Architecture of ocean monitoring and forecasting systems

An OOFS, with a global to regional scale, is based on numerical modelling of the ocean dynamics, biogeochemistry, and wave and data assimilation schemes for blending observations into the model and for providing the most accurate initial condition for the forecast (Tonani et al. 2015). An OOFS at coastal scale may usually use information from global/regional scales in terms of initial and boundary conditions to initialise and force its ocean model core in a very limited area in order to provide very accurate spatial-temporal solutions and may not necessarily use data assimilation methods.

In general, to produce a forecast we need to:
1. know what the ocean is doing now (initial condition);
2. calculate how the ocean will change in future (forecast);
3. use oceanographic expertise to validate and refine the output (products).

These three steps, represented in Figure 4.1, are based on a few basic components: observations, numerical model, and oceanographic expertise. Most of the systems rely on data assimilation techniques (see Section 4.4 for a general introduction and Section 5.5 for more details about numerical schemes) for blending observation and models; therefore, data assimilation can be considered as one of the essential components of the system. In the case of coastal forecasting systems, downscaling from global/regional scale is the preferred approach as described in Section 5.4.4.

Step 1 is the production of the most accurate initial condition about the variables the forecasting system is aiming to predict. This means that we need the best knowledge of the present status of each variable at every model grid point. This information is difficult to retrieve from observations because their spatial/temporal coverage is usually very sparse. Model simulations instead provide a uniform coverage in space and time and, thanks to data assimilation techniques observations, they can be blended into the model simulation, improving their accuracy. For data assimilation, it is common to use observations from multiple sources, maximising the data coverage and the type of variables measured by in situ and satellite instruments. The initial condition for the forecast is usually the result of a complex set of multiple simulations with data assimilation covering past hours or days. For global and regional oceanographic systems it is common to have a data assimilation cycle of the order of a few days. These simulations of the past provide not only the best knowledge for initialising the forecast of the present but also valuable information on the near present that can be included in the final product delivered to the users. 

The model usually needs some external forcing as input. The type of information needed at its boundaries (e.g. ocean/atmosphere, lateral boundaries, along the coast, etc.) can vary from model to model. An ocean dynamical model usually needs an atmospheric forcing from a real time weather prediction system to resolve the processes at the ocean/atmosphere interface. A regional/coastal model requires river runoff data at the interface with the coast and input values for its variables at the lateral boundaries. In case of coupled models (see Chapter 5 and Chapter 10, for example), external forcing fields might not be needed.

Step 2 is the projection in the future, the production of the forecast that is done by running the numerical model for hours, days or months in the future. The forecast lead time can vary from hours to days. Many systems have a forecast lead time of 3-15 days. The same forcing fields described in Step 1 are needed also for the forecast. The forcing fields could be from another forecast like the atmospheric forcing, that usually is from a weather prediction system, or they can be provided by climatological values or persisting the last available value.

Once the model has produced the forecast, it is validated and its output post processed to a standard format for the delivery to the users (Step 3 in Figure 4.1). The validation of the forecast cannot be done via direct inter-comparison with observations but is based on the validation of its initial condition and on studies covering an extended period in the past of the model skills.

As explained before, observations are a key component but have to be made available in real time and in a standard format. Observations in real time are usually ready to be used within a few hours from their acquisition but sometimes they can have delays of more than 24 hours. Timing of data availability will influence the design of the production cycle that has to compromise between using the maximum number of the observations and reducing the delay in the forecast release. The choice to be made has also to consider the need to release a new forecast as soon as possible even if this could imply a degradation of its accuracy.

The timeliness of the forcing fields is another limiting factor in the design of the production chain. We can take as an example a wave forecasting system in which the accuracy of the predicted fields is strongly correlated with the accuracy of the winds. We have to wait until the latest and more accurate wind forecast is made available before starting our production. Different solutions can be implemented depending on the characteristics of each system. The computational time needed for running each of the three steps described is a very important aspect as, depending on the cost for running a specific system, it could be a limiting factor.

Timeliness is of paramount importance for the users and the production time should be reasonably short to avoid delivering forecasts referring to the past. A rule of thumb is that the production time needs to be consistently less than the production frequency. It means that for a daily cycle (production of a forecast once a day) the production time should be of the order of a few hours.

Even if the information provided in this section is focused on a forecasting system, with few modifications it can be also applied to a multi-year production system to produce a reanalysis. The main difference is that in this case you are not projecting in the future but in the past. This implies that you can blend observations and model simulations at each time step. The model is continuously corrected by the observations, increasing the accuracy of the simulations. The atmospheric forcing usually is also more accurate because it is an analysis and not a forecast, and hence the observations have been subject to a more restrictive data quality control compared to the real time ones.

The multi-year production is composed only of Step 1 and Step 3. In this case, in Step 1 the model and data assimilation cover a few hours/days spans over multiple decades of years. As the multi-year products are not limited by the timeliness, usually their major constraints are the computational time that can be extremely expensive as well as the availability of homogenous sources of forcing. These differences with respect to other forecast products have to be taken into account in the design of the production cycle. 

In the next subsections the architecture details at the basis of an OOFS will be introduced.

The data access and pre-processing component should make available all the needed dataset that will be used to perform the analysis, and then the forecast (Step 2). Automatic acquisition of the data is mandatory for an operational system. It could be quite demanding depending on the dataset, the centres (or data providers) involved in data production and treatment, and the available network to connect the centres. For most of the dataset used in OOFS, at least a daily update is needed.

For atmospheric forcing the volume of the dataset can be big, and an efficient connection to Operational Meteorological Centres in charge of operational production of atmospheric analysis and forecast is critical. For example, the volume of hourly surface forcing fields from the ECMWF at global scale is 34 GB per day. Then, data pre-processing is necessary to interpolate the atmospheric fields to the ocean grid, if there is inhomogeneity between frequency of available forcings during the length of the specific run, atmospheric datasets must also be interpolated temporally. When a regional ocean model is employed instead of a global model, the retreatment of the atmospheric dataset may substantially reduce the volume of the atmospheric dataset and reduce the overall storage cost.

In-situ ocean observations can be downloaded in real time using WMO GTS or from dedicated interface such as the service developed in the Copernicus Marine Service (Le Traon et al., 2019), in which in situ observations are made available, documented, quality controlled, and homogenised, all very important tasks to be performed before assimilating such dataset in an OOFS. Satellite observations need to be pre-processed by a dedicated centre before their assimilation in an ocean operational system. Satellite observations are processed at various levels ranging from Level 0 to Level 4 which need to be made available depending on the data type. For example, Copernicus Marine Service also provides a unique access point to download all the available satellite observations in real time.

Ocean analysis is based on a model, observations, and data assimilation scheme to provide the initial state of the forecast on the basis of a minimum error principle, i.e. the data assimilation modelling system (Figure 4.1). This component is central processing unit (CPU) consuming and should be performed on a supercomputer. High performance computing power is one of the most important constraints to define the resolution of the analysis system, along with the number of observations that will be assimilated in the system and the frequency and length of the analysis cycle. In an operational framework, the analysis cycle should be performed in a range of a few minutes to a few hours (maximum), choosing the best compromise between performances, quality of the analysis, and robustness of the operational system. This component will provide the initial state for the ocean forecast. The resulting time series of analysed ocean state is defined as the best analysis time series.

To perform an ocean analysis, we need the initial state of the model, based on the prior state of the model at the end of the previous analysis cycle, in situ and satellite observations, and atmospheric forcing analysis fields, collected and formatted in the previous acquisition and pre-processing phase (including all the static files that are necessary for the data assimilation modelling system). Outputs of this component are 3D fields to update the best analysis time series and restart files to initialise the next ocean forecast. Other diagnostics, metrics or post-processing may be computed online directly during the analysis cycle to optimise the system, and used as additional products for dissemination and archiving. Such products are also used during the validation phase (e.g.the mixed layer depth, the collocation between model out put and observations, transports, etc.).

Note that in some coastal forecasting systems there is no direct data assimilation. If the model domain is small, in some occasions there is simply no available data to be assimilated. In these cases, the system relies totally on the boundary conditions and initial 3D fields derived from a larger scale model (see Section 5.4.4 for downscaling examples).

The ocean forecast at some range is based on the numerical model initialised by the ocean analysis and forced by the atmospheric forecast fields as provided by the operational atmospheric centre. In most cases, the same model is used for both the forecast component and the analysis component, even if differences in terms of resolution and physical parameterizations could be envisaged especially in the framework of an ensemble forecast. The same constraints mentioned above about high performance computing apply in order to perform forecasts that are usually updated at least every day. Forecast range will also depend on the computing resources and on the main processes that have to be forecasted with a reasonable skill (to be defined by the developer of the forecasting system). The forecasting cycle should be performed in a range of a few minutes to a few hours. Inputs of the forecasting cycle are the initial state produced by the data assimilation modelling system (e.g. ocean analysis), all the static files needed to integrate the model, and the atmospheric forcing for the full forecast length. The forecast output is updated every day and consists of 3D and 2D ocean fields; it may include diagnostics, metrics and other post-processed dataset that can be useful to assess the quality of the product, to highlight specific features of the forecasted ocean properties and for the final delivery to users.

The post processing phase is devoted to building all the products that will be delivered to the users. It consists of files or datasets that are provided according to a) standard file format (e.g. according to CF Conventions, 🔗1); b) on a specific grid; and c) with homogeneous variables and meta-data. Such products may be then used to compute new products as ocean monitoring indicator (OMI), ensemble mean and standard deviation in the framework of ensemble forecast. This post processing should be performed on a supercomputer in which all datasets provided by the analysis and forecast components are stored in order to save resources in the computing centre. Computing cost of this stage could be really high (for example, due to the interpolation procedure in the case that the products are delivered on a specific grid) and would also include large data transfer and input/output access. The inputs of the post-processing component are represented by all datasets produced during the analysis and forecast cycles, while the outputs are all the products that will be delivered for internal and external users.

The objective of the validation component is to provide information on the quality of the operational system. The quality of the analysis is compared to already known or expected results (based on literature or climatological datasets) or to available observations. Quality of the forecast is performed by computing forecast skill in comparison to the analysis with the observation in delayed mode. The final step is to provide all this information to forecasters and users. Input of this component are model products, diagnostics and metrics computed during previous steps and the output could be numerical fields, time series and/or interactive maps that allows, through web interfaces or other kinds of applications, direct querying, comparison of different periods, and validation of the production.

The goal of the dissemination phase is to make all the products available to users on a dedicated infrastructure. This phase may be complex and the associated cost is very dependent on objectives and user needs. If the dissemination of the model is only internal, outputs could be made available through an intranet, using in-place storage capacities. Other approaches are mandatory for a more complex system providing a very large dataset and long-time series and designed to be accessed by several thousands of users, including a catalogue of products continuously maintained and updated, dedicated services for viewing, extracting and downloading the data. Cloud storage facilities are now the best infrastructure to disseminate operational ocean products.

The monitoring component is an important part of an operational system as it allows operators and forecasters to monitor the performances along all the production phases, from data access to dissemination. KPIs should be monitored during this phase, including availability of inputs and outputs during each phase, timeliness, time of delivery and delay of each component, anomaly and/or errors identified during each phase. Monitoring phase should be used to provide information to the users and to decide on a go/no-go to disseminate the products externally. Monitoring phase should be presented on a dedicated dashboard.