MetOcean data types

Martin Schultz, Forschungszentrum Jülich, Germany () - 11 July 2014

An essential element of a robust data model for interoperable web applications such as JOIN is the definition and recognition of data types. As explained in the Coverage primer, most datasets in the fields of atmospheric and ocean sciences (short: MetOcean sciences) are discrete grid point coverages, but in order to fully understand and process such data, we also need information on the data coordinates, i.e. we must know how to place the data onto the Earth surface, to which altitude (or pressure) they refer, and for which date and time they are valid.

This document attempts a systematic classification of MetOcean data types as a preparatory step for defining and coding the JOIN data model, but also as information resource to anyone dealing with MetOcean data interoperability. I will be happy to collect feedback and receive criticism or updates concerning additional data types. The basis of this work comes from the community data model of Unidata, and from a CF convention track ticket which I opened, because I found inconsistencies among various tyologies of data (often confused with coverages), and because I felt that none of the existing typologies provides a complete hierarchy of MetOcean data types which is needed for a flexible data model.

Discrete grid point coverages

As explained in the coverage primer, the majority of MetOcean datasets can be subsumed under the class of "gridded datasets", where the grid is defined on discrete points and can have between 1 and 5 dimensions, which we call x, y, z, t, and i (first and second horizontal dimension, vertical dimension, time dimension, and ensemble dimension). Through association with a coordinate reference system (or at least a coordinate system), these dimensions become coordinates. Usually, the coordinates will be 1-dimensional, in which case we refer to the multi-dimensional grid as rectified (see the discussion on rectified grids in the overage primer). The most prominent exceptions are the 4-dimensional pressure or altitude coordinates of numerical model grids (see vertical coordinates). Scalar (numerical) features (i.e. 0-dimensional datasets) are considered as 1-element 1-dimensional coverages, i.e. point type datasets.

The following classification is structured according to the number of spatial dimensions of the MetOcean dataset. The t and i dimensions are treated independently and lead to the definition of subclasses of the spatial dataset types. For example, a point coverage type can be extended to a point_timeseries, a point_collection, or a point_timeseries_collection, while a rectified_grid_2d_xy may become a rectified_grid_2d_xy_timeseries, a rectified_grid_2d_ensemble, or a rectified_grid_2d_xy_timeseries_ensemble.

The notation x(i), y(i), [z(i)], v(i,t) lists the existing coordinates and their dimensions and the variable v with its dimensions. Normally, we implictly assume that x and y are longitude and latitude coordinates, and that z is either a pressure or altitude coordinate. However, this is not a fundamental requirement and at least in theory, the datatype classes given below should also work with other coordinate types (the conversion to longitudes, latitudes, etc. is actually a role of the coordinate reference system).

Note that all coverage types define only the locations of one (central) point of each "grid cell". The description of boundaries may add extra complexity for example if the 2d nonrectified grid describes a curved plane that cuts across an arbitrary part of the 3d sphere.

Zero (spatial) dimensions: Points and trajectories

  • point: a single location with optional vertical coordinate. All coordinates and the value are scalars unless they are extended to timeseries or collections. One-element one-dimensional or even multi-dimensional coordinates and variables that can be collapsed into scalars (example: extraction of a single grid point from a model grid) should not be considered point data, unless they are actually converted to scalar values.
    • point: x, y, [z], v
    • point_timeseries: x, y, [z], v(t)
    • point_collection: x(i), y(i), [z(i)], v(i)
    • point_timeseries_ensemble: x(i), y(i), [z(i)], v(i,t). In this case, all points share the same time axis. It is also conceivable to store multiple point_timeseries with individual time axes in one collection. However, as this will often result in sparsely populated matrices (timeseries of different lengths), we ignore this case for now. A suitable name would be point_collection_timeseries.

Notes:
1. Data from irregular grids (e.g. icosahedric grids) may also be described as point_collection (or point_collection_timeseries).
2. Conceivably, x and y may not always be defined or may have different dimensionalities (for example points along one latitude line). It should be good practice to always specify x and y, and always define them with the same dimensions. If this is not the case we should flag this as error and deal with it when it actually occurs.

  • trajectory: a location in space that changes with time. A trajectory will therefore always be a trajectory_timeseries or else it collapses into a point.
    • trajectory_timeseries: x(t), y(t), [z(t)], v(t)
    • trajectory_timeseries_ensemble: x(i,t), y(i,t), [z(i,t)], v(i,t). The trajectories of a collection share the same time axis.


Figure 1: Point and trajectory data types

One-dimensional datasets

  • rectified_grid_1d: a list of values that are arranged along a 1-dimensional coordinate axis (i.e. x, y, or z). The coordinate values need not be regularly spaced, although this will often be the case (and in practice we may indeed write subclasses for regular grids). Generally, coordinate values will be sorted in ascending order (latitude or vertical grids may sometimes be sorted in descending order instead).
    • rectified_grid_1d_x: x(x), [y], [z], v(x)
      • rectified_grid_1d_x_timeseries: x(x), [y], [z], v(t,x)
      • rectified_grid_1d_x_ensemble: x(x), [y], [z], v(i,x)
      • rectified_grid_1d_x_timeseries_ensemble: x(x), [y], [z], v(i,t,x)
    • rectified_grid_1d_y: [x], y(y), [z], v(y).
      • rectified_grid_1d_y_timeseries: [x], y(y), [z], v(t,y).
      • rectified_grid_1d_y_ensemble: [x], y(y), [z], v(i,y).
      • rectified_grid_1d_y_timeseries_ensemble: [x], y(y), [z], v(i,t,y).
    • rectified_grid_1d_z: [x], [y], z(z), v(z).
      • rectified_grid_1d_z_timeseries: [x], [y], z(z), v(t,z).
      • rectified_grid_1d_z_ensemble: [x], [y], z(z), v(i,z).
      • rectified_grid_1d_z_timeseries_ensemble: [x], [y], z(z), v(i,t,z).

Notes:
1. As discussed in the coverage primer, datasets with the frequently used sigma- or hybrid sigma-pressure vertical coordinates are not rectified coverages. Yet, as long as we have a single column, or all columns in the ensemble share the same z coordinate, such data can be treated as rectified coverages. In the case of simple altitude or pressure coordinates this is also true.

  • nonrectified_grid_1d: a list of values that are arranged along an arbitrary "line" in 3-dimensional space. Any x, y, z coordinate may be a 1-dimensional array, a scalar, or missing. All 1-dimensional coordinate axes must have the same dimension (i.e. length). Examples are: x, y(n), z(n), v(n) (here, x is a scalar value), x(n), y(n), v(n) (here *z* is a missing coordinate, i.e. one usually assumes that this line is at the Earth surface), or @x(n), y(n), z(n), v(n) (here, all spatial coordinates are defined).
    • nonrectified_grid_1d_timeseries: x(n), y(n), z(n), v(t,n)
    • nonrectified_grid_1d_ensemble: x(n), y(n), z(n), v(i,n)
    • nonrectified_grid_1d_timeseries_ensemble: x(n), y(n), z(n), v(i,t,n)

Notes:
1. Conceivably, the x, y, or z coordinates of a nonrectified_grid_1d_ensemble could be different for each ensemble member. If this is a relevant datatype (I am not aware of such data myself), then one should probably use the term "collection" rather than "ensemble" for this and define the appropriate classes accordingly.
2. Even though datasets from models with irregular grids (e.g. icosahedric, or various ocean model grids) may formally be similar to nonrectified_grid_1d data types, they are actually point_collection data, because the grid type requires all coordinates to be sorted in ascending or descending order.


Figure 2: 1-dimensional discrete grid point data types

Two-dimensional datasets

  • rectified_grid_2d: an array of values that are arranged along two 1-dimensional coordinate axes (i.e. longitude/latitude, longitude/vertical, or latitude/vertical). The coordinate values need not be regularly spaced. Generally, coordinate values will be sorted in ascending order (latitude or vertical grids may sometimes be sorted in descending order instead).
    • rectified_grid_2d_xy: x(x), y(y), [z], v(y,x); timeseries and ensemble will change the values to
      • rectified_grid_2d_xy_timeseries: x(x), y(y), [z], v(t,y,x)
      • rectified_grid_2d_xy_ensemble: x(x), y(y), [z], v(i,y,x)
      • rectified_grid_2d_xy_timeseries_ensemble: x(x), y(y), [z], v(i,t,y,x)
    • rectified_grid_2d_xz: x(x), [y], z(z), v(z,x)
      • rectified_grid_2d_xz_timeseries: x(x), [y], z(z), v(t,z,x)
      • rectified_grid_2d_xz_ensemble: x(x), [y], z(z), v(i,z,x)
      • rectified_grid_2d_xz_timeseries_ensemble: x(x), [y], z(z), v(i,tz,x)
    • rectified_grid_2d_yz: [x], y(y), z(z), v(z,y)
      • rectified_grid_2d_yz_timeseries: [x], y(y), z(z), v(t,z,y)
      • rectified_grid_2d_yz_ensemble: [x], y(y), z(z), v(i,z,y)
      • rectified_grid_2d_yz_timeseries_ensemble: [x], y(y), z(z), v(i,t,z,y)
  • nonrectified_grid_2d: an array of values that are arranged along an arbitrary "plane" in 3-dimensional space. At least one of x, y, z coordinate will be a 2-dimensional array, the other coordinates may be 2-d, 1-d, scalar, or missing. All 2-dimensional coordinate axes must have the same shape. Example: x(k,n), y(k,n), z(n), v(k,n).
    • nonrectified_grid_2d_timeseries: x(k,n), y(k,n), z(k,n), v(t,k,n)
    • nonrectified_grid_2d_ensemble: x(k,n), y(k,n), z(k,n), v(i,k,n)
    • nonrectified_grid_2d_timeseries_ensemble: x(k,n), y(k,n), z(k,n), v(i,t,k,n)
  • profile_trajectory_timeseries: x(t), y(t), z(t,z), v(t,z). Vertical profiling from a moving object (e.g. aircraft) yields a special type of 2-dimensional grid, similar to a trajectory (see figure 4 below). If only one time point is extracted from such a "grid", it can be collapsed into a (rectified) 1-dimensional grid coverage (rectified_grid_1d_z).


Figure 3: 2-dimensional discrete grid point data types


Figure 4: Example of a profile_trajectory_timeseries datatype (image from http://www.esrl.noaa.gov/csd/groups/csd3/instruments/topaz/)

Three-dimensional datasets

  • rectified_grid_3d: x(x), y(y), z(z), v(z,y,x). An array of values that are arranged along three 1-dimensional coordinate axes (for example longitude/latitude/altitude or pressure).
    • rectified_grid_3d_timeseries: x(x), y(y), z(z), v(t,z,y,x)
    • rectified_grid_3d_ensemble: x(x), y(y), z(z), v(i,z,y,x)
    • rectified_grid_3d_timeseries_ensemble: x(x), y(y), z(z), v(i,t,z,y,x)
    • sigma_model_grid_3d: x(x), y(y), z(z), v(z,y,x). Similar to the generic rectified_grid_3d, but with special handling of vertical coordinates (see note 1 below).
    • sigma_model_grid_3d_timeseries: x(x), y(y), z(z), v(t,z,y,x)
    • sigma_model_grid_3d_ensemble: x(x), y(y), z(z), v(i,z,y,x)
    • sigma_model_grid_3d_timeseries_ensemble: x(x), y(y), z(z), v(i,t,z,y,x)

Notes:
1. As mentioned several times, model datasets with sigma- or hybrid sigma-pressure vertical coordinates do not constitute rectified coverages. Since this is a very common case, however, and the vertical coordinates of such data are described with 1-dimensional coefficients, we introduce a special class hierarchy under the general category of rectified grids (the handling of such data is more similar to recitifed grids than to nonrectified grids).

  • nonrectified_grid_3d: : x(k,m,n), y(k,m,n), z(k,m,n), v(k,m,n). An array of values whose positions are determined by 3-dimensional coordinate arrays.
    • nonrectified_grid_3d_timeseries: x(k,m,n), y(k,m,n), z(k,m,n), v(t,k,m,n)
    • nonrectified_grid_3d_ensemble: x(k,m,n), y(k,m,n), z(k,m,n), v(i,k,m,n)
    • nonrectified_grid_3d_timeseries_ensemble: x(k,m,n), y(k,m,n), z(k,m,n), v(i,t,k,m,n)

Similarly to the 2-dimensional profile_trajectory_timeseries one can also define a cross_section_trajectory_timeseries which depicts for example an "aircraft flight corridor". This will require some further thinking, because the orientation of the cross sections will usually be constant relative to the aircraft, which means that it will change over time when projected onto a latitude-longitude coordinate system. A possible representation could be x(t,k,m,n), y(t,k,m,n), z(t,k,m,n), v(t,k,m,n)


Figure 5: Examples for 3-dimensional MetOcean data types


(under preparation...)
Rules for detection of coordinates
Rules for detection of coverage type

datatypes_point.png (10.1 KB) Schultz, Martin, 07/11/2014 02:19 pm

datatypes_1d.png (17.6 KB) Schultz, Martin, 07/11/2014 02:19 pm

datatypes_3d.png (10.1 KB) Schultz, Martin, 07/11/2014 03:39 pm

datatypes_2d.png (68.8 KB) Schultz, Martin, 07/15/2014 01:16 pm

datatypes_profile_trajectory.png (46 KB) Schultz, Martin, 07/15/2014 01:16 pm