Multidimensional WCS

Introduction

This page serves as a summary of the interest in augmenting Geoserver's WCS capabilities to serve 3D or 4D geospatial data. This common desire originates from diverse scientific communities which have a need for coverage servers which operate on coverage types pertinent to the community's discipline. A common data requirement is gridded meterological input, which can be applied in oceanography, climate change, and other scientific disciplines.

Adit Santokhee represents a group currently involved in an effort to expose the methods of the Grid Access Data Service (GADS)Java library as a web service conforming as closely as possible to the OGC's WCS specification. This effort has the scope defined in this attached document. In summary, they would like to support:

  • Simple subsetting of 2/3/4D gridded data (no resampling)
  • Asynchronous data delivery (due to large files)
  • Secure data access.

The remainder of this page discusses how this might come about.

Related Efforts

Wanting to publish large datasets on intelligent servers which are capable of eliminating wasteful network transfers (e.g. by allowing users to query for a particular variable or region) is about as common as the large datasets themselves. Wanting to then plot a specific variable for a specific region on a map with other annotations is about as common as wanting public funding. Nothing illustrates the impact of phenomenon X better than a map. This really helps to explain why study of phenomenon Xis important to your audience (in many cases, the web-surfing public). As such, adding the ability to manipulate large domain-specific datasets to a fully functional map-making machine is a slam dunk. It's one stop shopping for raw data and pretty pictures.

Making do with what we have

A couple of sites have already sprung up which illustrate the kind of functionality we desire in the long term. These sites all approach the problem of presenting multidimensional data differently.

OGC Web Coverage Server (WCS) 1.0

The OGC Web Coverage Serveris the reference from which all departures must be measured. This is the current standard for publication of gridded data on the web. It represents a first cut at the problem of publishing gridded data.

We will try to adhere to the WCS standard as much as possibleì, however, there is ample evidence that people who wish to publish multidimensional gridded data attempt to add to it, as documented in the GALEON interoperability experimentwebsite. While we have every desire to use the standard as it is, we have no need to repeat mistakes, paint ourselves into a corner, or go out of our way to build something which will change with the next version of the spec. Our strategy is to look at the facilities offered by WCS, determine how it falls short of meeting our needs, and compare with similar facilities in other, more mature, specifications. If possible, we will try to adopt conventions which are legal (if perhaps not "recommended") according to the current specification. Failing this, we will attempt to make minor alterations only where necessary, documenting them with respect to the pristine standard.

Definition of a Coverage, WCS, and WFS

An ISO coverage, as defined in ISO 19123, is really any set of features which possess a common description of their attributes. Because the same attributes are located at different spatio-temporal locations, one may view the collection of features as a single conceptual entity. This entity can provide location-dependent values for the attributes it advertises. The component features do not need to be regularly gridded nor do they need to represent continuous values across the domain. The single requirement is that the same quantities are measured at different locations.

This definition of a coverage tends to wash out the distinction between a Web Feature Server and a Web Coverage Server. Both servers serve coverages. However, Web Feature Servers publish discrete coverages and Web Coverage Servers publish gridded coverages (where the horizontal grid must be regular, but other axes (vertical, time, etc) need not be). In both cases, the item published has a spatio-temporal distribution to all of the items published. This definition does help us reinforce the fact that our scope is limited to publication of gridded multidimensional coverages. Publication of other types of coverages is delegated to the WFS.

Strengths, Weaknesses and Constraints

This section attempts to evaluate WCS 1.0 with respect to our goals. Such an evaluation should give a fair estimation of where we stand and how much work is required.

Strengths
  1. WCS 1.0 permits publication of multidimensional gridded data.
  2. WCS 1.0 permits selection along time and vertical axes.
  3. WCS 1.0 permits time axis to be irregularly gridded (what about other axes?)
  4. WCS 1.0 permits selection of file format for returned data.
  5. WCS 1.0 permits coverage-relevant axis descriptions. Any grid dimension associated with an axis description may be treated as an independant variable and used to subset the grid.
Weaknesses
  1. The mechanism provided to address 3D spatial data links to a single CRS id. This must of course be defined as a 3D CRS by the authority.
Constraints
  1. A coverage may return more than a single value, but all returned values must be of the same physical property. (e.g. a coverage may return a vector of radiances by wavelength, or a vector of reflectances by wavelength, but may not return a matrix of radiances and reflectances by wavelength.)
  2. CRSes may only be used if they have previously been defined by an authority factory. To define your own projection, you must use the "user-defined" section in the EPSG namespace. While this may have the effect of allowing the server to understand the associated projection, it does not communicate the CRS definition to a client.

OGC Web Map Server 1.3.0 / ISO 19128

Thus far, the only standard in OGC's web server suite to run the gauntlet of ISO standardization is the Web Mapping Services Implementation spec. This free OGC standardmight very well be identical to the ISO version, but this is unknown at the present time. What is known is that the OGC version normatively defines the means of specifying multiple dimensions in the context of a WMS.

This is a WMS spec and not a WCS spec. Having said this, the discussion of multidimensional datasets is pragmatic and clear. In terms of how to select data, the

In summary it appears that the standards organizations are finally beginning to address higher dimensionality. In light of this eventuality, we should adopt their conventions unless there is a compelling reason not to. If we find that changes are necessary, the WMS 1.3.0 methodology should be adopted as a baseline and our final solution should be expressed as in terms of departures from this standard.

How to Define a CRS

As of WMS 1.3.0, a server may declare a custom CRS which has not been defined by an authority factory. There are now two types of CRS references. The Label type requires the familiar syntax "authority:ID". Now there is a URL CRS reference. Using the URLreference, the WMS provides a fully qualified URL to a publicly accessible file which defines the CRS in a way that is compatible with 19111. The actual file format to use is not specified. However, if one courageously adopts the convention that GML is the preferred means of describing a CRS from scratch, a means of automatically communicating a complete CRS definition from server to client is now provided.

How to Advertise Dimensions

Simone writes here

How to Advertise Bounds on Elevation and Time Dimensions

Syntax   Meaning
 value  A single value.
 value1,value2,value3,...*  A list of multiple values
 min/max/resolution  An interval defined by its lower and upper bounds and its resolutions
 min1/max1/res1,min2/max2/res2,...*  A list of multiple intervals

 * Whitespace is allowed following commas in a list in a <Dimension> element of service metadata

A resolution value of zero (as in min/max/0) means that the data are effectively at infinitely-fine resolution for the purposes of making requests on the server. For instance, an instrument which continuously monitors randomly occurring data may have no explicitly defined temporal resolution.

All dimensions in a service metadata response are server-defined, with two exceptions. The dimensions named time and elevation are special cases predefined as follows:

<Dimension name="time" units="ISO8601" />
<Dimension name="elevation" units="verticalCRSid" unitSymbol="symbol" />

The time units identifier "ISO 8601" refers to the ISO 8601:2000 time representation specified in Annex D of the WMS 1.3.0 spec. No unitSymbol is defined for the time dimension. If a server opts to specify time values in a different representation it shall use a dimension name other than "time" (e.g. "date" or  "seconds").The elevation units identifier "verticalCRSid" refers to a vertical CRS as in 6.7.5 of WMS 1.3.0 spec. The unitSymbol attribute shall be chosen to match the units of the specified vertical CRS.

EXAMPLE 1 The verticalCRSid "CRS:88" refers to the vertical CRS defined in B.6 (elevation in meters in the North American Vertical Datum 1988). The unitSymbol "m" would be used.

EXAMPLE 2 The following are some examples of server-defined dimensions:

<Dimension name="time" units="ISO8601" default="2003-10-17">1996-01-01/2003-10-17/P1D</Dimension>

<Dimension name="temperature" units="Kelvin" unitSymbol="K" default="300">230,300,400</Dimension>

<Dimension name="elevation" units="CRS:88" unitSymbol="m" default="0">0/10000/100</Dimension>

Dimension declarations are inherited by enclosed Layers, as specified in 7.2.4.8 of WMS 1.3.0 spec.

A layer may provide a dimension declaration that has the same name attribute (case-insensitive matching) as a declaration in another layer. However, the dimension shall not be re-declared using the same name with conflicting units or unitSymbol attributes. Extent values, and the attributes default, nearestValue and multipleValues, may be different for each layer.

How to Request a Horizontal Slice

Some geographic information may be available at multiple elevations (for example, ozone concentrations at different heights in the atmosphere). A WMS may announce available elevations in its service metadata, and the GetMap operation includes an optional parameter for requesting a particular elevation.

  • Elevation value(s) can be associated with a dataset by using the ELEVATION dimension. The associated ELEVATION dimension has a number of mandatory and optional fields (see Table C.1 of Annex C in the WMS 1.3.0 spec).
  • When requesting a hor. data slice, and if a data object has an Elevation dimension defined, requests may include the parameter
ELEVATION=value

where, depending on the context, "value" can be a single value, a comma-separated list, or an interval of the form start/end without a resolution.
The absence of ELEVATION parameter will result to a request with the default value (if defined) for the ELEVATION dimension.

How to Request a Particular Time

To request a particular time from a dataset, the dataset must have the special "time" dimension associated with it. This association is described in the "How to Advertise Dimensions" section above. In essence, declaring that a layer has the special dimension "time" tells the server that "TIME=" parameters in a getMap request apply to this layer. Conversely, all layers which do not have an association with the "time" dimension are considered constant with respect to time and consequently ignore temporal parameters in queries.

Section C.3.6 in the WMS 1.3.0 specification contains two examples of GetMap requests which include time as a parameter. These are repeated here for convenience. The first example requests a single time slice as a GIF, and the second example specifies a range of times to be returned in a movie format (mpeg). These examples assume that "ozone" is a collection of 2D ozone maps at a particular elevation (this is a slight modification from the example in the standard.)

GetMap parameter Value
VERSION 1.3.0
REQUEST GetMap
LAYERS ozone
CRS CRS:84
BBOX -180,-90,180,90
WIDTH 600
HEIGHT 300
TIME 2000-08-03
FORMAT image/gif
GetMap parameter Value
VERSION 1.3.0
REQUEST GetMap
LAYERS ozone
CRS CRS:84
BBOX -180,-90,180,90
WIDTH 600
HEIGHT 300
TIME 2000-07-01/2000-07-31/P1D
FORMAT image/mpeg

Note that the BBOX parameter does not attempt to address the temporal request. It is purely spatial, and purely 2D (even if we were talking about a 3D data set.)

What good is a Sample Dimension

Sample dimension parameters allow the client to request a particular layer along one or more dimensional axes other than time or elevation.

A request parameter name is constructed by concatenating the prefix "dim_" with the sample dimension Name (the value of the name attribute of the corresponding <Dimension> element in the service metadata). The resulting "dim_name" is case-insensitive. The use of the "dim_" prefix is to avoid clashes between server-defined dimension names and other request parameters defined by this International Standard. (The "time" and "elevation" dimensions, being predefined, do not use the prefix.)

To include a sample dimension value in a request URL, the dim_name parameter is followed by an equals sign "=" and a valid value, a comma-separated list, or an interval, as described in Table C.2 of WMS 1.3 spec.

EXAMPLE A WMS Layer is described as having an extent along a dimension named "wavelength" as follows:

<Dimension name="wavelength" units="Angstrom" unitSymbol="Ao">3000,4000,5000,6000<Dimension>.

A GetMap request for a portrayal of the data at 4000 Angstroms would include the parameter "DIM_WAVELENGTH=4000".

The WMS 1.3.0 specification supports the notion of a sample dimension, but how exactly is it applicable to our problem? Can we define a sample dimension which contains arbitrary, unrelated quantities like salinity, temperature, etc? Or must we define an axis for each quantity (e.g. a temperature axis, a salinity axis, etc.)? (This second option does not appear to be too useful, and probably is not possible.)

An alternative data representation would be to expose each variable as a separate coverage with it's own units.

We must investigate how to use the sample dimension capability.

GeoTools Multidimensional Coverage

As GeoTools does most of GeoServer's behind-the-scenes work, the developmental [ISO coverage implementation]will be relevant to this GeoServer enhancement. This, however, is envisioned as a long term infrastructure investment and not a short term solution.

GALEON OGC Interoperability Experiment

Phase 1 of the GALEON IE is complete. Copied from The OGC's GALEON page, this experiment was to determine whether:

  1. a viable WCS getCapabilities geo-interface (gateway in earlier versions) can be built on existing THREDDS inventory catalog services
  2. the ncML-G data model is adequate for providing describeCoverage responses for netCDF datasets
  3. there are any solutions to the previously identified limitations to geoTIFF encoding format for representing from 5D netCDF files in such a way that the relationships among layers is preserved
  4. the proposed ncML-GML encoding format is a practical solution to serving 5D data from netCDF files, either embedded (ASCII or attached binary) or linked (OPeNDAP link or other URL)
  5. netCDF itself is a viable WCS binary encoding format
  6. existing WCS clients are able to access analyze and display 5D data from netCDF files
  7. 5D geospatial data sets can be served efficiently through standard database technology

In the above, THREDDS and OpenDAP are web publishing mechanisms currently in use for large datasets, typically in Grib, NetCDF, or HDF format. NetCDF and ncML-GMLare transfer mechanisms under investigation. Lastly the experiment investigates a number of issues surrounding the expansion of the WCS spec itself.

This interoperability experiment pertains to the web interface presented to clients. It does not traffic in internal details. It is therefore highly relevant when we start to consider what wemust add to the WCS spec. In particular, now that Phase 2 is commencing, it may very well be pertinent to note that one of the objectives is to increase the number of datasets served.

  • OGC's GALEON Page
  • The GALEON Wiki
  • Reports from people involved in the IE (UK and others) are attached to this page.
    Question
    Is the way in which multiple dimensions are exposed during the GALEON IE compatible with (or identical to) the way WMS 1.3.0 handles it?

Geoserver Security

Chris Holmes wants to begin work on GeoServer security soon anyway. Go to GeoCollab Security Discuss for more details.

Requirements

All projects need requirements. That's how you know when you're done. This project will:

  1. Add secure data access to GeoServer.
  2. Add asynchronous data delivery in NetCDF format to GeoServer.
  3. Change the WCS interface implemented by GeoServer such that it can handle >2D.
  4. Make GeoServer capable of serving a "vanilla" 2D geographic image if:
    1. The data type is compatible.
    2. The client has requested a horizontal 2D slice.

This effort

The objective of this effort is to implement an efficient and secure publishing and mapping mechanism for nontraditional GIS layers, such as meterological grids. This effort, as it is being led by Adit, will meet his needs. However, we may be able to leverage the previous work by the GALEON IE and the normative methods of OGC WMS 1.3.0 in order to avoid dead-ends. This effort is not part of the GALEON IE, nor is it an OGC effort.

Conventions and Restrictions

A previous section identified constraints, weaknesses and strengths of the existing WCS 1.0 specification. This section outlines what we intend to do in order to accomodate the weaknesses and constraints.

Change: Allow SRS Specification via URL

As noted in the section on WCS 1.0, one of the constraints imposed by the specification is that you may only refer to CRSes which have already been defined by some authority. These CRSes exist under the EPSG, CRS, or AUTO authority names. In short, client-server communication only includes CRS identifiers, never the entire CRS definition with all the parameters. This is problematic when data from models may exist in a perfectly well-defined CRS which has not been blessed by an authority.

As such, we shall support the URL method of SRS specification. In addition, we should explore adding support to Geoserver such that nonstandard CRS definitions registered with the server are automatically converted to GML and hosted under the WMS/WFS/WCS url namespace. In this way, custom projections are automatically communicated from server to client.

Convention: Only Use a Horizontal CRS

The CRS applies to the BBOX and Envelope of a coverage or of a result. For the 2D case, authorities have defined a plethora of ready to use CRSes. There are a few 3D CRSes, but going 3D severely limits your options. We notice that in the more mature WMS specification, the CRS and BBOX are limited to 2D and the vertical axis is handled separately by a <Dimension> named "elevation". This dimension specifies a vertical CRS as the units.

We adopt this practice as a convention with our use of WCS. BBOX shall specify the horizontal extent and vertical extent shall be specified separately by the parameter "ELEVATION", which is associated to individual coverages bearing the axisDescription with the name "elevation". The axisDescription defines the units and the datum of the vertical axis.


Question

According to some groups with experience in WCS implementation, it would be a bad decision to use the WMS "2-d horizontal plus vertical" approach (for specifying the CRS, and also for subsetting:

- i.e. BBOX=x1,y1,x2,y2&ELEVATION=z1,z2).

Much better would be to allow a full 3-d BBOX

If you imagine a 4-d data volume (x,y,z,t) it could be required to subset this in any way, e.g. a single point, or a line, or a plane, or a cube, all at either a single point in time or over a period of time.

e.g.:

- single point in time and space: BBOX=x1,y1,z1,x2,y2,z2&TIME=t1/t1

- single line vertically at a point in time: BBOX=x1,y1,Z1,x1,y1,Z2&TIME=t1/t1

- 'Hovmuller' (2-d slice) of a single line over time: BBOX=x1,y1,Z1,x1,y1,Z2&TIME=T1/T2

etc.....

Examples of arbitrary slices of a 4-d data volume can be found on a demo WCS at http://glue.badc.rl.ac.uk/cgi-bin/TPAC/WCS (the GetCoverage examples down the bottom of teh page)  

 

Convention: One Coverage, One Variable

To keep things simple, we limit ourselves to one variable per coverage. This does not necessarily mean that each variable is stored in a separate file on the filesystem. It does mean that if you want salinity, you request data from the salinity coverage. If you want temperature, you request data from the temperature coverage. We do not support (and neither does WCS 1.0) using a single request to get both salinity and temperature data from a coverage which contains both. Requesting data from a coverage is similar to loading data from a named variable in netCDF.

This convention, while vastly more restrictive than the conceptual definition of a coverage in ISO 19123, is only marginally more strict than the WCS specification.

Restriction: Temporal Specification

In accordance with the WCS 1.0 specification, the temporal extent of a coverage shall be advertised with the TemporalDomain element of the domainSet. Requesting a subset along the temporal dimension shall be performed using the special TIME parameter of the GetCoverage request. This does not deviate from the standard.

Rough approach

This effort has at least two independent components:

  1. Augmentation of the WCS specification to handle >2D data.

          It's worth noting that with a 4-d dataset, there are:

    •  a single way to select a 0-d data point
    •  4 ways to select a 1-d data line (along the x, y, z, or t dimensions)
    •  6 ways to select a 2-d slice (x-y, x-z, x-t, y-z, y-t, z-t)
    •  4 ways to select a 3-d volume (x,y,z, x-y-t, x-z-t, y-z-t)
    •  a single way to select a 4-d volume 

     2.   All behind-the-scenes work needed to service requests.

In the near term, the interface of the augmented WCS needs to be nailed down and formalized. This is gives implementors (not necessarily Geoserver implementors) a means of writing interoperable code. In my ([~bnordgren@fs.fed.us]) opinion, this interface should be as close as possible to the WMS 1.3.0 methodology because I believe this stands the best chance of future-proofing our effort. If possible, electing to not use fancy options (asynchronous data delivery or security) should reduce to the WMS 1.3.0 strategy.

Augmentation of the WCS

Previous work is documented on the GALEON website and in documents attached to this page. The GALEON documentation appears to be in the form of final reports from individual participants. We should probably review responses from all the participants and combine with the extra requirements of security and asynchronous data delivery to synthesize a good "next step". We should probably strive for a minimum departure from the WCS spec so that it still functions with plain old 2D data...

Watch this space.

Back end

The short term solution is to not do anything inside GeoServer itself. All requests will be relayed to an external GADS server and sent back to the client. (Is this correct?)

Project Events

  1. Simple demo of GADS coupled to Geoserver WCS by Aug 17th. This project has integrated GADS with Geoserver as of Aug 15th, the requests cause a slice to be extracted and a URL to download the data is returned.
    Adit's GADS/Geoserver installation

Project Documents

OGC WMS Specification
OGC WCS Specification

  Name Size Creator (Last Modifier) Creation Date Last Mod Date Comment  
File GADS_geoserver.odt 18 kB Adit Santokhee Aug 03, 2006 Aug 03, 2006  
PDF File GADS_geoserver.pdf 126 kB Adit Santokhee Aug 03, 2006 Aug 03, 2006  

Added by bnordgren@fs.fed.us, last edited by Alexander Petkov on Sep 03, 2008  (view change)
View Attachments (2) Info