Skip to content

Cloud Native GeoServer externalized configuration guide

Introduction

GeoServer Cloud provides extensive configuration options through Spring Boot's externalized configuration system. This document outlines the various configuration options available for customizing your GeoServer Cloud deployment.

JNDI Datasources

A custom and very simple JNDI implementation is used through the org.geoserver.cloud:spring-boot-simplejndi maven module, allowing to configure JNDI data sources through Spring-Boot's configuration properties.

jndi:
  datasources:
    # the data source names (i.e. ds1, ds2) will be bound as java:comp/env/jdbc/ds1 and java:comp/env/jdbc/ds2:
    ds1:
      enabled: true
      wait-for-it: true
      wait-timeout: 60
      url: jdbc:postgresql://host:5432/database
      username: sa
      password: sa
      connection-timeout: 250
      idle-timeout: 60000

On Kubernetes, you can for example mount it as a config map in all the services.

PgConfig backend: database-backed configuration files

The pgconfig catalog backend stores GeoServer's catalog and configuration in PostgreSQL, but treats a handful of directories, data/, tmp/, logs/, temp/, and legendsamples/, as pod-local by default: files under them are read and written directly on each pod's own filesystem instead of going through the database. This keeps large or transient files, such as uploaded rasters, temporary output, and logs, out of the catalog database. It also means a file written under one of these directories on one pod is invisible to every other pod, since there is no shared filesystem between them.

The geoserver.backend.pgconfig.db-backed-file-patterns property lists Ant-style path patterns, evaluated against the resource store path (for example data/workspace/store/file.properties), for files that are stored in the database even though they live under one of these otherwise pod-local directories. The default value targets ImageMosaic configuration files:

geoserver:
  backend:
    pgconfig:
      db-backed-file-patterns:
        - "data/**/*.properties"
        - "data/**/sample_image.dat"

ImageMosaic stores across pods

These defaults cover the configuration files gt-imagemosaic writes alongside a coverage store: indexer.properties, datastore.properties, the generated <coverage>.properties, and sample_image.dat. Because these files propagate through the database, an ImageMosaic coverage store created through the REST API, with its granule index backed by PostGIS instead of a shapefile index, works on every pod: each pod materializes the store's directory locally from the database before opening the reader, whether the store was just created, or a granule was just added or removed. Deletions propagate too: removing a coverage or deleting the store removes the database copies of its config files, and a store later created at the same path starts clean instead of inheriting the deleted store's configuration.

This is what makes it possible to create a mosaic store with PUT .../coveragestores/{store}/file.imagemosaic?configure=none, add granules with POST .../coveragestores/{store}/remote.imagemosaic, and render the resulting layer from a wms pod that never handled any of the REST calls, without a shared filesystem.

Limitation: inline granule uploads remain pod-local. POST .../coveragestores/{store}/file.imagemosaic with a raster zip, and PUT .../coveragestores/{store}/file.geotiff, write the granule's raster data directly to the pod's local filesystem, which db-backed-file-patterns does not propagate; only the small configuration files are covered by the defaults. Reference granules by URL instead: remote.imagemosaic for http(s):// and s3:// locations, such as Cloud Optimized GeoTIFFs on object storage, or external.imagemosaic for a path on a filesystem shared across every pod.

A complete worked example, including incremental REST-driven adds and an externally managed PostGIS index populated outside GeoServer, is available at examples/cog-imagemosaic-pgconfig/.

PgConfig backend: caches

The pgconfig backend keeps data in memory to spare the database repeated queries. Three caches work in each service instance, and the services running GWC (gwc, wms, webui and restconfig) add a fourth for built tile layers. They are on by default and have a size bound. The bound of the tile layers counts tile layers, not bytes: a kept tile layer takes about 15 KiB for a layer and about 100 KiB for a group of 30 layers, as measured on a lab catalog.

All properties below sit under geoserver.backend.pgconfig.cache.

Property Default Meaning
enabled true Keeps catalog and configuration objects in the entity cache. With false, GWC reads each tile layer and its settings from the database on each tile request, with two to four statements per tile request.
max-heap-percent 5 Maximum size of the entity cache, in percent of the maximum heap.
compression zstd How the entity cache stores objects: zstd or none.
miss-ttl 30s How long a lookup with no result is remembered.
tile-layers 1000 Number of GWC tile layers kept built by each instance of a service running GWC, for tile requests. 0 turns this off.
listing-write-through false Adds the objects read by catalog listings to the entity cache. A listing larger than the cache would push out the objects read most often.
request.enabled true Keeps the decoded objects of a request until the request ends.
request.max-entries 8000 Maximum number of objects kept for one request.
request.max-total-entries 100000 Maximum number of objects kept for the requests in flight together, in one service instance.
resources.enabled true Keeps the paths of the resource store, and the content of the files read, in memory.
resources.max-heap-percent 1 Maximum size of the file content kept, in percent of the maximum heap, and 64 MiB at most.

Durations accept the usual Spring Boot forms, such as 30s and 10m. For example:

geoserver:
  backend:
    pgconfig:
      cache:
        max-heap-percent: 5
        request:
          max-entries: 8000

Entity cache

The entity cache holds catalog and configuration objects as compressed bytes. The requests of a service instance share it. Local changes and changes made through other instances, received on the event bus, remove the affected entries. An entry does not expire on a timer. It stays until an event removes it, until a reset or a reload of GeoServer empties the cache, until the event bus reconnects, or until the memory bound pushes it out. A lookup that finds nothing is remembered for miss-ttl. A configuration that still sets entry-ttl starts normally: the setting is ignored. The cache never grows past max-heap-percent of the heap.

The GWC settings of a layer or layer group are kept in the entity cache next to the layer or group.

Layer groups are loaded in pages ahead of their first use. The load runs at start-up, on a reload, when the event bus reconnects, and once the cache serves again after a reset or a reload. It fills at most half of the cache, and the other half stays for the objects read by requests. Only the services running the layer group containment cache (geoserver.security.layergroup-containmentcache=true, set to true in config/geoserver.yml for the wms, gwc and webui services) load the groups. The other services read them on demand. A catalog listing made by GWC also puts at most half of the cache.

The services running GWC (gwc, wms, webui and restconfig) also keep the tile layers built by their lookups, up to tile-layers, so that a tile request does not decode and build its tile layer again. The requests share a kept tile layer. It is dropped when the catalog changes (any removal of entries from the entity cache), after a GWC configuration, gridset, tile layer or blob store event, a reset, a reload or a reconnection of the event bus.

Changes written to the pgconfig tables by another program than GeoServer Cloud are not supported. They show after a reset through the restconfig service or after a reconnection of the event bus.

Request cache

The request cache decodes each object once per request. A request keeps at most request.max-entries objects. The requests in flight together keep at most request.max-total-entries objects in a service instance. Past these limits, a request still works: it looks objects up without keeping them.

Resource store cache

The resource store holds files such as styles and templates. The cache loads the paths of the whole store in one query, and a path missing from the index is missing from the store. The index is loaded at first use and again after a clear: when the event bus reconnects, and when GeoServer resets. The content of the files is kept once read, within resources.max-heap-percent of the heap and 64 MiB at most.

The first read of a file in a styles folder (styles at the root, or workspaces/<workspace>/styles) writes the missing or outdated files of that folder to the pod's disk, in one query. This spares a GetCapabilities on a new pod one query per style file. If those files add up to more than 16 MB, they are read one by one. Subfolders of a styles folder load file by file.

Changes made through another instance are applied through the event bus. Changes made outside the store, by another program or by an instance without the events, are found at the next load of the index. That load updates the index, drops the kept content of the changed files, and tells the listeners of the service instance about the changed paths. For example, GeoServer then parses a changed style again.

Sizing

A catalog of 580 layers takes less than 1 MB in the entity cache. With zstd and its shipped dictionary, the stored bytes are 6 to 10 times smaller than the JSON. A decoded object takes about 2 KB in the request cache. The default of 8000 objects per request is twice the number read by a request on a catalog of 580 layers. The resource store index takes about 200 bytes per row. A catalog of 100,000 layers with one style per layer has about 100,000 rows. The file content is bounded by resources.max-heap-percent, 1 percent of the heap by default and 64 MiB at most.

Metrics

The Prometheus actuator endpoint exposes the caches.

  • The catalog cache, the configuration cache and the resource store cache are named gs-catalog, gs-config and gs-resources. Their series are the Caffeine cache metrics, such as cache_gets_total and cache_evictions_total, and cache_weight_bytes. They have the tags cache and name with the cache name, and cache_manager="pgconfig".
  • resourcestore_index_rows is the number of rows in the resource store index.
  • catalog_request_cache_lookups_total counts the lookups in the request caches, with result="hit" or result="miss". catalog_request_cache_rejected_total counts the objects not kept because a request cache was full or closed, or because the requests in flight held their request.max-total-entries. A request adds its counts when it ends.
  • catalog_request_cache_entries is the number of objects held by the requests in flight together, against request.max-total-entries.

Removed property

The geoserver.catalog.caching.enabled property has no effect. The pgconfig caches replace the optional catalog cache of the services. geoserver.backend.pgconfig.cache.enabled turns the entity cache on or off, and request.enabled and resources.enabled control the two other caches. A service started with the old property logs a warning that names the new one, whatever the old value. With the Compose overlay compose/toxiproxy.yml, WMS_CATALOG_CACHING=false sets GEOSERVER_BACKEND_PGCONFIG_CACHE_ENABLED=false for the wms service.

GeoServer configuration properties

HTTP proxy for cascaded OWS (WMS/WMTS/WFS) Stores

Cascaded OWS stores make use of a SPI (Service Provider Interface) extension point to configure the appropriate GeoTools HTTPClientFactory.

We provide a Spring Boot Auto-configuration that can be configured through regular spring-boot externalized properties and only affects GeoTools HTTP clients instead of the whole JVM.

The usual way to set an http proxy is through the http.proxyHost, http.proxyPort, http.proxyUser, http.proxyPassword Java System Properties.

In the context of Cloud Native GeoServer containerized applications, this presents a number of drawbacks:

  • Standard Java proxy parameters only work with System properties, not OS environment variables, and setting system properties is more cumbersome than env variables (you have to modify the container run command).
  • http.proxyUser/Password are not standard properties, though commonly used, it's kind of JDK implementation dependent.
  • Setting `-Dhttp.proxy* System properties affects all HTTP clients in the container, meaning requests to the config-service, consul-service, etc., will also try to go through the proxy, or you need to go through the extra burden of figuring out how to ignore them.
  • If the proxy is secured, and since the http client used may not respect the http.proxyUser/Password parameters, the apps won't start since they'll get HTTP 407 "Proxy Authentication Required".

The following externalized configuration properties apply, with these suggested default values:

# GeoTools HTTP Client proxy configuration, allows configuring cascaded WMS/WMTS/WFS stores
# that need to go through an HTTP proxy without affecting all the http clients at the JVM level
# These are default settings. The enabled property can be set to false to disable the custom
# HTTPClientFactory altogether.
# The following OS environment variables can be set for easier configuration:
# HTTP(S)_PROXYHOST, HTTP(S)_PROXYPORT, HTTP(S)_PROXYUSER, HTTP(S)_PROXYPASSWORD, HTTP(S)_NONPROXYHOSTS
geotools:
  httpclient:
    proxy:
      enabled: true
      http:
        host: ${http.proxyHost:}
        port: ${http.proxyPort:}
        user: ${http.proxyUser:}
        password: ${http.proxyPassword:}
        nonProxyHosts: ${http.nonProxyHosts:localhost.*}
        # comma separated list of Java regular expressions, e.g.: nonProxyHosts: localhost, example.*
      https:
        host: ${https.proxyHost:${geotools.httpclient.proxy.http.host}}
        port: ${https.proxyPort:${geotools.httpclient.proxy.http.port}}
        user: ${https.proxyUser:${geotools.httpclient.proxy.http.user}}
        password: ${https.proxyPassword:${geotools.httpclient.proxy.http.password}}
        nonProxyHosts: ${https.nonProxyHosts:${geotools.httpclient.proxy.http.nonProxyHosts}}

Configure HTTP proxy with environment variables in compose.yml

As mentioned above, regular JVM proxy configuration works with Java System properties but not with Operating System environment variables.

The above geotools.httpclient.proxy config properties though allow to do so easily as in the following compose.yml snippet:

version: "3.8"
...
services:
...
  wms:
    image: geoservercloud/geoserver-cloud-wms:<version>
    environment:
      HTTP_PROXYHOST: 192.168.86.26
      HTTP_PROXYPORT: 80
      HTTP_PROXYUSER: jack
      HTTP_PROXYPASSWORD: insecure
...

Configure admin user through environment variables

For Cloud-Native deployments, an AuthenticationProvider exists that allows to set an administrator account (username and password) through environment variables GEOSERVER_ADMIN_USERNAME/GEOSERVER_ADMIN_PASSWORD, or Java System Properties geoserver.admin.username and geoserver.admin.password.

Useful for devOps to set the admin password through a Kubernetes secret, instead of having to tweak the security configuration XML files with an init container or similar.

This authentication provider will be the first one tested for an HTTP Basic authorization, only if both the above mentioned username and password config properties are provided, and regardless of the authentication chain configured in GeoServer.

If only one of the geoserver.admin.username and geoserver.admin.password config properties is provided, the application will fail to start.

If enabled (i.e. both admin username and password provided), a failed attempt to log in will cancel the authentication chain, and no other authentication providers will be tested.

If the default admin username is used, it effectively overrides the admin password set in the xml configuration. If a separate administrator username is given, the regular admin user is disabled.

Change the default geoserver path

By default, GeoServer Cloud services are available under geoserver/cloud/web path. You can change it by using the GEOSERVER_BASE_PATH or Java System Properties geoserver.base-path.

Client address behind a reverse proxy

The services read the client address of a request for their logs, the control-flow rules, the brute force prevention, and the IP based security rules. The gateway tells them this address in the X-Forwarded-For header, and resolves it in the same way as Tomcat's remote IP valve. A trusted proxy in front of the gateway lists the client address in the X-Forwarded-For header of the request, and the gateway takes the right-most address of the header not matching the trusted proxies. For a request from any other client, the client address is the address of the connection.

geoserver.gateway.trusted-proxies is a regular expression matching the addresses of the trusted proxies, as seen by the gateway. By default, the gateway trusts no proxy. Behind a reverse proxy or a Kubernetes ingress controller, list its addresses, otherwise the services see the proxy address as the client address of the requests:

geoserver:
  gateway:
    trusted-proxies: 10\.42\.\d+\.\d+

On Kubernetes, the expression must match the pod addresses of the ingress controller. The ingress controller sees the real client address only when the Service in front of it has externalTrafficPolicy: Local.

The gateway logs a warning the first time a request with an X-Forwarded-For header comes from an address other than the trusted proxies.

The same setting selects the proxies allowed to set the request ID of the logs, see Write JSON logs.

Concurrent requests

Two settings of the configuration repository decide how many requests a service runs at once.

The control-flow extension admits the requests, with the rules of geoserver_control_flow.yml. Each service loads the first document of the file and the document of its own profile. The limits scale with cpu.cores, the number of processors seen by the JVM, equal to the CPU limit of the container:

Service Rules Default per core
wms ows.wms, ows.wms.getmap 8, 4
wfs ows.wfs, ows.wfs.getfeature 8, 2
wcs ows.wcs, ows.wcs.getcoverage 8, 1
wps ows.wps.execute 1
gwc ows.gwc, ows.wms.getmap 8, 4
restconfig, webui none, control-flow is disabled

A request above a limit waits in the queue of the rule for timeout seconds, 10 by default, then gets a 503. An environment variable sets one rule on one container, for instance GEOSERVER_EXTENSION_CONTROLFLOW_PROPERTIES_OWS_WMS_GETMAP=6, and CONTROL_FLOW=false disables the extension. The file documents the other rules: per user, per IP address, per output format, and request rates.

Tomcat gives each request a worker thread before control-flow sees it, and keeps the requests above its thread count waiting with no timeout. geoserver_spring.yml gives the services 500 threads, as many requests as one gateway replica sends to one address at once, the size of its connection pool. The count holds regardless of the services in the pod. TOMCAT_THREADS_MAX sets it on a container. A thread waiting in a control-flow queue costs memory only.

When the CPU request of a container is far below its limit, cpu.cores counts CPU not guaranteed to the container. Set the request equal to the limit, or pass the cores to size for: JAVA_OPTS=-Dcpu.cores=2.

Use GeoServer ACL

To use GeoServer ACL with GeoServer Cloud, you need to do the following steps: - Have a database available for storing the GeoServer ACL configuration - Add a GeoServer ACL instance to your deployment - Update the GeoServer spring configuration to enable ACL based security (you can use the acl spring profile)

Use OAuth authentication

You can enable OAuth authentication by replacing the default gateway image by the geOrchestra gateway (for example georchestra/gateway:23.1-RC1).

Filter vector and raster data formats

GeoServer Cloud provides the ability to filter which GeoTools DataAccessFactory (vector) and GridFormatFactorySpi (raster) implementations are available in the application. This allows you to customize each deployment to only include the formats you need, improving security, reducing the attack surface, and potentially improving startup time.

Configuration properties

The following configuration properties can be used to control which formats are available:

geotools:
  data:
    filtering:
      # Master switch for the entire filtering system
      enabled: true

      # Vector format configuration (DataAccessFactory implementations)
      vector-formats:
        "[PostGIS]": true
        "[Shapefile]": true
        "[GeoPackage]": true
        "[Oracle NG]": ${oracle.enabled:false}
        "[Web Feature Server (NG)]": true
        "[Microsoft SQL Server]": false
        # Add more vector format entries as needed

      # Raster format configuration (GridFormatFactorySpi implementations)
      raster-formats:
        "[GeoTIFF]": true
        "[ImageMosaic]": ${mosaic.enabled:true}
        "[ArcGrid]": false
        "[WorldImage]": true
        "[ImagePyramid]": false
        # Add more raster format entries as needed

Format names

The format names used in the configuration are the user-friendly display names returned by the respective factories:

  • For vector formats: The name returned by DataAccessFactory.getDisplayName()
  • For raster formats: The name returned by AbstractGridFormat.getName()

Since these names often contain special characters, they should be properly escaped in the YAML configuration using quotes and brackets.

Placeholder resolution

Both vector and raster format configurations support Spring property placeholder resolution, allowing you to create dynamic configurations using environment variables or system properties. For example:

vector-formats:
  "[Oracle NG]": ${oracle.enabled:false}
  "[PostGIS]": ${postgis.enabled:true}

This will enable the Oracle format only if the oracle.enabled property is set to true, otherwise it will default to false.

Implementation details

The filtering system uses different approaches for vector and raster formats:

  • Vector formats: Directly deregisters disabled DataAccessFactory implementations using DataAccessFinder.deregisterFactory() and DataStoreFinder.deregisterFactory()
  • Raster formats: Uses a custom FilteringFactoryCreator wrapper around the standard GridFormatFinder registry to filter formats on-the-fly

For more details, refer to the README files in the respective starter modules: - Input formats: src/starters/input-formats/README.md