Cloud Native GeoServer

Dockerized GeoServer micro-services

View the Project on GitHub

Introduction

Cloud Native GeoServer is GeoServer ready to use in the cloud through dockerized microservices.

The Cloud Native GeoServer project splits the GeoServer geospatial services and API offerings into individually deployable components of a microservices based architecture. This project provides clear constructive changes to GeoServer when required, based on prior experience with GeoServer clustering solutions.

Cloud Native GeoServer is built with GeoServer software components. Components are adapted and/or extended in a functional decomposition by business capability. The result is each service (OWS service, the Web UI, the REST API ) is available as a self-contained, individually deployable and scalable micro-services.

Q: Does that mean GeoServer’s .war is deployed several times, with each instance exposing a given “business capability”?

Absolutely not, this is not a clustered GeoServer approach and does not use war files. Each microservice is its own self-contained application, including only the GeoServer components needed for the service. Many GeoServer components provide a lot of functionality (such as different output formats). In these cases, care is taken to only load the functionality that is needed for a light-weight experience.

Contents

Background and motivation

GeoServer is the most widely used and deployed Open Source geospatial server in the world. It allows publishing, transforming, and editing geospatial data from a wide number of formats, through an extense number of user-facing services. Besides these out-of-the-box capabilities, it also counts with over 40 supported extensions, plus a bigger number of experimental extensions (so-called “community modules”).

Being a monolithic application, installation and configuration of GeoServer is very easy, as all its different components are bundled together, offering a simple, cohesive approach to software delivery.

People in charge of deploying and maintaining GeoServer instances usually face some very common challenges when it comes to server dimensioning, configuration, system health monitoring and corrective actions; difficulties that get incremented as systems need to ensure a certain capacity to handle request load, service availability, and overall performance.

Traditional deployments, though feasible of providing high availability, tend to require a fixed number of application instances, a single-master/multiple-slaves cluster architecture, and/or the installation and configuration of additional components to keep instances in sync.

Ensuring a given load capacity often results in over-provisioning of server resources to handle peaks on demand, which can have a negative impact on an organization’s budget, especially if deploying on a public cloud provider such as Amazon AWS, Microsoft Azure, or Google Cloud Platform.

Though a gateway/load balancer can be put in front and still make the cluster instances auto-scalable, all instances expose the whole set of functionalities GeoServer provides, despite which service(s) are under high demand, which can lead to unnecessary consumption of resources, opening attack vectors, and losing availability of unrelated capabilities when some component fails.

Vision

Cloud Native GeoServer is a sibling project to GeoServer, targeting organizations and system administrators in seek of a steady, prescribed way of provisioning GeoServer capabilities in a dynamic environment, where each business capability can be enabled, configured, dimensioned, and deployed independently.

Cloud Native GeoServer is not a re-write of GeoServer. On the contrary, it builds on top of existing GeoServer software components, adapting and/or extending them, feeding from, and contributing back to GeoServer’s mainline development. Traditional GeoServer installations are still recommended for simple deployments that can/need to expose all its capabilities.

This project reformulates how these capabilities are bundled together, in an attempt to achieve functional decomposition by business capability; which roughly means each OWS service, the Web UI, the REST API, and probably other components such as the Catalog and Configuration subsystem, become self-contained, individually deployable and scalable micro-services, loosely coupled through an event based architecture.

These containerized applications allow deployment strategies to vary from single-server docker composition, to multi-node deployments using docker swarm or Kubernetes.

Goals and benefits

Quick Start

The following instructions are meant as quick start up guide to run the system as a docker composition in a single machine. For more advanced deployment options and strategies, check out the Deployment Guide.

Make sure you have docker and docker-compose installed:

Download this docker-compose.yml file to a directory in your computer. For example:

$ mkdir gscloud && cd gscloud
$ wget "https://geoserver.github.io/geoserver-cloud/deploy/docker-compose/stable/docker-compose.yml"

Open a terminal and enter the directory where you just downloaded that file, and run docker-compose pull to fetch the docker images from Dockerhub:

$ docker-compose pull
Pulling rabbitmq  ... done
Pulling database  ... done
Pulling discovery ... done
Pulling config    ... done
Pulling gateway   ... done
Pulling catalog   ... done
Pulling wfs       ... done
Pulling wms       ... done
Pulling wcs       ... done
Pulling rest      ... done
Pulling webui     ... done

Then start the services with this command:

$ docker-compose --compatibility up -d

Note the --compatibility argument is required when using a v3 docker-compose file, in order to enable resource the CPU and Memory limits configured in the file for each service.

Wait for the services to start up, check with docker-compose ps until they’re healthy. May some service had exited, re-run docker-compose up -d, initial startup may cause timeouts as services compete for resources on a single machine.

Then browse to:

From a usability point of view, there should be no appreciable differences with a traditional GeoServer deployment.

What’s going on inside the local-machine cluster you’ve just run, is that each GeoServer business capability is running as a separate process in a separate docker container, and they’re all in synch in a loosely coupled way using a message driven event-bus.

Experiment dynamic service scaling and registration with docker-compose scale <service>=<instances>, for example:

docker-compose scale wfs=3 wcs=0
Starting gscloud_wfs_1 ... done
Creating gscloud_wfs_2 ... done
Creating gscloud_wfs_3 ... done
Stopping and removing gscloud_wcs_1 ... done

Technology Overview

With GeoServer being a traditional, Spring Framework based, monolithic Servlet/WebMVC application, a logical choice has been made to base the GeoServer derived microservices in the Spring Boot framework.

Additionally, Spring Cloud technologies enable crucial capabilities such as dynamic service discovery, externalized configuration, distributed events, API gateway, and more.

Only a curated list of the vast amount of GeoServer extensions will be supported, as they are verified and possibly adapted to work with this project’s architecture.

System Architecture Overview

The following diagram depicts the System’s general architecture:

Cloud Native GeoServer Architecture Diagram

  • Hexagons represent microservices;
  • coloured rectangles, logical groupings of components;
  • lines connecting a group to another component: connector applies to all services of the outgoing end, to all components of the incoming end;
  • white rectangles, components that are platform/deployment choices. For example:
    • “Event bus” could be a cloud infrastructure provider’s native service (event queue), or a microservice implementing a distributed event broker;
    • “Catalog/Config backend” is the software component used to access the catalog and configuration. Might be a microservice itself, catalog/config provider for “data directory”, database, or other kind of external service store, catalog/config backend implementations;
    • “Catalog/Config storage” is the storage mechanism that backs the catalog/config software component. Might be a shared “data directory” or database, a “per instance” data directory or database, and so on, depending on the available catalog/config backend implementations, and how they’re configured and provisioned;
    • “Geospatial data sources” is whatever method is used to access the actual data served up by the microservices.

Note the above diagram represents the overall system’s architecture. This is not a deployment diagram. Deployment involves choice of platforms, configurations, and more; without affecting the general architecture. Some microservices/components, though planned and represented in the architecture diagram, have not yet been developed/integrated. For instance: the logging, tracing, and monitoring components, as well as the GWC and WPS microservices.

Project Status

Version 0.2.0 has been released against GeoServer 2.18.2.

Project is being deployed in production since v0.1.0.

Camptocamp, as the original author, is in process of donating the project to OsGeo/Geoserver.

Q: So, is this production ready? Not at all. Cloud Native GeoServer is being used in production for the functionalities needed by Camptocamp’s customer funding this project so far. That does not mean it’s ready for general availability. For instance, these are the most outstanding issues:

  • The security subsystem needs a review. If you try to change the admin password through the web-ui, there’s a bug that would leave you with a broken authentication.
  • The WPS is non functional and temporarily removed from the build
  • There’s no GWC integration so far
  • The catalog-service uses GeoServer’s jdbcconfig community module. It’ll be replaced by a more performance and scalable solution. A lot of work has been put in improving the catalog and configuration design to make that easier. Additionally, we need to work on centralized logging, tracing, and monitoring.

Developer’s Guide

Follow the Developer’s Guide to learn more about the System’s design and how to get started contributing to it.

Deployment Guide

Check out the Deployment Guide to learn about deployment options, configuration, and target platforms.