Templates

This page contains information about the design and implementation of a GeoServer templating system. Here we will nail down the requirements and features of such a system, as well as evaluate a number of the candidate technologies for implementation.

• Velocity
• Freemarker

Requirements

Ease of Use

While the idea of using a template is somewhat of a developer concept, templates are something that end users will interfact with directly, some of which may have little or no programming knowledge. We want creating a new template to be as simple as possible.

Ease of Configuration

Perhaps falling under "ease of use", users will need to be able to add new templates to GeoServer as easily as possible. Adding a new template should be as simple as creating it with a text editor, and saving it to particular location.

Performance

As with anything else, performance is always a key issue. At the end of the day the template system will be applied to large collections of features. The templating engine must be capable of processing these in an efficient, streaming manner.

Secondly, adding a templating engine between pulling our data off of disk and showing to the user is a level of indirection that could lead to a potential bottleneck. We want to make sure we choose an engine that performs well.

Flexibility

Often templating engines out of the box work great for normal java objects or POJO;s as they are commonly refered. However in our domain we are not so lucky as we work with Features. The structure of a feature is something that a templating engine will not know about out of the box, so it needs to be flexible enough and provide an API to allow us to "teach" it about features. The point of which is to allow the end template designer to be able to interfact with features as if they were interacting with a POJO.

Use Cases

An important initial question is "How will templates be used by GeoServer?". A couple of scenarios have already popped up on the mailing lists and in feature requests which are well suited to templates. In particular...

KML Placemarks

In KML placemarks contain a description element, which can contain HTML describing various things about the placemark. When GeoServer outputs KML a placemark is generated for every feature which matche3. d the original request. Currently GeoServer defaults to creating a simple HTML table containing all the attributes of the feature as shown below:

Users have asked for the ability to:

• Show different content, like an image, or a link to some other resource
• Only show a subset of attributes

The possibilities are endless.

GetFeatureInfo Format

The WMS GetFeatureInfo operation defines the info_format parameter which specifies the format that should be used to relay the information about a set of features matching a particular criteria. GeoServer provides a couple of differnt formats out of the box such as plain text, and GML. One of interest is HTML, in which matching features are output in a simple table shown below:

Design

The general workflow of any templating system is generally the same. A service has some data that has been requested, and needs to apply a template to it in order to present it.

Template Loading

The first step in using a template is loading it. Which begs the question "How will templates be loaded by GeoServer?". Since a template is just a file this really depends on how templates are stored. A logical place to store them is in the GeoServer data directory since it is where all configuration for GeoServer gets stored. We have a couple of options for this.

Single Directory

One possibility would be to create a new sub directory in the GeoServer data directory structure called "templates". This could be a single location for all templates to be stored.

data_dir/
  catalog.xml
  services.xml
  featureTypes/
  styles/
  ...
  templates/
     foo.template
     bar.template

FeatureTypes Directory

More often then not, a template will be applied to a single feature collection or single feature. Which means the template is specific to a particular feature type. For his reason it would make sense to store the template in the appropriate directory under "featureTypes", since that is where all configuration specific to a single feature type ( eg. info.xml ), is located.

data_dir/
  catalog.xml
  services.xml
  featureTypes/
     featureType1/
        info.xml
        foo.template
        bar.template
     featureType2/
        info.xml/
        foo.template
        bar.template
  styles/
  ...

A case could be made for both of the above. Some times applying a template will be specific to a feature type, sometimes not. So why not both? Most template engines can be configured to load templates from any location, so it should be possible to configure multiple locations.

Feature Access

More often then not templates will be applied to features and feature collections. So the question becomes how do we want to expose our feature model to someone writing a template? In our feature model a feature can be represented as a row in a table. For sake of example lets consider the following feature collection:

Simple Feature Access

Normally in a template, properties of an object are available via a simple variable syntax. Lets consider for a moment that our feature was modelled as a java bean:

class Feature {
  
  String getID();

  Geometry getGeometry();

  Integer getIntProperty();

  Double getDoubleProperty();

  String getStringProperty();
  
}

In a template, we would be able to access properties like the following:

fid = ${fid}
geometry = ${geometry}
intProperty = ${intProperty}
doubleProperty = ${doubleProperty}
stringProperty = ${stringProperty}

Resulting in:

fid = fid.1
geometry = POINT(1 1)
intProperty = 1
doubleProperty = 1.1
stringProperty = one

This is nice because it is a simple syntax and is something that is generally standard across most templating engines and syntaxes. So it would desirable to adopt this same syntax for our features.

Dynamic Feature Access

While the above syntax is nice and simple, it may be a bit too simple for some users. For one it requires that the template writer know exactly what the attributes of the feature type are before hand. This may not be desirable. Perhaps some more dynamic or reflective access is required.

One solution would be to introduce a special variable called attributes, which could be a simple list of the attributes for the feature. Each attribute could contain a name,value pair. Access in a template would look something like:

for each a in ${attributes} do
  ${a.name} = ${a.value}
end

Resulting in:

geometry = POINT(1 1)
intProperty = 1
doubleProperty = 1.1
stringProperty = one

Implementation

There are a number of templating libraries at our disposal each with its own features and advantages. A extensive list can be found at http://java-source.net/open-source/template-engines. We will evaluate some of the more popular ones here. Please feel free to add additional information to this section in particular if you are familiar with a particular templating technology.

For each evaluation, we will try to answer the following questions.

1. How can we invoke a template?
2. How can we control the template lookup mechanism?
3. How can we provide easy access to features and what does access look like?

Velocity

Velocity is probably the most well-known of template engines.

Template Invocation

The general recipe for invoking a velocity template is as follows:

import org.apache.velocity.app.VelocityEngine;
import org.apache.velocity.Template;

...

/*
 *  create a new instance of the engine
 */

VelocityEngine ve = new VelocityEngine();

/*
 *  configure the engine.  In this case, we are using
 *  ourselves as a logger (see logging examples..)
 */

ve.setProperty( VelocityEngine.RUNTIME_LOG_LOGSYSTEM, this);

/*
 *  initialize the engine
 */

ve.init();

...

Template t = ve.getTemplate("foo.vm");

More details here.

Template Loading

Configuring template locations is pretty easy with velocity. It provides a simple properties format for controlling how templates are loaded. For instance, setting templates to be loaded from a particular directory:

//create the template engine
VelocityEngine engine = new VelocityEngine();

//set the location to loade templates from
Properties properties = new Properties();
properties.put( "file.resource.loader.path", GeoServerDataDirectory.getConfigDir( "templates" ) );
engine.init( properties );

Getting a bit more advanced, overriding the template lookup mechanism is also pretty easy. Consider a custom template loader which can load templates dynamically from a GeoServer data directory by extending the velocity FileResourceLoader.

public class DataDirectoryFileResourceLoader extends FileResourceLoader {

	public synchronized InputStream getResourceStream(String templateName)  throws ResourceNotFoundException {

              //first search the templates directory under data directory
	      File templates = GeoserverDataDirectory.findConfigFile( "templates");
              File template = new File( templates, templateName );
              if ( template.exists() ) {
                   return new FileInputStream( template );
              }

              //next search each feature type directory
              File featureTypes = GeoserverDataDirectory.findConfigFile( "featureTypes" );
	      File[] directories = featureTypes.listFiles();
              for ( int i = 0 ; i < directories.length; i++ ) {
                   File featureTypeDirectory = directories[i];
                   File template = new File( featureTypeDirectory, templateName );
                   if ( template.exists() ) {
                            return new FileInputStream( template );
                   }
              }

              //could not find
              throw new ResourceNotFoundException();
        }

}

Pointing the engine at the custom template loader is achieved again with a property:

//create the template engine
VelocityEngine engine = new VelocityEngine();

//set the custom template loader
Properties properties = new Properties();
properties.put( "file.resource.loader.class", GeoServerDataDirectory.class.getName() );
engine.init( properties );

Feature Access

I found that Velocity was a bit lacking when it came to teaching it about non java bean based object models like the geotools feature. Setting properties in a "context" is somewhat tedious and uses a map-like api:

//we have access to some feature
Feature feature = ...; 

//create the context and set the feature attributes
VelocityContext context = new VelocityContext();
for ( int i = 0; i < feature.getNumberOfAttributes(); i++) {
  //get the attribute name and value
  String name = feature.getFeatureType().getAttributeType( i ).getName();
  Object value = feature.getAttribute( i );

  //set the property in the context
  context.put( name, value );
}

//also set the feature id
context.set( "fid", feature.getID() );

//execute the template
engine.mergeTemplate( "foo.vm", ... );

Access in a template then becomes:

fid = ${fid}
geometry = ${geometry}
intProperty = ${intProperty}
doubleProperty = ${doubleProperty}
stringProperty = ${stringProperty}

There a couple of things to note about this approach:

1. It is somewhat tedious for the person who is setting up the template
2. It falls apart if we try to add a collection directly to the context

For 1, there is some light at the end of the tunnel. Velocity does contain the FieldMethodizer class which it used to be able to provide access to fields of a class directly. A quick subclass hack can be used to adapt it to feature attributes.

public class FeatureMethodizer extends FieldMethodizer {
		
	Feature f;
	
	public FeatureMethodizer( Feature f ) {
		super();
		this.f = f;
	}
	
	public Object get(String fieldName) {
             if ( "fid".equals( fieldName ) ) {
                  feturn f.getID();
             }

             return f.getAttribute( fieldName );
	}
	
}

Instances of this class can then wrap up features as they are thrown in the context:

context.put( "f", new FeatureMethodizer( feature ) );

For number 2, consider the situation of having a FeatureCollection and wanting to apply the template to the entire collection.

//we have access to a feature collection
FeatureCollection featureCollection = ...; 

//create the context and set the feature attributes
VelocityContext context = new VelocityContext();

//also set the feature id
context.put( "features", featureCollection );

//execute the template
engine.mergeTemplate( "foo.vm", ... );

We lose the ability to set features in the context, which means that access in the template becomes more complicated.

#foreach ( $f in $features ) 
fid = $f.getID()
geometry = $f.getAttribute( "geometry" )
intProperty = $f.getAttribute( "intProperty" )
...
#end

On the one hand we lose our very simple access to features as things become much more verbose. On the other hand we provide the template writer with a lot of flexibility as they have full access to the feature api.

Freemarker

Freemarker is another popular java templating engine.

Template Invocation

The general recipe for invoking a freemarker template is as follows:

import freemarker.template.*;
import java.util.*;
import java.io.*;

/* ------------------------------------------------------------------- */    
/* You usually do it only once in the whole application life-cycle:    */    
    
/* Create and adjust the configuration */
Configuration cfg = new Configuration();
cfg.setDirectoryForTemplateLoading( new File("/where/you/store/templates"));
cfg.setObjectWrapper(new DefaultObjectWrapper());

/* ------------------------------------------------------------------- */    
/* You usually do these for many times in the application life-cycle:  */    
                
/* Get or create a template */
Template temp = cfg.getTemplate("test.ftl");

/* Create a data model */
Map root = new HashMap();
root.put("user", "Big Joe");
Map latest = new HashMap();
root.put("latestProduct", latest);
latest.put("url", "products/greenmouse.html");
latest.put("name", "green mouse");

/* Merge data model with template */
Writer out = new OutputStreamWriter(System.out);
temp.process(root, out);
out.flush();
}

More info here.

Template Loading

Similar to Velocity customizing template loading relatively straight forward. Loading a template from a particular location looks like:

//Create the configuration
Configuration cfg = new Configuration();

//set the location to load templates from
cfg.setDirectoryForTemplateLoading( GeoServerDataDirectory.getConfigDir( "templates" ) );

And again, its also easy to create a custom template loading implementation:

public class GeoServerTemplateLoader implements TemplateLoader {

	/**
	 * Delegate file based template loader
	 */
	FileTemplateLoader loader;
	
	public GeoServerTemplateLoader() throws IOException {
		//create a file template loader to delegate to
		loader = new FileTemplateLoader();
	}
	
	public Object findTemplateSource(String path) throws IOException {
		
		//first check the templates directory
		File templates = GeoserverDataDirectory.findConfigFile( "templates" );
		if ( templates.exists() ) {
			template = (File) loader.findTemplateSource( 
				new File( templates, path ).getAbsolutePath()
			);
		}
		if ( template != null ) {
			return template;
		}
		
		//next, try relative to feature types
		
		File featureTypes =  GeoserverDataDirectory.findConfigFile( "featureTypes" );
		File[] directories = featureTypes.listFiles();
		for ( int i = 0; i < dirs.length; i++ ) {
                        File featureTypeDirectory = directories[ i ],
;
			Template template= = (File) loader.findTemplateSource(
				new File( featureTypeDirectory, path ).getAbsolutePath()	
			);
			if ( template != null ) {
				return null;
			}

		}
			
	}

	public long getLastModified(Object source) {
		return loader.getLastModified( source );
	}

	public Reader getReader(Object source, String encoding) throws IOException {
		return loader.getReader( source, encoding );
	}
	
	public void closeTemplateSource(Object source) throws IOException {
		loader.closeTemplateSource( source );
	}

}

Feature Access

Freemarker provides an interface called BeansWrapper for explicitly wrapping up non java bean objects. An implementation for wrapping up features could look like:

public class FeatureWrapper extends BeansWrapper {
	
	public TemplateModel wrap(Object object) throws TemplateModelException {
	
		//check for feature collection
		if ( object instanceof FeatureCollection ) {
			//create a model with just one variable called 'features'
			HashMap map = new HashMap();
			map.put( "features", new SimpleSequence( (FeatureCollection)object, this ) ); 
			return new SimpleHash( map );
		}
		else if ( object instanceof Feature ) {
			Feature feature = (Feature) object;
			
			//create the model
			HashMap map = new HashMap();
			
			//first add the feature id
			map.put( "fid", feature.getID() );
			
			//next add variables for each attribute, variable name = name of attribute
			for ( int i = 0 ; i < feature.getNumberOfAttributes(); i++ ) {
				AttributeType type = feature.getFeatureType().getAttributeType( i );
				
                                map.put( type.getName(), feature.getAttribute( i ) );
                        }
			
			return new SimpleHash( map );
		}
		
		return null;
	}

One of the really nice things about this implementation is that it handles both features and feature collections transparently. Using it in a template becomes:

//Create the configuration
Configuration cfg = new Configuration();

//set the feature object wrapper
cfg.setObjectWrapper( new FeatureWrapper() );

Proposed Example of Templating for KML

KML has had the most requests to be "templatable": people want to limit or style the amount of information returned to the user for their placemarks.

The possible requirements for a user are:

• Template associated with the SLD file, specifically for KML rendering
• A template for each rule in each feature type
• The ability to surround the information the template produces with user defined HTML

In the SLD file, an example of how a template could be specified:

<FeatureTypeStyle>
	<FeatureTypeName>Feature</FeatureTypeName>
	<Rule>
		<Name>my rule</Name>
		<PointSymbolizer>...</PointSymbolizer>
		<VendorOption name="kmlDescriptionTemplate">myPointKml.template</VendorOption>
	</Rule>
</FeatureTypeStyle>

The template could be specified in a vendor option, or our own tag if we want to modify the SLD schema.

It should be noted that KML features will be rendered with a placemark whether or not a text symbolizer is defined.

In the template, it could look something like this:

The name of this feature is: <b>${name}</b>
You can find it at ${pointGeom}
To find out more information about this feature visit this <a href="http://example.com/listfeature.php?fid=${fid}">link</a>