GeoServer Blog

The 37 gigabyte GML file

As GeoServer matures one of the main focuses becomes the ability to scale - dealing with both large amounts of data and large amounts of users. I got a bit of time to play with GeoServer a week or so ago, and wanted to test out a bit of the large data side of things. On our geoserver demo site, we’ve got about 19 gigabytes of data that we’re serving up. It’s all available through the WMS with OpenLayers on the front end, but the data is never exposed all at once. One of GeoServer’s strengths is the WFS, which provides access to the raw vector data, so I wanted to try to download the whole dataset.

GeoServer has some great fundamental design, as its built in such a way that data is never really held in memory, it streams from the database in to GeoTools java objects and then out to the appropriate output format. So in theory we should be able to stream GML from databases of any size. So that’s what I did, and there were absolutely no memory errors or other problems. Due to the verbose nature of GML the file GeoServer produced was about 37 gigabytes - containing road, landmark, and water data for the whole US, and country boundaries and place names for the world. The data was came down at 4.97 MB/s, which I don’t think is too bad for transforming it to GML on the fly. One interesting thing I noticed that with larger datasets there’s a noticeable pause before the data starts returning. With small datasets the streaming nature of GeoServer tends to produce results right away. So we’ll have to do some investigation to see what’s taking that time - hopefully it’s something we have control over and not buried in PostGIS or some such. I do believe that GeoServer can handle databases of any size, and would love to hear reports from people out there working with even bigger sets of data.

In the next couple weeks we’re also going to have Justin set up a better testing suite for scalability, using JMeter. A number of people have done testing against GeoServer with it before, but in a more ad hoc way. This will build the tests in to the source distribution, so that we are sure it gets run with every release, so we don’t have any regressions. There have been many speed improvements of late, and many more to come, so we want to be sure that other changes in the code don’t accidentally affect things. One more tidbit of scalability news, Gabriel reported that he attended a meeting where someone reported some GeoServer benchmarks, successfully supporting 1000 requests.

Read More

GeoServer 1.5.0-RC2 released

The GeoServer team is proud to announce the second release candidate for the 1.5 series. This release brings us closer to the stable 1.5.0 release of GeoServer that contains the new and exciting coverage support (WCS). To compliment the release we have created more documentation and tutorials, along with some tools to help you process your raster images.

You can download the latest version here.

Changes in this version include:

  • Bug fixes
  • Kml returns proper PNG files
  • Kml strips out illegal characters now
  • Improved error messages
  • UI improvements
  • Inline features are back
  • Links to old homepage updated
  • Performance optimization

The full changelog for this release can be found here.

Read More

Call For Workshops

Several deadlines are approaching to turn in proposals for conferences and events, most notably FOSS4G and Google’s Summer of Code, and we are currently working out what we would like to present. This is a great time to get input from the community to see what you would like us to talk about, demo, teach, and share with everyone. So lets talk briefly about the two events and what we have planned so far:

FOSS4G: This year it is being held in Victoria BC, Canada, and is a great conference to learn about everything geospatial that is open source. Last year we presented a three hour workshop to teach people the basics of setting up and using GeoServer. To plan for this year we would like to start by getting input now on what you would like us to cover. So far we have proposed doing more than one workshop: a basic one and a more advanced one. The basic workshop would cover: installing GeoServer, simple WMS queries, simple WFS queries, and some work with SLD. The advanced workshop, if we have one, could contain a number of topics from advanced WFS-T to creating your own GeoServer plugins in Java. If you have any ideas on what you would like to have us talk about, please let us know. We want to make sure you get the most out of the conference.

Google Summer of Code: This provides a unique opportunity to get your hands dirty with some open source code and get paid at the same time! We are thinking about an entry or two and have started a discussion page. Jot some ideas down on the wiki page or talk about it in this blog post. We are open for any ideas. Current ideas we have are: an SLD editor, GPS to shapefile tool, a server use/availability Google desktop widget.

Read More

TileCache Tutorial

One of the nicest little pieces of software we’ve come across in the last few months has been MetaCarta’s TileCache, which performs a very specific job - caching WMS requests for use in clients that understand tiles - and does it very well. We are making good use of it on our demo site, to give an even better user experience in terms of the speed the map shows up.

Chris and Schuyler gave lots of advice on how to use it in conjunction with OpenLayers, so we thought it’d be worthwhile to write up a new tutorial for GeoServer users looking to make use of it. And there are a few tips and tricks that will likely be useful to others, especially some of the nice things you can do in OpenLayers that make it work better with TileCache, like setting multiple host names to cheat the browsers connection limit and setting OL to try to reload when it gets pink tiles.

We hope to eventually be able to ship a nice integrated GeoServer plug-in that will run a TileCache with a nice GUI and enable easy integration with nice distributed java caches. The ideal would be to share code with TileCache and run in jython, but if that’s not possible we will likely do a hand port and leverage JAI. A big kudos to the MetaCarta programmers, it’s a very nice piece of code.

Read More

GeoServer 1.5.0-RC1 released

We have just released our first release candidate for the 1.5 series of GeoServer that you can download here. The main feature of 1.5 is support for ‘raster’ formats, like geotiff, arcgrid, gtopo30, and more. These are accessible not just through the WMS, but there also through the new Web Coverage Service (WCS) interface.  In line with GeoServer’s mission of making geospatial data more accessible, this allows access to the raw information of rasters, just as WFS does for vector formats. The WCS is passing all OGC compliance tests, and we plan on getting it certified when we go to 1.5.0. This release brings many fixes and improvements and it is also now backwards compatible to 1.3 and 1.4 data directories, so you do not have to port your data over to the new structure. Along with speed improvements, it also had a little reorganization of the user interface and has a couple tutorials to go along with it:

Since this is a release candidate, we would appreciate it if you could all download it and give it a quick try. Mostly for testing the backwards compatibility to old data directories, and for deployment in your existing systems to make sure there is no loss in functionality.

Along with this release are some coverage data assistance tools from Geo-Solutions. They will make your life of processing coverage data easier. It is in an early release stage so please report any bugs or problems back to us. We will be adding more documentation to it in the mean time.

Read More