GeoServer on steroids - GeoSolutions Demo Server

GeoServer on steroids All you wanted to know about how to make GeoServer faster but you never asked (or you did and no one answered)

Ing. Andrea Aime, GeoSolutions Ing. Simone Giannecchini, GeoSolutions

FOSS4G 2013, Nottingham 20th September 2013

GeoSolutions 

Founded in Italy in late 2006



Expertise







•

Image Processing, GeoSpatial >0.5


Scale dependent rules 

Too often forgotten or little used, yet very important: 







Hide layers when too zoomed in (raster/vector example) Progressively show details Add more expensive rendering when there are less features

Key to any high performance / good looking map


Example


Hide as you zoom in 



Add a MinScaleDenominator to the rule

This will make the layer disappear at 1:75000 (towards 1:1)


Alternative rendering 



Simple rendering at low scale (up to 1:2000) More complex rendering when zoomed in (1:1999 and above)


Alternative rendering


Point symbols

• 600 loc for 6 different points types • Painful… FOSS4G 2013, Nottingham 20th September 2013

Prepare expiration="7200" />


More Tweaks 

Use the right formats  

   



JPEG for background data (e.g. ortos) PNG8 + precomputed palette for background data (e.g. ortos) PNG full for overlays with transparency PNG8 full for overlays with transparency

Don’t compress things twice!

The format impacts also the disk space needed! (as well as the generation time) Check this blog post FOSS4G 2013, Nottingham 20th September 2013

Resource control


WMS request limits 





Max memory per request: avoid large requests, allows to size the server memory (max concurrent request * max memory) Max time per request: avoid requests taking too much time (e.g., using a custom style provided with dynamic SLD in the request)

Max errors: best effort renderer, but handling errors takes time


WFS request limits 

Max feature returned, configured as a global limit



Return feature bbox: reduce amount of generated GML



Per layer max feature count


WCS request limits


Control flow 



Control how many requests are executed in parallel, queue others: 

Increase throughput



Control memory usage



Enforce fairness

More info here FOSS4G 2013, Nottingham 20th September 2013

Control flow

17%

$GEOSERVER_DATA_DIR/controlflow.properties # don't allow more than 16 GetMap requests in parallel ows.wms.getmap=16 FOSS4G 2013, Nottingham 20th September 2013

Auditing 

Log each and every request



Log contents driven by customizable template



Summarize and analyze requests with offline tools



More info here


JVM and deploy configuration


Premise 





The options discussed here are not going to help visibly if you did not prepare the data and the styles They are finishing touches that can get performance up once the major data bottlenecks have been dealt with Check “Running in production” instructions here


JVM settings 









--server: enables the server JIT compiler --Xms2048m -Xmx2048m: sets the JVM use two gigabytes of memory --XX:+UseParallelOldGC -XX:+UserParallelGC: enables multi-threaded garbage collections, useful if you have more than two cores --XX:NewRatio=2: informs the JVM there will be a high number of short lived objects --XX:+AggressiveOpt: enable experimental optimizations that will be defaults in future versions of the JVM


Native JAI and JDK 



Install native JAI and use a recent Sun JDK! Benchmark over a small data set (the effect is not as visible on larger ones)


Setup a local cluster 

Java2D locks when drawing antialiased vectors 



Limits scalability severely

Use Apache mod_proxy_balance and setup a GeoServer each 2/4 cores mod_proxy_balance

GeoServer

GeoServer

GeoServer


Clustering advantage 



FOSS4G 2010 vector benchmarks (roads/buildings/isolines and so on, over the entire Spain) GeoServer was benchmarked without local clustering

66%


Benchmarking


Using JMeter 



Good benchmarking tool Allows to setup multiple thread groups, different parallelelism and request count, to ramp up the load



Can use CSV files to generate semi-randomized requests



Reports results in a simple table

http://jakarta.apache.org/jmeter/


Using JMeter 









Thread group: how many threads Loop: how many requests HTTP sampler: the request CSV: read request params from CSV Summary table


Generating the CSV 



Simple randomized generation tool built during WMS shootouts, wms_request.py Generate csv with the bbox and width/height to be used in JMeter scripts: ./wms_request.py -count 1200 -region -180 -90 180 90 -minres 0.002 -maxres 0.1 -minsize 256 256 -maxsize 1024 1024



Get it here along with a corresponding JMeter script:

http://demo1.geo-solutions.it/share/jmeter_2011.zip


Checking results 

Results table



Run the benchmarks 2-3 times, let the results stabilize



Save the results, check other optimizations, compare the results


Real world deploy


Deploy configuration


Raster data 







Whole Italy at 50cm per pixel Over 4TB, updated fully every 3 years (old data still available for historical access) Custom pyramid 

100 m per pixel: one image



20m per pixel: mosaic of 20 tiles



4m per pixel: mosaic of few hundred tiles



0.5m per pixel: 9000 tiles

Each tile is 10000x10000, with overviews


Vector data 









Cadastral data for the whole Italy, with full history (interval of validity for each parcel) 100 million polygons A query extracts a subset relative to a certain time interval and area the user is allowed to see

No data from this table is ever shown below 1:50000 (SLD scale dependencies) Physical table level partitioning (Oracle style) of the table based on geographic area to parallelize and cluster data loading, plus spatial indexing and indexes on commonly filtered upon attributes


The End

Questions? [email protected] [email protected] FOSS4G 2013, Nottingham 20th September 2013