% Template For OSGeo Journal
% Language: Latex
% See article_long.tex for more examples, including use of urls, code examples, etc.

% Head

\title{A New Zealand case study}
\subtitle{Open Source, Open Standards, Open Data}
\author{by Brent Alexander Wood 

\medskip
\noindent{NIWA, New Zealand. \email{pcreso@pcreso.com}}}

\maketitle

\subsection*{Abstract}

The National Institute for Water and Atmospheric Research (NIWA) is New Zealand's leading agency providing freshwater,
ocean, climate, atmosphere and fisheries related research. Open Source software is widely used internally, both
infrastructurally and in desktop systems. 

In 2011, the New Zealand Government passed ``The Declaration on Open and Transparent Government{}''. This requires
central government agencies to make taxpayer funded information freely available to the public, and encourages regional
and local government, as well as agencies such as NIWA to comply. NIWA works closely with central and regional
government, utilities, NGO's and primary industry, making information discovery and delivery using common and open
standards critical.

NIWA is using Open Source applications to meet these open data discovery and delivery requirements. Open Geospatial
Consortium (OGC) standards compliance ensures interoperability. Standards adopted to date include SFS (Postgis), CSW
(Geonetwork), WMS/WFS (Mapserver, Geoserver, Openlayers, Quantum GIS, Quantum Map) and SOS (52\textsuperscript{o}N,
Quantum Map). Some proprietary applications are also used. These are also OGC compliant and fit within NIWA's OGC based
architecture.  This paper describes the role that open source software and open standards play in NIWA's strategies and
architecture for environmental information management, discovery and delivery and gives implementation examples. 


\textbf{Keywords: }
environmental data,  interoperability, NIWA, OGC, open data, open source, OSGEO, web services.


\section[1.0 Introduction]{Introduction}
This paper presents several examples of using Open Source GIS related tools to manage and deliver environmental data
captured by a variety of research programmes undertaken by the National Institute for Water and Atmospheric Research in
New Zealand (NIWA, \url{http://www.niwa.co.nz}). 

The systems are fully compliant with open industry standards developed by the Open Geospatial Consortium (OGC) which are
the basis of New Zealand government standards for spatial data interoperability (NZGO, 2011), to comply with the
recommendations of the New Zealand Government's 2011 Declaration on Open and Transparent Government: 

{}``To support this declaration, the government asserts that the data and information it holds on behalf of the public
must be open, trusted and authoritative, well managed, readily available, without charge where possible, and reusable,
both legally and technically. Personal and classified data and information must be protected.

Public service and non-public service departments are directed and State Services agencies encouraged to commit to the
release of high value public data for re-use in accordance with the declaration and principles.``

NIWA has therefore a responsibility to ensure the publicly funded environmental data it holds, as well as the results of
research and analyses on these data, are made readily available. This is being done primarily using Open Source tools
to implement data (and metadata) discovery and delivery capabilities for NIWA held data.

\section{NIWA's environmental information needs.}
NIWA needs integrated and interoperable systems to manage its various and heterogeneous environmental data and
information holdings, and to provide search and access facilities to enable discovery and delivery, both internally,
within its various science centres, and externally, for clients, collaborators, other agencies and the public.  

\subsection{NIWA's internal requirements}

NIWA is New Zealand's leading environmental science provider in the aquatic, climate, atmospheric and fisheries domains.
As a New Zealand Crown Research Institute (CRI), NIWA operates as a commercial business, although the New Zealand
government is the only shareholder. Internally, NIWA is divided into science centres, each focused on one of NIWA's
environmental science domains. Given the highly inter-related nature of physical, chemical and biological data in fresh
water, coastal and oceans science, and of climate research on freshwater availability, etc, there is within NIWA a
critical need for information management, discovery and delivery systems that support all the areas of science NIWA
undertakes, and are not domain specific silos within the organisation. 

\ \ Virtually all environmental data has a critical spatial component, so NIWA's environmental data solutions are very
much spatial data soutions, hence the significant use of OSGEO applications and tools in NIWA (as well as other open
source tools), for integrated spatial data management, discovery and delivery systems.


\subsection{Client interoperability 
requirements}


NIWA clients include a wide range of goverrnment agencies (central, regional and local), utilities (eg. wind and hydro
power generation, electricity transmission), industry (e.g. agriculture, irrigation). NIWA also collaborates with
international agencies \& has international obligations on behalf of New Zealand. NIWA's data management and delivery
systems therefore need to support a wide range of relationships with local, national and international agencies.

To enable data and information discovery and delivery services accross such a wide range of users and clients from
internal, regional, national and international levels, a high degree of compliance with industry standards across
several domains is necessary. 

NIWA is following the OGC and ISO standards recommended for New Zealand Government agencies, and is both internally and
collaboratively developing a range of standards, tools and systems for users and clients which provide suitably
interoperable information discovery and delivery facilities. These make  use of Open Source tools and components from
the Open Source Geospatial Foundation (OSGEO), but include other Open Source applications and tools, as well as some
commercial applications.

Where there are no New Zealand recommended standards for particular domains, NIWA is collaborating with other New
Zealand agencies to adopt or devise suitable standards, and is also developing working applications as exemplars for
other New Zealand agencies.

\subsection{NIWA's history with open source}


NIWA has a long history of supporting and internal use of open source applications. In support of their 
research projects, various NIWA staff have used open source applications for some years. Also, Linux servers have been
used as web and database servers for some time, generally being used to replace older UNIX servers during the 1990's.
In the geospatial arena, given New Zealand's location in the world, NIWA has to contend with well known issues relating
to the 180\textsuperscript{o} meridian. To help work around these, NIWA funded the original implementation of the
ST\_Shift\_Longitude() Postgis function (\url{http://postgis.net/docs/manual-2.0/ST_Shift_Longitude.html}), as well as
the  cs2cs parameter ``+lon\_wrap{}'' for
Proj.4 (\url{http://trac.osgeo.org/proj/wiki/GenParms#lon_wrapover-LongitudeWrapping}). NIWA also developed the new
vector format for Generic Mapping Tools (GMT) v5.0 (Wessel et al, 2011), along with funding the GDAL/OGR driver for
this format, enabling GMT to interoperate with many other open source GIS related applications and tools.
NIWA also supports other open source tools such as GERRIS (Popinet, 2003), CYLC (Oliver, 2012) and has twice been
a finalist in the New Zealand Open Source Awards.


\subsection{Data management and delivery strategy}

To support information needs as outlined above NIWA has adopted a strategy for information management, discovery and
delivery. Part of the strategy is the support of core infrastructure components through standardized database
solutions; these can be open source or commercial, depending on the science and user needs. Metadata and Data are
exposed using (open or industry) standard compliant web services where possible. NIWA's strategy for meeting its
spatial data interoperability needs is largely based on OGC standards enabled by open source applications. This
approach ensures the delivery systems are decoupled from the underlying data management systems (see Figure 1).

The main information infrastructure components used at NIWA:

\begin{itemize}
\item Metadata Catalog (Geonetwork)
\item Station Catalog (Postgis)
\item Image Catalog (AtlasMD - commercial product)
\item Spatial Layer Archive (ESRI -- commercial product)
\item Taxa Catalog and Information System (Posgis)
\item Timeseries databases (various bespoke, commercial and open source products)
\item Observation databases (Postgis)
\item Marine and atmospheric data archives (netCDF, Postgis)
\item Product Catalog (commercial and open source products)
\item Webservice solutions (Mapserver, Geoserver and industry standard solutions)
\item Publishing  and Reporting System (Jasper)
\item Web delivery (Openlayers and others)
\end{itemize}


\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img001.pdf}
   \caption{Internally managed data (or metadata) from multiple data stores are provided using OGC compliant (CSW, WFS,
WMS, SOS) and other web services to the internet, enabling internal and external consumption, via both web and desktop
clients.}
   %\label{fig:figure_label}
\end{figure}

As one can see NIWA{}'s information infrastructure is based on a complex system of interacting components, with open
source and open standards solutions making up a large component of that infrastructure.

\section{Use of open 
 source and open standards for 
 NIWA's environmental information discovery 
 and delivery}

In 2011, NIWA established an Environmental Information Centre (EIC, https://www.niwa.co.nz/our-science/ei). This was a
significant change for NIWA, as all previous NIWA Research Centres were focused on specific science domains. The new
centre has a mandate to work across the science domains, fulfilling a strategic  role and supporting data monitoring,
management and delivery capabilities throughout NIWA, as well as working with clients and other stakeholders in New
Zealand's environmental research and management arena to facilitate data sharing and re-use. 

The EIC is supporting the use of open source tools to achieve its goals, and in line with the freedoms that open
source provides, supporting the re-use of the systems and standards both inside NIWA and externally to clients and
other agencies.

The following sections describe examples for NIWA's various uses of open source applications and open standards to
enable this strategy.

\subsection{IPY/CAML metadata catalogue}

One of the first open source applications formally used by NIWA to meet contracted requirements for data discovery and
delivery (rather than as an in-house research tool) was Geonetwork (\url{http://geonetwork-opensource.org/}). 

NIWA undertook a research voyage to the Ross Sea as part of the International Polar Year/Census of Antarctic Marine
Life (IPY/CAML) during 2008, which included a requirement to make all datasets and reports publicly available. 

At this time, Land Information New Zealand (LINZ), the government agency responsible for land titles, geodetic and
cadastral survey systems, topographic information, hydrographic information, managing Crown property and a variety of
other functions, was also establishing a metadata catalogue, a national catalogue for geospatial data holdings
(\url{http://www.geodata.govt.nz}), in conjunction with the
government agency responsible for science funding, the Ministry of Research, Science and Technology (MORST). This was
also based on an underlying Geonetwork catalogue, with website development undertaken by the local web development
company, Silverstripe, using their open source CMS of the same name (\url{http://www.silverstripe.org/}). NIWA chose to also
contract the initial implementation and hosting of its catalogue to Silverstripe.  

A Geonetwork metadata catalogue was established and populated to successfully provide discovery and delivery
capability. This has since been moved to sit within a more generic web portal established to provide information
discovery and delivery facilities for NIWA research projects,
(\url{http://www.os2020.org.nz/new-zealand-ipy-caml-project/}).

One side effect of these initiatives, in line with the increasing New Zealand Government's support of open data, was
a workshop on metadata catalogues run by LINZ, using Geonetwork as the enabling tool. This was attended by
representatives from central and regional government, universities, NIWA and other CRI's, utilities and private
businesses, and was a seminal influence in the ongoing development of information discovery and delivery systems in New
Zealand. 


\subsection{Ocean Survey 20/20 Bay of Islands web portal.}

In 2010, NIWA undertook an extensive multidisciplinary survey off the north east coast of the North Island (see Figure
2) with a particular focus on the Bay of Islands, as part of the New Zealand Ocean Survey 20/20 initiative. 

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img002.png}
   \caption{Map showing the Bay of Islands survey area, including the seabed bathymetry from shallow inshore areas to
water up to 50 metres deep.}
   %\label{fig:figure_label}
\end{figure}

This
project included a requirement to provide public access to the project's data and reports, as well as describing and
documenting the survey and its many participating agencies and staff. A web portal was chosen as the appropriate
solution, and was developed using the following open source tools (mostly from the OSGEO stack) enabling both server
and client capability:
\begin{itemize}
\item Geonetwork metadata catalogue for information discovery and download
\item Postgis spatial database server
\item Mapserver to provide OGC WMS/WFS contextual and data layers
\item Openlayers as the web map client 
\item Silverstripe as the enabling CMS, with both:
\end{itemize}
\begin{enumerate}
\item a map administration tool supporting an embedded Openlayers WMS/WFS client
\item a CSW client providing embedded user friendly access to the Geonetwork metadata catalogue.
\end{enumerate}

This solution is based on a client/server architecture which allows the data management and delivery (server)
functionality to be separated from the portal (client). Using OGC web services enables the portal to display content
from various independent data repositories, including Postgis databases and ESRI ArcServer/SDE data stores. This also
enables other agencies and users to reuse the WMS and WFS layers in their own applications, web sites or desktop
mapping applications as desired.

Silverstripe developed an Openlayers administration module for the open source Silverstripe CMS for this project.
This tool allows the web site administrators to manage the web portal{}'s map map pages, by creating and managing maps
and map layers. Layers are assigned to maps, and are defined as a URL for an OGC WMS or WFS service, along with
information required to manage the map and the layers it comprises. This tool separates the roles of GIS and web site
administration. GIS staff are responsible for systems managing the spatial data and serving the map layers, and web
site administrators manage the maps and layers within the web site.  The tool also enables re-labeling of WFS data
fields for feature queries, and integrates metadata catalogue searches from the map, by allowing a set of keywords to
be associated with a map layer, with the ability to for a user to request a catalogue search from the map interface,
allowing direct access to relevant reports, datasets, map images and other documents directly from the map interface.  

Some of the datasets and samples captured during the survey were analysed and made available as OGC web service map
layers using ESRI ArcServer. Some 20,000 seabed photographs captured during the survey are managed in Atlas, a
commercial digital asset management tool used by NIWA as an image database. NIWA, Silverstripe and the Atlas developers
implemented a custom web API to provide the Atlas images to the portal map via OpenLayers (see Figure 3).

The Silverstripe CMS also manages web pages describing the methods used during the survey, including physical gear which
was deployed, manual sampling methods, and for analytical results, descriptions of the analyses undertaken. The
embedded Openlayers administration tool includes a link to these pages, so the user can view the layer's associated
method information directly from the Openlayers layer menu, simply by clicking on the layer name (see Figure 4). This
provides a map-centric approach to the web portal, with users able to access a variety of non-map content directly from
the map.


\medskip
% The following is used for inserting an image
\begin{figure}
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img003.png}
   \caption{A custom web service allows users to view over 20000 seabed images from the Bay of Islands survey by clicking
on a point displayed inthe Openlayers map.}
   %\label{fig:figure_label}
\end{figure}


% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img004.png}
   \caption{Clicking on the name of a layer in the OpenLayers layer menu displays the layer method page from the CMS, as
configured in the Silverstripe OpenLayers administration module, providing a map-centric user interface embedded within
the CMS.
}
   %\label{fig:figure_label}
\end{figure}

The original Silverstripe portal has since been redesigned to support multiple projects within a single
implementation, as well as being able to be redeployed as a completely new portal instance where appropriate. This
maximises the potential for re-use of the portal as a complete spatial data discovery and delivery system, either by
adding new projects to an existing portal, or by redeploying new instances of the portal. It is now being used as a
generic web interface for NIWA research projects which require a web presence or interface
(\url{http://www.os2020.org.nz/}).



\subsection{OS2020 Chatham/Challenger 
 survey web portal}
 
 % The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img005.png}
   \caption{The ``species picker'' allows the user to select a species, to display two separate layers, the observed
locations of the species as a WFS layer, on the Chatham Rise to the east of New Zealand and the Challenger Plateau to
the west, and the modelled, predicted species distribution, via WMS.
}
   %\label{fig:figure_label}
\end{figure}



Another survey within the OS20/20 research programme examined the biodiversity of the Chatham Rise and Challenger
Plateau, off the east and west coasts of New Zealand respectively. After the survey was completed, an additional
contract required NIWA to make the data and reports available to the public by reusing the successful Bay of Islands
portal. Additional biodiversity content required some enhancements to the Silverstripe map administration tool,
noteably regarding a ``species picker'' facility (see Figure 5) for users to select from a list of available species,
providing the ability to display both a map  layer showing where the species was observed, as well as (where available)
a layer showing the predicted distribution. Following the client/server via web service architecture of the portal, the
list of species is also provided via a web service, so is driven from the server databases. This project also required
the seamless display of data across the 180\textsuperscript{o} meridian, enabled using Postgis, Mapserver and
Openlayers.


\subsection{The New Zealand Marine Biosecurity Porthole}

In 2011, NIWA was approached by Biosecurity New Zealand to create a new instance of the portal to provide access to New
Zealand marine biosecurity data. This was done (\url{http://www.marinebiosecurity.org.nz/}), and provides an example of
the easy redeployment and reuse of systems built using open source tools (see Figure 6). Biosecurity survey data was
loaded into a Postgis database with Mapserver used to deliver the data via OGC WMS and WFS services. The system uses
the Silverstripe CMS, along with both the Openlayers web mapping tool and the Geonetwork metadata catalogue to provide
marine biosecurity data and reports to stakeholders in New Zealand.


\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img006.png}
   \caption{The New Zealand Marine Biosecurity Porthole provides access to New Zealand's marine biosecurity data and
related information for stakeholders.
}
   %\label{fig:figure_label}
\end{figure}



\subsection{NIWA Data Catalogue}

NIWA's Environmental Information Centre has initiated several new projects and has continued to maintain and enhance
existing ones, relying heavily on open source components, building on NIWA's experience with the Bay of Islands portal.


The NIWA Data Catalogue (\url{http://dc.niwa.co.nz/niwa_dc/srv/en/main.home}) was implemented to provide a searchable
discovery facility for NIWA staff (internally) as well as clients and other users (externally). Based on our successful
deployment of Geonetwork for project based information discovery and delivery systems, NIWA chose Geonetwork for its
institutional catalogue. 

NIWA has developed a custom tool to facilitate the entry of metadata records, called Postcard (see Figure 7). This
presents the metadata fields to the user within a series of tabs, each covering a different aspect of the metadata,
including ``who'', ``where'', ``when'', etc, which provides a user friendly metadata entry tool. The tool is able
to automatically fill some fields based on the user id, and the knowledge that NIWA is the responsible agency. All
records are saved to GeoNetwork, but are not published until approved by the NIWA metadata curator, who is notified by
an email from the Postcard tool about each new record. 


\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img007.png}
   \caption{The NIWA postcard tool provides a user friendly interface for metadata entry. Tabs break up the content into
subject areas, pick lists provide a simple controlled vocabularies for appropriate fields, The tool automatically
emails NIWA's metadata curator of each record entered. The curator then checks the new record meets NIWA guidelines
before enabling public access to it.  }
   %\label{fig:figure_label}
\end{figure}


Records from NIWA's catalogue are automatically and selectively harvested into other catalogues, including national and
international catalogues. NIWA has been pleasantly surprised with how often such catalogues are also using Geonetwork,
making harvesting and interoperability relatively straightforward.

NIWA has adopted the Australian Marine Communities Metadata Profile as the preferred profile for its metadata. This is
an ISO19115/19139 compliant profile, based on the ANZLIC (Australia and New Zealand Land Information Council) profile,
with enhancements for marine metadata (AODC, 2008), which has been supported by GeoNetwork almost since its inception.

\subsection{NIWA Environmental Information browser}

The Environmental Information Browser (EIB) is a web based facility (\url{http://ei.niwa.co.nz/}) enabling concurrent
searches on several NIWA catalogues and databases which deliver information through OGC web services (see Figure 8).
Users can direct searches against specific systems, or across several systems or datasets at once. Searches support
keywords, as well as geographic and temporal constraints. The EIB is in a state of ongoing development as further
databases and new functionality are added. The EIB itself is a bespoke application implemented using the open source
Symfony framework, which makes extensive use of OGC web services to search and deliver content from the various
underlying systems. The spatial databases it accesses generally implemented using Postgis, served as OGC WMS and WFS
services via Geoserver or MapServer. The data catalogues are GeoNetwork instances searched using OGC CSW.

Given that the underlying NIWA systems accessed by EIB are providing CSW, WMS and WFS web services, other
applications and agencies are able to seamlessly embed content from these NIWA systems in their own applications.

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img008.png}
   \caption{NIWA's Environmental Information Browser provides a single search facility across several systems at once,
with spatial, temporal and keyword filters.  }
   %\label{fig:figure_label}
\end{figure}


\subsection{NIWA Taxonomic Information System}

NIWA is developing three linked applications to provide a suite of taxonomic data capabilities for internal and external
users, which together comprise the Taxonomic Information System (TIS, \url{https://tad.niwa.co.nz/}). 

While taxonomic data are not inherently spatial themselves, they are critical information supporting a wide range of
environmental science, which is decidedly spatial in nature. NIWA in integrating these with our spatial biodata
systems, such as the web portals described above, so the taxonomic systems being developed and their open source
components are included here.

Open source components used for these applications include:
\begin{itemize}

\item PHP The main programming language for the web interface
(\url{http://php.net/})
\item The Symfony framework (\url{http://www.symfony-project.org/})
\item JQuery -- javascript library for web user interfaces
(\url{http://jquery.org/})
\item Jasper Reports (\url{http://community.jaspersoft.com/wiki/community-wiki})
\item D3 Data Driven Documents
(\url{http://d3js.org})
\item PostgreSQL database (\url{http://www.postgresql.org})
\item Apache Webserver (\url{http://www.apache.org})
\item Apache Tomcat Java Servlet and JSP framework
(\url{http://tomcat.apache.org})
\item Jenkins extendable open source continuous integration server
(\url{http://jenkins-ci.org})
\item Apache Ant build scripting language
(\url{http://ant.apache.org})
\item Suse Linux Enterprise Server (SLES 11) Linux Platform (\url{https://www.suse.com/}
\item Apache Subversion source code revision control
(\url{http://subversion.apache.org})
\item Firefox browser \& Firebug (\url{http://www.mozilla.org})

\end{itemize}


\subsubsection{Taxonomic Reference System}

Quality taxonomic information is critical to support NIWA's biodiversity, biosecurity and fisheries research, as well as
the National Invertebrate Collection (NIC), New Zealand's primary museum collection of aquatic invertebrates. The
Taxonomic Reference System (TRS) provides NIWA's link to the New Zealand Organism Register (NZOR), the national
taxonomic reference database. TRS stores a subset of the NZOR content, those taxa relevant to NIWA, as an institutional
reference dataset for staff. For taxa where NIWA is the national authority, TRS will provide authoritative taxa data to
NZOR. TRS is implemented on a Postgres database.

Given the maritime and international nature of much of NIWA's biodata, the World Register of Marine Species (WoRMS,
\url{http://www.marinespecies.org/about.php}) is also a valuable reference for staff, and TRS provides direct access to
the WoRMS taxonomic hierarchy as well as NZOR, and the local NIWA version. (see Figure 9).

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img009.png}
   \caption{The NIWA Taxonomic Reference System (TRS) provides direct access to taxonomic information from NIWA's internal
taxonomic database, as well as the New Zealand Organisms Register (NZOR) and the World Register of Marine Species
(WoRMS).
 }
   %\label{fig:figure_label}
\end{figure}

\subsubsection{Taxonomic Attribute Database}

TAD, the Taxonomic Attribute Database, is a Postgres database storing taxon traits (attributes) which are associated
with a TRS taxon via the TRS taxon ID. These traits include text, photographs, dates, etc. and often comprise the
contents of various fields used for producing taxon based fact sheets or ID guides (see Figure 10). As with the Bay of
Islands web portal, images are stored in the Atlas digital asset management application, and retrieved using a web
service, the TAD record stores  the Atlas image key.

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img010.png}
   \caption{The Taxonomic Attribute Database (TAD) stores taxon trait information (such as text describing the
biosecurity risk) for taxa recorded in TRS, and has a web based interface to manage and edit the content, which is
accessible to authorised users both within NIWA and externally.
 }
   %\label{fig:figure_label}
\end{figure}

\subsubsection{Publishing and Reporting  System}

The Publishing and Reporting System (PRS) manages templates describing the layout of such documents. It uses the open
source version of Jasper Reports to create PDF or HTML documents from content stored in TRS or TAD, arranged according
to the selected template in PRS. Several documents (templates) are described in PRS, with data in TRS/TAD, including a
New Zealand freshwater pests guide, a national freshwater fish atlas and an otolith guide (used for prey identification
in fish, bird and marine mammal stomach contents, and to identify fish food species from middens in archeological
research). Users can select a single species or list of species to include in the specified report (see Figure 11).

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img011.png}
   \caption{The NIWA Publishing and Reporting System generates documents such as this fact sheet for a freshwater pest
species, with content retrieved from TRS and TAD, and the layout determined by a template.
 }
   %\label{fig:figure_label}
\end{figure}


\subsection{NIWA Environmental Observation 
 and Monitoring System}

The NIWA Environmental Observation and Monitoring System (NEMO) is an attribute/value database implemented using
Postgres. The generic database structure supporting NEMO is capable of storing and managing virtually any measurement
or observation data, and several datasets that used to be managed independently have now been migrated to NEMO. 

As NEMO has a variety of OGC web services already implemented on the underlying database, these services are
available ready for use on any dataset added to NEMO. 

To facilitate the loading of datasets into NEMO, NIWA has developed the NIWA Observations Ingestion System (NOIS) --
a flexible framework for ingesting various types of observations from data providers, through web services.

\subsection{NIWA Quantum Map}

NIWA Quantum Map (see Figure 12) is a customised version of Quantum GIS provided by NIWA
(\url{http://www.niwa.co.nz/software/quantum-map}), developed under contract by Sourcepole AG in Switzerland. Quantum
Map has some custom functionality added to simplify access to data provided by OGC web services, and some of the more
powerful Quantum GIS analytical capabilities have been removed, to provide a simpler mapping tool for users to access
data provided by OGC web services by NIWA and other agencies.

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/26-Wood/figs/a26991ED-img012.png}
   \caption{NIWA Quantum Map, showing a project with OGC WMS and WFS layers from NIWA's Ocean Survey 20/20 portal, as
well as from other datasets and OGC services. The NIWA web data tool is shown in the foreground, listing WFS layers
available from the Wellington City Council (WCC). These layers can be added to a user's ``favourites'' list, added to
the map, or downloaded and cached locally. }
   %\label{fig:figure_label}
\end{figure}

The main NIWA tool provided in Quantum Map is an enhanced OGC client. This is able to retrieve a list of WMS and WFS
service providers on the NIWA web site (\url{http://www.niwa.co.nz/ei/feeds/report}) and to open the map layers from
these sources. It is also able to cache these layers (both WMS and WFS) locally for offline use.

Given NIWA's decision to use OGC web services for environmental data delivery and discovery, Quantum Map is an important
part of the NIWA strategy for open data access. Having such a tool freely available means that by simply providing an
OGC WMS or WFS service on a dataset, NIWA is enabling access for anyone using this client application (or any other OGC
client). Developing web portals for every dataset is no longer necessary for users to discover, access and visualise
NIWA spatial data. As can be seen at the above URL, not just NIWA datasets are listed and available for use with
Quantum Map. Central and local government agencies, private businesses and other research agencies also provide map
layers via OGC web services, and are readily available for Quantum map users.

An important subset of NIWA's environmental data includes time series data, from both climate and hydrometric stations,
managed in NIWA's national climate and hydrometric databases respectively. During 2013 NIWA is developing OGC SOS
(Sensor and Observation Service) web services for these databases, and will be providing SOS clients for these in
Quantum Map. NIWA is discussing the best approach for delivering such time series data with stakeholders and interested
parties, with the likely approach being a combination of WFS for discovery and SOS for delivery, much as described at
FOSS4G2010 (Hollmann et al. 2010).

NIWA is also looking at the CSW support available for QGIS, and adapting this to enable Quantum Map to directly access
NIWA's institutional Geonetwork catalogue, and the OGC web services described there.

NIWA hopes that improved plugin support provided in future Quantum GIS will enable the various NIWA tools from Quantum
Map can also be made available as QGIS plugins, so that Quantum Map becomes simply a version of QGIS with preloaded
plugins, supporting all the platforms that QGIS does.

\section{Discussion}

Much as described in the US Department of Defence's guidelines on the use of Open Source software,  adapted below from
(Scott et al, 2011), NIWA has found several advantages in using FOSS software for open standards compliant systems
enabling data and metadata interoperability with clients, collaborators and stakeholders, including:
\begin{itemize}
\item Increased Agility/Flexibility: NIWA has full access to source code, in-house developers can work with the
developer community to adapt existing tools to better meet NIWA needs, generally within the normal development process
with no forking or bespoke systems required. Open source applications can be adapted to make use of common
infrastructural facilities in NIWA, and therefore a more efficient implementation is achieved, much like an optimised
institutional software suite. Support is available directly from the core development team, enabling rapid response
times for bug fixes and enhancements.
\item Less risk, better planning. Open source software projects generally maintain a public web site listing bugs,
problems, requested enhancements, etc, as well as the fixes. Perusing such sites provides a wealth of detail regarding
any problems, as well as the response times to solve issues etc. This sort of information is often impossible to find
for commercial software.
\item Faster delivery: Because developers only need to focus on changes to, and integration of, existing software
capabilities they can significantly reduce the time to delivery for new capabilities. 
\item Increased Innovation: With access to source code for existing capabilities, developers and contractors can focus
on innovation and the new requirements that are not yet met by the existingsource code capabilities, working within a
community of existing developers and sharing both information.
\end{itemize}

\section{Conclusion}

NIWA has had a long and successful involvement with open source software, Recent New Zealand government Open Data
initiatives support NIWA's goal to make data more accessible to clients, stakeholders and the public and other
interested parties. NIWA has made a significant commitment to delivering open data, via open standards, implemented
using robust open source applications.

NIWA's experiences using open source applications as research tools, and to provide environmental information
management, discovery and delivery capability for institutional, national and international systems have been
overwhelmingly positive. The wide support that exists for OGC standards in both the open source and proprietary GIS
spaces has meant that a high level of interoperability exists for users of tools from both camps, enabling common
frameworks despite different underlying tools. 


\paragraph{\textup{Acknowledgement}} NIWA acknowledges the support and funding provided by Land Information New Zealand through the Oceans Survey 20/20
programme and the New Zealand Geospatial Office; the Ministry for Primary Industries, the Ministry of Business, Innovation and Employment and the Terrestrial and Freshwater Biodiversity Information System (TFBIS) Programme managed by the Department of Conservation. 

NIWA recognises and appreciates the contribution OGC has made to providing standards that enable environmental data
interoperability, discovery and delivery, and the contribution OSGEO and others have made to providing quality open
source applications which enable the OGC standards to be put into practice, as well as many other geospatial
applications.  



\section*{References}


\begin{footnotesize}
%\begin{thebibliography}{99}

\begin{description}

\item[] 
AODC (2008) Marine Community Profile of ISO19115, Australian Ocean Data Centre. (last checked 2013)\url{ http://www.aodc.gov.au/files/MarineCommunityProfilev1.4.pdf}

\item[] 
GeoNetwork Opensource. GeoNetwork is a catalog application to manage spatially referenced resources.Ê \url{http://geonetwork-opensource.org/} (last checked 2013)

\item[] 
GeoServer. GeoServer is an open source software server written in Java that allows users to share and edit geospatial data.\url{ http://geoserver.org} (last checked 2013)

\item[] 
Hollmann, C., Stasch C., Jirka, S., (2010) Coupling 52\textsuperscript{o} North's Sensor Observation Service and Geoserver's Web Feature Service. Presentation at FOSS4G 2010 (last checked 2013) \url{http://2010.foss4g.org/presentations_show.php?id=3800 }

\item[] 
Jasper Reports. \url{http://www.jaspersoft.com} (last checked 2013)

\item[] 
MapServer. MapServer is an Open Source platform for publishing spatial data and interactive mapping applications to the web. \url{http://mapserver.org/} (last checked 2013)

\item[] 
Oliver, H., (2012) The Cylc Forecast Suite Metascheduler User Guide. (last checked 2013) \url{http://cylc.github.com/cylc/CylcUserGuide.pdf}

\item[] 
OpenLayers. OpenLayers is a open source JavaScript library for displaying map data in most modern web browsers. \url{http://openlayers.org/} (last checked 2013)

\item[] 
Popinet, S. (2003)   Gerris:  a tree-based adaptive solver for the incompressible Euler equations in complex geometries Journal of Computational Physics, Volume 190, Issue 2, September 2003, Pages 572-600

\item[] 
Quantum GIS. Quantum GIS (QGIS) is a user friendly Open Source Geographic Information System (GIS) licensed under the GNU General Public License. \url{http://www.qgis.org/}

\item[] 
PostgreSQL. PostgreSQL is an open source object-relational database system. \url{http://www.postgresql.org} (last checked 2013)

\item[] 
Scott, J., Wheeler, D., Lucas, M., Herz,J., (2011) Open Technology Development (OTD):Lessons Learned \& Best Practices for Military Software. \url{http://www.oss-institute.org/OTD2011/OTD-lessons-learned-military-FinalV1.pdf} (last checked 2013)

\item[] 
SilverStripe CMS. SilverStripe CMS is an open source web content management system. \url{http://www.silverstripe.org} (last checked 2013)

\item[] 
Wessel, P., Smith, W. H. F., Scharroo, R., Luis, J. F.  (2011) The Generic Mapping Tools Technical Reference and Cookbook. (last checked 2013) \url{http://gmt.soest.hawaii.edu/5/GMT_Docs.pdf}

%\end{thebibliography}



\end{description}


\end{footnotesize}



\endinput
