\begin{article}

\title{Usability Trumps Features}

\subtitle{User needs and the redesign of a web-based GIS to support community environmental monitoring}

\author{Martin J. Bunch and Michael D. MacLennon}

\maketitle

\section{Abstract}

Web-distributed tools that complement community-based environmental
monitoring (CBEM) initiatives can improve processing of and access to
information, supporting environmental education and better informing
decision-making.  To this end a web-based geographic information
system known as `Juturna' was developed to support CBEM in the
vicinity of Toronto, Canada.  This web-GIS facilitates input,
analysis, and reporting of community data. However, use of the system
steadily declined in activity since this initiative started in 2004.
Lay users reported that the system was complicated and confusing, and
so discouraged use.  Also, it employed expensive proprietary software,
which was a disincentive for the local Conservation Authority and
collaborating NGO to adopt the system.


To revitalize use of the website and provide support to the CBEM
program, we undertook to redesign the web-GIS using open source
software.  To understand why the original web-GIS was not well used
and to inform redesign of the system, we implemented a user-centered
design methodology. Methods included user testing, rapid prototyping
and stakeholder interviews. The process was invaluable in prioritizing
user tasks, defining characteristics of users of the website, and
identifying those components of the web-GIS most confounding to
them. Findings were used to inform re-development of the web-GIS
through an iterative process that led to the creation of two
prototypes that were evaluated by the user audience and so informed
the design of a new (more accessible) website.

{\bf Keywords:} web-GIS, public participation GIS, user-centered design, iterative development, community based environmental monitoring.


\section{Introduction}

Community-based environmental monitoring (CBEM) is becoming an
important resource for many environmental monitoring programs. For
Conservation Authorities these community-based programs help offset
the cost of employing trained scientific experts, and allow members of
the public to re-connect with the natural environment as
stewards. Creating online tools and resources that complement these
community-based initiatives can help improve access to information and
allow for more informed decisions to be made on issues that affect the
health of these monitored ecosystems.

Through one such CBEM initiative, the Toronto and Region Conservation
Authority (TRCA) in collaboration with Citizens' Environment Watch
(CEW) invited members of the public to participate in the collection
of water monitoring data in Toronto's watersheds. Part of this
community-based monitoring program was the implementation of a
geographic information system, accessible via the world wide web.
This system (known as `Juturna' after the Roman goddess of wells,
springs and streams) facilitates the input, analysis, and reporting of
community data. However, use of the web-GIS steadily declined in
activity since this initiative started in 2004, while at the same time
public interest in the community monitoring program remained strong.
Lay users reported that the system was complicated and confusing, and
so discouraged use.  Also, the web-GIS employed expensive proprietary
software, which was a disincentive for the conservation authority and
collaborating NGO to ultimately adopt the system.

In an effort to revitalize use of the website and provide support to
the community-based environmental monitoring program, we undertook to
understand why a seemingly excellent and functional tool was not well
used by its intended audience, and then to redesign the web-GIS using
open source software.  In this paper we briefly introduce the
community monitoring program so as to present their information and
analysis requirements, and we discuss the application of user-centered
design (UCD) methodology as a means of understanding the failure of
the initial pilot and to guide redevelopment of the web-GIS to make it
more usable to its audience. Specifically, we set out to understand:

\begin{itemize}
\item What are the business goals and the website functions most important to the stakeholders? 
\item How can the Juturna~1.0 system be improved with respect to ease of use its overall usefulness to its audience?
\end{itemize}

This paper presents a case study in which the pursuit of these
questions informed the redevelopment of a web-based public
participation GIS.  While many professionals in the field may take
some of the lessons we present for granted, we offer this case study
for those not having system design experience and/or training, in the
hopes of saving them missed opportunity, time and resources.

\section{Community-based environmental monitoring and the `Juturna' web-GIS}

Environmental monitoring and the gathering of environmental
information are important exercises for understanding the impact of
human stressors on the environment. In Ontario, Canada such activities
are generally carried out by Conservation Authorities,
quasi-governmental organizations that act as stewards to ensure that
environmentally sensitive areas remain intact and undisturbed by these
stresses. However, reductions in funds for environmental monitoring
activities that are allocated by provincial and federal governments
has led to increasing examples of local communities partnering with
conservation groups so as to continue monitoring ecologically
sensitive areas \citep{mayfield:2001} and as a means of
empowering and educating members of the public.  The regional
watershed monitoring program (RWMP) conceived by the Toronto and
Region Conservation Authority (TRCA) is one such initiative that
implements a collaborative approach to monitoring Toronto's
watersheds. Part of this approach was the development of a
community-based water monitoring program that allowed members of the
community to take part in monitoring the health of the streams and
rivers in the Toronto region (Figure \ref{fig:martin}).

\begin{figure}[htbp] % 1
\begin{center}
\includegraphics[width=\textwidth]{peer_review/bunch/images/figure1}
\caption{One of the authors (Martin Bunch, right) participating as a volunteer monitor to collect benthic macroinvertebrate samples on a transect in Black Creek near York University, Toronto.}
%Figure 1: 
\label{fig:martin}
\end{center}
\end{figure}

As part of the RWMP the Conservation Authority facilitated annual
workshops to train members of the public as volunteers for gathering
biological and abiotic data from streams, and to identify the benthic
macroinvertebrate (BMI) organisms (mostly bugs and worms but also
potentially clams, snails and crayfish) captured in their samples. The
purpose of these workshops was not only to educate members of the
public about water monitoring and the watershed ecosystem but also to
ensure that data is collected as accurately as possibly. TRCA, in
cooperation with the environmental NGO Citizen's Environment Watch
developed monitoring protocols that use benthic macro-invertebrates
and biotic indicators to measure aquatic health. CEW committed to
provide education, equipment, and support for community-based
ecological monitoring work, to teach annual workshops, and to operate
the volunteer monitoring program.

Community volunteers who participated in the program generated data
that reported the frequency of various organisms in their samples,
geomorphic and vegetative characteristics of monitoring stations, and
the station location.  This information can be used to construct
metrics that indicate whether a stream's health is unimpaired,
potentially impaired or impaired, based on the frequency distribution
of the organisms collected and their known sensitivities to stressors
\citep{barbour:1999,castela:2008,hilsenhoff:1988}.  TRCA staff involved in BMI monitoring
would also generate water chemistry data.  While TRCA scientists are
practiced at managing and analysing this data, the lay volunteers
participating in the monitoring program generally are not.

To address this, TRCA enlisted researchers (our predecessors) at York
University's Faculty of Environmental Studies to develop a system to
assist in the uploading, storing, reporting and analysis of volunteer
data, as well as data generated by TRCA staff. This resulted in the
creation of the web-GIS called `Juturna.' The website itself also
served other purposes for different individuals with a variety of
interests in environmental monitoring. For instance, the general
public could use the website to access TRCA's aquatic monitoring data
to perform rapid analysis of watershed and subwatershed data.

In October 2004 the pilot project was implemented.  It was one of the
first community-based environmental monitoring projects to utilize the
internet and rely on a complex web-based Geographic Information System
for interacting with the water monitoring data \citep{borisko:2004}. The
web-GIS allowed users to find the location of their monitoring station
using postal code searches, aerial photos, street networks, surface
water layers, land use, and points of interest.  They could then
create a point location with which to associate the monitoring station
data, upload data and generate reports that included calculation of
Benthic indices (indicating stream health).

The pilot project was met with great interest throughout Toronto's
environmental conservation community and the initiative was presented
at several conferences. The annual water monitoring workshops that
were used to train new volunteer water monitors were always filled to
capacity.  Coming into this project (Martin Bunch inherited the
project as a faculty member, and Michael MacLennan was a graduate
student at the time), we were very impressed with the system.  It was
highly functional, allowing users to upload everything from their BMI
data to digital photos of their monitoring stations (which were
incorporated into the station reports) (Figure \ref{fig:typical}).  It produced
extremely descriptive and well organized reports at the monitoring
station, subwatershed and watershed level, it had an attractive web
interface, sophisticated administrative tools, and most of all it
worked consistently and well.

\begin{figure*}[htbp] % 2
\begin{center}
\includegraphics[width=0.75\textwidth]{peer_review/bunch/images/figure2}
\caption{ A typical water monitoring station web page in Juturna~1.0, including 1) digital photos of the monitoring station, 2) site description, 3) health indicators and 4) a report generation panel.}
\label{fig:typical}
\end{center}
\end{figure*}

However, we were surprised to discover that there was a steady decline
in the number of visitors to the website, despite the ongoing interest
in the water monitoring workshops by members of the public. More
disturbing was that in the first monitoring season (2005) supported by
Juturna, data for only 13 volunteer monitoring stations were uploaded -- less
than half of what was expected.  This suggested that Juturna~1.0
was not effectively supporting the needs of volunteer users.
Furthermore, informal feedback suggested that users may have become
frustrated with the data upload process and abandoned the process
before completing it.

As we looked into modifying the system to deal with this, two other
issues arose.  First, software needed to be updated and data made
current.  Our University had licences for the proprietary software to
support the update (ArcIMS, ArcSDE Crystal Reports, MS Windows Server)
and also access to the needed data (digital air photos, vector layers
for roads, surface waters, watersheds, parks and other features)
except for proprietary postal code data. However this meant that the
program was dependant on the University for software maintenance and
support and on the University and/or the TRCA for data updates.
Purchasing both the software and data licensing was not feasible for
an NGO like CEW whose annual budget for this project was quite small.
Second, the back-end systems of this extremely complicated application
indicated that maintenance was going to be difficult.  There was only
very sparse documentation on system architecture and design, and the
code was not commented.  It was obvious that the design of the
original system did not consider maintenance or ongoing development
(URLs, for example, were hard coded in the system).

In the winter of 2005--2006, it became clear that the website needed to
be redeveloped to alleviate these problems.  In early spring of 2006,
financial support was provided by the Ontario Ministry of Environment
to redevelop the website using Free and Open-Source (FOSS) software.
The primary open source applications we used were MapServer,
PostgreSQL/PostGIS on a Linux platform with SQL, Ruby, HTML, and Java
components. The open source approach provided us with the opportunity
to develop the site in a user-centered design paradigm with the users
at forefront of design decisions.

\section{User-Centered Design and Iterative Development }

Good design of any product (Internet-based, computer based or
otherwise) is most successful if the design group understands as much
as they can about the users' intended use of the product
\citep{brinck:2002,kuniavsky:2003,lazar:2005}. User-centered
design (UCD) methodology takes this to heart.  It is an approach to
software development and web design that allows for a deeper
understanding of the user.  Having observed that there were serious
usability problems with Juturna~1.0, we undertook to implement a
user-centered design methodology in the re-development process of the
Juturna website.

The UCD procedure consisted of a series of development stages that
incorporated information gathered from the implementation of several
usability methods.  \cite{kuniavsky:2003} defines this kind of approach as
an iterative development framework. The idea of iterative development
(ID) is founded on the premise of continual refinement through a trial
and error process whereby each successive iteration incorporates what
is learned from the applied usability methods. The end goal of the
project being a product (the Juturna~2.0 web-GIS) that is as usable as
possible, meeting the needs of its intended audience.

Usability can be measured on three factors; functionality,
desirability and efficiency \citep{kuniavsky:2003}. Determining how these
are achieved depends on how well the development team understands its
user audience. In order to do this \cite{brinck:2002} suggest
adopting a `pervasive usability' approach within an iterative design
process. The purpose of this approach is to employ a methodology in
which user input is incorporated in every step of the development
process. For example, when a change is made to the design of the
interface it is tested by users to determine if this change was
effective in achieving a usability benchmark. This allows for
different usability methods to be implemented at each stage of the
development process (Figure \ref{fig:pervasive}). In this project the majority of the
usability tools were implemented in the first three phases of the
project. These tools include stakeholder interviews, use cases, user
flows, and user testing.

\vspace{0.15in}

 
\begin{figure}[htbp] % 3
\begin{center}
\includegraphics[width=\textwidth]{peer_review/bunch/images/figure3}
\caption{Pervasive Usability process (after \cite{brinck:2002}).}
\label{fig:pervasive}
\end{center}
\end{figure}

\section{Methods and the development process}

Following the process outlined by \cite{brinck:2002} and \cite{kuniavsky:2003}
our design process was implemented in five stages: requirements
analysis, conceptual design, mockups and prototypes, production and
launch.  These stages were operated by the application of the
techniques presented in Table \ref{table:ucdtechniques}.  The process
was used both to evaluate the Juturna~1.0 system (to understand its
failure), as well as to inform redevelopment of the new open source
system.

\subsection{Stage One: Requirements Analysis}

The first stage in the redevelopment of Juturna identified the goals
and parameters of the project. Using stakeholder interviews, a
baseline evaluation of the existing website in terms of its current
functionality as well as its usability was performed.  This was
followed by stakeholder interviews to explore the purpose of the site
from a business perspective, understand the stakeholder organizations'
perspectives of the users, and to set the context for the website's
future direction.  These interviews contributed both to evaluation of
the existing Juturna~1.0 system and to determining the target
audience, user goals, business goals and technical requirements for
the redeveloped website. At this stage user needs and target usability
goals were determined as a means of guiding the following stages.
This process identified five user groups: volunteers, TRCA staff,
system administrators, consultants and partners, and the lay public.

\begin{table*}
\begin{center}
\begin{tabular}{|p{0.15\textwidth}|p{0.85\textwidth}|}
\hline
{\bf Technique} & {\bf Description} \\
\hline
Semi-structured interviews
&
Semi-structured Interviews (SSI) occurred over a one hour period in a
comfortable setting chosen by the interviewee.  A limited set of
open-ended questions were used to direct a discussion.  More specific
questions were asked depending on the responses of the interviewee.
Themes explored in the SSI included, for example, history and context
of the organization with respect to water monitoring, and the efficacy
of the Juturna~1.0 system.
\\
\hline
Use cases
&
User scenarios were developed for each of the identified user groups.
Use Case Scenarios are narratives about how a particular type of user
of the system would interact with it.  These narratives provide some
context and ``feel'' for how the system needs to work.  From these
scenarios a functional requirements list and user flows may be
constructed.
\\\hline
Task analysis
&
Tasks are expressed as simple flow charts (``user flows'') that describe
the possible flow of interaction of particular category of user with
the website.  These assist in developing the information architecture
of the system, that is, the design and organization of the navigation
structure and consideration of position and placement priority of page
components like the log-in form, web-GIS functions, benthic data entry
forms and data access rules.
\\\hline
Wireframe mockups
&
Wireframe mockups are rapid prototype application interfaces.  This is
a block diagram that illustrates the overall navigation of a website
and the blocks of elements (such as content and functionality) that
will be present on the screen.
\\
\hline
Heuristic evaluation
&
Heuristic evaluation systematically assesses a user interface design
for usability in form and function based on a set of rules.  In this
case, rules were derived from Nielsen's 10 usability heuristics:
visibility of system status; match between system and the real world;
user control and freedom; consistency and standards; error prevention;
recognition rather than recall; flexibility and efficiency of use;
aesthetic and minimalist design; help users recognize, diagnose, and
recover from errors; help and documentation \citep{nielsen:2005}.
\\
\hline
User evaluation
&
For the original Juturna~1.0, for prototypes, and for the re-designed
open source system, user testing involving various user groups was
undertaken. Five evaluators were used to satisfy the minimum number to
catch eighty percent of the usability problems, which is between four
and five (\citep{nielsen:1994-2006};\citep{rubin:1994}. Six tasks exploring four main functionalities were evaluated
by participants. These were 60 minute interviews in which evaluators
undertook tasks on the system, while the moderator took notes and
recorded quantitative metrics about speed of completion and ease of
use.  Users, sitting at a computer accessing Juturna, responded to
questions such as ``You would like to generate a report of the Humber
watershed for the 2001 year.  How would you generate this report?'' by
undertaking the task on the system, and vocalizing their thoughts as
they did so.
\\
\hline
\end{tabular}
\caption{Primary techniques employed in user-centered design of Juturna~2.0}
\label{table:ucdtechniques}
\end{center}
\end{table*}

\subsection{Stage Two: Conceptual design }

Analysis and evaluation of the Juturna~1.0 site against data derived
from stakeholder interviews was then used to define a concept for the
architecture of the initial design of Juturna~2.0. This analysis
allowed us to produce flowcharts that summarized how the major user
tasks were to be completed in the new system (e.g., Figure \ref{fig:userflow}). These
documents provided a means of sketching the website's organization in
a more formal technical document known as a `use case' that defined
user scenarios.  These scenarios were developed based on interviews
with the major stakeholders and our previous experience with the
Juturna~1.0 website.

\begin{figure*}[htbp] % 4
\begin{center}
\includegraphics[width=0.75\textwidth]{peer_review/bunch/images/figure4}
\caption{`User flow' describing possible flows of interaction of the administrator user with the website. }
\label{fig:userflow}
\end{center}
\end{figure*}

\subsection{Stage Three: Mockups and Prototypes}

An interim representation of the new website was implemented using
mockups and prototypes. This allowed for early evaluation of the new
design before the final system was produced. Rapidly producing a
prototype of the final website using simple diagrams allows for
changes to be implemented easily and quickly, preventing extensive
modifications from occurring later in the development process. Figure
\ref{fig:wireframe} presents a wireframe prototype used for this purpose.  The
wireframes lacked content but had operational navigational structure.
User testing of these mockups, and consultation with the stakeholders,
identified changes to be implemented in the production stage.


\begin{figure}[htbp] % 5
\begin{center}
\includegraphics[width=\textwidth]{peer_review/bunch/images/figure5}
\caption{Wireframe prototype of the administrator console.}
\label{fig:wireframe}
\end{center}
\end{figure}

\subsection{Stage Four: Production}

At the production stage the product was created. Content was added to
the website, graphic elements were developed, and coding was produced
for the components of the website. Evaluation at this stage occurred
in the form of heuristic evaluation to assess the level of usability
in comparison to Juturna~1.0. This was a formal comparison of the
website with usability rules, so as to predict usability of the system
\citep{hvannberg:2007,nielsen:1994}).  User
testing of the website also occurred to ensure that the website
functioned properly.

\subsection{Stage Five: Launch}

At this stage the product was finally launched and made available to
the public. Prior to making the website available, a quality assurance
test was performed to ensure that all necessary website functions were
working properly.  This was not an endpoint as the system continues to
be refined as further development phases are implemented, new
functionality is added, and user needs change.

\section{Assessing Juturna~1.0 and building Juturna~2.0}

User testing occurred using the original Juturna~1.0 system (so as to
provide insight into its failure and to inform development of the new
open source version of Juturna), on a wireframe mockup of the new
system, and on a redeveloped version of the system.  Primary functions
evaluated included: data upload, data download, report creation, user
account management, and data assessment functions. These functions
were chosen because they were determined through the UCD process as
important to the website for all user groups.  User groups identified
were: Volunteer users (community monitors), consultant users or
partners (who may need to access raw data in the system),
Administrator users, TRCA users (staff), and Lay users (the general
public).  Volunteer users and TRCA users were the primary users of the
site, and their activity encompassed that of the others.  Thus, user
testing was undertaken on the basis of these two groups.  This process
led to four main areas of improvement in the system relating to: task
efficiency, topology of navigation structure, reducing users' memory
load and bullet proofing against user error.

\subsection{Improving task efficiency}

We were able to improve the efficiency of several tasks in the
redevelopment of the website.  For example, the action of logging into
the website is one of the most frequently performed activities by
all but the lay user group.  Juturna~1.0 placed the log in function
several pages deep within the sub-navigation structure of the site,
instead of in an apparent location such as the monitoring sites
landing page.  In redeveloping the site we placed the log in function
on the home page to allow users to easily locate the log in console
(similar in location to several popular websites like google.com,
gmail.com, msn.com and hotmail.com).  This simple modification reduced
the total time spent completing these tasks and also saves the user
from the frustration of having to search the website for the log in
page.

Figure \ref{fig:clickevents} illustrates this point in relation to the number of mouse
clicks required by the user to log into the restricted portion of the
website. The original website required the user to click four
different links on four separate web pages before he or she can start
uploading data. The redeveloped website reduced this to one mouse
click by placing the log in function on the home page of the website,
and also placed data uploading functionality as one of the first tasks
a user's log in screen.

\begin{figure}[htbp] % 6
\begin{center}
\includegraphics[width=\textwidth]{peer_review/bunch/images/figure6}
\caption{Illustration of click events needed to log in.}
\label{fig:clickevents}
\end{center}
\end{figure}

The purpose of this approach was to improve the speed at which a user
could start a task by reducing the cognitive load required to remember
which links to click to get to the data upload screen. Quantitative
metrics (Table \ref{table:taskoneqm}) as well as user vocalization
during user-testing interviews was useful in elucidating issues with
task efficiency.

\begin{table*}
\begin{center}
\begin{tabular}{|l|r|r|r|r|r|r|}
\hline
\multicolumn{7}{|l|}{\bf Task One: Report Generation}
\\
\multicolumn{7}{|l|}{ }
\\
\hline
 &Craig &Derrick&Samantha&Tanya&Chris & Average\\
\hline
\bf Time to Complete&
2&
1&
2&
3&
1&
1.8\\
\hline
\bf Errors&
1&
2&
2&
3&
1&
1.8\\
\hline
\bf Key &
\multicolumn{3}{|p{0.3\textwidth}|}{
{\bf Time to Complete}
\begin{enumerate}
\setcounter{enumi}{-1}
\item{Fail}
\item{Succeed very slowly in a roundabout way}
\item{Succeed a little slowly}
\item{Succeed quickly}
\end{enumerate}
}&
\multicolumn{3}{|p{0.3\textwidth}|}{
{\bf Number of Errors}
\begin{enumerate}
\setcounter{enumi}{-1}
\item Fail because of errors
\item Many errors
\item Some errors
\item Few or no errors
\end{enumerate}
}\\
\hline
\end{tabular}

\caption{Task One Quantitative Metrics (names are pseudonyms)}
\label{table:taskoneqm}
\end{center}
\end{table*}


\subsection{Topology of navigation structure}

Initial evaluation of the Juturna~1.0 navigation structure found that
the navigation was largely inconsistent and arbitrarily defined across
the website. For instance, the home page of Juturna~1.0 presented the
primary navigation on the left side of the page while all other pages
moved the navigation structure to the top of the page. This slight
change presents an inconsistent design that has the potential to be
highly disorienting to users, giving them the idea that they have
moved to another website. To further confuse the design, common names
associated with primary navigation on the home page also changed when
the user navigated to other pages on the site.  For example, the
primary navigation on the home page consisted of several links labeled
``Administrator Area (secure)'', ``Community Monitoring'' and ``Report
Generation Area (secure)'' while the primary navigation menu found on
all other pages labeled these same links as simply ``Admin'',
``Community'' and ``Report Generator.''

Additionally the web pages of Juturna~1.0 tended to be organized in an
ordered sequence with certain pages enabled with a sequentially meshed
navigation structure. Figure \ref{fig:navigation1} shows the topological structure of
Juturna~1.0.  Use of an ordered sequence or linear navigation
structure works well when users must complete a series of tasks in a
specific way (e.g., where a user must upload data in a particular
fashion). However, this kind of linear approach is problematic when
implemented over an entire website because it prevents the user from
exploring a site in a free and open manner, which allows them to
decide the order in which information is consumed.  The meshed
navigation structure implemented in Juturna~2.0 (Figure \ref{fig:navigation2}) alleviates
this problem by allowing users to navigate to any and every page on
the website while still allowing for some areas of the site to adhere
to a sequential structure. The downside to this approach is that users
have the ability to step in and out of different stages of the linear
structure which may break the data uploading process. It is best to
monitor for this and communicate this error if a user deviates from
the sequential structure.

\begin{figure}[htbp] % 7
\begin{center}
\includegraphics[width=0.8\textwidth]{peer_review/bunch/images/figure7}
\caption{Navigation topology used in Juturna~1.0.}
\label{fig:navigation1}
\end{center}
\end{figure}

\begin{figure}[htbp] % 8
\begin{center}
\includegraphics[width=0.8\textwidth]{peer_review/bunch/images/figure8}
\caption{Navigation topology used in Juturna~2.0.}
\label{fig:navigation2}
\end{center}
\end{figure}

Instead of maintaining a navigation structure that implements both
sequential and meshed architectures, we designed the first prototype
with a hierarchical and meshed topology that used navigation titles
that are task-based. As illustrated in Figure \ref{fig:navigation2}, this approach
organizes the website around the principle of tasks that a user is
able to accomplish on the site allowing them to easily orient
themselves and get to the task they want to accomplish. For example,
the navigational labels for a user account having permissions to
upload data and create reports were changed to `create test site' and
`generate report' instead of the more general and ambiguous title of
`community monitoring.' Designing the website in this manner created
an architecture that will be familiar to the user for two
reasons. First, the hierarchical structure is the most commonly used
navigation structure on the Internet and is therefore well understood
by the user audience. Second, because its organization is task based,
there is a greater potential that it appears more intuitive to the
user audience because it fits the user's understanding of the purpose
of the site.

\subsection{Reducing users' memory load}

Efforts were also made to minimize the user's memory load by making
options and links more visible. \cite{brinck:2002} argue that the
user should not have to remember information from one part of the
website to another. Instructions for use of the system should also be
visible and easily retrievable whenever appropriate.  Juturna~1.0 does
not minimize the user's memory load because it makes use of a
collapsible navigation structure. The system should also always keep
users informed about its state. For example, when a user clicks an
element on screen, the response from the system should be fast and
clearly indicate the navigation path chosen by the user, e.g., by
representing visited links by a color change. These are some of the
ways websites can inform the user of changes to its state as they
interact with the system. Juturna~1.0 did not keep the user informed
in an effective manner. Instead it forced users to explore the website
through a trial and error process and did not indicate what pages the
user has visited. This is particularly relevant in the navigation menu
where users must `discover' what links are available by expanding the
hidden menus. Figure \ref{fig:hiddennav} shows the Juturna~1.0 navigation menu of the
monitoring link as it appeared when the user dragged their mouse over
the link. This design reveals a highly complex secondary navigation
structure that requires the user to remember the location of
sub-navigation items that are always hidden.

\begin{figure}[htbp] % 9
\begin{center}
\includegraphics[width=\textwidth]{peer_review/bunch/images/figure9}
\caption{Hidden navigation menu of Juturna~1.0's navigation pane.}
\label{fig:hiddennav}
\end{center}
\end{figure}

Juturna~2.0's interface is more consistent in keeping the user
informed and orienting them to the purpose of the website. The
navigation menu does not contain any hidden links and there are lots
of visual signs in the form of page titles that show the user where
they are in the web site.  Figure \ref{fig:visual} illustrates the title of a web
page in Juturna~2.0. Having the title correspond to a link in the
navigation bar at left shows the user their location in the navigation
structure of the website at all times. This approach reduces the load
on the user's memory by utilizing location cues.

\begin{figure}[htbp] % 10
\begin{center}
\includegraphics[width=\textwidth]{peer_review/bunch/images/figure10}
\caption{Visual indicators in Juturna~2.0 help users know where they are at all times.}
\label{fig:visual}
\end{center}
\end{figure}

\subsection{Bullet proofing}

It was also determined through the analysis of the Juturna~1.0 system
that if improvements were made to the system's ability to prevent and
catch user errors the website would be significantly improved. Juturna~1.0,
contained many flaws in its ability to catch and prevent user
errors in several key functions including data downloading, data
uploading and report generation. The inability of the system to catch
these errors greatly limited the ability of users to complete tasks.
This is particularly apparent in the data downloading
function. Initially this function required the user to select several
criteria including the geographic extent of their data, the sampling
season, the sampling year and the data type, prior to downloading
data. However, there was no information present in the selection
parameters to let the user know whether they were requesting data that
did or did not exist in the database. As a result, the user was faced
with the responsibility of discovering what data was present through a
trial and error process. The initial redevelopment prototype attempted
to alleviate this problem by using a series of selection menus that
only listed the parameters present in the database.

\section{Conclusions}

Because we were a small team (a professor drawing upon graduate
student labour, and contracting out some programming work) with a
small amount of funds, the process described above did not demonstrate
the rapid response of agile design.  However, it did yield responsive
development -- the new Juturna system is leaps and bounds ahead of the
earlier system in terms of usability.  Paradoxically, it is also much
less impressive.  Juturna~2.0 has fewer functions, and is less
visually powerful than its predecessor. This is because many of the
functions incorporated in the first version were not essential to its
mission.  Some of those functions and capabilities may return in
future iterations of development, but only if users indicate a strong
need for them and if they can be implemented without requirements for
proprietary software or data.

Currently, Juturna~2.0 resides at \verb|www.juturna.ca| and is a functional
web-GIS supporting community volunteer monitoring organized by
Citizens' Environment Watch.  After the first redevelopment phase of
the system was completed in late 2007 there was a delay in
implementing Juturna (though it was online and functional) due to a
reorganization of the monitoring program by CEW and changing in-house
needs at TRCA due collaborative data management initiatives with other
CAs.  This necessitated another iteration of the design process in
2008.  The system is now being populated with volunteer data from the
2009 season and will support a new orientation of the CEW program to
operate with schools in the Toronto region in 2010.  Future plans for
Juturna target interoperability, as well as modifications to make the
system easily adaptable to other watersheds outside of the TRCA
jurisdiction.  Once this is happens we intend to make the system
available in a free and open-source manner.

\section{Acknowledgements}

The authors wish to acknowledge the contributions and collaboration of
our partners at the TRCA and Citizens' Environment Watch, as well as
Anna Goldberg and Joseph Zhao at Malone Given Parsons Limited for
programming work, and Matt Milan for advice in the redevelopment of
Juturna~2.0.  This project has received funding support from the
Ontario Ministry of the Environment. Such support does not indicate
endorsement by the Ministry of the contents of this material.

\address{
 Martin J. Bunch
\\Faculty of Environmental Studies
\\York University
\\4700 Keele Street
\\Toronto ON Canada
\\M3J 1P3
\\ ~
\\Tel: 416-736-2100 ext: 22630
\\Fax: 416-736-5679
\\Email: bunchmj@yorku.ca
}

\bibliographystyle{plainnat}
{\bibliography{peer_review/bunch/usabilitytrumpsfeatures}}

\end{article}

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "master"
%%% End: 

