% Template For OSGeo Journal
% Language: Latex
% See article_long.tex for more examples, including use of urls, code examples, etc.

% Head

\title{Gestural Interaction with Spatiotemporal Linked Open Data}
\subtitle{}
\author{by Thomas Bartoschek$^{1}$, Gerald Pape$^{1}$, Christian Kray$^{1}$, Jim Jones$^{1}$ and Tomi Kauppinen$^{2}$

\medskip
\noindent{1: University of M\"unster, Germany. 2: Aalto University, Finland \email{bartoschek@uni-muenster.de}}}

\maketitle

\subsection*{Abstract}

Exploring complex spatiotemporal data can be very challenging for non-experts. Recently, gestural
interaction has emerged as a promising option, which has been successfully applied to various domains, including
simple map control. In this paper, we investigate whether gestures can be used to enable non-experts to explore and
understand complex spatiotemporal phenomena. In this case study we made use of large amounts of Linked Open Data about
the deforestation of the Brazilian Amazon Rainforest and related ecological, economical and social factors. The results
of our study indicate that people of all ages can easily learn gestures and successfully use them to explore the
visualized and aggregated spatiotemporal data about the Brazilian Amazon Rainforest.

\textbf{Keywords: }
Gestural interaction, spatiotemporal phenomena, Linked Open Data.


\section{1 Introduction }

In recent years, gestural interaction has been successfully used to enable untrained users to control
different types of applications. Some operations arguably map naturally to certain types of gestures. Examples of
this are pointing at an item to select it or moving a body part to trigger a similar motion of the screen content.
These properties of gesture control are typically used to facilitate access to simple data such as photographs, media
or basic maps. Given this, our hypothesis is that gestural interaction is equally well suited to enable the exploration
of more complex (linked) spatiotemporal data. The goal is thus to create methods for communicating results of the
field of Geographic Information Science to the public, for example in an exhibition, in a science museum or
science center, where large surfaces are important for better visibility. 


Gesture control is a promising approach to enable interaction with large surfaces in particular since
it can improve ease of learning and help overcome reachability issues [15] and due to its high level of learnability
[7, 23]. While previous work has shown this for basic spatial operations (panning, zooming, re-arranging objects), it
is not clear whether gestural control can work equally well when interacting with spatiotemporal data. 

Interfacing research on Human-Computer Interaction with recent Linked Open Data and visualisation
techniques for complex spatiotemporal data is a contribution towards Linked Open Science [14] to support transparency
and openness of science via facilitating the exploration of scientific observations. Clearly, Linked Open Science
needs Linked Open Data (LOD)\footnote{\url{http://linkeddata.org}} to allow for publishing of very
different kinds of data on the web, and to interconnect them together and to space and time. Additionally Linked
Open Science needs Open Source Software, to make its results reproducable and freely available. 

In this paper we present and evaluate an application based on open source technologies enabling the
exploration of Linked Spatiotemporal Data integrated into an exhibit. We present a set of gestures that visitors to a
science fair can use to explore large amounts of linked data related to deforestation on large screens and we also
report on results from an initial survey. In the following, we first briefly describe the background before
presenting the prototype system, the linked open spatiotemporal datasets [13], their visualizations and the gestures
we have implemented. We then describe the survey we conducted and summarise the results we obtained. The penultimate
section discusses possible implications of our findings on the design of similar systems. The paper closes by
summarising the main contributions and giving a brief outlook on future work. 

\section{2 Related Work }

There is a broad range of previous work investigating the use of different types of gestures (e.g. arm-, body-, or headgestures) to facilitate interaction with various types of systems (e. g. desktop computers,
mobile phones or public displays). The recent introduction of affordable off-the-shelf solutions to track
gestures [16] led to the development of gestural interfaces for a variety of application scenarios. Gestures have thus
been used successfully as a means to enable laypeople to control different types of applications (e. g. simple map exploration, motion games, remote control of TVs, or art installations [1, 2, 21]).
Bhuiyan [1] provides a chronologically ordered overview of gesture-controlled user interfaces. Wachs [21] surveys
possible application scenarios for gestural interaction but also compiles a set of basic requirements for gesture-based
systems. Technically, the cited systems either make use of multi-touch surfaces [8], near-or on-body sensors [4], or
contactfree/camera-based technologies [3]. 

Frequently, a skeletal model is used to further improve recognition [12]. Since there now are several
viable options to detect gestures fairly reliably, the design and use of specific gestures has recently become an
important area of research. Wobbrock [22] presented initial research into what gestures people produce naturally and
identified a set of common gestures for use on multi-touch surfaces. Jokisch carried out a survey on how users
discover gestures for map interaction on large multi-touch surfaces [10]. In his study, adults (20-30 years old) easily
discovered simple gestures (i.e. pinch-to-zoom) but had great difficulties finding complex ones, while
children (8-9 years old) had more difficulties discovering both. 

A frequent example application scenario for gestural interaction is the exploration of
two-dimensional maps or simple spatial data (e. g. [6, 17]. Map interaction was also a frequent example scenario to
demonstrate FTIR-based surfaces [8]. Daiber [4] introduced a set of physical multi-touch interaction primitives and let
user groups choose corresponding spatial interaction tasks. Recently, the manipulation of a virtual globe has also
received some attention [4, 11, 19]. Boulos [11] introduced a system for gestural interaction with Google Earth and
Google Street View by describing gestures for the basic functionalities. Stannus [19] reported on a comparison study,
where they asked participants to perform different common navigation tasks with a virtual globe using either a
mouse, a 3D-mouse or gesture control. They found that users often prefer classical input methods, i. e. a standard
mouse for finding places in Google Earth, only displaying the standard imagery and no complex data. The gestural
interaction was rated and commented positively in terms of naturalness. The study was of a qualitative nature, having
only 10 paricipants and reported about technical problems in the prototype for the gestural interaction. 


The work presented in this paper also investigates gestural interaction with a virtual globe but
extends previous work by enabling interaction with the temporal dimension (to explore historical data and trends). In
contrast to previous work, the prototypical application we used relied on linked spatiotemporal data on deforestation
in South America. To the best of our knowledge our effort to provide exploring Linked Data with gestural
interaction methods is first of its kind. 

\section{3 Exploring Data about the Brazilian Amazon Rainforest }

In order to investigate the suitability of gestural interaction for the exploration of linked
spatiotemporal data, we designed an interactive system aimed at enabling untrained users to explore linked data sets
and visualizations on the deforestation of the Amazon region in South America. The system was exhibited at a science
fair, where we gathered feedback from visitors. The physical setup of our system was based on a triangular construction
with three rear projection surfaces (see Figure 1 and Figure 2). 

\medskip
% The following is used for inserting an image
\begin{figure}[tpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img001.png}
   \caption{Physical setup of prototype: three back-projection screens arranged in a triangular
configuration; each screen is controlled by a Kinect motion tracking device.}
   %\label{fig:figure_label}
\end{figure}


\medskip
% The following is used for inserting an image
\begin{figure}[tpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img002.png}
   \caption{Gestural Interaction with the prototype on two sides providing access to different
facets of the complex data se.}
   %\label{fig:figure_label}
\end{figure}


Each of the three screens was controlled by a
dedicated gesture tracking device (i.e. a Kinect sensor). The data was rendered on the screen using a modified
version of the free open source system NASA World Wind Java SDK\footnote{\url{http://worldwind.arc.nasa.gov/java}} (a virtual globe system). Gestural interaction was implemented via the open source API for natural
interaction OpenNI\footnote{\url{http://www.openni.org}}, a computer vision middleware NITE\footnote{\url{http://www.primesense.com/solutions/nite-middleware}} and time series analysis [20]. 


The Linked Data in use represents deforestation observations from the Brazil Amazon Rainforest.
Moreover, there are related social, economical and ecological data from selected Open Data sources. The social,
economical and ecological dimensions were selected following the Triangle of Sustainability concept [18]. The software
supports several operations to explore the data: panning---i.e. moving the map---, zooming in and zooming out, as well
as moving forwards and backwards through time. The temporal operation is particularly important to support
understanding of the effects and spread of the deforestation over time, and correlated phenomena like increased
GDP and increased population. 
    
The approach is completely based on free and open source software technologies and open data, making
the exploration of the Brazilian Amazon Rainforest deforestation available to a broad audience in science, education,
politics and society. In the next sections we introduce the open datasets and describe their visualization in NASA
World Wind and the user interface with its gestural interaction. 

\subsection{3.1 Linked Brazilian Amazon Rainforest Data }

Linked Open Data (LOD) is a set of standards and practices for exposing data on the web. LOD also
allows to explicitly store the various connections in the data. We made use of our earlier contribution to open up and
link the data [13] about the Brazilian Amazon Rainforest for the visualization. These statistical and spatial datasets
were described and structured using mainly two vocabularies, namely the Open Time and Space Core Vocabulary
(TISC)\footnote{\url{http://observedchange.com/tisc/ns}} and the Open Linked Amazon Vocabulary
(OLA)\footnote{\url{http://observedchange.com/amazon/ns}} . The entire dataset originates from official sources
of the Brazilian Government, namely from IBGE\footnote{\url{http://www.ibge.gov.br/english/}} (Brazilian
Institute of Geography and Statistics) and INPE\footnote{\url{http://www.inpe.br/ingles/}} (Brazilian
National Institute for Space Research). 

Due the recently announced policy of the Brazilian government to publish all their statistical data
on the web\footnote{\url{http://acessoainformacao.ibge.gov.br/en/}} it was possible to retrieve this data and integrate it into the existing Linked Data about the Brazilian Amazon Rainforest. We realized the
retrieval via a custom-made application, which connected to the IBGE's servers, downloaded the relevant data and
transformed it into Linked Data to be used for the visualizations and exploration.

\subsubsection{3.2.1 Statistical Dataset }

The statistical dataset, containing a time-series from 2004 to 2009, includes the following
information: 

\begin{description}

\item[Soybeans ] -- Soybean crops classified by hectares of planted and harvested
area, value in thousand of reais (Brazilian Currency), metric tons and kg hectare. Those crops are linked to an
observation and additionally linked to their respectively entries in DBPedia. 

\item[GDP] -- Gross Domestic Product from each municipality of Par\'a state. 

\item[Cattle] -- The total head of cattle grouped by municipality. 

\item[Population per municipality]  -- Census 2000 and 2010, and population projections
from 2001 to 2009. 

\end{description}

\subsubsection{3.2.2 Spatio-temporal Dataset }

Observations of deforestation were made using remote sensing techniques, i.e. satellites. Similarly, land use data and natural and social factors of change were collected. All
these data, structured and provided by INPE[5], were aggregated to grid cells of 25 km x 25 km. In Figure 3, a
visualization of the dataset can be seen, where the deforestation values from cells, which spatially overlap the
municipalities of Par\'a in Brazil, were aggregated and were plotted together with the corresponding
population on the virtual globe. The spatial dataset, also containing a time-series from 2004 to 2009, has the
following information: 

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img003.png}
   \caption{Visualization of the LOD Datasets on Population (height) and Deforestation (color) in NASA
World Wind.}
   %\label{fig:figure_label}
\end{figure}



\begin{description}

\item[Deforestation] -- The observed amount of deforestation aggregated to grid cells. 

\item[Municipalities and Federal States] -- A complete compound of Brazilian Federal
States, which belong to the Amazon area (according to the legal definition) and their municipalities, together
with their geographical location and covered area. 

\item[Mesoregions] -- Spatial dataset that contains
the mesoregions\footnote{Mesoregions are subdivisions of Brazilian
states. These subdivisions group various municipalities, based on neighboring municipalities and their common
characteristics. They were created by the Brazilian Institute of Geography and Statistics for statistical purposes.} area in Par\'a state. With this variable it was possible to group the statistical
variables, originally grouped by municipality, to their corresponding mesoregion. 

\end{description}


\subsection{3.2 Visualization }

The Brazilian Amazon Rainforest Dataset was visualized in NASA World Wind, divided in three
different variables with the deforestation phenomena, namely GDP, Population and Head of Cattle. Each variable can
be projected on the screens of the triangle and correlated with the yearly amount of deforestation, so the user can
navigate year by year through the spatio-temporal datasets and observe its correlation with the deforestation
phenomena. Figure 4 shows an example of the deforestation correlated with the GDP of each state of
Par\'a in the year 2005, where the geometries height represents the GDP per person and their colour the
amount of deforestation. 

 
\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img004.png}
   \caption{GDP (per person) and yearly deforestation in 2005.}
   %\label{fig:figure_label}
\end{figure}

Together with the variables and the deforestation visualization, several facts and pictures, related
to each variable, were displayed in the application as annotations. These facts and pictures were programmed to be
displayed according to the globe altitude, in a way that the user could have additional information to the application
current view. Figure 5 shows an earth view of the Brazilian Amazon deforestation correlated with population in the year
2008. By this altitude the application was programmed to display in the left-hand side facts relevant to the current
view. The facts displayed in the Figure 5 left side, written in German, say: ``The population of
Par\'a represents approximately the same population of Switzerland with an area 36,75 times larger'' and ``Par\'a, population: 7.321.439, Total Deforestation: 24,13\%,
Population Growth: 255.920''. This facts are hidden, giving place to other facts, if the user zooms
the globe in or out. In the Figure 5 right-hand side, a legend is displayed to classify the geometries colours and and
below it an image that explains the meaning of the geometries height, which in this case is population. 

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img005.png}
   \caption{Earth view of the Brazilian Amazon deforestation correlated with population in the year
2008.}
   %\label{fig:figure_label}
\end{figure}


\subsection{3.3 Gestures for Interacting with Spatiotemporal Data }

Due to the importance of temporal control during the exploration of the datasets, we tried to come up
with a gesture, which is easy to learn and to perform, and which also has a natural mapping to the temporal domain. Our
initial approach was to use a clock metaphor, which would let the user move the hand of a virtual clock to control time
by rotating their hand clockwise/counterclockwise. However, after some initial investigations we discarded this idea
due to concerns about awkward and uncomfortable movements as well as difficulties in recognizing it reliably. 

Eventually, we implemented three different gestures to control the spatial and temporal
attributes of our map. In designing them, we tried to mimic known gestures for multi-touch enabled surfaces such as
pinch-to-zoom and touch-and-drag-to-pan. While we were able to successfully apply this principle for the gestures
controlling the spatial aspect, we did not find widely-used, well-known gestures for temporal control and thus
designed them from scratch. The system currently supports the following gestures (cf. Fig. 6): 

\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img006.png}
   \caption{Complete gesture set implemented in the prototype (from left to right): one hand wipe
(panning), two hand spread (zooming) and wiping with hand above head (controlling time).}
   %\label{fig:figure_label}
\end{figure}

\begin{description}

\item[One hand wipe] -- The first gesture is a wiping motion with one hand, which can
be moved in any direction. This gesture allows the user to control which section of the map is shown, i. e. to pan the
map in the direction of the hand motion. It is possible to perform this gesture with either arm, but not with both at
the same time. 


\item[Two hand spread] -- The second gesture is used for zooming in and out of the map, i.
e. to change the scale of the map. To perform this gesture, users have to move both hands and forearms simultaneously.
If they move them closer together, the map is zoomed in, and if they move them further apart, it is zoomed out. 

\item[Wiping with hand above head] -- The third gesture controls the temporal aspect of
the map. Users can carry it out by raising one hand over their head and -- while keeping it above head -- performing a
horizontal wipe gesture with the other hand. Wiping to the left moves the timeline back by one year, wiping to the
right moves it forward by one year. Visitors to the science fair could learn about the gestures by studying one of the
posters we set up around the system, by observing other visitors interacting with the system, by talking to previous
visitors to the exhibit, or by directly asking a member of staff continuously present near the system. The
gestures can be seen in the project video\footnote{\url{http://goo.gl/oRTFO}}.


\end{description}


\section{4 Evaluating the Approach }

In order to gain a first understanding of gestural interaction with spatiotemporal data, we
carried out a questionnaire-based survey amongst visitors to the exhibit. Our goals were to get insights into how easy
people found it to use and learn the gesture set, and whether it enabled them to get a deeper understanding of the
complex spatiotemporal data they explored. In addition, we were interested in the fitness of the gesture set for
the actions they were mapped to. 

\subsection{4.1 Procedure }

We opportunistically asked visitors of the science fair, who had used our exhibit for several minutes
(i. e. the gesture controlled exploration system for deforestation data) whether they would like to participate in our
study. If they agreed, we asked them to fill out a short questionnaire. In addition, we informally observed the
use of the system while noting any patterns, behavior, incidents and comments we became aware of. Participants were thanked for taking part in the study but did not receive any payments in return for their time. 

\subsection{4.2 Material }

The questionnaire we used consisted of 28 open and closed questions in total. Participants were
first asked to provide some demographic information about themselves such as their age, gender and their primary
hand. We also gathered information about people's familiarity with common commercial systems that provide gesture
control interfaces (e. g. Nintendo Wii, Sony Playstation Move/Eye). The second group of questions investigated task
load for each gesture and were taken from the NASA TLX questionnaire [9]. We then asked participants questions relating
to the use of gestures to control the spatiotemporal data. They had to rate how well gestures and specific map
actions fit together, and to indicate whether they thought the gestural interaction interfered with the
exploration and understanding of the linked Brazilian Amazon data being displayed. Finally, participants had to rank
the three gestures overall. 

\subsection{4.3 Results }

In total, we collected 43 responses. 28 of the participants were male and 15 female. The average age
was 21.8 (mean deviation of 6.67); the youngest participant was 10, the oldest one 59. Seven of participants were left
handed, 36 right handed. 19 participants stated they are familiar with gesture-based interfaces in general, while the
other 24 stated the opposite. However, when asked about specific commercial systems (e. g. Nintendo Wii), it
became apparent that some people had not considered these to be gesture-based systems when answering the previous
question: 23 people indicated familiarity with Nintendo Wii, four with PS move, six with PS Eye Toy and eight with
Microsoft Kinect. Two participants named other gesture-based systems. For this question, multiple answers per
participants were allowed. 

Figure 7 summarises the results for task load according to the standard NASA TLX questionnaire. The
questionnaire uses a scale from zero to 20, where lower values correspond to lower work load, performance, effort
or frustration, and higher values to higher work load etc. Participants rated all three gestures in a very similar way.
Ratings for mental, physical and temporal demand ranged between 2.79 (std. dev. 3.23) and 4.07 (std. dev. 4.10), with the temporal control gesture generally being rated slightly worse than the other two.
Performance ratings were highest for the two hand spread gesture (14.23, std. dev. 3.75), followed by the single hand
wipe (13.12, std. dev. 4.2) and the wiping with hand above head gesture (12.67, std. dev. 4.6). The results in the
effort category were nearly identical, whereas there were notable differences in the frustration category:
here, the temporal control gesture attracted the worst rating (3.77, std. dev. 3.86), while the other two gestures were
rated very similarly (single hand wipe: 3.79, std. dev. 3.51 and two hand spread: 3.93, std. dev. 3.81). 

 
\medskip
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/a38621RV-img007.png}
   \caption{NASA TLX (task load) results for the three different gestures.}
   %\label{fig:figure_label}
\end{figure}
 
Results of the ranking task are summarised in Table 1. Overall, participants slightly preferred the
two hand spread over the one hand wipe; the wiping with one hand above head gesture was clearly the least preferred
gesture amongst the three implemented. We recorded a similar trend in terms of how well participants thought the
gestures mapped to the associated actions. The wiping with hand above head gestures was rated lowest at 13.07 (std. dev. 5.14), while the two hand spread
received a 14.02 rating (std. dev. 4.18) and the one hand wipe a score of 14.23 (std. dev. 3.75). The question, if the
gestural interaction was rather distracting or helpful in the exploration and understanding of the linked
spatiotemporal data on the Brazilian Amazon deforestation was answered with an average score of 13.92 (std. dev. 4.52).
Ratings were out of 20 from zero being worst to 20 being best. 

\medskip

{\centering
Table 1: Overall ranking of three gestures (1: best to 3: worst); the table lists
number of participant, who ranked each gesture either 1, 2 or 3.
}
%% ---- Images  -----
% The following is used for inserting an image
\begin{figure}[htpb]
   \centering
   \includegraphics[width=1\textwidth]{foss4g2013papers/38-Bartoschek/figs/table1.pdf}
   %\caption{The image caption goes here}
   %\label{fig:figure_label}
\end{figure}


\subsection{4.4 Observations }


In addition to the formal survey, we were also observing visitors interacting with the system: after having used our
system, a number of people reported that they found the controls mentally challenging during the first few
minutes, and were only able to engage with the content after that period of time. Some visitors were confused by the
direction of the temporal change as they were expecting a change in the opposite direction. One participant commented
in the questionnaire that our gesture is the opposite of turning the pages in a calendar. In addition, were received
several positive comments from older people, who found the system very accessible, particularly if compared to their
first time use of a mouse. However, there were also a number of people were unwilling to try out the system at
all. 

Furthermore, we noted some technical issues. The time between recognising a user and registering them as the user
controlling the system increased considerably with the number of bystanders/onlookers `seen' by the Kinect sensor. We
also observed some problems in recognising children, which depending on their arm length were not detected very well.



\section{5 Discussion }


Generally speaking, the results indicate that participants were able to easily learn and use all three gestures without
excessive work load, effort or frustration. Possibly due to people's lack of familiarity with the gesture used
to control time, the gesture consisting of wiping with one hand above head was rated slightly worse than the other two
gestures. It was particularly encouraging that participants considered the gestures helpful to engage with the
complex spatiotemporal dataset visualised by the application. In addition, several older users positively commented
positively on the system and its interface, indicating that gestural control could hold some potential to reach this
growing user group as well. 

However, since we were not able to specify specific tasks or learning goals beforehand, it is unclear whether this
self-assessment reflects actual learning performance. We also did not measure actual performance or error rates
during the study so that in this area, further research is required to confirm whether ease of learning and use
translate to good performance as well. 

Finally, we would like to mention the challenge of displaying large amounts of Linked Spatiotemporal Data on virtual
globes. In our solution we chose to use only the data from Par\'a state since it is an interesting and
representable part of the complete dataset about the Brazilian Amazon Rainforest. The data from Par\'a
state already has very interesting combination of each chosen dimension (ecological, economical and social factors). In
addition we visualized links to other countries based on the export statistics. Given these experiences we see very
potential research challenges emerging for visualizing larger datasets on virtual globes. 

\section{6 Conclusions }


In this paper we investigated the use of gestural interaction with complex spatiotemporal data, aggregated from Linked
Open Data sources. We introduce a prototype virtual globe application, based on open source software technologies. We
proposed three gestures to explore multi-dimensional deforestation data, two of which were inspired by common
multi-touch gestures, and one designed from ground up to control the timeline. In order to evaluate the gestures and
the overall approach, we conducted a survey study at a science fair. The results indicate that the gestures were easy
to learn and use (possibly after a brief initial learning phase), and were considered helpful in engaging with the
complex data visualised by the application. This was true for the different age groups that participated. 


In summary, our findings provide initial evidence that gestural interaction can also be successful in more complex
scenarios, and thus deserves further research. One promising next step would be to conduct a study investigating
naturally produced gestures (following the method pioneered by Wobbrock [22]). Additionally, modifying gestures by
sensing hand and finger postures could be very useful when interacting with complex spatiotemporal data (e.g. controlling the number of years time is moved forwards or backwards, by extending the corresponding number of
fingers while performing the gesture). 

In terms of visualizing connections between very different phenomena ---ecological, economical and social--- Linked
Data offered a good basis since connections could be made explicit via aggregation procedures, and via using time
and space as major integrators. As a contribution the Linked Data about the Brazilian Amazon Rainforest we created and
used is available also for others to use in visualizations and applications. 

By interlinking aspects of Human-Computer Interaction, Linked Open Data approaches and novel visualization techniques
for complex spatiotemporal data via open source software we contributed towards a Linked Open Science [14] to support
transparency and openness of science. 



\section*{References}


\begin{footnotesize}
%\begin{thebibliography}{99}

\begin{description}

\item[] 
[1] Bhuiyan, M., and Picking, R.  Gesture-controlled user interfaces, what have we done and what's next? Tech. rep.,
Centre for Applied Internet Research (CAIR), Wrexham, UK, 2009. 

\item[] 
[2] Bhuiyan, M., and Picking, R.  A gesture controlled user interface for inclusive design and evaluative study of
its usability. JSEA 4, 9 (2011), 513--521. 

\item[] 
[3] Cohen, C., Beach, G., and Foulk, G. A basic hand gesture control system for pc applications. In Applied Imagery
Pattern Recognition Workshop, AIPR 2001 30th (oct 2001), 74 --79. 

\item[] 
[4] Daiber, F., Sch\"oning, J., and Kr\"uger, A. Whole body interaction with
geospatial data. In Smart Graphics. IBM Int. Symposium on Smart Graphics, May 28-30, Salamanca, Spain, In: A. Butz, B. Fisher, and M. Christie, Eds., vol. 5531/2009, Springer (2009), 81--92. 

\item[] 
[5] de Espindola, G. M. Spatiotemporal trends of land use change in the Brazilian Amazon. PhD thesis, National
Institute for Space Research (INPE), S[2DC?]ao Jos\'e dos Campos, Brazil, 2012. 

\item[] 
[6] ESRI Inc. ArcGIS WFP API controlled by kinect. \url{http://www.youtube.com/watch?v=oICqUeIQlKo.} (accessed 02-22-2013).

\item[] 
[7] Fuhrmann, S., MacEachren, A., Dou, J., Wang, K., and Cox, A. Gesture and speech-based maps to support use of gis for
crisis management: a user study. AutoCarto 2005 (2005). 

\item[] 
[8] Han, J. Y. Low-cost multi-touch sensing through frustrated total internal reflection. In Proc. of UIST '05,
ACM (New York, NY, USA, 2005), 115--118. 

\item[] 
[9] Hart, S. G., and Stavenland, L. E.  Development of NASA-TLX (Task Load Index): Results of empirical and
theoretical research. In Human Mental Workload, P. A. Hancock and N. Meshkati, Eds. Elsevier, 1988, ch. 7, 139--183. 

\item[] 
[10] Jokisch, M., Bartoschek, T., and Schwering, A. Usability testing of the interaction of novices with a
multi-touch table in semi public space. In: HCI: Interaction Techniques and Environments,
Springer (2011), 71--80. 

\item[] 
[11] Kamel Boulos, M., Blanchard, B., Walker, C., Montero, J., Tripathy, A., and Gutierrez-Osuna, R. Web GIS in practice: a microsoft kinect natural user interface for Google Earth navigation. International Journal of Health Geographics
10, 1 (2011), 45. 

\item[] 
[12] Kar, A. Skeletal tracking using microsoft kinect the microsoft kinect sensor. Methodology 1 (2010), 1--11. 

\item[] 
[13] Kauppinen, T., de Espindola, G., Jones, J., S\'anchez, A., Gr\"aler, B., and
Bartoschek, T. Linked Brazilian Amazon Rainforest Data. Semantic Web Journal 1 (2012). 

\item[] 
[14] Kauppinen, T., and de Espindola, G. M. Linked Open Science--- communicating, sharing and evaluating data,
methods and results for executable papers. Proc. of the International Conf. on Computational Science (ICCS
2011), Procedia Computer Science 4, 0 (2011), 726--731. 

\item[] 
[15] Malik, S., Ranjan, A., and Balakrishnan, R. Interacting with large displays from a distance with vision-tracked
multi-finger gestural input. In Proceedings of the 18th annual ACM symposium on User interface software and
technology, ACM (2005), 43--52. 

\item[] 
[16] Owen Linderholm. Kinect Introduced at E3. \url{http://blogs.technet.com/b/microsoft blog/archive/2010/06/14/kinectintroduced-at-e3.aspx}. 
accessed 06-19-2012. 

\item[] 
[17] Rauschert, I., Agrawal, P., Sharma, R., Fuhrmann, S., Brewer, I., and MacEachren, A. Designing a human-centered,
multimodal gis interface to support emergency management. In Proc. 10th ACM international symposium on
Advances in GIS, ACM (2002), 119--124. 

\item[] 
[18] Serageldin, I. Promoting sustainable development -toward a new paradigm. In Valuing the Environment, I.
Serageldin and A. Steer, Eds., Proceedings of the first annual international Conference on Environmentally
Sustainable Development, World Bank (1995), 13--21. 

\item[] 
[19] Stannus, S., Rolf, D., Lucieer, A., and Chinthammit, W. Gestural navigation in google earth. In Proceedings of
the 23rd Australian Computer-Human Interaction Conference, OzCHI '11, ACM (New York, NY, USA, 2011), 269--272. 

\item[] 
[20] The OpenNI organization. OpenNI home. \url{http://www.openni.org/}. accessed 02-22-2013. 

\item[] 
[21] Wachs, J. P., K\"olsch, M., Stern, H., and Edan, Y. Vision-based hand-gesture applications.
Commun. ACM 54, 2 (Feb. 2011), 60--71. 

\item[] 
[22] Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proc.
of the 27th int. conf. on Human factors in computing systems, CHI '09, ACM (New York, NY, USA, 2009),
1083--1092. 


\item[] 
[23] Wu, M., and Balakrishnan, R. Multi-finger and whole hand gestural interaction techniques for multi-user
tabletop displays. In Proc. of the 16th annual ACM symposium on User interface software and technology, ACM
(2003), 193--202. 

%\end{thebibliography}



\end{description}


\end{footnotesize}



\endinput
