Usability Problems in GI Web Applications: A lesson from Literature

Organizations and individuals are becoming more aware of the benefits of geoinformation (GI) in supporting decision making. That being the case, there is a growing need to develop methods and technologies that make GI easily accessible, retrievable and usable to users. However, usability problems in GI web applications reportedly hinder users from effectively carrying out their intended operations. This paper contributes to a better understanding of GI-specific usability problems in GI web applications by: 1) Extracting usability problems from previous studies, 2) Classifying the problems, and 3) Identifying critical components of GI web applications. A systematic literature review approach is adopted to provide usability evaluation studies. 201 Usability problems were extracted from 12 studies, classified and qualitatively analysed. The results indicate a significant need for a GI specific usability framework, to provide support for GI specific usability evaluation and to provide generic solutions to reoccurring problems.


Introduction
Geoinformation has become the epitome of research and spatial decision-making. Consequently, GI web applications have become increasingly available online, providing free access to geoinformation and services. Furthermore, GI web applications are used by a wide range of users with different characteristics, needs and expectations.
Presently, there are no standardized guidelines for GIspecific usability evaluation (Resch & Zimmer, 2013). The design, development and evaluation of GI web applications relies on the Human Computer Interaction (HCI) and software engineering principles. Consequently, GI web applications continue to suffer from low usability as there are no generic solutions to GI-specific usability problems (Henzen, 2018). Different usability evaluation methods are used, such as user testing and expert evaluation using eye tracking, user survey, user observation techniques and as of lately remote evaluation techniques have also been embraced (Unrau & Kray, 2021). Ideally, the usability evaluation test results should incite the development of generic solutions to recurring usability problems and improvement of usability evaluation methods. On the contrary, the evaluation results mostly end with recommendations (Holzinger, 2005). Therefore, this research aims to collectively analyse usability issues to identify recurring GI-specific usability problems and critical GI web application components to provide input for the GI-specific usability pattern-based framework. The pattern-based framework is developed to provide generic solutions to GI usability problems.
A systematic literature review approach has been adopted to provide data, to gather a broad perspective on usability issues encountered in different GI web applications. By doing so, we carefully consider dynamic characteristics of GI web applications and the diversity of users to fully understand the provided usability issues. The usability problems are extracted from the literature articles, qualitatively analysed and classified to identify common usability problems. A similar study by (Unrau & Kray, 2019) was done focusing on evaluation methods and common practice for the usability evaluation of GIS applications in general.

Methods
The methodology comprised of two main components: a) Systematic literature review and b) Usability problem extraction. Both components comprised of 3 main steps as illustrated below. This methodology is adopted from (Dias, Pereira, & Freire, 2017). This section dealt with extracting studies on usability evaluation for GI web applications. 'GI web applications' and 'Usability testing and evaluation' were the main search terms translated into a search query as shown in figure 2. Google Scholar database was identified as the data source because its open access, provides advance searching functionalities. The database could be accessed through an external data mining software called 'Publish or Perish' that was used to extract data using BOOLEAN operators (Harzing, 2007). The extracted papers were evaluated and selected in three main steps. First, the title, abstract and keywords were screened, to include studies that addressed usability evaluation of GI web applications or its components. The second step, analysed the objectives of the evaluation study, the methods and techniques used to ensure user or expert evaluation was carried out. A clear understanding of the evaluated application type was also acquired to ensure mobile applications and desktop GIS are excluded. Finally, the results and discussion sections were fully read, to ensure usability problems were identified and can be extracted for further analysis.

Problem extraction
The usability problems in this study, are issues or user concerns that pose as an obstruction or difficulty when interacting with the application. For a problem to be selected, the criteria required it is well described, and can be easily understood and can be classified according to GI web application components and usability heuristics explained in the next section. Problems were extracted manually.

Problem Selection
Extracted problems were further screened and relevant problems were selected. Excluded issues include internet connections, hardware problems, as well as unclear statements such as "Participants gave the wrong answer" (Stanislav, et al., 2019). Such a statement cannot be classified as it does not provide enough information. The study also considered statements indicating a user concern or a suggestion on the use of the application. For example, "You have to have some sort of scale, so you can filter" (Kalantari, Syahrudin, Rajabifard, Subagyo, & Hubbard, 2020). Unfortunately, some studies did not document all identified issues and some issues were extracted from the discussions and results section as the author's analysis and interpretations of user testing data.

Classification of usability problems
This research believes that usability of GI web applications comprises of web usability elements and map usability elements. Understanding and classifying usability problems from this perspective helps identify GI-specific usability problems, and the GI components that require improvement, to better support users in effectively using the services and provided geoinformation. Usability problems are classified into subcategories of Map usability elements (Map elements and basic navigation tools; Basemaps, layers and symbology; Metadata content & representation) and Web usability elements (Data discovery, access and download functions, User Interface elements and Application logic) accordingly. The distinction between the two are that Map usability elements are specific to GI data and services such as map navigation tools, data content and metadata services etc. While Web usability elements are system functionalities that are not specific to GI web applications and can be found on other types of applications. These are general functions such as searching buttons, navigation and menus, help functionalities, user guidance elements etc. A problem can only belong to one category.
Subsequently, usability problems were analysed and classified according to Nielsen's 10 usability heuristics. Nielsen's heuristics are a prominent set of guidelines that provide advice on usability characteristics of an interface (Nielsen J., 1993). They were considered among others because they are common in literature (Dias, Pereira, and Freire 2017). Definitions and more details for each heuristic are found at (Nielsen, 1994) Initially, 1000 studies were extracted through the query and 11 additional papers were obtained from citations, references, and google scholar search recommendations. After extraction, the title, abstract and keywords were screened manually to make sure the studies addressed usability evaluation of GI web applications. Resulting in 909 exclusions. The excluded studies are those not addressing usability evaluation of GI web applications, literature review and mobile or desktop GIS studies. Subsequently, full text review was done manually to make sure the studies meet the criteria stated in Figure  1. A total of 73 studies were excluded due to duplication, mobile or desktop-GIS, unpublished work or focused on evaluating usability methods and not identifying usability problems. Subsequently, the methods and results sections for remaining 17 were screened to ensure usability problems were identified and users were involved in evaluation. Finally, 12 studies were selected. The excluded 5 did not have usability problems identified. The included studies are listed in figure below:

Systematic Literature review component
Study Application type # of problems [

Usability Problem Extraction component
A total of 239 usability problems were extracted from the 12 studies. 31 of these problems were excluded because they are: -Related to internet connection, -Hardware problems, -Unclear descriptions not giving enough information to classify the problem. E.g. "Users gave wrong answers".
The remaining 208 were further screened and 7 problems were excluded because they were duplicates, leaving 201 problems for further analysis and classification.

GI-specific Usability Problems Vs Web Usability Problems
About 53% of the extracted usability problems relates to the map User Interface (UI) elements, data content and representation. These are problems experienced at identifying elements, using and interpreting map and metadata content. This does not only present low usability for map UI, but also illustrates the need to coordinate map elements with usability aspects in order to develop usable map UIs. According to the analysis, the most critical components of GI web applications that poses high usability risk are: 1) Metadata content, 2) Map elements & navigation tools, 3) Basemaps, layers and symbology, 4) Searching functionalities & results, and 5) Data manipulation and analysis functions. In essence, the map UI suffers more from low usability in comparison to the general UI. Figure 6. Usability problems according to GI web elements Web usability elements here, are system functionalities and tools for data discovery and access functions, as well as general application elements such as search buttons, filters, UI navigation tools, download functionalities, user feedback and guidance elements and overall application interaction functions. Figure 6 depicts a relative high challenge for data discovery features compared to other features. For some provided analysis functionalities, users aren't able to figure out how to utilize them and the systems do not provide clear support on how to use them.

Recognition, rather than recall -20%
System design should facilitate easier recognition of features, map elements, tools, actions and available options without requiring much time from the user. By displaying features in a way that allows users to recognize and locate suitable options and features of interest. Most issues relate to user challenge in recognizing the right tools, users unable to predict the next steps and behaviour of the tools, and also inability to identify right values and parameters needed for their tasks. This could be because of the visibility of features according to user needs and interests. For instance, "It was also difficult to find the layers in order to activate the desirable one, since the list of the layers is located on a hidden flap on the right side" (Jesus, Brito, & Fernandes, 2017). The hidden table of content makes it difficult for users to find and select layers of interest. According to (Nielsen J., 1993), designs developed for recognition relies to a great extent on visibility of the object of interest to the user. Interestingly, problems such as: "Biggest problem was determining exact value of selected parameters" (Stanislav, et al., 2019), indicates that the instructions on the use of system functionalities to provide users with help in choosing right options are either not visible or non-existent. This may also indicate potential lack of literacy and domain knowledge in users as a cause and further investigations are needed to understand why.

Consistency and standards -14%
According to (Nielsen J., 1993), people spend most of their time on products other than yours. Failure to maintain consistency may increase the user's cognitive load by forcing them to learn something new. Most of the problems in this category relate to inconsistent metadata content, misplaced and missing metadata content and inconsistencies between tool and data content. For instance, "Why put this in the data source? Class should not be in the data sources" (Kalantari, Syahrudin, Rajabifard, Subagyo, & Hubbard, 2020), shows user frustration due to misplaced metadata content. More care should be taken in using known metadata convention and standards and evaluating metadata systems for consistency and usability.

Error prevention -10%
An application that provides good error notifications is relatively acceptable. Problems relating to errors encountered through usersystem dialogue, errors in displaying features and errors in using the tools are classified here. Most issues are data-tool and implementation errors. For example, "Sometimes a facet persists into the next search and doesn't clear" (Blake, Majewicz, Tickner, & Lam, 2017). In this case, the previous search continues into the next search, disrupting the searching activity of the user. Users without a clear interpretation of this issue may not be able to know how to continue. Another data-tool issue is: "If the tab is closed, the map loses the classification of the data" (Jesus, Brito, & Fernandes, 2017). When a user creates a self-generated map, closing a certain tab while still on the map causes the loss of classification.
Overall, there are a lot of errors in system functionalities that indicates a lack of thorough evaluation before launch. Evaluation of data content to system functionalities should never be underrated to prevent these type of errors.

Aesthetics and minimal design -7%
For GI web applications to communicate and provide geoinformation effectively, it is only necessary to consider the aesthetics and design of the map UI. The problems in this category pertain to the appearance and placement of interface elements. This category suggests that the amount, the look and feel of features not to obstruct the visibility of other features. Most issues relate to map UI elements, cartographic symbolization and generalization. Symbol scaling and different scale representations were among the issues observed to cause obstruction to other map features. Navigation issues were also reported, poorly arranged legends, inappropriate fonts sizes and inappropriate locations of important map elements were also recorded. This category indicates that as application UIs become simple and intuitive, cartographic interfaces need to do the same. Care should be taken to match cartographic objects of interest with user needs for user generated maps, particularly the visibility of cartographic objects should be well considered.

Conclusions
The usability heuristic guidelines provide guidance on usability characteristics of an interface. The extracted usability problems were qualitatively analyzed and classified according to Nielsen's 10 usability heuristics. Classifying usability problems from this perspective, uncovers the root cause of the problems by evaluating the heuristic guideline, the UI and the problems at hand.
The top five heuristics with the most usability problems combined represent 85% of the extracted usability problems. And by analyzing the usability problems and the heuristics above, we discuss the major findings below: Major findings: -Lack of functionalities and options according to user needs and expectations.

-
Lack of practice to maintain metadata standards for consistency and lack of methods to evaluate metadata systems for usability.

-
Lack of evaluation practice to identify implementations and data-tool errors before application launch. -Lack of standardized guidelines for cartographic elements and UI design.

-
Lack of user guidance, warning errors and visibility of map status.

User challenges and expectations:
In regards to user challenges and expectations, there is a lack of understanding on how to operate system functionalities, such as distance measurements, decoding a legend and lack of skills to create selfgenerated maps.
Users struggle to identify the tools to use, struggle to use the functionalities aimed to provide help and support in choosing the right options and parameters.
However, we observed user expectation for more system functionalities, such as flexible options to visualize data, more interactivity and automated help functions.
In conclusion, this study provides a clear indication on the way forward in order to contribute to the improvement of the usability of GI web applications. Future research could extend this study by addressing the question on how these usability problems provide an input for the GI usability pattern framework?