Investigating drivers’ geospatial abilities in unfamiliar environments

The prominence of landmarks in aiding pedestrian navigation has been highlighted in various studies; people rely strongly on visual landmarks, especially when navigating in unfamiliar environments. The paper describes the design and implementation of a study for assessing drivers’ spatial abilities, when navigating in an unfamiliar environment. Two types of route directions based on references to either landmarks or street names were given to two groups of participants. Three geospatial learning tasks are used to evaluate these abilities: map sketching, distance, and direction estimation. The findings showed that landmark-based route instructions help drivers develop a better cognitive map of the route. On the other hand, instructions either based on landmarks or on street information do not have an effect on distance or direction estimates. Nonetheless, qualitative analysis of directions and distances estimations gave interesting results. Findings associated with self-assessment of environmental spatial abilities using the Santa Barbara Sense of Direction Scale (SBSOD) seem to support prediction of at least one of the drivers’ abilities among those assessed in this study.


Introduction
A large portion of research on human navigation has focused on the evaluation of navigation and orientation abilities depending on the purpose and the properties of the route, the route instructions, and navigation systems. At large, it has been extensively documented that landmarks play a predominant role in human navigation and spatial memory (Daniel and Denis, 2004).
Research has focused on the enrichment of navigation instructions using reference to landmarks (Basiri et al., 2016, Burnett et al., 2001, Denis et al., 2007, Klippel and Winter, 2005, Raubal and Winter, 2002. In addition, other surveys have shown that by using landmark references instead of references to street names during navigation, people experience fewer difficulties and gain a better understanding and recognition of the route (Tom and Denis, 2003).
Although navigation systems provide useful instructions for successfully moving from an origin to a destination, users have problems developing survey knowledge of the environment they passed through (Sester and Dalyot, 2015). The enrichment of route instructions with salient landmarks (Raubal andWinter, 2002, Wage et al, 2018) and other survey knowledge (Sester and Dalyot, 2015) is considered a powerful alternative to conventional turn-by-turn instructions for facilitating spatial representation of the environment. Michon and Denis (2001) performed two studies to investigate the role of landmarks in route instructions in urban environments. In the first study, participants learned a route and then were asked to generate route instructions for guiding pedestrians unfamiliar with the environment in finding their way. Participant-generated route instructions were analysed on the basis of the number, frequency, and spatial distribution of landmarks along the route. In the second study, participants followed a route with minimal navigational instructions and then were asked to correct, enrich or revise the instructions given to them. The results revealed the prominence of landmarks in developing a mental representation of the environment along the route and in providing information about critical actions to be performed. Denis (2003, 2004) performed a series of studies to compare the effectiveness of route instructions in an urban environment based on references either to landmarks or to street names. Their results highlighted the guiding value of landmarks compared to street names regarding mental representation of routes by pedestrians.
Although much empirical work is conducted on navigation and orientation and other geospatial abilities of pedestrians in unfamiliar environments, to our knowledge there are no similar studies for assessing the abilities of non-professional drivers, as opposed to studies that assess abilities of driving professionals, such as taxi and bus drivers (Chase, 1983, Maguire et al., 2006. The aim of this study is to evaluate spatial abilities of non-professional drivers as well as their ability to build cognitive maps of an unknown environment depending on the form of navigation instructions they are given. The study was based on existing literature on the evaluation of large-scale (environmental) spatial abilities (Hegarty et al, 2006, Sas andMohd Noor, 2009).
For this purpose, the study uses a series of geospatial tasks to assess corresponding participants' geospatial abilities. Map sketching is used to evaluate the perception of the area and of the route and consequently the completeness and accuracy of the cognitive maps formed. Furthermore, the study examines the extent to which the formation of cognitive maps is influenced by the instructions' reference to landmarks or street names. Participants were also asked to estimate directions and distances; geospatial tasks that indicate the level of human geospatial abilities (Allen et al. 1996, Hegarty et al, 2006. Additionally, the degree to which each person can successfully assess the level of their geospatial abilities and therefore predict their performance in specific geospatial tasks is also assessed.
The paper is organized as follows: Section 2 details the survey design process and implementation procedure and provides information on availability of the survey data and of the analysis workflow. Section 3 highlights survey results, while conclusions and survey limitations are presented in Section 4.

Participants
The study involved 25 participants (12 men), whose age ranged from 25 to 71 years old. Participants were selected based on the following criteria: • To have a driving license and own a car.
• Not to be familiar with the area where the study took place so that the effect of prior knowledge is excluded. • To be a relatively experienced driver to minimize the effect of stress and driving inexperience on the results.

Study Area and Route Selection
For surveys on human navigation abilities and performance, the route selection constitutes a task performed by researchers in a non-systematic way, impeding replicability and resulting in idiosyncratic route selections that only fulfil the needs of a particular survey. A methodological framework which enables researchers choosing a route for wayfinding experiments according to predefined weighted criteria has not been formulated and implemented until very recently (Mazurkiewicz et al., 2021). Many of the referred surveys as the authors point out, lack justification of route selection and mention of route properties, thus, we provide as much information as possible on the study's route in what follows.
The route was selected to satisfy the needs of both navigation methods used in this study and more specifically to meet the following criteria: 1. To be simple, i.e., to consist of straight-line segments and limited number of navigational decision points since, simpler environments are considered more effective for investigating spatial abilities (Sas and Mohd Noor, 2009). 2. To have adequate landmarks, visible road signs, street addresses and numbers all along, to help drivers at points where actual changes of direction are necessary or likely to occur (Allen, 2000). 3. The formulation of route directions should be based on principles and practices for communicating route knowledge effectively (ibid). 4. The selected route should constitute the optimal route connecting start and destination as suggested by Michon and Denis (2001).
Landmarks are located along the route: (a) at points where the participants had to change direction (turning/ decision points), (b) along straight line segments so that they could verify that they were driving at the right direction, and (c) at start and destination points. Landmarks have been selected on the basis of the five salient characteristics of valued landmarks identified by Burnett et al. (2001): permanence, visibility, location usefulness, uniqueness, and brevity.
The experiment took place in the metropolitan area of Athens, Greece, over a 1.2 km route that consisted of four road segments and three turning points (Fig. 1). The route started at a well-known fast-food restaurant (Everest) (node A) and ended at a bank (EuroBank) (node E). Other landmarks included the intersection with a motorway (Attiki Odos), the municipal athletic center (Gym), a playground, a square with coffee shops, an elevated water tank, and a chain store (PetCity); the above constitute the landmarks referred to navigation instructions given to half of the participants (see following section).

Procedure
Participants were informed about the general aim of the study and were given the following instructions: "You will be given directions to drive a simple route. The objective is to form a general perception of the route and to observe all those aspects that will allow you to provide sufficient driving instructions to someone else who will drive the same route to reach the same destination. The route will be driven only once".
Participants were not informed about the spatial tasks they had to perform afterwards or about the nature of driving instructions they would be given. There was no time limit regarding either the driving part of the survey or the completion of the questionnaire.
Before starting the navigation process, participants filled in a questionnaire with personal data (sex, age, education, and profession) as well as their level of familiarity with the study area (ranging from 1: not at all familiar to 5: very familiar). They also filled in the Santa Barbara Sense of Direction Scale (SBSOD), a selfreport measure of environmental spatial abilities (Hegarty et al., 2002).
Participants were then asked to drive the route, following navigation instructions given by the co-driver (survey administrator). Participants were tested individually and the duration of the whole process ranged from 45 to 60 minutes. All participants followed the same route only once.
Participants were divided in two groups. The first group, which consisted of 12 participants (males; 6), with age ranging 25-67 years old (M=39.1, SD=16.1) and being slightly familiar with the study area (M=2.25, SD=1.1), was given navigation instructions referring to landmarks along the route. The second, which involved 13 participants (males; 6), with ages between 25-70 years old (M=48.6, SD=16.9) and somewhat familiar with the area (M=2.63, SD=1.3), was given navigation instructions referring to distances and street names similar to the directions derived from navigation apps such as Google Maps.
Although the driving instructions differed between the groups, the same rules applied for both. Participants were given purely verbal instructions without any visual aid. After identifying the starting point by themselves, each driving instruction was given to them immediately after the execution of the previous turning point and not close to the next. Once participants arrived at their destination, they returned to the starting point by a completely different route. They were then asked to make distance and direction estimates between landmarks and to draw the route's sketch map.

Data and Software Availability
Questionnaires and sketches were collected anonymously. All statistical analyses, which results are detailed in the following section, have been performed in R (R Core Team, 2021) using the tidyverse package (Wickham et al., 2019). Driving directions given to participants, an Exemplary Questionnaire in English, the collected survey data in tabular form, the R code of the statistical analysis workflow, and all necessary metadata supporting this publication, are available on figshare and are accessible via the following DOI: https://doi.org/10.6084/m9.figshare.14460102.v4. The workflow underlying this paper was successfully reproduced by an independent reviewer during the AGILE reproducibility review and a reproducibility report was published at https://doi.org/10.17605/OSF.IO/DX92A.

Results
Three measures were evaluated corresponding to the tasks performed: map sketching, distance estimates, and direction estimates. Τhe results of the SBSOD questionnaire are also included in this section, as well as a qualitative performance assessment of the three geospatial tasks compared to SBSOD scores.

SBSOD Scores
As mentioned, participants answered the 15-question SBSOD scale. SBSOD has been proven a powerful tool of self-assessment in terms of large-scale spatial abilities and it is used in a plethora of surveys that seek to assess spatial abilities in real/ physical spatial environments (Davies et al., 2017). Both groups were tested for normality, however, according to the Shapiro-Wilk test, the first group does not come from a normally distributed population (W(12)=0.858, p=.046), while the second does (W(13)=0.896, p=0.118). Thus, a Mann-Whitney U test was performed to test whether the two groups' distributions are notably different or not. The test showed that there is not a statistically significant difference (W=47.5, p=.102) between the two groups regarding their SBSOD scores.

Map Sketching
Participants were asked to sketch a map of the driven route ( Fig. 2). They were given the following additional instruction: "You are invited to add any detail you noticed along the route (e.g., buildings, shops, traffic lights, etc.) which could help others reach the same destination by following this route in the future".
Map sketching assessment methods, both qualitative and quantitative, vary, with particular emphasis on identifying landmarks, their relative position, and quadrant accuracy. To rate participants' performance in this task, qualitative errors were measured based on the methodology outlined in Hegarty et al. (2006), in particular: • The number of landmarks omitted, relative to the number of landmarks referred in the instructions. • The number of road segments missing from the map or those mistakenly added. • The number of wrong direction turns, i.e., right turn instead of left. • Errors are summed up to provide the participant's performance score. The closer the scoring to zero, the better the performance, indicating less errors in the map sketching task. .68), t(21,12)=-3.25, p=.004. The 95% confidence interval of the difference is -3.03 to -0.67. These results suggest that the navigation strategy has an effect on the number of landmarks omitted from the sketched maps of the two groups. On average, 2 of the 8 mentioned landmarks were omitted by the 1 st Group participants, while 4 out of 8 by the 2 nd Group participants. The result makes sense, as the 1 st Group was given instructions based on references to these landmarks and therefore had an advantage in identifying them and placing them correctly on the map. Drawn from the previous result, an analysis of the overall correctly placed landmarks on map sketches by both groups was also performed. Fig.2 shows that a 1 st Group participant added seven out of the eight landmarks mentioned in the driving instructions, while also correctly placing seven additional landmarks along the route. According to the Welch's t-test conducted, there is difference in the number of landmarks scores for landmark-based (M=9.83, SD=2.04) and street-based navigation (M=7.08, SD=2.56); t(22.53)=2.99, p=.007. Participants in the landmark-based navigation group scored higher in correctly identifying and placing landmarks than participants in the street-based navigation group by an average of more than two landmarks. The 95% confidence interval of the difference is 0.84 to 4.67. However, it is important to bear in mind that group 1 participants are somewhat favoured since they have received information on eight landmarks along the route, through the instructions they had be given throughout the driving process. Thus, we can say that, during the map sketch task, they were able to recall the landmark-based information they were given, while group 2 participants were able to build a cognitive map of the route indicating a substantial number of landmarks along the route even though they were not given this kind of information. Examining the results from both perspectives, omission and addition of landmarks, overall group 2 participants managed to include an average number of seven landmarks in their sketch maps without having been given such information, while group 1 participants, who had information on eight landmarks along the route, managed to include in their sketch maps, an average of almost two additional landmarks than those already informed about.
This finding partly supports the prominence of landmarks in developing a mental representation of the environment along the route and in providing information about critical actions to be performed as proven by Michon and Denis (2001). In our case, participants in both groups turned to landmarks as aids in representing and configuring the environment along the driven route by adding traffic lights, different kinds of stores, gas stations, churches, etc. resulting in a variety of information about the spatial environment on their sketch maps.

Direction Estimates
The second task involved estimating four directions by answering the following type of questions: "Imagine standing at … looking at ... Draw an arrow from the center of the circle pointing to …". Direction estimates were made by means of a circular pointing dial as described in Hegarty et al. (2006). Although the wording of the direction questions differed for the two groups ( Fig. 3) according to the driving instructions they were given, both groups were asked to make the same direction estimates.
Direction estimates were scored using the absolute accuracy score (AAS) as introduced by Jacobson et al. (1998). AAS represents the average percentage difference between the objective direction values (ODV) and the cognitive direction estimates (CDE). Pointing error is measured as the smallest angle between the correct target direction from the estimated one; no pointing error must exceed 180 o (Sholl et al., 2000) (Eq.1). Thus, estimations between 270 o and 359 o should be handled so that their scores fall below 180 o (Kitchin and Blades, 2002). The final answer is made positive. Qualitative analysis of both groups' direction estimates indicates the following:

AAS =
• Participants practically excelled in estimating directions #1 and #4. Only four gave wrong answers deviating 5 o -15 o from the correct directions. We can assume that participants can estimate directions parallel to a travelled route easily and correctly, especially if they do not have to reverse their perspective of the travelled route. • On the contrary, for direction #3, which requires participants to reverse the travelled route, an average deviation of almost 33 o from the correct direction is noted. • Finally, direction #2 is also wrongly estimated with deviation reaching 29 o in average.
The above partly comply with the finding of Pagkratidou et al. (2020), where participants pointed more accurately to locations of higher axial integration and connectivity (node D directly connected to, and thus easily accessible from C), but pointed less accurately from those locations (from D to A). Visual connectivity seems also to play an important role as shown by the correctness of estimations regarding direction AE.

Distance Estimation
The final task involved the estimation of four distances, which correspond to the four segments of the route (Fig.  1). These are: AB=650m, BC=60m, CD=400m, DE=55m long.
To evaluate distance estimates, the score for each distance estimate was the absolute difference between the estimate and the correct distance (|Estimated-True Value|). Thus, high score values indicate poorer estimations, while low score values more accurate ones. A participant's total score in the distance assessment task was calculated as the average of the four distances estimates scores. This was preferred over the score appearing in a number of similar works (Kirasic, and Rowley Bernicki, 1990, Kirasic et al., 1992, Kirasic, 2000, Hegarty et al., 2006 as the correlation between log estimated distances and log actual distances, since in those surveys, subjects were instructed "to provide a number to represent the distance from a reference point to a target point and were told that all subsequent estimates would refer to this distance" (Kirasic, and Rowley Bernicki, 1990), which does not hold in our case.
Group 1 participant #3 was excluded from further analysis of distance estimates, since their estimates are disproportionately different from the actual distances. Once included in the analysis, they would affect the statistics of the group and our small sample sizes are sensitive to extreme outliers.
According to the Shapiro-Wilk normality tests, both groups come from populations that follow normal distribution (W=0.903, p=.199, W=0.979, p=.974 for the 1 st and 2 nd group respectively).
The Welch's t-test that was conducted to compare scores in this particular task between groups, showed that there is no statistically significant difference in the distance estimation scores for landmark-based (M=103.75, SD=69.27) and street-based navigation (M=58.56, SD=27.60); t(12.68)=2.03, p=.064, CI (95%): -2.99, 93.38. These results suggest that navigation instructions either based on landmarks or on streets may not have an effect on distance estimation for this study.
When distance estimates were analysed individually, the two groups seem to present difference in distance estimation performance related to the third road segment (CD). Specifically, the Mann-Whitney test indicates statistically significant difference between the two groups regarding this estimate (W=118, p=.005).
Results show that participants of the 2 nd Group were more accurate in their estimates of the CD road segment (Group1: M=193,64, SD=144.38, Group2: M=38,46, SD=65.04). However, this is most likely because this information was given to them as it was part of the driving instructions they received. Since the overall distance estimates do not exhibit any statistically significant difference between the groups, we can assume that better performance of group 2 participants in one distance estimate is not indicative of better spatial skills.
Overall distance estimation proved to be the most problematic task for participants. Although the distances are relatively small and the route is very simple, participants do not demonstrate the ability to estimate them effectively. Among the 96 estimates (24 participants, 4 estimates each) only 16 were correct (16.66%). Among those, 11 concern segment CD which was given as metric information to group 2 participants as part of their navigation instructions. The number of over-and under-estimations is similar (38, 42: 39.59%, 43.75%, respectively), so we cannot say that participants have the tendency to over-or underestimate distances. The pattern of underestimating shorter distances (in near vista space; distances less than 75 m) and overestimating longer ones (in far vista space; distances longer than 75m), as shown in Daum and Hecht (2009), may not be applicable to drivers since other factors such as traffic, traffic lights, speed limits, weather conditions etc., may affect their ability to estimate travelled distances. Fig. 4 highlights another aspect of distance estimation; relative estimation errors. Real relative estimation errors (red dots), for both groups, range from less than -8% to more than 22%, without being proportional to distances. Distances DE and BC, which differ only 5 meters in length, present extensive difference in average overunderestimation, with DE, the shortest distance, being slightly underestimated (5%) and BC, a 5m longer distance, being overestimated more than 20% of its actual length. Absolute relative estimation errors (blue dots) reveal another interesting finding; participants' estimations for shorter distances are worse than for longer ones. Deviation of absolute estimates from the actual distances for shorter ones is almost 10% larger than for longer ones.

Relations among Variables
Judging from the measures of dependence (spearman correlations) between the variables, shown in Tabs.1 and 2 for the 1 st and 2 nd Group respectively, statistically significant relation exists only between SBSOD and Map Errors for both groups which is actually a moderate negative relation, meaning that participants with high SBSOD scored also well in Map Sketching tasks (fewer errors). For the 2 nd group, an additional moderate negative relation is revealed between distance and direction estimates, meaning that high scores in distance estimates do not necessarily entail the same for direction estimates among participants of the group. Upon scrutiny of the 2 nd group results, it can be stated that participants with very good direction estimates did not perform that well in distance estimation. In fact, most of their distance estimates are below average scores.

Variables
All other spearman correlation calculations are not statistically significant and are rather attributed to chance (p>0.05).
An analysis of individual results of the participants' selfassessment of environmental spatial abilities using the SBSOD, predicted to some extent their geospatial abilities. From our sample, we are not able to strongly confirm the view that "the most valid tools for assessing environmental spatial abilities are self-report questionnaires" (Sas and Mohd Noor, 2009). Specifically, participants from both groups who reported to have a very good sense of direction, estimated directions slightly better than the average of the population. Similarly, participants who reported a very good ability to construct a "mental map" of the environment (question #15 of SBSOD), performed better in the map sketch task (all participants from both groups who scored 6 and higher at #15, made fewer mistakes in their sketch maps than others). On the other hand, distance estimation results cannot be used to draw similar conclusions regarding their relationship to the reported ability (question #3 of SBSOD), since the overall performance in distance estimation was moderate.

Conclusions
As it stands, much survey work is performed in pedestrian navigation for assessing spatial abilities in unfamiliar environments, but limited similar research has been undertaken for drivers' performance.
The aim of the present study was to evaluate spatial abilities and the formation of cognitive maps of nonprofessional drivers based on verbal navigation instructions and not the drivers' capacity to successfully navigate the route. These abilities have been assessed based on direction and distance estimations, and map sketching. From the results, we can conclude that drivers' formation of cognitive maps involves landmarks as a prominent element of representing the environment along a route. Drivers who had not received any landmark information through driving instructions, managed to recall landmarks along the route almost as many as those drivers who had been given this kind of information through driving instructions.
Regarding direction estimates, participants were able to point to the correct directions when there is no reverse driven route involved and when directions are parallel to the route. In any other case, drivers seemed to experience difficulties in estimating directions.
Concerning distance estimation, study drivers did not seem to follow the pattern of underestimating shorter distances and overestimating longer ones, since other factors such as traffic, traffic lights, speed limits, weather conditions etc., may have affected their ability to estimate driven distances.
SBSOD scale has been used in a number of surveys where pedestrian navigation was involved and has exhibited some degree of predictive power to spatial skills at large scale. However, SBSOD has not been correlated to drivers' spatial abilities, since no such studies have been conducted to the authors' knowledge.
The study showed some interesting preliminary qualitative results, which encourage to pursue studies with a larger number of participants that will allow drawing extensive conclusions regarding the relationship between performance in geospatial tasks, the ability to construct cognitive maps of the environment and the instructions given when driving a route.
The study route was selected to be simple intentionally based on previous research associating route simplicity with a better investigation of spatial abilities. Once this has been understood, a slightly more complicated route including more line segments and turns and compliant with the recent methodology for systematic route selection (Mazurkiewicz et al., 2021) would probably allow to reveal drivers' geospatial abilities in unfamiliar urban environments to a larger extent. Nonetheless, more experimental research has to be pursued for studying drivers' performance in cognitive maps development and geospatial abilities in general.