The focus of this task was to conduct human factor testing on a large number of collision warning formats. This work was being conducted to try and determine the factors that make up the best combination of components in a warning system and to reduce the number of warning systems that will eventually be tested in the HRL Driving Simulator (HDS) which is a real time, interactive driving simulator.
The primary objective of the task was to conduct all necessary human factors studies to determine the best visual and auditory warnings for use in the HDS. Once the tests were completed and the data analyzed, the best combinations of warnings would be determined, and a final set would be chosen and integrated into the HDS for use during Task 6.3 (Driver-Vehicle Interface Studies).
This task was carried out by personnel from DE/AED and the University of California at Davis (UC-Davis). The initial warnings tested include both visual (icons and text) and auditory warnings (synthesized and digitized speech, tones, and spatial cues). At the conclusion of Task 6.1, the following goals were attained:
A collision warning system will be of little use to the consumer, if the driver can not effectively be made aware of potentially dangerous roadway situations. Consequently, an important feature of the collision warning system will be the proper method to warn the driver of potential crash events. This task has been a cooperative effort between UC-Davis and DE/AED. UC-Davis screened the initial set of collision avoidance warning concepts and conducted the rapid prototyping study of these warning interfaces. DE/AED provided feedback to the UC-Davis research group, and aided in the review of the results gathered from UC-Davis.
A final set of auditory and visual warning alerts were defined for testing using the following categorization scheme:
A videotape was prepared documenting all 50 alerts used in the experiment. In addition to the categorization based upon modality (i.e. A through F), each of the 50 alerts was assigned a type (i.e. number 1-6) which indicated the content of the information in the alert (i.e. whether the alert gave a general warning, gave an indication of the direction of the impending collision, gave an indication of what to do, and combinations of the above).
Seven dependent measures were developed, all 7 point likert-type scales capturing the following alert attributes:
Subjects rated all alerts on all dimensions yielding a sample of 350 observations per subject. In addition, a focus group type of discussion was conducted to capture additional qualitative information about each alert in the category. Finally, a subset of the subjects were asked to provide importance rankings for the first 6 attributes of the alerts (i.e. how important were each of the attributes in their ratings of the utility of the alerts). Independent variables included age in three categories (less than 35, 35 - 55, greater than 55) and gender. Number of years driving, number of crashes in the last two years and annual mileage driven was also collected.
The most significant challenge to this task was the recruitment of subjects that provide an even distribution of age. In particular, it was very difficult to recruit subjects over the age of 55. The initial recruitment activities yielded only 2 subjects older than 55. These recruitment activities included posting fliers around the UC-Davis campus and distributing announcements directly to university departments. Finally, a more aggressive effort was undertaken to recruit elderly subjects. Distributing an electronic mail message throughout the UC-Davis computer system, and networking within UC-Davis yielded an additional 11 subjects that were tested. While this complicated data coding, it was believed to be important enough to the goals of the study to make the extra effort to obtain older driver data.
The presentation of the alerts to the subjects was randomized to control for order effects. A script was prepared for the session moderator to use in conducting the experiments. Subjects were simply told that we were interested in their opinions and feelings about the alerts, that there were no right or wrong answers and that the alerts should be thought of as appearing in the instrument panel or transmitted through their radio. Consent forms were signed and the subjects provided their responses in a Response Booklet that facilitated data coding and subject response.
All sessions were conducted in the same room, with the identical configuration of the display screen and the subjects. In the front of the room were large display boards (approximately 3 x 4 feet) that displayed the seven full rating scales with anchor points. In addition, each subject was provided with a 1 page summary of the scales to use for ready reference. It took approximately 2 hours to complete the rating and focus group discussion for all of the alerts.
A total of 74 subjects were recruited and data was collected over four sessions during September 1995. The number of subjects in each session varied from a low of 5 to a high of 17, but was typically about 10.
A set of statistical measures were derived from the raw data including: age and gender distribution; means, standard deviations, minimum and maximum values for each alert and scale (50 alerts by 7 scales); means, standard deviations, minimum and maximum values for each alert category (i.e. A - F) and for each alert type (i.e. 1 - 6) separated by age and gender; bi-variate correlation among the seven ratings scales individually and by alert type and category; mean, importance rankings by gender (age was not possible due to only 30 observations of these measures); and focus group comments pertaining to each of the set of means and standard deviations. The SAS statistical software package was used to analyze the responses from the 74 subjects.
Analyses of variance was used to determine how the first six scales predicted the rating of overall utility of the alert. Such a model has coefficients that may be interpreted as the relative importance of each factor (predictor variables) in the rating of overall utility (dependent variable). This model largely substantiated the direct rankings of factor importance provided by a subset of our users. That is, getting attention is by far the most important factor in relative importance, and understandability as an alert is next in urgency. The last three factors are nearly equal in where to look for a problem. A series of additional models were requested and estimated from the data including the prediction of overall utility, and attention and urgency as a function of the six alert types. This is used to quantitatively test for differences in mean ratings between alert types.
UC-Davis completed all analyses and submitted a Task Final Report in February 1996. The report included a description of all experimental procedures and protocols, experimental controls, subject recruitment and interfaces tested. A series of analyses were reported which consisted of a comparison of means of the ratings of all the alerts tested. The report included recommendations concerning devices to be tested in the HDS as well as more generic characteristics of desirable devices.
In an attempt to determine the visual and auditory warnings that would be used in the Task 6.3 simulation studies, DE/AED evaluated the results gathered from the UC-Davis study. The UC-Davis study presented the top 10 warnings found to have the greatest `overall utility' for young and old subjects. From these results, commonalties across the young and old subjects were found. From the set of warnings presented to the subjects, old and young subjects highly rated auditory voices that stated "Caution", "Caution-Front" and "Crash-Left Side". One auditory tone that was rated high was the `clock alarm' tone. The text presentation of "Caution-Front" was also highly rated. The icons that were rated high for both groups were the car icon with the word "Caution" printed above it, and the car icon with a sweeping triangular area. Warnings currently designed for the DE/AED demonstration vehicles were also gathered, and a set of warnings was generated for further study.
Two types of potential visual warning icons were examined, cautionary and emergency. Figure 3.42 illustrates the various types of cautionary-level warnings that could be used to signify a potential front-end collision is possible. These warnings would appear in amber. Warnings A and D are currently being used in the DE/AED "Gold Car", and warnings B and C were gathered from the UC-Davis study results. Figure 3.43 illustrates the various types of emergency-level warnings that could be used to signify that a front end collision is imminent if no action is taken. These warnings would appear in red. Warnings A and E are currently being used in the DE/AED "Gold Car", and warnings B, C and D were gathered from the UC-Davis results. We added investigation of warning B, since it is very similar to the "Gold Car" icon.
In addition to the visual warning icons, two types of potential auditory warnings were also included for future examination, tone and voice. From the UC-Davis study, the top auditory warning tone and an additional tone used in the DE/AED demonstration vehicles will be further studied. Voice auditory warnings from the UC-Davis study and the DE/AED demonstration vehicles were also selected for further study. These included:
Using the results obtained from the UC-Davis study, DE/AED conducted further studies to finalize the warnings that will be used during the Task 6.3 simulator studies. To accomplish this, the study determined which visual warning(s) were understandable. In "real-world" applications, the visual warning will need to be quickly identified and understood. For instance, the visual warning may be paired with auditory tones, and/or proprioceptic warnings or it may be presented alone. In such cases, auditory tones and proprioceptic warnings will not provide a clear indication of the warning situation. Therefore, the visual warning will need to provide enough information to cue the driver to the potential hazard. To gather this information, the icons were presented to a group of subjects, and they were asked to rank order the warnings based on how `immediate they understand the warning' if it was presented on a HUD. From these results, the most understandable warnings were determined.
Subsequent to this study, further rapid prototyping work will be conducted in the simulator. The most understandable visual icons were incorporated into the simulator along with the auditory warnings. Subjects were asked to drive the simulator for a short period of time, during which combinations of the visual and auditory warnings were presented. Subjects were then asked to provide subjective feedback on the visual and auditory warnings that they experienced through a set of questions similar to those posed during the UC-Davis study. Questions on the warnings overall utility, attention-getting ability, urgency, annoyance, communication of type of collision, and what action should be taken, were all gathered.
From results gathered during the DE/AED experiment, a final set of visual and auditory warnings was chosen for use in the simulator. The visual icon that was chosen was icon C in Figure 3.42, while the auditory tone was simply a tone with no voice message used. Both of these warnings were then successfully integrated into the HDS architecture for use during the Task 6.3 studies.
During the performance of the work required to complete Task 6.1 all of the objectives that were desired were met. However, a possible deficiency may have occurred because the warning icons may not be the most recognizable during time critical situations. This is due to the subjects being tested under none critical situations where they were able to think more clearly about the icons they were viewing. Even with this problem, the following major accomplishments and program objectives were realized:
One of the drawbacks of the study conducted during Task 6.1 was that the subjects that were initially tested were not facing critical decisions that were time limited. Therefore, they were allowed to take their time when reviewing the information that was presented to them. In a real driving situation where a warning is being presented to the driver, the driver does not have the luxury to think about what the warning means, instead, they must be able to recognize it and respond as quickly as possible. Therefore, additional work needs to be conducted to see if subjects can "blindly" recognize the warnings that they are given. In addition, work needs to be performed in order to determine the warning cues that are best at conveying the necessary information to the driver when the driver has had no training and knows nothing about the system.