The focus of this task was to conduct closed course testing using human subjects driving a vehicle equipped with working warning systems. From the test data collected, the accuracy and reaction time of driver responses was to be evaluated. Two separate closed course tests were to be conducted using a side zone warning system and a forward zone warning system. In order to successfully complete the task, several different issues had to be addressed. First, a test vehicle had to be equipped with the desired warning systems. This feat was accomplished under Task 2.3, but a few additional modifications had to be made in order to use the vehicle during this portion of the project. Second, a suitable test track had to be procured and a potential subject population obtained. Finally, the closed course testing had to be conducted and all of the data that was obtained (both objective and subjective) had to be reduced, analyzed, and documented.
The actual closed course testing consisted of two separate and individual studies. The first study dealt with a side zone warning system that was designed to assist drivers making lane change maneuvers in traffic. The warning system provided information to the driver as to whether or not another vehicle was located in their blind zone as they tried to safely negotiate a lane change. The second study consisted of a forward zone warning system that was designed to detect potential head on collisions while traversing through traffic. The warning systems provided information to the driver warning of other vehicles that the driver could potentially collide with. During each phase of the study only a single warning system was used, and the subjects only needed to concentrate on the specific system being tested. Additionally, the scenarios were designed so that the focus was on the system being tested and the critical events that occurred were designed to specifically activate the intended system.
At the completion of Task 6.4 the following objectives were to be reached:
Before any of the actual closed course testing could be conducted a few issues had to be resolved so that the tests could be conducted safely. This entailed modifying the DE/AED "Gold Car", setting up the data collection protocols, obtaining a test track, and adjusting the critical events so that they could be used on the test track that was chosen.
The "Gold Car" is a Cadillac that is equipped with multiple warning systems. For our purposes it had to have both a side zone and forward zone warning system. The warning systems include the sensor, Multiple Target Tracker (MTT), Collision Avoidance Processor (CAP), and the final warnings that were presented to the subjects. The installation of the warning systems occurred on another task; however, a couple of additional modifications had to made. For safety, a hand brake was installed as part of the front passenger's seat. This was installed so that the experimenter that was riding in the car during testing could attempt to avoid potential collisions by applying the brakes. The other modification involved a control box that was designed so that the experimenter could easily control every component of the warning system.
During the initial phases of the Task 6.4 design, there was a concern involving changing the warning conditions during the actual test runs. The solution was to build a control box. The box tied directly into the computer system and allowed the experimenter to toggle various components of the system on and off. For example, the audio portion of the alarm could be activated so that the driver could hear it, or could be deactivated so that no auditory alert was presented. All of this could be controlled by simply setting a toggle switch to either the on or off position. As the test schedule proceeded, the experimenter could easily configure the next run without even leaving the passenger seat. This meant that the entire set of runs for each subject could be conducted without stopping the car (if that was what the subject desired).
The next critical issue that had to be overcome was the data collection. This was handled using an on board laptop computer system. The laptop computer was directly linked to the CAP using a serial communication connection. While the tests were being conducted, the information that was being collected and analyzed by the CAP was also stored on the laptop computer. Using a program that was supplied by DE/AED and another written by STI, the data could then be dissected and the critical information retrieved and stored into spreadsheets for subsequent data analysis using the statistical analysis program Statistica.
During the "Gold Car" configuration, a concurrent effort was being directed to find a test track that was suitable for conducting the closed course testing. This presented quite a problem because there were several definite characteristics that were required. Because of the nature of the events that would be occurring during the test runs, the track had to be sealed off so that no other traffic (besides our test vehicles) would be present. In addition, the track also had to have multiple lanes and long straight-aways. These constraints were necessary because there would be passing involved and vehicles interacting with the test vehicle at long distances. Several different test tracks were considered including the General Motor's (GM) Desert Proving Grounds in Mesa, Arizona, and the California Transportation Department's Automated Highway System test facility near San Diego, California. Unfortunately, due to high costs, security restrictions, and scheduling problems, neither of these facilities as well as others that were considered could be used. Finally, the use of the test track at the GM Technical Center was approved.
The test track at the GM Technical Center is approximately 2.6 miles long with roughly 1 and .8 mile straight sections that lead into wide curves on either end. The track was reserved for exclusive use (6 hours a day) for both of the tests that would be conducted. Finally, for the entire length of the test track, there were 2 lanes of traffic so that passing would not be a problem.
Having the testing located at the GM Technical Center also eliminated another potential problem, obtaining subjects. No matter where the testing was to be conducted, there were going to be problems with the subject population. These problems included having a large enough population so that the 24 required subjects could participate, scheduling the subjects so that they would fit into the proposed test schedule, and obtaining permission so that the subjects could be given access to the test facility. The GM test track eliminated these problems because the size of the work force at the Technical Center was large and subjects were able to easily schedule test runs around their work schedules, and they all had access to the facility. In addition, a GM representative was able to handle the entire subject scheduling on site, thus greatly reducing the workload that would be required if the scheduling was being conducted from across the country.
With the test track facility secured, the final step was adjusting the critical events that would be occurring during testing so that they would work on the test track. As part of the Task 6.3 effort, a test plan was designed and delivered to a review board. The review board approved the test plan and therefore no deviations from the plan would be acceptable. However, we were allowed to make the test plan fit the test track that was being used. The details of the final test plan can be found in the document "Task 6.3 - Vehicle Test Plan".
One of the objectives of the Task 6.3 simulation study was to determine the 2 warning systems that would subsequently be tested during the Task 6.4 Closed Course Testing portion of the project. This meant that the 2 best side warnings and 2 best forward warnings would be instrumented on the "Gold Car" and used during the tests. The systems that were tested were:
| Side Warning Systems: | Forward Warning Systems: |
| Visual icons in side mirrors | HUD + Tactile feel (seat shaker) |
| Visual icons in side mirrors + Audio | HUD + Tactile feel + Audio |
During the actual tests, a laptop computer was being used to collect various information about both the vehicle's and the subject's performance. All of this data was taken directly from the CAP computer at each frame time during the test run. In addition to this objective data, some subjective data was also obtained from the subjects. This data was collected in the form of a post test questionnaire. At the completion of all of their test runs, each subject was required to fill out a questionnaire regarding their impressions of the systems that they had just experienced. Most of the questions required the subjects to rank their impressions on a scale from 0 to 6. Other questions required a Yes or No response, and all of the questions allowed the subjects to add any comments that they felt were necessary. As a final piece of data, the subjects were also asked to choose which of the 2 systems that they experienced, was best. The questionnaires that were used can be found in the report, "Task 6.3 - Vehicle Test Plan".
Because of problems that were encountered when the Task 6.3 simulation studies were being conducted, the closed course testing could not be started until June of 1997. In order to conduct the closed course tests, the warning systems that would be tested had to be determined and this was one of the objectives of Task 6.3. Therefore, no closed course tests could be run until the simulation data was collected and analyzed. This unfortunate occurrence created a huge delay in the overall project schedule and was primarily responsible for the schedule running way over the initial estimates.
In hindsight, after all of the test were completed (both side and forward), it was easy to go back and notice several general problems that occurred during testing. The first and most important issue was safety. Although the reason the tests were conducted was to obtain data and driver feedback with the warning systems activated, we could not put the subjects in extreme danger and therefore the tests were less realistic than normal driving. There were 2 reasons for this, first we did not want any of the subjects or experimenters to get injured, and second since we only had one test vehicle we did not want to damage it and take the risk of not being able to conclude the testing. Therefore, the tests that were conducted had reduced realism in order that they would be conducted safely.
In order to comply with the desired safety requirements, only the test vehicle and one other vehicle that the test vehicle could interact with were on the test track. There was no defined secondary task to try and distract the subjects as they drove, allowing the subjects the ability to constantly check their mirrors and scan the entire field of view. This caused a perception problem with the subjects because although the warnings were given for the events that they experienced the overall experiment did not sufficiently mimic real life. The events themselves were reminiscent of real life events but they were conducted in a controlled environment and the subject's were able to partially anticipate the events. This in turn caused problems with the statistical data analysis because you never knew if the warning caused the subjects to respond or if simple visual observations resulted in their responses.
Subjects were able to anticipate events due in part to the test track that was used. At first the test track seemed fine because the types of events that were scripted to occur only required a continuous track, with at least 2 driving lanes and a long straightaway. However, in hindsight the number of lanes became a limiting constraint that reduced what we were able to do with the chase vehicle. For example, in the side zone study, the subject always knew which side the chase vehicle was going to be on and therefore could partially anticipate what was going to happen. By having at 3 lanes on the test track, the chase vehicle could be on either side and becomes harder to keep track of.
The side zone closed course tests were conducted during the last 3 weeks in June. A total of 24 subjects (22 male and 2 female) participated in the testing. Their ages ranged from 28 to 59 years with the majority of the subjects in the 40 to 50 year old category. Since gender and age effects were not being considered, the use of predominately middle aged male subjects was not a concern.
As is detailed in the vehicle test plan document, subjects were requested to complete 36 laps around the test track, with every 3 laps consisting of a different speed (40 or 50 mph) and warning condition (no warning, icons in mirror, or icons with audio). Each of these combinations was repeated once giving a total of 12 scenarios. As the subject drove the test track, a chase vehicle would continually move in and out of the test vehicle's blind zone, thus causing the warning system to activate. The system was activated 8 times during the 3 laps that comprised a scenario. Because a total of 36 laps were driven and the subject was required to complete the post testing questionnaire, a total of 3 hours was required to test each subject. Therefore, a total of 3 weeks were necessary to check out the vehicle and test track, and to conduct the entire study.
It is important to note that the warning system was not active during the side zone testing. There was a major concern that the test vehicle and chase vehicle would have to be extremely close in order to activate the system. Since there was only a single test vehicle, and the possibility for a mishap was high, a decision was made to activate the warnings manually using the control box instead of the live system. This was done by scripting the side scenarios so that a critical event (vehicle in the side zone) would occur at a certain time during the test run. During these critical occurrences, the chase vehicle would move close to the vehicle but not close enough so that if the test vehicle suddenly changed lanes the 2 vehicles would collide. At the same time, the experimenter inside the test vehicle would activate the warning. When the subject began changing lanes the experimenter would then deactivate the warning. This setup worked well and the testing occurred without any potential collisions occurring.
As part of the original plan, a mechanical side mirror was also going to be used during testing. In these cases the mirror would display an image of a car in the side mirror with and without the warning icon. The idea for this mirror came about during the safety discussions, however once the mirror was built and tested, a decision was made not to use it. There were 2 basic reasons for this decision. First, initial subjects that drove the test vehicle with the mirror did not think is was very realistic and therefore they ignored it. Second, in order to use the mechanical mirror, it would have to be continually switched with the real mirror, causing delays in an already tight test schedule and increasing the likelihood that something would break.
The actual tests were conducted with virtually no major problems. The only real problem that was encountered was getting all of the test runs completed during the 3-hour session. Because of some minor computer problems, and some subjects arriving late, not all of the subjects were able to drive the entire 12 scenarios. Every subject ran each of the 6 basic scenarios once, and then it was a matter of driving as many repeat runs as possible.
During each run data about the vehicle's response and the subject's reactions was collected. Using the objective data that was collected directly by the laptop computer, a multivariate analysis was performed to see if the warning conditions that the subjects were experiencing had any significant effect on the way they reacted. For the multivariate analysis the independent variables were the warning condition, the speed and the subjects. The dependent variables investigated were the maneuver time (time to start and complete the lane change), maximum steer angle during the lane change, maximum longitudinal acceleration, maximum lateral acceleration, and the maximum speed differential during the lane change. Since the side-Near Object Detection System (NODS) does not include the capability to measure range and range rate between vehicles, this information could not be analyzed. Furthermore, although reaction times were obtained (for steering and braking) they were not seriously analyzed because of the randomness of the subject's responses. For the side zone tests there was no clear threat and therefore different subjects did different things. For example some subjects were very aggressive and changed lanes quickly when the command was issued. This gives the impression of quick reaction times. However, other subjects continued to drive ahead until the side zone vehicle was clearly behind them and then changed lanes. The differences in these types of behavior make comparing reaction times very difficult.
A multivariate statistical analysis was performed on the data and showed that both the speed and warning independent variables had a significant influence on several dependent variables although the interaction of speed and warning only influenced the maximum steering angle variable. The plots in Figure 3.51 show that the visual plus auditory warning condition tends to give the best performance regardless of the speed condition.
Figure 3.51: Side Zone Mean Subject Responses
Additionally, the plots in Figure 3.51 provide some insight into what the subject's were doing. In the mean steer response plot, the subjects were inputting larger steering angles (potentially more dangerous) when they had no warnings as compared to when they were receiving warnings. Also, the differential speed plot (MAXV) shows that when the visual alert was combined with the audio alert, the subjects tended to increase their speed during the lane change, putting additional separation between the 2 vehicles and therefore allowing for a safer lane change maneuver. Overall, the visual icons combined with the auditory alert tend to give the best performance.
In addition to the multivariate analysis that was done on the objective data, analysis was also performed on the subjective data that was collected. At the conclusion of each subject's test set, they were required to fill out the post run questionnaire. This was done to obtain a subjective opinion on the warning systems that were experienced. Like the results of the simulation study, subjects perceived the systems to be effective at getting their attention, easy to sense, and good at presenting a sense of urgency. In addition to the system's effectiveness, subjects also found the systems to be annoying. Once again this was a similar response to the simulation study, however, in this case the annoyance factor was less than that reported during the simulation study. These results are shown in Figure 3.52.
Figure 3.52: Subject Perception of Side Zone Systems.
The subject responses indicated that there was a tradeoff between system effectiveness and system annoyance. This was the same as with the simulation study, but once again the annoyance factor was less in the closed course testing than it was in the simulation study. This is evident in the overall response where more than 70 percent of the subjects preferred the system that combined the visual and audio warnings in the same system.
Subject confidence in the systems was good with subjects responding that they had above average confidence that the systems would help prevent crashes from occurring. At the same time, only 30 percent of the subjects stated that they were relying on the systems. This was much lower than the 50 percent that was observed in the simulation study. Most importantly, every subject with the exception of 1, responded that if cost were not an issue, they would purchase a vehicle with a side zone warning system. This is a good indication that the subjects walked away with an overall positive attitude for the warning systems.
While filling out the questionnaires, subjects took the time to write in comments about the systems and the various components that they contained. The visual icons produced mostly positive responses because subjects associated the orange triangle icons with caution, making them readily understandable and easy to learn. A majority responded that it was an effective display that conveyed the necessary information in a clear concise manner. In addition, locating the icons in the side mirrors enforced good driving habits by forcing the driver to check the rear view mirror in order to see the icon. However, there were concerns about seeing the icons during bad weather or if something was in the passenger seat and thus blocking the view of the passenger side mirror.
As for the auditory tone, subjects stated that it was the most effective alert at getting their attention. This was the same result as was found during the simulation study. As with the simulation study, there was concern about the auditory tone becoming increasingly annoying as time went on and especially in heavy traffic conditions. Subjects responded that the system would have been better if the auditory alert was connected to the turn signal so that it would only activate when the turn signal was used (this was the way the simulation was configured).
Summing up the side zone closed course testing, several conclusions can be drawn:
The forward zone closed course tests were conducted during the last 2 weeks in August of 1997. A total of 24 subjects (22 male and 2 female) participated in the testing. Their ages ranged from 24 to 59 years with more than half being over 40 years old. Since gender and age effects were not being considered, the use of predominately middle aged male subjects was not a concern.
As was stated earlier, the warning conditions that were presented to the subjects consisted of a HUD with tactile feel, and a HUD with tactile feel and an auditory alert. The HUD that was used consisted of 4 different symbols that could appear in front of the driver and inside their field of view. The 4 symbols are shown in Figure 3.53 and consist of:
The auditory tone that was used had 2 separate components, a beeping sound during cautionary warning conditions and the word "BRAKE" continuously repeated during severe warning conditions. The auditory warnings were played until the threat no longer existed or until the subject pressed the brake pedal. The tactile feel was comprised of a seat shaker. Anytime a threat was detected the seat would shake. The same shaking intensity was used for both cautionary and severe warnings.
As is detailed in the vehicle test plan document, subjects were requested to complete 12 laps around the test track, with each lap consisting of a different speed (40 or 50 mph) and warning condition (no warning, HUD with tactile feel, or HUD with tactile feel and audio). This gives a total of 6 basic scenarios and each of these combinations was repeated once giving a total of 12 scenarios. As the subject drove around the test track, a chase vehicle was used to pull in front of the test vehicle in order to activate the warning system. When the vehicle pulled in front of the subject it would do one of 2 things, either coast along at a slower speed so that the test vehicle would eventually overtake it, or apply the brakes. In addition to these 2 critical events, there was a third event that involved a stationary target and placing the chase vehicle in the other lane to essentially force the subject towards the stationary target. In all cases, the subject was required to take whatever means was necessary to try and avoid a collision.
Unlike the side zone study where the actual sensors were not used, the forward zone study was conducted with a fully active warning system. There was no input from the experimenters except to instruct the subject on which alarm they would be experiencing, what speed they should be traveling at, and which lane they should be positioned in. On each individual lap around the test track, the subject was given a particular speed and warning condition and then would encounter all 3 of the critical events. Because the stationary target was fixed at a certain location on the track, the subjects always encountered it at the same location. The other 2 events (coasting and braking) were mixed in at different times to try and keep the subjects off guard. With the 12 laps plus completing the post test questionnaire, a total of a hour and a half was required to run each subject. Therefore, the testing could be conducted in as little as 6 days if the weather and schedule worked out correctly.
Figure 3.53: Forward Zone HUD Icons
The weather was a major concern because the test track was located in Michigan and the tests were being conducted in August. Since it was possible that the subjects may have to input large quick steering maneuvers and possibly hard braking, it would be far too dangerous to conduct any testing on a wet test track. Fortunately, we were only rained out on 1 day and the rest of the testing was conducted with little interference due to the weather.
The actual tests were conducted with virtually no major problems occurring and each subject was able to complete their entire set of runs. The only problem encountered was a computer-related problem when storing the data at the end of a session. Basically the computer was run continuously during each subject's session. At the end of the session the data was saved to the laptop's hard disk and copied to an external disk at the end of the day. The files would then be checked at the end of the day to see if data was there. For some unknown reason, when conducting the actual data analysis, several of the early subjects had data files that were corrupted. In most cases the data files contained a majority of the data collected, however in a couple of cases less than 30 percent of the data collected was useable. This was not detected in the field because the data at the beginning of the file was fine and passed the initial data checks.
During each run data, about the vehicle's response and the subject's reactions was collected. Using the objective data that was collected directly by the laptop computer, a multivariate analysis was performed to see if the warning conditions that the subjects were experiencing had any significant effect on the way they reacted. For the multivariate analysis the independent variables were the warning condition, the speed and the critical event. The dependent variables investigated were the minimum Time To Collision (TTC based on range and range rate from the CAP), minimum range between vehicles, brake response time, and steering response time. During the simulation testing we also looked at throttle response time, however, this was not available from the CAP so it was not evaluated.
A multivariate statistical analysis was performed on the data and showed that the critical event had a highly significant effect on the data and that neither the warning condition nor speed did. This can be seen in Figure 3.54 where plots of the mean TTC, mean minimum range between vehicles and mean steering reaction time are all shown for each critical event type. The plots show that each of the critical events had a different effect on the variables being plotted. We believe that the warning condition did not have an influence on the variables because of the event diversity and the ability of the subjects to anticipate the events and respond to them in a variety of ways. This was evident in the braking response times (not shown in Figure 3.54) where approximately half of the subjects either did not brake or began braking before the alert was given. This occurrence was independent of the critical event that was being experienced.
Like they did at the end of each side zone session, subjects filled out a questionnaire requesting their opinions about the warning systems that they had just experienced. The results from the questionnaire were similar to those from the simulation study and showed that there was a definite tradeoff between system effectiveness and system annoyance. Subjects thought that the systems were very effective at getting their attention, were very easy to sense and provided some sense of urgency. The only significant difference between these results and the simulation results was that in the simulation study, the subjects responded that the systems provided a better sense of urgency. This may be attributed to more complex situations in the simulator where there was more potential for danger.
Similar to the simulation study, the component that provided the most effective way of getting the subject's attention was the auditory alert. As with all the other testing (both side and forward, and simulator and closed course testing), subjects thought that the auditory alert was annoying and had the potential to become extremely annoying out in every day traffic. This is the case even though the subjects thought that the systems were very effective. These results are shown in Figure 3.55 where once again we see that the systems are rated very high in effectiveness but are also shown to be moderately annoying.
Figure 3.54: Mean Values of Selected Forward Zone Dependent Variables
Some of the other results that were obtained from the questionnaires included 85 percent of the subjects specifying that if cost were not an issue they would purchase a vehicle with a warning system on it. This was a definite increase over the simulation survey where 65 percent of the subjects stated that they would. In general the subjects stated that they had above average confidence that the systems would help prevent crashes from occurring, but only 20 percent of the subjects claimed that they were relying on the system which was almost identical to the simulation results. Finally, the warnings were less understandable then those presented in the simulation study but were still easy to learn and would probably require a minimal amount of training. The lower understandability is attributed to the warnings changing based on either cautionary or severe situations. In the simulator there was only a single warning for all critical conditions.
Finally, the subjects were asked to choose the best system of the 2 that they experienced. Half of the subjects preferred the system that combined the HUD with the tactile feel. A third preferred the system combining the HUD with the tactile feel and the auditory. The remaining said that they would prefer a system with only the HUD. This was not unexpected since over 70 percent of the subjects specifically commented about the HUD's effectiveness and how much they liked it. In fact this was the component that was best liked by the subjects. The main reason for this is because the warning was presented in their field of view, so they did not have to take their eyes off of the roadway scene and this provided more time for them to react. In addition, the HUD icons used in the closed course study were less complex and confusing then the icon used in the simulator and subjects perceived them to be informative and effective.
Subjects had mixed feelings about the tactile feel. Most of the positive responses indicated that it provided a clear message about what was going on and would be perfect if their eyes were not on the roadway. However, some subjects thought it was unnecessary and therefore distracting. Another major concern was how well it would work on rough roadways and would the driver be able to feel it.
As for the auditory tone, it was very effective at getting the subjects attention, but as the runs progressed it became increasingly annoying. This was especially true during false alarms. Subjects voiced concerns about the auditory alert in noisy areas such as construction zones and some also objected to the use of the word "BRAKE" for the severe warning. In these cases, subjects observed that braking might not necessarily be the correct response in all situations.
Figure 3.55: Subject Perception of Forward Zone Systems
A subject that has not really been discussed up to this point but is very important and therefore warrants its own special attention deals with false alarms. Over 80 percent of the subjects complained about false alarms that occurred during their test runs. This was by far the dominant response on the questionnaires. When we say false alarm what we mean is the system detecting something and warning the driver and the driver does not perceive it as a danger. During this testing there were several places along the test track were a false detection may have triggered the system to present a warning to the driver. For example, in one of the curves between the straight sections of the track there was a guardrail that caused a false alarm to be triggered. Even though the guardrail is a potential threat (if you go straight you will hit it) the driver sees it as a false alarm because they are steering through the curve and they do not perceive the guardrail as a threat. After the subjects received several of these false alarms, they started to become annoyed and this reduced the overall effectiveness of the systems because they are no longer sure if the system is reliable. As was stated above, this was the most dominant response and could cause drivers to not use the system. The general consensus was that the false alarms must be greatly reduced for these systems to ever become useful to the average driver. It should also be noted that negative responses to the false alarms were more prominent in the closed course testing than in the simulation study.
Summing up the forward zone closed course testing, several conclusions can be drawn:
During the performance of the work required to complete Task 6.4, several different problems and issues were encountered, and handled. The following major accomplishments and program objectives were realized:
In general, all of the desired goals for this task of the project were met. However, because of the safety issues that were raised and the limited capability to constrain the driver and force them to do what we wanted, the objective data that was collected did not provide overwhelming statistically significant results. As with the simulation study, this is a result of human subjects adapting to the situations that we were presenting and based on our initial expectations, less was accomplished with the data than was originally expected. However, once again, the subjective results that were obtained from the subjects provided incredible insight into the systems and should be considered a major benefit and accomplishment of this task.
Work needs to continue on improving the MTT and CAP so that false alarms can be minimized. With today's sensors being good enough to detect lots of objects in the vehicle's path, it is necessary to try and reduce the number of false detections that are presented to the driver.
The major emphasize of this study was to see how drivers reacted to a warning that they were given during dangerous driving situations. The data collection and reduction focused primarily on how the subject reacted and their overall opinions of the system's effectiveness and annoyance. Now that we have some idea in these areas, additional work needs to be performed to see if subjects can "blindly" recognize the warnings that they are given. In addition, work needs to be performed in order to determine the warning cues that are best at conveying the necessary information to the driver when the driver has had no training and knows nothing about the system.
Some of the data collection and critical event problems could definitely be improved if the driver were distracted in some way while driving during critical situations. By taking their eyes off the road, the driver is letting the warning system take over for them and thus a situation is created where the system does what is intended to do, provide assistance to a non-attentive driver. A better secondary task such as adjusting the radio, or having to do something in the back seat of the vehicle could accomplish this task. Any future controlled testing should examine the secondary task problem thoroughly before the tests are conducted. In addition to trying to take the driver's eyes off the road, adding additional interactive traffic and more lanes would greatly increase the realism of the task and force the driver to concentrate on more than just a single vehicle.
Since it is basically impossible to conduct every situation in a closed and controlled environment, placing test vehicles equipped with the warning systems out into every day real traffic situations would be the best course of action. This would allow subjects to evaluate the systems in a realistic way by taking them into heavy stop-and-go traffic, high speed traffic, over rough roadways and construction zones, and on curves and hills. By doing this, a true indication of effectiveness versus annoyance can be obtained.