Lane sensing is an essential component of a Forward Collision Warning System (FCWS). To assess threats correctly, the FCWS must know both the lane path and the host vehicle's position in the lane. This is especially important to prevent false alarms that would annoy and possibly confuse the driver. The primary objective of this task was to develop and demonstrate a robust, real-time lane-sensing system that determines the lane path and vehicle position in the lane on limited access highways. A secondary objective was to advance the state of the art in lane sensing technology.
Soon after the program began there were two major shifts in the project's underlying assumptions. These were motivated by the discoveries of: (a) There was no existing set of roadway image data to use for algorithm development and testing, and (b) There was no existing software that could be transferred from GM to ERIM to serve as a starting point for our efforts. To meet these challenges, ERIM added tasks to develop a data-collection system and to survey the technical literature in the field to create an algorithm resource. These tasks produced the following accomplishments.
To develop the Lane Sensing Module (LSM), ERIM created a suite of hardware and software that enabled the LSM research team to collect high-fidelity image data, perform controlled experiments, and quantitatively evaluate the results. A thorough literature search was also conducted, creating a searchable database of previous research and development, and compiled an extensive library of image-data sequences along with a modular library of software algorithm components. These tools enabled us to make substantial progress in advancing lane-sensing technology and give us the capability to continue that progress, through sound scientific investigation and the engineering of reliable solutions. No other organization in the world has equivalent capabilities.
Because of the delay in the acquisition of a radar subsystem, the integration of the LSM with the other equipment were not able to be carried out in a timely fashion. Therefore, the primary result of the lane-sensing task was a demonstration of the LSM as a stand-alone system.
At the outset of the task, a functional requirements specification for a lane-sensing module was prepared. This report defined preliminary requirements for the lane sensing function of the FCWS. Then, a preliminary system design for the LSM was created, with specifications for algorithm development and the real-time demonstration of the system.
A searchable database, Figure 3.39, was constructed using Lotus Notes GroupWare so the team could take advantage of previous research. A literature search and review was conducted and the results were scanned into the database using an optical character scanner. The database now contains reports on most of the important work that has been done in this field, along with reviewers' notes and comments.
Figure 3.39: Lane Sensing Requirement Database Process
ERIM also worked with GMR to review crash statistics as they relate to lane sensing. The relative frequency and severity of some types of crashes indicate a real safety need for improving lane sensing at night, especially in rural areas.
An analysis of the image sensor's functional requirements was key, since image quality is the limiting factor in system performance. Spectral, geometric, and sensitivity requirements were prepared. A spectral analysis of road materials, lane-marker paint, windshield glass, typical sources of illumination, and sensor response characteristics was prepared. This analysis was used to select the equipment for collecting training data for the algorithms and for choosing the sensor/filter combination for the demonstration system.
As was pointed out earlier, it was necessary to construct a video data acquisition system (VDAS). To begin this effort we had to define the requirements, including the camera, the storage subsystem, and the control unit. Literature from equipment vendors was gathered and analyzed, and a set of criteria was developed to rank the available products. Requirements were also defined for vehicle instrumentation to obtain motion and attitude information (roll, pitch, speed, etc.), and existing products were evaluated. The VDAS was used to collect a library of real-world roadway data, to both develop and test the lane-sensing algorithms. Later, the VDAS became the foundation of the demonstration system.
The software modules that implement the algorithm use techniques from ERIM and GM's existing technology, along with other researchers' approaches. Specific techniques were selected for each module based on expected performance and ease of implementation. The basic steps in the final algorithm, and the baseline techniques associated with them, are summarized in Figure 3.40.
Figure 3.40: Video Truthing Software
In addition, an interactive software package was assembled for Truthing the video data. The software enables an operator to locate the lane markers in sequences of recorded images. These locations were recorded and used later to score the performance of the algorithm. The software that automatically scored the algorithm was also developed by ERIM International, making it possible to perform large-scale, automatic testing of alternative algorithms and parameter sensitivity.
Training data for developing the lane sensing algorithms was collected first with existing ERIM International equipment and later with the VDAS.
The ACAS Consortium successfully demonstrated lane-sensing algorithms for the Department of Transportation sponsors on March 24, 1997. This demonstration was presented in ERIM's instrumented Vehicle Systems Test Bed van while driving on US 23. The remarkable aspect of this demonstration was that it was performed at night, Figure 3.41 using only the vehicle's headlights for illumination. This has never been done before. During the demonstration, vehicle state information (speed, roll, and pitch) was passed from sensors on the vehicle. The computer displayed an image of the road with the sensed lane markers highlighted, as shown in Figure 3.41.
Figure 3.41: Night Time Lane Sensing