Realization of lane departure warning system based on machine vision
Abstract: At present, traffic accidents caused by lane departures on expressways have caused huge losses, so that the study of lane departure warning systems has become a hot spot in society. Based on DSP calculation and FPGA calculation, this paper constructs a vision-based lane departure warning system. In this paper, the framework and hardware implementation of the system are given, and the transmission, storage and processing of large-capacity data in the vision system are studied in detail based on the proposed system. Finally, based on the constructed system, the feasibility of the system is verified through a series of typical visual processing algorithms and software algorithms applied to lane deviation decision.
1 Introduction
Car driving is an action behavior involving three factors: driver, car and road. The driver feels the road information through a series of physiological organs such as his eyes, and makes a series of driving decisions by understanding and judging the road information and tracking the driving status of the car. However, due to a series of factors such as long-term driving, carelessness and other factors, as well as the driver's perception of road information and vehicle status, there are certain "blind spots", resulting in traffic accidents becoming a major safety hazard in society. This article starts from the three aspects of driver, car and road, and studies a set of car active safety systems that simulate the driving behavior of people to improve the safety of vehicle driving, and implement active safety technologies such as lane departure warning.
In machine vision-based systems, real-time performance has always been a major issue, especially for systems such as lane departure warning, the algorithm to be implemented covers the entire visual computing process from image preprocessing to high-level visual computing , Both the bottom layer can be highly parallel realization of large data volume convolution, grayscale transformation and other regularization algorithms, including mid-level feature extraction and other pattern recognition algorithms and high-level some visual processing methods to achieve the understanding of the road Wait. For a long time, the versatility and efficiency of embedded systems have always been a pair of contradictions that are difficult to reconcile [1]. How to construct a more general vision processing system has a higher level of parallel and serial algorithms at all levels of vision Implementation efficiency is a problem to be solved in this article. The emergence of high-speed DSP processing chips and large-scale reconfigurable chips provides us with a good solution.
A large-scale reconfigurable chip, such as FPGA, integrates a large-capacity gate system based on a look-up table structure, which is very suitable for constructing a fine-grained SIMD processing array or pipeline structure, which is typical for Figure 2 The underlying image processing algorithm [1] has natural adaptability. TI's C6000 series of DSP computing chips use a Harvard structure with 8 processing units inside, and also integrate a dedicated hardware multiplier, which is suitable for completing some serial visual processing algorithms. The calculation process is more complicated, such as road understanding, which has a better realization effect.
Based on the above analysis, this paper combines a high-speed DSP computing chip and a reconfigurable FPGA chip to construct a high-speed embedded vision processing system. FPGA is used as an auxiliary logic control and front-end processing module. It mainly completes the early image acquisition and some similar histograms. The high-density image processing work such as graph statistics and convolution can be flexibly configured on the hardware structure, and the underlying algorithm can be completed by implementing a series of parallel structures or pipeline structures to avoid the resources caused by DSP to complete these fine-grained algorithms. Waste and time loss; DSP as the main computing center, complete high-level serial-based visual processing tasks, such as road understanding and interpretation, lane deviation decision, etc., DSP's high-speed processing ability can better accomplish these Highly serialized algorithm. The basic block diagram of the system is shown in Figure 2.
2. System design and implementation
To realize the vision-based automotive active safety system of Figure 2, data storage and transmission is a major issue, especially for applications with very high real-time requirements such as automotive active safety. How to obtain the final judgment of road understanding and make the correct decision on whether the vehicle deviates from the original collected image data through a series of memory data movement and signal processing in the shortest time The main goal of the system. This article will introduce the system in detail according to the sequence of data flow in the system.
2.1, data acquisition module
In this system, image data is the main sensor signal, and obtaining high-quality image data is of great significance for later processing and function realization. However, for the actual application environment, due to the influence of weather, light and other factors, or even a series of special situations, it is difficult for general image sensors to have good robustness under various conditions. In order to better adapt to various road and weather conditions and ensure driving safety, we have adopted a CMOS digital camera chip suitable for road use to independently develop a visual acquisition module. The camera chip can be fully programmed and controlled through 159-bit SPI control words. Better adjustment of exposure time, gain and noise. At the same time, the camera chip adopts a multi-slope exposure mode output. In different exposure modes, the exposure time is also different, to ensure that when the lighting is too strong or the lighting conditions are very weak, you can get more Good image.
For the SPI control of the camera chip, in order to adaptively adjust the working mode under different backgrounds, the DSP will give the FPGA a certain communication signal to complete the system level of the top layer of the camera chip according to the final understanding and judgment of the image. Control, and the modification and adjustment of the underlying real SPI control word is done by the FPGA, which can avoid the inconsistency of the working frequency of the SPI and the frequency of the external bus of the DSP and cause the waste of DSP work efficiency, and the FPGA to complete such work Is easier.
2.2, data transmission, storage module
In this system, from the front-end image acquisition module to the final output of the alarm signal, the data is continuously transmitted and moved between FPGA, SDRAM and DSP. Next, we will mainly discuss how to proceed between these three. High-speed and effective moving and processing are the main modules of this article. The data flow in the system mainly has the following aspects:
I. The pre-processed image data in FPGA must first be stored in the main memory chip SDRAM, which is the basis of the high-end back-end processing.
â…¡. DSP reads the latest image data from SDRAM for processing, and stores the intermediate results in SDRAM. Such read and write processes are likely to be cyclic. The block storage characteristics of SDRAM are suitable for such performance requirements. .
â…¢. For the convenience of system debugging, it is very likely that the results of intermediate processing are required to be displayed. Therefore, we attach a display debugging module to the FPGA. During debugging, the FPGA can read the intermediate processing results from the SDRAM for display.
â…£. There is another case. In some cases, we may require the image data to be directly transmitted between the DSP and the FPGA. This is also a category that we must consider.
Based on the above various possible situations, considering that the DSP chip we use supports both FIFO and SDRAM chip seamless connection, we designed the data transmission scheme of Figure 3:
First of all, the image data output by the front-end acquisition module is 8-bit, and for the SDRAM memory chip and DSP processing chip we use, the data bus is 64-bit. In order not to cause waste of resources, in the process of FPGA preprocessing the image, we simultaneously stitch 8-bit image data into 64 data outputs, so that the system resources can be fully utilized.
Secondly, a series of parallel pre-processed image data in FPGA is stored in FIFO developed inside FPGA. The FPGA gives a handshake signal to inform the DSP that data transfer from the FIFO to the SDRAM can begin, and the data transmission process is completely controlled and completed by the DSP through the EDMA transmission method. When the core processor DSP completes the current operation, the latest data is read from the SDRAM for processing, which can make full use of the rich peripheral resources of the DSP chip.
At the same time, from the data flow diagram in Figure 3, we find that in the process of data transmission from FPGA to SDRAM, compared with DSP, the FIFO module developed in FPGA for storing data and the external main memory SDRAM module are connected to DSP On the same address bus and data bus, on this type of transmission where the source device and the destination device are connected to the same external bus, the DSP chip we use supports an optimized EDMA transmission mode-PDT transmission, which The data transmission process is optimized, so that the EDMA transmission mode of the DSP can be directly transferred from the source device FIFO to the destination device SDRAM without passing through the buffer area in the DSP, so that the data transmission can be completed in one cycle. Compared with the traditional transmission mode, the FPGA is first transferred to the internal buffer area of ​​the DSP, and then written into the SDRAM by the DSP. The PDT transmission mode can basically shorten the data transmission time by half. For large-capacity data transmission such as images It is of great significance.
The data transfer process from the front-end pre-processing module to the main memory SDRAM is completed, followed by the high-speed data transmission process of the back-end. The data transmission between DSP and SDRAM can reach a high speed rate of more than 100MHz, especially the EDMA transmission method supported by DSP, which allows the transmission of external data and the internal Cache of DSP without CPU interference. This can improve the efficiency of CPU work. In addition, there are two levels of cache inside the DSP chip, and the second level cache L2 can reach 1MB, so we consider using the popular ping-pong transmission method. When the CPU is processing ping or pong data in the cache, Under the premise that the remaining processing time can be predicted, we can transfer the data to be processed next to the corresponding pong or ping storage area by EDMA in advance, so that the data transmission time is contained in the processing process, saving The corresponding transmission time improves efficiency.
Combining the above analysis of the entire data transmission process, we can derive the data movement process shown in Figure 4. It can be seen from the figure that the data from the Sensor is preprocessed in the FPGA, and the data processing process in the DSP can be transmitted in a pipelined manner under the SDRAM cache processing. FPGA is used as the front-end preprocessing module The completion of the processing of large-capacity data has a very large effect on the improvement of system efficiency.
There are many introductory articles about the front-end image preprocessing method and lane departure decision algorithm [2] [3] [4]. In this article, we will not do too much description. The main implementation process is shown in Figure 5. Through some typical algorithms in [2], we verify that the automotive active safety system designed in this paper fully meets the real-time requirements under the application conditions of automobile driving, and the system is highly scalable.
4 Conclusion
This article introduces in detail the implementation of a lane departure warning system based on DSP calculation and FPGA logic calculation, which mainly solves the high-speed transmission, storage and processing of large-capacity image data in the system. In the design process, this paper makes full use of the adaptability of the logic resources of FPGA to the underlying vision algorithm and the high-speed processing capability of the DSP chip to meet the high-level serial vision algorithm. Through a series of experiments, it was verified that the system meets the high real-time requirements in the context of automotive safety.
3. System software algorithm
Airis Vape,Airis Aura Disposable Vape,Airis Aura Vape Pen Device,Airis Aura Disposable E-Cigarette
Shenzhen Uscool Technology Co., Ltd , https://www.uscoolvape.com