Roadside perception simulation system for vehicle road cooperation

Love is parallel to the world 2021-08-08 15:25:36 阅读数:967

roadside perception simulation vehicle road


Roadside perception simulation system for vehicle road cooperation

  • Roadside perception simulation system for vehicle road cooperation
  • 1 Simulation system architecture
  • 2 Simulation scene construction
    • 2.1 Static environment static environment
    • 2.2 Dynamic traffic
    • 2.3 Roadside units
  • 3 Data acquisition and processing
    • 3.1 Simulate point cloud data generation
    • 3.2 Truth data generation and processing
    • 3.3 Simulation data output
  • 4 experiment
    • 4.1 Lidar installation height analysis
    • 4.2 Vehicle recognition based on simulated point cloud data
  • 5 Conclusion


Roadside perception simulation system for vehicle road cooperation

PDF Document address :

Intelligent transportation system (Intelligent Transport System, ITS) The safety and efficiency of road traffic can be effectively improved through artificial intelligence and information communication technology [1,2], At present, it has been widely recognized , It contains “ Smart cars ” and “ The way of wisdom ” Two parts . Car road cooperation is ITS The advanced stage of development , It is used to realize the communication between vehicles and between vehicles and roadside system , Enable the vehicle to better perceive the surrounding environment , Receive information about assisted driving , So that the road supervision department can deal with traffic accidents more effectively [3,4].

among , Roadside awareness is an important part of vehicle road collaborative application development , By deploying sensors on the roadside , The collected pavement information is processed through V2X Communication to the vehicle , Make the vehicle have the perception ability beyond the line of sight . in application , To achieve the best roadside perception , Different scenarios often require different RSU To configure , RSU The selection and installation of is a time-consuming and labor-consuming process , in addition , The identification of traffic participants is the core of roadside perception , The recognition algorithm based on machine learning needs a lot of label data , Manual tagging is verified to be an extremely inefficient way . With the continuous improvement of computer hardware performance in recent years , The application of simulation technology in the field of intelligent transportation has become a necessary means for all kinds of R & D institutions to accelerate the development process [5,6].

At present The simulation in the field of intelligent transportation mainly focuses on the verification of automatic driving algorithm , Car road coordination V2X Communications , On board sensor data acquisition And so on .

Gelbal Based on dSPACE Scalexio Systems and Carsim The simulation software constructs a hardware in the loop simulation system for the development of automatic driving algorithm [7], Amini A method based on virtual image synthesis and transformation is proposed , Data driven simulation tools , Research on end-to-end autopilot control strategy [8], Szendrei Based on SUMO A set of hardware in the loop (HWIL) is designed for rapid modeling and testing of vehicle road collaborative applications V2X Simulation architecture [9], Choudhury When the integration is built VISSIM、Matlab and NS3, be used for V2X Simulation test environment for protocol and application [10], Su Et al. Proposed a method using GPU Vehicle lidar simulation method for computing 3D object point cloud in virtual environment [11], Baidu uses real point cloud background combined with virtual traffic volume to simulate the way in which vehicle lidar perceives the virtual environment [12],Dworak And so on CARLA The simulation software simulates the laser radar to collect pure virtual point cloud data , By comparing with the data in the public test set , It is found that the simulated point cloud in the simulation environment can be used as a supplement to the real data [13].

From the perspective of analysis , At present, few people are involved in the simulation of roadside perception , However, as an application development of vehicle road cooperation, it is also indispensable , This paper will start with the simulation of roadside perception , This paper introduces the related system construction and two application cases on this basis .

1 Simulation system architecture

The classic autopilot simulation platform includes virtual scene 、 Dynamic case simulation 、 Sensor simulation 、 Vehicle dynamics simulation and other independent modules [14], Pictured 1 Shown . The simulation of roadside sensing focuses on the interaction between roadside sensors, vehicles and the environment , therefore , The biggest difference from the automatic driving simulation platform is that the sensor type is roadside sensor rather than on-board sensor , But in order to maximize the relevant characteristics of the real world , Still need to include a graphics engine 、 Physical engine and middleware system for communication with the outside world as the basic support , Pictured 2 Shown


The literature [15] from V2X、 Traffic flow 、 Non autonomous vehicles 、 sensor 、 Graphic rendering 、 This paper summarizes the current mainstream simulation software used in the field of intelligent transportation from the aspects of automatic driving vehicle dynamic model and so on , As shown in the table 1 Shown . among , TF Represents traffic flow , DM A driving model representing a non autonomous vehicle , SE Represents the sensor , VI Represents rendering quality , VD Represents the dynamic model of an autonomous vehicle . in addition , In the form , i Indicates that secondary development is required , o Indicates that there are no related functions , – – Very bad ,– Indicates poor , + It means better , ++ Very good .


From the figure 2 You know , The roadside sensing simulation system designed in this paper needs to focus on sensor simulation , Environment and traffic rendering , Vehicle dynamics simulation, etc , Through the table 1 The data analysis can meet these requirements Carla、LGSVL、Righthook、SCANeR、VTD as well as CarMaker, among Carla and LGSVL For open source software .

LGSVL It's based on the game engine Unity Developed a simulation software mainly used for automatic driving development and test , It supports simulation environments 、 Customization of sensors and communication content , chart 3 by LGSVL workflow [16], This article will be based on LGSVL Develop a simulation system for roadside perception . among , Develop a simulation environment suitable for roadside perception by using the custom scene function , Create roadside sensing unit by using the function of custom vehicle and sensor model , The user-defined communication content is used to realize the collection and transmission of roadside sensing data .


2 Simulation scene construction

Determining the simulation scenario is the premise of simulation test , The scene simulated in this paper is the driverless vehicle test field of the team , chart 4 Is its plan diagram , The main gate leads to the east gate L The main road and its surroundings are the key simulation areas of this time . The simulation scene construction of roadside perception simulation system includes static environment 、 Dynamic traffic 、 Roadside units, etc , The construction methods of simulation scenarios usually include building scenarios based on modeling software , Build a scene based on the completed game , Build scene based on augmented reality method , Generate scenes based on high-precision maps , This paper uses modeling software to build virtual scene , The modeling software is open source 3D Modeling software blender[17].


2.1 Static environment static environment

It mainly includes lanes for vehicles , The buildings in the scene , Green plants in the area 、 Street lamp, etc , These constitute the objective environment of the simulation scene , And it does not change with the change of other conditions in the simulation test process , adopt blender After modeling, through Unity The static environment of the simulation system is obtained after HD rendering, as shown in the figure 5 Shown .


2.2 Dynamic traffic

Dynamic traffic is the key component of the simulation test scenario , It mainly refers to the control system with dynamic characteristics in simulation 、 Traffic 、 People flow and other parts , Including traffic light simulation , Motor vehicle simulation , Pedestrian simulation, etc . Dynamic traffic simulation scene construction methods mainly include Construction of real traffic case data , Generalization construction based on real case data , And the construction of micro traffic simulation system . LGSVL The dynamic traffic model is constructed by micro simulation method , Built in map annotation tool is used to complete the creation of high-definition map in 3D environment , Based on high-definition map, the vehicle can drive according to the lane , Follow the traffic lights , The speed limit , Intersection decision-making and other functions . chart 6 For map marking on the driving lane .


LGSVL Built in rich vehicle models , Including hatchback , Hatchback , SUV, The jeep , truck , School bus, etc , Look by combining different colors , Dozens of vehicle models can be produced , It basically covers the common types of vehicles on the road . meanwhile , LGSVL Support customization and creation of more types of vehicles . chart 7 For the effect of adding a vehicle model in a static environment .


2.3 Roadside units

Roadside unit is the core component of vehicle road coordination , Responsible for collecting vehicle road information 、 Processing and transmission , It is also the key research object of roadside perception simulation system for vehicle road cooperation proposed in this paper . LGSVL As a simulation software mainly for automatic driving , It does not have the composition type of roadside unit , however LGSVL Support high customization of vehicle and sensor models , This paper uses LGSVL This function is used to create a roadside sensing unit for vehicle road cooperation .

Common roadside units include cameras 、 Laser radar 、 Millimeter wave radar 、 Industrial computer, etc , According to the actual situation in the park , Combine the roadside unit with the solar street lamp , stay blender The three-dimensional model of roadside unit is constructed in, as shown in the figure 8(a) Shown , stay LGSVL in , The loadable resources corresponding to roadside units are obtained by the same method as the new vehicle model , In the end in LGSVL Load the roadside unit and configure the corresponding sensor parameters, as shown in the figure 8(b) Shown .


3 Data acquisition and processing

Data acquisition is the essential use of roadside sensors , Depending on the sensor type , The contents and processing methods of data collection are also different , Such as The camera collects image information , Lidar collects three-dimensional point cloud data . Due to the high cost of lidar , And the post-processing of 3D data is more complex , Using simulation to simulate the physical characteristics of lidar and the corresponding data collection and processing has become an important supplement to the real road test . Taking roadside lidar as an example, this paper introduces the generation, processing and output process of its simulation data .

3.1 Simulate point cloud data generation

The idea of lidar simulation is to refer to the scanning mode of real lidar , Simulate the emission of each real radar ray , By intersecting with all objects in the scene , If there is an intersection within the maximum detection range of lidar , The corresponding point cloud coordinates are returned . Assume that the simulated lidar is L Line , The horizontal resolution is R, The horizontal scanning range is 360°, Get the number of rays emitted per frame N by :


By the type (1) Sum graph 9 You know , When the lidar frequency is high , When the environment in the scene is complex and the model is fine enough , The amount of calculation of intersection by simulating rays is very large , Take lidar as an example 64 Line , Horizontal resolution 0.4°, frequency 10 Hz For example , The lidar radiation emitted per second alone is as high as 576000 strip , On this basis, it is also necessary to traverse all object models in the scene except lidar for each ray . In order to achieve the effect of real-time simulation , Can use CPU Parallel or GPU The way of calculation to improve the calculation efficiency , LGSVL use GPU Calculate point cloud data .

Real point cloud data, in addition to location coordinates , Another key message is the reflection intensity , The reflection intensity mainly reflects the reflectivity of different physical materials to the near-infrared light used by lidar . therefore , The intensity value also needs to be considered when simulating point cloud data , LGSVL The metallicity and color values in the model material are obtained and normalized, and the value range is 0~255 Strength value between .

3.2 Truth data generation and processing

With simulated point cloud data , Generally, it also needs to cooperate with truth value data , Data set for model recognition training . The truth value data corresponds to the manual label data in the real data , The data content includes the position of the identifiable object 、 toward 、 Bounding box size 、 Speed 、 Type, etc , Different from the manual labeling process , Truth data is known relative to the simulation system , You only need to output the truth value data and point cloud data synchronously , Therefore, the efficiency of the output tag can be greatly improved . stay LGSVL Create a new truth data sensor in , The data type is Detected3DObject, Pictured 9 Shown , among , Id A sequence of objects identified within the same frame of data , Label Label objects , Position Is the position of the object , Rotation Orient the object toward , Scale Is the size of the object bounding box , LinearVelocity and AngularVelocity They are the linear velocity and angular velocity of the object . In order to realize the matching between truth data and point cloud data , It is necessary to keep the configuration parameters of the truth data sensor and the lidar sensor consistent , Such as position and posture 、 Effective range 、 Frequency, etc. .

Unity in , The attitude angle is represented by four elements , Pictured 10 Medium Rotation value , At the same time, the representation of the coordinate system is left-handed system , The label data commonly used for model training is expressed by Euler angle in right-hand coordinate system . Euler angle has 12 Kind of said , Each represents 12 A rotation order [18], In this paper ZYX Rotation order of . hypothesis Unity The attitude angle of the four elements in is expressed as quaternion=(x, y, z, w), Corresponding ZYX Euler angle is euler=(roll, pitch, yaw), Then there is a relationship between the two :


3.3 Simulation data output

LGSVL Support includes ROS, ROS2, CyberRT And so on , This paper is based on Rosbridge The communication realizes the output of simulated point cloud data and truth data . Rosbridge For the wrong ROS The program provides a way to use ROS Functional JSON API, Used to direct to ROS Send based on JSON Specification of commands [19]. Rosbridge Contains a WebSocket The server , Used with Web Browsers interact , Simulation system and ROS The communication between is shown in the figure 11 Shown .


Because the simulated point cloud data and the true value data are collected by different sensors , In order to realize the mutual matching of each frame file , This paper uses to obtain the current ROS Time is named as point cloud data and truth data of each frame , As the current ROS Time is, The point cloud data file collected at the corresponding time is saved as nm.pcd, The truth data file is nm.txt. Import the simulated point cloud data and truth data of the same frame into Rviz As shown in the figure 12 Shown .


4 experiment

4.1 Lidar installation height analysis

In reality , Due to the high cost of lidar , In the roadside layout, it is necessary to optimize the layout of lidar so that the effective coverage area of a single lidar can be used as much as possible . For single side arrangement only RSU On the road , Because the shapes of various vehicles are quite different , It is possible that the trolley is blocked by the crane , Thus, it poses a challenge to the over the horizon function provided by vehicle road cooperation , In order to reduce the blind area of lidar caused by large vehicle shielding , The simplest and most effective method is to increase the installation height of lidar . To obtain the minimum installation height of lidar, it is necessary to integrate the lidar parameters , Road environmental parameters , Test under various conditions such as vehicle parameters , It is unrealistic to pass a real road test , With the help of the roadside perception simulation system proposed in this paper, it can be completed simply and intuitively . In the experiments , The number of selected lines is 16 Line , The vertical angle is 30°, The effective distance is 120 m Laser radar of , The length of the cart is 10.5 m, Height 4.4 m, Trolley length 4.6 m, Height 1.4 m, By changing the lidar height and inclination value , get 6 The point cloud coverage of group lidar in the simulation environment is shown in the figure 13 Shown , It can be seen from the picture that , As the lidar altitude increases , The more likely the car is covered by point clouds , At the same time, in order to make the point cloud cover most of the lane , The tilt angle also needs to be increased , When the lidar altitude is 10 rice , The inclination angle is 55° when , The car can have better point cloud coverage .


4.2 Vehicle recognition based on simulated point cloud data

Compared with the two-dimensional image collected by camera, object recognition , Object recognition based on LIDAR point cloud data is not affected by ambient light , It has higher robustness , Therefore, it plays an important role in vehicle road coordination . Accordingly , Due to the huge amount of point cloud data in a single frame , Also use the method of deep learning , Recognition based on point cloud is more difficult than image recognition , Especially the process of making label data , It's extremely difficult to do it manually . Through the simulation system, a large number of label data can be generated quickly and accurately , However, whether the simulated data can replace the real data still needs to be verified by experiments .

This paper designs 4 Group experiments were carried out to verify , The first 1 The group was trained with real data and tested with real data , The first 2 The group was trained with simulated data and tested with simulated data , The first 3 The group was trained with real data and tested with simulated data , The first 4 The group was trained with simulated data and tested with real data , 4 The same training network was used in the experiment group , The data volume of training set and test set is calculated by 4:1 obtain , The final results are shown in table 2 Shown .


among , Precision For the accuracy of recognition , Relative to the samples detected in the test set , Recall For the recall rate , Relative to the entire test set , F1 score It is the harmonic average of accuracy rate and recall rate . From the table 2 It can be seen that , Whether it's testing simulated data with real data , Or simulated data to test real data , The final results show that all kinds of evaluation indicators can be close to pure real data , Thus we can see that , The simulated point cloud data output by the simulation system can better restore the characteristics of the real data .

5 Conclusion

With the rapid development of Intelligent Transportation , Simulation technology plays a more and more important role , Especially for automatic driving and vehicle road coordination, there have been many simulation applications and Research , However, few people are involved in roadside perception oriented simulation . This paper presents a roadside perception simulation system for vehicle road cooperation , The system is based on autopilot simulation software LGSVL Carry out secondary development and construction , The development content includes simulation environment , Roadside unit and data acquisition and communication , Finally, the application of the simulation system is explained through two experiments . In Experiment 1, the relationship between lidar height and road point cloud coverage is analyzed with the help of simulation environment , It can provide reference for the actual installation position of lidar , Experiment 2 compares the mutual verification results between the vehicle recognition model obtained from the point cloud data output in the simulation environment and the model obtained from the real data , It is concluded that the simulation of lidar and environment of the simulation system designed in this paper can restore the real situation to a high extent .

in addition , In this study , Since the lidar sensor and the truth data sensor are considered as separate individuals , There is a problem that complete synchronization cannot be achieved in time , There will be a slight gap between truth data and point cloud data in space , secondly , There are still some differences between the simulation environment and the real environment , Like green plants 、 Shed, etc , It leads to the situation that each index of cross validation in Experiment 2 is slightly lower than that of self validation , These will be mainly considered in the follow-up research . meanwhile , Exploring more application scenarios of roadside perception simulation system in vehicle road cooperation is also the direction of future research .


版权声明:本文为[Love is parallel to the world]所创,转载请带上原文链接,感谢。