HIT_Vanni 2022-08-06 06:35:48 阅读数:967

autopilotlidarcameramillimeterwave

【论文】详见知网链接、ELSEVIER链接、IEEE链接,【开源项目】详见github链接

【编译运行】详见DWD_sensor_fusion编译运行

- Driverless cars the rest of the multi sensor data fusion algorithm research
- 0. 主要成果
- 1.主要研究内容
- 2.Multi-sensor multi-target detection
- 3.Sensor coordinate transformation and multiple target tracking
- 4.Multisensor dynamic redundancy weight allocation algorithm research
- 4.1 Summary of dynamic weighting allocation algorithm
- 4.2 Considering the dynamic weight average variance of deviation distribution algorithm
- 4.3 Considering the variance rate of dynamic weighting allocation algorithm
- 4.4 Sensors exit decreasing dynamic weighting allocation and test
- 4.5 Weight normalized and weight distribution rule
- 4.6 The final weight allocation result

- 5. Based on dynamic redundancy weight of multi-sensor data fusion

1.Based on laser radar、摄像头、The multi-sensor redundancy of the millimeter wave radar system,Inspection end using laser radar objectCNN_SEG、Object detection cameraYOLOV4And generating millimeter wave objectBox-Muller算法,On track again for each sensor object USES the kalman filter combined with the Hungarian matchingSORT算法.

2.Based on kalman filter variance of multi-sensor redundancy weight distribution system,The system is picked up from the grind ofDWD算法,The system USES a consider the distribution of weight average variance of deviation and the variance of the average rate of change of the weight,And supplemented by weight drop sensor and sensor exit mechanism,Finally finally each sensor is obtained by weighting normalization weights,And check the weight distribution curve through the experiment,Can see the weight drop mechanism and sensor during exit mechanism

3.Based on multisensor dynamic redundancy weight perception of multi-sensor fusion system,该系统利用DWDAlgorithm for the weight of each sensor allocation effect on to provide a basis for the fusion of multi-sensor fusion end,A weighting sensor is higher,The sensor fusion results in the process of integration of the higher the decisive role,反之,The lower the weighting sensor,The sensor fusion results in the process of fusion of the lower the decisive role,If the weight sensor is zero,Namely the sensor exit.

1.Prior to redundant multi-sensor perceptual system set up：

Select the 3 d laser radar-毫米波雷达-Visual redundancy, prior to the multi-sensor perceptual system.激光雷达使用CNN_SEG算法,摄像头采用YOLOV4算法,Millimeter wave radar usingBox-Muller算法.

2.Multiple sensor coordinate transform data alignment problems：

USES the camera coordinate system on the basis of,Compared with the color camera coordinate transformation need rotation matrix and translation matrix,With these two matrices can be coordinate system of two sensor coordinate transformation,And other sensors, such as laser radar、Conversion of the coordinates of the millimeter wave radar to camera coordinates.

3.Sensor multi-target tracking state estimation problem：

The essence of the world's nonlinear,For intelligent vehicle,The perception of the outside world is so,Is very necessary to introduce the nonlinear state estimation method of intelligent vehicle in multiple target tracking,The associated good target for filter,Then the sensor target tracking state,In sensor tracking part,采用了SORT（Simple Online And Realtime Tracking）算法,The algorithm including the Hungarian matching data association and kalman filter tracking.

4.Multisensor forward redundancy weight distribution：

Through the variance of the average deviation of each sensors、Variance of the average rate of change,And give each sensor corresponding weight,To simulate the sensor fault,The whole system can still perception of target accurately,This paper developed a dynamic weighting allocation algorithm——DWD（Dynamic Weight Distribution）算法.

5.Multi-sensor target associated with information fusion problem：

For each sensor information fusion is made,For each sensor of the consistency of the same target detected by explain.Will some sensor fails or precision falling target to eliminate or reduce their impact on the fusion results.

CNN_SEGAlgorithm is adopted based on a bird's eye view overlooking the direction（Bird View, BV）Point cloud training of neural network is used to detect the obstacles.Its ability to identify the object types have six classes（car, bicycle, track,bus, pedestrian, sign and others）.Point cloud data is used in the testKitti开源数据.感谢知乎大神@MouJiaJunapolloPerception algorithm of analyticcnn_seg

cnnseg其实分为两步,The first step is to use a deep learning network forecast five layer information,center offset, objectness, positiveness, object height, class probability,The second step is to use the five layer information clustering[14].具体步骤如下：

根据objectness层信息,Detect obstacles grid point target

With images and point cloud to illustrateobjectness层的作用.objectness输出的是一个[0,1]的二维矩阵,With the raster map for point cloud（在cnn_seg中,The point cloud is constructed2.5D栅格,Each grid corresponding to the8维特征）.通过对objectnessConfidence in threshold（默认0.5）,就可将objectThe pixel detected.According to information below with point cloud reflection intensity are detected area.根据center offset层信息,To detect obstacles grid points clustering,By clustering goals

center offsetIs a two-dimensional matrix of two layers of,分别是x轴的offset和y轴的offset,The pair of layers together,Corresponding to each gridoffset,Length in the direction of the arrow and saidoffset.根据positiveness和object height层信息,The background as well as the high point in each point cloud cluster filtering

positiveness是一个filter信息.From the previous step actually can see,The point cloud cluster,There are a lot of background region.Through the statistics of each point cloud clusterpositiveness的均值,在经过阈值（默认0.1）,The clustering can filter out some background.

object height也是一个filter信息,Filter out the high point in grid.

- 根据class probability,Classifying each clustering goal,Get the final detection target

通过以上几步,cnnsegHave made the point cloud segmentation into point cloud cluster,Type are respectively cart,小车,行人,自行车,未知.Through the statistics of each class of each point cloud cluster probability value,Higher scores for their kind,For each point cloud clusters so as to realize the classification of,得到最终的target.

下图中,Said the car in red,Blue cart,Green said pedestrians.

The figure below shows this algorithm to detect the object,As shown in figure of square box,With the type of the object and the distance from the default of.

Below detection point cloud topic of output after,As you can see by the side of the road vehicle, and to drive a car.Another topic to detect target output,You can see the various types of object detection is right,至此,The part of the algorithm testing done.

This training adoptsXeon W-2150B/3.00GHz CPUAnd two2080Ti GPU进行训练,使用10000张Kitti数据集,共训练50000次,The training time about68小时,Training categories a total of5类（pedestrian、car、bicycle、truck、bus）The picture on the right for trainingloss值（公式如下）记录：

The trained weights fileAP（Average Precision）及mAP（mean Average Precision）Accuracy test,其结果如表2-1,Precision at a higher level：

类别 | AP（%） |
---|---|

Pedestrian | 89 |

Bicycle | 86 |

Car | 94 |

Truck | 78 |

Bus | 79 |

mAP | 85.2 |

This part of the algorithm is the detection effect of,Use of the input image asKittiThe left side of the data sets the topic of color camera,话题名为/left_color_cam,The test results,Orange box ascar,蓝色框为truck.由图中可以看出YOLOV4The effect of detection of various objects are relatively good. | |

Because the sensors to detect target is roughly in a gaussian distribution,So using the up and down or so four edges of laser radar target to join gaussian random number to deal with laser radar target,The results as the millimeter wave radar target,由于C++语言可以用randFunction to generate uniformly distributed random number,So this research intends to adoptBox-Muller算法[16],The algorithm can generate gaussian distribution according to the uniform distribution,具体算法见知网链接

经过测试,Decided to adopt the scope for(-30,30)的高斯分布,This requires an expectation for0,方差为10的高斯分布,即Y~N（0,10）.The actual output below,This distribution for an expectation for-1.891986493192650,方差为9.79255102687986的高斯分布.

Coordinate transformation method is adoptedGeiger, A等对kittiData sets coordinate transformation method,Eventually transform laser point cloud to camera coordinates,And generate the millimeter wave radar target.Coordinate change codegithub有.

Object scatter plot

Kalman filter module's main function is through kalman filter algorithm on to the next frame to the identified to target state prediction and optimal estimation.卡尔曼滤波过程：

SORT（Simple Online And Realtime Tracking）算法[19]Including Hungary matching and kalman filter.

First of all to show the Hungarian matching,上下两图分别为T1时刻和T2时刻,假设T1时刻成功跟踪了某个单个物体,ID为1,绘制物体跟踪BBox（紫色）.T2时刻物体检测BBox总共有3个（橙色）,预测T2时刻物体跟踪的BBox（紫色）有1个,解决紫色物体跟踪BBoxHow the object detection with orangeBBox关联的算法,就是SORTTracking algorithm to solve the core issue of.

经检验,SORTPart of the program implementationIOU匈牙利匹配,After the match the target location of the output is good for subsequent kalman filter.使用2D MOT 2015 benchmark dataset[20]Open source data set to test the module,Many objects matching index is as follows,Most of the object matching degree is higher,Matching module performance good

In the process of the weight,Study a dynamic weighting allocation（DWD,Dynamic Weight Distribution）算法,Set the initial value variance, first of all, for1,Check multiple object tracking good variance changes as shown in figure.Will track more than20Frame objects to screen out,According to these variance stable value of the object,By calculating arithmetic average to establish a stable value variance centerσ0=0.65514125,It is set as the initial value of variance,初始值σ0Can make the object of variance fast convergence.

In the process of testing for a long time,All the variances of the tracker output as follows,Figure appeared in some outliers,For these outliers,Should be in the design of variance threshold method eliminate.

After eliminating outliers,Find all the variance inσmax=0.6730585,σmin=0.639224范围内波动,And most of the points are concentrated inσ0The initial value position,So the maximum threshold is set toσmax,The minimum variance threshold is set toσmin.

The specific algorithm see paper知网链接

对于激光雷达、Millimeter wave radar and camera objects are processed using this method to,平均偏差越大,Means that the sensor detection object that there might be a problem,权重就越小.

The specific algorithm see paper知网链接

对于激光雷达、Millimeter wave radar and camera objects are processed using this method to,The greater the average rate of change,Means that the sensor detection object that there might be a problem,权重就越小.

Automatic weight distribution is primarily to test the system down and sensors exit mechanism does work,When a sensor in the detection and tracking effect is not good situation,The system should be to reduce the weight of the sensor of them,Serious when will make the sensor directly exit.By using large gaussian noise processing per frame to check whether the weight of each sensor will reduce or quit.

The above consider the variance of the average deviation of weight distribution, and considering the variation of the variance weight distribution,Two parts of each distribution0.5的权重,Normalization is to turn data into(0,1)之间的小数.This research adopts the weights of normalization method for zooming interval method,Zoom in interval method and the trans,The original longer values in the interval scale to(0, 1)Between the smaller after numerical.

According to the previous section described,According to whether sensors exit can get the following eight kinds of situation,OIs not out,X为退出：

DWDAlgorithm by more than the initial value setting、Considering variance average deviation of the weight、Considering variance average rate of change of the weight、Decreasing weight sensor and sensor exit mechanism、权重归一化,Get the final multi-sensor weight,另外,The system not only applies to three sensors' system,If you want to increase or decrease in perceived sensor,Only need to change the rules of sensor dynamic weighting system,总体思想不变,So the system has a strong ductile.

System most of the time in the weight of three sensors distribution,But the system also has some time on the weight distribution of two sensors,例如5-10帧、170-185帧附近,There are some time for the single sensor weight distribution,例如155帧附近、158帧附近、160帧附近、220帧附近、233帧附近,最特殊的,System also has carried on the weight distribution of three sensors exit at the same time,Due to the three sensors are exit weights allocation according to1/3平均分配,So average distribution situation is the three sensors are out of,例如163-170帧、212-216帧.

The specific algorithm see paper知网链接

下图中C31、L16、R17Become an object,C33、L17、R18Become an object.

下图中C14、L4、R4Successful fusion object as an object,C11、L5、R5Successful fusion as an object,In addition the frame is a special case,即C18Object only camera tracking success,But no laser radar and the millimeter wave radar for the object tracking,But testing box are output,So go for the fusion of the object is successful.

Using dynamic redundancy weight and do not use the dynamic redundancy weight directly compared fusion,Can successfully blended into the target number of frames significantly more,对比结果如下.Figure in the green for the use of dynamic weighting fusion results,Blue for not using the results of the dynamic weight,After using dynamic redundancy weight,Number of frames produce fusion object by the182帧提高到228帧,The data set to the total number of frames250帧;Fused object also has obvious improvement,Will object to the success of each frame fusion sum,Fusion successfully the number of objects by the827个提高到1089个,This shows to join dynamic redundancy weight of multi-sensor fusion has a positive role in,Can enhance the success rate of fusion.

【论文】详见知网链接、ELSEVIER链接、IEEE链接,【开源项目】详见github链接

版权声明：本文为[HIT_Vanni]所创，转载请带上原文链接，感谢。 https://car.inotgo.com/2022/218/202208060519211890.html

- 爱玛指挥官a808、爱玛壹号a130，都市潮男的“时尚宠儿”
- 预算20万左右，小鹏p5、深蓝sl03怎么选？
- 全新本田xr-v的“新颜王”落地，整个底盘包围非常粗犷
- 小鹏g9将于8月10日正式上市，配备智能驾驶辅助
- 全新标致408上市，内部采用参数化式的点阵式设计
- 低配版、中配版、高配版有什么区别？买车小白学会区分
- 东风日产全新纯电suvariya，颜值与实力并存
- 东风风行用场景定义汽车品类，风行游艇带来15万级家庭用车最优解
- 最新款丰田家用轿车亮相，外观颜值不输大众polo
- 海南各地新能源汽车补贴陆续出炉 最全汇总看这里
- The new Peugeot 408 is launched, and the interior adopts a parametric lattice design
- Subsidies for new energy vehicles in Hainan have been released one after another. The most complete summary is here.
- The latest Toyota family sedan unveiled, looks as good as the Volkswagen polo
- Dongfeng Fengxing uses scenarios to define car categories, and Fengxing Yachts brings the best solution for 150,000-class family vehicles
- Dongfeng nissan pure electric suvariya, new appearance level and strength
- What's the difference between the low-end, mid-range, and high-end versions?Buying a car, novices learn to distinguish
- 丰田凯美瑞外观渲染图曝光，装配了全新矩阵式LED头灯组
- 全球首款宝马变色概念车技惊四座，可变色相当拉风
- 新丰田兰德酷路泽气场足够强，有越野范的suv
- 爱驰u6play电气化时代，利用了大面积的黑色饰板
- 拓宽车用铝材前景，华南铝工业展开幕在即_搜狐汽车_搜狐网
- KFC Crazy Thursday---马拉车/二分+字符串哈希
- Aiways u6play electrified era, using a large area of black trim
- The aura of the new Toyota Land Cruiser is strong enough to have an off-road SUV
- The world's first BMW color-changing concept car is amazing
- Toyota Camry exterior renderings exposed, equipped with new matrix LED headlights
- KFC Crazy Thursday---horse cart/binary + string hash
- 使用wpf实时模拟现实车辆运行轨迹
- 跨界s长安欧尚x5第20万辆新车下线，主打年轻运动的风格
- 2023款川崎发布，红色涂装外围还有金色的线条搭配
- 英飞凌产品缺陷事件引发车用IGBT芯片市场震动
- 日产西玛将于2023年停产，顺应时代潮流
- 马自达cx-50海外售价曝光，或将下半年交付
- 走出吉利、独立上市的亿咖通：要像华为一样，打造自己的“问界”_搜狐汽车_搜狐网
- 全球缺“尿”，为何小小的尿素让韩国这么急_搜狐汽车_搜狐网
- Global shortage of "urine", why small urea let South Korea so nasty _ _ sohu cars sohu net
- Infineon's product defect incident caused a shock in the automotive IGBT chip market
- Going out of Geely and independently listed Yigatong: To be like Huawei, create your own "inquiry world"_Sohu Auto_Sohu.com
- Mazda cx-50 overseas price exposure, or will be delivered in the second half of the year
- Nissan Sima to end production in 2023, following the trend of the times
- 一汽-大众探岳x正式亮相，提升了科技感
- 解锁新成就，欧尚X5第20万辆下线_搜狐汽车_搜狐网
- 车长超5.2米，宽度超2米，续航700km，极氪MPV才是真旗舰！_搜狐汽车_搜狐网
- 叉车轮胎的种类有哪些？
- 手推车轮 trolley wheel
- Conductor over 5.2 m, width than 2 meters, the range of 700 km, extremely krypton MPV is a flagship indeed!_ sohu auto _ sohu net
- Unlocking new achievements, the 200,000th Auchan X5 rolls off the assembly line
- FAW-Volkswagen Tanyue X officially debuted, enhancing the sense of technology
- trolley wheel
- 案例推荐 | 支撑超万家停车场：科拓停车基于 Pulsar 的消息总线架构演进
- 强观察丨钠离子电池时代真的要来了吗？
- The types of forklift truck tires?
- Case recommendation | Supporting more than 10,000 parking lots: Ketuo Parking's evolution based on Pulsar's message bus architecture
- Strong observation丨Is the era of sodium-ion batteries really coming?
- 未来有机会转变为新能源车型，日产西玛将于2023年年中迎来停产
- 汉马科技：7月中重卡销量同比增长12.64%
- 东风日产7月份销量86474台，轩逸市场表现如何？
- 五菱星辰混动版“弹射起步”瞬移神器，百公里油耗4.6l
- 东风本田和广汽本田卖出17.38万台，一起来看看吧
- 营收、现金流双线大增！北汽蓝谷半年度财报释放诸多向好信号