Challenges of Lane Detection Using Deep Neural Networks in Severe Heavy Rain: A Synthetic Evaluation Dataset Based on the CARLA Simulator

Hyeonjae Jeon; Seongjeong Park; Sungho Son; Jungki Lee; Jinung An; Gyeungho Choi; Yongseob Lim

doi:10.22680/kasa2024.16.4.092

All Issue

2024 Vol.16, Issue 4 Preview Page

Challenges of Lane Detection Using Deep Neural Networks in Severe Heavy Rain: A Synthetic Evaluation Dataset Based on the CARLA Simulator 국CARLA 시뮬레이터 기반 합성 평가 데이터셋을 활용한 극한 폭우 상황에서의 심층 신경망을 이용한 차선 인식 성능 평가문제목

31 December 2024. pp. 92-101

PDF XML

Abstract

Autonomous driving technology nowadays targets to level 4 or beyond, but the researchers are faced with some limitations for developing reliable driving algorithms in diverse challenges. To promote the autonomous vehicles to spread widely, it is important to properly deal with the safety issues on this technology. Among various safety concerns, the sensor blockage problem by severe weather conditions can be one of the most frequent threats for lane de-tection algorithms during autonomous driving. To handle this problem, the importance of the generation of proper datasets is becoming more significant. In this paper, a synthetic lane dataset with sensor blockage is suggested in the format of lane detection evaluation. Rain streaks for each frame were made by an experimentally established equation. Using this dataset, the degradation of the diverse lane detection methods has been verified. The trend of the per-formance degradation of deep neural network-based lane detection methods has been analyzed in depth. Finally, the limitation and the future directions of the network-based methods were presented.

Keywords

Autonomous vehicles(자율주행차)

Artificial intelligence(인공지능)

Lane detection(차선 인식)

Performance benchmarking(성능 벤치마킹)

References

F. Hauer, T. Schmidt, B. Holzmüller, and A. Pretschner, "Did We Test All Scenarios for Automated and Autonomous Driving Systems?," IEEE Intelligent Transportation Systems Conference (ITSC), 2019, pp. 2950~2955, doi: 10.1109/ITSC.2019.8917326.

10.1109/ITSC.2019.8917326

S. Ionita, "Autonomous vehicles: From paradigms to technology," IOP Conf. Ser.: Mater. Sci. Eng., Vol. 252, No. 1, Paper 012098, Oct. 2017.

10.1088/1757-899X/252/1/012098

C. Gkartzonikas and K. Gkritza, "What have we learned? A review of stated preference and choice studies on autonomous vehicles," Transp. Res. Part C: Emerg. Technol., Vol. 98, pp. 323~337, 2019.

10.1016/j.trc.2018.12.003

D. J. Yeong, G. Velasco-Hernandez, J. Barry, and J. Walsh, "Sensor and sensor fusion technology in autonomous vehicles: A review," Sensors, Vol. 21, No. 6, 2021, Art. No. 2140.

10.3390/s2106214033803889PMC8003231

P. Jing, G. Xu, Y. Chen, Y. Shi, and F. Zhan, "The determinants behind the acceptance of autonomous vehicles: A systematic review," Sustainability, Vol. 12, No. 5, 2020, Art. No. 1719.

10.3390/su12051719

K. Othman, "Public acceptance and perception of autonomous vehicles: A comprehensive review," AI Ethics, Vol. 1, No. 3, pp. 355~387, 2021.

10.1007/s43681-021-00041-834790943PMC7908960

J. Cui, L. S. Liew, G. Sabaliauskaite, and F. Zhou, "A review on safety failures, security attacks, and available countermeasures for autonomous vehicles," Ad Hoc Netw., Vol. 90, 2019, Art. No. 101823.

10.1016/j.adhoc.2018.12.006

Y. Hamzeh, Z. El-Shair, and S. A. Rawashdeh, "Effect of adherent rain on vision-based object detection algorithms," SAE Int. J. Adv. Curr. Practices Mobility, Vol. 2, No. 6, pp. 3051~3059, 2020.

10.4271/2020-01-0104

M. Hnewa and H. Radha, "Object detection under rainy conditions for autonomous vehicles: A review of state-of-the-art and emerging techniques," IEEE Signal Processing Magazine, Vol. 38, No. 1, pp. 53~67, Jan. 2021, doi: 10.1109/MSP.2020.2984801.

10.1109/MSP.2020.2984801

N. Goberville et al., "Analysis of LiDAR and camera data in real world weather conditions for autonomous vehicle operations," SAE Int. J. Adv. Curr. Prac. Mobil., Vol. 2, pp. 2428~2434, 2020.

10.4271/2020-01-0093

K. Yoneda, N. Suganuma, R. Yanase, and M. Aldibaja, "Automated driving recognition technologies for adverse weather conditions," IATSS Res., Vol. 43, No. 4, pp. 253~262, 2019.

10.1016/j.iatssr.2019.11.005

L. Tang, Y. Shi, Q. He, A. W. Sadek, and C. Qiao, "Performance test of autonomous vehicle lidar sensors under different weather conditions," Transportation Research Record, Vol. 2674, No. 1, pp. 319~329, 2020.

10.1177/0361198120901681

J. P. Espineira, J. Robinson, J. Groenewald, P. H. Chan, and V. Donzella, "Realistic LiDAR with noise model for real-time testing of automated vehicles in a virtual environment," IEEE Sensors, Vol. 21, No. 8, pp. 9919~9926, Apr. 2021.

10.1109/JSEN.2021.3059310

G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez, "The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3234~3243, 2016.

10.1109/CVPR.2016.352

M. Wrenninge and J. Unger, "Synscapes: A photorealistic synthetic dataset for street scene parsing," arXiv preprint arXiv:1810.08705, 2018.

A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, "CARLA: An open urban driving simulator," in Proc. Conf. Robot Learn., 2017, pp. 1~16.

G. Rong et al., "LGSVL simulator: A high fidelity simulator for autonomous driving," in Proc. IEEE 23rd Int. Conf. Intell. Transp. Syst., 2020, pp. 1~6, doi: 10.1109/ITSC45102.2020.9294422.

10.1109/ITSC45102.2020.9294422

S. Shah, D. Dey, C. Lovett, and A. Kapoor, "Airsim: High-fidelity visual and physical simulation for autonomous vehicles," in Field and Service Robotics, Cham, Switzerland: Springer, 2018, pp. 621~635.

10.1007/978-3-319-67361-5_40

B. Yu and A. K. Jain, "Lane boundary detection using a multi-resolution hough transform," in Proceedings of the International Conference on Image Processing, Vol. 2, pp. 748~751, 1997.

10.1109/ICIP.1997.638604

Muthalagu, A. Bolimera, and V. Kalaichelvi, "Lane detection technique based on perspective transformation and histogram analysis for self-driving cars," Comput. Elect. Eng., Vol. 85, Jul. 2020, Art. No. 106653.

10.1016/j.compeleceng.2020.106653

X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang, "Spatial as deep: Spatial CNN for traffic scene understanding," in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, No. 1, 2018.

10.1609/aaai.v32i1.12301

T. Zheng, H. Fang, Y. Zhang, W. Tang, Z. Yang, H. Liu, and D. Cai, "Resa: Recurrent feature-shift aggregator for lane detection," in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, No. 4, pp. 3547~3554, 2021.

10.1609/aaai.v35i4.16469

Z. Qin, H. Wang, and X. Li, "Ultra fast structure-aware deep lane detection," in European Conference on Computer Vision, pp. 276~291, Springer, Cham, 2020.

10.1007/978-3-030-58586-0_17

L. Tabelini, R. Berriel, T. M. Paixao, C. Badue, A. F. De Souza, and T. Oliveira-Santos, "Keep your eyes on the lane: Real-time attention-guided lane detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 294~302, 2021.

10.1109/CVPR46437.2021.00036

H. Jeon et al., "CARLA Simulator-Based Evaluation Framework Development of Lane Detection Accuracy Performance Under Sensor Blockage Caused by Heavy Rain for Autonomous Vehicle," IEEE Robotics and Automation Letters, Vol. 7, No. 4, pp. 9977~9984, Oct. 2022, doi: 10.1109/LRA.2022.3192632.

10.1109/LRA.2022.3192632

DGIST-ARTIV, "How to use annotation_tool [annotation_main.py]," 2020. [Online]. Available: https://github.com/DGIST-ARTIV/VISION/tree/master/%EC%B0%A8%EC%84%A0/lane_annotation_tool.

D. Neven, B. De Branbandere, S. Georgoulis, M. Proesmans, and L. Van Gool, "Towards end-to-end lane detection: An instance segmentation approach," in 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 286~291, 2018, IEEE.

10.1109/IVS.2018.850054730590295

J. M. Snider, "Automatic steering methods for autonomous automobile path tracking," Robotics Institute, Pittsburgh, PA, USA, Tech. Rep. CMU-RITR-09-08, 2009.

Information

Publisher :Korean Auto-vehicle Safety Association
Publisher(Ko) :한국자동차모빌리티안전학회
Journal Title :Journal of Auto-vehicle Safety Association
Journal Title(Ko) :자동차안전학회지
Volume : 16
No :4
Pages :92-101
Received Date : 2024-10-14
Revised Date : 2024-12-10
Accepted Date : 2024-12-10
DOI :https://doi.org/10.22680/kasa2024.16.4.092

Journal of Auto-vehicle Safety Association ISSN:2005-9396(Print) 자동차안전학회지

All Issue