CNN-based Visual Localization for Autonomous Vehicles under Different Weather Conditions

Ghintab, Shahad S.; Hassan, Mohammed Y.

doi:10.30684/etj.2022.135917.1289

	CNN-based Visual Localization for Autonomous Vehicles under Different Weather Conditions
Engineering and Technology Journal
Article 9, Volume 41, Issue 2, February 2023, Pages 375-386 PDF (1 M)
Document Type: Research Paper
DOI: 10.30684/etj.2022.135917.1289
Authors
Shahad S. Ghintab^* ¹; Mohammed Y. Hassan²
¹University of Technology-Iraq, Baghdad, Iraq
²Professor, University of Technology-Iraq, Baghdad-Iraq
Abstract
Autonomous vehicles (AV) are expected to improve, transform, and revolutionize ground transportation. Previous techniques are dependent on localization employing pricey inaccuracy Global Positioning sensor. Furthermore, the performance loss is caused by drifting errors of Simultaneous Localization and Mapping. Regarding categorizing and analyzing texture, cameras are much more accessible and practical. This work contributes to obtaining high accuracy for AV localization and reducing errors in predicted positions. Based on the light, accurate, and robust proposed Convolutional Neural Network (CNN), it will scale down the computational complexity and shorten the training time. Considering various weather and time of day conditions such as bright sunny, hard rain noon, and wet cloudy noon with a vision-only system Red, Green, and Blue (RGB) low-cost camera sensors. To check the positional accuracy of the CNN, RGB images are combined with depth images using the IHS method. The k-Mean technique evaluates the similarity between a specific image and all street images to obtain precise coordinates. The Simulation findings demonstrate the superiority of the suggested technique for different weather conditions, which has an accuracy of up to 94.74% and a Mean Squared Error MSE in a distance of 0 meters, as opposed to [19], where the MSE in the projected position is 4.8 meters. Another indication of the proposed method's effectiveness is that it yielded reliable results when its validity was tested on images from a dataset that had not been trained.
Highlights
A Convolutional Neural Network (CNN) was developed for autonomous localization layer-by-layer in urban driving situations. To check the positional accuracy of the CNN, RGB images are combined with depth images using the IHS method. With an accuracy rate of 94.74%, the simulation results demonstrated the effectiveness of the suggested strategy. The Simulation findings demonstrate the superiority of the suggested technique for different weather conditions.
Keywords
Autonomous Vehicle; Localization; Deep learning; Different weather conditions; HIS, K-mean Algorithm

References
[1] S. Chen, B. Liu, C. Feng, C. Vallespi-Gonzalez, and C. Wellington, 3D Point Cloud Processing and Learning for Autonomous Driving, Mar. 2020, [Online]. Available: doi.org/10.48550/arXiv.2003.00601 [2] . Fayyad, M. A. Jaradat, D. Gruyer, and H. Najjaran, Deep Learning Sensor Fusion for Autonomous Vehicles Perception and Localization: A Review Deep Learning Sensor Fusion for Autonomous Vehicle Perception and Localization: A Review, 20 (2020) 4220. doi.org/10.3390/s20154220 [3] T. G. R. Reid et al., Localization Requirements for Autonomous Vehicles, SAE Int. J. Connect. Autom. Veh., 2 (2019) 173-190. doi.org/10.4271/12-02-03-0012 [4] A. L. Ballardini, S. Fontana, D. Cattaneo, M. Matteucci, and D. G. Sorrenti, Vehicle localization using 3D building models and point cloud matching, Sensors, 21 (2021) 5356. doi.org/10.3390/s21165356 [5] R. Pirník, M. Hruboš, D. Nemec, T. Mravec, and P. Božek, Integration of inertial sensor data into control of the mobile platform, Adv. Intell. Syst. Comput., 511 (2017) 271–282. doi: 10.1007/978-3-319-46535-7_21 [6] P. Božek, Y. Nikitin, P. Bezák, G. Fedorko, and M. Fabian, Increasing the Production System Productivity Using Inertial Navigation, Manuf. Technol., 15 (2015) 274-278. doi: 10.21062/ujep/x.2015/a/1213-2489/MT/15/3/274 [7] T. T. O. Takleh, N. A. Bakar, S. A. Rahman, R. Hamzah, and Z. A. Aziz, A brief survey on SLAM methods in autonomous vehicle, Int. J. Eng. Technol., 7 (2018) 38–43. doi: 10.14419/ijet.v7i4.27.22477 [8] A. Noureldin, A. El-Shafie, and M. Bayoumi, GPS/INS integration utilizing dynamic neural networks for vehicular navigation, Inf. Fusion, 12 (2011) 48–57. doi: 10.1016/j.inffus.2010.01.003 [9] H. U. Kim and T. S. Bae, Deep learning-based GNSS network-based real-time kinematic improvement for autonomous ground vehicle navigation, J. Sensors, 2019 (2019)8. doi: 10.1155/2019/3737265 [10] E. Parisotto, D. S. Chaplot, J. Zhang, and R. Salakhutdinov, Global Pose Estimation with an Attention-based Recurrent Network, Feb. 2018, [Online]. Available: doi.org/10.48550/arXiv.1802.06857 [11] D. Chen, Y. Yu, and X. Gao, Semi-Supervised Deep Learning Framework for Monocular Visual Odometry. [12] D. Eigen, C. Puhrsch, and R. Fergus, Depth map prediction from a single image using a multi-scale deep network, Adv. Neural Inf. Process. Syst., 3 (2014) 2366–2374. [13] J. Byun, M. Roh, K.-I. Na, J. C. Sohn, and S. Kim, LNAI 7508 - Navigation and Localization for Autonomous Vehicle at Road Intersections with Low-Cost Sensors, 2012. [14] S. Bag, Deep Learning Localization for Self-driving Cars, 2017. [Online]. Available: https://scholarworks.rit.edu/theses [15] S. Alzu’bi and Y. Jararweh, Data Fusion in Autonomous Vehicles Research, Literature Tracing from Imaginary Idea to Smart Surrounding Community, 2020 5th Int. Conf. Fog Mob. Edge Comput. FMEC. 2020, 306–311.doi: 10.1109/FMEC49853.2020.9144916 [16] . Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab, Deeper depth prediction with fully convolutional residual networks, Proc. - 2016 4th Int. Conf. 3D Vision, 3DV, 2016, 239–248. doi: 10.1109/3DV.2016.32 [17] G. Plastiras, C. Kyrkou, and T. Theocharides, Efficient convnet-based object detection for unmanned aerial vehicles by selective tile processing, ACM Int. Conf. Proceeding Ser., 2018, 1–6.doi:10.1145/3243394.3243692 [18] L. Alzubaidi et al., Review of deep learning: concepts, CNN architectures, challenges, applications, future directions, J. Big Data, 8 (2021). doi:10.1186/s40537-021-00444-8 [19] V. Vaquero Gomez, Lidar-Based Scene Understanding for Autonomous Driving Using Deep Learning. [Online]. Available: http://www.tdx.cat/?locale- [20] D. Mishra and B. Palkar, Image Fusion Techniques: A Review, Int. J. Comput. Appl., 130 (2015) 7–13.doi: 10.5120/ijca2015907084 [21] F. A. Al-Wassai, N. V. Kalyankar, and A. A. Al-Zuky, The IHS Transformations Based Image Fusion, 2 (2011),[Online]. Available: doi.org/10.48550/arXiv.1107.4396 [22] H. Atiyah and M. Y., Outdoor Localization in Mobile Robot with 3D LiDAR Based on Principal Component Analysis and K-Nearest Neighbors Algorithm, Eng. Technol. J., 39 (2021) 965–976. doi: 10.30684/etj.v39i6.2032 [23] M. Y. Hassan and G. Kothapalli, Comparison between neural network based PI and PID controllers, 2010 7th Int. Multi-Conference Syst. Signals Devices, SSD-10, no. May 2014, 2010, 1-6.doi:10.1109/SSD.2010.5585598 [24] B. Zhou, J. Liu, W. Sun, R. Chen, C. Tomlin, and Y. Yuan, pbSGD: Powered stochastic gradient descent methods for accelerated nonconvex optimization, IJCAI Int. Jt. Conf. Artif. Intell.,2021-Janua,2020,3258–3266.doi: 10.24963/ijcai.2020/451 [25] Q.-Y. Zhou, J. Park, and V. Koltun, Open3D: A Modern Library for 3D Data Processing. [Online]. Available: http://www.open3d. [26] S. Bag, Deep Learning Localization for Self-driving Cars, 2017. [Online]. Available: http://scholarworks.rit.edu/theses [27] Y. Wu, Y. Li, X. Ge, Y. Gao, and W. Qian, An Efficient Method for Calculating the Error Statistics of Block-Based Approximate Adders, IEEE Trans. Comput., 68 (2019) 21–38.doi: 10.1109/TC.2018.2859960
Statistics Article View: 129 PDF Download: 498