Framework for Evaluating Vision-based Autonomous Steering Control Model

Recent advances in deep learning methods for visual perception tasks have contributed greatly to the development of algorithm models for autonomous driving. However, there is a lack of efforts to construct a dataset or to provide a fair evaluation method specialized for autonomous driving tasks. In this study, we constructed a dataset for the training and evaluation of an algorithm model for vision-based autonomous steering control (V-ASC). In addition, we developed a benchmark environment to analyze and provide qualitative and quantitative evaluation results. In particular, considering the characteristics of the V-ASC evaluation, it is possible to evaluate the accuracy of not only the prediction result of the steering value of each frame but also the autonomous driving result after the continuous frame change. We implemented a software-in-the-loop simulator (S-ILS) that provides a view-transformed image frame corresponding to the steering value change based on the actual vehicle's dynamic model and the camera sensor model. We also developed a baseline V-ASC model based on the handcrafted feature and the newly proposed convolutional neural network (CNN) based end-to-end driving model to verify the evaluation environment of the constructed dataset and simulator. The comparison between the two methods confirmed that the end-to-end CNN technique exhibits superior accuracy in tracking the ground-truth (GT) result based on the human driving result; further, this technique is also superior in terms of the autonomy result of the test-driving scenario.