A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity

Seongjeong Park; Jinu Pahk; Lennart Lorenz Freimuth Jahn; Yongseob Lim; Jinung An; Gyeungho Choi

doi:10.1007/978-3-031-44851-5_16

A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity

Seongjeong Park
, Jinu Pahk
, Lennart Lorenz Freimuth Jahn
, Yongseob Lim
, Jinung An
, Gyeungho Choi

Daegu Gyeongbuk Institute of Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Autonomous driving simulations require highly realistic images. Our preliminary study found that when the CARLA Simulator image was made more like reality by using DCLGAN, the performance of the lane recognition model improved to levels comparable to real-world driving. It was also confirmed that the vehicle’s ability to return to the center of the lane after deviating from it improved significantly. However, there is currently no agreed-upon metric for quantitatively evaluating the realism of simulation images. To address this issue, based on the idea that FID (Fréchet Inception Distance) measures the feature vector distribution distance using a pre-trained model, this paper proposes a metric that measures the similarity of simulation road images using the attention map from the self-attention distillation process of ENet-SAD. Finally, this paper verified the suitability of the measurement method by applying it to the image of the CARLA map that implemented a real-world autonomous driving test road.

Original language	English
Title of host publication	Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023
Editors	Soon-Geul Lee, Jinung An, Nak Young Chong, Marcus Strand, Joo H. Kim
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	203-212
Number of pages	10
ISBN (Print)	9783031448508
DOIs	https://doi.org/10.1007/978-3-031-44851-5_16
State	Published - 2024
Event	18th International Conference on Intelligent Autonomous Systems, IAS18 2023 - Suwon, Korea, Republic of Duration: 4 Jul 2023 → 7 Jul 2023

Publication series

Name	Lecture Notes in Networks and Systems
Volume	795
ISSN (Print)	2367-3370
ISSN (Electronic)	2367-3389

Conference

Conference	18th International Conference on Intelligent Autonomous Systems, IAS18 2023
Country/Territory	Korea, Republic of
City	Suwon
Period	4/07/23 → 7/07/23

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

Keywords

Attention Map
CARLA
DCLGAN
ENet-SAD
FID (Frechet Inception Distance)

Access to Document

10.1007/978-3-031-44851-5_16

Cite this

Park, S., Pahk, J., Jahn, L. L. F., Lim, Y., An, J., & Choi, G. (2024). A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity. In S.-G. Lee, J. An, N. Y. Chong, M. Strand, & J. H. Kim (Eds.), Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023 (pp. 203-212). (Lecture Notes in Networks and Systems; Vol. 795). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-44851-5_16

Park, Seongjeong ; Pahk, Jinu ; Jahn, Lennart Lorenz Freimuth et al. / A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity. Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023. editor / Soon-Geul Lee ; Jinung An ; Nak Young Chong ; Marcus Strand ; Joo H. Kim. Springer Science and Business Media Deutschland GmbH, 2024. pp. 203-212 (Lecture Notes in Networks and Systems).

@inproceedings{2946f1e572db4d75973c39d7eef8719c,

title = "A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity",

abstract = "Autonomous driving simulations require highly realistic images. Our preliminary study found that when the CARLA Simulator image was made more like reality by using DCLGAN, the performance of the lane recognition model improved to levels comparable to real-world driving. It was also confirmed that the vehicle{\textquoteright}s ability to return to the center of the lane after deviating from it improved significantly. However, there is currently no agreed-upon metric for quantitatively evaluating the realism of simulation images. To address this issue, based on the idea that FID (Fr{\'e}chet Inception Distance) measures the feature vector distribution distance using a pre-trained model, this paper proposes a metric that measures the similarity of simulation road images using the attention map from the self-attention distillation process of ENet-SAD. Finally, this paper verified the suitability of the measurement method by applying it to the image of the CARLA map that implemented a real-world autonomous driving test road.",

keywords = "Attention Map, CARLA, DCLGAN, ENet-SAD, FID (Frechet Inception Distance)",

author = "Seongjeong Park and Jinu Pahk and Jahn, \{Lennart Lorenz Freimuth\} and Yongseob Lim and Jinung An and Gyeungho Choi",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; 18th International Conference on Intelligent Autonomous Systems, IAS18 2023 ; Conference date: 04-07-2023 Through 07-07-2023",

year = "2024",

doi = "10.1007/978-3-031-44851-5\_16",

language = "English",

isbn = "9783031448508",

series = "Lecture Notes in Networks and Systems",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "203--212",

editor = "Soon-Geul Lee and Jinung An and Chong, \{Nak Young\} and Marcus Strand and Kim, \{Joo H.\}",

booktitle = "Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023",

}

Park, S, Pahk, J, Jahn, LLF, Lim, Y , An, J & Choi, G 2024, A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity. in S-G Lee, J An, NY Chong, M Strand & JH Kim (eds), Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023. Lecture Notes in Networks and Systems, vol. 795, Springer Science and Business Media Deutschland GmbH, pp. 203-212, 18th International Conference on Intelligent Autonomous Systems, IAS18 2023, Suwon, Korea, Republic of, 4/07/23. https://doi.org/10.1007/978-3-031-44851-5_16

A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity. / Park, Seongjeong; Pahk, Jinu; Jahn, Lennart Lorenz Freimuth et al.
Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023. ed. / Soon-Geul Lee; Jinung An; Nak Young Chong; Marcus Strand; Joo H. Kim. Springer Science and Business Media Deutschland GmbH, 2024. p. 203-212 (Lecture Notes in Networks and Systems; Vol. 795).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity

AU - Park, Seongjeong

AU - Pahk, Jinu

AU - Jahn, Lennart Lorenz Freimuth

AU - Lim, Yongseob

AU - An, Jinung

AU - Choi, Gyeungho

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024

Y1 - 2024

N2 - Autonomous driving simulations require highly realistic images. Our preliminary study found that when the CARLA Simulator image was made more like reality by using DCLGAN, the performance of the lane recognition model improved to levels comparable to real-world driving. It was also confirmed that the vehicle’s ability to return to the center of the lane after deviating from it improved significantly. However, there is currently no agreed-upon metric for quantitatively evaluating the realism of simulation images. To address this issue, based on the idea that FID (Fréchet Inception Distance) measures the feature vector distribution distance using a pre-trained model, this paper proposes a metric that measures the similarity of simulation road images using the attention map from the self-attention distillation process of ENet-SAD. Finally, this paper verified the suitability of the measurement method by applying it to the image of the CARLA map that implemented a real-world autonomous driving test road.

AB - Autonomous driving simulations require highly realistic images. Our preliminary study found that when the CARLA Simulator image was made more like reality by using DCLGAN, the performance of the lane recognition model improved to levels comparable to real-world driving. It was also confirmed that the vehicle’s ability to return to the center of the lane after deviating from it improved significantly. However, there is currently no agreed-upon metric for quantitatively evaluating the realism of simulation images. To address this issue, based on the idea that FID (Fréchet Inception Distance) measures the feature vector distribution distance using a pre-trained model, this paper proposes a metric that measures the similarity of simulation road images using the attention map from the self-attention distillation process of ENet-SAD. Finally, this paper verified the suitability of the measurement method by applying it to the image of the CARLA map that implemented a real-world autonomous driving test road.

KW - Attention Map

KW - CARLA

KW - DCLGAN

KW - ENet-SAD

KW - FID (Frechet Inception Distance)

UR - https://www.scopus.com/pages/publications/85192441580

U2 - 10.1007/978-3-031-44851-5_16

DO - 10.1007/978-3-031-44851-5_16

M3 - Conference contribution

AN - SCOPUS:85192441580

SN - 9783031448508

T3 - Lecture Notes in Networks and Systems

SP - 203

EP - 212

BT - Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023

A2 - Lee, Soon-Geul

A2 - An, Jinung

A2 - Chong, Nak Young

A2 - Strand, Marcus

A2 - Kim, Joo H.

PB - Springer Science and Business Media Deutschland GmbH

T2 - 18th International Conference on Intelligent Autonomous Systems, IAS18 2023

Y2 - 4 July 2023 through 7 July 2023

ER -

Park S, Pahk J, Jahn LLF, Lim Y , An J , Choi G. A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity. In Lee SG, An J, Chong NY, Strand M, Kim JH, editors, Intelligent Autonomous Systems 18 - Volume 1 Proceedings of the 18th International Conference IAS18-2023. Springer Science and Business Media Deutschland GmbH. 2024. p. 203-212. (Lecture Notes in Networks and Systems). doi: 10.1007/978-3-031-44851-5_16

A Study on Quantifying Sim2Real Image Gap in Autonomous Driving Simulations Using Lane Segmentation Attention Map Similarity

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this