SPET: Transparent SRAM Allocation and Model Partitioning for Real-time DNN Tasks on Edge TPU

Changhun Han; Hoon Sung Chwa; Kilho Lee; Sangeun Oh

doi:10.1109/DAC56929.2023.10247661

SPET: Transparent SRAM Allocation and Model Partitioning for Real-time DNN Tasks on Edge TPU

Changhun Han
, Hoon Sung Chwa
, Kilho Lee
, Sangeun Oh

Department of Electrical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Deep neural networks (DNNs) have been deployed in many safety-critical real-time embedded systems. To support DNN tasks in real-time, most previous studies focused on GPU or CPU. However, Edge TPU has not yet been studied for real-time guarantees. This paper presents a real-time DNNs framework for Edge TPU to satisfy multiple DNN inference tasks' timing requirements. The proposed framework provides 1) SRAM allocation and model partitioning techniques and 2) a MIP-based algorithm that determines the amount of SRAM and the number of segments for each task. The experiment result shows that our framework provides 79% higher schedulability than the existing Edge TPU system.

Original language	English
Title of host publication	2023 60th ACM/IEEE Design Automation Conference, DAC 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9798350323481
DOIs	https://doi.org/10.1109/DAC56929.2023.10247661
State	Published - 2023
Event	60th ACM/IEEE Design Automation Conference, DAC 2023 - San Francisco, United States Duration: 9 Jul 2023 → 13 Jul 2023

Publication series

Name	Proceedings - Design Automation Conference
Volume	2023-July
ISSN (Print)	0738-100X

Conference

Conference	60th ACM/IEEE Design Automation Conference, DAC 2023
Country/Territory	United States
City	San Francisco
Period	9/07/23 → 13/07/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Access to Document

10.1109/DAC56929.2023.10247661

Cite this

@inproceedings{0db2e3f5fbfa4b30a06c3017887ef772,

title = "SPET: Transparent SRAM Allocation and Model Partitioning for Real-time DNN Tasks on Edge TPU",

abstract = "Deep neural networks (DNNs) have been deployed in many safety-critical real-time embedded systems. To support DNN tasks in real-time, most previous studies focused on GPU or CPU. However, Edge TPU has not yet been studied for real-time guarantees. This paper presents a real-time DNNs framework for Edge TPU to satisfy multiple DNN inference tasks' timing requirements. The proposed framework provides 1) SRAM allocation and model partitioning techniques and 2) a MIP-based algorithm that determines the amount of SRAM and the number of segments for each task. The experiment result shows that our framework provides 79\% higher schedulability than the existing Edge TPU system.",

author = "Changhun Han and Chwa, \{Hoon Sung\} and Kilho Lee and Sangeun Oh",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 60th ACM/IEEE Design Automation Conference, DAC 2023 ; Conference date: 09-07-2023 Through 13-07-2023",

year = "2023",

doi = "10.1109/DAC56929.2023.10247661",

language = "English",

series = "Proceedings - Design Automation Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2023 60th ACM/IEEE Design Automation Conference, DAC 2023",

}

Han, C, Chwa, HS, Lee, K & Oh, S 2023, SPET: Transparent SRAM Allocation and Model Partitioning for Real-time DNN Tasks on Edge TPU. in 2023 60th ACM/IEEE Design Automation Conference, DAC 2023. Proceedings - Design Automation Conference, vol. 2023-July, Institute of Electrical and Electronics Engineers Inc., 60th ACM/IEEE Design Automation Conference, DAC 2023, San Francisco, United States, 9/07/23. https://doi.org/10.1109/DAC56929.2023.10247661

SPET: Transparent SRAM Allocation and Model Partitioning for Real-time DNN Tasks on Edge TPU. / Han, Changhun; Chwa, Hoon Sung; Lee, Kilho et al.
2023 60th ACM/IEEE Design Automation Conference, DAC 2023. Institute of Electrical and Electronics Engineers Inc., 2023. (Proceedings - Design Automation Conference; Vol. 2023-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - SPET

T2 - 60th ACM/IEEE Design Automation Conference, DAC 2023

AU - Han, Changhun

AU - Chwa, Hoon Sung

AU - Lee, Kilho

AU - Oh, Sangeun

PY - 2023

Y1 - 2023

N2 - Deep neural networks (DNNs) have been deployed in many safety-critical real-time embedded systems. To support DNN tasks in real-time, most previous studies focused on GPU or CPU. However, Edge TPU has not yet been studied for real-time guarantees. This paper presents a real-time DNNs framework for Edge TPU to satisfy multiple DNN inference tasks' timing requirements. The proposed framework provides 1) SRAM allocation and model partitioning techniques and 2) a MIP-based algorithm that determines the amount of SRAM and the number of segments for each task. The experiment result shows that our framework provides 79% higher schedulability than the existing Edge TPU system.

AB - Deep neural networks (DNNs) have been deployed in many safety-critical real-time embedded systems. To support DNN tasks in real-time, most previous studies focused on GPU or CPU. However, Edge TPU has not yet been studied for real-time guarantees. This paper presents a real-time DNNs framework for Edge TPU to satisfy multiple DNN inference tasks' timing requirements. The proposed framework provides 1) SRAM allocation and model partitioning techniques and 2) a MIP-based algorithm that determines the amount of SRAM and the number of segments for each task. The experiment result shows that our framework provides 79% higher schedulability than the existing Edge TPU system.

UR - https://www.scopus.com/pages/publications/85173109091

U2 - 10.1109/DAC56929.2023.10247661

DO - 10.1109/DAC56929.2023.10247661

M3 - Conference contribution

AN - SCOPUS:85173109091

T3 - Proceedings - Design Automation Conference

BT - 2023 60th ACM/IEEE Design Automation Conference, DAC 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 9 July 2023 through 13 July 2023

ER -

SPET: Transparent SRAM Allocation and Model Partitioning for Real-time DNN Tasks on Edge TPU

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this