Abstract
In this paper, we characterize and model the performance and power consumption of Edge TPU, which efficiently accelerates deep learning (DL) inference in a low-power environment. Systolic array, as a high throughput computation architecture, its usage in the edge excites our interest in its performance and power pattern. We perform an extensive study for various neural network settings and sizes using more than 10,000 DL models. Through comprehensive exploration, we profile which factors highly influence the inference time and power to run DL Models. We show our key remarks for the relation between the performance/power and DL model complexity to enable hardware-aware optimization and design decisions. For example, our measurement shows that energy/performance is not linearly-proportional to the number of MAC operations. In fact, as the computation and DL model size increase, the performance follows a stepped pattern. Hence, the accurate estimate should consider other features of DL models such as on-chip/off-chip memory usages. Based on the characterization, we propose a modeling framework, called PETET, which perform online predictions for the performance and power of Edge TPU. The proposed method automatically identifies the relationship of the performance, power, and memory usages to the DL model settings based on machine learning techniques.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022 |
Editors | Cristiana Bolchini, Ingrid Verbauwhede, Ioana Vatajelu |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 612-615 |
Number of pages | 4 |
ISBN (Electronic) | 9783981926361 |
DOIs | |
State | Published - 2022 |
Event | 2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022 - Virtual, Online, Belgium Duration: 14 Mar 2022 → 23 Mar 2022 |
Publication series
Name | Proceedings of the 2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022 |
---|
Conference
Conference | 2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022 |
---|---|
Country/Territory | Belgium |
City | Virtual, Online |
Period | 14/03/22 → 23/03/22 |
Bibliographical note
Publisher Copyright:© 2022 EDAA.