Online Performance and Power Prediction for Edge TPU via Comprehensive Characterization

Yang Ni, Yeseong Kim, Tajana Rosing, Mohsen Imani

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

In this paper, we characterize and model the performance and power consumption of Edge TPU, which efficiently accelerates deep learning (DL) inference in a low-power environment. Systolic array, as a high throughput computation architecture, its usage in the edge excites our interest in its performance and power pattern. We perform an extensive study for various neural network settings and sizes using more than 10,000 DL models. Through comprehensive exploration, we profile which factors highly influence the inference time and power to run DL Models. We show our key remarks for the relation between the performance/power and DL model complexity to enable hardware-aware optimization and design decisions. For example, our measurement shows that energy/performance is not linearly-proportional to the number of MAC operations. In fact, as the computation and DL model size increase, the performance follows a stepped pattern. Hence, the accurate estimate should consider other features of DL models such as on-chip/off-chip memory usages. Based on the characterization, we propose a modeling framework, called PETET, which perform online predictions for the performance and power of Edge TPU. The proposed method automatically identifies the relationship of the performance, power, and memory usages to the DL model settings based on machine learning techniques.

Original languageEnglish
Title of host publicationProceedings of the 2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022
EditorsCristiana Bolchini, Ingrid Verbauwhede, Ioana Vatajelu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages612-615
Number of pages4
ISBN (Electronic)9783981926361
DOIs
StatePublished - 2022
Event2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022 - Virtual, Online, Belgium
Duration: 14 Mar 202223 Mar 2022

Publication series

NameProceedings of the 2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022

Conference

Conference2022 Design, Automation and Test in Europe Conference and Exhibition, DATE 2022
Country/TerritoryBelgium
CityVirtual, Online
Period14/03/2223/03/22

Bibliographical note

Publisher Copyright:
© 2022 EDAA.

Fingerprint

Dive into the research topics of 'Online Performance and Power Prediction for Edge TPU via Comprehensive Characterization'. Together they form a unique fingerprint.

Cite this