A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence

Jinhwi Kim; Apostolos Galanopoulos; Jude Vivek Joseph; Jeongho Kwak

doi:10.1109/ICTC49870.2020.9289270

A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence

Jinhwi Kim, Apostolos Galanopoulos, Jude Vivek Joseph, Jeongho Kwak

Department of Electrical Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

In this paper, we explore an impact of GPU/CPU scaling of a state-of-the-art AI embedded device on its energy consumption and AI performance. We use Nvidia Jetson TX2 as an experiment device thanks to its tractability to scale GPU/CPU and modify AI framework and libraries. Via extensive experiment in various ML (Machine Learning) scenarios, i.e., face recognition and objective detection, we demonstrate a clear tradeoff between GPU/CPU scaling, energy consumption (of GPU/CPU as well as entire device) and training/inference speed. Finally, we envision a future work aiming to optimize processing and networking resources simultaneously at an extended scenario that multiple AI embedded devices cooperate with each other for a common AI application.

Original language	English
Title of host publication	ICTC 2020 - 11th International Conference on ICT Convergence
Subtitle of host publication	Data, Network, and AI in the Age of Untact
Publisher	IEEE Computer Society
Pages	22-24
Number of pages	3
ISBN (Electronic)	9781728167589
DOIs	https://doi.org/10.1109/ICTC49870.2020.9289270
State	Published - 21 Oct 2020
Event	11th International Conference on Information and Communication Technology Convergence, ICTC 2020 - Jeju Island, Korea, Republic of Duration: 21 Oct 2020 → 23 Oct 2020

Publication series

Name	International Conference on ICT Convergence
Volume	2020-October
ISSN (Print)	2162-1233
ISSN (Electronic)	2162-1241

Conference

Conference	11th International Conference on Information and Communication Technology Convergence, ICTC 2020
Country/Territory	Korea, Republic of
City	Jeju Island
Period	21/10/20 → 23/10/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Keywords

GPU/CPU scaling
energy consumption
inference speed
training speed

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/ICTC49870.2020.9289270

Cite this

Kim, J., Galanopoulos, A., Vivek Joseph, J., & Kwak, J. (2020). A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence. In ICTC 2020 - 11th International Conference on ICT Convergence: Data, Network, and AI in the Age of Untact (pp. 22-24). Article 9289270 (International Conference on ICT Convergence; Vol. 2020-October). IEEE Computer Society. https://doi.org/10.1109/ICTC49870.2020.9289270

@inproceedings{dfa326fbca9b4e86bb255a454d83af67,

title = "A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence",

abstract = "In this paper, we explore an impact of GPU/CPU scaling of a state-of-the-art AI embedded device on its energy consumption and AI performance. We use Nvidia Jetson TX2 as an experiment device thanks to its tractability to scale GPU/CPU and modify AI framework and libraries. Via extensive experiment in various ML (Machine Learning) scenarios, i.e., face recognition and objective detection, we demonstrate a clear tradeoff between GPU/CPU scaling, energy consumption (of GPU/CPU as well as entire device) and training/inference speed. Finally, we envision a future work aiming to optimize processing and networking resources simultaneously at an extended scenario that multiple AI embedded devices cooperate with each other for a common AI application.",

keywords = "GPU/CPU scaling, energy consumption, inference speed, training speed",

author = "Jinhwi Kim and Apostolos Galanopoulos and {Vivek Joseph}, Jude and Jeongho Kwak",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 11th International Conference on Information and Communication Technology Convergence, ICTC 2020 ; Conference date: 21-10-2020 Through 23-10-2020",

year = "2020",

month = oct,

day = "21",

doi = "10.1109/ICTC49870.2020.9289270",

language = "English",

series = "International Conference on ICT Convergence",

publisher = "IEEE Computer Society",

pages = "22--24",

booktitle = "ICTC 2020 - 11th International Conference on ICT Convergence",

}

Kim, J, Galanopoulos, A, Vivek Joseph, J & Kwak, J 2020, A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence. in ICTC 2020 - 11th International Conference on ICT Convergence: Data, Network, and AI in the Age of Untact., 9289270, International Conference on ICT Convergence, vol. 2020-October, IEEE Computer Society, pp. 22-24, 11th International Conference on Information and Communication Technology Convergence, ICTC 2020, Jeju Island, Korea, Republic of, 21/10/20. https://doi.org/10.1109/ICTC49870.2020.9289270

A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence. / Kim, Jinhwi; Galanopoulos, Apostolos; Vivek Joseph, Jude et al.
ICTC 2020 - 11th International Conference on ICT Convergence: Data, Network, and AI in the Age of Untact. IEEE Computer Society, 2020. p. 22-24 9289270 (International Conference on ICT Convergence; Vol. 2020-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence

AU - Kim, Jinhwi

AU - Galanopoulos, Apostolos

AU - Vivek Joseph, Jude

AU - Kwak, Jeongho

PY - 2020/10/21

Y1 - 2020/10/21

N2 - In this paper, we explore an impact of GPU/CPU scaling of a state-of-the-art AI embedded device on its energy consumption and AI performance. We use Nvidia Jetson TX2 as an experiment device thanks to its tractability to scale GPU/CPU and modify AI framework and libraries. Via extensive experiment in various ML (Machine Learning) scenarios, i.e., face recognition and objective detection, we demonstrate a clear tradeoff between GPU/CPU scaling, energy consumption (of GPU/CPU as well as entire device) and training/inference speed. Finally, we envision a future work aiming to optimize processing and networking resources simultaneously at an extended scenario that multiple AI embedded devices cooperate with each other for a common AI application.

AB - In this paper, we explore an impact of GPU/CPU scaling of a state-of-the-art AI embedded device on its energy consumption and AI performance. We use Nvidia Jetson TX2 as an experiment device thanks to its tractability to scale GPU/CPU and modify AI framework and libraries. Via extensive experiment in various ML (Machine Learning) scenarios, i.e., face recognition and objective detection, we demonstrate a clear tradeoff between GPU/CPU scaling, energy consumption (of GPU/CPU as well as entire device) and training/inference speed. Finally, we envision a future work aiming to optimize processing and networking resources simultaneously at an extended scenario that multiple AI embedded devices cooperate with each other for a common AI application.

KW - GPU/CPU scaling

KW - energy consumption

KW - inference speed

KW - training speed

UR - http://www.scopus.com/inward/record.url?scp=85098986064&partnerID=8YFLogxK

U2 - 10.1109/ICTC49870.2020.9289270

DO - 10.1109/ICTC49870.2020.9289270

M3 - Conference contribution

AN - SCOPUS:85098986064

T3 - International Conference on ICT Convergence

SP - 22

EP - 24

BT - ICTC 2020 - 11th International Conference on ICT Convergence

PB - IEEE Computer Society

T2 - 11th International Conference on Information and Communication Technology Convergence, ICTC 2020

Y2 - 21 October 2020 through 23 October 2020

ER -

A Study on Energy-Process-Latency Tradeoff in Embedded Artificial Intelligence

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this