Classification performance analysis regarding margin of energy-based model

Myeong K. Kang; Kyo H. Park; Seong W. Kim; Min J. Kim; Sang C. Lee

doi:10.1145/3178264.3178280

Classification performance analysis regarding margin of energy-based model

Myeong K. Kang, Kyo H. Park, Seong W. Kim, Min J. Kim, Sang C. Lee

Division of Intelligent Robot

Nexys

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

When you train an energy-based model, it is important to set a good margin. However, it is almost impossible to train a good margin using stochastic gradient descent (SDG). Because the margin will saturate to zero while minimizing the cost function. For that reason, we usually set the margin to a non-trainable scalar to penalize offending answers linearly to more apart than a certain distance. A good performance setting relates to the length of margin and dimension to which the feature being mapped. In this paper, we will show that a large margin does not always lead to a better performance and affirm the well-tuned margin can achieve better results.

Original language	English
Title of host publication	Proceedings of 2017 International Conference on Industrial Design Engineering, ICIDE 2017
Publisher	Association for Computing Machinery
Pages	34-37
Number of pages	4
ISBN (Electronic)	9781450348669
DOIs	https://doi.org/10.1145/3178264.3178280
State	Published - 29 Dec 2017
Event	2017 International Conference on Industrial Design Engineering, ICIDE 2017 - Dubai, United Arab Emirates Duration: 28 Dec 2017 → 31 Dec 2017

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	2017 International Conference on Industrial Design Engineering, ICIDE 2017
Country/Territory	United Arab Emirates
City	Dubai
Period	28/12/17 → 31/12/17

Bibliographical note

Publisher Copyright:
© 2017 Association for Computing Machinery.

Keywords

Energy based models
Hinge loss
Margin
Negativelog-likelihood loss
Triplet loss

Access to Document

10.1145/3178264.3178280

Cite this

Kang, M. K., Park, K. H., Kim, S. W., Kim, M. J., & Lee, S. C. (2017). Classification performance analysis regarding margin of energy-based model. In Proceedings of 2017 International Conference on Industrial Design Engineering, ICIDE 2017 (pp. 34-37). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3178264.3178280

@inproceedings{e2892aedd0594f79aaafc70f35fa262b,

title = "Classification performance analysis regarding margin of energy-based model",

abstract = "When you train an energy-based model, it is important to set a good margin. However, it is almost impossible to train a good margin using stochastic gradient descent (SDG). Because the margin will saturate to zero while minimizing the cost function. For that reason, we usually set the margin to a non-trainable scalar to penalize offending answers linearly to more apart than a certain distance. A good performance setting relates to the length of margin and dimension to which the feature being mapped. In this paper, we will show that a large margin does not always lead to a better performance and affirm the well-tuned margin can achieve better results.",

keywords = "Energy based models, Hinge loss, Margin, Negativelog-likelihood loss, Triplet loss",

author = "Kang, \{Myeong K.\} and Park, \{Kyo H.\} and Kim, \{Seong W.\} and Kim, \{Min J.\} and Lee, \{Sang C.\}",

note = "Publisher Copyright: {\textcopyright} 2017 Association for Computing Machinery.; 2017 International Conference on Industrial Design Engineering, ICIDE 2017 ; Conference date: 28-12-2017 Through 31-12-2017",

year = "2017",

month = dec,

day = "29",

doi = "10.1145/3178264.3178280",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "34--37",

booktitle = "Proceedings of 2017 International Conference on Industrial Design Engineering, ICIDE 2017",

}

Kang, MK, Park, KH, Kim, SW, Kim, MJ & Lee, SC 2017, Classification performance analysis regarding margin of energy-based model. in Proceedings of 2017 International Conference on Industrial Design Engineering, ICIDE 2017. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 34-37, 2017 International Conference on Industrial Design Engineering, ICIDE 2017, Dubai, United Arab Emirates, 28/12/17. https://doi.org/10.1145/3178264.3178280

Classification performance analysis regarding margin of energy-based model. / Kang, Myeong K.; Park, Kyo H.; Kim, Seong W. et al.
Proceedings of 2017 International Conference on Industrial Design Engineering, ICIDE 2017. Association for Computing Machinery, 2017. p. 34-37 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Classification performance analysis regarding margin of energy-based model

AU - Kang, Myeong K.

AU - Park, Kyo H.

AU - Kim, Seong W.

AU - Kim, Min J.

AU - Lee, Sang C.

PY - 2017/12/29

Y1 - 2017/12/29

N2 - When you train an energy-based model, it is important to set a good margin. However, it is almost impossible to train a good margin using stochastic gradient descent (SDG). Because the margin will saturate to zero while minimizing the cost function. For that reason, we usually set the margin to a non-trainable scalar to penalize offending answers linearly to more apart than a certain distance. A good performance setting relates to the length of margin and dimension to which the feature being mapped. In this paper, we will show that a large margin does not always lead to a better performance and affirm the well-tuned margin can achieve better results.

AB - When you train an energy-based model, it is important to set a good margin. However, it is almost impossible to train a good margin using stochastic gradient descent (SDG). Because the margin will saturate to zero while minimizing the cost function. For that reason, we usually set the margin to a non-trainable scalar to penalize offending answers linearly to more apart than a certain distance. A good performance setting relates to the length of margin and dimension to which the feature being mapped. In this paper, we will show that a large margin does not always lead to a better performance and affirm the well-tuned margin can achieve better results.

KW - Energy based models

KW - Hinge loss

KW - Margin

KW - Negativelog-likelihood loss

KW - Triplet loss

UR - https://www.scopus.com/pages/publications/85049828166

U2 - 10.1145/3178264.3178280

DO - 10.1145/3178264.3178280

M3 - Conference contribution

AN - SCOPUS:85049828166

T3 - ACM International Conference Proceeding Series

SP - 34

EP - 37

BT - Proceedings of 2017 International Conference on Industrial Design Engineering, ICIDE 2017

PB - Association for Computing Machinery

T2 - 2017 International Conference on Industrial Design Engineering, ICIDE 2017

Y2 - 28 December 2017 through 31 December 2017

ER -

Classification performance analysis regarding margin of energy-based model

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this