Supervised Classification via Energy-Based Learning on CIFAR10 CIFAR100 and CELEBA

Wahyuningdiah Trisari Harsanti Putri; Quintin Kurnia Dikara; Heryudi Ganesha; Deshinta Arrova Dewi

doi:10.15408/jti.v19i2.48655

Authors

Wahyuningdiah Trisari Harsanti Putri Informatics Study Program, Faculty of Engineering Science, Universitas Paramadina, Jakarta, Indonesia https://orcid.org/0000-0002-9745-7933
Quintin Kurnia Dikara Informatics Study Program, Faculty of Engineering Science, Universitas Paramadina, Jakarta, Indonesia
Heryudi Ganesha Devoteam Indonesia, Jakarta, Indonesia
Deshinta Arrova Dewi Faculty of Data Science and Information Technology, INTI International University, Malaysia

DOI:

https://doi.org/10.15408/jti.v19i2.48655

Keywords:

Energy Based Model, JEM, machine learning, vision, CIFAR, CELEBA

Abstract

This research evaluates a Joint Energy Model (JEM) approach to perform supervised classification on the CIFAR10, CIFAR100, and CELEBA datasets. We implement a range of architectures, including custom shallow and deep CNNs and an adapted ResNet-18 modified with 3x3 first-layer convolution to prevent the loss of critical spatial information for small image classification. Following a rigorous experimental protocol involving multiple independent runs, we provide statistical validation via mean and standard deviation for all performance metrics. Our models achieved F1-scores exceeding 86% for both CIFAR datasets and a 90.34% accuracy of CELEBA multi-label classification. Beyond accuracy, we conducted a systematic analysis of the learned energy landscapes. While results confirm a systemic calibration failure in standard discriminative training for CIFAR objects, they reveal the emergence of a robust, globally coherent “face manifold” detector for the CELEBA dataset. These findings quantify the calibration limitations of standard classifiers and demonstrate how post-hoc energy interpretation can reveal the underlying structure of learned visual representations.

Author Biographies

Wahyuningdiah Trisari Harsanti Putri, Informatics Study Program, Faculty of Engineering Science, Universitas Paramadina, Jakarta, Indonesia

Informatics Study Program, Faculty of Engineering Sciences.
Quintin Kurnia Dikara, Informatics Study Program, Faculty of Engineering Science, Universitas Paramadina, Jakarta, Indonesia

Informatics Study Program, Faculty of Engineering Sciences.
Heryudi Ganesha, Devoteam Indonesia, Jakarta, Indonesia

InfraApp Tribe Lead/Project Manager, PT. Devoteam Cloud Services

References

[1] I. Goodfellow et al., “Generative adversarial networks,” Commun. ACM, vol. 63, no. 11, pp. 139–144, 2020, doi: 10.1145/3422622.

[2] M. H. Guo et al., “Attention mechanisms in computer vision: A survey,” Comput. Vis. Media (Beijing)., vol. 8, no. 3, pp. 331–368, 2022, doi: 10.1007/s41095-022-0271-y.

[3] T. Singh and D. K. Vishwakarma, “A deep multimodal network based on bottleneck layer features fusion for action recognition,” Multimed. Tools Appl., vol. 80, no. 24, pp. 33505–33525, 2021, doi: 10.1007/s11042-021-11415-9.

[4] L. Yuan et al., “Florence: A New Foundation Model for Computer Vision,” 2021.

[5] X. Zhao, L. Wang, Y. Zhang, X. Han, M. Deveci, and M. Parmar, A review of convolutional neural networks in computer vision, vol. 57, no. 4. Springer Netherlands, 2024. doi: 10.1007/s10462-024-10721-6.

[6] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017, doi: 10.1145/3065386.

[7] I. Ahmed, M. Kajol, U. Hasan, and P. P. Datta, “ChatGPT vs. Bard: A Comparative Study,” 2023.

[8] A. Borji and M. Mohammadian, “Battle of the Wordsmiths : Comparing ChatGPT, GPT-4, Claude, and Bard,” 2023.

[9] S. Bubeck et al., “Sparks of Artificial General Intelligence: Early experiments with GPT-4,” 2023.

[10] N. M.-D. Doris M. and Brennan, “Can ChatGPT and Bard generate aligned assessment items? A reliability analysis against human performance,” Journal of Applied Learning & Teaching, vol. 6, no. 1, pp. 75–80, 2023.

[11] S. Kabir, D. N. Udo-Imeh, B. Kou, and T. Zhang, “Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions,” ArXiv, 2023.

[12] T. P. Lillicrap et al., “Continuous control with deep reinforcement learning,” 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, 2016.

[13] M. Fang, X. Wang, and N. Z. Gong, “Provably Robust Federated Reinforcement Learning,” 2025, doi: 10.1145/3696410.3714728.

[14] G. Sreekumar and V. N. Bodetti, “IN PHYRE Discovers : Large Multimodal Models Struggle in Inductive Physical Reasoning,” arXiv:2506.12263v1, pp. 1–35, 2025.

[15] H. Shen et al., “PHY X : Does Your Model Have the ‘ Wits ’ for Physical Reasoning ?,” arXiv:2505.15929v2, 2025.

[16] C. Xue, V. Pinto, C. Gamage, E. Nikonova, P. Zhang, and J. Renz, “Phy-Q as a measure for physical reasoning intelligence,” Nat. Mach. Intell., vol. 5, no. January, pp. 83–93, 2023, doi: 10.1038/s42256-022-00583-4.

[17] Y. LeCun, “A Path Towards Autonomous Machine Intelligence,” Iclr, pp. 1–62, 2022.

[18] D. Ha and J. Schmidhuber, “Recurrent world models facilitate policy evolution,” Adv. Neural Inf. Process. Syst., vol. 2018-Decem, no. C, pp. 2450–2462, 2018.

[19] T. Osogami, “Boltzmann machines and energy-based models,” pp. 1–36.

[20] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016. Accessed: Jul. 28, 2026. [Online]. Available: https://www.deeplearningbook.org/

[21] W. Grathwohl, K. C. Wang, J. H. Jacobsen, D. Duvenaud, K. Swersky, and M. Norouzi, “Your Classifier Is Secretly an Energy Based Model and You Should Treat It Like One,” 8th International Conference on Learning Representations, ICLR 2020, pp. 1–23, 2020.

[22] M. Welling, D. Bren, and Y. W. Teh, “Bayesian Learning via Stochastic Gradient Langevin Dynamics,” 2011.

[23] A. Hyvarinen, “Estimation of Non-Normalized Statistical Models by Score Matching,” Journal of Machine Learning Research, vol. 6, p. 1, 2005, [Online]. Available: https://www.jmlr.org/papers/volume6/hyvarinen05a/hyvarinen05a.pdf

[24] F. K. Gustafsson, M. Danelljan, R. Timofte, and T. B. Schön, “How to Train Your Energy-Based Model for Regression,” 31st British Machine Vision Conference, BMVC 2020, 2020, doi: 10.5244/c.34.43.

[25] W. Grathwohl, K. C. Wang, J. H. Jacobsen, D. Duvenaud, K. Swersky, and M. Norouzi, “Your Classifier Is Secretly an Energy Based Model and You Should Treat It Like One,” 8th International Conference on Learning Representations, ICLR 2020, pp. 1–23, 2020.

[26] J. Hu, L. Shen, S. Albanie, G. Sun, and E. Wu, “Squeeze-and-Excitation Networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 8, pp. 2011–2023, 2020, doi: 10.1109/TPAMI.2019.2913372.

[27] V. Singh et al., “Beyond Cosine Decay: On the effectiveness of Infinite Learning Rate Schedule for Continual Pre-training,” pp. 1–21, 2025.

[28] H. Guo, J. Jin, and B. Liu, “Stochastic Weight Averaging Revisited,” Applied Sciences (Switzerland), vol. 13, no. 5, 2023, doi: 10.3390/app13052935.