Classification of Sign Language in Real Time Using Convolutional Neural Network
Abstract
Communication between people is essential for daily life activities. However, humans are created with their own strengths and weaknesses. One of them is the difficulty of communication and interaction for people with hearing and speech impairments. Sign language is a language for people who have difficulty hearing and speaking. However, sign language is not popular in society, and people who have it will have more difficulties. This research aims to classify hand gestures of sign language into letters using a convolutional neural network (CNN). The dataset is obtained from Kaggle, with a total of 34,627 data divided by the ratio of training and testing data of 80:20. From the test results, the letters of the alphabet that can be translated are: A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, S, T, U, V, W, X, Y, and Z. Furthermore, validation accuracy is obtained. In this study, a very high validation accuracy was obtained. The easiest letters to guess are V and N, while the most difficult letters to guess are n, c, j, and z. With different preprocessing, the loss value can be reduced, giving a higher accuracy of 95.4%.
Keywords
Full Text:
PDFReferences
Z. K. S. Domas and R. Rakhmadi, “Peningkatan Performa Decision Tree dengan AdaBoost untuk Klasifikasi Kekurangtransparanan Informasi Anti-Korupsi,” Appl. Inf. Syst. Manag., vol. 5, no. 2, pp. 75–82, Oct. 2022, doi: 10.15408/aism.v5i2.24887.
A. Oktavianto and S. F. Persada, “Persepsi Publik Tentang Pembelajaran Daring di Indonesia: Studi Menggunakan ELK Stack dan Python untuk Analisis Sentimen di Twitter,” J. Tek. ITS, vol. 9, no. 2, pp. A170–A175, 2021, doi: 10.12962/j23373539.v9i2.54277.
S. Mujilahwati, “Visualisasi Data Hasil Klasifikasi Naïve Bayes Dengan Matplotlib Pada Python,” Pros. SNST Fak. Tek., vol. 1, no. 1, pp. 205–211, 2021, [Online]. Available: https://publikasiilmiah.unwahas.ac.id/index.php/PROSIDING_SNST_FT/article/view/.
P. A. Winata, “Klasifikasi Naive Bayes Keparahan Trauma Pasien Menggunakan Data Neuro Cognitive Dan Data Physiologic dengan Python,” Seminar Nas. Mat. Geom. Stat. dan Komputas, 2022, [Online]. Available: https://jurnal.unej.ac.id/index.php/prosiding/article/view/33500/11662.
S. Mujilahwati, M. Sholihin, and R. Wardhani, “Optimasi Hyperparameter TensorFlow dengan Menggunakan Optuna di Python: Study Kasus Klasifikasi Dokumen Abstrak Skripsi,” J. Media Inform. Budidarma, vol. 5, no. 3, pp. 1084–1089, 2021, doi: 10.30865/mib.v5i3.3090.
L. A. Septiandi, E. M. Yuniarno, and A. Zaini, “Deteksi Kedipan dengan Metode CNN dan Percentage of Eyelid Closure (PERCLOS),” J. Tek. ITS, vol. 10, no. 1, pp. A56–A63, 2021, doi: 10.12962/j23373539.v10i1.61174.
W. Kurniawan and A. Harjoko, “Pengenalan Bahasa Isyarat dengan Metode Segmentasi Warna Kulit dan Center of Gravity,” Indones. J. Electron. Instrum. Syst., vol. 1, no. 2, pp. 67–78, 2011, doi: 10.22146/ijeis.1964.
B. Nugroho and E. Y. Puspaningrum, “Kinerja Metode CNN untuk Klasifikasi Pneumonia dengan Variasi Ukuran Citra Input,” J. Teknol. Inf. dan Ilmu Komput., vol. 8, no. 3, pp. 533–538, 2021, doi: 10.25126/jtiik.2021834515.
A. A. Gafar and J. Y. Sari, “Sistem Pengenalan Bahasa Isyarat Indonesia dengan Menggunakan Metode Fuzzy K-Nearest Neighbor,” J. Ultim., vol. 9, no. 2, pp. 122–128, 2018, doi: 10.31937/ti.v9i2.671.
A. Willyanto, D. Alamsyah, and H. Irsyad, “Identifikasi Tulisan Tangan Aksara Jepang Hiragana Menggunakan Metode CNN Arsitektur VGG-16,” J. Algoritm., vol. 2, no. 1, pp. 1–11, Oct. 2021, doi: 10.35957/algoritme.v2i1.1450.
N. Sazqiah et al., “Pengenalan Aksara Lampung Menggunakan Metode CNN (Convolutional Neural Network),” Semin. Nas. Ins. Prof., vol. 2, no. 1, pp. 1–5, 2022, doi: 10.23960/snip.v2i1.165.
I. Rahayuningsih, A. D. Wibawa, and E. Pramunanto, “Klasifikasi Bahasa Isyarat Indonesia Berbasis Sinyal EMG Menggunakan Fitur Time Domain (MAV, RMS, VAR, SSI),” J. Tek. ITS, vol. 7, no. 1, pp. A175–A180, 2018, doi: 10.12962/j23373539.v7i1.29967.
N. A. Hasma, F. Arnia, R. Muharar, and M. K. Muchamad, “Pengenalan Gerakan Isyarat Bahasa Indonesia Menggunakan Algoritma SURF dan K-Nearest Neighbor,” KITEKTRO: Jurnal Komputer, Informasi Teknologi dan Elektro, vol. 7, no. 1, pp. 50–54, 2022.
A. Lianardo, “Klasifikasi Gejala Penyakit Daun pada Tanaman Singkong Berbasis Vision Menggunakan Metode CNN dengan Arsitektur Mobilenet,” e-Proceeding of Engineering, vol. 8, no. 6, pp. 3176–3179, 2022, doi: 10.34818/eoe.v9i6.18980.
Darmatasia, “Pengenalan Sistem Isyarat Bahasa Indonesia (SIBI) Menggunakan Gradient-Convolutional Neural Network,” J. Instek, vol. 6, no. 1, pp. 56–65, 2021, doi: 10.24252/instek.v6i1.18637.
R. I. Borman, B. Priyopradono, and A. R. Syah, “Klasifikasi Objek Kode Tangan pada Pengenalan Isyarat Alphabet Bahasa Isyarat Indonesia (BISINDO),” Semin. Nas. Inform. dan Apl., no. September, pp. 1–4, 2018.
F. Devina, C. Citra, and E. Tanuwijaya, “Klasifikasi Bahasa Isyarat Amerika menggunakan Convolutional Neural Network,” J. Sist. dan Teknol. Inf., vol. 10, no. 1, pp. 139–144, Jan. 2022, doi: 10.26418/justin.v10i1.47184.
S. Ependi et al., “Klasifikasi Pendeteksi Wajah Berhijab Mengunakan Metode CNN (Convlutional Neural Network),” J. Pendidik. Tambusai, vol. 6, no. 1, pp. 3157–3164, 2022, [Online]. Available: https://jptam.org/index.php/jptam/article/view/3363.
A. Anton, N. F. Nissa, A. Janiati, N. Cahya, and P. Astuti, “Application of Deep Learning Using Convolutional Neural Network (CNN) Method for Women’s Skin Classification,” Sci. J. Informatics, vol. 8, no. 1, pp. 144–153, 2021, doi: 10.15294/sji.v8i1.26888.
B. D. Prasetya, F. S. Pamungkas, and I. Kharisudin, “Pemodelan dan Peramalan Data Saham dengan Analisis Time Series menggunakan Python,” Prism. Pros. Semin. Nas. Mat., vol. 3, pp. 714–718, 2020, [Online]. Available: https://journal.unnes.ac.id/sju/index.php/prisma/ ISSN.
R. Magdalena, S. Saidah, N. K. C. Pratiwi, and A. T. Putra, “Klasifikasi Tutupan Lahan Melalui Citra Satelit SPOT-6 dengan Metode Convolutional Neural Network (CNN),” J. Edukasi dan Penelit. Inform., vol. 7, no. 3, pp. 335–339, 2021, doi: 10.26418/jp.v7i3.48195.
V. Data, P. Covid, M. B. Tamam, and A. Hozairi, “Indonesia dan Malaysia Data Visualization of the Spread of Covid 19 in Indonesia and Malaysia,” Jurnal SimanteC, vol. 11, no. 1, pp. 13–18, Dec. 2022, doi: 10.21107/simantec.v11i1.14252.
N. Yu and K. Darling, “A Low-cost Approach to Crack Python CAPTCHAs using AI-based Chosen-plaintext Attack,” Appl. Sci., vol. 9, no. 10, pp. 1–17, 2019, doi: 10.3390/app9102010.
N. Hanum Harani, C. Prianto, and M. Hasanah, “Deteksi Objek dan Pengenalan Karakter Plat Nomor Kendaraan Indonesia Menggunakan Metode Convolutional Neural Network (CNN) Berbasis Python,” J. Tek. Inform., vol. 11, no. 3, pp. 47–53, 2019.
DOI: https://doi.org/10.15408/aism.v6i1.29820 Abstract - 0 PDF - 0
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
EDITORIAL ADDRESS:
Department of Information Systems, Faculty of Science and Technology,
Universitas Islam Negeri (UIN) Syarif Hidayatullah Jakarta
Faculty of Science and Technology Building, 3rd Floor, 1st Campus, Universitas Islam Negeri (UIN) Syarif Hidayatullah Jakarta
Jl. Ir. H. Juanda No. 95, Ciputat Timur, Kota Tangerang Selatan, Banten 15412, Indonesia.
Tlp/Fax: +622174019 25/+62217493315.
E-mail: aism.journal@apps.uinjkt.ac.id, Website: https://journal.uinjkt.ac.id/index.php/aism
Applied Information System and Management (AISM) by the Department of Information Systems, Faculty of Science and Technology, Universitas Islam Negeri (UIN) Syarif Hidayatullah Jakarta, Indonesia is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://journal.uinjkt.ac.id/index.php/aism.