Segmentation and Classification of Hand Gestures for Indian Sign Language with YCbCr and HSV Color Models with Dissimilar Signers Under Different Lighting Conditions using AlexNet
Keywords:
Indian sign language recognition system, invariant background, independent signers, support vector machine, AlexNetAbstract
Indian Sign Language (ISL) Recognition System can be implemented by considering the signs as hand gestures and developing a hand gesture recognition system. The impact of illumination changes on the performance of segmentation and classification where different lighting conditions such as normal, bright, dim, and artificial light under different color spaces like RGB, HSV and YCbCr are studied, and performance parameters are analyzed. It is observed that YCbCr color space is effective for complex images with different illumination. Occlusions of the hand with the face put additional constraints on the segmentation process. The effect of the varied skin color of different signers is also subjected to the study in this work. In India/Kerala, the skin color of the people varies drastically. It is difficult to fix the pixel value of the skin color as it affects the segmentation process. It will select other objects with the same color as the skin in the background. These constraints are overcome by selecting large, connected components in the frame as hand and are segmented. After modeling and analysis of the input hand image, gesture classification is done using SVM and AlexNet to recognize the gesture. The accuracy obtained with SVM is 93.8% and with AlexNet is 98.5%.
References
ISL Dictionary Launch. (2022). [Online]. ISLRTC. Islrtc.nic.in. Available from: https://islrtc.nic.in/isl-dictionary-launch
Manisha Kowdiki, Arti Khaparde. Automatic hand gesture recognition using hybrid meta-heuristic-based feature selection and classification with Dynamic Time Warping. Comput Sci Rev. 2021; 39: 100320.
Dolly Indraa, Purnawansyaha, Sarifuddin Madendab, Eri Prasetyo Wibowo. Indonesian Sign Language Recognition Based on Shape of Hand Gesture. The 5th Information Systems International Conference 2019; Procedia Comput Sci. 2019; 161: 74–81. 10.1016/j.procs.2019.11.101
Ibrahim Nada B, Selim Mazen M, Zayed Hala H. An Automatic Arabic Sign Language Recognition System (ArSLRS). J King Saud Univ – Computer and Information Sciences. 2018; 30(4): 470–477.
Pan T-Y, Lo L-Y, Yeh C-W, Li J-W, Liu H-T, Hu M-C. Sign language recognition in complex background scene based on adaptive skin colour modelling and support vector machine. Int J Big Data Intelligence. 2018; 5(1/2): 21–30.
Kumud Tripathi, Neha Baranwal, Nandi GC. Continuous Dynamic Indian Sign Language Gesture Recognition with Invariant Backgrounds. In: Advanced Computing, Networking and Security. Vol. 7135 of the series Lecture Notes in Computer Science. Springer; 2015; 106–116.
Bauer B, Karl-Friedrich. Towards an Automatic Sign Language Recognition System Using Subunits. LNAI 2298, GW-2001, Springer. 2002; 34–47.
Ghotkar AS, Kharate DGK. Study of vision based hand gesture recognition using Indian sign language. International Conference on Smart Sensing and Intelligent Systems. 2014; 7(1): 96–115.
Khamar Basha Shaika, Ganesan P, Kalist V, Sathish BS, Merlin Mary Jenithab J. Comparative Study of Skin Color Detection and Segmentation in HSV and YCbCr Color Space. 3rd International Conference on Recent Trends in Computing 2015 (ICRTC-2015); Procedia Comput Sci. 2015; 57: 41–48.
Hazem Khaled, Sayed Samir G, Saad Sayed M, Hossam Ali. Hand Gesture Recognition Using Modified 1$ and Background Subtraction Algorithms. Math Probl Eng. 2015; 2015: Article ID 741068. https://doi.org/10.1155/2015/741068
Suzuki S, Be K. Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics and Image Processing (CVGIP). 1985; 30(1): 32–46.
Yanan Xu, Dong-Won Park, Gou Chol Pok. Hand Gesture Recognition Based on Convex Defect Detection. Int J Appl Eng Res. 2017; 12(18): 7075–7079. ISSN 0973-4562.
Jonghoon Seo, Seungho Chae, Jinwook Shim, Dongchul Kim, Cheolho Cheong, Tack-Don Han. Fast Contour-Tracing Algorithm Based on a Pixel-Following Method for Image Sensors. Sensors. 2016; 16(3): 353. doi:10.3390/s16030353
Reshna S, Jayaraju M. Spotting and recognition of hand gesture for Indian sign language recognition system with skin segmentation and SVM. 2017 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), Chennai, India. 2017; 386–390. DOI: 10.1109/WiSPNET.2017.8299784.
