Real-time Sign Language Detection Using Computer Vision and Machine Learning Techniques
Keywords:
Hand gestures, OpenCV, Convolutional Neural Network, Dataset, Features extraction and Detection, Word prediction, Word FormationAbstract
Communication plays an important role in our day-to-day lives as it assists people in sharing, and connecting to each other. Despite knowing the significance of the same, some deaf-mute pupils who are not able to communicate freely. These groups of people use sign language for sharing their thoughts. It could be a tough task to understand sign language for a group of humans who are not deaf-mute. We developed a system that makes use of American Sign Language as the solution to the identified problem. The strategy involves implementing the module which detects pre-defined American sign language through hand gestures. For detection of movement of gestures, we use basic hardware components like a camera and OpenCV, media pipelined images and the image is passed to the CNN model for identification followed by word formation. The principal results of using this system will aid deaf-mute humans to share their ideas fruitfully and communicate with normal individuals proficiently and without hesitation.
References
Kumuda S, Mane Preethi K. Smart Assistant for Deaf and Dum Using Flexi le Resisti e Sensor : Implemented on LabVIEW Platform. 2020 IEEE International Conference on Inventive Computation Technologies (ICICT). 2020; 994–1000. DOI.org (Crossref), https://doi.org/10.1109/ICICT48043.2020.9112553.
Soni NS, Nagmode MS, Komati RD. Online hand gesture recognition & classification for deaf & dumb. 2016 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, India. 2016; 1–4. doi: 10.1109/INVENTIVE.2016.7830112.
PSG, JJ, SR, SY, PGS, Hiremath, Pendari NT. Dynamic Tool for American Sign Language Finger Spelling Interpreter. 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN), Greater Noida, India. 2018; 596–600. doi: 10.1109/ICACCCN.2018.8748859.
Sarbaresh G, Karthi A. Design and implementation of a sign-to-speech/text system for deaf and dumb people. 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI), Chennai, India. 2017; 1840–1844. doi: 10.1109/ICPCSI.2017.8392033.
Adewale V, Olamiti A. Conversion of sign language to text and speech using Machine Learning Techniques. Journal of Research and Review in Science (JRRS). 2018; 5(1): 58–65. https://doi.org/10.36108/jrrslasu/8102/50(0170)
Gowda N, Preksha, Surekha. A Review on Conversion of Sign Language in to Text and Audio. Int J Adv Res Innov Ideas Educ. 2022; 8(3): 193–196. Available from: https://ijariie.com/Admin UploadPdf/A_Review_on_Sign_Language_Conversion_Into_Text_and_Audio_ijariie16512.pdf
Saini S, Chaudhry S, Sapna. Virtual Assistant for Deaf and Dumb. Int J Adv Res Innov Ideas Educ. 2018; 4(3): 148–152. Available from: http://ijariie.com/AdminUploadPdf/ Virtual_talk_Assistance_for_Deaf_and_Dumb_ijariie8294.pdf
Mumhammad AL-Querishi, Thariq Khalid, Riad Souissi. Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues. IEEE Access. 2021; 9: 126917–126951. https://www.researchgate.net/publication/354426043_Deep_Learning_for_Sign_Language_Recognition_Current_Techniques_Benchmarks_and_Open_Issues
Ekbote J, Joshi M. Indian sign language recognition using ANN and SVM classifiers. In2017 International conference on innovations in information, embedded and communication systems (ICIIECS) 2017 Mar 17 (pp. 1-5). IEEE.
Samonte MJ, Guingab CJ, Relayo RA, Sheng MJ, Tamayo JR. Using Deep Learning in Sign Language Translation to Text.
