AI Role in Sign Language Interpretation

Authors

  • Yashraj P. Chavan Student, Department of Computer Engineering, Vishwaniketan's Institute of Management Entrepreneurship and Engineering Technology (ViMEET), Khalapur, Maharashtra, India
  • Paras P. Wala Student, Department of Computer Engineering, Vishwaniketan's Institute of Management Entrepreneurship and Engineering Technology (ViMEET), Khalapur, Maharashtra, India
  • Nihal S. Gavandi Student, Department of Computer Engineering, Vishwaniketan's Institute of Management Entrepreneurship and Engineering Technology (ViMEET), Khalapur, Maharashtra, India
  • Piyush N. Patil Student, Department of Computer Engineering, Vishwaniketan's Institute of Management Entrepreneurship and Engineering Technology (ViMEET), Khalapur, Maharashtra, India
  • Babeetta Bbhagat Assistant Professor, Department of Computer Engineering, Vishwaniketan's Institute of Management Entrepreneurship and Engineering Technology (ViMEET), Khalapur, Maharashtra, India

Keywords:

Data Acquisition and Control (DAC), Sign Language Translator, Gesture Recognition

Abstract

People who are mute or deaf encounter communication challenges when interacting with others. It can be difficult for individuals with similar conditions to effectively convey their thoughts, especially when not everyone understands sign language. This article aims to create a Data Acquisition and Control (DAC) system capable of translating sign language into written text that can be comprehended by a wider audience. This system is referred to as the “Sign Language Translator and Gesture Recognition”. We have designed an intelligent web interface that captures hand gestures and translates them into readable text. The content within this textbook can be wirelessly transmitted to a smartphone or displayed on an embedded TV screen. It is evident from the experimental outcomes that affordable sensors can capture gestures, measuring finger positions and their movements. The current system's performance demonstrates its ability to accurately interpret 20 out of 26 letters, achieving a recognition accuracy of 96%. Sign languages rely on hand and finger shapes, motion, body language, and facial expressions to convey meanings, and they vary from one country to another. As an example, Irish deaf individuals use Irish Sign Language, while in India, people use Indian Sign Language. Different sign languages possess their distinct alphabets, semantics, and vocabularies. Variations in dialects among regions are prevalent, even within a single nation. Variations in some of the signals are employed by various creative groups.

References

Stokoe WC, Casterline DC, Croneberg CG. A dictionary of American Sign Language on linguistic principles. Washington: Gallaudet College Press; 1965.

Quinto-Pozos D. Beyond bilingual programming: Interpreter education in the US amidst increasing linguistic diversity. International Journal of Interpreter Education (IJIE). 2018; 10(1): 6.

Parton BS. Sign language recognition and translation: A multidisciplined approach from the field of artificial intelligence. J Deaf Stud Deaf Educ. 2006 Jan 1; 11(1): 94–101.

Bragg D, Koller O, Bellard M, Berke L, Boudreault P, Braffort A, Caselli N, Huenerfauth M, Kacorri H, Verhoef T, Vogler C. Sign language recognition, generation, and translation: An interdisciplinary perspective. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility. 2019 Oct 24; 16–31.

Nguyen AT, Nguyen TD, Phan HD, Nguyen TN. A deep neural network language model with contexts for source code. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). 2018 Mar 20; 323–334.

Rastgoo R, Kiani K, Escalera S. Sign language recognition: A deep survey. Expert Syst Appl. 2021 Feb 1; 164: 113794.

Schembri Adam, Jordan Fenlon, Ramas Rentelis, Sally Reynolds, Kearsy Cormier. Building the British Sign Language Corpus. Lang Doc Conserv. 2013; 7: 136–154.

Camgoz NC, Hadfield S, Koller O, Ney H, Bowden R. Neural sign language translation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2018; 7784–7793.

Davis JE. Hand talk: Sign language among American Indian nations. Cambridge University Press; 2010 Jul 29.

Borghouts J, Neary M, Palomares K, De Leon C, Schueller SM, Schneider M, Stadnick N, Mukamel DB, Sorkin DH, Brown D, McCleerey-Hooper S. Understanding the potential of mental health apps to address mental health needs of the deaf and hard of hearing community: mixed methods study. JMIR Hum Factors. 2022 Apr 11; 9(2): e35641.

Feldman DM, Caggiano C, Beldon-Feldman J. Why language matters: What forensic psychologists need to know about the deaf community, American Sign Language, and interpreters. Pract Innov. 2023 Sep; 8(3): 249.

Leeson L, Wurm S, Vermeerbergen M, editors. Signed language interpreting: preparation, practice and performance. Routledge; 2014 Apr 8.

Szabó B, Hercegfi K. User‐centered approaches in software development processes: Qualitative research into the practice of Hungarian companies. J Softw: Evolution and Process. 2023 Feb; 35(2): e2501.

Ahmed MA, Zaidan BB, Zaidan AA, Alamoodi AH, Albahri OS, Al-Qaysi ZT, Albahri AS, Salih MM. Real-time sign language framework based on wearable device: analysis of MSL, DataGlove, and gesture recognition. Soft Comput. 2021 Aug; 25(16): 11101–22.

Bantupalli K, Xie Y. American sign language recognition using deep learning and computer vision. In 2018 IEEE International Conference on Big Data (Big Data). 2018 Dec 10; 4896–4899.

Halder A, Tayade A. Real-time vernacular sign language recognition using mediapipe and machine learning. Int J Res Publ Rev. 2021; 2(5): 9–17. Journal homepage: www. ijrpr. com ISSN. 2021;2582:7421.

Shenoy K, Dastane T, Rao V, Vyavaharkar D. Real-time Indian sign language (ISL) recognition. In 2018 IEEE 9th international conference on computing, communication and networking technologies (ICCCNT). 2018 Jul 10; 1–9.

Cheok MJ, Omar Z, Jaward MH. A review of hand gesture and sign language recognition techniques. Int J Mach Learn Cybern. 2019 Jan 31; 10(1–3): 131–53.

Downloads

Published

2023-11-21

Issue

Vol. 10 No. 2 (2023): Journal of Open Source Developments

Section

Review Article