Ethiopian Sign Language Recognition By Using Image Processing

Abstract

Sign language is an independent language that uses gestures and body language to convey meaning. Sign language facilities the communication between deaf and community. The main aim of this study to develop a model recognizes sign language using image processing. There is the gap of communication between hearing impaired and hearing peoples. Due to this communication gap deaf people encounter so many problems in their daily life since they are living with the people who communicate with spoken languages. So the researcher will develop a model which can eliminate this gap of communication between the hearing disables and hearing people. Even if some works have been done on Ethiopian sign language recognition they consider only the hand gesture. By having this information we have proposed a system that can recognize the signs of Ethiopian sign language and convert to text by facial expression.

Introduction

Deafness is the inability to hear. The problem may occur in one or both of the ears. Basically deafness may be inherited or caused by accidents. Sign language is a language in which hearing disable people communicate with each other and with hearing people. According to world health organization. A person who is not able to hear thresholds of 25 dB in one or in both ears is said to have hearing loss. Hearing loss may be mild, moderate, severe, or profound. It can affect one ear or both ears, and leads to difficulty in hearing conversational speech or loud sounds. The majority of people with disabling hearing loss live in low- and middle-income countries [5]. Sign Language (SL) continues to be the most important means of communication between the deaf-mute among themselves, and with the rest of the society. Image processing techniques are then used to detect and track hands and fingers, as well as facial expressions of the signer [7].

In the existing system, there are two basic communication ways which help deaf people to communicate with the hearing people and with themselves. The first one is communication through oral means (lip reading) which is very useful to communicate with the hearing people. The other one is manual means of communication which is expressed using hands and the upper body parts [2]. Communicate deaf people use different technology like text messaging, E-mail.

Hearing disable people need to communicate, express their feeling, get access of education and so on like that of hearing people. To get this access there must be a communication medium between the hearing and speaking disables and the healthy people. This communication medium should be sign language. As literatures show sign language is independent and fully fledged language which can convey information like spoken language. Sign language is the use of hand gesture, head motion facial expression, and leap movement to convey information. It is mostly used by hearing disable persons to communicate each other and with hearing peoples. However hearing people uses sign language rarely [10].

Even if all sign languages in across the world uses hand gesture, facial expression, and leap movement sign languages differ from one country to another country [6]. Hence there are so many sign languages in the world. For example, British Sign Language (BSL), Australian Sign Language (Auslan), American Sign Language (ASL), Taiwan Sign Language (TSL), New Zealand Sign Language (NZSL)[6].

Ethiopian Sign Language (EthSL) is one of the under-researched languages of Ethiopia although it is used by more than a million members of the Deaf community [4]. More works have been done on different sign languages recognition around the world. However works done on EthSL are limited.

Statement of problem

To use sign language an individual must have to learn the language first. However in Ethiopia Sign language has been serving in Deaf education for the last five decades, though how it is employed does not get enough attention from researchers [8]. Hence the chance of learning the language is very low. In addition to this, the numbers of hearing persons who can understand sign language are very negligible. A communication gap to interact with the hearing people to accomplish their daily tasks. To minimize this communication gap there should be a system which can recognize the sign language signs and make hearing people to understand what the hearing disables want to say by using text. There are a lot of works done worldwide to recognize sign languages. There are some researches On EthSl recognition also. However they mainly focus on static signs which can only recognize the base alphabets and the numbers only. Problems that are encountered in the formal education process that includes Lack of sufficient interpreters for each school in which deaf students are enrolled [3].

Save your time!
We can take care of your essay

• Proper editing and formatting
• Free revision, title page, and bibliography
• Flexible prices and money-back guarantee

Place Order

There are some works done on continuous signs too. However they recognize only static signs and hand gestures. According to [1], SLR system is incomplete without the signer’s facial expressions corresponding to the sign gesture.

Literature review

In this literature review discusses previous research critically about key areas of the theoretical basis of the sign language. Nada B. Ibrahimet al in [9] proposed a system to recognize Arabic sign language recognition. In this article hand motion and head motion tracking has been done by skin blob detection to track the motion and Euclidian distance measurement is used for classification to achieve recognition accuracy of 97% accuracy.

Abdi Tsegaye [11] uses hand motion trajectory detection to HMTD to detect isolated Ethiopian sign languages. They have applied Modified Hausdorff Distance (MHD) and achieved the recognition accuracy of 71.88%. Samuel teshome [6] propose a system to detect Ethiopian sign language recognition.

Tefera Gimbi[2] proposed Recognition of Isolated Signs in Ethiopian Sign language. In this study the data collected by camera then features are extracted from the segmented body parts and these values are converted into symbols using k-means algorithm. Hidden Markov Models trained using these symbols and achieved recognition accuracy of 87%.They have used KNN algorithm and achieved recognition accuracy of 40%.

Alaaddin I. Sidig et al in [10] in 2017 used Hartly transform (HT), furior transformation (FT), gabour transformation (GT) for image preprocessing and support vector machine (SVM) for classification to recognize Arabic sign language recognition and achieved 87% accuracy.

Techniques

According to [9] sign language recognition can be performed in two different ways. The first one is sensor based way needs variety of electro mechanical devices which are incorporated with many sensors to detect signs. In addition to this sensor based way needs smart gloves the signer should put on his hand while signing. The second way is visual-based SLR that uses one camera. The second way have low cost and high mobility than the former one. Hence in this study will focus on visual-based EthSL recognition. Object detection is used to recognize facial recognition and object motion tracking is used to track the motion of hands and head. The overall method includes input video, convert video to image frame, feature extraction (object detection), and at the last recognition (classification).

First capture video by using digital camera then preprocessing and segment the video to image frame then feature extraction.After extracting features we are going to recognition stage. For classification there are so many algorithms. These are Hidden Markov Model (HMM), artificial neural network (ANN), Support Vector Machine (SVM), Neuro-fuzzy inference system (NFIS).

Reference

1. Pradeep Kumar , ParthaPratim Roy, Debi ProsadDogra ,Independent Bayesian classifier combination based sign language recognition using facial expression ,2018
2. Tefera Gimbi, proposed Recognition of Isolated Signs in Ethiopian Sign language,2014.
3. Legesse Zerubabel, Ethiopian Finger Spelling Classification, 2008.
4. EyasuHailu Language Use in Ethiopian Sign Language, 2018.
5. WHO global estimates on prevalence of hearing loss march in 2018
6. Samuel teshome , isolated word-level Ethiopiansign language recognition ,2013
7. Mohamed Deriche, Salihu Aliyu, and Mohamed Mohandes, An Intelligent Arabic Sign Language Recognition System using a Pair of LMCs with GMM Based Classification,2019.
8. Elizabeth Demissie , research paper AddisAbaba university, Sign Language use as Medium of Instruction: The Caseof Grade One and Two at Mekanissa School for the Deaf, 2011.
9. Nada B. Ibrahim , Mazen M. Selim, Hala H. Zayed, journal of king saud university, An Automatic Arabic Sign Language Recognition System (ArSLRS), 2018.
10. Alaaddin I. Sidig, HamzahLuqman, Sabri A. Mahmoud, Transform-based Arabic sign language recognition,2017
11. Abdi thegaye , research paper, Offline candidate hand gesture selection andtrajectory determination for continuous Ethiopiansign language,2011