Sign language is the main interactive human communication language in the world. Also, it is a collection of hand gestures for each alphabet letter, facial expressions, and body movements that are used to indicate emotion. Sign language, compared to oral language, is not universal; it differs depending on the nation, and even by region in the country [1].
Modern technology contributes to the development of an appropriate and efficient tool to assist the deaf community [2]. Furthermore, communication between deaf people and hearing people should reduce the necessity for a deaf person to always have a human translator. Deaf people have devised and utilized signs among themselves for thousands of years. Signing developed to become comprehensive and sophisticated languages among the many cultures of deaf people all over the world [3]. A disability issue might make daily life more challenging, although advances in technology and computer systems have made life simpler and easier [4].
The great majority of well-known sign languages including one-handed manual alphabet in America Sign Language (ASL) and a two-handed manual alphabet in British Sign Language (BSL). Both are important standards for classifying sign languages, but they are not in the same language. One significant distinction between ASL and BSL is that BSL letters are fully described for hand-shape, orientation, and placement, whereas ASL letters are simply given for hand shape and orientation [5]. Kurdish Sign Language (KuSL) in those countries where Kurdish people live has not been recognized and documented in the form of a standard sign language. Thus, KuSL has been built as a standard for KuSL based on ASL, by attempting to standardize and popularize the language among members of the deaf community and those concerned. Such efforts resulted in the KuSL standard using machine learning techniques.
The convolutional neural network (CNN) approach has recently gained popularity in image classification due to promising results produced in real-world applications. As a result, several CNN-based sign language translators may be found in cutting-edge technologies [6][7]. Although CNNs are effective tools for categorizing hand shapes in Sign SLR, they lack hand segmentation capacity, requiring the use of specialized methods [8]. CNN is an effective machine-learning tool for learning features from raw data input and outperforms typical machine learning models for image classification. It is a type of deep neural network that was first introduced for 2D input [9]. Two-dimensional (2D) CNN has been used successfully in a variety of domains, including image classification, face recognition, and natural language processing, because of its excellent characteristics in parameter tuning, rising efficacy, and minimal complexity [10].
The current paper proposes a classification of KuSL based on ASL alphabet and ArSL2018 dataset that could be a benchmark for Kurdish Sign Language Recognition Standard (KuSLRS) and could help Kurdish deaf community. A2D-CNN architecture is proposed to classify Kurdish letter based Arabic-script letter.
1.1. Project Aim and Objective
The main objective of this study is to recognize alphabets in KuSL by identifying hand shapes based on an ASL alphabet and Ar2018 dataset. The datasets have been adapted to work with the Kurdish alphabet. The proposed model recognizes hand movements from a camera and converts them to meaningful text in Kurdish-Arabic script.
Simplifying a communication between deaf and normal people through the proposed model can reduce the role of human interpreter. This is the first step in using KuSL to overcome the communication gap between the public communities and the Kurdish deaf population.
1.2. Related Work
The majority of researchers have taken one of three approaches: sensor-based gloves, computer vision, or hybrid systems that combine glove-based and computer vision systems. The first algorithm ignores facial expressions, which are extremely important in sign language identification. Computer-vision systems, on the other hand, are capable of recording the entire gesture, not to mention their mobility, which distinguishes them from glove-based systems [11] [12].
Recently, Several studies have been performed on several sign languages, including ASL [13][14][15][16][17][18][19][20], BSL [21]. Arabic sign language (ArSL) [1][22][23], Turkish sign language [24][25],Persian sign language [26][27] [28], Indian sign language [29][30], and to the best of the authors’ knowledge, the only three available researches have been on KuSL, including 12 classes [31], 10 classes [32], and 84 classes [33].
The proposed model increased the performance of the Kurdish sign recognition for KuSL. Two datasets have been modified and combined; ArSL2018 and ASL dataset that contains 34 signs for Kurdish alphabet, including 71400 samples. The three major challenges in Sign language recognition are the number of classes, models, and machine learning techniques.
Static gestures and dynamic gestures are the two forms of gestures. Finger spelling is used to demonstrate these actions. The system’s structure, whether it deals with static language signs expressed in a single picture or dynamic language signs portrayed in a series of images, on one side and/or continuous input on the other, manages the data gathering technique, feature extraction, and data classification. Because static signs are not time series, they can be simply expressed in a global feature [33]. According to [26] The Persian sign language (PSL) system applies wavelet transform and neural networks (NN) to recognize static gestures of the Persian alphabet. A digital camera is used to capture the pictures needed for the specified alphabets, and the color photos are cropped, scaled, and transformed to grayscale images to achieve 94.06% as an accuracy classification result. On the other hand, hand gestures were recorded with LM and facial data from the signer was taken with Kinect in order to perform recognition on a training dataset of 51 dynamic sign using HMM. By using multiple classification algorithms, the study reached 96.05% and 94.27% for one-handed and two-handed gestures, respectively [34]. In some studies, a combination of static and dynamic sign recognition has been applied as presented in [17] and [19].
In terms of data collection, several ways include employing a Kinect sensor, recording from a digital camera, capturing from continuous video, and using public datasets. A two-stage Hand posture recognition (HPR) method for Sign Language Recognition was proposed utilizing a Kinect sensor. In the first stage, an efficient method for hand detection and tracking was presented. Deep Neural Networks (DNNs) were used in the second step to automatically learn features from hand posture images that are insensitive to movement, and recognition accuracy reached 98.12% [14]. A unique hand gesture recognition approach is presented in order to recognize 32 characters from the Persian Sign Language (PSL) alphabet. The technique is based on the YCbCr color space, a single Gaussian model, and the Bayes rule. It recognizes hand regions in complicated backgrounds with non-uniform lighting. The system is evaluated using 480 PSL posture pictures obtained using a USB webcam. The overall recognition rate was 95.62%. Moreover, a dynamic hand gesture recognition system is proposed to identify Kurdish Sign Language based on two lines of features obtained from the two for hand shape representation, feature selections were employed, and an Artificial Neural Network classifier is used to recognize hand gestures with 98% accuracy [32]. Furthermore, The Restricted Boltzmann Machine (RBM) is used to recognize hand sign language from visual input. The input image is cropped five times, and the hand of these cropped images is recognized using a Convolutional Neural Network (CNN). Massey University Gesture Dataset 2012, ASL Fingerspelling Dataset from the University of Surrey's Center for Vision, NYU, and ASL Fingerspelling A datasets are used to train the proposed multi-modal model [18].
Every day, researchers in this field are working on identification of fingerspelling, hand shape, and emotion recognition. However, alphabetic recognition is in progress every day, because it is principle of word recognition and sentence translation systems. A deep neural network (DNN) model was presented to train 24,000 images of 24 signs from the American Sign Language alphabet and fine-tuned using 5200 images of 26 generated signs. The system was then enhanced using a bidirectional long-short-term memory neural network that scored 98.07 percent training accuracy with a lexicon of 370 words. The presented American Sign Language alphabet translator (ASLAT) is based on a graphical user interface that includes a frame displaying real-time images captured by the camera, as well as three lines of text containing the SLR response of the current image, a sentence obtained from the sign sequence to word converter, and the corrected sentence obtained from word spelling correction (WSC) [8]. A British Sign Language (BSL) fingerspelling alphabet was investigated to recognize words from video. While in BSL, both hands participate in the recognition process. This is a challenging task compared to the other sign languages. On a dataset of 1,000 low-quality webcam images, the study obtains a word recognition accuracy of 98.9% [21]. In addition, a letter-based Kurdish Sign Language (KuSL) is presented to recognize hand gestures using three algorithms: scale invariant feature transform (SIFT), speeded-up robust features (SURF), and a novel approach called Grid based gesture descriptor. The new technique proved to be the best way for recognizing Kurdish hand signals, with recognising hand gestures in 67% accuracy while the first two algorithms have 42% accuracy. Even though the Kurdish Alphabet-Arabic script only has 36 letters, developing an effective Kurdish sign language recognition system is difficult owing to imaging condition differences such as position, lighting, and occlusion.
1.3. Research Contributions
The proposed model perform KuSL recognition system uses CNN, the model translates images into Kurdish alphabets in real time.
The main contributions of the proposed research include:
-
The proposed model is the first approach for recognizing Kurdish letter based on static hand shape sign.
-
Design a novel fully labelled dataset for KuSL. The dataset obtained from ASL and ArSL2018 based on hand shape recognition will be publicly available and free for all researchers.
-
Alphabet recognition based only one hand shape, without the need of motion signs.
-
An approach for generating real-time KuSL systems using CNN that has a high accuracy based on various users.
The remaining section of the paper is arranged as follows: The Materials and Methodology in Section 2. Then, section 3 contains Results and Discussion. Finally, conclusions and future work will be presented in Section 4.