Extraction of Voice Parameters for AI-Based Human Machine Interaction

Study Case Extraction of Voice Parameters for AI-Based HumanMachine Interaction Michael Kurniawan (2018123456) University of Cincinnati, 2600 Clifton Ave, Cincinnati, OH 45221, United States kurniaml@mail.uc.edu Abstract In the field of Artificial Intelligence, machines are designed to work and behave like humans by adopting the principle that every aspect of intelligence can be learned and realized in a system. One of the most important forms of human communication is voice or speech. In order for human voice to be learned and represented in a digital system, voice signals must be processed and extracted into a number of relevant acoustic parameters. Because each individual's voice characteristics are unique, the signal extraction results will produce different parameters. Various artificial intelligencebased applications utilize this voice processing, such as Speech Recognition and Speaker Recognition systems. This paper examines the Linear Predictive Coding (LPC) method as one of the main techniques for generating voice parameters based on words spoken by various speakers. These parameters can be used in speech recognition systems to control electronic devices through voice commands, as well as in speaker recognition systems for user authentication, for example in voice-based password mechanisms. Keywords: Voice signal processing, LPC, ANN, AI INTRODUCTION Artificial intelligence (AI) is developed by imitating the way humans learn, namely through a process of adaptation or training. This paper discusses the application of AI in machines that are capable of recognizing human voices through a process of acoustic pattern learning (Machine Learning), thereby creating interaction between humans and machines (Human–Machine Interaction). To simulate the process of human voice MICHAEL KURNIAWAN 1 Study Case formation, this study uses the Linear Predictive Coding (LPC) algorithm, while the recognition process uses Artificial Neural Network (ANN) to mimic the mechanism of human neural networks. Human voice signals carry various information such as language structure, intonation, and the speaker's emotional expression. The exchange of information through voice is very important in everyday life. Language structure and acoustic characteristics of speech also reflect a person's cognitive abilities and socio-cultural development [1], [2]. The process of human voice formation involves three main components: the power source, the vibration-forming organs, and the resonance cavity. Air from the lungs as the power source can exit through the mouth and produce oral sounds, or through the nose to produce nasal sounds. In some conditions, air can flow through the mouth and nose simultaneously, producing a nasalized sound. Figure 1 shows a diagram of the articulatory organs that play a role in forming speech sounds. Figure 1. Diagram of the mechanism of human speech formation. The character of the sound produced depends on the position and condition of the vocal cords. When the vocal cords are closed, the sound produced is unvoiced, for example [s]. Conversely, when the vocal cords are relaxed, the airflow is freer and produces a voiced sound, for example [a]. After passing through the pharyngeal cavity, the air flows to the upper throat. If the uvula closes the path to the nasal cavity, the air will exit through the mouth and produce sounds such as [p], [g], and [f]. If the uvula is lowered, the air flows into the nose and produces nasal sounds such as [m] and [n]. MICHAEL KURNIAWAN 2 Study Case Based on this mechanism, a number of parameters are needed to model speech formation. The speech mechanism model is shown in Figure 2(b). In this model, the vocal cords are modeled through two types of excitation functions: pulses (for voiced sounds) and random noise (for unvoiced sounds). The power source is represented as gain, while the resonance cavity is modeled as a filter. Because the filter in the synthesis model is all-pole, this model is also known as an autoregressive (AR) model [3]. The LPC model is expressed in the z-domain as: 𝑆(𝑧) = 𝐸(𝑧) ⋅ 𝐺 ⋅ 1 𝐴(𝑧) (1) with the filter function: 𝑝 𝐴(𝑧) = ∑ 𝑎(𝑖)𝑧 −𝑖 , 𝑎(0) = 1 (2) 𝑖=0 and the z-transform relationship: 𝑆(𝑧) ⟷ 𝑠(𝑛) (3) Figure 2. (a) Diagram of human speech formation, (b) Speech synthesis model. The discrete signal s(n) is the result of the inverse z-transform of S(z). For a normalized discrete period T=1, the following applies: 𝑆(𝑧) ⟷ 𝑠(𝑛 The same applies to excitation: 𝐸(𝑧) ⟷ 𝑒(𝑛) If the sound is voiced, e(n) is a periodic pulse. If unvoiced, e(n) is random noise. After being amplified by factor G, this signal passes through the all-pole filter 1/A(z) and produces the speech signal s(n). The LPC analysis model is obtained from the equation: MICHAEL KURNIAWAN 3 Study Case 𝐸(𝑧) = 1 𝑆(𝑧)𝐴(𝑧) 𝐺 (4) With LPC parameters in the form of filter coefficients, gain, pitch period P, and sound type (voiced/unvoiced). In LPC simulation, the analysis process is divided into two parts: 1. LPC Analysis Algorithm to calculate filter coefficients and gain. 2. Pitch Algorithm to calculate pitch period and determine sound type [4]. The LPC parameters generated by each person are different, so they can be used for voice recognition. The system will extract LPC parameters from the incoming voice signal, store them in a digital database, and compare them with the stored data for the identification process. Artificial neural networks (ANN) are one approach to speech recognition designed to mimic the function of the human brain. ANNs learn by gradually updating the weights at the synapses [5]. ANNs are able to learn voice parameters and match them to stored patterns. If a new pattern appears, the ANN stores it as additional data. Figure 3 shows a block diagram of the speech learning and recognition process. Figure 3. Diagram of the learning and speech recognition process. In this system, each incoming speech signal is extracted using LPC. The extraction results are studied by the ANN and stored in a pattern database. New data that is tested will be compared with existing patterns; if it matches, the speech is recognized, if not, the ANN will add a new pattern to the database. Thus, the system continues to develop as more voice data is added. MICHAEL KURNIAWAN 4 Study Case RESEARCH METHOD The research method used in this paper focuses on the application of the Linear Predictive Coding (LPC) algorithm. Through this method, a set of human voice parameters is obtained that correspond to the form of words spoken by various speakers. These parameters can be used for the development of speech recognition systems, speaker recognition, text-to-speech recognition, and various human-machine interaction applications based on speech signals. Speech Signal Extraction To generate unique voice parameters, the extraction process is carried out using an LPC analysis model. There are two main algorithms in the analysis stage, namely the LPC analysis algorithm and the pitch period algorithm. The LPC analysis algorithm is used to calculate the filter and gain coefficients, while the pitch algorithm is used to determine the pitch period and voiced/unvoiced parameters. The flowchart of the LPC parameter extraction process is shown in Figure 4, with the following steps: 1. Frame formation 2. LPC analysis algorithm: a. Calculation of prediction coefficients b. Calculation of reflection coefficients c. Calculation of gain 3. Pitch period algorithm: a. Determination of the pitch period b. Determination of sound type (voiced/unvoiced) According to classical research by Markel [3], the frame length for human speech signals ranges from 15 ms to 40 ms. For example, for a 1-second utterance with a frame length of 30 ms, approximately 33 frames will be obtained. MICHAEL KURNIAWAN 5 Study Case Figure 4. Speech Signal Extraction Procedure Frame Formation Procedure If the sampling frequency used is 8 kHz, then a 1-second speech signal produces 8,000 sample points. With a frame length of 30 ms, each frame contains approximately 240 samples. The procedure is as follows: • First frame: n = 0 to n = 239 • Second frame: n = 240 to n = 479 • And so on until the 33rd frame The flowchart of the frame formation process is shown in Figure 5. Figure 5. Frame Formation Procedure MICHAEL KURNIAWAN 6 Study Case LPC Analysis Algorithm Procedure This stage calculates the prediction coefficient, reflection coefficient, and gain using the autocorrelation method. The autocorrelation process flow diagram is shown in Figure 6. The steps are as follows: 1. Calculate autocorrelation based on: 𝑁−1−𝑖 1 𝑟𝑖 = ∑ 𝑁 𝑠(𝑛)𝑠(𝑛 − 𝑖) 𝑛=0 where: p: prediction order r_i: i-th autocorrelation value N: number of samples per frame s(n): n-th sample 2. In the initial stage (p = 0), the prediction coefficient is selected as a_0 (0)=1, so that we obtain: 𝐸0 = 𝑟0 3. For the next stage (1 ≤ p ≤ maximum order): a. Set a_p (0)=1 b. Calculate the reflection coefficient:. 𝑘𝑝 = − 1 𝐸𝑝−1 𝑝−1 ∑ 𝑎𝑝−1 (𝑖)𝑟𝑝−𝑖 𝑖=0 c. Determine the prediction coefficient: 𝑎𝑝 (𝑝) = 𝑘𝑝 d. Calculate the error energy: 𝐸𝑝 = 𝐸𝑝−1 (1 − 𝑘𝑝2 ) 4. Update the prediction coefficient: 𝑎𝑝 (𝑖) = 𝑎𝑝−1 (𝑖) + 𝑘𝑝 𝑎𝑝−1 (𝑝 − 𝑖) 5. Calculate the gain: 𝐺 = √𝐸𝑝 MICHAEL KURNIAWAN 7 Study Case Figure 6. LPC Algorithm Analysis Procedure Pitch Period Algorithm Procedure This algorithm is used to determine the pitch period and the voiced/unvoiced sound type. The flowchart is shown in Figure 7. Referring to Rabiner's method [4], the steps are: 1. Calculate signal autocorrelation: 𝑁−1−𝑖 1 𝑟𝑖 = ∑ 𝑁 𝑠(𝑛)𝑠(𝑛 − 𝑖) 𝑛=0 2. Normalize to r(0): 𝑟(𝑖) 𝑟(0) 3. Find the maximum value in the range i = 24–120 (human pitch 3–15 ms, fs = 8 kHz). 4. Decision: a. If r(i_“max” )/r(0)<0.25: unvoiced, pitch = 0 b. If ≥ 0.25: voiced, pitch = i_“max” MICHAEL KURNIAWAN 8 Study Case Figure 7. Pitch Period Algorithm Procedure Speech Recognizer This program is used to reconstruct speech signals based on the extracted parameters. Each frame is analyzed until the last frame. 1. If the frame is voiced, a pulse generator is used. 2. If unvoiced, a hiss generator (noise) is used. The excitation signal passes through gain amplification, then is filtered by an allpole filter (vocal tract filter) to produce a reconstruction signal. Figure 8. Speech Recognizer Procedure MICHAEL KURNIAWAN 9 Study Case RESULTS AND DISCUSSION Figure 9 shows a block diagram of the LPC analysis and synthesis system. The signal s(n) is processed by the LPC analysis algorithm into vocal tract parameters with order p = 10 (reflection coefficients k₁–k₁₀) and gain. The pitch algorithm produces pitch period and voiced/unvoiced parameters. The synthesis process then reconstructs the signal using the 1/A(z) filter. Figure 9. LPC Analysis–Synthesis Figure 10 compares the original signal s(n) with the reconstructed signal of the word “usai” spoken by a 31-year-old male for frames 1–4. There are a total of 33 frames, each containing 240 samples (30 ms, fs = 8 kHz). Reconstruction uses a bit rate of 2.4 kbit/s. Figure 10. Comparison of Original and Reconstructed Signals MICHAEL KURNIAWAN 10 Study Case Table 1. ASCII conversion of the original signal and the reconstructed signal in frames 1–4. In frame 4, there are approximately 5.5 pitch periods, so the pitch ≈ 240/5.5 ≈ 43 samples, which corresponds to the value in Table 1. In frame 3, there is a misinterpretation by the LPC system: although the original signal appears voiced, the system classifies it as unvoiced (v/uv = false, pitch = 0). This shows that in a single frame, the system can only select one type of v/uv. If a frame contains a mixture of voiced and unvoiced, distortion will occur, as in frame 2. Distortion can be reduced by shortening the frame length so that there is only one type of v/uv per frame. However, this results in an increase in the number of parameters that must be encoded, thereby increasing the bit rate. Similar observations occur in other words such as “word,” “medicine,” “fish,” “tail,” “reluctant,” “blue,” “various,” “satay,” “officer,” “school,” and “fire.” MICHAEL KURNIAWAN 11 Study Case CONCLUSION The analysis results show that the LPC method is capable of extracting speech signals into parameters that are unique to each word and each speaker. These parameters can be applied to various applications such as speech recognition, speaker recognition, text-to-speech, and other human-machine interaction applications. The learning and pattern recognition processes can be improved using Artificial Neural Networks (ANN), which is one of the main concepts in artificial intelligence (AI). REFERENCE MICHAEL KURNIAWAN 12