Implementation of Kinect Sensors on Mobile Robots

Study Case Implementation of Kinect Sensors on Mobile Robots for Inspecting Objects Containing Chemicals Michael Kurniawan (2018123456) University of Cincinnati, 2600 Clifton Ave, Cincinnati, OH 45221, United States kurniaml@mail.uc.edu Abstract The threat of explosions caused by suspicious objects remains a serious problem. Traditional methods for disarming bombs, such as the use of special explosive-proof vests, still pose a high risk to humans. This study developed a Mobile Robot equipped with a Kinect Sensor, MQ-5 Gas Sensor, HSV method, and image processing. Testing showed that the robot was able to inspect the color of the target object using the Kinect sensor and map the iso-butane gas levels at a specific location with the MQ-5 sensor. Image processing results show that for green objects using the HSV method, the hue value is 29–84, saturation is 86–255, and value is 6–255 with 2000 lux illumination. When the room illumination is only 80 lux, the robot misdetects green objects as yellow. Gas sensor data shows that wind direction and speed >5 km/h affect gas level reading fluctuations. The Kinect depth sensor is capable of measuring the distance between the mobile robot and objects within a range of 40–400 cm. Keywords: HSV method, Kinect sensor, MQ-5 sensor. INTRODUCTION In today's technological era, crime and terrorism rates are on the rise. Terrorists use technology to assemble and detonate bombs, often using easily obtainable hazardous chemicals. Based on data from official sources in September 2015, the number of perpetrators of terrorist crimes prosecuted up to that month was 1,143, with details as follows: 501 released, 328 serving sentences, 98 returned to their families, 127 deceased (108 due to law enforcement, 3 executed, 16 suicides) [1]. MICHAEL KURNIAWAN 1 Study Case To reduce the risk of human accidents, the latest technology in the form of mobile robots can replace dangerous tasks, such as detecting objects containing chemicals in suspicious items. Some existing mobile robots still have limitations, for example, in approaching targets accurately or lacking cameras to detect objects [2]. In addition, some robots are not equipped with semiconductor gas sensors for gas content inspection. Based on these conditions, this study proposes a mobile robot that can move on flat surfaces, approach target objects using Kinect sensors, and detect iso-butane levels with MQ-5 gas sensors. THEORETICAL BASIS Kinect Sensors Kinect was originally developed for the Xbox 360 gaming platform as a motion sensing device. Kinect has an RGB sensor, depth sensor, and microphone array (see Figure 1), which enable 3D motion capture, facial recognition, and voice recognition. Kinect has good performance, small dimensions, open source library support for Windows, Linux, and Mac, and a relatively affordable price [3]. The Kinect sensor is capable of recording video at a frame rate of 60 Hz, combining the output from an RGB camera (8-bit VGA, 640×480 pixels, Bayer filter) and a monochrome depth sensor (same resolution, 2048 sensitivity levels/11-bit) [3]. Figure 1. Components of the Kinect sensor [3]. MICHAEL KURNIAWAN 2 Study Case Figure 2. HSV color space model [3]. Images with HSV Color Space HSV is a color model that resembles human perception. • H (hue): represents color (e.g., red vs. pink). • S (saturation): color intensity. • V (value): amount of light, distinguishing, for example, dark red vs. light red . The HSV model can be seen in Figure 2. Dilation and Erosion Filters • Dilation: addition of pixels around an object. The output pixel value is the maximum of all input values. In a binary image, input 1 → output 1 [4]. (see Figure 3) • Erosion: reduction of pixels around an object. The output pixel value is the minimum of all inputs. In a binary image, input 0 → output 0 [4]. (see Figure 4) Figure 3. Dilation on a grayscale image [4]. MICHAEL KURNIAWAN 3 Study Case Figure 4. Erosion on a grayscale image [4]. MQ-5 Sensor Figure 5 shows the MQ-5 gas sensor and its module. Although not as sensitive as some other gas sensors, the MQ-5 is universal and capable of detecting various types of gases: H2, CO, CH4, C2H5OH, C3H8, C4H10, and other hydrocarbon gases. • Rs: sensor resistance at a specific gas concentration • Ro: sensor resistance at 1000 ppm H2 in open air Figure 5. (a) MQ-5 sensor, and (b) MQ-5 sensor module. Figure 6 shows the sensitivity characteristics of the MQ-5 sensor, as well as the effects of temperature and humidity. The y-axis shows the sensor resistance ratio (Rs/Ro). Figure 6. Characteristics of the MQ-5 sensor [5]. Table 1. Description of the MQ-5 gas sensor module [6] MICHAEL KURNIAWAN 4 Study Case Figure 7. Block diagram of the mobile robot system architecture. Figure 8. Hardware system design. SYSTEM DESIGN System Architecture The block diagram in Figure 7 shows a simplified mobile robot system for detecting objects containing hazardous chemicals. The robot uses a Kinect sensor to recognize MICHAEL KURNIAWAN 5 Study Case object colors, an MQ-5 sensor to detect gas levels, and a DC motor driver to control wheel speed based on pixel data sent from the Kinect. The purpose of the system design is for the robot to move in the direction of the object detected through HSV input values, approach the object, stop right in front of the object, and measure gas levels. In Figure 8, the mobile robot system is divided into three main parts: sensors, data processing, and control. The sensor section consists of Kinect to detect the target color and MQ-5 for gas content. The data processing section uses a microcontroller that processes input from sensors and a PC to control actuators, in the form of DC motors. Figure 9. Top view of mechanical design. Figure 10. Kinect sensor design. Mobile Robot Design The robot's mechanical design (Figure 9) uses two DC motors for the left and right rear wheels, as well as one omnidirectional front wheel so that the robot can maneuver well in various directions. Kinect Design The Kinect v1 sensor uses serial communication via USB to the PC. The sensor voltage is supplied separately from the main power supply (AC 220V). The Kinect v1 sensor design schematic can be seen in Figure 10. MQ-5 Gas Sensor Design The MQ-5 sensor module generates voltage when it detects iso-butane gas in the air. The output pin is connected to A0 Arduino Uno, with a 5V DC supply and ground. The sensor consists of a metal oxide semiconductor layer on an alumina substrate, equipped with an integrated heater. The heater typically uses 5V and requires ~3 minutes to reach stability before gas readings. MICHAEL KURNIAWAN 6 Study Case Figure 11. MQ-5 sensor design. Figure 12. Mobile robot system design implementation. RESULTS AND DISCUSSION Mobile Robot Mechanical Design Implementation The mechanical design (Figure 12) integrates the MQ-5 sensor, Kinect v1, DC motor driver, 12V motor, battery, microcontroller, and PC. All components are arranged to support the mobility of the robot, sensors, and data processing. Image Processing Testing Figure 13. Original image. MICHAEL KURNIAWAN 7 Study Case Figure 14. Image resulting from RGB to HSV conversion. Figure 15. Image resulting from dilation and erosion filters. Figure 16. Image resulting from final feature extraction. Testing was conducted by setting a threshold value to obtain a binary image: green in RGB is displayed, while other colors become black. The threshold value was adjusted according to lighting conditions. The processing results can be seen in Figures 13–16, including the ADC reading from the MQ-5 sensor. HSV Color Value Testing Testing was carried out by adjusting the hue, saturation, and value ranges. For example, green was detected with: • Hue: 29–84 • Saturation: 86–255 • Value: 6–255 under 2000 lux lighting. The results can be seen in Table 2 and Figures 17–19. MICHAEL KURNIAWAN 8 Study Case Figure 17. HSV value input testing. Figure 18. Green object under 2000 lux lighting. MICHAEL KURNIAWAN 9 Study Case Figure 19. Lighting test results using a lux meter on a smartphone. Distance Testing Using the Kinect Depth Sensor Distance testing using the Kinect Depth Sensor (Figures 20–22) shows: • Minimum distance: 40 cm • Maximum distance: 400 cm The measurement data compared to manual measurements using a ruler shows high precision results. Figure 20. Depth sensor testing implementation. Figure 21. Kinect depth sensor distance testing graph. Figure 22. Depth sensor interface display. MICHAEL KURNIAWAN 10 Study Case MQ-5 Sensor Testing The MQ-5 sensor was tested using iso-butane gas. The sensor was calibrated in neutral air to obtain the Ro value. The test was conducted in a room at 25°C with the air conditioner set to level 3. Changes in Rs/Ro, ppm, and gas concentration over time t (seconds) were recorded. • Distance between gas tube and sensor: 1 meter • PPM value when detecting gas: 1129.27 • PPM value without gas: 45.14 Figure 23. Mobile robot testing with iso-butane gas. Figure 24. PPM data graph against time. Figure 25. Rs/Ro graph against time. MICHAEL KURNIAWAN 11 Study Case CONCLUSION Based on the research results: 1. The Kinect sensor detects green objects with HSV: hue 29–84, saturation 86– 255, value 6–255 at 2000 lux illumination. Errors occur at low illumination (80 lux). 2. The Kinect depth sensor detects a minimum distance of 40 cm and a maximum distance of 400 cm. 3. Wind speed and direction affect the MQ-5 sensor readings. The sensor can map iso-butane levels up to ±1 meter in a vacuum. PPM values increase in accordance with gas concentration. REFERENCE MICHAEL KURNIAWAN 12 Study Case MICHAEL KURNIAWAN 13