Human Behavior Recognition in Surveillance Systems
- Topics: Surveillance
- Words: 3096
- |
- Pages: 7
- This essay sample was donated by a student to help the academic community. Papers provided by EduBirdie writers usually outdo students' samples.
Download
Short on time?
Get essay writing helpDownload
This paper is a literature review that focuses on detecting and analyzing the human behavior and its application in surveillance systems. Surveillance systems play a very important role in tracking and monitoring human behavior generally, this is one of the reasons why it has recently become a major interesting research topic. Human effort is not very effective in monitoring human behavior via surveillance systems as human operators miss a lot of information in different video frames.
Sometimes, it is difficult to differentiate a normal behavior from an abnormal behavior as what is normal in a context may be abnormal in another context. Ordinary surveillance systems on their own cannot classify human behavior into normal or abnormal behavior and for this reason, they require machine learning techniques. With machine learning techniques, surveillance systems can detect different objects, find anomalies in the monitor screen automatically and alert the necessary authority.
We aim in this paper to write a literature review article aimed to provide a foundation of knowledge on human behavior recognition and establish similarities and differences between previous studies by analyzing major formative work in the field of surveillance systems.
Human action detection, motion tracking, scene modelling, and behavior understanding (human activity recognition and discovery of activity patterns) have received a lot of attention in the computer vision and machine learning communities (Popoola & Wang, 2012) due to the uprising of several fields that these tools can be implemented efficiently in.
Researchers have developed a large interest in the areas of computer vision and machine learning communities to analyze the problem of human behavior in video clips. This is because in the last decade, systems used to consist of a low-level/mid-level computer vision system (to detect and analyze moving objects such as humans or cars) and a higher level interpretation module (to categorize motion into behaviors, for example, a pointing gesture or a car turning right). However, there have been few efforts to understand human behaviors that have vast extent in time, particularly when they involve serious interactions between people (Oliver, Rosario, & Pentland, 2000). Therefore, our main goal of this paper is to focus on detecting and analyzing the human behavior and its application in surveillance systems.
Video surveillance systems have become more important in today’s life, especially with the developments of computer science, communication technology, and internet engineering. Governments install video surveillance systems that work 24/7 in major government buildings and important facilities (e.g. airports, borders, roads) to fight against crime and keep an eye on places where incidents such as theft and accidents are likely to occur (Mishra & Saroha, 2016). The increasing need of security and safety in different areas has pushed video surveillance systems to emerge from the first generation “dumb camera” surveillance that needs a human eye to evaluate its images to the second generation “computer‐linked camera” surveillance systems that evaluate its own video images and reduce the human factor in surveillance (Surette, 2005). It is difficult to monitor simultaneous events in surveillance displays due to many limitations, therefore efforts are being made to design intelligent surveillance systems that are capable of distinguishing between what is a normal or an abnormal behavior within the context, because a normal behavior in one context may be abnormal in another (Kale, Jawale, Dhende, Abak, & Kosht, 2019).
Intelligent surveillance systems can detect different objects and find anomalies in the monitor screen automatically. As a result, it provides security personnel with the quickest and best way to alert and provide useful information (Zhao et al., 2014). These intelligent surveillance systems use various machine learning techniques and methods in order to understand human actions/behaviors. However, understanding of human actions using machine learning is a complex, diverse, and challenging area that has received much attention within the past ten years. Applications have included video surveillance and human–computer interfaces (Popoola & Wang, 2012) in addition to many more. Previous surveys and the number of conferences, papers, and good surveys on detecting or tracking human motion and gestures as well as anomaly detection and behavior recognition or analysis show that it is a highly researched area.
Video surveillance generally involves acquiring and processing visual data from a scene, in order to detect target(s) along time and space so as to recognition suspicious behaviors in awkward situations and if possible, generate alarms. It begins with change detection and motion data capture for moving targets (using tracking or non-tracking methods), to enable successive high-level event analysis (Popoola & Wang, 2012).
As a result, we aim in this paper to write a literature review article aimed to provide a foundation of knowledge on human behavior recognition and establish similarities and differences between previous studies by analyzing major formative work in the field of surveillance systems in order to draw a conclusion on the best practices of recognizing human behavior in surveillance systems and the best machine learning method used.
The applications of security surveillance in content based video retrieval and annotation, human computer interaction, human fall detection, video summarization, robotics, etc., has made human behavior recognition an important area of research in computer vision. Video surveillance systems are employed in the monitoring and analyzing of human behavior and activities. The main aim of intelligent surveillance system is to detect suspicious human or object behavior (otherwise known as abnormal behavior) in certain scenes and provide real time notification to the relevant person (Kale, Jawale, Dhende, Abak, & Kosht, 2019). J (2015) defines an abnormal behavior to be one that deviates from usual known behavior. (Kale and Patil, 2016), states that human behavior recognition “is a systematic approach to understand and analyze the movement of people in a camera captured content”.
The analysis of human behavior with the use of surveillance systems involves a pre-processing phase that detects the foreground and the background, tracking, feature extraction and then a machine learning method for classifying behavior into normal behavior or abnormal behavior (Farzad & Asli, 2015). Machine learning, as a branch of artificial intelligence, enables machines to perform given tasks skillfully with intelligent software (Mohammed, Khan, & Bashier, 2016).
Supervised learning: Supervised learning is basically concerned with labeled data. Hence, it is easy to apply its algorithms to pattern classification and data regression problems. It is used to discover mapping rules for predicting outputs of unknown inputs (Wang & Sng, 2015).
Unsupervised learning: Unlike supervised learning, unsupervised learning is concerned with unlabeled data. Unsupervised learning can be used to handle intra-class variation as it does not require labels of data. This learning algorithm deals with a form of network learning that gives correct response without any interference of an external behavior (Mishra & Saroha, 2016).
Semi-supervised Learning: Semi-supervised learning is a combination of both supervised learning and unsupervised learning. (He & He, 2018) proposed an end-to-end Unsupervised Multi-Object Detection framework for video surveillance, where a neural model learns to detect objects from each video frame by minimizing the image reconstruction error.
Clustering Algorithms: Clustering refers to the process of data gathering with identical features into groups. These groups are otherwise known as clusters (Al-Dhamari, Sudirman, & Mahmood, 2017). Abnormal detection with the aid of clusters is made possible using a clustering algorithm to analyze and then classify the data. (Lei, Jiang, Wu, & Du, 2016) further classified clustering algorithms into hierarchical clustering, partitioning clustering, model-based clustering, density-based clustering, grid-based clustering, etc.
Classification: A classification technique is used to manage or group of different objects individually. Object classification detect object and provide information that is needed for video analytics. It gives more detailed description of events or behaviors of objects. Object classification is useful for detection accuracy and creates high level metadata that describes another data in the video sequence (Mishra & Saroha, 2016).
Candamo, Shreve, Goldgof, Sapper, and Kasturi (2010) presented a survey research on human behavior recognition algorithms for surveillance in transit scenes with image processing techniques. The methods of detection Candamo et al. (2010) researched includes slow movement (individual), fighting (for multiple person) individual-vehicle recognition and object recognition. It was stated in their research that algorithms for human behavior recognition can aid in averting certain occurrences.
Popoola and Wang (2012) in their review on video based abnormal behavior recognition pointed out some methods of detecting abnormal human behvior using machine learning techniques.
In this paper, we review research works in the area of detecting and analyzing human behavior together with its application in surveillance systems for a period of nineteen years (19) years, i.e., only research work done between 2000 and 2019 were selected. Information gathered was done by studying the abstract of different research works including journals, conferences materials, academic articles and surveys that have to do with – “human behavior”, “intelligent surveillance systems”, “abnormal behavior detection”, “machine learning in surveillance systems” and then extracting necessary information for our research purpose. Forty (40) research works were reviewed and are all documented in English language including journals from IEEE, Engineering and Technology International Journal of Computer and Information Engineering, Journal of Engineering and Applied Sciences, etc.
Abnormal behaviors can sometimes be difficult to detect in video surveillance by a human observer (Sultani, Chen, & Shah, 2018), since there are no clear distinctions of abnormal behaviors from normal behaviors in certain circumstances especially in emergency situations like traffic accidents, crimes and terrorist attacks (Shao, Cai, & Wang, 2018). In other words, the abnormal behavior to be detected is determined by the defined normal behavior (Eisa & Moreira, 2017). Walking, running or sitting down are common normal behaviors that we exhibit as humans in our daily life (Chaaraoui, Padilla-Lopez, Ferrandez-Pastor, Nieto-Hidalgo, & Florez-Revuelta, 2014). However, abnormal behaviors may include abandoned object detection, suspicious human behavior, and intruder detection (Kotkar & Sucharita, 2017).
We can classify abnormal human behavior recognition into two dimensions – individual abnormal behavior recognition and crowd abnormal behavior recognition (Dhiman & Vishwakarma, 2018).
Meng, Miao, and Leung (2017) presented an Online Daily Habit Modeling and Anomaly Detection (ODHMAD) model with a list of defined activities to be detected and their various sensors. It provides online analysis of old person’s sensory behavior data for daily behavior detection.
In order to detect behaviors (normal and abnormal), visual data need to be obtained from the surveillance system for the main purpose of recognizing suspicious behaviors and alert the authority (J, 2015). It starts by tracking targets (with the use of either tracking or non-tracking methods) when there is a change in motion for analysis. Detecting abnormal behavior involves the search of unexpected problem in a data pattern (Kotkar & Sucharita, 2017).
According to Al-Dhamari et al. (2017), sensors are major components for surveillance systems. They aid in sighting targets that exhibit abnormal behaviors. Visible Light Camera (VLC), Thermal Camera (TC), Forward Looking Infrared (FLIR) camera are examples of sensors used in surveillance.
Feature extraction is done to detect area of interest of human movement (Xu et al., 2018), after which the ectracted features are then trained or tested with model and finally classified into normal or abnormal behavior.
Some machine learning techniques that can be used to detect and classify abnormal human behaviors include:
The Auto-Encoder acquires similar structures of normal video sequences and does estimation of normal patterns.
Video surveillance systems have wide range of applications like traffic monitoring and human activity understanding (Harish, Swati, Sonali, & Sangmesh, 2015), hospital for monitoring patient, in industry and process plant to monitor the activity of the production line (Jadhav, Suryawanshi, & Jadhav, 2017). Rai, Husain, Maity, & Yadav (2018), says that governments generally use surveillance systems during investigations and for the protection the masses.
In this paper, we did a literature review on the existing knowledge of human behavior recognition and its applications in the field of surveillance systems. We introduced the most significant machine learning techniques and methods used in detecting and classifying abnormal human behaviors.
Obviously, based on our study in the previous sections, we can see that human behavior detection algorithms face several challenges such as:
Having these challenges and many more shows that solving the abnormal behavior detection issue is not straightforward. Using a certain method in one environment/application does not mean that it will work well in other applications. Therefore, there is a need to always examine the performance of machine learning abnormal detection methods in order to improve its work within the wanted context.
Applying these methods in surveillance systems was also among the topics that we discussed throughout our paper. We firstly introduced surveillance systems first generations (dumb camera) that needs a human eye to evaluate its images. Then, we focus on the second generation “intelligent surveillance systems” that are capable of distinguishing between what is a normal or an abnormal behavior and can detect different objects automatically. As we mentioned in previous sections, the detection of abnormal behaviors using surveillance systems is used in several applications such as traffic monitoring and preventing crimes. Data obtained from intelligent surveillance systems experience pre-processing phase that detects the foreground and the background, feature extraction, and then one of the mentioned machine learning method is applied for classifying behaviors.
At the end, it is of note that more work should be devoted in order to build up real-time intelligence surveillance frameworks that can particularly overcome the scalability of video analysis especially in the real videos that contain a lot of moving objects, activities, and a crowd.
EduBirdie considers academic integrity to be the essential part of the learning process and does not support any violation of the academic standards. Should you have any questions regarding our Fair Use Policy or become aware of any violations, please do not hesitate to contact us via support@edubirdie.com.