Intelligent surveillance video analysis

With the development of wireless technology, surveillance video technology is constantly updating its iterations, and has entered the development process of digitalization, high definition and intelligence from the analog era. Intelligentization involves the processing of surveillance video images. In plain language, the goal of surveillance video image processing is to “see clearly” the monitored scenes, including denoising, deblurring, de-jittering, de-smog, and super-resolution reconstruction. Measures such as dynamic range expansion; the goal of surveillance video image analysis is to "see clearly" the objects of monitoring, including license plate recognition, face recognition, and human flow statistics; the goal of intelligent surveillance video analysis is to "understand" the content of the surveillance scene. Semantics, which gives the results of analysis that people can understand, such as what is happening here, what is here. It includes a variety of intelligent analysis techniques, such as behavior recognition, intrusion detection, object detection, group behavior recognition, and illegal vehicle detection.

In these three levels, it is clear that intelligent surveillance video analysis is our ultimate goal, instead of artificial observation, analysis and judgment of video surveillance content, even more accurate and faster than our humans in some aspects. However, in order to achieve this goal, it is necessary to provide intelligent analysis systems with as clear a video image as possible, as accurately as possible the result of target segmentation and recognition. We call these technologies the basic technology of intelligent video analysis, namely video image processing. And analysis. If the images provided do not meet this requirement, it is difficult to expect the intelligent analysis system to obtain the correct analysis results. Therefore, we say that intelligent video analysis is important, and the specific method of intelligent analysis is important, but the basic technology of providing video images is especially important in the initial development stage of current video analysis intelligence.

At present, from the application point of view, although intelligent video analysis has made great progress, there is still too much content to research and develop. Intelligent video analysis technology is still in the "initial stage" of video processing and video analysis. This part is now The main focus of the application. The basic monitoring video processing and analysis technology includes many aspects. In addition to the classic video processing technologies such as image enhancement and image denoising, most of the current concerns are image defogging, dynamic range expansion, and super-resolution reconstruction. License plate recognition, face recognition, ..., and many other technologies. The following is an example of the VAIS (Video Analysis Instrument System), a well-known video analysis system in the video surveillance industry, focusing on the basic processing techniques of video image processing and video image analysis.

Video image processing

2.1 Defogging and anti-reflection treatment

Due to the atmospheric scattering effect, the video images acquired under foggy weather conditions are relatively blurred, which seriously affects the visual effect of the image. The main reason is that the target light is attenuated by the fog during the propagation process, resulting in the loss of image detail and insufficient definition. The participation of ambient light causes the image color to be distorted and the color tone is dull.

In order to improve such foggy images, video de-fogging and anti-reflection technology (referred to as fog-through technology) can be used to clear the images that are ambiguous due to fog and dust, and to extract more information contained in the images. Provides good conditions for the next step in intelligent analysis of images.

The current translucent treatment methods are mainly divided into two categories: foggy image enhancement and foggy image restoration. The foggy image enhancement method is relatively simple. It does not consider the cause of image degradation. It only processes the color of the image, which can effectively improve the contrast of the foggy image, highlight the details of the image, and improve the visual effect of the image, but may Cause a certain loss of information.

The foggy image restores the mechanism of image quality degradation in foggy days, establishes a fog image degradation model, and then uses the image restoration method to recover the foggy degraded image, compensates for the distortion caused by the degradation process, and obtains a good estimate of the fog-free image. , thereby improving the image quality of foggy days. This method is highly targeted, and the defogging effect is natural and the information loss is small. The key to the processing is the estimation of the parameters in the model.

2.2 Super-resolution reconstruction

As people's requirements for monitoring image quality are getting higher and higher, improving the resolution of surveillance images has become an urgent requirement for the entire security industry. The direct way to improve image resolution is to increase the sensor density of the acquisition device (such as a camera). However, on the one hand, high-density image sensors are relatively expensive and unaffordable in general applications; on the other hand, imaging systems are currently approaching the limit due to their sensor array density limitations.

Under such circumstances, an effective way to solve this problem is to improve the spatial resolution of the image by using a signal processing-based software method, that is, super-resolution (SR: Super-Resolution) image reconstruction, which will be acquired. Low resolution images are reconstructed into high resolution images by a certain algorithm. The core method of SR reconstruction of VAIS is to exchange spatial resolution with time bandwidth (acquisition of multi-frame image sequence of the same scene) to realize conversion of time resolution to spatial resolution, so that the visual effect of reconstructed image exceeds any frame low resolution. image.

2.3 Dynamic Range Expansion

In many video surveillance situations, images with color (brightness) dynamic range deviations are often generated due to light, and overall or partial "exposure" is insufficient or "exposure" is excessive. The processing of such video images is mainly to correct the deviation of the color dynamic range, including two aspects: the processing of the backlight image and the processing of the dark light (such as night scene) image.

(1) Processing of dark light images

The image acquired in the night scene is insufficient due to insufficient light intensity, resulting in reduced image brightness and contrast, loss of color and other details, which increases image noise, thereby reducing image quality, seriously affecting image usage and further intelligent analysis processing. At present, the popular method of dark-light image processing is the Retinex algorithm, which separates the luminance image and the reflection image from a given image, and enhances the ratio of the luminance image and the reflection image in the original image under the condition of constant color. The purpose of the dim light image. However, this method is very computationally intensive and difficult to apply to real-time processing. For this reason, the statistical analysis of the VAIS system shows that the inverted dark light image has similar statistical characteristics to the foggy sky image. Based on this, a new fast algorithm based on image defogging method to expand the dynamic range of the image is designed. The inverted dark image is processed, and after the processing is completed, the image is expanded to expand the dynamic range.

(2) Processing of backlit images

During the video capture process, if the camera is facing the light source, this part is very bright, and the target object in the scene is much darker than the light source. Due to the limitation of the dynamic range of the camera, the brightness of the target area is very dark, and the details are not visible. This is the so-called backlight phenomenon. Obviously, the key to the backlight image compensation process is to improve the brightness of the target area, reduce the brightness of the light source part, and ensure the smooth transition of the natural transition of the entire image. The VAIS system uses a method similar to dim light image processing for backlight images, that is, a method based on image defogging to process a backlit image.

Third, video image analysis

3.1 Face recognition

Face recognition technology based on image processing can be divided into two categories. One is cooperative face recognition, which is the common identification method for attendance, access control, conference sign-in and other occasions. Currently, this type of technology has been compared. Mature; the other is non-cooperative face detection and recognition technology. Zui is typically the face detection and recognition of target characters in surveillance video. This type of technology is not perfect enough, and there are still many problems to be solved.

At present, most of the applications of surveillance video are non-cooperative face detection and recognition. Due to the inability to obtain the cooperation of target personnel and various adverse environmental factors, the accuracy of face images in video is not high. The face is blurred and the angle changes greatly, which makes it difficult to perform high-precision face detection and face recognition.

The VASI system focuses on non-cooperative face recognition technology, which is dedicated to algorithm optimization in face recognition of small sample libraries, and thus has high face detection and recognition rate. The VAIS system transmits all the face images captured and detected in the video scene to the face recognition part of the VAIS, reconstructs the blurred faces using the WSR and LCR algorithms, and then performs more with the face data of the VAIS system database. The second comparison screens out one or more face data close to the Zui for the user to confirm the target identity, or plays a role of prompting and alerting the user to remind the user to perform further identity verification. This method of face recognition is very useful in many occasions, such as preliminary identification and prompting of VIP or blacklist users in hotels, banks, airports, terminals, customs, etc.

3.2 License plate recognition

The license plate recognition technology mainly includes two types, one is the relatively stationary bayonet license plate recognition, and the other is the license plate recognition of the vehicle in motion.

For the license plate recognition of the parking bay type, the vehicle is relatively slow or slow, and the vehicle is relatively easy to detect and the recognition rate is high. The VAIS system recognizes such license plates. As long as the license plate image captured by the camera reaches 30 pixels or more, the license plate number can be accurately identified, and the accuracy is almost the same.

For the identification of license plates of vehicles in motion, such as vehicles on the road, even escaped vehicles, due to the complex environment, high-speed sports, other vehicle occlusion and other unfavorable factors, greatly increasing the technical difficulty of license plate segmentation and car number identification. . In this case, the VAIS system utilizes the associations and constraints existing between the targets in the monitoring scene, and can identify most of the fuzzy license plate numbers with relatively high accuracy. At present, VAIS has successfully assisted the department to quickly detect different levels of difficult cases from license plate recognition. In the future, with the continuous improvement of VAIS fuzzy license plate automatic recognition technology, the accuracy of license plate recognition will continue to increase.

FIG. 5 is an example of automatic identification of a suspect vehicle license plate in a cell. After the VAIS system is used to segment the fuzzy license plate, the running identification algorithm automatically outputs the license plate number to 624M4. Fig. 6 is an example of automatic recognition of a vehicle license plate for a fast-moving vehicle on a highway, and the identified license plate number is 422AF.

Fourth, the conclusion

In the rapid development of intelligent surveillance video analysis technology, when we encounter difficulties or errors, the source of the problem is still the poor quality of the image itself provided to the intelligent analysis system, such as the image is not clear, the target division is not accurate. The image content is dim, etc. Therefore, strengthening the processing and preliminary analysis of the video image is one of the necessary conditions for obtaining the accurate result of the intelligent video analysis.

Above we briefly introduced several important video image processing and analysis techniques, including image fogging, equalization of dark and backlit images, face detection and recognition, license plate detection and recognition technologies, etc. It is the basic function of the VAIS intelligent video analysis system, and has achieved good results in practical applications. In addition to the special video processing methods described above, there are a variety of other pre-processing methods, such as processing of the region of interest, anti-shake processing, dynamic target locking processing, tracking and analysis of the target.

Only by laying the foundation for video processing and analysis, it is possible to conduct intelligent video analysis on this basis. VAIS is working hard in this direction, combined with massive video data (big data) mining, using artificial intelligence machines. Learning methods, especially the deep learning methods of the conclusion artificial neural network, are step by step to improve the speed and accuracy of intelligent surveillance video analysis, step by step to practical.