CN106355171A - Video monitoring internetworking system - Google Patents

Abstract

The invention provides a video monitoring internetworking system. The video monitoring internetworking system is used for identifying personnel by two types of voices and images. The video monitoring internetworking system comprises a collection system, a voice recognition system and an image recognition system, wherein the collection system is used for collecting the voices and images; the voice recognition system comprises a dictionary scene voice module, a similarity comparison module and a voice recognition engine module; the image recognition system comprises a preprocessing module, a feature extraction module, a training module, a re-recognition module and an evaluation module. The video monitoring internetworking system has the advantage that the personnel can be effectively recognized.

Description

A video monitoring network system

technical field

The invention relates to the field of video monitoring, in particular to a video monitoring networking system.

Background technique

Video surveillance is an important part of the security system. The traditional surveillance system includes front-end cameras, transmission cables, and video surveillance platforms. Cameras can be divided into network digital cameras and analog cameras, which can be used as front-end video image signal collection. It is a comprehensive system with strong preventive capabilities. Video surveillance is widely used in many occasions because of its intuition, accuracy, timeliness and rich information content. In recent years, with the rapid development of computer, network, image processing and transmission technology, video surveillance technology has also made great progress.

Contents of the invention

The invention aims to provide a video monitoring network system capable of quickly and effectively identifying personnel.

The object of the present invention adopts following technical scheme to realize:

Provided is a networked video monitoring system capable of identifying personnel through voice and image, including a collection system, a voice recognition system and an image recognition system, the collection system collects voice and images, and the voice recognition system The system includes a dictionary scene speech module, a similarity comparison module and a speech recognition engine module, and the image recognition system includes a preprocessing module, a feature extraction module, a training module, a re-identification module and an evaluation module; the preprocessing module is used to determine pedestrians The position of the person in the image is to obtain a rectangular area containing the person; the feature extraction module is used to extract appearance features in the rectangular area containing the person; the training module is used to train multiple cross-modal projection models, each The cross-modal projection model contains two projection functions, which respectively map the images in different cameras into a common feature space and complete the similarity calculation; the re-identification module is used to identify whether the database contains and query Pedestrian images with the same personnel and confirm the identity of the query personnel; the evaluation module is used to evaluate the performance of the system.

The beneficial effect of the present invention is that the effective identification of personnel is realized.

Description of drawings

The present invention is further described by using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. For those of ordinary skill in the art, without paying creative work, other embodiments can also be obtained according to the following accompanying drawings Attached picture.

Fig. 1 is a schematic diagram of structural connection of the present invention.

Reference signs:

Acquisition system 1, voice recognition system 2, image recognition system 3.

detailed description

The present invention is further described in conjunction with the following examples.

Referring to Fig. 1, a kind of video surveillance networking system of the present embodiment can identify people through voice and image, including acquisition system 1, voice recognition system 2 and image recognition system 3, and the acquisition system 1 is paired with Voice and image are collected, and described voice recognition system 2 comprises dictionary scene voice module, similarity comparison module and voice recognition engine module, and described image recognition system 3 comprises preprocessing module, feature extraction module, training module, re-identification module and An evaluation module; the preprocessing module is used to determine the position of the person in the pedestrian image, and obtains a rectangular area containing the person; the feature extraction module is used to extract the appearance feature in the rectangular area containing the person; the training module is used for Training multiple cross-modal projection models, each cross-modal projection model contains two projection functions, which respectively map images in different cameras into a common feature space and complete similarity calculations; the re-identification The module is used to identify whether there is a pedestrian image consistent with the queryer in the database and to confirm the identity of the queryer; the evaluation module is used to evaluate the performance of the system.

Preferably, the dictionary scene speech module is adapted to sequentially collect the dictionary and scene speech in the user vocabulary, and save the collected feature vector as a template;

The similarity comparison module is adapted to compare the feature vectors of the speech input speech signal with each feature vector template stored in the dictionary scene speech module in turn, and output the one with the highest similarity as the speech recognition result.

This alternative embodiment achieves effective identification of persons.

Preferably, the templates in the dictionary scene speech module include monitoring system term templates and human voice plus dictionary templates.

This alternative embodiment speeds up the recognition speed.

Preferably, the preprocessing module includes an image fusion unit, and the image fusion unit is used to fuse images from different sources so as to better obtain comprehensive features of the images, including: separate the two source images that need to be fused Use biorthogonal wavelet transform for wavelet decomposition to determine the wavelet coefficients of the decomposed image; select the wavelet coefficients of the decomposed image according to the set ratio for the low-frequency coefficients to form the wavelet low-frequency coefficient matrix of the fused image; use consistent texture for the high-frequency coefficients The property measure analyzes the edge characteristics of different high and low frequency coefficients in a specific area, calculates the texture consistency measure of the image area, and determines the high-frequency wavelet coefficient matrix of the fused image according to predetermined rules, and the calculation formula of the texture consistency measure of the image area is defined for:

$\mathrm{<span\; class="notranslate">\; E.</span>}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; 8</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; EF</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; EF</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; EF</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}$

In the formula, EF(x) represents the texture consistency measure of the image region x, EF _l represents the texture consistency measure of each high-frequency component image in the image region x in the horizontal direction, and EF _c represents each high-frequency component of the image region x The texture consistency measure of the image in the vertical direction, EF _d represents the texture consistency measure of each high-frequency component image of the image area x in the diagonal direction; the wavelet low-frequency coefficient matrix of the fusion image, the fusion image The high-frequency wavelet coefficient matrix is subjected to discrete biorthogonal wavelet inverse transform, and finally the fused image is obtained.

This preferred embodiment sets the image fusion unit, which can better distinguish the false edges of the image according to the texture consistency measure, and makes the detail information more abundant and real while ensuring the overall visual effect; defines the texture consistency measure of the image area The calculation formula speeds up the speed of image fusion.

Preferably, the predetermined rules include:

(1) If more than 88% of the pixel values in the image area have a larger texture consistency measure, define the image area as an edge area, and select the wavelet coefficient of the high-frequency image with the largest corresponding edge texture consistency measure to form the fusion image The high-frequency wavelet coefficient matrix of ;

(2) If more than 88% of the pixel values in the image area have a small texture consistency measure, define the image area as a smooth area, and calculate the energy and matching degree of the two source images in the image area respectively. According to the energy and matching Determine the proportion of the wavelet coefficients of the two source images in the wavelet coefficients of the fusion image, and determine the high-frequency wavelet coefficient matrix of the fusion image according to the following formula:

R _G =β _A R _A +β _B R _B

In the formula, R _G represents the high-frequency wavelet coefficient matrix of the fused image, _RA and β _A represent the wavelet coefficient of a source image and the proportion of the wavelet coefficient in the wavelet coefficient of the fused image, and R _B and β _B respectively Indicates the wavelet coefficient of another secondary source image and the proportion of the wavelet coefficient in the wavelet coefficient of the fused image, where β _A + β _B =1.

This preferred embodiment determines the high-frequency wavelet coefficient matrix of the fused image according to a predetermined rule, which improves the fusion effect and speed.

Preferably, the extraction of appearance features in a rectangular area containing people includes:

(1) Carry out the illumination normalization processing of image, specifically comprise: a, set image as I, utilize LOG logarithm to convert image I to logarithmic domain, utilize differential Gaussian filter to carry out smoothing process to image I; b, to image I Image I performs global contrast equalization processing;

(2) Carry out image size normalization processing;

(3) Carry out image segmentation, for each image block, carry out feature vector extraction;

(4) Concatenate the feature vectors of all image blocks, and then perform PCA feature dimensionality reduction on the concatenated images.

In this preferred embodiment, a feature extraction module is provided to perform illumination normalization processing on the image before feature extraction, which reduces image distortion caused by illumination changes and makes feature extraction more accurate.

Preferably, the training module includes a sample classification unit and a cross-modal projection model learning unit; the sample classification unit specifically performs:

Let the feature spaces corresponding to two cameras C ₁ and C ₂ be and d ₁ and d ₂ represent the dimensions of the feature space of the two cameras respectively, assuming that the training data set is K pairs of cross-camera image features s _k ＝s(x _k ，y _k )∈{-1,+1} indicates the category label of the sample pair, -1 indicates heterogeneity, +1 indicates the same category, and the training set is divided into negative sample sets according to the category label and a set of positive samples |D ₁ |+|D ₂ |=K;

The cross-modal projection model learning unit specifically executes:

Suppose the set of cross-modal projection models H=[h ₁ h ₂ ,…,h _L ], L sub-models are used to deal with L kinds of data differences, each sub-model is composed of a pair of projection functions, h _l =[p _Xl ( x), p _Yl (y)], omitting the subscript l, the projection functions p _X (x) and p _Y (y) project x∈X and y∈Y into a common feature space:

In the formula, Represents the projection vector, a, b∈R are linear deviations, p _x (x) and p _Y (y) project the original features into {-1,+1} space;

There are also projection functions q _X (x) and q _Y (y) that project x∈X and y∈Y to another common feature space:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; a</span>}\\ {\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; b</span>}\end{array}\right.$

Establish the relationship between the data categories and the common feature space, and define the objective function:

In the formula, E represents expectation, Indicates the importance trade-off index of similar sample pairs and heterogeneous sample pairs;

In the formula, w _k represents the sample weight of the sample pair {x _k , y _k } in this sub-model learning, s _k ＝s(x _k ，y _k )∈{-1,+1} represents the category label of the sample pair,

The parameters {u,v,a,b} are learned by minimizing the objective function to obtain the corresponding projection function.

This preferred embodiment adopts multiple cross-modal projection models, which can fully cope with various data distribution differences.

Preferably, whether the identification database contains pedestrian images consistent with the inquiring person and confirming the identity of the inquiring person, including:

Suppose the set of queried persons is {f _i , STA(f _i )}, i=1, 2,...,N, f _i represents the i-th queried person, STA(f _i ) represents the identity of the th queried person , for the set of query personnel {g _j , STA(g _j ), j=1,2,...,M:

STA(g _j )=STA(f)

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; argmax</span>}}\mathrm{<span\; class="notranslate">\; Z</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; g</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>$

The similarity Z(g _j , f _i ) between g _j and f _i is expressed as:

Z(g _j ，f _i )＝sign(u ^T g _j +a)·sign(v ^T f _i +b)+||(u ^T g _j +a)-(v ^T f _i +b)||

Set the threshold T, T∈[1, 2], if Z(g _j , f _i )<T, there is no image consistent with the query person among the queried persons;

If Z(g _j , f _i )≥T, the inquired persons are sorted in descending order of similarity, and the ones at the top have the same identity as the inquiring persons.

This preferred embodiment improves the identification accuracy and efficiency of personnel in the video surveillance networking system.

Preferably, said evaluating the performance of the image recognition system includes:

Define the evaluation function:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}{{\mathrm{<span\; class="notranslate">\; N</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

In the formula, N represents the number of queries, and S _n represents the number of times the correct result can be found in the first n bits. The larger the value of the evaluation function, the better the re-identification performance of the system.

The evaluation module is set in this preferred embodiment, which is beneficial to improve the video surveillance networking system.

A group of recognition results of the video surveillance networking system of the present invention are shown in the following table:

N Average time for person identification Person recognition accuracy 6 0.14s 95.5% 12 0.12s 95.3% 18 0.16s 95.7%

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting the protection scope of the present invention, although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand , the technical solution of the present invention may be modified or equivalently replaced without departing from the spirit and scope of the technical solution of the present invention.

Claims (3)

1. A video surveillance networked system, characterized in that it can identify personnel by two modes of voice and image, including a collection system, a voice recognition system and an image recognition system, and the collection system collects voice and image, The speech recognition system includes a dictionary scene speech module, a similarity comparison module and a speech recognition engine module, and the image recognition system includes a preprocessing module, a feature extraction module, a training module, a re-identification module and an evaluation module; the preprocessing module It is used to determine the position of the person in the pedestrian image, and obtain the rectangular area containing the person; the feature extraction module is used to extract the appearance feature in the rectangular area containing the person; the training module is used to train multiple cross-modal projections model, each cross-modal projection model contains two projection functions, which respectively map images in different cameras to a common feature space and complete similarity calculations; the re-identification module is used to identify Whether it contains a pedestrian image consistent with the queryer and confirm the identity of the queryer; the evaluation module is used to evaluate the system performance. 2. A kind of video surveillance networking system according to claim 1, is characterized in that, the dictionary scene voice module is suitable for collecting the dictionary in the user vocabulary, the scene voice successively, and carries out the characteristic vector of collection as template save; The similarity comparison module is adapted to compare the feature vectors of the speech input speech signal with each feature vector template stored in the dictionary scene speech module in turn, and output the one with the highest similarity as the speech recognition result. 3. A video monitoring networking system according to claim 2, wherein the templates in the dictionary scene speech module include a monitoring system term template and a human voice plus dictionary template.