KR102278200B1 - Image processing apparatus, medium, and method - Google Patents

Abstract

The image processing apparatus includes a reference from a first acquisition unit configured to acquire a composite image obtained by synthesizing a plurality of images from a plurality of first imaging units directed to a predetermined area, and a second imaging unit directed to the predetermined area A second acquisition unit configured to acquire an image, a comparison unit configured to compare the synthesized image with the reference image, and an output unit configured to output a comparison result from the comparison unit.

Description

Image processing apparatus, medium and method {IMAGE PROCESSING APPARATUS, MEDIUM, AND METHOD}

The present invention relates to an image processing apparatus, medium and method.

BACKGROUND ART Conventionally, there is an imaging device having a plurality of imaging units capable of generating a composite image and an imaging unit capable of wide-area imaging. Wide-area monitoring can be realized by generating a composite image from a plurality of imaging units and confirming details that cannot be recognized in the composite image using the image from an imaging unit capable of wide-area imaging.

However, due to the arrangement of the photographing units, there is a case where a region having a double image or a region having a defect occurs in the seam portion of the synthesis. Depending on the position of the subject, overlapping or missing of the subject may occur, and the subject may not be photographed accurately. Hereinafter, a region having a double phase and a region having a defect are collectively referred to as an overlap/deletion region.

As a solution to the above problem, there is a method of supplementing the overlap or deficiency by using another imaging device. For example, Japanese Patent Laid-Open No. 2015-204512 discloses a method of detecting a missing area of a monitoring system composed of a plurality of cameras and giving a movement instruction to a movable camera to prevent the missing area.

However, in the technique disclosed in Japanese Patent Application Laid-Open No. 2015-204512, when the number of moving cameras is smaller than the number of missing areas, it is impossible to always properly image all the missing areas. Further, in the technique disclosed in Japanese Patent Laid-Open No. 2015-204512, the defect region can be detected, but the position of the defect region cannot be grasped.

One aspect of the present invention provides an image processing technique capable of more appropriately dealing with overlapping/deleting areas.

One aspect of the present invention has the following configuration.

a first acquisition unit configured to acquire a composite image obtained by synthesizing a plurality of images from a plurality of first imaging units directed to a predetermined area;

a second acquisition unit configured to acquire a reference image from a second imaging unit directed to the predetermined area;

a comparator configured to compare the composite image with the reference image;

and an output unit configured to output a comparison result from the comparison unit.

According to one aspect of the present invention, it is possible to provide an image processing technique capable of more appropriately dealing with overlapping/deleting areas.

Further features of the present invention will become apparent from the following detailed description of the embodiments given with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features and aspects of the invention and, together with the description, serve to explain the principles of the invention.
1 is a schematic diagram of a network camera system according to an embodiment.
FIG. 2 is a block diagram showing the configuration and function of the surveillance camera unit of FIG. 1 .
Fig. 3A is a schematic diagram showing the relationship between the angle of view of the fixed camera and the position of the subject.
3B is a schematic diagram of a subject imaged by the surveillance camera unit.
FIG. 3C is a diagram showing a multi-eye panoramic image when an imaging region is imaged with a multi-eye camera.
Fig. 4A is a diagram showing a multi-eye panoramic image when an imaging region is imaged with a multi-lens camera.
Fig. 4B is a diagram showing a PTZ panoramic image when the same imaging area is imaged with a PTZ camera.
Fig. 4C is an explanatory diagram showing a method for determining overlapping portions and missing portions.
4D is a schematic diagram showing a procedure for determining overlapping portions and missing portions.
Fig. 4E is an explanatory diagram showing a method of generating an overlapping portion and a missing portion.
FIG. 5 is a flowchart showing the flow of a series of processes of the comparison unit of FIG. 2 .
6A and 6B are diagrams illustrating a PTZ panoramic image displayed on a display accompanied by an overlap indicator and a missing indicator.
7A is a diagram illustrating a deficit indicator before movement and a deficit indicator after movement.
Fig. 7B is a diagram showing a PTZ panoramic image accompanied by a deficit indicator after movement.
FIG. 8 is a flowchart showing a flow of a series of processes in the rotation amount calculation unit of FIG. 2 .
9A is a diagram illustrating a multi-view panoramic image displayed on a display with an overlap indicator.
9B is a diagram illustrating a complementary polyocular image displayed on a display with a deficit indicator.
FIG. 10 is a flowchart showing a flow of a series of processes in the monitoring camera unit of FIG. 1 .
11 is a flowchart showing a procedure when the installer adjusts the posture of the monitoring camera unit.

Hereinafter, the same code|symbol shall be attached|subjected to the same or equivalent component, member, and process shown in each figure, and overlapping description is abbreviate|omitted as appropriate. In addition, in each figure, a part of the member which is not important for description is abbreviate|omitted and displayed.

The following embodiment provides an image processing apparatus supporting the desired installation of a surveillance camera unit having a plurality of fixed cameras and a PTZ camera capable of generating a wide range of images. This image processing apparatus acquires a composite image obtained by synthesizing a plurality of images from a plurality of fixed cameras, acquires a wide range image (hereinafter referred to as a reference image) from a PTZ camera, and displays the acquired composite image A difference between and a display range of the reference image is calculated, and the reference image is displayed on the display together with an indicator indicating the calculated difference. Thereby, it becomes easy to grasp the overlapping area and the missing area|region of a combined image, and it can implement|achieve easily the adjustment of the attitude|position and installation position of the monitoring camera unit in consideration of the position of a subject and the position of an overlap/missing area.

1 is a schematic diagram of a network camera system 10 according to an embodiment. The network camera system 10 includes a surveillance camera unit 106 and a workstation 121 . The surveillance camera unit 106 and the workstation 121 are connected via a network 120 . The network 120 may be the Internet, an intranet, a LAN, a WAN, a WiFi network, a public telephone network, a dedicated line, a wiring, or a combination thereof, and may be a wireless network or a wired network. The workstation 121 is a management device for managing the surveillance camera unit 106 via the network 120 . The workstation 121 includes a display 104 (for example, display means such as an LCD or an organic EL panel) and an operation unit 901 . The operation unit 901 accepts a user's operation. The operation unit 901 is provided with, for example, a keyboard, a mouse, a button, or a touch panel, and may be a device that accepts a well-known audio input. The surveillance camera unit receives an instruction given to the operation unit 901 via the network 120 . At this time, at least one of the display 104 and the operation unit 901 may be installed in the monitoring camera unit 106 .

The surveillance camera unit 106 includes a multi-lens camera 101 and a PTZ camera 102 . The multi-lens camera 101 includes a plurality of (four in the present embodiment) fixed cameras 105 . Both the multi-lens camera 101 and the PTZ camera 102 are installed in the housing 122 of the surveillance camera unit 106 . Each of the four fixed cameras 105 included in the multi-lens camera 101 is fixed to the housing 122 . Each fixed camera 105 may have a zoom function. The PTZ camera 102 is configured to be movable with respect to the housing 122 and may have, for example, a well-known pan/tilt/zoom function.

When the surveillance camera unit 106 is used, the multi-lens camera 101 and the PTZ camera 102 are directed to the same area in real space (hereinafter referred to as an imaging area). The multi-lens camera 101 captures an imaging area with the four fixed cameras 105, combines the obtained four images, and outputs a combined image obtained as a result of the synthesis, that is, a multi-lens panoramic image. The PTZ camera 102 uses a pan/tilt/zoom function to image an imaging area, and outputs the obtained reference image, that is, a PTZ panoramic image.

In the present embodiment, when the surveillance camera unit 106 is installed and its posture (eg, an angle displayed in azimuth) is adjusted, the difference between the PTZ panoramic image and the multi-view panoramic image is displayed on the display 104 . With reference to the indicator, the installer can orient the surveillance camera unit 106 so that the object of interest within the imaging area deviates from the differential.

Fig. 2 is a block diagram showing the configuration and function of the monitoring camera unit 106 according to the present embodiment. Each block shown here can be realized by hardware, i.e., an element such as a mechanical device or a CPU of a computer, and can be realized by software, i.e., a computer program, etc. However, the functional block shown here can be realized by the linkage between hardware and software. come true Accordingly, the fact that these functional blocks may be realized in various ways by a combination of hardware and software will be understood by those skilled in the art to which this specification applies.

The surveillance camera unit 106 further includes a processing unit 125 . Each member of the surveillance camera unit 106 , that is, the multi-lens camera 101 , the PTZ camera 102 , and the processing unit 125 is connected to the bus 126 , and is configured to be able to communicate with each other via the bus 126 . The monitoring camera unit 106 may include a volatile or nonvolatile memory such as ROM, RAM, HDD (all not shown).

At this time, in the present embodiment, the case where the processing unit 125 , the operation accepting unit 131 , the rotation amount calculation unit 902 , and the multi-view panorama complementing unit 601 are all installed in the monitoring camera unit 106 will be described. is not limited to For example, at least one of the processing unit 125 , the operation accepting unit 131 , the rotation amount calculating unit 902 , and the multi-view panorama complementing unit 601 may be provided in the workstation 121 . When the processing unit 125 is installed in the workstation 121 , the surveillance camera unit transmits an image from the camera to the workstation 121 via the network 120 , and transmits difference information from the workstation 121 to the network. (120) is received.

The processing unit 125 includes a multi-eye panorama acquisition unit 127 , a PTZ panorama acquisition unit 128 , a comparison unit 129 , a display control unit 130 , an operation accepting unit 131 , and a rotation amount calculation unit ( 902) and a multi-eye panorama complementing unit 601 .

The multi-eye panorama acquisition unit 127 acquires a multi-eye panorama image obtained by synthesizing four images from four fixed cameras 105 directed to the imaging area from the multi-eye camera 101 via the bus 126 .

The PTZ panorama acquisition unit 128 acquires a PTZ panorama image from the PTZ camera 102 directed to the imaging area from the PTZ camera 102 via the bus 126 .

The comparison unit 129 compares the multi-eye panorama image acquired by the multi-eye panorama acquisition unit 127 with the PTZ panorama image acquired by the PTZ panorama acquisition unit 128 . For example, the comparison unit 129 specifies the difference between the multi-view panoramic image and the PTZ panoramic image. The imaging area has a blind spot used as a blind spot of the multi-eye camera 101 and an overlapping area that is imaged over and over by the multi-eye camera 101 . The difference specified by the comparator 129 corresponds to the blind area and the overlap area.

Hereinafter, a difference between a multi-view panoramic image and a PTZ panoramic image will be described with reference to FIGS. 3A to 3C . Fig. 3A is a schematic diagram showing the positional relationship between the angles of view of the fixed cameras 105a to 105d and the subjects 201a to 201d. 3A corresponds to a view of the surveillance camera unit 106 viewed from above. An individual imaging area at the angle of view of each fixed camera 105a to 105d is part of the overall imaging area 210 . At this time, in this embodiment, in order to make description more understandable, depth etc. are ignored. In addition, although four images are synthesize|combined in this embodiment, the number of images to be synthesize|combined is not limited to this, For example, it can be determined according to the number of fixed cameras.

The four fixed cameras 105 of the multi-lens camera 101 are arranged to be spaced apart from each other in the circumferential direction. Therefore, between the individual imaging area 205b of a certain fixed camera 105b and the individual imaging area 205c of the adjacent fixed camera 105c, a blind area 204 is formed near the surveillance camera unit 106, and an overlapping area 203 is formed at a distance. this happens A blind area 204 and an overlap area 203 exist along a boundary 202 between adjacent individual imaging areas 205b and 205c.

3B is a schematic diagram of subjects 201a to 201d imaged by the monitoring camera unit 106 . In Fig. 3B, an automobile is used as an example of the subjects 201a to 201d. Further, the subjects 201a to 201d are arranged side by side along the boundary 202 between the adjacent individual imaging areas 205b and 205c. When the imaging area 210 is imaged by the PTZ camera 102, a PTZ panoramic image as shown in FIG. 3B is obtained.

FIG. 3C is a diagram illustrating a multi-eye panoramic image 207 when the imaging area 210 is imaged by the multi-eye camera 101 . The multi-eye panoramic image 207 is obtained by synthesizing four images 212a to 212d obtained by individual imaging areas captured by the four fixed cameras 105a to 105d. Combination of these images can be realized using a known image synthesizing technique. In FIG. 3C , overlaps and deletions occur in the images 211a to 211d of the subjects 201a to 201d due to the existence of the blind area 204 and the overlapping area 203 . Since the subject 201a is located in the overlapping area 203, it occurs in both the individual imaging areas 205b and 205c of the adjacent fixed cameras 105b and 105c. Accordingly, in the multi-eye panoramic image 207, the subject 201a is displayed as a double image (image 211a). Since the subjects 201c and 201d are located in the blind area 204, a part of the subject is missing from all of the individual imaging areas 205b and 205c of the adjacent fixed cameras 105b and 105c. In contrast, all of the subjects 201c and 201d are missing from all of the individual imaging areas 205b and 205c. Accordingly, the multi-eye panoramic image 207 displays only a part of the subjects 201c and 201d (images 211c and 211d), but does not display all of them.

Returning to Fig. 2, the comparator 129 includes a portion (hereinafter referred to as a missing portion) that is not included in the multi-view panoramic image 207 representing the same imaging area 210 among the PTZ images serving as a reference, and overlaps with each other. A portion to appear (hereinafter, referred to as an overlapping portion) is specified. As described above, the missing portion corresponds to the blind area 204 , and the overlapping portion corresponds to the overlapping area 203 .

The comparison unit 129 includes a small region extraction unit 107 , a pixel value comparison unit 108 , a retention unit 109 , and a difference region calculation unit 110 . The small region extraction unit 107 extracts a small region from the multi-eye panorama image 207 acquired by the multi-eye panorama acquisition unit 127 . The small region extraction unit 107 extracts a small region corresponding to the positional position from the PTZ panorama image acquired by the PTZ panorama acquisition unit 128 . The pixel value comparison unit 108 compares the extracted pixel values of the two small regions. The holding unit 109 holds the comparison result. After repeatedly performing the above-described small region comparison for the entire multi-view panoramic image, the difference region calculating unit 110 calculates an overlapping portion and a missing portion.

Hereinafter, a method of calculating the overlapping portion and the missing portion will be described with reference to FIGS. 4A to 4E . FIG. 4A is a diagram illustrating a multi-eye panoramic image 207 when the imaging area 210 is imaged by the multi-eye camera 101 . Fig. 4a corresponds to Fig. 3c. 4B is a diagram showing a PTZ panoramic image 301 when the same imaging area 210 is imaged by the PTZ camera 102 . Fig. 4C is an explanatory diagram showing a method for determining overlapping portions and missing portions. 4D is a schematic diagram showing a procedure for determining overlapping portions and missing portions. Fig. 4E is an explanatory diagram showing a method of generating an overlapping portion and a missing portion.

The comparison unit 129 compares the multi-view panoramic image 207 and the PTZ panoramic image 301 for each predetermined small area 401 . When the difference between the pixel values of the small region 401 is equal to or greater than the threshold value, the comparison unit 129 determines that overlap or deletion has occurred, and when the difference between the pixel values is smaller than the threshold value, the overlap or deletion does not occur. judge For example, as shown in Fig. 4C, the comparison unit 129 compares the pixel values in the small region 401-1, and determines that no overlap or deletion occurs because the difference between the pixel values is smaller than the threshold value. do. Further, the comparison unit 129 performs comparison in the small region 401-N1, and since the difference between the pixel values is equal to or greater than a threshold value, it is determined that overlapping or missing has occurred. In this case, the pixel value difference may be an average difference between pixel values of pixels included in the small region, or the number or ratio of pixels with which the pixel values in the small region match (or do not match).

The determination of overlap/deletion is sequentially performed by moving the small area 401 as shown in Fig. 4D. The initial position of the small area 401 is the lower left corner of each image (the small area at the initial position is denoted as small area 401-1), and after the comparison of the small area 401-1 is completed, the comparison processing is performed on the small area adjacent to the right Go to 401-2. Then, after reaching the small region 401-N2 at the right end, the comparison processing moves to the small region 401-N3 above it, and then similarly moves to the left when the processing is continued. When the upper left small area 401-N4 or the upper right small area 401-N5 is reached, the comparison unit 129 ends the determination processing. When the determination processing for all the small areas is completed, the difference area calculation unit 110 calculates an overlapping portion and a missing portion by synthesizing the determination results as shown in Fig. 4E. In the example of FIG. 4E , the difference area calculation unit 110 calculates the missing portion 402 in the PTZ panoramic image 301 .

5 is a flowchart showing the flow of a series of processing in the comparison unit 129 . First, in step S505, the comparison unit 129 extracts and compares the small area 401 of the multi-eye panoramic image 207 and the small area 401 of the PTZ panoramic image 301 . In step S506, the comparison unit 129 determines the magnitude relationship between the difference between the pixel values and the threshold value. If the difference is equal to or greater than the threshold value, in step S507, the comparison unit 129 determines that there is an overlap/deletion in the small region 401, and associates the position of the small region 401 with the determination result, and the holding unit ( 109) is stored. The determination result may be expressed by, for example, a flag indicating the presence or absence of overlap/deletion. If the difference is smaller than the threshold value, in step S508, the comparison unit 129 determines that there is no overlap/deletion in the small region 401, and the holding unit associates the position of the small region 401 with the determination result. It is stored in (109). Then, in step S509, the comparator 129 determines whether or not there is an unprocessed small region 401 . If there is an unprocessed small area 401, in step S511, the comparison unit 129 moves to the next small area 401, and the process returns to step S505. If it is determined in step S509 that there is no unprocessed small region 401, in step S510, the comparison unit 129 reads the determination result of each small region 401 from the holding unit 109, and overlaps and missing parts. calculate the part

Returning to FIG. 2 , the display control unit 130 functions as an output unit for outputting the comparison result obtained by the comparison unit 129 . The display control unit 130 causes the display 104 to display the PTZ panoramic image 301 together with the difference indicator indicating the difference specified by the comparison unit 129 via the network 120 . The difference indicator includes a deficit indicator indicating the missing portion 402 calculated by the difference region calculating unit 110 and an overlap indicator indicating the overlapping portion calculated by the difference region calculating unit 110 .

FIG. 6A is a diagram illustrating a PTZ panoramic image 301 accompanied by an overlap indicator 510 displayed on the display 104 . FIG. 6B is a diagram illustrating a PTZ panoramic image 301 accompanied by a deficit indicator 511 displayed on the display 104 . Unlike the multi-eye panoramic image 207 , the PTZ panoramic image 301 can display the subject 201 without overlapping or missing. For this reason, by superimposing the overlapping indicator 510 and the missing indicator 511 on the PTZ panoramic image 301, the positional relationship between the subject 201 and the overlapping area 203 or the blind area 204 can be recognized. have.

Returning to FIG. 2 , the operation accepting unit 131 sends an instruction for moving the overlapping indicator 510 or the missing indicator 511 to the operating unit 901 from the user, from the operating unit 901 to the network 120 . received through The display control unit 130 moves the overlap indicator 510 or the missing indicator 511 over the PTZ panoramic image 301 (reference image) displayed on the display 104 according to the received instruction. The user can move the displayed overlap indicator 510 or missing indicator 511 over the PTZ panoramic image 301 using the operation unit 901 . The rotation amount calculating unit 902 calculates the amount of movement of the housing 122 accompanying the movement of the overlapping indicator 510 or the missing indicator 511 . The amount of movement is, for example, the angle of rotation of the housing 122 that is actually needed to move the overlapping area 203 or the blind area 204 .

Hereinafter, movement of the deficit indicator 511 will be described with reference to FIGS. 7A and 7B . As an example, consider a case in which the deficit indicator 511 moves in the direction (left direction) of the arrow 710 on the PTZ panoramic image 301 displaying the deficit indicator 511 . 7A is a diagram illustrating a deficit indicator 511 before movement and a deficit indicator 711 after movement. In FIG. 7A , the loss indicator 511 before movement has a solid outline, and the loss indicator 711 after movement has a broken line outline, but the actual display form is not limited thereto.

7B is a diagram illustrating a PTZ panoramic image 301 accompanied by a deficit indicator 711 after movement. The deficit indicator has moved over the PTZ panoramic image 301 , but at this point the position of the corresponding blind area 204 has not yet moved. 7B shows the results of the simulation regarding rotating the surveillance camera unit 106 . For this reason, the display control unit 130 may display the post-movement deficit indicator 711 in a different manner from the previous deficit indicator 511 . For example, in the example of FIG. 7B , the defect indicator 711 after movement is indicated with a broken line outline.

On the PTZ panoramic image 301 displayed on the display 104, when the deficit indicator 511 moves according to an instruction from the user, the PTZ panoramic image 301 that was hidden by the deficit indicator 511 before moving. ) is shown. For example, in the state of FIG. 7A , the window portion of the image 712d of the subject 201d closest to the monitoring camera unit 106 is hidden by the defect indicator 511 . In the state of FIG. 7B after moving the deficit indicator 511, this window part is visible. In this way, the movement of the deficit indicator reveals an image that was hidden under the deficit indicator, so that the user can easily know how much or how to move the deficit indicator so that the object of interest is made visible.

Hereinafter, with reference to FIG. 7A, the calculation method of the rotation angle in the rotation amount calculation part 902 is demonstrated. As an example, consider the case where the deficit indicator 511 moves as shown in FIG. 7A . In Fig. 7A, coordinates are assigned to the PTZ panoramic image 301 as follows: (0, 0) at the lower left, (a, 0) at the lower right, (0, b) at the upper left, (0, b) at the upper right ( a, b) is assigned to the vertices of the deficit indicator 511 before movement (x1, y1), and (x2, y1) is assigned to the vertices of the deficit indicator 711 after movement. At this time, the coordinates are assigned in association with the small region 401 used for calculating the overlapping part and the missing part. The horizontal direction of FIG. 7A is the x direction, and the vertical direction is the y direction. The rotation angle of the monitoring camera unit 106 (or the housing 122 thereof) with respect to the movement amount in the x direction is calculated by the following formula (1). Here, α is the angular range corresponding to the PTZ panoramic image 301 . For example, in the case of generating the PTZ panoramic image 301 in all directions, ? = 360 degrees.

Rotation angle for movement = (x2-x1)x(α÷a) … (One)

8 is a flowchart showing the flow of a series of processes in the rotation amount calculation unit 902 . First, in step S801 , the rotation amount calculating unit 902 acquires the angular range α corresponding to the PTZ panoramic image 301 . For example, when the omnidirectional PTZ panoramic image 301 is generated, the angle range is 0 degrees to 360 degrees. Next, in step S802, the rotation amount calculating unit 902 calculates the coordinates (a, b) of the end of the PTZ panoramic image 301 and the coordinates (x1, y1) of the vertex of the deficit indicator 511 before movement. acquire After acquiring the coordinates, in step S803, the user moves the deficit indicator using the operation unit 901 . In step S804, the rotation amount calculation unit 902 calculates the coordinates (x2, y1) of the vertices of the deficit indicator 711 after movement. In step S805, the rotation amount calculation unit 902 calculates the rotation angle required for the movement of the deficit indicator by using the equation (1). In step S806, the rotation amount calculation unit 902 notifies the user of the calculated rotation angle. The notification may be performed by displaying the rotation angle on the display 104 via the display control unit 130, or may be performed by synthesizing and outputting an audio indicating the rotation angle.

Returning to FIG. 2 , the multi-eye panorama complementing unit 601 complements the multi-eye panoramic image 207 based on the comparison result from the comparison unit 129 . The multi-eye panorama complementing unit 601 complements the multi-eye panoramic image 207 according to the missing portion 402 calculated by the comparison unit 129 . Since the multi-eye panoramic image 207 has a different deficit from the PTZ panoramic image 301, by adjusting the multi-eye panoramic image 207 to compensate for the missing part 402, the sense of incongruity can be reduced. The display control unit 130 causes the display 104 to display the multi-eye panorama image supplemented by the multi-eye panorama complementation unit 601 .

9A is a diagram illustrating a multi-eye panoramic image 207 accompanied by an overlap indicator 911 displayed on the display 104 . The overlap indicator 911 indicates the overlapping portion calculated by the difference region calculating unit 110 . 9B is a diagram illustrating a complementary polyocular image 910 accompanied by a deficit indicator 912 displayed on the display 104 . The deficit indicator 912 indicates the missing portion 402 calculated by the difference region calculating unit 110 . The multi-eye panorama complementing unit 601 generates the complementary multi-eye image 910 by inserting the deficit indicator 912 into the multi-eye panoramic image 207 .

The operation of the surveillance camera unit 106 having the above configuration will be described.

10 is a flowchart showing a flow of a series of processes in the monitoring camera unit 106 . First, in step S171 , the surveillance camera unit 106 generates a multi-eye panoramic image 207 using the multi-eye camera 101 . In step S172 , the surveillance camera unit 106 uses the pan/tilt/zoom function of the PTZ camera 102 to scan a wide area by, for example, rotating the PTZ camera 102 , and a PTZ panoramic image 301 . ) is created. In step S173, the surveillance camera unit 106 compares the multi-eye panoramic image 207 and the PTZ panoramic image 301 to calculate an overlapping part and a missing part. After the calculation, in step S174 , the monitoring camera unit 106 superimposes the overlapping indicators and missing indicators corresponding to the calculated overlapping and missing portions on the PTZ panoramic image 301 , and displaying the image on the display 104 . make it In step S175, the surveillance camera unit 106 complements the multi-eye panoramic image 207 based on the calculated missing part. In step S176 , the surveillance camera unit 106 causes the display 104 to display the multi-eye panoramic image (supplemented multi-eye image) after supplementation.

11 is a flowchart showing a procedure for the installer to adjust the posture of the monitoring camera unit 106 . In step S180, the installer installs the surveillance camera unit 106 at a desired position. In step S181 , the installer uses the display 104 of the workstation 121 to confirm how the object of interest is included in the imaging area 210 . In step S182, the installer determines whether or not the object of interest is hidden by difference indicators (eg, overlapping or missing indicators). If it is not hidden, the installation is complete. If it is hidden, in step S183 , the installer moves the difference indicator so that the object of interest appears in the PTZ panoramic image 301 displayed on the display 104 . For example, the installer operates the operation unit 901 to move the difference indicator so that the object of interest deviates from the difference display. In step S184 , the installer receives a notification of the rotation angle corresponding to the movement amount from the monitoring camera unit 106 or the workstation 121 . In step S185, the installer rotates the entire monitoring camera unit 106 so that the notified rotation angle is realized.

According to the monitoring camera unit 106 of the present embodiment, it is easy to grasp the positions of the overlapping portion and the missing portion of the multi-eye panoramic image 207 , so that the subject 201 and the overlapping area 203 or the blind area 204 are separated. The positional relationship can be easily grasped. Thereby, installation of the monitoring camera unit 106 becomes easier in consideration of this positional relationship. For example, when there is a subject to be observed when the monitoring camera unit 106 is installed, the monitoring camera unit 106 may be installed so that the overlapping area 203 and the blind area 204 are separated from the subject.

In the above, the configuration and operation of the monitoring camera unit 106 according to the present embodiment have been described. This embodiment is an example, and it is understood by those skilled in the art that various modifications are possible by the combination of each component and each processing step, and that such modifications also fall within the scope of the present invention.

In this embodiment, although the case where the deficit indicator 511 is moved in parallel in the x direction has been described, it is not limited to this. It is also possible to move the display of other suns. For example, the deficit indicator 511 may be moved in the y direction.

In this embodiment, although the case where the PTZ camera 102 is used as an imaging part which generate|occur|produces a reference image was demonstrated, it is not limited to this. For example, a camera having a wider angle of view than the fixed camera 105 such as a camera (fisheye camera) provided with a fisheye lens can be used.

Other embodiments

An embodiment of the present invention is a computer executable recorded on a storage medium (which may also be referred to in more detail as a 'non-transitory computer-readable storage medium') for performing one or more functions of the aforementioned embodiment(s) of the present invention. One or more circuits (eg, application specific circuits (ASICs)) that read and execute possible instructions (eg, one or more programs) and/or perform one or more functions of the embodiment(s) described above. by the computer of the system or apparatus comprising a computer or, for example, by the computer of the system or apparatus by reading and executing computer-executable instructions from a storage medium to perform one or more functions of the embodiment(s) described above. A computer may have one or more central processing units (CPUs), microprocessors (MPUs) or other circuitry, and may include a network of separate computers or separate computer processors. The computer-executable instructions may be given to a computer, for example, from a network of storage media, which may include, for example, one or more hard disks, random access memory (RAM), read-only memory (ROM), A storage of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray disk (BD) ^TM ), a flash memory device, a memory card, and the like may be provided.

The present invention provides a program for realizing one or more functions of the above embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device read and execute the program It is also possible to execute It is also executable by a circuit (for example, an ASIC) that realizes one or more functions.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the present invention is not limited to these embodiments. The protection scope of the following claims should be construed in the broadest way to cover all modifications, equivalent structures and functions.

Claims (12)

a first acquisition unit configured to acquire a first image obtained by synthesizing a plurality of images from a plurality of first imaging units arranged such that the imaging regions partially overlap each other;
a second acquisition unit configured to acquire a second image obtained by a plurality of images captured by rotating the second imaging unit in a tilt direction and a pan direction;
a comparison unit configured to compare the first image and the second image and to specify an area that cannot be imaged by the plurality of first image sensing units;
and an output unit configured to output information indicating the region based on a result specified by the comparison unit.
The method of claim 1,
The comparison unit is an image processing apparatus for specifying a difference between the first image and the second image.
3. The method of claim 2,
The output unit displays the second image together with an indicator indicating the difference on the display unit.
4. The method of claim 3,
a receiving unit configured to receive an instruction to move the indicator representing the difference;
The output unit is an image processing apparatus that moves the indicator indicating the difference according to the instruction, on the second image displayed on the display unit.
5. The method of claim 4,
and a calculation unit configured to calculate an amount of movement of the housing of the image processing apparatus accompanying movement of the indicator indicating the difference.
5. The method of claim 4,
When the indicator indicating the difference moves according to the instruction on the second image displayed on the display unit, a portion of the second image that was hidden by the indicator indicating the difference before moving appears on the second image.
The method of claim 1,
a complementing unit configured to complement the first image based on the result from the comparison unit;
The output unit displays the first image supplemented by the complementation unit on a display unit.
The method of claim 1,
The comparison unit is configured to specify a region to be imaged overlapped by the plurality of first imaging units,
and the output unit is configured to output information indicating the area based on a result specified by the comparison unit.
The method of claim 1,
The second imaging unit is an image processing apparatus provided in a housing in which the plurality of first imaging units are provided.
10. The method of claim 9,
and each of the plurality of first imaging units is fixed to the housing, and the second imaging unit is configured to be movable with respect to the housing.
on the computer,
providing an imaging device including a plurality of first imaging units and second imaging units;
On the reference image obtained from a second image capturing unit and displayed on a display unit together with an indicator indicating a difference between a reference image and a synthesized image obtained by synthesizing a plurality of images from the plurality of first image capturing units, an object of interest is moving the indicator away from the difference;
A non-transitory computer-readable storage medium storing a computer program for executing a method comprising the step of adjusting the posture of the imaging device according to the amount of movement.
providing an imaging device including a plurality of first imaging units and second imaging units;
On the reference image obtained from a second image capturing unit and displayed on a display unit together with an indicator indicating a difference between a reference image and a synthesized image obtained by synthesizing a plurality of images from the plurality of first image capturing units, an object of interest is moving the indicator away from the difference;
and adjusting a posture of the imaging device according to a movement amount.

Images