CN104567758B - Stereo imaging system and its method - Google Patents

Abstract

A stereo imaging system and method thereof are disclosed. The system includes: a ray source emitting a plurality of ray fan beams; a plurality of columns of detectors respectively arranged opposite to the ray source at predetermined angles, when the inspected object moves along a direction intersecting the ray fan beams, The multi-column detectors respectively detect the intensity values of the corresponding ray fan beams penetrating the inspected object to form transmission images corresponding to each column of detectors; the reconstruction device converts any two transmission images among the multi-column transmission images As a dual-view image, the depth information of the object on the transmission image is calculated, and the calculated depth information is superimposed and fused to obtain the three-dimensional information of the inspected object for three-dimensional reconstruction. Use the transmission image collected by a multi-row linear detector with a certain angle to perform three-dimensional reconstruction to restore the lost depth information of the transmission image, so that the transmitted object presents a certain three-dimensional effect at different viewing angles, which is convenient for better image analysis. .

Description

Stereo imaging system and method thereof

technical field

The present application relates to the field of radiation imaging, in particular to a single-source multi-detector stereoscopic imaging system and a method thereof.

Background technique

Security inspection is of great significance in the fields of anti-terrorism and combating drug trafficking and smuggling. Countries all over the world are paying more and more attention to the security inspection of public places, and the inspection requirements for customs containers and luggage items are also getting higher and higher.

At present, the radiation imaging system is the mainstream for safety inspection. For ordinary transmission images that use linear detectors to image, the data obtained is two-dimensional images. The detection image has material depth information loss, and image information projection overlaps seriously, which affects people's perception of objects. Shape recognition and recognition.

Contents of the invention

Aiming at one or more problems in the prior art, a stereoscopic imaging system and method thereof are proposed.

In one aspect of the present invention, a stereoscopic imaging system is proposed, including: a radiation source emitting a plurality of fan beams of radiation; multiple columns of detectors respectively arranged opposite to the radiation source at predetermined angles, when the object to be inspected is placed along the When moving in a direction intersecting with the ray fan beams, the multi-row detectors respectively detect the intensity values of the corresponding ray fan beams penetrating the inspected object to form transmission images corresponding to each row of detectors; the reconstruction device , using any two transmission images among the multiple transmission images as a dual-view image, calculating the depth information of the object on the transmission image, superimposing and fusing the calculated depth information, obtaining the 3D information of the inspected object, and performing 3D reconstruction.

In another aspect of the present invention, a method for a stereoscopic imaging system is proposed, the system includes a ray source and a plurality of columns of detectors respectively arranged opposite to the ray source at predetermined angles, and the method includes the steps of: The ray source emits a plurality of ray fan beams; when the inspected object moves along a direction intersecting the ray fan beams, the multi-column detectors respectively detect the intensity values of the corresponding ray fan beams penetrating the inspected object, Form a transmission image corresponding to each column of detectors; use any two transmission images in multiple transmission images as a dual-view image, calculate the depth information of the object on the transmission image, and superimpose and fuse the calculated depth information to obtain the inspected The 3D information of the object is used for 3D reconstruction.

Use the transmission image collected by a multi-row linear detector with a certain angle to perform three-dimensional reconstruction to restore the lost depth information of the transmission image, so that the transmitted object presents a certain three-dimensional effect at different viewing angles, which is convenient for better image analysis. .

Description of drawings

The following figures illustrate embodiments of the invention. These figures and embodiments provide, in a non-limiting, non-exhaustive manner, some embodiments of the invention, in which:

Fig. 1 shows a top view of an image acquisition system according to an embodiment of the present invention;

Fig. 2 shows a side view of an image acquisition system according to an embodiment of the present invention;

Fig. 3 is a schematic diagram describing the process of obtaining depth information;

Fig. 4 is a schematic diagram describing the layout of the detector column direction;

5 is a schematic diagram describing the principle of geometric correction in the column direction;

Figure 6 shows a schematic diagram of a container truck;

Fig. 7 shows the process of box identification and modeling;

Figure 8 shows the process of body modeling;

Fig. 9 shows a schematic diagram of a three-dimensional reconstruction effect achieved by the technology according to an embodiment of the present invention.

detailed description

Specific embodiments of the present invention will be described in detail below, and it should be noted that the embodiments described here are only for illustration, not for limiting the present invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one of ordinary skill in the art that these specific details need not be employed to practice the present invention. In other instances, well-known structures, circuits, materials, or methods have not been described in detail in order to avoid obscuring the present invention.

Throughout this specification, reference to "one embodiment," "an embodiment," "an example," or "example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in the present invention. In at least one embodiment. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "an example," or "example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, particular features, structures or characteristics may be combined in any suitable combination and/or subcombination in one or more embodiments or examples. In addition, those of ordinary skill in the art should understand that the term "and/or" used herein includes any and all combinations of one or more of the associated listed items.

Three-dimensional images are widely used in real life. Compared with two-dimensional images, it can better describe the real scene and obtain more vivid visual effects. In the field of radiation imaging, CT imaging can very well reconstruct the three-dimensional structure of objects, but three-dimensional CT imaging has the disadvantages of complex scanning equipment structure and long measurement time. Therefore, solving the stereoscopic imaging problem by using multiple viewing angles for perspective images not only has good academic value, but also has good application value: multi-view stereoscopic imaging technology for radiographic images can give an approximate three-dimensional image, and use small angle rotation Or use the three-dimensional display terminal display to give the inspector three-dimensional perception and improve user experience.

In view of the above problems, in the embodiment of the present invention, the transmission image collected by a multi-row linear detector with a certain angle is used to perform three-dimensional reconstruction to restore the lost depth information of the transmission image, so that the transmitted object can be presented at different angles of view A certain stereoscopic effect facilitates better image analysis. For example, in the field of container security inspection, X-rays can be used to scan the container without opening the box, and then restore part of its three-dimensional information. Using this part of information in a new display method, it can provide users with new image viewing assistance and experience.

According to an embodiment of the present invention, a stereoscopic imaging system is proposed, including a ray source, a multi-column detector, and a reconstruction device such as a computer. The radiation source emits a plurality of radiation fan beams. A plurality of columns of detectors are arranged opposite to the radiation source at a predetermined angle, and when the object to be inspected moves along a direction intersecting with the radiation fan beam, the multi-row detectors respectively detect the transmission of the corresponding radiation fan beam The intensity values of the inspected object form transmission images corresponding to each column of detectors. The reconstruction device uses any two transmission images among the multiple transmission images as a dual-view image, calculates the depth information of the object on the transmission image, superimposes and fuses the calculated depth information, obtains the three-dimensional information of the inspected object, and performs three-dimensional reconstruction.

Figure 1 shows the top view of the image acquisition system. In the figure, three viewing angles are taken as examples. In practical applications, the viewing angles can be appropriately increased according to actual needs. The ray fan beams from the ray source 110 to the three left, middle and right detectors 120, 130, 140 form a predetermined opening angle, for example, an angle θ is formed between the left detector row and the middle detector row, and the right detector row and the middle detector row The angle θ is also formed between the columns, and three images under different angles are obtained by scanning. The middle rays are in the main beam direction of X-rays, and the left and right rays are symmetrically distributed on both sides of the main beam direction. The embodiment of the present invention takes three viewing angles as an example, and pairs of viewing angles can be added symmetrically on both sides of the X-ray main beam direction to form an image acquisition system.

Fig. 2 shows a side view of the image acquisition system, the imaging system mainly includes a radiation source 110, a collimation device (not shown), a data acquisition system (not shown) and the like. When the object to be detected passes through the detection area at a certain speed, multiple images corresponding to the number of detector rows can be generated simultaneously, that is, the number of images is the same as the number of detector rows. In this imaging model, an appropriate reference coordinate system can be established to perform 3D reconstruction and multi-view viewing of the acquired multi-view images.

Figure 3 shows the principle of obtaining depth information by using two perspective images. Objects with different depths will have a certain position difference in the two images, and the depth information can be extracted through this position difference. There are two points A and B at different depths. In the right detector scanning image, B is on the right of A, and in the left detector scanning image, B is on the left of A. The relative position difference between the two points is L, which is determined by L and the opening angle α The depth difference H between points A and B can be obtained:

When the beam angle α is small

That is, the relative depth of the object is proportional to the relative position difference of the object in the left and right images.

Depth information of objects on the image can be obtained by combining any two images obtained from different perspectives, and the obtained 3D information can be superimposed and fused, so that the 3D information can be restored to the maximum extent. The present invention takes three viewing angles as an example, and three images can be collected at the same time, the left image, the middle image and the right image, and there are three combinations of two combinations. For the combination of left and right views, the coordinate system of one of the images is used as the reference coordinate system, and the three-dimensional coordinates can be solved by the following formula:

Among them, v is the speed of the object passing through the acquisition area, f is the frequency of ray occurrence, α is the angle between two viewing angles, and Δx is the difference in the position of the same feature point in different views. The distance from the X-ray source to the plane where the detector is located is L. Taking the left view image as a reference image, use the obtained depth information z to modify the x and y coordinates as follows:

Among them, x _i and y _i are the point coordinates on the image, and dy is the resolution in the y direction. Similar solutions can be used for other image combination methods, but the coordinate calculation and correction formulas need to be adjusted appropriately.

In addition, there is geometric distortion in the direction of the detector column, which requires geometric correction in the direction of the detector column. The reasons for the existence of geometric distortion and the correction scheme will be introduced in detail below.

In the process of image acquisition, since the parameters such as the position and opening angle of the device are relatively fixed, the calibration steps in similar optical images can be ignored. It is just a matter of selecting the appropriate reference frame.

According to some embodiments of the present invention, the following algorithm is used to obtain the point-to-point matching of the two views: first, the traditional feature extraction and matching methods are used to establish the correspondence between the feature points of the two views; Kernel, superposition to establish a Hilbert space, using space smoothness constraints to iteratively obtain a smooth space vector field. This space vector field represents the corresponding relationship between the points on the two images.

In addition, the SIFT flow algorithm can also solve the point-to-point correspondence between the two images very well. The basic principle of the SIFTflow algorithm is: first calculate the SIFT feature vector for each pixel on the image, that is, Dense SIFT, and then use the flow estimation algorithm to match the SIFT vector description. However, SIFT Flow treats all pixels of the image to be matched equally, ignoring the difference of image pixels. For a picture, the amount of information of each pixel is different. The pixel of the prominent object in the picture has a large amount of information, while the background (such as a pure black background) has a small amount of information. Due to the equal treatment, the matching values of pixels with little or no information will affect the matching values of important pixels, which is equivalent to introducing interference to the matching process, resulting in inaccurate overall results. Therefore, this patent uses an improved energy function, which increases the influence proportion of the area with a large amount of information, weakens the influence proportion of the background area, and shields the influence of the background area on the matching process, in order to obtain more accurate matching results Create conditions. In addition, the Belief propagation algorithm is used to optimize the objective function. In this way, an optimal flow field can be obtained. This optimal flow field is the correspondence between the two images.

In view of the geometric distortion in the direction of the detector column, due to the radial characteristics of X-rays, the linear detectors are arranged in a fan shape (arc), which can adapt to the characteristics of X-rays and reduce geometric distortion. It can be seen that the fan-shaped (arc) arrangement of detectors is the optimal choice. However, in practical applications, the arrangement of the detectors is affected by the mechanical structure and space, and sometimes only an approximate fan-shaped (arc) arrangement can be used. Figure 4 shows an example of the arrangement of the detectors. The detector can be linearly installed on the vertical arm frame or linearly installed on the cross arm frame, or both can be installed, that is, the L-shaped arm frame. For example, the L-shaped detector row includes a plurality of detector modules 131 , 132 and 133 etc., and receives the radiation beam emitted by the radiation source 110 . Moreover, the angles of the two booms can be adjusted appropriately. In practical applications, an appropriate arrangement can be selected according to the requirements of the mechanical structure. In the embodiment of the present invention, an L-shaped arm frame is used. This approximate fan-shaped (arc-shaped) arrangement has different radii of the fan shapes. Although the geometric distortion in the direction of the detector column is reduced, it cannot be eliminated. This geometric distortion will seriously affect the effect of 3D reconstruction, which requires geometric correction during the reconstruction process.

Figure 5 shows the principle of geometric correction, assuming that the size of the detector column direction is S, and the number of detection pens is N, then the resolution in the direction of the X-ray main beam is and is considered to be standard resolution. The distance from the ray source O to the detector module in the main beam direction of the ray, that is, the radius of the fan is D, and the distance from other modules is ΔD. The ΔD of different modules can be obtained according to the detector layout diagram. Then the size S1 of the module relative to the X-ray main beam direction module is Then the resolution of the module is

According to the difference of the resolution, the geometric correction in the direction of the detector column is realized by using the image difference method.

Fig. 6 shows a container truck to be inspected, one of the three views collected in the embodiment of the present invention.

For the container truck to be inspected, the cargo in the container is the content of interest, and the container itself is a regular cube. Through the three-dimensional modeling of the box, the three-dimensional information of the box can be perfectly restored. The obtained 3D information can also be used as a reference for 3D reconstruction of goods in the box. Figure 7 shows the process of box position identification and modeling, and the details are as follows:

In step S110, eliminate the influence of dose instability

Use prior information to eliminate the influence of dose instability on the algorithm

U _x =-log(I/α)

Among them, α is the air value in the current column of the image (the image collected when there is only air in the acquisition area), I is the reading of a certain detector pen in the current column, and the converted image gray value Ux is insensitive to dose instability.

In step S120, find the top edge

For example, using morphological methods to solve the morphological gradient, and using Hough transform to detect the upper edge of the container. If not, an error is returned at step S150.

In step S140, remove the head of the car body and only keep the box part

Detect the upper edge of the container truck chassis, the processing method is the same as that of the container upper edge.

Then, determine the specific location of the box

With the upper edge of the box body and the upper edge of the vehicle chassis, the specific position and size of the box body are obtained.

In step S160, box modeling

Use the monitored container position and size to determine the type of container and select an appropriate 3D model in the template library to import during 3D reconstruction.

During the security inspection process, for the container truck to be inspected, the truck part is often not the focus of the inspection. During the 3D reconstruction, the modeling method can be used to quickly and effectively restore its 3D information. Firstly, the position and size of the truck should be identified and used as features, and then the most suitable 3D truck template should be found in the template library.

Figure 8 shows the process of vehicle body recognition and modeling. In the present invention, the edge information is used as the main basis to judge the position of the vehicle in the image, and the model and the position of the front of the truck are recognized on the premise of box recognition. When detecting the vehicle position, in order to obtain a robust result, the processing method is divided into three steps: preprocessing to remove stripes, and obtain an image with a gentle background change; calculate the gradient of the image, and quantize the gradient map to remove the influence of small gradient fluctuations; In the value quantization gradient map, find the largest continuous area of horizontal and vertical projection, that is, the vehicle position.

In step S210, the method of preprocessing to remove streaks: remove streaks in horizontal and vertical directions respectively. Taking the horizontal as an example, first find the projection sequence Projection of the image in the vertical direction. Median filtering is performed on the Projection. If the difference before and after filtering is large, it is judged to be a stripe, and the value of this row is replaced by the value of the latest non-stripe image row.

In step S220, the method for calculating the gradient is: quantizing the image, and calculating the gradient after quantization.

Find the vehicle position: Find the horizontal and vertical projections of the gradient map, and detect the largest continuous area after subtracting the minimum value (that is, remove the influence of possible stripes). This area is the vehicle location.

In step S230, the aforementioned method is used to identify the box.

In step S240, on the premise of box body modeling, the position and orientation of the vehicle body are determined by identifying the vehicle body. During 3D reconstruction, the 3D model of the vehicle in the template library can be used to replace the reconstruction of the vehicle body.

In the security inspection industry, the three-dimensional reconstruction of perspective images is for the purpose of assisting inspection, and does not require precise measurement. Therefore, the present invention pays more attention to using the depth information of the perspective image to three-dimensionalize the perspective image, with the purpose of displaying the overlapping perspective images of objects in layers. Based on this, two ways can be used to display the three-dimensional effect: the first, using three-dimensional display tools such as OpenGL, develop three-dimensional display software, and utilize different angles to rotate to display the three-dimensional effect; second, utilize the hardware display terminal, use the naked eye in the present invention The 3D display presents three-dimensional data. Figure 9 describes the 3D reconstruction effect displayed by the OpenGL software package.

Using the method of the above embodiment, the depth information of the two-dimensional image is obtained based on the multi-view perspective image, and the two-dimensional perspective image is three-dimensionalized to provide an auxiliary inspection solution. In addition, the use of modeling technology to achieve three-dimensional modeling of containers, improve the accuracy of three-dimensional reconstruction. Third, use the modeling technology to realize the 3D modeling of the truck and improve the 3D display effect.

Furthermore, by using the display software, the 3D effect can be displayed from different angles, combined with the hardware display terminal, the 3D display effect can be improved.

While this invention has been described with reference to a few exemplary embodiments, it is to be understood that the terms which have been used are words of description and illustration, rather than of limitation. Since the present invention can be embodied in many forms without departing from the spirit or essence of the invention, it should be understood that the above-described embodiments are not limited to any of the foregoing details, but should be construed broadly within the spirit and scope of the appended claims. , all changes and modifications falling within the scope of the claims or their equivalents shall be covered by the appended claims.

Claims (9)

1. a kind of stereo imaging system, including：

Radiographic source, send multiple ray fan-beams；

Multiple linear detector arrays, be oppositely arranged respectively with predetermined angle and the radiographic source, when inspected object along During the direction motion intersected with the ray fan-beam, it is saturating that the multiple linear detector array separately detects corresponding ray fan-beam The intensity level of the inspected object is penetrated, forms the transmission image corresponding with each row detector；

Reconstructing device, using any two width transmission image in several transmission images as double-visual angle figure, calculate thing on transmission image The depth information of body, fusion is overlapped to the depth information of calculating, obtains the three-dimensional information of inspected object, carry out Three-dimensional Gravity Build,

Wherein, the multiple linear detector array is L-arm frame, and the reconstructing device is using following formula to line detector battle array Resolution ratio on column direction is corrected：

Wherein, S represents the size along the direction of linear detector array, and N represents detecting module in linear detector array Number, D represent that radiographic source represents other detector modules relative to main beam to ray main beam directional detector module distance, Δ D The distance change of directional detector module.

2. stereo imaging system as claimed in claim 1, wherein, the reconstructing device is based on the object of different depth in two width Alternate position spike between image calculates depth information.

3. stereo imaging system as claimed in claim 1, wherein, the reconstructing device using the depth information z calculated to x and Y-coordinate is corrected.

4. stereo imaging system as claimed in claim 1, wherein, depth information is calculated as below in the reconstructing device：

2 points of A provided with different depth, B, B is right in A in right detector scanning image, and B is in A in left detector scanning image A left side, 2 points of relative position difference is L, and A, 2 depth difference H of B can be obtained by L and subtended angle α：

5. stereo imaging system as claimed in claim 1, wherein, the reconstructing device is based on the width in several transmission images Transmission image identifies casing, determines the position of casing, and carry out three-dimensional modeling to the casing.

6. a kind of method of stereo imaging system, the system includes radiographic source and respectively with predetermined angle and the radiographic source The multiple linear detector arrays being oppositely arranged, methods described include step：

Multiple ray fan-beams are sent from radiographic source；

When inspected object moves along the direction intersected with the ray fan-beam, the multiple linear detector array difference The intensity level that corresponding ray fan-beam transmits the inspected object is detected, forms the transmission plot corresponding with each row detector Picture；

Using any two width transmission image in several transmission images as double-visual angle figure, the depth for calculating object on transmission image is believed Breath, fusion is overlapped to the depth information of calculating, obtains the three-dimensional information of inspected object, carries out three-dimensional reconstruction,

Wherein, the multiple linear detector array is L-arm frame, using following formula to point on linear detector array direction Resolution is corrected：

Wherein, S represents the size along the direction of linear detector array, and N represents detecting module in linear detector array Number, D represent that radiographic source represents other detector modules relative to main beam to ray main beam directional detector module distance, Δ D The distance change of directional detector module.

7. method as claimed in claim 6, wherein, the alternate position spike of the object based on different depth between the two images is counted Calculate depth information.

8. method as claimed in claim 6, wherein, x and y coordinates are corrected using the depth information z of calculating.

9. method as claimed in claim 6, wherein, depth information is calculated as below：

2 points of A provided with different depth, B, B is right in A in right detector scanning image, and B is in A in left detector scanning image A left side, 2 points of relative position difference is L, and A, 2 depth difference H of B can be obtained by L and subtended angle α：