ES2868129T3 - Image processing method and electronic device - Google Patents

Abstract

A computer implemented method for image processing comprising: performing (101, 201, 301, 401) edge identification in an image; determining (102, 202, 302, 402) a filtering core for filtering processing on the image in accordance with an edge identification result; performing (103, 203, 303, 403) filtering processing on the image based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image; and performing (104) enhancement processing for the high-frequency image using a first gain coefficient of the high-frequency image to obtain an improved high-frequency image and performing image fusion for the low-frequency image and the image enhanced high-frequency for a processed image; wherein low-pass filtering processing is performed on the image based on the filtering core to obtain the low-frequency image, and the low-frequency image is subtracted from the original image to obtain the high-frequency image; characterized by: determining (102, 202, 302, 402) the filtering core for filtering processing on the image according to the result of the edge identification comprises: determining an edge coefficient on the image according to the result of edge identification, where the edge coefficient indicates edge information in the image, where the higher the edge coefficient, the more edge information the image comprises, while the lower the edge coefficient, the less edge information comprises the image; and determining a filter core size in accordance with the edge coefficient, wherein the filter core size is positively correlated with the edge coefficient; or the determination (102, 202, 302, 402) of the filtering core for a filtering processing on the image according to the result of the edge identification comprises: dividing the image into a region without background where the information of edge and a background region; and adaptively determining a size of the filter core according to a location where the edge information is found, wherein the size of the filter core for the bottomless region is larger than that of the background region.

Description

DESCRIPTION

Image processing method and electronic device

Technical field

The present disclosure relates generally to the technical field of electronic device and, in particular, to a method for image processing and an electronic device.

Background

With the rapid development of electronic gadgets, electronic gadgets such as mobile phones and tablet computers all have image capture functions, and users have increasingly high requirements for the quality of images captured by electronic gadgets.

In order to improve the definition of image details, adjusting the contrast of images is a common method in image processing. However, the adjustment of the image contrast is currently applied for the processing of medical images such as X-ray images. The limit of the field of application leads to the use of fixed parameters in the image contrast processing and they are not applicable to images containing complex information collected by electronic devices. The image processed by the current contrast adjustment method has fogging and ash problems.

In Zheng Xianhui et al. "Single-Image-Based Rain and Snow Removal Using Multi-guided Filter", November 3, 2013 (2013-11-03), INTELLIGENT VIRTUAL AGENT. VAT 2015. LNCS; [LECTURE NOTES IN COMPUTER SCIENCE; READ. NOTES COMPUTER], SPRING, BERLIN, HEIDELBERG, a method is provided to remove rain and snow using a multi-guided filter. However, in Zheng Xianhui et al., The general guided filter, which is independent of the size of the filter core, is used as the image filter.

In Bamber David C et al. "Novel adaptive kernels for image sharpening in the presence of noise", VISUAL INFORMATION PROCESSING XX, SPIE, 1000 20TH ST. BELLINGHAM WA 98225-6705 USA, vol. 8056, No. 1, May 13, 2011 (2011 -05-13), pages 1-7, an image sharpening method is provided based on the adaptive image filter kernel. However, in Bamber David C et al., The adaptive image filter kernel is not related to edge information.

In He K et al: "Guided Image Filtering", IEEE TRANSACTION ON PATTERN NANLYSIS AND MACHINE INTELLIGENT, vol. 35, January 1, 2013 (2013-01-01), page 1397-1409, a guided image filtering method is provided based on a filtering core related to edge information. However, in He K et al, the filtering core is not determined according to the result of the edge identification. Therefore, the fixed filtering core is adopted for the differences of the image captured by the electronic device, which causes the loss of image information as the fixed filtering core is poorly applicable and is not suitable for images captured by the electronic device with different scenes and different complexities.

Disclosure Summary

In accordance with one aspect of the present disclosure, a method for image processing is provided. The method is applied for an electronic device that includes a camera. The method includes obtaining an image captured by the camera and performing edge identification on the image; determining a filter core for a filter processing on the image in accordance with an edge identification result; performing the image filtering processing based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image; and perform enhancement processing for the high-frequency image using a first gain coefficient of the high-frequency image to obtain an improved high-frequency image and perform image blending for the low-frequency image and the high-frequency image Enhanced to get a processed image. Low-pass filtering processing is performed on the image based on the filtering core to obtain the low-frequency image, and the low-frequency image is subtracted from the original image to obtain the high-frequency image. Determining the filtering core for a filtering processing on the image according to an edge identification result comprises: determining an edge coefficient on the image according to the edge identification result, wherein the edge coefficient indicates edge information in the image, where the higher the edge coefficient, the more edge information the image includes, while the lower the edge coefficient, the less edge information the image includes; and determining a size of the filtering core according to the edge coefficient, wherein the size of the filtering core is positively correlated with the edge coefficient or determining the filtering core for a filtering processing on the image according to the edge identification result comprises: dividing the image into a bottomless region where the edge information is located and a background region; and adaptively determine a size of the filter core according to a location where it is finds the edge information, where the size of the filter core for the bottomless region is larger than that of the bottom region. In the image processing method, different kinds of filtering cores are adopted for the differences of the image captured by the electronic device, which avoids the problem of loss of image information, as the filtering core Fixed is poorly applicable and is suitable for images captured by the electronic device with different scenes and different complexities.

In accordance with another aspect of the present disclosure, an electronic device is provided. The electronic device includes a camera configured to capture an image, a non-transient memory that stores a computer program, and a processor, wherein the processor is configured to execute the computer program to perform a method in accordance with the above aspect.

Brief description of the drawings

FIG. 1 is a flow chart of a method for image processing in accordance with one embodiment of the present disclosure.

FIG. 2 is a flow chart of another method for image processing in accordance with one embodiment of the present disclosure.

FIG. 3 is a flow chart of yet another method for image processing in accordance with one embodiment of the present disclosure.

FIG. 4 is a flow chart of yet another method for image processing in accordance with one embodiment of the present disclosure.

FIG. 5 is a structural diagram of an image processing apparatus in accordance with one embodiment of the present disclosure.

FIG. 6 is a structural diagram of an electronic device in accordance with one embodiment of the present disclosure.

FIG. 7 is a structural diagram of another electronic device in accordance with one embodiment of the present disclosure.

Detailed description

The technical solutions of the present disclosure will be further described in accordance with detailed implementations and attached figures. The invention is defined by the appended claims.

A method for image processing is provided. The method is applied for an electronic device that includes a camera. The method includes obtaining an image captured by the camera and performing edge identification on the image; determining a filtering core for filtering processing on the image in accordance with the result of the edge identification; performing the image filtering processing based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image; and performing enhancement processing for the high-frequency image and performing image fusion for the low-frequency image and the enhanced high-frequency image to obtain a processed image.

In one embodiment, the image is in a chrominance-luminance separate color mode, and the chrominance-luminance separate color mode includes color YUV mode and color LAB mode. Performing filtering processing includes performing filtering processing for the luminance components of the image based on the filtering core.

In one embodiment, the camera includes an image sensor; and generating the image in the chrominance-luminance separate color mode includes converting raw signals captured by the image sensor into a color RGB mode image; and generating the color YUV mode image based on the color RGB mode image.

In one embodiment, determining a filtering core for a filtering processing on the image according to an edge identification result includes determining an edge coefficient on the image according to the edge identification result, wherein the edge coefficient indicates edge information in the image; and determining a size of the filter core in accordance with the edge coefficient, wherein the size of the filter core is positively correlated with the edge coefficient.

In one embodiment, the image is divided into a backgroundless region where the edge information is located and a background region. A filter core size is adaptively determined according to a location where the edge information is found, where the size of the filter core for the bottomless region is larger than that of the bottom region.

In one embodiment, performing enhancement processing for the high-frequency image includes: determining a first gain coefficient of the high-frequency image, wherein the first gain coefficient includes gain values of all pixels in the high-frequency image. high frequency, where a gain value of one pixel is determined according to a local standard deviation of a window with a preset size and with a pixel as a center, the local standard deviation of the window is obtained by calculating a local variance of window; and performing enhancement processing for the high-frequency image in accordance with the first gain coefficient.

In one embodiment, the value of a pixel gain is inversely proportional to the local standard deviation of the window with the pixel as a center.

In one embodiment, before performing enhancement processing for the high-frequency image in accordance with the first gain coefficient, the method further includes correcting the first gain coefficient in accordance with a scene identification result for the image. . The result of scene identification includes a ratio of high-frequency information to low-frequency information in the image.

In one embodiment, the first gain coefficient is reduced when the ratio of high-frequency information to low-frequency information in the image is greater than a preset value.

In one embodiment, the gain values in the first gain coefficient have different adjustment values. One of the gain values is subtracted by a corresponding adjustment value so that the first gain coefficient is reduced.

In one embodiment, the adjustment values are determined according to the luminance components of all pixels, wherein an adjustment value for a pixel is negatively correlated with a luminance component of the pixel.

In one embodiment, the method further includes determining a second low-frequency image gain coefficient; performing enhancement processing for the low-frequency image in accordance with the second gain coefficient; and performing image fusion for the enhanced high-frequency image and the enhanced low-frequency image to obtain the processed image.

In one embodiment, performing the enhancement processing for the low-frequency image according to the second gain coefficient includes identifying a flat region and a non-flat region in the low-frequency image according to the luminance information in the low-frequency image; and performing the image enhancement for the non-planar region in accordance with the second gain coefficient. The processed image is obtained by performing image fusion for the planar region, the enhanced non-planar region, and the enhanced high-frequency image.

In one embodiment, the low-frequency image is divided into a plurality of image regions, and each of the image regions has a pixel difference value that indicates a difference condition of the luminance information of all pixels in each of the image regions. The flat region includes image regions whose pixel difference values are less than or equal to a preset value. The non-planar region includes image regions whose pixel difference values are greater than the preset value. In one embodiment, the pixel difference value of each of the image regions is calculated in the following formula:

where A is the pixel difference value, p is a total number of pixels in each of the image regions, gb ( b = 1,2 , ... p) is a luminance component of one pixel in each one of the image regions, S is a mean value of local luminance of each of the image regions, and both p and b are positive integers greater than 0. The preset value is determined according to a mean value of difference local calculated according to a number of the image regions and the pixel difference value of each of the image regions.

An electronic device is provided. The electronic device includes a camera configured to capture an image, a non-transient memory that stores a computer program, and a processor, wherein the processor is configured to execute the computer program to perform a method as described above.

FIG. 1 is a flow chart of a method for image processing in accordance with one embodiment of the present disclosure. The method can be performed by an image processing apparatus, wherein the apparatus can be performed by software and / or hardware, and can usually be integrated into an electronic device. Referring to FIG. 1, the method can include the actions / operations in the following blocks.

At block 101, the method obtains an image captured by a camera and performs edge identification on the image.

To be exemplary, the electronic device in embodiments of the present disclosure may be a smart device configured with an image capturing apparatus, such as a mobile phone and a tablet computer.

In this embodiment, the image may be an image taken by the camera in accordance with a shooting instruction. The image may also be an image captured by the camera and displayed on a screen of the electronic device for users to preview, before the shooting instruction for the camera is executed.

Edge identification is performed on the image to extract a boundary line between an object and the background in the image. The contour points in the image can be detected first, and then the detected contour points are connected according to a link rule. Meanwhile, bypassed limit points are also detected and connected, and false limits are removed. The purpose of edge identification is to discover information about shape and reflection or transmittance in the image. By way of example, adjacent pixel values or luminance values of pixels in the image can be detected row by row and column by column to determine pixels with extremely changed pixel values or luminance values as edge pixels. Edge pixels can be connected to form a border. Alternatively, edge identification can also be performed on the image to calculate the image based on a Roberts edge operator, Sobel edge detection operator, or Laplacan edge operator, which is not limited.

At block 102, the method determines a filter core to perform filter processing on the image in accordance with an edge identification result.

The result of the edge identification may be to output the edge information in the image or to generate feature values indicating the edge information based on the identified edge information. The filter kernel is an operator kernel of a filter to perform filter processing on the image. Filtering cores with different sizes have different filtering effects. For example, for a filter with a smaller filter core, when performing filter processing, small details in the image can be preserved, while for a filter with a larger filter core, when performing filter processing, a large outline can be preserved in the image. By way of example, the filter core can be, among others, 3x3, 5x5, 7x7 or 9x9, etc.

In this embodiment, there is great difference between the image contents, which were captured when the electronic device shoots different objects. Edge identification is performed on the image and a filtering core applicable to the image is determined, so that the image contents are preserved in the filtering process, thus avoiding the loss of detailed information or contour information in the image. . Alternatively, determining a filtering core to perform filtering processing on the image according to the result of the edge identification includes: determining an edge coefficient on the image according to the result of the edge identification; and determining a filter core size to perform filter processing on the image in accordance with the edge coefficient, wherein the filter core size is positively correlated with the edge coefficient. The image edge coefficient is a characteristic value that is used to indicate edge information. That is, the edge coefficient of the image is used to indicate the edge information in the image. As an example, the higher the edge coefficient, the more edge information the image includes, while the lower the edge coefficient, the less edge information the image includes. To preserve the information in the image, the size of the filter core is positively correlated with the edge coefficient. That is, the higher the edge coefficient of the image, the greater the filtering core applied to the image. For example, when an image includes a white desktop and black buttons and spots uncovered on the desktop, it can be learned that, the image is relatively flat and contains less edge information, an edge coefficient derived from identifying the edges of the image. is smaller and consequently the filtering core applied to the image is smaller, for example it can be a 3x3 filtering core. When an image includes a plurality of tables, chairs, cabinets and a plurality of objects on the table, it can be learned that, the image is more complicated and contains more edge information, an edge coefficient derived from image edge identification is larger, and consequently the filtering core applied to the image is larger, for example, it may be a 9x9 filtering core.

Alternatively, a position of the edge information in the image is determined in accordance with the result of the edge identification for the image. The image is divided into a plurality of regions. The region with the edge information is filtered by a larger filter core. The background region of the image is filtered by a smaller filter core. In other words, the image is divided into a bottomless region and a background region, and the edge information is in the bottomless region. A size of the filter core is adaptively determined according to a location where the edge information is found, where the size of the filter core for the bottomless region is larger than that of the background region. Therefore, the image is filtered based on an adaptive filtering kernel, thus preserving the contour information and the detailed information of the image, to avoid the loss of the image information.

In block 103, the method performs filtering processing on the image based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image.

In this embodiment, a filter for filtering the image is a low-pass filter, and correspondingly, low-pass filtering processing is performed on the image. Specifically, low-pass filtering processing is performed on the image to obtain a low-frequency image corresponding to the original image, and the low-frequency image is subtracted from the original image to obtain a high-frequency image corresponding to the image. original. Specifically, the pixel difference values of the corresponding pixels between the original image and the low-frequency image to obtain a high-frequency image corresponding to the original image.

It should be noted that since the image may be an image in a multi-color mode, such as an image in RGB color mode, or an image in YUV color mode, the data of objects subjected to filtering processing is different. for images in different color modes. For example, for a color RGB mode image, the data object for filtering processing is an RGB value, and for a color YUV mode image, the data object for filtering processing is a separate Y component (i.e. luminance component).

At block 104, the method performs enhancement processing for the high-frequency image and performs image fusion for the low-frequency image and the enhanced high-frequency image to obtain a processed image.

The high-frequency image includes content information in the original image, and enhancement processing is performed for the high-frequency image, so that the contrast between the enhanced high-frequency image and the low-frequency image could be improved. A dynamic range is adjusted for the image to highlight objects in the image and improve image definition. By way of example, performing enhancement processing for the high frequency image may include setting gain values for the pixels in the high frequency image; multiplying the gain values by the pixel values or the luminance values of the pixels, respectively; and performing image fusion for the enhanced high-frequency image and the low-frequency image to obtain a processed image. The gain values for performing enhancement processing for the high-frequency image can be a fixed value. That is, the gain values for all pixels are identical. Alternatively, the gain values for performing enhancement processing for the high-frequency image can also be calculated according to the respective pixel and differ according to the respective pixel. Correspondingly, when the enhancement processing is performed for the high-frequency image, the pixel value or the luminance value of each of the pixels is multiplied by a corresponding gain value to obtain an enhanced image with high quality.

In the method for image processing according to embodiments of the present disclosure, edge identification is performed on a captured image, an adaptive filter core is determined according to the image identification result, the image is filtered. on the image according to the filter core applied to the image to obtain a high-frequency image and a low-frequency image, and enhancement processing is performed so that the high-frequency image generates a processed image. Through the above technical solution, different kinds of filtering cores are adopted for the differences of the image captured by the electronic device, which avoids the problem of loss of image information as the fixed filtering core it is poorly applicable, and it is suitable for images captured by the electronic device with different scenes and different complexities.

FIG. 2 is a flow chart of another method for image processing in accordance with one embodiment of the present disclosure. Referring to FIG. 2, the method can include the actions / operations in the following blocks.

At block 201, the method obtains an image in a chrominance-luminance separate color mode that is captured by a camera and performs edge identification in the image.

By way of example, the image in the chrominance-luminance separate color mode may be an image in the color YUV mode or the color LAB mode. In an image in the chrominance separate color mode, the luminance component and the chrominance component can be extracted separately, so that the image can be processed in terms of luminance and chrominance. By way of example, during the processing of the luminance components, no impact will be exerted on the chrominance components of the image.

It should be noted that RGB color mode, YUV color mode and LAB color mode can be converted to each other. Using a mobile phone as an example, when an image is captured by an image capture apparatus inside the mobile phone, one way to generate an image in the color YUV mode may include: converting the raw data obtained by an image sensor on an image in RGB color mode; and generating a color YUV mode image based on the color RGB mode image. The image capturing apparatus may be a camera, wherein the camera may include a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. Signals from a light source captured by the CCD image sensor or CMOS image sensor can be converted to digital signals as raw data, the raw data can be converted to image data in RGB color mode and Image data in RGB color mode is further converted to image data in YUV color mode. With the image capture apparatus of the mobile phone, an image in the joint photographic expert group (JPG) format can be formed by the image in the color YUV mode.

Image colors in color RGB mode, which is converted from raw data, may not be actual image colors, and therefore color RGB mode image data formed here is not can be processed in any way. Colors formed from image data in color YUV mode are actual image colors and therefore image data can be processed in color YUV mode. During the common processing of an image, RGB data can generally be processed, where the conversion of the color mode, which is raw data -> an image in RGB color mode -> an image in YUV color mode, can be perform for the raw data captured by the image sensor. The RGB color mode image can be processed to obtain a color RGB mode processed image, and then the color RGB mode processed image can be converted to a color YUV mode image to obtain a JPG format image. Accordingly, when images are processed in other color modes, the images must be converted to YUV color mode images, and then JPG format images can be obtained after the images are converted to YUV color mode images. colour.

In this embodiment, the image obtained in the chrominance-luminance separate color mode may be a color YUV mode or color LAB mode image. When the method for image processing of the present disclosure is applied in a mobile phone, alternatively, the image in the chrominance-luminance separate color mode may be an image in the YUV color mode, and the image in the color YUV mode Color YUV can be processed without redundant image conversion procedures after being captured by the image capture apparatus, reducing the image conversion process and improving image processing efficiency.

At block 202, the method determines a filter core to perform filter processing on the image in accordance with an edge identification result.

In block 203, the method performs filtering processing on the image based on the filtering core to obtain a high-frequency image and a low-frequency image corresponding to the image.

Alternatively, taking a color YUV mode image as an example, the color YUV mode image data is stored in planar format. In other words, the components Y, U, and V are respectively stored in different arrays. When the luminance components of all pixels in the image are transverse, the matrix is read to store the Y component, and then the respective luminance component of the respective pixel in the image can be obtained.

In this embodiment, the luminance components of the image are filtered, which means that the image is processed in the basic color mode. Therefore, the chrominance component of the image is not damaged, and color distortion and loss of detailed information during image processing is prevented. For example, only the Y component is filtered in the color YUV mode to get a high frequency image and a low frequency image associated with the Y component, and the Y component is adjusted and converted in the post enhancement processing, which does not affect a ratio between the U component and the V component at all. As such, the image color is guaranteed not to be distorted during processing and the image contrast is enhanced without damaging the image color. .

At block 204, the method calculates a local variance of a window with a preset size and with a reference pixel as a center in the high-frequency image.

At block 205, the method determines a gain value of the reference pixel in accordance with a local standard deviation corresponding to the local variance of the window.

At block 206, the method determines a first high-frequency image gain coefficient in accordance with gain values of all reference pixels in the high-frequency image.

For example, a reference pixel is any pixel (i, j), whose luminance component is x (i, j), where i is the horizontal coordinate of the reference pixel in the image, and j is the vertical coordinate of the pixel reference in the image. The window size can be (2n 1) ■ (2n 1), where n is an integer greater than or equal to 0. The above window size is for example only. In other embodiments, the above window can be in the shape of a rectangle. That is, the above window has the shape of (2n 1) ■ (2 m 1).

The local variance of the window can be calculated using the following formula:

Where,

In the above formula, mx ( i, j) is a local average value of the window, and x ( k, l) is a luminance component of a pixel in the window, where k and l are integers greater than or equal to 0 .

Ox ( i, j) is the local standard deviation of the window with the reference pixel as the center. Alternatively, the gain value of the reference pixel is inversely proportional to the local standard deviation. For example, the gain value of the reference pixel can be D / Ox ( i, j). Alternatively, the gain value of the reference pixel is greater than 1 to achieve the enhancement of the luminance components of the pixels in the high-frequency image, where D is a constant.

In this embodiment, through the above manner, the gain value of each of the pixels in the high-frequency image is determined, and the first gain coefficient of the high-frequency image is further determined. The first gain coefficient includes the gain values of all the pixels in the high-frequency image.

At block 207, the method corrects the first gain coefficient in accordance with a scene identification result for the image.

By way of example, the result of scene identification for the image may be to determine a ratio of high-frequency information to low-frequency information in the image. That is, the result of scene identification may include a radio from high-frequency information to low-frequency information in the image. When the ratio of the high-frequency information to the low-frequency information is relatively large, it indicates that the high-frequency image contains a large number of pixels to be enhanced. When the high-frequency image is enhanced according to the first gain coefficient, noise in the image is easily amplified to generate excess image, which affects image quality. In this embodiment, when the ratio of high-frequency information to low-frequency information is relatively large, the first gain coefficient is adaptively reduced, so that the first gain coefficient is corrected. Thus, it is understood that the first gain coefficient is adaptively adjusted in accordance with the result of scene identification so that the image does not enhance noise. Each of the pixels in the high frequency image has a different adjustment value for the pixel gain value. Alternatively, the adjustment value for the gain value is determined in accordance with the luminance component of the respective pixel. For example, the adjustment value for the gain value is negatively correlated with the luminance component of the respective pixel. For example, a light area that has a large luminance component is set with a small setting value for the gain coefficient, and a dark area that has a small luminance component is set with a large setting value for the gain coefficient. gain. Specifically, the correction of the first gain coefficient is that a corresponding adjustment value for each of the pixels in the high-frequency image is subtracted from an original gain value, then the corrected gain value is obtained, and then further corrected. the first gain coefficient of the high-frequency image. The coefficient of profit it is corrected according to the luminance components of all pixels, which prevents noise amplification caused when the dark region is enhanced with high gain.

At block 208, the method performs enhancement processing for the high frequency image in accordance with the corrected first gain coefficient.

At block 209, the method performs image fusion for the low-frequency image and the enhanced high-frequency image to obtain a processed image.

In the method for image processing according to embodiments of the present disclosure, the luminance components of the image in the chrominance-luminance separate color mode are filtered to obtain an associated high-frequency image and low-frequency image. With the luminance components, an adaptive gain value of each of the pixels is determined according to the luminance component of the pixel in the high-frequency image to generate the first gain coefficient of the high-frequency image, and the High-frequency image is enhanced to get the image processed. Since independent luminance components are analyzed and processed, the image color cannot be affected. It improves the contrast of the image in a case where it is guaranteed that there is no color distortion and a range of the image is dynamically expanded to make the details in the image definite, which improves the image quality.

FIG. 3 is a flow chart of yet another method for image processing in accordance with one embodiment of the present disclosure. Referring to FIG. 3, the method can include the actions / operations in the following blocks.

At block 301, the method obtains an image in a chrominance-luminance separate color mode that is captured by a camera and performs edge identification in the image.

At block 302, the method determines a filter core to perform filter processing on the image in accordance with an edge identification result.

In block 303, the method performs filtering processing on the image based on the filtering core to obtain a high-frequency image and a low-frequency image corresponding to the image.

At block 304, the method determines a first gain coefficient of the high-frequency image and a second gain coefficient of the low-frequency image, performs enhancement processing for the high-frequency image according to the first coefficient of gain and performs enhancement processing for the low-frequency image according to the second gain coefficient.

It should be noted that the low-frequency image includes background information and detailed image information. For example, an original image includes a black button and a dot spot on the white desktop, and the high-frequency image obtained by filtering processing may include the black button, and a low-frequency image includes the white desktop and the spot spot on the desktop, where the spot spot is low-frequency image details. In this embodiment, since the enhancement processing is performed synchronously for the high-frequency image and the low-pass image, all the content information in the image is enhanced at the same time, which avoids the loss of the detailed information in the low-frequency image.

The way to determine the second gain coefficient for the low-frequency image is the same as for the high-frequency image, which is not described here.

Alternatively, before the enhancement processing for the high-frequency image is performed according to the first gain coefficient and the enhancement processing for the low-frequency image is performed according to the second gain coefficient, the first gain coefficient and the second gain coefficient are corrected according to a scene identification result of the image, and then the high-frequency image is enhanced based on the first corrected gain coefficient, and the low-frequency image is enhanced based on to the second corrected gain coefficient.

At block 305, the method performs image fusion for the enhanced low-frequency image and the enhanced high-frequency image to obtain a processed image.

In the method for image processing according to embodiments of the present disclosure, for a chrominance-luminance separate color mode image, the luminance components of the image are filtered to obtain the low-frequency image and the image of high frequency, the first gain coefficient and the second gain coefficient are calculated respectively, the high-frequency image is enhanced according to the first gain coefficient, the low-frequency image is enhanced according to the second gain coefficient, and image fusion is performed for enhanced high-frequency image and low-frequency image Enhanced to get the processed image. Therefore, the contrast in the high-frequency image and the low-frequency image is simultaneously enhanced, the loss of details during image processing is prevented, and the image definition is enhanced in a case where no image distortion occurs. .

FIG. 4 is a flow chart of yet another method for image processing in accordance with one embodiment of the present disclosure. Referring to FIG. 4, the method can include the actions / operations in the following blocks.

At block 401, the method obtains an image in a chrominance-luminance separate color mode that is captured by a camera and performs edge identification for the image.

At block 402, the method determines a filter core to perform filter processing on the image in accordance with an edge identification result.

At block 403, the method performs filtering processing for the luminance components of the image based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image.

At block 404, the method determines a first gain coefficient of the high-frequency image, and performs enhancement processing for the high-frequency image in accordance with the first gain coefficient.

At block 405, the method identifies a flat region and a non-flat region in the low-frequency image according to the luminance information of all pixels in the low-frequency image.

At block 406, the method splits the low-frequency image into the flat region and the non-flat region.

At block 407, the method determines a second low-frequency image gain coefficient, and performs image enhancement for the divided non-planar region in accordance with the second gain coefficient.

At block 408, the method performs image fusion for the planar region, the enhanced non-planar region, and the enhanced high-frequency image to obtain a processed image.

In some embodiments, identifying the flat region and a non-flat region in the low-frequency image according to the luminance information of all the pixels in the low-frequency image may include performing partitioning processing for the low-frequency image. frequency to obtain a plurality of image regions and determine the pixel difference values of the image regions; determining the image regions whose pixel difference values are less than or equal to a preset value to belong to the flat region; and determining the image regions whose pixel difference values are greater than the preset value to belong to the non-planar region. That is, the flat region includes the image regions whose pixel difference values are less than or equal to a preset value and the non-flat region includes the image regions whose pixel difference values are greater than the preset value.

For any of the image regions, the pixel difference value of the image region can be calculated by the following formula:

v

1 'ST'

A = -> ( gb - g)

p ¿—i

_{b = l _}

Where A is a pixel difference value of the image region; p is a total number of pixels in the image region; gb ( b = 1,2, --- p) is a luminance component of a pixel in the image region; 8 is an average local luminance value of the image region; and both p and b are positive integers greater than 0.

The pixel difference value indicates a difference condition of the luminance information of all the pixels in the image region. For example, a larger pixel difference value indicates that the luminance information of all pixels in the image region differs greatly, and a smaller pixel difference value indicates that the luminance information of all pixels in the image region it has a greater similarity. Image regions whose pixel difference values are less than or equal to the preset value are joined to form the flat region, and image regions whose pixel difference values are greater than the preset value are joined to form the non-planar region . Alternatively, the preset value for determining the flat region and the non-flat region is related to an average value Á of local difference of the low-frequency image. That is, the preset value for determining the flat region and the non-flat region is determined from according to the mean value 4 of local difference. Specifically, the local difference average value 4 of the low-frequency image is determined according to the number of image regions and the pixel difference value A of the respective image region. The preset value can be A4 by inputting a coefficient A. In other words, when a pixel difference value A < A4, the image region with the pixel difference value A belongs to the flat region; and when pixel difference value A> A4, the image region with pixel difference value A belongs to the non-planar region.

As an example, for an image that includes a black button and dotted spots on the white desktop, the high-frequency image obtained by filtering processing may include the black button, and the low-frequency image includes the white desktop and dot spots on the desktop. In the low-frequency image, a dotted portion is a non-flat region, and the white desktop as a background region is a flat region. In the above embodiment, it can be learned that the pixel gain value is inversely proportional to the local standard deviation. Since the local standard deviation of the flat region is quite small, the gain value of the pixel is relatively large and therefore noise amplification will be introduced. In this embodiment, a flat region and a non-flat region are identified and divided from the image, and only the non-flat region is enhanced. That is, no adjustment is made for the luminance components in the flat region, which prevents amplification of noise in the flat region when the image is enhanced.

In the method for image processing according to embodiments of the present disclosure, the planar region and the non-planar region are divided from the low-frequency image, and the non-planar region has been enhanced to avoid loss of detail in the image. low-frequency image, keep the luminance of the flat region and avoid noise amplification.

In one embodiment, a preview image or a color YUV mode captured image captured by a camera can be displayed on a screen interface of an electronic device (eg, a mobile phone), and can be extracted a color magnification instruction entered by a user. The color magnification instruction can be entered such that the user clicks on a virtual controller (for example, a proportional integral controller) on the screen interface, or by user touch gestures or voice commands. Edge identification is performed on the image displayed on the screen interface according to an image processing instruction, a filter core of the low-pass filter is determined according to an edge identification result, and a low-pass filtering for the luminance components of the image to obtain a high-frequency image and a low-frequency image related to the luminance components, the gain coefficients for the high-frequency image and the low-pass image are calculated , respectively, the flat region and the non-flat region are identified and divided of the low-frequency image, the high-frequency image and the non-flat region of the low-frequency image are enhanced, and image fusion is performed for the the flat region, the enhanced non-flat region, and the enhanced high-frequency image to obtain the processed image. In this embodiment, since a color YUV mode image is processed directly, it saves time for color mode conversion, and since only the Y components of the image are processed, it does not affect the relationship between U and V, and it has no change in chrominance components. That is, the color of the image will not be damaged and there will be no distortion in any area of the image. As the high-frequency image and the non-flat region of the low-frequency image are enhanced, the details in the image are improved and the flat region of the low-frequency image is maintained. Therefore, the noise is controlled to avoid amplification of the noise.

FIG. 5 is a structural diagram of an image processing apparatus in accordance with one embodiment of the present disclosure. The apparatus can be implemented by software and / or hardware and can usually be integrated into an electronic device to execute the method for image processing. Referring to FIG. 5, the apparatus may include an edge identification module 501, a filter core determination module 502, a filter processing module 503, and an image enhancement module 504.

The edge identification module 501 is configured to obtain an image captured by a camera and perform edge identification on the image.

The filter core determining module 502 is configured to determine a filter core to perform filter processing on the image in accordance with an edge identification result.

The filter processing module 503 is configured to perform filter processing on the image based on the filter core to obtain a high-frequency image and a low-frequency image corresponding to the image.

The image enhancement module 504 is configured to perform enhancement processing for the high-frequency image and performs image fusion for the low-frequency image and the enhanced high-frequency image to obtain a processed image.

In the image processing apparatus according to embodiments of the present disclosure, different types of filtering core are adopted for the differences of the images captured by the electronic device, which avoids the problem of loss of image. the image information as the fixed filtering core is poorly applicable and is suitable for images captured by the electronic device with different scenes and different complexities.

Based on the above embodiments, the filter core determination module 502 is configured for the following actions / operations.

Determine an edge coefficient in the image according to the result of edge identification.

Determine a filter core size to perform filter processing on the image according to the edge coefficient, where the filter core size is positively correlated with the edge coefficient.

Based on the above embodiments, the image enhancement module 504 may include the following.

A local variance determination unit, which is configured to calculate a local variance of a window with a preset size and with a reference pixel as a center in the high-frequency image.

A gain value determining unit, which is configured to determine a gain value of a reference pixel in accordance with a local standard deviation corresponding to the local variance of the window. A first gain coefficient determining unit, which is configured to determine a first gain coefficient of the high-frequency image in accordance with gain values of all reference pixels.

A first enhancement processing unit, which is configured to perform enhancement processing for the high-frequency image in accordance with the first gain coefficient.

Based on the above embodiments, the apparatus further includes the following.

A gain coefficient correction module, which is configured to correct the first gain coefficient according to a scene identification result for the image before performing enhancement processing for the high-frequency image according to the first profit coefficient.

Based on the above embodiments, the apparatus further includes the following.

A low-frequency image enhancement processing module, which is configured to determine a second gain coefficient of the low-frequency image and perform enhancement processing for the low-frequency image in accordance with the second gain coefficient.

Accordingly, the image enhancement module 504 is configured to perform image blending for the enhanced high-frequency image and the enhanced low-frequency image to obtain the processed image. Based on the above embodiments, the low-frequency image enhancement processing module may include the following.

An image identification unit, which is configured to identify a flat region and a non-flat region in the low-frequency image according to the luminance information of all the pixels in the low-frequency image.

An image dividing unit, which is configured to divide the low-frequency image into the flat region and the non-flat region.

A non-planar region enhancement processing unit, which is configured to perform image enhancement for the divided non-planar region according to the second gain coefficient.

Accordingly, the image enhancement module 504 is configured to perform image fusion for the planar region, the enhanced non-planar region, and the enhanced high-frequency image to obtain a processed image. Based on the above embodiments, the image identification unit is configured for the following actions / operations.

Perform partitioning processing for the low-frequency image to obtain a plurality of image regions and determine the pixel difference values of the image regions.

Determine the image regions whose pixel difference values are less than or equal to a preset value to belong to the flat region.

Determine the image regions whose pixel difference values are greater than the preset value to belong to the non-planar region.

Based on the above embodiments, the image is an image in a separate chrominance color mode. The chrominance-luminance separate color mode may include a color YUV mode and a color LAB mode.

Based on the above embodiments, the apparatus further includes the following.

A generation module for the color YUV mode image, which is configured to convert raw signals captured by a camera image sensor into an RGB color mode image and generate a color YUV mode image in based on the image in RGB color mode.

The present disclosure may further provide a storage medium, which may include instructions executable by a computer processor, wherein the instructions can be used to execute the method for image processing while being executed by the computer processor, the method may include the following actions / operations.

The method obtains an image captured by a camera and performs edge identification on the image.

The method determines a filtering core to perform filtering processing on the image in accordance with an edge identification result.

The method performs filtering processing on the image based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image.

The method performs enhancement processing for the high-frequency image and performs image fusion for the low-frequency image and the enhanced high-frequency image to obtain a processed image.

A storage medium can be any type of non-transient memory devices or storage devices. The term "storage medium" is intended to include an installation medium, such as CD-ROM devices, a floppy disk, or a tape; non-transient memory of a computer system or random access memory, such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM and the like; non-volatile memory, such as flash memory, magnetic medium (such as hard disk or optical storage); a record or other type of items for storage. The storage medium can also include other types of storage or a combination thereof. Furthermore, the storage medium can be located in a first computer system from which programs can be executed, or located in a second computer system different from the first, where the second computer system can be connected to the first computer system through a network. . (such as the internet). The second computer system can provide systemic instructions for the first computer to execute. The term "storage medium" can include two or more storage mediums that reside in different locations (eg, different computer systems connected via the Internet). The storage medium can store program instructions, which can be executed by one or more processors (for example, a specific embodiment can be a computer program).

One embodiment of the present disclosure may provide a storage medium that includes computer-executable instructions, wherein the computer-executable instructions may not be limited to the operations mentioned above for image processing, but may include any related operation of the methods of processing. image processing provided by any of the embodiments of the present disclosure.

The present disclosure may provide an electronic device, which may integrate an image processing apparatus provided by embodiments of the present disclosure. FIG. 6 is a structural diagram of an electronic device in accordance with one embodiment of the present disclosure. Electronic device 600 may include non-transient memory 601, processor 602, and computer program stored in non-transient memory 601 and executed by processor 602, wherein processor 602 can execute computer program to perform the method for processing. imaging as described in the present disclosure.

In the electronic device according to embodiments of the present disclosure, different types of filtering cores are adopted for the differences of the images captured by the electronic device, which avoids the problem of loss of the image information as The fixed filtering core is poorly applicable, and it is suitable for images captured by the electronic device with different scenes and different complexities.

FIG. 7 is a structural diagram of yet another electronic device provided by embodiments of the present disclosure. The electronic device may include a housing (not shown in the figure), a non-transient memory 701, a central processing unit (CPU) 702, a circuit board (not shown in the figure), and an electrical circuit (not shown in the figure). The circuit board can be arranged within the space defined by the housing. CPU 702 and non-transient memory 701 can be arranged on the circuit board. The electrical circuit can provide power to each of the circuits or elements of the electronic device. Non-transient memory 701 can store codes for executable programs. CPU 702 can execute computer programs corresponding to executable program codes by reading executable program codes stored in non-transient memory 701. In such a way, the following actions / operations can be achieved.

Obtain an image captured by a camera and perform edge identification on the image.

A filtering core is determined to perform filtering processing on the image in accordance with an edge identification result.

Filtering processing is performed on the image based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image.

Enhancement processing is performed for the high-frequency image, and image fusion is performed for the low-frequency image and the high-frequency image to obtain a processed image.

The electronic device may further include a peripheral interface 703, a radio frequency (RF) circuit 705, an audio frequency circuit 706, a speaker 711, a power management chip 708, an input / output subsystem 709 (E / S), another input / control device 710, a touch screen 712, another input / control device 710 and an external interface 704, each of which can communicate via one or more communication buses or lines 707 of sign.

It should be understood that, the electronic device 700 shown in the figure is just one example of electronic devices, the electronic device 700 may contain more or less of the components than those illustrated in the figure, two or more of the components may be combined, or they may be combined. you can change the arrangement of the components. The components illustrated in the figure can be made by hardware, software or a combination thereof, where the hardware and software can include one or more signal processors and / or application specific integrated circuits.

The electronic device for the image processing operations provided by the present disclosure will be described in detail herein, and a mobile phone can be used as an example of the electronic device.

Non-transient memory 701 can be accessed by CPU 702, peripheral interface 703, and the like. Non-transient memory 701 may include high speed random access memory or non-volatile memory, such as a disk-type storage device, flash memory device, or other volatile solid-state memory device.

Peripheral interface 703 can connect device input and output to CPU 702 and non-transient memory 701.

The I / O Subsystem 709 can set the input and output peripherals. For example, the touch screen 712 and other input / control devices 710 can be connected to the peripheral interface 703. The I / O subsystem 709 may include a display controller 7091 and one or more input controllers 7092 to control other input / control devices 710. The one or more input controllers 7092 may receive electrical signals from other input / control devices 710 or send electrical signals to other input / control devices 710, where the other input / control devices 710 may include a physical button (a button pressure, a rocker button and the like), a marking plate, a slide switch, a control lever or a click wheel. It should be noted that the 7092 input controller can be connected to any of the following: a keyboard, an infrared port, a USB interface, and indicating equipment, such as a mouse.

The touch screen 712 can provide an input interface and an output interface between a user and the user's electronic device. The visual output can be displayed to the user, where the visual output can include a graphic, text, icon, video, and the like.

The I / O subsystem 709 display controller 7091 can receive electrical signals from the touch screen 712 or send electrical signals to the touch screen 712. The touch screen 712 can detect screen contact, the screen controller 7091 can convert the detected contact into an interaction with a user interface object displayed on the touch screen 712, which realizes the human-computer interaction. The user interface object displayed on the touch screen 712 may be an icon for running a game, an icon for connecting to a certain network, and the like. It should be noted that the device may also include an optical mouse, which is a touch-sensitive surface with no visual output display or an extension of the touch-sensitive surface defined by the touch screen.

The RF circuit 705 can be used to establish communication between the mobile phones and the wireless network (ie, one side of the network), performing the receiving and sending of data between the mobile phone and the wireless network. For example, receiving and sending messages, emails and the like. To be specific, the RF circuit 705 can receive and send RF signals, which are also called electromagnetic signals, the RF circuit 705 can convert electrical signals into electromagnetic signals or convert electromagnetic signals into electrical signals and communicate with communications networks and other devices through electromagnetic signals. The RF circuit 705 may include a known circuit to perform the above functions, wherein the known circuit may include, but is not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM), and the like.

Audio frequency circuit 706 can receive audio frequency data from peripheral interface 703 and convert the audio frequency data to electrical signals, which can be sent to speaker 711.

The speaker 711 can reconvert the audio signals received from the wireless network through the RF circuit of the mobile phone into sound and reproduce the sound for the user.

Power management chip 708 can supply power to connected hardware through CPU 702, I / O subsystem, and peripheral interface and performs power management.

The image processing apparatus, storage medium and electronic device provided by the above-mentioned embodiments can execute the method for image processing provided by any of the embodiments and has corresponding functional modules for executing the methods and achieving the effects. beneficial. Technical details that are not described in detail in the above embodiment may refer to the method for image processing described in any of the embodiments.

The invention is defined by the appended claims.

Claims (13)

1. A computer-implemented method of image processing comprising: perform (101,201,301,401) edge identification in an image; determining (102, 202, 302, 402) a filtering core for filtering processing on the image in accordance with an edge identification result; performing (103, 203, 303, 403) filtering processing on the image based on the filtering core to obtain a low-frequency image and a high-frequency image corresponding to the image; and performing (104) enhancement processing for the high-frequency image using a first gain coefficient of the high-frequency image to obtain an improved high-frequency image and performing image fusion for the low-frequency image and the image enhanced high-frequency for a processed image; wherein low-pass filtering processing is performed on the image based on the filtering core to obtain the low-frequency image, and the low-frequency image is subtracted from the original image to obtain the high-frequency image; characterized by: determining (102, 202, 302, 402) of the filtering core for filtering processing on the image according to the result of the edge identification comprises: determining an edge coefficient on the image according to the result of the edge identification, where the edge coefficient indicates edge information in the image, where the higher the edge coefficient, the more edge information the image comprises, while the lower the edge coefficient, the less edge information understand the image; and determining a size of the filter core in accordance with the edge coefficient, wherein the size of the filter core is positively correlated with the edge coefficient; or the determination (102, 202, 302, 402) of the filtering core for a filtering processing on the image according to the result of the edge identification comprises: dividing the image into a backgroundless region where the edge information is found and a background region; and adaptively determining a size of the filter core according to a location where the edge information is found, wherein the size of the filter core for the bottomless region is larger than that of the background region. The method according to claim 1, wherein the electronic device (600, 700) comprises a camera, and the image is captured by the camera and in a separate chrominance-luminance color mode, and the color mode chrominance-luminance separate comprises color YUV mode and color LAB mode; and the embodiment (103, 203, 303, 403) of the filtering processing comprises: perform filtering processing for the luminance components of the image based on the filtering core. 3. The method according to claim 2, wherein the camera comprises an image sensor; and generating the image in the chrominance-luminance separate color mode comprises: converting raw signals captured by the image sensor into a color RGB mode image; and generating the color YUV mode image based on the color RGB mode image. The method according to claim 1, wherein the enhancement processing embodiment (104) for the high frequency image comprises: determining (206) the first gain coefficient of the high-frequency image, wherein the first gain coefficient comprises gain values of all pixels in the high-frequency image, wherein a one-pixel gain value is determined from According to a local standard deviation of a window with a preset size and with a pixel as a center, the local standard deviation of the window is obtained by calculating a local variance of the window; and performing (208) enhancement processing for the high frequency image in accordance with the first gain coefficient. The method according to claim 4, wherein the gain value of a pixel is inversely proportional to the local standard deviation of the window with the pixel as a center. The method according to claim 4, before performing (208) enhancement processing for the high frequency image according to the first gain coefficient, further comprises: correcting (207) the first gain coefficient in accordance with a scene identification result for the image; wherein the result of scene identification comprises a ratio of high frequency information to low frequency information in the image. The method according to claim 6, wherein the first gain coefficient is reduced when the ratio of high-frequency information to low-frequency information in the image is greater than a preset value. The method according to claim 7, wherein the gain values in the first gain coefficient have different adjustment values; and one of the gain values is subtracted by a corresponding adjustment value so that the first gain coefficient is reduced. The method according to claim 8, wherein the adjustment values are determined in accordance with the luminance components of all pixels in the high-frequency image, wherein an adjustment value for a pixel is negatively correlated with a luminance component of the pixel. The method according to any one of claims 1-9, further comprising: determining (304) a second low-frequency image gain coefficient; performing (304) enhancement processing for the low-frequency image in accordance with the second gain coefficient; and performing (305) image fusion for the enhanced high-frequency image and the enhanced low-frequency image to obtain the processed image. The method according to claim 10, wherein the embodiment (304) of the enhancement processing for the low-frequency image according to the second gain coefficient comprises: identifying (405) a flat region and a non-flat region in the low-frequency image according to the luminance information in the low-frequency image; and performing (407) image enhancement for the non-planar region in accordance with the second gain coefficient; and wherein the processed image is obtained by performing (408) image fusion for the planar region, the enhanced non-planar region, and the enhanced high-frequency image. The method according to claim 11, wherein the low-frequency image is partitioned into a plurality of image regions and each of the image regions has a pixel difference value indicating a difference condition. of the luminance information of all the pixels in each of the image regions; the flat region comprises the image regions whose pixel difference values are less than or equal to a preset value; the non-planar region comprises the image regions whose pixel difference values are greater than the preset value; the pixel difference value of each of the image regions is calculated in the following formula:

where A is the pixel difference value, p is a total number of pixels in each of the image regions, gb ( b = 1,2, --- p) is a luminance component of one pixel in each one of the image regions, 8 is a mean local luminance value of each of the image regions, and both p and b are positive integers greater than 0; and the preset value is determined in accordance with a local difference mean value calculated in accordance with a number of the image regions and the pixel difference value of each of the image regions. 13. An electronic device (600, 700), comprising a camera configured to capture an image, a non-transient memory (601) that stores a computer program and a processor (602), characterized in that the processor is configured to execute the computer program for performing a method in any one of claims 1-12.