JP2761449B2 - Image processing system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to image processing systems and methods, and more particularly to a parallel processing system using a pipelined image processing technique.

[0002]

2. Description of the Related Art Single Instruction-Multiple Data Stream (SIMD)
And the multiple instruction-multiple data stream (MIMD) parallel processing system has the following restrictions .

In a SIMD system, partitions of the input data segment make it difficult to achieve proper processor load balancing during parallel execution of the data segment. Also, the incremental data bandwidth requirements on the input data path can burden the design when it is desired to increase the number of parallel processors in a SIMD system.

In a MIMD system, there is generally a complexity in data flow communication and synchronization between multiple processors.

These problems are particularly apparent when blocks of image data are being processed, for example, when the image data is being compressed (encoded) or decompressed (decoded). In that case, the source of the image data, such as a camera, or the sink of the image data, such as a display monitor, operates in real time at a fixed image data rate, and the image data processing system determines the desired source or sink.・Synchronize with image / data transfer speed
Obtain should be. That is, the image data processing system prevents loss of input image data, and
(Or) It must be able to process the image data at a rate that prevents the creation of undesirable visible artifacts when the image data is displayed.

In order to obtain optimal processing performance from a process pipeline architecture, the pipeline data flow should be maintained as completely as possible. However, this is made difficult by data flow variations that commonly occur during operation of the system. Also, full monitoring of pipeline control can represent an expensive overhead element for the host system or dedicated monitoring coprocessor.

US Pat. No. 5,046,080 discloses a video horn system for providing video horn services within a narrow band digital network. The host CPU 20 transmits an image via the bus 21 and the shared memory 24.
Connected to bus 28. The CODEC 25 includes a plurality of processing elements (PE) connected as a pipeline. Each PE includes a digital signal processor (DSP) and performs some of the functions of image coding. The FIFO memory module 64 includes a flag circuit and a FIFO memory. The flag circuit controls the data flow between the processing elements. Shared memory is used to store messages and performs synchronization between processing elements. It does not describe how to optimize the pipeline architecture to operate more efficiently with blocks of data.

[0008]

SUMMARY OF THE INVENTION A first object of the present invention is to provide a process pipeline in which data buffering and local control functions are inserted into the process pipeline at the boundaries of separate process pipeline functions. To provide an architecture.

A second object of the present invention is to provide a process pipeline architecture in which separate stages of the pipeline include a local data buffer and possibly an address buffer, and a local controller for monitoring local synchronization and timing. Is to do.

It is a third object of the present invention to provide a system for encoding and decoding blocks of video / image data and to provide a method for optimizing a pipeline architecture.

[0011]

SUMMARY OF THE INVENTION The above and other problems are solved and an object of the present invention is achieved by an image processing system constructed with a process pipeline architecture having a plurality of serially connected computation routines or functions. Achieved. Buffers between adjacent, serially connected routines or functions, e.g., first-in-first-out buffers
(FIFO) and Local State Machine (LSM) are inserted. This sequential process pipeline processing architecture minimizes synchronization complexity in a MIMD environment, but also increases the flexibility of the image processing system to changes in functionality . New functions or routines are added and inserted by determining new processing boundaries in the serial process pipeline structure and inserting new functions or routines at the determined boundaries.

Buffers, such as FIs, provide the pipeline structure with the ability to handle a wide range of data flow variations.
FO is inserted between processing stages.

According to another aspect of the invention, a local state machine (LSM) control block is inserted at the boundary between two adjacent processes to manage a local synchronization task,
The problem of increasing the overhead required by a typical host or dedicated coprocessor is solved. process·
To determine the correct order of processes in a pipeline architecture, LSM uses a FIFO flag with the interprocess initial connection protocol.

That is, the present invention provides a sequential process pipeline architecture with embedded buffers and local state machine controls between processes. Buffer and LS
M in combination provides a dynamic storage buffer and also provides data flow control for the process sequence. These characteristics increase the flexibility of the system and the scalability of the system to various applications, including but not limited to image processing applications.

More specifically, one aspect of the present invention provides a sequential process pipeline for use in a data processing system. A sequential process pipeline includes a plurality of processing stages that each execute a process on data organized as data blocks. A plurality of processing stages are connected in series with one another to deliver blocks of data between them. The first processing stage is connected to the data processing system at a data processing system resource capture boundary to input data from the data processing system at a first data rate and output data to the data processing system. Final processing stage, the data source and operating at a second data rate different from the first data rate
(Or) Connected to a data sink.

Each of the processing stages includes a data buffer inserted between the processing stage and an adjacent processing stage. Further, the first processing stage includes an interface data buffer inserted between the data processing system resource capture boundary and the first processing stage.

Each size (SIZE OF BUFFER) of a data buffer inserted between adjacent processing stages is determined by the following equation.

SIZE OF BUFFER (i) = (RP (i) −RP (i + 1)) × Pmax Here, in the case of the processing stage for executing the process (i),
i is the process sequence number associated with the processing stage in the process pipeline. Pmax is the process (i)
Is the maximum time required to completely process a data block, RP (i) is the data rate generated by process (i), and RP (i + 1) is the subsequent process (i + 1) Is the data rate from

The sequential process pipeline further includes a local controller associated with each of the processing stages to control the execution of the process by the processing stages.

In a preferred embodiment of the invention, the data block consists of image data and one of the processing stages performs a discrete cosine transform process to generate coefficients,
One of the processing stages performs a discrete cosine transform quantization process, and one of the processing stages performs a process to reduce the entropy of the quantized discrete cosine transform coefficients.

In the preferred embodiment, the first processing stage is connected to the second processing stage through a plurality of buffers, including an image data input buffer, an image data output buffer, and an address buffer. You. The address buffer stores an address for identifying a head address of a block of an address in the image memory. Each of the blocks of addresses in the image memory stores a block of decompressed image data. The local controller is responsive to writing the address to the address buffer to initiate operation of a processing stage that performs a discrete cosine transform process and a discrete cosine transform quantization process.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description of the present invention is constituted as follows. Section (A) describes the sequential process pipeline architecture, one aspect of the present invention, in connection with FIG. 1, and section (B) describes FIG. 1 in connection with FIGS. Example based video /
The entire configuration and operation of the preferred embodiment of the image processing pipeline is described, and section (C) is described in FIGS.
A preferred embodiment of the video / image processing pipeline will be described in more detail with reference to FIG.

Section (A) FIG. 1 illustrates a portion of an image processing system 10 constructed and operated in accordance with the present invention, in particular, multiple stages (stage (i-1) through stage (i-1)).
FIG. 3 is a simplified block diagram showing a stage (i + 1)) sequential process pipeline 12. The host processor 14 is connected to the stage (i-1) of the process pipeline 12 at a system resource capture boundary (SRIB) 14a.

Generally, the sequential process pipeline 12
Can be considered as a series of processing units linked in a predetermined sequence of processes, partitioned based on those routines or functions. These processes are intended to work in concert to achieve the desired computational end result.

Stage (i) of Process Pipeline 12
And stage (i + 1) comprises, for example, first and second buffers 16a and 16b, local state machine 18 and process block
Including 20. The process pipeline architecture of the present invention is characterized by the systolic nature of the local state machine 18 having an associated buffer 16a, 16b, each functioning as a data repository. LSM
Process block 20, buffer 16a under control of 18
And control and data are transferred between the storage devices via the buffer 16b.

The self-timing, LSM drive control embedded in the buffers 16a, 16b and the sequential process pipeline 12 performs a series of image processing operations and / or other operations on digital image data. Well suited for Assuming that the processes are self-triggered in the correct sequence, this approach provides automatic, flexible process pipeline control on a block-by-block basis through the process pipeline.

Various aspects of the operation of the process pipeline 12 will now be described in detail.

Process Synchronization Based on a given demand for data throughput intended to implement the process pipeline 12 and tracking along a sequential process in the process pipeline 12, the worst case processing time of the process Can be identified.

The maximum required time, expressed as the number of processing cycles of each calculation routine or function (hereinafter referred to as Pma
x) represents the maximum number of processing cycle counts required to completely process a block of data. Therefore, the magnitude of Pmax can be regarded as a global synchronization interval for each calculation process. The global synchronization clock obtained as a result of execution by host processor 14 informs individual process blocks 20 when to begin processing. This notification
This is done by an available line connected to stage (i-1) across SRIB 14a.

Underlying the global synchronization performed by the host processor 14 is an inter-process data flow communication managed locally by each embedded local state machine 18. The self-activating local state machine 18, which manages and synchronizes each of the processes partitioned by the process pipeline 12, uses a protocol in determining the correct timing to achieve process synchronization, as described in detail below. Run.

The number of processing cycles of a non-computed event, such as the processing cycles required to perform shared system memory resource arbitration, can sometimes dominate the overall processing effort. Therefore, non-computed events are preferably provided with an override mode if possible. As used herein, the maximum time to provide a non-computed event,
Or the number of cycles is indicated by Tmax. This is a resource dependent variable of the image processing system 10. As used herein
Tmax may be present if and only if the resource arbitration overhead (or the maximum time required to perform resource arbitration) of image processing system 10, referred to as ys, is greater than Pmax.
In FIG. 1, the available signal line 14b connected between the host processor 14 and the first LSM 18 of the sequential pipeline is A
LSM used as sys indicator or otherwise used 1
Acts as an override to Pmax of 8. Therefore, the available signal line 14b is
Provides global synchronization of pipeline 12.

Therefore, the protocol of each process can be described as follows in a pseudo code example. IF (Asys is greater than Pmax) AND IF (buffer (i-1)
Is NOT EMPTY) Tmax = Asys; Start PROCESS (i) for every Tmax cycle; ELS
E IF (Asys not greater than Pmax) AND IF (buffer (i-1) is NOT EMPTY) Initiate PROCESS (i) every Pmax cycle; END IF.

Maximum system dependent processing time (ie, Tmax)
Triggers by overriding LSM 18 by default, otherwise using the local self-timing reference denoted by Pmax. That is, a process sometimes has to wait for slower system arbitration until the longest processing cycle (ie, Tmax) allows the process to start. Thus, this dynamic control process pipeline 12 mode integrates and adapts the resource timing of the image processing system 10.

Buffer 16 (1) between process blocks 20
6a and 16b) logically smooth out data flow density variations. Boundary buffer 16 also serves another purpose: separating data dependent processes from data independent processes. In many signal processing routines, the relative sample rate of the dataflow node is independent of the data. Thus, the situation of the present invention implemented in the sequential process pipeline 12 distinguishes data dependent processes from data independent processes.

The determination of where to provide process boundaries and how to determine the size of the associated buffer 16 between subsequent processes is dictated by factors that can be classified into the following three categories: (1) System arbitration overhead incurred from various resources (ie, Asys). (2) Processing characteristics of the relative sample rate of data flow within a process (ie, is a given process data independent ?). (3) Input and output data bandwidth requirements of each process block 20 in the process pipeline 12, i.e., constraints

To accommodate short term speed mismatches between two adjacent process blocks 20, the size of the inserted buffer (SIZE OF BUFFER) is determined by:

SIZE OF BUFFER (i) = (RP (i) −RP (i + 1)) × Pmax (1) where Pmax is a processing time interval of each calculation process, that is, a maximum processing time. (i) is the data rate generated by process (i), RP (i + 1) is the data rate from subsequent process (i + 1), and i is the process pipeline.
12 is the process sequence number.

To achieve continuous data flow in the process pipeline 12, the size of the buffer 16 across the SRIB 14a is considered separately. When an intersection with the resources of the system 10 occurs, Pmax in equation (1) is replaced by Tmax, and SRIB
SIZE_SRI of interface data buffer in
B (the size of SRIB) is determined by the following equation.

SIZE_SRIB = (RP (i)-RP (i + 1)) x Tmax (2)

From the foregoing, the context of the present invention provides a method for defining a sequential process pipeline for use in a data processing system. The method includes the step of defining a plurality of data processing stages each performing a process on data organized as data blocks.
Each of these stages is defined, for example, to separate data-dependent processes from data-independent processes. The next step (b) of the method is to connect a plurality of defined data processing stages in series with one another in a selected order to achieve the desired processing function. The third step of the method is to provide an interface between adjacent ones of the serially connected data processing stages, with a data block buffer and a local controller for controlling the transfer of data blocks across the interface. provide.

Section (B) FIG. 2 is a simplified circuit block diagram illustrating an embodiment of an image data encoding and decoding system constructed according to the process pipeline architecture of FIG.

The purpose of the system is to perform compression (encoding) and decompression (decoding) of the image data.
In the case of encoding, the image in image memory 22 is passed to a compression / decompression (CODEC) device 24, and the resulting compressed data is stored in output FIFO 26 after being entered. When decoding image data, the compressed data in input FIFO 28 is passed to CODEC 24, and the resulting decompressed data is stored in image memory 22 for display and / or later processing. The local processor 30 includes a local memory 32, an output FIFO 26, an input FIFO 28, and an address FIFO.
Connected between 34. The address FIFO 34 stores the start address of a block of decoded image data in the image memory 22, as described later. Control register 36 has an input connected to the output of input FIFO 28 for receiving and storing the data block header (H). This heading information is stored in CODEC 24
Contains control information.

In this embodiment of the invention, CODEC 24 processes the image data using a compression algorithm based on the discrete cosine transform (DCT). A good DCT method is IS
O / IEC JTC1 / SC2 / WG8 CCITT SGVIII JPEG-8-R5.2 May 1
Integrated Photo Expert Group described in 0, 1990
(JPEG) Based on baseline. This algorithm converts the source image into an 8x8 block image
Process each block after dividing it into data elements. The processing of a block of image data is illustrated in the symmetric logic flow diagram of FIG.

Prior to compression, the image data is level shifted by subtracting 128 (8-bit precision).
The DCT unit processes the block of 64 events and generates coefficients that are quantized according to the weights of the 64 entry look-up table.
The weights are based on the known response of the human eye to various frequencies. The result is a frequency dependent quantization encoded by a lossless entropy encoding scheme. DA Huffman, "AM
ethod for the Construction of Minimum-Redundancy C
odes ", Proc. IRE, Sept. 1952, pp. 1098-1101, Huffman Variable Length Coding (VLC), and WK Pra
tt, Digital Image Processing, John Wiley & Sons, p
Run length coding (RLC), described in p. 632, (1978), is the two most frequently used lossless entropy coding techniques. Both of these techniques (VLC and RLC) are DCT
Take advantage of the fact that streams of coefficients often contain long runs of zeros.

The characteristics of the process pipeline 12 described above provide the following advantages when used in a CODEC-based image processing system.

Because of the inherent synchronization mechanism, as long as the processes in the process pipeline 12 occur in the correct sequence, the user of the image processing system 10 does not need to worry about the exact timing of the data transfer.

The FIFO memory inserted between the processes smoothes the fluctuation of the instantaneous transfer speed of the data flow.

In addition, the self-activated local state machine control makes the inter-boundary processing wait state transparent, thereby freeing the programmer from considering the global time schedule of the process propagating through the process pipeline 12.

Encoding In the case of encoding (compression), the uncompressed source image is first stored in the image memory 22. Local processor 30 first prepares CODEC 24 for the encoding process by loading an internal table and performing other necessary functions. The encoding process is controlled by LSM 18.
Before starting to encode a block of data, LSM 18 reads the input FIFO and recovers the block header (H). Then, the header including the CODEC 24 control information is stored in the control register 36 and applied to the CODEC 24. The local processor 30 then, for each image data block to be processed,
Image memory 22 Source address to address FIFO 3
Load 4

FIG. 4 shows the input of the image data encoding operation.
An example of the contents of FIFO 28 is shown. Obviously, the input FIFO 28 includes a plurality of header blocks, each of which includes CODEC 24 control information for an associated data block of image data stored in the image memory 22. FIG. 6 shows an example of the contents of the address FIFO 34 of FIG. 2 with respect to the contents of the input FIFO 28 of FIG. FIG. 7 shows an example of the contents of the output FIFO of FIG. 2 during and after the image encoding operation.

After providing the header information of the block of image data to the control register 36, the LSM 18 sends a read request to the image memory 22 to read the block of image data into the CODEC 24. CODEC 24 encodes the image data block and outputs the compressed image data to output FI for later retrieval by local processor 30.
Deliver to FO 26. When CODEC 24 finishes compressing a complete block of data, it ends the block (E
OB) signal. LSM 18 responds by repeating the above operation by retrieving the header information of the next image data block from input FIFO 28 in response to any further blocks to be processed. LSM 18 is the address FIFO 34 E
As shown by the state of the MPTY (empty) signal line, the address FIFO
If the source address of the image memory 22 further exists in the block 34, it is determined that there is a block to be further processed.

Decoding Process For decoding (decompression), a block of compressed image data is placed into the input FIFO by the local processor 30. Local processor 30 initiates and maintains any required block control parameters, as will be referred to later with respect to DCTQ 42 and DCT 40 control bits. The decryption process is LSM
Controlled by 18. Before starting to decode a block of data, LSM 18 reads input FIFO 28 and searches for the header of the block of image data to be processed. And
The header containing the CODEC 24 control information is stored in the control register 36 and used for CODEC 24. The local processor 30 also loads the address FIFO 34 with the target address of the image memory 22 for each image data block to be processed.

FIG. 3 shows an example of the contents of the input FIFO 28 relating to the image data decoding operation. Obviously, the block of image data and the end of block (EO
B) Identifiers are scattered. FIG. 5 shows an example of the contents of the address FIFO 34 of FIG. 2 with respect to the input FIFO 28 of FIG.

Following this operation, the LSM 18 re-enters the input FIFO
Read 28 and retrieve the first word of the compressed block of data. CODEC 24 begins decoding the data and sends an indication of the availability of additional data to LSM 18. LSM 18 each
Continue reading input FIFO 28 in response to CODEC 24 availability indication. CODEC 24 identifies the EOB of the block of image data read from input FIFO 28 and indicates this condition to LSM 18 by driving the EOB signal line. LS
M18 is a codec block of decoded image data
Waiting for transfer from 24 to image memory 22, and if there are more blocks to process as indicated by the state of the EMPTY status signal line in address FIFO 34, the above process is repeated.

Section (C) FIG. 8 is a detailed circuit block diagram of a preferred embodiment of the image data encoding and decoding system shown in FIG.

This embodiment of the invention uses the Micro Channel Interface Specification (Micro Channel is IBM
Provided on a plug-in circuit card to interface with a data processing system having a system bus (registered trademark of the company).

In this embodiment, the sizes of local memory 32 and buffer (i) (output FIFO 26 and input FIFO 28) are determined as described in section (A). That is,
Circuit card has three 1K x 18-bit output FIFOs 26, inputs
It comprises a 256 Kbyte static RAM (SRAM) such as a FIFO 28 and an address FIFO 34 and a local memory 32.
The circuit card comprises a local processor 30 and two more COs
A processing device (DCT device 40, DCTQ device 42) constituting the DEC 24 is provided. Devices 40 and 42 meet the Integrated Photo Expert Group (JPEG) baseline specification.

In a preferred embodiment of the present invention, DCT device 40
The DCTQ unit 42 is an L64740 DCT quantization processor, also commercially available from LSI Logic, Inc. The L64730 discrete cosine transform processor is commercially available from SI Logic, Inc. (Mile Petas, Calif.). DCT device 40
Is described in Publication L64730 (July 1990), and the structure and operation of DCTQ device 42 is described in Publication L67440. These publications are also available from LSI Logic. These devices each have a data buffer (buffer 40a or buffer 42a) inside. The external 64-byte conversion buffer 44 stores the DCT device 40 and DCTQ as described below.
Operates with the device 42.

Programmable logic device (PLD), specifically SYSTEM CTRL (system control unit) LSM 18a and XLATE
Control logic is included in the CTRL (Conversion Control) LSM 18b. The circuit card functions to perform image data encoding / decoding specific processing.

The aforementioned JPEG compression / decompression algorithm shown in FIG. 9 can be divided into the following three tasks. (1) Non-adaptive Discrete Cosine Transform (DCT) based on 8 x 8 block image elements (2) Run-length coding (RLC
= Run Length Coding)-Uniform intermediate step quantization of DCT coefficients using weighting function (3) Huffman VLC to reduce entropy of quantized DCT coefficients

[0058] The system shown in FIG. 8, DCT unit 40 performs a non-adaptive DCT function, DCTQ device 42 quantization function and run
Performs long coding (RLC) and the local processor 30
Partition these tasks to perform the decompression function. Thus, according to FIG. 10, the local processor 30
The DCTQ device 42 corresponds to the process (i) of FIG.
(i + 1), and the DCT device 40 performs the process (i +
Corresponds to 2).

In general, the VLC function performed by local processor 30 is a table look-up operation corresponding to the mapping between source symbols and variable length codewords when encoding image data. When decoding image data, VLC is typically accomplished by tracing along the encoding tree until the decoded data is reached. The VLC coder maps the input data into variable length codewords. As mentioned above, for the baseline JPEG method used in the preferred embodiment, the quantized DCT
A Huffman encoder is used to reduce the entropy of the coefficients. That is, the Huffman encoder reduces the average number of bits representing a symbol without destroying the initial identity of the symbol.

In the case of image data decoding, under the control of the host processor 14, compressed image data is locally transmitted from the host bus 46 via the host bus interface 48 and the system bus 50. Moved to SRAM 32. The system bus 50 and the image bus 52 are both housed inside a circuit card and are connected to each other by a system bus gate 54. System resource capture boundary
Synchronization of the data transfer at 14a is achieved by two flags contained in interface 48. Specifically, the ready (RDY) flag 48a indicates that the compressed image
Set by host processor 14 to indicate that a block of data is stored in SRAM 32.
RDY flag 48a is read by local processor 30 to decompress blocks of image data by DCT device 40 and DCT
Used to start by Q device 42.

[0061] The image by the DCT device 40 and the DCTQ device 42
Upon completion of the encoding of the block of data and the VLC performed by the local processor 30, an acknowledgment (ACK) flag 48
b is set by local processor 30 to indicate that a block of compressed image data is stored in SRAM 32. ACK flag 48b is set to host processor 14.
Is used to start the movement of the block of image data read and compressed to the mass storage device 15, for example.

The local processor 30 has an interface function,
Headline dissection of compressed data streams and VLC
Implemented in a digital signal processor (DSP) that performs One suitable embodiment of the DSP processor 30 is a texa
This is a TM320C30 device commercially available from SU Instruments. It should be understood that a digital signal processor is not required to implement the present invention, and that any digital data processor having the desired speed and processing reliability can be used. RISC (Reduced Instructi
On Set) processors are another suitable processor type.

The DSP processor 30 performs the VLC decompression function, and executes run length coding (R) from the data stored in the SRAM 32.
LC) Assemble blocks of image data. The DSP processor 30 also generates a heading (H) for each block. The header includes setup data of the DCT device 40 and the DCTQ device 42 for each block. The resulting data block is transferred by the DSP processor 30 as a 16-bit word to the data input FIFO 28. Each data block is preceded by its corresponding setup header word and ends with an EOB symbol, as shown in FIG. Each block of data is run length coded from 1 word to 64 words
(RLC) May contain image data.

The DSP processor 30 initializes the DCTQ device 42 by first loading the internal quantization table,
And for each block, the image memory of that block
Supply 22 target addresses. The target address is loaded into the address FIFO 34 as shown in FIG. This address is later used to store the decompressed block of image data in image memory 22.

When the system control unit 18a finds that the address FIFO 34 contains at least one address, it starts image data processing. Processor 30
This status is indicated by the EMPTY status flag in address FIFO 34, which is cleared when the first block address is loaded. In response to the FMPTY being de-displayed, the system controller 18a reads the header of the first block from the input FIFO 28 and stores the header information in the register (REG) 36. R
EG 36 outputs control signals applied to DCT device 40 and DCTQ device 42, and sets their internal characteristics for processing subsequent block data output from input FIFO 28.

That is, the DCTQ device 42
Four control bits connected to the DE.0, COMP0, COMP1, and RESPRED input terminals are received from REG 36. These terminals are used to control the following functions described in the aforementioned DCTQ publication L67440. MODE.0 One of two internal quantization tables
Select one COMP0, COMP1 Select DC predictor RESPRED Reset internal DC predictor when high

The DCT device 40 receives one control bit from the REG 36 connected to the INTER input terminal. The signal at this terminal, when high, indicates that the device is to operate in inter-frame mode.

In operation, DCTQ unit 42 generates 64 coefficients per data block. After passing through the DCT device 40, these
It becomes 4-byte decoded image data. Since the DCT device 40 produces a shuffled (but coherent) sequence of output data, the data buffer 44 performs the transformation. This conversion is accomplished by XLATE CTRL 18b which controls the address input of data buffer 44 to store the decoded image data in the desired order. And the reordered 64 byte decoded image
Data is stored in the image memory 22 and the address FI
Start with the address contained in FO 34. This address is applied to the address bus (ADDR) portion of the image bus 52 via the gate (GT) 58, and the memory controller (MEM CTRL) 22
It is latched by the address latch (AL) provided in a. The contents of the AL are automatically incremented to store the entire 64-byte decoded data block. The dynamic RAM controller (DRAM CTRL) 22b controls the access timing of 4 Mbytes of the dynamic RAM constituting the image memory 22.

This process continues until the address FIFO 34 is empty indicating that there are no more blocks of compressed image data to process.

In the case of image data encoding, the source
The image is contained in image memory 22. The source image can be obtained, for example, from a camera that produces digital image data at 30 frames / second.
The DSP processor 30 performs DCT and DCTQ for each block.
Load the heading (FIG. 4) into the input FIFO 28, load the appropriate quantization table into the DCTQ device 42,
And the image memory 22 target address of each block
By loading (FIG. 6) into the address FIFO 34,
Prepare the circuit card for the encoding process .

When the system controller 18a detects the presence of at least one address in the address FIFO 34, it starts the encoding process. The system control unit 18a has an address FIF
Read the first address from O34 and transfer it to the address latch of memory controller 22a. afterwards,
The system controller 18a starts accessing the corresponding block of image data in the image memory 22. Then, the system control unit 18a reads the heading of the first block from the input FIFO 28 and stores the heading in the REG 36. As described above, in order to control the encoding processing parameters of the image block, the stored header information is stored in the DCT device.
40 and DCTQ device 42. Under the control of XLATECTRL 18b, image data is read from image memory 22 into buffer 44, and a block of 64 bytes is delivered to DCT device 40 under control of XLATECTRL 18b. In the encoding mode, buffer 44 provides synchronization of the input data stream to both DCT device 40 and DCTQ device 42. DCT device 40
Is output via the DCTQ device 42,
A stream of compressed 16-bit images "events". The image "event" is stored in output FIFO 26 for later retrieval and VLC by DSP processor 30. DSP
Processor 30 outputs FI to detect when the first byte of compressed image data is stored therein.
It has an input connected to the EMPTY flag of FO 26.

As long as the address exists in the address FIFO 34, the system control unit 18a reads the block address from the address FIFO 34 and the block header from the input FIFO 28, and reads the encoded image block.
Control of writing data to output FIFO 26 continues.

Obviously, the system controller 18a is triggered by the presence of the start address which starts the local processing of the image data, and as long as the image data block address exists in the address FIFO 34, the processing of the image data is performed. Keep controlling. Therefore, the address FIFO EMPT
Use of the Y status flag is a fixed processing cycle stage (DCTQ
The operation of the device 42 and the DCT device 40) is separated from the variable processing cycle stages implemented by the processor 30. Similarly,
Use of the EMPTY status flag in output FIFO 26 decouples processor 30 from the fixed cycle DCT and DCTQ stages. In addition, all images and
Data processing (encoding and decoding) is performed by the host processor 14.
Is executed asynchronously and in parallel with other processing tasks that can be executed by

Although emphasis has been placed on processing a single block of data, it should be understood that during normal operation several blocks will generally be moving through the processing pipeline. For example, the compressed image data is stored in the SRAM 32 after being decompressed, and the processor 30 is performing the VLC operation on the data block in preparation for the decompression and has already been VLCed by the processor 30. At least one data block is enqueued in the input FIFO 28 and is controlled by the CO under control of the header information associated with the current data block.
DEC 24 is processing the current data block.

The correspondence between the preferred embodiment of the present invention of FIG. 9 and the new process pipeline architecture of FIG. 1 is shown in FIG. Obviously, synchronization of the data flow at the system resource acquisition boundary 14a is achieved by the processor 30 in relation to the RDY flag 48a and the ACK flag 48b, respectively. The SRAM 32 functions as the input and output buffer (i-1) in FIG. Stage (i) is processor 30,
System control unit LSM 18a, input FIFO 28 and output FIFO 26
Consists of The processing that takes place in stage (i) (VLC) is data dependent and requires a variable number of processing cycles. Synchronization of the data flow between stage (i) and stage (i + 1) is achieved by the system controller LSM 18a in relation to the EMPTY flag of the address FIFO 34 (not shown in FIG. 10). Stage (i + 1) and stage (i + 2) are DCTQ device 42 and DC
It consists of a T device 40. These stages operate independently of the data and have a fixed number of processing cycles per data block. The buffer (i + 1) in FIG.
This is realized by the internal buffers 40a and 42a of the device 40. 6
To (from) 4-byte buffer 44 and image memory 22
Is controlled by the conversion control unit LSM 18b.

Although described above in connection with a system for compressing and decompressing blocks of image data in real time, the process pipeline architecture of FIG. 1 is used for data processing applications other than image processing. be able to. Some modifications may be made to the preferred embodiment of the present invention shown in FIG. For example, the operation of the image processing system can be enhanced by the addition of a block 64 that performs motion compensation according to the MPEG (Movie Expert) standard. With this improvement, two frames of image data stored in image memory 22 are retrieved and processed by DCT device 40. Other functions can be provided by connecting the video interface 62 to the image bus 52 and moving image data in and out through the video interface 62. These other functions include image scaling, color conversion, filtering, rotation, reconstruction and depth queuing (d
epth cueing). Similarly, the image
The DCT device 40 and DCTQ device 42 can be replaced by other image processing circuits that perform functions other than data encoding and decoding.

[Brief description of the drawings]

FIG. 1 is a simplified circuit block diagram illustrating portions of an image processing system, particularly a multi-stage sequential process pipeline constructed and operated in accordance with the present invention.

FIG. 2 is a simplified circuit block diagram illustrating an embodiment of an image data encoding and decoding system constructed in accordance with the process pipeline architecture of FIG.

FIG. 3 is a diagram showing contents of an example of image data decoding operation of the input FIFO of FIG. 2;

FIG. 4 is a diagram showing the content of an example of image data encoding operation of the input FIFO of FIG. 2;

5 is a diagram showing the contents of an example of image data decoding operation in FIG. 3 of the address FIFO in FIG. 2;

6 is a diagram showing the contents of an example of the image data encoding operation of FIG. 4 of the address FIFO of FIG. 2;

7 is a diagram showing contents of an example of image data encoding operation of FIG. 4 of the output FIFO of FIG. 2;

FIG. 8 is a detailed circuit block diagram of a preferred embodiment of the image data encoding and decoding system shown in FIG.

Figure 9: JPEG DCT (Integrated Photo Expert Group,
3 is a logic flow diagram illustrating an image data encoding model based on a discrete cosine transform).

FIG. 10 is a block diagram showing the correspondence between the process pipeline architecture of FIG. 1 and the main functions of FIG. 8 in more detail.

[Explanation of symbols]

 10 Image Processing System 12 Sequential Process Pipeline 14 Host Processor 14a System Resource Capture Boundary (SRIB) 15 Mass Storage 16a Buffer 16b Buffer 18 Local State Machine (LSM) 18a System Controller 18b Conversion Controller 20 Process Block 22 Image memory 22a Memory controller 24 Compression / decompression (CODEC) device 26 Output FIFO 28 Input FIFO 30 Local processor / DSP processor 32 Local memory / SRAM 34 Address FIFO 36 Control register (REG) 40 DCT device 40a Data buffer 42 DCTQ device 42a Data buffer 44 64-byte conversion buffer 46 Host bus 48 Host bus interface 48a Enable (RDY) flag 48b Acknowledgment (ACK) flag 50 System bus 52 Image bus 54 System bus gate 62 Video interface 64 Motion compensation block

Continued on the front page (72) Inventor Thomas Akos Hovath United States 12582, Stormville, NY, Judith Drive 55 2218 (72) Inventor Andy Gen-Chen Lean United States 11566, Kenneth Road, Merrick, New York 2062 (72) Inventor Thomas McCarthy United States 10566, Peakskill, New York Crestview Avenue 29 (56) Reference Document JP-A-62-144283 (JP, A) JP-A-2-188879 (JP, A) JP-A-61-131122 (JP, A) JP-A-63-137375 (JP, A) 203283 (JP, A) JP-A-62-138973 (JP, A)

Claims (2)

(57) [Claims] 1. A local memory for storing compressed image data received from a host or output to the host, and a predetermined signal connected to the local memory for a block of compressed image data. A processor for performing processing; an image memory for storing uncompressed image data; a CODEC connected between the processor and the image memory for performing compression and decompression of image data; And buffer means connected between the CODEC and the CODEC, the buffer means receiving CODEC control information from the processor during compression, and a block of the CODEC control information and signal-processed image data upon decompression. From the processor, and Input buffer means for supplying to the CODEC, output buffer means for receiving a block of image data compressed at the time of compression from the CODEC and supplying it to the processor, and an uncompressed image stored in the image memory at the time of compression Address buffer means for receiving a head address of a block of data from the processor, and receiving a head address of a block of decompressed image data to be stored in the image memory upon decompression from the processor; Processing system. 2. The codec comprises: first processing means for performing a discrete cosine transform quantization process; second processing means for performing a discrete cosine transform; and between the first processing means and the second processing means. A second buffer connected between the second processing unit and the image memory; and controlling data transfer via the first buffer and the second buffer. The image processing system according to claim 1, further comprising: