CN103095687B - Metadata processing method and device - Google Patents

Abstract

The present invention provides a metadata processing method and device, wherein the metadata processing method includes receiving a write request, the write request carries an object identifier; according to the write request, at least one modification instruction corresponding to the object identifier is generated, and the modification instruction includes a data identifier And the target value corresponding to the data identifier, the data identifier is the identifier of the metadata that needs to be modified in at least one metadata corresponding to the object identifier; the modification instruction is sent to the client and the data server corresponding to the data identifier for the client The terminal and the data server respectively modify the metadata corresponding to the data identifier to the target value according to the modification instruction. In this way, the metadata stored in the client and the data server can be updated synchronously, ensuring the consistency of data storage in the distributed system, and avoiding write conflicts due to multiple clients sending write requests at the same time , which effectively guarantees the performance of the distributed system to provide metadata services.

Description

Metadata processing method and device

technical field

Embodiments of the present invention relate to network technology and computer technology, and in particular to a metadata processing method and device.

Background technique

There are mainly two ways for distributed systems to provide metadata services, namely central metadata services and peer-to-peer metadata services.

Centralized metadata services, including metadata clusters and data clusters. Wherein, the metadata cluster is composed of multiple metadata servers respectively used for storing metadata, and the data cluster is composed of multiple data servers respectively used for storing data. In the centralized metadata service, the metadata cluster usually uses one of the metadata servers as the central node to communicate with the client and provide external services. After the metadata is modified, the central node updates the metadata in other metadata servers in the metadata cluster.

However, when the distributed system adopts the centralized metadata service method, each client’s request to modify or access the metadata needs to be routed to the central node of the metadata cluster, so that the central node not only needs to provide external Metadata modification and access services also need to synchronize metadata changes to other metadata servers in the metadata cluster. Excessive reliance on a single node will cause the performance of metadata services to be limited by the performance of the central node.

When the distributed system adopts the method of peer-to-peer metadata service, the metadata and data are stored in the data cluster composed of multiple data servers. There is no central node in the data cluster, and the client can directly modify or access the data stored in each The metadata in the data server overcomes the problem that the performance of the metadata service is limited by the performance of the central node.

However, since multiple clients can perform read or write operations on each data server, when multiple clients write concurrently, write conflicts will occur. In such a case, a large amount of retry information will be sent between the data server and the client, resulting in low performance of the data server in providing metadata services during concurrent writing.

Contents of the invention

A first aspect of the present invention is to provide a metadata processing method comprising:

receiving a write request, where the write request carries an object identifier;

According to the write request, at least one modification instruction corresponding to the object identifier is generated, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is at least one corresponding to the object identifier The identifier of the metadata that needs to be modified in a metadata;

sending the modification instruction to the client and the data server corresponding to the data identifier, so that the client and the data server can modify the metadata corresponding to the data identifier to the target value.

With reference to the metadata processing method of the first aspect, in a first possible implementation manner, after generating at least one modification instruction corresponding to the object identifier according to the write request, the method further includes:

sending the at least one modification instruction to a data server for storing the modification instruction;

Correspondingly, the method also includes:

After sending the at least one modification instruction to the client and the data server corresponding to the data identifier, send a delete message to the data server for storing the modification instruction to indicate that the data for storing the modification instruction The server deletes the at least one modification instruction.

With reference to the first possible implementation of the first aspect, in a second possible implementation, the method further includes:

Obtaining undeleted modification instructions from the data server for storing the modification instructions;

Send the undeleted modification instruction to the client and the data server corresponding to the data identifier in the undeleted modification instruction.

With reference to the first aspect, the first or the second possible implementation manner, in a third possible implementation manner, the object identifier is a file identifier or a virtual disk identifier.

The second aspect of the present invention is to provide a metadata processing method, including:

receiving a read request, where the read request carries an object identifier;

If no metadata corresponding to the object identifier is found in the stored correspondence between the object identifier and the metadata, at least one access instruction corresponding to the object identifier is generated according to the read request, and the The access instruction includes a data identifier of metadata corresponding to the object identifier;

Send the access instruction to the data server corresponding to the data identifier;

receiving target metadata returned by the data server according to the data identifier.

In combination with the metadata processing method of the second aspect, in a first possible implementation manner, the read request also carries a client identifier;

Correspondingly, after receiving the target metadata returned by the data server according to the data identifier, the method further includes:

Sending the target metadata to a client corresponding to the client identifier, so that the client saves the target metadata as metadata corresponding to the object identifier.

With reference to the second aspect or the first possible implementation manner, in the second possible implementation manner, the object identifier is a file identifier or a virtual disk identifier.

A third aspect of the present invention is to provide a client, including:

a receiving unit, configured to receive a write request, where the write request carries an object identifier;

A processing unit, configured to generate at least one modification instruction corresponding to the object identifier according to the write request, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is the same as the object identifier An identifier of metadata that needs to be modified in at least one piece of metadata corresponding to the object identifier;

A sending unit, configured to send the modification instruction to the client and the data server corresponding to the data identifier, so that the client and the data server can respectively send the element corresponding to the data identifier according to the modification instruction. The data is modified to the stated target value.

In combination with the client provided in the third aspect, in a first possible implementation, the processing unit is further configured to:

After generating at least one modification instruction corresponding to the object identifier according to the write request, sending the at least one modification instruction to a data server for storing the modification instruction;

Correspondingly, the sending unit is also used for:

After sending the at least one modification instruction to the client and the data server corresponding to the data identifier, send a delete message to the data server for storing the modification instruction to indicate that the data for storing the modification instruction The server deletes the at least one modification instruction.

With reference to the first possible implementation of the third aspect, in a second possible implementation, the client further includes:

an acquisition unit, configured to acquire undeleted modification instructions from the data server for storing modification instructions;

Correspondingly, the sending unit is also used for:

Send the undeleted modification instruction to the client and the data server corresponding to the data identifier in the undeleted modification instruction.

With reference to the third aspect, the first or the second possible implementation manner, in the third possible implementation manner, the object identifier in the write request received by the receiving unit is a file identifier or a virtual Disk ID.

A fourth aspect of the present invention is to provide a client, including:

a receiving unit, configured to receive a read request, where the read request carries an object identifier;

The processing unit generates at least one access instruction corresponding to the object identifier according to the read request when no metadata corresponding to the object identifier is found in the stored correspondence between the object identifier and the metadata , the access instruction includes a data identifier of metadata corresponding to the object identifier;

a sending unit, configured to send the access instruction to a data server corresponding to the data identifier;

Correspondingly, the receiving unit is further configured to receive target metadata returned by the data server according to the data identifier.

With reference to the client provided in the fourth aspect, in a first possible implementation manner, the read request received by the receiving unit also carries a client identifier;

Correspondingly, the sending unit is also used for:

After receiving the target metadata returned by the data server according to the data identifier, sending the target metadata to a client corresponding to the client identifier, so that the client can save the target metadata is the metadata corresponding to the object ID.

With reference to the fourth aspect or the first possible implementation manner, in a second possible implementation manner, the object identifier in the read request received by the receiving unit is a file identifier or a virtual disk identifier.

The metadata processing method and device provided by the embodiments of the present invention generate at least one modification instruction corresponding to the object identifier according to the write request, and the modification instruction includes the data identifier of the metadata to be modified and the target for modifying the metadata value, each modification command is sent to the client and data server that store the corresponding metadata, so that the metadata stored in the client and the data server can be updated synchronously, ensuring that the data in the distributed system Consistency of storage. In addition, since the write requests of each client need to be sent to the data server in the data cluster through the client as the master client, avoiding the situation of write conflicts caused by multiple clients sending write requests at the same time , which effectively guarantees the performance of the distributed system to provide metadata services.

Description of drawings

FIG. 1 is a flowchart of a metadata processing method provided by an embodiment of the present invention;

FIG. 2 is a flowchart of another metadata processing method provided by an embodiment of the present invention;

FIG. 3 is a flowchart of another metadata processing method provided by an embodiment of the present invention;

FIG. 4 is a flowchart of another metadata processing method provided by an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a client provided by an embodiment of the present invention;

Fig. 6 is a schematic structural diagram of another client provided by an embodiment of the present invention

FIG. 7 is a schematic structural diagram of another client provided by an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of another client provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of another client provided by an embodiment of the present invention.

Detailed ways

The distributed system provided by various embodiments of the present invention includes a central cluster, a data cluster, and at least one client.

Wherein, the central cluster is composed of one or more central nodes, and the central nodes may be servers. The central node is not used to store metadata, but is used to select one client as the master client when there are multiple clients. When there are multiple central nodes, one of the central nodes is the main control central node, and the other central nodes are the backup central nodes of the main control central node, so that when the main control central node fails, the backup central node can complete the main control node. The work of the control center node. The central node referred to below refers to the master control center node.

A data cluster is composed of one or more data servers, and metadata and/or data of objects are stored in each data server.

The client is a device for users to operate on the objects stored in the data cluster. The client can be a personal computer or a server, and the corresponding system can be preset in the client so that the user can operate on the client. Objects are manipulated and controlled.

When there are multiple clients in the distributed system, the central cluster sets one of the multiple clients as the master client. When each client needs to communicate with the data cluster, it needs to pass through the master client. end, the master client communicates with the data server in the data cluster.

Specifically, the method for the user to perform read and write operations on objects in the distributed system through the client is as follows.

Fig. 1 is a flowchart of a metadata processing method provided by an embodiment of the present invention. As shown in Fig. 1, the method includes:

101. Receive a write request. Wherein, the write request carries an object identifier.

Specifically, when a user operates a distributed system through a client, it includes the following two situations. One case is that the client used by the user is the master client, and the write request received by the master client is sent by the user; the other case is that the client used by the user is not the master client, then After receiving the write request sent by the user, the client needs to send the write request to the master client, and the write request received by the master client is sent by other clients.

Regardless of whether the write request received by the master client is sent by the user or sent by other clients, the write request carries an object identifier. Objects are distributed in one or more data servers in the form of metadata and data in the data cluster. The object identifier is the identifier of the object on the user-facing side.

102. Generate at least one modification instruction corresponding to the object identifier according to the write request. Wherein, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is an identifier of metadata that needs to be modified in at least one piece of metadata corresponding to the object identifier.

Specifically, the corresponding relationship between object identifiers and metadata data identifiers is pre-stored in the master control client, and one object identifier may correspond to one or more metadata identifiers. Therefore, after receiving the write request, the master client can find the data identifier of the metadata corresponding to the object identifier according to the object identifier in the write request. Moreover, according to the received write request, the master client can know which metadata in the object needs to be modified and the modified target value, so that the master client can generate the corresponding object ID according to the write request. One or more modification instructions.

In each generated modification instruction, not only the data identifier of the metadata to be modified is included, but also the target value for modifying the metadata is required. Therefore, the data server that receives the modification instruction can modify the corresponding metadata to the target value according to the modification instruction.

103. Send the modification instruction to the client and the data server corresponding to the data identifier. Step 103 is performed for the client and the data server to modify the metadata corresponding to the data identifier to the target value according to the modification instruction.

Specifically, the master client not only stores the correspondence between the object identifier and the data identifier of the metadata, but also stores the correspondence between the data identifier of the metadata and the client, and the correspondence between the data identifier of the metadata and the data server . That is to say, the master client can find out which metadata is stored in which client and which data server according to the data identifier.

Correspondingly, the master control client sends one or more modification instructions to the corresponding client and data server respectively. Therefore, not only the data server can modify the corresponding metadata to the target value according to the modification instruction, but the client that stores the metadata can also modify the metadata to the target value accordingly, ensuring that the data stored in the client and the data server This metadata is consistent.

The metadata processing method provided by the embodiment of the present invention generates at least one modification instruction corresponding to the object identifier according to the write request, and the modification instruction includes the data identifier of the metadata to be modified and the target value for modifying the metadata, Send each modification instruction to the client and data server that store the corresponding metadata, so that the metadata stored in the client and the data server can be updated synchronously, ensuring the accuracy of data storage in the distributed system. Consistency. In addition, since the write requests of each client need to be sent to the data server in the data cluster through the client as the master client, it avoids the situation of write conflicts caused by multiple clients sending write requests at the same time, effectively It guarantees the performance of the distributed system to provide metadata services.

Fig. 2 is a flowchart of another metadata processing method provided by an embodiment of the present invention. As shown in Fig. 2, the method includes:

201. Receive a write request.

Specifically, reference may be made to the implementation manner described in step 101.

Further, the object identifier is a file identifier or a virtual disk identifier.

The objects stored in the data cluster can be files or virtual disks. Correspondingly, when the object is a file, the object identifier is the file identifier, and the metadata is the metadata of the file; when the object is a virtual disk, the object identifier is the virtual disk identifier, and the metadata is the metadata of the virtual disk.

It should be noted that files and virtual disks are only two object forms stored in the data cluster, and the optional object forms are not limited thereto. Metadata of other forms of objects that can be stored in the distributed system can also be processed by using the metadata processing methods and clients described in the various embodiments of the present invention.

202. Generate at least one modification instruction corresponding to the object identifier according to the write request.

Specifically, reference may be made to the implementation manner described in step 102 .

203. Send the at least one modification instruction to a data server for storing the modification instruction.

Specifically, after generating at least one modification instruction, the master client needs to send these modification instructions to the data server for storing the modification instruction.

The manner in which the data server stores the modification instruction may be similar to the manner in which other data is stored, that is, the modification instruction is stored in the data server as a part of special data.

Further, at least one modification instruction generated by the master client may be a transaction log.

Preferably, it may be a redo log in the transaction log.

It should be noted that the form of the optional modification instruction is not limited to this.

204. Send the at least one modification instruction to the client and the data server corresponding to the data identifier respectively.

Specifically, reference may be made to the implementation manner described in step 103 .

205. Send a delete message to the data server for storing the modification instruction. Step 205 is executed to instruct the data server for storing the modification instruction to delete the at least one modification instruction.

Specifically, after sending each modification instruction to the corresponding client and data server respectively, the master control client sends a delete message to the data server for storing the modification instruction. After receiving the deletion message, the data server for storing the modification instruction deletes the corresponding modification instruction.

Since the modification instruction corresponds to the object identifier, when there are multiple modification instructions corresponding to the object identifier, the plurality of modification instructions can be regarded as a group of modification instructions. The data server for storing modification instructions treats each set of modification instructions as a whole.

After the main control client sends each modification instruction in each group of modification instructions to the corresponding client and data server in turn, it sends a delete message to the data server for storing the modification instructions to instruct the data server to delete the group Modify instruction to delete.

The main control client stores at least one modification instruction in the data server in order to improve the performance of the metadata service.

If the master client fails, the central node will reselect a client from other clients as the new master client.

The new master control client needs to obtain the currently undeleted modification instructions from the data server used to store the modification instructions. Since when a group of modification instructions are all executed, the data server will delete this group of modification instructions according to the instruction of the master client, therefore, if there is an undeleted modification instruction in the data server, it means that it has not been deleted One or more of the modification instructions for is not yet executed.

After the new master control client obtains the undeleted modification instructions from the data server for storing the modification instructions, it sends the undeleted modification instructions to the The corresponding client and data server of the data identifier. Since the modification instructions that have not been deleted include data identifiers and target values, the new master client sends each modification instruction to the client and data server corresponding to the data identifiers in the modification instructions in turn, so that the modified The instruction client and data server modify the metadata corresponding to the data identifier to the target value.

Similarly, after the new master client sends the obtained undeleted modification instruction to the corresponding client and data server, it sends a delete message to the data server for storing the modification instruction, to indicate that the data server will Executed modification commands are deleted.

The master control client also stores the generated modification instructions in the data server of the data cluster, and after the master control client executes each modification instruction, it instructs the data server storing the modification instructions to delete the corresponding modification instructions. This makes it possible for a new master client to obtain the set of modification instructions even if the master client fails before all the modification instructions are executed. By re-executing the group of modification instructions by the new master client, all the modifications to the object indicated by the write request can be executed, and the modification to the metadata is guaranteed to be maintained in the client and the data server unanimous.

Further, when the master client fails, after the central node selects a new master client from other clients, it also sends a message to other clients except the new master client, to These clients are instructed to send their client IDs and the corresponding relationship between their stored object IDs and data IDs to the new master client. Therefore, the new master client can know the corresponding relationship between the object identifier and the data identifier stored in each client. For the new master client to find the object identifier according to the data identifier, and then find the client corresponding to the data identifier.

Fig. 3 is a flowchart of another metadata processing method provided by an embodiment of the present invention. As shown in Fig. 3, the method includes:

301. Receive a read request. Wherein, the read request carries an object identifier.

Specifically, when the client used by the user is a client other than the master client, when the client receives the read request sent by the user, first, according to the object identifier carried in the read request, the The corresponding relationship between the stored object identifier and metadata is searched, and if at least one metadata corresponding to the object identifier is found, it indicates that the at least one metadata is stored in the client. Correspondingly, the client only needs to return the found metadata to the user.

When the client used by the user is the master client, the master client searches the corresponding relationship between the corresponding identifier and metadata stored in it according to the object identifier in the received read request. The object identifier corresponds to at least one piece of metadata, which means that the at least one piece of metadata is stored in the master client. Correspondingly, the master control client returns the found metadata to the user.

When the client used by the user is a client other than the client, and the metadata corresponding to the object identifier is not found on it, the read request is sent to the master client.

After the master client receives the read request, if it can find the metadata corresponding to the object identifier, it will return the metadata to the client. Correspondingly, after receiving the metadata, the client stores the metadata as metadata corresponding to the object identifier.

However, after receiving the read request, if the master client cannot find the metadata corresponding to the object identifier, it needs to read the metadata from the data cluster.

302. When no metadata corresponding to the object identifier is found in the stored correspondence between the object identifier and the metadata, generate at least one access instruction corresponding to the object identifier according to the read request. Wherein, the access instruction includes a data identifier of metadata corresponding to the object identifier.

Specifically, the corresponding relationship between the object identifier and the data identifier of the metadata is pre-stored in the master client. If the master client does not find the metadata corresponding to the object identifier in the read request, then generate one or more access instructions corresponding to the object identifier according to the read request.

Since the master client can know which metadata in the object needs to be accessed according to the received read request, the master client can generate one or more access instructions according to the read request. Each access instruction includes the data identification of the metadata to be accessed. The data identifier of the metadata to be accessed corresponds to the object identifier in the read request.

303. Send the access instruction to the data server corresponding to the data identifier.

Specifically, since the corresponding relationship between the data identifier and the data server is stored in the master control client. Therefore, after generating the access instruction, the master client sends the access instruction to the data server corresponding to the data identifier according to the data identifier in the access instruction.

If there are multiple access instructions generated by the master client, each access instruction is sent to the data server corresponding to each data identifier in turn.

304. Receive target metadata returned by the data server according to the data identifier.

Specifically, after the master control client sends the access instruction to the corresponding data server, it can wait for the data server to return the corresponding target metadata. The target metadata received by the master client is the metadata corresponding to the data identifier in the access instruction.

In the metadata processing method provided by the embodiment of the present invention, when no metadata corresponding to the object identifier in the read request is stored in the master client, one or more access instructions corresponding to the object identifier are generated, and each The access instruction is sent to the data server corresponding to the data identifier of the metadata, so as to receive the target metadata corresponding to the data identifier from the data server. Therefore, when the corresponding metadata is not stored in the client or the master client, it is necessary to obtain the corresponding metadata by accessing the data server, and when the corresponding metadata is stored on the client and the master client, then It can be directly or indirectly returned to the user, effectively improving the performance and efficiency of the metadata service.

Fig. 4 is a flowchart of another metadata processing method provided by an embodiment of the present invention. As shown in Fig. 4, the method includes:

401. Receive a read request. Wherein, the read request carries an object identifier and a client identifier.

Specifically, reference may be made to the implementation manner described in step 301.

Further, the object identifier is a file identifier or a virtual disk identifier.

The objects stored in the data cluster can be files or virtual disks. Correspondingly, when the object is a file, the object identifier is the file identifier, and the metadata is the metadata of the file; when the object is a virtual disk, the object identifier is the virtual disk identifier, and the metadata is the metadata of the virtual disk.

It should be noted that files and virtual disks are only two object forms stored in the data cluster, and the optional object forms are not limited thereto. Metadata of other forms of objects that can be stored in the distributed system can also be processed by using the metadata processing methods and clients described in the various embodiments of the present invention.

Further, when the read request received by the master client comes from other clients, the read request may also carry the client identifier. The master client sends the metadata found on it, or the target metadata received from the data server, to the corresponding client identified by the client. Moreover, the master client will also store the corresponding relationship between the client ID and the object ID, so that when performing a write operation, it not only sends the modification instruction to the data server corresponding to the data ID, but also relationship, and send the modification command to the client corresponding to the data identifier.

402. When no metadata corresponding to the object identifier is found in the stored correspondence between object identifiers and metadata, generate at least one access instruction corresponding to the object identifier according to the read request.

Specifically, reference may be made to the implementation manner described in step 302 .

403. Send the access instruction to the data server corresponding to the data identifier.

Specifically, reference may be made to the implementation manner described in step 303 .

404. Receive target metadata returned by the data server according to the data identifier.

Specifically, reference may be made to the implementation manner described in step 304 .

405. Send the target metadata to a client corresponding to the client identifier, so that the client saves the target metadata as metadata corresponding to the object identifier.

Specifically, after receiving the target metadata from the data server, the master client sends the target metadata to the client corresponding to the client identifier. The client receiving the target metadata saves the target metadata as metadata corresponding to the object identifier carried in the read request. Therefore, when the client receives the read request carrying the object identifier again, it can find the metadata from the stored correspondence between the object identifier and the metadata, and then directly return the metadata to the user , without sending the read request to the master client, effectively improving the performance and efficiency of the metadata service.

In each of the above embodiments, a heartbeat mechanism is set between the main control client and the central node. If the central node can always detect the heartbeat of the master client, it can determine that the master client is in a normal working state; if the central node cannot detect the heartbeat of the master client within the preset If the control client fails, then re-select a new master client from other clients. In addition, the master client can also request the central node to reselect a new master client from other clients by directly sending a request message to the central node.

Further, other clients except the master client can also keep heartbeat with the central node. Similarly, the client whose heartbeat can be detected by the central node is a client in normal working state; the client whose heartbeat cannot be detected by the central node may be in a faulty working state. When the central node reselects the master client from the clients, it preferably selects from the clients whose heartbeat can be detected.

Furthermore, when other clients other than the master client recover from a faulty working state to a normal working state, they not only need to establish a heartbeat with the central node, but also need to identify their client IDs and their stored object IDs. The corresponding relationship with the data identifier is sent to the master client, so that the master client can update the relevant information of the client.

FIG. 5 is a schematic structural diagram of a client provided by an embodiment of the present invention. As shown in FIG. 5 , the client includes a receiving unit 11 , a processing unit 12 and a sending unit 13 .

The receiving unit 11 is configured to receive a write request, where the write request carries an object identifier;

The processing unit 12 is configured to generate at least one modification instruction corresponding to the object identifier according to the write request, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is corresponding to the object identifier The identifier of the metadata that needs to be modified in at least one metadata corresponding to the object identifier;

The sending unit 13 is configured to send the modification instruction to the client and the data server corresponding to the data identifier, so that the client and the data server can respectively send the modification instruction to the client corresponding to the data identifier. Metadata is modified to the target value.

Further, the processing unit 12 is also used for:

After generating at least one modification instruction corresponding to the object identifier according to the write request, sending the at least one modification instruction to a data server for storing the modification instruction;

Correspondingly, the sending unit 13 is also used for:

After sending the at least one modification instruction to the client and the data server corresponding to the data identifier, send a delete message to the data server for storing the modification instruction to indicate that the data for storing the modification instruction The server deletes the at least one modification instruction.

FIG. 6 is a schematic structural diagram of another client provided by an embodiment of the present invention. As shown in FIG. 6 , the client may further include an acquisition unit 14 .

An acquisition unit 14, configured to acquire undeleted modification instructions from the data server for storing modification instructions;

Correspondingly, the sending unit 13 is also used for:

Send the undeleted modification instruction to the client and the data server corresponding to the data identifier in the undeleted modification instruction.

Further, the object identifier in the write request received by the receiving unit 11 is a file identifier or a virtual disk identifier.

Specifically, for the method for processing metadata by the client provided in the embodiment of the present invention, reference may be made to the operation steps described in the corresponding method embodiment above, which will not be repeated here.

The client provided by the embodiment of the present invention generates at least one modification instruction corresponding to the object identifier according to the write request, and the modification instruction includes the data identifier of the metadata to be modified and the target value for modifying the metadata, and converts each The modification instructions are sent to the client and the data server that store the corresponding metadata, so that the metadata stored in the client and the data server can be updated synchronously, ensuring the consistency of data storage in the distributed system , in addition, since the write requests of each client need to be sent to the data server in the data cluster through the client as the master client, avoiding the situation of writing conflicts caused by multiple clients sending write requests at the same time, effectively ensuring It improves the performance of the distributed system to provide metadata services.

FIG. 7 is a schematic structural diagram of another client provided by an embodiment of the present invention. As shown in FIG. 7 , the client includes a receiving unit 21 , a processing unit 22 and a sending unit 23 .

The receiving unit 21 is configured to receive a read request, wherein the read request carries an object identifier;

The processing unit 22, when the metadata corresponding to the object identifier is not found in the stored correspondence between the object identifier and the metadata, according to the read request, generate at least one access item corresponding to the object identifier an instruction, the access instruction including a data identifier of metadata corresponding to the object identifier;

A sending unit 23, configured to send the access instruction to a data server corresponding to the data identifier;

Correspondingly, the receiving unit 21 is further configured to receive target metadata returned by the data server according to the data identifier.

Further, the read request received by the receiving unit 21 also carries a client identifier;

Correspondingly, the sending unit 23 is also used for:

After receiving the target metadata returned by the data server according to the data identifier, sending the target metadata to a client corresponding to the client identifier, so that the client can save the target metadata is the metadata corresponding to the object ID.

Further, the object identifier in the read request received by the receiving unit is a file identifier or a virtual disk identifier.

Specifically, for the method for processing metadata by the client provided in the embodiment of the present invention, reference may be made to the operation steps described in the corresponding method embodiment above, which will not be repeated here.

The client provided by the embodiment of the present invention, when the metadata corresponding to the object identifier in the read request is not stored, generates one or more access instructions corresponding to the object identifier, and sends each access instruction to the metadata server. A data server corresponding to the data identifier is used to receive target metadata corresponding to the data identifier from the data server. Therefore, when the corresponding metadata is not stored in the client or the master client, it is necessary to obtain the corresponding metadata by accessing the data server, and when the corresponding metadata is stored on the client and the master client, then It can be directly or indirectly returned to the user, effectively improving the performance and efficiency of the metadata service.

FIG. 8 is a schematic structural diagram of another client provided by an embodiment of the present invention. As shown in FIG. 8, the client includes:

Processor 31 , memory 32 , bus 33 and communication interface 34 . The processor 31 , the memory 32 and the communication interface 34 are connected through the bus 33 to complete mutual communication.

The processor 31 may be a single-core or multi-core central processing unit (Central Processing Unit, CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or be configured to implement one or more integrated circuits of the embodiments of the present invention. circuit.

The memory 32 can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory.

The memory 32 is used to store the program 321 . Specifically, the program 321 may include program codes, where the program codes include computer operation instructions.

A communication interface 34, configured to receive a write request, where the write request carries an object identifier;

Processor 31 runs program 321 to perform:

According to the write request, at least one modification instruction corresponding to the object identifier is generated, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is at least one corresponding to the object identifier The identifier of the metadata that needs to be modified in a metadata;

sending the modification instruction to the client and the data server corresponding to the data identifier, so that the client and the data server can modify the metadata corresponding to the data identifier to the target value.

Specifically, for the method for processing metadata by the client provided in the embodiment of the present invention, reference may be made to the operation steps described in the corresponding method embodiment above, which will not be repeated here.

FIG. 9 is a schematic structural diagram of another client provided by an embodiment of the present invention. As shown in FIG. 9, the

Clients include:

Processor 41 , memory 42 , bus 43 and communication interface 44 . The processor 41 , the memory 42 and the communication interface 44 are connected through the bus 43 to complete mutual communication.

The processor 41 may be a single-core or multi-core central processing unit (Central Processing Unit, CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or be configured to implement one or more integrated circuits of the embodiments of the present invention. circuit.

The memory 42 can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory.

The memory 42 is used to store the program 421 . Specifically, the program 421 may include program codes, where the program codes include computer operation instructions.

A communication interface 44, configured to receive a read request, wherein the read request carries an object identifier;

Processor 41 runs program 421 to perform:

If no metadata corresponding to the object identifier is found in the stored correspondence between the object identifier and the metadata, at least one access instruction corresponding to the object identifier is generated according to the read request, and the The access instruction includes a data identifier of metadata corresponding to the object identifier;

Send the access instruction to the data server corresponding to the data identifier;

receiving target metadata returned by the data server according to the data identifier.

Specifically, for the method for processing metadata by the client provided in the embodiment of the present invention, reference may be made to the operation steps described in the corresponding method embodiment above, which will not be repeated here.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (10)

1. A metadata processing method applied to a distributed system, the distributed system comprising a data server and at least two clients, characterized in that the at least two clients comprise a master client, the master The control client is the client that needs to pass through when the at least two clients communicate with the data server; the method is executed by the master control client, and the method includes: receiving a write request, where the write request carries an object identifier; According to the write request, at least one modification instruction corresponding to the object identifier is generated, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is at least one corresponding to the object identifier The identifier of the metadata that needs to be modified in a metadata; sending the modification instruction to the client and the data server corresponding to the data identifier, so that the client and the data server can modify the metadata corresponding to the data identifier to the target value; Wherein, the object identifier is a file identifier or a virtual disk identifier. 2. The metadata processing method according to claim 1, wherein after generating at least one modification instruction corresponding to the object identifier according to the write request, the method further comprises: sending the at least one modification instruction to a data server for storing the modification instruction; Correspondingly, the method also includes: After sending the at least one modification instruction to the client and the data server corresponding to the data identifier, send a delete message to the data server for storing the modification instruction to indicate that the data for storing the modification instruction The server deletes the at least one modification instruction. 3. The metadata processing method according to claim 2, wherein the method further comprises: Obtaining undeleted modification instructions from the data server for storing the modification instructions; Send the undeleted modification instruction to the client and the data server corresponding to the data identifier in the undeleted modification instruction. 4. A metadata processing method, applied to a distributed system, the distributed system includes a data server and at least two clients, wherein the at least two clients include a master client, and the master The control client is the client that needs to pass through when the at least two clients communicate with the data server; the method is executed by the master control client, and the method includes: receiving a read request, where the read request carries an object identifier; If no metadata corresponding to the object identifier is found in the stored correspondence between the object identifier and the metadata, at least one access instruction corresponding to the object identifier is generated according to the read request, and the The access instruction includes a data identifier of metadata corresponding to the object identifier; Send the access instruction to the data server corresponding to the data identifier; receiving target metadata returned by the data server according to the data identifier; Wherein, the object identifier is a file identifier or a virtual disk identifier. 5. The metadata processing method according to claim 4, wherein the read request also carries a client identifier; Correspondingly, after receiving the target metadata returned by the data server according to the data identifier, the method further includes: Sending the target metadata to a client corresponding to the client identifier, so that the client saves the target metadata as metadata corresponding to the object identifier. 6. A metadata processing device, applied to a distributed system, the distributed system includes a data server and at least two clients, wherein the at least two clients include a master client, and the master The control client is the client that needs to pass through when the at least two clients communicate with the data server; the device is the main control client, and the device includes: a receiving unit, configured to receive a write request, where the write request carries an object identifier; A processing unit, configured to generate at least one modification instruction corresponding to the object identifier according to the write request, the modification instruction includes a data identifier and a target value corresponding to the data identifier, and the data identifier is the same as the object identifier An identifier of metadata that needs to be modified in at least one piece of metadata corresponding to the object identifier; A sending unit, configured to send the modification instruction to the client and the data server corresponding to the data identifier, so that the client and the data server can respectively send the data identifier to the element corresponding to the data identifier according to the modification instruction. The data is modified to the target value; Wherein, the object identifier in the write request received by the receiving unit is a file identifier or a virtual disk identifier. 7. The device according to claim 6, wherein the processing unit is further configured to: After generating at least one modification instruction corresponding to the object identifier according to the write request, sending the at least one modification instruction to a data server for storing the modification instruction; Correspondingly, the sending unit is also used for: After sending the at least one modification instruction to the client and the data server corresponding to the data identifier, send a delete message to the data server for storing the modification instruction to indicate that the data for storing the modification instruction The server deletes the at least one modification instruction. 8. The device according to claim 7, wherein the client further comprises: an acquisition unit, configured to acquire undeleted modification instructions from the data server for storing modification instructions; Correspondingly, the sending unit is also used for: Send the undeleted modification instruction to the client and the data server corresponding to the data identifier in the undeleted modification instruction. 9. A metadata processing device, which is applied to a distributed system, and the distributed system includes a data server and at least two clients, wherein the at least two clients include a master client, and the master The control client is the client that needs to pass through when the at least two clients communicate with the data server; the device is the main control client, and the device includes: a receiving unit, configured to receive a read request, where the read request carries an object identifier; The processing unit generates at least one access instruction corresponding to the object identifier according to the read request when no metadata corresponding to the object identifier is found in the stored correspondence between the object identifier and the metadata , the access instruction includes a data identifier of metadata corresponding to the object identifier; a sending unit, configured to send the access instruction to a data server corresponding to the data identifier; Correspondingly, the receiving unit is further configured to receive target metadata returned by the data server according to the data identifier; Wherein, the object identifier in the read request received by the receiving unit is a file identifier or a virtual disk identifier. 10. The device according to claim 9, wherein the read request received by the receiving unit also carries a client identifier; Correspondingly, the sending unit is also used for: After receiving the target metadata returned by the data server according to the data identifier, sending the target metadata to a client corresponding to the client identifier, so that the client can save the target metadata is the metadata corresponding to the object ID.