CN103457844B - Method for vNIC binding of multiple Infiniband gateways - Google Patents

Abstract

The invention provides a vNIC binding method of multiple Infiniband gateways, which distinguishes different vNICs virtualized by multiple Infiniband gateways by utilizing identification information of the gateways, binds the distinguished vNICs to specific IP information, and enables different IP sections to be transmitted through the determined gateways, thereby realizing the multiplication of bandwidth between an Infiniband network and an Ethernet network, breaking through the bottleneck between the Infiniband network and the Ethernet network, realizing high-speed data exchange between different types of networks, selecting an IP network section and the Infiniband gateway passing through the IP network section according to different machine names corresponding to different IP network sections in application, realizing the load balance of the multiple Infiniband gateways, and particularly having more obvious effect in large data application environments such as seismic data processing and the like.

Description

Method for vNIC binding of multiple Infiniband gateways

technical field

The invention relates to a vNIC binding technology in an Infiniband gateway environment, in particular to a vNIC (Virtual Network Interface Card, virtual network adapter) binding method of multiple Infiniband gateways.

Background technique

With the development of petroleum geophysical exploration technology, seismic exploration methods are developing in the direction of multi-dimensional, multi-component, and ultra-multi-trace. Currently, the main construction method is 1ms or 2ms sampling and the number of receiving traces is less than 5000. According to 1ms sampling and 5000 channel receiving calculations, the real-time data flow is about 20MB/s. If it is continuous acquisition (such as air gun operations at sea), the storage data speed must be above 20MB/s. According to the development trend of seismic exploration towards full wave field and high-density acquisition, perhaps in the near future, 10,000 or 20,000 channels will be sampled in 1ms or 0.5ms, and the real-time data acquisition flow rate at that time can reach up to 160MB/s, which makes seismic exploration The total amount of data collected is astonishing (usually the total amount of data for a construction season of the 3D team can reach TB level).

At the same time, seismic data processing technology has also achieved rapid development, from pre-stack time migration, pre-stack depth migration, to two-way wave wave equation migration, and full waveform inversion, multi-wave multi-component and other processing technologies. With the development of seismic data software processing technology, the performance of hardware resources required for processing has also been rapidly improved. From the previous PC cluster with hundreds of nodes for processing to the current GPU cluster with dozens of nodes, the cluster density is getting higher and higher. The processing power of the cluster server is getting stronger and stronger. This makes the number of servers required for a seismic data processing operation less and less. At the same time, due to the development of seismic exploration and acquisition technology, the source data of seismic data is getting larger and larger. The original data, and the result data whose data is several times larger than the original data, have higher and higher requirements for the network in the cluster system. On the other hand, the development of the data center now makes the network architecture of the cluster not only limited to the Ethernet network, but also the Infiniband network, and the large-scale storage resource devices used by the cluster may be stored in the Ethernet or in the Infiniband network. , so how to realize the sharing of different storage resources in Ethernet and Infiniband networks, and at the same time satisfy the rapid transmission of terabytes of data volume is a daunting challenge and an important issue for current geophysical exploration data centers.

At present, the existing single Infiniband gateway vNIC binding technology can realize the integration of Infiniband network and Ethernet, but because it can only realize the vNIC binding of one Infiniband gateway, all data transmission between Ethernet and Infiniband network can only be through This will greatly limit the data transmission bandwidth between the two networks, especially when the Ethernet network and the Infiniband network are relatively large data centers, this network bandwidth bottleneck is particularly obvious, and in the seismic data processing data center, Due to the large amount of data, many concurrent data accesses, and high concurrency, network congestion often occurs between the Ethernet network and the Infiniband network, resulting in large data transmission delays and even network interruptions, which greatly affects the data center. Cluster operation and business processing limit the development of data centers.

Contents of the invention

The purpose of the embodiments of the present invention is to provide a vNIC binding method for multiple Infiniband gateways, so as to increase the bandwidth between the Ethernet network and the Infiniband network, realize high-speed data exchange between the Ethernet network and the Infiniband network, and satisfy seismic data processing, etc. The need for real-time processing of big data.

In order to achieve the above object, the present invention provides a vNIC binding method of multiple Infiniband gateways, comprising the following steps:

Perform port-channel port binding on the multiple Ethernet ports connected to the Infiniband gateways of the Ethernet switch, so that the port-channel ports correspond to the Infiniband gateways one by one;

Binding the Ethernet ports in each of the Infiniband gateways as port-channel ports respectively, generating correspondingly configured vNICs, and configuring VLAN (Virtual Local Area Network, Virtual Local Area Network) information for each port-channel port;

The server node in the Infiniband network obtains each vNIC information, and obtains the corresponding Infiniband gateway identifier from the vNIC information;

The server node in the Infiniband network binds the specified IP address information to the vNIC generated corresponding to each Infiniband gateway according to the Infiniband gateway identifier, so that different IP segments are transmitted through the determined Infiniband gateway.

Wherein, after the port-channel port binding is done respectively to the Ethernet switches connected to the multiple Ethernet ports of each Infiniband gateway, it also includes:

Load balancing mode is set for each port-channel port in the Ethernet switch;

Create multiple VLANs for the Ethernet switch, and configure corresponding VLAN information;

Add each port-channel port in the Ethernet switch to a designated VLAN respectively.

Wherein, after the server node in the Infiniband network binds the specified IP address information to the correspondingly generated vNIC of each Infiniband gateway according to the Infiniband gateway identifier, it also includes:

The server nodes in the Infiniband network restart each vNIC;

Perform opensm subnet management configuration on the Infiniband network, and restart the subnet management service.

Wherein, the multiple Ethernet ports connected to each Infiniband gateway to the Ethernet switch are respectively port-channel port bound, specifically including:

Set the Ethernet port of each Infiniband gateway to switchport mode;

Enable each Ethernet port;

Create a port-channel port binding so that it corresponds to the Infiniband gateway one by one;

Add description information of the created port-channel port;

Add the corresponding Ethernet port for the created port-channel port;

Enable the port-channel port created as described.

Wherein, the Ethernet ports in each Infiniband gateway are respectively bound as port-channel ports to generate correspondingly configured vNICs, and VLAN information is configured for each port-channel port, specifically including:

Each of the Infiniband gateways creates a port-channel port respectively;

Enable the port-channel port of each Infiniband gateway;

Set the port-channel management mode of each Infiniband gateway;

Create designated VLANs for the port-channels of each Infiniband gateway;

Open the eport port;

Disable the lacp (Link Aggregation Control Protocol, Link Aggregation Control Protocol) protocol.

Wherein, the server node in the Infiniband network binds the specified IP address information to the correspondingly generated vNIC of each Infiniband gateway according to the Infiniband gateway identifier, specifically including:

Confirm which gateway the vNIC belongs to one by one according to the Infiniband gateway identification information in each vNIC;

Write the corresponding network card configuration information of the Infiniband gateway to which the vNIC belongs to the configuration file of the vNIC.

Wherein, the described multiple VLANs are created for the Ethernet switch, and corresponding VLAN information is configured, specifically including:

Create a plurality of VLANs corresponding to the Infiniband gateway one by one for the Ethernet switch;

Add VLAN description information for each VLAN;

Configure the network information of each VLAN;

Add the port-channel port in the designated Ethernet switch for each VLAN, and make the port-channel port, VLAN and Infniband gateway correspond one-to-one;

Enable the VLAN segment.

Wherein, before the port-channel port binding is performed on the multiple Ethernet ports connected to each Infiniband gateway by the Ethernet switch, it also includes:

Install EOIB drivers for all nodes in the Infiniband network, and modify the EOIB parameters in its Infiniband configuration file to be set to start automatically at boot;

Connecting the server node to an Infiniband switch, connecting the Infiniband switch to a corresponding Infiniband gateway, and connecting the Infiniband gateway to the Ethernet switch.

The vNIC binding method of multiple Infiniband gateways in the embodiment of the present invention distinguishes different vNICs virtualized by multiple Infiniband gateways by using the identification information of the gateways, and binds the distinguished vNICs to specific IP information, so that different IP segments pass through the determined Gateway for transmission, thereby realizing the multiplication of bandwidth between the Infiniband network and the Ethernet network, breaking through the bottleneck between the Infiniband network and the Ethernet network, and realizing high-speed data exchange between two different types of networks, and in applications according to different Different machine names corresponding to the IP network segment are used to select the IP network segment and the passed Infiniband gateway to achieve load balancing of multiple Infiniband gateways, especially in big data application environments such as seismic data processing.

Description of drawings

The drawings described here are used to provide further understanding of the present invention, constitute a part of the application, and do not limit the present invention. In the attached picture:

FIG. 1 is a flowchart of a vNIC binding method for multiple Infiniband gateways according to an embodiment of the present invention.

Fig. 2 is that in the vNIC binding method of multiple Infiniband gateways in the embodiment of the present invention, port-channel port binding is performed on multiple Ethernet ports connected to each Infiniband gateway by an Ethernet switch, the load balancing mode is set, and added to a specific Flow chart of VLAN;

Fig. 3 is the vNIC binding method of multiple Infiniband gateways in the embodiment of the present invention, the Ethernet ports in each Infiniband gateway are respectively port-channel port bound, generate correspondingly configured vNICs, and configure VLAN information for each port-channel port Part of the flowchart.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments and descriptions of the present invention are used to explain the present invention, but not to limit the present invention.

The specific embodiment of the present invention will be further described in detail below in conjunction with the accompanying drawings.

Please refer to Figure 1, Figure 1 is a vNIC binding method for multiple Infiniband gateways according to an embodiment of the present invention, which specifically includes the following steps:

Step S1, install EOIB drivers for all nodes in the Infiniband network, and modify EOIB parameters in their Infiniband configuration files, so as to set EOIB to start automatically when booting. Among them, EOIB is the EOIB agreement.

Step S2, connect the server node to the Infiniband switch, connect the Infiniband switch to the corresponding Infiniband gateway, and connect the Infiniband gateway to the Ethernet switch.

Step S3, perform port-channel port binding on the multiple Ethernet ports connected to the Infiniband gateways on the Ethernet switch, so that the port-channel ports correspond to the Infiniband gateways one by one, set the load balancing mode, and add to a specific VLAN , Bind multiple Ethernet ports on the Ethernet switch side, increase the link bandwidth between the Ethernet and Infiniband network through port aggregation, and play a role in load balancing.

Step S4, bind the Ethernet ports in each Infiniband gateway as port-channel ports, generate correspondingly configured vNICs, and configure VLAN information for each port-channel port. In this way, the Ethernet port binding on the Infiniband gateway side is completed, and the port binding is realized at both ends of the Ethernet switch at the same time, so as to increase the bandwidth between the Ethernet and the Infiniband network and play a load balancing role. By configuring the vNIC after port-channel is completed, the corresponding vNIC is configured for each server through the EOIB protocol.

Step S5, the server node in the Infiniband network obtains the information of each vNIC.

Step S6, the server node in the Infiniband network obtains the Infiniband gateway identifier BX_GUID from the obtained vNIC information. Specifically, query all vNIC information on the server node of the Infiniband network, obtain the Infiniband gateway name (BX_NAME) and Infiniband gateway ID (BX_GUID) in the corresponding vNIC information, and distinguish the gateway information corresponding to different vNICs, so as to realize different Infiniband gateway configurations Different IP segments, the basis for judging that the same IP segment is configured for vNICs generated by the same Infiniband gateway.

Step S7, according to different gateway identifiers, bind different vNICs to specified IP address information, so that different IP segments are transmitted through the determined Infiniband gateway.

Step S8, the server node in the Infiniband network restarts each vNIC.

Step S9, performing opensm subnet management configuration on the Infiniband network, and restarting the subnet management service. .

In addition, in the embodiment of the present invention, since different data transmission jobs can be assigned to different nodes, different machine names can be specified in the data transmission job so that different data transmissions can pass through different gateways, for example, BX1_ibm** is The server node name corresponding to the network segment generated by the ibm** node through Infiniband gateway 1, BX2_ibm** is the server node name corresponding to the network segment generated by the Infiniband gateway 2 of the ibm** node, and finally reduces the Infiniband gateway load and improves The purpose of transmission efficiency.

As shown in Figure 2, in the above step S3, the multiple Ethernet ports connected to the Infiniband gateways of the Ethernet switch are respectively port-channel port bound, so that the port-channel and the Infiniband gateways correspond one-to-one, and a specific VLAN is configured The specific steps of the information are as follows:

Step S301, setting the Ethernet port of each Infiniband gateway to switchport mode.

Step S302, enabling each Ethernet port.

Step S303 , creating port-channel port bindings so that they correspond one-to-one with Infiniband gateways.

Step S304, adding description information of the created port-channel port.

Step S305, adding a corresponding Ethernet port to the created port-channel port.

Step S306, enabling the created port-channel port;

Step S307, creating multiple VLANs corresponding to the Infiniband gateways for the Ethernet switch.

Step S308, adding VLAN description information for each VLAN.

Step S309, configuring the network information of each VLAN.

Step S310, adding designated port-channel ports in the Ethernet switch for each VLAN, and making a one-to-one correspondence between port-channel ports, VLANs, and Infniband gateways.

Step S311, enable the VLAN segment.

As shown in Figure 3, the Ethernet ports in the Infiniband gateways in the Infiniband gateways in the above step S4 are port-channel port bound respectively, correspondingly configured vNICs are generated, and VLAN information is configured for each port-channel port. The specific process is as follows :

Step S401, each Infiniband gateway creates a port-channel port respectively.

Step S402, enable the port-channel port of each Infiniband gateway.

Step S403, setting the port-channel management mode of each Infiniband gateway.

Step S404, creating designated VLANs for the port-channels of each Infiniband gateway.

Step S405, open the eport port.

Step S406, closing the lacp protocol.

The vNIC binding scheme of multiple Infiniband gateways in the embodiment of the present invention enables data to be transmitted through multiple gateways, greatly widens the bandwidth between the Ethernet network and the Infiniband network, and solves the bandwidth bottleneck problem between the two networks. At the same time, through automatic Identifying different Infiniband gateways can enable the network configuration of each server node to be automatically bound to different gateways, so that the node network is automatically distributed to different gateways, so that it can not only greatly increase the bandwidth between the two networks, but also increase the internal data of the cluster. Load balancing and reasonable distribution among different Infiniband gateways during transmission.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, within the spirit and principles of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.

Claims (8)

1. A vNIC binding method of many Infiniband gateways, comprising the following steps: Perform port-channel port binding on the multiple Ethernet ports connected to the Infiniband gateways of the Ethernet switch, so that the port-channel ports correspond to the Infiniband gateways one by one; Bind the Ethernet ports in each Infiniband gateway as port-channel ports respectively, generate a correspondingly configured vNIC virtual network adapter, and configure VLAN information for each port-channel port; The server node in the Infiniband network obtains each vNIC information, and obtains the corresponding Infiniband gateway identifier from the vNIC information; According to the Infiniband gateway identifier, the server node in the Infiniband network binds the specified IP address information to the correspondingly generated vNIC of each Infiniband gateway, so that different IP segments are transmitted through the determined Infiniband gateway. 2. the vNIC binding method of many Infiniband gateways according to claim 1, after the multiple ethernet ports that are connected to each Infiniband gateway to the Ethernet switches are respectively done port-channel port binding, it also includes: Load balancing mode is set for each port-channel port in the Ethernet switch; Create a plurality of VLAN virtual local area networks for the Ethernet switch, and configure corresponding VLAN information; Add each port-channel port in the Ethernet switch to a designated VLAN respectively. 3. the vNIC binding method of many Infiniband gateways according to claim 1, the server node in the Infiniband network according to the Infiniband gateway identification, binds the specified IP address information to the vNIC correspondingly generated by each Infiniband gateway After that, also include: The server nodes in the Infiniband network restart each vNIC; Perform opensm subnet management configuration on the Infiniband network, and restart the subnet management service. 4. the vNIC binding method of many Infiniband gateways according to claim 1, described to be connected to a plurality of ethernet ports of each Infiniband gateway to Ethernet switch respectively do port-channel port binding, specifically comprise: Set the Ethernet port of each Infiniband gateway to switchport mode; Enable each Ethernet port; Create a port-channel port binding so that it corresponds to the Infiniband gateway one by one; Add description information of the created port-channel port; Add the corresponding Ethernet port for the created port-channel port; Enable the port-channel port created as described. 5. the vNIC binding method of many Infiniband gateways according to claim 1, described Ethernet port in each Infiniband gateway is done port-channel port binding respectively, generates the vNIC of corresponding configuration, and for each port-channel Port configuration VLAN information, including: Each of the Infiniband gateways creates a port-channel port respectively; Enable the port-channel port of each Infiniband gateway; Set the port-channel management mode of each Infiniband gateway; Create designated VLANs for the port-channels of each Infiniband gateway; Open the eport port; Disable the lacp link aggregation control protocol. 6. the vNIC binding method of many Infiniband gateways according to claim 1, the server node in the Infiniband network according to the Infiniband gateway identification, binds the specified IP address information to the vNIC correspondingly generated by each Infiniband gateway, Specifically include: Confirm which gateway the vNIC belongs to one by one according to the Infiniband gateway identification information in each vNIC; Write the network card configuration information in the server node corresponding to the Infiniband gateway to which the vNIC belongs to the configuration file of the vNIC. 7. The vNIC binding method of many Infiniband gateways according to claim 2, described creating a plurality of VLANs for Ethernet switches, and configuring corresponding VLAN information, specifically includes: Create a plurality of VLANs corresponding to the Infiniband gateway one by one for the Ethernet switch; Add VLAN description information for each VLAN; Configure the network information of each VLAN; Add the port-channel port in the designated Ethernet switch for each VLAN, and make the port-channel port, VLAN and Infniband gateway correspond one-to-one; Enable the VLAN segment. 8. The vNIC binding method of many Infiniband gateways according to claim 1, before the multiple Ethernet ports that are connected to each Infiniband gateway to the Ethernet switch, do port-channel port binding respectively, further comprising: Install EOIB drivers for all nodes in the Infiniband network, and modify the EOIB parameters in its Infiniband configuration file to be set to start automatically at boot; Connecting the server node to an Infiniband switch, connecting the Infiniband switch to a corresponding Infiniband gateway, and connecting the Infiniband gateway to the Ethernet switch.