> For the complete documentation index, see [llms.txt](https://docs.tibero.com/zetadata.en/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tibero.com/zetadata.en/network-configuration.md).
# Network Configuration
## Overview
ZetaData uses InfiniBand for fast network communication between DB nodes and Storage nodes. Follow the steps below to set up a ZetaData network using InfiniBand.
1. Install the InfiniBand driver.
2. Verify the operation of the InfiniBand driver.
3. Configure kernel parameters.
4. Configure a network (refer to the network configuration manual of each InfiniBand manufacturer).
{% hint style="info" %}
Note
If there is no InfiniBand or there is a problem with usage, then the network can be configured by
\
using the TCP protocol instead of the RDMA protocol.
Refer to the 'SSVR\_USE\_TCP parameter' in "Initialization Parameters" for instructions.
{% endhint %}
The figure below shows the IP address configuration for InfiniBand on the ZetaData node that will be used in the examples going forward.
###
***
## **1.** InfiniBand Driver Installation
To use InfiniBand, install a driver and library appropriate for the OS. Check the library by using each OS
\
command and install it first.
The following is an example of installing a library for each OS.
Ubuntu
```sql
$ sudo apt-cache search libibverb
```
Install the package 'Development files for the libibverbs library' searched with the above command.
```sql
$ sudo apt-get install libibverbs-dev
```
RedHat series (RHEL, CentOS, Fedora)
```sql
$ yum install libibverbs-devel libibverbs-devel-static
```
The following is an example of installing the InfiniBand driver on Red Hat 7,8 and 9 provided by Mellanox.
1\. Install the Melanox driver.
```sql
$ ./mlnxofedinstall
```
2. Create a network configuration file for InfiniBand. Below is an example of the configuration file on DB node 0.
```
$ echo "CONNECTED_MODE=yes
> TYPE=InfiniBand
> BOOTPROTO=none
> DEFROUTE=yes
> #IPV4_FAILURE_FATAL=no
> #IPV6INIT=yes
> #IPV6_AUTOCONF=yes
> #IPV6_DEFROUTE=yes
> #IPV6_FAILURE_FATAL=no
> NAME=ib0
> #UUID=6842318b-35d2-475a-8ae2-b3f9ec6b787a
> DEVICE=ib0
> ONBOOT=yes
> IPADDR=10.10.10.11
> NETMASK=255.255.255.0
> #PREFIX=32
> #IPV6_PEERDNS=yes
> #IPV6_PEERROUTES=yes" > /etc/sysconfig/network-scripts/ifcfg-ib0
```
3. Execute the opensmd service for switches that do not support the subnet manager function.
```
$ systemctl enable opensmd.service
$ service opensmd start
```
4. Restart the server.
###
***
## **2.** Verification of the InfiniBand driver operation
The following is an example of verifying the operation of the InfiniBand driver provided by Mellanox.
#### DB node #0
```sql
$ ip addr show ib0
8: ib0: mtu 65520 qdisc pfifo_fast \
state UP qlen 1024
link/infiniband 80:00:00:2c:fe:80:00:00:00:00:00:00:50:65:f3:ff:ff:88:81:d0 \
brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
inet 10.10.10.11/24 brd 10.10.10.255 scope global ib0
valid_lft forever preferred_lft forever
inet6 fe80::5265:f3ff:ff88:81d0/64 scope link
valid_lft forever preferred_lft forever
$ ibv_rc_pingpong
local address: LID 0x0006, QPN 0x008382, PSN 0xc85390, GID ::
remote address: LID 0x0002, QPN 0x02e388, PSN 0xb0671b, GID ::
8192000 bytes in 0.01 seconds = 10726.02 Mbit/sec
1000 iters in 0.01 seconds = 6.11 usec/iter
```
#### DB node #0
```sql
$ ip addr show ib0
8: ib0: mtu 65520 qdisc pfifo_fast \
state UP qlen 1024
link/infiniband 80:00:00:2c:fe:80:00:00:00:00:00:00:50:65:f3:ff:ff:88:71:40 \
brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
inet 10.10.10.12/24 brd 10.10.10.255 scope global ib0
valid_lft forever preferred_lft forever
inet6 fe80::5265:f3ff:ff88:7140/64 scope link
valid_lft forever preferred_lft forever
$ ibv_rc_pingpong 10.10.10.11
local address: LID 0x0002, QPN 0x02e388, PSN 0xb0671b, GID ::
remote address: LID 0x0006, QPN 0x008382, PSN 0xc85390, GID ::
8192000 bytes in 0.00 seconds = 13317.62 Mbit/sec
1000 iters in 0.00 seconds = 4.92 usec/iter
```
{% hint style="info" %}
**Note**
In the above example, only tests were performed between DB nodes using the ibv\_rc\_pinpong command, but it is recommended to perform InfiniBand driver operation tests from all nodes, including SSVR nodes, to each ZetaData node.
{% endhint %}
###
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.tibero.com/zetadata.en/network-configuration.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.