配置裸金属服务器上的网卡IP(集群)
操作场景
SAP HANA集群的内部通信平面,在申请裸金属服务器时无法自动创建,需要手动配置该平面网卡的IP地址,满足SAP系统的部署要求。
操作步骤
- 以“root”用户,使用密钥登录裸金属服务器。
- 查询网卡信息。
ip link
返回信息示例如下:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 link/ether 38:4c:4f:89:55:8d brd ff:ff:ff:ff:ff:ff 5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 link/ether 38:4c:4f:89:55:8e brd ff:ff:ff:ff:ff:ff 6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 8888 qdisc noqueue state UP mode DEFAULT group default link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff
- 设置udev规则。
执行以下命令创建“80-persistent-net.rules”文件。
cp /etc/udev/rules.d/70-persistent-net.rules /etc/udev/rules.d/80-persistent-net.rules
将2中查询到的,且“80-persistent-net.rules”中未体现的网卡MAC地址和名称,写入该文件中,使得裸金属服务器重启复位后,网卡名称和顺序不会发生改变。
网卡的MAC地址和名称中的字母,请使用小写字母。
vim /etc/udev/rules.d/80-persistent-net.rules
修改后的示例如下:
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="38:4c:4f:29:0b:e0", NAME="eth0" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="38:4c:4f:29:0b:e1", NAME="eth1" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="38:4c:4f:89:55:8d", NAME="eth2" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="38:4c:4f:89:55:8e", NAME="eth3"
修改完成后,保存并退出。
- 查询网卡的IP信息。
ifconfig
返回信息示例如下,其中的“bond0”为申请裸金属服务器时自动分配的网卡平面IP地址。
bond0 Link encap:Ethernet HWaddr FA:16:3E:3D:1C:E0 inet addr:10.0.3.2 Bcast:10.0.3.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe3d:1ce0/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:8888 Metric:1 RX packets:852 errors:0 dropped:160 overruns:0 frame:0 TX packets:1121 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:125429 (122.4 Kb) TX bytes:107221 (104.7 Kb) eth0 Link encap:Ethernet HWaddr FA:16:3E:3D:1C:E0 UP BROADCAST RUNNING SLAVE MULTICAST MTU:8888 Metric:1 RX packets:428 errors:0 dropped:10 overruns:0 frame:0 TX packets:547 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:64670 (63.1 Kb) TX bytes:50132 (48.9 Kb) eth1 Link encap:Ethernet HWaddr FA:16:3E:3D:1C:E0 UP BROADCAST RUNNING SLAVE MULTICAST MTU:8888 Metric:1 RX packets:424 errors:0 dropped:7 overruns:0 frame:0 TX packets:574 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:60759 (59.3 Kb) TX bytes:57089 (55.7 Kb) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:8 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:520 (520.0 b) TX bytes:520 (520.0 b)
- 查询组成bond的网卡的名称。
已组成bond且在使用中的网卡,不能用于内部通信平面,因此需要查询相应的网卡名称。
cd /etc/sysconfig/network
vi ifcfg-bond0
返回信息示例如下,可见“bond0”由“eth0”和“eth1”组成。
BONDING_MASTER=yes TYPE=Bond STARTMODE=auto BONDING_MODULE_OPTS="mode=4 xmit_hash_policy=layer3+4 miimon=100" NM_CONTROLLED=no BOOTPROTO=dhcp DEVICE=bond0 USERCONTRL=no LLADDR=fa:16:3e:3d:1c:e0 BONDING_SLAVE1=eth1 BONDING_SLAVE0=eth0
查询完成后,退出。
- 查询所有网卡的状态。
ip link
返回信息示例如下:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 link/ether 38:4c:4f:89:55:8d brd ff:ff:ff:ff:ff:ff 5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 link/ether 38:4c:4f:89:55:8e brd ff:ff:ff:ff:ff:ff 6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 8888 qdisc noqueue state UP mode DEFAULT group default link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff
- 将所有状态为“qdisc mq state DOWN”的网卡,设置为“qdisc mq state UP”,示例中为“eth2”和“eth3”。
ip link set eth2 up
ip link set eth3 up
- 重新查询网卡的状态。
ip link
返回信息示例如下:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 38:4c:4f:89:55:8d brd ff:ff:ff:ff:ff:ff 5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 38:4c:4f:89:55:8e brd ff:ff:ff:ff:ff:ff 6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 8888 qdisc noqueue state UP mode DEFAULT group default link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff
- 查看8中对应的网卡的状态,获取状态为“qdisc mq state UP”的网卡名称。
只有状态为“qdisc mq state UP”且未被使用过的网卡,才能组成bond,示例中为“eth2”和“eth3”。
“eth2”的LLADR为“38:4c:4f:89:55:8d”,“eth3”的LLADR为“38:4c:4f:89:55:8e”。
- 创建“eth2”和“eth3”网卡的配置文件。
可通过复制已有网卡配置文件的方式快速创建。
cp ifcfg-eth0 ifcfg-eth2
cp ifcfg-eth1 ifcfg-eth3
- 修改“eth2”和“eth3”网卡的配置文件
vi ifcfg-eth2
vi ifcfg-eth3
“eth2”网卡配置文件的修改示例如下。
其中,参数参数“MTU”配置为“8888”,,“BOOTPROTO”需要配置为“STATIC”,参数“DEVICE”、“LLADDR”根据实际需要填写。
STARTMODE=auto MTU=8888 NM_CONTROLLED=no BOOTPROTO=STATIC DEVICE=eth2 USERCONTRL=no LLADDR=38:4c:4f:89:55:8d TYPE=Ethernet
“eth3”网卡配置文件的修改示例如下:
STARTMODE=auto MTU=8888 NM_CONTROLLED=no BOOTPROTO=STATIC DEVICE=eth3 USERCONTRL=no LLADDR=38:4c:4f:89:55:8e TYPE=Ethernet
修改完成后,保存并退出。
- 将“eth2”和“eth3”组bond,假设为“bond1”。
创建ifcfg-bond1文件并修改配置。
cp ifcfg-bond0 ifcfg-bond1
vi ifcfg-bond1
“bond1”网卡配置文件的修改示例如下。
其中,参数“MTU”配置为“8888”,“BONDING_MODULE_OPTS”配置为“mode=1 miimon=100”,“BOOTPROTO”需要配置为“STATIC”,“DEVICE”、“BONDING_SLAVE1”、“BONDING_SLAVE0”、“IPADDR”、“NETMASK”、“NETWORK”根据实际需要填写,“LLADDR”配置为参数“BONDING_SLAVE1”对应网卡的LLADDR。
BONDING_MASTER=yes TYPE=Bond MTU=8888 STARTMODE=auto BONDING_MODULE_OPTS="mode=1 miimon=100" NM_CONTROLLED=no BOOTPROTO=STATIC DEVICE=bond1 USERCONTRL=no LLADDR=38:4c:4f:89:55:8d BONDING_SLAVE1=eth2 BONDING_SLAVE0=eth3 IPADDR=10.0.2.2 NETMASK=255.255.255.0 NETWORK=10.0.2.0
修改完成后,保存并退出。
- 使配置文件生效。
- 创建临时目录,并将网络配置文件复制到该目录下。
mkdir /opt/tmp/xml
cp /etc/sysconfig/network/ifcfg* /opt/tmp/
cp /etc/sysconfig/network/config /opt/tmp/
cp /etc/sysconfig/network/dhcp /opt/tmp/
- 停止待组成bond1的网卡。
ip link set eth3 down
- 将网卡配置文件转换成操作系统可辨识的配置文件。
/usr/sbin/wicked --log-target=stderr --log-level=debug3 --debug all convert --output /opt/tmp/xml /opt/tmp/
- 重新启用待组成bond1的网卡。
/usr/sbin/wicked --log-target=stderr --log-level=debug3 --debug all ifup --ifconfig /opt/tmp/xml/eth2.xml eth2
ip link set eth3 up
/usr/sbin/wicked --log-target=stderr --log-level=debug3 --debug all ifup --ifconfig /opt/tmp/xml/eth3.xml eth3
/usr/sbin/wicked --log-target=stderr --log-level=debug3 --debug all ifup --ifconfig /opt/tmp/xml/bond1.xml bond1
- 创建临时目录,并将网络配置文件复制到该目录下。
- 重新查询IP地址信息,可查看到IP地址已分配。
ip addr show
示例如下:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond0 state UP group default qlen 1000 link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond1 state UP group default qlen 1000 link/ether 38:4c:4f:89:55:8d brd ff:ff:ff:ff:ff:ff 5: eth3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 8888 qdisc mq master bond1 state UP group default qlen 1000 link/ether 38:4c:4f:89:55:8d brd ff:ff:ff:ff:ff:ff 6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 8888 qdisc noqueue state UP group default link/ether fa:16:3e:3d:1c:e0 brd ff:ff:ff:ff:ff:ff inet 10.0.3.2/24 brd 10.0.3.255 scope global bond0 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe3d:1ce0/64 scope link valid_lft forever preferred_lft forever 7: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 8888 qdisc noqueue state UP group default link/ether 38:4c:4f:89:55:8d brd ff:ff:ff:ff:ff:ff inet 10.0.2.2/24 brd 10.0.2.255 scope global bond1 valid_lft forever preferred_lft forever inet6 fe80::3a4c:4fff:fe29:b36/64 scope link valid_lft forever preferred_lft forever
- 删除创建的临时目录。
cd /opt
rm -rf tmp/
- 参见上述步骤,完成其他裸金属服务器上的配置。