How to create shared virtual IP using Pacemaker
Shared resources in a cluster need to be made accessible via an IP address. For this, we need to create a clustered IP address for our Pacemaker cluster, which will automatically be assigned to one of our cluster nodes at any point in time. Pacemaker will automatically re-assign the IP address to another node in our cluster if anything is wrong with the node currently assigned the IP address.
Shared virtual IP could be created on a Pacemaker cluster using pcs tool at the terminal.
Steps to create floating IP address using Pacemaker:
-
Install Pacemaker, Corosync and pcs on all of your available nodes.
-
Create a Pacemaker cluster from one of your nodes.
Related: How to create a Pacemaker cluster
-
Create a cluster IP resource using pcs from one of your cluster nodes.
$ sudo pcs resource create cluster_ip ocf:heartbeat:IPaddr2 ip=192.168.111.10 cidr_netmask=24 op monitor interval=30s
-
Ping the newly created IP address from all of your cluster nodes.
$ ping -c3 192.168.111.10 PING 192.168.111.10 (192.168.111.10) 56(84) bytes of data. 64 bytes from 192.168.111.10: icmp_seq=1 ttl=64 time=0.054 ms 64 bytes from 192.168.111.10: icmp_seq=2 ttl=64 time=0.086 ms 64 bytes from 192.168.111.10: icmp_seq=3 ttl=64 time=0.048 ms --- 192.168.111.10 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.048/0.062/0.086/0.019 ms
-
View details of the created cluster IP resource using pcs from one of your cluster nodes.
$ sudo pcs status resources cluster_ip Resource: cluster_ip (class=ocf provider=heartbeat type=IPaddr2) Attributes: cidr_netmask=24 ip=192.168.111.10 Operations: monitor interval=30s (cluster_ip-monitor-interval-30s) start interval=0s timeout=20s (cluster_ip-start-interval-0s) stop interval=0s timeout=20s (cluster_ip-stop-interval-0s)
-
View details of cluster and created resources using pcs from one of your cluster nodes.
$ sudo pcs status Cluster name: clustername Stack: corosync Current DC: node-02 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum Last updated: Mon Sep 30 05:38:30 2019 Last change: Mon Sep 30 05:31:00 2019 by root via cibadmin on node-01 2 nodes configured 1 resource configured Online: [ node-01 node-02 ] Full list of resources: cluster_ip (ocf::heartbeat:IPaddr2): Started node-01 Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled
Make sure the status for the pacemaker, corosync and pcsd daemon is enabled.
If the status is disabled, re-enable it which is to set the services to automatically start during system boot.$ sudo systemctl enable pacemaker corosync pcsd Created symlink from /etc/systemd/system/multi-user.target.wants/pacemaker.service to /usr/lib/systemd/system/pacemaker.service. Created symlink from /etc/systemd/system/multi-user.target.wants/corosync.service to /usr/lib/systemd/system/corosync.service. Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.
-
Get the name of the ethernet interface on the node where the cluster_ip resource is started on.
$ ip address show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:cd:b0:bc brd ff:ff:ff:ff:ff:ff inet 192.168.111.11/24 brd 192.168.111.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::a444:1cd9:da0c:900f/64 scope link tentative noprefixroute dadfailed valid_lft forever preferred_lft forever inet6 fe80::4f8c:f47a:308c:75af/64 scope link noprefixroute valid_lft forever preferred_lft forever
-
Disable the ethernet interface on the node where the cluster_ip resource is started on to test failover.
$ sudo ip link set ens33 down
You will be disconnected from the node if you are connecting remotely using ssh via the disabled interface
-
Check the cluster status on any other node to see if the cluster_ip resource correctly started on another node.
$ sudo pcs status Cluster name: clustername Stack: corosync Current DC: node-02 (version 1.1.20-5.el7_7.1-3c4c782f70) - partition with quorum Last updated: Wed Oct 2 22:38:57 2019 Last change: Wed Oct 2 22:21:59 2019 by root via cibadmin on node-01 2 nodes configured 1 resource configured Online: [ node-02 ] OFFLINE: [ node-01 ] Full list of resources: cluster_ip (ocf::heartbeat:IPaddr2): Started node-02 Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/enabled
The status of the node with disabled ethernet interface should now be OFFLINE .
-
Ping the cluster_ip's address again to test that the IP address is still reachable.
$ ping -c3 192.168.111.10 PING 192.168.111.10 (192.168.111.10) 56(84) bytes of data. 64 bytes from 192.168.111.10: icmp_seq=1 ttl=64 time=0.059 ms 64 bytes from 192.168.111.10: icmp_seq=2 ttl=64 time=0.076 ms 64 bytes from 192.168.111.10: icmp_seq=3 ttl=64 time=0.061 ms --- 192.168.111.10 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.059/0.065/0.076/0.011 ms
-
Re-enable the ethernet interface on the OFFLINE node to for the node to re-join our cluster.
$ sudo ip link set ens33 up
-
Re-check if pacemaker, corosync and pcsd services are enabled (to automatically start during system boot) on all nodes.
$ sudo systemctl status pacemaker corosync pcsd | grep Loaded Loaded: loaded (/usr/lib/systemd/system/pacemaker.service; disabled; vendor preset: disabled) Loaded: loaded (/usr/lib/systemd/system/corosync.service; disabled; vendor preset: disabled) Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor preset: disabled)
Certain pcs commands tend to disable previously enabled services. Enable using the following command:
$ sudo systemctl enable pacemaker corosync pcsd Created symlink from /etc/systemd/system/multi-user.target.wants/pacemaker.service to /usr/lib/systemd/system/pacemaker.service. Created symlink from /etc/systemd/system/multi-user.target.wants/corosync.service to /usr/lib/systemd/system/corosync.service. Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.