GlusterFS – ‘Host not connected’ when creating a volume on VBox VMs [closed]

I’m trying to deploy a simple GlusterFS cluster on CentOS using Virtualbox. I only have two nodes (gluster-node1 and gluster-node4) with two bricks each and I’m creating a volume with replica level 2.

When I issue the volume create command on gluster-node4 I get this error: Host gluster-node1.test not connected. The glustersd daemon is running on both host and if I check the peer status it returns State: Peer in Cluster (Disconnected) on both host.
I checked the /var/log/glusterfs/etc-glusterfs-glusterd.vol.log file on gluster-node4 and found this error E [socket.c:2157:socket_connect_finish] 0-management: connection to 192.168.1.61:24007 failed (No route to host), but if I try to ping or ssh to the gluster-node1 host it works without any problem.

As far as I can see there is no rule in iptables that would block this connection and I can see a route in the routing table.

# iptables --list -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.0.2.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1003   0        0 eth1
0.0.0.0         10.0.2.2        0.0.0.0         UG    0      0        0 eth0

(eth1 is an Host-only interface on the Vbox VMs)

Am I missing something? Any suggestions?

— UPDATE —

Both peers have entries in the /etc/hosts

# more /etc/hosts (on gluster-node1)
127.0.0.1   gluster-node1.test gluster-node1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.64    gluster-node4 gluster-node4.test

# more /etc/hosts (on gluster-node4)
127.0.0.1   gluster-node4.test gluster-node4 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.61    gluster-node1 gluster-node1.test

Answer

Attribution
Source : Link , Question Author : dstroppa , Answer Author : Thomas Berger