Hi, thanks for your anser,
It appear strange to me, because with usual ethernet IPv4 you can have 2 different interfaces with separated networks (that's what I'm already doing for ethernet network on master node). I though it should be possible to define a third network using IPoIB, isnt'it?
EDIT:
in fact I think the problem is more related to hardware recognition by kernel, as I noticed this Link encap:UNSPEC in the ifconfig result. But I think the correct modules are loaded, or do you see something missing:
root@node02:~# lsmod | grep ib_
ib_ucm 22539 0
ib_srp 42367 0
scsi_transport_srp 20226 1 ib_srp
ib_ipoib 122897 0
ib_umad 22133 0
ib_cm 42799 4 ib_ucm,ib_srp,ib_ipoib,rdma_cm
ib_sa 33766 6 mlx4_ib,ib_srp,ib_ipoib,rdma_ucm,rdma_cm,ib_cm
ib_mad 51572 4 mlx4_ib,ib_umad,ib_cm,ib_sa
ib_uverbs 60698 2 ib_ucm,rdma_ucm
ib_core 101271 13 ib_ucm,mlx5_ib,mlx4_ib,ib_srp,ib_ipoib,ib_umad,rdma_ucm,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad,ib_uverbs
ib_addr 18748 3 rdma_cm,ib_uverbs,ib_core
compat 13709 19 ib_ucm,mlx5_ib,mlx5_core,mlx4_en,mlx4_ib,mlx4_core,ib_srp,scsi_transport_srp,ib_ipoib,ib_umad,rdma_ucm,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad,ib_uverbs,ib_core,ib_addr
EDIT 2:
actually, I didn't start opensm now I did and IPoIB is working. But I get a warning during driver installation, telling that I should use LSPCI to improve the performance of the PCI MaxReadReq to 4096. I do it, but after reboot, the value is the same as before. Is there a way to fix it without adding it in an init.d script?