Configuration:
Host: CentOS 6.4
Mellanox Card: ConnectX-3 EN
Mellanox Driver: MLNX_OFED_LINUX-2.1-1.0.0-rhel6.4-x86_64 with SRIOV enabled
Virtualization: KVM
I have two servers with the above config, with Mellanox ports from one server are connected to other one using 10G SFP modules with fibre cables.
What happens is doing soft-reboot(typing reboot in a terminal) causes the the link led(green led) to not come up after the reboot.
- Ifconfig down/up doesn't work
- Unplugging replugging fibre cables doesn't work
- Unloading reloading drivers didn't help
- Unplugging replugging the SFP module is the only solution that works.
The above problem happens only randomly, which makes finding the root cause hard.
Edit1: Doing the self diagnostics test with ethtool shows link test failed.
host1# ethtool -t eth7
The test result is FAIL
The test extra info:
Interrupt Test 0
Link Test 1
Speed Test 0
Register Test 0
Loopback Test 0
Thanks!