Re: VMware ESXi 6.0 virtual ib_ipoib interfaces
Hi David, 1) Driver version 2.4.0 is an Ethernet driver only so if you have a VPI card, please make sure the port protocol is configured as 2 which is Ethernet. #/opt/mellanox/bin/mlxconfig -d...
View ArticleRe: ConnectX-3 RoCE fails send WRs when running multiple QPs
I afraid it is tricky to understand without debugging and source code, but most likely there is a problem there. Try to print the values of lkey/rkey, QP numbers, protection domains, other related data...
View ArticleRe: 'State: Initializing' but works
Thank you eddie.notz for your response. Yes.Both port 1 and port 2 status had been 'active' till then. (I did not apply any updates etc in recent and all updates are always manually installed to avoid...
View ArticleRe: Which ESXi driver to use for SRP/iSER over IB (not Eth!)?
7 days later and no reply from Mellanox... Seriously?
View ArticleRe: VMware ESXi 6.0 virtual ib_ipoib interfaces
Hi Sopie we don't want to use it as Ethernet card but as IB over IP. That's the problem that I can't get it to work as a 40GB IBoIP card in ESX 6 anymore, which worked perfectly with ESX 5.5 and the...
View ArticleHow to change speed from FDR to QDR using ibportstate command in centos 6.2
We have Mellanox Infiniband FDR cards and cables supporting it. We want to reduce the speed of the adapters from 56 Gbps to 40 Gbps. We tried using ibportstate <base lid> <port id> of the...
View ArticleRe: How to change speed from FDR to QDR using ibportstate command in centos 6.2
ibportstate <lid > <port> fdr10 0 espeed 30 ibportstate <lid> <port> reset
View ArticleIs it possible to connect 2 ethernet switches (sx1036) located up to 10km...
Hi guys, I have a question that is real important for me; How can I connect my two sites so that I am able to use RDMA between them? I have 2 racks, each located in a different datacenter. Each rack...
View ArticleRe: How to setup IPoIB correctly?
I have been kept digging, and here is what I have observed: I just did a [root@sc2u0n0 ~]# lsinitrd /boot/initramfs-3.10.0-327.28.2.el7.x86_64.img|less[...] drwxr-xr-x 5 root root 0 Aug...
View ArticleRe: VMware ESXi 6.0 virtual ib_ipoib interfaces
Driver 2.4.0 include IPoIB driver and works!Absolutlely! driver 2.4.0 located in ethernet driver section, but why this driver 2.4.0 include IPoIB driver & works?
View ArticleRe: Is it possible to connect 2 ethernet switches (sx1036) located up to 10km...
You can do that using MetroX and InfiniBand connection for 10Km, we don't support RoCE for such distance.This is mainly due to packet loss and buffer size of the switch on each side (depends on the...
View ArticleRe: How to setup IPoIB correctly?
I resolved the problem by moving all IPoIB IPv4 addresses to a different subnet.
View ArticleClik here to view.
Re: Which ESXi driver to use for SRP/iSER over IB (not Eth!)?
Looking at the current driver selection across different OS/cards/medium (IB vs. ETH) it looks like the only space consistently supported by Mellanox is Linux. Indeed, for Linux you have...
View ArticleRe: I have 2 colocation sites located 6-7km apart. I can rent a 10gb wave...
please refer to this post: Is it possible to connect 2 ethernet switches (sx1036) located up to 10km apart without losing RDMA?
View ArticleRe: Can multiple versions of mlnx-ofed exist in the same IB fabric?
Hi Greg, It is recommended to perform the upgrade during a maintenance windows and not in production, because there are a lot of differences between 2.X and 3.X versions and since part of the driver...
View ArticleClik here to view.
Is this the best our FDR adapters can do?
We have a small test setup illustrated below. I have done some ib_write_bw tests. Got "decent" numbers, but not as fast as I anticipated. First, some background of the setup: Two 1U storage servers...
View ArticleRe: Is this the best our FDR adapters can do?
One thing to keep in mind is that you'll hit the bandwidth of the PCIe bus.I've not used the ib_write test myself - but I'm fairly sure that it's not actually handling data - just accepting it and...
View ArticleRe: How to change speed from FDR to QDR using ibportstate command in centos 6.2
Hi, Thanks for the help, however, unfortunately this didn't help. Without firing command: CA PortInfo:# Port info: Lid 144 port...
View ArticleRe: Which ESXi driver to use for SRP/iSER over IB (not Eth!)?
KVM?No! KVM also have many limitations.For example EoIB, etc.Infiniband communication rely on SM- Subnet ManagerSM consist of some components and API.But SM architecture didn't designed for hypervisor...
View Article