Re: how do I configure Mellanox sn2700 to connect to a 40Gb port on a RS8264...
Ok I do have a SB7700 across the isle... I will test with that switch.
View ArticleClik here to view.
permanent setting log_num_mtt and log_mtts_per_seg
I am using MHEH29 with MLNX_OFED 1.5.3. on centos 6.2.To increase memory size, I set log_num_mtt and log_mtts_per_seg as#echo "22" > /sys/module/mlx4_core/parameters/log_num_mtt#echo "1" >...
View ArticleSM LID is not configured warning
Hi All, I have a pair of 4036 switches connected to a few server nodes. Each server node has a dual port Connectx2 card with one link to each switch. The switches then have two connections between...
View ArticleClik here to view.
Re: how do I configure Mellanox sn2700 to connect to a 40Gb port on a RS8264...
One question, Is this an IBM (BLADE, BNT) switch? On some systems, you need a license to enable the 40 Gbps ports.
View ArticleRe: SM LID is not configured warning
Are you running single NTttcp receiver/sender between servers or are there multiple of these going on concurrently ? Is PortXmitWait increasing ? It is indicative of congestion. Perhaps some machine is...
View ArticleRe: SM LID is not configured warning
Hi Hal,Thanks for the help on this.I am running between two servers with the following:ntttcp.exe -r -a 16 -t 15 -m 16,*,10.1.4.11ntttcp.exe -s -a 16 -t 15 -m 16,*,10.1.4.11 Which two servers does not...
View ArticleRe: SM LID is not configured warning
PortXmitWait on sending side means some link is slow (I suspect sending side is faster than receiving side). I'm not familiar with Windows performance tuning. Is ntttcp throughput in bytes/sec or...
View ArticleRe: SM LID is not configured warning
Here is the full output, throughput is MegaBytes/Sec. To compare a normal 1GBE connection usually tests round 112 MB/S. These have been in the 1500-2000 range.Copyright Version 5.31Network activity...
View ArticleRe: SM LID is not configured warning
112MB/s * 8 bits/byte = 892 Mb/s which is close to 1 Gbps so that makes sense to me. 1500-2000 MB/s = 12-16 Gb/s. 40 Gbps IB is 32 Gbps data rate max so that's .375-.5 of the 4x QDR IB "line" rate.
View ArticleRe: SM LID is not configured warning
Indeed, I feel like 75-90% of line rate is what I should expect. My experience with infiniband is pretty much none though, so maybe this is the expected throughput for IPoIB? I kinda doubt it. It...
View ArticleRe: SM LID is not configured warning
In general, IPoIB does not show off IB performance but this largely depends on packet sizes used. The fact that the throughput is independent of which machines used indicates that it's either host...
View ArticleRe: SM LID is not configured warning
Thanks Hal, I appreciate the help. I will probably create a fresh question about speed benchmarks/tuning as its a bit off topic from what this started as.
View ArticleSlow speeds and PortXmitWait - Any tips?
Hi Everyone,I have a new setup using 3 windows server hosts running WinOF with dual port Connectx-2 cards. These connect to two 4036 switches (one port to each switch and two links between switches)....
View ArticleClik here to view.
insmod mlx4_ib.ko giving crash on ARM board
Hi,I have connected Mellanox network card to PCie switch, on ARM processor board. To set IP address I am loading infini band drivers and mellanox Ethernet drivers . But is giving crash while inserting...
View ArticleClik here to view.
mckey program does not work
I used mckey to test RDMA CM multicast setup but it fails. Any help is appreciated.First of all, I started a machine to run as the receiver by "mckey -m 10.22.1.1". Then in another machine, I started...
View ArticleClik here to view.
MLNX-OS Download
Hello all, I apologize if this question has been answered previously. I've done quite a bit of searching and haven't had luck finding an answer. I have a MSX1036 and an MSX6036 switch that I'm using...
View ArticleClik here to view.
connectx-3 ethernet physical connection issue brocade vdx6940
I am having issues with ConnectX-3 dual port 40GB adapters connected to a Brocade VDX6940 switch. It appears to be a problem with the physical layer negotiation. Randomly when I reboot the server, a...
View ArticleMac address is assigned to wrong port of Mellanox
Hi all, I am trying to get sriov working with Mellanox. I have KVM hypervisor, and I have 2 port Mellanox cards. I am using port 1 of all cards.This is my output of lspci | grep Mel[root@hyp-1 hyp-1]#...
View ArticleRe: insmod mlx4_ib.ko giving crash on ARM board
Hi,We had a similar issue that was fixed recently. Please cherry-pick this commit and see if it works for you:https://patchwork.ozlabs.org/patch/618376/
View Article