Same behaviour using the version of IMB provided in the gcc version of HPC-X 1.5.370.

Runs fine on a single node, but hangs if scheduled across two nodes.

The LD_LIBRARY_PATH only contains pointers to HPC-X (line changes inserted by me below for reeadability

LD_LIBRARY_PATH=/lsf/9.1/linux2.6-glibc2.3-x86_64/lib:
/hpc/base/ctt/packages/hpcx/1.5.370/gcc/hcoll/lib:
/hpc/base/ctt/packages/hpcx/1.5.370/gcc/fca/lib:
/hpc/base/ctt/packages/hpcx/1.5.370/gcc/mxm/lib:
/hpc/base/ctt/packages/hpcx/1.5.370/gcc/ompi-v1.10/lib

Hello,

I'm trying to configure an infiniband network between two servers running Windows Server 2008 R2 using a Mellanox switch and copper cables. The ports all identify as active and LinkUp, subnet is up; but if I try to ibping any of the ports, it tells me that the port location could not be resolved. Am I missing something in the configuration? It seems like at this point I should be able to move data from one server to the other but I can't seem to figure out how to do so. Help?

Thanks!

Hello,

My environment:

- Ubuntu 14.04 with kernel linux-image-4.2.0-27-generic

- OFED 3.2-2.0.0.0

During installation of linux kernel > 4.2.0-27 DKMS reports errors.

Setting up linux-image-4.2.0-30-generic (4.2.0-30.36~14.04.1) ...

Running depmod.

update-initramfs: deferring update (hook will be called later)

Examining /etc/kernel/postinst.d.

run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.2.0-30-generic /boot/vmlinuz-4.2.0-30-generic

run-parts: executing /etc/kernel/postinst.d/dkms 4.2.0-30-generic /boot/vmlinuz-4.2.0-30-generic

Traceback (most recent call last):

  File "/usr/share/apport/package-hooks/dkms_packages.py", line 98, in <module>

    with apport.fileutils.make_report_file(report) as f:

AttributeError: 'module' object has no attribute 'make_report_file'

Traceback (most recent call last):

  File "/usr/share/apport/package-hooks/dkms_packages.py", line 98, in <module>

    with apport.fileutils.make_report_file(report) as f:

AttributeError: 'module' object has no attribute 'make_report_file'

Error! Bad return status for module build on kernel: 4.2.0-30-generic (x86_64)

Consult /var/lib/dkms/srp/1.5.5/build/make.log for more information.

Error! Bad return status for module build on kernel: 4.2.0-30-generic (x86_64)

Consult /var/lib/dkms/iser/1.7.0/build/make.log for more information.

 

: Unable to find an initial ram disk that I know how to handle.

Will not try to make an initrd.

 

When there is no OFED package in the system all goes fine.

cat /var/lib/dkms/srp/1.5.5/build/make.log

 

DKMS make.log for srp-1.5.5 for kernel 4.2.0-30-generic (x86_64)

Fri Mar 18 13:03:56 UTC 2016

makefile:17: /usr/src/ofa_kernel/default/configure.mk.kernel: No such file or directory

grep: srp.spec: No such file or directory

make: *** No rule to make target `/usr/src/ofa_kernel/default/configure.mk.kernel'.  Stop.

makefile:17: /usr/src/ofa_kernel/default/configure.mk.kernel: No such file or directory

grep: srp.spec: No such file or directory

make: *** No rule to make target `/usr/src/ofa_kernel/default/configure.mk.kernel'.  Stop.

 

cat /var/lib/dkms/iser/1.7.0/build/make.log

 

DKMS make.log for iser-1.7.0 for kernel 4.2.0-30-generic (x86_64)

Fri Mar 18 13:03:56 UTC 2016

makefile:48: /usr/src/ofa_kernel/default/configure.mk.kernel: No such file or directory

grep: iser.spec: No such file or directory

make: *** No rule to make target `/usr/src/ofa_kernel/default/configure.mk.kernel'.  Stop.

makefile:48: /usr/src/ofa_kernel/default/configure.mk.kernel: No such file or directory

grep: iser.spec: No such file or directory

 

Prior to linux image 4.2.0-27 updates was successful too.

When I am using my connect-ib card for rdma communications, I had some seg fault with my application. I was able to get some trace information using fwtrace as follows:

[503:12:30.862093402]  I2       rxs_handle_tpt_syndrome: gvmi=0x0000, tag_type=0x1, context_number=0x01f039, rxw_opcode=0x02, psn=0x9189ac

[503:12:32.737484478]  I2       starting RXS-tpt/inv error handler for slice: 0x8

[503:12:32.737527108]  I2       rxs_handle_tpt_syndrome: gvmi=0x0000, tag_type=0x1, context_number=0x01f039, rxw_opcode=0x02, psn=0xd0b44c

[503:12:32.739025278]  I4       handle_any_sx_error: ec_id=0x00, gvmi=0x0, req_res_=0, qpi=0x1f039, error_is_send=1

[503:12:32.739035670]  I4       handle_any_sx_error: ec_id=0x00, gvmi=0x0, req_res_=0, qpi=0x1f039, synd=a956

Does this mean something is wrong with my card?

Hi,

I guess the trace is from mlxtrace tool. It is pretty hard to guess what went wrong since we don't have your piece of code and on call you are getting a segfault. My guess is that these error might related to some issue in MLX5_EVENT_TYPE_CQ_ERROR flow. I also guess dmesg will print something else that might be more related.

Hi,

Can you share "vstat" from PS from both of your machines, and ibping syntax and error?

Hello,

Have you tried seeing that you cannot actually allocate DMA of the requires size with the default settings ? The parameters in mlx4 (ConnectX-3Pro and below) were configurable as the default values on some occasions were too low for some users. The mlx5 driver should have a much larger default pool.

Hello, into my infiniband cluster infrastructure i have a Intel IB switch (Intel True Scale Edge Switch 12300), i bough a new switch Mellanox voltaire 4036 and connected to the intel switch,

actually the intel switch is set as master with priority 0 but if i enable the 4036 switch subnet manager it do not became slave but master....

Is it normal or simply is not possible to activate the subnet manager in 2 different switch brands ?

Thanks

Hi

Please notice that under WinOF 5.1 driver and also ofed_3_2_win7x64 under the headers are located under

: libibmad ,  libibumad

ib_mad.h

ibumad.h

http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_driver

Hi There,

Our WinOFED driver for Windows comes with added support for various cmdlets from Powershell by using the default installation

Hi,

Sure, thanks!

vstat from machine running subnet manager:

vstat from other machine:

ibping error from machine running subnet manager:

ibping error from other machine:

ConnectX4 doesn't have this limitation

I found a cool feature on-demand paging in the user manual. This features seems to be a good fit for my application. However, while I am evaluating the performance of on-demand paging, I see there is a unexpected behavior. For example, if I just issue rdma operation from same buffers in thousands of iterations. I see there are some page faults reported by the hca even after I prefetch these buffers in the beginning. That's why I was looking at the trace information to see if my card is ok or not. Do you have any idea why this is happening?

I have successfully built and installed ofed 3.2-2.0.0.0 on an Oracle Enterprise Linux 6.7 system. I tested this ofed installation in ethernet emulation mode with iperf3  and a ConnectX-4 port looping back to the other ConnectX-4 port on the same system. I disabled the internal loopback and configured nat to force the data to be issued over the physical IB cable.

I used the same build and install sequence on a simulator system with a custom kernel based on Oracle Enterprise Linux 7.1. This system is failing "mst start"

I used MLNX_OFED_LINUX-3.2-2.0.0.0-oel7.1-x86_64.tgz and the mlnx_add_kernel_support.sh completed without issue.

Here is the uname info for the custom kernel the system is running:

# uname -a

Linux  4.1.12-32.el7uek-axnp.debug.070000.009400 #1 SMP Fri Jan 29 12:18:44 PST 2016 x86_64 x86_64 x86_64 GNU/Linux

The extra modules directory is populated with what seems like the correct directories and files:

# ls /usr/lib/modules/4.1.12-32.el7uek-axnp.debug.070000.009400/extra

dmadriver_api_mod.ko  knem             pdgCommon.ko     slbmgr_api_mod.ko

i40e.ko               ksimod.ko        pdgPm8018.ko     slbmgrmod.ko

iser                  mlnx-ofa_kernel  pdgQlSan.ko      srp

kernel-mft            nvdimm.ko        psg_services.ko  tdsmod.ko

The mst modules are present:

# ls ./usr/lib/modules/4.1.12-32.el7uek-axnp.debug.070000.009400/extra/kernel-mft/

mst_pciconf.ko  mst_pci.ko

However "mst start" fails for "version magic":

# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI modulemodprobe: ERROR: could not insert 'mst_pci': Exec format error

- Failure: 1

Loading MST PCI configuration modulemodprobe: ERROR: could not insert 'mst_pciconf': Exec format error

- Failure: 1

Create devices

mst_pci driver not found

Unloading MST PCI module (unused) - Success

Unloading MST PCI configuration module (unused) - Success

From dmesg:

[ 2264.895260] mst_pci: version magic '4.1.12-32.el7uek-axnp.debug SMP mod_unload ' should be '4.1.12-32.el7uek-axnp.debug.070000.009400 SMP mod_unload '

[ 2265.048972] mst_pciconf: version magic '4.1.12-32.el7uek-axnp.debug SMP mod_unload ' should be '4.1.12-32.el7uek-axnp.debug.070000.009400 SMP mod_unload '

It looks like the .070000.009400 is being truncated off the "version magic" in the mst_pci and mst_pciconf modules.

Is there a string length limit for the kernel versioning in mlnx_add_kernel_support.sh?

I am also trying to boot via PXE on Infiniband with ConnectIB card and I have found out, there are available Driver Update disc for Ethernet on FlexBoot site: DUD MLNX_EN for RHEL7.1 Driver

Is there anything similar for PXE over Infiniband  on CentOS 7.2?

Thank you for your answer

I have a ISR 9024S-M which can not take admin password. Looks like the partition is fed up by log file. Is there any way to free the partition or how to factory reset it? Please help!

Here is the screen dump from RS232 console:

(none) login: admin

login:

(none) login:

PPCBoot 1.1.6 (Nov 16 2004 - 08:21:51)

CPU:   IBM PowerPC 440 Rev. C

Board: Artesyn Technologies PM/PPC-440gp

       Artesyn Monitor Version 1.5.0

        VCO: 800 MHz

        CPU: 400 MHz

        PLB: 133 MHz

        OPB: 66 MHz

        EPB: 66 MHz

I2C:   ready

DRAM:  ECC Init ....Done 128 MB

FLASH: 64 MB

Initializing board as Monarch....

Scanning for PCI devices....

Scanning PCI devices on bus 0

LOCALE  VEND DEV  REV  BASE0    BASE1    BASE2    BASE3    BASE4    BASE5

------- ---- ---- --- -------- -------- -------- -------- -------- --------

00,0d,0 10b5 9056 ba  80000000 b0800000 80200000 80400000 ........ ........

In:    serial

Out:   serial

Err:   serial

Net:   EMAC0, EMAC1

BEDBUG:ready

Hit any key to stop autoboot:  0

=> <INTERRUPT>

=> <INTERRUPT>

=> <INTERRUPT>

=> <INTERRUPT>

=> <INTERRUPT>

=> <INTERRUPT>

=> <INTERRUPT>

=>  nmap -v -sn 192.168.2.0/24 -e ib1

Usage:

nm      - memory modify (constant address)

=> imm 53 90.1

00000090:I2c read: failed 4

Error reading the chip,

? 01

Error writing the chip.

00000091:I2c read: failed 4

Error reading the chip,

? 53

Error writing the chip.

00000092:I2c read: failed 4

Error reading the chip,

? 51

Error writing the chip.

00000093:I2c read: failed 4

Error reading the chip,

? 50

Error writing the chip.

00000094:I2c read: failed 4

Error reading the chip,

? => help

as      - assemble memory

autoscr - run script from memory

base    - print or set address offset

bdinfo  - print Board Info structure

bootelf - Boot from an ELF image in memory

bootm   - boot application image from memory

bootp   - boot image via network using BootP/TFTP protocol

bootvx  - Boot vxWorks from an ELF image

bootd   - boot default, i.e., run 'bootcmd'

break   - set or clear a breakpoint

cmp     - memory compare

coninfo - print console devices and informations

continue- continue from a breakpoint

cp      - memory copy

crc32   - checksum calculation

date    - get/set/reset date & time

dhcp    - invoke DHCP client to obtain IP/boot params

ds      - disassemble memory

echo    - echo args to console

erase   - erase FLASH memory

flinfo  - print FLASH memory information

getdcr  - Get an IBM PPC 4xx DCR's value

go      - start application at address 'addr'

help    - print online help

imd     - i2c memory display

imm     - i2c memory modify (auto-incrementing)

inm     - memory modify (constant address)

imw     - memory write (fill)

icrc32  - checksum calculation

iprobe  - probe to discover valid I2C chip addresses

iloop   - infinite loop on address range

isdram  - print SDRAM configuration information

iminfo  - print header information for application image

irqinfo - print information about IRQs

loadb   - load binary file over serial line (kermit mode)

loads   - load S-Record file over serial line

loop    - infinite loop on address range

md      - memory display

mm      - memory modify (auto-incrementing)

mtest   - simple RAM test

mw      - memory write (fill)

next    - single step execution, stepping over subroutines.

nm      - memory modify (constant address)

printenv- print environment variables

protect - enable or disable FLASH write protection

rarpboot- boot image via network using RARP/TFTP protocol

rdump   - Show registers.

reset   - Perform RESET of the CPU

run     - run commands in an environment variable

saveenv - save environment variables to persistent storage

setdcr  - Set an IBM PPC 4xx DCR's value

setenv  - set environment variables

sleep   - delay execution for some time

where   - Print the running stack.

step    - single step execution.

tftpboot- boot image via network using TFTP protocol

               and env variables ipaddr and serverip

version - print monitor version

?       - alias for 'help'

pci_enum - Enumerates and displays PCI Bus Enumeration Info

pci_info - Displays PCI Bus Enumeration Info

prog_i2c - Programs default i2c bootstrap settings

moninit - Copies PPCBoot monitor into Soldered Flash

monship - Erases soldered flash, inits environment variables

initenv - Initializes Board Environment variables

um      - destructive memory test

=> imm

Usage:

imm     - i2c memory modify (auto-incrementing)

=> where

Not at a breakpoint

=> version

PPCBoot 1.1.6 (Nov 16 2004 - 08:21:51)

=> coninfo

List of available devices:

serial   80000003 SIO stdin stdout stderr

=> dhcp

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 0

BOOTP broadcast 1

BOOTP broadcast 2

BOOTP broadcast 3

BOOTP broadcast 4

BOOTP broadcast 5

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 6

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 7

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 8

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 9

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 10

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 11

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 12

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 13

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 14

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 15

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Abort

BOOTP broadcast 16

eth_send -1

Retry count exceeded; starting again

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 17

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 18

Retry count exceeded; starting again

Trying EMAC0

Waiting for PHY auto negotiation to complete... TIMEOUT !

Trying EMAC1

ENET Speed is 100 Mbps - FULL duplex connection

BOOTP broadcast 19

Abort

=> help

as      - assemble memory

autoscr - run script from memory

base    - print or set address offset

bdinfo  - print Board Info structure

bootelf - Boot from an ELF image in memory

bootm   - boot application image from memory

bootp   - boot image via network using BootP/TFTP protocol

bootvx  - Boot vxWorks from an ELF image

bootd   - boot default, i.e., run 'bootcmd'

break   - set or clear a breakpoint

cmp     - memory compare

coninfo - print console devices and informations

continue- continue from a breakpoint

cp      - memory copy

crc32   - checksum calculation

date    - get/set/reset date & time

dhcp    - invoke DHCP client to obtain IP/boot params

ds      - disassemble memory

echo    - echo args to console

erase   - erase FLASH memory

flinfo  - print FLASH memory information

getdcr  - Get an IBM PPC 4xx DCR's value

go      - start application at address 'addr'

help    - print online help

imd     - i2c memory display

imm     - i2c memory modify (auto-incrementing)

inm     - memory modify (constant address)

imw     - memory write (fill)

icrc32  - checksum calculation

iprobe  - probe to discover valid I2C chip addresses

iloop   - infinite loop on address range

isdram  - print SDRAM configuration information

iminfo  - print header information for application image

irqinfo - print information about IRQs

loadb   - load binary file over serial line (kermit mode)

loads   - load S-Record file over serial line

loop    - infinite loop on address range

md      - memory display

mm      - memory modify (auto-incrementing)

mtest   - simple RAM test

mw      - memory write (fill)

next    - single step execution, stepping over subroutines.

nm      - memory modify (constant address)

printenv- print environment variables

protect - enable or disable FLASH write protection

rarpboot- boot image via network using RARP/TFTP protocol

rdump   - Show registers.

reset   - Perform RESET of the CPU

run     - run commands in an environment variable

saveenv - save environment variables to persistent storage

setdcr  - Set an IBM PPC 4xx DCR's value

setenv  - set environment variables

sleep   - delay execution for some time

where   - Print the running stack.

step    - single step execution.

tftpboot- boot image via network using TFTP protocol

               and env variables ipaddr and serverip

version - print monitor version

?       - alias for 'help'

pci_enum - Enumerates and displays PCI Bus Enumeration Info

pci_info - Displays PCI Bus Enumeration Info

prog_i2c - Programs default i2c bootstrap settings

moninit - Copies PPCBoot monitor into Soldered Flash

monship - Erases soldered flash, inits environment variables

initenv - Initializes Board Environment variables

um      - destructive memory test

=> ds

Usage:

ds      - disassemble memory

=> as

Usage:

as      - assemble memory

=> help

as      - assemble memory

autoscr - run script from memory

base    - print or set address offset

bdinfo  - print Board Info structure

bootelf - Boot from an ELF image in memory

bootm   - boot application image from memory

bootp   - boot image via network using BootP/TFTP protocol

bootvx  - Boot vxWorks from an ELF image

bootd   - boot default, i.e., run 'bootcmd'

break   - set or clear a breakpoint

cmp     - memory compare

coninfo - print console devices and informations

continue- continue from a breakpoint

cp      - memory copy

crc32   - checksum calculation

date    - get/set/reset date & time

dhcp    - invoke DHCP client to obtain IP/boot params

ds      - disassemble memory

echo    - echo args to console

erase   - erase FLASH memory

flinfo  - print FLASH memory information

getdcr  - Get an IBM PPC 4xx DCR's value

go      - start application at address 'addr'

help    - print online help

imd     - i2c memory display

imm     - i2c memory modify (auto-incrementing)

inm     - memory modify (constant address)

imw     - memory write (fill)

icrc32  - checksum calculation

iprobe  - probe to discover valid I2C chip addresses

iloop   - infinite loop on address range

isdram  - print SDRAM configuration information

iminfo  - print header information for application image

irqinfo - print information about IRQs

loadb   - load binary file over serial line (kermit mode)

loads   - load S-Record file over serial line

loop    - infinite loop on address range

md      - memory display

mm      - memory modify (auto-incrementing)

mtest   - simple RAM test

mw      - memory write (fill)

next    - single step execution, stepping over subroutines.

nm      - memory modify (constant address)

printenv- print environment variables

protect - enable or disable FLASH write protection

rarpboot- boot image via network using RARP/TFTP protocol

rdump   - Show registers.

reset   - Perform RESET of the CPU

run     - run commands in an environment variable

saveenv - save environment variables to persistent storage

setdcr  - Set an IBM PPC 4xx DCR's value

setenv  - set environment variables

sleep   - delay execution for some time

where   - Print the running stack.

step    - single step execution.

tftpboot- boot image via network using TFTP protocol

               and env variables ipaddr and serverip

version - print monitor version

?       - alias for 'help'

pci_enum - Enumerates and displays PCI Bus Enumeration Info

pci_info - Displays PCI Bus Enumeration Info

prog_i2c - Programs default i2c bootstrap settings

moninit - Copies PPCBoot monitor into Soldered Flash

monship - Erases soldered flash, inits environment variables

initenv - Initializes Board Environment variables

um      - destructive memory test

=> flinfo

Bank # 1: CFI conformant FLASH (16 x 16)  Size: 16 MB in 128 Sectors

Erase timeout 4096 ms, write timeout 0 ms, buffer write timeout 1024 ms, buffer

size 32

  Sector Start Addresses:

D0000000      D0020000      D0040000      D0060000      D0080000

D00A0000      D00C0000      D00E0000      D0100000      D0120000

D0140000      D0160000      D0180000      D01A0000      D01C0000

D01E0000      D0200000      D0220000      D0240000      D0260000

D0280000      D02A0000      D02C0000      D02E0000      D0300000

D0320000      D0340000      D0360000      D0380000      D03A0000

D03C0000      D03E0000      D0400000      D0420000      D0440000

D0460000      D0480000      D04A0000      D04C0000      D04E0000

D0500000      D0520000      D0540000      D0560000      D0580000

D05A0000      D05C0000      D05E0000      D0600000      D0620000

D0640000      D0660000      D0680000      D06A0000      D06C0000

D06E0000      D0700000      D0720000      D0740000      D0760000

D0780000      D07A0000      D07C0000      D07E0000      D0800000

D0820000      D0840000      D0860000      D0880000      D08A0000

D08C0000      D08E0000      D0900000      D0920000      D0940000

D0960000      D0980000      D09A0000      D09C0000      D09E0000

D0A00000      D0A20000      D0A40000      D0A60000      D0A80000

D0AA0000      D0AC0000      D0AE0000      D0B00000      D0B20000

D0B40000      D0B60000      D0B80000      D0BA0000      D0BC0000

D0BE0000      D0C00000      D0C20000      D0C40000      D0C60000

D0C80000      D0CA0000      D0CC0000      D0CE0000      D0D00000

D0D20000      D0D40000      D0D60000      D0D80000      D0DA0000

D0DC0000      D0DE0000      D0E00000      D0E20000      D0E40000

D0E60000      D0E80000      D0EA0000      D0EC0000      D0EE0000

D0F00000      D0F20000      D0F40000      D0F60000      D0F80000

D0FA0000      D0FC0000      D0FE0000

Bank # 2: CFI conformant FLASH (16 x 16)  Size: 16 MB in 128 Sectors

Erase timeout 4096 ms, write timeout 0 ms, buffer write timeout 1024 ms, buffer

size 32

  Sector Start Addresses:

D1000000      D1020000      D1040000      D1060000      D1080000

D10A0000      D10C0000      D10E0000      D1100000      D1120000

D1140000      D1160000      D1180000      D11A0000      D11C0000

D11E0000      D1200000      D1220000      D1240000      D1260000

D1280000      D12A0000      D12C0000      D12E0000      D1300000

D1320000      D1340000      D1360000      D1380000      D13A0000

D13C0000      D13E0000      D1400000      D1420000      D1440000

D1460000      D1480000      D14A0000      D14C0000      D14E0000

D1500000      D1520000      D1540000      D1560000      D1580000

D15A0000      D15C0000      D15E0000      D1600000      D1620000

D1640000      D1660000      D1680000      D16A0000      D16C0000

D16E0000      D1700000      D1720000      D1740000      D1760000

D1780000      D17A0000      D17C0000      D17E0000      D1800000

D1820000      D1840000      D1860000      D1880000      D18A0000

D18C0000      D18E0000      D1900000      D1920000      D1940000

D1960000      D1980000      D19A0000      D19C0000      D19E0000

D1A00000      D1A20000      D1A40000      D1A60000      D1A80000

D1AA0000      D1AC0000      D1AE0000      D1B00000      D1B20000

D1B40000      D1B60000      D1B80000      D1BA0000      D1BC0000

D1BE0000      D1C00000      D1C20000      D1C40000      D1C60000

D1C80000      D1CA0000      D1CC0000      D1CE0000      D1D00000

D1D20000      D1D40000      D1D60000      D1D80000      D1DA0000

D1DC0000      D1DE0000      D1E00000      D1E20000      D1E40000

D1E60000      D1E80000      D1EA0000      D1EC0000      D1EE0000

D1F00000      D1F20000      D1F40000      D1F60000      D1F80000

D1FA0000      D1FC0000      D1FE0000

Bank # 3: CFI conformant FLASH (16 x 16)  Size: 16 MB in 128 Sectors

Erase timeout 4096 ms, write timeout 0 ms, buffer write timeout 1024 ms, buffer

size 32

  Sector Start Addresses:

D2000000      D2020000      D2040000      D2060000      D2080000

D20A0000      D20C0000      D20E0000      D2100000      D2120000

D2140000      D2160000      D2180000      D21A0000      D21C0000

D21E0000      D2200000      D2220000      D2240000      D2260000

D2280000      D22A0000      D22C0000      D22E0000      D2300000

D2320000      D2340000      D2360000      D2380000      D23A0000

D23C0000      D23E0000      D2400000      D2420000      D2440000

D2460000      D2480000      D24A0000      D24C0000      D24E0000

D2500000      D2520000      D2540000      D2560000      D2580000

D25A0000      D25C0000      D25E0000      D2600000      D2620000

D2640000      D2660000      D2680000      D26A0000      D26C0000

D26E0000      D2700000      D2720000      D2740000      D2760000

D2780000      D27A0000      D27C0000      D27E0000      D2800000

D2820000      D2840000      D2860000      D2880000      D28A0000

D28C0000      D28E0000      D2900000      D2920000      D2940000

D2960000      D2980000      D29A0000      D29C0000      D29E0000

D2A00000      D2A20000      D2A40000      D2A60000      D2A80000

D2AA0000      D2AC0000      D2AE0000      D2B00000      D2B20000

D2B40000      D2B60000      D2B80000      D2BA0000      D2BC0000

D2BE0000      D2C00000      D2C20000      D2C40000      D2C60000

D2C80000      D2CA0000      D2CC0000      D2CE0000      D2D00000

D2D20000      D2D40000      D2D60000      D2D80000      D2DA0000

D2DC0000      D2DE0000      D2E00000      D2E20000      D2E40000

D2E60000      D2E80000      D2EA0000      D2EC0000      D2EE0000

D2F00000      D2F20000      D2F40000      D2F60000      D2F80000

D2FA0000      D2FC0000      D2FE0000

Bank # 4: CFI conformant FLASH (16 x 16)  Size: 16 MB in 128 Sectors

Erase timeout 4096 ms, write timeout 0 ms, buffer write timeout 1024 ms, buffer

size 32

  Sector Start Addresses:

D3000000      D3020000      D3040000      D3060000      D3080000

D30A0000      D30C0000      D30E0000      D3100000      D3120000

D3140000      D3160000      D3180000      D31A0000      D31C0000

D31E0000      D3200000      D3220000      D3240000      D3260000

D3280000      D32A0000      D32C0000      D32E0000      D3300000

D3320000      D3340000      D3360000      D3380000      D33A0000

D33C0000      D33E0000      D3400000      D3420000      D3440000

D3460000      D3480000      D34A0000      D34C0000      D34E0000

D3500000      D3520000      D3540000      D3560000      D3580000

D35A0000      D35C0000      D35E0000      D3600000      D3620000

D3640000      D3660000      D3680000      D36A0000      D36C0000

D36E0000      D3700000      D3720000      D3740000      D3760000

D3780000      D37A0000      D37C0000      D37E0000      D3800000

D3820000      D3840000      D3860000      D3880000      D38A0000

D38C0000      D38E0000      D3900000      D3920000      D3940000

D3960000      D3980000      D39A0000      D39C0000      D39E0000

D3A00000      D3A20000      D3A40000      D3A60000      D3A80000

D3AA0000      D3AC0000      D3AE0000      D3B00000      D3B20000

D3B40000      D3B60000      D3B80000      D3BA0000      D3BC0000

D3BE0000      D3C00000      D3C20000      D3C40000      D3C60000

D3C80000      D3CA0000      D3CC0000      D3CE0000      D3D00000

D3D20000      D3D40000      D3D60000      D3D80000      D3DA0000

D3DC0000      D3DE0000      D3E00000      D3E20000      D3E40000

D3E60000      D3E80000      D3EA0000      D3EC0000      D3EE0000

D3F00000      D3F20000      D3F40000      D3F60000      D3F80000 (RO)

D3FA0000 (RO) D3FC0000 (RO) D3FE0000 (RO)

=>

PPCBoot 1.1.6 (Nov 16 2004 - 08:21:51)

CPU:   IBM PowerPC 440 Rev. C

Board: Artesyn Technologies PM/PPC-440gp

       Artesyn Monitor Version 1.5.0

        VCO: 800 MHz

        CPU: 400 MHz

        PLB: 133 MHz

        OPB: 66 MHz

        EPB: 66 MHz

I2C:   ready

DRAM:  ECC Init ....Done 128 MB

FLASH: 64 MB

Initializing board as Monarch....

Scanning for PCI devices....

Scanning PCI devices on bus 0

LOCALE  VEND DEV  REV  BASE0    BASE1    BASE2    BASE3    BASE4    BASE5

------- ---- ---- --- -------- -------- -------- -------- -------- --------

00,0d,0 10b5 9056 ba  80000000 b0800000 80200000 80400000 ........ ........

In:    serial

Out:   serial

Err:   serial

Net:   EMAC0, EMAC1

BEDBUG:ready

Hit any key to stop autoboot:  0

booting secondary, press ^c to abort

booting...

## Booting image at d0a00000 ...

   Image Name:   Linux Multiboot-Image 2.4.21-pmp

   Created:      2006-08-24  11:41:03 UTC

   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)

   Data Size:    20363540 Bytes = 19886 kB = 19 MB

   Load Address: 00000000

   Entry Point:  00000000

   Contents:

   Image 0:   780651 Bytes = 762 kB = 0 MB

   Image 1: 19582876 Bytes = 19123 kB = 18 MB

   Verifying Checksum ... OK

   Uncompressing Multi-File Image ... OK

   Loading Ramdisk to 06cb1000, end 07f5df9c ... OK

Linux version 2.4.21-pmppc440-V02 (root@murzim) (gcc version 2.95.4 20010319 (pr

erelease/franzo/20011204)) #1 Thu Aug 24 14:39:17 IDT 2006

Artesyn PM/PPC440 Copyright 2003, Artesyn Communication Products, LLC

On node 0 totalpages: 32768

zone(0): 4096 pages.

zone(1): 28672 pages.

zone(2): 0 pages.

Kernel command line: console=ttyS0,38400 root=/dev/ram0 rw ip=192.168.0.101:::25

5.255.255.0:0.0.0.0:eth1:none

Calibrating delay loop... 599.65 BogoMIPS

Memory: 108300k available (1336k kernel code, 572k data, 60k init, 0k highmem)

Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)

Inode cache hash table entries: 8192 (order: 4, 65536 bytes)

Mount cache hash table entries: 512 (order: 0, 4096 bytes)

Buffer-cache hash table entries: 8192 (order: 3, 32768 bytes)

Page-cache hash table entries: 32768 (order: 5, 131072 bytes)

POSIX conformance testing by UNIFIX

PCI: Probing PCI hardware

Linux NET4.0 for Linux 2.4

Based upon Swansea University Computer Society NET3.039

Initializing RT netlink socket

Starting kswapd

Journalled Block Device driver loaded

devfs: v1.12c (20020818) Richard Gooch (rgooch@atnf.csiro.au)

devfs: boot_options: 0x0

JFFS2 version 2.1. (C) 2001 Red Hat, Inc., designed by Axis Communications AB.

i2c-core.o: i2c core module

i2c-dev.o: i2c /dev entries driver module

i2c-core.o: driver i2c-dev dummy driver registered.

i2c-algo-bit.o: i2c bit algorithm module

i2c-algo-pcf.o: i2c pcf8584 algorithm module

i2c-proc.o version 2.6.1 (20010825)

ST M41T00 Real-Time Clock Driver 0.01 03 Jan 2003

i2c-core.o: driver m41t00 registered.

pty: 256 Unix98 ptys configured

Serial driver version 5.05c (2001-07-08) with MANY_PORTS SHARE_IRQ SERIAL_PCI en

abled

ttyS00 at 0xfdeec200 (irq = 0) is a 16550A

ttyS01 at 0xfdeeb300 (irq = 1) is a 16550A

IBM gpio driver version 07.25.02

GPIO #0 at 0xc9070700

RAMDISK driver initialized: 16 RAM disks of 65536K size 1024 blocksize

loop: loaded (max 8 devices)

init_ppc405_enet(): vtr: ENET driver v1.1

Zmii bridge in RMII mode

eth0: Phy @ 0x4, type BCM5221 (0x004061e4)

Reset ethernet interfaces

Zmii bridge in RMII mode

eth1: Phy @ 0x5, type BCM5221 (0x004061e4)

Reset ethernet interfaces

Using buffer write method

Using buffer write method

Using buffer write method

Using buffer write method

Concatenating MTD devices:

(0): "Flash Bank"

(1): "Flash Bank"

(2): "Flash Bank"

(3): "Flash Bank"

into device "All Flash Banks"

Creating 3 MTD partitions on "All Flash Banks":

0x03500000-0x03f00000 : "Primary Kernel"

0x00a00000-0x03500000 : "Secondary Kernel"

0x00000000-0x00a00000 : "Jffs"

IBM IIC driver

i2c-dev.o: Registered 'IBM IIC adapter' as minor 0

i2c-core.o: client [m41t00] registered to adapter [IBM IIC adapter](pos. 0).

i2c-core.o: adapter IBM IIC adapter registered as adapter 0.

i2c-dev.o: Registered 'IBM IIC adapter' as minor 1

i2c-core.o: adapter IBM IIC adapter registered as adapter 1.

pci_hotplug: PCI Hot Plug PCI Core version: 0.5

NET4: Linux TCP/IP 1.0 for NET4.0

IP Protocols: ICMP, UDP, TCP

IP: routing cache hash table of 1024 buckets, 8Kbytes

TCP: Hash tables configured (established 8192 bind 8192)

NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.

Voltaire PPC440 I2C EEPROM Driver 0.02 04 Mar 2004

i2c-core.o: driver ppc440_i2c_eeprom registered.

i2c-core.o: client [ppc440_i2c_eeprom] registered to adapter [IBM IIC adapter](p

os. 1).

klr_init_module: p_klr_buffer = c6000000

klr_init_module: ------------------------------------- memory overrun

Kernel logger buffer is at:0xc6000000, size: 4083, initializing new buffer

RAMDISK: Compressed image found at block 0

Freeing initrd memory: 19123k freed

EXT2-fs warning: checktime reached, running e2fsck is recommended

VFS: Mounted root (ext2 filesystem).

Freeing unused kernel memory: 60k init

init started:  BusyBox v0.60.5 (2006.05.17-12:13+0000) multi-call binary

Mounting proc file system.

ibsw-init.sh start...

Sat Mar 12 20:28:01 UTC 2016

INSTALL FLAG  0

starting syslogd & klogd ...

Starting ISR:                   /usr/voltaire/scripts/ibsw-init.sh: cannot creat

e /mnt/jffs/nvigor-Mar_12_2016-20_28_01.log: No space left on device

(none) login:

Hi,

I recommend opening a ticket to support@mellanox.com and provide a code snippet of what you're trying to do so it will go into the appropriate resource.

the first 15 ports are the external ports of the switch blade.

it's not editable

A couple of comments on this:

1. SM Priority 0 is lowest priority and 15 is highest. You might want to set Intel switch SM to be higher SM priority than Mellanox/Voltaire switch SM priority.

2. Best practice is not to run different SM flavors in same subnet. There are many SM behaviors that are vendor specific and failover between the different flavors may be problematic.

Hope this helps,

-- Hal