First, I would make sure you are using supported adapters, cables, and SFPs for both the Mellanox cards and the Dell switch. There are known issues when mixing certain types of SFPs and cables, and with certain kinds of Twinax.
If all the hardware is supported, please make sure the WinOF driver is installed and that the Mellanox port is configured to Ethernet (for VPI adapters).
I would then try directly connecting the NICs to eachother to bypass the Dell switch. Try running some IO using direct connect to verify your cards and cables are working. If this works --> either a compatibility issue with SFPs/cables and Dell switch, or the switch is configured incorrectly.