PCIe cards with their own NIC/Storage/OS?

Question

I'm looking for a PCIe card that runs its own linux os and can talk with the host system. Ideally, it would have its own NIC and NVME interface for storage. Does anything like this exist?

Avlin67 · Accepted Answer

what you want is a DPU. you can even run storage and network on pcie switches without a host CPU !https://www.servethehome.com/zfs-without-a-server-using-the-...

LargoLasskhyfv · Answer

Many larger FPGA-development kits. NVME for storage maybe not right now, but as an add-on via 'Mezzanine' card atop the dev-kit.
Basically some dual- to quad-core arm embedded into the fpga, various amounts of memory, sometimes expandable via SODIMM, and so on.
NOT cheap!
https://www.xilinx.com/products/design-tools/embedded-softwa... would be cybertrash from Xilinx as offer to run on the ARM-Cores.
edit: Former Altera, now Intel-FPGA isn't really better.
Prepare for a world of hurt, if doing anything with FPGAs, and their proprietary tool chains. Prepare for another world of hurt, if thinking they would be supported by open-source tool chains in a sufficient way, or that the the few alternative solutions which are fully supported by them would be enough.
I'll show myself out now...(rambling)

johndoe0815 · Answer

Perhaps not exactly what you are looking for, but there are Smart NIC PCIe cards running Linux (usually for data center applications), see e.g. these links:
https://datatracker.ietf.org/meeting/105/materials/slides-10...
https://legacy.netdevconf.info/0x14/pub/slides/39/Netdev%200...
https://smartnicssummit.com/proceeding_files/a0q5f000000lcIn...

LinuxBender · Answer

Does anything like this exist?They do but many are closed source. Dell and HP have their own ilo/drac cards that run Linux but they have since locked them down. A while back someone here linked an unrestricted ilo card, I will see if I still have the link.[Edit] I believe this [1] is what someone linked here a while ago. In the embeded video, he talks about some other more inclusive options.[1] - https://hackaday.com/2022/09/20/raspberry-pi-grants-remote-a...

floatinglotus · Answer

You want a smartNIC/DPU, and you should probably run SONiC on it! (DASH)

btgeekboy · Answer

Have a look at the CCR2004-1G-2XS-PCIe from MikroTik. It&rsquo;s a router on a PCIe card, with its own OS.What&rsquo;s your usecase? Might be able to get better recommendations if you can tell us what you&rsquo;re trying to do.

wmf · Answer

I recommend the SolidRun SolidNET or Mellanox Bluefield. FPGAs cost a fortune and Mikrotik looks janky.

PCIe cards with their own NIC/Storage/OS?

I'm looking for a PCIe card that runs its own linux os and can talk with the host system. Ideally, it would have its own NIC and NVME interface for storage. Does anything like this exist?

what you want is a DPU. you can even run storage and network on pcie switches without a host CPU !
https://www.servethehome.com/zfs-without-a-server-using-the-...

You want a smartNIC/DPU, and you should probably run SONiC on it! (DASH)

Have a look at the CCR2004-1G-2XS-PCIe from MikroTik. It’s a router on a PCIe card, with its own OS.
What’s your usecase? Might be able to get better recommendations if you can tell us what you’re trying to do.

I recommend the SolidRun SolidNET or Mellanox Bluefield. FPGAs cost a fortune and Mikrotik looks janky.