However I have not worked with FPGAs yet so I wonder if I'm grossly overestimating their intent.
Its nice we have the handful of Lattice chips with open toolchains, but 9 times out of ten people still use other FPGAs for their reference designs unfortunately.
...
Snark aside, FPGAs currently on the market probably can't reprogram while computing (unless its a special hidden feature or something) its not something I've ever seen in documentation that talks about loading the chip configuration at power on time. However at a meta "many FPGA system" level, you could certainly have hardware hot-plug capable systems "offline" a chip unit, reload its code using a hardware management system that looks after this aspect (like using an Lights Out Management system to reboot a racked server in a datacenter)... then power it back up again with the new configuration and it gets added back into the system automatically due to the hot-plug system... so I guess the answer is "yes, but..."
Or say if FPGAs are faster, more compact, or power efficient then the parts of the network that changes slowest could be kept in them to complement the more dynamic parts. I think we still have quite a ways to go before these optimizations since running the network is much easier than training it.
https://en.wikipedia.org/wiki/General-purpose_computing_on_g...