HACKER Q&A
📣 matesz

Why there is little talk about hardware acceleration in DBMSs?


I was looking for some hardware acceleration examples in database systems for a while [1] e.g. with FPGA, GPU or CAM but couldn't find much. Just today I've encountered this article from Alibaba [2] in which they state that according to them they developed the first hardware accelerated database system which is used in production:

"This marks the first time that hardware acceleration was applied to the storage engine of an OLTP database".

Doing hardware acceleration seems very logical in database systems and I'm wondering why any FAANG or similar company didn't do it before. Unless I'm mistaken?

[1] original HN question - https://news.ycombinator.com/item?id=25006725

[2] https://www.alibabacloud.com/blog/new-insights-into-x-engine-the-new-storage-engine-of-rds-for-mysql_596209


  👤 detaro Accepted Answer ✓
It's something lots of research groups experiment with, but apparently it doesn't pay off all that often.

Working with large memory datasets is something most accelerators aren't good at, because for most their memory access speed outside a limited local set of data is limited, so they can't easily outcompete traditional optimizations/scaling. It can make sense for some cases, e.g. there are some few GPU-accelerated databases, and you can spend lots of money on fancy FPGA accelerators, but the competition is always "why not just use a bigger/more traditional servers instead of exotic special things".