Best way to provide access to large data sets

I've been researching methods for exposing large data sets to users of my website, and came across the Atlassian Data Lake [1], and was curious - is this a thing that companies are doing now?

For personal projects and smaller scale non-dynamic data sets I've used Datasette [2], but haven't seen anyone providing anything at scale beyond either an API for accessing data or some sort of periodic file export.

[1] https://www.atlassian.com/platform/analytics/what-is-atlassian-data-lake

[2] https://datasette.io/