HACKER Q&A
📣 breck

Literature on Ultra-Wide Tables?


I'm currently building a model on the value of ultra-wide tables when doing observational science outside of laboratory settings.

By ultra-wide I mean tables with thousands of columns.

I've got my basic concepts down, now looking to read the literature on what has been written before.

Anyone have any pointers on people or terms to search?

  👤 nicolaskruchten Accepted Answer ✓
This isn't literature per se but I worked on a machine-learning database system (MLDB) which supported tables with millions of columns... We extended SQL's SELECT statement with some functionality to deal with querying by pattern or by exclusion: https://docs.mldb.ai/doc/#builtin/sql/SelectExpression.md.ht... and https://docs.mldb.ai/doc/#builtin/datasets/Datasets.md.html for some references
👤 thorin
You don't really say why you want to use ultra wide tables? I can tell you that they are quite painful to look at as a user, microsoft dynamics crm which I use has some tables with many hundreds of columns and it's painful to access and performance is bad in certain scenarios.

Are you writing huge amounts of data?

How are you reading the data?

I'm assuming many of the columns will be empty for most records?