Is the world run by badly updated Excel sheets?

I work in manufacturing and I see Excel everywhere. Here's my two cents:

Excel is universal: I can share an excel sheet with Tim from accounting, Jessica from purchasing, Jared from the warehouse, Kathy on the shop floor, John from sales, and Kim from a vendor, and I can expect them all to be able to read and make changes to it.

Another important point is it's longevity. I often find myself opening cost sheets created 30 years ago, and they work without issue in the current version of Excel. I expect they'll work 30 years from now too.

Is it the optimum tool for all jobs? No, far from it, but I've yet to encounter a situation where it turned out to be a poor choice. (Note: Obviously your database shouldn't be in Excel, but you better have an "export to Excel" feature if you expect it to be useful to others.)

"Badly designed" and "not updated" are not unique problems to Excel and I'd argue it's a whole lot easier for most people to fix an Excel sheet than even, say, a python script.

A good portion of businesses are in my experience. Thing is, it works. That's the real no/low code solution. When I make a spreadsheet, I wish for more structure and such, but I looks through the eyes of a developer. When I just use spreadsheets to get things done, I make a lot more progress.

We use Metabase at my work where we can pull a bunch of great data for dashboards and tables for non-developers and it's great. We can also build nice internal dashboards with all sorts of aggregated data. Still, a lot of the team uses just a mix of a few Metabase dashboards and spreadsheets with manually imported CSVs from services like Stripe. It works well for them. As a developer, I'd want to optimize and automate some of that away, but there's a good chance I'd spend more time than them.

I think you can look at spreadsheets as a form of tech debt. If the debt's interest rate is low (as it is in many situations where spreadsheets are used for long periods of time), then you might never need to get rid of it. If it has a high interest rate (not scaling or working well given the current work load), then it's time to move over to specialized software. Time wise, it's not unlikely that the interest rate is low and it won't need to be swapped out for a long time, or even ever.

While these spreadsheets could use some (or a lot) of work, they seem to be working just fine for the businesses for the most part. As developers, I think we like structure and efficiency, sometimes to a fault.

I won't take a stab at answering your question directly, but I'll pose something to think about:

It is the nature of many a developer to scoff at non-programmers running the world with supposedly nightmarish Excel sheets. And there is probably a lot of valid criticism.

But it works.

And any time I see Engineers trying to engineer things to their own taste, they just make things worse. You'll either make some portal with an inferior feature set or you'll reimplement a crappy Excel. Work with the Excel status quo, not against it. Don't look down at the problem from high atop a mountain, but rather admire just how incredibly effective Excel is as a swiss army programming tool that basically anyone in a business can use.

I am a software developer by trade and I’m writing mostly backend stuff for 22 years. Here’s my take on Excel.

Recently I went through the process of creating a non-trivial spreadsheet with bunch of tables, custom formulas and such. Things like costs with cost groups, person-days, person-availability, social costs, package calculations, profit and loss projections…

I’m mind blown. Excel is a powerful technology. It’s amazing how it tracks cell references in formulas and calculations automatically when rows/columns are added or removed.

Real-time visual feedback is unprecedented. You have 20 interconnected formulas? Awesome, change one value and everything recalculates in front of you.

It took me 22 years to arrive at the conclusion that some tasks are just simpler with a spreadsheet. Especially when I want to present the outcome to non-dev people. You go into a meeting and someone asks you “what happens if they travel so and so often instead of what you have now”. Or “what if we add these two things into that cost group and won’t offer x and y, can you also hide that for a second just to compare”. This is so easy to amend in Excel.

Good luck opening your editor and changing the code live because you haven’t planned for a third travel option.

Google Sheets is okay. I'm missing tables since I started using them in Excel. Named tables with totals row is just so handy.

I don't have a problem with spreadsheets personally. Complexity is bad; it's best if it gets the job done.

My issue is that they're very bad to use on mobile. It just feels off - like when the first smartphones arrived and you had to zoom in on every single website because they weren't optimized for small screens.

Additionally, you lose a lot of portability. My most recent example of this is discovering that macros don't work on mobile while trying to use a sheet that relied on them.

I would maybe even pay money for a better spreadsheet, or even DB frontend user experience on mobile.

Thing with Excel spreadsheets is that they're more than good enough for 95% of the scenarios they're used for. Everyone knows how to use Excel, and the backward compatibility of Excel sheets go back 30 years.

I agree. Have a look at the EU Spreadsheet Risks Group, especially the horror stories.

http://www.eusprig.org/

http://www.eusprig.org/horror-stories.htm

They get posted to HN from time to time. https://news.ycombinator.com/from?site=eusprig.org

I've suggested to orgs going through some sort of business or digital or whatever transformation to leverage excel/spreadsheets as prototypes for data input processes. For example, if there is a question about how to optimize some sort of process - where there might be a need to do some data input, etc.? Well, then start an experiment with a custom spreadsheet, then see how things work, get feedback from users, etc. If participants find that the spreadheet template never needs to change, AND there's value in turning it into something more robust, then you have data and empirical evidence to substantiate the building of a dedicated application for said process. If the experiment shows that too much needs to change, then a more comploex app could need to be built...or hey, guess what, just stick to the excel file. As a former developer, i myself occasionally get those feelings of snobbery about excel...but then i remind myself that the point is to get stuff done...and as long as the tech debt/whatever debt is acceptable by the business, then who cares if a clever excel file helps get the job done.

Lol yes, but don’t forget about MS Access.

This is from a few years ago, but I heard that JPM (at the time) had over 20k MS Access database on their file servers.

The company I worked for had over 12k. I was able to reduce the count by a good chunk by automating a lot of processes and moving workloads to sql queries that retrieved data from our data warehouses, but still a boatload.

The world is run on horribly formatted spreadsheets.

There are tools that are better for specific use cases, but Excel just can do the vast majority of use cases 95% good enough, so it's not worth changing tools.

On top of that, it's so common that everybody just knows how to use it. Don't need to train people on some super niche excel alternative when you can just use Excel. That benefit is probably understated.

Yes.

Badly designed is an opinion. It works for them and matches their thought process, ergo it's perfectly designed for their workflow.

We use Excel to design Test Plan ATC’s, you know Given When Then test cases.

We have to follow the chief architects template that contains all these fancy column counting macros. I absolutely hate this because no one has any idea where the source template is, causing us to copy and paste one from the previous jira ticket, which inevitably breaks.

If there was a gui program that let you fill out a test plan form, it would make my teams lives easier, but instead all I know of is Gherkin. Learning someone’s domain specific language to fill out test plans sounds egotistically asinine. Idc how powerful it is, I need something simple that normal non hobby developers can jump into and do their job. Sigh

I mean it's not all Excel, sometimes it's badly updated Smartsheets, badly updated Appian dashboards etc. Even Trello ends up as a dumpster fire. Only place I've seen it work is orgs with Google Sheets and Docs

During the worst of the pandemic, my boss saved the company by creating a new line of business. He did it all in Excel, building in weeks what would have taken months using our software development practices.

"Is the world run by badly updated Excel sheets?"

Yes. My experience is that my performance is evaluated via a spreadsheet that calibrated me against others on the team/dept. Spreadsheets are commonly used at most places. There are errors in some. However, I think the bigger errors are in the systems they represent rather than strictly in their implementation. Also, stale data is common since many people manually update stuff and aren't diligent about it.

I started my career with writing scripts to lift excel data into databases and process it into a workable state.

Cell validation is a relatively unknown feature and almost nobody can resist the urge to write comments where data should be and vice versa.

Other than that it's pretty neat software. I find myself using it to perform string manipulation on computers without the usual coreutils suite.

Yes, I can personally attest to this. I was guilty of having to maintain one such spreadsheet that was an absolute abomination to both programming as well as business processes.

See: https://news.ycombinator.com/item?id=33317169

I think it depends. Used to work in asset management. Seven investors in our group managing $4.0bn in equity and debt investments.

We had complex, up-to-date excel models for every investment. We’d update them whenever the company released financials or when there were significant changes in rates.

Relevant meme:

https://starecat.com/content/wp-content/uploads/the-whole-wo...

>What is your experience?

There likely is C Code generated by an Excel sheet on a plane you have flown on. It certainly is not as bad as it sounds, but also nobody (not an exaggeration, I talked to the guy who created it) knows exactly how it works.

I've noticed the migration from excel to smartsheet for many companies/tasks.

Coming out of BigCo consulting, pretty much every decision in our sales and solutioning pipeline had Excel somewhere beneath it, simply because it was the fastest way to do financial and project modeling for rapidly-changing requirements.

In general, I found that most clients have both "god spreadsheets" that seem to continually accrue data and control operations in often opaque ways, and nightmarishly-complex reporting spreadsheets generated by upstream systems that, if one thing goes wrong, explode into bits of #VALUE!s. It's neither good nor bad, just a fact of life. One client we engaged with -- one of the largest companies in the world -- had what we called "the ten-billion-dollar spreadsheet," because that's literally the amount of annual funds it controlled, even though it was completely invisible at the executive level. (Sad to say, but my lasting contribution to humanity is probably helping unlock an additional billion dollars of capitalist investment a year by moving a key part of that spreadsheet to a simple but far more accurate regression model.)

We had a delay on one of the projects because the client dropped the laptop running anonymizing job in excel for data our devs needed to test the app. It was big bank client. So yeah.

Yes.

The entire financial sector is a case of "turtles all the way down" excel. And no it doesn't get any better when the numbers start having more zeroes.

I heard from first hand experience that a portion of option dealing was run on MS Office VBA, the oldest code dated from 1998.

In short, yes.

The long answer, definitely. However, this is what two generations of white-collar workers are trained on.

yes. in the end everything runs on relational data.

It does have flavors though and sometimes comes in access or sql form too =)

Absolutely.