HACKER Q&A
📣 akkad33

How do you keep big software systems bug free


We work on a highly complex domain specific software product in my company. Ours is a growing platform used for mathematic computations with many interlinked modules and new features being developed every day. Over the last months we have developed many new features and old code is used and modified in the process. We have found it hard to implement a failsafe system that ensures system correctness. It has been hard to ensure new features developed are correct and bug free and to ensure we do not break older features. The systems we have in place for catching bugs are human overseen and automatic end to end tests, integration tests in the code and unit tests. Unfortunately because of the sheer explosion of edge cases it is impossible to have a test for every possible edge case. Furthermore I feel our specs are transmitted in an ad-hoc manner through Jira tickets and it is easy to miss something or have some important detail get lost in the detail. I have been thinking of the feasibility of maintaining specs in a formal software language or using systems like Prolog to automatically generate edge cases to ease testing. I have no clue if this is feasible. How are such problems commonly handled in your field?

  👤 RetiredRichard Accepted Answer ✓
"Unfortunately because of the sheer explosion of edge cases it is impossible to have a test for every possible edge case"

You just can't keep any system bug free, the only thing you can do is improve your detection systems, codebase, and documentation and over time bugs will pop up less and less

👤 mrkeen
Everyone's tips will be different, but I'll throw out a few:

* Make illegal states unrepresentable

* pure, static functions wherever possible

* use test generators (a la quick check)

* if you can't unit test important code, try refactoring until you can, rather than immediately reaching for integration tests and mocks.

I don't think you'll be able to model your system well enough to be able to run it through a formal language (I'm thinking about TLA+). But Leslie Lamport did say something about "the algorithm is not the code". That is, if you have any tricky algorithms that you invented for this project, you can formally prove those algorithms in TLA+ and then translate them to code.

👤 chrisjj
> It has been hard to ensure new features developed are correct and bug free

Count yourself lucky. Most teams find it impossible! :)