How do you manage flaky E2E tests at scale?

Question

I&rsquo;m curious how folks + teams here deal with flaky end-to-end tests once a product and test suite gets "big" (> thousands test cases).I&rsquo;ve seen a few patterns over the years: retries everywhere, quarantining tests, rewriting flows, adding more waits than anyone feels good about, or just slowly losing trust in CI signal. None of them feel great once you have hundreds or thousands of tests running across multiple environments.I&rsquo;m especially interested in how QA and engineering teams split responsibility. Do you treat flakiness as a test problem, a product problem, or infrastructure noise? At what point do you decide a test is no longer worth keeping?Asking partly out of personal frustration and partly because I&rsquo;ve been working on tooling around browser automation and want to sanity check the problems I&rsquo;m seeing against the pains others are feeling day to day.Would love to hear real stories from people running E2E at scale, what actually worked, and what you wish you had done earlier.Thanks in advance.

benoau · Accepted Answer

I think testing via browser automation is fantastic, API-driven web browsers are just amazing tools, but it's also the source of a lot of the flakiness because of the inherent difficulties determining when a page is completely in a state that it can do what you're expecting of it. Playwright improves a lot on this over Puppeteer but it's imperfect, I often end up with a wait for a selector, load state, or function evaluating when it's actually okay to proceed and sometimes I'll use a combination to really make sure, because what it's really waiting for is not just achieving some state but what actually happens after it does.

apothegm · Answer

You find out why they&rsquo;re flaky, and fix them. If they&rsquo;re flaky in testing, there&rsquo;s probably something flaky in real world use. If they&rsquo;re flaky because of something specific to the test environment then they&rsquo;re not testing what you think they&rsquo;re testing, so fix them or get rid of them.