After so many months, finally we have a green CI – we can consider pull requests and changes that break the CI as “unstable”. This is a huge milestone, and I want to both thank everybody that tracked these failures to check if they were legit, to the people that fixed the legit failures and that fixed the flaky tests. This has been an amazing experience, and it helps build trust on the project!
If you look at the commits from the Atom project and check which ones have passing tests, you’ll see a horrible situation: they mostly don’t pass. There’s even a recent commit called “disable tests” that passes, and then the next commits also fail – so Atom was already failing tests when these were disabled (how is that possible, I won’t dare to think).
So it’s a huge milestone to have a CI that confidently (or at least we hope!) say if we broke something new! But on this post, I want to show not only how cool is that, but also the process and how to avoid flaky tests on the near future – both for Pulsar and for other projects