Makes a lot of sense. I figure contract tests is more or less what I have been doing then.
I think there is that misconception that unit tests are about validating each line of code and preventing logic bugs. Though obviously you understand that this isn’t just about that, or not at all about that I would argue. Unit tests won’t prevent accidental breaking changes, but you can at least add new tests every time this happen, so you’re at least guaranteed that the next maintainer won’t be doing the same mistake.
In an ideal world we could probably have nothing but integration tests, but in my experience if you only do integration testing you end up working really hard maintaining tests that are prone to break and costly to run. Unit tests are cheap to code and cheap to run, it is a great place to enforce your “contracts” with tests that are theoretically immutable. After those tests are out of the way, you can focus only on the actual interaction between your systems with the more expensive tests.
Anyway, you have a good take I am just blabbering. This is based on my personal experience as someone who only cared integration tests but was later converted by a person much smarter than I am. And then later on by joining a team that exclusively did integration testing, hundred of tests. It was hell (imo), we had to change dozens of tests for every little bit of modification. The rest of the team seemed fine with it though. We rarely shipped bugs but progress was incredibly slow for a module of such low complexity. Testing wasn’t the only issue with that codebase though.
The one part of that that sounds weird to me is needing to change integration tests frequently when changing the code. Were you changing the behavior of existing functionality a lot?
Yes, and the test suites were insane. The program was outputting a lot of data, and we basically asserted on anything and everything for any given integration. I mentioned that testing wasn’t the only issue, well there was a lot of issues. Unfortunately the behaviour changes were requested by the stakeholders and there was no way around it. That being said, had this thing we maintained been properly developed those changes would have been a breeze imo. The actual requirements were very simple.
But anyway, I realize this is maybe an extreme example to paint integration tests negatively, but the point remain. In this scenario, every time we changed a bit of code it broke dozens of integration tests instead of breaking just a relevant integration test, had everything that could have been unit tested been written that way. The integration tests could probably also had been less… exhaustive, but it was probably for the best considering the codebase.
Makes a lot of sense. I figure contract tests is more or less what I have been doing then.
I think there is that misconception that unit tests are about validating each line of code and preventing logic bugs. Though obviously you understand that this isn’t just about that, or not at all about that I would argue. Unit tests won’t prevent accidental breaking changes, but you can at least add new tests every time this happen, so you’re at least guaranteed that the next maintainer won’t be doing the same mistake.
In an ideal world we could probably have nothing but integration tests, but in my experience if you only do integration testing you end up working really hard maintaining tests that are prone to break and costly to run. Unit tests are cheap to code and cheap to run, it is a great place to enforce your “contracts” with tests that are theoretically immutable. After those tests are out of the way, you can focus only on the actual interaction between your systems with the more expensive tests.
Anyway, you have a good take I am just blabbering. This is based on my personal experience as someone who only cared integration tests but was later converted by a person much smarter than I am. And then later on by joining a team that exclusively did integration testing, hundred of tests. It was hell (imo), we had to change dozens of tests for every little bit of modification. The rest of the team seemed fine with it though. We rarely shipped bugs but progress was incredibly slow for a module of such low complexity. Testing wasn’t the only issue with that codebase though.
The one part of that that sounds weird to me is needing to change integration tests frequently when changing the code. Were you changing the behavior of existing functionality a lot?
Yes, and the test suites were insane. The program was outputting a lot of data, and we basically asserted on anything and everything for any given integration. I mentioned that testing wasn’t the only issue, well there was a lot of issues. Unfortunately the behaviour changes were requested by the stakeholders and there was no way around it. That being said, had this thing we maintained been properly developed those changes would have been a breeze imo. The actual requirements were very simple.
But anyway, I realize this is maybe an extreme example to paint integration tests negatively, but the point remain. In this scenario, every time we changed a bit of code it broke dozens of integration tests instead of breaking just a relevant integration test, had everything that could have been unit tested been written that way. The integration tests could probably also had been less… exhaustive, but it was probably for the best considering the codebase.