Mika Döffinger
Developer
David Schopf
Senior Developer
Pascal Bouwmann
Senior Developer (ehem.)
How Consumer-Driven Contract Testing prevents us from implementing breaking changes in APIs without proper versioning.
Contract Testing
According to the Pact documentation, contract testing changes the development workflow by changing the way testing is performed during the software lifecycle. It states the following:
Do you set your house on fire to test your smoke alarm? No, you test the contract it holds with your ears by using the testing button. Pact provides that testing button for your code, allowing you to safely confirm that your applications will work together without having to deploy the world first.
That sounds promising, which is why we wanted to try this out.
If you are not familiar with contract testing, check out the official documentation of Pact at https://docs.pact.io/. There is also a workshop tutorial for utilizing contract testing, e.g. in JavaScript, at https://github.com/pact-foundation/pact-workshop-js.
Our Project Setup
We had a project at hand which did not use contract testing yet, i.e. we did not start from scratch but performed a migration.
Our project consists of two frontends, one web frontend developed in Angular and one native iOS mobile frontend. The service layer consists of microservices which expose a REST API via an API gateway. The services are written in TypeScript. Our goal was to introduce contract testing into an existing project and analyze how the development workflows change. Hence, we limited the scope such that we used the web frontend as our consumer and one microservice as our provider. To simplify the setup, we used PactFlow as a fully managed Pact Broker, that stores and manages consumer-driven contracts between services, enabling teams to share and verify these contracts to ensure compatibility between consumers and providers.
The source code of both the web frontend and the microservice are stored in GitHub repositories. The CI/CD pipelines are hosted in GitHub Actions. The frontend is deployed to Google Firebase by one build and deploy pipeline. The microservice is deployed to a Cloud Run instance in the Google Cloud Platform. We have two separate pipelines for the microservice: a build pipeline which creates a Docker image and a deployment pipeline which deploys the service to Cloud Run.
Change The Contract
Currently, our provider exposes the /address/id endpoint and returns an object of the format
{
"id": number,
"street": string
}
Our current contract defines an interaction for the GET call against /address/1 which should return
{
"id": 1,
"street": "Elm Street"
}
{
"id": 1,
"street": "Elm Street",
"postbox": "P.O. Box 1234"
}
When the consumer pipeline successfully runs contract tests and publishes a new contract to the Pact Broker, this action triggers a webhook that initiates a GitHub Action in the provider repository named contract_requiring_verification_published, which then verifies the contract against the provider versions in the main branch, test environment, and production environment.
All verifications fail because the provider does not fulfill the new contract. The endpoint is missing the postbox field. In the next step of the consumer pipeline, the can-I-deploy action for the test environment is executed and blocks the deployment of the consumer due to the failed verification.
Provider Adjusts Implementation to Fulfill Contract
At the provider, we create a new feature branch with the same name as the consumer’s feature branch. The matching name is important due to the selectors of Pact, see https://docs.pact.io/provider/recommended_configuration#if-using-branchesenvironments. We add the postbox field to the response object of the /address/:id endpoint. The provider pipeline executes the provider verification for the contracts on the feature and main branches of the consumer and the deployed versions of the consumer on test and production environments.
All verifications pass because the provider fulfills the new contract without breaking the existing contract. The consumer with the old contract is satisfied with the new API response because it ignores the new postbox field. The new contract is satisfied because it receives a new postbox field.
With specifying the environment to check against, the can-I-deploy step in the provider pipeline is green and the provider can be deployed to test.
Deployment of The Consumer
The consumer pipeline of the feature branch can be re-triggered. The pipeline finishes successfully due to the successful provider verifications.
The changes can be merged to the main branch and deployed to the test environment. The can-I-deploy check for production still fails because the new provider version has not been deployed yet.
Deployment to Production
As we have previously seen, the provider API is backward-compatible. Hence, the provider can be deployed to production. After a new provider verification for production, also the can-I-deploy check for consumer to production succeeds and the consumer is deployed to production.
The consumer initially changed the contract which is fulfilled by the provider in all environments. When calling /address/1 with GET method, the following response is returned.
{
"id": 1,
"street": "Elm Street",
"postbox": "P.O. Box 1234"
}
Introducing a Breaking Change
After some time it turns out that the IDs of the addresses should be strings instead of numbers. The contract is changed on a feature branch of the consumer such that the response looks like:
{
"id": "1",
"street": "Elm Street",
"postbox": "P.O. Box 1234"
}
All provider verifications fail because in all provider versions the id is still a number whereas in the new contract the id is a string.
To fix the provider verifications, the provider changes the data type of the ID field to string on its feature branch.
The verification succeeds for the feature branch contract of the consumer. However, as the contracts on the main branch and in the test and production environments still declare the id field as number, the verification fails there. The provider cannot be merged into the main branch as the verification fails. The only way to continue is to bypass the checks, merge the provider and the consumer feature branches and deploy the new versions simultaneously, which is bad practice.
How could we have avoided the dead lock?
The Solution: API Versioning
Contract testing helps us ensure that APIs are backwards-compatible. If the new version is incompatible with older ones, we will reach a situation as described above. With proper API versioning, we can reach a state where the provider verifications succeed, and we can merge and deploy new versions of providers and consumers.
Let us implement the API properly from the beginning by introducing versioning. For APIs, there are several ways of handling versions. We choose to add the version number to the URL. Instead of exposing the address endpoint at /address/:id, the provider exposes it at /v1/address/:id. The version 1 of our API is born.
Now, our contract defines an interaction for the GET call against /v1/address/1 which should return
{
"id": 1,
"street": "Elm Street",
"postbox": "P.O. Box 1234"
}
Again, we want to change the id field to be a string. As this change is not backwards-compatible, we introduce version 2 of our API, i.e. the data structure at /v1/address/1 stays the same. Instead, we create a new endpoint at /v2/address/1 which returns the new data structure
{
"id": "1",
"street": "Elm Street",
"postbox": "P.O. Box 1234"
}
We add a new interaction to our contract. The first – already present – interaction is the call against v1, the second interaction is our call to the new endpoint at v2.
The provider still fulfills the contracts on the main branch and in the test and production environments, as version 1 did not change. The provider can be merged into the main branch and deployed to test and production environments.
Afterwards, the consumer can run its can-I-deploy checks. All checks are green, as version 1 and version 2 of the API are deployed to the target environments. The consumer can be deployed to all environments.
Conclusion
Contract testing is a great tool to manage APIs and ensure integrity of all components (consumers and providers). It supports implementing API best practices like versioning by design.
While setting up contract testing may require an initial investment of time and resources, especially in terms of defining contracts and integrating them into existing workflows, the long-term benefits far outweigh the costs. Contract testing is particularly valuable in projects that involve multiple microservices, where clear communication and compatibility between consumer and provider teams are critical. It enhances collaboration, reduces integration issues, and supports faster delivery cycles by providing confidence that changes will not break downstream dependencies.
In our example, version 1 of the API is not used anymore and could be decommissioned. The decommissioning consists of two parts. One part is to remove the respective interaction from the contract. The second one is to remove the version 1 endpoints from the provider. The order of deployments differs from the process explained above. Feel free to think through the process of removing obsolete endpoints.
For more complex project setups, it is not so easy to detect which endpoints are not used anymore. For this, the record-deployment tool is beneficial.
Try it out on your own!