The final component of a DataOps environment is the testing environment. Ideally, teams build tests prior to the development of any code or functionality. The orchestration tool then runs the tests before and after every stage in the pipeline. Catching bugs and problems upstream at the point of inception and integration saves oodles of time, money, and stress down the road. Development teams that use testing as a fundamental building block move faster than those that do not. This is counterintuitive at first for most developers who want to dive into a project without building tests before they start. But once tests are in place, developers can quickly identify issues before they get nestled deep into code where they are harder to fix. With continuous testing and monitoring, teams can set targets for performance, measure their output, and continuously improve their cycle times and quality.

Tests are the bedrock of automation. Without tests, automation is a runaway freight train that inevitably crashes. With tests, data teams can sleep well at night knowing they have built all the safeguards necessary to keep the train on the track. And if the train starts to deviate, they are proactively notified and can take action before business users experience problems. ‘Test automation is a huge part of what we do,” says Intel’s Martinez. “Without it, we can’t maintain a high level of quality at the scale and speed with which we operate. We have more than 1,000 tests in our test automation framework, and we keep adding tests all the time. We continually measure our progress over time, both individually and as a group. We are only as good as our test practices, and we strive to improve here.’

Pre-deployment testing is performed to confirm that the models are ready to deploy and deliver operational value, and to provide a basis for post-deployment comparison. Where earlier stages of testing—unit, user, and integration—focus on building things right, pre-deployment testing focuses on deploying the right things. Pre-deployment testing may also include regression testing when preparing to deploy revisions to a previously deployed model. Test automation fills an essential role when preparing to deploy.