How do you organize and trigger integration tests in a remote CI pipeline when they are too slow or resource-intensive to run on every pull request? I usually separate tests into tests/unit and tests/integration, or mark them with pytest markers configured in pyproject.toml, and I know how to select either group locally. I'm mainly looking for best practices around CI triggers—for example, running fast tests on every change, while starting integration or end-to-end tests manually, before merging, or as a required step before release and deployment.
4 Answers
For especially expensive tests, running them before the merge is usually a good compromise. Hardware-in-the-loop and event-driven systems can take much longer because they need real devices, queues, polling, or propagation time. In those cases, keep a shorter representative suite on ordinary changes and reserve the full configuration for a manual run or a required merge/release gate.
Use the event that triggered the pipeline to choose the pytest markers. For example, run the fast suite for normal pull requests, and run tests marked with something like e2e or integration when the change is being merged or a release is being prepared. A manual workflow with parameters or checkboxes is also useful when you need to select a particular environment or hardware configuration.
I run integration tests after the build has confirmed that a release is going to be created, but before publishing the release. The tests are explicitly marked in pytest and selected with options such as -m integration or -m 'not slow'. This keeps the normal feedback loop quick while still preventing a release from being created when the end-to-end checks fail.
A common setup is to run unit tests on every change, then make the integration suite a manual CI job. You can also require it for merges into the main branch or as a dependency of a release/deployment job. That gives you an on-demand option without letting long-running tests slow down every pull request.
That distinction is useful: unit tests usually isolate one function or class and check inputs, outputs, and errors, while integration tests verify that components work together—for example, an API endpoint communicating with a database or an event moving through several services.