I benchmarked several Python runtime validation tools—Beartype, Typeguard, type_enforced, and Pydantic—on Python 3.14.5 running on Ubuntu 24.04 with a Ryzen Threadripper 3770 and 128 GB of RAM. Each test measured the average time for one function call across 100 runs, excluding an initial warm-up.
The cases included primitive values, unions, dictionaries, lists, and nested collections. Beartype, Typeguard, and type_enforced were tested with one-item sampling, while Pydantic and a separate type_enforced configuration validated every collection element.
The results show that sampled validation remains nearly constant as collection sizes increase. For example, Beartype stayed around 0.45–0.78 microseconds, Typeguard around 3.7–6.5 microseconds, and type_enforced with one sampled item around 0.15–0.30 microseconds. Full validation scaled with collection size: Pydantic reached 881.63 microseconds for a 10,000-item list of unions, while type_enforced reached 122.87 microseconds for the same case.
However, sampled validation can miss invalid data. The warning markers in the results indicate tests where an intentionally invalid collection element was outside the sampled position and therefore was not detected. I also collected comparable results on Python 3.13 and 3.12, which were broadly similar but sometimes slower depending on the structure.
Are there important decorator or configuration optimizations I may have missed? Are there other runtime validation libraries or test methods that would make this comparison more complete?
3 Answers
The additional Python 3.13 and 3.12 numbers are useful. They suggest that interpreter-version differences are relatively small compared with the algorithmic difference between sampling and full traversal. For practical decisions, benchmark representative application inputs, include invalid cases, repeat more than 100 times, and report distributions such as median and percentiles rather than only the average.
Pydantic is primarily a data-validation and parsing library rather than a dedicated runtime type-checking decorator. It is still reasonable to include it if the comparison is about runtime validation broadly, especially because it can coerce input and validate structured data. The results should make that distinction explicit since its work is not identical to a simple type check.
That distinction makes sense. I included it because many projects use Pydantic models or validated function calls as their runtime input boundary, even though its scope is broader than checking annotations alone.
The biggest issue is not the small timing differences but the different correctness guarantees. Sampling one item gives impressive flat timings, but it does not establish that the collection satisfies its annotation. A benchmark should either deliberately place invalid values where each implementation is guaranteed to inspect them or report correctness and performance as separate measurements. Otherwise the fastest configurations are partly measuring how little work they perform.
Exactly. A warning helps communicate the limitation, but it would be clearer to provide separate tables for sampled checks and full-containment validation instead of mixing them together.
Agreed. The current results are mainly intended as a quick comparison. More repetitions, percentile timings, and separate valid and invalid datasets would make the methodology stronger.