I benchmarked several Python runtime validation approaches on Python 3.14.5 running on Ubuntu 24.04 with a Ryzen Threadripper 3770 and 128 GB of RAM. The tools tested were Beartype with one sampled collection item, Typeguard with one sampled item, type_enforced with both one-item sampling and full validation, and Pydantic's validate_call with full validation. Each benchmark measured the average execution time of one function call over 100 runs after discarding a warm-up run. Tests covered primitive values, unions, dictionaries, lists, and nested collections ranging from a few items to 10,000 items. The sampled approaches stayed nearly constant as collection sizes increased, while full validation became progressively slower. However, the sampled modes can miss invalid values that occur outside the inspected position, which is marked with a warning in the results. Pydantic is primarily a data-validation and parsing library rather than a dedicated runtime type-checking decorator, but it is included because it validates and may coerce function arguments at runtime. I would appreciate feedback on whether the decorators were configured optimally and suggestions for other tools or more representative benchmark designs.
3 Answers
Pydantic is worth comparing for runtime validation, but it is not quite the same category as Beartype or Typeguard. Its main job is parsing and validating data, often with coercion, so its cost and behavior should be discussed separately from simple type checks. The results are still useful if the comparison clearly distinguishes validation semantics from type enforcement.
The benchmark should ideally include Python 3.12 and 3.13 as well as 3.14. The follow-up measurements suggest the overall pattern remains similar across versions: sampled checks are mostly flat with collection size, while complete validation scales with the number and nesting of items. Typeguard is generally slower for the simple cases, and the full-validation modes become especially expensive for large dictionaries and nested lists.
The warning on sampled collection checks is more important than the raw timing. A one-item sample can report success even when another element has the wrong type, so those numbers are not directly comparable with tools that inspect every item. A stronger benchmark would deliberately place an invalid value in each collection and measure both detection rate and runtime, or separate the sampling and full-validation results into clearly different categories.
That is fair. The warnings were intended to make the limitation visible, but testing every invalid position or reporting detection reliability separately would make the comparison clearer.