Parsing multipart/form-data for large file uploads involves scanning and moving a lot of data, so it seems like an obvious candidate for Cython or Rust. However, benchmarks of several Python multipart parsers show that a carefully designed pure-Python implementation can sometimes perform surprisingly well, depending on the workload and API style. This raises a broader question: are there other situations where pure Python can beat Cython, Rust, or C implementations, or are these results mainly explained by differences in algorithms, implementation quality, and how often native code crosses the Python boundary?
4 Answers
Pure Python can also look competitive when it calls optimized native libraries such as NumPy or Numba, although that is not the same as Python executing the core algorithm itself. More generally, algorithm choice matters more than the language label: a better algorithm in Python can beat a worse one in Rust, while equivalent optimized native code should normally have the edge.
Cython is not automatically fast. If variables remain Python objects, the generated code still has to use the Python C API for type checks and operations. It only gets a major advantage when the hot loop uses C-level types and avoids repeated Python-object interaction. The same applies to Rust extensions: conversions and FFI calls can add overhead if they happen for every chunk or field.
Multipart parsing is close to a best-case workload for Python because much of the expensive work can be delegated to CPython's C-level bytes operations. Python may only coordinate a small number of calls to search for delimiters and create views, rather than processing every byte in a Python loop. A native binding that repeatedly marshals objects across the boundary or copies buffers can lose despite being compiled.
That advantage disappears when the work is a genuinely tight loop that stays in Python bytecode. Numeric kernels, custom hashing, and per-pixel processing are typical cases where Cython or Rust can be dramatically faster.
The benchmark does not really show Python beating an equivalent, well-written Rust or C implementation. It compares different Python packages, some of which use native extensions and have different designs and trade-offs. A correctly implemented native parser should generally win on the same task, but real-world libraries can be slower because of extra copying, allocations, or an inefficient interface.
Exactly—“if both are written correctly” is doing most of the work here. Language comparisons only mean much when the implementations perform the same work and have similar optimization goals.
The implementation details matter a lot. A native parser that creates an owned buffer for every part may lose to code using bytes.find and memoryview slices if those operations avoid copies.