I'm diving into Cython for the first time while working on a Python project similar to `zipgrep`/`ugrep`. My project streams through archive contents, searching for a given pattern without keeping much in memory. I've seen some benchmarks showing that my tool is about 15-30 times faster than `zipgrep`, but 2-8 times slower than `ugrep`, which is expected since `ugrep` is C++ based. I've tried using `cythonize` from `Cython.Build` with setuptools and also Nuitka, but I didn't notice any performance boost, even though both compiled successfully. I'm wondering if it's worth manually writing `.c` files and converting parts of my code to `cdef`, or will Python's inherent overhead always keep it slower than something like `ugrep`?
1 Answer
Cython has some profiling tools that can really help identify which parts of your code could benefit from avoiding Python interactions. This can show if you can get significant performance gains or just minor improvements. For getting close to native C++ speed, profiling is essential!
That's a logical approach! I'm currently using cProfile with snakeviz to find hotspots. Do you think Cython's tools would help me achieve a noticeable speed boost?