Hey everyone, I'm a fullstack developer who's gotten really comfortable with TypeScript, and now I'm facing some challenges while working on a Python backend without an ORM. My setup involves a separate query.py file for each service, where I define my SQL queries as Python functions that take in some parameters and generate SQL strings on-the-fly. The query functions return untyped dictionaries and lists, and we only marshal these into Pydantic models at the route level with FastAPI. I was told this approach is because of performance concerns, but with most query results limited to around 100 rows due to pagination, I'm doubtful that the performance gain is worth the lack of type safety. I'm looking for a solution that strikes a balance—maybe a wrapper class that only validates fields that are actually accessed? How significant is the overhead for type validation in Python compared to SQL query speeds and network latency? Any thoughts?
5 Answers
You might want to check out `TypedDict` for a more structured approach! It allows you to define expected keys and types in a dictionary-like format without adding runtime overhead. Another option could be using dataclasses where you can add type validation in the `__post_init__` method, which is pretty handy.
Using type validation at your app's input/output boundaries while relying on static typing elsewhere can really help you find that middle ground you're looking for. Just ensure you're leveraging good static type checkers like Mypy or Pyright, they’ve really improved recently!
If you're interested in faster validation, consider using msgspec instead of Pydantic. It has shown great benchmarks for performance. Here’s where you can find more about it: [msgspec benchmarks](https://jcristharif.com/msgspec/benchmarks.html).
Just a heads up, type hints and runtime validation serve different purposes in Python. If you're validating things like formats for strings or integers, you'll want return typed dicts from your queries. They’ll have no extra runtime costs over normal dicts, plus SQL is generally typed, so that helps too! What kind of validation are you really aiming for, by the way?
I get that! I'm just a bit concerned about tricky data types like dates and UUIDs. But I think typed dicts should cover most of my bases.
Looks like you're up against some pretty rigid decisions. Some devs might prefer not having typed dicts to avoid increased maintenance costs, which seems short-sighted if you're dealing with complex data. Pydantic has its strengths, but others suggest checking out alternatives like msgspec for speedier validations.
Subclassing to implement a custom `__getitem__` for type validation sounds like overkill to me. I mean, TypeScript doesn’t do runtime validation either!