I've been thinking about ways to enforce coding rules for projects without relying solely on LLM rules. Has anyone experimented with using custom linting and type-checking scripts instead? It seems like a more sustainable approach to establish guardrails. Of course, LLM rules would still be necessary for overarching concepts, but I believe custom linting extensions could get us pretty far. Has anyone pursued this route? Was it effective, or did it end up creating more issues?
4 Answers
I used to think this was a solid plan! However, after trying it out, I'm now seeing that it could make things trickier rather than easier. Check out my previous post for more context on my experience.
I believe using linters can be beneficial for certain production applications where code structure and patterns matter. But if you’re aiming to manage LLM hallucinations, you’ll probably have to dive into a more thorough evaluation process. Also, tools like VS Code can catch some common mistakes while coding.
I’m just vibecoding out here! Can someone explain what a linter actually is?
Why not ask your chatbot about it? That could be fun!
I’ve been using mypy along with ruff for linting and type-checking in Python. Honestly, I’ve encountered fewer errors since I instructed the LLM to always include type hints!
It's basically getting feedback from a compiler when your LLM bot writes code! That's what linting is all about.