Has anyone managed to configure a fully local AI assistant in VS Code that provides inline code suggestions comparable to Copilot, particularly for PowerShell? When I connect my account to VS Code, Copilot can often turn a single comment into most of a working script. For example, a comment describing a script that finds enabled Active Directory users whose passwords expire within seven days and emails them an HTML notification may produce a surprisingly complete starting point with one Tab press.
I have tried Ollama with several models and extensions, including Continue, Twinny, Code Llama, and various Qwen models. Code Llama has been the most usable so far, but it still frequently produces poor results. In one test, the comment `# find the password expiration date for each AD user` generated an incomplete PowerShell loop, used questionable properties, and then repeated the same block several times. I tested `codellama:7b-code-q6_K`, along with Qwen models around 1.5B, 7B, and 13B parameters. They all fit comfortably in my available VRAM.
I prefer running the models locally so I can keep scripts, names, email addresses, keys, and other sensitive details on my own hardware. I have configured my hosted AI account not to use my data for training, but I am still not fully comfortable sending work-related code to an external service. Are there better local models, inference engines, or VS Code extensions that provide reliable inline completion rather than just a chat interface?
4 Answers
Ollama is convenient, but it is not always the fastest or most flexible way to run local coding models. It may be worth comparing it with llama.cpp and looking into GPU offloading, context length, prompt formatting, and speculative decoding. Those settings can make a noticeable difference, though local models can still produce repetition or invalid PowerShell. Larger coding models generally perform much better than small ones, but they require substantially more memory.
For routine PowerShell work, a personal library of tested functions and profile helpers is still more predictable than any model. AI can save time, but generated scripts should be checked carefully because even good models can invent cmdlets, use the wrong properties, or misunderstand platform-specific APIs. A practical compromise is to keep your trusted functions locally and use the model for scaffolding, comments, refactoring, and remembering the shape of an older script.
A single comment turning into an entire script is closer to a chat or agent workflow than traditional inline completion. Inline models usually predict the next few lines from the surrounding code, while a chat-based tool can reason through requirements, ask questions, and generate a larger solution. You may get better results by describing the task in a chat panel first, then using inline completion for smaller edits and boilerplate.
That makes sense, although in a mixed team where some people are uncomfortable with the command line, generating a well-documented first draft from one comment is genuinely useful. I am mainly trying to understand whether local tools can get anywhere near that workflow.
For local coding assistance, a Qwen model in the roughly 27B range is likely to be a better starting point than the smaller models you tested, assuming your hardware can run it at a usable speed. Models in the 1.5B–13B range often struggle with multi-step PowerShell tasks and may repeat generated blocks. The strongest hosted models are still ahead of most local options unless you have high-end hardware, so expectations need to be realistic. Also, avoid putting credentials directly in scripts whenever possible; use a secret store, environment-based authentication, or encrypted credential files instead.
I normally keep credentials in encrypted exported credential files and avoid hardcoding them. Plaintext secrets mainly happen during short-lived testing, such as experimenting with Graph requests. The larger local models may be beyond my current hardware, but I may still compare a quantized version.
I used that approach for years, but a strong coding assistant has become a major productivity boost. PowerShell is only one of many responsibilities in my work, so generating a useful first draft quickly can be more valuable than searching through old scripts and rebuilding the context each time. I still review and test everything, but the hosted tools I have tried are much better at this than the local models so far.