Has anyone configured VS Code with a local AI assistant that gives inline code suggestions anywhere near the quality of Copilot, especially for PowerShell? When I connect my account to VS Code, Copilot can often turn a single comment into most of an entire script. For example, a comment describing a task like finding enabled Active Directory users whose passwords expire within seven days and emailing them an HTML-formatted notification can produce a surprisingly complete starting point after pressing Tab.
I would prefer to run everything locally for security and privacy reasons, without constantly replacing names, email addresses, keys, or other sensitive values before pasting code into the editor. I have tried several Ollama models and extensions, including Continue, Twinny, Code Llama, and multiple Qwen models. Code Llama has been the best so far, but it still performs poorly. Given a comment such as "find the password expiration date for each AD user," it may generate a partially incorrect loop and then repeat the same block several times. I tested codellama:7b-code-q6_K as well as 1.5B, 7B, and 13B Qwen models, all of which fit comfortably in my available VRAM. Am I using the wrong models, extensions, or inference setup, or are local models simply not comparable to hosted coding assistants yet?
4 Answers
A one-line comment that expands into a complete script is closer to a chat or agent workflow than ordinary inline autocomplete. Inline completion is optimized for predicting the next few tokens, while generating a whole reliable script requires more context, planning, and usually an interactive prompt. You may get better results by asking the local model to draft the script in a chat panel, then using inline completion for smaller edits.
Ollama is convenient, but it is not always the fastest or most configurable option. It may be worth testing llama.cpp and learning about settings such as context size, GPU offload, quantization, and speculative decoding. Larger coding models generally produce better results, but local models can still repeat themselves or invent PowerShell syntax, even when the model technically fits in VRAM.
For repetitive PowerShell work, a personal profile and reusable functions can be more dependable than asking a model to recreate everything from a comment. That said, local AI is useful for explaining existing functions, adapting a known pattern, or filling in smaller pieces. The quality gap between local models and the strongest hosted coding assistants is still noticeable for complex scripts.
I have relied on reusable functions for years, but a good coding assistant saves a lot of time when the task is unusual or I need to reconstruct a script I wrote long ago. Hosted tools have become much more reliable for this than older AI systems, which is why I am trying to find a local equivalent.
Among local options, a Qwen coding model in the 20B-plus range would likely be a better experiment than the small 1.5B–13B models. Models in that size class can still be less capable than the best hosted assistants, though. Also, avoid placing secrets directly in scripts whenever possible; use a secret store, encrypted credential files, or a secure prompt mechanism instead.
I normally keep credentials in encrypted XML files and avoid hardcoding them. Plaintext secrets occasionally appear during short-lived testing, particularly with Graph requests, but that is something I am trying to minimize.
The stronger local coding models may require substantially more memory than the smaller models being tested. If the hardware cannot run them at a useful speed, a hosted model will usually remain ahead for whole-script generation.