I'm exploring whether an AI agent can run completely client-side using JavaScript, with no dedicated hosting costs and no user data sent to a remote server. The model could run directly in the browser through WebLLM, or connect to a local Ollama or vLLM instance for an offline setup.
The basic design is an asynchronous loop that maintains conversation state, calls the model, executes requested tools, and updates the application or DOM. A simplified WebLLM-style implementation might look like this:
```javascript
import { CreateMLCEngine } from "@mlc-ai/web-llm";
async function agentLoop(task) {
const engine = await CreateMLCEngine("gemma-4-instruct");
const messages = [{ role: "user", content: task }];
while (true) {
const response = await engine.chat.completions.create({
messages,
tools: availableTools
});
const message = response.choices[0].message;
if (!message.tool_calls) return message.content;
for (const call of message.tool_calls) {
const result = await executeTool(
call.function.name,
call.function.arguments
);
messages.push({
role: "tool",
tool_call_id: call.id,
content: result
});
}
}
}
```
What are the main practical limitations of this browser-first approach, especially around hosting requirements, WebGPU or WebAssembly support, tool execution, context size, and safely allowing the agent to modify application state?
2 Answers
Zero-cost static hosting is realistic, but the requirements depend on how the model runs. CPU-based multithreaded WebAssembly commonly needs SharedArrayBuffer, which usually means serving the app with COOP and COEP headers and enabling cross-origin isolation. WebGPU-backed WebLLM deployments can avoid that particular requirement by using the client’s graphics hardware, making static hosting on services such as object storage or static site platforms much simpler. Browser and device support still needs to be tested carefully.
The bigger challenge is reliability and safety rather than the loop itself. Keep the agent as an explicit state machine with bounded retries, clear tool schemas, and checks for malformed or dangerous model output. Be especially cautious before allowing writes to the DOM, IndexedDB, local storage, or files. A safer design is to apply proposed changes to an in-memory representation first, validate them, and then commit them. Browser models also have limited context and can put pressure on VRAM, so trimming history, limiting tool results, and adding circuit breakers are useful.
Exactly. Client-side execution improves privacy, but it does not make model output trustworthy. Every state-changing tool should validate arguments and enforce its own permissions instead of relying on the model to behave.
That distinction is important. WebGPU can avoid the cross-origin-isolation setup for the model path, although the app still needs fallback behavior for browsers without usable WebGPU support.