What’s the Pythonic way to add an LLM-powered REPL to a server?

The usual approach is to call the model through its provider’s Python SDK or an HTTP client rather than launching a command-line process. Put the conversation loop in your service, expose carefully defined tools for things like Zigbee actions, and stream responses back to the client from an async endpoint. A subprocess can work for a prototype, but managing its lifetime, errors, and concurrent sessions becomes awkward in a real server.

A tool-calling framework such as PydanticAI can be a good fit. You define typed tools and structured outputs, then let the model choose from the operations your server explicitly exposes. That gives you a conversational loop without giving the model arbitrary access to your machine, and schema validation helps catch malformed arguments before anything touches your network.

First decide what the LLM is actually needed for. If the goal is simply remote control, a normal authenticated API may be more reliable, with the model acting as a natural-language layer on top. Provider APIs and SDKs are also easier to change than a process-based integration, especially if you keep your own small abstraction around chat messages, streaming, and tool calls.

Agent or REPL libraries can handle conversation history, tool calls, streaming, and context management, but they don’t automatically make the setup safe. An agent with unrestricted LAN or shell access is effectively a remote administrator. Use strong authentication, limit tools to a small allowlist, validate every argument, isolate the service, add confirmations for destructive actions, and log all requests and device changes.