Text Your AI to Control Your Home

"67% of smart speaker owners report frustration with misunderstood voice commands. 43% cite privacy concerns about always-on microphones." — Smart speaker adoption research, 2024

The Story

You're in bed. Your partner is asleep. The lights are still on downstairs. You know this because you can see the glow under the bedroom door, and you know yourself — you'll lie there thinking about it until you either get up or fall asleep annoyed.

You could say "Hey Siri, turn off the downstairs lights." But your partner is sleeping two feet away, and wake words aren't quiet. You could open the Home Assistant app — unlock your phone, find the app, wait for it to load, navigate to the right room, tap the right switches. Three taps minimum, plus a loading screen that somehow takes longer at midnight than it does at noon.

Or you could do what you'd do if you had a housemate who was still awake: send a text.

That's it. No wake word. No app. No loading screen. A text message that took less time to type than it took to read this paragraph.

The Idea

A plugin wraps your Home Assistant instance, exposing device control as tools the agent can call. You talk to it in natural language through whatever channel you're already using — Telegram from your phone, iMessage from your watch, Slack from your desk. The agent translates intent into action.

The agent understands context, not just commands. "Make it cozy in the living room" isn't a command any smart home system recognises. But the agent knows what you mean — because it remembers the last time you said "cozy" and what you ended up adjusting.

The plugin exposes five tool categories, each with an assigned risk level:

• lights.set — Low risk. Toggling lights, adjusting brightness and colour temperature. Approved automatically.
• thermostat.set — Medium risk. Temperature changes within a safe range (16-26°C) approved automatically. Anything outside that range requires confirmation.
• sensors.read — Low risk. Reading temperature, humidity, motion sensors, door/window states. No side effects.
• scenes.activate — Low risk. Triggering predefined scenes ("Movie Night", "Good Morning"). Only scenes you've already configured in Home Assistant.
• alarm.set — Critical risk. Arming or disarming the security system. Always requires explicit approval via escalation.

The agent works across time, channels, and context.

And when the stakes are real, the Judge steps in.

What Makes It Better Than Voice

Voice assistants were supposed to be the future of smart home control. The reality is more complicated. Sixty-seven percent of smart speaker owners report frustration with voice recognition. Forty-three percent cite privacy concerns about always-on microphones. Forty percent of smart speakers sit unused after the first year.

Voice fails at historical queries. "Was the heating on last night?" requires the assistant to query logs, aggregate data, and explain a timeline. Alexa can't do this. Google Home can't do this. They're designed for imperative commands, not questions about state over time.

Voice fails at aggregation. "What's the energy usage breakdown this week?" requires pulling data from multiple devices, calculating totals, comparing to previous periods. Voice assistants have no mechanism for this kind of multi-step reasoning.

Voice fails at context. "Turn off the lights" followed by "actually, keep the hallway on" requires the assistant to remember what it just did and partially undo it. Most voice assistants treat each command as independent.

Text works when voice doesn't. Partner sleeping. Baby napping. On a phone call. In a meeting. Noisy kitchen. Not at home. Every one of these is a situation where voice control fails and text just works. You don't need to be in the same room, the same building, or even the same country.

No always-on microphone. Text-based control means no device in your home is listening to everything you say, waiting for a wake word. The privacy difference isn't incremental — it's categorical.

How It Works

• Plugins — A plugin wraps the Home Assistant REST API using JSON-RPC over stdio. The plugin runs as a subprocess with its own process isolation — it can talk to Home Assistant but can't access anything else on your system. Standard plugin protocol means anyone can write a Home Assistant plugin, or adapt it for other platforms (Hubitat, openHAB, SmartThings).
• Channels — Telegram for your phone, iMessage through integration, Slack from your desk, the desktop app when you're at your computer. Same agent, same memory, same home control — regardless of where you're texting from.
• Judge — Every device action is evaluated for risk before execution. Lights are low risk. Thermostat within range is medium risk. The alarm is always critical. Risk levels are configurable — if you want door locks to require approval too, you set that once.
• Escalation — Critical actions trigger an approval flow. The agent sends you an inline keyboard (Telegram) or a confirmation prompt (other channels) and waits for your explicit approval. No timeout auto-approval. If you don't respond, nothing happens.
• Memory — The agent learns your patterns. "Cozy" means warm white at 40% with blinds closed because that's what you did last time. "Bedtime" means all lights off, thermostat to 19°C, alarm in home mode. These aren't programmed scenes — they're learned preferences from your natural language.
• Scheduler — "Turn the porch light on at sunset" creates a scheduled job. "Run the robot vacuum every weekday at 2pm" creates a recurring schedule. The scheduler works with the plugin system, so any device action can be automated with a text message.

What Breaks Without This

Voice assistants have a 20-40% command failure rate in real-world conditions. They have no memory between commands, no ability to reason about historical state, and no mechanism for complex multi-device coordination. They also require you to be physically present and speaking out loud — which eliminates half the situations where you actually want to control your home.

The Home Assistant app is powerful but designed for power users. The mobile interface requires multiple taps to do anything, loads slowly on cellular connections, and presents every device and automation with equal visual weight. It's a control panel, not a conversation partner.

Home Assistant's built-in AI agent (Assist) lives inside Home Assistant itself. You have to open Home Assistant to use it. It can't reach you proactively, can't work across channels, and can't remember conversations. It's a voice interface bolted onto a dashboard, not an agent that lives in your messaging app.

Siri and Alexa Shortcuts require you to pre-program every automation. "If I say X, do Y." This works for the five commands you use every day. It fails completely for the long tail of natural language requests that make up real life — questions about state, multi-step adjustments, contextual commands, requests that reference previous interactions.

Build This