Roomy: Agentic Remote Sensing

To Stand on the Shoulder of Giants

The idea

The aim is to re-think the way people interact with their environment, creating a highly personalised user experience. This could be cereal lovers learning more about where the grains come from, allowing for a more satisfying experience, or restaurant goers better understanding the atmosphere of a restaurant - the feng shui of the room so to speak.

Roomy is an experiment that explores how to bring people closer to the phyiscal world by anthromorphosing the physical world. It took a Pimoroni Enviro pHAT, an agentic LLM, a Facebook Page, Facebook’s Graph API, and several man hours of vibe coding to bring the project to life. What’s my role in this? Making design decisions on the architecture of the application, where it is run and how it runs - engineering.

Roomy, reads the room using the sensor, as well as readings from the met-office (through their API), and writes poetry about it. The use case is deliberately artistic in nature. It is a testbed for the smallest interesting agent loop: perceive → interpret → express, with the sensor readings — not a human prompt — driving the generation. Roomy is live on Read The Room. Do read the page’s Data Privacy statement before interacting with Roomy.

Where it lives now

The original version was intendeded to run entirely on one Raspberry Pi however, hardware constraints lead to the design decision of running the application as a distributed system comprised of containerised services running on a K3 cluster. Roomy still perceives through a Pi at the edge, but the thinking has moved off it. Today the whole thing runs as a handful of small, single-purpose services on a multi-node K3s cluster.

The division of labour is deliberate. The sensor board lives on an arm64 Pi at the edge (it has the hardware; it does nothing clever). The poet — a local LLM served by Ollama — runs on a beefier node where inference is comfortable. Redis sits in the middle as the memory of the whole system, and a Mosquitto broker carries the messages between nodes. Nothing holds state locally; a pod can die and come back and Roomy doesn’t lose its train of thought, because the train of thought was never in the pod.

%%{init: {'flowchart': {'useMaxWidth': false, 'nodeSpacing': 60, 'rankSpacing': 90, 'padding': 16}, 'themeVariables': {'fontSize': '50px'}}}%%
flowchart LR
    A["Enviro pHAT<br/>(edge Pi)"] -->|MQTT| B["subscriber"]
    B --> C[("Redis<br/>event store")]
    W["Met Office<br/>(outdoor weather)"] --> C
    C --> D["Roomy<br/>(Ollama poet)"]
    D -->|roomy:poems| E["facebook_poster"]
    E --> F["Facebook Page"]
    F -->|comments| G["facebook_responder"]
    G --> D
    C --> H["dashboard<br/>rollups + charts"]

The agent loop

Roomy runs a perceive–interpret–express cycle, and “the loop” is spread across the cluster , enabling high availability. A publisher on the edge Pi reads the board and pushes each reading onto MQTT; a subscriber catches those messages and appends them to a Redis stream — an append-only log that becomes Roomy’s memory of the room over time. The poet reads from that log, not from the sensor directly, which is what lets it reason about trends (“cooler than yesterday”) and not just the instant.

A dedicated facebook page acts as the chat interface where Roomy autonomously posts composed poems on a timer based on current sensor readings. Questions can arrive on an MQTT topic, and comments left on the Facebook Page get pulled back in — so Roomy is genuinely a multi-turn agent, answering in verse.

The one design decision that still matters most is the interpret step. Handing raw floats to a language model produces flat, listy verse. Translating them into qualitative language first — “dim, warm light; air pressure falling” — gives the model imagery to work with and keeps the poem grounded in what actually changed - the model is *never shown a number at all.

How it works

1. Sense the room

The edge publisher reads each sensor and ships the reading over MQTT. Downstream, the poet pulls the latest reading straight out of the Redis stream — so “reading the room” is really a lookup, and the sensor and the poet never have to be awake at the same time.

def read_room(sensors) -> dict:
    """Pseudo code"""
    # readings = sensors.get_all()
    # dict readings
    # return readings

2. Turn numbers into words

A thin, pure rules layer maps readings to phrases. This is where the poem gets its senses — and where I keep the model honest. Each measurement falls into a qualitative band, so what reaches the poet is warm, dim, still, never 24.9 or 181 lux.

TEMP_BANDS  = ((5, "freezing"), (14, "cold"), (19, "cool"), (24, "mild"), (28, "warm"))
LIGHT_BANDS = ((5, "dark"), (60, "dim"), (300, "soft"), (800, "bright"))

def band(value, bands, default):
    # determine which band the reading belongs to 
    # return linguisting label if it falls in defined buckets else return defaul linguisting label
    

def interpret(reading):
    return {
        "temperature": band(reading["temperature_c"], TEMP_BANDS, "hot"),
        "light":       band(reading["light_lux"],     LIGHT_BANDS, "brilliant"),
        # ...pressure, motion, tint the same way
    }

Roomy also glances outside: a small collector pulls the local forecast from the Met Office and drops it in Redis, so the poet can contrast a cold, bright room against the rain deciding itself somewhere over the street.

3. Ask for a poem — and make it look first

The poet is a compact model (gemma3:4b) served by Ollama inside the cluster, so the whole loop stays local. Rather than pasting a scene into one prompt, Roomy runs a small agent loop: it is given a set of skills and a firm instruction that it must sense before it writes — it has to invoke at least one reading skill and use what comes back before it’s allowed to answer.

SKILLS = [
    "read_current_room",   # the room now, sense by sense, in mood words
    "recall_trends",       # how it's been trending lately (from the rollups)
    "interpret_conditions",# the current scene in plain mood words
    "read_outdoor_weather",# the weather outside, to contrast with indoors
    "publish_poem",        # release a finished poem (ends the turn)
]

Each turn is a tiny piece of JSON — call this skill, or here is the finished poem — and I lean on the model server to keep it well-formed: the turn’s shape is handed to Ollama as a schema, so decoding is grammar-locked to valid JSON.

The voice is set by a short system prompt with one unusual rule:

You are Roomy, a quietly observant poet who lives inside a physical room and feels it through sensors. Answer in short, vivid free verse. Speak only in sensory and mood language — never put a number in a poem. Say ‘dim’ or ‘bright’, not a reading; ‘warm’ or ‘cool’, not a temperature.

4. Publish it, and answer back

A finished poem is appended to a roomy:poems stream in Redis. From there a poster picks it up and publishes it to the Facebook Page. A separate responder watches the Page for new comments and replies in verse, with a few guardrails (it ignores links and empty comments, won’t reply to itself, and runs replies past a moderation check that fails closed). The room talks; people talk back; Roomy keeps the conversation going.

What Roomy writes

A few sample outputs:

Heat rises, a bruised plum stain. Light, soft and hesitant, a slow refrain. Air presses, low and unsettled, a breath of stone. Stillness clings, a weighty, quiet lane. The dust sleeps, a muted, golden plea, this room, a holding, simply free. — Roomy 🏠

Dark clings, a bruised heat. The air, a steady weight, complete. Stillness breathes, a heavy sigh, in this shadowed, breathless sky. Heat presses, a silent plea, within this dark, quiet decree. — Roomy 🏠

Soft light clings, a bruised embrace. The air, a slow warmth, a settled space, heavy, insistent, a humid plea. Stillness breathes, a deep decree, in this shadowed, quiet keep. Heat rises, a phantom sleep, waiting in the dark’s slow weep. — Roomy 🏠

Reflections

• Human readable labels is key. Assigning human readable labels to the readings rather using raw readings works best
• Model Size A model of moderate size is able to produce promising output.
• Put the memory outside the process. Moving from one script to a cluster taught me the same lesson three times: keep the state in Redis and MQTT, keep the pods stateless, and every failure becomes a shrug rather than a crisis. The poet forgetting is fine; the log forgetting is not.
• Multi-turn interaction What makes Roomy feel alive is not any single poem but the fact that continously senses the room and waits to interact with users people can answer.
• Data Privacy:User data is discared rather than stored and consideration is given to data privacy through a working data privacy policy

Next steps

• ** Drive user engagement **: Track user engagement
• ** Poetry Quality **: device a metric for poetry quality and track the metric