Porting Home Assistant integrations to Personal Agent — plan & gap analysis

Research against a fresh shallow clone of home-assistant/core (1 484 integrations, homeassistant/components/*), mapped onto our existing integration tier (integrations/<domain>/ + src/personal_agent/integrations/ in the backend repo). This is the porting counterpart to docs/home-assistant-adoption.md (which deepens our entity/flow/dashboard subsystems). This doc answers: can we port all HA integrations, what does it take, and which framework pieces are still missing.

0. TL;DR

• HA core ships 1 484 integrations; ~884 have a config flow, pulling in 1 127 distinct PyPI requirements. They are not a uniform set — they split sharply by transport.
• Our integration tier is already an HA-shaped base: folder discovery + manifest.yaml + ConfigFlow + PersonalAgentIntegration (≈ async_setup_entry) + an entity system with device_class/state_class/unit/category/actions/device grouping + pull/push/webhook sync + capability providers. The shape matches; the breadth and transports do not.
• “Port all” is the wrong target. ~½ of HA assumes local radio/serial hardware on the same host as the runtime (Zigbee, Z‑Wave, Bluetooth, Matter/Thread, KNX, USB, serial). We run multi‑tenant in k8s/cloud — there is no LAN, no radio, no per‑tenant USB stick. Those need a local bridge (the Rust Computer Service) before they make any sense.
• The high‑value, directly‑portable slice is the cloud‑API integrations (cloud_polling / cloud_push, ~500): REST/OAuth services with no local hardware (weather, calendars, transit, energy, media, package tracking, smart‑home clouds). These map cleanly onto our existing model with no new framework — only the gaps in §4 (OAuth2, richer entity domains, discovery for the LAN tier).
• Recommended path: (1) close the small framework gaps that unblock the cloud tier (OAuth2 application‑credentials flow, a handful of richer entity domains, a codegen harness), (2) bulk‑port the cloud tier via a manifest‑driven generator, (3) add a device‑agent LAN bridge to unlock local‑IP devices, (4) treat radio/serial hubs and voice as separate programs, not ports.

1. What “all HA integrations” actually is

The single most important axis for us is iot_class locality:

• cloud_* (≈504) — talks to a vendor cloud over HTTPS. Runs fine from our cloud.
• local_* (≈672) — talks to a device on the same LAN as the HA host. Needs LAN reach.
• calculated/assumed_state/system/helper/virtual — derive state or are pure framework.

2. Concept mapping — HA → Personal Agent (what already lines up)

Takeaway: the per‑entry contract is essentially complete, and most framework gaps have since landed (OAuth, typed entity domains, history/stats, dependency extras, options/reauth, integration actions - see §4). The remaining gaps are discovery (§4.2), transport for the local tier (§4.3/§4.5 - no device‑agent LAN bridge yet), and the bulk‑port codegen harness (Phase 0).

3. Portability tiers — how to bucket all 1 484

The numbers overlap (a hub can be both cloud and local); treat them as planning buckets, not a partition. Tier A is the program. Tier B is the stretch. Tiers C/E are separate initiatives.

4. Framework / function gaps — what we still lack

Ordered by how much they unblock. Each is additive on the existing tier.

!!! note “Status (verified against the backend repo)” Most of the Phase-0/1 framework gaps have since SHIPPED: §4.1 OAuth2 (integrations/oauth2.py, FlowResultType.EXTERNAL_STEP, admin oauth2_application_credentials store), §4.4 richer entity domains (integrations/entity_domains.py, the 10 canonical typed descriptors), §4.6 history + statistics (entity_state_history + entity_state_statistics roll-up), §4.7 dependency extras (manifest.requirements_extra), §4.8 options/reauth/reconfigure (flow_manager flow_kind), and §4.9 integration-level actions (integrations/actions.py). Still open: §4.2 discovery, §4.3 radio/serial, §4.5 Computer Service LAN bridge (the Computer Service has no LAN/discovery runner yet), and the Phase-0 codegen harness (tools/ha_port/ does not exist).

4.1 - OAuth2 / application‑credentials config flow (P0 - SHIPPED)

HA has a first‑class config_entry_oauth2_flow + application_credentials: the flow redirects to the provider, captures the code at a callback, exchanges + refreshes tokens, and stores them on the entry. The original gap: our ConfigFlow only did in‑app forms (FORM/CREATE_ENTRY/ABORT) with no external‑redirect step, no token store, no refresh.

• Done: FlowResultType.EXTERNAL_STEP (+ external_url, async_external_step) with the callback resuming async_step_<step_id>; an OAuth2Session helper (integrations/oauth2.py) building the authorize URL, exchanging the code, and refreshing on expiry; an admin oauth2_application_credentials store (OAuth2ApplicationCredential model + admin_oauth_credentials router) keyed by domain; the canonical redirect URI {APP_ORIGIN}/oauth/callback injected via FlowContext.redirect_uri. Tokens are stashed as secret_fields on the flow and envelope‑encrypted at rest; never logged or in Temporal inputs (Contracts #5/#15).

4.2 — Discovery (zeroconf / SSDP / DHCP / Bluetooth / USB / HomeKit) (P2, UX for the LAN tier)

~250 HA integrations declare discovery so a device is found, not hand‑entered. We have none; every entry is manual host/credentials. Not a hard blocker for cloud (Tier A never discovers), but the LAN tier (Tier B) leans on it heavily.

• Add (only with the §4.5 bridge): a discovery channel from the device‑agent (it is on the LAN) → posts mDNS/SSDP/DHCP hits back → a “discovered integration” inbox that pre‑fills a config flow. Pure‑cloud deployments simply never see discoveries. Defer until a concrete Tier‑B port needs it.

4.3 — Local radio/serial transport subsystems (P3, gates Tier C)

Zigbee, Z‑Wave, Matter/Thread, Bluetooth/BLE, KNX, MQTT broker, serial/USB. These are whole stacks (zigpy, zwave‑js‑server, python‑matter‑server, bleak, aiomqtt) that assume a radio adapter on the host. A cloud multi‑tenant runtime cannot host them.

• Only viable via a co‑located bridge (§4.5) running the radio stack next to the hardware, exposing a normalized device API back to PA. This is a product line, not a port. Recommend defer; if pursued, start with MQTT (a broker the user already runs → cleanest bridge) and Matter (IP‑based, future‑proof) before Zigbee/Z‑Wave (per‑adapter drivers).

4.4 - Richer entity domains (P1 - SHIPPED)

The original gap: our EntityStateTypeDescriptor was generic (state + attributes + actions), while HA has ~40 typed platforms with domain semantics: climate (hvac_modes, target_temp, presets), media_player (transport state, source list, volume), cover (position/tilt), light (color modes/temp), fan, lock, vacuum, alarm_control_panel, number/select/button/switch, weather, calendar, todo, camera/image, update, device_tracker, event, date/time/datetime/text.

• Done: integrations/entity_domains.py ships the 10 canonical typed descriptors - climate, media_player, cover, light, lock, vacuum, weather, calendar, todo, camera - each a pre‑populated EntityStateTypeDescriptor with HA‑aligned standard action sets (filterable per entry via _filter_actions). The bare generic descriptor stays the fallback (backward‑compatible).

4.5 — Device‑agent LAN bridge (the local tier’s enabler) (P2, gates Tier B+C)

The Rust Computer Service (repo personal-agent-org/computer-service) already connects back over the device WS. Extend it into a LAN integration runner: a slim host that the user runs at home, which (a) reaches local devices, (b) optionally hosts radio stacks (§4.3), © relays discovery (§4.2), and (d) runs the local‑transport half of a ported integration while the config flow, entities, governance, and agent stay in the cloud. This is what turns “cloud‑only product” into “can talk to your house” without putting tenant hardware in our k8s.

4.6 - Long‑term statistics / recorder (P2 - SHIPPED)

HA’s recorder + statistics back history graphs and “has been X for Y” conditions. Done: the sync engine writes one entity_state_history row per state change, and entities/statistics.py rolls older numeric data into per‑period min/max/mean (hour buckets over raw rows, day over hour) in entity_state_statistics. Both are fail‑closed scope_isolation RLS and bounded by the admin entity_history_retention_days setting (default 30); history_retention.py is the coarse backstop sweep.

4.7 - Dependency packaging at scale (P1 - SHIPPED, extras model)

1 127 distinct PyPI requirements: porting hundreds of integrations means hundreds of new transitive deps, unworkable in one image.

• Done: the opt‑in extras model. manifest.requirements_extra names the pyproject [project.optional-dependencies] key that installs an integration’s requirements (defaults to integration-<domain>). An operator installs only what they need (pip install 'personal-agent[integration-foo]'); when the requirements aren’t importable the loader marks the integration unavailable and names the extra in the message instead of loading‑then‑crashing. The full‑trust, first‑party trust model is unchanged. (A per‑integration sidecar/venv was not pursued.)

4.8 - Options / reauth / reconfigure / lifecycle (P1 - SHIPPED)

Done: the FlowManager opens an entry‑edit with flow_kind ∈ reconfigure | options | reauth, entering async_step_<flow_kind> (each falling back to async_step_user); simple_flow.py provides a standard async_step_reconfigure. Config rows carry a state column (loaded | not_found | error) and a runtime health blob (ok | degraded | error from async_health). Reconfigure preserves untouched fields and decrypts existing secrets for the form.

4.9 - Service/automation surface parity (P3 - SHIPPED)

HA services.yaml (339 integrations) are global, schema’d actions; entity actions are entity‑scoped (async_call_action). Done: integrations/actions.py adds the integration‑level registry - PersonalAgentIntegration.integration_actions() returns IntegrationActionDescriptors (name → tool, fields → schema) and async_call_integration_action runs them, returning a structured IntegrationActionResult. Covers the few integration‑level services (e.g. “send a notification”, “run a scene”) that don’t fit the entity‑action model.

5. Recommended plan (phased)

Phase 0 — Harness & decision (1–2 wk). Build a manifest‑driven codegen + triage tool (tools/ha_port/): read a HA component’s manifest.json + entity platforms, classify it into a Tier (§3), and emit a PA integrations/<d>/ skeleton (manifest.yaml + config_flow stub + entity‑type stubs mapped from HA platforms). Output a ranked backlog (Tier A first, by popularity). Deliverable: the backlog + 3 hand‑finished Tier‑A ports as templates.

Phase 1 - Unblock the cloud tier (P0/P1 framework). - DONE. §4.1 OAuth2 flow, §4.4 richer entity domains (the 10 canonical ones), §4.7 dependency extras, and §4.8 options/reauth have all landed (plus §4.6 history and §4.9 actions). These were the only hard framework prerequisites for Tier A.

Phase 2 - Bulk‑port Tier A (cloud APIs, ~500). - NOT STARTED. The Phase‑1 framework is in place, but no bundled folder integration yet uses the OAuth flow, the typed entity domains, or integration‑level actions. Generator‑assisted, hand‑finished in priority order. Each port = manifest + config flow (often OAuth) + a thin client + entity types + actions. Most need no device‑agent. Wire each through the existing governance/classification/untrusted gates (Contracts #13/#14) — a cloud integration that returns attacker‑influenced text declares trust_tier: untrusted.

Phase 3 — LAN bridge + Tier B (local‑IP devices). Land §4.5 device‑agent LAN runner + §4.2 discovery relay (§4.6 history already landed). Port the local‑IP devices (ESPHome, Hue bridge, Sonos, printers, routers) running their transport half on the bridge. Gate: ESPHome or a Hue bridge controllable from chat via a home‑run bridge.

Phase 4 (optional, separate program) — Tier C radios + Tier E voice. Only if there’s product pull. Start MQTT/Matter. Voice is the model‑pipeline team, not this tier.

6. Per‑integration porting recipe (the repeatable unit)

integrations/<domain>/
  manifest.yaml      # from HA manifest.json: domain, name, iot_class, requirements,
                     #   config_flow, single_instance, codeowners, documentation;
                     #   set required_tier + trust_tier per our governance
  __init__.py        # class PersonalAgentIntegration:
                     #   async_setup_entry  ← HA async_setup_entry (build toolset/client)
                     #   entity_types()     ← HA entity platforms → EntityStateTypeDescriptor
                     #   async_sync_entities← HA coordinator.async_update (pull)
                     #   async_call_action  ← HA services / entity methods
                     #   *_provider()       ← if it backs a capability (weather/search/notify)
  config_flow.py     # HA async_step_user (+ OAuth EXTERNAL_STEP, §4.1)
  client.py          # the vendor SDK/REST wrapper (HA uses the requirement lib directly)
  mapping.py         # HA state/attrs → EntityStateRecord
  translations/<l>.json

Rules of thumb when porting:

• Pull over push first — implement async_sync_entities (poll); add async_handle_webhook/ listener only for *_push integrations that matter.
• Map HA platforms to our device_class/state_class/unit verbatim (we reuse HA’s vocabulary) so semantics carry over.
• Don’t hardcode per‑domain lists anywhere in core — derive from declarations (Contract: comms triage / listeners already do this).
• Governance: set required_tier by data sensitivity; trust_tier: untrusted for any external/attacker‑influenced text source so the assembler gates high‑privilege tools (Contract #13).
• Keys (OAuth tokens / API keys) live in the encrypted secret envelope, decrypted only inside SetupContext.secrets; never logged, never in Temporal inputs (Contract #5/#15).

7. Non‑goals / explicit cut lines

• Not porting HA’s frontend Lit cards, Lovelace strategies, or hassio/supervisor/backup/ cloud/onboarding system integrations — they’re HA‑host internals.
• Not hosting radio stacks in the cloud runtime (security + multi‑tenancy + no hardware).
• Not replicating recorder as a TSDB — a bounded entity_state_history + stats roll‑up only.
• Not the voice assist pipeline in this tier — it belongs to the model pipeline.
• “All 1 484” is explicitly re‑scoped to “all that make sense for a cloud‑first agent”: Tier A in full, Tier B via the bridge, Tier C/E as separate, opt‑in programs.

8. Bottom line

Shipped since this plan (verified against the backend repo): OAuth2 application‑credentials flow (§4.1), dependency extras (§4.7), the 10 canonical typed entity domains (§4.4), options / reauth / reconfigure lifecycle (§4.8), entity history + long‑term statistics (§4.6), and the integration‑level action registry (§4.9).

Still open:

1. Device‑agent LAN bridge (runs the local‑transport half at home) - P2, gates Tier B/C.
2. Discovery relay (zeroconf/SSDP/DHCP/BLE/USB via the bridge) - P2.
3. Local radio/serial subsystems (Zigbee/Z‑Wave/Matter/Thread/MQTT/KNX) - P3, separate program.
4. A manifest‑driven port generator (tools/ha_port/) to make the bulk feasible - Phase 0.

Everything else (the per‑entry contract, entity sync, actions, devices, webhooks, capability providers, governance, multi‑tenancy) already exists and matches HA’s shape.

Compiled 2026‑06 from a shallow home-assistant/core clone (1 484 integrations). Pairs with docs/home-assistant-adoption.md; numbers are from the clone’s manifests at time of writing.