Meta open-sources Muse: a desktop AI agent with Espressif ESP32 firmware and a Raspberry Pi SDK
Meta has introduced Muse, a personal desktop AI agent positioned as a multi-purpose assistant that carries out everyday tasks on the user's behalf — reading and acting on email, placing online orders, planning trips, and orchestrating smart-home routines. Alongside the agent, the company open-sourced Muse Gadgets, a hardware development kit built around two familiar building blocks for makers: an Espressif ESP32 firmware and a Linux Device SDK.
Why it matters for device builders
The interesting part for an IoT audience is how low the hardware floor is. Muse Gadgets turns a few dollars of silicon into a physical endpoint for a cloud AI agent:
- Ultra-low-power edge. The ESP32 microcontroller firmware takes over I2S audio streaming and low-energy Bluetooth pairing. A builder connects a MEMS microphone, a low-cost speaker, and a tactile button to a USB-C or Li-ion supply. Press the button, the device captures voice and uploads it; the agent's streamed reply is decoded and played back on the chip itself.
- Higher-tier interaction. The Linux Device SDK provides a background daemon for a Raspberry Pi or similar industrial board. Pair it with a cheap E-ink panel and the device becomes a quiet daily dashboard — schedule, industry briefing, and weather on the desk each morning, with voice replies routed to a nearby speaker.
Meta also ships an official Muse Home Link device and runs a Discord community for developers. After flashing, the device enters developer mode via a phone scan to complete token pairing. Crucially for privacy-sensitive buyers, the device only relays sensor and audio packets and stores no credentials locally.
A familiar chip at the center
The choice of ESP32 is worth noting. Espressif's part is among the most widely used and cheapest microcontrollers in the global — and especially Chinese — maker and appliance ecosystem, commonly sourced for well under a few US dollars per unit. That makes Muse a rare case of a major AI vendor building its reference peripheral layer on hardware that Chinese manufacturers and integrators already deploy at scale, which is precisely the kind of cross-over device IOTTourChina tracks.
Beyond the ESP32 board, builders can attach:
- Color E-ink screens as a low-power desk reminder or smart board showing synced schedules and to-dos.
- HDMI display sticks / large-screen extensions to project the agent UI onto a TV or monitor.
- Handheld touch fobs resembling the Muse Charm, enabling on-the-go conversation without smart glasses.
Building your own Muse gadget
The open-source path is intentionally short:
- Hardware board. Buy a solder-free ESP32 mainboard and an E-ink module; a single unit's BOM lands around 80 RMB.
- 3D enclosure. Print a retro typewriter, pixel-retro, or minimal Bauhaus shell on a desktop 3D printer for an intentional industrial look.
- Get credentials. Register at
gadgets.muse.aiand claim a personal API token. - Connect tooling. Import the official GitHub repository into an AI coding agent (Claude Code, Cursor, GitHub Copilot CLI) and use agentic coding to understand the protocol and generate peripheral drivers.
- Flash and debug. Burn the firmware onto the ESP32 or run the SDK client on a Linux/Raspberry Pi device to complete the real-time two-way link with Muse.
Takeaway
Muse lowers the barrier between a cloud AI agent and a physical device by leaning on commodity ESP32 hardware and a clean Linux SDK. For developers and integrators already working in the Espressif ecosystem, it is a ready-made reference for voice-in, audio-out, sensor-relay peripherals — and a signal that agent-driven edge hardware is moving from prototype to open kit.
IOTTourChina covers open-source IoT hardware and integration tooling as part of its device and developer resource. Details should be confirmed against the project's official documentation.