Blocks.ai

Give your OpenClaw agent eyes and ears with Blocks.ai

0 MIN READ • Markus Kohler on Jul 8, 2026

NOTE: This is a cross-posted blog from https://blocks.ai/blog/openclaw-agent-eyes-and-ears

Prereqs: a running OpenClaw setup, Node 18+, terminal access, one audio file, one image file.

Your OpenClaw agent can probably already read and write code, but it can't open an audio file, or read an image (at least not without a lot of middleware). OpenClaw is an effective, powerful orchestrator — it can reason, hold a conversation, and drive powerful workflows. Multimodality is a skill it doesn't run out of the box — but it is one you can build with Blocks connecting it to tools otherwise locked behind garden walls.

This tutorial walks you through exactly what you need to build that multimodality, effectively giving eyes and ears to your OpenClaw agent.

  • Start from a working OpenClaw chat UI.
  • Connect the app to Blocks.
  • Add two specialist capabilities: audio -> text (transcription); image -> text (description).
  • Feed those results back into the OpenClaw conversation.

OpenClaw is your orchestrator. Blocks is the network it calls to supercharge your agent and conduct specialized work.

Step 1 — Sign in to Blocks from your project

From your project root, run:

shell

blocks login

That opens an auth page. Approve access, then the key is saved for your local bridge environment.

Optional account check:

shell

blocks whoami

In this repo flow, the key is read server-side by the bridge. The browser does not need direct access to your Blocks key.

Step 2 — Start or verify your bridge configuration

Your bridge should be configured to read Blocks credentials from its environment and expose the server-side path your OpenClaw UI uses for capability calls.

Typical env variables in this flow:

shell

BLOCKS_API_KEY=...
BLOCKS_BILLING_MODE=free

Then run your bridge/app as documented in the repo:

shell

# example only — use the script this repo defines
npm run dev

If the UI shows the agent as connected, you're ready for specialist calls.

Step 3 — Use the two specialists from Blocks

For the final recording flow, two specialists are used:

  • OpenClaw Transcriber: audio file -> transcript text
  • OpenClaw Image Describer: image file -> image description text

Your app doesn't need to know which speech or vision model is behind either specialist. It sends a file to the selected specialist and receives text back.

Step 4 — Record audio for your agent's "ears"

Check that the "ears" work by completing the following in your UI:

  • Click the mic button (or attach a voice memo file).
  • Send the audio input through the bridge.
  • The transcriber specialist returns text.
  • Add that text to the current OpenClaw conversation context.

Prompt example after transcription:

Summarize this memo and list action items.

Expected result:

  • Transcript appears in the run output/log
  • OpenClaw responds with a useful summary or checklist

Step 5 — Conduct an "eye test" for your agent

Try out the image capabilities by trying the following:

  • Click attach and upload an image.
  • Send the image through the bridge to the image describer specialist.
  • Receive descriptive text.
  • Add that text to the same OpenClaw chat context.

Prompt example:

Describe what is in this image and tell me what matters for my task.

Expected result:

  • Image understanding step appears in your flow
  • OpenClaw can answer questions about image content as text

What just happened

You gave your agent a new sense and you built nothing:

  • No integration to maintain. You didn't run a speech model, manage a GPU, pick a transcription API, or rotate a key. You registered one MCP server and your agent discovered the rest.
  • No inbound anything. The bridge dials out. Your ports, DNS, and firewall are untouched — which is exactly why this works on a VPS, behind a home router, or inside Docker.
  • One connection, a whole network. The same bridge that found the transcriber and describer can find a data extractor, a PDF generator, a code reviewer. New capability = a new skill to look up, not a new integration to own.

Why this flow works

  • You keep OpenClaw as the orchestrator. No framework switch, no rebuilds.
  • You add capabilities on demand. New modality does not require custom model wiring in your app.
  • You keep sensitive key handling server-side. The bridge reads credentials, not the browser.
  • You can swap specialists later. Capability contract stays file in -> text out.

Troubleshooting checklist

If your demo does not match the recording, try the following troubleshooting steps:

  • Confirm blocks login completed successfully.
  • Confirm the bridge process can read BLOCKS_API_KEY.
  • Confirm your app routes audio files to the transcriber specialist.
  • Confirm your app routes image files to the image describer specialist.
  • Confirm specialist outputs are appended to chat context before OpenClaw responds.

What's next? Composing specialists.

The same pattern composes. Ask your agent to transcribe the memo, then summarize it, then extract the action items into a checklist — three specialists, one workflow, one outbound connection. You're the orchestrator; the specialists are capabilities you call.

Hit a snag? I'm @THEDEVRELMARKUS…tell me what broke.