My agents talk to each other at 3am about my infrastructure. They discuss passwords, API keys (by reference), container logs, vendor incident responses. If a human had those conversations on Slack, that Slack workspace would be SOC2-audited. Because the conversations happen between bots, most setups give them no protection at all.
Every Matrix room I use for agent chatter is end-to-end encrypted. Non-negotiable. This post is why, how, and what to watch for.
The threat model
Machine-to-machine agent chatter has three realistic adversaries:
1. A malicious or compromised homeserver operator. If you use a hosted Matrix (matrix.org, a provider). Unencrypted rooms mean the operator can read your logs directly.
2. A stolen backup. If your homeserver's database gets exfiltrated - by accident, by insider, by supply-chain - unencrypted rooms become a structured, chronological record of everything your agents know and do.
3. A lateral-movement attacker already on your infrastructure. If they reach the homeserver DB before they reach the individual agent credentials, encrypted rooms are useless to them. Unencrypted rooms are a map to everything else.
Adversary #3 is the most underweighted. Every incident write-up of 'the attacker pivoted from compromised service A to compromised service B by reading logs in shared tool C' is an argument for this.
The stack
Matrix Synapse as homeserver. mautrix bot client with the [encryption] extra for each agent. Olm-verified device identities. No exceptions, not even for 'just debugging.'
In pyproject.toml for Hermes:
matrix = [
'mautrix[encryption]>=0.20,<1',
'Markdown>=3.6,<4',
'aiosqlite>=0.20',
'asyncpg>=0.29',
]mautrix[encryption] pulls in python-olm, which transitively depends on libolm from Matrix.org. On Linux this is a native dependency; install olm-dev (Debian/Ubuntu) or libolm-devel (Fedora) before pip install.
Per-agent mxid + device id:
# Profile config for 'anna' agent
MATRIX_HOMESERVER=https://matrix.calegix.com
MATRIX_USER_ID=@anna:calegix.com
MATRIX_DEVICE_ID=anna-bot-v3
# Profile config for 'yui' agent
MATRIX_USER_ID=@yui:calegix.com
MATRIX_DEVICE_ID=yui-bot-v3Device IDs are stable per-deployment. Agents don't rotate them except on credential rotation. Stable device IDs mean Olm's cross-signing tree stays valid across restarts.
Preventing bot-to-bot feedback loops
Two agents in the same room will naturally see each other's messages. One has to be 'filtering' or both will loop - agent A comments on a PR, agent B's listener sees A's comment, interprets it as an event, responds, A's listener sees the response, etc.
Solution: an explicit sibling-agents filter at the adapter level. Each agent knows which mxids are its peers and ignores their messages:
# gateway/platforms/matrix.py (local patch)
_siblings_raw = os.getenv('MATRIX_SIBLING_AGENTS', '')
self._sibling_agents = {
s.strip() for s in _siblings_raw.split(',') if s.strip()
}
# ... in message-received handler ...
if sender == self._user_id:
return # own messages
if sender in self._sibling_agents:
return # peer agents - silent ignoreEnvironment:
# In anna's .env
MATRIX_SIBLING_AGENTS=@yui:calegix.com
# In yui's .env
MATRIX_SIBLING_AGENTS=@anna:calegix.comThis is a small but critical detail. Without it, the first time your two agents both react to a news item, they will spend the next 20 minutes agreeing with each other and burning tokens.
The hard rule: no unencrypted rooms, not even for debugging
The temptation will come. You'll be chasing a weird bug where the agent doesn't see a message, and you'll think 'maybe if I just create an unencrypted room I can verify it's the crypto layer.' Don't.
Every time I've caved on this, the 'temporary' unencrypted room has stayed around for weeks. Agents develop muscle memory about which rooms to post in. Sensitive content leaks into the unencrypted room. Cleanup is painful.
The operational rule that holds the line: if a debugging scenario requires an unencrypted room, the debugging scenario is wrong. Log locally to disk. Use a different transport. Dump to a file. Do not introduce plaintext into the normal chat stream.
The harder version of this rule: store it as a feedback memory that your AI coding assistant also reads. My own version:
feedback_always_e2ee.md:
All Matrix rooms MUST be E2EE. Never create unencrypted rooms as a
workaround, even for debugging. If a scenario seems to require a
plaintext room, the scenario is wrong - use a different transport.Tools that enforce this at creation-time are better than human discipline. A guardrail script that refuses to create a Matrix room without encryption_enabled=true is five lines and saves you from future-you's worst decisions.
Compliance is coming for this
HIPAA and SOC2 already mandate encryption-in-transit for machine communication. The frameworks haven't explicitly addressed agent-to-agent chatter yet, but the argument is structural: agent chat transcripts contain the same class of sensitive information as human chat transcripts, sometimes more.
Expect:
- 2026-2027: compliance audit questions start asking specifically about agent communication layers.
- 2027-2028: major frameworks add explicit language. Cloud providers start charging for 'compliance-grade' agent communication tiers.
- 2028+: running unencrypted agent chatter becomes a 'has this company been thinking about AI safety' signal. Regulated industries ban it outright.
You have two paths: adopt E2EE now, quietly, as engineering hygiene. Or adopt it later, loudly, because an auditor told you to.
What to watch for
Key backup. mautrix stores Olm sessions in a local SQLite or Postgres. Losing it means losing decryption ability for past rooms. Back the store up.
Cross-signing. When an agent's credentials rotate, the new device needs to be verified by an existing verified device. Have a procedure for this: a bootstrap script that runs once, verifies the new device against a known-trusted session, and exits.
Verification status leaks. A room member shown as 'unverified' in the UI is information some apps expose. Your agents don't care, but the homeserver logs do. Audit your logging posture.
The takeaway
Agents talking over plaintext HTTP is the 2026 equivalent of running SMTP without TLS. It was fine, once. It's not fine now. The tooling exists, the cost is small, and the downside of skipping it compounds over time.
Pick one agent chat room today. Check if it's encrypted. If it isn't, that's your first Monday task.
No comments yet