LABEL CHECK / DOSSIER
Hermes Agent
closure trend: high release velocity; vendor sweeper (hermes-sweeper) auto-closed several open deltas 2026-07-12–07-15 with implemented_on_main verdicts of mixed reliability (one fix confirmed real but merged to main, not yet in a tagged release; one closure contested by the reporter's own follow-up comments); structural deltas (skill auto-invocation, 13.9K overhead floor, delegate_task billing, cross-platform env-override) remain open through v0.18.2
Verdict
A fast-moving, well-connected personal agent chassis whose headline self-improvement claim is not functional in any released version.
Claim ledger — 14 rows
Delta ledger — 5 open
Skill auto-invocation non-functional — self-improvement loop has no completion path
closes when:`skill_view()` auto-triggers reliably across N conversations, re-verified
evidence:[1]Re-verified open 2026-06-28 against v0.17.0Re-verified OPEN 2026-07-09 against v0.18.2 (state=open, last activity 2026-06-30)
13.9K-token fixed overhead floor contradicts marketed savings
closes when:Overhead floor materially reduced, re-measured on a real session
evidence:[1]Re-verified open 2026-06-28 against v0.17.0Re-verified OPEN 2026-07-09 against v0.18.2 (state=open, updated 2026-06-30)
Compression threshold_percent not updated on model switch/fallback (wrong compression trigger after `/model`)
closes when:`threshold_percent` recalculates (or is re-read from per-model config) on every model switch, fallback activation, and primary restore; re-verified
evidence:[1]Re-verified open 2026-06-28 against v0.17.0Re-verified OPEN 2026-07-09 against v0.18.2; corrected mechanism: issue is per-model threshold_percent sync on model switch, NOT integer 75→75-token coercion (that earlier characterization is unverified)UPDATE 2026-07-17: #18617 CLOSED 2026-07-12 by hermes-sweeper (not_planned/implemented_on_main), citing commit 76381e2a8 which re-derives threshold_percent on update_model() in both directions — content confirmed via `gh api` to match the reported bug, with regression test tests/agent/test_compression_small_ctx_threshold_floor.py. Commit is merged to main but is 41 commits ahead of the v0.18.2 tag (v2026.7.7.2) — not yet in any tagged release as of 2026-07-17. Falsification criterion met on main only; delta stays open pending a release that includes it.
`delegate_task` costs never persisted — 79% billing undercount
closes when:Subagent costs flush to state.db; recorded ≈ actual
evidence:unverified source, single measurement (no primary issue URL as of 2026-07-09)Re-verified open 2026-06-28 against v0.17.0Re-check 2026-07-09 against v0.18.2: gh search for delegate_task cost/billing/state.db found no matching issue; related open delegate_task bugs exist (#59386 schema, #52484 recursion, etc.) but not the 79% undercount. Delta retained open; primary-source gap remains.
Env override re-enables explicitly disabled platforms
closes when:`platforms.X.enabled: false` survives presence of the platform's bot-token env var (Telegram/Feishu and peers); re-verified
evidence:[1][2]Prior 2026-06-28 note citing #53697 was a mis-attribution (#53697 is Telegram streaming default bypass, not platform enable override)Re-verified OPEN 2026-07-09 against v0.18.2: #55113 (TELEGRAM_BOT_TOKEN forces platform on despite platforms.telegram.enabled: false) OPEN; #47804 (Feishu env forces enable) OPEN; #48820 (Windows env override ignores yaml) OPENUPDATE 2026-07-17: #55113 (Telegram) CLOSED 2026-07-15 by hermes-sweeper (not_planned/implemented_on_main), citing an explicit-disable marker in gateway/config.py and commit 156176a44e (2026-06-07); that commit has a diverged git history vs. the v0.18.2 tag, lowering confidence in the citation, but the closure comment's own re-check against current main is independently plausible. #47804 (Feishu) re-confirmed OPEN. #48820 (Windows) re-confirmed OPEN. Falsification criterion requires Telegram AND Feishu and peers — not met while Feishu/Windows remain open.
Closed deltas — 9 (falsification-gated)
`skills_guard` "ask" verdict hard-blocks agent-created skills
closed by:Closed not-planned + implemented-on-main 2026-06 (#13686); behavior change not TD-re-verified
Gateway deadlock on background skill creation, no recovery path
closed by:PR #39737 merged 2026-06 (#10138)
Single event loop is a systemic root cause — originally 7 P1/P2 blocking bugs; 5 of 7 now closed (#29184/#20525 via PRs, #28300/#20269 sweeper, #25351 not-planned), #13281 and root-cause #21026 remain open
closed by:All 7 tracked issues closed on tracker by 2026-07-12 (final two via hermes-sweeper automated triage, not_planned/implemented_on_main); #21026's cited architecture change (owned executor) confirmed in v0.18.2 release, but root-cause attribution contested by reporter follow-up and not TD-re-verified by soak test → closed_partial
Telegram FD/pool exhaustion wedges gateway while reporting connected — vendor pool-drain fix shipped; TD soak not re-run
closed by:PR #54121 merged 2026-06-28 (#53524 closed completed); pool-timeout drain ships, but 48h soak criterion not TD-re-verified → closed_partial
Slack cross-channel send silently fails (`isdigit()` on alphanumeric channel IDs)
closed by:PR #16198 merged (#15927) — channel ID validation fixed
Docker `--insecure` was default
closed by:v0.15.1 — now explicit env opt-in
WhatsApp accepted all senders by default
closed by:v0.13.0 — unknown senders rejected by default
Secret redaction was opt-in
closed by:v0.13.0 — on by default
Per-turn token cost (variable portion)
closed by:v0.15.0 — 47% reduction (399K→213K, 31-turn chat)
Methodology
- sampling
- Label-check methodology (D0086) — material-claims rule: a claim is sampled if it appears in the product's own description of itself and would plausibly influence an adoption decision. Claims sampled from official docs (hermes-agent.nousresearch.com), release notes, and the project README.
- verification depth
- source-reviewed — not executed by Theory Delta
- verification statement
- Evidence is drawn from the project's own code, issue tracker, release notes, and production reports filed by third parties; select claims depth-verified against source on 2026-06-08; re-verified against the issue tracker and v0.17.0 releases on 2026-06-28; re-verified again on 2026-07-09 against v0.18.2 (tag v2026.7.7.2) via `gh issue view` / `gh api` on every open-delta backing issue and key claim issues (#4589, #21026, #13281, #4379, #18617, #17244, #50918, #53524/#54121, #55113, #47804) plus hermes-agent-self-evolution README/push date. Drained reverify-queue deltas #2667/#2561 on 2026-07-17 against v0.18.2 (still latest tagged release, no newer tag as of this pass): re-checked #4589/#4379 (no change, still open), #21026/#13281 (closed 2026-07-12 by hermes-sweeper automated triage — cross-checked the cited commit SHAs with `gh api repos/.../commits/<sha>` and `gh api repos/.../compare/<tag>...<sha>` for ancestry against the v0.18.2 tag; treated as closed_partial given one contested root-cause attribution and one diverged-history citation), #17244 (closed, SSE transport confirmed shipped in v2026.5.7 but dynamic discovery still manual), #18617 (closed, fix commit confirmed real and on-target but not yet in a tagged release — 41 commits ahead of v2026.7.7.2), #55113/#47804/#48820 (Telegram closed, Feishu/Windows still open — HM-10 falsification criterion not met). Theory Delta has not executed Hermes in its own environment for this dossier. Star counts (~212K as of 2026-07-09) are recorded in the backing block only and remain excluded from the adoption assessment.
- backing confidence
- secondary-research
- strongest case against
- Hermes's release cadence is fast enough that any specific issue may close within weeks; v0.15–v0.18 refactors and the PR #54121 Telegram pool drain may have changed runtime behavior not fully reflected in older open threads. The self-improvement bugs may be model-size or configuration-sensitive rather than universal. HM-09 (delegate_task billing undercount) still lacks a primary issue URL. Re-verification cadence for this tool should remain high (decay_class: volatile). Note 2026-06-28: prior delta HM-11 (claimed P0 session-auth exposure) was removed because no primary-source URL was ever attached. Note 2026-07-17: an automated vendor triage bot (hermes-sweeper) closed five open issues 2026-07-12–07-15 with self-certified "implemented_on_main" verdicts; spot-checking the cited commit SHAs found one genuinely on-target fix not yet in a tagged release, one closure whose root-cause attribution is contested by the original reporter's own follow-up comments, and two citations with diverged git history relative to the tag under audit — future re-verification passes should treat hermes-sweeper closures as a lead requiring independent commit inspection, not as ground truth.