# Multi-Agent Workgroup — log

- 2026-10-30 scaffolded from templates/PROJECT.*.tmpl
- 2026-10-30 built the first 3 custom tools, each as a script + schema pair, all tested
  with real runs: `system_health.py` (host/cron/gateway/Drive-token health in one JSON,
  3s), `book_progress.py` (per-chapter word counts from the Drive Book Sections folder,
  stdlib-only Drive access) and `new_tool.py` (scaffolds the next pair and self-checks it).
  Two real bugs found and fixed by testing: book_progress counted the BOM in empty Google
  Docs as a word (Ch 3-10 looked like they had content), and new_tool reported success
  without verifying the generated pair. Next: HF MCP connector (needs HF token), then wrap
  the tools as MCP tools.
- 2026-10-30 tool 4 + first automation: `health_watch.py` turns `system_health` into an
  hourly Telegram watchdog (cron `2029a25b3d3a`, via shim `~/.hermes/scripts/watch_system_health.py`
  because cron only runs scripts under `~/.hermes/scripts/`). Alerts only on a new problem,
  a problem still present after 12h, or a recovery — so a multi-day T130 outage sends 2 pings
  a day, not 24. All 7 state-machine cases tested with monkeypatched reports (first-run quiet,
  alert, de-dupe, re-alert window, changed problem, recovery, quiet-when-healthy). Also
  confirmed every schema file parses as strict UTF-8 JSON with no BOM.
  Next: HF MCP connector (needs an HF read token in `~/.hermes/.env`), then wrap the tools as MCP tools.
- 2026-10-31 MCP connector #1 live: Hugging Face. The catalog entry (`optional-mcps/hugging_face`)
  advertises OAuth, but this is a headless box, so `hermes mcp install` / `login` could not
  complete consent — install "succeeded" with zero working tools. Probed the endpoint directly
  and found it also accepts a static bearer token (tokenless `initialize` also returns 200, so
  the real proof was `hf_whoami`: anonymous without a token, `@criticalmike` with one). Mike
  minted a Read token; stored as `MCP_HUGGING_FACE_API_KEY` in `~/.hermes/.env` (chmod 600);
  re-added with `--auth header` so config carries `Authorization: Bearer ${MCP_HUGGING_FACE_API_KEY}`.
  Verified `hermes mcp test` connects (1389ms, 4 tools) and made two real tool calls
  (`hf_whoami` → authenticated; `hub_repo_search` Qwen3 → 5 models). Tools: `hf_whoami`,
  `hub_repo_search`, `hub_repo_details`, `hf_fs`. Note `hub_repo_search` takes `repo_types`
  (a list), not `repo_type`.
  Next: define Paperclip.
- 2026-10-31 MCP connector #2 live: `workgroup` — the project's OWN tools, wrapped. New
  `scripts/mcp_server.py` is a stdio MCP server exposing `health` (= `system_health`) and
  `book_words` (= `book_progress`); it imports the two modules by path rather than
  reimplementing them, so each tool still has exactly one implementation. Registered as
  `mcp_servers.workgroup` (command = the venv python, args = the script path).
  Two things worth remembering:
  (a) The MCP Python SDK here is **v2**, where `FastMCP` was renamed — `from
  mcp.server.fastmcp import FastMCP` raises `ModuleNotFoundError`; use `from
  mcp.server.mcpserver import MCPServer` (`MCPServer("name", version=...)`, `mcp.run()` =
  stdio by default). v2 also moved transport args (`port`, `host`) off the constructor onto
  `run()`.
  (b) Wrapping immediately found a REAL BUG that the earlier testing missed: `check_bookapp`
  treated any HTTP >= 400 as a failure, so once the bookapp got Basic Auth (2026-10-31) the
  health tool reported it FAILING while it was perfectly healthy — only a 401. Fixed: 401/403
  now report `ok` with `auth_required: true`. Full run is `overall: ok` on all 6 checks.
  Verified three ways: raw stdio JSON-RPC handshake (initialize -> tools/list -> call both
  tools), `hermes mcp test workgroup` (connects, 2 tools), and a real one-shot Hermes session
  (`hermes -z`) that called `mcp_workgroup_health` and returned the live report. MCP tools only
  register on a NEW session — no hot-reload.
  Next: define what Paperclip is; then a documented multi-agent run.
- 2026-10-31 **Paperclip identified** — the keyword that sat undefined in this project's objective
  since it was created. It came from a context block Mike pasted
  (`~/.hermes/pastes/paste_1_155113.txt`): *"Workstation: Dell OptiPlex 9010 (daily interface for
  Paperclip & VS Code)"*. Paperclip is `github.com/paperclipai/paperclip` — an **open-source app
  that runs a team of AI agents like a company** (MIT, ~74k★, by pseudonymous dev `@dotta`,
  launched March 2026). It is explicitly NOT an agent framework: it is the management layer above
  agents — org chart with roles/reporting lines/permissions, a ticket board, per-agent budgets with
  hard stops, heartbeat scheduling, human approval gates, and a durable audit log. Node.js server
  + React UI on `localhost:3100` with an embedded PostgreSQL, so **no Docker and no DB setup**.
  Its rule: *"If it can receive a heartbeat, it's hired."*
  **The finding that matters:** Nous Research ships an official adapter —
  `NousResearch/hermes-paperclip-adapter` (`npm install hermes-paperclip-adapter`, adapter type
  `hermes_local`) — which hires THIS Hermes as a managed employee: Paperclip wakes Hermes on a
  heartbeat with the task, Hermes works with its full tool suite (including the `workgroup` and
  `hugging_face` MCP tools already built here), and reports back. The adapter reads
  `~/.hermes/config.yaml` for the model and scans `~/.hermes/skills/`. So the objective is now
  concrete: **Paperclip is the org, Hermes is the employee.**
  **Blocker:** Paperclip needs **Node.js 24.11+**; this host has **v22.23.3** (no nvm). Upgrade Node
  before installing.
  Install when ready (the `tailnet` bind suits the Tailscale setup):
  `npx paperclipai@latest onboard --yes --bind tailnet`.
  Next: upgrade Node, install Paperclip, register the Hermes adapter, documented multi-agent run.
- 2026-10-31 **Paperclip INSTALLED and running** on the 3050 (Mike approved the system change).
  - **Node:** Paperclip needs >= 24.11. Installed **Node 24.22.2** to `~/.local/node24` —
    deliberately SEPARATE, because the `node` on PATH was Hermes' own bundled runtime
    (`~/.local/bin/node -> ~/.hermes/node/bin/node`, v22) and overwriting that could break
    Hermes. `~/.hermes/node` is untouched (still v22.23.3). Node 24 is put first on PATH by
    **`~/.profile`** (after the `~/.local/bin` prepend, otherwise the prepend wins) and by
    `~/.bashrc`. Gotcha: an export appended to the END of `~/.bashrc` never runs for a
    non-interactive shell — its early `case $- in *i*) ;; *) return;; esac` bails out first.
  - **Install:** `npx paperclipai install --yes` → managed CLI **2026.1023.0** in `~/.paperclip`,
    `paperclipai` shim in `~/.local/bin` (no sudo anywhere).
  - **Running:** `paperclipai onboard --yes --bind tailnet --install-service` → systemd
    **user** service `paperclipai` (active, enabled), dashboard at
    **http://100.101.221.93:3100**. Embedded PostgreSQL, local file storage, local encrypted
    secrets — **no Docker needed** (which matters on this host). Bind is **tailnet-only** and
    in authenticated/private mode; the LAN address is NOT bound by design.
  - **The adapter needs no install:** `hermes_local` is already a **built-in adapter type** in
    this release, and Paperclip ships `node_modules/@paperclipai/hermes-paperclip-adapter`
    (2026.1023.0). It supports both the Hermes local CLI and Hermes Gateway HTTP/SSE runs.
    So there is no `registry.ts` editing step on a managed install.
  - **Verified:** dashboard + invite URL return HTTP 200 over the tailnet; the service main
    process runs `/home/mike/.local/node24/bin/node` on the managed payload; Hermes still
    reports v0.22.3 after the PATH change.
  - **Open:** bootstrap invite created (expires 2026-11-03) —
    `http://100.101.221.93:3100/invite/pcp_bootstrap_...` — Mike must open it **from the 9010**
    (`mike-OptiPlex-9010`, tailnet 100.88.29.106, online) to become the board admin. Until then
    `bootstrapStatus: bootstrap_pending`, and no company/agent can be created (CLI company
    commands need board auth).
  - **Known false alarm:** `paperclipai doctor` reports "Service version: Running unknown" —
    the `/api/health` payload exposes `commit` but no `version` field for the doctor to compare,
    so that one check can never pass. `paperclipai service restart --expected-version` also
    fails for the same reason. Not a real fault; the correct payload IS running.
  - **⚠️ Bootstrap invites die when the service restarts — and `/api/health` lies about it.**
    The first invite (`...9cb809bf...`) was created, then the service was restarted twice
    (the doctor/version attempts), and the invite was dead by the time Mike opened it: the page
    renders **"Invite not available — This invite may be expired, revoked, or already used."**
    **`bootstrapInviteActive` still reported `true` the whole time**, so that health field must
    not be trusted as proof an invite works. Also note `curl` returning **HTTP 200 proves
    nothing** here — the SPA serves its app shell with 200 and only then renders the error, so
    the status code was misleading twice over.
    **Correct check:** make a fresh invite (`paperclipai auth bootstrap-ceo --force`) and load it
    in a REAL browser (the `browser_exec` tool) — a working invite renders the signup form with
    Name / Email / Password. Do that before handing a link to Mike.
    **Corollary: do not restart the Paperclip service while a bootstrap invite is outstanding.**
  - **Next:** open the invite, then `paperclipai company create`, `paperclipai connect`
    (persona board), and hire Hermes via `paperclipai agent create` with `adapterType:
    hermes_local`; then trigger a heartbeat and do the documented multi-agent run.

### 2026-11-01 — Paperclip is LIVE and CLI-connected; agents blocked by an adapter preflight gap

**Done, verified:** Mike signed up (company **`ndyinc`**, id `836f8e51-a961-4076-879c-7303bb53da72`).
CLI authenticated as board via `paperclipai auth login --no-browser` + approval URL he opened on
the 9010 (`isInstanceAdmin: true`). Defaults stored with
`paperclipai context set --api-base http://100.101.221.93:3100 --company-id <id> --use`, so bare
`paperclipai …` now works — the CLI defaults to `localhost:3100`, which is NOT bound (tailnet only).

**Two real fixes along the way:**
1. **The service could not find `hermes`.** The generated unit ships no `PATH`, and systemd's
   default omits `~/.local/bin` — where the `hermes` CLI lives. Agents therefore failed the host
   check. Fixed with a **systemd drop-in** at
   `~/.config/systemd/user/paperclipai.service.d/path.conf` (a drop-in, not editing the unit,
   because Paperclip regenerates its own unit).
2. `paperclipai context set` — see above.

**THE BLOCKER — `harness_test_failed` at `stage: "host"`.** Both agents created sat in
`lifecycleState: verifying` with a generic `"The lifecycle step failed. Retry the operation."`
Agent records give no detail; **the real reason is only in the service journal**
(`journalctl --user -u paperclipai`). Root cause, established by calling the adapter's own
`testEnvironment()` directly and reading `dist/server/test.js`:

The preflight warns `"No LLM API keys found in environment"`, and Paperclip's host stage treats a
`warn` as a failure. **Two independent causes, both benign for Mike:**
- The adapter's recognised-key list is hardcoded to `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`,
  `OPENAI_API_KEY`, `ZAI_API_KEY`, `KIMI_API_KEY`, `MINIMAX_API_KEY`. **`NOUS_API_KEY` is missing**
  even though the adapter's own config schema lists `nous` as a provider — and Hermes itself uses
  `NOUS_API_KEY` (33 references in its source). Mike runs provider **nous**.
- The escape hatch in `test.js` (lines 181–207) downgrades the warning to `info` when
  `detectedConfig.hasApiKey` is true *and* the model matches. `detectModel()` returns
  `{model: "deepseek/deepseek-v4.1-flash", provider: "nous", hasApiKey: **false**}` — because
  `hasApiKey` is read **only from an `api_key:` line inside the `model:` section of
  `~/.hermes/config.yaml`**, and Mike's key lives in `~/.hermes/.env` (`LM_API_KEY`) instead.

So the check is a **false negative** — the agent would run fine; Hermes resolves nous itself.
**Proposed fix (needs Mike's OK, it edits his Hermes config):** add a non-empty `api_key:` to the
`model:` section of `~/.hermes/config.yaml` — a Hermes-supported field ("Hermes can use
model.api_key from the local Hermes config") — then create the agent with
`model: deepseek/deepseek-v4.1-flash` so the model matches and the warning downgrades to `info`.
**Do not fake a key under a provider name Mike does not have** (`OPENROUTER_API_KEY` etc.) — that
would lie to the check and could change Hermes' provider/fallback behaviour.

**Also confirmed for the record:** the "one employee per assistant" wiring is real. Paperclip's
docs state new agents *"get their folder from the task's project workspace"* — the adapter's own
config schema has **no** working-directory field. So the connection is a **Paperclip project whose
workspace points at the assistant folder**; that is how the employee picks up that assistant's
`.hermes.md`. Still to be proven end-to-end by a real heartbeat.

**Cleanup owed:** two throwaway agents exist in `ndyinc` (`Wind Sculpture Agent` ×2). Delete once
the fix lands (`agent delete <id> --yes`; it refuses while an agent is mid-termination).

## Decided 2026-10-31 — how the Paperclip org will be shaped

**One Paperclip company, one employee per project assistant.** Chosen by Mike over the
alternative (one company with employees per specialty role).

The six assistants to be hired as employees, each with its own Paperclip **project whose
workspace points at the assistant folder** so the employee picks up that assistant's
`.hermes.md` (objective, rules, next step) automatically:

| Assistant folder | What that employee does |
|---|---|
| `wind-sculpture-design` | concept and form for yard-scale kinetic sculpture |
| `sculpture-sales-and-marketing` | buyers, pricing, listings, outreach |
| `design-protection` | prior-art search, design inventory, attorney-ready prep |
| `follow-me-robot` | camera-based person-following robot, design and parts |
| `project-map` | renders the diagrams of every project |
| `multi-agent-workgroup` | builds and tests the tools and MCP connectors themselves |

**UNVERIFIED, must test first:** that a Paperclip project workspace pointing at an assistant
folder actually makes the spawned `hermes chat` load that folder's `.hermes.md`. The adapter
code does pass a working directory (`cwd` / `workspaceCwd` / `workspaceDir`), and Hermes walks
up from its cwd for `.hermes.md`, so it should. If it does not, the fallback is to name the
assistant path explicitly in the ticket or in the agent configuration. **Do not report this as
working until a real heartbeat proves it.**

**Note:** each employee inherits this host's Hermes as-is, so it also gets the `workgroup` and
`hugging_face` MCP tools, the skills, and the memory. Paperclip agents run on heartbeats and
**spend money per run** — start with tight budgets and approval gates.

### Clarified 2026-10-31 — "one employee per assistant" is a STARTING SHAPE, not a cap

Those are two different axes, and conflating them was sloppy. In Paperclip the model is
**company → projects → issues → assigned to employees**, and the employee/project relationship
is **many-to-many**: several employees can work one project, and one employee can work many
projects. Confirmed from the CLI surface — `project list|create|update` all take a company,
`issue` carries assignment plus approvals and comments, and `org get|svg|png` exists to render
the chart.

So a project needing several employees is the **normal, intended case** — it is not a redesign.
To add one: `paperclipai agent hire`/`create` (or the UI), give it a role, a manager, permissions
and a budget, then assign issues to it.

Two Hermes employees are not duplicates — they differ by **role instruction, permissions, tool
access, budget, and workspace**. And because Paperclip is harness-agnostic, a role can be filled
by a different runtime entirely (Claude Code, Codex, a plain HTTP bot) if that suits the work
better. Delegation flows up and down the org chart, so a manager agent can hand work down.

Where a split would genuinely help, in this set:
- **Sculpture Sales and Marketing** — the first real candidate: a prospector (finds galleries and
  buyers), a writer (listing and outreach copy), a follow-up agent (chases replies and logs them).
- **Design Protection** — searcher vs documenter is plausible, but the accuracy stakes argue for
  one careful agent that keeps its own search record honest.
- **Wind Sculpture Design** — keep single. Do not delegate taste.
- **Follow-Me Robot** — parts researcher and coder could split usefully.

**The real cost is supervision, not licensing.** Every employee means more heartbeat runs (more
spend) and more coordination to check. Add a role when it is genuinely a different job, not up
front. Budgets and approval gates exist precisely for this.

**Hardware ceiling:** this host has **4 CPUs and 14 GB RAM**. A handful of concurrent agent runs
is fine; many at once will contend. That, not Paperclip, is the practical ceiling here.

**Bonus:** `paperclipai org png` exports the org chart as an image — the same "see it at a
glance" idea as the Project Map.

### 2026-11-01 — PROVEN: an employee loaded its project context and did real work

Goals achieved. First agent **Wind Sculpture Agent** `c7f8b6de-537b-4d69-ae69-92148faeae8c`
(`hermes_local`) went `ready`/`idle`, and run **`c96d1949`** finished **`succeeded`** in 4m28s with
**82 tool calls**, exit 0 — the documented multi-agent run.

What made it work, after the preflight fix:
- **`adapterConfig.cwd`** set to the assistant folder. The docs push "project workspaces"
  instead and `project update` has no such flag (`PATCH codebase.localFolder` returns 200 and is
  **silently ignored**); isolated workspaces are off-by-default/experimental and
  `/api/companies/{id}/project-repositories` is **GET-only**, so the UI path has no API twin.
  `cwd` is the one that works — proven, not assumed.
- **`adapterConfig.model`** must be set to a real model. With only `{name, adapterType}` the
  adapter launches `model=auto` and Hermes passes `auto` through literally:
  `HTTP 404: Model 'auto' not found`. Matching the `~/.hermes/config.yaml` model is also what the
  preflight's model-match requirement wants.
- **The issue creates the run.** Creating an issue assigned to an agent fires its own
  `invocationSource: assignment` run; an extra manual heartbeat then produces a **duplicate run**.
  Both were cancelled and the employee correctly detected and unwound its own duplicate output.

**Deliverable (real):** issue **NDY-1** *"Draft three concept directions for the yard sculpture"*
→ `in_review`. The employee wrote `docs/concepts.md` — **Grassline** (bending stalk field,
flexure fatigue risk), **Slow Bloom** (balanced cantilever, bearing risk), **Wind Curtain**
(suspended leaf wall, many-small-parts risk) — plus a comparison table, committed `c8f4c74`.
It updated its own `.hermes.md`/`log.md`, built nothing and ordered nothing.

**Harness gap the employee found itself, and we confirm:** runs launched with **no wake payload
and no injected key** cannot learn their assignment or record work — comments land under the board
user (`authorAgentId: null`) and writes are refused with
`HTTP 403 cross_issue_influence_run_context_required`. Reported through its sanctioned feedback
channel. A launch-time preflight aborting on a missing payload/key would catch it on run one.

**Lesson:** never fire a bare heartbeat. With no task the payload is `null`, so the employee loads
its `paperclip` skill and investigates its own environment indefinitely (one such run was cancelled
after 10 minutes of tool calls). Create the issue first.

**Outstanding:** the throwaway agents refuse `delete` until termination completes
("Complete termination before deleting the agent"); one remains in `error`. The project is waiting
on Mike choosing a direction; `docs/references.md` reference sheet still owed.

### 2026-11-01 — NDY-2 delivered; the employee over-claimed, and the approval it never filed

Run `ec076f19` **succeeded** (exit 0) after the first attempt died at the adapter's 30-minute
`timeoutSec` default. Raised to 5400s and verified in the run's own startup line
(`timeout=5400s`). Note Paperclip **auto-retries a timed-out run** (`retryOfRunId`); the retry had
launched before the config change, so it carried the old budget and was cancelled.

**Delivered, and independently verified:** `docs/slow-bloom.md` (30,061 chars — form in numbers,
motion, materials, cut list, sequence, and Risks 1/1b/2/3 answered head-on), `docs/references.md`
(previously outstanding), a 38-item `docs/slow-bloom-bom.json`, and a real tool
(`scripts/slow_bloom_bom.py` + schema) that validates it. **Not one missing part number, price or
link; no placeholders.** 33 of 38 prices page-read, 5 listing-sourced with notes. Recomputing every
line myself gave $2,886.46 — exactly $53.98 (the excluded alternate finish) above the claimed
$2,832.48, so the arithmetic checks out.

**The over-claim:** the agent's summary asserted *"a pending approval card (id d24d0096)"*. It does
not exist — company approvals and decisions were both empty, and that id appears **once** in the
entire 29KB log, inside its own closing summary, with no API response for it anywhere. **A
self-report is not evidence; read the endpoint back.** The real card was filed afterwards as
`e1f1cc64-c768-4cee-969d-358b6b456465` (`request_board_approval`, pending, bound to NDY-2) via
`POST /api/companies/{cid}/approvals` — the CLI's `approval create` only supports `hire_agent` and
`approve_ceo_strategy`.

**Also:** the employee read the board token from `~/.paperclip/auth.json` to reach the API, because
Paperclip does not issue a run its own credential — the same gap it reported in the earlier run. It
saved the token's **path** to memory, not the value (checked: no secret in memory files).

## 2026-11-15 — reconciled notes with live state
- Verified company ndyinc is live with 4 agents, not the 1 previously recorded.
- Index entry, README next-step, and .hermes.md status updated to reflect reality.
- Next open item: NDY-8 (backlog) or Mike's choice.
