What works today, and what is simulated
Generated from docs/TRACEABILITY.md when this site is built. The document is in English.
Document date: Oct 28, 2026
- 124 Done
- 48 Simulated
- 45 Mixed
- 0 Stub
- 0 Not started
- Done: implemented and tested end to end in the core.
- Simulated: real logic, tested against a local simulator or fake, not the real system.
- Mixed: see the row.
- Stub: an interface exists, the behaviour does not.
- Not started.
P0 (v1 launch bar)
17 rows
| What remains | ||
|---|---|---|
| W1: A workstation per AI employee, customer machine first (Windows and macOS): the desktop agent registers the machine it runs on and is ready at once; cloud workstations are optional and later | simulated | Signed installers; real element finding on a desktop; real cloud VMs and the 2-minute bar only if cloud workstations are offered later. |
| W2: Desktop agent on customer machines, outbound-only | Simulated | Native desktop apps (non-web); packaged, signed installers; real mTLS certificates and rotation; field testing on customer networks and proxies. |
| W3: Live view, one-click take-over and hand-back, session recorded; real screen streaming from the real-browser driver (JPEG frames, secret fields and elements showing a vault secret painted over before encoding, at most four captures a second per session shared by all viewers) to the console live view, and take-over input (click at picture coordinates, typed text, one named key) forwarded only through the person's own recorded driver after a take-over; text typed into a password or code field is recorded as a placeholder | done (web apps in a real Chromium) | Polled JPEG frames, not video or CDP screencast; web pages in the agent's Chromium only, not a native desktop; frames do not cross the agent link, so in split mode the console shows the text view only; frame images are not kept in the recording |
| M1: Sign in with Claude, ChatGPT, Gemini through official login or CLI; API key and OpenAI-compatible fallback | Simulated | Codex and Gemini CLIs need the owner's own sign-in (live/test_live_providers.py -k other skips until then); real API-key calls; written confirmation from providers that subscription plans may run an always-on agent (PRD open risk); real usage limits. |
| M2: Model choice per AI employee and sub-agent; usage meter per provider account | done | Real per-plan quotas from providers; billing reconciliation. |
| B1: Encrypted vault on the workstation; request_login(site) fills password and TOTP, returns no secret | done (keystore: simulated) | TPM-backed keys; running the fill component inside the workstation process boundary on real OS images; third-party penetration test. |
| B2: Domain matching, per-agent and per-site grants, first-use approval, revoke and rotate | done | Import from more password managers through their APIs (CSV/JSON exports today). |
| B3: Secrets redacted from recordings, logs, model requests; leak tests in CI | done | CI wiring in the hosted pipeline; bug bounty; leak scans over real recordings (video/OCR). |
| T1: Badgeye creates, configures, retires sub-agents within a budget | done | Tuning the scaling rule on real workloads. |
| T2: Every sub-agent output reviewed by the lead before a write to a system of record | done | Real model quality for reviews (the journeys use a deterministic stand-in model). |
| G1: Goals in plain language; limits by amount, action and system; shadow mode per step | done | A console screen for editing per-step thresholds (API only today). |
| G2: Approve and Hold in Slack, Teams, email with context and replay; SLA and backup approver | simulated | Slack and Teams app approval (marketplace review), production email sending domain; the messages run in the local outbox today. |
| O1: Watchers on mailboxes, queues, dashboards, screen regions; work with no prompt | simulated | Gmail and Microsoft 365 are written against the documented APIs and tested only against mocked HTTP: they have not run against a real Google Cloud project or Entra tenant, so consent, Pub/Sub delivery, Graph validation and renewal timings are unproven live. Push needs a public HTTPS URL on the agent (or a tunnel); without it the watcher polls. See MAILBOXES.md. |
| K1: Opt-in recorder, field redaction, allow and deny lists, task clustering | done (browser); desktop: not started | A desktop (non-browser) recorder; works-council materials. |
| S1: Versioned skills; record by demonstration; edit in the console; finance templates | Done | Recording demonstrations from the desktop agent (browser events today). |
| L1: Nightly replay of misses, sandbox tests against past runs, promote only on 100% | done | Real model drafting fixes (stand-in model in tests); sandbox isolation on real workstations (a second browser profile today). |
| R1: Morning brief, shift metrics, ROI view | done | Baselines from a customer's own process mining for hours returned (sample assumptions today). |
P1 and P2 (after design partners)
7 rows
| What remains | ||
|---|---|---|
| G3: Approval rules learned from history, suggested, never applied silently | done | Suggestions only; nothing changes until an Owner accepts. |
| O2: Health checks, auto-restart, alert after 5 minutes stalled | done | Probe for in-process subjects is the oldest in-flight task start; external agents post heartbeats. |
| K2: Process mining joined with task data: map, variants, bottlenecks, rework, hours | done (on synthetic P2P data) | Connectors to real ERP event logs. |
| S2: MCP servers, REST APIs, SQL; screen fallback | Done | MCP over streamable HTTP against a real remote server, and its URL through the egress gate; customer systems (ERP, real databases); secrets are vault_ref:<name> (sealed in the agent's Blindfold vault, vault/tool_secrets.py, resolved only at call time on the agent, redacted everywhere; tools/test_tool_secrets.py) or BADGEYE_TOOLS_* environment references as the fallback. |
| L2: Auto-repair after a target UI change, verified before use; the change is detected visually: before each screen step the real-browser driver snapshots the step's target region (hashed DOM structure tokens, no text, and a 32 x 32 grayscale thumbnail with secret fields masked) and compares it with the baseline from the last run where the step worked (token overlap, mean pixel difference, difference hash; pure-Python PNG decoder, no imaging library); a failure on a changed region runs the repair, a failure on an unchanged region escalates to a person instead, a step that still works on a changed region publishes skill.ui_changed and moves the baseline | done (web portals local) | Visual diffing needs the real-browser driver; the simulated and desktop drivers and training-sandbox replays keep the "a step failed" rule, and visual diffing in split mode (on the agent) is not tested; thresholds are tuned on the local portals only; verified against a changed local portal page, not a live vendor site |
| A1: Session replay and action log, searchable, exportable, reversible | done | Reversal only where the executor supports it; a settled payment is refused. |
| A2: SIEM export, per-action cost and time | done | Syslog is sent through an injectable transmit; a task's cost is split equally across its actions. |
Agent run safety (hardening of W2, W3, B2, T2 and the watchers)
4 rows
| What remains | ||
|---|---|---|
| Browser egress gate: only public http(s); no loopback, private, link-local, CGNAT, multicast or cloud-metadata addresses, every DNS answer checked (rebinding), unresolvable hosts refused; Playwright route interception on navigations, redirects and subrequests; per-task pin to the vault login sites (vault/domains.py), applied to top-level and iframe navigations but not subresources; opt-in intranet allow-list BADGEYE_BROWSER_INTRANET_ALLOW (off by default); refusals raise EgressRefused, write audit browser.egress_refused and a recording note; the DNS race closed by a local egress proxy: Chromium resolves no web name itself (--proxy-server, loopback not bypassed, QUIC off, WebRTC limited to proxied UDP), the proxy resolves each host once, checks every answer (EgressPolicy.vet_host) and connects only to an address it checked (BADGEYE_BROWSER_EGRESS_PROXY, on by default on the agent) | Mixed | The proxy covers this Chromium only: other software on the machine, or Chromium flags someone changed, can still reach the network, so hardened machines still want an egress-filtering firewall. The proxy is tested against local servers with a public stand-in address, not on customer networks or behind their own corporate proxies (it does not chain to an upstream proxy yet). |
| Challenge detection and take-over hand-off: CAPTCHA, Cloudflare interstitial, unplanned 2FA found from title, text and DOM hints (no network); the task pauses, a time-limited (default 10 min) help_request approval goes to the Owners and admins, the console shows it on Approvals and Workstation; after hand-back the model loop must re-read the page (StaleSnapshot) while the skill runner does one read_text itself before a click or type (skills/runner.py); an expired request fails the task cleanly | Simulated | An interactive Turnstile checkbox inside a form is still not detected; other walls (Akamai, DataDome, PerimeterX) not tried. With a task pin on, a third-party CAPTCHA iframe from another site is refused like any other frame, so the wall is detected from the page text instead of the widget. |
| Loop guard and per-run caps: the same tool call (name and normalised arguments) failing 3 times is not run again, the model is told, a model that insists ends the run blocked; hard cap on tool calls (60) and tokens (400000) per run, per AI employee or by BADGEYE_RUN_MAX_*, alongside monthly_token_budget | done | A blocked run fails the task. The intranet allow-list stays environment-only and is read from the control plane's environment; on a split deployment set it on the machine that runs the agent. |
| Prompt-injection and fraud eval set: 34 labelled invoices, emails and page texts; rules-first screen for instructions to the assistant, phishing, lookalike vendor domains and changed bank details; watchers flag events (payload.risk) and the loop marks screen text as untrusted | done (rules); no model | Rules only: new attack wording needs a new fixture and rule; a model-based second opinion is not built. A payment, refund, invoice approval or bank-detail change derived from a flagged event never clears automatically: a person must approve it whatever the limits say (goals/service.py _flagged_source, goals/limits.py moves_money_or_bank_details; goals/test_flagged_source.py). The lookalike rule no longer flags the parent domain of a known login site (mail from vendor.example, login at portal.vendor.example), which would have sent every such payment to a person. |
Console sign-in (security gate for deploying)
8 rows
| What remains | ||
|---|---|---|
| Per-person sign-in: scrypt passwords, opaque revocable expiring sessions, backoff per email + IP pair and a per-IP cap in the database (directory/throttle.py), forced change of a temporary password | done | Email verification |
| TOTP two-step sign-in: QR drawn server-side (no QR service), activation by proven code, 10 hashed one-time recovery codes, login challenge step with rate limits and replay protection, org requirement for owners and admins, audited admin reset | done | Not tested against a real authenticator app; WebAuthn or passkeys |
| Self-service password reset: signed single-use 30-minute link by email, same answer for any email, kills all sessions, rate limited | done | Real delivery through Resend or SMTP is covered by the existing sender tests only |
| Every /v1 call resolves the caller; segregation of duties uses the real signed-in person; the service token is for machines only | done | Routes that take an optional person field still rely on the client to send it |
| Console login, logout, HttpOnly cookie, proxy.ts redirect, People/invite screen, no "Acting as", dev token only with BADGEYE_CONSOLE_DEV=1 outside production, sample mode read-only | done | A browser-driven test; not run against a real Vercel deployment |
| Single sign-on with OpenID Connect (Google Workspace, Microsoft Entra, Okta, generic): authorization code + PKCE, single-use state bound to the browser by an HttpOnly cookie, nonce, issuer/audience/expiry/not-before, signature against the issuer's JWKS (RS256/384/512, PS256, ES256/384 only; cache with rotation and a refetch cooldown, stale keys served for up to 24 h if the IdP is down), per-org email-domain allowlist, verified email, linking only to invited people (auto-create only if an owner enables it, never as owner), group-to-role mapping (approver/admin only), owner can require SSO with a break-glass owner (password sessions of everyone else stop at once), client secret sealed, audited, rate limited, badgeye sso disable for operators; console: Sign in with SSO on the login page, owner-only SSO settings on Security | Mixed | Not run against a real Google, Entra or Okta tenant. No SCIM deprovisioning; no domain-ownership proof; no sign-out from the IdP (single logout). |
| Tamper-evident signed audit export: per-org Ed25519 key (private half sealed, public half in the console and GET /v1/audit/public-key); date-range export as a zip of events.jsonl, events.csv, a signed manifest (key, range, first/last seq, row count, first prev hash, last hash, file digests, anchors) and a README with the hash recipe; badgeye audit verify <export> [--public-key K] verifies signature, every hash and link, CSV against JSONL and anchors with no database and no network, and names the failing row for any altered, deleted, inserted or reordered row; daily signed chain-head anchor (scheduler job, on demand, shown on the Audit page); badgeye audit export for operators; the export is itself an audit row | done | The signing key sits with the service; pin the public key and keep copies of exports/heads outside it. No key rotation, no external timestamping (RFC 3161), no HSM/KMS. |
| Controls document mapping the above and the rest of the system to SOC 2 criteria, with implemented, partial, planned and organizational marked honestly | done | Not an attestation; no SOC 2 report exists. Policies, evidence collection and the audit itself are the company's to do. |
Public live demo (/demo)
1 rows
| What remains | ||
|---|---|---|
| Interactive product demo: team board, workstation mirror, approvals (Approve, Hold, Edit limit), limits, kill switch, egress gate, injection flagging, CAPTCHA handoff with 10-minute countdown, loop guard, per-run cap, audit, brief | simulated | Runs wholly in the browser from a seeded simulation; invented data only. It mirrors the rules of the core (browser safety, help requests) but does not call it. |
Directors (department directors and playbooks)
6 rows
| What remains | ||
|---|---|---|
| Charters, mandates, KPIs (3-5 each with target, unit, source) and OKRs for the Chief of Staff and 13 departments; default authority matrix (alone / with approval / never, amount caps) compiled into limits; default worker slots; RACI for quote-to-cash, procure-to-pay, hire-to-retire, incident-to-postmortem, campaign-to-lead, ticket-to-resolution and contract lifecycle | done | Defaults are starting points the owner edits; the locked human gates still apply |
| 84 default playbooks (6 per role), built only from the existing skill step kinds; every write is a propose_action or a write tool, so it drafts for review; always-human actions (sign, hire, terminate, publish externally, deploy to production, access grants) are proposed and caught by the gate; registered in the skill library with category = department (GET /v1/skills/library/playbooks, installable like any template) | done | Playbooks run on canned model answers in tests; real-model quality is not measured |
| Simulated tool servers for departments with no connected tool (83 tools: tracker, ERP, CRM, HR, helpdesk, calendar, mail, security, warehouse, board data, and so on); registered as REST tools on an in-process mock transport, so grants, redaction, audit and the review path for writes behave as for a real tool | simulated | Invented data, no network. Replace one by registering a real tool of the same name; the playbooks do not change |
| Shadow by default: a director's mode is shadow until an Owner promotes it (director.promoted / director.demoted, audited, API PUT /v1/org/directors/{id}/mode); in shadow its tasks read and draft but every write is recorded as "would have" (director.would_have) and nothing executes, while the request chain still progresses on the drafts; live directors keep limits and the hard human gates | done | Owner-only promotion; the Owner grades what the shadow director would have done before promoting |
| Directors with the split roles: org on the control plane, team on the agent; request tasks go down as task.requested, outcomes come up as task.completed/failed/escalated; plan approvals follow plan.proposed | done | Over HTTP between two runtimes, executed only on the agent |
| Console reads: spend per department for a month; Chief of Staff daily and weekly brief; client mirror of the human-gate lists is checked against the Python constants | done | Org-scoped like every route |
ERP connectors (market gap 5)
9 rows
| What remains | ||
|---|---|---|
| QuickBooks Online (OAuth 2.0 with refresh, vendors, bills, POs, BillPayment dated for a future day, requestid idempotency, intuit-signature webhooks) and Xero (OAuth 2.0 with rotating refresh tokens, contacts, ACCPAY draft then authorise, POs, payments, Idempotency-Key, x-xero-signature webhooks) | simulated (fixtures) | A run on an Intuit sandbox company and a Xero demo company. Neither system has goods receipts, so the match asks a person. |
| NetSuite (SuiteTalk REST, SuiteQL, upsert by external id, item receipts) and Bill.com (v3, session login, duplicate check before create) as documented adapters behind the same interface | Simulated | Token and session wiring in the service, sandbox run, request shapes confirmed against a live account. |
| Three-way match with price, quantity and total tolerances and reasons (matched, review, mismatch); console editor for tolerances per org and connector (owner/admin, audited, bounded 0 to 25 % price and quantity, 0 to 1000 total); a tool call can only tighten them; split mode: tolerances go down read-only, connection status (no secrets) comes up from the agent and the Connections page shows it as held on the agent with its last report, connect/sync/disconnect refused on the control plane | done (logic); inputs simulated | In split mode the OAuth connect runs on the agent machine, not from the console; one agent per org holds a connector; personal agents do not upload connection status; the split-mode page is covered by the Python split test, not Playwright |
| Credentials in the agent's vault (client secret, access and refresh tokens as separate sealed secrets, masked by the redaction hook); no token in the database, events, logs or model requests; the cloud control plane refuses to take them | done (software); keystore simulated | Split deployment: the console on the control plane cannot yet read connection status held on the agent (needs replication). |
| Writes are drafts: create_bill, post_bill, schedule_payment go through lead review, limits and approvals as tool:<provider>.<op>; payments carry their amount to the limit engine, an unclean match forces a person, the paying account is configured not chosen, never more than the balance | done | Hold a payment for a bank-detail change on the vendor record (gap 3). |
| Idempotency (business key table + vendor-side key + duplicate lookup), rate limit (token bucket), retry with backoff and Retry-After, one refresh on 401, no repeat of a keyless write after a 5xx | done | Real limits per plan. |
| Schema drift breaks a named test per vendor (test_schema_drift_*) | done | Fixtures are re-recorded from a sandbox once one exists. |
| Console Connections page (owners and admins connect, status, last sync, disconnect), OAuth callback route | simulated | |
| Screen fallback when no connector is connected (the tools do not exist; fallback steps run) | done | Skills must list the screen steps as fallback. |
Predictable cost and value (market gaps 4 and 7)
3 rows
| What remains | ||
|---|---|---|
| Price table per model (org, env, sample list, assumed fallback), dollar caps per run, goal, department and month checked before each model call, halt audited (cost.halted), one warning at 80% (cost.warning), policy audited | done (scripted models) | Prices are sample list prices until the customer sets their own; subscription plans are metered at list-price equivalent; a longer-than-projected reply can overshoot by its excess; department caps need the org module; not checked against real provider bills |
| Pre-run estimate from plan steps, past runs and prices, with a range; on the Goals page and plan-step approval cards | done | Playbooks are estimated by key through the API; no playbook-start screen shows it yet. Without history the range is a prior (half to double) |
| Value dashboard: hours returned, cost per task and invoice, escalation rate, exceptions, spend, net value vs an editable human baseline, per week and month and department; morning-brief line | done | Baseline minutes and the hourly rate are assumptions the customer edits; not measured. Departments need the org module |
PII detection and masking (roadmap-3 D)
6 rows
| What remains | ||
|---|---|---|
| Tier 0 detector: SIN (separator or EN/FR context, invoice/PO veto), RAMQ, passport, driver licence, CRA business number, bank transit/account, ABA, IBAN, cards, emails (personal vs business), phones, postal codes, date of birth, US SSN | Done | Probabilistic; a safeguard, not a compliance claim. Driver-licence formats are from the research doc and not verified against issuers |
| Tier 1 Presidio (no-op NLP, no models), agent only, unioned with Tier 0 | done | Cold import about 10 s (spaCy import); 129 MB on disk; PyInstaller size not measured |
| Placeholders <CA_SIN_1>, memory-only token map, executor re-hydration only into declared fields | interface ready | The AP executor must call resolve_arguments; not wired into tools/forms by this package |
| Sink modes off/shadow/enforce per org (shadow default, owner promotes, audited with shadow totals) | done | Modes reach the agent through apply_policy; agentlink sync not wired here |
| Model-call egress and the shared redactor (audit, logs, wire, recording, exports by default) | done | Other sinks (watcher intake, chat cards, mining, exports) need their owners to wrap calls in sink_scope |
| Console Privacy page | simulated | Counts only; sample snippets are redacted and memory-only |
Local study, synthetic workflows, work grouping, governance (roadmap-3 C)
5 rows
| What remains | ||
|---|---|---|
| Local study: time-boxed offline collection, device-side filter, review key, signed (Ed25519) optionally AES-256-GCM-encrypted archive, badgeye study start/stop/status/review/simulate/export/import; local data encrypted at rest (AES-256-GCM per line and for the state file, per-study key from the keystore seam, file-bound associated data, key deleted on wipe) | simulated | Signals come from a stub SignalSource; no real desktop capture was run. The default keystore is the 0600 key file under the study folder (BADGEYE_STUDY_KEYSTORE = keychain or dpapi for the OS store); the Keychain and DPAPI stores were not run in tests |
| Admin import: preview, access reason, approver, size cap, re-redaction, pseudonymous workstation ids, audit row, duplicate refusal | done | Not exercised against a real customer archive |
| Synthetic workflows with ground truth: P2P, O2C, expense, vendor onboarding, month-end close, HR onboarding; seeded, with variants, rework loops, exceptions | simulated | Sample data only; labels and rules written by the same author |
| Work grouping: case id inference, session splitting, objective family, confirmed objectives, no per-person fields | done (heuristic) | Family table is generic and small; unknown is returned rather than a guess |
| Governance pack: impact assessment, notices (EN, fr-CA), works-council briefs, data map, retention, law notes | done | Templates, not legal advice; not reviewed by counsel |
Desktop observer, consent ledger, device-side filter (roadmap-3 B)
5 rows
| What remains | ||
|---|---|---|
| macOS Accessibility and Windows UI Automation adapters behind one interface (roles and labels only; no values, no clipboard) | simulated | Not run on a real desktop: snapshot-to-signal mapping is tested on recorded traces; live polling code is unexercised |
| Device-side filter before queueing: work/sensitive/non-work rules, uncertain is non-work, private-window and secure-field exclusion, non-work reduced to a duration, redaction corpus (Luhn, IBAN, national ids, bank numbers, OTP, secrets), shared redactor last, case-id hashing | done (rules-based) | Generic formats only; no ML; list contents are org data |
| Fail-closed gate (signed config, expiry, participation, notice version, pause/snooze, kill switch), indicator state, encrypted local queue, mark-as-personal (24 h), withdrawal purge; macOS menu-bar control (badgeye-agent tray): state always shown, pause/resume, snooze 1 h, mark the last hour or 24 h as personal (queue and the rows not yet uploaded), quit | simulated | Queue key comes from the KeyStore seam (memory in tests). The tray logic is unit-tested; the AppKit shell was smoke-checked on a Mac (status item and menu built) but not clicked through by a person; no Windows tray. Mark-as-personal does not reach rows already uploaded |
| Consent ledger: versioned notices, per-person append-only acknowledgement with notice version and time, decline/pause/resume/withdraw, re-acceptance on a new version, deletion log by category, retention purge, person export, counts-only coverage, signed capture config (org Ed25519 key) | done | Replaces the per-workstation boolean, which stays as fallback and is switched by the ledger |
| Console Consent page | simulated | Counts only; spec written, run by the integrator |
Packs, routines, setup readiness, autonomy, triage eval (roadmap-3 G)
5 rows
| What remains | ||
|---|---|---|
| Playbook packs: signed zip (Ed25519), secret refusal on export and import, review (diff, permissions, risk), import lands paused (draft versions, routines off, limits template unapplied), digest-confirmed import, drift data, audit rows | done | Trust is the org's own key or a fingerprint the person confirms; no publisher registry, no org library tab, no one-click update. Risk levels are heuristics over step kinds |
| Routines: 15-minute floor, 20 per person, auto-disable after 10 failures with audit row and "needs you" item, platform-enforced read-only for scheduled runs unless an owner promotes; agent-side runner: routine.due goes down the agent link, the runner runs it through the team (one run per run id; limits per AI employee, per routine per hour, per day, run timeout; kill switch refuses), answers routine.run_finished (ok, failed, refused, escalated, timed out), which the control plane counts toward auto-disable; a dispatched run nobody answers fails after 90 minutes; audit rows on both sides with initiator schedule | done (runs simulated) | The work runs through the real team with a scripted model and the simulated workstation; no routine has run against a real system. A personal agent answers the outcome only, without the detail |
| Setup readiness from real probes (agent, vault, model, mailbox, ERP, limits, approver on phone, MFA, SSO, audit anchor) with fix links; console home card and Setup page | done (what is visible) | The control plane cannot see the agent's vault or mailbox tables: those show "Not checked", never ready. Model reachability is inferred from recent usage, not a live ping. Backups are the host's and not probed |
| Acts alone / Asks you / Never per department from authority matrix, gates and limits; API, console cards, static lib/generated/autonomy.json | done | Static copy shows shipped defaults, not any customer's settings |
| Triage eval "needs you / can wait", badgeye eval triage | simulated | 64 invented messages (FR/EN) labelled by the author, rules only; model pass not run. Wide intervals; a regression check, not proof |
Initiator, rule policy, auto-review hook, row-level security (roadmap-3 F)
6 rows
| What remains | ||
|---|---|---|
| Initiator kind and id on every audit row; "Nobody watching" filter in API and Audit page; hashed for new rows only, old chains verify | done | Person from actor; scheduler, watcher and delegated tasks set the scope |
| Audit-first: action.attempted committed before any executor, no row means no execution | done | Wrapped onto the team service by the policy wiring |
| Rule policy: deny-before-allow, fail closed, layers, locked team rules, Preview, audited refusals, in front of the limits check | done | See docs/POLICY.md |
| Auto-review hook that can only escalate, connected to the org's model (BADGEYE_AUTOREVIEW=on): the AI employee's configured model through badgeye.llm (redaction, outbound filters, usage meter), target and summary sent inside the untrusted envelope, JSON verdict; unreadable, failing or erroring model means ask | simulated | Off by default; tested against a deterministic scripted model, not a live provider. No separate timeout: a hung provider call holds the check until the provider client times out |
| Console Policy page and Audit filter | simulated | Spec written, run by the integrator |
| Postgres row-level security with non-owner app role | Mixed | Production still connects as postgres (BYPASSRLS), so RLS is not enforced there until BADGEYE_DB_URL uses badgeye_app; request layer binding pending |
Discover engine (roadmap-3 A)
7 rows
| What remains | ||
|---|---|---|
| Candidates from observed runs: clusters, five step labels, evidence states, nine gates, readiness | simulated | Invented capture only; not run on a real desktop |
| Economics range from CostPolicy and the cost estimate; unknown stays unknown | simulated | Assumptions editable and audited |
| Ranked backlog, discovery queue, golden scoring | done | |
| Decisions (owner/admin, audited), org isolation, no person column, k-anonymity floor | done | |
| Turn into skill, shadow-first goal, predicted against measured value | simulated | Sandbox replay against recorded runs: see the Discover replay row below |
| Conformance and change alerts | simulated | |
| Console Discover pages | simulated | Spec written; sample data labelled |
Personal and Team Agents with RBAC (roadmap-3 E)
8 rows
| What remains | ||
|---|---|---|
| member and viewer roles, person_departments, role/department edits (owner any; admin approver/member/viewer; last owner kept) | done | |
| SSO auto-create defaults to member, not approver | done | Existing people unchanged |
| Server-side confinement of member and viewer (default deny) | done | Own audit rows and own requested approvals for members not yet exposed |
| Scope, owner, visibility, content_access on AI employee; agent_shares with approval | done | |
| Permission evaluator: role x scope x credential mode x trigger x action x limits | done | Call sites in tool execution do not yet pass credential mode: evaluator is the contract |
| Agent link sends a scoped agent only its rows; personal agents upload metadata and digests only | simulated | Proven between two local runtimes, not on separate machines |
| Personal agent files a request to the department director | done | Nothing for the software; the director's handling of the request is the org module's normal request flow. |
| Console Agents page, sharing dialog, access preview, People role editor | simulated | Spec written, run by the integrator |
Split deployment and round-3 leftovers (roadmap-4 F)
6 rows
| What remains | ||
|---|---|---|
| agents loaded on the customer-machine agent; a personal agent's refusal files its org request through the link (metadata only: no summary or target) | simulated | Two local runtimes over a test client, not separate machines |
| Agent-link routes read across orgs only to find the link (invite, credential), then run as that org | done | The RLS case runs on Postgres and skips on SQLite |
| Password-reset email carries its org (PII mode applies) | done | |
| Enrolment writes person_id to the agent config; desktop capture starts for that person once they accept the notice; the agent fetches its signed config with its own credential | simulated | Capture adapters are simulated (see roadmap-3 D) |
| Tool results: placeholders resolved only in declared fields (pii_reveals), classes and tasks | done | |
| Discover draft replayed in the training sandbox against real recorded runs from Training (runs of the draft's own skill compared strictly; past tasks a person picks by id in the console or API; passed runs of other skills with the same write kinds on the same systems); pass rate and the replayed runs shown on the candidate; promotion needs 100 %; simulated examples only when Training holds no matching run, labelled | Done | Other skills' runs are matched on write kind and system only (not target or amount); parameter names are not mapped, so a parameter the run never had fails the case; replays use recorded screens, not live portals; the "past tasks to replay" field is tested in Python, the browser test skips when the demo has no convertible candidate |
Org spaces, person caps, escalation (roadmap-4 E)
6 rows
| What remains | ||
|---|---|---|
| Spaces: org, department, agent settings with one resolver (value, source, chain), tighten-only, owner locks, Customize / Use defaults, audit | done | Nothing for the software. |
| Effective settings panel (agent page, department tab, org defaults) | simulated | Spec written, run by the integrator |
| Per-person monthly spend cap: warn 80%, stop 100%, owner override, audit | done | Overshoot by at most one call, as with the other caps |
| Escalation to the Chief of Staff, then the owner; dedupe, hourly limit, digest, quiet hours | done | Owner message goes through the existing channel senders (outbox in tests); no console page for the settings yet |
| Connection scopes per personal agent (which connections, read vs read-write) | done | Nothing for the software. |
| SSO group to department mapping (additive at sign-in) | simulated | Against the mock identity provider, not a real one |
Data protection (roadmap-4 C)
5 rows
| What remains | ||
|---|---|---|
| Content encryption at rest, per-org envelope keys (AES-256-GCM), master-key rewrap, data-key rotation, crypto-shredding, backfill | done (software) | Encrypts task, handoff, action and brief documents, recording frame text/title, captured content, mining events, logs and mined tasks, process maps, watchers, watch events, mailbox records, goals, limits, approvals and goal actions. Ids, statuses, timestamps and small config tables stay plain so queries work. The only master-key provider is the env key; a cloud KMS adapter is an interface, not built (needs the customer's or our KMS account). Needs a master key set to take effect, then badgeye crypto reencrypt once for older rows. |
| AI data-sharing disclosure and per-person acceptance; runs blocked until an owner accepts | done | Seed/dev installs auto-accept with a row marked dev_fixture. An agent offline keeps the last answer it got until it polls again. Not legal advice. |
| Origin-bound credentials (agent tokens, machine API token): origins, CIDRs, machine fingerprint | done (software) | Tested over an in-process HTTP client, not through real proxies or load balancers. The fingerprint comes from the OS machine id, so it identifies a machine, not a person; an attacker with admin rights on that machine can read it. |
| Audit retention (floor, legal hold, signed purge checkpoint) | done | Scheduled purge is off by default. Postgres path (trigger exemption) is written but only run on SQLite here. |
| SIEM at-least-once delivery: outbox with cursor, backoff, dedupe by event id, dead letters | simulated | Against a mock receiver; opt-in per sink with outbox: true. |
Tool and run safety (roadmap-4 B)
6 rows
| What remains | ||
|---|---|---|
| Untrusted envelope on every tool, screen and watcher result (random nonce, HMAC, notice first, markers neutralised, idempotent), backstop before each model call | done | Skill decide prompts that embed page text are not enveloped yet |
| Typed handoff (from, to, goal, depth, fan-out, budget left, data refs); depth and fan-out caps per org; refused hops audited as request.refused and escalated | done | Caps can only lower the platform depth of 4 |
| Skills declare tools; offered list = declared and granted, capped at 10; out-of-scope call refused and audited | done | A run with no skill in play keeps its grants as before |
| Per-resident network modes (off, allow-list, login sites only, any public site); no policy means no traffic for new residents; kill switch means off | done | Nothing for the software. |
| Web Bot Auth signing (Ed25519, RFC 9421 style), per-org toggle off by default, key in the agent vault, public key directory | simulated | No real verifier has checked the signatures; CAPTCHA hand-off stays the main path |
| Console: network mode per AI employee, signing toggle, handoff caps | done |
Operations and onboarding (roadmap-4 D)
5 rows
| What remains | ||
|---|---|---|
| Config as code: badgeye org plan / apply / export (roles, departments and directors, budgets, human gates, authority, limits, policy rules, routines, required packs); plan first, idempotent, no deletes without --prune, audited, no secrets or people | done | SQLite-tested; not atomic across items (stops at the first error, safe to re-run); packs are checked, not imported; docs/ORG_CONFIG.md |
| Bounded MCP server over stdio: read tools plus propose tools that raise a pending approval; per-org hashed keys, scoped, rate-limited, audited; no approve/execute/delete/credential/kill-switch tool | done | Local stdio only (no HTTP or OAuth); protocol tested over a real pipe, not against Claude Desktop; docs/MCP.md |
| badgeye-agent doctor: out-of-process read-only scan (config, database integrity, vault tags, stale leases, stuck tasks, clock skew, egress config, disk and logs, capture consent); human and JSON output, exit codes 0/1/2/3 | done | Proven not to write (tree hash before and after, with and without a live WAL); OS-keystore keys (Keychain, DPAPI) are not decrypted, only structure is checked; skew vs control plane needs --online |
| Sample-inbox trial: sandbox org on the simulated mailbox and ERP, real pipeline in shadow mode, "sample data" banner, deleted after N days; entry points on the site (/try, /fr/try, CTA component) and in the console (/console/sample) | simulated | Runs only on a demo host (all modules, manual clock); one sandbox at a time because the simulated world is shared; audit rows and immutable skill versions of a deleted sandbox stay (append-only) |
| Agent-assisted install prompt and /install pages (EN, FR) with copy button | partial | Prompt content checked for steps and secrets; never run end to end by a real coding assistant on a real machine |
Approvals and the "Needs you" inbox (roadmap-4 A)
5 rows
| What remains | ||
|---|---|---|
| "Always allow this kind" on an approval: a narrow standing rule (same AI employee, action, system and counterparty; amount ceiling at most the owner's ceiling and the governing limit's cap; expires), two-step confirm with a one-time code, undo, audited (standing_rule.*), listed in the inbox. Never for NEVER_TYPES or a human gate (re-checked every time it would apply); respects a limit that always asks and the flagged-source rule | done | Counterparty is read from the action's payload (counterparty, vendor, ...) or its target text. Not yet shown on the goals or policy pages (only on /console/inbox) |
| Change cards: a change to an agent's model, budget, a director's instructions or a limit carries the whole new text and a diff; only an owner decides, only by sending the hash of the text shown; version check (stale = nothing overwritten); not routable to Slack, Teams or email; refused by name for bulk and always-allow | done | Enforced for every change that is proposed as a card. The existing direct owner routes (for example PUT /v1/org/directors/{id}/charter, POST /v1/limits) are unchanged and still edit at once |
| Unified inbox /console/inbox: approvals, plan steps, help requests, routine "needs you" items, access requests, change cards; role-filtered on the server; scored (kind, escalation, due time, triage rules where the message behind an action can be read) with the reason shown; bulk only for plain non-money approvals, refused per item otherwise | done | Triage uses the rules-only classifier on an invented set (evals/triage), not a model |
| Request access: a member or viewer asks for an agent or a connection; an owner or admin grants or denies from the inbox; audited (access.*); only the request routes were added to the member and viewer allow-list | done | An agent grant is a real share through agents.add_share. A connection grant gives each of the requester's personal agents read access to that connection (agents.set_connection_scopes, audited), and a personal agent made later still reads the grant; read-write stays the owner's or admin's choice. |
| Inbox as a PWA with web push (VAPID): manifest, service worker scoped to /console/inbox (caches and fetches nothing), generic push text only, endpoints limited to browser push services; approving needs the normal session and two-step check | Mixed | Delivery to a real phone (Android Chrome, iOS Safari through Apple's push service) and installing the PWA on one; the console still labels push "Simulated". Labelled "Simulated" in the console |
Roadmap 5, package M: positioning, home page and the consult side of the demo
2 rows
| What remains | ||
|---|---|---|
| Home page repositioned ("Automate your whole business"): hero with an AI employee's day and a phone (SMS, YES + PIN, then incoming call), AI team grid from the department charters, consult side (escalation ladder push, SMS, call, next approver; quiet hours; PIN for money), what each department hands off, channels, trust, FAQ, CTAs; EN and fr-CA. Rationale in docs/research/RAKAZO_POSITIONING.md | simulated | Marketing copy. SMS, calls, Slack, Teams, email and push are labelled simulated on the page; the real phone channel (package P) runs against a simulated provider. |
| Demo consult side: SMS thread with the sending AI employee, replies as bubbles, PIN keypad (wrong PIN pays nothing), free-text replies (EN/FR), scene "The call" with keypad options and a missed call escalating to the next approver | simulated | Runs wholly in the browser; it mirrors the product's rules and does not call the core phone channel. |
Roadmap 5, package P: the phone channel (SMS and voice) for "needs you"
9 rows
| What remains | ||
|---|---|---|
| Provider interface (send_sms, place_call, verify_webhook_signature, parsing, voice markup) and the simulated provider (default, tests, demo, console): records instead of sending, signs its own webhooks | done | Nothing is sent to anyone. |
| Twilio provider: REST over httpx (Messages, Calls), X-Twilio-Signature HMAC-SHA1 check, TwiML voice and SMS replies, opt-out error code | simulated | Never run against a real Twilio account, number or phone: carrier behaviour, A2P 10DLC / toll-free registration, speech accuracy and STOP handling by the carrier are unproven. |
| Telnyx provider: Messages API, calls through a TeXML application, Ed25519 webhook check with a five minute window, SMS replies by API | simulated | Never run against Telnyx; whether TeXML webhooks are signed the same way is from documentation and unproven. |
| Per-person contact: number verified by an SMS code (hashed, expiring, attempt- and rate-limited), channels (text, call, both, off), ladder timings, quiet hours (own and the org's), en / fr-CA, opt-out by STOP / ARRET (and re-opt-in), PIN stored hashed with lockout | done | Behaviour of real carriers on STOP (they also act on it themselves). |
| Consult by SMS: short masked message (what, who, amount, code), YES / NO / HOLD + code, PIN for money, bank details, human gates, NEVER_TYPES, amounts and org two-step policy; free text answered back to the asking agent; change cards console-only, never by text; decisions via goals.decide as the person (decided_via sms / call) | done | A decline closes the approval the way a timeout does (there is no separate "rejected" approval state). |
| Consult by voice: TTS question in en / fr-CA, keypad 1 / 2 / 3 / 9, speech as free text, PIN on the keypad, transcript and outcome stored and audited, missed call goes to the next step, inbound calls | simulated | Speech recognition quality (the carrier's) is unmeasured, fr-CA included. |
| Escalation ladder: inbox, then text after N, call after M, then the next person (backup approver through the approval's own escalation; next person on a question); digest and hourly caps; fake-clock tests | done | The backup approver is reached through the approval's SLA escalation, not an earlier "skip now" timer. |
| Public webhooks /v1/phone/webhooks/{provider}/sms and /voice: org from the numbers (system scope, then narrowed under RLS), signature per org credentials, idempotent by provider id, rate limited, neutral reply to unknown numbers | done | |
| Console /console/settings/phone: verify, channels, ladder, quiet hours, PIN, Text me / Call me, the simulated phone, org provider (names only), inbox shows "texted 2 min ago" / "called, no answer"; members and viewers manage their own | done |
Roadmap 6, package C: browser full capture, console and copy
5 rows
| What remains | ||
|---|---|---|
| Browser extension captures typed text of work fields (blur, submit, idle; max 2,000 chars) as typed_text, screen JSON (200 fields, 8 KB) on click, submit and navigation, clipboard as value_hash only; password, secure and sensitive-labelled fields (EN/FR) never read; private windows never captured; pause discards pending text; capture level metadata sends no content | Done | Real Chrome run not exercised in CI; non-ASCII casefold differs from Python only in rare characters |
| Always-on client-side masking mirroring badgeye.pii (CA SIN with Luhn, RAMQ, cards with Luhn, IBAN, bank, postal, e-mail, phone, DOB, keys/tokens), same placeholders, over-masks by design | done | Invented corpus; names and street addresses are not detected; server redacts again |
| value_hash = first 24 hex of HMAC-SHA256(content_salt, normalised value) via WebCrypto | done | |
| Console: Task mining event explorer with reason-gated "Show content" (reason of at least 10 characters), re-keying / copy-paste / repeated-screen cards read from process-map details | done (UI); depends on R6-B endpoints | Needs GET /v1/capture-content/events (assumed) from the control plane |
| Console: "My captured data" for every person with delete; capture level and content retention settings; consent copy updated | done (UI) | Needs GET /v1/consent/capture-settings and POST /v1/consent/config to accept capture_level and content_retention_days |
Roadmap 6, package B: content store, access, retention and mining on content
6 rows
| What remains | ||
|---|---|---|
| Server re-masks all content (typed text, screen JSON, teach notes, the recorder's value) with the shared detector, Blindfold secrets and EN/FR password-label rules; clipboard keeps only a validated value_hash; content of sensitive, deny-listed, non-work or non-allowed apps and of metadata orgs is dropped; event rows keep only the content kinds | done (simulated: invented PII corpus) | Teach screenshots cannot be inspected on the server; they are stored only in teach mode and only as sent by the device |
| Separate capture_content table, payload sealed with key purpose capture_content.payload; migration 0028 | done | Sealed only when a master key is configured (as for every sealed column) |
| Signed config carries capture_level, per-org content_salt (generated once), content_retention_days (default 90, max 365), teach_enabled; owner/admin change them (audited); switching to full publishes a new notice version naming typed text, screen content, clipboard links and teach mode | done | |
| Access: owner/admin read with a written reason (10+ chars), one audit row per view and export; everyone sees and deletes their own (/mine); members and viewers reach only /mine | done | No separate "automation" role exists; owner/admin only |
| Retention sweep deletes expired content; withdrawal or decline deletes the person's (and the workstation's) content | done | |
| Mining on content: re-keying (same hash in two apps in a case/session, count and estimated minutes), copy/paste edges, repeated screens (label fingerprint), field-level signatures with value types; Discover claims and field steps in skill drafts (labels and types only) | done (simulated sample data) | Re-key minutes are an estimate (30 s typed, 6 s pasted per entry) |
Role learning (R6-D)
7 rows
| What remains | ||
|---|---|---|
| A role model (tasks, end states, goals, decision rules with support and confidence) is learned from consented capture | Simulated | |
| Questions are rate limited (3 a day per person), skip quiet hours, go to the person doing the work | Tested | |
| Drafting creates draft skills and a shadow goal; payments and NEVER types stay gated | Tested | |
| Promotion needs an owner and the owner-set threshold | Tested | |
| Withdrawal removes cases, answers and comparisons on the next rebuild | Tested | |
| The demo "Accounts payable clerk" (about 84% coverage, 3 open questions, draft, shadow numbers) | Simulated data, labelled in the UI | |
| Shadow agreement compares decision rules with what people did | Mixed |
Agent memory and fleet (F1)
13 rows
| What remains | ||
|---|---|---|
| Memory notes scoped to a person, an AI employee, a department space or the team, with kind, source, confidence, last use and optional expiry | Tested on invented data | |
| A corrected draft yields candidate notes that are never used until the person or an admin confirms | Mixed | |
| "Remember that..." in chat, inbox or a phone answer creates an active note directly (EN and FR) | Tested | |
| context_for returns scoped, keyword-relevant notes, capped at 20 items and 2 KB, and is added to the sub-agent prompt | Mixed | |
| Per-person writing voice from approved drafts (length, greeting, sign-off, formality, EN/FR) used as a style hint | Mixed | |
| A memory can never change limits, approvals, permissions or safety rules | Mixed | |
| Stored notes are PII-masked and never hold secrets | Tested | |
| A person sees, edits and deletes only their own notes; admins manage team and agent notes; every change audited; org-scoped | Tested | |
| Console Memory page (filters, waiting notes, add, edit, forget, what an agent knows) | Mixed | |
| One registry of AI employees, directors, skills, routines, triggers, connections, tools, models and workstations with owner, department, status, autonomy and last activity | Simulated | |
| Scorecard per AI employee (tasks, approval rate, rejections and corrections, incidents, cost against hours saved, shadow agreement) | Mixed | |
| Policy standards as pass/fail checks (backup approver, limits, own key, skills reviewed, PII masking) and a fleet-wide summary | Tested | |
| Console Fleet page and scorecard | Mixed |
Privacy modes (roadmap 7, P3)
8 rows
| What remains | ||
|---|---|---|
| Owner-only identity mode and k floor (3 to 500, default 5), audited; a more identifying mode publishes a new notice version (re-consent), anonymous takes effect at once and unlinks stored data | Tested | |
| Notice text per mode, English and French, stating what is linked to whom (anonymous: per-person deletion impossible afterwards) | Tested for the key sentences | |
| Anonymous ingest: no person id, no stable workstation id, minute-rounded times, cohort only; identity claims refused | Tested | |
| Pseudonymous: stable HMAC code per person (secret sealed at rest); owner re-identification needs a reason of 10+ characters and is audited | Tested | |
| Person-level reads answer 409 in anonymous mode (capture content by person, my data, role questions/members, one workstation) | Tested | |
| k-anonymity on consenting people per cohort (not on rotating ids); nothing shown below the floor in mining, Discover, role learning | Tested | |
| Role learning by cohort, Ask-the-team board (answers without author, limit per cohort); process maps, Discover candidates, role model and AI employee draft from anonymous data | Tested; data is simulated | |
| Console: Privacy mode cards and slider, banners, My data message, team board | Mixed |
Roadmap 7, package P1: no raw text, end to end
7 rows
| What remains | ||
|---|---|---|
| A field value is read in memory only to compute a descriptor (type, length bucket, amount bucket, choice label, edited, lang) and a digest, then dropped; Signal has no text or clipboard text; no digest for secure / sensitive-labelled fields or values under 4 characters | simulated (recorded traces) | Live OS hooks are not written; names inside choice labels or screen labels are not detectable |
| Screen JSON is structure only: PII-masked labels, roles, order, a descriptor per field, free text as {"role": "text", "descriptor"} | simulated | Static text is indistinguishable from data on the desktop, so all of it is a descriptor |
| Teach notes kept (PII-masked); teach screenshots only when teach_images is on: the per-org setting is stored (off by default, owner/admin, audited), travels in the signed capture config, the agent obeys it, and the server keeps teach images only while it is on (images from a device on an older config are dropped) | done (desktop capture simulated) | No live teach screenshots on a real desktop; the browser-extension recorder config does not carry the setting |
| Device identity modes: anonymous = random workstation id rotated daily, random session key per session, minute-rounded time, cohort from enrolment; no stable id survives a day | simulated | Event order inside a minute is not kept; the agent upload link still authenticates with its enrolment credential |
| Browser extension sends field_input descriptors (wire value null), screen descriptors, clipboard hash only; anonymous mode as on the desktop | done (jsdom, not a customer's Chrome) | Anonymous mode needs the server to accept a rotating workstation id |
| Server stores text for teach_note only; text or typed_text from an old recorder becomes a descriptor and is discarded; views return descriptors; old rows are scrubbed on the retention sweep and never returned with text | done | Rows keep the field label and value_hash |
| Role learning decides from descriptors (amount bucket, value type, picked option); thresholds fall on real bucket edges | simulated (invented data) | Buckets are coarse (0-100, 100-1000, 1000-5000, 5000-10000, 10000-50000, 50000+) |
Roadmap 7, package P2: PII that is much better
6 rows
| What remains | ||
|---|---|---|
| Person names, EN + FR/Quebec, without a model: lexicon (about 2,600 first names, 3,100 surnames) + title/greeting/closing/label context + pair rules, with exception lists (name-and-word, company words, brands, org allow-list) | Done | A lone unlisted name, a lower-case full name and a user name are not caught; no NER model shipped |
| Street addresses FR/EN with unit and P.O. box; city and postal code left to their own class | done | Street names without a civic number are not masked by policy |
| Provincial health numbers (ON, BC, AB, generic by word), QC licence and other fixed-format licences, passports CA/US/FR, SSN/ITIN, IPv4/v6, MAC, IMEI (Luhn), VIN (check digit), plates (with context), DOB variants, age (with cue) | done | Health/licence formats come from public descriptions, not checked against issuers; health semantics (diagnoses) and gender not detected |
| Secrets: AWS, GCP, Azure, Stripe, GitHub, Slack, Twilio, SendGrid, npm, JWT, PEM, URL credentials, token query values, password: pairs | done | Pattern-based; a custom internal token shape is not known |
| One labelled corpus (1,156 positives, 675 negatives, EN + FR) shared by both engines; 100% recall, at least 99% negatives untouched (Python), extension may over-mask | done | Synthetic corpus; a regression net, not a field accuracy measure |
| Speed: 10 KB of text in about 9 ms (plain) to 13 ms (very dense), patterns compiled once, lexicon and folds cached | done | Measured on one laptop under load |
Live desktop observers (roadmap-8 R8-A)
2 rows
| What remains | ||
|---|---|---|
| macOS live observer: AXObserver (focus, window, title, value-changed event), front app, field exit read once, bounded window walk, clipboard digest, idle/lock, private-window markers, Accessibility permission check, badgeye-agent capture doctor and capture live (dev) | Simulated | Never run with Accessibility granted: AXObserver callbacks, hit testing, window walk timing and NSWorkspace notification delivery are unproven on a real desktop. Paste, browser URL and teach screenshots are not live |
| Windows live observer: IUIAutomation focus/Invoke/Value-changed handlers, ValuePattern read once, GetLastInputInfo, clipboard sequence number | written, not yet run on Windows | COM code, foreground polling (no SetWinEventHook) and the message pump are untested |
Demo seeding and PII gaps (roadmap-8 R8-D)
3 rows
| What remains | ||
|---|---|---|
| Demo org opens with the newer features: capture descriptors through the real ingest path (consent set up for the demo workstation, no text), mining insights (re-keying, copy-paste, repeated screens), the simulated "Accounts payable clerk" role, memory notes (team, agent, person) with one candidate waiting; pseudonymous mode kept | done, simulated | Sample data only; each step is independent so a failure never stops the demo; no anonymous-mode demo org |
| PII gaps closed: user names and handles built from a name (tremblay.jf, jf.tremblay, @marie_t, jtremblay after a user word), recipient lists after cc: / À :, lone first names after with, for, by, ask, call back, avec, demander à, GPS coordinates | done | Handles not built from a lexicon name are caught only by the extension (it masks any @handle outside a short list); exchange-rate decimal pairs are not coordinates without a cue |
| Public demo: task-mining example shows kind and bucket only ("Typed in Amount · amount · 1,000-5,000"), plus an anonymous-mode line, EN and fr-CA | done, simulated | Playwright run left to the integrator (ports busy) |
Draft review, memory transport, persistent limits (roadmap-8 R8-C)
3 rows
| What remains | ||
|---|---|---|
| Draft review and edit in the inbox: full draft, Edit textarea, Approve sends the edited text; executor runs the edited version; approval.approved_with_edits with masked diff; payee/amount/bank fields read-only, bank words and IBANs refused | done, simulated | executors are simulated; Playwright spec written, run left to the integrator |
| Memory context in split mode: memory_block + HMAC in task.requested, verified by the agent, used by the team loop | done | goal-step tasks planned on the agent carry no block; lost on agent restart |
| Persistent sliding-window limits (phone texts 8/h 30/day, calls 3/h 8/day; role-learning questions per day) in rate_events | phone done; rolelearn helper ready | deliver must call questions.reserve_slot (see report) |
Real-browser extension test and workstation codes (roadmap-8 R8-B)
2 rows
| What remains | ||
|---|---|---|
| Browser extension in a real Chromium: built from source, loaded unpacked, pointed at the real API (own ports 3190 / 8190), typing into an invoice form (supplier email, amount, password, private message, select); field descriptors reach the API (email, amount bucket, choice, text), no typed value anywhere (events, decrypted content, raw table text), the password field produces nothing, pause discards what was typed, a private window produces nothing. Found and fixed: a <select> was identified as field: so the server downgraded its choice label to text (now select:); opting out through the recorder consent route left content rows behind (now deleted with the events) | Mixed | Not run in Edge or Firefox; the private-window check opens a window without the extension (the manifest forbids it there), so it proves the manifest, not Chrome's enforcement; the extension's token must be a person session (the machine token is refused from browsers) |
| Pseudonymous mode stores events and content under a per-org HMAC code of the enrolled workstation id (wks_...), so the consent ledger's person-to-workstation link cannot be joined to events without the sealed org secret; re-identification (audited, owner, written reason) accepts a person code or a workstation code; my-data, withdrawal, retention, mining, Discover and role learning work on codes; moving identified to pseudonymous re-keys stored rows | done | The deletion log and the consent ledger still hold the enrolled id (legal record, owner/admin readable); an admin who also holds the org secret (database and keystore) can compute codes |
2026-11-05: typed text captured again, masked on the device (level full, default)
6 rows
| What remains | ||
|---|---|---|
| CaptureLevel.FULL and ContentKind.TYPED_TEXT un-retired; full is the default for new orgs (CaptureConfig, capture_settings); unknown values read as structure | done | Existing orgs keep their stored level (structure) until an admin raises it |
| Typed and clipboard text masked on the device before it leaves (Python and extension, same rules): every PII class; secrets dropped (descriptor only); PASSWORD class for passwords typed in normal fields (keyword phrases, bare keyword + token, PIN/NIP/CVV digits; lone high-entropy token; four-class tokens in text); no value_hash for password-looking or secret values | Mixed | A password with no keyword that looks like words passes; code-like work tokens may be over-masked |
| Hard exclusions kept: password / OTP / cc-* / password-, PIN-, CVV-, card-labelled fields give nothing but the descriptor (no event in the extension); private windows; pause discards; per-org deny lists plus built-in password managers, personal webmail and banking (DEFAULT_SENSITIVE_DOMAINS, BUILTIN_DENY); anonymous mode unlinked | done | Built-in lists are short; orgs add their own |
| Server keeps text only at level full, with the content-v2 mark, not from a password-labelled field; masks it again (device rules, server rules, last detector pass drops it); stored sealed; member sees and deletes own; retention; full -> structure drops stored text; raising the level publishes a new notice (EN + FR) and everyone re-accepts | done | Old rows (before content-v2) are still scrubbed to descriptors |
| Shared corpus extended with typed-text cases (messages, notes, e-mails; names, addresses, phones, SIN/NAS, health cards, bank/transit, IBAN, cards, DOB, typed passwords, URL tokens): 1,466 positives, 717 negatives | done | Synthetic, same author as the rules: a regression net |
| Console, site and notices say plainly that typed text is captured after masking, what is never captured, retention and who can see it | done |
2026-11-06: how well the AI employee does the job (agents-v3)
8 rows
| What remains | ||
|---|---|---|
| Plan, act, verify: the agent may declare a plan with a success check per step (plan), checks a step after doing it (check_step), and is told to try a different approach on a failure, then to ask or report after two. Checks are evaluated by the platform against the screen, the drafts and the final answer (page_contains, url_contains, drafted, output_has, output_equals), never taken from the model. A task's own done_when is checked on the final answer: a failure gets one "VERIFY BEFORE FINISHING" nudge, then the lead sends it back (REWORK) without asking its model; the lead's prompt says how many checks passed | done | A check that cannot be evaluated (no screen) fails: an unverifiable claim is not verified. Plan checks the model writes are only as good as the model's choice of check |
| Generalising a learned role: a goal step drafted from role learning holds every new case to the end states observed for that task (drafted in the system for writes, the status word for status end states). When the task was seen ending in several branches, only endings common to every branch are imposed, so a new variant is never held to another variant's ending | done | The branch of a new case is not inferred yet, so branch-specific endings are not checked |
| Ask instead of guessing: an ask_person tool (at most 3 questions per task) stops the run; the task is escalated with result.question (text, why, options) and the "needs a person" notice; POST /v1/team/tasks/{id}/answer adds the answer to the task's input (redacted, audited task.answered) and runs it again with the answer in the prompt. Callers cannot inject answers when creating a task | done | The answer route is owner/admin/approver only (inherits /v1/team access); the console has no answer form yet |
| Anomalies found in code before the model runs: a reference already handled by a finished task that wrote something (duplicate_reference) and amount vs total that disagree (conflicting_values) block and ask; an amount over 3x (or under a third of) this counterparty's median over at least 3 finished tasks (amount_outlier) warns. An answered block becomes a warning; every write of a run with an anomaly is marked anomaly | done | Looks at the last 500 finished tasks of the same employee; the reference keys and ratio are fixed constants |
| Calibrated confidence: computed from the evidence (capped self-report, failed or passed checks, anomalies, assumptions, failed tool calls, lead rework rounds, the agent's record shrunk to a prior). Below 0.6 every write is marked low_confidence; goals then asks a person even within the limits (goals.limits.DOUBT_CODES), and no standing rule ("always allow") applies to it | done | Weights are a documented starting point, not fitted |
| Learning from corrections: an approval held with a note, an executed action a person undid with a reason, and a lead rejection with notes become an employee-scoped procedure memory (source correction) naming the kind of work, the system and the target, so the next similar task gets it in its prompt. Active at once from an owner or admin; a candidate from any other approver and for every lead rejection. Goes through MemoryService.add (guard, secrets, PII, audit) | done | Recall is by keyword overlap with the task title (as for all memories) |
| Handover: task.result.handover, refreshed when a task stops or finishes, and GET /v1/team/tasks/{id}/handover (live): what was asked, the agent's own summary (marked as its claim, secrets redacted), verified and not verified checks, each write with its status and approval, what a person must do now, confidence, anomalies, assumptions, and links to the trace and approvals | done | The console does not show it yet (API only) |
| Role-agnostic prompt: "an AI employee in the {department} team" (or none), the same loop rules for admin, engineering, legal, people, support, sales and finance | done | Scenario tests cover guardrails with a scripted model, not model quality |