brokecrew/docs/superpowers/specs/2026-09-26-brokecrew-design.md

BrokeCrew System Design Specification

• Project Name: BrokeCrew
• Date: 2026-09-26
• Status: Approved Design
• Architecture Style: Async-first, modular CLI & Core Library (Approach A: Dual-Target Crate)

---

1. Executive Summary

BrokeCrew is an open-source, production-grade Rust CLI and library that enables developers to orchestrate a coordinated team of AI personas powered by OpenRouter models. BrokeCrew treats free-tier models as first-class citizens while eliminating their notorious instability through live model catalog discovery, multi-candidate fallback chains, and automatic rate-limit/budget guarding. When tasks demand advanced capabilities, a two-stage routing engine seamlessly escalates to paid model candidates. Multi-agent handoff pipelines allow personas to collaborate sequentially (e.g. coder -> reviewer) with visually distinct terminal rendering and real-time Server-Sent Events (SSE) streaming.

---

2. System Architecture & Boundaries

To preserve strict separation of concerns and allow future frontend expansions (such as a GUI binary or daemon service) without refactoring core logic, BrokeCrew adopts a dual-target architecture:

1. Core Library (brokecrew / src/lib.rs):
   • Contains all domain types, OpenRouter HTTP client, live model registry, rate limiter, routing engine, persona managers, and pipeline executors.
   • Completely agnostic of terminal I/O, UI styling, and CLI argument parsing.
   • Exposes clean async APIs and channels (tokio::sync::mpsc) for streaming tokens and pipeline lifecycle events.
2. CLI Executable (src/main.rs):
   • Thin entrypoint that handles clap argument parsing, signal trapping, terminal spinner management (indicatif), colored text formatting (owo-colors), and OS keyring integration.

brokecrew/
├── Cargo.toml
├── src/
│   ├── lib.rs              # Library root: re-exports domain modules
│   ├── main.rs             # Thin binary CLI entrypoint
│   ├── cli.rs              # clap command definitions & interactive prompts
│   ├── config.rs           # Configuration loader, paths & credential resolution
│   ├── client.rs           # ModelClient trait, OpenRouter HTTP client, SSE parser
│   ├── registry.rs         # Live OpenRouter model catalog cache & sync
│   ├── persona.rs          # Persona & ModelCandidate types, fallback cascade
│   ├── pipeline.rs         # Multi-agent typed pipeline execution
│   ├── budget.rs           # Account-wide rate limiter & local budget persistence
│   └── router/
│       ├── mod.rs          # Router coordinator (Stage A -> Stage B -> Stage C)
│       ├── keyword.rs      # Stage A: regex/keyword pre-filter
│       └── llm.rs          # Stage B: compact LLM classifier; Stage C: complexity escalation
├── config/
│   └── personas.example.toml # Default production personas & pipelines
├── tests/
│   ├── router_tests.rs     # Table-driven unit tests for classification
│   ├── fallback_tests.rs   # Fault-injection candidate cascade tests
│   └── wiremock_client.rs  # WireMock integration tests for SSE & error handling
├── .github/workflows/ci.yml
├── CODEOWNERS
├── CONTRIBUTING.md
├── CHANGELOG.md
├── LICENSE-MIT
├── LICENSE-APACHE
└── README.md

---

3. Detailed Component Specifications

3.1 Persona & Candidate Fallback System (persona.rs)

Personas are defined declaratively in personas.toml:

• name: Unique identifier (e.g., "coder", "reviewer", "architect").
• system_prompt: Instructions governing persona behavior and tone.
• trigger_words: High-confidence keywords/phrases triggering Stage A routing.
• candidates: Ordered list of ModelCandidate structs:
  • id: OpenRouter model identifier (e.g. "qwen/qwen-2.5-coder-32b-instruct:free").
  • tier: Free or Paid.
  • is_stealth: Boolean indicating experimental or unlisted prerelease models.

Candidate Selection & Fallback Walk

When dispatching a prompt to a persona:

1. Filter candidates:
   • Verify model presence in the cached ModelRegistry (unless in offline bypass mode).
   • If allow_stealth is false, exclude candidates marked is_stealth: true.
   • If escalation to Tier::Paid was triggered, filter or prioritize Tier::Paid candidates; otherwise prefer Tier::Free.
2. Sequentially attempt completion on each candidate:
   • On success: Return DispatchResult containing the reply, the model used, and whether fallback occurred.
   • On ClientError::RateLimited (429) or ClientError::ServerError (5xx) or missing model: Log structured audit event and immediately attempt the next candidate in the chain.
   • On ClientError::Unauthorized (401) or ClientError::PaymentRequired (402): Abort candidate walking immediately and surface the error.
3. If all candidates fail: Return an aggregated AllCandidatesFailed error detailing each attempt.

---

3.2 Two-Stage Router & Complexity Escalation (router/)

Routing determines which persona handles an unassigned user message and whether the request requires higher-tier execution.

Stage A: Zero-Cost Keyword Pre-Filter (keyword.rs)

• Evaluates prompt against persona trigger_words using word-boundary matching and case-insensitive scanning.
• If high-confidence matches point unambiguously to a persona (e.g. "write a python script" -> coder), routing completes instantly with zero API calls and zero latency.

Stage B: Low-Token LLM Classifier (llm.rs)

• Invoked only when Stage A is ambiguous or inconclusive.
• Calls a lightweight, fast model (e.g. meta-llama/llama-3.2-3b-instruct:free or user-configured router model).
• Strict system prompt: "You are a message router. Respond with ONLY the exact persona name from this list: [coder, reviewer, general]. Do not include explanations, punctuation, or any other words."
• Request configured with max_tokens = 10 and temperature = 0.0.
• Output is trimmed and validated against active personas. If unparseable or unexpected, defaults cleanly to default_persona (e.g. "general").

Stage C: Complexity Escalation (llm.rs)

• Assesses prompt complexity heuristics:
  • Estimated token volume and instruction density.
  • Multi-file or architectural keywords ("refactor the entire repo", "implement full backend with database migrations and tests").
  • Explicit user flag --escalate.
• When complexity exceeds complexity_threshold (configurable in config.toml, default 7/10), the router instructs the persona candidate walker to prefer paid-tier candidates.

---

3.3 Model Registry & Resilience (registry.rs)

• Maintains a cached snapshot of OpenRouter's live catalog:
  • Fetches GET https://openrouter.ai/api/v1/models on startup.
  • Periodic background sync every 30 minutes via tokio::spawn.
  • Caches id, pricing (prompt, completion), context_length, and supported_parameters.
• Safe resilience:
  • If a free model is removed from OpenRouter or marked offline, BrokeCrew skips it proactively before wasting a network roundtrip.
  • If registry fetch fails (e.g. initial offline state), BrokeCrew operates in degraded mode using static fallback configurations with warning logs.

---

3.4 Rate Limiting & Account Budget Awareness (budget.rs)

OpenRouter enforces strict account-wide rate limits:

• 20 requests / minute rolling window.
• Daily Quotas:
  • 50 requests/day for accounts with < $10 lifetime credit purchase.
  • 1,000 requests/day for accounts with ≥ $10 lifetime credit purchase.

Tracking Mechanism:

• Local state persisted in ~/.config/brokecrew/budget.json (or platform config directory) using atomic file writes (tempfile + rename).
• Every outbound call—including Stage B router queries—is accounted for.
• State tracks:
  • Recent call timestamps for the rolling 60-second window.
  • Cumulative daily counter reset at UTC midnight.
• Prior to firing any request:
  • If rolling minute count >= 20: Pause and wait for window decay, or warn the user.
  • If daily count reaches 80% of quota: Emit warning in CLI.
  • If daily count hits quota ceiling: Halt execution with clear explanation rather than triggering a silent 429.

---

3.5 Multi-Agent Handoff Pipeline (pipeline.rs)

Defined in personas.toml:

[[pipeline]]
name = "build-and-review"
steps = [
  { persona = "coder" },
  { persona = "reviewer", pass_previous_output = true }
]

• Steps execute sequentially; step N receives the accumulated output of step N-1 when pass_previous_output = true.
• Output is emitted as streaming events over a channel:
  • PipelineEvent::StepStarted { persona, model, tier }
  • PipelineEvent::Chunk(String)
  • PipelineEvent::StepCompleted { persona, total_tokens }
• The CLI formats each step distinctly with colored headers and persona avatars, ensuring personas never blend into an anonymous voice.

---

3.6 Security & Keyring Integration (config.rs)

• API keys are resolved with the following priority:
  1. CLI parameter --api-key <KEY>
  2. Environment variable OPENROUTER_API_KEY
  3. OS Credential Manager via keyring crate (keyring::Entry::new("brokecrew", "openrouter_api_key"))
  4. Local configuration file (config.toml)
• Key sanitization:
  • Struct fields containing API keys implement a custom Debug that redacts values to "[REDACTED]".
  • tracing spans and error formatters never output raw tokens or authorization headers.

---

4. Acceptance Criteria & Verification

1. Cargo Install & Zero-Config Execution:
   • cargo build --release succeeds without warnings (-D warnings).
   • Default personas.toml provides coder, reviewer, architect, and general personas with working free OpenRouter candidates.
2. Resilience & Fallback Walk:
   • Simulating 503 or 429 on Candidate 1 routes seamlessly to Candidate 2 with visible fallback logs.
3. Budget Guarding:
   • brokecrew status accurately displays remaining quota and rolling minute usage.
4. Pipeline Separation:
   • brokecrew chat --pipeline build-and-review "Create a LRU cache" displays [coder] output followed by [reviewer] output in distinct colored blocks.
5. Security & Privacy:
   • No API keys leaked across any logging level (--json-logs or stdout).
6. Cross-Platform & CI:
   • Full test suite passes on Linux, macOS, and Windows runners in GitHub Actions.