From edab67dbbca36a7134b09c380bfd841196fa74a2 Mon Sep 17 00:00:00 2001 From: oiupoyt Date: Sat, 26 Sep 2026 13:28:17 +0530 Subject: [PATCH] docs: add formal system design specification for BrokeCrew CLI --- .../specs/2026-09-26-brokecrew-design.md | 192 ++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-26-brokecrew-design.md diff --git a/docs/superpowers/specs/2026-09-26-brokecrew-design.md b/docs/superpowers/specs/2026-09-26-brokecrew-design.md new file mode 100644 index 0000000..be05ac3 --- /dev/null +++ b/docs/superpowers/specs/2026-09-26-brokecrew-design.md @@ -0,0 +1,192 @@ +# BrokeCrew System Design Specification + +- **Project Name:** BrokeCrew +- **Date:** 2026-09-26 +- **Status:** Approved Design +- **Architecture Style:** Async-first, modular CLI & Core Library (Approach A: Dual-Target Crate) + +--- + +## 1. Executive Summary + +BrokeCrew is an open-source, production-grade Rust CLI and library that enables developers to orchestrate a coordinated team of AI personas powered by OpenRouter models. BrokeCrew treats free-tier models as first-class citizens while eliminating their notorious instability through live model catalog discovery, multi-candidate fallback chains, and automatic rate-limit/budget guarding. When tasks demand advanced capabilities, a two-stage routing engine seamlessly escalates to paid model candidates. Multi-agent handoff pipelines allow personas to collaborate sequentially (e.g. coder -> reviewer) with visually distinct terminal rendering and real-time Server-Sent Events (SSE) streaming. + +--- + +## 2. System Architecture & Boundaries + +To preserve strict separation of concerns and allow future frontend expansions (such as a GUI binary or daemon service) without refactoring core logic, BrokeCrew adopts a dual-target architecture: +1. **Core Library (`brokecrew` / `src/lib.rs`)**: + - Contains all domain types, OpenRouter HTTP client, live model registry, rate limiter, routing engine, persona managers, and pipeline executors. + - Completely agnostic of terminal I/O, UI styling, and CLI argument parsing. + - Exposes clean async APIs and channels (`tokio::sync::mpsc`) for streaming tokens and pipeline lifecycle events. +2. **CLI Executable (`src/main.rs`)**: + - Thin entrypoint that handles `clap` argument parsing, signal trapping, terminal spinner management (`indicatif`), colored text formatting (`owo-colors`), and OS keyring integration. + +``` +brokecrew/ +├── Cargo.toml +├── src/ +│ ├── lib.rs # Library root: re-exports domain modules +│ ├── main.rs # Thin binary CLI entrypoint +│ ├── cli.rs # clap command definitions & interactive prompts +│ ├── config.rs # Configuration loader, paths & credential resolution +│ ├── client.rs # ModelClient trait, OpenRouter HTTP client, SSE parser +│ ├── registry.rs # Live OpenRouter model catalog cache & sync +│ ├── persona.rs # Persona & ModelCandidate types, fallback cascade +│ ├── pipeline.rs # Multi-agent typed pipeline execution +│ ├── budget.rs # Account-wide rate limiter & local budget persistence +│ └── router/ +│ ├── mod.rs # Router coordinator (Stage A -> Stage B -> Stage C) +│ ├── keyword.rs # Stage A: regex/keyword pre-filter +│ └── llm.rs # Stage B: compact LLM classifier; Stage C: complexity escalation +├── config/ +│ └── personas.example.toml # Default production personas & pipelines +├── tests/ +│ ├── router_tests.rs # Table-driven unit tests for classification +│ ├── fallback_tests.rs # Fault-injection candidate cascade tests +│ └── wiremock_client.rs # WireMock integration tests for SSE & error handling +├── .github/workflows/ci.yml +├── CODEOWNERS +├── CONTRIBUTING.md +├── CHANGELOG.md +├── LICENSE-MIT +├── LICENSE-APACHE +└── README.md +``` + +--- + +## 3. Detailed Component Specifications + +### 3.1 Persona & Candidate Fallback System (`persona.rs`) + +Personas are defined declaratively in `personas.toml`: +- `name`: Unique identifier (e.g., `"coder"`, `"reviewer"`, `"architect"`). +- `system_prompt`: Instructions governing persona behavior and tone. +- `trigger_words`: High-confidence keywords/phrases triggering Stage A routing. +- `candidates`: Ordered list of `ModelCandidate` structs: + - `id`: OpenRouter model identifier (e.g. `"qwen/qwen-2.5-coder-32b-instruct:free"`). + - `tier`: `Free` or `Paid`. + - `is_stealth`: Boolean indicating experimental or unlisted prerelease models. + +#### Candidate Selection & Fallback Walk +When dispatching a prompt to a persona: +1. Filter candidates: + - Verify model presence in the cached `ModelRegistry` (unless in offline bypass mode). + - If `allow_stealth` is `false`, exclude candidates marked `is_stealth: true`. + - If escalation to `Tier::Paid` was triggered, filter or prioritize `Tier::Paid` candidates; otherwise prefer `Tier::Free`. +2. Sequentially attempt completion on each candidate: + - On success: Return `DispatchResult` containing the reply, the model used, and whether fallback occurred. + - On `ClientError::RateLimited` (429) or `ClientError::ServerError` (5xx) or missing model: Log structured audit event and immediately attempt the next candidate in the chain. + - On `ClientError::Unauthorized` (401) or `ClientError::PaymentRequired` (402): Abort candidate walking immediately and surface the error. +3. If all candidates fail: Return an aggregated `AllCandidatesFailed` error detailing each attempt. + +--- + +### 3.2 Two-Stage Router & Complexity Escalation (`router/`) + +Routing determines which persona handles an unassigned user message and whether the request requires higher-tier execution. + +#### Stage A: Zero-Cost Keyword Pre-Filter (`keyword.rs`) +- Evaluates prompt against persona `trigger_words` using word-boundary matching and case-insensitive scanning. +- If high-confidence matches point unambiguously to a persona (e.g. "write a python script" -> `coder`), routing completes instantly with zero API calls and zero latency. + +#### Stage B: Low-Token LLM Classifier (`llm.rs`) +- Invoked only when Stage A is ambiguous or inconclusive. +- Calls a lightweight, fast model (e.g. `meta-llama/llama-3.2-3b-instruct:free` or user-configured router model). +- Strict system prompt: `"You are a message router. Respond with ONLY the exact persona name from this list: [coder, reviewer, general]. Do not include explanations, punctuation, or any other words."` +- Request configured with `max_tokens = 10` and `temperature = 0.0`. +- Output is trimmed and validated against active personas. If unparseable or unexpected, defaults cleanly to `default_persona` (e.g. `"general"`). + +#### Stage C: Complexity Escalation (`llm.rs`) +- Assesses prompt complexity heuristics: + - Estimated token volume and instruction density. + - Multi-file or architectural keywords (`"refactor the entire repo"`, `"implement full backend with database migrations and tests"`). + - Explicit user flag `--escalate`. +- When complexity exceeds `complexity_threshold` (configurable in `config.toml`, default 7/10), the router instructs the persona candidate walker to prefer paid-tier candidates. + +--- + +### 3.3 Model Registry & Resilience (`registry.rs`) + +- Maintains a cached snapshot of OpenRouter's live catalog: + - Fetches `GET https://openrouter.ai/api/v1/models` on startup. + - Periodic background sync every 30 minutes via `tokio::spawn`. + - Caches `id`, `pricing` (prompt, completion), `context_length`, and `supported_parameters`. +- Safe resilience: + - If a free model is removed from OpenRouter or marked offline, BrokeCrew skips it proactively before wasting a network roundtrip. + - If registry fetch fails (e.g. initial offline state), BrokeCrew operates in degraded mode using static fallback configurations with warning logs. + +--- + +### 3.4 Rate Limiting & Account Budget Awareness (`budget.rs`) + +OpenRouter enforces strict account-wide rate limits: +- **20 requests / minute** rolling window. +- **Daily Quotas**: + - 50 requests/day for accounts with < $10 lifetime credit purchase. + - 1,000 requests/day for accounts with ≥ $10 lifetime credit purchase. + +#### Tracking Mechanism: +- Local state persisted in `~/.config/brokecrew/budget.json` (or platform config directory) using atomic file writes (`tempfile` + rename). +- Every outbound call—including Stage B router queries—is accounted for. +- State tracks: + - Recent call timestamps for the rolling 60-second window. + - Cumulative daily counter reset at UTC midnight. +- Prior to firing any request: + - If rolling minute count >= 20: Pause and wait for window decay, or warn the user. + - If daily count reaches 80% of quota: Emit warning in CLI. + - If daily count hits quota ceiling: Halt execution with clear explanation rather than triggering a silent 429. + +--- + +### 3.5 Multi-Agent Handoff Pipeline (`pipeline.rs`) + +Defined in `personas.toml`: +```toml +[[pipeline]] +name = "build-and-review" +steps = [ + { persona = "coder" }, + { persona = "reviewer", pass_previous_output = true } +] +``` + +- Steps execute sequentially; step $N$ receives the accumulated output of step $N-1$ when `pass_previous_output = true`. +- Output is emitted as streaming events over a channel: + - `PipelineEvent::StepStarted { persona, model, tier }` + - `PipelineEvent::Chunk(String)` + - `PipelineEvent::StepCompleted { persona, total_tokens }` +- The CLI formats each step distinctly with colored headers and persona avatars, ensuring personas never blend into an anonymous voice. + +--- + +### 3.6 Security & Keyring Integration (`config.rs`) + +- API keys are resolved with the following priority: + 1. CLI parameter `--api-key ` + 2. Environment variable `OPENROUTER_API_KEY` + 3. OS Credential Manager via `keyring` crate (`keyring::Entry::new("brokecrew", "openrouter_api_key")`) + 4. Local configuration file (`config.toml`) +- Key sanitization: + - Struct fields containing API keys implement a custom `Debug` that redacts values to `"[REDACTED]"`. + - `tracing` spans and error formatters never output raw tokens or authorization headers. + +--- + +## 4. Acceptance Criteria & Verification + +1. **Cargo Install & Zero-Config Execution**: + - `cargo build --release` succeeds without warnings (`-D warnings`). + - Default `personas.toml` provides `coder`, `reviewer`, `architect`, and `general` personas with working free OpenRouter candidates. +2. **Resilience & Fallback Walk**: + - Simulating 503 or 429 on Candidate 1 routes seamlessly to Candidate 2 with visible fallback logs. +3. **Budget Guarding**: + - `brokecrew status` accurately displays remaining quota and rolling minute usage. +4. **Pipeline Separation**: + - `brokecrew chat --pipeline build-and-review "Create a LRU cache"` displays `[coder]` output followed by `[reviewer]` output in distinct colored blocks. +5. **Security & Privacy**: + - No API keys leaked across any logging level (`--json-logs` or stdout). +6. **Cross-Platform & CI**: + - Full test suite passes on Linux, macOS, and Windows runners in GitHub Actions.