oiupoyt/brokecrew
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
brokecrew/docs/superpowers/specs/2026-09-26-brokecrew-design.md

10 KiB
Raw Blame History

BrokeCrew System Design Specification

  • Project Name: BrokeCrew
  • Date: 2026-09-26
  • Status: Approved Design
  • Architecture Style: Async-first, modular CLI & Core Library (Approach A: Dual-Target Crate)

1. Executive Summary

BrokeCrew is an open-source, production-grade Rust CLI and library that enables developers to orchestrate a coordinated team of AI personas powered by OpenRouter models. BrokeCrew treats free-tier models as first-class citizens while eliminating their notorious instability through live model catalog discovery, multi-candidate fallback chains, and automatic rate-limit/budget guarding. When tasks demand advanced capabilities, a two-stage routing engine seamlessly escalates to paid model candidates. Multi-agent handoff pipelines allow personas to collaborate sequentially (e.g. coder -> reviewer) with visually distinct terminal rendering and real-time Server-Sent Events (SSE) streaming.

2. System Architecture & Boundaries

To preserve strict separation of concerns and allow future frontend expansions (such as a GUI binary or daemon service) without refactoring core logic, BrokeCrew adopts a dual-target architecture:

  1. Core Library (brokecrew / src/lib.rs):
    • Contains all domain types, OpenRouter HTTP client, live model registry, rate limiter, routing engine, persona managers, and pipeline executors.
    • Completely agnostic of terminal I/O, UI styling, and CLI argument parsing.
    • Exposes clean async APIs and channels (tokio::sync::mpsc) for streaming tokens and pipeline lifecycle events.
  2. CLI Executable (src/main.rs):
    • Thin entrypoint that handles clap argument parsing, signal trapping, terminal spinner management (indicatif), colored text formatting (owo-colors), and OS keyring integration.
brokecrew/
├── Cargo.toml
├── src/
│   ├── lib.rs              # Library root: re-exports domain modules
│   ├── main.rs             # Thin binary CLI entrypoint
│   ├── cli.rs              # clap command definitions & interactive prompts
│   ├── config.rs           # Configuration loader, paths & credential resolution
│   ├── client.rs           # ModelClient trait, OpenRouter HTTP client, SSE parser
│   ├── registry.rs         # Live OpenRouter model catalog cache & sync
│   ├── persona.rs          # Persona & ModelCandidate types, fallback cascade
│   ├── pipeline.rs         # Multi-agent typed pipeline execution
│   ├── budget.rs           # Account-wide rate limiter & local budget persistence
│   └── router/
│       ├── mod.rs          # Router coordinator (Stage A -> Stage B -> Stage C)
│       ├── keyword.rs      # Stage A: regex/keyword pre-filter
│       └── llm.rs          # Stage B: compact LLM classifier; Stage C: complexity escalation
├── config/
│   └── personas.example.toml # Default production personas & pipelines
├── tests/
│   ├── router_tests.rs     # Table-driven unit tests for classification
│   ├── fallback_tests.rs   # Fault-injection candidate cascade tests
│   └── wiremock_client.rs  # WireMock integration tests for SSE & error handling
├── .github/workflows/ci.yml
├── CODEOWNERS
├── CONTRIBUTING.md
├── CHANGELOG.md
├── LICENSE-MIT
├── LICENSE-APACHE
└── README.md

3. Detailed Component Specifications

3.1 Persona & Candidate Fallback System (persona.rs)

Personas are defined declaratively in personas.toml:

  • name: Unique identifier (e.g., "coder", "reviewer", "architect").
  • system_prompt: Instructions governing persona behavior and tone.
  • trigger_words: High-confidence keywords/phrases triggering Stage A routing.
  • candidates: Ordered list of ModelCandidate structs:
    • id: OpenRouter model identifier (e.g. "qwen/qwen-2.5-coder-32b-instruct:free").
    • tier: Free or Paid.
    • is_stealth: Boolean indicating experimental or unlisted prerelease models.

Candidate Selection & Fallback Walk

When dispatching a prompt to a persona:

  1. Filter candidates:
    • Verify model presence in the cached ModelRegistry (unless in offline bypass mode).
    • If allow_stealth is false, exclude candidates marked is_stealth: true.
    • If escalation to Tier::Paid was triggered, filter or prioritize Tier::Paid candidates; otherwise prefer Tier::Free.
  2. Sequentially attempt completion on each candidate:
    • On success: Return DispatchResult containing the reply, the model used, and whether fallback occurred.
    • On ClientError::RateLimited (429) or ClientError::ServerError (5xx) or missing model: Log structured audit event and immediately attempt the next candidate in the chain.
    • On ClientError::Unauthorized (401) or ClientError::PaymentRequired (402): Abort candidate walking immediately and surface the error.
  3. If all candidates fail: Return an aggregated AllCandidatesFailed error detailing each attempt.

3.2 Two-Stage Router & Complexity Escalation (router/)

Routing determines which persona handles an unassigned user message and whether the request requires higher-tier execution.

Stage A: Zero-Cost Keyword Pre-Filter (keyword.rs)

  • Evaluates prompt against persona trigger_words using word-boundary matching and case-insensitive scanning.
  • If high-confidence matches point unambiguously to a persona (e.g. "write a python script" -> coder), routing completes instantly with zero API calls and zero latency.

Stage B: Low-Token LLM Classifier (llm.rs)

  • Invoked only when Stage A is ambiguous or inconclusive.
  • Calls a lightweight, fast model (e.g. meta-llama/llama-3.2-3b-instruct:free or user-configured router model).
  • Strict system prompt: "You are a message router. Respond with ONLY the exact persona name from this list: [coder, reviewer, general]. Do not include explanations, punctuation, or any other words."
  • Request configured with max_tokens = 10 and temperature = 0.0.
  • Output is trimmed and validated against active personas. If unparseable or unexpected, defaults cleanly to default_persona (e.g. "general").

Stage C: Complexity Escalation (llm.rs)

  • Assesses prompt complexity heuristics:
    • Estimated token volume and instruction density.
    • Multi-file or architectural keywords ("refactor the entire repo", "implement full backend with database migrations and tests").
    • Explicit user flag --escalate.
  • When complexity exceeds complexity_threshold (configurable in config.toml, default 7/10), the router instructs the persona candidate walker to prefer paid-tier candidates.

3.3 Model Registry & Resilience (registry.rs)

  • Maintains a cached snapshot of OpenRouter's live catalog:
    • Fetches GET https://openrouter.ai/api/v1/models on startup.
    • Periodic background sync every 30 minutes via tokio::spawn.
    • Caches id, pricing (prompt, completion), context_length, and supported_parameters.
  • Safe resilience:
    • If a free model is removed from OpenRouter or marked offline, BrokeCrew skips it proactively before wasting a network roundtrip.
    • If registry fetch fails (e.g. initial offline state), BrokeCrew operates in degraded mode using static fallback configurations with warning logs.

3.4 Rate Limiting & Account Budget Awareness (budget.rs)

OpenRouter enforces strict account-wide rate limits:

  • 20 requests / minute rolling window.
  • Daily Quotas:
    • 50 requests/day for accounts with < $10 lifetime credit purchase.
    • 1,000 requests/day for accounts with ≥ $10 lifetime credit purchase.

Tracking Mechanism:

  • Local state persisted in ~/.config/brokecrew/budget.json (or platform config directory) using atomic file writes (tempfile + rename).
  • Every outbound call—including Stage B router queries—is accounted for.
  • State tracks:
    • Recent call timestamps for the rolling 60-second window.
    • Cumulative daily counter reset at UTC midnight.
  • Prior to firing any request:
    • If rolling minute count >= 20: Pause and wait for window decay, or warn the user.
    • If daily count reaches 80% of quota: Emit warning in CLI.
    • If daily count hits quota ceiling: Halt execution with clear explanation rather than triggering a silent 429.

3.5 Multi-Agent Handoff Pipeline (pipeline.rs)

Defined in personas.toml:

[[pipeline]]
name = "build-and-review"
steps = [
  { persona = "coder" },
  { persona = "reviewer", pass_previous_output = true }
]
  • Steps execute sequentially; step N receives the accumulated output of step N-1 when pass_previous_output = true.
  • Output is emitted as streaming events over a channel:
    • PipelineEvent::StepStarted { persona, model, tier }
    • PipelineEvent::Chunk(String)
    • PipelineEvent::StepCompleted { persona, total_tokens }
  • The CLI formats each step distinctly with colored headers and persona avatars, ensuring personas never blend into an anonymous voice.

3.6 Security & Keyring Integration (config.rs)

  • API keys are resolved with the following priority:
    1. CLI parameter --api-key <KEY>
    2. Environment variable OPENROUTER_API_KEY
    3. OS Credential Manager via keyring crate (keyring::Entry::new("brokecrew", "openrouter_api_key"))
    4. Local configuration file (config.toml)
  • Key sanitization:
    • Struct fields containing API keys implement a custom Debug that redacts values to "[REDACTED]".
    • tracing spans and error formatters never output raw tokens or authorization headers.

4. Acceptance Criteria & Verification

  1. Cargo Install & Zero-Config Execution:
    • cargo build --release succeeds without warnings (-D warnings).
    • Default personas.toml provides coder, reviewer, architect, and general personas with working free OpenRouter candidates.
  2. Resilience & Fallback Walk:
    • Simulating 503 or 429 on Candidate 1 routes seamlessly to Candidate 2 with visible fallback logs.
  3. Budget Guarding:
    • brokecrew status accurately displays remaining quota and rolling minute usage.
  4. Pipeline Separation:
    • brokecrew chat --pipeline build-and-review "Create a LRU cache" displays [coder] output followed by [reviewer] output in distinct colored blocks.
  5. Security & Privacy:
    • No API keys leaked across any logging level (--json-logs or stdout).
  6. Cross-Platform & CI:
    • Full test suite passes on Linux, macOS, and Windows runners in GitHub Actions.