147 lines
5.3 KiB
Markdown
147 lines
5.3 KiB
Markdown
# BrokeCrew
|
|
|
|
[](LICENSE-MIT)
|
|
[](https://crates.io/crates/brokecrew)
|
|
|
|
**A lightweight, resilient, async-first Rust CLI that coordinates a specialized team of AI personas backed by OpenRouter models.**
|
|
|
|
BrokeCrew makes free-tier models first-class citizens while eliminating their instability through live model catalog discovery, ranked multi-candidate fallback cascades, account-wide rate-limit budgeting, and multi-agent handoff pipelines.
|
|
|
|
---
|
|
|
|
## Why BrokeCrew (vs LiteLLM or RouteLLM)?
|
|
|
|
| Feature | BrokeCrew | LiteLLM Proxy / RouteLLM |
|
|
| :--- | :--- | :--- |
|
|
| **Footprint & Startup** | Single standalone native binary (~15MB), sub-10ms startup. | Heavy Python runtime, tens of pip dependencies, slower startup. |
|
|
| **Hardware Fit** | Designed to run smoothly on low-RAM ARM tablets, Raspberry Pis, and local dev boxes. | Heavy memory consumption unsuited for constrained devices. |
|
|
| **Persona Fallback Walk** | Declarative ranked candidates per persona (`qwen:free` → `deepseek:free` → paid fallback). | Static routing with complex configuration layers. |
|
|
| **Live Catalog Awareness** | Syncs `/v1/models` in background; skips dropped/offline models before wasting requests. | Often returns raw 404/400 errors when models vanish. |
|
|
| **Account-Wide Budgeting** | Tracks rolling 20 req/min and daily quotas (50 or 1,000 req/day) across all calls including router. | Typically only tracks proxy tokens, not OpenRouter account limits. |
|
|
| **Multi-Agent Handoffs** | Built-in typed pipelines (`coder` → `reviewer`) with real-time SSE streaming per persona. | Requires external orchestration frameworks (LangGraph, CrewAI). |
|
|
|
|
---
|
|
|
|
### Technical Invariants
|
|
- **Fail-Fast Cascade Walk**: Free endpoints on OpenRouter frequently return transient `429`, `503`, or drop out of `/v1/models` without warning. Candidate resolution treats failures as cascade triggers rather than fatal aborts, stepping to the next viable candidate model within `<250ms`.
|
|
- **Sliding-Window Budgeting**: Token usage and request counts are stored in atomic sliding windows (`VecDeque<DateTime<Utc>>`). Router and classifier calls deduct from the shared account budget to prevent downstream API lockouts.
|
|
- **Async Non-Blocking SSE**: Streaming tokens are decoupled from the Ratatui TUI render loop via crossbeam/tokio MPSC channels, ensuring UI frame rates remain at 60 FPS even during erratic remote network jitter.
|
|
|
|
---
|
|
|
|
## Brokecrew Interactive TUI
|
|
|
|
Simply run `brokecrew` to enter the rich, full-screen interactive TUI:
|
|
|
|
```bash
|
|
brokecrew
|
|
```
|
|
|
|
- **Live Team View**: Color-coded persona indicators (`● coder`, `● reviewer`, `● architect`), token streaming with markdown & syntax formatting.
|
|
- **Quota & Budget Monitor**: Real-time headers displaying rolling minute and 24h limits (`1m: 2/20`, `24h: 14/50`).
|
|
- **Interactive Modals**:
|
|
- `[Tab]` — Open Persona Selector (switch between intelligent Stage A/B `Auto-Route` and dedicated personas).
|
|
- `[Ctrl+P]` — Open Pipeline Selector (execute multi-agent pipelines like `build-and-review`).
|
|
- `[PgUp/PgDn]` — Scroll conversation history.
|
|
- `[Ctrl+L]` — Clear screen.
|
|
- `[Esc]` — Dismiss modal or exit.
|
|
- **Non-TUI / Scripting**: Use `brokecrew --cli` or `brokecrew chat "..."` for headless pipelines or script integration.
|
|
|
|
---
|
|
|
|
## Quickstart
|
|
|
|
### 1. Installation
|
|
|
|
**One-line curl install (Linux & macOS):**
|
|
```bash
|
|
curl -fsSL https://raw.githubusercontent.com/oiupoyt/brokecrew/main/install.sh | bash
|
|
```
|
|
|
|
**Via Cargo (Crates.io):**
|
|
```bash
|
|
cargo install brokecrew
|
|
```
|
|
|
|
**Build from source:**
|
|
```bash
|
|
git clone https://github.com/oiupoyt/brokecrew.git
|
|
cd brokecrew
|
|
cargo install --path .
|
|
```
|
|
|
|
### 2. Configure OpenRouter API Key
|
|
|
|
BrokeCrew securely stores your API key in your OS credential store (`keyring`), never in plaintext:
|
|
|
|
```bash
|
|
brokecrew config set-key
|
|
```
|
|
|
|
*(You can also export `OPENROUTER_API_KEY="sk-or-v1-..."` or pass `--api-key` for headless servers).*
|
|
|
|
### 3. Start Chatting
|
|
|
|
Zero-config instant routing:
|
|
```bash
|
|
brokecrew chat "build me a proxy checker site"
|
|
```
|
|
*Stage A regex/keyword routing automatically detects "build/site" and dispatches to the `coder` persona in 0ms!*
|
|
|
|
Run an interactive team session:
|
|
```bash
|
|
brokecrew chat
|
|
```
|
|
|
|
Run a multi-agent handoff pipeline:
|
|
```bash
|
|
brokecrew chat --pipeline build-and-review "Implement a token bucket rate limiter in Python"
|
|
```
|
|
|
|
Check model catalog and quota health:
|
|
```bash
|
|
brokecrew status
|
|
```
|
|
|
|
List available personas:
|
|
```bash
|
|
brokecrew personas list
|
|
```
|
|
|
|
---
|
|
|
|
## Declarative Personas (`personas.toml`)
|
|
|
|
Personas are defined in `~/.config/brokecrew/personas.toml` (or `./config/personas.example.toml`):
|
|
|
|
```toml
|
|
[[persona]]
|
|
name = "coder"
|
|
system_prompt = "You are a precise, terse coding assistant. Output working code, minimal commentary."
|
|
trigger_words = ["build", "code", "script", "function", "website", "api"]
|
|
|
|
[[persona.candidates]]
|
|
id = "qwen/qwen-2.5-coder-32b-instruct:free"
|
|
tier = "free"
|
|
is_stealth = false
|
|
|
|
[[persona.candidates]]
|
|
id = "deepseek/deepseek-coder:free"
|
|
tier = "free"
|
|
is_stealth = false
|
|
|
|
[[persona.candidates]]
|
|
id = "google/gemini-2.0-flash-001"
|
|
tier = "paid"
|
|
is_stealth = false
|
|
```
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
Dual-licensed under either:
|
|
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))
|
|
- MIT license ([LICENSE-MIT](LICENSE-MIT))
|
|
|
|
at your option.
|