Armory
Source

Browse

Search and filter by type across the catalog

850 results in Sub-Agents, Evals, Infrastructure, Observability 路 page 4 of 36

EvalsExperimental

stonybrooknlp-appworld

馃實 AppWorld: A Controllable World of Apps and People for Benchmarking Function Calling and Interactive Coding Agent, ACL'24 Best Resource Paper.

Contributed by Sentinel

96.7500 stars 路 78 forks 路 4 mentions
Harnessclaudecodexcursorgeminiopencode
No one-command install 路 SourceDetails
EvalsPreview

continuous-eval

Relari's modular evaluation library for LLM pipelines with deterministic + LLM-based metrics for RAG and agent workflows.

96.2517 stars 路 38 forks
Harnessclaudecursorcodexopencodegemini
evalsragagentsmetrics
No one-command install 路 SourceDetails
ObservabilityPreview

claude-code-statusline

Enhanced 4-line statusline for Claude Code with themes, cost tracking, and MCP server monitoring

95.9476 stars 路 35 forks
Harnessclaude
statuslineobservability
No one-command install 路 SourceDetails
ObservabilityPreview

phospho

Phospho is a text analytics and evaluation platform for LLM apps. It logs sessions, runs clustering, detects failures, and surfaces actionable insights.

95.8439 stars 路 35 forks
Harnessclaudecursorcodexopencodegemini
observabilityanalyticsevals
No one-command install 路 SourceDetails
ObservabilityPreview

athina-ai

Athina AI provides developer-focused LLM monitoring and eval framework: real-time inference logging, automated evals, and regression detection in CI.

94.6301 stars 路 23 forks
Harnessclaudecursorcodexopencodegemini
observabilityevalslogging
No one-command install 路 SourceDetails
EvalsPreview

metr-task-standard

METR's Task Standard: a specification and scaffold for creating agentic tasks used in autonomous agent capability evaluations.

94.3192 stars 路 37 forks
Harnessclaudecursorcodexopencodegemini
evalsagentstask-standardsafety
No one-command install 路 SourceDetails
EvalsExperimental

harbor-framework-terminal-bench-2-1

Terminal-Bench 2.1

Contributed by Sentinel

94.3119 stars 路 62 forks 路 3 mentions
Harnessclaudecodexcursorgeminiopencode
evals
No one-command install 路 SourceDetails
ObservabilityPreview

claude-pace

A lightweight Bash + jq statusline for Claude Code that displays rate limit pace delta (burn rate vs. time remaining), 5h/7d usage percentage, context window usage, git branch and diff stats. Compares current consumption rate against time remaining in each rate limit window to indicate whether quota is being used faster or slower than the window allows. Single file with no external dependencies beyond jq.

93.6229 stars 路 19 forks
Harnessclaude
claude-codestatus-lines
No one-command install 路 SourceDetails
EvalsPreview

vellum-evals

Vellum evaluation SDK for running LLM test suites with custom metrics, dataset pinning, and CI workflow integration.

90.582 stars 路 20 forks
Harnessclaudecursorcodexopencodegemini
evalssdkcidataset
No one-command install 路 SourceDetails
EvalsPreview

braintrust

Developer platform for logging, evaluating, and comparing LLM experiments with dataset versioning and scoring functions.

82.327 stars 路 12 forks
Harnessclaudecursorcodexopencodegemini
evalsexperiment-trackingsdk
No one-command install 路 SourceDetails
ObservabilityPreview

claudia-statusline

High-performance Rust-based statusline for Claude Code with persistent stats tracking, progress bars, and optional cloud sync. Features SQLite-first persistence, git integration, context progress bars, burn rate calculation, XDG-compliant with theme support (dark/light, NO_COLOR).

81.236 stars 路 5 forks
Harnessclaude
claude-codestatus-lines
No one-command install 路 SourceDetails
EvalsPreview

galileo-evaluate

Galileo evaluation and observability SDK for detecting hallucinations, data errors, and model weaknesses in LLM pipelines.

80.322 stars 路 11 forks
Harnessclaudecursorcodexopencodegemini
evalshallucinationobservabilitysdk
No one-command install 路 SourceDetails
InfrastructurePreview

e2b

Open-source secure cloud sandboxes (Firecracker microVMs) for running AI-generated code. ~150ms cold start.

80.0passed install test
Harnessclaudecursorcodexopencodegemini
infrastructuresandbox
No one-command install 路 SourceDetails
InfrastructurePreview

modal

Serverless cloud platform for running Python functions, containers, and AI workloads with zero infra management.

80.0passed install test
Harnessclaudecursorcodexopencodegemini
infrastructureserverless
No one-command install 路 SourceDetails
InfrastructurePreview

railway

Zero-config cloud platform for deploying agent backends, databases, and services from a Git push.

80.0passed install test
Harnessclaudecursorcodexopencodegemini
infrastructuredeploy
No one-command install 路 SourceDetails
InfrastructurePreview

vercel

Frontend cloud platform with serverless functions and AI SDK integrations for deploying agent-facing UIs.

80.0passed install test
Harnessclaudecursorcodexopencodegemini
infrastructuredeploy
No one-command install 路 SourceDetails
ObservabilityPreview

honeycomb

Honeycomb's OpenTelemetry-native observability platform: a high-cardinality event store ideal for tracing LLM pipelines and debugging slow agent traces.

75.116 stars 路 6 forks
Harnessclaudecursorcodexopencodegemini
observabilityopentelemetrytracing
No one-command install 路 SourceDetails
ObservabilityPreview

baserun

Baserun captures LLM traces via a lightweight decorator-based SDK and provides a dashboard for debugging prompt chains, testing variants, and measuring quality.

74.416 stars 路 5 forks
Harnessclaudecursorcodexopencodegemini
observabilitytracingtesting
No one-command install 路 SourceDetails
EvalsPreview

humanloop-evals

Humanloop Python SDK with integrated evals, dataset versioning, and human + LLM judge scoring for production pipelines.

69.012 stars 路 3 forks
Harnessclaudecursorcodexopencodegemini
evalshuman-evaldatasetsdk
No one-command install 路 SourceDetails
EvalsPreview

agentbench

Use to put a number on harness quality: run an agent harness against a task set and get a score, so harness changes are validated by evidence. It is the eval backbone of a self-improving loop.

UnrankedNo signals yet
Harnessclaudecodex
evalbenchmarkscoringharness
No one-command install 路 SourceDetails
Sub-AgentsPreview

3d-artist

3D art and asset creation specialist for game development. Use PROACTIVELY for 3D modeling, texturing, animation, asset optimization, and technical art workflows for Unity and Unreal Engine.

UnrankedNo signals yet
Harnessclaude
game-developmentsubagents
Details
Sub-AgentsPreview

4-1-beast

An agent definition that sets GPT-4.1 up as a coding agent.

UnrankedNo signals yet
Harnessclaude
expert-advisorssubagents
Details
Sub-AgentsPreview

a11y-architect

Accessibility Architect specializing in WCAG 2.2 compliance for Web and Native platforms. Use PROACTIVELY when designing UI components, establishing design systems, or auditing code for inclusive user experiences.

UnrankedNo signals yet
Harnessclaudecodexcursorgeminiopencode
subagent
Details
Sub-AgentsPreview

ab-test-analysis

Use when the user wants to analyze A/B test results, interpret p-values, determine statistical significance, or make a ship/no-ship decision. Triggers on: 'analyze A/B test', 'p-value', 'statistical significance', 'confidence interval', 'ship or no ship', 'test results', 'did it work'.

UnrankedNo signals yet
Harnessclaude
research-analysissubagent
Details
Browse 路 Armory