227 results in CLIs & Tools, Evals, Observability · page 9 of 10
ObservabilityPreview
baserun
Baserun captures LLM traces via a lightweight decorator-based SDK and provides a dashboard for debugging prompt chains, testing variants, and measuring quality.
CLI that detects dead `.claude/rules/` files by checking if `paths:` globs actually match files in your repo. Catches silent rule failures where renamed directories or typos in glob patterns cause rules to never apply. Features CI mode (exit 1 on dead rules), JSON output, and verbose mode showing matched files.
Use to run many agents in parallel as a visible grid of terminal panes: create an NxM layout, name and monitor panes, broadcast or target prompts, and save/restore whole company configurations.
Armory: a ranked catalog of 64,000+ open-source agent-harness components with a CLI (`armory search|install|init|rank`), an MCP server and a REST API; every row carries one 0–100 score from public signals.
Use to scaffold a complete agent-native harness: the CLAUDE.md, rules, skills, hooks, sub-agents, and memory layout, so a new project starts with the full discipline stack instead of an empty repo.
Use to orchestrate a swarm of sub-agents under a CEO pattern when one agent isn't enough but a full visible grid is overkill: break a mission into roles, dispatch them, and coordinate via signal files.
Use to track and cap what an agent run costs: meter token/compute spend, set budgets, and surface cost as a first-class signal so an autonomous agent doesn't quietly burn through its limit.
Use to put a number on harness quality: run an agent harness against a task set and get a score, so harness changes are validated by evidence. It is the eval backbone of a self-improving loop.
Use to apply deterministic, zero-LLM code transforms: fast, repeatable edits that don't need a model, so an agent offloads mechanical changes to a cheap tool instead of spending tokens reasoning through them.
Use to author, test, and share agent skills from the command line: scaffold a SKILL.md, validate its structure, and package it for reuse, turning a one-off procedure into a portable capability.
Fiddler AI Observability platform monitors LLM applications for hallucinations, toxicity, bias, and drift, with explainability and alerting for production AI.