Browse

openai-evals

OpenAI's official framework for evaluating LLMs and LLM-powered systems, with a registry of community eval sets.

lm-evaluation-harness

EleutherAI's unified framework for evaluating language models on hundreds of academic benchmarks.

browserbase-bb

Use when an agent must operate the live web (navigate, act, and extract on real pages) via a cloud browser driven by act/extract/observe primitives, with a local-Chromium escape hatch using the same code.

gibsonai-memori

Memori is agent-native memory infrastructure. A LLM-agnostic layer that turns agent execution and conversation into structured, persistent state for production systems. Built for enterprise, Memori works with the data infrastructure you already run, no rip-and-replace, and deploys across managed cloud, single-tenant cloud, VPC, and on-premises.

Contributed by Sentinel

comet-opik

Opik by Comet is an open-source LLM evaluation and tracing platform: log traces, run automated evals, create datasets, and track prompt improvements over time.

browser-use-webui

Gradio web UI on top of the browser-use framework. It lets users run AI browser agents interactively, configure LLM providers, watch live recordings, and replay task sessions without writing Python.

opentelemetry-genai

OpenTelemetry semantic conventions and instrumentation for GenAI/LLM spans, traces, and metrics via the GenAI semconv working group.

cua-computer-use-agent

trycua/cua open-source computer-use agent framework: Apple Silicon-native, runs lightweight macOS/Linux VMs with sub-second cold starts; provides a unified Python interface for screen capture, click, and type actions.

deepeval

Open-source LLM evaluation framework with 14+ metrics (hallucination, faithfulness, answer relevancy) and CI support.

ragas

Reference-free evaluation of retrieval-augmented generation pipelines; measures faithfulness, answer relevance, and context precision.

mcp-tunnels-cloudflared

Use to let a hosted agent reach a private-data MCP server behind your firewall (an outbound tunnel plus a proxy, with per-server OAuth) so internal tools are usable without exposing them to the public internet.

semantica-agi-semantica

Use when an agent's stored context needs provenance — you must be able to say where a remembered fact came from.

nanobrowser

Open-source Chrome extension that runs a multi-agent browser automation system locally. Planner, Navigator, and Validator agents collaborate inside the browser with no external API calls for web tasks.

nevamind-ai-memu

Use when one person's memory should follow them across several different agents.

portkey-ai-gateway

Open-source AI gateway providing a unified API across 100+ LLM providers with built-in observability, request logging, fallbacks, caching, and load balancing.

browserless

Browserless.io headless browser service. It provides a Docker-deployable or cloud-hosted Chrome endpoint with REST and WebSocket APIs for screenshot, PDF, scraping, and Puppeteer/Playwright remote sessions.

bytebot

Bytebot open-source computer-use agent: a Docker-based Ubuntu desktop with AI-controlled mouse and keyboard; exposes an HTTP API for agents to send click, type, screenshot, and macro commands.

evermind-ai-everos

Use when you want the agent's memory to be plain Markdown on your own disk rather than a hosted database.

memtensor-memos

Use when memory should persist across tasks and be reused, not just retrieved once per conversation.

phoenix

Arize Phoenix: open-source LLM observability with built-in evals, span tracing, and dataset curation for RAG and agents.

ccstatusline

A highly customizable status line formatter for Claude Code CLI that displays model info, git branch, token usage, and other metrics in your terminal.

openllmetry

OpenTelemetry-based observability for LLM applications. It auto-instruments OpenAI, Anthropic, LangChain, and 20+ providers with zero code changes.

steel-browser

Open-source browser API optimised for AI agents. It provides session management, stealth settings, proxy rotation, and a REST/WebSocket interface on top of Chromium for cloud-scale agent browser access.

plastic-labs-honcho

Memory library for building stateful agents

Contributed by Sentinel