Armory
Source
Browse
Skills

nemo-evaluator-sdk

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

Score
Unranked
Evidence
No signals yet
Last commit
Not known
Listed

Install

armory install nemo-evaluator-sdk --cli claude

writes the skill into.claude/skills/nemo-evaluator-sdk/SKILL.mdListed as compatible

Configuration
# fetches the source and writes it to:
.claude/skills/nemo-evaluator-sdk/SKILL.md

Needs the armory CLI · not on npm yet, build it from cli/ in the repository

What it is

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

When to use it

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.

How to install / invoke

# Copy the skill into your .claude/skills/ directory
curl -sL https://raw.githubusercontent.com/davila7/claude-code-templates/main/cli-tool/components/skills/ai-research/evaluation-nemo-evaluator/SKILL.md -o .claude/skills/evaluation-nemo-evaluator/SKILL.md

Notes

Extracted from davila7/claude-code-templates, evaluation-nemo-evaluator category.