Language: 日本語English

plugin-eval Codex Plugin

Author
OpenAI Codex
Category
Developer Tools
Topics
Claude Code Customization & Workflow · Code Quality, Review & Testing
Version
0.1.2
First seen
2026-07-12
Last confirmed
2026-07-13
Explanation last updated
2026-07-12

The explanation below is AI-generated. Please verify it against the sources.

plugin-eval is a Codex plugin and companion local Node.js CLI (Node >=20) that lets engineers evaluate Codex skills and plugins directly from chat. According to the README, it offers a chat-first "start" entrypoint that routes natural requests like "Evaluate this skill" to explicit local commands. Core commands include analyze, explain-budget, measurement-plan, init-benchmark, benchmark, report, and compare. The README states that analyze, explain-budget, and measurement-plan are deterministic local workflows that only read files and write local reports, while benchmark runs real codex exec sessions in isolated temp workspaces and stores artifacts under .plugin-eval/. It is published under the MIT license by OpenAI Codex (per plugin.json).

Overview

According to plugin.json and the README, plugin-eval is a Codex plugin bundle (with a manifest at .codex-plugin/plugin.json and chat-facing skills under skills/) that is also distributed as a local Node.js CLI (scripts/plugin-eval.js), not a hosted SaaS. It is currently marked private in package metadata, so the README describes installing it from a local checkout (via node ./scripts/plugin-eval.js or npm link) rather than from the public npm registry, and installing the Codex plugin via a local marketplace.json entry.

What you can do with plugin-eval

  • Ask Codex in chat to evaluate a named skill or plugin, explain a score, or list what to fix first (per plugin.json defaultPrompt examples)
  • Run plugin-eval start --request "" --format markdown as the chat-first router that recommends the local command sequence
  • Run analyze for a static local report, explain-budget for token budget explanations, and measurement-plan for guidance on what to measure
  • Run init-benchmark to create starter benchmark configuration, then benchmark to execute a live local codex exec session in an isolated temp workspace
  • Run report and compare to format results and compare before/after benchmark runs
  • Use compatibility aliases plugin-eval guide (for start) and plugin-eval recommend-measures (for measurement-plan)
  • Install the plugin per-user (~/plugins) or per-workspace via a local marketplace.json symlink setup described in the README

Sources

Original description (English)

Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking.

History of plugin-eval

Back to list