plugin-eval Codex Plugin
Codex PluginCodexDeveloper ToolsClaude Code Customization & WorkflowCode Quality, Review & TestingThe explanation below is AI-generated. Please verify it against the sources.
plugin-eval is a Codex plugin paired with a Node.js CLI that lets engineers evaluate a local skill or plugin directly from chat. According to the README, it recognizes plain-language requests such as asking for an evaluation, asking why a score came out a certain way, or asking what to fix first, and routes them to local commands like analyze, explain-budget, measurement-plan, init-benchmark, and benchmark. The analyze, explain-budget, and measurement-plan commands run as deterministic local checks, while the benchmark command executes real codex exec sessions inside an isolated temporary workspace and saves the resulting artifacts locally. It requires Node.js 20 or newer and can be set up either as a globally linked CLI command or as a Codex plugin registered through a local marketplace.json file. The overall purpose is to help developers judge the quality of other plugins and skills, prioritize fixes, and understand or measure their token usage.
What you can do with plugin-eval
- Ask in chat to evaluate a local skill or plugin and receive an analysis report
- Ask why a plugin or skill received a particular score
- Get a ranked list of what to fix first
- Request an explanation of token budget before running any live measurement
- Run a guided benchmark that measures real token usage through local codex exec sessions
- Compare benchmark results from before and after a change