LLM applications emit spans following OpenInference semantic conventions. Prompts are stored in different span attributes depending on the span kind and instrumentation:
Works with
AI-first code editor with Composer
Before installing skills in Cursor, ensure your development environment meets these requirements:
node --versionarize-prompt-optimizationExecute the skills CLI command in your project's root directory to begin installation:
Fetches arize-prompt-optimization from arize-ai/arize-skills and configures it for Cursor.
The CLI shows a list of agents. Use arrow keys and space to select Cursor:
Confirm successful installation by checking the skill directory location:
Restart Cursor to activate arize-prompt-optimization. Access via /arize-prompt-optimization in your agent's command palette.
We perform automated surface-level scans (Gen AI Scanner, Socket, Snyk) during installation. These checks detect common vulnerabilities but do not guarantee complete security. Always review skill source code and verify the publisher's reputation before production use.
Skills execute code in your environment. Always review source, verify the publisher, and test in isolation before production.
Submit your Claude Code skill and start earning
Create detailed user stories, acceptance criteria, and feature specs
Example
Generate user stories for 'password reset feature' with acceptance criteria, edge cases, and test scenarios
Reduce spec writing time by 50%, ensure comprehensive coverage
Research competitors, compare features, identify gaps
Example
Analyze 5 competitor products, create feature comparison matrix, suggest differentiation opportunities
Complete competitive research in 2 hours instead of 2 days
Evaluate features using frameworks (RICE, ICE, Kano) and create prioritized backlogs
Example
Score 20 feature ideas using RICE framework, generate prioritized roadmap with rationale
0
total installs
0
this week
8
GitHub stars
0
upvotes
Run in your terminal
0
installs
0
this week
8
stars
LLM applications emit spans following OpenInference semantic conventions. Prompts are stored in different span attributes depending on the span kind and instrumentation:
| Column | What it contains | When to use |
|---|---|---|
attributes.llm.input_messages |
Structured chat messages (system, user, assistant, tool) in role-based format | Primary source for chat-based LLM prompts |
attributes.llm.input_messages.roles |
Array of roles: system, user, assistant, tool |
Extract individual message roles |
attributes.llm.input_messages.contents |
Array of message content strings | Extract message text |
attributes.input.value |
Serialized prompt or user question (generic, all span kinds) | Fallback when structured messages are not available |
attributes.llm.prompt_template.template |
Template with {variable} placeholders (e.g., "Answer {question} using {context}") |
When the app uses prompt templates |
attributes.llm.prompt_template.variables |
Template variable values (JSON object) | See what values were substituted into the template |
attributes.output.value |
Model response text | See what the LLM produced |
attributes.llm.output_messages |
Structured model output (including tool calls) | Inspect tool-calling responses |
attributes.openinference.span.kind = 'LLM'): Check attributes.llm.input_messages for structured chat messages, OR attributes.input.value for a serialized prompt. Check attributes.llm.prompt_template.template for the template.attributes.input.value contains the user's question. The actual LLM prompt lives on child LLM spans -- navigate down the trace tree.attributes.input.value has tool input, attributes.output.value has tool result. Not typically where prompts live.These columns carry the feedback data used for optimization:
| Column pattern | Source | What it tells you |
|---|---|---|
annotation.<name>.label |
Human reviewers | Categorical grade (e.g., correct, incorrect, partial) |
annotation.<name>.score |
Human reviewers | Numeric quality score (e.g., 0.0 - 1.0) |
annotation.<name>.text |
Human reviewers | Freeform explanation of the grade |
eval.<name>.label |
LLM-as-judge evals | Automated categorical assessment |
eval.<name>.score |
LLM-as-judge evals | Automated numeric score |
eval.<name>.explanation |
LLM-as-judge evals | Why the eval gave that score -- most valuable for optimization |
attributes.input.value |
Trace data | What went into the LLM |
attributes.output.value |
Trace data | What the LLM produced |
{experiment_name}.output |
Experiment runs | Output from a specific experiment |
Proceed directly with the task β run the ax command you need. Do NOT check versions, env vars, or profiles upfront.
If an ax command fails, troubleshoot based on the error:
command not found or version error β see references/ax-setup.md401 Unauthorized / missing API key β run ax profiles show to inspect the current profile. If the profile is missing or the API key is wrong: check .env for ARIZE_API_KEY and use it to create/update the profile via references/ax-profiles.md. If .env has no key either, ask the user for their Arize API key (https://app.arize.com/admin > API Keys).env for ARIZE_SPACE_ID, or run ax spaces list -o json, or ask the user.env for ARIZE_DEFAULT_PROJECT, or ask, or run ax projects list -o json --limit 100 and present as selectable options.env, load if present, otherwise ask the user# List LLM spans (where prompts live)
ax spans list PROJECT_ID --filter "attributes.openinference.span.kind = 'LLM'" --limit 10
# Filter by model
ax spans list PROJECT_ID --filter "attributes.llm.model_name = 'gpt-4o'" --limit 10
# Filter by span name (e.g., a specific LLM call)
ax spans list PROJECT_ID --filter "name = 'ChatCompletion'" --limit 10
# Export all spans in a trace
ax spans export --trace-id TRACE_ID --project PROJECT_ID
# Export a single span
ax spans export --span-id SPAN_ID --project PROJECT_ID
# Extract structured chat messages (system + user + assistant)
jq '.[0] | {
messages: .attributes.llm.input_messages,
model: .attributes.llm.model_name
}' trace_*/spans.json
# Extract the system prompt specifically
jq '[.[] | select(.attributes.llm.input_messages.roles[]? == "system")] | .[0].attributes.llm.input_messages' trace_*/spans.json
# Extract prompt template and variables
jq '.[0].attributes.llm.prompt_template' trace_*/spans.json
# Extract from input.value (fallback for non-structured prompts)
jq '.[0].attributes.input.value' trace_*/spans.json
Once you have the span data, reconstruct the prompt as a messages array:
[
{"role": "system", "content": "You are a helpful assistant that..."},
{"role": "user", "content": "Given {input}, answer the question: {question}"}
]
If the span has attributes.llm.prompt_template.template, the prompt uses variables. Preserve these placeholders ({variable} or {{variable}}) -- they are substituted at runtime.
# Find error spans -- these indicate prompt failures
ax spans list PROJECT_ID \
--filter "status_code = 'ERROR' AND attributes.openinference.span.kind = 'LLM'" \
--limit 20
# Find spans with low eval scores
ax spans list PROJECT_ID \
--filter "annotation.correctness.label = 'incorrect'" \
--limit 20
# Find spans with high latency (may indicate overly complex prompts)
ax spans list PROJECT_ID \
--filter "attributes.openinference.span.kind = 'LLM' AND latency_ms > 10000" \
--limit 20
# Export error traces for detailed inspection
ax spans export --trace-id TRACE_ID --project PROJECT_ID
# Export a dataset (ground truth examples)
ax datasets export DATASET_ID
# -> dataset_*/examples.json
# Export experiment results (what the LLM produced)
ax experiments export EXPERIMENT_ID
# -> experiment_*/runs.json
Join the two files by example_id to see inputs alongside outputs and evaluations:
# Count examples and runs
jq 'length' dataset_*/examples.json
jq 'length' experiment_*/runs.json
# View a single joined record
jq -s '
.[0] as $dataset |
.[1][0] as $run |
($dataset[] | select(.id == $run.example_id)) as $example |
{
input: $example,
output: $run.output,
evaluations: $run.evaluations
}
' dataset_*/examples.json experiment_*/runs.json
# Find failed examples (where eval score < threshold)
jq '[.[] | select(.evaluations.correctness.score < 0.5)]' experiment_*/runs.json
Look for patterns across failures:
eval.*.explanation tells you WHY something failedUse this template to generate an improved version of the prompt. Fill in the three placeholders and send it to your LLM (GPT-4o, Claude, etc.):
You are an expert in prompt optimization. Given the original baseline prompt
and the associated performance data (inputs, outputs, evaluation labels, and
explanations), generate a revised version that improves results.
ORIGINAL BASELINE PROMPT
========================
{PASTE_ORIGINAL_PROMPT_HERE}
========================
PERFORMANCE DATA
================
The following records show how the current prompt performed. Each record
includes the input, the LLM output, and evaluation feedback:
{PASTE_RECORDS_HERE}
================
HOW TO USE THIS DATA
1. Compare outputs: Look at what the LLM generated vs what was expected
2. Review eval scores: Check which examples scored poorly and why
3. Examine annotations: Human feedback shows what worked and what didn't
4. Identify patterns: Look for common issues across multiple examples
5. Focus on failures: The rows where the output DIFFERS from the expected
value are the ones that need fixing
ALIGNMENT STRATEGY
- If outputs have extra text or reasoning not present in the ground truth,
remove instructions that encourage explanation or verbose reasoning
- If outputs are missing information, add instructions to include it
- If outputs are in the wrong format, add explicit format instructions
- Focus on the rows where the output differs from the target -- these are
the failures to fix
RULES
Maintain Structure:
- Use the same template variables as the current prompt ({var} or {{var}})
- Don't change sections that are already working
- Preserve the exact return format instructions from the original prompt
Avoid Overfitting:
- DO NOT copy examples verbatim into the prompt
- DO NOT quote specific test data outputs exactly
- INSTEAD: Extract the ESSENCE of what makes good vs bad outputs
- INSTEAD: Add general guidelines and principles
- INSTEAD: If adding few-shot examples, create SYNTHETIC examples that
demonstrate the principle, not real data from above
Goal: Create a prompt that generalizes well to new inputs, not one that
memorizes the test data.
OUTPUT FORMAT
Return the revised prompt as a JSON array of messages:
[
{"role": "system", "content": "..."},
{"role": "user", "content": "..."}
]
Also provide a brief reasoning section (bulleted list) explaining:
- What problems you found
- How the revised prompt addresses each one
Format the records as a JSON array before pasting into the template:
# From dataset + experiment: join and select relevant columns
jq -s '
.[0] as $ds |
[.[1][] | . as $run |
($ds[] | select(.id == $run.example_id)) as $ex |
{
input: $ex.input,
expected: $ex.expected_output,
actual_output: $run.output,
eval_score: $run.evaluations.correctness.score,
eval_label: $run.evaluations.correctness.label,
eval_explanation: $run.evaluations.correctness.explanation
}
]
' dataset_*/examples.json experiment_*/runs.json
# From exported spans: extract input/output pairs with annotations
jq '[.[] | select(.attributes.openinference.span.kind == "LLM") | {
input: .attributes.input.value,
output: .attributes.output.value,
status: .status_code,
model: .attributes.llm.model_name
}]' trace_*/spans.json
After the LLM returns the revised messages array:
1. Extract prompt -> Phase 1 (once)
2. Run experiment -> ax experiments create ...
3. Export results -> ax experiments export EXPERIMENT_ID
4. Analyze failures -> jq tMake data-driven prioritization decisions faster
Draft PRDs, status updates, and stakeholder presentations
Example
Create executive summary of Q3 roadmap, monthly progress report, feature launch announcement
Save 3-5 hours/week on communication overhead
Prerequisites
Time Estimate
30-60 minutes to see productivity improvements
Steps
Common Pitfalls
β Do
β Don't
π‘ Pro Tips
β Use when
Use for user story writing, competitive research, roadmap prioritization, stakeholder communication, and PRD drafting. Best for reducing repetitive documentation and research work.
β Avoid when
Avoid for strategic product vision (requires deep customer empathy), pricing decisions (needs market and financial expertise), or when face-to-face customer discovery is more valuable than speed.
mattpocock/skills
parcadei/continuous-claude-v3
cursor/plugins
pproenca/dot-skills
ailabs-393/ai-labs-claude-skills
mattpocock/skills
Registry listing for arize-prompt-optimization matched our evaluation β installs cleanly and behaves as described in the markdown.
arize-prompt-optimization reduced setup friction for our internal harness; good balance of opinion and flexibility.
arize-prompt-optimization has been reliable in day-to-day use. Documentation quality is above average for community skills.
Registry listing for arize-prompt-optimization matched our evaluation β installs cleanly and behaves as described in the markdown.
Solid pick for teams standardizing on skills: arize-prompt-optimization is focused, and the summary matches what you get after install.
arize-prompt-optimization reduced setup friction for our internal harness; good balance of opinion and flexibility.
Useful defaults in arize-prompt-optimization β fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
I recommend arize-prompt-optimization for anyone iterating fast on agent tooling; clear intent and a small, reviewable surface area.
arize-prompt-optimization reduced setup friction for our internal harness; good balance of opinion and flexibility.
Keeps context tight: arize-prompt-optimization is the kind of skill you can hand to a new teammate without a long onboarding doc.
showing 1-10 of 67