Compare prompts fairly
Test whether a prompt change improves a recurring task.
What to have ready
- The actual recurring task, current prompt and proposed alternative.
- Comparable inputs, observable outcomes and a permitted evaluation environment.
A task to start with
Compare a prompt change on a recurring task.
An illustrative request
Use the prompt-evaluation workflow with the baseline, candidate, representative inputs and actual acceptance rule. Keep comparable conditions and separate semantic correctness, format and execution success.
How to check the result
Inspect raw outputs and failed runs; validate scoring with correct, incorrect and malformed controls before claiming improvement.
This preview explains the task and checking approach. It is not a demonstrated customer outcome or a guarantee.
See what a useful orientation looks like.
The free worked example includes source files, an actual entry-point trace and executed checks you can compare with the code.
Open the worked example