AI with MattLearn as you work.Member area
← All skill previewsPlanning & learning

Compare prompts fairly

Test whether a prompt change improves a recurring task.

What to have ready

  • The actual recurring task, current prompt and proposed alternative.
  • Comparable inputs, observable outcomes and a permitted evaluation environment.

A task to start with

Compare a prompt change on a recurring task.

An illustrative request

Use the prompt-evaluation workflow with the baseline, candidate, representative inputs and actual acceptance rule. Keep comparable conditions and separate semantic correctness, format and execution success.

How to check the result

Inspect raw outputs and failed runs; validate scoring with correct, incorrect and malformed controls before claiming improvement.

This preview explains the task and checking approach. It is not a demonstrated customer outcome or a guarantee.

See what a useful orientation looks like.

The free worked example includes source files, an actual entry-point trace and executed checks you can compare with the code.

Open the worked example