Compare two evaluations of the same rows (two versions of a prompt, two models), row by row: one line per metric both have.
compare(await evaluate(category, rows), await evaluate(categoryV2, rows));// [{ metric: "exact_match", before: 0.6, after: 0.8, diff: 0.2, low: -0.05, high: 0.45, better: 2, worse: 0, same: 8, n: 10 }] Copy
compare(await evaluate(category, rows), await evaluate(categoryV2, rows));// [{ metric: "exact_match", before: 0.6, after: 0.8, diff: 0.2, low: -0.05, high: 0.45, better: 2, worse: 0, same: 8, n: 10 }]
Compare two evaluations of the same rows (two versions of a prompt, two models), row by row: one line per metric both have.