# How reliable the labels are

> Two validation studies on how consistent and how accurate the model's labels and predicted scores are, and why enrichment is reliable.

Before you build an analysis on Dimensions, it's fair to ask two things. If the model labels the same text again, does it give the same answer? And does its answer match what the customer actually said or felt? Dimension Labs ran a study on each.

All figures on this page come from the Agent Validation study (Dimension Labs docs, May 2026) unless a line says otherwise. The method is published in our [research](https://www.dimensionlabs.io/research).

## Repeatable: the same answer on every run

The first study asked whether the model gives the same predicted score when it reads the same text again, across thousands of conversations.

### The study

- **Data:** 9,977 fan surveys from five MLB teams.
- **Task:** the model read each open-text comment and predicted the 0 to 10 rating the fan gave on the same survey.
- **Runs:** the same prompt, three independent times, against the same text.
- **Prompt:** deliberately plain. No examples, no calibration, no domain-specific instructions.
- **Model:** GPT-4.1.

### Results

| Measure | Definition | Model |
|---|---|---|
| **Consensus** | All three runs gave the same score | 80.9% |
| **Pairwise exact** | Two runs gave the same score | 87% |
| **Pairwise tolerance** | Two runs within 1 point of each other | 99.9% |

On a 0 to 10 scale, two runs landed within one point of each other almost every time, and all three runs agreed exactly on four conversations in five.

## Recoverable: a score for every conversation

Customers rarely leave a rating. A predicted score fills the gap: the model reads the conversation and estimates the rating the customer would have given. This study checked those estimates against real ratings.

### The study

- **Data:** support conversations from Givebutter between November 2025 and April 2026. Of about 70,260 conversations in that window, 3,736 carried a usable customer satisfaction rating.
- **Task:** the model read each of the 3,736 rated conversations and predicted the customer's 1 to 5 rating, without seeing the rating.
- **Baseline:** tone-based sentiment on the same conversations.
- **Model:** GPT-4.1.

3,736 of 70,260 is 5.3%, fewer than 1 conversation in 18 (calculated from the study's counts). The other 94.7% have no rating at all. A predicted score is generated for every conversation, so it recovers a satisfaction signal for the conversations that would otherwise have none.

### Results

| Measure | Definition | Model | Sentiment baseline |
|---|---|---|---|
| **Correlation** | Agreement with the customer's own 1 to 5 rating (Pearson r) | 0.47 | 0.36 |
| **Low-rating detection** | Accuracy separating low ratings (1 to 3) from high (4 to 5) | 91% | 74% |
| **Exact match** | Predicted score equals the customer's rating | 66% | not measured |
| **Within 1 point** | Predicted score within one point of the customer's rating | 93% | not measured |

The predicted score tracked customers' own ratings more closely than sentiment did, and separated satisfied conversations from dissatisfied ones more reliably.

:::note[Read these numbers with the skew in mind]
Ratings in this data skew strongly positive: 91% are 4 or 5. A model that leans toward high scores gets some exact matches and within-1 matches for free, so those two figures partly reflect the skew. The clearer evidence is correlation and low-rating detection, where the base rate gives no free lift.
:::

## Why labeling with a language model is reliable

Enrichment is a **recognition** task, not a **generation** task, and language models are much more reliable at recognition.

| Task type | What it does | Risk of making things up |
|---|---|---|
| **Generation** | Writes something new: an essay, a summary, code | Higher. The model produces content that didn't exist before. |
| **Recognition** | Finds whether a concept is present in existing text, and labels it | Lower. The model matches against text that is already there, though it can still return a value outside your list. See [Values you didn't define](/troubleshooting/enrichment#values-you-didnt-define). |

Used for enrichment, a model gives you:

- **Consistency.** One labeler for every conversation, with no drift as volume grows.
- **Context.** It reads meaning and nuance, not just keywords.
- **Reliability.** Labels are tied to text that exists, and a verbatim Dimension can quote the exact words behind each label.

## What this means for your own data

These studies used plain prompts. Your own prompts can do better, and you should still check them:

1. **Test a prompt on a sample** before running it on everything. See [Test before you run](/dimensions/test-before-you-run).
2. **Pair judgements with quotes**, so any label can be checked against the customer's words. See [Design custom Dimensions](/dimensions/custom).
3. **Leave a score empty when there's no signal**, and add a confidence Dimension when conversations vary a lot. See [Predicted scores](/dimensions/predicted-scores).
4. **Keep collecting surveys.** Comparing predicted and real ratings on the conversations that have both is your own version of the recoverable study. Ask the agent: *"For conversations that have a CSAT rating, how closely does the predicted CSAT match? Show exact match, within 1 point, and correlation."*

:::note
The studies used GPT-4.1. The model behind enrichment and the agent can change over time, so treat these figures as evidence for the method, and run the check in step 4 on your own data.
:::