Prompts are Code: Building an Eval Harness for GTM AI Pipelines

The standard RevOps workflow for deploying an LLM lead qualification pipeline usually looks like this: spend an hour in the Anthropic Workbench or OpenAI Playground, paste in a few dozen inbound leads, tweak the system prompt until the results look "mostly right," and then copy-paste that string into an n8n node or a Python Lambda function.

This is "vibe-based development." It works until a product update changes your sign-up flow, or a model provider updates their underlying weights. Suddenly, your CRM is filled with leads tagged as "Mid-Market" that should have been "Enterprise."

Because the failure is semantic—the LLM is still returning valid JSON, just wrong data—the routing logic continues to fire. High-value prospects end up in a generic automated sequence instead of on a strategic account executive’s calendar. To build production-grade revenue systems, we have to treat prompts like code. That means implementing a lightweight evaluation (eval) harness with versioned test sets and deterministic assertions.

The Silent Failure of Prompt Drift

In traditional software, if a routing service breaks, it usually throws a 500 error and the pipeline halts. In LLM-powered GTM systems, failures are silent. A prompt change designed to improve "intent detection" might accidentally lower the threshold for "Enterprise" qualification, flooding your sales team with junk.

Even if you don’t change your prompt, model providers frequently update their snapshots. A shift in how a model interprets the word "scalable" in a lead's job description can alter your scoring distribution overnight. Without a regression test, you’re flying blind until a frustrated AE complains about lead quality three weeks later.

Step 1: The Golden Dataset

A "Golden Dataset" is a curated collection of inputs and expected outputs that represent the ground truth for your business. For a RevOps triage pipeline, you don’t need thousands of rows. A high-leverage starting point is 25 to 50 real-world leads covering your typical inbound spectrum.

Your dataset should include:

Clean Positives: Obvious ICP matches.
Clean Negatives: Competitors, students, or "test@test.com" sign-ups.
Hard Cases: Founders of seed-stage startups with high LinkedIn engagement, or consultants from Tier 1 firms using personal emails.
Historical Failures: Every time a lead is misrouted in production, sanitize it and add it to the set. This ensures that specific mistake never happens again.

Structure this as a simple version-controlled JSON file:

[
  {
    "input": "Lead: Jane Doe, Title: Head of DevOps, Company: LargeCorp (10k+ employees), Note: Evaluating SOC2 compliance tools.",
    "expected": {
      "segment": "Enterprise",
      "priority": "High",
      "routing_key": "ae_direct"
    }
  }
]

Step 2: Designing the Eval Harness

You don’t need a $2,000/month enterprise observability platform to start testing. You can build a deterministic testing script using pytest that runs locally or as a GitHub Action.

The goal is to automate three specific checks:

Schema Compliance: Does the output follow the exact JSON structure required by your CRM API?
Classification Accuracy: Did the model correctly identify the segment?
Extraction Precision: Did it correctly pull variables (like headcount or product interest) used in downstream logic?

A Basic Python Assertion

Using a simple testing framework allows you to run your prompts against your golden set in seconds.

import pytest
from my_gtm_logic import qualify_lead

GOLDEN_SET = [
    {"input": "...", "expected_segment": "Enterprise"},
    {"input": "...", "expected_segment": "SMB"}
]

@pytest.mark.parametrize("test_case", GOLDEN_SET)
def test_prompt_regression(test_case):
    # Call your LLM function with the current prompt
    response = qualify_lead(test_case["input"])
    
    # 1. Structural Check: Ensure the automation won't break
    assert "segment" in response, "Response missing required segment key"
    
    # 2. Logic Check: Ensure the classification hasn't drifted
    assert response["segment"] == test_case["expected_segment"], \
        f"Expected {test_case['expected_segment']} but got {response['segment']}"

Step 3: Guaranteeing Structure with Snapshots

To eliminate structural 400 errors when your automation tries to update Salesforce or HubSpot, leverage native Structured Outputs.

Anthropic and OpenAI now allow you to supply a JSON schema that the API enforces during generation. Use this instead of begging the model to "only return JSON" in the system prompt. This guarantees that your routing_key field is always a string and always present.

Crucially, stop using generic model pointers like claude-3-5-sonnet. Pin your code to specific model snapshots (e.g., claude-3-5-sonnet-20241022). Only upgrade the version when you are ready to re-run your entire eval suite and verify the new weights don’t break your lead logic.

Handling Ambiguity and Confidence

A common failure mode in GTM engineering is forcing an LLM to be certain when the data is trash. This leads to hallucinations.

Update your prompt schema to include a confidence_score (0.0 to 1.0) and a reasoning field. During your eval run, flag any lead where the model returns a score below 0.7.

If a prompt tweak increases the number of leads hitting this "low confidence" threshold, your automation efficiency is regressing—even if the accuracy of the high-confidence leads remains high. In production, these low-confidence leads should be routed to a manual RevOps queue rather than being auto-processed.

The Development Cycle

The evaluation step sits between the playground and the deployment.

Prototype: Tweak the prompt in the workbench until it handles a new edge case.
Test: Run your eval script against the golden dataset.
Review: Check for "regression noise." Did fixing the new edge case break three old ones?
Deploy: Once the script shows 100% schema compliance and 0% regression, push the prompt to your production workflow (n8n, Lambda, or your CRM's internal tool).

Is This Overkill?

If you have ten inbound leads a week, yes. You can manually review those in ten minutes.

But once you hit a scale where manual triage is a bottleneck, the cost of a broken routing rule dwarfs the cost of maintaining a test suite. If an enterprise lead sits in an unmonitored "SMB" bucket for three days because of a silent prompt failure, you’ve lost more revenue than the engineering time required to build these assertions.

GTM teams are building complex revenue logic. We cannot afford to manage that logic through trial and error. By implementing a lightweight eval harness, you treat your revenue stack with the same rigor you'd expect from any other engineering team.