GTM Galaxy

Article

The Job Title Normalization Waterfall: Balancing Speed and Accuracy in GTM Systems

Claudine Baumbach / / Pacific

Most GTM systems are built on a foundation of messy, self-reported text. One lead identifies as "VP of Demand Gen," the next as "Head of Global Growth Marketing," and the third as "Digital Marketing Guru."

If your lead routing, scoring, or automated sequencing depends on knowing exactly who someone is, you face a trade-off. You can use brittle regex rules that miss 30% of your leads, or you can send every title to an LLM, adding a full second of latency to your routing engine and burning tokens on simple strings.

I ran a benchmark across 1,000 messy B2B job titles to test three approaches: deterministic regex, vector embeddings, and small LLMs. The data shows that while small LLMs are the most accurate, they are too slow for synchronous GTM workflows. The answer is a tiered normalization waterfall.

The Experiment: 1,000 Titles, Three Models

I used a dataset of 1,000 job titles exported from a HubSpot instance. The goal was to map these titles to a standard two-dimensional grid: Seniority (CXO, VP, Director, Manager, IC) and Function (Engineering, Sales, Marketing, Ops, Finance).

Tier 1: The Deterministic Baseline (Regex & Keywords)

Most RevOps teams start and end here. This involves a long list of CONTAINS or MATCHES logic in a tool like LeanData or a Python script.

  • The Setup: A library of 150 keywords and 40 regex patterns mapped to standard personas.
  • Latency: <1ms. It is effectively instantaneous.
  • The Results: Accuracy sat at roughly 62%.
  • The Failure Mode: Regex is allergic to context. It struggles with compound titles. If a title is "Director of Sales Operations," a simple regex might bucket it into "Sales" or "Operations" depending on the order of operations, missing the nuance. It also fails on creative titles like "Marketing Evangelist" unless you manually add every variation to your lookup table.

Tier 2: Vector Embeddings (Semantic Similarity)

Vector embeddings represent text as a list of numbers (vectors) in a multi-dimensional space. I used the all-MiniLM-L6-v2 model, which is small (90MB) and fast enough to run on a cheap Lambda function.

  • The Setup: I created "anchor vectors" for 50 standard titles. For every incoming title, the system calculated the cosine similarity between the lead's title and the anchor list.
  • Latency: 5ms to 12ms.
  • The Results: Accuracy improved to 78%.
  • The Failure Mode: Embeddings are great at function but often ignore hierarchy. In vector space, "VP of Engineering" and "Engineering Intern" are mathematically close because they share the "Engineering" neighborhood. The model often ignores the weight of seniority modifiers. Using standard cosine distance, an "Assistant to the CEO" often maps to "Chief Executive Officer" because the semantic overlap is too high for a 384-dimensional model to distinguish the power dynamic.

Tier 3: Small LLMs (GPT-4o-mini)

Small, high-performance LLMs are the "heavy lifters." They don't just look at word proximity; they understand the logic of a job title.

  • The Setup: A structured output prompt asking for a JSON response with the classified seniority and function.
  • Latency: 600ms to 1,100ms.
  • The Results: Accuracy hit 96%.
  • The Failure Mode: The failure isn't the data—it's the clock. If you are running an inbound routing rule where a lead needs to be assigned to an AE within seconds of a form fill, adding a one-second delay for every enrichment step is a massive tax. While $0.15 per 1M tokens is cheap, the synchronous overhead remains an architectural bottleneck.

The Case for the Normalization Waterfall

If we rely only on Regex, our data is dirty. If we rely only on LLMs, our systems are slow. A waterfall architecture delivers high precision with sub-50ms median latency by escalating complexity only when needed.

1. The Exact Match Cache

Before doing any math, check a hash map of previously normalized titles. If "VP Growth" was already mapped to "VP, Marketing" yesterday, just use that. This handles 40% of recurring traffic with sub-1ms latency.

2. The Regex Guardrails

Run high-confidence regex rules to identify "Known Low Value" titles (e.g., "Student," "Intern," "N/A") and clear-cut standard titles. If these match, exit the pipeline early.

3. Vector Similarity with a Threshold

Use all-MiniLM-L6-v2 to compare the title against your anchor list.

  • If Similarity > 0.92: Accept the match. This handles standard variations (e.g., "VP of Sales" vs "Sales VP").
  • If Similarity is 0.70 - 0.91: This is the "Ambiguous Zone." Pass it to the LLM.

4. The LLM Fallback

Only use the LLM for titles that are truly non-standard. In my test, this was only 15% of the dataset. This keeps the median latency low while ensuring the "Chief Happiness Officer" still gets routed to the right team.

Implementation: A Practical Prototype

You don't need a vector database like Pinecone for this. A simple NumPy array of your anchor vectors is enough to run locally.

import numpy as np
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer('all-MiniLM-L6-v2')
standard_titles = ["VP Sales", "Software Engineer", "Marketing Manager"]
anchor_embeddings = model.encode(standard_titles)

def normalize_title(input_title):
    # 1. (Omitted) Check Hash Map Cache
    # 2. (Omitted) Check Regex Guardrails
    
    # 3. Vector Tier
    input_embedding = model.encode(input_title)
    hits = util.semantic_search(input_embedding, anchor_embeddings, top_k=1)
    score = hits[0][0]['score']
    
    if score > 0.92:
        return standard_titles[hits[0][0]['corpus_id']]
    
    # 4. LLM Fallback for ambiguous titles
    return call_llm_for_structured_classification(input_title)

By implementing this, I reduced the median latency of the normalization service from 850ms (pure LLM) to 42ms (waterfall), while maintaining 95%+ accuracy.

Addressing the Trade-offs

"Why not just use ZoomInfo or Clearbit?" Third-party enrichment is excellent, but it often fails on the "long tail" of smaller companies where their database is thin, and on "form fill" edge cases where a lead uses a new title. You need a normalization layer to catch the 20% of leads that enrichment tools return as null or Other.

"Is a waterfall too complex to maintain?" For a startup with 10 leads a day, yes. Just call the LLM and move on. But once you hit the scale where you are routing thousands of leads a week across global territories, speed to lead matters. Every second added to routing is a second added to your first automated touchpoint, which correlates directly with lower conversion rates.

Monitoring Drift

The final piece is a human-in-the-loop audit. Every week, sample 50 titles where the waterfall chose the LLM path. If the LLM is consistently correcting a specific title type, move that logic up to the Regex or Vector anchor list.

GTM engineering isn't about choosing the smartest model; it's about building a system that is smart enough to know when it needs to be fast and when it needs to be right.

— C.B.