GTM Galaxy

Article

The Round-Robin Race Condition: Benchmarking Lead Skew Under Concurrent Inbound Bursts

Claudine Baumbach / / Pacific

Most custom lead routing logic is built on a fundamental misunderstanding of how concurrent webhooks interact with a database. We write code that reads a value, increments a counter in memory, and saves it back. This works perfectly at 2:00 PM on a quiet Tuesday. It collapses during a product launch or a high-traffic webinar when fifty leads arrive in the same second.

When RevOps receives complaints that a specific rep was "skipped" or that one person was slammed with three leads at once, we usually check the logs, see that the rep eventually got a lead, and attribute the complaint to noise. But if your inbound volume spikes, those complaints are often grounded in a hard technical reality: the race condition.

The Anatomy of the Collision

A standard round-robin script follows a simple read-modify-write cycle. It queries the database for the rep who hasn't received a lead in the longest time, assigns the new lead, and updates that rep’s last_assigned_at timestamp.

Imagine two concurrent requests hitting your endpoint. Request A reads the database and sees that Rep 1 is the oldest. Before Request A can commit the update to Rep 1’s timestamp, Request B hits the database. Request B also sees that Rep 1 is the oldest because the first transaction hasn't finished. Both requests assign their leads to Rep 1.

When the updates finally land, the timestamp is overwritten twice for the same person. Rep 2, who should have received the second lead, is left at the front of the queue, effectively skipped for that cycle. In a high-concurrency burst, this happens dozens of times, creating massive skew.

Benchmarking the Skew

To quantify this failure, I ran a synthetic benchmark simulating a marketing spike. I tested three routing architectures against 100 concurrent requests aimed at a pool of ten reps using a custom Node.js harness.

1. The Naive Approach (Lockless)

This uses a standard SELECT followed by an UPDATE. Under 100 concurrent requests, the distribution was a disaster. Rep 1 received 18 assignments, while Rep 9 received 4. Approximately 30% of assignments were duplicates or skips. The system effectively "lost" the state of the rotation because the reads were happening faster than the writes could persist.

2. The Atomic SQL Approach (FOR UPDATE SKIP LOCKED)

I modified the query to use row-level locking. The logic tells the database: "Find the oldest rep, lock that row so no one else can read it, and if someone else already has a lock, move to the next available rep immediately."

UPDATE reps 
SET last_assigned_at = NOW() 
WHERE id = (
  SELECT id 
  FROM reps 
  ORDER BY last_assigned_at ASC 
  LIMIT 1 
  FOR UPDATE SKIP LOCKED
)
RETURNING id, name;

The results were perfect. Each of the ten reps received exactly 10 leads. The SKIP LOCKED clause is the engine here; it prevents the "thundering herd" problem where requests queue up waiting for a single row to unlock. Instead, the database engine intelligently distributes the load across the available rows.

3. The Queued Worker (Redis)

This approach pushes inbound leads into a Redis list (FIFO) and uses a single worker to process them serially. This also resulted in a perfect 10/10 distribution. However, it introduces architectural complexity—you now have to manage a queue, a worker process, and potential dead-lettering for failed assignments.

The Latency Myth

A common objection to database locking is that it "doesn't scale" or slows down the system. My benchmarks suggest otherwise.

  • Naive Approach: 42ms average response time.
  • Atomic SQL Approach: 59ms average response time.

A 17ms penalty is negligible in a GTM context. Salesforce and HubSpot have webhook timeout windows ranging from 10 to 60 seconds. We are operating three orders of magnitude below the threshold where this latency would cause a timeout. The database engine is significantly faster at managing these locks than application-level logic will ever be.

Choosing Your Architecture

When should you use an atomic lock versus a full-blown queue?

Use Atomic SQL Locks when:

  • The assignment logic is "light" (e.g., just updating a timestamp or an ID).
  • You are operating within a single database (Postgres or MySQL).
  • You need to keep the architecture simple and easy to debug.

Use a Queued Worker when:

  • The assignment requires heavy external lifting, such as calling enrichment APIs (Clearbit, 6sense) or checking complex territory maps in Salesforce.
  • Holding a database lock while waiting for a 2-second API response is a recipe for a system-wide bottleneck.
  • You need built-in retries for when the downstream CRM is down.

The Counterargument: Is This Overkill?

If you are a small B2B startup receiving five leads a day, you don't need row-level locking. The probability of two leads arriving in the same millisecond is statistically zero. In that case, the simplest code is the best code.

However, the moment you run a Product Hunt launch or a significant paid campaign, your infrastructure will be tested. If your system isn't built for concurrency, it will fail at the most expensive possible time: when you have the most leads to lose.

Commercial routing tools like LeanData or Chili Piper handle this locking internally. But for custom PLG sign-up flows, specialized inbound routing via iPaaS, or custom-coded handlers, the race condition is a silent killer of sales team morale and lead equity. If you want a fair rotation, stop relying on simple read-write cycles. Make your updates atomic.

— C.B.