GTM Galaxy

Article

The Idempotent Webhook: Building a Reliable Lead Intake Layer in n8n

Claudine Baumbach / / Pacific

Most GTM teams treat duplicate records as a data hygiene problem to be solved with a cleanup tool or a manual merge task. In reality, duplicates are usually the result of a predictable architectural failure: connecting a high-volume lead source directly to a CRM via a basic webhook or native integration.

When a form submission or a payment event fires, it doesn't just travel from Point A to Point B. It travels across networks, through API rate limits, and against database locks. When any part of that chain slows down, the sender—whether it’s Typeform, Stripe, or a custom app—will retry the delivery. Without a "shock absorber" in the middle, those retries become duplicates.

To build a resilient revenue system, you need to implement a standard software engineering pattern: idempotency.

The Mechanism of Duplicate Creation

Modern SaaS tools follow a "retry on failure" policy for webhooks. If your receiver (or the CRM it’s talking to) doesn't respond with a 200 OK within a strict window—usually 10 to 30 seconds—the sender assumes the message was lost and tries again.

In a typical GTM stack, duplicates occur in three specific scenarios:

  1. The Latency Loop: Your CRM takes 35 seconds to process a complex lead routing script, but the form provider times out at 30 seconds. The provider sends a second request while the first is still being processed.
  2. The Rate Limit Rejection: You hit your HubSpot API burst limit (e.g., 100 requests per 10 seconds). The first request fails; the integration layer retries, but if the first request eventually succeeded in the background, you now have two records.
  3. The Overlapping Retry: A temporary network blip causes a failure. The sender triggers its retry logic immediately, but the first request eventually reaches the destination. Both are processed.

What is Idempotency?

An idempotent operation is one that can be performed multiple times without changing the result beyond the initial application. In GTM engineering, this means if your system receives the same unique event_id five times, it should create exactly one record in your CRM and safely ignore the other four.

By building an idempotent receiver, you decouple the receipt of data from the processing of data.

The Build: n8n + PostgreSQL

We will build a receiver that logs the unique ID of every incoming webhook to a database. Before sending data to the CRM, the system checks this log. If the ID exists, the workflow stops.

1. The Persistence Layer

A lightweight PostgreSQL database is the best choice here because it handles concurrent requests far better than a spreadsheet. Use a simple table to track unique event IDs.

CREATE TABLE webhook_logs (
    id SERIAL PRIMARY KEY,
    provider_event_id VARCHAR(255) UNIQUE NOT NULL,
    processed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    payload JSONB
);

The UNIQUE constraint on provider_event_id is critical. It ensures that even if two identical requests hit your database at the exact same millisecond, the database engine itself will reject the second one.

2. The n8n Workflow Logic

Your n8n workflow should follow this flow to ensure reliability:

Node 1: Webhook Trigger Set the HTTP Method to POST. Identify the unique identifier provided by the source. For Typeform, this is event_id. For Stripe, it’s id (prefixed with evt_).

Node 2: The Idempotency Check (Postgres Node) Instead of a separate "Select" then "Insert" (which creates a race condition), use an INSERT ... ON CONFLICT statement. This is an atomic operation.

INSERT INTO webhook_logs (provider_event_id, payload) 
VALUES ('{{ $json.event_id }}', '{{ JSON.stringify($json) }}')
ON CONFLICT (provider_event_id) DO NOTHING
RETURNING id;

Node 3: The Filter (If Node) Check the output of the Postgres node:

  • If id is present: This is a new, unique event. Proceed to your HubSpot/Salesforce nodes.
  • If id is null: This is a duplicate. The database rejected the write. Stop the workflow here.

Architectural Trade-offs

Native integrations—like the direct HubSpot-to-Typeform sync—are adequate for low-volume teams. If you process five leads a week, building this is overkill. You are introducing a new point of failure: if your n8n instance or Postgres database goes down, lead intake stops.

However, for teams running high-spend performance marketing, the cost of a broken lead flow or a messy CRM outweighs the maintenance of this small architectural layer.

To manage the database size, add a simple cleanup workflow in n8n that runs once a week to purge old logs:

DELETE FROM webhook_logs WHERE processed_at < NOW() - INTERVAL '30 days';

Moving Beyond Point-to-Point

Revenue operations is often treated as a series of toggles inside SaaS platforms. But as systems scale, "hoping for the best" with native syncs isn't a strategy.

Building an idempotent receiver gives you a clean audit log of every raw payload the vendor sent you. If a CRM sync fails due to a mapping error, you don't have to ask the prospect to fill out the form again. You have the raw data in your webhook_logs, ready to be replayed.

The CRM is a tool for managing relationships, but it’s a poor system for technical event logging. By moving the intake logic to a mediated architecture, you ensure your sales team spends their time on calls, not on manual data deduplication.

— C.B.