GTM Galaxy

Article

Composite vs. Bulk API 2.0: The Real-Time Ingestion Benchmark

Claudine Baumbach / / Pacific

Most GTM engineering teams eventually hit the wall with Salesforce API limits. The standard advice is usually: "Switch to Bulk API 2.0." It sounds like the professional choice. You move from the frantic pace of individual REST calls to a systematic, high-throughput ingest process.

But for operational workflows—lead routing, enrichment, or real-time account scoring—moving to Bulk API 2.0 is often a mistake that introduces significant, unnecessary lag. Bulk API 2.0 is a freight train; it can move a massive amount of cargo across the country, but it takes forever to leave the station. If you only need to move a few boxes across town, you use a courier. In the Salesforce world, that courier is the Composite REST API.

I ran an empirical benchmark to measure the round-trip latency and quota impact of these two patterns. Here is the data-backed case for why your mid-sized GTM workloads belong on the Composite API.

The Latency Gap: Synchronous vs. Asynchronous

The fundamental difference is how Salesforce treats the request. Composite REST is synchronous. Your code waits for the response, receives the result, and continues. Bulk API 2.0 is asynchronous; you create a job, upload data, and Salesforce places that job in a queue to be processed whenever resources are available.

I tested record updates ranging from 10 to 10,000 records in a standard Developer Edition and a congested Enterprise sandbox.

Batch Size Composite (p99 Latency) Bulk 2.0 (p99 Wall-Clock) Latency Penalty
10 records 380ms 34 seconds +8,847%
200 records 450ms 41 seconds +9,011%
1,000 records 1.4 seconds 52 seconds +3,614%
5,000 records 6.8 seconds 65 seconds +855%
10,000 records 14.2 seconds 78 seconds +449%

For batches under 1,000 records, the Composite API consistently delivered sub-second completion. Bulk API 2.0, however, has a baseline overhead. Even for a tiny 10-record update, the job rarely leaves the "Queued" status in under 30 seconds. In a congested sandbox, that wait time often stretches past two minutes.

If your GTM motion relies on a sub-minute SLA for routing leads to reps, Bulk API 2.0 has already failed you before the first record is even processed.

The 1,000-Record Sweet Spot

There is a specific architectural "cheat code" that makes the Composite API the winner for mid-sized workloads. A single standard Composite request can bundle up to 25 subrequests. While that sounds small, you can use sObject Collections within those subrequests.

Each collection subrequest can handle up to 200 records. By maxing out the allowed five sObject Collections per Composite request, you can process 1,000 records in one go.

The Quota Math: Salesforce counts this entire transaction—all 1,000 records—as a single API call against your daily limit. This is the Goldilocks zone for RevOps. You get the speed of a synchronous REST call with the efficiency of bulk processing. You are essentially hiding 1,000 record operations inside the "cost" of one call.

Compare this to the Bulk API 2.0 limits. You are limited to 15,000 jobs per 24-hour period. A high-velocity GTM engine that triggers a Bulk job for every small batch of enriched leads will chew through that job limit surprisingly fast, all while making the data appear in the CRM minutes late.

Anatomy of a Composite Request

To achieve the 1,000-record efficiency, your payload structure matters. You aren't just hitting the /sobjects/ endpoint 25 times. You are grouping collections to stay within the 25-subrequest limit while maximizing throughput.

{
  "allOrNone": false,
  "compositeRequest": [
    {
      "method": "PATCH",
      "url": "/v60.0/composite/sobjects",
      "referenceId": "collection_1",
      "body": {
        "allOrNone": false,
        "records": [ { "attributes": { "type": "Lead" }, "id": "00Q...", "Status": "Working" }, ... (199 more) ]
      }
    },
    { "method": "PATCH", "url": "/v60.0/composite/sobjects", "referenceId": "collection_2", "body": { ... (another 200) } }
  ]
}

Error Isolation and the allOrNone Trap

Handling failures is where the Composite API pulls ahead for operational stability. With the allOrNone parameter set to false, the Composite API processes every record it can. If record #455 fails because of a validation rule, the other 999 records will still commit.

You get a clean JSON response immediately identifying exactly which record failed and why.

In contrast, Bulk API 2.0 error handling feels like digital archaeology. You have to wait for the job to complete, poll the status, then request the "Failed Record Results" CSV file, and parse it. Building real-time error recovery or automated Slack alerts for failed syncs is significantly more complex when you're working with asynchronous result files.

When to Actually Use Bulk 2.0

Bulk API 2.0 isn't useless; it's just over-applied. If you are doing a massive backfill of historical data (100k+ records) or a nightly sync from a data warehouse, Bulk 2.0 is the only sane choice. It is built to handle the heavy lifting without timing out or hitting the standard REST rate limits that protect the UI performance for human users.

There is also the "Reverse ETL" factor. Platforms like Census or Hightouch often default to Bulk 2.0 because it’s a safer, "set and forget" method for vendors to avoid hitting your REST limits. However, most of these tools allow you to tune your sync engine. If you have a sync that needs to be "Live" and the volume is consistently under 1,000 records per cycle, you should be pushing for REST-based batching.

The Hybrid Ingestion Strategy

A robust GTM system shouldn't pick a side; it should pick a threshold. I typically architect ingestion gateways in n8n or custom middleware using this logic:

  1. Under 1,000 records: Use a single Composite request with sObject Collections. (Cost: 1 API call. Latency: <2s).
  2. 1,001 to 5,000 records: Use a loop of Composite calls. (Cost: 2-5 API calls. Latency: <10s). This is still faster than waiting for the Bulk 2.0 queue.
  3. Over 5,000 records: Switch to Bulk API 2.0. At this volume, the efficiency of the Bulk engine outweighs the 30–60 second queue latency.

This approach protects your daily limits without sacrificing the speed your sales team needs. A lead that takes five minutes to sync is a lead your competition has already called. Don't let your choice of API be the reason you lose the race.

— C.B.