Designing a Bulk-Safe Sync Engine at 100K Records/Day

The requirement sounds innocent: "when a value changes on a record, propagate it to related records." The trap is the word related. One change can fan out to a parent, to sibling records in the same group, and to a linked object in a different hierarchy. Do that naively, per-record, and you'll trip Too many SOQL queries or Too many DML statements the first time someone bulk-loads.

A sync engine that scales rests on four ideas: collect before you query, dedupe before you write, guard against recursion, and isolate independent groups.

1. Collect, then query in bulk

Never query inside a loop. Walk the trigger set once, gather every key you'll need, then issue a single SOQL with an IN clause.

// Gather the parent + group keys from the changed records
Set<Id> groupIds = new Set<Id>();
Set<Id> linkedIds = new Set<Id>();
for (Topic__c t : changed) {
    if (t.Group__c != null)  groupIds.add(t.Group__c);
    if (t.Opportunity__c != null) linkedIds.add(t.Opportunity__c);
}

// One query per related object — not one per record
Map<Id, List<Topic__c>> siblingsByGroup = new Map<Id, List<Topic__c>>();
for (Topic__c sib : [
    SELECT Id, Group__c, Name, Status__c
    FROM Topic__c WHERE Group__c IN :groupIds
]) {
    if (!siblingsByGroup.containsKey(sib.Group__c))
        siblingsByGroup.put(sib.Group__c, new List<Topic__c>());
    siblingsByGroup.get(sib.Group__c).add(sib);
}

2. Dedupe before you write

When several changed records point at the same parent or sibling, the cascade will try to update the same record multiple times. Collect proposed changes into a Map<Id, SObject> keyed by record Id — the map naturally collapses duplicates so a record appears exactly once in your final DML.

Map<Id, Topic__c> toUpdate = new Map<Id, Topic__c>();

for (Topic__c t : changed) {
    for (Topic__c sib : siblingsByGroup.get(t.Group__c)) {
        if (sib.Id == t.Id) continue;          // don't sync to self
        if (sib.Status__c == t.Status__c) continue; // no-op, skip
        toUpdate.put(sib.Id, new Topic__c(
            Id = sib.Id, Status__c = t.Status__c
        ));
    }
}

if (!toUpdate.isEmpty()) update toUpdate.values(); // one DML

The map key is the dedup. Two source records targeting the same sibling write one row, not two — and the second simply overwrites the first entry in the map before any DML happens.

3. Guard against recursion

Your update fires the same trigger again. Without a guard, a group of N records can re-enter N times. A static set of already-processed Ids — checked at the top of the handler — short-circuits the re-entry.

private static Set<Id> synced = new Set<Id>();
// ... at handler entry:
changed = filterUnprocessed(changed, synced);
if (changed.isEmpty()) return;
synced.addAll(new Map<Id, Topic__c>(changed).keySet());

Note the guard tracks the records you're about to write, not just the trigger set — otherwise the cascade's own updates slip past it.

4. Isolate independent groups

This is the subtle one. If two unrelated groups happen to be in the same DML batch, a failure in group A shouldn't roll back group B's legitimate sync. Two options:

• Partial DML with Database.update(records, false) and inspect Database.SaveResult[] — lets the good rows commit while you log the bad ones.
• Per-group Savepoints when the groups must each be all-or-nothing internally but independent of each other.

Database.SaveResult[] results =
    Database.update(toUpdate.values(), false); // allOrNone = false
for (Integer i = 0; i < results.size(); i++) {
    if (!results[i].isSuccess()) {
        logError(toUpdate.values()[i].Id, results[i].getErrors());
    }
}

When synchronous isn't enough

If the fan-out is large or touches many objects, the right move is to hand off to async. A Queueable with the collected Id set keeps the synchronous transaction lean and gives you a fresh set of governor limits for the heavy write.

if (linkedIds.size() > THRESHOLD) {
    System.enqueueJob(new TopicCampaignSyncJob(linkedIds));
} else {
    syncInline(linkedIds);
}

The Queueable fallback pattern — inline for small volumes, async above a threshold — is what lets the same code path serve both a single-record UI edit and a 100K-record bulk load.

The checklist

• No SOQL or DML inside a loop — ever.
• Every write goes through a Map<Id, SObject> so records are touched once.
• A recursion guard tracks written Ids, not just trigger Ids.
• Independent groups fail independently (partial DML or savepoints).
• An async escape hatch for high-volume fan-out.

Get these five right and the engine doesn't care whether it's processing one record or a hundred thousand.