Many businesses start integration the same way.
First, connect the marketplace to the ERP. Then connect the ERP to the OMS. Then connect the OMS to the e-Invoice system. Then add one more connector for payments, one more for refunds, and one more because someone said it would be “quick.”
At small volume, this can look fine. Orders move. Data appears. Dashboards glow politely.
Then scale arrives and the cracks start to show.
A connector says the order was sent, but the invoice was not created. A retry creates a duplicate. A refund reaches the marketplace but not finance. A fee is posted twice. A document gets rejected, but the order status still says “done.” Suddenly, the business has automation, but not trust.
That is where the idea of a Transaction Integrity Layer becomes useful.
To be clear, “transaction integrity layer” does not appear to be a widely established formal industry standard in the way terms like API gateway or event bus are. It is better understood as an architecture concept: a dedicated layer that protects the correctness, traceability, and replay safety of business transactions as they move across systems. (ResearchGate)
In simple English, it is the part of your architecture that answers this question:
“Did the business event happen correctly, exactly as intended, across every system that matters?”
That is a much harder question than “Did the connector run?”
A connector moves data. A transaction integrity layer protects meaning.
This difference is the whole story.
A connector is usually built to pass data from System A to System B. That is useful, but narrow. It focuses on transport.
A transaction integrity layer focuses on business truth. It makes sure one marketplace order becomes one controlled commercial event with the right status, references, retries, adjustments, and audit trail across systems such as OMS, ERP, payments, settlement, and Malaysia’s MyInvois submission flow. MyInvois itself is built around structured submission, retrieval, validation, cancellation, rejection, and document search APIs, which shows that compliance depends on state control, not just file transfer. (MyInvois SDK)
That is why many connector-led designs fail at scale. They are good at moving messages, but weak at protecting the business meaning of those messages.
It is a bit like hiring a very fast courier who never checks whether the parcel contains the correct item, reached the correct person, or got delivered twice. Fast, yes. Reliable, not necessarily.
Why connectors look good early on
Connectors are popular for good reasons.
They are fast to deploy. Vendors can demo them nicely. Project teams can say “integration complete” earlier. For low-volume operations, manual cleanup can hide design weaknesses.
At this stage, the architecture often feels successful because the number of edge cases is still small. Few retries. Few timeouts. Few duplicate events. Few returns. Few disputes. Few month-end surprises.
But distributed systems do not fail only when software is “bad.” They fail because networks time out, acknowledgements get lost, remote services slow down, retries happen, consumers reprocess messages, and systems recover in awkward sequences. AWS explicitly notes that retries are common and useful, but they can also create unwanted side effects unless operations are made safe through idempotency. Thoughtworks and other distributed-systems sources make the same broader point: once you distribute work across services, you inherit failure modes such as asynchrony, node failure, network issues, and consistency trade-offs. (Amazon Web Services, Inc.)
That is why connector success in a demo environment often tells you very little about connector success during festive sales, refund spikes, partial outages, or month-end close.
Why connectors fail at scale
1. They do not own idempotency
At scale, retries are normal. If one side is unsure whether a request succeeded, it retries. If a message broker uses at-least-once delivery, consumers may receive the same message more than once. Microservices.io is very clear on this point: with at-least-once delivery, a consumer can be invoked repeatedly for the same message, so the consumer must be idempotent. Confluent likewise notes that Kafka’s default delivery guarantee is at-least-once unless stronger semantics are configured. (microservices.io)
A plain connector often assumes “receive once, process once.” Real systems laugh at that assumption.
A transaction integrity layer treats duplicates as expected. It uses transaction keys, idempotency tokens, deduplication logic, and deterministic replay rules so the same marketplace order or invoice request does not create duplicate commercial effects. AWS recommends idempotency tokens precisely because they let clients retry without accidentally creating extra records or extra side effects. (Amazon Web Services, Inc.)
2. They track payload delivery, not business state
A connector may tell you, “Message delivered to endpoint.” Helpful, but incomplete.
Finance needs answers like these:
- Was the order accepted?
- Was the invoice validated?
- Was the fee posted?
- Was the refund linked to the original sale?
- Was the adjustment sent within the allowed time window?
- Did the final status reach every dependent system?
MyInvois reflects this state-driven reality. It is not only a submission endpoint. It supports getting submissions, retrieving document source, retrieving document details including validation results, cancelling documents, rejecting documents, and searching document history. (MyInvois SDK)
That is exactly the kind of lifecycle a transaction integrity layer should manage. A connector alone usually does not.
3. They break under partial failure
The ugliest failures are partial ones.
Imagine this common sequence: the marketplace order is imported, stock is reserved, payment is confirmed, but the ERP posting times out. The connector retries, and the ERP receives the request twice. Later, the e-Invoice submission runs once, but the settlement record reflects two postings. Now operations think the order is fine, while finance thinks the numbers are haunted.
This is why retry logic without integrity controls is dangerous. AWS notes that retries simplify client behavior, but without safe idempotent handling they can create undesirable duplicate effects. Circuit breakers, backoff, and jitter help reduce cascading failures, but they do not by themselves solve transaction correctness. (Amazon Web Services, Inc.)
A transaction integrity layer is built for partial failure. It records transaction intent, current state, prior attempts, and recovery path. It knows whether to retry, ignore, compensate, or escalate.
4. They cannot explain the truth at month-end
This is where finance teams stop smiling.
At low volume, people can reconcile by hand. At scale, they need a transaction trail that is complete, searchable, and replayable. Malaysia’s MyInvois environment itself assumes this need by exposing document retrieval, recent-document retrieval, search, validation result lookup, and taxpayer TIN validation functions. The official SDK also documents extensive validation layers, including structure validation, core field validation, taxpayer validation, referenced-document validation, code validation, duplicate-document validation, and currency validation. (MyInvois SDK)
A connector typically says, “I sent the payload.” A finance team needs to say, “This order became this validated document, with this unique reference, this fee, this refund, this settlement, and this ledger impact.”
Only one of those answers survives an audit calmly.
So what exactly is a transaction integrity layer?
A practical definition would be this:
A transaction integrity layer is a control layer between systems that guarantees a business transaction is processed consistently, traceably, and safely across retries, failures, corrections, and downstream dependencies.
It usually does five important jobs.
First, it creates or enforces a canonical transaction identity. One order, one payment, one refund, one invoice request, one adjustment event. Not five slightly different references floating around like lost luggage.
Second, it manages state transitions. For example: received, validated, enriched, posted, submitted, accepted, rejected, cancelled, adjusted, settled.
Third, it enforces idempotency and deduplication. The same event can arrive twice without causing double impact.
Fourth, it supports reconciliation and replay. If one downstream system fails, the transaction can be resumed or replayed safely without corrupting records.
Fifth, it preserves evidence. That matters for finance, compliance, and operations.
In Malaysia, this matters even more for e-Invoice flows because MyInvois expects structured documents, validates them, returns validation metadata, and supports later cancellation, rejection, and retrieval actions. That means the architecture must preserve not just data movement, but the life story of each transaction. (MyInvois SDK)
What features should a real transaction integrity layer have?
For commerce, ERP, and e-Invoice environments, the features that actually matter are quite practical.
It should have a transaction ledger, meaning an internal record of every commercial event and every status change.
It should have idempotency keys so retries do not create duplicates. AWS and Confluent both treat this as foundational for reliability in distributed processing. (AWS Documentation)
It should have stateful orchestration, not just fire-and-forget delivery.
It should support compensation logic. If one step fails after others succeeded, the system should know whether to cancel, reverse, or issue an adjustment rather than simply panic in JSON.
It should maintain referential links between original and follow-up documents. This matters in Malaysia because adjustments can depend on original submitted documents, and the SDK explicitly includes referenced-document and duplicate-document validation rules. (ClearTax)
It should provide observability for business users, not only technical logs. A finance controller should be able to see what happened without reading stack traces like bedtime poetry.
And it should support reconciliation views across order, invoice, fee, refund, payout, and ledger.
A simple Malaysian example
Imagine a seller in Selangor that sells on Shopee, Lazada, TikTok Shop, and its own website.
With only connectors, each channel sends orders downstream. The ERP receives them. Another connector sends invoice data. Another handles refunds. Another pushes settlement files. Everything looks modern until a big campaign day creates delays, retries, and duplicate messages.
Now the finance team sees three bad outcomes at once: duplicate fee postings, missing refund linkage, and e-Invoice submission errors because referenced document states no longer match reality.
With a transaction integrity layer, every commercial event gets a controlled transaction record first. The layer checks whether the event is new or duplicate, validates the state, links it to prior events, and only then allows downstream posting or submission. If a downstream service times out, the layer preserves the truth and retries safely. If MyInvois returns validation feedback or a later cancellation is needed, the same transaction record keeps the chain intact. Official MyInvois APIs are designed around exactly this kind of structured control and retrieval model. (MyInvois SDK)
That does not make the system magical. It makes it governable.
Final thought
The biggest integration mistake at scale is thinking that moving data is the same as protecting a transaction.
It is not.
Connectors are useful. You still need them. But once order volume, channels, retries, refunds, and compliance requirements grow, connectors alone become too thin. They move information, but they do not reliably protect business truth.
A transaction integrity layer is the missing middle. It turns a pile of integrations into a system of record for what actually happened.
And that is why connectors fail at scale.
Not because they are useless.
Because they are not enough.