Business Problem
Over eight years, the platform had accumulated integrations with accounting packages, listing portals, payment providers, e-signature services, and communication tools, 25+ external systems, each built ad hoc by whichever team needed it. Auth handling, retry logic, rate limiting, and error reporting were reimplemented differently every time, with predictably different bugs. Integration failures were the largest incident category, and every third-party outage or API change became a multi-team fire drill.
The cost showed up in two ways: new integrations took roughly six weeks each because every one restarted from zero, and operational load kept climbing because 25 bespoke implementations meant 25 distinct failure behaviors nobody fully knew. Product wanted to double the integration catalog; operations wanted the pager to stop. Both pointed at the same answer: a shared integration platform.
Architecture
I designed a hub with three layers. An egress gateway (Go) fronted all outbound third-party traffic: centralized credential injection, per-provider rate limiting, circuit breaking, retries with budget-aware backoff, and uniform request/response logging. A connector runtime (Node.js/TypeScript) hosted per-provider connectors written against a strict SDK contract covering auth flows, pagination, webhook verification, and error taxonomy. A sync orchestration layer managed scheduled and event-driven data flows with cursor persistence and dead-letter handling.
Credentials moved into a dedicated vault service handling OAuth2 flows, token refresh, and encryption, with connectors receiving short-lived injected tokens rather than storing anything. This retired eleven separate homegrown token-refresh implementations, three of which were found storing credentials with reversible encryption during the audit that preceded the migration.
Observability was uniform by construction: every proxied call carried OpenTelemetry traces with provider, connector, and tenant attributes, feeding Grafana dashboards and per-provider SLO alerts. When a third party degraded, the on-call saw one dashboard telling them which provider, which tenants, and whether the circuit breaker had already contained it, instead of correlating five services' logs.
System Design
Single egress gateway giving one enforcement point for rate limits, retries, circuit breaking, and credential injection across all 25+ providers.
Strict connector SDK contract (typed auth, pagination, webhook, and error interfaces) making connector quality a property of the platform rather than of each author.
Centralized OAuth2 vault with short-lived token injection, eliminating credential storage from all connector code.
Provider-level circuit breakers with tenant-visible status, converting third-party outages from platform incidents into communicated degradations.
Cursor-persisted sync orchestration with idempotent upserts and dead-letter queues, making every data flow resumable and replayable.
Uniform tracing and per-provider SLOs, so integration health became a dashboard instead of tribal knowledge.
My Responsibilities
Owned the hub architecture and the migration strategy for all existing integrations; led a team of five across two squads.
Built the Go egress gateway, including the rate limiter and circuit-breaker implementations tuned per provider.
Designed the connector SDK and wrote the first three connectors as reference implementations with an accompanying conformance test suite.
Led the credential audit and the vault migration, coordinating disclosure and remediation for the weakly-encrypted credential findings.
Defined the per-provider SLOs with operations and built the alerting that replaced ad hoc integration monitoring.
Ran the 14-month migration of 25 legacy integrations, sequenced by incident frequency so the noisiest moved first.
Implementation
The gateway went first and delivered value before any connector migrated: pointing legacy integrations' outbound traffic through it immediately gave uniform logging, rate limiting, and circuit breaking to code that had never had them. Integration incident count started falling a quarter before the first connector was rewritten.
The connector SDK was extracted from, not designed ahead of, the first three reference connectors, an accounting package, a listing portal, and an e-signature provider, chosen for maximal diversity of auth and sync patterns. The conformance suite that every connector must pass encodes the lessons from those three, and it is why connector number 25 behaves like connector number 4.
Migration sequencing was ruthlessly incident-driven: the top five noisiest integrations moved first and accounted for most of the operational win. By the back half of the program, connector rewrites were routine enough that two were completed by engineers outside the platform team, which was the real validation of the SDK.
Results
Integration-related incidents down 73% year over year after full migration; integration incidents left the top-five incident categories entirely.
New integration build time reduced from ~6 weeks to 4 days median, measured across the eight integrations added post-platform.
Gateway proxied 6M calls per day with p95 added overhead under 40 ms.
Eleven bespoke token-refresh implementations retired; three weak credential-storage findings remediated during the vault migration.
Third-party outages contained by circuit breakers with tenant-visible status pages, cutting outage-driven support tickets by more than half.
Integration catalog grew from 25 to 33 systems in the year after launch with no growth in the platform team's operational load.
Lessons Learned
Ship the gateway before the framework; centralizing egress delivered most of the operational win before a single connector was rewritten.
Extract SDK contracts from real reference implementations rather than designing them speculatively.
A common error taxonomy is the least glamorous and most compounding investment in any integration platform.
Sequencing migrations by incident frequency front-loads the value and builds the political capital a long program needs.
Conformance test suites are how platform teams scale beyond themselves; outside teams shipping connectors was the success metric that mattered.
Future Improvements
Self-service connector development portal with sandboxed testing against recorded provider fixtures.
Declarative sync definitions for simple providers, generating connectors from an API spec plus mapping config instead of code.
Per-tenant integration usage analytics exposed in the product, turning the hub's telemetry into a customer-facing feature.
Automated provider API change detection by diffing observed response shapes against expected schemas, alerting before breakage hits users.
Challenges
Migrating live integrations without data gaps
Accounting and payment syncs could not miss records during cutover. Each migration ran old and new paths in parallel with a comparison harness diffing outputs for two weeks, then cut over with the legacy path in warm standby. Cursor handoff was rehearsed in staging with production-shaped data. All 25 migrations completed without a confirmed data gap.
Rate limits shared by adversarial workloads
A tenant's bulk historical sync and another tenant's real-time payment status check hit the same provider quota. The gateway implemented two-tier budgeting, a reserved interactive lane and a preemptible bulk lane per provider, so latency-sensitive calls never queued behind backfills. Interactive p95 through the gateway stayed under 40 ms of added overhead.
25 different error vocabularies
Every provider fails differently, and the old integrations surfaced raw provider errors to users. The SDK enforced mapping into a common taxonomy (auth-expired, rate-limited, provider-down, invalid-data, retriable-unknown) with defined platform behavior per class. Support could finally answer 'what does this error mean' from one runbook, and automated recovery handled the retriable classes uniformly.