Compliance Matrix Engine

The platform tracked regulated entities, properties, licenses, and safety certificates, across multiple jurisdictions, each with its own regulation set and renewal timelines. Compliance status was computed by a tangle of SQL views and PHP cron jobs that took six hours nightly and nobody fully understood. When a jurisdiction changed a rule, updates took two to three weeks of risky development.

Worse, the output was not auditable. When a customer disputed a compliance flag, or a regulator asked why an entity was marked compliant on a specific date, reconstructing the answer meant archaeology through logs and database backups. Legal flagged this as a material risk. The business needed rule changes deployable in days, evaluations explainable per entity per date, and the nightly batch finished before business hours in every timezone served.

I designed a standalone evaluation engine in Go, exposed over gRPC, that separated three concerns cleanly: rule definitions, entity facts, and evaluation results. Rules were versioned declarative documents, conditions over typed entity attributes with effective-date ranges, stored in PostgreSQL and authored through an internal UI with validation. Facts were ingested from the main platform through a change-data feed.

Evaluation was a pure function: (rule set version, fact snapshot, evaluation date) always produced the same result, and the engine stored the full input references with every output. This made historical reconstruction exact rather than approximate; answering 'why was this entity compliant on March 3rd' became a single indexed query returning the rule version, the facts used, and the decision trace.

The nightly batch ran as a Kubernetes job that partitioned entities across workers. Because evaluation was pure and inputs were snapshotted, workers scaled horizontally with zero coordination beyond partition assignment, and any failed partition could be re-run idempotently.

Declarative, versioned rule documents with effective-date ranges, so a regulation change scheduled for a future date could be authored and reviewed weeks ahead.

Pure-function evaluation with snapshotted inputs, giving deterministic replay and exact historical reconstruction for audits.

Decision traces stored per evaluation: every fired condition and its input values, compressed to keep storage around 40 bytes per check.

Horizontally partitioned batch workers on Kubernetes, scaling to 30 pods to finish 1.4M evaluations in 35 minutes.

Incremental re-evaluation triggered by fact changes during the day, so an updated certificate reflected in compliance status within a minute instead of waiting for the nightly run.

Rule linting and dry-run diff in CI: every rule change PR showed exactly which entities would change status before merge.

Designed the rule document model and evaluation semantics in collaboration with the compliance domain experts.

Built the Go evaluation engine, the gRPC API, and the batch orchestration on Kubernetes.

Implemented the decision-trace storage format and the audit reconstruction query path.

Created the dry-run diff tooling that let compliance officers preview the blast radius of a rule change before deployment.

Wrote the Terraform modules and GitHub Actions pipelines for the engine's infrastructure.

Migrated all 40+ legacy rule sets, pairing with domain experts to translate undocumented SQL logic into reviewed declarative rules.

The engine core was around 8,000 lines of Go with the evaluation semantics property-tested heavily: for any rule set and fact set, evaluation had to be deterministic, total, and order-independent. Property testing caught three subtle bugs around date-boundary conditions that example-based tests had missed.

Migration ran rule set by rule set over five months. Each set went through translation, differential validation against the legacy system, sign-off by a compliance officer using the dry-run diff report, then cutover. The differential harness stayed on for 30 days after each cutover as a regression tripwire.

Prometheus metrics covered evaluation throughput, per-rule-set latency, and diff counts during migrations, with Grafana dashboards the compliance team themselves watched. Giving the domain team direct visibility built the trust needed to retire the legacy system completely.

Nightly batch window shrank from 6 hours to 35 minutes for 1.4M entity checks.

Rule change lead time dropped from 2-3 weeks of development to same-day authoring with next-day deployment after review.

100% of audit requests answered with exact historical reconstruction; average response time for audit queries fell from days of manual work to under a minute.

Intraday incremental evaluation reflected fact changes in compliance status within 60 seconds.

Roughly 9,400 latent compliance calculation bugs identified and deliberately resolved during migration.

Compliance team became self-sufficient in rule authoring; engineering involvement in rule changes dropped to review-only.

Purity and snapshotted inputs are the cheapest route to auditability; explanation becomes a query instead of a forensic project.

A differential harness against the legacy system is worth building properly, it becomes your migration engine, your test suite, and your trust-building tool.

Bitemporal modelling is expensive to retrofit and cheap to do up front in any system where regulators ask about the past.

Dry-run diffs turn scary rule deployments into reviewable pull requests for non-engineers.

Add a rule simulation sandbox where compliance officers can test hypothetical regulation changes against production-shaped data.

Expose evaluation-as-a-service to customer systems via a public gRPC/REST API with per-customer rate limits.

Auto-generate plain-language explanations of decision traces for customer-facing compliance reports.

Explore incremental view maintenance to replace the remaining nightly batch entirely with continuous evaluation.