Decoupling Monolithic Databases: A Log-Based Change Data Capture Framework for Phased Microservices Evolution

Abstract

The migration of established enterprise systems from monolithic architectures to microservices is frequently constrained not by application logic, which containerisation and orchestration render tractable, but by the data tier. Where multiple business domains operate against a shared relational schema, tight coupling at the data layer limits deployment independence and negates much of the agility that microservice decomposition is intended to deliver. Conventional remedies are unsatisfactory: batch Extract-Transform-Load introduces latency incompatible with operational coexistence, while application-level dual writing requires distributed transaction management and admits partial-failure states that are difficult to reconcile. This paper presents a framework for decomposing a monolithic data tier using log-based Change Data Capture (CDC) as the synchronisation mechanism, with the Confluent Platform and Kafka Connect serving as the reference implementation. The framework organises the migration into four sequential and individually reversible phases - passive shadowing, transformation and routing, dual-state reconciliation, and staged cutover - each aligned with the incremental substitution logic of the Strangler Fig pattern. We describe the implementation architecture spanning source connector configuration, schema governance via a registry-enforced data contract, and sink delivery to polyglot targets. We further analyse the principal trade-offs the approach entails, comparing log-based capture against the transactional outbox pattern, characterising the consistency window introduced by the shift from strong to eventual consistency, and addressing tombstone propagation, distributed tracing, and compensating transactions. Finally, we specify an evaluation protocol by which implementations of the framework may be empirically assessed. The framework is offered as a design reference for practitioners undertaking phased data-tier decomposition.