Skip to main content

Dub Replication Modes

Dubs support two modes for capturing data from your source database:

  1. Logical Replication – continuously streams changes using native database mechanisms.
  2. Full Copy – fetches the full dataset at each snapshot.

Logical Replication​

Logical replication leverages native mechanisms in PostgreSQL and MySQL/MariaDB to subscribe to changes (i.e., transactions) in real time. This mode maintains a continuously updated, sanitized replica of your source database.

How it works​

  • During setup, the agent connects to your database, verifies the configuration, and creates a logical replication subscription.
  • An initial full copy of the database is performed to bootstrap the replica.
  • After the initial copy, all subsequent changes (INSERTs, UPDATEs, DELETEs) are streamed continuously through the masking agent.
  • Dubhub also tracks and applies schema changes to keep the replica aligned with the source:
    • In MySQL/MariaDB, DDL changes are captured natively through the binary log.
    • In PostgreSQL, standard logical replication does not include DDL changes, so Dubhub detects and applies them independently.

Dubhub detects:

  • Row-level changes streamed from the source
  • New tables and structural changes to existing tables
  • Additional objects such as indexes, sequences, and empty tables (captured during snapshots)

Note: Starting with PostgreSQL 16, logical replication can run on a read replica - allowing you to avoid adding load to your primary database.

Benefits​

Logical replication is ideal for large or frequently updated datasets:

  • Faster snapshot creation – Since changes are continuously applied, snapshots can be created in seconds, keeping data fresh.
  • Quicker Docker clone downloads – After the initial snapshot download, delta snapshots are much smaller, making Docker clone downloads faster.

Considerations​

Logical replication requires changes to the database configuration, which may include enabling replication settings and restarting the database (See the database configuration guides under Advanced Topics for more information). This setup is more involved but enables faster, more efficient snapshots going forward.

Full Copy​

In this mode, Dubhub queries each table directly and captures the full dataset every time a snapshot is taken.

Benefits​

  • Simple setup – No database configuration or restarts required.

Tradeoffs​

  • Slower snapshot creation – The entire dataset is copied every time.
  • Higher resource usage – Less efficient for large datasets or frequent snapshot schedules.

Choosing a Mode​

If you're just getting started and logical replication isn't already enabled, we recommend beginning with Full Copy mode. It's quick to set up and works out of the box. When you're ready for optimized performance, you can switch to Logical Replication for faster, incremental snapshots.