diff --git a/architecture/powersync-service.mdx b/architecture/powersync-service.mdx index 90c0c5e1..792266e0 100644 --- a/architecture/powersync-service.mdx +++ b/architecture/powersync-service.mdx @@ -93,6 +93,15 @@ After that, data is incrementally replicated using a change data capture stream. The PowerSync Service preserves source transaction boundaries when replicating. For Convex, mutations are ACID transactions. When a mutation writes multiple documents, Convex exposes those writes in `document_deltas` with the same commit timestamp, and PowerSync replicates all writes from the same mutation together as one batch. Clients do not observe a partial result from a single Convex mutation. +### Replication Batches + +Replication is batched. The Service accumulates changes in memory and periodically flushes them to bucket storage. Each flush writes three different kinds of record: + +- **Bucket operations**: entries appended to the [operation history](#operation-history) of each affected bucket, as `PUT` or `REMOVE` operations. A single source row produces one operation per bucket it belongs to, so a row that is shared across many buckets produces many operations. +- **Parameter index entries**: lookup entries that let the Service resolve which buckets a client should sync. These are written for rows that feed a stream's parameters. +- **Source records**: one record per source row, holding the row along with the buckets and lookups it currently belongs to. The Service compares against this record to work out which bucket operations and parameter index entries to remove when the row later changes or is deleted. + +Each flush is logged with these counts, along with the size and duration of the write. See [Instance Logs](/maintenance-ops/monitoring-and-alerting#instance-logs) for how to view them. ## Streaming Sync diff --git a/maintenance-ops/monitoring-and-alerting.mdx b/maintenance-ops/monitoring-and-alerting.mdx index 7e866391..314e2e44 100644 --- a/maintenance-ops/monitoring-and-alerting.mdx +++ b/maintenance-ops/monitoring-and-alerting.mdx @@ -263,6 +263,31 @@ rid: For diagnosing sync latency, see [Diagnosing Sync Latency](/debugging/troubleshooting#diagnosing-sync-latency). +### Replication Flush Entries + +Replication logs record a `Flushed` entry each time a batch of replicated changes is written to bucket storage. These entries are the most direct view of replication throughput: + +``` +Flushed: 1200 ops, 30 index entries, 450 records. 512kb in 240ms. Last op_id: 88421. Replication lag: 3s +``` + +These values are also available as structured properties under `flushed` on each entry: + +| Property | Description | +| ------------------------- | --------------------------------------------------------------------------------- | +| `bucket_ops_count` | Bucket operations appended to operation history. | +| `parameter_indexes_count` | Parameter index entries written for rows that feed stream parameters. | +| `source_records_count` | Source rows persisted, along with the buckets and lookups they belong to. | +| `size` | Size of the flush in bytes. | +| `duration` | Time taken to write the batch, in milliseconds. | +| `replication_lag_seconds` | Age of the oldest uncommitted change in the batch, in seconds. | + +Learn more about these counts in [Replication Batches](/architecture/powersync-service#replication-batches). + + + `replication_lag_seconds` is only present when the Service can determine the oldest uncommitted change in the batch. + + ## Custom Metadata in Sync Logs Custom metadata in sync logs allows clients to attach additional context to their PowerSync connection for improved observability and analytics. This metadata appears in the Sync & API logs, making it easier to track, debug, and analyze sync behavior across your app. For example, you can tag connections with app version, feature flags, or business context.