feat: explain the configuration for copy_data retries#66
Open
feat: explain the configuration for copy_data retries#66
Conversation
Deploying pgdog-docs with
|
| Latest commit: |
4779ffa
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://df223eef.pgdog-docs.pages.dev |
| Branch Preview URL: | https://push-prqnkmuwyull.pgdog-docs.pages.dev |
levkk
reviewed
Apr 28, 2026
|
|
||
| The two cases behave differently: | ||
|
|
||
| - **Destination shard down** — PgDog opens a fresh connection on the next attempt. |
Contributor
There was a problem hiding this comment.
Let's use tables here, e.g.:
| Situation | Behavior |
|---|---|
| Destination shard down | PgDog opens a fresh connection on the next attempt. |
levkk
reviewed
Apr 28, 2026
|
|
||
| Most drops are clean: table copies run inside an implicit transaction, so Postgres rolls back on disconnect and the destination is left empty. | ||
|
|
||
| The awkward case: the connection drops *after* `COPY` commits on some or all shards but before PgDog records it as done. The rows survive, and the next attempt hits primary key violations immediately. |
Contributor
There was a problem hiding this comment.
The wording here is not ideal. We should be a bit more neutral about edge cases / bugs we haven't fixed yet :)
Contributor
Author
There was a problem hiding this comment.
did the rewording of the new sections it attempt to improve this
levkk
reviewed
Apr 28, 2026
|
|
||
| ### Replication slot not dropped after abort | ||
|
|
||
| Aborting `COPY_DATA` without restarting it leaves the **permanent** replication slot sitting on the source. Postgres won't recycle WAL until it's gone. Drop it manually: |
Contributor
There was a problem hiding this comment.
I'm not sure this is true or we have a bug. We do drop that slot on copy abort, but I guess it may not always work, so it's a good idea to have this documented nonetheless.
Contributor
Author
There was a problem hiding this comment.
warning is generated in any case https://github.com/pgdogdev/pgdog/blob/a6afd9b510c8aedf2d14cfe1882db5d27f856884/pgdog/src/admin/stop_task.rs#L37
levkk
reviewed
Apr 28, 2026
| |-|-| | ||
| | [Pre-data](#pre-data-phase) | Create identical tables on all shards along with the primary key constraint (and index). Secondary indexes are _not_ created yet. | | ||
| | [Post-data](#post-data-phase) | Create secondary indexes on all tables and shards. This is done after [moving data](hash.md), as a separate step, because it's considerably faster to create indexes on whole tables than while inserting individual rows. | | ||
| | [Post-data](#post-data-phase) | Create secondary indexes on all tables and shards. This is done after [moving data](move.md), as a separate step, because it's considerably faster to create indexes on whole tables than while inserting individual rows. | |
Contributor
There was a problem hiding this comment.
When changing URLs, make sure to add a redirect to mkdocs.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
child of pgdogdev/pgdog#916