Best way to show user chronological history of whole database

[andrewf]

Hello folks,

I'm writing an end-user app where the user basically sees and edits each document individually. I want to show them a history of all the changes they've made (or synced). I first tried using the changes API for this, but the fact that it only shows leaf revisions makes it wildly unsuitable, since non-leaf revisions are the main reason a user will want to look at their history.

The least crazy idea I can think of right now is to (a) add my own last-modified timestamp to each document (yes, with the potential shenanigans of wrong clocks), (b) use changes to pull the most recently modified documents, and (c) manually pull all revisions of those documents and sort them by last-modified. This seems... slow, probably? It definitely seems inelegant. Is there a better approach here?

Thanks.

[garethbowen]

The problem is the old revs will be cleaned up the next time compaction runs. As far as I know the only way to do this would be to never edit or delete a doc, but create a new document every time and do the comparison with business logic.

[SourceR85]

As @garethbowen suggested: rely on the revisions of CouchDB/PouchDB for a git-style history isn't a good Idea.

My quick-n-dirty/straight-from-the-head idea: create documents for each new edit, and pump some meta information about the changes (e.g. diff-tree) and a reference to your previous doc.
PouchDB will be a great companion if you put a little bit logic into your _id string (e.g. ${hash}-${numeric_revision}).
Query your docs is then quite simple and fast with the allDocs function (using start- and endkey, or get the newest one with descending: true and limit: 1).

[andrewf]

Putting the revision in the ID seems like it would work. But I don't see how that's better than just not running compaction, which I'm not forced to do, right? And then when I sync, I have to do manually do all the conflict detection and resolution Pouch does for me.

It's also important that my documents link to each other, so all those links would become searches instead of plain lookups (I definitely don't want to deal with deconflicting user edits and recursive link updates if I can help it).

[SourceR85]

I have to do manually do all the conflict detection and resolution Pouch does for me.

Conflict resolution isn't part of PouchDB's responsibility.
In fact: you get the conflicts of a document via get(id, {conflicts: true}), solving them is your job (or the job of your users, if you provide them a merge editor).

[SourceR85]

It's also important that my documents link to each other

Sounds like a job for a view 😉

[andrewf]

Oh, indeed, I'm already using a view for tracking the relationships.

And yes, of course I'm handling the actual resolution in my business logic. It's still nice to have Pouch teeing them up for me, rather than writing code to go hunting for them when frankly I'd rather be doing UI...

[andrewf]

At that point I wonder if it would be better to just store the history inside each document, or something along those lines.

[andrewf]

So I've got a handful of possible solutions:

• Store each "item" as a set of version/diff documents collected by a view or range query, with a shared id prefix (or somesuch), compute current version as a reduction over history.
• Store history/past versions as part of the document (and compact aggressively, since that'll go quadratic really quick, but storing one or two revisions of history is probably fine).
• Store history/journal as a separate doc from the item with a shared id prefix with the item (same quadratic caveat).
• Store history as a set of separate docs (view, etc) with a shared id prefix.

Are there any other broad categories of solution I'm missing? It seems like there must be a few, but my brain is still stuck in the box.

(...where "item" is the actual doc I'm trying to version-control. I'm pretty well committed to UUIDs for these guys, fwiw, since their structure is very dynamic)

[janl]

no, you got it covered. Note that JSON Patch works neatly for this type of thing: https://neighbourhood.ie/blog/2020/09/15/use-json-patch-to-resolve-conflicts/