Current state
Vdirsyncer keeps a “status” file locally. The status file contains metadata from items on both storages the last time they were synchronised. My current implementation runs the synchronisation process, and then returns a new status, which should be saved to disk until next time.
I decided on this API since it made the whole codebase easier to reuse in different scenarios, including situation where the local system has no persistent storage (and the status could be saved onto a storage server, or wherever feasible).
This idea turned out to be problematic.
Handling interruptions
If synchronisation suffers a fatal interruption (e.g.: power failure, SIGKILL,
etc), none of the updated status is saved, which could result in problems
during the next synchronisation. Specifically, if content changes on either side
before another synchronisation, it would result in conflict. Deleted content
might be mistakenly restored on the other side. While annoying, I don’t think it
can ever result in data being deleted.
I initially decided to leave this in its current state for the alpha0
release, and to address it before a stable beta release.
Regrettably, this wasn’t the only issue with the status management.
Recovery after one-time failures
Due to a transient network issue, a single synchronisation failed mid-way (it had plenty of operations pending). Upon retrying, there were no operations to run. Something was wrong.
The problem turned out to be that I was updating the status file with the new metadata even if synchronisation failed. So the next time that synchronisation runs, it looks like the file hasn’t changed, and there is nothing to do. In reality, it hasn’t changed since the last failed synchronisation, but it has changed since the last successful one.
Unlike the problem above, I feel that this one is problematic enough to delay
an alpha0 release.
To address I am considering using the previous state as input when synchronising, and only updating status data for items which are successfully synchronised. However, during the first run, there is no previous state, and for items that are identical on both sides, no action needs to be taken. Currently, the status gets updated based on the actions that are executed, so “no action” means that items never end up in the status database.
To work around this last issue, a new action is needed, a no-op in terms of synchronisation that merely saves the item into the status file. This would only really be relevant during the first-time synchronisation, or if some other tool synchronised both storages.