‹ back home

vdirsyncer: rewriting status management

2024-01-27 #status-update #vdirsyncer

Current state

Vdirsyncer keeps a “status” file locally. The status file contains metadata from items on both storages the last time they were synchronised. My current implementation runs the synchronisation process, and then returns a new status, which should be saved to disk until next time.

I decided on this API since it made the whole codebase easier to reuse in different scenarios, including situation where the local system has no persistent storage (and the status could be saved onto a remote host, or wherever feasible).

This turned out to be bad idea.

Handling interruptions

If synchronisation suffers a fatal interruption (e.g.: power failure, SIGKILL, etc), none of the updated status is saved, which could result in problems during the next synchronisation. If content changes on either side before another synchronisation, it would result in conflict. Deleted content might be mistakenly restored on the other side, but I don’t think it can ever result in data being deleted.

I initially decided to leave this in its current state for the alpha0 release, and to address it before a stable beta release.

Regrettably, this wasn’t the only issue with the status management.

Recovery after one-time failures

Due to a transient network issue, a single synchronisation failed mid-way (it had plenty of operations pending). Upon retrying, there were no operations to run. Something was wrong.

The problem turned out to be that the status file is updated with the new file metadata even if synchronisation fails. So the next time that synchronisation runs, it looks like the file hasn’t changed, and there is nothing to do. In reality, it hasn’t changed since the last failed synchronisation, but it has changed since the last successful one.

Unlike the problem above, I feel that this one is problematic enough to delay an alpha0 release.

To address I am considering using the previous state as input when synchronising, and only updating data for items which are successfully synchronised. However, during the first run, there is no previous state, and for items that are identical on both sides, no action needs to be taken. Currently, the status gets updated based on the actions that are executed, so “no action” means that items never end up in the status database.

To work around this last issue, a new action is needed, a no-op in terms of synchronisation that merely saves the item into the status file. This would only really be relevant during the first-time synchronisation, or if some other tool synchronised both storages.

Have comments or want to discuss this topic?
Send an email to ~whynothugo/public-inbox@lists.sr.ht (mailing list etiquette)

— § —