One of the most popular features present in vdirsyncer and still missing in pimsync is support for Google’s CalDAV service. While CalDAV itself is standard, Google’s implementation is unique in a couple fo ways (while still technically being standards-compliant).

There are two major differences that tools like pimsync and vdirsyncer need to keep in mind.

Calendar paths

[permalink]

The first is not a big deal, and relates to how Google defines the paths for each calendar. Most servers host all of a users’s calendars under a single path, such as:

/users/hugo@whynothugo.nl/personal/
/users/hugo@whynothugo.nl/work/
/users/hugo@whynothugo.nl/travel/

Both vdirsyncer and pimsync follow a convention based on this: the last component in a calendar’s path is treated as its name, and is also used as a name for a local directory which mirrors that calendar.

Google uses something that looks vaguely like so:

/user/some-user/calendars/aluMp7OOf2/events/
/user/some-user/calendars/GCmYjRTK1t/events/
/user/some-user/calendars/LZ81HRKLRv/events/

Note that each calendar is a collection named “events” and is itself contained in a collection with a unique name.

Vdirsyncer implements Google support as a special type of storage, and uses the second-to-last component as the unique name for a calendar.

For pimsync, I intend to do something similar; I’ll implement a behaviour to use the second-to-last component as a calendar name, but leave this behind an independent configuration directive. This setting will automatically default to True if you’re syncing with https://www.googleapis.com, but shall also be usable in other settings where a similar behaviour is required.

OAuth instead of Basic Auth

[permalink]

Google only offers OAuth for authentication to their CalDAV services. The flow in this authentication mechanism is quite different from Basic Auth (and Digest Auth), which is what most other providers and implementation use.¹

When using Basic authentication, a user simply provides a username and password, and pimsync uses that to authenticate itself.

When using Google’s flavour of CalDAV, the steps are a lot more:

1. Obtain a client_id and client_secret via Google’s Web interface. For web-based services, the operator of the service provisions these credentials and users don’t need to deal with this, but for software that runs on a local host, end users need to do this themselves. This only needs to be done once, but can be quite off-putting and confusing for new users.²
2. Configure pimsync with this client_id and client_secret and initiate the authentication flow. This opens a web-browser, where the user logs in through Google’s website.
3. After logging in, the user is redirected to a URL they configured in step 1. Vdirsyncer runs an HTTP service in a local port to handle this redirection, but it’s always a source of issues when users are configuring a remote host which doesn’t expose arbitrary ports. The only flow usable by desktop applications was deprecated by Google some years ago without a usable substitute.
4. Finally, pimsync can handle the tokens returned by Google to perform authentication. There are two tokens: one is used to authenticate requests and expires quite often. The other token is used to renew the first. Both of these need to be periodically persistent, which is what makes the whole mechanism stateful.

Personal thoughts on the situation

[permalink]

Personally, I’d suggest to folks to just stay away from Google and use some other hosting provider that isn’t contiguously campaigning against user’s digital rights and digital autonomy. But the reality is that a lot of folks simply don’t have a choice. The most typical scenario is one’s employer using Google Workspaces to coordinate internally. People who are in such situation simply need to access calendars on Google’s servers and don’t have any choice on their service provider.

And then, if tools like pimsync don’t support Google’s CalDAV services, individuals and organisations would have a much harder time migrating away onto another platform.

Proposed solutions

[permalink]

I have been pondering for a long time two potential solutions…

An OAuth proxy

[permalink]

The first approach is to implement an OAuth proxy. This proxy is configured once by an operator, and then usable by themselves or by their family and community. The proxy is configured once with a pair of client_id and client_secret, negating the need for everyone else to provision their own. When users create their account on the proxy, they also log in with their Google accounts, and the access tokens are stored by the proxy, encrypted with their password. Finally, users simply configure pimsync to talk to this server, which decrypts their tokens with their password and relays requests to Google’s services.

I like the approach in this solution: each of the two components involved do one thing (i.e.: good separation of concerns), it doesn’t further increase complexity in pimsync, and the proxy is also usable by any CalDAV or CardDAV application which supports Basic Auth. It doesn’t just enable pimsync to use Google’s CalDAV services, but any other CalDAV application, with the application requiring any custom support.

Alas, for the typical sole user configuring one or two of their own computers, this doesn’t make life a lot easier — it merely makes things harder by having to configure two separate services and glue then together.

I still think that the proxy is a useful tool to be implemented in future, but it won’t be the primary solution for pimsync. I have a much longer design specification this idea, so please reach out if you’d be interested in implementing it.

Native OAuth support

[permalink]

The second potential solution is to implement OAuth support in pimsync itself. Most feedback I’ve received has also hinted that this is the most user-friendly approach, and this is what I plan to implement in future.

As I mentioned a few months ago, libdav itself uses a middleware approach for extension functionality to its CalDAV and CardDAV clients. Functionality like implementing authentication and setting custom headers is already feasible with this approach, so this library doesn’t need any custom support for OAuth. This is great news, because it allows keeping this library compact and well-focused.

In order to make this feature a reality, the following functionality needs to be implemented:

1. Support configuring OAuth configuration flows. The OAuth parameters themselves shall be provided separately, to avoid having to repeat them for multiple storages (the example below re-uses the same configuration for CalDAV and CardDAV). The client_id and client_secret parameters can be specified as an external command with the same syntax as passwords. A full example:

auth google_foo {
    type oauth
    auth_url https://accounts.google.com/o/oauth2/v2/auth
    client_id 123
    client_secret 123
    token_file ~/.local/share/pimsync/foo.tokens
    scope https://www.googleapis.com/auth/calendar
    scope https://www.googleapis.com/auth/carddav
}

storage calendars_foo {
    type caldav
    url https://apidata.googleusercontent.com/caldav/v2/
    auth google_foo
}

storage contacts_foo {
    type caldav
    url https://www.googleapis.com/.well-known/carddav
    auth google_foo
}

2. A middleware that implements an OAuth flow on top of the tower service trait. Existing Rust libraries which implement OAuth all seem to implement too much, including their own internal IO (i.e.: the opposite of sans-io). They are not composible with other libraries, including libdav. I’ll implement a minimal library for OAuth clients, and a lightweight middleware to fit it into the tower stack.

3. Include the library itself as a dependency of pimsync, and write the final glue code to use the middleware in cases of a matching configuration.

4. Finally, if the URL matches any of the two URLs above, use the alternative logic for calendar names that I mentioned above in Calendar Paths.

Note that all OAuth related URLs can be provided in the configuration, so the implementation is usable with any server which wishes to implement OAuth in future, and not just the one that exists right now.