How not to write a JSContact <> iCalendar converter
As part of my work to implement JMAP support for pimsync, I need to convert JSCalendar entries to and from iCalendar (and likewise, JSContact entries to and from vCard).
Fortunately, there were draft RFCs documenting how to do all these conversions1. Essentially, all fields in iCalendar format have an equivalent field in JSCalendar format and there’s special syntax to keep around unsupported fields in a lossless way so that they can be recovered when converting back.
While reading the RFC I paid special attention to recurrence rules, which I feared might be a great issue. Fortunately, recurrence rules are represented in more or less the same format, so I don’t need to parse them. The same applies for other recurrence-related fields.
iCalendar events can have either (1) a start datetime2 and an end datetime, or (2) a start datetime and a duration. JSCalendar only supports start datetime and duration, so anything in the first format needs to be converted. I didn’t think much of this in the two times that I read the entire RFC. This was a grave
mistake.
My general design
The general approach to my design was as follows:
When reading an iCalendar/vCard file, read each event/todo/contact and once we reach the end of it, return a JSON object with its equivalent on the other type.
When reading a JSCalendar/JSContact file, there’s a function to convert each JSON object into its counterpart, and these functions call each other, forming a sort of tree in the call stack.
vCard <> JSContact
My simple design worked wonders for vCard <> JSContact, and I managed to implement a rough prototype for pimsync3.
I also had unit tests converting JSContact into vCard and back, and a few JSCalendar items into iCalendar and back. Everythiing worked wonders (for the fields that were implemented, a few were just left as “pending”).
At this point, my design seemed solid, so I started building more around it, and slowly working on the iCalendar<>JSCalendar aspect of it. Again, I noticed that converting ’end datetime’ into durations requires a bit more thinking and left it for later, without giving it a second thought.
Stable UIDs
Potentially, vCard files may lack a UID, and conversion needs to add one. When converting the same input multiple times, the process SHOULD add the same UID. This is something which didn’t entirely fit well with my design, but pimsync should take care of this beforehand anyway: all its synchronisation process requires that all entries have a UID anyway.
Timezones are nightmares
This isn’t the first time that timezones sprung up to haunt me in what seemed like a perfectly good design for something that was going to work fine.
It turns out that if we have the start datetime for an event and an end datetime, calculating the duration isn’t that trivial. To do so, we need to take the timezone into account. Does an event start on the day before a daylight transition and end the following day at the same time? Then its duration is 23 or 25 hours, but not 24 hours!
Given how my entire converter converted and serialised each component in isolation, I didn’t have the timezone in scope when converting an event. In fact, the timezone might be present further down in the same file, so it hasn’t even been parsed yet.
Now, it’s far from impossible to re-implement things in a different way to take this into account, but it did imply that I had to scrap most of my code and start over, with a design that keeps some form of “partially converted” representation in memory until all necessary information is found further down in the same file.
Motivation
I admit that being hit by this situation strongly reduced my motivation. Work on JMAP support along with JSCalendar and JSContact has been taking far more than anticipated, and at this stage I mostly want to get it over with and move on to other more pressing features.
As I was dealing with having to re-start the same work with an entirely new approach, the folks from Stalwart Labs posted an email to the JMAP mailing list announcing calcard: a project to convert iCalendar/JSCalendar and vCard/JSContact.
I peeked at the code and was impressed. It’s has mostly minimal dependencies, and takes into account all possible fields. They first convert items into an intermediate representation, and then into the other type. I wasn’t a fan of this approach at first, but then realised it’s the only thing that really works out if you need to work around the timezone issue that blocked me.
calcard bundles rust-rrule, a library to parse recurrence rules which I
have used and contributed to in the past. I’m happy that they’ve reused an
existing implementation that works, and I also have some fixes for rrule which
can hopefully trickle down too.
Moving on
At this point, it makes more sense to adopt calcard for pimsync’s JMAP backend. I’ll contribute fixes or improvements upstream as needed, which should be a better use of time and result in a more robust solution.
This is a strange status update: usually I share something that works, or at least an experiment that demonstrates that an idea does not. In this case, I failed to implement a converter due to a flawed design. There is no greater lesson to be learnt from it either.
While I worked on this project, one of these drafts became a proper standard, RFC 9555. ↩︎
“datetime” in this context refers to a date with an optional time. ↩︎
I’m serious about this being a rough prototype; it entirely ignores race conditions since it’s more of a proof of concept than a version that’s ever supposed to be used. ↩︎