fuse for unprivileged users
Early this year I signed up to finish my university degree, a licentiate in computer science. I finished all the coursework years ago, and only my dissertation was missing. That absorbed a lot of time during May and June, but it’s now finally complete.
My research (PDF in Spanish) went into mounting fuse filesystems without any form of privilege escalation (some platforms use setuid, others require root). At a high level, a service runs with elevated privileges, and allows authorised users to mount filesystems via the kernel’s fuse interface. The bulk of the work was implementing the proof of concept.
Clients wishing to mount a new filesystem connect to the service via a socket, then specify a mount path, and finally exchange the file descriptor used for communicating fuse operations with the kernel.
The proof of concept allows not only mounting filesystems without setuid binaries, but it also allows handing over the file descriptor to a process inside a sandbox (even inside a docker container), so that a fuse service can run in highly confined environments—a notable departure from requiring privileges to run setuid binaries.
The source is an ugly cannibalisation of the libfuse project (and therefore under the LGPLv2/GPLv2 licences), and I hope to be able to release it at some point, even if only as a prototype. In theory, the general design should be usable in BSDs as well.
pimsync + davmail
Pimsync has now been confirmed to work with DavMail. DavMail exposes a pretty standard CalDAV server, but a couple of early adopters found a bug in pimsync which prevented it from completing the discovery process.
Fixing the bug was straightforward, and pimsync v0.4.4 has now been reported to work fine with DavMail.
JMAP and libjmap
My other point of focus these past weeks has been JMAP support for pimsync.
Status quo
Existing libraries for interacting with JMAP focus on email, and have no support for Calendars or Contacts (although both of these JMAP extensions are experimental).
The main contender was jmap-client. Adding support for Calendars and
Contacts is possible, but the general design doesn’t seem to be a good fit for
pimsync. The main blocker is that this library brings its own I/O (see: sans
I/O) and (high level) HTTP client. This makes it a poor fit for pimsync, which
already has its own I/O and HTTP client implementations. Pimsync also needs a
lower-level HTTP client due to the configuration options exposed.
Cyrus as a JMAP server
In order to work on a JMAP client, I needed a JMAP server. The only implementation which currently supports calendars and contacts is Cyrus IMAP.
I tried to build Cyrus from source in an isolated environment, but came across some build issues on Alpine. I reported these upstream. The developers provided fixes for each issue along the way. Cyrus IMAP now builds and works fine on Alpine, and likely on other musl-based distributions too.
While iterating on the above, I used cyrus-docker-test-server to continue on my own work. Running a server is as simple as:
docker build -t cyrus-testserver .
docker run -it \
-p 8080:8080 -p 8143:8143 -p 8110:8110 -p 8024:8024 -p 8001:8001 \
cyrus-testserver
Hacking on a JMAP client
My first prototype code worked, but was quite ugly, since I learnt quite a few quirks about JMAP while writing it. It doesn’t matter how many times you read the RFCs, you’ll still run into surprises when writing the client. The current code is a second prototype.
Some of the design choices in jmap-client make a bit of sense to me now, but
I’m still not entirely convinced by others. I’ve reached out asking if they’d
consider moving to a sans I/O approach. If so, I believe it’s best to
collaborate with them and attempt to mould their client library to fit my
requirements (even though I’d have to use a fork in the short term in order to
properly support Calendars and Contacts until they are fully standardised).
After having written the current JMAP implementation, I believe that requests would be best designed with a builder pattern. It would make writing JMAP-based applications a lot more pleasant for downstream consumers. Such an implementation would be quite time consuming, and it’s hard to justify it for pimsync’s more basic needs.
libjmap
I’ve extracted my JMAP client implementation into a new library, libjmap. It
exposes the few basic functions required by pimsync:
impl<C> JmapClient<C>
where
C: Service<http::Request<String>, Response = Response<Incoming>> + Sync + Send + 'static,
<C as Service<http::Request<String>>>::Error: std::error::Error + Send + Sync,
{
pub async fn discover_session_url(
mut client: C,
scheme: Scheme,
domain: String,
) -> Result<Uri, DiscoverError>;
pub async fn create_collection<T: Collection>(&self, name: &str) -> Result<Record>;
pub async fn create_record<T: Collection>(
&self,
calendar_id: &str,
record: &JsonValue,
) -> Result<Record>;
pub async fn delete_collection<T: Collection>(&self, collection_id: &str) -> Result<String>;
pub async fn delete_record<T: Collection>(&self, record_id: &str) -> Result<String>;
pub async fn get_collections<T>(&self) -> Result<Vec<T>>;
pub async fn get_records<T: Collection>(&self, calendar_id: &str) -> Result<Vec<Record>>;
// …
}
The get_collections function also fetches properties (display name, colour,
etc), but a function to update properties is still missing.
I’m still missing the support for the state parameter. Each time that the
server returns any data, it returns a state, an opaque value which changes any
time that data on the server changes. This allows sending requests specifying
operations which are only executed if the data has not changed on the server
since we last saw it. This mainly avoids race conditions. It’s closely related
to HTTP’s Etag header, with the biggest difference being that the state
changes any time any item changes, while an Etag applies only to individual
items.
As such, the current (incomplete) JMAP backend in pimsync is subject to race conditions if multiple clients operate on the same server at the same time.
pimsync JMAP support
JMAP support in pimsync is still incomplete, and unreleased.
I need to convert data in JSCalendar format into iCalendar and vice versa, for which I’ve been following the draft RFC. The code at this point works, but mostly handles the basic cases. There is no public release of this yet.
Finally, there is a bit of a mismatch between how JMAP handles a global state
and pimsync handles a per-item ETag. I have a plan on how to work around this,
although this unfortunately adds a bit of extra network overhead.
I believe that some refactors can be done to pimsync which would improve its
integration with JMAP (and also with the missing singlefile storage), but I
have no plans to execute on these any time soon.
Upcoming work
In the coming weeks I’ll continue with the JMAP work. Once that is finished, I’ll move onto OAuth and Google support. After that, I intend to implement any missing features which are present in vdirsyncer, so that pimsync can fully replace it for all use cases.