The term dotfiles refers to files with filenames starting with a dot. These are treated as “hidden” files and usually not listed by default in most tools unless an extra flag is provided, a “show hidden files” checkbox is checked or some similar mechanism.
Additionally, the term dotfiles is nowadays often used to describe a specific collection of these files: the ones that contain a user’s configuration associated helper scripts and some other related files. From the Archwiki:
User-specific application configuration is traditionally stored in so called dotfiles (files whose filename starts with a dot). It is common practice to track dotfiles with a version control system such as Git to keep track of changes and synchronize dotfiles across various hosts.
I’ve been keeping my dotfiles in a git repository since 2012, and it’s been of great value. For one, when I reconfigure a program I’m not terrified of “what if this doesn’t really work out”: if the new approach is unconvincing, I can easily roll back to the previous version. Any I can even inspect how something was configured months ago for reference. This alone completely changes how I approach improving workflow or making configuration changes.
That aside, it serves as a great mechanism to keep my configuration in sync across devices; I can commit a change on my desktop and later pull that change into my laptop (or Linux-based phone). At the time, I used this approach to sync configuration of tools across my work computer and home computer.
Approaches on tracking dotfiles
The idea is solid, and there’s a lot of different approaches to achieve similar results. Most of them have downsides, so here’s an overview of everything I’ve tried in over a decade.
Tracking all of $HOME
My first approach was to just create a git repository in $HOME, add the
directories and files that I want, and ignore everything else. By “ignore” I
mean “configure git to ignore” these files so that they don’t continuously show
up as untracked files. This unclutters git’s output which would otherwise
always list irrelevant files.
Keeping this list of ignored files up to date was a pain. There’s always
something new coming up. Of course, I could have ignored * and then un-ignore
everything else, but this results is directories in .config sometimes being
ignored-by-default. It’s very easy to not notice that a specific directory was
not tracked until it’s too late to go back.
Not great at all.
Another downside of this approach is that $HOME is now technically a git
repository, and so are all of its subdirectories. Shell integrations that show
the git-status of the current directory suddenly trigger on any directory,
slowing everything down and displaying meaningless output. If I run git status on a directory that I think is a repository, git yields confusing
results too because it’s not really a repository the way I though it was; it is
actually part of the dotfiles repository.
All in all, this approach as confusing and had issues.
Homesick
For years I used a tool called homesick. The general idea is that the
dotfiles repository is in a dedicated directory, and homesick creates
symlinks to these files in the “real” location in my home directory.
So, for example if my dotfiles repository is in $HOME/.dotfiles, then
$HOME/config/git is a symlink to $HOME/dotfiles/.config/git
A nice upside of this symlinking approach is that it’s easy to see which files
are tracked in my dotfiles repository. I can just run ls -l $HOME/.config,
and very easily visualise which files are symlinks. Those that are not symlinks
are not tracked.
This worked really well for years. The biggest pain points were a few major
updates of Ruby. homesick is written in Ruby, and sometimes when a new Ruby
update rolled out, homesick broke and it needed to be updated.
The last time it happened, it went unfixed for a long time (if it’s ever been fixed at all). I have no idea of what the issue was, or how to fix it either (I’m no Ruby expert). I guess I could have manually installed an older Ruby version, but that has its own set of issues considering that this tool is used to bootstrap new hosts.
My own sync tool
I tried most other tools around, but none of them convinced me. Some of them had more complicated abstractions or rules, others had too many dependencies or tricky setup scenarios.
So I ended up writing my own very simple tool that did a subset of what
homesick does. Specifically, it implemented the subset of features that I
use. This was initially in Python, and later in Rust (which helped iron out a
lot of edge cases). Every once in a while I find a tiny tweak or improvement
that I can make, but overall it’s been really stable.
Most of what this tool does is find files in the dotfiles repository, and make sure that symlinks exist in their expect location, pointing to the in-repository version.
It works well, and does what it must, but has an issue that’s present in all the tools that use a “symlinking” approach, which is what the next section is about:
Sandboxes and containers
When a configuration file for a given program is in my dotfiles, it’s stored in
a repository in $HOME/.dotfiles, and a symlink is placed under the real
location (e.g.: $HOME/.config/git) pointing to it.
This usually works, but breaks in a specific scenario: sandboxing. If an
application runs in a sandbox and is given access to its configuration location
(e.g.: $HOME/.config/some-app), it can “see” the symlink, but trying to
follow it fails due to $HOME/.dotfiles not being accessible inside the
sandbox.
This happens with all sorts of sandboxing, including Firejail, Flatpak or when
simply running a docker container with permissions to read $HOME/.config.
For example, I use neovim with some sandboxed/containerised language servers.
Language servers are given read-only access to the directory of the files being
edited. If I am editing $HOME/.config/nvim/init.lua, then the language server
can only access this directory. But because these files are symlinks, it tries
to read the target files (e.g.: where these symlinks point to), but fails
because, again, these are not accessible inside the sandbox.
Finally I should note: some very few tools don’t play well with symlinks: when they edit a file, they recreate it, so they remove the symlink and place a regular file, which requires that I later resolve the conflict manually before this can be sync into my dotfiles repository. This is a very rare though, and a very minor problem.
Moving forward
I don’t think the symlink situation is great. As more tools run with an isolated view of the filesystem (which is a great practice security-wise), the issues around it will worsen.
So far I’ve considered no longer using symlinks, but instead copying (or maybe
hard-linking) the files in-place out of the dotfiles repository. This likely
requires triggering conflict resolution manually when a file was edited in its
external location and a modified version is pulled by git. This scenario
shouldn’t be very frequent and isn’t terrible anyway; I can just trigger
vimdiff to handle this manually.
The main downsides of this approach are:
- It’s very hard to visualise which files are tracked and which ones aren’t. I
might add configuration for a new program in
.config, and forever miss that it hasn’t been committed. - There’s no tracking entire directories; only individual files.
The second problem can be solved by keeping list of directories into which the sync tool should automatically operate recursively.
For the first problem, however, I cannot see any obvious solution. Perhaps writing a bunch of code to fine untracked directories? I suspect this would permanently print out a lot of noise much like the situation with my first approach to dotfile-tracking.