Globally applying overlays onto Nixpkgs

These days, it is fashionable to have the Nixpkgs for your NixOS system managed alongside its configuration, pinned by some tool of choice. In Pinning NixOS with npins, this time without flakes for real, I've talked about how I want the same Nixpkgs to be used for both my system configuration as well as all my ad-hoc nix-shells. However, there is a key distinction between any import <nixpkgs> {} used in the wild and the pkgs provided within the NixOS module system: Only the latter has its overlays applied.

I'm a heavy user of overlays, and this discrepancy has bitten me more than once. For example, if I pin a certain version of a tool used for a service, and then do nix-shell -p that-tool to do some maintenance on the service, I suddenly have a version mismatch.

The reason for this dilemma is that NIX_PATH contains, well, a path, and that overlays are configuration for a Nixpkgs instance: import <nixpkgs> { overlays = …; }. Evaluating pkgs.path will yield the original path from <nixpkgs>, with any applied configuration erased.

Spray'n'pray: Using global overlays

Nixpkgs has a slightly niche feature: It can automatically load overlays impurely from the environment at instantiation time. This is implemented in <nixpkgs>/pkgs/top-level/impure.nix:

39
  overlays ? let
40
      isDir = path: builtins.pathExists (path + "/.");
41
      pathOverlays = try (toString <nixpkgs-overlays>) "";
42
      homeOverlaysFile = homeDir + "/.config/nixpkgs/overlays.nix";
43
      homeOverlaysDir = homeDir + "/.config/nixpkgs/overlays";

as we can see here, it will source and automatically apply overlays from the following locations:

• <nixpkgs-overlays>, set by putting nixpkgs-overlays=/path/to/my-overlay.nix into NIX_PATH,
• ~/.config/nixpkgs/overlays.nix,
• and all files in ~/.config/nixpkgs/overlays/* (n.b.: hard-coding ~/.config is not XDG basedir spec compliant!).

This means that if any of these provide any overlays, then any call to import <nixpkgs> {} will behave as if overlays = [ … ] had been passed to it. Equipped with this, we can simply have:

{
  channel.enable = false;
  nix.nixPath = [
    "nixpkgs-overlays=/etc/nixos/nixpkgs-overlays"
  ];
  
  environment.etc = {
    "nixos/nixpkgs-overlays".source = put path to overlays here;
  };
}

I decided against putting my overlay into ~/.config/nixpkgs/overlays.nix, because I wanted a truly global solution; I'm afraid things could break pre-login when /home is still encrypted. If Nixpkgs implemented the full XDG basedir specification, then we'd be able to simply put our overlay somewhere into /etc/xdg and wouldn't have to bother with also setting NIX_PATH.

With this solution, the overlay will be applied to all Nixpkgs instances, not only those of our system's Nixpkgs as found via import <nixpkgs> or behind the scenes in nix-shell -p. Notably, this will also apply to any pinned dev shell à la import sources.pkgs {}, no matter how old or new it is. Impurely meddling with dev environments is a recipe for confusing error messages, and likely to fail: Overlays are typically written to work against a specific Nixpkgs version, and trying to apply it to some very old pinned Nixpkgs pin will likely throw on a missing attribute.

This can be a desired outcome, and can work with an overlay carefully designed for such an environment. However, in many other cases it is desirable to only override Nixpkgs instances that aren't explicitly pinned by their own tooling.

A more selective approach: applyPatches

So, NIX_PATH wants a path. Let's give it one. The plan is to patch our system's Nixpkgs so that it will automatically apply a specific overlay when imported. Other Nixpkgs instances, pinned from other sources, will be unaffected. We can simply inject ourselves into Nixpkgs's default.nix entry point, like this:

-  import ./pkgs/top-level/impure.nix
+  args: (import ./pkgs/top-level/impure.nix) (args // { overlays = [ @PUT_OVERLAY_HERE@ ] ++ args.overlays or []; })

Applying the patch is slightly involved but straightforward. We need an unpatched Nixpkgs instance for bootstrapping, for accessing applyPatches and other library functions.

{
  pkgsPath = pkgsBootstrap.applyPatches {
    src = sources.pkgs; # Nixpkgs pin here
    patches = [
      (
        # Replace `${./overlay.nix}` with the correct path in there
        pkgsBootstrap.writeText "overlays.patch" ''
          diff --git a/default.nix b/default.nix
          index bdab048245e2..617d15b6bfd5 100644
          --- a/default.nix
          +++ b/default.nix
          @@ -27,4 +27,4 @@ if !builtins ? nixVersion || builtins.compareVersions requiredVersion builtins.n

          ${" "}else

          -  import ./pkgs/top-level/impure.nix
          +  args: (import ./pkgs/top-level/impure.nix) (args // { overlays = [ ${./overlay.nix} ] ++ args.overlays or []; })
        ''
      )
    ];
  };
}

Side note: This is why one does not put patches inline in Nix code. All lines of code that start with a space will be fucked up by the editor any time the indentation changes, moreover the patches list wants paths anyways so it requires wrapping in writeText. The ${" "} in the string is a hack to work around the editor eating that one whitespace. Putting the patch in a file with @SUBSTITUTIONS@ for the overlay path is the way to go here.

This solution still has a kink that needs to be ironed out: As it is, the provided overlay.nix would have to be self-contained, i.e. not import any other Nix code. Often times, this is not the case: A system configuration's overlay may import packages from npins or locally vendored packages. One solution is to put the entire configuration surrounding our overlay into the store (this is what flakes do), but this means that the source path changes any time any bit of configuration is touched, even if it has nothing to do with the overlay. Instead, pkgs.lib.fileset offers some functionality to only include the files we care about:

{
  overlaySource = pkgsBootstrap.lib.fileset.toSource {
    root = ./.;
    fileset = pkgsBootstrap.lib.fileset.unions [
      ./overlay.nix
      # Add all files/folders that the overlay imports here
      ./npins
      ./pkgs
    ];
  };
}

In the patch, we then simply replace ${overlay.nix} with ${overlaySource}/overlay.nix.

Conclusion

If you have an overlay that you want to apply to all Nixpkgs instances, use global overlays. If you have an overlay that you want to globally apply to a specific Nixpkgs instance, apply the overlay as a patch instead.

I'm honestly unsure if I'd recommend this approach to anybody else. It works for me and solves this very specific issue I have, though I can imagine most people don't care much about it.