Something went wrong. Try again.
What happens if I push all of nixpkgs
Something went wrong. Try again.
Nix
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642/* Functions for working with path values. */# See ./README.md for internal docs{ lib }:let
inherit (builtins) isString isPath split match typeOf storeDir ;
inherit (lib.lists) length head last genList elemAt all concatMap foldl' take drop ;
listHasPrefix = lib.lists.hasPrefix;
inherit (lib.strings) concatStringsSep substring ;
inherit (lib.asserts) assertMsg ;
inherit (lib.path.subpath) isValid ;
# Return the reason why a subpath is invalid, or `null` if it's valid subpathInvalidReason = value: if ! isString value then "The given value is of type ${builtins.typeOf value}, but a string was expected" else if value == "" then "The given string is empty" else if substring 0 1 value == "/" then "The given string \"${value}\" starts with a `/`, representing an absolute path" # We don't support ".." components, see ./path.md#parent-directory else if match "(.*/)?\\.\\.(/.*)?" value != null then "The given string \"${value}\" contains a `..` component, which is not allowed in subpaths" else null;
# Split and normalise a relative path string into its components. # Error for ".." components and doesn't include "." components splitRelPath = path: let # Split the string into its parts using regex for efficiency. This regex # matches patterns like "/", "/./", "/././", with arbitrarily many "/"s # together. These are the main special cases: # - Leading "./" gets split into a leading "." part # - Trailing "/." or "/" get split into a trailing "." or "" # part respectively # # These are the only cases where "." and "" parts can occur parts = split "/+(\\./+)*" path;
# `split` creates a list of 2 * k + 1 elements, containing the k + # 1 parts, interleaved with k matches where k is the number of # (non-overlapping) matches. This calculation here gets the number of parts # back from the list length # floor( (2 * k + 1) / 2 ) + 1 == floor( k + 1/2 ) + 1 == k + 1 partCount = length parts / 2 + 1;
# To assemble the final list of components we want to: # - Skip a potential leading ".", normalising "./foo" to "foo" # - Skip a potential trailing "." or "", normalising "foo/" and "foo/." to # "foo". See ./path.md#trailing-slashes skipStart = if head parts == "." then 1 else 0; skipEnd = if last parts == "." || last parts == "" then 1 else 0;
# We can now know the length of the result by removing the number of # skipped parts from the total number componentCount = partCount - skipEnd - skipStart;
in # Special case of a single "." path component. Such a case leaves a # componentCount of -1 due to the skipStart/skipEnd not verifying that # they don't refer to the same character if path == "." then []
# Generate the result list directly. This is more efficient than a # combination of `filter`, `init` and `tail`, because here we don't # allocate any intermediate lists else genList (index: # To get to the element we need to add the number of parts we skip and # multiply by two due to the interleaved layout of `parts` elemAt parts ((skipStart + index) * 2) ) componentCount;
# Join relative path components together joinRelPath = components: # Always return relative paths with `./` as a prefix (./path.md#leading-dots-for-relative-paths) "./" + # An empty string is not a valid relative path, so we need to return a `.` when we have no components (if components == [] then "." else concatStringsSep "/" components);
# Type: Path -> { root :: Path, components :: [ String ] } # # Deconstruct a path value type into: # - root: The filesystem root of the path, generally `/` # - components: All the path's components # # This is similar to `splitString "/" (toString path)` but safer # because it can distinguish different filesystem roots deconstructPath = let recurse = components: base: # If the parent of a path is the path itself, then it's a filesystem root if base == dirOf base then { root = base; inherit components; } else recurse ([ (baseNameOf base) ] ++ components) (dirOf base); in recurse [];
# The components of the store directory, typically [ "nix" "store" ] storeDirComponents = splitRelPath ("./" + storeDir); # The number of store directory components, typically 2 storeDirLength = length storeDirComponents;
# Type: [ String ] -> Bool # # Whether path components have a store path as a prefix, according to # https://nixos.org/manual/nix/stable/store/store-path.html#store-path. componentsHaveStorePathPrefix = components: # path starts with the store directory (typically /nix/store) listHasPrefix storeDirComponents components # is not the store directory itself, meaning there's at least one extra component && storeDirComponents != components # and the first component after the store directory has the expected format. # NOTE: We could change the hash regex to be [0-9a-df-np-sv-z], # because these are the actual ASCII characters used by Nix's base32 implementation, # but this is not fully specified, so let's tie this too much to the currently implemented concept of store paths. # Similar reasoning applies to the validity of the name part. # We care more about discerning store path-ness on realistic values. Making it airtight would be fragile and slow. && match ".{32}-.+" (elemAt components storeDirLength) != null;
in /* No rec! Add dependencies on this file at the top. */ {
/* Append a subpath string to a path.
Like `path + ("/" + string)` but safer, because it errors instead of returning potentially surprising results. More specifically, it checks that the first argument is a [path value type](https://nixos.org/manual/nix/stable/language/values.html#type-path"), and that the second argument is a [valid subpath string](#function-library-lib.path.subpath.isValid).
Laws:
- Not influenced by subpath [normalisation](#function-library-lib.path.subpath.normalise):
append p s == append p (subpath.normalise s)
Type: append :: Path -> String -> Path
Example: append /foo "bar/baz" => /foo/bar/baz
# subpaths don't need to be normalised append /foo "./bar//baz/./" => /foo/bar/baz
# can append to root directory append /. "foo/bar" => /foo/bar
# first argument needs to be a path value type append "/foo" "bar" => <error>
# second argument needs to be a valid subpath string append /foo /bar => <error> append /foo "" => <error> append /foo "/bar" => <error> append /foo "../bar" => <error> */ append = # The absolute path to append to path: # The subpath string to append subpath: assert assertMsg (isPath path) '' lib.path.append: The first argument is of type ${builtins.typeOf path}, but a path was expected''; assert assertMsg (isValid subpath) '' lib.path.append: Second argument is not a valid subpath string: ${subpathInvalidReason subpath}''; path + ("/" + subpath);
/* Whether the first path is a component-wise prefix of the second path.
Laws:
- `hasPrefix p q` is only true if [`q == append p s`](#function-library-lib.path.append) for some [subpath](#function-library-lib.path.subpath.isValid) `s`.
- `hasPrefix` is a [non-strict partial order](https://en.wikipedia.org/wiki/Partially_ordered_set#Non-strict_partial_order) over the set of all path values.
Type: hasPrefix :: Path -> Path -> Bool
Example: hasPrefix /foo /foo/bar => true hasPrefix /foo /foo => true hasPrefix /foo/bar /foo => false hasPrefix /. /foo => true */ hasPrefix = path1: assert assertMsg (isPath path1) "lib.path.hasPrefix: First argument is of type ${typeOf path1}, but a path was expected"; let path1Deconstructed = deconstructPath path1; in path2: assert assertMsg (isPath path2) "lib.path.hasPrefix: Second argument is of type ${typeOf path2}, but a path was expected"; let path2Deconstructed = deconstructPath path2; in assert assertMsg (path1Deconstructed.root == path2Deconstructed.root) '' lib.path.hasPrefix: Filesystem roots must be the same for both paths, but paths with different roots were given: first argument: "${toString path1}" with root "${toString path1Deconstructed.root}" second argument: "${toString path2}" with root "${toString path2Deconstructed.root}"''; take (length path1Deconstructed.components) path2Deconstructed.components == path1Deconstructed.components;
/* Remove the first path as a component-wise prefix from the second path. The result is a [normalised subpath string](#function-library-lib.path.subpath.normalise).
Laws:
- Inverts [`append`](#function-library-lib.path.append) for [normalised subpath string](#function-library-lib.path.subpath.normalise):
removePrefix p (append p s) == subpath.normalise s
Type: removePrefix :: Path -> Path -> String
Example: removePrefix /foo /foo/bar/baz => "./bar/baz" removePrefix /foo /foo => "./." removePrefix /foo/bar /foo => <error> removePrefix /. /foo => "./foo" */ removePrefix = path1: assert assertMsg (isPath path1) "lib.path.removePrefix: First argument is of type ${typeOf path1}, but a path was expected."; let path1Deconstructed = deconstructPath path1; path1Length = length path1Deconstructed.components; in path2: assert assertMsg (isPath path2) "lib.path.removePrefix: Second argument is of type ${typeOf path2}, but a path was expected."; let path2Deconstructed = deconstructPath path2; success = take path1Length path2Deconstructed.components == path1Deconstructed.components; components = if success then drop path1Length path2Deconstructed.components else throw '' lib.path.removePrefix: The first path argument "${toString path1}" is not a component-wise prefix of the second path argument "${toString path2}".''; in assert assertMsg (path1Deconstructed.root == path2Deconstructed.root) '' lib.path.removePrefix: Filesystem roots must be the same for both paths, but paths with different roots were given: first argument: "${toString path1}" with root "${toString path1Deconstructed.root}" second argument: "${toString path2}" with root "${toString path2Deconstructed.root}"''; joinRelPath components;
/* Split the filesystem root from a [path](https://nixos.org/manual/nix/stable/language/values.html#type-path). The result is an attribute set with these attributes: - `root`: The filesystem root of the path, meaning that this directory has no parent directory. - `subpath`: The [normalised subpath string](#function-library-lib.path.subpath.normalise) that when [appended](#function-library-lib.path.append) to `root` returns the original path.
Laws: - [Appending](#function-library-lib.path.append) the `root` and `subpath` gives the original path:
p == append (splitRoot p).root (splitRoot p).subpath
- Trying to get the parent directory of `root` using [`readDir`](https://nixos.org/manual/nix/stable/language/builtins.html#builtins-readDir) returns `root` itself:
dirOf (splitRoot p).root == (splitRoot p).root
Type: splitRoot :: Path -> { root :: Path, subpath :: String }
Example: splitRoot /foo/bar => { root = /.; subpath = "./foo/bar"; }
splitRoot /. => { root = /.; subpath = "./."; }
# Nix neutralises `..` path components for all path values automatically splitRoot /foo/../bar => { root = /.; subpath = "./bar"; }
splitRoot "/foo/bar" => <error> */ splitRoot = # The path to split the root off of path: assert assertMsg (isPath path) "lib.path.splitRoot: Argument is of type ${typeOf path}, but a path was expected"; let deconstructed = deconstructPath path; in { root = deconstructed.root; subpath = joinRelPath deconstructed.components; };
/* Whether a [path](https://nixos.org/manual/nix/stable/language/values.html#type-path) has a [store path](https://nixos.org/manual/nix/stable/store/store-path.html#store-path) as a prefix.
:::{.note} As with all functions of this `lib.path` library, it does not work on paths in strings, which is how you'd typically get store paths.
Instead, this function only handles path values themselves, which occur when Nix files in the store use relative path expressions. :::
Type: hasStorePathPrefix :: Path -> Bool
Example: # Subpaths of derivation outputs have a store path as a prefix hasStorePathPrefix /nix/store/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo/bar/baz => true
# The store directory itself is not a store path hasStorePathPrefix /nix/store => false
# Derivation outputs are store paths themselves hasStorePathPrefix /nix/store/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo => true
# Paths outside the Nix store don't have a store path prefix hasStorePathPrefix /home/user => false
# Not all paths under the Nix store are store paths hasStorePathPrefix /nix/store/.links/10gg8k3rmbw8p7gszarbk7qyd9jwxhcfq9i6s5i0qikx8alkk4hq => false
# Store derivations are also store paths themselves hasStorePathPrefix /nix/store/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo.drv => true */ hasStorePathPrefix = path: let deconstructed = deconstructPath path; in assert assertMsg (isPath path) "lib.path.hasStorePathPrefix: Argument is of type ${typeOf path}, but a path was expected"; assert assertMsg # This function likely breaks or needs adjustment if used with other filesystem roots, if they ever get implemented. # Let's try to error nicely in such a case, though it's unclear how an implementation would work even and whether this could be detected. # See also https://github.com/NixOS/nix/pull/6530#discussion_r1422843117 (deconstructed.root == /. && toString deconstructed.root == "/") "lib.path.hasStorePathPrefix: Argument has a filesystem root (${toString deconstructed.root}) that's not /, which is currently not supported."; componentsHaveStorePathPrefix deconstructed.components;
/* Whether a value is a valid subpath string.
A subpath string points to a specific file or directory within an absolute base directory. It is a stricter form of a relative path that excludes `..` components, since those could escape the base directory.
- The value is a string.
- The string is not empty.
- The string doesn't start with a `/`.
- The string doesn't contain any `..` path components.
Type: subpath.isValid :: String -> Bool
Example: # Not a string subpath.isValid null => false
# Empty string subpath.isValid "" => false
# Absolute path subpath.isValid "/foo" => false
# Contains a `..` path component subpath.isValid "../foo" => false
# Valid subpath subpath.isValid "foo/bar" => true
# Doesn't need to be normalised subpath.isValid "./foo//bar/" => true */ subpath.isValid = # The value to check value: subpathInvalidReason value == null;
/* Join subpath strings together using `/`, returning a normalised subpath string.
Like `concatStringsSep "/"` but safer, specifically:
- All elements must be [valid subpath strings](#function-library-lib.path.subpath.isValid).
- The result gets [normalised](#function-library-lib.path.subpath.normalise).
- The edge case of an empty list gets properly handled by returning the neutral subpath `"./."`.
Laws:
- Associativity:
subpath.join [ x (subpath.join [ y z ]) ] == subpath.join [ (subpath.join [ x y ]) z ]
- Identity - `"./."` is the neutral element for normalised paths:
subpath.join [ ] == "./." subpath.join [ (subpath.normalise p) "./." ] == subpath.normalise p subpath.join [ "./." (subpath.normalise p) ] == subpath.normalise p
- Normalisation - the result is [normalised](#function-library-lib.path.subpath.normalise):
subpath.join ps == subpath.normalise (subpath.join ps)
- For non-empty lists, the implementation is equivalent to [normalising](#function-library-lib.path.subpath.normalise) the result of `concatStringsSep "/"`. Note that the above laws can be derived from this one:
ps != [] -> subpath.join ps == subpath.normalise (concatStringsSep "/" ps)
Type: subpath.join :: [ String ] -> String
Example: subpath.join [ "foo" "bar/baz" ] => "./foo/bar/baz"
# normalise the result subpath.join [ "./foo" "." "bar//./baz/" ] => "./foo/bar/baz"
# passing an empty list results in the current directory subpath.join [ ] => "./."
# elements must be valid subpath strings subpath.join [ /foo ] => <error> subpath.join [ "" ] => <error> subpath.join [ "/foo" ] => <error> subpath.join [ "../foo" ] => <error> */ subpath.join = # The list of subpaths to join together subpaths: # Fast in case all paths are valid if all isValid subpaths then joinRelPath (concatMap splitRelPath subpaths) else # Otherwise we take our time to gather more info for a better error message # Strictly go through each path, throwing on the first invalid one # Tracks the list index in the fold accumulator foldl' (i: path: if isValid path then i + 1 else throw '' lib.path.subpath.join: Element at index ${toString i} is not a valid subpath string: ${subpathInvalidReason path}'' ) 0 subpaths;
/* Split [a subpath](#function-library-lib.path.subpath.isValid) into its path component strings. Throw an error if the subpath isn't valid. Note that the returned path components are also [valid subpath strings](#function-library-lib.path.subpath.isValid), though they are intentionally not [normalised](#function-library-lib.path.subpath.normalise).
Laws:
- Splitting a subpath into components and [joining](#function-library-lib.path.subpath.join) the components gives the same subpath but [normalised](#function-library-lib.path.subpath.normalise):
subpath.join (subpath.components s) == subpath.normalise s
Type: subpath.components :: String -> [ String ]
Example: subpath.components "." => [ ]
subpath.components "./foo//bar/./baz/" => [ "foo" "bar" "baz" ]
subpath.components "/foo" => <error> */ subpath.components = # The subpath string to split into components subpath: assert assertMsg (isValid subpath) '' lib.path.subpath.components: Argument is not a valid subpath string: ${subpathInvalidReason subpath}''; splitRelPath subpath;
/* Normalise a subpath. Throw an error if the subpath isn't [valid](#function-library-lib.path.subpath.isValid).
- Limit repeating `/` to a single one.
- Remove redundant `.` components.
- Remove trailing `/` and `/.`.
- Add leading `./`.
Laws:
- Idempotency - normalising multiple times gives the same result:
subpath.normalise (subpath.normalise p) == subpath.normalise p
- Uniqueness - there's only a single normalisation for the paths that lead to the same file system node:
subpath.normalise p != subpath.normalise q -> $(realpath ${p}) != $(realpath ${q})
- Don't change the result when [appended](#function-library-lib.path.append) to a Nix path value:
append base p == append base (subpath.normalise p)
- Don't change the path according to `realpath`:
$(realpath ${p}) == $(realpath ${subpath.normalise p})
- Only error on [invalid subpaths](#function-library-lib.path.subpath.isValid):
builtins.tryEval (subpath.normalise p)).success == subpath.isValid p
Type: subpath.normalise :: String -> String
Example: # limit repeating `/` to a single one subpath.normalise "foo//bar" => "./foo/bar"
# remove redundant `.` components subpath.normalise "foo/./bar" => "./foo/bar"
# add leading `./` subpath.normalise "foo/bar" => "./foo/bar"
# remove trailing `/` subpath.normalise "foo/bar/" => "./foo/bar"
# remove trailing `/.` subpath.normalise "foo/bar/." => "./foo/bar"
# Return the current directory as `./.` subpath.normalise "." => "./."
# error on `..` path components subpath.normalise "foo/../bar" => <error>
# error on empty string subpath.normalise "" => <error>
# error on absolute path subpath.normalise "/foo" => <error> */ subpath.normalise = # The subpath string to normalise subpath: assert assertMsg (isValid subpath) '' lib.path.subpath.normalise: Argument is not a valid subpath string: ${subpathInvalidReason subpath}''; joinRelPath (splitRelPath subpath);
}