about summary refs log tree commit diff
path: root/tvix/nix-compat (follow)
AgeCommit message (Collapse)AuthorFilesLines
7 days r/8174 fix(tvix/nix-compat): async nar reader requires wire featureFlorian Klink2-2/+2
It uses it internally. Change-Id: I8cb8fc9f567260d57f3a203407333d83beddf537 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11719 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-08 r/8095 refactor(nix-compat/nar/reader): reuse prev_name allocationedef6-48/+71
We reuse the prev_name allocation for Entry, instead of allocating and returning a separate Vec. We encode the `prev_name: None` case as an empty vector, since we don't allow empty names anyway, and the sorting is equivalent. Change-Id: I975b37ff873805f5ff099bc82128706891052247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11607 Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8094 feat(nix-compat/wire/bytes): read bytes into an existing bufferedef1-1/+60
For small bytestrings (like NAR names), we can read into a preallocated fixed-size buffer, instead of allocating a Vec every time. Change-Id: Id8da9e9cea99c814361230c0ec02606b731c79a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11606 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-08 r/8088 refactor(nix-compat/wire/bytes): style fixesedef1-13/+10
Change-Id: I65c3c43df83e0c364a4b7f1f3054c5b676bd07d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11605 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8087 fix(nix-compat/store_path): use Box<str>edef1-8/+5
We don't actually build up names in place here, so we don't need a capacity field. Saves 8 bytes. Change-Id: Icb01b45561e28fd525b726612f56d4640bc834c7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11604 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8086 feat(nix-compat/wire/bytes/reader): support buffered readingedef2-4/+194
If our underlying reader supports AsyncBufRead, then we can too. Change-Id: If4b948c983400ca591c1c475bbcf7dc00d562040 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11545 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8085 fix(nix-compat/wire/bytes/reader): always read trailer tag dataedef3-16/+73
Even if we have an aligned (or zero) size (and thus have no padding), we may still have a non-padding trailer. Since an exact read of only the user data bytes must always read the trailer bytes as well, this has to happen for aligned sizes too. For zero-sized trailers, we solve this by reading immediately, since no read is required of the user at all. user_len thus becomes a NonZeroU64, and the definition of "body data" is adjusted to always exclude the final block. Change-Id: I2c6bb51270fb22ad72a65076a3468673ab56ef68 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11597 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8084 fix(nix-compat/wire/bytes/reader): drop allow(dead_code)edef1-1/+0
We're using this in the NAR reader now. Change-Id: I28f17b1ccedd62ffcaf2fa32b517f16bcd036d94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11603 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-08 r/8083 refactor(nix-compat/wire/bytes/reader): drop prev_fillededef1-2/+1
We specifically structured the code this way to allow using `this.filled` as-is, so we should use it. Change-Id: I7e11bddceb4d5f37b1dd4c453a9d53b85fc1f6c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11602 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-04 r/8075 feat(nix-compat/nar/reader): async supportedef6-1/+566
This is a first cut at the async NAR reader, with some rough edges. Poisoning is left unimplemented for now, pending future work. Change-Id: Ifaafe0581a5e0e165a13357b909fb441f7bd8bab Reviewed-on: https://cl.tvl.fyi/c/depot/+/11524 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-04 r/8074 feat(nix-compat/wire/bytes/reader): parametrise on trailer tagedef3-10/+31
This allows using BytesReader with a custom tag, eg the closing parens for the NAR reader. No public constructor is provided for custom-tagged readers, since this feature isn't public API. Change-Id: I82e73d064edc4b6783ead1d6fe46a5b35f45c844 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11543 Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-02 r/8068 chore(nix-compat): Document daemon protocol versionBrian Olsen1-0/+21
This documents which Nix version uses which daemon protocol version so that any implementor of the protocol can make informed choices about how far back to support legacy. Change-Id: If2fc11702f7f731816d51928efc9ea9d939e1c00 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11579 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-02 r/8064 refactor(nix-compat): derivation_or_fod_hash -> hash_derivation_moduloFlorian Klink2-37/+40
There's no need for us to come up with our own names for this. Also update the comments/docstrings a bit, and inline the intermediate hash_derivation_modulo calculation. Change-Id: I09dab8ffe1ebfb6601841e98119eee4ff25d8f39 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11578 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-01 r/8053 fix(nix-compat/store_path): permit store paths with dots againedef1-10/+8
Nix 2.4 accidentally permitted this behaviour, but the revert came too late to beat Hyrum's law. It is now considered permissible. Link: https://github.com/NixOS/nix/pull/9867 Change-Id: Ie97777af6765fe1c12c8aa593afe1c9b69125775 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11553 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-30 r/8043 fix(tvix/nix-compat/wire): RangeInclusive<usize> for memory size boundsedef2-32/+30
u64 is an inappropriate type for something memory-sized, and most callers end up with off-by-ones when using `..` rather than `..=`, including the tests for the module itself. Change-Id: If3b7bea27eb0a6c01e0a5d7e64966acbbb664268 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11550 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-30 r/8042 refactor(nix-compat/wire): drop primitive functionsedef5-109/+28
These may as well be inlined, and hardly need tests, since they just alias AsyncReadExt::read_u64_le / AsyncWriteExt::write_u64_le. Boolean reading is worth making explicit, since callers may differ on how they want to handle values other than 0 and 1. Boolean writing simplifies to `.write_u64_le(x as u64)`, which is also fine to inline. Change-Id: Ief9722fe886688693feb924ff0306b5bc68dd7a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11549 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-30 r/8041 refactor(nix-compat/wire/bytes): branchless padding computationedef1-7/+8
Change-Id: Ie07c2516a485c78afa6f9a3c8256e9708c4c42c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11548 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-30 r/8040 fix(tvix/nix-compat/wire/bytes): return UnexpectedEof for unexpected EOFedef1-4/+1
Change-Id: I8cb79f93742cc3a9088701bc8c2795de893eb6a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11547 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-29 r/8034 refactor(nix-compat/wire/bytes/reader): move size reading to async ctoredef1-137/+65
with_size only existed to allow separating the phases of reading size and reading data, and similarly the Size state only existed to allow folding size reading into poll_read. Neither of these are necessary if we make the constructor async, and handle the size reading there. As a bonus, this makes BytesReader::len sensibly typed, and permits implementing is_empty, as Clippy demands of us. Change-Id: I72173ec06d60b7998d16a3ecfc5e6ac5424bbed3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11542 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-29 r/8032 refactor(nix-compat/wire/bytes/reader): use fn for PhantomDataedef1-1/+1
We don't actually hold a Tag, we just want to bind the type. Change-Id: Ida67c026f852ed54c3f18df914cf5c31e6227fd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11541 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-29 r/8031 refactor(tvix/nix-compat/wire/bytes): use RangeInclusive for limitsedef1-14/+7
The (min, max) pair is already a RangeInclusive in essence, so we might as well represent it that way. Change-Id: I2f67f3c47dc36b87e866ff5dc2e0cd28f01fbb04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11540 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-29 r/8030 refactor(nix-compat/wire/bytes): fold TrailerReader into BytesReaderedef2-91/+41
The TrailerReader has no purpose separate from BytesReader, and the code gets a fair bit simpler this way. EOF handling is simplified, since we just rely on the implicit behaviour of the existing case. Change-Id: Id9b9f022c7c89fbc47968a96032fc43553af8290 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11539 Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-29 r/8029 feat(nix-compat/wire/bytes/reader): expose the remaining data lengthedef2-0/+31
The API is a bit odd here, because we don't have a distinct type for a known-length reader. Change-Id: I4a1dd07fbed0a400004dbe4aa2095c51898ad3bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11538 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name>
2024-04-29 r/8028 refactor(nix-compat/nar/reader): always enable poisoning *API*edef1-64/+66
The poisoning API is now always available, whether debug_assertions is enabled or not. When debug assertions are not enabled, it is equivalent to a unit struct, and is always considered ready and unpoisoned. Change-Id: I950237f4816d480330d9acab32661ed4f1663049 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11525 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-26 r/8017 chore(nix-compat/nar/wire): space Tag::Buf docs properlyedef1-0/+1
Change-Id: If15b14b5328b9e7f85ea3fa7644d327dfb50e1a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11523 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-26 r/8016 feat(nix-compat/wire/bytes/reader): split out reading the trailer blockedef1-56/+134
We separate ingesting the trailer block into a Future of its own, parametrised on the specifics of the trailer pattern. This is intended to be used for future work on an async NAR reader, which needs to read a terminating parenthesis as well as the regular padding. Thanks to @griff for suggesting separating the ingestion into its own Future. Change-Id: I36c2503baa67937046a63e9bf0cfc38201394025 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11522 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-26 r/8015 refactor(nix-compat/wire/bytes/reader): drop random whitespaceedef1-2/+2
Change-Id: Ic683eab435576acc8f7e03f5684767ffa468851a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11521 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-26 r/8014 refactor(nix-compat/wire/bytes/reader): drop Pin::get_mutedef2-4/+4
Pin<&mut T> is DerefMut when T: Unpin, so we don't actually need to explicitly call get_mut. Change-Id: Iaa312ec49c87100010e09c94f319e57e31da0cd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11520 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-25 r/8013 refactor(nix-compat/wire): move BytesPacketPosition into writeredef2-20/+17
We don't use it in the reader anymore. Change-Id: I98fe204a747711464e9e7ca17df06fa9854eb344 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11519 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-25 r/8012 fix(nix-compat/bytes): make BytesReader less hazardousedef2-178/+317
We now *never* return the final bytes until we've read the padding in full, so read_exact is safe to use. This is implemented by TrailerReader, which splits the phases of reading (and validating) the final 8-byte block, and providing the contained payload bytes to the caller. Change-Id: I0d05946a8af9c260a18d71d2b763ba7a68b3c27f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11518 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-25 r/8011 feat(nix-compat/wire/bytes): allow specifying a pre-read sizeedef1-0/+41
Change-Id: I9c94239c308cfbc2e6dae871ba77fb33507433c9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11517 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-25 r/8010 refactor(nix-compat/wire/bytes): drop pin_project, clean upedef1-63/+57
We already require R: Unpin in the constructor, so there's not much use to pin projection. Change-Id: Ia7bf734dc3aa86ffa6d1d5de778939baa9676bb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11516 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-25 r/8009 refactor(nix-compat/wire/bytes): don't parametrise on RangeBoundsedef1-12/+20
This is semantically a RangeInclusive, since we can only have 0..=u64::MAX at most, and monomorphising on the bounds doesn't buy us anything. Change-Id: Ib601d7fd77d703d6c8c5ec27ac9e67bb122ce1c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11515 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-25 r/8007 test(tvix/nix-compat): add debug assertions for nar readertcmal2-84/+426
Adds debug assertions to ensure that the reader's variants are upheld. If any of the following happens, then the currently in use reader must be abandoned: * A directory or file reader encounters an error * A directory or file reader is dropped before being fully read from Additionally, a directory reader must not be read from again after it has returned None. These checks are only used when debug_assertions are on, so vanish in release mode. Resolves two TODO items added by edef Change-Id: I27bd9643a632798db5351957506c166b9bd5ca4e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11508 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Autosubmit: Aria Shrimpton <me@aria.rip> Tested-by: BuildkiteCI
2024-04-24 r/8004 docs(tvix/nix-compat/wire/bytes/reader): fix typoFlorian Klink1-1/+1
This is a ReadBuf, not a BufRead. Change-Id: Ie80e894f4b24b77cdd60409ddfaa66dae0ffeec9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11511 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-04-24 r/8003 docs(tvix/nix-compat): add missing reference to BytesReaderFlorian Klink1-2/+2
Change-Id: Ideed83d191b55e131720e598b7591e8375a26cfd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11510 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de>
2024-04-19 r/7975 chore(nix-compat): remove unused test-case and test-generatorFlorian Klink1-7/+0
This is especially nice, as it allows us getting rid of the other versions of proc-macro2 and quote. Change-Id: I9fdd012ee6c0ded3e18ec30504b83ac2032d1390 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11474 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7974 chore(nix-compat/drv): migrate from test_{case,resources} to rstestFlorian Klink1-76/+88
Also, replace the useless read_file helper with std::fs::read. Change-Id: I704ab42d1ffb9089716533125ff390b7441a8591 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11473 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7973 chore(nix-compat/derivation/parser): migrate from test_case to rstestFlorian Klink1-23/+37
Change-Id: I7c29e7d9e2b9cd66483ed44a9f2b3d291061f81d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11472 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7972 chore(nix-compat/nixhash): migrate from test_case to rstestFlorian Klink1-12/+14
Change-Id: I053f2c7fc9fdf59de5d8697eab32644b860e40be Reviewed-on: https://cl.tvl.fyi/c/depot/+/11471 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7971 chore(nix-compat/aterm): migrate from test_case to rstestFlorian Klink2-22/+30
Change-Id: If05fa3d0f41d45e3574e0c5ddff406cb33dc52f4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11470 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-04-19 r/7970 chore(tvix/nix-compat/store_path): migrate from test_case to rstestFlorian Klink1-16/+20
Change-Id: Ic466a27d61b95ca4d297abd6eb976c083e8b40af Reviewed-on: https://cl.tvl.fyi/c/depot/+/11469 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7969 chore(nix-compat/nixbase32): migrate from test_case to rstestFlorian Klink1-14/+17
Change-Id: I9ed5e728ff0706a37d0a238210a8dd23eb306033 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11468 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-19 r/7968 chore(nix-compat/narinfo): migrate from test_case to rstestFlorian Klink2-33/+42
Change-Id: Ibd3cd1bc0d9deaf018a66748cea56af51451703c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11467 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-18 r/7955 feat(tvix/nix-compat/nix_daemon/version): add ProtocolVersionFlorian Klink4-22/+156
This provides a nice wrapper struct to deal with versions. Change-Id: I6acc03bc9f8d84a0583196073b52776c45d3fe92 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11454 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-18 r/7954 feat(tvix/nix-compat): enable more features by defaultFlorian Klink1-0/+3
There's no point in not enabling these, and rust-analyzer is quite useless too. nix-compat users outside of here can still explicitly disable default features if they don't want to pull in these features. Change-Id: I8f2f24e7734ad152d1733bc3421098fc91af290a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11455 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-13 r/7911 feat(tvix/nix-compat/path_info): derive Hash for ExportedPathInfoIlan Joselevich1-1/+1
Change-Id: Icd2431f27043f120eee1d156cfcdc9bc85519ead Reviewed-on: https://cl.tvl.fyi/c/depot/+/11412 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7905 feat(tvix/nix-compat/wire/bytes): add #[inline] for ensure_* funcsFlorian Klink2-0/+2
While the compiler might realize it's a good idea to inline these, especially considering this is in the same crate [1], it doesn't hurt to add that annotation too. Suggested in cl/11385. [1]: https://nnethercote.github.io/perf-book/inlining.html Change-Id: If679fce0e1aab9ec681ba60f689d386d6fd92d61 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11403 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-04-13 r/7898 refactor(tvix/nix-compat): move nar writer to tokioFlorian Klink3-58/+52
There's little reason to keep the nar writer using Async{Read,Write} traits from futures, while everything else async in tvix (and nix-compat) uses tokio. Change-Id: I8cd1efcd0dd5bb76471de997603c7b701a5095de Reviewed-on: https://cl.tvl.fyi/c/depot/+/11391 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Brian Olsen <me@griff.name>
2024-04-13 r/7897 refactor(tvix/nix-compat): move worker_protocol into nix_daemon modFlorian Klink4-30/+32
This doesn't have much to do with the plain "wire" format, it's merely one user of it. Also, use the more "public" `wire::` API to read/write bytes, strings, bools and u64s. Change-Id: I98dddcc3004dfde7a0c009958fe84a840f77b188 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11390 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Brian Olsen <me@griff.name>