Comments on: v.sh part 2: Ada utilities http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/ Fri, 21 Aug 2020 17:09:10 +0000 http://polimedia.us hourly 1 By: bvt http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-177 bvt Thu, 02 Apr 2020 19:06:06 +0000 http://bvt-trace.net/?p=69#comment-177 <blockquote> Any thoughts on what if anything it should do on encountering file names with embedded whitespace (which on unix can even include newlines)? </blockquote> In the context of vpatches I don't see a good solution, but for the sake of honestly I considered aborting vdiff process or at least reporting that the vpatch is borked to stderr, as most likely no V presser/patcher would be able to do anything with it.

Any thoughts on what if anything it should do on encountering file names with embedded whitespace (which on unix can even include newlines)?

In the context of vpatches I don't see a good solution, but for the sake of honestly I considered aborting vdiff process or at least reporting that the vpatch is borked to stderr, as most likely no V presser/patcher would be able to do anything with it.

]]>
By: Jacob Welsh http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-176 Jacob Welsh Thu, 02 Apr 2020 15:14:00 +0000 http://bvt-trace.net/?p=69#comment-176 > shipping vpatch signatures without a vpatch à-la trinque becomes impossible Although that was for a genesis, where the diff hunks are fully determined anyway. I'd say it makes a good example of the importance of choosing optionality wisely, because putting it in one place closes it off in another. > it does have some crap that should go. Ah yes, the "quoting is good mmkay, so let's do it, sometimes, if we feel like it". Any thoughts on what if anything it should do on encountering file names with embedded whitespace (which on unix can even include newlines)? > shipping vpatch signatures without a vpatch à-la trinque becomes impossible

Although that was for a genesis, where the diff hunks are fully determined anyway.

I'd say it makes a good example of the importance of choosing optionality wisely, because putting it in one place closes it off in another.

> it does have some crap that should go.

Ah yes, the "quoting is good mmkay, so let's do it, sometimes, if we feel like it".

Any thoughts on what if anything it should do on encountering file names with embedded whitespace (which on unix can even include newlines)?

]]>
By: bvt http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-175 bvt Thu, 02 Apr 2020 09:43:58 +0000 http://bvt-trace.net/?p=69#comment-175 I don't have a reference specifically, but the clue for me was reading somewhere that phf took care that vdiff would reproduce all TRB vpatches accurately from the pressed trees. If you think about it, if vdiffs are allowed to produce differing outputs, shipping vpatch signatures without a vpatch <a href="http://trinque.org/2018/11/27/cuntoo-bootstrapper/#selection-55.96-55.182" rel="nofollow">à-la trinque</a> becomes impossible. To still have this possibility without cementing the diff algorithm, signature would have to cover only the vpatch headers, with filenames and input and output hashes, leaving the hunk bodies out. And, well, the "standard utility" is also implemented in some ~1000 LoC even with busybox. This is not to say that vdiff could not use some improvement: it does have some <a href="http://btcbase.org/patches/vtools_fixes_rootdir_files/tree/vtools/src/util.c#L91" rel="nofollow">crap that should go</a>. I don't have a reference specifically, but the clue for me was reading somewhere that phf took care that vdiff would reproduce all TRB vpatches accurately from the pressed trees. If you think about it, if vdiffs are allowed to produce differing outputs, shipping vpatch signatures without a vpatch à-la trinque becomes impossible. To still have this possibility without cementing the diff algorithm, signature would have to cover only the vpatch headers, with filenames and input and output hashes, leaving the hunk bodies out.

And, well, the "standard utility" is also implemented in some ~1000 LoC even with busybox. This is not to say that vdiff could not use some improvement: it does have some crap that should go.

]]>
By: Jacob Welsh http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-174 Jacob Welsh Thu, 02 Apr 2020 04:06:03 +0000 http://bvt-trace.net/?p=69#comment-174 Oh I see. The exact pattern looks to be <a href="http://btcbase.org/patches/vtools_fixes_rootdir_files/tree/vtools/src/vpatch.adb#L654" rel="nofollow">"^diff "</a>. I can't say I like the idea of needing a 2k+ line GNUistic vdiff compared to <100 lines of awk on top of a standard utility, or this bitcoin-like situation where "the code is the spec and realistically there can be only one implementation", or the idea that all vdiffs must produce the same output anyway as that closes off improvement of the algorithm. Do you have a ref handy for why identical output is important? I recall complaints about locale-dependent file sort order from the logs but not beyond that. Anyway, thank you for the caution and I will probably do as you suggest, as this doesn't seem like a battle I can or need to win. Oh I see. The exact pattern looks to be "^diff ".

I can't say I like the idea of needing a 2k+ line GNUistic vdiff compared to <100 lines of awk on top of a standard utility, or this bitcoin-like situation where "the code is the spec and realistically there can be only one implementation", or the idea that all vdiffs must produce the same output anyway as that closes off improvement of the algorithm. Do you have a ref handy for why identical output is important? I recall complaints about locale-dependent file sort order from the logs but not beyond that. Anyway, thank you for the caution and I will probably do as you suggest, as this doesn't seem like a battle I can or need to win.

]]>
By: bvt http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-172 bvt Wed, 01 Apr 2020 20:59:39 +0000 http://bvt-trace.net/?p=69#comment-172 Yes, it's nice, however it's phf who came up with this: vpatch utility won't detect a header unless it is preceded by a line that matches "^diff -uNr", and this is what I carried over to the rest of the v.sh components -- in the end, vpatch is the ultimate downstream consumer. Also, vdiff produces this header unconditionally. IMO if you want to get rid of Ada dependency, it's better to move vdiff to keksum code instead of relying on whatever diff busybox has -- also because if you diff two trees, it's not guaranteed that phf's vdiff and busybox+wrapper will produce identical hunks inside the vpatch. Yes, it's nice, however it's phf who came up with this: vpatch utility won't detect a header unless it is preceded by a line that matches "^diff -uNr", and this is what I carried over to the rest of the v.sh components -- in the end, vpatch is the ultimate downstream consumer. Also, vdiff produces this header unconditionally.

IMO if you want to get rid of Ada dependency, it's better to move vdiff to keksum code instead of relying on whatever diff busybox has -- also because if you diff two trees, it's not guaranteed that phf's vdiff and busybox+wrapper will produce identical hunks inside the vpatch.

]]>
By: Jacob Welsh http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-171 Jacob Welsh Wed, 01 Apr 2020 18:54:56 +0000 http://bvt-trace.net/?p=69#comment-171 > Upon detecting the "diff -uNr" in the colunm 0 of a line, read input and output information of a hunk. This wasn't previously a required part of the unified diff format, and at least busybox diff doesn't include it (<a href="http://fixpoint.welshcomputing.com/2020/adventures-in-the-forest-of-v/?b=III.&e=#select" rel="nofollow">see also</a>). It also means one might need to be more careful in any commentary at the top of a patch file. Still, I'm thinking this is the saner way to go since it simplifies any tool that needs to parse patches while complicating only theoretical vdiffs that don't conform; nicely anti-Postel if we can afford it. > Upon detecting the "diff -uNr" in the colunm 0 of a line, read input and output information of a hunk.

This wasn't previously a required part of the unified diff format, and at least busybox diff doesn't include it (see also). It also means one might need to be more careful in any commentary at the top of a patch file.

Still, I'm thinking this is the saner way to go since it simplifies any tool that needs to parse patches while complicating only theoretical vdiffs that don't conform; nicely anti-Postel if we can afford it.

]]>
By: TMSR OS, February 2020 Statement « Dorion Mode http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-158 TMSR OS, February 2020 Statement « Dorion Mode Mon, 09 Mar 2020 17:59:52 +0000 http://bvt-trace.net/?p=69#comment-158 [...] his awk + posix shell + Ada Vtron in a 3 part series (1, 2, [...] [...] his awk + posix shell + Ada Vtron in a 3 part series (1, 2, [...]

]]>
By: Diana Coman http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-133 Diana Coman Thu, 20 Feb 2020 19:38:37 +0000 http://bvt-trace.net/?p=69#comment-133 I kept thinking this over and despite my initial uneasiness, I think you are actually right in that a vpatch with *all* hunks creating files is indeed a genesis since it does not/can not properly speaking have an antecedent as it doesn't require/rely on anything being present. I kept thinking this over and despite my initial uneasiness, I think you are actually right in that a vpatch with *all* hunks creating files is indeed a genesis since it does not/can not properly speaking have an antecedent as it doesn't require/rely on anything being present.

]]>
By: bvt http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-132 bvt Mon, 17 Feb 2020 21:46:20 +0000 http://bvt-trace.net/?p=69#comment-132 @Diana Coman: If you consider a case of two vpatches that only create files, a genesis by intention (which V can't know) and a vpatch that adds manifest, those two vpatches can either conflict (if they create files with the same name) or commute (if there is no intersection between the sets of created names). How would a ~stateless system of V know which one is the genesis, and which one is just a leaf with manifest? AFAIK in V as described by Ben Vulpes, these will be ordered in flow somehow, and you would be able to press genesis, genesis+manifest OR manifest, manifest+genesis, depending on the exact flow the tool will give you; in v.sh, pressing one of them *will not* pull in the other, however you can select multiple leafs for pressing, not just one, so you can press any combination of them. If you consider a case of more vpatches, then genesis is not special vpatch in any way (so I guess Is_Genesis is rather misleading name) - it just happens that all other vpatches have this one vpatch - genesis - as their antecedent, while not all vpatches have the vpatch with manifest in their antecedent set. @Diana Coman:
If you consider a case of two vpatches that only create files, a genesis by intention (which V can't know) and a vpatch that adds manifest, those two vpatches can either conflict (if they create files with the same name) or commute (if there is no intersection between the sets of created names). How would a ~stateless system of V know which one is the genesis, and which one is just a leaf with manifest? AFAIK in V as described by Ben Vulpes, these will be ordered in flow somehow, and you would be able to press genesis, genesis+manifest OR manifest, manifest+genesis, depending on the exact flow the tool will give you; in v.sh, pressing one of them *will not* pull in the other, however you can select multiple leafs for pressing, not just one, so you can press any combination of them.

If you consider a case of more vpatches, then genesis is not special vpatch in any way (so I guess Is_Genesis is rather misleading name) - it just happens that all other vpatches have this one vpatch - genesis - as their antecedent, while not all vpatches have the vpatch with manifest in their antecedent set.

]]>
By: Diana Coman http://bvt-trace.net/2020/02/vsh-part-2-ada-utilities/#comment-131 Diana Coman Mon, 17 Feb 2020 18:57:56 +0000 http://bvt-trace.net/?p=69#comment-131 <blockquote> A vpatch is genesis if all its hunks are file creations. </blockquote> Hmm, this risks to misfire for instance on older trees that have the manifest added at some point rather than baked in from the start. Arguably they are due for a regrind anyway but the above definition (and its implicit reliance on the manifest file mechanism) sits rather uncomfortably with me. To my mind a vpatch is genesis if it doesn't have any ancestors - would this be all that harder to use at this specific point?

A vpatch is genesis if all its hunks are file creations.

Hmm, this risks to misfire for instance on older trees that have the manifest added at some point rather than baked in from the start. Arguably they are due for a regrind anyway but the above definition (and its implicit reliance on the manifest file mechanism) sits rather uncomfortably with me. To my mind a vpatch is genesis if it doesn't have any ancestors - would this be all that harder to use at this specific point?

]]>