As I mentioned before, I have written a V presser before starting v.sh, and not mentioning a word about it was a kind of debt on my side. So this article will provide a retrospective on the vy.scm, as it was called. I will not focus so much on the internals of the tool itself, but rather share some thoughts about its development and the implementation language.
I started writing vy.scm in ~2018 shortly before joining the TMSR, as the first tool to actually get access to the TRB sources in the pressed form, from practical point of view, and in general because I appreciated the simplicity and elegance of V. At that point, I was working through Starting Forth/Thinking Forth textbooks1, and really liked one aspect of Forth: because Forth is so simple to implement, everyone has his own implementation tuned to ones needs. This, however, makes code sharing impossible - in spite of the ANSI standard, there are always enough incompatibilities, for example in the dictionary implementation, so its the ideas that are shared instead; the developers interested in actually putting them to use can fit these ideas into their Forth as they wish.
This was the underlying implementation philosophy for vy.scm. Instead of looking at how v.pl/v.py worked inside, I looked at the vtree structure on the file system, looked at vpatches, read the description of V in Ben Vulpes' blog, and implemented it by relying only on the aforementioned sources2. This necessarily meant that the functionality that others may have relied on was dropped, as the general expectation was that everyone who cares to use V would implement the tools with all the functionality that he wishes to have. Upon some forced reflection, this approach does not strike me as particularly sane.
What I was blind to when starting writing vy.scm was that while implementing a simplest presser is a day of work, the time necessary to implement an actually useful program is much bigger, at least for me. So going too far with this development approach too far means that anyone who wishes to start using V, must now be stuck for a few weeks figuring out how to implement some basic piece of software anew. IMO this stresses the need of standardization: at the time of starting a blog, I did not even consider an option of a static blog, or any other platform. I knew that the standard blog engine was MP-WP, and this is what was there to use, it was not even a matter of "wanted" or "needed". Among pressers, no implementation has received a blessing that strong.
Another aspect of vy.scm implementation is the language: while "scm" file extension hints that it was implemented in Scheme, this is not exactly true. I implemented it in Scheme Shell3, in which I was interested because it looked like a viable language for reliable scripting: it had subprocess management facilities that are below (but close to) a pain limit that distinguished a shell from a programming language, it had the data structures that shell lacked, it had better scoping rules - what's not to like?
Not to like, as it turned out, were: an implementation of a size worthy of a full-scale language (~4Mb worth of sources, IIRC) and usability of a scripting language: there is a upper limit to memory allocated by scsh, that can be reliably changed only by re-bootstrapping the scsh image from its host Scheme48, it has no copy-on-write for strings, meaning that I had to implement an interning table to take care of the vpatch headers4. Another issue is that the original (i.e. working) scsh is 32-bit only, and runs on the old Scheme48. And so while Scheme48 was updated to 64-bit version in meantime, scsh was verschlimmbessert into working-but-not-quite state: it boots to the prompt, however some functions present in the original version are not exported, and these minor incompatibilities are all over the system. The practical consequence is that you're stuck with one of the two incompatible versions of system, depending on what versions was packed by the maintainers, or a mess on your hands that's too big to fix. So in the end, I kept vy.scm only on the "main battle tank^W^Wmachine", and used vk.pl on all remote servers.
Some of my conclusions out of this experience:
- Standardization: saves everyone resources that are best spent on something more useful. I can see why not aiming for it can kill organizations.
- Fits-in-head: the sources of scsh were too big for me to fix quickly, while scsh was not good enough to continue using. This was one of the motivations for starting v.sh.
- Scripting: it is better to use shit-and-sticks that Unix provides (because you're stuck with it anyway), than go for exotic tools with unmaintainable and unmaintained runtimes. IMO the programming language to kill the shell is still not there.
The sources for those interested (archaeologists):
curl http://bvt-trace.net/vpatches/vy-genesis.vpatch > vy-genesis.vpatch
curl http://bvt-trace.net/vpatches/vy-genesis.vpatch.bvt.sig > vy-genesis.vpatch.bvt.sig
curl http://bvt-trace.net/vpatches/vy-2.vpatch > vy-2.vpatch
curl http://bvt-trace.net/vpatches/vy-2.vpatch.bvt.sig > vy-2.vpatch.bvt.sig
- Thinking Forth is a nice read independent of the programming language you use. [↩]
- I did grep out the gpg invocations from v.pl. [↩]
- A modified R5RS Scheme written as an extension/modification of Scheme48. [↩]
- I was blind at the time to the option of just (intern)'ing all of them. I was aware that this option exists, but I found it unclean and unappealing at that time. Looking back, I think it was wrong, and I should have taken the "unclean" option: this would saved a small but non-negligible amount of time and lines of code. [↩]
Very interesting, and IMHO makes the idea of polishing off a proper Ada Scheme more appealing.
Seems like there is no hash verification though (just as there wasn't in my orig. pythonistic prototype Vtron) ?
Hear, hear!
The thing with standardization is that there has to be someone owning it - aka in practical terms it can only be done by an authority. Lacking that, it's pretty much a matter of "better stick to what shit you are already stuck with, anyway" indeed.
@Stanislav:
Right, but IMO it would still need scsh-like piping to be useful on POSIX platforms, which has its implications: e.g. scsh has non-lispy reader: case-sensitive, and with "." as a symbol and not as a cons pair constructor. Re hash verification, phf's vpatch verifies the hash of the input and output files, no extra ksum invocations are necessary.
@Diana:
Fully agree with both statements.
@bvt:
Theoretically -- would not need pipe, if a native (i.e. in Scheme) sig verifier were added.
Yes, my "EACH and every single individual dork" is rather "rhetoric exageration" as driven by frustration than a factual statement. You are right however in your (inferred) intuition that had the donkey substrate not given out of its own meanwhile, we'd be counting the sheep and summing up experiences to produce (mandatory) standards just about now.
Otherwise, this "better tool" thing is a fool's errand. There are "available" in the very limited sense here discussed uncounted (because uncountable) multitudes of such non-tools, bees carved out of birds, maimed tanks masquerading as machine guns (always inadequately deployed, too, for bonus horror). They are not things. A "scheme" replacement for bash is not a thing, not anymore than any other stupid idea of rambunctious boyhood, the socks with hoodies and their ilk. You are welcome to repeat the experience as many times as you can carry, but it'll always come out to the same way : doing just short of enough in evidently too much space, with obviously too much weight, by clearly too much money. Hitlerism, in a word, something which by well documented experience can sink organisations also, and just as well.
As to the shit-and-sticks, the point was made (and the attempt to bury such toxic facts followed immediately). Nevertheless, the point stands : A + epsilon is not a simplification over A alone, even if "I could just dwell in the epsilon my whole life and pretend A doesn't exist" is a solution "that seems to work okay" for... again... retarded bois. Moreover, secondarily but not to be entirely omitted : those "shit and sticks" work way the fuck better (in actual, lived practice) than anything else "available" (which it isn't, even though it may "convincingly" pretend to be). Always "fixing" the parts that didn't need any fixing at all is how the useless bois club stays useless over time, how its shit is always "inexplicably" late and so on. Forever, or at least that'd be their hope.
Well, if you look at all the things that were historically proposed as "replacements", all of them are dead now (I guess with the exception of fish shell, which is only a minor departure from bash). In this sense, I personally would not attempt such project personally. And the problem is not only too much space, the hypothetical replacement should have enough stability in time as well (no breaking of syntax/features), which combined with very large space to cover makes writing a hypothetical replacement an ~impossible task. Perhaps, the time for providing better shell replacement was in mid-80ties, and there is no way out now.
That said, IMO it's not like there is a usable Scheme implementation either (most of them are either overcomplicated or too limited) - so I still won't be stopping anyone from doing a Scheme in Ada; and having access to shell utilities (and I would add direct access to Linux syscalls as well) is the only way it could conceivably used for anything useful. Of course, most likely the point, that in a year or so, those who started to use it for long-term things would curse rewriting everything back to bash, will stand.
This is a nice reference for this article, not that I can somehow argue against it. And yes, when something needs to be done, it's those utilities that are used, irrespective of their potential warts.
Which is also why, (this goes to @Stanislav) I never liked esthlos' CL-V: why reimplement Keccak in CL when there is perfectly working one in Ada, available over pipe, and in this case - without warts at all? This is why IMO implementing signature verifier in CL/Scheme is wrong approach (yes, I know that if you parse the signature, verification is a ~3 lines of code, if the Scheme has actual bignums): you can use GPG, or perhaps Peh as well, why duplicate the work? And then rewrite phf's vpatch in CL as well, for the argument of purism?
As per my own notes, such things (like "masamune", like "peh", like "CL-V", like all the rest of the exponentially growing pile of wank) were tolerated, nothing more, and that strictly under the heading of "bois are mentally inferior to girls for purely biological reasons, so they have to be given space to hang themselves, after which they perhaps arrive to understand just how fucking stupid they were, learn something from the experience, and consequently make something of themselves".
This theoretical hope doesn't pan out in postmodern practice ; as it turns out out, of such pharmaceutical abundance as besets us on all sides some drug can always be found or tailored for every hurt boi, so that ever-precious neoteny may be preserved.
[...] dilligently working on owning and improving the vtools suite, clearly communicating on it and on his own history of V use that informs his choices of implementation, I find myself alternating between the two - while the old v.pl still has its niche for quick [...]