Following the the recent discussion in the logs, and reports by lobbes and spyked, time has come for me to publish a work plan as well.
I would divide all tasks into two groups: larger TODO items that I find important but that may take unpredictable amount of time, and smaller items work, on which someone else is perhaps working already.
The larger items are the following:
1. Adapt assembly FFA code to at least Chapter 14B1. After that, if the performance improvement is significant enough, I can switch to reading rest of FFA in leisure mode. There was also an idea of making a speed monster asm-only FFA variant2. I have to admit that without a concrete and pressing use-case, I'd be hesitant to take up this project - it does not seem worthwhile spending time polishing assembly if the expected returns are 0.5% performance improvement in a rarely used command. Nevertheless, I find that speed gains for modexp code would be a welcome addition to FFA. Also, I put this item first because I really don't like to work on several things at once, even if it is inevitable sometimes.
2. Ripping out kernel RNG as mentioned several times in the logs. This would involve some amount of investigative work at the beginning, that should take totally predictable amounts of time3. Then, come up with a plan to provide raw random bits to kernel from FG. The standard way of doing this4 comes in contrary to the "buffered RNG is not useful RNG" rule from the logs, so some way that avoids buffering must be found.
3. Some GNAT work, given that ave1 is busy with ARM64 SJLJ runtime. There is a bug with the variable finalization in static libraries reported by asciilifeform. Also, per my reading of GNAT runtime source, it might be possible to have asynchronous thread aborts with ZCX as well, with a small amount of code changes and slightly increased code smell - this would be something to have a look at as well5.
There also are some items that may be nice to work on, but are either not a priority right now, or have someone working on them already:
- For vtools, I have a candidate fix for the vdiff issue with large files6, and a small usability improvement for ksum - handling input from stdin.
- Writeup about the firefly board, how to make it more or less usable, and why it may be not worth spending time on this board at all.
- Another item in my TODO was avoiding gcc5 step in the cuntoo build process. When ave1gnat gets added to cuntoo, this step will be immediately eliminated, so I guess the correct way to approach this is provide help to mod6 if he need any.
- Testing the cuntoo-based TRB rotor (made by mod6) on ARM64. Some basic testing showed the the configure scripts of the dependencies (bdb, openssl) do not support aarch64, which will most likely cause a rewrite to a saner build system.
There are other items fluctuating in the logs, like TMSR ECC code for TRB, but I guess it's too early to put them into my TODO list.
The timeline for the work would be something like following:
- FFA work should take around two weeks - starting 02.07.2019 (so until around 16.07.2019) - to work through the next chapters, including the mathematical proof in Chapter 14. The assembly code is unlikely to change, but there is always a requirement to do measurements, investigate where bottlenecks are, see if something else really calls for asming, if yes - amend plans accordingly.
- After that, I would be open to work on any of the items 2 or 3, as the forum sees fit. So far I haven't seen any pressing need for item 2, thus there is a question whether working on ave1gnat could be a priority direction, or should be left fully for ave1.
Comments are welcome.
- OK, at least this should take predictable amount of time. [↩]
- If only I could find the log reference... [↩]
- Two, at most three weeks as far I can see now. [↩]
- A ring buffer fed by a userspace process that may read bits from a FG or two, pass zeros or any other known sequence, and generally speaking be under user's control. [↩]
- Unless ZCX is fully proscribed, of course. [↩]
- Basically, I adapted ksum code to hash a file chunk-by-chunk. [↩]
> I'd be hesitant to take up this project
I don't think it's a priority atm, fwiw.
> Ripping out kernel RNG
This however is a major priority ; especially because it can turn the currently worked-upon cuntoo from a flavour project some group stylistically flavours to a must-have item with no possible competition in the professional (as opposed to toy&smartphone) IT space.
> "buffered RNG is not useful RNG" rule
This rule only applies to new development. When we write a kernel, we'll write it correctly, to treat entropy demands as events rather than whatever the fuck current haphazard nondesigned cOpS (coincidentally-operationg pseudo-system) do.
Until then, simply filling the kernel entropy pools as they currently stand with FG-sourced noise rather than the current FtMeade sourced noise is good enough.
That's your work-around right there.
Point 3 is quite interesting as well.
> For vtools, I have a candidate fix
This should be released right now, I don't expect wrapping to take more than a few hours ?
> Writeup about the firefly board
Please.
> avoiding gcc5 step in the cuntoo build process
As you say, this should wait.
> Testing the cuntoo-based TRB rotor
This is liable to be both very intensive and also drive large changes downstream. I get that it appears sexy, but empty the pipe and clear your table before you start with this. Not like we have the spare hands to delve into build systems right this instant anyway.
Cheers.
Re: 100% ASMistic FFA: FWIW I cannot currently think of any good reason to do this.
Re: Firefly, I looked at this board in the past, rejected it on account of expense (it cost more than twice than the board I ended up using in the pilot plant.) Would still be interesting to read about it, however, if you tested it (would seem to me that my existing OS would run there without modification?)
Re: FG-eating kernel: the principal difficulty is to ensure that no two processes can ever end up being fed the same sequence of FG outputs. IMHO a direly needed item, however. And one that I thought would surely end up on my own conveyor, AFAIK no one has taken it up yet, if you do it -- I promise to test and comment.
@Stanislav Datskovskiy: My expectation was that the board will work reasonably well with storage, i.e. m.2 SSD. The problem is that its m.2 slot is not designed to be used for storage as is -- it has a weird pinout that is compatible with wifi cards or some other crap; to use an SSD with that slot you need an adapter board that will make the slot pinout match the standard (available from the vendor, and no, it's not b-key to m-key, it's something board-specific). I could've also used an usb3 SSD, however in this case any cheaper boards will work as well. Another problem is that I failed to get a usable TCP speed on this board -- the sustained download/upload speed (TCP/http, no sslisms involved) was ~100kbps, over 1G wire; and the connection was never stable. The network tweaks that I knew about did not work out, and I did not have enough time to go into device tree tweaking (that some do), or picking kernel version on which the network would work reliably (I never tested wifi with the board).
So in summary you have an expensive board that is clearly not designed to work with storage, big.LITTLE -> not designed to work with stock kernel, which requires time investment with unclear outcome to be even useful due to unstable wired network. It very much seems that that rk3399 is a multimedia chipset, that is supposed to be used in various toys, and lacks the things that you'd want in a small server.
The core of the work that I did was to build uboot with a configuration capable of booting the board off SSD (without which you can't really gentoo), and then to tweak uboot script to ignore the kernel arguments in the device tree, and use uboot-provided ones -- I had little experience with ARM boards before. But yes, your rootfs worked nicely with it.
Re: RK3399: sounds like that old Czech folk tale, where they had: "the king demands your hand in marriage." "is he young?" "he's old!" "is he kind?" "no, vicious!" "old and vicious, just like a husband ought to be!"
[...] Mogosanu (WoT: spyked) for December 2019 and Eric Benevides for December 2019, bvt (WoT: bvt) for July 2019 and Young Hands. [↩] Articles publishing a vpatch, should have context about the vpatch, your [...]