Living in the free world

March 29th, 2020

In the eighties we grew up knowing we we free. We knew this because the propaganda of that time told us so. As an easily impressed child I could only image how black live was on the other side of the wall. There nobody was free, it was so horrible that those people in eastern Europe were not allowed to travel to western Europe1. This was so thoroughly put into our heads, even in songs it was clear how great and free we were and how terrible and repressed they were. Also, how stupid, to let your government repress you so. And worse, to have neighbors who would inform on you. No, we here in the west, we are smart, we are free, we will never let this happen to us.

Well, of course already back then this was a lie. But, this week it became clear that how far we have come. No one is to leave their home, no one is allowed to get within 1.5 meter of each other. Last week-end, some would not listen and as the weather was nice, the beaches were full. This was not to be allowed, so no you will be fined if they see you in a group. And, zero protest. Today the last active KLM boeing 747 landed at Schiphol. People who went to see it land where fined2. But that's OK as they were warned. The news is full of heroes of the people3. It looks exactly what we were taught about so many years ago.

O, and let's not talk how we halved the number of beds in hospitals in the past 40 years. Well, we will see what the future brings.

  1. if they were able too they would never return or so the logic went []
  2. This makes no sense, the wind is very strong today most of your exhaled air will immediately disperse into nothing. Plus, these people usually look from their cars and keep their distance []
  3. The supermarket employees are heroes. The teachers who sit at home and make a video for the kids are heroes. The bus drivers of empty buses are heroes. The list goes on []

Failure Modes

January 25th, 2020

To name some; going awol, going off on an inconsequential angle, moving in circles or in short "being an engineer", or "engineering". All failure modes1 that I have discovered also apply to me.

At a previous job I made the asked for product but also 2 Integrated Development Environments out of parts laying around. Including a parser framework, a web based UI, a structural editor, an automatic test executor. When the first one became functioning for a while and my production level improved because of it, I decided it needed a rewrite. I was doing my required job and to the outer world productive (and I seemed to be sought out for what I did) but, I could have spent all that time to pull myself out of software engineering.

This past year, I've tried to get better. Not better as in improving myself, but get better as recovering from an illness. So, no engineering, but figuring out what we need to do and then deciding how to do it and making sure someone does it. Plus, communicate to all involved, committing to goals, setting goals. In short "management". I've loved it and this and will continue, but I still have engineering sickness. I wanted to make a package to make box diagrams. I did not communicate in the logs or at work, when the outcome would be too much unknown. I started to learn 2 programming languages; Lisp and Erlang. All off these, flight into save and known activities. Relapses into engineering.

So more convalescent therapy this year.

  1. In our effort to build a working production machine we have a todo list per sub-project. This todo list contains for at least a year a work-item to figure our all the ways some system might faile. With the tool almost done we have still not made this list of "failure modes". []

Off the road again

September 9th, 2019

Yes, we landed at Schiphol last Saturday. The whole week, the memories of the trip where slowly slipping out of my mind. Before the whole thing is gone, I'll try to salvage some bits in this post.

Some casual observations. The whole country is low maintenance, as in they will not do any maintenance1. It's so visible that you can guess the age of any building or road by the state it is in. For example, looking at the new pathways at the park in Kiev; "This looks so good, only minimal damage, must be about a year old". Indeed, at the gate / walk in point of the park a plate with some text and "2018". Another instance, we walked over a nice piece of green in between two driving lanes in one city. These pieces were rebuilt in different ages. The most recent already had grass growing between the tiles and some minor breakage. As we walked along, the breakage, grass and all around state went down. It was something to see. Now that I'm back, I've also started to notice lack of maintenance here, so what now?

20190814_175532

Playing chess in the park. I hope this is still here when I'm that old.

20190814_203338

hmm

As for restaurants. They provide many, the current hype seems to be sushi2. As with any hype, plenty of pizza restaurants also providing sushi. We did not try, I wanted to try the local food. Which was harder to get then "foreign" shit. Another observation; Queues at the McD, apparently this is a luxury item.

20190818_131131

Meat market in Vinnitsa, just the smell of fresh meat. We ate some raw bacon here that was very nice. In reference to the market in Kiev to seen on trilema.

20190818_131807

Way too cheap food. This restaurant turned out to be a hole in the wall for alcoholics. The salad was fine, the meatball too much bread.

We went to Lviv for a couple of days. This city is way different from the other cities I've seen so far in Ukraine. The whole city has a feel of an Italian city about 25 years ago. Traffic is a mess. Buildings need to be renovated (and some are). A lot of tourists3. We asked at the Hotel, where can we eat some good vareniki? we were directed to the worst restaurant on whole trip. You would start to hate the local cuisine based on this restaurant alone4.

Next we went to the Carpathians, we went by train to Mykulychyn5.

20190823_132844

A view

The hotel was next to a mountain spring. The water from the faucet was directly from this spring. Can you image the great taste? The food in the hotel was homemade, in the end we spent more on the food than the stay in this hotel.

20190823_142138

Building a house. No plan, just how your father did it.

After a week of swimming and hiking we went to Kiev to catch our plane. Here we had made a mistake of taking the sleep train back to Kiev. As we got no sleep and the whole experience was overall terrible. The two days in Kiev were nice, this city is simply on another scale.

20190831_152216

The clouds on the way home.

  1. I found two exceptions to this rule; schools and trains. The schools are well maintained. These get a fresh coat of paint every year, are clean and have well kept grounds. The difference is so striking that you can easily spot a school. Some bit of conversation;

    Me: That must be school

    Wife (from ukr): What?, no!  it's too big for this area.

    Me: It must be.

    Wife (to cab driver): What's that building?

    TD: School
    []

  2. We visited about 3/4 years ago ((a) I should have made a report then. (b) I did not see any sushi restaurants then. []
  3. mostly Polish []
  4. Ofcourse, now I cannot find it on google maps. []
  5. Train stops at a strip of stones that could be called a station []

On the road

August 12th, 2019

Saturday we flew to Ukraine. The flight wit LOT (Polish Airlines) was supposed to be at 7:00. At 22:00 on Friday I get an SMS to inform me that the flight is canceled, please call if you need any assistance. On, to call a generic number, wait for half an our on the line to get transferred to the problem desk. Wait for half an hour to get disconnected. And retry. About 2 hours later we had an alternative flight. This flight was to start at 9:35 and contain a transfer in Munich.

Guess where?

Guess where?

Advise to future travellers, never book a flight with a transfer in Munich. The Airport seems to consist of about 3 different sub-airports.  You'll land at the furthest end of one of the sub-airports and then have to transport yourself to the furthest other sub-airport. This is over these moving bands / flat escalators plus also a shuttle. Note that your transfer will include an extra baggage check, in which they will merrily do an extra swap for bomb residuals on the children. And, of course we need to go through customs, these Germans have 2 boots out of 20 or so open just to produce nice queue. We try to go through the automated machine, this only works for the adults. I ask the customs officer if we can skip the line, turns out it's OK to just push your children through the automated machine at the same time you are going through it. So, some extra running and shuttles later we arrive just on time as the next flight is delayed. And an uneventual 2 hours later we find ourselfs here;

airport_ua_2019_1024By the time we were ready to go through customs, the queue had dissolved and all went very smooth. As you can see the walkways to the planes have advertisements these days and the in Kiev looks a lot worse than the one in Schiphol. This turns out to be true for the whole country, but that will be another post. That post will most probably be how these people have money to build something but then immediately forget about it and let it rot. To finish up; the food is good.

Scraping btcbase.org

August 7th, 2019

I wanted to add a comment to trilema, but apparently it looks like spam. So here the comment for http://trilema.com/2019/trilema-goes-dark.

Currently I'm stuck on this reversing the original logs from the output at btcbase.org because of two problems;

1. The new format using block quotes [link][text] sometimes produces links  that are ambiguous and cannot be distinguished from the old way (for example [link][link] or [absolute btcbase link][text] vs [relative btcbase link][text]).
2. Sometime the conversion to html went wrong, for example in http://btcbase.org/log/2016-12-29#1592569. I could make some exceptions for these.
3. It seems that the btcbase.org html output code puts special tokens at the end into the description of the link; i.e. (comma's and dots and closing brackets etc are all eaten and put in the description of the link).

Here are my scripts, to preserve the current status;

The scraper: http://ave1.org/tarpit/log/scrape_log.py

To do all: http://ave1.org/tarpit/log/scrape_all.py

Lsip, Common Lisp The Language, Purpose, adnotated,

August 2nd, 2019

Common Lisp is intended to meet these goals:

Commonality
Common Lisp originated in an attempt to focus the work of several implementation groups, each of which was constructing successor implementations of MacLisp1 for different computers. These implementations had begun to diverge because of the differences in the implementation environments2: microcoded personal computers (Zetalisp3, Spice Lisp4), commercial timeshared computers (NIL-the ``New Implementation of Lisp''5 ), and supercomputers (S-1 Lisp6 ). While the differences among the several implementation environments of necessity7 will continue to force certain incompatibilities among the implementations, Common Lisp serves as a common dialect to which each implementation makes any necessary extensions8.

Portability
Common Lisp intentionally excludes features that cannot be implemented easily on a broad class of machines. On the one hand, features that are difficult or expensive to implement on hardware without special microcode are avoided or provided in a more abstract and efficiently implementable form. (Examples of this are the invisible forwarding pointers and locatives of Zetalisp9. Some of the problems that they solve are addressed in different ways in Common Lisp10.) On the other hand, features that are useful only on certain ``ordinary'' or ``commercial'' processors are avoided or made optional. (An example of this is the type declaration facility, which is useful in some implementations and completely ignored in others. Type declarations are completely optional and for correct programs affect only efficiency, not semantics.) Common Lisp is designed to make it easy to write programs that depend as little as possible on machine-specific characteristics, such as word length, while allowing some variety of implementation techniques.

Consistency
Most Lisp implementations are internally inconsistent in that by default the interpreter and compiler may assign different semantics to correct programs11. This semantic difference stems primarily from the fact that the interpreter assumes all variables to be dynamically scoped, whereas the compiler assumes all variables to be local unless explicitly directed otherwise. This difference has been the usual practice in Lisp for the sake of convenience and efficiency but can lead to very subtle bugs. The definition of Common Lisp avoids such anomalies by explicitly requiring the interpreter and compiler to impose identical semantics on correct programs so far as possible12.

Expressiveness
Common Lisp culls what experience has shown to be the most useful and understandable constructs from not only MacLisp but also Interlisp, other Lisp dialects, and other programming languages13. Constructs judged to be awkward or less useful have been excluded. (An example is the store construct of MacLisp.)

Compatibility
Unless there is a good reason to the contrary, Common Lisp strives to be compatible with Lisp Machine Lisp14, MacLisp, and Interlisp15, roughly in that order16.

Efficiency
Common Lisp has a number of features designed to facilitate the production of high-quality17 compiled code in those implementations whose developers care to invest effort in an optimizing compiler18. One implementation of Common Lisp, namely S-1 Lisp19, already has20 a compiler that produces code for numerical computations that is competitive in execution speed to that produced by a Fortran compiler [11]. The S-1 Lisp compiler extends the work done in MacLisp to produce extremely21 efficient numerical code22[19].

Power
Common Lisp is a descendant of MacLisp, which has traditionally placed emphasis on providing system-building tools23. Such tools may in turn be used to build the user-level packages such as Interlisp provides24; these packages are not, however, part of the Common Lisp core specification. It is expected such packages will be built on top of the Common Lisp core25.

Stability
It is intended that Common Lisp will change only slowly and with due deliberation26. The various dialects that are supersets of Common Lisp may serve as laboratories within which to test language extensions, but such extensions will be added to Common Lisp only after careful examination and experimentation27.

The goals of Common Lisp are thus very close to those of Standard Lisp [31] and Portable Standard Lisp [51]2829. Common Lisp differs from Standard Lisp primarily in incorporating more features, including a richer and more complicated set of data types and more complex control structures30.

This book is intended to be a language specification rather than an implementation specification31 (although implementation notes are scattered throughout the text32 ). It defines a set33 of standard language concepts and constructs that may be used for communication of data structures and algorithms in the Common Lisp dialect34. This set of concepts and constructs is sometimes referred to as the ``core Common Lisp language'' because it contains conceptually necessary or important features35. It is not necessarily implementationally36 minimal37. While many features could be defined in terms of others by writing Lisp code, and indeed may be implemented that way, it was felt that these features should be conceptually primitive so that there might be agreement among all users as to their usage38. (For example, bignums and rational numbers could be implemented as Lisp code given operations on fixnums. However, it is important to the conceptual integrity of the language that they be regarded by the user as primitive, and they are useful enough to warrant a standard definition39.)

For the most part, this book defines a programming language, not a programming environment. A few interfaces are defined for invoking such standard programming tools as a compiler, an editor, a program trace facility, and a debugger, but very little is said about their nature or operation. It is expected that one or more extensive programming environments will be built using Common Lisp as a foundation, and will be documented separately40.

change_begin
There are now many implementations of Common Lisp, some programmed by research groups in universities41 and some by companies that sell them commercially42, and a number of useful programming environments have indeed grown up around these implementations43. What is more, all the goals stated above have been achieved, most notably that of portability44. Moving large bodies of Lisp code from one computer to another is now routine45.
change_end

  1. Long dead,  nothing to do with apple, still has some web presence []
  2. So even before symbolics? []
  3. this was the lisp on the lisp machine, which then split off into symbolics []
  4. became CMUCL, which branched off to SBCL. Should I read SBCL code as an example of a good lisp program? []
  5. From wiki: Although unsuccessful in meeting its goals as a used language, NIL was important in several ways. First, it brought together Jon L. White, Guy L. Steele Jr., and Richard P. Gabriel, who were later to define Common Lisp. Second, Jonathan Rees worked on part of the NIL project during a year away from Yale University. On returning to Yale, he was hired by the computer science department to write a new Lisp, which became the optimizing, native code Scheme system named T. In part, NIL begat this name, since "T is not NIL".

    Shall I put in another link to the MP article?
    []

  6. dead, can anyone find the source? it was supposed to have been written in lisp []
  7. why is this necessary? []
  8. Aha, so common lisp only describes the common parts, the rest is all up in the air, and so we have the situation of today []
  9. These sound very cool, what were they? []
  10. I hope these are documented somewhere []
  11. What? This is going to be fun to learn []
  12. vague, first "explicit" and "require", next "so far as possible" []
  13. This sounds like advertising []
  14. Other name for ZetaLisp as far as I can figure out,  why use this name instead? []
  15. Deady, very []
  16. I you re-read froin the start, then the first lisp mentioned was MacLisp and now "Lisp Machine Lisp" is the most important one. What gives? []
  17. What defines "high-quality" in this context? []
  18. In the others, they don't care, the losers []
  19. The dead version []
  20. had []
  21. how extreme? []
  22. No information in this sentence other than to provide an excuse for the link []
  23. Empty sentence []
  24. Apple traditionally place emphasis on providing media producing tools. Such tools may in turn be used to produce the images such as any random porn site provides. []
  25. Hahahahahaha uch uch uch. []
  26. This came true,  but for different reasons than the author probably had foreseen []
  27. This was to be done by scientist, between those spare moments when they where not looking at the climate []
  28. Fuck, even here they were split, 3 ways!!! []
  29. The PSL code is available. The first version of this site on the wayback machine is from 2016 and says  *** Very soon major version update will be released. ***". The same words are still there. []
  30. We are BETTER and more COMPLEX than those fuckers []
  31. So this one is not CLHS, but something else? []
  32. Although on second thought, maybe it still could be an implementation specification? []
  33. In this book we defined a set []
  34. You can give me any set of words and I'll make a sentence out of it! It will not mean anything, but it sort of hints at something cool []
  35. Let's just continue, yes? What is it concepts, constructs or features or a bit of all of these? []
  36. Look mam, I made a new word []
  37. It's a whole lot of shit, sorry we could not minimize it []
  38. Fuck you []
  39. And fuck you again []
  40. and die in flames []
  41. the losers []
  42. the bastards []
  43. you know, "a number", like 0. That's a perfectly fine number. You can use it to add or multiply, it is a bit harder to use in division. []
  44. Hahahahaha uch uch uch, BTW can you  make multiple referrals to the same footnote in mp-wp? []
  45. Yes I just saw some large bodies moving about on the street! and their talk was definitely a bit off with losts of sss in there []

Lisp, Why?, step #5

August 1st, 2019

Why do you want strings ? What do you use them for ? Why are you using strings for that ? And so following.

Mircea Popescu

For years, I have wanted to break free of the tools of the Inca; Python, Matlab, Octave. These go-to languages to quickly and easily make something sort of work. I'm now working in a position where another set of tools is used, Word, Outlook, Excel. These are a step worse. Each time, I trade short term gains for long term costs. And those costs are mounting. This cannot stand. So, I want to work towards some sort of solid ground (learn Ada, pin down gnat,  read ffa). And also, use Common Lisp for those cases where I would normally go for Python and friends.

Next part of the why, I want to make diagrams like so. I've found no tools to help me with this on the long term;

  1. GUI Based diagram editors; Every editor comes with handy ways to make boxes with text,  support to put links between these boxes  and to make a sort of layout. But, for every change you will have to work on the entire diagram. Plus I do not want to become a proficient mouse worker.
  2. Graphviz; It sort of works, diagrams can be entered but to layout a diagram you I'll need to perform a feedback loop (change a parameter (like the order of the links), make graph, look at result, change another parameter, ...). The problem is that this loop is needed for every change, or simply give up on layout and then why are you doing it?.
So I jumped to the conclusion to produce some LISP code whereby I can define pieces of text1, define links between the pieces, specify locations for the pieces of text and finally the code will produce a diagram.  I decided to use svg as the the output format of this code. First task; given a piece of text, produce a svg file with this text and a box around the text.
As svg is just xml, I thought why not just string together the pieces needed, concatenate these into one big text and output the resulting mess.  And so have a kapstok to start learning LISP. My first step was to delve into the lisp spec was to find the call to do help with the concatenation magic. My other step was to piece together some higher level parts. And now on to a re-think desing.
As for the code so far;
(defun strcat (a b) (concatenate 'string a b))
(defun spacedstrcat (a b) (concatenate 'string a " " b))

(defun strlistcat (list)(reduce (lambda (a b) (concatenate 'string a b)) alist))

(defun xmlresttag (tag content)
  (if (and content (> (length content) 0))
      (strcat content (strcat (strcat "</" tag) ">"))
      "/>"))

(defun xmlbegintag (tag) (strcat "<" tag))
(defun xmlattribute (attribute) )
(defun xmlattibutes (attributes &key id class)
  (strlistcat (map xmlattribute attributes))

(defun xmlelement (tag content attributes &key id class)
  (spacedstrcat(spacedstrcat (begintag tag) (xmlattributes attributes :id id :class class))
               (xmlresttag (tag content))))

(defun rectangle (x y width height &key (content "") id class)
  ())

(defun assert-equals (a b)
  (if (string= a b) ()
      (progn (pprint `("not equals" ,a ,b) )
             (cerror "assertion failed" "abort"))))

(defun test$strcat ()
  (assert-equals (strcat "a" "b") "ab")
  (assert-equals (strcat "this" " is a test") "this is a test"))

(defun test$spacedstrcat ()
  (assert-equals (spacedstrcat "a" "b") "a b")
  (assert-equals (spacedstrcat "this" "is a test") "this is a test"))

(defun test$xmlresttag ()
  (assert-equals (xmlresttag "first" "") "/>")
  (assert-equals (xmlresttag "second" NIL) "/>")
  (assert-equals (xmlresttag "hello" "world") "world</hello>"))

(defun test$xmlbegintag ()
  (assert-equals (xmlbegintag "first") "<first"))

(defun run-tests ()
  (test$strcat)
  (test$spacedstrcat)
  (test$xmlresttag)
  (test$xmlbegintag))

The whole artifact is resting on the weak shoulders of concatenate and the wrong concept of  merging pieces of text into bigger pieces of text. I am thinking that constructing lists of strings will be a big improvement and then have a function whereby the whole list is output as svg text.

  1. I'm  avoiding the terms string and sentence []

Lisp, a question about concatenation, step #4

July 31st, 2019

For some reason a simple thing turns out to be hard1. This is what I want to do; I want to concatenate a list of strings into a single string. I do not want this;

(concatenate 'string "no" "not" "a fixed" "list," "fuck" "it")

But this;

(somefunction alist)

Where alist is;

(list "yes" "just" "a" "list")

The answers I found so far are;

  1. Many many examples of concatenate, which does not fit in this case.
  2. Using format. I reject this because this is a different sub-language. Something like this should be doable in lisp, not a special format language.
  3. Complex functions using loop or doiterate. Really?
The one I like best so far is;
(apply #'concatenate 'string list)

But is this really the way to go? or should I make something based on reduce?

Like this;

(reduce (lambda (a b) (concatenate 'string a b)) (list "why" "no" "sharp-quote" "for" "the"  "lambda?"))
  1. I must not be thinking very lispy yet or something []

Lisp, Function NULL, Step 3.

July 30th, 2019

I wanted to check for an empty string, and so came by the Function NULL. Finally, a well specified function. The description;

Returns t if object is the empty list; otherwise, returns nil.

Luckily, in Common LISP the "empty list" and "nil" are one and the same, so this sentence can be written as is1.

And the examples;

 (null '()) =>  T
 (null nil) =>  T
 (null t) =>  NIL
 (null 1) =>  NIL

Also fine, the four cases you would expect here.

But then, the last line of the notes;

 (null object) ==  (typep object 'null) ==  (eq object '())

Now, how does help? the author could not be bothered to add some text here? Is "==" somehow part of the syntax?

  1. Try this; Returns t if object is nil; otherwise, returns nil.

    Or this; Returns t if object is the empty list; otherwise, returns the empty list.

    I seem to also getting infected with this LISP virus to look for the edge cases []

Lisp, documenting my encounter with, step #2

July 29th, 2019

Dear blog, yesterday seem to have disappeared in the fog of time without me writing to you, I'm so sorry1. I promised to write a bit about the Special Operator Progn.I have no idea yet why this is called a "Special Operator"2. The writer of the specification was hopelessly confused when writing this one3. First lines;

progn form* => result*

Arguments and Values:

forms---an implicit progn.

results---the values of the forms.

As this will turn out to be important later, I checked the meaning of the asterisk symbol. This is not to be found in the spec as far as I can see, but the author does refer to an((yes the documentation says an, but which one?)) extended BNF, and although the meaning of "extended" is apparently free to interpretation, the asterisk probably stands for 0 or more4.  So, conclusion zero or more forms in, zero or more results out.

Next the description:

Description:

progn evaluates forms, in the order in which they are given.

The values of each form but the last are discarded.

The first line parses, the second seem to be in direct contradiction to the "Arguments and Values". Which is it? all the values of all the forms or only the last one? Maybe the examples will make this clear, but first another descriptive line:

If progn appears as a top level form, then all forms within that progn are considered by the compiler to be top level forms.

More links, these have to wait, but this does not parse as is, so if it is a "bottom level form" then all forms are considered "bottom level", or also mid level? or what? What does this mean and how does this  sentence clarify?

On to the examples;

 (progn) =>  NIL
 (progn 1 2 3) =>  3
 (progn (values 1 2 3)) =>  1, 2, 3
 (setq a 1) =>  1
 (if a
      (progn (setq a nil) 'here)
      (progn (setq a t) 'there)) =>  HERE
 a =>  NIL

First line, a special case, not described or obvious5, but at least now we know that zero forms will result in NIL.  Second, confirms to the description. Third, does this help? a single form will result in the result of that single form. Fourth line and fifth, here we go again, completely over the top side issues added, or does this have to do with the "top level" thing? and then how would this be different in a function definition?

Finally;

Notes:

Many places in Common Lisp involve syntax that uses implicit progns. That is, part of their syntax allows many forms to be written that are to be evaluated sequentially, discarding the results of all forms but the last and returning the results of the last form. Such places include, but are not limited to, the following: the body of a lambda expression; the bodies of various control and conditional forms (e.g., case, catch, progn, and when).

This was how I got to the progn, the if ((special operator, operator, form, function, macro or whatever)) does not support the "implicit progn".  This is not how you specify a language! So we have here this syntax, sometimes it is used, sometimes it is not used, here is some of the sometimes it is used. So let say you have 10 people in the spec commission, responsible for writing it. You divide all functions, forms and whatnot between those 10, each has to then do about 100 or whatever. For each you list the implicits and not-implicits or not applicables. You get 3 tables6. You put them in.  Done. O wait, you think this "implicit progns" links points to this?

  1. and also I'll end this style of starting an entry! []
  2. another one for the queue []
  3. and this happened to be one of first things I needed!! []
  4. Would it have been so hard to describe the BNF format fully in the specification. Or to at least provide a reference? []
  5. do they know what an example means? []
  6. or there abouts, whatever counting is hard []