The Intellectual Wilderness There is nothing more useless than doing efficiently that which should not be done at all.

2020.04.5 14:44

For Data Reporting, Keep Reliable URLs

Filed under: Science & Tech,Society — Tags: , , , , , , , , — zxq9 @ 14:44

It would be really nice if governments that are trying to be transparent by reporting stats publicly on a website would stop changing their entire URL scheme every week. It is hard to track things down and even the search engines aren’t keeping up very well.

2020.02.3 15:19

X-Y Problems

Filed under: Computing,Science & Tech,Society — Tags: , , , , , — zxq9 @ 15:19

People obsess about their X-Y problems to the point of ignoring accepted wisdom, plugging their ears to the deafening silence of the solution’s instructive whisper, picking themselves up as hard as they can by their own knees and wondering why they can’t fly.

They then run off and formalize their wrong solution as a PR into a core project.

If core maintainers aren’t mindful they’ll incorporate these disturbances into a previously still space, and if they are indelicate they will piss off the misguided (but industrious) boob who made the PR who is already by this point fanatically dedicated to his wrong solution and the idea that nobody “gets it” but him.

Ah, another day at the Bazaar.

2019.12.28 23:56

Starting a simple GUI project in Erlang with ZX

A few days ago I wrote a tutorial about how to create a CLI program in Erlang using a new code utility called ZX that makes launching Erlang a little bit more familiar for people accustomed to modern dynamic language tooling.

Today I want to do another one in the same spirit, but with a simple GUI program as the example.

In the previous tutorial I did a CLI utility that converts files containing JSON to files containing Erlang terms. It accepts two arguments: the input file path and the desired output file path.

Today we’ll do a slightly more interesting version of this: a GUI utility that allows hand creation/editing of both the input before conversion and the output before writing. The program will have a simple interface with just three buttons at the top: “Read File”, “Convert” and “Save File”; and two text editing frames as the main body of the window: one on the left with a text control for JSON input, and one on the right a text control for Erlang terms after conversion.

First things, first: We have to choose a name and create the project. Since we did “Termifier” with a package and module name “termifier” before, today we’ll make it called “Termifier GUI” with a package and appmod “termifierg” and a project prefix “tg_”. I’ve clipped out the creation prompt output for brevity like before, but it can be found here: zx_gui_creation.txt.

ceverett@okonomiyaki:~/vcs$ zx create project

### --snip snip--
### Prompts for project meta
### --snip snip--

Writing app file: ebin/
Project otpr-termifierg-0.1.0 initialized.

If we run this using ZX’s zx rundir command we see a GUI window appear and some stuff happen in the terminal (assuming you’re using a GUI desktop and WX is built into the Erlang runtime you’re using).

The default templated window we see is a GUI version of a “Hello, World!”:

If we try the same again with some command line arguments we will see the change in the text frame:

The output occurring in the terminal is a mix of ZX writing to stdout about what it is building and WX’s GTK bindings warning that it can’t find an optional style module (which isn’t a big deal and happens on various systems).

So we start out with a window that contains a single multi-line text field and accepts the “close window” event from the window manager. A modest, but promising start.

What we want to do from here is make two editable text fields side by side, which will probably require increasing the main window’s size for comfort, and adding a single sizer with our three utility buttons at the top for ease of use (and our main frame, of course, being derived from the wxEvtHandler, will need to connect to the button click events to make them useful!). The text fields themselves we probably want to make have fixed-width fonts since the user will be staring at indented lines of declarative code, and it might even be handy to have a more natural “code editor” feel to the text field interface, so we can’t do any better than to use the Scintilla-type text control widget for the text controls.

Now that we know basically what we want to do, we need to figure out where to do it! To see where to make these changes we need to take a little tour of the program. It is four modules, which means it is a far different beast than our previous single-module CLI program was.

Like any project, the best way to figure out what is going on is to establish two things:

  1. How is the code structured (or is there even a clear structure)?
  2. What is called to kick things off? (“Why does anything do anything?”)

When I go into termifierg/src/ I see some very different things than before, but there is a clear pattern occurring (though it is somewhat different than the common Erlang server-side “service -> worker” pattern):

ceverett@okonomiyaki:~/vcs$ cd termifierg/src/
ceverett@okonomiyaki:~/vcs/termifierg/src$ ls -l
合計 16
-rw-rw-r-- 1 ceverett ceverett 1817 12月 27 12:50 termifierg.erl
-rw-rw-r-- 1 ceverett ceverett 3166 12月 27 12:50 tg_con.erl
-rw-rw-r-- 1 ceverett ceverett 3708 12月 27 12:50 tg_gui.erl
-rw-rw-r-- 1 ceverett ceverett 1298 12月 27 12:50 tg_sup.erl

We have the main application module termifierg.erl, the name of which we chose during the creation process, and then we also have three more modules that use the tg_ prefix we chose during creation: tg_con, tg_gui and tg_sup. As any erlanger knows, anything named *_sup.erl is going to be a supervisor, so it is logical to assume that tg_sup.erl is the top (in this case the only) supervisor for the application. It looks like there are only two “living” modules, the *_con one, which seems short for a “control” module, and the *_gui one, which seems likely to be just the code or process that controls the actual window itself.

We know that we picked termifierg as the appmod for the project, so it should be the place to find the typical OTP AppMod:start/2 function… and sure enough, there it is: termifierg:start/2 is simply call to start things by calling tg_sup:start_link/0. So next we should see what tg_sup does. Being a supervisor its entire definition should be a very boring declaration of what children the supervisor has, how they depend on one another (order), and what restart strategy is being employed by that supervisor.

(Protip: magical supervision is evil; boring, declarative supervision is good.)

init([]) ->
     RestartStrategy = {one_for_one, 0, 60},
     Clients   = {tg_con,
                  {tg_con, start_link, []},
     Children  = [Clients],
     {ok, {RestartStrategy, Children}}.

Here we see only one thing is defined: the “control” module called tg_con. Easy enough. Knowing that we have a GUI module as well, we should expect that the tg_con module probably links to the GUI process instead of monitoring it, though it is possible that it might monitor it or maybe even use the GUI code module as a library of callback functions that the control process itself uses to render a GUI on its own.

[NOTE: Any of these approaches is valid, but which one makes the most sense depends entirely on the situation and type of program that is being written. Is the GUI a single, simple interface to a vast and complex system underneath? Does each core control component of the system have its own window or special widget or tab to render its data? Are there lots of rendered “views” on the program data, or a single view on lots of different program data? Is it OK for updates to the GUI to block on data retrieval or processing? etc.]

Here we see a program that is split between interface code and operation code. Hey! That sounds a lot like the “View” and “Control” from the classic “MVC” paradigm! And, of course, this is exactly the case. The “Model” part in this particular program being the data we are handling which is defined by the Erlang syntax on the one hand and JSON’s definition on the other (and so are implicit, not explicit, in our program).

The tg_con process is the operation code that does things, and it is spawn_linking the interface that is defined by tg_gui. If either one crashes it will take the other one down with it — easy cleanup! For most simple programs this is a good model to start with, so we’ll leave things as they are.

The tg_gui process is the one that interfaces with the back-end. In this simple of a program we could easily glom them all together without getting confused, but if we add even just a few interesting features we would bury our core logic under the enormous amounts of verbose, somewhat complex code inherent in GUI applications — and that becomes extremely frustrating to separate out later (so most people don’t do it and their single-module-per-thingy WX code becomes a collection of balls of mud that are difficult to refactor later, and that stinks!).

Since we already know what we want to do with the program and we already proved the core code necessary to accomplish it in the previous tutorial, we can move on to building an interface for it.

This is what the final program looks like, using the same example.json from the CLI example:

At this point we are going to leave the blog and I will direct you instead to the repository code, and in particular, the commit that shows the diff between the original template code generated by ZX and the modified commit that implements the program described at the beginning of this tutorial. The commit has been heavily commented to explain every part in detail — if you are curious about how this Wx program works I strongly recommend reading the code commit and comments!

2019.12.25 23:04

Starting a simple CLI project in Erlang with ZX

Filed under: Computing,Science & Tech — Tags: , , , , , — zxq9 @ 23:04

Yesterday I wrote a post about a new tooling suite for developers and users that makes dealing with Erlang more familiar to people from other languages. Using the tool for packaging and deployment/launch makes writing and deploying end-user programs in Erlang non-mysterious as well, which is a great benefit as Erlang provides a wonderful paradigm for making use of modern overwhelmingly multi-core client systems.

It is still in beta, but works well for my projects, so I’ll leave a quick tutorial here that shows the basic flow of writing a simple CLI utility in Erlang using ZX.

In this example we’ll make a program that accepts two arguments: a path to a file with JSON in it and a path to a file where the data should be written back after being converted to Erlang terms.

To start a project we do zx create project and follow the prompts.
(The snippet below excludes the full output for brevity, but you can view the entire creation prompt log here: zx_cli_creation.txt.)

ceverett@okonomiyaki:~/vcs$ zx create project

### --snip snip--
### Prompts for project meta
### --snip snip--

Writing app file: ebin/
Project otpr-termifier-0.1.0 initialized.

After the project is created we see a new directory in front of us called “termifier” (or whatever the project is named). We can execute this now just to make sure everything is going as expected:

ceverett@okonomiyaki:~/vcs$ ls
ceverett@okonomiyaki:~/vcs$ zx rundir termifier
Recompile: src/termifier
Hello, World! Args: []
ceverett@okonomiyaki:~/vcs$ zx rundir termifier foo bar baz
Hello, World! Args: ["foo","bar","baz"]

Ah! So we already have something that builds and runs, very similar to how an escript works, except that using ZX we can easily add dependencies from Zomp package realms and package and execute this program on any system in the world that has ZX on it via Zomp ourselves.

…not that we have any reason to deploy a “Hello, World!” program to the wider public, of course.

Notice here that the first time we run it we see a message Recompile: src/termifier. That means the module termifier is being compiled and cached. On subsequent runs this step is not necessary unless the source file has changed (the compiler detects this on its own).

Next lets search Zomp for the tag “json” to see if there are any packages that list it as a tag, and if there are any let’s get a description so maybe we can find a website or docs for it:

ceverett@okonomiyaki:~/vcs$ zx search json
ceverett@okonomiyaki:~/vcs$ zx describe otpr-zj
Package : otpr-zj-1.0.5
Name    : zj
Type    : lib
Desc    : A tiny JSON encoder/decoder in pure Erlang.
Author  : Craig Everett
Web     :
Repo    :
Tags    : ["json","json encoder","json decoder"]

Ah. Checking the website it is clear we can use this to decode JSON by simply calling zj:decode(JSON). Easy. So let’s add it to the project as a dependency and invoke it in src/termifier.erl:

ceverett@okonomiyaki:~/vcs$ cd termifier
ceverett@okonomiyaki:~/vcs/termifier$ zx set dep otpr-zj-1.0.5
Adding dep otpr-zj-1.0.5
ceverett@okonomiyaki:~/vcs/termifier$ cd src
ceverett@okonomiyaki:~/vcs/termifier/src$ vim termifier.erl

Inside termifier.erl we can see the templated code for start/1:

start(ArgV) ->
    ok = io:format("Hello, World! Args: ~tp~n", [ArgV]),

Lets change it so it does what we want (note here I’m going a bit further and writing a function write_terms/2 based on an older post of mine — this performs the inverse procedure of file:consult/1):

start([InPath, OutPath]) ->
    {ok, Bin} = file:read_file(InPath),
    {ok, Terms} = zj:decode(Bin),
    ok = write_terms(OutPath, Terms),
start(_) ->
    ok = io:format("ERROR: Two arguments are required."),

write_terms(Path, Terms) when is_list(Terms) ->
    Format = fun(Term) -> io_lib:format("~tp.~n", [Term]) end,
    Text = lists:map(Format, Terms),
    file:write_file(Path, Text);
write_terms(Path, Term) ->
    write_terms(Path, [Term]).

Note that we are calling zj:decode/1 in the body of start/1 above, knowing that ZX will find it for us and configure the environment at execution time. And now let’s give it a go!

ceverett@okonomiyaki:~/vcs$ zx rundir termifier example.json example.eterms
Recompile: src/zj
Recompile: src/termifier
ceverett@okonomiyaki:~/vcs$ cat example.json 
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
ceverett@okonomiyaki:~/vcs$ cat example.eterms 
{"color" => "Red","fruit" => "Apple","size" => "Large"}.

From here I could run zx package termifier, submit it to a realm (either the default public realm, or a new one I can create and host myself by doing zx create realm and then zx run zomp), and anyone could then run the command zx run termifier [in path] [out path] and ZX will take care of finding and building the necessary packages and launching the program.

That’s all there is to it. ZX’s template for CLI applications is quite minimal (as you can see) and is more similar to an escript than a more traditional OTP-style, supervised Erlang application. ZX has templates, however, for full-blown OTP applications, GUI code (structured also in the OTP paradigm), minimalist CLI programs like we see above, pure library code, and escripts (sometimes escript is exactly the tool you need!).

Happy coding!

2019.12.24 23:20

Erlang: A new packager and launching tool

I’ve never really liked the way that Erlang’s existing tools generally force one to accept “releases” as the One True Way to structure, build, test, launch and deploy Erlang programs. We have escript, of course, which is fantastic, but it does come with a few handicaps such as making it a somewhat cubmersome and magical process to write OTP-structured programs as scripts.

“Why do would you care about this? Isn’t Erlang what you use for writing game servers and streaming servers and web servers and… the key word being ‘server’?”

I care because I also write a lot of client-side code and Erlang provides a very smooth paradigm for writing networked GUI and CLI programs. While how to best structure those has been a bit of an exploratory art of its own since the first release of Erlang’s GUI bindings, the best way to deploy them to client systems (that is “end-user systems” — yes, the unwashed, non-technical masses) has been one of those subjects people tend to wave off because, well, nobody wants to discuss it and, “HAH! Who would write client-side Erlang without endless armies of tech support goons around the product?!?”.

Well, now we can (or at least I can).

I’ve started on a suite of tools that handles source templating for projects (GUI apps, CLI utilities, escripts, traditional OTP applications and library projects), packaging of them, distribution and resolution of the packages, building, and launching (in a “first-run means automatic deployment” paradigm) in a way that a developer familiar with other dynamic scripting languages would probably feel more at home with.

I’m still messing about with a few of the minor features (like text search for packages based on tags, a GUI application browser, an installer for Windows, and so on) but the core of it is in place for developers already.

If you’re an Erlanger or perhaps just Erlang-curious but never were quite sure how to get started with writing Erlang programs because structuring an Erlang project was always a bit of a mystery you never had time to work your way through, give it a try and please give me some feedback so I can improve the tools.

The tool itself is called ZX and the distribution service node code is called Zomp. To run the installer it all you need is an Erlang runtime somewhere in $PATH.

Link here: ZX/Zomp

2019.12.20 10:59

PSA: Reinventing the Wheel

Filed under: Computing,Science & Tech — Tags: , , , — zxq9 @ 10:59

Reinventing the wheel is not always a bad thing, and sometimes it is even called for. If you are engaging in reinvention of some wheels, just ask yourself if you have a reason for doing it.

Good enough reasons:

  • Self education
  • “The existing thingy doesn’t quite do what I want”
  • Creating an open source alternative
  • Simplifying an existing idea
  • Bringing “a lib for X” to a new language
  • You find a particular thing enjoyable to write

Bad reasons:

  • Ego at the expense of project progress
  • Thinking it will win you an argument
  • NIH syndrome (though there can valid reasons for NIH, too!)

2019.11.21 17:24

Testing Textually Composed Numbers for Primality

Filed under: Computing,Science & Tech — Tags: , , , , , — zxq9 @ 17:24

Last night on Twitter one of my favorite accounts, @fermatslibrary, put up an interesting post:

Start at 82 and write it down, then 81, write that down, etc. until you reach one, and you’ve written a (huge) prime number. Wow!

This seemed so strange to me, so of course I got curious:

After some doodling around I wrote a script that checks whether numbers constructed by the method above are prime numbers when the number construction is performed in various numeric bases (from 2 to 36, limited to 36 because for now I’m cheating using Erlang’s integer_to_list/2). It prints the results to the screen as the process handling each base finishes and writes a text file “texty_primes-result-[timestamp].eterms” at the end for convenience so you can use file:consult/1 on it later to mess around.

There are a few strange aspects to large primes, one of them being that checking whether or not they are prime can be a computationally intense task (and nobody knows a shortcut to this). To this end I wrote the Miller-Rabin primality test into the script and allow the caller to decide how many rounds of Miller-Rabin to run against the numbers to check them. So far the numbers that have come out have matched what is expected, but once the numbers get extremely large (and they get pretty big in a hurry!) there is only some degree of confidence that they are really prime, so don’t take the output as gospel.

I wrote the program in Erlang as an escript, so if you want to run it yourself just download the script and execute it.
The script can be found here: texty_primes

A results file containing the (very likely) prime constructions in bases 2 through 36 using “count-back from X” where X is 1 to 500 can be found here: texty_primes-result-20191121171755.eterms
Analyzing from 1 to 500 in bases 2 through 36 took about 25 minutes on a mid-grade 8-core system (Ryzen5). There are some loooooooooong numbers in that file… It would be interesting to test the largest of them for primality in more depth.

(Note that while the script runs you will receive unordered “Base X Result” messages printed to stdout. This is because every base is handed off to a separate process for analysis and they finish at very different times somewhat unpredictably. When all processing is complete the text file will contain a sorted list of {Base, ListOfPrimes} that is easier to browse.)

An interesting phenomenon I observed while doing this is that some numeric bases seem simply unsuited to producing primes when numbers are generated in this manner, bases that themselves are prime numbers in particular. Other bases seem to be rather fruitful places to search for this phenomenon.

Another interesting phenomenon is the wide number of numeric bases in which the numbers “21”, “321”, “4321” and “5321” turn out to be prime. “21” and “4321” in particular turn up quite a lot.

Perhaps most strangely of all is that base 10 is not a very good place to look for these kinds of primes! In fact, the “count back from 82” prime is the only one that can be constructed starting between 1 and 500 that I’ve found. It is remarkable that anyone discovered that at all, and also remarkable that it doesn’t happen to start at 14,562 instead of 82 — I’m sure nobody would have noticed this were any number much higher than 82 the magic starting point for constructing a prime this way.

This was fun! If you have any insights, questions, challenges or improvements, please let me know in the comments.

2019.08.3 05:20

Building Erlang 22.0 on Debian/Ubuntu

Filed under: Computing,Science & Tech — Tags: , , — zxq9 @ 05:20

Every time I switch to a new system and have to build a new release of Erlang with kerl I sit and scratch my head to remember which dependencies are required. Once you’re set up or have a prep script it is just too easy to forget which thing is needed for what over the next few years.

Here is my list of pre-build package installs on Ubuntu 18.04 — note that they are in three groups instead of just being a single long apt install command (why apt couldn’t manage to install these all at once is beyond me…):


  • gcc
  • curl


  • g++
  • dpkg-dev

Group 3:

  • build-essential
  • automake
  • autoconf
  • libncurses5-dev
  • libssl-dev
  • flex
  • xsltproc
  • libwxgtk3.0-dev

2019.06.22 23:20

Social Prophecy

If you want to feel creeped out in 2019…

Backstory setting for the Cyberpunk 2020 game rulebook Neo Tribes: The Nomads of North America, published in 1994.
Sagan nailed it in The Demon Haunted World.

2019.06.20 22:39


Filed under: Science & Tech — Tags: , , , , — zxq9 @ 22:39

SpaceX has made rocket booster landings so routine they are almost boring for anyone other than a space tech geek (but I’m a space tech geek, so there’s that). Blue Origin has a full-blown lunar cargo and machinery ferrying service in the works. The first 60 Starlink satellites have already launched. Acoustic simulation is now essentially a solved problem. VR latencies are finally being measured in single milliseconds. AMD is rapidly approaching its goal of having 1000+ cores available on the consumer end of computing devices.

These aren’t prospective projects, they are in development now and most of the underlying tech already works now. It is an absolutely amazing time to be alive.

And yet today I saw a picture of a US Navy destroyer mounted completely dry, high above the water, on a specialized cargoship… (a “wet” drydock? I have no idea what this is even called) and in spite of all the amazing things going on elsewhere in technology I am totally stunned at this. I’m sure it is just a normal thing for the operators of such craft to see, but wow that’s amazing:

Older Posts »

Powered by WordPress