The Intellectual Wilderness There is nothing more useless than doing efficiently that which should not be done at all.

2018.06.29 15:36

Hayabusa2: Approach to Ryugu timelapse

Filed under: Science & Tech,Space — Tags: , , , , , — zxq9 @ 15:36

GIF of the year.

2018.06.27 00:09

Erlang: R21 doc mirror

Filed under: Computing — Tags: , , , — zxq9 @ 00:09

Erlang doc mirror for R21 is now up.
(For those times when erlang.org takes a nap…)

http://zxq9.com/erlang/docs/reg/21.0/

Erlang: ZJ docs

Filed under: Computing — Tags: , , , , , — zxq9 @ 00:06

Docs for the ZJ Erlang JSON encoder/decoder are now available here:

http://zxq9.com/projects/zj/docs/

The binary_encode/1 function will probably be live tomorrow, along with a proper v1.0 release.

2018.06.26 14:52

Your tests don’t tell you what you think they do

Filed under: Computing,Science & Tech — Tags: , , , , , , , , — zxq9 @ 14:52

Yesterday I wrote a tiny JSON encoder/decoder in Erlang. While the Erlang community wasn’t in dire need of yet another JSON parser, the ones I saw around do things just a tiny bit differently than I want them to and writing a module against RFC-8259 isn’t particularly hard or time consuming.

Someone commented on (gasp!) the lack of tests in that module. They were right. I just needed the module to do two things, the code is boring, and I didn’t write tests. I’m such a rebel! Or a villain! Or… perhaps I’m just someone who values my time.

Maybe you’re thinking I’m one of those coding cowboys who goes hog wild on unsafe code! No. I’m not. Nothing could be further from the truth. What I have learned over the last 30 years of fiddling about with software is that hand-written tests are mostly a waste of time.

Here’s what happens:

  1. You write a new thingy.
  2. You throw all the common cases at it in the shell. It seems to work. Great!
  3. Being a prudent coder you basically translate the things you thought to throw at it in the shell into tests.
  4. You hook it up to an actual project you’re using somewhere — and it breaks!
  5. You fix the broken bits, and maybe add a test for whatever you fixed.
  6. Then other people start using it in their projects and stuff breaks quite a lot more ZOMG AHHH!

Where in here did your hand-written tests help out? If you write tests to define the bounds of the problem before you actually wrote your functions then tests might help out quite a lot because they deepen your understanding of the problem before you really tackle it head-on. Writing tests before code isn’t particularly helpful if you already thoroughly understand the problem and just need something to work, though.

When I wrote ZJ yesterday I needed it to work in the cases that I care about — and it did, right away. So I was happy. This morning, however, someone else decided to drop ZJ into their project and give it a go — and immediately ran into a problem! ZJ v0.1.0 returns an error if it finds trailing commas in JSON arrays or objects! Oh noes!

Wait… trailing commas aren’t legal in JSON. So what’s the deal? Would tests have discovered this problem? Of course not, because hand-written tests would have been bounded by the limits of my imagination and my imagination was hijacked by an RFC all day yesterday. But the real world isn’t an RFC, and if you’ve ever dealt with JSON in the wild that you’re not generating you’ll know that all sorts of heinous and malformed crap is clogging the intertubes, and most of it sports trailing commas.

My point here isn’t that testing is bad or always a waste of time, my point is that hand-written tests are themselves prone to the exact same problems the code being tested is: you wrote them so they carry flaws of implementation, design and scope, just like the rest of your project.

“So when is testing good?” you might ask. As mentioned earlier, those cases where you are trying to model the problem in your mind for the first time, before you’ve written any handling code, is a great time to write tests for no other reason than they help you understand the problem. But that’s about as far as I go with hand-writing tests.

The three types of testing I like are:

  • type checks
  • machine generated (property testing)
  • real-world (user testing)

A good type checker like Dialyzer (or especially ghc’s type system, but that’s Haskell) can tell you a lot about your code in very short order. It isn’t unusual at all to have sections of code that are written to do things that are literally impossible, but you wouldn’t know about until much later because, due simply to lack of imagination, quite often hand-written tests would never have executed the code, or not in a way that would reveal the structural error.
Typespecs: USE THEM

Good property testing systems like PropEr and QuickCheck generate and run as many tests as you give them time to (really, it is just constrained by time and computing resources), and once they discover breakages can actually fuzz the problem out to pinpoint the exact failing cases and very often indicate the root cause pretty quickly. It is amazing. If you ever experience this you’ll never want to hand write tests again.
Property Testing: USE IT

What about user testing? It is simply necessary. You’ll never dream up the insane stuff to try that users will, and neither will a property-based test generation system. Your test and development environment will often bear little resemblance to your users’ environments (a few weirdos out there still use Windows!), the things you might think to store in your system will rarely look anything like the sort of stuff they will wind up storing in it (you were thinking text, they were thinking video), and the frequency of operation that you assumed might look realistic will almost never been anywhere close to the mark (your one-off utility program that you assumed would run in isolation initiated by a user command in ~/bin/ may become the core part of a massively parallelized service script executed every minute by a cron job running as root).
Your Users: COMMUNICATE WITH THEM

Ultimately, hand-written tests tend to reveal a lot more about the author of the tests than the status of the software being tested.

2018.06.25 21:03

Tiny strings-as-strings JSON in portable Erlang

There are several JSON libs for Erlang at this point, and as there is no correct mapping between JSON types and Erlang types, all make different tradeoffs that either work or don’t for your project. Beyond that, various interface and implementation differences exist due to the tradeoffs inherent in manipulating elements of the Black Tongue known as lolscript:

  • Accept values to encode as magic tagged tuples so you can specify exactly what you want VS being ambiguous
  • Never allow “naked” values (everything must be in a list/array or a map or a [whatever]) VS “hanging” values
  • Treat all strings ever as binaries because “strings are big” VS treating all strings (and binaries) as strings because strings are easy to manipulate (io_lists…)
  • Decode JSON “objects” as proplists VS decode JSON objects to dicts or maps VS add an “options” argument to the decode function
  • Encode and decode values various ways based on optional switches VS “sane defaults” (aka “works for me”)
  • Achieve lolspeed via NIFs and only work on *nix VS maintain portability via pure Erlang
  • etc.

No combination is correct for every situation, hence the proliferation of libraries. In addition to proliferation, something as simple as what is described by RFC-8259 shouldn’t require a 20k LoC dependency to manage, at least not in Erlang of all languages.

The general strings-as-strings + portability tradeoffs were made by mochiweb years ago, with mochijson2 being the go-to JSON parser for lots of projects. Now that “tuple calls” have finally been retired after years of obsolescence and deprecation, mochijson2 is finally giving up the ghost as well (as it was based on tuple calls). As a replacement that makes mostly the same tradeoffs but is arguably simpler, I wrote a single-module JSON encoder/decoder lib. It treats all strings as strings, is in pure Erlang, and is utterly boring in how simple the code is. Nothing magical to see. At all. So don’t get excited.

If you need to read things in and read things out, in JSON, and don’t really care about lolspeed but want to understand what is happening, then ZJ is for you: ZJ project @ gitlab

Note that if you have roughly the same requirements but you want to make the strings-as-binaries tradeoff then JSX is the lib for you.

 

2018.05.30 01:00

Erlang: Eventually Things Will Change

Filed under: Computing,Science & Tech — Tags: , , , , , , , — zxq9 @ 01:00

I finally got a few days to really dedicate to the whole Zomp/ZX thing and wrote some docs.

If you actually click this link soon you’ll see an incomplete pile of poo, but it is a firm enough batch of poo that I can show it now, and you can get a very basic idea what this system is supposed to do:

Zomp/ZX docs

Some pages are missing and things are still a bit self-conflicted. The problem is that until you really use a system like this a bit it is hard to know what the actual requirements need to be. So that’s been a long internal journey.

If my luck holds I’ll have something useful out in short order, though. Here’s to keeping fingers crossed and creating useful on-ramps for new programmers in desperate need of easy-to-use power tools. While we can all only hope the gods will help them when it comes to tackling their actual human-relevant problems, the environment in which they render their solutions should not be actively hostile.

New Privacy Notices: Nothing Changes

Filed under: Computing,Politics / Geopolitics,Society — zxq9 @ 00:54

My anti-comment-spam plugin called Akimset, which has saved me the trouble of reviewing about two-million spam comments over the last several months, now requires me to display a privacy notice at the bottom of all my comment-enabled posts.

The People are now Protected. Sort of like how ancestral British are now protected from the likes of Tommy Robinson. Lovely.

But in the real world, none of this matters. The TLS certificate scheme in which users of nearly every allegedly secure bit of the web base their trust is itself a sham of epic proportions. Branch prediction causes amazing things to happen if you’re running on a physical system with any code on it you don’t trust also running (oh, haha, sounds like using browsers or, especially, using the “cloud”). Don’t even get me started on what the not-as-bad “they” are going to find when they dig into out-of-order execution.

And none of this is new. It has been that way since the beginning, people are only just now finding out. Somewhat hilarious.

As always: “possession is 90% of the law”. This applies in an absolute way. Keep your important bits close, physically. Be able to point to them. Be able to point to the man who is responsible for backing it up, keeping your off-site backups, etc.

2018.05.12 15:13

Language is deeper than mere communication

Filed under: Society — Tags: , , , , , , , , — zxq9 @ 15:13

I visited a behavioral psychologist friend’s home in the U.S. about a year ago. While there I got a call from someone back home. We had been speaking English but of course my phone call was in Japanese. He mentioned to me that very often when a person who speaks several languages associates a language strongly with a specific culture, place and social group that person actually changes when they switch languages for more than about a sentence. He mentioned this because he watched me closely while I was on the phone and found the Jekyll/Hyde thing interesting to observe first hand.

The whole idea seemed very odd to me at the time, but he insisted that inside of us are a bunch of different flavors of our own psyche, or several semi-conflicted psyches all cohabiting — and our external personality is a sort of amalgamated manifestation of psychological combinations we find appropriate for a given situation. Or something like this. I’m not a psychologist, so this is probably a horrible mangling of an idea he explained quite succinctly to someone (me) that doesn’t grok the first thing about this.

He went on to explain that (again, bad paraphrasing) language is the gateway to many thought processes, because at very high levels of consciousness we abstract complex ideas behind words, even in our own heads most of the time, and that unspoken context and meaning carries a lot of weight as well — but that since we cannot “hear” this context in the monologues of our mind, we just don’t give it much conscious consideration. So basically, switching languages also switches the context of your thinking to some degree, and context is how you pick which flavors of your psyche are appropriate to manifest at a given moment, and so on. So switching languages also makes you sort of switch programs in your head.

Anyway, he’s an expert, so I take this into consideration, and that was that. Fascinating idea, isn’t it?

I just realized today something interesting when a friend (who doesn’t speak Japanese) asked me “what is takoyaki” and I realized that, explained in proper English, it doesn’t sound very good. “Sort of like hushpuppies or donut-holes with pickled ginger in the batter and octopus pieces inside” is a rather unsatisfying explanation. It only gets worse if you explain what 鰹節 is (and people think how it is prepared looks like carpentry — which is only funny because it is true).

But! If I explain using a few loan words, it is great: “Like a hushpuppy with beni-shoga and tako inside!” See? No trouble.

「タコ」 and 「イカ」 sound delicious. “Octopus” and “squid” not so much. That’s probably why Americans order “fried calamari” instead of saying “fried squid” — scores way lower on the Captain Nemo’s Dinner Horror Scale.

2018.04.26 21:29

How to Exit Vim (or vi)

Filed under: Computing — Tags: , , , , , — zxq9 @ 21:29

I never thought I would write a post about this, but evidently it is a problem, so here we go.

The Procedure

  1. Realize you’re in a parallel universe where “bogomips” is a word and computrons and thunks are the competing currencies.
  2. Freak out, probably pressing something awkward in the process, and then in a paralyzed daze watch as help screens, text versions of your favorite recipe website, prayers to Cthulu, and descriptions of long missing persons flash by.
  3. Press <ESC> about a half dozen times in a rush of maniacal panic.
  4. Enter the sequence: “:q!” and hit <ENTER>.

Note that step #3 is actually necessary.

An alternative to #4 is to press the capital letters “ZQ”. (Note: capital letters. Not lower-case. That would mean something totally different to vim. Duh.)

But WATCH OUT! If you have accidentally opened multiple buffers in your moment of weakness (step #2) then your victory stroke would have slain naught but one of the portals to Oblivion and, just like that game where you play a DIY avatar that you’ll never manage to edit into looking anything like anyone who has ever lived, the only way to save Tamriel once and for all (yeah, right) is to continue typing “:q!<ENTER>” over and over until each and every gate to madness has been closed, you are proclaimed hero and the Oblivion Crisis is finally over.

Er, I mean, until you’ve exited vim all the way.

Anyway, “successfully exiting vim” is evidently a tooth-cutting experience and lends tremendous street cred among node.js and Elixir programmers, so hopefully this explanation will help you out when/if the time comes and Fate dares to test your mettle.

Explanation

The first thing to understand is that vim (a.k.a. “The One True Text Editor”) has modes.

When you start vim it will be in “command mode”. In command mode anything you enter is a command: opening files, yanking lines of text, constructing match sequences or whatever else. Command mode is often called “beep mode” because you can always return to it by pressing <ESC> a bunch of times, pretty much no matter what else is going on, and if you’re already in command mode when you press <ESC> it will beep at you — hence the habit of doing something, it not working out, and automatically diddling <ESC> about four or five times until it starts beeping at you.

Other modes include “edit mode” where you can do things like type (the way you expect to do in a text editor), visual mode, and a few others that you either already know or definitely don’t if you are here reading this article. From any other mode, all of them being something other than beep mode, you can press <ESC> a bunch and get back to beep mode, which is where you want to be if your goal is to exit vim.

Once you’re in beep mode, you can exit abruptly (without saving anything) by typing a colon (to enter a command), the letter “q” (to “q”uit) and an exclamation mark to indicate “do it now no matter what!”.

2018.04.15 21:44

Erlang: R20.3 doc mirror

Filed under: Computing — Tags: , , , , , — zxq9 @ 21:44

The main Erlang website has been super snappy for the last several months so I had slacked off on mirroring the documentation. Today it seems there are a few problems (DDOS-type symptoms, but I have no idea what is going on) so I’ve gone ahead and mirrored the R20 docs.

I also updated the “Erlang Stuff” page — though that page is going to get a few more changes once the Erlang tooling suite I’m working on is out, as now tinder, firekit, flint and zx are now all incorporated into the new (much better) thing… but more on that thing later once I’m done.

« Newer PostsOlder Posts »

Powered by WordPress