Archive for June, 2018

Erlang: ZJ v1.0.1 + JSON Test Suite (aka “What happens when you go public”)

Friday, June 29th, 2018


ZJ has been run through the JSON Test Suite at the recommendation of Michał Muskała, patched for compliance to all the cases required (and many optional behaviors), and now updated to v1.0.1. Complete results on this ZJ wiki page.


When I wrote ZJ my intention was only to scratch my own little itch: I needed a tiny, portable, single module JSON encoder/decoder for use on a relatively restricted set of data. I wanted a particular mapping between types, and writing it to work the way I wanted was easy.

I announced it when I felt meaningfully “done” because I thought someone else might like it. Some did. Some had suggestions. Some folks wondered where the tests were or if I was going to include any. I didn’t give these things much thought because it did what I wanted it to do already.

But then Oleg Tarasenko — someone I know well, have worked with in the past and respect very much — made a public query. In short, “Hey, what’s the plan with this? I want to use it, too.” Well crap. I don’t really have the time — or rather, I have other things I’d rather spend my time on but it’s Oleg and so I suddenly care again. So now I’m roped into something — maybe not hand-writing a bunch of tests or cranking up PropEr (which I’ve never been good at teaching how to create structured string input like JSON), but since the wonderful JSON Test Suite project already exists I can at least validate it as a JSON decoder using its pre-built example data.

With everything fresh in my mind it only took a few hours to clean everything up to the point that ZJ is actually now fully compliant with all the “must” features, many of the implementation specific features, and a few extensions that are technically illegal but occur frequently in practice.

That’s the sort of thing that happens when you share you work. It starts off as a “works for me” tool, and then your friends get a hold of it. Because you’ve known them a while and trust them you care a lot about what they have to say — even when they are just making gentle recommendations (or maybe especially when they are making gentle recommendations). The thing is, your friends are your friends because they have your best interests at heart, and in a software community just as in a neighborhood community very often your best interests are also their best interests. In fact, having another option for JSON parsing is better for everyone because there really is no One True Mapping, and it is especially good for everyone if a core piece of generic infrastructure has as permissible a license as possible (thanks, Marc) and the project can verify its claims to correctness — or at least vet its level of wrongness.

Zj is a tiny project, but even such a small project demonstrates the positive dynamic that exists among people within a community. So thanks Oleg Tarasenko, Michał Muskała, Loïc Hoguin, and others who took the time to discuss the vagaries of JSON and prod me a little to clean things up a bit more. I’m much happier with v1.0.1 than v1.0.0 (and will like v1.0.2 even more because of this and that).

Appeal to New Coders

Joe Armstrong convinced me about two years ago that it was better to release half-grown ideas to let them germinate and grow in the light of the sun than rot and be forgotten in the darkness of obscurity and personal temerity. Once again his position is validated. If you’re just getting into FOSS for the first time, release stuff. Don’t pester folks about it, but do announce your work so that eyes other than yours can get on it. Nothing but good comes of that (but sometimes it can be embarrassing — because you’ll occasionally be wrong, or lazy, or wrong because you were lazy…).

Hayabusa2: Approach to Ryugu timelapse

Friday, June 29th, 2018

GIF of the year.

Erlang: R21 doc mirror

Wednesday, June 27th, 2018

Erlang doc mirror for R21 is now up.
(For those times when takes a nap…)

Erlang: ZJ docs

Wednesday, June 27th, 2018

Docs for the ZJ Erlang JSON encoder/decoder are now available here:

The binary_encode/1 function will probably be live tomorrow, along with a proper v1.0 release.

Your tests don’t tell you what you think they do

Tuesday, June 26th, 2018

Yesterday I wrote a tiny JSON encoder/decoder in Erlang. While the Erlang community wasn’t in dire need of yet another JSON parser, the ones I saw around do things just a tiny bit differently than I want them to and writing a module against RFC-8259 isn’t particularly hard or time consuming.

Someone commented on (gasp!) the lack of tests in that module. They were right. I just needed the module to do two things, the code is boring, and I didn’t write tests. I’m such a rebel! Or a villain! Or… perhaps I’m just someone who values my time.

Maybe you’re thinking I’m one of those coding cowboys who goes hog wild on unsafe code! No. I’m not. Nothing could be further from the truth. What I have learned over the last 30 years of fiddling about with software is that hand-written tests are mostly a waste of time.

Here’s what happens:

  1. You write a new thingy.
  2. You throw all the common cases at it in the shell. It seems to work. Great!
  3. Being a prudent coder you basically translate the things you thought to throw at it in the shell into tests.
  4. You hook it up to an actual project you’re using somewhere — and it breaks!
  5. You fix the broken bits, and maybe add a test for whatever you fixed.
  6. Then other people start using it in their projects and stuff breaks quite a lot more ZOMG AHHH!

Where in here did your hand-written tests help out? If you write tests to define the bounds of the problem before you actually wrote your functions then tests might help out quite a lot because they deepen your understanding of the problem before you really tackle it head-on. Writing tests before code isn’t particularly helpful if you already thoroughly understand the problem and just need something to work, though.

When I wrote ZJ yesterday I needed it to work in the cases that I care about — and it did, right away. So I was happy. This morning, however, someone else decided to drop ZJ into their project and give it a go — and immediately ran into a problem! ZJ v0.1.0 returns an error if it finds trailing commas in JSON arrays or objects! Oh noes!

Wait… trailing commas aren’t legal in JSON. So what’s the deal? Would tests have discovered this problem? Of course not, because hand-written tests would have been bounded by the limits of my imagination and my imagination was hijacked by an RFC all day yesterday. But the real world isn’t an RFC, and if you’ve ever dealt with JSON in the wild that you’re not generating you’ll know that all sorts of heinous and malformed crap is clogging the intertubes, and most of it sports trailing commas.

My point here isn’t that testing is bad or always a waste of time, my point is that hand-written tests are themselves prone to the exact same problems the code being tested is: you wrote it so it carried flaws of implementation, design and scope, just like the rest of your project.

“So when is testing good?” you might ask. As mentioned earlier, those cases where you are trying to model the problem in your mind for the first time, before you’ve written any handling code, is a great time to write tests for no other reason than they help you understand the problem. But that’s about as far as I go with hand-writing tests.

The three types of testing I like are:

  • type checks
  • machine generated (property testing)
  • real-world (user testing)

A good type checker like Dialyzer (or especially ghc’s type system, but that’s Haskell) can tell you a lot about your code in very short order. It isn’t unusual at all to have sections of code that are written to do things that are literally impossible, but you wouldn’t know about until much later because, due simply to lack of imagination, quite often hand-written tests would never have executed the code, or not in a way that would reveal the structural error.
Typespecs: USE THEM

Good property testing systems like PropEr and QuickCheck generate and run as many tests as you give them time to (really, it is just constrained by time and computing resources), and once they discover breakages can actually fuzz the problem out to pinpoint the exact failing cases and very often indicate the root cause pretty quickly. It is amazing. If you ever experience this you’ll never want to hand write tests again.
Property Testing: USE IT

What about user testing? It is simply necessary. You’ll never dream up the insane stuff to try that users will, and neither will a property-based test generation system. Your test and development environment will often bear little resemblance to your users’ environments, the things you might think to store in your system will rarely look anything like the sort of stuff they will wind up storing in it, and the frequency of operation that you assumed might look realistic will almost never been anywhere close to the mark.

Ultimately, hand-written tests tend to tell a lot more about the author of the tests than the status of the software being tested.

Tiny strings-as-strings JSON in portable Erlang

Monday, June 25th, 2018

There are several JSON libs for Erlang at this point, and as there is no correct mapping between JSON types and Erlang types, all make different tradeoffs that either work or don’t for your project. Beyond that, various interface and implementation differences exist due to the tradeoffs inherent in manipulating elements of the Black Tongue known as lolscript:

  • Accept values to encode as magic tagged tuples so you can specify exactly what you want VS being ambiguous
  • Never allow “naked” values (everything must be in a list/array or a map or a [whatever]) VS “hanging” values
  • Treat all strings ever as binaries because “strings are big” VS treating all strings (and binaries) as strings because strings are easy to manipulate (io_lists…)
  • Decode JSON “objects” as proplists VS decode JSON objects to dicts or maps VS add an “options” argument to the decode function
  • Encode and decode values various ways based on optional switches VS “sane defaults” (aka “works for me”)
  • Achieve lolspeed via NIFs and only work on *nix VS maintain portability via pure Erlang
  • etc.

No combination is correct for every situation, hence the proliferation of libraries. In addition to proliferation, something as simple as what is described by RFC-8259 shouldn’t require a 20k LoC dependency to manage, at least not in Erlang of all languages.

The general strings-as-strings + portability tradeoffs were made by mochiweb years ago, with mochijson2 being the go-to JSON parser for lots of projects. Now that “tuple calls” have finally been retired after years of obsolescence and deprecation, mochijson2 is finally giving up the ghost as well (as it was based on tuple calls). As a replacement that makes mostly the same tradeoffs but is arguably simpler, I wrote a single-module JSON encoder/decoder lib. It treats all strings as strings, is in pure Erlang, and is utterly boring in how simple the code is. Nothing magical to see. At all. So don’t get excited.

If you need to read things in and read things out, in JSON, and don’t really care about lolspeed but want to understand what is happening, then ZJ is for you: ZJ project @ gitlab

Note that if you have roughly the same requirements but you want to make the strings-as-binaries tradeoff then JSX is the lib for you.