Erlang: Writing Terms to a File for file:consult/1

I notice that there are a few little helper functions I seem to always wind up writing given different contexts. In Erlang one of these is an inverse function for file:consult/1, which I have to write any time I use a text file to store config data*.

Very simply:

write_terms(Filename, List) ->
    Format = fun(Term) -> io_lib:format("~tp.~n", [Term]) end,
    Text = lists:map(Format, List),
    file:write_file(Filename, Text).

[Note that this *should* return the atom 'ok' — and if you want to check and assertion or crash on failure, you want to do ok = write_terms(Filename, List) in your code.]

This separates each term in a list by a period in the text file, which causes file:consult/1 to return the same list back (in order — though this detail usually does not matter because most conf files are used as proplists and are keysearched anyway).

An annoyance with most APIs is a lack of inverse functions where they could easily be written. Even if the original authors of the library don’t conceive of a use for an inverse of some particular function, whenever there is an opportunity for this leaving it out just makes an API feel incomplete (and don’t get me started on “web APIs”… ugh). This is just one case of that. Why does Erlang have a file:consult/1 but not a file:write_terms/2 (or “file:deconsult/2” or whatever)? I don’t know. But this bugs me in most libs in most languages — this is the way I usually deal with this particular situation in Erlang.

[* term_to_binary/1 ←→ binary_to_term/1 is not an acceptable solution for config data!]

9 thoughts on “Erlang: Writing Terms to a File for file:consult/1

  1. Jihyun Yu

    You can use lists:map/2 instead of lists:foldl/3 + lists:reverse/1, for example


    write_terms(Filename, List) ->
    file:write_file(Filename, lists:map(fun(Term) -> io_lib:format("~tp.~n", [Term]) end, List).

    Reply
    1. zxq9 Post author

      @Jihyun Yu
      Indeed! The most recent time I was doing this I had been doing some other things, and just cut the other code out because it worked. Then without thinking I had just reversed the list to make the list retain its order — totally overlooking the fact that I was reimplementing map in the process.

      Just had a good laugh at myself. (^.^)

      Reply
    1. zxq9 Post author

      Because the binary data on disk is not something that an administrator, user or anyone else can edit if your system goes to shit. Its not scriptable with non-Erlang tools… [insert 40-year-old arguments against non-text settings files] and so on.

      Settings should be something that a user who knows nothing about Erlang should be able to modify. You strip them of that when you start putting settings in binary files that only an Erlang program can read. Considering the ease of simply not putting settings in binary files, and that it gains you nothing, there is no reason to contemplate this.

      Reply
  2. Mark Sebald

    This is exactly what I was looking for! Thanks!
    I was considering using term_to_binary() and binary_to_term(), before I found your post.
    I do wonder about that name. I just can’t connect “consult” with reading terms from a file. Oh well as long as it works. Thanks again!

    Reply
    1. zxq9 Post author

      The wording is indeed silly. file:consult(File) doesn’t feel as natural as file:read_terms(File) and file:write_terms(File, Terms). I get the effect they were going for, but what is the reverse operation of file:consult(File)? I have no idea — it paints you into a corner semantically.

      I’m finally working on breaking out a few chunks of the utilities I’ve written into some other projects over the last year as their own library applications — and this pair of functions will be in there alongside my wxErlang meta-widget wrappers, string translation library and other odds and ends.

      term_to_binary/1 < -> binary_to_term/1 is a really great way of sidestepping the establishment of some arbitrary (and almost always lossy) serialization across the network. It lets you easily establish multi-cluster constellations of services composed of arbitrary node-sized clusters. That’s great any place that disterl doesn’t fit (especially when you don’t control all the nodes, due to the implicit trust among nodes in a cluster — you can’t write client-server software as peer nodes, for example) but it just doesn’t quite cut it for serializing data to disk in many cases (settings, options, config, history cache, logging, etc.).

      Reply
  3. Michael Terry

    Note that write_file’s default encoding is latin1, so it will mangle any non-latin1 utf8, as I just found out.

    Reply
    1. zxq9 Post author

      Yes, indeed. There is a utf8 version I actually use in several projects that I should locate and paste here in addition to the above example. Thank you for pointing this out!

      Reply

Leave a Reply to naruto Cancel reply

Your email address will not be published. Required fields are marked *