The Intellectual Wilderness

[Part 2 of a short series on messaging systems. (Part 1)]

Having implemented messaging systems of various sizes and scopes in all sorts of environments, I’ve come up with a few guidelines for myself:

1. If messaging is not the core service, make it an orthogonal network service.
2. If possible make the messages ephemeral.
3. If the messages must persist use the lightest storage solution possible and store as little as possible.
4. Accept that huge message traffic will mean partitioning, partitions will be eventually consistent, and this is OK.
5. You don’t need full text search.
6. If you really do need full text search then use a DB that is built for this — its a major time-sink to hack it in later and get it right.
7. If the messages are threaded annotations over other existing relational data, swallow your pride and consult your DBA.
8. VERSION. Your. DATA. And. PROTOCOLS.
9. If anything about messages over existing data records feels hacky, awkward, or like it might put pressure on the existing DB, separate message storage and accept that some data integrity may be delayed or lost from time to time.
10. Messaging is likely more important to your users than you (or they) think it is.
11. The messages themselves are likely less important to your users than you (or they) think they are.
12. If you can skip a feature, DO.
13. YAGNI.
14. You don’t need [AdvancedMessagingProtocol] (aka XMPP).
15. If you *really* need XMPP it will be painfully obvious (and if you do chances are you certainly don’t need the extensions).
16. [insert more things about avoiding feature creep]

Adding messaging to an existing system can get a little messy if you’re not really sure why you are doing it. If you know that your users really do have a need for messaging within your system, but you don’t know what features or type of messaging it should be, then think carefully about how you would want to use it as a user yourself.

Do users work together within your system to accomplish immediate goals? A concurrent multiuser system (multiplayer game, concurrent design tool, pair programming environment, etc.) benefits most from an ephemeral, instant chat system that centers around the immediate task. When the task is over the messages become meaningless and should be allowed to decay. It may be nice to give users an easy way to export or save significant messages, but that is really a client-side issue and is orthogonal to how the messaging system itself works.

Is the system used to coordinate real-world tasks or events? A real-world coordination system benefits most from a threaded comment/discussion system that centers around those tasks, and must only enhance but not replace the existing task-assignment and tracking features of the system. Threaded annotation is powerful here, but the threads need only persist as long as the task records do and messaging should never be mistaken for a task assignment tool (it can be leveraged as a part of task notification but never assignment or tracking). Remember ON DELETE CASCADE? This is where it is super helpful.

Do users self-organize into groups whose sole purpose is communication? Social group systems benefit most from mail systems implemented directly within the system they use to organize themselves. Such systems may also benefit from some form of ephemeral immediate chat, but it is critical that we keep in mind that that indicates a need for two different messaging systems, not One Message System To Rule Them All.

Many different flavors of message systems exist between the extremes of “persistent point-to-point mail” and “ephemeral, instant, channelized (group) chat”. Consider:

• Persistent chat (Campfire, SocialObstructionist StackOverflow chat, etc.))
• Message boards (“forums” — though this term is far broader in actual meaning…)
• Usenet-style newsgroups
• Mailing list systems
• Email bridges
• IRC + bridges + bots
• Anything else you can imagine…

Some other things to think about are the nature of the users relationship to one another. That’s not just about communication channels and point-to-point delivery issues, it is also about what concept of identity exists within the system. Is the system one with strong authentication, total or partial anonymity, or a hybrid? This will dictate everything about your approach to permissions — from moderation, channel creation and administrative control to whether private messages are permissible and have a huge impact on what the implementation of a messaging system will require in terms of access to the original host system it is intended to support.

The issues of identity, authentication and public visibility are largely orthogonal to questions of persistence duration, storage and message-to-record relationship, but they can become intertwined issues without you realizing it whenever it comes time to design the storage schema, the serialization format(s), or the protocol(s). Of course these are three flavors of basically the same issue — though the modern trend is shy away from thinking about this either by hiding behind HTTP (like, uh, who even knows how to program sockets anymore? zomg!) and sticking your fingers in your ears when someone says that “schemaless JSON is the schema” or that XML can do the job of all three because it is the pinnacle of data representations. Consider whether this may change in the future. Keep YAGNI in mind, but when it comes to schemas, serialization and protocols it is always good to design something that can be extended without requiring core modification.

[Part 1 of a short series on messaging systems. (Part 2)]

Messaging is a feature that seems to wind up on the ZOMG MUST HAVE!!1! feature list long after initial system deployment quite a bit these days. I’ve been getting asked about this a lot lately, so I’ve written down my generic advice here for reference. (If you want more specific advice feel free to contact me, though — its interesting and fun to hear about all the different things people are working on.)

I’ve been implementing and deploying messaging systems in one form or another since I first got involved with computers as a kid in the very late 80’s and early 90’s. Back then most multiplayer games were basically text message dispatch and routing systems with thematic computation surrounding those messages, so this was an area quite a few people my age dealt with — without realizing that it would one day be considered an “enterprise” feature. (And to think, we used to do this in Pascal, and assembler, and C, and hilariously inadequate variants of BASIC… as children! That nonsense was only reasonable to us because they were the only tools we had, those were labors of love, and we didn’t know any better.)

The most important thing to recognize about messaging systems is that the term “messaging” is hopelessly broad. It doesn’t really mean anything by itself. Every over-the-wire protocol is a message system definition, for example. Every drop-it-in-a-spool system is also a messaging system. Every combat notification system in a game is a messaging system. Every websocket thing you’ve ever done is a messaging system. This list could go on for quite a while, but I assume you get the idea.

Messaging systems take on different characteristics depending on their context of use. Some messaging systems are persistent, some are ephemeral, some have selective decay, some are instant, some are asynch, some are channeled, some are global, some are private, some are selective-access, some are moderated, some include extra semantic data that can be interpreted in the message body, some run everything through a central data service (“ooooh, ahhhh, ‘the cloud'”), some are peered, some are anonymous, some are verified, some broadcast, some are point to point, some are free, some are paid in per-message increments, some are paid by aggregate use, etc. The list keeps going as long as you can think of things that can be said about any system, actually.

It is important to keep in mind that most of the adjectives in the last paragraph are not mutually exclusive. Here is the fun part, though: if you need two that are, then you need two messaging systems. (That’s why they make chocolate and vanilla, after all.)

That’s a lot of different aspects to something as conceptually simple as “messaging”. The obvious problem there is that messaging is actually not simple at all in practice. When you decide that you need a “messaging solution” you really must carefully consider what that means for your users and your system. Your users’ state of mind, user experience, utility value of messaging and desire to use messaging in the context of their use of your system are all folded together.

Consider this: Email is a messaging system and we’ve all already got that, why is this not sufficient for your use? If you can’t answer that question quickly then sit back and let it stew for a bit — you will learn something about your users in the process of answering this question because it forces you to think for a moment as a user of your system instead of as a builder of it.

Let’s consider some reasons you might want to add a killer messaging feature to your existing product or system.

• You can’t figure out the difference between “organic pageviews” and traffic resulting from the robot invasion.
• You want more pageviews and you don’t care how you get them — messaging, you think, can be a source of bajillions of clicks!
• Your users deal with data objects about which they constantly converse, but can’t do so within your system.
• Your users have things to express that are best represented in a way that captures the history of their thoughts in sequence.
• Your native applications do exactly what they should, but breaking out of it to send a message is distracting.
• Your system already passes data among users, but requires human annotation to be fully meaningful.
• Your system deals in machine-to-machine messages already, but users still have to call each other on the phone.
• You are looking for a way to attach your brand to one more thing users see every day to reinforce the hypnotic effect you wish your brand had on them.
• You require an anonymous(ish) way for users to communicate out-of-band relative to other online services.

A conditional flow would look something like this:

stupid_reasons = [pageviews,
                  domain_valuation,
                  brand_enhancement,
                  pagerank,
                  BULLSHIT_WEB_IDEAS(SEO && ALL_FLAGS),
                  ADS([ad_search, ad_rank, ad_tech, ad_{*}, ...]),
                  ...];

if (member(your_reason, stupid_reasons)) {
    if (idea_of(You) || you_are(TheManagement)) {
        admit(being_wrong);
        assert cancer_killing_internet = You;
        abort(stupid_idea);
        quick_grieve(dying_product);
    } else if (idea_of(TheManagement)) {
        abort(current_job, GRACEFULLY);
        // It won't last much longer anyway.
    }
    acquire(funding_contacts);
    contact(me, IRL);
    // I've got way better ideas.
    // You should work with me instead of sticking with losers.
} else if (knee_jerk(messaging_feature)) {
    abort(stupid_idea);
    focus(core_features);
} else {
    congratulate(You);
    panic(FEATURE_BURDEN);
    CAREFULLY(implement());
}

quick_grieve(hopeless_thing)
{
    shock(hopeless_thing, DEAD);
    anger(cant_coerce([model, users, investors, world]),
          TANTRUM_LEVEL(11));
    bargain([me, you, imaginary_entities, investors, users, world, HAL],
            perceived_value(hopeless_thing, PERSPECTIVE(MYOPIC)));
    mourn(hopeless_thing);
    accept(DEFEAT);

    return AGGRESSIVE_OUTLOOK([POSITIVE && EXPERIENCED]);
}

Did you find yourself at congratulations(), or somewhere else? Stop and think for a while if your reason belongs in the stupid_reasons list or not. How much will it actually enhance your users’ experience with the product or interaction with each other or you? Are you going to use it yourself, or are you going to continue to use email, IRC, whatever corporate chat service is popular this week, Twitter, etc? (And if you say “Well, I/we don’t use my/our product, so I don’t know…” then you are already rather far beyond admitting defeat, you just don’t know it yet.)

Be painfully honest with yourself here because messaging is one of those things that you can’t add in and then just take away later without consequences. Even if very few people use it and it turns out to be a technical burden to you someone will wind up using it no matter how shitty it turns out to actually be, a few of these people will come to depend on it, and to these people you will be an asshole if you ever remove it. Don’t be an asshole to customers. That’s how you wind up with a “[your_product]sucks.com” — and the kind of users who come to depend on weird internal messaging systems that suck are exactly the kind of people who will register a domain like that and talk shit about you. That crap will stay on the web forever and the stink may stay on your product forever, even if you change your product and they come back loving it.

So let’s assume that your system will exist in a context where it really does add value to the user’s life in some way. goto congratulations()! Messaging will be a big win if done properly and unobtrusively. As a reward for being diligent and thoughtful you now you get to wade through a swamp of design issues. But don’t worry, I’ll give you the fanboat tour and point the way through the muck.

Next we’ll look at some common features of messaging systems, what they mean for your implementation and most importantly what they mean for your users within the context of your system. [Continue to Part 2]