This is a subject worthy of a series of its own, but a short scolding will have to do for now…
The “Web” is not an applications environment. At all. Web security is a joke. The whole idea is a joke. That’s like trying to have “newspaper security” because the whole point of HTTP is untrusted, completely public, unthrottled publication of textual data and that’s it. Non-textual data is linked in, not even native within a document (ever heard of, say, an <img> tag or that whole series that starts <a something=””>?) and various programs that can use HTTP to read HTML and related markup can (or can’t) fetch extra stuff, but the text is the core of it. Always.
People get wrapped around the axle trying to develop applications that run within a browser and always hit funny walls when trying to deliver interactivity. We’ve got a whole constellation of backhacks to HTTP now that are used to pretend that a website can be an application — in fact at this point I think probably more time is spent working within this kludged edge case than any other specific area of computing, and that’s really saying something. It says a lot about the need for an applications protocol that really can do the things we wish the Web could do and it speaks volumes about how little computing is actually understood by the majority of people who call themselves developers today.
If “security” means being selective about who can see what, then you need to not be using the web at all. Websites are all-or-nothing, whether we delude ourselves that we can put layers of “security” backhacks over it to act against its nature or not.
If “security” means being selective about who can make changes to the data presented on a website, you need to make modifications to data by means other than web forms. And yes, I mean you should be doing the “C” “U” and “D” parts of CRUD from within a real program working over a protocol that is made for this purpose (and therefore inherently securable, not hacked up to be sort-of securable like HTTPS) and ditch your web admin interfaces*. That means the Web can be used as a presentation face for your data, as it was intended (well, dynamic pages is a bit of a hack of its own, but it doesn’t fundamentally alter the concept underlying the design of the protocols involved), and your data input and interactivity faces are just applications other than the browser. Think World of Warcraft, the game, the WoW Armory, and the WoW Forums. Very different faces to match very different use cases and security contexts, but all are connected by a unified data concept centered on character.
Piggybacking ssh is a snap and the forgotten ASN.1 protocol is actually easier to write against than JSON or XML-RPC, but you might have to write your own library handler, though this is true even for JSON as well, depending on your environment.
[*And a note here: the blog part of this site is currently running on WordPress, and that’s a conscious decision against having real security based on what I believe the use case to be and the likelihood of someone taking a hard look at screwing my site over. This approach requires moderation and a vigilant eye. My business servers don’t work this way at all and can serve pages built from DB data and serve static pages built from other data, but that’s it — there is no web interface that permits data alteration at all on those sites I consider to be actually important. Anyway, even our verbiage is screwed up these days. Think about how I just used the word “site”. (!= site server service application)]
I can hear you whining now, “But that requires writing real applications that can touch the database or file system or whatever and that’s <i>hard</i> and requires actual study! And maybe I’ll have to write an actual data model instead of rolling with whatever crap $ORM_FRAMEWORK spatters out and maybe that means I have to actually know something about databases and maybe that means I’ll discover that MySQL is not good at handling normalized data and that might lead me to use Postgres or DB2 on projects and management and marketing droids don’t understand things that aren’t extensively gushed over by the marketing flax at EC trade shows and… NOOOoooo~~~!” But I would counter that all the layers of extra bullshit that are involved in running a public-facing “Web 2.0” site today are way more complicated, convoluted and extremely removed from a sane computing stack and therefore, at this point, much more involved, costly and “hard” to develop and maintain than real applications are.
In fact, since all of your “web applications” are built on top of a few basic parts, but in between most web developers use gigantic framework sets that are a lot larger and harder to learn and way shorter-lived than the underlying stack, your job security and applications security will both improve if you actually learn the underlying SQL, protocol workings, and interfaces involved in the utility constellation that comprises your “application” instead of just leaving everything up to the mystery of whatever today’s version of Rails, Dreamweaver or $PRODUCT is doing to/with your ideas.
I’m not bashing all frameworks, some of them are helpful and anything that speeds proper development is probably a good thing (and “proper” is a deliberately huge and gray area here — which is why development is still more art than science and probably always will be). I am, however, bashing the ideas that:
- Tthe web is an applications development environment
- Powertools can be anything other than necessarily complex
At this point the mastery burden for web development tools in most cases outstrips the mastery burden for developing normal applications. This means they defeat their own purpose, but people just don’t know that because so few have actually developed a real application or two and discovered how that works. Instead most have spent lots of time reading up on this or that web framework and showing off websites to friends and marks telling them they’re working on a “new app”. Web apps feel easier for reasons of familiarity, not because they are actually less complex.
Hmm… this is a good sentence. Meditate on the last sentence in the above paragraph.
The dangerously seductive thing about web development is that the web is very good at giving quick, shallow results measurable in pixels. The “shallow and quick” part being the dangerous bit and the “pixels” being the seductive bit. There is a reason that folks like Fred Brooks has insinuated that the drive to “just show pixels” is the bane of good programming and also called pixels and pretty screens “chicken lipstick”. Probably the biggest challenge to professional software development is the fact that if you’re really developing original works screens are usually the last thing you have to show, after a huge amount of the work is already done. Sure, you can show how the program logic works and you can trace through routines in an interpreter if you’re using a script language somewhere in the mix like Python or Scheme, but those sorts of concrete progress from a programmer’s perspective don’t connect to most clients’ concerns or imaginations. And so we have this drive to show pixels for everything.
Fortunately I’m the boss where I work so I can get around this to some degree, but its extremely easy to see how this drive to show pixels starts dicking things up right from the beginning of a project. Its very tempting to tell a client “yeah, give me a week and I’ll show you what things should look like”. Because that’s easy. Its a completely baseless lie, but its easy. As a marketer, project manager, or anyone else who contacts clients directly, it is important to learn how to manage expectations while still keeping the client excited. A lot of that has to do with being blunt and honest about everything, and also explaining at least a little of how things actually work. That way instead of saying “I’ll show you how it will look” you can say “I’ll show you a cut-down of how it will work” and then go write a script that does basically what your client needs. You can train them to look for the right things from an interpreter and showing them a working model instead of just screens (which are completely functionless for all they know) as a developer you get a chance to do two awesome things at once: prototype and discover where your surprise problems are going to probably come from early on, and if you work hard enough at it write your “one to throw away” right out the gate at customer sale prototyping time.
But web screens are still seductive, the only thing most people read books about these days (if they read at all) and the mid-term future of general computing looks every bit as stupid, shallow and rat-race driven as the Web itself.
I can’t wait to destroy all this.
Haha, just kidding… but seriously…
Interesting writing, I agree on almost everything.
One thing is, sometimes you simply can’t see the problems ahead until reaching there. That is a hard task to predict in developing and that’s why the web might have become what it’s today – full of hacks. It’s always easier to be smart at the end, but harder to construct and solve from what we have now to come out best by taking into account a lot of important points.
I believe that most of the web2 generation understands programming and computing from a rather higher layer of view point and they don’t see to the basics and therefore they do what you have written above: using JS for everything which may not need it at all and would be better off without it. (I myself prefer sites without JS at all, I’m just creating a server side solution for a scenario, but completely without JS – I simply don’t care for it).
Interesting question is where this can lead us to? Worse or even better? I used to write apps in assembly and use a lot of different languages and frameworks at different layers. Is it good that I understand the whole stuff how it works to its base? Or can its complexity as a whole make me less open minded? And therefore the new generation knowing less can take it much further much better and change it on their own as they like? Because they like the Web for sure. It gives quick success as you also wrote – and it’s easy to start with. But shouldn’t that be the whole point? What’s better will attract us more, so we get closer to what we like and leave behind which is harder and takes more effort. I think that’s the nature of everything, not everything is always perfect.
Anyway G is planning to redesign the Web from scratch (didn’t take much info on this very topic) and that’s the way the cookie crumbles. Once it was enough, now it’s time to rebuild it.
The basic problem here isn’t that we have higher level tools available. That is always good. The basic problem also isn’t people like you who understand what is happening at the lower level. That is an overwhelming good.
As for being “confined” by knowledge, I really don’t think that happens. You can be confined by details because you are simply overwhelmed by them if you actually try to write everything in assembler, sure, or even C. But you can step away from that with the press of a button and start writing stuff in Clojure or Haskell or whatever other super high level language you want. It doesn’t have to hold you back. But consider the reverse.
The average CS graduate today can’t write a compiler. I see CS graduates today basically re-implementing awk and bison in poor fashion within their own projects because they just don’t know about them — and these are not exactly low level tools, merely ones that they don’t know about but desperately need.
How many times have you seen format handlers written in hundreds of lines of Python that would amount to a few lines of awk? How many times have you seen JS timestamping, result sorting or any other number of silly things written that force data to traverse the network multiple times to get settled that could be handled in a single correct query on the DB side and further, the DB already has built-in commands for (the underuse of the Postgres AGE command comes to mind…)? I hear complaints about execution speed on the Web, but few are doing these older, much faster techniques. Client side scripting can possibly relocate some of the burden, but that’s only if the problem is paralellizable in the first place, and I see almost nobody doing studies on whether or not their web app is, in fact, loaded with juicy, parallelizable problems.
We can observe right now as the no-SQL (and related) movement is rediscovering why relational data is such a good idea, and watching them fumble to reimplement it without using the same vocabulary. Not saying relational data is perfect for every fit, but it fits primary data mappings better than anything else so far discovered, but only if understood.
Anyway, knowing the low level doesn’t constrain you, but not knowing it will always place hard barriers on what you can and can’t do. The high-level guy always depends absolutely on the idea that the low-level guys know what they are doing, but the reverse is not true.
My problem with the Web is that it is designed to be a document publishing medium and we are trying to use it as a universal application interface. If we need an application browser, why not build one? Its entirely possible and would give us a chance to tackle so many of the problems that have plagued web development.
Bouncing outside of a design is acceptable as a hack, but once things are understood it is always better to rethink the problem and determine if you’re solving a fundamentally different problem than you thought you were — and in the case of application services being provided within a document publication protocol this is definitely the case.
Nearly everything you mentioned in the above comment could be viewed as a manifestation of Greenspun’s Tenth Rule of Programming, that any sizable project not written in Common Lisp is destined to recreate at least half of it internally anyway indicating a huge waste of time due either to programmer ignorance or poor choices about tools.