Computers were in wide use by the late 1960s. They were called mainframes, and they were large machines usually tucked away in the basements of large companies or universities. Computers began life serving two functions: handling large databases (people, financial accounts, etc), and carrying out scientific calculations. But people recognized the great potential of computers as an educational medium by the early 1960s.
Computers improved by leaps and bounds. Their power grew rapidly, and the software used on them grew ever more sophisticated. But despite many admirable efforts, the educational software never quite seemed to take off.
In the late 1970s a new computer revolution started: microcomputers, later called personal computers or home computers. These smaller machines didn’t have as much power as the mainframes, but because millions were sold, inducing millions of people to mess around with them, an avalanche of new ideas developed. The big surprise was the popularity of computer games. This new entertainment form mushroomed into a large industry during the 1980s.
Meanwhile, the computer scientists were making dramatic advances in language design, operating system architectures, database managers, and artificial intelligence. But educational software limped along, growing more elaborate as the hardware improved, but never truly taking off.
Nowadays, our computers are immensely powerful and they are supported by oceans of software. They handle a huge range of services affecting much of our lives. Think of all the fields that have been revolutionized by computers: writing letters, papers, and books; creating beautiful images; composing music; solving scientific problems; communicating with people all over the planet by text, sound, or video; making movies; purchasing products; watching movies; and playing games. Of course, the amalgamation of masses of data about millions of people has generated huge new opportunities, both beneficial and frightening. Artificial intelligence is making gigantic leaps in its ability to recognize faces, understand our spoken words, and other astounding feats.
Yet amid all of this huge progress, our use of the computer as a medium of expression has not advanced. The most telling evidence comes from the field of educational software. Despite billions of dollars invested in educational software, despite the fact that it looks better, sounds better, and plays better than ever, it just hasn’t done much in terms of replacing traditional educational technologies. By rights, educational software should by now have done to schools what Amazon.com did to bookstores, what Netflix is doing to television networks, and what smartphones have done to landline phones. But it hasn’t.
The failure shows up most clearly in spending figures. Globally, we spend about $4.4 trillion on educational products annually. This figure does not include teachers or schools; it is only the size of the industry that caters to educational institutions. Educational software captures just $6 billion of this figure, about 0.14% of the total.
We have failed in our efforts to use the computer as a medium of expression. It can calculate, it can sort, crunch, list, and all manner of other things, but we’re just not using it for the expression of ideas, whether artistic or educational.
You will of course point to the many ways that computers have improved work in traditional media. Nobody writes a book with a typewriter anymore. Photography has gone digital, and digital post-processing is a standard part of the photographer’s work. Movies rely heavily on computers. All over the world, amateurs are using computers to create images, animations, music, and text that are much better than what was possible just thirty years ago.
But these are not using the computer as a medium of expression; they are using it as a tool for old media of expression. Their output is still literature, music, or video — media that have been around for more than a century. Nothing they have produced could not have been done (theoretically) a hundred years ago. I’m not denigrating their work — it’s great! But we must be honest in acknowledging that the computer is used as a tool. There is nothing in these beautiful works that could not (theoretically) have been done without a computer.
But let’s not forget games! Surely these contradict my thesis. They are unquestionably creatures of the computer; without a computer, they could not exist. And they are surely artistic expressions, right?
I don’t think so. There’s no question that games are entertaining, and there’s no question that some games have artistic elements included. But on closer examination, the artistic component of games proves to be non-computer. Lots of games have impressive stories built into them — but the stories are not part of the gameplay. They are little novels or movies tacked onto the game. They are not the heart and soul of the game; they are a dramatic veneer to the same old game mechanics that we have been playing for decades. Once you strip away all the impressive cosmetic features, modern games are played in a manner not much different from the gameplay of thirty years ago.
Crawford’s First Rule of Software Design is “Always ask, ‘What does the player DO?’” It is the player’s actions that comprise the heart and soul of the game, and the player’s actions in modern games are not much different from those of times of yore. Those actions do not address anything of artistic merit. There’s lots of art surrounding the game, but none in it. They are like a thaumatrope, a device that creates the illusion of two images merging into one. In a game, we alternate between an artistic story or expression and a non-artistic action. But alternating between the two does not unite them.
Step way back and look at the entire world of computer software. You cannot find any software that is artistically interesting. There are a great many output files — images, music, videos, stories — that are artistically impressive. But there are no artistically meritorious programs out there.
To summarize: software designers have been fabulously successful in exploiting the power of the computer for a huge range of activities. But they have failed badly in using the computer as a medium of expression. That’s the problem this course attempts to address.
An older essay addressing this same issue.
Why We Failed
There’s a simple explanation for our failure to utilize the computer as a medium of expression: we’re not using interactivity, and interactivity is the essence of the computer as a medium of expression. The point is best made by reviewing the parade of the most successful genres of computing. For the purposes of this analysis, I divide the history of computing into three eras.
Before the Internet (1978 - 1992 or so)
Back then, computers were completely standalone; they weren’t connected to other computers. Nevertheless, there were four clear classes of software that were immensely popular and sold zillions of copies:
1. Word Processing Software
Nowadays we take word processing for granted, largely because it has been around for so long. But back in the 1980s, word processing was revolutionary stuff. Before this software came along, you had to write everything down in longhand or use a typewriter. In both cases, mistakes were a major pain. In longhand, you’d cross out the mistake and write over it. On a typewriter, you backed up and used a product called “WhiteOut”, which was a fast-drying white paint, to paint over the error. The result wasn’t particularly clean, but it was acceptable. But if you had to make major changes in the document, you simply threw away your work and started over. It was frustrating. Getting things down on paper was a huge hassle, slow and error-prone. You young whippersnappers who have been using word processors all your lives don’t appreciate how big a step forward they represented.
Word processing didn’t just remove the tedium from getting words down on paper; it introduced a completely new style. Instead of carefully planning the entire document from scratch and preparing an outline, you just starting typing — making changes as you went. You could try writing your letter one way, then read it, and if it didn’t sound right, you could re-arrange it, try a different wording, add some new material in the middle of the letter. You could play around with the letter until it was satisfactory.
That was the key that made word processing so powerful: the ability to play around with the text to get it right: to INTERACT with it. Interaction made all the difference in the world. If you could somehow delete all the interaction from a word processor, you’d end up with something no better than a typewriter.
2. Spreadsheets
These don’t attract much attention outside of business these days, but in businesses, spreadsheets are as basic as desks and paper. During the 1980s, spreadsheets were the topic of much awe and wonderment. That’s because a spreadsheet allowed you to mess around with the numbers going into the calculation. You could explore “What if” scenarios. What if the cost of a crucial part increases by 10%? What if sales has a bad quarter and we get only 90% of the sales we projected? What if the interest rate goes down? You could play with all these numbers to explore the possibilities of a business venture, getting a much better idea of the risks and benefits. No longer did you have to rely on gut instincts to make crucial decisions. And the trick that made it all possible was the ability to play with the numbers — to INTERACT with them. If you could somehow magically remove the interactivity from a spreadsheet, you’d end up with something no better than a pencil and paper.
3. Games
Now we move on to games, which exploded in the 1980s from a craft to a major business. It should be obvious to all that the appeal of games lies in the interaction they provide. To put it more sharply, we can imagine the opposite situation: a game without any interaction is, well, dead. It’s not a game at all.
4. Painting/Drawing Software
MacPaint and MacDraw started it. There had been a few similar programs before then, but the low resolution of the displays and the impossibility of printing the results made such programs all but useless. MacPaint and MacDraw were teamed with the ImageWriter, a printer designed to print by the pixel rather than by the character. You could create an image in MacPaint and print a perfect copy of the image.
MacPaint
MacPaint, MacDraw, and MacWrite were the commercial introduction of the concept of “WYSIWYG” (What You See Is What You Get) developed at Xerox PARC. The significance of WYSIWYG is that it made the interaction between user and document direct and immediate. Before them, interaction with the computer was indirect: you gave the computer commands that made it do things; the command you gave seldom had any visual connection with the result. The dramatic improvement in interaction brought about by WYSIWYG proved to be revolutionary.
The Internet Before Smartphones (1992 or so to 2010 or so)
The second era is distinguished by the Internet and the World Wide Web. These two innovations triggered huge changes. The first big change was email. You whippersnappers cannot appreciate just how difficult communications were before email. The only fast way to communicate was by the telephone, and as often as not the other party wasn’t able to answer. If you were lucky, you might be able to leave a message, but then you wouldn’t know if or when they heard the message. And every now then answering machines would lose messages. Sometimes they’d return your call when you weren’t there, resulting in a game called “telephone tag” that was played out over the course of days.
Email changed all that. You could be certain that your message got through, and most people checked their email a couple of times a day, so your message was likely to reach the other person fairly soon. You couldn’t attach files like you can today, but email communications were much quicker and more certain than telephone tag. Your communications — your interactions — with other people were speeded up and made more certain. Better interaction was the reason for email’s success.
Another big change was the dramatic increase in public discussion. Now, there had been public discussion groups on bulletin board systems (“BBS”s) since the mid-1980s (does anybody remember CompuServe or BIX?) But the Web brought millions more people into the mix, and discussions of news stories were frequent and intense. All of a sudden, people were able to debate issues in public. Of course, most of these “debates” were really just flamewars, but the level of political interaction among people jumped way up.
Another consequence of the Internet was the ability of otherwise disconnected groups of people with special interests to join forces. Nowadays you can find a group interested in almost any subject. Homemade model tanks; bird feeder construction; armadillo enthusiasts; movie star fans — just about every possible human interest has its expression somewhere on the web. Again, the power of this capability is the interaction it offered with like-minded people.
The downside of this is the ease with which bad people can join together. Every hate group has its web presence; you can find all sorts of pages detailing how to make bombs, poisons, or untraceable guns; you can learn how to commit a variety of crimes; you can share your hatred with like-minded souls. Yes, they’re interacting with each other, too.
We also saw big changes in the way we purchase goods. Amazon and eBay were both started in the late 1990s; they both revolutionized commerce. You could find all sorts of strange and otherwise impossible-to-find things on eBay; it dramatically boosted the economic interactions among people. Amazon was initially confined to books, but with it you could find all sorts of books that you never knew existed. Before Amazon, the best way to find good books was to browse a bookstore, but you always knew that you were seeing only a fraction of good books out there. Amazon dramatically intensified the individual’s interaction with the book publishing industry.
Smartphones and Social Media (2010 — present)
The rise of smartphones and social media has propelled us into an era of even greater interaction. Social media allowed us to interact more frequently and more intensively with friends and family. Social media acted as a kind of public diary, where people could record their daily experiences. Again, the central appeal of social media lay in the big jump in interaction among people.
Smartphones further accelerated interaction. No longer was interaction with others confined to the computer at home or the laptop you lugged around. It was right there in your smartphone, ready for interaction in a flash. In any of the various waits imposed upon you by doctors, dentists, or queues, you could quickly check your smartphone for the latest useless trivia.
Conclusion
The point of all this is that interactivity has always been the magic ingredient behind the runaway success of all the big computer developments. Interactivity is what makes computers and smartphones valuable. It’s not the graphics, it’s not the sound, it’s not the animation — it’s the interaction. That’s the core conclusion that slips past so many designers:
It’s the interaction, stupid!!!
So how do we get more interaction? How do we turn our non-interactive failures (such as educational software) and turn them into interactive successes? The answer to these questions is lengthy and requires a great deal of background preparation. Brace yourself.
