You may have noticed that the programming examples have been getting larger and larger with the passing chapters. This is partly because the ideas I have been presenting have been getting more and more involved, requiring larger and more involved examples. You may also have guessed that real programs must be larger than the examples I am giving, and indeed they are. The size of a program is often measured by the number of lines of code written by the programmer. By this way of measuring, most of the programs you have seen so far are mere 5-line or 6-line programs. Real programs run considerably larger. A modern operating system like that for Macintosh or Windows will run to maybe 100 million lines of code.
What is staggering about so large a program is not the sheer amount of code itself so much as the complexity represented by all that code. A program is not an inert mass of information like a book. Word for word, character for character typed into the computer, a program is a far more complex effort than a book. This is because the words in a book, in comparison to the words in a computer program, are pretty much a loosely connected jumble. The words I chose to use in the last chapter have very little impact on the words I choose for this chapter.
A computer program, by contrast, is an immensely more demanding creature. It acts like a gigantic engine, with thousands of gears and wheels and pulleys, all packed into a very small space, everything very tightly connected. The overwhelming complexity of a huge program is enough to try the courage of any programmer. How can one person, or even a group of people, possibly keep track of this maze of interconnections and relationships?
Ancient Solutions
The problem we face here is not a new one. The creation and maintenance of complex structures has plagued civilization since its earliest days, for a civilization is itself a complex structure requiring maintenance. The task that falls on any government — to regulate commerce, collect taxes, adjudicate disputes — is as complex as the devious ways of its many citizens. The first civilization to develop effective techniques for dealing with these problems was Rome.
What was the source of Roman power? How were the Romans able to first create and then maintain an empire over a span of nearly two thousand years? Historians cite many factors, but a crucial factor often underestimated by the layman is the role of the Roman bureaucracy. We normally think of Roman legions marching across Europe, conquering everything in sight, but a much more important factor in Roman success was the mousy bureaucrat following in the wake of the legion, papyri in hand. Rome did not invent bureaucracy, but the Romans refined and developed the art of bureaucracy far beyond anything the world had known before. Roman administrative skills made it possible to raise, equip, and train the legions that conquered the territories; these same skills insured that the conquered lands were smoothly and efficiently governed. A newly-won territory quickly became a prosperous component of the Empire rather than a poverty-stricken and sullen vassal. Throughout the Empire, a large and efficient bureaucracy coordinated the flow of goods and people, and brought peace and prosperity to a larger area, for a longer time, than the world has known before or since. Such is the power of bureaucracy.
Exactly what is a bureaucracy? Three primary elements determine the form of a bureaucracy. The bureaus themselves constitute the first element. A bureau is a group of people performing a function. A bureau can be a small, one-person operation, or it can be as large as the Department of Defense. The second element is the assignment of functions to bureaus. Each bureau is responsible for a single function, be it broad or narrow. Each function is assigned to a single bureau. The third element is the set of communications procedures within the bureaucracy. The various bureaus must coordinate their actions; to do this requires a clear and simple communication system for transmitting work orders.
These three elements characterize a bureaucracy, but they do not explain its strengths. Why does a bureaucracy work? What is the source of its ability to handle complex problems?
Modularity
One strength of the bureaucracy is its modularity. The bureaucracy is broken up into discrete chunks that are much easier to understand. Consider, for example, the United States government. What is it? Well, we could start off by breaking it into three chunks, the legislative, the executive, and the judicial. Each of those three chunks includes within it a great many people. If you wanted more detail, we could break the executive branch into the various departments (State, Defense, Commerce, Labor, etc). We could then break one of the departments down into its subcomponents, going down further and further. Each module within the structure can be broken down to smaller components, and the modules can be reassembled to form the whole. This breaking down and putting together is one of the big ideas of Western civilization. It parades under the name analysis and synthesis. It is the basis for many of our civilization’s achievements. The bureaucracy is an example of analysis and synthesis applied to large organizations. Take all the problems that we require the US government to handle; break them down into components, assigning each component to a bureau. If a component is itself too large to digest, break it down further into sub-components. Continue this process of breaking down into sub-components as necessary. Once the problems have been broken apart and assigned, allow each bureau to tackle its small problem, then put the pieces together. The result? A Social Security program, an environmental protection policy, or an aircraft carrier.
Analysis and synthesis appears in many other areas. It is fundamental to scientific inquiry. The scientist approaches a complex and little-understood phenomenon and starts by breaking it down into its component aspects, identifying those aspects that can be explained with existing theory and isolating the aspect that represents a mystery. This makes it possible to focus intense attention on the single mysterious item. Once the core problem has been cracked, the components can be reassembled to produce a new theory of stellar evolution, a new chemical, or a cure for cancer.
An engineer follows the same pattern in designing a machine. Break the problem up into components. Put one team of engineers on the fuel system. Have another team tackle the suspension, while a third can worry about engine cooling. Send them off on their respective tasks; when they are done, assemble their work into a new car.
The intelligent hammer
A crucial requirement for successful analysis and synthesis is that the problem be broken up in an intelligent manner. If one attempts to subdivide a problem the way one partitions a vase with a hammer, one gets only a shattered mess. The hidden skill in successful analysis and synthesis is the ability to see clean, natural ways to subdivide the problem. And the basis for clean, natural subdivision, the key criterion, is the simplicity of interaction between the modules.
A problem in analysis and synthesis is essentially a problem of untangling. Suppose that I constructed a tangle of balls connected by springs. Some balls might have many springs attached to them, while other balls might have only one or two springs. How would you go about untangling this mess? If you studied it, you would undoubtedly find at least one group of balls that was tightly interconnected with lots of springs, but connected to other groups of balls by only a single spring. This would form the basis of your untangling effort. You would begin by separating the first group from the main mass. As you pick through the tangle, you would search for easily-separated groups. In short, you would analyze the tangle on the basis of the lowest interaction between groups.
This is the key idea to intelligent analysis. One must scan the problem, looking for patterns that break it up into modules that interact with each other in the simplest way. If each module has but one simple interaction with all other modules, then the situation is highly modular and ideal for analysis and synthesis. If some modules have multiple interactions with other modules, then the situation is less modular and will prove more difficult to handle.
An example is in order. Let’s say that you are a manager in a large corporation and are about to hire a new employee. You have interviewed a number of candidates and have made your decision. To implement it, you merely send a memo to the Personnel Department listing four items: the candidate’s name, the date that this person will start work, the salary to be offered, and the personnel requisition under which the candidate is being hired. The Personnel Department will take care of all the details: notifying the candidate of the job offer, obtaining the candidate’s Social Security number, home address, telephone number, filling out all the forms for the government, opening a personnel file on the candidate, and all the myriad other tasks that are required for employment in a large corporation. Your interaction with Personnel is small and simple: only four items of information are required from you. Yet, those four pieces of information trigger a great deal of work inside Personnel. In short, there are few springs between you and Personnel, and many springs inside Personnel. That’s a highly modular situation.
Just for laughs, let us consider a situation with very low modularity. Suppose, for example, that you were responsible for notifying the government of the candidate’s pay, but Personnel was responsible for notifying the government of the candidate’s claimed deductions. Then both you and Personnel would have to obtain the candidate’s name and Social Security number, and probably an internal employee number. You would need to check your information with Personnel, and they would need to check their information with you, and you would both need to check your information with the candidate. There is plenty of opportunity for a snafu here, with slightly different or inconsistent information being reported to the government. In terms of my tangled springs analogy, this situation has lots of springs running between you and the government, you and Personnel, and you and the candidate. A messy, tangled situation like this makes for a disaster. The essence of good modularit has: lots of internal communication within modules and the absolute minimum of external communication between modules.
There is one other benefit of the highly modular environment: once you have shot off your message to another bureau, you can forget about it. Personnel has their little form, number P-503, that you fill out and send off to them. If you fill it out properly, you need not worry about anything else. They’ll take care of all the little details. Indeed, they are probably taking care of details that you are completely unaware of — new government regulations about hiring, health insurance arrangements, alternate people to contact in an emergency, and so forth. Once a module, or bureau, or engine subassembly has been set up and its inputs determined, you treat it as a black box whose internal workings are of no concern. In future decision-making, you merely tell yourself, “So long as I ship the right inputs, or forms, or whatever, to that module, it will spew out the results I need.” It simplifies your thinking.
Modularity in computers
So what does all this have to do with computers? As it happens, the concepts of modularity, analysis and synthesis, and clear communications procedures are built right into computer programming languages. They are built into what are called Objects. Think of an Object as kind of like a bureau in the bureaucracy of the program.
In JavaScript, everything is an Object! Even the smallest, tiniest parts of a program are Objects. So your task in designing a program is to assemble a bunch of tiny bureaus into bigger and bigger bureaus, each one handling a specific task.
Objects are like… well, everything!
The absolute simplest kind of Object in a computer is a number. Visualize a piece of paper with a number on it:
Simplest possible Object
You already know, however, that it is possible to assemble numbers into an array, which in the real world we call a table. In JavaScript, a table is an Object, too.
A table of numbers
We can get even more complicated, storing a bunch of information about an employee in a file folder:
A file folder with information on an employee
This is an Object, too, but it has a bunch of different information grouped together. JavaScript has a way of grouping a bunch of different kinds of numbers into a single file folder, also called an Object. Next we group the file folder Objects into an even bigger Object:
And that can be grouped in another Object that’s even bigger and contains an even greater variety of things:
A really big Object
Of course, we have almost gotten rid of all those paper Objects; nowadays, most of that stuff sits inside our computers. So when we group all that stuff together, we get a modern office:
A really, REALLY big Object
At this point, I must explain a confusing bit of terminology. There are plain old everyday objects — you know, things — and then there are programming Objects. I try to make the difference clear by capitalizing something that is a programming Object.
But there’s something new in this photo: people. People aren’t objects. For the purposes of this essay, the difference between plain old objects and people is that people do things and objects don’t do anything. Here we come to The Biggest Idea of All: the difference between object and process. This dichotomy underlies almost everything we know. Here is a detailed explanation of the concept. I first wrote about the concept more than thirty years ago; here’s one of my first essays on the subject.
Until we brought people into the picture, objects were dead piles of data. They didn’t do anything. That’s what the people are for: they DO things! In a bureaucracy, information provides the data and people provide the processing. But JavaScript Objects have both data (smaller objects) and processing, using what we call methods. To make matters seem even more confusing at first, methods are also Objects.
{Hint: in JavaScript, everything is an Object: numbers, data, images, methods. When in doubt, call it an Object.}
While methods are Objects, they are Objects of a kind completely different from data-Objects; methods are program code. Again, this is probably a bit confusing, but think of it this way: what good is a bureau without bureaucrats? Every bureau has its papers, its filing cabinets, its computers, and it has its bureaucrats. The bureaucrats are the ones who do the processing of all the data in all those filing cabinets. If you think of a JavaScript program as a bureaucracy, then you have data-Objects that are processed by program-Objects.
The reason for this rather confusing arrangement is exactly the same reason why each bureau has its own set of data and its own set of bureaucrats. In programming theory, we call it encapsulation: keeping together all things (both data and code) that work together, and keeping everything else outside.
Of course, you can’t take encapsulation to its ultimate extreme, in which every bureau (or Object) has absolutely nothing to do with any other bureau (or Object). Bureaus must be able to communicate with each other in order to handle the bigger tasks of the overall bureaucracy. They do this with bureaucratic forms. Here’s a form you’ve probably seen before:
You use this form to communicate your tax information to the Internal Revenue Service of the United States Government. This form organizes all your data in a way that’s easy for the IRS bureaucrats to evaluate. This form even requires you to do a little of the processing (Line 37: “Subtract line 36 from line 22. This is your adjusted gross income.”)
Programming Objects in JavaScript have their own kind of forms; they’re called parameters and you fill them in when you use an Object. However, at this point I must stop, because things get a little complicated here. There are methods, and there are functions, and there are parameters, and there are arguments, and they each have their own precise purpose, but this book is meant to be an introduction for absolute beginners. When you’re ready to get down and dirty with JavaScript, you’ll need an actual textbook. For the purposes of this chapter, just remember that the whole idea of Objects with data and methods is a formalization of the ideas we’ve been using in bureaucracies for hundreds of years.