Uncategorized

Hackers and Painters (2003)

Hackers and Painters

May 2003

When I finished grad school in computer science I went to art school to study
painting. A lot of people seemed surprised that someone interested in computers
would also be interested in painting. They seemed to think that hacking and
painting were very different kinds of work– that hacking was cold, precise,
and methodical, and that painting was the frenzied expression of some primal
urge.

Both of these images are wrong. Hacking and painting have a lot in common. In
fact, of all the different types of people I’ve known, hackers and painters are
among the most alike.

What hackers and painters have in common is that they’re both makers. Along
with composers, architects, and writers, what hackers and painters are trying
to do is make good things. They’re not doing research per se, though if in the
course of trying to make good things they discover some new technique, so much
the better.

I’ve never liked the term “computer science.” The main reason I don’t like it
is that there’s no such thing. Computer science is a grab bag of tenuously
related areas thrown together by an accident of history, like Yugoslavia. At
one end you have people who are really mathematicians, but call what they’re
doing computer science so they can get DARPA grants. In the middle you have
people working on something like the natural history of computers– studying
the behavior of algorithms for routing data through networks, for example. And
then at the other extreme you have the hackers, who are trying to write
interesting software, and for whom computers are just a medium of expression,
as concrete is for architects or paint for painters. It’s as if mathematicians,
physicists, and architects all had to be in the same department.

Sometimes what the hackers do is called “software engineering,” but this term
is just as misleading. Good software designers are no more engineers than
architects are. The border between architecture and engineering is not sharply
defined, but it’s there. It falls between what and how: architects decide what
to do, and engineers figure out how to do it.

What and how should not be kept too separate. You’re asking for trouble if you
try to decide what to do without understanding how to do it. But hacking can
certainly be more than just deciding how to implement some spec. At its best,
it’s creating the spec– though it turns out the best way to do that is to
implement it.

Perhaps one day “computer science” will, like Yugoslavia, get broken up into
its component parts. That might be a good thing. Especially if it meant
independence for my native land, hacking.

Bundling all these different types of work together in one department may be
convenient administratively, but it’s confusing intellectually. That’s the
other reason I don’t like the name “computer science.” Arguably the people in
the middle are doing something like an experimental science. But the people at
either end, the hackers and the mathematicians, are not actually doing science.

The mathematicians don’t seem bothered by this. They happily set to work
proving theorems like the other mathematicians over in the math department, and
probably soon stop noticing that the building they work in says “computer
science” on the outside. But for the hackers this label is a problem. If what
they’re doing is called science, it makes them feel they ought to be acting
scientific. So instead of doing what they really want to do, which is to design
beautiful software, hackers in universities and research labs feel they ought
to be writing research papers.

In the best case, the papers are just a formality. Hackers write cool software,
and then write a paper about it, and the paper becomes a proxy for the
achievement represented by the software. But often this mismatch causes
problems. It’s easy to drift away from building beautiful things toward
building ugly things that make more suitable subjects for research papers.

Unfortunately, beautiful things don’t always make the best subjects for papers.
Number one, research must be original– and as anyone who has written a PhD
dissertation knows, the way to be sure that you’re exploring virgin territory
is to to stake out a piece of ground that no one wants. Number two, research
must be substantial– and awkward systems yield meatier papers, because you can
write about the obstacles you have to overcome in order to get things done.
Nothing yields meaty problems like starting with the wrong assumptions. Most of
AI is an example of this rule; if you assume that knowledge can be represented
as a list of predicate logic expressions whose arguments represent abstract
concepts, you’ll have a lot of papers to write about how to make this work. As
Ricky Ricardo used to say, “Lucy, you got a lot of explaining to do.”

The way to create something beautiful is often to make subtle tweaks to
something that already exists, or to combine existing ideas in a slightly new
way. This kind of work is hard to convey in a research paper.

So why do universities and research labs continue to judge hackers by
publications? For the same reason that “scholastic aptitude” gets measured by
simple-minded standardized tests, or the productivity of programmers gets
measured in lines of code. These tests are easy to apply, and there is nothing
so tempting as an easy test that kind of works.

Measuring what hackers are actually trying to do, designing beautiful software,
would be much more difficult. You need a good sense of design to judge good
design. And there is no correlation, except possibly a negative one, between
people’s ability to recognize good design and their confidence that they can.

The only external test is time. Over time, beautiful things tend to thrive, and
ugly things tend to get discarded. Unfortunately, the amounts of time involved
can be longer than human lifetimes. Samuel Johnson said it took a hundred years
for a writer’s reputation to converge. You have to wait for the writer’s
influential friends to die, and then for all their followers to die.

I think hackers just have to resign themselves to having a large random
component in their reputations. In this they are no different from other
makers. In fact, they’re lucky by comparison. The influence of fashion is not
nearly so great in hacking as it is in painting.

There are worse things than having people misunderstand your work. A worse
danger is that you will yourself misunderstand your work. Related fields are
where you go looking for ideas. If you find yourself in the computer science
department, there is a natural temptation to believe, for example, that hacking
is the applied version of what theoretical computer science is the theory of.
All the time I was in graduate school I had an uncomfortable feeling in the
back of my mind that I ought to know more theory, and that it was very remiss
of me to have forgotten all that stuff within three weeks of the final exam.

Now I realize I was mistaken. Hackers need to understand the theory of
computation about as much as painters need to understand paint chemistry. You
need to know how to calculate time and space complexity and about Turing
completeness. You might also want to remember at least the concept of a state
machine, in case you have to write a parser or a regular expression library.
Painters in fact have to remember a good deal more about paint chemistry than
that.

I’ve found that the best sources of ideas are not the other fields that have
the word “computer” in their names, but the other fields inhabited by makers.
Painting has been a much richer source of ideas than the theory of computation.

For example, I was taught in college that one ought to figure out a program
completely on paper before even going near a computer. I found that I did not
program this way. I found that I liked to program sitting in front of a
computer, not a piece of paper. Worse still, instead of patiently writing out a
complete program and assuring myself it was correct, I tended to just spew out
code that was hopelessly broken, and gradually beat it into shape. Debugging, I
was taught, was a kind of final pass where you caught typos and oversights. The
way I worked, it seemed like programming consisted of debugging.

For a long time I felt bad about this, just as I once felt bad that I didn’t
hold my pencil the way they taught me to in elementary school. If I had only
looked over at the other makers, the painters or the architects, I would have
realized that there was a name for what I was doing: sketching. As far as I can
tell, the way they taught me to program in college was all wrong. You should
figure out programs as you’re writing them, just as writers and painters and
architects do.

Realizing this has real implications for software design. It means that a
programming language should, above all, be malleable. A programming language is
for thinking of programs, not for expressing programs you’ve already thought
of. It should be a pencil, not a pen. Static typing would be a fine idea if
people actually did write programs the way they taught me to in college. But
that’s not how any of the hackers I know write programs. We need a language
that lets us scribble and smudge and smear, not a language where you have to
sit with a teacup of types balanced on your knee and make polite conversation
with a strict old aunt of a compiler.

While we’re on the subject of static typing, identifying with the makers will
save us from another problem that afflicts the sciences: math envy. Everyone in
the sciences secretly believes that mathematicians are smarter than they are. I
think mathematicians also believe this. At any rate, the result is that
scientists tend to make their work look as mathematical as possible. In a field
like physics this probably doesn’t do much harm, but the further you get from
the natural sciences, the more of a problem it becomes.

A page of formulas just looks so impressive. (Tip: for extra impressiveness,
use Greek variables.) And so there is a great temptation to work on problems
you can treat formally, rather than problems that are, say, important.

If hackers identified with other makers, like writers and painters, they
wouldn’t feel tempted to do this. Writers and painters don’t suffer from math
envy. They feel as if they’re doing something completely unrelated. So are
hackers, I think.

If universities and research labs keep hackers from doing the kind of work they
want to do, perhaps the place for them is in companies. Unfortunately, most
companies won’t let hackers do what they want either. Universities and research
labs force hackers to be scientists, and companies force them to be engineers.

I only discovered this myself quite recently. When Yahoo bought Viaweb, they
asked me what I wanted to do. I had never liked the business side very much,
and said that I just wanted to hack. When I got to Yahoo, I found that what
hacking meant to them was implementing software, not designing it. Programmers
were seen as technicians who translated the visions (if that is the word) of
product managers into code.

This seems to be the default plan in big companies. They do it because it
decreases the standard deviation of the outcome. Only a small percentage of
hackers can actually design software, and it’s hard for the people running a
company to pick these out. So instead of entrusting the future of the software
to one brilliant hacker, most companies set things up so that it is designed by
committee, and the hackers merely implement the design.

If you want to make money at some point, remember this, because this is one of
the reasons startups win. Big companies want to decrease the standard deviation
of design outcomes because they want to avoid disasters. But when you damp
oscillations, you lose the high points as well as the low. This is not a
problem for big companies, because they don’t win by making great products. Big
companies win by sucking less than other big companies.

So if you can figure out a way to get in a design war with a company big enough
that its software is designed by product managers, they’ll never be able to
keep up with you. These opportunities are not easy to find, though. It’s hard
to engage a big company in a design war, just as it’s hard to engage an
opponent inside a castle in hand to hand combat. It would be pretty easy to
write a better word processor than Microsoft Word, for example, but Microsoft,
within the castle of their operating system monopoly, probably wouldn’t even
notice if you did.

The place to fight design wars is in new markets, where no one has yet managed
to establish any fortifications. That’s where you can win big by taking the
bold approach to design, and having the same people both design and implement
the product. Microsoft themselves did this at the start. So did Apple. And
Hewlett-Packard. I suspect almost every successful startup has.

So one way to build great software is to start your own startup. There are two
problems with this, though. One is that in a startup you have to do so much
besides write software. At Viaweb I considered myself lucky if I got to hack a
quarter of the time. And the things I had to do the other three quarters of the
time ranged from tedious to terrifying. I have a benchmark for this, because I
once had to leave a board meeting to have some cavities filled. I remember
sitting back in the dentist’s chair, waiting for the drill, and feeling like I
was on vacation.

The other problem with startups is that there is not much overlap between the
kind of software that makes money and the kind that’s interesting to write.
Programming languages are interesting to write, and Microsoft’s first product
was one, in fact, but no one will pay for programming languages now. If you
want to make money, you tend to be forced to work on problems that are too
nasty for anyone to solve for free.

All makers face this problem. Prices are determined by supply and demand, and
there is just not as much demand for things that are fun to work on as there is
for things that solve the mundane problems of individual customers. Acting in
off-Broadway plays just doesn’t pay as well as wearing a gorilla suit in
someone’s booth at a trade show. Writing novels doesn’t pay as well as writing
ad copy for garbage disposals. And hacking programming languages doesn’t pay as
well as figuring out how to connect some company’s legacy database to their Web
server.

I think the answer to this problem, in the case of software, is a concept known
to nearly all makers: the day job. This phrase began with musicians, who
perform at night. More generally, it means that you have one kind of work you
do for money, and another for love.

Nearly all makers have day jobs early in their careers. Painters and writers
notoriously do. If you’re lucky you can get a day job that’s closely related to
your real work. Musicians often seem to work in record stores. A hacker working
on some programming language or operating system might likewise be able to get
a day job using it. [1]

When I say that the answer is for hackers to have day jobs, and work on
beautiful software on the side, I’m not proposing this as a new idea. This is
what open-source hacking is all about. What I’m saying is that open-source is
probably the right model, because it has been independently confirmed by all
the other makers.

It seems surprising to me that any employer would be reluctant to let hackers
work on open-source projects. At Viaweb, we would have been reluctant to hire
anyone who didn’t. When we interviewed programmers, the main thing we cared
about was what kind of software they wrote in their spare time. You can’t do
anything really well unless you love it, and if you love to hack you’ll
inevitably be working on projects of your own. [2]

Because hackers are makers rather than scientists, the right place to look for
metaphors is not in the sciences, but among other kinds of makers. What else
can painting teach us about hacking?

One thing we can learn, or at least confirm, from the example of painting is
how to learn to hack. You learn to paint mostly by doing it. Ditto for hacking.
Most hackers don’t learn to hack by taking college courses in programming. They
learn to hack by writing programs of their own at age thirteen. Even in college
classes, you learn to hack mostly by hacking. [3]

Because painters leave a trail of work behind them, you can watch them learn by
doing. If you look at the work of a painter in chronological order, you’ll find
that each painting builds on things that have been learned in previous ones.
When there’s something in a painting that works very well, you can usually find
version 1 of it in a smaller form in some earlier painting.

I think most makers work this way. Writers and architects seem to as well.
Maybe it would be good for hackers to act more like painters, and regularly
start over from scratch, instead of continuing to work for years on one
project, and trying to incorporate all their later ideas as revisions.

The fact that hackers learn to hack by doing it is another sign of how
different hacking is from the sciences. Scientists don’t learn science by doing
it, but by doing labs and problem sets. Scientists start out doing work that’s
perfect, in the sense that they’re just trying to reproduce work someone else
has already done for them. Eventually, they get to the point where they can do
original work. Whereas hackers, from the start, are doing original work; it’s
just very bad. So hackers start original, and get good, and scientists start
good, and get original.

The other way makers learn is from examples. For a painter, a museum is a
reference library of techniques. For hundreds of years it has been part of the
traditional education of painters to copy the works of the great masters,
because copying forces you to look closely at the way a painting is made.

Writers do this too. Benjamin Franklin learned to write by summarizing the
points in the essays of Addison and Steele and then trying to reproduce them.
Raymond Chandler did the same thing with detective stories.

Hackers, likewise, can learn to program by looking at good programs– not just
at what they do, but the source code too. One of the less publicized benefits
of the open-source movement is that it has made it easier to learn to program.
When I learned to program, we had to rely mostly on examples in books. The one
big chunk of code available then was Unix, but even this was not open source.
Most of the people who read the source read it in illicit photocopies of John
Lions’ book, which though written in 1977 was not allowed to be published until
1996.

Another example we can take from painting is the way that paintings are created
by gradual refinement. Paintings usually begin with a sketch. Gradually the
details get filled in. But it is not merely a process of filling in. Sometimes
the original plans turn out to be mistaken. Countless paintings, when you look
at them in xrays, turn out to have limbs that have been moved or facial
features that have been readjusted.

Here’s a case where we can learn from painting. I think hacking should work
this way too. It’s unrealistic to expect that the specifications for a program
will be perfect. You’re better off if you admit this up front, and write
programs in a way that allows specifications to change on the fly.

(The structure of large companies makes this hard for them to do, so here is
another place where startups have an advantage.)

Everyone by now presumably knows about the danger of premature optimization. I
think we should be just as worried about premature design– deciding too early
what a program should do.

The right tools can help us avoid this danger. A good programming language
should, like oil paint, make it easy to change your mind. Dynamic typing is a
win here because you don’t have to commit to specific data representations up
front. But the key to flexibility, I think, is to make the language very
abstract. The easiest program to change is one that’s very short.

This sounds like a paradox, but a great painting has to be better than it has
to be. For example, when Leonardo painted the portrait of Ginevra de Benci in
the National Gallery, he put a juniper bush behind her head. In it he carefully
painted each individual leaf. Many painters might have thought, this is just
something to put in the background to frame her head. No one will look that
closely at it.

Not Leonardo. How hard he worked on part of a painting didn’t depend at all on
how closely he expected anyone to look at it. He was like Michael Jordan.
Relentless.

Relentlessness wins because, in the aggregate, unseen details become visible.
When people walk by the portrait of Ginevra de Benci, their attention is often
immediately arrested by it, even before they look at the label and notice that
it says Leonardo da Vinci. All those unseen details combine to produce
something that’s just stunning, like a thousand barely audible voices all
singing in tune.

Great software, likewise, requires a fanatical devotion to beauty. If you look
inside good software, you find that parts no one is ever supposed to see are
beautiful too. I’m not claiming I write great software, but I know that when it
comes to code I behave in a way that would make me eligible for prescription
drugs if I approached everyday life the same way. It drives me crazy to see
code that’s badly indented, or that uses ugly variable names.

If a hacker were a mere implementor, turning a spec into code, then he could
just work his way through it from one end to the other like someone digging a
ditch. But if the hacker is a creator, we have to take inspiration into account.

In hacking, like painting, work comes in cycles. Sometimes you get excited
about some new project and you want to work sixteen hours a day on it. Other
times nothing seems interesting.

To do good work you have to take these cycles into account, because they’re
affected by how you react to them. When you’re driving a car with a manual
transmission on a hill, you have to back off the clutch sometimes to avoid
stalling. Backing off can likewise prevent ambition from stalling. In both
painting and hacking there are some tasks that are terrifyingly ambitious, and
others that are comfortingly routine. It’s a good idea to save some easy tasks
for moments when you would otherwise stall.

In hacking, this can literally mean saving up bugs. I like debugging: it’s the
one time that hacking is as straightforward as people think it is. You have a
totally constrained problem, and all you have to do is solve it. Your program
is supposed to do x. Instead it does y. Where does it go wrong? You know you’re
going to win in the end. It’s as relaxing as painting a wall.

The example of painting can teach us not only how to manage our own work, but
how to work together. A lot of the great art of the past is the work of
multiple hands, though there may only be one name on the wall next to it in the
museum. Leonardo was an apprentice in the workshop of Verrocchio and painted
one of the angels in his Baptism of Christ. This sort of thing was the rule,
not the exception. Michelangelo was considered especially dedicated for
insisting on painting all the figures on the ceiling of the Sistine Chapel
himself.

As far as I know, when painters worked together on a painting, they never
worked on the same parts. It was common for the master to paint the principal
figures and for assistants to paint the others and the background. But you
never had one guy painting over the work of another.

I think this is the right model for collaboration in software too. Don’t push
it too far. When a piece of code is being hacked by three or four different
people, no one of whom really owns it, it will end up being like a common-room.
It will tend to feel bleak and abandoned, and accumulate cruft. The right way
to collaborate, I think, is to divide projects into sharply defined modules,
each with a definite owner, and with interfaces between them that are as
carefully designed and, if possible, as articulated as programming languages.

Like painting, most software is intended for a human audience. And so hackers,
like painters, must have empathy to do really great work. You have to be able
to see things from the user’s point of view.

When I was a kid I was always being told to look at things from someone else’s
point of view. What this always meant in practice was to do what someone else
wanted, instead of what I wanted. This of course gave empathy a bad name, and I
made a point of not cultivating it.

Boy, was I wrong. It turns out that looking at things from other people’s point
of view is practically the secret of success. It doesn’t necessarily mean being
self-sacrificing. Far from it. Understanding how someone else sees things
doesn’t imply that you’ll act in his interest; in some situations– in war, for
example– you want to do exactly the opposite. [4]

Most makers make things for a human audience. And to engage an audience you
have to understand what they need. Nearly all the greatest paintings are
paintings of people, for example, because people are what people are interested
in.

Empathy is probably the single most important difference between a good hacker
and a great one. Some hackers are quite smart, but when it comes to empathy are
practically solipsists. It’s hard for such people to design great software [5],
because they can’t see things from the user’s point of view.

One way to tell how good people are at empathy is to watch them explain a
technical question to someone without a technical background. We probably all
know people who, though otherwise smart, are just comically bad at this. If
someone asks them at a dinner party what a programming language is, they’ll say
something like “Oh, a high-level language is what the compiler uses as input
to generate object code.” High-level language? Compiler? Object code? Someone
who doesn’t know what a programming language is obviously doesn’t know what
these things are, either.

Part of what software has to do is explain itself. So to write good software
you have to understand how little users understand. They’re going to walk up to
the software with no preparation, and it had better do what they guess it will,
because they’re not going to read the manual. The best system I’ve ever seen in
this respect was the original Macintosh, in 1985. It did what software almost
never does: it just worked. [6]

Source code, too, should explain itself. If I could get people to remember just
one quote about programming, it would be the one at the beginning of Structure
and Interpretation of Computer Programs.

Programs should be written for people to read, and only incidentally for
machines to execute.

You need to have empathy not just for your users, but for your readers. It’s in
your interest, because you’ll be one of them. Many a hacker has written a
program only to find on returning to it six months later that he has no idea
how it works. I know several people who’ve sworn off Perl after such
experiences. [7]

Lack of empathy is associated with intelligence, to the point that there is
even something of a fashion for it in some places. But I don’t think there’s
any correlation. You can do well in math and the natural sciences without
having to learn empathy, and people in these fields tend to be smart, so the
two qualities have come to be associated. But there are plenty of dumb people
who are bad at empathy too. Just listen to the people who call in with
questions on talk shows. They ask whatever it is they’re asking in such a
roundabout way that the hosts often have to rephrase the question for them.

So, if hacking works like painting and writing, is it as cool? After all, you
only get one life. You might as well spend it working on something great.

Unfortunately, the question is hard to answer. There is always a big time lag
in prestige. It’s like light from a distant star. Painting has prestige now
because of great work people did five hundred years ago. At the time, no one
thought these paintings were as important as we do today. It would have seemed
very odd to people at the time that Federico da Montefeltro, the Duke of
Urbino, would one day be known mostly as the guy with the strange nose in a
painting by Piero della Francesca.

So while I admit that hacking doesn’t seem as cool as painting now, we should
remember that painting itself didn’t seem as cool in its glory days as it does
now.

What we can say with some confidence is that these are the glory days of
hacking. In most fields the great work is done early on. The paintings made
between 1430 and 1500 are still unsurpassed. Shakespeare appeared just as
professional theater was being born, and pushed the medium so far that every
playwright since has had to live in his shadow. Albrecht Durer did the same
thing with engraving, and Jane Austen with the novel.

Over and over we see the same pattern. A new medium appears, and people are so
excited about it that they explore most of its possibilities in the first
couple generations. Hacking seems to be in this phase now.

Painting was not, in Leonardo’s time, as cool as his work helped make it. How
cool hacking turns out to be will depend on what we can do with this new medium.

Notes

[1] The greatest damage that photography has done to painting may be the fact
that it killed the best day job. Most of the great painters in history
supported themselves by painting portraits.

[2] I’ve been told that Microsoft discourages employees from contributing to
open-source projects, even in their spare time. But so many of the best hackers
work on open-source projects now that the main effect of this policy may be to
ensure that they won’t be able to hire any first-rate programmers.

[3] What you learn about programming in college is much like what you learn
about books or clothes or dating: what bad taste you had in high school.

[4] Here’s an example of applied empathy. At Viaweb, if we couldn’t decide
between two alternatives, we’d ask, what would our competitors hate most? At
one point a competitor added a feature to their software that was basically
useless, but since it was one of few they had that we didn’t, they made much of
it in the trade press. We could have tried to explain that the feature was
useless, but we decided it would annoy our competitor more if we just
implemented it ourselves, so we hacked together our own version that afternoon.

[5] Except text editors and compilers. Hackers don’t need empathy to design
these, because they are themselves typical users.

[6] Well, almost. They overshot the available RAM somewhat, causing much
inconvenient disk swapping, but this could be fixed within a few months by
buying an additional disk drive.

[7] The way to make programs easy to read is not to stuff them with comments. I
would take Abelson and Sussman’s quote a step further. Programming languages
should be designed to express algorithms, and only incidentally to tell
computers how to execute them. A good programming language ought to be better
for explaining software than English. You should only need comments when there
is some kind of kludge you need to warn readers about, just as on a road there
are only arrows on parts with unexpectedly sharp curves.

Read More

Show More

Related Articles

Check Also
Close
Back to top button
Close
Close