The essence of computer science

The other day, I was watching this lecture series from Harold Abelson of MIT, who explains better than I ever could the difference between computer science and other engineering disciplines.

Computer science deals with idealized components. We know as much as we want about the [pieces] that we fit together. That means that in building a large program, there is not that much difference between what I can build and what I can imagine. So as opposed to other kinds of engineering, the constraints imposed on building large software systems are the limitations of our own minds. In that sense computer science is an abstract form of engineering, where you ignore the constraints imposed by the physical world.

Of course this guy is coming from the perspective of an academic setting, and there are a few caveats that will easily be noticed by those in the industry. The constraints imposed by the physical world are still relevant to some degree. For example, we have the constraints of the computer hardware that we're working on, including memory and processor usage. In the web programming domain, you are constrained by the bandwidth of the network, etc. And, of course, in the end, the system has to "do the job" / "satisfy the customer", which is another constraint from the real world.

That said, I think his statement is still for the most part true, since the physical constraints apply to the system as a whole, but not necessarily the parts. For example a civil engineer has physical constraint on each bolt & screw, an electrical engineer has a physical constraint on each wire, each capacitor, each power supply, etc., an architect / building engineer has constraints on each concrete block he puts into the building. So for engineers in other disciplines both the parts and the whole are constrained. The computer scientist is only constrained in the end, while the intermediate processes and components have a tremendous amount of flexibility.

light at the end of the tunnel

And then I started working in the game industry. Here I found 5 types of programmers.

• The bad programmers: This was the old breed, and slowly dying, that simply wrote shitty code, period. If you're lucky, it did JUST the thing that it was intended to do, for 80% of the cases. If you needed to have the other 20% working, or if you wanted to add a new feature to the existing code base you couldn't. The whole thing needed to be rewritten from scratch.
• The super programmers: Those were the guys of the old-school philosophy, who wrote huge amounts of code, that nobody could touch or try to understand. The code was poorly written, with tons of messy dependencies, and yet it did a lot of useful things, and people were afraid to touch it, for fear of causing the collapse of the card house. In many ways, they were similar to the guys in the "bad programmers" category, except that they wrote a LOT of bad code. In fact they wrote so much of it, that it couldn't all be rewritten. Those giants were people the company couldn't live without. And yet when one of those guys left, somebody had to step in and do it right.
• The cleaners: This was another category of super-programmers who were good at cleaning up the mess created by other programmers, regardless of how poorly it was written. 
• The well-intentioned programmers: These guys seemed to care about software engineering principles, correct design, design patterns, refactoring, and other such concepts, upon which the academians looked down. Occasionally they would even get around to it. Unfortunately, in practice that idea slipped, since under the time pressure of deadlines people often reverted to their evil old ways and forgot about nice structure, readability, maintenance, and sharing. 
• The good programmers: finally there was the select few who could do the impossible: write a lot of code, write it well, so no cleanup was necessary. Anyone coming in after these guys would breathe a sigh of relief over how easy it was to understand their code and add features to it. Or maybe their successors wouldn't even appreciate this fact, nevertheless their job in fixing bugs or adding features would be much easier on this clean code base. 

Of course I wanted to be in the last category, and found myself more in the next-to-last. I was armed with knowledge of design patterns and other cool goodies. Unfortunately, many of the books written about software engineering and proper code design had one fatal flaw. They assumed a top-down approach where proper design preceded implementation, and the textbook-clean examples lead you by the hand towards the right solution. It was akin to doing exercises from a chess book in the flavor of "checkmate in 2, black starts". You know the solution exists, and you know it's a clean one, and you know it won't take too long to figure it out. 

In the real world, you have to play the real chess game, and work with real code, that wasn't necessarily written with you or your features in mind. In the real world, the design patterns aren't handed down to you from a textbook, and instead have to be arrived at from the bottom-up. You have to take code from an arbitrary state A to a final state Z, where

A = typically a mess of some sort, which does some of the things you want right, some of them wrong, and some of them not at all

Z = a functionining application with bugs from A fixed, some new desired features, and (as a side benefit) something which is also easy to maintain for the next guy

So let the journey begin. 

Signs of hope

Apparently, not all was lost. Slowly I started to see signs around me that I wasn't entirely alone. There were books written on design of software, the concept of design patterns was introduced and formalized, and my university even started teaching courses in it. There seemed tope that I was on the right track. Yet once again, as soon as I talked to people in the real world, they thought concepts of proper software design were cool in principle, but too much of a bother to use in practice. There were projects to finish, research results to get, and since noone in the outside world cared how your code looked, you could get away with writing it however the hell you please, which 99% of the time meant writing it badly. Interetingly enough people whom I talked to were aware of the fact that their code was written badly, respected me for trying to do a better job, and yet didn't really take the time to improve themselves in this regard.

One day I had an eye opening experience. I was trying to get into somebody else's research code in grad school, to figure out how it worked, so I could use it. Back and forth, back and forth, I went between the classes and the functions which all seemed to be in one file. It was too much to read through on screen, so I decided to send it to the printer. I let it go, and when I came to the printer it was out of paper, even though at least 20 pages were already on the table.WTF? I went back to see how many pages were actually sent to the printer, and to my utter astonishment I found out the file was 115 pages long! No wonder I couldn't make sense of it. I gave up on trying to reuse this guy's code, and vowed to myself to never write code like this.

I noticed also that when people looked at my code, they could fairly easily understand what it was doing, but when I looked at theirs, I would typically be lost. Maybe I was onto something. Maybe the "proper" way of writing would save me in the long run? So I pressed on, partly because I hoped that in the end it would pay off, and partly, as much as I hate to admit it, because I simply couldn't stop. Refactoring just felt so good.

Refactoraholics Anonymous

Hello, my name is Aleks and I'm a refactoraholic.

First time I noticed the interest in refactoring when I was around 18-19, just starting to write my first decent-sized pascal and C++ programs. First I wrote my programs the ugly way, and then I wanted to write them better. While this drive in and of itself was natural, I found myself more interested in making the code pretty than the end result of what it does. I noticed getting this addictive feeling when I refactoring, and ended up rewriting old projects until they were "perfect", even though there would be no functional benefit from it. Such purification activity on code seemed to have no purpose, as the code either:

a) already did what it was supposed to do or

b) was part of a curriculum course that I already finished, and thus working extra wouldn't improve my grade, or

c) it was part of a research project in grad school which needed more results, and instead of getting more results I found myself spending more time on refactoring.

It was very much a compulsion, whose power I didn't realize. Each time I did refactoring, it felt like it just HAD to be done, and each time it had very detremental effect on the project at hand. Naturally the existing functionality broke, my project's deadline slipped. I spent sleepless nights, and long weekends trying to get myself out of the mess. The code was prettier, more modular, more decoupled, more readable, but functionally it deteriorated, and it took me a while to get it to its original stability and use cases. Before I discovered source control, starting refactoring meant that I had to finish it, which cost me a lot of sweat and tears. Even when I did discover source control, I was very reluctant to back out the changes because I always gravitated towards a more modular version of the code, regardless of the cost.

 I cursed myself for starting the process, knowing full well I would have been better off not going there. But it was too late. I was in the middle of refactoring, everything was broken, I would be lucky if the code compiled let alone work as intended. Oh yeah, in the meantime I needed to get new functionality in, and by the time I cleaned up the mess, I would forget where I was and why I started the whole cleanup in the first place.

I considered going to a refactoring-shrink, who may have been able to advise me properly out of the situation, but who knows I could have gotten one of those old-school hacker dudes, who would have told me to be a man and just write the whole project in C already. I was too timid to come out of the refactoring closet, and too stubborn to stop, I was stuck in an evil loop. At the same time while I was aware of the problem, I was a little bit in denial, since refactoring felt so good. It was like a drug to me, I couldn't get enough. And despite all the pain it's put me thru, I kept at it. Each new project was refactored, again and again, and it's almost like I didn't learn from my mistakes.

I felt completely alone. My parents didn't know about my addiction, I didn't think they would understand. Most of my non-programming friends naturally couldn't relate at all. How do you explain refactoring of code to someone who doesn't even understand what code is all about?

But even most of my fellow programmers didn't share my passion, they thought I was a weirdo, spending endless hours on a useless activity.

I had a passion for writing code well, but it didn't seem to pay off. Was I doomed to an eternity of toil and frustration?... (to be continued)