I just recently finished reading a book, no, rather, a comic, by Scott McCloud, called Understanding Comics: The Invisible Art. It was a fantastic read, very illuminating in many aspects. A lot of things I didn’t understand about art in general are finally beginning to be less murky.

One of the most striking concepts was the six stages of art. And it resonated. I realized that those six stages correspond as well to the development and progress of a programmer.

In his comic, the six stages are as follows: Idea/Purpose, Form, Idiom, Structure, Craft, Surface.

The Six Steps of Art

Let’s start at the top: Surface.

Surface applies to the most basic of programming. Simple statements, flow control, basic functions. Every programmer begins their growth at this stage. They learn how basic math works, properties, etc. They learn how to order statements, if conditionals, and loop guards.

Next is Craft.

Craft is where they take the basics, the Surface, and begin constructing bigger things. Functions, classes, basic data structures. Here, they learn how to construct a complex piece of smaller simpler pieces, but they are only just barely beginning to learn why and how it works that way. Any data structures they build are based off of basic data structures, and they don’t yet understand deeply the fundamental ideas of efficiency, logic, and computability. The good programmer also begins to learn about source control, using their tools, building, and deploying their work.

Structure is next in their growth path.

Here, they begin to learn how to organize their complex pieces into a working piece. How to access and parse text files, basic database interactions. They’ll do things the slow, wrong way, but its necessary to reach the stage of development. Here, they begin to worry about efficiency, the right way of doing things. Object Oriented programming starts to make sense as a conceptual model(which is all it really is). The now competent programmer is capable of learning other basic conceptual models.

Idiom is the biggest part of the growth.

Here, they begin to learn about Design Patterns, and begin to apply them. They apply them at every stage, from the basic Surface, to the overall Structure of the program. They also begin to learn about the ideas and concepts behind data protocols, to use them where needed. This is when programmers become cynical about Silver Bullets. This is when they finally realize there is no substitute for skill, experience, and effort. Many programmers do extremely well staying at this stage. However, to become a top-notch programmer, one needs to see and explore the limits of programming.

That is where Form comes in. Programmers at this stage explore the efficacy of Design Patterns, breaking them where and when needed, even developing their own versions. They begin to develop solid ideas about how API’s should work, about what is computable, about efficiency, and about Logic. These are the programmers that take already established tools and concepts, and bring us exciting new ones. Notable examples are: Bittorrent, Linux, and the Mouse. They looked at what was being done, and brought something new, and world-changing to the world of computing.

The final, highest level of development is Ideas/Purpose. This is where programming Gods like Knuth and Stallman rest. This is where Computer Scientists do their heady, ivory tower research that is oh-so-important to the world at large. This is where concepts of algorithms, data structures, efficiency, and logic are explored in their abstract. This is where Computer Science is rooted, and where the birth of the computer industry started. This is where Boole developed his ideas of Boolean logic, where Cray designed his supercomputers, and the very basis of all digital electronics.

Now, this is the normal path most programmers take in their growth. Even though, in College and University, we’re taught the ideas and purpose of programming from almost the start… when we don’t have the basic building blocks of understanding of logic and program construction.

As my friend Khumba just said: “You can’t play Protoss until you’ve played a few games with Terran.” You need to understand the basic building blocks of logic, statements and flow-control and the experience with those to understand the higher level concepts of functions, routines, threads, classes, modules, executables, algorithms.

The thing is though, we can start at any stage we like, from Ideas/Purpose, to the Idioms of programming. But the greatest, best programmers amongst us will explore all of these stages in depth, learning what they need from it to improve themselves and their art. Because thats what Programming is. It is an art, and we are the artists, working with the pure stuffs of thought. We translate the abstract to binary logic, to 1′s and 0′s, and for that… we are artists.

If you disagree, or agree, let me know. Theres a handy comment box right down there.

No, seriously, it really is. It doesn’t matter how cool the subsystem is, how efficiently you can pack megaquads of data into a byte. What really does matter is how a user interacts with a system. I have a lesson here, to be shared, and I don’t really care if anyone listens.

A bit of backstory: I just recently bought one of these, the Neo Freerunner. I was intrigued by the hardware capabilities, as well as the absurdly low price for a phone with that much hardware.

However, I’ve since discovered that in many aspects the phone is useless. I can understand that its only for expert users, so consider this a bug report, of problems I’ve encountered no matter how intelligent or “expert” I am.

The phone itself is 100% unmodified hardware, and shipped with the default image, Openmoko 2007.2.

That software, though kludgy, slow, and occasionally frustrating, actually did what it was supposed to. I could type anything I wanted, with effort, use a backspace key, dial phone numbers.

There were some very annoying bugs, for example, the point where I was trying to add an email to a friend’s contact list. Normally, in any other kind of display, the area I’m typing into would scroll up, so I could see what I was typing. Not so here. I had to contend with guessing if I had hit the right buttons. Very annoying, and I quickly gave up.

Another bug, and I do consider these bugs, is the wifi and gps lacked a decent gui to configure them out of the box. The phone comes with specific hardware, it should have applications that can use them. Again, I know, for expert users, but I wanted a phone with wifi so I could browse the internet. If I need to log in with my laptop via ssh to the phone, then why not just surf with my laptop!

Now, I’ve been playing with the Om2008.8 release, came out today. Of course, I expected an official release to, you know, have basic issues solved. As of the time that I write this, there are 13 issues. THIRTEEN! Just hours after the “official” release. I could understand if these were rare or serious bugs, with video or weird interactions. But no, what we see are basic issues with the GSM portion! There are severe bugs in the phone part of the software… yes, this phone really is for expert users.

Usability is incredibly, incredibly important. I can’t stress this enough. The user does not know, nor care, about how free the code is, nor how awesome it is. They care that it works, and does really cool things. Basic usability needs to be addressed, if not by FIC, then by the community. They have steps. Some. Use cases to be filled out.

I’ve been following the Linux Haters blog, and its a lot of fun. I don’t have pretensions that one is better, or more superior. Its a great window into usability, and community matters, something everyone involved in OSS should be working on. Here’s a few guidelines if you’re going to make a touchscreen app:

• If what you want to show is bigger than the screen, then by all means, scroll. BUT. And this is a major but. You need to somehow inform the user there is more content. Here’s a sample: the default keyboard app in the latest upgrade. It shows just the lower case letters, nothing else. If I had not read online, I would not have known you could “flick” the keyboard to get at more keys.
• Be tolerant of mistakes. Always, Always, ALWAYS have an available backspace key for keyboards. This is more for the OS makers than the app, but it is still critically important.
• If you invent a brand-new interface or way of interaction, you should offer tutorials. Sony Ericsson phones when they are first turned on, offer new users a complete tour of how to use each app, and button. This is phenomenal in terms of making people happy. It shows you cared to make something like that, and that you want them to be able to use the app.
• Responsiveness. I can’t stress this enough. When your app is doing something, there should be some thing showing that its working away. Especially when booting up. My phone’s screen turns off at arbitrary times during bootup. This is unacceptable, as then it looks like it didn’t boot up.
• This is directed at the Openmoko devs themselves: a Notes application, please! You’ve given us a smartphone, with wifi, gps, an sd card… let us make notes. Arbitrary notes. About anything. Free of context, or context can be added. I don’t care if I can install one, one should be included by default.

Now, another consideration with something like this, to help usability, is at the very least, write documentation. Do you have any idea how awesome I would think this phone was, if I had a manual on how to fucking use the applications that came with this phone? Of course, the fact I need a manual on these apps just demonstrates how utterly unintuitive they are. Even if its assumed people can figure it out, we have an amazing resource to do this, thousands of interested individuals, with time, and the phone, able to modify a wiki. Provide information on how to USE these applications. Full, and in-depth, showing that if you flick the keyboard… you get other keys! OMG! Amazing… how am I supposed to know that, without delving into a mailing list?

Now, while I do really like the hardware, and I’m all for defending rights and freedoms, I also want usable programs. 90% of the time, they are completely usable, and excellent, and the only fault is me. Apache for one is humonguosly documented, and all faults for screwing things up lays totally with me. Abiword is another excellent program. I just use it for typing. I don’t need typography, or colors, or any shit like that. Open Office is simply too bulky for my simple needs, and doesn’t read Excel files properly.

Now, just a small warning, anyone that posts saying, “It works for me.” their comment is deleted immediately, no other reason needed. Any freetards yelling at me about criticizing their beliefs or whatever, deleted as well. If you have something to say, be calm and reasonable about it. Provide evidence, technical, screenshots, advice, whatever. I want to hear debate and discussion, but there can be no discussing with a Freetard. I use what works best for me, and for your information, I am currently using Ubuntu 8.04, on both my desktop and my laptop, with absolutely no hardware issues.

Thats right, the dreaded two m’s. If you use C or C++, its quite dreaded. However, for about the first time ever, I ran into memory management issues in Python! Well, partially due to my own mistakes, and partially due to the way Python handles memory.

Memory handling schemes go awry

Well, not quite. More like, a leaky abstraction, compounded by my own idiot mistakes in the code. When memory is allocated in a loop, it is normally a smart optimization to hold onto that memory. What this means is that there are some cool things that you can do, like, refer to variables that were allocated inside a loop, and they will be in the local scope. They will just have the latest value thats been assigned to that variable.

However, in my case it backfired. The memory was held onto… and never reused. Essentially, I had a memory leak.

My own stupid mistakes

What happened, is that I was loading in a variable from a pickled file. However, what I had done was:

elements = pickle.load(open('file', 'rb'))

Normally, this would be okay, however, due to the size of the files, and the fact I was doing this in a loop, over different files, is that there was a now opened multi-megabyte file floating around, that has not been closed or reallocated. Actually, several. I came home after two hours to find that almost all of my RAM and 70% of my swap space had been used. Thats 70% of 3.8 GB. Yeah, great job Zeroth.

The lesson here is, that what works well for a small loop, or small files, does not neccesarilly work well for large files or large loops. Always make sure you close every file you open, and if worst comes to worst, allocate memory by using the del keyword in Python. I’m using this, and there is very moderate memory growth, as compared to the quite massive almost 6 GB of memory the code was taking up before.

So the proper way to do this is:

f=open('file', 'rb')
elements = pickle.load(f)
f.close()
del f

I do need to make this clear. This fault was not due to Python, or any problems or bugs in it, but rather with my own sloppy code. Even the best of us make mistakes, and I’m definitely not the best of us.

I hate SanDisk Cruzer Thumb drives. Right now, I am running a temporary internet cafe for a conference of oceanographers and meteorologists, and so they need to upload their presentations. Therefore, they plug in their thumb drives, and I’ve seen the performance, reliability, and design of different models and brands. After plugging in literally 200 different thumb drives, I really really hate the SanDisk Cruzer drives.

The software is slow and buggy. When you plugin a SanDisk Cruzer drive, it loads in software it calls U3, to enable connection with the thumb drive. This software is slow, full of eye-candy, and offers nothing new. And the most grievous point: Why must a thumb drive require custom software? I can understand drivers, but custom software? Please. No one cares, and no one wants the extra software. I cannot imagine a single extra use that the software could provide, beyond encryption or security. Even that is better and faster done by external products, like TrueCrypt, or GPG.

When I say slow, I mean slow. Every other brand, even old old usb drives are done loading in Windows XP in less than 30 seconds on four year old machines. The U3 software means I have to wait well over two minutes, with the drive appearing and disappearing not once, not twice, but thrice. When cheap no-name brands of thumb drives can be done on an old machine in less than 30 seconds, your software is slow.

Just don’t buy SanDisk thumb drives. Maybe they might get the hint, and stop forcing the abysmal U3 software on their consumers.

Design is a very big thing, for a lot of people. But I find its often misunderstood, or incorrectly applied. To start with, design does not mean the same thing to Interior decorators as it does to architects. But most people see design as more of the interior decorator kind, than the architect kind.

Interior decorators do what I call appearance level design. And the benefits of appearance level design are not all that important. They make people feel better, more at home. Its a more impressive room/house/office.

That is not real design, in my view. Real design is figuring the best placement of bathrooms and appliances in the kitchen for maximum comfort and productivity. Real design is invisible, unseen, uncredited.

Here’s a little task for you, to see what design really is about. Walk into any room of your house/apartment/mud hut/palace, and choose a common household item. I’ll take a knife block as an example. Now, spend five or ten minutes thinking about how you could improve a knife block, to A) make it easier to use or B) add new functionality that is not just combining two opposite functions (like a clock on a VHF antenna).

If you can come up with an improvement that is not just glitz, that actually makes the item better than it was before, then the item does not have good design. If however, you can come up with nothing, and you realize changing anything would be for the worst, the item has good design.

My example the knife block, works really well for its purpose: hold knives, keep them sharp, and keep them organized. Can there be an improvement, that would make using it easier, faster, more efficient at its purpose? After ten minutes, all I could come up with was improving the knives themselves. The actual knife block works great. Its easy to pull out a knife, you know by the size of the hole how big the knife is, and its easy to put back in. Some knife blocks have built-in sharpeners, and thats about the only improvement that could be made.

The fact is, it takes experience and introspection to see design around us all the time. In our games, in TV shows, in our world all around us.

Another excellent example is in Final Fantasy III for the DS. The actual interface is a hodge-podge of button-clicking and touch-screen use, not very good. However, in the item list, for each category of items, there is an 8×8 icon representing it. This sits beside each item in your inventory, allowing you at a glance to see what it is. Axes for viking weapons, a glove icon for accessories, a spear icon for dragoons. Its simple, effective, nearly invisible. I caught myself using the icons, and thats when I actually noticed them. Someone consciously decided to make icons for each kind of item, and displayed them. Why? To make it easier to deal with a large inventory!

Good design is all around us. And good design takes conscious effort. Any time you find yourself making a snap decision on the placement of an item, or icon, or graphic interface, ask yourself why it goes there, and is there any way I can improve this?

As a personal example of good design, I was tasked to make it so that users of an application would know that the file has not been saved yet. My first response was a dialog on exit(or any other context change, more on that in future posts) if the file was not saved. Then I thought, okay, the user, she’s likely not going to want to have to click through a dialog all the time, so how else can I do this? I added an icon to the upper bar, that appears when the file is modified. Its red, shaped like a stop sign, and has the word stop on it. Its placed right by the exit button.

Nowhere else in the application is red used, so when you go to load a new file, exit, you see the icon if you did not already. It disappears when you save, and it appears when you modify a file. No dialogs, no clicks, simple information conveyed quickly and easily. That is good design. It makes the workflow faster, the user quickly and easily learns what the icon means, and it helps them!

Next time, ask yourself why you are making the decision this way. If you can improve it, do so!