Programming in 30-minute increments
edited January 02 2015

Day-to-day my time is divided by business and project management, phone calls, email, client visits or meetings, the occasional service call, server maintenance and notifications, errands, and lots of other little projects and client site updates and so on. That can make it difficult to spend long amounts of unbroken time on software development.

A lot has been written by and for programmers about “flow”. Ninlabs has a great overview. The conventional thinking is that software developers need long, unbroken stretches of time to be able to focus enough to build software. Software development — like many other disciplines — involves a lot of different pieces and mentally organizing them all can be challenging. If it takes 10 to 15 minutes to regain my concentration, and I’m getting interrupted every 30 minutes, then with conventional approaches, I won’t get any development work done.

2014 was the first year in business that allowed me to dedicate significant portions of my time to programming. At first, I focused on trying to force my environment to change: putting in headphones, asking not to be interrupted, closing my email tab, using shared office space, using coffee shops. None of it gave reliable results. I had to change the way I code instead, so I’ve been working on breaking my software development process down into 30-minute chunks. About a year later, it’s mostly worked out OK. Here’s what I’ve learned.

1. Stop trying to build blue-sky software architectures. Bug fixes, documentation, tests, and small functions can be built in 30-minute chunks and don’t require a lot of mental startup time. Just sit down, pull up your project file, look at your to-do list, and do something. An amazing amount of software can be made out of lots of small functions and a little good documentation.

2. Keep to-do lists right in your project files. I keep three kinds of to-do lists; one kind is for longer-term efforts in a project file, and might look something like:

    /**********************************************************************
    *                                                                     *
    *   TODO                                                              *
    *                                                                     *
    *       make daemonizable (start/stop support)                        *
    *                                                                     *
    *       summary/diagnostic email logs to sysadmin -- in progress      *
    *                                                                     *
    *       reputation tracking for embedded URLs, domains, ccTLDs and    *
    *       gTLDs, sender addresses, and content keywords -- in progress  *
    *                                                                     *
    *       calculate reputation using weighted arithmetic mean           *
    *       https://en.wikipedia.org/wiki/Weighted_arithmetic_mean        *
    *                                                                     *
    *       installer.sh, install grouchy to /usr/local/sbin per          *
    *       http://www.pathname.com/fhs/pub/fhs-2.3.html#PURPOSE16        *
    *       and libs to /usr/local/lib/grouchy/                           *
    *                                                                     *
    *       Add a test-run command that will feed a randomly-selected     *
    *       sample message from the archive into the check-mailbox        *
    *       process, but won't actually commit any iptables commands or   *
    *       database updates or send any user notifications.              *
    *                                                                     *
    *       Pre-load hosted domains after/during config file load.        *
    *                                                                     *
    *       Check configuration file values (db access, archive readable, *
    *       etc.) -- config_check()                                       *
    *                                                                     *
    *       Add critical(), warning(), notice(), and debug() functions    *
    *       for console logging and admin notification.                   *
    *                                                                     *
    **********************************************************************/
    

Another kind is usually somewhere near the top or bottom of the project file, and looks like:

    //  Five-minute TODOs
    //      Fix the extract_markers() function (and related) -- add a flags
    //      parameter to it so that it will filter results or produce
    //      different return types as needed.
    //
    //      Add a third optional parameter to extract_markers() to return
    //      only selected subsets of markers (e.g. 'from*') (see notes in function)
    

And the last kind is simple “TODO” tags scattered throughout the file, like:

    //  TODO:
    //      This function needs to also do the URL filtering embedded in the find_matching_messages() function
    //      (removing partially duplicated URLs)
    

The conventional approach is to keep this sort of feature and issue tracking in a separate bug tracker. That’s absolutely necessary for large software projects being worked on by large teams, but for smaller projects and smaller teams, keeping simple, structured notes right in the project file makes it easy to sit down, pull up the project file, and chip away at something. I’ve built significant parts of this particular software over the past year in quick sessions at cafes between appointments. Going to a bug tracker first would mean another distraction before writing the first line of code. It becomes another system to maintain, another thing that produces communication that eats up minutes here and there. You want to remove as many obstacles as you can between you and the first line of code you’ll write in a 30-minute session.

There’s the added benefit that this approach is flexible. I don’t have to sit down and spit out a whole new complicated function if I’m not feeling up to it. Instead I can choose to read through the code and fix some documentation or quick-fix an easy bug or two.

3. Marauding hordes of small functions can beat the crap out of big classes. I could build my project from the top down, starting out with a big application class, and then building some classes for the most common types of data it will be working with, and trying to drill my way into the squishy center of the code before I completely understand the problem the code is supposed to solve. Or, I could build lots of functions that look like this:

    function last_received_date ($message)
    {
        //  Search the received: headers of $message for the most recent
        //  date/timestamp.
        $headers = $message->received_lines();
        $date = '';
        $last_candidate = 0;
        foreach ($headers as $line)
        {
            $dates = tokenize($line)->dates();
            foreach ($dates as $candidate)
            {
                $timestamp = strtotime($candidate);
                if ( $timestamp > $last_candidate )
                {
                    $date = $candidate;
                    $last_candidate = $timestamp;
                }
            }
        }
        return $date;
    }
    

This function could also be written as a member function in a class that inherits a $message object and so on and so forth — but then that would require more up-front effort, and besides, I don’t even know if this particular function will be needed in the final product. Instead, just sit down and write a simple function that takes some input and spits out some output, and leverage other libraries I’m familiar with.

4. Throw out the current rule book. I’ve heard a lot of silly things about programming in recent years, like, “functions can’t have more than 30 or 50 or 100 lines” — I don’t remember the exact number now. If you’re coding in 30-minute batches, you’re going to have some bad, ugly code in your project. It’s unavoidable. You won’t have the luxury of building a big beautiful architecture for your software. Come to terms with that and accept it. Embrace an iterative development process where your program starts out doing one particular small job, and then gets better at it, and then gets more features added, and so on.

To illustrate the point, the code samples I’ve used here come from a side project I’ve been working on here and there throughout 2014. I finally broke down and started using Github not very long ago, so you can browse the project there. It’s software that is running, live, on my mail server right now, helping to block spam from reaching my users’ mailboxes. But, it started out as nothing more than a simple whois lookup function. At first the function could only take an IP address as a parameter, issue a whois request, and then spit out the result as text. Then, the function learned how to read the whois results and structure the output. Eventually it became its own library file, and then a new project was started, that combined the whois library with a URL library with an email library, and that project started out just by reading in a test set of email headers, finding an IP address, doing a whois lookup, and returning some information about the network that the email came from.

Vast swaths of this project are still rough, but it’s good enough for me to use. (I wouldn’t recommend anyone else using it yet.) I’m constantly going back and refactoring parts of the code, refining it here and there as I improve my understanding of the problem the software is supposed to solve. Design patterns? Beautiful architecture? 7-statement functions? Ain’t nobody got time fo’ dat.

5. Don’t forget that your software sucks. It’s easy for this approach to produce giant ugly piles of spaghetti code, with dependencies scattered everywhere and out-of-date comments peppered in for the kind of seasoning that will give you heartburn later. Keep an eye towards what the project should look like. Iterative software development shouldn’t imply bug-ridden. Writing a test is a good 30-minute exercise. Several of my most complicated, inner-most library functions have extensive tests they have to pass every time they’re changed. Today, the application itself gained the ability to be run in a test mode. Bug hunting is one of my favorite ways to spend a 30-minute chunk; every time I do it, the software gets smarter.

6. Keep the project fresh in your head. I try to focus on only a couple of projects per week, and during that week, I’m thinking about the projects when I’m in the bathroom, in the shower, driving, going for a walk, getting lunch, going to the store. The project is always loaded into my brain, so that when I do finally get a few minutes to work on it, I’m eager and ready and I know exactly what needs to be done.

7. Schedule productivity days. About once or twice a month, I try to schedule a day with a buddy of mine, a writer. He always has something he should be writing or editing. I always have a project that needs work. On these days we go somewhere and we sit down and we make each other work. No phone calls, no email, no distractions. This is when architecture happens. This is when I can sit down and flesh out big new features without being afraid of losing my place. I don’t come up for air until the feature is complete. It doesn’t have to be bug-free! That’s a 30-minute job for later. It does have to work; it needs to pass at least one test. The new feature has to be mostly complete.

So that’s how I do it: 3500 lines of code, in my spare time, with tests, 30 minutes at a time, with a couple of good days mixed in. Flow would be nice, but this will have to do for now.

And in a funny way, there are aspects of this project (and others) that have really benefited from this approach, and not just because I’m able to keep most of them moving along despite a relentless marching line of interruptions. There are a great many mistakes that I haven’t made in these projects, a lot of time that I haven’t wasted, simply because the projects were built out of piles of cheap functions while I developed better and better understanding of the engineering challenges involved. Almost all of the classes and libraries that I’ve built in the past couple of years have grown not from the top down, but incrementally, from the bottom up, starting out as collections of functions. Each class or library does exactly what I have needed it to do — no more, no less.