In my post on how to care, I committed to start blogging more regularly, rather than sporadically. I began last week by spending an hour on my commute each day doing writing and research for the post I published Saturday. On Sunday I took a break from the entire internet. I plan to continue with that schedule. This coming week I’ll be eliminating some time-wasting websites from my life and replacing them with reviewing and acting on my next actions lists, including one for improving my blog. However, because I don’t want my blog to be primarily about blogging, or about my own personal growth, I’m not going to be publicly committing and reporting on my commitments here in the future. For those who care, they can see my current efforts on my public Daytum page.
Since starting at Fog Creek, I’ve been learning about Mercurial from day one, since I’m working on Kiln. It was a big change from my work at Microsoft, where we used a VCS that was much closer to the Subversion model than the Mercurial model. One of my areas of focus in Kiln has been the import tool for teams migrating from Subversion. As I’ve tried to wrap my head around Subversion, Mercurial, and converting between the two, I’ve started to realize that many of the cultural differences between the two communities stem from basic technical strengths and weaknesses between the two products. Feel free to substitute Git for Mercurial, if that’s your cup of DVCS tea.
You could argue that the cultural differences led to the technical differences between the two camps. I suppose that’s probably true for the earliest contributors to the products, but it’s more likely that the technical strengths and weaknesses of each product appealed to those who naturally thought in certain ways, thus leading to the natural congregation of people with similar outlooks on creating software.
But enough of that. On to the differences.
Single project repositories vs. multiple project repositories
One change you’ll run up against, which was initially quite disconcerting for me, is that each project in Mercurial was contained in it’s own repository. I was used to one huge repository with different subdirectories for different projects. Consequently, because the code for Kiln is broken up into 5-10 separate repositories, I’ve spent the last few months asking others on my team if it wouldn’t be better to just combine some or all of our repositories. About once a month. I admit that I still think some combining would be good, but I’m beginning to understand more fully the mindset that leads to lots of small repositories.
This difference is one of the easiest to trace to basic differences in how the products work. Mercurial is much more narrowly focused, as a product, than Subversion is. Mercurial is all about tracking changes to a set of files. Subversion is all about tracking changes to each file separately. Mercurial tracks some repository wide information, such as branches, tags, and repository settings. Subversion allows you to branch, tag, and set properties on the whole repository or any subdirectory, or any random unrelated set of files, if you so desire.
Of the two, Subversion is far more general purpose in nature. It tracks changes to each file and directory separately, only keeping an overall revision number that tracks the chronological order of changes. Because of that, there are many features in Subversion that allow you to operate on a portion of the repository. You can check out a specific subdirectory, map files from another subdirectory, keep your working directory files at different revisions per file (called mixed revisions), and set properties on directories that apply to a directory and all of it’s children, such as which files to ignore when doing an svn status. Its even possible to set different permissions for different parts of the repository.
In contrast, Mercurial manages a single set of files in a repository. Directories are not first class objects in Mercurial, as they are in Subversion, they’re just artifacts of file names. Although internally, Mercurial tracks changes to each file separately there is no way to put the working directory into a mixed revisions state. The DAG cannot handle that type of freedom. Of course, the fact that Mercurial requires you to download the entire repository history to create your own working directory also puts downward pressure on the size of a repository. And because permissions are the same throughout a single repository, if different code needs different permissions, it also needs to be in a different Mercurial repository. The same is true for other settings, such as which files to ignore when doing hg status.
All of these differences naturally lead to smaller repositories that typically contain one project in Mercurial, and larger repositories that typically contain many, if not all projects, in Subversion. If you’re coming from Subversion, you’re going to want to get used to it. Fortunately it appeals to your innate desire to componentize — that is an innate desire, right? For me it is, and Mercurial makes it easier to do it at the project level. Of course, because Mercurial does less, it leaves to other systems the management of multiple repositories (see bitbucket and Kiln).
Sam Hart, when he decided to switch from Subversion to Mercurial, discussed this exact phenomenon:
“If you’re like me, when you originally set up SVN you did so in the laziest way possible.
“Setting up SVN repos is more work than it should be. It involves using commands that you normally never have to touch (svnadmin), setting up new entries for those repos in your http server’s configuration files (if you’re using Apache and WebDAV), and setting up user permissions to those repos. Thus, the lazy way to set them up is to make one central SVN repo under which you have multiple sub-repos. This has the advantage of making your repository very easy to maintain. However [it] has a big disadvantage in that a user with write access to any sub-repo will have write access to the entire repo.
“In Hg, on the other hand, setting up a new repository is much easier, and maintaining multiple repositories more manageable. So, if you’re like me, you may be tempted to remedy past sins by splitting your single gargantuan SVN repo into smaller Hg repos.”
Commit often vs commit when “ready”
Another change you’ll need to adjust to is to commit often. You’re probably used to making a bunch of related (or unrelated) changes, then doing some testing. You may build a version of your product and have others do testing. You’ll probably run automated tests, possibly multiple sets of automated tests. And finally, you’ll check in.
If you do this in a team using Mercurial they’ll wonder where you disappeared to while your code was being written, complain about how large the code reviews are, and be frustrated at how slowly you iterate on your code towards a good solution.
On my team at Microsoft, we had a concept of a shippable chunk of software. This helped guide the creation of branches in our centralized VCS. We could work in the branch, possibly with one or two other developers, until we had something we could reasonably ship, then merge the branch back into the main development repository. Depending on the rules for checking in to a VCS, whether centralized or distributed, software teams develop an understanding, either explicit or implicit of what a “committable chunk” is. What amount of code is worth committing, either for review or sharing with others.
The key change in mindset for me has been to make my own “committable chunks” much smaller than they used to be. No longer do I make hundreds of changes in tens of files, tying up another developer for hours in code reviews. It’s easy to make frequent commits locally, and push those to a personal branch on Kiln regularly for review.
But DVCS’s don’t just make it easy to have smaller committable chunks. They make it easier to manage committable chunks of all sizes. Because I work against a personal repository and merging is so easy, I can commit almost minute by minute to my personal repository, push multiple times a day to the feature branch I’m working on, push occasionally from the feature branch to the main development branch, and handle multi-feature pushes from development to a stable release branch. Obviously, those are all possible with a CVCS, but they always took so much time and effort to manage the branches, do the merge, and verify that nothing broke. In practice, that meant that steps were left out, and things slipped through the cracks.
Now, my changes to code are clearer, my original intentions more obvious, and I feel far better with my code in source control. I can look at changes at a small granular level, or I can look at the big merges.
Branch always vs. branch rarely
Closely related to a cultural norm of small, frequent commits is a norm of branching. Every clone of a Mercurial repository introduces a new branch once a change has been made. It’s also easy to branch many times within that clone. When I first made the switch, I didn’t really understand this. I knew I could work separately from other developers in my own repository, but I didn’t think of it conceptually as branching. It was more like I had my own sandbox, which I could then merge with the main repository when I checked in. And the idea of easily branching within my own repository still seems new to me.
But I’m learning to embrace the value in branching. As with frequent commits, it’s the ease of merging that makes the benefits of branching so readily available. And I’m beginning to value the power of having branches within my local repository. I can work on bug fixes separate from a major refactoring work, and easily (and quickly) switch between the two using a simple “hg up” command – even when I’m offline. That’s great for the times when I’m deep into feature code and a sudden urgent bug pops up that needs to be fixed and released immediately. I can also switch back and forth between work on two different features , which is great when I get stuck and just need a mental break from one of them. Also, it makes it super easy to prototype out new ideas without messing with my regular development.
One counterpoint to the ease of branching is that it may isolate developers. John DeRosa registers his concern about this:
“Additionally, I think distributed SCMs like Mercurial have a not-yet-fully-appreciated problem in making it too easy to not [ever] check code back into the main pool. With a local repository, a developer can feel protected from accidents and continue working happily for quite a long time. And then, say a year down the road, he/she does a massive check-in and discovers an integration problem. Branches, or a local repository that is effectively a private branch, should be easy to make — but not too easy.”
Let me explain why I don’t buy it. First, “a year down the road”?! Seriously? It says something that you have to imagine a scenario so horrible and unlikely in order to envision easy branching as a bad thing. I think that the author likely didn’t realize how easy merging usually is with a DVCS like Mercurial. And he must have totally forgotten that this lone maverick developer could have been merging the main development line into her own repository every day or week. The right solution to this imagined (and barely imaginable) scenario is not to eliminate easy branching, since without it the lone developer will do the same thing, but be much more likely to lose her work because it won’t be stored in a repository. The right solution is to fix a broken culture that enables someone to go a full year with no accountability for their work.
Source code files vs. all code files
Another important difference relates to what files you put in the repository. Because Mercurial and other DVCS’s don’t handle versioning of large files well, it is much more tempting to store them in a different way. This most obviously manifests itself in the storage of built binaries. If they are largish, and you want to keep lots of copies of them (nightly builds backed up for QA purposes, or even just weekly or monthly builds) then your repository becomes quite large and unwieldy very quickly. These types of files typically don’t diff well, making diffs between versions very large, and because the files are very large themselves, it means that downloading the repository takes much longer.
In this area, Subversion currently has a clear advantage. Only the files in the working directory are downloaded to client computers, so storing the history of large binary files only requires storage scaling on the server. Bandwidth is significantly reduced. Because it’s fairly simple, many Subversion installations have used it to track changes to built binaries and other very large files. The challenges to scale and management are limited to one machine, the server.
Naturally, users of Mercurial push handling of these large files to other systems. Their VCS is the location for their source files, typically a bunch of text files, which external tools then build into large binaries which are almost never stored in the same or another Mercurial repository. It is true that efforts are underway to alleviate this weakness in Mercurial, though I’m sure some don’t see it as a problem at all. The bfiles extension is an attempt to limit provide a more centralized model for certain large files. Of course, it has tradeoffs, but the fact that it’s being actively developed indicates that, at least for many, the tradeoffs are worth it.
For now, I’m happy that this aspect of Mercurial motivates me to automate more, to maintain more of the components of my products as code (in some form) that is compiled (using some method) to create these human-unreadable products.
Conclusion
There are obviously different ways to look at these cultural and philosophical differences between Subversion users and Mercurial users. One might look over the differences and conclude that Subversion seems much more flexible than Mercurial. Therefore, it must be better. Another might see how much better Mercurial handles basic source control features, such as branching, merging, and tags, and conclude that it is therefore a better product. It’s pretty obvious to me that these two views are quite related.
Michael Haggerty makes this point quite well in his post Git, Mercurial, and Bazaar—simplicity through inflexibility. The discussion is about the merging differences between Git and Subversion, but the principles apply to Mercurial as well. He argues that the very flexibility of Subversion is what makes merging more burdensome:
“Starting with release 1.5, Subversion, ironically, supports a much more flexible model of merging than the DAG-based DVCSs. Changes from any commit can be merged to any branch at the single-file level of granularity, enabling all of the operations listed above and some even weirder things (for example, a change that was originally applied to one file can be “merged” onto a completely different file). If your workflow demands this sort of thing, Subversion might hold significant advantages for you.
“But there are also many disadvantage to Subversion’s flexibility:
- Subversion’s merging model is more complicated than that of DAG-based VCSs, and therefore more complicated to implement and less predictable.
- It is much harder to visualize the history of a Subversion project (contrast that to DVCSs, whose history can be displayed as a single DAG).
- Subversion merges are innately slow, because of the large quantities of metadata that have to be manipulated.
- The bookkeeping of SVN merge info requires more user conscientiousness, and mistakes are not as easy to spot and fix.”
While he doesn’t take a stand on which is better, a CVCS like Subversion, or a DVCS like Mercurial or Git, I will. Mercurial (and other DAG based DVCS’s) provides a level of intrinsic guidance to developers through the limitations it has. Like many other great products, it is defined in part by what is not included. One might easily say that it is defined in large part by that. Products like the original iPod and iPhone both have this same feature. By focusing on the most important features, and specifically limiting users choice in other areas (changing batteries, how to buy and download apps, etc.), Apple created products that are wildly successful. True, they may not be as flexible as an Android, Blackberry, or Windows Phone. But they got the right things right.
And I think Mercurial is a step in that direction. I don’t think it’s there yet, but I don’t think anything else is any closer. Some other DVCS’s (git, at least) are also heading in the right direction, though they may be coming from a different starting point. Mercurial creates a philosophical and cultural starting point because of the technical choices that define both its strengths as well as its weaknesses. That philosophical starting point is a fundamentally better starting point for software development. It leads to greater componentization, greater granularity of history, more productive use of development time, and more automation.
Earlier this month, Merlin Mann wrote about one principle that is more important than we typically acknowledge: First, care. He begins by discussing the common challenge of staying focused on the important stuff. Staying focused is the easy thing, he reminds us: just do one thing at a time. Of course, he knows that an obviously tautological statement like that is not the solution, so he tells us what is: we must care more about the one thing we’re doing than anything else. If we don’t, then we’ll naturally flit from task to task without the sense of focus we desire. Eventually our lack of focus becomes the most important thing we focus on, but that just makes the problem worse. The only real solution is to care so much about something that the question changes from “How do I maintain focus?” to “How do I get rid of everything unrelated to the one important thing I’m working on?”
A False Start …
Fortunately, I read this article at just the right time – one month into developing my nightly post-mortem and planning habit. Because I actually did care about that habit, I recognized the truth in Merlin’s post. But I also recognized that the rest of my activities perfectly exemplify people focused on their lack of focus. And because I was less than a month away from starting a new habit, I knew what it should be: It was time to eliminate my focus issues.
Of course, I was thinking about it all wrong. And if you had actually read Merlin’s post, you’d know that. Go back, try again. The whole point of his post is that you cannot make eliminating distraction your focus. In reality, I needed to replace my bad habits with good ones. My bad habits consist of mindless internet surfing to a variety of sites. When I’m stuck on a problem at home or at work, I gravitate to a web browser and drown out my “stuckness” by reading all about politics, or watching movie trailers, or learning about some cool new technology, or following random news that isn’t really all that important. Oh, I usually get back to the problem eventually, and usually solve it … eventually. And then I go back to the mindless surfing.
Another False Start …
So I listed the triggers for mindless surfing, as well as the sites I would regularly visit. Once that list was in place I started thinking about what to replace it with. I came up with all kinds of ideas, ranging from practicing code katas, to working through my book reading list, to exercise, to doing a better job of reviewing and acting on my next actions lists, to cultivating my blog.
Over the last few weeks I tried to narrow this down. Okay … not really. What I really tried to do was come up with 8-9 small steps that replaced my lack of focus with 8-9 new habits. Focus Fail! Well, it wasn’t quite that bad, but I was trying to replace my lack of focus with a few different things, which would mean that I would still have a lack of focus. And of course, that made it hard to actually come up with the steps to take, and what I did come up with was unorganized and lacked, well, focus. And besides, it seemed about 10 times harder than my nightly post-mortem habit.
Lessons from False Starts …
However, in the process, I discovered some important benefits to Leo’s idea of anticipating the start of a new habit without actually starting it. First, you don’t jump into something prematurely. If I had tried to start with the ideas I had a week ago, or two weeks ago, I’m pretty sure I would have failed to keep up the changes. Oh I may have stuck with them for a few weeks, or even the whole two months, but the lack of natural cohesion would have eventually broken up the habit.
Second, it’s pretty easy to try out some of the ideas you have by doing “test runs”. Basically, I tried one of the small steps I would take for a day or two, just to see what it was like. Writing last week’s post on the habit creation process was one of these experiments. The key to this is to make sure your “test runs” are not a priority. My priority throughout was still the nightly post-mortem, but with my spare mental energy I also tried out some of the ideas I had, and some of them are even sticking. However, they are not my current focus, and if they fall by the wayside that’s ok. Other activities haven’t stuck at all, or were obviously not useful, so they won’t be part of my next habit.
What I re-realized just a few days ago is that I need one thing to focus on. I knew when I began this process what it would probably be: developing my blogging into a regular habit, rather than something I do once every few months. But at the time, I just didn’t care enough. I didn’t care enough to overcome my fear of failure. I didn’t care enough to push off the other interesting things I could be doing with this magical free time I’ll be creating for myself. Which wouldn’t be created, of course, if I maintained my current lack of focus. I didn’t care enough to commit myself and really do the hard work it will take to develop a blogging habit.
How to Care …
When I realized that, I discovered another important truth: Merlin’s instruction to “First, care”, can be spurred on by “Commit to something”. Caring is important. Caring about lots of different things isn’t going to solve your focus issues, however. But sometimes committing yourself wholeheartedly to something can increase how much you care. With the assumption that I’ll focus on this until I it’s a lifelong habit, and then go to work on the many other failings and weaknesses in my life, I can truly focus on it and nothing else. I can care about it more than anything else. I know I’ll get to the other stuff, eventually, which is no worse than before I made the commitment. And I’ll only get better at committing and following through with practice, thus increasing my chances of actually succeeding at all the other stuff.
So that’s my commitment – make blogging a regular habit. But what to blog about? I don’t know exactly. I’ve enjoyed writing this post and the last about habit development. I’m a developer at Fog Creek, and that means there are a bunch of technical topics that interest me also, and maybe Fog Creek needs another blogger. I could just as easily blog about religion or politics, both important subjects to me. I don’t have any active hobbies to write about as I’m not currently running, and don’t have the money yet for flight lessons. Writing about one of those hobbies, or another one, could help me get motivated to do more related to that. Or I could blog about blogging (like I am right now!) and the things I’m doing to get better at it. Or not.
I guess it comes back to the motivations I have for blogging, beyond the fact that I just committed to doing it. I want to blog to become a better writer, but I can do that with any topic. I want to blog to create a public reputation within the software and business communities I’m a part of. I want to blog to explore topics that are interesting to me, because forcing myself to express and understand those ideas is an important way that I learn. I spent a year and a half teaching a Sunday School class and learned more about the scriptures because I had to express myself well in order to teach. I’d like to have a similar experience with other areas of knowledge. I want to become an active part of at least one online community.
It’s that last point that will probably make this habit a little bigger than just blogging. I don’t want to just talk into my blog, disconnected from the rest of the humanity – I want to be involved in conversations. And that’s what I’m committing to. I’m still working out the individual steps, and over the next couple months I’ll be committing to those steps each week here on my blog.
I first read the posts at 6Changes.com just before Christmas. At the time, I was preparing for the yearly planning that my wife and I do each January. I already wanted to make some changes in my own life, and 6changes.com was like a small revelation. It convinced me to tackle an important change, gave me a set of things to do, and happened to be at just the level of detail I needed. I’ve gone through enough attempts at self-improvement to know they don’t all stick. That naturally made me wary of some of the claims Leo makes, but it also helped me to recognize the truth of many of his ideas.
In summary, he advocates the following regimen for making a change in your life:
- Choose a habit to develop
- Only work on one at a time
- Build anticipation up to a starting date
- Commit publicly to the overall habit
- Break the habit into 8-9 small steps
- Choose habits and steps that can be done daily
- Add one of the small steps each week for two months
- Commit publicly to each step when you start it
- Report publicly on your progress
You can check out his site for more on the reasoning behind these steps.
I have spent the last 7 weeks following this plan step by step. I recount the experience here because a detailed blow-by-blow of one person’s attempt would have helped me when I started.
Preparation …
After reading through the ideas at 6changes.com, I wanted a habit that would help me continue developing new habits, as well as get me doing some of the simple things that I wanted to make sure got done each day. My nightly post-mortem and planning session was born. This includes a variety of small but important steps: writing in my journal, getting to inbox zero on my various inboxes, going through my tickler file, planning the next day, and nightly prayer.
I followed Leo’s advice and didn’t start right away. I thought through the small steps I wanted to take each week, and made plans to start the first full week of January, giving me a few days to recover from New Years and having family in town. I came up with 8 small changes to my nightly routine that, combined, would make a big difference. And because I was starting out so small, it was really quite easy. Also, I told my wife about the new goal, and I started tracking it at Daytum.
I broke my habit into the following small steps:
- Organize desk
- Record date in journal, scan inboxes, and clear daily plan
- Write in journal
- Process inbox items
- Process ticklers for tomorrow
- List things I want to get done tomorrow
- Prepare to pray
- Nightly Prayer
Getting Started …
I already prayed each night before bed, so my first step was just to organize and clean up my desk, before my prayer. Organizing my work area took all of about 1 minute and typically just involved me plugging in my laptop, cell phone, and Zune. Sometimes there was more to put away, like when my son decided to do his homework on my desk and just leave it all there. But it never took more than a couple minutes to finish, so it was easy, even when I stayed up quite late.
The next week, I opened up a page in OneNote and recorded the date (Alt-Shift-D for you keyboard fanatics). Then I closed it. I scanned my inboxes, but did not process anything. And I looked at my calendar. Total added time: 30 seconds. That was the beginning of my journal writing habit. I made it to the end of the second week without missing a day.
The third week, I expanded my journal entry by actually writing a little about the day, sometimes spending a few minutes recounting something I’ve been thinking about or an interesting story. This added about 5 minutes to the process. My entries aren’t typically very long. Sometimes, I add some thoughts about what I hope to do the next day. So far, I haven’t missed a day writing a full journal entry since January 17th.
Building on the Habit …
Now to the most obviously beneficial change: processing my inboxes. This could have been daunting, initially, since I hadn’t been doing a good job of this. That is to say, all my inboxes were full of crusty old stuff that had been lying around for weeks or months. Even though I knew I would start working on this part of the resolution near the end of January, I didn’t try to get ahead of myself by changing my inbox processing earlier or doing a big purge the day before. I just eased into it. I figured if I went through one or two items each day from each inbox I’d be pretty close to inbox zero by the end of the week.
Of course, I went through a lot more than 1 or 2 items each day that first week. I had to, just to keep up. But it’s pretty easy to delete or archive all the random stuff that doesn’t require any action. And I made sure to finish each day with less in my inboxes than I had started with. Ever since then I’ve been at inbox zero in my personal email and physical inbox every single night. My work email is a slightly different beast that I tackle at work anyway. I do take care of the easy stuff at night though. And now that I’m current on all that it usually takes just a few minutes.
Next step: tickler file. Ever since first reading about the tickler file in Getting Things Done, I’ve thought it would be a great tool to use. Of course, using it absolutely requires some sort of daily habit, or it’s just another place to lose track of things that are important. Well, now I had a daily habit that I’d kept up for a month without fail, so I added going through my tickler file, which I keep in OneNote, each night. That was a huge change, in that I now had an easy way to remind myself of something at any point in the future.
The one modification I’ve made to the tickler described in GTD, is the addition of four “week” folders for the four weeks of the month, which I go through on Sunday. Then I only need seven “day” folders instead of 31. It’s easy to remember to go through the longer time periods when I should because I put reminders to do so in the shorter time periods. For example, my Sunday tickler file has a reminder to go through the weekly tickler file, and my “4th week” tickler file has a reminder to go through the monthly tickler file, etc.
Once I had the habit of looking back by writing in my journal, dealing with the stuff at hand by processing my inboxes and tickler file, it was time to look forward by planning the next day. For me, it’s a really simple process that just involves listing in order the things I expect to do the next day. I usually include my plans for the commute (1.5 hours one way on the train and subway), focus goals for work, and how to spend the evening with my family. I also add reminders to my calendar for things I cannot forget. This addition makes it easy for me to just get up in the morning and go. I don’t have to think as much about what I should be doing.
Going Forward …
All that brings us up to Valentine’s day. I’m spending this week and next improving my nightly prayer, which is still too perfunctory. And I’ve started working out a plan for the next habit to create. I have a lot more confidence going into the next one, because of the experiences I’ve had over the last 7 weeks.
Overall, it’s been a great experience, one that I hope to repeat in March and April.
Well, I’m now at Fog Creek. I love it here. The benefits they mention in their recruiting materials are all real and all great. I love the food, I love the snacks, I love the flexibility, I love the views of New York and the Hudson.
More important than that, I love the work.
Microsoft
Over the last few years, working on Outlook at Microsoft, I became more and more aware of good engineering practices. At Microsoft, so much of product development is handled by others, that initially, it was easy to sequester myself in my office, write lots of code, then fix lots of bugs, and feel a certain sense of accomplishment. I had to learn what it meant to write code that could be localized, code that was considered to have quality, by the standards there. I also saw lots of code, much of it good and much of it bad. I became very familiar with old code and still managed to forget plenty of code that I wrote after only a few months.
But I also started learning about what good code could be. I started to develop a certain feel for good code, and also started to recognize the natural impediments to writing good code, some of which are inherent in shipping a product that people will buy. I learned about TDD and tried to practice it in my work. I shared the ideas and concepts with my team. I started to get impatient with pace of adoption at Microsoft, while at the same time recognizing some of the reasons it was slow as valid.
As I approached my 6 year mark at Microsoft, I really started feeling like it was time for a change, both professionally and personally. Our family was ready to live somewhere that didn’t involve 8-9 months of rain each year. So I started feeling around for jobs. The Fog Creek opening landed on my radar at just the right time and, long story short, I started on September 14th.
One part of that long-story-made-short bears mentioning. After making the decision to take the job at Fog Creek, I was once again impressed at what a great place Microsoft is. My coworkers and management were very supportive of both me and the move. I’ve reflected on my time there and feel very good about what I learned, what I accomplished, and the products and features I was able to work on. I would definitely recommend it as a great place to work.
I’ve had a few friends and acquaintances ask me how I went about making this decision, and I felt like it would be good to share some of the reasoning that went into it for others. I’m not going to write a “10 reasons to switch jobs now” list or a “7 reasons small companies are better than large ones” list or a “How to land a dream job in New York City” essay. Rather, I’d like to talk about how I see my career and hopefully pull out some universal principles for any career in software development.
Own your career
First of all, you need to own your career. I’m not going to go into a lot of detail on this one, because any good advice on careers in general, or software development careers in particular, will offer plenty of detail on this one. I just want to call it out as a necessary pre-requisite to all the other ideas.
Know what you want
As my wife will tell you, I don’t normally know what I want. I tend to be pretty ambivalent about a lot of things. About 6 months before this career change, however, I started taking about 30 minutes a day just to think about and write about my career. I went back over all the notes I’d ever written down, all the goals I’d ever considered, all the crazy ideas I’d ever had. I organized them all and tried to figure out what made me tick. This was a pretty simple exercise, easy enough to do before breakfast (what can I say, I’m a morning person). But it paid huge dividends almost immediately. I went into work more excited (still at Microsoft). More importantly, I started to really care about where I would end up. I don’t know if my 30 minutes a day strategy would work for everyone, but you really need to become passionate about something, or your career won’t go anywhere.
For me, that passion boiled down to becoming a software craftsman. Which leads to another important principle in creating your ideal career: strive to be your very best. I connected with the ideal of a being software craftsman, which is admittedly still being defined. Whether that includes TDD as pre-requisite, or whether that means trying to be a Duct Tape Programmer is secondary to the ideal of crafting the best code possible within a set of interesting, creativity-inspiring constraints.
Broad and deep
A few years ago I had some interesting conversations with a friend of mine about whether education should initially take a student deep or broad. I don’t know that we ever answered that question, but we both agreed that a good education is ultimately both. At Microsoft, as at most big companies, I got to go deep. I learned what it meant to work on a single code base, in a single language. During my 6 years there I worked on two or three major feature areas. So much of the product development and marketing was handled by others, and many of them I never met or knew. I was able to gain a lot of knowledge about a very limited set of technologies, product areas, and functional roles. A big part of my desire for a change, any change, was to do something new. I wanted to come to a small company so I could at least be closer to the sales, marketing, and customer support side of the business. I wanted to change my technical focus, and learn more about new technologies: web development, new languages, whatever. It was time for me to broaden my experiences.
Make incremental improvements
When I told my wife that I had submitted my resume to a company based in New York City, her first reaction was incredulity: “New York City! I never want to live there!”. I assured her it was just for interview practice, because I felt the same way. I bring this up because sometimes our career goals will take us unexpected places. I personally would love it if Joel woke up one day, realized how much more profitable Fog Creek could be if it were based in Fort Collins, Colorado, and decided to move the whole company there. But I doubt that will happen. And that’s ok. I realized, and so did my wife, that we weren’t going to get the perfect situation at this point (especially in this economy). We knew even before the job search began that there would be some compromises. As it is, I’m extremely happy with the compromises we made, and we’ll probably be at Fog Creek for quite a while. I expected to give up more and not get as much. But no, we don’t live in New York City. They couldn’t handle our three boys, so we’re actually enjoying a great place in New Jersey.
Sometimes, be the worst
I know this advice has been around for a while in the body of software development career advice (links!), but that’s another reason I made this move, and it’s already started to bear fruit. I was by no means the best on my team at Microsoft. But I’d been there a while and I was too comfortable. I knew, coming to Fog Creek, that it would not be comfortable, that I would stretch myself technically, socially, and in other ways. But that wasn’t just something I had to live with. It was a necessary change for me, one that I looked forward to. It was time to grow, to learn, to stretch.
In saying this, I recognize that you probably shouldn’t go through your whole career this way. It would probably get pretty depressing if you always felt that you didn’t quite measure up. Plateaus aren’t bad things, they allow us to step back, to look over the things we’ve learn, to rest and recuperate in a way. And being the best is also a valuable position to be in, when you can mentor others and learn from their fresh new ideas.
And so…
At this point, I’m very happy with the move. I’m excited to be working on Fog Creek’s new product, Kiln. I’m just as excited for the new version of Outlook to ship, and am itching to get my hands on a copy of Windows 7.
Life is good.
In my last post I mentioned that I was doing some practice with the binary search algorithm. I wanted to approach it with a slightly different mindset and see what kind of an algorithm that led to. So I decided to think of it as a walk. I would go “visit” locations in the array and see if I should turn right or left at each one. Doing this in C meant that I could do some crazy stuff with arrays – nothing I would do in shipping code, but fun to play with nevertheless. Here is what I came up with:
1 char ChopWalk(int value, int *array, int size, int *poffset) 2 { 3 int walkto = size / 2; 4 int direction = 0; 5 char found = 0; 6 7 if (walkto == size) 8 return 0; 9 10 if (value == array[walkto]) 11 { 12 found = 1; 13 direction = 0; 14 } 15 else 16 { 17 if (value > array[walkto]) 18 direction = +1; 19 else if (value < array[walkto]) 20 direction = -1; 21 found = ChopWalk(value, &array[walkto + direction], direction * (size / 2), poffset); 22 } 23 24 *poffset += walkto + direction; 25 26 return found; 27 } 28 29 int Chop(int value, int *array, int size) 30 { 31 int result = 0; 32 if (ChopWalk(value, array, size, &result)) 33 return result; 34 else 35 return -1; 36 }
So for the last couple of weeks I’ve been practicing the binary search algorithm, as laid out by Dave Thomas here. I’ve done both iterative and recursive variations in C++ and then in C. I love the value of katas for learning new languages and features. I know both C++ and C, but doing this simple problem in both programs allowed me to work on learning new areas in each. First, in C++ I took the time to use the standard template library (stl), because I’m not very familiar with it. I just used vectors in my tests, but it helped me to become familiar with the stl documentation and the concepts of iterators as used in the stl.
Next, I did the same variations in C. This was fun because C is so basic, and it’s been a long time since I coded in straight C. You don’t worry about concepts like objects or functional programming (though they may help you think about the problem). It gets you really close to the underlying physical model of computation. You’re forced to think more about how the memory is laid out, and how the algorithm takes advantage of that, whereas with C++ and the stl the whole problem and the solution are more abstract, just slightly more removed from hardware.
In addition to the languages themselves, it’s also a chance to practice my use of tools. I used the work to hone my skills in vim, to understand our build system at work more fully, and to consider testing frameworks.
Uncle Bob’s tweet from a few weeks opened my eyes to the value of practice in honing my development skills, and led me to read a lot more about practicing and code katas. One of the things I love about having a kata, like the prime factors kata, is that you can focus on different skills and techniques each time through. You can do it in different languages, with different testing frameworks, using different text editors or build systems, on different operating systems, etc. I’ve done it a few times in the last few weeks, mostly in C++ (my current language for work projects). But I’ve used Visual Studio, vim, my work’s build environment, and other variations. I still haven’t used a decent testing framework, but that’s one I want to add.
Having gone through the prime factors kata a few times now, there are some changes that I feel should really be made near the end. Ok, the rest of this post isn’t going to make much sense until you at least read through the prime factors kata. Go do it now.
Commenters rightly pointed out that having the additional
if (n > 1)
clause added in test 4 isn’t intuitive and violates the TDD principle of just adding enough code to make the tests pass. But they then added the caveat that without it the step to loops and the candidate variable in test 7 is pretty big for the refactoring step, mostly in terms of the mental leap. I agree.
But if you really do the easiest thing you can as you add tests, you don’t run into that problem at all. The key is when you get to the test for finding the prime factors of 9 (i.e. the seventh test in the slides). The slides have you first creating a variable, candidate, to represent the number 2. This may makes sense as a refactoring step done before writing this test, but lets ignore that for now, and assume you haven’t done it. Then the easiest way to get the seventh test passing is just to copy the entire while loop below itself and change the 2’s to 3’s. Not only is that dead simple (Ctrl-C, Ctrl-V), it also makes the following steps much more obvious. At this point it’s easy to see how important the candidate variable is, so you create it, and increment it between the two while loops. Then the two while loops are identical so you just nest them inside another while loop. The remaining refactoring work to for loops is the same as the slides.
One of my biggest frustrations with the Windows Mobile task application is that it synchronizes your completed tasks. After a while, this greatly outnumbers your active tasks, making the task app really slow. So I periodically move my completed tasks to a separate task folder (I like to have them around for searching). Alternatively, you could just delete them once a week or so. The following Outlook macro code can be used to move all the completed tasks in your tasks folder to the first tasks subfolder.
Dim fld As Folder Dim fldComplete As Folder Set fld = Application.GetNamespace("MAPI").GetDefaultFolder(olFolderTasks) Set fldComplete = fld.Folders.Item(1) For x = fld.Items.Count To 1 Step -1 Dim objTask As TaskItem Set objTask = fld.Items(x) If (objTask.Complete) Then objTask.Move fldComplete End If Next x
Jeff Atwood recently discussed The Dark Side of Extensions, and advocated taking the five most useful popular extensions to Firefox and integrating them into the product as features. Before the Firefox team decides to go do that, it might be useful for them to step back and consider what they want Firefox to be: a solution or a platform.
The platform vs solution question eventually rises in any successful software project. The reason is that successful projects are used and usually by a lot of people. The more people use your software, the more people want to change it to fit their particular needs, address weak areas in the product, or adapt it for new uses. Once this pressure starts being applied, from your customers, you have a choice. You can hire extra people, work extra hard and try to take care of the most frequently requested changes as quickly as possible. Or you can do the work to make your software “extensible”, essentially to turn it into a platform. Of course, you can also do something in between.
The most obvious example of a platform in software is the operating system for a personal computer. Windows, Linux, and Mac OS X, are all platforms for personal computing. Of the three Linux is probably the most pure as a platform. Mac OS X is the least pure, because Apple sees it more as one part of a larger solution that they offer to users, and so don’t give their developers the same level of support that either Microsoft or the open source community does.
Those who advocate a pure platform approach like Linux, because you can include or exclude “bloat” (i.e. features) that you never use. This was very apparent in the comments to Jeff’s post, where tons of people said, in effect, “I don’t want a bloated browser like IE or Opera – I just want it to include the things I use.” Mozilla is definitely going down the platform route with Firefox, and the users who therefore choose Firefox reflect that.
Microsoft tends to take a middle of the road approach with its software. Windows and IE both attempt to be solutions and platforms at the same time. And both had great success because they got the killer “apps”: Windows had Office and IE had all the websites that only worked in IE for years. Heck, even now my wife won’t switch to Firefox because some pages “don’t look good”, despite Firefox’s apparently better support for standards.
Apple obviously approaches the problem by trying to produce a solution, and that has been extremely successful for them in certain markets (can anyone say iPod? iPhone?). In the past they haven’t had as much success as a platform, so it will be interesting to see how the new iPhone SDK works out for both Apple and it’s developers.
Ultimately, I’m not sure that creating a platform for end-user extensions is the right way to go in the browser market. Platforms are market makers. Windows created a huge market for all types of software by creating a successful platform for personal computers. The real market that browsers enable is the market for cool web sites and services, not the market for browser extensions (how many browser extensions are profitable products?). A browser needs to focus on becoming a platform for web sites and services, and becoming an easy way to get to those for end users. Browsers are not the only user interface to the internet, but they are the primary one and can remain so only if they provide an excellent platform for creating web sites. Of course, there are huge problems in trying to do that. But if a browser can do that, in a way that gives it a competitive advantage, it will become the platform for the web and be more successful, and probably more profitable, than Windows.