Good software is developed by good software developers. It involves a discipline not found in most programmers. Rigorous version and configuration management, checklists for style and review, “desk” checking reviews before commits, automated (continuous) builds, and fully automated test frameworks are all necessary steps to successfully, reliably delivering executable software that works. And most importantly when software fails you know exactly how you created it from what source and tool chain (and conceivably what version of the tool chain) using what version of the recipe so you can reliably rebuild the failure and fix both it and the test environment. Great developers may be great architects, but every great developer is a good disciplined developer first, and they really don't know how to build software otherwise. Software construction discipline is fundamental to good software. Every great software product and successful project has at least one developer near the top that maintains the level of necessary software construction discipline. And since culture comes from the top of any organization, it cascades down through the committer culture. I've been fortunate enough across the ~25 years I was around software development projects (IT) and products (ISV) to always have at least 10% of the organization and sometimes as high as 30% be such developers. The rest of the team had an easy time contributing appropriately because of the culture at the top set by the few. We all know what needs to be done — just few of us do it. (I don't do it easily — I had to work bloody hard even to attempt the discipline, literally dragging myself through a check list.) So what does this have to do with free and open source software? While I have no proof that the statistics are linked, I find it interesting that the general statistics around failed IT projects (~80% are never deployed) and moribund projects on forges like SourceForge (~90%) are very close. I have encountered the rare and the few disciplined software developers in many places — corp IT, product teams, FOSS projects. But most developers I know aren't like those disciplined developers. Without such discipline at the top, I believe no project can succeed regardless of whether it's ISV, academic, IT, or government, and whether open or closed. Said another way, only a strong disciplined developer culture can lead to a successful project, FOSS or otherwise. So lets link this idea of disciplined software construction to a collaborative community. Everyone seems to agree with O'Reilly's observations that one needs an "architecture of participation" to build a successful vibrant community. This applies to any community, and not simply open source projects. I'd argue that without the essential grounding in software construction discipline, you can't build such an architecture through which a community gets involved. Here’s why. Every project community needs a simple way for the broader world to get involved with the project such that the community grows: there needs to be an on ramp. The goal is to allow any old stranger with an interest in what you're building to become engaged enough to become passionate enough to contribute. Ways to make this easy are:
- The labelled source snap shot you make available must build at all times.
- There is a simple script/tool/makefile/recipe that builds the source.
- There is a test environment (however simple) that can confirm at the very least the build worked on known environments.
At this point in history, said stranger has been able to download and build the project, and then trivially confirm that what they build “runs”. I worked on the Rotor team at Microsoft in 2003. We released a million lines of code in a shared source project that represented a working cut through the .NET CLR matching the ECMA CLI standard. It trivially built and there was a test harness that trivially demonstrated that what was built worked on Windows XP, Mac OSX, and FreeBSD. The only documentation released with the software was a trivial couple of pages that told you the script to set your environment, the build command, the test command, and a brief discussion of how the source tree was organized. Within 24 hours, we received our first righteous enhancement of ~20 lines of code that provided a 10% improvement in JIT performance. In the next 24 hours we received our first well-formed bug report, the code to properly fix it, and the test case to demonstrate the fix. We had made it trivially easy to participate. Strangers that might be interested could focus on doing something interesting to them, rather than struggling to build anything useful. Wasting a potential contributor’s time does not make for a good first impression, and in a world where time is at a premium, you may lose them in frustration for good. Good developers probably don’t want to waste time on projects that can’t even get the basics right. Thinking you’re weeding out people that “can’t figure it out” wastes enormous potential. Now we get to the next steps.
- There is a published guideline for how code needs to be formatted.
- There is a published guideline for what minimal bar needs to pass for a contribution (e.g. formatting, issue logged, test harness passes cleanly, perhaps new test cases)
- How to contribute is published. (What makes a good bug report, How to submit, etc.)
Without this collection of guidance, existing project committers are essentially asking to waste time with poor submissions and answering the same questions over and over again on whatever physical comms tools they put in place (e.g. IRC, email distribution, etc.). Even building out an FAQ to answer these basic things that should have been handled by an automated script is a waste of time. Lastly.
- Tell people what to contribute.
Some potential community members will simply want to use the project for their own research/learning. Great! Out of such experimentation, new innovations and directions will appear, and hopefully if the first six steps exist, the work will be contributed even if it only lives in a side branch or side tree. Some potential community members will want to use the project for their own daily work, and giving them the ability to hand back bug reports and fixes or extensions is how the quality and functionality in the project improves. Some folks care and want to use the project in their daily lives, but also want actively to contribute and here you want to give them some direction as to what to do. Is there a future features list? Directions to the bug list? Would you prefer contributions that reflect translations of the code? (Did the project choose a reasonable strategy to allow folks to add new “language packs" of messages, etc. Did you publish the simple instructions for how to add a language?) Do you need better icons? Is there a list of how-to docs you would love to have? (Did you publish the instructions for templates or useable submissions?) If you don't have the software construction discipline to deliver the first three steps, you will be unable to deliver the next three steps or rather they won't work well and will consume more resource than the value they bring. If you don't get those steps in place, the last step becomes a waste of time. For me, looking back, if you don't get the simple developer discipline in place for steps 1-3, then the software project (FOSS, IT, academic, government or otherwise) is doomed to failure because it will rapidly degenerate into an unsustainable mess, completely incapable of supporting growth in the contributors and potential committers, and ultimately unable to sustain a growing number of users. It will always be "that toy/experiment in the corner" for which commercial products will always appear to do a better job. You see this level of construction discipline in successful FOSS projects. As the Apache project grew and evolved into the Apache Software Foundation, it has published a rigorous development methodology. So too with the evolution of the Eclipse Project into the Eclipse Foundation. The Linux kernel project would never have become what it is without the discipline at it’s core. This to me is the critical problem. There are lots of cool problems to solve on the planet. Without passion, however, you won't solve them. When you find someone with passion, even if that passion is only for getting their unique personal job done with the FOSS project in front of them, you have to make it so trivially easy to contribute that they're already hooked before they even realize it. Without developer discipline at the core, that on ramp won’t exist. The interested don't get to discover they're passionate. Culture comes from the top, even in a meritocracy.