Collaborator8.2-Blog-CTA-Demo

How to Turn Your Pile of Code into an Open Source Project

open source project

You’ve written some code, you think it would be useful to the world, and you’d like to give back to the open source world. But how do you do it? An experienced open source guy provides a checksheet for developers to release an open source project and get it noticed.

You can’t just dump stuff on GitHub.

I assume that as part of your motivation to release your code, you want people to:

  • Use your project
  • Get things done with it
  • Enjoy using it
  • Join your project and contribute
  • Find out about it so they can do the first four things

These things don’t just happen on their own. It takes foundational work. If you don’t care about these five items, then dumping your code in a GitHub repository does just fine. Otherwise, read on.

Be proud of your work

Before you take any action, you must have the mindset that your project is useful, high quality, and relevant enough for at least some people to be interested in using or contributing to. If you don’t have that internal sense of pride, the rest of the steps you take will be half-measures.

Back on Sesame Street in the 70′s, there was a song called “Sing” that applies. Part of the song goes:

“Don’t worry if it’s not good enough
For anyone else to hear
Just sing”

So, too, it is with your project. Don’t compare your coding skills to anyone else’s. Don’t compare your project to others. Don’t think that your project has to be better than any other, whatever “better” may mean. Don’t imagine what negative things people might have to say, because after all, this is the Internet and people would say negative things about free money falling from the sky.

If your project is good, and you like it, and you take pride in it, put it out there to help benefit the rest of the world.

Create a Project Summary Paragraph

You’re going to be talking about your project a lot, so write a project summary paragraph that you can cut and paste where you need it. You’re going to use that paragraph a lot, because you never want to assume people know what your project is based on its name alone.

Your project summary paragraph might look like this:

Lobot is a CSS debugger and optimizer that finds duplicated CSS tags, invalid syntax, and suggests optimizations. Lobot runs entirely as a Firefox plug-in under Firefox 18 and higher. It works on Linux, Windows, and Mac OS X.

or

Rancor is a Perl module that simplifies the aggregating of data from multiple NoSQL databases, including CouchDB and MongoDB. It runs on Perl 5.12 and higher, and requires Moose.

or

ack is a code-searching tool, similar to grep but optimized for programmers. It runs in pure Perl, is highly portable, and runs on any platform that runs Perl.

The summary paragraph should explain what problem it solves, for whom, and what the user needs to take advantage of it. This covers the high-level information that people who have never heard of your project need to know.

Stick that text in an Evernote note or some place where it’s easily accessible. Revise it as necessary over time.

Choose a site to host your project

Your code needs a publicly visible repository in which to reside, and project hosting sites like GitHub do it for you for free.

GitHub has become the default site on which to host your open source project these days, but it’s not the only option. GitHub certainly makes it drop-dead simple to create a project and a repository, and I have nothing but love for its service, but that doesn’t necessarily mean it’s the one your project should use.

Other options for hosting open source projects include:

These days, unless you have a reason to host with another version control system, go with Git; you have other choices to make about your project host. The VCS is the least interesting part of the selection process compared with figuring out which services fit your project best.

Probably the most important feature of a code hosting service is its issue tracker. For one thing, it’s the part of the site that you as project leader use the most. Think of all the annoyances of that you’ve had with issue tracking systems, both in open source and at your day job. Does the hosting service share these problems?

Is there an integrated wiki? I’m a big fan of wikis, but not all wikis are the same. Google Code’s wiki feature has a weird syntax and only users with commit rights to the project can make edits. For my projects, I wanted a small number of committers, but many wiki editors.

What do the project home pages look like? If you’re not going to give your project a dedicated website and URL (more on that later), then consider closely what your users (and potential users!) would see every time they visit your project.

Does the site give you a place to publish releases for download? Not all do. If your service doesn’t, you need somewhere else to publish releases.

Once you’ve chosen a site and created a project account, go ahead and import the source code to the version. That’s the part you knew you had to do anyway.

Create an issue tracker

Once you create your project on a hosting site and you have an issue tracker at your disposal, populate it with all of your existing issues for the code. This includes bugs, but may also include any future tasks. I recommend putting both of them in the issue tracker together, because they are often so closely related, and it gives volunteers for your project twice the opportunities to choose from to help.

Spend time creating high quality tickets. These tickets are a cultural example for future contributors to follow, so be on your best ticket-writing behavior. Be clear and explicit so that anyone else reading the ticket understands.

When you create issues for future work to be done, try to create granular tickets with explicit tasks or deliverables that new volunteers can work on. It’s not very useful to have a ticket that says:

#34: Increase test coverage. Our test coverage sucks. Write more tests.

because a new volunteer isn’t going to know where to go. You know the old joke about “How do you eat an elephant?” “One bite at a time” That ticket above is basically “Eat an elephant.” You need to chop it up into bites, like so:

#34: Add tests to the report generator

We need more tests in the report generator, especially because of the problems we’ve had with pagination in the past (see tickets #14 and #20). Look at `tests/printing/setup.t` for a starting point.

Then, create tickets #35-39 that include other bites that volunteers can work on.

Start a change log

If you haven’t done so, start a change log that tracks all the changes in your package over time. The change log should be kept in your source code repository. As you work on the project, document every change that could affect the user of the package.

Always remember who your change log is for. It’s primarily for someone who’s already used your project; that user wants to see what is in the next release and whether they should upgrade.

A change log is broken into sections for each version of the package that you released, or that you’re going to release. Every version should have at least these three headings:

  • Incompatibilities with previous versions
  • New features
  • Bug fixes

Everything in the change log should be explained in clear prose English for the benefit of the user. This specifically means that a dump of commit messages is not a change log.

If a bug was fixed, include a reference to the bug in the change log. If the contribution came from someone else, thank the person explicitly in the change log. An example:

Fixed ticket #17: Alerts are no longer generated if the user’s frammitz is zonked. Thanks to Chris Jones for the fix.

Create a mailing list

Your users want a community in which to ask questions and to make suggestions. Give them a central place to meet with other users.

The simplest way to do this is to go to Google Groups and create a mailing list. All the mailing list features you are likely to need are included, and choosing Google also means your mailing list archives are immediately findable in a Google search. This helps future users who look for information about your project without even knowing that a mailing list exists.

You may want to create two mailing lists, such as lobot-users and lobot-dev. If you don’t have a lot of interest in the project at the start, I suggest having one mailing list (lobot) and, if necessary in the future, create a second one (lobot-dev).

Whatever name you choose, if the mailing list software lets you create a long description of your project, paste in your Project Summary Paragraph. (It would sure get tiring typing in the same stuff about your project over and over, so aren’t you glad I said to create it once and keep it handy?)

Lately I’ve seen comments from people who disdain mailing lists in favor of thread-based discussions in a forum or the bug tracking system. In my opinion, those are too cumbersome, and they require users to monitor something on the Web rather than their email inbox.

Create a README

At the root of every project, you need a file called README. This plain text file should include at least these bits of information:

  • Your project summary paragraph
  • The author’s name and email address
  • The URL of the project website, if there is one
  • The URL of the project hosting page (GitHub, etc.)
  • Where to find the project bug/issue tracker
  • Where to find the project mailing list

GitHub specifically displays your README on the project home page, but it also is bundled up in all your releases, so it’s good to have for people to look at even if they’re not online.

Create a release

Users require releases of your software. It’s a disservice to your users to point at the Git repo and say “Just pull from the master branch and install it.” Users don’t want to have to use version control just to get a release of the code. Create a proper tarball (.tar.gz) that is easily downloadable by anyone.

Announce each release. Your announcements should not assume that the reader is familiar with your project.

Every release you create includes your Project Summary Paragraph at the top, before you say anything about what is in the release. Don’t assume that someone reading your release announcement knows what the project is.

Give credit to your contributors by name in your announcements, as well as in your change log (see above). The GCC project repeatedly violates this idea in their announcements. They claim that the contributors are “too numerous to mention.” Really? How many is too numerous? Do you think you’re going to run out of room in your email? What “too numerous to mention” really means is “I didn’t feel it was important enough to take the time to create a list of names” and implies, “All the time you put into this isn’t valued.”

How long do you think it might take to create a list of people who contributed to your project? Half an hour? An hour? Open source projects run on personal motivation and gratitude. They are scarce resources. Spending an hour to create a list of all your contributors so that they are publicly thanked is one of the best hours you can possibly spend on the project. When people feel appreciated, they are motivated to contribute more.

Create some documentation

I know, documentation is no fun for most developers, but every project needs it. A few dozen lines of an introduction of how to use the project is better than nothing. You may want to write it in the form of an FAQ, where you list any questions you were asked during development.

You don’t have to write a full manual the first time out, but you do have to give people something to start with.

Writing documentation is a big topic, big enough for an article of its own. Rather than go into more detail I refer you to my article, 13 Things People Hate About Your Open Source Docs.

Create a website

If there’s one gripe I have with GitHub’s home pages (and it applies to other project hosting sites, too) it’s that their project home pages focused entirely on the code and are not at all inviting to new users who are exploring a project.

Your GitHub repo home page is not a website. It is a front end to the code, but many of your users don’t care about the code. Your site doesn’t have to be fancy, but it does need to be aimed at new and potential users of your project.

For example, here is the ack project page on GitHub:

github-homepage

and here is the ack website at http://beyondgrep.com/.

beyond-grep-homepage

I also strongly suggest that you get a dedicated domain for your project, even if all you do with it is redirect to your project hosting site. That gives your project a permanent URL so that if you change project hosts, the base URL still works. Domain names are $10 per year, and it’s hard to beat the kind of value in this case.

Creating a project website can be a big task, and strictly speaking you could skip it and still have a successful project, but I think it goes a long way to helping make your project attractive to newcomers. You may want to push this off until after your project’s initial release to the world.

Announce it

Wait until you have things reasonably started before you do this. Attention is the most scarce resource, and people will only pay attention to your announcement once. If most of what you have to say is, “Mailing list coming soon, bug tracker coming soon, etc.” then people won’t take you seriously and won’t give you another look.

On the other hand, don’t wait to announce it until you’ve made everything perfect, because you will never have everything perfect.

Your announcement doesn’t have to be fancy or verbose. Just let people know that your project exists and point them to your project’s URL. It can be as simple as this:

Announcing the initial release of Lobot, a CSS debugger.

Lobot is a CSS debugger and optimizer that finds duplicated CSS tags, invalid syntax, and suggests optimizations. Lobot runs entirely as a Firefox plug-in under Firefox 18 and higher. It works on Linux, Windows and Mac OS X.

Lobot is available at https://lobot.example.com/

For anyone doing any amount of CSS work, I hope Lobot makes your work easier. Please try it out and let me know what you think at lobot@example.com.

Post that all over. Post it to your blog. Tweet a link to the blog post on your Twitter feed. Post it to Facebook or Google+ or whatever social media you live on. If your local user group has a mailing list, post it there. Post it to your project mailing list as its first post for its archive.

There are potential users everywhere and you don’t know where they are.

Congratulations

Congratulations. You’ve created a new open source project to help benefit others, and with any luck, to help improve the project for your own use as well.

See also:

subscribe-2

Comments

  1. @Andy, I really enjoyed reading this. Such great information to consider. Writing docs is a pain especially when your project is still in the beginning stages. I am currently working on a project and the docs is small but there is info there. However, I do plan on writing an extensive manual. I am going to check out the other post about open source docs, but do you have any other suggestions about writing a manual as well as distributing it besides in web form?

    • Andy Lester says:

      I’m no documentation expert, but I can tell you what I know.

      * Any documentation is better than nothing. When you’re starting on docs, just braindump everything you can. Don’t edit, at least at first. Just get out everything you can. You can edit and polish later.

      * Write a FAQ. People look for the FAQ first, and it’s an easy format to write. Anyone ever has trouble with the software, add a question to the FAQ. Anyone mails you about something, you add it to the FAQ.

      * Reread it every so often. Remove stuff that doesn’t apply any more, or update sections that have had new items added. Rewrite what you need to. Just as good code can go bad over time and needs refactoring, so too it is with documentation.

      * Don’t be afraid to repeat yourself. The same sentence or paragraph might belong in the introductory docs, and the FAQ, and the main body of the manual. Don’t make finding important information dependent on the user having picked the right part of the manual to read.

      Not sure exactly what you’re asking about distributing. I suggest writing in the simplest format possible. If you can use Markdown, do it, because then that’s easily translatable to HTML, or to pretty much any other format. Write once, format many. Is that what you were meaning?

      Not sure if that part about distributing is what you were after.

      • Thanks for the response. I had one other question. I keep going back and forth on whether or not to create a dedicated project page as you’ve mentioned or just redirect people to the main site. Are there any pros or cons in going with one option rather than the other? What are your thoughts on that? Thanks.

    • I have been trying to write the documentation first, then code. I don’t write the whole documentation then code the whole project, I just write small chunks at a time. It also helps me be clear about what I am coding, because I can read back about my intention. At the end I have both working code and documentaation completed at the same time. Which is awesome. I dont code anything without first writing down what I am intending to do. I think it is a really good habit to get into. We have to be able to describe our work as much as we can make it work. That makes a good coder.

      • Interesting concept. Should I assume you add screenshots afterwards as well? Thanks for sharing.

        • Andy Lester says:

          If screenshots make sense, then by all means add them. Screenshots are invaluable in letting the user see what the project does without having to install a thing. See the other article on documentation linked to in the article for more on that.

  2. To me documentation comes in two kinds, manuals and references. References can be read out of order. (Like an encyclopedia) A manual is to be read in order. (Like a recipe) Too many times I have seen open source manuals look like reference books. Manuals should describe a process on how to get from A to B. Like how to set the clock on your microwave oven. Manuals are extremly helpful for users. References are extremely helpful to developers. Make sure you know the difference and where possible write both manual and reference documentation.

  3. It would be great to have a comment about choosing a licence, companies can not (/should not) use code that does not have an explicit licence.

    It makes it far easier for a company to use code if it is a standard licence.

    • Martin Iturbide says:

      I also share your thoughts. But It is also useful for single developers too, so they know under which conditions they can contribute to the project. Also users cal also know under which conditions are they using the software.

      The license has to be part of the “readme”.

    • Andy Lester says:

      You’re absolutely right, and that’s a glaring omission on my part. It never crossed my mind as I wrote the article because I never go through the process of choosing a license, because I always use Artistic License 2.0.

  4. Ajoy Bhatia says:

    Andy specifically wrote that a dump of commit messages is not a change log. The change log should be in clear prose English for the benefit of the user. It is so that a user may understand what the changes/new features/bug fixes are in the release, and decide to upgrade or not.

  5. Please let me know if you’re looking for a author for
    your site. You have some really good articles and I believe I
    would be a good asset. If you ever want to take some of
    the load off, I’d absolutely love to write some material for your blog in exchange for a link back to mine.
    Please send me an email if interested. Kudos!

  6. Thanks a lot for sharing this. This will help me a lot in turning my pile of code in to open source project.

Trackbacks

  1. […] Going from home-grown code to an Open Source project. […]

  2. […] Slashdot recently highlighted an article from Andy Lester about how to start an open source project. The advice was targeted toward the developer who wants to get other people to actually use their project. I read over it and wanted to hold onto a link to the article’s advice, which is here: How to Turn Your Pile of Code into an Open Source Project […]

Speak Your Mind

*