On Plone documentation
A few historical lessons
I used to be quite involved in Plone documentation, at one point heading up the documentation team. Unfortunately, my ability to do that job was one of the early casualties of a more demanding career, and, apart from my book and a few contributed pieces of documentation, I've not been very active in the world of Plone documentation.
Recently, in particular after the Plone Conference last month, we've seen an explosion on the Plone documentation team mailing list. I've only read some of the posts - it's really far too much to keep up with - but I'm getting the feeling of history repeating itself, so I thought I'd bring up some of the lessons we learned the last time we had this round of discussions.
First of all, I'm very grateful to Mikko Othamaa, who summarised the discussion here.
Providing quality documentation is one of those problems that most open source projects find very hard to solve. Most people are not great writers, and many who would contribute documentation don't have the necessary experience or in-depth knowledge to be effective authors.
Writing documentation is actually harder than writing code. It's usually OK for code to evolve over time, and variations and inconsistencies can mostly be tolerated. Documentation by its very nature has to be consistent, concise and clear, and speak to the right audience.
In Plone, we have, in my opinion, a four main documentation challenges:
- The current corpus of documentation is vast and has grown organically. It is very hard to get an overview of it all, both as a reader and as an editor.
- Some documentation is out of date or even just plain wrong. This often causes more harm than the absence of documentation.
- Some documentation is simply lacking.
- Even where there is documentation available, there is often not a clear path that outlines what a new reader should read, and in which order he should read it.
I also think we have some challenges with the current documentation team itself. This is no way a criticism directed at those hard-working individuals who care about Plone's documentation. It is, however, important that we think about the systemic problems and try to address them.
Firstly, the documentation team lacks some structure. This is being addressed, by identifying owners of various areas, but I get the feeling that we frequently see short bursts of impassioned discussion, followed by long periods of not very much happening. Forgive my cynicism here, but save for a few great efforts (notably the introduction of the original Plone Help Center, and the documentation sprint that resulted in a great amount of re-organisation), I worry that more is being written about the documentation effort itself, than is being written as actual documentation.
Secondly, there seems to be an emphasis on technology solutions. I don't think it greatly matters whether we have a wiki or not, or whether we are able to re-use paragraphs across documents, or whether we have a fancy workflow. Any solution, no matter how open or closed, will fail if it is not continually maintained by people who make a sustained commitment.
Thirdly, there needs to be an injection of realism. A lot of grand plans are hatched, but the truth is that there is not a lot of available volunteer time for documentation. The current debate was in part borne out of frustration that the editorial team could not keep up.
TYpes of documentation
I think it's a mistake to try and solve all the problems with "Plone documentation" in the same way. At a very high level, let's consider the following types of documentations:
- End-user and administrator documentation: How do I get started? How do I set up a server? What's the right way to configure caching?
- Customisation and add-on development documentation: What is Plone's API? Where are the extension points I should use? What is current good practice?
- Detailed developer documentation: How do I contribute to Plone? What are the current focus areas and roadmap? How does a particular component work?
These probably need to be addressed in different ways.
User-focused documentation requires the most editorial control. Here, it pays to have a corpus small enough that it can be effectively reviewed and updated. It is also vital to have clear lines of responsibility over different sections. This documentation should be highlighted and easy to find - if readers stumble across something, this is where they should be stumbling.
Customisation/add-on developer documentation and more ad-hoc "how-to" style documents are by their nature less structured. The current Plone Help Center is largely based around this type of documentation: Anyone can contribute, but there is a light review process (which may be failing, either because things are not reviewed frequently enough, or because those who review demand perfection). Barriers to entry should be low, but there needs to be some degree of "self certification" so that readers understand what audience and version the document pertains to. For this type of documentation, we rely more on technology and less on editorial control.
Finally, detailed developer documentation belongs with the code it describes. The Zope 3 people figured out early on that to get developers to write documentation, you have to trick them: hence the reliance on doc tests and interfaces with detailed doc strings. This documentation is not produced by a dedicated documentation team, but is ensured through a culture of writing properly documented code.
I have a feeling of deja-vu as I write this, but here is my take on what we should do:
1. Produce a "Plone manual"
A while back, we called this "trails", and the idea was to highlight the "good" bits of documentation by tying them together into step-by-step guides aimed at different audiences and covering various broadly defined topics. For reasons I never understood, there was a lot of resistance to this idea. However, no matter which approach you take, we need to do this, and we need to re-use (even if just by copying and pasting) what's out there already, since we can't possibly write everything from scratch.
This documentation will have a few important facets:
- Each section of the manual needs an explicit owner, who is responsible for regularly reviewing and updating this text.
- It needs to be reviewed at least each time we make a new Plone release.
- We should have some versioning, so that people on older versions can use the older manual.
- The manual can only be updated via a proper editoral review process.
2. Open the flood-gates for "how-to" style documentation.
If someone wants to contribute documentation, by all means let them! If we have the type of editorially controlled manual outlined above, then we can afford to have a large number of "how-to" documents, even if they are not all to the same level of quality.
- Keep this documentation separate, so that it does not confuse users who really should be looking at the manual.
- Publish documents quickly (maybe immediately) so that contributors get the satisfaction of seeing their documentation help others.
- Keep an eye on what is contributed to ensure it is not giving bad advice.
- Rely on topic, version and audience keywords to automatically organise the documentation and make it searchable.
3. Enforce and expose developer documentation
We're getting better at separating our code into well-defined modules, and demanding documentation and tests for those modules. This documentation is great if you know what you're looking for, and generally we do not need to review it in as much detail. Where code is part of Plone core, provision of adequate documentation should be part of the Framework Team's review process.
Most developers are pretty happy to find this documentation in the code. However, we could and should use Sphinx to publish it on the web as well.
4. Allow feedback
There is enough documentation, and enough users that we can tap into the "folksonomy". At the very least, allow comments and offer ratings of the type "was this helpful to you". Use that to influence what documentation is most easy to find.
5. Use the tools we already have
It turns out we already have most of the tools we need to do this. A lot of the discussion recently has been around the use of wikis. I'm not against wikis per se, but I'll note that before we have the PHC, Plone had a ZWiki instance as documentation. The result was rather messy!
Wikis only really work for documentation if there are people actively gardening them. It seems to me that we need the categorisation and management tools offered by the PHC to effectively organise the documentation that's there. Furthermore, I suspect that those arguing for a wiki are more concerned about how easy it is to add an article, than how easy it is to edit one that you didn't write.
My suggestion, bearing in mind what we already have, would be:
- Use the "Reference Manual" section of the PHC to store the "Plone Manual", probably maintained as a series of Reference Manual structures organised by topic.
- Tweak the PHC listing pages so that they highlight the manual, and group documentation by audience, rather than by type.
- Use a post-hoc review process for contributed how-to documentation: Documents are first published as "public draft". The documentation team's review list shows these, allowing the team to either publish documents as "reviewed", or reject them on grounds of factual inaccuracy. This would eliminate the current bottleneck of reviews.
- Accept that how-to documentation won't be perfect. The review process for such documentation should be about rooting out serious errors or non-serious documentation, not demanding that every contributor is an eloquent Plone guru.
- Add ratings and improve commenting. Simple ratings of the type "was this helpful?" and maybe "this is out of date", coupled with a few reports, should help the documentation team identify and improve documentation that's doing more harm than good. For this to work, we need a way of alerting the documentation team when new comments are added, or when documents drop below a certain rating.
- We have a Trac category for documentation requests. We should allow people to make requests for new documentation, and expose the wish list in a way that lets people vote for their favourite requests. There are Trac plugins to do this already. The list of documentation in demand should be publicised, to encourage new contributions.
Apart from the lack of ratings in the PHC, everything else described above should be imminently possible. I'm sure others will have other ideas, of course. Ultimately I don't think it matters, though: unless we focus on low hanging fruit and on the documentation teams' processes, we'll probably end up going around in circles quite a few more times. Good for my book sales, perhaps, but not so good for Plone. :)