Oh, how we love to hate Plone
Or, "how I learned to stop worrying and love the open source project"
Sometimes, people seem like the really hate Plone. Actually, I take some solace in their rants, because normally these are dedicated and passionate users who have been burned by some problem, or who just want Plone to be a little bit more. I guess we set high expectations, which is good - it motivates people to contribute to fill the gaps, and it improves our standing in the world.
However, it can be a bit difficult sometimes being on the receiving end of this criticism. Most of us who contribute to Plone do so out of … well, a lot of reasons, but love of the community is normally one of them, as is the desire to produce something great that we can be proud of. When we hear that people give up on Plone (I always find it really hard to talk to people who noisily and completely just give up, whatever the context) or slander it publicly, whether out of frustration, ignorance, malice or genuine concern, it’s easy to become defensive.
To a certain extent, this comes from something of an assumptions mis-match. The people “on the inside” see the community as very transparent and open-ended. And it is. The barriers to entry are very low if you have the attitude to want to constructively contribute something, be it code or documentation or co-ordination or money. To people “on the outside” or wider periphery of the community, it must sometimes seem as if Plone is run like a company with a boss and a board that you can sack and an impenetrable elite of developers who call all the shots behind closed doors.
Maybe it’s not so different, but it’s very hard to direct blame at a community of volunteers. If you tell me that Plone sucks because your four attempts at migrating from 2.0.5 to 3.0 went to hell, I would probably take it somewhat personally. Why did I spend hundreds of hours building things that made you want to upgrade if this is the thanks? Of course, it’s never like that, because the criticism is directed at Plone-the-community, who may or may not have failed at something, but the actual work is done by individuals who are all very much equals. Perhaps the release manager and the Framework Team (of which I’m a member currently) feels a bit more pressure, but really Plone is extremely “flat” socially.
There are a few sources of this kind of mis-understanding or distrust. Some of them we can probably usefully improve to make everyone’s lives easier. Others are more about educating the user community to understand what they are dealing with.
The first one, I think, is marketing. I think the Plone community, like most open source projects, is not terribly good at marketing. We are extremely lucky to have some very talented and energetic people who really do a tremendous job at something awfully difficult (Paul once told me, “If someone says they’re doing marketing for Plone, shoot them; you’ll be doing them a favour.”). The problem is that we don’t have (and don’t want) some “senior management” issuing decrees about the features that will be in Plone 4, codenamed Longtooth that we can put out a press release about. Why not? Because you can’t tell volunteers to work overtime (although they often do) to finish off features that were long promised (more on that in a bit). That makes the job of a marketeer difficult. It’s equally difficult to know who to market to, how to get the word out there, and which word to put out. Just because a particular community member puts out a particular idea, that doesn’t mean it will happen. That takes a particular process and a good dose of reality in terms of what people can achieve.
So following on from that is process. Outsiders always have a tendency to want to add more formalism to Plone’s process, but that kind of misses the point. Some formalism is needed, if it enables people to work on Plone more effectively, for example by avoiding duplication of effort. When it starts restricting them, however, it’s bad. Most people have a day job that gives them plenty of management overheads, and when they go home and want to hack on Plone, they just want to be productive. Surprisingly, this doesn’t actually mean we all pull in different directions. The transparency, friendship and mutual respect that exists in the community means there are incredibly few real disputes, and if they arise, they are normally resolved to mutual satisfaction. We tend to debate until we mostly agree and let pragmatism do the rest. This depends greatly on Plone development being decentralised, which in turn stimulates innovation.
We do have a process, however. We choose a release manager (who is paid a nominal fee by the Plone Foundation - this is pretty much the Foundation’s only involvement in the actual development process, which is very much by design), and a Framework Team. Contributors produce “bundles” that show off a particular feature with some initial code, and the FWT votes on which ones to recommend to the release manager. The release manager sets a timetable, and helps people get their code in on time (or rejects code if need be). This process is documented through the roadmap which lists PLIPs (Plone Improvement Proposals) that are proposed, begun or merged into the main codebase. Probably anything much more formalised than this would just get in the way.
The most common criticism against this approach is “where is the big picture?” or, rather, “where will Plone be two versions from now?”. The answer is probably that it doesn’t matter. Unless those people who are writing Plone actually want to work on those features, they ain’t gonna happen. It is the discourse of the community, combined with people’s customers and our users’ feedback, which generate enthusiasm and excitement for new features. Sometimes new features simply happen because someone thought it’d be cool to write it. The release manager’s job is to balance that against being able to release something on time(ish), without undue risk that something may not be maintained in the future. Plone core is actually quite conservative: we often ask people to release things as add-ons rather than lobby to push them into the core, if they can work equally well that way.
Related to the process is the timetable. Now is a difficult time for people who are wondering whether they should start a project on Plone 2.5 or 3.0. Will 3.0 be releaesd on time? No. We have already shifted the timetable a few times. But that’s not a bad thing. Could I start development against 3.0alpha2? Maybe, but only if you actually want to be involved in the process of getting Plone 3.0 finished, and even then we won’t swear to anything. Again, we don’t have the CEO shouting from the top. We set some goals because they help us manage our task. We roughly aim for one release every six months, though in my own observations, every nine months is more realistic given the time it takes to physically get a release out, get people into the habit of fixing bugs and loose ends, and a certain degree of “release fatigue” which kicks in after that fairly stressful and monumental task it is to polish off a release we’d like to call “dot-zero”. One thing we have learned recently, is the importance of strategically timed sprints (such as the Archipelago Sprint a year ago, the Baarn sprint next weekend and the Sorrento Sprint at the end of February), and putting our timetable around them. But the bottom line is, unless you are interested in helping to push the release forward, you need to accept that it’s only ready when it’s ready. The people who are making it ready probably care as much as you about getting it out sooner rather than later.
Another reason people get frustrated is communication. Sometimes realities change. Sometimes, there is no clear answer to some architectural or practical question, just yet. The discourse of the community is rich and complex - as it has to be. Consensus tends to emerge among the contributors. However, it’s not reasonable to expect users to have time to read the mailing lists and lurk in the chatroom all the time. The most public-visible things we have are the aforementioned release roadmap and PLIPs, which do a pretty good job of describing features. We also have an announcement mailing list and news items on plone.org, we could use these better. It’s just that sometimes it’s hard to know which announcements are important to make.
For example, I recently learned that a respected peripheral contributor and investor in the community had long been under the misconception that Plone 3.5 would run on Zope 3, ditching Zope 2 entirely. He even heard this directly from someone fairly “high up”, a while ago. This is a particularly good example, because the relationship between Zope 2 and Zope 3 is fairly nebulous. To be clear: Plone will run on Zope 2 and the CMF, as it always has, for a very, very long time. At the same time, Zope 3 (which is not a direct successor to Zope 2, but rather a collection of small and loosely associated components built from a clean slate based on the experiences and mistakes from Zope 2) is being used as a library to enhance Zope 2 itself as well as Plone. Almost all the new work that is going into 3.0 has a heavy focus on using Zope 3 concepts and technologies, and this has boosted our productivity immensely, but that is in addition to, not instead of, the Zope 2/CMF stack. In other words, we don’t want to rewrite Plone from scratch and ditch compatability with every existing third party component and installation. Breathe out.
The challenge is that this relationship is mostly clear to the Plone developer community. A lot of the details were in fact only worked out over the past year or so. But if it’s clear to “us” and not clear (to us) that it’s unclear to the “rest of you”, then we may not even realise we need to communicate something more clearly.
This particular issue also impacts concerns about migration and integration. Plone is a big and complex system. It performs a lot of functionality “out-of-the-box”. People then customise this, install add-on components and extend Plone to meet their needs. This is exactly how it should be used, but because Plone is such an open-ended and flexible system, it’s incredibly hard to ensure that we never, ever break anything during a major version upgrade. In fact, we always do. Sometimes, it’s necessary to enable new features. Sometimes, 100% backwards compatability would be too high a burden and stifle innovation. We do take migration and general integration very, very seriously, because almost all of us have customers we would eventually need to migrate ourselves. However, this is one area where customers may well have to pay if they have non-standard installations and are unable to deal with problems themselves.
Similarly, not all third party products are created equal. Some are outright crap. But politically and practically, policing this is very hard. We (especially George Lee and Alex Clark - thank you!) are working on improving the Plone Software Center (which runs the products section on plone.org) to make it easier to see which products are “good” and which ones need attention. There is a project underway to look at this in the wider context of Plone, but again, this is difficult, because it depends on volunteers to do something which is a bit nebulous - defining processes and writing clear guidelines.
Related to this are critcisims about the complexity of our software stack. Yes, Plone is big and builds on a complex stack, parts of which date back almost a decade. Frustration at the complexity of any new system is fairly common. In Plone’s case, this is doubly so because Plone is quite advanced when it comes to certain types of functionality. We have solved complex problems with the tools we have available. Sorry - it’s never going to be as easy to understand as PHP, because that’s apples to oranges. Regardless, people could take some comfort in the fact that thousands of developers have managed to get productive (sometimes very productive) by learning Plone. Clearly, it’s not impossible. My own experience when I started learning Plone about three years ago was that it fit my brain quite well and I took it fairly quickly, but at the same time there is still detail I don’t know about. The most important thing here is to learn the core principles and learn how to help yourself discover what you need to know to achieve a particular task. You don’t have to understand the inner workings of the ZPublisher to be able to usefully build content types with Archetypes.
Which leads us to everyone’s favourite sticking point: documentation. Plone’s documentation is in part pretty good and in part pretty poor. It’s sometimes hard for users to find out which documentation is good and which is bad, though. Plone is not like a programming language where you can measure documentation quality by coverage of the API. People do thousands of different things with Plone. We actively encourage people to contribute documentation when they have found a way to solve a particular problem that wasn’t documented before. However, we can’t always ensure that everyone contributes to the same quality, or that they will keep the documentation up-to-date perpetually. There is a review cycle, but realistically, making 100% perfect documentation for Plone would require as many and as dedicated people again as the set of people who write most of the code for Plone. And worse, whereas fluctuations in quality of code can be mitigated against and hidden (most of the time) from the user, the same is not true in terms of quality documentation.
Again, plans are in motion to improve this situation. Central to the plans is the creation of “trails” of documentation - core skills and techniques for various audiences brought together into a set of documentation pieces that are more easily accessible, with “second-tier” documentation allowed to grow more organically as people contribute short pieces on specific things. This would hopefully also allow the documentation team to manage the quality and relevance of a subset of the documentation more tightly, breaking it down from an impossible task (go to the how-to section of plone.org and count the number of items there - a blessing and a curse) into one that is merely challenging. And again, we need more volunteers, more dedication and more time.
So what can you do? Educating our users and peripheral contributors is an important task. If you are on the outside peering in, here are a few tips:
Get involved! If you have a question, even a stupid one, ask someone in the community via the mailing lists or chatroom. It’s not hard, and if you ask intelligent questions, you will get friendly answers.
Don’t give up! We want you to use Plone, honestly. People will normally try to help if asked nicely.
Learn to help yourself! Invest time in understanding Plone, both the community and the software it’s produced. You don’t have to be an expert to be able to ask intelligent questions or deduce something from the documentation.
Read a book! Paid authors with professional editors will almost always produce better “introductory” documentation than volunteers writing a quick how-to or tutorial. Some people learn best from books, others by exploration. Find out what works for you, and expect that everything won’t just be obvious from the first glance.
Pay someone! If Plone is being used in a serious capacity in your organisation, then you should have a budget for extensions, maintenance, migrations and/or support. Deciding to use Plone should be a business decision based on some kind of cost-and-benefit analysis. We hope you find it more beneficial than costly, but it may not be the right tool in every single situation. If you ask on the mailing lists whether people think Plone would be suitable for a particular type of problem, you will get an honest answer. And again, take some comfort in the fact that a large number of people feed their families delivering Plone-basd software, so clearly it’s not always a dead end.
Give constructive criticism! If you feel something is wrong, then let us know - preferably with some workable way of making things better. We don’t like unhappy users more than you like being one.
Now, if only I’d spent half as much time fixing Plone 3 bugs as writing this, that may have deflected some future criticism from Plone 3. Back to work!