You are here: Home Articles DCWorkflow's hidden gems
Navigation
OpenID Log in

 

DCWorkflow's hidden gems

by Martin Aspeli last modified Apr 08, 2009 06:34 AM

A few things you may not have known about the venerable portal_workflow tool

A while ago, Laurence, Alex and I were sat over a beer thinking about all the parts of Plone we'd like to rip out. Alex asked, "what part of Plone is actually really good". And the first thing that came to (Laurence's) mind was DCWorkflow, better known as the portal_workflow tool.

On my most recent project, I've started appreciating it even more. Our workflow system is actually really cool, and can do a bunch of things that you may not have used, but which may come in handy one day. In general, if you are trying to solve a problem that involves one or more of:

  • the state (or status) of a content object
  • per-object security
  • user-triggered events/check points
  • approvals, sign-off or handover

then you should look to solve those using the workflow system.

How well do you know DCWorkflow?

As a refresher, DCWorkflow is a states-and-transitions system. That means that your workflow starts in a particular state (the initial state) and then moves to other states via transitions (also called actions in CMF).

When the workflow enters a particular state (including the initial state), the workflow is given a chance to update permissions on the object. A workflow manages a number of permissions - typically the "core" CMF permissions like View, Modify portal content and so on - and will set those on the object at each state change. Note that this is an event-driven thing, not a real-time security check: only by changing the state is the security information updated. That's why you need to click "Update security settings" at the bottom of the portal_workflow screen in the ZMI when you change your workflows' security settings and want to update existing objects.

A state can also assign local roles to groups. This is akin to assigning roles to groups on Plone's Sharing tab, but the mapping of roles to groups happens on each state change, much like the mapping of roles to permissions. Thus, you can say that in the pending_secondary'state, members of the Secondary reviewers group has the Reviewer local role. This is powerful stuff when combined with the more usual role-to-permission mapping.

State changes result in a number of variables being recorded, such as the actor (the user that invoked the transition), the action (the id of the transition), the date and time and so on. The list of variables is dynamic, so each workflow can define any number of variables linked to TALES expressions that are invoked to calculate the current value at the point of transition. And of course, the workflow keeps track of the current state. The state is exposed as a special type of workflow variable called the state variable. Most workflows in Plone uses the name review_state as the state variable.

Workflow variables are recorded for each state change in the workflow history. This allows you to see when a transition occurred, who effected it, and what state the object was in before or after. In fact, the "current state" of the workflow is internally considered to be the most recent entry in the workflow history.

Workflow variables are also the basis for worklists. They are basically canned queries run against the current state of workflow variables. Plone's review portlet shows all current worklists from all installed workflows. This can be a bit slow, but it does meant that you can use a single portlet to display an amalgamated list of all items on all worklists that apply to the current user. Most Plone workflows have a single worklist that matches on the review_state variable, e.g. showing all items in the pending state.

If states are the static entities in the workflow system, transitions (actions) are dynamic. Each state defines zero or more possible exit transitions, and each transition defines exactly one target state. It is also possible to mark a transition as "stay in current state". This can be useful if you want to do something in reaction to a transition and record that the transition happened in the workflow history, but not change the state (or security) of the object.

Transitions are controlled by one or more guards. These can be permissions (the preferred approach), roles (mostly useful for the Owner role - in other cases it is normally better to use permissions) or TALES expressions. A transition is available if all its guard conditions pass. A transition with no guard conditions is available to everyone (including anonymous!).

Transitions are user-triggered by default, but may be automatic. An automatic transition triggers immediately following another transition provided its guard conditions pass. It will not necessarily trigger as soon as the guard condition becomes true (that would involve continually re-evaluating guards for all active workflows on all objects!), but you can explicitly ask the workflow sytem to check them and trigger them - see below.

When a transition is triggered, the IBeforeTransitionEvent and IAfterTransitionEvent events are triggered. These are low-level events from Products.DCWorkflow that can tell you a lot about the previous and current states. There is a higher level IActionSucceededEvent in Products.CMFCore that is more commonly used to react after a workflow action has completed.

In addition to the events, you can configure workflow scripts. These are either created through-the-web or (more commonly) as External Methods, and may be set to execute before a transition is complete (i.e. before the object enters the target state) or just after it has been completed (the object is in the new state). Note that if you are using event handlers, you'll need to check the event object to find out which transition was invoked, since the events are fired on all transitions. The per-transition scripts are only called for specific transitions.

Workflows are mapped to types via the portal_workflow tool. There is a default workflow, indicated by the string (Default). Some types have no workflow, which means that they hold no state information and typically inherit permissions from their parent. It is also possible for types to have multiple workflows. You can list multiple workflows by separating their names by commas. This is called a workflow chain.

Note that in Plone, the workflow chain of an object is looked up by multi-adapting the object and the workflow to the IWorkflowChain interface. The adapter factory should return a tuple of string workflow names (IWorkflowChain is a specialisation of IReadSequence, i.e. a tuple). The default obviously looks at the mappings in the portal_workflow tool, but it is possible to override the mapping, e.g. in resopnse to some marker interface.

Multiple workflows applied in a single chain always co-exist in time. Typically, you need each workflow in the chain to have different state variables. The standard portal_workflow API (in particular, doActionFor(), which is used to change the state of an object) also asumes the transition ids are unique. If you have two workflows in the chain and both currently have a submit action available, only the first workflow will be transitioned if you do portal_workflow.doActionFor(context, 'submit'). Plone will show all available transitions from all workflows in the current object's chain in the State drop-down, so you do not need to create any custom UI for this. However, note that Plone always assumes the state variable is called review_state (which is also the variable indexed in portal_catalog). Therefore, the states of any secondary workflow probably won't show up unless you build some custom UI.

In terms of security, remember that the role-to-permission (and group-to-local-role) mappings are event-driven and are effected after each transition. If you have two concurrent workflows that manage the same permissions, the settings from the last transition invoked will apply. If they manage different permissions (or there is a partial overlap) then only the permissions managed by the most-recently-invoked workflow will change, leaving the settings for other permissions untouched. See also the note about subtractive workflows below.

Multiple workflows can be very useful in case you have concurrent processes. For example, an object may be published, but require translation. You can track the review state in the main workflow and the translation state in another. If you index the state variable for the second workflow in the catalog (the state variable is always available on the indexable object wrapper so you only need to add an index with the appropriate name to portal_catalog) you can search for all objects pending translation, for example using a Collection. See also the note about reactive workflows below.

Some interesting things I've done lately

All that, you get for free. I've also been working on some additional products around workflow.

Workflows from spreadsheet

When you have complex workflows managing lots of permissions, it can be pretty difficult to get a good overview of what states and transitions you have, how they relate to one another, and what the permission settings are. This is especially true if you're working on the filesystem and making repeatable workflows to be installed via GenericSetup. Like most GenericSetup handlers, the workflow tool handler uses an XML syntax, and managing ten states with ten permissions and a mixture of roles in XML is not fun.

Enter collective.wtf. The name has something to do with how I was feeling at the time. It lets you write your workflows as spreadsheets (which you need to save to CSV format). It tries to remove all the boilerplate to get Plone-standard workflows and avoids as much repetition as possible. I use an OpenOffice template that provides nicely formatted workflows and then save to CSV for the final import.

collective.wtf also provides a number of debugging aids around workflows. For example, you can easily get a CSV view of an currently installed workflows to sanity-check permissions, and there is a view that runs some heuristics on your installed workflows to check against Plone conventions and best practice.

Subtractive workflows

I needed to model a situation where a content object could go through a number of states managing the 'View' permission. In some states, the object would ordinarily be published to anonymous, in other states to authenticated members. However, it also had to be possible to mark an object as "confidential". A confidential item should never, ever be shown to the Anonymous, Authenticated or Member roles, but would otherwise go through the usual workflow.

To avoid having to create a parallel workflow for confidential items, I created collective.subtractiveworkflow. With this workflow, the permissions you chose for a given state are taken away from the selected roles, rather than granted to them (group/local role mappings are unaffected). I use it as a secondary workflow. If the user selects 'mark confidential' from the state drop-down, the subtractive workflow moves to the 'confidential' state, where View is taken away from Anonymous, Authenticated and Member. In the default 'non-confidential' state, no roles are ticked and so no permissions are taken away.

Remember from above that when a workflow manages permissions, only the last state applies. For something like a subtractive workflow, that is a problem, because its very role is to moderate the permissions managed by the previous state. Therefore, collective.subtractiveworkflow installs an event handler that will correctly calculate the "merged" set of permissions across all states for any workflow event that takes place on a multi-workflow chain where at least one element in the chain is subtractive.

Reactive workflows

Remember also that automatic transitions are only checked and potentially invoked immediately following a state change. In fact, they are only invoked after a state change in the same workflow, on the same object.

Laurence Rowe wrote Products.ReactiveWorkflow, which I've stolen and rolled into collective.wtf. It contains two event handlers which are not registered by default, but can be registered in ZCML, or called directly e.g. from a workflow post-transition script, to:

  • invoke any available automatic transitions for all workflows on the current object
  • invoke any available automatic transitions on the parent object

With these two event handlers installed, you can make complex co-reactive workflows. You could have an automatic transition in one workflow with a guard that is "freed" by a specific action on the second workflow, or you could have a state change in a a child object effect a state change in its parent.

Here is an example: The system I'm working on has an Idea folderish type. When it enters the ready-for-review state, the Review type becomes addable (the workflow manages the add permission). The Review type can be added by any user with the Reviewer role. There are also users with the OfficialReviewer role. A review is added as draft, and can then be transitioned to the complete state by its creator. There is an automatic transition from complete to official protected by a permission only granted to an OfficialReviewer. Thus, when an official reviewer completes his or her review, the review object automatically enters the official state. In the parent idea, there is an automatic transition from ready-for-review to review-completed that is guarded on whether or not there is an official review in the object. With the reactive workflow event handlers, that triggers as soon as the child review object is transitioned.

To have a bit more control in the GUI, you can also use content rules to make these types of checks and trigger automatic or regular transitions in the current or parent object. To help with that, a new product collective.contentrules.parentchild provides a set of conditions and actions that you case.

Smart guards

In the example above, there was a condition "the object contains a Review object in the official review state". This looks like object/@@wf-utils/contains/Review/official. The @@wf-utils view contains a few handlers like this, using traversal logic to receive arguments. I haven't managed to release that as a separate product yet, but if there's demand, I'll do so (or post the code).

Detailed workflow steps portlet

Plone's workflow UI is pretty un-obtrusive, but sometimes you want to push the workflow steps in people's faces. For this project, we need to have a rich text description associated with each workflow state, including HTML and some dynamic elements (mostly links). The same goes for each state, which should be presented as an action link and associated rich text description.

We achieve that using collective.portlet.workflowsteps. This pulls text from the description of the current state and transitions. It allows HTML and simple variable interpolation to enter the current object's URL, the portal root URL and so on. We assign an instance of this portlet as a content-type portlet, making it available on the types where it makes sense without taking up space everywhere.

The True Review portlet

We also have another custom portlet: collective.portlet.truereview. Unlike the standard review portlet, it does not use worklists. Instead, it searches for objects (usually limited by type and review state) where the current user has the Review portal content permission. This means that the review list contains only items that the user can actually review, as opposed to all items in the pending state. Since we use local roles extensively to grant review rights to specific users and groups, this makes the review list a lot more useful. As a bonus, this product also lets you configure collections using "user can review" as a criteria, making it easier to set up custom review lists.

Baton permissions

Finally, not so much a product as a pattern. I call it "baton permission". I've used this in the past where review of an item passes between multiple roles, with one role at a time responsible for the review. In such workflows, all review checkpoint transitions are guarded by the Review portal content permission, and this permission is also managed by the workflow itself. In each state, the workflow controls who is allowed to review the object and transition it to the next state.

Documentation and releases

Hopefully, this post has been useful (if long!). I'd welcome some help from anyone who wants to help turn the generic sections about DCWorkflow into a more comprehensive tutorial. That'd probably require some screenshots, a bit more detail on the UI and GenericSetup formats, and a description of the standard roles, but a lot of the details are outlined above.

Also, the products I've described are released to PyPI. At some point, I will also upload them to plone.org. I'm just saying that because Alex Clark will almost certainly remind me. :-)

Document Actions

DCWorkflow on the bridge

Posted by http://www.openidfrance.fr/encolpe at Apr 08, 2009 08:18 AM
This is a very good article of the possibility of actual version of DCWorkflow.
The CSV import is very important improvement to help human reader to create new workflows. The product collective.workflowed is also a good AJAX component to manipulate existing workfows.

Alas, this product is so mature that there is mainly few improvements in it for the five past years. Additional features like workflow delegations were implemented in specific products instead to be reversed to the community. Some deprecated features like worklists are always present after years. It could be a very good Google Summer of Code project for this year or the next one.

DCWorkflow on the bridge

Posted by Martin Aspeli at Apr 08, 2009 09:56 AM
Enclope,

I don't think worklists are to be deprecated. They are very useful for letting the workflow itself broadcast what items require attention. There may be ways to make them more efficient, though, and it'd be nice to be able to use "current user" as a match variable, which would eliminate the need for the True Review portlet.

Otherwise, I don't really see why it's so crucial to change it. Just because it's old doesn't mean it's broken. Things like delegations can be modelled with the existing model, and it's pretty easy to make your own type of workflow if you have very specific requirements. collective.subtractiveworkflow was a half day's work, and the only thing that took time was to test it properly.

Martin

DCWorkflow on the bridge

Posted by http://www.openidfrance.fr/encolpe at Apr 08, 2009 11:19 AM
The only advantage of worklists are they give a direct access to the workflow variables values and use them to make a search in the portal_catalog. If you remove that point it just adds a dependance on CMFCore in DCWorkflow.

DCworkflow is the last component of CMF that is refactored when CMF is evolving. It uses a lot of Zope 2 magic and can be optimized. I don't want to change it, I'm just raising the fact that we can improve it.

DCWorkflow on the bridge

Posted by Martin Aspeli at Apr 08, 2009 08:31 PM
Worklists as a concept are important. The workflow author should be able to define the worklists. If we just do catalog searches for 'pending' or whatever, that makes a lot of assumptions about how the workflow is set up.

What Zope 2 magic would you refactor out?

Excellent Refresher

Posted by https://launchpad.net/~shane-hathawaymix at Apr 08, 2009 10:58 AM
I've been surprised at how well DCWorkflow has survived over the years. Its basic design has hardly changed since the initial release. Maybe that's because it successfully combines all of those related concepts in a relatively small package that's fairly easy to wrap your head around.

Thanks for the nice overview.

WorkflowField

Posted by http://david.wglick.org/ at Apr 08, 2009 12:14 PM
This is a great start on a new plone.org tutorial.

One workflow-related product you didn't mention that I recently used to good effect is Products.WorkflowField, which is an Archetypes field that lets you activate a workflow transition from an object's edit page.

Great Post on Plone's Workflow / DCWorkflow!

Posted by http://ctxlken.myopenid.com/ at Apr 08, 2009 02:05 PM
Martin,

Thanks so much for this informative post. I wasn't aware of some of the new workflow-related products in the collective and look forward to trying them out (David- thanks for pointing out the Archetypes Field type as well - that can be very handy to keep things simple for users.)

Easy Worfkow Development:
I'll have to try out your new CSV approach. I typically use the swim lanes / state diagram approach outlined in ye 'ol DCWorkflow HowTo: http://www.zope.org/[…]/DCWorkflow_doc.pdf

I've found that by creating the workflows via point-and-click in the ZMI and then exporting the XML out via GenericSetup (portal_setup tool in Plone site in ZMI), that it's really a quick and simple way to generate the proper XML code and to drop the exported workflows.xml and /workflows folder into my custom product.

But perhaps there would be some value in using the CSV approach, where the CSV serves as both a source for XML code generation and nice documentation matrix (of the swim lanes, states, roles, permission), but just not in UML form.

Multi-workflow Adaptation:
I didn't realize this was possible (mapping multiple, parallel workflows to a content type.)

It makes me wonder whether it's possible to 'pipeline' one workfow into another rather than having parallel workflows apply to the same object. For instance, if I have Workflow 1 with states A->B->C and have Workflow 2 with just steps A->B, could I have them both pipeline to Workflow 3 with states X->Y->Z? This would allow for more workflow reuse rather than duplication of common (X->Y->Z) states/transitions.

One Gotcha with Roles/Groups Guards:
I recently stubmled upon the fact that to list multiple roles or groups in the Guards on a transition, you must use the semicolon (;) as a delimiter. Seemed odd to me, but fine now that I know.

GSOC Proposal:
Probably too late for this, but to make Plone more competitive with other full-featured 'Enterprise' CMS tools (it already far surpasses other open source CMS tools with regard to workflow), it would be nice to leverage an Eclipse plug-in (or ArgoUML or Poseidon?) that allows for UML State/Activity Diagrams to be exported as the necessary XML code to be merged into a Zope/Plone product as a custom workflow, just as GenericSetup allows today.

Many commercial tools provide the ability to do this (via Eclipse or Visio) drag-drop states/diagrams, insert post/pre-transition script hooks, etc.

Thanks again for the post. I'm working with some students as part of a Plone product development product this Spring and will have them read your post. It's great!

-Ken

Great Post on Plone's Workflow / DCWorkflow!

Posted by Martin Aspeli at Apr 08, 2009 08:38 PM
Hi Ken,

Doing TTW-and-export makes a lot of sense. I still find it tricky to do that because there are so many screens to go around, and you can't copy the permission settings from one state to another wholesale. I often have multiple states that need the exact same permissions. collective.wtf will export to CSV as well as import, though.

It's possible to design for pipelining by effectively having one workflow "wait" with a guard on its outbound transition, or simply not have any outbound transitions from its last state at all. They are still "in parallel" but it probably doesn't matter. I'm not sure how you'd handle the review_state though. Maybe the last applied wins. I'd have to check.

ArchGenXML has got workflow generation from state diagrams. I haven't used it in a while, but it's quite a good solution. See also collective.workflowed which gives you an AJAXy, through-the-web way to draw diagrams for workflows.

Martin

workflow portlet

Posted by http://laurencerowe.myopenid.com/ at Apr 09, 2009 11:17 AM
One thing I've found useful in a workflow portlet is to show both allowed and blocked transitions (the first with the green tick icon, the other with the red cross icon). This lets you put information such as "the page must be given a category before it can be published" in the workflow description.

Chained workflow UI support

Posted by https://claytron.myopenid.com/ at Apr 11, 2009 02:36 PM
Martin, awesome write-up on DCWorkflow! We wrote a package that makes the chained workflows show up properly in the Plone UI called sixfeetup.workflow.chained. It has not been released to the public yet but your post has given me a bit of energy to get that released.

Basically it modifies the state dropdown and the folder contents to show the chained states to the user. It also adds a new class to them so you can style them as needed. We found this very necessary when implementing chained workflows since the 'chained' state would be invisible.

Hopefully I can find some time to sprint on the package to get it ready for release in the near future. I look forward to trying out some of the packages you mentioned in the post.
Sponsor message

Detailed reviews of seo companies which provide web design services for clients but also covering domain registration companies to save costs!

Plone Book
Professional Plone 4 Development

I am the author of a book called Professional Plone Development. You can read more about it here.

About this site

This Plone site is kindly hosted by: 

Six Feet Up