The Über-Buildout Mark II - Windows (IIS) and Unix (nginx), production and development
A multi-platform buildout for development and production deployment on Unix and Windows
Previously, I wrote about how to use a single buildout to manage the various aspects of a production server. That buildout, however, only works on Unix platforms. It builds nginx, varnish and haproxy, all of which are Unix-only (there is an nginx build for Windows, but it won't work as an xdv XSLT proxy). For my current project, I also need to deploy to Windows Server 2003. And I need to support both production and development environments. Enter the Über-Buildout Mark II.
Update: I've used this setup in a few projects now. To make it easier to find and collaborate on, I've added it to the collective at http://svn.plone.org/svn/collective/buildout/uber You can export the version you want (with svn export, not svn co) and use it as a starting point for your own build. Please also help keep the build up to date, and feel free to add new variants.
This buildout is spread across a number of files:
- build.cfg contains recipes describing how to build the various services.
- base.cfg configures all generic services, eggs, and so on. It extends the Plone 3.3 versions.cfg and uses the buildout.dumppickedversions extension to write a file with all the versions picked.
- devel.cfg extends base.cfg and limits the parts built to exclude most of the production services, setting up a ZEO client with debug mode on and various debugging tools installed. It uses mr.developer to track individual packages from an internal svn repository (as well as external ones). This is the day-to-day buildout on Linux or OSX.
- production.cfg extends base.cfg to set production host names, ports and users. It also depends on the dumped versions file generated by buildout.dumppickedversions. This means that the production server is always running the "last known good" configuration from the testing environment.
- windows-settings.cfg contains a few Windows-specific overrides, mainly to do with path separators.
- windows-devel.cfg extends devel.cfg, turning off some Windows-incompatible services like supervisor.
- windows-production.cfg extends production.cfg, replacing supervisor, nginx, haproxy and varnish with generated configuration files for IIS and Squid 3.
This project uses the xdv engine for theming. For development, we use dv.xdvserver, which applies the theme via a separately run server, configured in devel/server.ini. For production on Unix, we deploy a static XSLT file to nginx. On Windows, we use collective.xdv (not yet released - see collective svn for now) to apply the transform in-Plone. collective.xdv is used on Unix too, even if it's not actually applying the transform, since it provides a way to declare which style sheets are to be rendered through the theme, via condition expressions in portal_css.
The buildout also sets up CacheFu, LDAP and ZODB blob support on Plone 3.3rc2.
The Unix production configuration is similar to the aforementioned blog post:
- nginx serving port 80
- varnish web cache
- nginx (again) applying the xdv theme transform as an XSLT
- haproxy performing load balancing
- Four ZEO clients
- ZEO server
- Supervisor process control
On Windows, the setup is currently:
- IIS 6 serving port 80, using the ManagedFusion URLRewriter plugin to provide Apache-style URL rewrite rules. It also serves the /static directory containing CSS and image resources.
- Squid 3, installed from binaries and running as a service, acting as a cache server and load balancer.
- 4 ZEO clients with collective.xdv applying the theme when invoked via the canonical domain.
- ZEO server
- All servers are managed as Windows services.
The URLRewriter configuration and the Squid configuration file are both generated by the buildout, though the installation of Squid and configuration of the IIS site is a one-off activity that you have to do manually.
The various configuration files and a skeletal buildout may be downloaded here. It should work out of the box, and give you a pretty good idea about how to set all of this up for your own project. There is a brief README.txt with some more details.
On Windows, note in particular that you must start Squid using the generated squid.conf, and you must set up IIS according to the instructions, with the 'theme' directory as the web root and a wildcard mapping for the URLRewriter plugin. Don't forget to disable "verify file exists" when you set up the wildcard mapping, and note that you need Microsoft .NET 2.0SP1 or later installed on the server.
If you do end up using this, consider contributing a more in-depth tutorial on this stuff to plone.org. I don't have time to do it properly right now, but it would be a really good contribution.