You are here: Home Articles Open Source-as-a-Service
Navigation
OpenID Log in

 

Open Source-as-a-Service

by Martin Aspeli last modified Apr 28, 2009 12:15 AM

Google App Engine is the future of computing one day, perhaps. In the meantime, it's a lot of fun.

This past long weekend (a holiday here in Western Australia), I played a little with Google App Engine. Unless you've been living under a rock, you should know that this is a way to deploy your own applications to Google's infrastructure. It's not a VPS system (or much like Amazon's EC2): you don't get a "server" at all. Instead, you write your applications and upload them to the GAE using a proprietary toolset. GAE then runs your application for you, and takes care of scalability, persistence, backups and all that boring stuff. You really have no idea (or need to care, in theory) about how all of that is managed. In this respect, GAE delivers that most hyped of recent buzz word: cloud computing.

GAE really is an amazing feat of engineering. It works. Things seem to scale (not that I've actually tested that). There's a whole infrastructure for billing and quota management. You get a lot for free, but you also have a predictable billing model once you scale beyond a certain size. I haven't done the sums, but for a business, the cost savings on hardware and maintenance could be huge and a predictable cost structure would be a boon.

That said, GAE is cloud computing in its infancy. You can't just pick up your ERP installation, say, and drop it into GAE. For one thing, it's pretty much focused on web applications. More importantly, you really need to write your applications for the GAE to begin with. In that sense, it's more like a development environment, complete with client-side tools and mocks (that work very well!), a server-side administration console and documentation. It's not a million miles away from what Zope 2 tried to be years and years ago: a way to program and deploy applications "through the web" (or "through-an-upload") on a shared platform.

When you write a GAE application, Google provides some basic services: storage (via a proprietary, non-relational, entity-based system that I assume is based on Google's BigTable underneath), authentication (via Google accounts), caching (via memcached), image manipulation (PIL), cron jobs and so on. These have specific APIs that you write to. Beyond that, you are operating in a sandbox. You can't read or write files, open sockets and so on.

At the moment, Python is the primary GAE language. Google have written a small, WSGI-capable framework called 'webapp', which works fine for simple applications. You can deploy other frameworks with your application, though, and Django (0.96) is available pre-installed. You get most of Python 2.5, though a few system calls are unavailable due to the sandbox.

Excitingly, Java support is now being tested, which should open GAE up to a much larger audience. Google have done the right thing here and used existing standards, such as Servlets, JSPs, JDO and JPA. There's also an Eclipse plug-in. That should make GAE development a lot more accessible to the armies of J2EE developers out there, so long as they're willing to accept the sandbox.

The biggest hurdle for porting existing applications - and possibly also when writing new applications - is probably going to be the storage engine. It's very easy to work with and I rather like it, but it's not a relational database, and it certainly isn't something you will have used before, being proprietary to GAE. It also supports limited number of data types and query operations (you can't do inner joins, for instance - you need to load the entities and process them in code). The data store is probably the secret to GAE scaling well, so it's not at all surprising it has to be this way. And for a large number of web apps, it's going to be plenty. But it is a trade-off that makes GAE good for some types of applications and unsuitable for others.

I'm not quite sure how to define the types of applications that GAE is good at. It is definitely a tool for web applications. GAE works well for things like the Google Summer of Code administrator application, or Google Moderator. It's probably better for writing small and clever web applications than big and complicated ones. If you're happy with using Google Accounts for authentication and your application doesn't need a very powerful database, then the 'webapp' framework is going to get you going very, very quickly. If you need more structure to your applications, you'll want to deploy Django or Pylons or some other, more established framework, of course, and invest a bit more time in setting up your project.

What about Plone? GAE is not going to be able to host a Plone site any time soon (we'd probably need to ditch the ZODB and use the GAE data model, for starters), but it may be able to host a small front-end web application that is populated with data from a Plone site using something like transmogrifier or ContentMirror pushing data over web services.  I'm also toying with a service that I hope will help with release management around Zope/Plone products (more on that later), which is a great fit for GAE. Tools that crunch ratings or let you build buildouts, maybe things like specialised issue trackers or other "service" type applications - these are good matches for GAE. If you write a GAE application, deployment and management becomes a non-issue, which makes it a lot more attractive to throw up ten small, specialised applications where perhaps before you'd have written one big one.

For me, this is the most exciting aspects of GAE. It opens up the possibility of open source communities creating and deploying "Open Source-as-a-Service" (I'm claiming the rights to the acronym OSaaS right now...), much like we've been developing and distributing "Open Source Software" for years. I suspect those services will be built in support of the traditional open source software projects, at least initially. There's not really the same kind of business model around this as there is around providing support and customisation, for example. However, by outsourcing some of our problems to Google, we can focus on what we do best and probably have a lot of fun in process.

And what of the GAE experience itself? I've had a lot of fun working with it. The 'webapp' framework is rather minimalist, and so probably won't scale conceptually if you go beyond a dozen or so controllers, but it's really easy to learn. The GAE documentation is good, and the client side tools and admin console feel like Google software: minimalist, a bit quirky perhaps, but ultimately useful and clever. If you haven't played with GAE and you have a spare long weekend, I can highly recommend it.

Document Actions

Zope at GAE

Posted by Myroslav Opyr at Apr 28, 2009 01:41 AM
Hi,

have you seen port of Zope 3 (as application server) to GAE? Persistent is working in sandboxed environment and there is DatastoreStorage for ZODB. Check out blog post at http://talk.quintagroup.com/blogs/myroslav/zope-at-gae and discussion at http://friendfeed.com/[…]/

Regards,

m.

Zope at GAE

Posted by Martin Aspeli at Apr 28, 2009 02:11 AM
Hi,

Yes, I did see that and probably should've mentioned it. It's Zope 3, though, not Zope 2. The main problem there is C extensions.

Even if that could be overcome, I'm still not convinced it'd actually work for something like Plone, though. Maybe in a more stripped-down version. It'd be interesting for sure. But as I said, I also don't know if Plone is a good match for GAE.

Martin

Zope at GAE

Posted by Myroslav Opyr at Apr 28, 2009 03:48 AM
Hi,

Grok starts up in 20 seconds at GAE. With 30 sec limit it is quite limited application. As soon as AppEngine suffers from any instabilities of their infrastructure, Grok apps stop working first.

The only possibility for Zope applications (I'm looking at Plone 4) to work properly at GAE is to redo startup dramatically. To have only minimal framework loaded during startup. And everything else to load as needed.

One of the steps in right direction is resolution of Dependency Problem (http://faassen.n--tree.net/blog/view/weblog/2009/01/29/0). The less dependencies, the more chances that loading minimal framework and modules required to handle arbitrary request will fit in 30 sec limit.

And Zope applications are aggressively unloaded from memory, due to fact that they are huge comparing to conventional GAE apps. It is more economical to keep 100 times more small apps in memory then single Grok app.

Regards,

m.

GAE lack of map/reduce is a show stopper

Posted by http://silviot.myopenid.com/ at Apr 28, 2009 08:50 AM
GAE hasn't (yet?) support fo long running processes, scheduled tasks and map/reduce jobs.
I agree with Gregable ( http://gregable.com/[…]/google-appengine-critical-flaw.html ) about this. In particular without a chance to migrate data (probably huge amounts of data) as you further develop your application is a critical flaw.
Someone ( http://architects.dzone.com/[…]/emulating-a-long-running-proce for instance ) has tried to emulate scheduled long running processes, but it's still a hack, not supported by google.
In GAE you can't count more than 1000 items, so if you want to show counters in your app (12340 news about Plone today ;-) ) you will have to update your counters each time you add/remove an entry.
That said, I think GAE is a great tool.

GAE lack of map/reduce is a show stopper

Posted by Martin Aspeli at Apr 28, 2009 09:00 AM
Right. :-)

To my mind, these are "flaws" only if you take GAE for something it's not. It's not going to be a panacea for all types of programs or applications. It's not something that's going to work for all (or even most) applications, even if there are workarounds. But it's a great tool for those "small and clever" applications. In a way, that probably describes most of Google's applications too, so it's not so surprising that they cater to this audience.

Martin
Sponsor message

Proper web design for high traffic websites which run on dedicated server is needed in order to access them through wireless internet providers.

Plone Book
Professional Plone 4 Development

I am the author of a book called Professional Plone Development. You can read more about it here.

About this site

This Plone site is kindly hosted by: 

Six Feet Up