The value proposition of using the cloud is that “things will be easy this way.” Building and operating large scale clouds, however, is rarely easy. As Mark Lucovsky said in his recent interview, “Who said large scale distributed systems are simple?” While solutions like Cloud Foundry make building core cloud technology less daunting, it’s still not easy.
In this guest blog, Lucas Carlson, Founder and CEO of AppFog, illustrates the story of how AppFog was built using Cloud Foundry open PaaS technology and what they learned along the way
Core Cloud Technology is Hard
Running a simple Cloud Foundry-based service is not very hard. Check out Micro Cloud FoundryTM.
Building one that is suitable for production and enterprise workloads is fairly hard.
Building a Cloud Foundry-based service with one API end-point to run in 6+ different availability zones on 4+ independent infrastructure vendors and that is suitable for production and enterprise workloads, at scale and in general availability? Now that is really hard.
Hitting the Ground Running with Cloud Foundry
As you can imagine, ever since AppFog announced GA, people have been asking how we did it. How did we figure out how to scale up a public cloud offering for Cloud Foundry that delivers one interoperable interface across multiple clouds, multiple availability zones and multiple vendors?
First: The Cloud Foundry codebase is really very good. In choosing to build using Cloud Foundry we eliminated a number of the potential serious challenges and major time sinks likely with this sort of project.
Second: The leverage that we were able to gain from the additional tools, code and services from within the Cloud Foundry ecosystem dramatically accelerated our development.
Third: Having a non-Cloud Foundry-based public PaaS already in GA and already at scale was a massive boost for us. We had the operational experience in-house needed to pull off a great PaaS.
AppFog’s ops experience was invaluable for us. Although PaaS is closely tied to the NoOps movement, NoOps doesn’t mean no ops, it only means no ops for the developer. As anyone who has run Cloud Foundry knows, a lot of ops is necessary and essential to running PaaS.
And most importantly, we had a vibrant community of tens of thousands of developers (the users of PHP Fog) that we could consult to help us define, test, and evolve AppFog through private beta to Public Beta and now to GA.
We Listened and Delivered
It was this community that helped us figure out what AppFog at GA would need to be, how it would need to work, and what our pricing would be. This was what devs asked us for:
- The ability to easily scale apps across dozens or hundreds of load-balanced instances, even on the free plan
- Must run apps on the fastest servers available in the infrastructure (yes, we mean, for example, M2.4XL)
- Must be able to run apps across multiple infrastructures (Rackspace, HP, AWS, Azure–and with no price difference)
- Must simplify pricing. No more weird packages and bundles and names. Make it simple and affordable and most of all make it fair.
- True interoperability in the cloud, including single-click, zero-code migrations between cloud vendors, eliminating all traces of vendor lock-in
- Target of 30-second deployments to AWS, Rackspace, HP, Azure, etc.
- Support for any and all languages in the Cloud Foundry codebase
- Support for any and all relational and NoSQL data stores
This is a daunting list. But it’s what devs wanted.
So let’s get into the nitty gritty… how did we actually do this?
How We Built a PaaS Around Cloud Foundry
This is a complex story. Fortunately, a few months ago, Jeremy Voorhis gave a talk at QCon in London describing how AppFog built a commercial system by using the Cloud Foundry OSS bits alongside our own set of custom extensions and enhancements. This is far and away the best place to start for understanding the Cloud Foundry-to-AppFog evolution. The slides for the talk can be found here and the video here.
We won’t go over everything in Jeremy’s hour-long talk here, but we’d like to cover some of the main points.
First, Jeremy clears up some of the confusion surrounding the signifier “Cloud Foundry.” On one hand, CloudFoundry.com is a hosted PaaS offering from VMware that runs on the vSphere virtualization platform based on the open source project found on CloudFoundry.org. VMware also has open sourced the tooling it uses to manage CloudFoundry.com–Cloud Foundry BOSH–and exposes the exact BOSH releases used live in production on their hosted service.
We do not run on the hosted VMware service but instead we are built from the same open source Cloud Foundry project. This open source distributed system enabling teams to manage and deploy apps and services over their lifecycle in the cloud is the core of the AppFog service.
AppFog is unique within the current Cloud Foundry ecosystem because we have built an infrastructure-agnostic Cloud Foundry service.
The beauty of Cloud Foundry is that it can, in principle, be implemented in conjunction with infrastructure offerings ranging from an OpenStack-based IaaS like Rackspace or HP to AWS, Joyent, Citrix, and beyond. This inherent multi-cloud portability is why the Cloud Foundry project was so inspiring to us.
For us, the ability to deploy and redeploy across infrastructures within the lifecycle of an application is the key to building a next-generation PaaS. It also, we think, fulfills the underlying mission of PaaS to: (a) sharpen product team focus in development; (b) produce a shorter feedback loop within app lifecycles, and; (c) provide the possibility of near-instant horizontal scalability.
Cloud Foundry’s Role in the Evolution of Cloud Computing
Even for those who are familiar with the space of cloud computing in general, it’s important to review historical context. For us at AppFog, Cloud Foundry has provided some of the crucial building blocks to assembling the first next-generation PaaS. But what do we mean by that?
First, it’s important to get some background. PaaS in its initial forms was enabled by a number of historical breakthroughs such as:
- The rise of data centers via companies like Rackspace in the 1990s
- The emergence of server virtualization via VMware in the early ‘00s
- The introduction of APIs to virtualized resources via AWS in the mid ‘00s
- The more recent rise of the DevOps paradigm with their associated and derived tools like Puppet and Chef.
But this trajectory really began coming to fruition in just the past two years. Tools like Cloud Foundry and OpenStack have emerged that enable full application lifecycle management. What this means is that the “platform” in “platform as a service” can now encompass all aspects of the application lifecycle outside of development. Jeremy refers to this as “NoOps” in the talk. What it means is emphatically not that no one does Ops work anymore. Instead, it means that responsibility for Ops work is passed on to the platform layer and remains there for most if not all of the app’s lifecycle.
Without a comprehensive tool like Cloud Foundry, we never would have been able to overcome the gap between (a) first-generation PaaS platforms, which we were essentially little more than gateways for (unwieldy) infrastructure management, and (b) next-generation PaaS like AppFog, which provide a far more comprehensive set of user benefits.
AppFog’s Modifications of Cloud Foundry
AppFog began as an effort to bring a great user interface and multi-cloud experience to the hosted Cloud Foundry ecosystem. We saw a number of stumbling blocks standing between the Cloud Foundry codebase and a full-fledged PaaS offering. The current Cloudfoundry.org project has CLI and various IDE integrations, but our vision included the full user experience, including fast sign-up, a web console, pre-built example applications that became what we call Jumpstarts, add-ons, and more.
While some Cloud Foundry users are indeed quite comfortable operating the system from the command line, Eclipse or the Spring Tool Suite (just as many AppFog users do), we also wanted to provide users with an elegant and intuitive console from which they could manage all of their apps. An example of the AppFog console is below:
The creators of the system wanted Cloud Foundry to remain agnostic toward the UX side of things and didn’t want to make its UX implementation overly opinionated.
We know that simplicity and intuitiveness are not ends in themselves, and that it’s all too easy to produce an elegant UX on top of a subpar underlying system. But there’s also a lot to be said for seeking to accommodate the needs of as many end users as possible and we believe that we’ve done that.
Here are some of the many innovations that we introduced on top of and below Cloud Foundry:
- A Varnish-based caching tier used to achieve significant speed gains
- Single-click clone from one CF-based infrastructure to another for true workload portability
- A Cloud Foundry API compatibility layer to auto-route requests to completely autonomous CF instances running in totally different parts of the planet
- Deep cross-datacenter nagios integration so we know about problems before they affect you
In addition, Cloud Foundry enabled AppFog to create a deeply holistic, self-communicating architecture. It has also enabled us to design AppFog as a system that assumes that failure happens and provides a set of protocols for handling failure, be it at the infrastructure level or elsewhere.
Carrying Cloud Foundry Forward
Building a full-fledged, commercial grade PaaS that spans multiple clouds, across multiple clouds, connecting it all together with a state of the art end-user experience is an immense undertaking even when starting with the base CloudFoundry.org system. It takes a dedicated team with a great deal of developer empathy, a team willing to step into the customers shoes and think like they think.
But we think that the result has been more than worth the effort. We were very fortunate to have a core system like Cloud Foundry available to us. We’ve given back in many ways, including adding initial PHP runtime support to the project, and continue to give back as we optimize Cloud Foundry to run well in public clouds.
We are incredibly excited about AppFog going GA. We are even more excited about the thousands and thousands of developers who have already signed up and deployed thousands and thousands of apps.
We believe that the success of AppFog demonstrates not just the power of PaaS, but also the strength of the Cloud Foundry codebase and the power of its open source ecosystem and community. This is a win not just for AppFog, but for everyone invested in the success of Cloud Foundry.
About the Author: Lucas Carlson is an entrepreneur and professional programmer who specializes in the development of web applications at scale. Lucas is the Founder and CEO at the leading cross-cloud PaaS company AppFog. He has authored over a dozen Ruby libraries and contributed to a range of significant Rails products including Rails and RedCloth. Lucas founded, ran and judged the popular Ruby on Rails contest Rails Day and has been a well-received speaker at many major programming conferences. Lucas was the first engineer at MOG and was responsible for development and technical direction for that seminal music streaming service. Lucas lives in Portland, Oregon, and loves statistics and Go.