The Cloud Foundry Blog

Using Cloud Foundry Services with Ruby: Part 1 – Auto-reconfiguration

Right from the launch, Cloud Foundry supported auto-reconfiguration of Spring and Rails apps that use a relational database service.  This allowed deploying such an app without changing a single line of code.  Recently, we extended this support for Spring apps to cover all services (Redis, Mongo, and Rabbit).  We are now extending this support for all services for Rails and making this available for Sinatra apps as well.  In this blog, we will explore how auto-reconfiguration works with Rails and Sinatra applications.

Auto-reconfiguration in action

To demonstrate auto-reconfiguration, we will grab an application from github and deploy it to Cloud Foundry without modification.  Let’s use lamernews, a Sinatra app that uses Redis.

mycomp:lamernews$ vmc push lamernews 
Would you like to deploy from the current directory? [Yn]: 
Application Deployed URL ["lamernews.cloudfoundry.com"]: 
Detected a Sinatra Application, is this correct? [Yn]: 
Memory Reservation ("64M", "128M", "256M", "512M", "1G") ["128M"]: 
Creating Application: OK 
Would you like to bind any services to 'lamernews'? [yN]: y 
The following system services are available 
1: mongodb 
2: mysql 
3: neo4j 
4: postgresql 
5: redis 
Please select one you wish to provision: 5 
Specify the name of the service ["redis-52216"]: 
Creating Service: OK 
Binding Service [redis-52216]: OK 
Uploading Application: 
Checking for available resources: OK 
Processing resources: OK 
Packing application: OK 
Uploading (1K): OK 
Push Status: OK 
Staging Application: OK 
Starting Application: OK 

Looks like the app deployed successfully.   Lamernews uses Redis to store comments, so let’s comment on a post and verify it stores successfully.


Now I press “Send comment”, and it looks like my comment was applied.

Let’s take a look at the lamernews code that initializes the Redis connection:

 
RedisHost = "127.0.0.1" 
RedisPort = 10000 
$r = Redis.new(:host => RedisHost, :port => RedisPort) if !$r 

As you can see, the code is attempting to connect to Redis on localhost, however it worked just fine when we deployed to Cloud Foundry.  How is this possible?  Cloud Foundry will automatically detect initialization of several popular clients anywhere in your code and swap out your connection parameters for those of a service bound to your application. Read on to find out more about how this works!

Auto-reconfiguration for Sinatra

Your application consists of business logic and interaction with services such as database and messaging.  In a Sinatra application, you may initialize these services in Sinatra::Base#configure(). However, this is certainly not a requirement.  You are free to initialize your services wherever you want, perhaps lazily in response to a request. Additionally, there are several different client libraries you can use for connection to data and messaging services (ActiveRecord, DataMapper, Mongo Ruby Driver, MongoMapper, etc). For example, consider the following code that creates a Redis client:

require 'redis' 
module Demo 
class App < Sinatra::Base configure do 
  redis = Redis.new(:host => '127.0.0.1', :port => '6379') 
end 
... 
end 
end

We can make one easy observation: The Redis host and port point to a server on localhost.  When you push this application to Cloud Foundry and bind a Redis service, the URL for that service is not going to be 127.0.0.1:6379!  So without an additional mechanism, such application will fail on startup.  This is where the auto-reconfiguration mechanism comes into play.  The auto-reconfiguration mechanism leverages Ruby metaprogramming to intercept the Redis initialization and replace the connection parameters with those of the Redis service bound to the application.  The result is that the user application works in local deployment and in Cloud Foundry without any change. When your Sinatra application is staged during the deployment process, Cloud Foundry will make two modifications:

  1. It will add an additional cf-autoconfig gem to your Bundle
  2. It will wrap the execution of your main Sinatra file (e.g. app.rb) in an auto_stage.rb file that ensures that all dynamic class modification is done prior to application execution.

Auto-reconfiguration for Rails

Cloud Foundry already provides auto-reconfiguration of your database in Rails by modifying the production settings in your config/database.yml during staging.  We have now added auto-reconfiguration of Mongo, Redis, and Rabbit clients as well. For example, you may have the following in config/initializers/redis.rb:

 $redis = Redis.new(:host => '127.0.0.1', :port => 6379, :password => 'mypass') 

Once again, we can see that the Redis host and port point to a server on localhost.  Cloud Foundry will automatically replace these localhost connection parameters with those of the Redis service bound to your application. While it’s fairly common to put these types of connections in a Rails Initializer File, auto-reconfiguration should work just as well if you create the connection somewhere else within your application. When your Rails application is staged during the deployment process, Cloud Foundry will make two modifications:

  1. It will add an additional cf-autoconfig gem to your Bundle
  2. It will add an Initializer file to config/initializers that ensures that all dynamic class modification is done prior to loading other Initializers (and thus before your application executes).

Supported Clients

The following table shows the supported clients for auto-reconfiguration.

Client Minimal Supported Version
redis-rb 2.0
Mongo Ruby Driver 1.2.0
amqp 0.8
carrot 1.0
mysql2 (Sinatra only) 0.2.7
pg PostgreSQL client (Sinatra only) 0.11.0

In some cases, you don’t need to be using these clients directly. For example, the popular object mapper for Mongo, 

mongo_mapper, uses the Mongo Ruby Driver.  Therefore, if your application uses mongo_mapper, Cloud Foundry can auto-reconfigure it to connect to your Mongo service. Note that the mysql and postgresql gems are listed as Sinatra only.  This is because we auto-reconfigure relational database connections in Rails without metaprogramming, by modifying your production database settings in your database.yml file.

Under the Hood

As mentioned, we leverage Ruby metaprogramming to intercept a common set of method calls that create connections.  We then replace the connection parameters with those of the service bound to your application.  We do this with some well-known metaprogramming patterns: Open Class or Class Extension and Around Alias.  These are described thoroughly in the excellent book Metaprogramming Ruby. Here is an example from our Redis auto-reconfiguration support:

 
require 'cfruntime/properties' 
module AutoReconfiguration 
module Redis 
def self.included( base ) 
  base.send( :alias_method, :original_initialize, :initialize) 
  base.send( :alias_method, :initialize, :initialize_with_cf ) 
end 
def initialize_with_cf(options = {}) 
  service_props = CFRuntime::CloudApp.service_props('redis') 
  cfoptions = options 
  cfoptions[:host] = service_props[:host] 
  cfoptions[:port] = service_props[:port] 
  cfoptions[:password] = service_props[:password] 
  original_initialize cfoptions 
end 
end 
end 
require 'redis' 
class Redis 
  include AutoReconfiguration::Redis 
end

The code starts by opening the Redis class and adding the methods defined in our AutoReconfiguration::Redis module.  The first method, self.included, sets up an Around Alias that directs all of your calls to Redis.new to the new initialize_with_cf method.  This method utilizes our cf-runtime gem library look up the Redis service connection properties and then calls the original Redis initialize method with the changed parameters. We use a similar approach for each supported client.  Feel free to browse the code in our github repo for more details.  Let us know if there are other clients or hook points we should support (feel free to submit a pull request!).

Limitations

Auto-reconfiguration of services work only if there is exactly one service of a given service type.  For example, you may bind only one relational database service (MySQL or Postgres) to an application. If an application doesn’t follow these limitations, auto-reconfiguration will not take place.  In those cases, you can still take advantage of the cf-runtime gem described in the next blog post to manually configure service access. The auto-reconfiguration mechanism expects typical Ruby applications.  If your application configuration is complex, it may not work.  In those cases, you can opt out of auto-reconfiguration as we will describe next.

Opting out of auto-reconfiguration

There may be situations where you will like to opt out of auto-reconfiguration.  Cloud Foundry offers a few ways to opt out of the auto-reconfiguration mechanism.

  1. Create a file in your Sinatra or Rails application called “config/cloudfoundry.yml”. Add the entry “autoconfig: false”.
  2. Include the ‘cf-runtime’ gem in you Gemfile. Cloud Foundry uses this mechanism to opt out, since applications either want to have the auto-reconfiguration behavior or simply take control over service creation completely. We do not see the value in auto-reconfiguring some services and manually configuring others. If you are using cf-runtime and not using Bundler, you can still opt-out of auto-reconfiguration by creating the cloudfoundry.yml file as described above.

Conclusion

Auto-reconfiguration in Cloud Foundry is a wonderful way to get started deploying your Rails and Sinatra apps quickly.  As your application matures or makes use of multiple services, you may need finer control of your service connections.  In the next blog in this series, Thomas Risberg will explain how to use the new cf-runtime gem for simplified connections to Cloud Foundry services.

– Jennifer Hickey, The Cloud Foundry Team

Don’t have a Cloud Foundry account yet?  Sign up for free today

This entry was posted in Cloud Foundry. Bookmark the permalink.

9 Responses to Using Cloud Foundry Services with Ruby: Part 1 – Auto-reconfiguration

  1. Pingback: Using Cloud Foundry Services with Ruby: Part 2 – Run-time Support for Ruby Applications | Blog

  2. Pingback: Cloud Foundry Further Improves Support for Ruby Applications | Blog

  3. Jennifer, this looks great and very clever. Where is the source for the cf-autoconfig gem?

    Nic

  4. Bonus problem, the cf-autoconfig gem isn’t yet hosted on rubygems.org (http://rubygems.org/gems/cf-autoconfig – not found)

    and

    $ gem install cf-autoconfig -V
    GET http://rubygems.org/latest_specs.4.8.gz
    302 Moved Temporarily
    GET http://production.s3.rubygems.org/latest_specs.4.8.gz
    304 Not Modified
    GET http://gems.engineyard.com/latest_specs.4.8.gz
    301 Moved Permanently
    GET http://rubygems.org/latest_specs.4.8.gz?
    302 Moved Temporarily
    GET http://production.s3.rubygems.org/latest_specs.4.8.gz?
    200 OK
    ERROR: Could not find a valid gem ‘cf-autoconfig’ (>= 0) in any repository
    ERROR: Possible alternatives: autoconfig, cnuconfig

    • Jennifer Hickey says:

      We weren’t planning to publish this gem to rubygems, as it is currently just used internally by the CF stagers, and we didn’t foresee a situation where someone would use it elsewhere. Is there a particular use case for embedding the gem in a different app?

  5. Ways to get the cf-autoconfig source, sort of…

    git clone git://github.com/cloudfoundry/vcap.git
    cd vcap
    gem install cf-runtime open_gem
    gem install staging/lib/vcap/staging/plugin/resources/cf-autoconfig-0.0.2.gem
    gem open cf-autoconfig

    Tada.

  6. Pingback: Cloud Foundry Improves Support For Background Processing | Blog

  7. Pingback: Running Resque Workers on Cloud Foundry | Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>