The insider's guide to everything sparqcode
Header image

Streaming Data With Rails 3.1 or 3.2

Posted by Mike on February 4, 2012 in Uncategorized - (0 Comments)

We recently upgraded one of our services to use Rails 3.2. The downside I discovered pretty quickly is that rails broke support for the 3.0 way of streaming data.

The 3.0 way, simplified a bit:

self.response_body =  proc{ |response, output|
  @records.each do |record|
     output.write << CSV.generate_line(record.csv_values)
}

In >= 3.1, the proc method has been deprecated. What are we to do then? Well it's actually pretty simple if we go digging in the Rack source code.

      if body.respond_to? :to_str
        write body.to_str
      elsif body.respond_to?(:each)
        body.each { |part|
          write part.to_s
        }
      else
        raise TypeError, "stringable or iterable required"
      end

Basically, if we set response_body to something that responds to 'each' then rack will automatically send those chunks back to the browser without having to do anything special on our end. In my case I wrote a little streamer class that would take care of it.

class RecordCsvStreamer
   attr_reader :records

   def initialize(record_scope)
      @records = record_scope
   end

   def each
     records.each do |record|
        yield csv_row(record)
     end
   end

   def csv_row(record)
   # however you want to format it
   end
end

Then in my controller I do

class RecordController
  def index
    @records = Record.all

    self.response_body = new RecordCsvStreamer(@records)
  end
end

Happy streaming!

If you’re used to a language like C++ or Java the common way to share behavior across classes was to use inheritance. This has some problems because you should only really use inheritance in a is-a relationship. A politican class would probably behave like the fool class, but I don’t think it’s appropriate for the politician to be a fool because then any changes to the fool would also the change the politician.

Now to have classes behave like others we use a strategy called encapsulation. Using the previous example, you would do something like this:

class Fool
  def speak
  end
end

class Politician < Fool
  def initialize
    @fool = Fool.new
  end

  def speak
    @fool.speak
  end
end

Now what if the implementation for speak was the same across the fool and politician classes and would always be the same? Well we could use a mixin to define that behavior.

module Speaking
  def speak
  end
end

class Fool
  include Speaking
end

class Politician
  include Speaking
end

Now this has one downside, lets say the Speaking module was written like this:

module Speaking
  def speak
    "I say " + @motto
  end
end

This definition assumes and requires that both the Fool class and the Politician class have a @motto instance variable. This would also apply to any method definitions that are referenced as well.

The mixin is not the golden hammer but it is a very powerful tool to have in your toolbox.

Publishing A Gem With Jewler

Posted by Mike on April 13, 2011 in Uncategorized - (0 Comments)

In order to do this you will need two things:

  1. the jeweler gem
  2. an account with rubygems.org

Jeweler is pretty useful utility for managing your gem’s lifecycle. To get the jeweler gem run

gem install jeweler

To create your project run

jeweler yournewgemname

It will create a new git repository called yournewgemname. You are basically ready to go at this point. You can add your source code files, test your gem, etc at this point.

Publishing

To publish your gem first you have to create a gem verison number

rake version:write

And then to publish your gem to rubygems.org

rake gemcutter:release

What’s Your Mobile Presence?

Posted by Mike on April 6, 2011 in Thoughts - (0 Comments)

Soon there is going to be more cellular phone subscriptions in the United States than there are people. Does this scare you? Unless you’ve figured out how to present yourself online, it should. What do I mean by mobile presence? Well it’s not hard to imagine a future where all of your customers access your website and information about you/your company entirely through the customer’s mobile phone. I think a lot of people out there still do not have a handle on how to deal with this shift in access patterns. I’ll share my thoughts on different ways you can deal with your mobile presence.

Don’t

A lot of the modern smartphones (iPhone and Android specifically) have very good browsers in them. It’s quite likely that your web page already renders just fine and your customers will be able to access the information they need. There’s a slight problem with this though, smart phones work on a different aspect ratio than your computer does so a lot of pages don’t look right, require a lot of scrolling, and generally make it harder for the user to navigate effectively.

Mobile Friendly Pages/JQuery Mobile

Another option is to make a mobile centric version of the page. Your mobile site is designed with the constraints of the mobile device in mind. The problem with this approach is that not all mobile phone browsers are created equal. As I mentioned both iPhone and Android have very capable browsers, but the experience you get on older blackberries and Windows Mobile can be downright painful.

The fine folks over at JQuery have been working to solve this problem with jquery mobile. Basically it’s a javascript framework that will adjust your UI to match the capabilities of the phone viewing it and do its best to make your page look like a native phone app. Even though jquery mobile is still in the alpha stage, it’s still remarkably mature. It’s really easy to get a really nice looking app up and running quickly, I highly recommend checking it out if you’re building a mobile web site.

Facebook

If you don’t already, you should have a Facebook page. The nice thing about maintaining an up to date facebook page is that Facebook has already done all the hard work of creating a mobile app and optimizing their site for mobile devices. If you direct your users to your facebook page, all you need to worry about is making sure your personal or company page is up to date.

SPARQ.ME

Available to all of our registered users is a feature called SPARQ.ME. It lets you quickly and easily create a mobile friendly site for yourself or for your business. You can check out my SPARQ.ME here. Like Facebook, we do all the work to make sure it looks good on mobile phones, but we let you customize the page with custom widgets so you can tailor the page to who you think will be viewing it.

In Conclusion…

Think about your mobile presence. As mobile phone usage increases in the US, being accessible through a mobile device is going to become more and more important. It’s better to think about this kind thing now rather than being blindsided by it in the future.

Recently I implemented a feature for our up and coming referral program that lets you refer people using your Yahoo and Google contacts. As part of this I needed to work with Yahoo’s own implementation of the OAuth standard. This interaction caused me quite a bit of pain, so I decided to write up a little tutorial to help out other struggling with this problem.

In order to access a Yahoo contact your first order of business will be to get an application key and an application secret from https://developer.apps.yahoo.com/projects.

Yahoo verifies the callback url you provide to be from the same domain as what you registered for (you can’t register with localhost). Because of this you will need to override your hosts file to match your domain, in my case I had an entry like

127.0.0.1 www.sparqcode.com

With all that in place, you will need two oauth consumers: one to authenticate and one to actually consume the data from Yahoo.

  def yahoo_oauth_consumer
    OAuth::Consumer.new(YahooOauthCredentials::KEY,
        YahooOauthCredentials::SECRET, {
          :site                 => 'https://api.login.yahoo.com',
          :scheme               => :query_string,
          :request_token_path   => '/oauth/v2/get_request_token',
          :access_token_path    => '/oauth/v2/get_token',
          :authorize_path       => '/oauth/v2/request_auth'
        })
  end

  def yahoo_calendar_consumer
    OAuth::Consumer.new(YahooOauthCredentials::KEY,
        YahooOauthCredentials::SECRET,
        {
          :site => 'http://social.yahooapis.com'
        })
  end

In your initial view, you need to redirect to Yahoo’s authorization URL.

    request_token = yahoo_oauth_consumer.get_request_token(:oauth_callback => yahoo_callback_contact_import_url)
    redirect_to request_token.authorize_url

Once your user has granted your application access to her contact list, you need to actually parse the response.

  def yahoo_callback
    if(params[:oauth_token] && params[:oauth_verifier])
      request_token = OAuth::RequestToken.new(yahoo_oauth_consumer, session['request_token'], session['request_secret'])

      access_token = request_token.get_access_token(:oauth_verifier => params[:oauth_verifier])

      access_token.consumer = yahoo_calendar_consumer
      yahoo_guid = access_token.params[:xoauth_yahoo_guid]

      response = access_token.get("/v1/user/#{yahoo_guid}/contacts?format=json&count=max")
      json = ActiveSupport::JSON.decode(response.body)

      @name_email_map = parse_yahoo_contacts_response(json)
    end
  end

  def parse_yahoo_contacts_response(json)
    name_email_map = {}

    return name_email_map if json['contacts']['contact'].nil?

    json['contacts']['contact'].each do |contact| 

      name = nil
      email = nil

      contact['fields'].each do |field|
        field['type']

        if field['type'] == 'name'
          name = "#{field['value']['givenName']} #{field['value']['familyName']}"
        end 

        if field['type'] == 'email'
          email = field['value']
        end
      end

      if(email)
        name ||= email
        name_email_map[name] = email
      end
    end

    name_email_map
  end

Any questions please leave a comment and I’ll do my best to answer.

Full code for handling storing session variables and redirecting


  def yahoo_import
    request_token = yahoo_oauth_consumer.get_request_token(:oauth_callback => yahoo_callback_contact_import_url)
    session['request_token'] = request_token.token
    session['request_secret'] = request_token.secret
    redirect_to request_token.authorize_url
  end

  def yahoo_callback
    if(params[:oauth_token] && params[:oauth_verifier])
      request_token = OAuth::RequestToken.new(yahoo_oauth_consumer, session['request_token'], session['request_secret'])

      access_token = request_token.get_access_token(:oauth_verifier => params[:oauth_verifier])

      yahoo_guid = access_token.params[:xoauth_yahoo_guid]

      access_token.consumer = yahoo_calendar_consumer
      yahoo_guid = access_token.params[:xoauth_yahoo_guid]

      response = access_token.get("/v1/user/#{yahoo_guid}/contacts?format=json&count=max")
      json = ActiveSupport::JSON.decode(response.body)

      @name_email_map = parse_yahoo_contacts_response(json)

    end
  end

This took me about an hour to get working properly so I figured I would share this with everyone. The big thing to keep in mind is that multipart emails don’t automatically pick up which template they need to render, so you need to specify render_message.

	def email_file(email, zipfile)
		recipients email
		content_type "multipart/mixed"
		subject "Yo"
		from "noreply@sparqcode.com"

		part 'text/html' do |p|
			p.body = render_message("the_template_you_want_to_render", {})
		end

		attachment "application/zip" do |a|
			a.body = zipfile.read
			a.filename = "somefile.zip"
		end
	end

I found facebook’s documentation confusing, almost hair-rippingly so, when it came to posting a message to somebody’s wall through facebook.  Facebook uses a method called OAuth2 to do authentication.  If you’re not familiar with OAuth2, from a high level it’s a security scheme that lets you do things on behalf of somebody else without ever having to obtain that other person’s username or password.

To prepare you will need a few things.

The OAuth2 gem

gem install oauth2

You will also need to setup a Facebook Application.

And finally you should probably put your application credentials somewhere easily accessible.  Using some tricks with const_missing I created a FacebookOauthCredentials file that will have relevant information regardless of what environment I’m in.

class FacebookOauthCredentials

  production = {
    :APP_ID => '',
    :API_KEY => '',
    :APP_SECRET => ''
  }

  test = {
    :APP_ID => '',
    :API_KEY => '',
    :APP_SECRET => ''
  }

  development = {
    :APP_ID => '',
    :API_KEY => '',
    :APP_SECRET => ''
  }

  OAUTH_CREDENTIALS = {
    "development" => development,
    "test" => test,
    "production" => production
  }

  def self.const_missing(key)
    value = OAUTH_CREDENTIALS[Rails.env][key]

    unless value
      super
    end
    value
  end
end

This lets me do something like FacebookOauthCredentials::APP_ID and it will be relevant no matter what environment I’m in.

Now we need a way to use those credentials to authenticate with Facebook.  I added this in application_controller.rb:

def facebook_client
    OAuth2::Client.new(FacebookOauthCredentials::APP_ID, FacebookOauthCredentials::APP_SECRET, :site => 'https://graph.facebook.com')
 end

Now we need to actually talk to facebook, this action is linked to directly from one of the pages in the app.:

def facebook_session_create(poster)
    redirect_to facebook_client.web_server.authorize_url(:scope => 'publish_stream', :redirect_uri => 'http://www.yourdomain.com/facebook_callback')
end

In this example it will ask the user for permission to publish to the user’s stream. Regardless of whether or not the user accepts the user is redirected to the :redirect_uri parameter.

In the code below, when I call get_access_token the redirect_uri paramter is not actually used as a redirect, it is used to validate against the redirect_uri parameter that was passed in the facebook_session_create step.

  def facebook_callback
    if(params[:code])
      access_token = facebook_client.web_server.get_access_token(params[:code], :redirect_uri => 'http://www.yourdomain.com/facebook_callback'

      access_token.post('/me/feed', :message => "hi, I'm posting a message from ruby"
    end

    redirect_to "http://www.facebook.com"
  end

And that’s all there is to it. Happy posting!

Embedding a Url in a QR Code

Posted by Mike on March 9, 2011 in Thoughts - (0 Comments)

This may seem like a silly topic to a lot of people.  Using Google’s charts API is very easy to generate a QR Code that points to your website.  Like so much in life, the devil is in the details.  If you’re CNN and you’re just embedding a URL in your QR code you’re going to end up with something like this:

The biggest problem with this method is that it will be very hard to scan unless the person scanning has a high resolution camera with a macro lens.  The reason this QR code is hard to scan is because it there’s a lot of data.  Simply speaking, the larger the data payload in a QR Code smaller the dots are, and the harder it is for phones to scan.  How do you combat this?  Url shortening.

The gist of URL shortening is to take some url like http://www.google.com and turn it into something smaller like http://ab.cd/1234.  The idea being is that any arbitrarily long url will turn into something very small and QR Code friendly.  Here is that same CNN url shortened:

There are two ways you can shorten a URL: Case sensitive and Case insensitive.  Case sensitive shortening converts your url into a series of characters consisting of a-z, A-Z, and 0-9.  Case insensitive just uses a-z and 0-9.

The benefit of case sensitive is that you get more URLS per character, for example bit.ly will basically be able to store every URL in the universe with six characters.  The downside is that if you ever have to type your url in on a phone manually the user will have trouble.  Case insensitive on the other hand doesn’t practically require any more characters, but it means if the user ever needs to type in the url they will have a much easier time.  I’ll cover the actual algorithms to compress the URL in a future post.

When you encode URLs in a QR code you should definitely shorten it, but the method you use should be determined by how you expect people to use it.

 

This problem has been bugging me for quite a while and the usual googling did not get me a solution that worked.

After some experimentation and a bit more research, I found the answer in the comments section of a blog post here: http://aflatter.de/2010/06/testing-headers-and-ssl-with-cucumber-and-capybara/

So let me share the solution I ended up with. First, we have to hack the Capybara Driver and allow the headers to be overridden. This part I just copied verbatim from the blog post. You can put this file anywhere in features/support.

module RackTestMixin

  def self.included(mod)
    mod.class_eval do
      # This is where we save additional entries.
      def hacked_env
        @hacked_env ||= {}
      end

      # Alias the original method for further use.
      alias_method :original_env, :env

      # Override the method to merge additional headers.
      # Plus this implicitly makes it public.
      def env
        original_env.merge(hacked_env)
      end
    end
  end

end

Capybara::Driver::RackTest.send :include, RackTestMixin

module HeadersHackHelper

  def add_headers(headers)
    page.driver.hacked_env.merge!(headers)
  end

end

World(HeadersHackHelper)

Now, we need to add a new step definition. This is where we diverge from the blog post.

Given /^my user agent is "(.+)"$/ do |agent|
  add_headers({'HTTP_USER_AGENT'=> agent})
end

So that’s it for testing user agent strings. Now, I was trying to figure this out because I wanted cucumber to run tests for different mobile phone types. So it made sense for me to throw in another step definition that would make this easier.

Given /^I have an? (.+)$/ do |phone_name|
  add_headers({‘HTTP_USER_AGENT’=> SAMPLE_AGENT_STRING[phone_name]})
end

SAMPLE_AGENT_STRING can be defined as a constant hash like below:

#Sample user agent strings
SAMPLE_AGENT_STRING={
  “iPhone” => “Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7″,
  “Android” => “HTC_Eris Mozilla/5.0 (Linux; U; Android 4.0; en-ca; Build/GINGERBREAD) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1″,
  “Windows Mobile” => “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; HTC_Touch_Diamond2_T5353; Windows Phone 6.5)”,
  “Windows Phone 7″ => “Mozilla/4.0 (compatible; MSIE 7.0; Windows Phone OS 7.0; Trident/3.1; IEMobile/7.0) Asus;Galaxy6″,
  “Blackberry” => “BlackBerry9700/5.0.0.351 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/123″,
  “Palm Pre” => “Mozilla/5.0 (webOS/1.0; U; en-US) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/1.0 Safari/525.27.1 Pre/1.0″,
  “Nokia N97″ => “Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124″
}

Now I can write scenarios like:

Given I have an iPhone
And I am viewing “/detect-phone”
Then I should see “You are using an iPhone!”


Several months ago we needed to implement a billing system. Conceptually, recurring payments are simple – ask for a customer’s payment card, save it, and then process it at later dates through a payment gateway.

Implementing it, on the other hand, is tricky, all because of one thorny issue: security. Hosting our site on the EC2 cloud didn’t make PCI compliance matters easier either.

That’s when we found Braintree.

Braintree’s a relatively recent newcomer to the payment game. They’re young, and they’re growing fast. Since 2007, they’ve been picked up by hotshot Web 2.0 companies like 37signals, Github, Disqus.

Now I can’t say it enough: I. Heart. Braintree.

Their secret sauce? Excellent customer service, a clean API, and a little thing they like to call Transparent Redirect.

Here’s how it works.

A standard transaction model usually goes something like this,

  1. User enters credit card information
  2. Credit card information is sent to a secure server hosted by you
  3. Card is processed through the payment gateways’s API.

All the extraneous work is done in securing your own server. To bypass that, the other option is to redirect – albiet jarringly – to a 3rd-party site.

But here’s the cleverness of Braintree’s transparent redirect.

  1. User enters credit card information
  2. Credit card information is sent directly to Braintree’s server, bypassing your servers
  3. Braintree silently redirects to a callback you supply
  4. Your callback handles the credit card through encrypted tokens

The best part is that during this whole process users will never know they’ve left the site. And since you never look at or deal with sensitive information directly, the need to take any extra steps to secure your servers is minimal.

From a code level, here are the steps to process a transaction.

1. Encrypt transaction details

tr_data = Braintree::TransparentRedirect.transaction_data(
                   :redirect_url => "http://example.com/your_callback",
                   :transaction => {
                     :type => "sale",
                     :amount => "10.00"
                    })

2. Construct the form and embed the transaction details

3. Handle the callback

def your_callback
  result = Braintree::TransparentRedirect.confirm(query_string)
  if result.success?
    puts "Hoorah!"
  else
    puts "Oh FUBAR!!"
  end
end

That’s it. Simple eh?

They have APIs written for Ruby, Java, PHP, Phython, and .NET.

Did I also mention their excellent customer service? Check out how fast they reply!