Backstory

One job ago I worked on a web product that had a pretty good coverage of Selenium tests. Those tests would spawn a Firefox instance and, as such, tended to be slow. On my quad-core Mac Pro they would clock in at a bit over 25 minutes.

Htmlunit_logoA co-worker of mine saw that this was not ideal, time-wise, and set about fixing it. He pulled out Selenium and plugged-in HtmlUnit (via Celerity/Culerity), updated the tests and — BEHOLD! — the test suite ran in four minutes.

Totally. F’ing. Epic.

Fast-forward to now

Webkit_LogoI’m starting a new web product project and ,naturally, I have some full-stack testing in from the beginning. This time I use capybara-wekbit and all is well. For a while.

Then some warts appear: a case come up where capybara-webkit doesn’t quite render something the way Chrome, Safari,or IE do so although the tests pass, it doesn’t actually work in a real browser.  I spend some serious time reworking the code to make it work properly in capybara-webkit as well as the commodity browsers.

All is good again for a time, but then it happened once more. Again, it’s fixed.

The third time this happens, I stop and think about it:

This stuff MUST work in real browsers, and I usually end up testing that by hand. So what’s the point of running tests through this headless browser? Why am I sinking time in to making this work in a browser that, quite literally, NO ONE will actually use?

The revelation was made: the browser being used is part of the stack in full-stack testing.

And, as such, trying to avoid using those browsers directly is, then, avoiding testing part of your stack. A lot has been said that you should develop on the same system as production: same interpreter, same database, same OS. So why should your tests not follow the same logic? Why test using an abstracted browser that, functionally, is never really used by a human?

Full Stack

Unit-testing is a fair point: when you are unit testing JS, a headless browser is probably a good thing to use. It will be faster and (assuming ECMA compliance) just as good as a real browser. But unit-testing is not full-stack testing.

This was a turning point for me. Before that, I saw using Selenium as using a crutch. Now I see that using Selenium is the whole point. You are not just testing your code, you are also testing how your code interacts with the browser. To wit, the browser is in actuality part of your product.

Ok, so, fact: doing Selenium tests with one browser is slow. Doing then with multiple browsers is even slower. But the solution to this problem is not to remove the browser from the equation, is is to make the browser a manageable part of the equation. Farming, parallelism, concurrency, those are all viable ways to make the speed issue more manageable. But removing the browsers from the testing process is not.

Suppose you’re using the following command to interact with an iframe within capybara:

page.driver.browser.switch_to.frame(‘iframe’)

Sometimes it works, but often you get en error like:

Element is not currently visible and so may not be interacted with

You may be falling prey the retry methods built in to capybara. See, capybara is waiting for something to appear in the dom and keeps retrying failing code until it does.

But what you did, by changing scope with “#swich_to” is made it so a previous passing command is now failing. Technically, you’ve made only what is in the iframe visible to capybara, so when the retry happens, things that were previously there are now not visible; they are still in the DOM, but not in current scope, hence “not visible”.

Try this. In a “background” or “before(:each)” before the test in question:

background { @window_handle = page.driver.browser.window_handle }

..and then, before your assertions:

page.driver.browser.switch_to.window(@window_handle)

That will ensure proper DOM scope on the retries.

This is the first part of what I hope will be a pretty useful set of tutorials for those moving from the world of Perl in to Ruby. For the overview of topic I hope to cover in this series you can go back and look at the “Overview” page. In this first part I want first to talk about how Perl and Ruby are similar and then go over some of the most visible differences.

Continue reading

I originally gave a presentation with this same title at the February 2012 ut.rb meeting.  Although it was focused at the Perl programmers that were present at that meeting it was so well received I’ve decided to spend more time on it an develop it in to a series of blog posts.

Of course this is meant to give Perl programmers a foothold in a new language but it is also useful for anyone coming in to the language from somewhere else.

My Perl credentials

I started developing in Perl in 2000 and was a full-time Perlista from 2000-’07 using versions 5.005 to 5.6, but I’ve never used the Perl 6 series.  My first gig was doing financial code for a credit-card processing company, then I moved on to start a web site (Quizilla.com) that eventually went to many tens-of-millions of daily page views on Perl code I wrote.

Overview of what’s on tap

In this series I plan on covering:

  • Part 1: basic similarities and shared conventions. Differences between use of semicolons, and parenthesis.
  • Part 2: Bagging on Perl’s OO system.  Objects and methods under Ruby.  Symbols.
  • Part 3: Perl sigils counterparts in Ruby.  New meaning of $, @ and @@.  Constants in Ruby.
  • Part 4: String interpolation, deliberate/implicit returns on methods.
  • Part 5: Bang (!) and Boolean (?) methods.
  • Part 6: blocks (very basically).
  • Part 7: bonus Rails-specific content: method_missing() and “magic methods”.

When /^I reload the page$/ do
# puts "DEBUG - The page for #{current_path}"
case Capybara::current_driver
when :selenium
visit page.driver.browser.current_url
when :racktest
visit [ current_path, page.driver.last_request.env['QUERY_STRING'] ].reject(&:blank?).join('?')
when :culerity
page.driver.browser.refresh
else
raise "unsupported driver, use rack::test or selenium/webdriver"
end
end

By default, when using celerity and celerity, there is no handler for file downloads. If your action in your scenario results in being served a file with a mime type you don’t understand it will end up being unceremoniously lost.

You can’t read “page.body” because that will return the page you are on, not the file you want to download.

You have a link to the action that will result in the file getting served to you:


When /^I press "([^"]*)" and download the file$/ do |element_id|

@file = page.find_by_id(element_id).click.getWebResponse.getContentAsStream.to_io.read

end

That will read the results of your click in to the @file variable that you can then parse in later steps.

Today I put out the first preliminary version of a gem I call ‘validate_block’. It comes from a discussion I had with @jakemallory yesterday where I said I wish I could logically scope the ‘validates_*’ commands in an ActiveRecord object.

It didn’t seem too hard and so last night I threw something together and it’s up on Rubygems.org now.

The main idea is that this gem will let you take a block with a lot of duplication like this:


validates_presence_of :hair, :unless => :bald?
validates_length_of :hair, :within => 3..15, :unless => :bald?
validates_inclusion_of :hair_color, :in => HAIR_COLORS, :unless => bald?

..and convert it in to this:


validate_block :unless => :bald? do
  presence_of :hair
  length_of :hair, :within => 3..15
  inclusion_of :hair_color, :in => HAIR_COLORS
end

As you can see, it eliminates the need to prefix each item with ‘validate_’ and adds the :unless condition to the whole group (you can put any AR Validation options in there).

Check it out at github. That is all.

You may have a point when you’re writing specs that are meant to anticipate a 404 Not Found from a resource. You have a rescue like this:

begin

rescue ActiveResource::ResourceNotFound
return nil
end

..and now you need a test for it. In Rspec it’s not obvious, but easy. First, you need to stub your actions in your spec:

Something.stub!(:some_method).and_raise(ActiveResource::ResourceNotFound)

That sounds good, but it won’t work because it will fail with:

ArgumentError: wrong number of arguments (0 for 1)

Arguments? I need to pass arguments to an error? I guess I do. Maybe the status code?

ActiveResource::ResourceNotFound.new(404)

But no:

NoMethodError: undefined method `code’ for 404:Fixnum

Ah ha! It needs some kind of object passed in to new()! But, what is it? We have no way of knowing off hand what object should be there. Wait, though, we’re INSIDE an environment that exists for making throw-away objects!

Something.stub!(:some_method).and_raise(
ActiveResource::ResourceNotFound.new(mock('err', :code => '404'))
)

Try that, you’ll get your exception!