Using Selenium WebDriver with PhantomJS on a shared host

I’m currently trying to set up a cronjob to do end-to-end autotesting of the OAuth code I wrote for Tandem Exchange, which is occasionally failing these days due to the OAuth provider systems being unavailable or very very slow (which is the same thing). It’s actually one of the biggest problems in web authentication actually, the fact that once you do start relying on OAuth providers to authenticate users on your website, you inherit a critical dependency on their servers always being reachable. (In our case, I’ve been seeing stupid downtimes by QQ, one of the largest Chinese social networks.) And it’s not something you can test with mock objects.

Set up a virtualenv:

Install Selenium in the virtualenv:

Download and untar the PhantomJS executable somewhere on the PATH:

During the first attempts at using PhantomJS, I had problems with SSL-based addresses always returning a .current_url of “about:blank“, which you have to fix using a somewhat obscure flag in the webdriver.PhantomJS() constructor.

The fix looks like this, in Python:

And when the unit test runs (via nosetests):

For the most part the Python-based selenium module works, but it is pretty verbose, as it sticks to the original Webdriver API very closely. There are higher-level abstractions, such as Splinter, but I get the impression that making sure it starts PhantomJS properly will be an ordeal in and of itself. I’ve gotten Facebook and Google OAuth testing working headlessly, which is pretty cool, but the next step of getting the QZone OAuth test working is getting jammed up on the fact that QZone is behaving exactly as users see it behave (which is to say, problematically).

But then, that’s the objective of the unit testing, to reveal the source of the latest OAuth problems.

Actually, I’ve also noticed that it’s pretty slow not only from the shared-hosting server we’re using, but also from where I am currently located. So two different points on the globe. I can’t help but wonder how much the Great Firewall of China is slowing things down.

Chrome USB Web Debugging Is Awesome

I’ve been doing a little bit of tuning for mobile devices on some web work that I’d been doing, and wanted to give the Chrome USB Web Debugging a try. For those living under a rock for the past two years, Google added the ability to live-debug websites on mobile devices using the standard Android Debug Bridge and a copy of mobile Chrome. This means you can debug your websites using a real browser, running on real Android tablets and phones. And, because most mobile browsers are based on some form of Webkit (or Blink), this should lead to reasonable market coverage.

In this case, I tested it using my Nexus 7, by activating the option in mobile Chrome:


After installing the ADB plugin for Google Chrome on the Desktop, the following extension icon shows up in the browser window:


Once you’ve started ADB via the plugin menu, you can then “View Inspection Targets” and the following list of inspection targets will open up:


Inspecting the Chrome instance on the Nexus 7 device will open up the familiar Developer Tools window, acting on the mobile device target:


Inspecting a DOM node on the desktop, highlights it on the mobile device, like you’re used to:


The great thing is that the debugging session is seamless. You are put in full control of an interactive debugger, all of the CSS inspection and modification is there, you can modify anything you want the Javascript console, set breakpoints, inspect, modify, and continue running stopped code. It all just works, which is pretty much the nicest thing you can say about a piece of software (and incidentally the hardest thing to achieve in software engineering).

In short, it’s a great way to code, debug, and optimize a website using a real mobile web browser.