iMacros enhanced using Javascript

Introduction

The other day, I had to automate interactions on a web page, for testing purposes. I stumbled across the iMacros Firefox extension, which turned out to be very mighty.

Basics

With iMacros you can automate sequences of events on web pages in order to test them in terms of behavior and/or performance. The extension has its own scripting engine and macro based language, which looks as follows:

TAB OPEN 
TAB T=2
URL GOTO=http://www.google.com

Which basically pops up a new tab and opens the specified URL in it. It’s possible to simulate key press or mouse events. You can simulate clicks on certain page elements, which can be selected by html attributes or xpath expressions, for instance. Plus, it’s also possible to set variables and pass extracted page contents to them. The complete command reference can be found here.

However, the code is pretty static. There is no native language element to insert loops in the code. The firefox extension itself has both a single play and a loop play button, where the latter simply invokes your script several times. Then there’s a variable called {{!LOOP}}, which contains the current loop iteration as an integer. This is as far as you can get with the builtin language.

But, and this is where it gets really nice (and also nasty), the scripting engine of iMacros can execute Javascript. They introduced a bunch of Javascript calls, which kind of act as a language binding. All the Javascript calls can be found here. By using Javascript, the example above can be rewritten as follows:

var macro = "CODE:";
macro += "TAB OPEN\n";
macro += "TAB T=2\n";
macro += "URL GOTO=http://www.google.com\n";
iimPlay(macro);

So you see, this enables us to execute those iMacros commands from Javascript. This means, we can use loops, variables, arrays, conditionals and so on. By using the {{!EXTRACT}} variable inside iMacros code and the iimGetLastExtract() method, it’s even possible to pass values from iMacros to the Javascript world. So we could extract content from web pages and process them in Javascript.

Since all the ugly string operations like the += style concatenation and the \n line endings, I wrote a little helper class for that, the MacroBlock:

class MacroBlock {
    constructor() {
        this.code = 'CODE:';
    }
    add(line) {
        this.code += line + '\n';
    }
    play() {
        iimPlay(this.code);
    }
    lastExtract() {
        return iimGetLastExtract();
    }
}

With these tools on hand, I was able to perform a lot of my tasks. As always, there are limitations, which required special treatment and I want to share my solutions with you.

Logging

I was sick of the annoying popup dialog created by alert(). Firefox has a very nice Javascript console builtin to their developer tools. However, I was not able to use that in my script and unfortunately, I don’t know why. According to this page, the following code should work:

Components.utils.import("resource://gre/modules/Console.jsm");
console.log("Hello from Firefox code"); //output messages to the console

But it doesn’t. But I found a way to use a log file:

function log(msg) {
    var m = new MacroBlock();
    m.add('SET !EXTRACT "'+datetime+" | "+msg+'"');
    m.add('SAVEAS TYPE=EXTRACT FOLDER="/tmp" FILE="imacros.log"');
    m.play();
}

Which allows you to do a $ tail -f /tmp/imacros.log and you get a console’ish output. The function above uses the SAVEAS command of iMacros.

Modularity

At a certain point, my desired functionality was complete, but the code was only structured by functions. I wanted to create modules and include them in other scripts. The good news is: there is a way. The bad news is: it’s only working in the firefox variant of iMacros. We can use Firefox’ internal XUL engine for that as follows.

Let’s assume, you want to create a module with a class named MyClass. Create a file called my_class.jsm with the following content:

var EXPORTED_SYMBOLS = ['MyClass'];
class MyClass {}

With the EXPORTED_SYMBOLS variable, you need to tell the XUL engine, which symbols you want to export/make available. Also notice, that the module has the file extension *.jsm!

Now, in one of your main scripts, you can include the module above as follows. Assume you have a script called main.js with the following content:

Components.utils.import("/path/to/file/my_class.jsm");
var obj = new MyClass();

That’s pretty sweet, isn’t it!?

Tabs

The iMacros engine has a TAB command. However, I had problems with that. Since I call iimPlay() several times between opening and closing a tab, the iMacros engine will lose knowledge over the tabs and you will not be able to reference tabs by their index (TAB T=nn) in subsequent calls. So I decided to simply use keyboard shortcuts the browser offers to open/close a tab. Namely CTRL+T, CTRL+W, CMD+T (OSX) and CMD+W (OSX). The functions look as follows:

function openTab() {
    var m = new MacroBlock();
    m.add('EVENT TYPE=KEYPRESS SELECTOR=* CHAR="t" MODIFIERS="[CTRL]"');
    m.play();
}

Keyboard shortcuts/OS detection

I originally wrote some scripts on a linux machine and the openTab() method above, using the keyboard shortcuts, worked just fine. When I continued working on the scripts on my mac (OSX), I realized that the method is not working any more. That was due to the fact that the shortcuts on OSX use different modifier keys, namely META instead of CTRL. So I decided to implement some sort of OS detection. Here it is:

function detectOS() {
    var l = new MacroBlock();
    l.add('SET !EXTRACT {{!URLCURRENT}}');
    l.play();
    var remUrl = iimGetLastExtract();

    var m = new MacroBlock();
    m.add('URl GOTO="http://www.whatsmyua.com/"');
    m.add('TAG POS=1 TYPE=A ATTR=ID:"ua-link" EXTRACT=TXT');
    m.play();
    var ua = iimGetLastExtract();

    if (ua.includes("Macintosh")) {
        config.os = "osx";
    } else {
        config.os = "non-osx";
    }

    var k = new MacroBlock();
    k.add('URL GOTO="'+remUrl+'"');
    k.play();

    return ua;
}

It’ll use one of the classic what is my user agent pages to detect the OS, extract the user-agent string and set a flag in a config object that can be used later on.