notatechblogofmarc: 2012

Tuesday, September 18, 2012

x-tags!

Browser technology advanced a lot in recent years. The hype about HTML5 spotlighted how much browsers improved. Still, HTML5 is a low level technology compared to other UI toolkits common in languages like .NET, Java and Flex. These are much more component based providing big building blocks for developers to assemble applications quickly. Sure, there is extjs and several others libs providing equally rich component sets made on top of low-level HTML/JavaScript/CSS.

Anyway, with the Web Components spec currently in the works of standard bodies and browser makers, HTML5 is going to take a next step towards a full-fledged RIA platform. Web Components roughly includes templates, decorators and custom tags. Rather a technical implication than a feature is the shadow DOM which divides the DOM in initial content and shadow elements.

For custom elements there is already a working implementation available. At x-tags.org Mozilla provides an polyfill in JavaScript which is on standards track. The strategy is obvious: create a specification of a feature, provide polyfill implementation for early use which finally can be replaced by a native implementation. Where this is possible, this is a good plan! It also helps to get early feedback on something like a 'reference implementation'. So how does this x-tag thingy work in use? It's pretty simple. See how to create a map:

<head>
<link rel="stylesheet" href="/x-tags/map/map.css"/>
<script src="/x-tags/x-tag.js"> </script>
<script src="/x-tags/map/map.js"> </script>
<style>
#detail-map {
box-sizing: border-box;
width: 100%;
height: 480px;
}
x-map {
border: 1px solid #black;
}
</style>
</head>
<body>
<x-map id="detail-map" data-key="xxx"></x-map>
</body>

The code samples shows how to include the xtag library providing support for all browsers including IE9 and better (so it completely safe for the mobile web). The map.js script registers a custom element and defines form and behavior. Once registered, instances of the element are recognized in the DOM and initialized accordingly. The web component spec defines a couple of life cycle events for custom elements (like created, inserted or attributeChanged). This way logic shipped with a custom element can be hooked in and do its augmenting work.

These technical details are basically interesting for component or framework developers only. An application developer does not have to fiddle around often with these details. More often he is interested in a set of components to quickly wire up an application. That's what x-tags is meant for. At x-tags.org you will find a JavaScript implementation of x-tag which ships with a bunch of nice components. I built a first FirefoxOS application with these and found them very handy. There is an x-slidebox and an x-flipbox element with which I quickly was able to construct a basic navigation behavior for my application in a declarative way.

<x-slides>
<x-slide>
<x-map id="map" data-key="xxx"></x-map>
</x-slide>
<x-slide>
<x-flipbox class=" x-flip-direction-right">
<x-card>
<div>Front side</div>
<div>Back side</div>
</x-card>
</x-flipbox>
</x-slide>
</x-slides>

Here again we have to include the JavaScript and CSS files in the header or at the end of the document:

All in all x-tags or better custom elements are a promising feature of HTML5 to make development of large applications and library integration more convenient. Thanks to JavaScript polyfills, it can be used much earlier than before all browser support it natively. I'm quite sure JavaScript libraries will jump on that train once available because it is easy to do and it offers a nice way to integrate third-party behavior in a declarative and semantic way.

See a short slide deck about Mozillas x-tag polyfill or jump to the demo page.

Sunday, September 16, 2012

Developing and deploying a FirefoxOS application

At the MozillaCamp2012 in Warsaw I came upon the x-tag implementation of Mozilla. Eager to try some x-tags and the Mozilla implementation I started a first FirefoxOS application. While I currently have no hardware device on which I can run b2g applications, we can find other ways.

While having a hardware device is essential to develop and test an application, it is a nice asset of a platform to have other ways to run an application. While developing a FirefoxOS application, the turnaround can be extremely quick for a great part of the app. When e.g. it comes to fiddle around with the UI to make it beautiful and shiny it's a bless to be able to simply reload the app in a browser and having all the beloved development tools available for debugging, adjusting styles and the like. Once it fits after a dozen or so development cycles, it can be deployed and tested on the device again.

For FirefoxOS aka b2g aka boot2gecko applications there are many ways to be run while development:

(re-)loaded in a web browser
(re-)installed on an FirefoxOS emulator running on a desktop computer
(re-)installed on a FirefoxOS mobile device

While reloading the app in the browser is the fastest, browsers lacks decent support for Web APIs so far. Anyway, having these three ways available boosts development in many regards.

Possible productive deployment platforms are:

a mobile phone device running FirefoxOS (not commercially available yet)
a mobile tablet device running FirefoxOS (same as phone)
a sandboxed browser (Web API support required)
a desktop application (e.g. deployed via FirefoxAurora)

All ways have their own appeal and taken them all together with a single app is great.

Mozilla provides a product line demonstrating the ability of doing this right now. As it is fully based on standards or standard proposal it is more than a proprietary FirefoxOS or Mozilla application. It's a convincing proposal on how web application technology can be leveraged on every platform with a similar degree of integration into the hosting device or platform as native counterparts or even run directly on a mobile and web enabled linux kernel like b2g. There might be never a full "write-once, run-everywhere" for high-class, high-fidelity apps, but given support for standardized, installable HTML5 apps on desktop and mobile is a great asset, pushing competitors further in many regards.

See this YouTube video with the screencast on how I deployed and run an web application in a web browser, the firefoxOS phone device emulator and deployed as a desktop application on OSX:

Saturday, February 25, 2012

Navigating the web programmatically with Phantom.js

Phantom.js is a headless webkit browser which let's you navigate the web in a programmatic manner from the command line. A JavaScript API allows to control the browser for example to load a web page, check its contents or run JavaScript within the page. Obviously this is a powerful tool for many purposes. Running test cases comes to mind quikly.
I created a first example to load a qUnit test case, run the test and, after a short delay, grab the test result from the DOM elements:

var page = new WebPage(),
    url = "http://localhost:3000/repo/test/model/model.qunit",
    delayInMillis = 200;

page.onLoadFinished = function (status) {
    
    window.setTimeout(function() {
        
        var result = page.evaluate(function () {
            return {
                passed: document.querySelector("#qunit-testresult .passed").innerText,
                failed: document.querySelector("#qunit-testresult .failed").innerText,
                total: document.querySelector("#qunit-testresult .total").innerText
            };
        });
        
        console.log(JSON.stringify(result));
        phantom.exit();
        
    }, delayInMillis);
};

page.open(url); 

The sample creates a WebPage object and registers a listener for the loadFinished event. The page is loaded by simply call page.open(url). As soon as the page has been loaded completely the handler is called. To get the results of the qUnit test a functions is evaluated. This function is called within the page just loaded so it can access the DOM to get the test result from the DOM as rendered by qUnit. To make sure all tests have finished at the time the script evaluates the DOM, a timeout is used to delay the action.

Once phantom.js is on the PATH the still naive script above can be called from the command line:

# phantomjs load-qunit.js

The ability to inject JavaScripts into pages is very powerful. It makes integration testing of a web application quite easy as we can inject scripts to emulate user inputs triggering events. Afterwards the resulting action can be asserted by checking the changes in the DOM. This adds another possible testing layer above unit and component testing which could be done with qUnit alone. It's not cross browser, but it's kind of lightweight and fast and requires less setup then Selenium or JsTestDriver. A good tool well suited to be run on both a developer workstation or an integration test server.

While being headless (there is still a dependency to X on Linux; see work around) Phantom.js is still able to render pages into a bitmap. This makes it easy to create a detailed test report especially when something went wrong. A rendered picture of the failing page can support debugging and complete other data which can be drawn from the DOM in case of a failing test.

It's dead easy:

var page = new WebPage(),
    url = "http://localhost:3000/repo/test/model/model.qunit",
    delayInMillis = 200,

    screenshotFile = "failedTest.png"
    viewportWith = 1024,
    viewportHeight = 1000;

page.viewportSize = { 
    width: viewportWith, 
    height: viewportHeight 
};

page.onLoadFinished = function (status) {
    
    window.setTimeout(function() {
        
        var result = page.evaluate(function () {
            return {
                passed: document.querySelector("#qunit-testresult .passed").innerText,
                failed: document.querySelector("#qunit-testresult .failed").innerText,
                total: document.querySelector("#qunit-testresult .total").innerText
            };
        });
        
        console.log(JSON.stringify(result));



        if (result.failed > 0) {

            page.render(screenshotFile);

        }



        phantom.exit();
        
    }, delayInMillis);
};

page.open(url); 

As demonstrated Phantom.js allows for automating many tasks done with a webbrowser. Not only testing but many other possible applications of Phantom.js can be very useful for web developers. It's obvious that testing with Phantom.js is not enough for the mainstream web as here cross browser testing is a must and solutions like Selenium are available.

Installation on OSX was as easy as downloading and extracting an archive file. Examples shipped with the distribution and the official documentation allowed for a quick jump start. So Phantom.js is another usefule webkit application which brings the web and it's technology out of the usual browser environement. Try it out!

Screenshot taken with Phantom.js after qUnit test has run:

Tuesday, February 14, 2012

Using jslint

JSLint and JSHint are tools which help to avoid bugs related to some JavaScript oddities. Linting my JavaScript code has become common for me and I found some issues in my code which I would not have found without it. It also helps refactoring code. For instance when moving code to a separate file, JSLint tells me about variables and functions I want to use in the script but are still in the old file. I don't have to run a test case to find the bug. The following option helps me out in this particular case:

undef: false # true if variables and functions need not be declared before used.

On OSX I use Textmate for JavaScript code. JSLintMate is a bundle for Textmate which integrates JSLint and JSHint into the editor and fires it whenever I save a JavaScript file. So, customizing my lint options has become an issue quickly. At the end of this post you can find my current JSLint options which are stored in the .jslinrc file in my home directory and hence are passed to JSLint by default.

Within a JavaScript file I can put additional instructions in comments. The jslint instruction can be used to pass options overriding or completing the default options:

/*jslint browser: true, devel: true */

The above tells JSLint to include global variables for the browser environment and allow calls to development tools like the console.

Other globals can be added by the global instruction:

/*global jQuery: true, craftjs: true */

This can be very useful to tell JSLint which globals are provided by third-party libraries as jQuery in the snippet above. Besides libraries there might also be globals created by my own scripts in other files which are linked into the page or prepended by a build tool like sprockets or craft.js.

Besides JSLint, JSHint becomes more and more popular. The mentioned craft.js I'm currently working on uses JSHint. Both roughly offer the same functionality. JSHint offers additional options and hence offers more control over the lint process. The JSLint instructions within the JavaScript comments are supported by JSHint as well. That's why JSLint and JSHint can be used side by side without cluttering the source code even more.

Conclusion

Being used of using IDE features for languages like Java makes a JavaScript developer feel lonely. Not only can JavaScript IDEs not offer such a rich feature set like Java IDEs can. JavaScript even does not have a compiler which tells a developer about typos and other mistakes. JSLint/JSHint can help to some extend by providing instant feedback when writing JavaScript code and supports the endevour of delivering high-quality JavaScript code.

my current jslint default options

adsafe: true # true if ADsafe rules should be enforced. See http://www.ADsafe.org/
bitwise: false # true if bitwise operators should be allowed.
cap: false # true if upper case HTML should be allowed
css: false # true if CSS workarounds should be tolerated
debug: false # true if debugger statements should be allowed (set to false before going into production)
eqeq: false # true if the == and != operators should be tolerated.
es5: false # true if ECMAScript 5 syntax should be allowed
evil: false # true if eval should be allowed
forin: false # true if unfiltered 'for in' statements should be allowed
fragment: false # true if HTML fragments should be allowed
indent: 4 # Number of spaces that should be used for indentation - used only if 'white' option is set
maxerr: 50 # The maximum number of warnings reported (per file)
maxlen: 120 # Maximum line length
newcap: false # true if Initial Caps with constructor functions is optional.
nomen: true # true if names should not be checked for initial or trailing underbars.
on: false # true if HTML event handlers (e.g. onclick="...") should be allowed
passfail: false # true if the scan should stop on first error (per file)
plusplus: true # true if ++ and -- should be allowed
regexp: false # true if . and [^...] should be allowed in RegExp literals.
safe: false # true if the safe subset rules are enforced (used by ADsafe)
sloppy: true # true if the ES5 'use strict'; pragma is not required
sub: false # true if subscript notation may be used for expressions better expressed in dot notation
undef: false # true if variables and functions need not be declared before used.
vars: false # true if multiple var statement per function should be allowed.
white: false # true if strict whitespace rules should be ignored.

predef: '' # Names of predefined global variables - comma-separated string or a YAML array

browser: false # true if the standard browser globals should be predefined
rhino: false # true if the Rhino environment globals should be predefined
windows: false # true if Windows-specific globals should be predefined
widget: false # true if the Yahoo Widgets globals should be predefined
devel: true # true if functions like alert, confirm, console, prompt etc. are predefined
node: false # true if the node.js environment globals should be predefined

Thursday, February 2, 2012

Leveraging node.js for unit testing in JavaScript development

I played around with node.js to find a way to properly unit test my JavaScripts. While doing that I stumbled upon a nice feature node.js offers by the vm module. The function

vm.runInNewContext(script, globalObject, filename);

allows to run a script and pass a context to it. This context is nothing else than the evil global object of JavaScript we all know about and work around. So the above code allows us to run a script like the one below which is stored in test.js and has two dependencies to the global object:

(function () {

    "use strict";

    var renderer = require("./renderer");

    assert.ok(renderer);

}());

By using runInNewContext we can now load the test.js script and run it with an empty context. This is done by test-runner.js:

(function () {

    "use strict";

    var vm = require("vm"), fs = require("fs"),

        filename = "app/test.js",

        testScript = fs.readFileSync(filename).toString(),

        globalObject = {};

    vm.runInNewContext(testScript, globalObject, filename);

}());

Now we can run the test runner with node:

#  node test-runner.js

To get an error saying:

ReferenceError: require is not defined

This is exactly what we expected because the globalObject passed to runInNewContext is empty and hence the context has no require to offer as we are used to in a node.js environment. However, being able to pass our own global object, we can mock dependencies the test requires. This is exactly what we need to mock away our collaboraters and unit test our modules in an isolated manner. In Java, libraries like EasyMock do some magic to provide mocks for interfaces and classes. A great and very useful tool for test driven development in Java. In JavaScript mocking a function is a snap compared to that. The dynamic type system and the functional approach with late binding makes it easy to mock a function or an entire object with literals. Hence the test-runner.js script needs to setup the globalObject properly to provide an appropriate global object on which test.js can rely on:

The above script now offers a require function as a property of the global object. It simply returns an empty object which is enough to fulfill our test case. Now our test script fails a liitle bit later because it misses the assert function to check the imported module. In this case we don't want to reimplement the assert function node.js already offers. Instead we just pass the real implementation:

globalObject = {

     assert: require("assert"),

     require: function(module) {

            return {};

}

}

Running the test runner passes now without complaint, because we successfully mocked all dependencies for our test case.

The exampled I showed here is very simplicistic. But it clearly shows the opportunity the runInNewContext function of the vm module offers for unit testing JavaScript code. We can not only mock the global require function, but also other global properties as for instance the document or window properties in a browser or a jQuery Ajax call $.ajax({}).

While it realy sounds like a big burden to mock away an entire browser environment, the impact of such an development style on JavaScript code would be huge and raise not only quality and testability but also leads to flexible reusable modules, objects and functions. While I have not applied such an development approach for JavaScript by now, my experiences with TDD in Java makes me sure this would be a good idea for JavaScript as well. Not having Eclipse and its great refactorings for Java (a kindom for Ctrl+1!!) makes it not that easy and powerful, but the benefits make it worth doing it.

As shown above, node.js in general and vm.runInContext in particular offer a nice starting point for such an endeavour.

notatechblogofmarc

follow me on twitter