Launching iOS and Android simulators with Node.js

I wanted a script to open iOS and Android device simulators and open their browsers to a URL (my karma.js test URL) so wrote one in node.js, using child_process to execute bash commands to get the job done.

It’s far from perfect and will probably explode if you don’t have devices already installed, but it’s more than suitable for my needs. This is a post for self reference more than anything.

It:

  • Gets the list of all available iOS devices and picks the fourth one in the list (position 3 in the array generated from the text output) (at the time of writing, on my system this is the iPhone 5)
  • Executes the command to load the selected iOS emulator
  • Gets the list of all available Android AVDs (Android Virtual Devices) and selects the first one (unlike iOS devices through xcode, AVDs have to be manually installed in Android Studio so it can be a pain if you don’t have more than 1 set up)
  • Executes the command to load the AVD
  • Starts a loop to check if the AVD has loaded yet – it does this by checking the AVD’s bootanim property. If this is ‘stopped’ we know the device has loaded (I couldn’t find a better way of detecting device load. If anyone has one, please let me know)
  • Once the Android emulator has loaded – we can assume the iOS emulator has also loaded as the AVD always takes much longer to load – we execute the command to unlock the screen on the AVD.
  • We execute the commands to launch the browser to our required URL.
var process 		= require('child_process'),
	url 			= "http://192.168.10.126:9876",
	iosdevice, bootChecker;
 
process.exec('xcrun simctl list', function(error, stdout, stderr){
	var splitRows = stdout.split("\n");
	iosdevice = splitRows[3].split(" (")[0];
	launchIOSDevice();
});
 
process.exec('~/Library/Android/sdk/tools/android list avd', function(error, stdout, stderr){
	var splitRows = stdout.split('\n'),
		deviceObj = {};
	// console.log(splitRows);
	splitRows.forEach(function(row){
		var splitRow = row.split(':');
		deviceObj[splitRow[0].replace(/\s/g, '')] = splitRow[1];
	});
	console.log(deviceObj.Name.replace(/\s/g, ''));
 
	process.exec('~/Library/Android/sdk/tools/emulator -avd ' + deviceObj.Name.replace(/\s/g, ''));
 
	isAndroidEmulatorBooted(function(response){
		if (!response){
			kickOffBootChecker();
		} else {
			unlockAndLoadURL();
		}
	});
 
});
 
function launchIOSDevice(){
	console.log('xcrun instruments -w "'+iosdevice+'"');
	process.exec('xcrun instruments -w "'+iosdevice+'"', function(error, stdout, stderr){})
}
 
function kickOffBootChecker() {
	bootChecker = setInterval(function(){
		isAndroidEmulatorBooted(function(response){});
	},1000);
}
 
function isAndroidEmulatorBooted(callback){
	process.exec('~/Library/Android/sdk/platform-tools/adb shell getprop init.svc.bootanim', function(error, stdout, stderr){
 
		if (stdout.toString().indexOf("stopped")>-1){
			clearInterval(bootChecker);
			unlockAndLoadURL();
			return callback(true);
		} else {
			console.log('loading Android emulator ...');
			return callback(false);
		}
	});
}
 
function unlockAndLoadURL(){
	//unlock the device
	console.log('unlocking the android emulator');
	process.exec('~/Library/Android/sdk/platform-tools/adb shell input keyevent 82');
 
	//gotourl
	console.log('launching the url on android');
	process.exec('~/Library/Android/sdk/platform-tools/adb shell am start -a android.intent.action.VIEW -d ' + url);
 
	console.log('android emulator takes longer to load so we can be sure the ios emu has loaded. launch url there too');
	process.exec('xcrun simctl openurl "'+iosdevice+'" ' + url);
}

Running Karma on iOS and Android Simulators with Appium

At my day job I’ve been tasked with improving our webapp testing strategy. This need came about after a bug was introduced in one of our webapps and not picked up by our current testing process. The bug was related to inconsistencies in the implementation of the Javascript date API between iOS Safari and Android Chrome browsers.

We currently employ a (pretty loose) TDD workflow for our Backbone webapps. Writing our specs with Jasmine and using Grunt as our task runner. This is all integrated into our Jenkins CI. The Jasmin specs are all run though an instance of Phantom in Jenkins via the (one of many) grunt task(s) when we push our code up and release though Jenkins. The build either passes or fails.

From investigating all the various options and frameworks, all signs pointed to Karma. Karma would give us:

  • Seamless integration into our current work flow
  • Immediate feedback – you write a test for a new piece of functionality, hit save and it immediately tells you it’s failed. Write the functionality, hit save and it tells you right away if it passes your test.
  • Testing on multiple real devices

It was a fairly painless process to set Karma up using the official docs and get my tests running on the browsers on my dev environment (Safari, Chrome, Canary, Firefox, Phantom) and there are plenty of resources online for this so I won’t go into this here. What I did have issues with, especially with the lack of resources online, was running my specs on mobile devices.

I came across an iOS launcher plugin that had looked promising to get me going with iOS Safari but it’s not been touched by the developer in a couple of years and after a bit of experimentation discovered that doesn’t appear to work with node versions > 0.10. At the time of writing, I’m working with 0.12.

More research led me to Selenium, a browser automator. From there, I came across Appium which uses the same WebDriver protocol as Selenium but also enables you to automate mobile browsers. Download the Appium desktop app here

To interact with Appium, there is the Karma webdriver launcher (npm install karma-webdriver-launcher).

The idea is pretty simple (but the config is a little fiddly):

  • Make sure you have xcode with iOS simulator installed and the Android Development Studio with at least 1 AVD (android virtual device) set up.
  • Fire up the Appium desktop app to start an Appium server on the machine with the virtual devices (a mac for iOS testing and windows/linux/mac for Android testing). This can be the same machine as you run your Karma instance or a machine accessible by it.
  • Configure your karma.conf.js file to let it know where your virtual devices are
  • Run your tests and Karma will pipe the tests out to the Appium server with the WebDriver API. Appium will then fire up the emulators and run the tests in the specified browsers.

karma.conf.js

Karma will launch the browers to localhost:9876 (or whatever other port you set it to) so if running your Karma instance on a different machine to your test machine, you will want to let your emulated browsers where to go to run the tests. Obviously localhost on the test machine is of no use if localhost there doesn’t have your Karma instance. You can simply do this by specifying the hostname in your karma.conf.js. ie:

hostname: '192.168.10.126',

You will need to declare your custom launchers for your emulators too. In my case, this looks like this:

customLaunchers: {
    'iOS-Safari' : {
        base: 'WebDriver',
        platformName: "iOS",
        deviceName: 'iPhone 5',
        config: webdriverConfig,
        browserName: 'Safari',
    },
    'Android' : {
        base: 'WebDriver',
        platformName: "Android",
        deviceName: 'Android',
        config: webdriverConfig,
        browserName: 'Browser',
    }
},

webdriverConfig just contains the IP and port of the Appium server. In my case:

var webdriverConfig = {
    hostname: '192.168.10.126',
    port: 4723
}

Here is my complete karma.conf.js:

// Karma configuration
// Generated on Thu Aug 06 2015 11:58:03 GMT+0100 (BST)
var webdriverConfig = {
    hostname: '192.168.10.126',
    port: 4723
}
module.exports = function(config) {
 
  config.set({
 
    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: '',
 
    browserNoActivityTimeout: 1000000,
 
    // frameworks to use
    // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
    frameworks: ['jasmine', 'requirejs'],
 
    // list of files / patterns to load in the browser
    files: [
      'test/test-main.js',
      {pattern: 'environment-config.js', included: false},
      {pattern: 'environment-config.json', included: false},
      {pattern: 'js/**/*.*', included: false},
      {pattern: 'test/jasmine/spec/**/*.spec.js', included: false},
    ],
 
    junitReporter: {
        outputDir: 'reports/karma',
        outputFile: undefined,
        suite: ''
    },
 
    // list of files to exclude
    exclude: [
    ],
 
    // preprocess matching files before serving them to the browser
    // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
    preprocessors: {
    },
 
    // test results reporter to use
    // possible values: 'dots', 'progress'
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
    reporters: ['progress', 'junit'],
 
    // web server port
    port: 9876,
 
    hostname: '192.168.10.126', //points to accessible Karma server
 
    // enable / disable colors in the output (reporters and logs)
    colors: true,
 
    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_INFO,
 
    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: true,
 
    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['iOS-Safari', 'Chrome'],
 
    customLaunchers: {
        'iOS-Safari' : {
            base: 'WebDriver',
            platformName: "iOS",
            deviceName: 'iPhone 5',
            config: webdriverConfig,
            browserName: 'Safari',
        },
        'Android' : {
            base: 'WebDriver',
            platformName: "Android",
            deviceName: 'Android',
            config: webdriverConfig,
            browserName: 'Browser',
        }
    },
 
    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: true
 
  })
}

Appium Android Config

Screen Shot 2015-08-20 at 16.34.33

 

You can get the device name and platform version of your installed AVDs with the command:

~/Library/Android/sdk/tools/android list avd

 

This will give you something that looks like this:

Screen Shot 2015-08-20 at 16.39.37

Nothing changed in advanced settings except you must put in your Android SDK path so Appium knows where to find the SDK.Screen Shot 2015-08-20 at 16.34.50

Appium iOS Config

Screen Shot 2015-08-20 at 16.34.58

Appium General Config

The server address here is the address and port of the Appium instance (note in my example, Appium and Karma are running on the same machine under different ports but as mentioned above, you can have the Appium server running on an entirely different machine as long as it is accessible by the Karma service)

Screen Shot 2015-08-20 at 16.35.05

Next steps

I’ve not yet solved the running of iOS and Android tests simultaneously on the same machine but I am currently trying to solve this by setting up a Linux VM on the test machine to run the Android simulator so I can pipe to 2 machines effectively and test on both at the same time.

Also to tackle is running the tests on Chrome on the Android simulator. Currently I’m only testing on the “stock” browser. Desktop Chrome tests will most likely cover me in every single instance, though for completeness I want to get this solved.

Error on “npm install gulp –save-dev”

Using a PC at work so unable to reap the benefits that I get from CodeKit 2 on my mac at home, I’ve mean meaning to get Grunt or Gulp into my workflow.

Always one to jump on a bandwagon, I decided to choose Gulp over Grunt as the newer task runner seems to be gaining traction over the warthog. A sucker for a logo, I also prefer the Gulp logo.

Starting a new project this afternoon, I decided to spend the final hours of the working week getting set up and accustomed to Gulp.

Our dev server at the office is running Ubuntu 11.10 so it was a bit of a struggle updating dependencies to actually get node.js installed. A bit of brainstorming with Sam (read: Sam, the unix genius, took the reins did it for me) and node was installed.

Following the setup instructions I hit a hurdle when trying to install Gulp to my project folder. With the command:

npm install gulp --save-dev

I was getting a lengthy error:

npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data
npm WARN package.json [email protected] No README data
npm ERR! Error: EACCES, chown '/path/to/dir/htdocs/node_modules/gulp/package.json'
npm ERR!     at Error (native)
npm ERR!  { [Error: EACCES, chown '/path/to/dir/htdocs/node_modules/gulp/package.json']
npm ERR!   stack: 'Error: EACCES, chown \'/path/to/dir/htdocs/node_modules/gulp/package.json\'\n    at Error (native)',
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   path: '/path/to/dir/htdocs/node_modules/gulp/package.json',
npm ERR!   fstream_finish_call: 'chown',
npm ERR!   fstream_type: 'File',
npm ERR!   fstream_path: '/path/to/dir/htdocs/node_modules/gulp/package.json',
npm ERR!   fstream_class: 'FileWriter',
npm ERR!   fstream_stack:
npm ERR!    [ '/usr/local/lib/node_modules/npm/node_modules/fstream/lib/writer.js:308:19',
npm ERR!      '/usr/local/lib/node_modules/npm/node_modules/graceful-fs/polyfills.js:143:7',
npm ERR!      'Object.oncomplete (evalmachine.:93:15)' ] }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

npm ERR! System Linux 3.0.0-12-generic-pae
npm ERR! command "node" "/usr/local/bin/npm" "install" "gulp" "--save-dev"
npm ERR! cwd /path/to/dir/htdocs
npm ERR! node -v v0.13.0-pre
npm ERR! npm -v 1.4.26
npm ERR! path /path/to/dir/htdocs/node_modules/gulp/package.json
npm ERR! fstream_path /path/to/dir/htdocs/node_modules/gulp/package.json
npm ERR! fstream_type File
npm ERR! fstream_class FileWriter
npm ERR! fstream_finish_call chown
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! stack Error: EACCES, chown '/path/to/dir/htdocs/node_modules/gulp/package.json'
npm ERR! stack     at Error (native)
npm ERR! fstream_stack /usr/local/lib/node_modules/npm/node_modules/fstream/lib/writer.js:308:19
npm ERR! fstream_stack /usr/local/lib/node_modules/npm/node_modules/graceful-fs/polyfills.js:143:7
npm ERR! fstream_stack Object.oncomplete (evalmachine.:93:15)
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /path/to/dir/htdocs/npm-debug.log
npm ERR! not ok code 0

After A LOT of trouble shooting, we discovered the issue was caused by npm being unable to change of the owner of package.json because the drive the site resided on (a mounted NAS) was in NTFS format. Rather than pull my hair out and try and find a work around, I moved the site to the main drive and everything was gravy.

CodeKit 2 replacing hostname instances with preview URL

After settings up a new project for a quick static build using CodeKit 2, I noticed some strange (and buggy) behaviour an interesting feature.

It appears that when previewing a project through CodeKit 2’s preview, it replaces all instances of the hostname specified in the project’s “External Server Address” with the preview URL.

In my case, I’d set up my host in MAMP Pro as “GWAC”, pointing to this in CodeKit 2’s
“External Server Address”

mampScreen Shot 2014-09-09 at 8.49pm

 

Wherever the string “GWAC” appears in my files, it is replaced with CodeKit’s preview URL. ie:

Screen Shot 2014-09-09 at 8.53pm

becomes:

Screen Shot 2014-09-09 at 8.54pm

 Update

This is expected behaviour. Makes sense:

 

PHP Recursive Function to Build a Site Tree / Site Map

A problem faced fairly regularly is the need to generate a site tree either for a menu or a site map.

Assuming your page structure in your database is along the lines of:

intPageId strPageName intParentId
1 Top Level Page X 0
2 Top Level Page Y 0
3 Child Page X 1
4 Grandchild Page X 3
5 Grandchild Page Y 3

Giving you a visual structure of:

Site tree structure

I’m a developer, not a designer. Don’t judge me

You can achieve this fairly simply in with the following PHP recursive function to get your page structure into an array. Note, I have used the Laravel query builder in this example as the project I wrote this for was a Laravel project, but this can be swapped out with a query builder from whatever framework you are using or with PHP’s PDO (or any of the lesser PHP APIs for working with databases).

function generateSiteTree($startAt)
{
	if ($children = \DB::table('cms_pages')->where('intParentId', '=', $startAt)->get())
	{
		$thisLevel = array();
		foreach ($children as $child) 
		{
			$thisLevel[$child->intPageId] = $child;
			$thisLevel[$child->intPageId]->children = generateSiteTree($child->intPageId);
		}
		return $thisLevel;
	}
 
}
 
$tree = generateSectionSiteTree(0);
 
 
print_r($tree);

That’s it.

Bonus recursive function

The project I wrote this function for required me to be able to reel off the site tree only for the top level parent of the current page, regardless of however deep you currently are.

This is easily achieved with this function:

function recurseToFindSectionParent($pageId)
{
	$current = \DB::table('cms_pages')->where('intPageId', '=', $pageId)->first();
 
	if ($current->intParentId == 0) //we've reached the top of the chain
	{
		return $current->intPageId;
	}
	else //keep going
	{
		return recurseToFindSectionParent($current->intParentId);
	}
}
$topParentID = recurseToFindSectionParent($currentPageId);

You can now feed the ID of the top level parent into the generateSiteTree function:

$tree = generateSiteTree($topParentID);

Upload Progress Bar with jQuery

Another quick post detailing a method for creating an upload progress bar with jQuery for file uploads. More for self reference than anything.

This example is using XHR.upload.onprogress to return a loaded (bytes uploaded) and total (total bytes of the file) value so an upload progress can be computed. We are using the jQuery form plugin for uploading files. We take the “loaded” and “total” values to display and update a progress indicator (status bar and the displayed percentage). Once the AJAX upload has completed, we reset and hide the progress indicator.

Boom.

$(this).ajaxSubmit({
    target: '#output',
	xhr: function()
	{
		myXhr = $.ajaxSettings.xhr();
        if (myXhr.upload)
        {
            $('#progressBar').show();
            myXhr.upload.addEventListener('progress', function(ev){
            	if (ev.lengthComputable) {
	                var percentComplete = Math.round((ev.loaded / ev.total) * 100);
			$('#percentage').text(percentComplete + '%');
			$('#status').css('width', percentComplete+'%');
	        }
            }, false);
            myXhr.upload.addEventListener('load', function(ev){
            	$('#progressBar').hide();
            	$('#percentage').text('0%');
		$('#status').css('width', '0%');
            }, false);
        }
        return myXhr;
	},
	dataType: 'json',
        success:  afterSuccess
});

HTML for the progress indicator:

<div id="progressBar">
	<div id="status"></div>
	<div id="percentage">0%</div>
</div>

CSS for the progress indicator:

#progressBar{
	width: 350px;
	text-align: center;
	height: 16px;
	position: fixed;
	left: 50%;
	margin-left: -175px;
	z-index: 9;
	border:1px solid #9d9d9d;
	border-radius: 5px;
	overflow: hidden;
	top: 45px;
	display: none;
}
#progressBar #status{
	width: 0%;
	background:#00bd67;
	height: 16px;
	transition:1s linear width; /*this gives a smooth effect as the progress increases*/
}
#progressBar #percentage{
	position: absolute;
	text-align: left;
	z-index: 11;
	top: 0;
	left:0;
	font-size: 12px;
	color: #ffffff;
	width: 350px;
	padding-left: 5px;
	line-height: 15px;
}
Upload progress bar with jQuery

Upload progress bar with jQuery

Note: When developing this, I had an issue with the progress being reported waaaaay too fast. On file uploads of ~70Mb the progress would shoot to 100% in a couple of seconds and the upload would churn away in the background until it was complete. I made a post on Stack Overflow and someone suggested that it could be due to antivirus on my machine. I tried on a machine without AV and sure enough it worked great. I’ve tried turning off AV on my work dev machine and it behaved as expected (I’m unsure if it’s anything to do with the brand of anti virus but I have avast on my work machine).

IE8 prompting to download jQuery JSON response

I had a few JSON calls which IE8 was prompting me to download/save the response rather than simply hand it to jQuery to do it’s thing.

The returned JSON had the “correct” header and no other browsers were experiencing any issue:

header('Content-type: application/json');

To get around the IE8 issue, I had to change the header of the returned JSON to return text/plain:

header('Content-type: text/plain');

and ensure that my JSON calls had the dataType “json” specified. Without this explicitly specified, the browser was unable to parse the returned JSON.

$.ajax({
	type: "POST",
	dataType: "json",
	url: URL,
	data:data,
	success: function(data){
		alert(data.status)
	}
});

Boom!

As an extra note, I was also using dropzone.js for drag and drop file uploads. Dropzone.js doesn’t allow you to force the datatype of the response so I had to convert the response to JSON before doing anything with it with:

JSON.parse(responseText);

(Note, JSON.parse is only supported in modern browsers, this was not an issue though for me though as only modern browsers were able to use the dropzone functionality anyway and I had a fallback in place for older browsers)

PHP sessions not setting in Internet Explorer (IE)

I just made a pretty interesting discovery in that PHP sessions do not set in IE when there is one or more underscores (_) in the URL.

IE was the last browser I checked an almost completed project in, having developed in Chrome and Firefox. My heart sank when I found I was unable to log in. I hadn’t experienced this issue on any other browser and didn’t have a clue where to even start.

I was able to log in on the staging environment (where I show progress to the client) but not on my local dev so it had to be an environment issue. The only difference between the environments was the URL. I had a play around and found that removing the underscores from my local dev environment URL fixed this issue and I was able to log in. Boom.

jQuery $ is undefined in IE iFrame

I have a very jQuery heavy app which I am including in an iFrame on a site I am working on. Developing in Firefox and Chrome, it was only at the last minute when I came to test the site in Internet Explorer. Much to my dismay I was getting “$ is undefined”. There were no issues when running the site directly in IE (not through the iFrame).

I tried numerous fixes recommended by Stack Overflow including delaying the loading of the iFrame until the main content had loaded, but in the end, it was simply fixed by hosting jQuery from my own server rather than pulling it in from the CDN.

Cookie Clicker AutoHotKey script

I’ve been stupidly addicted to Cookie Clicker for the last few days. There are a few of us in the office with sore fingers from our in house competition to see who can get the highest rate clicks per second (CPS). At the time of writing, I’m firing at 1,853,140,156.5 CPS.

To gain a leg up on the competition (albeit an unethical one), I went after an AutoHotKey script to take care of some extra clicking for me while I’m out of the office. To save me learning AHK and writing my own, I came across this post: http://www.autohotkey.com/board/topic/97540-cookie-clicker-game/

Here’s the Cookie Clicker AutoHotKey:

$~m::
if (IsCookieActive()) {
	Loop, 9
	If stop = 1
		Break
	Else
	{
		WinGetActiveStats WindowName, WindowW, WindowH, WindowX, WindowY
		if ErrorLevel
			sleep 0
		else
			click 267, 460
	}
}
return

Shift::
If stop = 1
	stop = 0
Else
	stop = 1
return


IsCookieActive() {
	WinGetActiveTitle, title
	titleContains := "cookies"
	; MsgBox %title%
	IfInString, title, %titleContains%
	{
		return true
	} 
	else 
	{
		return false
	}
}

To Use: Change the loop value (currently set to 9) to the number of cookie clicks you wish to execute. Run the AHK script. Start it by pressing ‘m’ and stop it by double pressing ‘shift’.