Staying Sane With Asynchronous Programming: Promises and Generators
Callback Hell no more with Promises and its close ally, Generators.
Callback Hell, also known as Pyramid of Doom, is an anti-pattern seen in code of programmers who are not wise in the ways of asynchronous programming.
async1(function(){
async2(function(){
async3(function(){
async4(function(){
....
});
});
});
});
It consists of multiple nested callbacks which makes code hard to read and debug. It is understandable how one might unknowingly get caught in Callback Hell while dealing with asynchronous logic.
JavaScript callback hell illustrated
If you are not expecting your application logic to get too complex, a few callbacks seem harmless. But once your project requirements start to swell, you will quickly find yourself piling layers of nested callbacks. Congrats! Welcome to Callback Hell.
Q: What's worse than callback hell?
A: Not fixing it.
So it is definitely recommended to do it right from the get-go and avoid deeply-nested callbacks. My favourite solution for this will be the usage of the Promise object. I have been dealing with Node.js for my last few projects and Promise managed to keep my sanity in check. But if you are looking for something more edgy, you will love Generators. I will touch more in depth about both approaches below.
Promise
A little preview of how a promise-based approach solves the callback hell:
// Callback approach
async1(function(){
async2(function(){
async3(function(){
....
});
});
});
// Promise approach
var task1 = async1();
var task2 = task1.then(async2);
var task3 = task2.then(async3);
task3.catch(function(){
// Solve your thrown errors from task1, task2, task3 here
})
// Promise approach with chaining
async1(function(){..})
.then(async2)
.then(async3)
.catch(function(){
// Solve your thrown errors here
})
Let's break it down on what Promise is doing for us here:
- Flattened callbacks
- Return values from asynchronous function
- Throw and Catch exceptions
With flattened code hierarchy, the code are now much more readable. But that's just the by-product of Promise. The main thing about Promise is that it helps us achieve (2) and (3). Achieving (2) and (3) is important because that is what synchronous function can do for us.
For Promise to make (2) and (3) work, the asynchronous function itself should return a Promise Object. This Promise object has two methods, then and catch. The methods will later be called depending on the state (fulflled || rejected) of the Promise Object.
asyncWithPromise() // Returns a promise object
.then(function(){ // if object's state is fulfilled, go here
...
})
.catch(function(){ // if object's state is rejected, go here
...
})
Let's start making some Promises
"So how do we convert an asynchronous function to return a Promise Object?", you may ask. Here's a simple example for writing a promise-returning asynchronous function.
Note: I will be using Q.js library for the demonstration of Promise and Generator. But if you are working in the ES6-friendly environment like IO.js, feel free to use the native syntax.
//asyncWithPromise is promise-returning
var asyncWithPromise = function(){
return new Q.Promise(function(resolve,reject){
if(everythingWorks){
resolve("This is true"); // State will be fulfilled
} else {
reject("This is false"); // State will be rejected
}
})
}
The callback from the Promise constructor gives us two parameters, resolve and reject function, that will affect the state of the Promise object. If everything works, call resolve, otherwise call reject. You can think of reject as throwing an exception. Note that you can pass in values to resolve and reject which will be further passed on to the respective handlers, then and catch.
Convert callback-based to Promise-based
But let's say if you are unable to edit the asynchronous function (perhaps due to a third-party library), you are still able to wrap the function and return a Promise.
//async1 doesn't return promise so we need to wrap it
var wrapperPromise = function(){
return new Q.Promise(function(resolve,reject){
//Resolve or reject in the callback function
async1(function(result){
if(result){
resolve(result);
} else {
reject(result);
}
})
});
}
Control Flow
In asynchronous programming, there are two control flow patterns that you will encounter, Serial Flow and Parallel Flow.
Serial Flow
Serial Flow happens when you need a group of asynchronous tasks to finish in series before you execute your code.
/**
* Serial Control Flow
* 1. Check for valid URL
* 2. If URL is valid, continue else return;
* 3. Lowercase the url
* 4. Using the lowercased url and name, generate an anchor tag
**/
var sampleLink = {
"name": "google",
"url" : "http://GOOGLE.com"
};
checkUrl(function(validUrl){
if(!validUrl) {return "Invalid URL"}
lowerCase(sampleLink.url,function(lowerCaseLink){
generateLink(sampleLink.name,lowerCaseLink,function(link){
console.log("Solve with Callback: ",link);
});
});
})
In the serial flow example above, we used 3 asynchronous functions that are mock API endpoints. Using a callback-based approach, we managed to tackle this serial flow problem. However, we can handle this in a cleaner manner using Promises.
/****************************************
* Mock Asynchronous API endpoints
*
* + lowerCase : Return lowercase string
* + checkUrl : Check for valid URL
* + generateLink : Generate HTML anchor tag
****************************************/
var lowerCase = function(str,cb){
setTimeout(function(){
cb(str.toLowerCase());
},Math.random()1000);
};
var checkUrl = function(url,cb){
setTimeout(function(){
cb(/^(http:\/\/)\w./.test(url));
},Math.random()1000);
};
var generateLink = function(name,url,cb){
setTimeout(function(){
cb(name.link(url));
},Math.random()1000);
};
We shall first wrap these asynchronous functions to return promise. Using the same wrapping method we mentioned above, we created 3 functions that return us promises when the asynchronous tasks complete.
var lowerCasePromise = function(str){
return Q.Promise(function(resolve,reject){
lowerCase(str,resolve);
});
};
var checkUrlPromise = function(url){
return Q.Promise(function(resolve,reject){
checkUrl(url,function(validUrl){
if(!validUrl) return reject("Invalid URL");
resolve(validUrl);
});
});
};
var generateLinkPromise = function(name,link){
return Q.Promise(function(resolve,reject){
generateLink(name,link,resolve);
});
};
With these three promise-returning functions, we solve the serial flow problem again. However, before I show you the "correct" answer. Let me show you a bad example of using Promises:
Bad example of Promises usage
checkUrlPromise(sampleLink.url)
.then(function(){
return lowerCasePromise(sampleLink.url);
})
.then(function(lowerCaseUrl){
generateLinkPromise(sampleLink.name,lowerCaseUrl)
.then(function(){
console.log("Solve with Promises: ",link);
})
})
.catch(function(err){
console.log(err);
})
At the first then handler, all we did in it is to return a Promise from the lowerCasePromise function. That seems a bit wasteful. In the next handler, I nested another callback from the generateLinkPromise function. What? Nested callbacks even with Promises? Yes, you can haz bad programming even with the right tools.
Let's see how we can improve on this:
Partial binding and Further Chaining
checkUrlPromise(sampleLink.url)
.then(lowerCasePromise.bind(null,sampleLink.url))
.then(function(lowerCaseUrl){
return generateLinkPromise(sampleLink.name, lowerCaseUrl);
})
.then(function(link){
console.log("Solve with Promises: ",link);
})
.catch(function(err){
console.log(err);
});
By using the bind method, we are able to reduce verbosity. Instead of handling the promise object of generateLinkPromise inside of another callback, we return that promise and pass it to the next then handler. Hence, visually we can still maintain a consistent code hierarchy.
Parallel Flow
Parallel Flow happens when you need a group of asynchronous tasks to be completed together before executing another task.
/****************************************
* Parallel Flow Control
*
* Return an array of string in lowercase in the SAME order.
****************************************/
var strArr = ["ABC","EFG","gooGLE","YAHoo!"];
var lowerCaseArr = [],
arrCounter = 0;
strArr.forEach(function(item,index){
lowerCase(item,function(lowerCaseString){
lowerCaseArr[index] = lowerCaseString;
arrCounter++;
if(arrCounter === strArr.length){
console.log(lowerCaseArr);
// ["abc","efg","google","yahoo!"]
}
});
});
The callback-based solution above is actually quite abstract and doesn't have good readability.
We shall re-use lowerCasePromise function to solve this problem.
var strArr = ["ABC","EFG","gooGLE","YAHoo!"];
var promisePool = strArr.map(function(str){
return lowerCasePromise(str);
});
Q
.all(promisePool)
.then(function(data){
console.log(data);
// ["abc","efg","google","yahoo!"];
}).catch(function(err){
console.log(err);
});
Using Q.all, once all promises have resolved, the then handler will be called with the array of resolved result passed in. The ordering of the array is also maintained. Simple and absolutely readable. If any of the promises is rejected, the catch handler will be called instead.
Generators
Generators, like Promise, is also one of the features in ES6 that can manage asynchronous programming. The great thing about Generators is that it can work hand-in-hand with Promise to bring us closer to synchronous programming.
/* Using the same Serial Flow example */
Q.spawn(function* (){
try {
if(yield checkUrlPromise(url)){
var str = yield lowerCasePromise(url);
var link = yield generateLinkPromise(name,str);
console.log("Solve with Generators: ",link);
} else {
console.log("Invalid Url")
}
}
catch(e){
console.log(e);
}
});
Pro-tip: If you cannot see how this might look synchronous, strip away the function *() and yield.
From the above example, we are coding in a synchronous manner with Generators- using the try and catch block and writing the code as if the result is returned immediately. There are also no verbosity with the then handler.
It took me awhile to understand generators because it's really abstract. I will attempt to explain generators from my point of view but you should totally check out other articles which had done a better job.
Run-to-completion
Javascript function are expected to run-to-completion - This means once the function starts running, it will run to the end of the function. However, Generators allow us to interrupt this execution and switch to other tasks before returning back to the last interrupted task.
/* Using the same Serial Flow example */
Q.spawn(function* (){
console.log('1');
// Pause here and jump out of this function until the promise resolve
var str = yield lowerCasePromise("HELLO");
console.log(str);
});
console.log('2');
// > 1
// > 2
// > "hello"
The weird-looking function *() is to inform the Javascript interpreter that this is a special generator function type while yield is the cue to interrupt the function.
When the code enter the function, it will hit console.log('1') first. It then continues to the next line which hit the yield expression. This pauses the function and allow other code to run which in this case is console.log('2'). But once lowerCasePromise's Promise has resolved, the paused function resumes and it continue to the last line of the function, console.log(str).
The Parallel Flow can be easily solve by Generators too but I will leave that as a homework for you.
Conclusion
I covered the problems of asynchronous programming and how you can tackle them with Promise and Generators. I have barely scratch the surface of Generators though but it should be enough to get you going.
If you want to try out Promise and Generators on the client-side, I have set up an example at JS Bin for you to get started. Be aware that I had only test on Chrome 41 which seems to provide full support for Generators.
As for the server-side, you can start using the Q.js for Promise. But you will need to switch on the --harmony-generators flag on Node.js 0.12.x or simply just switch to IO.js.
