suguru.dev

バンクーバーで働くエンジニアの備忘録

Benchmark: Async vs Neo-Async

What is Neo-Async?

Neo-Async is meant to be used as a drop-in replacement for the Async library. It implements the same API, and adds extra functionality. I have been trying to make it faster and safer than Async.

https://github.com/suguru03/neo-async

It doesn't mean I don't like Async. I would like to improve both libraries and to showcase some performance optimizations that might be implemented by other Node.js libraries. I especially admire @aearly for their good work.

Benchmark

Recently, Async has been released v2.0.0-rc.6 and I released Neo-Async v2.0.0-rc.1.
So I would like to check performance using benchmark and func-comparator.

How to check

I made a repository to measure performance between Async and Neo-Async.

https://github.com/suguru03/async-benchmark

$ git clone git@github.com:suguru03/async-benchmark.git
$ cd async-benchmark
$ npm install
$ node . // It might take more than one hour... 

Environment

  • Ubuntu v12.04
  • Node.js v6.2.1
  • async v2.0.0-rc.6
  • neo-async v2.0.0-rc.1
  • benchmark v2.1.0
  • func-comparator v0.7.1

Result

Neo-Async is 1.27 ~ 10.7 times faster than Async.

The value is the ratio (Neo-Async/Async) of the average speed.

Collections

function benchmark func-comparator
each 3.71 2.54
eachSeries 2.14 1.90
eachLimit 2.14 1.88
eachOf 3.30 2.50
eachOfSeries 1.97 1.83
eachOfLimit 2.02 1.80
map 4.20 4.11
mapSeries 2.40 3.65
mapLimit 2.64 2.66
mapValues 5.71 5.32
mapValuesSeries 3.82 3.23
mapValuesLimit 3.10 2.38
filter 8.11 8.76
filterSeries 5.79 4.86
filterLimit 4.00 3.32
reject 9.47 9.52
rejectSeries 7.39 4.64
rejectLimit 4.54 3.49
detect 6.67 6.37
detectSeries 3.54 3.73
detectLimit 2.38 2.62
reduce 4.13 3.23
reduceRight 4.23 3.24
transform 5.30 5.17
sortBy 2.24 2.37
some 6.39 6.10
someSeries 5.37 4.66
someLimit 3.39 2.84
every 6.85 6.27
everySeries 4.53 3.90
everyLimit 3.36 2.75
concat 9.18 9.35
concatSeries 7.49 6.09

Control Flow

funciton benchmark func-comparator
parallel 7.54 5.45
series 3.29 2.41
waterfall 5.12 4.27
whilst 1.96 1.95
doWhilst 2.07 1.96
until 2.10 1.99
doUntil 1.98 2.04
during 10.7 7.09
doDuring 5.98 6.03
queue 1.83 1.75
priorityQueue 1.79 1.75
times 3.84 3.65
race 1.45 1.27
auto 3.23 3.50
retry 9.43 6.78

Future Prospects

I'm not done with performance optimization; the control flow functions are my main focus right now. I will release v2.0.0 when satisfied with performance.

And I'm keeping in mind this issue.

Benchmark setup seems to be biased against async callback style libraries · Issue #985 · petkaantonov/bluebird · GitHub

I believe the asyncFlowF library doesn't implement the same interface as Async, but I would like Neo-Async to be beat it benchmark anyways. ;) Even as many JavaScript users switch to Promises, callback style is far from dead, so I want to keep improving Async and Neo-Async.

Next time, I will write about the difference between Async and Neo-Async. :)

Reference