STAC Report: IBM BigInsights vs Apache Hadoop, using SWIM

BigInsights performed jobs ~4x faster than pure Apache Hadoop and was ~11x faster in pure scheduling speed.

23 October 2013

STAC has just released a STAC Report detailing the performance difference between IBM InfoSphere® BigInsights® Enterprise Edition with Adaptive MapReduce (ver 2.1.0.1) on the one hand and Apache Hadoop (ver 1.1.2) on the other. We tested the two on the same hardware cluster using an adaptation of the Statistical Workload Injector for MapReduce (SWIM) methodology and a workload modeled on Facebook MapReduce jobs.

BigInsights completed the jobs about 4 times faster than Apache Hadoop running on the same environment. BigInsights was also about 11x faster using the corner-case “sleep” test of scheduling speed.

Qualified members of the STAC Benchmark Council can access the report here.