Scalable sentiment analytics

Authors

ASLAN BAKİROV
KEVSER NUR ÇOĞALMIŞ
AHMET BULUT

DOI

10.3906/elk-1311-128

Abstract

Spark has become a widely popular analytics framework that provides an implementation of the equally popular MapReduce programming model. Hadoop is an Apache foundation framework that can be used for processing large datasets on a cluster of computers using the MapReduce programming model. Mahout is an Apache foundation project developed for building scalable machine learning libraries, which includes built-in machine learning classifiers. In this paper, we show how to build a simple text classifier on Spark, Apache Hadoop, and Apache Mahout for extracting out sentiments from a text collection containing millions of text documents. Using a collection of 7 million movie reviews taken from IMDB, a Bayesian classifier was learned to predict sentiments for test reviews. Separate classifiers were learned on both Spark and Hadoop, i.e. our contenders for scalable sentiment analytics. Our empirical results showed that the sentiment learning task on Spark ran almost 10 times faster than the learning task on Hadoop.

Keywords

Sentiment analysis, MapReduce, Spark, Hadoop, Apache Mahout

First Page

1560

Last Page

1570

Recommended Citation

BAKİROV, ASLAN; ÇOĞALMIŞ, KEVSER NUR; and BULUT, AHMET (2016) "Scalable sentiment analytics," Turkish Journal of Electrical Engineering and Computer Sciences: Vol. 24: No. 3, Article 62. https://doi.org/10.3906/elk-1311-128
Available at: https://journals.tubitak.gov.tr/elektrik/vol24/iss3/62