Performance evaluation of DNN with other machine learning techniques in a cluster using Apache Spark and MLlib

Sentiment analysis on large data has become challenging due to the diversity, and nature of data.
Advancements in the internet, along with large data availability have obviated the traditional limitations
to distributed computing. The objective of this work is to carry out sentiment analysis on Apache Spark
distributed Framework to speed up computations and enhance machine performance in diverse environ
ments. The analysis, such as polarity identification, subjective analysis and email spam etc., are carried on
various text datasets. After pre-processing, Term Frequency-Inverse Document Frequency (TF-IDF) and
unsupervised Spark-Latent Dirichlet Allocation (LDA) clustering algorithms are used for feature extrac
tion and selection to improve the accuracy. Deep Neural Networks (DNN), Support Vector Machines
(SVM), Tree ensemble classifiers are used to evaluate the performance of the framework on single node
and cluster environments. Finally, the proposed work aims at building an approach for enhancing
machine performance, more in terms of runtime over accuracy.