International Journal of Applied Science and Engineering
Published by Chaoyang University of Technology

Rakshitha Prabhu 1*, Chandrashekara Seesandra Nashappa 2

1 Department of Computer Science and Engineering, S.J.C. Institute of Technology, Chikkaballapur, India

2 Department of Computer Science and Engineering, C. Byre Gowda Institute of Technology, India

 

Download Citation: |
Download PDF


ABSTRACT

Sentiment analysis is a crucial task in Natural Language Processing (NLP) that determines whether users have positive, neutral, or negative feelings about movies or products. NLP is being employed to address challenges related to the implementation of sentiment analysis. In these issues, basic polarity detection has been replaced by intricate emoticons that differentiate between various negative emotions. Utilizing a feature set to train the classifier, machine learning (ML) techniques can surpass lexicon-based methods in performance, but their effectiveness may be constrained to specific applications. The existing algorithms were processed to reduce the effects on social media and fake news detections but do not work on the classification and validation of tweets. To overcome these issues, a novel Deep Learning model with Weight Function based Bidirectional Encoder Representation from Transformers (WF-BERT) is proposed in this research to achieve high classification accuracy compared to the conventional ML techniques The weight functions of input and output elements in the proposed model are determined dynamically, based on the connection link between input and output. The obtained output results of the proposed model achieved an accuracy of 93.66%, recall of 87.30%, the precision of 88.00%, and F1-score of 87.90%. The proposed model outperforms existing language representation models with limited training data by including additional domain information.


Keywords: Bidirectional encoder representation from transformers, Machine learning algorithm, Natural language processing, Polarity detection, Sentiment analysis.


Share this article with your colleagues

 


REFERENCES

  1. Alattar, F., Shaalan, K. 2021. Using artificial intelligence to understand what causes sentiment changes on social media. IEEE Access, 9, 61756–61767.

  2. AlBadani, B., Shi, R., Dong, J. 2022. A novel machine learning approach for sentiment analysis on twitter incorporating the universal language model fine-tuning and SVM. Applied System Innovation, 5, 13.

  3. Aljedaani, W., Rustam, F., Mkaouer, M.W., Ghallab, A., Rupapara, V., Washington, P.B., Lee, E., Ashraf, I. 2022. Sentiment analysis on Twitter data integrating TextBlob and deep learning models: The case of US airline industry. Knowledge-Based Systems, 255, 109780.

  4. Alkhaldi, N.A., Asiri, Y., Mashraqi, A.M., Halawani, H.T., Abdel-Khalek, S., Mansour, R.F. 2022. Leveraging tweets for artificial intelligence driven sentiment analysis on the COVID-19 pandemic. Healthcare, 10, 910.

  5. Aslan, S., Kızıloluk, S., Sert, E. 2023. TSA-CNN-AOA: Twitter sentiment analysis using CNN optimized via arithmetic optimization algorithm. Neural Computing and Applications, 35, 10311–10328.

  6. Bibi, M., Abbasi, W.A., Aziz, W., Khalil, S., Uddin, M., Iwendi, C., Gadekallu, T.R. 2022. A novel unsupervised ensemble framework using concept-based linguistic methods and machine learning for twitter sentiment analysis. Pattern Recognition Letters, 158, 80–86.

  7. Cai, R., Qin, B., Chen, Y., Zhang, L., Yang, R., Chen, S., Wang, W. 2020. Sentiment analysis about investors and consumers in energy market based on BERT-BiLSTM. IEEE Access, 8, 171408–171415.

  8. Chintalapudi, N., Battineni, G., Amenta, F. 2021. Sentimental analysis of COVID-19 tweets using deep learning models. Infectious Disease Reports, 13, 329–339.

  9. Dashtipour, K., Gogate, M., Adeel, A., Larijani, H., Hussain, A. 2021. Sentiment Analysis of Persian Movie Reviews Using Deep Learning. Entropy, 23, 596.

  10. Dwivedi, D.N., Pathak, S. 2022. Sentiment analysis for COVID vaccinations using Twitter: Text clustering of positive and negative sentiments. Decision Sciences for COVID-19, International Series in Operations Research & Management Science, 320, 195–203.

  11. Faruque, M.A., Rahman, S., Chakraborty, P., Choudhury, T., Um, J.-S., Singh, T.P. 2022. Ascertaining polarity of public opinions on Bangladesh cricket using machine learning techniques. Spatial Information Research, 30, 1–8.

  12. Giménez, M., Palanca, J., Botti, V. 2020. Semantic-based padding in convolutional neural networks for improving the performance in natural language processing: A case of study in sentiment analysis. Neurocomputing, 378, 315–323.

  13. Gowda, S.R.S., Archana, B.R., Shettigar, P., Satyarthi, K.K. 2022. Sentiment analysis of Twitter data using naïve bayes classifier. ICDSMLA 2020, Proceedings of the 2nd International Conference on Data Science. Machine Learning and Applications, Lecture Notes in Electrical Engineering, 783, 1227–1234.

  14. Hidayat, T.H.J., Ruldeviyani, Y., Aditama, A.R., Madya, G.R., Nugraha, A.W., Adisaputra, M.W. 2022. Sentiment analysis of twitter data related to Rinca Island development using Doc2Vec and SVM and logistic regression as classifier. Procedia Computer Science, 197, 660–667.

  15. Jabalameli, S., Xu, Y., Shetty, S. 2022. Spatial and sentiment analysis of public opinion toward COVID-19 pandemic using twitter data: At the early stage of vaccination. International Journal of Disaster Risk Reduction, 80, 103204.

  16. Jain, R., Bawa, S., Sharma, S. 2022. Sentiment analysis of COVID-19 tweets by machine learning and deep learning classifiers. Advances in Data and Information Sciences. Lecture Notes in Networks and Systems, 318, 329–339.

  17. Karim, S., He, H., Laghari, A.A., Magsi, A.H., Laghari, R.A. 2021. Quality of service (QoS): Measurements of image formats in social cloud computing. Multimedia Tools and Applications, 80, 4507–4532.

  18. Karim, S., Zhang, Y., Laghari, A.A., Asif, M.R. 2018. Image processing based proposed drone for detecting and controlling street crimes. 2017 IEEE 17th International Conference on Communication Technology (ICCT), 1725–1730.

  19. Kaushik, N., Bhatia, M.K. 2022. Twitter sentiment analysis using K-means and hierarchical clustering on COVID pandemic. International Conference on Innovative Computing and Communications, Proceedings of ICICC 2021. Advances in Intelligent Systems and Computing, 1387, 757–769.

  20. Kumar, A., Srinivasan, K., Wen-Huang, C., Zomaya, A.Y. 2020. Hybrid context enriched deep learning model for fine-grained sentiment analysis in textual and visual semiotic modality social data. Information Processing & Management, 57, 102141.

  21. Kumar, P., Vardhan, M. 2022. PWEBSA: Twitter sentiment analysis by combining Plutchik wheel of emotion and word embedding. International Journal of Information Technology, 14, 69–77.

  22. Laghari, A.A., He, H., Shafiq, M., Khan, A. 2018. Assessment of quality of experience (QoE) of image compression in social cloud computing. Multiagent and Grid Systems, 14, 125–143.

  23. Leelawat, N., Jariyapongpaiboon, S., Promjun, A., Boonyarak, S., Saengtabtim, K., Laosunthara, A., Yudha, A.K., Tang, J. 2022. Twitter data sentiment analysis of tourism in Thailand during the COVID-19 pandemic using machine learning. Heliyon, 8, e10894.

  24. Li, M., Chen, L., Zhao, J., Li, Q. 2021. Sentiment analysis of Chinese stock reviews based on BERT model. Applied Intelligence, 51, 5016–5024.

  25. Malhotra, S., Kumar, V., Agarwal, A. 2021. Bidirectional transfer learning model for sentiment analysis of natural language. Journal of Ambient Intelligence and Humanized Computing, 12, 10267–10287.

  26. Mendoza-Urdiales, R.A., Núñez-Mora, J.A., Santillán-Salgado, R.J., Valencia-Herrera, H. 2022. Twitter sentiment analysis and influence on stock performance using transfer entropy and EGARCH methods. Entropy, 24, 874.

  27. Naresh, A., Krishna, P.V. 2021. An efficient approach for sentiment analysis using machine learning algorithm. Evolutionary Intelligence, 14, 725–731.

  28. Narwal, R., Aggarwal, H. 2022. Predicting online game-addicted behaviour with sentiment analysis using Twitter data. Machine Learning, Advances in Computing, Renewable Energy and Communication, Lecture Notes in Electrical Engineering, 768, 505–517.

  29. Neelakandan, S., Paulraj, D., Ezhumalai, P., Prakash, M. 2022. A deep learning modified neural network (DLMNN) based proficient sentiment analysis technique on Twitter data. Journal of Experimental & Theoretical Artificial Intelligence.

  30. Neogi, A.S., Garg, K.A., Mishra, R.K., Dwivedi, Y.K. 2021. Sentiment analysis and classification of Indian farmers’ protest using twitter data. International Journal of Information Management Data Insights, 1, 100019.

  31. Nezhad, Z.B., Deihimi, M.A. 2022. Twitter sentiment analysis from Iran about COVID 19 vaccine. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 16, 102367.

  32. Palomino, M.A., Aider, F. 2022. Evaluating the effectiveness of text pre-processing in sentiment analysis. Applied Sciences, 12, 8765.

  33. Patel, R., Passi, K. 2020. Sentiment analysis on Twitter data of world cup soccer tournament using machine learning. IoT, 1, 218–239.

  34. Pathak, A.R., Pandey, M., Rautaray, S. 2021. Topic-level sentiment analysis of social media data using deep learning. Applied Soft Computing, 108, 107440.

  35. Pota, M., Ventura, M., Fujita, H., Esposito, M. 2021. Multilingual evaluation of pre-processing for BERT-based sentiment analysis of tweets. Expert Systems with Applications, 181, 115119.

  36. Qian, C., Mathur, N., Zakaria, N.H., Arora, R., Gupta, V., Ali, M. 2022. Understanding public opinions on social media for financial sentiment analysis using AI-based techniques. Information Processing & Management, 59, 103098.

  37. Rodrigues, A.P., Fernandes, R., Akash, A., Abhishek, B., Shetty, A., Atul, K., Lakshmanna, K., Shafi, R.M. 2022. Real-time twitter spam detection and sentiment analysis using machine learning and deep learning techniques. Computational Intelligence and Neuroscience, 2022, 5211949.

  38. Sankar, H., Subramaniyaswamy, V., Vijayakumar, V., Kumar, S.A., Logesh, R., Umamakeswari, A. 2020. Intelligent sentiment analysis approach using edge computing‐based deep learning technique. Software: Practice and Experience, 50, 645–657.

  39. Singh, C., Imam, T., Wibowo, S., Grandhi, S. 2022. A deep learning approach for sentiment analysis of COVID-19 reviews. Applied Sciences, 12, 3709.

  40. Soumya, S., Pramod, K.V. 2020. Sentiment analysis of malayalam tweets using machine learning techniques. ICT Express, 6, 300–305.

  41. Srikanth, J., Damodaram, A., Teekaraman, Y., Kuppusamy, R., Thelkar, A.R. 2022. Sentiment analysis on COVID-19 Twitter data streams using deep belief neural networks. Computational Intelligence and Neuroscience, 2022, 8898100.

  42. Sunitha, D., Patra, R.K., Babu, N.V., Suresh, A., Gupta, S.C. 2022. Twitter sentiment analysis using ensemble based deep learning model towards COVID-19 in India and European countries. Pattern Recognition Letters, 158, 164–170.

  43. Swamy, S., Kundale, J., Jadhav, D. 2022. Sentiment analysis of multilingual mixed-code, Twitter data using machine learning approach. International Conference on Innovative Computing and Communications, Advances in Intelligent Systems and Computing, 1388, 683–697.

  44. Tao, J., Fang, X. 2020. Toward multi-label sentiment analysis: A transfer learning based approach. Journal of Big Data, 7, 1.

  45. Villavicencio, C., Macrohon, J.J., Inbaraj, X.A., Jeng, J.-H., Hsieh, J.-G. 2021. Twitter sentiment analysis towards covid-19 vaccines in the Philippines using naïve bayes. Information, 12, 204.

  46. Wang, D., Wang, X., Yin, S. 2021a. A new recursive neural network and center loss for expression recognition. International Journal of Electronics and Information Engineering, 13, 97–104.

  47. Wang, X., Kou, L., Sugumaran, V., Luo, X., Zhang, H. 2021b. Emotion correlation mining through deep learning models on natural language text. IEEE Transactions on Cybernetics, 51, 4400–4413.

  48. Wang, X., Tang, M., Yang, T., Wang, Z. 2021c. A novel network with multiple attention mechanisms for aspect-level sentiment analysis. Knowledge-based systems, 227, 107196.

  49. Wang, Y., Guo, J., Yuan, C., Li, B. 2022. Sentiment analysis of Twitter data. Applied Sciences, 12, 11775.

  50. Xu, F., Pan, Z., Xia, R. 2020. E-commerce product review sentiment classification based on a naïve Bayes continuous learning framework. Information Processing & Management, 57, 102221.

  51. Yadav, V., Verma, P., Katiyar, V. 2023. Long short term memory (LSTM) model for sentiment analysis in social data for e-commerce products reviews in Hindi languages. International Journal of Information Technology, 15, 759–772.

  52. Zhao, Y., Li, H., Yin, S. 2022. A multi-channel character relationship classification model based on attention mechanism. International Journal of Mathematics and Computer Science, 8, 28–36.


ARTICLE INFORMATION

Received: 2023-07-24
Revised: 2023-09-06
Accepted: 2023-09-18
Available Online: 2023-12-13


Cite this article:

Prabhu, R., Nashappa, C.S. 2024. A dynamic weight function based BERT auto encoder for sentiment analysis. International Journal of Applied Science and Engineering, 21, 2023182. https://doi.org/10.6703/IJASE.202403_21(1).006

  Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Other people also read ...

doaj logo

 

2.3
2022CiteScore
 
 
48th percentile
Powered by  Scopus
 
 

SCImago Journal & Country Rank

IJASE - Most Read Articles

IJASE - Most popular articles