International Journal of Applied Science and Engineering
Published by Chaoyang University of Technology

Niraj Kumar*, Subhash Chandra Yadav

Department of Computer Science and Engineering, Central University of Jharkhand, Ranchi, India

 

Download Citation: |
Download PDF


ABSTRACT

The social media platform has become one of the prime modes of interaction between different natures of peoples, where they used to share their feelings in the form of textual messages. Due to the easy availability of plenty of social media tools like Twitter, Flickr, Imgur, Facebook etc. more and more people are indulging themselves in propagating enormous amounts of information on diverse nature of topics/various issues, and that has become a huge source of data to be analyzed by the researchers to extract useful information. This research article comprises a brief study of different text classification models, which uses deep learning algorithm in Natural Language Processing task. However, it remains a challenging issue for most of the researchers to get absolute architecture, layout and appropriate techniques for classifying text data. Further, the study reveals a brief discussion on the relevance of various deep learning models available for text classification along with their feature assessment also a comparative study of the various available deep-learning models have also been done during the work.


Keywords: Text classification, Neural network, Attention mechanism, Transformer, RNN, Deep learning.


Share this article with your colleagues

 


REFERENCES

  1. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135–146.

  2. Chen, J., Huang, H., Tian, S., Qu, Y. 2009. Feature selection for text classification with Naïve Bayes. Expert Systems with Applications, 36, 5432–5435.

  3. Chen, Y. 2015. Convolutional neural network for sentence classification (Master's thesis, University of Waterloo).

  4. Cheng, J., Dong, L., Lapata, M. 2016. Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601-06733.

  5. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P. 2011. Natural language processing from scratch. Journal of Machine Learning Research, 12, 2493–2537.

  6. Devlin, J., Chang, M.W., Lee, K., Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810–04805.

  7. Dhuliawala, S., Kanojia, D., Bhattacharyya, P. 2016. Slangnet: A wordnet like resource for english slang. In Proceedings of the Tenth International Conference on Language Resources and Evaluation, 4329–4332.

  8. Fahad, S.A., Yahya, A.E. 2018. Inflectional review of deep learning on natural language processing. International Conference on Smart Computing and Electronic Enterprise IEEE., 1–4.

  9. Gupta, G., Malhotra, S. 2015. Text document tokenization for word frequency count using rapid miner. International Journal of Computer Applications, 975, 8887.

  10. Gupta, V., Lehal, G.S. 2009. A survey of text mining techniques and applications. Journal of Emerging Technologies in Web Intelligence, 1, 60–76.

  11. Hinton, G.E., Osindero, S., Teh, Y.W. 2006. A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527–1554.

  12. Joachims, T. 2005. Text categorization with support vector machines: Learning with many relevant features. In Machine Learning: ECML-98: 10th European Conference on Machine Learning Chemnitz, Germany, April 21–23, 1998 Proceedings, 137–142. Berlin, Heidelberg: Springer Berlin Heidelberg.

  13. Johnson, R., Zhang, T. 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. In International Conference on Machine Learning PMLR, 526–534.

  14. Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., Mikolov, T. 2016. Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612–03651.

  15. Kaggle, 2021a, https://www.kaggle.com/datasets/yelpdataset.

  16. Kaggle, 2021b, https://www.kaggle.com/lakshmi25npathi/imdbdataset-of-50k-movie-reviews

  17. Kalchbrenner, N., Grefenstette, E., Blunsom, P. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404–2188.

  18. Karhunen, J., Raiko, T., Cho, K. ,2015. Unsupervised deep learning: A short review. Advances in independent component analysis and learning machines, 125–142.

  19. Kim, Y., Jernite, Y., Sontag, D., Rush, A.M. 2016. Character-aware neural language models. In Thirtieth AAAI Conference on Artificial Intelligence.

  20. Kowsari, K., Brown, D.E., Heidarysafa, M., Meimandi, K.J., Gerber, M.S., Barnes, L.E. 2017. Hdltex: Hierarchical deep learning for text classification. In 16thIEEE International Conference on Machine Learning and Applications IEEE. 364–371.

  21. Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P.N., Bizer, C. 2015. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic web, 6, 167–195.

  22. Liu, P., Qiu, X., Huang, X. 2016. Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605–05101.

  23. Liu, X., Shen, Y., Duh, K., Gao, J. 2017. Stochastic answer networks for machine reading comprehension.arXiv preprint arXiv:1712–03556.

  24. Liu, Y., Sun, C., Lin, L., Wang, X. 2016. Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv preprint arXiv:1605–09090.

  25. LozaMencía, E., Fürnkranz, J. 2008. Efficient pairwise multilabel classification for large-scale problems in the legal domain. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases 50-65.

  26. Luštrek, M., Gams, M., Martinčić-Ipšić, S. 2016. What makes classification trees comprehensible. Expert Systems with Applications, 62, 333–346.

  27. Bijaksana, M.A., Li, Y., Algarni, A. 2013. A pattern based two-stage text classifier. In Machine Learning and Data Mining in Pattern Recognition: 9th International Conference, MLDM, Proceedings 9, 169–182. Springer Berlin Heidelberg.

  28. Haddoud, M., Mokhtari, A., Lecroq, T., Abdeddaïm, S. 2016. Combining supervised term-weighting metrics for SVM text classification with extended term representation. Knowledge and Information Systems, 49, 909–931.

  29. Mahto, D., Yadav, S.C. 2022. Hierarchical Bi-LSTM based emotion analysis of textual data.Bulletin of the Polish Academy of Sciences. Technical Sciences,70.

  30. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52ndAnnual Meeting of the Association for Computational Linguistics: System Demonstrations, 55–60.

  31. Mawardi, V.C., Susanto, N., Naga, D.S. 2018. Spelling correction for text documents in Bahasa Indonesia using finite state automata and Levinshtein distance method. In MATEC Web of Conferences, 164, 01047. EDP Sciences.

  32. Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Hodjat, B. 2019. Evolving deep neural networks. InArtificial intelligence in the age of neural networks and brain computing, Academic Press, 293-312.

  33. Mikolov, T., Chen, K., Corrado, G., Dean, J. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301–3781.

  34. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J. 2013. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26. http://qwone.com/~jason/20Newsgroups/

  35. Pahwa, B., Taruna, S., Kasliwal, N. 2018. Sentiment analysis-strategy for text pre-processing. International Journal of Computer Applications, 180, 15–18.

  36. Pennington, J., Socher, R., Manning, C.D. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on empirical methods in Natural Language Processing (EMNLP), 1532–1543.

  37. Prusa, J.D., Khoshgoftaar, T.M. 2016. Designing a better data representation for deep neural networks and text classification. In EEE 17thInternational Conference on Information Reuse and Integration, 411–416.

  38. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1, 9.

  39. Rajpurkar, P., Jia, R., Liang, P. 2018. Know what you don't know: Unanswerable questions for SQuAD. arXiv preprint arXiv:1806.03822.

  40. Saif, H., Fernández, M., He, Y., Alani, H. 2014. On stopwords, filtering and data sparsity for sentiment analysis of twitter.

  41. Sampson, G. 2005. The’LanguageInstinct’Debate: Revised Edition. A&C Black.
  42. Santos, C.D., Tan, M., Xiang, B., Zhou, B.2016. Attentive pooling networks. arXiv preprint arXiv:1602–03609.

  43. Sarzynska-Wawer, J., Wawer, A., Pawlak, A., Szymanowska, J., Stefaniak, I., Jarkiewicz, M., Okruszek, L. 2021. Detecting formal thought disorder by deep contextualized word representations.Psychiatry Research,304, 114135.

  44. Sun, C., Qiu, X., Xu, Y., Huang, X. 2019. How to fine-tune bert for text classification? InChina national conference on Chinese computational linguistics Springer, Cham, 194–206.

  45. Tai, K.S., Socher, R., Manning, C.D. 2015. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503–00075.

  46. Tang, D., Qin, B., Liu, T. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1422–1432.

  47. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Polosukhin, I. 2017. Attention is all you need. Advances in Neural Information Processing Systems, 30.

  48. Wang, G., Li, C., Wang, W., Zhang, Y., Shen, D., Zhang, X., Carin, L. 2018. Joint embedding of words and labels for text classification. arXiv preprint arXiv:1805–04174.

  49. Wang, H., Raj, B.2017. On the origin of deep learning.arXiv preprint arXiv:1702–07800.

  50. Xie, Q., Dai, Z., Hovy, E., Luong, T., Le, Q. 2020. Unsupervised data augmentation for consistency training.Advances in Neural Information Processing Systems,33, 6256–6268.

  51. Yang, Y., Pedersen, J.O. 1997. A comparative study on feature selection in text categorization. In Icml, 97, 35.

  52. Yang, Y., Yih, W.T., Meek, C. 2015. Wikiqa: A challenge dataset for open-domain question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2013–2018.

  53. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E. 2016. Hierarchical attention networks for document classification. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1480–1489.

  54. Zhang, X., Zhao, J., LeCun, Y. 2015. Character-level convolutional networks for text classification. Advances in Neural Information Processing Systems, 28.

  55. Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., Xu, B. 2016. Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling. arXiv preprint arXiv:1611–06639.

  56. Zhou, X., Wan, X., Xiao, J. 2016. Attention-based LSTM network for cross-lingual sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 247–256.


ARTICLE INFORMATION

Received: 2022-12-09
Revised: 2023-01-05
Accepted: 2023-03-08
Available Online: 2023-05-11


Cite this article:

Kumar, N., Yadav, S.C. Comprehensive analysis of deep learning based text classification models and applications. International Journal of Applied Science and Engineering, 20, 2022342. https://doi.org/10.6703/IJASE.202309_20(3).002

  Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Other people also read ...

doaj logo

 

2.3
2022CiteScore
 
 
48th percentile
Powered by  Scopus
 
 

SCImago Journal & Country Rank

IJASE - Most Read Articles

IJASE - Most popular articles