Dang Tien Dat 1, Nguyen Anh Minh 1, Tran Ngoc Thang 1, Rung-Ching Chen 2, Nguyen Linh Giang 3, Nguyen Thi Ngoc Anh 1*

1 Faculty of Mathematics and Informatics, Hanoi University of Science and Technology, Vietnam
2 Department of Information Management, Chaoyang University of Technology, Taiwan
3 School of Information and Communication Technology, Hanoi University of Science and Technology, Vietnam

 

Download Citation: |
Download PDF


ABSTRACT

Our study focuses on the diversity of user preferences and the dynamics of the user-product relationship, particularly in the context of periodic product usage. The principal objective of this research is to explore multi-objective optimization for a recommendation system tailored to periodic products. Our methodology employs a multi-objective reinforcement learning (MORL) algorithm. Additionally, we have proposed integrating the optimistic linear support algorithm into a MORL algorithm to collect good weight vectors. We also proposed using user clustering to ensure the model remembers user’s preferences in early episodes. The findings of this research demonstrate that our proposed multi-objective approach yields significantly higher effectiveness when contrasted with conventional single-objective methodologies.


Keywords: Deep reinforcement learning, Multi-objective recommendation system, Optimistic linear support, Periodic product


Share this article with your colleagues

 


REFERENCES

  1. Afsar, M.M., Crump, T., Far, B., 2022. Reinforcement learning based recommender systems: A survey. ACM Computing Surveys, 55(7), 1–38.

  2. Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., Zaremba, W., 2017. Hindsight experience replay. Advances in Neural Information Processing Systems, 30.

  3. Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A., 2017. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), 26–38.

  4. Chen, L., Zhu, G., Liang, W., Wang, Y., 2023. Multi-objective reinforcement learning approach for trip recommendation. Expert Systems with Applications, 226, 120145.

  5. Covington, P., Adams, J., Sargin, E., 2016. Deep neural networks for YouTube recommendations. In Proceedings of the 10th ACM conference on recommender systems, pp. 191–198.

  6. Cui, L., Ou, P., Fu, X., Wen, Z., Lu, N., 2017. A novel multi-objective evolutionary algorithm for recommendation systems. Journal of Parallel and Distributed Computing, 103, 53–63.

  7. Gomez-Uribe, C.A., Hunt, N., 2015. The Netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems, 6(4), 1–19.

  8. Hayes, C.F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., Roijers, D.M., 2022. A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agent Systems, 36(1), 26.

  9. Hochreiter, S., Schmidhuber, J., 1997. Long short-term memory. Neural Computation, 9(8), 1735–1780.

  10. Hou, Y., Gu, W., Dong, W., Dang, L., 2023. A deep reinforcement learning real-time recommendation model based on long and short-term preference. International Journal of Computational Intelligence Systems, 16(1), 4.

  11. Hu, Y., Da, Q., Zeng, A., Yu, Y., Xu, Y., 2018. Reinforcement learning to rank in e-commerce search engine: Formalization, analysis, and application. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 368–377.

  12. Jahn, J., 1985. Scalarization in multi objective optimization. Springer Vienna, pp. 45–88.

  13. Joachims, T., Freitag, D., Mitchell, T., 1997. Webwatcher: A tour guide for the world wide web. International Joint Conference on Artificial Intelligence, pp. 770–777.

  14. Keat, E.Y., Sharef, N.M., Yaakob, R., Kasmiran, K.A., Marlisah, E., Mustapha, N., Zolkepli, M., 2022. Multiobjective deep reinforcement learning for recommendation systems. IEEE Access, 10, 65011–65027.

  15. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D., 2015. Continuous control with deep reinforcement learning. International Conference on Learning Representations, abs/1509.02971.

  16. Linden, G., Smith, B., York, J., 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, 7(1), 76–80.

  17. Liu, F., Tang, R., Li, X., Ye, Y., Chen, H., Guo, H., Zhang, Z., 2018. Deep reinforcement learning based recommendation with explicit user-item interactions modeling. Neurocomputing, 307, 139–150.

  18. Mulani, J., Heda, S., Tumdi, K., Patel, J., Chhinkaniwala, H., Patel, J., 2020. Deep reinforcement learning based personalized health recommendations. “Deep learning techniques for biomedical and health informatics”, Springer, pp. 231–255.

  19. Paparella, V., Anelli, V.W., Boratto, L., Di Noia, T., 2023. Reproducibility of multiobjective reinforcement learning recommendation: Interplay between effectiveness and beyond-accuracy perspectives. In Proceedings of the 17th ACM Conference on Recommender Systems, pp. 467–478.

  20. Roijers, D.M., Vamplew, P., Whiteson, S., Dazeley, R., 2013. A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48, 67–113.

  21. Roijers, D.M., Whiteson, S., Oliehoek, F.A., 2015. Computing convex coverage sets for faster multi-objective coordination. Journal of Artificial Intelligence Research, 52, 399–443.

  22. Sarker, A., Shen, H., Kowsari, K., 2020. A data-driven reinforcement learning based multiobjective route recommendation system. IEEE 17th international conference on mobile ad hoc and sensor systems (mass), pp. 103–111.

  23. Sherstinsky, A., 2020. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena, 404, 132306.

  24. Song, L., Tekin, C., Van Der Schaar, M., 2014. Online learning in large-scale contextual recommender systems. IEEE Transactions on Services Computing, 9(3), 433–445.

  25. Stamenkovic, D., Karatzoglou, A., Arapakis, I., Xin, X., Katevas, K., 2022. Choosing the best of both worlds: Diverse and novel recommendations through multi-objective reinforcement learning. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 957–965.

  26. Sutton, R.S., Barto, A.G., 2018. Reinforcement learning: An introduction. MIT press.

  27. Von Lucken, C., Baran, B., Brizuela, C., 2014. A survey on multi-objective evolutionary algorithms for many-objective problems. Computational optimization and applications, 58, 707–756.

  28. Wang, Z., Schaul, T., Hessel, M., Hasselt, H.V., Lanctot, M., De Freitas, N., 2016. Dueling network architectures for deep reinforcement learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning-Volume 48, pp. 1995–2003.

  29. Wu, H., Ma, C., Mitra, B., Diaz, F., Liu, X., 2022. A multi-objective optimization framework for multi-stakeholder fairness-aware recommendation. ACM Transactions on Information Systems, 41(2), 1–29.

  30. Yalnizyan-Carson, A., Richards, B.A., 2022. Forgetting enhances episodic control with structured memories. Frontiers in Computational Neuroscience, 16, 757244.

  31. Yang, R., Sun, X., Narasimhan, K., 2019. A generalized algorithm for multi-objective reinforcement learning and policy adaptation. Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA, Article 1311, 14636–14647.

  32. Zhao, X., Zhang, L., Xia, L., Ding, Z., Yin, D., Tang, J., 2017. Deep reinforcement learning for list-wise recommendations. In Proceedings of the 12th ACM Conference on Recommender Systems. Association for Computing Machinery, New York, NY, USA, 95–103.

  33. Zheng, G., Zhang, F., Zheng, Z., Xiang, Y., Yuan, N.J., Xie, X., Li, Z., 2018. DRN: A deep reinforcement learning framework for news recommendation. In Proceedings of the 2018 world wide web conference, pp. 167–176.

  34. Zheng, Y., Wang, D.X., 2021. A survey of recommender systems with multi-objective optimization. Neurocomputing, 474, 141–153.


ARTICLE INFORMATION

Received: 2023-12-31
Revised: 2024-03-03
Accepted: 2024-06-03
Available Online: 2024-08-26


Cite this article:

Dat, D.T., Anh Minh, N., Thang, T.N., Chen, R.C., Linh Giang, N., Thi Ngoc Anh, N. 2024. Building the multi-objective periodic recommendation system through integrating optimistic linear support and user clustering to multi-object reinforcement learning. International Journal of Applied Science and Engineering, 21, 2023542. https://doi.org/10.6703/IJASE.202406_21(3).003

  Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Other people also read ...