Development of a hyperparameter optimization method for recommendatory models based on matrix factorization
DOI:
https://doi.org/10.15587/1729-4061.2021.239124Keywords:
Bayesian optimization, Gaussian process, covariance function, matrix factorization, recommendatory systemsAbstract
Many advanced recommendatory models are implemented using matrix factorization algorithms. Experiments show that the quality of their performance depends significantly on the selected hyperparameters. Analysis of the effectiveness of using various methods for solving this problem of optimizing hyperparameters was made. It has shown that the use of classical Bayesian optimization which treats the model as a «black box» remains the standard solution. However, the models based on matrix factorization have a number of characteristic features. Their use makes it possible to introduce changes in the optimization process leading to a decrease in the time required to find the sought points without losing quality.
Modification of the Gaussian process core which is used as a surrogate model for the loss function when performing the Bayesian optimization was proposed. The described modification at first iterations increases the variance of the values predicted by the Gaussian process over a given region of the hyperparameter space. In some cases, this makes it possible to obtain more information about the real form of the investigated loss function in less time.
Experiments were carried out using well-known data sets for recommendatory systems. Total optimization time when applying the modification was reduced by 16 % (or 263 seconds) at best and remained the same at worst (less than 1-second difference). In this case, the expected error of the recommendatory model did not change (the absolute difference in values is two orders of magnitude lower than the value of error reduction in the optimization process). Thus, the use of the proposed modification contributes to finding a better set of hyperparameters in less time without loss of quality
References
- Ricci, F., Rokach, L., Shapira, B., Kantor, P. B. (Eds.) (2011). Recommender Systems Handbook. Springer, 842. doi: https://doi.org/10.1007/978-0-387-85820-3
- Rendle, S., Zhang, L., Koren, Y. (2019). On the Difficulty of Evaluating Baselines: A Study on Recommender Systems. arXiv.org. Available at: https://arxiv.org/abs/1905.01395
- Dacrema, M. F., Cremonesi, P., Jannach, D. (2019). Are we really making much progress? A worrying analysis of recent neural recommendation approaches. Proceedings of the 13th ACM Conference on Recommender Systems. doi: https://doi.org/10.1145/3298689.3347058
- Aggarwal, C. C. (2016). Recommender Systems. Springer, 498. doi: https://doi.org/10.1007/978-3-319-29659-3
- Fathan, G., Bharata Adji, T., Ferdiana, R. (2018). Impact of Matrix Factorization and Regularization Hyperparameter on a Recommender System for Movies. Proceeding of the Electrical Engineering Computer Science and Informatics, 5 (5), 113–116. doi: https://doi.org/10.11591/eecsi.v5i5.1685
- Harper, F. M., Konstan, J. A. (2016). The MovieLens Datasets. ACM Transactions on Interactive Intelligent Systems, 5 (4), 1–19. doi: https://doi.org/10.1145/2827872
- Galuzzi, B. G., Giordani, I., Candelieri, A., Perego, R., Archetti, F. (2020). Hyperparameter optimization for recommender systems through Bayesian optimization. Computational Management Science, 17 (4), 495–515. doi: https://doi.org/10.1007/s10287-020-00376-3
- Nechaev, A. A., Meltsov, V. Yu. (2021). Investigating the hyperparameter configuration space of matrix factorization recommendation models. Nauchno-tekhnicheskiy vestnik Povolzh'ya, 5, 96–100. Available at: https://elibrary.ru/item.asp?id=46124660
- Tran, T., Lee, K., Liao, Y., Lee, D. (2018). Regularizing Matrix Factorization with User and Item Embeddings for Recommendation. Proceedings of the 27th ACM International Conference on Information and Knowledge Management. doi: https://doi.org/10.1145/3269206.3271730
- Feurer, M., Hutter, F. (2019). Hyperparameter Optimization. The Springer Series on Challenges in Machine Learning, 3–33. doi: https://doi.org/10.1007/978-3-030-05318-5_1
- Snoek, J., Larochelle, H., Adams, R. P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. arXiv.org. Available at: https://arxiv.org/abs/1206.2944
- McLeod, M., Osborne, M. A., Roberts, S. J. (2018). Practical Bayesian Optimization for Variable Cost Objectives. arXiv.org. Available at: https://arxiv.org/abs/1703.04335
- Chen, Y., Chen, B., He, X., Gao, C., Li, Y., Lou, J.-G., Wang, Y. (2019). λOpt. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. doi: https://doi.org/10.1145/3292500.3330880
- Xian, Z., Li, Q., Li, G., Li, L. (2017). New Collaborative Filtering Algorithms Based on SVD++ and Differential Privacy. Mathematical Problems in Engineering, 2017, 1–14. doi: https://doi.org/10.1155/2017/1975719
- Shi, W., Wang, L., Qin, J. (2020). User Embedding for Rating Prediction in SVD++-Based Collaborative Filtering. Symmetry, 12 (1), 121. doi: https://doi.org/10.3390/sym12010121
- Cline, A. K., Dhillon, I. S. (2006). Computation of the Singular Value Decomposition. Handbook of Linear Algebra, 45-1–45-13. doi: https://doi.org/10.1201/9781420010572-45
- Nechaev, A. (2021). Speeding up Bayesian Optimization of Matrix Factorization Recommender Models Hyperparameters. GitHub. Available at: https://github.com/dapqa/speeding-up-bo-for-cf-public
- Optimized-for-speed Eigen implementations of SVD, SVD++ and TimeSVD++ algorithms. Available at: https://github.com/dapqa/svdistic
- Nogueira, F. (2014). Bayesian Optimization: Open source constrained global optimization tool for Python. Available at: https://github.com/fmfn/BayesianOptimization
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Alexander Nechaev, Vasily Meltsov, Dmitry Strabykin
This work is licensed under a Creative Commons Attribution 4.0 International License.
The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.
A license agreement is a document in which the author warrants that he/she owns all copyright for the work (manuscript, article, etc.).
The authors, signing the License Agreement with TECHNOLOGY CENTER PC, have all rights to the further use of their work, provided that they link to our edition in which the work was published.
According to the terms of the License Agreement, the Publisher TECHNOLOGY CENTER PC does not take away your copyrights and receives permission from the authors to use and dissemination of the publication through the world's scientific resources (own electronic resources, scientometric databases, repositories, libraries, etc.).
In the absence of a signed License Agreement or in the absence of this agreement of identifiers allowing to identify the identity of the author, the editors have no right to work with the manuscript.
It is important to remember that there is another type of agreement between authors and publishers – when copyright is transferred from the authors to the publisher. In this case, the authors lose ownership of their work and may not use it in any way.
