Performance Evaluation and Improvement of High-Dimensional Optimization Algorithms based on Bayesian Model

DOI: https://doi.org/10.62517/jes.202502301

Author(s)

Xuefeng Liu*

Affiliation(s)

West Yunnan University of Applied Sciences Department of Public Foundation Studies, Pu'er, Yunnan, China

Abstract

Looking at the complex face of multi-dimensional optimization problems, traditional optimization techniques face the dual limitations of efficiency and accuracy. This experiment aims to improve the execution level of high-dimensional optimization algorithms through the Bayesian optimization approach. It integrates multi-scale modeling, refined sampling algorithms, improved kernel function construction and calculation speed optimization methods, and conducts an empirical analysis of the optimization effect of SVM algorithm parameters on the MNIST dataset. Bayesian optimization significantly enhances the efficiency and classification accuracy of the algorithm in multi-dimensional parameter space search. The accuracy of the optimized SVM classifier climbs to 97.2%. Empirical research confirms that Bayesian optimization shows excellent performance in high-dimensional optimization tasks. In resource-limited computing situations, it is an instant and efficient parameter optimization solution. Future research will focus on exploring the core secrets of reducing computational complexity and extending the application boundaries to the field of multi-dimensional machine learning.

Keywords

High-Dimensional Optimization; Bayesian Optimization; Support Vector Machine; MNIST Dataset

References

[1] Binois M, Wycoff N. A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization[J]. ACM Transactions on Evolutionary Learning and Optimization, 2022, 2(2): 1-26. [2] Maddox WJ, Balandat M, Wilson AG, et al. Bayesian optimization with high-dimensional outputs[J]. Advances in neural information processing systems, 2021, 34: 19274-19287. [3] Moriconi R, Deisenroth MP, Sesh Kumar K S. High-dimensional Bayesian optimization using low-dimensional feature spaces[J]. Machine Learning, 2020, 109: 1925-1943. [4] Wu J, Chen XY, Zhang H, et al. Hyperparameter optimization for machine learning models based on Bayesian optimization[J]. Journal of Electronic Science and Technology, 2019, 17(1): 26-40. [5] Raponi E, Wang H, Bujny M, et al. High dimensional Bayesian optimization assisted by principal component analysis[C]//Parallel Problem Solving from Nature–PPSN XVI: 16th International Conference, PPSN 2020, Leiden, The Netherlands, September 5- 9, 2020, Proceedings, Part I 16. Springer International Publishing, 2020: 169-183.