Popovich, P.; Fotopoulos, G.; Papadopoulos, N. Review Non-convex Optimization Method for Machine Learning. Preprints2024, 2024100278. https://doi.org/10.20944/preprints202410.0278.v1
APA Style
Popovich, P., Fotopoulos, G., & Papadopoulos, N. (2024). Review Non-convex Optimization Method for Machine Learning. Preprints. https://doi.org/10.20944/preprints202410.0278.v1
Chicago/Turabian Style
Popovich, P., Greg Fotopoulos and Nicholas Papadopoulos. 2024 "Review Non-convex Optimization Method for Machine Learning" Preprints. https://doi.org/10.20944/preprints202410.0278.v1
Abstract
Non-convex optimization is a critical tool in advancing machine learning, especially for complex models like deep neural networks and support vector machines. Despite challenges such as multiple local minima and saddle points, non-convex techniques offer various pathways to reduce computational costs. These include promoting sparsity through regularization, efficiently escaping saddle points, and employing subsampling and approximation strategies like stochastic gradient descent. Additionally, non-convex methods enable model pruning and compression, which reduce the size of models while maintaining performance. By focusing on good local minima instead of exact global minima, non-convex optimization ensures competitive accuracy with faster convergence and lower computational overhead. This paper examines the key methods and applications of non-convex optimization in machine learning, exploring how it can lower computation costs while enhancing model performance. Furthermore, it outlines future research directions and challenges, including scalability and generalization, that will shape the next phase of non-convex optimization in machine learning.
Keywords
Deep Neural Networks; Support Vector Machine; Non-convex Optimization; Gradient Method \and Machine Learning
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
