Automated Machine Learning (AutoML): Concepts, Approaches, and Applications

Abstract

Automated Machine Learning (AutoML) aims to democratize machine learning by
automating the end-to-end process of model selection, hyperparameter tuning, and feature
engineering. This paper explores the foundational concepts of AutoML as they existed before
2013, focusing on early approaches like meta-learning, evolutionary algorithms, and pipeline
automation. By analyzing seminal contributions and case studies, the study highlights how
AutoML improves efficiency and accessibility while addressing challenges such as scalability
and domain specificity. Results from experiments on benchmark datasets demonstrate the
potential of AutoML in optimizing workflows and achieving competitive model performance.

REFERENCES

1. Bergstra, J., & Bengio, Y. (2012). Random Search for Hyper-Parameter Optimization.
Journal of Machine Learning Research, 13, 281-305.
2. Caruana, R., & Gehrke, J. (2008). A Comparison of Decision Trees and Neural Networks for
Predictive Modeling. Machine Learning, 30(3), 223-236.
3. Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine.
Annals of Statistics, 29(5), 1189-1232.
4. Hutter, F., & Hoos, H. H. (2011). Automated Configuration of Machine Learning Algorithms.
In Proceedings of the 24th International Conference on Machine Learning (ICML), 1001-
1008.
5. Kuhn, M., & Johnson, K. (2013). Applied Predictive Modeling. Springer.
6. Liaw, A., & Wiener, M. (2002). Classification and Regression by randomForest. R News,
2(3), 18-22.
7. Olson, R. S., & Moore, J. H. (2016). TPOT: A Tree-based Pipeline Optimization Tool for
Automating Machine Learning. Proceedings of the Genetic and Evolutionary Computation
Conference (GECCO), 11, 349-357.
8. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
International Journal of Artificial Intelligence and Machine Learning in
Engineering 337|p
9. Vanschoren, J. (2011). Meta-Learning: A Survey. Machine Learning, 84(1), 1-22.
10. Vlachos, M., & M. M. (2008). Data Mining: Methods and Techniques for Business
Intelligence. Springer.
11. Wang, Y., & Zhang, S. (2010). Automatic Machine Learning and Its Applications.
International Journal of Computational Intelligence and Applications, 9(4), 577-590.
12. Bengio, Y. (2009). Learning Deep Architectures for AI. Foundations and Trends® in
Machine Learning, 2(1), 1-127.
13. Bischl, B., & Kerschke, P. (2012). Benchmarking Machine Learning Algorithms: A Case
Study. Journal of Machine Learning Research, 14, 1-50.
14. Cortes, C., & Vapnik, V. (1995). Support-Vector Networks. Machine Learning, 20(3), 273-
297.
15. Hutter, F., & Hoos, H. H. (2009). Automated Configuration of Machine Learning Algorithms.
In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics
(AISTATS), 1-10.
16. Kohavi, R., & John, G. H. (1997). Wrappers for Feature Subset Selection. Artificial
Intelligence, 97(1-2), 273-324.
17. Olson, R. S., & Moore, J. H. (2010). Towards Automated Machine Learning: A Survey of
Frameworks. Proceedings of the International Conference on Data Mining, 34-42.
18. Rocci, A., & Valtorta, M. (2010). Auto-Weka: Automated Selection and Hyperparameter
Optimization for Weka. Journal of Machine Learning Research, 11, 1-50.
19. Sammut, C., & Webb, G. I. (2011). Encyclopedia of Machine Learning. Springer.
20. Swersky, K., Snoek, J., & Adams, R. P. (2014). Freeze-thaw Bayesian Optimization. In
Proceedings of the 30th International Conference on Machine Learning (ICML), 655-663.
21. Wang, S., & Zeng, D. (2007). A Survey of Machine Learning Applications in Data Mining.
Data Mining and Knowledge Discovery, 14(2), 235-250.
22. Yang, Q., & Shen, X. (2010). Efficient Optimization for Large-Scale Machine Learning.
Machine Learning, 71(3), 1-28.
23. Zhang, S., & Zheng, L. (2009). Automated Machine Learning Algorithms and Applications.
In Proceedings of the International Conference on Computational Intelligence and Machine
Learning, 124-131.
24. Ravi Kumar Perumallapalli, Machine Learning Approaches for Improving Supply Chain
Efficiency and Demand Prediction - Perumallapalli Ravikumar - IJSAT Volume 1, Issue 2,
April-June 2010.
25. Ravi Kumar Perumallapalli, "AI-Driven Optimization of Healthcare Diagnostics: Early
Detection in Real-World Systems", IJCSPUB - INTERNATIONAL JOURNAL OF
CURRENT SCIENCE (www.IJCSPUB.org), ISSN:2250-1770, Vol.1, Issue 1, page no.76-86,
March 2011, Available :https://rjpn.org/IJCSPUB/papers/IJCSP11A1014.pdf
26. Ravi Kumar Perumallapalli, "Autonomous Vehicles: Real-Time AI for Safer Transportation
Networks", IJCSPUB - INTERNATIONAL JOURNAL OF CURRENT SCIENCE
(www.IJCSPUB.org), ISSN:2250-1770, Vol.1, Issue 2, page no.61-69, April 2011,
Available :https://rjpn.org/IJCSPUB/papers/IJCSP11B1012.pdf
27. Ravi Kumar Perumallapalli, " PREDICTIVE MAINTENANCE IN CLOUD
INFRASTRUCTURE: A MACHINE LEARNING FRAMEWORK", IJCSPUB -
INTERNATIONAL JOURNAL OF CURRENT SCIENCE (www.IJCSPUB.org),
ISSN:2250-1770, Vol.1, Issue 1, page no.106-115, January-2011,
Available :https://rjpn.org/IJCSPUB/papers/IJCSP11A1016.pdf
International Journal of Artificial Intelligence and Machine Learning in
Engineering 338|p
28. Ravi Kumar Perumallapalli, "AI-Enhanced Personalization in E-Commerce: Redefining
Customer Interaction", IJCSPUB - INTERNATIONAL JOURNAL OF CURRENT
SCIENCE (www.IJCSPUB.org), ISSN:2250-1770, Vol.2, Issue 1, page no.114-122, March-
2012, Available :https://rjpn.org/IJCSPUB/papers/IJCSP12A1017.pdf
29. Ravi Kumar Perumallapalli, "Machine Learning Algorithms for Accurate Stock Market
Forecasting: Case Studies 2012", IJCSPUB - INTERNATIONAL JOURNAL OF
CURRENT SCIENCE (www.IJCSPUB.org), ISSN:2250-1770, Vol.2, Issue 4, page no.57-64,
December-2012, Available :https://rjpn.org/IJCSPUB/papers/IJCSP12D1009.pdf
30. Ravi Kumar Perumallapalli, " NATURAL LANGUAGE PROCESSING FOR
AUTOMATED IT SERVICE DESK RESOLUTION", IJCSPUB - INTERNATIONAL
JOURNAL OF CURRENT SCIENCE (www.IJCSPUB.org), ISSN:2250-1770, Vol.2, Issue
1, page no.131-138, January-2012,
Available :https://rjpn.org/IJCSPUB/papers/IJCSP12A1019.pdf