Machine learning today is a major subfield of artificial intelligence, and a field that has significant applications in other subfields of AI and tech in general, such as in natural language processing and computer vision. In this post, I hold that the definition of this subject, at least as it is given today, is fundamentally informal. There are a couple candidates for definitions that we will consider.
First, as of February 13, 2021, Wikipedia defines ML as the study of algorithms which improve with experience. (This is also the definition I have adopted.) But if we wanted to formalize this, how would we formally define “improvement”? We could try to model this with an objective function, but the resulting formalization may be too narrow in scope, since ML can also include things such as unsupervised learning where we may not have explicit objective functions.
At this point, I thought of defining ML as the study of algorithms which “update state according to input data,” but these also appear with databases, which we don’t consider to be part of ML.
Another definition is the one provided by Andrew Ng, as the science of getting computers to perform tasks they were not explicitly programmed for. But this is an informal specification. In terms of being programmed, all algorithms need to still be programmed at some level: all algorithms, including ML algorithms, still need to be written in code. Thus, what we consider “explicit” here would be an informal judgment.
It therefore seems that, while we have common formal concepts like models and are able to understand informally what ML is about, a formal definition may still escape us.
Edited in 2025.
