Inappropriate Mathematics for Machine Learning

Inappropriate Mathematics for Machine Learning: Reviewers and students are sometimes greatly concerned by the distinction between:

1. An open set and a closed set.
2. ASupremum and a Maximum.
3. An event which happens with probability 1 and an event that always happens.

I don’t appreciate this distinction in machine learning & learning theory. All machine learning takes place (by definition) on a machine where every parameter has finite precision. Consequently, every set is closed, a maximal element always exists, and probability 1 events always happen.

The fundamental issue here is that substantial parts of mathematics don’t appear well-matched to computation in the physical world, because the mathematics has concerns which are unphysical. (Via Machine Learning (Theory).)

John is deeply confused here. One of the most important jobs of mathematical abstraction is to make it easier to reason about whole classes of problems whose size is data dependent. That's why we use Turing machines in the theory of computation, and the continuum when working with approximations and rates of change whose precision cannot be specified in advance. Constructive mathematics, however clever, is a niche pursuit because it makes simple arguments about rates, approximation, and existence much harder than they are with Weierstrass epsilon-delta arguments and their hugely successful development in analysis, topology, probability, and applied mathematics. It would be wonderful if there was a constructive mathematics that was as easy to work with as those classical areas. I spent quite a bit of time and effort studying intuitionistic and constructivist methods a long while back, and my frustrated conclusion then was that these approaches that are supposedly closer to computation actually make the simplest classical arguments a huge chore, for very uncertain payoff.

Sticking my neck out: constructivism is misguided because it believes in a single fabric for mathematics. It refuses to accept that mathematics is a patchwork of methods that work at different levels of abstraction and are not fully inter-translatable. Which is not surprising if you recognize that mathematics is a big messy workshop of tools for abstract thought, not the incomplete projection of a Platonic ideal.