I have not seen mentioned so far in this thread the following book, which I can't recommend more highly: Understanding Machine Learning: From Theory To Algorit

I have not seen mentioned so far in this thread the following book, which I can't recommend more highly: Understanding Machine Learning: From Theory To Algorithms – Shai Shalev-Shwartz