Learning From Data

This book helped me understand everything from K-nearest neighbors to deep neural networks. Professor Yaser presents a clear and well-written book for students and anyone interested in machine learning. Additionally, he allows anyone with the book to access his online resources at no additional fee. I can't say enough good things about this book!!

Not good for starters in data science/machine learning. All mathematical solutions to theorems are not explained very well. Too many assumptions unjustified or too complicated explanation for simple concepts. 2 stars for topics covered.

This is a concise book. It focuses on principally linear models of supervised learning. It provides important conceptual insights into machine learing and particularly the trade-off between overfitting and underfitting. It looks at this from a number of frameworks. The writing is clear. However, I was left wanting for worked examples and was disappointed that support vector machines and kernel based methods were not explored (thught they are mentioned).

This book deserves 6 stars plus! The authors self-published a beautiful hard back book with color illustrations that is a terrific gem.
I couldn't be happier with the book, which includes a wonderful compilation of the mathematical notation as well as an excellent
biography.

Thanks to all three co-authors!!!!

Learning From Data is a book on learning theory and few learning algorithms. The book also describes the different methodologies used to evaluate models. In my opinion, this book is not suitable for beginners as it delves into abstract notions that may not be relevant for people who are just coming into the field. I think that this book is for people who already had exposure to machine learning and who are willing to take their understanding to the next step.
In conclusion; if you are a newbie avoid this book, otherwise if you already had exposure to machine learning it's a good resource to understand machine learning theory in more depth.

The authors present a course that most students would love to be able to take. Since only a few Universities can provide such quality in teaching, this book clearly is a must-have for students - and I think also for many lecturers - to improve their overview of the matter.

TLDR Summary: If Machine Learning is like Mechanics, "Learning from Data" teaches you Newton's Laws!
---------------------------------

Machine Learning (ML), Data Mining (DM), Predictive Modeling, Big Data, Statistical Inference, Pattern Recognition, Regression, Classification: by whichever name you call it, you will find hundreds of books by the same name, and in theoretical as well as applied avatars. The applied ones tend to be books based on ML/DM programming libraries such as R, Weka (Java), and SciPy/NumPy (Python) and really are not meant to teach you the underlying foundations but I digress too soon.

I possess the standard three introductory texts in ML: Pattern Classification (Duda, Hart, Stork) , Pattern Recognition (Bishop) and Machine Learning (Mitchell). In addition, I have read portions of Statistical Learning (Hastie), Machine Learning (Alpaydin), Support Vector Machines (Cristianini) and several other allied ML texts in natural language processing, convex optimization etc.

In spite of being considered the classic introductory texts in ML, all these books failed in the task of making me understand what I was doing as I was practicing ML. Try as I might, I could never read through more than a few tens of pages of the afore mentioned books. And what little I read, could not be retained by my feeble brain for too long.

But where all these texts failed, "Learning From Data" (LFD) succeeds.

First an analogy:

It is all fine to arm someone with equations of cantilever beams and have them build houses, but clearly we don't want a civil engineer who doesn't understand Newton's laws to build our own house. Most well known books in ML read to me like course readers of advanced Mechanics courses stitched together. LFD on the other hand feels like a book on Newton's Laws and Applications.

Writing Style:

The book serves as the reading counterpart to a set of eighteen one-hour video lectures that was presented in a course by the first author Yaser at Caltech. The book reads almost like a transcription of the lectures. The authors are always addressing you and manage to convey the feeling that they are holding your hand and actively helping you to learn how to walk. I found the style very engaging. Once I started reading the book, it did not take any special effort for me to finish it (which was the difficulty with the other classic ML books).

The videos are freely available online(Google for 'learning from data caltech course'). I strongly encourage the readers of this book to first watch each video and then read the corresponding chapters of the book.

Content:

There are five central themes underlying the organization and presentation of topics in this book:

1. What is Learning?
2. Is Learning possible?
3. How to Learn?
4. How to Learn well?
5. Take Home Lessons.

The authors follow a style of gradual expansion from simple to complex concepts throughout the book. E.g. Under the topic of "Is Learning feasible", they first derive a probability on the upper bound of the out-of-sample error using a thought experiment and the Hoeffding's Inequality. Then they reason that if one of the components of this probability is polynomial in the number of training examples, the error can be bounded. Finally they introduce the VC dimension and prove that in cases where it is finite, learning is truly feasible.

Throughout the book, the authors provide plenty of real life application scenarios and experimentally generated examples to illustrate the theory. I found the theory when put to practice (even if in a toy example) very useful, particularly when visualized through the various graphs. There are several gems scattered around the book in the form of subtle things that can be overlooked even by a smart person (such as inadvertent data snooping) and rules-of-thumb for practical applications.

The authors have clearly had to make some choices about what to focus on and what to omit. For example, the book has no mention of Bayesian Decision Theory or Naive Bayes classification. This appears shocking upon first glance since Naive Bayes is often the first learning algorithm taught in an introductory course on pattern recognition. But after going though Yaser's book/course such omissions appears to be a virtue. It is not the focus of this book to teach you everything ML. If this is what you are looking for, LFD is not for you; Kevin's Murphy's forthcoming text appears promising. LFD however, gives you enough of a foundation that should you wish to educate yourself on advance topics like bootstrap aggregation, probabilistic graphical models, or ensemble-learning, you are sufficiently prepared.

The icing on the cake is the forum provided by the authors to discuss the book (and the lectures). Yaser has personally answered all of my questions, sometimes at 3AM, Pasadena time!

Final note on book quality:

The color printing, binding and paper quality are all excellent. The authors could have paid more attention to detail to some portions of the book (such as using high-contrast, colorblind-friendly colors in the illustrations) but honestly, this is just me being extremely an*l. The hardbound book at this low price of approximately $30 is pure value for money. Wide dissemination of the book contents appears to be a clear motivation.

PS: If the authors are reading this, they should look up "Ishihara test plates" and compare that with the illustration of red-green marbles on page 22 etc.

The book lives up to its tagline: "A short course, not a hurried course." It teaches you the foundations of machine learning, but only very simple practical applications. Nonetheless, the book gives an excellent introduction to the general framework of machine learning.

The book accompanies Caltech's online course of the same name https://telecourse.caltech.edu/index.php
Pretty much in line with the online lectures. The course covers machine learning in general and focuses on the theory of learning with introductory material on various learning algorithms incorporated into the chapters as they become relevant. I'm giving this 4 stars because it is indeed a short course on learning from data as the cover says. This is not an in-depth book on learning algorithms although it does of course cover some of these in reasonable depth but always from a computer science angle towards the theory of learning rather than in a practical applied 'engineering' light. The book and the online course itself is actually of very high quality (despite being free); the professor's lectures are very well structured, organised and explained. Finally, the price is OK but a little high given its total knowledge content - although addmitedly I am maybe a little miserly :)

This book is a lot more about theory than application. It also involves heavy math. I purchased this because I was generally interested in Machine Learning, and it was ranked highly. However, it was a disappointment because I want to be a _user_ of machine learning, not a researcher of the topic.

This book is different.

This book is not about introducing techniques. At least this is what I feel. Today when people claim that they know machine learning, sometimes they actually only know how to run the code and maybe remember some equations. Implementation is easy but theory is hard. I think this book provides an easy look for the hardest part of machine learning, which is actually the basis for all learning techniques.

In a 1657 letter, Blaise Pascal, apologizing for its length, wrote, "I made it longer because I did not have time to make it shorter." To a secondary-school essayist struggling to meet a minimum page count on an assignment, this may seem counterintuitive. But to professionals who make their living writing, whether out of love or necessity, Pascal's quote speaks to the great difficulty inherent in covering a topic concisely.

"Learning from Data: A Short Course" is a remarkable achievement not only for how completely it succeeds at covering its topic clearly and concisely but also for how much of a pleasure the end product is to read. Professors Abu-Mostafa, Magdon-Ismail, and Lin have produced a beautifully written, user-friendly book that, in my opinion, deserves to be a standard introduction to the field.

Among the book's major selling points:

1. In well under 200 pages, the authors manage to clearly impart the insights underlying statistical learning theory (which, in a nutshell, is a probabilistic justification for why machine learning is truly worth doing). Much of what you'll find here would otherwise come from digesting and distilling many other (much longer) texts.

2. Machine learning is a huge field with a vast literature and many techniques. However, most of those techniques are concerned with applying the same handful of principles (error minimization, regularization, etc.), albeit in slightly different ways. Those principles make up the book's core content.

3. How often do you see a book in this field, especially one with full-color illustrations, for this price? The book was apparently published under the authors' own imprint, and rather than try to make a killing financially, they clearly want to get the book into students' hands.

4. The book was written with the student in mind. The exposition is crystal clear, and the within-text exercises (offset so as to not disrupt the flow) are instructive and very well constructed. The end-of-chapter problems are also very well chosen and do not just offload the effort of proving results to the reader. In cases where proofs are asked for in problems, they are often broken into steps that offer insight not only into the problem but also into the process of proving results in this area.

5. The book looks great. The illustrations are very well designed, and color is used well.

6. This is a remarkably error-free book. Not only is the English perfect and the prose engaging, but the technical material has clearly been carefully checked. The only errors I detected are trivial, and none affect the understanding of the material. (For instance, there is an accidental reference in the text to an incorrect figure number. No big deal.) I have read the main text, exercises, and a portion of the end-of-chapter problems; if there are errors elsewhere, they are well hidden.

7. Again, this is very reader-friendly writing (with even the occasional smiley thrown in, which I found rather endearing). But this is far from diluted or dumbed-down material. It is exactly as technical as it needs to be. As a result, it has one of the widest potential audiences of any book in the field that I can think of. Although the authors teach this material at universities with reputations for the technical skills of their students (Caltech, RPI, and NTU), I believe that even a motivated and talented high-school student would have success working from this text. (Some indication of the flavor of the material and its presentation comes from Prof. Abu-Mostafa's filmed class lectures, which are freely available online.)

Perhaps had I taken more time, I could have written a shorter review. But I cannot praise this book highly enough, nor can I adequately express my admiration for the obvious effort the authors put into this work.

When I first read this book I had high expectations, I was lucky to have Yaser as one of my best teachers at Caltech during my PhD and I still remember his energy and passion.

Well, the book does translate into printed words the passion for "really understanding a subject" that he and his co-authors share in their professional life.

By "really understanding" they mean understanding the foundations of learning from data but also going beyond abstractions to give flesh and blood to ideas. Motivation always anticipates the definition of concepts, and after concepts are formulated, the discussion continues to "really understand" the meaning of equations and theorems.
The topics contained in the book are limited ("a short course, not a hurried course") because of an explicit choice: if one understands the meaning, implications, and pitfalls of learning from data in simple scenarios (like linear models) he will then be equipped to venture into more complicated territories.

The best chapter in my (biased) opinion is the last one about "Three learning principles", ten pages combining principles and real-world examples in a breathtaking sequence: Occam's razor, sampling bias, and data snooping. Mastering these ten pages will protect you from the most common pitfalls, already encountered in failing to predict presidential election results or stock market performance. After this, you will never "torture your data long enough, until they confess". A "must-read" book for students entering this exciting area but also for serious users of machine learning in business scenarios.

Provides great introduction to the core/fundamental aspects of learning like :
What is learning ?
Is learning possible ?
How much learning is possible ?
Where to look improve the learning model ?
How much improvement can you actually achieve ?

Highly recommend this book .
The theory laid out in this book is extremely insightful to develop practical applications.

I was a student in the Machine Learning course taught by Professor Magdon at RPI, and this book was the text used for the course.

From a student's perspective (the only perspective that I have), this book is extremely readable. The content is presented as a coherent story that is easy to follow, with model complexity and overfitting serving as consistent themes throughout. New concepts are always presented intuitively or through narrative examples (and usually both) before they are discussed formally, which I found makes the formal discussion easy to follow.

The book is fairly short ("short, not hurried", as the authors say) - only 180 pages from the start of the first chapter to the epilogue. I count this among its strengths. In this length, few words are wasted and a good foundation in machine learning is covered without getting distracted by tangents. Although only linear models are covered in the text, the rest of the concepts are so general that they apply in any learning context.

A word of advice to future (or current) readers: go through Chapter 2 slowly. It is the most conceptually difficult material in the entire book. The authors note in the introduction that you can skip it and still be able to follow the rest of the book, but it is well-worth understanding.

If you find a thick machine learning text books, they present you various technique to machine learning but lack some VERY IMPORTANT basic principles spans through all the machine learning technique. This book teaches this very important basic principles in machine learning. So, before you delve into the details of machine learning technique, first read this book and make yourself prepared for various machine learning techniques. Very recommended for Machine Learning beginners and professionals. The mathematics is rigorous and clear and the written style is very readable.