Not all heroes wear capes

Not all heroes wear capes


Disclaimer: All opinions expressed here are solely my own and do not represent those of my employer.

Impossible. That was the first word that came out of my mouth when I saw the course homepage for the first time. Impossible. There's no way that just anyone could learn deep learning. It was 2017. There were hardly any frameworks for developers to try out deep learning (Hearing the word Caffe still gives me shivers). As was the case with all of machine learning past, it appeared that only an elite few could partake in this revolutionary breakthrough. The rest of us had to watch on with envy.  Despite having a  graduate degree in an adjacent field, Computer vision, and a few years of industry experience under my belt, I had to spend hours poring through the literature to understand what deep learning was about.

So when this course publicly claimed not only that anyone could learn this technology, but also that anyone could harness it for their own use, it shook me to my bones. It didn't stop there. The course for all it promised cost a grand total of $0. It was free. I had to check this out.

From the 2017 Course homepage

A week after I took the course, I was reeling. I had just woken up from a deep slumber as though a bucket of ice water had just been thrown on my face. I felt like Neo, just unplugged from the Matrix.

Equations and formality were eschewed. Code was used to communicate ideas in a simple, inviting fashion. The syllabus, like my world, was turned upside down. It all made sense.

Why didn't everyone teach this way?

Jeremy Howard changed the way I learn forever. You'd imagine that a world-class teacher like him would have a Ph.D., hundreds of publications to his name, and deliver lectures in Greek symbols like Euclid or Archimedes.

Jeremy holds a degree in philosophy and has never formally studied machine learning.

My jaw dropped to the floor when I first learnt this. Yes, you have my permission to pick up yours too. So how did he do it?  As any good engineer would, I decided to find out.

Inverting an inverted pyramid

The first thing that strikes you about the approach is how it turns everything upside down. Every single course, every single textbook I'd encountered since Pre-K started bottom-up. You'd learn numbers, then arithmetic, then algebra, and so on. You've been there, so you know the drill right? "You have to learn to crawl before you can walk".

In the very first lesson of the fastai course, every student trains a neural network to classify images. Not just any neural network. A state-of-the-art one. This is before any equation or jargon is introduced. It didn't matter if we wanted to classify dogs, pokemon, exotic pineapples, or even stuffed teddy bears. We built something that worked for a problem we cared about (no matter how trivial).

The way is top-down.

When we teach kids basketball, we don't start by telling them that the ball is 29.5 inches in circumference, made of a rubber bladder, and has a leather surface. We don't teach them the aerodynamics of a ball in flight before teaching them to shoot. Doing that gives you a team of all-star scientists, not the Harlem globe trotters. We throw them the damn ball and let them have fun.

Enjoyable learning begins and ends with inspiration - The moment you see what you can do with something, the moment you have fun, that's the moment you thirst to learn more. When struggles come by as they always do, that accumulated enjoyment and inspiration pushes you to move past them. In fastai, students were hooked. They were engaged. We were given the ball to play with first and couldn't wait to learn about the rules and tools.

The top-down mentality is just one arrow in Jeremy's quiver.

The curiosity coefficient

It's an incredible achievement to teach something so clearly that students can't wait to learn more. It's an entirely different level of achievement to inspire students to make what they learnt even better.

Have you ever had one of those sweaters that are warm and fuzzy except for one spot where it itches like crazy? Usually, that spot is on your back and it's nigh impossible to reach it with your hands, much like the futile efforts of an archer trying to hit a bullseye standing on top of an angry giraffe. In those trying times, you do everything you can to get rid of said itch.

That is the kind of intensity that fastai students cultivate when it comes to solving problems. In live coding demos, we'd wade deep into the murky waters of machine learning with Jeremy. We often unlocked incredible insights which sparked our curiosity further.

Take for example the story of a previously undiscovered bug in Pytorch related to initializing neural networks. We watched in awe as Jeremy walked us through how he found it and why it was wrong. You'd never get that kind of hands-on experience anywhere else.

We refined something as trivial as matrix multiplication over and over until it was 50x faster than its initial form. These are but two of the many experiences when we as students were shown the compounding effects of small improvements. We wouldn't have known these improvements were so powerful (or necessary) if we weren't taught to ask "why".

When you are constantly immersed in a space where the two questions "How does this work?" and "How can it be made better?" are asked repeatedly, curiosity becomes muscle memory.

Curiosity by itself is powerful. Curiosity married with consistency is significantly more potent.

A shared purpose

The end (or should I say beginning?) of all education is to be able to use it in the real world in some way.

One of the significant limitations of traditional education is the lack of "transfer". Transfer is the ability of a learner to apply the skills and knowledge they've gained in one context to a problem in another context.

After taking the fastai course, students have been able to start successful companies, invent new techniques to win competitions, publish cutting-edge research that transformed the field, and more. Transfer is a non-issue for fastai students.

I believe this is in no small part due to the nurturing community of students and alumni who share their knowledge freely and openly with others in the course forums. I've been part of several course forums in my life, and I can honestly say that few if any reach the quality of discussions of the fastai forums.

Whether it's sharing what one is working on, or a stumbling block that's preventing one from realizing an idea, all posts are welcomed, and this sparks exciting conversations. Through fostering this culture of sharing and learning, Jeremy has given capes to his students so that they can make the world a better place.

As I reflect on what I've learnt (and how), I'm sure there are several other aspects that I've missed but these are the ones that have resonated most with me.

Jeremy has changed the way I approach learning. He is the kind of teacher I aspire to be.