Syllabus and Class Info
My teaching style!
Welcome to one of the most beautiful subjects on the planet Earth, where every species thinks algorithmically. Unlike most mathematics, whose spectacular beauty can be seen only after years of study, computation can be appreciated by all. Nothing here is therefore copyright, so share it all with your loved ones.
Why software is eating the world
A quick quiz
What is the value of rearranging 3, 1, 5 in the correct ascending order to 1, 3, 5?
Apple blocked Facebook's ability to rearrange numbers on your iPhone and Facebook lost $230 billion in market cap on Feb 3, 2022. This is more than the market cap of all but 10-20 companies!
Facebook would look around in your iPhone apps, compute your emotional distance from various products, and serve ads only for the products you were closest to. Call of Duty = 1, Prada handbag = 2, GTA5 = 100. So no GTA5 ads for you.
Can the intuition behind this bitcoin maths from the original Satoshi Nakamoto white paper be explained simply? We'll do it!
Could you have predicted YouTube's future from its first video?
Have you heard of EXCITE?
Well, they passed on a chance to buy Google for $750,000 (Google originally wanted $1M but came down).
How about Blockbuster Video (now dead) passing on a chance to buy Netflix for $50m?
In both cases, these companies did not think algorithmically. EXCITE didn't realize how efficiently Google's method would scale to a large internet. Blockbuster didn't realize Netflix had managed to build a mega videostore clerk who could handle everything from scheduling, shipping, and giving personalized recommendations to millions of customers. In one stroke, Netflix's electronic megabrain outdid the brains of countless videostore clerks! Now Tesla is building a similar megabrain that will instantly drive millions of cars.
Our NN is initially in Python for rapid iteration, then converted to C++/C/raw metal driver code for speed (important!). Also, tons of C++/C engineers needed for vehicle control & entire rest of car. Educational background is irrelevant, but all must pass hardcore coding test.— Elon Musk (@elonmusk) February 3, 2020
Why should you take this class instead of just reading the slides above?
In theory, a good online library is all you need. But the reality is that there are many subtle aspects of your own thinking that you can resolve only by sharing them with a superb group of graduate students, who will then give you amazing illumination. My class is not me monotonously reading out slides to gagged students, but an open forum for active discussion and listening.
My take on Transformer Neural Nets with Attention