Machine Learning Essentials: Must-Reads for Every AI Enthusiast

OpenAI's Ilya Sutskever: 90% AI Guide

Photo by Igor Omilaev on Unsplash

Generative AI is changing many industries. Scientists and experts need to learn about AI fast. What should be in a “Machine Learning Basics” guide?

Ilya Sutskever, a big name in AI, made a list of important articles.

Ilya believes that mastering these contents gives you an understanding of 90% of the important aspects in the current AI field.

Ilya studies special AI parts like transformer, RNN, and LSTM, and how complex AI can be.

Ilya's paper list

Ilya suggests Google’s important paper “Attention Is All You Need,” from 2017. This paper introduced the transformer architecture, now widely used in AI, especially for making AI models.

Ilya suggests a blog named “The Annotated Transformer,” by Cornell’s Professor Rush and others, from 2018.

This blog breaks down the paper line by line and simplifies it. It rearranges and shortens the original content, making it easier to understand. Later, in 2022, researchers like Austin Huang updated the blog using PyTorch.

In terms of RNNs, Ilya suggests reading a blog by AI expert Andrej Karpathy from 2015. He highlights the “amazing effectiveness” of RNNs.

screenshot

Ilya likes a paper called “Recurrent Neural Network Regularization.” It was made by Zaremba from NYU and Ilya in 2015. Ilya worked at Google Brain then.