leonardo-zhou.com

Neural Networks, Transformers, and ChatGPT

I like math.

I loved math and problem solving from an early age, and I continue to foster that love through competitive mathematics. From my early steps in AI, I was intrigued by how it really worked at the core of it all.

Reducing AI to pure math

I wanted to try to reduce the basis of AI into pure, simple math that anyone with that mathematical background could understand. So by going through Andrej Karpathy's playlist "Neural Networks: Zero to Hero," I broke down neural networks into piece-by-piece mathematical processes, and even manually calculated backpropagation formulas for a simple multilayer perceptron. Check out my article on my progress here.

Now bringing that math to life

But math isn't enough in the real world. Products matter. So I built ChatGPT from scratch by following the same tutorial from Andrej Karpathy. Although it isn't great at being an assistant and blabbers about in Shakespearean-like garbled English, it is a GPT and I'm proud of that.

Progress

Step 1: Basic LLM infrastructure

I implemented a basic voice assistant by connecting a speech-to-text API, LLM API, and text-to-speech API to have a responsive voice assistant. Read the article here.

Step 2: Automated functions

By allowing the user to activate commonplace commands such as calling a specific contact, opening a URL or an app, and recalling the date and time, this feels a little more like a Siri, albeit a ripoff one. Read the article here.

Step 3: Computer Vision

By adding computer vision capabilities, this levels up BetterSiri to the standards of JARVIS, an assistant that can hear and see (and comprehend all of that) to provide a meaningful answer. Read the article here.

MidnightFocus