ChatGPT 101: Pre-Training

How chatGPT works behind the scene: Part one

OpenAI’s GPT-3, including the chatbot version known as ChatGPT, is based on a transformer architecture and uses deep neural networks to generate text.

The system design of ChatGPT consists of several components. Part one, called pre-training, of several I will be publishing here. I will walk you through each component.

The first component is pre-training. Pre-training aims to enable the model to generate coherent and meaningful text. This involves training the model on a large corpus of…

--

--

💍Husband 📝Writer 🔧Engineer, bringing a unique blend of 🎨creativity, 💪commitment, and 💻technical expertise to everything.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Subedi🌀

💍Husband 📝Writer 🔧Engineer, bringing a unique blend of 🎨creativity, 💪commitment, and 💻technical expertise to everything.