Member-only story
How chatGPT works behind the scene: Part four
Hold on a sec! 🛑 Before diving into this part, make sure you’ve caught up with the third installment 🔙 by clicking on the link! Trust me, it’ll make the journey even smoother 🚀
The model is the most important part of the chatGPT system. It’s made up of many smaller pieces called layers. These layers work together to take the input (what the user has said) and figure out what word the chatGPT should say next. The model does this by giving each possible word a score based on how likely it thinks that word should come next. The word with the highest score is what the chatGPT will say next. This process is repeated until the chatGPT has generated a complete response.
The model consists of several key components:
Attention Mechanisms:
The model uses attention mechanisms to weigh the importance of different parts of the input sequence when generating a response.
Multi-Head Attention:
The model uses multiple attention heads to allow the model to attend to multiple different parts of the input sequence simultaneously.
