You posting in vague or half way commentary will get you into trouble fast. And many Public LLM's will invite you to have a "Conversation" however we must understand how the LLM works and why it loses its mind over time.
The entire conversation is stored and replayed with every single interaction.
This means the first thing you say in a conversation with the LLM is the best and most pure in the pipeline. Lets break this down more.
if you say
Show me pictures of catsit responds
Here are pictures of catsThen you state
show me pictures of cats and dogsThis is how the LLM sees your interaction
1 2 3 4 |
Show me pictures of cats Here are pictures of cats show me pictures of cats and dogs |
As your conversation grows so do the amount of variables and conditions as that pool of information it has to work with grows it pulls from the entire data strucutre.
Now if you go back to say
show me pictures of just cats its going to see
1 2 3 4 5 |
Show me pictures of cats Here are pictures of cats show me pictures of cats and dogs show me pictures of just cats |
And the probability of dogs showing up should by any logical sense be No dogs allowed. But the LLM still has dogs in the equation so now its not a 0 sum chance. And this is the inherent issue with stacking the model and long conversations.
you have 4 cats and 1 dog in the variable making Cats a 75% Confidence and Dogs at 15% but not logically 0%
Understanding this limit ensures we write better prompting.