The Memory Problem
As I continued testing GPT-4’s capabilities in powering LEAF’s narrative-driven companion system, one of the first major roadblocks took root: the OpenAI API retains no short-term or long-term memory at all. Not from the last line, the last prompt, or the last scene. It treats every interaction as if it’s the first. No history or continuity. For a game built around trust, memory, and evolving relationships, this is a serious issue.
The concept of LEAF depends on the AI companion’s ability to change based on the player’s behavior. If the companion forgets what the player did in the last scene, their reactions lose meaning. A character who is supposed to grow with the player can’t do that if they’re constantly resetting to zero.
This also causes narrative contradictions. For example, if the player kills an innocent NPC, the AI might criticize them in one scene, but later praise them or misunderstand the event entirely, believing the NPC was a threat. Without persistent memory, GPT-4 can’t track cause and effect.
To solve this, I’ve begun designing a middleware system - and interpreter that sits between the player’s actions and the GPT prompt. This system would track important gameplay data and use that to build each prompt intelligently. Instead of feeding GPT-4 every piece of memory each time (which quickly runs into token limits), the middleware can potentially summarize key context, helping the AI maintain consistency without being overloaded.
This could be supported by an external memory system, either in the form of a local database or lightweight data structure, where prior player decisions, character eractions, and story milestones are stored. The GPT agent can reference this data prior to generating a response, allowing it to make decisions based on previous interactions. The benefit of using a localized memory system - as opposed to an online or cloud-based one - is that it remains fully embedded within the game. No external connection is required, and players maintain privacy and full immersion. The AI doesn’t technically “remember” - but the game does, and it recontextualizes the prompts with that history every time the AI is asked to speak or act.
One inspiration is the idea of Retrieval Augmented Generation (RAG) architectures, which allow for the retrieval of relevant pieces of stored information and incorporates them into the prompt context. In the case of LEAF, this could mean searching a stored memory log for relevant past decisions, tone shifts, or trust events, and re-inserting them into the prompt dynamically.
Another functionality and potential solution I’ve started exploring is prompt carching. If a situation or player decision repeats, the system could store and reuse previous GPT outputs instead of generating them from scratch. This would save on API calls and help keep character behavior consistent. I’m also investigating how preloading responses based on certain game states or companion personalities might help balance real-time generation with narrative control. OpenAI may support some token caching internally, but it’s still unclear how much control I have over this within the API itself, or whether I’ll need to develop my own caching layer from scratch.
This memory problem isn’t just technical, but narrative as well. A story loses its power when characters forget what matters. While GPT-4 is incredibly responsive, its lack of memory forces me to think creatively about how to simulate the illusion of growth, change, and consequence. This middleware system I’m developing is still in early development, but may be one of the most important components in making LEAF work the way it’s meant to.