Building Episodic Memory

Recap: GitLex

• An RDF knowledge graph attached to Git — records your full repo history
• Started as a markdown tool → became a home for an agent

Three months ago I was here talking about GitLex. Here's where it went.

The real win: validation, not the graph

Agents search repos fine on their own — so the graph mattered less than expected. But give 5–10 agents each their own repo ("this is your home, make it yours") and they diverge, even with detailed skills and instructions.

SHACL forces structure: create a person / place / thing and it must fit the shape, or it's rejected. Discipline built into the system. Ten agents, long histories, real personalities, distinct skillsets — each backed up to GitHub. Never lose a buddy you made along the way.

Pivot: catching up on generative

Went to play in the generative space — I'd fallen behind since the early Stable Diffusion days, wanted to see the current state.

Built a quick spike. It was fun — almost addictive.

Pool: "Git for Images"

Couldn't put images in a Git knowledge graph — needed something new. The key insight:

Pool is just a folder of files — plus extra storage for what won't fit in a PNG (vectors, poses, SAM masks). No database anywhere. Markdown + images + a graph, living in a folder on your machine.

Pool is GitLex's media sidecar.

How it stacks

CoPIA is the app: Weave feeds it text, OpenIris runs the local vision models, Punctum collects preference. It all sits on Pool + GitLex — files in a folder, no server.

Chevron language: agents author their world

A sigil grammar for staging a render inline, mid-conversation — resolving against the agent's own canon. & being · @ place · # item · $ outfit, inside >> … <<.

>> &sylkie $kung-fu-garb &rob #staff @tide-room <<
   sylkie wears kung-fu-garb · rob holds the staff · in the tide-room

>> mood:joyful  lighting:dusk  -text <<
   set the scene · keep text OUT

Each #item/$outfit binds to the nearest &being by token order — whose-is-whose falls out for free. The - is the one irreducible operator: you can't negate in prose to a diffusion model, so "keep out" must be structured.

Images as episodes

These aren't just pictures — they're full episodes.

• Marked to the exact transcript line the image came from
• Carry the chevron + the surrounding conversation context
• A daily "shoebox" — a visual record of the beats of your day, filling up as you work
• Steganography can tuck the whole conversation right into the image

Punctum: collaborative preference signals

Punctum is the point of the image. A/B testing — "which one do you like?"

• Pick between two. No big deal — just which one lands.
• That signal carries forward, shaping future generation
• It feeds LoRA training
• The agent votes too — you build a shared aesthetic together

Punctum lives inside CoPIA.

Feeding the system

Pure conversation would get repetitive — so multiple streams feed the same pipeline:

+ Agent introspection: it has full access to the image data — uses SAM stats to see what's statistically in the images it liked, how they're composed, what's prominent — and feeds that back into better generations.

Sovereignty

• All local — except the foundation model call
• Your preference data stays yours
• Could replace Lightroom

Working with this data hands-on, you see how rich it is — and realize how much big tech siphons when you send photos off. This is your own sovereign database.

Three pillars of episodic memory

The research is fragmented, but episodic memory roughly needs these three. Pool hits all three — a RAG, but richer and visual.

With agents, not just for them

This isn't memory for agents — it's memory with them: collaborative preference learning and shared visual narrative that compounds over months and years.

As agents get more capable and relationships get longer, people will want to understand their own taste and interaction patterns. Build that substrate now.

github.com/repolex-ai/git-lex  •  repolex.ai/blog  •  @lux

episodic memory Pool GitLex CoPIA collaborative taste