AI Memory — a persistent user model so the coach never starts cold

A context window is not memory. It forgets you the moment the conversation ends.

The problem

Every Arti conversation started from zero. The model could see the current thread and whatever context the prompt assembled, but nothing carried across sessions — so a manager who had spent months in the product got the same cold opening as someone on day one.

Stuffing more history into the prompt isn't the answer. It's expensive, it grows without bound, and most of a transcript is noise. What the coach actually needs is a distilled view: how this person works, what they're trying to improve, what they've already tried.

The approach

UserAIProfile — a per-user record holding versioned JSON profile data plus a last_generated_at stamp. A dedicated prompt synthesizes it from two sources the product already had: the user's past Arti exchanges, and their answers to team exercises.

The interesting engineering isn't the generation. It's deciding when to regenerate.

• Signals, not cron. post_save hooks on ArtiExchange and ExerciseInstance mean the profile reacts to real activity rather than a nightly sweep that's stale by morning.
• But signals fire constantly. Naively, every message a user sends triggers an LLM call to rebuild their profile. For a chatty user that's dozens of calls a day producing nearly identical output.
• So regeneration is gated twice. An activity threshold — has enough actually changed since last time? — and a minimum-hours-between-updates guard. Only when both pass does the signal enqueue work.
• And it's deferred. The enqueued job is a @delayed_job(minutes=5), so a burst of messages collapses into one regeneration after the burst settles rather than racing mid-conversation.
• Backfill without the signals. A generate_user_profiles management command handles batch generation for existing users, since signals only cover activity from the day they shipped.
• Visible and inspectable. A feature-flagged profile tab in the UI, plus Django admin, so the profile is something you can read and sanity-check rather than an opaque blob influencing answers.

Later, the profile's updated_at became one of the inputs to the validation token behind the /users/me response cache — memory changing is one of the things that has to invalidate a user's cached payload.

Stack

Python · Django · Django signals · RQ delayed jobs · Gemini (via an internal LLM abstraction) · PostgreSQL · React · TypeScript

Outcomes

• Persistent, per-user memory feeding the coaching surfaces, generated from data the product already collected rather than a new capture flow.
• Regeneration cost bounded by construction: activity thresholds plus a min-hours guard plus a debounce window, instead of one LLM call per user turn.
• Ships as the "Personalized Member Insights" capability in the company's flagship AI launch.

What I learned

1. Memory is a scheduling problem wearing a prompt-engineering costume. Writing the synthesis prompt took an afternoon. Getting regeneration to fire often enough to stay current and rarely enough to stay affordable took the rest of the work.
2. A post_save signal is an unbounded loop if you let it be. Anything that triggers an LLM call from a write hook needs a threshold, a cooling-off period, and a deferral — otherwise you've built an expensive amplifier on your busiest users.
3. Make derived state readable. Putting the generated profile behind a UI tab and in admin meant that when the coach said something odd, I could check whether the memory was wrong or the prompt was — instead of guessing.