Memory as a superpower

Thoughts about capturing, tracking, and surfacing signals over time (through VC lens)

I previously wrote about the data pipeline and context architecture we’ve built for our AI analyst at Montage (we call him Monty). It’s a system that helps us manage sourcing and market analysis by capturing information, structuring it, and making it retrievable at the right moment. That piece was mostly about the mechanics of how the stack is wired together: how to set up the decision tree structure, how to parse it in a semi-agentic way, what to watch out for. What I want to do here is zoom out and go deeper on one specific aspect of that architecture: the role of memory. Because while the pipeline itself is useful for keeping things organized, a big piece is how it builds and maintains context over time. It’s part of a broader design space in AI applications: building memory systems that can hold on to the right context, help us make sense of it, and surface it when it matters most.

Why Memory Matters

Humans are wired for intuition and synthesis, but it gets hard holding large volumes of context consistently. AI and software tools in general have the opposite profile: they can recall everything but don’t have the intuition to consistently weigh or judge it. Putting the two together, for our use case of applying this in venture, that means using memory systems to track signals at a scale no one can reasonably keep in their head, like hundreds of companies crossing the desk, patterns in who’s raising funding and where, clusters of similar ideas that might otherwise slip by unnoticed. There’s value is in knowing who you’re not meeting, where you might be blind to activity, and what themes keep reappearing in the background. Done right, memory compounds judgment: every conversation, every passed deal, every emerging pattern becomes fuel for sharper decisions going forward.

Building the Foundation: Memory as Ground Truth

The foundation of any useful memory system is ground truth — a reliable mirror of what you’ve actually seen and discussed. This is less about prediction and more about making sure the raw inputs are faithfully captured, structured, and retrievable without drifting into noise or hallucination. For us, that meant setting up a pipeline that enforces alignment between messy, real-world inputs and the structured worldview we keep in our decision tree. The goal is simple but powerful: take every funding announcement, every IC discussion, every CRM update, and anchor it to the right node in the taxonomy so it compounds into context we can actually use.

To make this concrete, here are a few of the inputs that flow into the system:

• Deal announcements from newsletters and databases, capturing who raised, how much, what they do, and who invested.

• Market news that provides signals about broader shifts in activity or emerging spaces.

• Our pipeline from Notion and CRM, with company statuses marked as new, in-diligence, passed, or invested.

• Internal discussions, especially transcripts of IC meetings, which get distilled into heuristics and attached to the right nodes.

Here’s an example: we talked about AI for drug discovery at IC, what type of approaches are exciting to us. It was automatically summarized into a clear heuristic.

Even in cases where there is scientific promise, we look for compelling software leverage, differentiated datasets, and go-to-market plans that avoid single-asset development traps. We are interested in techbio platforms that treat mRNA as an engineering substrate, not a modality. Closed-loop design systems that optimize for expression durability and tissue targeting unlock large therapeutic white spaces. Key filters: proprietary data generation, software leverage, and co-founder teams blending deep science with ML engineering

This was added to the relevant node in the database using the agentic parsing mechanism. While scanning early stage companies, the system immediately began to consider this new information and flagged a really cool company fitting that profile. Automating this flow of information from discussions into the sourcing engine is great because you get personalization at a level not possible when using a third party service for this, and the timeline is instant.

Of course, building ground truth isn’t just about capture, it’s about doing it consistently. A few of the technical challenges we’ve had to navigate include:

• Consistency in mapping: does a deal always land in the same node if you re-run classification?

• Duplicate-named nodes: like “Payments” under both fintech and commerce infrastructure, which can create ambiguity.

• Near-duplicate paths: companies that could fit in multiple places, forcing tradeoffs between precision and speed. You have to have some understanding or way of knowing which paths are similar

These need to be carefully understood and handled to make sure relevant information is easily found and correctly applied.

Tracking Change: Memory as Market Monitor

If the ground truth layer is about capturing what you’ve seen, the market monitor layer is about watching how that picture shifts over time. The goal isn’t to predict or explain trends, it’s simply to track deltas: what’s changing in the market compared to what you already know. In venture, timelines stretch across months and years, so the signals that matter often emerge slowly. A single new deal in a category doesn’t mean much; clusters, adjacencies, and repeated patterns do. What makes this layer interesting is that it compounds with time: the longer you run it, the clearer the signals become.

Some of the signals worth tracking include:

• Spotting branches heating up where the pipeline is thin.

• Highlighting clusters of passed deals, which may point to blind spots in your framework.

• Surfacing new entrants that resemble portfolio companies, whether competitive or synergistic.

• Measuring cluster density by counting deals per branch over time, showing which spaces are accelerating.

• Portfolio adjacency, where new companies are flagged if they’re close to existing investments.

• Warm intro signals, where repeated co-investors or angels appear in a branch.

The challenges here are different from the ground truth layer but closely related. You need to maintain an overview while aligning inputs from multiple data sources. As branches evolve, you have to ensure the taxonomy still makes sense and doesn’t fragment into redundant or overlapping paths. And when new spaces emerge, you need a way to reconcile them with what’s already in the tree. These are not problems to “solve” once but dynamics to manage as the system grows. Done right, the market monitor turns memory into a lens that lets you see shifts in the market with more clarity and less noise.

Surfacing What Matters: Memory as Proactive Radar

The last layer of the system is about turning memory from something reactive into something proactive. Most AI tools today are designed to respond when you ask a question or provide a prompt. What they don’t do is nudge you with the information you didn’t think to ask for. A proactive memory system fills that gap. The goal isn’t prediction (we’re not trying to guess the next market trend or forecast outcomes) but simply to surface things you might otherwise miss. With all the information already captured and organized in the tree, the question becomes: how do you make it work for you in the background?

One way is through retrieval layers. Each node in the tree can run standing queries against external sources, like news, patent databases, job postings, etc., and then attach the results with timestamps. This turns every theme into a living feed. You don’t need to go hunt down updates on “supply chain optimization” or “AI-enabled compliance”; the system quietly collects and slots them into the right branches.

The next step is contextual alerts. Instead of just storing this information, the system surfaces it at the right moment, in the right channel. That might look like a Slack message with notes such as:

• “Three new companies in supply chain transparency raised significant rounds this month.”

• “A startup in the same space as your portfolio company just announced a new product.”

• “You’ve been looking into healthcare RPA — here are similar companies, some pre-raise and some recently backed by notable investors.”

• “You mentioned interest in graph-augmented retrieval; here’s a new company doing exactly that.”

The point is not to drown you in alerts, but to surface a handful of signals that would otherwise slip by. This is where memory crosses from being passive record-keeping into an active radar.

That shift comes with its own challenges. Setting thresholds is critical: too high and you miss things, too low and you’re buried in noise. Parsing the tree to decide which paths are relevant requires careful design, otherwise alerts lose their precision. And the system has to balance consistency with adaptivity, keeping the taxonomy stable enough to be trusted, while flexible enough to catch new themes without generating noisy or hallucinatory classifications. Proactive radar is about giving you a quiet stream of nudges, so that you see more than you would on your own, without being overwhelmed.

Putting It Together

Seen as a stack, the three layers build on one another: ground truth gives you a reliable record of what you’ve seen, market monitoring shows how that record shifts over time, and proactive radar surfaces the signals you might otherwise miss. Together they turn memory from passive storage into an active lens, one that compounds into perspective. Instead of starting from scratch each time you evaluate a company or thesis, you’re always standing on the shoulders of your accumulated judgment: what you’ve seen, what’s moving, and what’s emerging.

What excites me about this isn’t just the application in venture. These same principles generalize to any domain where decisions depend on dynamic, evolving context. A policymaker trying to stay ahead of regulatory developments, a scientist tracking new literature in a niche field, an enterprise team managing shifting market landscapes, etc., all face the same challenge of grounding information, monitoring change, and surfacing what matters. In that sense, memory systems are a design pattern for how we extend our reasoning.

The open question is how far this can go. Most AI tools today are still built around reactivity: waiting for a prompt, retrieving documents, generating a response. But if memory becomes the core layer, we could see a shift toward AI systems that act less like search engines and more like evolving collaborators. They wouldn’t need to be “super intelligent” to be valuable; they’d simply need to remember well, adapt as the world shifts, and help us catch what we’d otherwise miss. That feels like a frontier worth building toward. And if you’re experimenting with systems like this, I’d love to compare notes.