PRAANA Architecture

PRAANA is a single-process TypeScript CLI coding agent built around two memory systems:

  1. Adaptive Context — within-session working memory
  2. Cognitive Memory — learns from experience across sessions, scores what's confirmed, forgets what's noise

No daemon, no RPC server, no multi-process coordination.

Directory Structure

src/
  main.ts        — CLI entry point, TTY guard, slash commands, pi-tui launch
  turn.ts        — Per-turn orchestration (prompt → LLM → tools → banners)
  session.ts     — Session lifecycle (create/resume/end) & memory init
  compile-classic.ts — Classic-mode prompt assembly (full verbatim history)
  compiler.ts    — Legacy budget-band compiler (unit tests only)
  state-graph.ts — Tiered state management (active/soft/hard) & two-pass auto-hydrate (substring + BM25)
  state-graph-checkpoint.ts — O(1) resume: persist/restore StateGraph snapshot (tiers, touchedTurn) after each turn
  event-log.ts   — Append-only JSONL event persistence with fsyncSync durability; in-memory parse cache (re-reads disk only on mtime/size change)
  token-estimate.ts — Unicode-aware token heuristic (Latin/CJK/emoji/ZWJ); canonical shared estimator for all budget calculations
  context-pressure.ts — Density-weighted effective-token accounting; raw-token safety net; resolves compaction config
  cosine-similarity.ts — Cosine similarity for Float32Array embedding vectors; shared by context scoring and memory dedup
  hash.ts        — SHA-256 hashing utility; shared by session scope construction and skill scope keys
  model-resolver.ts — /model parsing and provider+model resolution
  provider-catalog.ts — live /models fetch + 6h cache for OpenAI-compatible providers
  llm.ts         — Provider registry and model building via pi-ai
  config.ts      — Multi-source JSON/TOML config loading & deep-merge
  types.ts       — Core shared TypeScript types
  ui.ts          — CLI output formatting, banners, and text colors
  utils/
    bm25.ts      — Shared BM25 tokenization, scoring, and corpus statistics
  context-engine/
    index.ts          — ContextEngine facade (store, ledger, extraction, checkpoint, telemetry, workflow patterns)
    types.ts          — Engine-level interfaces; WorkflowPattern
    artifact-store.ts — Content-addressed blob store with distiller integration
    turn-ledger.ts    — Append-only typed turn records with BM25 search
    distiller.ts      — DistillerRegistry, sync/deferred dispatch, intensity selection
    classify.ts       — Fast regex-based content-type classification
    summarize.ts      — Generic head/tail summarisation; re-exports estimateTokens from token-estimate.ts
    extraction.ts     — Post-turn deterministic extraction (TurnDigest, activity, errors)
    turn-digest.ts    — TurnDigest extraction and state-graph diffing
    activity-log.ts   — ActivityEntry derivation and rolling buffer
    error-tracker.ts  — Open error tracking and test-pass/fail detection
    checkpoint.ts     — SessionCheckpoint reconciliation, rendering, replay from digests
    state-snapshot.ts — StateGraph snapshot and diff for decision/constraint extraction
    scoring.ts        — Context-unit scoring (pin + recency + max(BM25, semantic) + hydrate boost) and budget selection
    embedding-cache.ts — Per-turn embedding cache; precomputes vectors for all context units concurrently; invalidated when unit set changes between turns
    engine-compiler.ts — Multi-resolution budget-band compiler; injects workflow context section when matching patterns exist
    bm25.ts           — Re-exports shared BM25 utilities for context-unit and turn-search scoring
    density.ts        — SectionDensityKind → weight table; densityWeight() used by engine-compiler for weighted pressure
    turn-recorder.ts  — Per-turn event recording for the turn ledger
    event-lineage.ts  — Trace artifacts back to producing turns, decisions, files
    telemetry.ts      — Per-session telemetry: pressure events, retrieval rates, distiller savings
    workflow-tracker.ts — Workflow pattern tracking: extract tool sequences + artifact types, persist to workflow_patterns table, render compact Workflow Context section
    db.ts             — SQLite schema for all engine tables: artifacts, turns, checkpoint, scorecard, workflow_patterns
  distillers/
    index.ts       — Default distiller registry factory
    git-diff.ts     — Diff hunk compaction with context reduction
    npm-test.ts     — Test output: collect failures, summarise passes
    tsc-errors.ts   — Build error deduplication
    rg-results.ts   — Search result dedup and top-k
    generic.ts      — Log template mining + generic head/tail
  tools/
    index.ts     — Tool registry (all tool definitions combined)
    tool-def.ts  — Type helper for defining tools
    system.ts    — shell, read_file, write_file, edit_file, read_and_summarize, batch_write, batch_edit
    search-code.ts — search_code: ripgrep-backed structured code search (rg --json → file:line:column matches with context)
    knowledge.ts — recall, remember, retrieve_artifact, context_summary, search_turn_events, event_lineage
    memory.ts    — Adaptive Context state-graph tools (tasks, decisions, constraints, notes)
  memory/
    index.ts     — Memory store exports
    store.ts     — High-level MemoryStore API (remember, recall, session start/end); isSessionGood export
    db.ts        — SQLite schema: entries, entry_scopes, sessions, entries_vec, skill_stats, skill_cooccurrence
    embeddings.ts — Ollama embedder adapter (optional, opt-in)
    types.ts     — Memory-specific types
    summarizer.ts — LLM-based extraction logic (conversation transcript → learnings JSON)
    openai-summarizer.ts — Chat completion fetch adapter for summarization
  skills/
    index.ts          — SkillRuntime: discovery, load tracking, telemetry (engine mode); buildSkillMetadataCatalog with usefulness-ranked ordering
    skill-stats-store.ts — Cross-session skill effectiveness: loadUsefulness (dual-scope), flush (boost/decay), co-occurrence recording
    types.ts          — SkillRecord (with scope), LoadedSkill (with used flag), SkillEffect, telemetry types

Runtime Architecture (Turn Flow)

User input
  → main.ts interactive TTY entrypoint
  → runTurn(session, input) in turn.ts
      1) append user_message event
      2) select mode:
         Engine: context_engine.enabled=true AND ContextEngine initialized (default)
         Classic: context engine disabled, or enabled but init failed
      3) [engine only] auto-hydrate matching peripheral state (two-pass: substring keyword + BM25 relevance)
      4) [engine only] capture StateGraph snapshot (for post-turn diff)
      5) [engine only] ingest tool results through distillers; classic passes through
      6) compile prompt:
         Engine:  compileEngineWithMetrics (budget bands + scored units + checkpoint)
         Classic: compileClassicWithMetrics (system frame + skill catalog + full history)
      7) piStream() with tools (tool set varies by mode — see Tools)
      8) log tool_call / tool_result + context_action events
      9) append agent_message event
     10) [engine only] flush deferred distillations, append turn record to ledger
     11) [engine only] extract TurnDigest (deterministic, no LLM)
     12) [engine only] reconcile SessionCheckpoint from digest
     13) increment turn count
     14) [engine only] apply tier demotion rules + cleanupStaleSkills()
     15) persist state graph checkpoint (session.persistStateGraphCheckpoint())
     16) print per-turn memory/prompt banner

Session Boundary Flow

SESSION START:
  Session.create()
  → init EventLog + StateGraph
  → init MemoryStore (if enabled)
  → memory.sessionStart(context)
  → cache digest markdown

SESSION RESUME:
  Session.resume(sessionId)
  → load state_graph_checkpoint.json if present → O(1) StateGraph restore (tiers + touchedTurn preserved)
  → replay only context_action events that occurred after the checkpoint's last_event_id
  → restore model override from the last model_override system_note
  → re-init MemoryStore + regenerate digest

SESSION END:
  Session.end(reason, transcriptEvents)
  → memory.sessionEnd(reason, events)
  → if summarizer enabled: extractLearnings from transcript events
  → write learnings into Cognitive Memory
  → close event log

State Model (Adaptive Context)

Defined in src/types.ts:

export type StateObjectKind = "task" | "decision" | "constraint" | "note";
export type StateTier = "active" | "soft" | "hard";

export interface StateObject {
  id: string; // ULID
  kind: StateObjectKind;
  tier: StateTier;
  payload: StatePayload; // TaskPayload | DecisionPayload | ConstraintPayload | NotePayload
  created: number; // unix ms
  updated: number; // unix ms
  lastTouched: number; // unix ms
}

Tier Behavior in Prompts

Tier Prompt Representation When
Active Full payload Current working set
Soft Summarized one-liner Recently idle
Hard Minimal anchor (ID) Older context, available via hydrate

Tier Demotion

Managed dynamically after each turn in src/turn.ts:

Auto-Hydration

Before a turn prompt is compiled, stateGraph.autoHydrate(userInput) runs two passes against peripheral state objects:

  1. Substring keyword match — extracts keywords (alphanumeric strings $\ge 3$ characters, filtering common English stop words) and matches them against the searchable text representation of peripheral objects. Fast and catches exact references.
  2. BM25 relevance — scores remaining peripheral objects with bm25Relevance(query, text). Objects scoring $\ge 0.15$ (the BM25_HYDRATE_THRESHOLD) are promoted even when no substring keywords matched. This catches fuzzy overlap that substring matching misses — for example, when a user mentions "S3 upload" and a peripheral object about "AWS bucket" gets promoted.

Returns AutoHydrateResult[] with { id, text, method } (method is "substring" or "bm25"). The text field is passed through to context-unit scoring for hydrate_boost calculation (see Scoring below). Promotions are logged as context_action events with reason: "auto_hydrate" and hydrate_method: "substring" | "bm25".

Event Log

Location: ~/.praana/sessions/<session_id>/events.jsonl

In-memory cache: After the first read, EventLog maintains a parsed event index in memory. On every subsequent readAll, readLast, search, or replayContextActions call it checks mtime and size via statSync; if unchanged it returns the cached array directly — no JSONL re-parsing. append() updates the cache inline before writing to disk, so the cache never drifts. getLastEvent() is O(1) direct cache access.

Each line is a single JSON line representing an Event:

Writing uses writeSync followed by fsyncSync to guarantee durable persistence on every event. Rebuilding state on session resume replays context_action events.

Agents can search the full log in-session via the search_session_log tool (keyword AND/OR search). This is distinct from recall, which queries cross-session Cognitive Memory in SQLite.

Compiler

PRAANA has two runtime compile paths. Selection happens in turn.ts:

const useEngineCompiler = contextEngineEnabled && !!session.contextEngine;
const classicMode = !useEngineCompiler;

src/compiler.ts (compileWithMetrics) is retained for unit tests only; it is not used at runtime.

Engine mode (context_engine.enabled = true, engine initialized)

src/context-engine/engine-compiler.ts constructs a multi-resolution prompt from budget bands:

  1. Band 1 (always): System frame, AGENTS.md, skills — existing allocation.
  2. Band 2 (always): Active user request + last 2 full turns verbatim — ~3000 tokens.
  3. Band 3 (always): SessionCheckpoint (pinned) — ~2000 tokens. Reconciled deterministically from TurnDigest after each turn. Sections: active request, session narrative, plan history, constraints, decisions (with retained rationale), files, findings, errors, activity. See Session Checkpoint below.
  4. Band 4 (scored): Fact cards, artifact cards, activity entries — ~3000 tokens.
  5. Band 5 (scored): Older turn digests — ~2000 tokens.
  6. Band 6 (remaining): Memory digest, low-priority state — remainder.

Context units are scored with a 5-term model: pin_boost × W_pin + recency(age) × W_recency + max(BM25 × W_relevance, semantic_similarity × W_semantic) + hydrate_boost × W_hydrate. BM25 handles exact keyword overlap; embedding cosine similarity handles semantic paraphrases (e.g. "auth middleware" vs "authentication handler"). The effective relevance term is max(bm25, semantic) so either signal can win. The hydrate boost is the maximum bm25Relevance(ht, unitContent) across all auto-hydrated object texts — it gives a lift to context units (e.g. older turn digests) that discuss the same objects the user just implicitly referenced. Default weights: w_pin=1.0, w_recency=0.5, w_relevance=0.3, w_semantic=0.3, w_hydrate_boost=0.2. Embeddings are precomputed per compile pass and cached in a session-scoped LRU cache; when the embedder is unavailable or fails, scoring falls back to BM25-only.

Pressure monitoring uses density-weighted effective tokens: contextPressure = weightedTokens / usableBudget. Each section kind has a hardcoded weight in src/context-engine/density.ts (decisions/constraints/plans = 1.0; open errors = 0.8; narrative/file/peripheral = 0.6; verbatim turns = 0.9; turn digests/artifacts = 0.4; findings/activity/fixed errors = 0.25). Low-density filler counts less toward pressure so a prompt full of error traces does not trigger compaction as aggressively as one full of architectural decisions. >0.70 triggers compaction (drop older digests, tighten checkpoint findings/activity budgets). >0.85 triggers emergency (keep checkpoint decisions/constraints/plans/open errors only — omit findings, activity, fixed errors — plus last 2 verbatim turns and current artifact cards). Raw-token safety net: when estimated raw prompt tokens exceed the usable budget or raw fill exceeds emergency_at, emergency mode is forced even if weighted pressure is lower. Section token caps (findings 300, scored bands 3000/2000) bound worst-case raw size; /stats shows both raw and weighted fill.

Score logging: in debug mode, scores.jsonl records every context unit's score breakdown per turn. Inspect via /why <unit-id>.

Session Checkpoint

src/context-engine/checkpoint.ts maintains a structured within-session summary that survives when turns scroll out of the verbatim window (turns 7+). It is reconciled from TurnDigest data — never by the LLM — to prevent drift.

Section Behaviour
Active request Latest user intent (overwritten each turn)
Session narrative Rolling prose from meaningful turns (decisions, constraints, errors, file writes, intent changes). Max 20 entries, 400-token render budget
Plan Current plan + superseded one-liners with completion hints. Extracted from assistant step lists via pattern matching. 400-token budget
Constraints Append-only, never dropped. Includes implicit constraints auto-extracted from user messages
Decisions Summary + rationale; compact after 10 idle turns but rationale is never dropped. 800-token budget
Files in play Evicted after 10 idle turns
Findings Rolling artifact refs, oldest evicted at 30 entries. 300-token budget
Errors Open errors full detail; fixed errors one-liners
Activity Rolling 15 entries from activity log

Implicit knowledge capture has two layers: (1) the system frame nudges the agent to call add_constraint / add_note for preferences and corrections — this is the primary mechanism because the LLM is the language-understanding component, not regex; (2) extractImplicitConstraints() in turn-digest.ts is a minimal safety net that only captures the syntactically unambiguous "not X, Y" correction pattern. Patterns like "let's use", "we use", "I prefer", "how about" are NOT regex-matched because natural language is too varied — those are the LLM's responsibility via the nudge.

Classic mode (compile-classic.ts)

When classic mode is active, src/compile-classic.ts builds a flat prompt with no token-budget truncation and no Adaptive Context sections:

  1. System frame — cwd, session ID, tool descriptions, AGENTS.md project context, skills guidance (use load_skill when relevant).
  2. Skills catalog — name + description only; load_skill(skill_id) loads full body on demand. No pre-injected bodies, no residency tiers.
  3. Cognitive Memory — digest (if enabled).
  4. Conversation history — full verbatim event log (readAll()), excluding context_action and system_note. Tool results are not truncated.
  5. Current input — latest user message.

Runtime differences from engine mode:

Engine mode Classic mode
Compiler compileEngineWithMetrics compileClassicWithMetrics
History Budget bands + checkpoint + scored digests Full verbatim log
StateGraph tools Available Hidden from tool list
Engine tools Available Hidden
SkillRuntime Load tracker + eviction (engine only) Not present (plain agent)
Auto-hydrate / tier demotion Yes Skipped
TurnDigest / checkpoint Yes Skipped
/why scores Available (debug) N/A

Classic mode matches how agents like Claude Code and pi operate — a growing transcript with no working-memory tiering. Set measurement_mode = true to write engine telemetry while running classic mode (development/debug only).

Legacy compiler (compiler.ts)

The original 5-section budget-band allocator (compileWithMetrics) remains in the tree for regression tests only. It applied per-section token budgets, truncated recent turns, and included Adaptive Context active/peripheral sections. Runtime no longer uses this path.

Tools

Defined in src/tools/ using Zod schemas and normalized via zod-to-json-schema. The registered tool set depends on compile mode (describeTools({ contextEngineEnabled, classicMode })).

Shared tools (classic and engine mode)

System Tools (src/tools/system.ts)

Code Search (src/tools/search-code.ts)

Cognitive Memory Tools (src/tools/knowledge.ts)

Adaptive Context Tools (src/tools/memory.ts — engine mode only)

Context Engine Tools (src/tools/knowledge.ts — engine mode only)

Cognitive Memory

Key Components

Memory Kinds

Kind What it stores
fact Verifiable project knowledge
preference Working style preferences
decision Architectural choices with rationale
pattern Recurring approaches that work
mistake A failure and the lesson extracted from it
constraint A rule that must always hold

Scope Isolation (Multi-Context Safety)

To prevent cross-project context leaks, all memory queries and writes are isolated using scopes:

In project sessions, MemoryStore runs two queries and merges by entry id:

  1. Full project scopes (user + agent + context) — project-local facts and decisions.
  2. Global-only scopes (user + agent) — entries without a context: scope (preferences, cross-project patterns).

Global-only queries exclude entries that carry context:, so project facts stay local. Ranking is unified across the merged set; semantic conflicts between global and project entries are not auto-resolved.

Ranking and Decay

The memory retrieval system fuses multiple signals into a unified search score:

Schema Migrations

The SQLite schema evolves through additive ALTER TABLE migrations applied at every openMemoryDb() call. ensureLayerColumns() inspects PRAGMA table_info(entries) and runs each ALTER TABLE ... ADD COLUMN only if the column is missing — making the migrations idempotent and safe for existing installs. New columns always carry a DEFAULT so existing rows are populated automatically. The retracted column (added with the RETRACT opcode) defaults to 0, so all pre-existing entries remain visible until explicitly tombstoned.

Skill Effectiveness Feedback Loop

Source: src/skills/skill-stats-store.ts, src/memory/db.ts

At session start, SkillStatsStore.loadUsefulness() performs a dual-scope read from memory.db's skill_stats(skill_id, scope, usefulness, load_count, used_count, last_loaded_at) table — global skills first, project overrides on collision — and returns a Map<string, number> used to sort the skill catalog by descending usefulness. At session end (engine mode only), flush() updates each skill's score:

Co-occurrence pairs (skills resident together during a used turn) are recorded to skill_cooccurrence(scope, skill_a, skill_b, count) for a future ranking consumer (#161). The scope field (context:<hash(gitRoot)> for project skills, "" for global) prevents cross-project bleed — a project's custom skill score does not pollute scores in other repos.

Workflow Pattern Tracking

Source: src/context-engine/workflow-tracker.ts, src/context-engine/db.ts

After each session ends, persistSessionPattern() extracts:

  1. Tool sequence — globally-deduplicated first-occurrence-ordered list of tools called this session.
  2. Artifact types — frequency-sorted list of artifact kind labels (e.g. git_diff, tsc_errors, shell_output).

Both are stored in workflow_patterns(task_type, tool_sequence_json, artifact_types_json, hit_count, last_seen_at) keyed by hash(taskType + toolSequence + artifactTypes). Rows older than 30 days are pruned via pruneExpiredPatterns() inside runShutdownMaintenance().

At compile time, renderWorkflowContext() queries patterns matching the current taskType, selects the top-N by hit_count, and renders a compact Workflow Context section injected into the prompt just before the session checkpoint. This gives the engine a prior over which tools and artifact types are likely to appear — without adding it to the scored pool. Patterns for a different task type are never included, so a coding session's workflow history does not bleed into a debugging session.

Telemetry Scorecard

Source: src/context-engine/db.ts (scorecard table), src/context-engine/telemetry.ts, ScorecardTracker in session/turn

One row per session in the context-engine SQLite DB. No text content is stored — only numeric signals:

Signal group What's tracked
Context retrieve_artifact calls, repeat file-reads, search_turn_events calls, pressure events, compaction triggers
Memory recall call count, recall-used %, project-scoped validity/usefulness deltas at session start and end
Skills unique skill loads, total load events, reloads, underloads (loaded but never used), token cost
Resume read-path digests and skill catalog ids for dedup across resumed sessions

Active when: context_engine.enabled = true (always) or measurement_mode = true (classic/debug — scorecard-only DB, no engine tables). Counters persist across resume via persistProgress() called after every turn.

Query: /scorecard in-session shows the current session row. For cross-session A/B analysis, query the scorecard table in the context-engine DB directly (see #17).

Configuration

Config files are deep-merged from lower to higher precedence (later overrides earlier):

  1. Global JSON: ~/.praana/praana.config.json
  2. Global TOML: ~/.praana/config.toml
  3. Local JSON: ./praana.config.json
  4. Local TOML: ./praana.config.toml
[llm]
provider = "openrouter"               # openrouter | openai | deepseek | groq | xai | fireworks | together | ollama | opencode | anthropic | google | mistral | amazon-bedrock
model = "deepseek/deepseek-v4-pro"    # any model supported by the chosen provider

[memory]
enabled = true
summarizer = "openrouter"             # openrouter | disabled
db_path = "~/.praana/memory.db"

[compiler]
token_budget = 100000
recent_turns = 10
recent_turns_token_budget = 30000

[tiers]
idle_soft_after_turns = 20
idle_hard_after_turns = 50

[context_engine]
enabled = true                     # false = classic mode (full verbatim transcript)
measurement_mode = false           # write telemetry in classic mode (debug)
artifact_inline_threshold = 400
artifact_ttl_turns = 50
checkpoint_enabled = true          # narrative, plan history, constraints, decisions w/ rationale

[context_engine.distiller]
default_intensity = "full"         # lite | full

[context_engine.scoring]
w_pin = 1.0
w_recency = 0.5
w_relevance = 0.3
w_semantic = 0.3       # embedding similarity; combined with BM25 via max()
w_hydrate_boost = 0.2    # boost for context units overlapping auto-hydrated objects

[context_engine.pressure]
compact_at = 0.70
emergency_at = 0.85

[session]
log_dir = "~/.praana/sessions"

Env Vars

Mid-session model switching

/model is handled by model-resolver.ts and validated before the switch toast appears.

Syntax: /model [provider] <model-id> — provider is optional and space-separated only (not provider/model as a single token unless it is the model id on the current provider).

Resolution order:

  1. pi-ai static catalog (getModel(provider, id))
  2. Live provider catalog from GET {baseUrl}/models via provider-catalog.ts (6-hour disk cache at ~/.praana/provider-catalog-cache.json)
  3. Reject with an error toast if still unknown

Live catalog providers (OpenAI-compatible /models): OpenRouter, OpenCode, OpenAI, DeepSeek, Groq, xAI, Fireworks, Together, Ollama. Anthropic, Google, Mistral, and Bedrock rely on pi-ai only.

Persistence: successful switches write model_override and optionally provider_override system notes to the event log. session.ts restores the latest overrides on resume. Routing prefixes like openrouter/ or opencode/ are stripped before API calls.

UI and Slash Commands

PRAANA's current CLI requires an interactive TTY and launches the pi-tui terminal shell (src/ui/tui/). The TUI uses native scrollback, transcript replay, markdown rendering, identity/glance bars, slash-command/file autocomplete, toast overlays, buffered shell output, and full thinking-text rendering when /thinking on is enabled.

Slash commands are handled by src/slash-commands.ts:

CLI flags: praana --incognito, praana --debug, praana --config <path>, plus subcommands such as praana resume <session_id>, praana init, and praana memory dedupe. See src/app-banner.ts and src/cli-args.ts for the full list.