notedeck

One damus client to rule them all
git clone git://jb55.com/notedeck
Log | Files | Refs | README | LICENSE

commit 0e209c2bba52c78e529342e74a08ae9238ec0b9e
parent 6a5ada26f03fb9c0f67cdb695fba645421419276
Author: William Casarin <jb55@jb55.com>
Date:   Sun, 15 Feb 2026 19:13:07 -0800

wip: AI conversation nostr notes (shelved for redesign)

Initial implementation of JSONL to nostr event conversion:
- session_jsonl.rs: ProfileState-style Value wrapper parser
- path_normalize.rs: cwd-based path normalization
- session_events.rs: kind-1988 event builder with NIP-10 threading
- session_converter.rs: JSONL to ndb orchestrator
- session_loader.rs: ndb to Message loader (incomplete)
- Design doc with full spec

Shelved because the 1:1 JSONL-line-to-event model does not
cleanly handle mixed content blocks, tool_use/tool_result
pairing, or subagent conversation trees.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Diffstat:
Mcrates/notedeck_dave/src/lib.rs | 5+++++
Acrates/notedeck_dave/src/path_normalize.rs | 139+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Acrates/notedeck_dave/src/session_converter.rs | 73+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Acrates/notedeck_dave/src/session_events.rs | 377+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Acrates/notedeck_dave/src/session_jsonl.rs | 457+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Acrates/notedeck_dave/src/session_loader.rs | 104+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adocs/ai-conversation-nostr-design.md | 138+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
7 files changed, 1293 insertions(+), 0 deletions(-)

diff --git a/crates/notedeck_dave/src/lib.rs b/crates/notedeck_dave/src/lib.rs @@ -10,8 +10,13 @@ pub mod ipc; pub(crate) mod mesh; mod messages; mod quaternion; +mod path_normalize; pub mod session; +pub mod session_converter; pub mod session_discovery; +pub mod session_events; +pub mod session_jsonl; +pub mod session_loader; mod tools; mod ui; mod update; diff --git a/crates/notedeck_dave/src/path_normalize.rs b/crates/notedeck_dave/src/path_normalize.rs @@ -0,0 +1,139 @@ +//! Path normalization for JSONL source-data. +//! +//! When storing JSONL lines in nostr events, absolute paths are converted to +//! relative (using the session's `cwd` as base). On reconstruction, relative +//! paths are re-expanded using the local machine's working directory. +//! +//! This operates on the raw JSON string via string replacement — paths can +//! appear anywhere in tool inputs/outputs, so structural replacement would +//! miss nested occurrences. + +/// Replace all occurrences of `cwd` prefix in absolute paths with relative paths. +/// +/// For example, with cwd = "/Users/jb55/dev/notedeck": +/// "/Users/jb55/dev/notedeck/src/main.rs" → "src/main.rs" +/// "/Users/jb55/dev/notedeck" → "." +pub fn normalize_paths(json: &str, cwd: &str) -> String { + if cwd.is_empty() { + return json.to_string(); + } + + // Ensure cwd doesn't have a trailing slash for consistent matching + let cwd = cwd.strip_suffix('/').unwrap_or(cwd); + + // Replace "cwd/" prefix first (subpaths), then bare "cwd" (exact match) + let with_slash = format!("{}/", cwd); + let result = json.replace(&with_slash, ""); + + // Replace bare cwd (e.g. the cwd field itself) with "." + result.replace(cwd, ".") +} + +/// Re-expand relative paths back to absolute using the given local cwd. +/// +/// Reverses `normalize_paths`: the cwd field "." becomes the local cwd, +/// and relative paths get the cwd prefix prepended. +/// +/// Note: This is not perfectly inverse — it will also expand any unrelated +/// "." occurrences that happen to match. In practice, the cwd field is the +/// main target, and relative paths in tool inputs/outputs are the rest. +pub fn denormalize_paths(json: &str, local_cwd: &str) -> String { + if local_cwd.is_empty() { + return json.to_string(); + } + + let local_cwd = local_cwd.strip_suffix('/').unwrap_or(local_cwd); + + // We need to be careful about ordering here. We want to: + // 1. Replace "." (bare cwd reference) with the local cwd + // 2. Re-expand relative paths that were stripped of the cwd prefix + // + // But since normalized JSON has paths like "src/main.rs" (no prefix), + // we can't blindly prefix all bare paths. Instead, we reverse the + // exact transformations that normalize_paths applied: + // + // The normalize step replaced: + // "{cwd}/" → "" (paths become relative) + // "{cwd}" → "." (bare cwd references) + // + // So to reverse, we need context-aware replacement. The safest approach + // is to look for patterns that were likely produced by normalization: + // - JSON string values that are exactly "." → local_cwd + // - Relative paths in known field positions + // + // For now, we do simple string replacement which handles the most + // common case (the "cwd" field). Full path reconstruction for tool + // inputs/outputs would need the original field structure. + + // Replace "\"cwd\":\".\"" with the local cwd + let result = json.replace("\"cwd\":\".\"", &format!("\"cwd\":\"{}\"", local_cwd)); + + result +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_normalize_absolute_paths() { + let json = r#"{"cwd":"/Users/jb55/dev/notedeck","file":"/Users/jb55/dev/notedeck/src/main.rs"}"#; + let normalized = normalize_paths(json, "/Users/jb55/dev/notedeck"); + assert_eq!( + normalized, + r#"{"cwd":".","file":"src/main.rs"}"# + ); + } + + #[test] + fn test_normalize_with_trailing_slash() { + // cwd with trailing slash is stripped; the cwd value in JSON + // still contains the trailing slash so it becomes "" + "/" = "/" + // after replacing the base. In practice JSONL cwd values don't + // have trailing slashes. + let json = r#"{"cwd":"/tmp/project","file":"/tmp/project/lib.rs"}"#; + let normalized = normalize_paths(json, "/tmp/project/"); + assert_eq!(normalized, r#"{"cwd":".","file":"lib.rs"}"#); + } + + #[test] + fn test_normalize_empty_cwd() { + let json = r#"{"file":"/some/path"}"#; + let normalized = normalize_paths(json, ""); + assert_eq!(normalized, json); + } + + #[test] + fn test_normalize_no_matching_paths() { + let json = r#"{"file":"/other/path/file.rs"}"#; + let normalized = normalize_paths(json, "/Users/jb55/dev/notedeck"); + assert_eq!(normalized, json); + } + + #[test] + fn test_normalize_multiple_occurrences() { + let json = r#"{"old":"/Users/jb55/dev/notedeck/a.rs","new":"/Users/jb55/dev/notedeck/b.rs"}"#; + let normalized = normalize_paths(json, "/Users/jb55/dev/notedeck"); + assert_eq!(normalized, r#"{"old":"a.rs","new":"b.rs"}"#); + } + + #[test] + fn test_denormalize_cwd_field() { + let json = r#"{"cwd":"."}"#; + let denormalized = denormalize_paths(json, "/Users/jb55/dev/notedeck"); + assert_eq!( + denormalized, + r#"{"cwd":"/Users/jb55/dev/notedeck"}"# + ); + } + + #[test] + fn test_normalize_roundtrip_cwd() { + let original_cwd = "/Users/jb55/dev/notedeck"; + let json = r#"{"cwd":"/Users/jb55/dev/notedeck"}"#; + let normalized = normalize_paths(json, original_cwd); + assert_eq!(normalized, r#"{"cwd":"."}"#); + let denormalized = denormalize_paths(&normalized, original_cwd); + assert_eq!(denormalized, json); + } +} diff --git a/crates/notedeck_dave/src/session_converter.rs b/crates/notedeck_dave/src/session_converter.rs @@ -0,0 +1,73 @@ +//! Orchestrates converting a claude-code session JSONL file into nostr events. +//! +//! Reads the JSONL file line-by-line, builds kind-1988 nostr events with +//! proper threading, and ingests them into the local nostr database. + +use crate::session_events::{self, BuiltEvent, ThreadingState}; +use crate::session_jsonl::JsonlLine; +use nostrdb::{Ndb, IngestMetadata}; +use std::fs::File; +use std::io::{BufRead, BufReader}; +use std::path::Path; + +/// Convert a session JSONL file into nostr events and ingest them locally. +/// +/// Returns the ordered list of note IDs for the ingested events. +pub fn convert_session_to_events( + jsonl_path: &Path, + ndb: &Ndb, + secret_key: &[u8; 32], +) -> Result<Vec<[u8; 32]>, ConvertError> { + let file = File::open(jsonl_path).map_err(ConvertError::Io)?; + let reader = BufReader::new(file); + + let mut threading = ThreadingState::new(); + let mut note_ids = Vec::new(); + + for (line_num, line) in reader.lines().enumerate() { + let line = line.map_err(ConvertError::Io)?; + if line.trim().is_empty() { + continue; + } + + let parsed = JsonlLine::parse(&line).map_err(|e| { + ConvertError::Parse(format!("line {}: {}", line_num + 1, e)) + })?; + + let events = session_events::build_events(&parsed, &mut threading, secret_key) + .map_err(|e| ConvertError::Build(format!("line {}: {}", line_num + 1, e)))?; + + for event in events { + ingest_event(ndb, &event)?; + note_ids.push(event.note_id); + } + } + + Ok(note_ids) +} + +/// Ingest a single built event into the local ndb. +fn ingest_event(ndb: &Ndb, event: &BuiltEvent) -> Result<(), ConvertError> { + ndb.process_event_with(&event.json, IngestMetadata::new().client(true)) + .map_err(|e| ConvertError::Ingest(format!("{:?}", e)))?; + Ok(()) +} + +#[derive(Debug)] +pub enum ConvertError { + Io(std::io::Error), + Parse(String), + Build(String), + Ingest(String), +} + +impl std::fmt::Display for ConvertError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + ConvertError::Io(e) => write!(f, "IO error: {}", e), + ConvertError::Parse(e) => write!(f, "parse error: {}", e), + ConvertError::Build(e) => write!(f, "build error: {}", e), + ConvertError::Ingest(e) => write!(f, "ingest error: {}", e), + } + } +} diff --git a/crates/notedeck_dave/src/session_events.rs b/crates/notedeck_dave/src/session_events.rs @@ -0,0 +1,377 @@ +//! Convert parsed JSONL lines into kind-1988 nostr events. +//! +//! Each JSONL line becomes one or more nostr events. Assistant messages with +//! mixed content (text + tool_use blocks) are split into separate events. +//! Events are threaded using NIP-10 `e` tags with root/reply markers. + +use crate::path_normalize; +use crate::session_jsonl::{self, ContentBlock, JsonlLine}; +use nostrdb::{NoteBuilder, NoteBuildOptions}; +use std::collections::HashMap; + +/// Nostr event kind for AI conversation notes. +pub const AI_CONVERSATION_KIND: u32 = 1988; + +/// A built nostr event ready for ingestion, with its note ID. +#[derive(Debug)] +pub struct BuiltEvent { + /// The full JSON string: `["EVENT", {…}]` + pub json: String, + /// The 32-byte note ID (from the signed event). + pub note_id: [u8; 32], +} + +/// Maintains threading state across a session's events. +pub struct ThreadingState { + /// Maps JSONL uuid → nostr note ID (32 bytes). + uuid_to_note_id: HashMap<String, [u8; 32]>, + /// The note ID of the first event in the session (root). + root_note_id: Option<[u8; 32]>, + /// The note ID of the most recently built event. + last_note_id: Option<[u8; 32]>, +} + +impl ThreadingState { + pub fn new() -> Self { + Self { + uuid_to_note_id: HashMap::new(), + root_note_id: None, + last_note_id: None, + } + } + + /// Record a built event's note ID, associated with a JSONL uuid. + fn record(&mut self, uuid: Option<&str>, note_id: [u8; 32]) { + if self.root_note_id.is_none() { + self.root_note_id = Some(note_id); + } + if let Some(uuid) = uuid { + self.uuid_to_note_id.insert(uuid.to_string(), note_id); + } + self.last_note_id = Some(note_id); + } +} + +/// Build nostr events from a single JSONL line. +/// +/// Returns one or more events. Assistant messages with mixed content blocks +/// (text + tool_use) are split into multiple events, one per block. +/// +/// `secret_key` is the 32-byte secret key for signing events. +pub fn build_events( + line: &JsonlLine, + threading: &mut ThreadingState, + secret_key: &[u8; 32], +) -> Result<Vec<BuiltEvent>, EventBuildError> { + let msg = line.message(); + let is_assistant = line.line_type() == Some("assistant"); + + // Check if this is an assistant message with multiple content blocks + // that should be split into separate events + let blocks: Vec<ContentBlock<'_>> = if is_assistant { + msg.as_ref() + .map(|m| m.content_blocks()) + .unwrap_or_default() + } else { + vec![] + }; + + let should_split = is_assistant && blocks.len() > 1; + + if should_split { + // Build one event per content block + let mut events = Vec::with_capacity(blocks.len()); + for block in &blocks { + let content = session_jsonl::display_content_for_block(block); + let role = match block { + ContentBlock::Text(_) => "assistant", + ContentBlock::ToolUse { .. } => "tool_call", + ContentBlock::ToolResult { .. } => "tool_result", + }; + + let event = build_single_event( + line, + &content, + role, + threading, + secret_key, + )?; + threading.record(line.uuid(), event.note_id); + events.push(event); + } + Ok(events) + } else { + // Single event for the line + let content = session_jsonl::extract_display_content(line); + let role = line.role().unwrap_or("unknown"); + + let event = build_single_event( + line, + &content, + role, + threading, + secret_key, + )?; + threading.record(line.uuid(), event.note_id); + Ok(vec![event]) + } +} + +#[derive(Debug)] +pub enum EventBuildError { + Build(String), + Serialize(String), +} + +impl std::fmt::Display for EventBuildError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + EventBuildError::Build(e) => write!(f, "failed to build note: {}", e), + EventBuildError::Serialize(e) => write!(f, "failed to serialize event: {}", e), + } + } +} + +/// Build a single nostr event from a JSONL line. +fn build_single_event( + line: &JsonlLine, + content: &str, + role: &str, + threading: &ThreadingState, + secret_key: &[u8; 32], +) -> Result<BuiltEvent, EventBuildError> { + let mut builder = NoteBuilder::new() + .kind(AI_CONVERSATION_KIND) + .content(content) + .options(NoteBuildOptions::default()); + + // Set timestamp from JSONL + if let Some(ts) = line.timestamp_secs() { + builder = builder.created_at(ts); + } + + // -- Session identity tags -- + if let Some(session_id) = line.session_id() { + builder = builder.start_tag().tag_str("d").tag_str(session_id); + } + if let Some(slug) = line.slug() { + builder = builder.start_tag().tag_str("session-slug").tag_str(slug); + } + + // -- Threading tags (NIP-10) -- + if let Some(root_id) = threading.root_note_id { + builder = builder + .start_tag() + .tag_str("e") + .tag_id(&root_id) + .tag_str("") + .tag_str("root"); + } + if let Some(reply_id) = threading.last_note_id { + builder = builder + .start_tag() + .tag_str("e") + .tag_id(&reply_id) + .tag_str("") + .tag_str("reply"); + } + + // -- Message metadata tags -- + builder = builder.start_tag().tag_str("source").tag_str("claude-code"); + + if let Some(version) = line.version() { + builder = builder + .start_tag() + .tag_str("source-version") + .tag_str(version); + } + + builder = builder.start_tag().tag_str("role").tag_str(role); + + // Model tag (for assistant messages) + if let Some(msg) = line.message() { + if let Some(model) = msg.model() { + builder = builder.start_tag().tag_str("model").tag_str(model); + } + } + + if let Some(line_type) = line.line_type() { + builder = builder + .start_tag() + .tag_str("turn-type") + .tag_str(line_type); + } + + // -- Discoverability -- + builder = builder + .start_tag() + .tag_str("t") + .tag_str("ai-conversation"); + + // -- Source data (lossless) -- + let raw_json = line.to_json(); + let source_data = if let Some(cwd) = line.cwd() { + path_normalize::normalize_paths(&raw_json, cwd) + } else { + raw_json + }; + builder = builder + .start_tag() + .tag_str("source-data") + .tag_str(&source_data); + + // Sign and build + let note = builder + .sign(secret_key) + .build() + .ok_or_else(|| EventBuildError::Build("NoteBuilder::build returned None".to_string()))?; + + let note_id: [u8; 32] = *note.id(); + + let event = enostr::ClientMessage::event(&note) + .map_err(|e| EventBuildError::Serialize(format!("{:?}", e)))?; + + let json = event + .to_json() + .map_err(|e| EventBuildError::Serialize(format!("{:?}", e)))?; + + Ok(BuiltEvent { json, note_id }) +} + +#[cfg(test)] +mod tests { + use super::*; + + // Test secret key (32 bytes, not for real use) + fn test_secret_key() -> [u8; 32] { + let mut key = [0u8; 32]; + key[0] = 1; // non-zero so signing works + key + } + + #[test] + fn test_build_user_text_event() { + let line = JsonlLine::parse( + r#"{"type":"user","uuid":"u1","parentUuid":null,"sessionId":"sess1","timestamp":"2026-02-09T20:43:35.675Z","cwd":"/tmp/project","version":"2.0.64","message":{"role":"user","content":"Human: hello world\n\n"}}"#, + ) + .unwrap(); + + let mut threading = ThreadingState::new(); + let events = build_events(&line, &mut threading, &test_secret_key()).unwrap(); + + assert_eq!(events.len(), 1); + assert!(threading.root_note_id.is_some()); + assert_eq!(threading.root_note_id, Some(events[0].note_id)); + + // Verify the JSON contains our kind and tags + let json = &events[0].json; + assert!(json.contains("1988")); + assert!(json.contains("source")); + assert!(json.contains("claude-code")); + assert!(json.contains("role")); + assert!(json.contains("user")); + assert!(json.contains("source-data")); + } + + #[test] + fn test_build_assistant_text_event() { + let line = JsonlLine::parse( + r#"{"type":"assistant","uuid":"u2","parentUuid":"u1","sessionId":"sess1","timestamp":"2026-02-09T20:43:38.421Z","cwd":"/tmp/project","version":"2.0.64","message":{"role":"assistant","model":"claude-opus-4-5-20251101","content":[{"type":"text","text":"I can help with that."}]}}"#, + ) + .unwrap(); + + let mut threading = ThreadingState::new(); + // Simulate a prior event + threading.root_note_id = Some([1u8; 32]); + threading.last_note_id = Some([1u8; 32]); + + let events = build_events(&line, &mut threading, &test_secret_key()).unwrap(); + assert_eq!(events.len(), 1); + + let json = &events[0].json; + assert!(json.contains("assistant")); + assert!(json.contains("claude-opus-4-5-20251101")); // model tag + } + + #[test] + fn test_build_split_assistant_mixed_content() { + let line = JsonlLine::parse( + r#"{"type":"assistant","uuid":"u3","sessionId":"sess1","timestamp":"2026-02-09T20:00:00Z","cwd":"/tmp","version":"2.0.64","message":{"role":"assistant","model":"claude-opus-4-5-20251101","content":[{"type":"text","text":"Let me check."},{"type":"tool_use","id":"t1","name":"Read","input":{"file_path":"/tmp/test.rs"}}]}}"#, + ) + .unwrap(); + + let mut threading = ThreadingState::new(); + let events = build_events(&line, &mut threading, &test_secret_key()).unwrap(); + + // Should produce 2 events: one text, one tool_call + assert_eq!(events.len(), 2); + + // Both should have unique note IDs + assert_ne!(events[0].note_id, events[1].note_id); + } + + #[test] + fn test_threading_chain() { + let lines = vec![ + r#"{"type":"user","uuid":"u1","parentUuid":null,"sessionId":"s","timestamp":"2026-02-09T20:00:00Z","cwd":"/tmp","version":"2.0.64","message":{"role":"user","content":"hello"}}"#, + r#"{"type":"assistant","uuid":"u2","parentUuid":"u1","sessionId":"s","timestamp":"2026-02-09T20:00:01Z","cwd":"/tmp","version":"2.0.64","message":{"role":"assistant","content":[{"type":"text","text":"hi"}]}}"#, + r#"{"type":"user","uuid":"u3","parentUuid":"u2","sessionId":"s","timestamp":"2026-02-09T20:00:02Z","cwd":"/tmp","version":"2.0.64","message":{"role":"user","content":"bye"}}"#, + ]; + + let mut threading = ThreadingState::new(); + let sk = test_secret_key(); + let mut all_events = vec![]; + + for line_str in &lines { + let line = JsonlLine::parse(line_str).unwrap(); + let events = build_events(&line, &mut threading, &sk).unwrap(); + all_events.extend(events); + } + + assert_eq!(all_events.len(), 3); + + // First event should be root (no e tags) + // Subsequent events should reference root + previous + // We can't easily inspect the binary note, but we can verify + // the JSON contains "root" and "reply" markers + assert!(!all_events[0].json.contains("root")); + assert!(all_events[1].json.contains("root")); + assert!(all_events[1].json.contains("reply")); + assert!(all_events[2].json.contains("root")); + assert!(all_events[2].json.contains("reply")); + } + + #[test] + fn test_path_normalization_in_source_data() { + let line = JsonlLine::parse( + r#"{"type":"user","uuid":"u1","sessionId":"s","timestamp":"2026-02-09T20:00:00Z","cwd":"/Users/jb55/dev/notedeck","version":"2.0.64","message":{"role":"user","content":"check /Users/jb55/dev/notedeck/src/main.rs"}}"#, + ) + .unwrap(); + + let mut threading = ThreadingState::new(); + let events = build_events(&line, &mut threading, &test_secret_key()).unwrap(); + + // The source-data tag should have normalized paths + let json = &events[0].json; + // Should NOT contain the absolute cwd path in source-data + // (it's normalized to ".") + assert!(json.contains("source-data")); + // The source-data value should have relative paths + // This is a basic check — the full round-trip test will verify this properly + } + + #[test] + fn test_queue_operation_event() { + let line = JsonlLine::parse( + r#"{"type":"queue-operation","operation":"dequeue","timestamp":"2026-02-09T20:43:35.669Z","sessionId":"sess1"}"#, + ) + .unwrap(); + + let mut threading = ThreadingState::new(); + let events = build_events(&line, &mut threading, &test_secret_key()).unwrap(); + assert_eq!(events.len(), 1); + + let json = &events[0].json; + assert!(json.contains("queue-operation")); + } +} diff --git a/crates/notedeck_dave/src/session_jsonl.rs b/crates/notedeck_dave/src/session_jsonl.rs @@ -0,0 +1,457 @@ +//! Parse claude-code session JSONL lines with lossless round-trip support. +//! +//! Follows the `ProfileState` pattern from `enostr::profile` — wraps a +//! `serde_json::Value` with typed accessors so we can read the fields we +//! need for nostr event construction while preserving the raw JSON for +//! the `source-data` tag. + +use serde_json::Value; + +/// A single line from a claude-code session JSONL file. +/// +/// Wraps the raw JSON value to preserve all fields for lossless round-trip +/// via the `source-data` nostr tag. +#[derive(Debug, Clone)] +pub struct JsonlLine(Value); + +impl JsonlLine { + /// Parse a JSONL line from a string. + pub fn parse(line: &str) -> Result<Self, serde_json::Error> { + let value: Value = serde_json::from_str(line)?; + Ok(Self(value)) + } + + /// The raw JSON value. + pub fn value(&self) -> &Value { + &self.0 + } + + /// Serialize back to a JSON string (lossless). + pub fn to_json(&self) -> String { + // serde_json::to_string on a Value is infallible + serde_json::to_string(&self.0).unwrap() + } + + // -- Top-level field accessors -- + + fn get_str(&self, key: &str) -> Option<&str> { + self.0.get(key).and_then(|v| v.as_str()) + } + + /// The JSONL line type: "user", "assistant", "progress", "queue-operation", + /// "file-history-snapshot" + pub fn line_type(&self) -> Option<&str> { + self.get_str("type") + } + + pub fn uuid(&self) -> Option<&str> { + self.get_str("uuid") + } + + pub fn parent_uuid(&self) -> Option<&str> { + self.get_str("parentUuid") + } + + pub fn session_id(&self) -> Option<&str> { + self.get_str("sessionId") + } + + pub fn timestamp(&self) -> Option<&str> { + self.get_str("timestamp") + } + + /// Parse the timestamp as a unix timestamp (seconds). + pub fn timestamp_secs(&self) -> Option<u64> { + let ts_str = self.timestamp()?; + let dt = chrono::DateTime::parse_from_rfc3339(ts_str).ok()?; + Some(dt.timestamp() as u64) + } + + pub fn cwd(&self) -> Option<&str> { + self.get_str("cwd") + } + + pub fn git_branch(&self) -> Option<&str> { + self.get_str("gitBranch") + } + + pub fn version(&self) -> Option<&str> { + self.get_str("version") + } + + pub fn slug(&self) -> Option<&str> { + self.get_str("slug") + } + + /// The `message` object, if present. + pub fn message(&self) -> Option<JsonlMessage<'_>> { + self.0.get("message").map(JsonlMessage) + } + + /// For queue-operation lines: the operation type ("dequeue", etc.) + pub fn operation(&self) -> Option<&str> { + self.get_str("operation") + } + + /// Determine the role string for nostr event tagging. + /// + /// This maps the JSONL structure to the design spec's role values: + /// user (text) → "user", user (tool_result) → "tool_result", + /// assistant (text) → "assistant", assistant (tool_use) → "tool_call", + /// progress → "progress", etc. + pub fn role(&self) -> Option<&str> { + match self.line_type()? { + "user" => { + // Check if the content is a tool_result array + if let Some(msg) = self.message() { + if msg.has_tool_result_content() { + return Some("tool_result"); + } + } + Some("user") + } + "assistant" => Some("assistant"), + "progress" => Some("progress"), + "queue-operation" => Some("queue-operation"), + "file-history-snapshot" => Some("file-history-snapshot"), + _ => None, + } + } +} + +/// A borrowed view into the `message` object of a JSONL line. +#[derive(Debug, Clone, Copy)] +pub struct JsonlMessage<'a>(&'a Value); + +impl<'a> JsonlMessage<'a> { + fn get_str(&self, key: &str) -> Option<&'a str> { + self.0.get(key).and_then(|v| v.as_str()) + } + + pub fn role(&self) -> Option<&'a str> { + self.get_str("role") + } + + pub fn model(&self) -> Option<&'a str> { + self.get_str("model") + } + + /// The raw content value — can be a string or an array of content blocks. + pub fn content(&self) -> Option<&'a Value> { + self.0.get("content") + } + + /// Check if content contains tool_result blocks (user messages with tool results). + pub fn has_tool_result_content(&self) -> bool { + match self.content() { + Some(Value::Array(arr)) => arr + .iter() + .any(|block| block.get("type").and_then(|t| t.as_str()) == Some("tool_result")), + _ => false, + } + } + + /// Extract the content blocks as an iterator. + pub fn content_blocks(&self) -> Vec<ContentBlock<'a>> { + match self.content() { + Some(Value::String(s)) => vec![ContentBlock::Text(s.as_str())], + Some(Value::Array(arr)) => arr.iter().filter_map(ContentBlock::from_value).collect(), + _ => vec![], + } + } + + /// Extract just the text from text blocks, concatenated. + pub fn text_content(&self) -> Option<String> { + let blocks = self.content_blocks(); + let texts: Vec<&str> = blocks + .iter() + .filter_map(|b| match b { + ContentBlock::Text(t) => Some(*t), + _ => None, + }) + .collect(); + + if texts.is_empty() { + None + } else { + Some(texts.join("")) + } + } +} + +/// A content block from an assistant or user message. +#[derive(Debug, Clone)] +pub enum ContentBlock<'a> { + /// Plain text content. + Text(&'a str), + /// A tool use request (assistant → tool). + ToolUse { + id: &'a str, + name: &'a str, + input: &'a Value, + }, + /// A tool result (tool → user message). + ToolResult { + tool_use_id: &'a str, + content: &'a Value, + }, +} + +impl<'a> ContentBlock<'a> { + fn from_value(value: &'a Value) -> Option<Self> { + let block_type = value.get("type")?.as_str()?; + match block_type { + "text" => { + let text = value.get("text")?.as_str()?; + Some(ContentBlock::Text(text)) + } + "tool_use" => { + let id = value.get("id")?.as_str()?; + let name = value.get("name")?.as_str()?; + let input = value.get("input")?; + Some(ContentBlock::ToolUse { id, name, input }) + } + "tool_result" => { + let tool_use_id = value.get("tool_use_id")?.as_str()?; + let content = value.get("content")?; + Some(ContentBlock::ToolResult { + tool_use_id, + content, + }) + } + _ => None, + } + } +} + +/// Human-readable content extraction for the nostr event `content` field. +/// +/// This produces the text that goes into the nostr event content, +/// suitable for rendering in any nostr client. +pub fn extract_display_content(line: &JsonlLine) -> String { + match line.line_type() { + Some("user") => { + if let Some(msg) = line.message() { + if msg.has_tool_result_content() { + // Tool result content — summarize + let blocks = msg.content_blocks(); + let summaries: Vec<String> = blocks + .iter() + .filter_map(|b| match b { + ContentBlock::ToolResult { content, .. } => match content { + Value::String(s) => { + Some(truncate_str(s, 500)) + } + _ => Some("[tool result]".to_string()), + }, + _ => None, + }) + .collect(); + summaries.join("\n") + } else if let Some(text) = msg.text_content() { + // Strip "Human: " prefix if present (claude-code adds it) + text.strip_prefix("Human: ").unwrap_or(&text).to_string() + } else { + String::new() + } + } else { + String::new() + } + } + Some("assistant") => { + if let Some(msg) = line.message() { + // For assistant messages, we'll produce content for each block. + // The caller handles splitting into multiple events for mixed content. + if let Some(text) = msg.text_content() { + text + } else { + String::new() + } + } else { + String::new() + } + } + Some("progress") => line + .message() + .and_then(|m| m.text_content()) + .unwrap_or_else(|| "[progress]".to_string()), + Some("queue-operation") => { + let op = line.operation().unwrap_or("unknown"); + format!("[queue: {}]", op) + } + Some("file-history-snapshot") => "[file history snapshot]".to_string(), + _ => String::new(), + } +} + +/// Extract display content for a single content block (for assistant messages +/// that need to be split into multiple events). +pub fn display_content_for_block(block: &ContentBlock<'_>) -> String { + match block { + ContentBlock::Text(text) => text.to_string(), + ContentBlock::ToolUse { name, input, .. } => { + // Compact summary: tool name + truncated input + let input_str = serde_json::to_string(input).unwrap_or_default(); + let input_preview = truncate_str(&input_str, 200); + format!("Tool: {} {}", name, input_preview) + } + ContentBlock::ToolResult { content, .. } => match content { + Value::String(s) => truncate_str(s, 500), + _ => "[tool result]".to_string(), + }, + } +} + +fn truncate_str(s: &str, max_chars: usize) -> String { + if s.chars().count() <= max_chars { + s.to_string() + } else { + let truncated: String = s.chars().take(max_chars).collect(); + format!("{}...", truncated) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_parse_user_text_message() { + let line = r#"{"parentUuid":null,"cwd":"/Users/jb55/dev/notedeck","sessionId":"abc-123","version":"2.0.64","gitBranch":"main","type":"user","message":{"role":"user","content":"Human: Hello world\n\n"},"uuid":"uuid-1","timestamp":"2026-02-09T20:43:35.675Z"}"#; + + let parsed = JsonlLine::parse(line).unwrap(); + assert_eq!(parsed.line_type(), Some("user")); + assert_eq!(parsed.uuid(), Some("uuid-1")); + assert_eq!(parsed.parent_uuid(), None); + assert_eq!(parsed.session_id(), Some("abc-123")); + assert_eq!(parsed.cwd(), Some("/Users/jb55/dev/notedeck")); + assert_eq!(parsed.version(), Some("2.0.64")); + assert_eq!(parsed.role(), Some("user")); + + let msg = parsed.message().unwrap(); + assert_eq!(msg.role(), Some("user")); + assert_eq!(msg.text_content(), Some("Human: Hello world\n\n".to_string())); + + let content = extract_display_content(&parsed); + assert_eq!(content, "Hello world\n\n"); + } + + #[test] + fn test_parse_assistant_text_message() { + let line = r#"{"parentUuid":"uuid-1","cwd":"/Users/jb55/dev/notedeck","sessionId":"abc-123","version":"2.0.64","message":{"model":"claude-opus-4-5-20251101","role":"assistant","content":[{"type":"text","text":"I can help with that."}]},"type":"assistant","uuid":"uuid-2","timestamp":"2026-02-09T20:43:38.421Z"}"#; + + let parsed = JsonlLine::parse(line).unwrap(); + assert_eq!(parsed.line_type(), Some("assistant")); + assert_eq!(parsed.role(), Some("assistant")); + + let msg = parsed.message().unwrap(); + assert_eq!(msg.model(), Some("claude-opus-4-5-20251101")); + assert_eq!( + msg.text_content(), + Some("I can help with that.".to_string()) + ); + } + + #[test] + fn test_parse_assistant_tool_use() { + let line = r#"{"parentUuid":"uuid-1","cwd":"/tmp","sessionId":"abc","version":"2.0.64","message":{"model":"claude-opus-4-5-20251101","role":"assistant","content":[{"type":"tool_use","id":"toolu_123","name":"Read","input":{"file_path":"/tmp/test.rs"}}]},"type":"assistant","uuid":"uuid-3","timestamp":"2026-02-09T20:43:38.421Z"}"#; + + let parsed = JsonlLine::parse(line).unwrap(); + let msg = parsed.message().unwrap(); + let blocks = msg.content_blocks(); + assert_eq!(blocks.len(), 1); + + match &blocks[0] { + ContentBlock::ToolUse { id, name, input } => { + assert_eq!(*id, "toolu_123"); + assert_eq!(*name, "Read"); + assert_eq!(input.get("file_path").unwrap().as_str(), Some("/tmp/test.rs")); + } + _ => panic!("expected ToolUse block"), + } + } + + #[test] + fn test_parse_user_tool_result() { + let line = r#"{"parentUuid":"uuid-3","cwd":"/tmp","sessionId":"abc","version":"2.0.64","type":"user","message":{"role":"user","content":[{"tool_use_id":"toolu_123","type":"tool_result","content":"file contents here"}]},"uuid":"uuid-4","timestamp":"2026-02-09T20:43:38.476Z"}"#; + + let parsed = JsonlLine::parse(line).unwrap(); + assert_eq!(parsed.role(), Some("tool_result")); + + let msg = parsed.message().unwrap(); + assert!(msg.has_tool_result_content()); + + let blocks = msg.content_blocks(); + assert_eq!(blocks.len(), 1); + match &blocks[0] { + ContentBlock::ToolResult { + tool_use_id, + content, + } => { + assert_eq!(*tool_use_id, "toolu_123"); + assert_eq!(content.as_str(), Some("file contents here")); + } + _ => panic!("expected ToolResult block"), + } + } + + #[test] + fn test_parse_queue_operation() { + let line = r#"{"type":"queue-operation","operation":"dequeue","timestamp":"2026-02-09T20:43:35.669Z","sessionId":"abc-123"}"#; + + let parsed = JsonlLine::parse(line).unwrap(); + assert_eq!(parsed.line_type(), Some("queue-operation")); + assert_eq!(parsed.operation(), Some("dequeue")); + assert_eq!(parsed.role(), Some("queue-operation")); + + let content = extract_display_content(&parsed); + assert_eq!(content, "[queue: dequeue]"); + } + + #[test] + fn test_lossless_roundtrip() { + // The key property: parse → to_json should preserve all fields + let original = r#"{"type":"user","uuid":"abc","parentUuid":null,"sessionId":"sess","timestamp":"2026-02-09T20:43:35.675Z","cwd":"/tmp","gitBranch":"main","version":"2.0.64","isSidechain":false,"userType":"external","message":{"role":"user","content":"hello"},"unknownField":"preserved"}"#; + + let parsed = JsonlLine::parse(original).unwrap(); + let roundtripped = parsed.to_json(); + + // Parse both as Value to compare (field order may differ) + let orig_val: Value = serde_json::from_str(original).unwrap(); + let rt_val: Value = serde_json::from_str(&roundtripped).unwrap(); + assert_eq!(orig_val, rt_val); + } + + #[test] + fn test_timestamp_secs() { + let line = r#"{"type":"user","timestamp":"2026-02-09T20:43:35.675Z","sessionId":"abc"}"#; + let parsed = JsonlLine::parse(line).unwrap(); + assert!(parsed.timestamp_secs().is_some()); + } + + #[test] + fn test_mixed_assistant_content() { + let line = r#"{"type":"assistant","uuid":"u1","sessionId":"s","timestamp":"2026-02-09T20:00:00Z","message":{"role":"assistant","model":"claude-opus-4-5-20251101","content":[{"type":"text","text":"Here is what I found:"},{"type":"tool_use","id":"t1","name":"Glob","input":{"pattern":"**/*.rs"}}]}}"#; + + let parsed = JsonlLine::parse(line).unwrap(); + let msg = parsed.message().unwrap(); + let blocks = msg.content_blocks(); + assert_eq!(blocks.len(), 2); + + // First block is text + assert!(matches!(blocks[0], ContentBlock::Text("Here is what I found:"))); + + // Second block is tool use + match &blocks[1] { + ContentBlock::ToolUse { name, .. } => assert_eq!(*name, "Glob"), + _ => panic!("expected ToolUse"), + } + + // display_content_for_block should work on each + assert_eq!( + display_content_for_block(&blocks[0]), + "Here is what I found:" + ); + assert!(display_content_for_block(&blocks[1]).starts_with("Tool: Glob")); + } +} diff --git a/crates/notedeck_dave/src/session_loader.rs b/crates/notedeck_dave/src/session_loader.rs @@ -0,0 +1,104 @@ +//! Load a previous session's conversation from nostr events in ndb. +//! +//! Queries for kind-1988 events with a matching `d` tag (session ID), +//! orders them by created_at, and converts them into `Message` variants +//! for populating the chat UI. + +use crate::messages::{AssistantMessage, ToolResult}; +use crate::session_events::AI_CONVERSATION_KIND; +use crate::Message; +use nostrdb::{Filter, Ndb, Transaction}; + +/// Load conversation messages from ndb for a given session ID. +/// +/// Returns messages in chronological order, suitable for populating +/// `ChatSession.chat` before streaming begins. +pub fn load_session_messages( + ndb: &Ndb, + txn: &Transaction, + session_id: &str, +) -> Vec<Message> { + let filter = Filter::new() + .kinds([AI_CONVERSATION_KIND as u64]) + .tags([session_id], 'd') + .limit(10000) + .build(); + + let results = match ndb.query(txn, &[filter], 10000) { + Ok(r) => r, + Err(_) => return vec![], + }; + + // Collect notes with their created_at for sorting + let mut notes: Vec<_> = results + .iter() + .filter_map(|qr| { + ndb.get_note_by_key(txn, qr.note_key).ok() + }) + .collect(); + + // Sort by created_at (chronological order) + notes.sort_by_key(|note| note.created_at()); + + let mut messages = Vec::new(); + for note in &notes { + let content = note.content(); + let role = get_tag_value(note, "role"); + + let msg = match role.as_deref() { + Some("user") => Some(Message::User(content.to_string())), + Some("assistant") => { + Some(Message::Assistant(AssistantMessage::from_text( + content.to_string(), + ))) + } + Some("tool_call") => { + // Tool calls are displayed as assistant messages in the UI + Some(Message::Assistant(AssistantMessage::from_text( + content.to_string(), + ))) + } + Some("tool_result") => { + // Extract tool name from content if possible + // Content format is the tool output text + let tool_name = "tool".to_string(); + let summary = truncate(content, 100); + Some(Message::ToolResult(ToolResult { + tool_name, + summary, + })) + } + // Skip progress, queue-operation, file-history-snapshot for UI + _ => None, + }; + + if let Some(msg) = msg { + messages.push(msg); + } + } + + messages +} + +/// Extract the value of a named tag from a note. +fn get_tag_value<'a>(note: &'a nostrdb::Note<'a>, tag_name: &str) -> Option<String> { + for tag in note.tags() { + if tag.count() >= 2 { + if let Some(name) = tag.get_str(0) { + if name == tag_name { + return tag.get_str(1).map(|s| s.to_string()); + } + } + } + } + None +} + +fn truncate(s: &str, max_chars: usize) -> String { + if s.chars().count() <= max_chars { + s.to_string() + } else { + let truncated: String = s.chars().take(max_chars).collect(); + format!("{}...", truncated) + } +} diff --git a/docs/ai-conversation-nostr-design.md b/docs/ai-conversation-nostr-design.md @@ -0,0 +1,138 @@ +# AI Conversation Nostr Notes — Design Spec + +## Overview + +Represent claude-code session JSONL lines as nostr events, enabling: +1. **Session presentation resume** — reload a previous session's UI from local nostr DB without re-parsing JSONL +2. **Round-trip fidelity** — reconstruct the original JSONL from nostr events for claude-code `--resume` +3. **Future sharing** — structure is ready for publishing sessions to relays (with privacy considerations deferred) + +## Architecture + +``` +claude-code JSONL ──→ nostr events ──→ ndb.process_event (local) + │ + ├──→ UI rendering (presentation from nostr query) + └──→ JSONL reconstruction (for --resume) +``` + +## Event Structure + +**Kind**: Regular event (1000-9999 range, specific number TBD). Immutable — no replaceable events. + +Each JSONL line becomes one nostr event. Every message type (user, assistant, tool_call, tool_result, progress, etc.) gets its own note for 1:1 JSONL line reconstruction. + +### Note Format + +```json +{ + "kind": "<TBD>", + "content": "<human-readable presentation text>", + "tags": [ + // Session identity + ["d", "<session-id>"], + ["session-slug", "<human-readable-name>"], + + // Threading (NIP-10) + ["e", "<root-note-id>", "", "root"], + ["e", "<parent-note-id>", "", "reply"], + + // Message metadata + ["source", "claude-code"], + ["source-version", "2.1.42"], + ["role", "<user|assistant|system|tool_call|tool_result>"], + ["model", "claude-opus-4-6"], + ["turn-type", "<JSONL type field: user|assistant|progress|queue-operation|file-history-snapshot>"], + + // Discoverability + ["t", "ai-conversation"], + + // Lossless reconstruction (Option 3) + ["source-data", "<JSON-escaped JSONL line with paths normalized>"] + ] +} +``` + +### Content Field + +Human-readable text suitable for rendering in any nostr client: +- **user**: The user's message text +- **assistant**: The assistant's rendered markdown text (text blocks only) +- **tool_call**: Summary like `Glob: {"pattern": "**/*.rs"}` or tool name + input preview +- **tool_result**: The tool output text (possibly truncated for presentation) +- **progress**: Description of the progress event +- **queue-operation / file-history-snapshot**: Minimal description + +### source-data Tag + +Contains the **full JSONL line** as a JSON string with these transformations applied: +- **Path normalization**: All absolute paths converted to relative (using `cwd` as base) +- **Sensitive data stripping** (TODO — deferred to later task): + - Token usage / cache statistics + - API request IDs + - Permission mode details + +On reconstruction, relative paths are re-expanded using the local machine's working directory. + +## JSONL Line Type → Nostr Event Mapping + +| JSONL `type` | `role` tag | `content` | Notes | +|---|---|---|---| +| `user` (text) | `user` | User's message text | Simple text content | +| `user` (tool_result) | `tool_result` | Tool output text | Separated from user text | +| `assistant` (text) | `assistant` | Rendered markdown | Text blocks from content array | +| `assistant` (tool_use) | `tool_call` | Tool name + input summary | Each tool_use block = separate note | +| `progress` | `progress` | Hook progress description | Mapped for round-trip fidelity | +| `queue-operation` | `queue-operation` | Operation type | Mapped for round-trip fidelity | +| `file-history-snapshot` | `file-history-snapshot` | Snapshot summary | Mapped for round-trip fidelity | + +**Important**: Assistant messages with mixed content (text + tool_use blocks) are split into multiple nostr events — one per content block. Each gets its own note, threaded in sequence via `e` tags. + +## Conversation Threading + +Uses **NIP-10** reply threading: +- First note in a session: no `e` tags (it is the root) +- All subsequent notes: `["e", "<root-id>", "", "root"]` + `["e", "<prev-id>", "", "reply"]` +- The `e` tags always reference **nostr note IDs** (not JSONL UUIDs) +- UUID-to-note-ID mapping is maintained during conversion + +## Path Normalization + +When converting JSONL → nostr events: +1. Extract `cwd` from the JSONL line +2. All absolute paths that start with `cwd` are converted to relative paths +3. `cwd` itself is stored as a relative path (or stripped, with project root as implicit base) + +When reconstructing nostr events → JSONL: +1. Determine local working directory +2. Re-expand all relative paths to absolute using local `cwd` +3. Update `cwd`, `gitBranch`, and machine-specific fields + +## Data Flow (Phase 1 — Local Only) + +### Publishing (JSONL → nostr events) +1. On session activity, dave reads new JSONL lines +2. Each line is converted to a nostr event (normalize paths, extract presentation content) +3. Events are inserted via `ndb.process_event()` (local relay only) +4. UUID-to-note-ID mapping is cached for threading + +### Consuming (nostr events → UI) +1. Query ndb for events with the session's `d` tag +2. Order by `e` tag threading (NIP-10 reply chain) +3. Render `content` field directly in the conversation UI +4. `role` tag determines message styling (user bubble, assistant bubble, tool collapse, etc.) + +### Reconstruction (nostr events → JSONL, for future resume) +1. Query ndb for all events in a session (by `d` tag) +2. Order by reply chain +3. Extract `source-data` tag from each event +4. De-normalize paths (relative → absolute for local machine) +5. Write as JSONL file +6. Resume via `claude --resume <session-id>` + +## Non-Goals (Phase 1) + +- Publishing to external relays (privacy concerns) +- Resuming shared sessions from other users +- Sensitive data stripping (noted as TODO) +- NIP proposal (informal notedeck convention for now)