commit 0d51e25ab0cfaebb219a7dafd90c0408f500086c
parent 6601747eb49a5814f4ae9932ba46cd133e35353a
Author: William Casarin <jb55@jb55.com>
Date: Mon, 21 Apr 2025 12:48:33 -0700
dave: improve docs with ai
Signed-off-by: William Casarin <jb55@jb55.com>
Diffstat:
4 files changed, 708 insertions(+), 7 deletions(-)
diff --git a/crates/notedeck_dave/README.md b/crates/notedeck_dave/README.md
@@ -1,14 +1,77 @@
+# Dave - The Nostr AI Assistant
-# Dave, the nostr assistant
+Dave is an AI-powered assistant for the Nostr protocol, built as a Notedeck application. It provides a conversational interface that can search, analyze, and present Nostr notes to users.
-Dave is a notedeck app that implements a nostr ai assistant. It is also fairy
-simple and is a good reference on how to build a notedeck nostr app.
+## Overview
+
+Dave demonstrates how to build a feature-rich application on the Notedeck platform that interacts with Nostr content. It serves both as a useful tool for Nostr users and as a reference implementation for developers building Notedeck apps.
## Features
-- [x] Query and search the local nostr database for notes
+- [x] Interactive 3D avatar with WebGPU rendering
+- [x] Natural language conversations with AI
+- [x] Query and search the local Nostr database for notes
- [x] Present and render notes to the user
-- [ ] Chat history
-- [ ] Anonymous [lmzap][lmzap] backend
+- [x] Tool-based architecture for AI actions
+- [ ] Context-aware searching (home, profile, or global scope)
+- [ ] Chat history persistence
+- [ ] Anonymous [lmzap](https://jb55.com/lmzap) backend
+
+## Technical Details
+
+Dave uses:
+
+- Egui for UI rendering
+- WebGPU for 3D avatar visualization
+- OpenAI API (or Ollama with compatible models)
+- NostrDB for efficient note storage and querying
+- Async Rust for non-blocking API interactions
+
+## Architecture
+
+Dave is structured around several key components:
+
+1. **UI Layer** - Handles rendering and user interactions
+2. **Avatar** - 3D representation with WebGPU rendering
+3. **AI Client** - Connects to language models via OpenAI or Ollama
+4. **Tools System** - Provides structured ways for the AI to interact with Nostr data
+5. **Message Handler** - Manages conversation state and message processing
+
+## Usage as a Reference
+
+Dave serves as an excellent reference for developers looking to:
+
+- Build conversational interfaces in Notedeck
+- Implement 3D rendering with WebGPU in Rust applications
+- Create tool-based AI agents that can take actions in response to user requests
+- Query and present Nostr content in custom applications
+
+## Getting Started
+
+1. Clone the repository
+2. Set up your API keys for OpenAI or configure Ollama
+ ```
+ export OPENAI_API_KEY=your_api_key_here
+ # or for Ollama
+ export OLLAMA_HOST=http://localhost:11434
+ ```
+3. Build and run the Notedeck application with Dave
+
+## Configuration
+
+Dave can be configured to use different AI backends:
+
+- OpenAI API (default) - Set the `OPENAI_API_KEY` environment variable
+- Ollama - Use a compatible model like `hhao/qwen2.5-coder-tools` and set the `OLLAMA_HOST` environment variable
+
+## Contributing
+
+Contributions are welcome! See the issues list for planned features and improvements.
+
+## License
+
+GPL
+
+## Related Projects
-[lmzap]: https://jb55.com/lmzap
+- [nostrdb](https://github.com/damus-io/nostrdb) - Embedded database for Nostr notes
diff --git a/crates/notedeck_dave/docs/README.md b/crates/notedeck_dave/docs/README.md
@@ -0,0 +1,9 @@
+
+# Dave Developer Docs
+
+These are mostly generated by Claude+code2prompt because I was too lazy to make
+my own. I did review them at least and they seemed accurate at the time of
+this generation (2025-04-21)
+
+- [Developer Guide](./developer-guide.md)
+- [In-Depth Tools Guide](./tools.md)
diff --git a/crates/notedeck_dave/docs/developer-guide.md b/crates/notedeck_dave/docs/developer-guide.md
@@ -0,0 +1,263 @@
+# Dave Developer Guide
+
+This guide explains the architecture and implementation details of Dave, the Nostr AI assistant for Notedeck. It's intended to help developers understand how Dave works and how to use it as a reference for building their own Notedeck applications.
+
+## Architecture Overview
+
+Dave follows a modular architecture with several key components:
+
+```
+notedeck_dave
+├── UI Layer (ui/mod.rs, ui/dave.rs)
+├── Avatar System (avatar.rs, quaternion.rs, vec3.rs)
+├── Core Logic (lib.rs)
+├── AI Communication (messages.rs)
+├── Tools System (tools.rs)
+└── Configuration (config.rs)
+```
+
+### Component Breakdown
+
+#### 1. UI Layer (`ui/dave.rs`)
+
+The UI layer handles rendering the chat interface and processing user inputs. Key features:
+
+- Chat message rendering for different message types (user, assistant, tool calls)
+- Input box with keyboard shortcuts
+- Tool response visualization (note rendering, search results)
+
+The UI is built with egui and uses a responsive layout that adapts to different screen sizes.
+
+#### 2. 3D Avatar (`avatar.rs`)
+
+Dave includes a 3D avatar rendered with WebGPU:
+
+- Implements a 3D cube with proper lighting and rotation
+- Interactive dragging for manual rotation
+- Random "nudge" animations during AI responses
+- Custom WebGPU shader implementation
+
+#### 3. Core Logic (`lib.rs`)
+
+The `Dave` struct in `lib.rs` ties everything together:
+
+- Manages conversation state
+- Handles user interactions
+- Processes AI responses
+- Executes tool calls
+- Coordinates UI updates
+
+#### 4. AI Communication (`messages.rs`)
+
+Dave communicates with AI services (OpenAI or Ollama) through:
+
+- Message formatting for API requests
+- Streaming token processing
+- Tool call handling
+- Response parsing
+
+#### 5. Tools System (`tools.rs`)
+
+The tools system enables Dave to perform actions based on AI decisions:
+
+- `query` - Search for notes in the NostrDB
+- `present_notes` - Display specific notes to the user
+
+Each tool has a structured definition with:
+- Name and description
+- Parameter specifications
+- Parsing logic
+- Execution code
+
+## Key Workflows
+
+### 1. User Message Flow
+
+When a user sends a message:
+
+1. UI captures the input and triggers a `Send` action
+2. The message is added to the chat history
+3. A request is sent to the AI service with the conversation context
+4. The AI response is streamed back token by token
+5. The UI updates in real-time as tokens arrive
+
+### 2. Tool Call Flow
+
+When the AI decides to use a tool:
+
+1. The tool call is parsed and validated
+2. The tool is executed (e.g., querying NostrDB)
+3. Results are formatted and sent back to the AI
+4. The AI receives the results and continues the conversation
+5. The UI displays both the tool call and its results
+
+### 3. Note Presentation
+
+When presenting notes:
+
+1. The AI identifies relevant notes and calls `present_notes`
+2. Note IDs are parsed and validated
+3. The UI renders the notes in a scrollable horizontal view
+4. The AI references these notes in its response with `^1`, `^2`, etc.
+
+## Implementation Patterns
+
+### Streaming UI Updates
+
+Dave uses Rust's `mpsc` channels to handle streaming updates:
+
+```rust
+let (tx, rx) = mpsc::channel();
+self.incoming_tokens = Some(rx);
+
+// In a separate thread:
+tokio::spawn(async move {
+ // Process streaming responses
+ while let Some(token) = token_stream.next().await {
+ // Send tokens back to the UI thread
+ tx.send(DaveApiResponse::Token(content.to_owned()))?;
+ ctx.request_repaint();
+ }
+});
+```
+
+### Tool Definition
+
+Tools are defined with structured metadata:
+
+```rust
+Tool {
+ name: "query",
+ parse_call: QueryCall::parse,
+ description: "Note query functionality...",
+ arguments: vec![
+ ToolArg {
+ name: "search",
+ typ: ArgType::String,
+ required: false,
+ description: "A fulltext search query...",
+ // ...
+ },
+ // Additional arguments...
+ ]
+}
+```
+
+### WebGPU Integration
+
+The 3D avatar demonstrates WebGPU integration with egui:
+
+```rust
+ui.painter().add(egui_wgpu::Callback::new_paint_callback(
+ rect,
+ CubeCallback {
+ mvp_matrix,
+ model_matrix,
+ },
+));
+```
+
+## Using Dave as a Reference
+
+### Building a Notedeck App
+
+To build your own Notedeck app:
+
+1. Implement the `notedeck::App` trait
+2. Define your UI components and state management
+3. Handle app-specific actions and updates
+
+```rust
+impl notedeck::App for YourApp {
+ fn update(&mut self, ctx: &mut AppContext<'_>, ui: &mut egui::Ui) {
+ // Process events, update state
+ // Render UI components
+ // Handle user actions
+ }
+}
+```
+
+### Working with NostrDB
+
+Dave demonstrates how to query and present Nostr content:
+
+```rust
+// Creating a transaction
+let txn = Transaction::new(note_context.ndb).unwrap();
+
+// Querying notes
+let filter = nostrdb::Filter::new()
+ .limit(limit)
+ .search(search_term)
+ .kinds([1])
+ .build();
+
+let results = ndb.query(txn, &[filter], limit as i32);
+
+// Rendering notes
+for note_id in ¬e_ids {
+ let note = note_context.ndb.get_note_by_id(&txn, note_id.bytes());
+ // Render the note...
+}
+```
+
+### Implementing AI Tools
+
+To add new tools:
+
+1. Define a new call struct with parameters
+2. Implement parsing logic
+3. Add execution code
+4. Register the tool in `dave_tools()`
+
+```rust
+#[derive(Debug, Deserialize, Serialize, Clone)]
+pub struct YourToolCall {
+ // Parameters
+}
+
+impl YourToolCall {
+ fn parse(args: &str) -> Result<ToolCalls, ToolCallError> {
+ // Parse JSON arguments
+ }
+
+ // Execution logic
+}
+
+// Add to tools list
+pub fn dave_tools() -> Vec<Tool> {
+ vec![query_tool(), present_tool(), your_tool()]
+}
+```
+
+## Best Practices
+
+1. **Responsive Design**: Use the `is_narrow()` function to adapt layouts for different screen sizes
+2. **Streaming Updates**: Process large responses incrementally to keep the UI responsive
+3. **Error Handling**: Gracefully handle API errors and unexpected inputs
+4. **Tool Design**: Create tools with clear, focused functionality and descriptive metadata
+5. **State Management**: Keep UI state separate from application logic
+
+## Advanced Features
+
+### Custom Rendering
+
+Dave demonstrates custom rendering with WebGPU for the 3D avatar:
+
+1. Define shaders using WGSL
+2. Set up rendering pipelines and resources
+3. Implement the `CallbackTrait` for custom drawing
+4. Add paint callbacks to the UI
+
+### AI Context Management
+
+Dave maintains conversation context for the AI:
+
+1. Structured message history (`Vec<Message>`)
+2. Tool call results included in context
+3. System prompt with instructions and constraints
+4. Proper message formatting for API requests
+
+## Conclusion
+
+Dave is a sophisticated example of a Notedeck application that integrates AI, 3D rendering, and Nostr data. By studying its implementation, developers can learn patterns and techniques for building their own applications on the Notedeck platform.
diff --git a/crates/notedeck_dave/docs/tools.md b/crates/notedeck_dave/docs/tools.md
@@ -0,0 +1,366 @@
+# Dave's Tool System: In-Depth Guide
+
+One of the most powerful aspects of Dave is its tools system, which allows the AI assistant to perform actions within the Notedeck environment. This guide explores the design and implementation of Dave's tools system, explaining how it enables the AI to query data and present content to users.
+
+## Tools System Overview
+
+The tools system enables Dave to:
+
+1. Search the NostrDB for relevant notes
+2. Present notes to users through the UI
+3. Handle context-specific queries (home, profile, etc.)
+4. Process streaming tool calls from the AI
+
+## Core Components
+
+### 1. Tool Definitions (`tools.rs`)
+
+Each tool is defined with metadata that describes:
+- Name and description
+- Required and optional parameters
+- Parameter types and constraints
+- Parsing and execution logic
+
+```rust
+Tool {
+ name: "query",
+ parse_call: QueryCall::parse,
+ description: "Note query functionality...",
+ arguments: vec![
+ ToolArg {
+ name: "search",
+ typ: ArgType::String,
+ required: false,
+ default: None,
+ description: "A fulltext search query...",
+ },
+ // More arguments...
+ ]
+}
+```
+
+### 2. Tool Calls
+
+When the AI decides to use a tool, it generates a tool call:
+
+```rust
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ToolCall {
+ id: String,
+ typ: ToolCalls,
+}
+```
+
+The `ToolCalls` enum represents different types of tool calls:
+
+```rust
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub enum ToolCalls {
+ Query(QueryCall),
+ PresentNotes(PresentNotesCall),
+}
+```
+
+### 3. Tool Responses
+
+After executing a tool, Dave sends a response back to the AI:
+
+```rust
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ToolResponse {
+ id: String,
+ typ: ToolResponses,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub enum ToolResponses {
+ Query(QueryResponse),
+ PresentNotes,
+}
+```
+
+### 4. Streaming Processing
+
+Since tool calls arrive in a streaming fashion from the AI API, Dave uses a `PartialToolCall` structure to collect fragments:
+
+```rust
+#[derive(Default, Debug, Clone)]
+pub struct PartialToolCall {
+ id: Option<String>,
+ name: Option<String>,
+ arguments: Option<String>,
+}
+```
+
+## Available Tools
+
+### 1. Query Tool
+
+The query tool searches the NostrDB for notes matching specific criteria:
+
+```rust
+#[derive(Debug, Deserialize, Serialize, Clone)]
+pub struct QueryCall {
+ context: Option<QueryContext>,
+ limit: Option<u64>,
+ since: Option<u64>,
+ kind: Option<u64>,
+ until: Option<u64>,
+ search: Option<String>,
+}
+```
+
+Parameters:
+- `context`: Where to search (Home, Profile, Any)
+- `limit`: Maximum number of results
+- `since`/`until`: Time range constraints (unix timestamps)
+- `kind`: Note type (1 for posts, 0 for profiles, etc.)
+- `search`: Fulltext search query
+
+Example usage by the AI:
+```json
+{
+ "search": "bitcoin",
+ "limit": 10,
+ "context": "home",
+ "kind": 1
+}
+```
+
+### 2. Present Notes Tool
+
+The present notes tool displays specific notes to the user:
+
+```rust
+#[derive(Debug, Deserialize, Serialize, Clone)]
+pub struct PresentNotesCall {
+ pub note_ids: Vec<NoteId>,
+}
+```
+
+Parameters:
+- `note_ids`: List of note IDs to display
+
+Example usage by the AI:
+```json
+{
+ "note_ids": "fe1278a57ce6a499cca6a54971f7255e5a953c91243f891be54c50155a7b9a9c,a8943f1c99af5acd5ebb24e7dae860ab8c879bdf2ed4bd14bbc28a3a4b0c2f50"
+}
+```
+
+## Tool Execution Flow
+
+1. **Tool Call Parsing**:
+ - AI sends a tool call with ID, name, and arguments
+ - Dave parses the JSON arguments into typed structures
+ - Validation ensures required parameters are present
+
+2. **Tool Execution**:
+ - For query tool: Constructs a NostrDB filter and executes the query
+ - For present notes: Validates note IDs and prepares them for display
+
+3. **Response Formatting**:
+ - Query results are formatted as JSON for the AI
+ - Notes are prepared for UI rendering
+
+4. **Response Processing**:
+ - AI receives the tool response and incorporates it into the conversation
+ - UI displays relevant components (search results, note previews)
+
+## Technical Implementation
+
+### Note Formatting for AI
+
+When returning query results to the AI, Dave formats notes in a simplified JSON structure:
+
+```rust
+#[derive(Debug, Serialize)]
+struct SimpleNote {
+ note_id: String,
+ pubkey: String,
+ name: String,
+ content: String,
+ created_at: String,
+ note_kind: u64,
+}
+```
+
+## Using the Tools in Practice
+
+### System Prompt Guidance
+
+Dave's system prompt instructs the AI on how to use the tools effectively:
+
+```
+- You *MUST* call the present_notes tool with a list of comma-separated note id references when referring to notes so that the UI can display them. Do *NOT* include note id references in the text response, but you *SHOULD* use ^1, ^2, etc to reference note indices passed to present_notes.
+- When a user asks for a digest instead of specific query terms, make sure to include both since and until to pull notes for the correct range.
+- When tasked with open-ended queries such as looking for interesting notes or summarizing the day, make sure to add enough notes to the context (limit: 100-200) so that it returns enough data for summarization.
+```
+
+### UI Integration
+
+The UI renders tool calls and responses:
+
+```rust
+fn tool_calls_ui(ctx: &mut AppContext, toolcalls: &[ToolCall], ui: &mut egui::Ui) {
+ ui.vertical(|ui| {
+ for call in toolcalls {
+ match call.calls() {
+ ToolCalls::PresentNotes(call) => Self::present_notes_ui(ctx, call, ui),
+ ToolCalls::Query(search_call) => {
+ ui.horizontal(|ui| {
+ egui::Frame::new()
+ .inner_margin(10.0)
+ .corner_radius(10.0)
+ .fill(ui.visuals().widgets.inactive.weak_bg_fill)
+ .show(ui, |ui| {
+ Self::search_call_ui(search_call, ui);
+ })
+ });
+ }
+ }
+ }
+ });
+}
+```
+
+## Extending the Tools System
+
+### Adding a New Tool
+
+To add a new tool:
+
+1. Define the tool call structure:
+```rust
+#[derive(Debug, Deserialize, Serialize, Clone)]
+pub struct NewToolCall {
+ // Parameters...
+}
+```
+
+2. Add a new variant to `ToolCalls` enum:
+```rust
+pub enum ToolCalls {
+ Query(QueryCall),
+ PresentNotes(PresentNotesCall),
+ NewTool(NewToolCall),
+}
+```
+
+3. Implement parsing logic:
+```rust
+impl NewToolCall {
+ fn parse(args: &str) -> Result<ToolCalls, ToolCallError> {
+ // Parse JSON arguments...
+ Ok(ToolCalls::NewTool(parsed))
+ }
+}
+```
+
+4. Create tool definition and add to `dave_tools()`:
+```rust
+fn new_tool() -> Tool {
+ Tool {
+ name: "new_tool",
+ parse_call: NewToolCall::parse,
+ description: "Description...",
+ arguments: vec![
+ // Arguments...
+ ]
+ }
+}
+
+pub fn dave_tools() -> Vec<Tool> {
+ vec![query_tool(), present_tool(), new_tool()]
+}
+```
+
+### Handling Tool Responses
+
+Add a new variant to `ToolResponses`:
+```rust
+pub enum ToolResponses {
+ Query(QueryResponse),
+ PresentNotes,
+ NewTool(NewToolResponse),
+}
+```
+
+Implement response formatting:
+```rust
+fn format_tool_response_for_ai(txn: &Transaction, ndb: &Ndb, resp: &ToolResponses) -> String {
+ match resp {
+ // Existing cases...
+ ToolResponses::NewTool(response) => {
+ // Format response as JSON...
+ }
+ }
+}
+```
+
+## Advanced Usage Patterns
+
+### Context-Aware Queries
+
+The `QueryContext` enum allows the AI to scope searches:
+
+```rust
+#[derive(Debug, Deserialize, Serialize, Clone)]
+#[serde(rename_all = "lowercase")]
+pub enum QueryContext {
+ Home,
+ Profile,
+ Any,
+}
+```
+
+### Time-Based Queries
+
+Dave is configured with current and recent timestamps in the system prompt, enabling time-aware queries:
+
+```
+- The current date is {date} ({timestamp} unix timestamp if needed for queries).
+- Yesterday (-24hrs) was {yesterday_timestamp}. You can use this in combination with `since` queries for pulling notes for summarizing notes the user might have missed while they were away.
+```
+
+### Filtering Non-Relevant Content
+
+Dave filters out reply notes when performing queries to improve results:
+
+```rust
+fn is_reply(note: Note) -> bool {
+ for tag in note.tags() {
+ if tag.count() < 4 {
+ continue;
+ }
+
+ let Some("e") = tag.get_str(0) else {
+ continue;
+ };
+
+ let Some(s) = tag.get_str(3) else {
+ continue;
+ };
+
+ if s == "root" || s == "reply" {
+ return true;
+ }
+ }
+
+ false
+}
+
+// Used in filter creation
+.custom(|n| !is_reply(n))
+```
+
+## Conclusion
+
+The tools system is what makes Dave truly powerful, enabling it to interact with NostrDB and present content to users. By understanding this system, developers can:
+
+1. Extend Dave with new capabilities
+2. Apply similar patterns in other AI-powered applications
+3. Create tools that balance flexibility and structure
+4. Build effective interfaces between AI models and application data
+
+This architecture demonstrates a robust approach to enabling AI assistants to take meaningful actions within applications, going beyond simple text generation to deliver real utility to users.
