developer-guide.md (6959B)
1 # Dave Developer Guide 2 3 This guide explains the architecture and implementation details of Dave, the Nostr AI assistant for Notedeck. It's intended to help developers understand how Dave works and how to use it as a reference for building their own Notedeck applications. 4 5 ## Architecture Overview 6 7 Dave follows a modular architecture with several key components: 8 9 ``` 10 notedeck_dave 11 ├── UI Layer (ui/mod.rs, ui/dave.rs) 12 ├── Avatar System (avatar.rs, quaternion.rs, vec3.rs) 13 ├── Core Logic (lib.rs) 14 ├── AI Communication (messages.rs) 15 ├── Tools System (tools.rs) 16 └── Configuration (config.rs) 17 ``` 18 19 ### Component Breakdown 20 21 #### 1. UI Layer (`ui/dave.rs`) 22 23 The UI layer handles rendering the chat interface and processing user inputs. Key features: 24 25 - Chat message rendering for different message types (user, assistant, tool calls) 26 - Input box with keyboard shortcuts 27 - Tool response visualization (note rendering, search results) 28 29 The UI is built with egui and uses a responsive layout that adapts to different screen sizes. 30 31 #### 2. 3D Avatar (`avatar.rs`) 32 33 Dave includes a 3D avatar rendered with WebGPU: 34 35 - Implements a 3D cube with proper lighting and rotation 36 - Interactive dragging for manual rotation 37 - Random "nudge" animations during AI responses 38 - Custom WebGPU shader implementation 39 40 #### 3. Core Logic (`lib.rs`) 41 42 The `Dave` struct in `lib.rs` ties everything together: 43 44 - Manages conversation state 45 - Handles user interactions 46 - Processes AI responses 47 - Executes tool calls 48 - Coordinates UI updates 49 50 #### 4. AI Communication (`messages.rs`) 51 52 Dave communicates with AI services (OpenAI or Ollama) through: 53 54 - Message formatting for API requests 55 - Streaming token processing 56 - Tool call handling 57 - Response parsing 58 59 #### 5. Tools System (`tools.rs`) 60 61 The tools system enables Dave to perform actions based on AI decisions: 62 63 - `query` - Search for notes in the NostrDB 64 - `present_notes` - Display specific notes to the user 65 66 Each tool has a structured definition with: 67 - Name and description 68 - Parameter specifications 69 - Parsing logic 70 - Execution code 71 72 ## Key Workflows 73 74 ### 1. User Message Flow 75 76 When a user sends a message: 77 78 1. UI captures the input and triggers a `Send` action 79 2. The message is added to the chat history 80 3. A request is sent to the AI service with the conversation context 81 4. The AI response is streamed back token by token 82 5. The UI updates in real-time as tokens arrive 83 84 ### 2. Tool Call Flow 85 86 When the AI decides to use a tool: 87 88 1. The tool call is parsed and validated 89 2. The tool is executed (e.g., querying NostrDB) 90 3. Results are formatted and sent back to the AI 91 4. The AI receives the results and continues the conversation 92 5. The UI displays both the tool call and its results 93 94 ### 3. Note Presentation 95 96 When presenting notes: 97 98 1. The AI identifies relevant notes and calls `present_notes` 99 2. Note IDs are parsed and validated 100 3. The UI renders the notes in a scrollable horizontal view 101 4. The AI references these notes in its response with `^1`, `^2`, etc. 102 103 ## Implementation Patterns 104 105 ### Streaming UI Updates 106 107 Dave uses Rust's `mpsc` channels to handle streaming updates: 108 109 ```rust 110 let (tx, rx) = mpsc::channel(); 111 self.incoming_tokens = Some(rx); 112 113 // In a separate thread: 114 tokio::spawn(async move { 115 // Process streaming responses 116 while let Some(token) = token_stream.next().await { 117 // Send tokens back to the UI thread 118 tx.send(DaveApiResponse::Token(content.to_owned()))?; 119 ctx.request_repaint(); 120 } 121 }); 122 ``` 123 124 ### Tool Definition 125 126 Tools are defined with structured metadata: 127 128 ```rust 129 Tool { 130 name: "query", 131 parse_call: QueryCall::parse, 132 description: "Note query functionality...", 133 arguments: vec![ 134 ToolArg { 135 name: "search", 136 typ: ArgType::String, 137 required: false, 138 description: "A fulltext search query...", 139 // ... 140 }, 141 // Additional arguments... 142 ] 143 } 144 ``` 145 146 ### WebGPU Integration 147 148 The 3D avatar demonstrates WebGPU integration with egui: 149 150 ```rust 151 ui.painter().add(egui_wgpu::Callback::new_paint_callback( 152 rect, 153 CubeCallback { 154 mvp_matrix, 155 model_matrix, 156 }, 157 )); 158 ``` 159 160 ## Using Dave as a Reference 161 162 ### Building a Notedeck App 163 164 To build your own Notedeck app: 165 166 1. Implement the `notedeck::App` trait 167 2. Define your UI components and state management 168 3. Handle app-specific actions and updates 169 170 ```rust 171 impl notedeck::App for YourApp { 172 fn update(&mut self, ctx: &mut AppContext<'_>, ui: &mut egui::Ui) { 173 // Process events, update state 174 // Render UI components 175 // Handle user actions 176 } 177 } 178 ``` 179 180 ### Working with NostrDB 181 182 Dave demonstrates how to query and present Nostr content: 183 184 ```rust 185 // Creating a transaction 186 let txn = Transaction::new(note_context.ndb).unwrap(); 187 188 // Querying notes 189 let filter = nostrdb::Filter::new() 190 .limit(limit) 191 .search(search_term) 192 .kinds([1]) 193 .build(); 194 195 let results = ndb.query(txn, &[filter], limit as i32); 196 197 // Rendering notes 198 for note_id in ¬e_ids { 199 let note = note_context.ndb.get_note_by_id(&txn, note_id.bytes()); 200 // Render the note... 201 } 202 ``` 203 204 ### Implementing AI Tools 205 206 To add new tools: 207 208 1. Define a new call struct with parameters 209 2. Implement parsing logic 210 3. Add execution code 211 4. Register the tool in `dave_tools()` 212 213 ```rust 214 #[derive(Debug, Deserialize, Serialize, Clone)] 215 pub struct YourToolCall { 216 // Parameters 217 } 218 219 impl YourToolCall { 220 fn parse(args: &str) -> Result<ToolCalls, ToolCallError> { 221 // Parse JSON arguments 222 } 223 224 // Execution logic 225 } 226 227 // Add to tools list 228 pub fn dave_tools() -> Vec<Tool> { 229 vec![query_tool(), present_tool(), your_tool()] 230 } 231 ``` 232 233 ## Best Practices 234 235 1. **Responsive Design**: Use the `is_narrow()` function to adapt layouts for different screen sizes 236 2. **Streaming Updates**: Process large responses incrementally to keep the UI responsive 237 3. **Error Handling**: Gracefully handle API errors and unexpected inputs 238 4. **Tool Design**: Create tools with clear, focused functionality and descriptive metadata 239 5. **State Management**: Keep UI state separate from application logic 240 241 ## Advanced Features 242 243 ### Custom Rendering 244 245 Dave demonstrates custom rendering with WebGPU for the 3D avatar: 246 247 1. Define shaders using WGSL 248 2. Set up rendering pipelines and resources 249 3. Implement the `CallbackTrait` for custom drawing 250 4. Add paint callbacks to the UI 251 252 ### AI Context Management 253 254 Dave maintains conversation context for the AI: 255 256 1. Structured message history (`Vec<Message>`) 257 2. Tool call results included in context 258 3. System prompt with instructions and constraints 259 4. Proper message formatting for API requests 260 261 ## Conclusion 262 263 Dave is a sophisticated example of a Notedeck application that integrates AI, 3D rendering, and Nostr data. By studying its implementation, developers can learn patterns and techniques for building their own applications on the Notedeck platform.