notedeck

One damus client to rule them all
git clone git://jb55.com/notedeck
Log | Files | Refs | README | LICENSE

developer-guide.md (6959B)


      1 # Dave Developer Guide
      2 
      3 This guide explains the architecture and implementation details of Dave, the Nostr AI assistant for Notedeck. It's intended to help developers understand how Dave works and how to use it as a reference for building their own Notedeck applications.
      4 
      5 ## Architecture Overview
      6 
      7 Dave follows a modular architecture with several key components:
      8 
      9 ```
     10 notedeck_dave
     11 ├── UI Layer (ui/mod.rs, ui/dave.rs)
     12 ├── Avatar System (avatar.rs, quaternion.rs, vec3.rs)
     13 ├── Core Logic (lib.rs)
     14 ├── AI Communication (messages.rs)
     15 ├── Tools System (tools.rs)
     16 └── Configuration (config.rs)
     17 ```
     18 
     19 ### Component Breakdown
     20 
     21 #### 1. UI Layer (`ui/dave.rs`)
     22 
     23 The UI layer handles rendering the chat interface and processing user inputs. Key features:
     24 
     25 - Chat message rendering for different message types (user, assistant, tool calls)
     26 - Input box with keyboard shortcuts
     27 - Tool response visualization (note rendering, search results)
     28 
     29 The UI is built with egui and uses a responsive layout that adapts to different screen sizes.
     30 
     31 #### 2. 3D Avatar (`avatar.rs`)
     32 
     33 Dave includes a 3D avatar rendered with WebGPU:
     34 
     35 - Implements a 3D cube with proper lighting and rotation
     36 - Interactive dragging for manual rotation
     37 - Random "nudge" animations during AI responses
     38 - Custom WebGPU shader implementation
     39 
     40 #### 3. Core Logic (`lib.rs`)
     41 
     42 The `Dave` struct in `lib.rs` ties everything together:
     43 
     44 - Manages conversation state
     45 - Handles user interactions
     46 - Processes AI responses
     47 - Executes tool calls
     48 - Coordinates UI updates
     49 
     50 #### 4. AI Communication (`messages.rs`)
     51 
     52 Dave communicates with AI services (OpenAI or Ollama) through:
     53 
     54 - Message formatting for API requests
     55 - Streaming token processing
     56 - Tool call handling
     57 - Response parsing
     58 
     59 #### 5. Tools System (`tools.rs`)
     60 
     61 The tools system enables Dave to perform actions based on AI decisions:
     62 
     63 - `query` - Search for notes in the NostrDB
     64 - `present_notes` - Display specific notes to the user
     65 
     66 Each tool has a structured definition with:
     67 - Name and description
     68 - Parameter specifications
     69 - Parsing logic
     70 - Execution code
     71 
     72 ## Key Workflows
     73 
     74 ### 1. User Message Flow
     75 
     76 When a user sends a message:
     77 
     78 1. UI captures the input and triggers a `Send` action
     79 2. The message is added to the chat history
     80 3. A request is sent to the AI service with the conversation context
     81 4. The AI response is streamed back token by token
     82 5. The UI updates in real-time as tokens arrive
     83 
     84 ### 2. Tool Call Flow
     85 
     86 When the AI decides to use a tool:
     87 
     88 1. The tool call is parsed and validated
     89 2. The tool is executed (e.g., querying NostrDB)
     90 3. Results are formatted and sent back to the AI
     91 4. The AI receives the results and continues the conversation
     92 5. The UI displays both the tool call and its results
     93 
     94 ### 3. Note Presentation
     95 
     96 When presenting notes:
     97 
     98 1. The AI identifies relevant notes and calls `present_notes`
     99 2. Note IDs are parsed and validated
    100 3. The UI renders the notes in a scrollable horizontal view
    101 4. The AI references these notes in its response with `^1`, `^2`, etc.
    102 
    103 ## Implementation Patterns
    104 
    105 ### Streaming UI Updates
    106 
    107 Dave uses Rust's `mpsc` channels to handle streaming updates:
    108 
    109 ```rust
    110 let (tx, rx) = mpsc::channel();
    111 self.incoming_tokens = Some(rx);
    112 
    113 // In a separate thread:
    114 tokio::spawn(async move {
    115     // Process streaming responses
    116     while let Some(token) = token_stream.next().await {
    117         // Send tokens back to the UI thread
    118         tx.send(DaveApiResponse::Token(content.to_owned()))?;
    119         ctx.request_repaint();
    120     }
    121 });
    122 ```
    123 
    124 ### Tool Definition
    125 
    126 Tools are defined with structured metadata:
    127 
    128 ```rust
    129 Tool {
    130     name: "query",
    131     parse_call: QueryCall::parse,
    132     description: "Note query functionality...",
    133     arguments: vec![
    134         ToolArg {
    135             name: "search",
    136             typ: ArgType::String,
    137             required: false,
    138             description: "A fulltext search query...",
    139             // ...
    140         },
    141         // Additional arguments...
    142     ]
    143 }
    144 ```
    145 
    146 ### WebGPU Integration
    147 
    148 The 3D avatar demonstrates WebGPU integration with egui:
    149 
    150 ```rust
    151 ui.painter().add(egui_wgpu::Callback::new_paint_callback(
    152     rect,
    153     CubeCallback {
    154         mvp_matrix,
    155         model_matrix,
    156     },
    157 ));
    158 ```
    159 
    160 ## Using Dave as a Reference
    161 
    162 ### Building a Notedeck App
    163 
    164 To build your own Notedeck app:
    165 
    166 1. Implement the `notedeck::App` trait
    167 2. Define your UI components and state management
    168 3. Handle app-specific actions and updates
    169 
    170 ```rust
    171 impl notedeck::App for YourApp {
    172     fn update(&mut self, ctx: &mut AppContext<'_>, ui: &mut egui::Ui) {
    173         // Process events, update state
    174         // Render UI components
    175         // Handle user actions
    176     }
    177 }
    178 ```
    179 
    180 ### Working with NostrDB
    181 
    182 Dave demonstrates how to query and present Nostr content:
    183 
    184 ```rust
    185 // Creating a transaction
    186 let txn = Transaction::new(note_context.ndb).unwrap();
    187 
    188 // Querying notes
    189 let filter = nostrdb::Filter::new()
    190     .limit(limit)
    191     .search(search_term)
    192     .kinds([1])
    193     .build();
    194     
    195 let results = ndb.query(txn, &[filter], limit as i32);
    196 
    197 // Rendering notes
    198 for note_id in &note_ids {
    199     let note = note_context.ndb.get_note_by_id(&txn, note_id.bytes());
    200     // Render the note...
    201 }
    202 ```
    203 
    204 ### Implementing AI Tools
    205 
    206 To add new tools:
    207 
    208 1. Define a new call struct with parameters
    209 2. Implement parsing logic
    210 3. Add execution code
    211 4. Register the tool in `dave_tools()`
    212 
    213 ```rust
    214 #[derive(Debug, Deserialize, Serialize, Clone)]
    215 pub struct YourToolCall {
    216     // Parameters
    217 }
    218 
    219 impl YourToolCall {
    220     fn parse(args: &str) -> Result<ToolCalls, ToolCallError> {
    221         // Parse JSON arguments
    222     }
    223     
    224     // Execution logic
    225 }
    226 
    227 // Add to tools list
    228 pub fn dave_tools() -> Vec<Tool> {
    229     vec![query_tool(), present_tool(), your_tool()]
    230 }
    231 ```
    232 
    233 ## Best Practices
    234 
    235 1. **Responsive Design**: Use the `is_narrow()` function to adapt layouts for different screen sizes
    236 2. **Streaming Updates**: Process large responses incrementally to keep the UI responsive
    237 3. **Error Handling**: Gracefully handle API errors and unexpected inputs
    238 4. **Tool Design**: Create tools with clear, focused functionality and descriptive metadata
    239 5. **State Management**: Keep UI state separate from application logic
    240 
    241 ## Advanced Features
    242 
    243 ### Custom Rendering
    244 
    245 Dave demonstrates custom rendering with WebGPU for the 3D avatar:
    246 
    247 1. Define shaders using WGSL
    248 2. Set up rendering pipelines and resources
    249 3. Implement the `CallbackTrait` for custom drawing
    250 4. Add paint callbacks to the UI
    251 
    252 ### AI Context Management
    253 
    254 Dave maintains conversation context for the AI:
    255 
    256 1. Structured message history (`Vec<Message>`)
    257 2. Tool call results included in context
    258 3. System prompt with instructions and constraints
    259 4. Proper message formatting for API requests
    260 
    261 ## Conclusion
    262 
    263 Dave is a sophisticated example of a Notedeck application that integrates AI, 3D rendering, and Nostr data. By studying its implementation, developers can learn patterns and techniques for building their own applications on the Notedeck platform.