notedeck

One damus client to rule them all
git clone git://jb55.com/notedeck
Log | Files | Refs | README | LICENSE

tools.md (9356B)


      1 # Dave's Tool System: In-Depth Guide
      2 
      3 One of the most powerful aspects of Dave is its tools system, which allows the AI assistant to perform actions within the Notedeck environment. This guide explores the design and implementation of Dave's tools system, explaining how it enables the AI to query data and present content to users.
      4 
      5 ## Tools System Overview
      6 
      7 The tools system enables Dave to:
      8 
      9 1. Search the NostrDB for relevant notes
     10 2. Present notes to users through the UI
     11 3. Handle context-specific queries (home, profile, etc.)
     12 4. Process streaming tool calls from the AI
     13 
     14 ## Core Components
     15 
     16 ### 1. Tool Definitions (`tools.rs`)
     17 
     18 Each tool is defined with metadata that describes:
     19 - Name and description
     20 - Required and optional parameters
     21 - Parameter types and constraints
     22 - Parsing and execution logic
     23 
     24 ```rust
     25 Tool {
     26     name: "query",
     27     parse_call: QueryCall::parse,
     28     description: "Note query functionality...",
     29     arguments: vec![
     30         ToolArg {
     31             name: "search",
     32             typ: ArgType::String,
     33             required: false,
     34             default: None,
     35             description: "A fulltext search query...",
     36         },
     37         // More arguments...
     38     ]
     39 }
     40 ```
     41 
     42 ### 2. Tool Calls
     43 
     44 When the AI decides to use a tool, it generates a tool call:
     45 
     46 ```rust
     47 #[derive(Debug, Clone, Serialize, Deserialize)]
     48 pub struct ToolCall {
     49     id: String,
     50     typ: ToolCalls,
     51 }
     52 ```
     53 
     54 The `ToolCalls` enum represents different types of tool calls:
     55 
     56 ```rust
     57 #[derive(Debug, Clone, Serialize, Deserialize)]
     58 pub enum ToolCalls {
     59     Query(QueryCall),
     60     PresentNotes(PresentNotesCall),
     61 }
     62 ```
     63 
     64 ### 3. Tool Responses
     65 
     66 After executing a tool, Dave sends a response back to the AI:
     67 
     68 ```rust
     69 #[derive(Debug, Clone, Serialize, Deserialize)]
     70 pub struct ToolResponse {
     71     id: String,
     72     typ: ToolResponses,
     73 }
     74 
     75 #[derive(Debug, Clone, Serialize, Deserialize)]
     76 pub enum ToolResponses {
     77     Query(QueryResponse),
     78     PresentNotes,
     79 }
     80 ```
     81 
     82 ### 4. Streaming Processing
     83 
     84 Since tool calls arrive in a streaming fashion from the AI API, Dave uses a `PartialToolCall` structure to collect fragments:
     85 
     86 ```rust
     87 #[derive(Default, Debug, Clone)]
     88 pub struct PartialToolCall {
     89     id: Option<String>,
     90     name: Option<String>,
     91     arguments: Option<String>,
     92 }
     93 ```
     94 
     95 ## Available Tools
     96 
     97 ### 1. Query Tool
     98 
     99 The query tool searches the NostrDB for notes matching specific criteria:
    100 
    101 ```rust
    102 #[derive(Debug, Deserialize, Serialize, Clone)]
    103 pub struct QueryCall {
    104     context: Option<QueryContext>,
    105     limit: Option<u64>,
    106     since: Option<u64>,
    107     kind: Option<u64>,
    108     until: Option<u64>,
    109     search: Option<String>,
    110 }
    111 ```
    112 
    113 Parameters:
    114 - `context`: Where to search (Home, Profile, Any)
    115 - `limit`: Maximum number of results
    116 - `since`/`until`: Time range constraints (unix timestamps)
    117 - `kind`: Note type (1 for posts, 0 for profiles, etc.)
    118 - `search`: Fulltext search query
    119 
    120 Example usage by the AI:
    121 ```json
    122 {
    123   "search": "bitcoin",
    124   "limit": 10,
    125   "context": "home",
    126   "kind": 1
    127 }
    128 ```
    129 
    130 ### 2. Present Notes Tool
    131 
    132 The present notes tool displays specific notes to the user:
    133 
    134 ```rust
    135 #[derive(Debug, Deserialize, Serialize, Clone)]
    136 pub struct PresentNotesCall {
    137     pub note_ids: Vec<NoteId>,
    138 }
    139 ```
    140 
    141 Parameters:
    142 - `note_ids`: List of note IDs to display
    143 
    144 Example usage by the AI:
    145 ```json
    146 {
    147   "note_ids": "fe1278a57ce6a499cca6a54971f7255e5a953c91243f891be54c50155a7b9a9c,a8943f1c99af5acd5ebb24e7dae860ab8c879bdf2ed4bd14bbc28a3a4b0c2f50"
    148 }
    149 ```
    150 
    151 ## Tool Execution Flow
    152 
    153 1. **Tool Call Parsing**:
    154    - AI sends a tool call with ID, name, and arguments
    155    - Dave parses the JSON arguments into typed structures
    156    - Validation ensures required parameters are present
    157 
    158 2. **Tool Execution**:
    159    - For query tool: Constructs a NostrDB filter and executes the query
    160    - For present notes: Validates note IDs and prepares them for display
    161 
    162 3. **Response Formatting**:
    163    - Query results are formatted as JSON for the AI
    164    - Notes are prepared for UI rendering
    165 
    166 4. **Response Processing**:
    167    - AI receives the tool response and incorporates it into the conversation
    168    - UI displays relevant components (search results, note previews)
    169 
    170 ## Technical Implementation
    171 
    172 ### Note Formatting for AI
    173 
    174 When returning query results to the AI, Dave formats notes in a simplified JSON structure:
    175 
    176 ```rust
    177 #[derive(Debug, Serialize)]
    178 struct SimpleNote {
    179     note_id: String,
    180     pubkey: String,
    181     name: String,
    182     content: String,
    183     created_at: String,
    184     note_kind: u64,
    185 }
    186 ```
    187 
    188 ## Using the Tools in Practice
    189 
    190 ### System Prompt Guidance
    191 
    192 Dave's system prompt instructs the AI on how to use the tools effectively:
    193 
    194 ```
    195 - You *MUST* call the present_notes tool with a list of comma-separated note id references when referring to notes so that the UI can display them. Do *NOT* include note id references in the text response, but you *SHOULD* use ^1, ^2, etc to reference note indices passed to present_notes.
    196 - When a user asks for a digest instead of specific query terms, make sure to include both since and until to pull notes for the correct range.
    197 - When tasked with open-ended queries such as looking for interesting notes or summarizing the day, make sure to add enough notes to the context (limit: 100-200) so that it returns enough data for summarization.
    198 ```
    199 
    200 ### UI Integration
    201 
    202 The UI renders tool calls and responses:
    203 
    204 ```rust
    205 fn tool_calls_ui(ctx: &mut AppContext, toolcalls: &[ToolCall], ui: &mut egui::Ui) {
    206     ui.vertical(|ui| {
    207         for call in toolcalls {
    208             match call.calls() {
    209                 ToolCalls::PresentNotes(call) => Self::present_notes_ui(ctx, call, ui),
    210                 ToolCalls::Query(search_call) => {
    211                     ui.horizontal(|ui| {
    212                         egui::Frame::new()
    213                             .inner_margin(10.0)
    214                             .corner_radius(10.0)
    215                             .fill(ui.visuals().widgets.inactive.weak_bg_fill)
    216                             .show(ui, |ui| {
    217                                 Self::search_call_ui(search_call, ui);
    218                             })
    219                     });
    220                 }
    221             }
    222         }
    223     });
    224 }
    225 ```
    226 
    227 ## Extending the Tools System
    228 
    229 ### Adding a New Tool
    230 
    231 To add a new tool:
    232 
    233 1. Define the tool call structure:
    234 ```rust
    235 #[derive(Debug, Deserialize, Serialize, Clone)]
    236 pub struct NewToolCall {
    237     // Parameters...
    238 }
    239 ```
    240 
    241 2. Add a new variant to `ToolCalls` enum:
    242 ```rust
    243 pub enum ToolCalls {
    244     Query(QueryCall),
    245     PresentNotes(PresentNotesCall),
    246     NewTool(NewToolCall),
    247 }
    248 ```
    249 
    250 3. Implement parsing logic:
    251 ```rust
    252 impl NewToolCall {
    253     fn parse(args: &str) -> Result<ToolCalls, ToolCallError> {
    254         // Parse JSON arguments...
    255         Ok(ToolCalls::NewTool(parsed))
    256     }
    257 }
    258 ```
    259 
    260 4. Create tool definition and add to `dave_tools()`:
    261 ```rust
    262 fn new_tool() -> Tool {
    263     Tool {
    264         name: "new_tool",
    265         parse_call: NewToolCall::parse,
    266         description: "Description...",
    267         arguments: vec![
    268             // Arguments...
    269         ]
    270     }
    271 }
    272 
    273 pub fn dave_tools() -> Vec<Tool> {
    274     vec![query_tool(), present_tool(), new_tool()]
    275 }
    276 ```
    277 
    278 ### Handling Tool Responses
    279 
    280 Add a new variant to `ToolResponses`:
    281 ```rust
    282 pub enum ToolResponses {
    283     Query(QueryResponse),
    284     PresentNotes,
    285     NewTool(NewToolResponse),
    286 }
    287 ```
    288 
    289 Implement response formatting:
    290 ```rust
    291 fn format_tool_response_for_ai(txn: &Transaction, ndb: &Ndb, resp: &ToolResponses) -> String {
    292     match resp {
    293         // Existing cases...
    294         ToolResponses::NewTool(response) => {
    295             // Format response as JSON...
    296         }
    297     }
    298 }
    299 ```
    300 
    301 ## Advanced Usage Patterns
    302 
    303 ### Context-Aware Queries
    304 
    305 The `QueryContext` enum allows the AI to scope searches:
    306 
    307 ```rust
    308 #[derive(Debug, Deserialize, Serialize, Clone)]
    309 #[serde(rename_all = "lowercase")]
    310 pub enum QueryContext {
    311     Home,
    312     Profile,
    313     Any,
    314 }
    315 ```
    316 
    317 ### Time-Based Queries
    318 
    319 Dave is configured with current and recent timestamps in the system prompt, enabling time-aware queries:
    320 
    321 ```
    322 - The current date is {date} ({timestamp} unix timestamp if needed for queries).
    323 - Yesterday (-24hrs) was {yesterday_timestamp}. You can use this in combination with `since` queries for pulling notes for summarizing notes the user might have missed while they were away.
    324 ```
    325 
    326 ### Filtering Non-Relevant Content
    327 
    328 Dave filters out reply notes when performing queries to improve results:
    329 
    330 ```rust
    331 fn is_reply(note: Note) -> bool {
    332     for tag in note.tags() {
    333         if tag.count() < 4 {
    334             continue;
    335         }
    336 
    337         let Some("e") = tag.get_str(0) else {
    338             continue;
    339         };
    340 
    341         let Some(s) = tag.get_str(3) else {
    342             continue;
    343         };
    344 
    345         if s == "root" || s == "reply" {
    346             return true;
    347         }
    348     }
    349 
    350     false
    351 }
    352 
    353 // Used in filter creation
    354 .custom(|n| !is_reply(n))
    355 ```
    356 
    357 ## Conclusion
    358 
    359 The tools system is what makes Dave truly powerful, enabling it to interact with NostrDB and present content to users. By understanding this system, developers can:
    360 
    361 1. Extend Dave with new capabilities
    362 2. Apply similar patterns in other AI-powered applications
    363 3. Create tools that balance flexibility and structure
    364 4. Build effective interfaces between AI models and application data
    365 
    366 This architecture demonstrates a robust approach to enabling AI assistants to take meaningful actions within applications, going beyond simple text generation to deliver real utility to users.