metadata.md (2385B)
1 2 # Note Metadata 3 4 nostrdb supports a flexible metadata system which allows you to store additional information about a nostr note. 5 6 metadata is stored as a sorted list of TVs (tag, value). The lengths are a fixed size but can reference a data table. 7 8 The type follows the "it's ok to be odd" rule. odd tags are opaque, user defined types that can store data of any kind. 9 10 ## Binary format 11 12 The binary format starts with a header containing the number of metadata entries. The count is followed by a sorted, aligned list of these entries: 13 14 ```c 15 16 // 16 bytes 17 struct ndb_note_meta { 18 // 4 bytes 19 uint8_t version; 20 uint8_t padding; 21 uint16_t count; 22 23 // 4 bytes 24 uint32_t data_table_size; 25 26 // 8 bytes 27 uint64_t flags; 28 }; 29 30 // 16 bytes 31 // 16 bytes 32 struct ndb_note_meta_entry { 33 // 4 byte entry header 34 uint16_t type; 35 uint16_t flags; 36 37 // additional 4 bytes of aux storage for payloads that are >8 bytes 38 // 39 // for reactions types, this is used for counts 40 // normally this would have been padding but we make use of it 41 // in our manually packed structure 42 union { 43 uint32_t value; 44 45 /* if this is a thread root, this counts the total replies in the thread */ 46 uint32_t total_reactions; 47 } aux; 48 49 // 8 byte metadata payload 50 union { 51 uint64_t value; 52 53 struct { 54 uint32_t offset; 55 uint32_t padding; 56 } offset; 57 58 struct { 59 /* number of direct replies */ 60 uint16_t direct_replies; 61 uint16_t quotes; 62 63 /* number of replies in this thread */ 64 uint32_t thread_replies; 65 } counts; 66 67 // the reaction binmoji[1] for reaction, count is stored in aux 68 union ndb_reaction_str reaction_str; 69 } payload; 70 }; 71 72 ``` 73 74 The offset is to a chunk of potentially unaligned data: 75 76 ``` 77 size : varint 78 data : u8[size] 79 ``` 80 81 ### Rationale 82 83 We want the following properties: 84 85 * The ability to quickly iterate/skip over different metadata fields. This is achieved by a fixed table format, without needing to encode/decode like blocks 86 * The ability to quickly update metadata fields. To update flags or counts can be done via race-safe mutation operations. The mutation can be done inplace (memcpy + poke memory address) 87 * The table entries can be inplace sorted for binary search lookups on large metadata tables 88 89 ## Write thread mutation operations 90 91 We want to support many common operations: 92 93 * Increase reaction count for a specific reaction type 94 95 [binmoji]: https://github.com/jb55/binmoji