nostrdb

an unfairly fast embedded nostr database backed by lmdb
git clone git://jb55.com/nostrdb
Log | Files | Refs | Submodules | README | LICENSE

builder.md (73281B)


      1 # Builder Interface Reference
      2 
      3 <!-- vim-markdown-toc GFM -->
      4 
      5 * [Introduction](#introduction)
      6 * [Size Prefixed Buffers](#size-prefixed-buffers)
      7 * [Namespaces](#namespaces)
      8 * [Error Codes](#error-codes)
      9 * [Endianess](#endianess)
     10   * [Deprecated](#deprecated)
     11 * [Buffers](#buffers)
     12 * [Tables](#tables)
     13   * [Adding Fields](#adding-fields)
     14   * [Nested Tables](#nested-tables)
     15 * [Packing tables](#packing-tables)
     16 * [Strings](#strings)
     17 * [Structs](#structs)
     18   * [Fixed Length Arrays in Structs](#fixed-length-arrays-in-structs)
     19 * [Nested Buffers](#nested-buffers)
     20 * [Scalars and Enums](#scalars-and-enums)
     21 * [Vectors](#vectors)
     22 * [Unions](#unions)
     23   * [Union Vectors](#union-vectors)
     24   * [Unions of Strings and Structs](#unions-of-strings-and-structs)
     25 * [Error Handling](#error-handling)
     26 * [Type System Overview](#type-system-overview)
     27 * [Cloning](#cloning)
     28 * [Picking](#picking)
     29 * [Sorting Vectors](#sorting-vectors)
     30   * [Dangers of Sorting](#dangers-of-sorting)
     31   * [Scanning](#scanning)
     32 * [Example of different interface type users](#example-of-different-interface-type-users)
     33 * [Special Emitters](#special-emitters)
     34 
     35 <!-- vim-markdown-toc -->
     36 
     37 
     38 ## Introduction
     39 
     40 We assume a separate read-only file and add extensions to this with
     41 support from a builder library and a builder object.
     42 
     43 The underlying builder library supports two modes of operation that mix
     44 together: `create` which sends data directly to the target buffer
     45 (emitter object) and a stack driven `start/end` approach which allocates
     46 objects and vectors on the stack. The code generator chooses the most
     47 efficient approach given the circumstances.
     48 
     49 Unlike most FlatBuffer language interfaces, tables and vectors are not
     50 created back to front: They are either created completely in one
     51 operation, or they are constructed on a stack front to back until they
     52 can be emitted.  The final buffer is still constructed back to front.
     53 For big-endian platforms this may require temporary stack allocation of
     54 complete vectors where little endian platforms can emit directly.
     55 
     56 Tables and vectors stored in other tables or vectors must be completed
     57 before the can be stored, but unlike must language interfaces they can
     58 be constructed while a parent is also being constructed as long as
     59 nesting remains balanced. While this occasionally may require more
     60 stack, it may also avoid external temporary allocation.
     61 
     62 A builder object is required to start buffer construction. The builder
     63 must be initialized first and can be reset and reused between buffers,
     64 reusing stack allocation. The builder can have a customized emitter
     65 object but here we use the default. Finalizing the buffer depends
     66 the emitter and we can use a default finalizer only because we use the
     67 default emitter - it allocates and populates a linear buffer from a
     68 paged emitter ring buffer.
     69 
     70 Note that in most cases `flatcc_builder_finalize_buffer` is sufficient,
     71 but to be strictly portable, use
     72 `flatcc_builder_finalize_aligned_buffer` and `aligned_free`.
     73 `aligned_free` is often implemented as `free` in `flatcc/portable` but
     74 not on all platforms. As of flatcc version 0.5.0
     75 `flatcc_builder_aligned_free` is provided to add robustness in case the
     76 applications `aligned_free` implementation might differ from the library
     77 version due to changes in compile time flags.
     78 
     79 Generally we use the monster example with various extensions, but to
     80 show a simple complete example we use a very simple schema (`myschema.fbs`):
     81 
     82     table mytable { myfield1: int; myfield2: int; }
     83 
     84     #include "myschema_builder.h"
     85 
     86     void testfun() {
     87 
     88         void *buffer;
     89         size_t size;
     90         flatcc_builder_t builder, *B;
     91         mytable_table_t mt;
     92         B = &builder;
     93         flatcc_builder_init(B);
     94 
     95         /* Construct a buffer specific to schema. */
     96         mytable_create_as_root(B, 1, 2);
     97 
     98         /* Retrieve buffer - see also `flatcc_builder_get_direct_buffer`. */
     99         /* buffer = flatcc_builder_finalize_buffer(B, &size); */
    100         buffer = flatcc_builder_finalize_aligned_buffer(B, &size);
    101 
    102         /* This is read-only buffer access. */
    103         mt = mytable_as_root(buffer);
    104         assert(mytable_myfield1(mt) == 1);
    105         assert(mytable_myfield2(mt) == 2);
    106 
    107         /* free(buffer); */
    108         flatcc_builder_aligned_free(buffer);
    109 
    110         /*
    111          * Reset, but keep allocated stack etc.,
    112          * or optionally reduce memory using `flatcc_builder_custom_reset`.
    113          */
    114         flatcc_builder_reset(B);
    115 
    116         /* ... construct another a buffer */
    117 
    118         /* Reclaim all memory. */
    119         flatcc_builder_clear(B);
    120     }
    121 
    122 Note that a compiled schema generates a `myschema_reader.h` file and
    123 optionally a `myschema_builder.h` and some common support files. When
    124 building a buffer the `myschema_builder.h` must be used but when only
    125 reading then the `myschema_reader.h` file should be used instead. Here
    126 we are only concerned with building. When building, it is necessary to
    127 link with `libflatccrt.a` runtime library but when reading, all
    128 nesessary code is contained in the generated header files.
    129 
    130 The builder object only manages a stack of currently active objects and
    131 does not store an object that is complete. Instead it calls an emitter
    132 object with the partial data ready for emission, similar to a write
    133 function. A default emitter is provided which implements a ring buffer
    134 and the result may be written to a file, copied to a buffer or a
    135 finalized to an allocated buffer. The builder supports these methods
    136 directly for default emitter, and only the default emitter because
    137 emitters are otherwise defined by only one simple emit function - see
    138 `emit_test.c` for a simple example of a custom emitter.
    139 A custom allocator may be useful when working with small buffers in a
    140 constrained environment - the allocator handles temporary stacks,
    141 virtual table caches etc. but not the emitter.
    142 
    143 The allocator and emitter interface is documented in the builder library
    144 header pflatcc_builder.h] and the default implementation in
    145 [flatcc_emitter.h]. The default allocator is implemented as part of the
    146 flatcc_builder source.
    147 
    148 The builder can be reused between buffers using the `reset` operation.
    149 The default emitter can also be reused and will automaticallhy reset
    150 when the buffer is. For custom emitters, any reset operation must be
    151 called manually. The same applies to clear. The reset operations
    152 maintain allocated memory by also reduce memory consumption across
    153 multiple resets heuristically.
    154 
    155 
    156 ## Size Prefixed Buffers
    157 
    158 Buffers can be created with a size prefix of type `uoffset_t`. When
    159 doing this, the buffer is aligned relative to the size prefix such that
    160 buffers can be stacked in a file and for example be accessed via memory
    161 mapping.
    162 
    163 The usual `create_as_root` and `start_as_root` has a variant called
    164 `create_as_root_with_size` and `start_as_root_with_size`.
    165 
    166 To read a buffer with a size prefix use:
    167 
    168     size_t size;
    169     buffer = flatbuffers_read_size_prefix(rawbuffer, &size);
    170 
    171 The size the size of the buffer excluding the size prefix. When
    172 verifying buffers the buffer and size arguments should be used. See also
    173 [monster_test.c] for an example.
    174 
    175 Note that the size prefix ensures internal alignment but does not
    176 guarantee that the next buffer in a file can be appended directly
    177 because the next buffers alignment is unknown and because it potentially
    178 wastes padding bytes.  The buffer size at offset 0 can increased to the
    179 needed alignment as long as endianness is handled and the size of the
    180 size field is subtracted, and zeroes are appended as necesary.
    181 
    182 ## Namespaces
    183 
    184 The generated code is typically wrapped in a custom namespace and
    185 functions and definitions that are library specific are usually mapped
    186 into the namespace. We often use an empty namespace for custom types and
    187 `flatbuffers_` for library names, but usually a `foo_` prefix could also
    188 be used on both cases, where `foo` is a custom namespace.
    189 
    190 Note that the name `flatcc_emitter` is only used with the default emitter
    191 and the name [flatcc_builder] is only used for buffer management but not
    192 for constructing content. Once a valid buffer is ready the common and
    193 namespace (`flatbuffers`) and schema specific (or empty) namespace is used
    194 with schema specific operations.
    195 
    196 All schema specific content is prefixed with a namespace to avoid
    197 conflicts - although the namespace is empty if the schema doesn't
    198 specify any. Note that the same schema can have multiple
    199 namespaces. An example of a namespace prefixed operation:
    200 
    201     MyGame_Example_Monster_create_as_root(B, ... lots of args);
    202 
    203 To simplify this we can use a macro to prefix a namespace. The use
    204 of the name `ns` is arbitrary and we can choose different names for
    205 different namespaces.
    206 
    207     #undef ns
    208     #define ns(x) MyGame_Example_ ## x
    209 
    210 But the above doesn't work with nested calls to ns such as
    211 
    212     ns(Monster_color_add(B, ns(Color_Green));
    213 
    214 it would have to be:
    215 
    216     ns(Monster_color_add)(B, ns(Color_Green);
    217 
    218 Therefore we have a helper macro the does allow nesting:
    219 
    220     #undef ns
    221     #define ns(x) FLATBUFFERS_WRAP_NAMESPACE(MyGame_Example, x)
    222 
    223 The common namespace can also be wrapped for a more consistent
    224 appearance:
    225 
    226     #undef nsc
    227     #define nsc(x) FLATBUFFERS_WRAP_NAMESPACE(flatbuffers, x)
    228 
    229     nsc(string_ref_t) s;
    230     s = nsc(string_create_str(B, "hello, world!"));
    231 
    232 instead of
    233 
    234     flatbuffers_string_ref_t s;
    235     s = flatbuffers_string_create_str(B, "hellow, world!);
    236 
    237 
    238 ## Error Codes
    239 
    240 Functions return values can be grouped roughly into 4 groups: functions
    241 returning pointer, references, `size_t` lengths, and `int` status codes.
    242 Pointers and references return 0 on error. Sizes do not return error.
    243 Status codes return 0 on success or an error code that is usually -1.
    244 Status codes may be checked with `flatbuffers_failed(...)`.
    245 
    246 
    247 ## Endianess
    248 
    249 The function `flatbuffers_is_native_pe()` provide an efficient runtime
    250 check for endianness. Since FlatBuffers are little endian, the function
    251 returns true when the native endianness matches the protocol endianness
    252 which for FlatBuffers is little endian. We do not hardcode little endian
    253 because it enables us to support other protocols in the future - for
    254 example the struct conversions may be very useful for big endian network
    255 protocols.
    256 
    257 > As of flatcc 0.4.0 it is possible to compile flatcc with native
    258 > big-endian support which has been tested on AIX. More details in
    259 > [README Endianness](https://github.com/dvidelabs/flatcc#endianness)
    260 
    261 
    262 By testing `is_native_pe` dependencies on speficic compile time flags
    263 can be avoided, and these are fragile:
    264 
    265 During build, vectors and structs behave differently from tables: A
    266 table updates one field at a time, doing endian conversion along the
    267 way. A struct is either placed in a table, and is converted by the table
    268 specific operation, or it is placed in a vector. A vector only does the
    269 endian conversion when the vector is finished, so when a vector is not
    270 created atomically with a single `create` call, the elements are placed on a
    271 stack. By default this is in native format, but the user may choose to
    272 place buffer encoded structs or scalars in the vector and call
    273 `vec_end_pe`. The same `push` operation can be used to place a
    274 natively encoded struct and a buffer encoded struct in the vector
    275 because it does no conversion at that point. Therefore there is also no
    276 `push_pe` method that would mean to push an unconverted element unto
    277 the stack. Only for tables and entire vectors does the pe command make
    278 sense. If a vector wishes to push a buffer encoded struct when the
    279 vector is otherwise constructed in native encoding or vice versa, the
    280 vector may be extended empty and then assigned using any of the
    281 `assign`, `assign_from_pe` or `assign_to_pe` calls.
    282 
    283 We did not mention that a struct can also be a standalone object
    284 as a buffer root, and for that it has a `end_pe` call that essentially
    285 works like a single element vector without a length prefix.
    286 
    287 The `clone` operation is a more userfriendly `pe` operation which takes
    288 an object or a vector from an existing buffer and places it in a new
    289 buffer without endian conversion.
    290 
    291 ### Deprecated
    292 
    293 __NOTE: `FLATBUFFERS_LITTLEENDIAN` is deprecated and will be removed in
    294 a future version. It just complicates endina handling.__
    295 
    296 The header files tries to define `FLATBUFFERS_LITTLEENDIAN` to 0 or 1
    297 based on system definitions but otherwise leaves the flag undefined.
    298 Simply testing for
    299 
    300     #if FLATBUFFERS_LITTLEENDIAN
    301     ...
    302     #endif
    303 
    304 will not fail if the endianness is undetected but rather give the
    305 impression that the system is big endian, which is not necessarily true.
    306 The `flatbuffers_is_native_pe()` relates to the detected or system
    307 provided conversion functions if a suitable `endian.h` file after the
    308 header file gave up on its own detection (e.g. `le16toh(1) == 1`).
    309 Therefore, it is better to use `flatbuffers_is_native_pe()` in most
    310 cases. It also avoids making assumptions on whether the protocol is
    311 little or big endian.
    312 
    313 ## Buffers
    314 
    315 A buffer can most simply be created with the `create_as_root` call for
    316 a table or a struct as seen ealier. The `as_root` part is just a thin
    317 wrapper around buffer start and stop calls and using these allows for
    318 more flexibility. the `as_root` also automatically uses the defined file
    319 identifier if any.
    320 
    321 The build process begins with starting a buffer. The buffer may contain
    322 a struct or table, so one of these should be constructed subsequently.
    323 Structs are generally created inline in tables, only at the buffer level
    324 is a struct created independently. The api actually permits other
    325 formats, but it will not be valid flatbuffers then.
    326 
    327     flatcc_builder_ref_t root;
    328     flatcc_builder_init(B);
    329     /* 0 indicates no file identifier. */
    330     flatcc_builder_buffer_start(B, 0);
    331     root = /* ... construct a table or a struct */
    332     flatcc_builder_buffer_end(B, root);
    333 
    334 `buffer_start` takes a file identifier as second argument. If null or a
    335 string with null characters, the identifier is not stored in the buffer.
    336 
    337 Regardless of whether a struct or table is declared as root in the schema or
    338 not, there are methods to automatically start both the buffer and struct or buffer
    339 and table such as `Monster_start/end_as_root`. This is also valid for
    340 nested buffers. If the schema has a file identifier, it is used as
    341 identifier for the created object. The alternative
    342 `create_as_root_with_identifier` allows for explicitly setting an id or
    343 explicitly dropping an id by providing a null argument. The
    344 corresponding reader function `Monster_as_root(buffer)` also has a
    345 `Monster_as_root_with_identifier(buffer, id)`. Here the id is ignored if the id
    346 is null, and otherwise the operation returns null if the id does not match.
    347 For the most part ids are handled transparently by these defaults.
    348 
    349 The buffer can be started with block alignment and/or a custom
    350 identifier using the `flatcc_builder_buffer_start_aligned`:
    351 
    352     flatcc_builder_buffer_start_aligned(B, "myid", 16);
    353     ...
    354     flatcc_builder_buffer_end(B, root);
    355 
    356 The alignment can be 0 using the minimum required alignment, which is
    357 derived from the operations between `start/end`. The alignment argument
    358 is called `block_align` and is useful if the emitter operates on blocks
    359 such as encryption, cache line isolation, or compression blocks where
    360 the final buffer should align with the blocks used during construction.
    361 This can lead to significant zero padding just after the block header,
    362 depending on block size.
    363 
    364 The schema specified identifier is given as:
    365 
    366     flatbuffers_identifier
    367 
    368 and defaults to null. The schema specified extension is given as:
    369 
    370     flatbuffers_extension
    371 
    372 and defaults to null. Note that `flatbuffers_` is replaced by whatever
    373 namespace is chosen. Each specific schema type also has a named file
    374 exntension reflection the extension active when the type was defined,
    375 for example:
    376 
    377     MyGame_Example_Monster_file_identifier
    378 
    379 This define is used when `create_as_root` automatically sets a file
    380 identifier.
    381 
    382 NOTE: before flatcc 0.6.1, the identifier was named
    383 
    384     MyGame_Example_Monster_identifier (DEPRECATED)
    385 
    386 but that would conflict with a table field named `identifier` which
    387 happened often enough to be a problem. This naming is now removed on
    388 conflict and will be completely removed in a future version.
    389 
    390 When the buffer is ended, nothing special happens but only at this point
    391 does it really makes sense to access the resulting buffer. The default
    392 emitter provides a copy method and a direct buffer access method. These
    393 are made available in the builder interface and will return null for
    394 other emitters. See also [flatcc_builder.h] and the default emitter in
    395 `flatcc_emitter.h`.
    396 
    397 
    398 ## Tables
    399 
    400 ### Adding Fields
    401 
    402 If `Monster` is a table, we can create a Monster buffer (after
    403 builder init) as follows:
    404 
    405     Monster_start(B);
    406     Monster_Hp_add(B, 80);
    407     ...
    408     flatcc_builder_buffer_create(B, Monster_end(B));
    409 
    410 All scalar and enums are added similar to the `Monster_add_Hp` call. We
    411 will subsequently see how to deal with other types.
    412 
    413 A table can also be created in a single operation using `create`:
    414 
    415     Monster_ref_t m;
    416     m = Monster_create(B, 80, ...);
    417 
    418 The create arguments are those taken by the individual fields `add`
    419 operations which is either an scalar, enum, or a reference returned by
    420 another create or end call. Note that unlike the C++ interface, unions
    421 only take a single argument that is also accepted by the `add` operation
    422 of a union field. Deprecated fields are not included in the argument
    423 list.
    424 
    425 As of v0.5.3 the arguments are given in field id order which is usually
    426 the same as the schema listed order, except with id attributes are
    427 given explicitly. Using id order ensures version stability. Note that
    428 since deprecated fields are omitted, deprecated fields can still break
    429 existing code.
    430 
    431 BREAKING: Prior to flatcc v0.5.3 the create call would use the schema order
    432 also when fields have id attributes specifying a different order. This
    433 could break code across versions and did not match the C++ behavior.
    434 It was also document that the `original_order` attribute affected create
    435 argument order, but that was incorrect.
    436 
    437 NOTE: If the `original_order` attribute is set on a table, the `create`
    438 implementation adds fields to the table in schema listed order,
    439 otherwise it adds fields in order of decreasing size to reduce alignment
    440 overhead. Generally there should be no need to use the `original_order`
    441 attribute. This doesn't affect the call argument order although that
    442 was incorrectly document prior to v 0.5.3.
    443 
    444 NOTE: the `create` and `create_as_root` operations are not guaranteed to
    445 be available when the number of fields is sufficiently large because it
    446 might break some compilers. Currently there are no such restrictions.
    447 
    448 Scalars and enums do not store the value if it it matches the default
    449 value which is by default 0 and otherwise defined in the schema. To
    450 override this behavior, use `force_add`. In the monster example, health
    451 points default to 100 (percent), so if we wish to force store it in the
    452 buffer we could use:
    453 
    454     Monster_hp_force_add(B, 100);
    455 
    456 Only scalar fields and enums have a `force_add` operation since only these
    457 types have a default value, and other types have a meaningful
    458 interpretation of null. (It is not quite clear if empty tables separate
    459 from null/absent are valid in all implementations).
    460 
    461 `force_add` may be useful when roundtripping data from a database where it is
    462 relevant to distinguish between any valid value and null. Most readers will not
    463 be able to tell the difference, but it is possible to inspect a flatbuffer to
    464 see if a table field is present, present and default, or absent, meaning null.
    465 
    466 NOTE: As of mid 2020, FlatBuffers added optional scalar table fields with support in flatcc 0.6.1. These fields automatically imply `force_add` to represent null values when a field is absent and therefore these fields do not have a `force_add` method and these fields also do not have a default value other than `null`, i.e. null if not added.
    467 
    468 If Monster is declared as root, the above may also be called as:
    469 
    470     Monster_start_as_root(B);
    471     Monster_add_hp(B, 80);
    472     ...
    473     Monster_end_as_root(B);
    474 
    475 (Calling `Monster_end` instead would require `buffer_end` call
    476 subsequently, and is basically a violation of nesting).
    477 
    478 ### Nested Tables
    479 
    480 Tables can be nested, for example the Mini field may have type
    481 Monster table again (a recursive type):
    482 
    483     buffer_start(B);
    484     Monster_start(B);
    485     Monster_add_Hp(B, 80);
    486     Monster_start(B);
    487     Monster_hp_add(B, 81);
    488     ...
    489     Monster_mini_add(Monster_end(B));
    490     ...
    491     flatcc_builder_buffer_end(B, Monster_end(B));
    492 
    493 The child Monster table may be created before the parent or as above
    494 between the tables start and end. If created before, reference must be
    495 stored until it can be added. The only requirement is that start and
    496 end are balanced, that the sub-table is ended before the parent, and
    497 that both are created in the same buffer (nested buffers can be created
    498 while the parent buffer is still being created, similar to sub-tables,
    499 so it is possible to mess this up):
    500 
    501     Monster_ref_t root, mini;
    502 
    503     buffer_start(B);
    504     Monster_start(B);
    505     Monster_hp_add(B, 81);
    506     mini = Monster_end(B);
    507 
    508     Monster_start(B);
    509     Monster_hp_add(B, 80);
    510     Monster_mini_add(B, mini);
    511     root = Monster_end(B);
    512 
    513     flatcc_builder_buffer_end(B, root)
    514 
    515 
    516 Rather than adding a child table explicitly, it can be started and ended
    517 as an operation on the field name, here with `Monster_Mini_start/end`:
    518 
    519     Monster_ref_t root;
    520 
    521     Monster_start(B);
    522     Monster_add_Hp(B, 80);
    523     Monster_mini_start(B);
    524     Monster_hp_add(B, 81);
    525     Monster_mini_end(B);
    526     root = Monster_end(B);
    527 
    528     flatcc_builder_buffer_end(B, root);
    529 
    530 We can repeat the the table nesting as deep as we like, provided our
    531 builder is willing to allocate enough stack space.
    532 
    533 **Warning**: It is possible to use the wrong table type operations
    534 between `start/end` - don't do that. It is a tradeoff between usability
    535 and type safety.
    536 
    537 Note that vectors, strings and structs map several standard operations
    538 to a field name, for example `mytable_myfield_push(B, x)`. This is not the
    539 case with table fields which only map `start/end/create` in part because it
    540 would never terminate for recursive types and in part because each table
    541 is different making a generic mapping rather complex and with very long
    542 names.
    543 
    544 A table may be created with a constructor, but it requires all
    545 non-scalar objects to be references or pointers. Struct fields must be
    546 pointers to zero padded structs, and strings, vectors and tables must be
    547 references. The constructors are probably most useful for simple tables
    548 with mostly scalar values (here we use the original Monster fields and
    549 leaves out any we have invented for the sake of illustration):
    550 
    551 IMPORTANT: objects can generally only be created within a buffer
    552 context, i.e. after `buffer_start`. For example calling
    553 `flatbuffers_uint8_vec_create` before `Monster_create_as_root`
    554 technically violates this rule because the create call also starts the
    555 buffer. It is, however, allowed at the top level. For nested buffers
    556 (see later) this must be avoided because the vector would end up in the
    557 wrong buffer.
    558 
    559     Monster_ref_t m;
    560     uint8_t invdata[4] = { 1, 2, 3, 4 };
    561     Vec3_t vec;
    562 
    563     flatbuffers_uint8_vec_ref_t inventory =
    564         flatbuffers_uint8_vec_create(B, invdata, 4);
    565     m = Monster_create(B, &vec, 150, 80, name, inventory,
    566         Color_Red, Any_as_NONE());
    567     flatcc_builder_buffer_create(m);
    568 
    569 or
    570 
    571     Monster_create_as_root(B, &vec, 150, 80, name, inventory,
    572         Color_Red, Any_as_NONE());
    573 
    574 ## Packing tables
    575 
    576 By reordering the fields, the table may be packed better, or be better
    577 able to reuse an existing vtable. The `create` call already does this
    578 unless the attribute `original_order` has been set. Unions present a
    579 special problem since it is two fields treated as one and the type field
    580 will generally waste padding space if stored in order:
    581 
    582 To help pack unions better these can be added with the type
    583 seperate from the value reference using `add_type(B, test.type)`,
    584 `add_value(B, test)` where the value is only added if the type is
    585 not `NONE`. The `add_type` should be called last since it is the
    586 smallest type.
    587 
    588 The same field should not be added more than at most once. Internal
    589 reservations that track offset fields may overflow otherwise. An
    590 assertion will fail in debug builds.
    591 
    592 Required table fields will be asserted in debug builds as part of the
    593 `end/create` call.  Only offset fields can have a required attribute.
    594 
    595 The generated `monster_test_reader.h` from [monster_test.fbs] shows how
    596 the default packing takes place in generated `create` calls, see for
    597 example the typealias test. Note that for example vectors are stored
    598 together with integers like `uint32` because references to vectors have
    599 the same size as `uint32`.
    600 
    601 
    602 ## Strings
    603 
    604 Strings can be added to tables with zero terminated strings as source
    605 
    606     Monster_start(B);
    607     ...
    608     Monster_name_create_str(B, "Mega Monster");
    609     Monster_end(B);
    610 
    611 or strings potententially containing zeroes:
    612 
    613     #define MONSTER "Mega\0Monster"
    614     Monster_start(B);
    615     ...
    616     /* Includes embedded zero. */
    617     Monster_name_create(B, MONSTER, sizeof(MONSTER));
    618     Monster_end(B);
    619 
    620 or zero terminated source up to at most `max_len` characters.
    621 
    622     #define MONSTER "Mega\0Monster"
    623     Monster_start(B);
    624     ...
    625     /* "Mega" */
    626     Monster_name_create_strn(B, MONSTER, 12);
    627     Monster_end(B);
    628 
    629 The `create_str` and `create_strn` versions finds the string length via strlen
    630 and strnlen respectively. `append_string` also has `_str/_strn` versions.
    631 
    632 A string can also be created from an existing flatbuffer string in which
    633 case the length is expected to be stored 4 bytes before the pointer in
    634 little endian format, and aligned properly:
    635 
    636     Monster_name_clone(B, mybufferstring);
    637 
    638 or, create a string at most 4 characters long starting at 0-based index
    639 10, if present:
    640 
    641     Monster_name_slice(B, mybufferstring, 10, 4);
    642 
    643 If index or index + len goes beyond the source, the result is truncated
    644 accordingly, possibly resulting in an empty string.
    645 
    646 A string can also be create independently. The above is just shortcuts
    647 for that:
    648 
    649     flatbuffers_string_ref_t monster_name;
    650     monster_name = flatbuffers_string_create_str("Mega Monster");
    651     Monster_name_add(B, monster_name);
    652 
    653 Strings are generally expected to be utf-8, but any binary data will be
    654 stored. Zero termination or embedded control codes are includes as is.
    655 The string gets a final zero temination regardless, not counted in the
    656 string length (in compliance with the FlatBuffers format).
    657 
    658 A string can also be constructed from a more elaborate sequence of
    659 operations. A string can be extended, appended to, or truncated and
    660 reappended to, but it cannot be edited after other calls including calls
    661 to update the same string. This may be useful if stripping escape codes
    662 or parsed delimiters, etc., but here we just create the same "Mega
    663 Monster" string in a more convoluted way:
    664 
    665     flatbuffers_string_ref_t name;
    666     char *s;
    667     #define N 20
    668     Monster_start(B);
    669     ...
    670     flatbuffers_string_start(B);
    671     flatbuffers_string_append(B, "Mega", 4);
    672     flatbuffers_string_append(B, " ", 1);
    673     s = flatbuffers_string_extend(B, N);
    674     strncpy(s, "Monster", N);
    675     flatbuffers_string_truncate(B, N - strlen(s));
    676     name = flatbuffers_string_end(B);
    677     Monster_name_add(B, name);
    678     ...
    679     Monster_end(B);
    680 
    681 `flatbuffers_string_create...` calls are also available when creating
    682 the string separate from adding it to a table, for example:
    683 
    684     flatbuffers_string_h name;
    685     name = flatbuffers_string_create_str(B, "Mini Monster");
    686 
    687 It is guaranteed that any returned the string buffer is zero filled and
    688 has an extra zero after the requested length such that strlen can be
    689 called on the content, but only the requested bytes may be updated.
    690 
    691 Every call only returns the substring being added to the string in that
    692 operation. It is also possible to call `flatbuffers_string_edit` to get a
    693 modifiable pointer to the start of the string.
    694 
    695 `flatbuffers_string_reserved_len(B)` returns the current string length
    696 including any embedded zeroes, but excluding final zero termination. It
    697 is only valid until `string_end` is called.
    698 
    699 See [flatcc_builder.h] for detailed documentation. Essentially `extend`
    700 reserves zeroed space on the stack and returns a buffer to the new
    701 space, and truncate reduces the overall size again, and the string is
    702 then given the final length and a zero termination at the end.
    703 
    704 There is no endian conversion (except internally for the string length),
    705 because UTF-8 strings are not sensitive to endianness.
    706 
    707 Like tables, the string may be created while a parent container is being
    708 constructed, or before.
    709 
    710 Strings can also be used as vector elements, but we will get that when
    711 discussing vectors.
    712 
    713 ## Structs
    714 
    715 Structs in tables can be added as:
    716 
    717     Monster_pos_create(B, 1, 2, 3);
    718 
    719 The above essentially does the following:
    720 
    721     Vec3_t *v;
    722     v = Monster_pos_start(B);
    723     Vec3_assign(v, 1, 2, -3.2);
    724     Monster_pos_end(B);
    725 
    726 Some versions of the monster schema has extra test fields - these would
    727 break the assign approach above because there would be extra arguments.
    728 Instead we can rely on the zero intialization and assign known fields.
    729 
    730     Vec3_t *v;
    731     v = Monster_pos_start(B);
    732     v->x = 1, v->y = 2, v->z = -3.2;
    733     Monster_pos_end(B);
    734 
    735 `Monster_pos_end_pe(B)` can be used when the struct is known to be
    736 little endian (pe for protocol endian, meaning no conversion is necessary),
    737 for example copied from an existing buffer, but then `clone` is a better
    738 choice:
    739 
    740     Monster_pos_clone(B, &v);
    741 
    742 When the struct is created alone for use as root:
    743 
    744     Vec3_ref_t root;
    745     root = Vec3_create(B, 1, 2, 3)
    746     flatcc_builder_buffer_create(B, root);
    747 
    748 An existing struct can be added as:
    749 
    750     Vec3_t v;
    751     Vec3_assign(&v, 1, 2, 3);
    752     /* v does not have to be zero padded. */
    753     Monster_pos_add(B, &v);
    754 
    755 When adding a struct that is already little endian, presumably from an
    756 existing buffer, it can be cloned using:
    757 
    758     Monster_pos_clone(B, &v);
    759 
    760 Clone assumes the source struct is both little endian and that padding
    761 is already zeroed (example ignores error handling), and `end_pe`
    762 does nothing.
    763 
    764     *Monster_pos_start(B) = v;
    765     Monster_pos_end_pe(B);
    766 
    767 There are several assignment types that convert between host (native)
    768 endianness and buffer endiannes. We use `pe` to indicate
    769 `protocol_endian` rather than just `le` for `little endian` because it
    770 allows us to change endianness to big endian in the the future and it
    771 more clearly states the intention. While big endian is not allowed in
    772 FlatBuffers, big endian structs may be useful in other network
    773 protocols - but it is not currently supported because it would force
    774 little endian platforms to support byte-swapping. The operations are:
    775 
    776 `assign_from_pe`, `assign_to_pe`, `copy`, `copy_from_pe`,
    777 `copy_to_pe`, `to_pe` and `from_pe`.
    778 
    779 All the copy operations takes a const pointer as source, and
    780 `to/from_pe` is just copy with same source and destination:
    781 
    782     Vec3_t v, v2;
    783     Vec3_assign_to_pe(&v2, 1, 2, 3);
    784     Vec3_copy_from_pe(Vec3_clear(&v), &v2);
    785     Vec3_to_pe(&v);
    786 
    787 `from_pe` means from little endian to native endian, end `to_pe`
    788 is the opposite. On little endian platforms all copy operations behave
    789 the same and only move fields, not padding. `to/from_pe` conversion
    790 will leave deprecated fields either as they were, or zero them because
    791 the operation may be skipped entirely on protocol endian native platforms.
    792 
    793 While struct fields cannot be deprecated officially, they are supported
    794 if the schema compiler is flagged to accept then. The struct fields are
    795 renamed and assigned 0 when using assign or copy, and assign / create has
    796 no argument for them.
    797 
    798 Because padding can carry noise and unintended information, structs
    799 should be cleared before assignment - but if used as a source to copy
    800 the padding is not copied so only the destation need to be zeroed.
    801 
    802 If a struct is nested, the assign operation includes all fields as if
    803 the struct was flattened:
    804 
    805     typedef struct Plane Plane_t;
    806     struct Plane {
    807         Vec3_t direction;
    808         Vec3_t normal;
    809     };
    810     Plane_t plane;
    811     Plane_clear(&plane);
    812     Plane_assign(&plane, 1, 2, 3, 7, 8, 9);
    813 
    814 Structs can also be created standalone, similar to tables and vectors,
    815 but FlatBuffers only support this when the struct is used as root.
    816 
    817 Assuming Vec3 is declared as root, a buffer only holding a Vec3 struct
    818 can be created using:
    819 
    820     Vec3_create_as_root(B, 1, 2, 3);
    821 
    822 Important: do not store the above as a nested buffer - it would be
    823 missing the vector size field. If `Monster_playground` is a ubyte vector
    824 with `nested_flatbuffer` attribute, then
    825 `Monster_playground_start/end_as_root` may be used.
    826 
    827 Structs also support `start/end_as_root`. In this case `start` returns
    828 the struct pointer, and `end_pe_as_root` is supported:
    829 
    830     Vec3_t *v;
    831     v = Vec3_start_as_root(B);
    832     v->x = 1, v->y = 2, v->z = 3;
    833     Vec3_end_as_root(B);
    834 
    835 (Be careful with the different result codes since a tables `start_as_root`
    836 returns an integer result code where 0 is success while a struct returns
    837 a pointer that is null on failure.)
    838 
    839 The following also creates a buffer at top-level, but it may also be
    840 added as a nested buffer because the stack frame detects the nesting:
    841 
    842     Vec3_t *v;
    843     flatcc_builder_buffer_start(B);
    844     v = Vec3_start(B);
    845     v->x = 1, v->y = 2, v->z = 3;
    846     flatcc_builder_buffer_end(B, Vec3_end(B));
    847 
    848 or
    849     flatcc_builder_buffer_start(B);
    850     ...
    851     Monster_start(B);
    852     flatcc_builder_buffer_start(B);
    853     v = Vec3_start(B);
    854     v->x = 1, v->y = 2, v->z = 3;
    855     Monster_playground_add(B,
    856         flatcc_builder_buffer_end(B, Vec3_end(B)));
    857     flatcc_builder_buffer_end(B, Monster_end(B));
    858 
    859 or
    860 
    861     flatcc_builder_buffer_ref_t nested_root;
    862     flatcc_builder_buffer_start(B);
    863     nested_root = Vec3_create_as_root(B, 1, 2, 3);
    864     Monster_start(B);
    865     Monster_playground_add(B, nested_root);
    866     flatcc_builder_buffer_end(B, Monster_end(B));
    867 
    868 A `buffer_ref_t` can be used as `uint8_vec_ref_t` when the
    869 buffer is nested, and otherwise the reference cannot be used
    870 for anything other than testing for failure. The buffer content
    871 should match the type declared in a `nested_flatbuffers` attribute
    872 but it isn't enforced, and a root can be stored in any field of
    873 [ubyte] type.
    874 
    875 When `Monster_playground` is declared as nested:
    876 
    877     ...
    878     Monster_start(B);
    879     Monster_playground_create_as_root(B, 1, 2, 3);
    880     flatcc_builder_buffer_end(B, Monster_end(B));
    881     ...
    882 
    883 Be aware that `Vec3_t` is for native updates while `Vec3_struct_t` is a const
    884 pointer to an endian encoded struct used in the reader interface, and actually
    885 also as source type in the clone operation.
    886 
    887 ### Fixed Length Arrays in Structs
    888 
    889 As of flatcc 0.6.0 it is possible to have fixed length arrays as structs
    890 members. A fixed length array is equivalent to having a struct field repeated
    891 one or more times. The schema syntax is `name : [type:count];` similar to an
    892 ordinary struct field `name : type;`. The type is any type that can ba valid
    893 struct field type including enums and nested structs. The size cannot be 0 and
    894 the overall size is limited by the maximum struct size the array is contained
    895 within which is typically 65535 (2^16-1).
    896 
    897 For example, given the schema:
    898 
    899     struct MyStruct {
    900       counters:[int:3];
    901       // char is only valid as a fixed length array type
    902       name:[char:6];
    903     }
    904     table MyTable {
    905       mystruct:MyStruct;
    906     }
    907 
    908 The table can be created with:
    909 
    910     ns(MyStruct_t) *x;
    911     ns(MyTable_start_as_root(B));
    912     x = ns(MyTable_mystruct_start(B));
    913     x->counters[0] = 1;
    914     x->counters[1] = 2;
    915     x->counters[2] = 3;
    916     strncpy(x->name, "Kermit", sizeof(x->name));
    917     ns(MyTable_mystruct_end(B));
    918     ns(MyTable_end_as_root(B));
    919 
    920 Note that char arrays are not zero terminated but they are zero padded, so
    921 strncpy is exactly the right operation to use when assigning to char arrays,
    922 at least when they do not contain embedded nulls which is valid.
    923 Char arrays are expected to be ASCII or UTF-8, but an application may use
    924 other encodings if this is clear to all users.
    925 
    926 With assignment:
    927 
    928     int data[3] = { 1, 2, 3 };
    929     ns(MyStruct_t) *x;
    930     ns(MyTable_start_as_root(B));
    931     x = ns(MyTable_mystruct_start(B));
    932     // Careful: the name argument does not use strncpy internally
    933     // so the source must be at least the expected length
    934     // like other array arguments. Strings can have embedded nulls.
    935     ns(MyStruct_assign(x, data, "Kermit");
    936     ns(MyTable_mystruct_end(B));
    937     ns(MyTable_end_as_root(B));
    938 
    939 To read a struct the pointer to the struct is retrieved first
    940 
    941     int sum;
    942     int i;
    943     const char *name;
    944     size_t name_len;
    945     ns(MyTable_table_t) t;
    946     ns(MyStruct_struct_t) x;
    947 
    948     t = ns(MyTable_as_root(buf));
    949     x = ns(MyTable_mystruct_get(t));
    950     for (sum = 0, i = 0; i < ns(MyStruct_counters_get_len()); ++i) {
    951       sum += ns(MyStruct_counters_get(x, i)) +
    952       // char arrays are endian neutral, so we can use pointer access.
    953       name = ns(MyStruct_name_get_ptr(x);
    954       name_len = strnlen(name, ns(MyStruct_name_get_len()));
    955       printf("Added counters from %.*s", name_len, name);
    956       // char arrays can be accessed like other arrays:
    957       // ns(MyStruct_name_get(x, i);
    958     }
    959 
    960 An alternative to `strnlen` is strip trailing zeroes which will allow for
    961 char arrays embedded zeroes, but there is no direct support for this. The JSON
    962 printer uses this approach to shorten the printed char array string.
    963 
    964 The `_get` suffix can be ommitted in the above if the flatcc `-g` has not
    965 supplied to reduce the risk of name conflicts, but not for `_get_len` and
    966 `_get_ptr`.
    967 
    968 Note that it is not possible to have fixed length arrays as part of a table but
    969 it is possible to wrap such data in a struct, and it is also possible to have
    970 vectors of structs that contain fixed length arrays.
    971 
    972 
    973 ## Nested Buffers
    974 
    975 These are discussed under Structs and Table sections but it is worth
    976 noting that a nested buffers can also be added as pe ubyte vectors
    977 which is probably the original intention with nested buffers. However,
    978 when doing so it can be difficult to ensure the buffer is correctly
    979 aligned. The untyped `flatcc_builder` has various options to deal with
    980 this, but with generated code it is better to create a nested buffer
    981 inline when suitable (with nested `buffer_start/end` or
    982 `mytable_myfield_create_as_root`) - for example a message wrapper with
    983 a union of tables holding buffer for a specific message type. In other
    984 cases the buffer may truly be created independently of the current
    985 buffer and then it can be added with controlled alignment using either
    986 the `flatcc_builder` api for full control, or the `nest` operation on
    987 nested table and struct fields:
    988 
    989 To create and add a ubyte vector with a higher alignment than ubytes
    990 single byte alignment, the following operation is available as an
    991 operation on a nested buffer field:
    992 
    993     Monster_playground_nest(B, void *data, size_t size, uint16_t align);
    994 
    995 If alignment is unknown, it can be set to 0, and it will default to 8
    996 for nested table types, and to the struct alignment for struct buffers.
    997 
    998 Block alignment is inherited from the parent buffer so the child buffer
    999 ends up in its own set of blocks, if block alignment is being used. If
   1000 the nested buffer needs a different block alignment, the `flatcc_builder`
   1001 api must be used.
   1002 
   1003 All structs and tables have an `start/end/create_as_root` even if they
   1004 are not referenced by any `nested_flatbuffers` field and they will
   1005 create [ubyte] vectors containing a nested buffer but only [ubyte]
   1006 fields with `nested_flatbuffers` attribute will dedicated
   1007 `start/end/create_as_root` on the field name. Structs also have
   1008 `end_pe_as_root`.
   1009 
   1010 
   1011 ## Scalars and Enums
   1012 
   1013 Scalars keep their original type names `uint8_t`, `double`, etc, but
   1014 they get some operations similar to structs. These are contained in a
   1015 namespace which by default is `flatbuffers_`, for example:
   1016 
   1017     uint16_t *flatbuffers_uint16_to_pe(uint16_t *p);
   1018     uint16_t *flatbuffers_uint16_from_pe(uint16_t *p);
   1019     flatbuffers_bool_t *flatbuffers_bool_to_pe(flatbuffers_bool_t *p);
   1020     flatbuffers_bool_t *flatbuffers_bool_from_pe(flatbuffers_bool_t *p);
   1021 
   1022 These may be used freely, but are primarily present as an interface to
   1023 the vector operations also defined for structs.
   1024 
   1025 Enums have similar definitions which may be used to convert endianness
   1026 without being concerned with the underlying integer type, for example:
   1027 
   1028     Color_enum_t *Color_to_pe(Color_enum_t *p);
   1029 
   1030 ## Vectors
   1031 
   1032 Vectors can be created independently, or directly when updating a table - the
   1033 end result is the same. Builder vector operations always reference element
   1034 values by pointer, or by reference for offset types like tables and strings.
   1035 
   1036     uint8_t v;
   1037     Monster_inventory_start(B);
   1038     v = 1;
   1039     flatbuffers_uint8_vec_push(B, &v);
   1040     v = 2;
   1041     flatbuffers_uint8_vec_push(B, &v);
   1042     v = 3;
   1043     flatbuffers_uint8_vec_push(B, &v);
   1044     Monster_inventory_end(B);
   1045 
   1046 or
   1047 
   1048     flatbuffers_uint8_vec_ref_t inv;
   1049     uint8_t v;
   1050     flatbuffers_uint8_vec_start(B);
   1051     v = 1;
   1052     flatbuffers_uint8_vec_push(B, &v);
   1053     v = 2;
   1054     flatbuffers_uint8_vec_push(B, &v);
   1055     v = 3;
   1056     flatbuffers_uint8_vec_push(B, &v);
   1057     inv = flatbuffers_uint8_vec_end(B);
   1058     Monster_inventory_add(B, inv);
   1059 
   1060 Because it can be tedious and error-prone to recall the exact field
   1061 type, and because the operations are not type safe (any kind of push
   1062 would be accepted), some vector operations are also mapped to the field
   1063 name:
   1064 
   1065     uint8_t v;
   1066     Monster_inventory_start(B);
   1067     v = 1;
   1068     Monster_inventory_push(B, &v);
   1069     v = 2;
   1070     Monster_inventory_push(B, &v);
   1071     v = 3;
   1072     Monster_inventory_push(B, &v);
   1073     Monster_inventory_end(B);
   1074 
   1075 Note: vector operations on a type uses the `_vec_<operation>` syntax, for
   1076 example `uint8_vec_push` or `Monster_vec_push` while operations that are mapped
   1077 onto table field names of vector type do not use the `_vec` infix because it is
   1078 not a type name, for example `Monster_inventory_push`.
   1079 
   1080 A slightly faster operation preallocates the vector:
   1081 
   1082     uint8_t *v;
   1083     Monster_inventory_start(B);
   1084     v = Monster_inventory_extend(B, 3);
   1085     v[0] = 1, v[1] = 2, v[2] = 3;
   1086     v = Monster_inventory_extend(B, 2);
   1087     v[0] = 4, v[1] = 5;
   1088     Monster_inventory_end(B);
   1089 
   1090 Push just extends one element at time.  Note that `extend` returns the
   1091 pointer to the extended vector segment. The full vector can be accessed
   1092 with `edit` and `reserved_len` between `start/end` (recalling that pointers
   1093 cannot be reused across buffer calls):
   1094 
   1095     uint8_t *v, i;
   1096     uint8_t data[] = { 1, 2 };
   1097     Monster_inventory_start(B);
   1098     Monster_inventory_push(B, &data[0]);
   1099     Monster_inventory_push(B, &data[1]);
   1100     v = Monster_inventory_edit(B);
   1101     for (i = 1; i < Monster_inventory_reserved_len(B); ++i) {
   1102         v[i] = v[i - 1] + v[i];
   1103     }
   1104     Monster_inventory_end(B);
   1105 
   1106 Note that the name `reserved_len` is to avoid confusion with
   1107 `_vec_len` read operation. It also indicates that it is not the final
   1108 size since it may change with `truncate/extend`.
   1109 
   1110 A vector can also contain structs. Let us extend the Monster example
   1111 with a vector of positions, so we can have a breadcrumb trail:
   1112 
   1113     Monster_breadcrumbs_start(B);
   1114     Vec3_vec_push_create(B, 1, 2, 3);
   1115     Vec3_vec_push_create(B, 3, 4, 5);
   1116     Monster_breadcrumbs_end(B);
   1117 
   1118 or
   1119 
   1120     Monster_breadcrumbs_start(B);
   1121     Monster_breadcrumbs_push_create(B, 1, 2, 3);
   1122     Monster_breadcrumbs_push_create(B, 3, 4, 5);
   1123     Monster_breadcrumbs_end(B);
   1124 
   1125 or
   1126 
   1127     Vec3_t *trails[2];
   1128     Monster_breadcrumbs_start(B);
   1129     trails = Monster_breadcrumbs_extend(B, 2);
   1130     Vec3_create(&trails[0], 1, 2, 3);
   1131     Vec3_create(&trails[1], 4, 5, 6);
   1132     Monster_breadcrumbs_end(B);
   1133 
   1134 The `vec_start/exttend/end/end_pe/create/create_pe/clone/slice` are
   1135 translated into similar calls prefixed with the field name instead of
   1136 `vector` and except for `start`, the calls also add the vector to the
   1137 table if successful, for example:
   1138 
   1139     uint8_t data[] = { 1, 2, 3 };
   1140     Monster_inventory_create(B, data, 3);
   1141     Monster_breadcrumbs_slice(B, some_other_breadcrumbs, 0, 10);
   1142 
   1143 Vector operations that are allowed between `vec_start` and
   1144 `vec_end(_pe)` are also mapped. These are
   1145 `vec_extend/append/truncate/edit/reserved_len`, and `push/push_create/push_copy`.
   1146 `push_copy` ensures only valid fields are copied, not zero padding (or
   1147 the unofficial deprecated fields).
   1148 
   1149 A struct `push_clone` is the same as a `push_copy` operation
   1150 because structs are stored inline in vectors - with the
   1151 exception of union vectors which have `push_clone` that does the
   1152 right thing.
   1153 
   1154 The `add` call adds a vector created independently from the table field,
   1155 and this is what is going on under the surface in the other calls:
   1156 
   1157     Vec3_t x;
   1158     Vec3_vec_ref_t inv;
   1159 
   1160     /* Clear any padding in `x` because it is not allocated by builder. */
   1161     Vec3_assign(Vec3_clear(&x), 3, 4, 5);
   1162     Vec3_vec_start(B);
   1163     Vec3_vec_push_create(B, 1, 2, 3);
   1164     Vec3_vec_push(B, &v);
   1165     inv = Vec3_vec_end(B);
   1166 
   1167     Monster_breadcrumbs_add(B, inv);
   1168 
   1169 As always, a reference such as `inv` may only be used at most once, and
   1170 should be used once to avoid garbage.
   1171 
   1172 Note that `Vec3_vec_start` would create an independent struct instead of a
   1173 vector of structs. Also note that `vec_ref_t` is a builder specific
   1174 temporary type while `vec_t` is intended as a const pointer to the first
   1175 element in an existing buffer in little endian encoding with a size
   1176 prefix (to be used with clone, for example).
   1177 
   1178 An existing Vec3 struct can also be pushed with `Vec3_push(B, &v)`. The
   1179 argument must be zero padded. Because vectors are converted at the end,
   1180 there is no `push_pe`, but a struct may be in little endian using push
   1181 on all platforms if `vec_end_pe` is used at the end.
   1182 
   1183 A vector may also be created from an existing array:
   1184 
   1185     uint8_t data[] = { 1, 2, 3 };
   1186     Monster_inventory_add(B, flatbuffers_uint8_vec_create(B, data, 3));
   1187 
   1188 This also applies to arrays of structs as long as they are properly zero
   1189 padded. `create_pe` is similar but does not do any endian conversion,
   1190 and is similar to `clone` except there are no header prefix.
   1191 
   1192 Likewise an existing vector with proper zero padding may be appended
   1193 using the `extend` operation. The format must be native or little endian
   1194 depending on whether `vec_end` or `vec_end_pe` is called at the end.
   1195 
   1196 All vectors are converted to little endian when the `end` command is
   1197 called. `end_pe` prevents this from happening.
   1198 
   1199 `clone` and `slice` and can be used to copy an entire, or a partial
   1200 array from an existing buffer. The pointer must be to the first vector
   1201 element in little endian format, and it must have a size prefix and be
   1202 aligned (like any flatbuffer vector). `slice` takes a base-0 index and
   1203 a vector length where the result is truncated if the source is not
   1204 large enough.
   1205 
   1206     Monster_inventory_clone(B, v);
   1207 
   1208 or
   1209 
   1210     Monster_inventory_add(flatbuffers_int8_clone(B, v);
   1211 
   1212 or
   1213 
   1214     Monster_inventory_add(flatbuffers_int8_slice(B, v, 2, 4);
   1215 
   1216 or
   1217 
   1218     Monster_inventory_slice(B, v, 2, 4);
   1219 
   1220 A vector of strings an be constructed as (`friends` is a string
   1221 vector field that we just invented for the occasion):
   1222 
   1223     flatbuffers_string_ref_t friend, *p;
   1224     Monster_friends_start(B);
   1225       friend = flatbuffer_string_create_str(B, "Peter Pan");
   1226       Monster_friends_push_create_str(B, "Shrek");
   1227       Monster_friends_push_create_str(B, "Pinnochio");
   1228       Monster_friends_push_create_str(B, "Pinnochio");
   1229       Monster_friends_push_create(B, "Hector", 6);
   1230       Monster_friends_push(friend);
   1231       p = Monster_friends_extend(B, 1);
   1232       *p = flatbuffers_string_create_str("Cindarella");
   1233       Monster_friends_push_start(B);
   1234         flatbuffers_string_append("The Little");
   1235         flatbuffers_string_append("Mermaid");
   1236       Monster_friends_push_end(B);
   1237     Monster_friends_end(B);
   1238 
   1239 Vectors and strings have a second argument to start, see also the `spawn` example
   1240 below.
   1241 
   1242 Finally, vectors can contain tables. Table vectors are offset
   1243 vectors just like string vectors. `push_start` pushes a new table and
   1244 allows for updates until `push_end`. If we have a spawn vector of monsters in
   1245 the Monster table, we can populate it like this:
   1246 
   1247     Monster_spawn_start(B);
   1248       Monster_vec_push_start(B);
   1249         Monster_Hp_add(B, 27);
   1250       Monster_vec_push_end(B);
   1251       Monster_vec_push_create(B,
   1252         /* Approximate argument list for illustration only. */
   1253         &vec, 150, 80, name, inventory, Color_Red, Any_as_None());
   1254     Monster_spawn_end(B);
   1255 
   1256 The push operation has constructors `push_start/end/create` for both tables
   1257 struct, and string elements. String elements also have
   1258 `push_create_str/create_strn/clone/slice`. Structs also have
   1259 `push_copy`. Between `push_start` and
   1260 `push_end` the operations valid for the given table or string element can be
   1261 used (typically `add` for tables, and `append` for strings).
   1262 
   1263 Instead of `Monster_vec_push_start` we can also uses
   1264 `Monster_spawn_push_start` etc. - in this case the child type is the
   1265 same as the parent, but using the field specific `push_start` ensures we
   1266 get the right table element type.
   1267 
   1268 `Monster_spawn_push_start(B)` takes no length argument because it is a
   1269 table element, while `Monster_friends_push_start(B)` because it is a
   1270 string element (similar to a vector).
   1271 
   1272 `Monster_spawn_start(B)` should just be followed by push operations
   1273 rather than following up with `Monster_spawn_extend(B, n)` because we
   1274 risk loose references that can lead to crashes. But handled carefully
   1275 it is possible:
   1276 
   1277     Monster_vec_ref_t mvec;
   1278     Monster_spawn_start(B);
   1279     mvec = Monster_spawn_extend(B, 2);
   1280     mvec[0] = Monster_create(B, ...);
   1281     mvec[1] = Monster_create(B, ...);
   1282     Monster_spawn_end(B);
   1283 
   1284 We can also push a reference to an independently create monster table,
   1285 all as seen before with strings.
   1286 
   1287 As of flatcc version 0.5.2 it is also possible to clone tables.
   1288 Therefore we also have `push_clone` on vectors of tables.
   1289 
   1290 While the use of `extend` and `truncate` is possible with vectors of
   1291 strings and tables, they should be used with care because the elements
   1292 are references and will just end up as garbage if truncated. On the
   1293 other hand, unused elements should be truncated as 0 elements in an
   1294 offset vector is not valid.
   1295 
   1296 A vector of tables or strings can be created using an externally built
   1297 array of references, for example:
   1298 
   1299     Monster_ref_t monsters[20];
   1300     Monster_vec_ref_t mvec;
   1301     monsters[0] = Monster_create(B, ...);
   1302     ...
   1303     mvec = Monster_vec_create(B, monsters, 20);
   1304 
   1305 By convention, create calls bypass the internal stack when the endian
   1306 format is otherwise compatible, and thus feed the emitter directly.
   1307 This is not possible with table and string vectors because the
   1308 references in the source vectors must be translated into offsets.
   1309 Therefore these create calls are similar to start, append, end calls.
   1310 There is an internal, but unexposed `flatcc_builder` version
   1311 `create_offset_vector_direct` which destroys the source vector instead
   1312 of allocating a stack copy.
   1313 
   1314 ## Unions
   1315 
   1316 Unlike the C++ Flatbuffers library, we do not expose a separate union
   1317 type field except via a small struct with a union of typed references
   1318 and a type field. This struct is given to the create argument, and above
   1319 it is zero initialized meaning default None.
   1320 
   1321 Unions can be created with value specific `start/end/create` calls. The add
   1322 call is not specialized since it takes a union reference:
   1323 
   1324 
   1325     Monster_test_Weapon_start(B);
   1326     Weapon_rounds_add(B, 50);
   1327     Monster_test_Weapon_end(B);
   1328 
   1329 or
   1330 
   1331     Monster_test_Weapon_create(B, 50);
   1332 
   1333 or
   1334 
   1335     Monster_test_Weapon_add(B, Weapon_create(B, 50));
   1336 
   1337 or
   1338 
   1339     Monster_test_Pickup_start(B);
   1340     Pickup_location_create(B, 0, 0, 17);
   1341     Pickup_hint_create_str(B, "Jump High!");
   1342     Monster_test_Pickup_end(B);
   1343 
   1344 or
   1345 
   1346     Pickup_ref_t test;
   1347     Pickup_start(B);
   1348     Pickup_location_create(B, 0, 0, 17);
   1349     test = Pickup_end(B);
   1350     Monster_test_add(B, Any_as_Pickup(test));
   1351 
   1352 or
   1353 
   1354     Any_union_ref_t test;
   1355     Pickup_start(B);
   1356     Pickup_location_create(B, 0, 0, 17);
   1357     /* test.Pickup = Pickup_end(B); no longer possible as of v0.5.0 */
   1358     test.value = Pickup_end(B); /* As of v0.5.1. */
   1359     test.type = Any_Pickup;
   1360     Monster_test_add(B, test);
   1361 
   1362 The following is valid and will not return an error, but also has no effect:
   1363 
   1364     Monster_test_add(B, Any_as_NONE());
   1365 
   1366 
   1367 _Note: the union structure has been changed for v0.5.0, and v0.5.1.
   1368 Both unions and union vectors are now represented by a struct with the
   1369 fields { type, value } in the low level interfaces. Before 0.5.0 only
   1370 unions of tables were supported._
   1371 
   1372 
   1373 ### Union Vectors
   1374 
   1375 The `monster_test.fbs` schema has a field named manyany in the Monster
   1376 table. It is vector of unions of type Any.
   1377 
   1378 We can create a vector using
   1379 
   1380     Any_union_vec_ref_t anyvec_ref;
   1381 
   1382     Any_vec_start(B);
   1383     Any_vec_push(TestSimpleTableWithEnum_create(B));
   1384     anyvec_ref = Any_vec_end(B);
   1385     Monster_manyany_add(anyvec_ref);
   1386 
   1387 A union can be constructed with type specific `_push` or `_push_create` operations:
   1388 
   1389     Monster_manyany_start(B);
   1390     Monster_manyany_push(B, Any_as_TestSimpleTableWithEnum(ref));
   1391     Monster_manyany_end(B);
   1392 
   1393     Monster_manyany_start(B);
   1394     Monster_manyany_TestSimpleTableWithEnum_push(B, ref);
   1395     Monster_manyany_end(B);
   1396 
   1397     Monster_manyany_start(B);
   1398     Monster_manyany_TestSimpleTableWithEnum_push_create(B, args);
   1399     Monster_manyany_end(B);
   1400 
   1401 and other similar operations, much like other vectors.
   1402 
   1403 Note that internally `anyvec_ref` is really two references, one to type
   1404 vector and one to a table vector. The vector is constructed a single
   1405 vector of unions and later split into two before final storage. If it is
   1406 necessary to create a union vector from a vector of tables and types,
   1407 the low level builder interface has a `direct` call to do this.
   1408 
   1409 Union vectos generally use more temporary stack space because during
   1410 construction because each element as a struct of type and reference
   1411 which don't back as densely as a two separate tables. In addition the
   1412 separated type and table vectors must be constructed temporarily. The
   1413 finaly buffer result is resonably compatct since the type vector does
   1414 not use much space. Unions will also be somewhat slower to construct,
   1415 but not unreasonably so.
   1416 
   1417 
   1418 ### Unions of Strings and Structs
   1419 
   1420 _Note: as of v0.5.0 unions can also contain strings and structs in
   1421 addition to tables. Support for these types in other languages may vary,
   1422 but C++ does support them too._
   1423 
   1424 All union values are stored by reference. Structs that are not unions
   1425 are stored inline in tables and cannot be shared but unions of struct
   1426 type are stored by reference and can be shared. A union value is
   1427 therefore always a reference. This is mostly transparent because the
   1428 generated table field methods has `create/start/end` calls for each union
   1429 value type and addition to `add`.
   1430 
   1431 To illustrate the use of these variation we use the Movie table from
   1432 [monster_test.fbs]:
   1433 
   1434     namespace Fantasy;
   1435 
   1436     table Attacker {
   1437         sword_attack_damage: int;
   1438     }
   1439 
   1440     struct Rapunzel {
   1441         hair_length: uint16;
   1442     }
   1443 
   1444     struct BookReader {
   1445         books_read: int;
   1446     }
   1447 
   1448     union Character {
   1449         MuLan: Attacker = 2,  // Can have name be different from type.
   1450         Rapunzel = 8,         // Or just both the same, as before.
   1451         Belle: Fantasy.BookReader,
   1452         BookFan: BookReader,
   1453         Other: string,
   1454         Unused: string = 255
   1455     }
   1456 
   1457     table Movie {
   1458         main_character: Character;
   1459         antagonist: Character;
   1460         side_kick: Character;
   1461         cameo: Character;
   1462         characters: [Character];
   1463     }
   1464 
   1465 
   1466 and the mixed type test case from [monster_test.c]:
   1467 
   1468 
   1469     nsf(Character_union_ref_t) ut;
   1470     nsf(Rapunzel_ref_t) cameo_ref;
   1471     nsf(Attacker_ref_t) attacker_ref;
   1472     nsf(BookReader_ref_t) br_ref;
   1473     nsf(BookReader_t *) pbr;
   1474     nsf(Movie_table_t) mov;
   1475 
   1476     nsf(Movie_start_as_root(B));
   1477     br_ref = nsf(BookReader_create(B, 10));
   1478     cameo_ref = nsf(Rapunzel_create(B, 22));
   1479     ut = nsf(Character_as_Rapunzel(cameo_ref));
   1480     nsf(Movie_main_character_Rapunzel_create(B, 19));
   1481     nsf(Movie_cameo_Rapunzel_add(B, cameo_ref));
   1482     attacker_ref = nsf(Attacker_create(B, 42));
   1483     nsf(Movie_antagonist_MuLan_add(B, attacker_ref));
   1484     nsf(Movie_side_kick_Other_create_str(B, "Nemo"));
   1485     nsf(Movie_characters_start(B));
   1486     nsf(Movie_characters_push(B, ut));
   1487     nsf(Movie_characters_MuLan_push(B, attacker_ref));
   1488     nsf(Movie_characters_MuLan_push_create(B, 1));
   1489     nsf(Character_vec_push(B, nsf(Character_as_Other(nsc(string_create_str(B, "other"))))));
   1490     nsf(Movie_characters_Belle_push(B, br_ref));
   1491     pbr = nsf(Movie_characters_Belle_push_start(B));
   1492     pbr->books_read = 3;
   1493     nsf(Movie_characters_Belle_push_end(B));
   1494     nsf(Movie_characters_Belle_push(B, nsf(BookReader_create(B, 1))));
   1495     nsf(Movie_characters_Belle_push_create(B, 2));
   1496     nsf(Movie_characters_Other_push(B, nsc(string_create_str(B, "another"))));
   1497     nsf(Movie_characters_Other_push_create_str(B, "yet another"));
   1498     nsf(Movie_characters_end(B));
   1499     nsf(Movie_end_as_root(B));
   1500 
   1501 Note that reading a union of string type requires a cast which can be
   1502 seen in the full test case in [monster_test.c].
   1503 ## Error Handling
   1504 
   1505 The API generally expects all error codes to be checked but the
   1506 following table and vector operations will accept and return an error:
   1507 
   1508 - `add` null reference to table, vector, or string.
   1509 - `push` null reference to table or string.
   1510 - `buffer_end/create` null reference to root.
   1511 
   1512 This can simplify pushing or adding atomically created objects, for
   1513 example by adding a cloned vector to table field.
   1514 
   1515 It is especially important to check start operations because the builder
   1516 will not be in the expected stack frame context after failure and will
   1517 not have reserved necessary internal memory, for example when adding a
   1518 table field.
   1519 
   1520 On a server with reasonable amount of memory using the default
   1521 allocator, and with an emitter that will not return errors, and when it
   1522 can be expected that inputs will not exceed the size contraints of the
   1523 flatbuffer data types, and if the api is being used correctly, then there
   1524 are no reason for failure and error handling may be skipped. However,
   1525 it is sometimes desireable for servers to restrict a single clients
   1526 memory usage, and then errors are very likely unless the source data is
   1527 already limited. As an opposite example, an embedded device sending
   1528 small network packages using a fixed but large enough allocation pool,
   1529 would be in total control and need not be concerned with any errors.
   1530 
   1531 
   1532 
   1533 ## Type System Overview
   1534 
   1535 The generated methods for building buffers may look the same but
   1536 have different semantics. For example `_clone` on a table field
   1537 such as `Monster_enemy_clone` will actually create a table based
   1538 on the content of a table in a another buffer, then add that
   1539 table to the currently open table. But `Monster_clone` will
   1540 create clone and just return a reference without adding the
   1541 reference to any table. There is also `push_clone` which adds
   1542 an element to an open vector. The same applies to many other
   1543 operations.
   1544 
   1545 Basically there are
   1546 the following different types of methods:
   1547 
   1548 - Methods on native flatbuffer types, such as
   1549   `flatbuffer_string_start`.
   1550 - Methods on generated types types such as `Monster_start`
   1551 - Methods on field members such as as `Monster_emeny_start`
   1552 - Methods on vectors on vectors of the above such as
   1553   `flatbuffers_string_vec_start`, `Monster_vec_start`.
   1554   `Monster_inventory_vec_start`.
   1555 - Slight adaptions for buffer roots and nested buffer roots.
   1556 
   1557 For unions and union vectors the story is more complex - and the
   1558 api might need to be cleaned up further, but generally there are
   1559 both union type fields, union value fields, and union fields
   1560 representing both, and vectors of the same. In additional there
   1561 are pseudo fields for each union member because `create` on a
   1562 union does not make sense, but
   1563 `Monster_myvariant_MyTable_create` does create and `MyTable`
   1564 table and assigns it with the correct type to the field
   1565 `Monster_myvariant_type` and `Monster_myvariant.
   1566 
   1567 
   1568 ## Cloning
   1569 
   1570 As of flatcc v0.5.2 it is also possible to clone tables, unions,
   1571 vectors of tables, vectors of strings, and vectors of unions.
   1572 Previously many operations did have a clone or a `push_clone`
   1573 operator, but these were all raw byte copies. Table cloning and
   1574 union cloning is signficantly more involved as it a simple copy
   1575 will not work due to stored references, possible sharing of
   1576 references and because the required alignment of table is hard
   1577 to reason about without building a new table. Unions and union
   1578 vectors are even more difficult.
   1579 
   1580 That said, cloning is now implemented for all relevant data
   1581 types.
   1582 
   1583 All clone operations expect the content to originate from
   1584 another finalized buffer. For scalars and structs there are
   1585 copy operations that are almost the same as clone - they both
   1586 avoid endian conversion.
   1587 
   1588 Structs have a special case with clone and copy: Whenever a
   1589 struct is stored inline in the desitination buffer, it behaves
   1590 like copy. Whenever the destination is a buffer root, or a union
   1591 member, the result is a reference to an independent memory
   1592 block. When calling clone on a struct type the destination is
   1593 unknown and a indendpendent reference is created. If this is not
   1594 the intention a `copy` operation can be used. When used field
   1595 methods the destination type is known at the right thing will
   1596 happen.
   1597 
   1598 Cloning a table will, by default, expand any shared references
   1599 in the source into separate copies. This is also true when
   1600 cloning string vectors, or any other data that holds references.
   1601 Worst case this can blow up memory (which is also true when
   1602 printing JSON from a buffer).
   1603 
   1604 It is possible to preserve the exact DAG structure when cloning.
   1605 It may not worthwhile for simple use cases but it goes as
   1606 follows:
   1607 
   1608 The builder has a pointer to a `flatcc_refmap_t` object. This is
   1609 a fairly small stack allocated object that implements a
   1610 hashtable. By default this pointer is null, and we have the
   1611 above mentioned expansion. If it is not null, each newly cloned
   1612 object will have its reference stored in the refmap. The next
   1613 time the same object is cloned, the existing reference will be
   1614 taken from the refmap instead. See source comments in
   1615 `flatcc_refmap.h` and `flatcc_builder.h`, and `monster_test.c`
   1616 clone tests.
   1617 
   1618 Note that, for example, it might be relevant to preserve DAG
   1619 structure when cloning one object with all its sub-objects, but
   1620 if it is cloned a second time, a new copy is desired still while
   1621 preseving the inner DAG structure. This can be done by working
   1622 with multiple refmaps and simple swapping them out via
   1623 `flatcc_builder_set_refmap`. It is also possible to add
   1624 references manually to a refmap before cloning.
   1625 
   1626 Warning: the refmap MUST not hold any foreign references when
   1627 starting a nested root clone or when cloning inside a nested
   1628 buffer that has been started but not ended because it is
   1629 invalid to share references between buffers and there are no
   1630 safety checks for this.
   1631 
   1632 
   1633 ## Picking
   1634 
   1635 Picking is a method that is related to clone and also introduced
   1636 with flatcc 0.5.2. A pick method is only defined on a table
   1637 field or a struct field. Instead of taking an a read reference
   1638 of same type as the field, it takes a reference to to the same
   1639 container type (table or struct). Essentially pick means: find
   1640 myself in the other table, clone me, and and me to the new table
   1641 which is currently open. So clone takes an entire table where
   1642 pick takes a single field. Table cloning is implemented as a
   1643 sequence of pick method, one for each field as can be seen in
   1644 the generated builder source. A pick operation does nothting if
   1645 the field is not set. Pick also works with refmaps because it
   1646 does an internal clone operation.  In the generated code, only
   1647 clone on types will use the refmap but other clone and pick
   1648 operations do depend on these type clone methods.
   1649 
   1650 
   1651 ## Sorting Vectors
   1652 
   1653 Vectors can be sorted, but not by the primary builder interface because:
   1654 
   1655 String and table elements cannot be accessed after they have been
   1656 emitted. The emitter can do all sorts of async operations other than
   1657 actually building a buffer, for example encrypting blocks and / or send
   1658 partial buffers over the network. Scalars could be sorted, but the most
   1659 efficient way of emitting vectors does not create a temporary vector but
   1660 emits the source directly when endianess allows for it. Less
   1661 significant, the buffer producer is likely busy processing content and /
   1662 or on a resource constrained device. Altogether, it is much simpler to
   1663 not support sorting at this interface level.
   1664 
   1665 To understand how sorting is implemented, lets first look at how an
   1666 already sorted vector can be searched:
   1667 
   1668 Every vector of string, scalar and enum element types have a `find`
   1669 operation in the reader interface that performs a binary seach. Every
   1670 vector of table and struct elements have a `find_by_<field_name>` iff
   1671 there is a key attribute on at least one top-level scalar, enum or
   1672 string field type. FlatBuffers do not officially allow for multiple key
   1673 attributes, but if enabled, there will by a `find_by` operation for
   1674 every keyed element field. In addition there is a `find` operation that
   1675 maps to the first keyed field.
   1676 
   1677 The read interface returns a vector type, which is a const pointer, when
   1678 accessing a table field of vector type. The find operation takes such a
   1679 vector as first argument, and a key as second. Strings have variations
   1680 to allow for keys with a given length (similar to strcmp vs strncmp).
   1681 
   1682 This leads us to the sort interface:
   1683 
   1684 Every `find` and `find_by` operation has a matching `sort` and `sort_by`
   1685 operation table and struct vectors maps `sort` to the first keyed
   1686 `sort_by` operation. The sort operation takes a non-const vector which
   1687 has the type name suffix `_mutable_vec_t`. These
   1688 vectors are not available via the reader interface and must be cast
   1689 explicitly from `_vec_t` to `_mutable_vec_t`. When this is done, the
   1690 vector can be sorted in-place in the buffer without any memory
   1691 allocation and without any recursion.
   1692 
   1693 
   1694 
   1695 If the namespace is
   1696 `flatbuffers`, a string vector is sorted by:
   1697 
   1698     flatbuffers_string_vec_t vec;
   1699     vec = ...;
   1700     `flatbuffers_string_vec_sort((flatbuffers_string_mutable_vec_t)vec)`
   1701 
   1702 Scalar and enum vectors have similar inline sort operations, for
   1703 example:
   1704 
   1705     flatbuffers_uint8_vec_sort(flatbuffer_uint8_mutable_vec_t vec);
   1706 
   1707 For vectors of tables or structs the sort function is named by the key
   1708 field. Assuming the Monster table has a key attribute on the `Hp` field,
   1709 the following sort operation is available:
   1710 
   1711     MyGame_Example_Monster_vec_t monsters;
   1712     monsters = ...;
   1713     MyGame_Example_Monster_vec_sort_by_Hp(
   1714         (MyGame_Example_Monster_mutable_vec_t)monsters);
   1715 
   1716 Note: this is the reader interface. Any kind of `ref_t` type used by the
   1717 builder do not apply here. (Advanced: if an emitter builds a buffer, the
   1718 ref type can be used to find the actual vector pointer and then it can
   1719 be sorted by casting the pointer to a vector, even if the buffer isn't
   1720 finished).
   1721 
   1722 Multiple keys per table or struct is an optional feature. Each key will
   1723 have its own sort and find function similar to the above. The first key
   1724 also has the shortcut:
   1725 
   1726     MyGame_Example_Monster_vec_sort(m);
   1727 
   1728 The current implementation uses heap sort which is nearly as fast as
   1729 quicksort and has a compact implementation that does not require
   1730 recursion or external memory and is robust against DOS attacks by having
   1731 worst case O(n log n). It is, however, not a stable sort. The sort
   1732 assumes struct have a reasonable size so swap operations can be done
   1733 efficiently. For large structs a decicated sort operation building an
   1734 external index vector would be better, but this is not supported.
   1735 
   1736 Note that a DAG is valid so there can be multiple vectors referring to
   1737 the same table elements, and each can be sorted by a different key.
   1738 
   1739 The find operations are stable meaning they always return the lowest
   1740 index of any matching key or `flatbuffers_not_found` which is larger
   1741 than any other index.
   1742 
   1743 ### Dangers of Sorting
   1744 
   1745 If a buffer was received over, say, an untrusted network the buffer
   1746 should be verified before being accessed. But verification only makes it
   1747 safe to read a buffer, not to modify a buffer because for example two
   1748 vectors can be crafted to overlap each other without breaking any
   1749 verification rules.
   1750 
   1751 Thus, sorting is intended to be done shortly after the buffer is
   1752 constructed while it can still be trusted.
   1753 
   1754 Using find on a buffer that is supposed to be sorted, but isn't, can
   1755 yield unexpected search results, but the result will always be a one
   1756 element in the vector being searched, not a buffer overrun.
   1757 
   1758 ### Scanning
   1759 
   1760 Some vectors can be sorted by different keys depending on which version
   1761 version of `_sort_by` is being used. Obviously `_find_by` must match the
   1762 sorted key.
   1763 
   1764 If we need to search for a key that is not sorted, or if we simply do
   1765 not want to sort the vector, it is possible to use scanning operations
   1766 instead by using `_scan` or `_scan_by`. Scanning is similar to find
   1767 except that it does a linear search and it supports scanning from a
   1768 given position.
   1769 
   1770 More information on scanning in the
   1771 [README](https://github.com/dvidelabs/flatcc#searching-and-sorting)
   1772 file, and in the [monster_test.c] test file.
   1773 
   1774 
   1775 ## Example of different interface type users
   1776 
   1777 A resource constrained microcontroller is building flatbuffers from
   1778 sensor data using an emitter that sends UDP packages of the flatbuffer
   1779 as soon as enough data is ready. A server reassembles the packages or
   1780 discards them if any UDP package was lost. One the package is assembled,
   1781 the server sorts specific vectors such as temparture levels in the buffer
   1782 before it sends the buffer upstream to a storage service through a
   1783 TCP/IP connection. The analyzers perform taks such as detecting
   1784 abnormal temparature readings based on the sorted vector data.
   1785 
   1786 In the above example, the original sensor devices are not interested in
   1787 the reader interface nor the sort interface. While the sort and find
   1788 operations may be available, it is dead inline code that does not
   1789 inflate the binary codes image size, but the libflatccrt library is
   1790 linked in. The collecting server is not interested in the builder
   1791 interface and does not link with the `libflatccrt` library but uses
   1792 both the inline functions of the reader intrface and the sort interface.
   1793 The upstream data storage service uses no interface at all since it
   1794 treats the buffers as binary blobs in a database indexed by device and
   1795 time. The end users only use the read only interface to visualize and
   1796 analyze and has no need for the builder or the sort interface.
   1797 
   1798 
   1799 ## Special Emitters
   1800 
   1801 An emitter only need to implement one function to replace or wrap the
   1802 default emitter. See [flatcc_builder.h] on `flatcc_builder_emit_fun` for
   1803 details, and also `emit_test.c` for a very simple custom emitter that
   1804 just prints debug messages, and [flatcc_emitter.h].
   1805 
   1806 When adding padding `flatcc_builder_padding_base` is used as base in iov
   1807 entries and an emitter may detect this pointer and assume the entire
   1808 content is just nulls. Usually padding is of limited size by its very
   1809 nature so the benefit of handling this is also limited, but it, or a
   1810 similar user provided constants can be used for similar purposes:
   1811 
   1812 When creating a vector in a single operation from an external C-array,
   1813 no copying takes place on the internal builder stack. Therefore it is
   1814 valid to provide a null pointer or a valid array such as
   1815 `flatcc_builder_padding_base` that is is too small for the given length,
   1816 provided that the emitter is aware of it. This in turn can be used to
   1817 allocate space in the emitters internal datastructure so the vector can
   1818 be filled after the fact if so desired. Pointer tagging may be another
   1819 way to communicate special intent. Be aware that only `create` calls
   1820 support this - any `append`, `start/end` or other dynamic operation will
   1821 require valid inpout and will stack allocate temporary space.
   1822 
   1823 Emitters always receive a small table of iov entries that together form
   1824 a single object including necessary headers and padding, for example a
   1825 vector, a string, a nested buffer header, or a vtable. This is
   1826 guaranteed by the api, but there is no coordination to provide details
   1827 about which call is in order to keep the interface simple and fast. If
   1828 this is desired the user must hint the emitter out of band before
   1829 calling the relevant build operation. This can also be one indirectly by
   1830 setting `user_state` in the emitter and have the emitter inspect this
   1831 setting.
   1832 
   1833 When adding vectors piecemeal using `append` or similar as opposed to
   1834 zero or less than zero copy approach above, the memory cost is obviously
   1835 higher, but unless the individual objects grow large, the stack will
   1836 operate in hot cpu cache so the bandwidth from main memory to cpu and
   1837 back will not necessarily double. If the stack grows large it may also
   1838 be worthwhile trimming the stack with a custom allocator and custom
   1839 builder reset between buffers to reduce stack size and initialization
   1840 overhead.
   1841 
   1842 [monster_test.c]: https://github.com/dvidelabs/flatcc/blob/master/test/monster_test/monster_test.c
   1843 [flatcc_builder.h]: https://github.com/dvidelabs/flatcc/blob/master/include/flatcc/flatcc_builder.h
   1844 [flatcc_emitter.h]: https://github.com/dvidelabs/flatcc/blob/master/include/flatcc/flatcc_emitter.h
   1845 [monster_test.fbs]: https://github.com/dvidelabs/flatcc/blob/master/test/monster_test/monster_test.fbs