The format also doesn't contain information for format identification and versioning, which is also by design. FlatBuffers is a statically typed system, meaning the user of a buffer needs to know what kind of buffer it is. FlatBuffers can of course be wrapped inside other containers where needed, or you can use its union feature to dynamically identify multiple possible sub-objects stored. Additionally, it can be used together with the schema parser if full reflective capabilities are desired.
Versioning is something that is intrinsically part of the format (the optionality / extensibility of fields), so the format itself does not need a version number (it's a meta-format, in a sense). We're hoping that this format can accommodate all data needed. If format breaking changes are ever necessary, it would become a new kind of format rather than just a variation.
The most important and generic offset type (see flatbuffers.h) is offset_t, which is currently always a uint32_t, and is used to refer to all tables/unions/strings/vectors. 32bit is intentional, since we want to keep the format binary compatible between 32 and 64bit systems, and a 64bit offset would bloat the size for almost all uses. A version of this format with 64bit (or 16bit) offsets is easy to set when needed. Unsigned means they can only point in one direction, which typically is forward (towards a higher memory location). Any backwards offsets will be explicitly marked as such.
The format starts with an offset_t to the root object in the buffer.
The most important and generic offset type (see flatbuffers.h) is uoffset_t, which is currently always a uint32_t, and is used to refer to all tables/unions/strings/vectors (these are never stored in-line). 32bit is intentional, since we want to keep the format binary compatible between 32 and 64bit systems, and a 64bit offset would bloat the size for almost all uses. A version of this format with 64bit (or 16bit) offsets is easy to set when needed. Unsigned means they can only point in one direction, which typically is forward (towards a higher memory location). Any backwards offsets will be explicitly marked as such.
The format starts with an uoffset_t to the root object in the buffer.
We have two kinds of objects, structs and tables.
These are the simplest, and as mentioned, intended for simple data that benefits from being extra efficient and doesn't need versioning / extensibility. They are always stored inline in their parent (a struct, table, or vector) for maximum compactness. Structs define a consistent memory layout where all components are aligned to their size, and structs aligned to their largest scalar member. This is done independent of the alignment rules of the underlying compiler to guarantee a cross platform compatible layout. This layout is then enforced in the generated code.
These start with an soffset_t to a vtable (signed version of offset_t, since vtables may be stored anywhere), followed by all the fields as aligned scalars. Unlike structs, not all fields need to be present. There is no set order and layout.
These start with an soffset_t to a vtable (signed version of uoffset_t, since vtables may be stored anywhere), followed by all the fields as aligned scalars (or offsets). Unlike structs, not all fields need to be present. There is no set order and layout.
To be able to access fields regardless of these uncertainties, we go through a vtable of offsets. Vtables are shared between any objects that happen to have the same vtable values.
-The elements of a vtable are all of type voffset_t, which is currently a uint16_t. The first element is the number of elements of the vtable, including this one. The second one is the size of the object, in bytes (including the vtable offset). This size is used for streaming, to know how many bytes to read to be able to access all fields of the object. The remaining elements are N the offsets, where N is the amount of field declared in the schema when the code that constructed this buffer was compiled (thus, the size of the table is N + 2).
The elements of a vtable are all of type voffset_t, which is a uint16_t. The first element is the number of elements of the vtable, including this one. The second one is the size of the object, in bytes (including the vtable offset). This size is used for streaming, to know how many bytes to read to be able to access all fields of the object. The remaining elements are the N offsets, where N is the amount of fields declared in the schema when the code that constructed this buffer was compiled (thus, the size of the table is N + 2).
All accessor functions in the generated code for tables contain the offset into this table as a constant. This offset is checked against the first field (the number of elements), to protect against newer code reading older data. If this offset is out of range, or the vtable entry is 0, that means the field is not present in this object, and the default value is return. Otherwise, the entry is used as offset to the field to be read.
Strings are simply a vector of bytes, and are always null-terminated. Vectors are stored as contiguous aligned scalar elements prefixed by a count.
+Strings are simply a vector of bytes, and are always null-terminated. Vectors are stored as contiguous aligned scalar elements prefixed by a 32bit element count (not including any null termination).
The current implementation constructs these buffers backwards, since that significantly reduces the amount of bookkeeping and simplifies the construction API.
+The current implementation constructs these buffers backwards (starting at the highest memory address of the buffer), since that significantly reduces the amount of bookkeeping and simplifies the construction API.
Here's an example of the code that gets generated for the samples/monster.fbs. What follows is the entire file, broken up by comments:
// automatically generated, do not modify diff --git a/docs/html/md__schemas.html b/docs/html/md__schemas.html index f4b82c6a0..52bfc7537 100644 --- a/docs/html/md__schemas.html +++ b/docs/html/md__schemas.html @@ -97,7 +97,7 @@ root_type Monster;
short ushortint uint floatlong ulong double[type]). Nesting vectors require you wrap the inner vector in a struct/table rather than writing [[type]].[type]). Nesting vectors is not supported, instead you can wrap the inner vector in a table.string, which may only hold UTF-8 or 7-bit ASCII. For other text encodings or general binary data use vectors ([byte] or [ubyte]) instead.