Clarified binary encoding with an example.

Change-Id: I60c900a2f7cbd88fe264f7b0dc78ef86b1580655
Tested: in Chrome.

docs/html/md__internals.html

@@ -75,12 +75,13 @@ $(document).ready(function(){initNavTree('md__internals.html','');});
 <h3>Structs</h3>
 <p>These are the simplest, and as mentioned, intended for simple data that benefits from being extra efficient and doesn't need versioning / extensibility. They are always stored inline in their parent (a struct, table, or vector) for maximum compactness. Structs define a consistent memory layout where all components are aligned to their size, and structs aligned to their largest scalar member. This is done independent of the alignment rules of the underlying compiler to guarantee a cross platform compatible layout. This layout is then enforced in the generated code.</p>
 <h3>Tables</h3>
-<p>These start with an <code>soffset_t</code> to a vtable. This is a signed version of <code>uoffset_t</code>, since vtables may be stored anywhere relative to the object. This offset is substracted (not added) from the object start to arrive at the vtable start. This offset is followed by all the fields as aligned scalars (or offsets). Unlike structs, not all fields need to be present. There is no set order and layout.</p>
+<p>Unlike structs, these are not stored in inline in their parent, but are referred to by offset.</p>
+<p>They start with an <code>soffset_t</code> to a vtable. This is a signed version of <code>uoffset_t</code>, since vtables may be stored anywhere relative to the object. This offset is substracted (not added) from the object start to arrive at the vtable start. This offset is followed by all the fields as aligned scalars (or offsets). Unlike structs, not all fields need to be present. There is no set order and layout.</p>
 <p>To be able to access fields regardless of these uncertainties, we go through a vtable of offsets. Vtables are shared between any objects that happen to have the same vtable values.</p>
-<p>The elements of a vtable are all of type <code>voffset_t</code>, which is a <code>uint16_t</code>. The first element is the size of the vtable in bytes, including the size element. The second one is the size of the object, in bytes (including the vtable offset). This size could be used for streaming, to know how many bytes to read to be able to access all fields of the object. The remaining elements are the N offsets, where N is the amount of fields declared in the schema when the code that constructed this buffer was compiled (thus, the size of the table is N + 2).</p>
+<p>The elements of a vtable are all of type <code>voffset_t</code>, which is a <code>uint16_t</code>. The first element is the size of the vtable in bytes, including the size element. The second one is the size of the object, in bytes (including the vtable offset). This size could be used for streaming, to know how many bytes to read to be able to access all <em>inline</em> fields of the object. The remaining elements are the N offsets, where N is the amount of fields declared in the schema when the code that constructed this buffer was compiled (thus, the size of the table is N + 2).</p>
 <p>All accessor functions in the generated code for tables contain the offset into this table as a constant. This offset is checked against the first field (the number of elements), to protect against newer code reading older data. If this offset is out of range, or the vtable entry is 0, that means the field is not present in this object, and the default value is return. Otherwise, the entry is used as offset to the field to be read.</p>
 <h3>Strings and Vectors</h3>
-<p>Strings are simply a vector of bytes, and are always null-terminated. Vectors are stored as contiguous aligned scalar elements prefixed by a 32bit element count (not including any null termination).</p>
+<p>Strings are simply a vector of bytes, and are always null-terminated. Vectors are stored as contiguous aligned scalar elements prefixed by a 32bit element count (not including any null termination). Neither is stored inline in their parent, but are referred to by offset.</p>
 <h3>Construction</h3>
 <p>The current implementation constructs these buffers backwards (starting at the highest memory address of the buffer), since that significantly reduces the amount of bookkeeping and simplifies the construction API.</p>
 <h3>Code example</h3>
@@ -162,7 +163,29 @@ STRUCT_END(Vec3, 12);
 }
 </pre><p><code>CreateMonster</code> is a convenience function that calls all functions in <code>MonsterBuilder</code> above for you. Note that if you pass values which are defaults as arguments, it will not actually construct that field, so you can probably use this function instead of the builder class in almost all cases. </p><pre class="fragment">inline const Monster *GetMonster(const void *buf) { return flatbuffers::GetRoot&lt;Monster&gt;(buf); }
 </pre><p>This function is only generated for the root table type, to be able to start traversing a FlatBuffer from a raw buffer pointer. </p><pre class="fragment">}; // namespace MyGame
-}; // namespace Sample</pre> </div></div><!-- contents -->
+}; // namespace Sample
+</pre><h3>Encoding example.</h3>
+<p>Below is a sample encoding for the following JSON corresponding to the above schema: </p><pre class="fragment">{ pos: { x: 1, y: 2, z: 3 }, name: "fred", hp: 50 }
+</pre><p>Resulting in this binary buffer: </p><pre class="fragment">// Start of the buffer:
+uint32_t 20  // Offset to the root table.
+
+// Start of the vtable. Not shared in this example, but could be:
+uint16_t 16 // Size of table, starting from here.
+uint16_t 22 // Size of object inline data.
+uint16_t 4, 0, 20, 16, 0, 0  // Offsets to fields from start of (root) table, 0 for not present.
+
+// Start of the root table:
+int32_t 16     // Offset to vtable used (default negative direction)
+float 1, 2, 3  // the Vec3 struct, inline.
+uint32_t 8     // Offset to the name string.
+int16_t 50     // hp field.
+int16_t 0      // Padding for alignment.
+
+// Start of name string:
+uint32_t 4  // Length of string.
+int8_t 'f', 'r', 'e', 'd', 0, 0, 0, 0  // Text + 0 termination + padding.
+</pre><p>Note that this not the only possible encoding, since the writer has some flexibility in which of the children of root object to write first (though in this case there's only one string), and what order to write the fields in. Different orders may also cause different alignments to happen. </p>
+</div></div><!-- contents -->
 </div><!-- doc-content -->
 <!-- Google Analytics -->
 <script>

docs/source/Internals.md

@@ -73,7 +73,10 @@ code.
 
 ### Tables
 
-These start with an `soffset_t` to a vtable. This is a signed version of
+Unlike structs, these are not stored in inline in their parent, but are
+referred to by offset.
+
+They start with an `soffset_t` to a vtable. This is a signed version of
 `uoffset_t`, since vtables may be stored anywhere relative to the object.
 This offset is substracted (not added) from the object start to arrive at
 the vtable start. This offset is followed by all the
@@ -88,7 +91,7 @@ The elements of a vtable are all of type `voffset_t`, which is
 a `uint16_t`. The first element is the size of the vtable in bytes,
 including the size element. The second one is the size of the object, in bytes
 (including the vtable offset). This size could be used for streaming, to know
-how many bytes to read to be able to access all fields of the object.
+how many bytes to read to be able to access all *inline* fields of the object.
 The remaining elements are the N offsets, where N is the amount of fields
 declared in the schema when the code that constructed this buffer was
 compiled (thus, the size of the table is N + 2).
@@ -106,7 +109,8 @@ field to be read.
 Strings are simply a vector of bytes, and are always
 null-terminated. Vectors are stored as contiguous aligned scalar
 elements prefixed by a 32bit element count (not including any
-null termination).
+null termination). Neither is stored inline in their parent, but are referred to
+by offset.
 
 ### Construction
 
@@ -249,4 +253,35 @@ start traversing a FlatBuffer from a raw buffer pointer.
     }; // namespace MyGame
     }; // namespace Sample
 
+### Encoding example.
 
+Below is a sample encoding for the following JSON corresponding to the above
+schema:
+
+    { pos: { x: 1, y: 2, z: 3 }, name: "fred", hp: 50 }
+
+Resulting in this binary buffer:
+
+    // Start of the buffer:
+    uint32_t 20  // Offset to the root table.
+
+    // Start of the vtable. Not shared in this example, but could be:
+    uint16_t 16 // Size of table, starting from here.
+    uint16_t 22 // Size of object inline data.
+    uint16_t 4, 0, 20, 16, 0, 0  // Offsets to fields from start of (root) table, 0 for not present.
+
+    // Start of the root table:
+    int32_t 16     // Offset to vtable used (default negative direction)
+    float 1, 2, 3  // the Vec3 struct, inline.
+    uint32_t 8     // Offset to the name string.
+    int16_t 50     // hp field.
+    int16_t 0      // Padding for alignment.
+
+    // Start of name string:
+    uint32_t 4  // Length of string.
+    int8_t 'f', 'r', 'e', 'd', 0, 0, 0, 0  // Text + 0 termination + padding.
+
+Note that this not the only possible encoding, since the writer has some
+flexibility in which of the children of root object to write first (though in
+this case there's only one string), and what order to write the fields in.
+Different orders may also cause different alignments to happen.