Port FlatBuffers to Python.

Implement code generation and self-contained runtime library for Python.

The test suite verifies:
  - Correctness of generated Python code by comparing output to that of
    the other language ports.
  - The exact bytes in the Builder buffer during many scenarios.
  - Vtable deduplication correctness.
  - Edge cases for table construction, via a fuzzer derived from the Go
    implementation.
  - All code is simultaneously valid in Python 2.6, 2.7, and 3.4.

The test suite includes benchmarks for:
  - Building 'gold' data.
  - Parsing 'gold' data.
  - Deduplicating vtables.

All tests pass on this author's system for the following Python
implementations:
  - CPython 2.6.7
  - CPython 2.7.8
  - CPython 3.4.2
  - PyPy 2.5.0 (CPython 2.7.8 compatible)

.gitignore

@@ -48,3 +48,4 @@ FlatBuffers.xcodeproj/
 java/.idea
 java/*.iml
 java/target
+**/*.pyc

CMakeLists.txt

@@ -24,6 +24,7 @@ set(FlatBuffers_Compiler_SRCS
   src/idl_gen_cpp.cpp
   src/idl_gen_general.cpp
   src/idl_gen_go.cpp
+  src/idl_gen_python.cpp
   src/idl_gen_text.cpp
   src/idl_gen_fbs.cpp
   src/flatc.cpp

docs/source/PythonUsage.md

@@ -0,0 +1,115 @@
+# Use in Python
+
+There's experimental support for reading FlatBuffers in Python. Generate
+code for Python with the `-p` option to `flatc`.
+
+See `py_test.py` for an example. You import the generated code, read a
+FlatBuffer binary file into a `bytearray`, which you pass to the
+`GetRootAsMonster` function:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    import MyGame.Example as example
+    import flatbuffers
+
+    buf = open('monster.dat', 'rb').read()
+    buf = bytearray(buf)
+    monster = example.GetRootAsMonster(buf, 0)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Now you can access values like this:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    hp = monster.Hp()
+    pos = monster.Pos()
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To access vectors you pass an extra index to the
+vector field accessor. Then a second method with the same name suffixed
+by `Length` let's you know the number of elements you can access:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    for i in xrange(monster.InventoryLength()):
+        monster.Inventory(i) # do something here
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can also construct these buffers in Python using the functions found
+in the generated code, and the FlatBufferBuilder class:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    builder = flatbuffers.NewBuilder(0)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Create strings:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    s = builder.CreateString("MyMonster")
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Create a table with a struct contained therein:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    example.MonsterStart(builder)
+    example.MonsterAddPos(builder, example.CreateVec3(builder, 1.0, 2.0, 3.0, 3.0, 4, 5, 6))
+    example.MonsterAddHp(builder, 80)
+    example.MonsterAddName(builder, str)
+    example.MonsterAddInventory(builder, inv)
+    example.MonsterAddTest_Type(builder, 1)
+    example.MonsterAddTest(builder, mon2)
+    example.MonsterAddTest4(builder, test4s)
+    mon = example.MonsterEnd(builder)
+
+    final_flatbuffer = bulder.Output()
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Unlike C++, Python does not support table creation functions like 'createMonster()'.
+This is to create the buffer without
+using temporary object allocation (since the `Vec3` is an inline component of
+`Monster`, it has to be created right where it is added, whereas the name and
+the inventory are not inline, and **must** be created outside of the table
+creation sequence).
+Structs do have convenient methods that allow you to construct them in one call.
+These also have arguments for nested structs, e.g. if a struct has a field `a`
+and a nested struct field `b` (which has fields `c` and `d`), then the arguments
+will be `a`, `c` and `d`.
+
+Vectors also use this start/end pattern to allow vectors of both scalar types
+and structs:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.py}
+    example.MonsterStartInventoryVector(builder, 5)
+    i = 4
+    while i >= 0:
+        builder.PrependByte(byte(i))
+        i -= 1
+
+    inv = builder.EndVector(5)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The generated method 'StartInventoryVector' is provided as a convenience
+function which calls 'StartVector' with the correct element size of the vector
+type which in this case is 'ubyte' or 1 byte per vector element.
+You pass the number of elements you want to write.
+You write the elements backwards since the buffer
+is being constructed back to front. Use the correct `Prepend` call for the type,
+or `PrependUOffsetT` for offsets. You then pass `inv` to the corresponding
+`Add` call when you construct the table containing it afterwards.
+
+There are `Prepend` functions for all the scalar types. You use
+`PrependUOffset` for any previously constructed objects (such as other tables,
+strings, vectors). For structs, you use the appropriate `create` function
+in-line, as shown above in the `Monster` example.
+
+Once you're done constructing a buffer, you call `Finish` with the root object
+offset (`mon` in the example above). Your data now resides in Builder.Bytes.
+Important to note is that the real data starts at the index indicated by Head(),
+for Offset() bytes (this is because the buffer is constructed backwards).
+If you wanted to read the buffer right after creating it (using
+`GetRootAsMonster` above), the second argument, instead of `0` would thus
+also be `Head()`.
+
+## Text Parsing
+
+There currently is no support for parsing text (Schema's and JSON) directly
+from Python, though you could use the C++ parser through SWIG or ctypes. Please
+see the C++ documentation for more on text parsing.
+

docs/source/doxyfile

@@ -750,6 +750,7 @@ INPUT = "FlatBuffers.md" \
         "CppUsage.md" \
         "GoUsage.md" \
         "JavaUsage.md" \
+        "PythonUsage.md" \
         "Benchmarks.md" \
         "WhitePaper.md" \
         "Internals.md" \

include/flatbuffers/idl.h

@@ -35,24 +35,24 @@ namespace flatbuffers {
 // Additionally, Parser::ParseType assumes bool..string is a contiguous range
 // of type tokens.
 #define FLATBUFFERS_GEN_TYPES_SCALAR(TD) \
-  TD(NONE,   "",       uint8_t,  byte,   byte,    byte) \
-  TD(UTYPE,  "",       uint8_t,  byte,   byte,    byte) /* begin scalar/int */ \
-  TD(BOOL,   "bool",   uint8_t,  boolean,byte,    bool) \
-  TD(CHAR,   "byte",   int8_t,   byte,   int8,    sbyte) \
-  TD(UCHAR,  "ubyte",  uint8_t,  byte,   byte,    byte) \
-  TD(SHORT,  "short",  int16_t,  short,  int16,   short) \
-  TD(USHORT, "ushort", uint16_t, short,  uint16,  ushort) \
-  TD(INT,    "int",    int32_t,  int,    int32,   int) \
-  TD(UINT,   "uint",   uint32_t, int,    uint32,  uint) \
-  TD(LONG,   "long",   int64_t,  long,   int64,   long) \
-  TD(ULONG,  "ulong",  uint64_t, long,   uint64,  ulong)  /* end int */ \
-  TD(FLOAT,  "float",  float,    float,  float32, float)  /* begin float */ \
-  TD(DOUBLE, "double", double,   double, float64, double) /* end float/scalar */
+  TD(NONE,   "",       uint8_t,  byte,   byte,    byte,   uint8) \
+  TD(UTYPE,  "",       uint8_t,  byte,   byte,    byte,   uint8) /* begin scalar/int */ \
+  TD(BOOL,   "bool",   uint8_t,  boolean,byte,    bool,   bool) \
+  TD(CHAR,   "byte",   int8_t,   byte,   int8,    sbyte,  int8) \
+  TD(UCHAR,  "ubyte",  uint8_t,  byte,   byte,    byte,   uint8) \
+  TD(SHORT,  "short",  int16_t,  short,  int16,   short,  int16) \
+  TD(USHORT, "ushort", uint16_t, short,  uint16,  ushort, uint16) \
+  TD(INT,    "int",    int32_t,  int,    int32,   int,    int32) \
+  TD(UINT,   "uint",   uint32_t, int,    uint32,  uint,   uint32) \
+  TD(LONG,   "long",   int64_t,  long,   int64,   long,   int64) \
+  TD(ULONG,  "ulong",  uint64_t, long,   uint64,  ulong,  uint64) /* end int */ \
+  TD(FLOAT,  "float",  float,    float,  float32, float,  float32) /* begin float */ \
+  TD(DOUBLE, "double", double,   double, float64, double, float64) /* end float/scalar */
 #define FLATBUFFERS_GEN_TYPES_POINTER(TD) \
-  TD(STRING, "string", Offset<void>, int, int, int) \
-  TD(VECTOR, "",       Offset<void>, int, int, int) \
-  TD(STRUCT, "",       Offset<void>, int, int, int) \
-  TD(UNION,  "",       Offset<void>, int, int, int)
+  TD(STRING, "string", Offset<void>, int, int, int, int) \
+  TD(VECTOR, "",       Offset<void>, int, int, int, int) \
+  TD(STRUCT, "",       Offset<void>, int, int, int, int) \
+  TD(UNION,  "",       Offset<void>, int, int, int, int)
 
 // The fields are:
 // - enum
@@ -61,12 +61,13 @@ namespace flatbuffers {
 // - Java type.
 // - Go type.
 // - C# / .Net type.
+// - Python type.
 
 // using these macros, we can now write code dealing with types just once, e.g.
 
 /*
 switch (type) {
-  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
     case BASE_TYPE_ ## ENUM: \
       // do something specific to CTYPE here
     FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
@@ -83,13 +84,13 @@ switch (type) {
 __extension__  // Stop GCC complaining about trailing comma with -Wpendantic.
 #endif
 enum BaseType {
-  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
       BASE_TYPE_ ## ENUM,
     FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
   #undef FLATBUFFERS_TD
 };
 
-#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+#define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
     static_assert(sizeof(CTYPE) <= sizeof(largest_scalar_t), \
                   "define largest_scalar_t as " #CTYPE);
   FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
@@ -464,6 +465,13 @@ extern bool GenerateJava(const Parser &parser,
                          const std::string &file_name,
                          const GeneratorOptions &opts);
 
+// Generate Python files from the definitions in the Parser object.
+// See idl_gen_python.cpp.
+extern bool GeneratePython(const Parser &parser,
+                           const std::string &path,
+                           const std::string &file_name,
+                           const GeneratorOptions &opts);
+
 // Generate C# files from the definitions in the Parser object.
 // See idl_gen_csharp.cpp.
 extern bool GenerateCSharp(const Parser &parser,

python/flatbuffers/__init__.py

@@ -0,0 +1,17 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .builder import Builder
+from .table import Table
+from .compat import range_func as compat_range

python/flatbuffers/builder.py

@@ -0,0 +1,549 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from . import number_types as N
+from .number_types import (UOffsetTFlags, SOffsetTFlags, VOffsetTFlags)
+
+from . import encode
+from . import packer
+
+from . import compat
+from .compat import range_func
+from .compat import memoryview_type
+
+
+class OffsetArithmeticError(RuntimeError):
+    """
+    Error caused by an Offset arithmetic error. Probably caused by bad
+    writing of fields. This is considered an unreachable situation in
+    normal circumstances.
+    """
+    pass
+
+
+class NotInObjectError(RuntimeError):
+    """
+    Error caused by using a Builder to write Object data when not inside
+    an Object.
+    """
+    pass
+
+
+class ObjectIsNestedError(RuntimeError):
+    """
+    Error caused by using a Builder to begin an Object when an Object is
+    already being built.
+    """
+    pass
+
+
+class StructIsNotInlineError(RuntimeError):
+    """
+    Error caused by using a Builder to write a Struct at a location that
+    is not the current Offset.
+    """
+    pass
+
+
+class BuilderSizeError(RuntimeError):
+    """
+    Error caused by causing a Builder to exceed the hardcoded limit of 2
+    gigabytes.
+    """
+    pass
+
+
+# VtableMetadataFields is the count of metadata fields in each vtable.
+VtableMetadataFields = 2
+
+
+class Builder(object):
+    """
+    A Builder is used to construct one or more FlatBuffers. Typically, Builder
+    objects will be used from code generated by the `flatc` compiler.
+
+    A Builder constructs byte buffers in a last-first manner for simplicity and
+    performance during reading.
+
+    Internally, a Builder is a state machine for creating FlatBuffer objects.
+
+    It holds the following internal state:
+        Bytes: an array of bytes.
+        current_vtable: a list of integers.
+        vtables: a list of vtable entries (i.e. a list of list of integers).
+    """
+
+    __slots__ = ("Bytes", "current_vtable", "head", "minalign", "objectEnd",
+                 "vtables")
+
+    def __init__(self, initialSize):
+        """
+        Initializes a Builder of size `initial_size`.
+        The internal buffer is grown as needed.
+        """
+
+        if not (0 <= initialSize < (2**UOffsetTFlags.bytewidth - 1)):
+            msg = "flatbuffers: Cannot create Builder larger than 2 gigabytes."
+            raise BuilderSizeError(msg)
+
+        self.Bytes = bytearray(initialSize)
+        self.current_vtable = None
+        self.head = UOffsetTFlags.py_type(initialSize)
+        self.minalign = 1
+        self.objectEnd = None
+        self.vtables = []
+
+    def Output(self):
+        """
+        Output returns the portion of the buffer that has been used for
+        writing data.
+        """
+
+        return self.Bytes[self.Head():]
+
+    def StartObject(self, numfields):
+        """StartObject initializes bookkeeping for writing a new object."""
+
+        self.assertNotNested()
+
+        # use 32-bit offsets so that arithmetic doesn't overflow.
+        self.current_vtable = [0 for _ in range_func(numfields)]
+        self.objectEnd = self.Offset()
+        self.minalign = 1
+
+    def WriteVtable(self):
+        """
+        WriteVtable serializes the vtable for the current object, if needed.
+
+        Before writing out the vtable, this checks pre-existing vtables for
+        equality to this one. If an equal vtable is found, point the object to
+        the existing vtable and return.
+
+        Because vtable values are sensitive to alignment of object data, not
+        all logically-equal vtables will be deduplicated.
+
+        A vtable has the following format:
+          <VOffsetT: size of the vtable in bytes, including this value>
+          <VOffsetT: size of the object in bytes, including the vtable offset>
+          <VOffsetT: offset for a field> * N, where N is the number of fields
+                     in the schema for this type. Includes deprecated fields.
+        Thus, a vtable is made of 2 + N elements, each VOffsetT bytes wide.
+
+        An object has the following format:
+          <SOffsetT: offset to this object's vtable (may be negative)>
+          <byte: data>+
+        """
+
+        # Prepend a zero scalar to the object. Later in this function we'll
+        # write an offset here that points to the object's vtable:
+        self.PrependSOffsetTRelative(0)
+
+        objectOffset = self.Offset()
+        existingVtable = None
+
+        # Search backwards through existing vtables, because similar vtables
+        # are likely to have been recently appended. See
+        # BenchmarkVtableDeduplication for a case in which this heuristic
+        # saves about 30% of the time used in writing objects with duplicate
+        # tables.
+
+        i = len(self.vtables) - 1
+        while i >= 0:
+            # Find the other vtable, which is associated with `i`:
+            vt2Offset = self.vtables[i]
+            vt2Start = len(self.Bytes) - vt2Offset
+            vt2Len = encode.Get(packer.voffset, self.Bytes, vt2Start)
+
+            metadata = VtableMetadataFields * N.VOffsetTFlags.bytewidth
+            vt2End = vt2Start + vt2Len
+            vt2 = self.Bytes[vt2Start+metadata:vt2End]
+
+            # Compare the other vtable to the one under consideration.
+            # If they are equal, store the offset and break:
+            if vtableEqual(self.current_vtable, objectOffset, vt2):
+                existingVtable = vt2Offset
+                break
+
+            i -= 1
+
+        if existingVtable is None:
+            # Did not find a vtable, so write this one to the buffer.
+
+            # Write out the current vtable in reverse , because
+            # serialization occurs in last-first order:
+            i = len(self.current_vtable) - 1
+            while i >= 0:
+                off = 0
+                if self.current_vtable[i] != 0:
+                    # Forward reference to field;
+                    # use 32bit number to ensure no overflow:
+                    off = objectOffset - self.current_vtable[i]
+
+                self.PrependVOffsetT(off)
+                i -= 1
+
+            # The two metadata fields are written last.
+
+            # First, store the object bytesize:
+            objectSize = UOffsetTFlags.py_type(objectOffset - self.objectEnd)
+            self.PrependVOffsetT(VOffsetTFlags.py_type(objectSize))
+
+            # Second, store the vtable bytesize:
+            vBytes = len(self.current_vtable) + VtableMetadataFields
+            vBytes *= N.VOffsetTFlags.bytewidth
+            self.PrependVOffsetT(VOffsetTFlags.py_type(vBytes))
+
+            # Next, write the offset to the new vtable in the
+            # already-allocated SOffsetT at the beginning of this object:
+            objectStart = SOffsetTFlags.py_type(len(self.Bytes) - objectOffset)
+            encode.Write(packer.soffset, self.Bytes, objectStart,
+                         SOffsetTFlags.py_type(self.Offset() - objectOffset))
+
+            # Finally, store this vtable in memory for future
+            # deduplication:
+            self.vtables.append(self.Offset())
+        else:
+            # Found a duplicate vtable.
+
+            objectStart = SOffsetTFlags.py_type(len(self.Bytes) - objectOffset)
+            self.head = UOffsetTFlags.py_type(objectStart)
+
+            # Write the offset to the found vtable in the
+            # already-allocated SOffsetT at the beginning of this object:
+            encode.Write(packer.soffset, self.Bytes, self.Head(),
+                         SOffsetTFlags.py_type(existingVtable - objectOffset))
+
+        self.current_vtable = None
+        return objectOffset
+
+    def EndObject(self):
+        """EndObject writes data necessary to finish object construction."""
+        if self.current_vtable is None:
+            msg = ("flatbuffers: Tried to write the end of an Object when "
+                   "the Builder was not currently writing an Object.")
+            raise NotInObjectError(msg)
+        return self.WriteVtable()
+
+    def growByteBuffer(self):
+        """Doubles the size of the byteslice, and copies the old data towards
+           the end of the new buffer (since we build the buffer backwards)."""
+        if not len(self.Bytes) <= 2**20:
+            msg = "flatbuffers: cannot grow buffer beyond 2 gigabytes"
+            raise BuilderSizeError(msg)
+
+        newSize = len(self.Bytes) * 2
+        if newSize == 0:
+            newSize = 1
+        bytes2 = bytearray(newSize)
+        bytes2[newSize-len(self.Bytes):] = self.Bytes
+        self.Bytes = bytes2
+
+    def Head(self):
+        """
+        Head gives the start of useful data in the underlying byte buffer.
+        Note: unlike other functions, this value is interpreted as from the left.
+        """
+        return self.head
+
+    def Offset(self):
+        """Offset relative to the end of the buffer."""
+        return UOffsetTFlags.py_type(len(self.Bytes) - self.Head())
+
+    def Pad(self, n):
+        """Pad places zeros at the current offset."""
+        for i in range_func(n):
+            self.Place(0, N.Uint8Flags)
+
+    def Prep(self, size, additionalBytes):
+        """
+        Prep prepares to write an element of `size` after `additional_bytes`
+        have been written, e.g. if you write a string, you need to align
+        such the int length field is aligned to SizeInt32, and the string
+        data follows it directly.
+        If all you need to do is align, `additionalBytes` will be 0.
+        """
+
+        # Track the biggest thing we've ever aligned to.
+        if size > self.minalign:
+            self.minalign = size
+
+        # Find the amount of alignment needed such that `size` is properly
+        # aligned after `additionalBytes`:
+        alignSize = (~(len(self.Bytes) - self.Head() + additionalBytes)) + 1
+        alignSize &= (size - 1)
+
+        # Reallocate the buffer if needed:
+        while self.Head() < alignSize+size+additionalBytes:
+            oldBufSize = len(self.Bytes)
+            self.growByteBuffer()
+            updated_head = self.head + len(self.Bytes) - oldBufSize
+            self.head = UOffsetTFlags.py_type(updated_head)
+        self.Pad(alignSize)
+
+    def PrependSOffsetTRelative(self, off):
+        """
+        PrependSOffsetTRelative prepends an SOffsetT, relative to where it
+        will be written.
+        """
+
+        # Ensure alignment is already done:
+        self.Prep(N.SOffsetTFlags.bytewidth, 0)
+        if not (off <= self.Offset()):
+            msg = "flatbuffers: Offset arithmetic error."
+            raise OffsetArithmeticError(msg)
+        off2 = self.Offset() - off + N.SOffsetTFlags.bytewidth
+        self.PlaceSOffsetT(off2)
+
+    def PrependUOffsetTRelative(self, off):
+        """
+        PrependUOffsetTRelative prepends an UOffsetT, relative to where it
+        will be written.
+        """
+
+        # Ensure alignment is already done:
+        self.Prep(N.UOffsetTFlags.bytewidth, 0)
+        if not (off <= self.Offset()):
+            msg = "flatbuffers: Offset arithmetic error."
+            raise OffsetArithmeticError(msg)
+        off2 = self.Offset() - off + N.UOffsetTFlags.bytewidth
+        self.PlaceUOffsetT(off2)
+
+    def StartVector(self, elemSize, numElems, alignment):
+        """
+        StartVector initializes bookkeeping for writing a new vector.
+
+        A vector has the following format:
+          <UOffsetT: number of elements in this vector>
+          <T: data>+, where T is the type of elements of this vector.
+        """
+
+        self.assertNotNested()
+        self.Prep(N.Uint32Flags.bytewidth, elemSize*numElems)
+        self.Prep(alignment, elemSize*numElems)  # In case alignment > int.
+        return self.Offset()
+
+    def EndVector(self, vectorNumElems):
+        """EndVector writes data necessary to finish vector construction."""
+
+        # we already made space for this, so write without PrependUint32
+        self.PlaceUOffsetT(vectorNumElems)
+        return self.Offset()
+
+    def CreateString(self, s):
+        """CreateString writes a null-terminated byte string as a vector."""
+
+        if isinstance(s, compat.string_types):
+            x = s.encode()
+        elif isinstance(s, compat.binary_type):
+            x = s
+        else:
+            raise TypeError("non-string passed to CreateString")
+
+        self.Prep(N.UOffsetTFlags.bytewidth, (len(x)+1)*N.Uint8Flags.bytewidth)
+        self.Place(0, N.Uint8Flags)
+
+        l = UOffsetTFlags.py_type(len(s))
+
+        self.head = UOffsetTFlags.py_type(self.Head() - l)
+        self.Bytes[self.Head():self.Head()+l] = x
+
+        return self.EndVector(len(x))
+
+    def assertNotNested(self):
+        """
+        Check that no other objects are being built while making this
+        object. If not, raise an exception.
+        """
+
+        if self.current_vtable is not None:
+            msg = ("flatbuffers: Tried to write a new Object when the "
+                   "Builder was already writing an Object.")
+            raise ObjectIsNestedError(msg)
+
+    def assertNested(self, obj):
+        """
+        Structs are always stored inline, so need to be created right
+        where they are used. You'll get this error if you created it
+        elsewhere.
+        """
+
+        N.enforce_number(obj, N.UOffsetTFlags)
+        if obj != self.Offset():
+            msg = ("flatbuffers: Tried to write a Struct at an Offset that "
+                   "is different from the current Offset of the Builder.")
+            raise StructIsNotInlineError(msg)
+
+    def Slot(self, slotnum):
+        """
+        Slot sets the vtable key `voffset` to the current location in the
+        buffer.
+
+        """
+        if self.current_vtable is None:
+            msg = ("flatbuffers: Tried to write an Object field when "
+                   "the Builder was not currently writing an Object.")
+            raise NotInObjectError(msg)
+
+        self.current_vtable[slotnum] = self.Offset()
+
+    def Finish(self, rootTable):
+        """Finish finalizes a buffer, pointing to the given `rootTable`."""
+        N.enforce_number(rootTable, N.UOffsetTFlags)
+        self.Prep(self.minalign, N.UOffsetTFlags.bytewidth)
+        self.PrependUOffsetTRelative(rootTable)
+        return self.Head()
+
+    def Prepend(self, flags, off):
+        self.Prep(flags.bytewidth, 0)
+        self.Place(off, flags)
+
+    def PrependSlot(self, flags, o, x, d):
+        N.enforce_number(x, flags)
+        N.enforce_number(d, flags)
+        if x != d:
+            self.Prepend(flags, x)
+            self.Slot(o)
+
+    def PrependBoolSlot(self, *args): self.PrependSlot(N.BoolFlags, *args)
+
+    def PrependByteSlot(self, *args): self.PrependSlot(N.Uint8Flags, *args)
+
+    def PrependUint8Slot(self, *args): self.PrependSlot(N.Uint8Flags, *args)
+
+    def PrependUint16Slot(self, *args): self.PrependSlot(N.Uint16Flags, *args)
+
+    def PrependUint32Slot(self, *args): self.PrependSlot(N.Uint32Flags, *args)
+
+    def PrependUint64Slot(self, *args): self.PrependSlot(N.Uint64Flags, *args)
+
+    def PrependInt8Slot(self, *args): self.PrependSlot(N.Int8Flags, *args)
+
+    def PrependInt16Slot(self, *args): self.PrependSlot(N.Int16Flags, *args)
+
+    def PrependInt32Slot(self, *args): self.PrependSlot(N.Int32Flags, *args)
+
+    def PrependInt64Slot(self, *args): self.PrependSlot(N.Int64Flags, *args)
+
+    def PrependFloat32Slot(self, *args): self.PrependSlot(N.Float32Flags,
+                                                          *args)
+
+    def PrependFloat64Slot(self, *args): self.PrependSlot(N.Float64Flags,
+                                                          *args)
+
+    def PrependUOffsetTRelativeSlot(self, o, x, d):
+        """
+        PrependUOffsetTRelativeSlot prepends an UOffsetT onto the object at
+        vtable slot `o`. If value `x` equals default `d`, then the slot will
+        be set to zero and no other data will be written.
+        """
+
+        if x != d:
+            self.PrependUOffsetTRelative(x)
+            self.Slot(o)
+
+    def PrependStructSlot(self, v, x, d):
+        """
+        PrependStructSlot prepends a struct onto the object at vtable slot `o`.
+        Structs are stored inline, so nothing additional is being added.
+        In generated code, `d` is always 0.
+        """
+
+        N.enforce_number(d, N.UOffsetTFlags)
+        if x != d:
+            self.assertNested(x)
+            self.Slot(v)
+
+    def PrependBool(self, x): self.Prepend(N.BoolFlags, x)
+
+    def PrependByte(self, x): self.Prepend(N.Uint8Flags, x)
+
+    def PrependUint8(self, x): self.Prepend(N.Uint8Flags, x)
+
+    def PrependUint16(self, x): self.Prepend(N.Uint16Flags, x)
+
+    def PrependUint32(self, x): self.Prepend(N.Uint32Flags, x)
+
+    def PrependUint64(self, x): self.Prepend(N.Uint64Flags, x)
+
+    def PrependInt8(self, x): self.Prepend(N.Int8Flags, x)
+
+    def PrependInt16(self, x): self.Prepend(N.Int16Flags, x)
+
+    def PrependInt32(self, x): self.Prepend(N.Int32Flags, x)
+
+    def PrependInt64(self, x): self.Prepend(N.Int64Flags, x)
+
+    def PrependFloat32(self, x): self.Prepend(N.Float32Flags, x)
+
+    def PrependFloat64(self, x): self.Prepend(N.Float64Flags, x)
+
+    def PrependVOffsetT(self, x): self.Prepend(N.VOffsetTFlags, x)
+
+    def Place(self, x, flags):
+        """
+        Place prepends a value specified by `flags` to the Builder,
+        without checking for available space.
+        """
+
+        N.enforce_number(x, flags)
+        self.head = self.head - flags.bytewidth
+        encode.Write(flags.packer_type, self.Bytes, self.Head(), x)
+
+    def PlaceVOffsetT(self, x):
+        """
+        PlaceVOffsetT prepends a VOffsetT to the Builder, without checking for
+        space.
+        """
+        N.enforce_number(x, N.VOffsetTFlags)
+        self.head = self.head - N.VOffsetTFlags.bytewidth
+        encode.Write(packer.voffset, self.Bytes, self.Head(), x)
+
+    def PlaceSOffsetT(self, x):
+        """
+        PlaceSOffsetT prepends a SOffsetT to the Builder, without checking for
+        space.
+        """
+        N.enforce_number(x, N.SOffsetTFlags)
+        self.head = self.head - N.SOffsetTFlags.bytewidth
+        encode.Write(packer.soffset, self.Bytes, self.Head(), x)
+
+    def PlaceUOffsetT(self, x):
+        """
+        PlaceUOffsetT prepends a UOffsetT to the Builder, without checking for
+        space.
+        """
+        N.enforce_number(x, N.UOffsetTFlags)
+        self.head = self.head - N.UOffsetTFlags.bytewidth
+        encode.Write(packer.uoffset, self.Bytes, self.Head(), x)
+
+
+def vtableEqual(a, objectStart, b):
+    """vtableEqual compares an unwritten vtable to a written vtable."""
+
+    N.enforce_number(objectStart, N.UOffsetTFlags)
+
+    if len(a) * N.VOffsetTFlags.bytewidth != len(b):
+        return False
+
+    for i, elem in enumerate(a):
+        x = encode.Get(packer.voffset, b, i * N.VOffsetTFlags.bytewidth)
+
+        # Skip vtable entries that indicate a default value.
+        if x == 0 and elem == 0:
+            pass
+        else:
+            y = objectStart - elem
+            if x != y:
+                return False
+    return True

python/flatbuffers/compat.py

@@ -0,0 +1,27 @@
+""" A tiny version of `six` to help with backwards compability. """
+
+import sys
+
+PY2 = sys.version_info[0] == 2
+PY26 = sys.version_info[0:2] == (2, 6)
+PY3 = sys.version_info[0] == 3
+PY34 = sys.version_info[0:2] >= (3, 4)
+
+if PY3:
+    string_types = (str,)
+    binary_type = bytes
+    range_func = range
+    memoryview_type = memoryview
+    struct_bool_decl = "?"
+else:
+    string_types = (basestring,)
+    binary_type = str
+    range_func = xrange
+    if PY26:
+        memoryview_type = buffer
+        struct_bool_decl = "<b"
+    else:
+        memoryview_type = memoryview
+        struct_bool_decl = "?"
+
+# NOTE: Future Jython support may require code here (look at `six`).

python/flatbuffers/encode.py

@@ -0,0 +1,29 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import ctypes
+
+from . import number_types as N
+from . import packer
+from .compat import memoryview_type
+
+
+def Get(packer_type, buf, head):
+    """ Get decodes a value at buf[head:] using `packer_type`. """
+    return packer_type.unpack_from(memoryview_type(buf), head)[0]
+
+
+def Write(packer_type, buf, head, n):
+    """ Write encodes `n` at buf[head:] using `packer_type`. """
+    packer_type.pack_into(buf, head, n)

python/flatbuffers/number_types.py

@@ -0,0 +1,174 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import ctypes
+import collections
+import struct
+from ctypes import sizeof
+
+from . import packer
+
+
+# For reference, see:
+# https://docs.python.org/2/library/ctypes.html#ctypes-fundamental-data-types-2
+
+# These classes could be collections.namedtuple instances, but those are new
+# in 2.6 and we want to work towards 2.5 compatability.
+
+class BoolFlags(object):
+    bytewidth = 1
+    min_val = False
+    max_val = True
+    py_type = bool
+    name = "bool"
+    packer_type = packer.boolean
+
+
+class Uint8Flags(object):
+    bytewidth = 1
+    min_val = 0
+    max_val = (2**8) - 1
+    py_type = int
+    name = "uint8"
+    packer_type = packer.uint8
+
+
+class Uint16Flags(object):
+    bytewidth = 2
+    min_val = 0
+    max_val = (2**16) - 1
+    py_type = int
+    name = "uint16"
+    packer_type = packer.uint16
+
+
+class Uint32Flags(object):
+    bytewidth = 4
+    min_val = 0
+    max_val = (2**32) - 1
+    py_type = int
+    name = "uint32"
+    packer_type = packer.uint32
+
+
+class Uint64Flags(object):
+    bytewidth = 8
+    min_val = 0
+    max_val = (2**64) - 1
+    py_type = int
+    name = "uint64"
+    packer_type = packer.uint64
+
+
+class Int8Flags(object):
+    bytewidth = 1
+    min_val = -(2**7)
+    max_val = (2**7) - 1
+    py_type = int
+    name = "int8"
+    packer_type = packer.int8
+
+
+class Int16Flags(object):
+    bytewidth = 2
+    min_val = -(2**15)
+    max_val = (2**15) - 1
+    py_type = int
+    name = "int16"
+    packer_type = packer.int16
+
+
+class Int32Flags(object):
+    bytewidth = 4
+    min_val = -(2**31)
+    max_val = (2**31) - 1
+    py_type = int
+    name = "int32"
+    packer_type = packer.int32
+
+
+class Int64Flags(object):
+    bytewidth = 8
+    min_val = -(2**63)
+    max_val = (2**63) - 1
+    py_type = int
+    name = "int64"
+    packer_type = packer.int64
+
+
+class Float32Flags(object):
+    bytewidth = 4
+    min_val = None
+    max_val = None
+    py_type = float
+    name = "float32"
+    packer_type = packer.float32
+
+
+class Float64Flags(object):
+    bytewidth = 8
+    min_val = None
+    max_val = None
+    py_type = float
+    name = "float64"
+    packer_type = packer.float64
+
+
+class SOffsetTFlags(Int32Flags):
+    pass
+
+
+class UOffsetTFlags(Uint32Flags):
+    pass
+
+
+class VOffsetTFlags(Uint16Flags):
+    pass
+
+
+def valid_number(n, flags):
+    if flags.min_val is None and flags.max_val is None:
+        return True
+    return flags.min_val <= n <= flags.max_val
+
+
+def enforce_number(n, flags):
+    if flags.min_val is None and flags.max_val is None:
+        return
+    if not flags.min_val <= n <= flags.max_val:
+        raise TypeError("bad number %s for type %s" % (str(n), flags.name))
+
+
+def float32_to_uint32(n):
+    packed = struct.pack("<1f", n)
+    (converted,) = struct.unpack("<1L", packed)
+    return converted
+
+
+def uint32_to_float32(n):
+    packed = struct.pack("<1L", n)
+    (unpacked,) = struct.unpack("<1f", packed)
+    return unpacked
+
+
+def float64_to_uint64(n):
+    packed = struct.pack("<1d", n)
+    (converted,) = struct.unpack("<1Q", packed)
+    return converted
+
+
+def uint64_to_float64(n):
+    packed = struct.pack("<1Q", n)
+    (unpacked,) = struct.unpack("<1d", packed)
+    return unpacked

python/flatbuffers/packer.py

@@ -0,0 +1,28 @@
+"""
+Provide pre-compiled struct packers for encoding and decoding.
+
+See: https://docs.python.org/2/library/struct.html#format-characters
+"""
+
+import struct
+from . import compat
+
+
+boolean = struct.Struct(compat.struct_bool_decl)
+
+uint8 = struct.Struct("<B")
+uint16 = struct.Struct("<H")
+uint32 = struct.Struct("<I")
+uint64 = struct.Struct("<Q")
+
+int8 = struct.Struct("<b")
+int16 = struct.Struct("<h")
+int32 = struct.Struct("<i")
+int64 = struct.Struct("<q")
+
+float32 = struct.Struct("<f")
+float64 = struct.Struct("<d")
+
+uoffset = uint32
+soffset = int32
+voffset = uint16

python/flatbuffers/table.py

@@ -0,0 +1,117 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from . import encode
+from . import number_types as N
+
+
+class Table(object):
+    """Table wraps a byte slice and provides read access to its data.
+
+    The variable `Pos` indicates the root of the FlatBuffers object therein."""
+
+    __slots__ = ("Bytes", "Pos")
+
+    def __init__(self, buf, pos):
+        N.enforce_number(pos, N.UOffsetTFlags)
+
+        self.Bytes = buf
+        self.Pos = pos
+
+    def Offset(self, vtableOffset):
+        """Offset provides access into the Table's vtable.
+
+        Deprecated fields are ignored by checking the vtable's length."""
+
+        vtable = self.Pos - self.Get(N.SOffsetTFlags, self.Pos)
+        vtableEnd = self.Get(N.VOffsetTFlags, vtable)
+        if vtableOffset < vtableEnd:
+            return self.Get(N.VOffsetTFlags, vtable + vtableOffset)
+        return 0
+
+    def Indirect(self, off):
+        """Indirect retrieves the relative offset stored at `offset`."""
+        N.enforce_number(off, N.UOffsetTFlags)
+        return off + encode.Get(N.UOffsetTFlags.packer_type, self.Bytes, off)
+
+    def String(self, off):
+        """String gets a string from data stored inside the flatbuffer."""
+        N.enforce_number(off, N.UOffsetTFlags)
+        off += encode.Get(N.UOffsetTFlags.packer_type, self.Bytes, off)
+        start = off + N.UOffsetTFlags.bytewidth
+        length = encode.Get(N.UOffsetTFlags.packer_type, self.Bytes, off)
+        return bytes(self.Bytes[start:start+length])
+
+    def VectorLen(self, off):
+        """VectorLen retrieves the length of the vector whose offset is stored
+           at "off" in this object."""
+        N.enforce_number(off, N.UOffsetTFlags)
+
+        off += self.Pos
+        off += encode.Get(N.UOffsetTFlags.packer_type, self.Bytes, off)
+        ret = encode.Get(N.UOffsetTFlags.packer_type, self.Bytes, off)
+        return ret
+
+    def Vector(self, off):
+        """Vector retrieves the start of data of the vector whose offset is
+           stored at "off" in this object."""
+        N.enforce_number(off, N.UOffsetTFlags)
+
+        off += self.Pos
+        x = off + self.Get(N.UOffsetTFlags, off)
+        # data starts after metadata containing the vector length
+        x += N.UOffsetTFlags.bytewidth
+        return x
+
+    def Union(self, t2, off):
+        """Union initializes any Table-derived type to point to the union at
+           the given offset."""
+        assert type(t2) is Table
+        N.enforce_number(off, N.UOffsetTFlags)
+
+        off += self.Pos
+        t2.Pos = off + self.Get(N.UOffsetTFlags, off)
+        t2.Bytes = self.Bytes
+
+    def Get(self, flags, off):
+        """
+        Get retrieves a value of the type specified by `flags`  at the
+        given offset.
+        """
+        N.enforce_number(off, N.UOffsetTFlags)
+        return flags.py_type(encode.Get(flags.packer_type, self.Bytes, off))
+
+    def GetSlot(self, slot, d, validator_flags):
+        N.enforce_number(slot, N.VOffsetTFlags)
+        if validator_flags is not None:
+            N.enforce_number(d, validator_flags)
+        off = self.Offset(slot)
+        if off == 0:
+            return d
+        return self.Get(validator_flags, self.Pos + off)
+
+    def GetVOffsetTSlot(self, slot, d):
+        """
+        GetVOffsetTSlot retrieves the VOffsetT that the given vtable location
+        points to. If the vtable value is zero, the default value `d`
+        will be returned.
+        """
+
+        N.enforce_number(slot, N.VOffsetTFlags)
+        N.enforce_number(d, N.VOffsetTFlags)
+
+        off = self.Offset(slot)
+        if off == 0:
+                return d
+        return off

python/setup.py

@@ -0,0 +1,17 @@
+from setuptools import setup
+
+setup(
+    name='flatbuffers',
+    version='0.1',
+    license='BSD',
+    author='FlatBuffers Contributors',
+    author_email='me@rwinslow.com',
+    url='https://github.com/google/flatbuffers/python',
+    long_description=('Python runtime library and code generator for use with'
+                      'the Flatbuffers serialization format.'),
+    packages=['flatbuffers'],
+    include_package_data=True,
+    requires=[],
+    description=('Runtime library and code generator for use with the '
+                 'Flatbuffers serialization format.'),
+)

src/flatc.cpp

@@ -64,6 +64,10 @@ const Generator generators[] = {
     flatbuffers::GeneratorOptions::kCSharp,
     "Generate C# classes for tables/structs",
     flatbuffers::GeneralMakeRule },
+  { flatbuffers::GeneratePython,   "-p", "Python",
+    flatbuffers::GeneratorOptions::kMAX,
+    "Generate Python files for tables/structs",
+    flatbuffers::GeneralMakeRule },
 };
 
 const char *program_name = NULL;

src/idl_gen_cpp.cpp

@@ -61,7 +61,8 @@ static std::string TranslateNameSpace(const std::string &qualified_name) {
 static std::string GenTypeBasic(const Parser &parser, const Type &type,
                                 bool real_enum) {
   static const char *ctypename[] = {
-    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) #CTYPE,
+    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
+      #CTYPE,
       FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
     #undef FLATBUFFERS_TD
   };

src/idl_gen_general.cpp

@@ -196,7 +196,7 @@ static std::string FunctionStart(const LanguageParameters &lang, char upper) {
 static std::string GenTypeBasic(const LanguageParameters &lang,
                                 const Type &type) {
   static const char *gtypename[] = {
-    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
         #JTYPE, #NTYPE, #GTYPE,
       FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
     #undef FLATBUFFERS_TD

src/idl_gen_go.cpp

@@ -616,7 +616,8 @@ static bool SaveType(const Parser &parser, const Definition &def,
 
 static std::string GenTypeBasic(const Type &type) {
   static const char *ctypename[] = {
-    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) #GTYPE,
+    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
+      #GTYPE,
       FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
     #undef FLATBUFFERS_TD
   };

src/idl_gen_python.cpp

@@ -0,0 +1,664 @@
+/*
+ * Copyright 2014 Google Inc. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// independent from idl_parser, since this code is not needed for most clients
+
+#include <string>
+
+#include "flatbuffers/flatbuffers.h"
+#include "flatbuffers/idl.h"
+#include "flatbuffers/util.h"
+
+namespace flatbuffers {
+namespace python {
+
+static std::string GenGetter(const Type &type);
+static std::string GenMethod(const FieldDef &field);
+static void GenStructBuilder(const StructDef &struct_def,
+                             std::string *code_ptr);
+static void GenReceiver(const StructDef &struct_def, std::string *code_ptr);
+static std::string GenTypeBasic(const Type &type);
+static std::string GenTypeGet(const Type &type);
+static std::string TypeName(const FieldDef &field);
+
+
+// Hardcode spaces per indentation.
+const std::string Indent = "    ";
+
+// Most field accessors need to retrieve and test the field offset first,
+// this is the prefix code for that.
+std::string OffsetPrefix(const FieldDef &field) {
+  return "\n" + Indent + Indent +
+         "o = flatbuffers.number_types.UOffsetTFlags.py_type" +
+         "(self._tab.Offset(" +
+         NumToString(field.value.offset) +
+         "))\n" + Indent + Indent + "if o != 0:\n";
+}
+
+// Begin by declaring namespace and imports.
+static void BeginFile(const std::string name_space_name,
+                      const bool needs_imports,
+                      std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "# automatically generated, do not modify\n\n";
+  code += "# namespace: " + name_space_name + "\n\n";
+  if (needs_imports) {
+    code += "import flatbuffers\n\n";
+  }
+}
+
+// Begin a class declaration.
+static void BeginClass(const StructDef &struct_def, std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "class " + struct_def.name + "(object):\n";
+  code += Indent + "__slots__ = ['_tab']";
+  code += "\n\n";
+}
+
+// Begin enum code with a class declaration.
+static void BeginEnum(const std::string class_name, std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "class " + class_name + "(object):\n";
+}
+
+// A single enum member.
+static void EnumMember(const EnumVal ev, std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += Indent;
+  code += ev.name;
+  code += " = ";
+  code += NumToString(ev.value) + "\n";
+}
+
+// End enum code.
+static void EndEnum(std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "\n";
+}
+
+// Initialize a new struct or table from existing data.
+static void NewRootTypeFromBuffer(const StructDef &struct_def,
+                                  std::string *code_ptr) {
+  std::string &code = *code_ptr;
+
+  code += Indent + "@classmethod\n";
+  code += Indent + "def GetRootAs";
+  code += struct_def.name;
+  code += "(cls, buf, offset):";
+  code += "\n";
+  code += Indent + Indent;
+  code += "n = flatbuffers.encode.Get";
+  code += "(flatbuffers.packer.uoffset, buf, offset)\n";
+  code += Indent + Indent + "x = " + struct_def.name + "()\n";
+  code += Indent + Indent + "x.Init(buf, n + offset)\n";
+  code += Indent + Indent + "return x\n";
+  code += "\n\n";
+}
+
+// Initialize an existing object with other data, to avoid an allocation.
+static void InitializeExisting(const StructDef &struct_def,
+                               std::string *code_ptr) {
+  std::string &code = *code_ptr;
+
+  GenReceiver(struct_def, code_ptr);
+  code += "Init(self, buf, pos):\n";
+  code += Indent + Indent + "self._tab = flatbuffers.table.Table(buf, pos)\n";
+  code += "\n";
+}
+
+// Get the length of a vector.
+static void GetVectorLen(const StructDef &struct_def,
+                         const FieldDef &field,
+                         std::string *code_ptr) {
+  std::string &code = *code_ptr;
+
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name) + "Length(self";
+  code += "):" + OffsetPrefix(field);
+  code += Indent + Indent + Indent + "return self._tab.VectorLen(o)\n";
+  code += Indent + Indent + "return 0\n\n";
+}
+
+// Get the value of a struct's scalar.
+static void GetScalarFieldOfStruct(const StructDef &struct_def,
+                                   const FieldDef &field,
+                                   std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  std::string getter = GenGetter(field.value.type);
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self): return " + getter;
+  code += "self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(";
+  code += NumToString(field.value.offset) + "))\n";
+}
+
+// Get the value of a table's scalar.
+static void GetScalarFieldOfTable(const StructDef &struct_def,
+                                  const FieldDef &field,
+                                  std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  std::string getter = GenGetter(field.value.type);
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self):";
+  code += OffsetPrefix(field);
+  code += Indent + Indent + Indent + "return " + getter;
+  code += "o + self._tab.Pos)\n";
+  code += Indent + Indent + "return " + field.value.constant + "\n\n";
+}
+
+// Get a struct by initializing an existing struct.
+// Specific to Struct.
+static void GetStructFieldOfStruct(const StructDef &struct_def,
+                                   const FieldDef &field,
+                                   std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self, obj):\n";
+  code += Indent + Indent + "obj.Init(self._tab.Bytes, self._tab.Pos + ";
+  code += NumToString(field.value.offset) + ")";
+  code += "\n" + Indent + Indent + "return obj\n\n";
+}
+
+// Get a struct by initializing an existing struct.
+// Specific to Table.
+static void GetStructFieldOfTable(const StructDef &struct_def,
+                                  const FieldDef &field,
+                                  std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self):";
+  code += OffsetPrefix(field);
+  if (field.value.type.struct_def->fixed) {
+    code += Indent + Indent + Indent + "x = o + self._tab.Pos\n";
+  } else {
+    code += Indent + Indent + Indent;
+    code += "x = self._tab.Indirect(o + self._tab.Pos)\n";
+  }
+  code += Indent + Indent + Indent;
+  code += "from ." + TypeName(field) + " import " + TypeName(field) + "\n";
+  code += Indent + Indent + Indent + "obj = " + TypeName(field) + "()\n";
+  code += Indent + Indent + Indent + "obj.Init(self._tab.Bytes, x)\n";
+  code += Indent + Indent + Indent + "return obj\n";
+  code += Indent + Indent + "return None\n\n";
+}
+
+// Get the value of a string.
+static void GetStringField(const StructDef &struct_def,
+                           const FieldDef &field,
+                           std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self):";
+  code += OffsetPrefix(field);
+  code += Indent + Indent + Indent + "return " + GenGetter(field.value.type);
+  code += "o + self._tab.Pos)\n";
+  code += Indent + Indent + "return \"\"\n\n";
+}
+
+// Get the value of a union from an object.
+static void GetUnionField(const StructDef &struct_def,
+                          const FieldDef &field,
+                          std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name) + "(self):";
+  code += OffsetPrefix(field);
+
+  // TODO(rw): this works and is not the good way to it:
+  bool is_native_table = TypeName(field) == "*flatbuffers.Table";
+  if (is_native_table) {
+    code += Indent + Indent + Indent + "from flatbuffers.table import Table\n";
+  } else {
+    code += Indent + Indent + Indent;
+    code += "from ." + TypeName(field) + " import " + TypeName(field) + "\n";
+  }
+  code += Indent + Indent + Indent + "obj = Table(bytearray(), 0)\n";
+  code += Indent + Indent + Indent + GenGetter(field.value.type);
+  code += "obj, o)\n" + Indent + Indent + Indent + "return obj\n";
+  code += Indent + Indent + "return None\n\n";
+}
+
+// Get the value of a vector's struct member.
+static void GetMemberOfVectorOfStruct(const StructDef &struct_def,
+                                      const FieldDef &field,
+                                      std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  auto vectortype = field.value.type.VectorType();
+
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self, j):" + OffsetPrefix(field);
+  code += Indent + Indent + Indent + "x = self._tab.Vector(o)\n";
+  code += Indent + Indent + Indent;
+  code += "x += flatbuffers.number_types.UOffsetTFlags.py_type(j) * ";
+  code += NumToString(InlineSize(vectortype)) + "\n";
+  if (!(vectortype.struct_def->fixed)) {
+    code += Indent + Indent + Indent + "x = self._tab.Indirect(x)\n";
+  }
+  code += Indent + Indent + Indent;
+  code += "from ." + TypeName(field) + " import " + TypeName(field) + "\n";
+  code += Indent + Indent + Indent + "obj = " + TypeName(field) + "()\n";
+  code += Indent + Indent + Indent + "obj.Init(self._tab.Bytes, x)\n";
+  code += Indent + Indent + Indent + "return obj\n";
+  code += Indent + Indent + "return None\n\n";
+}
+
+// Get the value of a vector's non-struct member. Uses a named return
+// argument to conveniently set the zero value for the result.
+static void GetMemberOfVectorOfNonStruct(const StructDef &struct_def,
+                                         const FieldDef &field,
+                                         std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  auto vectortype = field.value.type.VectorType();
+
+  GenReceiver(struct_def, code_ptr);
+  code += MakeCamel(field.name);
+  code += "(self, j):";
+  code += OffsetPrefix(field);
+  code += Indent + Indent + Indent + "a = self._tab.Vector(o)\n";
+  code += Indent + Indent + Indent;
+  code += "return " + GenGetter(field.value.type);
+  code += "a + flatbuffers.number_types.UOffsetTFlags.py_type(j * ";
+  code += NumToString(InlineSize(vectortype)) + "))\n";
+  if (vectortype.base_type == BASE_TYPE_STRING) {
+    code += Indent + Indent + "return \"\"\n";
+  } else {
+    code += Indent + Indent + "return 0\n";
+  }
+  code += "\n";
+}
+
+// Begin the creator function signature.
+static void BeginBuilderArgs(const StructDef &struct_def,
+                             std::string *code_ptr) {
+  std::string &code = *code_ptr;
+
+  code += "\n";
+  code += "def Create" + struct_def.name;
+  code += "(builder";
+}
+
+// Recursively generate arguments for a constructor, to deal with nested
+// structs.
+static void StructBuilderArgs(const StructDef &struct_def,
+                              const char *nameprefix,
+                              std::string *code_ptr) {
+  for (auto it = struct_def.fields.vec.begin();
+       it != struct_def.fields.vec.end();
+       ++it) {
+    auto &field = **it;
+    if (IsStruct(field.value.type)) {
+      // Generate arguments for a struct inside a struct. To ensure names
+      // don't clash, and to make it obvious these arguments are constructing
+      // a nested struct, prefix the name with the struct name.
+      StructBuilderArgs(*field.value.type.struct_def,
+                        (field.value.type.struct_def->name + "_").c_str(),
+                        code_ptr);
+    } else {
+      std::string &code = *code_ptr;
+      code += (std::string)", " + nameprefix;
+      code += MakeCamel(field.name, false);
+    }
+  }
+}
+
+// End the creator function signature.
+static void EndBuilderArgs(std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "):\n";
+}
+
+// Recursively generate struct construction statements and instert manual
+// padding.
+static void StructBuilderBody(const StructDef &struct_def,
+                              const char *nameprefix,
+                              std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "    builder.Prep(" + NumToString(struct_def.minalign) + ", ";
+  code += NumToString(struct_def.bytesize) + ")\n";
+  for (auto it = struct_def.fields.vec.rbegin();
+       it != struct_def.fields.vec.rend();
+       ++it) {
+    auto &field = **it;
+    if (field.padding)
+      code += "    builder.Pad(" + NumToString(field.padding) + ")\n";
+    if (IsStruct(field.value.type)) {
+      StructBuilderBody(*field.value.type.struct_def,
+                        (field.value.type.struct_def->name + "_").c_str(),
+                        code_ptr);
+    } else {
+      code += "    builder.Prepend" + GenMethod(field) + "(";
+      code += nameprefix + MakeCamel(field.name, false) + ")\n";
+    }
+  }
+}
+
+static void EndBuilderBody(std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "    return builder.Offset()\n";
+}
+
+// Get the value of a table's starting offset.
+static void GetStartOfTable(const StructDef &struct_def,
+                            std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "def " + struct_def.name + "Start";
+  code += "(builder): ";
+  code += "builder.StartObject(";
+  code += NumToString(struct_def.fields.vec.size());
+  code += ")\n";
+}
+
+// Set the value of a table's field.
+static void BuildFieldOfTable(const StructDef &struct_def,
+                              const FieldDef &field,
+                              const size_t offset,
+                              std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "def " + struct_def.name + "Add" + MakeCamel(field.name);
+  code += "(builder, ";
+  code += MakeCamel(field.name, false);
+  code += "): ";
+  code += "builder.Prepend";
+  code += GenMethod(field) + "Slot(";
+  code += NumToString(offset) + ", ";
+  if (!IsScalar(field.value.type.base_type) && (!struct_def.fixed)) {
+    code += "flatbuffers.number_types.UOffsetTFlags.py_type";
+    code += "(";
+    code += MakeCamel(field.name, false) + ")";
+  } else {
+    code += MakeCamel(field.name, false);
+  }
+  code += ", " + field.value.constant;
+  code += ")\n";
+}
+
+// Set the value of one of the members of a table's vector.
+static void BuildVectorOfTable(const StructDef &struct_def,
+                               const FieldDef &field,
+                               std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "def " + struct_def.name + "Start";
+  code += MakeCamel(field.name);
+  code += "Vector(builder, numElems): return builder.StartVector(";
+  auto vector_type = field.value.type.VectorType();
+  auto alignment = InlineAlignment(vector_type);
+  auto elem_size = InlineSize(vector_type);
+  code += NumToString(elem_size);
+  code += ", numElems, " + NumToString(alignment);
+  code += ")\n";
+}
+
+// Get the offset of the end of a table.
+static void GetEndOffsetOnTable(const StructDef &struct_def,
+                                std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += "def " + struct_def.name + "End";
+  code += "(builder): ";
+  code += "return builder.EndObject()\n";
+}
+
+// Generate the receiver for function signatures.
+static void GenReceiver(const StructDef &struct_def, std::string *code_ptr) {
+  std::string &code = *code_ptr;
+  code += Indent + "# " + struct_def.name + "\n";
+  code += Indent + "def ";
+}
+
+// Generate a struct field, conditioned on its child type(s).
+static void GenStructAccessor(const StructDef &struct_def,
+                              const FieldDef &field,
+                              std::string *code_ptr) {
+  GenComment(field.doc_comment, code_ptr, nullptr, "# ");
+  if (IsScalar(field.value.type.base_type)) {
+    if (struct_def.fixed) {
+      GetScalarFieldOfStruct(struct_def, field, code_ptr);
+    } else {
+      GetScalarFieldOfTable(struct_def, field, code_ptr);
+    }
+  } else {
+    switch (field.value.type.base_type) {
+      case BASE_TYPE_STRUCT:
+        if (struct_def.fixed) {
+          GetStructFieldOfStruct(struct_def, field, code_ptr);
+        } else {
+          GetStructFieldOfTable(struct_def, field, code_ptr);
+        }
+        break;
+      case BASE_TYPE_STRING:
+        GetStringField(struct_def, field, code_ptr);
+        break;
+      case BASE_TYPE_VECTOR: {
+        auto vectortype = field.value.type.VectorType();
+        if (vectortype.base_type == BASE_TYPE_STRUCT) {
+          GetMemberOfVectorOfStruct(struct_def, field, code_ptr);
+        } else {
+          GetMemberOfVectorOfNonStruct(struct_def, field, code_ptr);
+        }
+        break;
+      }
+      case BASE_TYPE_UNION:
+        GetUnionField(struct_def, field, code_ptr);
+        break;
+      default:
+        assert(0);
+    }
+  }
+  if (field.value.type.base_type == BASE_TYPE_VECTOR) {
+    GetVectorLen(struct_def, field, code_ptr);
+  }
+}
+
+// Generate table constructors, conditioned on its members' types.
+static void GenTableBuilders(const StructDef &struct_def,
+                             std::string *code_ptr) {
+  GetStartOfTable(struct_def, code_ptr);
+
+  for (auto it = struct_def.fields.vec.begin();
+       it != struct_def.fields.vec.end();
+       ++it) {
+    auto &field = **it;
+    if (field.deprecated) continue;
+
+    auto offset = it - struct_def.fields.vec.begin();
+    BuildFieldOfTable(struct_def, field, offset, code_ptr);
+    if (field.value.type.base_type == BASE_TYPE_VECTOR) {
+      BuildVectorOfTable(struct_def, field, code_ptr);
+    }
+  }
+
+  GetEndOffsetOnTable(struct_def, code_ptr);
+}
+
+// Generate struct or table methods.
+static void GenStruct(const StructDef &struct_def,
+                      std::string *code_ptr,
+                      StructDef *root_struct_def) {
+  if (struct_def.generated) return;
+
+  GenComment(struct_def.doc_comment, code_ptr, nullptr);
+  BeginClass(struct_def, code_ptr);
+  if (&struct_def == root_struct_def) {
+    // Generate a special accessor for the table that has been declared as
+    // the root type.
+    NewRootTypeFromBuffer(struct_def, code_ptr);
+  }
+  // Generate the Init method that sets the field in a pre-existing
+  // accessor object. This is to allow object reuse.
+  InitializeExisting(struct_def, code_ptr);
+  for (auto it = struct_def.fields.vec.begin();
+       it != struct_def.fields.vec.end();
+       ++it) {
+    auto &field = **it;
+    if (field.deprecated) continue;
+
+    GenStructAccessor(struct_def, field, code_ptr);
+  }
+
+  if (struct_def.fixed) {
+    // create a struct constructor function
+    GenStructBuilder(struct_def, code_ptr);
+  } else {
+    // Create a set of functions that allow table construction.
+    GenTableBuilders(struct_def, code_ptr);
+  }
+}
+
+// Generate enum declarations.
+static void GenEnum(const EnumDef &enum_def, std::string *code_ptr) {
+  if (enum_def.generated) return;
+
+  GenComment(enum_def.doc_comment, code_ptr, nullptr, "# ");
+  BeginEnum(enum_def.name, code_ptr);
+  for (auto it = enum_def.vals.vec.begin();
+       it != enum_def.vals.vec.end();
+       ++it) {
+    auto &ev = **it;
+    GenComment(ev.doc_comment, code_ptr, nullptr, "# ");
+    EnumMember(ev, code_ptr);
+  }
+  EndEnum(code_ptr);
+}
+
+// Returns the function name that is able to read a value of the given type.
+static std::string GenGetter(const Type &type) {
+  switch (type.base_type) {
+    case BASE_TYPE_STRING: return "self._tab.String(";
+    case BASE_TYPE_UNION: return "self._tab.Union(";
+    case BASE_TYPE_VECTOR: return GenGetter(type.VectorType());
+    default:
+      return "self._tab.Get(flatbuffers.number_types." + \
+             MakeCamel(GenTypeGet(type)) + \
+             "Flags, ";
+  }
+}
+
+// Returns the method name for use with add/put calls.
+static std::string GenMethod(const FieldDef &field) {
+  return IsScalar(field.value.type.base_type)
+    ? MakeCamel(GenTypeBasic(field.value.type))
+    : (IsStruct(field.value.type) ? "Struct" : "UOffsetTRelative");
+}
+
+
+// Save out the generated code for a Python Table type.
+static bool SaveType(const Parser &parser, const Definition &def,
+                     const std::string &classcode, const std::string &path,
+                     bool needs_imports) {
+  if (!classcode.length()) return true;
+
+  std::string namespace_name;
+  std::string namespace_dir = path;
+  auto &namespaces = parser.namespaces_.back()->components;
+  for (auto it = namespaces.begin(); it != namespaces.end(); ++it) {
+    if (namespace_name.length()) {
+      namespace_name += ".";
+      namespace_dir += kPathSeparator;
+    }
+    namespace_name = *it;
+    namespace_dir += *it;
+    mkdir(namespace_dir.c_str(), S_IRWXU|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH);
+
+    std::string init_py_filename = namespace_dir + "/__init__.py";
+    SaveFile(init_py_filename.c_str(), "", false);
+  }
+
+
+  std::string code = "";
+  BeginFile(namespace_name, needs_imports, &code);
+  code += classcode;
+  std::string filename = namespace_dir + kPathSeparator + def.name + ".py";
+  return SaveFile(filename.c_str(), code, false);
+}
+
+static std::string GenTypeBasic(const Type &type) {
+  static const char *ctypename[] = {
+    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
+      #PTYPE,
+      FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
+    #undef FLATBUFFERS_TD
+  };
+  return ctypename[type.base_type];
+}
+
+static std::string GenTypePointer(const Type &type) {
+  switch (type.base_type) {
+    case BASE_TYPE_STRING:
+      return "string";
+    case BASE_TYPE_VECTOR:
+      return GenTypeGet(type.VectorType());
+    case BASE_TYPE_STRUCT:
+      return type.struct_def->name;
+    case BASE_TYPE_UNION:
+      // fall through
+    default:
+      return "*flatbuffers.Table";
+  }
+}
+
+static std::string GenTypeGet(const Type &type) {
+  return IsScalar(type.base_type)
+    ? GenTypeBasic(type)
+    : GenTypePointer(type);
+}
+
+static std::string TypeName(const FieldDef &field) {
+  return GenTypeGet(field.value.type);
+}
+
+// Create a struct with a builder and the struct's arguments.
+static void GenStructBuilder(const StructDef &struct_def,
+                             std::string *code_ptr) {
+  BeginBuilderArgs(struct_def, code_ptr);
+  StructBuilderArgs(struct_def, "", code_ptr);
+  EndBuilderArgs(code_ptr);
+
+  StructBuilderBody(struct_def, "", code_ptr);
+  EndBuilderBody(code_ptr);
+}
+
+}  // namespace python
+
+bool GeneratePython(const Parser &parser,
+                    const std::string &path,
+                    const std::string & /*file_name*/,
+                    const GeneratorOptions & /*opts*/) {
+  for (auto it = parser.enums_.vec.begin();
+       it != parser.enums_.vec.end(); ++it) {
+    std::string enumcode;
+    python::GenEnum(**it, &enumcode);
+    if (!python::SaveType(parser, **it, enumcode, path, false))
+      return false;
+  }
+
+  for (auto it = parser.structs_.vec.begin();
+       it != parser.structs_.vec.end(); ++it) {
+    std::string declcode;
+    python::GenStruct(**it, &declcode, parser.root_struct_def);
+    if (!python::SaveType(parser, **it, declcode, path, true))
+      return false;
+  }
+
+  return true;
+}
+
+}  // namespace flatbuffers
+
+

src/idl_gen_text.cpp

@@ -159,7 +159,8 @@ template<> void Print<const void *>(const void *val,
       type = type.VectorType();
       // Call PrintVector above specifically for each element type:
       switch (type.base_type) {
-        #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+        #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, \
+          PTYPE) \
           case BASE_TYPE_ ## ENUM: \
             PrintVector<CTYPE>( \
               *reinterpret_cast<const Vector<CTYPE> *>(val), \
@@ -225,7 +226,8 @@ static void GenStruct(const StructDef &struct_def, const Table *table,
       OutputIdentifier(fd.name, opts, _text);
       text += ": ";
       switch (fd.value.type.base_type) {
-         #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+         #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, \
+           PTYPE) \
            case BASE_TYPE_ ## ENUM: \
               GenField<CTYPE>(fd, table, struct_def.fixed, \
                               opts, indent + Indent(opts), _text); \
@@ -233,7 +235,8 @@ static void GenStruct(const StructDef &struct_def, const Table *table,
           FLATBUFFERS_GEN_TYPES_SCALAR(FLATBUFFERS_TD)
         #undef FLATBUFFERS_TD
         // Generate drop-thru case statements for all pointer types:
-        #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+        #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, \
+          PTYPE) \
           case BASE_TYPE_ ## ENUM:
           FLATBUFFERS_GEN_TYPES_POINTER(FLATBUFFERS_TD)
         #undef FLATBUFFERS_TD

src/idl_parser.cpp

@@ -25,14 +25,15 @@
 namespace flatbuffers {
 
 const char *const kTypeNames[] = {
-  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) IDLTYPE,
+  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
+    IDLTYPE,
     FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
   #undef FLATBUFFERS_TD
   nullptr
 };
 
 const char kTypeSizes[] = {
-  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
       sizeof(CTYPE),
     FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
   #undef FLATBUFFERS_TD
@@ -96,7 +97,7 @@ enum {
   #define FLATBUFFERS_TOKEN(NAME, VALUE, STRING) kToken ## NAME = VALUE,
     FLATBUFFERS_GEN_TOKENS(FLATBUFFERS_TOKEN)
   #undef FLATBUFFERS_TOKEN
-  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+  #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
       kToken ## ENUM,
     FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
   #undef FLATBUFFERS_TD
@@ -107,7 +108,8 @@ static std::string TokenToString(int t) {
     #define FLATBUFFERS_TOKEN(NAME, VALUE, STRING) STRING,
       FLATBUFFERS_GEN_TOKENS(FLATBUFFERS_TOKEN)
     #undef FLATBUFFERS_TOKEN
-    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) IDLTYPE,
+    #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
+      IDLTYPE,
       FLATBUFFERS_GEN_TYPES(FLATBUFFERS_TD)
     #undef FLATBUFFERS_TD
   };
@@ -205,7 +207,8 @@ void Parser::Next() {
           attribute_.clear();
           attribute_.append(start, cursor_);
           // First, see if it is a type keyword from the table of types:
-          #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+          #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, \
+            PTYPE) \
             if (attribute_ == IDLTYPE) { \
               token_ = kToken ## ENUM; \
               return; \
@@ -580,7 +583,8 @@ uoffset_t Parser::ParseTable(const StructDef &struct_def) {
       auto field = it->second;
       if (!struct_def.sortbysize || size == SizeOf(value.type.base_type)) {
         switch (value.type.base_type) {
-          #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+          #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, \
+            PTYPE) \
             case BASE_TYPE_ ## ENUM: \
               builder_.Pad(field->padding); \
               if (struct_def.fixed) { \
@@ -593,7 +597,8 @@ uoffset_t Parser::ParseTable(const StructDef &struct_def) {
               break;
             FLATBUFFERS_GEN_TYPES_SCALAR(FLATBUFFERS_TD);
           #undef FLATBUFFERS_TD
-          #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+          #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, \
+            PTYPE) \
             case BASE_TYPE_ ## ENUM: \
               builder_.Pad(field->padding); \
               if (IsStruct(field->value.type)) { \
@@ -648,7 +653,7 @@ uoffset_t Parser::ParseVector(const Type &type) {
     // start at the back, since we're building the data backwards.
     auto &val = field_stack_.back().first;
     switch (val.type.base_type) {
-      #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE) \
+      #define FLATBUFFERS_TD(ENUM, IDLTYPE, CTYPE, JTYPE, GTYPE, NTYPE, PTYPE) \
         case BASE_TYPE_ ## ENUM: \
           if (IsStruct(val.type)) SerializeStruct(*val.type.struct_def, val); \
           else builder_.PushElement(atot<CTYPE>(val.constant.c_str())); \

tests/MyGame/Example/Any.py

@@ -0,0 +1,8 @@
+# automatically generated, do not modify
+
+# namespace: Example
+
+class Any(object):
+    NONE = 0
+    Monster = 1
+

tests/MyGame/Example/Color.py

@@ -0,0 +1,9 @@
+# automatically generated, do not modify
+
+# namespace: Example
+
+class Color(object):
+    Red = 1
+    Green = 2
+    Blue = 8
+

tests/MyGame/Example/Monster.py

@@ -0,0 +1,278 @@
+# automatically generated, do not modify
+
+# namespace: Example
+
+import flatbuffers
+
+class Monster(object):
+    __slots__ = ['_tab']
+
+    @classmethod
+    def GetRootAsMonster(cls, buf, offset):
+        n = flatbuffers.encode.Get(flatbuffers.packer.uoffset, buf, offset)
+        x = Monster()
+        x.Init(buf, n + offset)
+        return x
+
+
+    # Monster
+    def Init(self, buf, pos):
+        self._tab = flatbuffers.table.Table(buf, pos)
+
+    # Monster
+    def Pos(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(4))
+        if o != 0:
+            x = o + self._tab.Pos
+            from .Vec3 import Vec3
+            obj = Vec3()
+            obj.Init(self._tab.Bytes, x)
+            return obj
+        return None
+
+    # Monster
+    def Mana(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(6))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int16Flags, o + self._tab.Pos)
+        return 150
+
+    # Monster
+    def Hp(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(8))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int16Flags, o + self._tab.Pos)
+        return 100
+
+    # Monster
+    def Name(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(10))
+        if o != 0:
+            return self._tab.String(o + self._tab.Pos)
+        return ""
+
+    # Monster
+    def Inventory(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(14))
+        if o != 0:
+            a = self._tab.Vector(o)
+            return self._tab.Get(flatbuffers.number_types.Uint8Flags, a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 1))
+        return 0
+
+    # Monster
+    def InventoryLength(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(14))
+        if o != 0:
+            return self._tab.VectorLen(o)
+        return 0
+
+    # Monster
+    def Color(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(16))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int8Flags, o + self._tab.Pos)
+        return 8
+
+    # Monster
+    def TestType(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(18))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Uint8Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Test(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(20))
+        if o != 0:
+            from flatbuffers.table import Table
+            obj = Table(bytearray(), 0)
+            self._tab.Union(obj, o)
+            return obj
+        return None
+
+    # Monster
+    def Test4(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(22))
+        if o != 0:
+            x = self._tab.Vector(o)
+            x += flatbuffers.number_types.UOffsetTFlags.py_type(j) * 4
+            from .Test import Test
+            obj = Test()
+            obj.Init(self._tab.Bytes, x)
+            return obj
+        return None
+
+    # Monster
+    def Test4Length(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(22))
+        if o != 0:
+            return self._tab.VectorLen(o)
+        return 0
+
+    # Monster
+    def Testarrayofstring(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(24))
+        if o != 0:
+            a = self._tab.Vector(o)
+            return self._tab.String(a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 4))
+        return ""
+
+    # Monster
+    def TestarrayofstringLength(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(24))
+        if o != 0:
+            return self._tab.VectorLen(o)
+        return 0
+
+# /// an example documentation comment: this will end up in the generated code
+# /// multiline too
+    # Monster
+    def Testarrayoftables(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(26))
+        if o != 0:
+            x = self._tab.Vector(o)
+            x += flatbuffers.number_types.UOffsetTFlags.py_type(j) * 4
+            x = self._tab.Indirect(x)
+            from .Monster import Monster
+            obj = Monster()
+            obj.Init(self._tab.Bytes, x)
+            return obj
+        return None
+
+    # Monster
+    def TestarrayoftablesLength(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(26))
+        if o != 0:
+            return self._tab.VectorLen(o)
+        return 0
+
+    # Monster
+    def Enemy(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(28))
+        if o != 0:
+            x = self._tab.Indirect(o + self._tab.Pos)
+            from .Monster import Monster
+            obj = Monster()
+            obj.Init(self._tab.Bytes, x)
+            return obj
+        return None
+
+    # Monster
+    def Testnestedflatbuffer(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(30))
+        if o != 0:
+            a = self._tab.Vector(o)
+            return self._tab.Get(flatbuffers.number_types.Uint8Flags, a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 1))
+        return 0
+
+    # Monster
+    def TestnestedflatbufferLength(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(30))
+        if o != 0:
+            return self._tab.VectorLen(o)
+        return 0
+
+    # Monster
+    def Testempty(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(32))
+        if o != 0:
+            x = self._tab.Indirect(o + self._tab.Pos)
+            from .Stat import Stat
+            obj = Stat()
+            obj.Init(self._tab.Bytes, x)
+            return obj
+        return None
+
+    # Monster
+    def Testbool(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(34))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.BoolFlags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashs32Fnv1(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(36))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int32Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashu32Fnv1(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(38))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Uint32Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashs64Fnv1(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(40))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int64Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashu64Fnv1(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(42))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Uint64Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashs32Fnv1a(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(44))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int32Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashu32Fnv1a(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(46))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Uint32Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashs64Fnv1a(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(48))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int64Flags, o + self._tab.Pos)
+        return 0
+
+    # Monster
+    def Testhashu64Fnv1a(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(50))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Uint64Flags, o + self._tab.Pos)
+        return 0
+
+def MonsterStart(builder): builder.StartObject(24)
+def MonsterAddPos(builder, pos): builder.PrependStructSlot(0, flatbuffers.number_types.UOffsetTFlags.py_type(pos), 0)
+def MonsterAddMana(builder, mana): builder.PrependInt16Slot(1, mana, 150)
+def MonsterAddHp(builder, hp): builder.PrependInt16Slot(2, hp, 100)
+def MonsterAddName(builder, name): builder.PrependUOffsetTRelativeSlot(3, flatbuffers.number_types.UOffsetTFlags.py_type(name), 0)
+def MonsterAddInventory(builder, inventory): builder.PrependUOffsetTRelativeSlot(5, flatbuffers.number_types.UOffsetTFlags.py_type(inventory), 0)
+def MonsterStartInventoryVector(builder, numElems): return builder.StartVector(1, numElems, 1)
+def MonsterAddColor(builder, color): builder.PrependInt8Slot(6, color, 8)
+def MonsterAddTestType(builder, testType): builder.PrependUint8Slot(7, testType, 0)
+def MonsterAddTest(builder, test): builder.PrependUOffsetTRelativeSlot(8, flatbuffers.number_types.UOffsetTFlags.py_type(test), 0)
+def MonsterAddTest4(builder, test4): builder.PrependUOffsetTRelativeSlot(9, flatbuffers.number_types.UOffsetTFlags.py_type(test4), 0)
+def MonsterStartTest4Vector(builder, numElems): return builder.StartVector(4, numElems, 2)
+def MonsterAddTestarrayofstring(builder, testarrayofstring): builder.PrependUOffsetTRelativeSlot(10, flatbuffers.number_types.UOffsetTFlags.py_type(testarrayofstring), 0)
+def MonsterStartTestarrayofstringVector(builder, numElems): return builder.StartVector(4, numElems, 4)
+def MonsterAddTestarrayoftables(builder, testarrayoftables): builder.PrependUOffsetTRelativeSlot(11, flatbuffers.number_types.UOffsetTFlags.py_type(testarrayoftables), 0)
+def MonsterStartTestarrayoftablesVector(builder, numElems): return builder.StartVector(4, numElems, 4)
+def MonsterAddEnemy(builder, enemy): builder.PrependUOffsetTRelativeSlot(12, flatbuffers.number_types.UOffsetTFlags.py_type(enemy), 0)
+def MonsterAddTestnestedflatbuffer(builder, testnestedflatbuffer): builder.PrependUOffsetTRelativeSlot(13, flatbuffers.number_types.UOffsetTFlags.py_type(testnestedflatbuffer), 0)
+def MonsterStartTestnestedflatbufferVector(builder, numElems): return builder.StartVector(1, numElems, 1)
+def MonsterAddTestempty(builder, testempty): builder.PrependUOffsetTRelativeSlot(14, flatbuffers.number_types.UOffsetTFlags.py_type(testempty), 0)
+def MonsterAddTestbool(builder, testbool): builder.PrependBoolSlot(15, testbool, 0)
+def MonsterAddTesthashs32Fnv1(builder, testhashs32Fnv1): builder.PrependInt32Slot(16, testhashs32Fnv1, 0)
+def MonsterAddTesthashu32Fnv1(builder, testhashu32Fnv1): builder.PrependUint32Slot(17, testhashu32Fnv1, 0)
+def MonsterAddTesthashs64Fnv1(builder, testhashs64Fnv1): builder.PrependInt64Slot(18, testhashs64Fnv1, 0)
+def MonsterAddTesthashu64Fnv1(builder, testhashu64Fnv1): builder.PrependUint64Slot(19, testhashu64Fnv1, 0)
+def MonsterAddTesthashs32Fnv1a(builder, testhashs32Fnv1a): builder.PrependInt32Slot(20, testhashs32Fnv1a, 0)
+def MonsterAddTesthashu32Fnv1a(builder, testhashu32Fnv1a): builder.PrependUint32Slot(21, testhashu32Fnv1a, 0)
+def MonsterAddTesthashs64Fnv1a(builder, testhashs64Fnv1a): builder.PrependInt64Slot(22, testhashs64Fnv1a, 0)
+def MonsterAddTesthashu64Fnv1a(builder, testhashu64Fnv1a): builder.PrependUint64Slot(23, testhashu64Fnv1a, 0)
+def MonsterEnd(builder): return builder.EndObject()

tests/MyGame/Example/Stat.py

@@ -0,0 +1,39 @@
+# automatically generated, do not modify
+
+# namespace: Example
+
+import flatbuffers
+
+class Stat(object):
+    __slots__ = ['_tab']
+
+    # Stat
+    def Init(self, buf, pos):
+        self._tab = flatbuffers.table.Table(buf, pos)
+
+    # Stat
+    def Id(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(4))
+        if o != 0:
+            return self._tab.String(o + self._tab.Pos)
+        return ""
+
+    # Stat
+    def Val(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(6))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Int64Flags, o + self._tab.Pos)
+        return 0
+
+    # Stat
+    def Count(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(8))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Uint16Flags, o + self._tab.Pos)
+        return 0
+
+def StatStart(builder): builder.StartObject(3)
+def StatAddId(builder, id): builder.PrependUOffsetTRelativeSlot(0, flatbuffers.number_types.UOffsetTFlags.py_type(id), 0)
+def StatAddVal(builder, val): builder.PrependInt64Slot(1, val, 0)
+def StatAddCount(builder, count): builder.PrependUint16Slot(2, count, 0)
+def StatEnd(builder): return builder.EndObject()

tests/MyGame/Example/Test.py

@@ -0,0 +1,24 @@
+# automatically generated, do not modify
+
+# namespace: Example
+
+import flatbuffers
+
+class Test(object):
+    __slots__ = ['_tab']
+
+    # Test
+    def Init(self, buf, pos):
+        self._tab = flatbuffers.table.Table(buf, pos)
+
+    # Test
+    def A(self): return self._tab.Get(flatbuffers.number_types.Int16Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(0))
+    # Test
+    def B(self): return self._tab.Get(flatbuffers.number_types.Int8Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(2))
+
+def CreateTest(builder, a, b):
+    builder.Prep(2, 4)
+    builder.Pad(1)
+    builder.PrependInt8(b)
+    builder.PrependInt16(a)
+    return builder.Offset()

tests/MyGame/Example/Vec3.py

@@ -0,0 +1,44 @@
+# automatically generated, do not modify
+
+# namespace: Example
+
+import flatbuffers
+
+class Vec3(object):
+    __slots__ = ['_tab']
+
+    # Vec3
+    def Init(self, buf, pos):
+        self._tab = flatbuffers.table.Table(buf, pos)
+
+    # Vec3
+    def X(self): return self._tab.Get(flatbuffers.number_types.Float32Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(0))
+    # Vec3
+    def Y(self): return self._tab.Get(flatbuffers.number_types.Float32Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(4))
+    # Vec3
+    def Z(self): return self._tab.Get(flatbuffers.number_types.Float32Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(8))
+    # Vec3
+    def Test1(self): return self._tab.Get(flatbuffers.number_types.Float64Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(16))
+    # Vec3
+    def Test2(self): return self._tab.Get(flatbuffers.number_types.Int8Flags, self._tab.Pos + flatbuffers.number_types.UOffsetTFlags.py_type(24))
+    # Vec3
+    def Test3(self, obj):
+        obj.Init(self._tab.Bytes, self._tab.Pos + 26)
+        return obj
+
+
+def CreateVec3(builder, x, y, z, test1, test2, Test_a, Test_b):
+    builder.Prep(16, 32)
+    builder.Pad(2)
+    builder.Prep(2, 4)
+    builder.Pad(1)
+    builder.PrependInt8(Test_b)
+    builder.PrependInt16(Test_a)
+    builder.Pad(1)
+    builder.PrependInt8(test2)
+    builder.PrependFloat64(test1)
+    builder.Pad(4)
+    builder.PrependFloat32(z)
+    builder.PrependFloat32(y)
+    builder.PrependFloat32(x)
+    return builder.Offset()

tests/PythonTest.sh

@@ -0,0 +1,78 @@
+#!/bin/bash -eu
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+pushd "$(dirname $0)" >/dev/null
+test_dir="$(pwd)"
+gen_code_path=${test_dir}
+runtime_library_dir=${test_dir}/../python
+
+# Emit Python code for the example schema in the test dir:
+${test_dir}/../flatc -p -o ${gen_code_path} monster_test.fbs
+
+# Syntax: run_tests <interpreter> <benchmark vtable dedupes>
+#                   <benchmark read count> <benchmark build count>
+interpreters_tested=()
+function run_tests() {
+  if $(which ${1} >/dev/null); then
+    echo "Testing with interpreter: ${1}"
+    PYTHONDONTWRITEBYTECODE=1 \
+    JYTHONDONTWRITEBYTECODE=1 \
+    PYTHONPATH=${runtime_library_dir}:${gen_code_path} \
+    JYTHONPATH=${runtime_library_dir}:${gen_code_path} \
+    COMPARE_GENERATED_TO_GO=0 \
+    COMPARE_GENERATED_TO_JAVA=0 \
+    $1 py_test.py $2 $3 $4
+    interpreters_tested+=(${1})
+    echo
+  fi
+}
+
+# Run test suite with these interpreters. The arguments are benchmark counts.
+run_tests python2.6 100 100 100
+run_tests python2.7 100 100 100
+run_tests python3 100 100 100
+run_tests pypy 100 100 100
+
+# NOTE: We'd like to support python2.5 in the future.
+
+# NOTE: Jython 2.7.0 fails due to a bug in the stdlib `struct` library:
+#       http://bugs.jython.org/issue2188
+
+if [ ${#interpreters_tested[@]} -eq 0 ]; then
+  echo "No Python interpeters found on this system, could not run tests."
+  exit 1
+fi
+
+# Run test suite with default python intereter.
+# (If the Python program `coverage` is available, it will be run, too.
+#  Install `coverage` with `pip install coverage`.)
+if $(which coverage >/dev/null); then
+  echo 'Found coverage utility, running coverage with default Python:'
+
+  PYTHONDONTWRITEBYTECODE=1 \
+  PYTHONPATH=${runtime_library_dir}:${gen_code_path} \
+  coverage run --source=flatbuffers,MyGame py_test.py 0 0 0 > /dev/null
+
+  echo
+  cov_result=`coverage report --omit="*flatbuffers/vendor*,*py_test*" \
+              | tail -n 1 | awk ' { print $4 } '`
+  echo "Code coverage: ${cov_result}"
+else
+  echo -n "Did not find coverage utility for default Python, skipping. "
+  echo "Install with 'pip install coverage'."
+fi
+
+echo
+echo "OK: all tests passed for ${#interpreters_tested[@]} interpreters: ${interpreters_tested[@]}."

tests/test.cpp

@@ -37,7 +37,7 @@ int testing_fails = 0;
 
 template<typename T, typename U>
 void TestEq(T expval, U val, const char *exp, const char *file, int line) {
-  if (expval != val) {
+  if (U(expval) != val) {
     auto expval_str = flatbuffers::NumToString(expval);
     auto val_str = flatbuffers::NumToString(val);
     TEST_OUTPUT_LINE("TEST FAILED: %s:%d, %s (%s) != %s", file, line,