diff --git a/docs/html/md__java_usage.html b/docs/html/md__java_usage.html index b9b9ee023..b3d353bc3 100644 --- a/docs/html/md__java_usage.html +++ b/docs/html/md__java_usage.html @@ -60,10 +60,11 @@ Monster monster = Monster.getRootAsMonster(bb); Vec3 pos = monster.pos();
Note that whenever you access a new object like in the pos
example above, a new temporary accessor object gets created. If your code is very performance sensitive (you iterate through a lot of objects), there's a second pos()
method to which you can pass a Vec3
object you've already created. This allows you to reuse it across many calls and reduce the amount of object allocation (and thus garbage collection) your program does.
Java does not support unsigned scalars. This means that any unsigned types you use in your schema will actually be represented as a signed value. This means all bits are still present, but may represent a negative value when used. For example, to read a byte b
as an unsigned number, you can do: (short)(b & 0xFF)
Sadly the string accessors currently always create a new string when accessed, since FlatBuffer's UTF-8 strings can't be read in-place by Java.
+The default string accessor (e.g. monster.name()
) currently always create a new Java String
when accessed, since FlatBuffer's UTF-8 strings can't be used in-place by String
. Alternatively, use monster.nameAsByteBuffer()
which returns a ByteBuffer
referring to the UTF-8 data in the original ByteBuffer
, which is much more efficient. The ByteBuffer
's position
points to the first character, and its limit
to just after the last.
Vector access is also a bit different from C++: you pass an extra index to the vector field accessor. Then a second method with the same name suffixed by Length
let's you know the number of elements you can access:
for (int i = 0; i < monster.inventoryLength(); i++) monster.inventory(i); // do something here -
If you specified a file_indentifier in the schema, you can query if the buffer is of the desired type before accessing it using:
if (Monster.MonsterBufferHasIdentifier(bb, start)) ... +
Alternatively, much like strings, you can use monster.inventoryAsByteBuffer()
to get a ByteBuffer
referring to the whole vector. Use ByteBuffer
methods like asFloatBuffer
to get specific views if needed.
If you specified a file_indentifier in the schema, you can query if the buffer is of the desired type before accessing it using:
if (Monster.MonsterBufferHasIdentifier(bb)) ...
You can also construct these buffers in Java using the static methods found in the generated code, and the FlatBufferBuilder class:
FlatBufferBuilder fbb = new FlatBufferBuilder();
Create strings:
int str = fbb.createString("MyMonster"); @@ -86,7 +87,7 @@ int inv = fbb.endVector();
You can use the generated method startInventoryVector
to conveniently call startVector
with the right element size. You pass the number of elements you want to write. You write the elements backwards since the buffer is being constructed back to front.
There are add
functions for all the scalar types. You use addOffset
for any previously constructed objects (such as other tables, strings, vectors). For structs, you use the appropriate create
function in-line, as shown above in the Monster
example.
To finish the buffer, call:
Monster.finishMonsterBuffer(fbb, mon); -
The buffer is now ready to be transmitted. It is contained in the ByteBuffer
which you can obtain from fbb.dataBuffer()
. Importantly, the valid data does not start from offset 0 in this buffer, but from fbb.dataStart()
(this is because the data was built backwards in memory). It ends at fbb,capacity()
.
The buffer is now ready to be transmitted. It is contained in the ByteBuffer
which you can obtain from fbb.dataBuffer()
. Importantly, the valid data does not start from offset 0 in this buffer, but from fbb.dataBuffer().position()
(this is because the data was built backwards in memory). It ends at fbb.capacity()
.
There currently is no support for parsing text (Schema's and JSON) directly from Java, though you could use the C++ parser through JNI. Please see the C++ documentation for more on text parsing.
diff --git a/docs/source/JavaUsage.md b/docs/source/JavaUsage.md index 15bd816dd..0c1e2d525 100755 --- a/docs/source/JavaUsage.md +++ b/docs/source/JavaUsage.md @@ -1,7 +1,7 @@ # Use in Java -FlatBuffers supports reading and writing binary FlatBuffers in Java. Generate code -for Java with the `-j` option to `flatc`. +FlatBuffers supports reading and writing binary FlatBuffers in Java. Generate +code for Java with the `-j` option to `flatc`. See `javaTest.java` for an example. Essentially, you read a FlatBuffer binary file into a `byte[]`, which you then turn into a `ByteBuffer`, which you pass to @@ -19,8 +19,8 @@ Note that whenever you access a new object like in the `pos` example above, a new temporary accessor object gets created. If your code is very performance sensitive (you iterate through a lot of objects), there's a second `pos()` method to which you can pass a `Vec3` object you've already created. This allows -you to reuse it across many calls and reduce the amount of object allocation (and -thus garbage collection) your program does. +you to reuse it across many calls and reduce the amount of object allocation +(and thus garbage collection) your program does. Java does not support unsigned scalars. This means that any unsigned types you use in your schema will actually be represented as a signed value. This means @@ -28,8 +28,12 @@ all bits are still present, but may represent a negative value when used. For example, to read a `byte b` as an unsigned number, you can do: `(short)(b & 0xFF)` -Sadly the string accessors currently always create a new string when accessed, -since FlatBuffer's UTF-8 strings can't be read in-place by Java. +The default string accessor (e.g. `monster.name()`) currently always create +a new Java `String` when accessed, since FlatBuffer's UTF-8 strings can't be +used in-place by `String`. Alternatively, use `monster.nameAsByteBuffer()` +which returns a `ByteBuffer` referring to the UTF-8 data in the original +`ByteBuffer`, which is much more efficient. The `ByteBuffer`'s `position` +points to the first character, and its `limit` to just after the last. Vector access is also a bit different from C++: you pass an extra index to the vector field accessor. Then a second method with the same name @@ -38,10 +42,14 @@ suffixed by `Length` let's you know the number of elements you can access: for (int i = 0; i < monster.inventoryLength(); i++) monster.inventory(i); // do something here +Alternatively, much like strings, you can use `monster.inventoryAsByteBuffer()` +to get a `ByteBuffer` referring to the whole vector. Use `ByteBuffer` methods +like `asFloatBuffer` to get specific views if needed. + If you specified a file_indentifier in the schema, you can query if the buffer is of the desired type before accessing it using: - if (Monster.MonsterBufferHasIdentifier(bb, start)) ... + if (Monster.MonsterBufferHasIdentifier(bb)) ... ## Buffer construction in Java @@ -105,8 +113,9 @@ To finish the buffer, call: The buffer is now ready to be transmitted. It is contained in the `ByteBuffer` which you can obtain from `fbb.dataBuffer()`. Importantly, the valid data does -not start from offset 0 in this buffer, but from `fbb.dataStart()` (this is -because the data was built backwards in memory). It ends at `fbb,capacity()`. +not start from offset 0 in this buffer, but from `fbb.dataBuffer().position()` +(this is because the data was built backwards in memory). +It ends at `fbb.capacity()`. ## Text Parsing diff --git a/java/flatbuffers/FlatBufferBuilder.java b/java/flatbuffers/FlatBufferBuilder.java index 4623661b5..c1fa7711b 100755 --- a/java/flatbuffers/FlatBufferBuilder.java +++ b/java/flatbuffers/FlatBufferBuilder.java @@ -245,6 +245,7 @@ public class FlatBufferBuilder { public void finish(int root_table) { prep(minalign, SIZEOF_INT); addOffset(root_table); + bb.position(space); } public void finish(int root_table, String file_identifier) { @@ -255,13 +256,20 @@ public class FlatBufferBuilder { for (int i = FILE_IDENTIFIER_LENGTH - 1; i >= 0; i--) { addByte((byte)file_identifier.charAt(i)); } - addOffset(root_table); + finish(root_table); } + // Get the ByteBuffer representing the FlatBuffer. Only call this after you've + // called finish(). The actual data starts at the ByteBuffer's current position, + // not necessarily at 0. public ByteBuffer dataBuffer() { return bb; } - // The FlatBuffer data doesn't start at offset 0 in the ByteBuffer: - public int dataStart() { + // The FlatBuffer data doesn't start at offset 0 in the ByteBuffer, + // but now the ByteBuffer's position is set to that location upon + // finish(). This method should not be needed anymore, but is left + // here as private for the moment to document this API change. + // It will be removed in the future. + private int dataStart() { return space; } @@ -273,7 +281,7 @@ public class FlatBufferBuilder { } // Utility function for copying a byte array that starts at 0. - public byte[] sizedByteArray(){ + public byte[] sizedByteArray() { return sizedByteArray(space, bb.capacity() - space); } } diff --git a/java/flatbuffers/Table.java b/java/flatbuffers/Table.java index 4daafff70..d48a325fb 100755 --- a/java/flatbuffers/Table.java +++ b/java/flatbuffers/Table.java @@ -38,6 +38,10 @@ public class Table { } // Create a java String from UTF-8 data stored inside the flatbuffer. + // This allocates a new string and converts to wide chars upon each access, + // which is not very efficient. Instead, each FlatBuffer string also comes with an + // accessor based on __vector_as_bytebuffer below, which is much more efficient, + // assuming your Java program can handle UTF-8 data directly. protected String __string(int offset) { offset += bb.getInt(offset); if (bb.hasArray()) { @@ -45,10 +49,11 @@ public class Table { } else { // We can't access .array(), since the ByteBuffer is read-only. // We're forced to make an extra copy: - bb.position(offset + SIZEOF_INT); byte[] copy = new byte[bb.getInt(offset)]; + int old_pos = bb.position(); + bb.position(offset + SIZEOF_INT); bb.get(copy); - bb.position(0); + bb.position(old_pos); return new String(copy, 0, copy.length, Charset.forName("UTF-8")); } } @@ -66,6 +71,21 @@ public class Table { return offset + bb.getInt(offset) + SIZEOF_INT; // data starts after the length } + // Get a whole vector as a ByteBuffer. This is efficient, since it only allocates a new + // bytebuffer object, but does not actually copy the data, it still refers to the same + // bytes as the original ByteBuffer. + // Also useful with nested FlatBuffers etc. + protected ByteBuffer __vector_as_bytebuffer(int vector_offset, int elem_size) { + int o = __offset(vector_offset); + if (o == 0) return null; + int old_pos = bb.position(); + bb.position(__vector(o)); + ByteBuffer nbb = bb.slice(); + bb.position(old_pos); + nbb.limit(__vector_len(o) * elem_size); + return nbb; + } + // Initialize any Table-derived type to point to the union at the given offset. protected Table __union(Table t, int offset) { offset += bb_pos; @@ -74,12 +94,12 @@ public class Table { return t; } - protected static boolean __has_identifier(ByteBuffer bb, int offset, String ident) { + protected static boolean __has_identifier(ByteBuffer bb, String ident) { if (ident.length() != FILE_IDENTIFIER_LENGTH) throw new AssertionError("FlatBuffers: file identifier must be length " + FILE_IDENTIFIER_LENGTH); for (int i = 0; i < FILE_IDENTIFIER_LENGTH; i++) { - if (ident.charAt(i) != (char)bb.get(offset + SIZEOF_INT + i)) return false; + if (ident.charAt(i) != (char)bb.get(bb.position() + SIZEOF_INT + i)) return false; } return true; } diff --git a/src/idl_gen_java.cpp b/src/idl_gen_java.cpp index 8ee42006d..91847b533 100755 --- a/src/idl_gen_java.cpp +++ b/src/idl_gen_java.cpp @@ -174,16 +174,16 @@ static void GenStruct(const Parser &parser, StructDef &struct_def, // of a FlatBuffer code += " public static " + struct_def.name + " getRootAs"; code += struct_def.name; - code += "(ByteBuffer _bb, int offset) { "; + code += "(ByteBuffer _bb) { "; code += "_bb.order(ByteOrder.LITTLE_ENDIAN); "; code += "return (new " + struct_def.name; - code += "()).__init(_bb.getInt(offset) + offset, _bb); }\n"; + code += "()).__init(_bb.getInt(_bb.position()) + _bb.position(), _bb); }\n"; if (parser.root_struct_def == &struct_def) { if (parser.file_identifier_.length()) { // Check if a buffer has the identifier. code += " public static boolean " + struct_def.name; - code += "BufferHasIdentifier(ByteBuffer _bb, int offset) { return "; - code += "__has_identifier(_bb, offset, \"" + parser.file_identifier_; + code += "BufferHasIdentifier(ByteBuffer _bb) { return "; + code += "__has_identifier(_bb, \"" + parser.file_identifier_; code += "\"); }\n"; } } @@ -285,6 +285,15 @@ static void GenStruct(const Parser &parser, StructDef &struct_def, code += offset_prefix; code += "__vector_len(o) : 0; }\n"; } + if (field.value.type.base_type == BASE_TYPE_VECTOR || + field.value.type.base_type == BASE_TYPE_STRING) { + code += " public ByteBuffer " + MakeCamel(field.name, false); + code += "AsByteBuffer() { return __vector_as_bytebuffer("; + code += NumToString(field.value.offset) + ", "; + code += NumToString(field.value.type.base_type == BASE_TYPE_STRING ? 1 : + InlineSize(field.value.type.VectorType())); + code += "); }\n"; + } } code += "\n"; if (struct_def.fixed) { diff --git a/tests/JavaTest.java b/tests/JavaTest.java index 04646c587..74d68db13 100755 --- a/tests/JavaTest.java +++ b/tests/JavaTest.java @@ -41,7 +41,7 @@ class JavaTest { // Now test it: ByteBuffer bb = ByteBuffer.wrap(data); - TestBuffer(bb, 0); + TestBuffer(bb); // Second, let's create a FlatBuffer from scratch in Java, and test it also. // We use an initial size of 1 to exercise the reallocation algorithm, @@ -95,7 +95,7 @@ class JavaTest { try { DataOutputStream os = new DataOutputStream(new FileOutputStream( "monsterdata_java_wire.bin")); - os.write(fbb.dataBuffer().array(), fbb.dataStart(), fbb.offset()); + os.write(fbb.dataBuffer().array(), fbb.dataBuffer().position(), fbb.offset()); os.close(); } catch(java.io.IOException e) { System.out.println("FlatBuffers test: couldn't write file"); @@ -103,20 +103,20 @@ class JavaTest { } // Test it: - TestBuffer(fbb.dataBuffer(), fbb.dataStart()); + TestBuffer(fbb.dataBuffer()); // Make sure it also works with read only ByteBuffers. This is slower, // since creating strings incurs an additional copy // (see Table.__string). - TestBuffer(fbb.dataBuffer().asReadOnlyBuffer(), fbb.dataStart()); + TestBuffer(fbb.dataBuffer().asReadOnlyBuffer()); System.out.println("FlatBuffers test: completed successfully"); } - static void TestBuffer(ByteBuffer bb, int start) { - TestEq(Monster.MonsterBufferHasIdentifier(bb, start), true); + static void TestBuffer(ByteBuffer bb) { + TestEq(Monster.MonsterBufferHasIdentifier(bb), true); - Monster monster = Monster.getRootAsMonster(bb, start); + Monster monster = Monster.getRootAsMonster(bb); TestEq(monster.hp(), (short)80); TestEq(monster.mana(), (short)150); // default @@ -145,6 +145,13 @@ class JavaTest { invsum += monster.inventory(i); TestEq(invsum, 10); + // Alternative way of accessing a vector: + ByteBuffer ibb = monster.inventoryAsByteBuffer(); + invsum = 0; + while (ibb.position() < ibb.limit()) + invsum += ibb.get(); + TestEq(invsum, 10); + Test test_0 = monster.test4(0); Test test_1 = monster.test4(1); TestEq(monster.test4Length(), 2); diff --git a/tests/MyGame/Example/Monster.java b/tests/MyGame/Example/Monster.java index 76f94760e..fa7bc21e5 100755 --- a/tests/MyGame/Example/Monster.java +++ b/tests/MyGame/Example/Monster.java @@ -8,8 +8,8 @@ import java.util.*; import flatbuffers.*; public class Monster extends Table { - public static Monster getRootAsMonster(ByteBuffer _bb, int offset) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (new Monster()).__init(_bb.getInt(offset) + offset, _bb); } - public static boolean MonsterBufferHasIdentifier(ByteBuffer _bb, int offset) { return __has_identifier(_bb, offset, "MONS"); } + public static Monster getRootAsMonster(ByteBuffer _bb) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (new Monster()).__init(_bb.getInt(_bb.position()) + _bb.position(), _bb); } + public static boolean MonsterBufferHasIdentifier(ByteBuffer _bb) { return __has_identifier(_bb, "MONS"); } public Monster __init(int _i, ByteBuffer _bb) { bb_pos = _i; bb = _bb; return this; } public Vec3 pos() { return pos(new Vec3()); } @@ -17,24 +17,30 @@ public class Monster extends Table { public short mana() { int o = __offset(6); return o != 0 ? bb.getShort(o + bb_pos) : 150; } public short hp() { int o = __offset(8); return o != 0 ? bb.getShort(o + bb_pos) : 100; } public String name() { int o = __offset(10); return o != 0 ? __string(o + bb_pos) : null; } + public ByteBuffer nameAsByteBuffer() { return __vector_as_bytebuffer(10, 1); } public byte inventory(int j) { int o = __offset(14); return o != 0 ? bb.get(__vector(o) + j * 1) : 0; } public int inventoryLength() { int o = __offset(14); return o != 0 ? __vector_len(o) : 0; } + public ByteBuffer inventoryAsByteBuffer() { return __vector_as_bytebuffer(14, 1); } public byte color() { int o = __offset(16); return o != 0 ? bb.get(o + bb_pos) : 8; } public byte testType() { int o = __offset(18); return o != 0 ? bb.get(o + bb_pos) : 0; } public Table test(Table obj) { int o = __offset(20); return o != 0 ? __union(obj, o) : null; } public Test test4(int j) { return test4(new Test(), j); } public Test test4(Test obj, int j) { int o = __offset(22); return o != 0 ? obj.__init(__vector(o) + j * 4, bb) : null; } public int test4Length() { int o = __offset(22); return o != 0 ? __vector_len(o) : 0; } + public ByteBuffer test4AsByteBuffer() { return __vector_as_bytebuffer(22, 4); } public String testarrayofstring(int j) { int o = __offset(24); return o != 0 ? __string(__vector(o) + j * 4) : null; } public int testarrayofstringLength() { int o = __offset(24); return o != 0 ? __vector_len(o) : 0; } + public ByteBuffer testarrayofstringAsByteBuffer() { return __vector_as_bytebuffer(24, 4); } /// an example documentation comment: this will end up in the generated code multiline too public Monster testarrayoftables(int j) { return testarrayoftables(new Monster(), j); } public Monster testarrayoftables(Monster obj, int j) { int o = __offset(26); return o != 0 ? obj.__init(__indirect(__vector(o) + j * 4), bb) : null; } public int testarrayoftablesLength() { int o = __offset(26); return o != 0 ? __vector_len(o) : 0; } + public ByteBuffer testarrayoftablesAsByteBuffer() { return __vector_as_bytebuffer(26, 4); } public Monster enemy() { return enemy(new Monster()); } public Monster enemy(Monster obj) { int o = __offset(28); return o != 0 ? obj.__init(__indirect(o + bb_pos), bb) : null; } public byte testnestedflatbuffer(int j) { int o = __offset(30); return o != 0 ? bb.get(__vector(o) + j * 1) : 0; } public int testnestedflatbufferLength() { int o = __offset(30); return o != 0 ? __vector_len(o) : 0; } + public ByteBuffer testnestedflatbufferAsByteBuffer() { return __vector_as_bytebuffer(30, 1); } public Monster testempty() { return testempty(new Monster()); } public Monster testempty(Monster obj) { int o = __offset(32); return o != 0 ? obj.__init(__indirect(o + bb_pos), bb) : null; }