Added accessor in Java to get vectors as ByteBuffers.
Also cleaned up ByteBuffer usage in general: ByteBuffer.position now universally indicates the start of a ByteBuffer. Change-Id: Ic4bfb98f9df9501b8fc82de2c45db7d7311135ac Tested: on Linux.
This commit is contained in:
Wouter van Oortmerssen
parent
32f2c1c3b9
commit
858e9961e2
|
|
@ -60,10 +60,11 @@ Monster monster = Monster.getRootAsMonster(bb);
|
|||
Vec3 pos = monster.pos();
|
||||
</pre><p>Note that whenever you access a new object like in the <code>pos</code> example above, a new temporary accessor object gets created. If your code is very performance sensitive (you iterate through a lot of objects), there's a second <code>pos()</code> method to which you can pass a <code>Vec3</code> object you've already created. This allows you to reuse it across many calls and reduce the amount of object allocation (and thus garbage collection) your program does.</p>
|
||||
<p>Java does not support unsigned scalars. This means that any unsigned types you use in your schema will actually be represented as a signed value. This means all bits are still present, but may represent a negative value when used. For example, to read a <code>byte b</code> as an unsigned number, you can do: <code>(short)(b & 0xFF)</code></p>
|
||||
<p>Sadly the string accessors currently always create a new string when accessed, since FlatBuffer's UTF-8 strings can't be read in-place by Java.</p>
|
||||
<p>The default string accessor (e.g. <code>monster.name()</code>) currently always create a new Java <code>String</code> when accessed, since FlatBuffer's UTF-8 strings can't be used in-place by <code>String</code>. Alternatively, use <code>monster.nameAsByteBuffer()</code> which returns a <code>ByteBuffer</code> referring to the UTF-8 data in the original <code>ByteBuffer</code>, which is much more efficient. The <code>ByteBuffer</code>'s <code>position</code> points to the first character, and its <code>limit</code> to just after the last.</p>
|
||||
<p>Vector access is also a bit different from C++: you pass an extra index to the vector field accessor. Then a second method with the same name suffixed by <code>Length</code> let's you know the number of elements you can access: </p><pre class="fragment">for (int i = 0; i < monster.inventoryLength(); i++)
|
||||
monster.inventory(i); // do something here
|
||||
</pre><p>If you specified a file_indentifier in the schema, you can query if the buffer is of the desired type before accessing it using: </p><pre class="fragment">if (Monster.MonsterBufferHasIdentifier(bb, start)) ...
|
||||
</pre><p>Alternatively, much like strings, you can use <code>monster.inventoryAsByteBuffer()</code> to get a <code>ByteBuffer</code> referring to the whole vector. Use <code>ByteBuffer</code> methods like <code>asFloatBuffer</code> to get specific views if needed.</p>
|
||||
<p>If you specified a file_indentifier in the schema, you can query if the buffer is of the desired type before accessing it using: </p><pre class="fragment">if (Monster.MonsterBufferHasIdentifier(bb)) ...
|
||||
</pre><h2>Buffer construction in Java</h2>
|
||||
<p>You can also construct these buffers in Java using the static methods found in the generated code, and the FlatBufferBuilder class: </p><pre class="fragment">FlatBufferBuilder fbb = new FlatBufferBuilder();
|
||||
</pre><p>Create strings: </p><pre class="fragment">int str = fbb.createString("MyMonster");
|
||||
|
|
@ -86,7 +87,7 @@ int inv = fbb.endVector();
|
|||
</pre><p>You can use the generated method <code>startInventoryVector</code> to conveniently call <code>startVector</code> with the right element size. You pass the number of elements you want to write. You write the elements backwards since the buffer is being constructed back to front.</p>
|
||||
<p>There are <code>add</code> functions for all the scalar types. You use <code>addOffset</code> for any previously constructed objects (such as other tables, strings, vectors). For structs, you use the appropriate <code>create</code> function in-line, as shown above in the <code>Monster</code> example.</p>
|
||||
<p>To finish the buffer, call: </p><pre class="fragment">Monster.finishMonsterBuffer(fbb, mon);
|
||||
</pre><p>The buffer is now ready to be transmitted. It is contained in the <code>ByteBuffer</code> which you can obtain from <code>fbb.dataBuffer()</code>. Importantly, the valid data does not start from offset 0 in this buffer, but from <code>fbb.dataStart()</code> (this is because the data was built backwards in memory). It ends at <code>fbb,capacity()</code>.</p>
|
||||
</pre><p>The buffer is now ready to be transmitted. It is contained in the <code>ByteBuffer</code> which you can obtain from <code>fbb.dataBuffer()</code>. Importantly, the valid data does not start from offset 0 in this buffer, but from <code>fbb.dataBuffer().position()</code> (this is because the data was built backwards in memory). It ends at <code>fbb.capacity()</code>.</p>
|
||||
<h2>Text Parsing</h2>
|
||||
<p>There currently is no support for parsing text (Schema's and JSON) directly from Java, though you could use the C++ parser through JNI. Please see the C++ documentation for more on text parsing. </p>
|
||||
</div></div><!-- contents -->
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# Use in Java
|
||||
|
||||
FlatBuffers supports reading and writing binary FlatBuffers in Java. Generate code
|
||||
for Java with the `-j` option to `flatc`.
|
||||
FlatBuffers supports reading and writing binary FlatBuffers in Java. Generate
|
||||
code for Java with the `-j` option to `flatc`.
|
||||
|
||||
See `javaTest.java` for an example. Essentially, you read a FlatBuffer binary
|
||||
file into a `byte[]`, which you then turn into a `ByteBuffer`, which you pass to
|
||||
|
|
@ -19,8 +19,8 @@ Note that whenever you access a new object like in the `pos` example above,
|
|||
a new temporary accessor object gets created. If your code is very performance
|
||||
sensitive (you iterate through a lot of objects), there's a second `pos()`
|
||||
method to which you can pass a `Vec3` object you've already created. This allows
|
||||
you to reuse it across many calls and reduce the amount of object allocation (and
|
||||
thus garbage collection) your program does.
|
||||
you to reuse it across many calls and reduce the amount of object allocation
|
||||
(and thus garbage collection) your program does.
|
||||
|
||||
Java does not support unsigned scalars. This means that any unsigned types you
|
||||
use in your schema will actually be represented as a signed value. This means
|
||||
|
|
@ -28,8 +28,12 @@ all bits are still present, but may represent a negative value when used.
|
|||
For example, to read a `byte b` as an unsigned number, you can do:
|
||||
`(short)(b & 0xFF)`
|
||||
|
||||
Sadly the string accessors currently always create a new string when accessed,
|
||||
since FlatBuffer's UTF-8 strings can't be read in-place by Java.
|
||||
The default string accessor (e.g. `monster.name()`) currently always create
|
||||
a new Java `String` when accessed, since FlatBuffer's UTF-8 strings can't be
|
||||
used in-place by `String`. Alternatively, use `monster.nameAsByteBuffer()`
|
||||
which returns a `ByteBuffer` referring to the UTF-8 data in the original
|
||||
`ByteBuffer`, which is much more efficient. The `ByteBuffer`'s `position`
|
||||
points to the first character, and its `limit` to just after the last.
|
||||
|
||||
Vector access is also a bit different from C++: you pass an extra index
|
||||
to the vector field accessor. Then a second method with the same name
|
||||
|
|
@ -38,10 +42,14 @@ suffixed by `Length` let's you know the number of elements you can access:
|
|||
for (int i = 0; i < monster.inventoryLength(); i++)
|
||||
monster.inventory(i); // do something here
|
||||
|
||||
Alternatively, much like strings, you can use `monster.inventoryAsByteBuffer()`
|
||||
to get a `ByteBuffer` referring to the whole vector. Use `ByteBuffer` methods
|
||||
like `asFloatBuffer` to get specific views if needed.
|
||||
|
||||
If you specified a file_indentifier in the schema, you can query if the
|
||||
buffer is of the desired type before accessing it using:
|
||||
|
||||
if (Monster.MonsterBufferHasIdentifier(bb, start)) ...
|
||||
if (Monster.MonsterBufferHasIdentifier(bb)) ...
|
||||
|
||||
|
||||
## Buffer construction in Java
|
||||
|
|
@ -105,8 +113,9 @@ To finish the buffer, call:
|
|||
|
||||
The buffer is now ready to be transmitted. It is contained in the `ByteBuffer`
|
||||
which you can obtain from `fbb.dataBuffer()`. Importantly, the valid data does
|
||||
not start from offset 0 in this buffer, but from `fbb.dataStart()` (this is
|
||||
because the data was built backwards in memory). It ends at `fbb,capacity()`.
|
||||
not start from offset 0 in this buffer, but from `fbb.dataBuffer().position()`
|
||||
(this is because the data was built backwards in memory).
|
||||
It ends at `fbb.capacity()`.
|
||||
|
||||
|
||||
## Text Parsing
|
||||
|
|
|
|||
|
|
@ -245,6 +245,7 @@ public class FlatBufferBuilder {
|
|||
public void finish(int root_table) {
|
||||
prep(minalign, SIZEOF_INT);
|
||||
addOffset(root_table);
|
||||
bb.position(space);
|
||||
}
|
||||
|
||||
public void finish(int root_table, String file_identifier) {
|
||||
|
|
@ -255,13 +256,20 @@ public class FlatBufferBuilder {
|
|||
for (int i = FILE_IDENTIFIER_LENGTH - 1; i >= 0; i--) {
|
||||
addByte((byte)file_identifier.charAt(i));
|
||||
}
|
||||
addOffset(root_table);
|
||||
finish(root_table);
|
||||
}
|
||||
|
||||
// Get the ByteBuffer representing the FlatBuffer. Only call this after you've
|
||||
// called finish(). The actual data starts at the ByteBuffer's current position,
|
||||
// not necessarily at 0.
|
||||
public ByteBuffer dataBuffer() { return bb; }
|
||||
|
||||
// The FlatBuffer data doesn't start at offset 0 in the ByteBuffer:
|
||||
public int dataStart() {
|
||||
// The FlatBuffer data doesn't start at offset 0 in the ByteBuffer,
|
||||
// but now the ByteBuffer's position is set to that location upon
|
||||
// finish(). This method should not be needed anymore, but is left
|
||||
// here as private for the moment to document this API change.
|
||||
// It will be removed in the future.
|
||||
private int dataStart() {
|
||||
return space;
|
||||
}
|
||||
|
||||
|
|
@ -273,7 +281,7 @@ public class FlatBufferBuilder {
|
|||
}
|
||||
|
||||
// Utility function for copying a byte array that starts at 0.
|
||||
public byte[] sizedByteArray(){
|
||||
public byte[] sizedByteArray() {
|
||||
return sizedByteArray(space, bb.capacity() - space);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -38,6 +38,10 @@ public class Table {
|
|||
}
|
||||
|
||||
// Create a java String from UTF-8 data stored inside the flatbuffer.
|
||||
// This allocates a new string and converts to wide chars upon each access,
|
||||
// which is not very efficient. Instead, each FlatBuffer string also comes with an
|
||||
// accessor based on __vector_as_bytebuffer below, which is much more efficient,
|
||||
// assuming your Java program can handle UTF-8 data directly.
|
||||
protected String __string(int offset) {
|
||||
offset += bb.getInt(offset);
|
||||
if (bb.hasArray()) {
|
||||
|
|
@ -45,10 +49,11 @@ public class Table {
|
|||
} else {
|
||||
// We can't access .array(), since the ByteBuffer is read-only.
|
||||
// We're forced to make an extra copy:
|
||||
bb.position(offset + SIZEOF_INT);
|
||||
byte[] copy = new byte[bb.getInt(offset)];
|
||||
int old_pos = bb.position();
|
||||
bb.position(offset + SIZEOF_INT);
|
||||
bb.get(copy);
|
||||
bb.position(0);
|
||||
bb.position(old_pos);
|
||||
return new String(copy, 0, copy.length, Charset.forName("UTF-8"));
|
||||
}
|
||||
}
|
||||
|
|
@ -66,6 +71,21 @@ public class Table {
|
|||
return offset + bb.getInt(offset) + SIZEOF_INT; // data starts after the length
|
||||
}
|
||||
|
||||
// Get a whole vector as a ByteBuffer. This is efficient, since it only allocates a new
|
||||
// bytebuffer object, but does not actually copy the data, it still refers to the same
|
||||
// bytes as the original ByteBuffer.
|
||||
// Also useful with nested FlatBuffers etc.
|
||||
protected ByteBuffer __vector_as_bytebuffer(int vector_offset, int elem_size) {
|
||||
int o = __offset(vector_offset);
|
||||
if (o == 0) return null;
|
||||
int old_pos = bb.position();
|
||||
bb.position(__vector(o));
|
||||
ByteBuffer nbb = bb.slice();
|
||||
bb.position(old_pos);
|
||||
nbb.limit(__vector_len(o) * elem_size);
|
||||
return nbb;
|
||||
}
|
||||
|
||||
// Initialize any Table-derived type to point to the union at the given offset.
|
||||
protected Table __union(Table t, int offset) {
|
||||
offset += bb_pos;
|
||||
|
|
@ -74,12 +94,12 @@ public class Table {
|
|||
return t;
|
||||
}
|
||||
|
||||
protected static boolean __has_identifier(ByteBuffer bb, int offset, String ident) {
|
||||
protected static boolean __has_identifier(ByteBuffer bb, String ident) {
|
||||
if (ident.length() != FILE_IDENTIFIER_LENGTH)
|
||||
throw new AssertionError("FlatBuffers: file identifier must be length " +
|
||||
FILE_IDENTIFIER_LENGTH);
|
||||
for (int i = 0; i < FILE_IDENTIFIER_LENGTH; i++) {
|
||||
if (ident.charAt(i) != (char)bb.get(offset + SIZEOF_INT + i)) return false;
|
||||
if (ident.charAt(i) != (char)bb.get(bb.position() + SIZEOF_INT + i)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -174,16 +174,16 @@ static void GenStruct(const Parser &parser, StructDef &struct_def,
|
|||
// of a FlatBuffer
|
||||
code += " public static " + struct_def.name + " getRootAs";
|
||||
code += struct_def.name;
|
||||
code += "(ByteBuffer _bb, int offset) { ";
|
||||
code += "(ByteBuffer _bb) { ";
|
||||
code += "_bb.order(ByteOrder.LITTLE_ENDIAN); ";
|
||||
code += "return (new " + struct_def.name;
|
||||
code += "()).__init(_bb.getInt(offset) + offset, _bb); }\n";
|
||||
code += "()).__init(_bb.getInt(_bb.position()) + _bb.position(), _bb); }\n";
|
||||
if (parser.root_struct_def == &struct_def) {
|
||||
if (parser.file_identifier_.length()) {
|
||||
// Check if a buffer has the identifier.
|
||||
code += " public static boolean " + struct_def.name;
|
||||
code += "BufferHasIdentifier(ByteBuffer _bb, int offset) { return ";
|
||||
code += "__has_identifier(_bb, offset, \"" + parser.file_identifier_;
|
||||
code += "BufferHasIdentifier(ByteBuffer _bb) { return ";
|
||||
code += "__has_identifier(_bb, \"" + parser.file_identifier_;
|
||||
code += "\"); }\n";
|
||||
}
|
||||
}
|
||||
|
|
@ -285,6 +285,15 @@ static void GenStruct(const Parser &parser, StructDef &struct_def,
|
|||
code += offset_prefix;
|
||||
code += "__vector_len(o) : 0; }\n";
|
||||
}
|
||||
if (field.value.type.base_type == BASE_TYPE_VECTOR ||
|
||||
field.value.type.base_type == BASE_TYPE_STRING) {
|
||||
code += " public ByteBuffer " + MakeCamel(field.name, false);
|
||||
code += "AsByteBuffer() { return __vector_as_bytebuffer(";
|
||||
code += NumToString(field.value.offset) + ", ";
|
||||
code += NumToString(field.value.type.base_type == BASE_TYPE_STRING ? 1 :
|
||||
InlineSize(field.value.type.VectorType()));
|
||||
code += "); }\n";
|
||||
}
|
||||
}
|
||||
code += "\n";
|
||||
if (struct_def.fixed) {
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ class JavaTest {
|
|||
// Now test it:
|
||||
|
||||
ByteBuffer bb = ByteBuffer.wrap(data);
|
||||
TestBuffer(bb, 0);
|
||||
TestBuffer(bb);
|
||||
|
||||
// Second, let's create a FlatBuffer from scratch in Java, and test it also.
|
||||
// We use an initial size of 1 to exercise the reallocation algorithm,
|
||||
|
|
@ -95,7 +95,7 @@ class JavaTest {
|
|||
try {
|
||||
DataOutputStream os = new DataOutputStream(new FileOutputStream(
|
||||
"monsterdata_java_wire.bin"));
|
||||
os.write(fbb.dataBuffer().array(), fbb.dataStart(), fbb.offset());
|
||||
os.write(fbb.dataBuffer().array(), fbb.dataBuffer().position(), fbb.offset());
|
||||
os.close();
|
||||
} catch(java.io.IOException e) {
|
||||
System.out.println("FlatBuffers test: couldn't write file");
|
||||
|
|
@ -103,20 +103,20 @@ class JavaTest {
|
|||
}
|
||||
|
||||
// Test it:
|
||||
TestBuffer(fbb.dataBuffer(), fbb.dataStart());
|
||||
TestBuffer(fbb.dataBuffer());
|
||||
|
||||
// Make sure it also works with read only ByteBuffers. This is slower,
|
||||
// since creating strings incurs an additional copy
|
||||
// (see Table.__string).
|
||||
TestBuffer(fbb.dataBuffer().asReadOnlyBuffer(), fbb.dataStart());
|
||||
TestBuffer(fbb.dataBuffer().asReadOnlyBuffer());
|
||||
|
||||
System.out.println("FlatBuffers test: completed successfully");
|
||||
}
|
||||
|
||||
static void TestBuffer(ByteBuffer bb, int start) {
|
||||
TestEq(Monster.MonsterBufferHasIdentifier(bb, start), true);
|
||||
static void TestBuffer(ByteBuffer bb) {
|
||||
TestEq(Monster.MonsterBufferHasIdentifier(bb), true);
|
||||
|
||||
Monster monster = Monster.getRootAsMonster(bb, start);
|
||||
Monster monster = Monster.getRootAsMonster(bb);
|
||||
|
||||
TestEq(monster.hp(), (short)80);
|
||||
TestEq(monster.mana(), (short)150); // default
|
||||
|
|
@ -145,6 +145,13 @@ class JavaTest {
|
|||
invsum += monster.inventory(i);
|
||||
TestEq(invsum, 10);
|
||||
|
||||
// Alternative way of accessing a vector:
|
||||
ByteBuffer ibb = monster.inventoryAsByteBuffer();
|
||||
invsum = 0;
|
||||
while (ibb.position() < ibb.limit())
|
||||
invsum += ibb.get();
|
||||
TestEq(invsum, 10);
|
||||
|
||||
Test test_0 = monster.test4(0);
|
||||
Test test_1 = monster.test4(1);
|
||||
TestEq(monster.test4Length(), 2);
|
||||
|
|
|
|||
|
|
@ -8,8 +8,8 @@ import java.util.*;
|
|||
import flatbuffers.*;
|
||||
|
||||
public class Monster extends Table {
|
||||
public static Monster getRootAsMonster(ByteBuffer _bb, int offset) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (new Monster()).__init(_bb.getInt(offset) + offset, _bb); }
|
||||
public static boolean MonsterBufferHasIdentifier(ByteBuffer _bb, int offset) { return __has_identifier(_bb, offset, "MONS"); }
|
||||
public static Monster getRootAsMonster(ByteBuffer _bb) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (new Monster()).__init(_bb.getInt(_bb.position()) + _bb.position(), _bb); }
|
||||
public static boolean MonsterBufferHasIdentifier(ByteBuffer _bb) { return __has_identifier(_bb, "MONS"); }
|
||||
public Monster __init(int _i, ByteBuffer _bb) { bb_pos = _i; bb = _bb; return this; }
|
||||
|
||||
public Vec3 pos() { return pos(new Vec3()); }
|
||||
|
|
@ -17,24 +17,30 @@ public class Monster extends Table {
|
|||
public short mana() { int o = __offset(6); return o != 0 ? bb.getShort(o + bb_pos) : 150; }
|
||||
public short hp() { int o = __offset(8); return o != 0 ? bb.getShort(o + bb_pos) : 100; }
|
||||
public String name() { int o = __offset(10); return o != 0 ? __string(o + bb_pos) : null; }
|
||||
public ByteBuffer nameAsByteBuffer() { return __vector_as_bytebuffer(10, 1); }
|
||||
public byte inventory(int j) { int o = __offset(14); return o != 0 ? bb.get(__vector(o) + j * 1) : 0; }
|
||||
public int inventoryLength() { int o = __offset(14); return o != 0 ? __vector_len(o) : 0; }
|
||||
public ByteBuffer inventoryAsByteBuffer() { return __vector_as_bytebuffer(14, 1); }
|
||||
public byte color() { int o = __offset(16); return o != 0 ? bb.get(o + bb_pos) : 8; }
|
||||
public byte testType() { int o = __offset(18); return o != 0 ? bb.get(o + bb_pos) : 0; }
|
||||
public Table test(Table obj) { int o = __offset(20); return o != 0 ? __union(obj, o) : null; }
|
||||
public Test test4(int j) { return test4(new Test(), j); }
|
||||
public Test test4(Test obj, int j) { int o = __offset(22); return o != 0 ? obj.__init(__vector(o) + j * 4, bb) : null; }
|
||||
public int test4Length() { int o = __offset(22); return o != 0 ? __vector_len(o) : 0; }
|
||||
public ByteBuffer test4AsByteBuffer() { return __vector_as_bytebuffer(22, 4); }
|
||||
public String testarrayofstring(int j) { int o = __offset(24); return o != 0 ? __string(__vector(o) + j * 4) : null; }
|
||||
public int testarrayofstringLength() { int o = __offset(24); return o != 0 ? __vector_len(o) : 0; }
|
||||
public ByteBuffer testarrayofstringAsByteBuffer() { return __vector_as_bytebuffer(24, 4); }
|
||||
/// an example documentation comment: this will end up in the generated code multiline too
|
||||
public Monster testarrayoftables(int j) { return testarrayoftables(new Monster(), j); }
|
||||
public Monster testarrayoftables(Monster obj, int j) { int o = __offset(26); return o != 0 ? obj.__init(__indirect(__vector(o) + j * 4), bb) : null; }
|
||||
public int testarrayoftablesLength() { int o = __offset(26); return o != 0 ? __vector_len(o) : 0; }
|
||||
public ByteBuffer testarrayoftablesAsByteBuffer() { return __vector_as_bytebuffer(26, 4); }
|
||||
public Monster enemy() { return enemy(new Monster()); }
|
||||
public Monster enemy(Monster obj) { int o = __offset(28); return o != 0 ? obj.__init(__indirect(o + bb_pos), bb) : null; }
|
||||
public byte testnestedflatbuffer(int j) { int o = __offset(30); return o != 0 ? bb.get(__vector(o) + j * 1) : 0; }
|
||||
public int testnestedflatbufferLength() { int o = __offset(30); return o != 0 ? __vector_len(o) : 0; }
|
||||
public ByteBuffer testnestedflatbufferAsByteBuffer() { return __vector_as_bytebuffer(30, 1); }
|
||||
public Monster testempty() { return testempty(new Monster()); }
|
||||
public Monster testempty(Monster obj) { int o = __offset(32); return o != 0 ? obj.__init(__indirect(o + bb_pos), bb) : null; }
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue