* First working hack of adding 64-bit. Don't judge :)
* Made vector_downward work on 64 bit types
* vector_downward uses size_t, added offset64 to reflection
* cleaned up adding offset64 in parser
* Add C++ testing skeleton for 64-bit
* working test for CreateVector64
* working >2 GiB buffers
* support for large strings
* simplified CreateString<> to just provide the offset type
* generalize CreateVector template
* update test_64.afb due to upstream format change
* Added Vector64 type, which is just an alias for vector ATM
* Switch to Offset64 for Vector64
* Update for reflection bfbs output change
* Starting to add support for vector64 type in C++
* made a generic CreateVector that can handle different offsets and vector types
* Support for 32-vector with 64-addressing
* Vector64 basic builder + tests working
* basic support for json vector64 support
* renamed fields in test_64bit.fbs to better reflect their use
* working C++ vector64 builder
* Apply --annotate-sparse-vector to 64-bit tests
* Enable Vector64 for --annotate-sparse-vectors
* Merged from upstream
* Add `near_string` field for testing 32-bit offsets alongside
* keep track of where the 32-bit and 64-bit regions are for flatbufferbuilder
* move template<> outside class body for GCC
* update run.sh to build and run tests
* basic assertion for adding 64-bit offset at the wrong time
* started to separate `FlatBufferBuilder` into two classes, 1 64-bit aware, the other not
* add test for nested flatbuffer vector64, fix bug in alignment of big vectors
* fixed CreateDirect method by iterating by Offset64 first
* internal refactoring of flatbufferbuilder
* block not supported languages in the parser from using 64-bit
* evolution tests for adding a vector64 field
* conformity tests for adding/removing offset64 attributes
* ensure test is for a big buffer
* add parser error tests for `offset64` and `vector64` attributes
* add missing static that GCC only complains about
* remove stdint-uintn.h header that gets automatically added
* move 64-bit CalculateOffset internal
* fixed return size of EndVector
* various fixes on windows
* add SizeT to vector_downward
* minimze range of size changes in vector and builder
* reworked how tracking if 64-offsets are added
* Add ReturnT to EndVector
* small cleanups
* remove need for second Array definition
* combine IndirectHelpers into one definition
* started support for vector of struct
* Support for 32/64-vectors of structs + Offset64
* small cleanups
* add verification for vector64
* add sized prefix for 64-bit buffers
* add fuzzer for 64-bit
* add example of adding many vectors using a wrapper table
* run the new -bfbs-gen-embed logic on the 64-bit tests
* remove run.sh and fix cmakelist issue
* fixed bazel rules
* fixed some PR comments
* add 64-bit tests to cmakelist
The distributions for C++ and Python include the generated reflection
bindings but are currently missing from the other language packages.
This will bring the Java package generated for releases closer to
feature parity with the C++ and Python release artifacts.
As Java does not support unsigned integer types, the value types
are "rounded up" (an uint32 is represented as a long) but persisted
correctly (an uint32 is persisted as 4 bytes).
This CL makes a cast operation explicit so that the compiler
does not throw warning messages.
Co-authored-by: Dominic Battre <battre@chromium.org>
Co-authored-by: Derek Bailey <derekbailey@google.com>
The MyGame/Example/LongEnum.java class did not compile because
Java expects an "L" suffix for literals of type long.
This CL fixes the code generation to include such a suffix.
Co-authored-by: Dominic Battre <battre@chromium.org>
* Start of mvn-ification of the test
* move to right locations
* Update the IO done in the test to read from resources / write to temp folders
* Add github workflow attempt to mvn test it instead of JavaTest.sh
* Pin the Kotlin benchmark's symlink for /java to the right location
* Inline equality assertions and format JavaTest.java
* fix android gradle source directory
Co-authored-by: Derek Bailey <derekbailey@google.com>
Untyped fixed vectors are not supported in FlexBuffers. There
was an assert to check for it, but on java, asserts are optional. This
change converts the assertion into a runtime exception.
Fixes#7358
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Revert "avoiding even more NoSuchMethod exceptions (#6729)"
This reverts commit 6fb2c90d9e.
* Revert "avoiding more NoSuchMethod exceptions (#6671)"
This reverts commit 752c7b576d.
* Revert "avoiding NoSuchMethod exception (#6658)"
This reverts commit 813d3632ec.
* Use Java 8 for Kotlin Linux builds to verify
Related with issue #6113.
`(old_buf_size & 0xC0000000) != 0` checks if we can duplicate old_buf_size
and still be under 2GB (by checking if bit 30 or 31 is 1). This doesn't
allow buffers larger than 1GB.
The strategy now is to allocate a buffer with the maximum array size
when we detect that we are overflowing the 2GB.
Also changed default buffer size to 1024.
Added a method FlatBufferBuilder::createSharedString that
enable string sharing for building messages on java.
The shared pool will only contains strings inserted by
this methods.
* [Java] Grow ArrayReadWriteBuf enough to match requested capacity.
Also handle requested capacity overflowing.
* [Java] Improve readability of ArrayReadWriteBuf.requestCapacity()
* [Java] prepare testBuilderGrowth() to fail again once FlexBuffersBuilder uses ByteBufferReadWriteBuf internally, add TestFail() as a better alternative to a plain "assert false"
* [Java] Revert some test changes and extract a small string that was used more than once; one could say three times, however, it might be worthy of a discussion if the third occasion also falls into this category, as it is an independent use case and thus would work in the same way even if the value changed.
Co-authored-by: Markus <markus@greenrobot>
The original implementation of map access is very naive:
- Encode String to UTF8 byte[]
- Creates a new KeyVector
- Performs a binary search to find the key
- return value
So every access to the Map there was useless allocations of Keys and KeyVector
and complete encoding of the search key, which for most comparisons would be wasteful.
This changes completely removes the use of KeyVector and compute the key
positions on the spot. Besides that, it compares keys codepoint-by-codepoint,
avoiding unnecessary allocations and reducing encoding for most cases.
Some benchmarks result in a 2.75x speedup.
To read and build flexbuffers on Java, one needs to wrap the data
using ByteBuffer. But for the common case of having ByteBuffers
backed by arrays, accessing from a ByteBuffer might be inefficient.
So this change introduces two interfaces: ReadBuf and ReadWriteBuf.
It allows one to read and writes data directly on an array. It also allow
other buffer implementations to be used with flexbuffers.
Another change is that FlexBuffersBuilder backed by array allows
the buffer to grow with the increase of the message size. Something
that could not be done with ByteBuffer.
* [Java][FlexBuffers] Deprecate typed vector strings due to design flaw
It will still be possible to read buffers with this type, but the
elements will be treated as FBT_KEY and will be read as null-terminated
string.
Trying to build a vector of strings as typed will throw an exception.
More information on https://github.com/google/flatbuffers/issues/5627
Also, fix another bug on strings, where long strings were not properly
aligned.
* [Java][FlexBuffers] Make FBT_VECTOR_STRING_DEPRECATED considered typed.
The logic for FlexBuffers.isVectorType() was changed
to not consider FBT_VECTOR_STRING_DEPRECATED a typed
vector, but that can lead to missinterpretation for
existing serialized data. So we are reverting.
* byte buffer factory returned buffer is used instead of the requested capacity
* byte buffer factory returned buffer is used instead of the requested capacity
* Comment fix
Empty objects that inherit from Sized would try to access internal
ByteBuffer when Sized::size was called. So we add a single byte in
the empty buffer, so when size() is called it would return 0
Derek Bailey