Commit Graph

433 Commits

Author SHA1 Message Date
Kamil Rojewski 30c4948541 Renamed JS generator to explicitly state it generates TS too (#5043) 2018-12-06 11:12:06 -08:00
Matt Frantz 2aa0d9a54d Support nulls in String compare, CreateSharedString (#5060) 2018-12-03 09:48:50 -08:00
Wouter van Oortmerssen f575b02fda Make string/vector field serialization order not depend on optimizer.
Multiple calls of e.g. CreateString inside a call to a CreateTable
could cause those strings to end up in different locations in the
wire format, since order or argument evaluation is undefined.

This is allowed by the FlatBuffer format, but it is not helpful,
especially when debugging the contents of binaries, or comparing
against a "golden" binary for tests etc.

Now making sure that all the CreateTableDirect calls first serialize
sub strings/vectors before calling CreateTable.

Also made similar changes to the serialization of "binary schemas".

Change-Id: I5747c4038b37a0d400aca2bc592bec751cf5c172
2018-11-16 17:08:35 -08:00
Vladimir Glavnyy 5f32f94810 Make the Parser independent from the global C-locale (#5028)
* Make the Parser independent from the global C-locale

* Set a specific test locale using the environment variable FLATBUFFERS_TEST_LOCALE

* Remove redundant static qualifiers
2018-11-16 09:24:06 -08:00
Shivendra Agarwal 9068b0ee63 flexbuffers blob to string implementation (#5039) 2018-11-15 14:57:05 -08:00
Shivendra Agarwal ab54e61805 FlexBuffer to JSON convertor for typed and fixedTypedvectors (#4947)
* FlexBuffer to JSON convertor for typed and fixedTypedvectors

* moving the common implementation to template

* signed unsigned comparison fix

* fix a formatting ({

* changing logic to append comma in vector of elements in json
2018-11-12 08:49:11 -08:00
Wouter van Oortmerssen 980a6d66d3 Fixed vectors of enums in object API serialized incorrectly.
And also added tests for them.

Change-Id: I91af1904105435601287291412e82d5066f476a5
Tested: on Linux.
2018-11-05 14:33:36 -08:00
Wouter van Oortmerssen 21591916af Fixed missing code from @javax.annotation PR.
https://github.com/google/flatbuffers/pull/4986 missed part of the
implementation.

Change-Id: Ie41771eb018a550c289c77ebe9ef5c50d1ad6751
2018-11-02 11:40:21 -07:00
Frank Benkstein 91fe9ba93f make unions with type aliases more usable (#5019)
Some generic C++ and Rust code is not generated when unions use type
aliases because of potential ambiguity. Actually check for this
ambiguity and only disable offending code only if it is found.
2018-11-01 12:51:25 -07:00
Vladimir Glavnyy 0fb1d44bc4 Add 'fsanitize=address,undefined' to tests and flatc targets (#5009)
* Add '-fsanitize' optional flags to flattests and flatc targets

Control: -DFLATBUFFERS_CODE_SANITIZE=(ON | OFF | "=memory,undefined")
Travis-CI: building with -DFLATBUFFERS_CODE_SANITIZE=ON

* Fix -pie flag

* Cleanup
2018-10-29 11:29:05 -07:00
Doug Muir ff687ae9c1 Make alignment checking optional. (#5011) 2018-10-22 16:14:44 -07:00
Frank Benkstein ca417426c7 make flatbuffers::IsFieldPresent safer (#4988)
Give the vtable offset enum inside each table the name
"FlatBuffersVTableOffset" and base type voffset_t so it can be used as a
dependent type in IsFieldPresent. This makes that function slightly
safer since it prevents calling it with arbitrary, non-table types.
Now, the only way to use IsFieldPresent incorrectly is to create your
own type which does not inherit from flatbuffers::Table but has a
dependent voffset convertible type "FlatBuffersVTableOffset".
2018-10-22 15:57:45 -07:00
Vladimir Glavnyy 55b30827f2 Add fuzzer test for scalar numbers in json. (#4996)
* Add fuzzer test for scalar numbers in json. Grammar-based regex used to check correctness.

* Fix conversation
2018-10-22 15:44:18 -07:00
Frank Benkstein efbb11e093 CI check generate code (#4998)
* call reflection code generation from tests

This simplifies instructions to contributors so they don't forget to update
reflection code.

* add error handling to generate_code scripts

Let them propagate their errors instead of swallowing them so they show
up when called in CI.

* apply editorconfig to shell scripts

* use ordered map in dart codegen

Using an unordered map in the codegen can lead to spurious diffs in the
generated dart code.

* add CI check for generate_code being run

* update reflection_generated.h

* disable diff-check for monster_test.bfbs

Work around #5008.
2018-10-22 15:41:12 -07:00
Sumant Tambe 802639e40d Efficient Conversion of a FlatBufferBuilder to a MessageBuilder (#4980)
* Efficient conversion of FlatBufferBuilder to grpc::MessageBuilder

* Added a variety of tests to validate correctness of the MessageBuilder move operations.
Disable MessageBuilder half-n-half tests on MacOS.

* Fix failing Android build

* Generalized the MessageBuilder move constructor to accept a deallocator
2018-10-18 10:32:59 -07:00
kostya-sh 76d31e1b5e Go - Use Go bool type for bool fields (#4962)
* Use Go bool type for bool fields, and store non-default bool field to test data
2018-10-15 16:55:59 -07:00
Frank Benkstein 20396a1760 disallow c style casts (#4981)
Fixes #4857.
2018-10-15 15:11:31 -07:00
Frank Benkstein a3d8391f7b don't use std::function in flatbuffers::Parser (#4995)
std::function makes code harder to debug because it requires stepping
through a separate destructor and call operator.  It's use unnecessary
in the Parser since the functions taking functors are private and are
only used within idl_parser.cpp.  Therefore the definitions can stay in
idl_parser.cpp as well.  Only care must be taken that the definitions
appear before use but that's already true and all compilers will
complain equally if it get's violated.  This change might also improve
performance since it might allow inlining where it wasn't possible
before but I haven't measured that.
2018-10-15 12:26:35 -07:00
Vladimir Glavnyy 4ed6fafdfa Refactoring of idl_parser (#4948)
* Refactoring of numbers parser

More accurate parse of float and double.
Hexadecimal floats.
Check "out-of-range" of uint64 fields.
Check correctness of default values and metadata.

* Remove locale-independent code strtod/strtof from PR #4948.

* small optimization

* Add is_(ascii) functions

* is_ascii cleanup

* Fix format conversation

* Refine number parser

* Make code compatible with Android build

* Remove unnecessary suppression of warning C4127
2018-10-11 10:37:47 -07:00
Michael Edwards a4f9d1bfcc Fix expected type of enum values in reflection tables (#4944)
Fixes #4930
2018-10-08 14:29:29 -07:00
cor3ntin 7c3c027295 Add missing const on Reference::As<> (#4975)
Reference::As<> was needlessly mutable wich made them
less safe and harder to use
2018-10-08 12:52:01 -07:00
Stewart Miles 569492e890 Disable armeabi builds for Android and re-enable CI builds. (#4970)
armeabi support was removed from the Android NDK so we should no
longer build it.  Since this fixes the Android build failures this
commit also re-enables Travis Android builds.

While re-enabling Android builds, some recent changes broke C++98
support so this fixes those issues as well which include:
- Conditionally compiling use of move constructors, operators and
  std::move.
- Changing sample to use flatbuffers::unique_ptr rather than
  std::unique_ptr.

Finally, added the special "default_ptr_type" value for the
"cpp_ptr_type" attribute.  This expands to the value passed to
the "--cpp-ptr-type" argument of flatc.
2018-10-08 12:43:57 -07:00
Vladimir Glavnyy 925c1d77fc Fix recursion counter check. Add control to override depth of nested … (#4953)
* Fix recursion counter check. Add control to override depth of nested objects.

* Change if-condition to `>=`
2018-10-04 09:27:37 -07:00
Wouter van Oortmerssen c0698cc33f Bumped version to 1.10.1 for all languages.
Change-Id: I9a6256d90ea800834a887afdcf888df412018933
2018-10-03 12:48:47 -07:00
Vladimir Glavnyy 7e711f80d7 Parser error message reports both the line number and the cursor position. (#4954) 2018-10-01 14:57:36 -07:00
Shivendra Agarwal 0bffce5aef Add more apis to query vector types from a reference (#4823)
* Add more apis to query vector types from a reference

https://github.com/google/flatbuffers/issues/4818

* changing order of apis

* another reordering

* removed vector element type api as not needed as for now
2018-09-27 12:34:27 -07:00
Vladimir Glavnyy d48f08acfe Add missed defined(clang) and fix the wrong placement of #pragma push/pop for MSVC compiler (#4946) 2018-09-27 12:31:43 -07:00
Sumant Tambe 49fed8c4f6 Add FlatBufferBuilder move semantics tests to the main test suite (#4902)
* Add FlatBufferBuilder move semantics tests to main

Do not eagerly delete/reset allocators in release and release_raw functions
Update android, vs2010 build files
New tests for various types of FlatBufferBuilders and move semantics

* Improve test failure output with function names
2018-09-24 12:03:31 -07:00
Michael Edwards b1a925dfc2 ToStringVisitor settings to allow pretty formatted JSON (#4933) 2018-09-24 09:29:49 -07:00
Thomas 33791dc7b0 Add compare operator to code generated for c++ (#4940)
* Add operator== for c++ genated code

New "--gen-compare" option for flatc to generate compare operators. The operators are defined based on object based api types.

Inspired by issue #263.

* Improve compare operator for c++.
Thanks for the code review.

- Improve robustness against future schema extensions
- Code style
- Fix --rust generation in generate_code.sh
2018-09-21 16:53:59 -07:00
Robert 3c54fd964b Port FlatBuffers to Rust (#4898)
This is a port of FlatBuffers to Rust. It provides code generation and a
runtime library derived from the C++ implementation. It utilizes the
Rust type system to provide safe and fast traversal of FlatBuffers data.

There are 188 tests, including many fuzz tests of roundtrips for various
serialization scenarios. Initial benchmarks indicate that the canonical
example payload can be written in ~700ns, and traversed in ~100ns.

Rustaceans may be interested in the Follow, Push, and SafeSliceAccess
traits. These traits lift traversals, reads, writes, and slice accesses
into the type system, providing abstraction with no runtime penalty.
2018-09-02 18:26:55 -07:00
Sumant Tambe e7578548a5 Add move semantics to MessageBuilder, FlatBufferBuilder, SliceAllocator, and vector_downward (#4893)
Unit tests
Update flatbuffers + gRPC build instructions
Update CMakeLists.txt with cmake variables for grpc and protobuf install paths
Update tests for travis build
2018-08-30 16:43:22 -07:00
Gabriel Nützi 99acd0bcd7 DetachedBuffer cannot be rewrapped (#4885)
* Simple ReleaseRaw implemented

* [doc]

* clear_buffer and clear_allocator introduced

* auto

* typos

* rename because of -Werror=shadow
2018-08-24 10:57:57 -07:00
Derek Bailey e1f48ad35a Updated preprocessor check for template aliases. (#4888) 2018-08-23 09:48:03 -07:00
Andy Martin c2c3a84aaf Add C#/Java generator behaviour for 'private' attribute (#4882)
* Added 'private' attribute, supported when generating C# and Java

* Added use of 'private' attribute in monster_test
2018-08-16 15:25:33 -07:00
iceboy 55289c55bf Use string_view if _HAS_CXX17 is true (#4876)
Current version of VC2017 is not setting __cplusplus to correct value, instead they use _MSC_VER, _MSVC_LANG and _HAS_CXX17 macros.
2018-08-13 16:19:51 -07:00
Wouter van Oortmerssen ed2415eb72 Fixed use of uoffset_t in verifier could cause wrap around.
The verifier must be resilient against any corrupt data, so
now using size_t thru-out to ensure any 64-bit offsets can
be represented.

Also added verification of alignment.

Change-Id: I87a22aa6b045c2d83b69b47a47153f2e15ad7e06
Tested: on Linux, also with libfuzzer.
2018-08-13 14:52:50 -07:00
Andrew Selle e203882d54 Clarify flatbuffer reflection struct sizes to be less error-prone. (#4870) 2018-08-10 10:55:54 -07:00
aardappel 4898809eca FlatBuffers implementation for the Lobster programming language
Language, see: http://strlen.com/lobster/ and https://github.com/aardappel/lobster
2018-07-29 13:23:00 -07:00
shassani ca5aaf62d3 Adds multi-line to FLatBufferToString (#4847)
Adding multi-line option to FlatBufferToString method in minireflect to allow more readable output for larger flatbuffers.
2018-07-27 12:45:12 -07:00
Wouter van Oortmerssen c80f8d18c1 Fixed FlatBufferBuilder::Required not checking vtable size.
As reported in: https://github.com/google/flatbuffers/issues/4846
Tested: on Linux.

Change-Id: Id5e82dc5a46a681119cfe5a15415d522aac0e1f2
2018-07-27 12:41:50 -07:00
Paul Reimer 118093b613 Update FLATBUFFERS_HAS_STRING_VIEW __cplusplus checks to the versions provided by the standard feature test macros. (#4841) 2018-07-26 15:40:41 -07:00
shassani 4235a25640 Adds ForceStringAlignment to flatbuffers. (#4828)
ForceStringAlignment is useful for memory aligning string fields in flatbuffers.
2018-07-19 16:00:31 -07:00
Shivendra Agarwal 88cd182349 Restricting (typed=false, fixed=true) combination in flexbuffer CreateVector (#4825)
https://github.com/google/flatbuffers/issues/4815
2018-07-19 14:59:06 -07:00
Shivendra Agarwal 7c824ef690 Fixing an assert in flexbuffers CreateVector (#4824)
An assert was blocking the creation of typedvectors. It was wrongly checking for limited types even though vector was not of fixedTyped.
2018-07-19 14:55:06 -07:00
Wouter van Oortmerssen 79f2adc50a Renamed Verifier methods
The name Verify was getting too overloaded, and confused
the VS compiler

Change-Id: I26423a4d513e4def2f4e41d7f278bb683fc12518
2018-07-16 16:42:10 -07:00
shassani 51d9641de6 flatbuffer force-empty option (#4822) 2018-07-16 16:05:06 -07:00
Wouter van Oortmerssen 8f1bebba05 C++ verifier now primarily uses offsets instead of pointers.
Fix for: https://bugs.chromium.org/p/chromium/issues/detail?id=834710

Before, the verifier would create pointers to objects, and then
verify they are inside the buffer. But since even constructing pointers
that are outside a valid allocation is Undefinied Behavior in C++, this
can trigger UBSAN (with -fsanitize=pointer-overflow).

Now instead the bounds checking is first performed using offsets
before pointers are even created.

Change-Id: If4d376e90df9847e543247e70a062671914dae1b
Tested: on Linux.
2018-07-16 15:51:28 -07:00
Wouter van Oortmerssen b188fde27e Fixed Uint() method actually storing a signed int.
Also fixed two constructors missing explicit.

Change-Id: I147b2d2517bb660d92d4b0167992c115ed65dca7
2018-07-06 12:30:43 -07:00
Derek Bailey ba5eb3b5cf Lua (5.3) Language addition (#4804)
* starting Lua port of python implmention. Syncing commit

* Bulk of Lua module port from Python done. Not tested, only static analysis. Need to work on binary strings. Started work on flatc lua code generation

* Fixed all the basic errors to produced a binary output from the builder, don't know if it is generated correctly, but it contains data, so that must be good

* fixed binary set command that was extending the array improperly

* continued improvement

* Moved lua submodules down a directory so their names don't clash with potential other modules. Added compat module to provide Lua versioning logic

* Successful sample port from Python

* working on testing Lua code with formal tests

* continued to work on tests and fixes to code to make tests pass

* Added reading buffer test

* Changed binaryarray implmentation to use a temporary table for storing data, and then serialize it to a string when requested. This double the rate of building flatbuffers compared to the string approach.

* Didn't need encode module as it just added another layer of indirection that isn't need

* profiled reading buffers, optimizations to increase read performance of monster data to ~7 monster / millisecond

* Writing profiler improvments. Get about
~2 monsters/millisecond building rate

* removed Numpy generation from Lua (came from the Python port)

* math.pow is deprecated in Lua 5.3, so changed to ^ notation. Also added .bat script for starting Lua tests

* adding results of generate_code.bat

* simple edits for code review in PR.

* There was a buffer overflow in inserting the keywords into the unorder set for both the Lua and Python code gens. Changed insertion to use iterators.

* fixed spacing issue

* basic documenation/tutorial updates. Updated sample_binary.lua to reflect the tutorial better

* removed windows-specific build step in Lua tests
2018-07-05 15:55:57 -07:00