Commit Graph

1525 Commits

Author SHA1 Message Date
tira-misu dba962ebb8 Enable flatbuffer to initialize Parser from bfbs (#4283) (#5077)
* Enable flatbuffer to initialize Parser from bfbs (#4283)

Now its possible to generate json data from bfbs data type and flatbuffers data
and visa versa.

* add deserialize functionality in parser from bfbs
* add small usage sample

* Fix build break

* Merge branch 'pr/1' into fix-issue4283

* Fix buildbreak

* Build monster_test.bfbs with --bfbs-builtins

Attribute flexbuffer has be included in bfbs. Only with this attribute test
will run. By initialization a parser by a bfbs the attribute has to be known
for this filed. monsterdata_test.golden has a flexbuffer field so parse would
fail.

* Fix generate_code.sh

* Revert automatic indent changes by IDE

* Auto detect size prefixed binary schema files

* Use identifier (bfbs) to detect schema files
2018-12-13 11:59:27 -08:00
Christian Lang 60a0f35fbc generated cmake command for flatc generation should depend on source file (#5087) 2018-12-13 11:56:51 -08:00
Rob Agar 58e8552da0 Get rid of reproducible build warning by removing build date & time from flatc version (#5086)
* disable reproducible build warning due to date/time macros

* wrapped GCC pragmas in #ifdef _GNUC_

* removed __DATE__ and __TIME__ macros from flatc.cpp
2018-12-10 12:48:09 -08:00
Austin Schuh d56a4055da Add flatbuffer_cc library support (#5061)
* Add flatbuffer_cc library support

* Update flags so all the tests pass

Tests now all pass!

* Modify the tests to use the generated code

This should be a simple serialize/deserialize test of the new generated
code to make sure the bazel rules are doing something sane.

* Use generated monster_test.fb in testing/test.cpp

cmake drops it's generated code in tests/monster_test_generated.h
Instead of checking that in, let's generate it with bazel.

* Make grpc tests depend on monster_test_generated.h

* Remove redundant cmake dependency

This should address @aardappel's feedback.

* Run flatc for Android as well

This will fix the last travis.ci failure

* Add generated output folder and fix flags

* Move flatbuffers_header_build_rules to the library that uses it

* Use --cpp-ptr-type to fix android

Android was the only target using the STL emulation layer.  It needed
the --cpp-ptr-type flatbuffers::unique_ptr flag to work.  Add it!

* Roll back changes to use autogenerated monster_test_generated.

Flip tests/test.cpp to use the autogenerated file as well.
2018-12-10 12:23:40 -08:00
Kamil Rojewski 30c4948541 Renamed JS generator to explicitly state it generates TS too (#5043) 2018-12-06 11:12:06 -08:00
Simon.S.King 80d148b175 [net] Fix issue #5036 when write a double value into bytebuffer at big endian machine with unsafe mode. (#5076) 2018-12-03 09:50:26 -08:00
Matt Frantz 2aa0d9a54d Support nulls in String compare, CreateSharedString (#5060) 2018-12-03 09:48:50 -08:00
Robert 79cd55bd3a
CI: Dockerized language port tests (#5066)
This runs a script in TravisCI that executes a bunch of small Docker image
scripts to test the language ports in isolated environments. This allows us to
test multiple language versions with little additional complexity.

Covers:

+ Java OpenJDK 10.0.2
+ Java OpenJDK 11.0.1
+ Node 10.13.0
+ Node 11.2.0
+ Python CPython 2.7.15
+ Python CPython 3.7.1
+ Rust 1.30.1
2018-11-29 22:03:06 -08:00
Robert b378b8eb69
Fix create_vector_of_strings to use the stack, and test it. (#5074) 2018-11-29 11:47:28 -08:00
Sumant Tambe 9635d494b3 Fix Java generator bug that ignores bidi streaming attribute (#5063)
* Fix Java generator bug that ignores streaming: bidi attribute
Tests

* Java gRPC client streaming test

* Java gRPC Bidi Streaming Test
2018-11-29 10:33:17 -08:00
Robert 0143f4e364
disable unimportant RAM-heavy test b/c Windows (#5047) 2018-11-16 22:03:51 -08:00
Russell Chou e161ade68c Make Parser skip files it's seen already. (#5048)
The use case is so we can batch compile a bunch of schemas, some which can depend on each other, without caring about their order.
2018-11-16 17:11:23 -08:00
Wouter van Oortmerssen f575b02fda Make string/vector field serialization order not depend on optimizer.
Multiple calls of e.g. CreateString inside a call to a CreateTable
could cause those strings to end up in different locations in the
wire format, since order or argument evaluation is undefined.

This is allowed by the FlatBuffer format, but it is not helpful,
especially when debugging the contents of binaries, or comparing
against a "golden" binary for tests etc.

Now making sure that all the CreateTableDirect calls first serialize
sub strings/vectors before calling CreateTable.

Also made similar changes to the serialization of "binary schemas".

Change-Id: I5747c4038b37a0d400aca2bc592bec751cf5c172
2018-11-16 17:08:35 -08:00
Vladimir Glavnyy 5f32f94810 Make the Parser independent from the global C-locale (#5028)
* Make the Parser independent from the global C-locale

* Set a specific test locale using the environment variable FLATBUFFERS_TEST_LOCALE

* Remove redundant static qualifiers
2018-11-16 09:24:06 -08:00
Kamil Rojewski d6b1ce09cf Fixed double generated file name postfix (#5046)
* Fixed double generated file name postfix

* Fixed missing file postfix on imports without reexports
2018-11-16 09:20:29 -08:00
Austin Schuh f23009f04f Make flatbuffer C++ tests work when vendored with bazel (#5040)
Tests for third_party code are run out of the main workspace.  This
isn't an issue when the main workspace is the
com_github_google_flatbuffers workspace, but is an issue when you are
running the tests from another repository.

To reproduce, use "git_repository" to add flatbuffers to a project and
then run:
  bazel test @com_github_google_flatbuffers//:flatbuffers_test
2018-11-16 09:19:34 -08:00
Kamil Rojewski 30bae01ea3 Convenient createX methods for TS (#5044) 2018-11-16 09:17:54 -08:00
Shivendra Agarwal 9068b0ee63 flexbuffers blob to string implementation (#5039) 2018-11-15 14:57:05 -08:00
Robert f8a0d3889b
Fix missing doxygen link to Rust docs 2018-11-13 11:10:03 -08:00
Antonio Park 688fc77460 Wrong int32 min-max range definition on numTypes.lua (#5031)
* Update numTypes.lua

int32 range was wrongly defined.

* Update numTypes.lua

Fix number 32 to 31
2018-11-12 09:40:24 -08:00
Uilian Ries dd8922878d Update Conan version Automatically (#5027)
* #5026 Add FindFlatBuffers.cmake to Conan Package

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #5026 Build Flatbuffers on OSX 10

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #5026 Auto update Conan package version

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #5026 Update Conan docker images

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #5026 Update tag var on Appveyor

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #5026 Filter appveyor brach name

Signed-off-by: Uilian Ries <uilianries@gmail.com>
2018-11-12 08:49:42 -08:00
Shivendra Agarwal ab54e61805 FlexBuffer to JSON convertor for typed and fixedTypedvectors (#4947)
* FlexBuffer to JSON convertor for typed and fixedTypedvectors

* moving the common implementation to template

* signed unsigned comparison fix

* fix a formatting ({

* changing logic to append comma in vector of elements in json
2018-11-12 08:49:11 -08:00
Wouter van Oortmerssen f445c1eb4a Fixed STLPort build
Change-Id: I2b9508f547667695475be64fa56f64f129603fd4
2018-11-05 16:33:35 -08:00
Wouter van Oortmerssen 980a6d66d3 Fixed vectors of enums in object API serialized incorrectly.
And also added tests for them.

Change-Id: I91af1904105435601287291412e82d5066f476a5
Tested: on Linux.
2018-11-05 14:33:36 -08:00
Wouter van Oortmerssen 21591916af Fixed missing code from @javax.annotation PR.
https://github.com/google/flatbuffers/pull/4986 missed part of the
implementation.

Change-Id: Ie41771eb018a550c289c77ebe9ef5c50d1ad6751
2018-11-02 11:40:21 -07:00
Sumant Tambe b6c3d7b899 Fix a memory leak in builder test (#5021)
* Fix a memory leak in builder test

* using RAII to free memory
2018-11-02 10:04:58 -07:00
Gabriel Nützi db0fcdd906 --keep-prefix for JS generator (#5018)
* keep include path

* add option --keep-prefix for js

* format contribution (format whole files before merge!)

* revert util.h : IsAbsPath ...

* JS Generator: only support relatives paths (keep it as it was)
2018-11-02 08:54:43 -07:00
Frank Benkstein 91fe9ba93f make unions with type aliases more usable (#5019)
Some generic C++ and Rust code is not generated when unions use type
aliases because of potential ambiguity. Actually check for this
ambiguity and only disable offending code only if it is found.
2018-11-01 12:51:25 -07:00
Robert 4c3b6c247d
Rust: test that no heap allocs happen on hot paths (#5022) 2018-10-31 18:08:14 -07:00
Wouter van Oortmerssen 160e8f2fdc Disallowed 0 byte structs.
This is because they are incompatible with C++ and possibly other
languages that make them minimum size 1 (to make sure multiple
such objects don't reside at the same address). Forcing them to size
1 was also not practical, as that is requires updating the logic
of a lot of implementations and thus possibly backwards incompatible.

More here: https://github.com/google/flatbuffers/issues/4122

Change-Id: I2bfdc8597b7cfd2235bb4074bb2ae06f81f8e57d
2018-10-29 17:40:19 -07:00
Wouter van Oortmerssen 705577de51 Revert "Convert net/FlatBuffers project to be compatible with netstandard (#4811)"
This reverts commit 72b05bc865.

Change-Id: I17510b7fa49b9861ce0a870b16f5c14a41082db5
2018-10-29 17:21:30 -07:00
Vladimir Glavnyy 0fb1d44bc4 Add 'fsanitize=address,undefined' to tests and flatc targets (#5009)
* Add '-fsanitize' optional flags to flattests and flatc targets

Control: -DFLATBUFFERS_CODE_SANITIZE=(ON | OFF | "=memory,undefined")
Travis-CI: building with -DFLATBUFFERS_CODE_SANITIZE=ON

* Fix -pie flag

* Cleanup
2018-10-29 11:29:05 -07:00
Wouter van Oortmerssen bd20a60d6a Fixed Java gRPC version numbers.
Change-Id: Iff00a48b370f45f80d68e6c6929483d608ce1885
2018-10-25 13:48:41 -07:00
Wouter van Oortmerssen b78c4332be Fixed npm publish appending multiple copies into .mjs
Change-Id: Id8c373b0f77d9a0f4922257339abb4e01e06c2d4
2018-10-25 13:48:41 -07:00
csmoe 636b516492 clean up unnecessary outer rust gen-snippet (#5013)
* clean up unneeded outer rust gen-snippet

* update generated rust file
2018-10-23 20:16:18 -07:00
Felix Hanley 560718e976 Use stdlib context in generated Go files (#4904) 2018-10-22 16:28:58 -07:00
Doug Muir ff687ae9c1 Make alignment checking optional. (#5011) 2018-10-22 16:14:44 -07:00
Frank Benkstein ca417426c7 make flatbuffers::IsFieldPresent safer (#4988)
Give the vtable offset enum inside each table the name
"FlatBuffersVTableOffset" and base type voffset_t so it can be used as a
dependent type in IsFieldPresent. This makes that function slightly
safer since it prevents calling it with arbitrary, non-table types.
Now, the only way to use IsFieldPresent incorrectly is to create your
own type which does not inherit from flatbuffers::Table but has a
dependent voffset convertible type "FlatBuffersVTableOffset".
2018-10-22 15:57:45 -07:00
Vladimir Glavnyy 55b30827f2 Add fuzzer test for scalar numbers in json. (#4996)
* Add fuzzer test for scalar numbers in json. Grammar-based regex used to check correctness.

* Fix conversation
2018-10-22 15:44:18 -07:00
Frank Benkstein efbb11e093 CI check generate code (#4998)
* call reflection code generation from tests

This simplifies instructions to contributors so they don't forget to update
reflection code.

* add error handling to generate_code scripts

Let them propagate their errors instead of swallowing them so they show
up when called in CI.

* apply editorconfig to shell scripts

* use ordered map in dart codegen

Using an unordered map in the codegen can lead to spurious diffs in the
generated dart code.

* add CI check for generate_code being run

* update reflection_generated.h

* disable diff-check for monster_test.bfbs

Work around #5008.
2018-10-22 15:41:12 -07:00
Frank Benkstein 5c0f914f38 forbid enum values that are out of range (#4977)
* forbid enum values that are out of range

Enum values that are out of range can lead to generated C++ code that does
not compile.  Also forbid boolean enums.

* update enum and union documentation slightly
2018-10-18 10:39:08 -07:00
Sumant Tambe 802639e40d Efficient Conversion of a FlatBufferBuilder to a MessageBuilder (#4980)
* Efficient conversion of FlatBufferBuilder to grpc::MessageBuilder

* Added a variety of tests to validate correctness of the MessageBuilder move operations.
Disable MessageBuilder half-n-half tests on MacOS.

* Fix failing Android build

* Generalized the MessageBuilder move constructor to accept a deallocator
2018-10-18 10:32:59 -07:00
Felix Frank ad8b1e5dbd [Python] Fast serialization of numpy vectors (#4829)
[Python] Fast serialization of numpy vectors (#4829)
2018-10-15 17:07:08 -07:00
kostya-sh 76d31e1b5e Go - Use Go bool type for bool fields (#4962)
* Use Go bool type for bool fields, and store non-default bool field to test data
2018-10-15 16:55:59 -07:00
Frank Benkstein 20396a1760 disallow c style casts (#4981)
Fixes #4857.
2018-10-15 15:11:31 -07:00
Frank Benkstein a3d8391f7b don't use std::function in flatbuffers::Parser (#4995)
std::function makes code harder to debug because it requires stepping
through a separate destructor and call operator.  It's use unnecessary
in the Parser since the functions taking functors are private and are
only used within idl_parser.cpp.  Therefore the definitions can stay in
idl_parser.cpp as well.  Only care must be taken that the definitions
appear before use but that's already true and all compilers will
complain equally if it get's violated.  This change might also improve
performance since it might allow inlining where it wasn't possible
before but I haven't measured that.
2018-10-15 12:26:35 -07:00
iceboy 241e87d143 add :runtime_cc target (#4994) 2018-10-15 12:18:15 -07:00
Frank Benkstein 35f0b41fed fix missing newline in flatc help message (#4992) 2018-10-11 15:54:01 -07:00
Frank Benkstein cb5422c398 fix #4974 (#4989) 2018-10-11 12:40:41 -07:00
Vladimir Glavnyy 4ed6fafdfa Refactoring of idl_parser (#4948)
* Refactoring of numbers parser

More accurate parse of float and double.
Hexadecimal floats.
Check "out-of-range" of uint64 fields.
Check correctness of default values and metadata.

* Remove locale-independent code strtod/strtof from PR #4948.

* small optimization

* Add is_(ascii) functions

* is_ascii cleanup

* Fix format conversation

* Refine number parser

* Make code compatible with Android build

* Remove unnecessary suppression of warning C4127
2018-10-11 10:37:47 -07:00