Commit Graph

1881 Commits

Author SHA1 Message Date
Wouter van Oortmerssen 3cd9b6434a Removed code_generators.cpp from library targets
Change-Id: Ia6e032a77983bf1838b8675f51d1c910acc991d8
2020-01-02 08:50:39 -08:00
Max Burke 355dfd48d1 [rust] Make enum names public (#5690)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Address comments raised in PR

* Update documentation comments per feedback

* Fix typo

* [rust] Make enum variant names public.

* Update generated test files

* Add test for public enum names
2019-12-31 10:28:58 -08:00
Michael Beardsworth bcd58a159b Correct inverted logic around include prefixes. (#5689)
4d1a9f8d9e inverted the logic around
keeping the include prefix. This change fixes the error.
2019-12-30 10:34:01 -08:00
lu-wang-g a2c12900aa Optimize Pack method using numpy (#5662)
Add the support to pack using numpy for scalar vectors when numpy is available.
2019-12-26 21:42:11 -05:00
Austin Schuh 901b89e733 [C++] Add Builder and Table typedefs (#5685)
* Add Builder and Table typedefs

This gives us a way to use templates to go from a builder to a table
and back again without having to pass both types in.

* Fix tests/cpp17/generated_cpp17/monster_test_generated.h
2019-12-26 14:56:46 -08:00
Wouter van Oortmerssen 31f8799083 Minor doc updates: FlexBuffers C#, Discord, CppUsage.
Change-Id: Ie34ff580eb2f41ff35f85271b10865f4a14d0dca
2019-12-26 12:36:41 -08:00
Austin Schuh 8023d99e21 Upgrade rules_go (#5684)
This should help with #5672 if I'm reading the report back from
buildkite properly.
2019-12-26 10:07:15 -08:00
Austin Schuh b4154405d1 Fix --incompatible_load_cc_rules_from_bzl (#5683)
Incompatible flag --incompatible_load_cc_rules_from_bzl will break
FlatBuffers once Bazel 1.2.1 is released.

Fixes #5671
2019-12-26 10:03:22 -08:00
Austin Schuh 04c17c7a76 Add support for absl::string_view when available (#5682)
For C++11 platforms, absl::string_view is sometimes available. This can
be used for string_view when std::string_view is not available.
2019-12-26 10:02:24 -08:00
Austin Schuh 62ec7d52ce [Bazel] Add support for compatible_with and restricted_to (#5681)
* Add support for compatible_with and restricted_to

These attributes have been available in Bazel for years.  Pass them
through so the flatbuffer rules can be used with them.  They let you
constrain which target platform is used.

While we are here, fix gen_reflections to work with bazel.

* Add docs
2019-12-26 09:58:48 -08:00
Robert Winslow 7de668053f
CI: New Docker tests for Python with numpy (#5677)
* New Docker tests for Python with numpy

* print numpy status in test suite
2019-12-24 02:14:55 -05:00
Wouter van Oortmerssen 3a70e0b308 Fixed struct initialization error on older versions of C#
"'this' object cannot be used before all of its fields are assigned to"

Change-Id: Icccdcc0d0be0fe0b87abe0eb28fe1cc91116fcfb
2019-12-23 17:35:54 -08:00
Wouter van Oortmerssen 9b13201356 Fixed warnings in FlexBuffers.java
- Missing return statement <- bug!
- Missing hashCode function.

Change-Id: I6333cac72adf8ead92ab2e6c7215650ce4571a73
2019-12-23 17:11:57 -08:00
Wouter van Oortmerssen 5e3916050c Fixed out of date licenses on gRPC Python files.
Change-Id: Ia6c3bf5d7da795db46c0baa8e9c7591de3400517
2019-12-23 14:01:41 -08:00
Wouter van Oortmerssen c957550511 Removed test proto output.
Change-Id: Iaf64bec068d03dd1b75670e9a28dde7392ebddb5
2019-12-23 12:33:16 -08:00
Vladimir Glavnyy 44bf719883 Add flatc '--cpp_std' switch (#5656)
* Add flatc '--cpp_std' switch and sandbox for C++17 code generator

- Added 'flac --cpp_std legacy' for compatibility with old compilers (VS2010);
- Added experimental switch 'flac --cpp_std c++17' for future development;
- Added C++17 sandbox test_cpp17.cpp;
- C++ code generator generates enums with explicit underlying type to avoid problems with the forward and backward schema compatibility;
- Adjusted CMakeLists.txt, CI and generate code scripts to support of introduced '--cpp_std';

* Fix --cpp_std values: c++0x, c++11, c++17

* Add 'cpp::CppStandard' enum

* Add testing engine into test_cpp17

* Rebase to upstream/master

* Set default '--cpp-std C++0x'

* Fix code generation (--cpp_std C++11) in CMakeLists.txt

- Fix dependency declaration of grpctest target

* Revert --cpp-std for the tests from explicit C++11 to flatc default value (C++0x)
2019-12-23 12:13:48 -08:00
Wouter van Oortmerssen 3e8f15df90 Fix for FlexBuffers FBT_VECTOR_STRING size bit-width.
For details, test.cpp/FlexBuffersDeprecatedTest(), and also
https://github.com/google/flatbuffers/issues/5627

Change-Id: I6e86e1138a5777e31055cfa2f79276d44732efbc
2019-12-23 11:49:59 -08:00
stefan301 602721a735 Added Check to VerifyAlignment (#5675)
* Added missing EndTable() call to VerifyObject()

VerifyObject called VerifyTableStart() but not EndTable(). This made Verifier::VerifyComplexity() increase depth_ with each table, not with the depth of tables.

https://groups.google.com/forum/#!topic/flatbuffers/OpxtW5UFAdg

* Added Check to VerifyAlignment

https://stackoverflow.com/questions/59376308/flatbuffers-verifier-returns-false-without-any-assertion-flatbuffers-debug-veri
2019-12-23 09:08:35 -08:00
Michael Beardsworth 13c05f4da3 Improve import handling for proto conversion (#5673)
* Keep include prefix when converting from proto.

This change preserves the include prefix when generating flatbuffers
from proto (with FBS_GEN_INCLUDES) defined.

* Improve handling of imports in proto conversion.

Previously, there was no runtime flag to make proto->fbs conversion keep
the import structure of a collection of files. This change makes proto
conversion respect the --no-gen-includes flag and skip the output of
"generated" symbols.
2019-12-23 08:50:29 -08:00
Light Lin ce3a1c43a2 [Dart] Fix prepare space for writeListInt64 and writeListUint64 (#5654)
* Fix prepare space for writeListInt64 and 

writeListUint64

* Fix align issues
2019-12-18 11:23:54 -08:00
Matt Brubeck aa75e5734b Make Rust constants public (#5659)
* Make Rust constants public

Otherwise they cannot be accessed by code that consumes the generated
bindings.

* Re-generate test code

* Add a test for enum constants
2019-12-17 12:18:59 -08:00
cryptocode 2790fee257 Add namespace qualification to union types (#5666) 2019-12-17 09:11:26 -08:00
Max Burke eddebec1b6 Bugfix for Rust generation of union fields named with language keywords (#5592)
* Bugfix for Rust generation of union fields named with language keywords

Looking at ParseField, it appears that in the case of unions, an extra field with a `UnionTypeFieldSuffix` is added to the type definition, however, if the name of this field is a keyword in the target language, it isn't escaped.

For example, if generating code for rust for a union field named `type`, flatc will generate a (non-keyword escaped) field named `type_type` for this hidden union field, and one (keyword escaped) called `type_` for the actual union contents.

When the union accessors are generated, they refer to this `type_type` field, but they will escape it mistakenly, generating code like this:

```
  #[inline]
  #[allow(non_snake_case)]
  pub fn type__as_int(&self) -> Option<Int<'a>> {
    if self.type__type() == Type::Int {
      self.type_().map(|u| Int::init_from_table(u))
    } else {
      None
    }
  }
```

Which will fail to build because the field is called `self.type_type()`, not `self.type__type()`.

* [Rust] Add crate-relative use statements for FBS includes.

At present if a flatbuffer description includes a reference to a type in
another file, the generated Rust code needs to be hand-modified to add
the appropriate `use` statements.

This assumes that the dependencies are built into the same crate, which
I think is a reasonable assumption?

* Revert "[Rust] Add crate-relative use statements for FBS includes."

This reverts commit d554d79fec.

* Address comments raised in PR

* Update documentation comments per feedback

* Fix typo
2019-12-15 15:18:22 -08:00
FujiZ 030fee36ab wrap multiple statements in do {} while(!IsConstTrue(true)) (#5655) 2019-12-13 10:14:14 -08:00
Gautham B A f9724d1bde [gRPC] Uncomment MessageBuilder (#5658)
The line where the MessageBuilder was
constructed was commented out (perhaps
an oversight).
2019-12-09 10:28:31 -08:00
Björn Harrtell b20801ca40 Supress unsigned-integer-overflow for PaddingBytes (#5647) 2019-12-09 10:06:04 -08:00
cryptocode a8e800bd7c Add --force-empty-vectors option (#5653)
The rationale for this option is that JSON clients typically want empty arrays (i.e [] in the JSON) instead of missing properties, but not empty strings when the value isn't set.
--force-empty is kept as-is, i.e. it will force both empty strings and vectors.

Closes #5652
2019-12-05 17:33:45 -08:00
Wouter van Oortmerssen d7530ae961 Fixed enum min/max values not properly escaped.
Change-Id: I503fbfaff1d2579807ca71a07cca8363dff75e52
2019-12-05 14:28:36 -08:00
bttk 99d11e279f Split Bazel targets into multiple packages (#5640)
* Split Bazel targets into multiple packages

* Merge /include/BUILD back into /BUILD
2019-12-02 14:14:48 -08:00
Google AutoFuzz Team 4fd8eb214b Remove a static_assert (#5643)
Having a static_assert on MSAN and ASAN prevents
the fuzzers from being used with different engines,
like TSAN, UBSAN, … but also with fuzzers that aren't
using MSAN/ASAN like afl for example.
2019-12-02 14:13:28 -08:00
lu-wang-g 65f8703572 Flatbuffers Python Object API (#5616)
* Flatbuffers Python Object API

Implement the logic to generate the Python object API that can
unpack the data from a buf class into an object class, and pack
the data of an object class to a buf class.

* Fix the build issues

Remove unused parameters and replace auto in the for-loop statement
with std::string to make it compatible with VS2010.

* Fix the build issues.

* Add support for Array type

Added logic to handle Array type in Python Object API. Updated the
generated code accordingly.

* Fix the old style casting from int to char

* Fixed another conversion from int to char

* Fixed the import for typing

Importing typing may cause errors when a machine do not have the
moduel typing installed. This PR fixes the issue by guarding
"import typing" with the "try/except" statement.

* Fix issue of iterating the vector of import list

* Update the generated examples using generate_code.sh

* Fix the import order for typing

The import list was stored in unordered_set, so that each generated
codes may have different import order. Therefore, it failed in the
consistency test where two generated copies need to have exactly the
same apperance.

* Optimize unpack using numpy

Use numpy to unpack vector whenever it is possible to improve unpack
performance.

Also, added codegen command for Python specificly in generate_code.sh,
because --no-includes cannot be turn on for Python.

* Fix the import order

* Update generate_code.bat for windows accordingly

* Replace error message with pass

Avoid printing error message for every Python2 users about typing.
Replace it with pass.
2019-12-02 14:11:28 -08:00
Ivan Shynkarenka 75823cc275 [Clang 10]: definition of implicit copy constructor for 'TableKeyComparatoris deprecated #5649 (#5650) 2019-12-02 14:10:09 -08:00
nilsocket 58e279244c [docs]: add missing semicolon (#5648)
Add missing semicolon for table field.
2019-12-02 14:01:25 -08:00
Tiago Silva 3c964e10ab [GO] Fix support for enums with underscores and Unions with imported members (#5600)
* Fix Enum Stringer method when Enum has underscores

Fixes #5481

* Fix go package importing itself when Union has imported members.
2019-11-26 12:25:36 -08:00
Vladimir Glavnyy c3c32ec942 Fix ambiguity of a type deduction in TEST_EQ macro if arguments have `enum class` type. (#5630) 2019-11-25 12:56:47 -08:00
Vladimir Glavnyy 075e8d676b Simplify declarations of x-macro FLATBUFFERS_TD (#5638) 2019-11-25 12:54:59 -08:00
FujiZ bcf1bd5c9e read vtable size through ReadScalar() (#5636) 2019-11-25 12:52:42 -08:00
Derek Bailey 136d75fa65 Changed null checks in test. Removed verifier pointer usage (#5634) 2019-11-21 10:25:31 -08:00
Vladimir Glavnyy 091fa1fd1b Add testing of C++ with sanitizers (CI-Docker) (#5631)
* Add C++ build testing with clang and gcc

This adds Dockerfiles which test building flatc and the C++ library against clang
and gcc. See discussion at #5119.  It is derived from the Travis CI tooling.

The GRPC tests are failing due to #5099 so those are commented out.

These are run from the .travis.yml file rather than the tests/docker/languages
folder because the builds may each take longer than 30 minutes and were hitting
Travis timeouts.

Parallel builds and build caching attempt to keep the build times low.

* Add GCC 8.3 and Clang 7.0 with sanitizers into CI (based on #5130)

- Add a docker based on Debian Buster.
- Add C++ building scripts for the docker.
- Leak-sanitizer requires SYS_PTRACE.
2019-11-18 12:16:41 -08:00
FujiZ ff3781dc2d add namespace prefix in FLATBUFFERS_MAX_BUFFER_SIZE (#5629) 2019-11-18 12:02:14 -08:00
Malar Kannan 6beb9f49cb Support for python grpc - continuing the work from the pull request #4270 #4705 (#5613)
* Support for python grpc

* add few fixes

* Fixes build

* Fix python generator

* Add tests

* Fix grpc python test

* Fix tests and add incomplete python generator

* Fix python generator

* Add python generator methods

* Fix Appveyor build

* grpc python support v0.1

* Update tests

* update grpctest

* Remove duplicated code and fix a brace

* tests for flatbuffers grpc python

* Updated tests + removed SerializeToString, From String

* remove pickle import

* include missing files in ci - BUILD and generated test result
2019-11-14 16:58:35 -08:00
Derek Bailey 80988ea869 Removed idl_gen_general.cpp and move contents to code_generators.cpp (#5625) 2019-11-14 15:47:58 -08:00
Derek Bailey 0f2ff7eaa9 Lua cleanup (#5624) 2019-11-14 15:46:46 -08:00
Derek Bailey dda095023d [C++] Adds basic schema evolution tests (#5611)
* Added basic schema evolution tests

* Add BUILD targets for evolution tests. Added to test/generate_code scripts

* Use vector.front() instead of vector.data()

* Added --scoped-enums option for evolution test
2019-11-14 15:44:18 -08:00
Derek Bailey adbcbba5d1 [C++, C#, Java] Separated C# and Java generators into their own classes (#5618)
* Cloned idl_gen_general.cpp to idl_gen_csharp.cpp and removed java references

* Java generator changes
2019-11-11 11:37:55 -08:00
Dmitry cbbd6aca04 add check for root_type specified for json schema generation (#5622) 2019-11-11 11:33:27 -08:00
messense 405c64e07d [Rust] Bump smallvec version to 1.0 (#5621)
See https://github.com/servo/rust-smallvec/pull/175 for changelog.
2019-11-11 09:52:26 -08:00
Derek Bailey 42c08cbca6 Ran src/clang-format-all.sh (#5617) 2019-11-11 09:29:37 -08:00
Wouter van Oortmerssen 33d5dd9bdd Improved pull request & clang-format instructions.
Change-Id: Ia439bcc61bac5788792244d10e573b1fba54b347
2019-11-07 14:37:33 -08:00
Derek Bailey 105dd528e9 Change monster_extra generation to use flatbuffers::unique_ptr (#5612) 2019-11-07 14:10:41 -08:00