Commit Graph

2232 Commits

Author SHA1 Message Date
Casper 16836ff95a
Add advance feature indicators to reflection (#6546)
* Add advance feature indicators to reflection

* deserialize too

* model advanced features as bitflags

* use uint64_t instead of AdvancedFeatures

* git clang format

* initialize advanced_features_

* remove whitespace

Co-authored-by: Casper Neo <cneo@google.com>
2021-04-27 10:10:59 -04:00
Casper c87179e73e
Rust Remove SafeSliceAccess for Arrays, and fix miri. (#6592)
* Fix Miri flag passing and bump Rust version.

* Fix Miri problems from Arrays PR.

SafeSliceAccess was removed for Arrays. It's kind of unsound.
It has two properties:
1. EndianSafe
2. Alignment 1

We only need 1. in create_vector_direct to memcpy data.
We both 1. and 2. for accessing things with slices as buffers are built on &[u8]
which is unaligned. Conditional compilation implements
SafeSliceAccess for >1byte scalars (like f32) on LittleEndian machines
which is wrong since they don't satisfy 2.

This UB is still accessible for Vectors (though not exercised our
tests) as it implements SafeSliceAccess. I'll fix this later by
splitting SafeSliceAccess into its 2 properties.

Co-authored-by: Casper Neo <cneo@google.com>
2021-04-26 19:28:25 -04:00
Casper c24031c36b
Mark endian_scalar as unsafe. (#6588)
* Mark endian_scalar as unsafe.

Also
- removed the deprecated flexbuffer slice from example
- fixed some cargo warnings

* Assertions and read_scalar made unsafe

* Clippy lints

* Add to Safety

Co-authored-by: Casper Neo <cneo@google.com>
2021-04-26 09:18:58 -04:00
mustiikhalil 4ccc52c7a0
[swift] Implements union strings (#6589) 2021-04-24 14:47:24 +03:00
Alexander Zaitsev b82fe07384
[Rust] Fix small mistyping (#6585)
Hi!

Just a veeeeeery small fix in the Rust sample. No functionality is affected.

That's not important at all but annoying for my eyes :)
2021-04-22 12:36:23 -07:00
罗泽轩 1e7f6c8c7c
[TS] Commit the generated tests code (#6579)
* [TS] Ignore generated monster_test_grpc files

* [TS] Commit the generated tests code
2021-04-22 11:15:18 -07:00
罗泽轩 84714b109b
chore: ensure bash is used to generate code (#6577)
In some platforms, like Ubuntu, `sh` is linked to `dash` but not `bash`.
2021-04-19 14:30:55 -07:00
罗泽轩 1045d7dd44
[Lua] Fix EnforceNumberAndPack for bool type (#6576) 2021-04-19 14:30:20 -07:00
罗泽轩 0c77775966
[TS] Remove duplicated build-ts ci (#6575)
Now we already have a real build-ts ci job, let's remove the commented
one.
2021-04-19 14:28:26 -07:00
Ádám Lippai c43ba17520
[Rust] Specify Minimum Supported Rust Version (#6573)
Related to #6572
2021-04-19 16:43:11 -04:00
Eddie Linder da3bb64ef6
[Rust] Add support for fixed size arrays (#6548)
* Add support for fixed size arrays

* clang-format

* Update rust image to 1.51 to support const generics

* Handle correctly big endian

* Add fuzz tests and clean code

* Add struct fuzz test and optimize struct arrays for api

* Bump flatbuffers crate version
2021-04-16 11:15:59 -04:00
Paulo Pinheiro 151900ba96
[Kotlin][FlexBuffers] Add support for Kotlin-iOS (#6564)
With this change, Flexbuffers for Kotlin now supports iOS(arm32, arm64, X64).
WatchOS and tvOS not supported.
2021-04-15 11:29:51 -07:00
Kamil Rojewski c012f29f91
silenced clippy warning (#6565) 2021-04-15 11:19:13 -04:00
Paulo Pinheiro df2df21ec1
[Kotlin] Bump kotlinx.benchmark dependency to 0.3.0 (#6560)
Kotlinx.benchmark project just abandoned bintray and moving to maven central,
removing the artifacts from bintray and causing missing dependencies in our build.
So we are updating the dependency to point to the new version on maven central.

More information in:

53ee45d0d9 (diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5)
2021-04-12 13:47:40 -07:00
Björn Harrtell 408e4db4af
[TS] Add Build TS to CI jobs (#6524)
* Add Build TS to CI jobs

* Add npm compile step

* Fix syntax

* Add required code gen for TS

* Exit on failure

* Make TS code gen identical to test run

* Remove duplicate TS code gen artifacts

* Remove bad gitignore

* Forgot parts of generate_code and make sure same settings for test run

* Don't forget about the bat

* Try and fix flatc args

* Another attempt to fix args

* Fix typo

* Commit up to date code gen

* Another attempt to fix sh/bat

* Move -o param to after -I

* Commit missing code gen file

* Fix grpc code gen and test

* Another grpc code gen fix

* Rework to not use ts folder

* Fix env vars in bat and make less churn

* Move TS code gen to dedicated folder

* Fix transpilation output folder and module paths

* Fixes to code gen

* Include generated js

* Moved ts code gen

* Remove test ts code gen folder
2021-04-12 10:41:59 -07:00
Paulo Pinheiro 4d2364f342
[Kotlin][FlexBuffers] Add support for Kotlin-JS (#6554)
Flexbuffers for Kotlin currently supports JVM and MacOS. This change
introduces support to JS as well.
2021-04-12 10:30:15 -07:00
Casper 261cf3b204
Default-empty vectors of enums (#6505)
* disable clippy

* Vector of enum default

* swift and tests

* git clang format

* Rewrite enum parser checks

* Remove Voids from more_defaults

* vector enum swift

* remove vector accessor from swift

* clang format

Co-authored-by: Casper Neo <cneo@google.com>
2021-04-06 07:23:45 -04:00
mustiikhalil cd67261bba
[CI] fixes buildkite (#6544) 2021-04-05 12:34:42 +03:00
mustiikhalil 1aa0c2f6a9
Limit formatter to pull requests only (#6540) 2021-04-01 20:09:43 +03:00
Casper 4133a39df8
Rust structz (#6539)
* Rust structz

* struct of structs test

* swift tmp variables

Co-authored-by: Casper Neo <cneo@google.com>
2021-03-29 19:56:45 -04:00
Paulo Pinheiro 1c26d2a1a0
[Kotlin][FlexBuffers] JSON support for Flexbuffers (#6417)
* [Kotlin][FlexBuffers] Add JSON support for FlexBuffers

* [Kotlin][Flexbuffers] Re-implement JSON parser with a tokenizer.
2021-03-29 15:57:23 -07:00
Jeong YunWon 276b1bc342
[grpc] Support latest version of grpc PoC (#6338)
* use grpcpp rather than deprecated grpc++

* grpcpp Deserialize take ByteBuffer instead of grpc_byte_buffer

* grpc 1.36.0

* Fix to use grpc 1.36.0

* Fix CMakeLists to refer gRPC

* add find_package(Threads)

* Update bazel deps

* Apply workaround about boringssl build error

* Install latest cmake for osx on travis-ci

Co-authored-by: Yeom Gyusun <omniavinco@gmail.com>
2021-03-25 12:12:35 -07:00
Kamil Rojewski 124654ffc4
fixed packing structs (#6530)
* fixed packing structs in nested buffers

* fixed packing structs
2021-03-25 10:25:00 -07:00
mustiikhalil 3b7d1e86b4
[GO] Tries to add go format to the CI (#6518)
* Trys to add go format to the CI

Adds Gofmt to CI

* Adds formatting for go dir in grpc

* Small fix
2021-03-18 11:03:44 -07:00
Michael 78f0c0d1d9
[C++] #6501 - Problem when mapping a native type multiple times (#6514)
* [C++] #6501 - Problem when mapping a native type multiple times
- idl.h:
added "native_type_pack_name"
- flatbuffers.h:
added CreateVectorOfNativeStructs variants which receive a pointer to the serialization function
- idl_gen_cpp.cpp:
adapted code generation in case "native_type_pack_name" attribute is present
- extended tests & docs; improved surrounding native_type docs a little

* integrated review feedback
2021-03-18 11:01:50 -07:00
Vladimir Glavnyy c992eafb5b
[fuzzer] Add `monster_debug` target (#6513)
Add the `monster_debug` target for better troubleshooting.
Improve oss-fuzz logs.
2021-03-18 10:56:28 -07:00
mustiikhalil ef8dd7792a
[Swift] Removes allman rule (#6519) 2021-03-17 13:22:49 +03:00
Vladimir Glavnyy 69b329fc87
[flexbuffers, json] Parse `nan` and `inf` (#6512) 2021-03-15 11:44:42 -07:00
Kamil Rojewski 6543ba5297
fixed packing structs in nested buffers (#6509) 2021-03-11 11:14:23 -08:00
Vladimir Glavnyy 0e453ac352
[idl_parser] Add kTokenNumericConstant token (#6432)
* [idl_parser] Add kTokenNumericConstant token

This commit adds the new token for correct parsing of signed numeric constants.
Before this expressions `-nan` or `-inf` were treated as kTokenStringConstant.
This was ambiguous if a real string field parsed.
For example, `{ "text_field" : -name }` was accepted by the parser as valid JSON object.

Related oss-fuzz issue: 6200301176619008

* Add additional positive tests fo 'inf' and 'nan' as identifiers

* Rebase to HEAD

* Move processing of signed constants to ParseSingleValue method.

* Add missed `--cpp-static-reflection` (#6324) to pass CI

* Remove `flatbuffers.pc` from repository to unblock CI (#6455).

Probably the generated flatbuffers.pc should not be a part of repo.

* Fix FieldIdentifierTest()
2021-03-11 11:12:06 -08:00
Vladimir Glavnyy e9b4ae69dc
Remove `flatbuffers.pc` from the repository (#6508) 2021-03-11 11:11:04 -08:00
mustiikhalil fc4fffea41
Bump grpc to 1.0.0 (#6507)
Adds void to keywords
2021-03-09 10:56:06 +03:00
dpacbach b240ab704d
Move Traits struct and Create method out of --cpp-static-reflection. (#6503) 2021-03-08 10:59:17 -08:00
Ben Niu 9a4f1f434b
Disable x64-specific optimizations for ARM64EC ReadInt64 (#6506)
ARM64EC is a new ARM64 ABI designed by Microsoft to support x64 application emulation on ARM64 CPUs. When compiling for ARM64EC, both the _M_X64 and _M_ARM64EC macros are defined. However, that causes problem in compiling this file, because the __movsb intrinsic, which is lowered to rep movsb, is not supported on ARM64, so the optimization for native x64 should be disabled for ARM64EC.
2021-03-08 10:53:04 -08:00
dpacbach fac64918dc
Add --cpp-static-reflection to generate_code.bat. (#6502) 2021-03-06 22:28:27 +07:00
David P. Sicilia a69815f72c
[C++17] Add compile-time reflection for fields. (#6324)
* [C++17] Add compile-time reflection for fields.

Included in this commit is the following:

  - The C++ generator has been modified so that,
    when in C++17 mode, it will emit Table and
    Struct field traits that can be used at com-
    pile time as a form of static reflection. This
    includes field types, field names, and a tuple
    of field getter results.

  - Diffs to the cpp17 generated files. No other
    generated files are affected.

  - A unit test that also serves as an example. It
    demonstrates how to use the full power of this
    reflection to implement a full recursive
    JSON-like stringifier for Flatbuffers types,
    but without needing any runtime access to the
    *.fbs definition files; the computation is
    done using only static reflection.

Tested on Linux with gcc 10.2.0.

Fixes #6285.

* Fix int-conversion warning on MSVC.

* Try to fix std::to_string ambiguity on MSVC.

* Fix clang-format diffs.

* Fix more clang-format diffs.

* Fix last clang-format diff.

* Enable C++17 build/test for VC 19 platform in CI.

* Forgot to add value to cmake command line variable.

* Various fixes/changes in response to @vglavnyy's feedback.

* Replace "fields pack" with index-based getters.

* Fix MSVC error.

* Fix clang-format diffs.

* getter_for method returns result instead of address-of-getter.

* Next round of reviewer suggestions.

* Use type instead of hardcoded struct name.

* Fix clang-format diff.

* Add test for FieldType since it is not used in the stringify test.

* Add fields_number field to Traits struct.

* Add --cpp-static-reflection flag and put those features behind it.

* Fix clang-format diffs.

* Remove <tuple> include.
2021-03-05 10:01:40 -08:00
Kamil Rojewski 4033ff5892
fixed invalid TS call and added test files (#6495)
e581013e3d broke TS generation - please don't use "replace" to refactor function names :)
2021-03-04 12:12:44 -08:00
Casper a083572512
disable clippy (#6494)
Co-authored-by: Casper Neo <cneo@google.com>
2021-03-01 13:51:22 -08:00
Vladimir Glavnyy 5319dedb1a
[idl_parser, JSON] Disable parsing of JSON for incomplete schemes (#6493)
This commit disable JSON parsing for an incomplete scheme if JSON object is embedded into one file with the scheme.
This should improve the quality of OSS-Fuzz inputs for the parser_fuzzer target.
2021-03-01 13:40:58 -08:00
Mark Spatz bd4e0b30a7
[idl_parser] Track included files by hash (#6434)
* [idl_gen] Delete ts::GenPrefixedImport()

I don't know what this is for, but it's the only piece of code external
to idl_parser.cpp that expects the key of Parser::included_files_ to be
a path. And it appears to be unused.

* [idl_parser] Track included files by hash

Parser::included_files_ is a map whose main purpose is to keep track of
which files have already been parsed in order to protect against
multiple inclusion. Its key is the path that the file was found at
during parsing (or, if it's an in-memory file, just its name).

This commit changes the key to be the 64 bit FNV-1a hash of the file's
name (just the name, not the complete path) xor'd with the hash of the
file's contents (unless it's an in-memory file, then we only hash the
name.)

This allows multiple include protection to function even in the face of
unique per-file include paths (fixes #6425).

* Ran tests/generate_code.sh

CI told me to do it.

* Gracefullt handle case where source_filename == nullptr

I just learned source_filename might also be null. In that case, we
should exclude it from the hash instead of hashing a zero length
string, just like we exclude source when it is null.

Presumably nameless files will never be included (they can't, can they?)
so this doesn't really matter, but I think it's prettier anyways.
2021-03-01 12:34:01 -08:00
Kamil Rojewski bf90612007
fix for noUncheckedIndexedAccess in ts (#6474) 2021-02-25 14:57:10 -08:00
mustiikhalil 8142fedd19
Working on a python example plus fixing python grpc code (#6456)
Refactored python grpc code gen

Adds example server & client + fixes ci

Fixes generated code

Making sure we encode the reply string as utf8

Adds Readme details to clarify issue regarding encoding when python is sending/receiving
2021-02-25 14:38:12 -08:00
Colin c0be1cb7a5
[rust] Remove debug code (#6475)
* Remove debug code

This was added for testing in the recent genericize PR for flexbuffer Reader.

* Added alloc tests -> MapReader::{is_empty, index_key, len}

* Added , accessible through Deref to deprecation warning
2021-02-24 13:00:18 -05:00
Derek Bailey 8cccdfba53
Revert "[C#] Fix truncated ArraySegment<byte> if elementSize != 1 (#6462)" (#6488)
This reverts commit cbbbaa61b3.
2021-02-23 11:30:07 -08:00
Björn Harrtell cbbbaa61b3
[C#] Fix truncated ArraySegment<byte> if elementSize != 1 (#6462)
* WIP: Fix returned truncated ArraySegment<byte> if elementSize is not byte

* Fix

* Regenerated test code
2021-02-23 09:34:20 -08:00
mustiikhalil ffc2ef77ca
[CI] Adds Code Generation tests on Github Actions (#6482)
Moves check-grpc-generation to scripts
2021-02-23 08:51:39 +03:00
Millian Poquet 1da6f4f18b
[CMake] generate pkg-config flatbuffers.pc file (#6455)
This commit enables CMake to generate a flatbuffers.pc file on install.

pkg-config eases the utilization of software libraries by enabling the
developers of a library to define how the library should be used.

It can be used as a tool by build systems to find and manage dependencies,
this is notably the default Meson behavior and optionally used by CMake.
2021-02-22 10:23:38 -08:00
mustiikhalil b5da526e6d
[Swift] Moves grpc example to grpc/examples (#6479)
Updates generated Code

Removes grpc generation code from tests dir

Small fix to generate.sh
2021-02-20 23:07:48 +03:00
mustiikhalil 3b5365762d
[TS] Moves grpc code to examples folder (#6476)
Adds readme
2021-02-20 20:31:00 +03:00
mustiikhalil e2f5438ac1
Fixes grammer (#6477)
Updates docs
2021-02-19 12:16:38 -08:00