If you used flatbuffers_ts_library with gen_reflections = True then it
attempted to use the flat-file compiler rather than flatc itself.
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Add binary schema reflection
* remove not-used parameter
* move logic from object API to base API
* forward declare
* remove duplicate code gen that was stompping on the edits
* reduce to just typedef generation
* fixed bazel rules to not stomp
* more bazel fixes to support additional generated files
* Migrate from rules_nodejs to rules_js/rules_ts (take 2)
This is the second version of patch #7923. The first version got
reverted because bazel query was failing:
$ bazel --nosystem_rc --nohome_rc query tests(set('//...')) except tests(attr("tags", "manual", set('//...')))
ERROR: Traceback (most recent call last):
File "/workdir/tests/ts/bazel_repository_test_dir/BUILD", line 6, column 22, in <toplevel>
npm_link_all_packages(name = "node_modules")
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/npm/defs.bzl", line 188, column 13, in npm_link_all_packages
fail(msg)
Error in fail: The npm_link_all_packages() macro loaded from @npm//:defs.bzl and called in bazel package 'tests/ts/bazel_repository_test_dir' may only be called in bazel packages that correspond to the pnpm root package '' and pnpm workspace projects ''
This was happening because the `.bazelrc` file only added
`--deleted_packages` to the `build` command. We also need it for the
`query` command. This second version of the patch fixes that.
Original commit message:
This patch migrates the current use of rules_nodejs to the new rules_js.
rules_js is the intended replacement of rules_nodejs as per this note:
https://github.com/aspect-build/rules_js#relationship-to-rules_nodejs
> rules_js is an alternative to the build_bazel_rules_nodejs Bazel module
> and accompanying npm packages hosted in
> https://github.com/bazelbuild/rules_nodejs, which is now
> unmaintained. All users are recommended to use rules_js instead.
There are a few notable changes in this patch:
1. The `flatbuffer_ts_library` macro no longer accepts a `package_name`
attribute. This is because rules_js appears to manage the import
naming of dependencies via top-level `npm_link_package` targets.
Users will have to migrate.
2. I added a few more arguments to `flatbuffer_library_public()`. These
helped with exposing esbuild to `ts/compile_flat_file.sh`.
3. I pinned the version of `typescript` in `package.json` so that
rules_ts can download the exact same version. rules_ts doesn't know
what to do if the version isn't exact.
4. Since rules_js uses the pnpm locking mechanism, we now have a
`pnpm-lock.yaml` file instead of a yarn lock file.
4. I added bazel targets for a few of the existing tests in `tests/ts`.
They can be run with `bazel test //test/ts:all`. Since there is no
flexbuffers bazel target, I did not add a bazel target for the
corresponding test.
5. I added a separate workspace in `tests/ts/bazel_repository_test_dir/`
to validate that the flatbuffers code can be imported as an external
repository. You can run the test with
`bazel test //test/ts:bazel_repository_test`. For this to work, I
needed to expose a non-trivial chunk of the flatbuffers code to the
test. I achieved this through some recursive `distribution`
filegroups. This is inspired by rules_python's workspace tests.
I did not do anything special to validate that the `gen_reflections`
parameter works the same. This patch doesn't change anything about
the TypeScript generation.
As a side note: I am not an expert with rules_js. This patch is my
attempt based on my limited understanding of the rule set.
Fixes#7817
* Fix the query
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* Start using pnpm
* Add @npm
* get more stuff set up
* Get the analysis phase passing.
* Get esbuild working?
* Get it compiling?
$ bazel build //tests/ts/...
* Try to get the test working
* test is passing
* Get the other tests working
* clarify comment
* clean up a bit
* Try to add another test
* Add another test
* clean up more
* remove unused reference
* Add e2e test
* Get more of the test working
* add lock file
* Get test working on its own
* Get e2e test passing
* fix infinite recursion
* Add comments
* clean up some more
* clean up more again
* Source typescript version from package.json
* run buildifier
* lint
* Fix unset `extra_env`
* Incorporate feedback
* run buildifier
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
* [TS/JS] Entry point per namespace
* Fix handling of outputpath and array_test
* Attempt to fix generate_code
* Fix cwd for ts in generate_code
* Attempt to fixup bazel and some docs
* Add --ts-flat-files to bazel build to get bundle
* Move to DEFAULT_FLATC_TS_ARGS
* Attempt to add esbuild
* Attempt to use npm instead
* Remove futile attempt to add esbuild
* Attempt to as bazel esbuild
* Shuffle
* Upgrade bazel deps
* Revert failed attempts to get bazel working
* Ignore flatc tests for now
* Add esbuild dependency
* `package.json` Include esbuild
* `WORKSPACE` Add fetching esbuild binary
* Update WORKSPACE
* Unfreeze Lockfile
* Update WORKSPACE
* Update BUILD.bazel
* Rework to suggest instead of running external bundler
* Add esbuild generation to test script
* Prelim bundle test
* Run test JavaScriptTest from flatbuffers 1.x
* Deps upgrade
* Clang format fix
* Revert bazel changes
* Fix newline
* Generate with type declarations
* Handle "empty" root namespace
* Adjust tests for typescript_keywords.ts
* Separate test procedure for old node resolution module output
* Fix rel path for root level re-exports
* Bazel support for esbuild-based flatc
Unfortunately, we lose typing information because the new esbuild method
of generating single files does not generate type information.
The method used here is a bit hack-ish because it relies on parsing the
console output of flatc to figure out what to do.
* Try to fix bazel build for when node isn't present on host
* Auto formatting fixes
* Fix missing generated code
Co-authored-by: Derek Bailey <derekbailey@google.com>
Co-authored-by: James Kuszmaul <jabukuszmaul+collab@gmail.com>
The headline here is adding a flatbuffer_ts_library rule for generating
typescript code in bazel. This entails some non-trivial other changes,
but ideally none are user-visible.
In particular:
* Added a --ts-flat-file flag that generates a single *_generated.ts
file instead of separate files for each typescript type. This makes
bazel much happier.
* Import the bazel rules_nodejs stuff needed to support building
typescript in bazel
* Move flatbuffers.ts to index.ts because I wasn't sure how to make
bazel comprehend the "main" attribute of the package.json. Happy
to take another stab at figuring that out if really needed.
* Fix another couple keyword escaping spots in typescript...
I'm not seeing the reason why we didn't attempt to support transitive
dependencies for flatbuffer_cc_library, and the current setup makes
having to propagate new dependencies to all of their recursive
dependents obnoxious.
* Add support for compatible_with and restricted_to
These attributes have been available in Bazel for years. Pass them
through so the flatbuffer rules can be used with them. They let you
constrain which target platform is used.
While we are here, fix gen_reflections to work with bazel.
* Add docs
* Stop building for Windows until the build passes
ERROR: D:/b/bk-windows-java8-bd0z/bazel/flatbuffers/BUILD:123:1: Couldn't build file _objs/flatbuffers_test/test.obj: undeclared inclusion(s) in rule '//:flatbuffers_test':
this rule is missing dependency declarations for the following files included by 'tests/test.cpp':
'tests/monster_test_generated.h'
'tests/monster_extra_generated.h'
The files in tests are being found instead of the generated files since
Windows doesn't have any sandboxing. For now, let's disable the rules
and come back to it.
* Fix buildifier warnings
Clean up docstrings and add a module docstring.
In flatbuffers, build_defs.bzl has been updated to have the
bazel rule `flatbuffer_cc_library` defined. Therefore, it should
be possible to build another application and using `flatbuffer_cc_library`
directly (by `load("@com_github_google_flatbuffers//:build_defs.bzl", "flatbuffer_cc_library")`)
However, when I tried to do the above, I saw the following errors in bazel:
```
ERROR: /root/.cache/bazel/_bazel_root/c27e9809996ce9a9c0ed8dd79ef0897b/external/arrow/BUILD.bazel:12:1: in deps attribute of cc_library rule @arrow//:arrow_format: target '@arrow//:runtime_cc' does not exist. Since this rule was created by the macro 'flatbuffer_cc_library', the error might have been caused by the macro implementation in /root/.cache/bazel/_bazel_root/c27e9809996ce9a9c0ed8dd79ef0897b/external/com_github_google_flatbuffers/build_defs.bzl:216:16
```
The reason for the bazel error was that `//:runtime_cc` and `//:flatc` does not have
the repo name prefixed.
By prefix `` the above bazel build error could be resolved.
This fix should help other programs to use flatbuffers directly through bazel.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add flatbuffer_cc library support
* Update flags so all the tests pass
Tests now all pass!
* Modify the tests to use the generated code
This should be a simple serialize/deserialize test of the new generated
code to make sure the bazel rules are doing something sane.
* Use generated monster_test.fb in testing/test.cpp
cmake drops it's generated code in tests/monster_test_generated.h
Instead of checking that in, let's generate it with bazel.
* Make grpc tests depend on monster_test_generated.h
* Remove redundant cmake dependency
This should address @aardappel's feedback.
* Run flatc for Android as well
This will fix the last travis.ci failure
* Add generated output folder and fix flags
* Move flatbuffers_header_build_rules to the library that uses it
* Use --cpp-ptr-type to fix android
Android was the only target using the STL emulation layer. It needed
the --cpp-ptr-type flatbuffers::unique_ptr flag to work. Add it!
* Roll back changes to use autogenerated monster_test_generated.
Flip tests/test.cpp to use the autogenerated file as well.
James Kuszmaul