Migrate from rules_nodejs to rules_js/rules_ts (take 2) (#7928)
* Migrate from rules_nodejs to rules_js/rules_ts (take 2)
This is the second version of patch #7923. The first version got
reverted because bazel query was failing:
$ bazel --nosystem_rc --nohome_rc query tests(set('//...')) except tests(attr("tags", "manual", set('//...')))
ERROR: Traceback (most recent call last):
File "/workdir/tests/ts/bazel_repository_test_dir/BUILD", line 6, column 22, in <toplevel>
npm_link_all_packages(name = "node_modules")
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/npm/defs.bzl", line 188, column 13, in npm_link_all_packages
fail(msg)
Error in fail: The npm_link_all_packages() macro loaded from @npm//:defs.bzl and called in bazel package 'tests/ts/bazel_repository_test_dir' may only be called in bazel packages that correspond to the pnpm root package '' and pnpm workspace projects ''
This was happening because the `.bazelrc` file only added
`--deleted_packages` to the `build` command. We also need it for the
`query` command. This second version of the patch fixes that.
Original commit message:
This patch migrates the current use of rules_nodejs to the new rules_js.
rules_js is the intended replacement of rules_nodejs as per this note:
https://github.com/aspect-build/rules_js#relationship-to-rules_nodejs
> rules_js is an alternative to the build_bazel_rules_nodejs Bazel module
> and accompanying npm packages hosted in
> https://github.com/bazelbuild/rules_nodejs, which is now
> unmaintained. All users are recommended to use rules_js instead.
There are a few notable changes in this patch:
1. The `flatbuffer_ts_library` macro no longer accepts a `package_name`
attribute. This is because rules_js appears to manage the import
naming of dependencies via top-level `npm_link_package` targets.
Users will have to migrate.
2. I added a few more arguments to `flatbuffer_library_public()`. These
helped with exposing esbuild to `ts/compile_flat_file.sh`.
3. I pinned the version of `typescript` in `package.json` so that
rules_ts can download the exact same version. rules_ts doesn't know
what to do if the version isn't exact.
4. Since rules_js uses the pnpm locking mechanism, we now have a
`pnpm-lock.yaml` file instead of a yarn lock file.
4. I added bazel targets for a few of the existing tests in `tests/ts`.
They can be run with `bazel test //test/ts:all`. Since there is no
flexbuffers bazel target, I did not add a bazel target for the
corresponding test.
5. I added a separate workspace in `tests/ts/bazel_repository_test_dir/`
to validate that the flatbuffers code can be imported as an external
repository. You can run the test with
`bazel test //test/ts:bazel_repository_test`. For this to work, I
needed to expose a non-trivial chunk of the flatbuffers code to the
test. I achieved this through some recursive `distribution`
filegroups. This is inspired by rules_python's workspace tests.
I did not do anything special to validate that the `gen_reflections`
parameter works the same. This patch doesn't change anything about
the TypeScript generation.
As a side note: I am not an expert with rules_js. This patch is my
attempt based on my limited understanding of the rule set.
Fixes #7817
* Fix the query
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-03 18:48:15 +00:00
|
|
|
load("@aspect_bazel_lib//lib:copy_to_bin.bzl", "copy_to_bin")
|
2019-12-26 18:03:22 +00:00
|
|
|
load("@rules_cc//cc:defs.bzl", "cc_test")
|
2023-05-04 23:12:45 +00:00
|
|
|
load("//:build_defs.bzl", "DEFAULT_FLATC_ARGS", "flatbuffer_cc_library")
|
2019-12-26 18:03:22 +00:00
|
|
|
|
|
|
|
|
package(default_visibility = ["//visibility:private"])
|
2019-12-02 22:14:48 +00:00
|
|
|
|
Migrate from rules_nodejs to rules_js/rules_ts (take 2) (#7928)
* Migrate from rules_nodejs to rules_js/rules_ts (take 2)
This is the second version of patch #7923. The first version got
reverted because bazel query was failing:
$ bazel --nosystem_rc --nohome_rc query tests(set('//...')) except tests(attr("tags", "manual", set('//...')))
ERROR: Traceback (most recent call last):
File "/workdir/tests/ts/bazel_repository_test_dir/BUILD", line 6, column 22, in <toplevel>
npm_link_all_packages(name = "node_modules")
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/npm/defs.bzl", line 188, column 13, in npm_link_all_packages
fail(msg)
Error in fail: The npm_link_all_packages() macro loaded from @npm//:defs.bzl and called in bazel package 'tests/ts/bazel_repository_test_dir' may only be called in bazel packages that correspond to the pnpm root package '' and pnpm workspace projects ''
This was happening because the `.bazelrc` file only added
`--deleted_packages` to the `build` command. We also need it for the
`query` command. This second version of the patch fixes that.
Original commit message:
This patch migrates the current use of rules_nodejs to the new rules_js.
rules_js is the intended replacement of rules_nodejs as per this note:
https://github.com/aspect-build/rules_js#relationship-to-rules_nodejs
> rules_js is an alternative to the build_bazel_rules_nodejs Bazel module
> and accompanying npm packages hosted in
> https://github.com/bazelbuild/rules_nodejs, which is now
> unmaintained. All users are recommended to use rules_js instead.
There are a few notable changes in this patch:
1. The `flatbuffer_ts_library` macro no longer accepts a `package_name`
attribute. This is because rules_js appears to manage the import
naming of dependencies via top-level `npm_link_package` targets.
Users will have to migrate.
2. I added a few more arguments to `flatbuffer_library_public()`. These
helped with exposing esbuild to `ts/compile_flat_file.sh`.
3. I pinned the version of `typescript` in `package.json` so that
rules_ts can download the exact same version. rules_ts doesn't know
what to do if the version isn't exact.
4. Since rules_js uses the pnpm locking mechanism, we now have a
`pnpm-lock.yaml` file instead of a yarn lock file.
4. I added bazel targets for a few of the existing tests in `tests/ts`.
They can be run with `bazel test //test/ts:all`. Since there is no
flexbuffers bazel target, I did not add a bazel target for the
corresponding test.
5. I added a separate workspace in `tests/ts/bazel_repository_test_dir/`
to validate that the flatbuffers code can be imported as an external
repository. You can run the test with
`bazel test //test/ts:bazel_repository_test`. For this to work, I
needed to expose a non-trivial chunk of the flatbuffers code to the
test. I achieved this through some recursive `distribution`
filegroups. This is inspired by rules_python's workspace tests.
I did not do anything special to validate that the `gen_reflections`
parameter works the same. This patch doesn't change anything about
the TypeScript generation.
As a side note: I am not an expert with rules_js. This patch is my
attempt based on my limited understanding of the rule set.
Fixes #7817
* Fix the query
---------
Co-authored-by: Derek Bailey <derekbailey@google.com>
2023-05-03 18:48:15 +00:00
|
|
|
# rules_js works around various JS tooling limitations by copying everything
|
|
|
|
|
# into the output directory. Make the test data available to the tests this way.
|
|
|
|
|
copy_to_bin(
|
|
|
|
|
name = "test_data_copied_to_bin",
|
|
|
|
|
srcs = glob([
|
|
|
|
|
"*.mon",
|
|
|
|
|
"*.json",
|
|
|
|
|
]),
|
|
|
|
|
visibility = ["//tests/ts:__subpackages__"],
|
|
|
|
|
)
|
|
|
|
|
|
2019-12-02 22:14:48 +00:00
|
|
|
# Test binary.
|
|
|
|
|
cc_test(
|
|
|
|
|
name = "flatbuffers_test",
|
|
|
|
|
testonly = 1,
|
|
|
|
|
srcs = [
|
2023-05-09 16:16:30 +00:00
|
|
|
"64bit/evolution/v1_generated.h",
|
|
|
|
|
"64bit/evolution/v2_generated.h",
|
|
|
|
|
"64bit/offset64_test.cpp",
|
|
|
|
|
"64bit/offset64_test.h",
|
|
|
|
|
"64bit/test_64bit_bfbs_generated.h",
|
|
|
|
|
"64bit/test_64bit_generated.h",
|
2022-09-21 18:05:05 +00:00
|
|
|
"alignment_test.cpp",
|
|
|
|
|
"alignment_test.h",
|
|
|
|
|
"alignment_test_generated.h",
|
2022-08-28 23:54:58 +00:00
|
|
|
"evolution_test.cpp",
|
|
|
|
|
"evolution_test.h",
|
2019-12-02 22:14:48 +00:00
|
|
|
"evolution_test/evolution_v1_generated.h",
|
|
|
|
|
"evolution_test/evolution_v2_generated.h",
|
2022-08-28 23:54:58 +00:00
|
|
|
"flexbuffers_test.cpp",
|
|
|
|
|
"flexbuffers_test.h",
|
|
|
|
|
"fuzz_test.cpp",
|
|
|
|
|
"fuzz_test.h",
|
|
|
|
|
"is_quiet_nan.h",
|
|
|
|
|
"json_test.cpp",
|
|
|
|
|
"json_test.h",
|
2022-11-18 19:04:46 +00:00
|
|
|
"key_field/key_field_sample_generated.h",
|
|
|
|
|
"key_field_test.cpp",
|
|
|
|
|
"key_field_test.h",
|
2022-08-28 23:54:58 +00:00
|
|
|
"monster_test.cpp",
|
|
|
|
|
"monster_test.h",
|
2020-07-21 18:08:01 +00:00
|
|
|
"monster_test_bfbs_generated.h",
|
2019-12-02 22:14:48 +00:00
|
|
|
"namespace_test/namespace_test1_generated.h",
|
|
|
|
|
"namespace_test/namespace_test2_generated.h",
|
2022-08-29 19:48:10 +00:00
|
|
|
"native_inline_table_test_generated.h",
|
2019-12-02 22:14:48 +00:00
|
|
|
"native_type_test_impl.cpp",
|
|
|
|
|
"native_type_test_impl.h",
|
2021-04-05 09:34:42 +00:00
|
|
|
"optional_scalars_generated.h",
|
2022-08-28 23:54:58 +00:00
|
|
|
"optional_scalars_test.cpp",
|
|
|
|
|
"optional_scalars_test.h",
|
|
|
|
|
"parser_test.cpp",
|
|
|
|
|
"parser_test.h",
|
|
|
|
|
"proto_test.cpp",
|
|
|
|
|
"proto_test.h",
|
|
|
|
|
"reflection_test.cpp",
|
|
|
|
|
"reflection_test.h",
|
2019-12-02 22:14:48 +00:00
|
|
|
"test.cpp",
|
|
|
|
|
"test_assert.cpp",
|
|
|
|
|
"test_assert.h",
|
|
|
|
|
"test_builder.cpp",
|
|
|
|
|
"test_builder.h",
|
2023-05-15 04:22:38 +00:00
|
|
|
"union_underlying_type_test_generated.h",
|
2019-12-02 22:14:48 +00:00
|
|
|
"union_vector/union_vector_generated.h",
|
2022-08-28 23:54:58 +00:00
|
|
|
"util_test.cpp",
|
|
|
|
|
"util_test.h",
|
2019-12-02 22:14:48 +00:00
|
|
|
],
|
|
|
|
|
copts = [
|
|
|
|
|
"-DFLATBUFFERS_TRACK_VERIFIER_BUFFER_SIZE",
|
|
|
|
|
],
|
|
|
|
|
data = [
|
2022-09-21 18:05:05 +00:00
|
|
|
":alignment_test.fbs",
|
2019-12-02 22:14:48 +00:00
|
|
|
":arrays_test.bfbs",
|
|
|
|
|
":arrays_test.fbs",
|
|
|
|
|
":arrays_test.golden",
|
|
|
|
|
":evolution_test/evolution_v1.fbs",
|
|
|
|
|
":evolution_test/evolution_v1.json",
|
|
|
|
|
":evolution_test/evolution_v2.fbs",
|
|
|
|
|
":evolution_test/evolution_v2.json",
|
|
|
|
|
":include_test/include_test1.fbs",
|
|
|
|
|
":include_test/sub/include_test2.fbs",
|
2022-11-18 19:04:46 +00:00
|
|
|
":key_field/key_field_sample.fbs",
|
2019-12-02 22:14:48 +00:00
|
|
|
":monster_extra.fbs",
|
|
|
|
|
":monster_test.bfbs",
|
|
|
|
|
":monster_test.fbs",
|
|
|
|
|
":monsterdata_extra.json",
|
|
|
|
|
":monsterdata_test.golden",
|
|
|
|
|
":monsterdata_test.json",
|
2021-05-14 20:56:52 +00:00
|
|
|
":name_clash_test/invalid_test1.fbs",
|
|
|
|
|
":name_clash_test/invalid_test2.fbs",
|
|
|
|
|
":name_clash_test/valid_test1.fbs",
|
|
|
|
|
":name_clash_test/valid_test2.fbs",
|
2019-12-02 22:14:48 +00:00
|
|
|
":native_type_test.fbs",
|
2021-04-05 09:34:42 +00:00
|
|
|
":optional_scalars.fbs",
|
2022-06-16 16:26:44 +00:00
|
|
|
":optional_scalars.json",
|
|
|
|
|
":optional_scalars_defaults.json",
|
2019-12-02 22:14:48 +00:00
|
|
|
":prototest/imported.proto",
|
2023-02-01 19:17:35 +00:00
|
|
|
":prototest/non-positive-id.proto",
|
|
|
|
|
":prototest/test.golden.fbs",
|
2019-12-02 22:14:48 +00:00
|
|
|
":prototest/test.proto",
|
2023-02-01 19:17:35 +00:00
|
|
|
":prototest/test_id.golden.fbs",
|
|
|
|
|
":prototest/test_include.golden.fbs",
|
|
|
|
|
":prototest/test_include_id.golden.fbs",
|
|
|
|
|
":prototest/test_suffix.golden.fbs",
|
|
|
|
|
":prototest/test_suffix_id.golden.fbs",
|
|
|
|
|
":prototest/test_union.golden.fbs",
|
|
|
|
|
":prototest/test_union_id.golden.fbs",
|
|
|
|
|
":prototest/test_union_include.golden.fbs",
|
|
|
|
|
":prototest/test_union_include_id.golden.fbs",
|
|
|
|
|
":prototest/test_union_suffix.golden.fbs",
|
|
|
|
|
":prototest/test_union_suffix_id.golden.fbs",
|
|
|
|
|
":prototest/twice-id.proto",
|
|
|
|
|
":prototest/use-reserved-id.proto",
|
2019-12-02 22:14:48 +00:00
|
|
|
":unicode_test.json",
|
|
|
|
|
":union_vector/union_vector.fbs",
|
|
|
|
|
":union_vector/union_vector.json",
|
|
|
|
|
],
|
|
|
|
|
includes = [
|
|
|
|
|
"",
|
|
|
|
|
"include/",
|
|
|
|
|
],
|
|
|
|
|
deps = [
|
2022-09-21 18:05:05 +00:00
|
|
|
":alignment_test_cc_fbs",
|
2019-12-02 22:14:48 +00:00
|
|
|
":arrays_test_cc_fbs",
|
|
|
|
|
":monster_extra_cc_fbs",
|
|
|
|
|
":monster_test_cc_fbs",
|
|
|
|
|
":native_type_test_cc_fbs",
|
|
|
|
|
"//:flatbuffers",
|
2023-04-28 20:40:38 +00:00
|
|
|
"//src:generate_fbs",
|
2019-12-02 22:14:48 +00:00
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Test bzl rules
|
|
|
|
|
|
2020-07-21 18:08:01 +00:00
|
|
|
cc_library(
|
|
|
|
|
name = "test_assert",
|
|
|
|
|
srcs = ["test_assert.cpp"],
|
|
|
|
|
hdrs = ["test_assert.h"],
|
|
|
|
|
visibility = ["//grpc/tests:__subpackages__"],
|
|
|
|
|
deps = ["//:flatbuffers"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
cc_library(
|
|
|
|
|
name = "test_builder",
|
|
|
|
|
srcs = ["test_builder.cpp"],
|
|
|
|
|
hdrs = ["test_builder.h"],
|
|
|
|
|
visibility = ["//grpc/tests:__subpackages__"],
|
|
|
|
|
deps = [
|
|
|
|
|
":monster_test_grpc",
|
|
|
|
|
":test_assert",
|
|
|
|
|
"//:flatbuffers",
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
cc_library(
|
|
|
|
|
name = "monster_test_grpc",
|
|
|
|
|
srcs = [
|
|
|
|
|
"monster_test.grpc.fb.cc",
|
|
|
|
|
"monster_test.grpc.fb.h",
|
|
|
|
|
"monster_test_generated.h",
|
|
|
|
|
],
|
|
|
|
|
hdrs = [
|
|
|
|
|
"monster_test.grpc.fb.h",
|
2023-05-04 23:12:45 +00:00
|
|
|
"monster_test_bfbs_generated.h",
|
2020-07-21 18:08:01 +00:00
|
|
|
"monster_test_generated.h",
|
|
|
|
|
],
|
|
|
|
|
includes = ["."],
|
|
|
|
|
visibility = ["//grpc/tests:__subpackages__"],
|
|
|
|
|
deps = [
|
|
|
|
|
"//:flatbuffers",
|
|
|
|
|
"@com_github_grpc_grpc//:grpc++",
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
|
2019-12-02 22:14:48 +00:00
|
|
|
flatbuffer_cc_library(
|
2022-02-22 04:33:47 +00:00
|
|
|
name = "include_test_fbs",
|
|
|
|
|
srcs = [
|
2019-12-02 22:14:48 +00:00
|
|
|
"include_test/include_test1.fbs",
|
|
|
|
|
"include_test/sub/include_test2.fbs",
|
|
|
|
|
],
|
2022-02-22 04:33:47 +00:00
|
|
|
include_paths = ["tests/include_test"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "monster_test_cc_fbs",
|
|
|
|
|
srcs = ["monster_test.fbs"],
|
2023-05-04 23:12:45 +00:00
|
|
|
outs = ["monster_test_bfbs_generated.h"],
|
|
|
|
|
flatc_args = DEFAULT_FLATC_ARGS + [
|
|
|
|
|
"--bfbs-comments",
|
|
|
|
|
"--bfbs-builtins",
|
|
|
|
|
"--bfbs-gen-embed",
|
|
|
|
|
"--bfbs-filenames tests",
|
|
|
|
|
],
|
2022-02-22 04:33:47 +00:00
|
|
|
include_paths = ["tests/include_test"],
|
2020-07-21 18:08:01 +00:00
|
|
|
visibility = ["//grpc/tests:__subpackages__"],
|
2022-02-22 04:33:47 +00:00
|
|
|
deps = [":include_test_fbs"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Test that running without --no-includes works properly (monster_test doesn't
|
|
|
|
|
# work cleanly due to the circular dependency in the include_tests/ files).
|
|
|
|
|
include_test_args = [
|
|
|
|
|
"--gen-object-api",
|
|
|
|
|
"--gen-compare",
|
|
|
|
|
"--gen-mutable",
|
|
|
|
|
"--reflect-names",
|
|
|
|
|
"--cpp-ptr-type flatbuffers::unique_ptr",
|
|
|
|
|
"--force-empty",
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "included_test_fbs",
|
|
|
|
|
srcs = ["included_test.fbs"],
|
|
|
|
|
flatc_args = include_test_args,
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "includer_test_fbs",
|
|
|
|
|
srcs = ["includer_test.fbs"],
|
|
|
|
|
flatc_args = include_test_args,
|
|
|
|
|
deps = [":included_test_fbs"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
cc_library(
|
|
|
|
|
name = "include_build_test",
|
|
|
|
|
srcs = ["include_build_test.cc"],
|
|
|
|
|
deps = [":includer_test_fbs"],
|
2019-12-02 22:14:48 +00:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "monster_extra_cc_fbs",
|
|
|
|
|
srcs = ["monster_extra.fbs"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "arrays_test_cc_fbs",
|
|
|
|
|
srcs = ["arrays_test.fbs"],
|
|
|
|
|
flatc_args = [
|
|
|
|
|
"--gen-object-api",
|
|
|
|
|
"--gen-compare",
|
|
|
|
|
"--no-includes",
|
|
|
|
|
"--gen-mutable",
|
|
|
|
|
"--reflect-names",
|
|
|
|
|
"--cpp-ptr-type flatbuffers::unique_ptr",
|
|
|
|
|
"--scoped-enums",
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "native_type_test_cc_fbs",
|
|
|
|
|
srcs = ["native_type_test.fbs"],
|
|
|
|
|
flatc_args = [
|
|
|
|
|
"--gen-object-api",
|
|
|
|
|
"--gen-mutable",
|
|
|
|
|
"--cpp-ptr-type flatbuffers::unique_ptr",
|
|
|
|
|
],
|
|
|
|
|
)
|
2022-09-21 18:05:05 +00:00
|
|
|
|
|
|
|
|
flatbuffer_cc_library(
|
|
|
|
|
name = "alignment_test_cc_fbs",
|
|
|
|
|
srcs = ["alignment_test.fbs"],
|
|
|
|
|
)
|