31 FlatBuffers release process

mustiikhalil edited this page 2022-09-27 23:37:09 +02:00

Table of Contents

• Update Version Identifiers
• Language Distributions
• Flatc and C++
• Python (PyPi)
• Java (Maven)
• C# (nuget)
• JavaScript/TypeScript (npm)
• Rust (crates.io)
• To Categorize / Old Notes

Steps to release a new version of FlatBuffers to github:

Update Version Identifiers

1. Update version in include/flatbuffers/base.h

2. Update version in CMake/Version.cmake

3. Update version-check in generated include/flatbuffers/reflection_generated.h

   This is needed because we have a catch-22 where we use this generated file within flatc, which makes that file. But it will complain that the flatbuffer libraries mismatch.

4. Update version-check in generated tests/evolution_v1_generated.h and tests/evolution_v2_generated.h

5. Update version in pom.xml

6. Update version in package.json

7. Update version in net/flatbuffers/Google.FlatBuffers.csproj

8. Update version in dart/pubspec.yaml

9. Update version in python/flatbuffers/_version.py

   See Python steps below to do it as part of the script

10. Find-Replace globally: FlatBuffersVersion_Y_Y_Y with FlatBuffersVersion_X_X_X

11. Find-Replace globally: FLATBUFFERS_Y_Y_Y() with FLATBUFFESR_X_X_X()

    Note: Skip the one in android/app/src/main/java for now.

12. Update version in FlatBuffers.podspec

Language Distributions

Flatc and C++

1. make clean

2. make -j

3. scripts/generated_code.py

4. Update CHANGELOG.md

5. Make a git commit with a message: FlatBuffers Version X.X.X

6. Go through the normal PR process to get the commit submitted

7. Generate an annotated tag for the release. Make sure this is done directly in the flatbuffer repo and not via a pull request.

   git tag -a -m "Flatbuffers Version X.X.X" vX.X.X

8. Push the tag to github (again not via a PR).

   git push origin vX.X.X

9. The result should be a tagged commit at the following: https://github.com/google/flatbuffers/tags

Python (PyPi)

1. cd python

2. Set VERSION environment to version.

   bash: export VERSION="X.X.X"

   fish: set -g -x VERSION X.X.X

3. python setup.py sdist

4. python setup.py bdist_wheel

5. Ensure twine in installed:

   python3 -m pip install --upgrade twine

6. twine upload dist/flatbuffers-X.X.X.tar.gz

   This will ask for your credentials for PyPi.

7. twine upload dist/flatbuffers-X.X.X-py2.py3-none-any.whl

8. Output should be on: https://pypi.org/project/flatbuffers/

Java (Maven)

1. Need gpg key installed in local keyring.

   1. gpg --import private.key

      If this times out, its because its asking for a password and doesn't show it to you.

      1. sudo apt install pinentry-tty
      2. sudo update-alternatives --config pinentry
      3. Select the pinentry-tty option.

   2. Set up .m2/settings.xml with

   <settings>
     <servers>
       <server>
         <id>ossrh</id>
         <username>FILL ME IN</username>
         <password>FILL ME IN</password>
       </server>
     </servers>
   </settings>
   

2. In root directory run mvn clean deploy -P release

   This will publish the version and you cannot republish the same version. So it might be good to do a "snapshot" deplyoment by appending -SNAPSHOT to the version.

3. It should prompt for your password during the signing stage.

4. It should eventually complete and show: https://search.maven.org/artifact/com.google.flatbuffers/flatbuffers-java

C# (nuget)

1. Its sometimes advisable to set a -beta.1 version in net/flatbuffers/Google.FlatBuffers.csproj first, to make sure everything is working with the published package. Then repeat without the beta tag.
2. cd net/flatbuffers
3. dotnet build Google.FlatBuffers.csproj -c Release
4. dotnet pack Google.FlatBuffers.csproj -c Release
5. dotnet nuget push .\bin\Release\Google.FlatBuffers.XXX.nupkg --api-key <API_KEY> --source https://api.nuget.org/v3/index.json
6. Should appear here in a few minutes: https://www.nuget.org/packages/Google.FlatBuffers/

JavaScript/TypeScript (npm)

1. npm login
2. npm publish
3. Should be live here relatively quickly: https://www.npmjs.com/package/flatbuffers

Rust (crates.io)

1. cd rust/flatbuffers/
2. Verify Cargo.toml version
3. cargo publish

To Categorize / Old Notes

• Change the version number in base.h, package.json, pom.xml (and various such files in grpc/), CMakeLists.txt (FlatBuffers_Library_SONAME_FULL), CMake/Version.cmake, dart/pubspec.yaml, rust/cargo.toml (not same as FlatBuffers version), Constants.java, FlatBufferConstants.cs, idl_gen_java/csharp/kotlin/swift.cpp
• re-build flatc
• runs generate_code.sh to ensure there's no missing files.
  • And also because the version number is in some generated files.
  • Also run SwiftTest.sh in tests/FlatBuffers.Test.Swift since it generates code (FIXME)
  • ../../../../flatc --swift --grpc greeter.fbs in tests/FlatBuffers.GRPC.Swift/Sources/Model (FIXME)
• (search for 1.12 and 1_12 to be sure).
• push to github.
• Update the documentation remote branch gh-pages that hosts what is shown at http://google.github.io/flatbuffers/ using something like (warning, this will go live instantly):
  • cd docs/source
  • doxygen
  • cd ../../.. (one up from flatbuffers root)
  • git clone -b gh-pages --single-branch https://github.com/google/flatbuffers.git gh-pages
  • cd gh-pages
  • cp -r ../flatbuffers/docs/html/* .
  • git add *
  • git commit
  • git push
  • cd ..
  • rm -rf gh-pages flatbuffers/docs/html
• Tag this release by version number, e.g.:
  • git tag -a -m "FlatBuffers release 1.0.0" v1.0.0
• git push yourgithubremotename v1.0.0
• On the github releases page, add a release description to this tag, with a description of what changed etc.
  • Also add flatc binaries from the last CI run for win/lin/mac
• For JS:
  • (version has already been changed in package.json)
  • (on npm <4.0): npm run append-esm-export
  • npm login
  • npm publish
  • (note npm adduser needs a newer node than what's available on my machine?)
  • Check changes are on: https://www.npmjs.com/package/flatbuffers
• For Java:
  • (version already changed in pom.xml)
  • mvn clean deploy -P release
  • Not needed / working?
    • mvn release:clean release:prepare
    • mvn release:perform
  • (see http://central.sonatype.org/pages/apache-maven.html for details).
  • This requires an account with oss.sonatype.org, a gpg key registered with a keyserver, and your user/pass in .m2/settings.xml (Maven) to work, among other mysterious things.
  • Note: some of the above commands tend to error out, since it is all some crazy async complex system.. but it will then succeed later anyway (shows up the new version on maven central).
  • For gRPC:
    • NOTE: since this depends on the main flatbuffers package, typically needs to wait until that is done uploading!
    • in dir grpc, mvn deploy
    • May need a mvn clean first if it fails: https://github.com/google/flatbuffers/issues/5856
• For Dart:
  • Run cd dart && sh publish.sh
  • Results here: https://pub.dev/packages/flat_buffers
• For Python:
  • cd python
  • export VERSION="1.9"
  • python setup.py sdist
  • python setup.py bdist_wheel
  • twine upload dist/flatbuffers-1.9.tar.gz
  • twine upload dist/flatbuffers-1.9-py2.py3-none-any.whl
  • rm -rf build
  • See release on: https://pypi.org/project/flatbuffers/
  • (more detail here: https://github.com/google/flatbuffers/issues/4507#issuecomment-396370441)
• For C#:
  • Follow these steps for a first time release: https://docs.microsoft.com/en-us/nuget/quickstart/create-and-publish-a-package-using-the-dotnet-cli
  • Then close https://github.com/google/flatbuffers/issues/4162