Commit Graph

22 Commits

Author SHA1 Message Date
Felix Frank ad8b1e5dbd [Python] Fast serialization of numpy vectors ()
[Python] Fast serialization of numpy vectors ()
2018-10-15 17:07:08 -07:00
Robert e1d5fda5d4 python: do not clobber minalign when we create objects () 2018-07-23 16:29:45 -07:00
Wouter van Oortmerssen e9912e9298 Added setup.cfg to default to python 2/3 wheels for pypi.
Change-Id: I64cf42aca79c32d21cd15c1415125ba97665d134
2018-06-25 15:08:06 -07:00
Robert Schmidtke 08cf50c54a Java/C#/Python prefixed size support ()
* initial changes to support size prefixed buffers in Java

* add slice equivalent to CSharp ByteBuffer

* resolve TODO for slicing in CSharp code generation

* add newly generated Java and CSharp test sources

* fix typo in comment

* add FinishSizePrefixed methods to CSharp FlatBufferBuilder as well

* add option to allow writing the prefix as well

* generate size-prefixed monster binary as well

* extend JavaTest to test the size prefixed binary as well

* use constants for size prefix length

* fuse common code for getRootAs and getSizePrefixedRootAs

* pulled file identifier out of if

* add FinishSizePrefixed, GetSizePrefixedRootAs support for Python

* Revert "extend JavaTest to test the size prefixed binary as well"

This reverts commit 68be4420dd.

* Revert "generate size-prefixed monster binary as well"

This reverts commit 2939516fdf.

* fix ByteBuffer.cs Slice() method; add proper CSharp and Java tests

* fix unused parameter

* increment version number

* pulled out generated methods into separate utility class

* pulled out generated methods into separate utility class for Python

* fix indentation

* remove unnecessary comment

* fix newline and copyright

* add ByteBufferUtil to csproj compilation

* hide ByteBuffer's internal data; track offset into parent's array

* test unsafe versions as well; compile and run in debug mode

* clarify help text for size prefix

* move ByteBuffer slicing behavior to subclass

* fix protection levels

* add size prefix support for text generation

* add ByteBufferSlice to csproj compilation

* revert size prefix handling for nested buffers

* use duplicate instead of slice for removing size prefix

* remove slice subclass and use duplicate for removing size prefix

* remove slice specific tests

* remove superfluous command line option
2018-03-12 11:30:46 -07:00
Michael Holler 0c8b4c7614 Add support for Python lib continuous deployment.
Use a combination of travis and twine to publish to PyPI. New
publications will be made:

* When `master` is updated. This will trigger the publication of a
  the Python artifact versioned an iso-compliant build datetime. In this
  way, the cutting edge version will always be available via PyPI.
* When a new git tag is pushed. Tag pushes trigger the publication of
  a python artifact with the same version as the git tag, with the
  leading `v` stripped if present (`v1.2.3` becomes `1.2.3`).

Publications rely on Travis having a PYPI_PASSWORD environment set in
the project settings. See the Travis CI documentation for information on
[setting environment variables which containing sensitive data][1]. Make
extra sure the "Display value in build log" switch is OFF.

In addition to setting the previously mentioned `PYPI_PASSWORD`
environment variable, the owner of the PyPI `flatbuffers` repository
should, after merging this commit into master, add his own commit to
change `mikeholler` in `.travis/deploy-python.sh` to his username. It's
also recommended that the owner of `flatbuffers` use a separate account
in the unlikely event that the environment variable somehow becomes
compromised. Again, this is very unlikely, since the environment
variable is only set for "safe" builds approved by maintainers (not on
random pull requests).

[1]: https://docs.travis-ci.com/user/environment-variables/#Defining-Variables-in-Repository-Settings
2017-11-27 13:14:33 -06:00
Robert 86fb05d320 Python: CreateByteVector function in builder. () 2017-10-06 08:34:07 -07:00
Wouter van Oortmerssen ac1015e3c4 Trimmed vtables of trailing zeroes.
This is something the format supports, but none of the builders
were doing. Can save 10-20% on FlatBuffer binary size!

Also fixed the Go tests.

Change-Id: I616c56ce9bbcfcaee23aa24f0532fcb60b6a8c75
Tested: on Linux.
2017-08-24 09:35:54 -07:00
Kevin Rose 3282a84e30 [Python] (scalar) vector reading speedup via numpy ()
* Add numpy accessor to python flatbuffers scalar vectors

* Update python tests to test numpy vector accessor

* Update appveyor CI to run Python tests, save generated code as artifact

* Update example generated python code

* Add numpy info to python usage docs

* Update test schema and python tests w/ multi-byte vector

* did not mean to push profiling code

* adding float64 numpy tests
2017-08-01 08:34:00 -07:00
Wouter van Oortmerssen 1a161a8333 Added missing licenses to some sh/cs/py files.
Bug: https://github.com/google/flatbuffers/issues/3872
Change-Id: I5d551168e9bc925e867e5e4ddf5d809418fd44eb
Tested: on Linux.
2016-05-23 09:51:41 -07:00
Alex Kerfoot 1e7310e6cd Fix CreateString with already-encoded string or bytearray in Python 2.7.
There was no way to pass an already-encoded string to 
`builder.CreateString` in Python 2.7:

- Passing a `bytearray` raised a TypeError because `bytearray` was not
  recognized as an instance of `compat.binary_type`.
- Passing a utf-8 encoded `str` would cause the string to be
  double-encoded, because `compat.string_types = (basestring,)` and
  `basestring` is the base class of `str` and `unicode`, so the logic
  would never reach the `elif isinstance(s, compat.binary_type)` case.
- Converting a utf-8 encoded bytearray to `bytes` like
  `builder.CreateString(bytes(encoded_string))` does not work because
  in Python 2.7, bytes is just an alias for `str` so it behaves as
  above.

This change allows either `bytes` or `bytearray` as an already-encoded
string to be passed to `CreateString` in versions of Python that support
`bytearray`, and falls back to `str` in older versions.

In Python 2, it restricts unencoded string types to `unicode`, so `str`
can be used as an encoded, binary representaiton.
2016-03-08 16:05:02 -08:00
rw a88cf317fd Python: remove ctypes dependency in runtime library. 2016-02-08 14:15:02 -08:00
Robert b7d97ff3de Merge pull request from faizanrashid/master
[BUG FIX] [MINOR] Fix encoding with unicode characters.
2016-01-19 17:56:26 -08:00
Mark Klara 69a31b807a Revamping the FlatBuffers docs.
Adding an API reference for the supported languages.

General docs cleanup, including a new `tutorial` section that
supports all of the supported languages.

Added samples for each supported language to mirror the new
tutorial page.

Cleaned up all the links by making them `@ref` style links,
instead of referencing the names of the generated `.html` files.

Removed all generated files that were unnecessarily committed.

Also fixed the C# tests (two were failing due to a missing file).

Bug: b/25801305

Tested: Tested all samples on Ubuntu, Mac, and Android. Docs were
generated using doxygen and viewed on Chrome.

Change-Id: I2acaba6e332a15ae2deff5f26a4a25da7bd2c954
2016-01-19 14:31:17 -08:00
Faizan Rashid af23683311 [BUG FIX] [MINOR] Fix encoding with unicode characters.
When passing a unicode string to builder.CreateString, the
default encoding assumed all characters can be encoded
using ascii. Added a fix so a user can specify the
encoding and how to handle errors when creating strings.
2015-12-31 09:45:09 +05:00
Faizan Rashid 853e34087a [BUG] [MINOR] Use buffer for specific py versions
Fix for Issue 1741
Minor bug where python versions 2.7.x where x < 5 do not support
unpacking from memoryview objects. Versions 2.7.5 and above will
use memoryview while 2.7 versions below 2.7.5  will use buffer
objects.

Manual testing was performed on versions 2.7.5 and 2.7.2 to
confirm both worked correctly.
2015-12-13 02:55:16 -05:00
rw 3232727ace Python: Improve Builder user interface.
+ Add state to the Builder object to track if we are inside a table,
  and if we are finished building the buffer.
+ Use this data to check that a buffer is being built correctly.
+ Raise an exception if a buffer is not being built correctly.
+ Test that the exceptions happen as expected.

Based on d236dea.
2015-11-11 19:43:53 -08:00
tguo-aa d9db48257b Add self.assertNotNested() in CreateString
And also add a test case.
If you try to nest CreateString you will get a clear exception.
2015-09-29 08:34:10 +08:00
Frank Stein 33e4ab65e9 Incorrect buffer size check fixed. 2015-08-14 00:50:39 +03:00
Frank Stein 7bcbb19569 MaxBufferSize() changed to MAX_BUFFER_SIZE. Also buffer will never grow more MAX_BUFFER_SIZE. 2015-08-07 21:19:02 +03:00
Frank Stein a1d801c375 2Gb buffer size checks fixed for Python Builder 2015-07-14 00:10:11 +03:00
rw 7a1b77322b Update Python package description: license, descriptions. 2015-05-15 12:20:20 -07:00
rw 48dfc69ee6 Port FlatBuffers to Python.
Implement code generation and self-contained runtime library for Python.

The test suite verifies:
  - Correctness of generated Python code by comparing output to that of
    the other language ports.
  - The exact bytes in the Builder buffer during many scenarios.
  - Vtable deduplication correctness.
  - Edge cases for table construction, via a fuzzer derived from the Go
    implementation.
  - All code is simultaneously valid in Python 2.6, 2.7, and 3.4.

The test suite includes benchmarks for:
  - Building 'gold' data.
  - Parsing 'gold' data.
  - Deduplicating vtables.

All tests pass on this author's system for the following Python
implementations:
  - CPython 2.6.7
  - CPython 2.7.8
  - CPython 3.4.2
  - PyPy 2.5.0 (CPython 2.7.8 compatible)
2015-05-12 15:40:29 -07:00