Rooba
/
flatbuffers
mirror of https://github.com/google/flatbuffers.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Updated benchmark chart with stats for pugixml. 

Change-Id: I23d7db5b62ca52fee49e1bedcf7d7d8d74957cf8
Tested: on Windows.
This commit is contained in:
Wouter van Oortmerssen 2014-07-18 16:15:10 -07:00
parent 127d35085a
commit d03ba640be
2 changed files with 24 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
docs/html/md__benchmarks.html
View File

@ -53,37 +53,36 @@ $(document).ready(function(){initNavTree('md__benchmarks.html','');});
<div class="title">Benchmarks </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>Comparing against other serialization solutions, running on Windows 7 64bit. We use the LITE runtime for Protocol Buffers (less code / lower overhead), and Rapid JSON, one of the fastest C++ JSON parsers around.</p>
<div class="textblock"><p>Comparing against other serialization solutions, running on Windows 7 64bit. We use the LITE runtime for Protocol Buffers (less code / lower overhead), Rapid JSON (one of the fastest C++ JSON parsers around), and pugixml, also one of the fastest XML parsers.</p>
<p>We compare against Flatbuffers with the binary wire format (as intended), and also with JSON as the wire format with the optional JSON parser (which, using a schema, parses JSON into a binary buffer that can then be accessed as before).</p>
<p>The benchmark object is a set of about 10 objects containing an array, 4 strings, and a large variety of int/float scalar values of all sizes, meant to be representative of game data, e.g. a scene format.</p>
<table class="doxtable">
<tr>
<th></th><th>FlatBuffers (binary) </th><th>Protocol Buffers LITE </th><th>Rapid JSON </th><th>FlatBuffers (JSON) </th></tr>
<th></th><th>FlatBuffers (binary) </th><th>Protocol Buffers LITE </th><th>Rapid JSON </th><th>FlatBuffers (JSON) </th><th>pugixml </th></tr>
<tr>
<td>Decode + Traverse + Dealloc (1 million times, seconds) </td><td>0.08 </td><td>302 </td><td>583 </td><td>105 </td></tr>
<td>Decode + Traverse + Dealloc (1 million times, seconds) </td><td>0.08 </td><td>302 </td><td>583 </td><td>105 </td><td>196 </td></tr>
<tr>
<td>Decode / Traverse / Dealloc (breakdown) </td><td>0 / 0.08 / 0 </td><td>220 / 0.15 / 81 </td><td>294 / 0.9 / 287 </td><td>70 / 0.08 / 35 </td></tr>
<td>Decode / Traverse / Dealloc (breakdown) </td><td>0 / 0.08 / 0 </td><td>220 / 0.15 / 81 </td><td>294 / 0.9 / 287 </td><td>70 / 0.08 / 35 </td><td>41 / 3.9 / 150 </td></tr>
<tr>
<td>Encode (1 million times, seconds) </td><td>3.2 </td><td>185 </td><td>650 </td><td>169 </td></tr>
<td>Encode (1 million times, seconds) </td><td>3.2 </td><td>185 </td><td>650 </td><td>169 </td><td>273 </td></tr>
<tr>
<td>Wire format size (normal / zlib, bytes) </td><td>344 / 220 </td><td>228 / 174 </td><td>1475 / 322 </td><td>1029 / 298 </td></tr>
<td>Wire format size (normal / zlib, bytes) </td><td>344 / 220 </td><td>228 / 174 </td><td>1475 / 322 </td><td>1029 / 298 </td><td>1137 / 341 </td></tr>
<tr>
<td>Memory needed to store decoded wire (bytes / blocks) </td><td>0 / 0 </td><td>760 / 20 </td><td>65689 / 40 </td><td>328 / 1 </td></tr>
<td>Memory needed to store decoded wire (bytes / blocks) </td><td>0 / 0 </td><td>760 / 20 </td><td>65689 / 4 </td><td>328 / 1 </td><td>34194 / 3 </td></tr>
<tr>
<td>Transient memory allocated during decode (KB) </td><td>0 </td><td>1 </td><td>131 </td><td>4 </td></tr>
<td>Transient memory allocated during decode (KB) </td><td>0 </td><td>1 </td><td>131 </td><td>4 </td><td>34 </td></tr>
<tr>
<td>Generated source code size (KB) </td><td>4 </td><td>61 </td><td>0 </td><td>4 </td></tr>
<td>Generated source code size (KB) </td><td>4 </td><td>61 </td><td>0 </td><td>4 </td><td>0 </td></tr>
<tr>
<td>Field access in handwritten traversal code </td><td>accessors </td><td>accessors </td><td>manual error checking </td><td>accessors </td></tr>
<td>Field access in handwritten traversal code </td><td>typed accessors </td><td>typed accessors </td><td>manual error checking </td><td>typed accessors </td><td>manual error checking </td></tr>
<tr>
<td>Library source code (KB) </td><td>15 </td><td>some subset of 3800 </td><td>87 </td><td>43 </td></tr>
<td>Library source code (KB) </td><td>15 </td><td>some subset of 3800 </td><td>87 </td><td>43 </td><td>327 </td></tr>
</table>
<h3>Some other serialization systems we compared against but did not benchmark (yet), in rough order of applicability:</h3>
<ul>
<li>Cap'n'Proto promises to reduce Protocol Buffers much like FlatBuffers does, though with a more complicated binary encoding and less flexibility (no optional fields to allow deprecating fields or serializing with missing fields for which defaults exist). It currently also isn't fully cross-platform portable (lack of VS support).</li>
<li>msgpack: has very minimal forwards/backwards compatability support when used with the typed C++ interface. Also lacks VS2010 support.</li>
<li>Thrift: very similar to Protocol Buffers, but appears to be less efficient, and have more dependencies.</li>
<li>XML: typically even slower than JSON, but has the advantage that it can be parsed with a schema to reduce error-checking boilerplate code.</li>
<li>YAML: a superset of JSON and otherwise very similar. Used by e.g. Unity.</li>
<li>C# comes with built-in serialization functionality, as used by Unity also. Being tied to the language, and having no automatic versioning support limits its applicability.</li>
<li>Project Anarchy (the free mobile engine by Havok) comes with a serialization system, that however does no automatic versioning (have to code around new fields manually), is very much tied to the rest of the engine, and works without a schema to generate code (tied to your C++ class definition). </li>

27
docs/source/Benchmarks.md
View File

@ -2,7 +2,8 @@
Comparing against other serialization solutions, running on Windows 7
64bit. We use the LITE runtime for Protocol Buffers (less code / lower
overhead), and Rapid JSON, one of the fastest C++ JSON parsers around.
overhead), Rapid JSON (one of the fastest C++ JSON parsers around),
and pugixml, also one of the fastest XML parsers.
We compare against Flatbuffers with the binary wire format (as
intended), and also with JSON as the wire format with the optional JSON
@ -13,17 +14,17 @@ The benchmark object is a set of about 10 objects containing an array, 4
strings, and a large variety of int/float scalar values of all sizes,
meant to be representative of game data, e.g. a scene format.
| | FlatBuffers (binary) | Protocol Buffers LITE | Rapid JSON | FlatBuffers (JSON) |
|--------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|
| Decode + Traverse + Dealloc (1 million times, seconds) | 0.08 | 302 | 583 | 105 |
| Decode / Traverse / Dealloc (breakdown) | 0 / 0.08 / 0 | 220 / 0.15 / 81 | 294 / 0.9 / 287 | 70 / 0.08 / 35 |
| Encode (1 million times, seconds) | 3.2 | 185 | 650 | 169 |
| Wire format size (normal / zlib, bytes) | 344 / 220 | 228 / 174 | 1475 / 322 | 1029 / 298 |
| Memory needed to store decoded wire (bytes / blocks) | 0 / 0 | 760 / 20 | 65689 / 40 | 328 / 1 |
| Transient memory allocated during decode (KB) | 0 | 1 | 131 | 4 |
| Generated source code size (KB) | 4 | 61 | 0 | 4 |
| Field access in handwritten traversal code | accessors | accessors | manual error checking | accessors |
| Library source code (KB) | 15 | some subset of 3800 | 87 | 43 |
| | FlatBuffers (binary) | Protocol Buffers LITE | Rapid JSON | FlatBuffers (JSON) | pugixml |
|--------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------| ----------------------|
| Decode + Traverse + Dealloc (1 million times, seconds) | 0.08 | 302 | 583 | 105 | 196 |
| Decode / Traverse / Dealloc (breakdown) | 0 / 0.08 / 0 | 220 / 0.15 / 81 | 294 / 0.9 / 287 | 70 / 0.08 / 35 | 41 / 3.9 / 150 |
| Encode (1 million times, seconds) | 3.2 | 185 | 650 | 169 | 273 |
| Wire format size (normal / zlib, bytes) | 344 / 220 | 228 / 174 | 1475 / 322 | 1029 / 298 | 1137 / 341 |
| Memory needed to store decoded wire (bytes / blocks) | 0 / 0 | 760 / 20 | 65689 / 4 | 328 / 1 | 34194 / 3 |
| Transient memory allocated during decode (KB) | 0 | 1 | 131 | 4 | 34 |
| Generated source code size (KB) | 4 | 61 | 0 | 4 | 0 |
| Field access in handwritten traversal code | typed accessors | typed accessors | manual error checking | typed accessors | manual error checking |
| Library source code (KB) | 15 | some subset of 3800 | 87 | 43 | 327 |
### Some other serialization systems we compared against but did not benchmark (yet), in rough order of applicability:
@ -36,8 +37,6 @@ meant to be representative of game data, e.g. a scene format.
with the typed C++ interface. Also lacks VS2010 support.
- Thrift: very similar to Protocol Buffers, but appears to be less efficient,
and have more dependencies.
- XML: typically even slower than JSON, but has the advantage that it can be
parsed with a schema to reduce error-checking boilerplate code.
- YAML: a superset of JSON and otherwise very similar. Used by e.g. Unity.
- C# comes with built-in serialization functionality, as used by Unity also.
Being tied to the language, and having no automatic versioning support