b.stefanuk.ca

Appearance

Using Flatbuffers ​

Note that the Flatbuffers tutorials are phenomenal. This guide doesn't aim to replace them, but to supplement.

Installing Flatbuffers (Linux) ​

sh
# Select a folder place the source code (e.g. ~/dev/third-party/)
mkdir -p ~/dev/third-party && cd ~/dev/third-party
# Clone the repository
git clone https://github.com/google/flatbuffers
cd flatbuffers
# Generate makefiles, build, and install
cmake -G "Unix Makefiles"
make
sudo make install
# Select a folder place the source code (e.g. ~/dev/third-party/)
mkdir -p ~/dev/third-party && cd ~/dev/third-party
# Clone the repository
git clone https://github.com/google/flatbuffers
cd flatbuffers
# Generate makefiles, build, and install
cmake -G "Unix Makefiles"
make
sudo make install

Schemas ​

The work with flatbuffers you need to define a schema. This file tells flatbuffers what each data structure consists of and how it's formatted. Schemas are written in the Interface Definition Language (IDL), which uses syntax similar to C style languages.

From the flatbuffer tutorial, here is an example schema:

c++
// Defines the namespace the generated code will be given
namespace MyGame.Sample;

// An enum that can be used in the schema is declared like this
enum Color:byte { Red = 0, Green, Blue = 2 }

// You can couple tables using the union keyword
union Equipment { Weapon, Backpack } // Optionally add more tables.

// Structs can also be used in the schema like this
struct Vec3 {
  x:float;
  y:float;
  z:float;
}

// This is the main object in the flatbuffer. It uses all
// the other components (enums, structs, tables) defined
// elsewhere in the schema
table Monster {
  pos:Vec3; // Struct.
  mana:short = 150;
  hp:short = 100;
  name:string;
  friendly:bool = false (deprecated);
  inventory:[ubyte];  // Vector of scalars.
  color:Color = Blue; // Enum.
  weapons:[Weapon];   // Vector of tables.
  equipped:Equipment; // Union.
  path:[Vec3];        // Vector of structs.
}

table Weapon {
  name:string;
  damage:short;
}

table Backpack {
    capacity:short;
}

// The root type declares the root table for the serialized data
root_type Monster;
// Defines the namespace the generated code will be given
namespace MyGame.Sample;

// An enum that can be used in the schema is declared like this
enum Color:byte { Red = 0, Green, Blue = 2 }

// You can couple tables using the union keyword
union Equipment { Weapon, Backpack } // Optionally add more tables.

// Structs can also be used in the schema like this
struct Vec3 {
  x:float;
  y:float;
  z:float;
}

// This is the main object in the flatbuffer. It uses all
// the other components (enums, structs, tables) defined
// elsewhere in the schema
table Monster {
  pos:Vec3; // Struct.
  mana:short = 150;
  hp:short = 100;
  name:string;
  friendly:bool = false (deprecated);
  inventory:[ubyte];  // Vector of scalars.
  color:Color = Blue; // Enum.
  weapons:[Weapon];   // Vector of tables.
  equipped:Equipment; // Union.
  path:[Vec3];        // Vector of structs.
}

table Weapon {
  name:string;
  damage:short;
}

table Backpack {
    capacity:short;
}

// The root type declares the root table for the serialized data
root_type Monster;

Compiling the Schema ​

Once you've defined your schema you have to compile with with flatc. If the above schema was called monster.fbs, this will generate a file called monster_generated.h (for C++) which should be included in whatever file you expect to conduct the serialization in.

Reading and Writing Flatbuffers (in process of deprecating) ​

First, create an instance of the FlatBufferBuilder which will contain the buffer as items are added to it.

C++
flatbuffers::FlatBufferBuilder builder(1024);
flatbuffers::FlatBufferBuilder builder(1024);

Using the builder you can use the CreateX shortcut (where X is the name of the table) to build whole tables quickly. Any objects that are contained within the root table have to be serialized before they're added.

Optionally, you can add each element to the flatbuffer manually by creating a builder object for the specific table and then adding elements one-by-one

C++
MonsterBuilder monster_builder(builder);
monster_builder.add_pos(&position);
monster_builder.add_hp(hp);
...
monster_builder.add_equipped_type(Equipment_Weapon);
monster_builder.add_equipped(axe.Union());
auto mon = monster_builder.Finish();
MonsterBuilder monster_builder(builder);
monster_builder.add_pos(&position);
monster_builder.add_hp(hp);
...
monster_builder.add_equipped_type(Equipment_Weapon);
monster_builder.add_equipped(axe.Union());
auto mon = monster_builder.Finish();

Writing Flatbuffers ​

  1. Create an instance of FlatBufferBuilder which will contain the buffer as items are added to it.
c++
// Create a flatbuffer builder
flatbuffers::FlatBufferBuidler fbb;
// Create a flatbuffer builder
flatbuffers::FlatBufferBuidler fbb;
  1. Using the builder you can use the CreateX shortcut to build whole tables quickly (where X is the name of the table). Any objects contained within the root table have to be serialized before they're added. Alternatively, you can add each element to the manually by creating a builder object for the specific table and then adding elements one-by-one.
c++
MonsterBuilder mon_builder(fbb);
mon_builder.add_pos(&position);
mon_builder.add_hp(hp);
...
auto monster = mon_builder.Finish(); // Apply to current offest
fbb.Finish(monster);                 // Apply the final offset
MonsterBuilder mon_builder(fbb);
mon_builder.add_pos(&position);
mon_builder.add_hp(hp);
...
auto monster = mon_builder.Finish(); // Apply to current offest
fbb.Finish(monster);                 // Apply the final offset

Writing Unions ​

Unions are useful for collection tables or structs that are logically associated, but may differ depending on the data being sent. To write a union, use either of methods in (2), and additionally specify the _type.

c++
mon_builder.add_equipped_type(Equipment_Weapon);
mon_builder.add_equipped(axe.Union()); // You must call .Union()
mon_builder.add_equipped_type(Equipment_Weapon);
mon_builder.add_equipped(axe.Union()); // You must call .Union()

Object API ​

To initialize the object API, add --gen-object-api as a flag to the Flatbuffer compiler, e.g., flatc. More information can be found in the FlatBuffers: Use in C++ guide.

Writing Flatbuffers with the Object API ​

c++
// Instantiate a struct with the xT type
::request::RequestT req_t;

// Populate the struct members directly
req_t.payload     = payload;
req_t.payload_len = req_payload_size;
req_t.command_id  = fb::camera::request::REQ_CMD_STREAM_START;

// Pack it with a builder
flatbuffers::FlatBufferBuilder fbbr;
fbbr.Finish(request::Request::Pack(fbbr, &req_t));

// Use the buffer pointer and size as usual
uint8_t *req_buf  = fbbr.GetBufferPointer();
int      req_size = fbbr.GetSize();
// Instantiate a struct with the xT type
::request::RequestT req_t;

// Populate the struct members directly
req_t.payload     = payload;
req_t.payload_len = req_payload_size;
req_t.command_id  = fb::camera::request::REQ_CMD_STREAM_START;

// Pack it with a builder
flatbuffers::FlatBufferBuilder fbbr;
fbbr.Finish(request::Request::Pack(fbbr, &req_t));

// Use the buffer pointer and size as usual
uint8_t *req_buf  = fbbr.GetBufferPointer();
int      req_size = fbbr.GetSize();

Reading Flatbuffers with the Object API ​

c++
// Instantiate a struct with the xT type
CameraRequestT cam_req;

// Unpack the data
GetCameraRequest(req.data())->UnPackTo(&cam_req);

// Access the members directly
cam_req.system_id
// Instantiate a struct with the xT type
CameraRequestT cam_req;

// Unpack the data
GetCameraRequest(req.data())->UnPackTo(&cam_req);

// Access the members directly
cam_req.system_id

Writing Unions with the Object API ​

c++
// This example uses a union of tables,
// first, create the table that goes in the union.
fb::camera::request::StreamOptionsT sopt_t;
sopt_t.stream_target = fb::camera::request::STREAM_TARGET_UDP;
sopt_t.udp_host      = "192.168.245.123";
sopt_t.udp_port      = 19200;
flatbuffers::FlatBufferBuilder      fbbs;
fbbs.Finish(fb::camera::request::StreamOptions::Pack(fbbs, &sopt_t));

// Next, create the union object by specifying the type,
// and using the .Set method on the union.
fb::camera::request::OptionsUnion   opt_union;
opt_union.type = fb::camera::request::Options_StreamOptions;
opt_union.Set(std::move(sopt_t));

// Add the union to the base table.
fb::camera::request::CameraRequestT cam_req_t;
cam_req_t.camera_name = camera_name;
cam_req_t.options     = opt_union;
flatbuffers::FlatBufferBuilder fbbt;
fbbt.Finish(fb::camera::request::CameraRequest::Pack(fbbt, &cam_req_t));
// This example uses a union of tables,
// first, create the table that goes in the union.
fb::camera::request::StreamOptionsT sopt_t;
sopt_t.stream_target = fb::camera::request::STREAM_TARGET_UDP;
sopt_t.udp_host      = "192.168.245.123";
sopt_t.udp_port      = 19200;
flatbuffers::FlatBufferBuilder      fbbs;
fbbs.Finish(fb::camera::request::StreamOptions::Pack(fbbs, &sopt_t));

// Next, create the union object by specifying the type,
// and using the .Set method on the union.
fb::camera::request::OptionsUnion   opt_union;
opt_union.type = fb::camera::request::Options_StreamOptions;
opt_union.Set(std::move(sopt_t));

// Add the union to the base table.
fb::camera::request::CameraRequestT cam_req_t;
cam_req_t.camera_name = camera_name;
cam_req_t.options     = opt_union;
flatbuffers::FlatBufferBuilder fbbt;
fbbt.Finish(fb::camera::request::CameraRequest::Pack(fbbt, &cam_req_t));

Reading Unions with the Object API ​

c++
// First, ensure that the type is what you intend
assert(cam_req.options.type == fb::camera::request::Options_StreamOptions);

// Next, cast the data to a variable
auto stream_opts = cam_req.options.AsStreamOptions();
// First, ensure that the type is what you intend
assert(cam_req.options.type == fb::camera::request::Options_StreamOptions);

// Next, cast the data to a variable
auto stream_opts = cam_req.options.AsStreamOptions();

Troubleshooting ​

A runtime errors similar to below

void flatbuffers::FlatBufferBuilder::NotNested(): Assertion `!nested' failed.
void flatbuffers::FlatBufferBuilder::NotNested(): Assertion `!nested' failed.

means that you are creating an offset item within the scope of the builder you're adding it too; flatbuffers doesn't allow this (for some reason). Move any calls to fbb.CreateString or similar outside of the inner builder, e.g., mon_builder or CreateX command.

ADDITIONAL RESOURCES ​

Gen-Object API