esp::gfx_batch::Renderer class
#include <esp/gfx_batch/Renderer.h>

Batch renderer.

A renderer optimized for drawing multiple scenes at once, applying multi-draw optimizations where possible to reduce draw overhead. While all scenes are rendered at once, each scene contains an independent set of rendered objects.

This class expects an active Magnum::GL::Context and renders into a provided framebuffer, making it suitable for use in GUI applications. For standalone operation use the RendererStandalone subclass instead, which manages the OpenGL context and a framebuffer on its own. Apart from the context and framebuffer, the high-level usage is same for both.

Usage

The renderer gets constructed using a RendererConfiguration with desired tile size and count set via RendererConfiguration::setTileSizeCount() as described in its documentation. Each tile corresponds to one rendered scene, organized in a grid, all scenes have the same rendered size.

First, files with meshes, materials and textures that are meant to be rendered from should get added via addFile(). The file itself isn't directly rendered, instead it's treated as a composite file with named root scene nodes being node hierarchy templates to be selectively added to particular scenes. Apart from picking particular root nodes, it's also possible to treat the whole file as a single hierarchy using RendererFileFlag::Whole. See the flag documentation for more information.

Then, particular scenes are populated using addNodeHierarchy(), referencing the node hierarchy templates via their names. The function takes an initial transformation and returns an ID of the node added into the scene. The ID can then be used to subsequently update its transformation via transformations(), for example in respose to a physics simulation or an animation. Each scene also has an associated camera and its combined projection and transformation matrix can be updated using updateCamera().

Finally, the draw() function renders the grid of scenes into a provided framebuffer.

Inner workflow of the batch renderer

The goal of the renderer is to do as much as possible using the least amount of draw calls. For that, it relies heavily on multi-draw and texture array support implemented in Magnum shaders.

At the beginning, all data from files added by addFile() get uploaded to the GPU. The data consists of meshes, texture image levels and packed material uniforms. Such data are uploaded only once and treated as immutable for the rest of the renderer lifetime.
On each addNodeHierarchy() call, a list of mesh views each together with material association, texture layer and transform and node assignment is added to a draw list. The draw list is further detailed below.
For each draw() and each non-empty scene, the following is done:
- The transformation passed to updateCamera() is uploaded to a uniform buffer.
- The renderer calculates hierarchical transformations for all nodes based on the matrices supplied via transformations(). Each item in the draw list is then assigned a corresponding calculated absolute transformation, and the list of transformations corresponding to all draws is uploaded to a uniform buffer.
- Then the draw list is processed, resulting in one or more multi-draw calls as described below.

Draw list and multi-draw call submission

In the ideal (and often impossible) case, there would be just a single file added with addFile(), containing exactly one mesh and exactly one texture array, with scene nodes referring to sub-views of them — i.e., mesh index ranges, texture layer indices and texture transformation matrices. Then, no matter how many times addNodeHierarchy() gets called, the whole draw list populated by it can be drawn with a single multi-draw call.

In practice however, the files may contain meshes with different vertex layouts (such as some having vertex colors and some not), textures of different formats and sizes, and different materials requiring different shaders (such as vertex-colored, alpha-masked etc.). Draw lists populated with addNodeHierarchy() are then partitioned into draw batches, where a particular draw batch contains all draws with a particular shader, from a particular mesh and with a particular texture. Each draw batch then corresponds to a single multi-draw call issued on the GPU.

Performance tradeoffs

The optimization goal here is to pick a balance between minimizing driver overhead from submitting many draw calls and paying extra cost for shader features, texture fetches and vertex attributes present also for draws that don't need them.

For example, if only some meshes use vertex colors, it's possible to add white vertex colors to the remaining meshes, unifying their layout and making it possible to draw them together in a single draw call. The cost of extra vertex processing bandwidth would probably be rather minimal. On the other hand, if some materials need alpha masking and some not, attempting to draw everything with alpha masking enabled may have a much larger cost than the additional draw call saved by this change.

Here it's important to also take into account differences on the web — there the driver overhead is significantly higher and it might be beneficial to put extra effort into reducing draw batch count even though it may result in slightly worse performance natively.

Creating batch-optimized files

While addFile() can consume any count of any regular scene/model files supported by Magnum, in order to properly take advantage of all performance features it's needed to combine the files into batch-optimized composite files containing mesh views and texture arrays.

Batch-optimized files can be stored in any file format that has sufficient flexibility for Magnum::Trade::MeshData layouts, is capable of storing 2D array textures and supports storing custom Magnum::Trade::SceneData fields. At the moment, a file format supporting all this is glTF, with files produced using the GltfSceneConverter plugin.

Meshes and mesh views

Composite meshes, which concatenate several meshes of the same layout together, can be added with the general Magnum::Trade::AbstractSceneConverter::add(const MeshData&, Containers::StringView) API and referenced from scene nodes via Magnum::Trade::SceneField::Mesh. Mesh views, referencing sub-ranges of the composite mesh, aren't a builtin Magnum::Trade::SceneData feature at the moment however, and thus have to be added as custom scene fields of the following names:

meshViewIndexOffset of type Magnum::Trade::SceneFieldType::UnsignedInt, containing offset of the mesh view in the index buffer in bytes,
meshViewIndexCount of type Magnum::Trade::SceneFieldType::UnsignedInt, containing index count, and
meshViewMaterial of type Magnum::Trade::SceneFieldType::Int containing the corresponding material ID. The builtin Magnum::Trade::SceneField::MeshMaterial is not used, because glTF bakes the material reference into the mesh itself, which means referencing the same mesh multiple times with different materials would cause it to be unnecessarily duplicated in the glTF.

The custom field IDs can be arbitrary, what's important is that they are associated with corresponding names using Magnum::Trade::AbstractSceneConverter::setSceneFieldName(). Furthermore, to prevent the file from being opened by unsuspecting glTF viewers, a private MAGNUMX_mesh_views extension should be added as both extensionUsed and extensionRequired in the plugin configuration. Inspecting the resulting file with magnum-sceneconverter may look similarly to this:

magnum-sceneconverter --info-scenes -i ignoreRequiredExtensions batch.gltf

Scene 0:
  Bound: 11 objects @ UnsignedInt (0.7 kB)
  Fields:
    Parent @ Int, 11 entries
    ImporterState @ Pointer, 11 entries
    Transformation @ Matrix4x4, 4 entries
    Mesh @ UnsignedInt, 7 entries
    Custom(0:meshViewIndexOffset) @ Float, 7 entries
    Custom(1:meshViewIndexCount) @ Float, 7 entries
    Custom(2:meshViewMaterial) @ Float, 7 entries

Texture arrays

For 2D texture arrays the KHR_texture_ktx extension is used. Because it's not approved by Khronos yet, it needs the experimentalKhrTextureKtx configuration option enabled in the converter plugin. Materials should reference layers of it via Magnum::Trade::MaterialAttribute::BaseColorTextureLayer and related attributes, together with supplying BaseColorTextureMatrix describing a sub-image of a particular layer. Inspecting the resulting file with magnum-sceneconverter may look similarly to this:

magnum-sceneconverter --info-textures --info-images --info-materials \
  -i experimentalKhrTextureKtx,ignoreRequiredExtensions batch.gltf

…
Material 3: yellow
  Type: PbrMetallicRoughness
  Base layer:
    BaseColor @ Vector4: ██ {1, 1, 0, 1}
    BaseColorTexture @ UnsignedInt: 0
    BaseColorTextureLayer @ UnsignedInt: 1
    BaseColorTextureMatrix @ Matrix3x3: {0.5, 0, 0,
 0, 0.5, 0.5,
 0, 0, 1}
Texture 0 (referenced by 4 material attributes):
  Type: Texture2DArray, image 0
  Minification, mipmap and magnification: Nearest, Nearest, Nearest
  Wrapping: {Repeat, Repeat, Repeat}
3D image 0 (referenced by 1 textures):
  Level 0: Array {4, 4, 2} @ RGB8Unorm (0.1 kB)

Derived classes

class RendererStandalone: Standalone batch renderer.

Constructors, destructors, conversion operators

Renderer(const RendererConfiguration& configuration) explicit: Constructor.
~Renderer() virtual

Public functions

auto flags() const -> RendererFlags: Global renderer flags.
auto tileSize() const -> Magnum::Vector2i: Tile size.
auto tileCount() const -> Magnum::Vector2i: Tile count.
auto sceneCount() const -> std::size_t: Scene count.
auto maxLightCount() const -> Magnum::UnsignedInt: Max light count.
auto addFile(Corrade::Containers::StringView filename, RendererFileFlags flags = {}, Corrade::Containers::StringView name = {}) -> bool: Add a file.
auto addFile(Corrade::Containers::StringView filename, Corrade::Containers::StringView importerPlugin, RendererFileFlags flags = {}, Corrade::Containers::StringView name = {}) -> bool: Add a file using a custom importer plugin.
auto hasNodeHierarchy(Corrade::Containers::StringView name) const -> bool: If given mesh hierarchy exists.
auto addNodeHierarchy(Magnum::UnsignedInt sceneId, Corrade::Containers::StringView name, const Magnum::Matrix4& bakeTransformation = {}) -> std::size_t: Add a mesh hierarchy.
auto addEmptyNode(Magnum::UnsignedInt sceneId) -> std::size_t
auto addLight(Magnum::UnsignedInt sceneId, std::size_t nodeId, RendererLightType type) -> std::size_t: Add a light.
void clear(Magnum::UnsignedInt sceneId): Clear a scene.
void clearLights(Magnum::UnsignedInt sceneId): Clear lights in a scene.
auto camera(Magnum::UnsignedInt sceneId) const -> Magnum::Matrix4: Get the combined projection and view matrices of a camera (read-only)
auto cameraDepthUnprojection(Magnum::UnsignedInt sceneId) const -> Magnum::Vector2: Get the depth unprojection parameters of a camera (read-only)
void updateCamera(Magnum::UnsignedInt sceneId, const Magnum::Matrix4& projection, const Magnum::Matrix4& view): Set the camera projection and view matrices.
auto transformations(Magnum::UnsignedInt sceneId) -> Corrade::Containers::StridedArrayView1D<Magnum::Matrix4>: Transformations of all nodes in the scene.
auto lightColors(Magnum::UnsignedInt sceneId) -> Corrade::Containers::StridedArrayView1D<Magnum::Color3>: Colors of all lights in the scene.
auto lightRanges(Magnum::UnsignedInt sceneId) -> Corrade::Containers::StridedArrayView1D<Magnum::Float>: Ranges of all lights in the scene.
void draw(Magnum::GL::AbstractFramebuffer& framebuffer): Draw all scenes into provided framebuffer.
auto sceneStats(Magnum::UnsignedInt sceneId) const -> SceneStats: Scene stats.

Function documentation

esp::gfx_batch::Renderer::Renderer(const RendererConfiguration& configuration) explicit

Constructor.

Parameters
configuration	Renderer configuration

Expects an active Magnum::GL::Context to be present.

RendererFlags esp::gfx_batch::Renderer::flags() const

Global renderer flags.

By default, no flags are set.

Magnum::Vector2i esp::gfx_batch::Renderer::tileSize() const

Tile size.

The default tile size is {128, 128}.

Magnum::Vector2i esp::gfx_batch::Renderer::tileCount() const

Tile count.

By default there's a single tile.

std::size_t esp::gfx_batch::Renderer::sceneCount() const

Scene count.

Same as the product() of tileCount(). Empty scenes are not rendered, they only occupy space in the output framebuffer.

Magnum::UnsignedInt esp::gfx_batch::Renderer::maxLightCount() const

Max light count.

By default there's zero lights, i.e. flat-shaded rendering.

bool esp::gfx_batch::Renderer::addFile(Corrade::Containers::StringView filename, RendererFileFlags flags = {}, Corrade::Containers::StringView name = {})

Add a file.

Parameters
filename	File to add
flags	File flags
name	Hierarchy name. Ignored if RendererFileFlag::Whole isn't set.

By default a file is treated as a so-called composite — a collection of node hierarchy templates, which you then add with Renderer::addNodeHierarchy() using a name corresponding to one of the root nodes. This means all root nodes have to be named and have their names unique.

If RendererFileFlag::Whole is set, the whole file is treated as a single mesh hierarchy template instead, named name, or if name is empty with the filename used as a name.

Returns true on success. Prints a message to Magnum::Error and returns false if the file can't be imported or the names are conflicting.

bool esp::gfx_batch::Renderer::addFile(Corrade::Containers::StringView filename, Corrade::Containers::StringView importerPlugin, RendererFileFlags flags = {}, Corrade::Containers::StringView name = {})

Add a file using a custom importer plugin.

Parameters
filename	File to add
importerPlugin	Importer plugin name or path
flags	File flags
name	Hierarchy name. Ignored if RendererFileFlag::Whole isn't set.

See addFile(Corrade::Containers::StringView, RendererFileFlags, Corrade::Containers::StringView) for more information.

bool esp::gfx_batch::Renderer::hasNodeHierarchy(Corrade::Containers::StringView name) const

If given mesh hierarchy exists.

Returns true if name is a mesh hierarchy name added by any previous addFile() call, false otherwise.

std::size_t esp::gfx_batch::Renderer::addNodeHierarchy(Magnum::UnsignedInt sceneId, Corrade::Containers::StringView name, const Magnum::Matrix4& bakeTransformation = {})

Add a mesh hierarchy.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()
name	Node hierarchy template name, added with addFile() earlier
bakeTransformation	Transformation to bake into the hierarchy
Returns	ID of the newly added node

The returned ID can be subsequently used to update transformations via transformations(). The returned IDs are not contiguous, the gaps correspond to number of child nodes in the hierarchy.

The bakeTransformation is baked into the added hierarchy, i.e. transformations() at the returned ID is kept as an identity transform and writing to it will not overwrite the baked transformation. This parameter is useful for correcting orientation/scale of the imported mesh.

std::size_t esp::gfx_batch::Renderer::addLight(Magnum::UnsignedInt sceneId, std::size_t nodeId, RendererLightType type)

Add a light.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()
nodeId	Node ID to inherit transformation from, returned from addNodeHierarchy() or addEmptyNode() earlier
type	Light type. Can't be changed after adding the light.

Expects that maxLightCount() isn't 0. If type is RendererLightType::Directional, a negative Magnum::Matrix4::backwards() is taken from nodeId transformation as the direction, if type is RendererLightType::Point, Magnum::Matrix4::translation() is taken from nodeId transformation as the position. The ID returned from this function can be subsequently used to update light properties via lightColors() and lightRanges().

void esp::gfx_batch::Renderer::clear(Magnum::UnsignedInt sceneId)

Clear a scene.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

Clears everything added by addNodeHierarchy(), addEmptyNode() and addLight().

void esp::gfx_batch::Renderer::clearLights(Magnum::UnsignedInt sceneId)

Clear lights in a scene.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

Clears everything added by addLight().

Magnum::Matrix4 esp::gfx_batch::Renderer::camera(Magnum::UnsignedInt sceneId) const

Get the combined projection and view matrices of a camera (read-only)

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

Magnum::Vector2 esp::gfx_batch::Renderer::cameraDepthUnprojection(Magnum::UnsignedInt sceneId) const

Get the depth unprojection parameters of a camera (read-only)

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

void esp::gfx_batch::Renderer::updateCamera(Magnum::UnsignedInt sceneId, const Magnum::Matrix4& projection, const Magnum::Matrix4& view)

Set the camera projection and view matrices.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()
projection	Projection matrix of the camera
view	View matrix of the camera (inverse transform)

Also computes the camera unprojection. Modifications to the transformation are taken into account in the next draw().

Corrade::Containers::StridedArrayView1D<Magnum::Matrix4> esp::gfx_batch::Renderer::transformations(Magnum::UnsignedInt sceneId)

Transformations of all nodes in the scene.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

Returns a view on all node transformations in given scene. Desired usage is to update only transformations at indices returned by addNodeHierarchy() and addEmptyNode(), updating transformations at other indices is possible but could have unintended consequences. Modifications to the transformations are taken into account in the next draw(). By default, transformations of root nodes (those which IDs were returned from addNodeHierarchy() or addEmptyNode()) are identities, transformations of other nodes are unspecified.

Corrade::Containers::StridedArrayView1D<Magnum::Color3> esp::gfx_batch::Renderer::lightColors(Magnum::UnsignedInt sceneId)

Colors of all lights in the scene.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

Returns a view on colors of all lights in given scene. Desired usage is to update only colors at indices returned by addLight(), updating colors at other indices is possible but could have unintended consequences. Modifications to the lights are taken into account in the next draw(). By default, colors of root lights (those which IDs were returned from addLight()) are 0xffffff_rgbf, colors of other lights are unspecified.

Corrade::Containers::StridedArrayView1D<Magnum::Float> esp::gfx_batch::Renderer::lightRanges(Magnum::UnsignedInt sceneId)

Ranges of all lights in the scene.

Parameters
sceneId	Scene ID, expected to be less than sceneCount()

Returns a view on ranges of all lights in given scene. Desired usage is to update only ranges at indices returned by addLight(), updating ranges at other indices is possible but could have unintended consequences. Modifications to the lights are taken into account in the next draw(). The value range is ignored for RendererLightType::Directional. By default, ranges of root lights (those which IDs were returned from addLight()) are Magnum::Constants::inf(), ranges of other lights are unspecified.

void esp::gfx_batch::Renderer::draw(Magnum::GL::AbstractFramebuffer& framebuffer)

Draw all scenes into provided framebuffer.

The framebuffer is expected to have a size at least as larger as the product of tileSize() and tileCount().

SceneStats esp::gfx_batch::Renderer::sceneStats(Magnum::UnsignedInt sceneId) const

Scene stats.

Mainly for testing and introspection purposes. The returned info is up-to-date only if draw() has been called before.