Chapter 2

Key Concepts

Mesh structure

How are meshes structured in the RMC? From the parts of a vertex, to the indices that turn them into triangles and the polygroups that batch them. Mesh Structure

Streams

Streams are the core data type of the RMC, containing one part of the mesh data. Learn more about streams!

Stream Builder

Stream builders allow for efficient, type safe access to a Stream. Learn more about stream builders!

Stream Linkage

Stream linkages are used to keep multiple streams in sync. Useful for things like position/tangents/texcoords/vertexcolor portions of a mesh Learn more about stream linkages!

Stream Set

Stream sets are the container for easily working with multiple Streams, and StreamLinkages. Learn more about stream sets!

Mesh Builder Local

Mesh Builder Local is a helper template for working with the most common mesh layout in an efficient, easy to use manner. Learn more about mesh builder local!

Subsections of Key Concepts

Meshes

Mesh Representation

Mesh data in the Realtime Mesh Comonent is represented as a indexed triangle list. With this you have two buffers:

  1. Vertex Buffer: Responsible for storing all the unique vertices. This is done ideally without duplicates, but also a single attribute being different causes the whole vertex to be different. The normal vertex attributes are:

    1. Position: Represents the position of the vertex in object space as a three component vector for X, Y, Z
    2. Normal: Represents the up-vector of the face this vertex is responsible for, or the common up-vector used for smooth shading all the adjoining triangle faces. Also known as Tangent-Z
    3. Tangent: Represents the forward-vector of the face this vertex is responsible for, or the common forward-vector for smooth shading all the adjoining triangle faces. Also known as Tangent-X
    4. UV Coorinates 1-8: Also konwn as Texture Coorinates, are the 2D coordinats in texture space 0-1 representing the area of a texture to apply to this face at this vertex. There can be any number of these channels between 1 and 8. Can also be used to feed arbitrary data through to the shader and sample in Material as TextureCoordinate.
    5. Color: Also known as Vertex Color is the color channel that can be used for supplying a color for this vertex, or possibly some other arbitrary data through to the material and read in the material as Vertex Color
  2. Index Buffer: Responsible for storing the mapping of vertices to triangles. Formed by a list of integers, with each group of 3 representing a triangle, and indexing the vertices to use for that triangles points.

The example below shows a simple setup where we render two triangles, using 6 entries in the index buffer, and only 4 vertices since 2 vertices are shared with two triangles each on the common edge. Index Triangle List Index Triangle List


Winding Order

One thing to be aware of and careful of is the order of your triangles indices. This affects one standard optimization of 3d rendering called backface culling. Unreal Engine uses Counter Clockwise Culling, so if the triangles points are visually ordered clockwise as referenced in the index list, it will not be rendered. This is a standard and important optimization as it removes many triangles that cannot be visible as they’re facing away from the view.

This means when you index your vertices, you should index the points visually counter clockwise.


Advanced Indexing

There are advanced things you can do with index buffers, in combinaton with the section groups/sections of the RMC, for example:

  • You could make one large index buffer and have a separate index buffer for shadows to simplify the geometry used. This can be extremely beneficial as shadows do not care about anything except position, so you could combine duplicates much more aggressively
  • You could make different versions of a sections that share a common set of vertices and switch between which one is rendered.

PolyGroups

Within the RMC, you have the option to use an extra data stream to define the polygroup. There should be 1 entry for each Triangle in the associated stream. This is used to easily separate the triangles into groups which can be rendered separately. The RMC assumes the polygroups are contiguous, so all triangles of a particular group are next to eachother. There are utilities to sort the triangles based on the polygroup if you want to build the triangles in an arbitrary order and sort them later.


Additional Resources

Streams

Streams are the core of the data structure for the RMC. They hold a single data type, like FVector3f or FColor, and they can handle 1-8 elements. They are meant to hold a single data component whether that’s positions, tangents, colors, texture coordinates, triangles, or something custom. You can work directly with a stream, but it’s generally not the best way. Usually you’ll want a FRealtimeMeshStreamSet


Stream Key

FRealtimeMeshStreamKey is a way of identifying the stream within things like a StreamSet, or to the RMC itself. It contains the StreamType, either Vertex or Index, as well as a stream name.

Some common stream keys

  • Vertex:Position - Contains the vertex position data
  • Vertex:Tangents - Contains the tangents, both TangentZ and TangentX, of the vertex data
  • Vertex:TexCoords - Contains the Texture Coordinates for 1-8 channels
  • Vertex::Colors - Contains the vertex color data
  • Index:Triangles - Contains the indexing data for a triangle list
  • Index:PolyGroups - Contains the poly group id for each triangle

Creating a Stream
// For example, this will create a stream with the key `Vertex:Position` and the datatype of FVector3f
auto PositionStream = FRealtimeMeshStream::Create<FVector3f>(FRealtimeMeshStreams::Position);

Bulk Adding Data
// If you have an array like the following (assuming it's filled with data)
TArray<FVector3f> IncomingData;

// You could add all of this at once through
PositionStream.Append(IncomingData);

Miscellaneous Helpers
// Fill the stream starting at position 10 and running for 20 elements with a default value
PositionStream.FillRange(10, 20, FVector3f(0, 0, 1));

// Zero a range of the stream starting at index 10 and running the next 20 elements
PositionStream.ZeroRange(10, 20);

// Preallocate the stream to hold 128 rows, this can cut down on reallocation and data copying.
PositionStream.Reserve(128);

Stream Builder

Stream Builder are a helper to allow for fast interaction with Streams of known or possibly unknown concrete data types. They allow you to treat a stream much like an array with all the common operations like Add/Remove/Append and indexed retrieval operators. It is possible to also use a streambuilder to work with a subset of a stream. For example if you wanted to work on only 1 channel of tex coords when it contains 4 total


Creating a Stream Accessor
// For example, this will create a stream with the key `Vertex:Position` and the datatype of FVector3f
auto PositionStream = FRealtimeMeshStream::Create<FVector3f>(FRealtimeMeshStreams::Position);

// This will create a simple StreamBuilder where you know the type is FVector3f. This will assert if the types do not match
// This does allow for the fastest interaction with the stream as no internal conversion is performed.
TRealtimeMeshStreamBuilder<FVector3f> PositionStreamBuilder(PositionStream);

// This will create a simple StreamBuilder where you want to work with it as if the data was FVector3d, but the actual streamdata is FVector3f.
// This will incur a slight overhead as it will do the conversion both ways internally, 
// but this is all done through templates, so it's the least overhead of the conversion available
TRealtimeMeshStreamBuilder<FVector3d, FVector3f> PositionStreamBuilder(PositionStream);

// This will create a StreamBuilder where you want to treat the data as a FVector3d, but you don't know the format of the stream. 
// This will perform a dynamic conversion internally, but the formats must be compatible. 
// In this case FVector3d, or FVector3f are safe, as well as custom data types of the same setup
TRealtimeMeshStreamBuilder<FVector3d, void> PositionStreamBuilder(PositionStream);

Creating a Strided Stream Builder
// This creates a texcoords stream with 4 channels using the packed type FVector2DHalf.
auto TexCoordsStream = FRealtimeMeshStream::Create<TRealtimeMeshTexCoords<FVector2DHalf, 4>>(FRealtimeMeshStreams::TexCoords);

// We can do the same stream builder setup as above to work with all 4 channels at the same time
TRealtimeMeshStreamBuilder<TRealtimeMeshTexCoords<FVector2DHalf, 4>> TexCoordStreamBuilder(TexCoordsStream);

// Here we use the strided builder to work with channel 1 as though it was an array of FVector2f and let the builder
// perform the conversion internally.  This can be done with with all the same variations as a normal builder for no-conversion, direct-conversion, or dynamic conversion.
TRealtimeMeshStridedStreamBuilder<FVector2f, FVector2DHalf> TexCoordStreamBuilder(TexCoordsStream, 1);

Working with Stream Builders
// Stream Builders let you treat a stream much like a TArray
// So you have all the normal functions, plus some specialized helpers.
// Some examples are: 

// Add a element
PositionStreamBuilder.Add(FVector3f(0, 0, 0));

// Emplace an element
TexCoordsStreamBuilder.Emplace({ FVector2f(0, 0), FVector2f(0, 0) });

// Reserve the number of elements, preallocating the internal storage
PositionStreamBuilder.Reserve(100);

// Append an initializer list of 3 elements
PositionStreamBuilder.Append({ FVector3f(0, 0, 0), FVector3f(1, 1, 1), FVector3f(2, 2, 2) });

// Remove element 1
PositionStreamBuilder.RemoveAt(1);

// Set the number of elements filling any new elements with the supplied value.
PositionStreamBuilder.SetNumWithFill(128, FVector3f(0, 0, 1));

Stream Linkage

Stream Linkages are used to tie multiple Streams together so that operations that affect the size of one stream affect them all together. This is useful to keep secondary vertex data streams the same size, and allow you to set data only as desired.


Creating a Stream Linkage
// Create our position stream
auto PositionStream = FRealtimeMeshStream::Create<FVector3f>(FRealtimeMeshStreams::Position);

// Create our tangents stream
auto TangentsStream = FRealtimeMeshStream::Create<FRealtimeMeshTangentsNormalPrecision>(FRealtimeMeshStreams::Tangents);

// Create our linkage that we'll use to bind the vertex streams together
FRealtimeMeshStreamLinkage VerticesLinkage;

// Bind the position stream, with a deafult value of zero vector.
// We can do this simply because we know the type
VerticesLinkage.BindStream(PositionStream, FRealtimeMeshStreamDefaultRowValue::Create(FVector3f::ZeroVector));

// Bind the tangents stream, we create the default value by passing it our wanted value, and getting the final layout from the stream.
// This lets it do the conversion once and then just blit this value into the stream.
VerticesLinkage.BindStream(TangentsStream, FRealtimeMeshStreamDefaultRowValue::Create(
    FRealtimeMeshTangentsNormalPrecision(FVector3f::ZAxisVector, FVector3f::XAxisVector),
    TangentsStream.GetLayout()));

// We can setup the two builders to make working with the streams easier    
TRealtimeMeshStreamBuilder<FVector3f> PositionStreamBuilder(PositionStream);
TRealtimeMeshStreamBuilder<FRealtimeMeshTangentsHighPrecision, FRealtimeMeshTangentsNormalPrecision> TangentsStreamBuilder(TangentsStream);

// This will add the position to the position stream. It will also set the tangents stream size to the same size and default the value
// You should *NOT* use .Add on the secondary streams when setting the value, you can just used the indexed set. 
// It doesn't matter which stream you call .Add() on, it will resize them all.
int32 Index = PositionStreamBuilder.Add(FVector3f(1, 0, 0));

// Here we just set the tangent entry for this index
// If you call .Add() again then you'll end up with 2 vertices in both streams, instead of 1 vertex with the tangent set for the existing vertex
TangentsStreamBuilder[Index] = FRealtimeMeshTangentsHighPrecision(FVector3f(0, 1, 0), FVector3f(0, 0, 1));

Stream Set

StreamSets are a container holding 1 or more Streams, and potentially some binding logic to tie streams together. At their core they function like a hashtable containing streams referenced by the StreamKey which is a name and buffer usage type (Vertex, Index). This is the most common way to pass data around, and can be used to store arbitrarily complex sets of data.


Creating a basic stream set
// First we create the empty stream set
FRealtimeMeshStreamSet StreamSet;

// Then we can add whatever streams to it we want
FRealtimeMeshStream& PositionStream = StreamSet.AddStream<FVector3f>(FRealtimeMeshStreams::Position);
FRealtimeMeshStream& TangentsStream = StreamSet.AddStream<FRealtimeMeshTangentsNormalPrecision>(FRealtimeMeshStreams::Tangents);

// We can choose to add the streams to a link pool if we want to
StreamSet.AddStreamToLinkPool("Vertices", FRealtimeMeshStreams::Position, FRealtimeMeshStreamDefaultRowValue::Create(FVector3f::ZeroVector));
StreamSet.AddStreamToLinkPool("Vertices", FRealtimeMeshStreams::Tangents, FRealtimeMeshStreamDefaultRowValue::Create(
    FRealtimeMeshTangentsNormalPrecision(FVector3f::ZAxisVector, FVector3f::XAxisVector),
    TangentsStream.GetLayout()));

// Then we can bind the stream builds to the contained streams like a individual stream
TRealtimeMeshStreamBuilder<FVector3f> PositionStreamBuilder(StreamSet.FindChecked(FRealtimeMeshStreams::Position));

Copying stream sets

Stream Sets by default don’t allow simple copy through assignment. This is because this can be a heavy operation to copy all the stream data. If you actually want to copy a stream you should use the explicit copy constructor

// Use the explicit copy constructor as implicit copy and assignment are disallowed for performance reasons.
FRealtimeMeshStreamSet StreamSet2(StreamSet);

You can move to stream set around from one storage to another. This is far more efficient as it doesn’t duplicate the internal data but instead moves it from one owner to the next. This will reset the source streamset in the process

// Move assignment is fast because it does not duplicate the data and instead moves the ownership of it.
FRealtimeMeshStreamSet StreamSet2 = MoveTemp(StreamSet);

Mesh Builder Local

Mesh Builder Local is a helper utility that makes building and working with the mesh data for the most common vertex format simpler. You can customize the exact data types and it will internally switch between the necessary conversion types just like TRealtimeMeshStreamBuilder.


Creating a basic mesh builder local
// First we create the empty stream set
FRealtimeMeshStreamSet StreamSet;

// Then we can bind the Mesh Builder Local to it.
// We can configure the stream data types through the template parameters
// We can also supply void to get dynamic conversion for unknown stream layouts.
TRealtimeMeshBuilderLocal<uint16, FPackedNormal, FVector2DHalf, 1> Builder(StreamSet);

// We can decide what vertex elements are enabled.
Builder.EnableTangents();
Builder.EnableTexCoords();
Builder.EnableColors();
Builder.EnablePolyGroups();

// We can add a vertex, and optionally set things like the tangents, color, texcoords.
// We can then get the new index from it to use later.
int32 V0 = Builder.AddVertex(FVector3f(-50.0f, 0.0f, 0.0f))
    .SetNormalAndTangent(FVector3f(0.0f, -1.0f, 1.0f), FVector3f(1.0f, 0.0f, 0.0f))
    .SetColor(FColor::Red)
    .SetTexCoord(FVector2f(0.0f, 0.0f));

// Now we can add a triangle giving it the indices of the vertices for the 3 corners, as well as optionally supplying the polygroup
Builder.AddTriangle(V0, V1, V2, 0);