World Interactive Plugin (Part 2)

In this section, we’ll cover the math behind packing and unpacking wind directions. We’ll also create a background component to replace the default black background in the render target.

First, let’s understand how the wind directions will be represented in our render targets. These correspond to the X and Y axes in Unreal Engine:

We want to visualize the X and Y wind direction values in our render target. These values range from -1 to 1, but since render targets can only store positive values, we need a way to remap this range.

To solve this, we can use the same approach employed in normal map packing. The values are mapped as follows:

-1 → 0 in the render target
0 → 0.5 in the render target
1 → 1 in the render target

This process is known as packing, and there is a formula for this.

Packed value = (Value + 1) / 2

Using this formula, we can convert values from the range [-1, 1] into [0, 1], making them compatible with the render target’s storage format.

This is an example of the material used on the Part 1 of this document. It renders the sphere normals, packed into our render target:

This is the result on the render target after drawing these components:

As you can see, there’s an issue: the black (empty) areas should map to [0.5, 0.5]. To address this, we need to add a background component.
There might be other ways to achieve this, but this is the approach I chose. I initially tried changing the render target’s clear color, but that had no effect.

The implementation is similar to the UWindMeshEmitter, but on BeginPlay, we call a method that checks whether the parent is a UPowerInteractionWindListener. If it is, we adjust the component’s size to match the render target.

void UWindBackgroundComponent::CheckParentAttachment()
{

  USceneComponent* Parent = GetAttachParent();

  if(Parent && !Parent->IsA<UPowerInteractionWindListener>())
  {
    UE_LOG(LogWindBackgroundComponent, Warning, TEXT("Wind Background Component %s is not attached to a Wind Listener"), *GetName());
  }
  else {

		ParentWindListener = Cast<UPowerInteractionWindListener>(Parent);

		//Add this component to the visible components on the listener
        ParentWindListener->ShowOnlyComponent(this);

		//Set the required size to fill all the listener view
        SetWorldScale3D(FVector(ParentWindListener->OrthoWidth / 100.0f));

  }
}

The mesh for this component is a plane with 100x100cm size, facing upwards. Now we can attach it to the world listener component on our pawn:

Next, we need to assign a simple material with an emissive color of [0.5, 0.5] to the component.
After applying it, the render target now looks like this:

At this stage, we could create an animated background material that creates wind waves, but for now, we will keep the setup simple.

After this step, I decided to move on to creating reactive grass. The goal is to develop a material that meets the following requirements:

Correctly maps the mesh position within the render target’s UV space.
Bends in the direction stored in the render target.

For the first step, I created a Material Function that takes a world position as a parameter, converts it into render target space, and returns both the raw render target color and the unpacked value.
The formula to unpack the render target value is:

Unpacked Value = (Value – 0.5) *2

To map world positions to the render target, we need two key pieces of information: the listener’s position and its width. (Note: the listener does not rotate I’ll explain the reason for that later). There are two possible approaches to provide these values:

(Discarded approach) Use dynamic material instances to pass the necessary values to each material. I ruled this out because creating a dynamic material every time a wind emitter component is instantiated could cause performance hitches, and each instance would require its own tick to update the listener values.
(Preferred approach) Use a Material Parameter Collection (MPC). This object allows us to define global values accessible by any material. With this method, we only need to update the values in the parameter collection once, and all materials will automatically reference the updated data.

So I created the parameter collection, and added these 2 vector values:

The next step is to reference this ParameterCollection on the WorldInteractionSubsystem and update it’s values on the tick method:

void UWorldInteractionSubsystem::UpdateWindParameters(float DeltaTime)
{

	if(ActiveWindListener.IsValid() && InteractionsParameterCollection.IsValid())
	{
		UMaterialParameterCollectionInstance* PCInstance = GetWorld()->GetParameterCollectionInstance(InteractionsParameterCollection.Get());

		if (PCInstance)
		{
			//Update the parameters in the material parameter collection based on the listener data
			ActiveListenerPosition = ActiveWindListener->GetComponentLocation();
			ListenerWidth = ActiveWindListener->OrthoWidth;
			PCInstance->SetVectorParameterValue(FName("WindListenerPosition"), FLinearColor(ActiveListenerPosition));
			PCInstance->SetVectorParameterValue(FName("WindListenerWorldSize"), FLinearColor(FVector(ListenerWidth)));
		}
	}
}

Now we are ready to make our material function:

The following code maps a position in world space to UV space within the render target texture.
We clamp the resulting values to ensure we only sample within the valid [0, 1] UV range.

Once we obtain the UV coordinates, we use them to read the render target and return both the raw color value and the unpacked value:

To test this Material Function, I created a material that draw the render target color directly on a mesh, using the pixel’s world location as input.

In this test material, the output of the function is simply connected to the Base Color:

Now we can apply the material to the ground mesh to see the results:

The test results were correct, so I proceeded to create the reactive grass material. The first step was populating the floor with grass. I used Unreal’s Foliage Tool, placing cylinders to represent individual grass blades:

The first step is to obtain the per-instance pivot point to read from the render target. Using the World Position node directly would result in each vertex being deformed differently, which would distort the mesh.

To calculate the pivot position for each foliage instance within the material, we follow these steps:

Next, we implement the rotation logic.

Since the grass mesh is small and simple, we can rotate it fully. For larger meshes, such as trees, a gradual rotation can be applied using a mask to bend the mesh from the base of the trunk to the top.

For now, we will rotate the entire grass mesh: