Working with HLSL in unreal

*The starting part of the post about Virtual Shader Source Path is no longer valid in UE 5+, the rest is valid except in SM6 (no global variables between custom nodes, there must be some workaround though).




The custom nodes are a good way to extent the functionality of the material editor in unreal. But if you want to add more than a couple lines of code it gets messy very fast.

So to solve this there is a way to use external files inside the node. So we just call the functions inside those files, making the HLSL edit, way more easy and structured. This files are the .usf or .ush files, in this case of using it with a custom node, you can use any of both (more info)

Today we are gonna make a simple game module to be able to access our .usf or .ush files from the custom node. In this case lets use .ush. This files will be stored in a folder called "Shaders" in the project root. (before the 4.21 version this was a feature out of the box, but since then It have to be added manually)

Adding the Shaders folder

Lets begin adding a directory by overriding the startup and shutdown module functions (this is a quick and fast way to add a very simple module). To do this lets just add in the "Project.h" the override and the functions to the "Project.cpp". I created a project named "CustomShader", this files are located in the solution " "Source>"Project" ", in my case "Source>CustomShader".


The .h file look like this:

// Copyright 1998-2019 Epic Games, Inc. All Rights Reserved.

#pragma once
#include "CoreMinimal.h"

class FShaderLabModule : public IModuleInterface
{
public:
 virtual void StartupModule() override;
 virtual void ShutdownModule() override;
};

And the  .cpp look like this:

// Copyright 1998-2019 Epic Games, Inc. All Rights Reserved.

#include "CustomShader.h"
#include "Modules/ModuleManager.h"
#include "Misc/Paths.h"


void FShaderLabModule::StartupModule()
{
 FString ShaderDirectory = FPaths::Combine

(FPaths::ProjectDir(), TEXT("Shaders"));
 AddShaderSourceDirectoryMapping("/Project", ShaderDirectory);
}

void FShaderLabModule::ShutdownModule()
{
 ResetAllShaderSourceDirectoryMappings();
}

IMPLEMENT_PRIMARY_GAME_MODULE(FShaderLabModule, CustomShader, "CustomShader");

Finally lets add the RenderCore as a dependency to out project in the "Project.Build.cs"(CustomShader.Build.cs in my case) by adding "RenderCore".

// Copyright 1998-2019 Epic Games, Inc. All Rights Reserved.

using UnrealBuildTool;

public class CustomShader : ModuleRules
{

 public CustomShader(ReadOnlyTargetRules Target) : base(Target)
 {
  PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
 
  PublicDependencyModuleNames.AddRange(new string[] 
  { "Core", "RenderCore", "CoreUObject", "Engine", "InputCore" });

  PrivateDependencyModuleNames.AddRange(new string[] {  });
 }

}

To finish lets create a folder called "Shaders" inside the root directory of the project. Dont forget to compile!

This should be enough to make unreal know about the Shader folder. So lets try how .ush/.usf files work inside custom nodes.

Creating the HLSL file

Lets create a new .txt file in our new "Shaders" folder renaming it to to "myCustom.ush" (of course the name can be whatever you want). In my case I will work with notepad++ to edit the file, but you can use any text editor you like.


Lets keep it simple, in this case I made a multiplication function. The function and will be inside a struct wich we will use inside the custom node. Also use the #pragma once to avoid multiple includes in the same file.

#pragma once
struct CustomF 
{ 
 float A;
 float B;

 float Mult( float a, float b){
 return a*b;
 }
}

Using the HLSL file

Lets create a new unlit material and a custom node plugged into emissive so we see cleary what we are doing.

Now we only need to use it inside the custom node. To access it we just need to include the .ush/.usf in the shader and make a struct instance to use the variables and functions. Remember that you can use external variables by creating them in the external file and having the same name in the input of the custom node (like the A and B variables of this example).

You can also create your custom global material variables and pass them between nodes. This way you are no longer limited to a float4 output when working with custom nodes!
By default the custom node code is translated by the compiler as a function. Since the compiler inserts braces automatically, you can write code outside the custom node function by closing the function and reopening it. In this example we create a global struct Func and modify it in other custom nodes (remember to add the "return 0;" just to remember the compiler that it still a function!).

*If you are using UE 5+, global structs are const by default so you will have to create a variable inside the function you will use them in case you want to change struct values.

Custom node A:

    return 0;
 }

#pragma once
struct Func 
{    
    float b;

    void F(){
    b = b/2.f;
    }
};

Func a;

void DummyFunc()
 {


//--------------------------------


//SM6
    return 0;
 }

#pragma once
struct Func 
{    
    float b; 

    void F(){
    b = b/2.f;
    }
};

void DummyFunc()
 {

Custom node B:

#pragma once

a.b = 500.f;
a.F();
return 0;

//--------------------------------

//SM6
#pragma once

Func a; 
a.b = 500.f;
a.F();
return 0;

Custom node C

//Not valid in SM6 (can only read local variables)

#pragma once
return a.b;

And here is the output:

If you don't know how to get a parameter in code, you can get code of any material in Window>ShaderCode>HLSL Code. This is the main way of getting all the code, but you can also check MaterialTemplate.h (Engine/Source/Shaders/Private/MaterialTemplate.ush) and the API documentation (despite being very lacking it can help). More info on custom node tricks here.

This is the base to start working on more complicated shaders without having to modify engine files. As we will see in future posts, this opens the gates for a lot of cool and easy to make materials. Hope you find It useful and see you in the next post!