The material and shader generation system in Torque3D is a powerful, and flexible tool for providing artists with a set of features they can use to bring their creations to life. A system, such as this, is of limited use to a team if it can not be modified and extended easily. This document will describe how the Torque3D material code constructs HLSL and GLSL shaders, explain how to add additional material shader features, and provide the programmer with a simple, re-usable feature template.

This document assumes familiarity with shading languages, and 3D API concepts.

Overview of the Torque3D Material System

1
singleton Material(greyrock_material)
2
{
3
   mapTo = "greyrock";  
4
   baseTex[0] = "grayrock_diffuse.dds";
5
   bumpTex[0] = "grayrock_normal_specular.dds";
6
   pixelSpecular = true;
7
   specular = "1 1 1 0";
8
   specularPower = 12;
9
};

Materials encapsulate the textures, shaders, shader-constants, and render-states that are needed to draw geometry in Torque3D. Materials are defined in script files and have a wide variety of possible data fields, which tie directly to members of the C++ class Material (found in ‘Engine/source/materials/materialDefinition.h’). These values are then used to create a subclass of  ProcessedMaterial. This document will focus on ProcessedShaderMaterial (found in ‘Engine/source/materials/processedShaderMaterial.h’) but familiarity with the ProcessedMaterial base class is recommended.

When a ProcessedShaderMaterial is initialized, it will first get handles for the textures defined in the Material, by calling ProcessedMaterial::_setStageData(). This method associates the source textures with a stage, and a FeatureType. Feature types are prefixed with MFT_, for example MFT_DiffuseMap. These feature types can be found in ‘Engine/source/materials/materialFeatureTypes.h’. To declare a new feature type use the DeclareFeatureType() macro. Use the ImplementFeatureType() macro to specify how and in-what-order the feature should be used (this will be discussed in greater detail later).

Once textures have been loaded, the ProcessedShaderMaterial initialization code determines the material features which should be enabled for each stage, based on what texture handles have been successfully created, and the values on the members of the Material. This is first performed in ProcessedShaderMaterial::_determineFeatures() and is then optionally validated/modified by a delegate. ProcessedShaderMaterial::_createPasses() is then called to create the shaders and constant-buffer handles for each pass defined by the Material.

During shader creation is where the ShaderGen system comes into play. The command SHADERGEN->getShader() is invoked, with parameters specifying the features requested, and the format of the vertex stream. ShaderGen uses implementations of the ShaderFeature class to create HLSL and GLSL shaders (ShaderFeatureHLSL and ShaderFeatureGLSL). When ShaderGen is initialized, it will call all registered ShaderGenInitDelegate and allow them to associate a ShaderFeature with a FeatureType (see ‘Engine/source/shaderGen/hlsl/shaderGenHLSLInit.cpp’). Some of these features have shader code assocated with them, such as MFT_LightMap and its implementation, LightmapFeatHLSL. Some features do not have code associated, and are used as flags for changing the behavior of other feature implementations, such as, MFT_IsDXTnm which is a flag indicating that the normal map data needs to be expanded before use.

Adding a Simple Shader Feature

We just took a quick overview at how a material property goes from script to shader code. Now lets start with shader code, and work back to script by implementing a simple shader feature which multiplies the output of the material it is part of by a constant value.

First declare the feature type in ‘Engine/source/materials/materialFeatureTypes.h’

1
DeclareFeatureType( MFT_ConstantMult );

Next, the corresponding implementation in ‘Engine/source/materials/materialFeatureTypes.cpp’

1
ImplementFeatureType( MFT_ConstantMult , MFG_PostProcess, 999.0f, true );

This specifies that the FeatureType, MFT_ConstantMult, should execute in the MaterialFeatureGroup MFG_PostProcess, the last group to execute (see MaterialFeatureGroup struct) and it should run at order 999.0f in that group, effectively making it the last thing to process the fragment before it is returned by the pixel shader.

Now that we have an entry point into the material system which will allow us to run code where we want: right before output, we need to write that shader code. Create a new file, ‘Engine/source/shaderGen/constantMult.cpp’. Note that this document demonstrates adding a feature differently than is demonstrated by other features in the engine. This is done, not only to make the code more concise, but to demonstrate an additional, more encapsulated way to extend the functionality in the ShaderGen system.

We are going to skip the ShaderFeatureHLSL and ShaderFeatureGLSL classes and implement a general feature for both shading languages (it is recommended that you look over the existing shader features as reference material).

 C++ | 
 copy code |
?
 
01
#include "shaderGen/shaderFeature.h"
02
#include "shaderGen/shaderOp.h"
03
#include "shaderGen/featureMgr.h"
04
#include "materials/materialFeatureTypes.h"
05
#include "gfx/gfxDevice.h"
06
 
07
class ConstantMultFeature : public ShaderFeature
08
{
09
public:
10
   virtual void processPix( Vector<ShaderComponent*> &componentList, const MaterialFeatureData &fd )
11
   {
12
      // Find the constant value
13
      Var *multConst = (Var *)( LangElement::find("multConst") );
14
      if( multConst == NULL )
15
      {
16
         multConst = new Var;
17
         multConst->setType( GFX->getAdapterType() == OpenGL ? "vec4" : "float4" );
18
         multConst->setName( "multConst" );
19
         multConst->constSortPos = cspPotentialPrimitive;
20
         multConst->uniform = true;
21
      }
22
 
23
      // Find output fragment
24
      Var *color = (Var*) LangElement::find( getOutputTargetVarName(DefaultTarget) );
25
      if ( !color )
26
      {
27
         color = new Var;
28
         color->setType( GFX->getAdapterType() == OpenGL ? "vec4" : "fragout" );
29
         color->setName( getOutputTargetVarName(DefaultTarget) );
30
 
31
         if(GFX->getAdapterType() != OpenGL)
32
            color->setStructName( "OUT" );
33
 
34
         output = new GenOp( "@ = @", color, multConst );
35
      }
36
 
37
      output = new GenOp( "   @ *= @;\r\n", color, multConst );
38
   }
39
 
40
   virtual String getName() { return "ConstantMultFeature"; }
41
 
42
   // These methods aren't used
43
   virtual Var* getVertTexCoord( const String &name ) { return NULL; }
44
   virtual LangElement *setupTexSpaceMat(  Vector<ShaderComponent*> &componentList, Var **texSpaceMat ) { return NULL; }
45
   virtual LangElement *expandNormalMap( LangElement *sampleNormalOp, LangElement *normalDecl, LangElement *normalVar, const MaterialFeatureData &fd ) { return NULL; }
46
   virtual LangElement *assignColor( LangElement *elem, Material::BlendOp blend, LangElement *lerpElem = NULL, ShaderFeature::OutputTarget outputTarget = ShaderFeature::DefaultTarget ) { return NULL; }
47
};
48
 
49
 
50
class ConstMultFeatureReg
51
{
52
   bool _handleGFXEvent(GFXDevice::GFXDeviceEventType evt)
53
   {
54
      if( evt == GFXDevice::deInit )
55
      {
56
         FEATUREMGR->registerFeature( MFT_ConstantMult, new ConstantMultFeature );
57
      }
58
      else if( evt == GFXDevice::deDestroy )
59
      {
60
         FEATUREMGR->unregisterFeature( MFT_ConstantMult );
61
      }
62
 
63
      return true;
64
   }
65
 
66
public:
67
   ConstMultFeatureReg()
68
   {
69
      GFXDevice::getDeviceEventSignal().notify(this, &ConstMultFeatureReg::_handleGFXEvent, 999.0f);
70
   }
71
};
72
 
73
static ConstMultFeatureReg sConstMultFeatureReg;

This shader feature can be used for both GLSL and HLSL, since its function is so simple. The more complex shader features which exist in Torque3D use different implementations for OpenGL and Direct3D, which are based off of ShaderFeatureGLSL and ShaderFeatureHLSL. This method of implementing shader features is useful and you should be familiar with it even though this document does not discuss it.

The ConstantMultFeature::processPix() method uses a variable, called ‘multConst’. This is a value that we want to expose to the Material definition in script. We will get to that later. It takes this value and multiplies it by the output color.

Exposing Data to the Shader Feature

The next step in this process is to assign data the shader constant that the above feature uses, ‘multConst’. The central location for assignment of values to shader constants is located in ProcessedShaderMaterial::_setShaderConstants(). Add this code in that method.

 C++ | 
 copy code |
?
 
904
if( handles->mMultConstSC->isValid() )
905
      shaderConsts->set( handles->mMultConstSC, mMaterial->mMultConstant );

This checks to see if a shader constant handle called ‘mMultConstSC’ is valid. A handle will only be valid if the shader code uses that shader constant. If that handle is valid, it will assign the value specifed in the Material to the shader constant. Next, add that constant handle member to the ShaderConstHandles class in ‘Engine/source/materials/processedShaderMaterial.h’.

1
GFXShaderConstHandle* mMultConstSC;

Next, add the code which will assign a handle to that member in ShaderConstHandles::init().

1
mMultConstSC = shader->getShaderConstHandle("$multConst");

Note that the name must begin with the ‘$’ character, and that the name assigned in the previous code must match this name.

17
multConst->setType( GFX->getAdapterType() == OpenGL ? "vec4" : "float4" );
18
multConst->setName( "multConst" );

(From above code block, provided for reference)

Now this handle will get assigned a value if the shader which is being used requests the ‘multConst’ constant. This will be true if the ShaderFeature that we assigned to MFT_ConstantMult gets added during shader generation.  This will only happen if the MaterialFeatureData structure specifies that MFT_ConstantMult is enabled by the material, so let’s take a look at ProcessedShaderMaterial::_determineFeatures(), and add the following code.

 C++ | 
 copy code |
?
 
219
if ( mMaterial->mMultConstant.alpha > -1.0f )
220
   fd.features.addFeature( MFT_ConstantMult );

This will enable the MFT_ConstantMult feature on the generated shader if the value assigned to the constant has an Alpha value greater than -1.0. If it does, the shader code which we wrote in the ConstantMultFeature will get added to the generated shader. This will cause the ‘mMultConstSC’ handle that we added to be valid for Material definitions which specify a value for ‘multConst’. That member still does not exist yet, so let’s go to the Material class, and add that member variable.

1
ColorF mMultConstant;

And also initialization in the Material constructor.

1
mMultConstant.set(1.0f, 1.0f, 1.0f, -1.0f);

This initializes the value of Alpha to -1.0, which will cause the feature to be disabled by default. Next we need to allow this value to be adjusted from the script definition of a Material, so expose it as a field in Material::initPersistFields().

1
addField("multConstant", TypeColorF, Offset(mMultConstant, Material));

Wrapping it all Up

Robot turned red using the new shader feature

We have now worked backwards, from shader code, all the way back to script. To test this new material feature, adjust one of the Material definitions in your game project to look something like this:

01
singleton Material(skin)
02
{
03
   mapTo = "skin";
04
   baseTex[0] = "skin";
05
   bumpTex[0] = "skin_n";
06
   pixelSpecular[0] = true;
07
   specular[0] = "0.6 0.7 0.6 0.7";
08
   specularPower[0] = 32.0;
09
 
10
   // Add this to your material
11
   multConstant = "1.0 0.0 0.0 1.0"; 
12
};

You should now have a red version of the existing material. 

This document should provide you with a good overview of how to add a ShaderFeature to Torque3D. The concepts introduced here form the foundation of the lighting and material system for Torque. A topic as diverse as material functionality can’t possibly be covered in one document, or with example code alone. The material features included in Torque3D provide a wide range of functionality examples for creating more complicated features.