The Vertex Paint Swaying System is a feature in the WayForward Engine that brings environments to life by procedurally manipulating static mesh geometry to create ambient vertex-based animation. The swaying mesh shader uses artist-applied vertex color information to translate vertices back and forth to make plants and trees look like they’re swaying in the wind. While the system was already in place, I upgraded it with several new features. More importantly, I created a maxscript to dramatically improve the artists’ workflow and results with vertex painting meshes for the swaying system.
I joined Shantae: Half Genie Hero’s production after the Kickstarter, when the team was reestablished. In the first few weeks, I was asked to analyze the Kickstarter pipeline and tools to learn how to use them and see where I can improve. When I was shown the Vertex-based swaying system, I noticed that creating proper swaying meshes involved a lot of trial and error. When redundant, repetitive tasks are involved in a process, it usually means that Tech Art can help out!
The Vertex Paint Swaying System was created after DuckTales to liven up static environments without resorting to making every static mesh become rigged and animated with bones. The system was mostly intended to simulate environments being affected by calm and heavy wind. Meshes are “animated” by displacing their vertices back and forth to simulate swaying. So, vertex color data needs to be smartly painted onto the mesh surfaces to give convincing results. That’s where I saw my opportunity to optimize this pipeline!
THE ORIGINAL PROCESS
I wrote the Offset to Paint maxscript to help automatically generate vertex paint on meshes for the Swaying System. Before my maxscript was created, all swaying meshes were vertex painted by hand. The biggest problem was that artists had to estimate how their vertex-painted meshes would look like when affected by wind in-game. Furthermore, they were only using one channel of color ( meaning only one direction ). After they painted the mesh, the artists would export it, test in-game, and repaint until the swaying mesh looked good enough in game.
HOW THE SCRIPT WORKS
My goal was to create a maxscript that would make the artists’ job both easier and improve the quality of the sway meshes. Rather than hand-painting vertices, artists duplicate the static mesh and manipulate the actual shape of the mesh itself to create essentially a blend shape for the original mesh to turn into. My script then calculates the vertex paint based on the offset difference between the original mesh and duplicated mesh vertices. The rgb channels are 1:1 with xyz coordinates.
The script loops through each vertex shared by the original mesh and duplicate mesh. The distance between them is translated into a color value.
// Offset Vertex
offset.x = abs( duplicate.x – original.x )
offset.y = abs( duplicate.y – original.y )
offset.z = abs( duplicate.z – original.z )
While vertex position coordinates can hold very large floating point numbers, rgb color values are 8-bit, and only have a range between 0-255. The WF Engine allows programmers to add data type properties to static meshes so that they can hold info such as for collisions. So, to solve the problem, I added a float3 property called influence to the sway meshes . Its a scalar value that is calculated for each xyz direction based on the max offset distance. By dividing each vertex offset by the max distance, all the vertices are normalized to range larger than 8-bit with max distance representing 255 for each color.
// Normalize from color data to distance
color.x = 255 * ( offset.x/maxDist.x )
color.y = 255 * ( offset.y/maxDist.y )
color.z = 255 * ( offset.z/maxDist.z )
There are two major limitations to how artists can manipulate vertices in the duplicated mesh. First, vertex paint only records offset positioning. So, while you can rotate and scale the vertices to offset their positions, the results in-game might not be 1:1 because of the second reason. The 0 to 255 space does not hold any sign value. The influence property mentioned above is what dictates whether the rgb values are positive or negative. This means that if one vertex is offset downward, than all must be offset downward as well. There is a possible solution that I haven’t tried yet, which is to split color space into two. 0-127 would be negative and 128-255 would be positive. However, that might cause issues with the influence property precision since there’s be half the number space to work with. I’d love to try that out sometime. The WindHelper already offsets vertices negatively or positively, depending on the wind direction anyway. However, sometimes moving vertices in the opposite direction will break the mesh ( vertices clipping through faces etc ). So, in addition to the influence property, I also added the locked boolean3 property. If true, then the vertices will always move in the original direction for that dimension.
EXTRACTING BAKED DATA
For a few months after I created the maxscript, if the artist wanted to revisit a sway mesh, but did not create a save file with the duplicated mesh, then they would have to start over. That’s why I created the Offset Vertex Color maxscript. It would essentially reverse engineer the duplicated mesh by using the vertex colors on the original mesh plus the influence property to offset all the vertices to how they would appear in-game. The artist could then further modify the reduplicated mesh and run the Offset to Paint maxscript again.
WINDHELPER AND LUA
In the WF Engine, meshes are caused to sway when the vertex shader detects that meshes have been tagged and contain vertex color data. The WindHelper class was created to drive the movement of the swaying static meshes. User-driven variables control how the WindHelper acts; such as wind speed and fluctuation intervals. Lastly, constancy and viscosity are two variables that control how the mesh vertices interpolate between new wind values to make the meshes look heavier ( like a tree trunk ) or springier ( like a palm branch ). Each level’s WindHelpers act as scalars for the vertex shader to dictate how much of vertex color data to apply to the mesh.
The artists control the WindHelper variables via a LUA script placed in the level editor. In addition, artists can tag meshes into different groups. Each tagged group has its own WindHelper instance so that an environment can have meshes affected by different wind values ( for example, ground plants vs underwater plants ).
The vertex paint is essentially dictating a new blend shape for the meshes. The wind essentially interpolates between the original and new shape ( 100% wind means maximum intended position of vertices to make new shape ). As mentioned in the previous section, I created the maxscripts to automate vertex painting and allow artists to create any mesh sway shape they want. I also expanded the vertex shader and WindHelper class to further improve the Swaying System.
THE VERTEX SHADER
The swaying mesh vertex shader relies on the WindHelper, mesh properties, and vertex colors
to displace each vertex to make the mesh look as if it’s swaying. Originally, the vertex shader only used the red channel, meaning the vertices could only sway in one direction. I extended it so that all three rgb values are used and therefore wind can modify meshes in all three dimensions. Because of my Offset to Paint maxscript, the Sway System became more about swaying between two shapes, instead of slightly displacing vertices.
WIND HELPER VARIABLES
The WindHelper has two states; calm and gust wind. Based on a user-driven timer, the WindHelper switches between these two states to help make the wind feel more natural and random. The two wind states have their own variable values that the artist set in the LUA script. Originally, each tagged group had its own Calm Wind variables, but the Gust Wind was global so that the gust would affect every tagged group simultaneously.
First, I changed it so that Gust Wind is local to each tagged group just like Calm Wind. The Global Wind variables are simply the wait intervals and duration of when it switches between Gust and Calm state. Second, I split the range and interval variables so that the minimum and maximum can be assigned unique values rather than simply a sign change from negative to positive. Third, I added a threshold variable that would ensure the random values would never be too low of a value. Lastly, I cleaned up the code and added more comments to make it clear for users what each value did. When I first opened the LUA file, I had to deduce several properties, especially the constancy and viscosity. I feel that cleaning up the code was a good call, especially for when new users are introduced to the swaying system.
Each tagged group has its own WindHelper instance so meshes in separate groups do not sway at the same rate. I added Positional Delay so that meshes can subtly be affected later by the new “air current”. This slight delay adds a little realism; the wind has a physical presence. I brainstormed if there was a way to accomplish this without needing to store previous WindHelper data.
However, to get the best results, we created a 128-sized circular array that stores previous WindHelper data and each sway mesh samples from the array based on its position. 128 cells gives us more than enough information to fit screen space. However, if the camera moves too fast, there is a chance that you can catch the “reset” of the sway mesh.
The last new feature that I added was requested by the Assistant Director of the project. He wanted each mesh to also be rotated based on the strength of the wind.
So, I added a rotation vector to the sway mesh properties. It was just a basic rotation on the pivot of the mesh, not based on vertices at all, but it used the same scalar from the WindHelper. The artists simply input how much they want the mesh to rotate at maximum. Adding rotation to the Sway System turned out to be a great idea, and combined with the vertex sway, really adds life to palm branches and other foliage of the scenes.
My maxsciprts improved the pipeline because it allows artists to distort the meshes exactly how they want them to look, rather than wrestling with vertex paint in hoping it looks as intended. I feel that I made some substantial changes to the Sway System by introducing new features and cleaning up the code. Creating scripts to automate and optimize pipelines is the most rewarding thing about being a Tech Artist. Its double rewarding when I create these scripts for team members to use, rather than just make my job easier!
In 2016, I started work at Sanzaru Games. They knee-deep developing "VR Sports Challenge" an Oculus Touch Controller Launch game! Hockey brawling was being added as an awesome mini game. Besides punching, the designers wanted Players to be able to grab their oponent by their jersey. Because rendering VR is so costly, the game budget could not afford dynamic cloths. So, we suggested blendshapes baked into the vertex color! Luckily, there's only 3 preset points of grabbing ( left, right, and collar ) All three blendshapes were stored in a single mesh's vertices.
I resurrected my Vertex Painting scripts from two years prior. While WayForward used 3DSMax, Sanzaru is a Maya household, so I needed to port my code from Maxscript to Python! With a chance to revisit, I resolved many shortcomings. Most important, I split the 8-bit color channels into negative and positive space. So, the Character Artists were not limited to offseting the vertices in one direction anymore!
The Hockey Jersey Material was extended to read the vertex color data to displace the vertices into the blendshape created in Maya. In addition, the Player's Hand transform was also passed to the material so that the weighted verts could be offset and rotated. The effect could only be pushed so far before the jersey mesh seemed broken. Once players moved their hand too far, they would snap out of the grab. I'm so happy that my scripts were usable in a more AAA title and engine!