Offsetting Animations in Shaders

Last updated 2023- 5-14, 4:16:12 A.M. UTC+00:00

They like to know it as "Time Displacement".

There's really only a couple of small things that'd prevent me from trying to do a whole vid in [Blender Shaders].
— MMaker, July 2020

And one of these small things was the "Time Displacement" effect we are very used to in After Effects.

Time Displacement in AE

From VSauce's "DISTORTIONS"

For the uninitiated, the Time Displacement effect is a compound effect. An arbitrary map's luminance is used as the weight of the effect. As is the case with any other weight map in AE, 0.5 represents no change, 0 full negative, and 1 full positive. Then we have the Max Displacement Time slider, this is the full magnitude (it's also possible to reverse the effect with a negative value here) of the displacement (in seconds). And the last slider, Time Resolution (in frames/s), is not relevant in our discussion of shaders. Using this effect often results in noticeable seams between two frames that are time-displaced, this is simply due to the fact that there are not enough in-between frames to fill in the pixels. So in effect, we can think of the weight map as if it is being posterized. This is not so much of a problem for procedural animations in shaders as they are not based on frames.

Time Displacement as a Hack for Halftone

Time Displacement has been used as a way to achieve the halftone effect natively in After Effects, it's far from ideal but one that nonetheless works.

AFTER EFFECTS SUPER FAST HALFTONE TUTORIAL #AFTEREFFECTS #MOTIONGRAPHICS #TUTORIAL

— MMaker (@stysmmaker) September 12, 2019

If all you wanted was halftone in shaders, you don't need to deal with any of this funkiness.

halftone in blender! pic.twitter.com/haYtapEUor

— lachrymaL (@lachrymaL_F) August 31, 2020

Abstract

So here's effect we're trying to achieve:

Learn how to have nice randomized tile animations to use as a matte for transitions.
I don't trust twitter, so to compensate for the video being compressed to oblivion, I shall talk the video through. pic.twitter.com/R2doB90r6P

— CH FR (@chfr_otomads) January 13, 2021

In theory, this isn't very difficult. We set up an animation of a circle scaling up, use UV manipulation techniques to get a grid of them, and then offset each block's animation completion with a noise/random value, changing the offset completion would then drive everything to completion. And indeed, it is not very hard to implement this idea, but this isn't the exact effect described in CH FR's video above, we have ignored the fact that, in AE, we have total control over what we are trying to offset, including the timing function/easing of the animation.

The problem then escalates in complexity. But I don't think the solution is all that difficult to understand or even very cumbersome to obtain. One could simply pass the result obtained from the previous paragraph (global completion with offset applied), normalized and clipped, into a timing function, and that would solve our problem. It is crucial that the offsetting happens before entering the shaping function.

Method

Consider the following animation (1).

On the left is a 2x2 grid of the same diagonal wipe animation, and on the right is the animation curve that describes the completion of the animation.

And now let's offset the animation in each square, I'll use specific values here, but the idea is the same with noise and random numbers, I'll also color code them so it's easier to see on the graph. (2)

If this is all you're trying to do, we're done here! But let's continue to try to implement the timing function to have the animation be eased. We'll be using a quintic curve here.

Naively applying the timing function yields the following result. (3)

Note on the Newton–Raphson method

This implementation of cubic bezier is based on this one from Golan Levin (I don't know why he defined the same function twice). Since the cubic bezier curve is a parametric 2D curve (that is, using two different functions x(t) and y(t) (both of which are cubics) to describe the X and Y coordinates of every point on the curve, the t is called the parameter and its domain is [0, 1]). We need to solve the cubic x(t) from our given X-value, which would be the current time of the playhead in our application, to obtain the parameter t which we then plug into y(t) to find the corresponding Y-coordinate. This implementation uses the numerical Newton-Raphson method to approximate the roots of x(t).

To use this method we actually need the derivative of the function we're trying to solve, fortunately a cubic's derivative is a quadratic and that's pretty easy to compute.

It converges pretty fast for our case so we're only doing 5 iterations. (We just use x for our initial guess t₀)

In the end we plug t₅ into y(t) to get the final value.

Again, we get this chaotic result because the offset happens after the shaping function. The values coming out of the function are being offset out of the range accepted for our animation completion. If we switch the order of operations here as stated in the abstract, we obtain the following. (4)

And this is pretty much what we want, we could even switch out the timing function for something else, such as the cubic bezier curve that everybody's used to. (5) Try dragging the Bezier handles!

A couple more trivial changes and we get our desired effect. (6)

Frame-based Time Displacement in Shaders

So would there be a way to time displace a video in shaders? Our pal Roughy ran into something similar while working on The Power of Terry. The goal was essentially to have a flipbook of all the frames of a video, each frame as a plane in 3D space. This turned out to be particularly challenging in Blender. We looked at drivers and scripting, drivers couldn't cut it and no one knew how to script, ultimately, Roughy did all of it by hand, manually assigning materials to every plane. A couple days later I dug into OSL and managed to do it with a filename-concatenation method, basically replacing the image texture input node. One unfortunate downside is that OSL is not implemented with GPU rendering in Cycles. While I have seen that V-Ray and Octane seem to have support for OSL running on the GPU, this seems to be a nonstandard implementation.

WTF Mr. Roosendaal?

Frame-based Time Displacement is similarly doable through OSL in Blender.