Split the rain pass's compute shader in two, with one governing brightness and the other governing glyph cycling. This allows glyphs to randomly cycle properly, and leaves room to store new properties.

This commit is contained in:
Rezmason
2022-09-07 12:35:27 -07:00
parent 76d37fc752
commit 2eb7b70926
8 changed files with 200 additions and 139 deletions

View File

@@ -50,7 +50,7 @@ While there have been a lot of attempts at #1 and #3, they're all missing import
- **Get the glow and color right.** Matrix symbols aren't just some shade of phosphorous green; they're first given a bloom effect, and then get tone-mapped to the green color palette.
- **Symbols change shape faster as they dim.** When symbols light up, they almost never change shape, but their cycle speed increases the darker and darker they get.
- **Two "raindrops" can occupy the same column.** This is complicated, because we can't allow them to collide. A useful approach to thinking about this is, each column's glyph brightness is a kind of [sawtooth wave](http://mathworld.wolfram.com/SawtoothWave.html).
- **Capture the glyph sequence.** Yes, the symbols in the sequels' opening titles, which are arguably the highest quality versions of the 2D effect, change according to a repeating sequence (see `glyph order.txt`).
- **Capture the glyph sequence.** Yes, the symbols in the sequels' opening titles, which are arguably the highest quality versions of the 2D effect, change according to a repeating sequence (see `glyph order.txt`). This is only a technical detail, and only applies to *Reloaded* and *Revolutions*— everyplace else, the symbols change randomly.
- **Make it free, open source and web based.** Because someone could probably improve on what I've done, and I'd like to see that, and maybe incorporate their improvements back into this project.
- **Support as many browsers and devices as possible.** This project used to rely on Three.js's GPUComputationRenderer, which only worked in browsers supporting WebGL's [oes_texture_float extension](https://caniuse.com/#search=OES_texture_float). The rewrite dropped this dependency, and gained support for a broader range of browsers and devices.
- **Whip up some artistic license and depict the *previous* Matrix versions.** The sequels describe [a paradisiacal predecessor](https://rezmason.github.io/matrix?version=paradise) to the Matrix that was too idyllic, [and another earlier, nightmarish Hobbesian version](https://rezmason.github.io/matrix?version=nightmare) that proved too campy. They depict some programs running older, differently colored code, so it's time someone tried rendering them.

View File

@@ -1,9 +1,18 @@
TODO:
Random cycling should actually be random
Compute shader should have separate channels for cycle and symbol
Update WebGPU
Compute shine and symbol in separate methods of compute shader
Try to identify cursors more consistently
Reformulate the basis
https://buf.com/films/the-matrix-resurrections
Base cursors and other colors on BUF clip
Show this stuff for showComputationTexture
Pixel grill?
Tune the colors
Maybe glow can be an SDF-derived effect instead, look into it
Migrate the rest of the project over
Migrate WebGPU
Audio system
Toggle (or number representing frequency)
@@ -27,21 +36,7 @@ Create a Resurrections font
Icomoon
Unicode PUA
Reformulate the basis
https://buf.com/films/the-matrix-resurrections
Why does my existing affect feel *sparser* than these?
Softer edges— base them on fwidth still, just soften them
Base cursors and other colors on BUF clip
Pixel grill?
Tighten the threshold
Tweak the colors
Maybe glow can be an SDF-derived effect instead, look into it
Migrate the rest of the project over
Migrate WebGPU
Resurrections
Support random glyph order
Support anomaly streaks
MSDF
They should line up in Photoshop without too much trouble, actually
@@ -55,8 +50,6 @@ Resurrections
Get the "normals" and color right
Note: even completely dark glyphs can have glint on their edges
"Golden hour"
Eventually improve expanded new glyph set
Make font
WebGPU
Why is it brighter than the regl version?

View File

@@ -55,16 +55,16 @@ const defaults = {
font: "matrixcode",
useCamera: false,
backgroundColor: [0, 0, 0], // The color "behind" the glyphs
cursorBrightness: 0, // The brightness of the "cursor" at the bottom of a raindrop
volumetric: false, // A mode where the raindrops appear in perspective
animationSpeed: 1, // The global rate that all animations progress
forwardSpeed: 0.25, // The speed volumetric rain approaches the eye
bloomStrength: 0.7, // The intensity of the bloom
bloomSize: 0.4, // The amount the bloom calculation is scaled
highPassThreshold: 0.1, // The minimum brightness that is still blurred
cycleSpeed: 0.5, // The speed glyphs change
cycleSpeed: 0.2, // The speed glyphs change
cycleFrameSkip: 1, // The global minimum number of frames between glyphs cycling
cycleStyleName: "cycleFasterWhenDimmed", // The way glyphs cycle, either proportional to their brightness or randomly
cursorEffectThreshold: 1, // The minimum brightness for a glyph to still be lit up as a cursor at the bottom of a raindrop
baseBrightness: -0.5, // The brightness of the glyphs, before any effects are applied
baseContrast: 1.1, // The contrast of the glyphs, before any effects are applied
brightnessOverride: 0.0, // A global override to the brightness of displayed glyphs. Only used if it is > 0.
@@ -108,15 +108,13 @@ const versions = {
width: 40,
},
operator: {
baseBrightness: -0.3,
baseContrast: 1,
cursorBrightness: 1,
bloomSize: 0.6,
bloomStrength: 0.75,
highPassThreshold: 0.0,
cycleSpeed: 0.2,
cycleSpeed: 0.01,
cycleFrameSkip: 8,
cycleStyleName: "cycleRandomly",
cursorEffectThreshold: 0.69,
brightnessOverride: 0.22,
brightnessThreshold: 0,
fallSpeed: 0.6,
@@ -134,12 +132,12 @@ const versions = {
nightmare: {
font: "gothic",
highPassThreshold: 0.7,
baseBrightness: -0.9,
baseBrightness: -0.8,
brightnessDecay: 0.75,
fallSpeed: 1.2,
hasThunder: true,
numColumns: 60,
cycleSpeed: 1,
cycleSpeed: 0.35,
paletteEntries: [
{ hsl: [0.0, 1.0, 0.0], at: 0.0 },
{ hsl: [0.0, 1.0, 0.2], at: 0.2 },
@@ -154,7 +152,7 @@ const versions = {
font: "coptic",
bloomStrength: 1,
highPassThreshold: 0,
cycleSpeed: 0.1,
cycleSpeed: 0.05,
baseBrightness: -0.1,
brightnessDecay: 0.05,
fallSpeed: 0.04,
@@ -174,11 +172,16 @@ const versions = {
},
resurrections: {
font: "resurrections",
glyphEdgeCrop: 0.1,
cursorBrightness: 1,
baseBrightness: -0.7,
baseContrast: 1.17,
highPassThreshold: 0,
numColumns: 70,
cycleStyleName: "cycleRandomly",
cycleSpeed: 0.15,
bloomStrength: 0.8,
fallSpeed: 0.2,
cycleSpeed: 0.05,
bloomStrength: 0.7,
fallSpeed: 0.3,
paletteEntries: [
{ hsl: [0.38, 0.9, 0.0], at: 0.0 },
{ hsl: [0.38, 1.0, 0.6], at: 0.92 },
@@ -222,7 +225,7 @@ const versions = {
numColumns: 20,
fallSpeed: 0.35,
cycleStyleName: "cycleRandomly",
cycleSpeed: 0.8,
cycleSpeed: 0.3,
glyphEdgeCrop: 0.1,
ditherMagnitude: 0,
paletteEntries: [
@@ -232,7 +235,6 @@ const versions = {
],
raindropLength: 1.4,
highPassThreshold: 0.2,
cursorEffectThreshold: 0.8,
renderer: "regl",
bloomStrength: 0,
@@ -242,14 +244,14 @@ const versions = {
useHoloplay: true,
},
['3d']: {
["3d"]: {
volumetric: true,
fallSpeed: 0.5,
cycleSpeed: 1,
cycleSpeed: 0.35,
baseBrightness: -0.9,
baseContrast: 1.5,
raindropLength: 0.3
}
raindropLength: 0.3,
},
};
versions.throwback = versions.operator;
versions.updated = versions.resurrections;

View File

@@ -53,24 +53,19 @@ export default ({ regl, config, lkg }) => {
// This double buffer is smaller than the screen, because its pixels correspond
// with glyphs in the final image, and the glyphs are much larger than a pixel.
const doubleBuffer = makeDoubleBuffer(regl, {
const shineDoubleBuffer = makeDoubleBuffer(regl, {
width: numColumns,
height: numRows,
wrapT: "clamp",
type: "half float",
});
const rainPassCompute = loadText("shaders/glsl/rainPass.compute.frag.glsl");
const computeUniforms = {
const rainPassShine = loadText("shaders/glsl/rainPass.shine.frag.glsl");
const shineUniforms = {
...commonUniforms,
...extractEntries(config, [
"brightnessThreshold",
"brightnessOverride",
"baseBrightness",
"baseContrast",
"brightnessDecay",
"cursorEffectThreshold",
"cycleSpeed",
"cycleFrameSkip",
"fallSpeed",
"hasSun",
"hasThunder",
@@ -80,17 +75,39 @@ export default ({ regl, config, lkg }) => {
"rippleThickness",
"loops",
]),
cycleStyle,
rippleType,
};
const compute = regl({
const shine = regl({
frag: regl.prop("frag"),
uniforms: {
...computeUniforms,
previousState: doubleBuffer.back,
...shineUniforms,
previousShineState: shineDoubleBuffer.back,
},
framebuffer: doubleBuffer.front,
framebuffer: shineDoubleBuffer.front,
});
const symbolDoubleBuffer = makeDoubleBuffer(regl, {
width: numColumns,
height: numRows,
wrapT: "clamp",
type: "half float",
});
const rainPassSymbol = loadText("shaders/glsl/rainPass.symbol.frag.glsl");
const symbolUniforms = {
...commonUniforms,
...extractEntries(config, ["cycleSpeed", "cycleFrameSkip", "loops"]),
cycleStyle,
};
const symbol = regl({
frag: regl.prop("frag"),
uniforms: {
...symbolUniforms,
shineState: shineDoubleBuffer.front,
previousSymbolState: symbolDoubleBuffer.back,
},
framebuffer: symbolDoubleBuffer.front,
});
const quadPositions = Array(numQuadRows)
@@ -113,6 +130,9 @@ export default ({ regl, config, lkg }) => {
"forwardSpeed",
"glyphVerticalSpacing",
// fragment
"brightnessThreshold",
"brightnessOverride",
"cursorBrightness",
"glyphEdgeCrop",
"isPolar",
]),
@@ -138,7 +158,8 @@ export default ({ regl, config, lkg }) => {
uniforms: {
...renderUniforms,
state: doubleBuffer.front,
shineState: shineDoubleBuffer.front,
symbolState: symbolDoubleBuffer.front,
glyphTex: msdf.texture,
camera: regl.prop("camera"),
@@ -181,7 +202,7 @@ export default ({ regl, config, lkg }) => {
{
primary: output,
},
Promise.all([msdf.loaded, rainPassCompute.loaded, rainPassVert.loaded, rainPassFrag.loaded]),
Promise.all([msdf.loaded, rainPassShine.loaded, rainPassVert.loaded, rainPassFrag.loaded]),
(w, h) => {
output.resize(w, h);
const aspectRatio = w / h;
@@ -231,7 +252,8 @@ export default ({ regl, config, lkg }) => {
[screenSize[0], screenSize[1]] = aspectRatio > 1 ? [1, aspectRatio] : [1 / aspectRatio, 1];
},
() => {
compute({ frag: rainPassCompute.text() });
shine({ frag: rainPassShine.text() });
symbol({ frag: rainPassSymbol.text() });
regl.clear({
depth: 1,
color: [0, 0, 0, 1],

View File

@@ -4,10 +4,11 @@
#endif
precision lowp float;
uniform sampler2D state;
uniform sampler2D shineState, symbolState;
uniform float numColumns, numRows;
uniform sampler2D glyphTex;
uniform float glyphHeightToWidth, glyphSequenceLength, glyphEdgeCrop;
uniform float brightnessOverride, brightnessThreshold, cursorBrightness;
uniform vec2 glyphTextureGridSize;
uniform vec2 slantVec;
uniform float slantScale;
@@ -17,7 +18,7 @@ uniform bool volumetric;
varying vec2 vUV;
varying vec3 vChannel;
varying vec4 vGlyph;
varying vec4 vShine, vSymbol;
varying float vDepth;
float median3(vec3 i) {
@@ -29,11 +30,10 @@ float modI(float a, float b) {
return floor(m + 0.5);
}
vec2 getSymbolUV(float glyphCycle) {
float symbol = floor((glyphSequenceLength) * glyphCycle);
vec2 getSymbolUV(float symbol) {
float symbolX = modI(symbol, glyphTextureGridSize.x);
float symbolY = (symbol - symbolX) / glyphTextureGridSize.x;
symbolY = glyphTextureGridSize.y - symbolY - 1.0;
symbolY = glyphTextureGridSize.y - symbolY - 1.;
return vec2(symbolX, symbolY);
}
@@ -61,38 +61,45 @@ void main() {
uv.y /= glyphHeightToWidth;
}
// Unpack the values from the data texture
vec4 glyph = volumetric ? vGlyph : texture2D(state, uv);
float brightness = glyph.r;
vec2 symbolUV = getSymbolUV(glyph.g);
float effect = glyph.a;
// Unpack the values from the data textures
vec4 shine = volumetric ? vShine : texture2D(shineState, uv);
vec4 symbol = volumetric ? vSymbol : texture2D(symbolState, uv);
vec2 symbolUV = getSymbolUV(symbol.r);
brightness = max(effect, brightness);
float brightness = shine.r;
// Modes that don't fade glyphs set their actual brightness here
if (brightnessOverride > 0. && brightness > brightnessThreshold) {
brightness = brightnessOverride;
}
brightness = max(shine.b * cursorBrightness, brightness);
brightness = max(shine.a, brightness);
// In volumetric mode, distant glyphs are dimmer
if (volumetric) {
brightness = brightness * min(1.0, vDepth);
brightness = brightness * min(1., vDepth);
}
// resolve UV to cropped position of glyph in MSDF texture
vec2 glyphUV = fract(uv * vec2(numColumns, numRows));
glyphUV -= 0.5;
glyphUV *= clamp(1.0 - glyphEdgeCrop, 0.0, 1.0);
glyphUV *= clamp(1. - glyphEdgeCrop, 0., 1.);
glyphUV += 0.5;
vec2 msdfUV = (glyphUV + symbolUV) / glyphTextureGridSize;
// MSDF: calculate brightness of fragment based on distance to shape
vec3 dist = texture2D(glyphTex, msdfUV).rgb;
float sigDist = median3(dist) - 0.5;
float alpha = clamp(sigDist/fwidth(sigDist) + 0.5, 0.0, 1.0);
float alpha = clamp(sigDist/fwidth(sigDist) + 0.5, 0., 1.);
if (showComputationTexture) {
vec4 debugColor = vec4(glyph.r - alpha, glyph.g * alpha, glyph.a - alpha, 1.0);
vec4 debugColor = vec4(shine.r - alpha, shine.g * alpha, shine.a - alpha, 1.);
if (volumetric) {
debugColor.g = debugColor.g * 0.9 + 0.1;
}
gl_FragColor = debugColor;
} else {
gl_FragColor = vec4(vChannel * brightness * alpha, 1.0);
gl_FragColor = vec4(vChannel * brightness * alpha, 1.);
}
}

View File

@@ -1,10 +1,11 @@
precision highp float;
// This shader is the star of the show. For each glyph, it determines its:
// This shader is the star of the show.
// It writes falling rain to four channels of a data texture:
// R: brightness
// G: progress through the glyph sequence
// B: unused!
// A: additional brightness for effects
// G: unused
// B: whether the cell is a "cursor"
// A: some other effect, such as a ripple
// Listen.
// I understand if this shader looks confusing. Please don't be discouraged!
@@ -15,19 +16,17 @@ precision highp float;
#define SQRT_2 1.4142135623730951
#define SQRT_5 2.23606797749979
uniform sampler2D previousState;
uniform sampler2D previousShineState;
uniform float numColumns, numRows;
uniform float time, tick, cycleFrameSkip;
uniform float animationSpeed, fallSpeed, cycleSpeed;
uniform float time, tick;
uniform float animationSpeed, fallSpeed;
uniform bool hasSun, hasThunder, loops;
uniform bool showComputationTexture;
uniform float brightnessOverride, brightnessThreshold, brightnessDecay;
uniform float brightnessDecay;
uniform float baseContrast, baseBrightness;
uniform float raindropLength, glyphHeightToWidth, glyphSequenceLength;
uniform int cycleStyle, rippleType;
uniform float raindropLength, glyphHeightToWidth;
uniform int rippleType;
uniform float rippleScale, rippleSpeed, rippleThickness;
uniform float cursorEffectThreshold;
// Helper functions for generating randomness, borrowed from elsewhere
@@ -67,16 +66,6 @@ float getBrightness(float rainTime) {
return value * baseContrast + baseBrightness;
}
float getCycleSpeed(float rainTime, float brightness) {
float localCycleSpeed = 0.;
if (cycleStyle == 0 && brightness > 0.) {
localCycleSpeed = pow(1. - brightness, 4.);
} else if (cycleStyle == 1) {
localCycleSpeed = fract(rainTime);
}
return animationSpeed * cycleSpeed * localCycleSpeed;
}
// Additional effects
float applySunShowerBrightness(float brightness, vec2 screenPos) {
@@ -131,77 +120,44 @@ float applyRippleEffect(float effect, float simTime, vec2 screenPos) {
return effect;
}
float applyCursorEffect(float effect, float brightness) {
if (brightness >= cursorEffectThreshold) {
effect = 1.;
}
return effect;
}
// Main function
vec4 computeResult(bool isFirstFrame, vec4 previousResult, vec2 glyphPos, vec2 screenPos) {
vec4 computeResult(float simTime, bool isFirstFrame, vec2 glyphPos, vec2 screenPos, vec4 previous, vec4 previousBelow) {
// Determine the glyph's local time.
float simTime = time * animationSpeed;
float rainTime = getRainTime(simTime, glyphPos);
// Rain time is the backbone of this effect.
// Determine the glyph's brightness.
float previousBrightness = previousResult.r;
float brightness = getBrightness(rainTime);
if (hasSun) {
brightness = applySunShowerBrightness(brightness, screenPos);
}
if (hasThunder) {
brightness = applyThunderBrightness(brightness, simTime, screenPos);
}
// Determine the glyph's cycle— the percent this glyph has progressed through the glyph sequence
float previousCycle = previousResult.g;
bool resetGlyph = isFirstFrame;
if (loops) {
resetGlyph = resetGlyph || previousBrightness <= 0.;
}
if (resetGlyph) {
previousCycle = showComputationTexture ? 0. : randomFloat(screenPos);
}
float localCycleSpeed = getCycleSpeed(rainTime, brightness);
float cycle = previousCycle;
if (mod(tick, cycleFrameSkip) == 0.) {
cycle = fract(previousCycle + 0.005 * localCycleSpeed * cycleFrameSkip);
}
if (hasSun) brightness = applySunShowerBrightness(brightness, screenPos);
if (hasThunder) brightness = applyThunderBrightness(brightness, simTime, screenPos);
// Determine the glyph's effect— the amount the glyph lights up for other reasons
float effect = 0.;
effect = applyRippleEffect(effect, simTime, screenPos); // Round or square ripples across the grid
effect = applyCursorEffect(effect, brightness); // The bright glyphs at the "bottom" of raindrops
// Modes that don't fade glyphs set their actual brightness here
if (brightnessOverride > 0. && brightness > brightnessThreshold) {
brightness = brightnessOverride;
}
float previousBrightnessBelow = previousBelow.r;
float cursor = brightness > previousBrightnessBelow ? 1.0 : 0.0;
// Blend the glyph's brightness with its previous brightness, so it winks on and off organically
if (!isFirstFrame) {
float previousBrightness = previous.r;
brightness = mix(previousBrightness, brightness, brightnessDecay);
}
vec4 result = vec4(brightness, cycle, 0.0, effect);
// Better use of the alpha channel, for demonstrating how the glyph cycle works
if (showComputationTexture) {
result.a = min(1., localCycleSpeed);
}
vec4 result = vec4(brightness, 0., cursor, effect);
return result;
}
void main() {
float simTime = time * animationSpeed;
bool isFirstFrame = tick <= 1.;
vec2 glyphPos = gl_FragCoord.xy;
vec2 screenPos = glyphPos / vec2(numColumns, numRows);
vec4 previousResult = texture2D( previousState, screenPos );
gl_FragColor = computeResult(isFirstFrame, previousResult, glyphPos, screenPos);
vec4 previous = texture2D( previousShineState, screenPos );
vec4 previousBelow = texture2D( previousShineState, screenPos + vec2(0., -1. / numRows));
gl_FragColor = computeResult(simTime, isFirstFrame, glyphPos, screenPos, previous, previousBelow);
}

View File

@@ -0,0 +1,80 @@
precision highp float;
// This shader governs the glyphs appearing in the rain.
// It writes each glyph's state to four channels of a data texture:
// R: symbol
// G: age
// B: unused
// A: unused
#define PI 3.14159265359
uniform sampler2D previousSymbolState, shineState;
uniform float numColumns, numRows;
uniform float time, tick, cycleFrameSkip;
uniform float animationSpeed, cycleSpeed;
uniform bool loops, showComputationTexture;
uniform float glyphSequenceLength;
uniform int cycleStyle;
// Helper functions for generating randomness, borrowed from elsewhere
highp float randomFloat( const in vec2 uv ) {
const highp float a = 12.9898, b = 78.233, c = 43758.5453;
highp float dt = dot( uv.xy, vec2( a,b ) ), sn = mod( dt, PI );
return fract(sin(sn) * c);
}
// Core functions
float getCycleSpeed(float brightness) {
float localCycleSpeed = 1.;
if (cycleStyle == 0 && brightness > 0.) {
localCycleSpeed = pow(1. - brightness, 4.);
}
return animationSpeed * cycleSpeed * localCycleSpeed;
}
// Main function
vec4 computeResult(float simTime, bool isFirstFrame, vec2 glyphPos, vec2 screenPos, vec4 previous, vec4 shine) {
float brightness = shine.r;
float previousSymbol = previous.r;
float previousAge = previous.g;
bool resetGlyph = isFirstFrame;
if (loops) {
resetGlyph = resetGlyph || brightness <= 0.;
}
if (resetGlyph) {
previousAge = randomFloat(screenPos + vec2(0.5));
previousSymbol = floor(glyphSequenceLength * randomFloat(screenPos));
}
float cycleSpeed = getCycleSpeed(brightness);
float age = previousAge;
float symbol = previousSymbol;
if (mod(tick, cycleFrameSkip) == 0.) {
age += cycleSpeed * cycleFrameSkip;
float advance = floor(age);
age = fract(age);
if (cycleStyle == 0) {
symbol = mod(symbol + advance, glyphSequenceLength);
} else if (cycleStyle == 1 && advance > 0.) {
symbol = floor(glyphSequenceLength * randomFloat(screenPos + vec2(simTime)));
}
}
vec4 result = vec4(symbol, age, 0., 0.);
return result;
}
void main() {
float simTime = time * animationSpeed;
bool isFirstFrame = tick <= 1.;
vec2 glyphPos = gl_FragCoord.xy;
vec2 screenPos = glyphPos / vec2(numColumns, numRows);
vec4 previous = texture2D( previousSymbolState, screenPos );
vec4 shine = texture2D( shineState, screenPos );
gl_FragColor = computeResult(simTime, isFirstFrame, glyphPos, screenPos, previous, shine);
}

View File

@@ -1,7 +1,7 @@
#define PI 3.14159265359
precision lowp float;
attribute vec2 aPosition, aCorner;
uniform sampler2D state;
uniform sampler2D shineState, symbolState;
uniform float density;
uniform vec2 quadSize;
uniform float glyphHeightToWidth, glyphVerticalSpacing;
@@ -11,7 +11,7 @@ uniform float time, animationSpeed, forwardSpeed;
uniform bool volumetric;
varying vec2 vUV;
varying vec3 vChannel;
varying vec4 vGlyph;
varying vec4 vShine, vSymbol;
varying float vDepth;
highp float rand( const in vec2 uv ) {
@@ -23,7 +23,8 @@ highp float rand( const in vec2 uv ) {
void main() {
vUV = (aPosition + aCorner) * quadSize;
vGlyph = texture2D(state, aPosition * quadSize);
vShine = texture2D(shineState, aPosition * quadSize);
vSymbol = texture2D(symbolState, aPosition * quadSize);
// Calculate the world space position
float quadDepth = 0.0;