Hello, Project Redraw

Back in 2022, when Christian Falch and I introduced React Native Skia, we showed an example that looks like this:

<Canvas>
  <Path
    path={hello}
    strokeWidth={60}
    color="#5F96E7"
  />
</Canvas>

We draw a path. The stroke width is a static number, and so is the color. That's it. In the back of my mind, I always had a fantasy: instead of passing a number, we could pass a function. That function receives geometric information as input, such as the tangent and the arc length of the path, and we use it to return what the stroke width or the color the path should be. In the function below the stroke width is a sine curve that depends on the arc length of the path.

Same for the color, instead of a static color value or a crude gradient, we could pass a function instead. For instance, in the example below, we interpolate the color based on the arc length of the path.

<Path
  path={hello}
  strokeWidth={(ctx) => {
    "use gpu";
    const wave = std.sin(ctx.t * 30);
    return 60 + wave * 30;
  }}
  color={(ctx) => {
    "use gpu";
    return interpolateColors(ctx.t, colors);
  }}
/>

This used to be just a fantasy but there have been a couple of advances that are making this possible today and therefore enable me to build project Redraw. The first one is WebGPU. Because WebGPU represents a unified runtime across graphic libraries such as Skia and Three.js, you can incrementally build your own primitives that will integrate with existing libraries, you do not need to rebuild the whole world yourself. But more importantly, WebGPU features such as compute shaders and storage buffers enable us to build robust acceleration structures for Bézier path rendering.

The second technical advance that makes this possible is TypeGPU. You may have noticed that the functions in the example above use the "use gpu" directive. This means that the function is compiled and executed on GPU. There are a couple of things I really like about this project. First, the compilation is done at bundling time so at runtime there is no heavy dependency or compilation step needed. The second aspect is a feature from TypeGPU named compTime that allows for statically known JS expressions to be compiled directly to the WebGPU shading language. This allows for a lot of convenience like in the code sample below where we use a statically known arbitrarily sized array as well as string values. At compilation-time, we know enough to parse the string values as color values and instantiate the interpolateColors to support that exact size of color array.

<Path
  path={hello}
  color={(ctx) => {
    "use gpu";
    return interpolateColors(
      ctx.t,
      // An arbitrary sized array of strings
      // in a shader? 🤔
      ["cyan", "magenta", "yellow"]
    );
  }}
/>

Now because the color function receives rich geometric information as parameter, namely the distance to the closest point on the path, the tangent of the path at that position, as well as the path arc length, we can build all sorts of complex path renderings such as the one seen below:

So far we have only mentioned stroke width and color, but Redraw paths have a third property: feather. Feathering softens a shape's edge as a function of its geometry rather than by blurring pixels after a rasterization step. It doesn't need rasterization nor pixel convolutions. That unlocks a couple of effects fun effects. The first is analytical motion blur: instead of stacking or sampling frames, we derive the blur amount and direction directly from an object's velocity. The second is non-raster backdrop filters: a frosted-glass effect normally needs an offscreen pass that copies and blurs the background, but here the blur is evaluated as vector geometry in a single pass. Both would be too slow and too complex to achieve with traditional raster techniques.

From my experience, physically-based renderings in the 2d world are usually done via raster effects which are quite slow and/or by painstakingly baking all of the lighting information manually for every single component. However I believe that like for 3d graphics, we can also procedurally add PBR to 2d user-interfaces. Developers and designers can build flat and delightful user-interfaces and then add depth and materiality with a few lines of code. In the example below, we procedurally add a bevel material to a simple 2d shape. We define the color, bevel geometry as well as lighting information.

Redraw is still at its experimental stage but we have a playground where you can try these demos. And if you are a subscriber, you can get early access to the project. With Redraw, I was scratching my own itch. Now I would love to understand better if it can help you. If you think Redraw can be helpful in your own creative work, reach out, I would love to hear from you.