Skip to content

davnotdev/spirv-webgpu-transform

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

154 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SPIRV WebGPU Transforms

Version Badge Docs Badge License Badge Downloads Badge

When porting native games to the web using WebGPU, it becomes neccessary to translate shaders (typically from SPIR-V) to WGSL using naga or tint. Unfortunately, the WGSL specification lacks support for many features that have shader programmers have become accustomed to. This project aims to transform common but unsupported SPIRV shaders into a form that naga and tint can transpile.

This also includes an interface for some dodging unsupported features or features that may not run on older browser versions.

Under heavy development. Remember, the published crate source may lag behind the github source!

Feature Summary

At the moment, the following transformations are supported:

Feature spirv-val naga tint
Combined Image Samplers
Immediates (Push Constants) ✅*
Binding Arrays
Mixed Depth / Comparison ⚠️*
isnan / isinf Patching
Storage Cube Patching
Unused Image Sampler Pruning

(1)* 99% OK, just one very specific padding related naga bug.

(2)* Simple cases are OK. With some special patches, naga can process these.

Combined Image Samplers

It is commonly known that WebGpu does not support combined image samplers. This makes adding WebGpu support for existing OpenGL or Vulkan renderers impossible without workarounds. This is one such workaround. By reading and modifying SPIRV byte code, combined image samplers can be split into their respective texture and sampler. Special edge cases such as the use of combined image samplers in function parameters and nested functions are also handled.

layout(set = 0, binding = 0) uniform sampler2D u_texture;
layout(set = 0, binding = 1) uniform sampler2DArray u_texture_array;

// is converted into...

layout(set = 0, binding = 0) uniform texture2D u_texture;
layout(set = 0, binding = 1) uniform sampler u_sampler;

// *texture2DArray doesn't exist in glsl, but in wgsl, this would be texture_2d_array<f32>
layout(set = 0, binding = 2) uniform texture2DArray u_texture_array;
layout(set = 0, binding = 3) uniform sampler u_sampler;

Additional Notes

  • Translating sampler2D[N] and sampler2DArray[N] is NOT supported.
  • After being split, the SPIR-V will not translate back to GLSL "one-to-one", the translation back to GLSL using either naga or tint creates a combined image sampler!
  • This implementation has not been updated to use the current function nesting implementation, so extremely strange function nesting patterns may cause issues

Tests

Test spirv-val Naga Tint
test.frag
test_nested.frag
test_arrayed.frag
test_mixed.frag

Immediates (Push Constants)

Immediates, more commonly known as push constants came to the WebGPU spec late, so browser support is still unreliable (as of writing). This patch replaces push constants with an equivalent uniform buffer.

layout(std430, push_constant) uniform PushBlock {
    vec4 color;
    mat4 transform;
    float scale;
    ...
} pc;

// is converted into...

layout(std140, set = N+1, binding = 0) uniform PushBlock {
    vec4 color;
    mat4 transform;
    float scale;
    ...
} pc;

// where N is the max set in the shader.

Additional Notes

  • The converted uniform is placed in set = N+1, where N is the highest descriptor set already in use.
  • Shaders that contain mat2 (or any matrix with 2-row columns: matCx2) in the push constant block will not pass through naga specifically.
  • To my knowledge, padding should not be a concern between uniforms, storage, and immediates.

Tests

Test spirv-val Naga Tint
immediates.frag
mat2_direct.frag ❌*
array_of_mat2.frag
nested_struct.frag
row_major.frag

* naga's SPIR-V front-end rejects MatrixStride 16 for mat2x2, this should be fixed soon (?).

Binding Arrays

Binding arrays are a feature commonly used in shaders and supported by WGSL compilers, just not on the web (yet?). This patch takes fixed size arrays of size N containing opaque or concrete types and splits them into N individual bindings. In wgpu, this covers following features:

  • TEXTURE_BINDING_ARRAY
  • BUFFER_BINDING_ARRAY
  • STORAGE_RESOURCE_BINDING_ARRAY
  • SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING
  • STORAGE_TEXTURE_ARRAY_NON_UNIFORM_INDEXING
struct Thing {
    float a;
};

#define MAX_RESOURCES 2
layout(set = 0, binding = 0) uniform sampler u_samplers[MAX_RESOURCES];
layout(set = 0, binding = 1) uniform texture2D u_textures[MAX_RESOURCES];
layout(set = 0, binding = 2) uniform texture2DArray u_texture_arrays[MAX_RESOURCES];
layout(set = 0, binding = 3) uniform image2D u_images[MAX_RESOURCES];
layout(set = 0, binding = 4) uniform Thing u_things[MAX_RESOURCES];
layout(set = 0, binding = 5, std140) buffer Thing u_buf_things[MAX_RESOURCES];

// is converted into...

struct Thing {
    float a;
};

layout(set = 0, binding = 0) uniform sampler u_samplers_0;
layout(set = 0, binding = 1) uniform sampler u_samplers_1;
layout(set = 0, binding = 2) uniform texture2D u_textures_0;
layout(set = 0, binding = 3) uniform texture2D u_textures_1;
layout(set = 0, binding = 4) uniform texture2DArray u_texture_arrays_0;
layout(set = 0, binding = 5) uniform texture2DArray u_texture_arrays_1;
layout(set = 0, binding = 6) uniform image2D u_images_0;
layout(set = 0, binding = 7) uniform image2D u_images_1;
layout(set = 0, binding = 8) uniform Thing u_things_0;
layout(set = 0, binding = 9) uniform Thing u_things_1;
layout(set = 0, binding = 10, std140) buffer Thing u_buf_things_0;
layout(set = 0, binding = 11, std140) buffer Thing u_buf_things_1;

Additional Notes

  • Combined image samplers are not supported, please run the combined image sampler pass first.
  • Nested resources (texture2D u[I][J][K]) are not supported.
  • Usage of additional SPIR-V capabilities such as SparseResidency or ImageQuery are not supported.

Tests

Test spirv-val Naga Tint
buffer_binding_array.frag
storage_binding_array.frag
texture_binding_array.frag
nested_texture_binding_array.frag
sampler_binding_array.frag
sampler_stub.frag
texture_array_binding_array.frag

Mixed Depth / Comparison

The WGSL spec differentiates between sampler and sampler_comparison as well as texture2d<T> and texture_depth_2d. In GLSL and SPIRV land, the rules on which can be used where are MUCH softer. In fact, in SPIRV, "whether or not depth comparisons are actually done is a property of the sampling opcode, not of this (image type) type declaration." WGSL technically does allow for the mixing of these types to some degree, but both naga and tint have trouble or simply CANNOT mix the two. For that reason, we need to decouple "regular" and "comparison" samplers and textures.

layout(set = 0, binding = 0) uniform sampler u_mixed_sampler;
layout(set = 0, binding = 1) uniform texture2D u_mixed_texture;

void main() {
    float g0 = textureProj(sampler2DShadow(u_mixed_texture, u_mixed_sampler), vec4(0.0, 0.0, 0.0, 0.0));
    vec4 g1 = textureLod(sampler2D(u_mixed_texture, u_mixed_sampler), vec2(0.0, 0.0), 0);
}

// is *ROUGHLY* converted into ...

layout(set = 0, binding = 0) uniform sampler u_mixed_sampler;
layout(set = 0, binding = 1) uniform sampler u_comparison_sampler;
layout(set = 0, binding = 2) uniform texture2D u_mixed_texture;
layout(set = 0, binding = 3) uniform texture2D u_comparison_texture;

void main() {
    float g0 = textureProj(sampler2DShadow(u_comparison_texture, u_comparison_texture), vec4(0.0, 0.0, 0.0, 0.0));
    vec4 g1 = textureLod(sampler2D(u_mixed_texture, u_mixed_sampler), vec2(0.0, 0.0), 0);
}

Tests

Test spirv-val Naga Tint
test_image.frag
test_wrong_type_image.spvasm
test_sampler.frag
test_mixed_dref.frag ❌*
test_nested_sampler.frag ❌*
test_nested2_sampler.frag ❌*
test_nested_image.frag ❌*
test_nested2_image.frag ❌*
test_hidden_dref.frag ❌*
test_hidden2_dref.frag ❌*
test_hidden3_dref.frag ❌*
test_cross_dref.frag ❌*

* With some special patches, naga can process these.

isnan / isinf Patching

WGSL does not have a mapping for the isnan and isinf functions from GLSL. This transformation replaces all isnan and isinf functions with an IEEE-754 appropriate implementation. Support also includes vector types (vecN input and bvecN output).

Tests

Test spirv-val Naga Tint
isnanisinf.frag
isnanisinf_vectored.frag
isnanisinf_immediate.frag

Additional Notes

  • Only 32-bit floats are supported, other bit widths are not supported

Storage Cube Patching

WGSL does not support GLSL's imageCube and equivalents. This transformation properly replaces imageCube with GLSL's image2DArray or WGSL's texture_storage_2d_array. The proper coordinate transformations are handled by the shader patch.

layout(set = 0, binding = 0) uniform writeonly imageCube u_ic;
// is converted into...
layout(set = 0, binding = 0) uniform writeonly image2DArray u_ic;

void main() {
    imageStore(u_ic, coord, value);
    // is converted into...
    imageStore(u_ic, _imageCubeDirectionToArrayed(coord), value);
}

Tests

Test spirv-val Naga Tint
storagecube.frag
storagecube_immediate.frag
storagecube_nested.frag

For function nesting, see this wgpu issue and this gpuweb issue

Additional Notes

  • You can nest imageCube usages in functions, but this will not translate to WGSL
  • imageCubeArray is not supported

Unused Image Sampler Pruning

In ubershader configurations, it is common to have uniforms that are referenced in some variants but not others. If you were to declare a uniform texture2D depth_buffer; that is unused, the WGSL translation is always texture_2d. However, in shaders that DO reference depth_buffer, you likely get a texture_depth_2d. This may cause conflicts in your bind group layouts or errors when creating bind groups. This patch prunes samplers and textures that are unused.

Tests

Test spirv-val Naga Tint
pruneunuseddref.frag
pruneunuseddref_nested.frag
pruneunuseddref_storage.frag

Additional Notes

  • Does not prune combined image samplers
  • Does not prune storage textures since they are separate category of texture

Library Usage

Add this to your Cargo.toml:

spirv-webgpu-transform = "0.1"

I recommend having a look at src/bin/spv_webgpu_transform.rs.

let spv = spirv_webgpu_transform::u8_slice_to_u32_vec(&spv_bytes);

// Tells you which bindings need to be corrected.
let mut out_correction_map = None;

// Using `combimgsampsplitter` as an example.
spirv_webgpu_transform::combimgsampsplitter(&spv, &mut out_correction_map).unwrap()

let out_spv_bytes = spirv_webgpu_transform::u32_slice_to_u8_vec(&out_spv);

CLI Usage

# Using the `combimg` operation as an example.

cargo install spirv-webgpu-transform
spv_webgpu_transform combimg in.spv out.spv
# or
git clone https://github.com/davnotdev/spirv-webgpu-transform
cd spirv-webgpu-transform
cargo r -- combimg in.spv out.spv

FFI Usage

See ffi/bin/spv_webgpu_transform.c


Enjoy!

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors