This is a work in progress research project trying to evaluate how a declarative/reactive graphics engine would work.
It's basically following the WebGPU Fundamentals guide, while applying the preachings of Steven Wittens at acko.net (the site with the cool banner).
See it live:
git clone [email protected]:mendes5/web-gpu-react.git
cd web-gpu-react
cp .env.example .env
nvm use
yarn
yarn dev
We have a useGPU
hook, and use it to basically to everything GPU related.
Example:
const instanceCount = useRefTrap(10);
const { updateBuffers } = useGPU(({ gpu, frame, action }) => {
// Setup code
const pipeline = gpu.createRenderPipeline({ ... });
// Rendering code uses `frame`
const main = frame.main(({ encoder }) => {
...
encoder.setPipeline(shader);
pass.draw(instanceCount.current)
...
}, [pipeline]);
// Non-rendering code uses action
const updateBuffers = action(({ invalidates, queue }) => {
invalidates(main);
queue.copyExternalImageToTexture(...);
});
return { updateBuffers };
}, []);
return <button onClick={updateBuffers} />
The idea is not to go to the lengths of usegpu.live and create helpers for lightning, animation, loading GLTF etc... I only want a single efficient, declarative, reactive way to manage GPU resources, the most low level building block. With that done we can do everything else in other packages.
By using the useGPU
a lot stuff is already done for you:
- Resource management:
- Sharing and caching of resources that can be global like samplers, and pipelines, kinda like react query, but for GPU stuff.
- Automatic setup/teardown of local resources like buffers and textures.
- Device management
- If the device is lost, then everything in the whole application will be re-created automatically.
- Multiple devices.
- Lazy rendering
- Only rerender if data changes
- Rerender if the screen gets resized
useRefTrap
makes the frames that depends on it re-render on mutations, so nouseState
is required to update stuff.
- Frame scheduling
- Actions always ran before render, but can also wait the render to finish to download GPU buffers.
- Runs frame callbacks at the desired frame rate if necessary, or only on data changes.
When your application is running at 60FPS no react code is actually running, all React does is wire up a bunch of resources on one render, and then if everything is memoized correctly, it should only re-render when the GPU resources needs to be created/destroyed.
It can become quite heavy if you code without care, but paths to optimize such code do exist.
That way you probably could create an FPS game, and react could be used as an kind of GPUResourceManager
class. But my idea is to put everything else in react too, like using JSX to build scene graphs, DSLs, visibility toggles etc... Thay way you could easily create data driven apps using the GPU.
- Support SSR (we aren't using the GPU there, but the
GPUDevice
components, and all other components should not block rendering of HTML on the server) - Support react strict mode
- Cache GPU calls so two
createBuffers
calls with the same parameters on the same component only create one buffer. - Support rendering without canvas.
- Create a frame scheduler
- Rerender on canvas resize
- Make the frame scheduler support lazy rendering
- Make the frame scheduler render in tree order.
- Make the frame scheduler support custom update timing.
- Make react
useRef
rerender lazy frames - Support async
useGPU
to allow forcreateRenderPipelineAsync
- Add "actions" to run imperative code on demand, before rendering
- Hide away the
device
entirely - Add a resource debugger
- Support multiple command encoders
- Improve async action support
- Add
memo
function, to cache CPU resource creation insideuseGPU
(i guess you can do it outside withuseMemo
but will make porting existing code easier) - Add a shader linker (probably use from
@use-gpu/shader
) - Adds better way to
createBindGroup
- Adds better way to use the
device.queue
- Adds better way to render to the
screen
- Adds better way to render to use the default presentation format.