Repository navigation
List of issues to discuss with TC39, WASM, JS engine teams #747
Description
-
mmap'ing ArrayBuffers into WASM heap. - read-only ArrayBuffers and write-only(?) ArrayBuffers (that work with
mmap), for buffer mapping - ArrayBuffers (non-Shared, same thread) that point at non-disjoint ranges of the same backing data
- ArrayBuffers whose base addresses aren't maximally-aligned (e.g. Float64Array not valid at offset 0, valid at offset 4)
- No one has asked for this yet, probably not very important
- "bitflags" equivalent functionality (e.g. Java-style EnumSet 🥺)
-
Ability to make ArrayBuffers non-transferrable.It turns out this already exists as[[ArrayBufferDetachKey]] -
SharedObjectTablea bit likeSharedArrayBufferbut instead of containing bytes, they contain references to shallow-cloneable objects ([Serializable]IDL interface objects). This should allow for example forGPUTextures created on one worker to be instantly visible to other workers.- Or (somewhat less ideal) synchronous
receiveMessage()onMessagePort. (WASM multi-threading ability #476) - Place to draft a proposal: https://hackmd.io/@webgpu/S1f56dN8s
- Or (somewhat less ideal) synchronous
Please add more issues on this thread.
Reacted by Dzmitry Malyshau, James Baicoianu, Linz, ExE Boss, Erik, Ankesh Bharti, Ino Itsuca, Suraj, Lv Yitian, munrocket and 3 more
Activity
kainino0x commentedon May 4, 2020
Non-transferable ArrayBuffers (mapped ArrayBuffers shouldn't be transferable to other threads). (We can do this without JS changes by faking transfers, by making them actually copies instead.)Kangz commentedon May 5, 2020
SharedObjectTablea bit likeSharedArrayBufferbut instead of containing bytes, they contain references to shallow-cloneable objects ([Serializable]IDL interface objects). This should allow for example forGPUTextures created on one worker to be instantly visible to other workers.kainino0x commentedon May 5, 2020
receiveMessage()onMessagePort.#476
sandersdan commentedon May 7, 2020
WebCodecs is also interested in read-only ArrayBuffers, for providing access to YUV planes in CPU memory that remain owned by decoders. Is there any previous work on this request?
kainino0x commentedon May 7, 2020
A discussion on the internal-gpu mailing list can be found here:
https://lists.w3.org/Archives/Member/internal-gpu/2020May/0001.html
in which Ken discusses Java's past experience with read-only ArrayBuffers.
I just forwarded it to you, @sandersdan.
Other discussions I just found:
kainino0x commentedon Jun 11, 2020
kainino0x commentedon Jun 30, 2020
Kangz commentedon Jul 24, 2020
kainino0x commentedon Jul 24, 2020
kainino0x commentedon May 4, 2021
It turns out that this (or synchronous receive-message) would also be extremely useful to WebAssembly for sending modules around for dynamic linking.
Kangz commentedon Aug 27, 2021
Another important thing to discuss with TC39 is adding a concept of non-transferrable
ArrayBuffer. Right now this group agreed to make the result ofGPUBuffer.getMappedRangenon-transferrable so we can detach them onGPUBuffer.unmapbut the JS spec doesn't have any way to express this at the moment.Contrary to the other TC39 topics in this issue, this one is important to figure out before WebGPU v1 as it is an important part of the WebGPU semantics (and not a future improvement).
Kangz commentedon Aug 27, 2021
This might also require a change to the HTML spec for structuredSerializeWithTransfer (an internal postMessage algorithm)
kainino0x commentedon Aug 27, 2021
As mentioned above:
We might be able to make this work entirely within our spec instead of having to go change the behavior of ArrayBuffer. In more detail:
If you tried to transfer a getMappedRange-created ArrayBuffer, it would appear to work entirely normally - old ArrayBuffer would get detached - except the underlying data would get copied into a new, run-of-the-mill ArrayBuffer, which means that any writes to the new ArrayBuffer would not appear in the GPUBuffer upon unmap().
kainino0x commentedon Aug 27, 2021
Filed new issue #2072.
pythonesque commentedon Aug 27, 2021
AFAIK, what you're describing is the behavior for non-transferable objects, though? Minus the fake detach, that is. I guess the point is that this wouldn't be an observable behavior change compared to the usual behavior (except in the context of WebGPU memory mapping, which isn't specified by TC39)?
11 remaining items
MikhailGorobets commentedon Aug 19, 2024
Any updates?
kainino0x commentedon Aug 27, 2024
@MikhailGorobets Not at this time. This issue tracks a bunch of things, what in particular are you interested in?
MikhailGorobets commentedon Aug 28, 2024
@kainino0x
We are using Emscripten to compile C++ into WASM. Currently, we are facing an issue when using
WGPUDevicein a multi-threaded environment (https://emscripten.org/docs/porting/pthreads.html). For example, when creating aWGPUSamplerin a thread different from the one where theWGPUDevicewas created, we receive an invalid pointer. The problem is descripted heregpuweb/explainer/index.bs
Line 935 in fb97a57
kainino0x commentedon Aug 28, 2024
Thanks, good to know. There was some progress in investigating this direction but we have not had resources to work on it in a while. It will also of course require WebGPU to have multithreading in the first place, which is #354.
In the meantime, there's also been a little bit of investigation into Emscripten having a way to proxy all WebGPU calls to a single thread so that neither of those will be necessary. This won't perform as well, but unfortunately we haven't prototyped it yet. See emscripten-core/emscripten#19645
erights commentedon Dec 23, 2024
Please see https://github.com/tc39/proposal-immutable-arraybuffer which proposes Immutable ArrayBuffers, is at Stage 2 of the tc39 process, has enthusiastic support from some implementations, and not (yet?) any serious objections. How applicable would this be to the concerns expressed above?
kainino0x commentedon Jan 3, 2025
@erights Thank you for letting us know about that! We are indeed interested, but right now the proposal is not compatible with WebGPU. I've filed tc39/proposal-immutable-arraybuffer#25 where we can discuss that in more depth.