This change vendors `wgpu` library in-tree and hooks up the initialization bits. It implements adapter and device initialization and adds a simple test.
Current status:
- [x] Architecture
- [x] figure out the IPC story
- [ ] move wgpu crates into a dedicated folder (let's follow up with this)
- [x] Review
- [x] WebIDL changes by DOM peers
- [x] Linux
- [x] avoid depending on spirv_cross - https://github.com/gfx-rs/wgpu/pull/371
- [x] macOS
- [x] due to cross-compiling shaders - https://github.com/gfx-rs/gfx/pull/3047
- [x] need the dependency update
- [x] stop using gcc - https://github.com/SSheldon/rust-objc-exception/pull/5
- [x] unexpected SSL header collision - https://bugzilla.mozilla.org/show_bug.cgi?id=1592398
- [x] undefined Metal symbols
- [x] Windows
- [x] due to "ipc-channel" not supporting Windows yet - https://github.com/servo/ipc-channel/pull/233~~
- [x] due to some exceptional stuff - https://github.com/grovesNL/spirv_cross/issues/121
- [x] undefined symbol: `D3D12CreateDevice`
- [x] d3d12.dll is not found, dxgi1_4 doesn't present
- [x] d3d11.dll and dxgi.dll need to be explicitly loaded on win32 mingw - https://github.com/gfx-rs/gfx/pull/3076
- [x] libbacktrace fails to link on win32 mingw
- [x] Android
- [x] spirv-cross fails to build - https://github.com/KhronosGroup/SPIRV-Cross/pull/1193
Update-1:
We decided to go with IPDL mechanism instead of Rust based ipc-channel (or any alternatives), which unblocks Windows build.
Update-2:
It appears that WebGPUThreading isn't needed any more as the child thread (and its event loop) is now managed by IPDL infrastructure. This PR removes it 🎉 .
Update-3:
InstanceProvider is also removed.
Update-4:
All set, the try is green, waiting for dependent changes to go in.
Differential Revision: https://phabricator.services.mozilla.com/D49458
This change vendors `wgpu` library in-tree and hooks up the initialization bits. It implements adapter and device initialization and adds a simple test.
Current status:
- [x] Architecture
- [x] figure out the IPC story
- [ ] move wgpu crates into a dedicated folder (let's follow up with this)
- [x] Review
- [x] WebIDL changes by DOM peers
- [x] Linux
- [x] avoid depending on spirv_cross - https://github.com/gfx-rs/wgpu/pull/371
- [x] macOS
- [x] due to cross-compiling shaders - https://github.com/gfx-rs/gfx/pull/3047
- [x] need the dependency update
- [x] stop using gcc - https://github.com/SSheldon/rust-objc-exception/pull/5
- [x] unexpected SSL header collision - https://bugzilla.mozilla.org/show_bug.cgi?id=1592398
- [x] undefined Metal symbols
- [x] Windows
- [x] due to "ipc-channel" not supporting Windows yet - https://github.com/servo/ipc-channel/pull/233~~
- [x] due to some exceptional stuff - https://github.com/grovesNL/spirv_cross/issues/121
- [x] undefined symbol: `D3D12CreateDevice`
- [x] d3d12.dll is not found, dxgi1_4 doesn't present
- [x] d3d11.dll and dxgi.dll need to be explicitly loaded on win32 mingw - https://github.com/gfx-rs/gfx/pull/3076
- [x] libbacktrace fails to link on win32 mingw
- [x] Android
- [x] spirv-cross fails to build - https://github.com/KhronosGroup/SPIRV-Cross/pull/1193
Update-1:
We decided to go with IPDL mechanism instead of Rust based ipc-channel (or any alternatives), which unblocks Windows build.
Update-2:
It appears that WebGPUThreading isn't needed any more as the child thread (and its event loop) is now managed by IPDL infrastructure. This PR removes it 🎉 .
Update-3:
InstanceProvider is also removed.
Update-4:
All set, the try is green, waiting for dependent changes to go in.
Differential Revision: https://phabricator.services.mozilla.com/D49458
Note: This changeset does not yet make it possible to propagate the
simd-accel feature to encoding_rs in standalone SpiderMonkey builds.
Differential Revision: https://phabricator.services.mozilla.com/D41355
This adds a mdns_service to mtransport to handle responding to mDNS queries.
All hostnames will be generated from UUIDs, so the responder assumes that it
is the only responder for a hostname which is registered with it. Because of
this, the responder does not first make a DNS query itself to see if any other
responder is handling a hostname, and does not wait a random amount of time
before replying, both of which are required by the specification to avoid
collisions with other responders.
Differential Revision: https://phabricator.services.mozilla.com/D38489
This adds a mdns_service to mtransport to handle responding to mDNS queries.
All hostnames will be generated from UUIDs, so the responder assumes that it
is the only responder for a hostname which is registered with it. Because of
this, the responder does not first make a DNS query itself to see if any other
responder is handling a hostname, and does not wait a random amount of time
before replying, both of which are required by the specification to avoid
collisions with other responders.
Differential Revision: https://phabricator.services.mozilla.com/D38489
Also remove most C++-side optimizations for avoiding calls to Rust
for short strings now that we have LTO between C++ and Rust. Since
LTO still leaves the overhead of one function call layer, inlined
function call avoidance optimization is left in place in the
IsUTF8 and in the 8-bit IsASCII cases for which perfherder flags
the difference as significant for the length 15.
Differential Revision: https://phabricator.services.mozilla.com/D40999
This patch introduces a new Rust crate called `static_prefs`.
It also changes generate_static_pref_list.py to generate two new files.
- StaticPrefsCGetters.cpp: contains C getters, which are just wrappers around
the C++ getters. This is included into Preferences.cpp.
- static_prefs.rs: contains declarations for the C getters, plus the `pref!`
macro which provides nice syntax for calling the C getters. This is included
into static_prefs/src/lib.rs.
The new code is only generated for prefs marked with the new `rust` field in
the YAML. It's opt-in because there's no point generating additional code for
900+ static prefs when only about 20 are currently used from Rust.
This patch only marks a single pref (`browser.display.document_color_use`) with
`rust: true`. That pref isn't accessed from Rust code in this patch, but it's
necessary because the generated Rust code is invalid if there are zero
Rust-accessed prefs. (The next patch will access that pref and others from Rust
code.
Differential Revision: https://phabricator.services.mozilla.com/D40791
This commit introduces a Rust XPCOM component,
`mozISyncedBookmarksMerger`, that wraps the Dogear crate for
merging and applying synced bookmarks.
How this works: `SyncedBookmarksMirror.jsm` manages opening
the connection, initializing the schema, and writing incoming
items into the mirror database. The new `mozISyncedBookmarksMerger`
holds a handle to the same connection. When JS code calls
`mozISyncedBookmarksMerger::apply`, the merger builds local and
remote trees, produces a merged tree, applies the tree back to Places,
and stages outgoing items for upload in a temp table, all on the
storage thread. It then calls back in to JS, which inflates Sync
records for outgoing items, notifies Places observers, and cleans up.
Since Dogear has a more robust merging algorithm that attempts to fix
up invalid trees, `test_bookmark_corruption.js` intentionally fails.
This is fixed in the next commit, which changes the merger to handle
invalid structure.
Differential Revision: https://phabricator.services.mozilla.com/D20076
This commit wraps just enough of the mozStorage API to support the
bookmarks mirror. It's not complete: for example, there's no way
to open, clone, or close a connection, because the mirror handles
that from JS. The wrapper also omits shutdown blocking and retrying on
`SQLITE_BUSY`.
This commit also changes the behavior of sync and async mozStorage
connections. Async (`mozIStorageAsyncConnection`) methods may be called
from any thread on any connection. Sync (`mozIStorageConnection`)
methods may be called from any thread on a sync connection, and from
background threads on an async connection. All connections now QI
to `mozIStorageConnection`, but attempting to call a sync method on
an async connection from the main thread throws.
Finally, this commit exposes an `OpenedConnection::unsafeRawConnection`
getter in Sqlite.jsm, for JS code to access the underlying connection.
Differential Revision: https://phabricator.services.mozilla.com/D20073
Until rust 1.28, there was no stable way to change the allocator used by
rust code. In bug 1280578, we hooked HeadAlloc/HeapFree/HeapRealloc,
that the default rust system allocator uses. On other platforms, rust
code just ended up using malloc/free/realloc like everything else.
As of rust 1.28, though, it is now possible to use the GlobalAlloc trait
and the #[global_allocator] attribute to set an allocator. On Windows,
this can allow us to hook mozjemalloc directly, rather than using an
indirection through HeapAlloc/etc. (which require an extra call to
GetProcessHeap), so let's do this. On other platforms, this just ends up
doing the same thing as the default rust system allocator (except for
the memalign limit on 32-bits platforms).
We still need the HeapAlloc/etc. hooks for some C++ code using it, though.
Another benefit is that the HeapAlloc GlobalAlloc implementation needs
to do its own memalign, which it does by overallocating and aligning
manually. We obviously don't need to do this when we using
memalign/_aligned_malloc directly.
Differential Revision: https://phabricator.services.mozilla.com/D14820
This removes the code added in bug 1458161, because the old versions of
rust that required it can't be used to build Gecko anymore. The variant
for newer versions of rust stays.
Differential Revision: https://phabricator.services.mozilla.com/D14528
The current rust panic hook keeps a string for the crash reporter, and
goes on calling the default rust panic hook, which prints out a crash
stack... when RUST_BOOTSTRAP is set *and* when that works. Notably, on
both mac and Windows, it only really works for local builds, but fails
for debug builds from automation, although on automation itself, we also
do stackwalk from crash minidumps, which alleviates the problem.
Artifact debug builds are affected, though.
More importantly, C++ calls to e.g. MOZ_CRASH have a similar but
different behavior, in that they dump a stack trace on debug builds, by
default (with exceptions, see below for one). The format of those stack
traces is understood by the various fix*stack*py scripts under
tools/rb/, that are used by the various test harnesses both on
automation and locally.
Additionally, the current rust panic hook, as it calls the default rust
panic hook, ends up calling abort() on non-Windows platforms, which ends
up being verbosely redirected to mozalloc_abort per
https://dxr.mozilla.org/mozilla-central/rev/237e4c0633fda8e227b2ab3ab57e417c980a2811/memory/mozalloc/mozalloc_abort.cpp#79
which then calls MOZ_CRASH. Theoretically, /that/ would also print a
stack trace, but doesn't because currently the stack trace printing code
lives in libxul, and MOZ_CRASH only calls it when compiled from
libxul-code, which mozalloc_abort is not part of.
With this change, we make the rust panic handler call back into
MOZ_CRASH directly. This has multiple advantages:
- This is more consistent cross-platforms (Windows is not special
anymore).
- This is more consistent between C++ and rust (stack traces all look
the same, and can all be post-processed by fix*stack*py if need be)
- This is more consistent in behavior, where debug builds will show
those stack traces without caring about environment variables.
- It demangles C++ symbols in rust-initiated stack traces (for some
reason that didn't happen with the rust panic handler)
A few downsides:
- the loss of demangling for some rust symbols.
- the loss of addresses in the stacks, although they're not entirely
useful
- extra empty lines.
The first should be fixable later one. The latter two are arguably
something that should be consistent across C++ and rust, and should be
changed if necessary, independently of this patch.
Depends on D11719
Depends on D11719
Differential Revision: https://phabricator.services.mozilla.com/D11720
The current rust panic hook keeps a string for the crash reporter, and
goes on calling the default rust panic hook, which prints out a crash
stack... when RUST_BOOTSTRAP is set *and* when that works. Notably, on
both mac and Windows, it only really works for local builds, but fails
for debug builds from automation, although on automation itself, we also
do stackwalk from crash minidumps, which alleviates the problem.
Artifact debug builds are affected, though.
More importantly, C++ calls to e.g. MOZ_CRASH have a similar but
different behavior, in that they dump a stack trace on debug builds, by
default (with exceptions, see below for one). The format of those stack
traces is understood by the various fix*stack*py scripts under
tools/rb/, that are used by the various test harnesses both on
automation and locally.
Additionally, the current rust panic hook, as it calls the default rust
panic hook, ends up calling abort() on non-Windows platforms, which ends
up being verbosely redirected to mozalloc_abort per
https://dxr.mozilla.org/mozilla-central/rev/237e4c0633fda8e227b2ab3ab57e417c980a2811/memory/mozalloc/mozalloc_abort.cpp#79
which then calls MOZ_CRASH. Theoretically, /that/ would also print a
stack trace, but doesn't because currently the stack trace printing code
lives in libxul, and MOZ_CRASH only calls it when compiled from
libxul-code, which mozalloc_abort is not part of.
With this change, we make the rust panic handler call back into
MOZ_CRASH directly. This has multiple advantages:
- This is more consistent cross-platforms (Windows is not special
anymore).
- This is more consistent between C++ and rust (stack traces all look
the same, and can all be post-processed by fix*stack*py if need be)
- This is more consistent in behavior, where debug builds will show
those stack traces without caring about environment variables.
- It demangles C++ symbols in rust-initiated stack traces (for some
reason that didn't happen with the rust panic handler)
A few downsides:
- the loss of demangling for some rust symbols.
- the loss of addresses in the stacks, although they're not entirely
useful
- extra empty lines.
The first should be fixable later one. The latter two are arguably
something that should be consistent across C++ and rust, and should be
changed if necessary, independently of this patch.
Depends on D11719
Differential Revision: https://phabricator.services.mozilla.com/D11720
The current rust panic hook keeps a string for the crash reporter, and
goes on calling the default rust panic hook, which prints out a crash
stack... when RUST_BOOTSTRAP is set *and* when that works. Notably, on
both mac and Windows, it only really works for local builds, but fails
for debug builds from automation, although on automation itself, we also
do stackwalk from crash minidumps, which alleviates the problem.
Artifact debug builds are affected, though.
More importantly, C++ calls to e.g. MOZ_CRASH have a similar but
different behavior, in that they dump a stack trace on debug builds, by
default (with exceptions, see below for one). The format of those stack
traces is understood by the various fix*stack*py scripts under
tools/rb/, that are used by the various test harnesses both on
automation and locally.
Additionally, the current rust panic hook, as it calls the default rust
panic hook, ends up calling abort() on non-Windows platforms, which ends
up being verbosely redirected to mozalloc_abort per
https://dxr.mozilla.org/mozilla-central/rev/237e4c0633fda8e227b2ab3ab57e417c980a2811/memory/mozalloc/mozalloc_abort.cpp#79
which then calls MOZ_CRASH. Theoretically, /that/ would also print a
stack trace, but doesn't because currently the stack trace printing code
lives in libxul, and MOZ_CRASH only calls it when compiled from
libxul-code, which mozalloc_abort is not part of.
With this change, we make the rust panic handler call back into
MOZ_CRASH directly. This has multiple advantages:
- This is more consistent cross-platforms (Windows is not special
anymore).
- This is more consistent between C++ and rust (stack traces all look
the same, and can all be post-processed by fix*stack*py if need be)
- This is more consistent in behavior, where debug builds will show
those stack traces without caring about environment variables.
- It demangles C++ symbols in rust-initiated stack traces (for some
reason that didn't happen with the rust panic handler)
A few downsides:
- the loss of demangling for some rust symbols.
- the loss of addresses in the stacks, although they're not entirely
useful
- extra empty lines.
The first should be fixable later one. The latter two are arguably
something that should be consistent across C++ and rust, and should be
changed if necessary, independently of this patch.
Depends on D11719
Differential Revision: https://phabricator.services.mozilla.com/D11720
