Use the SHT_RELR format which significantly improves the size reduction
from elfhack:
total size of .rel.* + .elfhack.* sections
x86 x86_64 android-arm
plain 3532904 10739544 3488888
current-elfhack 1085815 1155994 1042048
relr-elfhack 130219 193552 113840
Differential Revision: https://phabricator.services.mozilla.com/D134756
Elfhack currently can't deal with files larger than 4GiB because it
translates all ELF data structures to the 32-bits variant, even for
64-bits ELF files. So if the original file has e.g. sections that start
after the 4GiB boundary, they can't be represented in memory.
Practically speaking, this is not causing problems, but has prevented a
working elfhack for aarch64 because e.g. some relocation types don't fit
in the 32-bits ELF representation.
Differential Revision: https://phabricator.services.mozilla.com/D134745
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This only solves the easy half of the problem outlined in the bug,
leaving the other half for later.
iostream::tellg() actually returns streampos, which is able to support
files larger than 4GiB with libstdc++, but converting to an int
obviously truncated that, as well as transformed values between 2GiB and
4GiB into invalid negative numbers.
iostream::seekg() also takes a streampos, so storing the streampos as-is
is enough to address the problem with tellg()/seekg() sequences.
The other half of the problem involves elfhack converting 64-bits ELF
headers to 32-bits headers internally, which requires deeper changes.
This change however, is enough to support files up to 4GiB, which is
already a good first step.
Differential Revision: https://phabricator.services.mozilla.com/D94252
The `clobber` targets are superseded by `mach clobber`, so we don't need them for any reason. The `clean` target is meant to get you to a post-`configure` state, but it doesn't really work, and if it's necessary for you to be in that state for some reason you can just clobber and re-`configure`, so it doesn't seem worth it to get it working again. Instead, delete all of them. Also delete `everything` which is not useful when `clobber` doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D93514
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
We were never adjusting `last` in this loop, so we were computing the
wrong addresses for all sections beyond the first. Which in turn meant
that we would compute the wrong size for the section data we needed to
allocate.
Differential Revision: https://phabricator.services.mozilla.com/D28195
We were allocating ElfSection's data with `new[]` and modifying it with
`realloc` in some places, which causes allocator mismatches.
Consistently manage the data with `malloc`, `realloc`, and `free` instead.
Differential Revision: https://phabricator.services.mozilla.com/D27327
In bug 1470701, we added a dummy global variable so that it ends up in
the bss section, making it large enough for two pointers. Unfortunately,
in some cases, the symbol is eliminated by the linker because it is
unused. So we try to ensure it stays there.
Differential Revision: https://phabricator.services.mozilla.com/D11257
This is some sort of followup to bug 1423813, providing a minimalistic
way to undo elfhack when the elfhack sections are in separate segments,
which has been the case since bug 1385783 but didn't cause problems
on Android builds until bug 1423822.
Depends on D9622
Differential Revision: https://phabricator.services.mozilla.com/D9623
This effectively backs out bug 822584, which worked around a similar
problem to what we are facing with Android xpcshell, being that the
crash reporter doesn't handle the address space "fragmentation" induced
by elfhack. The work around worked, at the expense of some added
complexity.
It was used for B2G only, and has effectively been unused since B2G was
retired.
Differential Revision: https://phabricator.services.mozilla.com/D9622
When checking whether the new relocations sizes are going to be a win, we
need to account for the fact that there are non-elfhacked relocation
remaining.
Differential Revision: https://phabricator.services.mozilla.com/D5837
In some rare cases, it is possible for one of the eh_frame sections'
original address to be larger than the address of the injected code
section, which is added before the first executable section. Namely,
this happens in the rare case where that eh_frame section is smaller
than the injected code section, and is adjacent to the first executable
section. We obviously want to move the eh_frame sections in that case,
since one of them is in the way.
Bug 1423822 moved the injected code section before the .text section.
When linking with lld, the text section is usually page aligned, and
starting a PT_LOAD. We inject code at the beginning of the PT_LOAD,
which means the PT_LOAD is going to be extended at least a page
downwards. And it means the preceding PT_LOAD can't finish in that same
page, so the overhead of the injected code is needs to account for the
page alignment.
If the .eh_frame_hdr and .eh_frame sections are not between the elfhack
relocation and elfhack code sections, it's not going to change anything
to try to move it, so don't even try.
While here, adjust the adjacency test to error out when the section name
doesn't match, and account for the fact that the eh_frame_hdr section
might appear after eh_frame.
