This patch adjusts tools/fuzzing/ in such a way that the relevant parts can be
reused in the JS engine. Changes in detail include:
* Various JS_STANDALONE checks to exclude parts that cannot be included in
those builds.
* Turn LibFuzzerRegistry and LibFuzzerRunner into generic FuzzerRegistry and
FuzzerRunner classes and use them for AFL as well. Previously, AFL was
piggy-backing on gtests which was kind of an ugly solution anyway (besides
that it can't work in JS). Now more code like registry and harness is
shared between the two and they follow almost the same call paths and entry
points. AFL macros in FuzzingInterface have been rewritten accordingly.
This also required name changes in various places. Furthermore, this unifies
the way, the fuzzing target is selected, using the FUZZER environment
variable rather than LIBFUZZER (using LIBFUZZER in browser builds will give
you a deprecation warning because I know some people are using this already
and need time to switch). Previously, AFL target had to be selected using
GTEST_FILTER, so this is also much better now.
* I had to split up FuzzingInterface* such that the STREAM parts are in a
separate set of files FuzzingInterfaceStream* because they use nsStringStream
which is not allowed to be included into the JS engine even in a full browser
build (error: "Using XPCOM strings is limited to code linked into libxul.").
I also had to pull FuzzingInterface.cpp (the RAW part only) into the header
and make it static because otherwise, would have to make not only separate
files but also separate libraries to statically link to the JS engine, which
seemed overkill for a single small function. The streaming equivalent of the
function is still in a cpp file.
* LibFuzzerRegister functions are now unique by appending the module name to
avoid redefinition errors.
MozReview-Commit-ID: 44zWCdglnHr
This patch adjusts tools/fuzzing/ in such a way that the relevant parts can be
reused in the JS engine. Changes in detail include:
* Various JS_STANDALONE checks to exclude parts that cannot be included in
those builds.
* Turn LibFuzzerRegistry and LibFuzzerRunner into generic FuzzerRegistry and
FuzzerRunner classes and use them for AFL as well. Previously, AFL was
piggy-backing on gtests which was kind of an ugly solution anyway (besides
that it can't work in JS). Now more code like registry and harness is
shared between the two and they follow almost the same call paths and entry
points. AFL macros in FuzzingInterface have been rewritten accordingly.
This also required name changes in various places. Furthermore, this unifies
the way, the fuzzing target is selected, using the FUZZER environment
variable rather than LIBFUZZER (using LIBFUZZER in browser builds will give
you a deprecation warning because I know some people are using this already
and need time to switch). Previously, AFL target had to be selected using
GTEST_FILTER, so this is also much better now.
* I had to split up FuzzingInterface* such that the STREAM parts are in a
separate set of files FuzzingInterfaceStream* because they use nsStringStream
which is not allowed to be included into the JS engine even in a full browser
build (error: "Using XPCOM strings is limited to code linked into libxul.").
I also had to pull FuzzingInterface.cpp (the RAW part only) into the header
and make it static because otherwise, would have to make not only separate
files but also separate libraries to statically link to the JS engine, which
seemed overkill for a single small function. The streaming equivalent of the
function is still in a cpp file.
* LibFuzzerRegister functions are now unique by appending the module name to
avoid redefinition errors.
MozReview-Commit-ID: 44zWCdglnHr
Suppress these warnings intead of fixing them because libfuzzer is a third-party llvm library:
tools/fuzzing/libfuzzer/FuzzerDriver.cpp:450:10: warning: 'return' will never be executed [-Wunreachable-code-return]
tools/fuzzing/libfuzzer/FuzzerDriver.cpp:663:12: warning: 'return' will never be executed [-Wunreachable-code-return]
MozReview-Commit-ID: 9mWEuc5wCn9
tools/fuzzing/libfuzzer/harness/LibFuzzerTestHarness.h:118:10: warning: unused member function 'failed' [-Wunused-member-function]
MozReview-Commit-ID: 7hsvvnnolcl
Going further from the previous changes, all libfuzzer_main really does
is call the init function, and then proceed to call the fuzzer driver
with the testing function.
So instead of calling that function for it to do all that, the
LibFuzzerRunner can just call the init function itself, and then
call the fuzzer driver with the testing function.
The LibFuzzerRunner code lives in libxul. It's unnecessary complications
to have it call back a function in the firefox executable just so that
it calls another function that is in libxul. Passing the init and
testing functions to the libfuzzer_main function allows to just bypass
that roundtrip, simplifying the setup.
The function given to XRE_LibFuzzerSetMain is called from somewhere that
does have access to argc/argv already, so we can avoid passing them
to XRE_LibFuzzerSetMain.
This actually might fix subtle issues with argc/argv not really matching
reality when calling the LibFuzzerMain function in the current code:
some arguments are handled before the call, and both argc and argv are
modified from within XRE_main, but the values stored for the
LibFuzzerMain call still are the original ones.
Argv being a pointer, and it not being reallocated, the value stored for
the LibFuzzerMain call points to the changed one, but argc, being an
integer, is not modified accordingly.
In fact, it's actually worse, because while the Gecko code doesn't
reallocate argv, gtk_main might. So if some GTK flag is passed on the
command line, there's also a possibility that the LibFuzzerMain function
will do a use-after-free.
So all in all, it's just better to use the set of modified argc/argv
from XRE_main instead of storing them from main().
