errors, etc.''):
We now ReportStatementTooLarge only if
- a jump offset overflows 32 bits, signed;
- there are 2**32 or more span dependencies in a script;
- a backpatch chain link is more than (2**30 - 1) bytecodes long;
- a source note's distance from the last note, or from script main entry
point, is > 0x7fffff bytes.
Narrative of the patch, by file:
- js.c
The js_SrcNoteName array of const char * is now a js_SrcNoteSpec array of
"specifiers", structs that include a const char *name member. Also, due to
span-dependent jumps at the ends of basic blocks where the decompiler knows
the basic block length, but not the jump format, we need an offset operand
for SRC_COND, SRC_IF_ELSE, and SRC_WHILE (to tell the distance from the
branch bytecode after the condition expression to the span-dependent jump).
- jsarena.[ch]
JS arenas are used mainly for last-in-first-out allocation with _en masse_
release to the malloc pool (or, optionally, to a private freelist). But
the code generator needs to allocate and grow (by doubling, to avoid O(n^2)
growth) allocations that hold bytecode, source notes, and span-dependency
records. This exception to LIFO allocation works by claiming an entire
arena from the pool and realloc'ing it, as soon as the allocation size
reaches the pool's default arena size. Call such an allocation a "large
single allocation".
This patch adds a new arena API, JS_ArenaFreeAllocation, which can be used
to free a large single allocation. If called with an allocation that's not
a large single allocation, it will nevertheless attempt to retract the arena
containing that allocation, if the allocation is last within its arena.
Thus JS_ArenaFreeAllocation adds a non-LIFO "free" special case to match the
non-LIFO "grow" special case already implemented under JS_ARENA_GROW for
large single allocations.
The code generator still benefits via this extension to arenas, over purely
manual malloc/realloc/free, by virtue of _en masse_ free (JS_ARENA_RELEASE
after code generation has completed, successfully or not).
To avoid searching for the previous arena, in order to update its next
member upon reallocation of the arena containing a large single allocation,
the oversized arena has a back-pointer to that next member stored (but not
as allocable space within the arena) in a (JSArena **) footer at its end.
- jscntxt.c
I've observed for many scripts that the bytes of source notes and bytecode
are of comparable lengths, but only now am I fixing the default arena size
for cx->notePool to match the size for cx->codePool (1024 instead of 256).
- jsemit.c
Span-dependent instructions in JS bytecode consist of the jump (JOF_JUMP)
and switch (JOF_LOOKUPSWITCH, JOF_TABLESWITCH) format opcodes, subdivided
into unconditional (gotos and gosubs), and conditional jumps or branches
(which pop a value, test it, and jump depending on its value). Most jumps
have just one immediate operand, a signed offset from the jump opcode's pc
to the target bytecode. The lookup and table switch opcodes may contain
many jump offsets.
This patch adds "X" counterparts to the opcodes/formats (X is suffixed, btw,
to prefer JSOP_ORX and thereby to avoid colliding on the JSOP_XOR name for
the extended form of the JSOP_OR branch opcode). The unextended or short
formats have 16-bit signed immediate offset operands, the extended or long
formats have 32-bit signed immediates. The span-dependency problem consists
of selecting as few long instructions as possible, or about as few -- since
jumps can span other jumps, extending one jump may cause another to need to
be extended.
Most JS scripts are short, so need no extended jumps. We optimize for this
case by generating short jumps until we know a long jump is needed. After
that point, we keep generating short jumps, but each jump's 16-bit immediate
offset operand is actually an unsigned index into cg->spanDeps, an array of
JSSpanDep structs. Each struct tells the top offset in the script of the
opcode, the "before" offset of the jump (which will be the same as top for
simplex jumps, but which will index further into the bytecode array for a
non-initial jump offset in a lookup or table switch), the after "offset"
adjusted during span-dependent instruction selection (initially the same
value as the "before" offset), and the jump target (more below).
Since we generate cg->spanDeps lazily, from within js_SetJumpOffset, we must
ensure that all bytecode generated so far can be inspected to discover where
the jump offset immediate operands lie within CG_CODE(cg). But the bonus is
that we generate span-dependency records sorted by their offsets, so we can
binary-search when trying to find a JSSpanDep for a given bytecode offset,
or the nearest JSSpanDep at or above a given pc.
To avoid limiting scripts to 64K jumps, if the cg->spanDeps index overflows
65534, we store SPANDEP_INDEX_HUGE in the jump's immediate operand. This
tells us that we need to binary-search for the cg->spanDeps entry by the
jump opcode's bytecode offset (sd->before).
Jump targets need to be maintained in a data structure that lets us look
up an already-known target by its address (jumps may have a common target),
and that also lets us update the addresses (script-relative, a.k.a. absolute
offsets) of targets that come after a jump target (for when a jump below
that target needs to be extended). We use an AVL tree, implemented using
recursion, but with some tricky optimizations to its height-balancing code
(see http://www.enteract.com/~bradapp/ftp/src/libs/C++/AvlTrees.html).
A final wrinkle: backpatch chains are linked by jump-to-jump offsets with
positive sign, even though they link "backward" (i.e., toward lower bytecode
address). We don't want to waste space and search time in the AVL tree for
such temporary backpatch deltas, so we use a single-bit wildcard scheme to
tag true JSJumpTarget pointers and encode untagged, signed (positive) deltas
in JSSpanDep.target pointers, depending on whether the JSSpanDep has a known
target, or is still awaiting backpatching.
Note that backpatch chains would present a problem for BuildSpanDepTable,
which inspects bytecode to build cg->spanDeps on demand, when the first
short jump offset overflows. To solve this temporary problem, we emit a
proxy bytecode (JSOP_BACKPATCH; JSOP_BACKPATCH_PUSH for jumps that push a
result on the interpreter's stack, namely JSOP_GOSUB; or JSOP_BACKPATCH_POP
for branch ops) whose nuses/ndefs counts help keep the stack balanced, but
whose opcode format distinguishes its backpatch delta immediate operand from
a normal jump offset.
The cg->spanDeps array and JSJumpTarget structs are allocated from the
cx->tempPool arena-pool. This created a LIFO vs. non-LIFO conflict: there
were two places under the TOK_SWITCH case in js_EmitTree that used tempPool
to allocate and release a chunk of memory, during whose lifetime JSSpanDep
and/or JSJumpTarget structs might also be allocated from tempPool -- the
ensuing release would prove disastrous. These bitmap and table temporaries
are now allocated from the malloc heap.
- jsinterp.c
Straightforward cloning and JUMP => JUMPX mutating of the jump and switch
format bytecode cases.
- jsobj.c
Silence warnings about %p used without (void *) casts.
- jsopcode.c
Massive and scary decompiler whackage to cope with extended jumps, using
source note offsets to help find jumps whose format (short or long) can't
be discovered from properties of prior instructions in the script.
One cute hack here: long || and && expressions are broken up to wrap before
the 80th column, with the operator at the end of each non-terminal line.
- jsopcode.h, jsopcode.tbl
The new extended jump opcodes, formats, and fundamental parameterization
macros. Also, more comments.
- jsparse.c
Random and probably only aesthetic fix to avoid decorating a foo[i]++ or
--foo[i] parse tree node with JSOP_SETCALL, wrongly (only foo(i)++ or
--foo(i), or the other post- or prefix form operator, should have such an
opcode decoration on its parse tree).
- jsscript.h
Random macro naming sanity: use trailing _ rather than leading _ for macro
local variables in order to avoid invading the standard C global namespace.
with JSRESOLVE_ASSIGNING, wrongly), plus a few miscellaneous bugfixes.
- Combine the JSStackFrame members constructing, special, overrides, and
reserved into a uint32 flags member.
- Separate JOF_ASSIGNING from the JOF_SET bytecode format flag, and impute
JSRESOLVE_ASSIGNING from the presence of JOF_ASSIGNING among the current
opcode's format flags. To handle the for-in loop opcodes, which do more
than simply assign -- in particular, they do property lookups whose resolve
hook outcalls should not be flagged with JSRESOLVE_ASSIGNING -- a new frame
flag, JSFRAME_ASSIGNING, has been added.
- Fix interpreter version selection to respect JS_SetVersion, whose effect on
cx->version is "sticky".
- Fix js_DecompileValueGenerator to deal with JSOP_ENUMELEM -- it never had,
as this testcase shows (it crashes without this patch):
version(120);
eval("function fe(s) { for (it[s] in this); }");
try { fe('rdonly'); } catch (e) { print(e); }
- Fix scope chain for nested functions at top-level (JSOP_DEFFUN), in a part of another statement (JSOP_CLOSURE), and unnamed in an expression (JSOP_ANONFUNOBJ) to match ECMA-262 13.2. My bad: fp->varobj was used up till now, instead of fp->scopeChain; we still *bind* the name of a statement-level (top or not) nested function in fp->varobj. This fixes bug 69559. (r=rogerl, sr=jband)
- Add an Intern command to the shell, for GC vs. intern'ed atom testing.
- All jsvals for which JSVAL_IS_GCTHING evaluates to true must contain tagged
pointers into the GC heap -- therefore jsapi.c's JS_DefineConstDoubles cannot
"cheat" by tagging addresses of static jsdoubles to avoid js_NewNumberValue.
- Finalization is now interleaved with the Sweep phase, to avoid allocating
memory for finalization records while sweeping. Instead, the JSRuntime holds a
preallocated JSGCThing vector (gcFinalVec) that the Sweep phase fills and
flushes via gc_finalize_phase, repeatedly.
This means that finalizers cannot allocate a new GC thing, an incompatible but
plausible change. js_AllocGCThing asserts and then checks whether it is called
while rt->gcLevel is non-zero, and fails the allocation attempt if so. But this
fixes bug 38942, where the old sweep-then-finalize with a sweep => malloc
dependency could lead to memory exhaustion.
- Instead of scanning whole stackPool arenas, which led to UMRs (bug 27924) and
sometimes to gross over-scanning that depended on the GC bounds-checking all
thing pointers against its heap, we scan exactly those stack slots in use:
- arguments reachable from fp->argv;
- variables reachable from fp->vars;
- operands now reachable from fp->spbase, bounded above by the lesser of
fp->sp or fp->spbase + fp->script->depth for an interpreted frame; if the
latter, fp->sp has advanced logically above the operand budget, in order to
call a native method, and all unused slots from fp->sp up to depth slots
above fp->spbase must be set to JSVAL_VOID;
- stack segments pushed when calling native methods, prefixed by JSStackHeader
structs and linked from cx->stackSegments through each header.
The stack segment headers help the GC avoid scanning unused portions of the
stack: the generating pc slots running depth slots below fp->spbase, and slots
at the end of an arena that aren't sufficient to satisfy a contiguous allocation
for more args, vars, or operands.
- Exact GC means the stack pointer must remain above live operands until the
interpreter is done with them, so jsinterp.c got heavily whacked. Instead of
POPs of various kinds followed by a PUSH for binary operators (e.g.), we use
FETCH and STORE macros that index by -1 and -2 from sp, and minimize adjustments
to sp. When sp is homed to fp->sp, this allows js_DecompileValueGenerator to
find the value reliably, and if possible its generating pc.
- Finally, the O(n**2) growth rate of gc_find_flags has been fixed, using the
scheme sketched in bug 49816 and documented in a new major comment in jsgc.c.
Briefly, by allocating flags and things from one arena, we can align things on
1024-byte "thing page" boundaries, and use JSGCPageInfo headers in each page to
find a given thing's flags in O(1) time.
/be
- Add JS1.5 getter/setter support in all its glory:
* getter function SN() {return ++x} at top-level or as a closure binds an SN
property getter than returns the incremented value of x. Likewise for
setter function SN(y) {return y = x}.
* getters and setters may be defined in an object literal:
o = {p getter:function() {return ++this.x},
p setter:function(y){return this.x = y},
x:42};
* getter= and setter= operators (compound tokens) may be used to bind getter
and setter properties dynamically:
o = new Object;
o.p getter= function() {return ++this.x};
o.p setter= function(y){return this.x = y};
o.x = 42;
Waldemar is concerned that this form will collide semantically with JS2, so
I am not committing to keeping it in JS1.5. I'd like to check my code in
ASAP so shaver can use it, and I'd also like to see this form get used (or
not) during Mozilla betas. Caveat emptor, and if you find this "dynamic"
or "imperative" form necessary and hard to substitute, please let me know.
If this proves important to users, then I think JS1.5 should keep it.
- Cleaned up property flags (in a binary-incompatible fashion -- who cares?) by
eliminating JSPROP_ASSIGNHACK and JSPROP_TINYIDHACK.
- Added JS_DONT_PRETTY_PRINT flag to be ORed with the indent argument to the
several JS_Decompile*() API calls. This avoids any newlines or identation in
the decompiled string.
- Improved and extended (for getter/setter non-reservation) scanner lookahead
by using a circular (power-of-2 sized) token buffer.
- Fix ECMA Edition 3 deviation where function f(){function g(){}} bound f.g by
mistake (it should arrange to make a closure named g in activations of f, but
it should not bind a property of function f).
- js_DecompileValueGenerator had rusted due to bytecode/source-note changes,
or maybe parts of it never worked right. Anyway, it now does not induce a
crashing underflow in the decompiler. As part of this fix, it now takes a
checkStack flag telling whether to look for the jsval v argument on the JS
stack. The calls from ImportProperty, js_SetProperty, and js_DeleteProperty
pass in v a jsval for the property id, which should not be sought after on
the stack (it might happen to be there due to o['p'] = 2, but we want to
decompile o["p"], not "p").
- js_DecompileValueGenerator would load a generating pc even if the value v
did not match the pc's corresponding stack item! Oops. This lead to less
than idea diagnostics.
- js_DecompileValueGenerator was also not mapping JSOP_TRAP to the real op at
a sufficiently early and univeral point in its control flow.
- Fix PopOff to assert and check for stack underflow in the decompiler, and
beef up PushOff too (it asserted, but did not check).
- js_ReportIsNotFunction now avoids JS_InternString by indexing directly into
cx->runtime->atomState.typeAtoms with the result of JS_TypeOfValue.
- Removed unnecessary local GC root reserved by non-zero trailing member of
obj_eval's JSFunctionSpec initializer.
- Fix js_GetAtom fallibility by returning &dummy on assert-botch "can't happen"
index out of range case.
- js_InitAtomMap needn't bother nulling ale->next with tmp save
- js_InitAtomState explicit tail fusion for FROB via goto bad, and early memset
(I know, JSRuntime is cleared already and it contains the atom state ... but
jsatom.c doesn't know that).
- Clear all ATOM_ flags save ATOM_PINNED when creating a new atom.
- Cleanup xtra, ALIGNNUM, etc. useless variables, use JSVAL_ALIGN and JS_MAX.
+ merging of js/src and js/ref
+ elimination of most dependencies on NSPR
+ JS1.4 feature additions and accumulated bug fixes
More details are in last week's mozilla status report.
