Attachment 284951 Details for Bug 160125 – fixed the deltablue bug

[patch] fixed the deltablue bug

blah.patch (text/plain), 157.68 KB, created by Filip Pizlo on 2016-07-30 12:27:19 PDT

(hide)

Description:

Filename:

MIME Type:

Creator: Filip Pizlo

Created: 2016-07-30 12:27:19 PDT

Size: 157.68 KB

patch

obsolete

>Index: Source/JavaScriptCore/ChangeLog
>===================================================================
>--- Source/JavaScriptCore/ChangeLog	(revision 203920)
>+++ Source/JavaScriptCore/ChangeLog	(working copy)
>@@ -1,3 +1,213 @@
>+2016-07-29  Filip Pizlo  <fpizlo@apple.com>
>+
>+        Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace
>+        https://bugs.webkit.org/show_bug.cgi?id=160125
>+
>+        Reviewed by NOBODY (OOPS!).
>+        
>+        In order to make the GC concurrent (bug 149432), we would either need to enable concurrent
>+        copying or we would need to not copy. Concurrent copying carries a 1-2% throughput overhead
>+        from the barriers alone. Considering that MarkedSpace does a decent job of avoiding
>+        fragmentation, it's unlikely that it's worth paying 1-2% throughput for copying. So, we want
>+        to get rid of copied space. This change moves copied space's biggest client over to marked
>+        space.
>+        
>+        Moving butterflies to marked space means having them use the new Auxiliary HeapCell
>+        allocation path. This is a fairly mechanical change. At a high level this does these things:
>+        
>+        - We use AuxiliaryBarrier instead of CopyBarrier.
>+        
>+        - We use tryAllocateAuxiliary instead of tryAllocateStorage. I got rid of the silly
>+          CheckedBoolean stuff, since it's so much more trouble than it's worth.
>+        
>+        - The JITs have to emit inlined marked space allocations instead of inline copy space
>+          allocations.
>+        
>+        - Everyone has to get used to zeroing their butterflies after allocation instead of relying
>+          on them being pre-zeroed by the GC. Copied space would zero things for you, while marked
>+          space doesn't.
>+        
>+        This is still a work in progress. It still fails some internal fuzzing tests and I haven't
>+        measured performance yet.
>+
>+        * JavaScriptCore.xcodeproj/project.pbxproj:
>+        * bytecode/PolymorphicAccess.cpp:
>+        (JSC::AccessCase::generateImpl):
>+        * dfg/DFGSpeculativeJIT.cpp:
>+        (JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
>+        (JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage):
>+        (JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage):
>+        * dfg/DFGSpeculativeJIT32_64.cpp:
>+        (JSC::DFG::SpeculativeJIT::compile):
>+        * dfg/DFGSpeculativeJIT64.cpp:
>+        (JSC::DFG::SpeculativeJIT::compile):
>+        * ftl/FTLLowerDFGToB3.cpp:
>+        (JSC::FTL::DFG::LowerDFGToB3::compileCreateDirectArguments):
>+        (JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSize):
>+        (JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
>+        (JSC::FTL::DFG::LowerDFGToB3::initializeArrayElements):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl):
>+        (JSC::FTL::DFG::LowerDFGToB3::emitRightShiftSnippet):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocateHeapCell):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocateCell):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocateObject):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocatorForSize):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocateVariableSizedObject):
>+        (JSC::FTL::DFG::LowerDFGToB3::allocateJSArray):
>+        * heap/ConservativeRoots.cpp:
>+        (JSC::ConservativeRoots::~ConservativeRoots):
>+        (JSC::ConservativeRoots::grow):
>+        (JSC::ConservativeRoots::genericAddPointer):
>+        * heap/ConservativeRoots.h:
>+        (JSC::ConservativeRoots::size):
>+        (JSC::ConservativeRoots::roots):
>+        * heap/CopyToken.h:
>+        * heap/Heap.cpp:
>+        (JSC::Heap::copyBackingStores):
>+        * heap/Heap.h:
>+        (JSC::Heap::allocatorForObjectWithoutDestructor):
>+        (JSC::Heap::allocatorForObjectWithDestructor):
>+        (JSC::Heap::allocatorForAuxiliaryData):
>+        (JSC::Heap::storageAllocator):
>+        * heap/HeapInlines.h:
>+        (JSC::Heap::isCollecting):
>+        (JSC::Heap::heap):
>+        (JSC::Heap::allocateWithoutDestructor):
>+        (JSC::Heap::allocateObjectOfType):
>+        (JSC::Heap::subspaceForObjectOfType):
>+        (JSC::Heap::allocatorForObjectOfType):
>+        (JSC::Heap::allocateAuxiliary):
>+        (JSC::Heap::tryAllocateAuxiliary):
>+        (JSC::Heap::tryReallocateAuxiliary):
>+        (JSC::Heap::tryAllocateStorage):
>+        (JSC::Heap::didFreeBlock):
>+        (JSC::Heap::findGCObjectPointer):
>+        (JSC::Heap::isPointerGCObjectJSCell):
>+        (JSC::Heap::isValueGCObject):
>+        (JSC::Heap::isPointerGCObject): Deleted.
>+        * heap/MarkedAllocator.cpp:
>+        (JSC::MarkedAllocator::retire):
>+        (JSC::MarkedAllocator::tryAllocateWithoutCollectingImpl):
>+        (JSC::MarkedAllocator::tryPopFreeList):
>+        (JSC::MarkedAllocator::tryAllocateWithoutCollecting):
>+        (JSC::MarkedAllocator::allocateSlowCase):
>+        (JSC::MarkedAllocator::tryAllocateSlowCase):
>+        (JSC::MarkedAllocator::allocateSlowCaseImpl):
>+        (JSC::MarkedAllocator::tryAllocateBlock):
>+        (JSC::MarkedAllocator::addBlock):
>+        (JSC::MarkedAllocator::tryAllocateHelper): Deleted.
>+        (JSC::MarkedAllocator::tryAllocate): Deleted.
>+        (JSC::MarkedAllocator::allocateBlock): Deleted.
>+        * heap/MarkedAllocator.h:
>+        (JSC::MarkedAllocator::destruction):
>+        (JSC::MarkedAllocator::cellKind):
>+        (JSC::MarkedAllocator::heap):
>+        (JSC::MarkedAllocator::takeLastActiveBlock):
>+        (JSC::MarkedAllocator::init):
>+        (JSC::MarkedAllocator::tryAllocate):
>+        (JSC::MarkedAllocator::allocate):
>+        * heap/MarkedBlock.cpp:
>+        (JSC::MarkedBlock::tryCreate):
>+        (JSC::MarkedBlock::create): Deleted.
>+        * heap/MarkedBlock.h:
>+        (JSC::MarkedBlock::isAtomAligned):
>+        (JSC::MarkedBlock::cellAlign):
>+        (JSC::MarkedBlock::blockFor):
>+        * heap/MarkedSpace.h:
>+        (JSC::MarkedSpace::subspaceForObjectsWithDestructor):
>+        (JSC::MarkedSpace::subspaceForObjectsWithoutDestructor):
>+        (JSC::MarkedSpace::allocateAuxiliary):
>+        (JSC::MarkedSpace::tryAllocateAuxiliary):
>+        (JSC::MarkedSpace::forEachBlock):
>+        * heap/SlotVisitor.cpp:
>+        (JSC::SlotVisitor::append):
>+        (JSC::SlotVisitor::appendJSCellOrAuxiliary):
>+        (JSC::SlotVisitor::appendToMarkStack):
>+        (JSC::SlotVisitor::markAuxiliary):
>+        (JSC::SlotVisitor::noteLiveAuxiliaryCell):
>+        (JSC::SetCurrentCellScope::SetCurrentCellScope):
>+        (JSC::SlotVisitor::visitChildren):
>+        * heap/SlotVisitor.h:
>+        * inspector/JSJavaScriptCallFrame.cpp:
>+        * llint/LLIntOffsetsExtractor.cpp:
>+        (main):
>+        * parser/ModuleAnalyzer.cpp:
>+        * runtime/AuxiliaryBarrier.h: Added.
>+        (JSC::AuxiliaryBarrier::AuxiliaryBarrier):
>+        (JSC::AuxiliaryBarrier::clear):
>+        (JSC::AuxiliaryBarrier::get):
>+        (JSC::AuxiliaryBarrier::slot):
>+        (JSC::AuxiliaryBarrier::operator bool):
>+        (JSC::AuxiliaryBarrier::setWithoutBarrier):
>+        * runtime/AuxiliaryBarrierInlines.h: Added.
>+        (JSC::AuxiliaryBarrier<T>::AuxiliaryBarrier):
>+        (JSC::AuxiliaryBarrier<T>::set):
>+        * runtime/ButterflyInlines.h:
>+        (JSC::Butterfly::createUninitialized):
>+        (JSC::Butterfly::growArrayRight):
>+        * runtime/ClonedArguments.cpp:
>+        (JSC::ClonedArguments::createEmpty):
>+        * runtime/IntlCollator.cpp:
>+        * runtime/IntlDateTimeFormat.cpp:
>+        * runtime/IntlNumberFormat.cpp:
>+        * runtime/JSArray.cpp:
>+        (JSC::JSArray::unshiftCountSlowCase):
>+        * runtime/JSArray.h:
>+        (JSC::createArrayButterfly):
>+        (JSC::JSArray::create):
>+        (JSC::JSArray::tryCreateUninitialized):
>+        * runtime/JSCInlines.h:
>+        * runtime/JSCallee.cpp:
>+        * runtime/JSGenericTypedArrayViewInlines.h:
>+        (JSC::JSGenericTypedArrayView<Adaptor>::slowDownAndWasteMemory):
>+        * runtime/JSMapIterator.cpp:
>+        * runtime/JSModuleNamespaceObject.cpp:
>+        * runtime/JSModuleRecord.cpp:
>+        * runtime/JSObject.cpp:
>+        (JSC::getClassPropertyNames):
>+        (JSC::JSObject::visitButterfly):
>+        (JSC::JSObject::visitChildren):
>+        (JSC::JSObject::heapSnapshot):
>+        (JSC::JSObject::createInitialInt32):
>+        (JSC::JSObject::createInitialContiguous):
>+        (JSC::JSObject::createArrayStorage):
>+        (JSC::JSObject::convertUndecidedToInt32):
>+        (JSC::JSObject::convertUndecidedToContiguous):
>+        (JSC::JSObject::convertUndecidedToArrayStorage):
>+        (JSC::JSObject::convertInt32ToDouble):
>+        (JSC::JSObject::convertInt32ToArrayStorage):
>+        (JSC::JSObject::convertDoubleToArrayStorage):
>+        (JSC::JSObject::convertContiguousToArrayStorage):
>+        (JSC::JSObject::increaseVectorLength):
>+        (JSC::JSObject::ensureLengthSlow):
>+        (JSC::JSObject::copyButterfly): Deleted.
>+        (JSC::JSObject::copyBackingStore): Deleted.
>+        * runtime/JSObject.h:
>+        * runtime/JSObjectInlines.h:
>+        * runtime/JSPromise.cpp:
>+        * runtime/JSPromisePrototype.cpp:
>+        * runtime/JSPropertyNameIterator.cpp:
>+        * runtime/JSSetIterator.cpp:
>+        * runtime/JSStringIterator.cpp:
>+        * runtime/JSTemplateRegistryKey.cpp:
>+        * runtime/JSTypedArrayViewPrototype.cpp:
>+        * runtime/MapConstructor.cpp:
>+        * runtime/MapPrototype.cpp:
>+        * runtime/ProxyConstructor.cpp:
>+        * runtime/ProxyObject.cpp:
>+        * runtime/RegExpMatchesArray.h:
>+        (JSC::tryCreateUninitializedRegExpMatchesArray):
>+        * runtime/SamplingProfiler.cpp:
>+        (JSC::SamplingProfiler::processUnverifiedStackTraces):
>+        * runtime/SetConstructor.cpp:
>+        * runtime/SetPrototype.cpp:
>+        * runtime/WeakMapConstructor.cpp:
>+        * runtime/WeakSetConstructor.cpp:
>+        * tests/stress/array-storage-array-unshift.js: Added.
>+        * tests/stress/contiguous-array-unshift.js: Added.
>+        * tests/stress/double-array-unshift.js: Added.
>+        * tests/stress/int32-array-unshift.js: Added.
>+
> 2016-07-29  Yusuke Suzuki  <utatane.tea@gmail.com>
> 
>         Unreviewed, ByValInfo is only used in JIT enabled environments
>Index: Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj
>===================================================================
>--- Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj	(revision 203920)
>+++ Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj	(working copy)
>@@ -322,6 +322,8 @@
> 		0F38B01817CFE75500B144D3 /* DFGCompilationKey.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F38B01417CFE75500B144D3 /* DFGCompilationKey.h */; };
> 		0F38B01917CFE75500B144D3 /* DFGCompilationMode.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F38B01517CFE75500B144D3 /* DFGCompilationMode.cpp */; };
> 		0F38B01A17CFE75500B144D3 /* DFGCompilationMode.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F38B01617CFE75500B144D3 /* DFGCompilationMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
>+		0F38D2A21D44196800680499 /* AuxiliaryBarrier.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F38D2A01D44196600680499 /* AuxiliaryBarrier.h */; settings = {ATTRIBUTES = (Private, ); }; };
>+		0F38D2A31D44196D00680499 /* AuxiliaryBarrierInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F38D2A11D44196600680499 /* AuxiliaryBarrierInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
> 		0F392C891B46188400844728 /* DFGOSRExitFuzz.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F392C871B46188400844728 /* DFGOSRExitFuzz.cpp */; };
> 		0F392C8A1B46188400844728 /* DFGOSRExitFuzz.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F392C881B46188400844728 /* DFGOSRExitFuzz.h */; };
> 		0F3A1BF91A9ECB7D000DE01A /* DFGPutStackSinkingPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F3A1BF71A9ECB7D000DE01A /* DFGPutStackSinkingPhase.cpp */; };
>@@ -572,6 +574,7 @@
> 		0FA7A8EB18B413C80052371D /* Reg.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FA7A8E918B413C80052371D /* Reg.cpp */; };
> 		0FA7A8EC18B413C80052371D /* Reg.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FA7A8EA18B413C80052371D /* Reg.h */; settings = {ATTRIBUTES = (Private, ); }; };
> 		0FA7A8EE18CE4FD80052371D /* ScratchRegisterAllocator.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FA7A8ED18CE4FD80052371D /* ScratchRegisterAllocator.cpp */; };
>+		0FADE6731D4D23BE00768457 /* HeapUtil.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FADE6721D4D23BC00768457 /* HeapUtil.h */; };
> 		0FAF7EFD165BA91B000C8455 /* JITDisassembler.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FAF7EFA165BA919000C8455 /* JITDisassembler.cpp */; };
> 		0FAF7EFE165BA91F000C8455 /* JITDisassembler.h in Headers */ = {isa = PBXBuildFile; fileRef = 0FAF7EFB165BA919000C8455 /* JITDisassembler.h */; settings = {ATTRIBUTES = (Private, ); }; };
> 		0FB105851675480F00F8AB6E /* ExitKind.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0FB105821675480C00F8AB6E /* ExitKind.cpp */; };
>@@ -2518,6 +2521,8 @@
> 		0F38B01417CFE75500B144D3 /* DFGCompilationKey.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGCompilationKey.h; path = dfg/DFGCompilationKey.h; sourceTree = "<group>"; };
> 		0F38B01517CFE75500B144D3 /* DFGCompilationMode.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGCompilationMode.cpp; path = dfg/DFGCompilationMode.cpp; sourceTree = "<group>"; };
> 		0F38B01617CFE75500B144D3 /* DFGCompilationMode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGCompilationMode.h; path = dfg/DFGCompilationMode.h; sourceTree = "<group>"; };
>+		0F38D2A01D44196600680499 /* AuxiliaryBarrier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AuxiliaryBarrier.h; sourceTree = "<group>"; };
>+		0F38D2A11D44196600680499 /* AuxiliaryBarrierInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AuxiliaryBarrierInlines.h; sourceTree = "<group>"; };
> 		0F392C871B46188400844728 /* DFGOSRExitFuzz.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGOSRExitFuzz.cpp; path = dfg/DFGOSRExitFuzz.cpp; sourceTree = "<group>"; };
> 		0F392C881B46188400844728 /* DFGOSRExitFuzz.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGOSRExitFuzz.h; path = dfg/DFGOSRExitFuzz.h; sourceTree = "<group>"; };
> 		0F3A1BF71A9ECB7D000DE01A /* DFGPutStackSinkingPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGPutStackSinkingPhase.cpp; path = dfg/DFGPutStackSinkingPhase.cpp; sourceTree = "<group>"; };
>@@ -2765,6 +2770,7 @@
> 		0FA7A8E918B413C80052371D /* Reg.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = Reg.cpp; sourceTree = "<group>"; };
> 		0FA7A8EA18B413C80052371D /* Reg.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Reg.h; sourceTree = "<group>"; };
> 		0FA7A8ED18CE4FD80052371D /* ScratchRegisterAllocator.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ScratchRegisterAllocator.cpp; sourceTree = "<group>"; };
>+		0FADE6721D4D23BC00768457 /* HeapUtil.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HeapUtil.h; sourceTree = "<group>"; };
> 		0FAF7EFA165BA919000C8455 /* JITDisassembler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JITDisassembler.cpp; sourceTree = "<group>"; };
> 		0FAF7EFB165BA919000C8455 /* JITDisassembler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JITDisassembler.h; sourceTree = "<group>"; };
> 		0FB105821675480C00F8AB6E /* ExitKind.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ExitKind.cpp; sourceTree = "<group>"; };
>@@ -5246,6 +5252,7 @@
> 				C24D31E1161CD695002AA4DB /* HeapStatistics.h */,
> 				C2E526BB1590EF000054E48D /* HeapTimer.cpp */,
> 				C2E526BC1590EF000054E48D /* HeapTimer.h */,
>+				0FADE6721D4D23BC00768457 /* HeapUtil.h */,
> 				FE7BA60D1A1A7CEC00F1F7B4 /* HeapVerifier.cpp */,
> 				FE7BA60E1A1A7CEC00F1F7B4 /* HeapVerifier.h */,
> 				C25F8BCB157544A900245B71 /* IncrementalSweeper.cpp */,
>@@ -5607,6 +5614,8 @@
> 				F692A84D0255597D01FF60F7 /* ArrayPrototype.cpp */,
> 				F692A84E0255597D01FF60F7 /* ArrayPrototype.h */,
> 				0FB7F38A15ED8E3800F167B2 /* ArrayStorage.h */,
>+				0F38D2A01D44196600680499 /* AuxiliaryBarrier.h */,
>+				0F38D2A11D44196600680499 /* AuxiliaryBarrierInlines.h */,
> 				52678F8C1A031009006A306D /* BasicBlockLocation.cpp */,
> 				52678F8D1A031009006A306D /* BasicBlockLocation.h */,
> 				147B83AA0E6DB8C9004775A4 /* BatchedTransitionOptimizer.h */,
>@@ -5772,7 +5781,6 @@
> 				70DC3E081B2DF2C700054299 /* IteratorPrototype.h */,
> 				93ADFCE60CCBD7AC00D30B08 /* JSArray.cpp */,
> 				938772E5038BFE19008635CE /* JSArray.h */,
>-				539FB8B91C99DA7C00940FA1 /* JSArrayInlines.h */,
> 				0F2B66B417B6B5AB00A7AE3F /* JSArrayBuffer.cpp */,
> 				0F2B66B517B6B5AB00A7AE3F /* JSArrayBuffer.h */,
> 				0F2B66B617B6B5AB00A7AE3F /* JSArrayBufferConstructor.cpp */,
>@@ -5782,6 +5790,7 @@
> 				0F2B66BA17B6B5AB00A7AE3F /* JSArrayBufferView.cpp */,
> 				0F2B66BB17B6B5AB00A7AE3F /* JSArrayBufferView.h */,
> 				0F2B66BC17B6B5AB00A7AE3F /* JSArrayBufferViewInlines.h */,
>+				539FB8B91C99DA7C00940FA1 /* JSArrayInlines.h */,
> 				86FA9E8F142BBB2D001773B7 /* JSBoundFunction.cpp */,
> 				86FA9E90142BBB2E001773B7 /* JSBoundFunction.h */,
> 				657CF45619BF6662004ACBF2 /* JSCallee.cpp */,
>@@ -7261,6 +7270,7 @@
> 				0F338E1C1BF286EA0013C88F /* B3BlockInsertionSet.h in Headers */,
> 				0F9495881C57F47500413A48 /* B3StackSlot.h in Headers */,
> 				C4F4B6F31A05C944005CAB76 /* cpp_generator_templates.py in Headers */,
>+				0F38D2A21D44196800680499 /* AuxiliaryBarrier.h in Headers */,
> 				5DE6E5B30E1728EC00180407 /* create_hash_table in Headers */,
> 				9959E92B1BD17FA4001AA413 /* cssmin.py in Headers */,
> 				2A111246192FCE79005EE18D /* CustomGetterSetter.h in Headers */,
>@@ -7693,6 +7703,7 @@
> 				A1587D6E1B4DC14100D69849 /* IntlDateTimeFormat.h in Headers */,
> 				FE187A0F1C030D6C0038BBCA /* SnippetOperand.h in Headers */,
> 				A1587D701B4DC14100D69849 /* IntlDateTimeFormatConstructor.h in Headers */,
>+				0FADE6731D4D23BE00768457 /* HeapUtil.h in Headers */,
> 				A1587D751B4DC1C600D69849 /* IntlDateTimeFormatConstructor.lut.h in Headers */,
> 				A5398FAB1C750DA40060A963 /* HeapProfiler.h in Headers */,
> 				A1587D721B4DC14100D69849 /* IntlDateTimeFormatPrototype.h in Headers */,
>@@ -7815,6 +7826,7 @@
> 				C25D709C16DE99F400FCA6BC /* JSManagedValue.h in Headers */,
> 				2A4BB7F318A41179008A0FCD /* JSManagedValueInternal.h in Headers */,
> 				A700874217CBE8EB00C3E643 /* JSMap.h in Headers */,
>+				0F38D2A31D44196D00680499 /* AuxiliaryBarrierInlines.h in Headers */,
> 				A74DEF96182D991400522C22 /* JSMapIterator.h in Headers */,
> 				9959E92D1BD17FA4001AA413 /* jsmin.py in Headers */,
> 				E3D239C91B829C1C00BBEF67 /* JSModuleEnvironment.h in Headers */,
>Index: Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp
>===================================================================
>--- Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp	(revision 203920)
>+++ Source/JavaScriptCore/bytecode/PolymorphicAccess.cpp	(working copy)
>@@ -1206,34 +1206,20 @@ void AccessCase::generateImpl(AccessGene
>             size_t newSize = newStructure()->outOfLineCapacity() * sizeof(JSValue);
>             
>             if (allocatingInline) {
>-                CopiedAllocator* copiedAllocator = &vm.heap.storageAllocator();
>-
>-                if (!reallocating) {
>-                    jit.loadPtr(&copiedAllocator->m_currentRemaining, scratchGPR);
>-                    slowPath.append(
>-                        jit.branchSubPtr(
>-                            CCallHelpers::Signed, CCallHelpers::TrustedImm32(newSize), scratchGPR));
>-                    jit.storePtr(scratchGPR, &copiedAllocator->m_currentRemaining);
>-                    jit.negPtr(scratchGPR);
>-                    jit.addPtr(
>-                        CCallHelpers::AbsoluteAddress(&copiedAllocator->m_currentPayloadEnd), scratchGPR);
>-                    jit.addPtr(CCallHelpers::TrustedImm32(sizeof(JSValue)), scratchGPR);
>-                } else {
>+                MarkedAllocator& allocator = vm.heap.allocatorForAuxiliaryData(newSize);
>+                
>+                jit.move(CCallHelpers::TrustedImmPtr(&allocator), scratchGPR2);
>+                jit.emitAllocate(scratchGPR, scratchGPR2, scratchGPR3, slowPath);
>+                jit.addPtr(CCallHelpers::TrustedImm32(newSize + sizeof(IndexingHeader)), scratchGPR);
>+                
>+                if (reallocating) {
>                     // Handle the case where we are reallocating (i.e. the old structure/butterfly
>                     // already had out-of-line property storage).
>                     size_t oldSize = structure()->outOfLineCapacity() * sizeof(JSValue);
>                     ASSERT(newSize > oldSize);
>             
>                     jit.loadPtr(CCallHelpers::Address(baseGPR, JSObject::butterflyOffset()), scratchGPR3);
>-                    jit.loadPtr(&copiedAllocator->m_currentRemaining, scratchGPR);
>-                    slowPath.append(
>-                        jit.branchSubPtr(
>-                            CCallHelpers::Signed, CCallHelpers::TrustedImm32(newSize), scratchGPR));
>-                    jit.storePtr(scratchGPR, &copiedAllocator->m_currentRemaining);
>-                    jit.negPtr(scratchGPR);
>-                    jit.addPtr(
>-                        CCallHelpers::AbsoluteAddress(&copiedAllocator->m_currentPayloadEnd), scratchGPR);
>-                    jit.addPtr(CCallHelpers::TrustedImm32(sizeof(JSValue)), scratchGPR);
>+                    
>                     // We have scratchGPR = new storage, scratchGPR3 = old storage,
>                     // scratchGPR2 = available
>                     for (size_t offset = 0; offset < oldSize; offset += sizeof(void*)) {
>Index: Source/JavaScriptCore/dfg/DFGOperations.cpp
>===================================================================
>--- Source/JavaScriptCore/dfg/DFGOperations.cpp	(revision 203920)
>+++ Source/JavaScriptCore/dfg/DFGOperations.cpp	(working copy)
>@@ -852,7 +852,6 @@ char* JIT_OPERATION operationNewArrayWit
>         return bitwise_cast<char*>(exec->vm().throwException(exec, createRangeError(exec, ASCIILiteral("Array size is not a small enough positive integer."))));
> 
>     JSArray* result = JSArray::create(*vm, arrayStructure, size);
>-    result->butterfly(); // Ensure that the backing store is in to-space.
>     return bitwise_cast<char*>(result);
> }
> 
>Index: Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp
>===================================================================
>--- Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp	(revision 203920)
>+++ Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp	(working copy)
>@@ -3907,27 +3907,30 @@ void SpeculativeJIT::compile(Node* node)
>             m_jit.move(sizeGPR, scratchGPR);
>             m_jit.lshift32(TrustedImm32(3), scratchGPR);
>             m_jit.add32(TrustedImm32(sizeof(IndexingHeader)), scratchGPR, resultGPR);
>-            slowCases.append(
>-                emitAllocateBasicStorage(resultGPR, storageGPR));
>-            m_jit.subPtr(scratchGPR, storageGPR);
>+            m_jit.emitAllocateVariableSized(
>+                storageGPR, m_jit.vm()->heap.subspaceForAuxiliaryData(), resultGPR, scratchGPR,
>+                scratch2GPR, slowCases);
>+            m_jit.addPtr(TrustedImm32(sizeof(IndexingHeader)), storageGPR);
>             Structure* structure = globalObject->arrayStructureForIndexingTypeDuringAllocation(node->indexingType());
>             emitAllocateJSObject<JSArray>(resultGPR, TrustedImmPtr(structure), storageGPR, scratchGPR, scratch2GPR, slowCases);
>             
>             m_jit.store32(sizeGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
>             m_jit.store32(sizeGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
>             
>-            if (hasDouble(node->indexingType())) {
>-                JSValue nan = JSValue(JSValue::EncodeAsDouble, PNaN);
>-                
>-                m_jit.move(sizeGPR, scratchGPR);
>-                MacroAssembler::Jump done = m_jit.branchTest32(MacroAssembler::Zero, scratchGPR);
>-                MacroAssembler::Label loop = m_jit.label();
>-                m_jit.sub32(TrustedImm32(1), scratchGPR);
>-                m_jit.store32(TrustedImm32(nan.u.asBits.tag), MacroAssembler::BaseIndex(storageGPR, scratchGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
>-                m_jit.store32(TrustedImm32(nan.u.asBits.payload), MacroAssembler::BaseIndex(storageGPR, scratchGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
>-                m_jit.branchTest32(MacroAssembler::NonZero, scratchGPR).linkTo(loop, &m_jit);
>-                done.link(&m_jit);
>-            }
>+            JSValue hole;
>+            if (hasDouble(node->indexingType()))
>+                hole = JSValue(JSValue::EncodeAsDouble, PNaN);
>+            else
>+                hole = JSValue();
>+            
>+            m_jit.move(sizeGPR, scratchGPR);
>+            MacroAssembler::Jump done = m_jit.branchTest32(MacroAssembler::Zero, scratchGPR);
>+            MacroAssembler::Label loop = m_jit.label();
>+            m_jit.sub32(TrustedImm32(1), scratchGPR);
>+            m_jit.store32(TrustedImm32(hole.u.asBits.tag), MacroAssembler::BaseIndex(storageGPR, scratchGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
>+            m_jit.store32(TrustedImm32(hole.u.asBits.payload), MacroAssembler::BaseIndex(storageGPR, scratchGPR, MacroAssembler::TimesEight, OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
>+            m_jit.branchTest32(MacroAssembler::NonZero, scratchGPR).linkTo(loop, &m_jit);
>+            done.link(&m_jit);
>             
>             addSlowPathGenerator(std::make_unique<CallArrayAllocatorWithVariableSizeSlowPathGenerator>(
>                     slowCases, this, operationNewArrayWithSize, resultGPR,
>Index: Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp
>===================================================================
>--- Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp	(revision 203920)
>+++ Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp	(working copy)
>@@ -3869,27 +3869,31 @@ void SpeculativeJIT::compile(Node* node)
>             m_jit.move(sizeGPR, scratchGPR);
>             m_jit.lshift32(TrustedImm32(3), scratchGPR);
>             m_jit.add32(TrustedImm32(sizeof(IndexingHeader)), scratchGPR, resultGPR);
>-            slowCases.append(
>-                emitAllocateBasicStorage(resultGPR, storageGPR));
>-            m_jit.subPtr(scratchGPR, storageGPR);
>+            m_jit.emitAllocateVariableSized(
>+                storageGPR, m_jit.vm()->heap.subspaceForAuxiliaryData(), resultGPR, scratchGPR,
>+                scratch2GPR, slowCases);
>+            m_jit.addPtr(TrustedImm32(sizeof(IndexingHeader)), storageGPR);
>             Structure* structure = globalObject->arrayStructureForIndexingTypeDuringAllocation(node->indexingType());
>+            
>             emitAllocateJSObject<JSArray>(resultGPR, TrustedImmPtr(structure), storageGPR, scratchGPR, scratch2GPR, slowCases);
>             
>             m_jit.store32(sizeGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfPublicLength()));
>             m_jit.store32(sizeGPR, MacroAssembler::Address(storageGPR, Butterfly::offsetOfVectorLength()));
>             
>-            if (hasDouble(node->indexingType())) {
>+            if (hasDouble(node->indexingType()))
>                 m_jit.move(TrustedImm64(bitwise_cast<int64_t>(PNaN)), scratchGPR);
>-                m_jit.move(sizeGPR, scratch2GPR);
>-                MacroAssembler::Jump done = m_jit.branchTest32(MacroAssembler::Zero, scratch2GPR);
>-                MacroAssembler::Label loop = m_jit.label();
>-                m_jit.sub32(TrustedImm32(1), scratch2GPR);
>-                m_jit.store64(scratchGPR, MacroAssembler::BaseIndex(storageGPR, scratch2GPR, MacroAssembler::TimesEight));
>-                m_jit.branchTest32(MacroAssembler::NonZero, scratch2GPR).linkTo(loop, &m_jit);
>-                done.link(&m_jit);
>-            }
>+            else
>+                m_jit.move(TrustedImm64(JSValue::encode(JSValue())), scratchGPR);
>+            m_jit.move(sizeGPR, scratch2GPR);
>+            MacroAssembler::Jump done = m_jit.branchTest32(MacroAssembler::Zero, scratch2GPR);
>+            MacroAssembler::Label loop = m_jit.label();
>+            m_jit.sub32(TrustedImm32(1), scratch2GPR);
>+            m_jit.store64(scratchGPR, MacroAssembler::BaseIndex(storageGPR, scratch2GPR, MacroAssembler::TimesEight));
>+            m_jit.branchTest32(MacroAssembler::NonZero, scratch2GPR).linkTo(loop, &m_jit);
>+            done.link(&m_jit);
>             
>-            addSlowPathGenerator(std::make_unique<CallArrayAllocatorWithVariableSizeSlowPathGenerator>(
>+            addSlowPathGenerator(
>+                std::make_unique<CallArrayAllocatorWithVariableSizeSlowPathGenerator>(
>                     slowCases, this, operationNewArrayWithSize, resultGPR,
>                     globalObject->arrayStructureForIndexingTypeDuringAllocation(node->indexingType()),
>                     globalObject->arrayStructureForIndexingTypeDuringAllocation(ArrayWithArrayStorage),
>Index: Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp
>===================================================================
>--- Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp	(revision 203920)
>+++ Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp	(working copy)
>@@ -103,12 +103,12 @@ void SpeculativeJIT::emitAllocateRawObje
>     size += outOfLineCapacity * sizeof(JSValue);
> 
>     if (size) {
>-        slowCases.append(
>-            emitAllocateBasicStorage(TrustedImm32(size), storageGPR));
>-        if (hasIndexingHeader)
>-            m_jit.subPtr(TrustedImm32(vectorLength * sizeof(JSValue)), storageGPR);
>-        else
>-            m_jit.addPtr(TrustedImm32(sizeof(IndexingHeader)), storageGPR);
>+        m_jit.move(TrustedImmPtr(&m_jit.vm()->heap.allocatorForAuxiliaryData(size)), scratchGPR);
>+        m_jit.emitAllocate(storageGPR, scratchGPR, scratch2GPR, slowCases);
>+        
>+        m_jit.addPtr(
>+            TrustedImm32(outOfLineCapacity * sizeof(JSValue) + sizeof(IndexingHeader)),
>+            storageGPR);
>     } else
>         m_jit.move(TrustedImmPtr(0), storageGPR);
> 
>@@ -127,14 +127,20 @@ void SpeculativeJIT::emitAllocateRawObje
>         slowCases, this, operationNewRawObject, resultGPR, storageGPR,
>         structure, vectorLength));
> 
>-    if (hasDouble(structure->indexingType()) && numElements < vectorLength) {
>+    if (numElements < vectorLength) {
> #if USE(JSVALUE64)
>-        m_jit.move(TrustedImm64(bitwise_cast<int64_t>(PNaN)), scratchGPR);
>+        if (hasDouble(structure->indexingType()))
>+            m_jit.move(TrustedImm64(bitwise_cast<int64_t>(PNaN)), scratchGPR);
>+        else
>+            m_jit.move(TrustedImm64(JSValue::encode(JSValue())), scratchGPR);
>         for (unsigned i = numElements; i < vectorLength; ++i)
>             m_jit.store64(scratchGPR, MacroAssembler::Address(storageGPR, sizeof(double) * i));
> #else
>         EncodedValueDescriptor value;
>-        value.asInt64 = JSValue::encode(JSValue(JSValue::EncodeAsDouble, PNaN));
>+        if (hasDouble(structure->indexingType()))
>+            value.asInt64 = JSValue::encode(JSValue(JSValue::EncodeAsDouble, PNaN));
>+        else
>+            value.asInt64 = JSValue::encode(JSValue());
>         for (unsigned i = numElements; i < vectorLength; ++i) {
>             m_jit.store32(TrustedImm32(value.asBits.tag), MacroAssembler::Address(storageGPR, sizeof(double) * i + OBJECT_OFFSETOF(JSValue, u.asBits.tag)));
>             m_jit.store32(TrustedImm32(value.asBits.payload), MacroAssembler::Address(storageGPR, sizeof(double) * i + OBJECT_OFFSETOF(JSValue, u.asBits.payload)));
>@@ -6726,18 +6732,24 @@ void SpeculativeJIT::compileAllocateProp
>     
>     SpeculateCellOperand base(this, node->child1());
>     GPRTemporary scratch1(this);
>+    GPRTemporary scratch2(this);
>+    GPRTemporary scratch3(this);
>         
>     GPRReg baseGPR = base.gpr();
>     GPRReg scratchGPR1 = scratch1.gpr();
>+    GPRReg scratchGPR2 = scratch2.gpr();
>+    GPRReg scratchGPR3 = scratch3.gpr();
>         
>     ASSERT(!node->transition()->previous->outOfLineCapacity());
>     ASSERT(initialOutOfLineCapacity == node->transition()->next->outOfLineCapacity());
>     
>-    JITCompiler::Jump slowPath =
>-        emitAllocateBasicStorage(
>-            TrustedImm32(initialOutOfLineCapacity * sizeof(JSValue)), scratchGPR1);
>-
>-    m_jit.addPtr(JITCompiler::TrustedImm32(sizeof(IndexingHeader)), scratchGPR1);
>+    size_t size = initialOutOfLineCapacity * sizeof(JSValue);
>+    m_jit.move(
>+        JITCompiler::TrustedImmPtr(&m_jit.vm()->heap.allocatorForAuxiliaryData(size)),
>+        scratchGPR2);
>+    JITCompiler::JumpList slowPath;
>+    m_jit.emitAllocate(scratchGPR1, scratchGPR2, scratchGPR3, slowPath);
>+    m_jit.addPtr(JITCompiler::TrustedImm32(size + sizeof(IndexingHeader)), scratchGPR1);
>         
>     addSlowPathGenerator(
>         slowPathCall(slowPath, this, operationAllocatePropertyStorageWithInitialCapacity, scratchGPR1));
>@@ -6772,16 +6784,21 @@ void SpeculativeJIT::compileReallocatePr
>     StorageOperand oldStorage(this, node->child2());
>     GPRTemporary scratch1(this);
>     GPRTemporary scratch2(this);
>+    GPRTemporary scratch3(this);
>         
>     GPRReg baseGPR = base.gpr();
>     GPRReg oldStorageGPR = oldStorage.gpr();
>     GPRReg scratchGPR1 = scratch1.gpr();
>     GPRReg scratchGPR2 = scratch2.gpr();
>-        
>-    JITCompiler::Jump slowPath =
>-        emitAllocateBasicStorage(TrustedImm32(newSize), scratchGPR1);
>-
>-    m_jit.addPtr(JITCompiler::TrustedImm32(sizeof(IndexingHeader)), scratchGPR1);
>+    GPRReg scratchGPR3 = scratch3.gpr();
>+    
>+    JITCompiler::JumpList slowPath;
>+    m_jit.move(
>+        JITCompiler::TrustedImmPtr(&m_jit.vm()->heap.allocatorForAuxiliaryData(newSize)),
>+        scratchGPR2);
>+    m_jit.emitAllocate(scratchGPR1, scratchGPR2, scratchGPR3, slowPath);
>+    
>+    m_jit.addPtr(JITCompiler::TrustedImm32(newSize + sizeof(IndexingHeader)), scratchGPR1);
>         
>     addSlowPathGenerator(
>         slowPathCall(slowPath, this, operationAllocatePropertyStorage, scratchGPR1, newSize / sizeof(JSValue)));
>Index: Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp
>===================================================================
>--- Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp	(revision 203920)
>+++ Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp	(working copy)
>@@ -3766,7 +3766,7 @@ private:
>                 size, m_out.constInt32(DirectArguments::allocationSize(minCapacity)));
>             
>             fastObject = allocateVariableSizedObject<DirectArguments>(
>-                size, structure, m_out.intPtrZero, slowPath);
>+                m_out.zeroExtPtr(size), structure, m_out.intPtrZero, slowPath);
>         }
>         
>         m_out.store32(length.value, fastObject, m_heaps.DirectArguments_length);
>@@ -4062,9 +4062,11 @@ private:
>             LValue butterflySize = m_out.add(
>                 payloadSize, m_out.constIntPtr(sizeof(IndexingHeader)));
>             
>-            LValue endOfStorage = allocateBasicStorageAndGetEnd(butterflySize, failCase);
>+            LValue allocator = allocatorForSize(
>+                vm().heap.subspaceForAuxiliaryData(), butterflySize, failCase);
>+            LValue startOfStorage = allocateHeapCell(allocator, failCase);
>             
>-            LValue butterfly = m_out.sub(endOfStorage, payloadSize);
>+            LValue butterfly = m_out.add(startOfStorage, m_out.constIntPtr(sizeof(IndexingHeader)));
>             
>             LValue object = allocateObject<JSArray>(structure, butterfly, failCase);
>             
>@@ -6919,11 +6921,13 @@ private:
>                 
>                 LBasicBlock lastNext = m_out.insertNewBlocksBefore(slowPath);
>                 
>-                LValue endOfStorage = allocateBasicStorageAndGetEnd(butterflySize, slowPath);
>+                LValue startOfStorage = allocateHeapCell(
>+                    allocatorForSize(vm().heap.subspaceForAuxiliaryData(), butterflySize, slowPath),
>+                    slowPath);
> 
>                 LValue fastButterflyValue = m_out.add(
>-                    m_out.sub(endOfStorage, indexingPayloadSizeInBytes),
>-                    m_out.constIntPtr(sizeof(IndexingHeader) - indexingHeaderSize));
>+                    startOfStorage,
>+                    m_out.constIntPtr(structure->outOfLineCapacity() + sizeof(IndexingHeader)));
> 
>                 m_out.store32(vectorLength, fastButterflyValue, m_heaps.Butterfly_vectorLength);
>                 
>@@ -7717,10 +7721,11 @@ private:
> 
>     void initializeArrayElements(IndexingType indexingType, LValue vectorLength, LValue butterfly)
>     {
>-        if (!hasDouble(indexingType)) {
>-            // The GC already initialized everything to JSValue() for us.
>-            return;
>-        }
>+        LValue hole;
>+        if (hasDouble(indexingType))
>+            hole = m_out.constInt64(bitwise_cast<int64_t>(PNaN));
>+        else
>+            hole = m_out.constInt64(JSValue::encode(JSValue()));
> 
>         // Doubles must be initialized to PNaN.
>         LBasicBlock initLoop = m_out.newBlock();
>@@ -7735,9 +7740,7 @@ private:
>         LValue index = m_out.phi(Int32, originalIndex);
>         LValue pointer = m_out.phi(pointerType(), originalPointer);
>         
>-        m_out.store64(
>-            m_out.constInt64(bitwise_cast<int64_t>(PNaN)),
>-            TypedPointer(m_heaps.indexedDoubleProperties.atAnyIndex(), pointer));
>+        m_out.store64(hole, TypedPointer(m_heaps.indexedDoubleProperties.atAnyIndex(), pointer));
>         
>         LValue nextIndex = m_out.sub(index, m_out.int32One);
>         m_out.addIncomingToPhi(index, m_out.anchor(nextIndex));
>@@ -7798,12 +7801,13 @@ private:
>         LBasicBlock continuation = m_out.newBlock();
>         
>         LBasicBlock lastNext = m_out.insertNewBlocksBefore(slowPath);
>-        
>-        LValue endOfStorage = allocateBasicStorageAndGetEnd(
>-            m_out.constIntPtr(sizeInValues * sizeof(JSValue)), slowPath);
>+
>+        size_t sizeInBytes = sizeInValues * sizeof(JSValue);
>+        MarkedAllocator& allocator = vm().heap.allocatorForAuxiliaryData(sizeInBytes);
>+        LValue startOfStorage = allocateHeapCell(m_out.constIntPtr(&allocator), slowPath);
>         
>         ValueFromBlock fastButterfly = m_out.anchor(
>-            m_out.add(m_out.constIntPtr(sizeof(IndexingHeader)), endOfStorage));
>+            m_out.add(m_out.constIntPtr(sizeInBytes + sizeof(IndexingHeader)), startOfStorage));
>         
>         m_out.jump(continuation);
>         
>@@ -8373,7 +8377,7 @@ private:
>         setJSValue(patchpoint);
>     }
> 
>-    LValue allocateCell(LValue allocator, LBasicBlock slowPath)
>+    LValue allocateHeapCell(LValue allocator, LBasicBlock slowPath)
>     {
>         LBasicBlock success = m_out.newBlock();
>     
>@@ -8394,7 +8398,7 @@ private:
>         m_out.storePtr(
>             m_out.loadPtr(result, m_heaps.JSCell_freeListNext),
>             allocator, m_heaps.MarkedAllocator_freeListHead);
>-
>+        
>         return result;
>     }
>     
>@@ -8408,7 +8412,7 @@ private:
> 
>     LValue allocateCell(LValue allocator, Structure* structure, LBasicBlock slowPath)
>     {
>-        LValue result = allocateCell(allocator, slowPath);
>+        LValue result = allocateHeapCell(allocator, slowPath);
>         storeStructure(result, structure);
>         return result;
>     }
>@@ -8436,46 +8440,54 @@ private:
>             ClassType::allocationSize(0), structure, butterfly, slowPath);
>     }
>     
>-    template<typename ClassType>
>-    LValue allocateVariableSizedObject(
>-        LValue size, Structure* structure, LValue butterfly, LBasicBlock slowPath)
>+    LValue allocatorForSize(LValue subspace, LValue size, LBasicBlock slowPath)
>     {
>         static_assert(!(MarkedSpace::preciseStep & (MarkedSpace::preciseStep - 1)), "MarkedSpace::preciseStep must be a power of two.");
>         static_assert(!(MarkedSpace::impreciseStep & (MarkedSpace::impreciseStep - 1)), "MarkedSpace::impreciseStep must be a power of two.");
> 
>-        LValue subspace = m_out.constIntPtr(&vm().heap.subspaceForObjectOfType<ClassType>());
>-        
>         LBasicBlock smallCaseBlock = m_out.newBlock();
>         LBasicBlock largeOrOversizeCaseBlock = m_out.newBlock();
>         LBasicBlock largeCaseBlock = m_out.newBlock();
>         LBasicBlock continuation = m_out.newBlock();
>         
>-        LValue uproundedSize = m_out.add(size, m_out.constInt32(MarkedSpace::preciseStep - 1));
>-        LValue isSmall = m_out.below(uproundedSize, m_out.constInt32(MarkedSpace::preciseCutoff));
>+        LValue uproundedSize = m_out.add(size, m_out.constIntPtr(MarkedSpace::preciseStep - 1));
>+        LValue isSmall = m_out.below(uproundedSize, m_out.constIntPtr(MarkedSpace::preciseCutoff));
>         m_out.branch(isSmall, unsure(smallCaseBlock), unsure(largeOrOversizeCaseBlock));
>         
>         LBasicBlock lastNext = m_out.appendTo(smallCaseBlock, largeOrOversizeCaseBlock);
>         TypedPointer address = m_out.baseIndex(
>             m_heaps.MarkedSpace_Subspace_preciseAllocators, subspace,
>-            m_out.zeroExtPtr(m_out.lShr(uproundedSize, m_out.constInt32(getLSBSet(MarkedSpace::preciseStep)))));
>+            m_out.lShr(uproundedSize, m_out.constInt32(getLSBSet(MarkedSpace::preciseStep))));
>         ValueFromBlock smallAllocator = m_out.anchor(address.value());
>         m_out.jump(continuation);
>         
>         m_out.appendTo(largeOrOversizeCaseBlock, largeCaseBlock);
>         m_out.branch(
>-            m_out.below(uproundedSize, m_out.constInt32(MarkedSpace::impreciseCutoff)),
>+            m_out.below(uproundedSize, m_out.constIntPtr(MarkedSpace::impreciseCutoff)),
>             usually(largeCaseBlock), rarely(slowPath));
>         
>         m_out.appendTo(largeCaseBlock, continuation);
>         address = m_out.baseIndex(
>             m_heaps.MarkedSpace_Subspace_impreciseAllocators, subspace,
>-            m_out.zeroExtPtr(m_out.lShr(uproundedSize, m_out.constInt32(getLSBSet(MarkedSpace::impreciseStep)))));
>+            m_out.lShr(uproundedSize, m_out.constInt32(getLSBSet(MarkedSpace::impreciseStep))));
>         ValueFromBlock largeAllocator = m_out.anchor(address.value());
>         m_out.jump(continuation);
>         
>         m_out.appendTo(continuation, lastNext);
>-        LValue allocator = m_out.phi(pointerType(), smallAllocator, largeAllocator);
>-        
>+        return m_out.phi(pointerType(), smallAllocator, largeAllocator);
>+    }
>+    
>+    LValue allocatorForSize(MarkedSpace::Subspace& subspace, LValue size, LBasicBlock slowPath)
>+    {
>+        return allocatorForSize(m_out.constIntPtr(&subspace), size, slowPath);
>+    }
>+    
>+    template<typename ClassType>
>+    LValue allocateVariableSizedObject(
>+        LValue size, Structure* structure, LValue butterfly, LBasicBlock slowPath)
>+    {
>+        LValue allocator = allocatorForSize(
>+            vm().heap.subspaceForObjectOfType<ClassType>(), size, slowPath);
>         return allocateObject(allocator, structure, butterfly, slowPath);
>     }
>     
>@@ -8562,26 +8574,24 @@ private:
>         
>         unsigned vectorLength = std::max(BASE_VECTOR_LEN, numElements);
>         
>-        LValue endOfStorage = allocateBasicStorageAndGetEnd(
>-            m_out.constIntPtr(sizeof(JSValue) * vectorLength + sizeof(IndexingHeader)),
>-            slowPath);
>+        MarkedAllocator& allocator = vm().heap.allocatorForAuxiliaryData(
>+            sizeof(JSValue) * vectorLength + sizeof(IndexingHeader));
>+        LValue startOfStorage = allocateHeapCell(m_out.constIntPtr(&allocator), slowPath);
>         
>-        LValue butterfly = m_out.sub(
>-            endOfStorage, m_out.constIntPtr(sizeof(JSValue) * vectorLength));
>+        LValue butterfly = m_out.add(startOfStorage, m_out.constIntPtr(sizeof(IndexingHeader)));
>         
>-        LValue object = allocateObject<JSArray>(
>-            structure, butterfly, slowPath);
>+        LValue object = allocateObject<JSArray>(structure, butterfly, slowPath);
>         
>         m_out.store32(m_out.constInt32(numElements), butterfly, m_heaps.Butterfly_publicLength);
>         m_out.store32(m_out.constInt32(vectorLength), butterfly, m_heaps.Butterfly_vectorLength);
>-        
>-        if (hasDouble(structure->indexingType())) {
>-            for (unsigned i = numElements; i < vectorLength; ++i) {
>-                m_out.store64(
>-                    m_out.constInt64(bitwise_cast<int64_t>(PNaN)),
>-                    butterfly, m_heaps.indexedDoubleProperties[i]);
>-            }
>-        }
>+
>+        LValue hole;
>+        if (hasDouble(structure->indexingType()))
>+            hole = m_out.constInt64(bitwise_cast<int64_t>(PNaN));
>+        else
>+            hole = m_out.constInt64(JSValue::encode(JSValue()));
>+        for (unsigned i = numElements; i < vectorLength; ++i)
>+            m_out.store64(hole, butterfly, m_heaps.indexedDoubleProperties[i]);
>         
>         return ArrayValues(object, butterfly);
>     }
>Index: Source/JavaScriptCore/heap/ConservativeRoots.cpp
>===================================================================
>--- Source/JavaScriptCore/heap/ConservativeRoots.cpp	(revision 203920)
>+++ Source/JavaScriptCore/heap/ConservativeRoots.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2011 Apple Inc. All rights reserved.
>+ * Copyright (C) 2011, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -31,6 +31,7 @@
> #include "CopiedSpace.h"
> #include "CopiedSpaceInlines.h"
> #include "HeapInlines.h"
>+#include "HeapUtil.h"
> #include "JSCell.h"
> #include "JSObject.h"
> #include "JSCInlines.h"
>@@ -51,16 +52,16 @@ ConservativeRoots::ConservativeRoots(Mar
> ConservativeRoots::~ConservativeRoots()
> {
>     if (m_roots != m_inlineRoots)
>-        OSAllocator::decommitAndRelease(m_roots, m_capacity * sizeof(JSCell*));
>+        OSAllocator::decommitAndRelease(m_roots, m_capacity * sizeof(HeapCell*));
> }
> 
> void ConservativeRoots::grow()
> {
>     size_t newCapacity = m_capacity == inlineCapacity ? nonInlineCapacity : m_capacity * 2;
>-    JSCell** newRoots = static_cast<JSCell**>(OSAllocator::reserveAndCommit(newCapacity * sizeof(JSCell*)));
>-    memcpy(newRoots, m_roots, m_size * sizeof(JSCell*));
>+    HeapCell** newRoots = static_cast<HeapCell**>(OSAllocator::reserveAndCommit(newCapacity * sizeof(HeapCell*)));
>+    memcpy(newRoots, m_roots, m_size * sizeof(HeapCell*));
>     if (m_roots != m_inlineRoots)
>-        OSAllocator::decommitAndRelease(m_roots, m_capacity * sizeof(JSCell*));
>+        OSAllocator::decommitAndRelease(m_roots, m_capacity * sizeof(HeapCell*));
>     m_capacity = newCapacity;
>     m_roots = newRoots;
> }
>@@ -72,13 +73,14 @@ inline void ConservativeRoots::genericAd
> 
>     m_copiedSpace->pinIfNecessary(p);
> 
>-    if (!Heap::isPointerGCObject(filter, *m_blocks, p))
>-        return;
>-
>-    if (m_size == m_capacity)
>-        grow();
>-
>-    m_roots[m_size++] = static_cast<JSCell*>(p);
>+    HeapUtil::findGCObjectPointers(
>+        filter, *m_blocks, p,
>+        [&] (void* p) {
>+            if (m_size == m_capacity)
>+                grow();
>+            
>+            m_roots[m_size++] = bitwise_cast<HeapCell*>(p);
>+        });
> }
> 
> template<typename MarkHook>
>Index: Source/JavaScriptCore/heap/ConservativeRoots.h
>===================================================================
>--- Source/JavaScriptCore/heap/ConservativeRoots.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/ConservativeRoots.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2009 Apple Inc. All rights reserved.
>+ * Copyright (C) 2009, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -31,8 +31,8 @@
> namespace JSC {
> 
> class CodeBlockSet;
>+class HeapCell;
> class JITStubRoutineSet;
>-class JSCell;
> 
> class ConservativeRoots {
> public:
>@@ -44,11 +44,11 @@ public:
>     void add(void* begin, void* end, JITStubRoutineSet&, CodeBlockSet&);
>     
>     size_t size();
>-    JSCell** roots();
>+    HeapCell** roots();
> 
> private:
>     static const size_t inlineCapacity = 128;
>-    static const size_t nonInlineCapacity = 8192 / sizeof(JSCell*);
>+    static const size_t nonInlineCapacity = 8192 / sizeof(HeapCell*);
>     
>     template<typename MarkHook>
>     void genericAddPointer(void*, TinyBloomFilter, MarkHook&);
>@@ -58,12 +58,12 @@ private:
>     
>     void grow();
> 
>-    JSCell** m_roots;
>+    HeapCell** m_roots;
>     size_t m_size;
>     size_t m_capacity;
>     MarkedBlockSet* m_blocks;
>     CopiedSpace* m_copiedSpace;
>-    JSCell* m_inlineRoots[inlineCapacity];
>+    HeapCell* m_inlineRoots[inlineCapacity];
> };
> 
> inline size_t ConservativeRoots::size()
>@@ -71,7 +71,7 @@ inline size_t ConservativeRoots::size()
>     return m_size;
> }
> 
>-inline JSCell** ConservativeRoots::roots()
>+inline HeapCell** ConservativeRoots::roots()
> {
>     return m_roots;
> }
>Index: Source/JavaScriptCore/heap/CopyToken.h
>===================================================================
>--- Source/JavaScriptCore/heap/CopyToken.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/CopyToken.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2013, 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2013, 2015-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -29,7 +29,6 @@
> namespace JSC {
> 
> enum CopyToken {
>-    ButterflyCopyToken,
>     TypedArrayVectorCopyToken,
>     MapBackingStoreCopyToken,
>     DirectArgumentsOverridesCopyToken
>Index: Source/JavaScriptCore/heap/Heap.cpp
>===================================================================
>--- Source/JavaScriptCore/heap/Heap.cpp	(revision 203920)
>+++ Source/JavaScriptCore/heap/Heap.cpp	(working copy)
>@@ -598,12 +598,6 @@ void Heap::copyBackingStores()
>                         
>                         CopyWorkList& workList = block->workList();
>                         for (CopyWorklistItem item : workList) {
>-                            if (item.token() == ButterflyCopyToken) {
>-                                JSObject::copyBackingStore(
>-                                    item.cell(), copyVisitor, ButterflyCopyToken);
>-                                continue;
>-                            }
>-                            
>                             item.cell()->methodTable()->copyBackingStore(
>                                 item.cell(), copyVisitor, item.token());
>                         }
>Index: Source/JavaScriptCore/heap/Heap.h
>===================================================================
>--- Source/JavaScriptCore/heap/Heap.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/Heap.h	(working copy)
>@@ -83,13 +83,15 @@ typedef HashCountedSet<const char*> Type
> 
> enum HeapType { SmallHeap, LargeHeap };
> 
>+class HeapUtil;
>+
> class Heap {
>     WTF_MAKE_NONCOPYABLE(Heap);
> public:
>     friend class JIT;
>     friend class DFG::SpeculativeJIT;
>     static Heap* heap(const JSValue); // 0 for immediate values
>-    static Heap* heap(const JSCell*);
>+    static Heap* heap(const HeapCell*);
> 
>     // This constant determines how many blocks we iterate between checks of our 
>     // deadline when calling Heap::isPagedOut. Decreasing it will cause us to detect 
>@@ -102,11 +104,6 @@ public:
>     static bool testAndSetMarked(const void*);
>     static void setMarked(const void*);
> 
>-    // This function must be run after stopAllocation() is called and 
>-    // before liveness data is cleared to be accurate.
>-    static bool isPointerGCObject(TinyBloomFilter, MarkedBlockSet&, void* pointer);
>-    static bool isValueGCObject(TinyBloomFilter, MarkedBlockSet&, JSValue);
>-
>     void writeBarrier(const JSCell*);
>     void writeBarrier(const JSCell*, JSValue);
>     void writeBarrier(const JSCell*, JSCell*);
>@@ -150,7 +147,11 @@ public:
>     MarkedAllocator& allocatorForObjectWithoutDestructor(size_t bytes) { return m_objectSpace.allocatorFor(bytes); }
>     MarkedAllocator& allocatorForObjectWithDestructor(size_t bytes) { return m_objectSpace.destructorAllocatorFor(bytes); }
>     template<typename ClassType> MarkedAllocator& allocatorForObjectOfType(size_t bytes);
>+    MarkedAllocator& allocatorForAuxiliaryData(size_t bytes) { return m_objectSpace.auxiliaryAllocatorFor(bytes); }
>     CopiedAllocator& storageAllocator() { return m_storageSpace.allocator(); }
>+    void* allocateAuxiliary(JSCell* intendedOwner, size_t);
>+    void* tryAllocateAuxiliary(JSCell* intendedOwner, size_t);
>+    void* tryReallocateAuxiliary(JSCell* intendedOwner, void* oldBase, size_t oldSize, size_t newSize);
>     CheckedBoolean tryAllocateStorage(JSCell* intendedOwner, size_t, void**);
>     CheckedBoolean tryReallocateStorage(JSCell* intendedOwner, void**, size_t, size_t);
>     void ascribeOwner(JSCell* intendedOwner, void*);
>@@ -267,6 +268,7 @@ private:
>     friend class GCLogging;
>     friend class GCThread;
>     friend class HandleSet;
>+    friend class HeapUtil;
>     friend class HeapVerifier;
>     friend class JITStubRoutine;
>     friend class LLIntOffsetsExtractor;
>Index: Source/JavaScriptCore/heap/HeapInlines.h
>===================================================================
>--- Source/JavaScriptCore/heap/HeapInlines.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/HeapInlines.h	(working copy)
>@@ -28,6 +28,7 @@
> 
> #include "CopyBarrier.h"
> #include "Heap.h"
>+#include "IndexingHeader.h"
> #include "JSCell.h"
> #include "Structure.h"
> #include <type_traits>
>@@ -59,7 +60,7 @@ inline bool Heap::isCollecting()
>     return m_operationInProgress == FullCollection || m_operationInProgress == EdenCollection;
> }
> 
>-inline Heap* Heap::heap(const JSCell* cell)
>+inline Heap* Heap::heap(const HeapCell* cell)
> {
>     return MarkedBlock::blockFor(cell)->heap();
> }
>@@ -199,7 +200,7 @@ inline void* Heap::allocateWithoutDestru
> }
> 
> template<typename ClassType>
>-void* Heap::allocateObjectOfType(size_t bytes)
>+inline void* Heap::allocateObjectOfType(size_t bytes)
> {
>     // JSCell::classInfo() expects objects allocated with normal destructor to derive from JSDestructibleObject.
>     ASSERT((!ClassType::needsDestruction || (ClassType::StructureFlags & StructureIsImmortal) || std::is_convertible<ClassType, JSDestructibleObject>::value));
>@@ -210,7 +211,7 @@ void* Heap::allocateObjectOfType(size_t 
> }
> 
> template<typename ClassType>
>-MarkedSpace::Subspace& Heap::subspaceForObjectOfType()
>+inline MarkedSpace::Subspace& Heap::subspaceForObjectOfType()
> {
>     // JSCell::classInfo() expects objects allocated with normal destructor to derive from JSDestructibleObject.
>     ASSERT((!ClassType::needsDestruction || (ClassType::StructureFlags & StructureIsImmortal) || std::is_convertible<ClassType, JSDestructibleObject>::value));
>@@ -221,7 +222,7 @@ MarkedSpace::Subspace& Heap::subspaceFor
> }
> 
> template<typename ClassType>
>-MarkedAllocator& Heap::allocatorForObjectOfType(size_t bytes)
>+inline MarkedAllocator& Heap::allocatorForObjectOfType(size_t bytes)
> {
>     // JSCell::classInfo() expects objects allocated with normal destructor to derive from JSDestructibleObject.
>     ASSERT((!ClassType::needsDestruction || (ClassType::StructureFlags & StructureIsImmortal) || std::is_convertible<ClassType, JSDestructibleObject>::value));
>@@ -231,6 +232,37 @@ MarkedAllocator& Heap::allocatorForObjec
>     return allocatorForObjectWithoutDestructor(bytes);
> }
> 
>+inline void* Heap::allocateAuxiliary(JSCell* intendedOwner, size_t bytes)
>+{
>+    void* result = m_objectSpace.allocateAuxiliary(bytes);
>+#if ENABLE(ALLOCATION_LOGGING)
>+    dataLogF("JSC GC allocating %lu bytes of auxiliary for %p: %p.\n", bytes, intendedOwner, result);
>+#else
>+    UNUSED_PARAM(intendedOwner);
>+#endif
>+    return result;
>+}
>+
>+inline void* Heap::tryAllocateAuxiliary(JSCell* intendedOwner, size_t bytes)
>+{
>+    void* result = m_objectSpace.tryAllocateAuxiliary(bytes);
>+#if ENABLE(ALLOCATION_LOGGING)
>+    dataLogF("JSC GC allocating %lu bytes of auxiliary for %p: %p.\n", bytes, intendedOwner, result);
>+#else
>+    UNUSED_PARAM(intendedOwner);
>+#endif
>+    return result;
>+}
>+
>+inline void* Heap::tryReallocateAuxiliary(JSCell* intendedOwner, void* oldBase, size_t oldSize, size_t newSize)
>+{
>+    void* newBase = tryAllocateAuxiliary(intendedOwner, newSize);
>+    if (!newBase)
>+        return nullptr;
>+    memcpy(newBase, oldBase, oldSize);
>+    return newBase;
>+}
>+
> inline CheckedBoolean Heap::tryAllocateStorage(JSCell* intendedOwner, size_t bytes, void** outPtr)
> {
>     CheckedBoolean result = m_storageSpace.tryAllocate(bytes, outPtr);
>@@ -354,33 +386,6 @@ inline void Heap::didFreeBlock(size_t ca
> #endif
> }
> 
>-inline bool Heap::isPointerGCObject(TinyBloomFilter filter, MarkedBlockSet& markedBlockSet, void* pointer)
>-{
>-    MarkedBlock* candidate = MarkedBlock::blockFor(pointer);
>-    if (filter.ruleOut(bitwise_cast<Bits>(candidate))) {
>-        ASSERT(!candidate || !markedBlockSet.set().contains(candidate));
>-        return false;
>-    }
>-
>-    if (!MarkedBlock::isAtomAligned(pointer))
>-        return false;
>-
>-    if (!markedBlockSet.set().contains(candidate))
>-        return false;
>-
>-    if (!candidate->isLiveCell(pointer))
>-        return false;
>-
>-    return true;
>-}
>-
>-inline bool Heap::isValueGCObject(TinyBloomFilter filter, MarkedBlockSet& markedBlockSet, JSValue value)
>-{
>-    if (!value.isCell())
>-        return false;
>-    return isPointerGCObject(filter, markedBlockSet, static_cast<void*>(value.asCell()));
>-}
>-
> } // namespace JSC
> 
> #endif // HeapInlines_h
>Index: Source/JavaScriptCore/heap/HeapUtil.h
>===================================================================
>--- Source/JavaScriptCore/heap/HeapUtil.h	(nonexistent)
>+++ Source/JavaScriptCore/heap/HeapUtil.h	(working copy)
>@@ -0,0 +1,135 @@
>+/*
>+ * Copyright (C) 2016 Apple Inc. All rights reserved.
>+ *
>+ * Redistribution and use in source and binary forms, with or without
>+ * modification, are permitted provided that the following conditions
>+ * are met:
>+ * 1. Redistributions of source code must retain the above copyright
>+ *    notice, this list of conditions and the following disclaimer.
>+ * 2. Redistributions in binary form must reproduce the above copyright
>+ *    notice, this list of conditions and the following disclaimer in the
>+ *    documentation and/or other materials provided with the distribution.
>+ *
>+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
>+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
>+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>+ * THE POSSIBILITY OF SUCH DAMAGE.
>+ */
>+
>+#pragma once
>+
>+namespace JSC {
>+
>+// Are you tired of waiting for all of WebKit to build because you changed the implementation of a
>+// function in HeapInlines.h?  Does it bother you that you're waiting on rebuilding the JS DOM
>+// bindings even though your change is in a function called from only 2 .cpp files?  Then HeapUtil.h
>+// is for you!  Everything in this class should be a static method that takes a Heap& if needed.
>+// This is a friend of Heap, so you can access all of Heap's privates.
>+//
>+// This ends up being an issue because Heap exposes a lot of methods that ought to be inline for
>+// performance or that must be inline because they are templates.  This class ought to contain
>+// methods that are used for the implementation of the collector, or for unusual clients that need
>+// to reach deep into the collector for some reason.  Don't put things in here that would cause you
>+// to have to include it from more than a handful of places, since that would defeat the purpose.
>+// This class isn't here to look pretty.  It's to let us hack the GC more easily!
>+
>+class HeapUtil {
>+public:
>+    // This function must be run after stopAllocation() is called and 
>+    // before liveness data is cleared to be accurate.
>+    template<typename Func>
>+    static void findGCObjectPointers(
>+        TinyBloomFilter filter, MarkedBlockSet& markedBlockSet, void* passedPointer,
>+        const Func& func)
>+    {
>+        char* pointer = static_cast<char*>(passedPointer);
>+    
>+        MarkedBlock* candidate = MarkedBlock::blockFor(pointer);
>+        // It's possible for a butterfly pointer to point past the end of a butterfly. Check this now.
>+        if (pointer <= bitwise_cast<char*>(candidate) + sizeof(IndexingHeader)) {
>+            // We may be interested in the last cell of the previous MarkedBlock.
>+            char* previousPointer = pointer - sizeof(IndexingHeader) - 1;
>+            MarkedBlock* previousCandidate = MarkedBlock::blockFor(previousPointer);
>+            if (!filter.ruleOut(bitwise_cast<Bits>(previousCandidate))
>+                && markedBlockSet.set().contains(previousCandidate)
>+                && previousCandidate->cellKind() == HeapCell::Auxiliary) {
>+                previousPointer = static_cast<char*>(previousCandidate->cellAlign(previousPointer));
>+                if (previousCandidate->isLiveCell(previousPointer))
>+                    func(previousPointer);
>+            }
>+        }
>+    
>+        if (filter.ruleOut(bitwise_cast<Bits>(candidate))) {
>+            ASSERT(!candidate || !markedBlockSet.set().contains(candidate));
>+            return;
>+        }
>+    
>+        if (!markedBlockSet.set().contains(candidate))
>+            return;
>+
>+        auto tryPointer = [&] (void* pointer) {
>+            if (candidate->isLiveCell(pointer))
>+                func(pointer);
>+        };
>+    
>+        if (candidate->cellKind() == HeapCell::JSCell) {
>+            if (!MarkedBlock::isAtomAligned(pointer))
>+                return;
>+        
>+            tryPointer(pointer);
>+            return;
>+        }
>+    
>+        // A butterfly could point into the middle of an object.
>+        char* alignedPointer = static_cast<char*>(candidate->cellAlign(pointer));
>+        tryPointer(alignedPointer);
>+    
>+        // Also, a butterfly could point at the end of an object plus sizeof(IndexingHeader). In that
>+        // case, this is pointing to the object to the right of the one we should be marking.
>+        if (candidate->atomNumber(alignedPointer) > MarkedBlock::firstAtom()
>+            && pointer <= alignedPointer + sizeof(IndexingHeader))
>+            tryPointer(alignedPointer - candidate->cellSize());
>+    }
>+    
>+    static bool isPointerGCObjectJSCell(
>+        TinyBloomFilter filter, MarkedBlockSet& markedBlockSet, const void* pointer)
>+    {
>+        MarkedBlock* candidate = MarkedBlock::blockFor(pointer);
>+        if (filter.ruleOut(bitwise_cast<Bits>(candidate))) {
>+            ASSERT(!candidate || !markedBlockSet.set().contains(candidate));
>+            return false;
>+        }
>+        
>+        if (!MarkedBlock::isAtomAligned(pointer))
>+            return false;
>+        
>+        if (!markedBlockSet.set().contains(candidate))
>+            return false;
>+        
>+        if (candidate->cellKind() != HeapCell::JSCell)
>+            return false;
>+        
>+        if (!candidate->isLiveCell(pointer))
>+            return false;
>+        
>+        return true;
>+    }
>+    
>+    static bool isValueGCObject(
>+        TinyBloomFilter filter, MarkedBlockSet& markedBlockSet, JSValue value)
>+    {
>+        if (!value.isCell())
>+            return false;
>+        return isPointerGCObjectJSCell(filter, markedBlockSet, static_cast<void*>(value.asCell()));
>+    }
>+};
>+
>+} // namespace JSC
>+
>Index: Source/JavaScriptCore/heap/MarkedAllocator.cpp
>===================================================================
>--- Source/JavaScriptCore/heap/MarkedAllocator.cpp	(revision 203920)
>+++ Source/JavaScriptCore/heap/MarkedAllocator.cpp	(working copy)
>@@ -66,7 +66,7 @@ void MarkedAllocator::retire(MarkedBlock
>     block->didRetireBlock(freeList);
> }
> 
>-inline void* MarkedAllocator::tryAllocateHelper(size_t bytes)
>+inline void* MarkedAllocator::tryAllocateWithoutCollectingImpl(size_t bytes)
> {
>     if (m_currentBlock) {
>         ASSERT(m_currentBlock == m_nextBlockToSweep);
>@@ -120,11 +120,11 @@ inline void* MarkedAllocator::tryPopFree
>     return head;
> }
> 
>-inline void* MarkedAllocator::tryAllocate(size_t bytes)
>+inline void* MarkedAllocator::tryAllocateWithoutCollecting(size_t bytes)
> {
>     ASSERT(!m_heap->isBusy());
>     m_heap->m_operationInProgress = Allocation;
>-    void* result = tryAllocateHelper(bytes);
>+    void* result = tryAllocateWithoutCollectingImpl(bytes);
> 
>     m_heap->m_operationInProgress = NoOperation;
>     ASSERT(result || !m_currentBlock);
>@@ -148,6 +148,18 @@ ALWAYS_INLINE void MarkedAllocator::doTe
> 
> void* MarkedAllocator::allocateSlowCase(size_t bytes)
> {
>+    bool crashOnFailure = true;
>+    return allocateSlowCaseImpl(bytes, crashOnFailure);
>+}
>+
>+void* MarkedAllocator::tryAllocateSlowCase(size_t bytes)
>+{
>+    bool crashOnFailure = false;
>+    return allocateSlowCaseImpl(bytes, crashOnFailure);
>+}
>+
>+void* MarkedAllocator::allocateSlowCaseImpl(size_t bytes, bool crashOnFailure)
>+{
>     ASSERT(m_heap->vm()->currentThreadIsHoldingAPILock());
>     doTestCollectionsIfNeeded();
> 
>@@ -155,29 +167,34 @@ void* MarkedAllocator::allocateSlowCase(
>     ASSERT(!m_freeList.head);
>     m_heap->didAllocate(m_freeList.bytes);
>     
>-    void* result = tryAllocate(bytes);
>+    void* result = tryAllocateWithoutCollecting(bytes);
>     
>     if (LIKELY(result != 0))
>         return result;
>     
>     if (m_heap->collectIfNecessaryOrDefer()) {
>-        result = tryAllocate(bytes);
>+        result = tryAllocateWithoutCollecting(bytes);
>         if (result)
>             return result;
>     }
> 
>     ASSERT(!m_heap->shouldCollect());
>     
>-    MarkedBlock* block = allocateBlock(bytes);
>-    ASSERT(block);
>+    MarkedBlock* block = tryAllocateBlock(bytes);
>+    if (!block) {
>+        if (crashOnFailure)
>+            RELEASE_ASSERT_NOT_REACHED();
>+        else
>+            return nullptr;
>+    }
>     addBlock(block);
>         
>-    result = tryAllocate(bytes);
>+    result = tryAllocateWithoutCollecting(bytes);
>     ASSERT(result);
>     return result;
> }
> 
>-MarkedBlock* MarkedAllocator::allocateBlock(size_t bytes)
>+MarkedBlock* MarkedAllocator::tryAllocateBlock(size_t bytes)
> {
>     size_t minBlockSize = MarkedBlock::blockSize;
>     size_t minAllocationSize = WTF::roundUpToMultipleOf<MarkedBlock::atomSize>(sizeof(MarkedBlock)) + WTF::roundUpToMultipleOf<MarkedBlock::atomSize>(bytes);
>@@ -186,7 +203,7 @@ MarkedBlock* MarkedAllocator::allocateBl
> 
>     size_t cellSize = m_cellSize ? m_cellSize : WTF::roundUpToMultipleOf<MarkedBlock::atomSize>(bytes);
> 
>-    return MarkedBlock::create(*m_heap, this, blockSize, cellSize, m_attributes);
>+    return MarkedBlock::tryCreate(*m_heap, this, blockSize, cellSize, m_attributes);
> }
> 
> void MarkedAllocator::addBlock(MarkedBlock* block)
>Index: Source/JavaScriptCore/heap/MarkedAllocator.h
>===================================================================
>--- Source/JavaScriptCore/heap/MarkedAllocator.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/MarkedAllocator.h	(working copy)
>@@ -53,6 +53,7 @@ public:
>     DestructionMode destruction() const { return m_attributes.destruction; }
>     HeapCell::Kind cellKind() const { return m_attributes.cellKind; }
>     void* allocate(size_t);
>+    void* tryAllocate(size_t);
>     Heap* heap() { return m_heap; }
>     MarkedBlock* takeLastActiveBlock()
>     {
>@@ -71,10 +72,12 @@ public:
>    
> private:
>     JS_EXPORT_PRIVATE void* allocateSlowCase(size_t);
>-    void* tryAllocate(size_t);
>-    void* tryAllocateHelper(size_t);
>+    JS_EXPORT_PRIVATE void* tryAllocateSlowCase(size_t);
>+    void* allocateSlowCaseImpl(size_t, bool crashOnFailure);
>+    void* tryAllocateWithoutCollecting(size_t);
>+    void* tryAllocateWithoutCollectingImpl(size_t);
>     void* tryPopFreeList(size_t);
>-    MarkedBlock* allocateBlock(size_t);
>+    MarkedBlock* tryAllocateBlock(size_t);
>     ALWAYS_INLINE void doTestCollectionsIfNeeded();
>     void retire(MarkedBlock*, MarkedBlock::FreeList&);
>     
>@@ -113,21 +116,35 @@ inline void MarkedAllocator::init(Heap* 
>     m_attributes = attributes;
> }
> 
>+inline void* MarkedAllocator::tryAllocate(size_t bytes)
>+{
>+    MarkedBlock::FreeCell* head = m_freeList.head;
>+    if (UNLIKELY(!head)) {
>+        void* result = tryAllocateSlowCase(bytes);
>+        if (result && !ASSERT_DISABLED)
>+            memset(result, 0xCD, bytes);
>+        return result;
>+    }
>+    
>+    m_freeList.head = head->next;
>+    if (!ASSERT_DISABLED)
>+        memset(head, 0xCD, bytes);
>+    return head;
>+}
>+
> inline void* MarkedAllocator::allocate(size_t bytes)
> {
>     MarkedBlock::FreeCell* head = m_freeList.head;
>     if (UNLIKELY(!head)) {
>         void* result = allocateSlowCase(bytes);
>-#ifndef NDEBUG
>-        memset(result, 0xCD, bytes);
>-#endif
>+        if (!ASSERT_DISABLED)
>+            memset(result, 0xCD, bytes);
>         return result;
>     }
>     
>     m_freeList.head = head->next;
>-#ifndef NDEBUG
>-    memset(head, 0xCD, bytes);
>-#endif
>+    if (!ASSERT_DISABLED)
>+        memset(head, 0xCD, bytes);
>     return head;
> }
> 
>Index: Source/JavaScriptCore/heap/MarkedBlock.cpp
>===================================================================
>--- Source/JavaScriptCore/heap/MarkedBlock.cpp	(revision 203920)
>+++ Source/JavaScriptCore/heap/MarkedBlock.cpp	(working copy)
>@@ -35,14 +35,27 @@ namespace JSC {
> static const bool computeBalance = false;
> static size_t balance;
> 
>-MarkedBlock* MarkedBlock::create(Heap& heap, MarkedAllocator* allocator, size_t capacity, size_t cellSize, const AllocatorAttributes& attributes)
>+static void resetBadly(void* base, size_t size)
>+{
>+    for (size_t i = size / sizeof(EncodedJSValue); i--;) {
>+        static_cast<EncodedJSValue*>(base)[i] = JSValue::encode(
>+            bitwise_cast<JSCell*>(static_cast<intptr_t>(666 * 8)));
>+    }
>+}
>+
>+MarkedBlock* MarkedBlock::tryCreate(Heap& heap, MarkedAllocator* allocator, size_t capacity, size_t cellSize, const AllocatorAttributes& attributes)
> {
>     if (computeBalance) {
>         balance++;
>         if (!(balance % 10))
>             dataLog("MarkedBlock Balance: ", balance, "\n");
>     }
>-    MarkedBlock* block = new (NotNull, fastAlignedMalloc(blockSize, capacity)) MarkedBlock(allocator, capacity, cellSize, attributes);
>+    void* blockSpace = tryFastAlignedMalloc(blockSize, capacity);
>+    if (!blockSpace)
>+        return nullptr;
>+    if (Options::resetFreeCellsBadly())
>+        resetBadly(blockSpace, capacity);
>+    MarkedBlock* block = new (NotNull, blockSpace) MarkedBlock(allocator, capacity, cellSize, attributes);
>     heap.didAllocateBlock(capacity);
>     return block;
> }
>@@ -92,11 +105,11 @@ inline void MarkedBlock::callDestructor(
>     cell->zap();
> }
> 
>-template<MarkedBlock::BlockState blockState, MarkedBlock::SweepMode sweepMode, bool callDestructors>
>+template<MarkedBlock::BlockState blockState, MarkedBlock::SweepMode sweepMode, DestructionMode destructionMode, MarkedBlock::ResetMode resetMode>
> MarkedBlock::FreeList MarkedBlock::specializedSweep()
> {
>     ASSERT(blockState != Allocated && blockState != FreeListed);
>-    ASSERT(!(!callDestructors && sweepMode == SweepOnly));
>+    ASSERT(!(destructionMode == DoesNotNeedDestruction && sweepMode == SweepOnly));
> 
>     // This produces a free list that is ordered in reverse through the block.
>     // This is fine, since the allocation code makes no assumptions about the
>@@ -109,11 +122,13 @@ MarkedBlock::FreeList MarkedBlock::speci
> 
>         HeapCell* cell = reinterpret_cast_ptr<HeapCell*>(&atoms()[i]);
> 
>-        if (callDestructors && blockState != New)
>+        if (destructionMode == NeedsDestruction && blockState != New)
>             callDestructor(cell);
> 
>         if (sweepMode == SweepToFreeList) {
>             FreeCell* freeCell = reinterpret_cast<FreeCell*>(cell);
>+            if (resetMode == ResetBadly)
>+                resetBadly(freeCell, cellSize());
>             freeCell->next = head;
>             head = freeCell;
>             ++count;
>@@ -139,17 +154,25 @@ MarkedBlock::FreeList MarkedBlock::sweep
>         return FreeList();
> 
>     if (m_attributes.destruction == NeedsDestruction)
>-        return sweepHelper<true>(sweepMode);
>-    return sweepHelper<false>(sweepMode);
>+        return sweepHelperSelectResetMode<NeedsDestruction>(sweepMode);
>+    return sweepHelperSelectResetMode<DoesNotNeedDestruction>(sweepMode);
>+}
>+
>+template<DestructionMode destructionMode>
>+MarkedBlock::FreeList MarkedBlock::sweepHelperSelectResetMode(SweepMode sweepMode)
>+{
>+    if (Options::resetFreeCellsBadly())
>+        return sweepHelperSelectStateAndSweepMode<destructionMode, ResetBadly>(sweepMode);
>+    return sweepHelperSelectStateAndSweepMode<destructionMode, DontReset>(sweepMode);
> }
> 
>-template<bool callDestructors>
>-MarkedBlock::FreeList MarkedBlock::sweepHelper(SweepMode sweepMode)
>+template<DestructionMode destructionMode, MarkedBlock::ResetMode resetMode>
>+MarkedBlock::FreeList MarkedBlock::sweepHelperSelectStateAndSweepMode(SweepMode sweepMode)
> {
>     switch (m_state) {
>     case New:
>         ASSERT(sweepMode == SweepToFreeList);
>-        return specializedSweep<New, SweepToFreeList, callDestructors>();
>+        return specializedSweep<New, SweepToFreeList, destructionMode, resetMode>();
>     case FreeListed:
>         // Happens when a block transitions to fully allocated.
>         ASSERT(sweepMode == SweepToFreeList);
>@@ -160,8 +183,8 @@ MarkedBlock::FreeList MarkedBlock::sweep
>         return FreeList();
>     case Marked:
>         return sweepMode == SweepToFreeList
>-            ? specializedSweep<Marked, SweepToFreeList, callDestructors>()
>-            : specializedSweep<Marked, SweepOnly, callDestructors>();
>+            ? specializedSweep<Marked, SweepToFreeList, destructionMode, resetMode>()
>+            : specializedSweep<Marked, SweepOnly, destructionMode, resetMode>();
>     }
> 
>     RELEASE_ASSERT_NOT_REACHED();
>Index: Source/JavaScriptCore/heap/MarkedBlock.h
>===================================================================
>--- Source/JavaScriptCore/heap/MarkedBlock.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/MarkedBlock.h	(working copy)
>@@ -109,12 +109,14 @@ namespace JSC {
>             mutable ReturnType m_count;
>         };
> 
>-        static MarkedBlock* create(Heap&, MarkedAllocator*, size_t capacity, size_t cellSize, const AllocatorAttributes&);
>+        static MarkedBlock* tryCreate(Heap&, MarkedAllocator*, size_t capacity, size_t cellSize, const AllocatorAttributes&);
>         static void destroy(Heap&, MarkedBlock*);
> 
>         static bool isAtomAligned(const void*);
>+        void* cellAlign(void*);
>         static MarkedBlock* blockFor(const void*);
>         static size_t firstAtom();
>+        size_t atomNumber(const void*);
>         
>         void lastChanceToFinalize();
> 
>@@ -184,15 +186,16 @@ namespace JSC {
>         static const size_t atomAlignmentMask = atomSize - 1;
> 
>         enum BlockState { New, FreeListed, Allocated, Marked, Retired };
>-        template<bool callDestructors> FreeList sweepHelper(SweepMode = SweepOnly);
>+        template<DestructionMode> FreeList sweepHelperSelectResetMode(SweepMode = SweepOnly);
>+        enum ResetMode { DontReset, ResetBadly };
>+        template<DestructionMode, ResetMode> FreeList sweepHelperSelectStateAndSweepMode(SweepMode = SweepOnly);
> 
>         typedef char Atom[atomSize];
> 
>         MarkedBlock(MarkedAllocator*, size_t capacity, size_t cellSize, const AllocatorAttributes&);
>         Atom* atoms();
>-        size_t atomNumber(const void*);
>         void callDestructor(HeapCell*);
>-        template<BlockState, SweepMode, bool callDestructors> FreeList specializedSweep();
>+        template<BlockState, SweepMode, DestructionMode, ResetMode> FreeList specializedSweep();
>         
>         MarkedBlock* m_prev;
>         MarkedBlock* m_next;
>@@ -236,6 +239,16 @@ namespace JSC {
>         return !(reinterpret_cast<Bits>(p) & atomAlignmentMask);
>     }
> 
>+    inline void* MarkedBlock::cellAlign(void* p)
>+    {
>+        Bits base = reinterpret_cast<Bits>(atoms() + firstAtom());
>+        Bits bits = reinterpret_cast<Bits>(p);
>+        bits -= base;
>+        bits -= bits % cellSize();
>+        bits += base;
>+        return reinterpret_cast<void*>(bits);
>+    }
>+
>     inline MarkedBlock* MarkedBlock::blockFor(const void* p)
>     {
>         return reinterpret_cast<MarkedBlock*>(reinterpret_cast<Bits>(p) & blockMask);
>Index: Source/JavaScriptCore/heap/MarkedSpace.h
>===================================================================
>--- Source/JavaScriptCore/heap/MarkedSpace.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/MarkedSpace.h	(working copy)
>@@ -68,6 +68,7 @@ public:
>     void* allocateWithDestructor(size_t);
>     void* allocateWithoutDestructor(size_t);
>     void* allocateAuxiliary(size_t);
>+    void* tryAllocateAuxiliary(size_t);
> 
>     Subspace& subspaceForObjectsWithDestructor() { return m_destructorSpace; }
>     Subspace& subspaceForObjectsWithoutDestructor() { return m_normalSpace; }
>@@ -182,6 +183,11 @@ inline void* MarkedSpace::allocateAuxili
>     return auxiliaryAllocatorFor(bytes).allocate(bytes);
> }
> 
>+inline void* MarkedSpace::tryAllocateAuxiliary(size_t bytes)
>+{
>+    return auxiliaryAllocatorFor(bytes).tryAllocate(bytes);
>+}
>+
> template <typename Functor> inline void MarkedSpace::forEachBlock(const Functor& functor)
> {
>     forEachSubspace(
>Index: Source/JavaScriptCore/heap/SlotVisitor.cpp
>===================================================================
>--- Source/JavaScriptCore/heap/SlotVisitor.cpp	(revision 203920)
>+++ Source/JavaScriptCore/heap/SlotVisitor.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2012, 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2012, 2015-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -118,10 +118,42 @@ void SlotVisitor::clearMarkStack()
> 
> void SlotVisitor::append(ConservativeRoots& conservativeRoots)
> {
>-    JSCell** roots = conservativeRoots.roots();
>+    HeapCell** roots = conservativeRoots.roots();
>     size_t size = conservativeRoots.size();
>     for (size_t i = 0; i < size; ++i)
>-        append(roots[i]);
>+        appendJSCellOrAuxiliary(roots[i]);
>+}
>+
>+void SlotVisitor::appendJSCellOrAuxiliary(HeapCell* heapCell)
>+{
>+    if (!heapCell)
>+        return;
>+    
>+    ASSERT(!m_isCheckingForDefaultMarkViolation);
>+    
>+    if (Heap::testAndSetMarked(heapCell))
>+        return;
>+    
>+    MarkedBlock* block = MarkedBlock::blockFor(heapCell);
>+    switch (block->cellKind()) {
>+    case HeapCell::JSCell: {
>+        JSCell* jsCell = static_cast<JSCell*>(heapCell);
>+        
>+        if (!jsCell->structure()) {
>+            ASSERT_NOT_REACHED();
>+            return;
>+        }
>+        
>+        jsCell->setCellState(CellState::NewGrey);
>+
>+        appendToMarkStack(block, jsCell);
>+        return;
>+    }
>+        
>+    case HeapCell::Auxiliary: {
>+        noteLiveAuxiliaryCell(block);
>+        return;
>+    } }
> }
> 
> void SlotVisitor::append(JSValue value)
>@@ -168,17 +200,36 @@ void SlotVisitor::setMarkedAndAppendToMa
> 
> void SlotVisitor::appendToMarkStack(JSCell* cell)
> {
>+    appendToMarkStack(MarkedBlock::blockFor(cell), cell);
>+}
>+
>+void SlotVisitor::appendToMarkStack(MarkedBlock* block, JSCell* cell)
>+{
>     ASSERT(Heap::isMarked(cell));
>     ASSERT(!cell->isZapped());
>-
>+    
>     m_visitCount++;
>-    m_bytesVisited += MarkedBlock::blockFor(cell)->cellSize();
>+    m_bytesVisited += block->cellSize();
>     m_stack.append(cell);
> 
>     if (UNLIKELY(m_heapSnapshotBuilder))
>         m_heapSnapshotBuilder->appendNode(cell);
> }
> 
>+void SlotVisitor::markAuxiliary(const void* base)
>+{
>+    if (Heap::testAndSetMarked(base))
>+        return;
>+    
>+    noteLiveAuxiliaryCell(MarkedBlock::blockFor(base));
>+}
>+
>+void SlotVisitor::noteLiveAuxiliaryCell(MarkedBlock* block)
>+{
>+    m_visitCount++;
>+    m_bytesVisited += block->cellSize();
>+}
>+
> class SetCurrentCellScope {
> public:
>     SetCurrentCellScope(SlotVisitor& visitor, const JSCell* cell)
>@@ -202,9 +253,9 @@ private:
> ALWAYS_INLINE void SlotVisitor::visitChildren(const JSCell* cell)
> {
>     ASSERT(Heap::isMarked(cell));
>-
>+    
>     SetCurrentCellScope currentCellScope(*this, cell);
>-
>+    
>     m_currentObjectCellStateBeforeVisiting = cell->cellState();
>     cell->setCellState(CellState::OldBlack);
>     
>Index: Source/JavaScriptCore/heap/SlotVisitor.h
>===================================================================
>--- Source/JavaScriptCore/heap/SlotVisitor.h	(revision 203920)
>+++ Source/JavaScriptCore/heap/SlotVisitor.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2011-2013, 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2011-2013, 2015-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -37,8 +37,10 @@ namespace JSC {
> class ConservativeRoots;
> class GCThreadSharedData;
> class Heap;
>+class HeapCell;
> class HeapSnapshotBuilder;
> template<typename T> class JITWriteBarrier;
>+class MarkedBlock;
> class UnconditionalFinalizer;
> template<typename T> class Weak;
> class WeakReferenceHarvester;
>@@ -105,6 +107,10 @@ public:
> 
>     void harvestWeakReferences();
>     void finalizeUnconditionalFinalizers();
>+    
>+    // This informs the GC about auxiliary of some size that we are keeping alive. If you don't do
>+    // this then the space will be freed at end of GC.
>+    void markAuxiliary(const void* base);
> 
>     void copyLater(JSCell*, CopyToken, void*, size_t);
>     
>@@ -124,10 +130,13 @@ private:
>     friend class ParallelModeEnabler;
>     
>     JS_EXPORT_PRIVATE void append(JSValue); // This is private to encourage clients to use WriteBarrier<T>.
>+    void appendJSCellOrAuxiliary(HeapCell*);
>     void appendHidden(JSValue);
> 
>     JS_EXPORT_PRIVATE void setMarkedAndAppendToMarkStack(JSCell*);
>     void appendToMarkStack(JSCell*);
>+    void appendToMarkStack(MarkedBlock*, JSCell*);
>+    void noteLiveAuxiliaryCell(MarkedBlock*);
>     
>     JS_EXPORT_PRIVATE void mergeOpaqueRoots();
>     void mergeOpaqueRootsIfNecessary();
>Index: Source/JavaScriptCore/inspector/JSJavaScriptCallFrame.cpp
>===================================================================
>--- Source/JavaScriptCore/inspector/JSJavaScriptCallFrame.cpp	(revision 203920)
>+++ Source/JavaScriptCore/inspector/JSJavaScriptCallFrame.cpp	(working copy)
>@@ -29,11 +29,9 @@
> #include "DebuggerScope.h"
> #include "Error.h"
> #include "IdentifierInlines.h"
>-#include "JSCJSValue.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSJavaScriptCallFramePrototype.h"
> #include "ObjectConstructor.h"
>-#include "StructureInlines.h"
> 
> using namespace JSC;
> 
>Index: Source/JavaScriptCore/jit/JITOperations.cpp
>===================================================================
>--- Source/JavaScriptCore/jit/JITOperations.cpp	(revision 203920)
>+++ Source/JavaScriptCore/jit/JITOperations.cpp	(working copy)
>@@ -472,17 +472,6 @@ void JIT_OPERATION operationPutByIdDirec
>         repatchPutByID(exec, baseObject, structure, ident, slot, *stubInfo, Direct);
> }
> 
>-void JIT_OPERATION operationReallocateStorageAndFinishPut(ExecState* exec, JSObject* base, Structure* structure, PropertyOffset offset, EncodedJSValue value)
>-{
>-    VM& vm = exec->vm();
>-    NativeCallFrameTracer tracer(&vm, exec);
>-
>-    ASSERT(structure->outOfLineCapacity() > base->structure(vm)->outOfLineCapacity());
>-    ASSERT(!vm.heap.storageAllocator().fastPathShouldSucceed(structure->outOfLineCapacity() * sizeof(JSValue)));
>-    base->setStructureAndReallocateStorageIfNecessary(vm, structure);
>-    base->putDirect(vm, offset, JSValue::decode(value));
>-}
>-
> ALWAYS_INLINE static bool isStringOrSymbol(JSValue value)
> {
>     return value.isString() || value.isSymbol();
>Index: Source/JavaScriptCore/jit/JITOperations.h
>===================================================================
>--- Source/JavaScriptCore/jit/JITOperations.h	(revision 203920)
>+++ Source/JavaScriptCore/jit/JITOperations.h	(working copy)
>@@ -320,7 +320,6 @@ void JIT_OPERATION operationPutByIdStric
> void JIT_OPERATION operationPutByIdNonStrictBuildList(ExecState*, StructureStubInfo*, EncodedJSValue encodedValue, EncodedJSValue encodedBase, UniquedStringImpl*) WTF_INTERNAL;
> void JIT_OPERATION operationPutByIdDirectStrictBuildList(ExecState*, StructureStubInfo*, EncodedJSValue encodedValue, EncodedJSValue encodedBase, UniquedStringImpl*) WTF_INTERNAL;
> void JIT_OPERATION operationPutByIdDirectNonStrictBuildList(ExecState*, StructureStubInfo*, EncodedJSValue encodedValue, EncodedJSValue encodedBase, UniquedStringImpl*) WTF_INTERNAL;
>-void JIT_OPERATION operationReallocateStorageAndFinishPut(ExecState*, JSObject*, Structure*, PropertyOffset, EncodedJSValue) WTF_INTERNAL;
> void JIT_OPERATION operationPutByValOptimize(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue, ByValInfo*) WTF_INTERNAL;
> void JIT_OPERATION operationDirectPutByValOptimize(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue, ByValInfo*) WTF_INTERNAL;
> void JIT_OPERATION operationPutByValGeneric(ExecState*, EncodedJSValue, EncodedJSValue, EncodedJSValue, ByValInfo*) WTF_INTERNAL;
>Index: Source/JavaScriptCore/parser/ModuleAnalyzer.cpp
>===================================================================
>--- Source/JavaScriptCore/parser/ModuleAnalyzer.cpp	(revision 203920)
>+++ Source/JavaScriptCore/parser/ModuleAnalyzer.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -26,9 +26,7 @@
> #include "config.h"
> #include "ModuleAnalyzer.h"
> 
>-#include "IdentifierInlines.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSGlobalObject.h"
> #include "JSModuleRecord.h"
> #include "StrongInlines.h"
>Index: Source/JavaScriptCore/runtime/AuxiliaryBarrier.h
>===================================================================
>--- Source/JavaScriptCore/runtime/AuxiliaryBarrier.h	(nonexistent)
>+++ Source/JavaScriptCore/runtime/AuxiliaryBarrier.h	(working copy)
>@@ -0,0 +1,63 @@
>+/*
>+ * Copyright (C) 2016 Apple Inc. All rights reserved.
>+ *
>+ * Redistribution and use in source and binary forms, with or without
>+ * modification, are permitted provided that the following conditions
>+ * are met:
>+ * 1. Redistributions of source code must retain the above copyright
>+ *    notice, this list of conditions and the following disclaimer.
>+ * 2. Redistributions in binary form must reproduce the above copyright
>+ *    notice, this list of conditions and the following disclaimer in the
>+ *    documentation and/or other materials provided with the distribution.
>+ *
>+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
>+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
>+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>+ * THE POSSIBILITY OF SUCH DAMAGE.
>+ */
>+
>+#pragma once
>+
>+namespace JSC {
>+
>+class JSCell;
>+class VM;
>+
>+// An Auxiliary barrier is a barrier that does not try to reason about the value being stored into
>+// it, other than interpreting a falsish value as not need a barrier. It's OK to use this for either
>+// JSCells or any other kind of data, so long as it responds to operator!().
>+template<typename T>
>+class AuxiliaryBarrier {
>+public:
>+    AuxiliaryBarrier() { }
>+    
>+    template<typename U>
>+    AuxiliaryBarrier(VM&, JSCell*, U&&);
>+    
>+    void clear() { m_value = T(); }
>+    
>+    template<typename U>
>+    void set(VM&, JSCell*, U&&);
>+    
>+    const T& get() const { return m_value; }
>+    
>+    T* slot() { return &m_value; }
>+    
>+    explicit operator bool() const { return !!m_value; }
>+    
>+    template<typename U>
>+    void setWithoutBarrier(U&& value) { m_value = std::forward<U>(value); }
>+    
>+private:
>+    T m_value;
>+};
>+
>+} // namespace JSC
>+
>Index: Source/JavaScriptCore/runtime/AuxiliaryBarrierInlines.h
>===================================================================
>--- Source/JavaScriptCore/runtime/AuxiliaryBarrierInlines.h	(nonexistent)
>+++ Source/JavaScriptCore/runtime/AuxiliaryBarrierInlines.h	(working copy)
>@@ -0,0 +1,50 @@
>+/*
>+ * Copyright (C) 2016 Apple Inc. All rights reserved.
>+ *
>+ * Redistribution and use in source and binary forms, with or without
>+ * modification, are permitted provided that the following conditions
>+ * are met:
>+ * 1. Redistributions of source code must retain the above copyright
>+ *    notice, this list of conditions and the following disclaimer.
>+ * 2. Redistributions in binary form must reproduce the above copyright
>+ *    notice, this list of conditions and the following disclaimer in the
>+ *    documentation and/or other materials provided with the distribution.
>+ *
>+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
>+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
>+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>+ * THE POSSIBILITY OF SUCH DAMAGE.
>+ */
>+
>+#pragma once
>+
>+#include "Heap.h"
>+#include "VM.h"
>+
>+namespace JSC {
>+
>+template<typename T>
>+template<typename U>
>+AuxiliaryBarrier<T>::AuxiliaryBarrier(VM& vm, JSCell* owner, U&& value)
>+{
>+    m_value = std::forward<U>(value);
>+    vm.heap.writeBarrier(owner);
>+}
>+
>+template<typename T>
>+template<typename U>
>+void AuxiliaryBarrier<T>::set(VM& vm, JSCell* owner, U&& value)
>+{
>+    m_value = std::forward<U>(value);
>+    vm.heap.writeBarrier(owner);
>+}
>+
>+} // namespace JSC
>+
>Index: Source/JavaScriptCore/runtime/ButterflyInlines.h
>===================================================================
>--- Source/JavaScriptCore/runtime/ButterflyInlines.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/ButterflyInlines.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2012 Apple Inc. All rights reserved.
>+ * Copyright (C) 2012, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -37,10 +37,9 @@ namespace JSC {
> 
> inline Butterfly* Butterfly::createUninitialized(VM& vm, JSCell* intendedOwner, size_t preCapacity, size_t propertyCapacity, bool hasIndexingHeader, size_t indexingPayloadSizeInBytes)
> {
>-    void* temp;
>     size_t size = totalSize(preCapacity, propertyCapacity, hasIndexingHeader, indexingPayloadSizeInBytes);
>-    RELEASE_ASSERT(vm.heap.tryAllocateStorage(intendedOwner, size, &temp));
>-    Butterfly* result = fromBase(temp, preCapacity, propertyCapacity);
>+    void* base = vm.heap.allocateAuxiliary(intendedOwner, size);
>+    Butterfly* result = fromBase(base, preCapacity, propertyCapacity);
>     return result;
> }
> 
>@@ -119,7 +118,8 @@ inline Butterfly* Butterfly::growArrayRi
>     void* theBase = base(0, propertyCapacity);
>     size_t oldSize = totalSize(0, propertyCapacity, hadIndexingHeader, oldIndexingPayloadSizeInBytes);
>     size_t newSize = totalSize(0, propertyCapacity, true, newIndexingPayloadSizeInBytes);
>-    if (!vm.heap.tryReallocateStorage(intendedOwner, &theBase, oldSize, newSize))
>+    theBase = vm.heap.tryReallocateAuxiliary(intendedOwner, theBase, oldSize, newSize);
>+    if (!theBase)
>         return 0;
>     return fromBase(theBase, 0, propertyCapacity);
> }
>Index: Source/JavaScriptCore/runtime/ClonedArguments.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/ClonedArguments.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/ClonedArguments.cpp	(working copy)
>@@ -48,12 +48,15 @@ ClonedArguments* ClonedArguments::create
>     if (vectorLength > MAX_STORAGE_VECTOR_LENGTH)
>         return 0;
> 
>-    void* temp;
>-    if (!vm.heap.tryAllocateStorage(0, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)), &temp))
>+    void* temp = vm.heap.tryAllocateAuxiliary(nullptr, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)));
>+    if (!temp)
>         return 0;
>     Butterfly* butterfly = Butterfly::fromBase(temp, 0, structure->outOfLineCapacity());
>     butterfly->setVectorLength(vectorLength);
>     butterfly->setPublicLength(length);
>+    
>+    for (unsigned i = length; i < vectorLength; ++i)
>+        butterfly->contiguous()[i].clear();
> 
>     ClonedArguments* result =
>         new (NotNull, allocateCell<ClonedArguments>(vm.heap))
>Index: Source/JavaScriptCore/runtime/IntlCollator.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/IntlCollator.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/IntlCollator.cpp	(working copy)
>@@ -1,6 +1,7 @@
> /*
>  * Copyright (C) 2015 Andy VanWagoner (thetalecrafter@gmail.com)
>  * Copyright (C) 2015 Sukolsak Sakshuwong (sukolsak@gmail.com)
>+ * Copyright (C) 2016 Apple Inc. All Rights Reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -33,8 +34,7 @@
> #include "IntlCollatorConstructor.h"
> #include "IntlObject.h"
> #include "JSBoundFunction.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "ObjectConstructor.h"
> #include "SlotVisitorInlines.h"
> #include "StructureInlines.h"
>Index: Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/IntlDateTimeFormat.cpp	(working copy)
>@@ -34,8 +34,7 @@
> #include "IntlDateTimeFormatConstructor.h"
> #include "IntlObject.h"
> #include "JSBoundFunction.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "ObjectConstructor.h"
> #include "SlotVisitorInlines.h"
> #include "StructureInlines.h"
>Index: Source/JavaScriptCore/runtime/IntlNumberFormat.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/IntlNumberFormat.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/IntlNumberFormat.cpp	(working copy)
>@@ -34,11 +34,8 @@
> #include "IntlNumberFormatConstructor.h"
> #include "IntlObject.h"
> #include "JSBoundFunction.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "ObjectConstructor.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSArray.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSArray.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSArray.cpp	(working copy)
>@@ -41,6 +41,12 @@ using namespace WTF;
> 
> namespace JSC {
> 
>+#if 0
>+#define LOGIT() dataLog(RawPointer(this), ": ", WTF_PRETTY_FUNCTION, "\n");
>+#else
>+#define LOGIT() do { } while(false)
>+#endif
>+
> STATIC_ASSERT_IS_TRIVIALLY_DESTRUCTIBLE(JSArray);
> 
> const ClassInfo JSArray::s_info = {"Array", &JSNonFinalObject::s_info, 0, CREATE_METHOD_TABLE(JSArray)};
>@@ -61,6 +67,8 @@ Butterfly* createArrayButterflyInDiction
> 
> void JSArray::setLengthWritable(ExecState* exec, bool writable)
> {
>+    LOGIT();
>+    
>     ASSERT(isLengthWritable() || !writable);
>     if (!isLengthWritable() || writable)
>         return;
>@@ -238,11 +246,13 @@ void JSArray::getOwnNonIndexPropertyName
> // This method makes room in the vector, but leaves the new space for count slots uncleared.
> bool JSArray::unshiftCountSlowCase(VM& vm, bool addToFront, unsigned count)
> {
>+    LOGIT();
>+    
>     ArrayStorage* storage = ensureArrayStorage(vm);
>     Butterfly* butterfly = storage->butterfly();
>     unsigned propertyCapacity = structure(vm)->outOfLineCapacity();
>     unsigned propertySize = structure(vm)->outOfLineSize();
>-
>+    
>     // If not, we should have handled this on the fast path.
>     ASSERT(!addToFront || count > storage->m_indexBias);
> 
>@@ -254,7 +264,8 @@ bool JSArray::unshiftCountSlowCase(VM& v
>     //  * desiredCapacity - how large should we like to grow the vector to - based on 2x requiredVectorLength.
> 
>     unsigned length = storage->length();
>-    unsigned usedVectorLength = min(storage->vectorLength(), length);
>+    unsigned oldVectorLength = storage->vectorLength();
>+    unsigned usedVectorLength = min(oldVectorLength, length);
>     ASSERT(usedVectorLength <= MAX_STORAGE_VECTOR_LENGTH);
>     // Check that required vector length is possible, in an overflow-safe fashion.
>     if (count > MAX_STORAGE_VECTOR_LENGTH - usedVectorLength)
>@@ -273,15 +284,19 @@ bool JSArray::unshiftCountSlowCase(VM& v
>     DeferGC deferGC(vm.heap);
>     void* newAllocBase = 0;
>     unsigned newStorageCapacity;
>+    bool allocatedNewStorage;
>     // If the current storage array is sufficiently large (but not too large!) then just keep using it.
>     if (currentCapacity > desiredCapacity && isDenseEnoughForVector(currentCapacity, requiredVectorLength)) {
>         newAllocBase = butterfly->base(structure(vm));
>         newStorageCapacity = currentCapacity;
>+        allocatedNewStorage = false;
>     } else {
>         size_t newSize = Butterfly::totalSize(0, propertyCapacity, true, ArrayStorage::sizeFor(desiredCapacity));
>-        if (!vm.heap.tryAllocateStorage(this, newSize, &newAllocBase))
>+        newAllocBase = vm.heap.tryAllocateAuxiliary(this, newSize);
>+        if (!newAllocBase)
>             return false;
>         newStorageCapacity = desiredCapacity;
>+        allocatedNewStorage = true;
>     }
> 
>     // Step 3:
>@@ -311,13 +326,19 @@ bool JSArray::unshiftCountSlowCase(VM& v
>         ASSERT(count + usedVectorLength <= newVectorLength);
>         memmove(newButterfly->arrayStorage()->m_vector + count, storage->m_vector, sizeof(JSValue) * usedVectorLength);
>         memmove(newButterfly->propertyStorage() - propertySize, butterfly->propertyStorage() - propertySize, sizeof(JSValue) * propertySize + sizeof(IndexingHeader) + ArrayStorage::sizeFor(0));
>+        
>+        if (allocatedNewStorage) {
>+            // We will set the vectorLength to newVectorLength. We populated requiredVectorLength
>+            // (usedVectorLength + count), which is less. Clear the difference.
>+            for (unsigned i = requiredVectorLength; i < newVectorLength; ++i)
>+                newButterfly->arrayStorage()->m_vector[i].clear();
>+        }
>     } else if ((newAllocBase != butterfly->base(structure(vm))) || (newIndexBias != storage->m_indexBias)) {
>         memmove(newButterfly->propertyStorage() - propertySize, butterfly->propertyStorage() - propertySize, sizeof(JSValue) * propertySize + sizeof(IndexingHeader) + ArrayStorage::sizeFor(0));
>         memmove(newButterfly->arrayStorage()->m_vector, storage->m_vector, sizeof(JSValue) * usedVectorLength);
>-
>-        WriteBarrier<Unknown>* newVector = newButterfly->arrayStorage()->m_vector;
>+        
>         for (unsigned i = requiredVectorLength; i < newVectorLength; i++)
>-            newVector[i].clear();
>+            newButterfly->arrayStorage()->m_vector[i].clear();
>     }
> 
>     newButterfly->arrayStorage()->setVectorLength(newVectorLength);
>@@ -329,8 +350,10 @@ bool JSArray::unshiftCountSlowCase(VM& v
> 
> bool JSArray::setLengthWithArrayStorage(ExecState* exec, unsigned newLength, bool throwException, ArrayStorage* storage)
> {
>+    LOGIT();
>+    
>     unsigned length = storage->length();
>-
>+    
>     // If the length is read only then we enter sparse mode, so should enter the following 'if'.
>     ASSERT(isLengthWritable() || storage->m_sparseMap);
> 
>@@ -393,6 +416,8 @@ bool JSArray::setLengthWithArrayStorage(
> 
> bool JSArray::appendMemcpy(ExecState* exec, VM& vm, unsigned startIndex, JSC::JSArray* otherArray)
> {
>+    LOGIT();
>+    
>     if (!canFastCopy(vm, otherArray))
>         return false;
> 
>@@ -433,6 +458,8 @@ bool JSArray::appendMemcpy(ExecState* ex
> 
> bool JSArray::setLength(ExecState* exec, unsigned newLength, bool throwException)
> {
>+    LOGIT();
>+    
>     Butterfly* butterfly = m_butterfly.get();
>     switch (indexingType()) {
>     case ArrayClass:
>@@ -497,6 +524,8 @@ bool JSArray::setLength(ExecState* exec,
> 
> JSValue JSArray::pop(ExecState* exec)
> {
>+    LOGIT();
>+    
>     Butterfly* butterfly = m_butterfly.get();
>     
>     switch (indexingType()) {
>@@ -594,6 +623,8 @@ JSValue JSArray::pop(ExecState* exec)
> //  - pushing to an array of length 2^32-1 stores the property, but throws a range error.
> void JSArray::push(ExecState* exec, JSValue value)
> {
>+    LOGIT();
>+    
>     Butterfly* butterfly = m_butterfly.get();
>     
>     switch (indexingType()) {
>@@ -730,6 +761,8 @@ void JSArray::push(ExecState* exec, JSVa
> 
> JSArray* JSArray::fastSlice(ExecState& exec, unsigned startIndex, unsigned count)
> {
>+    LOGIT();
>+    
>     auto arrayType = indexingType();
>     switch (arrayType) {
>     case ArrayWithDouble:
>@@ -760,6 +793,8 @@ JSArray* JSArray::fastSlice(ExecState& e
> 
> bool JSArray::shiftCountWithArrayStorage(VM& vm, unsigned startIndex, unsigned count, ArrayStorage* storage)
> {
>+    LOGIT();
>+    
>     unsigned oldLength = storage->length();
>     RELEASE_ASSERT(count <= oldLength);
>     
>@@ -865,6 +900,8 @@ bool JSArray::shiftCountWithArrayStorage
> 
> bool JSArray::shiftCountWithAnyIndexingType(ExecState* exec, unsigned& startIndex, unsigned count)
> {
>+    LOGIT();
>+    
>     VM& vm = exec->vm();
>     RELEASE_ASSERT(count > 0);
> 
>@@ -963,6 +1000,8 @@ bool JSArray::shiftCountWithAnyIndexingT
> // Returns true if the unshift can be handled, false to fallback.    
> bool JSArray::unshiftCountWithArrayStorage(ExecState* exec, unsigned startIndex, unsigned count, ArrayStorage* storage)
> {
>+    LOGIT();
>+    
>     unsigned length = storage->length();
> 
>     RELEASE_ASSERT(startIndex <= length);
>@@ -1100,6 +1139,8 @@ bool JSArray::unshiftCountWithAnyIndexin
> 
> void JSArray::fillArgList(ExecState* exec, MarkedArgumentBuffer& args)
> {
>+    LOGIT();
>+    
>     unsigned i = 0;
>     unsigned vectorEnd;
>     WriteBarrier<Unknown>* vector;
>@@ -1166,6 +1207,8 @@ void JSArray::fillArgList(ExecState* exe
> 
> void JSArray::copyToArguments(ExecState* exec, VirtualRegister firstElementDest, unsigned offset, unsigned length)
> {
>+    LOGIT();
>+    
>     unsigned i = offset;
>     WriteBarrier<Unknown>* vector;
>     unsigned vectorEnd;
>Index: Source/JavaScriptCore/runtime/JSArray.h
>===================================================================
>--- Source/JavaScriptCore/runtime/JSArray.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSArray.h	(working copy)
>@@ -1,6 +1,6 @@
> /*
>  *  Copyright (C) 1999-2000 Harri Porten (porten@kde.org)
>- *  Copyright (C) 2003, 2007, 2008, 2009, 2012, 2015 Apple Inc. All rights reserved.
>+ *  Copyright (C) 2003, 2007, 2008, 2009, 2012, 2015-2016 Apple Inc. All rights reserved.
>  *
>  *  This library is free software; you can redistribute it and/or
>  *  modify it under the terms of the GNU Lesser General Public
>@@ -191,8 +191,8 @@ inline Butterfly* createArrayButterfly(V
>         vm, intendedOwner, 0, 0, true, baseIndexingHeaderForArray(initialLength),
>         ArrayStorage::sizeFor(BASE_VECTOR_LEN));
>     ArrayStorage* storage = butterfly->arrayStorage();
>-    storage->m_indexBias = 0;
>     storage->m_sparseMap.clear();
>+    storage->m_indexBias = 0;
>     storage->m_numValuesInVector = 0;
>     return butterfly;
> }
>@@ -215,12 +215,17 @@ inline JSArray* JSArray::create(VM& vm, 
>         if (hasDouble(structure->indexingType())) {
>             for (unsigned i = 0; i < vectorLength; ++i)
>                 butterfly->contiguousDouble()[i] = PNaN;
>+        } else {
>+            for (unsigned i = 0; i < vectorLength; ++i)
>+                butterfly->contiguous()[i].clear();
>         }
>     } else {
>         ASSERT(
>             structure->indexingType() == ArrayWithSlowPutArrayStorage
>             || structure->indexingType() == ArrayWithArrayStorage);
>         butterfly = createArrayButterfly(vm, 0, initialLength);
>+        for (unsigned i = 0; i < BASE_VECTOR_LEN; ++i)
>+            butterfly->arrayStorage()->m_vector[i].clear();
>     }
> 
>     return createWithButterfly(vm, structure, butterfly);
>@@ -235,33 +240,39 @@ inline JSArray* JSArray::tryCreateUninit
>     unsigned outOfLineStorage = structure->outOfLineCapacity();
> 
>     Butterfly* butterfly;
>-    if (LIKELY(!hasAnyArrayStorage(structure->indexingType()))) {
>+    IndexingType indexingType = structure->indexingType();
>+    if (LIKELY(!hasAnyArrayStorage(indexingType))) {
>         ASSERT(
>-            hasUndecided(structure->indexingType())
>-            || hasInt32(structure->indexingType())
>-            || hasDouble(structure->indexingType())
>-            || hasContiguous(structure->indexingType()));
>-
>-        void* temp;
>-        if (!vm.heap.tryAllocateStorage(0, Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)), &temp))
>-            return 0;
>+            hasUndecided(indexingType)
>+            || hasInt32(indexingType)
>+            || hasDouble(indexingType)
>+            || hasContiguous(indexingType));
>+
>+        void* temp = vm.heap.tryAllocateAuxiliary(nullptr, Butterfly::totalSize(0, outOfLineStorage, true, vectorLength * sizeof(EncodedJSValue)));
>+        if (!temp)
>+            return nullptr;
>         butterfly = Butterfly::fromBase(temp, 0, outOfLineStorage);
>         butterfly->setVectorLength(vectorLength);
>         butterfly->setPublicLength(initialLength);
>-        if (hasDouble(structure->indexingType())) {
>+        if (hasDouble(indexingType)) {
>             for (unsigned i = initialLength; i < vectorLength; ++i)
>                 butterfly->contiguousDouble()[i] = PNaN;
>+        } else {
>+            for (unsigned i = initialLength; i < vectorLength; ++i)
>+                butterfly->contiguous()[i].clear();
>         }
>     } else {
>-        void* temp;
>-        if (!vm.heap.tryAllocateStorage(0, Butterfly::totalSize(0, outOfLineStorage, true, ArrayStorage::sizeFor(vectorLength)), &temp))
>-            return 0;
>+        void* temp = vm.heap.tryAllocateAuxiliary(nullptr, Butterfly::totalSize(0, outOfLineStorage, true, ArrayStorage::sizeFor(vectorLength)));
>+        if (!temp)
>+            return nullptr;
>         butterfly = Butterfly::fromBase(temp, 0, outOfLineStorage);
>         *butterfly->indexingHeader() = indexingHeaderForArray(initialLength, vectorLength);
>         ArrayStorage* storage = butterfly->arrayStorage();
>         storage->m_indexBias = 0;
>         storage->m_sparseMap.clear();
>         storage->m_numValuesInVector = initialLength;
>+        for (unsigned i = initialLength; i < vectorLength; ++i)
>+            storage->m_vector[i].clear();
>     }
> 
>     return createWithButterfly(vm, structure, butterfly);
>Index: Source/JavaScriptCore/runtime/JSCInlines.h
>===================================================================
>--- Source/JavaScriptCore/runtime/JSCInlines.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSCInlines.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2014 Apple Inc. All rights reserved.
>+ * Copyright (C) 2014, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>Index: Source/JavaScriptCore/runtime/JSCallee.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSCallee.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSCallee.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2014 Apple Inc. All rights reserved.
>+ * Copyright (C) 2014, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -27,13 +27,9 @@
> #include "JSCallee.h"
> 
> #include "GetterSetter.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCell.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSGlobalObject.h"
>-#include "SlotVisitorInlines.h"
> #include "StackVisitor.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h
>===================================================================
>--- Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSGenericTypedArrayViewInlines.h	(working copy)
>@@ -510,25 +510,15 @@ ArrayBuffer* JSGenericTypedArrayView<Ada
>     // up. But if you do *anything* to trigger a GC watermark check, it will know
>     // that you *had* done those allocations and it will GC appropriately.
>     Heap* heap = Heap::heap(thisObject);
>+    VM& vm = *heap->vm();
>     DeferGCForAWhile deferGC(*heap);
>     
>     ASSERT(!thisObject->hasIndexingHeader());
> 
>-    size_t size = thisObject->byteSize();
>-    
>-    if (thisObject->m_mode == FastTypedArray
>-        && !thisObject->butterfly() && size >= sizeof(IndexingHeader)) {
>-        ASSERT(thisObject->m_vector);
>-        // Reuse already allocated memory if at all possible.
>-        thisObject->m_butterfly.setWithoutBarrier(
>-            bitwise_cast<IndexingHeader*>(thisObject->vector())->butterfly());
>-    } else {
>-        RELEASE_ASSERT(!thisObject->hasIndexingHeader());
>-        VM& vm = *heap->vm();
>-        thisObject->m_butterfly.set(vm, thisObject, Butterfly::createOrGrowArrayRight(
>-            thisObject->butterfly(), vm, thisObject, thisObject->structure(),
>-            thisObject->structure()->outOfLineCapacity(), false, 0, 0));
>-    }
>+    RELEASE_ASSERT(!thisObject->hasIndexingHeader());
>+    thisObject->m_butterfly.set(vm, thisObject, Butterfly::createOrGrowArrayRight(
>+        thisObject->butterfly(), vm, thisObject, thisObject->structure(),
>+        thisObject->structure()->outOfLineCapacity(), false, 0, 0));
> 
>     RefPtr<ArrayBuffer> buffer;
>     
>Index: Source/JavaScriptCore/runtime/JSMapIterator.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSMapIterator.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSMapIterator.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2013 Apple, Inc. All rights reserved.
>+ * Copyright (C) 2013, 2016 Apple, Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -26,12 +26,9 @@
> #include "config.h"
> #include "JSMapIterator.h"
> 
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSMap.h"
> #include "MapDataInlines.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSModuleNamespaceObject.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSModuleNamespaceObject.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSModuleNamespaceObject.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -27,14 +27,10 @@
> #include "JSModuleNamespaceObject.h"
> 
> #include "Error.h"
>-#include "IdentifierInlines.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSModuleEnvironment.h"
> #include "JSModuleRecord.h"
> #include "JSPropertyNameIterator.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSModuleRecord.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSModuleRecord.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSModuleRecord.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -28,14 +28,10 @@
> 
> #include "Error.h"
> #include "Executable.h"
>-#include "IdentifierInlines.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSMap.h"
> #include "JSModuleEnvironment.h"
> #include "JSModuleNamespaceObject.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSObject.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSObject.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSObject.cpp	(working copy)
>@@ -55,6 +55,12 @@
> 
> namespace JSC {
> 
>+#if 0
>+#define LOGIT() dataLog(RawPointer(this), ": ", WTF_PRETTY_FUNCTION, "\n");
>+#else
>+#define LOGIT() do { } while(false)
>+#endif
>+
> // We keep track of the size of the last array after it was grown. We use this
> // as a simple heuristic for as the value to grow the next array from size 0.
> // This value is capped by the constant FIRST_VECTOR_GROW defined in
>@@ -88,77 +94,6 @@ static inline void getClassPropertyNames
>     }
> }
> 
>-ALWAYS_INLINE void JSObject::copyButterfly(CopyVisitor& visitor, Butterfly* butterfly, size_t storageSize)
>-{
>-    ASSERT(butterfly);
>-    
>-    Structure* structure = this->structure();
>-    
>-    size_t propertyCapacity = structure->outOfLineCapacity();
>-    size_t preCapacity;
>-    size_t indexingPayloadSizeInBytes;
>-    bool hasIndexingHeader = this->hasIndexingHeader();
>-    if (UNLIKELY(hasIndexingHeader)) {
>-        preCapacity = butterfly->indexingHeader()->preCapacity(structure);
>-        indexingPayloadSizeInBytes = butterfly->indexingHeader()->indexingPayloadSizeInBytes(structure);
>-    } else {
>-        preCapacity = 0;
>-        indexingPayloadSizeInBytes = 0;
>-    }
>-    size_t capacityInBytes = Butterfly::totalSize(preCapacity, propertyCapacity, hasIndexingHeader, indexingPayloadSizeInBytes);
>-    if (visitor.checkIfShouldCopy(butterfly->base(preCapacity, propertyCapacity))) {
>-        Butterfly* newButterfly = Butterfly::createUninitializedDuringCollection(visitor, preCapacity, propertyCapacity, hasIndexingHeader, indexingPayloadSizeInBytes);
>-
>-        // Copy the properties.
>-        PropertyStorage currentTarget = newButterfly->propertyStorage();
>-        PropertyStorage currentSource = butterfly->propertyStorage();
>-        for (size_t count = storageSize; count--;)
>-            (--currentTarget)->setWithoutWriteBarrier((--currentSource)->get());
>-        
>-        if (UNLIKELY(hasIndexingHeader)) {
>-            *newButterfly->indexingHeader() = *butterfly->indexingHeader();
>-            
>-            // Copy the array if appropriate.
>-            
>-            WriteBarrier<Unknown>* currentTarget;
>-            WriteBarrier<Unknown>* currentSource;
>-            size_t count;
>-            
>-            switch (this->indexingType()) {
>-            case ALL_UNDECIDED_INDEXING_TYPES:
>-            case ALL_CONTIGUOUS_INDEXING_TYPES:
>-            case ALL_INT32_INDEXING_TYPES:
>-            case ALL_DOUBLE_INDEXING_TYPES: {
>-                currentTarget = newButterfly->contiguous().data();
>-                currentSource = butterfly->contiguous().data();
>-                RELEASE_ASSERT(newButterfly->publicLength() <= newButterfly->vectorLength());
>-                count = newButterfly->vectorLength();
>-                break;
>-            }
>-                
>-            case ALL_ARRAY_STORAGE_INDEXING_TYPES: {
>-                newButterfly->arrayStorage()->copyHeaderFromDuringGC(*butterfly->arrayStorage());
>-                currentTarget = newButterfly->arrayStorage()->m_vector;
>-                currentSource = butterfly->arrayStorage()->m_vector;
>-                count = newButterfly->arrayStorage()->vectorLength();
>-                break;
>-            }
>-                
>-            default:
>-                currentTarget = 0;
>-                currentSource = 0;
>-                count = 0;
>-                break;
>-            }
>-
>-            memcpy(currentTarget, currentSource, count * sizeof(EncodedJSValue));
>-        }
>-        
>-        m_butterfly.setWithoutBarrier(newButterfly);
>-        visitor.didCopy(butterfly->base(preCapacity, propertyCapacity), capacityInBytes);
>-    } 
>-}
>-
> ALWAYS_INLINE void JSObject::visitButterfly(SlotVisitor& visitor, Butterfly* butterfly, Structure* structure)
> {
>     ASSERT(butterfly);
>@@ -175,13 +110,16 @@ ALWAYS_INLINE void JSObject::visitButter
>         preCapacity = 0;
>         indexingPayloadSizeInBytes = 0;
>     }
>-    size_t capacityInBytes = Butterfly::totalSize(preCapacity, propertyCapacity, hasIndexingHeader, indexingPayloadSizeInBytes);
>+    
>+    HeapCell* base = bitwise_cast<HeapCell*>(butterfly->base(preCapacity, propertyCapacity));
> 
>+    ASSERT(Heap::heap(base) == visitor.heap());
>+
>+    // Keep the butterfly alive.
>+    visitor.markAuxiliary(base);
>+    
>     // Mark the properties.
>     visitor.appendValuesHidden(butterfly->propertyStorage() - storageSize, storageSize);
>-    visitor.copyLater(
>-        this, ButterflyCopyToken,
>-        butterfly->base(preCapacity, propertyCapacity), capacityInBytes);
>     
>     // Mark the array if appropriate.
>     switch (this->indexingType()) {
>@@ -225,19 +163,6 @@ void JSObject::visitChildren(JSCell* cel
> #endif
> }
> 
>-void JSObject::copyBackingStore(JSCell* cell, CopyVisitor& visitor, CopyToken token)
>-{
>-    JSObject* thisObject = jsCast<JSObject*>(cell);
>-    ASSERT_GC_OBJECT_INHERITS(thisObject, info());
>-
>-    if (token != ButterflyCopyToken)
>-        return;
>-    
>-    Butterfly* butterfly = thisObject->m_butterfly.get();
>-    if (butterfly)
>-        thisObject->copyButterfly(visitor, butterfly, thisObject->structure()->outOfLineSize());
>-}
>-
> void JSObject::heapSnapshot(JSCell* cell, HeapSnapshotBuilder& builder)
> {
>     JSObject* thisObject = jsCast<JSObject*>(cell);
>@@ -780,6 +705,7 @@ void JSObject::notifyPresenceOfIndexedAc
> 
> Butterfly* JSObject::createInitialIndexedStorage(VM& vm, unsigned length, size_t elementSize)
> {
>+    LOGIT();
>     ASSERT(length < MAX_ARRAY_INDEX);
>     IndexingType oldType = indexingType();
>     ASSERT_UNUSED(oldType, !hasIndexedProperties(oldType));
>@@ -807,6 +733,8 @@ ContiguousJSValues JSObject::createIniti
> {
>     DeferGC deferGC(vm.heap);
>     Butterfly* newButterfly = createInitialIndexedStorage(vm, length, sizeof(EncodedJSValue));
>+    for (unsigned i = newButterfly->vectorLength(); i--;)
>+        newButterfly->contiguousInt32()[i].setWithoutWriteBarrier(JSValue());
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure(vm), NonPropertyTransition::AllocateInt32);
>     setStructureAndButterfly(vm, newStructure, newButterfly);
>     return newButterfly->contiguousInt32();
>@@ -827,6 +755,8 @@ ContiguousJSValues JSObject::createIniti
> {
>     DeferGC deferGC(vm.heap);
>     Butterfly* newButterfly = createInitialIndexedStorage(vm, length, sizeof(EncodedJSValue));
>+    for (unsigned i = newButterfly->vectorLength(); i--;)
>+        newButterfly->contiguous()[i].setWithoutWriteBarrier(JSValue());
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure(vm), NonPropertyTransition::AllocateContiguous);
>     setStructureAndButterfly(vm, newStructure, newButterfly);
>     return newButterfly->contiguous();
>@@ -834,6 +764,7 @@ ContiguousJSValues JSObject::createIniti
> 
> ArrayStorage* JSObject::createArrayStorage(VM& vm, unsigned length, unsigned vectorLength)
> {
>+    LOGIT();
>     DeferGC deferGC(vm.heap);
>     Structure* structure = this->structure(vm);
>     IndexingType oldType = indexingType();
>@@ -849,6 +780,8 @@ ArrayStorage* JSObject::createArrayStora
>     result->m_sparseMap.clear();
>     result->m_numValuesInVector = 0;
>     result->m_indexBias = 0;
>+    for (size_t i = vectorLength; i--;)
>+        result->m_vector[i].setWithoutWriteBarrier(JSValue());
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure, structure->suggestedArrayStorageTransition());
>     setStructureAndButterfly(vm, newStructure, newButterfly);
>     return result;
>@@ -862,6 +795,11 @@ ArrayStorage* JSObject::createInitialArr
> ContiguousJSValues JSObject::convertUndecidedToInt32(VM& vm)
> {
>     ASSERT(hasUndecided(indexingType()));
>+
>+    Butterfly* butterfly = m_butterfly.get();
>+    for (unsigned i = butterfly->vectorLength(); i--;)
>+        butterfly->contiguousInt32()[i].setWithoutWriteBarrier(JSValue());
>+
>     setStructure(vm, Structure::nonPropertyTransition(vm, structure(vm), NonPropertyTransition::AllocateInt32));
>     return m_butterfly.get()->contiguousInt32();
> }
>@@ -881,6 +819,11 @@ ContiguousDoubles JSObject::convertUndec
> ContiguousJSValues JSObject::convertUndecidedToContiguous(VM& vm)
> {
>     ASSERT(hasUndecided(indexingType()));
>+
>+    Butterfly* butterfly = m_butterfly.get();
>+    for (unsigned i = butterfly->vectorLength(); i--;)
>+        butterfly->contiguous()[i].setWithoutWriteBarrier(JSValue());
>+
>     setStructure(vm, Structure::nonPropertyTransition(vm, structure(vm), NonPropertyTransition::AllocateContiguous));
>     return m_butterfly.get()->contiguous();
> }
>@@ -917,7 +860,9 @@ ArrayStorage* JSObject::convertUndecided
> 
>     unsigned vectorLength = m_butterfly.get()->vectorLength();
>     ArrayStorage* storage = constructConvertedArrayStorageWithoutCopyingElements(vm, vectorLength);
>-    // No need to copy elements.
>+    
>+    for (unsigned i = vectorLength; i--;)
>+        storage->m_vector[i].setWithoutWriteBarrier(JSValue());
>     
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure(vm), transition);
>     setStructureAndButterfly(vm, newStructure, storage->butterfly());
>@@ -938,11 +883,12 @@ ContiguousDoubles JSObject::convertInt32
>         WriteBarrier<Unknown>* current = &butterfly->contiguousInt32()[i];
>         double* currentAsDouble = bitwise_cast<double*>(current);
>         JSValue v = current->get();
>-        if (!v) {
>+        // NOTE: Since this may be used during initialization, v could be garbage. If it's garbage,
>+        // that means it will be overwritten later.
>+        if (!v.isInt32()) {
>             *currentAsDouble = PNaN;
>             continue;
>         }
>-        ASSERT(v.isInt32());
>         *currentAsDouble = v.asInt32();
>     }
>     
>@@ -966,13 +912,11 @@ ArrayStorage* JSObject::convertInt32ToAr
>     unsigned vectorLength = m_butterfly.get()->vectorLength();
>     ArrayStorage* newStorage = constructConvertedArrayStorageWithoutCopyingElements(vm, vectorLength);
>     Butterfly* butterfly = m_butterfly.get();
>-    for (unsigned i = 0; i < butterfly->publicLength(); i++) {
>+    for (unsigned i = 0; i < vectorLength; i++) {
>         JSValue v = butterfly->contiguous()[i].get();
>-        if (v) {
>-            newStorage->m_vector[i].setWithoutWriteBarrier(v);
>+        newStorage->m_vector[i].setWithoutWriteBarrier(v);
>+        if (v)
>             newStorage->m_numValuesInVector++;
>-        } else
>-            ASSERT(newStorage->m_vector[i].get().isEmpty());
>     }
>     
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure(vm), transition);
>@@ -1014,13 +958,11 @@ ArrayStorage* JSObject::convertDoubleToA
>     unsigned vectorLength = m_butterfly.get()->vectorLength();
>     ArrayStorage* newStorage = constructConvertedArrayStorageWithoutCopyingElements(vm, vectorLength);
>     Butterfly* butterfly = m_butterfly.get();
>-    for (unsigned i = 0; i < butterfly->publicLength(); i++) {
>+    for (unsigned i = 0; i < vectorLength; i++) {
>         double value = butterfly->contiguousDouble()[i];
>-        if (value == value) {
>-            newStorage->m_vector[i].setWithoutWriteBarrier(JSValue(JSValue::EncodeAsDouble, value));
>+        newStorage->m_vector[i].setWithoutWriteBarrier(JSValue(JSValue::EncodeAsDouble, value));
>+        if (value == value)
>             newStorage->m_numValuesInVector++;
>-        } else
>-            ASSERT(newStorage->m_vector[i].get().isEmpty());
>     }
>     
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure(vm), transition);
>@@ -1041,13 +983,11 @@ ArrayStorage* JSObject::convertContiguou
>     unsigned vectorLength = m_butterfly.get()->vectorLength();
>     ArrayStorage* newStorage = constructConvertedArrayStorageWithoutCopyingElements(vm, vectorLength);
>     Butterfly* butterfly = m_butterfly.get();
>-    for (unsigned i = 0; i < butterfly->publicLength(); i++) {
>+    for (unsigned i = 0; i < vectorLength; i++) {
>         JSValue v = butterfly->contiguous()[i].get();
>-        if (v) {
>-            newStorage->m_vector[i].setWithoutWriteBarrier(v);
>+        newStorage->m_vector[i].setWithoutWriteBarrier(v);
>+        if (v)
>             newStorage->m_numValuesInVector++;
>-        } else
>-            ASSERT(newStorage->m_vector[i].get().isEmpty());
>     }
>     
>     Structure* newStructure = Structure::nonPropertyTransition(vm, structure(vm), transition);
>@@ -2729,6 +2669,7 @@ unsigned JSObject::countElements()
> 
> bool JSObject::increaseVectorLength(VM& vm, unsigned newLength)
> {
>+    LOGIT();
>     // This function leaves the array in an internally inconsistent state, because it does not move any values from sparse value map
>     // to the vector. Callers have to account for that, because they can do it more efficiently.
>     if (newLength > MAX_STORAGE_VECTOR_LENGTH)
>@@ -2754,6 +2695,8 @@ bool JSObject::increaseVectorLength(VM& 
>             ArrayStorage::sizeFor(vectorLength), ArrayStorage::sizeFor(newVectorLength));
>         if (!newButterfly)
>             return false;
>+        for (unsigned i = vectorLength; i < newVectorLength; ++i)
>+            newButterfly->arrayStorage()->m_vector[i].clear();
>         newButterfly->arrayStorage()->setVectorLength(newVectorLength);
>         setButterflyWithoutChangingStructure(vm, newButterfly);
>         return true;
>@@ -2768,6 +2711,8 @@ bool JSObject::increaseVectorLength(VM& 
>         newIndexBias, true, ArrayStorage::sizeFor(newVectorLength));
>     if (!newButterfly)
>         return false;
>+    for (unsigned i = vectorLength; i < newVectorLength; ++i)
>+        newButterfly->arrayStorage()->m_vector[i].clear();
>     newButterfly->arrayStorage()->setVectorLength(newVectorLength);
>     newButterfly->arrayStorage()->m_indexBias = newIndexBias;
>     setButterflyWithoutChangingStructure(vm, newButterfly);
>@@ -2776,6 +2721,7 @@ bool JSObject::increaseVectorLength(VM& 
> 
> bool JSObject::ensureLengthSlow(VM& vm, unsigned length)
> {
>+    LOGIT();
>     Butterfly* butterfly = m_butterfly.get();
>     
>     ASSERT(length < MAX_ARRAY_INDEX);
>@@ -2799,8 +2745,12 @@ bool JSObject::ensureLengthSlow(VM& vm, 
> 
>     if (hasDouble(indexingType())) {
>         for (unsigned i = oldVectorLength; i < newVectorLength; ++i)
>-            butterfly->contiguousDouble().data()[i] = PNaN;
>+            butterfly->contiguousDouble()[i] = PNaN;
>+    } else {
>+        for (unsigned i = oldVectorLength; i < newVectorLength; ++i)
>+            butterfly->contiguous()[i].clear();
>     }
>+
>     return true;
> }
> 
>@@ -2821,6 +2771,7 @@ void JSObject::reallocateAndShrinkButter
> 
> Butterfly* JSObject::growOutOfLineStorage(VM& vm, size_t oldSize, size_t newSize)
> {
>+    LOGIT();
>     ASSERT(newSize > oldSize);
> 
>     // It's important that this function not rely on structure(), for the property
>Index: Source/JavaScriptCore/runtime/JSObject.h
>===================================================================
>--- Source/JavaScriptCore/runtime/JSObject.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSObject.h	(working copy)
>@@ -26,11 +26,11 @@
> #include "ArgList.h"
> #include "ArrayConventions.h"
> #include "ArrayStorage.h"
>+#include "AuxiliaryBarrier.h"
> #include "Butterfly.h"
> #include "CallFrame.h"
> #include "ClassInfo.h"
> #include "CommonIdentifiers.h"
>-#include "CopyBarrier.h"
> #include "CustomGetterSetter.h"
> #include "DeferGC.h"
> #include "Heap.h"
>@@ -103,7 +103,6 @@ public:
> 
>     JS_EXPORT_PRIVATE static size_t estimatedSize(JSCell*);
>     JS_EXPORT_PRIVATE static void visitChildren(JSCell*, SlotVisitor&);
>-    JS_EXPORT_PRIVATE static void copyBackingStore(JSCell*, CopyVisitor&, CopyToken);
>     JS_EXPORT_PRIVATE static void heapSnapshot(JSCell*, HeapSnapshotBuilder&);
> 
>     JS_EXPORT_PRIVATE static String className(const JSObject*);
>@@ -686,8 +685,6 @@ public:
>         
>     void setStructure(VM&, Structure*);
>     void setStructureAndButterfly(VM&, Structure*, Butterfly*);
>-    void setStructureAndReallocateStorageIfNecessary(VM&, unsigned oldCapacity, Structure*);
>-    void setStructureAndReallocateStorageIfNecessary(VM&, Structure*);
> 
>     JS_EXPORT_PRIVATE void convertToDictionary(VM&);
> 
>@@ -797,7 +794,6 @@ protected:
>     JSObject(VM&, Structure*, Butterfly* = 0);
>         
>     void visitButterfly(SlotVisitor&, Butterfly*, Structure*);
>-    void copyButterfly(CopyVisitor&, Butterfly*, size_t storageSize);
> 
>     // Call this if you know that the object is in a mode where it has array
>     // storage. This will assert otherwise.
>@@ -949,7 +945,7 @@ private:
>     JS_EXPORT_PRIVATE ArrayStorage* ensureArrayStorageSlow(VM&);
> 
> protected:
>-    CopyBarrier<Butterfly> m_butterfly;
>+    AuxiliaryBarrier<Butterfly*> m_butterfly;
> #if USE(JSVALUE32_64)
> private:
>     uint32_t m_padding;
>@@ -1487,8 +1483,16 @@ ALWAYS_INLINE bool JSObject::putDirectIn
>     
>     validateOffset(offset);
>     ASSERT(newStructure->isValidOffset(offset));
>-    setStructureAndReallocateStorageIfNecessary(vm, newStructure);
>-
>+    DeferGC deferGC(vm.heap);
>+    size_t oldCapacity = structure->outOfLineCapacity();
>+    size_t newCapacity = newStructure->outOfLineCapacity();
>+    ASSERT(oldCapacity <= newCapacity);
>+    if (oldCapacity == newCapacity)
>+        setStructure(vm, newStructure);
>+    else {
>+        Butterfly* newButterfly = growOutOfLineStorage(vm, oldCapacity, newCapacity);
>+        setStructureAndButterfly(vm, newStructure, newButterfly);
>+    }
>     putDirect(vm, offset, value);
>     slot.setNewProperty(this, offset);
>     if (attributes & ReadOnly)
>@@ -1496,27 +1500,6 @@ ALWAYS_INLINE bool JSObject::putDirectIn
>     return true;
> }
> 
>-inline void JSObject::setStructureAndReallocateStorageIfNecessary(VM& vm, unsigned oldCapacity, Structure* newStructure)
>-{
>-    ASSERT(oldCapacity <= newStructure->outOfLineCapacity());
>-    
>-    if (oldCapacity == newStructure->outOfLineCapacity()) {
>-        setStructure(vm, newStructure);
>-        return;
>-    }
>-
>-    DeferGC deferGC(vm.heap); 
>-    Butterfly* newButterfly = growOutOfLineStorage(
>-        vm, oldCapacity, newStructure->outOfLineCapacity());
>-    setStructureAndButterfly(vm, newStructure, newButterfly);
>-}
>-
>-inline void JSObject::setStructureAndReallocateStorageIfNecessary(VM& vm, Structure* newStructure)
>-{
>-    setStructureAndReallocateStorageIfNecessary(
>-        vm, structure(vm)->outOfLineCapacity(), newStructure);
>-}
>-
> inline bool JSObject::putOwnDataProperty(VM& vm, PropertyName propertyName, JSValue value, PutPropertySlot& slot)
> {
>     ASSERT(value);
>Index: Source/JavaScriptCore/runtime/JSObjectInlines.h
>===================================================================
>--- Source/JavaScriptCore/runtime/JSObjectInlines.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSObjectInlines.h	(working copy)
>@@ -1,7 +1,7 @@
> /*
>  *  Copyright (C) 1999-2001 Harri Porten (porten@kde.org)
>  *  Copyright (C) 2001 Peter Kelly (pmk@post.com)
>- *  Copyright (C) 2003-2006, 2008, 2009, 2012-2015 Apple Inc. All rights reserved.
>+ *  Copyright (C) 2003-2006, 2008, 2009, 2012-2016 Apple Inc. All rights reserved.
>  *  Copyright (C) 2007 Eric Seidel (eric@webkit.org)
>  *
>  *  This library is free software; you can redistribute it and/or
>@@ -24,6 +24,7 @@
> #ifndef JSObjectInlines_h
> #define JSObjectInlines_h
> 
>+#include "AuxiliaryBarrierInlines.h"
> #include "Error.h"
> #include "JSObject.h"
> #include "Lookup.h"
>Index: Source/JavaScriptCore/runtime/JSPromise.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSPromise.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSPromise.cpp	(working copy)
>@@ -28,12 +28,9 @@
> 
> #include "BuiltinNames.h"
> #include "Error.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSPromiseConstructor.h"
> #include "Microtask.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSPromisePrototype.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSPromisePrototype.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSPromisePrototype.cpp	(working copy)
>@@ -29,13 +29,11 @@
> #include "BuiltinNames.h"
> #include "Error.h"
> #include "JSCBuiltins.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSFunction.h"
> #include "JSGlobalObject.h"
> #include "JSPromise.h"
> #include "Microtask.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSPropertyNameIterator.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSPropertyNameIterator.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSPropertyNameIterator.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2015 Apple, Inc. All rights reserved.
>+ * Copyright (C) 2015-2016 Apple, Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -26,13 +26,9 @@
> #include "config.h"
> #include "JSPropertyNameIterator.h"
> 
>-#include "IdentifierInlines.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSPropertyNameEnumerator.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSSetIterator.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSSetIterator.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSSetIterator.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2013 Apple, Inc. All rights reserved.
>+ * Copyright (C) 2013, 2016 Apple, Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -26,12 +26,9 @@
> #include "config.h"
> #include "JSSetIterator.h"
> 
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSSet.h"
> #include "MapDataInlines.h"
>-#include "SlotVisitorInlines.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSStringIterator.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSStringIterator.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSStringIterator.cpp	(working copy)
>@@ -28,9 +28,7 @@
> #include "JSStringIterator.h"
> 
> #include "BuiltinNames.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>-#include "StructureInlines.h"
>+#include "JSCInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/JSTemplateRegistryKey.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSTemplateRegistryKey.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSTemplateRegistryKey.cpp	(working copy)
>@@ -1,5 +1,6 @@
> /*
>  * Copyright (C) 2015 Yusuke Suzuki <utatane.tea@gmail.com>.
>+ * Copyright (C) 2016 Apple Inc. All Rights Reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -26,9 +27,7 @@
> #include "config.h"
> #include "JSTemplateRegistryKey.h"
> 
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>-#include "StructureInlines.h"
>+#include "JSCInlines.h"
> #include "VM.h"
> 
> namespace JSC {
>Index: Source/JavaScriptCore/runtime/JSTypedArrayViewPrototype.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/JSTypedArrayViewPrototype.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/JSTypedArrayViewPrototype.cpp	(working copy)
>@@ -29,7 +29,7 @@
> #include "BuiltinNames.h"
> #include "CallFrame.h"
> #include "GetterSetter.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSFunction.h"
> #include "JSGenericTypedArrayViewPrototypeFunctions.h"
> #include "TypedArrayAdaptors.h"
>Index: Source/JavaScriptCore/runtime/MapConstructor.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/MapConstructor.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/MapConstructor.cpp	(working copy)
>@@ -29,12 +29,10 @@
> #include "Error.h"
> #include "GetterSetter.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSGlobalObject.h"
> #include "JSMap.h"
> #include "MapPrototype.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/MapPrototype.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/MapPrototype.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/MapPrototype.cpp	(working copy)
>@@ -31,13 +31,11 @@
> #include "ExceptionHelpers.h"
> #include "GetterSetter.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSFunctionInlines.h"
>+#include "JSCInlines.h"
> #include "JSMap.h"
> #include "JSMapIterator.h"
> #include "Lookup.h"
> #include "MapDataInlines.h"
>-#include "StructureInlines.h"
> 
> #include "MapPrototype.lut.h"
> 
>Index: Source/JavaScriptCore/runtime/Options.h
>===================================================================
>--- Source/JavaScriptCore/runtime/Options.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/Options.h	(working copy)
>@@ -182,6 +182,7 @@ typedef const char* optionString;
>     v(bool, testTheFTL, false, Normal, nullptr) \
>     v(bool, verboseSanitizeStack, false, Normal, nullptr) \
>     v(bool, useGenerationalGC, true, Normal, nullptr) \
>+    v(bool, resetFreeCellsBadly, false, Normal, nullptr) \
>     v(bool, eagerlyUpdateTopCallFrame, false, Normal, nullptr) \
>     \
>     v(bool, useOSREntryToDFG, true, Normal, nullptr) \
>Index: Source/JavaScriptCore/runtime/ProxyConstructor.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/ProxyConstructor.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/ProxyConstructor.cpp	(working copy)
>@@ -28,8 +28,7 @@
> 
> #include "Error.h"
> #include "IdentifierInlines.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "ObjectConstructor.h"
> #include "ObjectPrototype.h"
> #include "ProxyObject.h"
>Index: Source/JavaScriptCore/runtime/ProxyObject.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/ProxyObject.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/ProxyObject.cpp	(working copy)
>@@ -28,8 +28,7 @@
> 
> #include "Error.h"
> #include "IdentifierInlines.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "ObjectConstructor.h"
> #include "SlotVisitorInlines.h"
> #include "StructureInlines.h"
>Index: Source/JavaScriptCore/runtime/RegExpMatchesArray.h
>===================================================================
>--- Source/JavaScriptCore/runtime/RegExpMatchesArray.h	(revision 203920)
>+++ Source/JavaScriptCore/runtime/RegExpMatchesArray.h	(working copy)
>@@ -38,9 +38,9 @@ ALWAYS_INLINE JSArray* tryCreateUninitia
>     if (vectorLength > MAX_STORAGE_VECTOR_LENGTH)
>         return 0;
> 
>-    void* temp;
>-    if (!vm.heap.tryAllocateStorage(0, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)), &temp))
>-        return 0;
>+    void* temp = vm.heap.tryAllocateAuxiliary(nullptr, Butterfly::totalSize(0, structure->outOfLineCapacity(), true, vectorLength * sizeof(EncodedJSValue)));
>+    if (!temp)
>+        return nullptr;
>     Butterfly* butterfly = Butterfly::fromBase(temp, 0, structure->outOfLineCapacity());
>     butterfly->setVectorLength(vectorLength);
>     butterfly->setPublicLength(initialLength);
>Index: Source/JavaScriptCore/runtime/SamplingProfiler.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/SamplingProfiler.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/SamplingProfiler.cpp	(working copy)
>@@ -34,6 +34,7 @@
> #include "Executable.h"
> #include "HeapInlines.h"
> #include "HeapIterationScope.h"
>+#include "HeapUtil.h"
> #include "InlineCallFrame.h"
> #include "Interpreter.h"
> #include "JSCJSValueInlines.h"
>@@ -392,7 +393,7 @@ void SamplingProfiler::processUnverified
>             JSValue callee = JSValue::decode(encodedCallee);
>             StackFrame& stackFrame = stackTrace.frames.last();
>             bool alreadyHasExecutable = !!stackFrame.executable;
>-            if (!Heap::isValueGCObject(filter, markedBlockSet, callee)) {
>+            if (!HeapUtil::isValueGCObject(filter, markedBlockSet, callee)) {
>                 if (!alreadyHasExecutable)
>                     stackFrame.frameType = FrameType::Unknown;
>                 return;
>@@ -437,7 +438,7 @@ void SamplingProfiler::processUnverified
>                 return;
>             }
> 
>-            RELEASE_ASSERT(Heap::isPointerGCObject(filter, markedBlockSet, executable));
>+            RELEASE_ASSERT(HeapUtil::isPointerGCObjectJSCell(filter, markedBlockSet, executable));
>             stackFrame.frameType = FrameType::Executable;
>             stackFrame.executable = executable;
>             m_liveCellPointers.add(executable);
>Index: Source/JavaScriptCore/runtime/SetConstructor.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/SetConstructor.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/SetConstructor.cpp	(working copy)
>@@ -29,13 +29,11 @@
> #include "Error.h"
> #include "GetterSetter.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSGlobalObject.h"
> #include "JSSet.h"
> #include "MapData.h"
> #include "SetPrototype.h"
>-#include "StructureInlines.h"
> 
> namespace JSC {
> 
>Index: Source/JavaScriptCore/runtime/SetPrototype.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/SetPrototype.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/SetPrototype.cpp	(working copy)
>@@ -31,13 +31,11 @@
> #include "ExceptionHelpers.h"
> #include "GetterSetter.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSFunctionInlines.h"
>+#include "JSCInlines.h"
> #include "JSSet.h"
> #include "JSSetIterator.h"
> #include "Lookup.h"
> #include "MapDataInlines.h"
>-#include "StructureInlines.h"
> 
> #include "SetPrototype.lut.h"
> 
>Index: Source/JavaScriptCore/runtime/WeakMapConstructor.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/WeakMapConstructor.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/WeakMapConstructor.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2013 Apple, Inc. All rights reserved.
>+ * Copyright (C) 2013, 2016 Apple, Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -28,11 +28,9 @@
> 
> #include "Error.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSGlobalObject.h"
> #include "JSWeakMap.h"
>-#include "StructureInlines.h"
> #include "WeakMapPrototype.h"
> 
> namespace JSC {
>Index: Source/JavaScriptCore/runtime/WeakSetConstructor.cpp
>===================================================================
>--- Source/JavaScriptCore/runtime/WeakSetConstructor.cpp	(revision 203920)
>+++ Source/JavaScriptCore/runtime/WeakSetConstructor.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2015 Apple, Inc. All rights reserved.
>+ * Copyright (C) 2015-2016 Apple, Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -28,11 +28,9 @@
> 
> #include "Error.h"
> #include "IteratorOperations.h"
>-#include "JSCJSValueInlines.h"
>-#include "JSCellInlines.h"
>+#include "JSCInlines.h"
> #include "JSGlobalObject.h"
> #include "JSWeakSet.h"
>-#include "StructureInlines.h"
> #include "WeakSetPrototype.h"
> 
> namespace JSC {
>Index: Source/JavaScriptCore/tests/stress/array-storage-array-unshift.js
>===================================================================
>--- Source/JavaScriptCore/tests/stress/array-storage-array-unshift.js	(nonexistent)
>+++ Source/JavaScriptCore/tests/stress/array-storage-array-unshift.js	(working copy)
>@@ -0,0 +1,8 @@
>+//@ runDefault
>+var x = [2.5, 1.5];
>+x.length = 1000000000;
>+x.length = 2;
>+Array.prototype.unshift.call(x, 3.5);
>+if (x.toString() != "3.5,2.5,1.5")
>+    throw "Error: bad result: " + describe(x);
>+
>Index: Source/JavaScriptCore/tests/stress/contiguous-array-unshift.js
>===================================================================
>--- Source/JavaScriptCore/tests/stress/contiguous-array-unshift.js	(nonexistent)
>+++ Source/JavaScriptCore/tests/stress/contiguous-array-unshift.js	(working copy)
>@@ -0,0 +1,6 @@
>+//@ runDefault
>+var x = ['b', 'a'];
>+Array.prototype.unshift.call(x, 'c');
>+if (x.toString() != "c,b,a")
>+    throw "Error: bad result: " + describe(x);
>+
>Index: Source/JavaScriptCore/tests/stress/double-array-unshift.js
>===================================================================
>--- Source/JavaScriptCore/tests/stress/double-array-unshift.js	(nonexistent)
>+++ Source/JavaScriptCore/tests/stress/double-array-unshift.js	(working copy)
>@@ -0,0 +1,6 @@
>+//@ runDefault
>+var x = [2.5, 1.5];
>+Array.prototype.unshift.call(x, 3.5);
>+if (x.toString() != "3.5,2.5,1.5")
>+    throw "Error: bad result: " + describe(x);
>+
>Index: Source/JavaScriptCore/tests/stress/int32-array-unshift.js
>===================================================================
>--- Source/JavaScriptCore/tests/stress/int32-array-unshift.js	(nonexistent)
>+++ Source/JavaScriptCore/tests/stress/int32-array-unshift.js	(working copy)
>@@ -0,0 +1,6 @@
>+//@ runDefault
>+var x = [2, 1];
>+Array.prototype.unshift.call(x, 3);
>+if (x.toString() != "3,2,1")
>+    throw "Error: bad result: " + describe(x);
>+
>Index: Source/WTF/ChangeLog
>===================================================================
>--- Source/WTF/ChangeLog	(revision 203920)
>+++ Source/WTF/ChangeLog	(working copy)
>@@ -1,3 +1,18 @@
>+2016-07-29  Filip Pizlo  <fpizlo@apple.com>
>+
>+        Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace
>+        https://bugs.webkit.org/show_bug.cgi?id=160125
>+
>+        Reviewed by NOBODY (OOPS!).
>+        
>+        The JSC GC needs a tryFastAlignedMalloc(), and this exposes such a function.
>+
>+        * wtf/FastMalloc.cpp:
>+        (WTF::fastAlignedMalloc):
>+        (WTF::tryFastAlignedMalloc):
>+        (WTF::fastAlignedFree):
>+        * wtf/FastMalloc.h:
>+
> 2016-07-29  Mark Lam  <mark.lam@apple.com>
> 
>         Make StringView capable of being passed or returned in only 2 registers.
>Index: Source/WTF/wtf/FastMalloc.cpp
>===================================================================
>--- Source/WTF/wtf/FastMalloc.cpp	(revision 203920)
>+++ Source/WTF/wtf/FastMalloc.cpp	(working copy)
>@@ -1,7 +1,6 @@
>-// Copyright (c) 2005, 2007, Google Inc. All rights reserved.
>-
> /*
>- * Copyright (C) 2005-2009, 2011, 2015 Apple Inc. All rights reserved.
>+ * Copyright (c) 2005, 2007, Google Inc. All rights reserved.
>+ * Copyright (C) 2005-2009, 2011, 2015-2016 Apple Inc. All rights reserved.
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>  * are met:
>@@ -97,6 +96,11 @@ void* fastAlignedMalloc(size_t alignment
>     return _aligned_malloc(size, alignment);
> }
> 
>+void* tryFastAlignedMalloc(size_t alignment, size_t size) 
>+{
>+    return _aligned_malloc(size, alignment);
>+}
>+
> void fastAlignedFree(void* p) 
> {
>     _aligned_free(p);
>@@ -111,6 +115,13 @@ void* fastAlignedMalloc(size_t alignment
>     return p;
> }
> 
>+void* tryFastAlignedMalloc(size_t alignment, size_t size) 
>+{
>+    void* p = nullptr;
>+    posix_memalign(&p, alignment, size);
>+    return p;
>+}
>+
> void fastAlignedFree(void* p) 
> {
>     free(p);
>@@ -231,6 +242,11 @@ void* fastAlignedMalloc(size_t alignment
>     return bmalloc::api::memalign(alignment, size);
> }
> 
>+void* tryFastAlignedMalloc(size_t alignment, size_t size) 
>+{
>+    return bmalloc::api::tryMemalign(alignment, size);
>+}
>+
> void fastAlignedFree(void* p) 
> {
>     bmalloc::api::free(p);
>Index: Source/WTF/wtf/FastMalloc.h
>===================================================================
>--- Source/WTF/wtf/FastMalloc.h	(revision 203920)
>+++ Source/WTF/wtf/FastMalloc.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- *  Copyright (C) 2005-2009, 2015 Apple Inc. All rights reserved.
>+ *  Copyright (C) 2005-2009, 2015-2016 Apple Inc. All rights reserved.
>  *
>  *  This library is free software; you can redistribute it and/or
>  *  modify it under the terms of the GNU Library General Public
>@@ -53,6 +53,7 @@ WTF_EXPORT_PRIVATE void fastFree(void*);
> 
> // Allocations from fastAlignedMalloc() must be freed using fastAlignedFree().
> WTF_EXPORT_PRIVATE void* fastAlignedMalloc(size_t alignment, size_t);
>+WTF_EXPORT_PRIVATE void* tryFastAlignedMalloc(size_t alignment, size_t);
> WTF_EXPORT_PRIVATE void fastAlignedFree(void*);
> 
> WTF_EXPORT_PRIVATE size_t fastMallocSize(const void*);
>@@ -107,6 +108,7 @@ using WTF::fastMallocSize;
> using WTF::fastRealloc;
> using WTF::fastStrDup;
> using WTF::fastZeroedMalloc;
>+using WTF::tryFastAlignedMalloc;
> using WTF::tryFastCalloc;
> using WTF::tryFastMalloc;
> using WTF::tryFastZeroedMalloc;
>Index: Source/WebKit2/ChangeLog
>===================================================================
>--- Source/WebKit2/ChangeLog	(revision 203927)
>+++ Source/WebKit2/ChangeLog	(working copy)
>@@ -1,3 +1,14 @@
>+2016-07-29  Filip Pizlo  <fpizlo@apple.com>
>+
>+        Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace
>+        https://bugs.webkit.org/show_bug.cgi?id=160125
>+
>+        Reviewed by NOBODY (OOPS!).
>+        
>+        Need to #include more things now.
>+
>+        * WebProcess/Plugins/Netscape/JSNPObject.cpp:
>+
> 2016-07-28  Ryosuke Niwa  <rniwa@webkit.org>
> 
>         Crash with an Invalid Web Process IPC Message ID: WebPageProxy.AttributedStringForCharacterRangeCallback
>Index: Source/WebKit2/WebProcess/Plugins/Netscape/JSNPObject.cpp
>===================================================================
>--- Source/WebKit2/WebProcess/Plugins/Netscape/JSNPObject.cpp	(revision 203920)
>+++ Source/WebKit2/WebProcess/Plugins/Netscape/JSNPObject.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2010 Apple Inc. All rights reserved.
>+ * Copyright (C) 2010, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -32,6 +32,7 @@
> #include "NPJSObject.h"
> #include "NPRuntimeObjectMap.h"
> #include "NPRuntimeUtilities.h"
>+#include <JavaScriptCore/AuxiliaryBarrierInlines.h>
> #include <JavaScriptCore/Error.h>
> #include <JavaScriptCore/IdentifierInlines.h>
> #include <JavaScriptCore/JSGlobalObject.h>
>Index: Source/bmalloc/ChangeLog
>===================================================================
>--- Source/bmalloc/ChangeLog	(revision 203920)
>+++ Source/bmalloc/ChangeLog	(working copy)
>@@ -1,3 +1,27 @@
>+2016-07-29  Filip Pizlo  <fpizlo@apple.com>
>+
>+        Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace
>+        https://bugs.webkit.org/show_bug.cgi?id=160125
>+
>+        Reviewed by NOBODY (OOPS!).
>+        
>+        The JSC GC needs to be able to tryMemalign, and this refactors the code to expose such a
>+        function.
>+
>+        * bmalloc/Allocator.cpp:
>+        (bmalloc::Allocator::allocate):
>+        (bmalloc::Allocator::tryAllocate):
>+        (bmalloc::Allocator::allocateImpl):
>+        (bmalloc::Allocator::reallocate):
>+        * bmalloc/Allocator.h:
>+        * bmalloc/Cache.h:
>+        (bmalloc::Cache::allocate):
>+        (bmalloc::Cache::tryAllocate):
>+        * bmalloc/bmalloc.h:
>+        (bmalloc::api::malloc):
>+        (bmalloc::api::tryMemalign):
>+        (bmalloc::api::memalign):
>+
> 2016-07-13  Enrica Casucci  <enrica@apple.com>
> 
>         Update supported platforms in xcconfig files to match the sdk names.
>Index: Source/bmalloc/bmalloc/Allocator.cpp
>===================================================================
>--- Source/bmalloc/bmalloc/Allocator.cpp	(revision 203920)
>+++ Source/bmalloc/bmalloc/Allocator.cpp	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2014-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -64,6 +64,18 @@ void* Allocator::tryAllocate(size_t size
> 
> void* Allocator::allocate(size_t alignment, size_t size)
> {
>+    bool crashOnFailure = true;
>+    return allocateImpl(alignment, size, crashOnFailure);
>+}
>+
>+void* Allocator::tryAllocate(size_t alignment, size_t size)
>+{
>+    bool crashOnFailure = false;
>+    return allocateImpl(alignment, size, crashOnFailure);
>+}
>+
>+void* Allocator::allocateImpl(size_t alignment, size_t size, bool crashOnFailure)
>+{
>     BASSERT(isPowerOfTwo(alignment));
> 
>     if (!m_isBmallocEnabled) {
>@@ -80,7 +92,10 @@ void* Allocator::allocate(size_t alignme
>         return allocate(roundUpToMultipleOf(alignment, size));
> 
>     std::lock_guard<StaticMutex> lock(PerProcess<Heap>::mutex());
>-    return PerProcess<Heap>::getFastCase()->allocateLarge(lock, alignment, size);
>+    Heap* heap = PerProcess<Heap>::getFastCase();
>+    if (crashOnFailure)
>+        return heap->allocateLarge(lock, alignment, size);
>+    return heap->tryAllocateLarge(lock, alignment, size);
> }
> 
> void* Allocator::reallocate(void* object, size_t newSize)
>Index: Source/bmalloc/bmalloc/Allocator.h
>===================================================================
>--- Source/bmalloc/bmalloc/Allocator.h	(revision 203920)
>+++ Source/bmalloc/bmalloc/Allocator.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2014 Apple Inc. All rights reserved.
>+ * Copyright (C) 2014, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -43,12 +43,15 @@ public:
> 
>     void* tryAllocate(size_t);
>     void* allocate(size_t);
>+    void* tryAllocate(size_t alignment, size_t);
>     void* allocate(size_t alignment, size_t);
>     void* reallocate(void*, size_t);
> 
>     void scavenge();
> 
> private:
>+    void* allocateImpl(size_t alignment, size_t, bool crashOnFailure);
>+    
>     bool allocateFastCase(size_t, void*&);
>     void* allocateSlowCase(size_t);
>     
>Index: Source/bmalloc/bmalloc/Cache.h
>===================================================================
>--- Source/bmalloc/bmalloc/Cache.h	(revision 203920)
>+++ Source/bmalloc/bmalloc/Cache.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2014 Apple Inc. All rights reserved.
>+ * Copyright (C) 2014, 2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -41,6 +41,7 @@ public:
> 
>     static void* tryAllocate(size_t);
>     static void* allocate(size_t);
>+    static void* tryAllocate(size_t alignment, size_t);
>     static void* allocate(size_t alignment, size_t);
>     static void deallocate(void*);
>     static void* reallocate(void*, size_t);
>@@ -79,6 +80,14 @@ inline void* Cache::allocate(size_t size
>     return cache->allocator().allocate(size);
> }
> 
>+inline void* Cache::tryAllocate(size_t alignment, size_t size)
>+{
>+    Cache* cache = PerThread<Cache>::getFastCase();
>+    if (!cache)
>+        return allocateSlowCaseNullCache(alignment, size);
>+    return cache->allocator().tryAllocate(alignment, size);
>+}
>+
> inline void* Cache::allocate(size_t alignment, size_t size)
> {
>     Cache* cache = PerThread<Cache>::getFastCase();
>Index: Source/bmalloc/bmalloc/bmalloc.h
>===================================================================
>--- Source/bmalloc/bmalloc/bmalloc.h	(revision 203920)
>+++ Source/bmalloc/bmalloc/bmalloc.h	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
>+ * Copyright (C) 2014-2016 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -43,6 +43,12 @@ inline void* malloc(size_t size)
>     return Cache::allocate(size);
> }
> 
>+// Returns null on failure.
>+inline void* tryMemalign(size_t alignment, size_t size)
>+{
>+    return Cache::tryAllocate(alignment, size);
>+}
>+
> // Crashes on failure.
> inline void* memalign(size_t alignment, size_t size)
> {
>Index: Tools/Scripts/run-jsc-stress-tests
>===================================================================
>--- Tools/Scripts/run-jsc-stress-tests	(revision 203920)
>+++ Tools/Scripts/run-jsc-stress-tests	(working copy)
>@@ -420,7 +420,7 @@ $numPasses = 0
> # We force all tests to use a smaller (1.5M) stack so that stack overflow tests can run faster.
> BASE_OPTIONS = ["--useFTLJIT=false", "--useFunctionDotArguments=true", "--maxPerThreadStackUsage=1572864"]
> EAGER_OPTIONS = ["--thresholdForJITAfterWarmUp=10", "--thresholdForJITSoon=10", "--thresholdForOptimizeAfterWarmUp=20", "--thresholdForOptimizeAfterLongWarmUp=20", "--thresholdForOptimizeSoon=20", "--thresholdForFTLOptimizeAfterWarmUp=20", "--thresholdForFTLOptimizeSoon=20", "--maximumEvalCacheableSourceLength=150000"]
>-NO_CJIT_OPTIONS = ["--useConcurrentJIT=false", "--thresholdForJITAfterWarmUp=100"]
>+NO_CJIT_OPTIONS = ["--useConcurrentJIT=false", "--thresholdForJITAfterWarmUp=100", "--resetFreeCellsBadly=true"]
> FTL_OPTIONS = ["--useFTLJIT=true"]
> 
> $runlist = []

Actions: View | Formatted Diff | Diff

Attachments on bug 160125: 284422 | 284428 | 284455 | 284506 | 284540 | 284543 | 284550 | 284621 | 284635 | 284642 | 284705 | 284709 | 284728 | 284788 | 284814 | 284889 | 284898 | 284899 | 284920 | 284951 | 284959 | 284976 | 284977 | 285168 | 285169 | 285170 | 285254 | 285260 | 285349 | 285396 | 285399 | 285401 | 285444 | 285450 | 285467 | 285481 | 285575 | 285600 | 285601 | 285623 | 285625 | 285630 | 285665 | 285673 | 285678 | 285706 | 285746 | 285804 | 285893 | 285935 | 285950 | 286010 | 286011 | 286012 | 286013 | 286014 | 286015 | 286016 | 286017 | 286033 | 286034 | 286036 | 286048 | 286050 | 286063 | 286067 | 286069 | 286074 | 286087 | 286195 | 286196 | 286351 | 286354 | 286356 | 286359 | 286362 | 286617 | 286626 | 286627 | 286631 | 286633 | 286640 | 286754 | 287135 | 287156 | 287159 | 287178 | 287186 | 287244 | 287346 | 287454 | 287458 | 287528 | 287538 | 287552 | 287595 | 287629 | 287660 | 287666 | 287718 | 287723 | 287928 | 287970 | 287982 | 287985 | 287987