Bad performance when disable concurrent GC

[NDobrev]

Hi everyone,

I decide to disable ENABLE_CONCURRENT_GC, because in our application we don't want extra reserved memory for barriers table, but this result in serious performance implications.Basicly our application spend 95% of the time in GC, because it looks like every API call trigger cleanup.
So i decide to disable it for function ContextAPIWrapper_Core, but this result in a crash in JavascriptArray::DirectSetItem_Full.
Look like this line SparseArraySegment* current = (SparseArraySegment*)this->GetBeginLookupSegment(itemIndex, false);
returns wrong pointer(misinterpreted) or memory behind it is stomped.
I try to add 24 byte marker at begin of SparseArraySegmentBase which i set to zero and when crash occur “current” looks like :

Marker - >0x00000002bc9e8308 {0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00,
0x83, 0x46, 0x4d, 0xc8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x06, 0xc1, 0xff, 0xff, 0xff, 0xff}
Left -> 0xffffffff
Length ->0x00000000
size-> 0x00000005
Next -> 0xffffffff000206c1 { buffer=0xffffffff000206c1 {???, ???, ???, ???, ???, ???} left=???

Here is part of the call stack:

Js::SparseArraySegment<void*>::AllocateSegmentImpl(Memory::Recycler* recycler, Js::SparseArraySegmentBase* prev, uint32 index) Line 205 + 0x4 bytes C++
Js::SparseArraySegment<void*>::AllocateSegment(Memory::Recycler* recycler, Js::SparseArraySegmentBase* prev, uint32 index) Line 194 + 0x10 bytes C++
Js::JavascriptArray::DirectSetItem_Full<void*>(uint32 itemIndex, void* newValue) Line 1413 + 0x10 bytes C++
Js::JavascriptArray::DirectSetItemAt<void*>(uint32 itemIndex, Js::Var newValue) Line 455 C++
Js::JavascriptArray::SetItem(uint32 index, Js::Var value, enum class Js::PropertyOperationFlags flags) Line 12534 + 0xc bytes C++
Js::JavascriptOperators::OP_SetElementI(Js::Var instance, Js::Var index, Js::Var value, Js::ScriptContext* scriptContext, enum class Js::PropertyOperationFlags flags) Line 4362 C++
Js::InterpreterStackFrame::OP_SetElementI<Js::OpLayoutT_ElementI<Js::LayoutSizePolicy<(LayoutSize)0>>>(const Js::OpLayoutT_ElementI<Js::LayoutSizePolicy<(LayoutSize)0>>* playout, enum class Js::PropertyOperationFlags flags) Line 5168 + 0x3c bytes C++
Js::InterpreterStackFrame::OP_SetElementIStrict<Js::OpLayoutT_ElementI<Js::LayoutSizePolicy<(LayoutSize)0>>>(const Js::OpLayoutT_ElementI<Js::LayoutSizePolicy<(LayoutSize)0>>* playout) Line 5212 C++
Js::InterpreterStackFrame::ProcessUnprofiled() Line 268 + 0x26 bytes C++
Js::InterpreterStackFrame::Process() Line 3521 + 0x9 bytes C++
Js::InterpreterStackFrame::ProcessThunk(void* address, void* addressOfReturnAddress) Line 2525 + 0x9 bytes C++
Js::InterpreterStackFrame::InterpreterHelper(Js::ScriptFunction* function, Js::ArgumentReader args, void* returnAddress, void* addressOfReturnAddress, struct Js::InterpreterStackFrame::AsmJsReturnStruct* asmJsReturn) Line 2033 + 0x14 bytes C++
Js::InterpreterStackFrame::InterpreterThunk(Js::RecyclableObject* function, Js::CallInfo callInfo, ...) Line 1754 + 0x37 bytes C++

Do you have some ideas what is source the crash or some suggestions how to reduce time spent in GC when ENABLE_CONCURRENT_GC is disable.

[jaimecbernardo]

Hi,

I've also observed this behavior with concurrent GC off. The best solution for my case was turning the concurrent GC on, but I've collected some info that might helpful.

This behavior seems to be caused by heuristics that try to aggressively clean memory if your case is using memory above a certain threshold / has allocated lots of memory in a small time period. For parallel GC, that's ok, since each GC call just requests the background cleaning. The problem is that without concurrent GC, the runtime will have to wait for the GC call to finish before continuing and this GC call will be made at each allocation.

This behavior can be made better if you increase the GC heuristic values here: https://github.com/Microsoft/ChakraCore/blob/de80038b6ce0d224e121ff2439564a95dfbcd3a6/lib/Common/Memory/RecyclerHeuristic.cpp#L37-L59
One thing I think helps a lot is increasing the baseFactor to above the "max" memory in MB that is expected to be real working memory in your case. Still, if you're stressing the engine to always be creating more objects in a small amount of time, you'll still be able to observe the GC kicking in from time to time.
Another possible solution for this would be to work in some sort of mechanism that limits the calls to the GC so they are not made for each alloc request. (But they still need to be made eventually, nonetheless).

[rhuanjl]

As a user of CC I've found that if I want to limit GC calls the best way is to cut down the number of objects being created and disposed of in my JS code - work out ways to reuse objects rather than continually making new ones etc.

[NDobrev]

Yeah, but we don't have control over the code of our users, so we can just reduce the amount of garbage with good practices.

[rhuanjl]

One other suggestion I've thought of that may help - you could try creating your runtime with JsRuntimeAttributeEnableIdleProcessing and then include a JsIdle call at some point in your workflow/event loop. (This attribute tells CC that there will be a good time to recurring tasks like GC calls and so not to do them all the time and then JsIdle tells it when to do them)

[curtisman]

You mentioned that you want to disable concurrent GC because you don't want to the write barrier table.
Did you disable the write barrier table? You probably need to disable partial GC as well, since that relies on write barrier as well.

[NDobrev]

Thank you very much for the info and suggestions. Looks like JsRuntimeAttributeEnableIdleProcessing flag have good impact.