MLX/Transforms+Eval.swift:166: Fatal error: Unable to extract MLXArray from idle

[aPaleBlueDot]

I can't figure what I'm doing wrong to get this error. It occurs with both downloaded and locally saved models. Tried these 3 ModelConfiguration s:

gemma2bQuantized
mistral7B4bit
phi3_4bit

[davidkoski]

The error comes from here:

private func collect(_ item: Any, into arrays: inout [MLXArray]) {
...
    default:
        fatalError("Unable to extract MLXArray from \(item)")

This is complaining about a value passed in to eval() or asyncEval(). Based on the idle I suspect it is probably this:

    enum LoadState {
        case idle
        case loaded(ModelContainer)
    }

but how that gets passed in ..., that is a bit of a mystery. I suggest putting a breakpoint on that line (MLX/Transforms+Eval.swift:166) and looking at what calls it.

[aPaleBlueDot]

Indeed, somehow the model is probably not getting loaded. It's very likely that my Xcode config is incorrect, as it's untouched example code that gives this error. Putting a breakpoint now.

[davidkoski]

You may be missing entitlements for network access (to download the weights)

[aPaleBlueDot]

It downloads the model directory from HF before this error. I later switched to using a local folder, though nothing changed. Also noticed another warning prior to this one: BOOL _NSPersistentUIDeleteItemAtFileURL(NSURL *const __strong) Failed to stat item: file:///Users/hg/Library/Containers/mlx.LoRATrainingExample.GH/Data/Library/Saved%20Application%20State/mlx.LoRATrainingExample.GH.savedState/restorecount.plist

[aPaleBlueDot]

Attaching the screenshots of the call stack right before this exception is thrown.

[davidkoski]

Ah, I see. Delete this line in LoRAEvaluator.loadModel():

            eval(model)

I will prepare a PR for that, but in the meantime that should improve things -- we don't need that line.

[davidkoski]

Thank you for reporting this and helping to find the cause!

[aPaleBlueDot]

Thank you for your spotting it in the blink of an eye. This team rocks!