Update MLXChatExample for iOS Sandboxing

[rudrankriyam]

On iOS, due to sandboxing, using the default hub gives an error along the line because it cannot store in the downloads directory:

You do not have permission to save the file "model-name" in the folder "mlx-community"

We can directly use the default parameter of HubApi() that stores under the documents directory, accessible on both iOS and macOS.

Also, I wonder the need for the default static value then?

/// Extension providing a default HubApi instance for downloading model files
extension HubApi {
    /// Default HubApi instance configured to download models to the user's Downloads directory
    /// under a 'huggingface' subdirectory.
    static let `default` = HubApi(
        downloadBase: URL.downloadsDirectory.appending(path: "huggingface"))
}

Thank you creating this example @ibrahimcetin!

[rudrankriyam]

On second thoughts, would you prefer a OS dependent approach? Keep desired behavior of downloads folder on macOS, while documents for the rest OSes.

[ibrahimcetin]

I’ve added this because my Documents directory syncs with iCloud, which caused all Hugging Face models to sync to my iCloud account. I didn’t test this on iOS, so I didn’t notice the issue earlier.

For now, using this workaround might be a better approach:

#if(os(macOS))
static let `default` = HubApi(
        downloadBase: URL.downloadsDirectory.appending(path: "huggingface"))
#elseif(os(iOS))
static let `default` = HubApi()
#endif

Later, we can improve the handling of this in HubApi for a more robust solution.

[rudrankriyam]

Let's do it #else only so it takes care of visionOS as well

[ibrahimcetin]

Thank you.

I think HubApi should use HF_HOME if configured or default huggingface path as a default on macOS. What do you think @rudrankriyam?

[rudrankriyam]

I think the HF_HOME is for the token, and not the path to store the download? For a sample project, I would prefer keeping it to default and let the developers update the path according to their needs. What do you say?

[davidkoski]

This brings up an interesting point, I think: the Chat app doesn't have a sandbox on macOS. I think it probably should, though storing files in ~/Downloads is convenient. On iOS, visionOS, etc. applications automatically have sandboxes, so the lack of one doesn't do anything and is the trigger for this problem.

I suggest the following

• add App Sandbox to the Capabilities (MLXChatExample target -> Signing & Capabilities)
  • outgoing connections (for the download)
  • file access to Downloads (read/write)
• add Increased Memory Limit to the Capabilities (this will help when running on iOS and visionOS devices)

One other option (and this might be good for iOS too) is to consider this:

• https://developer.apple.com/documentation/foundation/optimizing-your-app-s-data-for-icloud-backup

Specifically data put in ~/Library/Caches can be purged from the device by the OS as needed and doesn't end up in the iCloud backup. On macOS maybe it is convenient to be in ~/Downloads so you can see the models, but on iOS it might be good to use URL.cachesDirectory:

• https://developer.apple.com/documentation/foundation/url/cachesdirectory

[rudrankriyam]

Oh yeah, having cachesDirectory vs the default is a better idea on iOS! I will make the changes and update the PR. Thanks for pointing out, @davidkoski!

[JoeJoe1313]

Has anyone tested the current proposed changes on iOS? I get a new error now:

Offline mode error: Repository not available locally

[rudrankriyam]

I did test the proposed changes and seeing this error for the first time. The default HubApi() is same as the one used in LLMEval. Is it happening for every model?

[rudrankriyam]

I have updated the changes as you mentioned @davidkoski, and tested it on iPad. Let me know what you think!

[JoeJoe1313]

I just tested the version with cachesDirectory and it's working.

[davidkoski]

Great, thank you for setting this up!