Skip to content

Conversation

hardikdava
Copy link
Contributor

Description

sv.DetectionDatasets is using image_name as keys for Dict. This can be leading to create issues like same there can be possibility of having same name in multiple dataset. Also, saving image_path can be useful for not loading the whole image dataset but only load when necessary.

This PR is only redesigning the existing code.

@hardikdava
Copy link
Contributor Author

@SkalskiP This is an initial PR is for lazy dataset loading and integration for yoloexplorer.

@hardikdava
Copy link
Contributor Author

@capjamesg Please check it out if anything changes for autodistill.

@hardikdava hardikdava marked this pull request as ready for review August 17, 2023 19:45
@hardikdava
Copy link
Contributor Author

@SkalskiP can we quickly merge this? Then we are good to go for yoloexplorer integration.

@SkalskiP SkalskiP requested review from SkalskiP and capjamesg August 18, 2023 10:24
@SkalskiP SkalskiP added api:datasets Dataset API version 0.14.0 Feature to be added in `0.14.0` release labels Aug 18, 2023
@SkalskiP SkalskiP added this to the version: 0.14.0 milestone Aug 18, 2023
@SkalskiP
Copy link
Collaborator

@hardikdava merge it or release it? :) Because the release will probably come in 1-2 weeks. But merge can be today.

@hardikdava
Copy link
Contributor Author

@SkalskiP let's merge it for now. I will make changes accordingly for initial integration.

@SkalskiP
Copy link
Collaborator

@hardikdava, we cant merge it. You made changes only in .from_* methods and not in .as_*. All methods that save annotations assume they operate on image names, not image paths, and try to join the path of the target directory with the source image path.

@hardikdava
Copy link
Contributor Author

@SkalskiP noted.

@SkalskiP
Copy link
Collaborator

@hardikdava let me know when you'll have a version ready for tests 🙏🏻

@hardikdava
Copy link
Contributor Author

Hey 👋 @SkalskiP , I have changed the required method. Let me know I have missed anything else.

@SkalskiP
Copy link
Collaborator

Hi, @hardikdava 👋🏻 do you maybe have some colab I could use for testing?

@SkalskiP
Copy link
Collaborator

Hi @hardikdava 👋🏻 I'm testing it now.

@SkalskiP
Copy link
Collaborator

@SkalskiP
Copy link
Collaborator

Hi @hardikdava 👋🏻! Merging.

@SkalskiP SkalskiP merged commit 6b0be9c into develop Aug 22, 2023
@SkalskiP SkalskiP deleted the fix/yoloexplorer branch January 2, 2024 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api:datasets Dataset API version 0.14.0 Feature to be added in `0.14.0` release
Projects
Status: Current Release: Done
Development

Successfully merging this pull request may close these issues.

2 participants