sketch inpaint from a1111 for non-inpaint models #4824
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
There is lots of things from auto1111 if we can port. Like highres fix. |
Contributor
|
Cool! |
|
Without a specially trained inpainting model, does this trick really work? My experiment is that the inpainted area does not respect the context well. |
Contributor
Author
|
@deepconv yes, it works:
I used this model https://civitai.com/models/28059?modelVersionId=89464 for tests. |
AmericanPresidentJimmyCarter
pushed a commit
to AmericanPresidentJimmyCarter/diffusers
that referenced
this pull request
Apr 26, 2024
* Create masked_stable_diffusion_img2img.py * add MaskedIm2ImPipeline to readme * Update README.md
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
sketch inpaint from a1111 for non-inpaint models.
Pipleline takes an image and a mask as inputs. It does inpainting by doing this step on each iteration
latent = latent * mask + image_latent * (1 - mask)
that is we replace latent code for non-masked parts of images with latents from the encoding of the image
code in a1111 works with both: inpainting and non-inpainting models. The pipeline from this pr will work only with non-inpaint ones