Issues with generation results after training

[GuileGaze]

I just finished training my first model (anime character on AnythingV3fp32) and I'm having a couple issues with its outputs.

The first and most important is that larger resolutions break things really bad. Fidelity is lost with regards to the original design and anything past about 1024x1024 ends up completely deformed (including img2img results). Example.
Secondly, the images themselves seem to struggle a bit when trying to "change" the image from the base design. This includes different poses, expressions, clothes, etc. For example, if I want the character to wear a black dress, it really struggles with the change. Example.
Lastly if I try to generate anything else, the features of the trained character are noticeably beginning to bleed into it. Example.

For reference, I trained the model on 15 reference images at 3000 steps with the text encoder set to 50% using the colab. Everything that wasn't a prompt on the colab was left default (ie, I didn't change anything in the code itself).

My main issue is things breaking at higher resolutions and the images being "samey" since I can always switch back to the base model if I want to generate anything other than the trained character (although being able to work in tandem would be nice). Is there anything I can do to fix this problem or should I look for another fork/colab if I want more fine-tuned results?

[TheLastBen]

the stable diffusion model is trained on 512px images, going over that might cause problems, there is the "highres.fix" checkbox in the A1111 UI, use it to fix that issue, and try training with high resolution images, ex 768x768 or 640x640 and set it to fp16 to save time.

[GuileGaze]

Would training on higher res images fix the problem, even if the problem is also occuring on img2img as well? I'm asking this because every other model I've used has had no issues with img2img upscaling.

Also, would using regularization images help improve the results in any way? I created 300 images of "person" in the model I want to train (since I'm not using the SD model). If so, would I be able to just change one of these lines and point it to my directory instead?

[TheLastBen]

from my experience, regularization images don't improve the result. if you train it enough on high res pics, you will get good results at 1024x1024
here is an example of a model I trained on 1024px images : (no highres.fix)

[GuileGaze]

I see, I have one last question then if you don't mind. If the model still isn't getting details exactly right, should I just continue to train the model even if I know it's most likely overfitting? Other results are already being influenced so I figure if I continue to train, I'd be using the model only for that specific character.

[TheLastBen]

if you resume the training, it won't overfit, you can resume as much as you want, make sure you resume without the text_encoder.

[GuileGaze]

Alright, as soon I get access to a GPU again I'll try things out. Thanks!