[quant] QoL improvements for pipeline-level quant config

[sayakpaul]

What does this PR do?

• Shifts PipelineQuantizationConfig to its module to keep the src/diffusers/quantizers/__init__.py clean.
• Adds a __repr__ method to PipelineQuantizationConfig so that users can investigate it in between their workflows.
• Sets the quantization_config attribute of a pipeline if it's not None.

I tested with the following script and got the quantization config printed nicely as below:

text_encoder_2 BitsAndBytesConfig {
  "_load_in_4bit": true,
  "_load_in_8bit": false,
  "bnb_4bit_compute_dtype": "bfloat16",
  "bnb_4bit_quant_storage": "uint8",
  "bnb_4bit_quant_type": "nf4",
  "bnb_4bit_use_double_quant": false,
  "llm_int8_enable_fp32_cpu_offload": false,
  "llm_int8_has_fp16_weight": false,
  "llm_int8_skip_modules": null,
  "llm_int8_threshold": 6.0,
  "load_in_4bit": true,
  "load_in_8bit": false,
  "quant_method": "bitsandbytes"
}
transformer BitsAndBytesConfig {
  "_load_in_4bit": true,
  "_load_in_8bit": false,
  "bnb_4bit_compute_dtype": "bfloat16",
  "bnb_4bit_quant_storage": "uint8",
  "bnb_4bit_quant_type": "nf4",
  "bnb_4bit_use_double_quant": false,
  "llm_int8_enable_fp32_cpu_offload": false,
  "llm_int8_has_fp16_weight": false,
  "llm_int8_skip_modules": null,
  "llm_int8_threshold": 6.0,
  "load_in_4bit": true,
  "load_in_8bit": false,
  "quant_method": "bitsandbytes"
}

Code

from diffusers import PipelineQuantizationConfig, DiffusionPipeline
import torch

components_to_quantize = ["transformer", "text_encoder_2"]
quant_config = PipelineQuantizationConfig(
    quant_backend="bitsandbytes_4bit",
    quant_kwargs={
        "load_in_4bit": True,
        "bnb_4bit_quant_type": "nf4",
        "bnb_4bit_compute_dtype": torch.bfloat16,
    },
    components_to_quantize=components_to_quantize,
)
pipe = DiffusionPipeline.from_pretrained(
    "hf-internal-testing/tiny-flux-pipe",
    quantization_config=quant_config,
    torch_dtype=torch.bfloat16,
).to("cuda")
print(pipe.quantization_config)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

So that we can do from diffusers import PipelineQuantizationConfig.

[SunMarc]

Thanks for this !

[yiyixuxu]

thanks!

[sayakpaul]

Failing tests are unrelated.