Support BitsAndBytesConfig
#6
by
brand17
- opened
Please add support of BitsAndBytesConfig
Hi,
Apologies for the late reply, yes you could able to done the quantization by using the BitsAndBytesConfig on google/gemma-3-12b-it model. The quantized model is already available from unsloth if you would like to explore more and experiment with it, please find the following page. Please feel free to reach out to us if you required any further assistance.
Thanks.