And How to Quantize LLMs After a Merge
In your notebook, I can't find dequantize_model function, so can't do experiment
The method dequantize_model is at the beginning of the section "Benchmarking: Inference throughtput and accuracy" in the notebook.
In your notebook, I can't find dequantize_model function, so can't do experiment
The method dequantize_model is at the beginning of the section "Benchmarking: Inference throughtput and accuracy" in the notebook.