o ¢Äi# ã@s`ddlZddlZddlZddlZddlmZe e¡Zdd„Z dd„Z dd„Z Gd d „d ƒZ dS) éN)ÚConv1DcCs<|jj\}}tj ||¡}|jjj ¡|j_|jj|j_|S)N) ÚweightÚshapeÚtorchÚnnÚLinearÚdataÚTÚ contiguousZbias)ÚmoduleZin_sizeZout_sizeÚlinear©r úb/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/transformers/quantize_helper.pyÚ_conv1d_to_linears  rcCsNt d¡t|jƒD]}|j|}t|tƒr t|ƒ}||j|<q t|ƒq dS)zsin-place This is for Dynamic Quantization, as Conv1D is not recognized by PyTorch, convert it to nn.Linear zreplace Conv1D with LinearN)ÚloggerÚdebugÚlistZ_modulesÚ isinstancerrÚconv1d_to_linear)ÚmodelÚnamer r r r rrs     úrcCs.t | ¡d¡tj d¡d}t d¡|S)Nztemp.pé)rÚsaveZ state_dictÚosÚpathÚgetsizeÚremove)rÚsizer r rÚ_get_size_of_pytorch_model's rc@s,eZdZeejfdd„ƒZeddd„ƒZdS)ÚQuantizeHelpercCsLt|ƒtjj|tjjh|d}t dt|ƒ›¡t dt|ƒ›¡|S)z{ Usage: model = quantize_model(model) TODO: mix of in-place and return, but results are different )Údtypez'Size of full precision Torch model(MB):z"Size of quantized Torch model(MB):) rrZ quantizationÚquantize_dynamicrrrÚinfor)rr Zquantized_modelr r rÚquantize_torch_model/s z#QuantizeHelper.quantize_torch_modelFcCsddlm}ddlm}||ƒjjdddt dtj   |¡d›¡||||dt j j id t d |›¡t d tj   |¡d›¡dS) Nr)ÚPath)r!T)ÚparentsÚexist_okz&Size of full precision ONNX model(MB):rZDefaultTensorType)Úuse_external_data_formatZ extra_optionszquantized model saved to:z!Size of quantized ONNX model(MB):)Úpathlibr$Zonnxruntime.quantizationr!ÚparentÚmkdirrr"rrrÚonnxZ TensorProtoÚFLOAT)Zonnx_model_pathZquantized_model_pathr'r$r!r r rÚquantize_onnx_model<s   ü z"QuantizeHelper.quantize_onnx_modelN)F)Ú__name__Ú __module__Ú __qualname__Ú staticmethodrZqint8r#r-r r r rr.s  r) Úloggingrr+rZtransformers.modeling_utilsrÚ getLoggerr.rrrrrr r r rÚs