o i3+@sddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z ddl m Z mZGddde ZddZdd Zd d Zed krIedSdS) N) QuantFormat QuantTypeStaticQuantConfigquantize)CalibrationDataReaderCalibrationMethodc@s*eZdZddZdefddZddZdS) OnnxModelCalibrationDataReaderc stj|_fddtjD}t|}g}|D]1i}fddttD}fdd|D}t ||ddD] \}} | ||j <qA| |qt |t |ks[Jt |dt |ksgJt |_dS)Ncs&g|]}|drtjj|qS)Ztest_data_set_) startswithospathjoin model_dir.0aselfi/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/quantization/static_quantize_runner.py s  z;OnnxModelCalibrationDataReader.__init__..csg|] }tj|qSr)r r r r)data_dirrrrscsg|]}|qSr)read_onnx_pb_data)rZ data_pathrrrrsF)strictr)r r dirnamer listdir onnxruntimeZInferenceSessionZ get_inputssortedzipnameappendlenitercalibration_data) r model_pathZ data_dirsZ model_inputsZ name2tensorsZ name2tensorZ data_pathsZ data_ndarrays model_inputZ data_ndarrayr)rrr__init__s     z'OnnxModelCalibrationDataReader.__init__returncCs t|jdS)z9generate the input data dict for ONNXinferenceSession runN)nextr"rrrrget_next!s z'OnnxModelCalibrationDataReader.get_nextcCsPt}t|d}||Wdn1swYtj|}|S)Nrb)onnxZ TensorProtoopenZParseFromStringreadZ numpy_helperZto_array)rZfile_pbtensorfretrrrr%s   z0OnnxModelCalibrationDataReader.read_onnx_pb_dataN)__name__ __module__ __qualname__r%dictr(rrrrrr s rcCstjdd}|jddddd|jdd dd d|jd gd d dd|jdgd ddd|jdddd|jdddd|jdddd|jdddd|jdddd|jdd gd!d"|jd#d$gd%d&d'|jd(d)d)d*gd+d'|jd,dd-d|jd.dd/d|jd0dd1d|jd2dd3d|jd4td5d6d7|jd8dd9d|jd:dd;d|jdd dd?d"|jd@d ddAd"|jdBdCdDdEgdFdG|jdHdIdJ|S)KNz%The arguments for static quantization) descriptionz-iz--input_model_pathTzPath to the input onnx model)requiredhelpz-oz--output_quantized_model_pathz'Path to the output quantized onnx modelz--activation_typeqint8quint8Zqint16Zquint16Zqint4Zquint4Z qfloat8e4m3fnr9z!Activation quantization type used)choicesdefaultr6z --weight_typer8zWeight quantization type usedz--enable_subgraph store_truez#If set, subgraph will be quantized.)actionr6z--force_quantize_no_input_checka By default, some latent operators like maxpool, transpose, do not quantize if their input is not quantized already. Setting to True to force such operator always quantize input and so generate quantized output. Also the True behavior could be disabled per node using the nodes_to_exclude.z--matmul_const_b_onlyz3If set, only MatMul with const B will be quantized.z--add_qdq_pair_to_weightzjIf set, it remains floating-point weight and inserts both QuantizeLinear/DeQuantizeLinear nodes to weight.z--dedicated_qdq_pairzFIf set, it will create identical and dedicated QDQ pair for each node.z)--op_types_to_exclude_output_quantization+z]If any op type is specified, it won't quantize the output of ops with this specific op types.)nargsr;r6z--calibration_methodminmaxr@ZentropyZ percentile distributionzCalibration method used)r;r:r6z--quant_formatqdq qoperatorzQuantization format usedz--calib_tensor_range_symmetriczoIf enabled, the final range of tensor during calibration will be explicitly set to symmetric to central point 0z--calib_moving_averagezIf enabled, the moving average of the minimum and maximum values will be computed when the calibration method selected is MinMax.z--disable_quantize_biaszWhether to quantize floating-point biases by solely inserting a DeQuantizeLinear node If not set, it remains floating-point bias and does not insert any quantization nodes associated with biases.z--use_qdq_contrib_opszIf set, the inserted QuantizeLinear and DequantizeLinear ops will have the com.microsoft domain, which forces use of ONNX Runtime's QuantizeLinear and DequantizeLinear contrib op implementations.z--minimum_real_rangeg-C6?aIf set to a floating-point value, the calculation of the quantization parameters (i.e., scale and zero point) will enforce a minimum range between rmin and rmax. If (rmax-rmin) is less than the specified minimum range, rmax will be set to rmin + MinimumRealRange. This is necessary for EPs like QNN that require a minimum floating-point range when determining quantization parameters.)typer;r6z --qdq_keep_removable_activationsz|If set, removable activations (e.g., Clip or Relu) will not be removed, and will be explicitly represented in the QDQ model.z*--qdq_disable_weight_adjust_for_int32_biaszIf set, QDQ quantizer will not adjust the weight's scale when the bias has a scale (input_scale * weight_scale) that is too small.z --per_channelz&Whether using per-channel quantizationz--nodes_to_quantizezfList of nodes names to quantize. When this list is not None only the nodes in this list are quantized.z--nodes_to_excludeznList of nodes names to exclude. The nodes in this list will be excluded from quantization when it is not None.z--op_per_channel_axisr)ZOP_TYPEZPER_CHANNEL_AXISa8Set channel axis for specific op type, for example: --op_per_channel_axis MatMul 1, and it's effective only when per channel quantization is supported and per_channel is True. If specific op type supports per channel quantization but not explicitly specified with channel axis, default channel axis will be used.)r?r=metavarr;r6z--tensor_quant_overridesz4Set the json file for tensor quantization overrides.)r6)argparseArgumentParser add_argumentfloat parse_args)parserrrrparse_arguments-s     rNcCs|siSt| }t|}Wdn1swY|D]}||D]}tj|dtjd|d<t|d|d<q%q|S)Nscale)ZdtypeZ zero_point)r+jsonloadnparrayZfloat32)filer.Zquant_override_dictr-Zenc_dictrrrget_tensor_quant_overridess   rUc Cst}t|jd}tjtjtjtjtjtj tj d}||j }||j }t |j}|j|j|j|j|j|j||j|j|j |j|j|j|jt|jd}tjtj tj!tj"d}t#j$t#j%d}t&|||j'||j(||d|j)|j*|j+ddd|d } t,|j|j-| ddS) N)r#r7)ZEnableSubgraphZForceQuantizeNoInputCheckZMatMulConstBOnlyZAddQDQPairToWeightZ"OpTypesToExcludeOutputQuantizationZDedicatedQDQPairZ QDQOpTypePerChannelSupportToAxisZCalibTensorRangeSymmetricZCalibMovingAverageZ QuantizeBiasZUseQDQContribOpsZMinimumRealRangeZQDQKeepRemovableActivationsZ"QDQDisableWeightAdjustForInt32BiasZTensorQuantOverridesrA)rCrDF) Zcalibration_data_readerZcalibrate_method quant_formatactivation_type weight_typeZop_types_to_quantizenodes_to_quantizenodes_to_exclude per_channelZ reduce_rangeZuse_external_data_formatZcalibration_providers extra_options)r$Z model_outputZ quant_config).rNrZinput_model_pathrZQInt8ZQUInt8ZQInt16ZQUInt16ZQInt4ZQUInt4Z QFLOAT8E4M3FNrWrXr3Zop_per_channel_axisZenable_subgraphZforce_quantize_no_input_checkZmatmul_const_b_onlyZadd_qdq_pair_to_weightZ'op_types_to_exclude_output_quantizationZdedicated_qdq_pairZcalib_tensor_range_symmetricZcalib_moving_averageZdisable_quantize_biasZuse_qdq_contrib_opsZminimum_real_rangeZqdq_keep_removable_activationsZ(qdq_disable_weight_adjust_for_int32_biasrUZtensor_quant_overridesrZMinMaxZEntropyZ Percentile DistributionrZQDQZ QOperatorrZcalibration_methodrVrYrZr[rZoutput_quantized_model_path) argsZ data_readerZarg2quant_typerWrXZ'qdq_op_type_per_channel_support_to_axisr\Zarg2calib_methodZarg2quant_formatZsqcrrrmainsj   r___main__)rHrPr numpyrRr*rZonnxruntime.quantizationrrrrZ"onnxruntime.quantization.calibraterrrrNrUr_r0rrrrs    =