o i @sddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl Z ddl Z ddl mZmZmZmZddlZddlmZmZmZde jd e jd e jfd d Z  d0de jd e jd edBded e jf ddZGdddZGdddZGdddeZGdddejdZ GdddZ!Gddde!Z"Gddde!Z#Gd d!d!e#Z$Gd"d#d#e#Z%Gd$d%d%e#Z&Gd&d'd'ejdZ'Gd(d)d)e'Z(dd*ej)d+difd,e*e Bd-ee*dBfd.d/Z+dS)1N)Sequence)Enum)Path) ModelProto TensorProtohelper numpy_helper) apply_plotload_model_with_shape_infersmooth_distributionpkqkreturncCs|tj|j|jd}|ddt|dd|dd|dd<|dk|dk@}d||<|dk|dk@}tj||<|S)z See https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.rel_entr.html#scipy.special.rel_entr. Python implementation. dtypeNr)npemptyshaperloginf)r rresc2c1r\/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/quantization/calibrate.pyrel_entrs2 rbaseaxiscCs|dus |dks Jd|dusJdt|tj}d|tj||dd}t|tj}t||\}}d|tj||dd}t||}tj||d}|dur]|t|}||jS) z Simplifeied version of entropy. Source: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html. This avoids taking a dependency on scipy just for this function. Nrz0base={base} must be a positive number or `None`.z qk is None?T)rkeepdimsr) rasarrayastypefloat32sumZbroadcast_arraysrrr)r rrrZvecsrrrentropy's   r'c@sLeZdZegdZegdZddZeddZeddZ d d Z d S) TensorData)avgstdlowesthighesthist hist_edgesbins)r)r*r+r,r.cKst||_|D]E\}}|tjvr td|dtjd|tjvrJt|ds6tdt |d||j t j t j fvrJtd|j d|t|||q dS)NzUnexpected value z not in .rUnexpected type z for k=zUnexpected dtype )listkeys_attrsitemsr(_allowed ValueError_floatshasattrtyperrZfloat16r$setattr)selfkwargskvrrr__init__Gs   zTensorData.__init__cC4t|dr t|dstdt|d|j|jfS)Nr+r,z0Attributes 'lowest' and/or 'highest' missing in r0)r9AttributeErrordirr+r,r<rrr range_valueS zTensorData.range_valuecCrA)Nr)r*z)Attributes 'avg' and/or 'std' missing in r0)r9rBrCr)r*rDrrravg_stdYrFzTensorData.avg_stdcs$fddjD}jj|d<|S)Ncsi|]}|t|qSr)getattr).0r>rDrr asz&TensorData.to_dict..CLS)r4 __class____name__r<datarrDrto_dict_s zTensorData.to_dictN) rM __module__ __qualname__ frozensetr6r8r@propertyrErGrPrrrrr(Cs     r(c@sfeZdZdeeeeBffddZddZddZ dd Z d d Z d d Z ddZ ddZddZdS) TensorsDatarOc Cs||_i|_|D]p\}}t|tstdt|dt|trf|tj kr;t |dkr;t |d|dd|j|<q t |dkrUt |d|d|d|dd |j|<q td |d d t |d |dt|t sutdt|d||j|<q dS)NzKeys must be strings not r0rr r+r,)r+r,r-r/zUnexpected tuple for rz , it has z elements: zValues must be TensorData not ) calibration_methodrOr5 isinstancestr TypeErrorr:tupleCalibrationMethodMinMaxlenr()r<r[rOr>r?rrrr@gs"   &"  zTensorsData.__init__ccs|jEdHdSNrOrDrrr__iter__yszTensorsData.__iter__cCs ||jvSrcrdr<keyrrr __contains__| zTensorsData.__contains__cCs |j|Srcrdrfrrr __getitem__rizTensorsData.__getitem__cCs(||jvr td|d||j|<dS)Nz)Only an existing tensor can be modified, z is not.)rO RuntimeError)r<rgvaluerrr __setitem__s zTensorsData.__setitem__cC |jSrc)rOr3rDrrrr3rizTensorsData.keyscCrnrc)rOvaluesrDrrrrorizTensorsData.valuescCrnrc)rOr5rDrrrr5rizTensorsData.itemscCs|jj|j|jd}|S)N)rKrOr[)rLrMrOr[rNrrrrPs zTensorsData.to_dictN)rMrQrRdictr]r(r_r@rerhrjrmr3ror5rPrrrrrUfs rUc@seZdZdZdZdZdZdS)r`rr rVrYN)rMrQrRraEntropy Percentile Distributionrrrrr`s r`c@sVeZdZeddZejdefddZddZ dd Z d d Z d e d e fddZ dS)CalibrationDataReadercCst|dr t|jp tS)Nget_next)r9callableruNotImplemented)clssubclassrrr__subclasshook__sz&CalibrationDataReader.__subclasshook__rcCt)z9generate the input data dict for ONNXinferenceSession runNotImplementedErrorrDrrrruszCalibrationDataReader.get_nextcCs|SrcrrDrrrrezCalibrationDataReader.__iter__cCs|}|dur t|Src)ru StopIteration)r<resultrrr__next__szCalibrationDataReader.__next__cCr{rcr|rDrrr__len__r~zCalibrationDataReader.__len__ start_index end_indexcCr{rcr|)r<rrrrr set_ranger~zCalibrationDataReader.set_rangeN)rMrQrR classmethodrzabcabstractmethodrprurerrintrrrrrrts rt) metaclassc@seZdZ     ddeeBdeedBfddZdgfd d Zd d Zd e fddZ ddZ ddZ de fddZdefddZdS)CalibraterBaseNaugmented_model.onnxF model_pathop_types_to_calibratecCsnt|tr tt||_nt|trt||_ntd||_||_||_||_ ||_ d|_ d|_ dg|_ dS)a :param model_path: ONNX model to calibrate. It should be a model file path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param symmetric: make range of tensor symmetric (central point is 0). :param use_external_data_format: use external data format to store model which size is >= 2Gb. :param per_channel: whether to compute ranges per each channel. z model_path should be model path.NCPUExecutionProvider)r\r]r rmodelr7raugmented_model_path symmetricuse_external_data_format per_channelZ augment_model infer_sessionexecution_providers)r<rrrrrrrrrr@s    zCalibraterBase.__init__rcCs||_|dS)zz reset the execution providers to execute the collect_data. It triggers to re-creating inference session. N)rcreate_inference_session)r<rrrrset_execution_providerss z&CalibraterBase.set_execution_providerscCs,t}tjj|_tj|j||jd|_dS)z9 create an OnnxRuntime InferenceSession. ) sess_options providersN) onnxruntimeZSessionOptionsZGraphOptimizationLevelZORT_DISABLE_ALLZgraph_optimization_levelZInferenceSessionrrr)r<rrrrrs  z'CalibraterBase.create_inference_sessionrc Csdd|jjD}|dd|jjD|dd|jjDdd|jjD}t}tjtj h}|jj D]4}|j rB|j |j vrkt |j|jD] }||vrj||}|jdrj|jjj|vrj||vrj||qJq7||fS)z select input/output tensors of candidate nodes to calibrate. returns: tensors (set): set of tensor name. value_infos (dict): tensor name to value info. cSi|]}|j|qSrnamerIvirrrrJz>CalibraterBase.select_tensors_to_calibrate..cSrrr)rIZotrrrrJrcSrrr)rIitrrrrJrcSh|]}|jqSrr)rIinitrrr z=CalibraterBase.select_tensors_to_calibrate.. tensor_type)graph value_infoupdateoutputinput initializersetrFLOATZFLOAT16noderop_type itertoolschainr:ZHasFieldr elem_typeadd) r<r value_infosrtensors_to_calibrateZtensor_type_to_calibrater tensor_namerrrrselect_tensors_to_calibrates$    z*CalibraterBase.select_tensors_to_calibratecC|jS)zP return: augmented onnx model. Call after calling augment_graph )rrDrrrget_augment_modelsz CalibraterBase.get_augment_modelcCr{)z abstract method: augment the input model to prepare for collecting data. It will: 1. augment the model to be able to collect desired statistics data 2. save augmented model to augmented_model_paths r|rDrrr augment_graphszCalibraterBase.augment_graph data_readercCr{)z abstract method: collect the tensors that will be used for range computation. It can be called multiple times. r|)r<rrrr collect_datazCalibraterBase.collect_datarcCr{)ze abstract method: compute data based on the calibration method stored in TensorsData r|rDrrr compute_data"rzCalibraterBase.compute_data)NrFFF)rMrQrRr]rrr@rrrrrrrtrrUrrrrrrs$  " rcsxeZdZ        ddeeBdeedBffdd Zd d Zd d Zd e fddZ ddZ de fddZ ZS)MinMaxCalibraterNrF{Gz?rrc s|tj|||||| dg|_d|_t|jjj|_dd|jjjD|_ ||_ |r6|dks2|dkr6t d||_ ||_ dS)aw :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param symmetric: make range of tensor symmetric (central point is 0). :param use_external_data_format: use external data format to store model which size is >= 2Gb :param moving_average: compute the moving average of the minimum and maximum values instead of the global minimum and maximum. :param averaging_constant: constant smoothing factor to use when computing the moving average. :param max_intermediate_outputs: maximum number of intermediate outputs before an intermediate range is computed. :param per_channel: whether to compute ranges per each channel. )rrrrrNcSrrrrIrrrrrLrz,MinMaxCalibrater.__init__..rr z;Invalid averaging constant, which should not be < 0 or > 1.)superr@intermediate_outputscalibrate_tensors_rangerbrrrnum_model_outputsmodel_original_outputsmoving_averager7averaging_constantmax_intermediate_outputs) r<rrrrrrrrrrLrrr@*s" zMinMaxCalibrater.__init__csj\}}ttttjdgtj d}jj j |ddfdd}|D] }||d||dq1t jjjjd d S) z Adds ReduceMin and ReduceMax nodes to all quantization_candidates op type nodes in model and ensures their outputs are stored as part of the graph output :return: augmented ONNX model rcSs6|jD]}tj||jr|jSqtd|d)Nz&Model does not contain a version for 'z'.) opset_importonnxZdefshasdomainversionrk)rrrrrrget_op_version^s  z6MinMaxCalibrater.augment_graph..get_op_versionc sd}|d|}|d}tjj||g|g||d}tjjd|g|g|d}ddjjjD}|d djjjD|d djjjD||vrV||j j j }nt d |d j rt||j j jj} d gtd| } |jdkr|jtd| n tt} ttj| tjd| } |j| jjj| jjj||gjjjt ||dgdS)Nr _Z_Reshape)r rZReshape)inputsoutputsrcSrrrrrrrrJwrzNMinMaxCalibrater.augment_graph..add_reduce_min_max..cSrrr)rIorrrrJxrcSrrrrIirrrrJyrz'Unable to guess tensor type for tensor zE, running shape inference before quantization may resolve this issue.rrVZaxesr)!rrZ make_noderrrrrrr:rrr7rrbrdimrange attributeappendZmake_attributer]uuiduuid4r from_arrayrarrayint64rrextendZmake_tensor_value_info) rZreduce_op_namer Z reduce_outputintermediate_outputZ reduce_nodeZ reshape_noderZ onnx_typeZ tensor_rankZ reduced_axesZreduce_axes_nameZ reduce_axesrZreshape_shape_namer<rradd_reduce_min_maxds>     z:MinMaxCalibrater.augment_graph..add_reduce_min_maxZ ReduceMinZ ReduceMaxZsave_as_external_dataN)rrr]rrrrrrrrrrrsaverr)r<ZtensorsrZ reshape_shapertensorrrrrSs .   zMinMaxCalibrater.augment_graphcC g|_dSrcrrDrrrclear_collected_datariz%MinMaxCalibrater.clear_collected_datarcCs |}|sn|j|jd||jdur$t|j|jkr$|qt|jdkr5|jdur5t d| }t |t sHt dt|d|dS)NTrNo data is collected.z+compute_data must return a TensorsData not r0)rurrrrunrrbrrr7rr\rUr^r:)r<rrtrrrrs    zMinMaxCalibrater.collect_datac Cs|s|S|D]o\}}t|tr|jd}|jd}n|\}}t||tr6||jd}||jd}n||\}}|jrR||j||} ||j||} n t||} t||} t|tsht||trqt| | d||<q| | f||<q|S)Nrr rW)r5r\r(rErrminmax) r<Z old_rangeZ new_rangergrlold_minold_maxnew_minnew_max min_value max_valuerrr merge_ranges(      zMinMaxCalibrater.merge_rangerc stjdkr jSfddttjdDfddjD}i|D]}|D] \}}|g|q.q(jdfddtdtdD}fdd D}g}tdtdD]X}jrt j ||dd } t j ||d dd } nt j ||dd } t j ||d dd } j rt j t | t | gdd } || | fqg|| | fqgttjtt||d d } jrۈj| _jS| _jS)z Compute the min-max range of tensor :return: dictionary mapping: {added node names: (ReduceMin, ReduceMax) pairs } rcsg|] }j|jqSr)r get_outputsrrrDrr sz1MinMaxCalibrater.compute_data..cg|] }tt|ddqSFstrictrpziprIr output_namesrrrNcsg|] }|ddqS)rr) rpartitionr)added_output_namesrrrsrVcs i|] }|jvr||qSr)rr)merged_output_dictr<rrrJsz1MinMaxCalibrater.compute_data..r!r Fr)rbrrrr5 setdefaultrrrrZnanmeannanminnanmaxrabsrUr`rarprr) r<output_dicts_listdr>r?Zcalibrate_tensor_namesZmerged_added_output_dictpairsrZmin_value_arrayZmax_value_arrayZmax_absolute_valueZnew_calibrate_tensors_ranger)rrrr<rrsJ    zMinMaxCalibrater.compute_data)NrFFFrNF)rMrQrRr]rrr@rrrtrrrUr __classcell__rrrrr)s& )I!rcsreZdZ         dd eeBd eedBffd d Zd dZddZde fddZ de fddZ Z S)HistogramCalibraterNrF percentile-X@samerrc svtj|||||dg|_d|_t|jjj|_dd|jjjD|_ d|_ ||_ ||_ ||_ | |_d|_| |_dS)a= :param model_path: ONNX model to calibrate. It is a model path. :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_bins: number of bins to create a new histogram for collecting tensor values. :param num_quantized_bins: number of quantized bins. Default 128. :param percentile: A float number between [0, 100]. Default 99.99. :param scenario: see :class:`DistributionCalibrater` )rrrrNcSrrrrrrrr.rz/HistogramCalibrater.__init__..)rr@rrrbrrrrr collectormethodnum_binsnum_quantized_binsrrscenario) r<rrrrrrrrrrrrrr@ s$ zHistogramCalibrater.__init__cCsV||j\|_}|jD]}||jvr|jjj||q tj|j|j |j ddS)z make all quantization_candidates op type nodes as part of the graph output. :return: augmented ONNX model rN) rrrrrrrrrrr)r<rrrrrr7s   z!HistogramCalibrater.augment_graphcCrrcrrDrrrrGriz(HistogramCalibrater.clear_collected_datarc sBddjD}ddjD |}|sn-jd|}g}t|D]\}}||vr<|t|q)||q)j|qt jdkrTt dfd djD}i|D]} | D] \} } | g| qhqbfd d D} j stjjjjjjd _ j | dS) zy Entropy Calibrator collects operators' tensors as well as generates tensor histogram for each operator. cSrrrrIZnode_argrrrrNrz3HistogramCalibrater.collect_data..cSsg|]}|jqSrrrrrrrOrz4HistogramCalibrater.collect_data..TNrrcrrrrrrrrercs i|] }|jvr||qSr)rr) merged_dictr<rrrJos z4HistogramCalibrater.collect_data..)rrrrrr)rZ get_inputsrrur enumeratercopyrrbr7r5r rHistogramCollectorrrrrrrcollectr) r<rZinput_names_setrrZ fixed_outputsZ output_indexrr rr>r?Zclean_merged_dictr)rrr<rrJsH      z HistogramCalibrater.collect_datarcCsh|jstdt|trtj}nt|trtj}nt|tr"tj }n t dt |dt ||j S)z Compute the min-max range of tensor :return: dictionary mapping: {tensor name: (min value, max value)} z9No collector created and can't generate calibration data.zUnknown calibrater z". This method must be overwritten.)rr7r\EntropyCalibraterr`rqPercentileCalibraterrrDistributionCalibraterrsr^r:rUcompute_collection_result)r<calrrrr~s   z HistogramCalibrater.compute_data) NrFrFrrrr)rMrQrRr]rrr@rrrtrrUrrrrrrr s& ,4rcsBeZdZ       d deeBdeedBffdd ZZS) r"NrFr'rrrc  tj||||||||ddS)a :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile', 'distribution']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_bins: number of bins to create a new histogram for collecting tensor values. :param num_quantized_bins: number of quantized bins. Default 128. )rrrrNrr@) r<rrrrrrrrrrrr@ zEntropyCalibrater.__init__)NrFr'FrrrMrQrRr]rrr@rrrrrr" r"csBeZdZ       d deeBdeedBffd d ZZS) r#NrFrrrrrc r')a :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile', 'distribution']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_quantized_bins: number of quantized bins. Default 128. :param percentile: A float number between [0, 100]. Default 99.99. )rrrrNr() r<rrrrrrrrrrrr@r)zPercentileCalibrater.__init__)NrFrFrrr*rrrrr#r+r#cs@eZdZ      d deeBdeedBffd d ZZS) r$NrF distributionrrrrc stj|||||||ddS)a :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile', 'distribution']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_bins: number of bins to create a new histogram for collecting tensor values. :param scenario: for float 8 only, if `scenario="same"`, the algorithm weights and float 8 follow the same distribution, if `scenario="p3"`, it assumes the weights follow a gaussian law and float 8 ~ X^3 where X is a gaussian law )rrrNr()r<rrrrrrrrrrr@s zDistributionCalibrater.__init__)NrFr,rrr*rrrrr$s r$c@s,eZdZdZejddZejddZdS)CalibrationDataCollectorzL Base class for collecting data for calibration-based quantization. cCr{)z Generate informative data based on given data. name_to_arr : dict tensor name to NDArray data r|r< name_to_arrrrrr!sz CalibrationDataCollector.collectcCr{)z? Get the optimal result among collection data. r|rDrrrr%sz2CalibrationDataCollector.compute_collection_resultN)rMrQrR__doc__rrr!r%rrrrr-s  r-c@sveZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ e dddZddZddZdS)r a` Collecting histogram for each tensor. Percentile and Entropy method are supported. ref: https://github.com//apache/incubator-mxnet/blob/master/python/mxnet/contrib/quantization.py ref: https://docs.nvidia.com/deeplearning/tensorrt/pytorch-quantization-toolkit/docs/_modules/ pytorch_quantization/calib/histogram.html cCs.i|_||_||_||_||_||_||_dSrc)histogram_dictrrrrrr)r<rrrrrrrrrr@s zHistogramCollector.__init__cCrrc)r1rDrrrget_histogram_dict sz%HistogramCollector.get_histogram_dictcCsHtd|jdvr||S|jdkr |jr||S||Std)Nz/Collecting tensor data and making histogram ...>r,r'rDOnly 'entropy', 'percentile' or 'distribution' methods are supported)printr collect_valuercollect_absolute_valuer7r.rrrr!#s     zHistogramCollector.collectcCsr|D]1\}}t|trB|D]}t|tjs$Jdt|d|qdd|D}t|dks.r z6The calibration expects only one element type but got rr)r/zMonly float32 or float16 is supported, every constant must be explicitly typedrVrYrz'old_min should be a numpy array but is rN)r5r\r2rndarrayr:rbr"r7flattensizer r rrabsoluter1 histogramrr#float64r9ZarangeZhstackrr)r<r/rdata_arrZarrZdtypesZ data_arr_nprrr-r. old_histogramrrold_histold_hist_edgesZ temp_amaxwidthZ new_bin_edgesrrrr62s\ &            z)HistogramCollector.collect_absolute_valuec Cs|D]k\}}t|}|}|jdkr!t|}t|}ntjd|jd}tjd|jd}tjt t |t ||jd}||j vrW|j |}| ||||||j |<qtj ||j| |fd\}} || |||f|j |<qdS)z1 Collect histogram on real value rrrN)r5rr"r9r:r r rrrr r1merge_histogramr<r) r<r/rr>rr thresholdr?r-r.rrrr5ls,         z HistogramCollector.collect_valuecCs|\}}}} } || kr)tj|t|| | fd\} } | ||t||t| || fS| dkrAtj|t|| |fd\} }| |7} n9t|}d| |}t|| |d}|d|}||| }tj||| |fd\} }| ||||7<| |t||t| ||fS)NrCrrVr )rr<rbrrr)r<r?r>rrZ new_thresholdr@rArrZ old_thresholdZnew_histrr-r.Z old_num_binsZ old_strideZhalf_increased_binsZ new_num_binsrrrrDs2    z"HistogramCollector.merge_histogramcCsl|jr t|jdkrtdtd|jd|jdkr |S|jdkr)|S|jdkr2|Std) Nrz=Histogram has not been collected. Please run collect() first.z0Finding optimal threshold for each tensor using z algorithm ...r'rr,r3)r1rbr7r4rcompute_entropycompute_percentilecompute_distributionrDrrrr%s   z,HistogramCollector.compute_collection_resultcCs|jdks |jdkrtd|j}|j}i}tdt|td|jtdd|d|d |D]\}}|d}|d }|}t ||} |j rrt | |d} tj || |j d  tj || |j d f||<n*d|d } t | d | } t | | } tj || |j d tj || |j d f||<|d} |d}||d| kr| ||d f||<||d |kr||d|f||<g|||ddR||<tjdddvrt||q8|S)Nrdz     z'HistogramCollector.compute_distributioncs|d}|d}|j}|d}|d}|djt||d}fddt|jD} t||ddD]} || } t|| d|} || || f| | |<t|| | } | }t|d| }t|| d}|d|7<|d|7<|dk tj }tj|tj d}| j|}t|D]}||}||}t| ||||<q|dt| ||d7<tj|jtj d}t|D] }||}||}t|||}|dkr||||||<qt |}t |}|dus|durtj tj d}n tj t||d}||| |<q5t|}| |}|d}|d }|d|kr2||df}|d|kr?|d|f}t|dd sIJt|dd sSJ|S) aFGiven a dataset, find the optimal threshold for quantizing it. The reference distribution is `q`, and the candidate distribution is `p`. `q` is a truncated version of the original distribution. Ref: http://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf rr rVcs(g|]}tjddtjddfqS)rr)rrrrrrrIs(z.NrrrYr)r:rrZzerosrrrdeepcopyr%r#rr rrr'Zargminr9)r<r<rr-r.rZzero_bin_indexZnum_half_quantized_binZ kl_divergenceZ thresholdsrrrZsliced_distributionpZleft_outliers_countZright_outliers_countZnonzerosZquantized_binsZnum_merged_binsindexstartendqZnormdivZmin_kl_divergence_idxrXrrrrrrW;sf        z(HistogramCollector.get_entropy_thresholdN)r )rMrQrRr0r@r2r!r6r5rDr%rGrF staticmethodr\rHrWrrrrr s :  .  (r rFrrc CsXd}|tjkr3|dd}|dd} |dd} |dd} |dd} t|||||| | | | d }na|tjkrV|d d } |d d }|dd}t|||||| |d }n>|tjkry|d d } |dd}|dd}t|||||| |d}n|tjkr|d d } |dd}t ||||| |d}|r| |r||_ | |St d|)NrFrrrrr)rrrrrrrrr)rrrrrrrT)rrrrrr)rrrzUnsupported calibration method )r`rarTrrqr"rrr#rsr$rrrr7)rrrZcalibrate_methodrrZ extra_optionsZ calibratorrrrrrrrrrrrrcreate_calibrators|               re)Nr),rrrrRrcollections.abcrenumrpathlibrnumpyrrrrrrrZ quant_utilsr r r r8rfloatrr'r(rUr`ABCMetartrrrr"r#r$r-r rar]rerrrrsn    #4nb""#