o iz^@sddlZddlZddlZddlZddlZddlZddlmZmZm Z m Z m Z m Z m Z ddlmZmZddlmZeeZddZdd d Zdd d Zd dZgdZdgddgddgdZGdddZ         dddZdddZdS)N)AttributeProto GraphProto ModelProto NodeProto TensorProtohelper numpy_helper) infer_shapesinfer_shapes_path)versioncCsdd|DS)z| Convert numpy float16 to python int. :param np_list: numpy float16 list :return int_list: python int list cSs.g|]}tt|dddddqS)HN)intbinviewzfill).0_rZ/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/transformers/float16.py $s.z%_npfloat16_to_int..r)Znp_listrrr_npfloat16_to_intsr"\o>@cCsdd}|t|dkjddkrF|t|dk}|t|dk}||kr7td|d|||krFtd|d||t|dkjddkr|t|dk}|t|dk}|| kr{td|d| || krtd|d| t|d||||}t|| |d| |}t|||td||}t|td|| | |}t|S)a? Convert float32 numpy array to float16 without changing sign or finiteness. Positive values less than min_positive_val are mapped to min_positive_val. Positive finite values greater than max_finite_val are mapped to max_finite_val. Similar for negative values. NaN, 0, inf, and -inf are unchanged. cSst||k||kSN)np logical_and)abcrrrbetween/sz&convert_np_to_float16..betweenrzthe float32 number z will be truncated to infz-inf) rwhereshapemaxminloggerdebugfloatfloat16)Znp_arraymin_positive_valmax_finite_valr!Z positive_maxZ positive_minZ negative_maxZ negative_minrrrconvert_np_to_float16's(   r-cCst|tstdt||jtjkrMtj|_|jr7tt |j||}t |}||j dd<g|jdd<|j rMt j|j dd}t|||}||_ |S)aConvert tensor float to float16. Args: tensor (TensorProto): the tensor to convert. min_positive_val (float, optional): minimal positive value. Defaults to 1e-7. max_finite_val (float, optional): maximal finite value. Defaults to 1e4. Raises: ValueError: input type is not TensorProto. Returns: TensorProto: the converted tensor. 3Expected input type is an ONNX TensorProto but got Nfloat32dtype) isinstancer ValueErrortype data_typeFLOATFLOAT16 float_datar-rarrayrZ int32_dataraw_data frombuffertobytes)tensorr+r, float16_dataZint_listZ float32_listZ float16_listrrrconvert_tensor_float_to_float16Is    r?cCst|j}t|j|j|Sr)rZto_arrayr$rZmake_tensor_value_infonamer5)r=r$rrrmake_value_info_from_tensorns rA)ZArrayFeatureExtractorZ BinarizerZCastMapZCategoryMapperZDictVectorizerZFeatureVectorizerZImputerZ LabelEncoderZLinearClassifierZLinearRegressorZ NormalizerZ OneHotEncoderRandomUniformLikeZ SVMClassifierZ SVMRegressorZScalerZTreeEnsembleClassifierZTreeEnsembleRegressorZ TreeEnsembleZZipMapZNonMaxSuppressionZTopKZRoiAlignRangeZCumSumZMinZMaxZUpsampler )ZResizeZ GroupNormZ SkipGroupNormc@s,eZdZdZdefddZdefddZdS) InitializerTrackerz'Class for keeping track of initializer. initializercCs||_g|_g|_dSr)rF fp32_nodes fp16_nodes)selfrFrrr__init__s zInitializerTracker.__init__nodecCs$|r |j|dS|j|dSr)rGappendrH)rIrKis_node_blockedrrradd_nodeszInitializerTracker.add_nodeN)__name__ __module__ __qualname____doc__rrJrrNrrrrrEsrEFc +s |dksJd|tttjjksJd|durin|} t|trb|} tt j tdkr]|s]t j t j| d} | j} t| | t | }d}Wdn1sWwYnt | }t|tsptdt|d}|stt j td krzt}Wnw|durt}|durg}t|}t|}td |d |d d |d|d|d|g}g}g}g}|dur||}||i}t}t}dd|jjD}dd|jjD}ttrfdd|D}fdd|D}nsg}g}t |jjD]V\}}|j|vr\dt|}|||j<|!|jdt|}|jj"!}|#|||_t$j%|jj&_'t(j)d|jg|gt$j%|dg}|jj*+||||!|qt |jjD]U\}}|j|vrdt|}|||j<|!|jdt|}|jj"!}|#|||_t$j%|jj&_'t(j)d|g|jgd|dg}|jj*+||||!|qdi}|rg} |D]}!t|!tr| |!jt|!t,r|!j-D]}|j.t$j/kr|j|vsJt0|||j<q|!j*D]}|j|vrqt1t2|jD]}|j||vr||j||j|<q t1t2|jD]}|j||vr:||j||j|<q&|j3|vpF|j|v}"t |jD](\}}||vrs|"pj|t45|j3gvoj|| 5|j3gv}#||6||#qL|"r||q|j3dkr|j7D]}$|$jdkr|$j8t$j/krt$j%|$_8nq|j3dvrd}%|j7D]}$|$jd krd}%|$j8t$j/krt$j%|$_8q|j3d!vr|%s|j7+t(9d t$j%g|j3t4vs|j3| vr|j7D]}$| |$qq||qt|!t:r'| |!j;|!j|!j=|||!j?D] }t>|||}qt|!t,rzt@A|!j|!j|!j"D]B}|jj&j't$j/krS|j|vrSt$j%|jj&_'|||jBd"rx|jjCj'j&j't$j/krx|j|vrxt$j%|jjCj'j&_'||q7q| }|s|DD]*}&|s|&jErt>|&j-|||&_-|tF|&j-|&jGr|stHd#|&jEq|D]o}'t |'jD]f\}}|t4|'j3vs|| 5|'j3gvrѐq|D]J}(||(jkr|jj"!}|#|(|'jd$t|}||_t$j/|jj&_'|'jd%t|}t(j)d|g|gd|dg}|jj*+|||'j|<nqӐqq| r(t$jInt$j/})|D]}'t1t2|'jD]T}|'j|}|D]I}(||(jkr|jj"!}|#|(|'jd$t|}||_|)|jj&_'|'jd%t|}t(j)d|g|g|)|dg}|jj*+|||'j|<nq?q6t1t2|'jD]T}|'j|}*|D]I}(|*|(jkr|jj"!}|#|(|'jd&t|}||_|)|jj&_'|'jd't|}t(j)d|g|*gd(|dg}|jj*+|||'j|<nqqq-|S))aConvert tensor float type in the input ONNX model to tensor float16. Args: model (ModelProto or str): The ONNX model or path of the model to convert. min_positive_val (float, optional): minimal positive value. Defaults to 5.96e-08. max_finite_val (float, optional): maximal finite value of float16. Defaults to 65504. keep_io_types (Union[bool, List[str]], optional): It could be boolean or a list of float32 input/output names. If True, model inputs/outputs should be left as float32. Defaults to False. disable_shape_infer (bool, optional): Skips running onnx shape/type inference. Useful if shape inference has been done. Defaults to False. op_block_list (List[str], optional): List of op types to leave as float32. Defaults to None, which will use `float16.DEFAULT_OP_BLOCK_LIST`. node_block_list (List[str], optional): List of node names to leave as float32. Defaults to None. force_fp16_initializers(bool): force converting all float initializers to float16. Default to false, which will convert only the one needed to avoid precision loss. force_fp16_inputs(Dict[str, List[int]]): Force the conversion of the inputs of some operators to float16, even if this script's preference it to keep them in float32. Raises: ValueError: input type is not ModelProto. Returns: ModelProto: converted model. rzginvalid min_positive_val. smallest positive float16 value: subnormal 5.96e-08, and normalized 6.104e-05z4invalid max_finite_val. largest float16 value: 65504Nz1.8.0)dirTz$Expected an ONNX ModelProto but got z1.2.0z"fp16 parameters: min_positive_val=z max_finite_val=z keep_io_types=z disable_shape_infer=z op_block_list=z node_block_list=z force_fp16_initializers=cS"g|] }|jjjtjkr|jqSrr4 tensor_type elem_typerr6r@rnrrrr"z,convert_float_to_float16..cSrTrrUrXrrrrrZcg|]}|vr|qSrrrX keep_io_typesrrr cr[rrrXr\rrr r^Zgraph_input_cast_Zgraph_input_castZCast)tor@Zgraph_output_cast_Zgraph_output_castrDr_)ZEyeLikeZ Multinomial RandomNormalZRandomNormalLike RandomUniformrB SequenceEmptyZ BernoulliFr1)r`rarb sequence_typezXinitializer is used by both fp32 and fp16 nodes. Consider add these nodes to block list:Z _input_cast_Z _input_castZ _output_cast_Z _output_cast )Jr)rZfinfor*r%r2strr parseonnx __version__tempfileNamedTemporaryFileospathdirnamer@r loadrr3r4r DEFAULT_OP_BLOCK_LISTsetr'r(rLgraphinputoutputlist enumerateadd value_infoZCopyFromrr7rVrWrZ make_noderKextendrrFr5r6rErangelenZop_typeALWAYS_FLOAT_INPUTSgetrN attributeiZmake_attributergZgraphstr?Ztensors itertoolschainZHasFieldrcvaluesrHrArGinfoZBFLOAT16)+modelr+r,r]Zdisable_shape_inferZ op_block_listZnode_block_listZforce_fp16_initializersZforce_fp16_inputsZ#use_bfloat16_as_blocked_nodes_dtypeZforce_fp16_inputs_dictZ model_pathZtmpfileZshape_infer_model_pathZfunc_infer_shapequeueZvalue_info_listZ node_listZmixed_float_type_node_listZ name_mappingZgraph_io_to_skipZio_castsZ fp32_inputsZ fp32_outputsr~rYZ output_nameZ node_nameZnew_value_infonew_nodeZ input_nameZfp32_initializersZ next_levelqrMZuse_fp32_weightattrZ has_dtypevaluerKrwZ accuracy_typersrr\rconvert_float_to_float16s $     ,                                               d $                 rcCst|tstdt||jtjkrtdd}|jr#t|j}|j r.tj |j dd}|dur6t dt |||}t t|t|S)zSMeasure the maximum absolute difference after converting a float tensor to float16.r.z#Expected tensor data type is float.Nr/r0zexternal data not loaded!)r2rr3r4r5r6r8rr9r:r; RuntimeErrorr-Zamaxabsr/)r=r+r,Z float32_datar>rrrfloat_to_float16_max_diffs    r)rr) rrFFNNFNF)rloggingrkrinumpyrrgrrrrrrrZonnx.shape_inferencer r packagingr getLoggerrOr'rr-r?rAror{rErrrrrrs<$   "%" >