o iY@stddlZddlZddlZddlmZddlZddlmZm Z m Z ddl m Z de de de de d ejf d d Zd e de de d ejfd dZde de fddZ d;de de de de dede d ejfddZdedeeejffddZde de de de dede de d e de de dede fd d!Z "dN)Path) ModelProto TensorProto numpy_helper) OnnxModel input_ids batch_sizesequence_lengthdictionary_sizereturncCsr|jjjtjtjtjfvsJtjj |||ftj d}|jjjtjkr*t |}|S|jjjtjkr7t |}|S)a`Create input tensor based on the graph input of input_ids Args: input_ids (TensorProto): graph input of the input_ids input tensor batch_size (int): batch size sequence_length (int): sequence length dictionary_size (int): vocabulary size of dictionary Returns: np.ndarray: the input tensor created )sizedtype) type tensor_type elem_typerFLOATINT32INT64nprandomrandintint32float32int64)rrr r datara/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/transformers/bert_test_data.pyfake_input_ids_datas   r segment_idscCsn|jjjtjtjtjfvsJtj||ftj d}|jjjtjkr(t |}|S|jjjtjkr5t |}|S)a,Create input tensor based on the graph input of segment_ids Args: segment_ids (TensorProto): graph input of the token_type_ids input tensor batch_size (int): batch size sequence_length (int): sequence length Returns: np.ndarray: the input tensor created r ) rrrrrrrrzerosrrr)rrr rrrrfake_segment_ids_data1s    r!max_sequence_lengthaverage_sequence_lengthcCsH|dkr||ks Jd||krtd|||Stdd|dS)N)rr)r"r#rrrget_random_lengthLs r&r% input_maskrandom_sequence_length mask_typec CsJ|jjjtjtjtjfvsJ|dkr8tj|tj d}|r,t |D] }t ||||<q!nt |D]}|||<q0n|dkr}tj ||ftj d}|rbt |D]}t ||}t |D]} d||| f<qWqLntj||ftj d} | |d| j dd| j df<n|dksJtj |ddtj d}|rt |D] }t ||||<qt |dD]6}|dkr|||d||dnd|||<|dkr|||d||dnd|d|d|<qn(t |D]}|||<qt |dD]}|||||<|||d|d|<q|jjjtjkrt|}|S|jjjtjkr#t|}|S)a"Create input tensor based on the graph input of segment_ids. Args: input_mask (TensorProto): graph input of the attention mask input tensor batch_size (int): batch size sequence_length (int): sequence length average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type - 1: mask index (sequence length excluding paddings). Shape is (batch_size). 2: 2D attention mask. Shape is (batch_size, sequence_length). 3: key len, cumulated lengths of query and key. Shape is (3 * batch_size + 2). Returns: np.ndarray: the input tensor created r$rr%Nr)rrrrrrrrZonesrranger&r shaperr) r'rr r#r(r)riZactual_seq_lenjtemprrrfake_input_mask_dataVsZ       "  0:    r0 directoryinputsc Cstj|s(zt|Wntytd|dYnwtd|dntd|dt|D]1\}\}}t ||}t tj |d|dd }| | Wd n1sbwYq6d S) zOutput input tensors of test data to a directory Args: directory (str): path of a directory inputs (Dict[str, np.ndarray]): map from input name to value zCreation of the directory z failedz#Successfully created the directory  zWarning: directory z$ existed. Files will be overwritten.Zinput_.pbwbN)ospathexistsmkdirOSErrorprint enumerateitemsr from_arrayopenjoinwriteSerializeToString)r1r2indexnamerZtensorfilerrroutput_test_datas   rF test_casesverbose random_seedc Cs|dusJtj|t|g} t|D]9} t||||}|j|i}|r0t|||||j<|r>t|||| | | ||j<|rKt| dkrKt d|| |q| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases dictionary_size (int): vocabulary size of dictionary for input_ids verbose (bool): print more information or not random_seed (int): random seed input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value NrzExample inputs) rrseedr+rrDr!r0lenr;append)rr rGr rHrIrrr'r#r(r) all_inputsZ _test_caseZinput_1r2rrrfake_test_datas" "        rN'rJc Cs6t|||| ||||||| | } t| |krtd| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): print more information or not input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value z$Failed to create test data for test.)rNrKr;) rr rGrJrHrrr'r#r(r)r rMrrrgenerate_test_datas"! rPcCs`|t|jkr dS|j|}||}|dur.|||}|dur.|jdkr.||jd}|S)NZCastr)rKinputfind_graph_inputZ get_parentZop_type) onnx_model embed_nodeZ input_indexrQZ graph_inputZ parent_noderrrget_graph_input_from_embed_node$s   rUrSinput_ids_namesegment_ids_nameinput_mask_namec Cs|}|dure||}|durtd|d}|r,||}|dur,td|d}|r@||}|dur@td|d|rEdnd|rKdnd}t||kr`td|dt||||fSt|dkrttdt||d } t| dkr| d} t|| d}t|| d}t|| d }|dur|D] } | j} d | vr| }q|durtd |||fSd}d}d}|D]} | j} d | vr| }qd | vsd| vr| }q| }q|r|r|r|||fStd)a Find graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_model (OnnxModel): onnx model object input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Raises: ValueError: Graph does not have input named of input_ids_name or segment_ids_name or input_mask_name ValueError: Expected graph input number does not match with specified input_ids_name, segment_ids_name and input_mask_name Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask Nz Graph does not have input named r$rzExpect the graph to have z inputs. Got r*z'Expect the graph to have 3 inputs. Got ZEmbedLayerNormalizationmaskz#Failed to find attention mask inputtokensegmentz?Fail to assign 3 inputs. You might try rename the graph inputs.)Z'get_graph_inputs_excluding_initializersrR ValueErrorrKZget_nodes_by_op_typerUrDlower) rSrVrWrXZ graph_inputsrrr'Zexpected_inputsZ embed_nodesrTrQZinput_name_lowerrrrfind_bert_inputs1sb                r_ onnx_filecCsTt}t|d}||Wdn1swYt|}t||||S)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_file (str): onnx model path input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask rbN)rr?ZParseFromStringreadrr_)r`rVrWrXmodelrErSrrrget_bert_inputss  rdcCsJt}|jddtdd|jddtddd|jd dtd d d|jd dtd dd|jddtddd|jddtddd|jddtddd|jddtd dd|jddtddd|jddddd|jdd|jdddd d|jdd!|jd"d#d$td%d&|jd'd(ddd)d|jdd*|jd+dtd,d-d|}|S).Nz--modelTzbert onnx model path.)requiredrhelpz --output_dirFz4output test data path. Default is current directory.)rerdefaultrfz --batch_sizer$zbatch size of inputz--sequence_lengthz maximum sequence length of inputz--input_ids_namezinput name for input idsz--segment_ids_namezinput name for segment idsz--input_mask_namezinput name for attention maskz --samplesz$number of test cases to be generatedz--seedr*z random seedz --verbose store_truezprint verbose information)reactionrf)rHz--only_input_tensorsz-only save input tensors and no output tensors)only_input_tensorsz-az--average_sequence_lengthz)average sequence length excluding padding)rgrrfz-rz--random_sequence_lengthz3use uniform random instead of fixed sequence length)r(z --mask_typer%z^mask type: (1: mask index, 2: raw 2D mask, 3: key lengths, cumulated lengths of query and key))argparseArgumentParser add_argumentstrint set_defaults parse_args)parserargsrrrparse_argumentss   rvrc output_dirrkc CsVt|||| \}}}t||||||||| | | }t|D]\}}tj|dt|}t||q| r5dSddl}d| vrCddgndg}|j ||d}dd| D}t|D]N\}}tj|dt|}| ||}t|D]4\}}t t|||}ttj|d |d d }||Wdn1swYqsqZdS) aICreate test data for a model, and save test data to a directory. Args: model (str): path of ONNX bert model output_dir (str): output directory batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): whether print more information input_ids_name (str): graph input name of input_ids segment_ids_name (str): graph input name of segment_ids input_mask_name (str): graph input name of input_mask only_input_tensors (bool): only save input tensors, average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type(int): mask type Ztest_data_set_NrZCUDAExecutionProviderZCPUExecutionProvider) providerscSsg|]}|jqSr)rD).0outputrrr Nsz-create_and_save_test_data..Zoutput_r4r5)rdrPr<r6r7r@rprF onnxruntimeZget_available_providersZInferenceSessionZ get_outputsrunrr>rZasarrayr?rArB)rcrwrr rGrJrHrVrWrXrkr#r(r)rrr'rMr-r2r1r|rxsessionZ output_namesresultZ output_nameZ tensor_resultrErrrcreate_and_save_test_data sJ"   rcCst}|jdkr |j|_|j}|dur(t|j}tj|j d|j d|j}|dur8t|}|j dddnt dt |j||j |j|j|j|j|j|j|j|j|j|j|jt d|dS)NrZbatch_Z_seq_T)parentsexist_okz7Directory existed. test data files will be overwritten.z Test data is saved to directory:)rvr#r rwrrcr6r7r@parentrr9r;rZsamplesrJrHrVrWrXrkr(r))rurwpr7rrrmainYs8   r__main__)r%)rO)NNN)!rmr6rpathlibrnumpyrZonnxrrrrSrrqZndarrayrr!r&boolr0rpdictrFrNrPrUtupler_rdrvrr__name__rrrrs.    I    F    4 ^ d      L '