o i3@sddlZddlZddlZddlZddlmZ dddZddZddZd d Z d d Z   dd dZ ddZ ddZ ddZedkr}eZededdlmZeejejsfejsaJdeeZnejZddlmZeeeZeD]ZeeqvdSdS)N) TensorProtocCst}|jdddtdd|jdddtdd|jd d dtd d d |jdddtddd |jddtd dd |jddtd dd |jddtddd |jddtddd |jddtddd |jddtdd d |jd!dtdd"d |jd#dtdd$d |jd%dd&gd'd(d)|jd*d+dd,d-d.|jdd/|jd0dtd1d2d |jd3dd,d4d.|jdd5|jd6dd,d7d.|jdd8|jd9d:dd,d;|jdd<||S)=Nz-iz--inputFz2Set the input file for reading the profile results)requiredtypehelpz-mz--modelzIonnx model path to run profiling. Required when --input is not specified.z-bz --batch_sizezbatch size of input)rrdefaultrz-sz--sequence_length zsequence length of inputz--past_sequence_lengthzpast sequence length for gpt2z--global_lengthz&number of global tokens for longformerz --samplesiz\number of samples to test. Set it large enough to reduce the variance of performance result.z --thresholdg{Gz?zfThreshold of run time ratio among all nodes. Nodes with larger ratio will show in top expensive nodes.z --thread_numznumber of threads to usez--input_ids_namez"input name for input IDs, for bertz--segment_ids_namez$input name for segment IDs, for bertz--input_mask_namez'input name for attention mask, for bertz--dummy_inputsr)bertgpt2 longformerrzEType of model inputs. The default will create dummy inputs with ones.)rrchoicesrz-gz --use_gpu store_truezuse GPU)ractionr)use_gpuz --providercudazExecution provider to usez--basic_optimizationz_Enable only basic graph optimizations. By default, all optimizations are enabled in OnnxRuntime)basic_optimizationz--kernel_time_onlyz.Only include the kernel time and no fence time)Zkernel_time_onlyz-vz --verbose)rr)verbose)argparseArgumentParser add_argumentstrintfloat set_defaults parse_args)argvparserr[/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/transformers/profiler.pyparse_argumentss       r c CsDddlm}||||| |dd}|D]}|d|} q|} | S)Nr)create_onnxruntime_sessionT)Zenable_all_optimization num_threadsZenable_profiling)benchmark_helperr!runZ end_profiling) Zonnx_model_pathrproviderr thread_num all_inputsr!sessionZinputs_ profile_filerrr run_profiles  r+cCs&t|dtkrt||dSdS)Nvalue)rZ WhichOneofrgetattr)dimrrrget_dim_from_type_protos&r/cCsdd|jjjDS)NcSsg|]}t|qSr)r/).0drrr sz-get_shape_from_type_proto..) tensor_typeshaper.)Z type_protorrrget_shape_from_type_protosr5c si|D]m}t|j}g}t|D]\}}t|tr!||qt|dkr+dSt|dkr7|||d<t|dkrC|||d<|jjj } | t j t j t j fvsTJ| t j kr\tjn | t j krdtjntj} tj|| d} | |j<qfddt|D} | S)a Create dummy inputs for ONNX model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples Returns: List[Dict]: list of inputs NrrZdtypecg|]}qSrrr0r) dummy_inputsrrr2z'create_dummy_inputs..)'get_graph_inputs_excluding_initializersr5r enumerate isinstancerappendlenr3 elem_typerFLOATINT32INT64numpyfloat32int64int32onesnamerange) onnx_model batch_sizesequence_lengthsamples graph_inputr4Z symbol_dimsir.rB data_typedatar'rr:rcreate_dummy_inputss0            rUc CsBddlm}m}|||||\} } } ||||dd| | | dd } | S)a-Create dummy inputs for BERT model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: List[Dict]: list of inputs r)find_bert_inputsgenerate_test_data{F)Z test_casesseedr input_ids segment_ids input_maskZrandom_mask_length)Zbert_test_datarVrW) rMrNrOrPinput_ids_namesegment_ids_nameinput_mask_namerVrWrZr[r\r'rrrcreate_bert_inputss r`cs|||||d}i|D]V}t|j}t|D]\}} t| tr4| |vr.td| || ||<q|jjj} | t j t j t j fvsFJ| t j krNt jn | t j krVt jnt j} t j|| d} | |j<qfddt|D} | S)aCreate dummy inputs for GPT-2 model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length past_sequence_length (int): past sequence length samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rNZseq_lenZ past_seq_lenZ total_seq_lensymbol is not supported: r7cr8rrr9r:rrr27r<z&create_gpt2_inputs..)r=r5rr>r?r RuntimeErrorr3rBrrCrDrErFrGrHrIrJrKrL)rMrNrOpast_sequence_lengthrPsymbolsrQr4rRr.rBrSrTr'rr:rcreate_gpt2_inputs s0       recs ||d}i|D]m}t|j}t|D]\}} t| tr0| |vr*td| || ||<q|jjj} | t j t j t j fvsBJ| t j krJt jn | t j krRt jnt j} d|jvrlt j|| d} d| ddd|f<nt j|| d} | |j<q fddt|D} | S) aCreate dummy inputs for Longformer model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length global_length (int): number of global tokens samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_longformer_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rNrOraglobalr7rNcr8rrr9r:rrr2fr<z,create_longformer_inputs..)r=r5rr>r?rrbr3rBrrCrDrErFrGrHrIrKZzerosrJrL)rMrNrO global_lengthrPrdrQr4rRr.rBrSrTr'rr:rcreate_longformer_inputs;s.         rhcCs|jdkr|jntjdd}dtjvrt|tjd<ddlm}ddlm }|||j }d}|j dkrEt ||j |j|j|j|j|j}n.|j dkrWt||j |j|j|j}n|j d krit||j |j|j|j}n t||j |j|j}t|j |j|j|j|j|}|S) NrF)ZlogicalZOMP_NUM_THREADS)load) OnnxModelr r r )r&psutil cpu_countosenvironronnxrirMrjmodelr;r`rNrOrPr]r^r_rercrhrgrUr+rr%r)argsr"rirjrMr'r*rrrr$jsV       r$__main__ Arguments) setup_loggerzMrequires either --model to run profiling or --input to read profiling results)process_results)N)NNN)rrmrFrkrorr r+r/r5rUr`rerhr$__name__ argumentsprintr#rtrinputrpr*Zprofile_result_processorruresultslinerrrrsD   / )0/6