o i"@sddlZddlZddlZddlmZddlZddlZddlZddl m Z ddlm Z ddl m Z ddlmZddlmZmZddlmZeeZGd d d ejjZdS) N)Path)convert_float_to_float16) ModelProto) OnnxModel) WhisperConfig)get_model_dynamic_axesget_sample_encoder_inputs)InferenceSessionc seZdZdZdedejjdeffdd Z dej fdd Z d d Z d d Z ddZdedefddZ   ddededededef ddZdededefddZZS)WhisperEncoderzWhisper encoder componentconfigmodel model_implcs@t||_|j|_||_|dkr|j|_dS|jj|_dSNZopenai)super__init__r devicer encoderr )selfr r r  __class__q/home/kim/smarthome/.venv/lib/python3.10/site-packages/onnxruntime/transformers/models/whisper/whisper_encoder.pyrs "zWhisperEncoder.__init__audio_featurescCs||}|jdkr |S|jSr)rr Zlast_hidden_state)rrZoutputsrrrforward%s zWhisperEncoder.forwardcC dg}|S)Nrr)r input_namesrrrr)zWhisperEncoder.input_namescCr)NZencoder_hidden_statesr)r output_namesrrrr-rzWhisperEncoder.output_namescCst|j||}|S)N)rr )rrr dynamic_axesrrrr1szWhisperEncoder.dynamic_axesuse_fp16_inputscCs|jdkr |r t|}|Sr)r r)rr rrrrfix_layernorm_weights5sz$WhisperEncoder.fix_layernorm_weightsTFonnx_model_pathproviderverboseuse_external_data_formatc Cst|j|jd|d}|}|}|||} t|jjdddt G} t j | d} t| jjddd|r<| n|} tjj||d| d||| dd|d tj| |d } || |} tj| ||dd Wd n1spwY||||d S) aExport encoder to ONNX Args: onnx_model_path (str): path to save ONNX model provider (str): provider to use for verifying parity on ONNX model verbose (bool, optional): print verbose information. Defaults to True. use_external_data_format (bool, optional): use external data format or not. Defaults to False. use_fp16_inputs (bool, optional): use float16 inputs for the audio_features. Defaults to False. Z batch_sizeZuse_fp16T)parentsexist_okz encoder.onnxr) argsfZ export_paramsrrrZ opset_versionZdo_constant_foldingr#)Zload_external_data)Zsave_as_external_dataZall_tensors_to_one_fileN)rr rrrrrparentmkdirtempfileTemporaryDirectoryospathjointorchonnxZexportZ load_modelr rsave verify_onnx)rr!r"r#r$rinputsrrrZ tmp_dir_nameZtemp_onnx_model_pathZout_pathr rrr export_onnxAsH    zWhisperEncoder.export_onnxc Cst|j|jd|d}||d}t||gd}|dd|did}t ||}t dt dt |dS) a"Verify ONNX model outputs and PyTorch model outputs match Args: onnx_model_path (str): path to save ONNX model provider (str): execution provider for ONNX model use_fp16_inputs (bool, optional): use float16 inputs for the audio_features r%r&r) providersNrz"Comparing encoder_hidden_states...z Max diff: )rr rrdetachcpunumpyr runnpabsloggerwarningmax) rr!r"rr7Z pt_outputssessZ ort_outputsdiffrrrr6s$ zWhisperEncoder.verify_onnx)TFF)__name__ __module__ __qualname____doc__rr3nnModulestrrZTensorrrrrrboolr r8r6 __classcell__rrrrr s: @r )loggingr0r.pathlibrr<r>r4r3Zfloat16rrZ onnx_modelrZ transformersrZwhisper_inputsrrZ onnxruntimer getLoggerrEr@rIrJr rrrrs