diff --git "a/trtexec_log.txt" "b/trtexec_log.txt" new file mode 100644--- /dev/null +++ "b/trtexec_log.txt" @@ -0,0 +1,11334 @@ +&&&& RUNNING TensorRT.trtexec [TensorRT v8403] # /home/fxmarty/Downloads/TensorRT-8.4.3.1/targets/x86_64-linux-gnu/bin/trtexec --onnx=decoder_model_sim.onnx --shapes=input_ids:1x10,attention_mask:1x10 --verbose --dumpProfile --profilingVerbosity=detailed +[12/30/2022-19:54:16] [I] === Model Options === +[12/30/2022-19:54:16] [I] Format: ONNX +[12/30/2022-19:54:16] [I] Model: decoder_model_sim.onnx +[12/30/2022-19:54:16] [I] Output: +[12/30/2022-19:54:16] [I] === Build Options === +[12/30/2022-19:54:16] [I] Max batch: explicit batch +[12/30/2022-19:54:16] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default +[12/30/2022-19:54:16] [I] minTiming: 1 +[12/30/2022-19:54:16] [I] avgTiming: 8 +[12/30/2022-19:54:16] [I] Precision: FP32 +[12/30/2022-19:54:16] [I] LayerPrecisions: +[12/30/2022-19:54:16] [I] Calibration: +[12/30/2022-19:54:16] [I] Refit: Disabled +[12/30/2022-19:54:16] [I] Sparsity: Disabled +[12/30/2022-19:54:16] [I] Safe mode: Disabled +[12/30/2022-19:54:16] [I] DirectIO mode: Disabled +[12/30/2022-19:54:16] [I] Restricted mode: Disabled +[12/30/2022-19:54:16] [I] Build only: Disabled +[12/30/2022-19:54:16] [I] Save engine: +[12/30/2022-19:54:16] [I] Load engine: +[12/30/2022-19:54:16] [I] Profiling verbosity: 2 +[12/30/2022-19:54:16] [I] Tactic sources: Using default tactic sources +[12/30/2022-19:54:16] [I] timingCacheMode: local +[12/30/2022-19:54:16] [I] timingCacheFile: +[12/30/2022-19:54:16] [I] Input(s)s format: fp32:CHW +[12/30/2022-19:54:16] [I] Output(s)s format: fp32:CHW +[12/30/2022-19:54:16] [I] Input build shape: input_ids=1x10+1x10+1x10 +[12/30/2022-19:54:16] [I] Input build shape: attention_mask=1x10+1x10+1x10 +[12/30/2022-19:54:16] [I] Input calibration shapes: model +[12/30/2022-19:54:16] [I] === System Options === +[12/30/2022-19:54:16] [I] Device: 0 +[12/30/2022-19:54:16] [I] DLACore: +[12/30/2022-19:54:16] [I] Plugins: +[12/30/2022-19:54:16] [I] === Inference Options === +[12/30/2022-19:54:16] [I] Batch: Explicit +[12/30/2022-19:54:16] [I] Input inference shape: attention_mask=1x10 +[12/30/2022-19:54:16] [I] Input inference shape: input_ids=1x10 +[12/30/2022-19:54:16] [I] Iterations: 10 +[12/30/2022-19:54:16] [I] Duration: 3s (+ 200ms warm up) +[12/30/2022-19:54:16] [I] Sleep time: 0ms +[12/30/2022-19:54:16] [I] Idle time: 0ms +[12/30/2022-19:54:16] [I] Streams: 1 +[12/30/2022-19:54:16] [I] ExposeDMA: Disabled +[12/30/2022-19:54:16] [I] Data transfers: Enabled +[12/30/2022-19:54:16] [I] Spin-wait: Disabled +[12/30/2022-19:54:16] [I] Multithreading: Disabled +[12/30/2022-19:54:16] [I] CUDA Graph: Disabled +[12/30/2022-19:54:16] [I] Separate profiling: Disabled +[12/30/2022-19:54:16] [I] Time Deserialize: Disabled +[12/30/2022-19:54:16] [I] Time Refit: Disabled +[12/30/2022-19:54:16] [I] Inputs: +[12/30/2022-19:54:16] [I] === Reporting Options === +[12/30/2022-19:54:16] [I] Verbose: Enabled +[12/30/2022-19:54:16] [I] Averages: 10 inferences +[12/30/2022-19:54:16] [I] Percentile: 99 +[12/30/2022-19:54:16] [I] Dump refittable layers:Disabled +[12/30/2022-19:54:16] [I] Dump output: Disabled +[12/30/2022-19:54:16] [I] Profile: Enabled +[12/30/2022-19:54:16] [I] Export timing to JSON file: +[12/30/2022-19:54:16] [I] Export output to JSON file: +[12/30/2022-19:54:16] [I] Export profile to JSON file: +[12/30/2022-19:54:16] [I] +[12/30/2022-19:54:16] [I] === Device Information === +[12/30/2022-19:54:16] [I] Selected Device: NVIDIA GeForce RTX 3060 Laptop GPU +[12/30/2022-19:54:16] [I] Compute Capability: 8.6 +[12/30/2022-19:54:16] [I] SMs: 30 +[12/30/2022-19:54:16] [I] Compute Clock Rate: 1.282 GHz +[12/30/2022-19:54:16] [I] Device Global Memory: 5946 MiB +[12/30/2022-19:54:16] [I] Shared Memory per SM: 100 KiB +[12/30/2022-19:54:16] [I] Memory Bus Width: 192 bits (ECC disabled) +[12/30/2022-19:54:16] [I] Memory Clock Rate: 6.001 GHz +[12/30/2022-19:54:16] [I] +[12/30/2022-19:54:16] [I] TensorRT version: 8.4.3 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::GridAnchor_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::GridAnchorRect_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::NMS_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::Reorg_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::Region_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::Clip_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::LReLU_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::PriorBox_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::Normalize_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::ScatterND version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::RPROI_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::BatchedNMS_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::BatchedNMSDynamic_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::BatchTilePlugin_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::FlattenConcat_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::CropAndResize version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::CropAndResizeDynamic version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::DetectionLayer_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::EfficientNMS_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::ProposalDynamic version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::Proposal version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::ProposalLayer_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::PyramidROIAlign_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::ResizeNearest_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::Split version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::SpecialSlice_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::InstanceNormalization_TRT version 2 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::CoordConvAC version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::DecodeBbox3DPlugin version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::GenerateDetection_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::MultilevelCropAndResize_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::MultilevelProposeROI_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::NMSDynamic_TRT version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::PillarScatterPlugin version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::VoxelGeneratorPlugin version 1 +[12/30/2022-19:54:16] [V] [TRT] Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1 +[12/30/2022-19:54:17] [I] [TRT] [MemUsageChange] Init CUDA: CPU +328, GPU +0, now: CPU 336, GPU 259 (MiB) +[12/30/2022-19:54:17] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +327, GPU +104, now: CPU 682, GPU 363 (MiB) +[12/30/2022-19:54:17] [I] Start parsing network model +[12/30/2022-19:54:17] [I] [TRT] ---------------------------------------------------------------- +[12/30/2022-19:54:17] [I] [TRT] Input filename: decoder_model_sim.onnx +[12/30/2022-19:54:17] [I] [TRT] ONNX IR version: 0.0.7 +[12/30/2022-19:54:17] [I] [TRT] Opset version: 13 +[12/30/2022-19:54:17] [I] [TRT] Producer name: pytorch +[12/30/2022-19:54:17] [I] [TRT] Producer version: 1.13.1 +[12/30/2022-19:54:17] [I] [TRT] Domain: +[12/30/2022-19:54:17] [I] [TRT] Model version: 0 +[12/30/2022-19:54:17] [I] [TRT] Doc string: +[12/30/2022-19:54:17] [I] [TRT] ---------------------------------------------------------------- +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::GridAnchor_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::GridAnchorRect_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::NMS_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::Reorg_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::Region_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::Clip_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::LReLU_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::PriorBox_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::Normalize_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::ScatterND version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::RPROI_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::BatchedNMS_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::BatchedNMSDynamic_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::BatchTilePlugin_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::FlattenConcat_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::CropAndResize version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::CropAndResizeDynamic version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::DetectionLayer_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::EfficientNMS_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::EfficientNMS_ONNX_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Explicit_TF_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::EfficientNMS_Implicit_TF_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::ProposalDynamic version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::Proposal version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::ProposalLayer_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::PyramidROIAlign_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::ResizeNearest_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::Split version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::SpecialSlice_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::InstanceNormalization_TRT version 2 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::CoordConvAC version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::DecodeBbox3DPlugin version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::GenerateDetection_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::MultilevelCropAndResize_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::MultilevelProposeROI_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::NMSDynamic_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::PillarScatterPlugin version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::VoxelGeneratorPlugin version 1 +[12/30/2022-19:54:18] [V] [TRT] Plugin creator already registered - ::MultiscaleDeformableAttnPlugin_TRT version 1 +[12/30/2022-19:54:18] [V] [TRT] Adding network input: input_ids with dtype: int32, dimensions: (-1, -1) +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: input_ids for ONNX tensor: input_ids +[12/30/2022-19:54:18] [V] [TRT] Adding network input: attention_mask with dtype: int32, dimensions: (-1, -1) +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: attention_mask for ONNX tensor: attention_mask +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: transformer.wte.weight +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: transformer.h.0.ln_1.weight +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: transformer.h.0.attn.bias +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: lm_head.bias +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2824 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2825 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2826 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2851 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2852 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2853 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2854 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2855 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2856 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2881 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2882 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2883 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2884 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2885 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2886 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2911 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2912 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2913 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2914 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2915 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2916 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2941 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2942 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2943 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2944 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2945 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2946 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2971 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2972 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2973 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2974 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2975 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_2976 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3001 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3002 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3003 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3004 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3005 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3006 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3031 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3032 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3033 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3034 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3035 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3036 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3061 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3062 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3063 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::MatMul_3064 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Weight at index 0: 9223372036854775807 is out of range. Clamping to: 2147483647 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Importing initializer: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Shape [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: input_ids +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape [Shape] inputs: [input_ids -> (-1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Shape for ONNX node: /transformer/Shape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Shape_output_0 for ONNX tensor: /transformer/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape [Shape] outputs: [/transformer/Shape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Gather [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Gather [Gather] inputs: [/transformer/Shape_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Constant_output_0 for ONNX node: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Gather for ONNX node: /transformer/Gather +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Gather_output_0 for ONNX tensor: /transformer/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Gather [Gather] outputs: [/transformer/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Shape_1 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: input_ids +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape_1 [Shape] inputs: [input_ids -> (-1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Shape_1 for ONNX node: /transformer/Shape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Shape_1_output_0 for ONNX tensor: /transformer/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape_1 [Shape] outputs: [/transformer/Shape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Gather_1 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Gather_1 [Gather] inputs: [/transformer/Shape_1_output_0 -> (2)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Constant_1_output_0 for ONNX node: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Gather_1 for ONNX node: /transformer/Gather_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Gather_1_output_0 for ONNX tensor: /transformer/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Gather_1 [Gather] outputs: [/transformer/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze [Unsqueeze] inputs: [/transformer/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Unsqueeze for ONNX node: /transformer/Unsqueeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Unsqueeze_output_0 for ONNX tensor: /transformer/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze [Unsqueeze] outputs: [/transformer/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Concat [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Concat [Concat] inputs: [/transformer/Constant_2_output_0 -> (1)[INT32]], [/transformer/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Constant_2_output_0 for ONNX node: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Concat for ONNX node: /transformer/Concat +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Concat_output_0 for ONNX tensor: /transformer/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Concat [Concat] outputs: [/transformer/Concat_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Reshape [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: input_ids +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Reshape [Reshape] inputs: [input_ids -> (-1, -1)[INT32]], [/transformer/Concat_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Reshape for ONNX node: /transformer/Reshape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Reshape_output_0 for ONNX tensor: /transformer/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Reshape [Reshape] outputs: [/transformer/Reshape_output_0 -> (-1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Shape_2 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape_2 [Shape] inputs: [/transformer/Reshape_output_0 -> (-1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Shape_2 for ONNX node: /transformer/Shape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Shape_2_output_0 for ONNX tensor: /transformer/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape_2 [Shape] outputs: [/transformer/Shape_2_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Gather_2 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Gather_2 [Gather] inputs: [/transformer/Shape_2_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Gather_2 for ONNX node: /transformer/Gather_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Gather_2_output_0 for ONNX tensor: /transformer/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Gather_2 [Gather] outputs: [/transformer/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Unsqueeze_1 for ONNX node: /transformer/Unsqueeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Unsqueeze_1_output_0 for ONNX tensor: /transformer/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Concat_1 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Concat_1 [Concat] inputs: [/transformer/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Concat_1 for ONNX node: /transformer/Concat_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Concat_1_output_0 for ONNX tensor: /transformer/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Concat_1 [Concat] outputs: [/transformer/Concat_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Reshape_1 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: attention_mask +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Reshape_1 [Reshape] inputs: [attention_mask -> (-1, -1)[INT32]], [/transformer/Concat_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Reshape_1 for ONNX node: /transformer/Reshape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Reshape_1_output_0 for ONNX tensor: /transformer/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Reshape_1 [Reshape] outputs: [/transformer/Reshape_1_output_0 -> (-1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/Reshape_1_output_0 -> (-1, -1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Unsqueeze_2 for ONNX node: /transformer/Unsqueeze_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Unsqueeze_2_output_0 for ONNX tensor: /transformer/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/Unsqueeze_2_output_0 -> (-1, 1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/Unsqueeze_2_output_0 -> (-1, 1, -1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, 1, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Unsqueeze_3 for ONNX node: /transformer/Unsqueeze_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Unsqueeze_3_output_0 for ONNX tensor: /transformer/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/Unsqueeze_3_output_0 -> (-1, 1, 1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Cast [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Cast [Cast] inputs: [/transformer/Unsqueeze_3_output_0 -> (-1, 1, 1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Cast for ONNX node: /transformer/Cast +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Cast_output_0 for ONNX tensor: /transformer/Cast_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Cast [Cast] outputs: [/transformer/Cast_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Cast_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Sub [Sub] inputs: [/transformer/Constant_7_output_0 -> ()[FLOAT]], [/transformer/Cast_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Constant_7_output_0 for ONNX node: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Sub for ONNX node: /transformer/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Sub_output_0 for ONNX tensor: /transformer/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Sub [Sub] outputs: [/transformer/Sub_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Mul [Mul] inputs: [/transformer/Sub_output_0 -> (-1, 1, 1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Constant_8_output_0 for ONNX node: /transformer/Constant_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Mul for ONNX node: /transformer/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Mul_output_0 for ONNX tensor: /transformer/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Mul [Mul] outputs: [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/wte/Gather [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.wte.weight +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/wte/Gather [Gather] inputs: [transformer.wte.weight -> (50400, 512)[FLOAT]], [/transformer/Reshape_output_0 -> (-1, -1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: transformer.wte.weight for ONNX node: transformer.wte.weight +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/wte/Gather for ONNX node: /transformer/wte/Gather +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/wte/Gather_output_0 for ONNX tensor: /transformer/wte/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/wte/Gather [Gather] outputs: [/transformer/wte/Gather_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Shape_3 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/wte/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape_3 [Shape] inputs: [/transformer/wte/Gather_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Shape_3 for ONNX node: /transformer/Shape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Shape_3_output_0 for ONNX tensor: /transformer/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Shape_3 [Shape] outputs: [/transformer/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Slice [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Slice [Slice] inputs: [/transformer/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Slice for ONNX node: /transformer/Slice +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Slice_output_0 for ONNX tensor: /transformer/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Slice [Slice] outputs: [/transformer/Slice_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/Squeeze [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Squeeze [Squeeze] inputs: [/transformer/Slice_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Squeeze for ONNX node: /transformer/Squeeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/Squeeze_output_0 for ONNX tensor: /transformer/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/Squeeze [Squeeze] outputs: [/transformer/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/wte/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/wte/Gather_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/ReduceMean for ONNX node: /transformer/h.0/ln_1/ReduceMean +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.0/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.0/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/wte/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Sub [Sub] inputs: [/transformer/wte/Gather_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Sub for ONNX node: /transformer/h.0/ln_1/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.0/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Sub [Sub] outputs: [/transformer/h.0/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Pow [Pow] inputs: [/transformer/h.0/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Constant_output_0 for ONNX node: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Pow for ONNX node: /transformer/h.0/ln_1/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.0/ln_1/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Pow [Pow] outputs: [/transformer/h.0/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.0/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/ReduceMean_1 for ONNX node: /transformer/h.0/ln_1/ReduceMean_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.0/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.0/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Add [Add] inputs: [/transformer/h.0/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Constant_1_output_0 for ONNX node: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Add for ONNX node: /transformer/h.0/ln_1/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Add_output_0 for ONNX tensor: /transformer/h.0/ln_1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Add [Add] outputs: [/transformer/h.0/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.0/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Sqrt for ONNX node: /transformer/h.0/ln_1/Sqrt +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.0/ln_1/Sqrt_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.0/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Div [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Sqrt_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Div [Div] inputs: [/transformer/h.0/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Div for ONNX node: /transformer/h.0/ln_1/Div +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Div_output_0 for ONNX tensor: /transformer/h.0/ln_1/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Div [Div] outputs: [/transformer/h.0/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Mul [Mul] inputs: [/transformer/h.0/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: transformer.h.0.ln_1.weight for ONNX node: transformer.h.0.ln_1.weight +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Mul for ONNX node: /transformer/h.0/ln_1/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.0/ln_1/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Mul [Mul] outputs: [/transformer/h.0/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/ln_1/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Add_1 [Add] inputs: [/transformer/h.0/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: transformer.h.0.ln_1.bias for ONNX node: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/ln_1/Add_1 for ONNX node: /transformer/h.0/ln_1/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.0/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/ln_1/Add_1 [Add] outputs: [/transformer/h.0/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2824 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.0/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2824 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2824 for ONNX node: onnx::MatMul_2824 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/q_proj/MatMul for ONNX node: /transformer/h.0/attn/q_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.0/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.0/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2825 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.0/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2825 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2825 for ONNX node: onnx::MatMul_2825 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/k_proj/MatMul for ONNX node: /transformer/h.0/attn/k_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.0/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.0/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2826 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.0/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2826 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2826 for ONNX node: onnx::MatMul_2826 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/v_proj/MatMul for ONNX node: /transformer/h.0/attn/v_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.0/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.0/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape [Shape] inputs: [/transformer/h.0/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape for ONNX node: /transformer/h.0/attn/Shape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape [Shape] outputs: [/transformer/h.0/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather [Gather] inputs: [/transformer/h.0/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather for ONNX node: /transformer/h.0/attn/Gather +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather [Gather] outputs: [/transformer/h.0/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_1 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_1 [Shape] inputs: [/transformer/h.0/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_1 for ONNX node: /transformer/h.0/attn/Shape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_1 [Shape] outputs: [/transformer/h.0/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_1 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_1 [Gather] inputs: [/transformer/h.0/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_1 for ONNX node: /transformer/h.0/attn/Gather_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_1 [Gather] outputs: [/transformer/h.0/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze for ONNX node: /transformer/h.0/attn/Unsqueeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_1 for ONNX node: /transformer/h.0/attn/Unsqueeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_2_output_0 for ONNX node: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_3_output_0 for ONNX node: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat for ONNX node: /transformer/h.0/attn/Concat +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat [Concat] outputs: [/transformer/h.0/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape [Reshape] inputs: [/transformer/h.0/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape for ONNX node: /transformer/h.0/attn/Reshape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape [Reshape] outputs: [/transformer/h.0/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_2 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_2 [Shape] inputs: [/transformer/h.0/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_2 for ONNX node: /transformer/h.0/attn/Shape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_2 [Shape] outputs: [/transformer/h.0/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_2 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_2 [Gather] inputs: [/transformer/h.0/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_2 for ONNX node: /transformer/h.0/attn/Gather_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_2 [Gather] outputs: [/transformer/h.0/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_3 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_3 [Shape] inputs: [/transformer/h.0/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_3 for ONNX node: /transformer/h.0/attn/Shape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_3 [Shape] outputs: [/transformer/h.0/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_3 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_3 [Gather] inputs: [/transformer/h.0/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_3 for ONNX node: /transformer/h.0/attn/Gather_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_3 [Gather] outputs: [/transformer/h.0/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_2 for ONNX node: /transformer/h.0/attn/Unsqueeze_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_3 for ONNX node: /transformer/h.0/attn/Unsqueeze_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_1 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_1 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_1 for ONNX node: /transformer/h.0/attn/Concat_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_1 [Concat] outputs: [/transformer/h.0/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_1 [Reshape] inputs: [/transformer/h.0/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_1 for ONNX node: /transformer/h.0/attn/Reshape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_1 [Reshape] outputs: [/transformer/h.0/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_4 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_4 [Shape] inputs: [/transformer/h.0/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_4 for ONNX node: /transformer/h.0/attn/Shape_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_4 [Shape] outputs: [/transformer/h.0/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_4 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_4 [Gather] inputs: [/transformer/h.0/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_4 for ONNX node: /transformer/h.0/attn/Gather_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_4 [Gather] outputs: [/transformer/h.0/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_5 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_5 [Shape] inputs: [/transformer/h.0/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_5 for ONNX node: /transformer/h.0/attn/Shape_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_5 [Shape] outputs: [/transformer/h.0/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_5 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_5 [Gather] inputs: [/transformer/h.0/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_5 for ONNX node: /transformer/h.0/attn/Gather_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_5 [Gather] outputs: [/transformer/h.0/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_4 for ONNX node: /transformer/h.0/attn/Unsqueeze_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_5 for ONNX node: /transformer/h.0/attn/Unsqueeze_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_2 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_2 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_2 for ONNX node: /transformer/h.0/attn/Concat_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_2 [Concat] outputs: [/transformer/h.0/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_2 [Reshape] inputs: [/transformer/h.0/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_2 for ONNX node: /transformer/h.0/attn/Reshape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_2 [Reshape] outputs: [/transformer/h.0/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Transpose [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose [Transpose] inputs: [/transformer/h.0/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Transpose for ONNX node: /transformer/h.0/attn/Transpose +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: present.0.value_20 for ONNX tensor: present.0.value +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose [Transpose] outputs: [present.0.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_6 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_6 [Shape] inputs: [/transformer/h.0/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_6 for ONNX node: /transformer/h.0/attn/Shape_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_6 [Shape] outputs: [/transformer/h.0/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_6 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_6 [Gather] inputs: [/transformer/h.0/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_6 for ONNX node: /transformer/h.0/attn/Gather_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_6 [Gather] outputs: [/transformer/h.0/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice [Slice] inputs: [/transformer/h.0/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice for ONNX node: /transformer/h.0/attn/Slice +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice [Slice] outputs: [/transformer/h.0/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_1 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_1 [Slice] inputs: [/transformer/h.0/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_1 for ONNX node: /transformer/h.0/attn/Slice_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_1 [Slice] outputs: [/transformer/h.0/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_2 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_2 [Slice] inputs: [/transformer/h.0/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_2 for ONNX node: /transformer/h.0/attn/Slice_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_2 [Slice] outputs: [/transformer/h.0/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_3 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_3 [Slice] inputs: [/transformer/h.0/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_3 for ONNX node: /transformer/h.0/attn/Slice_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_3 [Slice] outputs: [/transformer/h.0/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_7 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_7 [Shape] inputs: [/transformer/h.0/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_7 for ONNX node: /transformer/h.0/attn/Shape_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_7 [Shape] outputs: [/transformer/h.0/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_7 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_7 [Gather] inputs: [/transformer/h.0/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_29_output_0 for ONNX node: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_7 for ONNX node: /transformer/h.0/attn/Gather_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_7 [Gather] outputs: [/transformer/h.0/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Range [Range] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.0/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Range for ONNX node: /transformer/h.0/attn/Range +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Range_output_0 for ONNX tensor: /transformer/h.0/attn/Range_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Range [Range] outputs: [/transformer/h.0/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Cast_1 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Range_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cast_1 [Cast] inputs: [/transformer/h.0/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Cast_1 for ONNX node: /transformer/h.0/attn/Cast_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.0/attn/Cast_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cast_1 [Cast] outputs: [/transformer/h.0/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Cast_2 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cast_2 [Cast] inputs: [/transformer/h.0/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Cast_2 for ONNX node: /transformer/h.0/attn/Cast_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.0/attn/Cast_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cast_2 [Cast] outputs: [/transformer/h.0/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Div [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Cast_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Cast_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Div [Div] inputs: [/transformer/h.0/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.0/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Div for ONNX node: /transformer/h.0/attn/Div +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Div_output_0 for ONNX tensor: /transformer/h.0/attn/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Div [Div] outputs: [/transformer/h.0/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.0/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_32_output_0 for ONNX node: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Pow for ONNX node: /transformer/h.0/attn/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Pow_output_0 for ONNX tensor: /transformer/h.0/attn/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Pow [Pow] outputs: [/transformer/h.0/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.0/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reciprocal for ONNX node: /transformer/h.0/attn/Reciprocal +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.0/attn/Reciprocal_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.0/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reciprocal_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul [Mul] inputs: [/transformer/h.0/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Mul for ONNX node: /transformer/h.0/attn/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Mul_output_0 for ONNX tensor: /transformer/h.0/attn/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul [Mul] outputs: [/transformer/h.0/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Cast_3 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cast_3 [Cast] inputs: [/transformer/h.0/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Cast_3 for ONNX node: /transformer/h.0/attn/Cast_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.0/attn/Cast_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cast_3 [Cast] outputs: [/transformer/h.0/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Range_1 [Range] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Cast_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.0/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_34_output_0 for ONNX node: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Range_1 for ONNX node: /transformer/h.0/attn/Range_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Range_1_output_0 for ONNX tensor: /transformer/h.0/attn/Range_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Range_1 [Range] outputs: [/transformer/h.0/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Einsum [Einsum] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Range_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Einsum [Einsum] inputs: [/transformer/h.0/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.0/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Einsum for ONNX node: /transformer/h.0/attn/Einsum +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Einsum_output_0 for ONNX tensor: /transformer/h.0/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Einsum [Einsum] outputs: [/transformer/h.0/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Sin [Sin] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Sin [Sin] inputs: [/transformer/h.0/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Sin for ONNX node: /transformer/h.0/attn/Sin +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Sin_output_0 for ONNX tensor: /transformer/h.0/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Sin [Sin] outputs: [/transformer/h.0/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Cos [Cos] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cos [Cos] inputs: [/transformer/h.0/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Cos for ONNX node: /transformer/h.0/attn/Cos +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Cos_output_0 for ONNX tensor: /transformer/h.0/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Cos [Cos] outputs: [/transformer/h.0/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_8 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_8 [Shape] inputs: [/transformer/h.0/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_8 for ONNX node: /transformer/h.0/attn/Shape_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_8 [Shape] outputs: [/transformer/h.0/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_8 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_8 [Gather] inputs: [/transformer/h.0/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_8 for ONNX node: /transformer/h.0/attn/Gather_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_8 [Gather] outputs: [/transformer/h.0/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_3 [Reshape] inputs: [/transformer/h.0/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_3 for ONNX node: /transformer/h.0/attn/Reshape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_3 [Reshape] outputs: [/transformer/h.0/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/Constant_6_output_0 for ONNX node: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/ConstantOfShape for ONNX node: /transformer/h.0/attn/ConstantOfShape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.0/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.0/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Expand [Expand] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Expand [Expand] inputs: [/transformer/h.0/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.0/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Expand for ONNX node: /transformer/h.0/attn/Expand +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Expand_output_0 for ONNX tensor: /transformer/h.0/attn/Expand_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Expand [Expand] outputs: [/transformer/h.0/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Tile [Tile] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Expand_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Tile [Tile] inputs: [/transformer/h.0/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Tile for ONNX node: /transformer/h.0/attn/Tile +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Tile_output_0 for ONNX tensor: /transformer/h.0/attn/Tile_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Tile [Tile] outputs: [/transformer/h.0/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_6 for ONNX node: /transformer/h.0/attn/Unsqueeze_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_3 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_3 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_3 for ONNX node: /transformer/h.0/attn/Concat_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_3 [Concat] outputs: [/transformer/h.0/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Tile_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_4 [Reshape] inputs: [/transformer/h.0/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.0/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_4 for ONNX node: /transformer/h.0/attn/Reshape_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_4 [Reshape] outputs: [/transformer/h.0/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_9 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_9 [Shape] inputs: [/transformer/h.0/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_9 for ONNX node: /transformer/h.0/attn/Shape_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_9 [Shape] outputs: [/transformer/h.0/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_9 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_9 [Gather] inputs: [/transformer/h.0/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_9 for ONNX node: /transformer/h.0/attn/Gather_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_9 [Gather] outputs: [/transformer/h.0/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.0/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_7 for ONNX node: /transformer/h.0/attn/Unsqueeze_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_8 for ONNX node: /transformer/h.0/attn/Unsqueeze_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_4 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_4 [Slice] inputs: [/transformer/h.0/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_4 for ONNX node: /transformer/h.0/attn/Slice_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_4 [Slice] outputs: [/transformer/h.0/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.0/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_9 for ONNX node: /transformer/h.0/attn/Unsqueeze_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_10 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_10 [Shape] inputs: [/transformer/h.0/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_10 for ONNX node: /transformer/h.0/attn/Shape_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_10 [Shape] outputs: [/transformer/h.0/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_10 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_10 [Gather] inputs: [/transformer/h.0/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_10 for ONNX node: /transformer/h.0/attn/Gather_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_10 [Gather] outputs: [/transformer/h.0/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_5 [Reshape] inputs: [/transformer/h.0/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_5 for ONNX node: /transformer/h.0/attn/Reshape_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_5 [Reshape] outputs: [/transformer/h.0/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/ConstantOfShape_1 for ONNX node: /transformer/h.0/attn/ConstantOfShape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.0/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.0/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Expand_1 [Expand] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Expand_1 [Expand] inputs: [/transformer/h.0/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.0/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Expand_1 for ONNX node: /transformer/h.0/attn/Expand_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.0/attn/Expand_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Expand_1 [Expand] outputs: [/transformer/h.0/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Tile_1 [Tile] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Expand_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Tile_1 [Tile] inputs: [/transformer/h.0/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Tile_1 for ONNX node: /transformer/h.0/attn/Tile_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.0/attn/Tile_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Tile_1 [Tile] outputs: [/transformer/h.0/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_10 for ONNX node: /transformer/h.0/attn/Unsqueeze_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_4 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_4 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_4 for ONNX node: /transformer/h.0/attn/Concat_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_4 [Concat] outputs: [/transformer/h.0/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Tile_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_6 [Reshape] inputs: [/transformer/h.0/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.0/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_6 for ONNX node: /transformer/h.0/attn/Reshape_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_6 [Reshape] outputs: [/transformer/h.0/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.0/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_11 for ONNX node: /transformer/h.0/attn/Unsqueeze_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_12 for ONNX node: /transformer/h.0/attn/Unsqueeze_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_5 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_5 [Slice] inputs: [/transformer/h.0/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_5 for ONNX node: /transformer/h.0/attn/Slice_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_5 [Slice] outputs: [/transformer/h.0/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.0/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_13 for ONNX node: /transformer/h.0/attn/Unsqueeze_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Mul_1 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_1 [Mul] inputs: [/transformer/h.0/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Mul_1 for ONNX node: /transformer/h.0/attn/Mul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.0/attn/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_1 [Mul] outputs: [/transformer/h.0/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_6 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_6 [Slice] inputs: [/transformer/h.0/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_6 for ONNX node: /transformer/h.0/attn/Slice_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_6 [Slice] outputs: [/transformer/h.0/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_7 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_7 [Slice] inputs: [/transformer/h.0/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_7 for ONNX node: /transformer/h.0/attn/Slice_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_7 [Slice] outputs: [/transformer/h.0/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Neg [Neg] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Neg [Neg] inputs: [/transformer/h.0/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Neg for ONNX node: /transformer/h.0/attn/Neg +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Neg_output_0 for ONNX tensor: /transformer/h.0/attn/Neg_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Neg [Neg] outputs: [/transformer/h.0/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Neg_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.0/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_14 for ONNX node: /transformer/h.0/attn/Unsqueeze_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.0/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_15 for ONNX node: /transformer/h.0/attn/Unsqueeze_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_5 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_5 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.0/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_5 for ONNX node: /transformer/h.0/attn/Concat_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_5 [Concat] outputs: [/transformer/h.0/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_11 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_11 [Shape] inputs: [/transformer/h.0/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_11 for ONNX node: /transformer/h.0/attn/Shape_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_11 [Shape] outputs: [/transformer/h.0/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_8 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_8 [Slice] inputs: [/transformer/h.0/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_8 for ONNX node: /transformer/h.0/attn/Slice_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_8 [Slice] outputs: [/transformer/h.0/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_6 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_6 [Concat] inputs: [/transformer/h.0/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_6 for ONNX node: /transformer/h.0/attn/Concat_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_6 [Concat] outputs: [/transformer/h.0/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_7 [Reshape] inputs: [/transformer/h.0/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.0/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_7 for ONNX node: /transformer/h.0/attn/Reshape_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_7 [Reshape] outputs: [/transformer/h.0/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Mul_2 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_2 [Mul] inputs: [/transformer/h.0/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Mul_2 for ONNX node: /transformer/h.0/attn/Mul_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.0/attn/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_2 [Mul] outputs: [/transformer/h.0/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Add [Add] inputs: [/transformer/h.0/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Add for ONNX node: /transformer/h.0/attn/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Add_output_0 for ONNX tensor: /transformer/h.0/attn/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Add [Add] outputs: [/transformer/h.0/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_12 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_12 [Shape] inputs: [/transformer/h.0/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_12 for ONNX node: /transformer/h.0/attn/Shape_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_12 [Shape] outputs: [/transformer/h.0/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_11 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_11 [Gather] inputs: [/transformer/h.0/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_11 for ONNX node: /transformer/h.0/attn/Gather_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_11 [Gather] outputs: [/transformer/h.0/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_16 for ONNX node: /transformer/h.0/attn/Unsqueeze_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_9 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_9 [Slice] inputs: [/transformer/h.0/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_9 for ONNX node: /transformer/h.0/attn/Slice_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_9 [Slice] outputs: [/transformer/h.0/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.0/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_17 for ONNX node: /transformer/h.0/attn/Unsqueeze_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_18 for ONNX node: /transformer/h.0/attn/Unsqueeze_18 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_10 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_10 [Slice] inputs: [/transformer/h.0/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_10 for ONNX node: /transformer/h.0/attn/Slice_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_10 [Slice] outputs: [/transformer/h.0/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.0/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_19 for ONNX node: /transformer/h.0/attn/Unsqueeze_19 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Mul_3 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_3 [Mul] inputs: [/transformer/h.0/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Mul_3 for ONNX node: /transformer/h.0/attn/Mul_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.0/attn/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_3 [Mul] outputs: [/transformer/h.0/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_11 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_11 [Slice] inputs: [/transformer/h.0/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_11 for ONNX node: /transformer/h.0/attn/Slice_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_11 [Slice] outputs: [/transformer/h.0/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_12 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_12 [Slice] inputs: [/transformer/h.0/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_12 for ONNX node: /transformer/h.0/attn/Slice_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_12 [Slice] outputs: [/transformer/h.0/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Neg_1 [Neg] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Neg_1 [Neg] inputs: [/transformer/h.0/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Neg_1 for ONNX node: /transformer/h.0/attn/Neg_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.0/attn/Neg_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Neg_1 [Neg] outputs: [/transformer/h.0/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Neg_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.0/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_20 for ONNX node: /transformer/h.0/attn/Unsqueeze_20 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.0/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_21 for ONNX node: /transformer/h.0/attn/Unsqueeze_21 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_7 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_7 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.0/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_7 for ONNX node: /transformer/h.0/attn/Concat_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_7 [Concat] outputs: [/transformer/h.0/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_13 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_13 [Shape] inputs: [/transformer/h.0/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_13 for ONNX node: /transformer/h.0/attn/Shape_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_13 [Shape] outputs: [/transformer/h.0/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_13 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_13 [Slice] inputs: [/transformer/h.0/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_13 for ONNX node: /transformer/h.0/attn/Slice_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_13 [Slice] outputs: [/transformer/h.0/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_8 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_8 [Concat] inputs: [/transformer/h.0/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_8 for ONNX node: /transformer/h.0/attn/Concat_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_8 [Concat] outputs: [/transformer/h.0/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_8 [Reshape] inputs: [/transformer/h.0/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.0/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_8 for ONNX node: /transformer/h.0/attn/Reshape_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_8 [Reshape] outputs: [/transformer/h.0/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Mul_4 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_4 [Mul] inputs: [/transformer/h.0/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Mul_4 for ONNX node: /transformer/h.0/attn/Mul_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.0/attn/Mul_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Mul_4 [Mul] outputs: [/transformer/h.0/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Mul_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Add_1 [Add] inputs: [/transformer/h.0/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Add_1 for ONNX node: /transformer/h.0/attn/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Add_1_output_0 for ONNX tensor: /transformer/h.0/attn/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Add_1 [Add] outputs: [/transformer/h.0/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_9 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_9 [Concat] inputs: [/transformer/h.0/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_9 for ONNX node: /transformer/h.0/attn/Concat_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_9 [Concat] outputs: [/transformer/h.0/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_10 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_10 [Concat] inputs: [/transformer/h.0/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.0/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_10 for ONNX node: /transformer/h.0/attn/Concat_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_10 [Concat] outputs: [/transformer/h.0/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_1 [Transpose] inputs: [/transformer/h.0/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Transpose_1 for ONNX node: /transformer/h.0/attn/Transpose_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: present.0.key_169 for ONNX tensor: present.0.key +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_1 [Transpose] outputs: [present.0.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_2 [Transpose] inputs: [/transformer/h.0/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Transpose_2 for ONNX node: /transformer/h.0/attn/Transpose_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.0/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_2 [Transpose] outputs: [/transformer/h.0/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_14 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_14 [Shape] inputs: [/transformer/h.0/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_14 for ONNX node: /transformer/h.0/attn/Shape_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_14 [Shape] outputs: [/transformer/h.0/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_14 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_14 [Slice] inputs: [/transformer/h.0/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_14 for ONNX node: /transformer/h.0/attn/Slice_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_14 [Slice] outputs: [/transformer/h.0/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Squeeze [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Squeeze [Squeeze] inputs: [/transformer/h.0/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Squeeze for ONNX node: /transformer/h.0/attn/Squeeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.0/attn/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Squeeze [Squeeze] outputs: [/transformer/h.0/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_15 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: present.0.key +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_15 [Shape] inputs: [present.0.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_15 for ONNX node: /transformer/h.0/attn/Shape_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_15 [Shape] outputs: [/transformer/h.0/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_15 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_15 [Slice] inputs: [/transformer/h.0/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_15 for ONNX node: /transformer/h.0/attn/Slice_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_15 [Slice] outputs: [/transformer/h.0/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.0/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Squeeze_1 for ONNX node: /transformer/h.0/attn/Squeeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.0/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.0/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Sub [Sub] inputs: [/transformer/h.0/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.0/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Sub for ONNX node: /transformer/h.0/attn/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Sub_output_0 for ONNX tensor: /transformer/h.0/attn/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Sub [Sub] outputs: [/transformer/h.0/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.0/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_22 for ONNX node: /transformer/h.0/attn/Unsqueeze_22 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.0/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_23 for ONNX node: /transformer/h.0/attn/Unsqueeze_23 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_16 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.0/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_16 for ONNX node: /transformer/h.0/attn/Slice_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_16 [Slice] outputs: [/transformer/h.0/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.0/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_24 for ONNX node: /transformer/h.0/attn/Unsqueeze_24 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Slice_17 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_17 [Slice] inputs: [/transformer/h.0/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Slice_17 for ONNX node: /transformer/h.0/attn/Slice_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.0/attn/Slice_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Slice_17 [Slice] outputs: [/transformer/h.0/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_3 [Transpose] inputs: [/transformer/h.0/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Transpose_3 for ONNX node: /transformer/h.0/attn/Transpose_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.0/attn/Transpose_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_3 [Transpose] outputs: [/transformer/h.0/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Transpose_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/MatMul [MatMul] inputs: [/transformer/h.0/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.0/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/MatMul for ONNX node: /transformer/h.0/attn/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/MatMul_output_0 for ONNX tensor: /transformer/h.0/attn/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/MatMul [MatMul] outputs: [/transformer/h.0/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Where [Where] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Slice_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Where [Where] inputs: [/transformer/h.0/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.0/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Where for ONNX node: /transformer/h.0/attn/Where +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Where_output_0 for ONNX tensor: /transformer/h.0/attn/Where_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Where [Where] outputs: [/transformer/h.0/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Div_1 [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Where_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Div_1 [Div] inputs: [/transformer/h.0/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_112_output_0 for ONNX node: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Div_1 for ONNX node: /transformer/h.0/attn/Div_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Div_1_output_0 for ONNX tensor: /transformer/h.0/attn/Div_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Div_1 [Div] outputs: [/transformer/h.0/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Add_2 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Div_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Add_2 [Add] inputs: [/transformer/h.0/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Add_2 for ONNX node: /transformer/h.0/attn/Add_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Add_2_output_0 for ONNX tensor: /transformer/h.0/attn/Add_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Add_2 [Add] outputs: [/transformer/h.0/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Softmax [Softmax] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Add_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Softmax [Softmax] inputs: [/transformer/h.0/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Softmax for ONNX node: /transformer/h.0/attn/Softmax +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Softmax_output_0 for ONNX tensor: /transformer/h.0/attn/Softmax_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Softmax [Softmax] outputs: [/transformer/h.0/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Softmax_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: present.0.value +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/MatMul_1 [MatMul] inputs: [/transformer/h.0/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.0.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/MatMul_1 for ONNX node: /transformer/h.0/attn/MatMul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.0/attn/MatMul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/MatMul_1 [MatMul] outputs: [/transformer/h.0/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/MatMul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_4 [Transpose] inputs: [/transformer/h.0/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Transpose_4 for ONNX node: /transformer/h.0/attn/Transpose_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.0/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Transpose_4 [Transpose] outputs: [/transformer/h.0/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_16 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_16 [Shape] inputs: [/transformer/h.0/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_16 for ONNX node: /transformer/h.0/attn/Shape_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_16 [Shape] outputs: [/transformer/h.0/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_12 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_12 [Gather] inputs: [/transformer/h.0/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_12 for ONNX node: /transformer/h.0/attn/Gather_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_12 [Gather] outputs: [/transformer/h.0/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Shape_17 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_17 [Shape] inputs: [/transformer/h.0/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Shape_17 for ONNX node: /transformer/h.0/attn/Shape_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.0/attn/Shape_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Shape_17 [Shape] outputs: [/transformer/h.0/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Gather_13 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Shape_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_13 [Gather] inputs: [/transformer/h.0/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Gather_13 for ONNX node: /transformer/h.0/attn/Gather_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.0/attn/Gather_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Gather_13 [Gather] outputs: [/transformer/h.0/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_25 for ONNX node: /transformer/h.0/attn/Unsqueeze_25 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Gather_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.0/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Unsqueeze_26 for ONNX node: /transformer/h.0/attn/Unsqueeze_26 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.0/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.0/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Concat_11 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_11 [Concat] inputs: [/transformer/h.0/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Constant_115_output_0 for ONNX node: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Concat_11 for ONNX node: /transformer/h.0/attn/Concat_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.0/attn/Concat_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Concat_11 [Concat] outputs: [/transformer/h.0/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Concat_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_9 [Reshape] inputs: [/transformer/h.0/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/Reshape_9 for ONNX node: /transformer/h.0/attn/Reshape_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.0/attn/Reshape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/Reshape_9 [Reshape] outputs: [/transformer/h.0/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Reshape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2851 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.0/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2851 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2851 for ONNX node: onnx::MatMul_2851 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/attn/out_proj/MatMul for ONNX node: /transformer/h.0/attn/out_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.0/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.0/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2852 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.0/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2852 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2852 for ONNX node: onnx::MatMul_2852 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/fc_in/MatMul for ONNX node: /transformer/h.0/mlp/fc_in/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.0/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.0/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/fc_in/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.0/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: transformer.h.0.mlp.fc_in.bias for ONNX node: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/fc_in/Add for ONNX node: /transformer/h.0/mlp/fc_in/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.0/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_in/Add [Add] outputs: [/transformer/h.0/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul [Mul] inputs: [/transformer/h.0/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Constant_output_0 for ONNX node: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Mul for ONNX node: /transformer/h.0/mlp/act/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul [Mul] outputs: [/transformer/h.0/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Pow [Pow] inputs: [/transformer/h.0/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Constant_1_output_0 for ONNX node: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Pow for ONNX node: /transformer/h.0/mlp/act/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Pow [Pow] outputs: [/transformer/h.0/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.0/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Constant_2_output_0 for ONNX node: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Mul_1 for ONNX node: /transformer/h.0/mlp/act/Mul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.0/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Add [Add] inputs: [/transformer/h.0/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Add for ONNX node: /transformer/h.0/mlp/act/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Add [Add] outputs: [/transformer/h.0/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.0/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Constant_3_output_0 for ONNX node: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Mul_2 for ONNX node: /transformer/h.0/mlp/act/Mul_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.0/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Tanh [Tanh] inputs: [/transformer/h.0/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Tanh for ONNX node: /transformer/h.0/mlp/act/Tanh +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Tanh_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Tanh [Tanh] outputs: [/transformer/h.0/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Tanh_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Add_1 [Add] inputs: [/transformer/h.0/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Add_1 for ONNX node: /transformer/h.0/mlp/act/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Add_1 [Add] outputs: [/transformer/h.0/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.0/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/act/Mul_3 for ONNX node: /transformer/h.0/mlp/act/Mul_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.0/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.0/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2853 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.0/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_2853 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2853 for ONNX node: onnx::MatMul_2853 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/fc_out/MatMul for ONNX node: /transformer/h.0/mlp/fc_out/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.0/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.0/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/mlp/fc_out/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.0/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/mlp/fc_out/Add for ONNX node: /transformer/h.0/mlp/fc_out/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.0/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/mlp/fc_out/Add [Add] outputs: [/transformer/h.0/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/Add [Add] inputs: [/transformer/h.0/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/Add for ONNX node: /transformer/h.0/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/Add_output_0 for ONNX tensor: /transformer/h.0/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/Add [Add] outputs: [/transformer/h.0/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.0/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/wte/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/Add_1 [Add] inputs: [/transformer/h.0/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/wte/Gather_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.0/Add_1 for ONNX node: /transformer/h.0/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.0/Add_1_output_0 for ONNX tensor: /transformer/h.0/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.0/Add_1 [Add] outputs: [/transformer/h.0/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.0/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/ReduceMean for ONNX node: /transformer/h.1/ln_1/ReduceMean +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.1/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.1/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Sub [Sub] inputs: [/transformer/h.0/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Sub for ONNX node: /transformer/h.1/ln_1/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.1/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Sub [Sub] outputs: [/transformer/h.1/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Pow [Pow] inputs: [/transformer/h.1/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Pow for ONNX node: /transformer/h.1/ln_1/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.1/ln_1/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Pow [Pow] outputs: [/transformer/h.1/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.1/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/ReduceMean_1 for ONNX node: /transformer/h.1/ln_1/ReduceMean_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.1/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.1/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Add [Add] inputs: [/transformer/h.1/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Add for ONNX node: /transformer/h.1/ln_1/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Add_output_0 for ONNX tensor: /transformer/h.1/ln_1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Add [Add] outputs: [/transformer/h.1/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.1/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Sqrt for ONNX node: /transformer/h.1/ln_1/Sqrt +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.1/ln_1/Sqrt_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.1/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Div [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Sqrt_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Div [Div] inputs: [/transformer/h.1/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Div for ONNX node: /transformer/h.1/ln_1/Div +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Div_output_0 for ONNX tensor: /transformer/h.1/ln_1/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Div [Div] outputs: [/transformer/h.1/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Mul [Mul] inputs: [/transformer/h.1/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Mul for ONNX node: /transformer/h.1/ln_1/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.1/ln_1/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Mul [Mul] outputs: [/transformer/h.1/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/ln_1/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Add_1 [Add] inputs: [/transformer/h.1/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/ln_1/Add_1 for ONNX node: /transformer/h.1/ln_1/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.1/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/ln_1/Add_1 [Add] outputs: [/transformer/h.1/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2854 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.1/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2854 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2854 for ONNX node: onnx::MatMul_2854 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/q_proj/MatMul for ONNX node: /transformer/h.1/attn/q_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.1/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.1/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2855 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.1/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2855 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2855 for ONNX node: onnx::MatMul_2855 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/k_proj/MatMul for ONNX node: /transformer/h.1/attn/k_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.1/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.1/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2856 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.1/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2856 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2856 for ONNX node: onnx::MatMul_2856 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/v_proj/MatMul for ONNX node: /transformer/h.1/attn/v_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.1/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.1/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape [Shape] inputs: [/transformer/h.1/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape for ONNX node: /transformer/h.1/attn/Shape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape [Shape] outputs: [/transformer/h.1/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather [Gather] inputs: [/transformer/h.1/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather for ONNX node: /transformer/h.1/attn/Gather +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather [Gather] outputs: [/transformer/h.1/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_1 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_1 [Shape] inputs: [/transformer/h.1/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_1 for ONNX node: /transformer/h.1/attn/Shape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_1 [Shape] outputs: [/transformer/h.1/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_1 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_1 [Gather] inputs: [/transformer/h.1/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_1 for ONNX node: /transformer/h.1/attn/Gather_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_1 [Gather] outputs: [/transformer/h.1/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze for ONNX node: /transformer/h.1/attn/Unsqueeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_1 for ONNX node: /transformer/h.1/attn/Unsqueeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat for ONNX node: /transformer/h.1/attn/Concat +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat [Concat] outputs: [/transformer/h.1/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape [Reshape] inputs: [/transformer/h.1/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape for ONNX node: /transformer/h.1/attn/Reshape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape [Reshape] outputs: [/transformer/h.1/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_2 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_2 [Shape] inputs: [/transformer/h.1/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_2 for ONNX node: /transformer/h.1/attn/Shape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_2 [Shape] outputs: [/transformer/h.1/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_2 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_2 [Gather] inputs: [/transformer/h.1/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_2 for ONNX node: /transformer/h.1/attn/Gather_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_2 [Gather] outputs: [/transformer/h.1/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_3 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_3 [Shape] inputs: [/transformer/h.1/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_3 for ONNX node: /transformer/h.1/attn/Shape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_3 [Shape] outputs: [/transformer/h.1/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_3 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_3 [Gather] inputs: [/transformer/h.1/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_3 for ONNX node: /transformer/h.1/attn/Gather_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_3 [Gather] outputs: [/transformer/h.1/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_2 for ONNX node: /transformer/h.1/attn/Unsqueeze_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_3 for ONNX node: /transformer/h.1/attn/Unsqueeze_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_1 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_1 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_1 for ONNX node: /transformer/h.1/attn/Concat_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_1 [Concat] outputs: [/transformer/h.1/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_1 [Reshape] inputs: [/transformer/h.1/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_1 for ONNX node: /transformer/h.1/attn/Reshape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_1 [Reshape] outputs: [/transformer/h.1/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_4 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_4 [Shape] inputs: [/transformer/h.1/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_4 for ONNX node: /transformer/h.1/attn/Shape_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_4 [Shape] outputs: [/transformer/h.1/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_4 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_4 [Gather] inputs: [/transformer/h.1/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_4 for ONNX node: /transformer/h.1/attn/Gather_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_4 [Gather] outputs: [/transformer/h.1/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_5 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_5 [Shape] inputs: [/transformer/h.1/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_5 for ONNX node: /transformer/h.1/attn/Shape_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_5 [Shape] outputs: [/transformer/h.1/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_5 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_5 [Gather] inputs: [/transformer/h.1/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_5 for ONNX node: /transformer/h.1/attn/Gather_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_5 [Gather] outputs: [/transformer/h.1/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_4 for ONNX node: /transformer/h.1/attn/Unsqueeze_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_5 for ONNX node: /transformer/h.1/attn/Unsqueeze_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_2 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_2 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_2 for ONNX node: /transformer/h.1/attn/Concat_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_2 [Concat] outputs: [/transformer/h.1/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_2 [Reshape] inputs: [/transformer/h.1/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_2 for ONNX node: /transformer/h.1/attn/Reshape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_2 [Reshape] outputs: [/transformer/h.1/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Transpose [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose [Transpose] inputs: [/transformer/h.1/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Transpose for ONNX node: /transformer/h.1/attn/Transpose +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: present.1.value_189 for ONNX tensor: present.1.value +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose [Transpose] outputs: [present.1.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_6 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_6 [Shape] inputs: [/transformer/h.1/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_6 for ONNX node: /transformer/h.1/attn/Shape_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_6 [Shape] outputs: [/transformer/h.1/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_6 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_6 [Gather] inputs: [/transformer/h.1/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_6 for ONNX node: /transformer/h.1/attn/Gather_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_6 [Gather] outputs: [/transformer/h.1/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice [Slice] inputs: [/transformer/h.1/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice for ONNX node: /transformer/h.1/attn/Slice +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice [Slice] outputs: [/transformer/h.1/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_1 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_1 [Slice] inputs: [/transformer/h.1/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_1 for ONNX node: /transformer/h.1/attn/Slice_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_1 [Slice] outputs: [/transformer/h.1/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_2 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_2 [Slice] inputs: [/transformer/h.1/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_2 for ONNX node: /transformer/h.1/attn/Slice_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_2 [Slice] outputs: [/transformer/h.1/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_3 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_3 [Slice] inputs: [/transformer/h.1/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_3 for ONNX node: /transformer/h.1/attn/Slice_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_3 [Slice] outputs: [/transformer/h.1/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_7 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_7 [Shape] inputs: [/transformer/h.1/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_7 for ONNX node: /transformer/h.1/attn/Shape_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_7 [Shape] outputs: [/transformer/h.1/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_7 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_7 [Gather] inputs: [/transformer/h.1/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_7 for ONNX node: /transformer/h.1/attn/Gather_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_7 [Gather] outputs: [/transformer/h.1/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Range [Range] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.1/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Range for ONNX node: /transformer/h.1/attn/Range +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Range_output_0 for ONNX tensor: /transformer/h.1/attn/Range_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Range [Range] outputs: [/transformer/h.1/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Cast_1 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Range_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cast_1 [Cast] inputs: [/transformer/h.1/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Cast_1 for ONNX node: /transformer/h.1/attn/Cast_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.1/attn/Cast_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cast_1 [Cast] outputs: [/transformer/h.1/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Cast_2 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cast_2 [Cast] inputs: [/transformer/h.1/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Cast_2 for ONNX node: /transformer/h.1/attn/Cast_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.1/attn/Cast_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cast_2 [Cast] outputs: [/transformer/h.1/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Div [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Cast_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Cast_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Div [Div] inputs: [/transformer/h.1/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.1/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Div for ONNX node: /transformer/h.1/attn/Div +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Div_output_0 for ONNX tensor: /transformer/h.1/attn/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Div [Div] outputs: [/transformer/h.1/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.1/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Pow for ONNX node: /transformer/h.1/attn/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Pow_output_0 for ONNX tensor: /transformer/h.1/attn/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Pow [Pow] outputs: [/transformer/h.1/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.1/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reciprocal for ONNX node: /transformer/h.1/attn/Reciprocal +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.1/attn/Reciprocal_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.1/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reciprocal_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul [Mul] inputs: [/transformer/h.1/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Mul for ONNX node: /transformer/h.1/attn/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Mul_output_0 for ONNX tensor: /transformer/h.1/attn/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul [Mul] outputs: [/transformer/h.1/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Cast_3 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cast_3 [Cast] inputs: [/transformer/h.1/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Cast_3 for ONNX node: /transformer/h.1/attn/Cast_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.1/attn/Cast_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cast_3 [Cast] outputs: [/transformer/h.1/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Range_1 [Range] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Cast_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.1/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Range_1 for ONNX node: /transformer/h.1/attn/Range_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Range_1_output_0 for ONNX tensor: /transformer/h.1/attn/Range_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Range_1 [Range] outputs: [/transformer/h.1/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Einsum [Einsum] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Range_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Einsum [Einsum] inputs: [/transformer/h.1/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.1/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Einsum for ONNX node: /transformer/h.1/attn/Einsum +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Einsum_output_0 for ONNX tensor: /transformer/h.1/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Einsum [Einsum] outputs: [/transformer/h.1/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Sin [Sin] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Sin [Sin] inputs: [/transformer/h.1/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Sin for ONNX node: /transformer/h.1/attn/Sin +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Sin_output_0 for ONNX tensor: /transformer/h.1/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Sin [Sin] outputs: [/transformer/h.1/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Cos [Cos] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cos [Cos] inputs: [/transformer/h.1/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Cos for ONNX node: /transformer/h.1/attn/Cos +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Cos_output_0 for ONNX tensor: /transformer/h.1/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Cos [Cos] outputs: [/transformer/h.1/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_8 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_8 [Shape] inputs: [/transformer/h.1/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_8 for ONNX node: /transformer/h.1/attn/Shape_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_8 [Shape] outputs: [/transformer/h.1/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_8 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_8 [Gather] inputs: [/transformer/h.1/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_8 for ONNX node: /transformer/h.1/attn/Gather_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_8 [Gather] outputs: [/transformer/h.1/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_3 [Reshape] inputs: [/transformer/h.1/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_3 for ONNX node: /transformer/h.1/attn/Reshape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_3 [Reshape] outputs: [/transformer/h.1/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/ConstantOfShape for ONNX node: /transformer/h.1/attn/ConstantOfShape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.1/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.1/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Expand [Expand] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Expand [Expand] inputs: [/transformer/h.1/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.1/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Expand for ONNX node: /transformer/h.1/attn/Expand +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Expand_output_0 for ONNX tensor: /transformer/h.1/attn/Expand_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Expand [Expand] outputs: [/transformer/h.1/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Tile [Tile] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Expand_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Tile [Tile] inputs: [/transformer/h.1/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Tile for ONNX node: /transformer/h.1/attn/Tile +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Tile_output_0 for ONNX tensor: /transformer/h.1/attn/Tile_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Tile [Tile] outputs: [/transformer/h.1/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_6 for ONNX node: /transformer/h.1/attn/Unsqueeze_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_3 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_3 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_3 for ONNX node: /transformer/h.1/attn/Concat_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_3 [Concat] outputs: [/transformer/h.1/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Tile_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_4 [Reshape] inputs: [/transformer/h.1/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.1/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_4 for ONNX node: /transformer/h.1/attn/Reshape_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_4 [Reshape] outputs: [/transformer/h.1/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_9 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_9 [Shape] inputs: [/transformer/h.1/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_9 for ONNX node: /transformer/h.1/attn/Shape_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_9 [Shape] outputs: [/transformer/h.1/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_9 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_9 [Gather] inputs: [/transformer/h.1/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_9 for ONNX node: /transformer/h.1/attn/Gather_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_9 [Gather] outputs: [/transformer/h.1/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.1/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_7 for ONNX node: /transformer/h.1/attn/Unsqueeze_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_8 for ONNX node: /transformer/h.1/attn/Unsqueeze_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_4 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_4 [Slice] inputs: [/transformer/h.1/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_4 for ONNX node: /transformer/h.1/attn/Slice_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_4 [Slice] outputs: [/transformer/h.1/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.1/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_9 for ONNX node: /transformer/h.1/attn/Unsqueeze_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_10 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_10 [Shape] inputs: [/transformer/h.1/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_10 for ONNX node: /transformer/h.1/attn/Shape_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_10 [Shape] outputs: [/transformer/h.1/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_10 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_10 [Gather] inputs: [/transformer/h.1/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_10 for ONNX node: /transformer/h.1/attn/Gather_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_10 [Gather] outputs: [/transformer/h.1/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_5 [Reshape] inputs: [/transformer/h.1/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_5 for ONNX node: /transformer/h.1/attn/Reshape_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_5 [Reshape] outputs: [/transformer/h.1/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/ConstantOfShape_1 for ONNX node: /transformer/h.1/attn/ConstantOfShape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.1/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.1/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Expand_1 [Expand] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Expand_1 [Expand] inputs: [/transformer/h.1/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.1/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Expand_1 for ONNX node: /transformer/h.1/attn/Expand_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.1/attn/Expand_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Expand_1 [Expand] outputs: [/transformer/h.1/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Tile_1 [Tile] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Expand_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Tile_1 [Tile] inputs: [/transformer/h.1/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Tile_1 for ONNX node: /transformer/h.1/attn/Tile_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.1/attn/Tile_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Tile_1 [Tile] outputs: [/transformer/h.1/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_10 for ONNX node: /transformer/h.1/attn/Unsqueeze_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_4 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_4 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_4 for ONNX node: /transformer/h.1/attn/Concat_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_4 [Concat] outputs: [/transformer/h.1/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Tile_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_6 [Reshape] inputs: [/transformer/h.1/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.1/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_6 for ONNX node: /transformer/h.1/attn/Reshape_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_6 [Reshape] outputs: [/transformer/h.1/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.1/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_11 for ONNX node: /transformer/h.1/attn/Unsqueeze_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_12 for ONNX node: /transformer/h.1/attn/Unsqueeze_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_5 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_5 [Slice] inputs: [/transformer/h.1/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_5 for ONNX node: /transformer/h.1/attn/Slice_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_5 [Slice] outputs: [/transformer/h.1/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.1/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_13 for ONNX node: /transformer/h.1/attn/Unsqueeze_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Mul_1 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_1 [Mul] inputs: [/transformer/h.1/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Mul_1 for ONNX node: /transformer/h.1/attn/Mul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.1/attn/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_1 [Mul] outputs: [/transformer/h.1/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_6 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_6 [Slice] inputs: [/transformer/h.1/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_6 for ONNX node: /transformer/h.1/attn/Slice_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_6 [Slice] outputs: [/transformer/h.1/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_7 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_7 [Slice] inputs: [/transformer/h.1/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_7 for ONNX node: /transformer/h.1/attn/Slice_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_7 [Slice] outputs: [/transformer/h.1/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Neg [Neg] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Neg [Neg] inputs: [/transformer/h.1/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Neg for ONNX node: /transformer/h.1/attn/Neg +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Neg_output_0 for ONNX tensor: /transformer/h.1/attn/Neg_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Neg [Neg] outputs: [/transformer/h.1/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Neg_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.1/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_14 for ONNX node: /transformer/h.1/attn/Unsqueeze_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.1/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_15 for ONNX node: /transformer/h.1/attn/Unsqueeze_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_5 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_5 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.1/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_5 for ONNX node: /transformer/h.1/attn/Concat_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_5 [Concat] outputs: [/transformer/h.1/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_11 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_11 [Shape] inputs: [/transformer/h.1/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_11 for ONNX node: /transformer/h.1/attn/Shape_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_11 [Shape] outputs: [/transformer/h.1/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_8 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_8 [Slice] inputs: [/transformer/h.1/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_8 for ONNX node: /transformer/h.1/attn/Slice_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_8 [Slice] outputs: [/transformer/h.1/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_6 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_6 [Concat] inputs: [/transformer/h.1/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_6 for ONNX node: /transformer/h.1/attn/Concat_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_6 [Concat] outputs: [/transformer/h.1/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_7 [Reshape] inputs: [/transformer/h.1/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.1/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_7 for ONNX node: /transformer/h.1/attn/Reshape_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_7 [Reshape] outputs: [/transformer/h.1/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Mul_2 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_2 [Mul] inputs: [/transformer/h.1/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Mul_2 for ONNX node: /transformer/h.1/attn/Mul_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.1/attn/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_2 [Mul] outputs: [/transformer/h.1/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Add [Add] inputs: [/transformer/h.1/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Add for ONNX node: /transformer/h.1/attn/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Add_output_0 for ONNX tensor: /transformer/h.1/attn/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Add [Add] outputs: [/transformer/h.1/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_12 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_12 [Shape] inputs: [/transformer/h.1/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_12 for ONNX node: /transformer/h.1/attn/Shape_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_12 [Shape] outputs: [/transformer/h.1/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_11 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_11 [Gather] inputs: [/transformer/h.1/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_11 for ONNX node: /transformer/h.1/attn/Gather_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_11 [Gather] outputs: [/transformer/h.1/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_16 for ONNX node: /transformer/h.1/attn/Unsqueeze_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_9 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_9 [Slice] inputs: [/transformer/h.1/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_9 for ONNX node: /transformer/h.1/attn/Slice_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_9 [Slice] outputs: [/transformer/h.1/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.1/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_17 for ONNX node: /transformer/h.1/attn/Unsqueeze_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_18 for ONNX node: /transformer/h.1/attn/Unsqueeze_18 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_10 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_10 [Slice] inputs: [/transformer/h.1/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_10 for ONNX node: /transformer/h.1/attn/Slice_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_10 [Slice] outputs: [/transformer/h.1/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.1/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_19 for ONNX node: /transformer/h.1/attn/Unsqueeze_19 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Mul_3 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_3 [Mul] inputs: [/transformer/h.1/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Mul_3 for ONNX node: /transformer/h.1/attn/Mul_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.1/attn/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_3 [Mul] outputs: [/transformer/h.1/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_11 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_11 [Slice] inputs: [/transformer/h.1/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_11 for ONNX node: /transformer/h.1/attn/Slice_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_11 [Slice] outputs: [/transformer/h.1/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_12 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_12 [Slice] inputs: [/transformer/h.1/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_12 for ONNX node: /transformer/h.1/attn/Slice_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_12 [Slice] outputs: [/transformer/h.1/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Neg_1 [Neg] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Neg_1 [Neg] inputs: [/transformer/h.1/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Neg_1 for ONNX node: /transformer/h.1/attn/Neg_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.1/attn/Neg_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Neg_1 [Neg] outputs: [/transformer/h.1/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Neg_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.1/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_20 for ONNX node: /transformer/h.1/attn/Unsqueeze_20 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.1/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_21 for ONNX node: /transformer/h.1/attn/Unsqueeze_21 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_7 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_7 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.1/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_7 for ONNX node: /transformer/h.1/attn/Concat_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_7 [Concat] outputs: [/transformer/h.1/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_13 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_13 [Shape] inputs: [/transformer/h.1/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_13 for ONNX node: /transformer/h.1/attn/Shape_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_13 [Shape] outputs: [/transformer/h.1/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_13 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_13 [Slice] inputs: [/transformer/h.1/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_13 for ONNX node: /transformer/h.1/attn/Slice_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_13 [Slice] outputs: [/transformer/h.1/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_8 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_8 [Concat] inputs: [/transformer/h.1/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_8 for ONNX node: /transformer/h.1/attn/Concat_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_8 [Concat] outputs: [/transformer/h.1/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_8 [Reshape] inputs: [/transformer/h.1/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.1/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_8 for ONNX node: /transformer/h.1/attn/Reshape_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_8 [Reshape] outputs: [/transformer/h.1/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Mul_4 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_4 [Mul] inputs: [/transformer/h.1/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Mul_4 for ONNX node: /transformer/h.1/attn/Mul_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.1/attn/Mul_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Mul_4 [Mul] outputs: [/transformer/h.1/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Mul_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Add_1 [Add] inputs: [/transformer/h.1/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Add_1 for ONNX node: /transformer/h.1/attn/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Add_1_output_0 for ONNX tensor: /transformer/h.1/attn/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Add_1 [Add] outputs: [/transformer/h.1/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_9 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_9 [Concat] inputs: [/transformer/h.1/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_9 for ONNX node: /transformer/h.1/attn/Concat_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_9 [Concat] outputs: [/transformer/h.1/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_10 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_10 [Concat] inputs: [/transformer/h.1/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.1/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_10 for ONNX node: /transformer/h.1/attn/Concat_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_10 [Concat] outputs: [/transformer/h.1/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_1 [Transpose] inputs: [/transformer/h.1/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Transpose_1 for ONNX node: /transformer/h.1/attn/Transpose_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: present.1.key_338 for ONNX tensor: present.1.key +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_1 [Transpose] outputs: [present.1.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_2 [Transpose] inputs: [/transformer/h.1/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Transpose_2 for ONNX node: /transformer/h.1/attn/Transpose_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.1/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_2 [Transpose] outputs: [/transformer/h.1/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_14 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_14 [Shape] inputs: [/transformer/h.1/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_14 for ONNX node: /transformer/h.1/attn/Shape_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_14 [Shape] outputs: [/transformer/h.1/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_14 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_14 [Slice] inputs: [/transformer/h.1/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_14 for ONNX node: /transformer/h.1/attn/Slice_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_14 [Slice] outputs: [/transformer/h.1/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Squeeze [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Squeeze [Squeeze] inputs: [/transformer/h.1/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Squeeze for ONNX node: /transformer/h.1/attn/Squeeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.1/attn/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Squeeze [Squeeze] outputs: [/transformer/h.1/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_15 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: present.1.key +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_15 [Shape] inputs: [present.1.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_15 for ONNX node: /transformer/h.1/attn/Shape_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_15 [Shape] outputs: [/transformer/h.1/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_15 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_15 [Slice] inputs: [/transformer/h.1/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_15 for ONNX node: /transformer/h.1/attn/Slice_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_15 [Slice] outputs: [/transformer/h.1/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.1/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Squeeze_1 for ONNX node: /transformer/h.1/attn/Squeeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.1/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.1/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Sub [Sub] inputs: [/transformer/h.1/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.1/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Sub for ONNX node: /transformer/h.1/attn/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Sub_output_0 for ONNX tensor: /transformer/h.1/attn/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Sub [Sub] outputs: [/transformer/h.1/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.1/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_22 for ONNX node: /transformer/h.1/attn/Unsqueeze_22 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.1/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_23 for ONNX node: /transformer/h.1/attn/Unsqueeze_23 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_16 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.1/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_16 for ONNX node: /transformer/h.1/attn/Slice_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_16 [Slice] outputs: [/transformer/h.1/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.1/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_24 for ONNX node: /transformer/h.1/attn/Unsqueeze_24 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Slice_17 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_17 [Slice] inputs: [/transformer/h.1/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Slice_17 for ONNX node: /transformer/h.1/attn/Slice_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.1/attn/Slice_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Slice_17 [Slice] outputs: [/transformer/h.1/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_3 [Transpose] inputs: [/transformer/h.1/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Transpose_3 for ONNX node: /transformer/h.1/attn/Transpose_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.1/attn/Transpose_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_3 [Transpose] outputs: [/transformer/h.1/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Transpose_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/MatMul [MatMul] inputs: [/transformer/h.1/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.1/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/MatMul for ONNX node: /transformer/h.1/attn/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/MatMul_output_0 for ONNX tensor: /transformer/h.1/attn/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/MatMul [MatMul] outputs: [/transformer/h.1/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Where [Where] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Slice_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Where [Where] inputs: [/transformer/h.1/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.1/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Where for ONNX node: /transformer/h.1/attn/Where +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Where_output_0 for ONNX tensor: /transformer/h.1/attn/Where_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Where [Where] outputs: [/transformer/h.1/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Div_1 [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Where_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Div_1 [Div] inputs: [/transformer/h.1/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Div_1 for ONNX node: /transformer/h.1/attn/Div_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Div_1_output_0 for ONNX tensor: /transformer/h.1/attn/Div_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Div_1 [Div] outputs: [/transformer/h.1/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Add_2 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Div_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Add_2 [Add] inputs: [/transformer/h.1/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Add_2 for ONNX node: /transformer/h.1/attn/Add_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Add_2_output_0 for ONNX tensor: /transformer/h.1/attn/Add_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Add_2 [Add] outputs: [/transformer/h.1/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Softmax [Softmax] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Add_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Softmax [Softmax] inputs: [/transformer/h.1/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Softmax for ONNX node: /transformer/h.1/attn/Softmax +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Softmax_output_0 for ONNX tensor: /transformer/h.1/attn/Softmax_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Softmax [Softmax] outputs: [/transformer/h.1/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Softmax_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: present.1.value +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/MatMul_1 [MatMul] inputs: [/transformer/h.1/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.1.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/MatMul_1 for ONNX node: /transformer/h.1/attn/MatMul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.1/attn/MatMul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/MatMul_1 [MatMul] outputs: [/transformer/h.1/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/MatMul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_4 [Transpose] inputs: [/transformer/h.1/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Transpose_4 for ONNX node: /transformer/h.1/attn/Transpose_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.1/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Transpose_4 [Transpose] outputs: [/transformer/h.1/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_16 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_16 [Shape] inputs: [/transformer/h.1/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_16 for ONNX node: /transformer/h.1/attn/Shape_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_16 [Shape] outputs: [/transformer/h.1/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_12 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_12 [Gather] inputs: [/transformer/h.1/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_12 for ONNX node: /transformer/h.1/attn/Gather_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_12 [Gather] outputs: [/transformer/h.1/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Shape_17 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_17 [Shape] inputs: [/transformer/h.1/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Shape_17 for ONNX node: /transformer/h.1/attn/Shape_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.1/attn/Shape_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Shape_17 [Shape] outputs: [/transformer/h.1/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Gather_13 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Shape_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_13 [Gather] inputs: [/transformer/h.1/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Gather_13 for ONNX node: /transformer/h.1/attn/Gather_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.1/attn/Gather_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Gather_13 [Gather] outputs: [/transformer/h.1/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_25 for ONNX node: /transformer/h.1/attn/Unsqueeze_25 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Gather_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.1/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Unsqueeze_26 for ONNX node: /transformer/h.1/attn/Unsqueeze_26 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.1/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.1/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Concat_11 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_11 [Concat] inputs: [/transformer/h.1/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.1/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Concat_11 for ONNX node: /transformer/h.1/attn/Concat_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.1/attn/Concat_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Concat_11 [Concat] outputs: [/transformer/h.1/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Transpose_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Concat_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_9 [Reshape] inputs: [/transformer/h.1/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.1/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/Reshape_9 for ONNX node: /transformer/h.1/attn/Reshape_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.1/attn/Reshape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/Reshape_9 [Reshape] outputs: [/transformer/h.1/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/Reshape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2881 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.1/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2881 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2881 for ONNX node: onnx::MatMul_2881 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/attn/out_proj/MatMul for ONNX node: /transformer/h.1/attn/out_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.1/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.1/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2882 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.1/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2882 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2882 for ONNX node: onnx::MatMul_2882 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/fc_in/MatMul for ONNX node: /transformer/h.1/mlp/fc_in/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.1/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.1/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/fc_in/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.1/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/fc_in/Add for ONNX node: /transformer/h.1/mlp/fc_in/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.1/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_in/Add [Add] outputs: [/transformer/h.1/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul [Mul] inputs: [/transformer/h.1/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Mul for ONNX node: /transformer/h.1/mlp/act/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul [Mul] outputs: [/transformer/h.1/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Pow [Pow] inputs: [/transformer/h.1/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Pow for ONNX node: /transformer/h.1/mlp/act/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Pow [Pow] outputs: [/transformer/h.1/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.1/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Mul_1 for ONNX node: /transformer/h.1/mlp/act/Mul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.1/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Add [Add] inputs: [/transformer/h.1/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.1/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Add for ONNX node: /transformer/h.1/mlp/act/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Add [Add] outputs: [/transformer/h.1/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.1/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Mul_2 for ONNX node: /transformer/h.1/mlp/act/Mul_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.1/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Tanh [Tanh] inputs: [/transformer/h.1/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Tanh for ONNX node: /transformer/h.1/mlp/act/Tanh +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Tanh_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Tanh [Tanh] outputs: [/transformer/h.1/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Tanh_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Add_1 [Add] inputs: [/transformer/h.1/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Add_1 for ONNX node: /transformer/h.1/mlp/act/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Add_1 [Add] outputs: [/transformer/h.1/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.1/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.1/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/act/Mul_3 for ONNX node: /transformer/h.1/mlp/act/Mul_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.1/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.1/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2883 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.1/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_2883 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2883 for ONNX node: onnx::MatMul_2883 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/fc_out/MatMul for ONNX node: /transformer/h.1/mlp/fc_out/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.1/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.1/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/mlp/fc_out/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.1/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/mlp/fc_out/Add for ONNX node: /transformer/h.1/mlp/fc_out/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.1/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/mlp/fc_out/Add [Add] outputs: [/transformer/h.1/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/Add [Add] inputs: [/transformer/h.1/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/Add for ONNX node: /transformer/h.1/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/Add_output_0 for ONNX tensor: /transformer/h.1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/Add [Add] outputs: [/transformer/h.1/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.1/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/Add_1 [Add] inputs: [/transformer/h.1/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.1/Add_1 for ONNX node: /transformer/h.1/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.1/Add_1_output_0 for ONNX tensor: /transformer/h.1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.1/Add_1 [Add] outputs: [/transformer/h.1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/ReduceMean for ONNX node: /transformer/h.2/ln_1/ReduceMean +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.2/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.2/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Sub [Sub] inputs: [/transformer/h.1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Sub for ONNX node: /transformer/h.2/ln_1/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.2/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Sub [Sub] outputs: [/transformer/h.2/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Pow [Pow] inputs: [/transformer/h.2/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Pow for ONNX node: /transformer/h.2/ln_1/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.2/ln_1/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Pow [Pow] outputs: [/transformer/h.2/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.2/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/ReduceMean_1 for ONNX node: /transformer/h.2/ln_1/ReduceMean_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.2/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.2/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Add [Add] inputs: [/transformer/h.2/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Add for ONNX node: /transformer/h.2/ln_1/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Add_output_0 for ONNX tensor: /transformer/h.2/ln_1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Add [Add] outputs: [/transformer/h.2/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.2/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Sqrt for ONNX node: /transformer/h.2/ln_1/Sqrt +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.2/ln_1/Sqrt_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.2/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Div [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Sqrt_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Div [Div] inputs: [/transformer/h.2/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Div for ONNX node: /transformer/h.2/ln_1/Div +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Div_output_0 for ONNX tensor: /transformer/h.2/ln_1/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Div [Div] outputs: [/transformer/h.2/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Mul [Mul] inputs: [/transformer/h.2/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Mul for ONNX node: /transformer/h.2/ln_1/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.2/ln_1/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Mul [Mul] outputs: [/transformer/h.2/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/ln_1/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Add_1 [Add] inputs: [/transformer/h.2/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/ln_1/Add_1 for ONNX node: /transformer/h.2/ln_1/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.2/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/ln_1/Add_1 [Add] outputs: [/transformer/h.2/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2884 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.2/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2884 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2884 for ONNX node: onnx::MatMul_2884 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/q_proj/MatMul for ONNX node: /transformer/h.2/attn/q_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.2/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.2/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2885 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.2/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2885 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2885 for ONNX node: onnx::MatMul_2885 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/k_proj/MatMul for ONNX node: /transformer/h.2/attn/k_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.2/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.2/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::MatMul_2886 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.2/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2886 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: onnx::MatMul_2886 for ONNX node: onnx::MatMul_2886 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/v_proj/MatMul for ONNX node: /transformer/h.2/attn/v_proj/MatMul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.2/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.2/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape [Shape] inputs: [/transformer/h.2/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape for ONNX node: /transformer/h.2/attn/Shape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape [Shape] outputs: [/transformer/h.2/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather [Gather] inputs: [/transformer/h.2/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather for ONNX node: /transformer/h.2/attn/Gather +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather [Gather] outputs: [/transformer/h.2/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_1 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_1 [Shape] inputs: [/transformer/h.2/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_1 for ONNX node: /transformer/h.2/attn/Shape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_1 [Shape] outputs: [/transformer/h.2/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_1 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_1 [Gather] inputs: [/transformer/h.2/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_1 for ONNX node: /transformer/h.2/attn/Gather_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_1 [Gather] outputs: [/transformer/h.2/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze for ONNX node: /transformer/h.2/attn/Unsqueeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_1 for ONNX node: /transformer/h.2/attn/Unsqueeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat for ONNX node: /transformer/h.2/attn/Concat +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat [Concat] outputs: [/transformer/h.2/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape [Reshape] inputs: [/transformer/h.2/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape for ONNX node: /transformer/h.2/attn/Reshape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape [Reshape] outputs: [/transformer/h.2/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_2 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_2 [Shape] inputs: [/transformer/h.2/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_2 for ONNX node: /transformer/h.2/attn/Shape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_2 [Shape] outputs: [/transformer/h.2/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_2 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_2 [Gather] inputs: [/transformer/h.2/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_2 for ONNX node: /transformer/h.2/attn/Gather_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_2 [Gather] outputs: [/transformer/h.2/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_3 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_3 [Shape] inputs: [/transformer/h.2/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_3 for ONNX node: /transformer/h.2/attn/Shape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_3 [Shape] outputs: [/transformer/h.2/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_3 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_3 [Gather] inputs: [/transformer/h.2/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_3 for ONNX node: /transformer/h.2/attn/Gather_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_3 [Gather] outputs: [/transformer/h.2/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_2 for ONNX node: /transformer/h.2/attn/Unsqueeze_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_3 for ONNX node: /transformer/h.2/attn/Unsqueeze_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_1 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_1 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_1 for ONNX node: /transformer/h.2/attn/Concat_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_1 [Concat] outputs: [/transformer/h.2/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_1 [Reshape] inputs: [/transformer/h.2/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_1 for ONNX node: /transformer/h.2/attn/Reshape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_1 [Reshape] outputs: [/transformer/h.2/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_4 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_4 [Shape] inputs: [/transformer/h.2/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_4 for ONNX node: /transformer/h.2/attn/Shape_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_4 [Shape] outputs: [/transformer/h.2/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_4 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_4 [Gather] inputs: [/transformer/h.2/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_4 for ONNX node: /transformer/h.2/attn/Gather_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_4 [Gather] outputs: [/transformer/h.2/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_5 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_5 [Shape] inputs: [/transformer/h.2/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_5 for ONNX node: /transformer/h.2/attn/Shape_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_5 [Shape] outputs: [/transformer/h.2/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_5 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_5 [Gather] inputs: [/transformer/h.2/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_5 for ONNX node: /transformer/h.2/attn/Gather_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_5 [Gather] outputs: [/transformer/h.2/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_4 for ONNX node: /transformer/h.2/attn/Unsqueeze_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_5 for ONNX node: /transformer/h.2/attn/Unsqueeze_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_2 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_2 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_2 for ONNX node: /transformer/h.2/attn/Concat_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_2 [Concat] outputs: [/transformer/h.2/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_2 [Reshape] inputs: [/transformer/h.2/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_2 for ONNX node: /transformer/h.2/attn/Reshape_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_2 [Reshape] outputs: [/transformer/h.2/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Transpose [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Transpose [Transpose] inputs: [/transformer/h.2/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Transpose for ONNX node: /transformer/h.2/attn/Transpose +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: present.2.value_358 for ONNX tensor: present.2.value +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Transpose [Transpose] outputs: [present.2.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_6 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_6 [Shape] inputs: [/transformer/h.2/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_6 for ONNX node: /transformer/h.2/attn/Shape_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_6 [Shape] outputs: [/transformer/h.2/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_6 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_6 [Gather] inputs: [/transformer/h.2/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_6 for ONNX node: /transformer/h.2/attn/Gather_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_6 [Gather] outputs: [/transformer/h.2/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice [Slice] inputs: [/transformer/h.2/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice for ONNX node: /transformer/h.2/attn/Slice +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice [Slice] outputs: [/transformer/h.2/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_1 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_1 [Slice] inputs: [/transformer/h.2/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_1 for ONNX node: /transformer/h.2/attn/Slice_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_1 [Slice] outputs: [/transformer/h.2/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_2 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_2 [Slice] inputs: [/transformer/h.2/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_2 for ONNX node: /transformer/h.2/attn/Slice_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_2 [Slice] outputs: [/transformer/h.2/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_3 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_3 [Slice] inputs: [/transformer/h.2/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_3 for ONNX node: /transformer/h.2/attn/Slice_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_3 [Slice] outputs: [/transformer/h.2/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_7 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_7 [Shape] inputs: [/transformer/h.2/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_7 for ONNX node: /transformer/h.2/attn/Shape_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_7 [Shape] outputs: [/transformer/h.2/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_7 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_7 [Gather] inputs: [/transformer/h.2/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_7 for ONNX node: /transformer/h.2/attn/Gather_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_7 [Gather] outputs: [/transformer/h.2/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Range [Range] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.2/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Range for ONNX node: /transformer/h.2/attn/Range +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Range_output_0 for ONNX tensor: /transformer/h.2/attn/Range_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Range [Range] outputs: [/transformer/h.2/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Cast_1 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Range_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cast_1 [Cast] inputs: [/transformer/h.2/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Cast_1 for ONNX node: /transformer/h.2/attn/Cast_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.2/attn/Cast_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cast_1 [Cast] outputs: [/transformer/h.2/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Cast_2 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cast_2 [Cast] inputs: [/transformer/h.2/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Cast_2 for ONNX node: /transformer/h.2/attn/Cast_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.2/attn/Cast_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cast_2 [Cast] outputs: [/transformer/h.2/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Div [Div] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Cast_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Cast_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Div [Div] inputs: [/transformer/h.2/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.2/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Div for ONNX node: /transformer/h.2/attn/Div +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Div_output_0 for ONNX tensor: /transformer/h.2/attn/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Div [Div] outputs: [/transformer/h.2/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Pow [Pow] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Div_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.2/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Pow for ONNX node: /transformer/h.2/attn/Pow +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Pow_output_0 for ONNX tensor: /transformer/h.2/attn/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Pow [Pow] outputs: [/transformer/h.2/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Pow_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.2/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reciprocal for ONNX node: /transformer/h.2/attn/Reciprocal +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.2/attn/Reciprocal_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.2/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Mul [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reciprocal_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul [Mul] inputs: [/transformer/h.2/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Mul for ONNX node: /transformer/h.2/attn/Mul +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Mul_output_0 for ONNX tensor: /transformer/h.2/attn/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul [Mul] outputs: [/transformer/h.2/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Cast_3 [Cast] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cast_3 [Cast] inputs: [/transformer/h.2/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Cast_3 for ONNX node: /transformer/h.2/attn/Cast_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.2/attn/Cast_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cast_3 [Cast] outputs: [/transformer/h.2/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Range_1 [Range] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Cast_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.2/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Range_1 for ONNX node: /transformer/h.2/attn/Range_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Range_1_output_0 for ONNX tensor: /transformer/h.2/attn/Range_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Range_1 [Range] outputs: [/transformer/h.2/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Einsum [Einsum] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Range_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Mul_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Einsum [Einsum] inputs: [/transformer/h.2/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.2/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Einsum for ONNX node: /transformer/h.2/attn/Einsum +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Einsum_output_0 for ONNX tensor: /transformer/h.2/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Einsum [Einsum] outputs: [/transformer/h.2/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Sin [Sin] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Sin [Sin] inputs: [/transformer/h.2/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Sin for ONNX node: /transformer/h.2/attn/Sin +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Sin_output_0 for ONNX tensor: /transformer/h.2/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Sin [Sin] outputs: [/transformer/h.2/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Cos [Cos] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Einsum_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cos [Cos] inputs: [/transformer/h.2/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Cos for ONNX node: /transformer/h.2/attn/Cos +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Cos_output_0 for ONNX tensor: /transformer/h.2/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Cos [Cos] outputs: [/transformer/h.2/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_8 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_8 [Shape] inputs: [/transformer/h.2/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_8 for ONNX node: /transformer/h.2/attn/Shape_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_8 [Shape] outputs: [/transformer/h.2/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_8 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_8 [Gather] inputs: [/transformer/h.2/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_8 for ONNX node: /transformer/h.2/attn/Gather_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_8 [Gather] outputs: [/transformer/h.2/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Sin_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_3 [Reshape] inputs: [/transformer/h.2/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_3 for ONNX node: /transformer/h.2/attn/Reshape_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_3 [Reshape] outputs: [/transformer/h.2/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/ConstantOfShape for ONNX node: /transformer/h.2/attn/ConstantOfShape +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.2/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.2/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Expand [Expand] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Expand [Expand] inputs: [/transformer/h.2/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.2/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Expand for ONNX node: /transformer/h.2/attn/Expand +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Expand_output_0 for ONNX tensor: /transformer/h.2/attn/Expand_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Expand [Expand] outputs: [/transformer/h.2/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Tile [Tile] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Expand_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Tile [Tile] inputs: [/transformer/h.2/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Tile for ONNX node: /transformer/h.2/attn/Tile +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Tile_output_0 for ONNX tensor: /transformer/h.2/attn/Tile_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Tile [Tile] outputs: [/transformer/h.2/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_6 for ONNX node: /transformer/h.2/attn/Unsqueeze_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_3 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_3 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_3 for ONNX node: /transformer/h.2/attn/Concat_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_3 [Concat] outputs: [/transformer/h.2/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Tile_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_4 [Reshape] inputs: [/transformer/h.2/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.2/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_4 for ONNX node: /transformer/h.2/attn/Reshape_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_4 [Reshape] outputs: [/transformer/h.2/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_9 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_9 [Shape] inputs: [/transformer/h.2/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_9 for ONNX node: /transformer/h.2/attn/Shape_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_9 [Shape] outputs: [/transformer/h.2/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_9 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_9 [Gather] inputs: [/transformer/h.2/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_9 for ONNX node: /transformer/h.2/attn/Gather_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_9 [Gather] outputs: [/transformer/h.2/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.2/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_7 for ONNX node: /transformer/h.2/attn/Unsqueeze_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_8 for ONNX node: /transformer/h.2/attn/Unsqueeze_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_4 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_4 [Slice] inputs: [/transformer/h.2/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_4 for ONNX node: /transformer/h.2/attn/Slice_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_4 [Slice] outputs: [/transformer/h.2/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.2/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_9 for ONNX node: /transformer/h.2/attn/Unsqueeze_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_10 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_10 [Shape] inputs: [/transformer/h.2/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_10 for ONNX node: /transformer/h.2/attn/Shape_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_10 [Shape] outputs: [/transformer/h.2/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_10 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_10 [Gather] inputs: [/transformer/h.2/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_10 for ONNX node: /transformer/h.2/attn/Gather_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_10 [Gather] outputs: [/transformer/h.2/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Cos_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_5 [Reshape] inputs: [/transformer/h.2/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_5 for ONNX node: /transformer/h.2/attn/Reshape_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_5 [Reshape] outputs: [/transformer/h.2/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/ConstantOfShape_1 for ONNX node: /transformer/h.2/attn/ConstantOfShape_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.2/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.2/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Expand_1 [Expand] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Expand_1 [Expand] inputs: [/transformer/h.2/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.2/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Expand_1 for ONNX node: /transformer/h.2/attn/Expand_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.2/attn/Expand_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Expand_1 [Expand] outputs: [/transformer/h.2/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Tile_1 [Tile] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Expand_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Tile_1 [Tile] inputs: [/transformer/h.2/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Tile_1 for ONNX node: /transformer/h.2/attn/Tile_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.2/attn/Tile_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Tile_1 [Tile] outputs: [/transformer/h.2/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_10 for ONNX node: /transformer/h.2/attn/Unsqueeze_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_4 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_4 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_4 for ONNX node: /transformer/h.2/attn/Concat_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_4 [Concat] outputs: [/transformer/h.2/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Tile_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_6 [Reshape] inputs: [/transformer/h.2/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.2/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_6 for ONNX node: /transformer/h.2/attn/Reshape_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_6 [Reshape] outputs: [/transformer/h.2/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.2/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_11 for ONNX node: /transformer/h.2/attn/Unsqueeze_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_12 for ONNX node: /transformer/h.2/attn/Unsqueeze_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_5 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_5 [Slice] inputs: [/transformer/h.2/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_5 for ONNX node: /transformer/h.2/attn/Slice_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_5 [Slice] outputs: [/transformer/h.2/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.2/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_13 for ONNX node: /transformer/h.2/attn/Unsqueeze_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Mul_1 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_1 [Mul] inputs: [/transformer/h.2/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Mul_1 for ONNX node: /transformer/h.2/attn/Mul_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.2/attn/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_1 [Mul] outputs: [/transformer/h.2/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_6 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_6 [Slice] inputs: [/transformer/h.2/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_6 for ONNX node: /transformer/h.2/attn/Slice_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_6 [Slice] outputs: [/transformer/h.2/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_7 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_7 [Slice] inputs: [/transformer/h.2/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_7 for ONNX node: /transformer/h.2/attn/Slice_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_7 [Slice] outputs: [/transformer/h.2/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Neg [Neg] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Neg [Neg] inputs: [/transformer/h.2/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Neg for ONNX node: /transformer/h.2/attn/Neg +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Neg_output_0 for ONNX tensor: /transformer/h.2/attn/Neg_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Neg [Neg] outputs: [/transformer/h.2/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Neg_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.2/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_14 for ONNX node: /transformer/h.2/attn/Unsqueeze_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.2/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_15 for ONNX node: /transformer/h.2/attn/Unsqueeze_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_5 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_5 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.2/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_5 for ONNX node: /transformer/h.2/attn/Concat_5 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_5 [Concat] outputs: [/transformer/h.2/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_11 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_11 [Shape] inputs: [/transformer/h.2/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_11 for ONNX node: /transformer/h.2/attn/Shape_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_11 [Shape] outputs: [/transformer/h.2/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_8 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_8 [Slice] inputs: [/transformer/h.2/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_8 for ONNX node: /transformer/h.2/attn/Slice_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_8 [Slice] outputs: [/transformer/h.2/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_6 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_6 [Concat] inputs: [/transformer/h.2/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_6 for ONNX node: /transformer/h.2/attn/Concat_6 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_6 [Concat] outputs: [/transformer/h.2/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_7 [Reshape] inputs: [/transformer/h.2/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.2/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_7 for ONNX node: /transformer/h.2/attn/Reshape_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_7 [Reshape] outputs: [/transformer/h.2/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Mul_2 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_2 [Mul] inputs: [/transformer/h.2/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Mul_2 for ONNX node: /transformer/h.2/attn/Mul_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.2/attn/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_2 [Mul] outputs: [/transformer/h.2/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Add [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Mul_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Mul_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Add [Add] inputs: [/transformer/h.2/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Add for ONNX node: /transformer/h.2/attn/Add +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Add_output_0 for ONNX tensor: /transformer/h.2/attn/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Add [Add] outputs: [/transformer/h.2/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_12 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_12 [Shape] inputs: [/transformer/h.2/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_12 for ONNX node: /transformer/h.2/attn/Shape_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_12 [Shape] outputs: [/transformer/h.2/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_11 [Gather] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_11 [Gather] inputs: [/transformer/h.2/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_11 for ONNX node: /transformer/h.2/attn/Gather_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Gather_11 [Gather] outputs: [/transformer/h.2/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_16 for ONNX node: /transformer/h.2/attn/Unsqueeze_16 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_9 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_9 [Slice] inputs: [/transformer/h.2/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_9 for ONNX node: /transformer/h.2/attn/Slice_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_9 [Slice] outputs: [/transformer/h.2/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.2/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_17 for ONNX node: /transformer/h.2/attn/Unsqueeze_17 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_18 for ONNX node: /transformer/h.2/attn/Unsqueeze_18 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_10 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_10 [Slice] inputs: [/transformer/h.2/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_10 for ONNX node: /transformer/h.2/attn/Slice_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_10 [Slice] outputs: [/transformer/h.2/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.2/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_19 for ONNX node: /transformer/h.2/attn/Unsqueeze_19 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Mul_3 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_3 [Mul] inputs: [/transformer/h.2/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Mul_3 for ONNX node: /transformer/h.2/attn/Mul_3 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.2/attn/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_3 [Mul] outputs: [/transformer/h.2/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_11 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_11 [Slice] inputs: [/transformer/h.2/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_11 for ONNX node: /transformer/h.2/attn/Slice_11 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_11 [Slice] outputs: [/transformer/h.2/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_12 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_12 [Slice] inputs: [/transformer/h.2/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_12 for ONNX node: /transformer/h.2/attn/Slice_12 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_12 [Slice] outputs: [/transformer/h.2/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Neg_1 [Neg] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_12_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Neg_1 [Neg] inputs: [/transformer/h.2/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Neg_1 for ONNX node: /transformer/h.2/attn/Neg_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.2/attn/Neg_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Neg_1 [Neg] outputs: [/transformer/h.2/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Neg_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.2/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_20 for ONNX node: /transformer/h.2/attn/Unsqueeze_20 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_11_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.2/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_21 for ONNX node: /transformer/h.2/attn/Unsqueeze_21 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_7 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_7 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.2/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_7 for ONNX node: /transformer/h.2/attn/Concat_7 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_7 [Concat] outputs: [/transformer/h.2/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_13 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_13 [Shape] inputs: [/transformer/h.2/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_13 for ONNX node: /transformer/h.2/attn/Shape_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_13 [Shape] outputs: [/transformer/h.2/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_13 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_13 [Slice] inputs: [/transformer/h.2/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_13 for ONNX node: /transformer/h.2/attn/Slice_13 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_13 [Slice] outputs: [/transformer/h.2/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_8 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_13_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_8 [Concat] inputs: [/transformer/h.2/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_8 for ONNX node: /transformer/h.2/attn/Concat_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_8 [Concat] outputs: [/transformer/h.2/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_7_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_8 [Reshape] inputs: [/transformer/h.2/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.2/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_8 for ONNX node: /transformer/h.2/attn/Reshape_8 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Reshape_8 [Reshape] outputs: [/transformer/h.2/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Mul_4 [Mul] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_8_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_4 [Mul] inputs: [/transformer/h.2/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Mul_4 for ONNX node: /transformer/h.2/attn/Mul_4 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.2/attn/Mul_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Mul_4 [Mul] outputs: [/transformer/h.2/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Add_1 [Add] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Mul_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Mul_4_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Add_1 [Add] inputs: [/transformer/h.2/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Add_1 for ONNX node: /transformer/h.2/attn/Add_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Add_1_output_0 for ONNX tensor: /transformer/h.2/attn/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Add_1 [Add] outputs: [/transformer/h.2/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_9 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Add_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_9 [Concat] inputs: [/transformer/h.2/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_9 for ONNX node: /transformer/h.2/attn/Concat_9 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_9 [Concat] outputs: [/transformer/h.2/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_10 [Concat] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Add_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_3_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_10 [Concat] inputs: [/transformer/h.2/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.2/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_10 for ONNX node: /transformer/h.2/attn/Concat_10 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Concat_10 [Concat] outputs: [/transformer/h.2/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_9_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Transpose_1 [Transpose] inputs: [/transformer/h.2/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Transpose_1 for ONNX node: /transformer/h.2/attn/Transpose_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: present.2.key_507 for ONNX tensor: present.2.key +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Transpose_1 [Transpose] outputs: [present.2.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_10_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Transpose_2 [Transpose] inputs: [/transformer/h.2/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Transpose_2 for ONNX node: /transformer/h.2/attn/Transpose_2 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.2/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Transpose_2 [Transpose] outputs: [/transformer/h.2/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_14 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Transpose_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_14 [Shape] inputs: [/transformer/h.2/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_14 for ONNX node: /transformer/h.2/attn/Shape_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_14 [Shape] outputs: [/transformer/h.2/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_14 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_14 [Slice] inputs: [/transformer/h.2/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_14 for ONNX node: /transformer/h.2/attn/Slice_14 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_14 [Slice] outputs: [/transformer/h.2/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Squeeze [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_14_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Squeeze [Squeeze] inputs: [/transformer/h.2/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Squeeze for ONNX node: /transformer/h.2/attn/Squeeze +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.2/attn/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Squeeze [Squeeze] outputs: [/transformer/h.2/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_15 [Shape] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: present.2.key +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_15 [Shape] inputs: [present.2.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_15 for ONNX node: /transformer/h.2/attn/Shape_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Shape_15 [Shape] outputs: [/transformer/h.2/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_15 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_15 [Slice] inputs: [/transformer/h.2/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_15 for ONNX node: /transformer/h.2/attn/Slice_15 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_15 [Slice] outputs: [/transformer/h.2/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_15_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.2/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Squeeze_1 for ONNX node: /transformer/h.2/attn/Squeeze_1 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.2/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.2/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Sub [Sub] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Squeeze_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Sub [Sub] inputs: [/transformer/h.2/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.2/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Sub for ONNX node: /transformer/h.2/attn/Sub +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Sub_output_0 for ONNX tensor: /transformer/h.2/attn/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Sub [Sub] outputs: [/transformer/h.2/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Sub_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.2/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_22 for ONNX node: /transformer/h.2/attn/Unsqueeze_22 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Squeeze_1_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.2/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:18] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_23 for ONNX node: /transformer/h.2/attn/Unsqueeze_23 +[12/30/2022-19:54:18] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:18] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_16 [Slice] +[12/30/2022-19:54:18] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:18] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:18] [V] [TRT] /transformer/h.2/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.2/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_16 for ONNX node: /transformer/h.2/attn/Slice_16 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Slice_16 [Slice] outputs: [/transformer/h.2/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Squeeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.2/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_24 for ONNX node: /transformer/h.2/attn/Unsqueeze_24 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Slice_17 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Slice_17 [Slice] inputs: [/transformer/h.2/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Slice_17 for ONNX node: /transformer/h.2/attn/Slice_17 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.2/attn/Slice_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Slice_17 [Slice] outputs: [/transformer/h.2/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Transpose_3 [Transpose] inputs: [/transformer/h.2/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Transpose_3 for ONNX node: /transformer/h.2/attn/Transpose_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.2/attn/Transpose_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Transpose_3 [Transpose] outputs: [/transformer/h.2/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Transpose_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Transpose_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/MatMul [MatMul] inputs: [/transformer/h.2/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.2/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/MatMul for ONNX node: /transformer/h.2/attn/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/MatMul_output_0 for ONNX tensor: /transformer/h.2/attn/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/MatMul [MatMul] outputs: [/transformer/h.2/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Where [Where] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Slice_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Where [Where] inputs: [/transformer/h.2/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.2/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Where for ONNX node: /transformer/h.2/attn/Where +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Where_output_0 for ONNX tensor: /transformer/h.2/attn/Where_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Where [Where] outputs: [/transformer/h.2/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Div_1 [Div] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Where_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Div_1 [Div] inputs: [/transformer/h.2/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Div_1 for ONNX node: /transformer/h.2/attn/Div_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Div_1_output_0 for ONNX tensor: /transformer/h.2/attn/Div_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Div_1 [Div] outputs: [/transformer/h.2/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Add_2 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Div_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Add_2 [Add] inputs: [/transformer/h.2/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Add_2 for ONNX node: /transformer/h.2/attn/Add_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Add_2_output_0 for ONNX tensor: /transformer/h.2/attn/Add_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Add_2 [Add] outputs: [/transformer/h.2/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Softmax [Softmax] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Add_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Softmax [Softmax] inputs: [/transformer/h.2/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Softmax for ONNX node: /transformer/h.2/attn/Softmax +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Softmax_output_0 for ONNX tensor: /transformer/h.2/attn/Softmax_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Softmax [Softmax] outputs: [/transformer/h.2/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Softmax_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: present.2.value +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/MatMul_1 [MatMul] inputs: [/transformer/h.2/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.2.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/MatMul_1 for ONNX node: /transformer/h.2/attn/MatMul_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.2/attn/MatMul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/MatMul_1 [MatMul] outputs: [/transformer/h.2/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/MatMul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Transpose_4 [Transpose] inputs: [/transformer/h.2/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Transpose_4 for ONNX node: /transformer/h.2/attn/Transpose_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.2/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Transpose_4 [Transpose] outputs: [/transformer/h.2/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_16 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Shape_16 [Shape] inputs: [/transformer/h.2/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_16 for ONNX node: /transformer/h.2/attn/Shape_16 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Shape_16 [Shape] outputs: [/transformer/h.2/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_12 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Gather_12 [Gather] inputs: [/transformer/h.2/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_12 for ONNX node: /transformer/h.2/attn/Gather_12 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Gather_12 [Gather] outputs: [/transformer/h.2/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Shape_17 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Shape_17 [Shape] inputs: [/transformer/h.2/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Shape_17 for ONNX node: /transformer/h.2/attn/Shape_17 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.2/attn/Shape_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Shape_17 [Shape] outputs: [/transformer/h.2/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Gather_13 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Shape_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Gather_13 [Gather] inputs: [/transformer/h.2/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Gather_13 for ONNX node: /transformer/h.2/attn/Gather_13 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.2/attn/Gather_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Gather_13 [Gather] outputs: [/transformer/h.2/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_25 for ONNX node: /transformer/h.2/attn/Unsqueeze_25 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Gather_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.2/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Unsqueeze_26 for ONNX node: /transformer/h.2/attn/Unsqueeze_26 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.2/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.2/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Concat_11 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Concat_11 [Concat] inputs: [/transformer/h.2/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.2/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Concat_11 for ONNX node: /transformer/h.2/attn/Concat_11 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.2/attn/Concat_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Concat_11 [Concat] outputs: [/transformer/h.2/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Concat_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Reshape_9 [Reshape] inputs: [/transformer/h.2/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.2/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/Reshape_9 for ONNX node: /transformer/h.2/attn/Reshape_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.2/attn/Reshape_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/Reshape_9 [Reshape] outputs: [/transformer/h.2/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/Reshape_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2911 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.2/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2911 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2911 for ONNX node: onnx::MatMul_2911 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/attn/out_proj/MatMul for ONNX node: /transformer/h.2/attn/out_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.2/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.2/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2912 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.2/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2912 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2912 for ONNX node: onnx::MatMul_2912 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/fc_in/MatMul for ONNX node: /transformer/h.2/mlp/fc_in/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.2/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.2/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/fc_in/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.2/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/fc_in/Add for ONNX node: /transformer/h.2/mlp/fc_in/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.2/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_in/Add [Add] outputs: [/transformer/h.2/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Mul [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul [Mul] inputs: [/transformer/h.2/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Mul for ONNX node: /transformer/h.2/mlp/act/Mul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul [Mul] outputs: [/transformer/h.2/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Pow [Pow] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Pow [Pow] inputs: [/transformer/h.2/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Pow for ONNX node: /transformer/h.2/mlp/act/Pow +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Pow [Pow] outputs: [/transformer/h.2/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.2/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Mul_1 for ONNX node: /transformer/h.2/mlp/act/Mul_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.2/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Add [Add] inputs: [/transformer/h.2/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.2/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Add for ONNX node: /transformer/h.2/mlp/act/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Add [Add] outputs: [/transformer/h.2/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.2/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Mul_2 for ONNX node: /transformer/h.2/mlp/act/Mul_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.2/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Tanh [Tanh] inputs: [/transformer/h.2/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Tanh for ONNX node: /transformer/h.2/mlp/act/Tanh +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Tanh_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Tanh [Tanh] outputs: [/transformer/h.2/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Tanh_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Add_1 [Add] inputs: [/transformer/h.2/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Add_1 for ONNX node: /transformer/h.2/mlp/act/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Add_1 [Add] outputs: [/transformer/h.2/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.2/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.2/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/act/Mul_3 for ONNX node: /transformer/h.2/mlp/act/Mul_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.2/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.2/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2913 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.2/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_2913 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2913 for ONNX node: onnx::MatMul_2913 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/fc_out/MatMul for ONNX node: /transformer/h.2/mlp/fc_out/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.2/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.2/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/mlp/fc_out/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.2/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/mlp/fc_out/Add for ONNX node: /transformer/h.2/mlp/fc_out/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.2/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/mlp/fc_out/Add [Add] outputs: [/transformer/h.2/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/Add [Add] inputs: [/transformer/h.2/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/Add for ONNX node: /transformer/h.2/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/Add_output_0 for ONNX tensor: /transformer/h.2/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/Add [Add] outputs: [/transformer/h.2/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.2/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/Add_1 [Add] inputs: [/transformer/h.2/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.2/Add_1 for ONNX node: /transformer/h.2/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.2/Add_1_output_0 for ONNX tensor: /transformer/h.2/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.2/Add_1 [Add] outputs: [/transformer/h.2/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.2/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/ReduceMean for ONNX node: /transformer/h.3/ln_1/ReduceMean +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.3/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.3/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Sub [Sub] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Sub [Sub] inputs: [/transformer/h.2/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Sub for ONNX node: /transformer/h.3/ln_1/Sub +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.3/ln_1/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Sub [Sub] outputs: [/transformer/h.3/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Pow [Pow] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Pow [Pow] inputs: [/transformer/h.3/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Pow for ONNX node: /transformer/h.3/ln_1/Pow +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.3/ln_1/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Pow [Pow] outputs: [/transformer/h.3/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.3/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/ReduceMean_1 for ONNX node: /transformer/h.3/ln_1/ReduceMean_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.3/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.3/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Add [Add] inputs: [/transformer/h.3/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Add for ONNX node: /transformer/h.3/ln_1/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Add_output_0 for ONNX tensor: /transformer/h.3/ln_1/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Add [Add] outputs: [/transformer/h.3/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.3/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Sqrt for ONNX node: /transformer/h.3/ln_1/Sqrt +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.3/ln_1/Sqrt_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.3/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Div [Div] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Sqrt_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Div [Div] inputs: [/transformer/h.3/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Div for ONNX node: /transformer/h.3/ln_1/Div +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Div_output_0 for ONNX tensor: /transformer/h.3/ln_1/Div_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Div [Div] outputs: [/transformer/h.3/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Mul [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Div_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Mul [Mul] inputs: [/transformer/h.3/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Mul for ONNX node: /transformer/h.3/ln_1/Mul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.3/ln_1/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Mul [Mul] outputs: [/transformer/h.3/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/ln_1/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Add_1 [Add] inputs: [/transformer/h.3/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/ln_1/Add_1 for ONNX node: /transformer/h.3/ln_1/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.3/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/ln_1/Add_1 [Add] outputs: [/transformer/h.3/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2914 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.3/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2914 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2914 for ONNX node: onnx::MatMul_2914 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/q_proj/MatMul for ONNX node: /transformer/h.3/attn/q_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.3/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.3/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2915 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.3/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2915 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2915 for ONNX node: onnx::MatMul_2915 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/k_proj/MatMul for ONNX node: /transformer/h.3/attn/k_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.3/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.3/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2916 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.3/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2916 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2916 for ONNX node: onnx::MatMul_2916 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/v_proj/MatMul for ONNX node: /transformer/h.3/attn/v_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.3/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.3/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape [Shape] inputs: [/transformer/h.3/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape for ONNX node: /transformer/h.3/attn/Shape +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape [Shape] outputs: [/transformer/h.3/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather [Gather] inputs: [/transformer/h.3/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather for ONNX node: /transformer/h.3/attn/Gather +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather [Gather] outputs: [/transformer/h.3/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_1 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_1 [Shape] inputs: [/transformer/h.3/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_1 for ONNX node: /transformer/h.3/attn/Shape_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_1 [Shape] outputs: [/transformer/h.3/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_1 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_1 [Gather] inputs: [/transformer/h.3/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_1 for ONNX node: /transformer/h.3/attn/Gather_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_1 [Gather] outputs: [/transformer/h.3/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze for ONNX node: /transformer/h.3/attn/Unsqueeze +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_1 for ONNX node: /transformer/h.3/attn/Unsqueeze_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat for ONNX node: /transformer/h.3/attn/Concat +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat [Concat] outputs: [/transformer/h.3/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape [Reshape] inputs: [/transformer/h.3/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape for ONNX node: /transformer/h.3/attn/Reshape +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape [Reshape] outputs: [/transformer/h.3/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_2 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_2 [Shape] inputs: [/transformer/h.3/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_2 for ONNX node: /transformer/h.3/attn/Shape_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_2 [Shape] outputs: [/transformer/h.3/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_2 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_2 [Gather] inputs: [/transformer/h.3/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_2 for ONNX node: /transformer/h.3/attn/Gather_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_2 [Gather] outputs: [/transformer/h.3/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_3 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_3 [Shape] inputs: [/transformer/h.3/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_3 for ONNX node: /transformer/h.3/attn/Shape_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_3 [Shape] outputs: [/transformer/h.3/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_3 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_3 [Gather] inputs: [/transformer/h.3/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_3 for ONNX node: /transformer/h.3/attn/Gather_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_3 [Gather] outputs: [/transformer/h.3/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_2 for ONNX node: /transformer/h.3/attn/Unsqueeze_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_3 for ONNX node: /transformer/h.3/attn/Unsqueeze_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_1 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_1 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_1 for ONNX node: /transformer/h.3/attn/Concat_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_1 [Concat] outputs: [/transformer/h.3/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_1 [Reshape] inputs: [/transformer/h.3/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_1 for ONNX node: /transformer/h.3/attn/Reshape_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_1 [Reshape] outputs: [/transformer/h.3/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_4 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_4 [Shape] inputs: [/transformer/h.3/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_4 for ONNX node: /transformer/h.3/attn/Shape_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_4 [Shape] outputs: [/transformer/h.3/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_4 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_4 [Gather] inputs: [/transformer/h.3/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_4 for ONNX node: /transformer/h.3/attn/Gather_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_4 [Gather] outputs: [/transformer/h.3/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_5 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_5 [Shape] inputs: [/transformer/h.3/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_5 for ONNX node: /transformer/h.3/attn/Shape_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_5 [Shape] outputs: [/transformer/h.3/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_5 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_5 [Gather] inputs: [/transformer/h.3/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_5 for ONNX node: /transformer/h.3/attn/Gather_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_5 [Gather] outputs: [/transformer/h.3/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_4 for ONNX node: /transformer/h.3/attn/Unsqueeze_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_5 for ONNX node: /transformer/h.3/attn/Unsqueeze_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_2 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_2 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_2 for ONNX node: /transformer/h.3/attn/Concat_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_2 [Concat] outputs: [/transformer/h.3/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_2 [Reshape] inputs: [/transformer/h.3/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_2 for ONNX node: /transformer/h.3/attn/Reshape_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_2 [Reshape] outputs: [/transformer/h.3/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Transpose [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose [Transpose] inputs: [/transformer/h.3/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Transpose for ONNX node: /transformer/h.3/attn/Transpose +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: present.3.value_527 for ONNX tensor: present.3.value +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose [Transpose] outputs: [present.3.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_6 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_6 [Shape] inputs: [/transformer/h.3/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_6 for ONNX node: /transformer/h.3/attn/Shape_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_6 [Shape] outputs: [/transformer/h.3/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_6 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_6 [Gather] inputs: [/transformer/h.3/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_6 for ONNX node: /transformer/h.3/attn/Gather_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_6 [Gather] outputs: [/transformer/h.3/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice [Slice] inputs: [/transformer/h.3/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice for ONNX node: /transformer/h.3/attn/Slice +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice [Slice] outputs: [/transformer/h.3/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_1 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_1 [Slice] inputs: [/transformer/h.3/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_1 for ONNX node: /transformer/h.3/attn/Slice_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_1 [Slice] outputs: [/transformer/h.3/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_2 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_2 [Slice] inputs: [/transformer/h.3/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_2 for ONNX node: /transformer/h.3/attn/Slice_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_2 [Slice] outputs: [/transformer/h.3/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_3 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_3 [Slice] inputs: [/transformer/h.3/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_3 for ONNX node: /transformer/h.3/attn/Slice_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_3 [Slice] outputs: [/transformer/h.3/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_7 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_7 [Shape] inputs: [/transformer/h.3/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_7 for ONNX node: /transformer/h.3/attn/Shape_7 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_7 [Shape] outputs: [/transformer/h.3/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_7 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_7 [Gather] inputs: [/transformer/h.3/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_7 for ONNX node: /transformer/h.3/attn/Gather_7 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_7 [Gather] outputs: [/transformer/h.3/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Range [Range] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.3/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Range for ONNX node: /transformer/h.3/attn/Range +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Range_output_0 for ONNX tensor: /transformer/h.3/attn/Range_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Range [Range] outputs: [/transformer/h.3/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Cast_1 [Cast] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Range_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cast_1 [Cast] inputs: [/transformer/h.3/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Cast_1 for ONNX node: /transformer/h.3/attn/Cast_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.3/attn/Cast_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cast_1 [Cast] outputs: [/transformer/h.3/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Cast_2 [Cast] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cast_2 [Cast] inputs: [/transformer/h.3/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Cast_2 for ONNX node: /transformer/h.3/attn/Cast_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.3/attn/Cast_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cast_2 [Cast] outputs: [/transformer/h.3/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Div [Div] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Cast_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Cast_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Div [Div] inputs: [/transformer/h.3/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.3/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Div for ONNX node: /transformer/h.3/attn/Div +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Div_output_0 for ONNX tensor: /transformer/h.3/attn/Div_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Div [Div] outputs: [/transformer/h.3/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Pow [Pow] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Div_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.3/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Pow for ONNX node: /transformer/h.3/attn/Pow +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Pow_output_0 for ONNX tensor: /transformer/h.3/attn/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Pow [Pow] outputs: [/transformer/h.3/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.3/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reciprocal for ONNX node: /transformer/h.3/attn/Reciprocal +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.3/attn/Reciprocal_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.3/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Mul [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reciprocal_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul [Mul] inputs: [/transformer/h.3/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Mul for ONNX node: /transformer/h.3/attn/Mul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Mul_output_0 for ONNX tensor: /transformer/h.3/attn/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul [Mul] outputs: [/transformer/h.3/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Cast_3 [Cast] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cast_3 [Cast] inputs: [/transformer/h.3/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Cast_3 for ONNX node: /transformer/h.3/attn/Cast_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.3/attn/Cast_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cast_3 [Cast] outputs: [/transformer/h.3/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Range_1 [Range] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Cast_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.3/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Range_1 for ONNX node: /transformer/h.3/attn/Range_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Range_1_output_0 for ONNX tensor: /transformer/h.3/attn/Range_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Range_1 [Range] outputs: [/transformer/h.3/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Einsum [Einsum] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Range_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Einsum [Einsum] inputs: [/transformer/h.3/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.3/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Einsum for ONNX node: /transformer/h.3/attn/Einsum +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Einsum_output_0 for ONNX tensor: /transformer/h.3/attn/Einsum_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Einsum [Einsum] outputs: [/transformer/h.3/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Sin [Sin] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Einsum_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Sin [Sin] inputs: [/transformer/h.3/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Sin for ONNX node: /transformer/h.3/attn/Sin +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Sin_output_0 for ONNX tensor: /transformer/h.3/attn/Sin_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Sin [Sin] outputs: [/transformer/h.3/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Cos [Cos] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Einsum_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cos [Cos] inputs: [/transformer/h.3/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Cos for ONNX node: /transformer/h.3/attn/Cos +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Cos_output_0 for ONNX tensor: /transformer/h.3/attn/Cos_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Cos [Cos] outputs: [/transformer/h.3/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_8 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Sin_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_8 [Shape] inputs: [/transformer/h.3/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_8 for ONNX node: /transformer/h.3/attn/Shape_8 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_8 [Shape] outputs: [/transformer/h.3/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_8 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_8 [Gather] inputs: [/transformer/h.3/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_8 for ONNX node: /transformer/h.3/attn/Gather_8 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_8 [Gather] outputs: [/transformer/h.3/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Sin_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_3 [Reshape] inputs: [/transformer/h.3/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_3 for ONNX node: /transformer/h.3/attn/Reshape_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_3 [Reshape] outputs: [/transformer/h.3/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/ConstantOfShape for ONNX node: /transformer/h.3/attn/ConstantOfShape +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.3/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.3/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Expand [Expand] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Expand [Expand] inputs: [/transformer/h.3/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.3/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Expand for ONNX node: /transformer/h.3/attn/Expand +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Expand_output_0 for ONNX tensor: /transformer/h.3/attn/Expand_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Expand [Expand] outputs: [/transformer/h.3/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Tile [Tile] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Expand_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Tile [Tile] inputs: [/transformer/h.3/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Tile for ONNX node: /transformer/h.3/attn/Tile +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Tile_output_0 for ONNX tensor: /transformer/h.3/attn/Tile_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Tile [Tile] outputs: [/transformer/h.3/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_6 for ONNX node: /transformer/h.3/attn/Unsqueeze_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_3 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_3 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_3 for ONNX node: /transformer/h.3/attn/Concat_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_3 [Concat] outputs: [/transformer/h.3/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Tile_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_4 [Reshape] inputs: [/transformer/h.3/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.3/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_4 for ONNX node: /transformer/h.3/attn/Reshape_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_4 [Reshape] outputs: [/transformer/h.3/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_9 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_9 [Shape] inputs: [/transformer/h.3/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_9 for ONNX node: /transformer/h.3/attn/Shape_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_9 [Shape] outputs: [/transformer/h.3/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_9 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_9 [Gather] inputs: [/transformer/h.3/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_9 for ONNX node: /transformer/h.3/attn/Gather_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_9 [Gather] outputs: [/transformer/h.3/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.3/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_7 for ONNX node: /transformer/h.3/attn/Unsqueeze_7 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_8 for ONNX node: /transformer/h.3/attn/Unsqueeze_8 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_4 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_4 [Slice] inputs: [/transformer/h.3/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_4 for ONNX node: /transformer/h.3/attn/Slice_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_4 [Slice] outputs: [/transformer/h.3/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.3/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_9 for ONNX node: /transformer/h.3/attn/Unsqueeze_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_10 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Cos_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_10 [Shape] inputs: [/transformer/h.3/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_10 for ONNX node: /transformer/h.3/attn/Shape_10 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_10 [Shape] outputs: [/transformer/h.3/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_10 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_10 [Gather] inputs: [/transformer/h.3/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_10 for ONNX node: /transformer/h.3/attn/Gather_10 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_10 [Gather] outputs: [/transformer/h.3/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Cos_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_5 [Reshape] inputs: [/transformer/h.3/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_5 for ONNX node: /transformer/h.3/attn/Reshape_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_5 [Reshape] outputs: [/transformer/h.3/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/ConstantOfShape_1 for ONNX node: /transformer/h.3/attn/ConstantOfShape_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.3/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.3/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Expand_1 [Expand] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Expand_1 [Expand] inputs: [/transformer/h.3/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.3/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Expand_1 for ONNX node: /transformer/h.3/attn/Expand_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.3/attn/Expand_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Expand_1 [Expand] outputs: [/transformer/h.3/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Tile_1 [Tile] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Expand_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Tile_1 [Tile] inputs: [/transformer/h.3/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Tile_1 for ONNX node: /transformer/h.3/attn/Tile_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.3/attn/Tile_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Tile_1 [Tile] outputs: [/transformer/h.3/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_10 for ONNX node: /transformer/h.3/attn/Unsqueeze_10 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_4 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_4 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_4 for ONNX node: /transformer/h.3/attn/Concat_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_4 [Concat] outputs: [/transformer/h.3/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Tile_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_6 [Reshape] inputs: [/transformer/h.3/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.3/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_6 for ONNX node: /transformer/h.3/attn/Reshape_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_6 [Reshape] outputs: [/transformer/h.3/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.3/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_11 for ONNX node: /transformer/h.3/attn/Unsqueeze_11 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_12 for ONNX node: /transformer/h.3/attn/Unsqueeze_12 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_5 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_5 [Slice] inputs: [/transformer/h.3/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_5 for ONNX node: /transformer/h.3/attn/Slice_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_5 [Slice] outputs: [/transformer/h.3/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.3/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_13 for ONNX node: /transformer/h.3/attn/Unsqueeze_13 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Mul_1 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_1 [Mul] inputs: [/transformer/h.3/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Mul_1 for ONNX node: /transformer/h.3/attn/Mul_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.3/attn/Mul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_1 [Mul] outputs: [/transformer/h.3/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_6 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_6 [Slice] inputs: [/transformer/h.3/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_6 for ONNX node: /transformer/h.3/attn/Slice_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_6 [Slice] outputs: [/transformer/h.3/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_7 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_7 [Slice] inputs: [/transformer/h.3/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_7 for ONNX node: /transformer/h.3/attn/Slice_7 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_7 [Slice] outputs: [/transformer/h.3/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Neg [Neg] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Neg [Neg] inputs: [/transformer/h.3/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Neg for ONNX node: /transformer/h.3/attn/Neg +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Neg_output_0 for ONNX tensor: /transformer/h.3/attn/Neg_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Neg [Neg] outputs: [/transformer/h.3/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Neg_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.3/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_14 for ONNX node: /transformer/h.3/attn/Unsqueeze_14 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.3/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_15 for ONNX node: /transformer/h.3/attn/Unsqueeze_15 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_5 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_5 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.3/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_5 for ONNX node: /transformer/h.3/attn/Concat_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_5 [Concat] outputs: [/transformer/h.3/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_11 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_11 [Shape] inputs: [/transformer/h.3/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_11 for ONNX node: /transformer/h.3/attn/Shape_11 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_11 [Shape] outputs: [/transformer/h.3/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_8 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_8 [Slice] inputs: [/transformer/h.3/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_8 for ONNX node: /transformer/h.3/attn/Slice_8 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_8 [Slice] outputs: [/transformer/h.3/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_6 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_6 [Concat] inputs: [/transformer/h.3/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_6 for ONNX node: /transformer/h.3/attn/Concat_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_6 [Concat] outputs: [/transformer/h.3/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_7 [Reshape] inputs: [/transformer/h.3/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.3/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_7 for ONNX node: /transformer/h.3/attn/Reshape_7 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_7 [Reshape] outputs: [/transformer/h.3/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Mul_2 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_2 [Mul] inputs: [/transformer/h.3/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Mul_2 for ONNX node: /transformer/h.3/attn/Mul_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.3/attn/Mul_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_2 [Mul] outputs: [/transformer/h.3/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Mul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Mul_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Add [Add] inputs: [/transformer/h.3/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Add for ONNX node: /transformer/h.3/attn/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Add_output_0 for ONNX tensor: /transformer/h.3/attn/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Add [Add] outputs: [/transformer/h.3/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_12 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_12 [Shape] inputs: [/transformer/h.3/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_12 for ONNX node: /transformer/h.3/attn/Shape_12 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_12 [Shape] outputs: [/transformer/h.3/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_11 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_11 [Gather] inputs: [/transformer/h.3/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_11 for ONNX node: /transformer/h.3/attn/Gather_11 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_11 [Gather] outputs: [/transformer/h.3/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_16 for ONNX node: /transformer/h.3/attn/Unsqueeze_16 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_9 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_9 [Slice] inputs: [/transformer/h.3/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_9 for ONNX node: /transformer/h.3/attn/Slice_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_9 [Slice] outputs: [/transformer/h.3/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.3/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_17 for ONNX node: /transformer/h.3/attn/Unsqueeze_17 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_18 for ONNX node: /transformer/h.3/attn/Unsqueeze_18 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_10 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_10 [Slice] inputs: [/transformer/h.3/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_10 for ONNX node: /transformer/h.3/attn/Slice_10 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_10 [Slice] outputs: [/transformer/h.3/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.3/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_19 for ONNX node: /transformer/h.3/attn/Unsqueeze_19 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Mul_3 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_3 [Mul] inputs: [/transformer/h.3/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Mul_3 for ONNX node: /transformer/h.3/attn/Mul_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.3/attn/Mul_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_3 [Mul] outputs: [/transformer/h.3/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_11 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_11 [Slice] inputs: [/transformer/h.3/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_11 for ONNX node: /transformer/h.3/attn/Slice_11 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_11 [Slice] outputs: [/transformer/h.3/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_12 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_12 [Slice] inputs: [/transformer/h.3/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_12 for ONNX node: /transformer/h.3/attn/Slice_12 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_12 [Slice] outputs: [/transformer/h.3/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Neg_1 [Neg] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Neg_1 [Neg] inputs: [/transformer/h.3/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Neg_1 for ONNX node: /transformer/h.3/attn/Neg_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.3/attn/Neg_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Neg_1 [Neg] outputs: [/transformer/h.3/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Neg_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.3/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_20 for ONNX node: /transformer/h.3/attn/Unsqueeze_20 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.3/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_21 for ONNX node: /transformer/h.3/attn/Unsqueeze_21 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_7 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_7 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.3/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_7 for ONNX node: /transformer/h.3/attn/Concat_7 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_7 [Concat] outputs: [/transformer/h.3/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_13 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_13 [Shape] inputs: [/transformer/h.3/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_13 for ONNX node: /transformer/h.3/attn/Shape_13 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_13 [Shape] outputs: [/transformer/h.3/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_13 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_13 [Slice] inputs: [/transformer/h.3/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_13 for ONNX node: /transformer/h.3/attn/Slice_13 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_13 [Slice] outputs: [/transformer/h.3/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_8 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_8 [Concat] inputs: [/transformer/h.3/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_8 for ONNX node: /transformer/h.3/attn/Concat_8 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_8 [Concat] outputs: [/transformer/h.3/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_8 [Reshape] inputs: [/transformer/h.3/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.3/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_8 for ONNX node: /transformer/h.3/attn/Reshape_8 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_8 [Reshape] outputs: [/transformer/h.3/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Mul_4 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_4 [Mul] inputs: [/transformer/h.3/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Mul_4 for ONNX node: /transformer/h.3/attn/Mul_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.3/attn/Mul_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Mul_4 [Mul] outputs: [/transformer/h.3/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Mul_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Mul_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Add_1 [Add] inputs: [/transformer/h.3/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Add_1 for ONNX node: /transformer/h.3/attn/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Add_1_output_0 for ONNX tensor: /transformer/h.3/attn/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Add_1 [Add] outputs: [/transformer/h.3/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_9 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_9 [Concat] inputs: [/transformer/h.3/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_9 for ONNX node: /transformer/h.3/attn/Concat_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_9 [Concat] outputs: [/transformer/h.3/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_10 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_10 [Concat] inputs: [/transformer/h.3/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.3/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_10 for ONNX node: /transformer/h.3/attn/Concat_10 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_10 [Concat] outputs: [/transformer/h.3/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_1 [Transpose] inputs: [/transformer/h.3/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Transpose_1 for ONNX node: /transformer/h.3/attn/Transpose_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: present.3.key_676 for ONNX tensor: present.3.key +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_1 [Transpose] outputs: [present.3.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_10_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_2 [Transpose] inputs: [/transformer/h.3/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Transpose_2 for ONNX node: /transformer/h.3/attn/Transpose_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.3/attn/Transpose_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_2 [Transpose] outputs: [/transformer/h.3/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_14 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Transpose_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_14 [Shape] inputs: [/transformer/h.3/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_14 for ONNX node: /transformer/h.3/attn/Shape_14 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_14_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_14 [Shape] outputs: [/transformer/h.3/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_14 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_14_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_14 [Slice] inputs: [/transformer/h.3/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_14 for ONNX node: /transformer/h.3/attn/Slice_14 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_14_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_14 [Slice] outputs: [/transformer/h.3/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Squeeze [Squeeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_14_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Squeeze [Squeeze] inputs: [/transformer/h.3/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Squeeze for ONNX node: /transformer/h.3/attn/Squeeze +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.3/attn/Squeeze_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Squeeze [Squeeze] outputs: [/transformer/h.3/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_15 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: present.3.key +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_15 [Shape] inputs: [present.3.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_15 for ONNX node: /transformer/h.3/attn/Shape_15 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_15 [Shape] outputs: [/transformer/h.3/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_15 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_15 [Slice] inputs: [/transformer/h.3/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_15 for ONNX node: /transformer/h.3/attn/Slice_15 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_15 [Slice] outputs: [/transformer/h.3/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.3/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Squeeze_1 for ONNX node: /transformer/h.3/attn/Squeeze_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.3/attn/Squeeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.3/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Sub [Sub] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Squeeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Squeeze_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Sub [Sub] inputs: [/transformer/h.3/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.3/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Sub for ONNX node: /transformer/h.3/attn/Sub +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Sub_output_0 for ONNX tensor: /transformer/h.3/attn/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Sub [Sub] outputs: [/transformer/h.3/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.3/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_22 for ONNX node: /transformer/h.3/attn/Unsqueeze_22 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Squeeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.3/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_23 for ONNX node: /transformer/h.3/attn/Unsqueeze_23 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_16 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.3/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_16 for ONNX node: /transformer/h.3/attn/Slice_16 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_16 [Slice] outputs: [/transformer/h.3/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Squeeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.3/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_24 for ONNX node: /transformer/h.3/attn/Unsqueeze_24 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Slice_17 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_17 [Slice] inputs: [/transformer/h.3/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Slice_17 for ONNX node: /transformer/h.3/attn/Slice_17 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.3/attn/Slice_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Slice_17 [Slice] outputs: [/transformer/h.3/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_3 [Transpose] inputs: [/transformer/h.3/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Transpose_3 for ONNX node: /transformer/h.3/attn/Transpose_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.3/attn/Transpose_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_3 [Transpose] outputs: [/transformer/h.3/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Transpose_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Transpose_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/MatMul [MatMul] inputs: [/transformer/h.3/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.3/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/MatMul for ONNX node: /transformer/h.3/attn/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/MatMul_output_0 for ONNX tensor: /transformer/h.3/attn/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/MatMul [MatMul] outputs: [/transformer/h.3/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Where [Where] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Slice_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Where [Where] inputs: [/transformer/h.3/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.3/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Where for ONNX node: /transformer/h.3/attn/Where +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Where_output_0 for ONNX tensor: /transformer/h.3/attn/Where_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Where [Where] outputs: [/transformer/h.3/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Div_1 [Div] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Where_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Div_1 [Div] inputs: [/transformer/h.3/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Div_1 for ONNX node: /transformer/h.3/attn/Div_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Div_1_output_0 for ONNX tensor: /transformer/h.3/attn/Div_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Div_1 [Div] outputs: [/transformer/h.3/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Add_2 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Div_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Add_2 [Add] inputs: [/transformer/h.3/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Add_2 for ONNX node: /transformer/h.3/attn/Add_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Add_2_output_0 for ONNX tensor: /transformer/h.3/attn/Add_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Add_2 [Add] outputs: [/transformer/h.3/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Softmax [Softmax] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Add_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Softmax [Softmax] inputs: [/transformer/h.3/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Softmax for ONNX node: /transformer/h.3/attn/Softmax +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Softmax_output_0 for ONNX tensor: /transformer/h.3/attn/Softmax_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Softmax [Softmax] outputs: [/transformer/h.3/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Softmax_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: present.3.value +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/MatMul_1 [MatMul] inputs: [/transformer/h.3/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.3.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/MatMul_1 for ONNX node: /transformer/h.3/attn/MatMul_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.3/attn/MatMul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/MatMul_1 [MatMul] outputs: [/transformer/h.3/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/MatMul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_4 [Transpose] inputs: [/transformer/h.3/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Transpose_4 for ONNX node: /transformer/h.3/attn/Transpose_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.3/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Transpose_4 [Transpose] outputs: [/transformer/h.3/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_16 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_16 [Shape] inputs: [/transformer/h.3/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_16 for ONNX node: /transformer/h.3/attn/Shape_16 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_16 [Shape] outputs: [/transformer/h.3/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_12 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_16_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_12 [Gather] inputs: [/transformer/h.3/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_12 for ONNX node: /transformer/h.3/attn/Gather_12 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_12 [Gather] outputs: [/transformer/h.3/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Shape_17 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_17 [Shape] inputs: [/transformer/h.3/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Shape_17 for ONNX node: /transformer/h.3/attn/Shape_17 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.3/attn/Shape_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Shape_17 [Shape] outputs: [/transformer/h.3/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Gather_13 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Shape_17_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_13 [Gather] inputs: [/transformer/h.3/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Gather_13 for ONNX node: /transformer/h.3/attn/Gather_13 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.3/attn/Gather_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Gather_13 [Gather] outputs: [/transformer/h.3/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_12_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_25 for ONNX node: /transformer/h.3/attn/Unsqueeze_25 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Gather_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.3/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Unsqueeze_26 for ONNX node: /transformer/h.3/attn/Unsqueeze_26 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.3/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.3/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Concat_11 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_11 [Concat] inputs: [/transformer/h.3/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.3/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Concat_11 for ONNX node: /transformer/h.3/attn/Concat_11 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.3/attn/Concat_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Concat_11 [Concat] outputs: [/transformer/h.3/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Transpose_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Concat_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_9 [Reshape] inputs: [/transformer/h.3/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.3/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/Reshape_9 for ONNX node: /transformer/h.3/attn/Reshape_9 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.3/attn/Reshape_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/Reshape_9 [Reshape] outputs: [/transformer/h.3/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/Reshape_9_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2941 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.3/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2941 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2941 for ONNX node: onnx::MatMul_2941 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/attn/out_proj/MatMul for ONNX node: /transformer/h.3/attn/out_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.3/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.3/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2942 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.3/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2942 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2942 for ONNX node: onnx::MatMul_2942 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/fc_in/MatMul for ONNX node: /transformer/h.3/mlp/fc_in/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.3/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.3/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/fc_in/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.3/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/fc_in/Add for ONNX node: /transformer/h.3/mlp/fc_in/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.3/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_in/Add [Add] outputs: [/transformer/h.3/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Mul [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul [Mul] inputs: [/transformer/h.3/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Mul for ONNX node: /transformer/h.3/mlp/act/Mul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul [Mul] outputs: [/transformer/h.3/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Pow [Pow] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Pow [Pow] inputs: [/transformer/h.3/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Pow for ONNX node: /transformer/h.3/mlp/act/Pow +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Pow [Pow] outputs: [/transformer/h.3/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.3/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Mul_1 for ONNX node: /transformer/h.3/mlp/act/Mul_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.3/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Add [Add] inputs: [/transformer/h.3/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.3/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Add for ONNX node: /transformer/h.3/mlp/act/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Add [Add] outputs: [/transformer/h.3/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.3/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Mul_2 for ONNX node: /transformer/h.3/mlp/act/Mul_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.3/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Tanh [Tanh] inputs: [/transformer/h.3/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Tanh for ONNX node: /transformer/h.3/mlp/act/Tanh +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Tanh_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Tanh [Tanh] outputs: [/transformer/h.3/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Tanh_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Add_1 [Add] inputs: [/transformer/h.3/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Add_1 for ONNX node: /transformer/h.3/mlp/act/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Add_1 [Add] outputs: [/transformer/h.3/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.3/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.3/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/act/Mul_3 for ONNX node: /transformer/h.3/mlp/act/Mul_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.3/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.3/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2943 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.3/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_2943 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2943 for ONNX node: onnx::MatMul_2943 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/fc_out/MatMul for ONNX node: /transformer/h.3/mlp/fc_out/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.3/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.3/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/mlp/fc_out/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.3/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/mlp/fc_out/Add for ONNX node: /transformer/h.3/mlp/fc_out/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.3/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/mlp/fc_out/Add [Add] outputs: [/transformer/h.3/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/Add [Add] inputs: [/transformer/h.3/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/Add for ONNX node: /transformer/h.3/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/Add_output_0 for ONNX tensor: /transformer/h.3/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/Add [Add] outputs: [/transformer/h.3/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.3/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.2/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/Add_1 [Add] inputs: [/transformer/h.3/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.2/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.3/Add_1 for ONNX node: /transformer/h.3/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.3/Add_1_output_0 for ONNX tensor: /transformer/h.3/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.3/Add_1 [Add] outputs: [/transformer/h.3/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.3/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/ReduceMean for ONNX node: /transformer/h.4/ln_1/ReduceMean +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.4/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.4/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Sub [Sub] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.3/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Sub [Sub] inputs: [/transformer/h.3/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Sub for ONNX node: /transformer/h.4/ln_1/Sub +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.4/ln_1/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Sub [Sub] outputs: [/transformer/h.4/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Pow [Pow] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Pow [Pow] inputs: [/transformer/h.4/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Pow for ONNX node: /transformer/h.4/ln_1/Pow +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.4/ln_1/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Pow [Pow] outputs: [/transformer/h.4/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Pow_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.4/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/ReduceMean_1 for ONNX node: /transformer/h.4/ln_1/ReduceMean_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.4/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.4/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Add [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Add [Add] inputs: [/transformer/h.4/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Add for ONNX node: /transformer/h.4/ln_1/Add +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Add_output_0 for ONNX tensor: /transformer/h.4/ln_1/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Add [Add] outputs: [/transformer/h.4/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Add_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.4/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Sqrt for ONNX node: /transformer/h.4/ln_1/Sqrt +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.4/ln_1/Sqrt_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.4/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Div [Div] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Sub_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Sqrt_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Div [Div] inputs: [/transformer/h.4/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Div for ONNX node: /transformer/h.4/ln_1/Div +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Div_output_0 for ONNX tensor: /transformer/h.4/ln_1/Div_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Div [Div] outputs: [/transformer/h.4/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Mul [Mul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Div_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Mul [Mul] inputs: [/transformer/h.4/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Mul for ONNX node: /transformer/h.4/ln_1/Mul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.4/ln_1/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Mul [Mul] outputs: [/transformer/h.4/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/ln_1/Add_1 [Add] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Mul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Add_1 [Add] inputs: [/transformer/h.4/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/ln_1/Add_1 for ONNX node: /transformer/h.4/ln_1/Add_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.4/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/ln_1/Add_1 [Add] outputs: [/transformer/h.4/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2944 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.4/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2944 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2944 for ONNX node: onnx::MatMul_2944 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/q_proj/MatMul for ONNX node: /transformer/h.4/attn/q_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.4/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.4/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2945 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.4/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2945 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2945 for ONNX node: onnx::MatMul_2945 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/k_proj/MatMul for ONNX node: /transformer/h.4/attn/k_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.4/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.4/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Add_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::MatMul_2946 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.4/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2946 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: onnx::MatMul_2946 for ONNX node: onnx::MatMul_2946 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/v_proj/MatMul for ONNX node: /transformer/h.4/attn/v_proj/MatMul +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.4/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.4/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape [Shape] inputs: [/transformer/h.4/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape for ONNX node: /transformer/h.4/attn/Shape +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape [Shape] outputs: [/transformer/h.4/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather [Gather] inputs: [/transformer/h.4/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather for ONNX node: /transformer/h.4/attn/Gather +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather [Gather] outputs: [/transformer/h.4/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_1 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_1 [Shape] inputs: [/transformer/h.4/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_1 for ONNX node: /transformer/h.4/attn/Shape_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_1 [Shape] outputs: [/transformer/h.4/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_1 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_1 [Gather] inputs: [/transformer/h.4/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_1 for ONNX node: /transformer/h.4/attn/Gather_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_1 [Gather] outputs: [/transformer/h.4/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze for ONNX node: /transformer/h.4/attn/Unsqueeze +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_1 for ONNX node: /transformer/h.4/attn/Unsqueeze_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Concat [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat for ONNX node: /transformer/h.4/attn/Concat +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Concat [Concat] outputs: [/transformer/h.4/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Reshape [Reshape] inputs: [/transformer/h.4/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape for ONNX node: /transformer/h.4/attn/Reshape +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Reshape [Reshape] outputs: [/transformer/h.4/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_2 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_2 [Shape] inputs: [/transformer/h.4/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_2 for ONNX node: /transformer/h.4/attn/Shape_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_2 [Shape] outputs: [/transformer/h.4/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_2 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_2 [Gather] inputs: [/transformer/h.4/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_2 for ONNX node: /transformer/h.4/attn/Gather_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_2 [Gather] outputs: [/transformer/h.4/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_3 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_3 [Shape] inputs: [/transformer/h.4/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_3 for ONNX node: /transformer/h.4/attn/Shape_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_3 [Shape] outputs: [/transformer/h.4/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_3 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_3 [Gather] inputs: [/transformer/h.4/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_3 for ONNX node: /transformer/h.4/attn/Gather_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_3 [Gather] outputs: [/transformer/h.4/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_2 for ONNX node: /transformer/h.4/attn/Unsqueeze_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_3 for ONNX node: /transformer/h.4/attn/Unsqueeze_3 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_1 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Concat_1 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_1 for ONNX node: /transformer/h.4/attn/Concat_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Concat_1 [Concat] outputs: [/transformer/h.4/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Reshape_1 [Reshape] inputs: [/transformer/h.4/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_1 for ONNX node: /transformer/h.4/attn/Reshape_1 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Reshape_1 [Reshape] outputs: [/transformer/h.4/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_4 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_4 [Shape] inputs: [/transformer/h.4/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_4 for ONNX node: /transformer/h.4/attn/Shape_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_4 [Shape] outputs: [/transformer/h.4/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_4 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_4 [Gather] inputs: [/transformer/h.4/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_4 for ONNX node: /transformer/h.4/attn/Gather_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_4 [Gather] outputs: [/transformer/h.4/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_5 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_5 [Shape] inputs: [/transformer/h.4/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_5 for ONNX node: /transformer/h.4/attn/Shape_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_5 [Shape] outputs: [/transformer/h.4/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_5 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_5 [Gather] inputs: [/transformer/h.4/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_5 for ONNX node: /transformer/h.4/attn/Gather_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_5 [Gather] outputs: [/transformer/h.4/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_4 for ONNX node: /transformer/h.4/attn/Unsqueeze_4 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_5 for ONNX node: /transformer/h.4/attn/Unsqueeze_5 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_2 [Concat] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Concat_2 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_2 for ONNX node: /transformer/h.4/attn/Concat_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Concat_2 [Concat] outputs: [/transformer/h.4/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Reshape_2 [Reshape] inputs: [/transformer/h.4/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_2 for ONNX node: /transformer/h.4/attn/Reshape_2 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Reshape_2 [Reshape] outputs: [/transformer/h.4/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Transpose [Transpose] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_2_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Transpose [Transpose] inputs: [/transformer/h.4/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Transpose for ONNX node: /transformer/h.4/attn/Transpose +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: present.4.value_696 for ONNX tensor: present.4.value +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Transpose [Transpose] outputs: [present.4.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_6 [Shape] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_6 [Shape] inputs: [/transformer/h.4/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_6 for ONNX node: /transformer/h.4/attn/Shape_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Shape_6 [Shape] outputs: [/transformer/h.4/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_6 [Gather] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_6 [Gather] inputs: [/transformer/h.4/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_6 for ONNX node: /transformer/h.4/attn/Gather_6 +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_6_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Gather_6 [Gather] outputs: [/transformer/h.4/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Slice [Slice] inputs: [/transformer/h.4/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:19] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice for ONNX node: /transformer/h.4/attn/Slice +[12/30/2022-19:54:19] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Slice [Slice] outputs: [/transformer/h.4/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:19] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_1 [Slice] +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_1_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:19] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:19] [V] [TRT] /transformer/h.4/attn/Slice_1 [Slice] inputs: [/transformer/h.4/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_1 for ONNX node: /transformer/h.4/attn/Slice_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_1 [Slice] outputs: [/transformer/h.4/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_2 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_2 [Slice] inputs: [/transformer/h.4/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_2 for ONNX node: /transformer/h.4/attn/Slice_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_2 [Slice] outputs: [/transformer/h.4/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_3 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_3 [Slice] inputs: [/transformer/h.4/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_3 for ONNX node: /transformer/h.4/attn/Slice_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_3 [Slice] outputs: [/transformer/h.4/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_7 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_7 [Shape] inputs: [/transformer/h.4/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_7 for ONNX node: /transformer/h.4/attn/Shape_7 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_7 [Shape] outputs: [/transformer/h.4/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_7 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_7 [Gather] inputs: [/transformer/h.4/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_7 for ONNX node: /transformer/h.4/attn/Gather_7 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_7 [Gather] outputs: [/transformer/h.4/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Range [Range] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.4/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Range for ONNX node: /transformer/h.4/attn/Range +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Range_output_0 for ONNX tensor: /transformer/h.4/attn/Range_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Range [Range] outputs: [/transformer/h.4/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Cast_1 [Cast] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Range_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cast_1 [Cast] inputs: [/transformer/h.4/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Cast_1 for ONNX node: /transformer/h.4/attn/Cast_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.4/attn/Cast_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cast_1 [Cast] outputs: [/transformer/h.4/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Cast_2 [Cast] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cast_2 [Cast] inputs: [/transformer/h.4/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Cast_2 for ONNX node: /transformer/h.4/attn/Cast_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.4/attn/Cast_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cast_2 [Cast] outputs: [/transformer/h.4/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Div [Div] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Cast_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Cast_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Div [Div] inputs: [/transformer/h.4/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.4/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Div for ONNX node: /transformer/h.4/attn/Div +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Div_output_0 for ONNX tensor: /transformer/h.4/attn/Div_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Div [Div] outputs: [/transformer/h.4/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Pow [Pow] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Div_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.4/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Pow for ONNX node: /transformer/h.4/attn/Pow +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Pow_output_0 for ONNX tensor: /transformer/h.4/attn/Pow_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Pow [Pow] outputs: [/transformer/h.4/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Pow_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.4/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reciprocal for ONNX node: /transformer/h.4/attn/Reciprocal +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.4/attn/Reciprocal_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.4/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Mul [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reciprocal_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul [Mul] inputs: [/transformer/h.4/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Mul for ONNX node: /transformer/h.4/attn/Mul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Mul_output_0 for ONNX tensor: /transformer/h.4/attn/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul [Mul] outputs: [/transformer/h.4/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Cast_3 [Cast] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cast_3 [Cast] inputs: [/transformer/h.4/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Cast_3 for ONNX node: /transformer/h.4/attn/Cast_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.4/attn/Cast_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cast_3 [Cast] outputs: [/transformer/h.4/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Range_1 [Range] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Cast_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.4/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Range_1 for ONNX node: /transformer/h.4/attn/Range_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Range_1_output_0 for ONNX tensor: /transformer/h.4/attn/Range_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Range_1 [Range] outputs: [/transformer/h.4/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Einsum [Einsum] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Range_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Einsum [Einsum] inputs: [/transformer/h.4/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.4/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Einsum for ONNX node: /transformer/h.4/attn/Einsum +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Einsum_output_0 for ONNX tensor: /transformer/h.4/attn/Einsum_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Einsum [Einsum] outputs: [/transformer/h.4/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Sin [Sin] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Einsum_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Sin [Sin] inputs: [/transformer/h.4/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Sin for ONNX node: /transformer/h.4/attn/Sin +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Sin_output_0 for ONNX tensor: /transformer/h.4/attn/Sin_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Sin [Sin] outputs: [/transformer/h.4/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Cos [Cos] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Einsum_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cos [Cos] inputs: [/transformer/h.4/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Cos for ONNX node: /transformer/h.4/attn/Cos +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Cos_output_0 for ONNX tensor: /transformer/h.4/attn/Cos_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Cos [Cos] outputs: [/transformer/h.4/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_8 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Sin_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_8 [Shape] inputs: [/transformer/h.4/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_8 for ONNX node: /transformer/h.4/attn/Shape_8 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_8 [Shape] outputs: [/transformer/h.4/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_8 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_8 [Gather] inputs: [/transformer/h.4/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_8 for ONNX node: /transformer/h.4/attn/Gather_8 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_8 [Gather] outputs: [/transformer/h.4/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Sin_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_3 [Reshape] inputs: [/transformer/h.4/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_3 for ONNX node: /transformer/h.4/attn/Reshape_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_3 [Reshape] outputs: [/transformer/h.4/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/ConstantOfShape for ONNX node: /transformer/h.4/attn/ConstantOfShape +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.4/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.4/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Expand [Expand] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Expand [Expand] inputs: [/transformer/h.4/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.4/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Expand for ONNX node: /transformer/h.4/attn/Expand +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Expand_output_0 for ONNX tensor: /transformer/h.4/attn/Expand_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Expand [Expand] outputs: [/transformer/h.4/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Tile [Tile] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Expand_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Tile [Tile] inputs: [/transformer/h.4/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Tile for ONNX node: /transformer/h.4/attn/Tile +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Tile_output_0 for ONNX tensor: /transformer/h.4/attn/Tile_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Tile [Tile] outputs: [/transformer/h.4/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_6 for ONNX node: /transformer/h.4/attn/Unsqueeze_6 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_3 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_3 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_3 for ONNX node: /transformer/h.4/attn/Concat_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_3 [Concat] outputs: [/transformer/h.4/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Tile_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_4 [Reshape] inputs: [/transformer/h.4/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.4/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_4 for ONNX node: /transformer/h.4/attn/Reshape_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_4 [Reshape] outputs: [/transformer/h.4/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_9 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_9 [Shape] inputs: [/transformer/h.4/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_9 for ONNX node: /transformer/h.4/attn/Shape_9 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_9 [Shape] outputs: [/transformer/h.4/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_9 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_9 [Gather] inputs: [/transformer/h.4/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_9 for ONNX node: /transformer/h.4/attn/Gather_9 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_9 [Gather] outputs: [/transformer/h.4/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.4/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_7 for ONNX node: /transformer/h.4/attn/Unsqueeze_7 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_8 for ONNX node: /transformer/h.4/attn/Unsqueeze_8 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_4 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_4 [Slice] inputs: [/transformer/h.4/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_4 for ONNX node: /transformer/h.4/attn/Slice_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_4 [Slice] outputs: [/transformer/h.4/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.4/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_9 for ONNX node: /transformer/h.4/attn/Unsqueeze_9 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_10 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Cos_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_10 [Shape] inputs: [/transformer/h.4/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_10 for ONNX node: /transformer/h.4/attn/Shape_10 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_10 [Shape] outputs: [/transformer/h.4/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_10 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_10 [Gather] inputs: [/transformer/h.4/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_10 for ONNX node: /transformer/h.4/attn/Gather_10 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_10 [Gather] outputs: [/transformer/h.4/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Cos_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_5 [Reshape] inputs: [/transformer/h.4/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_5 for ONNX node: /transformer/h.4/attn/Reshape_5 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_5 [Reshape] outputs: [/transformer/h.4/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/ConstantOfShape_1 for ONNX node: /transformer/h.4/attn/ConstantOfShape_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.4/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.4/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Expand_1 [Expand] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Expand_1 [Expand] inputs: [/transformer/h.4/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.4/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Expand_1 for ONNX node: /transformer/h.4/attn/Expand_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.4/attn/Expand_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Expand_1 [Expand] outputs: [/transformer/h.4/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Tile_1 [Tile] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Expand_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Tile_1 [Tile] inputs: [/transformer/h.4/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Tile_1 for ONNX node: /transformer/h.4/attn/Tile_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.4/attn/Tile_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Tile_1 [Tile] outputs: [/transformer/h.4/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_10 for ONNX node: /transformer/h.4/attn/Unsqueeze_10 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_4 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_4 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_4 for ONNX node: /transformer/h.4/attn/Concat_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_4 [Concat] outputs: [/transformer/h.4/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Tile_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_6 [Reshape] inputs: [/transformer/h.4/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.4/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_6 for ONNX node: /transformer/h.4/attn/Reshape_6 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_6 [Reshape] outputs: [/transformer/h.4/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.4/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_11 for ONNX node: /transformer/h.4/attn/Unsqueeze_11 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_12 for ONNX node: /transformer/h.4/attn/Unsqueeze_12 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_5 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_5 [Slice] inputs: [/transformer/h.4/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_5 for ONNX node: /transformer/h.4/attn/Slice_5 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_5 [Slice] outputs: [/transformer/h.4/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.4/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_13 for ONNX node: /transformer/h.4/attn/Unsqueeze_13 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Mul_1 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_1 [Mul] inputs: [/transformer/h.4/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Mul_1 for ONNX node: /transformer/h.4/attn/Mul_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.4/attn/Mul_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_1 [Mul] outputs: [/transformer/h.4/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_6 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_6 [Slice] inputs: [/transformer/h.4/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_6 for ONNX node: /transformer/h.4/attn/Slice_6 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_6 [Slice] outputs: [/transformer/h.4/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_7 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_7 [Slice] inputs: [/transformer/h.4/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_7 for ONNX node: /transformer/h.4/attn/Slice_7 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_7 [Slice] outputs: [/transformer/h.4/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Neg [Neg] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Neg [Neg] inputs: [/transformer/h.4/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Neg for ONNX node: /transformer/h.4/attn/Neg +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Neg_output_0 for ONNX tensor: /transformer/h.4/attn/Neg_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Neg [Neg] outputs: [/transformer/h.4/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Neg_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.4/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_14 for ONNX node: /transformer/h.4/attn/Unsqueeze_14 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.4/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_15 for ONNX node: /transformer/h.4/attn/Unsqueeze_15 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_5 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_5 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.4/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_5 for ONNX node: /transformer/h.4/attn/Concat_5 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_5 [Concat] outputs: [/transformer/h.4/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_11 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_11 [Shape] inputs: [/transformer/h.4/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_11 for ONNX node: /transformer/h.4/attn/Shape_11 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_11 [Shape] outputs: [/transformer/h.4/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_8 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_8 [Slice] inputs: [/transformer/h.4/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_8 for ONNX node: /transformer/h.4/attn/Slice_8 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_8 [Slice] outputs: [/transformer/h.4/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_6 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_6 [Concat] inputs: [/transformer/h.4/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_6 for ONNX node: /transformer/h.4/attn/Concat_6 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_6 [Concat] outputs: [/transformer/h.4/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_7 [Reshape] inputs: [/transformer/h.4/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.4/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_7 for ONNX node: /transformer/h.4/attn/Reshape_7 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_7 [Reshape] outputs: [/transformer/h.4/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Mul_2 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_2 [Mul] inputs: [/transformer/h.4/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Mul_2 for ONNX node: /transformer/h.4/attn/Mul_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.4/attn/Mul_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_2 [Mul] outputs: [/transformer/h.4/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Add [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Mul_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Mul_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Add [Add] inputs: [/transformer/h.4/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Add for ONNX node: /transformer/h.4/attn/Add +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Add_output_0 for ONNX tensor: /transformer/h.4/attn/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Add [Add] outputs: [/transformer/h.4/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_12 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_12 [Shape] inputs: [/transformer/h.4/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_12 for ONNX node: /transformer/h.4/attn/Shape_12 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_12 [Shape] outputs: [/transformer/h.4/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_11 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_11 [Gather] inputs: [/transformer/h.4/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_11 for ONNX node: /transformer/h.4/attn/Gather_11 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_11 [Gather] outputs: [/transformer/h.4/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_16 for ONNX node: /transformer/h.4/attn/Unsqueeze_16 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_9 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_9 [Slice] inputs: [/transformer/h.4/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_9 for ONNX node: /transformer/h.4/attn/Slice_9 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_9 [Slice] outputs: [/transformer/h.4/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.4/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_17 for ONNX node: /transformer/h.4/attn/Unsqueeze_17 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_18 for ONNX node: /transformer/h.4/attn/Unsqueeze_18 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_10 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_10 [Slice] inputs: [/transformer/h.4/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_10 for ONNX node: /transformer/h.4/attn/Slice_10 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_10 [Slice] outputs: [/transformer/h.4/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.4/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_19 for ONNX node: /transformer/h.4/attn/Unsqueeze_19 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Mul_3 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_3 [Mul] inputs: [/transformer/h.4/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Mul_3 for ONNX node: /transformer/h.4/attn/Mul_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.4/attn/Mul_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_3 [Mul] outputs: [/transformer/h.4/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_11 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_11 [Slice] inputs: [/transformer/h.4/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_11 for ONNX node: /transformer/h.4/attn/Slice_11 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_11 [Slice] outputs: [/transformer/h.4/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_12 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_12 [Slice] inputs: [/transformer/h.4/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_12 for ONNX node: /transformer/h.4/attn/Slice_12 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_12 [Slice] outputs: [/transformer/h.4/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Neg_1 [Neg] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Neg_1 [Neg] inputs: [/transformer/h.4/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Neg_1 for ONNX node: /transformer/h.4/attn/Neg_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.4/attn/Neg_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Neg_1 [Neg] outputs: [/transformer/h.4/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Neg_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.4/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_20 for ONNX node: /transformer/h.4/attn/Unsqueeze_20 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.4/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_21 for ONNX node: /transformer/h.4/attn/Unsqueeze_21 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_7 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_7 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.4/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_7 for ONNX node: /transformer/h.4/attn/Concat_7 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_7 [Concat] outputs: [/transformer/h.4/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_13 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_13 [Shape] inputs: [/transformer/h.4/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_13 for ONNX node: /transformer/h.4/attn/Shape_13 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_13 [Shape] outputs: [/transformer/h.4/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_13 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_13 [Slice] inputs: [/transformer/h.4/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_13 for ONNX node: /transformer/h.4/attn/Slice_13 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_13 [Slice] outputs: [/transformer/h.4/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_8 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_8 [Concat] inputs: [/transformer/h.4/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_8 for ONNX node: /transformer/h.4/attn/Concat_8 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_8 [Concat] outputs: [/transformer/h.4/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_8 [Reshape] inputs: [/transformer/h.4/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.4/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_8 for ONNX node: /transformer/h.4/attn/Reshape_8 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_8 [Reshape] outputs: [/transformer/h.4/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Mul_4 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_4 [Mul] inputs: [/transformer/h.4/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Mul_4 for ONNX node: /transformer/h.4/attn/Mul_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.4/attn/Mul_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Mul_4 [Mul] outputs: [/transformer/h.4/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Add_1 [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Mul_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Mul_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Add_1 [Add] inputs: [/transformer/h.4/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Add_1 for ONNX node: /transformer/h.4/attn/Add_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Add_1_output_0 for ONNX tensor: /transformer/h.4/attn/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Add_1 [Add] outputs: [/transformer/h.4/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_9 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_9 [Concat] inputs: [/transformer/h.4/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_9 for ONNX node: /transformer/h.4/attn/Concat_9 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_9 [Concat] outputs: [/transformer/h.4/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_10 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_10 [Concat] inputs: [/transformer/h.4/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.4/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_10 for ONNX node: /transformer/h.4/attn/Concat_10 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_10 [Concat] outputs: [/transformer/h.4/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_1 [Transpose] inputs: [/transformer/h.4/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Transpose_1 for ONNX node: /transformer/h.4/attn/Transpose_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: present.4.key_845 for ONNX tensor: present.4.key +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_1 [Transpose] outputs: [present.4.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_10_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_2 [Transpose] inputs: [/transformer/h.4/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Transpose_2 for ONNX node: /transformer/h.4/attn/Transpose_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.4/attn/Transpose_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_2 [Transpose] outputs: [/transformer/h.4/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_14 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Transpose_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_14 [Shape] inputs: [/transformer/h.4/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_14 for ONNX node: /transformer/h.4/attn/Shape_14 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_14_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_14 [Shape] outputs: [/transformer/h.4/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_14 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_14_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_14 [Slice] inputs: [/transformer/h.4/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_14 for ONNX node: /transformer/h.4/attn/Slice_14 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_14_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_14 [Slice] outputs: [/transformer/h.4/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Squeeze [Squeeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_14_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Squeeze [Squeeze] inputs: [/transformer/h.4/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Squeeze for ONNX node: /transformer/h.4/attn/Squeeze +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.4/attn/Squeeze_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Squeeze [Squeeze] outputs: [/transformer/h.4/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_15 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: present.4.key +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_15 [Shape] inputs: [present.4.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_15 for ONNX node: /transformer/h.4/attn/Shape_15 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_15 [Shape] outputs: [/transformer/h.4/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_15 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_15 [Slice] inputs: [/transformer/h.4/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_15 for ONNX node: /transformer/h.4/attn/Slice_15 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_15 [Slice] outputs: [/transformer/h.4/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_15_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.4/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Squeeze_1 for ONNX node: /transformer/h.4/attn/Squeeze_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.4/attn/Squeeze_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.4/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Sub [Sub] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Squeeze_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Squeeze_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Sub [Sub] inputs: [/transformer/h.4/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.4/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Sub for ONNX node: /transformer/h.4/attn/Sub +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Sub_output_0 for ONNX tensor: /transformer/h.4/attn/Sub_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Sub [Sub] outputs: [/transformer/h.4/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Sub_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.4/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_22 for ONNX node: /transformer/h.4/attn/Unsqueeze_22 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Squeeze_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.4/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_23 for ONNX node: /transformer/h.4/attn/Unsqueeze_23 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_16 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.4/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_16 for ONNX node: /transformer/h.4/attn/Slice_16 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_16_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_16 [Slice] outputs: [/transformer/h.4/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Squeeze_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.4/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_24 for ONNX node: /transformer/h.4/attn/Unsqueeze_24 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Slice_17 [Slice] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_16_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_17 [Slice] inputs: [/transformer/h.4/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Slice_17 for ONNX node: /transformer/h.4/attn/Slice_17 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.4/attn/Slice_17_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Slice_17 [Slice] outputs: [/transformer/h.4/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_3 [Transpose] inputs: [/transformer/h.4/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Transpose_3 for ONNX node: /transformer/h.4/attn/Transpose_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.4/attn/Transpose_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_3 [Transpose] outputs: [/transformer/h.4/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Transpose_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Transpose_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/MatMul [MatMul] inputs: [/transformer/h.4/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.4/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/MatMul for ONNX node: /transformer/h.4/attn/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/MatMul_output_0 for ONNX tensor: /transformer/h.4/attn/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/MatMul [MatMul] outputs: [/transformer/h.4/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Where [Where] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Slice_17_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Where [Where] inputs: [/transformer/h.4/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.4/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Where for ONNX node: /transformer/h.4/attn/Where +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Where_output_0 for ONNX tensor: /transformer/h.4/attn/Where_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Where [Where] outputs: [/transformer/h.4/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Div_1 [Div] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Where_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Div_1 [Div] inputs: [/transformer/h.4/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Div_1 for ONNX node: /transformer/h.4/attn/Div_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Div_1_output_0 for ONNX tensor: /transformer/h.4/attn/Div_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Div_1 [Div] outputs: [/transformer/h.4/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Add_2 [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Div_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Add_2 [Add] inputs: [/transformer/h.4/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Add_2 for ONNX node: /transformer/h.4/attn/Add_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Add_2_output_0 for ONNX tensor: /transformer/h.4/attn/Add_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Add_2 [Add] outputs: [/transformer/h.4/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Softmax [Softmax] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Add_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Softmax [Softmax] inputs: [/transformer/h.4/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Softmax for ONNX node: /transformer/h.4/attn/Softmax +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Softmax_output_0 for ONNX tensor: /transformer/h.4/attn/Softmax_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Softmax [Softmax] outputs: [/transformer/h.4/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Softmax_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: present.4.value +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/MatMul_1 [MatMul] inputs: [/transformer/h.4/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.4.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/MatMul_1 for ONNX node: /transformer/h.4/attn/MatMul_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.4/attn/MatMul_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/MatMul_1 [MatMul] outputs: [/transformer/h.4/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/MatMul_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_4 [Transpose] inputs: [/transformer/h.4/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Transpose_4 for ONNX node: /transformer/h.4/attn/Transpose_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.4/attn/Transpose_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Transpose_4 [Transpose] outputs: [/transformer/h.4/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_16 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Transpose_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_16 [Shape] inputs: [/transformer/h.4/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_16 for ONNX node: /transformer/h.4/attn/Shape_16 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_16_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_16 [Shape] outputs: [/transformer/h.4/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_12 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_16_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_12 [Gather] inputs: [/transformer/h.4/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_12 for ONNX node: /transformer/h.4/attn/Gather_12 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_12 [Gather] outputs: [/transformer/h.4/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Shape_17 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Transpose_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_17 [Shape] inputs: [/transformer/h.4/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Shape_17 for ONNX node: /transformer/h.4/attn/Shape_17 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.4/attn/Shape_17_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Shape_17 [Shape] outputs: [/transformer/h.4/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Gather_13 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Shape_17_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_13 [Gather] inputs: [/transformer/h.4/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Gather_13 for ONNX node: /transformer/h.4/attn/Gather_13 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.4/attn/Gather_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Gather_13 [Gather] outputs: [/transformer/h.4/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_12_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_25 for ONNX node: /transformer/h.4/attn/Unsqueeze_25 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Gather_13_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.4/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Unsqueeze_26 for ONNX node: /transformer/h.4/attn/Unsqueeze_26 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.4/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.4/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Concat_11 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_11 [Concat] inputs: [/transformer/h.4/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.4/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Concat_11 for ONNX node: /transformer/h.4/attn/Concat_11 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.4/attn/Concat_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Concat_11 [Concat] outputs: [/transformer/h.4/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Transpose_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Concat_11_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_9 [Reshape] inputs: [/transformer/h.4/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.4/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/Reshape_9 for ONNX node: /transformer/h.4/attn/Reshape_9 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.4/attn/Reshape_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/Reshape_9 [Reshape] outputs: [/transformer/h.4/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/Reshape_9_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::MatMul_2971 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.4/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2971 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: onnx::MatMul_2971 for ONNX node: onnx::MatMul_2971 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/attn/out_proj/MatMul for ONNX node: /transformer/h.4/attn/out_proj/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.4/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.4/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/ln_1/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::MatMul_2972 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.4/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2972 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: onnx::MatMul_2972 for ONNX node: onnx::MatMul_2972 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/fc_in/MatMul for ONNX node: /transformer/h.4/mlp/fc_in/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.4/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.4/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/fc_in/Add [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.4/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/fc_in/Add for ONNX node: /transformer/h.4/mlp/fc_in/Add +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.4/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_in/Add [Add] outputs: [/transformer/h.4/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Mul [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul [Mul] inputs: [/transformer/h.4/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Mul for ONNX node: /transformer/h.4/mlp/act/Mul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul [Mul] outputs: [/transformer/h.4/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Pow [Pow] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Pow [Pow] inputs: [/transformer/h.4/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Pow for ONNX node: /transformer/h.4/mlp/act/Pow +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Pow_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Pow [Pow] outputs: [/transformer/h.4/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Pow_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.4/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Mul_1 for ONNX node: /transformer/h.4/mlp/act/Mul_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.4/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Add [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Add [Add] inputs: [/transformer/h.4/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.4/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Add for ONNX node: /transformer/h.4/mlp/act/Add +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Add [Add] outputs: [/transformer/h.4/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.4/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Mul_2 for ONNX node: /transformer/h.4/mlp/act/Mul_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.4/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Tanh [Tanh] inputs: [/transformer/h.4/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Tanh for ONNX node: /transformer/h.4/mlp/act/Tanh +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Tanh_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Tanh [Tanh] outputs: [/transformer/h.4/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Add_1 [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Tanh_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Add_1 [Add] inputs: [/transformer/h.4/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Add_1 for ONNX node: /transformer/h.4/mlp/act/Add_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Add_1 [Add] outputs: [/transformer/h.4/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.4/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.4/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/act/Mul_3 for ONNX node: /transformer/h.4/mlp/act/Mul_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.4/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.4/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::MatMul_2973 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.4/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_2973 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: onnx::MatMul_2973 for ONNX node: onnx::MatMul_2973 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/fc_out/MatMul for ONNX node: /transformer/h.4/mlp/fc_out/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.4/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.4/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/mlp/fc_out/Add [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.4/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/mlp/fc_out/Add for ONNX node: /transformer/h.4/mlp/fc_out/Add +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.4/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/mlp/fc_out/Add [Add] outputs: [/transformer/h.4/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/Add [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/Add [Add] inputs: [/transformer/h.4/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/Add for ONNX node: /transformer/h.4/Add +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/Add_output_0 for ONNX tensor: /transformer/h.4/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/Add [Add] outputs: [/transformer/h.4/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.4/Add_1 [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.3/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/Add_1 [Add] inputs: [/transformer/h.4/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.3/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.4/Add_1 for ONNX node: /transformer/h.4/Add_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.4/Add_1_output_0 for ONNX tensor: /transformer/h.4/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.4/Add_1 [Add] outputs: [/transformer/h.4/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.4/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/ReduceMean for ONNX node: /transformer/h.5/ln_1/ReduceMean +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.5/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.5/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Sub [Sub] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.4/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Sub [Sub] inputs: [/transformer/h.4/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Sub for ONNX node: /transformer/h.5/ln_1/Sub +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.5/ln_1/Sub_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Sub [Sub] outputs: [/transformer/h.5/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Pow [Pow] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Sub_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Pow [Pow] inputs: [/transformer/h.5/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Pow for ONNX node: /transformer/h.5/ln_1/Pow +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.5/ln_1/Pow_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Pow [Pow] outputs: [/transformer/h.5/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Pow_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.5/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/ReduceMean_1 for ONNX node: /transformer/h.5/ln_1/ReduceMean_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.5/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.5/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Add [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Add [Add] inputs: [/transformer/h.5/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Add for ONNX node: /transformer/h.5/ln_1/Add +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Add_output_0 for ONNX tensor: /transformer/h.5/ln_1/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Add [Add] outputs: [/transformer/h.5/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Add_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.5/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Sqrt for ONNX node: /transformer/h.5/ln_1/Sqrt +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.5/ln_1/Sqrt_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.5/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Div [Div] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Sub_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Sqrt_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Div [Div] inputs: [/transformer/h.5/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Div for ONNX node: /transformer/h.5/ln_1/Div +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Div_output_0 for ONNX tensor: /transformer/h.5/ln_1/Div_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Div [Div] outputs: [/transformer/h.5/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Mul [Mul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Div_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Mul [Mul] inputs: [/transformer/h.5/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Mul for ONNX node: /transformer/h.5/ln_1/Mul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.5/ln_1/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Mul [Mul] outputs: [/transformer/h.5/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/ln_1/Add_1 [Add] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Mul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Add_1 [Add] inputs: [/transformer/h.5/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/ln_1/Add_1 for ONNX node: /transformer/h.5/ln_1/Add_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.5/ln_1/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/ln_1/Add_1 [Add] outputs: [/transformer/h.5/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::MatMul_2974 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.5/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2974 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: onnx::MatMul_2974 for ONNX node: onnx::MatMul_2974 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/q_proj/MatMul for ONNX node: /transformer/h.5/attn/q_proj/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.5/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.5/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::MatMul_2975 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.5/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2975 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: onnx::MatMul_2975 for ONNX node: onnx::MatMul_2975 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/k_proj/MatMul for ONNX node: /transformer/h.5/attn/k_proj/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.5/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.5/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Add_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::MatMul_2976 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.5/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_2976 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: onnx::MatMul_2976 for ONNX node: onnx::MatMul_2976 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/v_proj/MatMul for ONNX node: /transformer/h.5/attn/v_proj/MatMul +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.5/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.5/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape [Shape] inputs: [/transformer/h.5/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape for ONNX node: /transformer/h.5/attn/Shape +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape [Shape] outputs: [/transformer/h.5/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather [Gather] inputs: [/transformer/h.5/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather for ONNX node: /transformer/h.5/attn/Gather +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather [Gather] outputs: [/transformer/h.5/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_1 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_1 [Shape] inputs: [/transformer/h.5/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_1 for ONNX node: /transformer/h.5/attn/Shape_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_1 [Shape] outputs: [/transformer/h.5/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_1 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_1 [Gather] inputs: [/transformer/h.5/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_1 for ONNX node: /transformer/h.5/attn/Gather_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_1 [Gather] outputs: [/transformer/h.5/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze for ONNX node: /transformer/h.5/attn/Unsqueeze +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_1 for ONNX node: /transformer/h.5/attn/Unsqueeze_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Concat [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat for ONNX node: /transformer/h.5/attn/Concat +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Concat [Concat] outputs: [/transformer/h.5/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Reshape [Reshape] inputs: [/transformer/h.5/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape for ONNX node: /transformer/h.5/attn/Reshape +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Reshape [Reshape] outputs: [/transformer/h.5/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_2 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_2 [Shape] inputs: [/transformer/h.5/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_2 for ONNX node: /transformer/h.5/attn/Shape_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_2 [Shape] outputs: [/transformer/h.5/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_2 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_2 [Gather] inputs: [/transformer/h.5/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_2 for ONNX node: /transformer/h.5/attn/Gather_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_2 [Gather] outputs: [/transformer/h.5/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_3 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_3 [Shape] inputs: [/transformer/h.5/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_3 for ONNX node: /transformer/h.5/attn/Shape_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_3 [Shape] outputs: [/transformer/h.5/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_3 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_3 [Gather] inputs: [/transformer/h.5/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_3 for ONNX node: /transformer/h.5/attn/Gather_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_3 [Gather] outputs: [/transformer/h.5/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_2 for ONNX node: /transformer/h.5/attn/Unsqueeze_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_3 for ONNX node: /transformer/h.5/attn/Unsqueeze_3 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_1 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Concat_1 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_1 for ONNX node: /transformer/h.5/attn/Concat_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Concat_1 [Concat] outputs: [/transformer/h.5/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Reshape_1 [Reshape] inputs: [/transformer/h.5/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_1 for ONNX node: /transformer/h.5/attn/Reshape_1 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Reshape_1 [Reshape] outputs: [/transformer/h.5/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_4 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_4 [Shape] inputs: [/transformer/h.5/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_4 for ONNX node: /transformer/h.5/attn/Shape_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_4 [Shape] outputs: [/transformer/h.5/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_4 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_4 [Gather] inputs: [/transformer/h.5/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_4 for ONNX node: /transformer/h.5/attn/Gather_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_4 [Gather] outputs: [/transformer/h.5/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_5 [Shape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_5 [Shape] inputs: [/transformer/h.5/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_5 for ONNX node: /transformer/h.5/attn/Shape_5 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Shape_5 [Shape] outputs: [/transformer/h.5/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_5 [Gather] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_5 [Gather] inputs: [/transformer/h.5/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_5 for ONNX node: /transformer/h.5/attn/Gather_5 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Gather_5 [Gather] outputs: [/transformer/h.5/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_4 for ONNX node: /transformer/h.5/attn/Unsqueeze_4 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_5 for ONNX node: /transformer/h.5/attn/Unsqueeze_5 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_2 [Concat] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Concat_2 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_2 for ONNX node: /transformer/h.5/attn/Concat_2 +[12/30/2022-19:54:20] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Concat_2 [Concat] outputs: [/transformer/h.5/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:20] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_2_output_0 +[12/30/2022-19:54:20] [V] [TRT] /transformer/h.5/attn/Reshape_2 [Reshape] inputs: [/transformer/h.5/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:20] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_2 for ONNX node: /transformer/h.5/attn/Reshape_2 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_2 [Reshape] outputs: [/transformer/h.5/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Transpose [Transpose] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Transpose [Transpose] inputs: [/transformer/h.5/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Transpose for ONNX node: /transformer/h.5/attn/Transpose +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: present.5.value_865 for ONNX tensor: present.5.value +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Transpose [Transpose] outputs: [present.5.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_6 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_6 [Shape] inputs: [/transformer/h.5/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_6 for ONNX node: /transformer/h.5/attn/Shape_6 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_6 [Shape] outputs: [/transformer/h.5/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_6 [Gather] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_6 [Gather] inputs: [/transformer/h.5/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_6 for ONNX node: /transformer/h.5/attn/Gather_6 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_6 [Gather] outputs: [/transformer/h.5/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice [Slice] inputs: [/transformer/h.5/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice for ONNX node: /transformer/h.5/attn/Slice +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice [Slice] outputs: [/transformer/h.5/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_1 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_1 [Slice] inputs: [/transformer/h.5/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_1 for ONNX node: /transformer/h.5/attn/Slice_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_1 [Slice] outputs: [/transformer/h.5/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_2 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_2 [Slice] inputs: [/transformer/h.5/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_2 for ONNX node: /transformer/h.5/attn/Slice_2 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_2 [Slice] outputs: [/transformer/h.5/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_3 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_3 [Slice] inputs: [/transformer/h.5/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_3 for ONNX node: /transformer/h.5/attn/Slice_3 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_3 [Slice] outputs: [/transformer/h.5/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_7 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_7 [Shape] inputs: [/transformer/h.5/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_7 for ONNX node: /transformer/h.5/attn/Shape_7 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_7 [Shape] outputs: [/transformer/h.5/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_7 [Gather] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_7 [Gather] inputs: [/transformer/h.5/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_7 for ONNX node: /transformer/h.5/attn/Gather_7 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_7 [Gather] outputs: [/transformer/h.5/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Range [Range] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.5/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Range for ONNX node: /transformer/h.5/attn/Range +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Range_output_0 for ONNX tensor: /transformer/h.5/attn/Range_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Range [Range] outputs: [/transformer/h.5/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Cast_1 [Cast] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Range_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cast_1 [Cast] inputs: [/transformer/h.5/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Cast_1 for ONNX node: /transformer/h.5/attn/Cast_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.5/attn/Cast_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cast_1 [Cast] outputs: [/transformer/h.5/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Cast_2 [Cast] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cast_2 [Cast] inputs: [/transformer/h.5/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Cast_2 for ONNX node: /transformer/h.5/attn/Cast_2 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.5/attn/Cast_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cast_2 [Cast] outputs: [/transformer/h.5/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Div [Div] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Cast_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Cast_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Div [Div] inputs: [/transformer/h.5/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.5/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Div for ONNX node: /transformer/h.5/attn/Div +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Div_output_0 for ONNX tensor: /transformer/h.5/attn/Div_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Div [Div] outputs: [/transformer/h.5/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Pow [Pow] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Div_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.5/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Pow for ONNX node: /transformer/h.5/attn/Pow +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Pow_output_0 for ONNX tensor: /transformer/h.5/attn/Pow_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Pow [Pow] outputs: [/transformer/h.5/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Pow_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.5/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reciprocal for ONNX node: /transformer/h.5/attn/Reciprocal +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.5/attn/Reciprocal_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.5/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Mul [Mul] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reciprocal_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul [Mul] inputs: [/transformer/h.5/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Mul for ONNX node: /transformer/h.5/attn/Mul +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Mul_output_0 for ONNX tensor: /transformer/h.5/attn/Mul_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul [Mul] outputs: [/transformer/h.5/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Cast_3 [Cast] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cast_3 [Cast] inputs: [/transformer/h.5/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Cast_3 for ONNX node: /transformer/h.5/attn/Cast_3 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.5/attn/Cast_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cast_3 [Cast] outputs: [/transformer/h.5/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Range_1 [Range] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Cast_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.5/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Range_1 for ONNX node: /transformer/h.5/attn/Range_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Range_1_output_0 for ONNX tensor: /transformer/h.5/attn/Range_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Range_1 [Range] outputs: [/transformer/h.5/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Einsum [Einsum] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Range_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Mul_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Einsum [Einsum] inputs: [/transformer/h.5/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.5/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Einsum for ONNX node: /transformer/h.5/attn/Einsum +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Einsum_output_0 for ONNX tensor: /transformer/h.5/attn/Einsum_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Einsum [Einsum] outputs: [/transformer/h.5/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Sin [Sin] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Einsum_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Sin [Sin] inputs: [/transformer/h.5/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Sin for ONNX node: /transformer/h.5/attn/Sin +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Sin_output_0 for ONNX tensor: /transformer/h.5/attn/Sin_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Sin [Sin] outputs: [/transformer/h.5/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Cos [Cos] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Einsum_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cos [Cos] inputs: [/transformer/h.5/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Cos for ONNX node: /transformer/h.5/attn/Cos +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Cos_output_0 for ONNX tensor: /transformer/h.5/attn/Cos_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Cos [Cos] outputs: [/transformer/h.5/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_8 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Sin_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_8 [Shape] inputs: [/transformer/h.5/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_8 for ONNX node: /transformer/h.5/attn/Shape_8 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_8 [Shape] outputs: [/transformer/h.5/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_8 [Gather] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_8 [Gather] inputs: [/transformer/h.5/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_8 for ONNX node: /transformer/h.5/attn/Gather_8 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_8 [Gather] outputs: [/transformer/h.5/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Sin_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_3 [Reshape] inputs: [/transformer/h.5/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_3 for ONNX node: /transformer/h.5/attn/Reshape_3 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_3 [Reshape] outputs: [/transformer/h.5/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/ConstantOfShape for ONNX node: /transformer/h.5/attn/ConstantOfShape +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.5/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.5/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Expand [Expand] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Expand [Expand] inputs: [/transformer/h.5/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.5/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Expand for ONNX node: /transformer/h.5/attn/Expand +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Expand_output_0 for ONNX tensor: /transformer/h.5/attn/Expand_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Expand [Expand] outputs: [/transformer/h.5/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Tile [Tile] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Expand_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Tile [Tile] inputs: [/transformer/h.5/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Tile for ONNX node: /transformer/h.5/attn/Tile +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Tile_output_0 for ONNX tensor: /transformer/h.5/attn/Tile_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Tile [Tile] outputs: [/transformer/h.5/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_6 for ONNX node: /transformer/h.5/attn/Unsqueeze_6 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_3 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_3 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_3 for ONNX node: /transformer/h.5/attn/Concat_3 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_3 [Concat] outputs: [/transformer/h.5/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Tile_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_4 [Reshape] inputs: [/transformer/h.5/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.5/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_4 for ONNX node: /transformer/h.5/attn/Reshape_4 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_4 [Reshape] outputs: [/transformer/h.5/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_9 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_9 [Shape] inputs: [/transformer/h.5/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_9 for ONNX node: /transformer/h.5/attn/Shape_9 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_9 [Shape] outputs: [/transformer/h.5/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_9 [Gather] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_9 [Gather] inputs: [/transformer/h.5/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_9 for ONNX node: /transformer/h.5/attn/Gather_9 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_9 [Gather] outputs: [/transformer/h.5/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.5/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_7 for ONNX node: /transformer/h.5/attn/Unsqueeze_7 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_8 for ONNX node: /transformer/h.5/attn/Unsqueeze_8 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_4 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_4 [Slice] inputs: [/transformer/h.5/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_4 for ONNX node: /transformer/h.5/attn/Slice_4 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_4 [Slice] outputs: [/transformer/h.5/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.5/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_9 for ONNX node: /transformer/h.5/attn/Unsqueeze_9 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_10 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Cos_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_10 [Shape] inputs: [/transformer/h.5/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_10 for ONNX node: /transformer/h.5/attn/Shape_10 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_10 [Shape] outputs: [/transformer/h.5/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_10 [Gather] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_10 [Gather] inputs: [/transformer/h.5/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_10 for ONNX node: /transformer/h.5/attn/Gather_10 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_10 [Gather] outputs: [/transformer/h.5/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Cos_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_5 [Reshape] inputs: [/transformer/h.5/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_5 for ONNX node: /transformer/h.5/attn/Reshape_5 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_5 [Reshape] outputs: [/transformer/h.5/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/ConstantOfShape_1 for ONNX node: /transformer/h.5/attn/ConstantOfShape_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.5/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.5/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Expand_1 [Expand] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Expand_1 [Expand] inputs: [/transformer/h.5/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.5/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Expand_1 for ONNX node: /transformer/h.5/attn/Expand_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.5/attn/Expand_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Expand_1 [Expand] outputs: [/transformer/h.5/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Tile_1 [Tile] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Expand_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Tile_1 [Tile] inputs: [/transformer/h.5/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Tile_1 for ONNX node: /transformer/h.5/attn/Tile_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.5/attn/Tile_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Tile_1 [Tile] outputs: [/transformer/h.5/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_10 for ONNX node: /transformer/h.5/attn/Unsqueeze_10 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_4 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_4 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_4 for ONNX node: /transformer/h.5/attn/Concat_4 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_4 [Concat] outputs: [/transformer/h.5/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Tile_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_6 [Reshape] inputs: [/transformer/h.5/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.5/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_6 for ONNX node: /transformer/h.5/attn/Reshape_6 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_6 [Reshape] outputs: [/transformer/h.5/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.5/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_11 for ONNX node: /transformer/h.5/attn/Unsqueeze_11 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_12 for ONNX node: /transformer/h.5/attn/Unsqueeze_12 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_5 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_5 [Slice] inputs: [/transformer/h.5/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_5 for ONNX node: /transformer/h.5/attn/Slice_5 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_5 [Slice] outputs: [/transformer/h.5/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.5/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_13 for ONNX node: /transformer/h.5/attn/Unsqueeze_13 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Mul_1 [Mul] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_1 [Mul] inputs: [/transformer/h.5/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Mul_1 for ONNX node: /transformer/h.5/attn/Mul_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.5/attn/Mul_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_1 [Mul] outputs: [/transformer/h.5/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_6 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_6 [Slice] inputs: [/transformer/h.5/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_6 for ONNX node: /transformer/h.5/attn/Slice_6 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_6 [Slice] outputs: [/transformer/h.5/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_7 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_7 [Slice] inputs: [/transformer/h.5/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_7 for ONNX node: /transformer/h.5/attn/Slice_7 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_7 [Slice] outputs: [/transformer/h.5/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Neg [Neg] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Neg [Neg] inputs: [/transformer/h.5/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Neg for ONNX node: /transformer/h.5/attn/Neg +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Neg_output_0 for ONNX tensor: /transformer/h.5/attn/Neg_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Neg [Neg] outputs: [/transformer/h.5/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Neg_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.5/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_14 for ONNX node: /transformer/h.5/attn/Unsqueeze_14 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.5/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_15 for ONNX node: /transformer/h.5/attn/Unsqueeze_15 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_5 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_5 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.5/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_5 for ONNX node: /transformer/h.5/attn/Concat_5 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_5 [Concat] outputs: [/transformer/h.5/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_11 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_11 [Shape] inputs: [/transformer/h.5/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_11 for ONNX node: /transformer/h.5/attn/Shape_11 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_11 [Shape] outputs: [/transformer/h.5/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_8 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_8 [Slice] inputs: [/transformer/h.5/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_8 for ONNX node: /transformer/h.5/attn/Slice_8 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_8 [Slice] outputs: [/transformer/h.5/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_6 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_6 [Concat] inputs: [/transformer/h.5/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_6 for ONNX node: /transformer/h.5/attn/Concat_6 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_6 [Concat] outputs: [/transformer/h.5/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_7 [Reshape] inputs: [/transformer/h.5/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.5/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_7 for ONNX node: /transformer/h.5/attn/Reshape_7 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_7 [Reshape] outputs: [/transformer/h.5/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Mul_2 [Mul] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_2 [Mul] inputs: [/transformer/h.5/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Mul_2 for ONNX node: /transformer/h.5/attn/Mul_2 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.5/attn/Mul_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_2 [Mul] outputs: [/transformer/h.5/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Add [Add] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Mul_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Mul_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Add [Add] inputs: [/transformer/h.5/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Add for ONNX node: /transformer/h.5/attn/Add +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Add_output_0 for ONNX tensor: /transformer/h.5/attn/Add_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Add [Add] outputs: [/transformer/h.5/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_12 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_12 [Shape] inputs: [/transformer/h.5/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_12 for ONNX node: /transformer/h.5/attn/Shape_12 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_12_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_12 [Shape] outputs: [/transformer/h.5/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_11 [Gather] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_12_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_11 [Gather] inputs: [/transformer/h.5/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_11 for ONNX node: /transformer/h.5/attn/Gather_11 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Gather_11 [Gather] outputs: [/transformer/h.5/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_16 for ONNX node: /transformer/h.5/attn/Unsqueeze_16 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_9 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_9 [Slice] inputs: [/transformer/h.5/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_9 for ONNX node: /transformer/h.5/attn/Slice_9 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_9 [Slice] outputs: [/transformer/h.5/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.5/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_17 for ONNX node: /transformer/h.5/attn/Unsqueeze_17 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_18 for ONNX node: /transformer/h.5/attn/Unsqueeze_18 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_10 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_10 [Slice] inputs: [/transformer/h.5/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_10 for ONNX node: /transformer/h.5/attn/Slice_10 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_10 [Slice] outputs: [/transformer/h.5/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.5/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_19 for ONNX node: /transformer/h.5/attn/Unsqueeze_19 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Mul_3 [Mul] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_3 [Mul] inputs: [/transformer/h.5/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Mul_3 for ONNX node: /transformer/h.5/attn/Mul_3 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.5/attn/Mul_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_3 [Mul] outputs: [/transformer/h.5/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_11 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_11 [Slice] inputs: [/transformer/h.5/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_11 for ONNX node: /transformer/h.5/attn/Slice_11 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_11 [Slice] outputs: [/transformer/h.5/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_12 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_12 [Slice] inputs: [/transformer/h.5/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_12 for ONNX node: /transformer/h.5/attn/Slice_12 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_12_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_12 [Slice] outputs: [/transformer/h.5/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Neg_1 [Neg] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_12_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Neg_1 [Neg] inputs: [/transformer/h.5/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Neg_1 for ONNX node: /transformer/h.5/attn/Neg_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.5/attn/Neg_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Neg_1 [Neg] outputs: [/transformer/h.5/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Neg_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.5/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_20 for ONNX node: /transformer/h.5/attn/Unsqueeze_20 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_11_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.5/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_21 for ONNX node: /transformer/h.5/attn/Unsqueeze_21 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_7 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_7 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.5/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_7 for ONNX node: /transformer/h.5/attn/Concat_7 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_7 [Concat] outputs: [/transformer/h.5/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_13 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_13 [Shape] inputs: [/transformer/h.5/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_13 for ONNX node: /transformer/h.5/attn/Shape_13 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_13 [Shape] outputs: [/transformer/h.5/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_13 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_13 [Slice] inputs: [/transformer/h.5/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_13 for ONNX node: /transformer/h.5/attn/Slice_13 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_13 [Slice] outputs: [/transformer/h.5/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_8 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_8 [Concat] inputs: [/transformer/h.5/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_8 for ONNX node: /transformer/h.5/attn/Concat_8 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_8 [Concat] outputs: [/transformer/h.5/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_7_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_8 [Reshape] inputs: [/transformer/h.5/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.5/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_8 for ONNX node: /transformer/h.5/attn/Reshape_8 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Reshape_8 [Reshape] outputs: [/transformer/h.5/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Mul_4 [Mul] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_8_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_4 [Mul] inputs: [/transformer/h.5/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Mul_4 for ONNX node: /transformer/h.5/attn/Mul_4 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.5/attn/Mul_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Mul_4 [Mul] outputs: [/transformer/h.5/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Add_1 [Add] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Mul_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Mul_4_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Add_1 [Add] inputs: [/transformer/h.5/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Add_1 for ONNX node: /transformer/h.5/attn/Add_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Add_1_output_0 for ONNX tensor: /transformer/h.5/attn/Add_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Add_1 [Add] outputs: [/transformer/h.5/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_9 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Add_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_9 [Concat] inputs: [/transformer/h.5/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_9 for ONNX node: /transformer/h.5/attn/Concat_9 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_9 [Concat] outputs: [/transformer/h.5/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_10 [Concat] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Add_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_3_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_10 [Concat] inputs: [/transformer/h.5/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.5/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_10 for ONNX node: /transformer/h.5/attn/Concat_10 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Concat_10 [Concat] outputs: [/transformer/h.5/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_9_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Transpose_1 [Transpose] inputs: [/transformer/h.5/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Transpose_1 for ONNX node: /transformer/h.5/attn/Transpose_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: present.5.key_1014 for ONNX tensor: present.5.key +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Transpose_1 [Transpose] outputs: [present.5.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_10_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Transpose_2 [Transpose] inputs: [/transformer/h.5/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Transpose_2 for ONNX node: /transformer/h.5/attn/Transpose_2 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.5/attn/Transpose_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Transpose_2 [Transpose] outputs: [/transformer/h.5/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_14 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Transpose_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_14 [Shape] inputs: [/transformer/h.5/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_14 for ONNX node: /transformer/h.5/attn/Shape_14 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_14_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_14 [Shape] outputs: [/transformer/h.5/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_14 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_14_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_14 [Slice] inputs: [/transformer/h.5/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_14 for ONNX node: /transformer/h.5/attn/Slice_14 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_14_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_14 [Slice] outputs: [/transformer/h.5/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Squeeze [Squeeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_14_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Squeeze [Squeeze] inputs: [/transformer/h.5/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Squeeze for ONNX node: /transformer/h.5/attn/Squeeze +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.5/attn/Squeeze_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Squeeze [Squeeze] outputs: [/transformer/h.5/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_15 [Shape] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: present.5.key +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_15 [Shape] inputs: [present.5.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_15 for ONNX node: /transformer/h.5/attn/Shape_15 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Shape_15 [Shape] outputs: [/transformer/h.5/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_15 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_15 [Slice] inputs: [/transformer/h.5/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_15 for ONNX node: /transformer/h.5/attn/Slice_15 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_15 [Slice] outputs: [/transformer/h.5/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_15_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.5/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Squeeze_1 for ONNX node: /transformer/h.5/attn/Squeeze_1 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.5/attn/Squeeze_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.5/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Sub [Sub] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Squeeze_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Squeeze_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Sub [Sub] inputs: [/transformer/h.5/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.5/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Sub for ONNX node: /transformer/h.5/attn/Sub +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Sub_output_0 for ONNX tensor: /transformer/h.5/attn/Sub_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Sub [Sub] outputs: [/transformer/h.5/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Sub_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.5/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_22 for ONNX node: /transformer/h.5/attn/Unsqueeze_22 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Squeeze_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.5/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_23 for ONNX node: /transformer/h.5/attn/Unsqueeze_23 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_16 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.5/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_16 for ONNX node: /transformer/h.5/attn/Slice_16 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_16_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_16 [Slice] outputs: [/transformer/h.5/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Squeeze_1_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.5/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:21] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_24 for ONNX node: /transformer/h.5/attn/Unsqueeze_24 +[12/30/2022-19:54:21] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:21] [V] [TRT] Parsing node: /transformer/h.5/attn/Slice_17 [Slice] +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_16_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:21] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:21] [V] [TRT] /transformer/h.5/attn/Slice_17 [Slice] inputs: [/transformer/h.5/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Slice_17 for ONNX node: /transformer/h.5/attn/Slice_17 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.5/attn/Slice_17_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Slice_17 [Slice] outputs: [/transformer/h.5/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Transpose_3 [Transpose] inputs: [/transformer/h.5/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Transpose_3 for ONNX node: /transformer/h.5/attn/Transpose_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.5/attn/Transpose_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Transpose_3 [Transpose] outputs: [/transformer/h.5/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Transpose_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Transpose_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/MatMul [MatMul] inputs: [/transformer/h.5/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.5/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/MatMul for ONNX node: /transformer/h.5/attn/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/MatMul_output_0 for ONNX tensor: /transformer/h.5/attn/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/MatMul [MatMul] outputs: [/transformer/h.5/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Where [Where] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Slice_17_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Where [Where] inputs: [/transformer/h.5/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.5/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Where for ONNX node: /transformer/h.5/attn/Where +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Where_output_0 for ONNX tensor: /transformer/h.5/attn/Where_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Where [Where] outputs: [/transformer/h.5/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Div_1 [Div] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Where_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Div_1 [Div] inputs: [/transformer/h.5/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Div_1 for ONNX node: /transformer/h.5/attn/Div_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Div_1_output_0 for ONNX tensor: /transformer/h.5/attn/Div_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Div_1 [Div] outputs: [/transformer/h.5/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Add_2 [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Div_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Add_2 [Add] inputs: [/transformer/h.5/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Add_2 for ONNX node: /transformer/h.5/attn/Add_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Add_2_output_0 for ONNX tensor: /transformer/h.5/attn/Add_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Add_2 [Add] outputs: [/transformer/h.5/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Softmax [Softmax] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Add_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Softmax [Softmax] inputs: [/transformer/h.5/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Softmax for ONNX node: /transformer/h.5/attn/Softmax +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Softmax_output_0 for ONNX tensor: /transformer/h.5/attn/Softmax_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Softmax [Softmax] outputs: [/transformer/h.5/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Softmax_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: present.5.value +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/MatMul_1 [MatMul] inputs: [/transformer/h.5/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.5.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/MatMul_1 for ONNX node: /transformer/h.5/attn/MatMul_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.5/attn/MatMul_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/MatMul_1 [MatMul] outputs: [/transformer/h.5/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/MatMul_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Transpose_4 [Transpose] inputs: [/transformer/h.5/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Transpose_4 for ONNX node: /transformer/h.5/attn/Transpose_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.5/attn/Transpose_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Transpose_4 [Transpose] outputs: [/transformer/h.5/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_16 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Transpose_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Shape_16 [Shape] inputs: [/transformer/h.5/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_16 for ONNX node: /transformer/h.5/attn/Shape_16 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_16_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Shape_16 [Shape] outputs: [/transformer/h.5/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_12 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_16_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Gather_12 [Gather] inputs: [/transformer/h.5/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_12 for ONNX node: /transformer/h.5/attn/Gather_12 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_12_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Gather_12 [Gather] outputs: [/transformer/h.5/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Shape_17 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Transpose_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Shape_17 [Shape] inputs: [/transformer/h.5/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Shape_17 for ONNX node: /transformer/h.5/attn/Shape_17 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.5/attn/Shape_17_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Shape_17 [Shape] outputs: [/transformer/h.5/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Gather_13 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Shape_17_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Gather_13 [Gather] inputs: [/transformer/h.5/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Gather_13 for ONNX node: /transformer/h.5/attn/Gather_13 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.5/attn/Gather_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Gather_13 [Gather] outputs: [/transformer/h.5/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_12_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_25 for ONNX node: /transformer/h.5/attn/Unsqueeze_25 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Gather_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.5/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Unsqueeze_26 for ONNX node: /transformer/h.5/attn/Unsqueeze_26 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.5/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.5/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Concat_11 [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Concat_11 [Concat] inputs: [/transformer/h.5/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.5/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Concat_11 for ONNX node: /transformer/h.5/attn/Concat_11 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.5/attn/Concat_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Concat_11 [Concat] outputs: [/transformer/h.5/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Transpose_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Concat_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Reshape_9 [Reshape] inputs: [/transformer/h.5/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.5/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/Reshape_9 for ONNX node: /transformer/h.5/attn/Reshape_9 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.5/attn/Reshape_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/Reshape_9 [Reshape] outputs: [/transformer/h.5/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/Reshape_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::MatMul_3001 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.5/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3001 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: onnx::MatMul_3001 for ONNX node: onnx::MatMul_3001 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/attn/out_proj/MatMul for ONNX node: /transformer/h.5/attn/out_proj/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.5/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.5/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/ln_1/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::MatMul_3002 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.5/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3002 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: onnx::MatMul_3002 for ONNX node: onnx::MatMul_3002 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/fc_in/MatMul for ONNX node: /transformer/h.5/mlp/fc_in/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.5/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.5/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/fc_in/Add [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.5/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/fc_in/Add for ONNX node: /transformer/h.5/mlp/fc_in/Add +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.5/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_in/Add [Add] outputs: [/transformer/h.5/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Mul [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul [Mul] inputs: [/transformer/h.5/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Mul for ONNX node: /transformer/h.5/mlp/act/Mul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul [Mul] outputs: [/transformer/h.5/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Pow [Pow] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Pow [Pow] inputs: [/transformer/h.5/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Pow for ONNX node: /transformer/h.5/mlp/act/Pow +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Pow_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Pow [Pow] outputs: [/transformer/h.5/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Pow_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.5/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Mul_1 for ONNX node: /transformer/h.5/mlp/act/Mul_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.5/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Add [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Add [Add] inputs: [/transformer/h.5/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.5/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Add for ONNX node: /transformer/h.5/mlp/act/Add +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Add [Add] outputs: [/transformer/h.5/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.5/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Mul_2 for ONNX node: /transformer/h.5/mlp/act/Mul_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.5/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Tanh [Tanh] inputs: [/transformer/h.5/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Tanh for ONNX node: /transformer/h.5/mlp/act/Tanh +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Tanh_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Tanh [Tanh] outputs: [/transformer/h.5/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Add_1 [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Tanh_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Add_1 [Add] inputs: [/transformer/h.5/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Add_1 for ONNX node: /transformer/h.5/mlp/act/Add_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Add_1 [Add] outputs: [/transformer/h.5/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.5/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.5/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/act/Mul_3 for ONNX node: /transformer/h.5/mlp/act/Mul_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.5/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.5/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::MatMul_3003 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.5/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_3003 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: onnx::MatMul_3003 for ONNX node: onnx::MatMul_3003 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/fc_out/MatMul for ONNX node: /transformer/h.5/mlp/fc_out/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.5/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.5/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/mlp/fc_out/Add [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.5/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/mlp/fc_out/Add for ONNX node: /transformer/h.5/mlp/fc_out/Add +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.5/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/mlp/fc_out/Add [Add] outputs: [/transformer/h.5/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/Add [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/Add [Add] inputs: [/transformer/h.5/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/Add for ONNX node: /transformer/h.5/Add +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/Add_output_0 for ONNX tensor: /transformer/h.5/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/Add [Add] outputs: [/transformer/h.5/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.5/Add_1 [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.4/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/Add_1 [Add] inputs: [/transformer/h.5/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.4/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.5/Add_1 for ONNX node: /transformer/h.5/Add_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.5/Add_1_output_0 for ONNX tensor: /transformer/h.5/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.5/Add_1 [Add] outputs: [/transformer/h.5/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.5/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/ReduceMean for ONNX node: /transformer/h.6/ln_1/ReduceMean +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.6/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.6/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Sub [Sub] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.5/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Sub [Sub] inputs: [/transformer/h.5/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Sub for ONNX node: /transformer/h.6/ln_1/Sub +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.6/ln_1/Sub_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Sub [Sub] outputs: [/transformer/h.6/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Pow [Pow] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Sub_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Pow [Pow] inputs: [/transformer/h.6/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Pow for ONNX node: /transformer/h.6/ln_1/Pow +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.6/ln_1/Pow_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Pow [Pow] outputs: [/transformer/h.6/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Pow_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.6/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/ReduceMean_1 for ONNX node: /transformer/h.6/ln_1/ReduceMean_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.6/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.6/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Add [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Add [Add] inputs: [/transformer/h.6/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Add for ONNX node: /transformer/h.6/ln_1/Add +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Add_output_0 for ONNX tensor: /transformer/h.6/ln_1/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Add [Add] outputs: [/transformer/h.6/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Add_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.6/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Sqrt for ONNX node: /transformer/h.6/ln_1/Sqrt +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.6/ln_1/Sqrt_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.6/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Div [Div] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Sub_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Sqrt_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Div [Div] inputs: [/transformer/h.6/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Div for ONNX node: /transformer/h.6/ln_1/Div +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Div_output_0 for ONNX tensor: /transformer/h.6/ln_1/Div_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Div [Div] outputs: [/transformer/h.6/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Mul [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Div_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Mul [Mul] inputs: [/transformer/h.6/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Mul for ONNX node: /transformer/h.6/ln_1/Mul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.6/ln_1/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Mul [Mul] outputs: [/transformer/h.6/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/ln_1/Add_1 [Add] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Add_1 [Add] inputs: [/transformer/h.6/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/ln_1/Add_1 for ONNX node: /transformer/h.6/ln_1/Add_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.6/ln_1/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/ln_1/Add_1 [Add] outputs: [/transformer/h.6/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::MatMul_3004 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.6/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3004 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: onnx::MatMul_3004 for ONNX node: onnx::MatMul_3004 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/q_proj/MatMul for ONNX node: /transformer/h.6/attn/q_proj/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.6/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.6/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::MatMul_3005 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.6/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3005 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: onnx::MatMul_3005 for ONNX node: onnx::MatMul_3005 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/k_proj/MatMul for ONNX node: /transformer/h.6/attn/k_proj/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.6/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.6/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Add_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::MatMul_3006 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.6/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3006 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: onnx::MatMul_3006 for ONNX node: onnx::MatMul_3006 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/v_proj/MatMul for ONNX node: /transformer/h.6/attn/v_proj/MatMul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.6/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.6/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape [Shape] inputs: [/transformer/h.6/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape for ONNX node: /transformer/h.6/attn/Shape +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape [Shape] outputs: [/transformer/h.6/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather [Gather] inputs: [/transformer/h.6/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather for ONNX node: /transformer/h.6/attn/Gather +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather [Gather] outputs: [/transformer/h.6/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_1 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_1 [Shape] inputs: [/transformer/h.6/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_1 for ONNX node: /transformer/h.6/attn/Shape_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_1 [Shape] outputs: [/transformer/h.6/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_1 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_1 [Gather] inputs: [/transformer/h.6/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_1 for ONNX node: /transformer/h.6/attn/Gather_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_1 [Gather] outputs: [/transformer/h.6/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze for ONNX node: /transformer/h.6/attn/Unsqueeze +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_1 for ONNX node: /transformer/h.6/attn/Unsqueeze_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat for ONNX node: /transformer/h.6/attn/Concat +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat [Concat] outputs: [/transformer/h.6/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape [Reshape] inputs: [/transformer/h.6/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape for ONNX node: /transformer/h.6/attn/Reshape +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape [Reshape] outputs: [/transformer/h.6/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_2 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_2 [Shape] inputs: [/transformer/h.6/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_2 for ONNX node: /transformer/h.6/attn/Shape_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_2 [Shape] outputs: [/transformer/h.6/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_2 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_2 [Gather] inputs: [/transformer/h.6/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_2 for ONNX node: /transformer/h.6/attn/Gather_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_2 [Gather] outputs: [/transformer/h.6/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_3 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_3 [Shape] inputs: [/transformer/h.6/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_3 for ONNX node: /transformer/h.6/attn/Shape_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_3 [Shape] outputs: [/transformer/h.6/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_3 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_3 [Gather] inputs: [/transformer/h.6/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_3 for ONNX node: /transformer/h.6/attn/Gather_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_3 [Gather] outputs: [/transformer/h.6/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_2 for ONNX node: /transformer/h.6/attn/Unsqueeze_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_3 for ONNX node: /transformer/h.6/attn/Unsqueeze_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_1 [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_1 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_1 for ONNX node: /transformer/h.6/attn/Concat_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_1 [Concat] outputs: [/transformer/h.6/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_1 [Reshape] inputs: [/transformer/h.6/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_1 for ONNX node: /transformer/h.6/attn/Reshape_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_1 [Reshape] outputs: [/transformer/h.6/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_4 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_4 [Shape] inputs: [/transformer/h.6/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_4 for ONNX node: /transformer/h.6/attn/Shape_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_4 [Shape] outputs: [/transformer/h.6/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_4 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_4 [Gather] inputs: [/transformer/h.6/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_4 for ONNX node: /transformer/h.6/attn/Gather_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_4 [Gather] outputs: [/transformer/h.6/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_5 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_5 [Shape] inputs: [/transformer/h.6/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_5 for ONNX node: /transformer/h.6/attn/Shape_5 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_5 [Shape] outputs: [/transformer/h.6/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_5 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_5 [Gather] inputs: [/transformer/h.6/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_5 for ONNX node: /transformer/h.6/attn/Gather_5 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_5 [Gather] outputs: [/transformer/h.6/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_4 for ONNX node: /transformer/h.6/attn/Unsqueeze_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_5 for ONNX node: /transformer/h.6/attn/Unsqueeze_5 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_2 [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_2 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_2 for ONNX node: /transformer/h.6/attn/Concat_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_2 [Concat] outputs: [/transformer/h.6/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_2 [Reshape] inputs: [/transformer/h.6/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_2 for ONNX node: /transformer/h.6/attn/Reshape_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_2 [Reshape] outputs: [/transformer/h.6/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Transpose [Transpose] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Transpose [Transpose] inputs: [/transformer/h.6/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Transpose for ONNX node: /transformer/h.6/attn/Transpose +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: present.6.value_1034 for ONNX tensor: present.6.value +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Transpose [Transpose] outputs: [present.6.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_6 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_6 [Shape] inputs: [/transformer/h.6/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_6 for ONNX node: /transformer/h.6/attn/Shape_6 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_6 [Shape] outputs: [/transformer/h.6/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_6 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_6 [Gather] inputs: [/transformer/h.6/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_6 for ONNX node: /transformer/h.6/attn/Gather_6 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_6 [Gather] outputs: [/transformer/h.6/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice [Slice] inputs: [/transformer/h.6/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice for ONNX node: /transformer/h.6/attn/Slice +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice [Slice] outputs: [/transformer/h.6/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_1 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_1 [Slice] inputs: [/transformer/h.6/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_1 for ONNX node: /transformer/h.6/attn/Slice_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_1 [Slice] outputs: [/transformer/h.6/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_2 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_2 [Slice] inputs: [/transformer/h.6/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_2 for ONNX node: /transformer/h.6/attn/Slice_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_2 [Slice] outputs: [/transformer/h.6/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_3 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_3 [Slice] inputs: [/transformer/h.6/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_3 for ONNX node: /transformer/h.6/attn/Slice_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_3 [Slice] outputs: [/transformer/h.6/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_7 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_7 [Shape] inputs: [/transformer/h.6/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_7 for ONNX node: /transformer/h.6/attn/Shape_7 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_7 [Shape] outputs: [/transformer/h.6/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_7 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_7 [Gather] inputs: [/transformer/h.6/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_7 for ONNX node: /transformer/h.6/attn/Gather_7 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_7 [Gather] outputs: [/transformer/h.6/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Range [Range] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.6/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Range for ONNX node: /transformer/h.6/attn/Range +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Range_output_0 for ONNX tensor: /transformer/h.6/attn/Range_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Range [Range] outputs: [/transformer/h.6/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Cast_1 [Cast] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Range_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cast_1 [Cast] inputs: [/transformer/h.6/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Cast_1 for ONNX node: /transformer/h.6/attn/Cast_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.6/attn/Cast_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cast_1 [Cast] outputs: [/transformer/h.6/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Cast_2 [Cast] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cast_2 [Cast] inputs: [/transformer/h.6/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Cast_2 for ONNX node: /transformer/h.6/attn/Cast_2 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.6/attn/Cast_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cast_2 [Cast] outputs: [/transformer/h.6/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Div [Div] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Cast_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Cast_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Div [Div] inputs: [/transformer/h.6/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.6/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Div for ONNX node: /transformer/h.6/attn/Div +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Div_output_0 for ONNX tensor: /transformer/h.6/attn/Div_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Div [Div] outputs: [/transformer/h.6/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Pow [Pow] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Div_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.6/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Pow for ONNX node: /transformer/h.6/attn/Pow +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Pow_output_0 for ONNX tensor: /transformer/h.6/attn/Pow_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Pow [Pow] outputs: [/transformer/h.6/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Pow_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.6/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reciprocal for ONNX node: /transformer/h.6/attn/Reciprocal +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.6/attn/Reciprocal_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.6/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Mul [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reciprocal_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Mul [Mul] inputs: [/transformer/h.6/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Mul for ONNX node: /transformer/h.6/attn/Mul +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Mul_output_0 for ONNX tensor: /transformer/h.6/attn/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Mul [Mul] outputs: [/transformer/h.6/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Cast_3 [Cast] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cast_3 [Cast] inputs: [/transformer/h.6/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Cast_3 for ONNX node: /transformer/h.6/attn/Cast_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.6/attn/Cast_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cast_3 [Cast] outputs: [/transformer/h.6/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Range_1 [Range] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Cast_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.6/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Range_1 for ONNX node: /transformer/h.6/attn/Range_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Range_1_output_0 for ONNX tensor: /transformer/h.6/attn/Range_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Range_1 [Range] outputs: [/transformer/h.6/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Einsum [Einsum] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Range_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Mul_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Einsum [Einsum] inputs: [/transformer/h.6/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.6/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Einsum for ONNX node: /transformer/h.6/attn/Einsum +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Einsum_output_0 for ONNX tensor: /transformer/h.6/attn/Einsum_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Einsum [Einsum] outputs: [/transformer/h.6/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Sin [Sin] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Einsum_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Sin [Sin] inputs: [/transformer/h.6/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Sin for ONNX node: /transformer/h.6/attn/Sin +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Sin_output_0 for ONNX tensor: /transformer/h.6/attn/Sin_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Sin [Sin] outputs: [/transformer/h.6/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Cos [Cos] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Einsum_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cos [Cos] inputs: [/transformer/h.6/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Cos for ONNX node: /transformer/h.6/attn/Cos +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Cos_output_0 for ONNX tensor: /transformer/h.6/attn/Cos_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Cos [Cos] outputs: [/transformer/h.6/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_8 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Sin_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_8 [Shape] inputs: [/transformer/h.6/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_8 for ONNX node: /transformer/h.6/attn/Shape_8 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_8 [Shape] outputs: [/transformer/h.6/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_8 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_8 [Gather] inputs: [/transformer/h.6/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_8 for ONNX node: /transformer/h.6/attn/Gather_8 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_8 [Gather] outputs: [/transformer/h.6/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Sin_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_3 [Reshape] inputs: [/transformer/h.6/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_3 for ONNX node: /transformer/h.6/attn/Reshape_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_3 [Reshape] outputs: [/transformer/h.6/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/ConstantOfShape for ONNX node: /transformer/h.6/attn/ConstantOfShape +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.6/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.6/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Expand [Expand] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Expand [Expand] inputs: [/transformer/h.6/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.6/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Expand for ONNX node: /transformer/h.6/attn/Expand +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Expand_output_0 for ONNX tensor: /transformer/h.6/attn/Expand_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Expand [Expand] outputs: [/transformer/h.6/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Tile [Tile] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Expand_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Tile [Tile] inputs: [/transformer/h.6/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Tile for ONNX node: /transformer/h.6/attn/Tile +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Tile_output_0 for ONNX tensor: /transformer/h.6/attn/Tile_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Tile [Tile] outputs: [/transformer/h.6/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_6 for ONNX node: /transformer/h.6/attn/Unsqueeze_6 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_3 [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_3 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_3 for ONNX node: /transformer/h.6/attn/Concat_3 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_3 [Concat] outputs: [/transformer/h.6/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Tile_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_3_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_4 [Reshape] inputs: [/transformer/h.6/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.6/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_4 for ONNX node: /transformer/h.6/attn/Reshape_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_4 [Reshape] outputs: [/transformer/h.6/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_9 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_9 [Shape] inputs: [/transformer/h.6/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_9 for ONNX node: /transformer/h.6/attn/Shape_9 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_9 [Shape] outputs: [/transformer/h.6/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_9 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_9 [Gather] inputs: [/transformer/h.6/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_9 for ONNX node: /transformer/h.6/attn/Gather_9 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_9 [Gather] outputs: [/transformer/h.6/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.6/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_7 for ONNX node: /transformer/h.6/attn/Unsqueeze_7 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_8 for ONNX node: /transformer/h.6/attn/Unsqueeze_8 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_4 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_4 [Slice] inputs: [/transformer/h.6/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_4 for ONNX node: /transformer/h.6/attn/Slice_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_4 [Slice] outputs: [/transformer/h.6/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.6/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_9 for ONNX node: /transformer/h.6/attn/Unsqueeze_9 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_10 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Cos_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_10 [Shape] inputs: [/transformer/h.6/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_10 for ONNX node: /transformer/h.6/attn/Shape_10 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_10_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_10 [Shape] outputs: [/transformer/h.6/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_10 [Gather] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_10_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_10 [Gather] inputs: [/transformer/h.6/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_10 for ONNX node: /transformer/h.6/attn/Gather_10 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_10_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Gather_10 [Gather] outputs: [/transformer/h.6/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Cos_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_5 [Reshape] inputs: [/transformer/h.6/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_5 for ONNX node: /transformer/h.6/attn/Reshape_5 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_5 [Reshape] outputs: [/transformer/h.6/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/ConstantOfShape_1 for ONNX node: /transformer/h.6/attn/ConstantOfShape_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.6/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.6/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Expand_1 [Expand] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Expand_1 [Expand] inputs: [/transformer/h.6/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.6/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Expand_1 for ONNX node: /transformer/h.6/attn/Expand_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.6/attn/Expand_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Expand_1 [Expand] outputs: [/transformer/h.6/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Tile_1 [Tile] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Expand_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Tile_1 [Tile] inputs: [/transformer/h.6/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Tile_1 for ONNX node: /transformer/h.6/attn/Tile_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.6/attn/Tile_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Tile_1 [Tile] outputs: [/transformer/h.6/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_10_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_10 for ONNX node: /transformer/h.6/attn/Unsqueeze_10 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_4 [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_4 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_4 for ONNX node: /transformer/h.6/attn/Concat_4 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_4 [Concat] outputs: [/transformer/h.6/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Tile_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_4_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_6 [Reshape] inputs: [/transformer/h.6/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.6/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_6 for ONNX node: /transformer/h.6/attn/Reshape_6 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Reshape_6 [Reshape] outputs: [/transformer/h.6/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.6/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_11 for ONNX node: /transformer/h.6/attn/Unsqueeze_11 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_9_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_12 for ONNX node: /transformer/h.6/attn/Unsqueeze_12 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_5 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_5 [Slice] inputs: [/transformer/h.6/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_5 for ONNX node: /transformer/h.6/attn/Slice_5 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_5 [Slice] outputs: [/transformer/h.6/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.6/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_13 for ONNX node: /transformer/h.6/attn/Unsqueeze_13 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Mul_1 [Mul] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Mul_1 [Mul] inputs: [/transformer/h.6/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Mul_1 for ONNX node: /transformer/h.6/attn/Mul_1 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.6/attn/Mul_1_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Mul_1 [Mul] outputs: [/transformer/h.6/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_6 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_6 [Slice] inputs: [/transformer/h.6/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_6 for ONNX node: /transformer/h.6/attn/Slice_6 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_6 [Slice] outputs: [/transformer/h.6/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_7 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_7 [Slice] inputs: [/transformer/h.6/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_7 for ONNX node: /transformer/h.6/attn/Slice_7 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_7 [Slice] outputs: [/transformer/h.6/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Neg [Neg] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_7_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Neg [Neg] inputs: [/transformer/h.6/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Neg for ONNX node: /transformer/h.6/attn/Neg +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Neg_output_0 for ONNX tensor: /transformer/h.6/attn/Neg_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Neg [Neg] outputs: [/transformer/h.6/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Neg_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.6/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_14 for ONNX node: /transformer/h.6/attn/Unsqueeze_14 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_6_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.6/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_15 for ONNX node: /transformer/h.6/attn/Unsqueeze_15 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_5 [Concat] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_5 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.6/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_5 for ONNX node: /transformer/h.6/attn/Concat_5 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Concat_5 [Concat] outputs: [/transformer/h.6/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_11 [Shape] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_5_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_11 [Shape] inputs: [/transformer/h.6/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_11 for ONNX node: /transformer/h.6/attn/Shape_11 +[12/30/2022-19:54:22] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Shape_11 [Shape] outputs: [/transformer/h.6/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_8 [Slice] +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_11_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:22] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:22] [V] [TRT] /transformer/h.6/attn/Slice_8 [Slice] inputs: [/transformer/h.6/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:22] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_8 for ONNX node: /transformer/h.6/attn/Slice_8 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_8 [Slice] outputs: [/transformer/h.6/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_6 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_6 [Concat] inputs: [/transformer/h.6/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_6 for ONNX node: /transformer/h.6/attn/Concat_6 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_6 [Concat] outputs: [/transformer/h.6/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Reshape_7 [Reshape] inputs: [/transformer/h.6/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.6/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_7 for ONNX node: /transformer/h.6/attn/Reshape_7 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Reshape_7 [Reshape] outputs: [/transformer/h.6/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Mul_2 [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Mul_2 [Mul] inputs: [/transformer/h.6/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Mul_2 for ONNX node: /transformer/h.6/attn/Mul_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.6/attn/Mul_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Mul_2 [Mul] outputs: [/transformer/h.6/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Add [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Mul_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Mul_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Add [Add] inputs: [/transformer/h.6/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Add for ONNX node: /transformer/h.6/attn/Add +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Add_output_0 for ONNX tensor: /transformer/h.6/attn/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Add [Add] outputs: [/transformer/h.6/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_12 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_12 [Shape] inputs: [/transformer/h.6/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_12 for ONNX node: /transformer/h.6/attn/Shape_12 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_12_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_12 [Shape] outputs: [/transformer/h.6/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_11 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_12_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Gather_11 [Gather] inputs: [/transformer/h.6/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_11 for ONNX node: /transformer/h.6/attn/Gather_11 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Gather_11 [Gather] outputs: [/transformer/h.6/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_16 for ONNX node: /transformer/h.6/attn/Unsqueeze_16 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_9 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_9 [Slice] inputs: [/transformer/h.6/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_9 for ONNX node: /transformer/h.6/attn/Slice_9 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_9 [Slice] outputs: [/transformer/h.6/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.6/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_17 for ONNX node: /transformer/h.6/attn/Unsqueeze_17 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_18 for ONNX node: /transformer/h.6/attn/Unsqueeze_18 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_10 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_10 [Slice] inputs: [/transformer/h.6/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_10 for ONNX node: /transformer/h.6/attn/Slice_10 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_10_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_10 [Slice] outputs: [/transformer/h.6/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_10_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.6/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_19 for ONNX node: /transformer/h.6/attn/Unsqueeze_19 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Mul_3 [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Mul_3 [Mul] inputs: [/transformer/h.6/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Mul_3 for ONNX node: /transformer/h.6/attn/Mul_3 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.6/attn/Mul_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Mul_3 [Mul] outputs: [/transformer/h.6/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_11 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_11 [Slice] inputs: [/transformer/h.6/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_11 for ONNX node: /transformer/h.6/attn/Slice_11 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_11 [Slice] outputs: [/transformer/h.6/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_12 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_12 [Slice] inputs: [/transformer/h.6/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_12 for ONNX node: /transformer/h.6/attn/Slice_12 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_12_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_12 [Slice] outputs: [/transformer/h.6/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Neg_1 [Neg] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_12_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Neg_1 [Neg] inputs: [/transformer/h.6/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Neg_1 for ONNX node: /transformer/h.6/attn/Neg_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.6/attn/Neg_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Neg_1 [Neg] outputs: [/transformer/h.6/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Neg_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.6/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_20 for ONNX node: /transformer/h.6/attn/Unsqueeze_20 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.6/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_21 for ONNX node: /transformer/h.6/attn/Unsqueeze_21 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_7 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_7 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.6/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_7 for ONNX node: /transformer/h.6/attn/Concat_7 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_7 [Concat] outputs: [/transformer/h.6/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_13 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_13 [Shape] inputs: [/transformer/h.6/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_13 for ONNX node: /transformer/h.6/attn/Shape_13 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_13 [Shape] outputs: [/transformer/h.6/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_13 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_13 [Slice] inputs: [/transformer/h.6/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_13 for ONNX node: /transformer/h.6/attn/Slice_13 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_13 [Slice] outputs: [/transformer/h.6/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_8 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_8 [Concat] inputs: [/transformer/h.6/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_8 for ONNX node: /transformer/h.6/attn/Concat_8 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_8 [Concat] outputs: [/transformer/h.6/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Reshape_8 [Reshape] inputs: [/transformer/h.6/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.6/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_8 for ONNX node: /transformer/h.6/attn/Reshape_8 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Reshape_8 [Reshape] outputs: [/transformer/h.6/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Mul_4 [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Mul_4 [Mul] inputs: [/transformer/h.6/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Mul_4 for ONNX node: /transformer/h.6/attn/Mul_4 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.6/attn/Mul_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Mul_4 [Mul] outputs: [/transformer/h.6/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Add_1 [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Mul_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Mul_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Add_1 [Add] inputs: [/transformer/h.6/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Add_1 for ONNX node: /transformer/h.6/attn/Add_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Add_1_output_0 for ONNX tensor: /transformer/h.6/attn/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Add_1 [Add] outputs: [/transformer/h.6/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_9 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_9 [Concat] inputs: [/transformer/h.6/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_9 for ONNX node: /transformer/h.6/attn/Concat_9 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_9 [Concat] outputs: [/transformer/h.6/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_10 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_10 [Concat] inputs: [/transformer/h.6/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.6/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_10 for ONNX node: /transformer/h.6/attn/Concat_10 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_10_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_10 [Concat] outputs: [/transformer/h.6/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_1 [Transpose] inputs: [/transformer/h.6/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Transpose_1 for ONNX node: /transformer/h.6/attn/Transpose_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: present.6.key_1183 for ONNX tensor: present.6.key +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_1 [Transpose] outputs: [present.6.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_10_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_2 [Transpose] inputs: [/transformer/h.6/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Transpose_2 for ONNX node: /transformer/h.6/attn/Transpose_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.6/attn/Transpose_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_2 [Transpose] outputs: [/transformer/h.6/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_14 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Transpose_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_14 [Shape] inputs: [/transformer/h.6/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_14 for ONNX node: /transformer/h.6/attn/Shape_14 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_14_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_14 [Shape] outputs: [/transformer/h.6/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_14 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_14_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_14 [Slice] inputs: [/transformer/h.6/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_14 for ONNX node: /transformer/h.6/attn/Slice_14 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_14_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_14 [Slice] outputs: [/transformer/h.6/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Squeeze [Squeeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_14_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Squeeze [Squeeze] inputs: [/transformer/h.6/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Squeeze for ONNX node: /transformer/h.6/attn/Squeeze +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.6/attn/Squeeze_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Squeeze [Squeeze] outputs: [/transformer/h.6/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_15 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: present.6.key +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_15 [Shape] inputs: [present.6.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_15 for ONNX node: /transformer/h.6/attn/Shape_15 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_15 [Shape] outputs: [/transformer/h.6/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_15 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_15 [Slice] inputs: [/transformer/h.6/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_15 for ONNX node: /transformer/h.6/attn/Slice_15 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_15 [Slice] outputs: [/transformer/h.6/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.6/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Squeeze_1 for ONNX node: /transformer/h.6/attn/Squeeze_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.6/attn/Squeeze_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.6/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Sub [Sub] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Squeeze_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Squeeze_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Sub [Sub] inputs: [/transformer/h.6/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.6/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Sub for ONNX node: /transformer/h.6/attn/Sub +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Sub_output_0 for ONNX tensor: /transformer/h.6/attn/Sub_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Sub [Sub] outputs: [/transformer/h.6/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Sub_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.6/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_22 for ONNX node: /transformer/h.6/attn/Unsqueeze_22 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Squeeze_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.6/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_23 for ONNX node: /transformer/h.6/attn/Unsqueeze_23 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_16 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.6/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_16 for ONNX node: /transformer/h.6/attn/Slice_16 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_16_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_16 [Slice] outputs: [/transformer/h.6/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Squeeze_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.6/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_24 for ONNX node: /transformer/h.6/attn/Unsqueeze_24 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Slice_17 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_16_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_17 [Slice] inputs: [/transformer/h.6/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Slice_17 for ONNX node: /transformer/h.6/attn/Slice_17 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.6/attn/Slice_17_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Slice_17 [Slice] outputs: [/transformer/h.6/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_3 [Transpose] inputs: [/transformer/h.6/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Transpose_3 for ONNX node: /transformer/h.6/attn/Transpose_3 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.6/attn/Transpose_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_3 [Transpose] outputs: [/transformer/h.6/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Transpose_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Transpose_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/MatMul [MatMul] inputs: [/transformer/h.6/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.6/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/MatMul for ONNX node: /transformer/h.6/attn/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/MatMul_output_0 for ONNX tensor: /transformer/h.6/attn/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/MatMul [MatMul] outputs: [/transformer/h.6/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Where [Where] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Slice_17_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Where [Where] inputs: [/transformer/h.6/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.6/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Where for ONNX node: /transformer/h.6/attn/Where +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Where_output_0 for ONNX tensor: /transformer/h.6/attn/Where_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Where [Where] outputs: [/transformer/h.6/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Div_1 [Div] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Where_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Div_1 [Div] inputs: [/transformer/h.6/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Div_1 for ONNX node: /transformer/h.6/attn/Div_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Div_1_output_0 for ONNX tensor: /transformer/h.6/attn/Div_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Div_1 [Div] outputs: [/transformer/h.6/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Add_2 [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Div_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Add_2 [Add] inputs: [/transformer/h.6/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Add_2 for ONNX node: /transformer/h.6/attn/Add_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Add_2_output_0 for ONNX tensor: /transformer/h.6/attn/Add_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Add_2 [Add] outputs: [/transformer/h.6/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Softmax [Softmax] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Add_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Softmax [Softmax] inputs: [/transformer/h.6/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Softmax for ONNX node: /transformer/h.6/attn/Softmax +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Softmax_output_0 for ONNX tensor: /transformer/h.6/attn/Softmax_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Softmax [Softmax] outputs: [/transformer/h.6/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Softmax_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: present.6.value +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/MatMul_1 [MatMul] inputs: [/transformer/h.6/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.6.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/MatMul_1 for ONNX node: /transformer/h.6/attn/MatMul_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.6/attn/MatMul_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/MatMul_1 [MatMul] outputs: [/transformer/h.6/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/MatMul_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_4 [Transpose] inputs: [/transformer/h.6/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Transpose_4 for ONNX node: /transformer/h.6/attn/Transpose_4 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.6/attn/Transpose_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Transpose_4 [Transpose] outputs: [/transformer/h.6/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_16 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Transpose_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_16 [Shape] inputs: [/transformer/h.6/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_16 for ONNX node: /transformer/h.6/attn/Shape_16 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_16_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_16 [Shape] outputs: [/transformer/h.6/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_12 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_16_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Gather_12 [Gather] inputs: [/transformer/h.6/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_12 for ONNX node: /transformer/h.6/attn/Gather_12 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_12_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Gather_12 [Gather] outputs: [/transformer/h.6/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Shape_17 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Transpose_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_17 [Shape] inputs: [/transformer/h.6/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Shape_17 for ONNX node: /transformer/h.6/attn/Shape_17 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.6/attn/Shape_17_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Shape_17 [Shape] outputs: [/transformer/h.6/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Gather_13 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Shape_17_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Gather_13 [Gather] inputs: [/transformer/h.6/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Gather_13 for ONNX node: /transformer/h.6/attn/Gather_13 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.6/attn/Gather_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Gather_13 [Gather] outputs: [/transformer/h.6/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_12_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_25 for ONNX node: /transformer/h.6/attn/Unsqueeze_25 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Gather_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.6/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Unsqueeze_26 for ONNX node: /transformer/h.6/attn/Unsqueeze_26 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.6/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.6/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Concat_11 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_11 [Concat] inputs: [/transformer/h.6/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.6/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Concat_11 for ONNX node: /transformer/h.6/attn/Concat_11 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.6/attn/Concat_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Concat_11 [Concat] outputs: [/transformer/h.6/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Transpose_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Concat_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Reshape_9 [Reshape] inputs: [/transformer/h.6/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.6/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/Reshape_9 for ONNX node: /transformer/h.6/attn/Reshape_9 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.6/attn/Reshape_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/Reshape_9 [Reshape] outputs: [/transformer/h.6/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/Reshape_9_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::MatMul_3031 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.6/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3031 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: onnx::MatMul_3031 for ONNX node: onnx::MatMul_3031 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/attn/out_proj/MatMul for ONNX node: /transformer/h.6/attn/out_proj/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.6/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.6/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/ln_1/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::MatMul_3032 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.6/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3032 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: onnx::MatMul_3032 for ONNX node: onnx::MatMul_3032 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/fc_in/MatMul for ONNX node: /transformer/h.6/mlp/fc_in/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.6/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.6/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/fc_in/Add [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.6/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/fc_in/Add for ONNX node: /transformer/h.6/mlp/fc_in/Add +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.6/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_in/Add [Add] outputs: [/transformer/h.6/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Mul [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul [Mul] inputs: [/transformer/h.6/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Mul for ONNX node: /transformer/h.6/mlp/act/Mul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Mul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul [Mul] outputs: [/transformer/h.6/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Pow [Pow] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Pow [Pow] inputs: [/transformer/h.6/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Pow for ONNX node: /transformer/h.6/mlp/act/Pow +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Pow_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Pow [Pow] outputs: [/transformer/h.6/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Pow_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.6/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Mul_1 for ONNX node: /transformer/h.6/mlp/act/Mul_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.6/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Add [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Add [Add] inputs: [/transformer/h.6/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.6/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Add for ONNX node: /transformer/h.6/mlp/act/Add +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Add [Add] outputs: [/transformer/h.6/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.6/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Mul_2 for ONNX node: /transformer/h.6/mlp/act/Mul_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.6/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Tanh [Tanh] inputs: [/transformer/h.6/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Tanh for ONNX node: /transformer/h.6/mlp/act/Tanh +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Tanh_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Tanh [Tanh] outputs: [/transformer/h.6/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Add_1 [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Tanh_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Add_1 [Add] inputs: [/transformer/h.6/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Add_1 for ONNX node: /transformer/h.6/mlp/act/Add_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Add_1 [Add] outputs: [/transformer/h.6/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Mul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.6/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.6/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/act/Mul_3 for ONNX node: /transformer/h.6/mlp/act/Mul_3 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.6/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.6/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::MatMul_3033 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.6/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_3033 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: onnx::MatMul_3033 for ONNX node: onnx::MatMul_3033 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/fc_out/MatMul for ONNX node: /transformer/h.6/mlp/fc_out/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.6/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.6/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/mlp/fc_out/Add [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.6/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/mlp/fc_out/Add for ONNX node: /transformer/h.6/mlp/fc_out/Add +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.6/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/mlp/fc_out/Add [Add] outputs: [/transformer/h.6/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/Add [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/Add [Add] inputs: [/transformer/h.6/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/Add for ONNX node: /transformer/h.6/Add +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/Add_output_0 for ONNX tensor: /transformer/h.6/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/Add [Add] outputs: [/transformer/h.6/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.6/Add_1 [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.5/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/Add_1 [Add] inputs: [/transformer/h.6/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.5/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.6/Add_1 for ONNX node: /transformer/h.6/Add_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.6/Add_1_output_0 for ONNX tensor: /transformer/h.6/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.6/Add_1 [Add] outputs: [/transformer/h.6/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/ReduceMean [ReduceMean] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/ReduceMean [ReduceMean] inputs: [/transformer/h.6/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/ReduceMean for ONNX node: /transformer/h.7/ln_1/ReduceMean +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/ReduceMean_output_0 for ONNX tensor: /transformer/h.7/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/ReduceMean [ReduceMean] outputs: [/transformer/h.7/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Sub [Sub] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.6/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/ReduceMean_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Sub [Sub] inputs: [/transformer/h.6/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.7/ln_1/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Sub for ONNX node: /transformer/h.7/ln_1/Sub +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Sub_output_0 for ONNX tensor: /transformer/h.7/ln_1/Sub_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Sub [Sub] outputs: [/transformer/h.7/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Pow [Pow] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Sub_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Pow [Pow] inputs: [/transformer/h.7/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Pow for ONNX node: /transformer/h.7/ln_1/Pow +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Pow_output_0 for ONNX tensor: /transformer/h.7/ln_1/Pow_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Pow [Pow] outputs: [/transformer/h.7/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Pow_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/ReduceMean_1 [ReduceMean] inputs: [/transformer/h.7/ln_1/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/ReduceMean_1 for ONNX node: /transformer/h.7/ln_1/ReduceMean_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/ReduceMean_1_output_0 for ONNX tensor: /transformer/h.7/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/ReduceMean_1 [ReduceMean] outputs: [/transformer/h.7/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Add [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/ReduceMean_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Add [Add] inputs: [/transformer/h.7/ln_1/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Add for ONNX node: /transformer/h.7/ln_1/Add +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Add_output_0 for ONNX tensor: /transformer/h.7/ln_1/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Add [Add] outputs: [/transformer/h.7/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Sqrt [Sqrt] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Add_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Sqrt [Sqrt] inputs: [/transformer/h.7/ln_1/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Sqrt for ONNX node: /transformer/h.7/ln_1/Sqrt +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Sqrt_output_0 for ONNX tensor: /transformer/h.7/ln_1/Sqrt_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Sqrt [Sqrt] outputs: [/transformer/h.7/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Div [Div] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Sub_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Sqrt_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Div [Div] inputs: [/transformer/h.7/ln_1/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.7/ln_1/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Div for ONNX node: /transformer/h.7/ln_1/Div +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Div_output_0 for ONNX tensor: /transformer/h.7/ln_1/Div_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Div [Div] outputs: [/transformer/h.7/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Mul [Mul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Div_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Mul [Mul] inputs: [/transformer/h.7/ln_1/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Mul for ONNX node: /transformer/h.7/ln_1/Mul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Mul_output_0 for ONNX tensor: /transformer/h.7/ln_1/Mul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Mul [Mul] outputs: [/transformer/h.7/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/ln_1/Add_1 [Add] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Mul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Add_1 [Add] inputs: [/transformer/h.7/ln_1/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/ln_1/Add_1 for ONNX node: /transformer/h.7/ln_1/Add_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/ln_1/Add_1_output_0 for ONNX tensor: /transformer/h.7/ln_1/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/ln_1/Add_1 [Add] outputs: [/transformer/h.7/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/q_proj/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::MatMul_3034 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/q_proj/MatMul [MatMul] inputs: [/transformer/h.7/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3034 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: onnx::MatMul_3034 for ONNX node: onnx::MatMul_3034 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/q_proj/MatMul for ONNX node: /transformer/h.7/attn/q_proj/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/q_proj/MatMul_output_0 for ONNX tensor: /transformer/h.7/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/q_proj/MatMul [MatMul] outputs: [/transformer/h.7/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/k_proj/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::MatMul_3035 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/k_proj/MatMul [MatMul] inputs: [/transformer/h.7/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3035 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: onnx::MatMul_3035 for ONNX node: onnx::MatMul_3035 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/k_proj/MatMul for ONNX node: /transformer/h.7/attn/k_proj/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/k_proj/MatMul_output_0 for ONNX tensor: /transformer/h.7/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/k_proj/MatMul [MatMul] outputs: [/transformer/h.7/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/v_proj/MatMul [MatMul] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Add_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::MatMul_3036 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/v_proj/MatMul [MatMul] inputs: [/transformer/h.7/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3036 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: onnx::MatMul_3036 for ONNX node: onnx::MatMul_3036 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/v_proj/MatMul for ONNX node: /transformer/h.7/attn/v_proj/MatMul +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/v_proj/MatMul_output_0 for ONNX tensor: /transformer/h.7/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/v_proj/MatMul [MatMul] outputs: [/transformer/h.7/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape [Shape] inputs: [/transformer/h.7/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape for ONNX node: /transformer/h.7/attn/Shape +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape [Shape] outputs: [/transformer/h.7/attn/Shape_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather [Gather] inputs: [/transformer/h.7/attn/Shape_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather for ONNX node: /transformer/h.7/attn/Gather +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather [Gather] outputs: [/transformer/h.7/attn/Gather_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_1 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_1 [Shape] inputs: [/transformer/h.7/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_1 for ONNX node: /transformer/h.7/attn/Shape_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_1_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_1 [Shape] outputs: [/transformer/h.7/attn/Shape_1_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_1 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_1 [Gather] inputs: [/transformer/h.7/attn/Shape_1_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_1 for ONNX node: /transformer/h.7/attn/Gather_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_1_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_1 [Gather] outputs: [/transformer/h.7/attn/Gather_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze for ONNX node: /transformer/h.7/attn/Unsqueeze +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_1 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_1 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_1 for ONNX node: /transformer/h.7/attn/Unsqueeze_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_1_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_1 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_1_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Concat [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_output_0 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_1_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat for ONNX node: /transformer/h.7/attn/Concat +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Concat [Concat] outputs: [/transformer/h.7/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape [Reshape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/q_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Reshape [Reshape] inputs: [/transformer/h.7/attn/q_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.7/attn/Concat_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape for ONNX node: /transformer/h.7/attn/Reshape +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Reshape [Reshape] outputs: [/transformer/h.7/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_2 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_2 [Shape] inputs: [/transformer/h.7/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_2 for ONNX node: /transformer/h.7/attn/Shape_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_2_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_2 [Shape] outputs: [/transformer/h.7/attn/Shape_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_2 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_2 [Gather] inputs: [/transformer/h.7/attn/Shape_2_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_2 for ONNX node: /transformer/h.7/attn/Gather_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_2_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_2 [Gather] outputs: [/transformer/h.7/attn/Gather_2_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_3 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_3 [Shape] inputs: [/transformer/h.7/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_3 for ONNX node: /transformer/h.7/attn/Shape_3 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_3_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_3 [Shape] outputs: [/transformer/h.7/attn/Shape_3_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_3 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_3 [Gather] inputs: [/transformer/h.7/attn/Shape_3_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_3 for ONNX node: /transformer/h.7/attn/Gather_3 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_3_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_3 [Gather] outputs: [/transformer/h.7/attn/Gather_3_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_2 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_2 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_2_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_2 for ONNX node: /transformer/h.7/attn/Unsqueeze_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_2_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_2 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_3 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_3 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_3_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_3 for ONNX node: /transformer/h.7/attn/Unsqueeze_3 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_3_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_3 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_1 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Concat_1 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_2_output_0 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_3_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_1 for ONNX node: /transformer/h.7/attn/Concat_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_1_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Concat_1 [Concat] outputs: [/transformer/h.7/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_1 [Reshape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/k_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Reshape_1 [Reshape] inputs: [/transformer/h.7/attn/k_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.7/attn/Concat_1_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_1 for ONNX node: /transformer/h.7/attn/Reshape_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_1_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Reshape_1 [Reshape] outputs: [/transformer/h.7/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_4 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_4 [Shape] inputs: [/transformer/h.7/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_4 for ONNX node: /transformer/h.7/attn/Shape_4 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_4_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_4 [Shape] outputs: [/transformer/h.7/attn/Shape_4_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_4 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_4 [Gather] inputs: [/transformer/h.7/attn/Shape_4_output_0 -> (3)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_4 for ONNX node: /transformer/h.7/attn/Gather_4 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_4_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_4 [Gather] outputs: [/transformer/h.7/attn/Gather_4_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_5 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_5 [Shape] inputs: [/transformer/h.7/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_5 for ONNX node: /transformer/h.7/attn/Shape_5 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_5_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_5 [Shape] outputs: [/transformer/h.7/attn/Shape_5_output_0 -> (3)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_5 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_5 [Gather] inputs: [/transformer/h.7/attn/Shape_5_output_0 -> (3)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_5 for ONNX node: /transformer/h.7/attn/Gather_5 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_5_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_5 [Gather] outputs: [/transformer/h.7/attn/Gather_5_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_4_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_4 for ONNX node: /transformer/h.7/attn/Unsqueeze_4 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_4_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_5_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_5 for ONNX node: /transformer/h.7/attn/Unsqueeze_5 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_5_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_2 [Concat] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_4_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_3_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Concat_2 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_2_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_3_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_2 for ONNX node: /transformer/h.7/attn/Concat_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_2_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Concat_2 [Concat] outputs: [/transformer/h.7/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_2 [Reshape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/v_proj/MatMul_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Reshape_2 [Reshape] inputs: [/transformer/h.7/attn/v_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.7/attn/Concat_2_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_2 for ONNX node: /transformer/h.7/attn/Reshape_2 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_2_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Reshape_2 [Reshape] outputs: [/transformer/h.7/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Transpose [Transpose] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_2_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Transpose [Transpose] inputs: [/transformer/h.7/attn/Reshape_2_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Transpose for ONNX node: /transformer/h.7/attn/Transpose +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: present.7.value_1203 for ONNX tensor: present.7.value +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Transpose [Transpose] outputs: [present.7.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_6 [Shape] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_6 [Shape] inputs: [/transformer/h.7/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_6 for ONNX node: /transformer/h.7/attn/Shape_6 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_6_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Shape_6 [Shape] outputs: [/transformer/h.7/attn/Shape_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_6 [Gather] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_6 [Gather] inputs: [/transformer/h.7/attn/Shape_6_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_6 for ONNX node: /transformer/h.7/attn/Gather_6 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_6_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_6_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Gather_6 [Gather] outputs: [/transformer/h.7/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Slice [Slice] inputs: [/transformer/h.7/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice for ONNX node: /transformer/h.7/attn/Slice +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Slice [Slice] outputs: [/transformer/h.7/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_1 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Slice_1 [Slice] inputs: [/transformer/h.7/attn/Reshape_1_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_1 for ONNX node: /transformer/h.7/attn/Slice_1 +[12/30/2022-19:54:23] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_1_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_1_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Slice_1 [Slice] outputs: [/transformer/h.7/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:23] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_2 [Slice] +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:23] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:23] [V] [TRT] /transformer/h.7/attn/Slice_2 [Slice] inputs: [/transformer/h.7/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:23] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_2 for ONNX node: /transformer/h.7/attn/Slice_2 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_2_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_2 [Slice] outputs: [/transformer/h.7/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_3 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_15_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_3 [Slice] inputs: [/transformer/h.7/attn/Reshape_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.0/attn/Constant_15_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_3 for ONNX node: /transformer/h.7/attn/Slice_3 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_3_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_3 [Slice] outputs: [/transformer/h.7/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_7 [Shape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_7 [Shape] inputs: [/transformer/h.7/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_7 for ONNX node: /transformer/h.7/attn/Shape_7 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_7_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_7 [Shape] outputs: [/transformer/h.7/attn/Shape_7_output_0 -> (4)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_7 [Gather] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_29_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_7 [Gather] inputs: [/transformer/h.7/attn/Shape_7_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_29_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_7 for ONNX node: /transformer/h.7/attn/Gather_7 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_7_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_7 [Gather] outputs: [/transformer/h.7/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Range [Range] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_31_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Range [Range] inputs: [/transformer/Constant_output_0 -> ()[INT32]], [/transformer/h.7/attn/Gather_7_output_0 -> ()[INT32]], [/transformer/h.0/attn/Constant_31_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Range for ONNX node: /transformer/h.7/attn/Range +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Range_output_0 for ONNX tensor: /transformer/h.7/attn/Range_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Range [Range] outputs: [/transformer/h.7/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Cast_1 [Cast] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Range_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cast_1 [Cast] inputs: [/transformer/h.7/attn/Range_output_0 -> (32)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Cast_1 for ONNX node: /transformer/h.7/attn/Cast_1 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Cast_1_output_0 for ONNX tensor: /transformer/h.7/attn/Cast_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cast_1 [Cast] outputs: [/transformer/h.7/attn/Cast_1_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Cast_2 [Cast] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cast_2 [Cast] inputs: [/transformer/h.7/attn/Gather_7_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Cast_2 for ONNX node: /transformer/h.7/attn/Cast_2 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Cast_2_output_0 for ONNX tensor: /transformer/h.7/attn/Cast_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cast_2 [Cast] outputs: [/transformer/h.7/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Div [Div] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Cast_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Cast_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Div [Div] inputs: [/transformer/h.7/attn/Cast_1_output_0 -> (32)[FLOAT]], [/transformer/h.7/attn/Cast_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Div for ONNX node: /transformer/h.7/attn/Div +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Div_output_0 for ONNX tensor: /transformer/h.7/attn/Div_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Div [Div] outputs: [/transformer/h.7/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Pow [Pow] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_32_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Div_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Pow [Pow] inputs: [/transformer/h.0/attn/Constant_32_output_0 -> ()[FLOAT]], [/transformer/h.7/attn/Div_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Pow for ONNX node: /transformer/h.7/attn/Pow +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Pow_output_0 for ONNX tensor: /transformer/h.7/attn/Pow_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Pow [Pow] outputs: [/transformer/h.7/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Reciprocal [Reciprocal] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Pow_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reciprocal [Reciprocal] inputs: [/transformer/h.7/attn/Pow_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Reciprocal for ONNX node: /transformer/h.7/attn/Reciprocal +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reciprocal_output_0 for ONNX tensor: /transformer/h.7/attn/Reciprocal_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reciprocal [Reciprocal] outputs: [/transformer/h.7/attn/Reciprocal_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Mul [Mul] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reciprocal_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul [Mul] inputs: [/transformer/h.7/attn/Reciprocal_output_0 -> (32)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Mul for ONNX node: /transformer/h.7/attn/Mul +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Mul_output_0 for ONNX tensor: /transformer/h.7/attn/Mul_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul [Mul] outputs: [/transformer/h.7/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Cast_3 [Cast] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cast_3 [Cast] inputs: [/transformer/h.7/attn/Gather_6_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Casting to type: float32 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Cast_3 for ONNX node: /transformer/h.7/attn/Cast_3 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Cast_3_output_0 for ONNX tensor: /transformer/h.7/attn/Cast_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cast_3 [Cast] outputs: [/transformer/h.7/attn/Cast_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Range_1 [Range] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Cast_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Range_1 [Range] inputs: [/transformer/h.0/attn/Constant_34_output_0 -> ()[FLOAT]], [/transformer/h.7/attn/Cast_3_output_0 -> ()[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Range_1 for ONNX node: /transformer/h.7/attn/Range_1 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Range_1_output_0 for ONNX tensor: /transformer/h.7/attn/Range_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Range_1 [Range] outputs: [/transformer/h.7/attn/Range_1_output_0 -> (-1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Einsum [Einsum] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Range_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Mul_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Einsum [Einsum] inputs: [/transformer/h.7/attn/Range_1_output_0 -> (-1)[FLOAT]], [/transformer/h.7/attn/Mul_output_0 -> (32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Einsum for ONNX node: /transformer/h.7/attn/Einsum +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Einsum_output_0 for ONNX tensor: /transformer/h.7/attn/Einsum_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Einsum [Einsum] outputs: [/transformer/h.7/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Sin [Sin] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Einsum_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Sin [Sin] inputs: [/transformer/h.7/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Sin for ONNX node: /transformer/h.7/attn/Sin +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Sin_output_0 for ONNX tensor: /transformer/h.7/attn/Sin_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Sin [Sin] outputs: [/transformer/h.7/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Cos [Cos] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Einsum_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cos [Cos] inputs: [/transformer/h.7/attn/Einsum_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Cos for ONNX node: /transformer/h.7/attn/Cos +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Cos_output_0 for ONNX tensor: /transformer/h.7/attn/Cos_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Cos [Cos] outputs: [/transformer/h.7/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_8 [Shape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Sin_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_8 [Shape] inputs: [/transformer/h.7/attn/Sin_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_8 for ONNX node: /transformer/h.7/attn/Shape_8 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_8_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_8 [Shape] outputs: [/transformer/h.7/attn/Shape_8_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_8 [Gather] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_8 [Gather] inputs: [/transformer/h.7/attn/Shape_8_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_8 for ONNX node: /transformer/h.7/attn/Gather_8 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_8_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_8 [Gather] outputs: [/transformer/h.7/attn/Gather_8_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_3 [Reshape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Sin_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_3 [Reshape] inputs: [/transformer/h.7/attn/Sin_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_3 for ONNX node: /transformer/h.7/attn/Reshape_3 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_3_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_3 [Reshape] outputs: [/transformer/h.7/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/ConstantOfShape [ConstantOfShape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/ConstantOfShape [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/ConstantOfShape for ONNX node: /transformer/h.7/attn/ConstantOfShape +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/ConstantOfShape_output_0 for ONNX tensor: /transformer/h.7/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/ConstantOfShape [ConstantOfShape] outputs: [/transformer/h.7/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Expand [Expand] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/ConstantOfShape_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Expand [Expand] inputs: [/transformer/h.7/attn/Reshape_3_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.7/attn/ConstantOfShape_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Expand for ONNX node: /transformer/h.7/attn/Expand +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Expand_output_0 for ONNX tensor: /transformer/h.7/attn/Expand_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Expand [Expand] outputs: [/transformer/h.7/attn/Expand_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Tile [Tile] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Expand_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Tile [Tile] inputs: [/transformer/h.7/attn/Expand_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Tile for ONNX node: /transformer/h.7/attn/Tile +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Tile_output_0 for ONNX tensor: /transformer/h.7/attn/Tile_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Tile [Tile] outputs: [/transformer/h.7/attn/Tile_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_8_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_6 for ONNX node: /transformer/h.7/attn/Unsqueeze_6 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_6_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_3 [Concat] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_3 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_6_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_3 for ONNX node: /transformer/h.7/attn/Concat_3 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_3_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_3 [Concat] outputs: [/transformer/h.7/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_4 [Reshape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Tile_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_4 [Reshape] inputs: [/transformer/h.7/attn/Tile_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.7/attn/Concat_3_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_4 for ONNX node: /transformer/h.7/attn/Reshape_4 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_4_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_4_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_4 [Reshape] outputs: [/transformer/h.7/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_9 [Shape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_9 [Shape] inputs: [/transformer/h.7/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_9 for ONNX node: /transformer/h.7/attn/Shape_9 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_9_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_9 [Shape] outputs: [/transformer/h.7/attn/Shape_9_output_0 -> (4)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_9 [Gather] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_9 [Gather] inputs: [/transformer/h.7/attn/Shape_9_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_9 for ONNX node: /transformer/h.7/attn/Gather_9 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_9_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_9 [Gather] outputs: [/transformer/h.7/attn/Gather_9_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_7 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_4_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_7 [Unsqueeze] inputs: [/transformer/h.7/attn/Reshape_4_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_7 for ONNX node: /transformer/h.7/attn/Unsqueeze_7 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_7_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_7 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_8 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_8 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_8 for ONNX node: /transformer/h.7/attn/Unsqueeze_8 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_8_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_8 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_8_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_4 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_4 [Slice] inputs: [/transformer/h.7/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_8_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_4 for ONNX node: /transformer/h.7/attn/Slice_4 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_4_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_4_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_4 [Slice] outputs: [/transformer/h.7/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_9 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_4_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_9 [Unsqueeze] inputs: [/transformer/h.7/attn/Slice_4_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_9 for ONNX node: /transformer/h.7/attn/Unsqueeze_9 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_9_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_9 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_10 [Shape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Cos_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_10 [Shape] inputs: [/transformer/h.7/attn/Cos_output_0 -> (-1, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_10 for ONNX node: /transformer/h.7/attn/Shape_10 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_10_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_10 [Shape] outputs: [/transformer/h.7/attn/Shape_10_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_10 [Gather] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_10 [Gather] inputs: [/transformer/h.7/attn/Shape_10_output_0 -> (2)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_10 for ONNX node: /transformer/h.7/attn/Gather_10 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_10_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_10 [Gather] outputs: [/transformer/h.7/attn/Gather_10_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_5 [Reshape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Cos_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_37_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_5 [Reshape] inputs: [/transformer/h.7/attn/Cos_output_0 -> (-1, 32)[FLOAT]], [/transformer/h.0/attn/Constant_37_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_5 for ONNX node: /transformer/h.7/attn/Reshape_5 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_5_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_5 [Reshape] outputs: [/transformer/h.7/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/ConstantOfShape_1 [ConstantOfShape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/ConstantOfShape_1 [ConstantOfShape] inputs: [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/ConstantOfShape_1 for ONNX node: /transformer/h.7/attn/ConstantOfShape_1 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/ConstantOfShape_1_output_0 for ONNX tensor: /transformer/h.7/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/ConstantOfShape_1 [ConstantOfShape] outputs: [/transformer/h.7/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Expand_1 [Expand] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/ConstantOfShape_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Expand_1 [Expand] inputs: [/transformer/h.7/attn/Reshape_5_output_0 -> (-1, 1)[FLOAT]], [/transformer/h.7/attn/ConstantOfShape_1_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Expand_1 for ONNX node: /transformer/h.7/attn/Expand_1 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Expand_1_output_0 for ONNX tensor: /transformer/h.7/attn/Expand_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Expand_1 [Expand] outputs: [/transformer/h.7/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Tile_1 [Tile] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Expand_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Tile_266 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Tile_1 [Tile] inputs: [/transformer/h.7/attn/Expand_1_output_0 -> (-1, 1)[FLOAT]], [onnx::Tile_266 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Tile_1 for ONNX node: /transformer/h.7/attn/Tile_1 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Tile_1_output_0 for ONNX tensor: /transformer/h.7/attn/Tile_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Tile_1 [Tile] outputs: [/transformer/h.7/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_10 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_10 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_10_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_10 for ONNX node: /transformer/h.7/attn/Unsqueeze_10 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_10_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_10 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_10_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_4 [Concat] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_4 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_10_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_4 for ONNX node: /transformer/h.7/attn/Concat_4 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_4_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_4_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_4 [Concat] outputs: [/transformer/h.7/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_6 [Reshape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Tile_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_4_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_6 [Reshape] inputs: [/transformer/h.7/attn/Tile_1_output_0 -> (-1, 2)[FLOAT]], [/transformer/h.7/attn/Concat_4_output_0 -> (2)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_6 for ONNX node: /transformer/h.7/attn/Reshape_6 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_6_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_6 [Reshape] outputs: [/transformer/h.7/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_11 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_11 [Unsqueeze] inputs: [/transformer/h.7/attn/Reshape_6_output_0 -> (-1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (_, _), unsqueezing to: (_, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_11 for ONNX node: /transformer/h.7/attn/Unsqueeze_11 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_11_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_11 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_12 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_12 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_9_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_12 for ONNX node: /transformer/h.7/attn/Unsqueeze_12 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_12_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_12 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_12_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_5 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_12_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_5 [Slice] inputs: [/transformer/h.7/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_12_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_5 for ONNX node: /transformer/h.7/attn/Slice_5 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_5_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_5 [Slice] outputs: [/transformer/h.7/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_13 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_13 [Unsqueeze] inputs: [/transformer/h.7/attn/Slice_5_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_13 for ONNX node: /transformer/h.7/attn/Unsqueeze_13 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_13_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_13 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Mul_1 [Mul] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul_1 [Mul] inputs: [/transformer/h.7/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Unsqueeze_13_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Mul_1 for ONNX node: /transformer/h.7/attn/Mul_1 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Mul_1_output_0 for ONNX tensor: /transformer/h.7/attn/Mul_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul_1 [Mul] outputs: [/transformer/h.7/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_6 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_6 [Slice] inputs: [/transformer/h.7/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_6 for ONNX node: /transformer/h.7/attn/Slice_6 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_6_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_6 [Slice] outputs: [/transformer/h.7/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_7 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_7 [Slice] inputs: [/transformer/h.7/attn/Slice_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_7 for ONNX node: /transformer/h.7/attn/Slice_7 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_7_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_7 [Slice] outputs: [/transformer/h.7/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Neg [Neg] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Neg [Neg] inputs: [/transformer/h.7/attn/Slice_7_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Neg for ONNX node: /transformer/h.7/attn/Neg +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Neg_output_0 for ONNX tensor: /transformer/h.7/attn/Neg_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Neg [Neg] outputs: [/transformer/h.7/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_14 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Neg_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_14 [Unsqueeze] inputs: [/transformer/h.7/attn/Neg_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_14 for ONNX node: /transformer/h.7/attn/Unsqueeze_14 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_14_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_14 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_15 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_15 [Unsqueeze] inputs: [/transformer/h.7/attn/Slice_6_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_15 for ONNX node: /transformer/h.7/attn/Unsqueeze_15 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_15_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_15 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_5 [Concat] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_14_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_15_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_5 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_14_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.7/attn/Unsqueeze_15_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_5 for ONNX node: /transformer/h.7/attn/Concat_5 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_5_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_5 [Concat] outputs: [/transformer/h.7/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_11 [Shape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_11 [Shape] inputs: [/transformer/h.7/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_11 for ONNX node: /transformer/h.7/attn/Shape_11 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_11_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_11 [Shape] outputs: [/transformer/h.7/attn/Shape_11_output_0 -> (5)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_8 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_8 [Slice] inputs: [/transformer/h.7/attn/Shape_11_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_8 for ONNX node: /transformer/h.7/attn/Slice_8 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_8_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_8 [Slice] outputs: [/transformer/h.7/attn/Slice_8_output_0 -> (3)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_6 [Concat] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_8_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_6 [Concat] inputs: [/transformer/h.7/attn/Slice_8_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_6 for ONNX node: /transformer/h.7/attn/Concat_6 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_6_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Concat_6 [Concat] outputs: [/transformer/h.7/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_7 [Reshape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_7 [Reshape] inputs: [/transformer/h.7/attn/Concat_5_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.7/attn/Concat_6_output_0 -> (4)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_7 for ONNX node: /transformer/h.7/attn/Reshape_7 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_7_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Reshape_7 [Reshape] outputs: [/transformer/h.7/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Mul_2 [Mul] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul_2 [Mul] inputs: [/transformer/h.7/attn/Reshape_7_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Unsqueeze_9_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Mul_2 for ONNX node: /transformer/h.7/attn/Mul_2 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Mul_2_output_0 for ONNX tensor: /transformer/h.7/attn/Mul_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul_2 [Mul] outputs: [/transformer/h.7/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Add [Add] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Mul_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Mul_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Add [Add] inputs: [/transformer/h.7/attn/Mul_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Mul_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Add for ONNX node: /transformer/h.7/attn/Add +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Add_output_0 for ONNX tensor: /transformer/h.7/attn/Add_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Add [Add] outputs: [/transformer/h.7/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_12 [Shape] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_12 [Shape] inputs: [/transformer/h.7/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_12 for ONNX node: /transformer/h.7/attn/Shape_12 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_12_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_12_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Shape_12 [Shape] outputs: [/transformer/h.7/attn/Shape_12_output_0 -> (4)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_11 [Gather] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_12_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_11 [Gather] inputs: [/transformer/h.7/attn/Shape_12_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_11 for ONNX node: /transformer/h.7/attn/Gather_11 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_11_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Gather_11 [Gather] outputs: [/transformer/h.7/attn/Gather_11_output_0 -> ()[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_16 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_16 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_16 for ONNX node: /transformer/h.7/attn/Unsqueeze_16 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_16_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_16 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_16_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_9 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_7_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_16_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_9 [Slice] inputs: [/transformer/h.7/attn/Unsqueeze_7_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_16_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_9 for ONNX node: /transformer/h.7/attn/Slice_9 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_9_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_9 [Slice] outputs: [/transformer/h.7/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_17 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_9_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_17 [Unsqueeze] inputs: [/transformer/h.7/attn/Slice_9_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_17 for ONNX node: /transformer/h.7/attn/Unsqueeze_17 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_17_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_17 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_18 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_18 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_11_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_18 for ONNX node: /transformer/h.7/attn/Unsqueeze_18 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_18_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_18 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_18_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_10 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_18_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_10 [Slice] inputs: [/transformer/h.7/attn/Unsqueeze_11_output_0 -> (1, -1, -1)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_18_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_10 for ONNX node: /transformer/h.7/attn/Slice_10 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_10_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_10 [Slice] outputs: [/transformer/h.7/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_19 [Unsqueeze] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_10_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_19 [Unsqueeze] inputs: [/transformer/h.7/attn/Slice_10_output_0 -> (1, -1, -1)[FLOAT]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Original shape: (1, _, _), unsqueezing to: (_, _, _, _) +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_19 for ONNX node: /transformer/h.7/attn/Unsqueeze_19 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_19_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Unsqueeze_19 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Mul_3 [Mul] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_19_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul_3 [Mul] inputs: [/transformer/h.7/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Unsqueeze_19_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Mul_3 for ONNX node: /transformer/h.7/attn/Mul_3 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Mul_3_output_0 for ONNX tensor: /transformer/h.7/attn/Mul_3_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Mul_3 [Mul] outputs: [/transformer/h.7/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_11 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_11 [Slice] inputs: [/transformer/h.7/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:24] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_11 for ONNX node: /transformer/h.7/attn/Slice_11 +[12/30/2022-19:54:24] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_11_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_11 [Slice] outputs: [/transformer/h.7/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:24] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_12 [Slice] +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_2_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_11_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:24] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:24] [V] [TRT] /transformer/h.7/attn/Slice_12 [Slice] inputs: [/transformer/h.7/attn/Slice_2_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/Constant_5_output_0 -> (1)[INT32]], [/transformer/Constant_11_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_12 for ONNX node: /transformer/h.7/attn/Slice_12 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_12_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_12_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_12 [Slice] outputs: [/transformer/h.7/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Neg_1 [Neg] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_12_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Neg_1 [Neg] inputs: [/transformer/h.7/attn/Slice_12_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Neg_1 for ONNX node: /transformer/h.7/attn/Neg_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Neg_1_output_0 for ONNX tensor: /transformer/h.7/attn/Neg_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Neg_1 [Neg] outputs: [/transformer/h.7/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_20 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Neg_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_20 [Unsqueeze] inputs: [/transformer/h.7/attn/Neg_1_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_20 for ONNX node: /transformer/h.7/attn/Unsqueeze_20 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_20_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_20 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_21 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_11_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_21 [Unsqueeze] inputs: [/transformer/h.7/attn/Slice_11_output_0 -> (-1, -1, 4, 32)[FLOAT]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (_, _, 4, 32), unsqueezing to: (_, _, _, _, _) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_21 for ONNX node: /transformer/h.7/attn/Unsqueeze_21 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_21_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_21 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_7 [Concat] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_20_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_21_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_7 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_20_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], [/transformer/h.7/attn/Unsqueeze_21_output_0 -> (-1, -1, 4, 32, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_7 for ONNX node: /transformer/h.7/attn/Concat_7 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_7_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_7_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_7 [Concat] outputs: [/transformer/h.7/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_13 [Shape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_7_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_13 [Shape] inputs: [/transformer/h.7/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_13 for ONNX node: /transformer/h.7/attn/Shape_13 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_13_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_13 [Shape] outputs: [/transformer/h.7/attn/Shape_13_output_0 -> (5)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_13 [Slice] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_13 [Slice] inputs: [/transformer/h.7/attn/Shape_13_output_0 -> (5)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_13 for ONNX node: /transformer/h.7/attn/Slice_13 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_13_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_13 [Slice] outputs: [/transformer/h.7/attn/Slice_13_output_0 -> (3)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_8 [Concat] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_8 [Concat] inputs: [/transformer/h.7/attn/Slice_13_output_0 -> (3)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_8 for ONNX node: /transformer/h.7/attn/Concat_8 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_8_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_8_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_8 [Concat] outputs: [/transformer/h.7/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_8 [Reshape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_7_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_8_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Reshape_8 [Reshape] inputs: [/transformer/h.7/attn/Concat_7_output_0 -> (-1, -1, 4, 32, 2)[FLOAT]], [/transformer/h.7/attn/Concat_8_output_0 -> (4)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_8 for ONNX node: /transformer/h.7/attn/Reshape_8 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_8_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_8_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Reshape_8 [Reshape] outputs: [/transformer/h.7/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Mul_4 [Mul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_8_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_17_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Mul_4 [Mul] inputs: [/transformer/h.7/attn/Reshape_8_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Unsqueeze_17_output_0 -> (1, -1, 1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Mul_4 for ONNX node: /transformer/h.7/attn/Mul_4 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Mul_4_output_0 for ONNX tensor: /transformer/h.7/attn/Mul_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Mul_4 [Mul] outputs: [/transformer/h.7/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Add_1 [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Mul_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Mul_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Add_1 [Add] inputs: [/transformer/h.7/attn/Mul_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Mul_4_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Add_1 for ONNX node: /transformer/h.7/attn/Add_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Add_1_output_0 for ONNX tensor: /transformer/h.7/attn/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Add_1 [Add] outputs: [/transformer/h.7/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_9 [Concat] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_9 [Concat] inputs: [/transformer/h.7/attn/Add_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Slice_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_9 for ONNX node: /transformer/h.7/attn/Concat_9 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_9_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_9_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_9 [Concat] outputs: [/transformer/h.7/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_10 [Concat] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_10 [Concat] inputs: [/transformer/h.7/attn/Add_1_output_0 -> (-1, -1, 4, 64)[FLOAT]], [/transformer/h.7/attn/Slice_3_output_0 -> (-1, -1, 4, 64)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_10 for ONNX node: /transformer/h.7/attn/Concat_10 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_10_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_10_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_10 [Concat] outputs: [/transformer/h.7/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Transpose_1 [Transpose] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_9_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_1 [Transpose] inputs: [/transformer/h.7/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Transpose_1 for ONNX node: /transformer/h.7/attn/Transpose_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: present.7.key_1352 for ONNX tensor: present.7.key +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_1 [Transpose] outputs: [present.7.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Transpose_2 [Transpose] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_10_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_2 [Transpose] inputs: [/transformer/h.7/attn/Concat_10_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Transpose_2 for ONNX node: /transformer/h.7/attn/Transpose_2 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Transpose_2_output_0 for ONNX tensor: /transformer/h.7/attn/Transpose_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_2 [Transpose] outputs: [/transformer/h.7/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_14 [Shape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Transpose_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_14 [Shape] inputs: [/transformer/h.7/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_14 for ONNX node: /transformer/h.7/attn/Shape_14 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_14_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_14_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_14 [Shape] outputs: [/transformer/h.7/attn/Shape_14_output_0 -> (4)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_14 [Slice] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_14_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_14 [Slice] inputs: [/transformer/h.7/attn/Shape_14_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_14 for ONNX node: /transformer/h.7/attn/Slice_14 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_14_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_14_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_14 [Slice] outputs: [/transformer/h.7/attn/Slice_14_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Squeeze [Squeeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_14_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Squeeze [Squeeze] inputs: [/transformer/h.7/attn/Slice_14_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Squeeze for ONNX node: /transformer/h.7/attn/Squeeze +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Squeeze_output_0 for ONNX tensor: /transformer/h.7/attn/Squeeze_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Squeeze [Squeeze] outputs: [/transformer/h.7/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_15 [Shape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: present.7.key +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_15 [Shape] inputs: [present.7.key -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_15 for ONNX node: /transformer/h.7/attn/Shape_15 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_15_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_15_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_15 [Shape] outputs: [/transformer/h.7/attn/Shape_15_output_0 -> (4)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_15 [Slice] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_15_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_69_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_15 [Slice] inputs: [/transformer/h.7/attn/Shape_15_output_0 -> (4)[INT32]], [/transformer/h.0/attn/Constant_69_output_0 -> (1)[INT32]], [/transformer/Constant_2_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_15 for ONNX node: /transformer/h.7/attn/Slice_15 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_15_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_15_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_15 [Slice] outputs: [/transformer/h.7/attn/Slice_15_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Squeeze_1 [Squeeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_15_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Squeeze_1 [Squeeze] inputs: [/transformer/h.7/attn/Slice_15_output_0 -> (1)[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (1,), squeezing to: () +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Squeeze_1 for ONNX node: /transformer/h.7/attn/Squeeze_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Squeeze_1_output_0 for ONNX tensor: /transformer/h.7/attn/Squeeze_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Squeeze_1 [Squeeze] outputs: [/transformer/h.7/attn/Squeeze_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Sub [Sub] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Squeeze_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Squeeze_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Sub [Sub] inputs: [/transformer/h.7/attn/Squeeze_1_output_0 -> ()[INT32]], [/transformer/h.7/attn/Squeeze_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Sub for ONNX node: /transformer/h.7/attn/Sub +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Sub_output_0 for ONNX tensor: /transformer/h.7/attn/Sub_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Sub [Sub] outputs: [/transformer/h.7/attn/Sub_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_22 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Sub_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_22 [Unsqueeze] inputs: [/transformer/h.7/attn/Sub_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_22 for ONNX node: /transformer/h.7/attn/Unsqueeze_22 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_22_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_22 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_22_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_23 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Squeeze_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_23 [Unsqueeze] inputs: [/transformer/h.7/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_23 for ONNX node: /transformer/h.7/attn/Unsqueeze_23 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_23_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_23 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_23_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_16 [Slice] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: transformer.h.0.attn.bias +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_22_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_23_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_6_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_16 [Slice] inputs: [transformer.h.0.attn.bias -> (1, 1, 2048, 2048)[BOOL]], [/transformer/h.7/attn/Unsqueeze_22_output_0 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_23_output_0 -> (1)[INT32]], [/transformer/Constant_6_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_16 for ONNX node: /transformer/h.7/attn/Slice_16 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_16_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_16_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_16 [Slice] outputs: [/transformer/h.7/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_24 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Squeeze_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_24 [Unsqueeze] inputs: [/transformer/h.7/attn/Squeeze_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_24 for ONNX node: /transformer/h.7/attn/Unsqueeze_24 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_24_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_24 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_24_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Slice_17 [Slice] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_16_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_24_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_5_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_17 [Slice] inputs: [/transformer/h.7/attn/Slice_16_output_0 -> (1, 1, -1, 2048)[BOOL]], [onnx::Unsqueeze_114 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_24_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_13_output_0 -> (1)[INT32]], [/transformer/Constant_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Slice_17 for ONNX node: /transformer/h.7/attn/Slice_17 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Slice_17_output_0 for ONNX tensor: /transformer/h.7/attn/Slice_17_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Slice_17 [Slice] outputs: [/transformer/h.7/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Transpose_3 [Transpose] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_9_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_3 [Transpose] inputs: [/transformer/h.7/attn/Concat_9_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Transpose_3 for ONNX node: /transformer/h.7/attn/Transpose_3 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Transpose_3_output_0 for ONNX tensor: /transformer/h.7/attn/Transpose_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_3 [Transpose] outputs: [/transformer/h.7/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/MatMul [MatMul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Transpose_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Transpose_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/MatMul [MatMul] inputs: [/transformer/h.7/attn/Transpose_2_output_0 -> (-1, 4, -1, 128)[FLOAT]], [/transformer/h.7/attn/Transpose_3_output_0 -> (-1, 4, 128, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/MatMul for ONNX node: /transformer/h.7/attn/MatMul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/MatMul_output_0 for ONNX tensor: /transformer/h.7/attn/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/MatMul [MatMul] outputs: [/transformer/h.7/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Where [Where] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Slice_17_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_8_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Where [Where] inputs: [/transformer/h.7/attn/Slice_17_output_0 -> (1, 1, -1, -1)[BOOL]], [/transformer/h.7/attn/MatMul_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Constant_8_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Where for ONNX node: /transformer/h.7/attn/Where +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Where_output_0 for ONNX tensor: /transformer/h.7/attn/Where_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Where [Where] outputs: [/transformer/h.7/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Div_1 [Div] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Where_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_112_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Div_1 [Div] inputs: [/transformer/h.7/attn/Where_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/h.0/attn/Constant_112_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Div_1 for ONNX node: /transformer/h.7/attn/Div_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Div_1_output_0 for ONNX tensor: /transformer/h.7/attn/Div_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Div_1 [Div] outputs: [/transformer/h.7/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Add_2 [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Div_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Mul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Add_2 [Add] inputs: [/transformer/h.7/attn/Div_1_output_0 -> (-1, 4, -1, -1)[FLOAT]], [/transformer/Mul_output_0 -> (-1, 1, 1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Add_2 for ONNX node: /transformer/h.7/attn/Add_2 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Add_2_output_0 for ONNX tensor: /transformer/h.7/attn/Add_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Add_2 [Add] outputs: [/transformer/h.7/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Softmax [Softmax] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Add_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Softmax [Softmax] inputs: [/transformer/h.7/attn/Add_2_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Softmax for ONNX node: /transformer/h.7/attn/Softmax +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Softmax_output_0 for ONNX tensor: /transformer/h.7/attn/Softmax_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Softmax [Softmax] outputs: [/transformer/h.7/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/MatMul_1 [MatMul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Softmax_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: present.7.value +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/MatMul_1 [MatMul] inputs: [/transformer/h.7/attn/Softmax_output_0 -> (-1, 4, -1, -1)[FLOAT]], [present.7.value -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/MatMul_1 for ONNX node: /transformer/h.7/attn/MatMul_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/MatMul_1_output_0 for ONNX tensor: /transformer/h.7/attn/MatMul_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/MatMul_1 [MatMul] outputs: [/transformer/h.7/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Transpose_4 [Transpose] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/MatMul_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_4 [Transpose] inputs: [/transformer/h.7/attn/MatMul_1_output_0 -> (-1, 4, -1, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Transpose_4 for ONNX node: /transformer/h.7/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Transpose_4_output_0 for ONNX tensor: /transformer/h.7/attn/Transpose_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Transpose_4 [Transpose] outputs: [/transformer/h.7/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_16 [Shape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Transpose_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_16 [Shape] inputs: [/transformer/h.7/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_16 for ONNX node: /transformer/h.7/attn/Shape_16 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_16_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_16_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_16 [Shape] outputs: [/transformer/h.7/attn/Shape_16_output_0 -> (4)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_12 [Gather] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_16_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Gather_12 [Gather] inputs: [/transformer/h.7/attn/Shape_16_output_0 -> (4)[INT32]], [/transformer/Constant_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_12 for ONNX node: /transformer/h.7/attn/Gather_12 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_12_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_12_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Gather_12 [Gather] outputs: [/transformer/h.7/attn/Gather_12_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Shape_17 [Shape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Transpose_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_17 [Shape] inputs: [/transformer/h.7/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Shape_17 for ONNX node: /transformer/h.7/attn/Shape_17 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Shape_17_output_0 for ONNX tensor: /transformer/h.7/attn/Shape_17_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Shape_17 [Shape] outputs: [/transformer/h.7/attn/Shape_17_output_0 -> (4)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Gather_13 [Gather] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Shape_17_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Gather_13 [Gather] inputs: [/transformer/h.7/attn/Shape_17_output_0 -> (4)[INT32]], [/transformer/Constant_1_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Using Gather axis: 0 +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Gather_13 for ONNX node: /transformer/h.7/attn/Gather_13 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Gather_13_output_0 for ONNX tensor: /transformer/h.7/attn/Gather_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Gather_13 [Gather] outputs: [/transformer/h.7/attn/Gather_13_output_0 -> ()[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_25 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_12_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_25 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_12_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_25 for ONNX node: /transformer/h.7/attn/Unsqueeze_25 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_25_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_25 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_25_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Unsqueeze_26 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Gather_13_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_26 [Unsqueeze] inputs: [/transformer/h.7/attn/Gather_13_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Unsqueeze_26 for ONNX node: /transformer/h.7/attn/Unsqueeze_26 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Unsqueeze_26_output_0 for ONNX tensor: /transformer/h.7/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Unsqueeze_26 [Unsqueeze] outputs: [/transformer/h.7/attn/Unsqueeze_26_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Concat_11 [Concat] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_25_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Unsqueeze_26_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/attn/Constant_115_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_11 [Concat] inputs: [/transformer/h.7/attn/Unsqueeze_25_output_0 -> (1)[INT32]], [/transformer/h.7/attn/Unsqueeze_26_output_0 -> (1)[INT32]], [/transformer/h.0/attn/Constant_115_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Concat_11 for ONNX node: /transformer/h.7/attn/Concat_11 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Concat_11_output_0 for ONNX tensor: /transformer/h.7/attn/Concat_11_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Concat_11 [Concat] outputs: [/transformer/h.7/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/Reshape_9 [Reshape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Transpose_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Concat_11_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Reshape_9 [Reshape] inputs: [/transformer/h.7/attn/Transpose_4_output_0 -> (-1, -1, 4, 128)[FLOAT]], [/transformer/h.7/attn/Concat_11_output_0 -> (3)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/Reshape_9 for ONNX node: /transformer/h.7/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/Reshape_9_output_0 for ONNX tensor: /transformer/h.7/attn/Reshape_9_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/Reshape_9 [Reshape] outputs: [/transformer/h.7/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/attn/out_proj/MatMul [MatMul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/Reshape_9_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::MatMul_3061 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/out_proj/MatMul [MatMul] inputs: [/transformer/h.7/attn/Reshape_9_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3061 -> (512, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: onnx::MatMul_3061 for ONNX node: onnx::MatMul_3061 +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/attn/out_proj/MatMul for ONNX node: /transformer/h.7/attn/out_proj/MatMul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/attn/out_proj/MatMul_output_0 for ONNX tensor: /transformer/h.7/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/attn/out_proj/MatMul [MatMul] outputs: [/transformer/h.7/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/fc_in/MatMul [MatMul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/ln_1/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::MatMul_3062 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_in/MatMul [MatMul] inputs: [/transformer/h.7/ln_1/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3062 -> (512, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: onnx::MatMul_3062 for ONNX node: onnx::MatMul_3062 +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/fc_in/MatMul for ONNX node: /transformer/h.7/mlp/fc_in/MatMul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/fc_in/MatMul_output_0 for ONNX tensor: /transformer/h.7/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_in/MatMul [MatMul] outputs: [/transformer/h.7/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/fc_in/Add [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: transformer.h.0.mlp.fc_in.bias +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/fc_in/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_in/Add [Add] inputs: [transformer.h.0.mlp.fc_in.bias -> (2048)[FLOAT]], [/transformer/h.7/mlp/fc_in/MatMul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/fc_in/Add for ONNX node: /transformer/h.7/mlp/fc_in/Add +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/fc_in/Add_output_0 for ONNX tensor: /transformer/h.7/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_in/Add [Add] outputs: [/transformer/h.7/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Mul [Mul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul [Mul] inputs: [/transformer/h.7/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Mul for ONNX node: /transformer/h.7/mlp/act/Mul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Mul_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Mul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul [Mul] outputs: [/transformer/h.7/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Pow [Pow] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Pow [Pow] inputs: [/transformer/h.7/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Pow for ONNX node: /transformer/h.7/mlp/act/Pow +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Pow_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Pow_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Pow [Pow] outputs: [/transformer/h.7/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Mul_1 [Mul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Pow_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul_1 [Mul] inputs: [/transformer/h.7/mlp/act/Pow_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_2_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Mul_1 for ONNX node: /transformer/h.7/mlp/act/Mul_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Mul_1_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul_1 [Mul] outputs: [/transformer/h.7/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Add [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/fc_in/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Mul_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Add [Add] inputs: [/transformer/h.7/mlp/fc_in/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.7/mlp/act/Mul_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Add for ONNX node: /transformer/h.7/mlp/act/Add +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Add_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Add [Add] outputs: [/transformer/h.7/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Mul_2 [Mul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/mlp/act/Constant_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul_2 [Mul] inputs: [/transformer/h.7/mlp/act/Add_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.0/mlp/act/Constant_3_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Mul_2 for ONNX node: /transformer/h.7/mlp/act/Mul_2 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Mul_2_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul_2 [Mul] outputs: [/transformer/h.7/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Tanh [Tanh] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Mul_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Tanh [Tanh] inputs: [/transformer/h.7/mlp/act/Mul_2_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Tanh for ONNX node: /transformer/h.7/mlp/act/Tanh +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Tanh_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Tanh_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Tanh [Tanh] outputs: [/transformer/h.7/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Add_1 [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Tanh_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Constant_7_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Add_1 [Add] inputs: [/transformer/h.7/mlp/act/Tanh_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/Constant_7_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Add_1 for ONNX node: /transformer/h.7/mlp/act/Add_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Add_1_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Add_1 [Add] outputs: [/transformer/h.7/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/act/Mul_3 [Mul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Mul_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul_3 [Mul] inputs: [/transformer/h.7/mlp/act/Mul_output_0 -> (-1, -1, 2048)[FLOAT]], [/transformer/h.7/mlp/act/Add_1_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/act/Mul_3 for ONNX node: /transformer/h.7/mlp/act/Mul_3 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/act/Mul_3_output_0 for ONNX tensor: /transformer/h.7/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/act/Mul_3 [Mul] outputs: [/transformer/h.7/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/fc_out/MatMul [MatMul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/act/Mul_3_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::MatMul_3063 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_out/MatMul [MatMul] inputs: [/transformer/h.7/mlp/act/Mul_3_output_0 -> (-1, -1, 2048)[FLOAT]], [onnx::MatMul_3063 -> (2048, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: onnx::MatMul_3063 for ONNX node: onnx::MatMul_3063 +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/fc_out/MatMul for ONNX node: /transformer/h.7/mlp/fc_out/MatMul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/fc_out/MatMul_output_0 for ONNX tensor: /transformer/h.7/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_out/MatMul [MatMul] outputs: [/transformer/h.7/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/mlp/fc_out/Add [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/fc_out/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_out/Add [Add] inputs: [transformer.h.0.ln_1.bias -> (512)[FLOAT]], [/transformer/h.7/mlp/fc_out/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/mlp/fc_out/Add for ONNX node: /transformer/h.7/mlp/fc_out/Add +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/mlp/fc_out/Add_output_0 for ONNX tensor: /transformer/h.7/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/mlp/fc_out/Add [Add] outputs: [/transformer/h.7/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/Add [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/attn/out_proj/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/mlp/fc_out/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/Add [Add] inputs: [/transformer/h.7/attn/out_proj/MatMul_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.7/mlp/fc_out/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/Add for ONNX node: /transformer/h.7/Add +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/Add_output_0 for ONNX tensor: /transformer/h.7/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/Add [Add] outputs: [/transformer/h.7/Add_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/h.7/Add_1 [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.6/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/Add_1 [Add] inputs: [/transformer/h.7/Add_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.6/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/h.7/Add_1 for ONNX node: /transformer/h.7/Add_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/h.7/Add_1_output_0 for ONNX tensor: /transformer/h.7/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/h.7/Add_1 [Add] outputs: [/transformer/h.7/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/ReduceMean [ReduceMean] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/ReduceMean [ReduceMean] inputs: [/transformer/h.7/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/ReduceMean for ONNX node: /transformer/ln_f/ReduceMean +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/ReduceMean_output_0 for ONNX tensor: /transformer/ln_f/ReduceMean_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/ReduceMean [ReduceMean] outputs: [/transformer/ln_f/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Sub [Sub] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.7/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/ReduceMean_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Sub [Sub] inputs: [/transformer/h.7/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/ln_f/ReduceMean_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Sub for ONNX node: /transformer/ln_f/Sub +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Sub_output_0 for ONNX tensor: /transformer/ln_f/Sub_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Sub [Sub] outputs: [/transformer/ln_f/Sub_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Pow [Pow] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Sub_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Pow [Pow] inputs: [/transformer/ln_f/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/h.0/ln_1/Constant_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Pow for ONNX node: /transformer/ln_f/Pow +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Pow_output_0 for ONNX tensor: /transformer/ln_f/Pow_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Pow [Pow] outputs: [/transformer/ln_f/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/ReduceMean_1 [ReduceMean] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Pow_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/ReduceMean_1 [ReduceMean] inputs: [/transformer/ln_f/Pow_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/ReduceMean_1 for ONNX node: /transformer/ln_f/ReduceMean_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/ReduceMean_1_output_0 for ONNX tensor: /transformer/ln_f/ReduceMean_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/ReduceMean_1 [ReduceMean] outputs: [/transformer/ln_f/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Add [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/ReduceMean_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/h.0/ln_1/Constant_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Add [Add] inputs: [/transformer/ln_f/ReduceMean_1_output_0 -> (-1, -1, 1)[FLOAT]], [/transformer/h.0/ln_1/Constant_1_output_0 -> ()[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Add for ONNX node: /transformer/ln_f/Add +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Add_output_0 for ONNX tensor: /transformer/ln_f/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Add [Add] outputs: [/transformer/ln_f/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Sqrt [Sqrt] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Add_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Sqrt [Sqrt] inputs: [/transformer/ln_f/Add_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Sqrt for ONNX node: /transformer/ln_f/Sqrt +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Sqrt_output_0 for ONNX tensor: /transformer/ln_f/Sqrt_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Sqrt [Sqrt] outputs: [/transformer/ln_f/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Div [Div] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Sub_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Sqrt_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Div [Div] inputs: [/transformer/ln_f/Sub_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/ln_f/Sqrt_output_0 -> (-1, -1, 1)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Div for ONNX node: /transformer/ln_f/Div +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Div_output_0 for ONNX tensor: /transformer/ln_f/Div_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Div [Div] outputs: [/transformer/ln_f/Div_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Mul [Mul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Div_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: transformer.h.0.ln_1.weight +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Mul [Mul] inputs: [/transformer/ln_f/Div_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.weight -> (512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Mul for ONNX node: /transformer/ln_f/Mul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Mul_output_0 for ONNX tensor: /transformer/ln_f/Mul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Mul [Mul] outputs: [/transformer/ln_f/Mul_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/ln_f/Add_1 [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Mul_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: transformer.h.0.ln_1.bias +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Add_1 [Add] inputs: [/transformer/ln_f/Mul_output_0 -> (-1, -1, 512)[FLOAT]], [transformer.h.0.ln_1.bias -> (512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/ln_f/Add_1 for ONNX node: /transformer/ln_f/Add_1 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/ln_f/Add_1_output_0 for ONNX tensor: /transformer/ln_f/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/ln_f/Add_1 [Add] outputs: [/transformer/ln_f/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/Unsqueeze_4 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Gather_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Unsqueeze_4 [Unsqueeze] inputs: [/transformer/Gather_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/Unsqueeze_4 for ONNX node: /transformer/Unsqueeze_4 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/Unsqueeze_4_output_0 for ONNX tensor: /transformer/Unsqueeze_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Unsqueeze_4 [Unsqueeze] outputs: [/transformer/Unsqueeze_4_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/Unsqueeze_5 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Gather_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Unsqueeze_5 [Unsqueeze] inputs: [/transformer/Gather_1_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/Unsqueeze_5 for ONNX node: /transformer/Unsqueeze_5 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/Unsqueeze_5_output_0 for ONNX tensor: /transformer/Unsqueeze_5_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Unsqueeze_5 [Unsqueeze] outputs: [/transformer/Unsqueeze_5_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/Unsqueeze_6 [Unsqueeze] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Squeeze_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::Unsqueeze_114 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Unsqueeze_6 [Unsqueeze] inputs: [/transformer/Squeeze_output_0 -> ()[INT32]], [onnx::Unsqueeze_114 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Original shape: (), unsqueezing to: (1,) +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/Unsqueeze_6 for ONNX node: /transformer/Unsqueeze_6 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/Unsqueeze_6_output_0 for ONNX tensor: /transformer/Unsqueeze_6_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Unsqueeze_6 [Unsqueeze] outputs: [/transformer/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/Concat_2 [Concat] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Unsqueeze_4_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Unsqueeze_5_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Unsqueeze_6_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Concat_2 [Concat] inputs: [/transformer/Unsqueeze_4_output_0 -> (1)[INT32]], [/transformer/Unsqueeze_5_output_0 -> (1)[INT32]], [/transformer/Unsqueeze_6_output_0 -> (1)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/Concat_2 for ONNX node: /transformer/Concat_2 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/Concat_2_output_0 for ONNX tensor: /transformer/Concat_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Concat_2 [Concat] outputs: [/transformer/Concat_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /transformer/Reshape_2 [Reshape] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/ln_f/Add_1_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Concat_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Reshape_2 [Reshape] inputs: [/transformer/ln_f/Add_1_output_0 -> (-1, -1, 512)[FLOAT]], [/transformer/Concat_2_output_0 -> (3)[INT32]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /transformer/Reshape_2 for ONNX node: /transformer/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /transformer/Reshape_2_output_0 for ONNX tensor: /transformer/Reshape_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] /transformer/Reshape_2 [Reshape] outputs: [/transformer/Reshape_2_output_0 -> (-1, -1, 512)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /lm_head/MatMul [MatMul] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /transformer/Reshape_2_output_0 +[12/30/2022-19:54:25] [V] [TRT] Searching for input: onnx::MatMul_3064 +[12/30/2022-19:54:25] [V] [TRT] /lm_head/MatMul [MatMul] inputs: [/transformer/Reshape_2_output_0 -> (-1, -1, 512)[FLOAT]], [onnx::MatMul_3064 -> (512, 50400)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: onnx::MatMul_3064 for ONNX node: onnx::MatMul_3064 +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /lm_head/MatMul for ONNX node: /lm_head/MatMul +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: /lm_head/MatMul_output_0 for ONNX tensor: /lm_head/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /lm_head/MatMul [MatMul] outputs: [/lm_head/MatMul_output_0 -> (-1, -1, 50400)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Parsing node: /lm_head/Add [Add] +[12/30/2022-19:54:25] [V] [TRT] Searching for input: lm_head.bias +[12/30/2022-19:54:25] [V] [TRT] Searching for input: /lm_head/MatMul_output_0 +[12/30/2022-19:54:25] [V] [TRT] /lm_head/Add [Add] inputs: [lm_head.bias -> (50400)[FLOAT]], [/lm_head/MatMul_output_0 -> (-1, -1, 50400)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Registering layer: lm_head.bias for ONNX node: lm_head.bias +[12/30/2022-19:54:25] [V] [TRT] Registering layer: /lm_head/Add for ONNX node: /lm_head/Add +[12/30/2022-19:54:25] [V] [TRT] Registering tensor: logits_1372 for ONNX tensor: logits +[12/30/2022-19:54:25] [V] [TRT] /lm_head/Add [Add] outputs: [logits -> (-1, -1, 50400)[FLOAT]], +[12/30/2022-19:54:25] [V] [TRT] Marking logits_1372 as output: logits +[12/30/2022-19:54:25] [V] [TRT] Marking present.0.key_169 as output: present.0.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.0.value_20 as output: present.0.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.1.key_338 as output: present.1.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.1.value_189 as output: present.1.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.2.key_507 as output: present.2.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.2.value_358 as output: present.2.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.3.key_676 as output: present.3.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.3.value_527 as output: present.3.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.4.key_845 as output: present.4.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.4.value_696 as output: present.4.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.5.key_1014 as output: present.5.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.5.value_865 as output: present.5.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.6.key_1183 as output: present.6.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.6.value_1034 as output: present.6.value +[12/30/2022-19:54:25] [V] [TRT] Marking present.7.key_1352 as output: present.7.key +[12/30/2022-19:54:25] [V] [TRT] Marking present.7.value_1203 as output: present.7.value +[12/30/2022-19:54:25] [I] Finish parsing network model +[12/30/2022-19:54:25] [V] [TRT] Applying generic optimizations to the graph for inference. +[12/30/2022-19:54:25] [V] [TRT] Original: 1094 layers +[12/30/2022-19:54:25] [V] [TRT] After dead-layer removal: 1094 layers +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2824 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2824 with (Unnamed Layer* 67) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2825 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2825 with (Unnamed Layer* 70) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2826 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2826 with (Unnamed Layer* 73) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2851 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2851 with (Unnamed Layer* 679) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2852 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2852 with (Unnamed Layer* 682) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2853 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2853 with (Unnamed Layer* 705) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2854 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2854 with (Unnamed Layer* 725) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2855 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2855 with (Unnamed Layer* 728) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2856 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2856 with (Unnamed Layer* 731) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2881 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2881 with (Unnamed Layer* 1329) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2882 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2882 with (Unnamed Layer* 1332) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2883 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2883 with (Unnamed Layer* 1350) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2884 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2884 with (Unnamed Layer* 1370) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2885 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2885 with (Unnamed Layer* 1373) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2886 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2886 with (Unnamed Layer* 1376) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2911 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2911 with (Unnamed Layer* 1974) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2912 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2912 with (Unnamed Layer* 1977) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2913 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2913 with (Unnamed Layer* 1995) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2914 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2914 with (Unnamed Layer* 2015) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2915 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2915 with (Unnamed Layer* 2018) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2916 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2916 with (Unnamed Layer* 2021) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2941 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2941 with (Unnamed Layer* 2619) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2942 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2942 with (Unnamed Layer* 2622) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2943 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2943 with (Unnamed Layer* 2640) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2944 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2944 with (Unnamed Layer* 2660) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2945 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2945 with (Unnamed Layer* 2663) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2946 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2946 with (Unnamed Layer* 2666) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2971 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2971 with (Unnamed Layer* 3264) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2972 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2972 with (Unnamed Layer* 3267) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2973 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2973 with (Unnamed Layer* 3285) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2974 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2974 with (Unnamed Layer* 3305) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2975 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2975 with (Unnamed Layer* 3308) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_2976 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_2976 with (Unnamed Layer* 3311) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3001 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3001 with (Unnamed Layer* 3909) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3002 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3002 with (Unnamed Layer* 3912) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3003 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3003 with (Unnamed Layer* 3930) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3004 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3004 with (Unnamed Layer* 3950) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3005 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3005 with (Unnamed Layer* 3953) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3006 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3006 with (Unnamed Layer* 3956) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3031 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3031 with (Unnamed Layer* 4554) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3032 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3032 with (Unnamed Layer* 4557) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3033 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3033 with (Unnamed Layer* 4575) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3034 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3034 with (Unnamed Layer* 4595) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3035 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3035 with (Unnamed Layer* 4598) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3036 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3036 with (Unnamed Layer* 4601) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3061 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3061 with (Unnamed Layer* 5199) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3062 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3062 with (Unnamed Layer* 5202) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3063 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3063 with (Unnamed Layer* 5220) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on /transformer/Reshape +[12/30/2022-19:54:25] [V] [TRT] Removing /transformer/Reshape +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/Reshape_1 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/Reshape_1 with /transformer/Unsqueeze_2 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/Reshape_1 + /transformer/Unsqueeze_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/Reshape_1 + /transformer/Unsqueeze_2 with /transformer/Unsqueeze_3 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.0/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.0/attn/Reshape_2 with /transformer/h.0/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.0/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.0/attn/Reshape_4 with /transformer/h.0/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.0/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.0/attn/Reshape_6 with /transformer/h.0/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 665) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 665) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.0/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.0/attn/Transpose_4 with /transformer/h.0/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.1/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.1/attn/Reshape_2 with /transformer/h.1/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.1/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.1/attn/Reshape_4 with /transformer/h.1/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.1/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.1/attn/Reshape_6 with /transformer/h.1/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 1316) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 1316) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.1/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.1/attn/Transpose_4 with /transformer/h.1/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.2/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.2/attn/Reshape_2 with /transformer/h.2/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.2/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.2/attn/Reshape_4 with /transformer/h.2/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.2/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.2/attn/Reshape_6 with /transformer/h.2/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 1961) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 1961) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.2/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.2/attn/Transpose_4 with /transformer/h.2/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.3/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.3/attn/Reshape_2 with /transformer/h.3/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.3/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.3/attn/Reshape_4 with /transformer/h.3/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.3/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.3/attn/Reshape_6 with /transformer/h.3/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 2606) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 2606) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.3/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.3/attn/Transpose_4 with /transformer/h.3/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.4/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.4/attn/Reshape_2 with /transformer/h.4/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.4/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.4/attn/Reshape_4 with /transformer/h.4/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.4/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.4/attn/Reshape_6 with /transformer/h.4/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 3251) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 3251) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.4/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.4/attn/Transpose_4 with /transformer/h.4/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.5/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.5/attn/Reshape_2 with /transformer/h.5/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.5/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.5/attn/Reshape_4 with /transformer/h.5/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.5/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.5/attn/Reshape_6 with /transformer/h.5/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 3896) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 3896) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.5/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.5/attn/Transpose_4 with /transformer/h.5/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.6/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.6/attn/Reshape_2 with /transformer/h.6/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.6/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.6/attn/Reshape_4 with /transformer/h.6/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.6/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.6/attn/Reshape_6 with /transformer/h.6/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 4541) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 4541) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.6/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.6/attn/Transpose_4 with /transformer/h.6/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.7/attn/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.7/attn/Reshape_2 with /transformer/h.7/attn/Transpose +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.7/attn/Reshape_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.7/attn/Reshape_4 with /transformer/h.7/attn/Unsqueeze_7 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.7/attn/Reshape_6 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.7/attn/Reshape_6 with /transformer/h.7/attn/Unsqueeze_11 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on (Unnamed Layer* 5186) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Removing (Unnamed Layer* 5186) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleShuffleFusion on /transformer/h.7/attn/Transpose_4 +[12/30/2022-19:54:25] [V] [TRT] ShuffleShuffleFusion: Fusing /transformer/h.7/attn/Transpose_4 with /transformer/h.7/attn/Reshape_9 +[12/30/2022-19:54:25] [V] [TRT] Running: ShuffleErasure on /transformer/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] Removing /transformer/Reshape_2 +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on onnx::MatMul_3064 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing onnx::MatMul_3064 with (Unnamed Layer* 5245) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on lm_head.bias +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing lm_head.bias with (Unnamed Layer* 5248) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] After Myelin optimization: 44 layers +[12/30/2022-19:54:25] [V] [TRT] Applying ScaleNodes fusions. +[12/30/2022-19:54:25] [V] [TRT] After scale fusion: 44 layers +[12/30/2022-19:54:25] [V] [TRT] Running: ConstantSplit on /transformer/h.0/attn/Constant_34_output_0 +[12/30/2022-19:54:25] [V] [TRT] Running: ConstantSplit on /transformer/Constant_7_output_0 +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0 with (Unnamed Layer* 236) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_1 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_1 with (Unnamed Layer* 889) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_2 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_2 with (Unnamed Layer* 1534) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_3 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_3 with (Unnamed Layer* 2179) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_4 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_4 with (Unnamed Layer* 2824) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_5 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_5 with (Unnamed Layer* 3469) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_6 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_6 with (Unnamed Layer* 4114) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] Running: ConstShuffleFusion on /transformer/Constant_7_output_0_clone_7 +[12/30/2022-19:54:25] [V] [TRT] ConstShuffleFusion: Fusing /transformer/Constant_7_output_0_clone_7 with (Unnamed Layer* 4759) [Shuffle] +[12/30/2022-19:54:25] [V] [TRT] After dupe layer removal: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After final dead-layer removal: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After tensor merging: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After vertical fusions: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After dupe layer removal: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After final dead-layer removal: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After tensor merging: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After slice removal: 50 layers +[12/30/2022-19:54:25] [V] [TRT] After concat removal: 50 layers +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.0/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.0/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.0/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.0/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.1/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.1/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.1/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.1/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.2/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.2/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.2/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.2/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.3/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.3/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.3/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.3/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.4/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.4/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.4/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.4/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.5/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.5/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.5/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.5/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.6/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.6/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.6/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.6/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.7/attn/Cos +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.7/attn/Cos from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Running: UnaryToPointwiseConversion on /transformer/h.7/attn/Sin +[12/30/2022-19:54:25] [V] [TRT] Swap the layer type of /transformer/h.7/attn/Sin from UNARY to POINTWISE +[12/30/2022-19:54:25] [V] [TRT] Trying to split Reshape and strided tensor +[12/30/2022-19:54:25] [V] [TRT] Graph construction and optimization completed in 0.302387 seconds. +[12/30/2022-19:54:26] [V] [TRT] Using cublasLt as a tactic source +[12/30/2022-19:54:26] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +840, GPU +360, now: CPU 1960, GPU 723 (MiB) +[12/30/2022-19:54:26] [V] [TRT] Using cuDNN as a tactic source +[12/30/2022-19:54:26] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +125, GPU +60, now: CPU 2085, GPU 783 (MiB) +[12/30/2022-19:54:26] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored. +[12/30/2022-19:54:26] [V] [TRT] Constructing optimization profile number 0 [1/1]. +[12/30/2022-19:54:26] [V] [TRT] Reserving memory for host IO tensors. Host: 0 bytes +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing reformatting costs +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float() *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.7/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00263306 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00263306 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.6/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00263963 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00263963 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.5/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00260713 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00260713 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.4/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00263497 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00263497 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.3/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00270223 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00270223 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.2/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00263023 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00263023 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.1/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00263323 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00263323 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Float(), Float(1) -> Float(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.0/attn/Range_1 (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00277108 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00277108 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.7/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.0028298 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.0028298 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.6/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00319685 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00319685 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.5/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00377408 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00377408 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.4/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00406209 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00406209 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.3/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00424551 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00424551 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.2/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00443775 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00443775 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.1/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00479878 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00479878 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: -> Int32(1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: /transformer/h.0/attn/Range (Fill) +[12/30/2022-19:54:26] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00537448 +[12/30/2022-19:54:26] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00537448 +[12/30/2022-19:54:26] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Fill Tactic: 0x0000000000000000 +[12/30/2022-19:54:26] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:26] [V] [TRT] *************** Autotuning format combination: Int32(1), Int32(1), Int32(1), Int32(1), Int32(1), Int32(1), Int32(1), Int32(1), Float(1), Float(1), Float(1), Float(1), Float(1), Float(1), Float(1), Float(1) -> Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1) *************** +[12/30/2022-19:54:26] [V] [TRT] --------------- Timing Runner: {ForeignNode[(Unnamed Layer* 224) [Shuffle].../transformer/h.0/attn/Einsum]} (Myelin) +[12/30/2022-19:54:27] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.0723589 +[12/30/2022-19:54:27] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.0723589 +[12/30/2022-19:54:27] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Myelin Tactic: 0x0000000000000000 +[12/30/2022-19:54:27] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:27] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:27] [V] [TRT] --------------- Timing Runner: PWN(/transformer/h.0/attn/Cos) (PointWiseV2) +[12/30/2022-19:54:27] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00277794 +[12/30/2022-19:54:27] [V] [TRT] Tactic: 0x0000000000000001 Time: 0.00298095 +[12/30/2022-19:54:27] [V] [TRT] Tactic: 0x0000000000000002 Time: 0.00278101 +[12/30/2022-19:54:28] [V] [TRT] Tactic: 0x0000000000000003 Time: 0.00340441 +[12/30/2022-19:54:28] [V] [TRT] Tactic: 0x0000000000000004 Time: 0.00298505 +[12/30/2022-19:54:28] [V] [TRT] Tactic: 0x0000000000000005 Time: 0.00286091 +[12/30/2022-19:54:28] [V] [TRT] Tactic: 0x0000000000000006 Time: 0.00434022 +[12/30/2022-19:54:28] [V] [TRT] Tactic: 0x0000000000000007 Time: 0.00340157 +[12/30/2022-19:54:28] [V] [TRT] Tactic: 0x0000000000000008 Time: 0.00302505 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000009 Time: 0.00304836 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x000000000000001c Time: 0.00279059 +[12/30/2022-19:54:29] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 0.00277794 +[12/30/2022-19:54:29] [V] [TRT] --------------- Timing Runner: PWN(/transformer/h.0/attn/Cos) (PointWise) +[12/30/2022-19:54:29] [V] [TRT] PointWise has no valid tactics for this config, skipping +[12/30/2022-19:54:29] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: PointWiseV2 Tactic: 0x0000000000000000 +[12/30/2022-19:54:29] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:29] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:29] [V] [TRT] --------------- Timing Runner: PWN(/transformer/h.0/attn/Sin) (PointWiseV2) +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000000 Time: 0.00276958 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000001 Time: 0.00296891 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000002 Time: 0.00278013 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000003 Time: 0.00341116 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000004 Time: 0.00298343 +[12/30/2022-19:54:29] [V] [TRT] Tactic: 0x0000000000000005 Time: 0.00280946 +[12/30/2022-19:54:30] [V] [TRT] Tactic: 0x0000000000000006 Time: 0.00433399 +[12/30/2022-19:54:30] [V] [TRT] Tactic: 0x0000000000000007 Time: 0.00341279 +[12/30/2022-19:54:30] [V] [TRT] Tactic: 0x0000000000000008 Time: 0.00304943 +[12/30/2022-19:54:30] [V] [TRT] Tactic: 0x0000000000000009 Time: 0.00298981 +[12/30/2022-19:54:30] [V] [TRT] Tactic: 0x000000000000001c Time: 0.0027629 +[12/30/2022-19:54:30] [V] [TRT] Fastest Tactic: 0x000000000000001c Time: 0.0027629 +[12/30/2022-19:54:30] [V] [TRT] --------------- Timing Runner: PWN(/transformer/h.0/attn/Sin) (PointWise) +[12/30/2022-19:54:30] [V] [TRT] PointWise has no valid tactics for this config, skipping +[12/30/2022-19:54:30] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: PointWiseV2 Tactic: 0x000000000000001c +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Float(32,1) -> Float(32,1) *************** +[12/30/2022-19:54:30] [V] [TRT] =============== Computing costs for +[12/30/2022-19:54:30] [V] [TRT] *************** Autotuning format combination: Int32(10,1), Float(32,1), Float(32,1), Int32(10,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1), Float(32,1) -> Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(5120,1280,128,1), Float(504000,50400,1) *************** +[12/30/2022-19:54:30] [V] [TRT] --------------- Timing Runner: {ForeignNode[(Unnamed Layer* 5216) [Shuffle].../lm_head/Add]} (Myelin) +[12/30/2022-19:54:46] [V] [TRT] Tactic: 0x0000000000000000 Time: 1.39147 +[12/30/2022-19:54:46] [V] [TRT] Fastest Tactic: 0x0000000000000000 Time: 1.39147 +[12/30/2022-19:54:46] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: Myelin Tactic: 0x0000000000000000 +[12/30/2022-19:54:46] [V] [TRT] Formats and tactics selection completed in 19.6311 seconds. +[12/30/2022-19:54:46] [V] [TRT] After reformat layers: 50 layers +[12/30/2022-19:54:46] [V] [TRT] Pre-optimized block assignment. +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 6235422720 +[12/30/2022-19:54:46] [V] [TRT] Total Activation Memory: 6235467776 +[12/30/2022-19:54:46] [I] [TRT] Detected 2 inputs and 17 output network tensors. +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_2 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_3 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_4 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_5 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_6 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Constant_34_output_0_clone_7 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0 + (Unnamed Layer* 236) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_1 + (Unnamed Layer* 889) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_2 + (Unnamed Layer* 1534) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_3 + (Unnamed Layer* 2179) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_4 + (Unnamed Layer* 2824) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_5 + (Unnamed Layer* 3469) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_6 + (Unnamed Layer* 4114) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/Constant_7_output_0_clone_7 + (Unnamed Layer* 4759) [Shuffle] Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.7/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.6/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.5/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.4/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.3/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.2/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.1/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Range_1 Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.7/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.6/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.5/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.4/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.3/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.2/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.1/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: /transformer/h.0/attn/Range Host Persistent: 0 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: {ForeignNode[(Unnamed Layer* 224) [Shuffle].../transformer/h.0/attn/Einsum]} Host Persistent: 24 Device Persistent: 0 Scratch Memory: 2304 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.0/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.0/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.1/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.1/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.2/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.2/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.3/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.3/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.4/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.4/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.5/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.5/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.6/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.6/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.7/attn/Cos) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: PWN(/transformer/h.7/attn/Sin) Host Persistent: 244 Device Persistent: 0 Scratch Memory: 0 +[12/30/2022-19:54:46] [V] [TRT] Layer: {ForeignNode[(Unnamed Layer* 5216) [Shuffle].../lm_head/Add]} Host Persistent: 24 Device Persistent: 0 Scratch Memory: 1177600 +[12/30/2022-19:54:46] [I] [TRT] Total Host Persistent Memory: 4160 +[12/30/2022-19:54:46] [I] [TRT] Total Device Persistent Memory: 0 +[12/30/2022-19:54:46] [I] [TRT] Total Scratch Memory: 1177600 +[12/30/2022-19:54:46] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 0 MiB +[12/30/2022-19:54:46] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 2.63702ms to assign 25 blocks to 42 nodes requiring 1207296 bytes. +[12/30/2022-19:54:46] [V] [TRT] Optimized block assignment. +[12/30/2022-19:54:46] [V] [TRT] Block size 1177600 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 1536 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [V] [TRT] Block size 512 +[12/30/2022-19:54:46] [I] [TRT] Total Activation Memory: 1207296 +[12/30/2022-19:54:46] [V] [TRT] Disabling unused tactic source: CUDNN +[12/30/2022-19:54:46] [V] [TRT] Disabling unused tactic source: CUBLAS, CUBLAS_LT +[12/30/2022-19:54:46] [V] [TRT] Disabling unused tactic source: EDGE_MASK_CONVOLUTIONS +[12/30/2022-19:54:46] [V] [TRT] Engine generation completed in 20.7639 seconds. +[12/30/2022-19:54:46] [V] [TRT] Deleting timing cache: 2 entries, served 14 hits since creation. +[12/30/2022-19:54:46] [V] [TRT] Engine Layer Information: +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_1, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_1[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_2, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_2[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_3, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_3[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_4, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_4[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_5, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_5[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_6, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_6[Float()] +Layer(Constant): /transformer/h.0/attn/Constant_34_output_0_clone_7, Tactic: 0x0000000000000000, -> (Unnamed Layer* 227) [Constant]_output_clone_7[Float()] +Layer(Constant): /transformer/Constant_7_output_0 + (Unnamed Layer* 236) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 236) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_1 + (Unnamed Layer* 889) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 889) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_2 + (Unnamed Layer* 1534) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 1534) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_3 + (Unnamed Layer* 2179) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 2179) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_4 + (Unnamed Layer* 2824) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 2824) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_5 + (Unnamed Layer* 3469) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 3469) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_6 + (Unnamed Layer* 4114) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 4114) [Shuffle]_output[Float(1)] +Layer(Constant): /transformer/Constant_7_output_0_clone_7 + (Unnamed Layer* 4759) [Shuffle], Tactic: 0x0000000000000000, -> (Unnamed Layer* 4759) [Shuffle]_output[Float(1)] +Layer(Fill): /transformer/h.7/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_7[Float()], (Unnamed Layer* 4759) [Shuffle]_output[Float(1)] -> /transformer/h.7/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.6/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_6[Float()], (Unnamed Layer* 4114) [Shuffle]_output[Float(1)] -> /transformer/h.6/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.5/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_5[Float()], (Unnamed Layer* 3469) [Shuffle]_output[Float(1)] -> /transformer/h.5/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.4/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_4[Float()], (Unnamed Layer* 2824) [Shuffle]_output[Float(1)] -> /transformer/h.4/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.3/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_3[Float()], (Unnamed Layer* 2179) [Shuffle]_output[Float(1)] -> /transformer/h.3/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.2/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_2[Float()], (Unnamed Layer* 1534) [Shuffle]_output[Float(1)] -> /transformer/h.2/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.1/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output_clone_1[Float()], (Unnamed Layer* 889) [Shuffle]_output[Float(1)] -> /transformer/h.1/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.0/attn/Range_1, Tactic: 0x0000000000000000, (Unnamed Layer* 227) [Constant]_output[Float()], (Unnamed Layer* 236) [Shuffle]_output[Float(1)] -> /transformer/h.0/attn/Range_1_output_0[Float(10)] +Layer(Fill): /transformer/h.7/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.7/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.6/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.6/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.5/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.5/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.4/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.4/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.3/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.3/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.2/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.2/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.1/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.1/attn/Range_output_0[Int32(32)] +Layer(Fill): /transformer/h.0/attn/Range, Tactic: 0x0000000000000000, -> /transformer/h.0/attn/Range_output_0[Int32(32)] +Layer(Myelin): {ForeignNode[(Unnamed Layer* 224) [Shuffle].../transformer/h.0/attn/Einsum]}, Tactic: 0x0000000000000000, /transformer/h.0/attn/Range_output_0[Int32(32)], /transformer/h.1/attn/Range_output_0[Int32(32)], /transformer/h.2/attn/Range_output_0[Int32(32)], /transformer/h.3/attn/Range_output_0[Int32(32)], /transformer/h.4/attn/Range_output_0[Int32(32)], /transformer/h.5/attn/Range_output_0[Int32(32)], /transformer/h.6/attn/Range_output_0[Int32(32)], /transformer/h.7/attn/Range_output_0[Int32(32)], /transformer/h.7/attn/Range_1_output_0[Float(10)], /transformer/h.6/attn/Range_1_output_0[Float(10)], /transformer/h.5/attn/Range_1_output_0[Float(10)], /transformer/h.4/attn/Range_1_output_0[Float(10)], /transformer/h.3/attn/Range_1_output_0[Float(10)], /transformer/h.2/attn/Range_1_output_0[Float(10)], /transformer/h.1/attn/Range_1_output_0[Float(10)], /transformer/h.0/attn/Range_1_output_0[Float(10)] -> /transformer/h.7/attn/Einsum_output_0[Float(10,32)], /transformer/h.6/attn/Einsum_output_0[Float(10,32)], /transformer/h.5/attn/Einsum_output_0[Float(10,32)], /transformer/h.4/attn/Einsum_output_0[Float(10,32)], /transformer/h.3/attn/Einsum_output_0[Float(10,32)], /transformer/h.2/attn/Einsum_output_0[Float(10,32)], /transformer/h.1/attn/Einsum_output_0[Float(10,32)], /transformer/h.0/attn/Einsum_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.0/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.0/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.0/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.0/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.0/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.0/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.1/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.1/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.1/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.1/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.1/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.1/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.2/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.2/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.2/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.2/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.2/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.2/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.3/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.3/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.3/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.3/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.3/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.3/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.4/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.4/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.4/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.4/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.4/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.4/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.5/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.5/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.5/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.5/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.5/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.5/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.6/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.6/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.6/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.6/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.6/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.6/attn/Sin_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.7/attn/Cos), Tactic: 0x0000000000000000, /transformer/h.7/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.7/attn/Cos_output_0[Float(10,32)] +Layer(PointWiseV2): PWN(/transformer/h.7/attn/Sin), Tactic: 0x000000000000001c, /transformer/h.7/attn/Einsum_output_0[Float(10,32)] -> /transformer/h.7/attn/Sin_output_0[Float(10,32)] +Layer(Myelin): {ForeignNode[(Unnamed Layer* 5216) [Shuffle].../lm_head/Add]}, Tactic: 0x0000000000000000, input_ids[Int32(1,10)], /transformer/h.0/attn/Sin_output_0[Float(10,32)], /transformer/h.0/attn/Cos_output_0[Float(10,32)], attention_mask[Int32(1,10)], /transformer/h.1/attn/Cos_output_0[Float(10,32)], /transformer/h.1/attn/Sin_output_0[Float(10,32)], /transformer/h.2/attn/Cos_output_0[Float(10,32)], /transformer/h.2/attn/Sin_output_0[Float(10,32)], /transformer/h.3/attn/Cos_output_0[Float(10,32)], /transformer/h.3/attn/Sin_output_0[Float(10,32)], /transformer/h.4/attn/Cos_output_0[Float(10,32)], /transformer/h.4/attn/Sin_output_0[Float(10,32)], /transformer/h.5/attn/Cos_output_0[Float(10,32)], /transformer/h.5/attn/Sin_output_0[Float(10,32)], /transformer/h.6/attn/Cos_output_0[Float(10,32)], /transformer/h.6/attn/Sin_output_0[Float(10,32)], /transformer/h.7/attn/Cos_output_0[Float(10,32)], /transformer/h.7/attn/Sin_output_0[Float(10,32)] -> present.0.value[Float(1,4,10,128)], present.0.key[Float(1,4,10,128)], present.1.value[Float(1,4,10,128)], present.1.key[Float(1,4,10,128)], present.2.value[Float(1,4,10,128)], present.2.key[Float(1,4,10,128)], present.3.value[Float(1,4,10,128)], present.3.key[Float(1,4,10,128)], present.4.value[Float(1,4,10,128)], present.4.key[Float(1,4,10,128)], present.5.value[Float(1,4,10,128)], present.5.key[Float(1,4,10,128)], present.6.value[Float(1,4,10,128)], present.6.key[Float(1,4,10,128)], present.7.value[Float(1,4,10,128)], present.7.key[Float(1,4,10,128)], logits[Float(1,10,50400)] +[12/30/2022-19:54:46] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB) +[12/30/2022-19:54:46] [I] Engine built in 30.1319 sec. +[12/30/2022-19:54:46] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 1616, GPU 671 (MiB) +[12/30/2022-19:54:46] [I] [TRT] Loaded engine size: 293 MiB +[12/30/2022-19:54:47] [V] [TRT] Deserialization required 179059 microseconds. +[12/30/2022-19:54:47] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB) +[12/30/2022-19:54:47] [I] Engine deserialized in 0.181391 sec. +[12/30/2022-19:54:47] [V] [TRT] Total per-runner device persistent memory is 0 +[12/30/2022-19:54:47] [V] [TRT] Total per-runner host persistent memory is 4160 +[12/30/2022-19:54:47] [V] [TRT] Allocated activation device memory of size 1207296 +[12/30/2022-19:54:47] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB) +[12/30/2022-19:54:47] [I] Using random values for input input_ids +[12/30/2022-19:54:47] [I] Created input binding for input_ids with dimensions 1x10 +[12/30/2022-19:54:47] [I] Using random values for input attention_mask +[12/30/2022-19:54:47] [I] Created input binding for attention_mask with dimensions 1x10 +[12/30/2022-19:54:47] [I] Using random values for output present.0.value +[12/30/2022-19:54:47] [I] Created output binding for present.0.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.0.key +[12/30/2022-19:54:47] [I] Created output binding for present.0.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.1.value +[12/30/2022-19:54:47] [I] Created output binding for present.1.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.1.key +[12/30/2022-19:54:47] [I] Created output binding for present.1.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.2.value +[12/30/2022-19:54:47] [I] Created output binding for present.2.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.2.key +[12/30/2022-19:54:47] [I] Created output binding for present.2.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.3.value +[12/30/2022-19:54:47] [I] Created output binding for present.3.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.3.key +[12/30/2022-19:54:47] [I] Created output binding for present.3.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.4.value +[12/30/2022-19:54:47] [I] Created output binding for present.4.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.4.key +[12/30/2022-19:54:47] [I] Created output binding for present.4.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.5.value +[12/30/2022-19:54:47] [I] Created output binding for present.5.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.5.key +[12/30/2022-19:54:47] [I] Created output binding for present.5.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.6.value +[12/30/2022-19:54:47] [I] Created output binding for present.6.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.6.key +[12/30/2022-19:54:47] [I] Created output binding for present.6.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.7.value +[12/30/2022-19:54:47] [I] Created output binding for present.7.value with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output present.7.key +[12/30/2022-19:54:47] [I] Created output binding for present.7.key with dimensions 1x4x10x128 +[12/30/2022-19:54:47] [I] Using random values for output logits +[12/30/2022-19:54:47] [I] Created output binding for logits with dimensions 1x10x50400 +[12/30/2022-19:54:47] [I] Starting inference +[12/30/2022-19:54:50] [I] The e2e network timing is not reported since it is inaccurate due to the extra synchronizations when the profiler is enabled. +[12/30/2022-19:54:50] [I] To show e2e network timing report, add --separateProfileRun to profile layer timing in a separate run or remove --dumpProfile to disable the profiler. +[12/30/2022-19:54:50] [I] +[12/30/2022-19:54:50] [I] === Profile (1874 iterations ) === +[12/30/2022-19:54:50] [I] Layer Time (ms) Avg. Time (ms) Median Time (ms) Time % +[12/30/2022-19:54:50] [I] /transformer/h.7/attn/Range_1 10.20 0.0054 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.6/attn/Range_1 9.32 0.0050 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.5/attn/Range_1 9.21 0.0049 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.4/attn/Range_1 9.49 0.0051 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.3/attn/Range_1 8.84 0.0047 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.2/attn/Range_1 8.83 0.0047 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.1/attn/Range_1 9.05 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.0/attn/Range_1 9.20 0.0049 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.7/attn/Range 9.11 0.0049 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.6/attn/Range 8.71 0.0046 0.0041 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.5/attn/Range 8.75 0.0047 0.0051 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.4/attn/Range 8.95 0.0048 0.0041 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.3/attn/Range 8.72 0.0047 0.0041 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.2/attn/Range 8.70 0.0046 0.0041 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.1/attn/Range 8.63 0.0046 0.0041 0.3 +[12/30/2022-19:54:50] [I] /transformer/h.0/attn/Range 8.84 0.0047 0.0041 0.3 +[12/30/2022-19:54:50] [I] {ForeignNode[(Unnamed Layer* 224) [Shuffle].../transformer/h.0/attn/Einsum]} 125.29 0.0669 0.0666 4.2 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.0/attn/Cos) 10.11 0.0054 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.0/attn/Sin) 9.35 0.0050 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.1/attn/Cos) 9.43 0.0050 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.1/attn/Sin) 9.41 0.0050 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.2/attn/Cos) 9.01 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.2/attn/Sin) 8.97 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.3/attn/Cos) 9.08 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.3/attn/Sin) 8.96 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.4/attn/Cos) 9.09 0.0049 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.4/attn/Sin) 9.00 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.5/attn/Cos) 9.14 0.0049 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.5/attn/Sin) 9.22 0.0049 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.6/attn/Cos) 9.06 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.6/attn/Sin) 9.08 0.0048 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.7/attn/Cos) 9.44 0.0050 0.0051 0.3 +[12/30/2022-19:54:50] [I] PWN(/transformer/h.7/attn/Sin) 9.45 0.0050 0.0051 0.3 +[12/30/2022-19:54:50] [I] {ForeignNode[(Unnamed Layer* 5216) [Shuffle].../lm_head/Add]} 2590.90 1.3826 1.3735 86.1 +[12/30/2022-19:54:50] [I] Total 3008.55 1.6054 1.5964 100.0 +[12/30/2022-19:54:50] [I] +&&&& PASSED TensorRT.trtexec [TensorRT v8403] # /home/fxmarty/Downloads/TensorRT-8.4.3.1/targets/x86_64-linux-gnu/bin/trtexec --onnx=decoder_model_sim.onnx --shapes=input_ids:1x10,attention_mask:1x10 --verbose --dumpProfile --profilingVerbosity=detailed