Fixed parameters:

• model_name_or_path: Bhumika/roberta-base-finetuned-sst2
• dataset:
  • path: glue
  • name: sst2
  • calibration_split: None
  • eval_split: validation
  • data_keys: ['sentence']
  • label_keys: ['label']
• quantization_approach: dynamic
• node_exclusion: []
• per_channel: False
• calibration: None
• framework: onnxruntime
• framework_args:
  • opset: 15
  • optimization_level: 1
• aware_training: False

Benchmarked parameters:

• operators_to_quantize: ['Add', 'MatMul'], ['Add']

Evaluation

Below, time metrics for

• Batch size: 8
• Input length: 128 | operators_to_quantize | | latency_mean (original, ms) | latency_mean (optimized, ms) | | throughput (original, /s) | throughput (optimized, /s) | | accuracy (original) | accuracy (optimized) | | :-------------------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: | :-: | :-----------------: | :------------------: | | ['Add'] | | | 454.70 | 361.81 | | | 2.50 | 3.00 | | | 1.0 | 1.0 | | ['Add', 'MatMul'] | | | 474.54 | 135.14 | | | 2.50 | 7.50 | | | 1.0 | 1.0 |