Hugging Face's logo

Optimum documentation

Dummy Input Generators

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v1.24.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Dummy Input Generators

It is very common to have to generate dummy inputs to perform a task (tracing, exporting a model to some backend, testing model outputs, etc). The goal of DummyInputGenerator classes is to make this generation easy and re-usable.

Base class

class optimum.utils.DummyInputGenerator

< >

( )

Generates dummy inputs for the supported input names, in the requested framework.

concat_inputs

< >

( inputsdim: int )

Parameters

  • inputs — The list of tensors in a given framework to concatenate.
  • dim (int) — The dimension along which to concatenate.

Concatenates inputs together.

constant_tensor

< >

( shape: typing.List[int]value: typing.Union[int, float] = 1dtype: typing.Optional[typing.Any] = Noneframework: str = 'pt' )

Parameters

  • shape (List[int]) — The shape of the constant tensor.
  • value (Union[int, float], defaults to 1) — The value to fill the constant tensor with.
  • dtype (Optional[Any], defaults to None) — The dtype of the constant tensor.
  • framework (str, defaults to "pt") — The requested framework.

Generates a constant tensor.

generate

< >

( input_name: strframework: str = 'pt'int_dtype: str = 'int64'float_dtype: str = 'fp32' )

Parameters

  • input_name (str) — The name of the input to generate.
  • framework (str, defaults to "pt") — The requested framework.
  • int_dtype (str, defaults to "int64") — The dtypes of generated integer tensors.
  • float_dtype (str, defaults to "fp32") — The dtypes of generated float tensors.

Generates the dummy input matching input_name for the requested framework.

pad_input_on_dim

< >

( input_dim: intdesired_length: typing.Optional[int] = Nonepadding_length: typing.Optional[int] = Nonevalue: typing.Union[int, float] = 1dtype: typing.Optional[typing.Any] = None )

Parameters

  • input_ — The tensor to pad.
  • dim (int) — The dimension along which to pad.
  • desired_length (Optional[int], defaults to None) — The desired length along the dimension after padding.
  • padding_length (Optional[int], defaults to None) — The length to pad along the dimension.
  • value (Union[int, float], defaults to 1) — The value to use for padding.
  • dtype (Optional[Any], defaults to None) — The dtype of the padding.

Pads an input either to the desired length, or by a padding length.

random_float_tensor

< >

( shape: typing.List[int]min_value: float = 0max_value: float = 1framework: str = 'pt'dtype: str = 'fp32' )

Parameters

  • shape (List[int]) — The shape of the random tensor.
  • min_value (float, defaults to 0) — The minimum value allowed.
  • max_value (float, defaults to 1) — The maximum value allowed.
  • framework (str, defaults to "pt") — The requested framework.
  • dtype (str, defaults to "fp32") — The dtype of the generated float tensor. Could be “fp32”, “fp16”, “bf16”.

Generates a tensor of random floats in the [min_value, max_value) range.

random_int_tensor

< >

( shape: typing.List[int]max_value: intmin_value: int = 0framework: str = 'pt'dtype: str = 'int64' )

Parameters

  • shape (List[int]) — The shape of the random tensor.
  • max_value (int) — The maximum value allowed.
  • min_value (int, defaults to 0) — The minimum value allowed.
  • framework (str, defaults to "pt") — The requested framework.
  • dtype (str, defaults to "int64") — The dtype of the generated integer tensor. Could be “int64”, “int32”, “int8”.

Generates a tensor of random integers in the [min_value, max_value) range.

random_mask_tensor

< >

( shape: typing.List[int]padding_side: str = 'right'framework: str = 'pt'dtype: str = 'int64' )

Parameters

  • shape (List[int]) — The shape of the random tensor.
  • padding_side (str, defaults to “right”) — The side on which the padding is applied.
  • framework (str, defaults to "pt") — The requested framework.
  • dtype (str, defaults to "int64") — The dtype of the generated integer tensor. Could be “int64”, “int32”, “int8”.

Generates a mask tensor either right or left padded.

supports_input

< >

( input_name: str ) → bool

Parameters

  • input_name (str) — The name of the input to generate.

Returns

bool

A boolean specifying whether the input is supported.

Checks whether the DummyInputGenerator supports the generation of the requested input.

Existing dummy input generators

class optimum.utils.DummyTextInputGenerator

< >

( task: strnormalized_config: NormalizedTextConfigbatch_size: int = 2sequence_length: int = 16num_choices: int = 4random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_num_choices_range: typing.Optional[typing.Tuple[int, int]] = Nonepadding_side: str = 'right'**kwargs )

Generates dummy encoder text inputs.

class optimum.utils.DummyDecoderTextInputGenerator

< >

( task: strnormalized_config: NormalizedTextConfigbatch_size: int = 2sequence_length: int = 16num_choices: int = 4random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_num_choices_range: typing.Optional[typing.Tuple[int, int]] = Nonepadding_side: str = 'right'**kwargs )

Generates dummy decoder text inputs.

class optimum.utils.DummyPastKeyValuesGenerator

< >

( task: strnormalized_config: NormalizedTextConfigbatch_size: int = 2sequence_length: int = 16random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None**kwargs )

Generates dummy past_key_values inputs.

class optimum.utils.DummySeq2SeqPastKeyValuesGenerator

< >

( task: strnormalized_config: typing.Union[optimum.utils.normalized_config.NormalizedSeq2SeqConfig, optimum.utils.normalized_config.NormalizedEncoderDecoderConfig]batch_size: int = 2sequence_length: int = 16encoder_sequence_length: typing.Optional[int] = Nonerandom_batch_size_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None**kwargs )

Generates dummy past_key_values inputs for seq2seq architectures.

class optimum.utils.DummyBboxInputGenerator

< >

( task: strnormalized_config: NormalizedConfigbatch_size: int = 2sequence_length: int = 16random_batch_size_range: typing.Optional[typing.Tuple[int, int]] = Nonerandom_sequence_length_range: typing.Optional[typing.Tuple[int, int]] = None**kwargs )

Generates dummy bbox inputs.

class optimum.utils.DummyVisionInputGenerator

< >

( task: strnormalized_config: NormalizedVisionConfigbatch_size: int = 2num_channels: int = 3width: int = 64height: int = 64**kwargs )

Generates dummy vision inputs.

class optimum.utils.DummyAudioInputGenerator

< >

( task: strnormalized_config: NormalizedConfigbatch_size: int = 2feature_size: int = 80nb_max_frames: int = 3000audio_sequence_length: int = 16000**kwargs )

< > Update on GitHub

←GPTQ quantization Normalized configurations→