Hugging Face's logo

Evaluate documentation

Hub methods

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Hub methods

Methods for using the Hugging Face Hub:

Push to hub

evaluate.push_to_hub

< >

( model_id: strtask_type: strdataset_type: strdataset_name: strmetric_type: strmetric_name: strmetric_value: floattask_name: str = Nonedataset_config: str = Nonedataset_split: str = Nonedataset_revision: str = Nonedataset_args: typing.Dict[str, int] = Nonemetric_config: str = Nonemetric_args: typing.Dict[str, int] = Noneoverwrite: bool = False )

Parameters

  • model_id (str) — Model id from https://hf.co/models.
  • task_type (str) — Task id, refer to https://github.com/huggingface/evaluate/blob/main/src/evaluate/config.py#L154 for allowed values.
  • dataset_type (str) — Dataset id from https://hf.co/datasets.
  • dataset_name (str) — Pretty name for the dataset.
  • metric_type (str) — Metric id from https://hf.co/metrics.
  • metric_name (str) — Pretty name for the metric.
  • metric_value (float) — Computed metric value.
  • task_name (str, optional) — Pretty name for the task.
  • dataset_config (str, optional) — Dataset configuration used in datasets.load_dataset(). See huggingface/datasets docs for more info: https://huggingface.co/docs/datasets/package_reference/loading_methods#datasets.load_dataset.name
  • dataset_split (str, optional) — Name of split used for metric computation.
  • dataset_revision (str, optional) — Git hash for the specific version of the dataset.
  • dataset_args (dict[str, int], optional) — Additional arguments passed to datasets.load_dataset().
  • metric_config (str, optional) — Configuration for the metric (e.g. the GLUE metric has a configuration for each subset)
  • metric_args (dict[str, int], optional) — Arguments passed during Metric.compute().
  • overwrite (bool, optional, defaults to False) — If set to True an existing metric field can be overwritten, otherwise attempting to overwrite any existing fields will cause an error.

Pushes the result of a metric to the metadata of a model repository in the Hub.

Example:

>>> push_to_hub(
...     model_id="huggingface/gpt2-wikitext2",
...     metric_value=0.5
...     metric_type="bleu",
...     metric_name="BLEU",
...     dataset_name="WikiText",
...     dataset_type="wikitext",
...     dataset_split="test",
...     task_type="text-generation",
...     task_name="Text Generation"
... )
←Saving methods Evaluator classes→