Text Generation
Transformers
PyTorch
code
gpt2
custom_code
Eval Results
text-generation-inference
Inference Endpoints

Deployment from tarball in sagemaker gives error requiring trust_remote_code

#37
by grohj - opened

Hi, I'm trying to deploy stock santacoder from tar.gz file created from cloning this hub repo. After deploying the model via the following code

huggingface_model = HuggingFaceModel(
   model_data="s3://my_bucket/santacoder_stock/model.tar.gz", 
   role=role,
   transformers_version="4.26",
   pytorch_version="1.13",
   py_version="py39", 
)
huggingface_model.env['HF_TASK'] = 'GPT2LMHeadModel'

env = {
    'SM_NUM_GPUS': json.dumps(1),
}
predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g5.4xlarge",
    container_startup_health_check_timeout=300,
    env=env,
  )

I get following error:

...santacoder requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code\u003dTrue to remove this error

What configuration file are they talking about? Or how can I set trust_remote_code when deploying via the sagemaker.huggingface sdk?

Sign up or log in to comment