Hugging Face's logo

Spaces:
nikravan
/
3ML_bot_RTL
Sleeping

App Files Community
nikravan commited on
Commit
9129e0d
•
1 Parent(s): a3e847e

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +1 -1
app.py CHANGED
@@ -32,7 +32,7 @@ h1 {
32
  display: block;
33
  }
34
  """
35
- inference_dtype=torch.float32
36
  quantization_config = BitsAndBytesConfig(
37
  load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16
38
  )
 
32
  display: block;
33
  }
34
  """
35
+ inference_dtype=torch.bfloat16
36
  quantization_config = BitsAndBytesConfig(
37
  load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16
38
  )