Failed to deploy the model in inference endpoint with NO error

#25

by Isgservices-builderai - opened Mar 8

Discussion

Isgservices-builderai

Mar 8

•

edited Mar 8

Have tried to create an inference point for this model couple of time and it failed on all occasions without any error. Here is the snippet of my log:
What could be the issue? I'm using Nvidia A100 - 2xGPU.160GB. Have increased Max Number of Tokens to 8K.

Cc @zqh11 - @jxji

fs-tom

Jun 6

Same result Nvidia L4 · 4x GPU · 96 GB

fs-tom

Jun 6

Have tried to create an inference point for this model couple of time and it failed on all occasions without any error. Here is the snippet of my log:
What could be the issue? I'm using Nvidia A100 - 2xGPU.160GB. Have increased Max Number of Tokens to 8K.

Cc @zqh11 - @jxji

I got Bin12345/AutoCoder running

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment