Llama Batch Inference of Llama-2-70B-Chat-GPTQ

#50

by Ivy111 - opened Mar 6

Mar 6

I want to do llama batch inference so that it can run in multiple parallel environments. Can I implement it using Llama-2-70B-Chat-GPTQ?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment