sbottazziunsam

/

LLaVA-Med_weights

like 2

Hello,
I downloaded the Llama weights from this link https://huggingface.co/nyanko7/LLaMA-7B/tree/main and followed the instructions in the LlaVA-Med delta repo (https://huggingface.co/microsoft/llava-med-7b-delta) but couldn't make it work yet because of GPU issues, so I don't know yet if they work.

Thank you for your reply！I used your weights andit worked, but I used the weights of llava-med to merge llama-7b-hf and found that it didn't work. Which llama weight did you use?

Hello,
I downloaded the Llama weights from this link https://huggingface.co/nyanko7/LLaMA-7B/tree/main. To convert them into HF format, I followed these instructions https://huggingface.co/docs/transformers/main/model_doc/llama#usage-tips . And then keep on with the guide in the LlaVA-Med delta repo (https://huggingface.co/microsoft/llava-med-7b-delta sections "Install" and "Serving"). Also found this youtube tutorial, maybe it's useful for you https://www.youtube.com/watch?v=qfkKz8JYfcc&t=8s.
May I asked how did you make my weights work? Did you use the "Medical Visual Chat " or the "Medical VQA"?

hello,
I loaded model.worker with your weights, and then used gradient_web for inference, and then it worked. I used my own weights before (similar to the steps you mentioned) and get an error response.So your weights should be correct.

Hello YiHongping, I just got back to LlavaMed and wanted to ask you if you could share the script you used to make the model work or if you followed a tutorial. Thank you!

hello,
I can make inferences directly using your weights, but when I follow the repo tutorial, the inferences will be garbled.

you need to change pydantic 1.10.13 and gradio 3.23.0,and other follows the repo

Hi, I have followed the steps mentioned in the documentation and got the LLaVa-Med weights. Now I need to upload the files and the checkpoints and then create a inference endpoints so that I can use this model with langchain framework to work on my project. Guide me through this

Hello, I'm sorry but I have not knowledge at all of langchain framework. I am also trying to run inference but couldn't manage to do it yet, I was trying to use the "eval" files and also the "serve" folder that are on the Github repo. What YiHongping told me about "model_worker" and "gradient_web" in a previous message I think it's reffered to the files in the "serve" folder in the repo.

@sbottazziunsam Thanks for the info, I have used your weights it working quite well.

@NithinM-29602 I'm glad they worked. May I ask you how did yo use them? Did you run inference? I'd appreciate if you could give me a detailed description of the stepts you took. Thank you

@NithinM-29602 I just followed this tutorial https://www.youtube.com/watch?v=qfkKz8JYfcc&t=8s. But instead of using the delta weight of mine, I used yours. Usually you need to specify the path of the final llava-med weights in the command and it worked.
Have a Great Day...