Where I can find remote code?
I'm in a setting where I can't use trust_remote_code=True
How I can come up with a complete local inference?
The trust_remote_code=True does not call out to any remote endpionts. It simply means that the model code is custom and not from the transformers library. When you download the model to your local machine, you will download all the model source code files too (you can see them here). From that point, you will be running inference locally.
So all you need to do is keep the trust_remote_code=True flag on, and it should work :)
Is there an example handler.py posted anywhere for using the HF custom inference deployment option?
+1 can you include code for custom inference in the README ?
The original ask of this issue has been addressed; for custom handler.py logic for HF Inference's product, I think that the HF discussion boards are a good place to go. I will also mention this thread to them and suggest they add an MPT example to their docs