String "<|endoftext" in the end of every model answer
There is "<|endoftext" at the end of every model answer. May be you need to apply this fix:
https://huggingface.co/tiiuae/falcon-40b-instruct/commit/662a9a4ffd96f4f73dd18141b60962f94b743c56
Please see this discussion:
https://huggingface.co/tiiuae/falcon-40b-instruct/discussions/11#64788db09c1f42c1f4de5c87
How to load this model
I ran this using the code provided in the model card but got this error
raise ValueError(
ValueError: Could not load model tiiuae/falcon-7b-instruct with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.falcon.modeling_falcon.FalconForCausalLM'>). See the original errors:
ValueError: The current device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format.
Please help me out here,
Thanks!