I want to shard another model how can I shard?
#4
by
mandeepbagga
- opened
Hi, hope you are doing well. I'm trying to use another model to finetune it further, but on google collab, memory is getting filled. So, I want to shard the model first I know it will cost me some money, but I'll be able to finetune it on collab later.
It will be cool if someone can tell me if we can use CPU ram to shard a model.
Thanks
Just load the model and do
model.push_to_hub("", max_shard_size="2000MB", use_auth_token=True)
Also push the tokenizer
tokenizer.push_to_hub("", use_auth_token=True)