replit/replit-code-v1_5-3b · Error with new version of transformers v4.35.0

Nov 12, 2023

There is an error with this model with the new version of transformers v4.35.0.
To reproduce:

from accelerate import init_empty_weights
from transformers import AutoModelForCausalLM, AutoConfig

model_id = "replit/replit-code-v1_5-3b"
config = AutoConfig.from_pretrained(
    model_id, trust_remote_code=True, revision="b85a5dd28ef6053ad095bf5113dd740a86a67597"
)
with init_empty_weights():
    model = AutoModelForCausalLM.from_config(
        config, trust_remote_code=True, code_revision="b85a5dd28ef6053ad095bf5113dd740a86a67597"
    )

Or run the following colab.
There is no error with the previous version of transformers v4.34.1.

lmnewey

Dec 14, 2023

•

edited Dec 14, 2023

I was having a similar issue, is the replit tokenizer issue ?

mike-ravkine

Dec 29, 2023

I can confirm this model no longer works with transformers past 4.34.1

The issue is this commit https://github.com/huggingface/transformers/commit/ac5893756bafcd745d93a442cf36f984545dbad8 moving a bunch of code around and https://huggingface.co/replit/replit-code-v1_5-3b/blob/main/hf_prefixlm_converter.py relying on it's old location.