metadata

tags:
  - text-generation-inference
  - text-generation
  - peft
library_name: transformers
widget:
  - messages:
      - role: user
        content: What is better for gut health protein or carbs ?
license: mit
pipeline_tag: text-generation

Model Trained Using WEBSPACEAI_TRANIER AGENT

WEBSPACEAI Trainer Agent is a powerful tool designed to streamline the training of large language models (LLMs) using PyTorch and TensorFlow. It offers efficient data management, enabling users to curate and organize datasets for optimal model performance. The platform supports comprehensive training strategies, allowing for iterative adjustments to enhance accuracy. With collaboration features and a user-friendly interface, the Trainer Agent makes AI training accessible to both technical and non-technical users, empowering organizations to develop sophisticated AI solutions tailored to their needs.

Usage


from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "PATH_TO_THIS_REPO"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype='auto'
).eval()

# Prompt content: "hi"
messages = [
    {"role": "user", "content": "hi"}
]

input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
output_ids = model.generate(input_ids.to('cuda'))
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)

# Model response: "Hello! How can I assist you today?"
print(response)