README.md

metadata

datasets:
  - togethercomputer/RedPajama-Data-V2
language:
  - de

LLäMMchen 1B

This is a German Tinyllama 1B language model trained from scratch using the the Tinyllama codebase on the German portion of RedPajama V2.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("LSX-UniWue/llammchen_1b")

tokenizer = AutoTokenizer.from_pretrained("LSX-UniWue/llammchen_1b")

Evaluation

We evaluated our results on the Superkleber benchmark.