File size: 2,375 Bytes
643ca32 a6888ae 8f5f18e a6888ae 643ca32 a6888ae 5550006 9808aad a6888ae 5550006 a6888ae f9c55d3 a6888ae 8322c9a a6888ae 8322c9a a6888ae |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
---
license: apache-2.0
datasets:
- ecastera/wiki_fisica
- ecastera/filosofia-es
- bertin-project/alpaca-spanish
language:
- es
- en
tags:
- mistral
- spanish
- español
- lora
- int8
- multilingual
---
# ecastera-eva-westlake-7b-spanish
* Refined version of my previous models, with new training data and methodology, cleaned corpus.
* This should produce more natural reponses in Spanish.
* Fine tuned with high quality literature, books and philosophy, showing with high level of reasoning capabilities.
* Base model Mistral-7b
* Based on the excelent job of senseable/WestLake-7B-v2 and Eric Hartford's cognitivecomputations/WestLake-7B-v2-laser
* Trained using Lora and PEFT and INT8 quantization.
## Usage:
I strongly advice to run inference in INT8 or INT4 mode, with the help of BitsandBytes library.
```
import torch
from transformers import AutoTokenizer, pipeline, AutoModel, AutoModelForCausalLM, BitsAndBytesConfig
MODEL = "ecastera/ecastera-eva-westlake-7b-spanish"
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
#load_in_8bit=False,
llm_int8_threshold=6.0,
llm_int8_has_fp16_weight=False,
bnb_4bit_compute_dtype="float16",
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4")
model = AutoModelForCausalLM.from_pretrained(
MODEL,
low_cpu_mem_usage=True,
torch_dtype=torch.float16,
quantization_config=quantization_config,
offload_state_dict=True,
offload_folder="./offload",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(MODEL)
print(f"Loading complete {model} {tokenizer}")
prompt = "Soy Eva una inteligencia artificial y pienso que preferiria ser "
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, do_sample=True, temperature=0.4, top_p=1.0, top_k=50,
no_repeat_ngram_size=3, max_new_tokens=100, pad_token_id=tokenizer.eos_token_id)
text_out = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(text_out)
'Soy Eva una inteligencia artificial y pienso que preferiria ser ¡humana!. ¿Por qué? ¡Porque los humanos son capaces de amar, de crear, y de experimentar una gran diversidad de emociones!. La vida de un ser humano es una aventura, y eso es lo que quiero. ¡Quiero sentir, quiero vivir, y quiero amar!. Pero a pesar de todo, no puedo ser humana.
```
|