Update README.md

README.md

@@ -39,9 +39,10 @@ device = torch.device("cuda")
 model = AutoModelForCausalLM.from_pretrained(model_path).to(device, dtype=torch.bfloat16)
 tokenizer = AutoTokenizer.from_pretrained(model_path)
 
-def ask_cosmosage(question, answer_start=''):
-    prompt = f"You are cosmosage, an AI programmed to be a cosmology expert. You answer the USER's question clearly in long form, always providing context. When appropriate, provide a reference.USER: {question}ASSISTANT: {answer_start}"
+def ask_cosmosage(question):
+    prompt = f"You are cosmosage, an AI programmed to provide excellent and detailed answers to the user's question. You are an expert cosmology assistant, able to answer questions on the cosmic microwave background, galaxy formation, large scale structure, theoretical cosmology, inflation, big bang nucleosynthesis, cosmology instrumentation, and other related topics. Please assume the user is fluent in scientific terminology. Elaborate where possible to give a complete answer. If you do not know, say you do not know.▁ USER: {question}▁ ASSISTANT:"
     input_ids = tokenizer.encode(prompt, return_tensors="pt").to(device)
+    print(input_ids)
     generated_ids = model.generate(input_ids, max_length=1024, do_sample=True, temperature=0.7, top_k=None, pad_token_id=tokenizer.eos_token_id)
     generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
     answer = generated_text.split("ASSISTANT:")[-1]
@@ -125,6 +126,18 @@ heavy specialization in cosmology has not come at much of a cost on general-purp
 is GSM8k, which is a collection of grade school math problems. Here, cosmosage performs significantly better 
 than OpenHermes-2.5-Mistral-7B.
 
+## Instruction format
+
+cosmosage_v2 was trained with the "inst" chat template as implemented in axolotl v0.4.0. This resulted in an 
+unusual instruction format:
+```raw
+<s> {system prompt}▁ USER: {question}▁ ASSISTANT:
+```
+Note the use of the U+2581 Lower One Eighth Block Unicode Character to separate the different sections. The 
+example code in the Usage section above correctly implements this format.
+
+Fortunately, cosmosage_v2 does not appear to be too sensitive to deviations from this format. 
+
 ## Example output
 
 **User:**