tiiuae

/

falcon-mamba-7b-Q4_K_M-GGUF

like 1

Follow

Technology Innovation Institute 947

Hi! I try using ChatML but it doesn't follow instruction well even for simple task.
For example, it should just answer:
The answer is this.

But sometimes, it answers like this:
Assistant AI: The answer is this.

Not only that... When I set the system message, it also sometimes ignores it ( but I don't know why since it's not always, but often. It's like 50 50 ).

So.. What the prompt format should I choose for this? I'm using llama-cli for this.

And somehow it's end the text by using [end of text]. Not that I mind it if it's 1 token and "[end of text]" returned as a whole. But it's not, so even I try to check with:

if(line.strip() != "[end of text]"):
        print(line, end="", flush=True)

It's still print out the [end of text].

Some of my test in python:

import subprocess

chat = """<|im_start|>system
Assistant is a large language model trained by Daryl.
<|im_end|>
<|im_start|>user
# Michael Jordan: The Greatness of a Legend


Michael Jordan is a legend in the world of basketball, and his greatness is legendary. Known for his incredible talent, work ethic, and dedication to the game, Jordan redefined basketball and became an inspiration for millions of basketball players worldwide. In this article, we will delve into the history of Michael Jordan and explore his rise to greatness.

**Early Life and Amateur Career**

Michael Jordan was born on February 17, 1963, in Wilmington, North Carolina. Growing up, he exhibited an innate talent for basketball. At the age of 5, he made his debut on the basketball court, scoring 5 points in his first game. As a young boy, Jordan honed his skills at St. Mark's Episcopal School, where he played basketball under the guidance of Coach Sam Smith.

Jordan played basketball for two seasons at St. Mark's before transferring to Emslie Banneker School, where he continued his basketball journey. During his high school years, Jordan averaged over 30 points per game, making him one of the top high school players in the nation.

**College Career**

After graduating from high school, Jordan attended the University of North Carolina at Chapel Hill. There, he continued his basketball journey under coach Dean Smith. Jordan became a key player for the Tar Heels and was named the ACC Player of the Year in his junior year. His college career was cut short when he suffered 
a serious injury that forced him to miss a year.

**College Career**

Despite his injury, Jordan remained steadfast in his determination to succeed. He returned to college in 1983 and began a career as a standout college basketball player. In his junior year, he earned the Associated Press Player of the Year Award and the Naismith College Player of the Year Award. During his college career, Jordan averaged over 30 points per game, solidifying his status as a basketball prodigy.

**College Career**

After graduating from college, Jordan entered the NBA Draft and was drafted by the Chicago Bulls in 1984. His rookie year was phenomenal, with Jordan earning the NBA Rookie of the Year Award and the NBA Slam Dunk Contest Award. During his rookie year, Jordan set the NBA record for most Slam Dunks in a season, scoring 45 in a single game.

**Professional Career**

In his professional career, Jordan continued to push the boundaries of greatness. He won the NBA scoring title in 1990, 1991, and 1992, all while leading his team to the NBA Finals. Jordan's rise to greatness continued with the NBA Finals MVP award in 1996, 1998, and 1999.

Throughout his career, Jordan broke numerous records, and his dedication to the game of basketball never wavered. His achievements on and off the court made him a legend in the world of basketball.

**Legacy**

Michael Jordan's legacy transcended basketball. Jordan's dedication to the sport and his relentless pursuit of greatness made him a model for young basketball players and a symbol of perseverance. Today, Michael Jordan's legacy remains a lasting inspiration for athletes worldwide.

In conclusion, Michael Jordan's rise to greatness is an inspiring tale of determination and relentless pursuit of excellence. Jordan's achievements, records, and legacy make him one of the greatest basketball players of all time. The world has witnessed a phenomenal player in Michael Jordan, and his legacy will never fade.

In article above, when is he drafted by Chicago Bulls? And please cite the article sentences that are relevant to the question.
<|im_end|>
<|im_start|>assistant
"""

process = subprocess.Popen(["ext/llama-cli.exe", 
                        "-m", "llm/falcon-mamba/falcon-mamba-7b-instruct.Q2_K.gguf",
                        "-p", chat],
                        stderr=subprocess.DEVNULL,
                        stdout=subprocess.PIPE) 

while process.poll() is None:
    line = process.stdout.read(1)
    if not line: break

    try:
        line = line.decode('utf-8', errors='ignore')
    except UnicodeDecodeError:
        continue

    if(line.strip() != "[end of text]"):
        print(line, end="", flush=True)

Thanks.