WOW ..just wow
That is the most capable opensource LLM for coding I ever seen... It has coding level of GPT-4 ... is incredible.
I easily made advanced applications by this llm .... never done that before so easily.
For instance Phind-CodeLlama-34B-v2 is far less capable comparing to WizardCoder-33B-V1.1. That is big step forward now.
Added from my posts:
for me not making errors what I am surprised.
I'am using newest llamacpp and model version q4k_m ( old q4 is much worse )
My code to run llamcpp
build\bin\main.exe --model models\new3\wizardcoder-33b-v1.1.Q4_K_M.gguf --mlock --color --threads 16 --keep -1 --batch_size 512 --n_predict -1 --repeat_penalty 1.1 --ctx_size 0 --interactive --instruct --reverse-prompt "<|im_end|>" -ngl 58 --simple-io --in-prefix " ### Instruction: " --in-suffix " ### Response: " -p "Below is an instruction that describes a task. Write a response that appropriately completes the request. "
Look on that code :
Can you make A snake game in python?
- first try. Look how long code is and was generated without any errors also game looks ..decent.
- Even has working score system.
import turtle
import time
import random
delay = 0.1
score = 0
high_score = 0
# Set up the screen
wn = turtle.Screen()
wn.title("Snake Game")
wn.bgcolor("black")
wn.setup(width=600, height=600)
wn.tracer(0) # Turns off the screen updates
# Snake head
head = turtle.Turtle()
head.speed(0)
head.shape("square")
head.color("white")
head.penup()
head.goto(0, 0)
head.direction = "stop"
# Snake food
food = turtle.Turtle()
food.speed(0)
food.shape("circle")
food.color("red")
food.penup()
food.goto(random.randint(-290, 290), random.randint(-290, 290))
segments = []
# Pen
pen = turtle.Turtle()
pen.speed(0)
pen.shape("square")
pen.color("white")
pen.penup()
pen.hideturtle()
pen.goto(0, 260)
pen.write("Score: 0 High Score: 0", align="center", font=("Courier", 24, "normal"))
# Functions
def go_up():
if head.direction != "down":
head.direction = "up"
def go_down():
if head.direction != "up":
head.direction = "down"
def go_left():
if head.direction != "right":
head.direction = "left"
def go_right():
if head.direction != "left":
head.direction = "right"
def move():
if head.direction == "up":
y = head.ycor()
head.sety(y + 20)
if head.direction == "down":
y = head.ycor()
head.sety(y - 20)
if head.direction == "left":
x = head.xcor()
head.setx(x - 20)
if head.direction == "right":
x = head.xcor()
head.setx(x + 20)
# Keyboard bindings
wn.listen()
wn.onkeypress(go_up, "Up")
wn.onkeypress(go_down, "Down")
wn.onkeypress(go_left, "Left")
wn.onkeypress(go_right, "Right")
# Main game loop
while True:
wn.update()
# Check for a collision with the border
if head.xcor()>290 or head.xcor()<-290 or head.ycor()>290 or head.ycor()<-290:
time.sleep(1)
head.goto(0,0)
head.direction = "stop"
# Hide the segments
for segment in segments:
segment.goto(1000, 1000)
# Clear the segments list
segments.clear()
score = 0
# Check for a collision with the food
if head.distance(food) < 20:
x = random.randint(-290, 290)
y = random.randint(-290, 290)
food.goto(x,y)
# Add a segment
new_segment = turtle.Turtle()
new_segment.speed(0)
new_segment.shape("square")
new_segment.color("grey")
new_segment.penup()
segments.append(new_segment)
# Shorten the delay
delay -= 0.001
score += 10
if score > high_score:
high_score = score
pen.clear()
pen.write("Score: {} High Score: {}".format(score, high_score), align="center", font=("Courier", 24, "normal"))
# Move the end segments first in reverse order
for index in range(len(segments)-1, 0, -1):
x = segments[index-1].xcor()
y = segments[index-1].ycor()
segments[index].goto(x, y)
# Move segment 0 to where the head is
if len(segments) > 0:
x = head.xcor()
y = head.ycor()
segments[0].goto(x, y)
move()
time.sleep(delay)
wn.mainloop()
- Second try to make snake game gave even better results , welcome screen , end screen etc ....
- look how long the code is and works without any corrections
import pygame, sys, time, random
from pygame.locals import *
# Constants that will be used in the game
FPS = 15
WINDOW_WIDTH = 640
WINDOW_HEIGHT = 480
CELLSIZE = 20
assert WINDOW_WIDTH % CELLSIZE == 0, "Window width must be a multiple of cell size."
assert WINDOW_HEIGHT % CELLSIZE == 0, "Window height must be a multiple of cell size."
CELLWIDTH = int(WINDOW_WIDTH / CELLSIZE)
CELLHEIGHT = int(WINDOW_HEIGHT / CELLSIZE)
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)
RED = (255, 0, 0)
GREEN = (0, 255, 0)
DARKGRAY = (40, 40, 40)
BGCOLOR = BLACK
UP = 'up'
DOWN = 'down'
LEFT = 'left'
RIGHT = 'right'
HEAD = 0 # syntactic sugar: index of the snake's head
def main():
global FPSCLOCK, DISPLAYSURF
pygame.init()
FPSCLOCK = pygame.time.Clock()
DISPLAYSURF = pygame.display.set_mode((WINDOW_WIDTH, WINDOW_HEIGHT))
pygame.display.set_caption('Snake')
showStartScreen()
while True: # main game loop
runGame()
showGameOverScreen()
def runGame():
# Game setup
snake = [{'x': 5, 'y': 10}, {'x': 4, 'y': 10}, {'x': 3, 'y': 10}]
direction = RIGHT
newDirection = direction
food = getRandomLocation()
while True: # single game loop
for event in pygame.event.get(): # event handling loop
if event.type == QUIT or (event.type == KEYDOWN and event.key == K_ESCAPE):
terminate()
elif event.type == KEYDOWN:
if event.key == K_LEFT and direction != RIGHT:
newDirection = LEFT
elif event.key == K_RIGHT and direction != LEFT:
newDirection = RIGHT
elif event.key == K_UP and direction != DOWN:
newDirection = UP
elif event.key == K_DOWN and direction != UP:
newDirection = DOWN
if isGameOver(snake):
return
direction = newDirection # only change direction when no QUIT event
# move the snake
if direction == UP:
newHead = {'x': snake[HEAD]['x'], 'y': snake[HEAD]['y'] - 1}
elif direction == DOWN:
newHead = {'x': snake[HEAD]['x'], 'y': snake[HEAD]['y'] + 1}
elif direction == LEFT:
newHead = {'x': snake[HEAD]['x'] - 1, 'y': snake[HEAD]['y']}
elif direction == RIGHT:
newHead = {'x': snake[HEAD]['x'] + 1, 'y': snake[HEAD]['y']}
snake.insert(0, newHead) # insert and del operations are fast at the beginning of a list
if (newHead == food): # have we eaten?
food = getRandomLocation() # spawn new food object
else: # move snake by removing tail
snake.pop()
# Draw the game world on the window
DISPLAYSURF.fill(BGCOLOR)
drawGrid()
drawSnake(snake)
drawObject(food, RED)
pygame.display.update()
FPSCLOCK.tick(FPS)
def drawGrid():
for x in range(0, WINDOW_WIDTH, CELLSIZE): # draw vertical lines
pygame.draw.line(DISPLAYSURF, DARKGRAY, (x, 0), (x, WINDOW_HEIGHT))
for y in range (0, WINDOW_HEIGHT, CELLSIZE): # draw horizontal lines
pygame.draw.line(DISPLAYSURF, DARKGRAY, (0, y), (WINDOW_WIDTH, y))
def drawSnake(snakeList):
for coord in snakeList:
x = coord['x'] * CELLSIZE
y = coord['y'] * CELLSIZE
snakeSegmentRect = pygame.Rect(x, y, CELLSIZE, CELLSIZE)
pygame.draw.rect(DISPLAYSURF, GREEN, snakeSegmentRect)
snakeInnerSegmentRect = pygame.Rect(x + 4, y + 4, CELLSIZE - 8, CELLSIZE - 8)
pygame.draw.rect(DISPLAYSURF, WHITE, snakeInnerSegmentRect)
def drawObject(coord, color):
x = coord['x'] * CELLSIZE
y = coord['y'] * CELLSIZE
foodRect = pygame.Rect(x, y, CELLSIZE, CELLSIZE)
pygame.draw.rect(DISPLAYSURF, color, foodRect)
def getRandomLocation():
return {'x': random.randint(0, CELLWIDTH - 1), 'y': random.randint(0, CELLHEIGHT - 1)}
def isGameOver(snake):
if (snake[HEAD]['x'] == -1 or snake[HEAD]['x'] == CELLWIDTH or
snake[HEAD]['y'] == -1 or snake[HEAD]['y'] == CELLHEIGHT): # hit the wall
return True
for snakeBody in snake[1:]: # hit itself
if snakeBody == snake[HEAD]:
return True
return False
def showStartScreen():
titleFont = pygame.font.Font('freesansbold.ttf', 100)
titleSurf1 = titleFont.render('Snake Game', True, WHITE, DARKGRAY)
titleSurf2 = titleFont.render('Press a key to start.', True, WHITE)
DISPLAYSURF.blit(titleSurf1, (int((WINDOW_WIDTH - titleSurf1.get_width()) / 2), int((WINDOW_HEIGHT - titleSurf1.get_height()) / 2)))
DISPLAYSURF.blit(titleSurf2, (int((WINDOW_WIDTH - titleSurf2.get_width()) / 2), int((WINDOW_HEIGHT - titleSurf1.get_height()) / 2) + 100))
pygame.display.update()
waitForPlayer()
def showGameOverScreen():
gameOverFont = pygame.font.Font('freesansbold.ttf', 150)
gameSurf = gameOverFont.render('Game', True, WHITE)
overSurf = gameOverFont.render('Over', True, WHITE)
DISPLAYSURF.blit(gameSurf, (int((WINDOW_WIDTH - gameSurf.get_width()) / 2), int((WINDOW_HEIGHT - gameSurf.get_height()) / 2)))
DISPLAYSURF.blit(overSurf, (int((WINDOW_WIDTH - overSurf.get_width()) / 2), int((WINDOW_HEIGHT - overSurf.get_height()) / 2) + 100))
drawPressKeyMsg()
pygame.display.update()
waitForPlayer()
gameOverFont = pygame.font.Font('freesansbold.ttf', 15)
playAgainSurf = gameOverFont.render('Play again? (yes or no)', True, WHITE)
DISPLAYSURF.blit(playAgainSurf, (int((WINDOW_WIDTH - playAgainSurf.get_width()) / 2), int((WINDOW_HEIGHT - playAgainSurf.get_height()) / 2) + 100))
pygame.display.update()
return playAgain()
def drawPressKeyMsg():
pressKeyFont = pygame.font.Font('freesansbold.ttf', 18)
pressKeySurf = pressKeyFont.render('Press a key to play again.', True, WHITE)
DISPLAYSURF.blit(pressKeySurf, (int((WINDOW_WIDTH - pressKeySurf.get_width()) / 2), int((WINDOW_HEIGHT - pressKeySurf.get_height()) / 2) + 100))
pygame.display.update()
def playAgain():
while True: # keep asking until player enters yes or no
for event in pygame.event.get():
if event.type == QUIT:
terminate()
if event.type == KEYDOWN:
if event.key == K_ESCAPE: # pressing escape quits
terminate()
if event.key == K_y or event.key == K_n:
return event.key == K_y # yes or no
def waitForPlayer():
while True:
for event in pygame.event.get():
if event.type == QUIT:
terminate()
if event.type == KEYDOWN:
if event.key == K_ESCAPE: # pressing escape quits
terminate()
return
def terminate():
pygame.quit()
sys.exit()
if __name__ == '__main__':
main()
or
Provide complete working code for a realistic looking tree in Python using the Turtle graphics library and a recursive algorithm.
- also first try.
import turtle
import random
# Function to draw trees using recursion
def draw_tree(branchLen,t):
if branchLen > 5: # Base case for recursive function
t.forward(branchLen)
t.right(20)
draw_tree(branchLen - 15,t) # Recursive call to draw right branch of tree
t.left(40)
draw_tree(branchLen - 15,t) # Recursive call to draw left branch of tree
t.right(20)
t.backward(branchLen) # Go back to the initial position after drawing each branch
else:
return
def main():
myWin = turtle.Screen()
t = turtle.Turtle()
t.left(90)
t.up()
t.backward(100)
t.down()
t.color("green")
# Draw a random tree with branch length between 50 and 100
draw_tree(random.randint(50, 100),t)
myWin.exitonclick()
main()
- Second attempt
import turtle
import random
# Set up the screen
wn = turtle.Screen()
wn.bgcolor("black")
turtle.speed(0)
turtle.setheading(90)
turtle.penup()
turtle.goto(-150, -200)
turtle.pendown()
turtle.pencolor('green')
# Recursive function to draw a tree
def draw_tree(branchLength):
angle = random.randint(15,45)
sf = random.uniform(0.6, 0.8) # scale factor for the branches
if branchLength < 3: # Base case to stop recursion
return
else:
turtle.forward(branchLength)
turtle.left(angle)
draw_tree(branchLength*sf)
turtle.right(angle*2)
draw_tree(branchLength*sf)
turtle.left(angle)
if branchLength < 15: # To make sure the leaves are small
turtle.pencolor('green')
turtle.dot()
turtle.backward(branchLength)
# Call the function
draw_tree(75)
turtle.done()
Try such quotatios on any other open source llm ..good luck to run on first try or will never be working code. Even if works is usually very basic snake game looks. Or that tree is broken badly.
This model is doing that without any problems even more complex code .... is very comparable to GPT-4 exceeding GPT-3.5 easily.
Would you like to share things that amazed you? Just for us to sneak peek a little bit:)
Well.. my first super simple test, it botched a variable name when it used it later in the code. ( only about 5 lines as i start small, then work my way up.. easily fixable, but still.... ) EDIT: python .. forgot to include that.
for me not making errors what I am surprised.
I'am using newest llamacpp and model version q4k_m ( old q4 is much worse )
My code to run llamcpp
build\bin\main.exe --model models\new3\wizardcoder-33b-v1.1.Q4_K_M.gguf --mlock --color --threads 16 --keep -1 --batch_size 512 --n_predict -1 --repeat_penalty 1.1 --ctx_size 0 --interactive --instruct --reverse-prompt "<|im_end|>" -ngl 58 --simple-io --in-prefix " ### Instruction: " --in-suffix " ### Response: " -p "Below is an instruction that describes a task. Write a response that appropriately completes the request. "
Look on that code :
Can you make A snake game in python?
- first try. Look how long code is and was generated without any errors also game looks ..decent.
- Even has working score system.
import turtle
import time
import random
delay = 0.1
score = 0
high_score = 0
# Set up the screen
wn = turtle.Screen()
wn.title("Snake Game")
wn.bgcolor("black")
wn.setup(width=600, height=600)
wn.tracer(0) # Turns off the screen updates
# Snake head
head = turtle.Turtle()
head.speed(0)
head.shape("square")
head.color("white")
head.penup()
head.goto(0, 0)
head.direction = "stop"
# Snake food
food = turtle.Turtle()
food.speed(0)
food.shape("circle")
food.color("red")
food.penup()
food.goto(random.randint(-290, 290), random.randint(-290, 290))
segments = []
# Pen
pen = turtle.Turtle()
pen.speed(0)
pen.shape("square")
pen.color("white")
pen.penup()
pen.hideturtle()
pen.goto(0, 260)
pen.write("Score: 0 High Score: 0", align="center", font=("Courier", 24, "normal"))
# Functions
def go_up():
if head.direction != "down":
head.direction = "up"
def go_down():
if head.direction != "up":
head.direction = "down"
def go_left():
if head.direction != "right":
head.direction = "left"
def go_right():
if head.direction != "left":
head.direction = "right"
def move():
if head.direction == "up":
y = head.ycor()
head.sety(y + 20)
if head.direction == "down":
y = head.ycor()
head.sety(y - 20)
if head.direction == "left":
x = head.xcor()
head.setx(x - 20)
if head.direction == "right":
x = head.xcor()
head.setx(x + 20)
# Keyboard bindings
wn.listen()
wn.onkeypress(go_up, "Up")
wn.onkeypress(go_down, "Down")
wn.onkeypress(go_left, "Left")
wn.onkeypress(go_right, "Right")
# Main game loop
while True:
wn.update()
# Check for a collision with the border
if head.xcor()>290 or head.xcor()<-290 or head.ycor()>290 or head.ycor()<-290:
time.sleep(1)
head.goto(0,0)
head.direction = "stop"
# Hide the segments
for segment in segments:
segment.goto(1000, 1000)
# Clear the segments list
segments.clear()
score = 0
# Check for a collision with the food
if head.distance(food) < 20:
x = random.randint(-290, 290)
y = random.randint(-290, 290)
food.goto(x,y)
# Add a segment
new_segment = turtle.Turtle()
new_segment.speed(0)
new_segment.shape("square")
new_segment.color("grey")
new_segment.penup()
segments.append(new_segment)
# Shorten the delay
delay -= 0.001
score += 10
if score > high_score:
high_score = score
pen.clear()
pen.write("Score: {} High Score: {}".format(score, high_score), align="center", font=("Courier", 24, "normal"))
# Move the end segments first in reverse order
for index in range(len(segments)-1, 0, -1):
x = segments[index-1].xcor()
y = segments[index-1].ycor()
segments[index].goto(x, y)
# Move segment 0 to where the head is
if len(segments) > 0:
x = head.xcor()
y = head.ycor()
segments[0].goto(x, y)
move()
time.sleep(delay)
wn.mainloop()
or
Provide complete working code for a realistic looking tree in Python using the Turtle graphics library and a recursive algorithm.
- also first try.
import turtle
import random
# Function to draw trees using recursion
def draw_tree(branchLen,t):
if branchLen > 5: # Base case for recursive function
t.forward(branchLen)
t.right(20)
draw_tree(branchLen - 15,t) # Recursive call to draw right branch of tree
t.left(40)
draw_tree(branchLen - 15,t) # Recursive call to draw left branch of tree
t.right(20)
t.backward(branchLen) # Go back to the initial position after drawing each branch
else:
return
def main():
myWin = turtle.Screen()
t = turtle.Turtle()
t.left(90)
t.up()
t.backward(100)
t.down()
t.color("green")
# Draw a random tree with branch length between 50 and 100
draw_tree(random.randint(50, 100),t)
myWin.exitonclick()
main()
Try such quotatios on any other open source llm ..good luck to run on first try or will never be working code. Even if works is usually very basic snake game looks. Or that tree is broken badly.
This model is doing that without any problems even more complex code .... is very comparable to GPT-4 exceeding GPT-3.5 easily.
@mirek190 yeah it probably is pretty good at coding since it uses deepseek coder as base model which alone is comparable or even better then phind. Then extra finetuning probably made it even better. I would say those are simple questions and most models are probably trained on them but it probably is still one of the best if not the best oss coder model.
@mirek190 yeah it probably is pretty good at coding since it uses deepseek coder as base model which alone is comparable or even better then phind. Then extra finetuning probably made it even better. I would say those are simple questions and most models are probably trained on them but it probably is still one of the best if not the best oss coder model.
I tested and used phind model.
The new wizard coder 33b v1.1 is far more ahead ...really close to gpt-4 or in some aspect the same level .
I asked it to compare two variations of code and point out the differences and it outputs total nonsense about the code but not about the differences, worse than GPT-3, no matter how much I change the prompt, generation parameters and with max_tokens of 1k. I am testing on Q4K_S. Maybe it's good for text to code task, but not for comparison.
Just tried wizardcoder-python-13b-v1.0.Q4_K. Asked a few questions first to 33B and then to 13B-python, questions like how to convert a specific decorator syntax into a regular function call, how to setup redis in persistent mode, a single-line shell snippet to output env.variables using python. 33B failed almost all of these questions, even with hints. 13B answered correctly within 3 attempts.
Strange for me works as harm 13b version is not even close to this one.
Try also codebooga-34b-v0.1
Try also codebooga-34b-v0.1
Well, i didnt know he was doing his own models now. I will also have to try that.
And for what its worth mixtral-8x7b-instruct-v0.1 has been doing great for me in early testing. Much better than others so far ( python code )
Mixtral-8x7b-instruct-v0.1 is great without the doubt but specialized WizardCoder-33B-V1.1 is better for coding from my tests.
Maybe mixtral-8x7b-instruct-v0.1 works for you better because you are using intensive reasoning from llm for coding and mixtral-8x7b-instruct-v0.1 is the best right now in it and common sense.
also humaneval (coding):
wizardCoder-33B-V1.1 - 79.9
wizardcoder-python-13b-v1.0 - 62.19
mixtral-8x7b-instruct-v0.1 - 40.2
codebooga-34b-v0.1 - 71.3
That is the most capable opensource LLM for coding I ever seen... It has coding level of GPT-4 ... is incredible.
I easily made advanced applications by this llm .... never done that before so easily.
For instance Phind-CodeLlama-34B-v2 is far less capable comparing to WizardCoder-33B-V1.1. That is big step forward now.Added from my posts:
for me not making errors what I am surprised.
I'am using newest llamacpp and model version q4k_m ( old q4 is much worse )My code to run llamcpp
build\bin\main.exe --model models\new3\wizardcoder-33b-v1.1.Q4_K_M.gguf --mlock --color --threads 16 --keep -1 --batch_size 512 --n_predict -1 --repeat_penalty 1.1 --ctx_size 0 --interactive --instruct --reverse-prompt "<|im_end|>" -ngl 58 --simple-io --in-prefix " ### Instruction: " --in-suffix " ### Response: " -p "Below is an instruction that describes a task. Write a response that appropriately completes the request. "
.....
This is a prime example of how to NOT judge a model.
Your snake example is a almost 1:1 copy of https://github.com/RichardHogg/Snake-game-for-beginners/blob/master/snake_game.py
So it was able to replicate a public available 6 year old game line by line, which is copyrighted without opensource license and it came without any attributions.
I find that super concerning.
So it was above to replicate a public available 6 year old game line by line, which is copyrighted without opensource license and it came without any attributions.
I find that super concerning.Why? its knowledge out on the interwebs. its fair game. Oppressive IP be damned.
1)first - it is illegal content to be used -it's copyrighted and you have no clue that you just copied someones work but you are liable for the damages if you use it
2)second - it means the model is not generalizing (doing your work) it's replicating fine tuning data. It is a strong signal of overfitting it with too much repetitions of training on too few parameters
You'd be way better off searching for "snake game in python" on Google, using the first source you like. That's not what AI is made for.
Item 1: if you cant tell, i dont give a flying F about IP. Personally, I think it hinders human progress and is immoral and oppressive to suppress knowledge.
Item 2: Agreed, its not a good test, or use case, for generative AI
You can violate copyright (and lose your entire future by getting sued in) but any normal user of AI, especially coding models, will care. As in not being sued for his entire future income.
So point 1 is even more important than point 2.
Point 2 means the model training is corrupt, Point 1 means people risk their future without knowing they risk it.