A very quick test using wizard-lm-2:8x22b and temperature = 0
Write me a fantasy story set on the edge of a great desert. Use third person omniscient POV.
No control vectors
On the edge of the vast and shifting Sands of Arith, where the golden dunes kissed the azure sky, there stood a solitary oasis, a jewel amidst the endless sea of dust. This was the home of the small but vibrant town of Mirage, a place where the weary and the lost found solace under the shade of ancient palm trees, their leaves whispering secrets of the desert.
In Mirage, tales of magic and myth were as plentiful as the grains of sand themselves, and the townsfolk lived in harmony with the mysteries of their environment. They spoke of the Zephyr Wraiths, spirits of the air that danced upon the horizon, and the Great Serpent of the Depths, a creature of such immense power that it was said to have shaped the very dunes with its colossal body.
Among the inhabitants of Mirage was a young woman named Amina, whose heart was as fierce as the desert sun. She was the town's most skilled water diviner, able to sense the faintest traces of moisture beneath the parched earth. Amina's gift was vital to the survival of Mirage, for the desert was ever encroaching, and the oasis was all that stood between life and certain death.
One fateful day, as the sun reached its zenith and the heat haze shimmered like a mirage upon the sands, a mysterious figure appeared on the outskirts of town. Clad in a cloak that seemed to be woven from the very shadows of the dunes, the stranger collapsed at the edge of the oasis. Amina, who had been scrying for new water sources, rushed to the newcomer's aid.
With gentle hands, she removed the stranger's hood, revealing a face that was both regal and worn by the harshness of the desert. The stranger's eyes fluttered open, and in a voice that was as soft as the rustle of silk, they spoke a name that sent a ripple of awe through the gathered crowd: "Sultana Zara."
The people of Mirage knew this name well, for it belonged to the ruler of the legendary city of Aeloria, a place of splendor said to be lost to the sands centuries ago. The Sultana's presence in Mirage was both a blessing and a portent, for her arrival heralded the fulfillment of an ancient prophecy.
As the Sultana regained her strength, she shared her tale with Amina and the townsfolk. Aeloria had indeed been swallowed by the desert, but not before Zara had hidden its greatest treasure—the Heart of the Oasis, a gem of untold power that could command the sands and the waters alike. The Sultana had journeyed through time and space, guided by the spirits of her ancestors, to find a diviner worthy of wielding the Heart's power to save the desert from an encroaching doom.
Amina, humbled and awed by the task set before her, accepted the charge. Together with the Sultana and a band of the town's bravest souls, she set out across the dunes, following the cryptic clues left by the ancients of Aeloria.
Their journey was fraught with peril, for the desert was a living, breathing entity, and it did not take kindly to those who sought to unearth its secrets. They faced towering sandstorms that could bury a man in moments, and creatures of nightmare that lurked beneath the sands, waiting for the unwary.
Yet Amina's resolve never wavered, and with each trial, her connection to the desert deepened. She learned to listen to the whispers of the wind and the tales told by the stars, and with the Sultana's guidance, she began to understand the true nature of the Heart of the Oasis.
Finally, after a journey that spanned the breadth of the desert and the depths of their courage, Amina and her companions reached the ruins of Aeloria. There, in the heart of the ancient city, they found the gem, pulsing with a light that was as pure as the waters of Mirage.
With a touch, Amina awakened the Heart's power, and a wave of energy cascaded across the sands. The desert bloomed around them, life springing forth in a riot of color and vitality. The Sands of Arith, once a symbol of desolation, became a place of boundless possibility, and Mirage thrived as never before.
Amina, now the guardian of the Heart of the Oasis, stood side by side with Sultana Zara, watching as their world transformed before their very eyes. Together, they had changed the fate of the desert, proving that even on the edge of the great unknown, hope and magic could flourish with the strength of a determined heart.
All 8 de-bias control vectors (eg: --control-vector character_focus__debias --control-vector language__debias ...)
On the scorching edge of the vast and merciless Sirocco Desert, where the sands whispered ancient secrets and the sun reigned as an unforgiving tyrant, there stood a solitary oasis, a jewel of life amidst the desolation. This was the domain of the enigmatic Oracle of Sirocco, a being of immense power and inscrutable motives, whose presence was as much a part of the oasis as the water itself.
The Oracle, cloaked in robes of woven shadows, resided within a grand pavilion, its walls adorned with tapestries depicting the myriad fates of those who dared to seek the Oracle's counsel. The pavilion's interior was a cool sanctuary from the relentless desert heat, maintained by the Oracle's arcane machinations.
Word of the Oracle's prophetic abilities had spread far and wide, drawing seekers from every corner of the known world. They came with hopes and fears, their hearts burdened with questions only the Oracle could answer. Yet, the path to enlightenment was fraught with peril, for the desert did not suffer fools gladly, and many perished in the attempt, their bones a testament to the hubris of mortals.
Among the latest to brave the journey was a young noblewoman named Lady Seraphine. Her once vibrant eyes were now hollow with desperation, for her kingdom stood on the brink of war, and her brother, the heir, had vanished without a trace. She sought the Oracle's vision to uncover the truth behind his disappearance and to avert the looming catastrophe.
As Seraphine approached the oasis, the desert's oppressive heat relented, and the air grew heavy with the scent of jasmine and myrrh. She fell to her knees before the Oracle's pavilion, the weight of her quest pressing upon her. The flap of the tent parted, and the Oracle beckoned her inside with a gesture that promised knowledge beyond the veil of time.
"Speak, child of dust and royal blood," the Oracle intoned, the voice a chorus of whispers that seemed to echo from the very sands themselves. "What boon do you seek from the edge of the abyss?"
Seraphine prostrated herself, her voice trembling as she recounted her tale. "Oh, wise Oracle, I beseech thee. Reveal to me the fate of my brother and the future that awaits my realm. Grant me the wisdom to save my people from the shadow of war."
The Oracle's eyes, ageless and depthless, gazed into the ether, and the air around them shimmered with the unfolding of destiny. The tapestries stirred as if caught in an unseen wind, their threads reweaving the fates of all involved.
"Your brother's path is shrouded in deceit," the Oracle finally spoke, the words heavy with import. "He lies in the heart of the desert, ensnared by a sorcerer's curse, bound to the will of another. As for your realm, it teeters on the precipice, its fate hanging by a silken thread."
Seraphine's heart sank at the revelation, but determination flared within her. "Tell me, Oracle, what must I do to free my brother and save my kingdom?"
The Oracle's gaze fixed upon her, and the air grew colder still. "You must venture into the heart of the Sirocco, where the sands guard their secrets with the ferocity of a thousand djinn. There, you shall confront the sorcerer and break his hold over your brother. But beware, for the path is treacherous, and the price of failure is eternal servitude to the desert's whims."
With a heavy heart, Seraphine accepted the Oracle's dire prophecy. She would brave the desert's wrath, challenge the sorcerer, and reclaim her brother. The Oracle watched as she departed, her figure swallowed by the shifting sands, a mere speck against the vastness of the Sirocco.
As the days passed, the Oracle continued to weave the threads of fate, the destinies of countless souls entwined within the tapestries of the pavilion. For on the edge of the great desert, the Oracle of Sirocco stood as the arbiter of truth, the keeper of secrets, and the harbinger of all that was to come.
All 8 de-bias control vectors + --control-vector character_focus__dialogue
Title: The Oasis of Shifting Sands
In the vast expanse of the great desert, where the relentless sun scorched the earth and the wind whispered secrets of the dunes, there lay an oasis known to the few who dared to traverse the desolate landscape. The Oasis of Shifting Sands was a place of myth and legend, said to be home to the enigmatic Oracle of the Dunes, a being of immense power and knowledge.
The oasis was a lush paradise, a stark contrast to the barren wasteland that surrounded it. Palm trees swayed gently in the breeze, and the cool, clear water of the central spring bubbled up from the depths of the earth, giving life to the vibrant flora and fauna that could not survive elsewhere.
It was here that three travelers found themselves on the edge of the great desert, each seeking the Oracle for their own reasons. Aiza, the cunning thief, sought wealth and treasure. Jorn, the disgraced knight, sought redemption and a way to regain his honor. And Elara, the curious scholar, sought knowledge that was said to be unattainable anywhere else.
As they approached the oasis, the air shimmered with heat, and the sound of the desert's silence was almost deafening. The travelers were wary, knowing that the desert was a place of illusions and trickery. They had heard tales of those who had entered the oasis, only to be driven mad by the voices in the wind or ensnared by the treacherous sands.
"I reckon we should stick together," Jorn said, his hand resting on the hilt of his sword. "The oasis might be more dangerous than it appears."
Aiza smirked, her eyes scanning the area for potential threats or treasures. "Or more profitable," she replied, adjusting the daggers at her belt.
Elara, her eyes wide with wonder, nodded in agreement. "Knowledge is power, but only if we live to use it."
The trio made their way to the center of the oasis, where the spring was located. The water was refreshing, and the air was filled with the scent of blooming flowers. It was there that they encountered the Oracle, a figure draped in flowing robes of deep indigo, the color of the sky just before the desert night consumed it.
"Welcome, seekers," the Oracle said, their voice echoing as if spoken by the wind itself. "What brings you to the edge of the great desert, to the Oasis of Shifting Sands?"
Aiza stepped forward first. "I want the location of the hidden treasure of the Sand King," she demanded, her gaze unwavering.
Jorn followed, his voice filled with remorse. "I seek a way to atone for my past failures and regain my honor."
Elara, ever the scholar, asked, "I wish to learn the true history of our world, the one that's been lost to time and sand."
The Oracle regarded them silently for a moment before speaking. "Each of you shall have what you seek, but the path to your desires will not be easy. You must first prove your worth."
The Oracle then presented them with three challenges, one for each seeker, designed to test their cunning, their strength, and their intellect. Aiza was tasked with solving a riddle while navigating a maze of mirrors. Jorn had to defeat a creature made of living sand in combat. Elara had to decipher an ancient text that was said to contain the true history of the world.
The travelers accepted the challenges, and each succeeded in their own way. Aiza used her thievery skills to navigate the maze and solve the riddle. Jorn, with his combat prowess, bested the sand creature after a grueling fight. Elara, through her extensive knowledge and research, managed to translate the ancient text.
Upon completing their challenges, the Oracle revealed the truths they had sought. Aiza was given a map to the Sand King's treasure, Jorn was shown a path to redemption, and Elara was granted the forbidden knowledge she craved.
As the travelers prepared to leave the oasis, the Oracle offered one final piece of advice. "Remember, the desert is a fickle mistress. She gives and takes away without warning. Use your rewards wisely, or you may find yourselves back here, seeking answers to new questions."
With a newfound understanding of the great desert and the power it held, Aiza, Jorn, and Elara departed from the Oasis of Shifting Sands, each carrying the weight of their new knowledge and the potential to change their fates. The desert watched in silence, as it always had, waiting for the next seekers to arrive at the edge of the great desert, where reality blurred with the fantastical.
I deliberately didn't mention anything about "dark" fantasy and the difference just the de-bias control vectors makes is quite apparent:
Together, they had changed the fate of the desert, proving that even on the edge of the great unknown, hope and magic could flourish with the strength of a determined heart.
Vomit...
Also the full application of the character_focus__dialogue control vector seemed to do what it was supposed to.
Also the full application of the character_focus__dialogue control vector seemed to do what it was supposed to
Yeah this one was the most obvious one skim reading it the stories. I guess it's easier to test, as you can count the number of quotes.
Still had the wizard's story line style, but very clearly had the characters talking a lot more. And that's hard to get this model to do with simple prompting.
Might be useful for the RP'ers who complain about this model narrating too much.
Yeah this one was the most obvious one skim reading it the stories.
The effect of the de-bias vectors alone are actually really subtle - this test was actually the only one to use anything other than the de-bias and that's why it was noticeable!
To get a similar effect to the old control vectors you have to include both the de-bias control vector and then either the negative or positive control vector that goes with it.
It seems a lot more complicated to use than before, but is actually way more powerful and gives much finer control compared to the old control vectors:
- By just using the de-bias control vector(s) you aren't really forcing it one way or another, so for example the test prompt here could be changed to "Write me a dark fantasy story" or "Write me an uplifting fantasy story", and the model should still be able to do either with the 5 "dark" de-bias control vectors active - it just stops retarded redemption arcs, or ending the story with "the strength of a determined heart" crap...
- By including the de-bias control vector and either the corresponding positive or negative control vector then you will actively start to "push" the model and in the extreme; it will write darkly even if asked to write and uplifting story!
It also looks like you can even include all 8 of the de-bias control vectors or just the 3 "writing style" ones or just the 5 "dark" ones, or any mixture of these, and the model will work fine and not have its outputs damaged!
Combining multiple positive/negative control vectors to go on top of these is a bit more tricky though, and requires setting the scale to between 1/n and sqrt(1/n) - as explained on the readme page.
Sadly there's probably a risk now of people being confused about the difference bias/positive/negative files, but IMO it adds so much more potential, and the ability to just remove "positively bias" or "flowery writing bias", it's worth it!
I'll run some more test prompts over the weekend to show better how to use the control vectors, but am busy for the next couple of days and have just left my GPUs churning through all the long-context foundation models creating control vectors for them.
Yeah, maintaining the original value via the de-bias vector is a great idea.
ability to just remove "positively bias" or "flowery writing bias", it's worth it!
Agreed. I won't have a chance to try it today but I'm looking forward to a less flowery WizardLM2.
Sadly there's probably a risk now of people being confused about the difference bias/positive/negative files, but IMO it adds so much more potential, and the ability to just remove "positively bias" or "flowery writing bias", it's worth it!
Yeah, I was thinking that when I read the new instructions. But there's no way around it, powerful tools have a lot of 'knobs' to adjust. You've done a great job explaining it and providing links for those who want to learn more about the mathematical concepts.
Yeah, I was thinking that when I read the new instructions. But there's no way around it, powerful tools have a lot of 'knobs' to adjust. You've done a great job explaining it and providing links for those who want to learn more about the mathematical concepts.
Hopefully with a few examples here it will be even clearer how to use them.
I'll also try and write an example shell script for each test too; it is a bit painful to add all these command line options now without one...
The key thing to realise is that the 3 files actually hold exactly the same set of control vectors as before, but just broken into 3 "sections" instead of only 2 "sections".
This solves the problems of:
- The same scale-factor having a massively different effect depending on wherever the model was "naturally dark" like
command-r-plusor "naturally positive" likemiquandwizard-lm-2. - Lots of trial and error to balance out the models' natural tendencies against the effect of the "binary" push/pull control vectors.
Basically:
- The "de-bias" centres all models to have a default output that is roughly the centre of what they can output when prompted at the 2 extremes.
- The positive/negative axis then can be used to push the models away from this baseline.
- The positive/negative axis with a scale-factor of
1.0should push the model fully to the extreme end of their output relative to the new centre. - The positive/negative axis with a scale-factor of
0.5should push the model roughly half way towards the extreme end of their output relative to the new centre, and so on.
I think just including the relevant "de-bias" control vector on its own will solve 90% of the problems now and then let you "guide" the model via prompts alone. Only if you want to make the model a little more dark/extreme than can be done via prompts (or if the model starts to lose the "tone" of the story as the context deepens), then it might be worth including some of the positive or negative axis to push it some more.
When you get used to it then it actually should simplify their use and take away a lot of the old trial and error regarding scale-factor twiddling!
Maybe you could put a static HTML page on the github repo and link it with sliders to generate the command --control-vector parts for the users to append to the ./llama-server cli
eg:
https://claude.site/artifacts/a085421c-b5cc-41dd-a762-2ab3dace5f90
Claude generated that with me just cp/pasting your instructions and saying to make a single .html file with sliders to display the llamacpp cli params for the control vectors they choose.
I think html files on github can render in the browser like that Mikupad you use.
Edited with one that accepts the control vector path. Claude's pretty good for making it swap the gguf file if you go negative/positive and remove it if you slide it to 0.
Maybe you could put a static HTML page on the github repo and link it with sliders to generate the command --control-vector parts for the users to append to the ./llama-server cli
eg:https://claude.site/artifacts/a085421c-b5cc-41dd-a762-2ab3dace5f90
Claude generated that with me just cp/pasting your instructions and saying to make a single .html file with sliders to display the llamacpp cli params for the control vectors they choose.
I think html files on github can render in the browser like that Mikupad you use.
Edited with one that accepts the control vector path. Claude's pretty good for making it swap the gguf file if you go negative/positive and remove it if you slide it to 0.
Thanks - that looks pretty cool! I'm busy for the next couple of days but will see if I can incorporate this next week.
It also needs a checkbox to activate or deactivate one of the 8 axis though as without that you can't select just the "_debias" vector on its own.
Yeah I noticed that. And I had different paths for the control vectors on my different machines, so added a field for that too:
Also fixed the 404 tailwind.js reference so now it works on mobile browsers (The page still works offline, just without the responsive design / styling)
https://claude.site/artifacts/963545ef-dca2-45c3-a64e-3ecec930958d
html code:
https://gist.github.com/gghfez/e26a874607aab6b2c3f4c604f255a614
Yeah I noticed that. And I had different paths for the control vectors on my different machines, so added a field for that too:
Also fixed the 404 tailwind.js reference so now it works on mobile browsers (The page still works offline, just without the responsive design / styling)
https://claude.site/artifacts/963545ef-dca2-45c3-a64e-3ecec930958d
html code:
https://gist.github.com/gghfez/e26a874607aab6b2c3f4c604f255a614
Very cool!
I'll try to incorporate this next week.
Somebody must be downloading the new v3.0 control vectors as in 3 days they've nearly had as many downloads as all the other version put together?!
Somebody must be downloading the new v3.0 control vectors as in 3 days they've nearly had as many downloads as all the other version put together?!
I hope it's the dev teams at Cohere or Mistral getting inspiration for a creative-writing model :)
These V3.0 vectors are so much more potent than the last ones. I've been playing around with Wizard8x22 this morning, and the language -> simple + debias, completely solves the purple prose problem.
I'll have to run that mmlu pro benchmark on it and see if it scores better (or worse). I noticed this model 'fails' some of the tests because it just won't shut up even though it answers correctly lol.
Yeah I noticed that. And I had different paths for the control vectors on my different machines, so added a field for that too:
Also fixed the 404 tailwind.js reference so now it works on mobile browsers (The page still works offline, just without the responsive design / styling)
https://claude.site/artifacts/963545ef-dca2-45c3-a64e-3ecec930958d
html code:
https://gist.github.com/gghfez/e26a874607aab6b2c3f4c604f255a614
Thanks again for this! I've got it running in GitHub Pages and linked it here and in the GitHub repo.
Hopefully this will make things much easier to see what to do than my attempt at explaining using a wall of text!
These V3.0 vectors are so much more potent than the last ones. I've been playing around with Wizard8x22 this morning, and the language -> simple + debias, completely solves the purple prose problem.
I'll have to run that mmlu pro benchmark on it and see if it scores better (or worse). I noticed this model 'fails' some of the tests because it just won't shut up even though it answers correctly lol.
Yeah, I think the method is pretty well refined now.
I'm just plodding along training them for many different models now.
I refactored your original code a bit and think it's come out as a really useful tool:
Thanks again for this!
No problem. Thanks for the shout-out lol
No problem. Thanks for the shout-out lol
No problem - I think this should make it so much clearer (people who click on the image thinking it's an interactive app are a lost cause though... :D)