AbstractPhila PRO

I've taken the benchmarks of the model from 50% to 86-93% spearman utilizing a quaternion-oriented attention head.

This is getting dangerously close to 99.9% mutation detection accuracy, with a model deemed 50% accurate - all by extracting geometric features from the constellation and training the ensemble head with the correct rules.

These are spearman result logits. These are in fact detecting the results.

This is the power of what I'm doing. From 50% to 90% in 48 hours with a single GPU.

Training your own alignment only requires a piece of the dataset you wish to run and about 8 hours or so. Run it, fall asleep, check on it in the morning. It'll be ready. Extract features, train your head in minutes. The spearman will be nearly perfect.

I'm currently preparing what I consider to be the final head that will need to be created. The quaternion head, which will be specifically predictive based on an ensemble of four divergent-methodology heads, each specifically tasked to solve the SVD in conjunction with the features. This system should extract any little bit of differentiation that exists. The imaginary head is the most crucial. Explaining this requires an entire paper of it's own.

I call this imaginary head the "Cletus" head, as it's inherently lesser accuracy in relation to the others. However, without it the combination does not coalesce correctly. Without the Cletus, the model does not reach full cohesion. This head is the most crucial, because it has the hardest job. It's actually the one who returned from the battlefield with the blueprint to describe everything it saw.

I expect the sheer geometric alignment alone to yield a new form of Adam tuning specific to introspective analytical alignment and with that a new format of optimizer dedicated to geometric preservation in conjunction with informational data accumulation. I also expect a new methodology for larger-buffer data movement kernel-wise, a structural boundary for SVD limitations within full spectrum, a substructure measured collapse state of SVD when projected, and multiple other models that will have hiccups and growing pains.

These tools are all building to the end-state format, which will express everything simultaneously in order to combine the necessary data from many many forms of models together, without requiring direct tooling to each model simultaneously.

Such finalized tools will include a reusable pretrained geometric patchwork that exhibits all the necessary traits of a geometric structure in it's frozen state, capable of being finetuned quickly into any other state, or simply utilized as a lookup beacon with the correct geometric transformer alignment.

The geometric transformer, which is specifically a revamped format for the transformer intentionally designed with the structural preservation of the overarching structure in mind, rather than falling directly to the naturalistic entropy of immediate solution over larger-scale contribution. This system will not replace rope, it will contribute to the concept of long-concept preservation and work hand-in-hand with systems like rope, attention, and original transformers simultaneously. ROPE based models will benefit most from this structure, as they are already trained intrinsically with alignment and rotation at their cores.

The geometric transformer by design takes nth inputs in as variant states, and those are transformed internally. Utilizing this by it's default state will yield by design, but it will require tuning and curation for specific use cases no matter which case. This is conceptually familiar to those who use transformers, and simultaneously intimidating to those who understand what I'm describing I'd think. I myself am a little intimidated that I'm this close as-is.

There are multiple other prototypes at work all leading to the geometric transformer, which will be both an empirically superior utility to any of the utilities I currently use, and embody the very essence of the geometric structure that I'm currently working with as a full trainable data mutation operation - meant to directly attenuate the structure of the observation, to the expectation of the autograd and gradients.

Getting pretty close to a few pieces, but not there yet.

I've spent the better part of the day refactoring the geolip-core github code so it would be better inline with the actual findings, and I'm currently having claude build the models using the geofractal router system.

With that I'll enable a dummy-clause structure, so the component code files can be snipped out and work standalone as necessary. Due to the geofractal router's rigidity as a structure, geovocab's problematic multi-layer formatting for formulas that often have strange hardware quirks, and a few of the more reusable systems requiring modularity - I'll just build it like this to allow for just... USE MINE INSTEAD mentality.

You'll be able to just snip them out and use them, like many of the representations within the "models" experiment folders. They are simply standalone that may or may not snap onto pieces of the larger wholes.

geolip-core will be built specifically with the geofractal router structure in mind, inheriting it's strengths, weaknesses, and hardware control - while simultaneously having a wrapper that simply says: use standard pytorch instead.

Using standard pytorch will disable much of the functionality, but the components in their standalone forms WILL WORK. The pipelines are another story.

Keep my attribution and naming in the comments please, this is a testament to a very long series of research that resulted in solutions to problems rather than trying to introduce more problems. Attribution is all I wish, and you can make your fortune from that. I believe many of the great minds of the past would agree; Nikola Tesla I believe would agree, accreditation is all I want, but not for my name - for them and my humble contribution. The greats who put the pieces together and solved the biggest problems.

As we grow, the shadows of giants are cast upon the surfaces of life and stone. Work hard, progress steadily, expand your mind, build your skills, and by the time you have any time to look down... you will be casting a shadow of your own. As we grow old and our shadows grow, others are born and see the cast shadows. Encouraging them through the same process is all I know how to do.

Wrote a triton kernel to approximate SVD at around 15000x on blackwell architecture, while the standard torch.linalg.svd basically sits in a swamp of slow.

It only tackles small kernel sizes for now, 3x3, which is the current experiment's encoder paradigm. Standard SVD causes death by a thousand cuts when using small matrixes. Smaller matrixes provide a much more robust access to certain elemental linkages on many spectrum.

The formula isn't perfect. It's absolutely lightning quick though. The svd_trison.py file has a profiler.

https://huggingface.co/AbstractPhil/geolip-hypersphere-experiments/blob/main/spectral/notebooks/experiment_2_manifold_structures.ipynb

https://huggingface.co/AbstractPhil/geolip-core/blob/main/svd_triton.py

Incoming the geofractal router structure.

It will provide the necessary hardware and software implications to create much larger structures and curate the code in a much more effective way to hardware control than simply leaving it up to random chance or ai.

As I expand the system I will be heavily testing to include systems like Ulysses, accelerate, and more to encompass a larger array of learning. This will be crucial to building a proper bert trainer as well.