Thank you ; and document to assist for all model usage.
Hey;
Thank you for quanting my models... all of them ... big list!
I have created the following document (30+ pages now) which covers a lot of critical information for choosing quants, then setting up the model for use as well as other information. This may be of use / help.
Contents:
QUANTS:
- QUANTS Detailed information.
- IMATRIX Quants
- QUANTS GENERATIONAL DIFFERENCES:
- ADDITIONAL QUANT INFORMATION
- ARM QUANTS / Q4_0_X_X
- NEO Imatrix Quants / Neo Imatrix X Quants
- CPU ONLY CONSIDERATIONS
Class 1, 2, 3 and 4 model critical notes
SOURCE FILES for my Models / APPS to Run LLMs / AIs:
- TEXT-GENERATION-WEBUI
- KOBOLDCPP
- SILLYTAVERN
- Lmstudio, Ollama, Llamacpp, and OTHER PROGRAMS
TESTING / Default / Generation Example PARAMETERS AND SAMPLERS
- Basic settings suggested for general model operation.
Generational Control And Steering of a Model / Fixing Model Issues on the Fly
- Multiple Methods to Steer Generation on the fly
- On the fly Class 3/4 Steering / Generational Issues and Fixes (also for any model/type)
- Advanced Steering / Fixing Issues (any model, any type) and "sequenced" parameter/sampler change(s)
- "Cold" Editing/Generation
Quick Reference Table / Parameters, Samplers, Advanced Samplers
- Quick setup for all model classes for automated control / smooth operation.
- Section 1a : PRIMARY PARAMETERS - ALL APPS
- Section 1b : PENALITY SAMPLERS - ALL APPS
- Section 1c : SECONDARY SAMPLERS / FILTERS - ALL APPS
- Section 2: ADVANCED SAMPLERS
DETAILED NOTES ON PARAMETERS, SAMPLERS and ADVANCED SAMPLERS:
- DETAILS on PARAMETERS / SAMPLERS
- General Parameters
- The Local LLM Settings Guide/Rant
- LLAMACPP-SERVER EXE - usage / parameters / samplers
- DRY Sampler
- Samplers
- Creative Writing
- Benchmarking-and-Guiding-Adaptive-Sampling-Decoding
ADVANCED: HOW TO TEST EACH PARAMETER(s), SAMPLER(s) and ADVANCED SAMPLER(s)
Link: