23 54

TTK Ciar

ttkciar

http://ciar.org/ttk/

ttkciar

AI & ML interests

distributed systems, materials engineering, narrow AI, AGI

Recent Activity

liked a model 1 day ago

FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview

liked a model 1 day ago

bartowski/Gigaberg-Mistral-Large-123B-GGUF

liked a model 1 day ago

ehristoforu/phi-4-45b

View all activity

Organizations

None yet

ttkciar's activity

liked 4 models 1 day ago

New activity in ND911/stable-diffusion-3.5-large-GGUF 14 days ago

GGUF files lack sd version metadata, required for stable-diffusion.cpp

#2 opened 14 days ago by

ttkciar

updated a model 6 months ago

ttkciar/Starling-LM-11B-alpha-Q4_K_M-GGUF

Updated Aug 2, 2024 • 41

New activity in qnguyen3/14b-gguf-arxiv 7 months ago

What is this model for?

#1 opened 7 months ago by

ttkciar

liked a model 7 months ago

gghfez/WizardLM-2-8x22B-Beige

Text Generation • Updated Jun 15, 2024 • 10 • 3

reacted to yushun0410's post with ❤️ 7 months ago

Post

4625

Hi Huggingfacers!

Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.

The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.

Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!

Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793

Code: https://github.com/zyushun/Adam-mini