Mayank Mishra's picture

1644263.7 TFLOPS

Mayank Mishra

mayank-mishra

·

https://mayank31398.github.io/

AI & ML interests

Large Language Models, Distributed Training and Inference

Recent Activity

upvoted a paper 20 days ago

upvoted a collection 21 days ago

New activity 29 days ago

ibm-granite/granite-3.0-2b-instruct

Articles

Improving Hugging Face Training Efficiency Through Packing with Flash Attention

Saving Memory Using Padding-Free Transformer Layers during Finetuning

Aurora-M: The First Open Source Biden-Harris Executive Order Red teamed Multilingual Language Model

Organizations

mayank-mishra's activity

upvoted a paper 20 days ago

SelfCodeAlign: Self-Alignment for Code Generation

Paper • 2410.24198 • Published 21 days ago • 20

upvoted a collection 21 days ago

SmolLM2

State-of-the-art compact LLMs for on-device applications: 1.7B, 360M, 135M • 10 items • Updated about 2 hours ago • 172

upvoted a collection about 1 month ago

Granite 3.0 Language Models

A series of language models trained by IBM licensed under Apache 2.0 license. We release both the base pretrained and instruct models. • 8 items • Updated 17 days ago • 89

upvoted 2 papers 3 months ago

The Mamba in the Llama: Distilling and Accelerating Hybrid Models

Paper • 2408.15237 • Published Aug 27 • 37

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published Aug 23 • 22

upvoted an article 3 months ago

Article

Improving Hugging Face Training Efficiency Through Packing with Flash Attention

Aug 21

• 22

upvoted a collection 3 months ago

Power-LM

Dense & MoE LLMs trained with power learning rate scheduler. • 4 items • Updated Oct 17 • 15

upvoted a paper 3 months ago

Transformer Explainer: Interactive Learning of Text-Generative Models

Paper • 2408.04619 • Published Aug 8 • 155

upvoted 4 papers 4 months ago

OpenDevin: An Open Platform for AI Software Developers as Generalist Agents

Paper • 2407.16741 • Published Jul 23 • 68

Enhancing Training Efficiency Using Packing with Flash Attention

Paper • 2407.09105 • Published Jul 12 • 14

Scaling Granite Code Models to 128K Context

Paper • 2407.13739 • Published Jul 18 • 19

The infrastructure powering IBM's Gen AI model development

Paper • 2407.05467 • Published Jul 7 • 2

upvoted a paper 5 months ago

MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention

Paper • 2407.02490 • Published Jul 2 • 23

upvoted a collection 5 months ago

Experimental Checkpoints

As name suggests • 1 item • Updated Jun 30 • 1

upvoted an article 5 months ago

Article

Aligning Large Language Models with BRAIn

By

•

Jun 11

• 8

upvoted a collection 6 months ago

Dolomite Engine Sample

This collections contains a sample dataset and model trained via dolomite-engine. Repo: https://github.com/ibm-granite/dolomite-engine/ • 2 items • Updated Jun 30 • 1

upvoted a paper 6 months ago

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

Paper • 2405.12981 • Published May 21 • 28

upvoted 2 papers 7 months ago

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization

Paper • 2404.03605 • Published Apr 4 • 1

Granite Code Models: A Family of Open Foundation Models for Code Intelligence

Paper • 2405.04324 • Published May 7 • 22

upvoted an article 7 months ago

Article

Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent

Apr 22

• 78