LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Paper
•
2310.08659
•
Published
•
22
PEFT
-
-
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models
Paper • 2309.14717 • Published • 44 -
ModuLoRA: Finetuning 3-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers
Paper • 2309.16119 • Published • 1 -
LoRA ensembles for large language model fine-tuning
Paper • 2310.00035 • Published • 2 -
NOLA: Networks as Linear Combination of Low Rank Random Basis
Paper • 2310.02556 • Published • 2 -
LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning
Paper • 2308.03303 • Published • 3 -
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
Paper • 2309.12307 • Published • 87 -
DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning
Paper • 2309.05173 • Published • 1 -
Scaled Prompt-Tuning for Few-Shot Natural Language Generation
Paper • 2309.06759 • Published • 1 -
Adapting Language Models to Compress Contexts
Paper • 2305.14788 • Published • 1 -
In-context Autoencoder for Context Compression in a Large Language Model
Paper • 2307.06945 • Published • 27 -
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt
Paper • 2305.11186 • Published • 1 -
A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning
Paper • 2304.14856 • Published • 1 -
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval
Paper • 2204.02292 • Published • 1 -
Composable Sparse Fine-Tuning for Cross-Lingual Transfer
Paper • 2110.07560 • Published • 1 -
Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification
Paper • 2308.07282 • Published • 1 -
Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models
Paper • 2308.10462 • Published • 2 -
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
Paper • 2306.00477 • Published • 1 -
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
Paper • 2303.08566 • Published • 1 -
LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning
Paper • 2308.11148 • Published • 2 -
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
Paper • 2212.09535 • Published • 1 -
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels
Paper • 2309.08513 • Published • 1 -
Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm
Paper • 2303.07910 • Published • 1 -
Exploring the Benefits of Differentially Private Pre-training and Parameter-Efficient Fine-tuning for Table Transformers
Paper • 2309.06526 • Published • 1 -
LoRAPrune: Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
Paper • 2305.18403 • Published • 2 -
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning
Paper • 2303.15647 • Published • 4 -
Parameter-Efficient Fine-Tuning with Layer Pruning on Free-Text Sequence-to-Sequence Modeling
Paper • 2305.08285 • Published • 1 -
Multi-Head Adapter Routing for Cross-Task Generalization
Paper • 2211.03831 • Published • 2 -
Visual Prompt Tuning
Paper • 2203.12119 • Published • 1 -
PVP: Pre-trained Visual Parameter-Efficient Tuning
Paper • 2304.13639 • Published • 1 -
Do We Really Need a Large Number of Visual Prompts?
Paper • 2305.17223 • Published • 1 -
Improving Visual Prompt Tuning for Self-supervised Vision Transformers
Paper • 2306.05067 • Published • 2 -
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning
Paper • 2210.08823 • Published • 1 -
VeRA: Vector-based Random Matrix Adaptation
Paper • 2310.11454 • Published • 28 -
When can transformers reason with abstract symbols?
Paper • 2310.09753 • Published • 2 -
LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery
Paper • 2310.18356 • Published • 22 -
Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Models
Paper • 2307.08303 • Published • 1 -
Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker
Paper • 2305.13729 • Published • 1 -
Soft-prompt Tuning for Large Language Models to Evaluate Bias
Paper • 2306.04735 • Published • 1 -
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding
Paper • 2306.04933 • Published • 1 -
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization
Paper • 2303.12314 • Published • 1 -
Contrastive Learning for Prompt-Based Few-Shot Language Learners
Paper • 2205.01308 • Published • 1 -
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning
Paper • 2305.18169 • Published • 1 -
Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval
Paper • 2308.08285 • Published • 1 -
Privacy-Preserving Prompt Tuning for Large Language Model Services
Paper • 2305.06212 • Published • 1 -
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning
Paper • 2211.03044 • Published • 1 -
Contrastive Demonstration Tuning for Pre-trained Language Models
Paper • 2204.04392 • Published • 1 -
Platypus: Quick, Cheap, and Powerful Refinement of LLMs
Paper • 2308.07317 • Published • 23 -
Bactrian-X : A Multilingual Replicable Instruction-Following Model with Low-Rank Adaptation
Paper • 2305.15011 • Published • 1 -
Sparse Finetuning for Inference Acceleration of Large Language Models
Paper • 2310.06927 • Published • 14 -
TART: A plug-and-play Transformer module for task-agnostic reasoning
Paper • 2306.07536 • Published • 11 -
Arbitrary Few Parameters are Good Enough for Adapting Large-scale Pre-trained Language Models
Paper • 2306.02320 • Published • 1 -
LiST: Lite Prompted Self-training Makes Parameter-Efficient Few-shot Learners
Paper • 2110.06274 • Published • 1 -
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
Paper • 2104.07650 • Published • 2 -
Effectiveness of Data Augmentation for Parameter Efficient Tuning with Limited Data
Paper • 2303.02577 • Published • 1 -
Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis
Paper • 2306.07664 • Published • 1 -
Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner
Paper • 2305.01711 • Published • 1 -
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models
Paper • 2210.12360 • Published • 1 -
LoRA: Low-Rank Adaptation of Large Language Models
Paper • 2106.09685 • Published • 30 -
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
Paper • 2309.10400 • Published • 26 -
Efficient Streaming Language Models with Attention Sinks
Paper • 2309.17453 • Published • 13 -
QLoRA: Efficient Finetuning of Quantized LLMs
Paper • 2305.14314 • Published • 45 -
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Paper • 2205.05638 • Published • 3 -
Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
Paper • 2307.05695 • Published • 22 -
An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models
Paper • 2309.09958 • Published • 18 -
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction
Paper • 2305.18752 • Published • 3 -
XPrompt: Exploring the Extreme of Prompt Tuning
Paper • 2210.04457 • Published • 1 -
GPT Understands, Too
Paper • 2103.10385 • Published • 8 -
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
Paper • 2111.00160 • Published • 1 -
Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models
Paper • 2310.05015 • Published • 1 -
Can pruning make Large Language Models more efficient?
Paper • 2310.04573 • Published • 1 -
S-LoRA: Serving Thousands of Concurrent LoRA Adapters
Paper • 2311.03285 • Published • 28 -
MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning
Paper • 2311.02303 • Published • 4 -
Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning
Paper • 2310.20587 • Published • 16 -
SteloCoder: a Decoder-Only LLM for Multi-Language to Python Code Translation
Paper • 2310.15539 • Published • 1 -
Beyond Universal Transformer: block reusing with adaptor in Transformer for automatic speech recognition
Paper • 2303.13072 • Published • 1 -
READ: Recurrent Adaptation of Large Transformers
Paper • 2305.15348 • Published • 2 -
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model
Paper • 2211.11363 • Published • 1 -
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model
Paper • 2305.15265 • Published • 1 -
MultiWay-Adapater: Adapting large-scale multi-modal models for scalable image-text retrieval
Paper • 2309.01516 • Published • 1 -
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning
Paper • 2212.03220 • Published • 1 -
PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques
Paper • 2304.12410 • Published • 1 -
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning
Paper • 2205.11005 • Published • 1 -
LLM4TS: Two-Stage Fine-Tuning for Time-Series Forecasting with Pre-Trained LLMs
Paper • 2308.08469 • Published • 2 -
Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs
Paper • 2304.14999 • Published • 2 -
Towards a Unified View of Parameter-Efficient Transfer Learning
Paper • 2110.04366 • Published • 2 -
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models
Paper • 2307.03084 • Published • 1 -
Composing Parameter-Efficient Modules with Arithmetic Operations
Paper • 2306.14870 • Published • 3 -
Model-Agnostic Syntactical Information for Pre-Trained Programming Language Models
Paper • 2303.06233 • Published • 1 -
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization
Paper • 2303.15822 • Published • 1 -
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
Paper • 2307.13269 • Published • 31 -
A Comprehensive Analysis of Adapter Efficiency
Paper • 2305.07491 • Published • 1 -
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning
Paper • 2301.12132 • Published • 1 -
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models
Paper • 2203.01104 • Published • 2 -
Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture
Paper • 2303.16753 • Published • 1 -
LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models
Paper • 2308.10252 • Published • 1 -
ConPET: Continual Parameter-Efficient Tuning for Large Language Models
Paper • 2309.14763 • Published • 1 -
SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations
Paper • 2305.13235 • Published • 1 -
Non-Intrusive Adaptation: Input-Centric Parameter-efficient Fine-Tuning for Versatile Multimodal Modeling
Paper • 2310.12100 • Published • 1 -
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Paper • 2311.06243 • Published • 17 -
A Unified Continual Learning Framework with General Parameter-Efficient Tuning
Paper • 2303.10070 • Published • 1 -
Efficient Model Adaptation for Continual Learning at the Edge
Paper • 2308.02084 • Published • 1 -
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model
Paper • 2208.08340 • Published • 1 -
MVP: Meta Visual Prompt Tuning for Few-Shot Remote Sensing Image Scene Classification
Paper • 2309.09276 • Published • 1 -
Approximated Prompt Tuning for Vision-Language Pre-trained Models
Paper • 2306.15706 • Published • 1 -
Incremental Task Learning with Incremental Rank Updates
Paper • 2207.09074 • Published • 1 -
IF2Net: Innately Forgetting-Free Networks for Continual Learning
Paper • 2306.10480 • Published • 1 -
Continual Learning with Pretrained Backbones by Tuning in the Input Space
Paper • 2306.02947 • Published • 1 -
Continual Learning with Dependency Preserving Hypernetworks
Paper • 2209.07712 • Published • 1 -
CLR: Channel-wise Lightweight Reprogramming for Continual Learning
Paper • 2307.11386 • Published • 1 -
MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning
Paper • 2304.09402 • Published • 2 -
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning
Paper • 2301.11660 • Published • 1 -
Pruning Pre-trained Language Models Without Fine-Tuning
Paper • 2210.06210 • Published • 1 -
Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond
Paper • 2304.05216 • Published • 1 -
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Paper • 2305.17691 • Published • 1 -
Plug-and-Play Document Modules for Pre-trained Models
Paper • 2305.17660 • Published • 1 -
SiRA: Sparse Mixture of Low Rank Adaptation
Paper • 2311.09179 • Published • 8 -
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning
Paper • 2309.05444 • Published • 1 -
LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
Paper • 2311.05556 • Published • 81 -
ProSG: Using Prompt Synthetic Gradients to Alleviate Prompt Forgetting of RNN-like Language Models
Paper • 2311.01981 • Published • 1 -
Augmented Large Language Models with Parametric Knowledge Guiding
Paper • 2305.04757 • Published • 2 -
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
Paper • 2311.13171 • Published • 1 -
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression
Paper • 2309.14021 • Published • 1 -
OpenPrompt: An Open-source Framework for Prompt-learning
Paper • 2111.01998 • Published • 1 -
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Paper • 2004.12406 • Published • 1 -
Less is More: Selective Layer Finetuning with SubTuning
Paper • 2302.06354 • Published • 1 -
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition
Paper • 2308.14929 • Published • 1 -
Robust low-rank training via approximate orthonormal constraints
Paper • 2306.01485 • Published • 1 -
Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training
Paper • 2303.13635 • Published • 1 -
Cuttlefish: Low-Rank Model Training without All the Tuning
Paper • 2305.02538 • Published • 1 -
Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximation
Paper • 2108.10808 • Published • 1 -
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Paper • 2110.15343 • Published • 1 -
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Paper • 2305.17331 • Published • 1 -
Continual Learning with Low Rank Adaptation
Paper • 2311.17601 • Published • 1 -
Sparse Low-rank Adaptation of Pre-trained Language Models
Paper • 2311.11696 • Published • 1 -
Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Paper • 2311.00915 • Published • 1 -
Punica: Multi-Tenant LoRA Serving
Paper • 2310.18547 • Published • 2 -
Greenformer: Factorization Toolkit for Efficient Deep Neural Networks
Paper • 2109.06762 • Published • 1 -
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
Paper • 2311.12023 • Published • 2 -
Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters
Paper • 2309.11042 • Published • 2 -
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning
Paper • 2311.11077 • Published • 24 -
eP-ALM: Efficient Perceptual Augmentation of Language Models
Paper • 2303.11403 • Published • 3 -
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation
Paper • 2209.09815 • Published • 1 -
Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning
Paper • 2310.11670 • Published • 1 -
Parameter Efficient Tuning Allows Scalable Personalization of LLMs for Text Entry: A Case Study on Abbreviation Expansion
Paper • 2312.14327 • Published • 6 -
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning
Paper • 2308.12043 • Published • 1 -
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation
Paper • 2210.07558 • Published • 1 -
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks
Paper • 2106.04489 • Published • 1 -
Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks
Paper • 2210.03265 • Published • 1 -
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer
Paper • 2205.12148 • Published • 2 -
Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning
Paper • 2309.06922 • Published • 1 -
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
Paper • 2312.00968 • Published • 1 -
MultiLoRA: Democratizing LoRA for Better Multi-Task Learning
Paper • 2311.11501 • Published • 33 -
Orthogonal Subspace Learning for Language Model Continual Learning
Paper • 2310.14152 • Published • 2 -
Tied-Lora: Enhacing parameter efficiency of LoRA with weight tying
Paper • 2311.09578 • Published • 14 -
ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
Paper • 2311.13600 • Published • 42 -
PELA: Learning Parameter-Efficient Models with Low-Rank Approximation
Paper • 2310.10700 • Published • 1 -
SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models
Paper • 2305.05189 • Published • 2 -
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Paper • 2401.01335 • Published • 64 -
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models
Paper • 2310.16240 • Published • 1 -
Parameter-Efficient Tuning with Special Token Adaptation
Paper • 2210.04382 • Published • 1 -
NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning
Paper • 2307.08941 • Published • 1 -
Trained Rank Pruning for Efficient Deep Neural Networks
Paper • 1812.02402 • Published • 1 -
TRP: Trained Rank Pruning for Efficient Deep Neural Networks
Paper • 2004.14566 • Published • 1 -
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference
Paper • 2304.04947 • Published • 1 -
Training Neural Networks with Fixed Sparse Masks
Paper • 2111.09839 • Published • 1 -
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
Paper • 2401.02731 • Published • 2 -
Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models
Paper • 2309.01479 • Published • 1 -
Efficient Storage of Fine-Tuned Models via Low-Rank Approximation of Weight Residuals
Paper • 2305.18425 • Published • 1 -
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles
Paper • 2401.00243 • Published • 1 -
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
Paper • 2401.00788 • Published • 21 -
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
Paper • 2106.04647 • Published • 1 -
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Paper • 2205.12410 • Published • 1 -
BitDelta: Your Fine-Tune May Only Be Worth One Bit
Paper • 2402.10193 • Published • 19 -
DoRA: Weight-Decomposed Low-Rank Adaptation
Paper • 2402.09353 • Published • 26 -
Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
Paper • 2402.14688 • Published -
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Paper • 2403.03507 • Published • 182 -
Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU
Paper • 2403.06504 • Published • 53 -
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
Paper • 2403.13372 • Published • 62 -
LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
Paper • 2405.00732 • Published • 118 -
LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment
Paper • 2312.09979 • Published • 1 -
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Paper • 2405.12130 • Published • 46 -
MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models
Paper • 2402.12851 • Published • 2 -
SKIP: Skill-Localized Prompt Tuning for Inference Speed Boost-Up
Paper • 2404.11916 • Published -
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution
Paper • 2405.17357 • Published -
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
Paper • 2405.19597 • Published -
Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
Paper • 2405.03003 • Published • 7 -
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
Paper • 2405.17604 • Published • 1 -
Sparse Matrix in Large Language Model Fine-tuning
Paper • 2405.15525 • Published -
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
Paper • 2406.02214 • Published -
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections
Paper • 2405.20271 • Published -
Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models
Paper • 2405.21050 • Published -
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
Paper • 2405.15179 • Published • 1 -
Spectral Adapter: Fine-Tuning in Spectral Space
Paper • 2405.13952 • Published -
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs
Paper • 2405.16325 • Published • 1 -
SinkLoRA: Enhanced Efficiency and Chat Capabilities for Long-Context Large Language Models
Paper • 2406.05678 • Published -
LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
Paper • 2406.00605 • Published • 2 -
Effectively Compress KV Heads for LLM
Paper • 2406.07056 • Published -
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation
Paper • 2406.10785 • Published • 1 -
SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
Paper • 2405.16057 • Published