17 29 58

PenutChen

penut85420

penut85420

AI & ML interests

LLM, Quantization

Recent Activity

commented a paper 5 days ago

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

upvoted a paper 8 days ago

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

liked a model 12 days ago

IamCreateAI/Ruyi-Mini-7B

View all activity

Organizations

penut85420's activity

upvoted a paper 8 days ago

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

Paper • 2411.17691 • Published Nov 26, 2024 • 10

upvoted a collection 15 days ago

ModernBERT

Collection

Bringing BERT into modernity via both architecture changes and scaling • 3 items • Updated 16 days ago • 112

upvoted a paper about 2 months ago

Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs

Paper • 2410.10739 • Published Oct 14, 2024 • 2

upvoted a paper 3 months ago

Instruction Following without Instruction Tuning

Paper • 2409.14254 • Published Sep 21, 2024 • 27

upvoted a paper 5 months ago

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference

Paper • 2407.14057 • Published Jul 19, 2024 • 45

upvoted a paper 6 months ago

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients

Paper • 2407.08296 • Published Jul 11, 2024 • 31

upvoted 7 papers 7 months ago

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Paper • 2406.11931 • Published Jun 17, 2024 • 58

In-Context Editing: Learning Knowledge from Self-Induced Distributions

Paper • 2406.11194 • Published Jun 17, 2024 • 15

GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities

Paper • 2406.11768 • Published Jun 17, 2024 • 20

Parameter-Efficient Fine-Tuning with Discrete Fourier Transform

Paper • 2405.03003 • Published May 5, 2024 • 7

Let's Fuse Step by Step: A Generative Fusion Decoding Algorithm with LLMs for Multi-modal Text Recognition

Paper • 2405.14259 • Published May 23, 2024 • 1

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Paper • 1909.11942 • Published Sep 26, 2019 • 2

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Paper • 1810.04805 • Published Oct 11, 2018 • 16

upvoted a paper 8 months ago

Layer-Condensed KV Cache for Efficient Inference of Large Language Models

Paper • 2405.10637 • Published May 17, 2024 • 19

upvoted a paper 9 months ago

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning

Paper • 2403.17919 • Published Mar 26, 2024 • 16

upvoted 3 papers 10 months ago

You Need to Pay Better Attention

Paper • 2403.01643 • Published Mar 3, 2024 • 1

Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU

Paper • 2403.06504 • Published Mar 11, 2024 • 53

Divide-or-Conquer? Which Part Should You Distill Your LLM?

Paper • 2402.15000 • Published Feb 22, 2024 • 22

upvoted a collection 11 months ago

Canonical models

Collection

This collection lists all the historical (pre-"Hub") canonical model checkpoints, i.e. repos that were not under an org or user namespace • 68 items • Updated Feb 13, 2024 • 14

upvoted a paper 11 months ago

Fast Vocabulary Transfer for Language Model Compression

Paper • 2402.09977 • Published Feb 15, 2024 • 2