Pruning on Feng's Blog

LLM/VLM Compression Foundations

Sun, 10 May 2026 00:00:00 +0000

I started looking at model compression because the numbers didn’t add up. My GPU has 24GB of VRAM and the models I want to run need 40GB. The gap is a factor of two, which quantization claims to solve. But then I found papers about pruning, and distillation, and token compression, and hardware-aware NAS, and suddenly the question wasn’t “which technique” but “which combination, in what order, for which hardware.”

This article is my attempt to organize what I’ve learned into a coherent map. It is not a survey — there are good surveys for that. It is a working notebook: what I understand, what surprised me, and what I still can’t explain.

Thesis: Compression works because neural networks are overparameterized for the expressivity they actually use. The hard part is knowing which bits are the ones that don’t matter — and that answer depends on what you’re compressing (text vs. vision-language), how you remove it (prune, quantize, or distill), what order you apply the steps (P-KD-Q), and what hardware runs the result.

Scope: This covers the foundations of LLM and VLM compression — the three pillars (pruning, quantization, distillation), token compression, NAS for compression, the empirical ordering evidence, failure modes, and hardware decision rules. It does not cover training-from-scratch efficiency, inference serving systems (vLLM, TensorRT-LLM) beyond their connection to compression, or retrieval-augmented generation.

Prerequisites: This assumes familiarity with transformer architectures, basic neural network training (backpropagation, gradient descent), floating-point representation, and cross-entropy loss.

1. Overparameterization is the precondition

If models weren’t overparameterized, compression wouldn’t work. The Lottery Ticket Hypothesis established this formally in 2018: dense, randomly-initialized networks contain subnetworks that, trained in isolation, match the full network’s accuracy. For modern LLMs, the numbers are concrete — up to 30% of parameters can be pruned with negligible loss, and models hold 98-99% of original capabilities at just 15% pruning.

This overparameterization isn’t a mistake. Sparse architectures are hard to train from scratch. We train dense and then compress because that’s what the optimization surface allows.

The shape of the redundancy matters, and it differs by modality. This is something I initially underestimated — I thought all redundancy was weight-level, but the token-level and modality-dependent patterns are just as important for practical compression decisions.

Modality	Redundancy pattern	Scale
Images	Spatial — neighboring patches share textures/colors	—
Video	Spatiotemporal — consecutive frames share backgrounds; at 10fps, 1000 tokens/frame, a 90-min video yields ~54M tokens	54M tokens/video
Audio	Salient info concentrates in sparse, brief segments and specific frequency bands	—
MLLM sequences	>50% of tokens get minimal attention; multimodal tokens are >80% of sequences in reasoning tasks	>80% of sequence

All compression exploits some version of this: there are bits you can throw away because they don’t change the output. The question is which bits.

2. The three pillars, and two newer additions

The literature converges on five categories. Three dominate practice:

Method	Mechanism	What it reduces	Tuning required
Quantization	Lower-bit weight/activation representation	Memory, potentially speed	Often tuning-free for LLMs
Pruning	Remove unimportant weights or structures	Parameters, compute	Recovery training at high ratios
Distillation	Transfer knowledge from large → small model	Parameters, compute	Training a student

Token compression and Neural Architecture Search sit alongside these — newer, less universal, but important for specific scenarios.

2.1 Quantization: the hardware-sensitive frontier

Quantization converts float32/float16 weights to fewer bits. The fundamental tension: non-uniform quantization achieves higher accuracy because weights aren’t uniformly distributed, but uniform quantization gets hardware support. You cannot have both accuracy and hardware efficiency simultaneously with existing methods.

A critical asymmetry drives the research: weights are easy to quantize, activations are hard because of outlier distributions. SmoothQuant addresses this by migrating quantization difficulty from activations to weights via per-channel scaling:

$$Y = X \cdot \text{diag}(s)^{-1} \times \text{diag}(s) \cdot W$$

This enables W8A8 quantization with minimal accuracy loss and a 2× throughput gain. The idea is simple — smooth the activation outliers into the weights where they do less damage — but the execution requires careful per-channel scaling factors.

The outlier problem, quantified

ICQuant reveals the structure of the problem in a way I find unusually clean: the top 5% of weight outliers consume about 50% of the total value range — meaning one full quantization bit gets wasted on just 5% of the weights. About 97% of weight channels have uniformly-distributed outlier positions (verified across Llama2/3/4 and Qwen2.5 families), which enables a per-channel partitioning strategy: separate codebooks for outliers and inliers, combined with index coding that costs ≈0.3 bits/weight vs. ≈1 bit for prior approaches.

The production baseline and the frontier

FP8 (E4M3) on NVIDIA H100/B200 is the modern production baseline — essentially lossless 50% memory reduction from FP16. 4-bit PTQ (AWQ, GPTQ) achieves virtually lossless quantization for models above 70B parameters. QuIP/QuIP# pushes to 2 bits by multiplying weight and Hessian matrices with randomized Hadamard transforms to make entries approximately i.i.d. Gaussian, enabling E8 lattice codebook quantization.

At the extreme frontier: LittleBit reaches 0.1 bits/weight through latent factorization; iFairy uses complex numbers {±1, ±i} for 2-bit “multiplication-free” inference via sign flips.

Edge and VLM-specific quantization

Edge deployment demands specialized methods. Q-VLM minimizes cross-layer dependency errors in LVLMs using activation entropy as a proxy. MBQ accounts for differential sensitivity between vision and language tokens, achieving up to 1.4× decoding speedup with a custom W3 kernel. P4Q introduces learnable prompts and a lightweight low-bit adapter to realign post-quantization feature distributions.

KV-cache quantization deserves separate mention. In PaLM-540B with batch size 512 and context length 2048, the KV cache alone needs 3TB — three times the model parameters. KIVI-style KV-cache quantization is now table-stakes for long-context serving.

2.2 Pruning: three strategies, three hardware outcomes

Pruning’s real-world impact depends entirely on the pattern, because hardware can only exploit certain sparsity structures:

Pruning type	What’s removed	Hardware speedup	Examples
Unstructured	Individual weights	None without sparse kernels	SparseGPT, Wanda
Semi-structured	Fixed patterns (2:4, 4:8)	Yes on NVIDIA Ampere+	SparseGPT 2:4, Wanda N:M
Structured	Whole layers/heads/channels	Yes on commodity hardware	LLM-Pruner, NIRVANA, UKMP

The key insight — and the one I keep coming back to — is that unstructured sparsity achieves the best accuracy but delivers zero speedup without special hardware. Structured pruning physically reduces matrix dimensions — immediate gains on any hardware, at higher accuracy cost. Semi-structured 2:4 sparsity is NVIDIA’s compromise: hardware-supported on Ampere GPUs, but one-shot methods like SparseGPT and Wanda still suffer at 60-80% sparsity or with tight 2:4 constraints.

Beyond uniform sparsity: per-dimension pruning

A critical limitation of prior methods is uniform sparsity within layers — all output dimensions of a weight matrix get the same pruning ratio. TRIM demonstrates this is deeply suboptimal: individual output dimensions differ significantly in sensitivity. By assigning unique per-row sparsity ratios via iterative metric-driven adjustment, TRIM reduces OPT-13B perplexity at 80% sparsity from 6461 (Wanda-based OWL) to 324 — over 95% reduction in perplexity at the same sparsity level.

NIRVANA redesigns structured pruning by combining magnitude-scaled gradient saliency ($|\partial f / \partial W \cdot W|$) with Adam-based NTK stability guarantees. The dual criterion balances output preservation with training stability — Proposition 4.1 proves $|\hat{\Theta} - \Theta| \leq O(\varepsilon)$ under the SignGD kernel.

Key design choices:

Adaptive sparsity allocation: parameter $\gamma$ controls MLP vs. attention pruning rates ($v_{\text{MLP}} = \gamma \cdot v_{\text{Attn}}$)
Hardware-aware dimension alignment: all hidden dimensions forced to multiples of 8 for Tensor Core compatibility
Global joint ranking across all layers/modules with a safeguard retaining ≥1 unit per layer to prevent collapse

At 50% sparsity, NIRVANA achieves WikiText2 perplexity (PPL) of 48.94 vs. 215.94 for LLM-Pruner on Llama3.1-8B. Ablation reveals that magnitude-based scoring alone causes extreme collapse (PPL ≈ 10⁵–10⁶), and removing adaptive allocation $\gamma$ raises PPL from 48.94 to 102.00.

FastForward Pruning reformulates sparsity allocation as a single-step RL problem. The RL state is defined solely by the global target sparsity $\sigma_t$ (enabling transfer learning), with a ratio-based reward function ( $PPL_{\text{dense}} / PPL_{\text{pruned}}$ ) that is scale-invariant for portability across model sizes. Results: 3.4× faster than EAS (6.13 vs. 23.6 GPU-hr) on LLaMA-V1 7B at 20% sparsity, with better PPL (6.64 vs. 6.89).

VLM-specific structured pruning: UKMP

Text-only pruning methods fail on LVLMs because they treat the language backbone in isolation, ignoring the vision-language interface. UKMP (AAAI 2025) introduces the first unified structured pruning framework purpose-built for LVLMs.

The UKMI metric combines three innovations:

Adaptive dual normalization: block-wise normalization (by parameter volume) prevents large modules from dominating; modality-wise normalization balances vision and language components
First-order gradient saliency: UKMP discards the second-order Fisher term because the convergence assumption of second-order derivatives is invalid when parameters are frozen — they retain first-order gradients
Angle distribution entropy: entropy over 100 cosine bins weights the Taylor importance, penalizing parameters whose removal would cause large angular shifts in feature space

Recovery uses a weight recalling module: low-rank $P_2 Q_2 W^p$ transformation parallel to LoRA, trained through three-phase progressive distillation (vision-only MSE → vision+language MSE → task loss + KL). This module is reparameterizable — it folds into base weights after training at no inference cost.

At 50% pruning, UKMP achieves 47.81% VQAv2 accuracy (vs. 36.40% next-best) and 96.92 NoCaps CIDEr (vs. 85.51). Even at 20% pruning, the pruned BLIP-2 beats similarly-sized full BLIP-2 on OK-VQA and GQA.

2.3 Distillation: transfer without the baggage

Knowledge distillation trains a smaller student model to mimic a larger teacher. The three challenges: what knowledge to transfer, which algorithm to use, and how to design the student-teacher pair.

White-box distillation using KL-divergence at high temperature ($\tau = 4.0$) reveals the teacher’s confidence across the full vocabulary, enabling finer-grained transfer than black-box methods relying on text outputs alone.

Curriculum distillation with selective reflection

SRD (Selective Reflection Distillation) demonstrates that not all training samples contribute equally — and that curriculum ordering matters. Easy-to-hard curriculum significantly outperforms reverse hard-to-easy ordering. An increasing temperature schedule ($\tau_0 = 1 \to \tau_n = 2$) is a key effectiveness driver; reversing it severely degrades results.

SRD achieves up to 39% training time reduction while using 75% of data, and consistently improves ROUGE-L by 3.92–15.53% across all 7 tested KD methods on 5 benchmarks. It is plug-and-play — no changes to model architectures, loss functions, or KD algorithms. It even enables distilled students to surpass teacher performance (26.07 vs. 25.15 ROUGE-L for OpenLLaMA2).

VLM-specific distillation

VLMs present unique challenges because cross-modal alignment must be preserved.

Switch-KD (CVPR 2026) unifies vision-language knowledge transfer within a shared text-probability space. The Visual-Switch Distillation pathway switches student visual outputs into the teacher’s language pathway ($S\text{-ViT} \to T\text{-Projector} \to T\text{-LLM}$), producing visual-switch logits that represent the teacher’s output distribution conditioned on student-encoded visual representations. This is supervised by DBiLD loss, which uses the Kneedle algorithm for adaptive top-k boundary detection and bidirectional reverse KL alignment on pairwise logit differences — outperforming forward KL by 0.5 points.

Switch-KD-0.5B achieves +3.6 Avg10 over TinyLLaVA-0.5B across 10 multimodal benchmarks and matches the 3B teacher with half the parameters. However, it requires feature-space and vocabulary consistency between teacher and student.

Align-KD rests on a critical architectural finding: cross-modal alignment in VLMs occurs primarily at the first attention layer’s text-query-vision component ($A_{1, t \leftarrow v}$). Distilling only this targeted attention map achieves the same performance as distilling all maps while saving up to 50% computation. Distilling the wrong component is harmful: vision-query-vision attention KD collapses performance to 43.7 (vs. 64.4 baseline).

Bridging black-box and white-box distillation

The strongest teachers (GPT-4, proprietary models) are black-box — only text outputs available via API. White-box KD requires internal parameters. GrayKD (AAAI 2026) bridges this with a single-stage framework using no proxy teacher. Black-box rationales are injected through a lightweight cross-attention module — student hidden states as queries, rationale embeddings as keys/values, with 15% random masking for augmentation.

The efficiency gain is dramatic: GrayKD uses 610M parameters total vs. 2.06B for conventional KD pipelines. GrayKD Triple achieves 27.64 Avg Rouge-L, beating PromptKD + White Teacher (26.44) — despite using the same black-box GPT-4o-mini teacher as lower-scoring methods. Rationale diversity is the dominant factor: switching from multi-rationale to single-rationale reuse drops Rouge-L by 1.14 points.

2.4 Token compression: compressing the input, not the model

Token compression operates upstream of the three traditional pillars: instead of compressing model weights, it compresses the input. Approaches are categorized by modality (image/video/audio) and mechanism (transformation-based, similarity-based, attention-based, query-based). The key advantage: token compression is post-optimization, requiring no retraining.

I find this category theoretically elegant but practically limited — it only helps when tokens dominate the compute budget, which is true for video and long-context multimodal tasks but less so for standard image+text inference.

2.5 NAS for compression

CompressNAS treats Tucker rank selection as a global search problem, using an MSE-based accuracy proxy comparing decomposed vs. reference layer feature vectors. Existing zero-cost proxies (NASWOT, GraSP, SNIP, ZiCo) fail monotonic trends at higher ranks. CompressNAS builds two lookup tables ($\Delta\text{acc}$, $\Delta\text{flash}$) and uses ILP-based NAS to select ranks globally given a hardware budget — 8× compression of ResNet-18 on ImageNet with <4% accuracy drop.

LLM-NAS solves a problem I hadn’t considered: LLM-driven architecture search exhibits exploration bias, repeatedly proposing designs within a narrow region of the search space. The fix is three innovations:

Complexity-driven partitioning into 6 disjoint niches defined by architectural complexity (nor_conv_3×3 count)
LLM-powered prompt co-evolution — prompts and architectures co-evolve across rounds
XGBoost zero-cost predictor aggregating 13 proxy metrics with Spearman correlation ~0.90 to ground truth

Search takes 3 minutes and 120 API calls vs. 2–17 GPU-days for supernet baselines. Removing partitioning drops hypervolume from 0.978 to 0.516. Removing the LLM entirely drops it to 0.843.

3. The P-KD-Q ordering: sequence matters

A systematic study on Qwen2.5-3B shows that compression ordering is not a detail — it determines whether the pipeline works at all.

Sequence	Compression	G-Eval	PPL	Verdict
P-KD-Q	3.68×	0.733	5.048	Best
KD-P-Q	3.68×	0.644	—	Intermediate
P-Q-KD	3.68×	0.610	—	Intermediate
KD-Q-P	3.68×	—	53.4	Collapse
Q-P-KD	3.68×	0.060	34.5	Near-zero
Q-KD-P	3.68×	0.080	24.1	Near-zero

The mechanism is specific and instructive — and worth pausing on because it explains an entire class of pipeline failures: NF4 quantization produces inference-only models incompatible with gradient-based training. Any sequence with Q before training steps is dead on arrival. The P-KD-Q sequence lets each step compound: pruning reduces the search space, distillation transfers knowledge to the pruned architecture, quantization reduces precision with minimal added loss.

A practical note: quantization alone achieves 3.00× compression (5886→1959 MB). Adding pruning and distillation adds only 0.68× more (to 3.68×) at significant complexity cost. For many use cases, quantization alone is the right answer.

4. Where compression fails

4.1 The alignment cliff in VLMs

VLM compression has a failure mode absent in text-only LLMs. At low compression ratios, structural pruning damages multimodal alignment (vision ↔ language) more than the language backbone; at high ratios, both degrade. This means for mild compression, fine-tuning only the multimodal projector is sufficient — you are repairing the alignment bridge, not the entire model.

UKMP addresses this directly through modality-wise adaptive normalization and its weight recalling module’s progressive three-phase distillation. Text-only importance metrics (magnitude, gradient) cannot detect which parameters mediate the vision-language interface. The convergence assumption of Fisher information is also invalid for VLMs: frozen parameters retain first-order gradients, making second-order importance estimates actively misleading.

4.2 Extreme sparsity collapse

One-shot pruning methods degrade severely at 60-80% sparsity with semi-structured patterns. NIRVANA’s ablation shows magnitude-based scoring alone causes PPL ≈ 10⁵–10⁶ at 50% sparsity. Attention-only pruning causes catastrophic collapse; joint pruning of attention and MLP yields the smoothest degradation.

4.3 Early quantization destroys trainability

Applying NF4 quantization before any other technique destroys trainability. Q-KD-P and Q-P-KD sequences achieve near-zero G-Eval scores (0.080, 0.060). The gradient-free nature of NF4-quantized models means they cannot participate in subsequent distillation or pruning recovery.

4.4 Layer sensitivity isn’t uniform

In partial 2:4 sparsification, later layers are more sensitive than earlier ones — skipping the last third of the model yields the best accuracy. For LVLMs, widthwise pruning of attention heads and MLP neurons outperforms wholesale layer removal. And within a single layer, individual output dimensions differ dramatically in sensitivity.

5. How hardware changes everything

5.1 The hardware-taxonomy mismatch

The compression technique that looks best on paper often delivers zero real-world speedup. Models optimized for GPU do not run fast on CPU and mobile, and vice versa.

If your target is…	Prefer…	Avoid…
Datacenter GPU (A100/H100)	Semi-structured 2:4 + quantization	Pure unstructured
Edge/CPU/Mobile	Structured pruning (widthwise)	Any unstructured or semi-structured
Long-context serving	KV-cache quantization	—
Extreme compression (≤45%)	Structured + distillation recovery	One-shot pruning alone

5.2 Memory bandwidth is the real bottleneck

During autoregressive decode, each token generation requires loading the entire model from memory — a classic memory-bandwidth-bound operation. This explains why quantization helps more than pruning for decode latency (smaller weights mean less data movement), why KV-cache quantization becomes critical at long contexts, and why joint algorithm-hardware optimization is the only path to order-of-magnitude gains.

The Titanus accelerator takes this to the extreme: chiplet-based digital computing-in-memory stores all static weights on-chip, eliminating repeated weight reloading during decode — a 39.4× reduction in off-chip memory access.

5.3 The edge reality

CLIP-B/16 at 149.6M parameters already exceeds Jetson Nano’s 4GB RAM (no dedicated GPU), causing frequent memory swaps that kill real-time performance. Edge deployment demands the full toolbox: pre-deployment compression, efficient fine-tuning, runtime optimization, and careful security/privacy handling.

5.4 A practical decision flow

For compressing an existing LVLM:

Extremely low resources, no recovery training: Widthwise pruning only. Accept accuracy loss.
Moderate compression (≤30%): Layerwise pruning + multimodal projector fine-tuning (5% of original data suffices).
High compression (≤45%): Widthwise pruning + supervised fine-tuning + hidden-state distillation.
For any combination: 4-bit quantization adds ~+0.1 PPL on top of sparsity.
Always: P-KD-Q ordering. Never quantize before training.

6. Where we are in 2025

The compression field has matured from experimental techniques into an engineering discipline with clear tiers:

Tier	Category	Examples	Status
Production	Foundational pruning	SparseGPT, Wanda, LLM-Pruner	Deployed
Production	4-bit quantization	AWQ, GPTQ, NF4 (QLoRA)	Deployed
Production	Inference engines	vLLM (PagedAttention), TensorRT-LLM	Deployed
Production	FP8 baseline	H100/B200 hardware-native	Deployed
Experimental	Extreme pruning	TRIM, NIRVANA, FastForward	Active research
Experimental	VLM-specific pruning	UKMP (UKMI + weight recalling)	Active research
Experimental	Ultra-low-bit quant	LittleBit (0.1-bit), ICQuant, iFairy	Active research
Experimental	Curriculum distillation	SRD	Active research
Experimental	VLM distillation	Switch-KD, Align-KD	Active research
Experimental	Black-box KD	GrayKD (610M params, no proxy)	Active research
Experimental	NAS for compression	CompressNAS, LLM-NAS, HAT	Active research

I expect several of the experimental rows to move to production within 12-18 months. TRIM-style per-dimension pruning and ICQuant-style index coding are both conceptually simple enough to integrate into existing pipelines. UKMP’s modality-aware pruning is clearly the right approach for VLMs — the question is whether it generalizes beyond BLIP-2 to LLaVA-style architectures.

7. A unifying mental model

Compression works because networks use fewer bits of information than they allocate parameters. The art is knowing which parameters carry that information. The answer depends on four things:

What redundancy exists: token-level, weight-level, layer-level — and modality-dependent. Video has spatiotemporal redundancy. MLLM sequences are >80% multimodal tokens that receive minimal attention. Within a single weight matrix, individual rows differ in sensitivity by orders of magnitude.
How you remove it: quantize, prune, or distill. Quantization targets precision. Pruning targets structure. Distillation targets knowledge transfer. Token compression targets the input directly.
What order you apply techniques: P-KD-Q is empirically optimal. Any sequence with Q before training steps fails catastrophically. This is not a heuristic — it follows from NF4’s gradient-free nature.
What hardware runs it: This is what determines whether a 50% parameter reduction translates to a 50% latency reduction or no reduction at all. Unstructured sparsity wins on accuracy but loses on every hardware metric. Structured pruning is the opposite. Semi-structured 2:4 is NVIDIA’s compromise.

What I find striking about the failure modes is how cleanly they carve the parameter space. In VLMs, the vision-language alignment is more fragile than the language backbone. In deep transformers, later layers carry disproportionate importance, and individual output dimensions within the same layer differ dramatically. Hardware is not an implementation detail — it defines which removal patterns become faster.

The frontier is hybrid, sequential, and precision-extreme: combining pruning, distillation, and quantization in the right order, with per-dimension granularity, pushing quantization to fractions of a bit — while ensuring the pipeline remains trainable throughout.

Boundary conditions

This model assumes the pretrained model is available. If you are training from scratch with a compression target, the entire framework shifts — you would design the architecture sparse from the start rather than compressing post-hoc.
The P-KD-Q ordering evidence comes from a single systematic study on Qwen2.5-3B. I have not seen replication on larger models or different architectures. The mechanism (NF4 gradient-free) is general, but the magnitude of the ordering effect at other scales is unknown.
UKMP has been validated primarily on BLIP-2. Its generalization to LLaVA, InternVL, or other VLM architectures is an open question.
Token compression is effective when tokens dominate compute (long video, long context). For single-image QA, the gains are modest.
The practical decision flow (Section 5.4) assumes access to fine-tuning resources. For truly zero-shot deployment, only quantization and token compression apply.
I have not covered training-time efficiency (mixed precision, gradient accumulation, ZeRO, FSDP), which interacts with compression in deployment pipelines but is a separate topic.

Open questions

Does the P-KD-Q ordering effect replicate on models above 70B parameters? The mechanism is architecture-agnostic, but the magnitude could scale differently.
Why do some layers tolerate 4-bit quantization while structurally similar layers fall apart at 6-bit? I suspect effective rank or singular value distribution, but I have not tested this.
Can UKMP’s modality-aware importance metric be adapted to video-language models, where the redundancy patterns are spatiotemporally structured rather than spatially structured?
What is the minimum viable recovery data for structured pruning of LVLMs at 50%+ sparsity? The current evidence says 5% of original data for moderate compression and full SFT for high compression, but the boundary between these regimes is fuzzy.
At what point does token compression become more practical than model compression for video understanding? The 54M-token number for 90-minute video suggests the crossover exists but I do not know where.

[[Q]] Six months from now: has UKMP been extended to LLaVA-style architectures, and does the modality-aware importance metric generalize beyond BLIP-2?

References

Frankle & Carbin, “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”, ICLR 2019.
Xiao et al., “SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models”, ICML 2023.
Frantar & Alistarh, “SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot”, ICML 2023.
Sun et al., “Wanda: A Simple and Effective Pruning Approach for Large Language Models”, ICLR 2024.
Ashkboos et al., “SliceGPT: Compress Large Language Models by Deleting Rows and Columns”, ICLR 2024.
Ma et al., “LLM-Pruner: On the Structural Pruning of Large Language Models”, NeurIPS 2023.
NIRVANA: “NIRVANA: Neural Implicit Removal via Verifiable Adam-based NTK Alignment for Structured Pruning”, 2025.
TRIM: “TRIM: Per-Dimension Structured Pruning for Large Language Models”, 2025.
FastForward: “FastForward Pruning: Efficient LLM Pruning via Single-Step Reinforcement Learning”, 2025.
UKMP: “UKMP: Unified Knowledge Maintenance Pruning for Vision-Language Models”, AAAI 2025.
ICQuant: “ICQuant: Index Coding Quantization for Large Language Models”, 2025.
Switch-KD: “Switch-KD: Knowledge Distillation with Visual Switch for Efficient Vision-Language Models”, CVPR 2026.
Align-KD: “Align-KD: Shallow-Layer Attention Alignment for Mobile Vision-Language Models”, 2025.
GrayKD: “GrayKD: Gray-Box Knowledge Distillation for Large Language Models”, AAAI 2026.
SRD: “Selective Reflection Distillation: Curriculum Knowledge Distillation for LLMs”, 2025.
CompressNAS: “CompressNAS: Neural Architecture Search for Model Compression”, 2025.
LLM-NAS: “LLM-NAS: Large Language Models for Neural Architecture Search”, 2025.
Compression Ordering: “A Systematic Study of Compression Ordering for Large Language Models”, 2025.
Multimodal Token Compression Survey, arXiv 2507.20198, 2025.
Efficient VLM Survey: “Efficient Vision-Language Models: A Survey”, 2025.

AI时代，个体何以为家？

Thu, 26 Mar 2026 00:00:00 +0000

在不确定性中锚定自我的行动纲领

我们这代人，正不偏不倚地撞上一个历史的岔路口。眼看着技术洪流卷裹着社会加速狂奔，身后扬起的是前所未有的集体焦虑。尤其是在这波来势汹汹的人工智能浪潮里，一个直击灵魂的拷问像海底的暗涌，悄然浮出水面：当机器的智能和效率，已经无限逼近，甚至开始甩开人类时，我们这些凡夫俗子的价值，到底该往哪儿搁？我的脑子可不爱只停留在对“世界末日”的悲观预设上，那太没劲了。相反，我想把这看起来不可逆转的趋势，当作探路的起点，去琢磨琢磨，我们到底该如何凭着那点儿知识的微光，和一股子豁出去的劲头，为自己在这喧嚣中铸就一份意义，也为未来争取那么一点点更广阔的主体性自由。

一、解构“价值重构”的底层逻辑

要我说，人类的价值定义，就像海边的沙滩，被历史的潮汐反复冲刷，变幻莫测。你想啊，从茹毛饮血的时代，你力气大、跑得快，就能直接换来一口吃的；到后来有了土地，生产资料成了硬通货；再到工业革命，机器轰鸣着取代了无数体力活和手艺人，每一次技术大跃进，都在不动声色地调整着生产要素那杆秤的砝码。数字互联刚冒头那会儿，不少人还以为个体的黄金时代真来了，结果呢？不过是把打工的舞台从实体工厂搬到了资本搭建的线上平台，真正的大头，还是进了平台所有者的腰包。

我琢磨来琢磨去，发现这些所谓的“进步”，嘴上说着要“解放人类”，骨子里玩的却是另一套把戏——悄悄地、系统性地“降低人在生产中的不可替代性”。它们把人那些独有的本事，一点点拆解成可以复制、可以掌控的工具，然后，咔嚓一下，个体在财富分配中的议价权就这么被无情地压低了。

而现在，AI的横空出世，特别是它开始渗透到认知、创意，甚至连情感领域都不放过的时候，我得说，这场“把人从价值链条上剔除”的游戏，算是彻底进入了终极阶段。这可是头一回，机器把替代的触角伸到了我们人类最引以为傲的智识高地。那些我们拍着胸脯说“这活儿只有人能干”的事儿，AI正以一种更高效、更廉价的方式，大摇大摆地接手。这意味着什么？意味着过去我们靠着苦读、靠着积累专业知识来挖的那条“护城河”，正在以肉眼可见的速度被填平。生产的核心要素，已经不再是简单的“资本+劳动”，而是彻底变成了“资本+AI”，而“劳动”这个砝码，正被无限趋近于零。这不是什么简单的技术性失业，各位，这可是一场社会财富分配逻辑的深层结构性大重构，它极有可能把我们的社会，推向一个前所未有的两极分化深渊。

二、破除幻想，直面现实壁垒

面对眼前这个庞大得让人有点喘不过气来的结构性挑战，社会上当然不缺那些急着要“破局”的声音。你看，有人把希望寄托在“技术平权”上，天真地以为只要人人都能用上AI工具，就能把差距给抹平了；也有人扯着嗓子呼吁“制度干预”，指望着靠政策来重新分配蛋糕；更有甚者，还乐观得有点可爱，觉得等AI把所有“有用”的活儿都干完了，人类的价值就能回归到艺术、情感这类“无用”之事上去了。

可我这个人，向来喜欢把事情掰开了揉碎了看。一番理性审视下来，这些听起来很美的“破局”路径，恐怕都得撞上现实的几堵高墙：

首先，什么“技术平权”，在我看来不过是个美丽的幻象。 咱们个人能拿到的，顶多是AI的“使用权”，注意，是“使用权”，可不是“所有权”！AI最核心的壁垒——那天文数字般的算力、海量的数据、还有动辄几十上百亿参数的大模型——依然牢牢地攥在少数资本巨头手里。你想想，游戏的规则，制定权永远属于那些掌握生产资料的人，不是吗？

其次，指望“制度干预”来力挽狂澜，也面临着一个“囚徒困境”。 资本这东西，早就把触角伸进了全球各国的政策体系，渗透之深，超乎想象。再加上各国之间为了抢占科技高地，那竞争简直是刺刀见红。在这种背景下，任何一个国家想要限制AI发展，都可能陷入“我限制了，别人没限制，那我不是傻了吗”的尴尬境地，结果往往是大家谁也别想管，只能眼睁睁看着资本的权重持续无限放大。

最后，那些关于“价值重定义”的美好愿景，也得面对残酷的现实。 没错，当AI把“有用”的活儿都抢走了，我们是能把重心放回艺术、情感这些“无用之事”上，可问题是，在现行的财富分配体系下，这些“无用之事”在经济上根本没有议价权啊！除非整个社会的基本价值体系来一次地动山摇式的根本性重构，否则，它们怎么可能成为我们个体安身立命的坚实基础呢？

所以你看，全民之所以焦虑得夜不能寐，根源压根儿不是什么“学不会AI”，而是我们这些清醒的旁观者，已经看透了：无论你个体如何拼命折腾，都很难成为AI时代的规则制定者，充其量，也只能是被这股洪流裹挟着前行的一员罢了。

然而，就算眼前这终局看起来板上钉钉、不可逆转，我心里头却依然有那么一份坚信：人类的能动性，可没那么容易就全然丧失。技术迭代的速度确实是指数级的，像坐上了火箭，可社会利益格局的重构啊，往往是线性的，像老牛拉破车。拦着AI全面落地，从来不只是技术本身，更多的是人性与利益那剪不断理还乱的复杂交织。你想想，那些既得利益群体的本能抗拒、国家对社会稳定的优先考量、还有咱们中国商业社会里头根深蒂固的人情世故和信任网络，这些可都是AI想全面替代，也得碰壁的无形壁垒。它们为我们争取到了一个宝贵的5到10年的“窗口期”。这个窗口期，可不是让我们躺平任嘲、坐以待毙的，恰恰相反，它提供了一个清醒的、不抱任何幻想的现实出口，一个让我们能够积累、能够转型、能够战略布局的绝佳机遇期。

三、行动纲领：在不确定中铸就主体性

启蒙思想教我们要理性自律，存在主义又在耳边低语，呼唤我们积极介入这个世界。到了AI时代，面对个体价值被大水稀释的宏大叙事，我们更该回归到最朴素的本源：把知识当作指路的火把，把行动当作渡河的舟楫，在这不确定性的大海里，牢牢地锚定住那个“我”，为自己铸就一份实实在在的主体性自由。

个体层面的自我铸就：

先来说说我们最能直接掌控、也最有希望出成绩的领域——个体层面的自我铸就。毕竟，自己这亩三分地，总得先耕耘好不是？

首先，要卯足劲儿培养那些AI暂时还摸不着边儿的“元技能”和“复合能力”。 这些是我们的独门绝技：你得练就一副批判性思维的火眼金睛，能透过现象看本质，在信息爆炸和一团浆糊的模糊情境里，依然能独立做出判断，去解决那些AI看了都头疼的开放性、非线性难题。 创新和创造力更是重中之重。AI再厉害，也只是个超级缝合怪，真正的原创性思考、那种直击人心的艺术表达、还有跨领域的融合创新，它还真学不来。我们要把AI当成激发灵感、提升效率的趁手工具，而不是让它来替代我们思考。别忘了情感智能和人际协作。领导力、沟通协调、同理心、谈判博弈、化解冲突、社群搭建……这些都是构建信任、连接人心的核心要素，是机器永远无法复制的柔软力量。我们还得成为AI的**“价值守门人”，在人机协作中，最终的伦理责任得由我们来扛。说到底，要确保技术这把双刃剑，是真真切切地服务于人类福祉，而不是反噬自身。最后，也是最关键的，是终身学习与超级适应性**。别害怕变化，要把学习当成一种生存本能，持续驾驭AI的“新语言”和“新工具”，让AI成为我们自身能力的“超级放大器”，而不是扼杀器。

其次，我们要积极积累那些能让我们“反脆弱”的资本。 毕竟，手里有粮，心里不慌： 物质资本当然不能少。趁着这宝贵的窗口期，该赚的钱得努力赚，该攒的资产得好好攒，给自己多增加点财务冗余，真遇到大风大浪，也能有个兜底的。 社会资本更是无价之宝。好好经营你的人际关系、信任网络，多参与社群活动。要知道，那些深厚的人际连接和互助网络，是AI永远也复制不来的“价值堡垒”，关键时刻能救命。 认知资本也不能忽视。别光盯着AI怎么用，更要深入理解它背后的底层逻辑，这样才能形成高层次的判断力和战略预见性，不至于被新概念牵着鼻子走。最后，也别忘了健康资本。身体是革命的本钱，身心健康，才能有足够的韧性去应对未来可能出现的各种挑战和压力，熬过这场大变局。

最最后，也是最最深刻的，我们要重新审视和重构生命的意义。 别再死盯着纯粹的经济生产和效率至上那套了，是时候把重心挪一挪，多关注体验、连接、创造和自我实现。在艺术创作里找灵感、在志愿服务中找到价值、在家庭生活里感受温暖、在户外探索中找回野性……这些非经济的领域，往往藏着更深层次的意义感和满足感。不妨学着拥抱“慢生活”和“反效率”的价值观，去对抗AI裹挟而来的那种过度功利主义和速度崇拜。要知道，守护人类独有的、那些不可量化的体验维度，才是我们作为“人”的最后一道防线。

组织与社会层面的积极介入：

当然，我们个体的努力并非孤立无援。涓涓细流汇成江海，通过积极的参与和倡议，我们完全可以汇聚起一股力量，去影响那些更广阔的系统，让它不至于偏航太远。

我们可以倡导“人机共生”的理念，推动岗位重塑。 鼓励那些组织和企业，别老想着让AI来“取代”人，而应该把它看作是人类能力的“增强器”。去设计那些人机协作的工作流程，创造出需要人类独特判断和创造力的新型“超级职位”，让AI成为我们的助手，而不是掘墓人。

我们要不遗余力地推动教育体系的革新。 大声疾呼，让教育别再是填鸭式的知识灌输，而是要彻底转向能力培养。把批判性思维、创造力、情商和伦理道德这些核心素养，摆到教学的C位，为我们的下一代，为他们能更好地适应AI时代做好万全准备。

同时，也要密切关注并支持新型分配机制的探索。 比如全民基本收入（UBI）、数据所有权、AI税等等，这些听起来有点超前的想法，或许正是未来社会财富分配的新方向。积极去了解它们的讨论和实践，促使技术带来的红利，能更广泛、更公平地惠及整个社会，而不是只流向少数人的口袋。

别忘了，还要积极参与到AI伦理与治理框架的构建中去。 关注AI的透明度够不够、它的决策能不能被解释、出了问题谁来负责。我们要大声倡导，把人类最核心的价值观融入AI的设计与应用中，给这匹脱缰的野马套上缰绳，避免技术最终失控，反噬我们自身。

四、锚定人类价值，共塑未来

说到底，我们今天所面对的，确实像是一场对人类个体价值的“终极清算”，一场足以颠覆现有社会结构的深远变革。但话说回来，我们这些浸润在启蒙理性与存在主义思潮中的个体，怎能轻易就把这“终局”当成无可逃避的宿命呢？知识，从来都是打破蒙昧、指引我们前行的火炬；而行动本身，更是在我们看清了世界的荒谬与挑战之后，依然选择勇敢地投身其中，为自己、也为我们所在的社群，一砖一瓦地铸就意义的过程。

真正的“体面”活法，绝不是在被告知“大势已去”之后，躺平了任由命运摆布。它应该是在这股技术洪流中，我们人类社会能够通过深思熟虑、而非盲目跟风，通过积极行动、而非被动接受，去共同构建一个以人类福祉为核心，以主体性自由为目标，一个真正可持续的未来。这并非要螳臂当车、逆转什么铁律，而是要先理解这些规律，然后在规律的缝隙里，找到我们人类能动性的最大化空间。所以，诸位，让我们把理性当作指引方向的灯塔，把勇气化作乘风破浪的风帆，在AI的滔天巨浪中，不仅仅是认识这个世界，更要撸起袖子，积极地去改造它，为人类的价值与尊严，锚定一个坚实而又光明的未来吧！

Hugo Qucik Introduction

Tue, 10 Mar 2026 21:20:12 +0100

First post with Hugo

In the root folder, execute command

hugo new content content/posts/my-first-post.md

This will create a new markdown file in the content/posts directory with the name my-first-post.md. The file will contain some default front matter, which you can edit to include your desired title, date, and other metadata.

---
title: "My First Post"
date: 2026-03-10T21:20:12+01:00
draft: true
---

The metadata are intoduced in Hugo as “front matter”. The title field specifies the title of the post, the date field indicates when the post was created, and the draft field is set to true, which means that the post will not be published until you change it to false. More information about front matter can be found in the Hugo documentation.

Quick review of the post content

Execute the command

hugo server -D

Then open your web browser and navigate to http://localhost:1313/. You should see your new post listed on the homepage. Click on the post title to view the full content of the post. Since the draft field is set to true, the post will not be visible on the public site until you change it to false and rebuild your site.

Publish the post

To publish the post, open the my-first-post.md file and change the draft field from true to false. Then save the file and rebuild your site by running the command:

hugo

After rebuilding, your post will be published and visible on the public site. You can verify this by navigating to http://localhost:1313/ again and checking that your post is now listed on the homepage.

About

Mon, 01 Jan 0001 00:00:00 +0000

A former physicist turned machine learning engineer, I have a passion for learning and sharing knowledge. With a background in physics, I bring a unique perspective to software development, combining analytical thinking with creativity. I enjoy exploring new technologies and applying them to solve real-world problems. In my free time, I like to read, travel, and experiment with new frontiers in deep learning and artificial intelligence. This blog is a platform for me to share my insights, experiences, and projects in the world of software engineering and beyond. I hope to inspire others to pursue their passions and contribute to the ever-evolving field of technology.