Posts by Tags

TL;DR: 本文梳理位置编码的主要演进路线：从可学习绝对位置编码（BERT、GPT）和正弦绝对编码（Transformer）的限制，到相对位置编码（T5、Transformer-XL）的改进，再到旋转位置编码（RoPE）的突破性创新。RoPE 的特点是「以绝对之形，行相对之实」：对每个位置做绝对旋转，但 Attention 内积只依赖相对距离。后续的 2D-RoPE 和 M-RoPE 把这个机制扩展到视觉和多模态场景，用于现代视觉语言模型（如 Qwen2-VL、Qwen3-VL）中的二维/三维时空位置建模。贯穿全文的问题是：模型到底如何表示「距离」，以及这种表示如何同时兼顾外推、KV Cache 和多模态坐标。

TL;DR: This article follows the main line of positional encoding research: the limitations of learned absolute embeddings in BERT and GPT, sinusoidal encoding in the original Transformer, the improvements introduced by relative positional encoding in models such as T5 and Transformer-XL, and finally Rotary Positional Encoding (RoPE). RoPE applies an absolute rotation at each position, yet the resulting attention inner product depends only on relative distance. Later extensions, including 2D-RoPE and M-RoPE, carry the same mechanism into vision and multimodal models such as Qwen2-VL and Qwen3-VL. The question throughout is how a model represents distance while preserving length extrapolation, KV caching, and multimodal coordinates.

TL;DR: 本文系统梳理了 Qwen-VL 系列四代模型的技术演进：从初代 Qwen-VL 建立了视觉-语言对齐和三阶段训练流程；到 Qwen2-VL 引入动态分辨率、M-RoPE 和视频输入；再到 Qwen2.5-VL 主要补推理效率、视频时间建模和后训练数据质量；最终走向更深层的视觉-语言融合（Qwen3-VL）。

TL;DR: This article traces four generations of the Qwen-VL family. The original Qwen-VL established visual-language alignment and a three-stage training pipeline. Qwen2-VL added dynamic resolution, M-RoPE, and video input. Qwen2.5-VL focused on inference efficiency, temporal video modeling, and post-training data quality. Qwen3-VL then moved visual-language fusion deeper into the model.

TL;DR: 本文梳理位置编码的主要演进路线：从可学习绝对位置编码（BERT、GPT）和正弦绝对编码（Transformer）的限制，到相对位置编码（T5、Transformer-XL）的改进，再到旋转位置编码（RoPE）的突破性创新。RoPE 的特点是「以绝对之形，行相对之实」：对每个位置做绝对旋转，但 Attention 内积只依赖相对距离。后续的 2D-RoPE 和 M-RoPE 把这个机制扩展到视觉和多模态场景，用于现代视觉语言模型（如 Qwen2-VL、Qwen3-VL）中的二维/三维时空位置建模。贯穿全文的问题是：模型到底如何表示「距离」，以及这种表示如何同时兼顾外推、KV Cache 和多模态坐标。

TL;DR: This article follows the main line of positional encoding research: the limitations of learned absolute embeddings in BERT and GPT, sinusoidal encoding in the original Transformer, the improvements introduced by relative positional encoding in models such as T5 and Transformer-XL, and finally Rotary Positional Encoding (RoPE). RoPE applies an absolute rotation at each position, yet the resulting attention inner product depends only on relative distance. Later extensions, including 2D-RoPE and M-RoPE, carry the same mechanism into vision and multimodal models such as Qwen2-VL and Qwen3-VL. The question throughout is how a model represents distance while preserving length extrapolation, KV caching, and multimodal coordinates.

TL;DR：在大语言模型（LLM）的 RLHF 里，强化学习是后训练阶段绕不开的一条线。从 OpenAI 提出的 PPO，到斯坦福大学提出的 DPO，再到 DeepSeek 提出的 GRPO，都极大推动了 LLM 的能力涌现。本文将从原理、公式推导到工程实现，系统梳理这三大算法的核心思想与演进逻辑，帮助读者建立完整的技术图谱。

TL;DR: Reinforcement learning runs through the post-training stage of RLHF for large language models. PPO from OpenAI, DPO from Stanford, and GRPO from DeepSeek have each changed how LLMs are trained after pre-training. This article covers their objectives, derivations, and implementations, then connects the design choices that led from one method to the next.

TL;DR: 本文系统梳理了 Qwen-VL 系列四代模型的技术演进：从初代 Qwen-VL 建立了视觉-语言对齐和三阶段训练流程；到 Qwen2-VL 引入动态分辨率、M-RoPE 和视频输入；再到 Qwen2.5-VL 主要补推理效率、视频时间建模和后训练数据质量；最终走向更深层的视觉-语言融合（Qwen3-VL）。

TL;DR: This article traces four generations of the Qwen-VL family. The original Qwen-VL established visual-language alignment and a three-stage training pipeline. Qwen2-VL added dynamic resolution, M-RoPE, and video input. Qwen2.5-VL focused on inference efficiency, temporal video modeling, and post-training data quality. Qwen3-VL then moved visual-language fusion deeper into the model.

TL;DR：在大语言模型（LLM）的 RLHF 里，强化学习是后训练阶段绕不开的一条线。从 OpenAI 提出的 PPO，到斯坦福大学提出的 DPO，再到 DeepSeek 提出的 GRPO，都极大推动了 LLM 的能力涌现。本文将从原理、公式推导到工程实现，系统梳理这三大算法的核心思想与演进逻辑，帮助读者建立完整的技术图谱。

TL;DR: Reinforcement learning runs through the post-training stage of RLHF for large language models. PPO from OpenAI, DPO from Stanford, and GRPO from DeepSeek have each changed how LLMs are trained after pre-training. This article covers their objectives, derivations, and implementations, then connects the design choices that led from one method to the next.

Paper: CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (CVPR 2025) Authors: Xing et al. TL;DR: CapRL 把图像描述质量转化为一个可验证问题：如果一段 caption 足够完整，纯文本 LLM 应该能只根据它回答图像相关的 MCQ。这个设定避开了 LVLM-as-a-Judge 对长度和风格的偏好，也给 RL 训练提供了更稳定的奖励信号。

Paper: CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (CVPR 2025) Authors: Xing et al. TL;DR: CapRL turns caption quality into a verifiable question: if a caption is complete enough, a text-only LLM should be able to answer image-related multiple-choice questions from that caption alone. This setup avoids the length and style preferences of LVLM-as-a-Judge and gives RL training a more stable reward signal.

TL;DR: 本文系统梳理了 Qwen-VL 系列四代模型的技术演进：从初代 Qwen-VL 建立了视觉-语言对齐和三阶段训练流程；到 Qwen2-VL 引入动态分辨率、M-RoPE 和视频输入；再到 Qwen2.5-VL 主要补推理效率、视频时间建模和后训练数据质量；最终走向更深层的视觉-语言融合（Qwen3-VL）。

TL;DR: This article traces four generations of the Qwen-VL family. The original Qwen-VL established visual-language alignment and a three-stage training pipeline. Qwen2-VL added dynamic resolution, M-RoPE, and video input. Qwen2.5-VL focused on inference efficiency, temporal video modeling, and post-training data quality. Qwen3-VL then moved visual-language fusion deeper into the model.

TL;DR: 本文梳理位置编码的主要演进路线：从可学习绝对位置编码（BERT、GPT）和正弦绝对编码（Transformer）的限制，到相对位置编码（T5、Transformer-XL）的改进，再到旋转位置编码（RoPE）的突破性创新。RoPE 的特点是「以绝对之形，行相对之实」：对每个位置做绝对旋转，但 Attention 内积只依赖相对距离。后续的 2D-RoPE 和 M-RoPE 把这个机制扩展到视觉和多模态场景，用于现代视觉语言模型（如 Qwen2-VL、Qwen3-VL）中的二维/三维时空位置建模。贯穿全文的问题是：模型到底如何表示「距离」，以及这种表示如何同时兼顾外推、KV Cache 和多模态坐标。

TL;DR: This article follows the main line of positional encoding research: the limitations of learned absolute embeddings in BERT and GPT, sinusoidal encoding in the original Transformer, the improvements introduced by relative positional encoding in models such as T5 and Transformer-XL, and finally Rotary Positional Encoding (RoPE). RoPE applies an absolute rotation at each position, yet the resulting attention inner product depends only on relative distance. Later extensions, including 2D-RoPE and M-RoPE, carry the same mechanism into vision and multimodal models such as Qwen2-VL and Qwen3-VL. The question throughout is how a model represents distance while preserving length extrapolation, KV caching, and multimodal coordinates.

Paper: CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (CVPR 2025) Authors: Xing et al. TL;DR: CapRL 把图像描述质量转化为一个可验证问题：如果一段 caption 足够完整，纯文本 LLM 应该能只根据它回答图像相关的 MCQ。这个设定避开了 LVLM-as-a-Judge 对长度和风格的偏好，也给 RL 训练提供了更稳定的奖励信号。

Paper: CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (CVPR 2025) Authors: Xing et al. TL;DR: CapRL turns caption quality into a verifiable question: if a caption is complete enough, a text-only LLM should be able to answer image-related multiple-choice questions from that caption alone. This setup avoids the length and style preferences of LVLM-as-a-Judge and gives RL training a more stable reward signal.