Higher-order Linear Attention
Paper
•
2510.27258
•
Published
•
5
Chmielewski
Eryk-Chmielewski
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
about 6 hours ago
Higher-order Linear Attention
upvoted
a
paper
about 6 hours ago
Continuous Autoregressive Language Models
liked
a model
about 10 hours ago
ByteDance-Seed/BFS-Prover-V2-32B
Organizations
upvoted
2
papers
about 6 hours ago
liked
7
models
about 10 hours ago
ByteDance-Seed/BFS-Prover-V2-32B
Text Generation
•
33B
•
Updated
•
384
•
10
racineai/CU-1
Object Detection
•
Updated
•
70
•
36
Salesforce/GTA1-32B
Image-Text-to-Text
•
33B
•
Updated
•
1.46k
•
5
inclusionAI/Ring-mini-linear-2.0-GPTQ-int4
Text Generation
•
3B
•
Updated
•
394
•
9
tiiuae/Falcon-E-3B-Instruct
Text Generation
•
0.9B
•
Updated
•
525
•
33
ai21labs/AI21-Jamba-Reasoning-3B
Text Generation
•
3B
•
Updated
•
5.37k
•
116
bytedance-research/DynamicCoT
8B
•
Updated
•
14
•
5
upvoted
a
paper
about 10 hours ago
upvoted
a
paper
about 11 hours ago
reacted to
Kseniase's
post with 🔥
about 11 hours ago
Post
7664
11 Fascinating new Policy Optimization techniques
Policy optimization (PO) algorithms are central to training AI models with preference-based feedback. In recent weeks, numerous new PO methods have emerged that build on or replace the popular PPO and GRPO, solving their issues. Here are 11 of them:
1. BAlanced Policy Optimization (BAPO) → BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping (2510.18927)
Dynamically adjusting the clipping bounds in PPO-style updates to balance positive and negative gradients and prevent entropy collapse
2. Training-Free GRPO → Training-Free Group Relative Policy Optimization (2510.08191)
Instead of using numeric rewards, it compares rollouts semantically to distill useful knowledge as a token prior, which is then applied during inference to guide the model’s behavior
3. Asymmetric Importance Sampling Policy Optimization (ASPO) → ASPO: Asymmetric Importance Sampling Policy Optimization (2510.06062)
Fixes imbalanced token weighting in LLM training. It flips the importance sampling ratios for positive tokens to correct over- and under-updates, and adds a soft dual-clipping step to keep gradients stable
4. In-Context Steered Policy Optimization (ICPO) → https://arxiv.org/abs/2510.26519
Uses a model’s own in-context learning ability to guide training with existing data. It combines Mixed-Policy GRPO with Implicit Expert Forcing to expand exploration and adds Expert Region Reject Sampling and Annealed Expert-Bonus Reward Shaping to ensure stability and balanced expert influence
5. Graph-Enhanced Policy Optimization (GEPO) → https://arxiv.org/abs/2510.26270
Builds a graph of an agent’s experiences to understand how different states connect, guide exploration and assign rewards more effectively
6. Information Gain-based Policy Optimization (IGPO) → Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn LLM Agents (2510.14967)
Uses the model’s own belief updates to create dense, informative feedback for smoother multi-turn learning
Read further below ⬇️
If you like this, also subscribe to the Turing post: https://www.turingpost.com/subscribe
Policy optimization (PO) algorithms are central to training AI models with preference-based feedback. In recent weeks, numerous new PO methods have emerged that build on or replace the popular PPO and GRPO, solving their issues. Here are 11 of them:
1. BAlanced Policy Optimization (BAPO) → BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping (2510.18927)
Dynamically adjusting the clipping bounds in PPO-style updates to balance positive and negative gradients and prevent entropy collapse
2. Training-Free GRPO → Training-Free Group Relative Policy Optimization (2510.08191)
Instead of using numeric rewards, it compares rollouts semantically to distill useful knowledge as a token prior, which is then applied during inference to guide the model’s behavior
3. Asymmetric Importance Sampling Policy Optimization (ASPO) → ASPO: Asymmetric Importance Sampling Policy Optimization (2510.06062)
Fixes imbalanced token weighting in LLM training. It flips the importance sampling ratios for positive tokens to correct over- and under-updates, and adds a soft dual-clipping step to keep gradients stable
4. In-Context Steered Policy Optimization (ICPO) → https://arxiv.org/abs/2510.26519
Uses a model’s own in-context learning ability to guide training with existing data. It combines Mixed-Policy GRPO with Implicit Expert Forcing to expand exploration and adds Expert Region Reject Sampling and Annealed Expert-Bonus Reward Shaping to ensure stability and balanced expert influence
5. Graph-Enhanced Policy Optimization (GEPO) → https://arxiv.org/abs/2510.26270
Builds a graph of an agent’s experiences to understand how different states connect, guide exploration and assign rewards more effectively
6. Information Gain-based Policy Optimization (IGPO) → Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn LLM Agents (2510.14967)
Uses the model’s own belief updates to create dense, informative feedback for smoother multi-turn learning
Read further below ⬇️
If you like this, also subscribe to the Turing post: https://www.turingpost.com/subscribe
- 1 reply
upvoted
an
article
3 days ago
Article
NVIDIA Isaac GR00T in LeRobot
By
and 4 others
•
•
18
upvoted
2
papers
4 days ago
liked
a
dataset
4 days ago
liked
a
model
5 days ago
liked
a
dataset
6 days ago
liked
2
models
6 days ago
