Scaling Laws 撞墙：互联网高质量数据耗尽后的 AI 进化危机

8分

2026年6月5日

Technology

当人类的高质量思想被 AI 训练吞噬殆尽，模型面临着“近亲繁殖”导致的崩溃风险。Lena 与 Eli 将深度拆解合成数据、推理时计算等前沿路径，带你寻找数据荒漠后的下一条智能增长曲线。

Scaling Laws 撞墙：互联网高质量数据耗尽后的 AI 进化危机のベスト引用

这种‘高质量文本荒’正迫使整个行业重新审视那个曾经坚不可摧的Scaling Laws：如果燃料烧光了，火箭还能继续飞吗？这不仅仅是技术参数的微调，而是关乎AI进化是否会陷入停滞的根本性危机。

Generated by Howard Zane

質問を入力

深度探讨当 AI 训练所需的互联网高质量文本数据耗尽后，模型能力是否会陷入停滞。分析 Scaling Laws 的延续性、合成数据的潜在风险（套娃效应）以及未来的技术突破方向（如推理时计算、物理世界数据）。

ホストの声

Lena

知識ソース

Prescriptive Scaling Laws for Data Constrained Training

https://arxiv.org/html/2605.01640v1

Will we run out of data to train large language models? | Epoch AI

https://epoch.ai/publications/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data

AI models collapse when trained on recursively generated data

https://www.nature.com/articles/s41586-024-07566-y

Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training

https://arxiv.org/pdf/2602.16065

The Era of the ‘Thinking’ Machine: How Inference-Time Compute is Rewriting the AI Scaling Laws | FinancialContent

https://www.financialcontent.com/article/tokenring-2026-1-28-the-era-of-the-thinking-machine-how-inference-time-compute-is-rewriting-the-ai-scaling-laws

The limiting factor in physical AI isn't compute or architecture - it's data - DEV Community

https://dev.to/yks/the-limiting-factor-in-physical-ai-isnt-compute-or-architecture-its-data-4lbp

よくある質問

模型坍塌被形象地比喻为AI的一种“遗传病”或“近亲繁殖”现象。当大语言模型在训练时不再接触新鲜的人类原创数据，而是反复读取由上一代AI生成的合成数据时，就会发生这种现象。从信息论的角度看，每一次迭代都是有损的信息变换，会导致互信息量指数级衰减。最终，模型会逐渐忘记稀有的边缘案例，变得平庸、单调，甚至出现语无伦次和知识长尾彻底消失的“痴呆”状态。

合成数据并非完全不可用，关键在于使用策略。脚本指出，纯合成数据训练会导致模型崩溃，但如果采取“混合累积”策略，将合成数据与原始的人类真实数据按比例混合，模型依然可以保持稳定。研究发现，只要训练集中保留超过70%的人类原始输入，或者通过强化学习引入人类反馈（RLHF）作为外部的低熵源，就能有效补偿信息损失，打破熵增的死循环。

推理时计算（Inference-time Scaling）是指将计算资源从训练阶段重新分配到推理阶段。与其单纯追求更大的参数量或更多的预训练数据，不如给模型更多的“思考时间”。通过引入树搜索算法（Tree Search）或优化采样策略，小尺寸模型在处理复杂任务（如数学竞赛）时，其表现甚至能超过比它大数倍的模型。这种方式利用现有权重进行深层信息挖掘，开启了AI进化的第二增长曲线。

开发者应实现从“粗放消耗”向“精准萃取”的范式转移。首先，应停止对Token数量的盲目追求，转而利用高质量分类器进行“数据分桶”，优先保证高价值数据的覆盖。其次，在垂直领域建议采用“小模型+强化学习+搜索算法”的组合。最后，对于具身智能（机器人）开发，应极度重视数据的“多样性”而非单纯的数量，因为在不同场景中采集少量演示的效果，远好于在单一场景中进行大量重复训练。

コロンビア大学卒業生が開発サンフランシスコ発

BeFreedは好奇心旺盛な仲間が集うグローバルコミュニティ

4.7

平均評価

7,840件以上のアプリ評価

BeFreed Community

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA

117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum

108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC

254

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful

4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon

201

"It is great for me to learn something from the book without reading it."

@OojasSalunke

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn

483

"Makes me feel smarter every time before going to work"

@Cashflowbubu

BeFreedがウェブ上でどのように話題になっているかをもっと見る

コロンビア大学卒業生が開発サンフランシスコ発

BeFreedは好奇心旺盛な仲間が集うグローバルコミュニティ

4.7

平均評価

7,840件以上のアプリ評価

BeFreed Community

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA

117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum

108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC

254

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful

4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon

201

"It is great for me to learn something from the book without reading it."

@OojasSalunke

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn

483

"Makes me feel smarter every time before going to work"

@Cashflowbubu

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA

117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum

108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC

254

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful

4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon

201

"It is great for me to learn something from the book without reading it."

@OojasSalunke

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn

483

"Makes me feel smarter every time before going to work"

@Cashflowbubu

"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."

@Moemenn

"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."

@Chloe, Solo founder, LA

117

"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."

@Raaaaaachelw

"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."

@Matt, YC alum

108

"Reading used to feel like a chore. Now it’s just part of my lifestyle."

@Erin, Investment Banking Associate , NYC

254

"Feels effortless compared to reading. I’ve finished 6 books this month already."

@djmikemoore

"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."

@Pitiful

4.5K

"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."

@SofiaP

"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"

@Jaded_Falcon

201

"It is great for me to learn something from the book without reading it."

@OojasSalunke

"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."

@Leo, Law Student, UPenn

483

"Makes me feel smarter every time before going to work"

@Cashflowbubu

BeFreedがウェブ上でどのように話題になっているかをもっと見る

7,840件以上のアプリ評価4.7

今すぐ学習の旅を始めよう

Scaling Laws 撞墙：互联网高质量数据耗尽后的 AI 进化危机

8分

2026年6月5日

Technology

Scaling Laws 撞墙：互联网高质量数据耗尽后的 AI 进化危机のベスト引用

这种‘高质量文本荒’正迫使整个行业重新审视那个曾经坚不可摧的Scaling Laws：如果燃料烧光了，火箭还能继续飞吗？这不仅仅是技术参数的微调，而是关乎AI进化是否会陷入停滞的根本性危机。