Scaling Laws 撞墙：互联网高质量数据耗尽后的 AI 进化危机
当人类的高质量思想被 AI 训练吞噬殆尽，模型面临着“近亲繁殖”导致的崩溃风险。Lena 与 Eli 将深度拆解合成数据、推理时计算等前沿路径，带你寻找数据荒漠后的下一条智能增长曲线。
Лучшая цитата из Scaling Laws 撞墙：互联网高质量数据耗尽后的 AI 进化危机
这种‘高质量文本荒’正迫使整个行业重新审视那个曾经坚不可摧的Scaling Laws：如果燃料烧光了，火箭还能继续飞吗？这不仅仅是技术参数的微调，而是关乎AI进化是否会陷入停滞的根本性危机。
Generated by Howard Zane
深度探讨当 AI 训练所需的互联网高质量文本数据耗尽后，模型能力是否会陷入停滞。分析 Scaling Laws 的延续性、合成数据的潜在风险（套娃效应）以及未来的技术突破方向（如推理时计算、物理世界数据）。
Часто задаваемые вопросы
模型坍塌被形象地比喻为AI的一种“遗传病”或“近亲繁殖”现象。当大语言模型在训练时不再接触新鲜的人类原创数据，而是反复读取由上一代AI生成的合成数据时，就会发生这种现象。从信息论的角度看，每一次迭代都是有损的信息变换，会导致互信息量指数级衰减。最终，模型会逐渐忘记稀有的边缘案例，变得平庸、单调，甚至出现语无伦次和知识长尾彻底消失的“痴呆”状态。
合成数据并非完全不可用，关键在于使用策略。脚本指出，纯合成数据训练会导致模型崩溃，但如果采取“混合累积”策略，将合成数据与原始的人类真实数据按比例混合，模型依然可以保持稳定。研究发现，只要训练集中保留超过70%的人类原始输入，或者通过强化学习引入人类反馈（RLHF）作为外部的低熵源，就能有效补偿信息损失，打破熵增的死循环。
推理时计算（Inference-time Scaling）是指将计算资源从训练阶段重新分配到推理阶段。与其单纯追求更大的参数量或更多的预训练数据，不如给模型更多的“思考时间”。通过引入树搜索算法（Tree Search）或优化采样策略，小尺寸模型在处理复杂任务（如数学竞赛）时，其表现甚至能超过比它大数倍的模型。这种方式利用现有权重进行深层信息挖掘，开启了AI进化的第二增长曲线。
开发者应实现从“粗放消耗”向“精准萃取”的范式转移。首先，应停止对Token数量的盲目追求，转而利用高质量分类器进行“数据分桶”，优先保证高价值数据的覆盖。其次，在垂直领域建议采用“小模型+强化学习+搜索算法”的组合。最后，对于具身智能（机器人）开发，应极度重视数据的“多样性”而非单纯的数量，因为在不同场景中采集少量演示的效果，远好于在单一场景中进行大量重复训练。
Создано выпускниками Колумбийского университета в Сан-Франциско
BeFreed объединяет глобальное сообщество любознательных людей
"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."
"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."
"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."
"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."
"Reading used to feel like a chore. Now it’s just part of my lifestyle."
"Feels effortless compared to reading. I’ve finished 6 books this month already."
"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."
"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."
"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"
"It is great for me to learn something from the book without reading it."
"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."
"Makes me feel smarter every time before going to work"
Создано выпускниками Колумбийского университета в Сан-Франциско
BeFreed объединяет глобальное сообщество любознательных людей
"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."
"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."
"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."
"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."
"Reading used to feel like a chore. Now it’s just part of my lifestyle."
"Feels effortless compared to reading. I’ve finished 6 books this month already."
"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."
"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."
"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"
"It is great for me to learn something from the book without reading it."
"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."
"Makes me feel smarter every time before going to work"
"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."
"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."
"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."
"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."
"Reading used to feel like a chore. Now it’s just part of my lifestyle."
"Feels effortless compared to reading. I’ve finished 6 books this month already."
"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."
"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."
"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"
"It is great for me to learn something from the book without reading it."
"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."
"Makes me feel smarter every time before going to work"
"Instead of endless scrolling, I just hit play on BeFreed. It saves me so much time."
"I never knew where to start with nonfiction—BeFreed’s book lists turned into podcasts gave me a clear path."
"Perfect balance between learning and entertainment. Finished ‘Thinking, Fast and Slow’ on my commute this week."
"Crazy how much I learned while walking the dog. BeFreed = small habits → big gains."
"Reading used to feel like a chore. Now it’s just part of my lifestyle."
"Feels effortless compared to reading. I’ve finished 6 books this month already."
"BeFreed turned my guilty doomscrolling into something that feels productive and inspiring."
"BeFreed turned my commute into learning time. 20-min podcasts are perfect for finishing books I never had time for."
"BeFreed replaced my podcast queue. Imagine Spotify for books — that’s it. 🙌"
"It is great for me to learn something from the book without reading it."
"The themed book list podcasts help me connect ideas across authors—like a guided audio journey."
"Makes me feel smarter every time before going to work"
