用潛在的離散代幣替換推理過程的一部分可改善大型語言模型推理

當在結構化推理軌跡上明確訓練時，LLMS已顯示出顯著改進，使其能夠求解數學方程，推斷邏輯結論並導航多步規劃任務。但是，處理這些冗長的推理軌跡所需的計算資源是很大的。這項工作引入了一種新穎的技術，將離散的潛在代幣整合到LLM推理中。

Large Language Models (LLMs) have shown remarkable improvements when explicitly trained on structured reasoning traces, enabling them to solve mathematical equations, infer logical conclusions, and perform multistep planning tasks. However, these models require significant computational resources to process lengthy reasoning traces. Researchers are actively exploring ways to enhance efficiency while maintaining the effectiveness of these models.

當對結構化推理軌跡進行明確訓練，使其能夠求解數學方程，推斷邏輯結論並執行多步計劃任務時，大型語言模型（LLM）已顯示出顯著的改進。但是，這些模型需要大量的計算資源來處理冗長的推理軌跡。研究人員正在積極探索提高效率的方法，同時保持這些模型的有效性。

One of the primary challenges in LLM reasoning is the high computational cost associated with training and inference. When models process step-by-step reasoning traces in natural language, much of the text is used to maintain coherence rather than contribute to reasoning. This leads to inefficient memory usage and increased processing time. Current methods aim to mitigate this issue by abstracting reasoning steps into compressed representations without losing critical information. However, models that attempt to internalize reasoning traces through continuous latent space or multi-stage training often perform worse than those trained with full reasoning details.

LLM推理的主要挑戰之一是與培訓和推理相關的高計算成本。當模型以自然語言處理逐步推理痕跡時，許多文本都用於保持連貫性而不是有助於推理。這導致記憶使用效率低下並增加了處理時間。當前的方法旨在通過將推理步驟抽象為壓縮表示形式，而不會丟失關鍵信息。但是，試圖通過連續的潛在空間或多階段訓練將推理痕跡內部化的模型通常比接受完整推理細節的訓練的模型差不多。

Existing solutions have focused on reducing redundancy in reasoning traces by compressing intermediate steps. Some approaches use continuous latent representations, while others involve iterative reductions of reasoning sequences. However, these methods require complex training procedures and fail to maintain performance comparable to explicit textual reasoning. Researchers sought an alternative approach that reduces computational demands while preserving reasoning capabilities. To address this, they have introduced a method that replaces parts of the reasoning process with latent discrete tokens, achieving improved efficiency without sacrificing accuracy.

現有的解決方案的重點是通過壓縮中間步驟來減少推理軌蹟的冗餘。一些方法使用連續的潛在表示，而另一些方法則涉及推理序列的迭代減少。但是，這些方法需要復雜的培訓程序，並且無法保持與明確的文本推理相當的性能。研究人員尋求一種替代方法，可以減少計算需求，同時保留推理能力。為了解決這個問題，他們引入了一種方法，該方法用潛在的離散令牌取代推理過程的一部分，從而提高了效率而不犧牲準確性。

A research team from Meta AI and UC Berkeley proposed a novel technique that integrates discrete latent tokens into LLM reasoning. They employed a vector-quantized variational autoencoder (VQ-VAE) to convert a portion of the stepwise reasoning process into compact representations. The method involves replacing early reasoning steps with latent abstractions while retaining later steps in textual form. This hybrid representation ensures the model maintains interpretability while reducing the token length of reasoning sequences. The key innovation is the randomized mixing of latent and text tokens, which enables the model to adapt seamlessly to new reasoning structures without extensive retraining.

來自Meta AI和UC Berkeley的研究團隊提出了一種新型技術，將離散的潛在代幣整合到LLM推理中。他們採用了載體定量的變分自動編碼器（VQ-VAE）將逐步推理過程的一部分轉換為緊湊的表示。該方法涉及用潛在抽象替換早期的推理步驟，同時以文本形式保留以後的步驟。這種混合表示可確保該模型在減少推理序列的令牌長度的同時保持可解釋性。關鍵創新是潛在和文本令牌的隨機混合，這使該模型能夠無縫地適應新的推理結構而無需進行大量重新訓練。

The researchers developed a training strategy incorporating latent tokens into LLM reasoning traces. During training, a controlled number of reasoning steps are replaced with their corresponding latent representations, ensuring that the model learns to interpret both abstracted and explicit reasoning structures. The randomization of latent token replacements allows adaptability across different problem types, improving the model’s generalization ability. Limiting the number of textual reasoning steps reduces input size, making LLMs more computationally efficient while maintaining reasoning performance.

研究人員制定了一種培訓策略，將潛在代幣納入LLM推理軌跡。在培訓期間，由其相應的潛在表示替換了許多受控的推理步驟，以確保模型學會解釋抽象和明確的推理結構。潛在令牌替換的隨機化允許在不同的問題類型上進行適應性，從而提高了模型的泛化能力。限製文本推理步驟的數量會減少輸入大小，從而使LLM在保持推理性能的同時更加有效。

Furthermore, the researchers ensured that the extended vocabulary, including newly introduced latent tokens, could be seamlessly integrated into the model without requiring major modifications. The proposed method demonstrated significant performance improvements across various benchmarks. The approach outperformed traditional chain-of-thought (CoT) models when applied to mathematical reasoning tasks. On the Math dataset, it achieved a 4.2% improvement over previous best-performing methods. In the GSM8K benchmark, the approach yielded a 4.1% gain, while in the Fresh-Gaokao-Math-2023 dataset, it outperformed existing models by 13.3%.

此外，研究人員確保可以將擴展的詞彙（包括新引入的潛在代幣）無縫地集成到模型中而無需進行重大修改。提出的方法顯示了各種基準的性能改善。當應用於數學推理任務時，該方法的表現超過了傳統的經營鏈（COT）模型。在數學數據集上，它比以前表現最好的方法提高了4.2％。在GSM8K基準中，該方法的增長率為4.1％，而在Fresh-Gaokao-Math-2023數據集中，它的表現優於現有模型的13.3％。

The reduction in reasoning trace length was equally noteworthy, with an average decrease of 17%, which resulted in faster inference times and lower memory consumption.

推理痕量長度的降低同樣值得注意，平均降低17％，導致推理時間更快，記憶消耗較低。

Evaluations on logical reasoning datasets such as ProntoQA and ProsQA further validated the approach’s effectiveness, with accuracy improvements of 1.2% and 18.7%, respectively. The model achieved 100% accuracy on simpler reasoning tasks, demonstrating its capacity for efficient logical deduction.

對ProntoQA和ProsQA等邏輯推理數據集的評估進一步驗證了該方法的有效性，精度提高了1.2％和18.7％。該模型在簡單的推理任務上達到了100％的精度，證明了其有效邏輯扣除的能力。

The introduction of latent tokens has provided a significant step forward in optimizing LLM reasoning without compromising accuracy. By reducing the dependence on full-text reasoning sequences and leveraging discrete latent representations, the researchers have developed an approach that maintains efficiency while improving model generalization. The hybrid structure ensures that essential reasoning components are preserved, offering a practical solution to the challenge of balancing interpretability and computational efficiency. As LLMs continue to evolve, such methods may pave the way for more resource-efficient artificial intelligence systems that retain high levels of reasoning capability.

潛在代幣的引入為優化LLM推理的前進提供了重要的一步，而不會損害準確性。通過減少對全文推理序列的依賴性並利用離散潛在表示，研究人員開發了一種方法，可以在改善模型概括的同時保持效率。混合結構確保保留基本的推理組件，從而為平衡可解釋性和計算效率的挑戰提供了實用的解決方案。隨著LLM的不斷發展，這種方法可能為保留高水平的推理能力的更多資源有效的人工智能係統鋪平了道路。