PatchScopes：大語模型神經元的手術（LLMS）

大型語言模型（LLM）徹底改變了人工智能領域，表現出了自然語言理解和產生的顯著能力。這些模型由相互聯繫的人工神經元層組成，通過稱為隱藏表示形式的數字向量進行通信。但是，在這些隱藏表示形式中編碼的含義是一個重大挑戰。機器學習能力的領域旨在彌合這一差距，而Google研究人員提出了一種了解LLM“思考”的方法。

Large Language Models (LLMs) have revolutionized the field of artificial intelligence, demonstrating remarkable capabilities in natural language understanding and generation. These models, comprised of layers of interconnected artificial neurons, communicate through vectors of numbers known as hidden representations. However, deciphering the meaning encoded within these hidden representations has been a significant challenge. The field of machine learning interpretability seeks to bridge this gap, and "Patchscopes" that Google researchers came up with a method to understand what LLM “thinks”.

大型語言模型（LLM）徹底改變了人工智能領域，表現出了自然語言理解和產生的顯著能力。這些模型由相互聯繫的人工神經元層組成，通過稱為隱藏表示形式的數字向量進行通信。但是，在這些隱藏表示形式中編碼的含義是一個重大挑戰。機器學習能力的領域旨在彌合這一差距，而Google研究人員提出了一種了解LLM“思考”的方法。

Patchscopes is a novel interpretability method that enables researchers to perform "surgery" on the neurons of an LLM. This involves cutting out and replacing hidden representations between different prompts and layers, allowing for a detailed inspection of the information contained within. The core concept is the "inspection prompt," which acts as a lens into the LLM's mind, facilitating the extraction of human-interpretable meaning. The framework leverages the inherent ability of LLMs to translate their own hidden representations into understandable text.

PatchScopes是一種新型的可解釋性方法，使研究人員能夠對LLM的神經元進行“手術”。這涉及切割和替換不同提示和層之間的隱藏表示形式，從而詳細檢查了其中包含的信息。核心概念是“檢查提示”，它充當LLM腦海中的鏡頭，促進了人類解動意義的提取。該框架利用LLM將其隱藏表示形式轉化為可理解的文本的固有能力。

Understanding the Transformer Architecture: A Foundation for Patchscopes

了解變壓器體系結構：PatchScopes的基礎

Patchscopes builds upon a deep understanding of LLMs and the transformer architecture, which forms the backbone of many modern language models. Transformer models process text by first tokenizing the input, breaking it down into smaller units (words or sub-words). Each token is then embedded into a high-dimensional vector space, creating an initial hidden representation.

PatchScopes建立在對LLM和Transformer架構的深入了解的基礎上，後者構成了許多現代語言模型的骨幹。變壓器模型首先對輸入進行處理，將其分解為較小的單元（單詞或子字）。然後將每個令牌嵌入高維矢量空間中，從而創建一個初始的隱藏表示形式。

The transformer architecture consists of multiple layers of transformer blocks. Each layer refines the hidden representation based on the output of the preceding layer and the relationships between tokens in the input sequence. This process continues through the final layer, where the hidden representation is used to generate the output text. Decoder-only models, which are the focus of Patchscopes, only consider preceding tokens when generating the next token, making them particularly well-suited for language generation tasks.

變壓器架構由多個變壓器塊組成。每一層都根據前面一層的輸出以及輸入序列中的令牌之間的關係來完善隱藏的表示。此過程繼續通過最後一層，其中隱藏的表示形式用於生成輸出文本。僅解碼器模型是PatchScopes的重點，僅在生成下一個令牌時考慮之前的令牌，這使其特別適合語言生成任務。

The Patchscopes framework operates on a simple yet powerful premise: LLMs possess the inherent ability to translate their own hidden representations into human-understandable text. By patching hidden representations between different locations during inference, researchers can inspect the information within a hidden representation, understand LLM behavior, and even augment the model's performance.

PatchScopes框架以簡單而強大的前提運行：LLM具有將自己的隱藏表示形式轉化為人類可行的文本的固有能力。通過在推理期間修補不同位置之間的隱藏表示形式，研究人員可以檢查隱藏表示形式中的信息，了解LLM行為，甚至增強模型的性能。

The process involves several key steps:

該過程涉及多個關鍵步驟：

Source Prompt: A source prompt is fed into the LLM, generating hidden representations at each layer. This prompt serves as the context from which information will be extracted.

源提示：源提示符被饋入LLM，在每一層中生成隱藏的表示形式。該提示是從中提取信息的上下文。

Inspection Prompt: An inspection prompt is designed to elicit a specific type of information from the LLM. This prompt typically includes a placeholder token where the hidden representation from the source prompt will be inserted.

檢查提示：檢查提示旨在從LLM中獲取特定類型的信息。該提示通常包括一個佔位符令牌，其中將插入源提示中隱藏的表示。

Patching: The hidden representation from a specific layer and token position in the source prompt is "patched" into the placeholder token in the inspection prompt. This effectively replaces the LLM's internal representation with the extracted information.

修補：在檢查提示符中，將源提示符中特定層和令牌位置的隱藏表示形式“修補”。這有效地用提取的信息有效地取代了LLM的內部表示。

Generation: The LLM continues decoding from the patched inspection prompt, generating text based on the combined information from the source and inspection prompts.

一代：LLM繼續從修補的檢查提示符中解碼，並根據來自源和檢查提示的組合信息生成文本。

Analysis: The generated text is analyzed to understand the information encoded in the hidden representation. This can involve evaluating the accuracy of factual information, identifying the concepts captured by the representation, or assessing the model's reasoning process.

分析：分析生成的文本以了解隱藏表示形式中編碼的信息。這可能涉及評估事實信息的準確性，識別代表捕獲的概念或評估模型的推理過程。

Case Study 1: Entity Resolution

案例研究1：實體解決

The first case study explores how LLMs resolve entities (people, places, movies, etc.) across different layers of the model. The goal is to understand at what point the model associates a token with its correct meaning. For example, how does the model determine that "Diana" refers to "Princess Diana" rather than the generic name?

第一個案例研究探討了LLM如何解決模型不同層的實體（人，地方，電影等）。目的是了解模型在什麼時候將令牌與正確的含義相關聯。例如，模型如何確定“戴安娜”是指“戴安娜公主”而不是通用名稱？

To investigate this, a source prompt containing the entity name is fed into the LLM. The hidden representation of the entity token is extracted at each layer and patched into an inspection prompt designed to elicit a description of the entity. By analyzing the generated descriptions, researchers can determine when the model has successfully resolved the entity.

為了進行調查，包含實體名稱的源提示將被送入LLM。實體令牌的隱藏表示形式在每一層中都提取，並將其修補成一個檢查提示，旨在引起對實體的描述。通過分析生成的描述，研究人員可以確定模型何時成功解決了該實體。

The results of this case study suggest that entity resolution typically occurs in the early layers of the model (before layer 20). This aligns with theories about layer function, which posit that early layers are responsible for establishing context from the prompt. The study also reveals that tokenization (how the input text is broken down into tokens) has a significant impact on how the model navigates its embedding space.

該案例研究的結果表明，實體分辨率通常發生在模型的早期層（第20層之前）。這與關於層函數的理論保持一致，該理論認為早期層是從提示中建立上下文的原因。該研究還表明，令牌化（輸入文本如何分解為令牌）對模型如何導航其嵌入空間有重大影響。

Case Study 2: Attribute Extraction

案例研究2：屬性提取

The second case study focuses on evaluating how accurately the model's hidden representation captures well-known concepts and their attributes. For example, can the model identify that the largest city in Spain is Madrid?

第二個案例研究的重點是評估模型的隱藏表示形式的準確程度捕獲了知名概念及其屬性。例如，該模型可以確定西班牙最大的城市是馬德里嗎？

To extract an attribute, a source prompt containing the subject (e.g., "Spain") is fed into the LLM. The hidden representation of the subject token is extracted and patched into an inspection prompt designed to elicit the specific attribute (e.g., "The largest city is x"). By analyzing the generated text, researchers can determine whether the model correctly identifies the attribute.

為了提取屬性，將包含受試者（例如“西班牙”）的源提示送入LLM。將主題令牌的隱藏表示形式提取並修補為旨在引起特定屬性的檢查提示（例如，“最大的城市是x”）。通過分析生成的文本，研究人員可以確定模型是否正確識別屬性。

This case study compares Patchscopes to a technique called "probing," which involves training a classifier to predict an attribute from a hidden representation. Unlike probing, Patchscopes does not

該案例研究將PatchScopes與一種稱為“探測”的技術進行了比較，該技術涉及訓練分類器以預測隱藏表示形式的屬性。與探測不同，PatchScopes沒有