“含義機器”可視化大規模的語言模型將單詞分解為令牌並處理它們

使用大規模語言模型（例如Chatgpt，Claude和Grok）的生成AI適當地對用戶單詞響應，就像人類一樣。但是，有很大的區別

The website 'Meaning Machine' provides a visually easy-to-understand view of how large-scale language models process language, which is different from how humans process language. Generative AI using large-scale language models such as ChatGPT, Claude, and Grok respond appropriately to user words just like a human. However, there is a big difference between how large-scale language models process language and how humans process language.

網站的“含義機器”提供了視覺上易於理解的視圖，以了解大規模的語言模型處理語言，這與人類處理語言的方式不同。使用大規模語言模型（例如Chatgpt，Claude和Grok）的生成AI適當地對用戶單詞響應，就像人類一樣。但是，大規模語言模型處理語言與人類如何處理語言之間存在很大的區別。

The website 'Meaning Machine' provides a visually easy-to-understand view of how large-scale language models process language.

網站的“意義機器”提供了視覺上易於理解的視圖，以了解大規模的語言模型處理語言。

Meaning Machine · Streamlit

意思是機器·簡化

Below that, the input sentence is split into words, and each word is shown with a numeric ID when it is represented as a 'token' by the large-scale language model.

在此之下，輸入句子被分為單詞，當大規模語言模型將其表示為“令牌”時，每個單詞都會以數字ID表示。

Joshua Hathcock, the developer of Meaning Machine, explains that large-scale language models do not process entire sentences together, but rather split words and character sets into numerical IDs called tokens and process them abstractly. For example, in the case of the GPT model on which ChatGPT is based, common words such as 'The,' 'young,' 'student,' 'didn't,' and 'submit' are often represented by a single token, but rare words are split into multiple tokens made up of combinations of subwords. The large-scale language model then identifies the grammatical role of each token and infers the subject, verb, object, etc. of the sentence. In the example sentence, the subject is 'student,' the verb is 'submit,' and the object is 'report.'

意義機器的開發人員Joshua Hathcock解釋說，大規模的語言模型不會將整個句子一起處理在一起，而是將單詞和字符設置為稱為令牌的數字ID並抽像地處理它們。例如，在GPT模型的情況下，ChatGpt所基於的，例如“'''Young，''學生，“沒有”和“提交”等常見詞，通常由一個令牌表示，但是稀有的單詞分為多個令牌，由子詞組合組成。然後，大規模的語言模型識別每個令牌的語法作用，並滲透句子的主題，動詞，對像等。在示例句子中，主題是“學生”，動詞為“提交”，對象為“報告”。

The large-scale language model tags each token with its part of speech (POS), maps dependencies in a sentence, and structures and represents the sentence.

大規模的語言模型標記每個令牌及其語音的一部分（POS），句子中的地圖依賴項以及結構並代表句子。

The meaning of the dependency strings is explained in the table at the bottom of the Meaning Machine page.

依賴關係字符串的含義在含義機器頁面底部的表中說明。

Each token is then converted into a list (vector) of hundreds of numbers that capture its meaning and context. The figure below shows each token in the example sentence visualized in two dimensions through dimensionality reduction.

然後將每個令牌轉換為捕獲其含義和上下文的數百個數字的列表（向量）。下圖顯示了示例句子中通過降低維度可視化的每個令牌。

Below that is a tree showing the dependencies of each token, which shows which tokens depend on which other tokens, and what the whole picture means.

下面是一棵樹，顯示了每個令牌的依賴性，該樹顯示了哪些令牌取決於其他令牌以及整個圖片的含義。

You can navigate through the dependencies by dragging the bar at the bottom of the diagram left and right.

您可以通過在左右圖底部拖動欄來瀏覽依賴項。

In Meaning Machine, you can enter any sentence you like into the input form at the top of the page to see how the large-scale language model converts each word into a token and how it captures the dependencies of the entire sentence.

在“含義機器”中，您可以將自己喜歡的任何句子輸入頁面頂部的輸入形式中，以查看大規模語言模型如何將每個單詞轉換為令牌，以及它如何捕獲整個句子的依賴項。

'These technical steps reveal something deeper: language models don't understand language the way humans do,' Hathcock said. 'They simulate language convincingly, but in a fundamentally different way. When you or I say 'dog,' we might recall the feel of fur, the sound of a bark, and even an emotional response. But when a large-scale language model sees the word 'dog,' it sees a vector of numbers formed by the frequency with which 'dog' appears near words like 'bark,' 'tail,' 'vet,' and so on. This is not wrong; it has statistical meaning. But this has no substance, no basis, no knowledge.' In other words, large-scale language models and humans process language fundamentally differently, and no matter how human-like a response may be, there are no beliefs or goals.

哈斯科克說：“這些技術步驟揭示了更深入的問題：語言模型不像人類那樣理解語言。” ``他們令人信服地模擬語言，但根本不同。當您或我說“狗”時，我們可能會回想起皮毛的感覺，樹皮的聲音，甚至是情感反應。但是，當一個大規模的語言模型看到“狗”一詞時，它看到了一個數字向量，該數字是由“狗”出現在'bark'''''''''tail'''''''''''等頻率的頻率中。這不是錯的；它具有統計含義。但這沒有實質，沒有基礎，沒有知識。 ”換句話說，大規模的語言模型和人類從根本上處理語言，無論反應可能有多像人類，都沒有信念或目標。

Despite this, large-scale language models are already widely used in society, creating people's resumes, filtering content, and sometimes even determining what is valuable. Since AI is already becoming a social infrastructure, Hathcock argued that it is important to know the difference in performance and understanding of large-scale language models.

儘管如此，大規模的語言模型已經在社會中廣泛使用，創建了人們的簡歷，過濾內容，有時甚至確定什麼是有價值的。由於AI已經成為社會基礎設施，因此Hathcock認為，了解大型語言模型的性能和理解的差異很重要。