Large Concept Models (LCMs) Offer Some Exciting Prospects

In today's column, I explore an intriguing new advancement for generative AI and large language models (LLMs) consisting of moving beyond contemporary words

Large concept models (LCMs) offer some exciting prospects. In today’s column, I explore an intriguing new advancement for generative AI and large language models (LLMs) consisting of moving beyond contemporary words-based approaches to sentence-oriented approaches.

The extraordinary deal is this. You might be vaguely aware that most LLMs currently focus on words and accordingly generate responses on a word-at-a-time basis. Suppose that instead of looking at the world via individual words, we could use sentences as a core element. Whole sentences come into AI, and complete sentences are generated out of AI.

To do this, the twist is that sentences are reducible to underlying concepts, and those computationally ferreted-out concepts become the esteemed coinage of the realm for this groundbreaking architectural upheaval of conventional generative AI and LLMs. The new angle radically becomes that we then design, build, and field so-called large concept models (LCMs) in lieu of old-fashioned large language models.

Let’s talk about it.

This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here). For my coverage of the top-of-the-line OpenAI ChatGPT o1 and o3 models and their advanced reasoning functionality, see the link here and the link here.

There is an ongoing concern in the AI community that perhaps AI researchers and AI developers are treading too much of the same ground right now. We seem to have landed on an impressive architecture contrivance for how to shape generative AI and LLMs and few want to depart from the success so far attained.

If it isn’t broken, don’t fix it.

The problem is that not everyone concurs that the prevailing architecture isn’t actually broken. By broken — and to quickly clarify, the issue is more of limitations and constraints than it is one of something inherently being wrong. A strong and vocal viewpoint is that we are hitting the topmost thresholds of what contemporary LLMs can accomplish. There isn’t much left in the gas tank, and we are soon to hit a veritable wall.

As such, there are brave souls who are seeking alternative architectural avenues. Exciting but a gamble at the same time. They might hit the jackpot and discover the next level of AI. Fame and fortune await. On the other hand, they might waste time on a complete dead-end. Smarmy cynics will call them foolish for their foolhardy ambitions. It could harm your AI career and knock you out of getting that sweet AI high-tech freewheeling job you’ve been eyeing for the longest time.

I continue to give airtime to those who are heads-down seriously aiming to upset the apple cart. For example, my analysis of the clever chain-of-continuous thought approach for LLMs merits dutiful consideration, see the link here. Another exciting possibility is the neuro-symbolic or hybrid AI approach that marries artificial neural networks (ANNs) with rules-based reasoning, see my discussion at the link here.

There is no doubt in my mind that a better mousetrap is still to be found, and all legitimate new-world explorers should keep sailing the winds of change. May your voyage be fruitful.

The approach I’ll be identifying this time around has to do with the existing preoccupation with words.

Actually, it might be more appropriate to say a preoccupation with tokens. When you enter words into a prompt, those words are converted into numeric values referred to as tokens. The rest of the AI processing computationally crunches on those numeric values or tokens, see my detailed description of how this works at the link here. Ultimately, the AI-generated response is in token format and must be converted back into text so that you get a readable answer.

In a sense, you give words to AI, and the AI gives you words in return (albeit via the means of tokenization).

Do we have to do things that way?

No, there doesn’t seem to be a fundamental irrefutable law of nature that says we must confine ourselves to a word-at-a-time focus. Feel free to consider alternatives. Let your wild thoughts flow.

Here is an idea. Imagine that whole sentences were the unit of interest. Rather than parsing and aiming at single words, we conceive of a sentence as our primary unit of measure. A sentence is admittedly a collection of words. No disagreement there. The gist is that the sentence is seen as a sentence. Right now, a sentence happens to be treated as a string of words.

Give the AI a sentence, and you get back a generated sentence in return.

Boom, drop the mic.

Making sense of sentences is a bit of a head-scratcher. How do you look at an entire sentence and identify what the meaning or significance of the sentence is?

Aha, let’s assume that sentences are representative of concepts. Each sentence will