Near Protocol Unveils Ambitious Plan to Build the World's Largest Open-Source AI Model

The 1.4 trillion parameter model would be 3.5 times bigger than Meta's current open-source Llama model.

Layer-1 blockchain Near Protocol (NEAR) has announced plans to build the world’s largest open-source artificial intelligence (AI) model during the opening day of its Redacted conference in Bangkok, Thailand.

The 1.4 trillion parameter model will be 3.5 times larger than Meta’s (META) current open-source Llama model. It will be created through competitive crowdsourced research and development (R&D) from thousands of contributors on the new Near AI Research hub.

Participants will be able to join the training of a small 500 million parameter model starting on Nov. 10. Near’s AI models will grow in size and sophistication across seven models. Only the best contributors will make the leap to working on progressively more complex and larger models.

The models will be monetized, and privacy will be preserved via the use of encrypted Trusted Execution Environments (TEE) to reward contributors and encourage constant updating as the technology progresses.

The project will be funded with token sales, Near co-founder Illia Polosukhin told Cointelegraph at the Redacted conference in Bangkok.

“It costs about $160 million, so it’s a lot, obviously, but actually, in crypto, it is raiseable money,” he said. Near is one of the few crypto projects with the ability to realize such an ambitious undertaking: Polosukhin was one of the authors of the groundbreaking transformer research paper that led to ChatGPT, and co-founder Alex Skidanov worked at OpenAI in the lead-up to the era-defining model’s release in late 2022.

Skidanov, who now heads up Near AI, conceded that training a model of this size and complexity would be a massive undertaking with a big hurdle to overcome.

“To train a model of this size, you’d need tens of thousands of GPUs in one place, which wouldn’t be ideal. But to use a decentralized network of compute you’d need a new technology that doesn’t exist today because all the distributed training techniques we have require very fast interconnect,” he said.

However, Skidanov added that emerging research from Deep Mind suggests it is possible.

“There is some emerging research from Deep Mind that shows you can actually do distributed training over slow interconnect with a technique called pipeline parallelism,” he said.

Polosukhin said he hasn’t spoken with existing projects like the Artificial Superintelligence Alliance but would be happy to see if there are synergies.

“Whatever happens, decentralized AI technology must win for all our sake. We had Edward Snowden as a guest speaker at the conference, and he painted a scary portrait of centralized AI turning the world into a giant surveillance state,” he said.

“This is probably the most important technology right now and probably in the future. And the reality is, if AI is controlled by one company, we effectively are going to do whatever that company says,” Snowden explained.