Last week, after briefly deposed CEO Sam Altman was reinstalled at OpenAI, two reports claimed that a top-secret project at the company had rattled some researchers there with its potential to solve intractable problems in a powerful new way.
“Given vast computing resources, the new model was able to solve certain mathematical problems,” Reuters reported, citing a single unnamed source. “Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success.” The Information said that Q* was seen as a breakthrough that would lead to “far more powerful artificial intelligence models,” adding that “the pace of development alarmed some researchers focused on AI safety,” citing a single unnamed source.
Reuters also reported that some researchers sent a letter expressing concerns about Q*’s potential power to the nonprofit board that ejected Altman, although a WIRED source familiar with the board’s thinking says that was not the case. And perhaps in part thanks to its conspiracy-evoking name, speculation about Q* surged over the Thanksgiving weekend, building a fearsome reputation for a project about which we know next to nothing. Altman himself appeared to confirm the existence of the project when asked about Q* in an interview with the Verge yesterday, saying “No particular comment on that unfortunate leak.”
What could Q* be? Combining a close read of the initial reports with consideration of the hottest problems in AI right now suggests it may be related to a project that OpenAI announced in May, claiming powerful new results from a technique called “process supervision.”
The project involved Ilya Sutskever, OpenAI’s chief scientist and cofounder, who helped oust Altman but later recanted—The Information says he led work on Q*. The work from May was focused on reducing the logical slipups made by large language models (LLMs). Process supervision, which involves training an AI model to break down the steps needed to solve a problem, can improve an algorithm’s chances of getting the right answer. The project showed how this could help LLMs, which often make simple errors on elementary math questions, tackle such problems more effectively.
Andrew Ng, a Stanford University professor who led AI labs at both Google and Baidu and who introduced many people to machine learning through his classes on Coursera, says that improving large language models is the next logical step in making them more useful. “LLMs are not that good at math, but neither are humans,” Ng says. “However, if you give me a pen and paper, then I’m much better at multiplication, and I think it’s actually not that hard to fine-tune an LLM with memory to be able to go through the algorithm for multiplication.”
Read the full article here
