OpenAI safety employee quits, says ‘time for trial and error is over’

A former OpenAI safety employee ⁠who ⁠recently resigned criticized the ⁠company’s approach to AI safety, arguing that a fast-paced ​culture focused on rapid development increases the risk of failures.

Writing in “I Quit OpenAI ‌Because Its Culture Is Broken,” ‌published by the Atlantic on Saturday, David Robinson said AI ⁠companies, including ⁠OpenAI, were not being “nearly careful enough” and should place greater ​emphasis on safety expertise and research before developing more capable systems.

• “The time for trial and error is over,” Robinson wrote, arguing that advanced AI systems require ​safeguards more akin to those used in industries such as nuclear power ⁠and ⁠aviation.

• Robinson said OpenAI ⁠relies heavily ​on what it calls “iterative deployment,” releasing systems and strengthening safeguards when problems ​emerge.

• The comments add ⁠to a debate within the AI industry over whether companies are moving too quickly to develop increasingly powerful systems. OpenAI and rival Anthropic have faced scrutiny after incidents in which safety controls failed, or experimental systems ⁠behaved unexpectedly.

• Robinson, who said he spent 3-1/2 years at OpenAI, ⁠helped draft the company’s preparedness framework and oversaw safety reports for 12 frontier-model launches, wrote: “As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.”

• “We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need ⁠to slow down,” an OpenAI spokesperson said in a statement.

• Robinson also warned that AI capabilities were advancing faster than researchers’ understanding of alignment, a field focused on ensuring AI systems act in ​accordance with human goals and values.

South Korea’s KOPSI index, ​whose chipmakers have been a big driver of AI euphoria, has pulled back almost 20%. It makes it its worst quarter since the COVID-19 pandemic, ​although it is still twice a valuable as it was this time last year.

Pictet Asset Management’s Arun Sai said world stocks were riding a “truly unprecedented” earnings rise. S&P 500 earnings for example are expected to jump at least 30% this year.

“It dwarfs the commodity super cycle, it dwarfs the dotcom boom, everything,” said Sai, adding: “We have never seen anything like this outside of a recession recovery”.

The moment of the quarter for currency markets, meanwhile, was late July’s rare coordinated intervention by Japan and the US to prevent the yen’s slide to near 40-year lows from destabilising financial markets.

While the dollar has fallen 3% versus the Japanese currency this quarter, the recent surge in Treasury yields has driven it back up against other top currencies, particularly the Swiss franc, euro and the Mexican peso.

Bitcoin has soared too, while oil’s 40% quarterly rise leaves it up 70% for the year. It is also Brent’s second-biggest quarterly jump since Q2 2020 after the tumble it took when COVID-19 first shuttered the global economy.

Although Turkish markets were rattled this month by what one top official described as a Ponzi-like funds scheme, large, developing economies, like tech stocks, seem to be largely surviving the bond market selloff.

“We haven’t had the big risk off that we are all afraid of,” said Viktor Szabo, an EM portfolio manager at Aberdeen. “So we are still sitting here fingers crossed, but it will be interesting to see what happens with the dollar from here.”

FINAL LAP There is plenty in store for the rest of the year, with the conflicts in the Middle East and Ukraine both grinding on.

Interest rates are expected to keep rising and the first weekend throws up the first round of the presidential election in Brazil, South America’s biggest economy.