Youshua Bengio launches $30M lab to rethink AI design

Machine learning pioneer Yoshua Bengio is launching a new nonprofit lab backed by roughly $30 million in funding to make AI systems act less like humans.Why it matters: While the move bucks a trend toward AI that acts independently, Bengio and others argue the current approach risks creating systems that may pursue their own self-preservation at the expense of humanity."We've been getting inspiration from humans as the template for building intelligent machines, but that's crazy, right?" Bengio said in an interview."If we continue on this path, that means we're going to be creating entities — like us — that don't want to die, and that may be smarter than us and that we're not sure if they're going to behave according to our norms and our instructions," he said.Driving the news: Bengio, a Montreal-based researcher who has long warned about the risks of a technology he helped develop, has raised about $30 million for the nonprofit, dubbed LawZero.LawZero currently has about 15 staffers, Bengio said, "but the goal is to hire many more."Bengio is among those who have called for tougher regulation of AI development and even the breakup of big tech companies. Earlier this year he gave a TED Talk urging greater caution and collaboration.The big picture: There's a growing sense of worry among critics — and even AI practitioners — that safety is taking a back seat as companies and countries race to be first with AI that can best humans in a wide variety of tasks, so-called artificial general intelligence (AGI).Bengio said there is also a high risk in concentrating control of advanced AI in a handful of companies."You don't want AGI or superintelligence to be in the hands of one person or one company only deciding what to do, or even one government," Bengio said. "So you need very strong checks and balances."Between the lines: Bengio says a large part of the problem is how current systems are trained. During initial training, the systems are taught to mimic humans and then they're honed by seeing which responses people find most appealing. "Both of these give rise to uncontrolled agency," Bengio said.Some early glimmers of this are already appearing, such as Anthropic's latest model which, in a test scenario, sought to blackmail its engineers to avoid shutdown.By contrast, Bengio says he wants to create AI systems that have intellectual distance from humans and act as more of a detached scientist than a personal companion or human agent."The training principle is completely different, but it can exploit a lot of the recent advances that have happened in machine learning," he said.Yes, but: Bengio told Axios that the $30 million should be enough to fund the basic research effort for about 18 months.But AI is expensive, and as a non-profit, it may be difficult to raise additional, larger funding rounds — as OpenAI and others have found.Bengio says he doesn't see future funding as a roadblock, as he's certain more investors now understand the plausible risks ahead.Governments could also be future backers of LawZero, Bengio said.

Comments

World news

Tigers maul man to death after he jumps into zoo enclosure

Express

German leader is meeting Trump for the first time, so will it be another Starmer love-in, or a Zelenskyy-style brush off?

about 5 hours ago

President Trump suspends entry to Harvard's international students

Axios

Comments

World news

Tigers maul man to death after he jumps into zoo enclosure

Ukraine LIVE: Huge Russian drone strike kills toddler in devastating Pryluky attack

Madeleine McCann LIVE: Major search update as man who knew suspect makes huge claim

Russia economy in meltdown as Moscow rocked by record-high 21% interest rates

Operation Spiderweb: Satellite images show Russian bombers destroyed in massive Ukrainian drone attack

Dead Sea Scroll breakthrough as scientists find proof behind ancient manuscripts

Attack on Russian nuclear bombers captured in new Ukraine drone footage

Trump escalates vendetta against Harvard with fresh bid to ban foreign students

Karoline Levitt ditches religious necklace after Jon Stewart’s brutal joke: ‘Some sort of weird Pinocchio cross’

Election of Mexico's first Indigenous Supreme Court justice in 170 years raises hope and skepticism

Should I hold my cash in a stocks and shares ISA – or is a cash ISA better?

House Democrats question Trump in probe of Elon Musk's alleged drug use

Sheffield crash: Boy, 16, dies in 'hit-and-run smash' involving Audi and electric bike

Draisaitl scores in OT, Oilers beat the Panthers in overtime in Game 1 of the Stanley Cup Final

Ros Atkins on... The White House claims on BBC Gaza coverage

What we know about Trump's latest travel ban

Burundi's ruling party seeks to tighten grip on power

Watch: Hungry elephant wanders into Thai grocery store

Trump suspends foreign student visas at Harvard

Trump Pushes to Restrict Harvard’s International Students From Entering U.S.

Friedrich Merz, Germany’s Chancellor, Heads to Meet Trump at White House

Who Is Johnnie Moore? Here’s What to Know About a Gaza Aid Group’s Chairman.

Ukraine-Russia war latest: Trump warns no ‘immediate peace’ after Putin call following Operation Spiderweb

Trump signs travel ban targeting 12 countries with 'hostile attitudes' to the US

"Ridiculous and false": Biden hits back at Trump over autopen investigation

Kirk drives in winning run in Jays’ walkoff win

Kirk delivers walkoff hit as Jays edge Phillies

Reason Brit 'drug mules' are 'flavour of the month' for criminal gangs is exposed

Psychic dubbed 'Japan's Baba Vanga' issues terrifying prediction for 2030

North Korea's Kim says he'll 'unconditionally support' Russia's war against Ukraine

Paramedic who survived Israeli attack that killed 15 aid workers describes alleged torture in custody

German leader is meeting Trump for the first time, so will it be another Starmer love-in, or a Zelenskyy-style brush off?

President Trump suspends entry to Harvard's international students

Trump orders investigation into Biden

After otter pup’s death, Vancouver Island dog owners urged to control their pets

Alberta premier says she’s invited Prime Minister Carney to Oilers playoff game

Family of B.C. Mountie who took own life angry officer she was involved with now with SPS

Alberta premier says she sees a ‘breakthrough’ in talks with Americans on energy

Valour FC gets out of basement in front of record-breaking crowd

Separatist Alberta Republican Party cries foul after byelection debate is cancelled

Vancouver to offer up land in bid to move SROs from Granville Entertainment District

London Knights hand out team awards to end 2024-2025 season

Trump issues new travel ban covering a dozen countries

Calgary may speed up building new neighbourhoods if it doesn’t require upfront costs

30-year-old worker drowns in industrial accident at B.C. mill

Confusion surrounds Vancouver’s Lapu Lapu ‘memorial event’

New president, new pressures put South Korea-U.S. alliance at crossroads

Data capturing hot spots and burned acres show Canada wildfire season off to wild start

NFL long snapper causes commotion in church after interrupting service to demand answers over porn claims

Biden’s frailty has Democrats scrambling to rebrand - or jump ship like Karine Jean-Pierre

Trump revives travel ban and restricts 19 countries

Trump orders investigation into Biden presidency claiming aides hid his ‘serious cognitive decline’

Diddy dangled Cassie Ventura’s friend off 17th-floor apartment balcony, jury told

Migrant deported to Mexico is returned to US by Trump administration after order by judge

'Hear how it's roaring?': Men of Weyakwin make a stubborn stand against Saskatchewan wildfires

China reveals first ever details of nuclear weapon 200x more powerful than Hiroshima bomb with huge 7,500-mile range

Hundreds of thousands in Gaza 'catastrophically food insecure'

Putin is playing the victim to get the US back onside

Putin vowed revenge against Ukraine for drone strike on airbases, Trump says

Trump signs travel ban on 12 countries and restrictions on seven more