Youshua Bengio launches $30M lab to rethink AI design

Machine learning pioneer Yoshua Bengio is launching a new nonprofit lab backed by roughly $30 million in funding to make AI systems act less like humans.Why it matters: While the move bucks a trend toward AI that acts independently, Bengio and others argue the current approach risks creating systems that may pursue their own self-preservation at the expense of humanity."We've been getting inspiration from humans as the template for building intelligent machines, but that's crazy, right?" Bengio said in an interview."If we continue on this path, that means we're going to be creating entities — like us — that don't want to die, and that may be smarter than us and that we're not sure if they're going to behave according to our norms and our instructions," he said.Driving the news: Bengio, a Montreal-based researcher who has long warned about the risks of a technology he helped develop, has raised about $30 million for the nonprofit, dubbed LawZero.LawZero currently has about 15 staffers, Bengio said, "but the goal is to hire many more."Bengio is among those who have called for tougher regulation of AI development and even the breakup of big tech companies. Earlier this year he gave a TED Talk urging greater caution and collaboration.The big picture: There's a growing sense of worry among critics — and even AI practitioners — that safety is taking a back seat as companies and countries race to be first with AI that can best humans in a wide variety of tasks, so-called artificial general intelligence (AGI).Bengio said there is also a high risk in concentrating control of advanced AI in a handful of companies."You don't want AGI or superintelligence to be in the hands of one person or one company only deciding what to do, or even one government," Bengio said. "So you need very strong checks and balances."Between the lines: Bengio says a large part of the problem is how current systems are trained. During initial training, the systems are taught to mimic humans and then they're honed by seeing which responses people find most appealing. "Both of these give rise to uncontrolled agency," Bengio said.Some early glimmers of this are already appearing, such as Anthropic's latest model which, in a test scenario, sought to blackmail its engineers to avoid shutdown.By contrast, Bengio says he wants to create AI systems that have intellectual distance from humans and act as more of a detached scientist than a personal companion or human agent."The training principle is completely different, but it can exploit a lot of the recent advances that have happened in machine learning," he said.Yes, but: Bengio told Axios that the $30 million should be enough to fund the basic research effort for about 18 months.But AI is expensive, and as a non-profit, it may be difficult to raise additional, larger funding rounds — as OpenAI and others have found.Bengio says he doesn't see future funding as a roadblock, as he's certain more investors now understand the plausible risks ahead.Governments could also be future backers of LawZero, Bengio said.

Comments

World news

Ukraine-Russia war latest: Trump warns no ‘immediate peace’ after Putin call following Operation Spiderweb

Independent

German leader is meeting Trump for the first time, so will it be another Starmer love-in, or a Zelenskyy-style brush off?

about 4 hours ago

President Trump suspends entry to Harvard's international students

Axios

Comments

World news

Ukraine-Russia war latest: Trump warns no ‘immediate peace’ after Putin call following Operation Spiderweb

Trump signs travel ban targeting 12 countries with 'hostile attitudes' to the US

"Ridiculous and false": Biden hits back at Trump over autopen investigation

Kirk drives in winning run in Jays’ walkoff win

Kirk delivers walkoff hit as Jays edge Phillies

Reason Brit 'drug mules' are 'flavour of the month' for criminal gangs is exposed

Psychic dubbed 'Japan's Baba Vanga' issues terrifying prediction for 2030

North Korea's Kim says he'll 'unconditionally support' Russia's war against Ukraine

Paramedic who survived Israeli attack that killed 15 aid workers describes alleged torture in custody

German leader is meeting Trump for the first time, so will it be another Starmer love-in, or a Zelenskyy-style brush off?

President Trump suspends entry to Harvard's international students

Trump orders investigation into Biden

After otter pup’s death, Vancouver Island dog owners urged to control their pets

Alberta premier says she’s invited Prime Minister Carney to Oilers playoff game

Family of B.C. Mountie who took own life angry officer she was involved with now with SPS

Alberta premier says she sees a ‘breakthrough’ in talks with Americans on energy

Valour FC gets out of basement in front of record-breaking crowd

Separatist Alberta Republican Party cries foul after byelection debate is cancelled

Vancouver to offer up land in bid to move SROs from Granville Entertainment District

London Knights hand out team awards to end 2024-2025 season

Trump issues new travel ban covering a dozen countries

Calgary may speed up building new neighbourhoods if it doesn’t require upfront costs

30-year-old worker drowns in industrial accident at B.C. mill

Confusion surrounds Vancouver’s Lapu Lapu ‘memorial event’

New president, new pressures put South Korea-U.S. alliance at crossroads

Data capturing hot spots and burned acres show Canada wildfire season off to wild start

NFL long snapper causes commotion in church after interrupting service to demand answers over porn claims

Biden’s frailty has Democrats scrambling to rebrand - or jump ship like Karine Jean-Pierre

Trump revives travel ban and restricts 19 countries

Trump orders investigation into Biden presidency claiming aides hid his ‘serious cognitive decline’

Diddy dangled Cassie Ventura’s friend off 17th-floor apartment balcony, jury told

Migrant deported to Mexico is returned to US by Trump administration after order by judge

'Hear how it's roaring?': Men of Weyakwin make a stubborn stand against Saskatchewan wildfires

China reveals first ever details of nuclear weapon 200x more powerful than Hiroshima bomb with huge 7,500-mile range

Hundreds of thousands in Gaza 'catastrophically food insecure'

Putin is playing the victim to get the US back onside

Putin vowed revenge against Ukraine for drone strike on airbases, Trump says

Trump signs travel ban on 12 countries and restrictions on seven more

Senators broach $2 trillion in spending cuts in W.H. meeting with Trump

Trump impeachment causes a new furious internal battle among House Democrats

Trump issues travel ban for 12 countries

Newark Airport forced to ground all flights causing travel chaos for passengers

Weapons charges laid in fatal 2024 Langley shooting

Union for B.C. port supervisors, maritime employers ratify new deal

The country where the left (not the far right) made hardline immigration laws

Alexei Navalny's work reaches Russian audiences by way of new satellite channel

Kelowna mayor questions pediatric ward closure at hospital

Replica Stanley Cup built for Calgary Flames offered to Edmonton Oilers

Saskatchewan wildfire evacuees wish for more communication

Lawsuit filed against Vernon Cadet Camp over alleged sexual abuse in 2007

Evacuees fleeing northern Saskatchewan wildfires struggle with uncertainty, loss

Plans for 'AI Fight Club' of defense, tech companies brewing in Washington

RAF chief expected to be next top British military officer: Reports

The 24 European countries that still don’t use the Euro as one more joins - full list

China taunts 'weak' Putin after Russia's 'Pearl Harbour' Ukraine attack nightmare

We gave $7B to California for a high-speed rail line and no track was ever laid: ‘Trains to nowhere’

Masters of the House: Trump and Vance set to attend ‘Les Miserables’ opening at the Kennedy Center

Trump administration signals it will slash funds for long-delayed California high-speed rail project

Mom dies from her injuries days after her 5-year-old was killed by a ‘drunk driver’ that slammed into their car

Man fatally shoot his in-laws while his kids hid in the next room, cops say