New Zealand’s next government should prioritise
addressing the risks of advanced AI, says Good Ancestors New
Zealand, after
an independent assessment found that one of the world’s
leading AI companies, Anthropic, is likely improving its
models at around twice its previous rate. METR judged that
most of that speed-up is likely coming from the company’s
own use of AI.
On 5 November the AI evaluation
non-profit METR published its assessment of research and
development at Anthropic, based on 11 weeks of access to the
company’s internal data and staff. It found Anthropic is
“likely improving the capabilities of their models on
well-specified tasks at around twice the rate” it did
between March 2024 and October 2025.
METR found no
evidence of significant bottlenecks to further progress over
the next 12 months and warned of “a significant risk” that
progress could speed up further. METR says its estimate
carries high uncertainty, and it would not be surprised if
the true rate were anywhere from 1.3 to 3 times the
historical rate. Anthropic also redacted a majority of the
evidence METR found most informative.
The finding
comes two months after Anthropic CEO Dario Amodei
called on AI developers to “pace the frontier” by
deliberately slowing the pace at which companies improve the
capabilities of AI models. OpenAI
CEO Sam Altman and
xAI’s Elon Musk publicly agreed. Amodei said slowing
down would need coordination across the industry and support
from governments.
Advertisement – scroll to continue reading
Good Ancestors Executive Director
Dan Foote says the assessment puts independent numbers on
the acceleration the companies themselves have warned
about.
“In September the people running the world’s
leading AI companies said the improvement of leading models
needed to slow down. An independent assessment now suggests
at least one of them is improving at around twice its
previous pace, and that AI is likely the main reason for
that speed-up.”
Foote says New Zealand does not yet
have the institutional arrangements in place to handle this
pace of change.
“When AI is used to speed up its own
development, the time governments have to understand and
respond to the risks gets shorter. The most serious of those
risks could be catastrophic and New Zealand needs to be
better prepared to tackle them.”
New Zealand has a
caretaker government until a new government is formed.
After the 2023 election, that process took around six weeks.
This time the official result, including special votes, is
due on 27 November and serious coalition negotiations are
expected to begin after that.
Under caretaker
conventions, Ministers avoid significant new policy
decisions until a government is formed. Officials are,
however, preparing briefings for incoming Ministers. Good
Ancestors is calling for those briefings to set out the
risks from advanced AI and the options for responding,
including a
New Zealand AI Safety Institute and a national AI Safety
Strategy.
“We are not asking New Zealand to slow
its use of AI. Whoever forms the next government will
inherit a technology that is moving faster than it was a
year ago and could have transformative impacts on New
Zealand in this term of government. The preparation should
start now, so the next government can act in its first
weeks,” says
Foote.