
Contents
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Technical Observatories for Better AI Governance Technical Observatories for Better AI Governance
-
How we can use technical metrics to improve government assessment and oversight of AI How we can use technical metrics to improve government assessment and oversight of AI
-
-
How the AI Community Uses Metrics and Measures to Understand AI Progress How the AI Community Uses Metrics and Measures to Understand AI Progress
-
How the AI community regulates and develops itself via measurements How the AI community regulates and develops itself via measurements
-
Computer Vision: ImageNet and its variations Computer Vision: ImageNet and its variations
-
Natural language processing: Moving from discrete tests to evaluation suites Natural language processing: Moving from discrete tests to evaluation suites
-
Where the AI community falls short, today, on measurements and evaluations Where the AI community falls short, today, on measurements and evaluations
-
Our benchmarks are becoming irrelevant at an increasing rate Our benchmarks are becoming irrelevant at an increasing rate
-
The emergence of new AI ethics papers and benchmarks The emergence of new AI ethics papers and benchmarks
-
-
How Governments Could Integrate These Metrics and Measures into Policymaking How Governments Could Integrate These Metrics and Measures into Policymaking
-
Why governments are not (yet) able to regulate AI via measurements, and what to do about it Why governments are not (yet) able to regulate AI via measurements, and what to do about it
-
How governments approach questions of measurement and assessment today How governments approach questions of measurement and assessment today
-
One way governments could integrate measures from the AI community for better governance One way governments could integrate measures from the AI community for better governance
-
Increase the amount of measurement and assessment work being done broadly, then coordinate the work with demands from regulators and other parties Increase the amount of measurement and assessment work being done broadly, then coordinate the work with demands from regulators and other parties
-
-
Conclusion Conclusion
-
Notes Notes
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
17 Information Markets and AI Development
Get accessJack Clark, Co-Founder of Anthropic; Co-Chair of the AI Index Steering Committee, Stanford Institute for Human-Centered Artificial Intelligence (HAI)
-
Published:26 January 2023
Cite
Abstract
Good governance depends on good information. This chapter discusses how we can better measure and assess the state of AI progress by closely looking at the trends in the technical literature. Due to rapid progress in AI during the last decade, we are beginning to see large-scale deployment of AI systems into society, but this is happening at a time when the capacity for (most) governments to have a well-calibrated understanding of the technology is declining. We expect that, if the status quo situation holds, it will be increasingly hard for governments to effectively govern AI developers. Therefore, we believe an intervention is needed to increase the chance of AI being governed effectively in the future. In this chapter, I discuss how metrics and measures are used by the AI research community, how these measures are themselves used by the AI community as informal devices of self-regulation, and how governments could better integrate this data into their policymaking apparatus and use these measures to steer the private and academic AI sectors. I conclude with some specific recommendations for how policymakers could develop infrastructure for monitoring AI publications and outline how this infrastructure could be merged with policymaking.
Sign in
Personal account
- Sign in with email/username & password
- Get email alerts
- Save searches
- Purchase content
- Activate your purchase/trial code
- Add your ORCID iD
Purchase
Our books are available by subscription or purchase to libraries and institutions.
Purchasing informationMonth: | Total Views: |
---|---|
January 2023 | 6 |
February 2023 | 10 |
March 2023 | 10 |
April 2023 | 8 |
May 2023 | 18 |
June 2023 | 14 |
July 2023 | 17 |
August 2023 | 20 |
September 2023 | 16 |
October 2023 | 44 |
November 2023 | 27 |
December 2023 | 23 |
January 2024 | 20 |
February 2024 | 19 |
March 2024 | 19 |
April 2024 | 8 |
May 2024 | 21 |
June 2024 | 16 |
July 2024 | 13 |
August 2024 | 15 |
September 2024 | 19 |
October 2024 | 26 |
November 2024 | 16 |
December 2024 | 16 |
January 2025 | 18 |
February 2025 | 15 |
March 2025 | 12 |
April 2025 | 13 |
May 2025 | 4 |
Get help with access
Institutional access
Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:
IP based access
Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.
Sign in through your institution
Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.
If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.
Sign in with a library card
Enter your library card number to sign in. If you cannot sign in, please contact your librarian.
Society Members
Society member access to a journal is achieved in one of the following ways:
Sign in through society site
Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:
If you do not have a society account or have forgotten your username or password, please contact your society.
Sign in using a personal account
Some societies use Oxford Academic personal accounts to provide access to their members. See below.
Personal account
A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.
Some societies use Oxford Academic personal accounts to provide access to their members.
Viewing your signed in accounts
Click the account icon in the top right to:
Signed in but can't access content
Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.
Institutional account management
For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.