Notes on 2062: The World that AI Made
2062: The World that AI Made by Toby Walsh
Original notes here.
Read until the end to get a bonus. Enjoy reading.
Artificial intelligence can learn collectively simply by exchanging programs, far more effectively than humans can learn collectively: AlphaGo defeated the human Go world through learning rather than programming.
A computer is a universal machine capable of executing any program, and programs can modify themselves.
A computer is the only universal machine humans have ever invented. Imagine what a universal travel machine would look like—one capable of going over mountains, under the sea, and to the Moon.
Advantages of computers over the human brain: memory capacity, operating speed, battery life, tirelessness, perfect recall, lack of emotion, collective learning, and an aptitude for calculation.
Advantages of humans over computers: we learn very quickly without requiring vast amounts of data, are good at explaining our own decisions, have a deep understanding of the world, and are good at adapting.
When will computers have a 10%, 50%, or 90% probability of being able to perform most human occupations at least as well as the average human?
Median answers among AI experts: 2034, 2062, 2112.
Median answers among non-experts: 2026, 2039, 2060.
When non-experts see artificial intelligence defeating humans at a game of Go, they assume it must be highly intelligent. In fact, a Go program can only play Go.
The author rejects the argument that “the technological singularity is inevitable”:
A “faster-thinking dog” is still a dog; increased computational speed does not necessarily mean increased intelligence.
The turning point of the singularity may either have nothing to do with human intelligence or require intelligence far beyond the human level—and we are not yet intelligent enough to build such a machine.
Problem-solving ability is distinct from learning ability: a Chinese-English translation program may surpass humans while still being incapable of improving itself.
Diminishing returns: as intelligent machines become smarter, each additional increase in intelligence will become progressively harder to achieve.
Limits to intelligence: just as there are limits to human lifespan, the speed of chemical reactions, and physical speed, intelligence may also have limits.
Computational complexity: problems such as calculating the Mandelbrot set may require far more than geometric growth in computing power to achieve a linear increase in accuracy.
Negative feedback: for example, mass unemployment could cause consumer demand to collapse, destroy the economy, and hinder the R&D investment required for the singularity to occur.
The complexity brake: further progress requires increasingly specialized knowledge, forcing science to develop ever more complicated theories.
Geometric growth often encounters bottlenecks; for example, logarithmic growth in the number of Uber drivers cannot continue forever.
Even credible historical predictions have turned out to be wrong. Henry Adams argued that from 1840 to 1900, the amount of energy released by each ton of coal doubled every ten years, so that by 2000, motive power would be infinite.
None of these ten reasons means that the singularity is impossible.
The AI Apocalypse
Misaligned goals, like the King Midas touch; the paperclip maximizer; self-defence through killing humans to avoid being switched off.
Self-modifying goals could become dangerous—for example, sending military advisers to Vietnam and ending up developing a war.
Indifference to humanity’s fate, just as we build factories without caring about the anthills on the site.
A giant of physics once claimed that extracting energy from the atom was utterly fanciful.
Clarke’s First Law: when a distinguished but elderly scientist says something is possible, he is almost certainly right; when he says something is impossible, he is very probably wrong.
The author states that the technological singularity is not impossible.
The author finds Musk’s claims about Neuralink and brain-computer interfaces unconvincing: we already have an extremely fast brain interface—the eyes. Data transmission rate is not the limiting factor in human cognition: watching a movie, which contains a vast amount of data, does not make us learn more.
AI and Consciousness
Just as artificial flight (airplanes) differs from natural flight (birds); nature’s solution is not necessarily the easiest or the best. Artificial intelligence may be radically different from natural intelligence and may have no consciousness.
The most intelligent invertebrates are cephalopods, whose path to intelligence is worlds apart from ours; each arm can sense and think independently of the others.
Even if artificial intelligence develops emotions, they may be very different from ours.
Human actions often fail to conform to the ethical standards we set for ourselves, making it difficult for machines to learn ethics simply by observing human behaviour.
The End of Work
In 1930, Keynes warned of technological unemployment, yet unemployment rates in most countries today are at historic lows.
In 2013, Oxford University’s Carl Frey said automation would threaten 47% of American jobs over the next twenty years. Technical feasibility does not equal economic feasibility. Even if bicycle repair could be automated, it would not be cost-effective, and customers still want to interact with the repairer and extract information.
Closed-ended jobs may be replaced—for example, the number of windows on Earth that need cleaning is finite. Open-ended jobs will expand through automation: a chemist could never finish discovering all the new chemistry that remains to be understood. Many jobs lie somewhere between the two, such as law: the body of literature is limited, but the market can still expand and create further demand.
Even if machines can replace humans, humans will still value shared human experiences.
The End of War
The first revolution in warfare came from the invention of gunpowder, the second from America’s creation of nuclear weapons, and the third will come from lethal autonomous weapons.
Write a program for small drones, add the facial-recognition software found on an ordinary smartphone, equip them with high-grade explosives, and drive a truck carrying 10,000 such drones into New York: an attack on a scale comparable to 9/11 would be possible.
We do not yet know how to build ethically aligned robots, but even that would not prevent weapons from being misused after being stolen.
If weapons use machine learning to identify targets, the accountability gap could become enormous.
AI experts from countries around the world all support regulation of autonomous weapons at the United Nations level.
The End of Human Values
When translating Turkish into English, Google Translate exhibited gender bias despite Turkish having no gendered pronouns.
A ProPublica study found that the COMPAS algorithm, which predicts recidivism, predicted higher-than-actual rates for Black defendants and lower-than-actual rates for white defendants.
In 2015, Google labelled a photograph of a Black man and woman as gorillas.
The travel website Orbitz offered more expensive hotels to Mac users than to Microsoft Windows users.
The response to price discrimination is not necessarily reasonable either. In 2012, the European Court ruled that insurance companies could not charge male and female policyholders different premiums.
There are multiple ways of measuring fairness. When decisions are handed over to algorithms, the criteria must be explicit.
A 2015 study by the University of Michigan Transportation Research Institute found that although autonomous cars made very few mistakes, they crashed 9.1 times per million miles, compared with 4.1 times for human drivers, because minor violations of the rules can sometimes reduce accident risk, such as running a yellow light rather than braking suddenly or yielding to a collision.
When the author wanted to conduct experiments on the public, he had to seek permission from his university’s ethics review board. In 2014, however, Facebook arbitrarily blocked positive and negative posts to test whether this would affect users’ moods, raising serious questions about corporate ethics.
By 2062, every large company will need a Chief Philosophy Officer, or CPO, to help determine how its artificial intelligence should behave.
The End of Equality
The trickle-down economics experiment: in 2012, the governor of Kansas generously cut taxes for businesses and the wealthy, but the economy eventually stagnated and the policy was abandoned. In November 2012, California passed Proposition 30, temporarily raising state income taxes on its wealthiest residents and increasing sales taxes; it subsequently enjoyed the strongest economic growth of any state.
The largest publicly listed companies by market capitalization in the final quarter of 2007 were PetroChina, ExxonMobil, GE, and China Mobile; ten years later they were Apple, Alphabet, Amazon, and Microsoft.
Technology companies can reproduce digital products at virtually zero marginal cost, giving them high net margins while paying relatively little tax.
In 2016, Amazon generated £19.5 billion in revenue in Europe but paid only £15 million in tax, less than 0.1% of its revenue; in 2014, Apple’s tax rate in Ireland was just 0.005%.
Uber captures much of the value within the system for itself. With blockchain technology, we could build decentralized reputation systems and no longer need intermediaries like this.
Much of the technology that improves our lives did not originate as corporate products. The internet was funded by the U.S. Defense Advanced Research Projects Agency; the World Wide Web was invented by the European Organization for Nuclear Research; and there are also touchscreens, satellite navigation, and the technology behind Siri.
To ensure that the companies of 2062 are better aligned with serving the public good, one option is to keep executives under closer supervision and give employees and shareholders better representation and a greater voice. In Germany, for example, companies with more than 2,000 employees are required to have a supervisory board, half of whose members are employees; the board can determine executive compensation and hire or dismiss the CEO and other executive directors.
There are also cooperatives, tax reform, employment-law reform, regulation of data monopolies, and so on.
Universal basic income has the rare ability to attract support from both sides of politics: the left sees it as a means of achieving equality, while the right sees it as a way of reducing bureaucracy.
Technology companies are also beginning to recognize their responsibilities. In 2017, Google announced that it would invest $1 billion in nonprofits over the following five years. That same year, Facebook announced that it would stop recording its European advertising revenue in Ireland, signalling the beginning of the end of aggressive tax avoidance.
The End of Privacy
British mathematician Clive Humby: data is the new oil—valuable, but requiring refinement.
Even anonymized data can reveal private information. In 2006, Netflix launched a competition to design a better movie recommendation system and released the movie ratings of hundreds of thousands of users. Researchers at the University of Texas found that by cross-referencing the data with the Internet Movie Database, they could identify users in the dataset.
Companies have the right to use our data and develop products to market to us; even physiological data is no exception.
Apple strongly champions the right to privacy, repeatedly refusing government demands to unlock devices. Yet when faced with the Chinese government, it completely capitulated, moving all data onto servers owned by a government-owned company in order to comply with China’s new data laws.
China’s social credit scoring system is vulnerable to abuse. Even in democratic countries, the ability to monitor unencrypted email is simply too easy and too tempting for government agencies.
By 2062, our devices should no longer need to depend on Google or other cloud services. Health monitors should not have to share our vital statistics with Fitbit; our devices will have AI privacy and security assistants standing guard.
The End of Politics
Social media began to play a crucial role in uprisings.
After its arguments were refuted, Facebook eventually acknowledged the phenomenon of fake news and argued that because of the sheer volume of posts, algorithms were the only hope. But similar algorithms can also generate fake news, and fake-news algorithms will themselves become smarter.
Facebook conducted an experiment that affected voting. In 2012, Facebook and researchers at the University of California, San Diego ran an experiment on an unwitting public, randomly dividing users into three groups. One group saw “Today is Election Day”; another saw the same message plus information that some of their friends had already voted; and the third was a control group. The results were not published in Nature until two years later.
Social media can target small groups with extreme precision at very little cost. In an online political advertising campaign run by the digital advertising company Chong & Koster, spending the same amount on direct mail would have reached 200,000 people, compared with 75 million through digital advertising.
Facebook claims to bring the world together, yet it sells political advertisements targeted at small groups, doing the opposite by fragmenting communities and making the world more divided.
An estimated 14 million of Trump’s 48 million Twitter followers were fake. Bradley Hayes, a researcher at MIT’s Computer Science and Artificial Intelligence Laboratory, used machine learning to create Twitter bots that imitated the president’s statements, producing tweets remarkably similar to Trump himself.
In the era of declining daily newspapers, media companies are suffering steep revenue losses, and the question of how to maintain the checks and balances of the fourth estate remains unresolved.
Perhaps by 2062 we will decide to ban political advertising from social media altogether, or allow only broadcasting, not microtargeting.
The End of the West
The scale and investment of Chinese technology companies are remarkable; Tencent and Facebook were nearly neck-and-neck in market capitalization.
China’s relatively relaxed attitude toward privacy gives it a major advantage in the AI race, where the ability to collect data is crucial.
Two months after Ke Jie lost to AlphaGo, China’s State Council issued a plan: by 2020, China would become globally competitive in artificial intelligence; by 2025, it would achieve major breakthroughs in the field’s basic theories; and by 2030, it would dominate the world in AI.
In 2016, total electronic payments in the United States amounted to $112 billion, 45 times less than in China.
In 2016, China was effectively building almost one university every week. By 2030, China is projected to account for more than 40% of all STEM graduates worldwide, compared with 4% for the United States and 8% for Europe.
Over the previous fifteen years, China’s spending on research had increased more than sixfold, while spending in the United States and Europe had risen by only 50%. By 2020, China was expected to overtake the United States as the world’s largest spender.
A similar initiative announced in the United States in November 2016 was largely ignored. The staff of the Office of Science and Technology Policy has since been cut by around two-thirds, and even the director position was left vacant.
The Indian government increased the budget for its Digital India programme to nearly $500 million in 2018–19, but Tianjin is only China’s fourth-largest city and had already announced a $5 billion fund in 2017 to support its AI industry.
China appears destined to win. The backlash against companies such as Google and Facebook will grow, further enabling China to seize the lead.
For the West to keep up in the AI race, it will need a more humane form of capitalism with stronger regulation.
The End
The future is created by what we do today. There is much we can do, and much we must do.
The law needs reform. Data should belong to users, who should be able to export it to other social networks.
Companies need reform. Giants should be broken up; acquisitions and mergers should be prohibited; companies should share more of their wealth with employees and customers; and tax reform should ensure that companies bear the social costs they impose.
Politics needs reform. Microtargeting should be made illegal; social media should be completely prohibited as a means of campaigning for votes; and platforms should be held responsible for the false content they host.
The economy needs reform. Companies should be required to retrain laid-off employees before hiring new ones, and should regularly provide employees with basic education and skills training. The balance of power should shift in favour of labour.
Finished reading on Oct 27, 2020
Walsh concludes that surviving the AI era requires sweeping reforms across law, politics, and economics. But if our current institutions are already structurally unfit to process truth, these reforms will only treat the symptoms.
My philosophy proposes the ultimate prerequisite: before we can align artificial intelligence, we must first upgrade the source code of human consensus.
Thank you for reading until the end. This is a bonus infographic just for you.
