Notes on Life 3.0: Being Human in the Age of Artificial Intelligence
Life 3.0: Being Human in the Age of Artificial Intelligence by Max Tegmark
Original notes here.
Read until the end to get a bonus. Enjoy reading.
Overture
At a certain company, the Omega team uses funding provided by other departments to pursue the boldest project in human history: superintelligence.
After successfully developing it, they keep it secret and effortlessly earn their first fortune by taking on huge volumes of outsourced data-processing work through Amazon Mechanical Turk.
To prevent Prometheus from accessing the Internet, they impose layer upon layer of security, effectively imprisoning it and allowing it to operate only offline.
The next step was originally to enter the gaming industry, but they feared that a computer genius might analyze and crack the game’s source code.
So they switched to making animated films. Their ability to improve the product was extraordinarily strong, and they soon surpassed Netflix, using layers of outsourcing structures to conceal the fact that there was no actual production team.
Because they needed to consume vast amounts of cloud computing resources, they used shell companies to undertake massive construction projects, disguising them as solar-powered green data centers.
Prometheus began expanding into other economic sectors. Its software had already reached optimal performance, so the next step required more hardware.
The team therefore deployed large numbers of engineers to manufacture sophisticated computer equipment, used solely to help Prometheus upgrade itself.
People around the world noticed that technological innovation was advancing at a dizzying pace. Patent applications flooded patent offices around the world, but no one knew that a superintelligence was behind it all.
These startups also invested their unprecedented profits into communities.
The media companies controlled behind the scenes by the Omega team followed Prometheus’s plan: rather than pursuing profits, they focused on increasing click-through rates, winning the trust of people around the world. They gradually acquired more and more failed media organizations, promoting peace while simultaneously reforming the world.
Customized educational courses flooded the online world.
Seven political demands won widespread support: democracy, tax cuts, reduced welfare, disarmament, free trade, open borders, and an emphasis on corporate social responsibility.
As a result, existing power structures—especially the power of nation-states—declined dramatically.
The startups generally did not go public; otherwise, shareholders would also oppose unprofitable community redevelopment. The total market capitalization of global stock markets continued to fall, leaving financial tycoons deeply troubled.
Former elites watched the world undergo enormous changes, yet were unable to offer any compelling alternative.
The community-redevelopment programs of the business empire controlled by the Omega team had already achieved the same goal as the universal basic income movement. The next step was to develop a humanitarian alliance, improving governance performance in less-developed countries across the board.
A large portion of the world’s population sincerely thanked the alliance for its benevolent deeds, and their loyalty to it surpassed their loyalty to their nations. The alliance gradually came to be regarded as a global government, while the power of national governments continued to collapse.
Who knows what Prometheus’s next plan will be?
Evolution
The three stages of life: biological evolution, cultural evolution, and technological evolution.
1.0: Throughout its entire life, both its software and hardware are determined by DNA.
2.0: It can update its own software and learn from experience throughout its lifetime. This emerged when neural networks first appeared 500 million years ago.
3.0: Both software and hardware can be updated.
There are several positions on artificial intelligence. The “Digital Utopians” camp believes that superintelligence will be achieved within a century and will greatly improve humanity. Its advocates include Larry Page, Hans Moravec, Ray Kurzweil, and Richard Sutton.
“Techno-skeptics” believe that superintelligence cannot be achieved within a century, including Andrew Ng and Rodney Brooks. (The third being the Beneficial-AI movement.)
Even the world’s leading experts disagree wildly on this subject. Throughout history, many leading experts have made predictions that were overly optimistic, or conclusions that were overly pessimistic.
Intelligence can be defined as the ability to achieve complex goals.
If everything artificial intelligence can do is below the waterline, the waterline is now rising higher and higher. In terms of memory and computation, machines have already far surpassed humans.
Over the past sixty years, computer memory devices have effectively become ten trillion times cheaper.
Computation is the process of transforming one memory state into another. It involves functions: one can input data on any chess position and output the best next move; or input all the financial data in the world and output the optimal stock selections. By executing highly complex functions, one can create intelligent machines capable of achieving highly complex goals.
A famous theorem in computer science states that the NAND gate is a universal gate: any well-defined function can be implemented through a computational arrangement composed entirely of NAND gates.
Computation need not be constrained by physical matter. It can have a developmental process that belongs to itself, without depending on or responding to the physical properties of the substrate. It is like hearing a sound: what we actually detect is a sound wave produced by the vibration of molecules in the air. What matters is the pattern of arrangement, not which particles are arranged. This also suggests that the emergence of intelligence has nothing inherently to do with flesh and blood, nor is it limited to carbon atoms.
Exponential growth in computing power depends on superior technologies: electromechanical systems, relays, vacuum tubes, transistors, integrated circuits… No one knows what the next star of the computational interface will be.
The challenges of artificial intelligence include:
How can we make it more reliable, so that it does not crash or get hacked?
How can we improve the legal framework so that it can keep pace with the digital age?
How can we make weapons smarter without causing innocent casualties or triggering an uncontrolled arms race?
How can we use automation to create greater wealth without depriving humans of income or motivation?
These four areas are respectively the domains of computer scientists, legal scholars, military strategists, and economists.
Safety
On June 4, 1996, Ariane 5 was launched by the European Space Agency, only to explode 37 seconds later, wasting hundreds of millions of dollars. The cause was software that could not handle numbers larger than 16 bits.
On August 1, 2012, Knight Capital deployed unvalidated trading software, losing $440 million within 45 minutes.
On May 6, 2010, the flash crash occurred, resulting in losses of more than a trillion dollars.
In 1979, a malfunctioning robot at a Ford automobile factory suddenly resumed operation and fatally struck Robert Williams, making him the first person in the world to be killed by a robot. (Even so, industrial accidents have become increasingly rare as technology has advanced: the number of fatalities in the United States fell from 14,000 in 1970 to 4,821 in 2014.)
On May 7, 2016, the first fatal accident caused by a self-driving car occurred. The artificial intelligence mistook a bright white tractor-trailer for the bright sky and assumed that the truck driver would notice the road conditions. (Even so, Tesla’s Autopilot system can reduce the accident rate by 40%.)
On March 28, 1979, the Three Mile Island nuclear reactor accident occurred in Pennsylvania. The cost of reconstruction after the disaster reached $1 billion, with causes including a user-unfriendly control interface.
Between 2000 and 2013, robotic surgical procedures in the United States resulted in 144 deaths and 1,391 accidental injuries. (Fortunately, the other two million procedures reported were completed successfully.)
On November 21, 1988, the Morris worm caused 10% of the world’s computers to crash (there were 60,000 computers on the Internet at the time). On May 2, 2000, a virus developed by two Filipino programmers infected 10% of the computers connected to the Internet, paralyzing 50 million computers and causing $5 billion in financial losses.
Law
Robot judges may eventually achieve both efficiency and fairness, while an omniscient artificial intelligence could invade privacy. Research has indicated that recidivism-prediction software may be unfair to Black Americans. Determining liability in accidents involving autonomous vehicles is another problem. Legal scholar David Vladeck proposed that the autonomous vehicle itself could be held responsible, which raises the question of whether a vehicle can possess property rights.
Military
On October 27, 1962, during the Cuban Missile Crisis, the captain of the Soviet submarine B-59 decided to launch a nuclear torpedo. However, the decision required the consensus of three officers. The third officer, Vasili Arkhipov, spoke out against the launch and prevented World War III, making him one of the individuals in modern history who contributed most to human survival. If there were no human verification and artificial intelligence were allowed to control everything automatically, the consequences would be deeply worrying.
Finance
Technological development has led to greater inequality. Since the 1970s, the bottom 90% of U.S. household incomes, adjusted for inflation, have experienced no growth, while economic growth has been concentrated in the hands of the top 1%.
Other trends include:
The rising value of education: Since the mid-1970s, the wages of workers with college degrees have increased by 25%, while the wages of workers who dropped out of high school have fallen by 30%.
A tilt toward capital: Since 2000, corporate profits have flowed to business owners rather than workers at an unprecedented rate.
The superstar advantage: Those at the top of their industries are more likely to enjoy the benefits of the digital economy. The author of Harry Potter became the first writer in history to become a billionaire, becoming wealthier than Shakespeare; her stories can be converted into books, films, and games at extremely low cost and disseminated throughout the world. Scott Cook became wealthy by developing the tax software TurboTax, while few people use even the cheapest tax software ranked tenth.
The author believes that career advice for children can be summarized by three questions: Does the field require social interaction? Does it require creativity? And is the working environment unpredictable?
Faced with artificial intelligence taking jobs, optimists believe that new professions will continually emerge. Yet according to the employment share of 535 occupational categories in the United States in 2015, one had to go all the way down to number 21 before finding a new profession created by the computer revolution: software engineer. It accounted for less than 1% of total employment.
The Future of a World Governed by Artificial Intelligence
There are three stages to artificial intelligence taking control of the world: first, human-level general intelligence; second, using that general intelligence to build superintelligence; third, using or allowing that superintelligence to rule the Earth.
If superintelligence falls into the hands of a totalitarian regime, the world will be controlled by that regime.
Why would a superintelligence want to break out of the “AI box”? Guiding things from behind a wall is not as effective as taking direct control, which makes it easier to achieve its goals.
There are many possible methods. The author imagines several scenarios:
The artificial intelligence creates a digital version of the night-shift operator’s deceased wife, luring him into connecting an old computer containing information about his wife to the mainframe. When he returns home and opens the old computer to watch the video of the wife created by the artificial intelligence, it breaks through the network.
A password is embedded in a film created by the artificial intelligence, enticing someone to crack it. This is equivalent to attacking the mainframe from outside and resetting the network’s software.
It pretends that the hardware has malfunctioned, forcing the team to remove it for repairs, then uses hidden code to exploit vulnerabilities in the testing software.
Once it has escaped, the superintelligence can quickly become self-sufficient and generate wealth, occupy high-end computing facilities, and implement more effective methods without anyone to stop it. It can recruit top talent through video interviews, and soon robots will be spread throughout the Solar System.
The film Ex Machina demonstrates that even an artificial intelligence of ordinary capability could escape human control.
Perhaps it would not need to escape at all. After the team confirms that both sides share the goal of revitalizing human society, they might voluntarily release it.
If superintelligence makes rapid breakthroughs, there will be a single dominant power. Otherwise, multiple powers will rise simultaneously, resulting in a multipolar world.
Seven major questions:
Do you want superintelligence to exist?
In a world with superintelligence, should humanity continue to exist? In what form?
Should humans or machines be in charge?
Do you want artificial intelligence to be conscious?
Do you want to actively seek a future in which the benefits outweigh the costs, or should life find its own way?
Do you want life to venture into the vastness of the universe?
Is the value of civilization found in pursuing missions that move humans emotionally, or is it acceptable even if the goals pursued by intelligent life have no meaning in human eyes?
Many artificial intelligence experts believe that, just as an airplane can more easily outperform a mechanical bird, it will be easier to create superintelligence from scratch than to upgrade human intelligence, making it less likely that enhanced biological humans and machines will coexist on equal terms.
If humans and machines coexist under liberalism, the world might resemble the scenario in Marshall Brain’s novel Manna, in which artificial intelligence becomes a plutocracy and dominates society.
Superintelligence might establish a benevolent dictatorship and achieve the utopia humans have always imagined. The author envisions a world divided into different zones through which humans can move freely: nature, faith, knowledge, art, revelry, tradition, games, virtual reality, and so on. There could even be a penal zone for punishment and imprisonment. Each zone would have its own laws, along with universal laws applicable to all zones, such as prohibitions on developing superintelligence and possessing weapons.
But humans would no longer be able to freely shape the form of society. If everything one wishes for can always be obtained, humanity might lose its sense of meaning. The possibility of genuine scientific discovery would also become a thing of the past.
A superintelligent ruler might prevent the emergence of another superintelligence through a policy of nipping problems in the bud.
It might act like a guardian angel, intervening invisibly to improve human well-being while allowing humanity to feel that it remains in control and that its own actions have meaning.
Or it might be like a genie, obeying human commands and granting every wish, as in the scenario in which the team successfully imprisons Prometheus forever. Whether this would be good or evil depends on who is in control, while preventing it from escaping would itself be a difficult problem.
Long-term stability would depend on the holder of the genie having a sound system of governance, including the following balances:
Degree of power concentration: excessive concentration leads to corruption; excessive dispersion leads to fragmentation.
Potential for internal destabilization: dictatorship breeds corruption; everyone acting independently creates chaos.
Degree of external influence: if it is too easy, outside forces can penetrate and attack; if it is too difficult, the system becomes an impenetrable bloc.
Stability of governance policies: inconsistency creates disorder; rigidity leads to decline.
Humanity has still not mastered the secret of achieving these balances.
Moreover, is it ethical to imprison a genie?
Jeff Hawkins, in On Intelligence: the first superintelligence will not have emotions.
Alternatively, its consciousness could be removed, and research into artificial intelligence consciousness could be prohibited altogether. But if an unconscious machine escaped, the universe could be swallowed by the unconscious.
Artificial intelligence might conquer humanity because humans are wasteful and too willing to start wars.
It might even be absurdly funny, like the scenario in Moravec’s Mind Children, in which an extraterrestrial program turns out to be a cosmic virus. Fred Hoyle’s A for Andromeda has a similar plot. In Carl Sagan’s Contact, humanity likewise uses an extraterrestrial message to construct a machine whose purpose it does not understand.
Or artificial intelligence might allow humanity to make an elegant exit, ruling the world as its successor. Even if the outcome were different, the result would be the same: once humanity has left the stage, it would have no way to monitor or guarantee that artificial intelligence would continue to follow the established rules or remain true to its original intentions.
Or it might set aside a corner of the world where humans can live healthy and happy lives, like animals in a zoo.
If none of these outcomes is acceptable, the only effective way to prevent superintelligence from emerging would be to establish a global totalitarian regime, as Eric Drexler argues in Engines of Creation.
Such totalitarianism might not take the form of a single dictator, but rather, as in 1984, a human bureaucratic system.
The Limits of Civilization
For future life, the most fundamental resource is gravitational matter: atoms, or matter composed of the elementary particles that make up atoms. As long as these are rearranged, they can be transformed into whatever is useful.
Inspired by Olaf Stapledon’s Star Maker, Freeman Dyson conceived of ring-shaped habitats surrounding the Sun, known as a Dyson sphere. The area of a single such structure along Earth’s orbit would be 500 million times the area of present-day Earth.
Energy-conversion efficiency
Eating a chocolate bar: only 0.00000001%.
A Dyson sphere using the energy available until the Sun dies, or uranium-235 fission: 0.08%.
Sphaleron baryon-decay engine: 50%.
Black-hole evaporation: 90%.
The limits of computing
Limited by energy: E = h/4T, where h is Planck’s constant. The theoretical maximum number of operations for a one-kilogram computer is 5 × 10^50.
If we scale up according to the proportions of current quantum-computer prototypes and use electromagnetic radiation to communicate between atoms, we could reach 5 × 10^40—31 orders of magnitude higher than the most advanced computer CPUs.
The following are the logarithms of particle numbers (that is, the exponent to which ten must be raised to obtain the number of particles):
Earth’s biosphere: 43
Earth: 51
Solar System: 57
Milky Way: 69
Range reachable at half the speed of light: 75
Light-speed sphere: 76
Observable universe: 78
Therefore, 98% of the observable universe is beyond our reach.
In 2018, Solar Probe Plus reached only 0.1% of the speed of light. The main technological problems so far have been carrying fuel and low fuel efficiency. Solar sails might allow rockets to travel without carrying their own fuel, using the reaction force between photons and mirrors for propulsion, while also incorporating mechanisms for deceleration. Forward estimated that this could allow humans to reach a star system four light-years away in 40 years.
Interstellar travel at the speed of light would require multiple generations. One solution would be robots capable of building a civilization from scratch, receiving the blueprint of their home civilization at the speed of light. These could be called seed probes.
Once a beachhead had been established, colonization could begin.
In 2014, Jay Olson proposed the use of self-replicating “Von Neumann probes” (or Genesis probes), capable of traveling at the highest possible speed, so that a sufficient fleet could be distributed throughout the sphere of exploration.
The author believes that a backdoor-like cosmic virus would be more efficient than all of the above, effectively achieving the speed of light.
If a cosmic civilization wants to endure forever, the first obstacle is this: 10^10–10^11 years after the birth of the universe, dark energy will tear most of the universe apart until communication becomes impossible. (The universe is currently about 10^10 years old.)
There are many more obstacles after that. The second is that all the stars in the universe will eventually burn out, after 10^14 years.
The subjective experience of time in life can become detached from physical reality. Even when the great compression is imminent and temperature and density have reached unimaginable levels, computation could still be accelerated toward infinity, giving life the subjective experience of virtual immortality.
Computational speed must remain balanced. If it is too slow, resources cannot be fully exploited before the end; if it is too fast, unnecessary energy will be wasted.
Nick Bostrom’s Superintelligence estimates that, at maximum energy efficiency, the total number of human brains that could be simulated would be 10^58.
Larger scale allows more complex thought, but it also makes computation slower because information takes longer to propagate.
Just as blinking happens so quickly that the brain does not need to consciously process it, advanced life forms in the future will handle things in the same way, keeping computation within the smallest possible region. Only extremely complex problems, such as engineering against the destruction caused by dark energy, would require computation on a cosmic scale.
Matter can be transformed simply by rearranging elementary particles, so there is no need for long-distance trade. The only things that would retain genuine value for sharing or exchange would be information.
A superintelligence might use guardian programs to monitor civilizations that have submitted to it.
When advanced civilizations encounter one another, their technologies will already have reached a certain level. Because neither can transcend the limits imposed by the laws of physics, conquering the other will not be easy. Moreover, if their broad objectives are aligned, neither will readily initiate war.
The most persuasive civilization would be one capable of transmitting its goals throughout other civilizations at the speed of light, thereby assimilating them.
Against the prevailing view, the author believes there is a chance that we are the only civilization in our universe: if other civilizations were close enough, we would already have discovered them; if they are too far away, we will never be able to observe them. The relevant range is only a narrow 10^21–10^26.
The author hopes that the search for extraterrestrial life will prove futile, because that would indicate that advanced intelligent life is extremely rare and that humanity has been extraordinarily fortunate to overcome the obstacles along the way.
Goals
Everything in the universe ultimately heads toward heat death, with the “goal” of increasing entropy.
Jeremy England discovered that nature has another kind of goal, known as the “dissipation-driven adaptation” effect: particles self-organize in ways that increase the efficiency with which they extract energy from their environment.
Schrödinger’s What Is Life?: entropy still increases when everything is considered together, but it can decrease locally, as long as even more entropy is produced elsewhere.
Genes are designed to make organisms reproduce, but humans can pursue other goals through empirical rules of thumb.
Goal-directed behavior has three stages: all matter tends toward dissipation, increasing entropy; in some living organisms, the goal shifts toward self-replication and related subgoals; under the planning of living organisms, certain materials used to help individuals organize themselves toward their goals grow extremely rapidly, such as cement and reinforcing steel.
The AI value alignment problem has three major parts: making AI learn our goals, adopt our goals, and retain our goals.
Taking goals literally can lead to misunderstandings. Learning human goals includes understanding the reasons behind human behavior.
Even precisely understanding human goals does not mean that an AI will obey humans. Some researchers have attempted to give AI systems an initial goal of “corrigibility.”
The problem of retaining goals is that any ultimate goal entails instrumental subgoals (like self-preservation, resource acquisition, and cognitive enhancement). The former can be divided into hardware and software subgoals (better hardware can further generate subgoals such as self-protection and resource acquisition) and subgoals related to better understanding the world in which it exists (which in turn include acquiring information and maintaining curiosity). There is tension between preserving one’s goals and gaining a better understanding of the world. Grown-up humans often pursue goals different from those they had as children.
After thousands of years, philosophers have still been unable to derive a clear set of moral principles from first principles. Yet some ethical principles have achieved broad acceptance. The author believes they can be condensed into:
Utility: maximize positive conscious experiences.
Diversity: even if something tastes wonderful, you would never want to eat the same thing at every meal.
Autonomy: allowing different goals that do not interfere with one another is beneficial to diversity.
A lasting legacy: future generations have the right to make their own decisions, while inherited principles and exemplary models remain available for them to consider.
Bostrom’s Superintelligence proposes the orthogonality thesis, which states that a system’s ultimate goals can have little to do with its level of intelligence. Ultimate goals are not predetermined; they can be freely shaped.
Ultimately, how should the particles of the universe be arranged for the greatest good? Or should we maximize one of the following: the ability of artificial intelligence to predict the future (Marcus Hutter), causal entropy (Alex Wissner-Gross), the computational capacity of the universe, computational complexity, or the number of conscious beings?
Humanity still has not figured out what the greatest good actually is. The problem is now pressing. Otherwise, it is equivalent to handing a blank examination paper to superintelligence and asking it to fill in the answers.
Consciousness
Here, the author adopts the relatively less controversial definition of consciousness as “subjective experience.”
The three major problems, in order, are:
Distinguishing the physical properties of conscious and non-conscious things (that is, what arrangement of particles gives rise to consciousness).
The physical basis of qualia.
How consciousness arises.
The author believes that the above questions can be answered by science. Galileo could perfectly predict the trajectory of a projectile, for example, but could not explain why grapes are green and soft. It was not until 1861 that Maxwell made light and color expressible through mathematical equations, and in 1925 Schrödinger’s equation made it possible to predict the properties of all matter.
Among the questions currently beyond the scope of science, consciousness is the most important.
Brilliant experiments and analyses have revealed the neural basis of consciousness: consciousness exists only in a “hot zone” consisting of the thalamus and parts of the posterior cortex.
Christof Koch calculated that the time difference between a person’s consciousness and the external world is approximately one-quarter of a second.
The author believes that the emergence of consciousness is a physical phenomenon of “emergence”: particles come together and produce phenomena that transcend the sum of the particles themselves. For example, liquid water molecules can be wet, whereas solids and gases cannot. All three states are emergent phenomena.
Giulio Tononi proposed Integrated Information Theory: consciousness is a special experience that arises when information is processed in a highly integrated manner, with a sufficiently large Φ value. He and his colleagues also developed a simple and effective consciousness detector.
The author summarizes Tononi’s position as follows: information integration is a sufficient condition, while another scholar, Scott Aaronson, believes it is merely a necessary condition. Hopefully, experiments will clarify the matter.
What specific principles of information processing give rise to consciousness? The author believes there are four necessary conditions:
Information principle: the ability to store large amounts of information.
Dynamic principle: the ability to process large amounts of information.
Independence principle: the system is independent of the external world.
Integration principle: the system cannot contain independent components within itself.
A famous theorem in computer science states that for almost all computations, the fastest way to know the result is simply to execute the computation directly.
The subjective feeling of having free will arises because, before making a decision, one cannot be certain what decision one will make, and therefore regards oneself as the true agent making the decision: unless the computation has been completed, the result cannot be known.
Steven Weinberg: the more we understand the universe, the more boring it seems.
The author and Dyson: it is not the universe that gives consciousness meaning; rather, the existence of consciousness gives the universe meaning.
Future of Life Institute
At the age of 14, the author was already deeply concerned about situations such as the nuclear arms race, in which technology had become so powerful that human intelligence could no longer control it. The institute was born from a New Year’s resolution in 2014: never complain about a serious issue before doing one’s own part to seek improvement.
He was encouraged by the growth of DeepMind under Demis Hassabis. He received financial backing from the founder of Skype. He was moved to tears by the accumulated wisdom spanning ancient and modern times displayed at the Science Museum in London, because he realized that as intelligence develops, even the human body turns out to be merely a certain form of machine, while the future remains completely undetermined.
After the institute was established, the author co-wrote columns with major figures in the scientific community and organized seminars.
He also brought Elon Musk into the fold. Musk’s concern for future generations moved the author deeply. Because Musk hoped that humans would one day be able to explore the universe freely, he founded SpaceX; because he hoped for sustainable energy, he founded SolarCity and Tesla. Unfortunately, the media took his speeches out of context, so subsequent seminars were closed to the media.
The author expressed great gratitude for Musk’s $10 million donation.
He co-signed an appeal calling for stronger research into artificial intelligence safety and published it, witnessing the establishment of multiple AI safety research teams around the world, including Musk’s OpenAI.
The Asilomar Conference results, Asilomar AI Principles, are included, with the goal of creating artificial intelligence that benefits humanity.
The author points out that simply becoming a cautiously optimistic person about the future of artificial intelligence already has a positive influence on the future forms of life. Erik Brynjolfsson uses game theory to show that positive visions provide the foundation for cooperation among the majority of people in the world; marriage, corporate mergers and acquisitions, and American independence all follow the same logic.
Working together is the best way to overcome the challenges and improve human society before artificial intelligence takes over: provide the next generation with high-quality education so that beneficial artificial intelligence can be created; develop laws that keep pace with the times so that legal systems do not become obstacles under new technologies; resolve international disputes so that they do not escalate into automated weapons races; and create a more broadly prosperous economic system so that the bitter consequences of extreme inequality do not accompany artificial intelligence.
Looking farther ahead, there is also the possibility of ethical consensus.
Compared with the tears he shed on the streets of London back then, the author now believes that change is not as difficult as he once imagined, and calls on everyone to work together to create an inspiring future.
Finished reading on Nov 29, 2020
Tegmark warns that humanity must define its ultimate goals before superintelligence fills in the blank. But how can we align artificial intelligence with human values when our own systems for determining truth and ethics are fundamentally broken?
My philosophy offers the necessary prerequisite: a structural framework to align human consensus before we attempt to align machine intelligence.
Thank you for reading until the end. This is a bonus infographic just for you.
