Critique of Brynjolfsson and McAfee's The Second Machine Age

In January Erik Brynjolfsson and Andrew McAfee published The Second Machine Age as a follow up to their first book Race Against the Machine.  Both are excellent (though they say essentially the same thing, so if you read The Second Machine Age there is no need to read Race Against the Machine).  Since those books have been published there has been a growing discussion of how new AI technologies will effect employment, and what we should do about it.  While I think the books do a great job of laying out the problem, and have promoted more discussion about these important issues, I was struck by how inadequate their policy recommendations seem to be.

The Problem

The books do a great job of illustrating how machines are rapidly encroaching on cognitive labor.  Whereas the first machine age was characterized by machines replacing human (and other animal) physical labor, the second machine age, which we are experiencing right now, is characterized by machines replacing human cognitive labor.  They give many examples (self driving cars, etc.) that really drive the point home.

If we understand human work to be composed of essentially two parts, physical work and cognitive work, then we should expect the consequences of the second machine age to be radically different from the consequences of the first.  As machines replaced physical labor, the human workforce merely moved to jobs that were inseparable from cognition and thus beyond the capacity of machines.  The type of jobs changed, but over the long term, the number of jobs was still adequate.  As machines replace cognitive labor, however, there is no other type of job for the human workforce to move to.  Machines won't replace all cognitive jobs at once, since their cognitive abilities will ramp up over time.  However, as machines become more and more cognitively capable they will squeeze humans into a smaller and smaller niche of remaining jobs.

Eventually, if machines become as cognitively capable as humans, there will be no work left which cannot be done more cost effectively by a machine.  But even before that point, unless we find some remedy, there will be massive unemployment as machines replace humans faster than we can reallocate humans to the types of cognitive jobs which machines have not yet mastered.  So what do we do about this?

Policy Recommendations of The Second Machine Age

The policy recommendations of The Second Machine Age boil down to a focus on education (MOOCs, Kahn Academy, etc., and higher salaries for teachers), entreprenuerism (immigration reform and cutting red tape), public science funding, and public infrastructure updates. Brynjolfsson and McAfee propose paying for these initiatives with taxes on activities with negative externalities and on economic rents.

Importantly, they reject any fundamental adjustments to capitalism:

We are also skeptical of efforts to come up with fundamental alternatives to capitalism. By ‘capitalism’ here, we mean a decentralized economic system of production and exchange in which most of the means of production are in private hands (as opposed to belonging to the government), where most exchange is voluntary (no one can force you to sign a contract against your will), and where most goods have prices that vary based on relative supply and demand instead of being fixed by a central authority. All of these features exist in most economies around the world today. Many are even in place in today’s China, which is still officially communist. 

These features are so widespread because they work so well. Capitalism allocates resources, generates innovation, rewards effort, and builds affluence with high efficiency, and these are extraordinarily important things to do well in a society. As a system capitalism is not perfect, but it’s far better than the alternatives. Winston Churchill said that, “Democracy is the worst form of government except for all those others that have been tried.” We believe the same about capitalism.

They also recommend against reconsideration of the basic income ("Will we need to revisit the idea of a basic income in the decades to come? Maybe, but it’s not our first choice.") because work is beneficial to human happiness. Instead they advocate a negative income tax that augments the incomes of the working poor. Presumably, the notion is that computers will not be ready to take over all human labor even in the "long run", so better to keep humans working alongside of computers for the foreseeable future (and they give a bunch of current examples of humans working with machines out-competing either working alone).

Finally they list some "Wild Ideas" that they don't endorse, but seem worth further consideration: public mutual funds providing inalienable income to citizens (maybe not that different from a basic income), incentives to develop human augmenting rather than human replacing tech, setting aside certain categories of work for humans, using vouchers to create a minimum standard of living and a more massive public infrastructure campaign.

Inadequacy of Their Policy Recommendations

Brynjolfsson and McAfee do a great job in the first two thirds of the book, but when it comes to policy recommendation, they seem to miss their own point. While their policy recommendations are not terrible general recommendations for economic prosperity, they are potentially terrible distractions when discussed in the context of machines rapidly replacing humans throughout every sector of our economy.

The important point, which they make so well, is that whereas the first machine age was about machines replacing human physical labor, the second machine age is about machines replacing human cognitive labor. Replacing physical labor simply meant that human labor had to be reallocated to cognitive labor, but as machines replace human cognitive labor there is nothing to reallocate them to, except on a temporary basis to whatever particular types of cognitive labor that machines have yet to master.

Improved education, entreprenuerism, public science funding, and public infrastructure are great general recommendations.  They are particularly good recommendations for the first machine age where the challenge was smoothly reallocating human labor to a different category of work.  But how do they address the particular problems of the second machine age where the set of cognitive tasks where humans still have a comparative advantage is constantly dwindling?  Where we can only guess at what major employment sector will evaporate over the space of just a few years (e.g. driving)?  And where, ultimately, there is no reason to expect there to be any tasks where humans have a comparative advantage?

Not only are their policy recommendations utterly inadequate to address the problems they set forth, but then they prematurely limit the discussion of policies that might actually address the problems to only those that do not "fundamentally" adjust capitalism.

Huh?!?  Machines are rapidly replacing all human labor, but lets not change our economic system? Hasn't our economic system been, at heart, a means for allocating, organizing, augmenting and incentivizing human labor?  How can it be that human labor will be rapidly disappearing, squeezed into a smaller and smaller niche by machines, and yet no big changes to our economic system are called for?

Particularly, at the point where machines have replaced virtually all human labor, will it still make sense to vest control over most of earth's productive capital with an elite capitalist class?  What important contribution will capitalists be making when they are already employing machines to make the important decisions about how to most effectively allocate, preserve and expand their productive capital?  At that point can't we just get rid of the capitalists and have a democratically controlled government give direction to the machines?

And if that is our ultimate destination, shouldn't the policies we adopt now be aimed at smoothing our transition to that destination, and ensuring that we can successfully navigate there at all?  (If so, it would seem the most important thing we can be doing right now is strengthening and safeguarding our democracy, perhaps by addressing economic inequality much more forcefully.)

But, Brynjolfsson and McAfee do not answer any of these questions in their books.  They simply say, in effect, "lets not have any big changes to our economic system, even though we are facing the elimination of HUMAN LABOR... surely that doesn't merit a rethinking of the system."

Since the rest of their The Second Machine Age is quite excellent, I am curious about their answer to this critique.  Perhaps they see the transition as taking centuries rather than decades.  In that case it might be more important to focus on maximizing human productivity, leaving less pressing questions regarding how to transform our economy for later generations.  Or, perhaps they do not anticipate that machines will replace humans, but that humans and machines will ultimately merge.  In that case, the issue of machines replacing humans may not be relevant in the long run.  Or, perhaps they think that advances in AI will hit a wall or plateau.  In that case the primary concern is just making reallocating humans to whatever jobs will long remain out of reach of machines.

 Who knows.  They don't say.

More on Kevin Kelly's singularity thoughts

Kevin Kelly has one more Singularity related post that I think is worth examining titled The Singularity Is Always Near. He makes two points which I think are interesting, the first of which he explains as follows.

Not all intelligences are capable of bootstrapping intelligence. We might call a mind capable of imaging another type of intelligence but incapable of replicating itself a Type 1 mind. A Type 2 mind would be an intelligence capable of replicating itself (making artificial minds) but incapable of making one substantially smarter. A Type 3 mind would be capable of creating an intelligence sufficiently smart that it could make another generation even smarter.

This was interesting to think about for a moment, but when considered in light of what we understand about the creation of the only intelligent mind that we know of, the human mind, the divisions don't make any sense.  The human mind was created through evolution.  Evolution has no intelligence at all.  Evolution works through random variation and selection.  If evolution can create a mind without any intelligent design at all (indeed without any intention at all), why would an intelligent species actively working to replicate the feat have a problem doing so?  Given enough time, the only capabilities an evolved mind would need in order to replicate its own intelligence would be (1) the ability to create random variations of a design and (2) the ability to evaluate whether the design was better or worse than the previous iteration.  In other words, if an evolved intelligence is capable of undertaking the task of attempting to replicate itself, then, given enough time, it will probably succeed.  The real question is how much time it will take.  Given the successes in the world of AI research to date, humans appear to be moving quite a bit faster than evolution did.

The second point that I found interesting was that the date of the Singularity, the location of the "elbow" of the exponential curve, is completely arbitrary.  He uses the following graphs to illustrate (borrowed from another blogger making the same point):

Of all Kelly's points that I've discussed on this blog, I think this is the most interesting.  In some ways it actually fits well with my contention that the Singularity will not be post-scarcity.  The abundance crowd, typified by Peter Diamandis, think that an explosion of material wealth, unlike anything we have ever experienced, will eliminate poverty and want, and lead to a age of abundance where resources are no longer scarce and human social relations are no longer largely defined by control over scarce resources.  What he seems to be overlooking is that compared to a few hundred years ago, we are already living in a time of abundance, but poverty and want still exist, and human social relations are still very much defined by control over scarce resources.  How can this be?  The answer is that the human consumptive drive is determined not by some absolute measure of what is needed for survival but rather by what is available.  The more there is, the more we want to consume.  And we define our sense of well-being in terms of that relative measure which always grows to swallow whatever resources are available.  Viewed in this light, the Singularity will not be a discrete moment that fundamentally changes the human condition, but rather will be a continuation of the processes that we have already been experiencing for hundreds of years.

So I think Kelly's point is a good one, but I also think he is overlooking something very important.  The dates that thinkers like Kurzweil have picked out for the Singularity may really be their forecasts for when certain facts that have always typified human existence will change forever:

• work:  when artificial intelligence is capable of doing all the things that humans can do, it will be more cost effective to have robots doing all of society's work, and work will no longer define the human experience of life.
• body:  human bodies universally follow a design laid out by evolution; however, when we can re-engineer our bodies to have bigger brains, extra arms, nanobot immune systems, or to eliminate biological components altogether, the human body will no longer define person-hood.
• death:  possibly the most important of these changes, there is nothing in the laws of physics that requires death (on human time scales anyway), and most likely we will eventually find ways to avoid it indefinitely.

Exponential technological progress may be properly viewed as a continuous phenomenon rather than a discrete one.  But there are discrete points along the way that will have tremendous emotional and spiritual significance for everyone.  Singularitans theorize that the elimination of work, body and death will happen in a compressed time frame.  The Singularity was originally conceived as the point beyond which we cannot fathom.  But the future is always hard to predict, and perhaps Kelly is right that we experience change as continuous rather than discrete.   Perhaps what people really mean by the Singularity is the point where technology has changed key elements that have defined what it means to be human for as long as humans have existed.

Further correspondence with Kevin Kelly regarding Pinnacle-ism vs Thinkism

A couple of days ago I posted a response to Kevin Kelly's thinkism article.  He responded and the exchange below ensued.

Kelly's response to my last post:

I am not a pinnacle-ist by any stretch. I think we had not even begun to create the variety and quality of artificial minds that are possible. In 100 years we'll look back and say we had not yet make any kind of AI by 2014. But unlike you, I don't think we can create these super smart minds by merely thinking about them. We need to invent the chips and software, and that requires making new real things, making mistakes, and trying stuff. We have to continue experimenting with human and mammalian brains to discover how they work. We can't just think about human minds, which is what you would prefer to do. We lack data, not just ideas. All this experimentation, investigation, trials, dead ends takes time. Each probe takes time. Evolution spent several million years working around the clock to experiment. Of course we will greatly accelerate this process, as we already have done in the last 100 years. Through technology we've accelerated evolution, particularly in the last 20 years. But interestingly, our IQ has not changed in the last 20 years. We used other kinds of technology than IQ to speed things up. The evidence is that increasing our own intelligence was not needed to create some artificial mind-like things. Will we need increased intelligence to make an intelligence smarter than us? And will intelligence compoundly accelerate. Those are unknown.

Increased intelligence can help make super intelligence, but unlike think-ists, I don't see any evidence it will be sufficient alone. Thinkism is the primitive idea that the only thing you need to invent new things, to make breakthroughs, to bootstrap into a new level is a higher IQ. High IQ is necessary but not sufficient to accomplish science and technology. But without the time-consuming work of testing, gathering data in real life (which works in real time), building complicated apparatus which never work the first time, trying many things which don't work at all, IQ alone is not going to get you progress. At least that's the evidence so far. Can we invent a type of science that does not require gathering data? Possible. But the evidence so far is the opposite. Lately we've been doing a kind of science that has less theory and only data collection, so a data-less science remains a possibility without evidence.

We can accelerate data collection (as we have) and we will accelerate the creation of artificial intelligence (as we have) but increased IQ is only a small part of the innovations needed to accelerate it further. Thinkism is the crazy idea that higher IQ will solve all problems.

My Response:

I never said or implied that experimentation will be unnecessary for smarter-than-human AIs, nor do I know of any well known Singularity thinkers (Kurzweil, Yudkosky, etc.) who've said anything that implies that.

Their point, which I agree with, is that experimentation will be accelerated to such an extreme degree that it appears to be a difference in kind. In my response to your thinkism article, I emphasized that AIs will still need to experiment by likening the Singularity to the difference between chimps and humans. Both humans and chimps experiment with their environments, but humans do it so much better that when you compare them, chimps appear to be making virtually no progress. Despite the fact that chimps do in fact learn new concepts and technologies and culturally propagate them, humans have moved so much faster that chimps appear to be at a nearly complete standstill.

The superior intelligence of humans allows them to skip over thousands of iterations of experiments that chimps need to do when, for instance, solving a puzzle of how to remove a banana from a box. Whereas the human solves that puzzle nearly instantly with a single experiment (and moves on to experiment on more challenging problems) the chimp spends all day puzzling about and poking the box, engaging in thousands of little experiments before mastering the puzzle.

Similarly, smarter-than-human AIs will still need to experiment, but their experiments and their thoughts about the results will be so much more sophisticated than humans, that in comparison human science will appear to have been at a near standstill.

I coined the word "pinnacle-ism" to respond to your "thinkism", because your intuition seems to be that a similar increases in intelligence (as that from chimp to human) will not result in similar improvements in experimental efficiency. The only way I can really imagine this to be the case is if somehow humans have reached some sort of experimental pinnacle, past which profoundly greater intelligence will no longer result in profoundly better experiments and profoundly improved insights gained from those experiments.

In your thinkism article you say:

There is no doubt that a super AI can accelerate the process of science, as even non-AI computation has already speed it up. But the slow metabolism of a cell (which is what we are trying to augment) cannot be sped up. If we want to know what happens to subatomic particles, we can’t just think about them. We have to build very large, very complex, very tricky physical structures to find out. Even if the smartest physicists were 1,000 smarter than they are now, without a Collider, they will know nothing new. Sure, we can make a computer simulation of an atom or cell (and will someday). We can speed up this simulations many factors, but the testing, vetting and proving of those models also has to take place in calendar time to match the rate of their targets

But, both humans and chimps need to experiment with the world in real-time and yet we do see a Singularity-like difference in the move from chimp to human intelligence. Humans also need to wait for the results of experiments, and yet our experimental efficiency is so much greater than chimps that it appears to be a different thing entirely. Why? Because a human can integrate vastly more information, construct more sophisticated models that account for that information, identify some important model variations, and envision experiments that more effectively cut through the noise to find the most promising variation. A human skips more than 99% of the experiments that a chimp must iterate through before mastering a puzzle. (Not to mention the fact that for most problems, a chimp is simply incapable of conceiving of the proper models.)

Your claim is that similar increases in intelligence will not result in similar increases in experimental efficiency. Why, apart from pinnacle-ism, would that be the case?

Kelly's Response:

My guess: We are no-where near any pinnacle. The acceleration of intelligence will continue but it will be much much slower than the Singularitans believe; this difference will be significant, a matter of kind, not degree.

My Response:

So you think it will be some (undefined) degree slower than the Singularitans believe. Can't really argue with that. Who knows...

I do agree that Singularitan's tend to be overly optimistic. Particularly the Abundance crowd (e.g. Diamndis), who overlook the fact that material wealth has been rapidly increasing for centuries, but human consumption has always increased fast enough to leave many people wanting. I don't believe in the post-scarcity singularity. That is a fantasy that has never really been justified with any compelling theory.

Kelly's Response:

Whenever possible I prefer to base my expectations on data. I have been seeking evidence that artificial intelligence follows Moore's Law in the last 50 years but have found none. The inputs -- processor speed, number of transistors, links, you name it -- have all been increasing exponentially, but there is no evidence the output -- the IQ of machine learning-- has. Artificial smartness is increasing linearly -- at best. And I see no evidence of it becoming exponential in the future either. If you have evidence of such, that could change my mind.

My Response:

Hmm... hard to say given that we are still so far away from anything resembling intelligent computers (and regardless, we don't have any good metric of general intelligence). If cutting edge AI is currently about .001 as smart as a person, and it doubles to .002, I'm not sure we would even notice the difference. Twice as smart as really stupid is still really stupid.

The only example I can think of is the one I keep citing from the biological world: a 3 times difference in brain mass between chimps and humans has resulted in what seems like a much greater than 3 times difference in intelligence (at least when measure in terms of technological achievement).

I imagine the intelligence curve would have increasing returns to computational power over some portions, decreasing over others, and roughly constant over others. I don't know of any principled way to know where humans are on the curve, but given that a 3 times increase in brain mass has resulted in the difference between chimps and us, I don't see any reason to think we have reached a portion of the curve that has severely diminishing returns to computational power.

Kelly's Response:

Don't confuse brain mass with IQ. It doesn't tell you much. Whales have brains about 4 times as large as ours. They have more cortical convolutions, too. Neanderthal brains were larger than ours. The computers with the most chips, FLOPS, or neurons won't necessarily be the smartest. You overestimate the importance of chimps vs humans.

My Response:

Actually I was simplifying by saying brain mass because most people don't know what encephalization quotient is (see http://en.wikipedia.org/wiki/Encephalization_quotient). When measuring by encephalization quotient whales don't have very big brains (dolphins do, but still much smaller than humans).

Anyway, I agree it's not all about size, but there is a very strong relationship between brain size and intelligence in animals.

But, by no means was I confusing brain size with IQ. As I said, I expect that if you were to express the relationship between brain size (computational power) and IQ (intelligence) as a curve, there would be regions of diminishing returns, increasing returns and constant returns to brain size. So obviously the two are not synonymous.

I may be overestimating the importantance of chimps vs humans, but unfortunately, we don't really have any other examples of intelligence, or the development of intelligence. Since human and animal intelligence are the only things that we know of that exhibit real intelligence, I think it is useful to look at the development of human intelligence as a guide. I prefer to use the little data that we do have rather than not using any data at all.

See the previous two parts of this exchange here and here.

Kevin Kelly's theory of Pinnacle-ism

Kevin Kelly recently responded to my post about his post about thinkism with the following:

I appreciate your response. Your main point is a huge assumption with no evidence behind it and as fine example of thinkism that I've seen: 

But the more important point is that intelligence is probably a substantial research bottleneck right now in ways that are impossible for us to comprehend.

If you have evidence of this please point me to it. Otherwise it is just fantasy.

To characterize his position, I'll coin a new term called pinnacle-ism.  Let me explain.  To attempt to grasp how greater intelligence might effect technological progress I looked to the only examples of intelligence that we know of: existing biological intelligences such as humans and apes. There we see that a relatively modest increase in thinking matter (the brain) of about 3 times, has increased technological progress to such a drastic degree that it appears a difference in kind rather than degree. Chimps experiment with the world around them too, but their experiments are primitive and their ability to interpret the results are so feeble by comparison, that to us they appear to make virtually no progress (even though they do in fact learn new concepts and technologies and culturally propagate them).

So that is what a 3 times difference in thinking matter (plus unknown tweaks to how it is used) has produced. Kelly's theory is that another 3 times increase in thinking matter (or one thousand, million, billion or trillion times increase) would make essentially no difference. It may decrease the time between experiments, but it would not, for instance, radically alter which experiments we choose to run or the interpretation of the results, even though that is exactly what the previous 3 times increase did.

I admit that looking at a single example of the results of an increase in intelligence in order to predict the results of future increases constitutes only anecdotal evidence. But anecdotal evidence is still evidence. And, in fact you can also look to the difference between smaller brained monkey's and apes and find increases in brain size produced a similar difference in kind there as well (tools use, cultural propagation, etc).

So I have provided some evidence that intelligence is a bottleneck for technological progress. Kelly looks at the evidence that increased intelligence has drastically increased technological progress in the past and concludes that the previous instances will be the final instances: going forward increased intelligence will have minimal impact. But usually when interpreting evidence we look for patterns in the past to help us predict patterns in the future, and we demand evidence for assertions that future patterns will be very different from past patterns. So I would turn his challenge around: "If you have evidence of this please point me to it. Otherwise it is just fantasy."

At any rate, the position that greater intelligence has lead to rapidly increasing pace of technological progress in the past, but will not in the future, seems to imply that humans have reached the pinnacle of intelligence with regards to technological progress.  Whereas increases in intelligence up to the level of a human have profoundly affected the pace of progress, there isn't much further to go.  We humans are at the pinnacle.  Kelly appears to be a pinnacle-ist.  But where is the evidence for his pinnacle-ism?

See the post that precipitated this exchange here, and ensuing discussion between Kelly and I here.

More intelligent Singularity skepticism

Here are two criticisms of the Singularity that are more interesting than snidely assuming (without explanation) that we'll never be able to create machines as intelligent as humans.

In the first, the author Kevin Kelly supposes that even if we have super intelligent machines, it probably won't change very much because evolution of technology does not depend simply on intelligence, but also requires experimentation.  Even if we can speed up the pace of analysis of the results of experiments, we will still need to conduct the experiments.  In some cases, such as particle physics, conducting those experiments requires enormous construction projections.  If we simply cut out all the time between experiments that is currently occupied by human thought we would have a faster pace, but not that much faster.

On the surface the point seems vaguely plausible, but on deeper reflection, I think it's almost certainly wrong.  First of all, one implication of smarter-than-human machine intelligence is that labor will be drastically less expensive, particularly the highest levels of cognitive labor.  Building a super collider will be much less expensive and much faster when you don't have to pay a single human and instead have an army of unpaid robots working 24/7 on construction and design.  Another implication is that investigation will be much more parallelized.  Creating minds that can do PhD level analysis will not require 30 years of schooling.  There will be more expert level intelligence and experimentation applied to more different paths of inquiry all simultaneously.

But the more important point is that intelligence is probably a substantial research bottleneck right now in ways that are impossible for us to comprehend.  The human brain is only 3 times larger than a chimp brain, but that 3 times difference in size results in a nearly infinite difference in capability.  A 3 times difference in brain matter is the difference between sitting in the jungle eating bananas and achieving all the greatest and smallest feats of human civilization.  The most simple concepts that we take for granted are inherently beyond the reach of a chimp, which will never learn to read or write, do simple math, or program a computer, much less fly to the moon or build a skyscraper.

Saying that a computer with 3 times the capacity of the human brain is unlikely to make technological progress much faster than a human because it will still need time to carry out experiments, is like saying humans are unlikely to make technological progress much faster than chimps because we still need time to carry out experiments.  But this is obviously false because a chimp cannot conceive of and carry out the same kind of experiments, and even if it could it would have no idea what to make of the results.

Furthermore, we aren't talking about a computer with 3 times the capacity of a human.  We are talking about computers with a thousand (ten years after parity), million (twenty years after), billion (thirty years after) or trillion (forty years after) times the capacity of a human.

Thus, when you think about it in a little more detail, to presume that somehow human technological progress is moving nearly as fast as possible because a greater intelligence would still have to carry out experiments is absurd.  And we aren't merely talking about a doubling or tripling of pace.  A chimp could never build a sky scrapper no matter how many millennium it had to learn.  Nor are we talking about an increase in progress comparable to the difference between humans and chimps which corresponds to only a 3 times difference in brain matter.  We are talking about a difference in pace that is so vast it is essentially inconceivable.

So while I think the thinkism argument is interesting, I think it is almost certainly completely wrong.

The second criticism I think is more compelling.  Here Kelly is not criticizing the Singularity, but rather the likely timeline for its occurrence, noting that Singularity prognosticators curiously pick dates right around the time when they would be likely to die, thus perhaps providing them some comfort that it will arrive just in time to save their lives.  I don't really think there is a good response to this apart from, lets wait and see.  Nobody knows how long it will take to replicate human intelligence in machines.  Apart from saying that it is possible and will probably take less than 15,000 years, the best we can do is speculate based on our intuitions about the nature of intelligence and the progress of technology.  My speculations are more in line with Kurzweil's than with those that think we are centuries away.  Is that just wishful thinking, strongly influenced by the hope of immortality?  Perhaps.  But it is also my best guess.  From all the reading and thinking I've done on the subject, and from my decades of following the evolution of technology, that is my gut estimate.  Unfortunately, it is impossible to remove the influence of hopefulness (whatever that may be) on my estimate.

I do think it's worth noting however that the same thing can also work in reverse.  There are people whose attachment to the uniqueness and divinity of human cognition leads them to estimate that the Singularity is very far away.  There are people whose political agendas depend on a distant estimate for the arrival of the singularity (e.g. if it is coming in thirty years likes Kurzweil thinks, climate change probably shouldn't be quite as big a concern).

We all have our biases.  When I sit around and think about the most likely time for the Singularity, it seems about thirty to fifty years away, even if I'm trying really hard not to be influenced by my own hopes for immortality.  What else can I do?  We'll just have to wait and see.

See Kevin Kelly's response to this post here and the ensuing discussion between the two of us here.

The religion of Singularity

I quite enjoyed this article arguing that the singularity is merely a bizarre modern day tech religion. Obviously I don't think the author is right. But he's not entirely wrong either. The article’s singularity skepticism hinges on whether we will ever achieve human level artificial intelligence, and one reasonable data point someone might consider in that regard is the current level of artificial intelligence. Currently the most cutting edge AI is extraordinarily primitive compared to human intelligence, but perhaps we could get a rough sense by comparing to other animals.  It's pretty clear that our robotic dogs are only the palest shadow of a biological dog. My guess is that our most sophisticated AI techniques probably couldn't pilot a cockroach successfully through its day (even if we had a super computer small enough to put into the brain of a cockroach). On the other hand, there is no biological intelligence (that we know of) that can play chess as well as my $1,000 laptop, or Jeopardy as well as Watson. Right now, biological and artificial intelligences excel at different tasks. Nevertheless, apart from a few very narrow examples (such as chess and Jeopardy) computer intelligence does not appear close to equaling the most important abilities by which we typically define human intelligence.

Furthermore, there really isn't any way to know when computer intelligence will get there. Kurzweil takes some guesses that seem plausible to me, but seem outlandish to many people in and out of the field of artificial intelligence. Everyone who believes that human intelligence is computable has their own intuitions about when computers will first reach that level of intelligence. Ultimately, Kurzweil's estimate is based on informed intuition, but so are the estimates of those who think we are no where close. Only time will tell who is right. Personally, my intuition is not that far from Kurzweil's--maybe 10 or 20 years later than his estimate. I accept the fact that that is a guess. But does the author of the article accept that his estimate is a guess?

I think the one thing we can confidently say on this point is that the ones who are really religious are those who, based on faith, believe that human intelligence cannot be replicated through human invention. There are many varieties of that faith: from an actual belief in god and the immortal soul--implying that human intelligence is not a manifestation of a physical process, and thus cannot be replicated though a physical human invention--to a mere belief that humans aren't up to the task of creating something so marvelously complicated. But whatever the variety, all refutations of the singularity based on the premise that we cannot replicate human intelligence appear to be based on faith or dogma.

Hopefully it is obvious to most readers how a refutation of the singularity based on the immortal soul endowed by god rests entirely on unsubstantiated dogma. But what about the assertion that humans may simply not be intelligent or inventive enough to create something as marvelously complicated as the human brain? I think the weight of science clearly opposes that view, and to see why it is helpful to consider how the human brain was created to begin with. Evolution is one of the most strongly supported theories in all of science. Animal brains are organs evolved through a process of random variation and natural selection of those variations that most contributed to survival and reproduction. The human brain is merely an extreme example of an animal brain, and was thus created through the same process of random variation and natural selection. For the purposes of this discussion, one important aspect of that process is that the variation was random. There was no intelligent designer sitting around and thinking about the next best variation to try. A belief that humans are not intelligent enough to ever design something as intelligent as we are thus hinges on the assumption that despite all of our intelligence, our attempts to improve machine understanding will be worse than random.

I've never seen any evidence to support that position and there is quite a bit of evidence to the contrary. Whereas it took evolution billions of years to evolve even the most rudimentary intelligence, in the space of just one hundred or so years since the first principles of computing were articulated, human's have designed machines intelligent enough to safely drive cars, defeat the best human chess and jeopardy players, analyze and respond usefully to human language, coordinate complex movements of quadrupeds and bipedal robots for walking and running, and so forth. So far we seem to be progressing quite a bit more efficiently than we would through random variation.

Exactly how much more efficiently is a reasonable topic for debate. But it is not reasonable to propose that humans are simply incapable of reproducing human level intelligence because the task it too immense for our weak intelligence. Given that we seem to be progressing much faster than evolution did, it can be reasonably argued that it will take a long time, but not that it is impossible. The argument for impossibility appears to contradict what we've learned through science about the evolution of our own intelligence.

Though it’s impossible to say how much faster AI is progressing than biological intelligence evolved, it is interesting to speculate about. When estimating how long it has taken humans to design cutting edge AI, do you start counting at birth of the first modern human, or at the birth of the first computer? And how intelligent is our cutting edge AI? There isn’t much evidence that it is as smart as a cockroach, for instance, because it has never been used to do all the things that a cockroach does. On the other hand there is no biological intelligence that can play chess or jeopardy as well as a computer. So it is hard to measure how far we have come.

Just for kicks, lets suppose that our cutting edge AI is roughly as intelligent as a cockroach. Life started evolving on earth in the neighborhood of 4 billion years ago, cockroaches first appear in the fossil record about 300 million years ago, and humans first appear about 200 thousand years ago. Thus the time it took evolution to go from cockroach to human was about 7.5% of the time it took evolution to go from nothing to cockroach. If you figure that it will take proportionally as long for humans to create human level intelligence as evolution did, and you start timing how long we've been working on it from the appearance of the first humans, then you might guess it will take another 15,000 years. But if you start counting from when the first principles of computing were laid out roughly 100 years ago, then you might guess it will take in the neighborhood of another decade.

Which is the more reasonable place to start counting? I don't think there is a good answer to that question, but I'll indulge in some recreational speculation nonetheless. Perhaps we can approach that question by examining the critical differences between evolution as a process for developing intelligence versus human invention. With evolution we take (1) random variation and (2) select those that have the highest reproductive fitness. It’s important to recognize that reproductive fitness is not the same thing as intelligence. Reproductive fitness is affected by many different factors and in many cases greater intelligence puts constraints on an animal that decrease it’s fitness. For instance, the brains of some bird species, whose bodies are highly streamlined to improve flight efficiency, actually seasonally shrink to reduce overall weight. In that case the added intelligence of a bigger brain is not worth the added weight. Similarly, an important factor that influences brain size in all animals is the comparatively higher energy consumption of brain matter. The larger the brain, the more energy it uses, and the more likely an animal is to starve to death during food shortages. For these reasons evolution often selects for less intelligent animals, and intelligence is only loosely related to reproductive fitness.

The process by which evolution created human intelligence can be contrasted with the process by which human are working to create artificial intelligence. AI researchers are (1) experimenting with carefully chosen and designed variation, with (2) the explicit goal of maximizing intelligence. Thus (1) where variation in evolution is random, in AI research it is intelligently designed and (2) whereas the fitness function for evolution is only loosely related to intelligence, in AI research the fitness function is very closely related to intelligence.

The second point is worth considering in more detail. The first time humans started to use intelligence as the fitness function for their inventions was the birth of the field of AI research. Before that there was no AI of any sort, and after that rudimentary forms of AI started to develop. All the progress that we have made in the field of AI has been made since the creation of the field of AI research, and this kind of focused directive is something that has never been part of the process of biological evolution. Because this is really the start of where humans embarked on the creation of AI, this seems like a more natural place to draw the comparison, though some might argue that it makes more sense to go back a few thousand years to the first developments of math and logic. At any rate, if you count from the birth of the AI field, we are looking at perhaps another decade before human level AI, and if you count from the beginning of known western philosophy we are looking at perhaps another century (assuming we've reached cockroach level AI, and that going from cockroach to human level AI takes proportionally as long as going from cockroach to human did).

Of course we won’t know the right comparison until after we achieve human level AI, but to me it is interesting to consider nonetheless merely to get a sense of the range of plausible possibilities. To summarize, the only example we can use to estimate the difficulty of developing human level intelligence is the only example of human level intelligence that we know of: humans. After taking about 4.2 billion years to evolve cockroach intelligence, it took only another 300 million years to evolve human intelligence. If the same proportions held for human design of AI, and you start counting at (1) appearance of the first modern humans, (2) the first records of the development of math and logic, or (3) the development of the field of AI, then we should achieve human level AI in roughly 15,000 years, 100 years, or 10 year, respectively.

But, despite the fact that there is a wide range of reasonable possibilities for the timeline of achieving human level AI, all the arguments that I have come across that humans will never achieve AI appear to be based on dogma, and to contradict what we know about how human intelligence was created and how humans are researching AI.  Thus to the extent that the author of the the article purports to refute the Singularity based on the impossibility of creating human intelligence, he appears to be doing so based on the very kind of dogmatic irrational thinking that he falsely assumes Singularity believers are engaged in.

But I do think there is another interesting aspect to the article.  Once someone understands and adopts the notion of the singularity, it can play something akin to the role that religion plays in a religious person's life. It is an organizing principle. It provides structure and hope. It doesn't explain how the universe began, or how it will end, but it perhaps provides a glimmer of hope that we, or our future incarnations, may someday have more insight into those questions. It will not give us solace that our loved ones who have perished are happily looking down upon us from heaven, but it does hold out the tantalizing possibility of immortality for ourselves, and our living loved ones, and perhaps resurrection of our lost ones. It does promise some possibility of eliminating war, starvation, disease, poverty, etc. And to the extent that its promises are things we deeply want, it can provide some meaning and direction to our lives.

So I think what the article gets wrong is not so much equating the singularity to religion, but rather ridiculing it because of the parallels. We live in a world where it is acceptable to believe that when you take communion the cracker you eat turns into the flesh and blood of Jesus Christ in your body, but it is somehow fringe and bizarre to believe that human intelligence is the manifestation of physical processes that we will one day be able to replicate?

If we humans do not drive ourselves to extinction first, we will one day reach the singularity. It will change everything. Any understanding of our place in the universe and the meaning we find in our lives that does not recognize and embrace that is missing something important. It makes sense for the singularity to have an impact on our spiritual lives. It would be weird if it didn't.

Future of Life Institute and my discussion with Andrew McAfee

I recently attended The Future of Technology: Benefits and Risks, which was the inaugural event of the Future of Life Institute.  I was glad to see more evidence that these issues are being taken seriously by high profile thinkers.  The event consisted of presentations by each of six panelists, and then a discussion moderated by Alan Alda.  For anyone who has already been following this area of thought, there wasn't much in the way of new ideas presented, but the panelists were able to draw on interesting examples from their own fields.  The panelist's intuitions about the most pressing threats seemed to differ substantially from one another, and I got the impression that they are still at a very early stage in their thinking about these issues.

For me the most interesting part was my brief discussion with Andrew McAfee (author of Second Machine Age and Race Against the Machine) after the formal discussion period ended.  The idea of post scarcity economics came up and I wanted to get his reaction to my analysis implying that the notion of post scarcity economics is actually a critical conceptual mistake among singularity thinkers.  After a brief back and forth where he alluded to past mistaken predictions of resource constraints such as those by Malthus, McAfee exclaimed "I'm not concerned about resource constraints", and it was obvious to me that he meant that to end the conversation so he could answer other people's questions.

Admittedly, it's hard to summarize and understand a complex idea in just a few minutes, especially in a chaotic environment after a presentation where dozens of people are approaching panelists to ask follow up questions.  So perhaps he has more interesting things to say on the topic.  Nevertheless, his response reinforced my sense that there is an important blindness among thinkers in this area.

His apparent blindness is all the more striking given the topic of his two books with Erik Brynjolfsson which are in large part explorations of how technological changes are increasing inequality and will continue to do so unless we adopt policies to address those tendencies.  As he says, he is a "technology optimist", and I guess part of what that means for him is that he does not believe in long run resource constraints.

I think this may be related to my other criticism of his books.  For him, dealing with increasing inequality in the face of rapid technological change, is about adopting sensible policies within our existing economic system.  For instance, he says "We are also skeptical of efforts to come up with fundamental alternatives to capitalism. By ‘capitalism’ here, we mean a decentralized economic system of production and exchange in which most of the means of production are in private hands..."  While I agree with him about some of the virtues of capitalism, it is hard for me to imagine a sensible economic system that is still recognizable as capitalism in a world where robots are performing virtually all labor and humans are by and large unemployed.  I plan to write more about that in subsequent posts.

It seems that McAfee imagines that with some policy tweaks to capitalism we can solve the issues that arise when computers replace humans in the work force and capital becomes the only factor of production.  If we merely implement those tweaks effectively we can overcome the obvious tendency for capital accumulation to lead to radical inequality in a post-labor economy and thereby make it to a post-scarcity utopia, which will somehow still be capitalist.  This seems pretty far fetched to me.  Why should we organize our economic system around private ownership of productive capital in a world where computers are superior to humans for nearly all productive work--including determining the most efficient ways to invest productive capital?

I think part of the reasoning for his position must have to do with a focus on the pre-singularity.  He is concerned with the period of time where humans retain an edge in some subset of productive activities.  Or perhaps he is assuming a pre-singularity merger of humans and computers such that the distinction between what computers can do and what people can do becomes less meaningful.  Perhaps he believes that if we are ultimately headed towards a world that loses the distinction between human and computer work, then it continues to make sense to have humans using their self interested judgement for how to invest their productive capital, for all the same reasons that we think capitalism is the least bad system right now.  Who knows.  He doesn't say in his books.  He just says, lets keep capitalism because past efforts to do something better have failed... even though many of the factors that make capitalism effective are set to radically change.  That is a profoundly unconvincing position.

Perhaps if I got to talk to a him a little longer he would have been able to fill out his vision in a more compelling manner (though I don't think he does that in either of his books on the subject).