The first AI blog was written by a major, highly respected figure in the field. It consisted, as a blog should, of a series of short essays on various subjects relating to the central topic. It appeared in the mid-80s, just as the ARPAnet was transforming over into the internet.

The only little thing I forgot to mention was that it didn’t actually appear in blog form, which of course hadn’t been invented. The WWW didn’t appear until the next decade. It appeared in book form, albeit a somewhat unusual one since it was, as mentioned, a series of short essays, one to a page. It was, of course, Marvin Minsky’s Society of Mind.

Of course, you’re reading a blog about AI right now. The difference is that that was Minsky, and this is merely me. If you haven’t read SOM, put down your computer and go read it now.

Good. You’re back.  Here’s why SoM is relevant to our subject of whether and how soon AI is possible:

It remains a curious fact that the AI community has, for the most part, not pursued Society of Mind-like theories. It is likely that that Minsky’s framework was simply ahead of its time, in the sense that in the 1980s and 1990s, there were few AI researchers who could comfortably conceive of the full scope of issues Minsky discussed—including learning, reasoning, language, perception, action, representation, and so forth. Instead the field has shattered into dozens of subfields populated by researchers with very different goals and who speak very different technical languages. But as the field matures, the population of AI researchers with broad perspectives will surely increase, and we hope that they will choose to revisit the Society of Mind theory with a fresh eye.  (Push Singh — further quotes from the same source)

In other words, here’s a comprehensive theory of what an AI architecture ought to look like that is the summary of the lifework of one of the founders and leaders of the field, and yet no one has seriously tried to implement it.  (When I say serious, I mean put as much effort into it as has gone into, say, Grand Theft Auto.)  (There has been a serious effort to implement the theoretical approach of the CMU wing of classical AI, namely SOAR.)

Part of the reason for this is that SoM is in some sense only half a theory:

Minsky sees the mind as a vast diversity of cognitive processes each specialized to perform some type of function, such as expecting, predicting, repairing, remembering, revising, debugging, acting, comparing, generalizing, exemplifying, analogizing, simplifying, and many other such ‘ways of thinking’. There is nothing especially common or uniform about these functions; each agent can be based on a different type of process with its own distinct kinds of purposes, languages for describing things, ways of representing knowledge, methods for producing inferences, and so forth.
To get a handle on this diversity, Minsky adopts a language that is rather neutral about the internal composition of cognitive processes. He introduces the term ‘agent’ to describe any component of a cognitive process that is simple enough to understand, and the term ‘agency’ to describe societies of such agents that together performs functions more complex than any single agent could.

… but SoM doesn’t have a lot to say about what the individual functions are or how implemented, outside a few examples.  Since AI has for the past few decades concentrated on immediate results, most of the work has been on parts of the problem that could be described as stuff that would be inside a single agent, or at most an agency.

A good example of this happened a few years ago with the winning of the DARPA Grand Challenge and thus the development of the self-driving car. A few months after that happened, I was having a conversation with an AI researcher at a conference.  I maintained that the difference between the results of the first and second races — nobody got more than a mile or so, and then a couple years later several cars finished the whole 130-mile course –  represented real progress.  He pooh-poohed the idea.  All the techniques used in the cars had been previously known and published, he said.  All that had happened was that they had been integrated together into a working system.

I think this attitude goes a long way to explaining the lack of work on SoM and other overall cognitive architecture theories.  But as I reasoned previously:

The difference was, the Wright brothers knew an extra Good Trick, which was how to control the plane in the air once it was flying.
So to develop a working AI, we need the power, which we don’t think is going to be a problem. We need the lift, which is the kind of techniques found in narrow AIs and discussed above. And finally we need the control.

SoM represents a theory of how the control might work.  Where does that leave us?  Can we simply take Minsky’s books and papers and build an AI with all the existing narrow skill programs acting as agents? Hardly.  There’s a lot of work to be done, and probably several new Good Tricks left to be found.

The bottom line, though, is that we are not facing a blank wall.  We are facing a corridor with a sign reading “This way to the egress.”  Indeed we are partway down the corridor already; robotics and self-driving cars have required the development of integrated cognitive architectures along the lines that will probably lead to success.  Note that Brooks’ subsumption architecture had a lot in common with SoM.

So there is at least a case to be made that we are into the home stretch.  Of course that’s where the race really heats up and all the excitement happens…

So far, in making my case that AI is (a) possible and (b) likely in the next decade or two, I’ve focused on techniques which are or easily could be part of a generally intelligent system, and which will clearly be enhanced by the two orders of magnitude increase in processing power we expect from Moore’s Law by 2020.  (Note — we certainly don’t have to wait till 2020 to find out.  Existing hardware is well into the usable range, probably for less than $1M.  But you don’t get too many researchers, and no hobbyists, doing their research on machines like that today. You will in 2020.)

To make a heavier-than-air airplane fly, you need an engine.  If you have an airframe with lift-to-drag ratio r, stall speed s, and weight w, and a propellor with thrust efficiency e, you need an engine with power p=swr/e to fly. Power<p, no fly. Power>p, fly.

Both of the major American flying machine efforts understood this.  Langley spent huge effort developing light, powerful engines.  The brothers Wright built their own aeroengine from scratch in their bicycle shop.

The difference was, the Wright brothers knew an extra Good Trick, which was how to control the plane in the air once it was flying.

So to develop a working AI, we need the power, which we don’t think is going to be a problem. We need the lift, which is the kind of techniques found in narrow AIs and discussed above. And finally we need the control.

What I just said is an example of reasoning by analogy.  To an extent much greater than usually realized, most cognition and reasoning is based on analogy.  When you perform a physical skill, the specific sequence of sensory and motor signals is never exactly any of the ones that happened during practice; but they’re close enough that the mapping is straight-forward.

This is something that is well-known to the AI mainstream:

But “the big feature of human-level intelligence is not what it does when it works but what it does when it’s stuck,” Minsky said. When faced with novelty, Minsky claims, human intelligence applies “reasoning by analogy” to make the most direct tap into the cognitive glue that fuses knowledge domains.
Reasoning by analogy is a way of adapting old knowledge, which almost never perfectly matches the present situation, by following a recipe of detecting differences and tweaking parameters. It all happens so quickly that no “thinking” seems to be involved.  (EE Times)

The particular kind of reasoning by analogy that would make an associative memory machine work well can be called analogical quadrature.  This is the form of problem done most famously by Melanie Mitchell’s Copycat program: you have three things A, B, and C, and you want to find a fourth D such that A:B::C:D.  In the associative memory scheme, you need to do not the actual action you did in the memory, but the action that fits the current situation the way the remembered action fit the remembered situation.

As a simple example, if the remembered action was done by someone else, the parallel could be mapping things so that the action is done by you this time. In other words, analogical quadrature enables imitation.

If you can somehow represent your concepts as points in an n-dimensional space, analogical quadrature is falling-down easy: D=C+B-A in ordinary vector algebra. Of course, sometimes the mapping into n-space is problematical, and we are thrown back on symbolic methods such as those of the FARGitecture.

Those have their own problems, essentially the same ones as any symbolic AI: the operations and ontology in, e.g., Copycat are all idiosyncratic and hand-coded, and there’s no clear way to build a learning machine that extends them automatically.

I’ll go out on a limb and guess that the ultimate solution will involve elements of both extremes.  Search will be needed both to find new operations for symbolic formulations, and to find appropriate mappings into n-space for the subsymbolic ones.  A few key insights — new Good Tricks — will be necessary to unify the known methods and give us a solid understanding of, and engine for, analogical quadrature.  That’ll be a huge step towards general AI.

AI researchers in the 80s ran into a problem: the more their systems knew, the slower they ran.  Whereas we know that people who learn more tend to get faster (and better in other ways) at whatever it is they’re doing.

The solution, of course, is: Duh. the brain doesn’t work like a von Neumann model with an active processor and passive memory.  It has, in a simplified sense, a processor per fact, one per memory.  If I hold up an object and ask you what it is, you don’t calculate some canonicalization of it as a key into an indexed database. You compare it simultaneously to everything you’ve ever seen (and still remember).  Oh, yeah, that’s that potted aspidistra that Aunt Suzie keeps in her front hallway, with the burn mark from the time she …

The processing power necessary to to that kind of parallel matching is high, but not higher than the kind of processing power that we already know the brain has.  It’s also not higher than the processing power we expect to be able to throw at the problem by 2020 or so.  Suppose it takes a million ops to compare a sensed object to a memory.  10 MIPS to do it in a tenth of a second.  A modern workstation with 10 gigaops could handle 1000 concepts. A GPGPU with a teraops could handle 100K, which is still probably in the hypohuman range.  By 2020, a same priced GPGPU could do 10M concepts, which is right smack in the human range by my best estimate.

Associative memory gets you a lot.  You don’t have to parse an unknown object for algorithmic retrieval.  You don’t have to come with some one-size-fits-all representation and/or classification scheme.  Indeed, each object in memory can have its own representation if necessary or useful.

It gets better.  The memories aren’t all, or even mostly, objects.  They’re typically actions.  Let’s suppose the actions are represented as situation-action-resulting situation triples — something like Minsky’s trans-frames.  Then we can use the associative memory to

• recognize, as described above
• predict: search on the situation and action; the prediction is the result in the best match
• plan: match on situation and desired result; do the action from the best match
• generalize: every time a was done, b happened
• model: by chaining predictions, etc

There was an attempt to do this kind of thing in mainstream AI under the name “case-based reasoning” a couple of decades ago, but it appears to have foundered for several reasons, not least of which was the inability to do heavy-duty parallel matching on extensive memory sets.

There are a number of things that need to be added to the scheme for it to be useful and robust, like embedding it in a hierarchical, multiagent architecture, the ability to do analogical quadrature, and the ability to find useful representations.  But that’s for another post.

Bayer (the same company that makes the aspirin) is now beginning to manufacture multi-walled carbon nanotubes in industrial quantities.  The pilot plant will produce 200 tons per year, and the market is expected to grow at 25% per year.

The MWCNTs are for materials use, meaning mostly fiber-reinforced composites, e.g. airplanes, tennis racquets, arrows,

and the like.  The major advantages over conventional polymers / fibers is that the CNTs are stronger and conductive (both electrically and thermally) — producing a plastic that is more like metal in many ways, but still much lighter.  The conductivity is supposed to be comparable to copper, i.e. good enough to use as wiring in many applications.  Looking at the data for CNTs as a polymer additive, the major effect on mechanical properties was to make them less stretchy (and about 10% stronger), while having a major effect on conductivity properties.  Nobody has yet, as far as I know, managed to figure out how to make a composite that has the really high tensile strength possibilities of the raw nanotubes.  Alternatively, CNTs in light metal matrices such as aluminum or magnesium seems to have significant possibilities.  Time will tell — but there’s still a major advance to be made.

The individual CNTs in the mix are on average 8 or so walls, 15 nm diameter, and over a micron long (i.e. an aspect ratio of at least 60 and probably in the hundreds).

Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!

So we will take it as given, or at least observed in some cases and reasonably likely in general, that AI can, at the current state of the programming art, handle any particular well-specified task, given enough (human) programming effort aimed at that one task.

We can be a bit more specific about what “well-specified” means.  In general, if the task has a static ontology that can be laid out by the programmers, it’s within the scope of current practice.  A huge part of the progress of early AI was in fact simply building up (hand-made) ontologies.  An ontology includes, BTW, not just a list of concept names, but the semantics: code to recognize, predict, simulate, and perform whatever things we’d expect a person to be able to do who we would describe as “understanding” the concept.

The difference between this “static AI” and real human-level intelligence is that people learn new concepts constantly.  We will learn several words a day our entire lives (estimates range from 1 to 10 and of course this depends on individual intelligence and environment). Concepts are constantly changing and growing, splitting and merging, being half-forgotten and rediscovered.

Not only the ability to create new concepts, but the fluidity and adaptability of the ones we already have, enable the robustness of human intelligence.

There’s been a lot less research on how to build concepts than there has been involving the formalization of existing ones in static form.  There’s a bias toward the latter since you get a machine that can do something useful much quicker that way.

However, there has been research in creating new concepts and we can say something about it.  It seems to be the area where the high computational resources make a difference.  The most general approach we have is search, in various forms. Deep Blue invented startling new chess strategies on the fly.  These robots evolved a number of concepts through simulated evolution.

(ps — if you want your research paper to be picked up by the pop-sci news and blogosphere, simply include the words “robot” and “predator” in it )

Going back to Lenat’s AM, it’s been understood that search, in various forms, is capable of the kind of learning we need, but also that it tends to run out of steam sooner rather than later.  In other words, it seems likely that a properly set-up search is capable of inventing a fairly sophisticated concept, but you need another setup for the next one.  It’s generally accepted that some sort of evolutionary search is going on in the brain, but the system that controls it, sets up the search spaces, defines the fitness functions, and so forth, is definitely not well understood.

Thus the key to understanding when and whether general AI can happen lies in the high-level organization that can guide the application of focused search to produce a growing set of concepts that work coherently together.

The brothers Montgolfier invented the hot air balloon upon the observation that smoke rises, and thus they figured that if they could catch it in a bag, the bag would be pulled upward.

Hot air ballooning is quite popular today; people think of balloons as being quaint and pretty and natural, or at least more natural than airplanes.

Actually, a modern hot-air balloon uses more fuel than an airplane does to fly the same payload for the same time. The reason, of course, is that hot air needs to be hot, but the balloon needs to be light, so that the material needs to be thin, which means in practice that heat is lost through the balloon, and needs to be regenerated by burning fuel.

With nanotech we could make a fabric of diamond sheets for strength, with vacuum for insulation, and thin metallic films (or graphene sheets) to reflect thermal radiation. That means that we could have a balloon that was much lighter than woven nylon, and yet enormously better insulated.

The air in a balloon may be typically heated to around 100C, making it 0.93 kg/m^3 (compared with 1.2 at 20C). Call it roughly 0.25 kg/m^3 lifting capacity.

But if we can insulate it, we could fill it with steam instead (at the same temperature). (Steam would condense on the walls of an uninsulated bag.) Steam at 100C has a density of 0.59, call it 0.6 kg lifting capacity. Since we aren’t losing heat (much), we could superheat it and get some extra lift, say a few 0.1kg/m^3, but there would likely be a tradeoff with insulation weight, energy rates to cover leakage, etc. Even without it, the balloon lifts its own weight, including the water in the steam (and probably 100 times that of just the balloon).

(Postscript: It’s occasionally assumed that diamond is strong enough to make air-buoyant vacuum-filled balloons. This doesn’t actually work. Hydrogen remains the champ, with a density of 0.09 kg/m^3, which is essentially negligible. But even so, a steam balloon would only have to have twice the volume of a hydrogen balloon with the same lift, as compared to 5 times for hot air.)

We’ve had a fair amount of interest in the Kartik M. Gada Humanitarian Innovation Prizes, mostly from RepRap types. They pointed out that we had a slight incompatibility in the specification of the open source requirements with those of the RepRap community itself. We’ve changed the requirements to allow either BSD or GPL.

To make a donation to the Gada Prize fund, click thru to the prize page, click on the “Join Now” button to the right, go to the Donation section, and select “Gada Prize” from the project pulldown.

At the Foresight congerence special-interest lunch on IQ tests for AI, Monica Anderson suggested a test involving separating text which had had spaces and punctuation removed, back into words.  As a somewhat whimsical version of the test, I suggested the Sigil of Scoteia:

The Sigil of Scoteia

In case you’re unfamiliar with it, it’s the frontispiece of the novel The Cream of the Jest by James Branch Cabell.  Why does the Sigil make a good AI test?

One reason is that it requires a considerably more holonic interpretation process than just separating the text would.  It’s in a handwritten script in an extremely idiosyncratic font — you need to have a good guess what the word is to figure out what the letters are, and vice versa.  Information must flow down as well as up the interpretation stack.  It takes a few minutes to figure out the Sigil; you can read the jammed-up letters version straight off:

|IAMESBRAN
CHCABELLMADETHISB
OOKSOTHATHEWHOWILLSMAY
READTHESTORYOFMANSETERNA
LLYUNSATISFIEDHUNGERINSEAR
CHOFBEAUTY|ETTARRESTAYSINACCE
SSIBLEALWAYSANDHERLOVLINES
SISHISTOLOOKONONLYINHISDRE
AMS|ALLMENSHEMUSTEVADEAT
THELASTANDMANYARETHE
WAYSOFHEREVASION

(stumbling perhaps over the name of one of the characters near the middle).

But then, once you have the words, you’ve only gotten started.  The test isn’t “separate this into words” — it’s “what does this mean?”

You could work out the words but not be able to explain them in context. You might be able to tell what the Sigil was physically in the book but be completely clueless as to its emotional meaning.  I claim that the question “what does this mean?” has different answers at every point across the diahuman range of intelligence.  Cabell was an unexcelled master of cryptic, poetic, romantic fantasy, based on a very thorough knowledge of mythology and keen insight into human nature.  Think of him as Tolkein multiplied by James Joyce, filtered through the light touch of Wodehouse.

Thus actually to “get it” with Cabell, you have to be able to understand things on lots of different levels at once.

I often feel smug at the dumbness of people in the sense that surely AI must be a low bar — how hard could it be to beat that, whatever that might have happened to be.  But there are other times when, contemplating people like Cabell, I feel like giving up, it’s just too hard.

If you can work out the words in the Sigil, you’re at the hypo/diahuman border.  If you can write a book like Cream of the Jest, you’re at the dia/epihuman border.

After months of trying, NASA is calling it quits on freeing the Spirit rover from the Martian sand that it’s been stuck in since May of 2009.  Unfortunately, after six years of tireless service, the end might be very near for the rover, which faces a severe Martian winter in its current position. NASA engineers will spend the next few weeks preparing Spirit to face the winter weather, and hope that it will be able to continue on as a stationary scientific platform.

“Spirit is not dead; it has just entered another phase of its long life,” said Doug McCuistion, director of the Mars Exploration Program at NASA Headquarters in Washington. “We told the world last year that attempts to set the beloved robot free may not be successful. It looks like Spirit’s current location on Mars will be its final resting place.”

The Martian winter will begin in May. Until then, NASA will try and use remaining power to change the inclination of Spirit in order to help it capture more sunlight. NASA says that unless Spirit can be positioned in a better position, it is unlikely that it will survive.

Meanwhile, Opportunity, Spirit’s sister, continues to amble onwards towards a crater called Endeavor. NASA has some good videos summarizing Spirit’s six years.