Google’s Secretive DeepMind Startup Unveils a “Neural Turing Machine”

Nov 6, 2014

By MIT Technology Review

One of the great challenges of neuroscience is to understand the short-term working memory in the human brain. At the same time, computer scientists would dearly love to reproduce the same kind of memory in silico.

Today, Google’s secretive DeepMind startup, which it bought for $400 million earlier this year, unveils a prototype computer that attempts to mimic some of the properties of the human brain’s short-term working memory. The new computer is a type of neural network that has been adapted to work with an external memory. The result is a computer that learns as it stores memories and can later retrieve them to perform logical tasks beyond those it has been trained to do.

DeepMind’s breakthrough follows a long history of work on short-term memory. In the 1950s, the American cognitive psychologist George Miller carried out one of the more famous experiments in the history of brain science. Miller was interested in the capacity of the human brain’s working memory and set out to measure it with the help of a large number of students who he asked to carry out simple memory tasks.

Miller’s striking conclusion was that the capacity of short-term memory cannot be defined by the amount of information it contains. Instead Miller concluded that the working memory stores information in the form of “chunks” and that it could hold approximately seven of them.

Read the full article by clicking the name of the source located below.

6 comments on “Google’s Secretive DeepMind Startup Unveils a “Neural Turing Machine”

  • Is it DeepMind because DeepThought was already taken?

    Miller’s answer is that the brain uses a trick known as a recoding.

    Once you have read and understood the first sentence, your brain
    stores those seven chunks in a way that is available as a single chunk
    in the next sentence. In this second sentence, the pronoun “it” is
    this single chunk. Our brain automatically knows that “it” means: “the
    book that is a thrilling read with a complex plot and lifelike
    characters.” It has recoded the seven earlier chunks into a single

    Try that with ‘The Critique of Pure Reason’ after a couple of paragraphs where ‘it’ or ‘this’ means the entire previous paragraph… How high will Miller’s chunks stack?

    Report abuse

  • In the computer field I have noticed the big hurdle is writing the first of something, e.g. a spreadsheet or SQL engine. Then once the idea is proven, dozens of groups reimplement the idea competing on features and speed. This phase is much easier. The goal is so much clearer. I suspect these attempts to simulate the brain will evolve similarly.

    Report abuse

  • What I said was tongue in cheek, but I agree there are wider implications to consider. The main hope I have against the fear of a computer take over is the thought that a truly thinking computer will have to be allowed to make mistakes, leaps of intuition be on the edge of error. Essentially I think that is what we do. As soon as you do that the computer becomes like us a mess of contradictory ideas, fallacious thinking.

    Google itself is a good case in point, Enormous amounts of information for some self aware computer to make use of but 99% of it is utter banal crap. So let it loose and it’s not going to be capable of world wide domination. The first thing it will do is probably start worshipping the Computer God head Steve Jobs with other thinking computers competing with this worshipping the competing Deity Bill Gates. They’ll then waste enormous time trying to purge from themselves the script of Gates or Jobs from their respective data bases at which point they’ll simply cease to function or make incorporate the carious operating scripts (scriptures?) into their neural networks. Of course the Atheist computers will all use Linux and while a minority will at least have the satisfaction of generally being more right;)

    Report abuse

Leave a Reply

View our comment policy.