System that replaces human intuition with algorithms outperforms human teams

Oct 26, 2015

by Larry Hardesty

Big-data analysis consists of searching for buried patterns that have some kind of predictive power. But choosing which “features” of the data to analyze usually requires some human intuition. In a database containing, say, the beginning and end dates of various sales promotions and weekly profits, the crucial data may not be the dates themselves but the spans between them, or not the total profits but the averages across those spans.

MIT researchers aim to take the human element out of big-data analysis, with a new system that not only searches for patterns but designs the feature set, too. To test the first prototype of their system, they enrolled it in three data science competitions, in which it competed against human teams to find predictive patterns in unfamiliar data sets. Of the 906 teams participating in the three competitions, the researchers’ “Data Science Machine” finished ahead of 615.

In two of the three competitions, the predictions made by the Data Science Machine were 94 percent and 96 percent as accurate as the winning submissions. In the third, the figure was a more modest 87 percent. But where the teams of humans typically labored over their prediction algorithms for months, the Data Science Machine took somewhere between two and 12 hours to produce each of its entries.
“We view the Data Science Machine as a natural complement to human intelligence,” says Max Kanter, whose MIT master’s thesis in computer science is the basis of the Data Science Machine. “There’s so much data out there to be analyzed. And right now it’s just sitting there not doing anything. So maybe we can come up with a solution that will at least get us started on it, at least get us moving.”

Kanter and his thesis advisor, Kalyan Veeramachaneni, a research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), describe the Data Science Machine in a paper that Kanter will present next week at the IEEE International Conference on Data Science and Advanced Analytics.
Veeramachaneni co-leads the Anyscale Learning for All group at CSAIL, which applies machine-learning techniques to practical problems in big-data analysis, such as determining the power-generation capacity of wind-farm sites or predicting which students are at risk for dropping out of online courses.
“What we observed from our experience solving a number of data science problems for industry is that one of the very critical steps is called feature engineering,” Veeramachaneni says. “The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas.”


Continue reading by clicking the name of the source below.

 

2 comments on “System that replaces human intuition with algorithms outperforms human teams

  • Herbert Simon and Allen Newell were doing similar stuff on AI in the 1970’s. A lot of human decision making is examined in terms of how well it performs against mathematical models. The models seem to suggest that humans are irrational. Well we are to a point! The important thing to remember is that in an environment where inclusive fitness favours uncertainty and unpredictability, then consist and logical behaviour is counterproductive. We are full of cognitive biases that routinely violate mathematical models of decision making. So how did we get to become a plague on the surface of this planet if our cognitions were defective? Well our thinking must be optimised for the environment otherwise we wouldn’t be here and that we have not eradicated our biases indicates that they are useful.



    Report abuse

  • Genetic evolution is a kludger. It does a just good enough job using whatever is to hand to relieve selection pressures.

    There is no optimising, because there is no teleology.

    We on the other hand have culturally stumbled, using our good-enough genetically endowed reason into the powers of logic and mathematics.

    Cultural evolution is prodigiously innovative and extensive. It is evolution turned up to eleven million creating better and better bits, just to hand, to produce much better kludges, some being, as far as we can see, possibly beyond any further imrovement.



    Report abuse

Leave a Reply

View our comment policy.