While computers have gotten incredibly adept at churning through large amounts of data, human intuition is usually needed to analyze that information and to put it in context. Well, for the moment, at least, but researchers from MIT are working on an algorithm that could change all that.
According to Gizmodo and Slashgear, Max Kanter and colleagues from the institute’s Computer Science and Artificial Intelligence Laboratory are working on a new computer system capable of detecting interesting or unusual patterns hidden in massive databases. Early tests of the computer show that it is more than a match for human intuition.
In fact, when matched against people in a trio of big-data analysis contests, MIT’s so-called Data Science Machine outperformed its flesh-and-blood counterparts, achieving better scores than 615 of the 906 teams in the competition. It was 94 and 96 percent as accurate as humans in two of the three games, and while it was only 87 percent as accurate in the third, it needed only a fraction of the time to complete its analysis (2-12 hours compared to months for the human teams).
“We view the Data Science Machine as a natural complement to human intelligence,” Kanter, whose master’s thesis in computer science inspired the Data Science Machine, explained in a statement. “There’s so much data out there to be analyzed. And right now it’s just sitting there not doing anything. So maybe we can come up with a solution that will at least get us started on it, at least get us moving.”
Algorithms could help speed up big data analysis
Kanter and his thesis adviser Kalyan Veeramachaneni, a research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), used several different techniques to give the Data Science Machine the ability to simulate human intuition, Slashgear said. For instance, it uses structural relationships in databases as hints to find relationships between information.
The researchers are currently using the technology to deal with problems such as learning which students are most likely to drop out of online courses. When it comes to predicting the dropout rate, the machine determined that two key indications were how long before a deadline students began working on a problem set, and the amount of time spent on a course website compared to other students in the same class. Neither statistic is officially recorded, but both can be inferred.
“What we observed from our experience solving a number of data science problems for industry is that one of the very critical steps is called feature engineering,” said Veeramachaneni. “The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas.”
While Kanter and Veeramachaneni admit that it is unlikely that this algorithm will ever be able to fully replace human intuition, it could speed up analyses of large collections of data, according to Gizmodo.
As Kanter explained, “There’s so much data out there to be analyzed, and right now it’s just sitting there not doing anything. So maybe we can come up with a solution that will at least get us started on it, at least get us moving.”
—–
Feature Image: Thinkstock
Comments