New computer system solves algebra word problems

Researchers have developed a new computer system that can automatically solve the type of word problems common in introductory algebra classes.
The work could lead to educational tools that identify errors in students' reasoning or evaluate the difficulty of word problems, researchers said.
It may also lead to systems that can solve more complicated problems in geometry, physics, and finance.
The computer system was developed by researchers in Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Laboratory, working with colleagues at the University of Washington.
The researchers' system exploits two existing computational tools. One is the computer algebra system Macsyma, whose initial development at MIT in the 1960s was a milestone in artificial-intelligence research.

For this research, Macsyma provided a way to distill algebraic equations with the same general structure into a common template.

The other tool is the type of sentence parser used in most natural-language-processing research. A parser represents the parts of speech in a given sentence and their syntactic relationships as a tree - a type of graph that, like a family-tree diagram, fans out at successive layers of depth.
For the researchers' system, understanding a word problem is a matter of correctly mapping elements in the parsing diagram of its constituent sentences onto one of Macsyma's equation templates.
To teach the system how to perform that mapping, and to produce the equation templates, the researchers used machine learning.

Researchers found a website on which algebra students posted word problems they were having difficulty with, and where their peers could then offer solutions.
From an initial group of roughly 2,000 problems, they culled 500 that represented the full range of problem types found in the larger set.
In a series of experiments, the researchers would randomly select 400 of the 500 problems, use those to train their system, and then test it on the remaining 100.
For the training, however, they used two different approaches. In the first approach, they fed the system both word problems and their translations into algebraic equations - 400 examples of each.

But in the second, they fed the system only a few examples of the five most common types of word problems and their algebraic translations. The rest of the examples included only the word problems and their numerical solutions.
In the first case, the system, after training, was able to solve roughly 70 per cent of its test problems; in the second, that figure dropped to 46 per cent.
But according to Nate Kushman, an MIT graduate student in electrical engineering and computer science and lead author on the new paper, that's still good enough to offer hope that the approach could generalise to more complex problems.