New algorithm can help us 'see' the time

Time flies but can we see how time moves? Researchers believe a new algorithm can - with roughly 80 percent accuracy - determine whether a given snippet of video is playing backward or forward.

By identifying subtle but intrinsic characteristics of visual experience, the research could lead to more realistic graphics in gaming and film.

"If you see that a clock in a movie is going backward, that requires a high-level understanding of how clocks normally move," said William Freeman, a professor of computer science and engineering at Massachusetts Institute of Technology (MIT).

To study shape perception, you might invert a photograph to make everything that is black white, and white black, and then check what you can still see and what you can not.

"Here we are doing a similar thing, by reversing time, then seeing what it takes to detect that change. We are trying to understand the nature of the temporal signal," Freeman added.

Einstein's theory of relativity envisions time as a spatial dimension like height, width, and depth.

But unlike those other dimensions, time seems to permit motion in only one direction: forward.

This directional asymmetry - the "arrow of time" - is something of a conundrum for theoretical physics.

To unlock this, Freeman and his collaborators designed algorithms that approached the problem in three different ways.

All three algorithms were trained on a set of short videos that had been identified in advance as running either forward or backward.

The algorithm that performed best begins by dividing a frame of video into a grid of hundreds of thousands of squares.

Then it divides each of those squares into a smaller, four-by-four grid.

For each square in the smaller grid, it determines the direction and distance that clusters of pixels move from one frame to the next.

The researchers divided their training data into three sets, sequentially training the algorithm on two of the sets and testing its performance against the third.

"The algorithm's success rates were 74 percent, 77 percent, and 90 percent, respectively," Freeman said.