The study suggests that the lower the fitness level, the faster the deterioration of vital nerve fibres in the brain.
This deterioration results in cognitive decline, including memory issues characteristic of dementia patients.
"This research supports the hypothesis that improving people's fitness may improve their brain health and slow down the aging process," said Kan Ding, from University of Texas (UT) Southwestern Medical Center in the US.
The study published in the Journal of Alzheimer's Disease focused on a type of brain tissue called white matter, which is composed of millions of bundles of nerve fibres used by neurons to communicate across the brain.
The researchers determined that lower fitness levels were associated with weaker white matter, which in turn correlated with lower brain function.
Unlike previous studies that relied on study participants to assess their own fitness, the new research objectively measured cardiorespiratory fitness with a scientific formula called maximal oxygen uptake.
Scientists also used brain imaging to measure the functionality of each patient's white matter.
Patients were then given memory and other cognitive tests to measure brain function, allowing scientists to establish strong correlations between exercise, brain health, and cognition.
The study adds to a growing body of evidence pointing to the benefits of exercising regularly.
"Evidence suggests that what is bad for your heart is bad for your brain. We need studies like this to find out how the two are intertwined and hopefully find the right formula to help prevent Alzheimer's disease," said Rong Zhang of UT Southwestern.