Damon, 42, said he has been able to keep his personal life mostly out of the public eye because he married a civilian, reported Contactmusic.
"I remember telling him that I walk my kids to school, and his face just fell. Because he should be able to do that, too. And he can't.
"I got lucky. I fell in love with a civilian. Not an actress and not a famous actress at that. Because then the attention doesn't double - it grows exponentially. Because then suddenly everybody wants to be in your bedroom," Damon said.
The actor also revealed that he and his family are about to move to Los Angeles, to a home just down the street from his best friend Ben Affleck, with whom he is starting a production company.