The deceased, from Punjab province, was working as a cook in the Pakistan Navy, police said.
"We are still investigating whether it was a sectarian killing or a case of a robbery," police official Imran Shaukat said.
The navy personnel was rushed to a hospital where he succumbed to his wounds during treatment.
This is the fifth such incident targeting a naval personnel in Karachi since last year.
Earlier this month, a security guard of a naval officer was shot dead here when gunmen opened fire on his car.
In February last year, a lieutenant commander of the Pakistan Navy died after being shot at in Kemari area.
In another incident in the same month, a naval officer and his wife were injured in a mysterious explosion in their car near the city's Karsaz area.
