The tourists, seeking a series-levelling win, went to tea on the second day at 126 for three despite good batting conditions at the Sinhalese Sports Club.
Opener Ahmed Shehzad anchored the innings with 58, but his dismissal just before tea turned the tide in Sri Lanka's favour.
Shehzad put on 47 for the first wicket with Khurram Manzoor and 63 for the second with Azhar Ali to lift the total to 110-1 before two wickets fell in the space of 12 runs.
Shehzad fell soon after, edging a pull off Dilruwan Perera to be caught down the leg-side by wicket-keeper Niroshan Dickwella.
At tea, Younis Khan was unbeaten on 10 and skipper Misbah-ul Haq had yet to score.
Shehzad was on 37 when he was reprieved by the Decision Review System (DRS) after being given out leg-before off Herath by English umpire Richard Illingworth.
Herath had separated the opening pair when he had Manzoor caught behind by the wicket-keeper for 23 two deliveries before lunch.
Number 11 batsman Chanaka Welegedara returned unbeaten on 27 after adding 36 for the last wicket with Herath (17).
Left-arm seamer Junaid Khan finished with five for 87, with all his five career five-wicket hauls coming against Sri Lanka.
Fellow seamer Wahab Riaz claimed three for 88.
Sri Lanka lead 1-0 in the two-match series after winning the first Test in Galle by seven wickets.
