The BJP leader's remarks come a day after the Congress attacked PM Modi on the sufferings of Dalits in Gujarat