The historian wrote a letter to CoA head Vinod Rai after quitting the panel recently