The doctor is such a bad liar - everything about his body language is giving him away, even without the weird reactions to someone who isn't there - and yet nobody notices. But perhaps he's just like that all the time and they think it's his "normal". After all they've never seen him before the big betrayal. For the audience though, because we get the flashbacks, it's so very obvious.