Considering he couldn't tell John's girlfriends apart in Belgravia and it turns out he doesn't know the Earth goes around the Sun and that we don't currently have a King on the throne because he "deletes" information he deems irrelevant to his work - it's actually very feasible for his character to not notice