joker being a challenge in hand-to-hand depends entirely on the writer and the series, it makes sense if the writer wants it to make sense.

it doesn't make any sense that nobody's killed the joker yet or that nobody else uses joker gas or that nobody's found a sample of batman's blood at a crime scene and used it to find out who he is or that tim drake was somehow the only person capable of deducing his identity or that nobody's found the batcave although all they'd have to do is follow his batmobile or that bruce wayne hasn't somehow poured enough money into arkham to make it impenetrable or that the villains somehow still keep getting their gimmick weapons every time they get out of prison.