Anything that runs WITHIN D2 is detectable. As in hacks that run in game, and no-CD loaders. But, anything that modifies D2's files while it isnt running (CD key stuff, rain remover, HC hack, etc) are not detectable.
As for detecting something malicious that is not detected by AV, you need to use a disassembler and a good knowledge of ASM to look through the program's code. It also helps to know about the game it is made for and what language it was programmed in (if there is extra code that doesnt make sense). Debugging also helps if you dont understand parts of the code. I doubt anyone that hasnt done this before will be able to, just because it is hard to understand professional code without experience on unprofessional code.
As for detecting something malicious that is not detected by AV, you need to use a disassembler and a good knowledge of ASM to look through the program's code. It also helps to know about the game it is made for and what language it was programmed in (if there is extra code that doesnt make sense). Debugging also helps if you dont understand parts of the code. I doubt anyone that hasnt done this before will be able to, just because it is hard to understand professional code without experience on unprofessional code.