A critical vulnerability of quantum key distribution systems using typical single-photon detectors is a family of optical manipulation attacks through blinding and "fake states". These allow a man-in-the-middle attack that are not revealed by standard protocols. Here, we present a self-testing method of photodetectors to efficiently reveal manipulation by anything but single photon-level signals. This self-testing neither relies on specific assumptions on the detection or manipulation mechanism, nor requires the technologically more complex protocol of a measurement-device independent quantum key distribution. We illustrate three different countermeasure examples where the legitimate receiver of the quantum key distribution system self-exposes its detectors to light from a local emitter. A detector manipulation attack, being optical or of other nature, is revealed by detecting an excess or a lack of detector events as a consequence of this self-illumination. In our experiments, the self-testing concept could successfully identify a detector manipulation attack within a very short time. Compared with existing countermeasures against detector manipulation attacks based on technical monitoring of detector states, or protocols that require a different topology, the method presented in this work has a low impact on the normal operation of detecting single-photon-level signals, and is significantly simpler to implement. This countermeasure can even be retrofit to existing quantum key distibution systems, and harden them against detector manipulation attacks.