Replay spoofing attacks are a major threat for speaker verification systems. Although many anti-spoofing systems or countermeasures are proposed to detect dataset-specific replay attacks with promising performance, they generalize poorly when applied on unseen datasets. In this work, the cross-dataset scenario is treated as a domain-mismatch problem and dealt with using a domain adversarial training framework. Compared with previous approaches, features learned from this newly-designed architecture are more discriminative for spoofing detection, but more indistinguishable across different domains. Only labeled source-domain data and unlabeled target-domain data are required during the adversarial training process, which can be regarded as unsupervised domain adaptation. Experiments on the ASVspoof 2017 V. 2 dataset as well as the physical access condition part of BTAS 2016 dataset demonstrate that a significant EER reduction of over relative 30% can be obtained after applying the proposed domain adversarial training framework. It is shown that our proposed model can benefit from a large amount of unlabeled target-domain training data to improve detection accuracy.