Due to the rapidly growing volume of publications, it is an urgent task to evaluate the potential impact of emerging works and identify valuable publications for the benefit of researchers with limited time. The impact of authors is significant in terms of personal promotion and fund raising. However, the criterion used to evaluate the impact of authors is largely based on credits they have received, not potential credits. We take the citation count, the most direct and widely used metric for publication popularity, as the potential impact and evaluate its predictability. To this end, we propose a neural network-based point process model for predicting the citation count of individual publications. Our approach integrates signals from paper-specific features and their citation traces, which reflect the trend of either losing or gaining popularity. The model is learned in an adversarial way, which mitigates the bias-exposure efficiently. We verify our model on the largest publicly available academic publication repository, and our model outperforms alternatives with a notable margin.