Currently, face recognition technology (FRT) has been applied ubiquitously. However, due to the abuse of personal face photos on social media, FRT has encountered unprecedented challenges which promote the development of face spoofing detection (also called face liveness detection or face anti-spoofing) technology. Traditional face spoofing detection methods usually extract features manually and distinguish real and fake faces through a single cue, which may make these methods have problems with low accuracy and generality. In addition, the effectiveness of existing methods is affected by illumination variations. To address the above issues, we propose a multi-scale color inversion dual-stream convolutional neural network, termed MSCI-DSCNN. One stream of the proposed model converts the input RGB images into grayscale ones and conducts multi-scale color inversion to obtain the MSCI images, which are then put into the improved MobileNet to extract face reflection features. The other stream of the network directly feeds RGB images into the improved MobileNet to extract face color features. Finally, the features extracted separately from the two branches are fused and then used for face spoofing detection. We evaluate the proposed framework on three publicly available databases, CASIA-FASD, REPLAY-ATTACK, and OULU-NPU, and achieve promising results. To further measure the generalization capability of the proposed approach, extensive cross-database experiments are performed and the results exhibit great effectiveness of our MSCI-DSCNN method.