Spiking neural networks (SNNs) are promising neural network models to achieve power-efficient and event-based computing on neuromorphic hardware. SNNs inherently contain noise and are robust to noisy inputs as well as noise related to the discrete 1-bit spike. In this paper, we find that SNNs are more robust to Gaussian noise in synaptic weights than artificial neural networks (ANNs) under some conditions. This finding will enhance our understanding of the neural dynamics in SNNs and of the advantages of SNNs compared with ANNs. Our results imply the possibility of using high-performance cutting-edge materials with intrinsic noise as an information storage medium in SNNs.