The fast expansion of photon detection technology has fertilized the rapid growth of single-photon sensing and imaging techniques. While promising significant advantages over their classical counterparts, they suffer from ambient and quantum noises whose effects become more pronounced at low light levels and thus must be addressed to achieve high quality results. Here we study how photon-counting noises degrade a single-pixel optical classifier via compressive sensing, and how its performance can be restored by using quantum parametric mode sorting (QPMS). Using MNIST handwritten digits as an example, we examine the effects of detector dark counts and in-band background noises, and demonstrate the effectiveness of mode filtering and upconversion detection in addressing those issues. We achieve 94% classification accuracy in the presence of 500 times stronger in-band noise than the signal received. Our results suggest a robust and efficient approach to single photon sensing in noisy environment.