Quantum logic gates with fidelity above fault-tolerant threshold are building blocks for scalable quantum technologies[1,2]. Compared to other types of qubits, photon is one of a kind due to its unparalleled advantages in long-distance quantum information exchange[3-5]. As a result, high-fidelity photonic quantum operations are not only indispensable for photonic quantum computation[6-8] but also critical for quantum network[2,9]. However, two-qubit photonic quantum logic gate with fidelity comparable to that of leading physical systems, i.e. 99.7% for superconducting circuits[10] and 99.9% for trapped ions[11], has not been achieved. A major limitation is the imperfection of single photons[12]. Here, we overcome this limitation by using high-quality single photons generated from Rydberg atoms as qubits for the interference-based gate protocol, and achieve a gate fidelity up to 99.84(3)%. Our work paves the way for scalable photonic quantum applications[13-15] based on near-optimal single-photon qubits and photon-photon gates.