Measurement-Device-Independent Quantum Key Distribution (MDI-QKD) is a two-photon protocol devised to eliminate eavesdropping attacks that interrogate or control the detector in realized quantum key distribution systems. In MDI-QKD, the measurements are carried out by an untrusted third party, and the measurement results are announced openly. Knowledge or control of the measurement results gives the third party no information about the secret key. Error-free implementation of the MDI-QKD protocol requires the crypto-communicating parties, Alice and Bob, to independently prepare and transmit single photons that are physically indistinguishable, with the possible exception of their polarization states. In this paper, we apply the formalism of quantum optics and Monte Carlo simulations to quantify the impact of small errors in wavelength, bandwidth, polarization and timing between Alice's photons and Bob's photons on the MDI-QKD quantum bit error rate (QBER). Using published single-photon source characteristics from two-photon interference experiments as a test case, our simulations predict that the finite tolerances of these sources contribute (4.04+/-20/N_sifted) to the QBER in an MDI-QKD implementation generating an N_sifted-bit sifted key.