It is unclear which noise exposure time window and noise characteristics during nighttime are most detrimental for sleep quality in real life settings. We have conducted a field study with 105 volunteers wearing a wrist actimeter to record their sleep during seven days, together with concurrent outdoor noise measurements at their bedroom window. Actimetry recorded sleep latency increased by 5.6 minutes (95% confidence interval: 1.6 to 9.6 minutes) per 10 dB(A) increase in noise exposure during the first hour after bedtime. Actimetry assessed sleep efficiency was significantly reduced by 2-3 percent per 10 dB(A) increase in measured outdoor noise (Leq, 1h) for the last three hours of sleep. For subjectively reported sleepiness, noise exposure during the last hour prior to wake up was most crucial with an increase in the sleepiness score of 0.31 units (95% CI: 0.08 to 0.54) per 10 dB(A) Leq,1h. Associations for estimated indoor noise were not more pronounced than for outdoor noise. Considering noise events in addition to equivalent sound pressure levels (Leq) only marginally improved the statistical models. Our study provides evidence that matching the nighttime noise exposure time window to the individual’s diurnal sleep-wake pattern results in a better estimate of detrimental nighttime noise effects on sleep. We found that noise exposure at the beginning and the end of the sleep is most crucial for sleep quality.