How to handle time in D&D 5e

I want to run a campaign where time is a central factor, in the sense that NPCs will be certain places at certain times, and learning these patterns is important, but I’m not sure how to handle it. In combat, I can use the “one turn is 6 seconds” rule, but how do I keep track of time elapsed while my players traipse around the town? While they talk to NPCs?