How can I get better at identifying the “right time” to use my spell slots?

I’ve played a bunch of DnD 5e, but my current campaign is my first time playing a Warlock outside of a one-shot; right now I’m almost to level 5. I’m playing more of a face/roleplay-focused role, so my invocations (Aspect of the Moon and Devil’s Sight) are generally more useful out of combat, but I made sure to take several combat-oriented cantrips, as well as shatter, to have at least one good damage-focused spell.

In general, I’m having a lot of fun, but I’m having some trouble getting used to having only two spell slots to play with beyond the very early levels.

A couple of sessions ago, I eagerly fired off both my spell slots at what looked like pretty juicy targets. I did plenty of damage… but when enemy reinforcements showed up, I was out of ammo – and in hindsight, I would have had even better targets to use my spells on if I had kept them in reserve.

In response, the next session, I resolved to try to keep a shorter leash on my spells. I played through a combat encounter trying to keep my spell slots in reserve, but I felt like limiting myself to cantrips, weapon attacks, and "utility actions" like shoving kept me from pulling my weight. I then felt silly for being so cautious when we decided to take a short rest at the end of the encounter anyway. I felt especially silly when the same entire story repeated itself later in the session.

Of course, I can (and do) encourage my party to take short rests when we have the opportunity to do so (as long as people could use the HP and it’s not just because I’m out of spell slots), but in general I feel like I’m doing a poor job of identifying the "right" times to pull from my limited resource pool.

Of course, this problem isn’t limited to Warlock – every spellcasting class has to make these judgment calls – but with such a low number of slots to work with, playing Warlock is definitely testing my resource management. I recognize that adeptly managing my limited spell slots is just part of the game, but is there anything in particular I can do that would help to pick out the best times to bust out a spell?