How to use Inspiring Leader so it’s not a permanent HP increase for the whole party?

I wanted to ask the question "Is Inspiring Leader basicly a permanent HP increase for the whole party?", but apparently that question has already been asked and answered, even though it doesn’t really solve my issue.

I am a DM for a party of 5 players where one of them (the Rogue) has the Inspiring leader feat. We’re all horrible at roleplay, so I don’t require him to actually make up a speech each long rest in order to get the benefits. Because I think it’s also a bit silly that he has to say "I use inspiring leader" each long rest, I simply ruled that he automatically uses it every morning.

And this is where something very rare happened! The players wanted me to make a harsher ruling! One of them said that they should only be allowed to get the benefits after an actual speach (or at least have the +n be dependent on whether a speech was given or not, and how good it was). Some others said that it was silly that they’d get a permanent HP increase and that there should be some mechanic tied to it.

I kinda agree with them, but I don’t know how to make a better ruling. I’m also afraid to nerf it too hard, because since it’s a feat he already sacrificed two +1 ASI’s to get it.

How should I rule Inspiring Leader so that it is not a permanent HP boost, but also not nerfed too hard?

Since this there probably isn’t a RAW solution for this, I’m mainly looking for experiences from both DMs and players who have had DM’s who made a special ruling concerning this feat.