Can I just hold my Spellcasting Focus, instead of a material (with a price) that I own, to cast a spell?

Let’s say that I want to cast Soul Cage. I know that I need the tiny silver cage worth 100gp. Can I just have it attached to my belt and cast the spell using my Spellcasting focus, or do I need to hold specifically the tiny silver cage?

I’m a Hexblade Warlock with a glaive and the Improved Pact Weapon feature, wich makes my summoned glaive my Spellcasting Focus. I wonder if there’s a way to kill an enemy with my glaive then use my reaction to cast Soul Cage.