Does a wizard need to hold a component pouch or focus in one hand in order for it to work?

One of my PCs is a (weird) wizard who is frequently switching quickly between items—one turn he’s wielding a quarterstaff two-handed, the next he’s got a component pouch and a dagger, and after that, he’s going back to the two-handed quarterstaff to prepare for an attack of opportunity.

We often find ourselves in situations where we’re not sure if the rules on action economy actually allow for him to cast some of these spells. For clarification, this question is two-fold:

  • If a focus is merely worn, can a wizard still cast his spells?
  • If it fits, can the wizard hold a component or focus in the same hand that performs somatic components?