Hidden Markov Models for Hand Gestures

I am completing a final year project for hand gesture recognition using Hidden Markov Models

I have a fair understanding of Hidden Markov Models and how they work using simple examples such as the Unfair Casino and some Weather examples.

I am looking to implement multiple Hidden markov models where each model corresponds to a single gesture, similarly to this paper where the observed states are the angles between the coordinates of different points. This would create a sequence of numbers from 0 to 18 as seen in Figure 3 and Figure 4. .

What would the hidden states be in terms of this scenario?

The weather example has the observations ‘Walk’, ‘Shop’ and ‘Clean’ which would be the numbers 0-18 in the hand gesture case, however I do not know what the states ‘Rainy’ and ‘Sunny’ would correspond to in the hand gesture scenario.