Cross your hands in front of your chest. For a human observer, they remain your left and right hands. For a collection of independent vision models, preserving that simple fact can require quite a lot of coordination.
MediaPipe Holistic combines pose, face and hand estimation into one pipeline. Google's December announcement explains a sensible division of work: estimate the body, use it to locate relevant regions, then examine the face and hands at a suitable resolution. Body context helps the smaller trackers maintain consistent identities.
Resolution is the first reason to avoid treating the whole image uniformly. A low-resolution frame may be enough to locate a shoulder or hip. Fingers occupy a much smaller area. If the hand model receives only a crop from that already reduced image, the information it needs may have disappeared. Cropping from the original frame preserves detail without forcing every stage to process every pixel.
This resembles a useful habit in graphics: spend effort where it changes the visible result. A face needs fine articulation, while a torso may need only a few stable reference points. Different models can specialize without abandoning a common understanding of the person.
Keep the estimates attached to a person
The harder question is how the estimates join up. Suppose a hand tracker loses a hand during a fast movement. A body estimate can provide a likely region in which to look again. Without that context, the tracker may search more broadly or attach itself to the wrong object. A pipeline has information that an isolated model does not.
For an application, I would want to preserve that relationship in the data model. A hand should belong to a person and a side of the body, rather than merely being the first hand in an array. A missing estimate should be represented explicitly. Otherwise a renderer may animate yesterday's valid data as though it described the current frame.
Gestures add another layer of interpretation. A pose estimate is not a command. If a person raises an arm to adjust their glasses, should that activate a menu? The interface needs context, activation rules and feedback. Tracking more body parts gives the developer more evidence, but also more opportunities to interpret ordinary movement incorrectly.
I would test a gesture interface with transitional movements rather than only its intended gestures. Hands entering the image, crossing each other, reaching behind the body and returning to rest are all informative. The pauses between commands should be as comfortable as the commands themselves.
The useful advance here is coordination. A single pipeline can make several specialized predictions agree well enough to support an interaction. That is a stronger foundation than a collage of individually impressive demos. It brings the software closer to observing a person whose face, hands and body are all participating in the same action.