By running this code below, I am able to get the demo app to run but, where can I get the source code so that I can get the hand metadata regarding finger positions and joints?

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_cpu \
    --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt

I will like to modify: bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_cpu so I can output a file with hand metadata for sign language analysis. But this seems to be a binary file. How can I use this or integrate this with a new app?

@research506 Please look at https://mediapipe.readthedocs.io/en/latest/multi_hand_tracking_desktop.html for how to build and run multi-hand tracking example on desktop.

On desktop, you can add an output stream poller to demo_run_graph_main.cc or demo_run_graph_main_gpu.cc, such as

ASSIGN_OR_RETURN(mediapipe::OutputStreamPoller multi_hand_landmarks_poller,
                   graph.AddOutputStreamPoller("multi_hand_landmarks"));

to listen for output packets on a particular output stream, such as "multi_hand_landamrks" above. See https://mediapipe.readthedocs.io/en/latest/hello_world_desktop.html for example.

Then, on every frame, check if a packet is available in the output stream:

mediapipe::Packet multi_hand_landmarks_packet;
if (!multi_hand_landmarks_poller.Next(&multi_hand_landmarks_packet)) break;
const auto& multi_hand_landmarks = multi_hand_landmarks_packet.Get<std::vector<std::vector<mediapipe::NormalizedLandmark>>>();

Note that you need a dependency on

#include "mediapipe/framework/formats/landmark.pb.h"

in the file you are modifying (and modify the corresponding rule in BUILD file as well).

And then you should be able to do what you want with these landmarks, such as print them:

int hand_index = 0;
for (const auto& hand_landmarks : multi_hand_landmarks) {
  int landmark_index = 0;
  for (const auto& landmark : hand_landmark) {
    std::cout << "[Hand<" << hand_index << ">] Landmark<" << landmark_index++ << ">: (" << landmark.x() << ", " << landmark.y() << ", " << landmark.z() << ")\n";
  }
 std::cout << "\n";
 ++hand_index;
}

My brother, this was the droid I was looking for. I will update this thread with a link to my repo where the code can be found. I owe you a beer