Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Connecting openFrameworks to Google MediaPipe Machine Learning Framework over UDP
| Date | Stars |
|---|---|
| 2026-07-31 | 309 |
| 2026-08-03 | 309 |
| 2026-08-06 | 309 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
## MediaPipe to openFrameworks
#### Notes on how to connect Google's [MediaPipe ML Framework](https://github.com/google/mediapipe) to openFrameworks
MediaPipe is a cross-platform framework for building multimodal applied machine learning pipelines. I want to be able to use it in external applications.
This tutorial walks through how to stream MediaPipe data out over UDP, so any external app and receive and use the data.
I show how to modify the mediapipe example _mediapipe/examples/desktop/hand_tracking_ to add in a new node that recieves hand tracking data as input, broadcasts that data over UDP on port 8080, and then passes the tracking data on to the rest of the graph as output.
> Tested on macOS Mojave (10.14.6) and openFrameworks 0.10.1
---
Beigin by installing MediaPipe on your system using [google's instructions](https://google.github.io/mediapipe/).
Then install and setup Google [Protobufs](https://developers.google.com/protocol-buffers) for openFrameworks using my previous [tutorial](https://github.com/madelinegannon/protobuf_tutorial).
If you've never used [Bazel](https://bazel.build/) before, the build system and organization of Mediapipe can be _really_ confusing. I try to go through step-by-step below, but you can find more information in the [MediaPipe Docs](https://mediapipe.readthedocs.io/en/latest/index.html).
---
#### We're going to modify MediaPipe's desktop hand tracking example to stream out landmarks and bounding rectangles over UDP.
## 1. Update the Graph Definition to Include our new PassThrough Calculator
_Modify mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt_
```
# Add New Node
node {
calculator: "MyPassThroughCalculator"
input_stream: "LANDMARKS:hand_landmarks"
input_stream: "NORM_RECT:hand_rect"
input_stream: "DETECTIONS:palm_detections"
output_stream: "LANDMARKS:hand_landmarks_out"
output_stream: "NORM_RECT:hand_rect_out"
output_stream: "DETECTIONS:palm_detections_out"
}
# Modify input_stream names of next node
# Subgraph that renders annotations and overlays them on top of the input
# images (see renderer_cpu.pbtxt).
node {
calculator: "RendererSubgraph"
input_stream: "IMAGE:input_video"
input_stream: "LANDMARKS:hand_landmarks_out"
input_stream: "NORM_RECT:hand_rect_out"
input_stream: "DETECTIONS:palm_detections_out"
output_stream: "IMAGE:output_video"
}
```
`NOTE` You can visualize the graph to test that inputs and outpus match up at https://viz.mediapipe.dev/
When you add in the custom MyPassThroughCalculator your graph should look like this:
## 2. Making a Custom Calculator
_Adding UDP, Detections, Landmarks, and Hand Rectangles to the PassThrough Calculator_
1. Copy the file _src/mediapipe/my_pass_though_calculator.cc_ to your Mediapipe calculators directory _mediapipe/calculators/core_.
2. The main differences between `my_pass_though_calculator.cc` and the original `pass_though_calculator.cc` are that it adds UDP streaming, and uses Landmark, Rect, and Detection protobufs in the `::mediapipe::Status Process()` function. Next we need to modify the graph file to declare the Tag of the calculators input and stream.
## 3. Modifying Calculators BUILD file
1. Add the following in _mediapipe/calculators/core/BUILD_ to include `my_pass_through_calculator` dependencies:
```
cc_library(
name = "my_pass_through_calculator",
srcs = ["my_pass_through_calculator.cc"],
visibility = [
"//visibility:public",
],
deps = [
"//mediapipe/framework:calculator_framework",
"//mediapipe/framework/port:status",
"//mediapipe/framework/foExcerpt of 10,186 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4074f157d65ea35b, llm:Repository topics and description: 'mediapipe, hand-tracking, machine-learning, openframeworks, udp' and README: tutorial to stream MediaPipe (ML hand-tracking) data over UDP to external apps (connect MediaPipe to openFrameworks).
matched fp:4074f157d65ea35b, llm:Repository topics and description: 'mediapipe, hand-tracking, machine-learning, openframeworks, udp' and README: tutorial to stream MediaPipe (ML hand-tracking) data over UDP to external apps (connect MediaPipe to openFrameworks).
matched fp:4074f157d65ea35b, llm:Repository topics and description: 'mediapipe, hand-tracking, machine-learning, openframeworks, udp' and README: tutorial to stream MediaPipe (ML hand-tracking) data over UDP to external apps (connect MediaPipe to openFrameworks).