Skip to content

Commit 81a2b06

Browse files
mrtjmrtj
committed
Initial commit
0 parents  commit 81a2b06

File tree

4 files changed

+224
-0
lines changed

4 files changed

+224
-0
lines changed

‎.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.ipynb_checkpoints
2+
*.so

‎README.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# pyds_tracker_meta
2+
3+
[pybind11](https://github.com/pybind/pybind11) wrapper to access Nvidia DeepStream tracker meta info (`NvDsPastFrame...` classes) from Python.
4+
5+
This library provides access to the `NvDsPastFrameObjBatch` type user metadata of DeepStream data streams. This metadata is generated by the object trackers of [nvtracker](https://docs.nvidia.com/metropolis/deepstream/dev-guide/#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.02.html#) plugin. The C API of these structures can be found in [nvds_tracker_meta.h](https://docs.nvidia.com/metropolis/deepstream/dev-guide/DeepStream_Development_Guide/baggage/nvds__tracker__meta_8h.html) header file of DeepStream SDK. The instances of these metadata structures contain information about the object tracking results in past frames.
6+
7+
Some trackers do not reported the state of the tracked object if there are no current matching detection results. If in a later frame a detection result confirms the state of the tracked object, the past history of the tracking is reported retroactively in `NvDsPastFrame...` metadata. This library provides Python access to this metadata. For more information refer to the [nvtracker](https://docs.nvidia.com/metropolis/deepstream/dev-guide/#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.02.html#) plugin documentation.
8+
9+
## Installation
10+
11+
### Prerequisites
12+
13+
1. Install [pybind11](https://github.com/pybind/pybind11). The recommended way is to [build it from source](https://pybind11.readthedocs.io/en/stable/basics.html?highlight=install#compiling-the-test-cases). Alternatively you might try simply `pip3 install pybind11`.
14+
2. You should have `gstreamer-1.0` and `gstreamer-video-1.0` packages installed in your system. If you are using DeepStream, you probably have these packages already.
15+
3. You will need also the standard `c++` compiler that you usually find in Linux distribution. `c++11` standard is used.
16+
17+
### Compile the source
18+
19+
1. The source should be compiled on your target platform (Jetson or x86).
20+
2. Set your DeepStream version and path in `build.sh`.
21+
3. Launch `build.sh`
22+
4. Copy the compiled library (eg. `pyds_tracker_meta.cpython-36m-aarch64-linux-gnu.so`) to your python `site-packages` folder, or alternatively to the folder where your python script will run.
23+
24+
## Usage
25+
26+
`pyds_tracker_meta` is meant to be used together with the standard [Python bindings for DeepStream](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps). Make sure you have `pyds` available.
27+
28+
Ensure you have set `enable-past-frame` property of the `gst-nvtracker` plugin to `1`. (See [nvtracker](https://docs.nvidia.com/metropolis/deepstream/dev-guide/#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.02.html#) plugin documentation.)
29+
30+
Most likely you will use this library from the buffer probe callbacks of a gstreamer plugin pad, when the object tracking results are available. The [deepstream-test2](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/apps/deepstream-test2) python app shows you how to set up such a callback.
31+
32+
The example snippet provided bellow shows how to cast a user meta to a past frame object batch, and how to access all fields of the metadata. Add the following lines to the `osd_sink_pad_buffer_probe` method found int `deepstream-test2.py`, just after the [`batch_meta` was acquired](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/2931f6b295b58aed15cb29074d13763c0f8d47be/apps/deepstream-test2/deepstream_test_2.py#L61):
33+
34+
```python
35+
def osd_sink_pad_buffer_probe(pad,info,u_data):
36+
37+
# ... code to acquire batch_meta ...
38+
39+
user_meta_list = batch_meta.batch_user_meta_list
40+
while user_meta_list is not None:
41+
user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
42+
43+
print('user_meta:', user_meta)
44+
print('user_meta.user_meta_data:', user_meta.user_meta_data)
45+
print('user_meta.base_meta:', user_meta.base_meta)
46+
if not pyds_tracker_meta.NvDsPastFrameObjBatch.user_meta_is_past_frame_obj_batch(user_meta):
47+
continue
48+
past_frame_object_batch = pyds_tracker_meta.NvDsPastFrameObjBatch.from_user_meta(user_meta)
49+
print('past_frame_object_batch:', past_frame_object_batch)
50+
print(' list:')
51+
for past_frame_object_stream in past_frame_object_batch.list:
52+
print(' past_frame_object_stream:', past_frame_object_stream)
53+
print(' streamID:', past_frame_object_stream.streamID)
54+
print(' surfaceStreamID:', past_frame_object_stream.surfaceStreamID)
55+
print(' list:')
56+
for past_frame_object_list in past_frame_object_stream.list:
57+
print(' past_frame_object_list:', past_frame_object_list)
58+
print(' numObj:', past_frame_object_list.numObj)
59+
print(' uniqueId:', past_frame_object_list.uniqueId)
60+
print(' classId:', past_frame_object_list.classId)
61+
print(' objLabel:', past_frame_object_list.objLabel)
62+
print(' list:')
63+
for past_frame_object in past_frame_object_list:
64+
print(' past_frame_object:', past_frame_object)
65+
print(' frameNum:', past_frame_object.frameNum)
66+
print(' tBbox.left:', past_frame_object.tBbox.left)
67+
print(' tBbox.width:', past_frame_object.tBbox.width)
68+
print(' tBbox.top:', past_frame_object.tBbox.top)
69+
print(' tBbox.right:', past_frame_object.tBbox.height)
70+
print(' confidence:', past_frame_object.confidence)
71+
print(' age:', past_frame_object.age)
72+
73+
try:
74+
user_meta_list = user_meta_list.next
75+
except StopIteration:
76+
break
77+
```

‎build.sh

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
NAME=pyds_tracker_meta
2+
PKGS="gstreamer-1.0 gstreamer-video-1.0"
3+
NVDS_VERSION="5.0"
4+
NVDS_PATH="/opt/nvidia/deepstream/deepstream-${NVDS_VERSION}/sources/includes/"
5+
6+
c++ -O3 -Wall -shared -std=c++11 \
7+
-fPIC `python3 -m pybind11 --includes` \
8+
pyds_tracker_meta.cpp \
9+
-o ${NAME}`python3-config --extension-suffix` \
10+
-I${NVDS_PATH} \
11+
`pkg-config --cflags ${PKGS}`

‎pyds_tracker_meta.cpp

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
#include <iostream>
2+
#include <pybind11/pybind11.h>
3+
#include <pybind11/stl.h>
4+
#include "nvds_tracker_meta.h"
5+
6+
namespace py = pybind11;
7+
8+
PYBIND11_MODULE(pyds_tracker_meta, m) {
9+
m.doc() = "pybind11 wrapper to access Nvidia DeepStream NvDsPastFrame* classes";
10+
11+
py::class_<NvDsPastFrameObjBatch>(m, "NvDsPastFrameObjBatch", "Batch of lists of buffered objects")
12+
13+
// Casting and type check
14+
.def_static(
15+
"cast",
16+
[](void *obj) { return (NvDsPastFrameObjBatch*)(obj); },
17+
"Cast given object/data to pyds_pastframemeta.NvDsPastFrameObjBatch",
18+
py::arg("obj"),
19+
py::return_value_policy::reference)
20+
.def_static(
21+
"from_user_meta",
22+
[](NvDsUserMeta *user_meta) {
23+
NvDsPastFrameObjBatch* obj = nullptr;
24+
if (user_meta && user_meta->base_meta.meta_type == NVDS_TRACKER_PAST_FRAME_META) {
25+
obj = (NvDsPastFrameObjBatch *) (user_meta->user_meta_data);
26+
}
27+
return obj;
28+
},
29+
py::arg("user_meta"),
30+
"If the data contained in the user meta is a NvDsPastFrameObjBatch instance, "
31+
"casts the data and returns it. Otherwise returns NULL.",
32+
py::return_value_policy::reference)
33+
.def_static(
34+
"user_meta_is_pfob",
35+
[](NvDsUserMeta *user_meta) {
36+
return (user_meta->base_meta.meta_type == NVDS_TRACKER_PAST_FRAME_META);
37+
},
38+
py::arg("user_meta"),
39+
"Returns true if the user_meta contains a NvDsPastFrameObjBatch instance.")
40+
41+
// Struct members
42+
.def_readonly("numAllocated", &NvDsPastFrameObjBatch::numAllocated, "Number of blocks allocated for the list.")
43+
.def_readonly("numFilled", &NvDsPastFrameObjBatch::numFilled, "Number of filled blocks in the list.")
44+
45+
// list is implemented as a real python iterator
46+
.def_property_readonly(
47+
"list",
48+
[](NvDsPastFrameObjBatch &self) {
49+
return py::make_iterator(self.list, self.list + self.numFilled);
50+
},
51+
py::keep_alive<0, 1>(),
52+
"Iterator of stream lists.")
53+
54+
// convenience methods for subscription access, lenght, use the batch directly as an iterator
55+
.def(
56+
"__getitem__",
57+
[](const NvDsPastFrameObjBatch &self, size_t i) {
58+
if (i >= self.numFilled) throw py::index_error();
59+
return self.list[i];
60+
})
61+
.def("__len__", [](const NvDsPastFrameObjBatch &self) { return self.numFilled; })
62+
.def("__iter__",
63+
[](const NvDsPastFrameObjBatch &self) {
64+
return py::make_iterator(self.list, self.list + self.numFilled);
65+
},
66+
py::keep_alive<0, 1>());
67+
68+
69+
py::class_<NvDsPastFrameObjStream>(m, "NvDsPastFrameObjStream", "List of objects in each stream.")
70+
.def_readonly("streamID", &NvDsPastFrameObjStream::streamID, "Stream id the same as frame_meta->pad_index.")
71+
.def_readonly("surfaceStreamID", &NvDsPastFrameObjStream::surfaceStreamID, "Stream id used inside tracker plugin.")
72+
.def_readonly("numAllocated", &NvDsPastFrameObjStream::numAllocated, "Maximum number of objects allocated.")
73+
.def_readonly("numFilled", &NvDsPastFrameObjStream::numFilled, "Number of objects in this frame.")
74+
75+
// list is implemented as a real python iterator
76+
.def_property_readonly(
77+
"list",
78+
[](NvDsPastFrameObjStream &self) { return py::make_iterator(self.list, self.list + self.numFilled); },
79+
py::keep_alive<0, 1>(),
80+
"Iterator of objects inside this stream.")
81+
82+
// convenience methods for subscription access, lenght, use the frame object stream directly as an iterator
83+
.def("__getitem__",
84+
[](const NvDsPastFrameObjStream &self, size_t i) {
85+
if (i >= self.numFilled) throw py::index_error();
86+
return self.list[i];
87+
})
88+
.def("__len__", [](const NvDsPastFrameObjStream &self) { return self.numFilled; })
89+
.def("__iter__",
90+
[](const NvDsPastFrameObjStream &self) {
91+
return py::make_iterator(self.list, self.list + self.numFilled);
92+
},
93+
py::keep_alive<0, 1>());
94+
95+
96+
py::class_<NvDsPastFrameObjList>(m, "NvDsPastFrameObjList", "One object in several past frames")
97+
.def_readonly("uniqueId", &NvDsPastFrameObjList::uniqueId, "Object tracking id.")
98+
.def_readonly("classId", &NvDsPastFrameObjList::classId, "Object class id.")
99+
.def_property_readonly(
100+
"objLabel",
101+
[](NvDsPastFrameObjList *self) {
102+
return std::string(self->objLabel, MAX_LABEL_SIZE);
103+
},
104+
"An array of the string describing the object class.")
105+
.def_readonly("numObj", &NvDsPastFrameObjList::numObj, "Number of frames this object appreared in the past.")
106+
107+
// list is implemented as a real python iterator
108+
.def_property_readonly(
109+
"list",
110+
[](NvDsPastFrameObjList &self) { return py::make_iterator(self.list, self.list + self.numObj); },
111+
py::keep_alive<0, 1>(),
112+
"Iterator of past frame info of this object.")
113+
114+
// convenience methods for subscription access, lenght, use the frame object list directly as an iterator
115+
.def("__getitem__",
116+
[](const NvDsPastFrameObjList &self, size_t i) {
117+
if (i >= self.numObj) throw py::index_error();
118+
return self.list[i];
119+
})
120+
.def("__len__", [](const NvDsPastFrameObjList &self) { return self.numObj; })
121+
.def("__iter__",
122+
[](const NvDsPastFrameObjList &self) {
123+
return py::make_iterator(self.list, self.list + self.numObj);
124+
},
125+
py::keep_alive<0, 1>());
126+
127+
128+
py::class_<NvDsPastFrameObj>(m, "NvDsPastFrameObj")
129+
.def_readonly("frameNum", &NvDsPastFrameObj::frameNum)
130+
.def_readonly("tBbox", &NvDsPastFrameObj::tBbox)
131+
.def_readonly("confidence", &NvDsPastFrameObj::confidence)
132+
.def_readonly("age", &NvDsPastFrameObj::age);
133+
134+
}

0 commit comments

Comments
 (0)