Limitations of object detection.
3008 8 2019-9-26
Uploading and Loding Picture ...(0/1)
o(^-^)o
BGA
Second Officer
Offline

I played a bit with the object detection stuff in the Robomaster S1 and I found some considerable issues that it would be great if DJI would be ablee to address. They all boil down to performance issues and I ma pretty sure this should have not been an issue considering the CPU the robot is using (you can confirm that by simply looking at its build in person tracking).
1 - A simple while loop that does object detection (by calling [color=rgba(0, 0, 0, 0.65)]vision_ctrl.get_car_detection_info()) is running at 10 FPS at most. This is simply way too slow and I suspect it is not the cost of doing the image detection itself but the fact that the python interpreter DJI is using is way too slow. You can do better DJI!
[color=rgba(0, 0, 0, 0.65)]2 - Image detection itself seems to be running in an even lower FPS (around 4 or 5). The end result of this is that even with a robot moving, 2 consecutive calls to get_car_detection_info() might return virtually the same information (+- some error). This is what ends up making the robot overshoot when trying to target something at faster speeds.
[color=rgba(0, 0, 0, 0.65)]

[color=rgba(0, 0, 0, 0.65)]For 1 there is nothing I can do unfortunatelly. Just hope that DJI will improve the performance of its interpreter.
[color=rgba(0, 0, 0, 0.65)]
[color=rgba(0, 0, 0, 0.65)]For 2, I used some code to re-read the detection info in case it seems that it did not change between 2 interactions. That helps, but this only makes the low rate of object detection be even more obvious.
[color=rgba(0, 0, 0, 0.65)]

[color=rgba(0, 0, 0, 0.65)]Anyway, if you are writing S1 programs, be warned.
[color=rgba(0, 0, 0, 0.65)]


2019-9-26
Use props
GG38
lvl.1

France
Offline

Maybe the actual limitation comes from the smart vision camera. High fps means more computer power : cost/battery life

I have a jevois at home. Wondering if i could use it instead

http://jevois.org/
2019-11-3
Use props
BGA
Second Officer
United States
Offline

GG38 Posted at 11-3 23:40
Maybe the actual limitation comes from the smart vision camera. High fps means more computer power : cost/battery life

I have a jevois at home. Wondering if i could use it instead

Nope, does not make sense. Person tracking works very well and it is arguably more computationally intensive to detect a person than to detect a well known object (in the end, they should be the same anyway due to the employment of machine learning).

2019-11-4
Use props
GG38
lvl.1

France
Offline

Can we teach the camera to recognize other things? Ie a specific object / animal?
2019-11-4
Use props
BGA
Second Officer
United States
Offline

GG38 Posted at 11-4 10:12
Can we teach the camera to recognize other things? Ie a specific object / animal?

There is no APi exposed for that, so no. Technically, it should be possible as long as the S1 has enough memory to store the new detection model (it does appear to have a considerable amount of RAM, although I am not sure about on-board storage).
2019-11-4
Use props
GG38
lvl.1

France
Offline

Lack of documentation?
2019-11-4
Use props
BGA
Second Officer
United States
Offline

GG38 Posted at 11-4 14:15
Lack of documentation?

It might be, but as this would be considerably more involved than anything else available through the API today, I do not think this is the case. I think they just did not come up with a public API for this at all.

As a side note, they intentionally removed introspection functions ("dir()"  for example) from the on-board python interpreter so there is no way to try to figure out stuff by using those. In other worlds, not having documentation about something or something not being actually available amounts to virtually the same thing unless you are very good a luck guesses.

2019-11-4
Use props
GG38
lvl.1

France
Offline

It might worth opening a case in the support page...
Screenshot_2019-11-05-08-43-11-296_dji.go.v4.png
Screenshot_2019-11-05-08-42-59-012_dji.go.v4.png
2019-11-4
Use props
BGA
Second Officer
United States
Offline

GG38 Posted at 11-4 23:44
It might worth opening a case in the support page...

I have been filing bug reports and feature requests. So fat I am not very confident they will result into anything being done that is not already on their plans anyway,
2019-11-5
Use props
Advanced
You need to log in before you can reply Login | Register now

Credit Rules