Please select Into the mobile phone version | Continue to access the computer ver.
Share FPV footage via WebRTC
345 2 8-6 23:30
Uploading and Loding Picture ...(0/1)
o(^-^)o
z37soft
lvl.4
Flight distance : 791079 ft
Japan
Offline

Hello! I am Nozomu from RedDotDrone.
I know there are a lot of people who are interested, so I would like to share my knowledge with you.

There are Mobile SDK, UX SDK, Payload SDK, Onboard SDK, etc., but I would like to share about the video preview included in DJIWidget. DJIWidget is a framework that accompanies the sample project and provides FPV footage and live streaming capabilities. However, since it is not provided as an SDK, there is no documentation. You will have to refer to the comments in the source code to understand the specifications.

(The following is for iOS.)

I focused on the DJIVideoPreviewer class. This class has an interface that allows you to register your own stream processor and frame processor.


@interface DJIVideoPreviewer (Processor)

/**
*  @param processor Processor registered to receive the H264 stream data.
*/
-(void) registStreamProcessorid<VideoStreamProcessor>)processor;

/**
*  @param processor Remove registered processor list.
*/
-(void) unregistStreamProcessorid)processor;

/*
*  @param processor Processor registered to receive the VideoFrameYUV frame data.
*/
-(void) registFrameProcessorid<VideoFrameProcessor>)processor;

/**
*  @param processor Remove registered processor list.
*/
-(void) unregistFrameProcessorid)processor;

@end


I tried to use this to intercept the footage sent from the drone so that it could be distributed as NDI.
NDI is a protocol developed by NewTek to transmit video over IP.
https://www.newtek.com/ndi/applications/

This makes it possible to send video to multiple devices via a network using an iPhone, even if the controller does not have a video output such as HDMI.

Rarely do we have an actual LAN environment in the field, but if you can install a router, this solution is powerful.
It is important to note that NDI can reduce the load and bandwidth by passing the YUV directly rather than RGB, so you should extract the video data from the DJIVideoPreviewer using YUV as much as possible. Therefore, set enableHardwareDecode and enableFastUpload to TRUE. Also, Mavic2 converts to RGB by going through the lens distortion correction process. The key is to skip this process or take out the video data before that process.

We use this to put video from drones on WebRTC tools like Zoom and Whereby, and to control drones from remote locations. Of course, remote control of the drone is a different function, so you can try your hand at it on your own.


I'm happy to help you with your ideas.
Best,

Nozomu Miura
https://reddotdrone.com


8-6 23:30
Use props
lukelu
lvl.1

United States
Offline

z37soft, Thanks for your sharing, fly drone from anywhere is wonderful ! We are recently looking for a good communication protocol for such tele-operation. How does the drone receive controlling commands from a remote computer ? Hope to hear your expertise.
11-7 13:05
Use props
z37soft
lvl.4
Flight distance : 791079 ft
Japan
Offline

lukelu Posted at 11-7 13:05
z37soft, Thanks for your sharing, fly drone from anywhere is wonderful ! We are recently looking for a good communication protocol for such tele-operation. How does the drone receive controlling commands from a remote computer ?  Hope to hear your expertise.

Hi lukelu,
I send telemetry on MQTT.
Of course, if security is required, I don't send it as is, but instead I use VPN and encryption to communicate with it.
Thanks.
11-7 22:43
Use props
Advanced
You need to log in before you can reply Login | Register now

Credit Rules