Hello! I am Nozomu from RedDotDrone.
I know there are a lot of people who are interested, so I would like to share my knowledge with you.

There are Mobile SDK, UX SDK, Payload SDK, Onboard SDK, etc., but I would like to share about the video preview included in DJIWidget. DJIWidget is a framework that accompanies the sample project and provides FPV footage and live streaming capabilities. However, since it is not provided as an SDK, there is no documentation. You will have to refer to the comments in the source code to understand the specifications.

（The following is for iOS.）

I focused on the DJIVideoPreviewer class. This class has an interface that allows you to register your own stream processor and frame processor.


@interface DJIVideoPreviewer (Processor)

/**
*  @param processor Processor registered to receive the H264 stream data.
*/
-(void) registStreamProcessorid<VideoStreamProcessor>)processor;

/**
*  @param processor Remove registered processor list.
*/
-(void) unregistStreamProcessorid)processor;

/*
*  @param processor Processor registered to receive the VideoFrameYUV frame data.
*/
-(void) registFrameProcessorid<VideoFrameProcessor>)processor;

/**
*  @param processor Remove registered processor list.
*/
-(void) unregistFrameProcessorid)processor;

@end


I tried to use this to intercept the footage sent from the drone so that it could be distributed as NDI.
NDI is a protocol developed by NewTek to transmit video over IP.
https://www.newtek.com/ndi/applications/

This makes it possible to send video to multiple devices via a network using an iPhone, even if the controller does not have a video output such as HDMI.

Rarely do we have an actual LAN environment in the field, but if you can install a router, this solution is powerful.
It is important to note that NDI can reduce the load and bandwidth by passing the YUV directly rather than RGB, so you should extract the video data from the DJIVideoPreviewer using YUV as much as possible. Therefore, set enableHardwareDecode and enableFastUpload to TRUE. Also, Mavic2 converts to RGB by going through the lens distortion correction process. The key is to skip this process or take out the video data before that process.

We use this to put video from drones on WebRTC tools like Zoom and Whereby, and to control drones from remote locations. Of course, remote control of the drone is a different function, so you can try your hand at it on your own.


I'm happy to help you with your ideas.
Best,

Nozomu Miura
https://reddotdrone.com

z37soft, Thanks for your sharing, fly drone from anywhere is wonderful ! We are recently looking for a good communication protocol for such tele-operation. How does the drone receive controlling commands from a remote computer ? Hope to hear your expertise.

lukelu Posted at 11-7 13:05
z37soft, Thanks for your sharing, fly drone from anywhere is wonderful ! We are recently looking for a good communication protocol for such tele-operation. How does the drone receive controlling commands from a remote computer ?  Hope to hear your expertise.

Hi lukelu,
I send telemetry on MQTT.
Of course, if security is required, I don't send it as is, but instead I use VPN and encryption to communicate with it.
Thanks.

thanks for sharing, was searching for it!