Tactical Guide-Frame drop in live streaming? Caused by your phone

Spacemarine

Some people use the RTMP streaming function of the MSDK to push real-time image transmission video to a custom RTMP server, during flight but have found that live streaming can sometimes drop frame.

How to troubleshoot this problem? There are three possible causes

1. The mobile phone itself

2. The RTMP server is experiencing a problem

3. The APP code developed by MSDK has a bug

We can easily locate the problem by excluding the variables, such as pushing the stream to a third party platform to see if it can solve the problem and whether using the official GO 4 or Sample Code can solve the problem.

However sometimes the system settings of the mobile device may also cause dropped frame. Some Android systems have been optimized to save power and the setting can effect the overall operation and limit the background operations of certain APPs. For Example, MIUI's phone runs in "Balanced Mode" by default. You can switch to "Perforamce Mode" to maximize the hardware perforamace but the battery will drain very fast. The system also provide "Power Saving Mode" which restricts the APP running in background. This will lead to a conosequce that the APP might be slow sometimes crash. In my experiment, I found the MIUI's phone is running streaming function smoothly while the operating system sets to "Performance Mode" and dropped frame has been found while the operating system is running under "Power Saving Mode".

z37soft

And in general, when a smartphone gets hot, the CPU clock is dropped.

TboltFly

Hello,
I am having trouble with my main app crashing after a few minutes of streaming to my cloud RTMP server.  The app crash is attached at bottom of this post.

A little background - the code is for a dji mini 3 running on a samsung  galaxy S22 using the sample app from sdk 5.1.7 derived from https://github.com/dji-sdk/Mobile-SDK-Android-V5/blob/dev-sdk-main/SampleCode-V5/android-sdk-v5-uxsdk/src/main/java/dji/v5/ux/sample/showcase/defaultlayout/DefaultLayoutActivity.java

Here is the code on the main app that runs on a fragment (the home fragment) and is performed "onCreateView".
IVideoChannel primaryChannel =
            MediaDataCenter.getInstance().getVideoStreamManager().getAvailableVideoChannel(VideoChannelType.PRIMARY_STREAM_CHANNEL);
and later
StreamSource primaryStreamSource = primaryChannel.getStreamSource();
if (VideoChannelState.ON == to && primaryStreamSource != null) {
         getActivity().runOnUiThread(() -> primaryFpvWidget.updateVideoSource(primaryStreamSource, VideoChannelType.PRIMARY_STREAM_CHANNEL));
         primaryChannel.addVideoChannelStateChangeListener(primaryChannelStateListener);
}

in a separate worker thread after on product connect,  I start the stream like this:

ILiveStreamManager liveStreamManager = MediaDataCenter.getInstance().getLiveStreamManager();

         LiveStreamSettings.Builder lssBuilder = new LiveStreamSettings.Builder();

         RtmpSettings.Builder b = new RtmpSettings.Builder();
         b.setUrl(liveURL);
         RtmpSettings rtmp = b.build();
         lssBuilder.setRtmpSettings(rtmp);
         lssBuilder.setLiveStreamType(LiveStreamType.RTMP);
         LiveStreamSettings lss = lssBuilder.build();

         liveStreamManager.setLiveStreamSettings(lss);

         // the video going to app display is primary - MUST BE PRIMARY!!! or dont see video
         // but is the trouble is 2 guys can't own the same socket - how to fix?
         // put it all on the same thread
         liveStreamManager.setVideoChannelType(VideoChannelType.PRIMARY_STREAM_CHANNEL);
         liveStreamManager.setLiveStreamQuality(StreamQuality.SD); // make quality lower? LKA
         //liveStreamManager.setLiveVideoBitrate(2048); // small
         liveStreamManager.setLiveVideoBitrateMode(LiveVideoBitrateMode.AUTO);

            liveStreamManager.startStream(new CommonCallbacks.CompletionCallback() {...}

What am I doing wrong? The goal is to have the same primary channel video on the app (phone) itself AND streamed up to my server.
Thanks in advance for your help.

2023-11-03 14:42:44.584 14829-14829/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:44.584 14829-14829/? A/DEBUG: pid: 12914, tid: 12914, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:44.589 12914-12988/com.tbolt.main E/fdsan: failed to exchange ownership of file descriptor: fd 297 is owned by unique_fd 0x6e6ca9d75c, was expected to be unowned

2023-11-03 14:42:44.620 12914-14680/com.tbolt.main E/co_b: (co_a:86) checkFrame() lastId = 18150 id=18156

2023-11-03 14:42:44.776 12914-12988/com.tbolt.main E/fdsan: attempted to close file descriptor 297, expected to be owned by unique_fd 0x6e6ca9dd9c, actually owned by unique_fd 0x6e6ca9d75c

2023-11-03 14:42:44.963 14834-14834/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:44.963 14834-14834/? A/DEBUG: pid: 12914, tid: 12914, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:44.974 12914-14824/com.tbolt.main E/fdsan: failed to exchange ownership of file descriptor: fd 297 is owned by unique_fd 0x6e6ca9d75c, was expected to be unowned

2023-11-03 14:42:45.081 12914-14687/com.tbolt.main W/DJIV5/co_c@1a50a50: (co_b:99) onOutputFrame() frame index:18161

2023-11-03 14:42:45.161 12914-14688/com.tbolt.main E/co_b: (co_a:104) checkFrame() len=6840 checkLen=11802 seiOffset：6854

2023-11-03 14:42:45.167 12914-12985/com.tbolt.main A/libc: Fatal signal 11 (SIGSEGV), code -6 (SI_TKILL) in tid 12985 (com.tbolt.main), pid 12914 (com.tbolt.main)

2023-11-03 14:42:45.201 12914-14685/com.tbolt.main E/co_b: (co_a:86) checkFrame() lastId = 18163 id=18166

2023-11-03 14:42:45.204 12914-12988/com.tbolt.main E/fdsan: attempted to close file descriptor 297, expected to be owned by unique_fd 0x6e6ca9abdc, actually owned by unique_fd 0x6e6ca9d75c

2023-11-03 14:42:45.332 14840-14840/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:45.332 14840-14840/? A/DEBUG: pid: 12914, tid: 12914, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:45.682 14846-14846/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:45.682 14846-14846/? A/DEBUG: pid: 12914, tid: 12914, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:45.710 12914-14680/com.tbolt.main E/co_b: (co_a:86) checkFrame() lastId = 18175 id=18178

2023-11-03 14:42:46.049 14852-14852/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:46.049 14852-14852/? A/DEBUG: pid: 12914, tid: 12914, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:46.368 12914-14687/com.tbolt.main W/DJIV5/co_c@1a50a50: (co_b:99) onOutputFrame() frame index:18194

2023-11-03 14:42:46.429 14859-14859/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:46.429 14859-14859/? A/DEBUG: pid: 12914, tid: 12914, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:46.812 14864-14864/? A/DEBUG: Cmdline: com.tbolt.main

2023-11-03 14:42:46.812 14864-14864/? A/DEBUG: pid: 12914, tid: 12985, name: com.tbolt.main >>> com.tbolt.main <<<

2023-11-03 14:42:46.856 2299-14876/? W/ActivityManager: crash : com.tbolt.main,10274

2023-11-03 14:42:46.856 2299-14876/? W/ActivityTaskManager: Force finishing activity com.tbolt.main/.MainActivity

2023-11-03 14:42:46.985 2299-5904/? W/InputManager-JNI: Input channel object 'bebdddd com.tbolt.main/com.tbolt.main.MainActivity (client)' was disposed without first being removed with the input manager!

oliverbent

To troubleshoot dropped frames in RTMP streaming using MSDK, consider these possible causes:

Mobile phone performance: Check if your phone's settings impact streaming. Some settings, like power-saving modes, can affect background app operations.

RTMP server issues: Verify that your custom RTMP server is functioning correctly.

APP code: Ensure your MSDK-based app is free of bugs.

To pinpoint the issue:

Test streaming on a third-party platform.
Try using official GO 4 or Sample Code.
System settings on Android devices can affect performance. For example, MIUI's "Performance Mode" can improve streaming, while "Power Saving Mode" may cause dropped frames due to restricted background operations.