james.mcmillan
lvl.2
Flight distance : 71739 ft
United States
Offline
|
Taking this discussion a bit further, think about how the Osmo gimbals work. They are not getting any feedback from what's being recorded. The feedback is coming from the Osmo's accelerometers or encoders (I don't know which) to keep the camera pointing in the same direction (at the vanishing point), regardless of the handle's position. We know this is true because the Osmo works even when it isn't connected to anything and the lens cover is still on.
To truly track an object, it is necessary for the object's position to be defined - in this case, on the imaging device (phone), then create some kind of feedback loop where the image is continuously analyzed to determine where the object is, then the Osmo must be told how to move the gimbals to re-center the object. For this to be successful, (at least) two things need to happen: 1) the pattern-recognition algorithm must be effective enough to recognize the object as it moves relative to the background, and 2) the "analysis/correction" process must be done fast enough that the object remains within the FOV. A number of things work against this: 1) by definition, since the Osmo's main function is to smooth out the resulting images, it doesn't move the gimbals very quickly or precisely, 2) the lighting on the object is probably changing as it moves, so that stresses the pattern-recognition algorithm, 3) since the object trying to be tracked is (almost always) three dimensional, it will probably be presenting a different profile as it moves, again stressing the pattern-recognition algorithm, and 4) doing all this in real time requires quite a bit of processing horsepower.
In the Osmo Mobile, all of this process except the actual gimbal recentering movements are accomplished in the device (phone) itself via software. To accomplish the same thing on the other Osmos, the image itself must first be downloaded from the Osmo to the device (phone), where the same processing steps described above occur. This image downloading takes time - and is affected by the speed of both the camera and the WiFi connection. Remember, we're talking about video - at 25+ fps. And, of course, the speed of the required processing is determined by the speed of the device itself (phone). The reality is, that the real-time image(s) available for processing (what we see on our devices from the Osmo) may only be running at 10 fps or less and at a reduced resolution. Further, the pattern-matching algorithm is probably only being performed on a subset of those images, otherwise it would fall way too far behind given the lack of computing power.
It's why object tracking that relies on pattern recognition doesn't work perfectly. At the very least, it is inconsistent. And, it explains why the best tracking is accomplished when the object being tracked can always be positively identified by something more robust than a pattern-recognition algorithm running on a low computing power device (phone). If an object's GPS coordinates can be accurately defined, object tracking is much more robust. In fact, Autoframe does have the ability to track via GPS coordinates, but it requires the object to be stationary and its GPS coordinates defined, or a moving object (person, car, etc.) must be carrying a GPS-enabled device (phone). However, since GPS generally doesn't work very accurately inside, it means pattern recognition is often the only option.
In my view, the weak point of doing tracking via pattern recognition is the fact that the algorithm can't be particularly sophisticated because there just isn't the necessary processing horsepower in our devices (phones) to make this work (well). The requirement to move the image from the camera to the device (phone) via WiFi (and probably at a lower resolution), only adds to the problem with the non-Osmo Mobile models. |
|