Is anyone making any progress with decoding/interpreting the Phantom 3 Flight Log files, in the *.DAT files?

Of course, we have the flylitchi online web-based java converter, which will take the *.txt log files, made by the DJI Pilot application, and turn it into a CSV file, for use with dashboard, and of course you can use the "User Center" in the DJI Pilot application to read some of the data from the Flight Log Files, and display it.  But it would really be nice if you can get the original log data, which is recorded onto the Phantom memory while actually flying, regardless of radio reception, and read from these.

There is a rumor going around that these *.DAT files are encrypted, but I'm pretty sure they are not.  They contain enough ascii at the beginning, and have enough pattern throughout them to know that's not the case.  If you look at the binary data, it's definitely broken down into visible chunks of data, and very frequently it has "runs" of a repeating byte, and that repeating byte counts up slowly, 1, 2, 3, 4, ..... up to 255, then back to zero, 1, 2, 3, 4, .... and so on, repeatedly.  It seems to any true encryption algorithm would render the entire file looking like random numbers, with no runs of patterns (unless different blocks were encrypted separately, like a form of EBD coding.)

Is anyone working on this?

Being able to read the *.DAT files directly and extract information would be much preferable to the *.txt log files which the DJI Pilot application produces and reads, because the *.DAT files would be written to the phantom even when there's no radio link (like when using ground station) or when radio link is buggy, or when you want to use third party controller applications, and skip DJI-Pilot directly.

I realize that many people who read this have their own "proprietary" reasons for wanting to crack the code, as they're hoping to profit from sellable applications.  That's fine, but surely there must be some hobbyists who are working on this?!?!?

- tj.

Ahhh! Patience Grasshopper!

Like many before me, I used the DJI Pilot App to get into Flight Data Mode. Once in that mode and pressed OK, I was instructed to connect the USB port to the PC.

The USB port in question is the one on the front of the craft , NOT the one on the gimbal. The USB port on the gimbal is used to either read data/media from the SD card or for updating, if necessary. However....

The day before yesterday I exchanged emails with the owner of the flyitchi web site. He said we could not use the data files that are derived from that USB port - they will have errors, i.e., corrupted in some way. He instructed me as follows.

Connect your mobile device to your PC. In my case I'm using a Galaxy S5, so it's very straightforward. I just connect the appropriate charging/data cable from the S5 to my PC. A dialog box should open up that will allow you to see the files on your phone/mobile device.

Look for the DJI folder (it might say DJI Files). In that folder, you will find the pilot log folder which contains the flight records for your craft. They should be in a *.txt format. Leaving that screen available, open your browser and navigate to www.flyitchi.com - then choose LOGS. In the lower left-hand corner you'll see a small dialog box asking for the file. Copy and paste one of the .txt files from the pilot logs on your mobile device and press CONVERT to CSV.

As long as the file is not corrupted, you'll get a download to your PC (I'm using WIN 7 - don't know what happens on a MAC). When the download is complete, click on it and it should open Excel, with the contents of the file displayed.

It is highly detailed. My most recent flight of just under 6 minutes displays about 2,000 lines of data. There's about a dozen or so different items that are displayed. The data more or less matches what I see in my Pilot Log data available through the Pilot App, but with details given every so many fractions of a second. No course data (like a line on a map) is displayed, but geographical coordinates are displayed in two of the fields (one for lat and one for long).

I think this flylitchi data was originally designed to be used with the Phantom 2 Vision+ to create autonomous flight plans. Other than be fascinated with the detail of the data, I'm not sure how they could be utilized to produce more info than you get in the summary logs in the pilot app.

Hope that info helps.

aburkefl@gmail. Posted at 2015-5-17 04:24
Ahhh! Patience Grasshopper! ...[In short, he explains above how he started to look into the *.DAT files, but found out it wasn't easy, so instead he uses the DJI-Pilot Application to generate *.txt files, then converts them with the flylitchi program into telemetry CXV files, and that works just fine for him...]

Aburkefl:

Thanks for the response.  Yes, I know how to do the above, but there would be significant benefits of being able to read and convert the *.DAT files, in spite of the fact that it's not easy.  For one thing, the stored *.DAT files on the Phantom itself won't be dependent on having a working data link back to the transmitter/controller, and the *.DAT files should record even during autonomous (groundstation) flights, once that feature is available.  And the *.DAT files will have a ton of information in them, including yaw, and other details, which the flylitchi program doesn't give access to (in spite of the fact that that data is in the *.txt files.)  And once the 3-rd party software comes out for the Phantom-3, the *.txt files will not be available on those 3rd party flying programs.

But I do appreciate you taking the time to respond!

- Thomas.

tejones777 Posted at 2015-5-17 09:33
Aburkefl:

Thanks for the response.  Yes, I know how to do the above, but there would be significa ...

My apologies. If I had read your post just a little more carefully....

Still curious though, what makes you feel there's even more/better data available in the .dat files if we can ever get to them?

Someone representing DJI claimed they would NEVER create waypoints and autonomous flight for the P3. Perhaps those who really want that feature will eventually get it via other developers using the SDK?

TJ,
As part of us developing our own Ground Station app, we made a little utility for viewing the Inspire 1 log files and we just tested it with Phantom 3 log files and it seems to work!

https://www.mapsmadeeasy.com/log_viewer

It currently only shows the the map of the flight path with a plot of altitude and velocity vs time but it is better than having all those sweet log files on your drone and not being able to do anything to do with them.

I tried the MapsMadeEasy and it works!  Just click on the Choose File button and select one of your .DAT files.  Depending on the size of the file (a progress bar is shown), your flight path will be displayed for the points and vectors contained in the loaded file.  I did this with files from my P3A.

mswall@mac.com Posted at 2015-5-27 09:18
I tried the MapsMadeEasy and it works!  Just click on the Choose File button and select one of your  ...

So, these P3A .DAT files can be pulled from the P3A USB port?

Yes they can.  Be patient and get everything connected - phone/tablet to RC - turn on.  Then turn on P3 without props installed.  Now go into settings in the Pilot app:
MC Settings --> select Advanced Settings
Advanced Settings --> Enter flight data mode - now wait until you get a dialog telling you that the quad is in flight data mode.  when this happens, connect THE USB CABLE DJI PROVIDED from the nose of the quad to your computer.  I use an iMac.  The data drive should mount and be available - I think its called NO NAME just like the SD card is.  You will find your .DAT files in there.
It did not work the first time for me because my P3 was too close to the iMac - then because I was not using the DJI USB cable - and then because I did not give it time to notify me that the quad was indeed in Flight Data Mode.  It will work.

any updates on this?

fredz Posted at 2015-6-29 02:29
any updates on this?

Yes - after retrieving your FLYxxx.DAT files from your Phantom, visit www.djilogs.com where you can convert your files directly to CSV files. Then you can either review the data in Excel or take the file to www.flightreplay.com

Can I find somewhere .DAT files structure ?

Is it possible to dave this data to your computer as the local files are filling my phone up and I dont want to delete them as they are good record???Would it not make sense to have a base station app for your computer???

Hi guys,

After experiencing my first 'flyaway' yesterday (thankfully I got her home with just a broken propeller) I decided to take the plunge and try to reverse engineer these files.

I've made some good progress; they are quite straightforward actually; there's a fixed length header and then it's basically an binary event log. Made up of a list of records of different types. Each record has a similar header, and the records for a given type are all the same size.

Record Format:
UInt16 identifying the record type (seems to always be of the form 0x55XX (where XX is the record type number)
UInt16 (Unkown)
UInt16 (Unknown)
UInt16 (Timestamp)
<payload, where of variable size depending on record type>

There seems to be a very simple sort of 'encryption' going on, where each byte in the payload requires and increment of 'timestamp' and then mod(0xFF).

I've got some simple numbers out like lat/long etc.. but will need a bit of help identifying what the values in all the other record types correspond to.

This weekend I'll write up what I have in as a python script or something and put it up on github, hopefully together we can figure it out.

Any keen hackers out there; I have been using 'Hexinator' for looking at the binary files; it is really quite good.

Interesting... I will help for sure .. But I don't think that Hexinator is enough to  decode all the stuff ... Anyway it's a good start. What you mean for  "each byte in the payload requires and increment of 'timestamp' "
??

pgs31 Posted at 2015-7-30 18:52
Hi guys,

After experiencing my first 'flyaway' yesterday (thankfully I got her home with just a bro ...

do you have any news on the script progress? I have found some information about the structure

wukong Posted at 2015-8-5 14:16
do you have any news on the script progress? I have found some information about the structure

Hey pgs31 any more progress with this? I started looking at Inspire dat files with 010 Editor... Haven't got too far yet...

Thx,

V.

vincenzomiceli Posted at 2015-9-19 11:52
Hey pgs31 any more progress with this? I started looking at Inspire dat files with 010 Editor... H ...

Any updates on this?

Capt_M Posted at 2015-10-23 19:50
Any updates on this?

Check out Another FLYXXX.DAT converter. It describes a tool I've developed to read  these files.

BudWalker Posted at 2015-11-4 03:44
Check out Another FLYXXX.DAT converter. It describes a tool I've developed to read  these files.

I am trying to find the actual date/time stamp for log entry.  Each line in the Log File is provided with the "Tick #"...but I don't believe this is referenced to a actual "Date/Time" stamp for the event.  

Has anyone solved this correlation issue?  Am I totally missing something??

lindsay.mohr Posted at 2015-11-23 11:30
I am trying to find the actual date/time stamp for log entry.  Each line in the Log File is provid ...

I'm just now seeing that you replied to my post. Do you know how to cause me to get an email if I get a reply.

To answer your question the .DAT doesn't appear to have a date/time. However, there is a new version of DatConverter called DatCon that has a GUI and is much easier to use. DatCon has a coulme that makes it easy to sync with the .txt produced by the Go App. So if you have the .txt and something like a HealthDrones report you can sync the two.

DatCon can be obtained by visiting https://datfile.net/downloads.html

BudWalker Posted at 2016-3-5 19:07
I'm just now seeing that you replied to my post. Do you know how to cause me to get an email if I  ...

Hi Bud,
thanks for DatConverter.

Is Available the source code?

Thanks ;)

analist Posted at 2016-4-28 03:08
Hi Bud,
thanks for DatConverter.

The source will be available. But, not yet. I'm assuming you're referring to DatCon. The source for DatConverter has already been published.

BudWalker Posted at 2016-4-28 13:32
The source will be available. But, not yet. I'm assuming you're referring to DatCon. The source for ...

Perfect,
let me know.

Thanks again

analist Posted at 2016-4-29 06:46
Perfect,
let me know.

The source for a nonGUI version of DatCon can be found at DatCon Source

BudWalker Posted at 2016-7-20 23:15
The source for a nonGUI version of DatCon can be found at DatCon Source

Excellent, thanks so much! Could you please provide compilation instructions?

foobarbecue Posted at 2016-7-31 11:54
Excellent, thanks so much! Could you please provide compilation instructions?

Nevermind, I figured it out! Just run javac src/*/*.java from inside DatCon/src.

foobarbecue Posted at 2016-7-31 13:00
Nevermind, I figured it out! Just run javac src/*/*.java from inside DatCon/src.

I'm just now seeing this. There seems to be now way to have this forum email me when someone replies to my post. I normally hang out on Phantom Pilots as BudWalker. Contact me there if you need to.

DJI is the only one who has the software to properly and fully read the DAT files.

Can I download a mac file that reads .DAT files? It is obvious that there is different kinds of .DAT files and I'm not sure which ones will work. VLC won't open .DAT files and neither will MailRaider Pro. Any help?

kalev@vaskas.or Posted at 2016-8-10 04:46
Can I download a mac file that reads .DAT files? It is obvious that there is different kinds of .DAT ...

Go to DatCon You'll find instructions on how to make DatCon run on a Mac

DJI-Ken Posted at 2016-8-1 12:43
DJI is the only one who has the software to properly and fully read the DAT files.

DJI should make this available to its customers.  I am dealing with an issue where I flew using DJI's GS Pro and it did not record any flight records to GS Pro or the GO app.  There for I had to retrieve the *.DAT files from the Inspire 1 itself and they need to be converted to upload to my logbook.  I fly under Part 107 regulations and flight logs are extremely important to comply with the FAA.

DemolitionMan14 Posted at 2017-3-15 08:08
DJI should make this available to its customers.  I am dealing with an issue where I flew using DJI's GS Pro and it did not record any flight records to GS Pro or the GO app.  There for I had to retrieve the *.DAT files from the Inspire 1 itself and they need to be converted to upload to my logbook.  I fly under Part 107 regulations and flight logs are extremely important to comply with the FAA.

It's a software program that you wouldn't even be able to read. It's just numbers and graphs and the flight data analysis team has tons of training to learn how to read the data.

And for recording flights for your 107, your flight record in the GO app is just fine as it shows all the data you need. (flight data & time, furthest and total distance, max altitude, location).

DJI-Ken Posted at 2017-3-15 08:20
It's a software program that you wouldn't even be able to read. It's just numbers and graphs and the flight data analysis team has tons of training to learn how to read the data.

And for recording flights for your 107, your flight record in the GO app is just fine as it shows all the data you need. (flight data & time, furthest and total distance, max altitude, location).

The problem is flights that I conducted using Ground Station Pro did not record a copy of the log to the GO App.  The only place they are is on the Inspire 1 in *.DAT format.  Which is why I am trying to convert the *.DAT files to DJI TXT or *.CSV files so they can be used by a 3rd party website to store and view the log.  It would be nice to be able to manual upload the *.DAT files that i need to my DJI GO App flight records.  Maybe a user friendly program that can interperate *.DAT files and upload in DJI TXT format to the GO app flight log storage would be nice to develop.  Or can I send DJI the *.DAT files to convert and upload to my account?