Hi there,

Is there any chance of getting some sort of API documentation on how to send control commands and access the video stream from PC without using the official application? Also, is it perhaps planned to open up the python API a bit more to allow for more complex computer vision tasks?

I have exactly the same question. The PC app unfortunately looks like a blown up version of the iphone app with terrible resolution. Like Toumal, I just want to be able to code outside of the official app and be able to debug and upload code. I really do hope they release something for advanced users. I get that this is an introductory toy for kids into robotics, but it would be if they can release the API for the rest of us.

It seems they are part way there. There is okay documentation in the existing programming guide to get us started, but there isn’t any detail on how to connect outside of the app. Hopefully it is or will be open so that third party app developers can have a crack at it.  

Also in the app you can save your code to the cloud, and pull stuff in from the cloud... but I haven’t found a way to access that cloud storage from outside of the app (I haven’t dug very far...). We need to be able to import code from outside of the app so that we can create libraries of shared capability. GitHub here we come.

Finally something to note is that everything is interpreted in the app, as far as I can tell. Logic isn’t being executed on the controller. I could be wrong here, but set your expectations accordingly.

The great thing I see is that all of these things are just opportunities for the community to innovate and create new addons and capability. As long as DJI provides some open access to facilitate modification without having to break warranty, we should be all good.

Basically, I'd like DJI to release enough information for people to control the bot from a PC without needing their app, as well as for the python API to be powerful enough to get as much functionality as needed for AI robot battles similar to Robocode (https://robocode.sourceforge.io/)

We've already seen another thread here asking for why you can't just fire the IR blaster via code, only the projectile launcher. This is the kind of stuff I'm talking about. I also want to run OpenCV on a computer so getting the video stream and sending control information to and from other programs is vital for this. This would also hugely increase the value proposition of the S1.

They really do need to improve the app for PC. The video feed is horrible, and it's incredibly annoying not being able to have the app in windowed mode. Also is anyone having problems with the premade programs acquiring targets?

Didn't mean to double post

My S1 arrived at the weekend and I’m pretty pleased with my new little toy.

What bothers me most is the little amount of available documentation. The S1 is advertised as programmable educational robot and I want to know how to access the robot without the really bad PC port of the closed source Robomaster application (made in Unity Engine).

I want to know how to directly connect via Python, what libraries & APIs are available, how to make use of the additional UART, PMW & USB ports... e.g. I wanna add a laser distance sensor to program more advanced stuff - without proper documentation this kind of stuff is nearly impossible.

Using nmap and Wireshark I was able to access the S1 via FTP. Looks like it’s running some kind of BusyBox implementation and the Camera is a D-Link DCS-932L. Don’t get me wrong, it is part of the fun to hack into a new product and find out all this information by yourself, but I don’t have the time to reverse engineer the whole thing.

The Robomaster Challenge documentation & software is way superior & detailed, I really don’t understand why DJI choose a different path for the S1.

@Sevre The main communication happens via UDP. This includes both the camera feed, telemetry data as well as the control commands from the PC.

What DJI could do is to provide basic documentation of the UDP format for us to create support libraries for python, java, etc. - that way, people who just want to use the provided app can do that, and those who want to do more can dive in and do everything.

I do some testing  under direct connect mode.  You do can  control DJI by generating UDP  packets。 For example， host ip 192.168.1.6 robomaster s1 ip 192.168.1.8， udp  src port 10607 dst port 10607 .generate udp packets   ， it  can trigger robomaster shoot.   but as @sevreNniarB said, caputure and analysis these packets will cost a lot of time.

sevreNniarB Posted at 7-8 03:00
My S1 arrived at the weekend and I’m pretty pleased with my new little toy.

What bothers me most is the little amount of available documentation. The S1 is advertised as programmable educational robot and I want to know how to access the robot without the really bad PC port of the closed source Robomaster application (made in Unity Engine).

Can't agree more! This robot is supposed to be programmable, and apparently DJI can make it programmable (just look at those robot battle code on GitHub!). So why reserve the ability from the commercial users?

wqiu Posted at 7-14 02:36
I do some testing  under direct connect mode.  You do can  control DJI by generating UDP  packets。 For example， host ip 192.168.1.6 robomaster s1 ip 192.168.1.8， udp  src port 10607 dst port 10607 .generate udp packets [view_image]  ， it  can trigger robomaster shoot.   but as @sevreNniarB said, caputure and analysis these packets will cost a lot of time.

@wqiu Hi, could you share the method or software to capture the s1 udp traffic?

Wow, I never would have thought of back-hacking the Wifi to control the S1!  That's some serious coding!

From what I can see, there is no current way to get info into the S1 other than the canned api for the few sensors given.

Yes I am seeing some difficulty with locking on to the Vision Markers, but my lighting is poor in my only work area, so I am installing bright LED headlights that plug straight into an open PWM port, which is similar to plugging into an RC receiver, such as in an RC car, to plug in the steering servo.  The canned api should allow for program control of the brightness.

The PWM ports allow for program control of standard RC servos - but I have yet to imagine what I would do with those.

I don't think those ports are I/O, just Output only.

In the app, don't forget to explore the Road to Mastery - there are some GREAT lessons in there, including the code for Blue Tape Line following, which works excellently by the way.

Can someone please link me to some recommended robot battle software on GitHub?

And yes I DITTO the call to make it all Open Source!  Especially the app!  

Fantasy Island:  It would be super cool to install a NVIDIA Jetson Nano as the controller board.  That would unlock all kinds of deep learning and trippy AI stuff.

As far as I can tell, vision processing, networking and overall code execution happen in the module that is located over the gun. The module in the back, where all the sensors and motors plugs in, appears to be mainly a CAN-Bus hub. It may do some of the PID processing for the motors too.

What this suggests to me is that a good place to assert control is by replacing the “head” module with something else, maybe a Raspberry Pi with a camera, and tapping into the CAN Bus.  It is extremely unlikely that any kind of encryption or protection has been applied to the CAN Bus.

This would let you use any of the various remote control / video streaming systems that already exist for the Pi and similar. Should be easily reversible, just by plugging in the original module.

I’ll try sniffing the CAN Bus with a logic probe later today.

zezrum Posted at 6-14 08:32
It seems they are part way there. There is okay documentation in the existing programming guide to get us started, but there isn’t any detail on how to connect outside of the app. Hopefully it is or will be open so that third party app developers can have a crack at it.  

Also in the app you can save your code to the cloud, and pull stuff in from the cloud... but I haven’t found a way to access that cloud storage from outside of the app (I haven’t dug very far...). We need to be able to import code from outside of the app so that we can create libraries of shared capability. GitHub here we come.

As I understand it, programs actually run on the robot, not on the phone/computer.

Rechard Posted at 8-1 06:23
@wqiu Hi, could you share the method or software to capture the s1 udp traffic?

I install "wireshark" on my  PC， run DJI APP on PC. then you can caputer udp packets.

You can also sniffer CAN bus control frames.
I find ID 0x0201,0x0202,0x0203,0x0204,0x0211,0x0212,0x0213,0x0214,0x0215,0x0216  from CAN frames so far, each of them correspond to one CAN node device. Most likely, the first two of them stand for DJI intellegent controller and motion controller.
Such as:
00000,="22:40:09.538",452.9922,ch1,rec,0x0201,data frame,0x06,x| A0 48 08 01 5C DB
00001,="22:40:09.538",512.3520,ch1,rec,0x0202,data frame,0x05,x| 00 00 00 D6 7C
00002,="22:40:09.538",512.3531,ch1,rec,0x0203,data frame,0x08,x| 55 16 04 FC 04 09 A2 9C
00003,="22:40:09.538",512.3532,ch1,rec,0x0203,data frame,0x08,x| 00 3F 76 35 FF 00 00 87

Bruno59 Posted at 8-2 14:49
As I understand it, programs actually run on the robot, not on the phone/computer.

I would second that motion.

As an example, running an Autonomous program is triggered on the Intelligent Controller and at that point, the robot cuts off any output coming from the outside, including requests to run the Custom Skill program.

Looks to me like DJI precipitated the release of the RoboMaster S1 to synch with the RoboMaster 2019 tournament. Production certainly seems to have been taken off guard, as S1s were shipped at a very slow pace.

The SDK/API documentation seems to have been the next victim. When you consider the quality of the SDK documentation for the DJI Tello and Tello EDU, the other consumer STEM Education robot in the DJI Catalog, one can only be but baffled by how thin the RoboMaster Docvumentation is.

Hopefully, over the coming weeks, we will see better material becoming available.

wqiu Posted at 8-17 07:07
You can also sniffer CAN bus control frames.
I find ID 0x0201,0x0202,0x0203,0x0204,0x0211,0x0212,0x0213,0x0214,0x0215,0x0216  from CAN frames so far, each of them correspond to one CAN node device. Most likely, the first two of them stand for DJI intellegent controller and motion controller.
Such as:

Can you share how you configured your sniffer? I have a Bus Pirate and a USB logic analyzer, both working with PulseView, but I have yet to figure out how to get the sniffer in-line between the top controller and the motor controller.

Did you build a cable? Or just use female jumpers to the pins exposed in the CAN BUS port?

Yes, I just use two female jumpers to the pins exposed in the CAN BUS port. There are four pins for each port. two pins are  11V power  souce. two pins are CAN interface.

gpvillamil Posted at 8-19 17:25
Can you share how you configured your sniffer? I have a Bus Pirate and a USB logic analyzer, both working with PulseView, but I have yet to figure out how to get the sniffer in-line between the top controller and the motor controller.

Did you build a cable? Or just use female jumpers to the pins exposed in the CAN BUS port?

The black wires all share the same differential signal. You can tap into the black bus by connecting to any unused black connector. There's one on the top of the turret with a rubber plug which may be used. All the hit detectors have a spare connector which may also be used.
The orange motor bus is separate from the black CAN bus.
If one motor bus connection isn't connected, then none of the motors will turn on.
The black CAN bus runs at 2,000,000 bits per second. The orange motor bus uses 921600 bps.
I had to make an adapter to sniff the motor bus.
The Reddit user ReasonablyClever pointed out the battery communicates to the robot via I2C.
The battery changer only uses the data line of the I2C bus to communicate with the battery.
The battery contains three 18650 Li-Ion cells.

Duane Degn Posted at 8-28 17:33
The black wires all share the same differential signal. You can tap into the black bus by connecting to any unused black connector. There's one on the top of the turret with a rubber plug which may be used. All the hit detectors have a spare connector which may also be used.
The orange motor bus is separate from the black CAN bus.
If one motor bus connection isn't connected, then none of the motors will turn on.

Extremely helpful, thanks!

Being able to tap into an unused port makes life a lot easier!

Do you have any thoughts on what the M0 port might be?

Duane Degn Posted at 8-28 17:33
The black wires all share the same differential signal. You can tap into the black bus by connecting to any unused black connector. There's one on the top of the turret with a rubber plug which may be used. All the hit detectors have a spare connector which may also be used.
The orange motor bus is separate from the black CAN bus.
If one motor bus connection isn't connected, then none of the motors will turn on.

The black wires all share the same differential signal. You can tap into the black bus by connecting to any unused black connector.
The black CAN-Bus nodes are configured in a Hub. Loose one node, everything else continues working normally.

There's one on the top of the turret with a rubber plug which may be used. All the hit detectors have a spare connector which may also be used.
The S1 is obviously designed for growth and expansion. I can't wait to see what surprises will pop-up in the future.

The orange motor bus is separate from the black CAN bus.
If one motor bus connection isn't connected, then none of the motors will turn on.
The Motor CAN-Bus (M-Bus) daisy chains the motor control nodes. It is logical, especially with mecanum wheels that the loss of one wheel makes the entire drivertrain useless.

From what I understand about CAN Bus, it's a shared bus, so disabling all the motors when one motor is disconnected is not a physical connectivity (there's a single wire from each motor to the controller), but rather a logical design decision, i.e. if not all motors are reporting their statuses on a bus, the controller will not turn any of them on.

-albertr

gpvillamil Posted at 8-29 12:28
Extremely helpful, thanks!

Being able to tap into an unused port makes life a lot easier!

"Do you have any thoughts on what the M0 port might be?"
I don't.
I've started documenting some of my hacking efforts on my Hackaday IO page.https://hackaday.io/project/167276-dji-robomaster-s1-hacks
I've added some photos of the inside of the battery and the charger.

Duane Degn Posted at 8-31 20:59
"Do you have any thoughts on what the M0 port might be?"
I don't.
I've started documenting some of my hacking efforts on my Hackaday IO page.https://hackaday.io/project/167276-dji-robomaster-s1-hacks

Here's a better link to my Hackaday IO page.

Duane Degn Posted at 8-31 21:03
Here's a better link to my Hackaday IO page.

The link feature didn't work the last two times I tried it. Maybe DJI doesn't like external links?
My Hackaday IO page can be found by searching my user name.

albertr Posted at 8-29 19:16
From what I understand about CAN Bus, it's a shared bus, so disabling all the motors when one motor is disconnected is not a physical connectivity (there's a single wire from each motor to the controller), but rather a logical design decision, i.e. if not all motors are reporting their statuses on a bus, the controller will not turn any of them on.

-albertr

I'm pretty sure you are correct. The act of disconnecting the motor doesn't itself disable the other motors rather the controller won't issue any drive commands if all four motors don't acknowledge their presence.
To get around this, I made a "T" pin to allow me to sniff the M Bus.

I connected jumper wires to these T pins and used a RS-495 chip to convert the differential lines into something I could feed into my logic analyzer.

The M Bus transmits at 921,600 bps.
The CAN bus uses 2 Mbps.
Here's a small portion of the traces captured.

I've included the logic analyzer file in my Hackaday IO project page (I use the same user name.)

Duane Degn Posted at 9-1 07:02
I'm pretty sure you are correct. The act of disconnecting the motor doesn't itself disable the other motors rather the controller won't issue any drive commands if all four motors don't acknowledge their presence.
To get around this, I made a "T" pin to allow me to sniff the M Bus.
[view_image]

"The M Bus transmits at 921,600 bps."
After investigating a bit more, I'm not so sure the baud is exactly 921,600.

921,600 baud is non-standard speed for CAN Bus. Could it be 1000 Kbaud?

-albertr

This is genius!  I love it, especially the logic analyzer, I used to use one all the time.   The baud rate calculation will always be approximate in a system like this.  While it's pretty to look at, I'm not sure what you can actually do with this data.

albertr Posted at 9-2 06:40
921,600 baud is non-standard speed for CAN Bus. Could it be 1000 Kbaud?

-albertr

The baud of the Motor Bus is not consistent. I'm pretty sure the Motion Controller uses 921,600 because the bits on the logic analyzer stay in sync with transitions. The replies from the motors (at least what I interpret as replies) use a different baud. It's close to 921,600 but the bits and transitions lose sync after a few bytes of data. It's NOT 1Mbps. It's within a few K of the Motion Controller baud.

Hmm, IMHO that is highly unlikely. The RX and TX should use the same clock frequency if it's synchronous communication. And CAN Bus doesn't use 921,600 baud, could it be just high speed serial UART connection? Or maybe there're something wrong with the sniffing hardware you are using?

-albertr

albertr Posted at 9-2 14:04
Hmm, IMHO that is highly unlikely. The RX and TX should use the same clock frequency if it's synchronous communication. And CAN Bus doesn't use 921,600 baud, could it be just high speed serial UART connection? Or maybe there're something wrong with the sniffing hardware you are using?

-albertr

Here's an overview of a M Bus communication.


I have the logic analyzer set to decode at 921600 bps. You can see in the close up of section A the transitions line up well with the bits read.

A close up section B shows the transitions no long keep sync with the bits being read at the same assumed baud.

I'm guessing the initial series of alternating highs and lows is supposed to tell the receiving device which baud to expect. This seems like a strange way to do things but I don't know how else to explain the difference in apparent bauds used.
I've attached a logic analyzer file to my Hackaday project page. I think Saleae has a viewer available for anyone interested in taking a closer look at the traces.
The traces attached to my Hackday project page are not the same as the one shown here. I'll make this file available if anyone is interested in seeing it.

did you figure out the protocol?
I have been working on the Agras T16 and I am in an identical boat, misalined bits when using the can protocol.
I switch to Async Serial at 921600 and thins are slightly misalined but look better.