I find it really hard to go down with the phantom 4 and not introduce some kind of shake. It seems like the sticks are very sensitive and if you don't move the stick perfectly downwards, it will start to go left or right ever so slightly. This is very annoying and I don't want that movement when recording video. The whole video is unuseable.

Anybody a solution for that?

Yes, I've noticed this myself.  If you notice, the sticks are not perpendicular and tilt outwards just ever so slightly.

I've found that lowering some figures in EXP and THIS LINK helped a ton.   

I will try that out, thx.