that's not that much of a difference unless you go to X5 Raw,
The sensor on the X5 is larger and has more dynamic range, plus the interchangable lens provide greater latitude in capturing the scene the way you want it.
In this forum thread, Ian Cresswell provides his insights on the merits of the X5R in the following Review and Behind the Scenes video.
I have the X3 and X5 and i use them for different situations. With the X5 with M Zuiko lenses i take zero chances vs with the X3 i am less stressed flying haha