I've noticed that one of my batteries has a very small ridge on the right side. It's barely visible but it's there.

Barely visible ridge on the right from top to bottom along the axis where the "CE" sign is at ...

Looking from the top, there's no evidence

Top view

The App tells the following:

Battery Details in-app

There's a tiny voltage difference in cell N°3.
If this would be a stack of 3 common 18350 Li-IMR cell, this wouldn't raise much of a concern on my end but since we're talking about a precious drone and not a vaping device I'm really unsure...

Should I replace it?

its fine as long as its niot soft/bloated or suffering poor balancing.  just keep an eye on it.  i have a battery with a small imprint on it and it runs like normal and im not very gentle on my batteries..

Hi there, thanks for reaching out DJI Forum. Please don't worry, as long as the battery is not bloated and not heating up that's fine. Should you have further question, please don't hesitate to contact us. Thank you.

Thanks for your answers!

I think it's good practice then to check the battery data as soon as it's plugged in the drone before take-off...

I think the battery is fine, I have 2 batteries that look just like your battery and the voltage discrepancy is normal....

I'd say you're fine. Just mark the battery so you know to check the voltage on it when you use it. If you start seeing an increase in cell deviation, then I'd pull it. New battery is $80 and not worth losing your drone over.