I don't think the shortest path function is nicely differentiable. Consider two points (origin and destination) with the same x-coordinate in the x,y-plane. Now suppose there is a horizontal wall obstructing the straight path from one point to the other. If you move the origin to the left, the shortest path goes around the obstruction's left side. If you love it to the right, the shortest path goes around the obstruction's right side. At some point in the middle, the shortest path function jumps from one path to the other, in a non-continous manner, which suggests that the function is not differentiable at this point. In fact, there must be a curve (not just a point) along which the shortest path function is not differentiable.

I don't know the specifics of your problem, and I think the non-differentiability may be worked around with some effort, but perhaps there's a simpler approach? A* has long been used to solve pathing problems in videogames. It may be worth checking out.