Different results in OD Cost Matrix analysis and Route analysis

I am using OD Cost Matrix layer in order to find distances between many origins and destinations points. It works fine, but I want to make sure the results make sense, and since OD Cost Matrix can only produce straight lines (and not true route shapes), I tried to calculate some specific origin-destination pairs using a Network Analyst regular Route layer and compare them to the OD Cost Matrix results.

However, in some of the cases the distances seem to be different between the two methods, even when I use the exact same analysis settings and network restrictions. Sometimes it's only a small gap, but in other cases it's probably a whole different route.

Shouldn't the two tools produce the exact same routes?