You're right about that... it doesn't matter because you can always query for the data needed.
However, one of the strengths of using an ORM is that you get a more "natural" of accessing your data...you work with a model, probably of real world entities connected to each other in associations. So if you need a list of addresses for your person you just navigate the Addresses property of your Person instance. I think this is by far more "natural" that having to do a query for the addresses.
From a performance point of view I think using navigable properties is also faster - especially because you can apply fetchplans to your model..either include them in the default fetchplan, meaning they will be fetched when the parent object is fetched. In the above Person-Addresses examples it means if you include the addresses field in the default fetchplan, each time you fetch a Person instance you also fetch the list of addresses for that person - in one query
If using the default fetchplan does not fit (it rarely does) all your scenarios in which you operate on the Person class you can also apply a named fetchplan before you fetch the Person instance needed. For example, in one case you need only to return a list of person names you will just use the default fetchplan...but in another case you need both the properties for person and the list of addresses... in the latter case you can instruct the fetch call (query or other means to load the instance needed) to use a specific named fetchplan... in this example it could be one named "PersonAndAddresses"
Lazy-loading fields and fetcplans are closely related you should definitely check them out in the docs
if you have not already done so.
So, my point is... it might seem to not matter if you query for the data instead... but I still think... if you use the features built in OA you will get a by far better (natural) way of accessing your data...and might even get a performance boost when using fetchplans