Well, I managed to work around the problem by subscribing to the AutoGeneratingColumn event and then just canceling it:
I'd still be curious to know what it happening behind the scenes to make this work when you set that property as the above solution is more of a hack.