For people reading this:
I manage to get an average value in CalculatedField by providing an additional ReqiredField based on a dedicated property used for counting. In my model I used a read-only property that just return 1. The aggregated value (SumAggregate) gives the total number of items, then you just have to divide.
Note: the total number of item could not be retrieved directly because actual implementation of IAggregateValues and IGroup are not accessible. PivotGrid is good but full of such limitations !