Serialization and Eager Loading

[Pedro]

Hello,

I'm having a problem with Lazy/Eager Loading and Serialization. I'll try to explain the issue conceptually first and then provide some code to backup my findings.

First, for what I understand, when I have a relationship like Entity1 -> Entity2 in a property like Entity1.Two, the ORM engine will replace the reference in Entity1 with an object of its own which will handle lazy loading behavior. After the initial load, when I first access a field in Entity2 (say Entity1.Two.Name) the engine will load data and fill the Entity2 actual object.

Second, I understand also that if I try to serialize an Entity1 object, it won't go automatically filling it's dependant relationships and the non-loaded lazy-load relationships will end up as null on the serialization.

The issue I'm facing it seems that this Telerik-LazyLoading-Object is not serializable and not even using a FetchStrategy to force an eager load seem to fix it.

So, talking code now, I have a very simple entity setup (to keep the example straightforward Entity1 has some simple binary serialization embedded):

[Serializable]
public class Entity1
{
    public Int32 Id { get; set; }
    public String Name { get; set; }
    public Int32 TwoId { get; set; }
    public Entity2 Two { get; set; }

    public byte[] GetSerialized()
    {
        using (System.IO.MemoryStream aStream = new System.IO.MemoryStream())
        {
            BinaryFormatter aSerializer = new BinaryFormatter();
            aSerializer.Serialize(aStream, this);
            aStream.Position = 0;
            return aStream.ToArray();
        }
    }

    public static Entity1 LoadFromSerializedString(byte[] serializedString)
    {
        using (System.IO.MemoryStream aStream = new System.IO.MemoryStream(serializedString))
        {
            BinaryFormatter aDeserializer = new BinaryFormatter();
            aStream.Position = 0;
            Entity1 e1 = (Entity1)aDeserializer.Deserialize(aStream);
            return e1;
        }
    }
}

[Serializable]
public class Entity2
{
    public Int32 Id { get; set; }
    public String Name { get; set; }
    public IList<Entity1> Ones { get; set; }
}

And also a Web Forms project for testing. The project consists of a single page with a single button. When we first load the page, we load an Entity1 which already exists with a Strategy which eager loads its Entity2 reference. We serialize Entity1 and store it in the viewstate. Later, when we click the button we land on the other branch and try to rebuild Entity1 by deserializing it.

public partial class Index : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            if (!Page.IsPostBack)
            {
                Domain.PresenterContext aContext = new Domain.PresenterContext();

                Telerik.OpenAccess.FetchOptimization.FetchStrategy aStrategy = new OpenAccess.FetchOptimization.FetchStrategy();
                aStrategy.LoadWith<Domain.Entity1>(x => x.Two);
                aStrategy.MaxFetchDepth = 10;
                aStrategy.MaxResultsLimit = 100;
                aContext.FetchStrategy = aStrategy;

                Domain.Entity1 e1 = aContext.Ones.Single(x => x.Name == "1_2");

                byte[] serialized = e1.GetSerialized();
                ViewState["e1"] = serialized;
            }
            else
            {
                byte[] serialized = (byte[])ViewState["e1"];
                Domain.Entity1 e1 = Domain.Entity1.LoadFromSerializedString(serialized);
            }
        }


        protected void Button1_Click(object sender, EventArgs e)
        {

        }
    }

What I expected was that since I eager loaded Entity2 reference, the serialization process wouldn't have a problem to include the reference. But it seems it's not  the case.

As it so happens, if I insert a breakpoint right after the entity1-fetching-line and inspect object e1 (acessing property "Two" and forcing the actual load) the serialization would run just fine and when I deserialize it, Entity2 would be present. If I don't inspect e1, Entity2 will end up null after deserializing. This behavior seem to be inaltered by the presence or not of the FetchStrategy, tough I have some strong evidence that the eager loading is acting ok. It just seems that despite eager loading, the Entity2 reference is still replaced by some sort of proxy which fails to serialize if I don't access it first.

Is it by design? Or have I misinterpreted the engine and I'm screwing up somewhere?

[Damyan Bogoev]

Hi Pedro,

I am afraid this is an issue with our fetch strategies and LINQ queries processing. We will investigate the cause for the issue and will fix it.

I am sorry for the inconvenience caused.

Regards,
Damyan Bogoev
the Telerik team

[Thomas]

Hi Pedro,

this is not an issue with the FetchStrategy, but a side effect of the workings of OpenAccess:
When an instance is loaded, not all fields are necessarily loaded as well. Some can be lazy loaded, which is controlled by the enhanced code in conjunction with a StateManager instance. This StateManager instance is not serialized per default; the same applies to the OpenAccessEnhancedFlags field that we introduce.
When an entity is now serialized, there is no upfront notification that serialization will take place. Hence a lazy loaded field will potentially be null, resulting in a null during a deserialization. This explains also why the effect is different when you look with the Visual Studio debugger in the object prior the serialization: then the lazy loading is triggered by the debugger inspection, and the field content therefore available afterwards.

What to do now? Either you call

((Telerik.OpenAccess.SPI.dataobjects.PersistenceCapable)entity).OpenAccessEnhancedPreSerialize();

which is the preferred way, or you just check for the .Count on the collection member before in code (hence triggering the lazy load); actually, all fields must be checked (accessed) prio to serialization.
Another way to do this is to set a surrogate selector in the binary formatter and let this selector perform the OpenAccessPreSerialize().
The third option is to call this OpenAccessPreSerialize() method in the ISerializable.GetObjectData() when you implement the ISerializable interface on your persistent classes.

I'm afraid there is not much we can do without changing a fundamental principle of OpenAccess (lazy loading), and we have no chance to get notified by the binary serialization process.

Regards,
Thomas
the Telerik team

[o]

can you please explain more about this:
OpenAccessEnhancedPreSerialize();

what does it do?

and also if you can explain how can i use that method to serialize an object graph?

[Thomas]

The OpenAccessEnhancedPreSerialize method is a method that the enhancement step during build generates. It informs the internal state manager  to populate the fields in the user instance with the values already fetched from the database. Notice: OpenAccess can fetch values from the database without making the values visible immediately in the user visible managed object. PreSerialize just closes this gap explicitly. The same would happen if all fields were accessed by enhanced code.

In order to serialize the whole object graph, the values of the logical fields need to be transported to the physical fields in the objects. Without this step, the serialization process will just see null values. PreSerialize will avoid this.

Regards,
Thomas
Telerik