Data not saving when using InsertAsync

[Simon]

I've used the Tasks sample as a baseline but I'm using manual code behind to save data to the data store, however data does not seem to be persisting between runs.

I save the data using "insertAsync" and check the context contents and my data is there.
But when I query the contect on the next run, the data store is empty.

Am I missing something obvious?

[Deyan]

Hello Simon,

Thanks for writing.

I am not quite sure why this happens based on the provided description. We have addressed some issues on our side in the SynchronizationContext class which could lead to similar problems.

Can you please share with us your project for further investigation so that we can make sure we have enough details to be able to help?

Additionally, we have released an Internal Build two days ago which contains the aforementioned fixes. Download it and paste the binaries into the Binaries folder of your Telerik RadControls for Windows Phone installation by backing up the old ones and see if there is a change in the behavior.

Thanks for your time.

Regards,
Deyan
Telerik

[Simon]

I've uploaded the package to SkyDrive - http://sdrv.ms/10CKDAW (uses NuGet package restore so if you need to run it, it should fetch the necessary packages)

The synchronisation code is in the MainViewModel class as in the test package.

P.S. I ended up having to un-install the Json reference and add the up to date Json Nuget package else it wouldn't run.

Will check out the latest dev build

[Simon]

Just checked the release notes for the latest internal build available:
"RadControls for Windows Phone [phone] Version"

Doesn't say anything about the cloud sync but I'll give it a whirl anyway

[Deyan]

Hi Simon,

Can you please share the steps I need to take in order to reproduce the issue with your project.

I was able to run it but I don't exactly know how to proceed in order to see what happens.

Thanks for your time!

Regards,
Deyan
Telerik

[Simon]

Apologies.

You shouldn't really have to do anything apart from login and register.

As it stands in the app:

• Login to Cloud
• Login with facebook (the process is compressed for testing
• Once start screen is active your profile details should be stored in the cloud and on the device.

However when the "AddUpdateContact" method is called in the MainViewModel, the local DB seems to be updated.
When next the app runs the DB is empty and it tries to insert your profile again.

When the test query for your details runs:

	var syncContact = Contacts.FirstOrDefault(c => c.FbId == contact.FbId);

the value is always null no matter how many times the app is run.

[Deyan]

Hello Simon,

Browsing through your code, I noticed some things which are not entirely correct:

internal async void AddUpdateContact(FindMeContact contact)
{
    try
    {
        var syncContact = Contacts.FirstOrDefault(c => c.FbId == contact.FbId);
        if (syncContact == null)
        {
            await this.contactsContext.AddAsync(contact);
        }
        else
        {
            syncContact.FbId = contact.FbId;
            syncContact.DisplayName = contact.DisplayName;
            syncContact.DisplayImageUri = contact.DisplayImageUri;
            syncContact.Tracked = contact.Tracked;
            await this.contactsContext.SynchronizeItemAsync(syncContact);
        }
    }
    catch (SynchronizationException ex)
    {

    }
}

Here, you are either adding a new contact in the context or synchronize an existing one.

The notes I would like to make are as follows:

1. The AddAsync method stores the item locally and schedules it for synchronization. It does not upload it in the Cloud. You will have to call the SynchronizeAsync method exposed by the SynchronizationContext to make sure the object is uploaded. The DeleteAsync method works in the same way.

2. In the case when the contact exists, you need to change its properties and again call the SynchronizeAsync method exposed by the SynchronizationContext to make sure the object is stored in the cloud. We are currently updating our Online Help to make sure these details are described.

I have also taken a look at your View Model classes. An important note to make is that you must call the PropertyChanged method for each property to make sure changes are tracked. Just as we do on our models shown in the online help here:

http://www.telerik.com/help/windows-phone/radcontrols-cloud-synchronization-manualintegration.html

Note the implementation of the Task class.

3. The SynchronizeItemAsync method is intended for internal use only. It is public because it is defined by an interface but we will hide it using an explicit implementation.

You can also use the SynchronizeAsync method exposed by your own objects that inherit from SynchronizableDataItem. Calling this method will add the item to the corresponding SynchronizationContext and will additionally initiate a Synchronization call to make sure the data is uploaded on the Cloud.

I hope this is helpful.

Let me know should you have additional questions.

Regards,
Deyan
Telerik

[Simon]

OK, so if I understand what your saying correctly, the "SynchroniseItemAsync" call is unnecessary.

By searching an entry and updating it's values should be enough to update a DB entry, then call the SynchroniseAsync() method afterwards to upload changes to the cloud?

Which sounds fine but doesn't explain why the DB was empty on the next run of the app and the line:

	var syncContact = Contacts.FirstOrDefault(c => c.FbId == contact.FbId);

always equates to false on each subsequent run of the application.  Or am I missing some kind of commit statement?

Sadly calling "OnPropertyChanged" wouldn't help my scenario because I'm creating data to be stored in the DB and accessed later, all changes are programmatic and controlled so the UI has little involvement in that process (except via command later)

[Deyan]

Hi Simon,

I missed to mention that the SynchronizeAsync should be called to make sure DB items are loaded. I now realized that this also needs to be mentioned in the help so it will be part of the upcoming online help update.

If you don't call the OnPropertyChanged method in the setters of your ViewModel's properties, nothing will be uploaded to the cloud since our Telerik Cloud Services (codenamed Everlive) SDK won't detect the changes.

Here, the OnPropertyChanged event is not related to sending changes to the UI but rather to our Everlive SDK.

So can you please try calling SynchronzieAsync after resolving each SynchronizationContext instance and see if it resolves the DB case?


Regards,
Deyan
Telerik

[Simon]

Funny you should say that because I recently just added that to each of the update / insert methods but can't test till I get home.

Like most web connected frameworks, it doesn't work without direct internet access.  Our works authenticated proxy breaks just about everything except the web browser :D

[Deyan]

Hi Simon,

Thanks for writing back.

Just check it out and let me know how it behaves. Unfortunately I won't be able to respond during the weekend, so in case you need help, I suggest you simply take a look at the sample code that comes with the Telerik Cloud Synchronization App project template (which you have started with).

Just take a look how tasks are added, updated and synchronized.

Calling the SynchronizeAsync certainly needs a connection for the cloud-related part of the synchronization but when it comes to loading items from the local DB it should work without connection.

We appreciate your feedback. Since the current release is CTP, this will help us identify potential problems, improve the API, etc.

Thanks again for your time!

Regards,
Deyan
Telerik

[Simon]

Oh i've no problems putting the Beta through it's paces, always good to find ways to improve systems and if that's before it's the official release, all the better 

[Simon]

Right, I think I found the root cause of my issue (well two things actually)

1st - seems the local DB is locked while a synchronisation is taking place so searching cannot happen.  Reports the view is empty.
So either the initial synchronisation / load needs to be awaited so that searching cannot happen until it is loaded, or searching the db should acknowledge a sync is taking place and handle it. 
Also there needs to be a way to check the synchronisation status and an event to notify synchronisation has completed

2nd - One flaw I had which was to do with the everlive service is that it is not clear enough that you NEED to create your content first before you can synchronise a table with the back end.
This needs to be highlighted a lot more (I'll probably end up doing a blog post about the setup when I get a chance)

I'm nearly there but not quite yet.

Seems a better solution to calling SynchroniseAsync is to just call OnPropertyChanged manually in code as it has the same effect but only synchronises what has changed

[Simon]

For now one way I found to work around this is to have the following before any insert or update methods

            while (this.contactsContext.IsSynchronizing)
            {
                await System.Threading.Tasks.Task.Delay(500);
            }

That way the operation doesn't attempt until the data store is ready, not ideal but it works

[Deyan]

Hello Simon,

1. You simply need to await the SynchronizeAsync call. That's the way this API is meant to be used. Awaiting this call will make sure the execution will wait for the synchronization to finish and will continue after that.

2. Using the Telerik Cloud Synchronization App wizard makes sure to create the content type needed to run the sample. This is also explained in the help. It seems, however, that more details are needed on using the Everlive portal so we will make sure to extend the contents.

Thanks for your feedback.

Regards,
Deyan
Telerik

[Simon]

OK, well Point one really needs highlighting especially since the demo app doesn't implement this, the only interaction it mentions is when the MainViewModel is initiated:

        internal void InitContext()
        {
            SynchronizationContextPool.RegisterContextForType<Task>(new EverliveSyncServiceProvider<Task>(CloudProviderHelper.CurrentProvider.CurrentUser.Id + "_local.db"));
            this.tasksContext = SynchronizationContextPool.GetContextForType<Task>();
            this.tasksContext.SynchronizationFilter = task => task.CreatedBy == CloudProviderHelper.CurrentProvider.CurrentUser.Id;
            this.OnPropertyChanged("Tasks");
        }

Nowhere is "Awaiting SynchronisationAsync" mentioned and if I try inserting a record on startup then you would run into the problem I mentioned.
I'm guessing that since the interface is disabled until synchronisation has completed (and the data retrieved) this wouldn't highlight in the test app but I would be very concerned about the lag (especially with slow connections) should lots of data be stored in the client service.  The process ultimately should load the cached data first and then refresh from the server for performance reasons.

As to the second point, yes I believe the startup docs need to clearly highlight that you HAVE to setup the storage on the service manually with detailed steps for how to do it, in particular I did find the data types available on the server confusing but worked around it in the end.

[Deyan]

Hi Simon,

Thanks for writing back.

Indeed, some hints the fact that Content Types should be created explicitly on the server are needed in the help and we will be adding this. We just want to make sure that we do not cover topics that are already available in the help contents on the Everlive.com portal.

As for your other question:

We might consider adding a new method on the SynchronizationContext which will allow you to explicitly load the locally stored items into the view.

Normally, the async-await pattern is indicated by the async suffix that is put on the methods and should remind you of the possibility to await the method. Please note that awaiting a method does not block the UI thread, it simply schedules the execution of the upcoming methods after the first one has finished.

I have now browsed through your previous communication and was not quite sure what exactly do you mean with the following:

"1st - seems the local DB is locked while a synchronisation is taking place so searching cannot happen.  Reports the view is empty.
So either the initial synchronisation / load needs to be awaited so that searching cannot happen until it is loaded, or searching the db should acknowledge a sync is taking place and handle it. 
Also there needs to be a way to check the synchronisation status and an event to notify synchronisation has completed"

1. We are internally using SQLite to store the items locally. It surely impelments some kind of locking mechanism while operations are performed on the database but you are not supposed to directly access this database. Accessing the View property should not pose any problems. Can you please share some further details about the scenario in which you experience problems inserting items so that we can make sure we examine it on our side?

Thanks for your cooperation.

Regards,
Deyan
Telerik

[Simon]

Ok, to put it simply.

The tasks sample does NOT await any of the synchronisation work, it calls the async load method (without Async) and then when it has completed it returns the data to the view and it is displayed on the form.

However if I was to run a second query programatically (even awaited) while the above is taking place then the query would return 0 results.

From what I can gather, either:
* the SQLite DB only caches while the app is running so until the sync has completed the local db is empty
* the SQLite db is celared when a new initial / full sync event takes place.
* Queries are run against the local db only (which if either the above is true) which returns no results while a sync is taking place

So while the app is starting queries don't appear to work until the initial sync has taken place.

[Deyan]

Hi Simon,

Thanks for writing back.

Indeed, the code in the Tasks sample does not await the SynchronizeAsync call. That is because nothing that is data related happens after that call. We just have a JumpList control bound to the View which displays the items as soon as they're available.

So, if you have locally stored data (let's say you're run the app, synchronized and closed the app to run it again), the local DB is never empty. It has the items stored after the last synchronization. What could be empty is the collection exposed by the View property which does not expose the DB itself. It is an in-memory representation of the items available locally. It is loaded upon initial synchronization. That is why your queries do not return results until after the initial synchronization.

So, based on our communication until now I would like to summarize your concerns:

1. Performance hit when having to wait for the synchronization to end to be able to access the locally stored items.

Possible solution: you can use a background worker now to make sure synchronization is performed on a separate thread. We could also implement this internally.

2. Missing ability to separately execute given synchronization stages like loading locally stored items only.
Possible solution: extent the API to ensure more flexible synchronization options are available.

3. Events that notify you which synchronization stage is currently taking place.

4. Direct local DB access with support for multiple queries running simultaneously.

Let me know if I am missing something as this is important for the planning of the next release :-)

Also, feel free to correct any of my observations if you feel that I do not understand you correctly.

Additionally, tt will be great if you share the code you are using. In this way I will better understand the scenario.

Regards,
Deyan
Telerik

[Simon]

1: - should be able to access the local db (cache) as a primary with notifications when data updated from sync
2: - yes
3: events and statuses for synchronisation process and possible local db state (dirty / updating / clean / etc)
4: critical, also important in offline scenarios so I can access data when synchronisation is not possble

Also in offline, it must be possible to cache logon credentials and use the cached version (with lifetime checking) so I can access the db authenticated locally.

[Deyan]

Hello Simon,

Thanks for getting back to me so quickly.

We really appreciate your feedback.

I will put down these requests on the TODO list.

Currently, we will consider this thread closed.

Please, feel free to open a new thread in case you have questions, feedback or need assistance in using the components.

Thanks again for your time!

Regards,
Deyan
Telerik

[Deyan]

Hello Simon,

This is a quick follow up to inform you about the latest developments of the CloudDataSync component. In our Q2 2013 SP1 release we've introduced the following new features:

- Support for separately executing different synchronization operations as described here:
    http://www.telerik.com/help/windows-phone/radcontrols-cloud-synchronization-    synccontext.html#ExecutingSeparateSynchronizationActions
- A simple mechanism for Data Relations which is described here:
    http://www.telerik.com/help/windows-phone/radcontrols-cloud-synchronization-relations.html

We are currently working on IQueryable access to the local database as well (the first version will be available soon via an Internal Build).

Let us know what you think!

Regards,
Deyan
Telerik

[Simon]

Thanks, just updated my build using the control panel and will keep an eye out for the latest internal build.

Will let you know how I get on.

[Deyan]

Hello Simon,

Thanks!

We will appreciate your feedback on how your experiments are going. Let us know if you need further assistance.

We will be happy to take a look at what you have when you have a working version of it. This will help us polish the API and see how we can simplify the integration of the Cloud Data Sync component further.

Regards,
Deyan
Telerik