Schemaless Data and Dynamic Schema

[R]

I am currently attempting to use the Kendo Grid to display data that is returned from my Node/Express/Mongoose/MongoDB stack.
Being that MongoDB is schemaless, I have the scenario where my collection will have documents that may or may not contain the same elements.

When trying to populate a grid with a returned JSON query similar to the following:

[
{FirstName: "TestA", LastName:"TestB", Age: 50},
{FirstName: "TestC", Age:50}
]

Using a grid datasource schema that is dynamically generated similar to:
 schema: {
                model: {
                    fields: {
                        FirstName: { type: "string" },
                        LastName: { type: "string"},
                        Age: { type: "number" }
                    }
                }
            }

Trying to display three columns: 
                    [
                           {
                                field: "FirstName",
                                title: "First Name"
                            }, {
                                field: "LastName",
                                title: "Last Name"
                            }, {
                                field: "Age",
                                title: "Age"
                            }
                    ]

I get an error when binding the data. In Firefox and IE it says "LastName is not defined". In Chrome I do not get an error the grid draws the text "[object HTMLCollection]" where you'd expect the field to be null or blank.

Is the grid intended to handle the scenario where an element may not be present in the returned JSON?
If not, are there an work arounds to this?

Thanks in Advance,
Rene

[R]

Anybody!? *fingers crossed*

[Andreas]

Hi R,

we have the same problem, as our DB is Mongo. So there is no schema at all. 

The KendoUI Grid dataSource can't handle this scenario as far as i know, so actually the json must contain all columns of the grid. W But we found a workaround. We changed the source code of the Grid :-)

The KendoUI Grid has many functions, one of them is called _rowsHTML.

Here is our version of that function:

_rowsHtml: function(data) {

            var that = this,

                html = "",

                idx,

                length,

                rowTemplate = that.rowTemplate,

                altRowTemplate = that.altRowTemplate;

 

            // new variables

            var cols = that.columns,

                columnLength = cols.length,

                i,

                record,

                property;

 

            for (idx = 0, length = data.length; idx < length; idx++) {

 

                // get the data record

                record = data[idx];

                 

                // loop through all grid columns

                for(i = 0; i < columnLength; i++) {

                    // fieldName of the grid column

                    property = cols[i].field;

 

                    // add the property to the record, if it not exits

                    if (record[property] === undefined){

                        record[property] = null;

                    }

                }

 

                if (idx % 2) {

                    html += altRowTemplate(record);

                } else {

                    html += rowTemplate(record);

                }

 

                that._data.push(record);

            }

 

            return html;

        }

I hope this helps you. Also it would be nice if someone from the KendoUI team comment this.

[R]

Andreas,

Thanks for the information. We did manage to come up with a similar work around. I did test out the schema-less approach using a different third party grid and it handled it with ease. Normally we would work through the source but unfortunately this is purely a proof of concept project and we have not made a purchase yet.

I appreciate the time you took to post. Atleast I know i'm not alone! haha

Thanks!

[Jonathan]

Telerik admin, do you have any suggestions?

I would like to dynamically set dataSource.options.schema.model.fields

I tried to do this in the data source parse method, but it did not work.

[Iliana Dyankova]

Hello Jonathan,

I am afraid changing this configuration on the fly is not supported out of the box - the data source should be recreated. Could you please provide more details about the exact scenario you are trying to implement? This way we would be able to advice you further and suggest a suitable solution if such is possible. 
 
Kind regards,
Iliana Nikolova
the Telerik team

[Jonathan]

An API call returns to me a dynamic set of data. The first row describes any information necessary for column generation/datatypes.

Right now I am loading the data through a standard ajax call, massaging the data, then setting up the datasource.

I guess the best thing to do would be to destroy and recreate the grid everytime, because the columns might change?

[Iliana Dyankova]

Hello Jonathan,

I am afraid that the dynamic change of grid columns is not supported at present - the grid should be recreated every time. 

Greetings,
Iliana Nikolova
the Telerik team

[joshua]

I made my own grid with knockout and dynamic columns using an observable array.... This has been elusive from any of the major data sets grids. slickgrid can do it but i bet its killing the grid behind the seen's. 

The problem is in how TD driven grids are built. 

My question is would it be easy to bind an observable to the column section, or do you only build the columns once like KOgrid which would not work for the same reasons.  

I have built a custom json pivot class(soon to be on github ;)) and im assuming the user above has done the same thing. It requires that the grid be built dynamically. If the grid built columns dynamically you could in theory gain efficiency just like the rows. Only loading the columns into memory as the user scrolls  and adding in new ones.

What im looking for is an evented grid control..... I think i will have to build one myself :(

The idea is that the grid is not a series of rows and columns but a set of individual cells that relate to the cells around tehm, but are build in an evented model.  This being that cells in any direction can be built changed grow or modified as the data changes and are not directly linked to the rigid row column arch... tThink of it a grid built like hadoop stores memory by relations not by forced structure... man it would be fast.  

[Brian Vallelunga]

Has there been any improvements on this? I would also like to dynamically define my data source's model based on the JSON that is returned by the data source's call itself.

I think the solution would be quite simple. Just as we can set the schema's data to be a function that takes the response and returns the data, allow us to do the same with the model. If it's set to a function, we can then return the model, parsed from the server's response.

Something like this is what I'm envisioning:

var dataSource = new kendo.data.DataSource({

    // other options set here

    schema: {

        model: function(response) { return response.dataModel; }

    }

{);

I'm not sure how else to accomplish this without making two calls to the server, one for the results metadata and one for the data itself, recreating the data source every time. Even then, I'm not sure how I would integrate in the data source with a grid's paging/sorting.

[Josh G]

I have been able to change the model of an existing DataSource by calling the define() on reader.model.  

var kendoTransport = {

        read: {

            url: window.application.baseurl + "api/data/Get" + healthVaultClassName + "Models?startDate=" + startDate + "&endDate=" + endDate + "",

            dataType: "json",

            type: "GET"

        }

    };

    var kendoDataSource = new kendo.data.DataSource({

        transport: kendoTransport,

        schema: {

            model: {}

        }

    });

 

// ... Later a partial view is loaded depending on the health vault class. The model is thusly modified.

 

    kendoDataSource.reader.model.define({

        id: "BloodGlucose",

        fields: {

            Key: { type: "string" },

            EffectiveDate: { type: "date" },

            GlucoseMeasurementType: { type: "string" },

            IsControlTest: { type: "boolean" },

            MeasurementContext: { type: "string" },

            ReadingNormalcy: { type: "string" },

            State: { type: "string" },

            When: { type: "date" },

            OutsideOperatingTemperature: { type: "string" },

            CommonData: { type: "string" },

            Value: { type: "number" },

            MgdL: { type: "number" },

            MmolL: { type: "number" }

        }

    });

I don't know if this answers your question, but hopefully it will help.

[genji]

R, you mentioned you used a third party grid that could handle schema-less approach, which 3rd party grid was that? I love Telerik but we are doing so much more mongodb these days and Telerik seems to not do much QA or thinking towards how their controls work in a mongo world of dynamic data.

You should be able to set the data of a grid to a json array like this:

[
    {
        "DateTime": "2017-05-16T23:32:52.6003189Z",
        "Name": "machine1",
        "TheWord": "Hello World"
    },
    {
        "DateTime": "2017-05-16T23:32:52.6003189Z",
        "Name": "machine1",
        "TheWord": "Hello World",
        "FailsToDisplayThis": "Test1"
    }
]

 

And it should just work but instead it will not display the column FailsToDisplayThis. 

[Stefan]

Hello Marc,

The provided data will be displayed correctly by the Kendo UI Grid without a schema if the columns property of the Grid is set:

https://dojo.telerik.com/UbuHE

Also, the columns can be changed dynamically using the setOptions method of the Grid:

http://docs.telerik.com/kendo-ui/knowledge-base/change-a-widget-options-dinamically

Regards,
Stefan
Progress Telerik

[genji]

I will try this; looks like it should solve the problem. Given a lot more usage of things like Mongodb are you guys looking at potentially doing things like in the example I gave if the second json object has a column you have not seen then just dynamically add the column since you are already looping through all of the data? This would alleviate the need for having to dynamically update/calculate all columns. I realize either way would solve the problem but it would be nice of the controls just worked by default with such data. thank you 

[Stefan]

Hello Marc,

Thank you for the suggestion, MongoDB is indeed gaining more and more popularity in the last few years.

I can recommend submitting a feature request in our feedback portal for supporting the described data format. Then based on its popularity among the Kendo UI community, we may implement it as a built-in feature in a future release:

http://kendoui-feedback.telerik.com/forums/127393-kendo-ui-feedback/category/63716-data-source

Regards,
Stefan
Progress Telerik