Hi,



I'm trying to setup offline fetching against collections that have relations and am seeing getting some incomplete responses on the relational fields.



I am testing by fetching a collection with relations first with `offline:false`, then performing the same fetch with `offline:true`.



Observations:



-- The 1st (online) fetch returns all data and relations (two) as expected



-- On the 2nd fetch, some of the documents do not have either relational field fully populated. All that is available in the relational field is _collection, _id and _type



-- The same record fails each time.



-- Occasionally there is a second record failing, but it doesn't happen every time.



-- All other records 13 (or 14) always return the data



-- I've double checked the data on the always failing record, and can't find any issues. And if there were, I'm assuming that it would cause the relation to fail regardless of offline: true/false?



-- I have a about 6 collections that have 1 or more relational fields, and all have at least 1 record not getting the relations populated when offline



I've gone through several iterations, but my latest test is something like this:



~~~

KINVEY_DEBUG = true;



Kinvey.init({

appKey: xxx,

appSecret: xxx,

sync: {

enable: true,

online: Ti.Network.getOnline() //have also tried to just set to true

}

})

.then(function(activeUser){

Kinvey.Sync.destruct(); //clear the existing cache for the test

});



//query from network the first time

var offLine = false;



function doFetch(){

Kinvey.DataStore.find('collection', null, {

offline: offLine,

relations: {

oneRelation: 'oneCollection',

twoRelation: 'twoCollection'

})

.then(function(response){

//toggle the flag for the next query

offLine = !offLine;



//do something with data

});

}



//call with a button click or something

doFetch();

~~~

Since you have debug mode on, what is exactly being logged? That will help me determine whether the local database even bothers to look up the relations in the first place.

On the offline query, it starts with



~~~

[INFO] : (

[INFO] : "Retrieving documents by query.",

[INFO] : {

[INFO] : }

[INFO] : )

[INFO] : (

[INFO] : "Using local persistence."

[INFO] : )

[INFO] : (

[INFO] : "Initiating a read request.",

[INFO] : {

[INFO] : }

[INFO] : )

[INFO] : (

[INFO] : "Executing a query.",

[INFO] : "SELECT value FROM 'recipes'",

[INFO] : (

[INFO] : )

[INFO] : )

[INFO] : (

[INFO] : "Executed the query.",

[INFO] : {

[INFO] : result = (

~~~



Then it lists all of the documents returned. There is a lot, I can email them if you like.



Next, is says "Retrieved the documents by query.", and appears to list all the documents again.



After that, it says "Retrieved relations for the document." along with each document individually.

You should see three separate `SELECT ... FROM` entries - one for the top-level document, and two for the relations. If that's the case, it tries to look up the related entities.

I only have a single `SELECT` entry. Note this is during a `find` to retrieve all documents.



If I do a `get` to retrieve a single document, I DO see three `SELECT` statements. Additionally, the `get` is retrieving all the relations for all documents that I have tested, even the document that does not get the relational fields during the `find` above.

> @adampax said:

> Then it lists all of the documents returned. There is a lot, I can email them if you like.

>

> Next, is says "Retrieved the documents by query.", and appears to list all the documents again.

>

> After that, it says "Retrieved relations for the document." along with each document individually.

>



Actually I missed one of the log entries. After executing the query and before 'retrieved' there are log entries for "Retrieving relations for a document." First, there is a 'Retrieving' entry with all of the documents in the collection listed. At the end are the options for fallback, offline, refresh, relations, and a success callback.



Next, each of the documents is listed after a 'Retrieving' entry. This is where I noticed that only the first document has the relations option listed. The rest of the 'Retrieving' entries do not have relations or success in them.



This is all happening around line 5095 of the Titanium 1.1.8 library (I would link to it but can't find an up to date master copy on github.com/kinvey).



`KinveyReference.get(document, options)` is called first with the entire collection, then the `document` array is iterated and `get` is called again on each document in the collection. Below this, on line 5115 `delete options.relations` is called, removing the relations property from the options argument that is still being used while iterating through the individual documents.



I can comment out `delete options.relations` and all the relations are now being resolved in my test, though as it states in the code, these options are being temporarily reset for other purposes, so I'm sure there is a different fix needed.



Even with this now working, there are still some items I don't understand.



-- Most of the documents already have their relations resolved in the 'Executed the query log', which comes before `Retrieving relations`. Why is that?



-- Looking in the SQLITE table in the app, I see that the relations are added to the documents here (which may explain my previous question). That being the case, why do the relations need to be retrieved?



-- The relations table only has one record in it -- the same record that is missing on the relation in the master collection. Shouldn't all the related records be here? Is this related to your previous question of how many SQL queries I'm seeing on the after the `DataStore.find` query?

Interesting. I think a different fix is replacing line 5097 (inside the `document.map` callback) with:

```

var clone = function(obj) {

var result = { };

for(var key in obj) {

if(obj.hasOwnProperty(key)) {

result[key] = obj[key];

}

}

return result;

};

return KinveyReference.get(member, clone(options));

```



Could you try and verify whether this works?



To answer your other questions, the resolved relations may be visible in the log as console.log is buggy when it comes to displaying objects (it takes the latest state, instead of the one present at the time of logging). In SQLite, the document should be stored without relations, and the relations reside as separate documents in the corresponding table.

Hi, yes cloning the options argument is working here. Are there other areas in the library where recursion is being performed and arguments are potentially being overwritten?



I am definitely seeing documents being stored with relations data in the SQLITE table.



EDIT: I am seeing relational data in _some_ of the documents, not all, which makes me wonder if there are still issues with recursion or something similar elsewhere in the library.

Yep, same problem with `KinveyReference.save` deleting parameters from the `options` argument around line 5220 or so.



Have to clone the options argument here as well.

Thanks for reporting - I will fix this in the library.

Hi @Mark - just wondering if you have any updates on this and where it might fit into the roadmap? As I am sure you know, it impacts the base HTML5 JS library as well. We've had the above workaround in place (commenting out "delete options.relations") for quite awhile now on all our offline-supported apps, but also have always believed that workaround is breaking other things and making relations caching a bit unstable - especially nested relations.



Anything we can do to help (test cases, etc) don't hesitate to let us know. Thanks.

It should be noted that the workaround mentioned here does not work for nested relations, [more info here](https://support.kinvey.com/discussion/201272404/deeply-nested-offline-relations-with-titanium#latest).