Start a new topic

Fetching Changed Data Best Practices

There seems to be 2 ways for clients to get changed data.



1) Business logic.

Client asks what's new. Business logic compiles from various tables and returns the results.

This seems to be running into some timeout issues as talked about here: http://support.kinvey.com/hc/communities/public/questions/200670433-Business-Logic-Execution-Time-Limit and

http://support.kinvey.com/hc/communities/public/questions/200567147-Business-Logic-Execution-Time-Limit



2) Client does heavy lifting.

Client queries a change table for what's new.

Client queries each of the tables in turn for the changed items.



Are either of these methods better resource wise on the Kinvey side?



If the client does the heavy lifting, do we have any idea how this affect mobile battery life?

What's the use case for this? How are your collections set up and how many collections do you need to query?
It's an app on mobile and native desktop that needs to sync data. There's about half a dozen fairly simple collections plus a change log collection.



The change log could be queried by the client or by business logic to get the UIDs of any changed items and their tables.



Happy to give more info.
How often are you querying for what's new?
I'd like fairly responsive data so it's currently planned at about every 10s while in the foreground. Less in the background.
Are you seeing timeouts when you attempt to run these queries? Querying by ID should be fairly quick..
We're still getting BL timeouts on a small data set by querying in parallel by date. Happens about 1% of the time randomly. Feels like it's a transport timeout rather than a logic timeout.



We decided not to query a change collection for changed UIDs and then query the actual collection. This is because if there are thousands of new UIDs, that query could be more expensive than just querying the collection by date. And potentially we could exceed the size limit that a query can be constructed on. (Give me: UID1 AND UID2.... AND UID10000, etc.)



Given the potential unreliability of this and the danger that a large query might always exceed 2s thus locking a client out, we're looking at dropping business logic and going with the client doing the heavy lifting.
Can you provide to me which particular timeout error you are seeing?
Michael,



The exact error we are seeing is:



{"error":"BLTimeoutError","description":"The Business Logic script did not complete. Please contact support","debug":"The execution of a background task timed out."}



Thanks. We've recently made some changes, what is the frequency that you are seeing this?
It used to happen about 20% of the time. Recently it's been about 1% of the time without anything really different on those queries.
Login or Signup to post a comment