Recommended bulk save : Progress Kinvey

T

Tzu-Hao Huang

started a topic almost 5 years ago

Hi, any idea the best and safest way to save several objects to the same collection?

Assume I have an array of cards (say 600 different entities) to save to Card collection

Should I just use a for loop to save?

for card in cards {
    cardStore.save(card)
}

Right now I'm currently using PromiseKit "when(fulfilled: generator, concurrently: concurrent action)" to save one by one.

The reason with concurrent 1 action is that I noticed saving action sometimes will crash with the RLMException "Can only add, remove, or create objects in a Realm in a write transaction - call beginWriteTransaction on an RLMRealm instance first." Especially with multiple saving action at once.

Even more with my current way of saving, for some users it crashed with the reason like the following

https://github.com/realm/realm-cocoa/issues/4831

I know that when realm is saving, it will create extra space on the database and I also know that in Kinvey SDK 3.20.0 Kinvey added a feature called "compactCacheOnLaunch" to get rid of those extra space on database. But is there a way to make cache compact other than launch time?

In summary, is there a safe way to save multiple objects to avoid

crash due to "Can only add, remove, or create objects in a Realm in a write transaction - call beginWriteTransaction on an RLMRealm instance first."
crash due to "Cannot allocate memory size for realm"

P

Pranav J

said almost 5 years ago

Hello,

Thanks for explaining your use-case in detail. I reviewed all the information provided by you. Unfortunately, In a single request, you can save/update only one entity at a time. For 'Bulk Upload'/'Bulk Saving', see the CSV/JSON import feature on the Kinvey console (navigate to the collection, click on 'Settings' option, then click 'Import Data'). The bulk update is not supported. You can fetch and delete multiple entities in a single request using a query.

Please take a look at this link for better understanding. Also, please open support tickets since you are a paying customer.

Thanks,
Pranav

Bilger Yahov

said almost 5 years ago

Hi Tzu-Hao Huang,

Thanks for contacting us regarding your concern. Have you had a chance to read my colleague - Pranav's suggestion?

We can give you another suggestion as well. Kinvey offers a service called Business Logic. Kinvey Business Logic is a NodeJS code execution runtime that allows you to customize the behavior of requests to your backend by providing means to implement server-side business logic. Kinvey Business Logic enables developers to focus on the unique functional business logic of a mobile application, while Kinvey handles the work of receiving, parsing, and routing requests, connecting to and querying data stores, and enabling mobile-specific functionality. Business logic can be invoked as part of a request to a Kinvey collection, as a custom endpoint to execute an arbitrary function, as common code created and shared among BL functions, and on a scheduled basis.

Inside a Business Logic script you can make use of predefined modules. There's a module called collectionAccess, which can be helpful for your case. Using the specified module you can directly access your data in MongoDB. Thus, you can insert a single document or an array of documents into the collection with a single call. For your specific case that would be inserting the array of cards.

I hope this answer shows you the right direction to go to.

Should you need further assistance with that, please feel free to open a support ticket.

Good luck!

Cheers,

Bilger Yahov,

Kinvey Technical Support Engineer.

T

Tzu-Hao Huang

said almost 5 years ago

Hi, the information being saved is users' study data, which is something generated by users and we can't really manually import through JSON import.

And we can't use Business Logic as well since this save action might happen offline. We use generic Kinvey's Datastore.save() and let SDK handle saving to cache and syncing with backend.

So I guess using PromiseKit to save the data concurrently is the way to go.

BTW, any idea why sometimes it's causing crash due to "Can only add, remove, or create objects in a Realm in a write transaction - call beginWriteTransaction on an RLMRealm instance first."? Since Kinvey's SDK handles the interaction with realm. I don't even know where to get RLMRealm instance to begin write transaction.

Lastly, is it possible in the future Kinvey can provide safe batch save to the cache and collection?

Thanks

Redirect Notice

How can we help you today?