Hi, I need to scrape a website at certain times a day, how can I parse the returned html to json and insert into datastore? I have been playing with feathersJS and there I used node-osmosis to parse the data.. Is there a way to use node-osmosis in kinvey?

Hi Johnny Doe,

It seems that the noted NodeJS library (node-osmosis) would parse HTML and thus easily extract parts of the HTML into JSON so that this data can be saved in Kinvey.

From your question I would assume that you are not only looking at storing the data in Kinvey, but also using Kinvey as a run-time environment to run your web scraper service on a regular (scheduled) basis. If that is the case, then you may use Kinvey Flexservice Runtime which is a NodeJS server-less run-time environment where you may execute a Kinvey Flex Service. In this service you may use the node-osmosis library to scrape the website and then save the data to Kinvey. To run this microservice on a regular basis you would use Kinvey Scheduled Code.

The Kinvey FlexService Runtime is available with our Business and Enterprise plans.

I hope this has answered your question.