samedi 9 juillet 2016

Approach for fetching data and analyse it in python

I am doing one project in which I have to analyse the huge data which is on tornadoes happened in USA till now. As python is good for data analysis, I opted it. But I have some questions for which I need some clarification: 1. I am using pandas for data analysis. What I did till now is I have created one big dataframe(of 1GB .csv file) which contains all .csv files. Now lets suppose I want to calculate total deaths happened in year 2000. So I wrote query for that. Query is fetching results but it is taking some time. Is it good to store all the data in dataframe and fetch data? Or is there any other faster approach. 2. Another approach create json string of entire file and query that json string. I haven't done anything in this approach. Is is a good one?

Thank You.

Aucun commentaire:

Enregistrer un commentaire