Etiquetas » Bigdata

How to load huge CSV datasets in Python Pandas

Some tips and tricks to load a big dataset using the very famous Python library.

Image by xresch from Pixabay

It may happen that you have a huge CSV dataset which occupies 4 or 5 GBytes (or even more) in your hard disk and you want to process it with Python… 640 palabras más


The Modern Rules Of Data Festival

After the storm

The Moat is all messy from the rainstorm, the flood washed in a lot of stuff, suffice to say it’s not clean. It is going to take a lot to clean this up in preparation for the final day of the Īgue festival. 326 palabras más


Migrating billion+ documents in Elastic Search to AWS S3 !!!

Enterprises today need their data monetized, if not, leverage on the insights from the enormous amount of data to generate newer incomes from newer products. Building infrastructure, platforms to gather, clean, prepare, process, standardize and making the data (big data) available for heterogeneous stake holders amidst the ecosystem that is vastly huge, constantly changing, is a daunting task. 1.023 palabras más


Moving data from oracle to HDFS

I divided the scope into two stages:

Stage -1 | Initial data load from Oracle database to HDFS (for bulk load)

  • Here I used oracle native utility (Copy2hadoop) for reducing the friction…
  • 533 palabras más


These days applications have become more data intensive than compute intensive. People face problems around complex, fast growing and changing data and steps to manage it. 1.290 palabras más


How big is big data?

  1. What is big data?
  2. Real life examples.
  3. Top facts about big data you must know.
  4. Jobs available in big data field.
  5. Courses to excel in big data.
755 palabras más