I know the question I’m posting is a bit broad, but unfortunately I have no idea of what to search for.
So, my real question is: what should have I to search for to understand how to store and read data that need to be processed to get reports about something?
I’m referring to applications like Google Adwords, SemRush, Facebook Analytics and, in general, any kind of application that collect a very big amount of data and then use it to build reports.
Currently I use MySQL but I’ve read many times that it isn’t the best solution to choose when approaching such kind of tasks.
So, which should be my starting point to understand which alternatives I have and, after, to choose one?
For the type of data you will be working with, you have two choices: a Data Warehouse or a Data Management Platform (DMP). I will give you a high level look at the two and hopefully this will help you start further research:
- A Data Warehouse allows you to pull in several data sources and build your own analyzations and visualizations on top of them. Cloud based products like Google BigQuery or Amazon Redshift make excellent data warehouse solutions because they can handle huge amounts of data without you having to maintain any hardware. BigQuery and Redshift have plugins that can integrate with products like Google Analytics and Facebook Insights to help load your data.
- Think of a DMP as a specialized data warehouse product that is developed to handle integrating customer and campaign data from websites and applications. While you can do some analyzations and visualizations with a DMP, its main role is to collect and segment the data that marketers will use to execute advertising and media buys. Salesforce DMP, Datalogix, and Oracle’s Blurkai are a few DMP product examples.
I recommend starting your search with Data Warehouses and DMPs, and then look into the different offerings in each area. As you look at products, evaluate who the audience will be (eg: Data Scientists vs Marketers) – then look at the differences in scalability, reliability, ease of use, company based and community based technical support, and of course price. I hope this helps.