The Evolution of Big Data
The term ‘data science’ has recently emerged to refer specifically to the profession that has recently merged statistics with computer science to help make sense of the vast pool of big data. In fact, this transition has been so profound that Harvard Business Review defines two eras in the use of analytics – before big data and after big data, or, perhaps more appropriately, Analytics 1.0 and 2.0, with another impending shift that they call Analytics 3.0.
The Wall Street Journal defines Analytics 1.0 as the initial era from about 1954 to 2009, which was characterized by:
- Relatively small and structured data sources, which came from internal sources
- Storing data in enterprise warehouses before analysis
- The great majority of analytical activity was descriptive analytics, or reporting
- Creating analytical models was a “batch” process often requiring several months
- Quantitative analysts were segregated from business people and decisions in “back rooms”
- Very few organizations “competed on analytics” — for most, analytics were marginal to their strategy
The Emergence of Big Data
2010 saw the world take notice of “big data,” particularly as internet giants like Facebook and Google began to collect and analyze their user data. Where small data was sourced from a company’s internal transactions, big data was large, unstructured, and sourced externally from the internet. The constant stream of data meant that it had to be captured and processed quickly, and the overall speed of analysis became much faster. Descriptive analytics were derived from the indexing and crawling of archival data, and they still crowded out the more predictive and prescriptive techniques that were to come with Analytics 3.0. A new generation of quantitative analysts, or “data scientists,” was born and big data and analytics began to form the basis for customer-facing products and processes.
As businesses currently evolve into Analytics 3.0, the Wall Street Journal identified a number of traits that are already apparent. Among them are:
- Businesses are combining data from various sources and in various formats to yield new insights in predictive and prescriptive models.
- Analytics are driving both internal decisions and data-based products and services for customers.
- Faster technologies are being coupled with agile analytical methods and machine learning techniques that produce insights at a much faster rate.
- Data scientists, who excel at extracting and structuring data, are working with conventional quantitative analysts who excel at modeling it and the combined teams are doing whatever is necessary to get the analytical job done.
- Companies are beginning to create “Chief Analytics Officer” roles or equivalent titles to oversee the building of analytical capabilities.
As big data continues to evolve, the University of Alabama at Birmingham’s Collat School of Businessis developing the data scientists of tomorrow through its Bachelor’s in Information Systems degree. It arms students with the technical skills needed to collect and analyze data, as well as the business skills needed to translate it into the intelligence that will give you and your business a competitive advantage.