A History of Data Collection, Storage, and Analysis

Data has tremendously changed over time—particularly the amount, collection, types, and analysis of it. Data has also progressed to what it is today by impacting everyone and every industry. The future of data and that of market research are also interrelated. As a result, it’s important to understand its history, especially as the next shift in data is happening now.

Early Forms of Data

The first forms of early data were in the form of tally or tick marks. These were collected in order to keep track or record inventories such as food for ancient civilizations. Later the abacus was invented to help with the calculations of such records. Then, other data related to astrological studies and time-keeping resulted in scientific discoveries. Ultimately, as more forms of data were discovered, the need for tools to collect, analyze, and store it also quickly resulted—even in the earliest of times of data history.

Data Evolving

As society advanced, more evolved needs for data resulted. For example, during the 1800’s the census started to take place—one of the largest sources of data yet. Because the census was gathering more and more data points, a need for a more efficient process for data collection and analysis was required. In fact, without it, compiling and analyzing the census data wouldn’t be possible to complete until close to the time the next census arose.

Herman Hollerith, an inventor of the times, recognized this need and developed what became known as the Hollerith Tabulating Machine—which tabulated data from the census and took analysis from years to months. This machine also allowed researchers to more easily extrapolate insights from the census to spend more time understanding the impact of the data. And while statistics has its place before the 1800’s, it was then applied more broadly as a result of this machine. Herman would later become known for founding IBM.

Fast forward a bit into the mid-1900’s and the invention of the computer, the crisis became data storage. Luckily, around this time, semiconductor memory chips were developed to replace traditional vacuum tubes (or RAM storage) used on the Hollerith Tabulating Machine and similar tools. But with the commercialization of computers, even more storage was needed to keep pace with the influx of data sources. For example, banks and other enterprise institutions started to see more and more value in the impact data tracking and analysis could have on their bottom line. Howard Dresner, a well- known leader and analyst at Gartner, put a now commonly known name to this phenomenon: business intelligence. So by the 1980’s magnetic disk storages—or floppy disks—were developed and became mainstream.

But the revolution of data had yet to reach its peak. We bet you can guess what changed the world of data as we know it by the 1990’s—the internet. More data, more data types, and new ways for collecting, using, and analyzing it were all a result of the internet. But as we are all likely very familiar with that story, we can skip ahead to what really matters—the future of data.

The Future of Data Is Already Here

We argue that the future of data is already here, largely due to big data. Big data was actually first coined in the 1990’s, but today, it’s really just data—and lots of it. Big data has also resulted in the data science industry. An industry formed to develop better tools and means for integrating different types of data. Now, the future of data is here—in the form of cognitive technologies. These technologies have already become products that leverage AI to perform more complex tasks through machine learning and natural language processing.

But it’s tough to predict the future of data and technology; the possibilities are truly limitless. In the past, some have been very successful at predicting it, like Nikola Tesla, who, in 1926 stated:

“When wireless is perfectly applied the whole earth will be converted into a huge brain, which in fact it is, all things being particles of a real and rhythmic whole. We shall be able to communicate with one another instantly, irrespective of distance. Not only this, but through television and telephony we shall see and hear one another as perfectly as though we were face to face, despite intervening distances of thousands of miles; and the instruments through which we shall be able to do this will be amazingly simple compared with our present telephone. A man will be able to carry one in his vest pocket.”

Shocking that 70 years ago Nikola could have predicted such an accurate depiction of the future. In the end, it’s always about data collection and analysis—we need faster and cheaper ways of analyzing it and storing it. While technology is evolving to provide more means of storage and greater amounts of it, the number of tools available for interpreting data can be overwhelming. But if you want to know more and take the first step, learn more about how big data and survey data can be combined to provide unique audience intelligence by checking out the webinar below.