CONVERSATIONS # 6

22 195 000 BOOKS AND COUNTING

The dangers of a data driven world:

What are we talking about here? We are seeing the dawn of a new age, an age where huge, gigantic datasets are analyzed seeking hidden correlations between seemingly unrelated things. This analysis is then voiced by the mathematicians in the form of some kind of prediction. The statistical analysis are churned from datasets that are so big, so stupendously  large, that one cannot contemplate them in real terms. The data store of Wall Mart for example, contains almost 600 terabytes of data.

How much is that?

The entire Library of Congress houses roughly 22 195 000 books. To store this digitally, will need roughly 20 terabytes. In contrast, Wall Mart’s data stores contain 600 terabytes, enough to store the equivalent of 665 850 000 books, or 30 Libraries of Congress. (Ayres, 11).

These vast stores of data, coupled with the growing ability to store it easily, cheaply and with great data integrity, and the huge strides, the giant leaps in technology that transformed the world the last 5 years, enabling almost every individual to possess computing power undreamed of just a decade ago, has opened this door which we are just about to enter.

It was Adolf Hitler who said that war was very much like opening a door into a dark room. You had no idea what waited for you on the inside once you enter. That’s where we are today. We are standing in front of this door, without any real inclination of what might be waiting inside for us.

The first thing one must understand when contemplating this development, is that we are indeed discussing the death, the funeral of human intuition. No longer will there be a flicker of understanding in the eyes of the people when they are confronted with decisions from government and corporate “machines”. Gone are the days when humans could intuitively predict the outcome of things. The new world order will differ. Decisions will be coming from “black box” functions, from these datasets mined for invisible and hidden correlations creating completely unexpected and new outcomes.

As such, we will enter a world that will be less comfortable, and on an interpersonal level, ironically, less predictable. In other words, a world filled with more stressors, where we will be in much less control of our lives and environment.

Even though it may look and sound trivial, we as human beings have developed a fine tuned social machine that consist of millions and millions of small interpersonal and relational nuances that makes the world go round. Small mannerisms make life cognitively clear and present, and create a safe environment for us. We call this culture,

With this dawn of quantitive analysis, the expected is thrown out unexpectedly to introduce the unexpected to the expectant crowd. No longer will social graces be the saving grace at the bus stop or station. No longer will the tried and tested rules of engagement be a safe haven for the parties when you want to rent a car of fly on an airline. Things have changed.

If for example your flight is cancelled, the airline will no longer reward their loyal clients by making the very first seats available to them. On the contrary. Now, by using data mining methods, the airline will identify their most disloyal clients, those they are most at risk of losing as future clients, and the will get preferential service over and above the loyal clients.

No matter how strong the arguments are for this approach; no matter how strong the arguments for the autonomy of the business and the freedom to contract as they please or where they find their best advantage, I abhor this. I abhor the hidden, the conceited way in which decisions are now made, I abhor the lack of decorum, the lack of protocol, and the replacement thereof by cash hungry money chasing little neurons hell-bent on maximizing todays income irrespective of the lack of culture and grace of it.

I also abhor that which I cannot trust, that which lack the basic integrity expected of the humans around us. In a world where corporations are free and actually rewarded for being dishonest and conceited with their clients, we enter relationships based on false premises, based on hidden premises built upon hidden correlations worked out by hidden little spiders that churn away our secrets while we sleep. The basis of this new order, is that it’s built on complete dishonesty, on utter non-disclosure, and on a much more predatory relationship between humans. This does not bode good for mankind.

Related articles

• What is The World’s Largest Private Employer? (socyberty.com)
• DaVinci Institute unveils eight competitions for mankind (zdnet.com)
• Featured Dataset: Foodista ” kasabi (kasabi.com)
• Big data and open source unlock genetic secrets (radar.oreilly.com)
• SDM 2011: Day 2 (geomblog.blogspot.com)
• Crunching Big Data: more than a byte-sized bet (canada.com)