Big Data

Big data is a buzz topic. I normally don't find buzz topics very interesting, but this one has intrigued me.

Phil's quick definition: Big data is what Google does.

You want to find something on the web, but the web is flipping huge. Google boils down that large information to something that you are looking for.

Here are some things to know about big data that might be helpful to know. 

1. It can't be done on your home computer. As incredible as our personal computers are, they are too slow for big data.
2. It is expensive for the equipment. The company I work for just started a big data project that is going to require 20 different servers to essentially run one analysis program. To give you a visual, that's like a computer the size of a refrigerator that costs $50,000.
3. The research projects that Universities have done with big data have started to draw attention of companies that want to spend money on big data projects. In other words, the industry is growing, and very few people are experts in the field (I think of my very smart brother who is about to graduate high school).

You may have seen studies that caught your attention from big data already. One study looked at twitter to gauge how people are feeling around the world at any given moment. More recently, a university did a study that included more data than just twitter. They included blogs, facebook, any social media that they can find to draw a correlation between how people felt on a given day, and how the stock market reacted for the next 3 days.

Other studies get more specific to a company. They might be trying to answer a question like, "After a commercial in the Superbowl, how did people feel about your company?" Theoretically, someone who pays $5 million for a commercial, could watch the entire country's reaction to its commercial in real time. They may even be able to predict sales for the next week more accurately depending on the reaction a commercial had.

My examples surround marketing as you can tell, but there are more practical uses for this too. When networks go down at gigantic companies, it causes problems. A website goes down, email is unavailable, all kinds of problems can occur. The company I work for is using big data to solve those problems. It analyzes an entire network of computers, with enormous amounts of data (logs from servers, reports from load balancers, and many other tools) to hone in on the problems of a huge network

That's all. I'm just interested, and thought you might not have heard about it.