Data mining is a term that people who even people who are not involved in the industry or in marketing or advertising are familiar with. That's because part of data mining happens over the internet and regular internet consumers have heard the term applied to their own online activities. In this article, we are going to thoroughly research the term data mining and everything that goes into it.
What Is Data Mining?
The first question that we have to answer is ‘what is data mining’. The technical term is somewhat complicated, but to break it down into something relatively simple, data mining is when a computer collects data specifically for the purpose of discovering patterns or doing statistical analysis. However, the process uses artificial intelligence to actually do the data mining. So, the data that is collected isn't just a free-for-all, where any data that exists is gathered up. Instead, there is intelligent method used to find data patterns and extract the information.
In fact, data analysis might be a better word for this than data mining, since the term actually refers to the process of finding those patterns rather than to the collection of the data itself. Data collection is something less complex than easier to understand. To demonstrate it further, imagine that you are collecting email addresses. If you collected every email address that you could find online, such as those that were posted on profiles, forums or in website contact information, you would be doing data collection.
But if you were to only collect emails from a certain group of people, or emails that came from a source that you knew would contain a certain group of people, then it might be a lot closer to data mining. So, if the email addresses that were collected we're all from women between the age ages of 23 to 33, that would be an example of data mining. You would have to use very intelligent methods in order to find those email addresses and ensure that they were qualified for you to collect them.
How Does Data Mining Work?
In order to understand the term better we should discuss how the process works. Let's use your local grocery store as an example. You have likely noticed that you have to have a savings card in order to shop in the United States these days. If you do not have the card, then you will have to pay a higher price than those who do.
The goal of these shopping cards is not to save you money, and in fact, they are a major inconvenience to customers that forget them, but instead are tools to help the grocery store understand your buying habits. The grocery store figures out what you are buying and when as well as a bunch of other data and uses that to improve their marketing and advertising, in-store displays or various other aspects of their business.
How is Data Mined?
Computer software is the labor behind the mining of data. There are a number of commercial programs on the market but many companies make their own data mining software that fits their specific purposes. You have to do this in many cases because each businesses needs are unique and what one software engineer might think a business needs could be something completely different from what they actually require. So the data that they need must specifically be identified by the software, to be mined in the way that they need it to.
For example, suppose that a gas station chain wants to find out when the best time is to offer a special on a fill-up or items within the store. The information gathered has to be classified so that it can be organized properly. Data mining helps to create that class of information that shows what people buy, when they buy it and all kinds of things about their visit to those gas stations.
The Process of Data Mining
Let's look at the five steps of data mining so that you can understand better how the process works and how it will work for your own company. The first step is going to be a two-part step. The first step is actually gathering the data. But in order to gather that data you have to have somewhere to put it. This means setting up a data warehouse. We are going to discuss the data warehouse in more detail below, but for now, just understand that it is a place where data is stored.
The second step is the storage and management of data. This data can be stored and various places like servers within the company or located somewhere on the cloud where people can access it from anywhere.
Third, the company's management decides how the data is going to be organized. This usually requires figuring out what the businesses goals are and how they would like to use that data to improve certain operations. There may be many uses for data that is pulled from the data warehouse and it will be divided up accordingly.
The fourth step is to use an application to sort the data based upon what the user wants. Many companies build their own software applications for this, since a business’s needs maybe unique and something off the market may not work for what they are looking for. The final step is to present the data to users in a format that is both easy to understand and easy to share. For example, the data may be managed in a graph or table of some kind.
Data Warehousing
Data warehousing is a storage facility for digital information. This is different than a database or a data mart. Think of a database as the small selection of products offered at your local gas station. A data mart is a bit bigger and might be compared to your local grocery store. There is a lot more stored there but essentially, the items that are available are all food-related and geared towards consumers.
A data warehouse, on the other hand, is more like Costco. There is a massive amount of information stored there - pretty much all of the information that a business could ever want. Of course, having all of the information in one place doesn't make it any easier to use. But that's why software program exists to organize the data, manage it properly and even move it to other locations so that it can be used.
The Classes of Data Mining Tasks
There are a few different classes that the data mining task can be divided up into. Let's go over each of them so that you can further understand the process. The first class that we will discuss is the summarization or generalization of the data. Data that is relevant to that task needs to be summarized and made to be abstract which will give an overview of that data. For example, the buying habits of someone on Amazon might be summarized in to the amount of money spent, the total time on the site, the effectiveness of also bought advertising and much more.
Next we will discuss the classification of data. A model must be created so that each object will be placed into a class based upon specific attributes. This will allow you to classify future objects and understand the class better.
The next class is going to association. Association is when you connect objects together. This is done by a method called The Association Rule. You determine the rules that connect two objects together. Association rules allow you to see the relationship between objects and when you see certain objects appear you know that other sets of objects will also be appearing. This can be demonstrated by the Grocery store checkout counter. You may notice that candy bars and gum always seem to go together. This is because someone has figured out that people that buy a candy bar might also buy some gum.
Next is what is called clustering. Clustering is the identification and classification of sets of objects or groups, particularly for objects with unknown classes. The objects are clustered based upon what is the same about them or what is different about them. Once the clusters have been created and the features have been decided upon objects can become better organized with clustering.
Finally, trend analysis is a vital part of data mining. A trend is at least in the sense of data mining an event that is measured overtime. To use a very simple example, the sales of a specific product; if a company a sold Corn Flakes and Bran Flakes cereal, examining that sales record can determine trends such as when Corn Flakes out sold Bran Flakes, if ever, certain times of the year when one was more popular than the other or certain times of the year where cereal sales were higher than others, as well as a ton of other information. Trend analysis looks for interesting patterns in the history of objects. Ups, downs, peaks, valleys other patterns can tell companies a lot about that particular set of objects.
