数据挖掘课件:chap3-data-exploration.ppt
- 【下载声明】
1. 本站全部试题类文档,若标题没写含答案,则无答案;标题注明含答案的文档,主观题也可能无答案。请谨慎下单,一旦售出,不予退换。
2. 本站全部PPT文档均不含视频和音频,PPT中出现的音频或视频标识(或文字)仅表示流程,实际无音频或视频文件。请谨慎下单,一旦售出,不予退换。
3. 本页资料《数据挖掘课件:chap3-data-exploration.ppt》由用户(罗嗣辉)主动上传,其收益全归该用户。163文库仅提供信息存储空间,仅对该用户上传内容的表现方式做保护处理,对上传内容本身不做任何修改或编辑。 若此文所含内容侵犯了您的版权或隐私,请立即通知163文库(点击联系客服),我们立即给予删除!
4. 请根据预览情况,自愿下载本文。本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
5. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007及以上版本和PDF阅读器,压缩文件请下载最新的WinRAR软件解压。
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 数据 挖掘 课件 chap3_data_exploration
- 资源描述:
-
1、 Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 Data Mining: Exploring DataLecture Notes for Chapter 3Introduction to Data MiningbyTan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 2 What is data exploration?lKey motivations of data exploration include He
2、lping to select the right tool for preprocessing or analysis Making use of humans abilities to recognize patternsu People can recognize patterns not captured by data analysis tools lRelated to the area of Exploratory Data Analysis (EDA) Created by statistician John Tukey Seminal book is Exploratory
3、Data Analysis by Tukey A nice online introduction can be found in Chapter 1 of the NIST Engineering Statistics Handbookhttp:/www.itl.nist.gov/div898/handbook/index.htmA preliminary exploration of the data to better understand its characteristics. Tan,Steinbach, Kumar Introduction to Data Mining 8/05
4、/2005 3 Techniques Used In Data Exploration lIn EDA, as originally defined by Tukey The focus was on visualization Clustering and anomaly detection were viewed as exploratory techniques In data mining, clustering and anomaly detection are major areas of interest, and not thought of as just explorato
5、rylIn our discussion of data exploration, we focus on Summary statistics Visualization Online Analytical Processing (OLAP) Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 4 Iris Sample Data Set lMany of the exploratory data techniques are illustrated with the Iris Plant data set. Can be o
6、btained from the UCI Machine Learning Repository http:/www.ics.uci.edu/mlearn/MLRepository.html From the statistician Douglas Fisher Three flower types (classes):u Setosau Virginica u Versicolour Four (non-class) attributesu Sepal width and lengthu Petal width and lengthVirginica. Robert H. Mohlenbr
7、ock. USDA NRCS. 1995. Northeast wetland flora: Field office guide to plant species. Northeast National Technical Center, Chester, PA. Courtesy of USDA NRCS Wetland Science Institute. Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 5 Summary StatisticslSummary statistics are numbers that s
8、ummarize properties of the data Summarized properties include frequency, location and spreadu Examples: location - mean spread - standard deviation Most summary statistics can be calculated in a single pass through the data Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 6 Frequency and M
9、odelThe frequency of an attribute value is the percentage of time the value occurs in the data set For example, given the attribute gender and a representative population of people, the gender female occurs about 50% of the time.lThe mode of a an attribute is the most frequent attribute value lThe n
10、otions of frequency and mode are typically used with categorical data Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 7 PercentileslFor continuous data, the notion of a percentile is more useful. Given an ordinal or continuous attribute x and a number p between 0 and 100, the pth percenti
11、le is a value of x such that p% of the observed values of x are less than . lFor instance, the 50th percentile is the value such that 50% of all values of x are less than . Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 8 Measures of Location: Mean and MedianlThe mean is the most common
12、measure of the location of a set of points. lHowever, the mean is very sensitive to outliers. lThus, the median or a trimmed mean is also commonly used. Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 9 Measures of Spread: Range and VariancelRange is the difference between the max and min
13、lThe variance or standard deviation is the most common measure of the spread of a set of points. lHowever, this is also sensitive to outliers, so that other measures are often used. Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 10 Visualization Visualization is the conversion of data in
14、to a visual or tabular format so that the characteristics of the data and the relationships among data items or attributes can be analyzed or reported.lVisualization of data is one of the most powerful and appealing techniques for data exploration. Humans have a well developed ability to analyze lar
15、ge amounts of information that is presented visually Can detect general patterns and trends Can detect outliers and unusual patterns Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 11 Example: Sea Surface TemperaturelThe following shows the Sea Surface Temperature (SST) for July 1982 Tens
16、 of thousands of data points are summarized in a single figure Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 12 RepresentationlIs the mapping of information to a visual formatlData objects, their attributes, and the relationships among data objects are translated into graphical elements
17、 such as points, lines, shapes, and colors.lExample: Objects are often represented as points Their attribute values can be represented as the position of the points or the characteristics of the points, e.g., color, size, and shape If position is used, then the relationships of points, i.e., whether
18、 they form groups or a point is an outlier, is easily perceived. Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 13 ArrangementlIs the placement of visual elements within a displaylCan make a large difference in how easy it is to understand the datalExample: Tan,Steinbach, Kumar Introduct
19、ion to Data Mining 8/05/2005 14 SelectionlIs the elimination or the de-emphasis of certain objects and attributeslSelection may involve the chossing a subset of attributes Dimensionality reduction is often used to reduce the number of dimensions to two or three Alternatively, pairs of attributes can
20、 be consideredlSelection may also involve choosing a subset of objects A region of the screen can only show so many points Can sample, but want to preserve points in sparse areas Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 15 Visualization Techniques: HistogramslHistogram Usually show
21、s the distribution of values of a single variable Divide the values into bins and show a bar plot of the number of objects in each bin. The height of each bar indicates the number of objects Shape of histogram depends on the number of binslExample: Petal Width (10 and 20 bins, respectively) Tan,Stei
22、nbach, Kumar Introduction to Data Mining 8/05/2005 16 Two-Dimensional HistogramslShow the joint distribution of the values of two attributes lExample: petal width and petal length What does this tell us? Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 17 Visualization Techniques: Box Plot
23、slBox Plots Invented by J. Tukey Another way of displaying the distribution of data Following figure shows the basic part of a box plotoutlier10th percentile25th percentile75th percentile50th percentile10th percentile Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 18 Example of Box Plots
24、 lBox plots can be used to compare attributes Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 19 Visualization Techniques: Scatter PlotslScatter plots Attributes values determine the position Two-dimensional scatter plots most common, but can have three-dimensional scatter plots Often add
25、itional attributes can be displayed by using the size, shape, and color of the markers that represent the objects It is useful to have arrays of scatter plots can compactly summarize the relationships of several pairs of attributesu See example on the next slide Tan,Steinbach, Kumar Introduction to
展开阅读全文