Data Exploration
Data exploration refers to the initial step in data analysis in which data analysts use data visualization and statistical techniques to describe dataset characterizations, such as size, quantity, and accuracy, in order to better understand the nature of the data.
Why is Data Exploration Important?
Humans process visual data better than numerical data, therefore it is extremely challenging for data scientists and data analysts to assign meaning to thousands of rows and columns of data points and communicate that meaning without any visual components.
Data Exploration in Machine Learning
Data exploration steps to follow before building a machine learning model include:
-
Variable identification: define each variable and its role in the dataset
-
Univariate analysis: for continuous variables, build box plots or histograms for each variable independently; for categorical variables, build bar charts to show the frequencies
-
Bi-variable analysis - determine the interaction between variables by building visualization tools
-
~Continuous and Continuous: scatter plots
-
~Categorical and Categorical: stacked column chart
-
~Categorical and Continuous: boxplots combined with swarmplots
-
Detect and treat missing values
-
Detect and treat outliers
Why is Data Exploration Tools?
-
Power BI
-
Tableau
-
Weka
-
Rapid Miner