Tableau is a very powerful and fast growing data visualization tool used in many industries. It helps in simplifying raw data into a visual format which is very easy to understand at the first glance. Tableau helps to create stories describing the data that can be understood by professionals at any level in an organization. The great thing about Tableau is that it doesn't require one to be from a technical background to work with it. It allows non-technical users to create customized dashboards. Thus this tool has garnered interest among people from all sectors such as business, researchers, different industries, etc. What makes Tableau stand out from the rest is that data analysis is very fast with it and the visualizations created are in the form of dashboards and worksheets. Let's introduce you all to this wonderful tool. If you don't already have Tableau installed, please download it by referring to this blog. The first screen that appears when you open Tableau is it's start screen. On the left hand side of the start screen you get options to load data in various formats such as Excel, Text, JSON, Pdf files etc. or you can load data directly from a server as well. Here, you can also find dataset that comes with Tableau so that you can explore various functions of the tool. On the right hand side there are several tutorial videos to help you get started with this tool. Loading Dataset In this tutorial we will work with the superstore dataset. Although it is already available in Tableau please download it from below here since some changes have been made to it for the purpose of this tutorial. NOTE: Please get familiar with the dataset. All the column names are self-explanatory. Our dataset is an Excel file therefore on the start screen we will click on 'Microsoft Excel' , in the dialogue box that pops up go to the directory containing the dataset file. Once you open the dataset, another page appears that let's you know that the data set is now connected. Here you can also see the name of sheets ('Orders', 'People', and 'Returns') available in this file on the left hand side under a tab named 'Sheets'. You can also preview the data before getting things started, by dragging the sheets to the 'Drag sheets here' space on the screen. You can also perform various join and sort operations on the data according to your needs in this space and preview it. NOTE: We won't be performing join on our data in this tutorial, since we don't need it. Even at this point you can add more data by clicking on the 'add' button near the 'Connections' tab. Congratulations! we have reached our first milestone. Worksheets Now, that we have loaded and viewed our data, we have reached the point where we can mold it to get some information out of it. The work space in Tableau is called 'Worksheet'. In order to navigate to the worksheet you need to click on the 'Sheet 1' tab on the bottom left corner of the screen. (And just so you know, you can also change the name of the sheet by right clicking on it and then selecting the 'Rename' option, the same way as in Microsoft Excel.) The above picture names various sections of the worksheet. The topmost bar displays the name of the workbook. A workbook contains sheets. A sheet can be a worksheet, a dashboard, or a story. From the Toolbar we can access commands and analysis and navigation tools In the Cards and Shelves workspace area we drag fields to add data to a view. The View area is the canvas in the workspace where we can create a visualization (also referred to as a "viz"). On the side bar there are two panes, namely, Data pane and Analytics pane. In the Data pane there are two section: dimensions and measures, which groups together qualitative and quantitative fields in the dataset respectively. Also, the different icons in green and blue corresponds to different data types. Even the colors of the icon are significant, blue colored icons tells us that the data in the field is discrete where as green colored icons tells that the data is continuous in nature. Sheet tabs represents each sheet in a workbook. This can include worksheets, dashboards, and stories. Status bar displays information about the current view. You now have all the basic information and are just one step away from creating visuals. We have just passed our second milestone! Creating Charts We will create a sales trend chart using the superstore data. One thing to note is that, the x and y axes are represented by columns and rows in Tableau workspace. To create a sales trend chart we want the Dates on the x-axis and Sales on the y-axis. Thus, we will drag the Dates field from the Data pane to the columns shelve and Sales to the rows shelve. So, we just created a line chart. But, is that it? Well, yes! it's pretty easy right? But wait. We are not done yet. In the above picture we can see that on dragging the required fields into the columns and rows shelves, an yearly sales trend chart was made. But what if we want to dig deeper to get more insight from the data? To do that, we will click on the '+' sign before the YEAR field in the columns shelve. This makes the chart appear like this: Clicking on the '+' sign transforms the chart into a quarterly sales trend per year. Let's click on the '+' sign before the QUARTER field in the columns shelve. Well, this gives us a more detailed monthly sales trend chart. But there are several discontinuities in the chart. Why is that? Any guesses? (HINT: the colors are telling you something!) I hope you guessed it right. The data fields are blue in color which tells that these fields are discrete in nature, thus the presence of discontinuity in the chart. To resolve it, we will first click on the '-' (minus) sign before the YEAR and QUARTER fields which will bring us back to square one. Then, we will click on the downward facing arrow on the right side of the YEAR field. This will open a dropdown list with several options. We will select the "YEAR" option in the continuous variables section to make the data in this field to be treated as continuous. Doing this makes the field continuous and thus the color of the YEAR field in the columns shelve changes from blue to green. Now if we click on the '+' sign again as we did above, we will get the following visuals. One more way to bring the same change is shown below, where we change the 'Order date' field in the Data pane from discrete to continuous variable before dragging it to the column shelve. We can change the color , size, label, style of the chart using the Marks card at the left of the View. We can also change the type of chart using the 'Show me' tab in the top right corner of the worksheet. We will explore these charts in the coming blogs. One last lesson before we part ways: To make chart reading easier and more accurate, Tableau offers 'Tooltip'. If you hover the pointer over the line in the chart, it will show you the value of the x and y axes. A nice and detailed chart is a click away. You have achieved your goal, now you can rest and rejuvenate to walk yourself through another blog. In the upcoming blogs we will explore more of Tableau, so keep an eye out for it. Until then, If you need implementation for any of the topics mentioned above or assignment help on any of its variants, feel free to contact us.

Tableau is one of the best and powerful visualization tools these days. It is widely used in the field of Business Intelligence but it is also utilized in other sectors as well such as research, statistics, various industries etc. It simplifies raw data into understandable format. We can also manipulate data in it to get desired results. It is very easy to use and doesn't require any technical background to work with. The visualizations are created in the form of Dashboards and Stories. Tableau can be majorly classified into two sections: Developers tool: This consists of the tools that allows us to do the actual work i.e. create dashboards, reports, charts, stories and other visualizations. The products under this category are Tableau Desktop and Tableau Public. Sharing tools: The name is self-explanatory; it helps in sharing the visualizations created using developers’ tools. The products under this category are Tableau Online, Tableau Server, and Tableau Reader. All in all, there are Five products in Tableau: Tableau Desktop, Tableau Public, Tableau Online, Tableau Server, and Tableau Reader. Tableau Desktop: As mentioned earlier, this is where all the major work is done. It has a lot of features that lets you create visualizations very easily. It provides connectivity to Data Warehouse and other file types such as excel, text, json, PDF etc. The visualization can be stored locally or publicly. Tableau Desktop can be further classified in two parts: Tableau Desktop private: where the workbooks are kept private, and the access is limited. The workbooks cannot be published online, it can only be distributed either Offline or in Tableau Public. Tableau Desktop Professional: Here the only major difference is that the workbooks can be published online and there is full access to all the features. Tableau Public: It is the same as Tableau Desktop, but it is a public version i.e., it is free but the workbooks created cannot be saved locally. they can only be uploaded to Tableau's Public cloud where it can be seen and accessed by everyone. There is no privacy offered in this version. It is best for an individual who wants to learn working with Tableau. Tableau Server: It is essentially used to share the workbooks across the organization. The work needs to be published in Tableau Desktop first to be able to upload it on the server. Once uploaded, anyone with a license can view the work. Though it isn't necessary for the licensed user to have Tableau Server installed. If a person has valid login credentials, then he/she can view the work on a web browser. The admin of the organization will always have full control over the server. Tableau Online: It is an online sharing tool of Tableau. Its functionalities are similar to Tableau Server, but the data is stored on servers hosted in the cloud which are maintained by the Tableau group. There is no storage limit on the data that can be published. It creates a direct link to over 40 data sources that are hosted in the cloud such as the MySQL, Hive, Amazon Aurora, Spark SQL and many more. To publish, both Tableau Online and Server require the workbooks created by Tableau Desktop. Data that is streamed from the web applications say Google Analytics, Salesforce.com are also supported by Tableau Server and Tableau Online. Tableau Reader: It is tool used to view workbooks created using Tableau Developer tools. It doesn't allow editing and modification in the workbook. Anyone having the workbook can view it using Tableau reader. In fact, if you want to share the dashboards created by you then the receiver needs Tableau Reader to be installed. Tableau has the ability to connect to any platform to extract data. Simple databases such as excel, PDF; and complex databases like Oracle, a database in the cloud such as Amazon web services, Microsoft Azure SQL database, Google Cloud SQL and various other data sources can be extracted by Tableau. Tableau Uses Following are the main uses and applications of Tableau: Business Intelligence Data Visualization Data Collaboration Data Blending Real-time data analysis Query translation into visualization To import large size of data To create no-code data queries To manage large size metadata Download and Installation: Tableau Public: Step 1: Go to https://public.tableau.com/en-us/s/download. Enter your email id and click on "DOWNLOAD THE APP" button. Step 2: The .exe file for Windows will start downloading, and you will be able to see the downloading progress in the bottom left corner of the website. Step 3: Open the downloaded file. Accept the terms and conditions and click on "Install" button. Step 4: After installation the application would open to its home page. Tableau desktop: Step 1. Go to https://www.tableau.com/products/desktop. Step 2. Click the "TRY IT FOR FREE" button. Step 3. It will redirect you to another page where you need to enter your email id and click on "DOWNLOAD FREE TRIAL" button. Step 4. This will start downloading latest version of Tableau. An .exe file for Windows is downloaded, and you can see the downloading progress in the bottom left corner of the website. Step 5. Open the downloaded file. This will open the setup wizard. Accept the terms and conditions by checking the box and click on "Install" button. Step 6. A pop-up message would open asking for the approval of Administrator to install the software. Approve it and the installation of the Tableau Desktop on Windows system will start. Step 7. Once the installation is completed, the Tableau Desktop application would open. Step 8. A registration window will appear: Click on Activate Tableau and enter your license details, if you do not have a license, enter your credentials. Click on Start Trial and wait for registration to complete. Step 9. Wait for registration to complete. You are ready to use Tableau Desktop. Sample Visualizations: This section is going to be a treat to the eyes. We can create Dashboards to tell stories using the data at hand. Let's take a look at some Dashboards telling different stories. We will see the versatility of Tableau and the amazing visuals it offers. A dashboard showing sales of audiobooks. A dashboard showing sales of a super store. We can also make interactive and animated dashboards. Here are some dashboards exhibiting such properties which makes story-telling using data more interesting and eye-catching. Some more amazing dashboards! You may like these blogs as well: Data visualization tools Exploring data visualization using matplotlib and seaborn If you need implementation for any of the topics mentioned above or assignment help on any of its variants, feel free to contact us.

In this blog we will discuss Health risk assessment using Deep Learning Algorithms. What is Predictive analysis ? Predictive analytics helps connect the data to effective action by drawing a reliable conclusion which a data analyst can predict the future based on the current and previous data. This term is mainly used for analytics and statistical techniques. Health Risk Assessment Deep Learning The healthcare sector is lacking in actionable knowledge. This industry faces challenges in essential areas like electronic record management, data integration, and computer-aided diagnoses and disease predictions. It needs to reduce the healthcare cost and healthcare movements. Rapidly expanding the field of predictive analytics and deep learning play a pivotal role in the evolution of large volumes of healthcare data research. Deep learning provides a wide range of techniques, tools and framework to address these challenges. Nowadays Health data is expanding rapidly in various formats. This health data offers more opportunities for health data analysis and enhancement of health services by innovative approaches. Predictive analytics helps healthcare life sciences and providers and applies many techniques from statistics, data mining, modeling, machine learning, and artificial intelligence to investigate current findings to make predictions about the future. It helps healthcare organizations to prepare for health care by optimizing the cost, diagnosing the diseases accurately, enhancement of patient care, resource optimization and improves clinical outcomes The concept of deep learning is to dig a large volume of data to automatically identify patterns and extract features from complex unsupervised data without the involvement of humans, which makes it an important tool in big data analysis. Deep learning plays an important role in diagnostic applications. Deep learning techniques can reveal clinically relevant information hidden in the large data with a guidance of relevant clinical questions to assist clinical decision-making and in turn provides the physicians the analysis of any disease accurately for better treatment, thus resulting in better medical decisions. Predictive analytics using Deep Learning Health risk assessment predictive analytics aims to predict the health related outcomes based on clinical or non clinical patterns in the data. There are two methods to build the predictive model First, The collection of patient data in clinical trials with a set of predefined protocols. For example, Lung cancer risk prediction model, UK prospective diabetes study (UKPDS), Heart disease Prediction etc. Second, The use of existing patient data collected in clinical practice, such as EHRs, insurance claims, and clinical registries. For instance, the inpatient mortality predictive model. Predictive models capture the characteristics of the specific event. The UKPDS risk engine can predict coronary heart disease and stroke in patients with type 2 diabetes. Deep learning is widely used for medical imaging analysis in several different application domains. Medical imaging techniques such as MRI scans, CT scans, ECG, used to diagnose dreadful diseases such as heart disease, cancer, brain tumor. Hence, with the help of deep learning, the doctors can analyze the disease better and provide patients with the best treatment. In addition, deep learning is used to analyze medical insurance fraud claims. Moreover, deep learning helps the insurance industry to send out discounts and offers to their target patients. Deep learning technique used to detect Alzheimer’s disease at an early stage in which the medical industry faces the challenges currently. Deep learning techniques are used to understand a genome and help patients get an idea about the disease that might affect them, which has a promising future also. Deep Learning Framework Deep learning combines advances in computing power and neural networks with many layers to learn complicated patterns in large amounts of data. It is an extension of the classical neural network and uses more hidden layers so that the algorithms can handle complex data with various structures. Deep learning collects a large volume of data, including patients records, medical reports, and insurance records, and applies its neural networks to provide the best outcomes. Therefore, it is important to involve a deep learning role to resolve healthcare issues due to its representational and recognition supremacy that assists healthcare personnel to determine, predict, analyze, and practice its theories for the delivery of healthcare. Deep Learning Models Feature engineering is the main difference between traditional machine learning algorithms and deep learning algorithms. It required domain expertise and a time consuming process. Deep learning involves automatic feature engineering. Traditional machine learning algorithm Deep learning algorithm Convolutional neural network (CNN) model is the most commonly used deep learning algorithm. CNNs are composed of neurons that have learnable weights and biases. Each neuron receives inputs and creates a dot product. The complete network expresses a single differentiated function that scores the input aligned data of health attributes in accordance with the classes of health risks. CNNs have been proven to be more efficient in training inputs with a restricted number of parameters and hidden units. CNN can achieve local connections and tied weights efficiently by pooling translation invariant features. This specialty is adaptive to our design, as the input health data has been normalized and the output health risks have been predefined in certain classes We developed a diabetes Risk Prediction model Using Deep learning Algorithms. This Work is used to predict the diabetes in a patient. The dataset used here is the Pima indians diabetes. The dataset consists of 768 entries having 9 features. Pregnancies - Number of times pregnant Glucose - Plasma glucose glucose concentration a 2 hours in an oral glucose tolerance test Blood Pressure - Diastolic Blood Pressure (mm hg) Skin Thickness - Triceps skinfold thickness (mm) Insulin - 2 Hours serum insulin (mu U/ml) BMI - Body mass index (weight in kg/(height in m)^2 Diabetes Pedigree Function - Diabetes Pedigree Function Age - Age (years) Outcomes - class variable (0 or 1) 268 of 768 are 1 the others are 0(1 means the patient is diabetic and 0 means the patient is non diabetic) Sample Of Dataset : Here we can see the Training and validation accuracy and training and validation loss of diabetic Risk assessment model. Thank You

In this article we will see how to perform SQL & Hive operations to Analyze data in pyspark. As we know, Pyspark is the Python API for Spark. Spark SQL is a framework which runs on spark. Spark SQL is a Spark module for structured data processing and the use of Spark SQL is to execute SQL queries either SQL or HiveQL. SQLContext allows us to connect the different data sources to write and read the data. Spark SQL also reads and writes data which is stored in hive. Hive is used for Handling Big-Data. We can store multiple tables records into a single table using hiveQL. In this blog we also see how multiple table records are stored into a single table.. SQLContext is the entry point of all relational functionality in spark. Code Snippet : from pyspark.sql import SQLContext,SparkContext,HiveContext sc = SparkSession.builder.appName(“SQl_Hive”).getOrCreate() sqlContext = SQLContext(sc) Now let's see how to load the data and read data using SQLContext. Here it is as shown below In the First Line we are reading the data using the sqlcontext. In the Second line we Register the data frame as a Table. Now we can read the data with the help of SQL queries. Code Snippet : # Create the DataFrame df_employee = sqlcontext.read.csv("WA_Fn-UseC_-HR-Employee-Attrition.csv",inferSchema=True,header=True) df_employee.registerTempTable("df_employee") Using the following command we can read data from the table. Code snippet : #Read all the data from dataset sqlcontext.sql("SELECT * FROM df_new").show() As we can see using SQLContext we can read the data and perform various SQL operations to extract the information from the table. We can execute any SQL queries to grab the information into the table like Join queries, aggregate functions, Sub Queries etc. Output : Let's check whose employee Monthly income is highest among all other employees from each department by performing the SQL queries on the table. Code Snippet : sqlContext.sql("SELECT EmployeeNumber,Department,MonthlyIncome FROM df_new WHERE MonthlyIncome IN (SELECT MAX(MonthlyIncome) FROM df_new GROUP BY Department);").show() As shown below we can see the output is those employees whose monthly income is highest in their department. The employee number, employee department which he belongs to and his Monthly income. Output : We can also create a hive table in pyspark using the following command named employee with the fields employee_number, Department, Job_role, Age, Gender, Monthly_income, Over_time, Marital_status. For creating the table we have to import SparkContext and HiveContext. Code Snippet : from pyspark import SparkContext,HiveContext sc = SparkContext(“appName = “Hive_operations”) hive_context = HiveContext(sc) Using the following command We are creating the table Code snippet : hive_context.sql("CREATE TABLE employee1 (Employee_Number INT, Department STRING, Job_Roll STRING, Age INT,\ Gender STRING ,Monthly_Income INT, Over_time STRING, Marital_Status STRING) row format delimited \ fields terminated BY ',' tblproperties('skip.header.line.count'='0') stored AS textfile; ") hive_context.sql("select * from employee1").show() We can see Empty table is created Output : Now we are loading the data into the table from the .csv file using a hive query and the file is present in our local system. Code Snippet : # load the data in hive table hive_context.sql("LOAD DATA LOCAL INPATH 'employee1.csv' INTO TABLE employee1") Output : Now we will see how to merge two tables into a single table. Suppose we have two tables with the same structure we want to all record in a single table using a hive query it is possible. We have two hive table employee1 and employee2 using the following command we merge two tables. We can also merge multiple tables into a single table. Code snippet : # create a new table and concatenate data from employee1 and employee2 table hive_context.sql("create table employee as (select * from employee1 union all select * from employee2)") Here we can see all records from the employee1 and employee2 tables are merged into a single table. Output : Conclusion : In this article we are seen How to perform the some basic operation using SQLContext and HiveContext using Pyspark. Thank you

In statistics, logistic regression is a predictive analysis that is used to describe data. It is used to find the relationship between one dependent column and one or more independent columns. Dependent column means that we have to predict and an independent column means that we are used for the prediction. Before building the logistic regression model we will discuss logistic regression, after that we will see how to apply Logistic Regression Classification on datasets using Pyspark. Logistic regression Logistic regression is the machine is one of the supervised machine learning algorithms which is used for classification to predict the discrete value outcomes. It uses the statistical approach to predict the outcomes of dependent variables based on the observation given in the dataset. There are three types of Logistic regression Binomial Logistic Regression Multinomial Logistic Regression Ordinal Logistic Regression Advantages of Logistic regression : It is simple and easy to implement machine learning algorithms yet provide great training efficiency in some cases. Due to this reason it does not require high computational power. This algorithm is proven to be very efficient when the dataset has features that are linearly separable. This algorithm allows models to be updated easily to reflect new data, ulike decision trees or support vector machines. The update can be done using stochastic gradient descent. Its outputs well-calibrated Probabilities along with classification results. Disadvantages It can't solve nonlinear problems with logistic regression since it has a linear decision surface. Logistic Regression is a statistical analysis model that attempts to predict precise probabilistic outcomes based on independent features. On high dimensional datasets, this may lead to the model being over-fit on the training set, which means overstating the accuracy of predictions on the training set and thus the model may not be able to predict accurate results on the test set. This usually happens in the case when the model is trained on little training data with lots of features. Now here we are going build the Logistic regression model on the dataset using Pyspark Why PySpark ? Spark is much faster. Spark is multi-threaded. It means two or more executions run concurrently. Whereas pandas are single threaded. Spark will only execute when you take Action. So Lets Start.. Steps : - 1. Import some important libraries and create the SparkSession. SparkSession is the entry point of the program. Load the dataset search_engine.csv using pyspark. Code snippet : import findspark findspark.init() #import SparkSession import pyspark from pyspark.sql import SparkSession spark=SparkSession.builder.appName('Logistic_Regression').getOrCreate() #Read the dataset df=spark.read.csv('search_engine.csv',inferSchema=True,header=True) After loading the data when you run the code you will get the following result. Output : PrintSchema : It displays the structure of data. Calculate Statistical data like Count, Average, Standard deviation, Minimum value, Maximum value for each column ( Exploratory Data analysis). Code snippet : #statistical Data Analysis df.describe().show() Calculate total number of countries, platforms and status are present in datasets. Code snippet : #count the country present in the datasets df.groupBy('Country').count().show() #count the search engine present in the datasets df.groupBy('Platform').count().show() # Count the status df.groupBy('Status').count().show() Output : Lets see the visualization data. Code Snippet : import matplotlib.pyplot as plt import seaborn as sns df11=df.toPandas() sns.set_style('whitegrid') sns.countplot(x='Country',hue='Platform',data=df11) Output : Categorical Data cannot deal with machine learning algorithms so we need to convert into numerical data. We use StringIndexer to encode a column of string categories to a column of indices and The ordering of the indices is done on the basis of popularity and the range. when you convert the column into numbers you will get the following result. Code snippet : #import required libraries from pyspark.ml.feature import StringIndexer # Convert the platform columns to numerical search_engine_indexer = StringIndexer(inputCol="Platform", outputCol="Platform_Num").fit(df) df = search_engine_indexer.transform(df) #Dsiplay the categorial column and numerical column df.select(['Platform','Platform_Num']).show(10,False) df.select(['Country','Country_Num']).show(10,False) Output : Sometimes in a dataset, columns are found that do not have a specific number of preferences. The data in the column is usually shown by category or value of category and even when the data label in the column is encoded. So Now we are using OneHotEncoder to split the column which contains numerical data. when you split the column by using OneHotEncoder you will get the following result. We can see the platform column into the search_engine_vector column. Code snippet : #import library onehotencoder from pyspark.ml.feature import OneHotEncoder #one hot encoding search_engine_encoder = OneHotEncoder(inputCol="Platform_Num", outputCol="Search_Engine_Vector").fit(df) df = search_engine_encoder.transform(df) Output : Now we are using VectorAssembler to concatenate the multiple columns into a vector column. It will combine all the features of multiple columns in one column. After applying the VectorAssembler we can see all the columns concatenated into feature columns. Code snippet : #import vector assembler library from pyspark.ml.feature import VectorAssembler #concatenate all the columns in vector column. df_assembler = VectorAssembler(inputCols=['Search_Engine_Vector','Country_Vector','Age', 'Repeat_Visitor','Web_pages_viewed'], outputCol="features") df = df_assembler.transform(df) Output : Now Split your data into train and test data. Normally this is 70% and 30%. This method is used to measure the accuracy of the model. apply the Logistic regression model. After applying the model you will get the following result. Status columns have original data, prediction column means it will predict the value calculated by this model and last column is the probability column. Code snippet : #split the data training_df,test_df=model_df.randomSplit([0.75,0.25]) #import the logistic regression from pyspark.ml.classification import LogisticRegression #Apply the logistic regression model log_reg=LogisticRegression(labelCol='Status').fit(training_df) #Training Results train_results=log_reg.evaluate(training_df).predictions train_results.filter(train_results['Status']==1).filter(train_results['prediction']==1).select(['Status','prediction','probability']).show(10,False) Output : Accuracy comes out to 0.9396. It obtains 93 % values that are correctly predicted by this model. That means our model is doing a great job identifying the Status. Calculate the Precision Rate for our ML model. Precision Rate comes out to 0.9389. It means 93.89% Positive Predictions are correctly predicted. Code snippet : #Calculate the matchine record out of the total records accuracy=float((true_postives+true_negatives) /(results.count())) print("Accuracy : " + str(accuracy)) recall = float(true_postives)/(true_postives + false_negatives) print("Precision Rate : " + str(recall)) Output : Thank you

In this article you will learn about optical character recognition (OCR)? How does optical character recognition work? Let's start What is OCR? OCR stands for optical character recognition. We have plenty of information in the form of printed documents, handwritten scripts and images. OCR is the process to recognize scanned images of both handwritten and printed characters and convert it into a machine readable and digital format. There are three main aspect of OCR approach: Preprocessing Character recognition Character segmentation and presentation of data The OCR can be implemented by using convolutional neural networks. CNN is popular in deep neural network architecture. How does Optical character recognition Work? Techniques Image Processing in OCR The aim of pre-processing is to improve the quality of image data so that OCR model gives you accurate output. Mostly the OCR model gives an accurate output with 300 DPI. Image scaling refers to the resizing of a digital image. When we scanned the document sometimes the document was not properly aligned. Skewed image defines the image which is not straight. It directly impacts the line segmentation of the OCR model which reduces the accuracy rate. It may need to be tilted a few degree clockwise or counterclockwise in order to make lines of text perfectly horizontal or vertical. Character Recognition in OCR : There are two types of OCR algorithm. Matrix matching compares the image which is scanned by OCR scanner as a character with a library of character matrices or templates. When an image matches one of these matrices of dots within a given level of similarity. The computer labels the image according to that ASCII character. Feature Extraction is OCR without strict matching to prescribed templates. Also known as Intelligent Character Recognition (ICR), or Topological Feature Analysis, this method varies by how much "computer intelligence" is applied by the manufacturer. The computer looks for general features such as open areas, closed shapes, diagonal lines, line intersections, etc. This method is more versatile than matrix matching. Matrix matching works best if the OCR encounters a limited repertoire of type styles, with little or no variation within each style. Where the characters are less predictable, feature, or topographical analysis is superior. Post-Processing in OCR It is the error correction technique to ensure high accuracy of the OCR model. OCR accuracy can be increased if the output is constrained by lexicon. In this way the algorithm can make a list of words that are allowed to occur in the scanned document. Convolutional Neural Network CNNs are made of a large number of interconnected neurons that have learnable weights and biases. In CNN architecture the neurons are organized as layers. It contains the hidden layers, input layer, and output layer. When the large number of hidden layers in the network is generally said to be a deep neural network. The hidden layer neurons of CNN are connected to a small region of the input space generated from the previous layer instead of connecting to all, as in the fully connected network like Multi Layered Perceptron networks. This method reduces the number of connection weights in CNN compared to MLP. CNN takes less time to train for networks of similar size. The input to the typical CNN are two dimensional arrays of data such as images. Unlike the regular neural network the layers of a CNN are arranged in three dimensions. Basically the input layer is a buffer to hold the input and go to the next layer CNN performs the core operation of feature extraction and convolutional operation of the input data. ReLU Rectified Linear Unit is an activation function used to introduce non linearity. It replaces the negative value with zero. It can speed up the learning process. Every output of the convolutional layer is passed through the activation function. Pooling layer reduces the spatial size of each feature map, hence the computation is reduced in the network. It also uses a sliding window that moves in stride across the feature map and transforms it into representative values. Fully connected layers connect every neuron in the layer to all the neurons in the previous layer. It learns non-linear combinations of features and is used to classify or estimate output. For classification problems, the fully connected layer is followed by a soft-max layer, It produces the probability of each class for the given input. And for regression problems, it is followed by a regression layer to predict the output. Thank you

In this article we will show you how to determine the color minor dots, contour and center in an black image background using python 3.7, Open source Computer Vision Library (OpenCV) and numpy. Advancement of Artificial intelligence computer vision came in the late 1960's. Computer vision should be able to detect the 3d object and 2d pictures. Opencv is a computer vision library. It has been written in the programming languages c and c++. It can be easily run on windows and linux and interface with python language. Computer vision is a field of artificial intelligence that trains computers to interpret and understand the visual world. Color detection using opencv, allows detection of specific color in an image. In this opencv detection system read the image, scan the object, match the color and give the result.if the Color is matched with a defined color pattern by RGB color model then the system gets the correct output as a result. Image processing is a method to perform some operations on an image, in order to get an enhanced image or to extract some useful information from it. It is a type of signal processing in which input is an image and output may be image or characteristics/features associated with that image. We are using the jupyter notebook to determine color, contour and center. Jupyter Notebook IDE: Jupyter Notebook is an open source web-based application which allows you to create and share documents containing live code, equations, visualisations, and narrative text. The IDE also includes data cleaning and transformation, numerical simulation, statistical modelling, data visualisation, and many others. Step involved in contour and color detection in Python 3.7 Lets begin with a given sample image in either .jpg or .png format and apply object detection in it. To implement this project the following packages of python 3.7 have to be downloaded and installed. Description of Library used Numpy : Numpy is the most basic yet a powerful package for mathematical and scientific computing and data manipulation in python. It is an open source library available in python. Cv2: OpenCV is a high performance library for digital image processing and computer vision, which is free and open source. Imutils : imutils are a series of convenience functions to make basic image processing functions such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV. Code Snippet : import cv2 import numpy as np import imutils Procedure Read an image : First a simple image in which processing is to be applied is to be read. It's done using the opencv library. Next convert the image into hsv color model. HSV or Hue Saturation Value is used to separate image luminance from color information. This makes it easier when we are working on or need luminance of the image/frame. HSV is also used in situations where color description plays an integral role. Here, convert the image from RGB to HSV color space and then define a specific range of H-S-V values to detect color Code Snippet : img = cv2.imread('data\\left.png') hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) Normal image and HSV image : Now define a Lower color range and upper color range and convert it into numpy array Code Snippet : #Red Color lower_red = np.array([0,100,100]) upper_red = np.array([7,255,255]) # yellow color lower_yellow = np.array([25,100,100]) upper_yellow = np.array([30,255,255]) #green color lower_green = np.array([40,70,80]) upper_green = np.array([70,255,255]) #blue color lower_blue = np.array([90,60,0]) upper_blue = np.array([121,255,255]) #dark teal color lower_dark_teal = np.array([80,100,100]) upper_dark_teal = np.array([90,255,255]) #dark yellow color lower_dark_yellow = np.array([20,100,100]) upper_dark_yellow = np.array([25,255,255]) To perform the actual color detection using OpenCV, see below we use the cv2.inRange function. cv2.inRange function requires three argument images, where we are going to perform a color detection, the second lower range of the color you want to detect, and the third is the upper range you want to detect. Binary mask is returned, where white pixels (255) represent pixels that fall into the upper and lower limit range and black pixels (0) do not. Code Snippet : yellow = cv2.inRange(hsv, lower_yellow, upper_yellow) green = cv2.inRange(hsv, lower_green, upper_green) blue = cv2.inRange(hsv, lower_blue, upper_blue) red = cv2.inRange(hsv, lower_red, upper_red) dark_teal = cv2.inRange(hsv, lower_dark_teal, upper_dark_teal) dark_yellow = cv2.inRange(hsv, lower_dark_yellow, upper_dark_yellow) Now we will have to use find Contours to find the number of contours in the Image and find the center of each of them. cnts1 = cv2.findContours(red,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) cnts1 = imutils.grab_contours(cnts1) cnts2 = cv2.findContours(yellow,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) cnts2 = imutils.grab_contours(cnts2) cnts3 = cv2.findContours(blue,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) cnts3 = imutils.grab_contours(cnts3) cnts4 = cv2.findContours(green,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) cnts4 = imutils.grab_contours(cnts4) cnts5 = cv2.findContours(dark_teal,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) cnts5 = imutils.grab_contours(cnts5) cnts6 = cv2.findContours(dark_yellow,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) cnts6 = imutils.grab_contours(cnts6) Now Draw the contours for each blob and find the center of blobs using cv2.moments. Image Moment is a particular weighted average of image pixel intensities, with the help of which we can find some specific properties of an image, like radius, area, centroid etc. To find the centroid of the image, we generally convert it into a binary format and then find its center. Using the putText we can display the color name of each blob. Code Snippet : for c in cnts1: cv2.drawContours(img,[c],-1,(0,255,0),3) # compute the center of the contour M = cv2.moments(c) if M["m00"] != 0: cX = int(M["m10"] / M["m00"]) cY = int(M["m01"] / M["m00"]) else: cX, cY = 0, 0 cv2.circle(img, (cX, cY), 7, (255, 255, 255), 1) cv2.putText(img, "red", (cX - 20, cY - 20),cv2.FONT_HERSHEY_SIMPLEX, 2.5, (255, 255, 255), 1) Using the following code we can get out Code Snippet : cv2.imshow("result",img) k = cv2.waitKey(0) if k == 27: break Output : Conclusion : Computer vision can be used to solve most of the problems with utmost sophistication. We can use both Python and MATLAB for Computer Vision, but we prefer Python because it takes less simulation time than MATLAB. Contours, Centers & colors were detected in the given sample images successfully. Thanks You,

In this article, we are going to see the recommendation system and How it works. After that we will develop a movie recommendation model system. Lets Begin. What is a recommender System ? Recommender Systems is techniques providing suggestions for items to be of use to a user. The suggestions provided are aimed at supporting their users in various decision-making processes such as what product to buy, which book to read, which movie to watch etc. The system has proven to be a valuable tool for online users to cope with information overload. Become one of the most popular and most powerful tools in electronic commerce. Various techniques for recommendation generation have been proposed during the last decade and many of them are successfully deployed in commercial environments. Recommendation Techniques There are three main technique to build the recommendation system Content based method Collaborative filtering methods Hybrid methods Content based method Content based system which uses characteristic information. This information about item keywords categories etc. and users preferences, profile etc. The system learns to recommend items that are similar to the one that users liked in the past. The similarity of items is calculated based on the features associated with the compared items. For example, if a user has positively rated a movie that belongs to the action or thriller genre, then the system can learn to recommend other movies from this genre. Collaborative Filtering methods In this method recommend items based on similarity measures between users and items. The items recommended to a user are those preferred by similar users. For example a user likes Product A and another user likes the same product A as well as another product B, The first user could also be interested in the second product. Their aim is to predict new interactions based on historical ones. There are two types of Collaborative filtering methods. Memory based Model based In a memory based method the first way is to identify the cluster of users and interactions of one specific user to predict the interactions of other similar users. Second way identifies clusters of items that have been rated by user A and utilizes them to predict the interaction of user A with a different but similar product B. In a model based is used data mining and machine learning techniques. The aim is to train the models to be able to make the prediction. Hybrid Methods Hybrid method combines the collaborative filtering and content based methods. Benefits of Recommender system Increase number of sales items Very few techniques to increase the number of sales items without increasing the marketing efforts. Once you build the automated recommendation system, you will get recurring additional sales without any efforts. Increase user satisfaction A well developed Recommendation model can also improve the experience of the user. The user will find the recommendations interesting, relevant and, with a properly designed human-computer interaction, and will also enjoy using the system Better understand what the user wants The description of the user’s preferences, either collected explicitly or predicted by the system. The service provider may then decide to re-use this knowledge for a number of other goals such as improving the management of the item’s stock or production Increase user fidelity A user should be loyal. Many recommendation models compute recommendations, leveraging the formal information acquired from the user in previous interaction. For example, rating of items. Application of Recommedation systems Product recommendation: Most important use of the recommender system is at online retailers. All Ecommerce websites and online vendors try to present each returning user with some suggestion of a product that they might like to buy. Movie recommendation : Netflix offers its customers recommendations of movies they might like. These recommendations are based on ratings provided by users Books recommendation : like kindle offers its customers recommendations of books they might like. These recommendations are based on ratings provided by users Now we are going to build the Movie recommendation model system. which provides you with the recommendations of the movies that are similar to the ones that have been watched in the past. For that The dataset I used here directly comes from netflix and I am importing this dataset from kaggle. The dataset contains the 4 text files. Each text file contains over 20 m rows. But here I am using only one text file for building the recommendation model due to processing time. Here I have already imported some packages which will be needed for build the model Code snippet : # import libraries import pandas as pd import numpy as np import matplotlib.pyplot as plt from surprise import Reader, Dataset, SVD from surprise.model_selection import cross_validate In this step I have defined a read_data function for reading the data and then stored data in a variable. Code snippet : # define function for reading data def read_data(data_loc): df = pd.read_csv(data_loc, header=None, names = ['Customer_Id','Ratings'],usecols=[0,1]) return df # data path path_1 = "/content/combined_data_1.txt" path_2 = "/content/combined_data_2.txt" path_3 = "/content/combined_data_3.txt" path_4 = "/content/combined_data_4.txt" data_1 = read_data(path_1) data_2 = read_data(path_2) data_3 = read_data(path_3) data_4 = read_data(path_4) Here we can see the shape of all text file data in each file contains over 20 million records In this step Count the total records, total movies, total customer and total ratings Code snippet : # total movies total_movies = all_data.isnull().sum()[1] # total customers total_customer = all_data['Customer_Id'].nunique() - total_movies # total ratings total_ratings = all_data['Customer_Id'].count() - total_movies print("Total Records ",all_data.shape[0]) print("Total Movies ",total_movies) print("Total Customers ",total_customer) print("Total ratings ",total_ratings) Output : In this step Now here Iterating the ratings data in a loop and calculating the percentage of each rating then and that data will be visualizing. Code snippet : erc = all_data.groupby('Ratings')['Ratings'].agg(['count']) ## Plotting the graph ax = erc.plot(kind = 'barh', legend = False, figsize = (15,10)) plt.title('Total : {:,} Movies, {:,} customers, {:,} ratings given'.format(total_movies, total_customer, total_ratings), fontsize=20) plt.axis('off') for i in range(1,6): ax.text(erc.iloc[i-1][0]/4, i-1, 'Rating {}: {:.0f}%'.format(i, erc.iloc[i-1][0]*100 / erc.sum()[0]), color = 'white', weight = 'bold') Now here I am extracting records ratings nan value is True and creating a new dataframe to know all that where does the movie counting start from. creating a numpy array containing movie ids according the 'ratings' dataset and then store it in list arr_movie create a new array Account for last record and corresponding length append the created array to the dataset after removing the 'nan' rows. After that these two columns are converted into an integer. Code Snippet : # # To count all the 'nan' values in the Ratings column in the 'ratings' dataset data_nan = pd.DataFrame(pd.isna(all_data.Ratings)) data_nan = data_nan[data_nan['Ratings'] == True] arr_movie = [] movie_id = 1 for i,j in zip(data_nan['index'][1:],data_nan['index'][:-1]): temp = np.full((1,i-j-1), movie_id) arr_movie = np.append(arr_movie, temp) movie_id += 1 final_rec = np.full((1,len(all_data) - data_nan.iloc[-1, 0] - 1),movie_id) arr_movie = np.append(arr_movie, final_rec) print('Movie numpy',arr_movie) print('Length',(len(arr_movie))) all_data = all_data[pd.notnull(all_data['Ratings'])] all_data['Movie_Id'] = arr_movie.astype(int) all_data['Customer_Id'] = all_data['Customer_Id'].astype(int) print('Data') print(all_data.iloc[::5000, :]) Output : Created a list of all the movies that are rated less often.It includes only top 30% rated movies, for that count the ratings and find the mean value by movie id. And this code will return output the minimum number of times of review, which is less often. Code Snippet : f = ['count','mean'] movie_gb_mi = all_data.groupby('Movie_Id')['Ratings'].agg(f) movie_gb_mi.index = movie_gb_mi.index.map(int) movie_benchmark = round(movie_gb_mi['count'].quantile(0.7),0) drop_movie_list = movie_gb_mi[movie_gb_mi['count'] < movie_benchmark].index print('Movie minimum times of review:',(movie_benchmark)) Similarly created a list of all the inactive users who rate less often for that count the ratings by customer_id. This will return the top 30 % minimum times of review by customers. Code Snippet : cust_gb_ci = all_data.groupby('Customer_Id')['Ratings'].agg(f) cust_gb_ci.index = cust_gb_ci.index.map(int) cust_benchmark = round(cust_gb_ci['count'].quantile(0.7),0) drop_cust_list = cust_gb_ci[cust_gb_ci['count'] < cust_benchmark].index print('Customer minimum times of review:',(cust_benchmark)) Dropping a list of all the movies which get rated less often. And also Dropping a list of all the inactive customers who rate less often. Code Snippet : print('Original Shape: ',all_data.shape) all_data = all_data[~all_data['Movie_Id'].isin(drop_movie_list)] all_data = all_data[~all_data['Customer_Id'].isin(drop_cust_list)] print('After droping the Shape is : ',(all_data.shape)) print('Data') all_data Output : Creating the matrix ratings for values, index for customer id and movie id for columns. we need it for our recommendation system Code Snippet : data_pivot = pd.pivot_table(all_data,values='Ratings',index='Customer_Id',columns='Movie_Id') print(data_pivot.shape) data_pivot Output : We have one more dataset of which is movie titles. taking only 2 lacs record for fast processing. After that applied the SVD algorithm on that dataset. Which is created. Code Snippet : movie_titles = pd.read_csv('/content/movie_titles.csv', encoding = "ISO-8859-1", header = None, names = ['Movie_Id', 'Year', 'Name']) movie_titles.set_index('Movie_Id', inplace = True) movie_titles.head() # reader reader = Reader() # get just top 2Lacs rows for faster run time data = Dataset.load_from_df(all_data[['Customer_Id', 'Movie_Id', 'Ratings']][:200000], reader) # Use the SVD algorithm. svd = SVD() # Compute the RMSE of the SVD algorithm cross_validate(svd, data, measures=['RMSE', 'MAE']) Here we can see the cross validation result of SVD algorithm Output : Taking one user customer id extract the record who has given five ratings to those movies Code Snippet : cust_1493615 = all_data[(all_data['Customer_Id'] == 1493615) & (all_data['Ratings'] == 5)] cust_1493615 = cust_1493615.set_index('Movie_Id') cust_1493615 = cust_1493615.join(movie_titles)['Name'] cust_1493615 Output : Now here predicting the movie above users who have rate 5 rating the movies. Now here see which movies he loves to watch. First dropping the list of all movies who rate less often from the movie title dataset. Taking full data set customer id movie id and ratings and store it in variable data1 After that fit train set on svd model and then predicted the ratings which can say here estimated score of this user Code snippet : customer_1493615 = movie_titles.copy() customer_1493615 = customer_1493615.reset_index() customer_1493615 = customer_1493615[~customer_1493615['Movie_Id'].isin(drop_movie_list)] # getting full dataset data1 = Dataset.load_from_df(all_data[['Customer_Id', 'Movie_Id', 'Ratings']], reader) trainset = data1.build_full_trainset() svd.fit(trainset) customer_1493615['Estimate_Score'] = customer_1493615['Movie_Id'].apply(lambda x: svd.predict(1493615, x).est) customer_1493615 = customer_1493615.drop('Movie_Id', axis = 1) customer_1493615 = customer_1493615.sort_values('Estimate_Score', ascending=False) customer_1493615 Our Final Prediction Output : Thank You,

In this article we will see what is scapy? How it works but before we must know about the network, protocol and their types, TCP port number and packets. Let's start. What is a network? A network consists of two or more computers in the network that use a set of common communication protocols over digital interconnection for the purpose of sharing information. Computer networks are categorized as Local Area Network LANs, Metropolitan Area Networks MANs and Wide Area Network WANs based on the maximum distance on the network. Networks that connect hosts in a room building, or building of campus are called LANs. The distance between hosts on a LAN can be anywhere from a few meters to about one kilometer. Networks that are used to connect hosts within a city, or between small cities are known as MANs. The distance between hosts on a MANs is about one to 20 kilometers. Networks that are used to connect hosts within a state or country are known as WANs. The distance between hosts on a WANs is in the range of ten of kilometers to thousands of kilometers. Protocol is a set of rules that allow two or more devices to exchange information over the network. It is a router that determines a packet's path from source to destination hardware implemented protocols in the network interface cards of two physically connected computers control the flow of bits on the wire between the two computers. A congestion control protocol controls the rate at which packets are transmitted between sender and receiver. Protocols are running everywhere on the internet. Transmission Control Protocol (TCP) is used for a set of rules to send the data into packet form over the network and ensure the successful delivery of the data on the destination site on the network . Internet Protocol is used for a set of rules to send and receive data at the IP address. An IP address is the unique identification for every device which is connected across the network. This Internet protocol has two versions IPv4 and IPv6. IPv4 IP stands for Internet Protocol and v4 stands for version 4. It has 12 fields and Each IP address is 32 bit long (Equivalent 4 bytes). IP addresses are typically written in so-called dot decimal notation. Each byte of the address is in the decimal form and Each value should be between 0 to 255. For example, a typical address would be 193.32.216.9. The 193 is the decimal equivalent for the first 8 bits of the address and the 32 is the decimal equivalent for the second 8 bits of the address etc. IPv6 New networks and IP nodes are being attached to the internet at a breathtaking rate. To respond to this need of large IP space, a new IP protocol, IPv6, was developed. IPv6 address moves 32 bit to a 128-bit address space. It allows 340 undecillion unique address spaces. IPv6 address contains letters and numbers and it has 8 groups. For example a typical address would be 2002:db8::8a2f:362:7797. In IPv6 there are three types of addresses. In addition to unicast and multicast addresses, a new type of address, called an anycast address, has also been introduced which allows a packet addressed to an anycast address to be delivered to any one of a group of hosts TCP port number An IP address is not sufficient to run a network application. On a computer running multiple applications or services. IP address just identifies only devices not their application. Port number identifies the application and services which are running on the computer. A TCP port number is a unique identification of each application and it is used with an IP address. A port is a 16 bit unsigned integer, and the total number of ports available in the TCP/IP model is 65,535 ports. Hence the range of port numbers is 0 to 65,535. Here is an example of port number 193.32.216.9:7. In that case 192.32.216.9 is an IP address and 7 is port number. What is Scapy Scapy is a python program and it is one of the best packet manipulation tools. It enables the user to send the dissect, sniff and forge packet. This allows their construction tools to scan, probe or attack on the network. In short, scapy is a most powerful tool for packet manipulation programs. We can easily forge and dissect packets with the help of scapy. Tracerouting, probing, unit testing and network scanning are some tasks handled by scapy. It can change most important tools like hping, arpspoof, arp-sk, arping, p0f etc. We can also set the source IP and destination IP and then we can send the packet from source IP to destination IP. Scapy does two things: sending packets and receiving answers. Scapy is also capable of sending invalid frames, injecting your own 802.11 frames, combining techniques. Scapy sends a packet, and receives an answer that matches requests with answers and returns a list of packet couples (request, answer) and a list of unmatched packets. This has the big advantage over tools like Nmap or hping that an answer is not reduced to (open/closed/filtered), but is the whole packet. When we communicate or exchange the information across the network all the data is transferred in the packet form. Packet is a small segment of a large message. These packets are then recombined by the device that receives them. Packet header stores all the information like source and destination IP address TCP ports etc. Why is scapy so special ? Fast Packet Design Scapy paradigm is to propose a domain specific language enabling a powerful and fast description of any kind of packet. Scapy enables the user to describe a packet or set of packets as layers that are stacked one upon another. Scapy does not oblige the user to use predetermined methods or templates. Probe once, interpret many Scapy gives all the information, i.e. all the stimuli sent and all the responses received. Scapy gives the complete raw data, that data may be used many times allowing the viewpoint to evolve during analysis For e.g. a TCP port scan may be probed and the data visualized as the result of the port scan. Pcap is an application programming interface for capturing network traffic. Wireshark is a network analyzer that is to create the .Pcap file to collect and record data from the network. Now here Using the rdcap read the pcap file displaying data. Code Snippet : from scapy.all import * import os d = " " data = "202011251400-78-5k.pcap" d = rdpcap(data) sessions = d.sessions() print(sessions) Output : Now Let's analyze the network data. Identify elephant TCP flows, i.e., those flows that transfer a large amount of data, in a packet trace captured at a vantage point in the ISP network. Now here Read the .Pcap file using PcapReader() and store the source IP, destination IP, source port number and destination port number and also packet length in the dictionary form. Code Snippet : from scapy.layers.inet import IP, TCP pkts = 0 flows = 0 ft = {} for pkt in PcapReader(data): pkts += 1 if IP in pkt: if TCP in pkt: ft[(pkt[IP].src,pkt[IP].dst,pkt[TCP].sport,pkt[TCP].dport)]= pkt.wirelen flows +=1 ft Output : Plotting Top 100 TCP flow size distribution Code Snippet : topn = 100 data = [i/1000 for i in list(self.ft.values())] data.sort() data = data[-topn:] fig = plt.figure() ax = fig.add_subplot(1,1,1) ax.hist(data, bins=20, log=True) ax.set_ylabel('# of flows') ax.set_xlabel('Data sent [KB]') ax.set_title('Top {} TCP flow size distribution.'.format(topn)) plt.savefig(sys.argv[1] + '.flows.pdf', bbox_inches='tight') plt.close() Output : Thank You.

In this article we will discuss Convolutional neural networks and perform image classification using keras. What is CNN ? CNN stands for convolutional neural networks. Convolutional is one type of scanning and the Neural network is a system of interconnected artificial neurons that exchange messages to each other. Nowadays CNN is widely used for pattern and image recognition techniques. They have a number of advantages compared to other techniques. Hence so many companies use these techniques. How does it work ? Machines identify the image as a pixel form and do these pixel scanning by filter and collect information of the input image. When we slide the filter on the image we get new values by matching from the pixel value. These values help us to identify all the image features. Image features are fed into the convolutional layer. Outputs of convolutional layers are fed into a fully connected layer. Finally CNN predicts the features of the image and we get the result. Different layers of CNN Convolutional layer Poling layer Fully connected layer Why is CNN required ? It gives high accuracy It require less memory for processing and execution Image detection using CNN is robust to distortion like shape change due to camera lens, different position, different lighting conditions etc. Now we are going to perform the Chest X-Ray image classification using Keras. We have a dataset of Chest X-Ray images (pneumonia). We will perform the image classification using keras. Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow. It is an open-source software library that provides a Python interface for artificial neural networks. It is an open-source software library that provides a Python interface for artificial neural networks. Keras follows best practices for reducing cognitive load: it offers consistent & simple APIs, it minimises the number of user actions required for common use cases, and it provides clear & actionable error messages. It also has extensive documentation and developer guides. First we will import some important libraries. We have a dataset of chest X-Ray images which is downloaded from kaggle. The Train dataset contains a total of 5216 images, where 1341 images belong to the Normal category, 3875 images depicted Pneumonia And, Test dataset contains a total of 624 images, where 234 images belong to the normal category, 390 images depicted Pneumonia. # Importing all necessary libraries from keras.preprocessing.image import ImageDataGenerator from keras.models import Sequential from keras.layers import Conv2D, MaxPooling2D from keras.layers import Activation, Dropout, Flatten, Dense from keras import backend as K import matplotlib.pyplot as plt from os from os.path from pathlib import * import pandas as pd import seaborn as sns Create neural network models and to train a dataset. The architecture of a CNN is a key factor in determining its performance and efficiency. The way in which the layers are structured, which elements are used in each layer and how they are designed will often affect the speed and accuracy with which it can perform various tasks. The function Sequential( ) is used to group a linear stack of layers into a tf.keras.Model. We defined an instance of Sequential named 'model' and using it we added 3 hidden layers to the neural network. The first two layers have 'relu' as their activation function and the last one has 'softmax' as its activation function. We then combined all the layers using model. compile and trained the model on our training data. Convolutional layer Apply Convolutional operation on the input data using conv_2d library and passing to the MaxPooling 2 layer. Conv_2d converts the image into multiple images. Activation is the activate function. This layer computes the dot product between their weights and a small region and they are connected to the input layer. It will change the dimension depending on the filter size. Relu Relu is a Rectifying Linear Unit. The Relu function returns the positive value or 0 in place of the previous negative value. It does not change the dimension of the previous layer. Max Pooling layer Apply max_pool_2d on the convolutional layer. This layer will reduce the spatial size of the convolved features and also reduce overfitting by providing an abstract representation of them. Below is an example of a max pooling layer. Fully connected layer This layer is the final layer of the CNN model. This layer computes all the input layers and determines the final prediction. In this layer all the input layer nodes are connected to every node in the second layer. We can use one or more than fully connected layers. Dropout layer Dropout layer is a regularization technique and it is used for reducing overfitting in CNN. In general the dropout layer is used in fully connected layers in deep learning models but it is possible to use after the max_pooling layer. Dropout layer should not be used on test data because dropout is a random process of disabling the neuron in a layer and your network will have different output every sequence of activation. Code Snippet : model = Sequential() model.add(Conv2D(32, (2, 2), input_shape=input_shape)) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(32, (2, 2))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(64, (2, 2))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(32, (2, 2))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Flatten()) model.add(Dense(64)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(1)) model.add(Activation('sigmoid')) model.compile(loss='binary_crossentropy',optimizer='rmsprop',metrics=['accuracy']) model.summary() Once a model is "built", you can call its summary() method to display its contents However, it can be very useful when building a Sequential model incrementally to be able to display the summary of the model so far, including the current output shape. In this case, you should start your model by passing an Input object to your model, so that it knows its input shape from the start We will plot a graph of the number of pneumonia affected X-Ray images and normal images present in the training dataset. Code Snippet : # count plot of training dataset sns.set_style('whitegrid') sns.countplot(x='Label',data=train_df) In the following count plot we can see the dataset contains pneumonia affected X-Ray are over 3500 and Normal X-Rays are below 1500. Output : We will plot some images which is present in the dataset Code snippet : # Display some pictures of the dataset fig, axes = plt.subplots(nrows=4, ncols=6, figsize=(15, 7),subplot_kw={'xticks': [], 'yticks': []}) for i, ax in enumerate(axes.flat): ax.imshow(plt.imread(train_df.Filepath[i])) ax.set_title(train_df.Label[i], fontsize = 15) plt.tight_layout(pad=0.5) plt.show() Output : We will now check the accuracy and loss of the model and will plot the same. Code Snippet : # Plot graph plt.plot(history.history['accuracy'],'r',label='training accuracy') plt.plot(history.history['val_accuracy'],'b',label='validation accuracy') plt.xlabel('epochs') plt.ylabel('accuracy') plt.legend() plt.show() plt.plot(history.history['loss'],'r',label='training loss') plt.plot(history.history['val_loss'],'b',label='validation loss') plt.xlabel('epochs') plt.ylabel('loss') plt.legend() plt.show() Output : Thank You

In this Article we are going to analyze the IPL dataset. In this analysis we are focusing on bowler performance during the Indian premier league. The data which has been used in this article has been gathered from Kaggle. The data set consists of data about IPL matches played from the 2008 to 2019. We have two dataset deliveries and matches. In Deliveries data set contains the 21 attributes and 179078 records and The matches dataset contains 18 attributes and 756 records. Our Objective To find top 10 Player names who takes most wickets To find top 10 Player names who have bowled most no balls To find top 10 Player names who have bowled most wide balls To find top 10 Player names by their bowling average To find top 10 Player names by their bowling strike rate To find top 10 Player names by their bowling Economy rate Number of winning teams To find top 10 Player names who most runs To find top 10 Player names who become the most of the time man of the match Our Goal : Basic Exploratory Analysis Features Analysis Dependencies/Libraries Required: In this step, we imported all the required libraries like seaborn, pandas(for preprocessing), math, matplotlib etc. import pandas as pd import numpy as np import matplotlib.pyplot as plt import seaborn as sns import math from IPython.display import display Loading the data ipl_deliveries = 'IPL Data 2008 to 2019\\deliveries.csv' ipl_match = 'IPL Data 2008 to 2019\\matches.csv' ipl_deliveries = pd.read_csv(ipl_deliveries) ipl_match = pd.read_csv(ipl_match) Output : Displaying deliveries dataset Attributes names of deliveries dataset Matches data Attributes name of matches dataset Data Preparation and Data Cleaning The data set found inaccurate records. the same team with two different names. So In this step, replace the team name and venue name also. Now the data is ready for analysis. Code snippet : ipl_match.team1.replace({'Rising Pune Supergiants' : 'Rising Pune Supergiant'},regex=True,inplace=True) ipl_match.team2.replace({'Rising Pune Supergiants' : 'Rising Pune Supergiant'},regex=True,inplace=True) ipl_match.winner.replace({'Rising Pune Supergiants' : 'Rising Pune Supergiant'},regex=True,inplace=True) ipl_match.venue.replace({'Feroz Shah Kotla Ground':'Feroz Shah Kotla', 'M Chinnaswamy Stadium':'M. Chinnaswamy Stadium', 'MA Chidambaram Stadium, Chepauk':'M.A. Chidambaram Stadium', 'M. A. Chidambaram Stadium':'M.A. Chidambaram Stadium', 'Punjab Cricket Association IS Bindra Stadium, Mohali':'Punjab Cricket Association Stadium', 'Punjab Cricket Association Stadium, Mohali':'Punjab Cricket Association Stadium', 'IS Bindra Stadium':'Punjab Cricket Association Stadium', 'Rajiv Gandhi International Stadium, Uppal':'Rajiv Gandhi International Stadium', 'Rajiv Gandhi Intl. Cricket Stadium':'Rajiv Gandhi International Stadium'},regex=True,inplace=True) Now here first we replace the name of id to match_id and then combined the data by their match_id Code Snippet : ipl_match.rename(columns={'id':'match_id'},inplace=True) combine_data = pd.merge(ipl_deliveries,ipl_match,on='match_id') pd.set_option('display.max_columns',None) combine_data.head(2) Now In this step, Gathered information about the bowler and then stored it into a dictionary. Code Snippet : bowler_performance={} for i in range(0, len(combine_data['bowler'])): try: total_balls = bowler_performance[combine_data['bowler'][i]][0] + 1 total_runs = bowler_performance[combine_data['bowler'][i]][1] + combine_data['total_runs'][i] total_wickets = bowler_performance[combine_data['bowler'][i]][2] wide_balls = bowler_performance[combine_data['bowler'][i]][3] no_balls = bowler_performance[combine_data['bowler'][i]][4] if(combine_data['wide_runs'][i] != 0): wide_balls = wide_balls + 1 if(combine_data['noball_runs'][i] != 0): no_balls = no_balls + 1 try: if(math.isnan(combine_data['dismissal_kind'][i])): bowler_performance[combine_data['bowler'][i]] = [total_balls, total_runs, total_wickets, wide_balls, no_balls] except: total_wickets = bowler_performance[combine_data['bowler'][i]][2] + 1 bowler_performance[combine_data['bowler'][i]] = [total_balls, total_runs, total_wickets, wide_balls, no_balls] except: try: if(math.isnan(combine_data['dismissal_kind'][i])): bowler_performance[combine_data['bowler'][i]] = [0, combine_data['total_runs'][i], 1, 0, 0 ] except: bowler_performance[combine_data['bowler'][i]] = [0, combine_data['total_runs'][i], 0, 0, 0 ] analysis_condition = [] analysis_condition.append(['Name', 'Total balls', 'Total runs', 'Total wickets','Wide balls', 'No balls']) for i in bowler_performance: analysis_condition.append([[i][0], bowler_performance[i][0], bowler_performance[i][1], bowler_performance[i][2], bowler_performance[i][3], bowler_performance[i][4]]) print(analysis_condition) Output : In this step we extract the information from the dictionary using a for loop and store it into a list after create a dataframe and store it all data from the list. Here To visualize the data defined a function to plot the bar plot Code Snippet : def bar_plot(data,x,y,titles): plt.figure(figsize=(20,10)) sns.barplot(x, y, data=data[:10]) plt.title(titles,size=20) plt.xticks(rotation=45,size=15) plt.yticks(size=15) plt.show() Calling the bar_plot function to visualize the top 10 Player names who take most wickets. In the Barplot we can see the SL malinga takes the most wickets in IPL session from 2008 to 2019 Code snippet : tw = bowler_data[:].sort_values(by='Total_wickets',ascending=False) bar_plot(tw,'Bowler_name','Total_wickets','Bowler Names vs Total Wickets') Call the defined function to visualize the top 10 Player names who have bowled most wide balls. In the Barplot we can see the SL malinga bowled the most wide ball in IPL session from 2008 to 2019 Code Snippet : twb=bowler_data[:].sort_values(by='Total_wide_balls',ascending=False) bar_plot(twb,'Bowler_name','Total_wide_balls','Bowler Names vs Total Wide Balls') Call the defined function to visualize the top 10 Player names who have bowled most No balls. In the Barplot we can see the S Sreesanth bowled the most No balls in IPL session from 2008 to 2019 Code snippet : tnb=bowler_data[:].sort_values(by='Total_No_balls',ascending=False) bar_plot(tnb,'Bowler_name','Total_No_balls','Bowler Names vs Total No Balls') Call the defined function to visualize the top 10 Player names Highest bowling average. In the Barplot we can see the K Goel is the number one position highest bowling average in IPL session from 2008 to 2019 Code snippet : tba=bowler_data[:].sort_values(by='Bowling_average',ascending=False) bar_plot(tba,'Bowler_name','Bowling_average','Bowler Names vs Bowling_average') Call the defined function to visualize the top 10 Player names Highest bowling strike rate. In the Barplot we can see the K Goel is the number one position highest bowling strike rate in IPL session from 2008 to 2019 Code snippet : tsr=bowler_data[:].sort_values(by='Strike_rate',ascending=False) bar_plot(tsr,'Bowler_name','Strike_rate','Bowler Names vs Top Strike Rate') Call the defined function to visualize the top 10 Player names Highest bowling Economy rate. In the Barplot we can see the K Goel is the number one position highest bowling Economy rate in IPL session from 2008 to 2019 Code snippet : ter=bowler_data[:].sort_values(by='Economy_rate',ascending=False) bar_plot(ter,'Bowler_name','Economy_rate','Bowler Names vs Top Economy Rate') We can see the graph most of the times Mumbai indians win the match in all IPL seasons and at the second and third position is chennai super kings and kolkata knight riders. Code snippet : plt.figure(figsize=(20,10)) ax = sns.countplot(x="winner", data=ipl_match) ax.set_title("Number of matches win",size = 15); plt.xticks(rotation=45,size=15); ax.set_xlabel('Teams',size = 15); ax.set_ylabel('Number of occurences',size = 15); plt.show() We can see in the graph that Virat kohli is in first position in making the highest runs. Code snippet : batsman_data = ipl_deliveries.groupby(['batsman']).sum().reset_index() best_batsman=batsman_data[:].sort_values(by='batsman_runs',ascending=False) bar_plot(best_batsman,'batsman','batsman_runs','Batsman Runs Vs Batsman Name') We can see in the graph that Chris Gayle is in first position for most of the time man of the match and at the second position is AB de Villiers. Code snippet : plt.figure(figsize=(20,10)) ax = sns.countplot(x="player_of_match", data=ipl_match ,order = ipl_match['player_of_match'].value_counts().index[:20:1]) ax.set_title("Top players Become man of the match",size = 15); plt.xticks(rotation=45,size=15); ax.set_xlabel('Teams',size = 15); ax.set_ylabel('Number of occurences',size = 15); plt.show() Thank You

In this Article we will perform the Real time Gender Detection using CNN. Let's start. Gender is our identity. In our social life it is also a significant element. Artificial intelligence gender recognition can be used in many fields, such as smart human-machine interface growth , health, cosmetics, electronic commerce etc. Recognition of people's gender from their facial images is an ongoing and active problem of research. The researchers suggested a number of methods to resolve this problem, but the criteria and actual performance are still inadequate. A statistical pattern recognition approach for solving this problem is proposed in this project. Convolutional Neural Network (ConvNet / CNN), a Deep Learning algorithm, is used as an extractor of features in the proposed solution. CNN takes input images and assigns value to different aspects / objects (learnable weights and biases) of the image and can differentiate between them. ConvNet requires much less preprocessing than other classification algorithms. While the filters are hand-made in primitive methods, ConvNets can learn these filters / features with adequate training.In this research, face images of individuals have been trained with convolutionary neural networks, and age and sex with a high rate of success have been predicted. More than 20,000 images contain age, gender and ethnicity annotations. The images cover a wide range of poses, facial expression, lighting, occlusion, and resolution. Introduction The aim is to predict the gender of individuals using image data sets. A growing number of applications, especially after the increase in social networks and social media, are being concerned with automatic gender classification. Age and gender are the two most fundamental facial qualities in social interaction. In smart applications, such as access control, human computer interaction, enforcement, marketing intelligence and visual supervision, etc, it is important to recognize gender using one facial image. In machine learning there are many algorithms that are the most common technologies used in this project. Convolutional Neural Network (CNN) is one of the most prevalent algorithms that has gained a high reputation in image feature extraction. Methodology Architecture Gender recognition using Convolutional neural network : A Convolutional Neural Network (ConvNet / CNN) is a Deep Learning algorithm, which allows an input image to take on different aspects/objects and can be distinguished from one image (learnable weights and biases). ConvNet requires much less preprocessing than other classification algorithms. While the filters are hand-made in primitive methods, ConvNets can learn these filters / features with adequate training. The ConvNet architecture is similar to that of neurons in the human brain and was influenced by the Visual Cortex organization. Within a limited area of the visual field known as the Receptive Field, only individual neurons respond to stimuli. The entire visual area is protected by a selection of these fields. About Dataset The data collection consists of more than 20,000 facial images with age, gender and ethnicity annotations. The images cover a wide range of poses, facial expression, lighting, occlusion, resolution .It can be used for a variety of tasks, for example face detection, age estimation, gender recognition, position of landmarks etc. The survey is focused on the age detection of the neural network (CNN) image dataset architecture. We need to split our dataset into three parts: training dataset, test dataset and validation dataset. The purpose of splitting data is to avoid overfitting which is paying attention to minor details/noise which is not necessary and only optimizes the training dataset accuracy. We need a model that performs well on a dataset that it has never seen (test data), which is called generalization. The training set is the actual subset of the dataset that we use to train the model. The model observes and learns from this data and then optimizes its parameters. The validation dataset is used to select hyperparameters (learning rate, regularization parameters). When the model is performing well enough on our validation dataset, we can stop learning using a training dataset. The test set is the remaining subset of data used to provide an unbiased evaluation of a final model fit on the training dataset. Description of Library used Numpy : Numpy is the most basic yet a powerful package for mathematical and scientific computing and data manipulation in python. It is an open source library available in python. Cv2: OpenCV is a high performance library for digital image processing and computer vision, which is free and open source. Matplotlib : Matplotlib is a plotting library for the python programming language and its numerical mathematics extensions in numpy. OS : The OS module in python provides a way of using operating system dependent functionality KERAS : Keras is an open-source high-level neural network API, written in python. It allows easy and fast prototyping Technology Image Processing : Image processing is a method to perform some operations on an image, in order to get an enhanced image or to extract some useful information from it. It is a type of signal processing in which input is an image and output may be image or characteristics/features associated with that image. Computer Vision : Computer vision is a field of artificial intelligence that trains computers to interpret and understand the visual world. Using digital images from cameras and videos and deep learning models, machines can accurately identify and classify objects. Tools Used Anaconda Navigator : Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing (data science, machine learning applications, large-scale data processing, predictive analytics, etc.), that aims to simplify package management and deployment. Jupyter Notebook IDE: Jupyter Notebook is an open source web-based application which allows you to create and share documents containing live code, equations, visualizations, and narrative text. The IDE also includes data cleaning and transformation, numerical simulation, statistical modelling, data visualization, and many others. Import libraries #import libraries import cv2,os import numpy as np from keras.utils import np_utils from keras.models import Sequential from keras.layers import Dense, Activation, Dropout, Conv2D, Flatten, MaxPooling2D from keras.callbacks import ModelCheckpoint from sklearn.model_selection import train_test_split from matplotlib import pyplot as plt from keras.models import load_model from random import shuffle from scipy import misc import imageio from keras.utils.np_utils import to_categorical Procedure First we are changing the current directory to the path where our image dataset is stored by using os.chdir( ) and then we get the list of all the files and directories in the specified path using os.listdir( ). #use the file path where your dataset is stored data_path = 'UTKFace' img_files = os.listdir(data_path) len(img_files) Shuffle( ) is used to randomize all the image files. shuffle(img_files) gender = [i.split('_')[1] for i in img_files] Using split( ) we get the gender of each image file and then store it into a list. target = [] for i in gender: i = int(i) target.append(i) Using cv2.imread( ) and cv2.resize( ), we read an image from each file as an array and resize its dimensions to 32x32 and then store it in a list named data. data=[] img_size=32 for img_name in img_files: img_path = os.path.join(data_path,img_name) img = cv2.imread(img_path) try: gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) resized = cv2.resize(gray,(img_size,img_size)) data.append(resized) except Exception as e: print("Exception: ",e) Next we normalize the data by converting the datatype of variable data to float32 and dividing it by 255. Convert the class vector(integers) to a binary class matrix using to_categorical() before training our model. #data values are normalized data = np.array(data)/255.0 # Reshaping of data data = np.reshape(data,(data.shape[0],img_size,img_size,1)) new_target = to_categorical(target, num_classes=2) # saving the file np.save('target',new_target) np.save('data',data) There are two ways to build keras models : sequential and functional. In our project we are using sequential by which we create a model layer-by-layer. When we use Conv2D as the first layer we must define the input shape. This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs. Activation is used for “relu” The MaxPooling2D layer is used for spatial data. Dropout layer is used to prevent a model from overfitting. Dense layer is a linear operation in which every input is connected to every output by a weight. # build Convolutional neural network leyar model = Sequential() model.add(Conv2D(100,(3,3),input_shape=data.shape[1:])) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(1,1))) model.add(Conv2D(100,(2,2))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(1,1))) model.add(Conv2D(100,(2,2))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(400,(2,2))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(1,1))) model.add(Flatten()) model.add(Dropout(0.3)) model.add(Dense(50,activation='relu')) model.add(Dense(2,activation='softmax')) Next we compile and fit the model. We summarize the model by using model.summary( ). Total params: 3,621,752, trainable params: 3,621,752 and Non-trainable params: 0. The total length of the dataset is 23708, out of which 90% is used for training, 10% for testing and 20% for validating the data. #Split the data train_data, test_data, train_target, test_target = train_test_split(data, target, test_size=0.1) We then evaluate the model on a test set to get the accuracy up to 0.88. Now the model is trained and it's ready to recognize the gender of any random image from a dataset. We plot a random sample of 15 test images, their predicted labels and ground truth. We get the output by displaying each image along with its title. In this project, Keras is used. Keras is an open-source neural network library. It is user friendly and provides several features such as activation.functions, layers, optimizers, etc. and it supports CNN . By using the appropriate class in Keras, deep learning models can be created on iOS and Android through the JVM (Java Virtual Machine). Keras enables the model to perform random transformations and normalization operations on batches of image data by working on different attributes such as height shift, width shift, rotation range, rescale, range of shear, range of zoom, horizontal rip and fill mode. Using these attributes the system can automatically rotate, translate, rescale, and zoom into or out of images, as well as apply shearing transformations, rip images horizontally, fill in newly created pixels, etc. For the purpose of image classification, ConvNet is used. Training Dataset : The training dataset is used as a set of examples used for training the model, i.e. to fit the different parameters. Validation Dataset : A validation dataset is used to fit the hyper-parameters of the classifier. A validation dataset is necessary because it helps in the reduction of overfitting. The validation dataset is independent of the training dataset. Test Dataset : The test dataset is used to test the performance of the classifier or model and to check the performance of characteristics such as accuracy, loss, sensitivity, etc. It is independent of the training and validation dataset. Real time Gender Recognition Steps to follow : Face detection with Haar cascade Gender Recognition with CNN 1. Face detection with Haar cascades : This is a part most of us at least have heard of. OpenCV/JavaCV provide direct methods to import Haarcascades and use them to detect faces. 2. Gender Recognition with CNN : CNN algorithm is used for Gender recognition. The CNN’s output layer (probability layer) in this CNN consists of two classes ‘Male’ and ‘Female’. img_size=32 #load the best model model = load_model('model-10.model') faceCascade=cv2.CascadeClassifier('haarcascade_frontalface_default.xml') #starts the webcam cap = cv2.VideoCapture('vp_debate.mp4') ## Label and Color labels_dict = {0:'Male',1:'Female'} color_dict = { 0:(0,0,255),1:(0,255,0)} while 1: ret,frame = cap.read() gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY) faces = faceCascade.detectMultiScale(gray,1.3,5) for x,y,w,h in faces: face_img = gray[y:y+w,x:x+h] resized = cv2.resize(face_img,(img_size,img_size)) normalized = resized/255.0 reshaped = np.reshape(normalized,(1,img_size,img_size,1)) result = model.predict(reshaped) label = np.argmax(result,axis=1)[0] cv2.rectangle(frame,(x,y),(x+w,y+h),color_dict[label],2) cv2.putText(frame,labels_dict[label],(x,y-10),cv2.FONT_HERSHEY_SIMPLEX,0.8,(255,255,255),2) cv2.imshow('Video',frame) key=cv2.waitKey(1) cap.release() cv2.destroyAllWindows() Conclusion we proposed a model to Classify the gender by feeding the CNN image dataset, a deep learning algorithm and trained in broad database face-recognition. In all, we think that the accuracy of the model is decent. but can be further improved by using more data, data increase and better network architecture.