Understanding Naive Bayes

by Apr 15, 2020Article

Naïve Bayes Classifier is machine learning model used to classify the object based on different features. The object or attribute that we are going to classify is also referred as dependent variable whereas the features that are used to predict the dependent variable is knows as independent variable (predictors).

Naïve Bayes classifier is a probabilistic model based on Bayes Theorem which states that:

Which means, calculating the probability of event y given that event X has already occurred. Here two assumptions are made to use it in Naïve Bayes.

The first assumption made here is that the predictors are independent from each other i.e. that one feature does not affect the presence of any other feature. That’s why it is called Naïve.

The second assumption made here is that the equal weight should be given to the predictors to make the prediction.

Let’s consider the below example to understand the implementation of theorem.

Here, we are trying to predict if the day is good for playing the golf or not given the features of the day. The row represents each day entry and columns represents the predictor features (except ‘PLAY GOLF’). Considering the first row, the day is not suitable for playing golf if the outlook is rainy, humidity is high, temperature is hot and it’s not windy. The first assumption for the Naïve Bayes is that the predictors are independent i.e. OUTLOOK is not dependent on TEMPRATURE, HUMIDITY, WINDY. Likewise, for other features. The second assumption is that the equal importance should be given to all the predictors.

From the above-mentioned Bayes Theorem, X can be considered as feature space and can be rewritten as below.

Where x1, x2, x3 …. xn are all the predictors (Here, OUTLOOK, TEMPRATURE, HUMIDITY, WINDY)

On substituting X into the original Bayes equation can be written as below-

Now all the values from the dataset can be substituted in the above equation. The denominator will remain constant for the given data, so it can be further simplified as below.

In our case, the class y has two outcomes and we need to select the class with the maximum probability, which can be written as:

This can be used to predict the class with the given features.

This algorithm is based on predictor independent assumption which hard to find the features in real world. It works well with multiclass prediction by providing the probability for each class. Due to these key points, this algorithm is mostly used in text classification / Spam filtering / Sentiment Analysis and have shown better results than other algorithms [1].

In Oracle, we can create Naïve Bayes Machine Learning model in Oracle Analytics Cloud as well as in ADW. OAC provides the drag and drop functionality and is quick to use for business users with less programming knowledge. Whereas ADW works directly on the database and is mostly used by IT team who has good knowledge of PL/SQL. In Oracle DB, Data Mining functions are defined to use different algorithms. To learn more on the usability of OAC and ADW, please refer to our video’s series.

Sources

  1. G. Chauhan, “All About Naive Bayes,” 2018. [Online]. Available: https://towardsdatascience.com/all-about-naive-bayes-8e13cef044cf.
  2. “GeeksforGeeks,” [Online]. Available: https://www.geeksforgeeks.org/naive-bayes-classifiers/.

 

Paritosh Gupta

Paritosh Gupta

Data Analytics Analyst

About the Author

Paritosh Gupta is Data Analytics Analyst at Vertice. He has 5+ years of experience in Data domain working with Data Extraction, Pre-processing, Data Analysis, ETL, developing and supporting reconciliation systems. He holds Master’s Degree in Data Analytics from Dublin City University and Bachelor of Engineering in Computer Science from Rajiv Gandhi Technical University.

Sign Up For Our Newsletter

* indicates required

Please select all the ways you would like to hear from Vertice.

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.