Probability = Given the distribution, what's the probability of data occuring.
Likelihood = Give the data, what's the likelihood that the distribution is correct? Or given data, what is the probability it belongs to a distribution
Bayes' Theorem: https://www.youtube.com/watch?v=R13BD8qKeTg
How is it different from Naive Bayes' Theorem?
Naive Bayes is considered “Naive” because it makes an assumption that is virtually impossible to see in real-life data: the conditional probability is calculated as the pure product of the individual probabilities of components. This implies the absolute independence of features — a condition probably never met in real life.
Lets take Naive Bayes on a intuition level to figure out what food tastes good together!
Naive Bayes will make the following assumption:
If you like Pickles, and you like Ice Cream, naive bayes will assume independence and give you a Pickle Ice Cream and think that you'll like it.
Another way to look: https://www.youtube.com/watch?v=YBvilAYd5sE
First: Bayes' Theorem:
It's all about prior information, how the prior information is crucial to the posterior or likelihood of occurence of an event.
Bayes' Theorem is something that updates the outcome in light of new information.
When new information comes to light, the prior probability of information changes.
So bayes' theorem is updating the probability of hypothesis occurs or is true given the new evidence
Bayes theorem is like a formula to tell how much you trust the evidence/event
Bayes' Theorem:
P(H|E) = Probability that a hypothesis is true given an event or evidence = Posterior Probability
P(H) = Probability that the hypothesis is true: Prior Probability = Probability before you had new information or evidence
P(E|H) = Probability of the event or evidence is seen given the hypothesis is true
Probability that Event E occurred given event/hypothesis H P(E|H) = likelihood of H given E = L(H|E)
P(E) = Probability that the event occurs or probability of seeing an evidence
Hypothesis is something you want to confirm given the event has occurred
Example:
Event occurred: A medical test showed that you have a disease but the test is not 100% accurate
Hypothesis: You actually have the disease
Prior information: How many people in population have the disease (a rough estimate of prior information)
After you do 1 test: Prior information: Probability you actually have the disease given the new infomation said so
previous posterior information now becomes prior information
P(H|E) = P(E|H) * P(H)
_____________
P(E)
or
P(H|E) = true pos / all positives
all positives = true pos + false pos
Say, you run a test and test positive for a disease
What information we have?
Disease affects 0.1% of the population
True Positive = 99% i.e. Test says disease / You have disease = 99%
And, Test says disease / You don't have disease (False Positive) = 1%
What are the chances you have the disease?
H = I have disease
E = Test says disease
P(E|-H) = 0.01
P(H) = 0.1% = 0.001 #Prior information about the occurance of disease in the population = You have disease
P(E|H) = 0.99
P(-H) = 0.999 = You don't have disease
P(E) = Test says disease =
Law of probability
Say you have apple and oranges in basket
P(Apple is Ripe in all fruits) = P(Aripe) = Naripe / N
P(Orange is Ripe in all fruits) = P(Oripe) = Noripe / N
P(Ripe) = P(Aripe) + P(Oripe)
P(R) = P(A ^ R) + P (O ^ R)
P(R|A) = P(R ^ A) / P(A)
P(A|R) = P(A ^ R) / P(R)
P(R ^ A) = P(R|A) * P(A)
P(O ^ A) = P(R|O) * P(O)
P(R) = P(R|A) * P(A) + P(R|O) * P(O)
Based on this:
P(E) = Test says disease = P(Test says disease ^ Has disease) + P(Test says disease ^ doesn't have disease)
= P(E^H) + P(E^-H)
= P(E|H) * P(H) + P(E|-H) * P(-H)
So,
P(H|E) = P(E|H) * P(H)
__________________________________
P(E|H) * P(H) + P(E|-H) * P(-H)
0.99 * 0.001
= _______________________
0.99*0.001+0.01*0.999
= 0.09 = 9%
So now you're 9% certain that you have disease
Now you want to further confirm it,
so you do another test (same test) and get positive again:
Now you use the prior probability as 9% and do the calculation, because the old information confirmed it
then you get 91% chance
Another way to look around it:
Another and most important thing is: It depends heavily on prior information
So, life lesson: just because you get highly negative outcome, doesn't always mean your outcome is bad, it might just mean that your prior was bad - your assumption was bad
https://www.youtube.com/watch?v=HZGCoVF3YvM
Another way of thinking about it is: Outcome is not only determined by or should be determined by an event, but a sequence of events or it should update prior information, not give its own info. we should look at prior info before making prediction
Naive Bayes Classifier:
Example:
