Probability and Information theory [Goodfellow Book]:
Quantifying uncertainity and then derviving new uncertain statements.
Information Theory:
Quantify the amount of uncertainity in a probability distribution.
Probability Theory:
Make uncertain statements and reason in presence of uncertainity.
Uncertain:
Means you don't know anything, doesn't have a probability distribution.
Stochastic (non-deterministic):
Means it changes in the way that are fully predictible.
Stochasticity = Uncertainity + Probability Distribution
Probability:
Frequentist Probability:
Repeated experiment 'inf' times, then, 40% of the experiments will have such.
Bayesian Probability:
But what is experiment not repeatable? Like diagnosing a patient with flu / finding if the sun has exploded.
Here we use probability as a degree of belief (Qualitative)
Quantifying uncertainity and then derviving new uncertain statements.
Information Theory:
Quantify the amount of uncertainity in a probability distribution.
Probability Theory:
Make uncertain statements and reason in presence of uncertainity.
Uncertain:
Means you don't know anything, doesn't have a probability distribution.
Stochastic (non-deterministic):
Means it changes in the way that are fully predictible.
Stochasticity = Uncertainity + Probability Distribution
Probability:
Frequentist Probability:
Repeated experiment 'inf' times, then, 40% of the experiments will have such.
Bayesian Probability:
But what is experiment not repeatable? Like diagnosing a patient with flu / finding if the sun has exploded.
Here we use probability as a degree of belief (Qualitative)