Probability Distributions

Random Variables

Random variables are variables that can take different values, and the value they take depends on a probabilistic process.

Examples:

Y=Resting heart rate Y=congestive heart failure status (1=Yes, 0=No)
80 0
82 1
78 0
90 1
65 0

Probability Distributions

A probability distribution is a function that maps a possible outcome of a random variable to its probability of occurrence.

Key Components of Probability Distributions

  • Support: values that the distribution can take

  • Parameters: values that characterize the distribution

  • PDF/PMF: \(f(x)\) , function whose output is a probability of a particular value or range of values

  • CDF: \(F(x)\) , function whose output is a cumulative probability up to a particular value

  • Expected value/mean: \(\mathbf{E}(X)\)

  • Variance: \(\mathbf{E}[(X-\mu)^2]\)

Example

\(X\) 1 2 3 4
\(P(X=x)\) 0.2 0.4 0.3 0.1

  • Is this a valid probability distribution?

  • Is this a discrete or continuous distribution?

  • What is the support?

  • What is \(P(X=3)\)?

  • What is \(F(3)\)?

  • What is \(\mathbf{E}(X)\)?

  • What is \(Var(X)\)?

Example answers

  • Is this a valid probability distribution? Yes - probabilities sum to 1

  • Is this a discrete or continuous distribution? Discrete

  • What is the support? {1,2,3,4}

  • What is \(P(X=3)\)? 0.3

  • What is \(F(3)\)? 0.2+0.4+0.3=0.9

Example answers

  • What is \(\mathbf{E}(X)\)?

    • \(\sum_x xP(X=x) = 1*0.2+2*0.4+3*0.3+4*0.1 = 2.3\)

  • What is \(Var(X)\)?

    • \(Var(X)=\mathbf{E}[(X-\mu)^2]=\mathbf{E}[X^2]-\mathbf{E}[X]^2\) (do this on your own)

    • \(\mathbf{E}[X]^2\) \(= (2.3)^2 = 5.29\)

    • \(\mathbf{E}[X^2]\) \(=\sum_x x^2 P(X=x)\) \(=1^2*0.2\) \(+2^2*0.4\) \(+3^2*0.3\) \(+4^2*0.1 = 6.1\)

    • \(Var(X)=6.1-5.29=0.81\)

Common Probability Distributions

  1. Normal
  2. Bernoulli
  3. Binomial
  4. Uniform
  5. Multinomial
  6. Poisson
  7. Negative Binomial
  8. Exponential