sampling method in detail

Basic

What it means to calcaulte the variance with mean 0?

\[ Var[X]= \int_{-\infty}^{\infty} x^2 f(x)dx \] when \(f(x)\) is \(N(1,2)\). Then \(f(x)\) is \[\frac{1}{\sqrt{2\pi\sigma^2}}\cdot\exp\left(-(x-\mu)^2/2\sigma^2\right)\] where \(\mu\) is 0 and \(\sigma\) is 2.

\[ Var[X]= \int_{-\infty}^{\infty} x^2 \frac{f(x)}{g(x)}g(x)dx \] where \(g(x)\) is N(0,3)+1

• Integrand of \(x\) along with density being integrated against \(f(x)\)

We are going to show the true mean, mean using simple monte carlo, and mean using importance sampling.

set.seed(1234)
mu = 0
sigma = 2
X = rnorm(1e5, mu, sigma)
X2 = X^2
mean(X2); var(X2)

## [1] 3.995663

## [1] 31.87845

exp(-1000)/(exp(-1000)+exp(-1001))

## [1] NaN

1/(1+exp(-1))

## [1] 0.7310586

Inverse Transform Sampling

If we know the exact form of the distribution, we can sample the points using the inverse CDF technique.

Rejection Sampling

Importance Sampling

• Not for generating samples. It is a method to estimate the expected value of a function directly