Taylor’s Formula

If \( f(x) = a_0 + a_1 x + a_2 x^2 + \cdots \),
$$
a_n = \frac{f^{(n)}(0)}{n!}
$$
with another base point
$$
f(x) = f(b) + f’(b)(x-b) + \frac{f’’(b)}{2} (x-b)^2 + \frac{ f^{(3)}(b)}{3!}(x-b)^3 + \cdots
$$
For example, \(\sqrt{x}\) is not appropriate to expand at \(0\) because it’s not differentiable at \(0\). so use \( b= 1\),
$$
x^{\frac{1}{2}} = 1 + \frac{1}{2}(x-1) + \frac{ \left( \frac{1}{2} \right) \left( \frac{1}{2} - 1 \right) }{2!} (x-1)^2 + \cdots
$$

likelihood function

어떤 probability distribution(parameter \(\theta\))에서 sample \(\mathbb x\)를 뽑을 확률을 \(p\)라고 하면,
$$
L(\theta | \mathbb x) = p (=p(\mathbb x | \theta) )
$$
이다. 예를들어 Bernoulli distribution(\(Bin(n,\pi)\))은 다음과 같이 정의되는데,
$$
f(x;\pi) = \pi ^ x (1-\pi)^{(1-x)} , x = 0 , 1, \pi \text{ is unknown parameter}
$$
likelihood function은 다음과 같다.
$$
L(\pi | x) = \frac{x!}{(n-x)!x!} \pi ^x (1-\pi)^{(n-x)}
$$
계산상 편의를 이유로 Log를 취한 log likelihood가 자주 쓰인다.

Gibbs sampling

1. initial value \(X^{(0)}\)
2. update \(j\)th component by \(p(x_j | x_1^{(i+1)}, \cdots , x_{j-1}^{(i+1)}, x_{j+1}^{(i)}, \cdots , x_n^{(i)})\)
3. repeat

(from wikipedia)


cf. block gibbs sampling
update  \(j\)th component by \(p(x_j | x_1^{(i)}, \cdots , x_{j-1}^{(i)}, x_{j+1}^{(i)}, \cdots , x_n^{(i)})\)