Random Variables

Formalisme untuk Variabel yang Nilainya Tidak Pasti

Why This Matters for Your Work

Random variable adalah jembatan antara ruang probabilitas abstrak ($\Omega$, events) dengan angka riil yang bisa kita hitung, plot, dan gunakan dalam model.

Setiap model statistik atau ML pada dasarnya adalah spesifikasi distribusi random variable: - OLS: $y_i = \mathbf{x}_i^T\beta + \varepsilon_i$ di mana $\varepsilon_i$ adalah random variable dengan distribusi tertentu - Logistic regression: $Y_i | \mathbf{x}_i \sim \text{Bernoulli}(\sigma(\mathbf{x}_i^T\beta))$ - Poisson regression: $Y_i | \mathbf{x}_i \sim \text{Poisson}(\exp(\mathbf{x}_i^T\beta))$ - Neural network: layer outputs adalah random variables (di bawah Bayesian interpretation)

Memahami expectation, variance, covariance secara formal memungkinkan kita menjawab: adalah estimator kita unbiased? Seberapa besar variabilitasnya? Bagaimana error propagates?

1 1. Definisi Formal

Definisi: Random Variable

Random variable $X$ adalah fungsi dari sample space $\Omega$ ke bilangan real $\mathbb{R}$: \[X: \Omega \to \mathbb{R}\]

Untuk setiap outcome $\omega \in \Omega$, $X(\omega)$ adalah nilai numerik yang terassigned.

Contoh: Lempar dua koin, $\Omega = \{HH, HT, TH, TT\}$. Define $X$ = jumlah kepala: - $X(HH) = 2$ - $X(HT) = X(TH) = 1$ - $X(TT) = 0$

$X$ “mengubah” outcomes yang abstrak menjadi angka. Kita bisa tulis $P(X = 1) = P(\{HT, TH\}) = 2/4 = 0.5$.

Discrete vs Continuous: - Discrete: $X$ mengambil nilai dalam set countable $\{x_1, x_2, \ldots\}$ - Continuous: $X$ bisa mengambil nilai dalam interval kontinu (misalnya $\mathbb{R}$ atau subset-nya)

2 2. Discrete Random Variables

2.1 Probability Mass Function (PMF)

Definisi: PMF

Probability Mass Function (PMF) dari discrete RV $X$: \[p(x) = P(X = x) = P(\{\omega \in \Omega : X(\omega) = x\})\]

Properti: 1. $p(x) \geq 0$ untuk semua $x$ 2. $\sum_{x} p(x) = 1$ (sum atas semua nilai yang mungkin)

2.2 Cumulative Distribution Function (CDF)

CDF memberikan probabilitas $X$ at most $x$: \[F(x) = P(X \leq x) = \sum_{t \leq x} p(t)\]

Properti CDF (berlaku untuk discrete dan continuous): - $F$ monotone non-decreasing - $\lim_{x \to -\infty} F(x) = 0$ dan $\lim_{x \to \infty} F(x) = 1$ - Right-continuous: $\lim_{h \to 0^+} F(x + h) = F(x)$

2.3 Expected Value (Expectation, Mean)

Definisi: Expected Value (Diskrit)

\[E[X] = \mu_X = \sum_{x} x \cdot p(x)\]

Interpretasi: rata-rata tertimbang dari semua nilai yang mungkin, dengan bobot probabilitasnya.

LOTUS (Law of the Unconscious Statistician): Untuk fungsi $g(X)$: \[E[g(X)] = \sum_x g(x) \cdot p(x)\]

Kita tidak perlu tahu distribusi dari $g(X)$ untuk menghitung expected value-nya!

2.4 Variance

Definisi: Variance

\[\text{Var}(X) = E[(X - \mu_X)^2] = \sum_x (x - \mu_X)^2 p(x)\]

Computational formula (lebih mudah dihitung): \[\text{Var}(X) = E[X^2] - (E[X])^2\]

Standard deviation: $\text{SD}(X) = \sqrt{\text{Var}(X)}$

Derivasi computational formula: \[E[(X-\mu)^2] = E[X^2 - 2\mu X + \mu^2] = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - \mu^2 \quad \square\]

3 3. Continuous Random Variables

3.1 Probability Density Function (PDF)

Definisi: PDF

Random variable $X$ continuous jika ada fungsi $f: \mathbb{R} \to \mathbb{R}_{\geq 0}$ (disebut PDF) sehingga untuk semua $a \leq b$:

\[P(a \leq X \leq b) = \int_a^b f(x) \, dx\]

Properti: 1. $f(x) \geq 0$ untuk semua $x$ 2. $\int_{-\infty}^{\infty} f(x) \, dx = 1$

Perhatian penting: $f(x)$ BUKAN probabilitas! $f(x)$ bisa $> 1$. Yang merupakan probabilitas adalah integral dari $f$.

Untuk continuous RV: $P(X = x) = \int_x^x f(t)dt = 0$. Probabilitas di tepat satu titik adalah nol. Jadi: \[P(a \leq X \leq b) = P(a < X \leq b) = P(a \leq X < b) = P(a < X < b)\]

3.2 CDF untuk Continuous RV

\[F(x) = P(X \leq x) = \int_{-\infty}^x f(t) \, dt\]

Dengan Fundamental Theorem of Calculus: \[f(x) = F'(x) \quad \text{(di mana } F \text{ differentiable)}\]

3.3 Expected Value dan Variance (Continuous)

\[E[X] = \int_{-\infty}^{\infty} x f(x) \, dx\]

\[E[g(X)] = \int_{-\infty}^{\infty} g(x) f(x) \, dx \quad \text{(LOTUS)}\]

\[\text{Var}(X) = \int_{-\infty}^{\infty} (x - \mu)^2 f(x) \, dx = E[X^2] - (E[X])^2\]

4 4. Properties of Expectation

Ini adalah properties yang paling sering digunakan:

Definisi: Linearity of Expectation

Untuk konstanta $a, b$ dan random variables $X, Y$:

\[E[aX + b] = a E[X] + b\]

\[E[X + Y] = E[X] + E[Y]\]

Ini berlaku selalu, tanpa syarat apapun — bahkan jika $X$ dan $Y$ tidak independent!

Ini adalah satu dari hasil paling powerful dalam probabilitas. Kita bisa hitung expected value dari sum yang kompleks dengan hanya menjumlahkan expected values individual.

Property lain: - $E[c] = c$ untuk konstanta $c$ - $E[XY] = E[X]E[Y]$ HANYA JIKA $X \perp Y$ (independent) - Secara umum: $E[XY] = E[X]E[Y] + \text{Cov}(X,Y)$

Contoh penggunaan linearity: Dalam OLS, $\hat{\beta} = (X^TX)^{-1}X^Ty$ dan $y = X\beta + \varepsilon$. Maka: \[E[\hat{\beta}] = E[(X^TX)^{-1}X^T(X\beta + \varepsilon)] = \beta + (X^TX)^{-1}X^T E[\varepsilon] = \beta\] jika $E[\varepsilon | X] = 0$. Ini adalah unbiasedness OLS!

5 5. Properties of Variance

Definisi: Variance Properties

Untuk konstanta $a, b$ dan random variables $X, Y$:

\[\text{Var}(aX + b) = a^2 \text{Var}(X)\]

\[\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y)\]

Jika $X \perp Y$: $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

Perhatikan: $b$ tidak mempengaruhi variance (konstanta tidak menambah variabilitas). Dan kalau $X, Y$ independent, variance bisa dijumlahkan (tanpa factor 2).

Generalisasi: Untuk $X_1, \ldots, X_n$ iid (independent and identically distributed): \[\text{Var}\left(\sum_{i=1}^n X_i\right) = n \cdot \text{Var}(X_1)\] \[\text{Var}(\bar{X}) = \text{Var}\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{\text{Var}(X_1)}{n}\]

Ini adalah kenapa standard error dari sample mean adalah $\sigma/\sqrt{n}$.

6 6. Covariance & Correlation

Definisi: Covariance

\[\text{Cov}(X, Y) = E[(X - \mu_X)(Y - \mu_Y)] = E[XY] - E[X]E[Y]\]

Computational formula: $\text{Cov}(X,Y) = E[XY] - E[X]E[Y]$

Sifat: - $\text{Cov}(X, X) = \text{Var}(X)$ - $\text{Cov}(X, Y) = \text{Cov}(Y, X)$ (simetris) - $\text{Cov}(aX + b, Y) = a \cdot \text{Cov}(X, Y)$ - $\text{Cov}(X + Y, Z) = \text{Cov}(X, Z) + \text{Cov}(Y, Z)$

Definisi: Correlation

\[\rho(X, Y) = \text{Corr}(X, Y) = \frac{\text{Cov}(X, Y)}{\text{SD}(X) \cdot \text{SD}(Y)}\]

Properti penting: $\rho(X, Y) \in [-1, 1]$ - $\rho = 1$: perfect positive linear relationship - $\rho = -1$: perfect negative linear relationship - $\rho = 0$: no linear relationship (tapi bisa ada nonlinear relationship!)

Hubungan independence dan correlation: - Independence $\Rightarrow$ uncorrelated ($\rho = 0$) - Uncorrelated $\not\Rightarrow$ independent (contoh: $X \sim U(-1,1)$, $Y = X^2$. Mereka uncorrelated tapi sangat dependent!)

7 7. Worked Example 1: Discrete RV — Jumlah Keberhasilan

Setup: Lempar koin biased dengan $P(H) = p = 0.6$, sebanyak $n = 3$ kali. $X$ = jumlah kepala.

PMF: \[p(k) = \binom{3}{k} p^k (1-p)^{3-k}, \quad k = 0, 1, 2, 3\]

$k$	$\binom{3}{k}$	$p^k$	$(1-p)^{3-k}$	$p(k)$
0	1	1	0.064	0.064
1	3	0.6	0.16	0.288
2	3	0.36	0.4	0.432
3	1	0.216	1	0.216

Expected value: \[E[X] = 0(0.064) + 1(0.288) + 2(0.432) + 3(0.216) = 0 + 0.288 + 0.864 + 0.648 = 1.8\]

Check: $E[X] = np = 3 \times 0.6 = 1.8$ ✓

Variance: \[E[X^2] = 0^2(0.064) + 1^2(0.288) + 2^2(0.432) + 3^2(0.216) = 0 + 0.288 + 1.728 + 1.944 = 3.96\] \[\text{Var}(X) = E[X^2] - (E[X])^2 = 3.96 - 1.8^2 = 3.96 - 3.24 = 0.72\]

Check: $\text{Var}(X) = np(1-p) = 3 \times 0.6 \times 0.4 = 0.72$ ✓

8 8. Worked Example 2: Continuous RV — Exponential Distribution

Setup: $X \sim \text{Exponential}(\lambda)$ dengan $\lambda > 0$.

PDF: $f(x) = \lambda e^{-\lambda x}$ untuk $x > 0$ (nol untuk $x \leq 0$).

Verifikasi: $\int_0^\infty \lambda e^{-\lambda x} dx = \lambda \cdot \frac{1}{\lambda} = 1$ ✓

Expected value (integrasi by parts: $u = x$, $dv = \lambda e^{-\lambda x}dx$): \[E[X] = \int_0^\infty x \cdot \lambda e^{-\lambda x} dx = \left[-xe^{-\lambda x}\right]_0^\infty + \int_0^\infty e^{-\lambda x} dx = 0 + \frac{1}{\lambda} = \frac{1}{\lambda}\]

Second moment: \[E[X^2] = \int_0^\infty x^2 \lambda e^{-\lambda x} dx = \frac{2}{\lambda^2}\]

(Gunakan integration by parts dua kali, atau Gamma function identity: $\int_0^\infty x^n e^{-\lambda x}dx = n!/\lambda^{n+1}$.)

Variance: \[\text{Var}(X) = E[X^2] - (E[X])^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2}\]

Jadi untuk Exponential($\lambda$): mean = $1/\lambda$ dan variance = $1/\lambda^2$.

lambda <- 2  # rate parameter

# Theoretical values
cat("Theoretical E[X]:", 1/lambda, "\n")

Theoretical E[X]: 0.5

cat("Theoretical Var(X):", 1/lambda^2, "\n")

Theoretical Var(X): 0.25

# Simulate to verify
set.seed(42)
x_sim <- rexp(100000, rate = lambda)
cat("\nSimulated E[X]:", round(mean(x_sim), 4), "\n")


Simulated E[X]: 0.4991

cat("Simulated Var(X):", round(var(x_sim), 4), "\n")

Simulated Var(X): 0.2503

# LOTUS: compute E[X^2]
cat("\nE[X^2] via LOTUS (simulation):", round(mean(x_sim^2), 4))


E[X^2] via LOTUS (simulation): 0.4994

cat("\nE[X^2] theoretical (2/lambda^2):", 2/lambda^2)


E[X^2] theoretical (2/lambda^2): 0.5

# CDF check
cat("\n\nP(X <= 1) theoretical:", pexp(1, rate=lambda))



P(X <= 1) theoretical: 0.8646647

cat("\nP(X <= 1) from integration:",
    integrate(function(x) lambda * exp(-lambda*x), 0, 1)$value)


P(X <= 1) from integration: 0.8646647

Connection: OLS Estimator sebagai Random Variable

Ini adalah salah satu aplikasi paling penting dari framework random variable dalam econometrics.

Model: $\mathbf{y} = X\boldsymbol{\beta} + \boldsymbol{\varepsilon}$, di mana $\boldsymbol{\varepsilon}$ adalah random vector.

OLS estimator $\hat{\boldsymbol{\beta}} = (X^TX)^{-1}X^T\mathbf{y}$ adalah random variable (lebih tepatnya, random vector) — karena ia adalah fungsi dari $\mathbf{y}$ yang random.

Dengan asumsi $E[\boldsymbol{\varepsilon}|X] = \mathbf{0}$: \[E[\hat{\boldsymbol{\beta}}|X] = E[(X^TX)^{-1}X^T(X\boldsymbol{\beta} + \boldsymbol{\varepsilon})|X]\] \[= (X^TX)^{-1}X^TX\boldsymbol{\beta} + (X^TX)^{-1}X^T E[\boldsymbol{\varepsilon}|X]\] \[= \boldsymbol{\beta} + \mathbf{0} = \boldsymbol{\beta} \quad \checkmark \text{ (unbiased)}\]

Dengan asumsi $\text{Var}(\boldsymbol{\varepsilon}|X) = \sigma^2 I$: \[\text{Var}(\hat{\boldsymbol{\beta}}|X) = (X^TX)^{-1}X^T \cdot \sigma^2 I \cdot X(X^TX)^{-1} = \sigma^2(X^TX)^{-1}\]

Ini adalah covariance matrix dari OLS estimator — dari sinilah standard errors diturunkan!

# Discrete RV: berapa E[X] dan Var(X)?
x_vals <- 0:5
probs <- dpois(x_vals, lambda = 2)  # Poisson(2)

# E[X]
ex <- sum(x_vals * probs)

# E[X^2]
ex2 <- sum(x_vals^2 * probs)

# Var(X)
var_x <- ex2 - ex^2

cat("Poisson(lambda=2):\n")

Poisson(lambda=2):

cat("E[X] =", round(ex, 4), "(theoretical:", 2, ")\n")

E[X] = 1.8947 (theoretical: 2 )

cat("Var(X) =", round(var_x, 4), "(theoretical:", 2, ")\n")

Var(X) = 1.7333 (theoretical: 2 )

# Continuous: integrate manually
cat("\nNormal(mu=3, sigma^2=4):\n")


Normal(mu=3, sigma^2=4):

f_normal <- function(x) dnorm(x, mean=3, sd=2)

# E[X]
ex_norm <- integrate(function(x) x * f_normal(x), -Inf, Inf)$value
cat("E[X] =", round(ex_norm, 4), "\n")

E[X] = 3

# Var(X)
ex2_norm <- integrate(function(x) x^2 * f_normal(x), -Inf, Inf)$value
var_norm <- ex2_norm - ex_norm^2
cat("Var(X) =", round(var_norm, 4), "\n")

Var(X) = 4

Practice Problems

Soal 1: Misalkan $X$ punya PMF: $P(X=1) = 0.2$, $P(X=2) = 0.5$, $P(X=3) = 0.3$. Hitung $E[X]$, $E[X^2]$, $\text{Var}(X)$, dan $\text{SD}(X)$.

Solusi 1: - $E[X] = 1(0.2) + 2(0.5) + 3(0.3) = 0.2 + 1.0 + 0.9 = 2.1$ - $E[X^2] = 1(0.2) + 4(0.5) + 9(0.3) = 0.2 + 2.0 + 2.7 = 4.9$ - $\text{Var}(X) = 4.9 - 2.1^2 = 4.9 - 4.41 = 0.49$ - $\text{SD}(X) = \sqrt{0.49} = 0.7$

Soal 2: Untuk $X \sim \text{Uniform}(a, b)$ dengan PDF $f(x) = 1/(b-a)$ pada $[a,b]$, buktikan bahwa $E[X] = (a+b)/2$ dan $\text{Var}(X) = (b-a)^2/12$.

Solusi 2: \[E[X] = \int_a^b \frac{x}{b-a}dx = \frac{1}{b-a}\cdot\frac{b^2-a^2}{2} = \frac{(b+a)(b-a)}{2(b-a)} = \frac{a+b}{2}\]

\[E[X^2] = \int_a^b \frac{x^2}{b-a}dx = \frac{b^3-a^3}{3(b-a)} = \frac{a^2+ab+b^2}{3}\]

\[\text{Var}(X) = \frac{a^2+ab+b^2}{3} - \left(\frac{a+b}{2}\right)^2 = \frac{a^2+ab+b^2}{3} - \frac{a^2+2ab+b^2}{4} = \frac{(b-a)^2}{12}\]

Soal 3: Jika $X$ dan $Y$ independent dengan $E[X] = 2$, $\text{Var}(X) = 4$, $E[Y] = 3$, $\text{Var}(Y) = 9$. Hitung: (a) $E[2X + 3Y - 1]$, (b) $\text{Var}(2X - Y)$, (c) $\text{Cov}(X+Y, X-Y)$.

Solusi 3: - (a) $E[2X + 3Y - 1] = 2(2) + 3(3) - 1 = 4 + 9 - 1 = 12$ - (b) $\text{Var}(2X - Y) = 4\text{Var}(X) + \text{Var}(Y) = 4(4) + 9 = 25$ - (c) $\text{Cov}(X+Y, X-Y) = \text{Cov}(X,X) - \text{Cov}(X,Y) + \text{Cov}(Y,X) - \text{Cov}(Y,Y)$ $= \text{Var}(X) - 0 + 0 - \text{Var}(Y) = 4 - 9 = -5$

9 Ringkasan

Konsep	Discrete	Continuous
Distribusi	PMF: $p(x) = P(X=x)$	PDF: $f(x)$, $P(a\leq X\leq b)=\int_a^b f$
CDF	$F(x) = \sum_{t\leq x} p(t)$	$F(x) = \int_{-\infty}^x f(t)dt$
Expectation	$E[X] = \sum_x x\,p(x)$	$E[X] = \int x\,f(x)dx$
Variance	$\text{Var}(X) = E[X^2] - (E[X])^2$	sama
LOTUS	$E[g(X)] = \sum_x g(x)p(x)$	$E[g(X)] = \int g(x)f(x)dx$

Key results: - Linearity of expectation: $E[X+Y] = E[X] + E[Y]$ (always) - Independence $\Rightarrow$ uncorrelated, tapi tidak sebaliknya - $\text{Var}(\bar{X}) = \sigma^2/n$ untuk iid sample

Next: Key Probability Distributions →

--- title: "Random Variables" subtitle: "Formalisme untuk Variabel yang Nilainya Tidak Pasti" format: html: toc: true toc-depth: 3 code-fold: false --- ::: {.callout-note title="Why This Matters for Your Work"} Random variable adalah **jembatan** antara ruang probabilitas abstrak ($\Omega$, events) dengan angka riil yang bisa kita hitung, plot, dan gunakan dalam model. Setiap model statistik atau ML pada dasarnya adalah spesifikasi distribusi random variable: - **OLS**: $y_i = \mathbf{x}_i^T\beta + \varepsilon_i$ di mana $\varepsilon_i$ adalah random variable dengan distribusi tertentu - **Logistic regression**: $Y_i | \mathbf{x}_i \sim \text{Bernoulli}(\sigma(\mathbf{x}_i^T\beta))$ - **Poisson regression**: $Y_i | \mathbf{x}_i \sim \text{Poisson}(\exp(\mathbf{x}_i^T\beta))$ - **Neural network**: layer outputs adalah random variables (di bawah Bayesian interpretation) Memahami expectation, variance, covariance secara formal memungkinkan kita menjawab: **adalah estimator kita unbiased? Seberapa besar variabilitasnya? Bagaimana error propagates?** ::: --- ## 1. Definisi Formal ::: {.callout-important title="Definisi: Random Variable"} **Random variable** $X$ adalah fungsi dari sample space $\Omega$ ke bilangan real $\mathbb{R}$: $$X: \Omega \to \mathbb{R}$$ Untuk setiap outcome $\omega \in \Omega$, $X(\omega)$ adalah nilai numerik yang terassigned. ::: **Contoh**: Lempar dua koin, $\Omega = \{HH, HT, TH, TT\}$. Define $X$ = jumlah kepala: - $X(HH) = 2$ - $X(HT) = X(TH) = 1$ - $X(TT) = 0$ $X$ "mengubah" outcomes yang abstrak menjadi angka. Kita bisa tulis $P(X = 1) = P(\{HT, TH\}) = 2/4 = 0.5$. **Discrete vs Continuous**: - **Discrete**: $X$ mengambil nilai dalam set countable $\{x_1, x_2, \ldots\}$ - **Continuous**: $X$ bisa mengambil nilai dalam interval kontinu (misalnya $\mathbb{R}$ atau subset-nya) --- ## 2. Discrete Random Variables ### Probability Mass Function (PMF) ::: {.callout-important title="Definisi: PMF"} **Probability Mass Function (PMF)** dari discrete RV $X$: $$p(x) = P(X = x) = P(\{\omega \in \Omega : X(\omega) = x\})$$ Properti: 1. $p(x) \geq 0$ untuk semua $x$ 2. $\sum_{x} p(x) = 1$ (sum atas semua nilai yang mungkin) ::: ### Cumulative Distribution Function (CDF) **CDF** memberikan probabilitas $X$ at most $x$: $$F(x) = P(X \leq x) = \sum_{t \leq x} p(t)$$ Properti CDF (berlaku untuk discrete dan continuous): - $F$ monotone non-decreasing - $\lim_{x \to -\infty} F(x) = 0$ dan $\lim_{x \to \infty} F(x) = 1$ - Right-continuous: $\lim_{h \to 0^+} F(x + h) = F(x)$ ### Expected Value (Expectation, Mean) ::: {.callout-important title="Definisi: Expected Value (Diskrit)"} $$E[X] = \mu_X = \sum_{x} x \cdot p(x)$$ Interpretasi: rata-rata *tertimbang* dari semua nilai yang mungkin, dengan bobot probabilitasnya. ::: **LOTUS (Law of the Unconscious Statistician)**: Untuk fungsi $g(X)$: $$E[g(X)] = \sum_x g(x) \cdot p(x)$$ Kita tidak perlu tahu distribusi dari $g(X)$ untuk menghitung expected value-nya! ### Variance ::: {.callout-important title="Definisi: Variance"} $$\text{Var}(X) = E[(X - \mu_X)^2] = \sum_x (x - \mu_X)^2 p(x)$$ **Computational formula** (lebih mudah dihitung): $$\text{Var}(X) = E[X^2] - (E[X])^2$$ Standard deviation: $\text{SD}(X) = \sqrt{\text{Var}(X)}$ ::: *Derivasi computational formula*: $$E[(X-\mu)^2] = E[X^2 - 2\mu X + \mu^2] = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - \mu^2 \quad \square$$ --- ## 3. Continuous Random Variables ### Probability Density Function (PDF) ::: {.callout-important title="Definisi: PDF"} Random variable $X$ **continuous** jika ada fungsi $f: \mathbb{R} \to \mathbb{R}_{\geq 0}$ (disebut **PDF**) sehingga untuk semua $a \leq b$: $$P(a \leq X \leq b) = \int_a^b f(x) \, dx$$ Properti: 1. $f(x) \geq 0$ untuk semua $x$ 2. $\int_{-\infty}^{\infty} f(x) \, dx = 1$ ::: **Perhatian penting**: $f(x)$ BUKAN probabilitas! $f(x)$ bisa $> 1$. Yang merupakan probabilitas adalah *integral* dari $f$. Untuk continuous RV: $P(X = x) = \int_x^x f(t)dt = 0$. Probabilitas di tepat satu titik adalah nol. Jadi: $$P(a \leq X \leq b) = P(a < X \leq b) = P(a \leq X < b) = P(a < X < b)$$ ### CDF untuk Continuous RV $$F(x) = P(X \leq x) = \int_{-\infty}^x f(t) \, dt$$ Dengan Fundamental Theorem of Calculus: $$f(x) = F'(x) \quad \text{(di mana } F \text{ differentiable)}$$ ### Expected Value dan Variance (Continuous) $$E[X] = \int_{-\infty}^{\infty} x f(x) \, dx$$ $$E[g(X)] = \int_{-\infty}^{\infty} g(x) f(x) \, dx \quad \text{(LOTUS)}$$ $$\text{Var}(X) = \int_{-\infty}^{\infty} (x - \mu)^2 f(x) \, dx = E[X^2] - (E[X])^2$$ --- ## 4. Properties of Expectation Ini adalah properties yang paling sering digunakan: ::: {.callout-important title="Definisi: Linearity of Expectation"} Untuk konstanta $a, b$ dan random variables $X, Y$: $$E[aX + b] = a E[X] + b$$ $$E[X + Y] = E[X] + E[Y]$$ **Ini berlaku selalu, tanpa syarat apapun — bahkan jika $X$ dan $Y$ tidak independent!** ::: Ini adalah satu dari hasil paling powerful dalam probabilitas. Kita bisa hitung expected value dari sum yang kompleks dengan hanya menjumlahkan expected values individual. **Property lain**: - $E[c] = c$ untuk konstanta $c$ - $E[XY] = E[X]E[Y]$ HANYA JIKA $X \perp Y$ (independent) - Secara umum: $E[XY] = E[X]E[Y] + \text{Cov}(X,Y)$ **Contoh penggunaan linearity**: Dalam OLS, $\hat{\beta} = (X^TX)^{-1}X^Ty$ dan $y = X\beta + \varepsilon$. Maka: $$E[\hat{\beta}] = E[(X^TX)^{-1}X^T(X\beta + \varepsilon)] = \beta + (X^TX)^{-1}X^T E[\varepsilon] = \beta$$ jika $E[\varepsilon | X] = 0$. Ini adalah **unbiasedness** OLS! --- ## 5. Properties of Variance ::: {.callout-important title="Definisi: Variance Properties"} Untuk konstanta $a, b$ dan random variables $X, Y$: $$\text{Var}(aX + b) = a^2 \text{Var}(X)$$ $$\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y)$$ Jika $X \perp Y$: $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$ ::: **Perhatikan**: $b$ tidak mempengaruhi variance (konstanta tidak menambah variabilitas). Dan kalau $X, Y$ independent, variance bisa dijumlahkan (tanpa factor 2). **Generalisasi**: Untuk $X_1, \ldots, X_n$ iid (independent and identically distributed): $$\text{Var}\left(\sum_{i=1}^n X_i\right) = n \cdot \text{Var}(X_1)$$ $$\text{Var}(\bar{X}) = \text{Var}\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{\text{Var}(X_1)}{n}$$ Ini adalah kenapa **standard error** dari sample mean adalah $\sigma/\sqrt{n}$. --- ## 6. Covariance & Correlation ::: {.callout-important title="Definisi: Covariance"} $$\text{Cov}(X, Y) = E[(X - \mu_X)(Y - \mu_Y)] = E[XY] - E[X]E[Y]$$ **Computational formula**: $\text{Cov}(X,Y) = E[XY] - E[X]E[Y]$ Sifat: - $\text{Cov}(X, X) = \text{Var}(X)$ - $\text{Cov}(X, Y) = \text{Cov}(Y, X)$ (simetris) - $\text{Cov}(aX + b, Y) = a \cdot \text{Cov}(X, Y)$ - $\text{Cov}(X + Y, Z) = \text{Cov}(X, Z) + \text{Cov}(Y, Z)$ ::: ::: {.callout-important title="Definisi: Correlation"} $$\rho(X, Y) = \text{Corr}(X, Y) = \frac{\text{Cov}(X, Y)}{\text{SD}(X) \cdot \text{SD}(Y)}$$ Properti penting: $\rho(X, Y) \in [-1, 1]$ - $\rho = 1$: perfect positive linear relationship - $\rho = -1$: perfect negative linear relationship - $\rho = 0$: no linear relationship (tapi bisa ada nonlinear relationship!) ::: **Hubungan independence dan correlation**: - Independence $\Rightarrow$ uncorrelated ($\rho = 0$) - Uncorrelated $\not\Rightarrow$ independent (contoh: $X \sim U(-1,1)$, $Y = X^2$. Mereka uncorrelated tapi sangat dependent!) --- ## 7. Worked Example 1: Discrete RV — Jumlah Keberhasilan **Setup**: Lempar koin biased dengan $P(H) = p = 0.6$, sebanyak $n = 3$ kali. $X$ = jumlah kepala. PMF: $$p(k) = \binom{3}{k} p^k (1-p)^{3-k}, \quad k = 0, 1, 2, 3$$ | $k$ | $\binom{3}{k}$ | $p^k$ | $(1-p)^{3-k}$ | $p(k)$ | |-----|----------------|-------|----------------|--------| | 0 | 1 | 1 | 0.064 | 0.064 | | 1 | 3 | 0.6 | 0.16 | 0.288 | | 2 | 3 | 0.36 | 0.4 | 0.432 | | 3 | 1 | 0.216 | 1 | 0.216 | **Expected value**: $$E[X] = 0(0.064) + 1(0.288) + 2(0.432) + 3(0.216) = 0 + 0.288 + 0.864 + 0.648 = 1.8$$ Check: $E[X] = np = 3 \times 0.6 = 1.8$ ✓ **Variance**: $$E[X^2] = 0^2(0.064) + 1^2(0.288) + 2^2(0.432) + 3^2(0.216) = 0 + 0.288 + 1.728 + 1.944 = 3.96$$ $$\text{Var}(X) = E[X^2] - (E[X])^2 = 3.96 - 1.8^2 = 3.96 - 3.24 = 0.72$$ Check: $\text{Var}(X) = np(1-p) = 3 \times 0.6 \times 0.4 = 0.72$ ✓ --- ## 8. Worked Example 2: Continuous RV — Exponential Distribution **Setup**: $X \sim \text{Exponential}(\lambda)$ dengan $\lambda > 0$. PDF: $f(x) = \lambda e^{-\lambda x}$ untuk $x > 0$ (nol untuk $x \leq 0$). **Verifikasi**: $\int_0^\infty \lambda e^{-\lambda x} dx = \lambda \cdot \frac{1}{\lambda} = 1$ ✓ **Expected value** (integrasi by parts: $u = x$, $dv = \lambda e^{-\lambda x}dx$): $$E[X] = \int_0^\infty x \cdot \lambda e^{-\lambda x} dx = \left[-xe^{-\lambda x}\right]_0^\infty + \int_0^\infty e^{-\lambda x} dx = 0 + \frac{1}{\lambda} = \frac{1}{\lambda}$$ **Second moment**: $$E[X^2] = \int_0^\infty x^2 \lambda e^{-\lambda x} dx = \frac{2}{\lambda^2}$$ (Gunakan integration by parts dua kali, atau Gamma function identity: $\int_0^\infty x^n e^{-\lambda x}dx = n!/\lambda^{n+1}$.) **Variance**: $$\text{Var}(X) = E[X^2] - (E[X])^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \frac{1}{\lambda^2}$$ Jadi untuk Exponential($\lambda$): **mean = $1/\lambda$** dan **variance = $1/\lambda^2$**. ```{r} #| label: exponential-rv #| code-summary: "R: Moments of Exponential distribution" lambda <- 2 # rate parameter # Theoretical values cat("Theoretical E[X]:", 1/lambda, "\n") cat("Theoretical Var(X):", 1/lambda^2, "\n") # Simulate to verify set.seed(42) x_sim <- rexp(100000, rate = lambda) cat("\nSimulated E[X]:", round(mean(x_sim), 4), "\n") cat("Simulated Var(X):", round(var(x_sim), 4), "\n") # LOTUS: compute E[X^2] cat("\nE[X^2] via LOTUS (simulation):", round(mean(x_sim^2), 4)) cat("\nE[X^2] theoretical (2/lambda^2):", 2/lambda^2) # CDF check cat("\n\nP(X <= 1) theoretical:", pexp(1, rate=lambda)) cat("\nP(X <= 1) from integration:", integrate(function(x) lambda * exp(-lambda*x), 0, 1)$value) ``` --- ::: {.callout-caution title="Connection: OLS Estimator sebagai Random Variable"} Ini adalah salah satu aplikasi paling penting dari framework random variable dalam econometrics. Model: $\mathbf{y} = X\boldsymbol{\beta} + \boldsymbol{\varepsilon}$, di mana $\boldsymbol{\varepsilon}$ adalah random vector. **OLS estimator** $\hat{\boldsymbol{\beta}} = (X^TX)^{-1}X^T\mathbf{y}$ adalah **random variable** (lebih tepatnya, random vector) — karena ia adalah fungsi dari $\mathbf{y}$ yang random. Dengan asumsi $E[\boldsymbol{\varepsilon}|X] = \mathbf{0}$: $$E[\hat{\boldsymbol{\beta}}|X] = E[(X^TX)^{-1}X^T(X\boldsymbol{\beta} + \boldsymbol{\varepsilon})|X]$$ $$= (X^TX)^{-1}X^TX\boldsymbol{\beta} + (X^TX)^{-1}X^T E[\boldsymbol{\varepsilon}|X]$$ $$= \boldsymbol{\beta} + \mathbf{0} = \boldsymbol{\beta} \quad \checkmark \text{ (unbiased)}$$ Dengan asumsi $\text{Var}(\boldsymbol{\varepsilon}|X) = \sigma^2 I$: $$\text{Var}(\hat{\boldsymbol{\beta}}|X) = (X^TX)^{-1}X^T \cdot \sigma^2 I \cdot X(X^TX)^{-1} = \sigma^2(X^TX)^{-1}$$ Ini adalah **covariance matrix** dari OLS estimator — dari sinilah standard errors diturunkan! ::: --- ```{r} #| label: moments-summary #| code-summary: "R: Menghitung moments dari distribusi" # Discrete RV: berapa E[X] dan Var(X)? x_vals <- 0:5 probs <- dpois(x_vals, lambda = 2) # Poisson(2) # E[X] ex <- sum(x_vals * probs) # E[X^2] ex2 <- sum(x_vals^2 * probs) # Var(X) var_x <- ex2 - ex^2 cat("Poisson(lambda=2):\n") cat("E[X] =", round(ex, 4), "(theoretical:", 2, ")\n") cat("Var(X) =", round(var_x, 4), "(theoretical:", 2, ")\n") # Continuous: integrate manually cat("\nNormal(mu=3, sigma^2=4):\n") f_normal <- function(x) dnorm(x, mean=3, sd=2) # E[X] ex_norm <- integrate(function(x) x * f_normal(x), -Inf, Inf)$value cat("E[X] =", round(ex_norm, 4), "\n") # Var(X) ex2_norm <- integrate(function(x) x^2 * f_normal(x), -Inf, Inf)$value var_norm <- ex2_norm - ex_norm^2 cat("Var(X) =", round(var_norm, 4), "\n") ``` --- ::: {.callout-warning title="Practice Problems" collapse="true"} **Soal 1**: Misalkan $X$ punya PMF: $P(X=1) = 0.2$, $P(X=2) = 0.5$, $P(X=3) = 0.3$. Hitung $E[X]$, $E[X^2]$, $\text{Var}(X)$, dan $\text{SD}(X)$. **Solusi 1**: - $E[X] = 1(0.2) + 2(0.5) + 3(0.3) = 0.2 + 1.0 + 0.9 = 2.1$ - $E[X^2] = 1(0.2) + 4(0.5) + 9(0.3) = 0.2 + 2.0 + 2.7 = 4.9$ - $\text{Var}(X) = 4.9 - 2.1^2 = 4.9 - 4.41 = 0.49$ - $\text{SD}(X) = \sqrt{0.49} = 0.7$ --- **Soal 2**: Untuk $X \sim \text{Uniform}(a, b)$ dengan PDF $f(x) = 1/(b-a)$ pada $[a,b]$, buktikan bahwa $E[X] = (a+b)/2$ dan $\text{Var}(X) = (b-a)^2/12$. **Solusi 2**: $$E[X] = \int_a^b \frac{x}{b-a}dx = \frac{1}{b-a}\cdot\frac{b^2-a^2}{2} = \frac{(b+a)(b-a)}{2(b-a)} = \frac{a+b}{2}$$ $$E[X^2] = \int_a^b \frac{x^2}{b-a}dx = \frac{b^3-a^3}{3(b-a)} = \frac{a^2+ab+b^2}{3}$$ $$\text{Var}(X) = \frac{a^2+ab+b^2}{3} - \left(\frac{a+b}{2}\right)^2 = \frac{a^2+ab+b^2}{3} - \frac{a^2+2ab+b^2}{4} = \frac{(b-a)^2}{12}$$ --- **Soal 3**: Jika $X$ dan $Y$ independent dengan $E[X] = 2$, $\text{Var}(X) = 4$, $E[Y] = 3$, $\text{Var}(Y) = 9$. Hitung: (a) $E[2X + 3Y - 1]$, (b) $\text{Var}(2X - Y)$, (c) $\text{Cov}(X+Y, X-Y)$. **Solusi 3**: - (a) $E[2X + 3Y - 1] = 2(2) + 3(3) - 1 = 4 + 9 - 1 = 12$ - (b) $\text{Var}(2X - Y) = 4\text{Var}(X) + \text{Var}(Y) = 4(4) + 9 = 25$ - (c) $\text{Cov}(X+Y, X-Y) = \text{Cov}(X,X) - \text{Cov}(X,Y) + \text{Cov}(Y,X) - \text{Cov}(Y,Y)$ $= \text{Var}(X) - 0 + 0 - \text{Var}(Y) = 4 - 9 = -5$ ::: --- ## Ringkasan | Konsep | Discrete | Continuous | |--------|---------|-----------| | Distribusi | PMF: $p(x) = P(X=x)$ | PDF: $f(x)$, $P(a\leq X\leq b)=\int_a^b f$ | | CDF | $F(x) = \sum_{t\leq x} p(t)$ | $F(x) = \int_{-\infty}^x f(t)dt$ | | Expectation | $E[X] = \sum_x x\,p(x)$ | $E[X] = \int x\,f(x)dx$ | | Variance | $\text{Var}(X) = E[X^2] - (E[X])^2$ | sama | | LOTUS | $E[g(X)] = \sum_x g(x)p(x)$ | $E[g(X)] = \int g(x)f(x)dx$ | **Key results**: - Linearity of expectation: $E[X+Y] = E[X] + E[Y]$ (always) - Independence $\Rightarrow$ uncorrelated, tapi tidak sebaliknya - $\text{Var}(\bar{X}) = \sigma^2/n$ untuk iid sample **Next**: [Key Probability Distributions →](04-distributions.qmd)