Relative Information Gain

시간 $t$ 에 대한 RIG 값은 다음과 같다.

$$RIG\left(w_t\right)=1-\frac{\log \mathrm{Loss}\left(w_t\right)}{p_t\log (p_t)-(1-p_t)\log (1-p_t)}$$

여기서 log loss 는 아래와 같다 (binary cross-entropy).

$$\log \mathrm{Loss}\left(w_t\right)=-y_t\log \left(p_t\right)-\left(1-y_t\right)\log \left(1-p_t\right)$$

The key desired property of off-line measures like RIG is its correlation with key online business metrics.

using RIG in offline experiments correlates highly with RPM gains in online settings.