官方淘宝店 易迪拓培训 旧站入口
首页 > 无线通信 > 通信技术学习讨论 > 问个信息论的题目

问个信息论的题目

12-16
McELIECE那本信息论上的一个题目
Let f(x) be any function defined for all x>=1. If X is a discret random
variable with rangeR={x_1,x_2,……,x_n},define the f-entropy of X by
H_f(X)=Σp_i*f(1/p_i),where p_i=P{X=x}.
i) If f(x)=log(x)/x,show that H_f(X)<log(e)/e
ii)Work harder on i) and show that in fact H_f(X)<=log(3)/3.with equality
   iff exactly three of the p_i's equal 1/3,and the rest are 0.
一点头绪都没有啊。哪位大侠给点头绪撒。谢谢了

Part (i) is not hard. Let g(x)=f(1/x)=-xlog(x), which has its maximum at x=1/e. Note that g(x) is concave, so that H_f(X)=\sum{p_i*g(p_i)}<=g(\sum{p_i*p_i})<=g(1/e)=1/e. Finally, you need to show that equality is not attainable.
No idea till now about part (ii).

因为H_f(X)=Σp_i*f(1/p_i)=E[f(x)]<=max{f(x)},而f(x)=log(x)/x因此f(x)<loge/e。
第二问用拉格朗日法,g(p1,...pn)=Σpi^2*log(1/pi)-lamda*(Σpi -1),解得内部极大值为logn/n,n>3时logn/n<log3/3,最大值在边界处不妨p1=0,依次类推下去即可。

H_f(x)是一个熵,然后他计算了熵的上限

Top