问个信息论的题目
12-16
McELIECE那本信息论上的一个题目
Let f(x) be any function defined for all x>=1. If X is a discret random
variable with rangeR={x_1,x_2,……,x_n},define the f-entropy of X by
H_f(X)=Σp_i*f(1/p_i),where p_i=P{X=x}.
i) If f(x)=log(x)/x,show that H_f(X)<log(e)/e
ii)Work harder on i) and show that in fact H_f(X)<=log(3)/3.with equality
iff exactly three of the p_i's equal 1/3,and the rest are 0.
一点头绪都没有啊。哪位大侠给点头绪撒。谢谢了
Let f(x) be any function defined for all x>=1. If X is a discret random
variable with rangeR={x_1,x_2,……,x_n},define the f-entropy of X by
H_f(X)=Σp_i*f(1/p_i),where p_i=P{X=x}.
i) If f(x)=log(x)/x,show that H_f(X)<log(e)/e
ii)Work harder on i) and show that in fact H_f(X)<=log(3)/3.with equality
iff exactly three of the p_i's equal 1/3,and the rest are 0.
一点头绪都没有啊。哪位大侠给点头绪撒。谢谢了
Part (i) is not hard. Let g(x)=f(1/x)=-xlog(x), which has its maximum at x=1/e. Note that g(x) is concave, so that H_f(X)=\sum{p_i*g(p_i)}<=g(\sum{p_i*p_i})<=g(1/e)=1/e. Finally, you need to show that equality is not attainable.
No idea till now about part (ii).
因为H_f(X)=Σp_i*f(1/p_i)=E[f(x)]<=max{f(x)},而f(x)=log(x)/x因此f(x)<loge/e。
第二问用拉格朗日法,g(p1,...pn)=Σpi^2*log(1/pi)-lamda*(Σpi -1),解得内部极大值为logn/n,n>3时logn/n<log3/3,最大值在边界处不妨p1=0,依次类推下去即可。
H_f(x)是一个熵,然后他计算了熵的上限
相关文章:
- 信息论方面有那些好的中文书啊?(05-08)
- 信息论中没没有信息量随着时间变化的概念(05-08)
- 上传了两本信息论的书(05-08)
- 求朱雪龙老师那本"应用信息论"习题答案!(05-08)
- Shannon著名信息论论文的据信(05-08)
- 问个信息论的问题(05-08)
射频专业培训教程推荐