小马哥课堂-统计学-标准误差

Posted 2023-05-02

参考技术A

在 小马哥课堂-统计学-中心极限定理 一节的例子中提到一个标准误差的概念，有同学对此不清楚，所以这里单独写一节，来对standard error进行阐述，希望能大家能有一个直观的理解。

The standard error (SE) of a statistic (usually an estimate of a parameter) is the standard deviation of its sampling distribution.If the parameter or the statistic is the mean, it is called the standard error of the mean (SEM).

The sampling distribution of a population mean is generated by repeated sampling and recording of the means obtained. This forms a distribution of different means, and this distribution has its own mean and variance. Mathematically, the variance of the sampling distribution obtained is equal to the variance of the population divided by the sample size. This is because as the sample size increases, sample means cluster more closely around the population mean.

Therefore, the relationship between the standard error and the standard deviation is such that, for a given sample size, the standard error equals the standard deviation divided by the square root of the sample size. In other words, the standard error of the mean is a measure of the dispersion of sample means around the population mean.

标准误差，通常是指 某个统计量(一般是某个分布的参数估计，例如正态分布的 参数的估计)的标准误差，即抽样分布的标准差。

对总体进行样本容量为n的抽样，样本容量为n，反复进行抽样，那么"每个样本"的均值 形成一个分布，该分布有自己的期望和方差。数学上， 抽样分布的方差等于 总体方差除以样本容量 。随着样本容量的增大，样本均值越来越接近于总体均值。因此，标准差和标准误的关系是:给定样本容量n，标准误等于 标准差除以 样本容量的平方根。换而言之，样本均值的标准误是衡量 样本均值和总体均值的离散程度。

我们知道，方差是衡量 随机变量 与其期望的离散程度；

又因为，样本均值的标准误是衡量 样本均值 和总体均值的离散程度；

所以，我们将 样本均值 看成是一个 随机变量 ，那么，标准误就是 随机变量 的标准差。概括言之(抽象成更一般的情况)，标准误是抽样分布的标准差。

The standard error of the mean (SEM) can be expressed as:

where

σ is the standard deviation of the population.
n is the size (number of observations) of the sample.

Since the population standard deviation is seldom known, the standard error of the mean is usually estimated as the sample standard deviation divided by the square root of the sample size (assuming statistical independence of the values in the sample).

where

s is the sample standard deviation (i.e., the sample-based estimate of the standard deviation of the population), and
n is the size (number of observations) of the sample.

从上面的结果可以看出， 抽样分布的方差等于 总体方差除以样本容量 ，而且随着样本容量和抽样次数的增加，标准误的值越来越小，即越接近总体方差。

ElasticSearch异常归纳(能力工场小马哥)

• 异常1: can not run elasticsearch as root

[WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-2] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.1.2.jar:5.1.2]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:100) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:176) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.2.jar:5.1.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.2.jar:5.1.2]
... 6 more

 

• 异常1解决方式: 

在es-linux环境中,不允许使用root用户运行ElasticSearch,所以添加一个新的普通用户就可以了(linux命令如下)

1 groupadd es　　　　　 -- 创建一个用户组(我使用的es作为组名)
2 useradd -g es es　　 -- 创建一个用户(我使用es作为用户名,并加入到es组里面)
3 passwd es　　　　　　 -- 为刚刚创建的es用户添加密码
4 su es　　　　　　　　　-- 切换到es用户下
5 $ElasticSearch_Home/bin/elasticsearch  --启动ElasticSearch