(3.91)
3.8 Sampling with replacement 可放回的采样
Up to now, we have considered only the case where we sample without replacement; and that is evidently appropriate for many real situations. For example, in a quality control application, what we have called simply ‘drawing a ball’ might consist of taking a manufactured item, such as an electric light bulb, from a carton of similar light bulbs and testing it to destruction. In a chemistry experiment, it might consist of weighing out a sample of an unknown protein, then dissolving it in hot sulfuric acid to measure its nitrogen content. In either case, there can be no thought of ‘drawing that same ball’ again.
到目前为止，我们只考虑了采样品无需放回去的情况;这显然适用于许多实际情况。例如，在质量控制中，我们所谓的“抽取一个球”可能实际上是从一个纸箱中取出制成品，例如电灯泡，并对其进行破坏性测试。在化学实验中，可能是根据重量对未知蛋白质进行采样，然后将其溶解在热硫酸中以测量其氮含量。在任何一种情况下，都不可能再次“抽到同一个球”。
But suppose now that, being less destructive, we sample balls from the urn and, after recording the ‘color’ (i.e. the relevant property) of each, we replace it in the urn before drawing the next ball. This case, of sampling with replacement, is enormously more complicated conceptually, but, with some assumptions usually made, ends up being simpler mathematically than sampling without replacement. Let us go back to the probability for drawing two red balls in succession. Denoting by B' the same background information as before, except for the added stipulation that the balls are to be replaced, we still have an equation like (3.9):
但是现在假设，测试的破坏性较小，我们从盒中对球采样，并在记录每个球的“颜色”（即相关属性）后，在采样下一个球之前将其放回到盒子中去。这种放回采样的情况,从概念上将更加复杂，但是，再做出一些通常的假设后，最终在数学上要比不放回采样的情况更简单。让我们回到连续两次取出红球的概率的问题。用B'表示与之前问题相同的背景信息，除了采样可以被放回去外，我们仍然有一个像（3.9）一样的等式：
and the first factor is still, evidently, (M/N); but what is the second one? Answering this would be, in general, a very difficult problem, requiring much additional analysis if the background information B' includes some simple but highly relevant common sense information that we all have. What happens to that red ball that we put back in the urn? If we merely dropped it into the urn, and immediately drew another ball, then it was left lying on the top of the other balls (or in the top layer of balls), and so it is more likely to be drawn again than any other specified ball whose location in the urn is unknown. But this upsets the whole basis of our calculation, because the probability for drawing any particular (i th) ball is no longer given by the Bernoulli urn rule which led to (3.11).
第一个因子显然还是（M/N）;但第二个因子应该是什么？一般来说，回答这个问题将是非常困难的，如果背景信息B'含有一些简单但和问题密切相关的常识信息，则我们需要进行更多的额外分析。我们放回盒里的那个红球会如何？如果我们只是将它放入盒中，并立即抽出另一个球，那么它就会停留在其他球的顶部（或者在所有球的最顶层），因此它比任何其他球(因为其位置未知)都更可能被抽到。但这会扰乱我们整个计算的基础，因为取出任何特定（第i个）球的概率不再由伯努利规则（3.11）给出。
3.8.1 Digression: a sermon on reality vs. models 题外话：关于现实与模型的布道
The difficulty we face here is that many things which were irrelevant from symmetry, as long as the robot’s state of knowledge was invariant under any permutation of the balls, suddenly become relevant, and, by one of our desiderata of rationality, the robot must take into account all the relevant information it has. But the probability for drawing any particular ball now depends on such details as the exact size and shape of the urn, the size of the balls, the exact way in which the first one was tossed back in, the elastic properties of balls and urn, the coefficients of friction between balls and between ball and urn, the exact way you reach in to draw the second ball, etc. In a symmetric situation, all of these details are irrelevant.
我们在这里面对的问题是许多事情是与对称性无关的，之前在机器人的知识状态,对于取出的球的任何排列都是不变的，但现在突然变得盒球的排列有关了，根据我们的推理的基本原则之一，即机器人必须将所有得到的信息考虑在内。但是现在取出任何特定一个球的概率将会受到诸如球的确切尺寸和形状以及球的大小，还有第一个取出的球被放回到盒子中的运动轨迹，还有球和盒的弹性特征和摩擦系数,球与球之间的摩擦力，以及取出第二个球的准确方式等等。在对称的情况下，所有这些细节都是无关紧要的。
Even if all these relevant data were at hand, we do not think that a team of the world’s best scientists and mathematicians, backed up by all the world’s computing facilities, would be able to solve the problem; or would even know how to get started on it. Still, it would not be quite right to say that the problem is unsolvable in principle; only so complicated that it is not worth anybody’s time to think about it. So what do we do?
即使我们能得到所有这些数据，即使是世界上最好的科学家和数学家团队,能使用世界上所有的计算资源,我们也不认为能够解决这个问题;甚至不知道如何开始解决。但是，原则上说这个问题无法解决是不正确的,只是太过复杂，以至于不值得任何人花时间去考虑它。那么我们该怎么办？
In probability theory there is a very clever trick for handling a problem that becomes too difficult. We just solve it anyway by:
概率论中，有一个非常聪明的技巧来处理过于困难的问题。办法有：
(1) making it still harder; (2) redefining what we mean by ‘solving’ it, so that it becomes something we can do; (3) inventing a dignified and technical-sounding word to describe this procedure, which has the psychological effect of concealing the real nature of what we have done, and making it appear respectable.
（1）让它变得更难点; （2）重新定义什么是“解决”，让问题变成我们能做的事情; （3）发明一个堂皇且和听起来很技术化的名词来描述这个过程，让它具有一种心理效应,掩盖我们已经做的工作的真实本质，并且让它让人敬畏。
In the case of sampling with replacement, we apply this strategy as follows.
在可放回抽样的情况下，我们应用此策略如下。
(1) Suppose that, after tossing the ball in, we shake up the urn. However complicated the problem was initially, it now becomes many orders of magnitude more complicated, because the solution now depends on every detail of the precise way we shake it, in addition to all the factors mentioned above. (2) We now assert that the shaking has somehow made all these details irrelevant, so that the problem reverts back to the simple one where the Bernoulli urn rule applies. (3) We invent the dignified-soundingword randomization to describe what we have done. This term is, evidently, a euphemism, whose real meaning is: deliberately throwing away relevant information when it becomes too complicated for us to handle.
（1）假设在将球放回去后，再晃动盒子。虽然原来的问题很复杂，但现在复杂度又增加几个数量级，因为除了上面提到的所有因素之外，现在还要考虑到晃动盒子的方式的每个精确的细节。 （2）我们现在断言，晃动盒子这个动作已经以某种方式使所有这些细节变得无关紧要，所以又回到了简单的伯努利实验的情况。 （3）我们发明一个堂而皇之的名字"随机化"来描述我们所做的事情。这个术语显然是一种委婉用语，其真正含义是：当它变得太复杂而我们无法处理时，有意丢弃一些相关信息。
We have described this procedure in laconic terms, because an antidote is needed for the impression created by some writers on probability theory, who attach a kind of mystical significance to it. For some, declaring a problem to be ‘randomized’ is an incantation with the same purpose and effect as those uttered by an exorcist to drive out evil spirits; i.e. it cleanses their subsequent calculations and renders them immune to criticism. We agnostics often envy the True Believer, who thus acquires so easily that sense of security which is forever denied to us.
我们用简洁的术语描述了这个过程，因为一些著作给概率论附加了一种神秘的意义,让人印象深刻,所以需要一副解毒剂。对于一些人来说，宣称一个问题是“随机的”是一个咒语，其目的和效果与驱魔人用来驱逐邪灵的目的和效果相同;即这"净化"了他们之后的计算并让他们免于受到批评。我们这种不可知论者经常嫉妒那些真正的信徒，因为他们很容易就得到了那些安全感,而我们永远也做不到。
However, in defense of this procedure, we have to admit that it often leads to a useful approximation to the correct solution; i.e. the complicated details, while undeniably relevant in principle, might nevertheless have little numerical effect on the answers to certain particularly simple questions, such as the probability for drawing r red balls in n trials when n is sufficiently small. But from the standpoint of principle, an element of vagueness necessarily enters at this point; for, while we may feel intuitively that this leads to a good approximation, we have no proof of this, much less a reliable estimate of the accuracy of the approximation, which presumably improves with more shaking.
然而，为了捍卫这一过程，我们得承认它通常会得出有用的,近似正确的解决方案;即，虽然这些复杂的细节从原则上不能否认会产生影响，但对于某些特别简单的问题的答案可能几乎没有数量上的影响，例如求当n足够小时在n次试验中抽取r个红球的概率。但从原则的角度来看，这是引入了一个模糊的因素;因为虽然我们可能直觉地认为这导致了一个很好的近似，但我们没有证据证明这一点，更不用说对近似精度的可靠估计，虽然这个精度可能会随着晃动盒子更多次而有所改善。
The vagueness is evident particularly in the fact that different people have widely divergent views about how much shaking is required to justify step (2). Witness the minor furor surrounding a US Government-sponsored and nationally televized game of chance some years ago, when someone objected that the procedure for drawing numbers from a fish bowl to determine the order of call-up of young men for Military Service was ‘unfair’ because the bowl hadn’t been shaken enough to make the drawing ‘truly random’, whatever that means. Yet if anyone had asked the objector: ‘To whom is it unfair?’ he could not have given any answer except, ‘To those whose numbers are on top; I don’t know who they are.’ But after any amount of further shaking, this will still be true! So what does the shaking accomplish?
需要晃动多少次才能达到（2）的要求,事实上不同的人观点差别巨大,使得引入的这个模糊因素如此显眼.多年前，在美国政府赞助下,为了决定服军役的征调顺序而在全国电视上播放的抽签过程,即从大鱼碗中抽取数字，引起了的轻微骚动,称这是“不公平的”,“因为碗没有被摇晃到足以使抽签是'真正随机'的，无论这意味着什么。然而如果有人问反对者：“是对谁不公平？”反对者无法给出任何答案,除了“碗中在最上面的那些数字所代表的人，不过我不知道他们是谁”。但即使再多摇晃一会后，这个现象仍然存在！那么再多晃几下到底有什么用呢？
Shaking does not make the result ‘random’, because that term is basically meaningless as an attribute of the real world; it has no clear definition applicable in the real world. The belief that ‘randomness’ is some kind of real property existing in Nature is a form of the mind projection fallacy which says, in effect, ‘I don’t know the detailed causes – therefore – Nature does not know them.’ What shaking accomplishes is very different. It does not affect Nature’s workings in any way; it only ensures that no human is able to exert any wilful influence on the result. Therefore, nobody can be charged with ‘fixing’ the outcome.
摇晃并不会使结果“随机”，说真实世界的拥有"随机"属性基本上没什么意义;它没有明确的定义能适用于现实世界。相信“随机性”大自然中存在的某种真实性质,这是一种臆想谬误，实际上这和说“我不知道详细的原因--所以--大自然也不知道原因”没什两样.但晃动盒子和这个不同。它不会以影响大自然的运作,以任何方式;它只能确保没有人能够对结果施加任何故意的影响。因此，没有人可以被指控“修改”了结果。
At this point, you may accuse us of nitpicking, because you know that after all this sermonizing, we are just going to go ahead and use the randomized solution like everybody else does. Note, however, that our objection is not to the procedure itself, provided that we acknowledge honestly what we are doing; i.e. instead of solving the real problem, we are making a practical compromise and being, of necessity, content with an approximate solution. That is something we have to do in all areas of applied mathematics, and there is no reason to expect probability theory to be any different.
在这一点上，你可能会指责我们有点吹毛求疵，因为你知道在啰嗦完上面这些后，我们解决随机问题的方案还是其他人一样。但请注意，只要我们诚实地承认我们正在做的事情,我们的反对意见针对的不是过程本身;即，我们并没有解决实际问题，只是对现实的妥协，不得不满足于一个近似的解决方案。这是我们在应用数学的所有领域都不得不做的事情，并且没有理由期望概率理论有任何不同。
Our objection is to the belief that by randomization we somehow make our subsequent equations exact; so exact that we can then subject our solution to all kinds of extreme conditions and believe the results, when applied to the real world. The most serious and most common error resulting from this belief is in the derivation of limit theorems (i.e. when sampling with replacement, nothing prevents us from passing to the limit n→∞ and obtaining the usual ‘laws of large numbers’). If we do not recognize the approximate nature of our starting equations, we delude ourselves into believing that we have proved things (such as the identity of probability and limiting frequency) that are just not true in real repetitive experiments.
我们的反对的是种信念,即随机化过程以某种方式,使我们的方程是准确的;当在现实世界中应用我们方案时,能够应付各种极端条件的情况，并且能够确信结果是准确的。由此信念可能导致的最严重和最常见的错误是,从极限定理开始推导时（即，在可放回采样中，没有什么能阻止我们在极限n→∞时得出通常的“大数定律”）。如果我们不认识我们的起始时的方程的本质是一种近似，我们就会自欺欺人地相信我们已经证明了一些结论（例如概率的均等性和频率的有限性）,这些结论在真实的重复实验中是错误的。
The danger here is particularly great because mathematicians generally regard these limit theorems as the most important and sophisticated fruits of probability theory, and have a tendency to use language which implies that they are proving properties of the real world. Our point is that these theorems are valid properties of the abstract mathematical model that was defined and analyzed. The issue is: to what extent does that model resemble the real world? It is probably safe to say that no limit theorem is directly applicable in the real world, simply because no mathematical model captures every circumstance that is relevant in the real world. Anyone who believes that he is proving things about the real world, is a victim of the mind projection fallacy.
在这点上危险非常大，由于数学家们普遍认为这些极限定律是概率论最重要和最复杂的成果，并倾向于在语言中暗示它们是现实世界的一个已经被实证的属性。我们的观点是这些定律是我们定义并分析的抽象数学模型的属性。问题是：该模型在多大程度上与现实世界相似？可以肯定地说，这些极限定律不能直接应用于现实世界，因为没有数学模型能够捕捉到现实世界中所有的相关因素。任何相信他正在证明的东西就是属于现实世界的人，都是臆想谬误的受害者。
Let us return to the equations. What answer can we now give to the question posed after Eq. (3.91)? The probability of drawing a red ball on the second draw clearly depends not only on N and M, but also on the fact that a red one has already been drawn and replaced. But this latter dependence is so complicated that we can’t, in real life, take it into account; so we shake the urn to ‘randomize’ the problem, and then declare to be irrelevant: . After drawing and replacing the second ball, we again shake the urn, declare it ‘randomized,’ and set , etc. In this approximation, the probability for drawing a red ball at any trial is M/N.
让我们回到数学等式。我们现在继续可以回答在等式(3.91)之后提出的问题。第二次抽中红球的概率显然不仅取决于N和M，而且还取决于一个红球已经被抽中和放回的事实。但后一种依赖是如此复杂，以至于在现实生活中我们无法对其进行处理,所以不能考虑在内;所以我们通过晃动盒子来'随机化'问题，然后声明是无关的,即。在取出并放回第二个球之后，我们再次摇动盒子，声明它被“随机化”了，并取,等等。在这种近似下，任何一次抽中红球的概率都是M/N.
This is not just a repetition of what we learned in (3.37); what is new here is that the result now holds whatever information the robot may have about what happened in the other trials. This leads us to write the probability for drawing exactly r red balls in n trials, regardless of order, as
这不仅仅是在重复我们在（3.37）中学到的东西;新内容是，机器人能得到所有和采样有关的信息,都体现在结论中体现了。因此,不考虑顺序,在n次中恰好抽到r个红球的概率，可写为
, (3.92)
which is just the binomial distribution (3.86). Randomized sampling with replacement from an urn with finite N has approximately the same effect as passage to the limit N →∞ without replacement.
即是二项分布（3.86）。在n个球的盒子中进行的可放回的随机化了的采样,与从在极限N→∞情况下且不用放回,得到了大致相同的结果。
Evidently, for small n, this approximation will be quite good; but for large n these small errors can accumulate (depending on exactly how we shake the urn, etc.) to the point where (3.92) is misleading. Let us demonstrate this by a simple, but realistic, extension of the problem.
显然，当n比较小时，这是一种非常号的近似;但当n足够大时，这些小误差可能累积（取决于我们如何晃动盒子,等等）到误导的程度(3.92)。让我们通过一个简单,但现实的扩展问题来证明这一点。