From 46d6bf56bef44b928017430264b59d4f3c376b08 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Tue, 4 Aug 2015 12:44:41 -0700 Subject: [PATCH 01/10] Initial split of book into notebook. --- notebooks/01_Bayes_Theorem.ipynb | 476 +++++++ notebooks/02_Computational_Statistics.ipynb | 999 +++++++++++++++ notebooks/03_Estimation.ipynb | 495 ++++++++ notebooks/04_More_Estimation.ipynb | 839 ++++++++++++ notebooks/05_Odds_and_Addends.ipynb | 527 ++++++++ notebooks/06_Decision_Analysis.ipynb | 625 +++++++++ notebooks/07_Prediction.ipynb | 472 +++++++ notebooks/08_Observer_Bias.ipynb | 695 ++++++++++ notebooks/09_Two_Dimensions.ipynb | 468 +++++++ .../10_Approximate_Bayesian_Computation.ipynb | 734 +++++++++++ notebooks/11_Hypothesis_Testing.ipynb | 285 +++++ notebooks/12_Evidence.ipynb | 555 ++++++++ notebooks/13_Simulation.ipynb | 570 +++++++++ notebooks/14_A_Hierarchical_Model.ipynb | 323 +++++ notebooks/15_Dealing_with_Dimensions.ipynb | 1120 +++++++++++++++++ notebooks/Preface.ipynb | 545 ++++++++ 16 files changed, 9728 insertions(+) create mode 100644 notebooks/01_Bayes_Theorem.ipynb create mode 100644 notebooks/02_Computational_Statistics.ipynb create mode 100644 notebooks/03_Estimation.ipynb create mode 100644 notebooks/04_More_Estimation.ipynb create mode 100644 notebooks/05_Odds_and_Addends.ipynb create mode 100644 notebooks/06_Decision_Analysis.ipynb create mode 100644 notebooks/07_Prediction.ipynb create mode 100644 notebooks/08_Observer_Bias.ipynb create mode 100644 notebooks/09_Two_Dimensions.ipynb create mode 100644 notebooks/10_Approximate_Bayesian_Computation.ipynb create mode 100644 notebooks/11_Hypothesis_Testing.ipynb create mode 100644 notebooks/12_Evidence.ipynb create mode 100644 notebooks/13_Simulation.ipynb create mode 100644 notebooks/14_A_Hierarchical_Model.ipynb create mode 100644 notebooks/15_Dealing_with_Dimensions.ipynb create mode 100644 notebooks/Preface.ipynb diff --git a/notebooks/01_Bayes_Theorem.ipynb b/notebooks/01_Bayes_Theorem.ipynb new file mode 100644 index 0000000..27a0eeb --- /dev/null +++ b/notebooks/01_Bayes_Theorem.ipynb @@ -0,0 +1,476 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Bayes’s Theorem {#intro}\n", + "\n", + "##Conditional probability\n", + "\n", + "The fundamental idea behind all Bayesian statistics is Bayes’s theorem,\n", + "which is surprisingly easy to derive, provided that you understand\n", + "conditional probability. So we’ll start with probability, then\n", + "conditional probability, then Bayes’s theorem, and on to Bayesian\n", + "statistics.\n", + "\n", + "A probability is a number between 0 and 1 (including both) that\n", + "represents a degree of belief in a fact or prediction. The value 1\n", + "represents certainty that a fact is true, or that a prediction will come\n", + "true. The value 0 represents certainty that the fact is false.\n", + "\n", + "Intermediate values represent degrees of certainty. The value 0.5, often\n", + "written as 50%, means that a predicted outcome is as likely to happen as\n", + "not. For example, the probability that a tossed coin lands face up is\n", + "very close to 50%.\n", + "\n", + "A conditional probability is a probability based on some background\n", + "information. For example, I want to know the probability that I will\n", + "have a heart attack in the next year. According to the CDC, “Every year\n", + "about 785,000 Americans have a first coronary attack.\n", + "()”\n", + "\n", + "The U.S. population is about 311 million, so the probability that a\n", + "randomly chosen American will have a heart attack in the next year is\n", + "roughly 0.3%.\n", + "\n", + "But I am not a randomly chosen American. Epidemiologists have identified\n", + "many factors that affect the risk of heart attacks; depending on those\n", + "factors, my risk might be higher or lower than average.\n", + "\n", + "I am male, 45 years old, and I have borderline high cholesterol. Those\n", + "factors increase my chances. However, I have low blood pressure and I\n", + "don’t smoke, and those factors decrease my chances.\n", + "\n", + "Plugging everything into the online calculator at\n", + ", I find that my risk of a\n", + "heart attack in the next year is about 0.2%, less than the national\n", + "average. That value is a conditional probability, because it is based on\n", + "a number of factors that make up my “condition.”\n", + "\n", + "The usual notation for conditional probability is\n", + "$\\mathrm{p}(A|B)$, which is the probability of $A$ given\n", + "that $B$ is true. In this example, $A$ represents the prediction that I\n", + "will have a heart attack in the next year, and $B$ is the set of\n", + "conditions I listed.\n", + "\n", + "##Conjoint probability\n", + "\n", + "**Conjoint probability** is a fancy way to say the\n", + "probability that two things are true. I write\n", + "$\\mathrm{p}(A {~\\mathrm{and}~}B)$ to mean the probability\n", + "that $A$ and $B$ are both true.\n", + "\n", + "If you learned about probability in the context of coin tosses and dice,\n", + "you might have learned the following formula:\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B)}} \\quad\\quad\\mbox{WARNING: not always true}$$\n", + "For example, if I toss two coins, and $A$ means the first coin lands\n", + "face up, and $B$ means the second coin lands face up, then\n", + "${{\\mathrm{p}(A)}} =\n", + "{{\\mathrm{p}(B)}} = 0.5$, and sure enough,\n", + "${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B)}} = 0.25$.\n", + "\n", + "But this formula only works because in this case $A$ and $B$ are\n", + "independent; that is, knowing the outcome of the first event does not\n", + "change the probability of the second. Or, more formally,\n", + "$\\mathrm{p}(B|A)$ = $\\mathrm{p}(B)$.\n", + "\n", + "Here is a different example where the events are not independent.\n", + "Suppose that $A$ means that it rains today and $B$ means that it rains\n", + "tomorrow. If I know that it rained today, it is more likely that it will\n", + "rain tomorrow, so\n", + "${{\\mathrm{p}(B|A)}} > {{\\mathrm{p}(B)}}$.\n", + "\n", + "In general, the probability of a conjunction is\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", + "for any $A$ and $B$. So if the chance of rain on any given day is 0.5,\n", + "the chance of rain on two consecutive days is not 0.25, but probably a\n", + "bit higher.\n", + "\n", + "##The cookie problem\n", + "\n", + "We’ll get to Bayes’s theorem soon, but I want to motivate it with an\n", + "example called the cookie problem.[^1] Suppose there are two bowls of\n", + "cookies. Bowl 1 contains 30 vanilla cookies and 10 chocolate cookies.\n", + "Bowl 2 contains 20 of each.\n", + "\n", + "Now suppose you choose one of the bowls at random and, without looking,\n", + "select a cookie at random. The cookie is vanilla. What is the\n", + "probability that it came from Bowl 1?\n", + "\n", + "This is a conditional probability; we want\n", + "$\\mathrm{p}({\\mbox{Bowl 1}} |\n", + " {\\mbox{vanilla}})$, but it is not obvious how to compute it. If\n", + "I asked a different question—the probability of a vanilla cookie given\n", + "Bowl 1—it would be easy:\n", + "$${{\\mathrm{p}({\\mbox{vanilla}} | {\\mbox{Bowl 1}})}} = 3/4$$\n", + "Sadly, $\\mathrm{p}(A|B)$ is *not* the same as\n", + "$\\mathrm{p}(B|A)$, but there is a way to get from one to\n", + "the other: Bayes’s theorem.\n", + "\n", + "##Bayes’s theorem {#bayess-theorem}\n", + "\n", + "At this point we have everything we need to derive Bayes’s theorem.\n", + "We’ll start with the observation that conjunction is commutative; that\n", + "is\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(B {~\\mathrm{and}~}A)}}$$\n", + "for any events $A$ and $B$.\n", + "\n", + "Next, we write the probability of a conjunction:\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", + "Since we have not said anything about what $A$ and $B$ mean, they are\n", + "interchangeable. Interchanging them yields\n", + "$${{\\mathrm{p}(B {~\\mathrm{and}~}A)}} = {{\\mathrm{p}(B)}}~{{\\mathrm{p}(A|B)}}$$\n", + "That’s all we need. Pulling those pieces together, we get\n", + "$${{\\mathrm{p}(B)}}~{{\\mathrm{p}(A|B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", + "Which means there are two ways to compute the conjunction. If you have\n", + "$\\mathrm{p}(A)$, you multiply by the conditional\n", + "probability $\\mathrm{p}(B|A)$. Or you can do it the other\n", + "way around; if you know $\\mathrm{p}(B)$, you multiply by\n", + "$\\mathrm{p}(A|B)$. Either way you should get the same\n", + "thing.\n", + "\n", + "Finally we can divide through by $\\mathrm{p}(B)$:\n", + "$${{\\mathrm{p}(A|B)}} = \\frac{{{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}}{{{\\mathrm{p}(B)}}}$$\n", + "And that’s Bayes’s theorem! It might not look like much, but it turns\n", + "out to be surprisingly powerful.\n", + "\n", + "For example, we can use it to solve the cookie problem. I’ll write $B_1$\n", + "for the hypothesis that the cookie came from Bowl 1 and $V$ for the\n", + "vanilla cookie. Plugging in Bayes’s theorem we get\n", + "$${{\\mathrm{p}(B_1|V)}} = \\frac{{{\\mathrm{p}(B_1)}}~{{\\mathrm{p}(V|B_1)}}}{{{\\mathrm{p}(V)}}}$$\n", + "The term on the left is what we want: the probability of Bowl 1, given\n", + "that we chose a vanilla cookie. The terms on the right are:\n", + "\n", + "- ${{\\mathrm{p}(B_1)}}$: This is the probability that we\n", + " chose Bowl 1, unconditioned by what kind of cookie we got. Since the\n", + " problem says we chose a bowl at random, we can assume\n", + " ${{\\mathrm{p}(B_1)}} = 1/2$.\n", + "\n", + "- ${{\\mathrm{p}(V|B_1)}}$: This is the probability of\n", + " getting a vanilla cookie from Bowl 1, which is 3/4.\n", + "\n", + "- $\\mathrm{p}(V)$: This is the probability of drawing a\n", + " vanilla cookie from either bowl. Since we had an equal chance of\n", + " choosing either bowl and the bowls contain the same number of\n", + " cookies, we had the same chance of choosing any cookie. Between the\n", + " two bowls there are 50 vanilla and 30 chocolate cookies, so\n", + " $\\mathrm{p}(V)$ = 5/8.\n", + "\n", + "Putting it together, we have\n", + "$${{\\mathrm{p}(B_1|V)}} = \\frac{(1/2)~(3/4)}{5/8}$$ which\n", + "reduces to 3/5. So the vanilla cookie is evidence in favor of the\n", + "hypothesis that we chose Bowl 1, because vanilla cookies are more likely\n", + "to come from Bowl 1.\n", + "\n", + "This example demonstrates one use of Bayes’s theorem: it provides a\n", + "strategy to get from $\\mathrm{p}(B|A)$ to\n", + "$\\mathrm{p}(A|B)$. This strategy is useful in cases, like\n", + "the cookie problem, where it is easier to compute the terms on the right\n", + "side of Bayes’s theorem than the term on the left.\n", + "\n", + "##The diachronic interpretation\n", + "\n", + "There is another way to think of Bayes’s theorem: it gives us a way to\n", + "update the probability of a hypothesis, $H$, in light of some body of\n", + "data, $D$.\n", + "\n", + "This way of thinking about Bayes’s theorem is called the\n", + "**diachronic interpretation**. “Diachronic” means that\n", + "something is happening over time; in this case the probability of the\n", + "hypotheses changes, over time, as we see new data.\n", + "\n", + "Rewriting Bayes’s theorem with $H$ and $D$ yields:\n", + "$${{\\mathrm{p}(H|D)}} = \\frac{{{\\mathrm{p}(H)}}~{{\\mathrm{p}(D|H)}}}{{{\\mathrm{p}(D)}}}$$\n", + "In this interpretation, each term has a name:\n", + "\n", + "- $\\mathrm{p}(H)$ is the probability of the hypothesis\n", + " before we see the data, called the prior probability, or just\n", + " **prior**.\n", + "\n", + "- $\\mathrm{p}(H|D)$ is what we want to compute, the\n", + " probability of the hypothesis after we see the data, called the\n", + " **posterior**.\n", + "\n", + "- $\\mathrm{p}(D|H)$ is the probability of the data under\n", + " the hypothesis, called the **likelihood**.\n", + "\n", + "- $\\mathrm{p}(D)$ is the probability of the data under\n", + " any hypothesis, called the **normalizing constant**.\n", + "\n", + "Sometimes we can compute the prior based on background information. For\n", + "example, the cookie problem specifies that we choose a bowl at random\n", + "with equal probability.\n", + "\n", + "In other cases the prior is subjective; that is, reasonable people might\n", + "disagree, either because they use different background information or\n", + "because they interpret the same information differently.\n", + "\n", + "The likelihood is usually the easiest part to compute. In the cookie\n", + "problem, if we know which bowl the cookie came from, we find the\n", + "probability of a vanilla cookie by counting.\n", + "\n", + "The normalizing constant can be tricky. It is supposed to be the\n", + "probability of seeing the data under any hypothesis at all, but in the\n", + "most general case it is hard to nail down what that means.\n", + "\n", + "Most often we simplify things by specifying a set of hypotheses that are\n", + "\n", + "Mutually exclusive:\n", + ": At most one hypothesis in the set can be true, and\n", + "\n", + "Collectively exhaustive:\n", + ": There are no other possibilities; at least one of the hypotheses has\n", + " to be true.\n", + "\n", + "I use the word **suite** for a set of hypotheses that has\n", + "these properties.\n", + "\n", + "In the cookie problem, there are only two hypotheses—the cookie came\n", + "from Bowl 1 or Bowl 2—and they are mutually exclusive and collectively\n", + "exhaustive.\n", + "\n", + "In that case we can compute $\\mathrm{p}(D)$ using the law\n", + "of total probability, which says that if there are two exclusive ways\n", + "that something might happen, you can add up the probabilities like this:\n", + "$${{\\mathrm{p}(D)}} = {{\\mathrm{p}(B_1)}}~{{\\mathrm{p}(D|B_1)}} + {{\\mathrm{p}(B_2)}}~{{\\mathrm{p}(D|B_2)}}$$\n", + "Plugging in the values from the cookie problem, we have\n", + "$${{\\mathrm{p}(D)}} = (1/2)~(3/4) + (1/2)~(1/2) = 5/8$$ which\n", + "is what we computed earlier by mentally combining the two bowls." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The M&M problem\n", + "\n", + "M&M’s are small candy-coated chocolates that come in a\n", + "variety of colors. Mars, Inc., which makes M&M’s, changes\n", + "the mixture of colors from time to time.\n", + "\n", + "In 1995, they introduced blue M&M’s. Before then, the color\n", + "mix in a bag of plain M&M’s was 30% Brown, 20% Yellow, 20%\n", + "Red, 10% Green, 10% Orange, 10% Tan. Afterward it was 24% Blue , 20%\n", + "Green, 16% Orange, 14% Yellow, 13% Red, 13% Brown.\n", + "\n", + "Suppose a friend of mine has two bags of M&M’s, and he\n", + "tells me that one is from 1994 and one from 1996. He won’t tell me which\n", + "is which, but he gives me one M&M from each bag. One is\n", + "yellow and one is green. What is the probability that the yellow one\n", + "came from the 1994 bag?\n", + "\n", + "This problem is similar to the cookie problem, with the twist that I\n", + "draw one sample from each bowl/bag. This problem also gives me a chance\n", + "to demonstrate the table method, which is useful for solving problems\n", + "like this on paper. In the next chapter we will solve them\n", + "computationally.\n", + "\n", + "The first step is to enumerate the hypotheses. The bag the yellow\n", + "M&M came from I’ll call Bag 1; I’ll call the other Bag 2.\n", + "So the hypotheses are:\n", + "\n", + "- A: Bag 1 is from 1994, which implies that Bag 2 is from 1996.\n", + "\n", + "- B: Bag 1 is from 1996 and Bag 2 from 1994.\n", + "\n", + "Now we construct a table with a row for each hypothesis and a column for\n", + "each term in Bayes’s theorem:\n", + "\n", + "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", + "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", + "| A | 1/2 | (20)(20) | 200 | 20/27|\n", + "| B | 1/2 | (14)(10) | 70 | 7/27|\n", + "\n", + "\n", + "The first column has the priors. Based on the statement of the problem,\n", + "it is reasonable to choose\n", + "${{\\mathrm{p}(A)}} = {{\\mathrm{p}(B)}} = 1/2$.\n", + "\n", + "The second column has the likelihoods, which follow from the information\n", + "in the problem. For example, if $A$ is true, the yellow\n", + "M&M came from the 1994 bag with probability 20%, and the\n", + "green came from the 1996 bag with probability 20%. If $B$ is true, the\n", + "yellow M&M came from the 1996 bag with probability 14%, and\n", + "the green came from the 1994 bag with probability 10%. Because the\n", + "selections are independent, we get the conjoint probability by\n", + "multiplying.\n", + "\n", + "The third column is just the product of the previous two. The sum of\n", + "this column, 270, is the normalizing constant. To get the last column,\n", + "which contains the posteriors, we divide the third column by the\n", + "normalizing constant.\n", + "\n", + "That’s it. Simple, right?\n", + "\n", + "Well, you might be bothered by one detail. I write\n", + "$\\mathrm{p}(D|H)$ in terms of percentages, not\n", + "probabilities, which means it is off by a factor of 10,000. But that\n", + "cancels out when we divide through by the normalizing constant, so it\n", + "doesn’t affect the result.\n", + "\n", + "When the set of hypotheses is mutually exclusive and collectively\n", + "exhaustive, you can multiply the likelihoods by any factor, if it is\n", + "convenient, as long as you apply the same factor to the entire column." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Monty Hall problem\n", + "----------------------\n", + "\n", + "The Monty Hall problem might be the most contentious question in the\n", + "history of probability. The scenario is simple, but the correct answer\n", + "is so counterintuitive that many people just can’t accept it, and many\n", + "smart people have embarrassed themselves not just by getting it wrong\n", + "but by arguing the wrong side, aggressively, in public.\n", + "\n", + "Monty Hall was the original host of the game show *Let’s Make a\n", + "Deal*. The Monty Hall problem is based on one of the regular\n", + "games on the show. If you are on the show, here’s what happens:\n", + "\n", + "- Monty shows you three closed doors and tells you that there is a\n", + " prize behind each door: one prize is a car, the other two are less\n", + " valuable prizes like peanut butter and fake finger nails. The prizes\n", + " are arranged at random.\n", + "\n", + "- The object of the game is to guess which door has the car. If you\n", + " guess right, you get to keep the car.\n", + "\n", + "- You pick a door, which we will call Door A. We’ll call the other\n", + " doors B and C.\n", + "\n", + "- Before opening the door you chose, Monty increases the suspense by\n", + " opening either Door B or C, whichever does not have the car. (If the\n", + " car is actually behind Door A, Monty can safely open B or C, so he\n", + " chooses one at random.)\n", + "\n", + "- Then Monty offers you the option to stick with your original choice\n", + " or switch to the one remaining unopened door.\n", + "\n", + "The question is, should you “stick” or “switch” or does it make no\n", + "difference?\n", + "\n", + "Most people have the strong intuition that it makes no difference. There\n", + "are two doors left, they reason, so the chance that the car is behind\n", + "Door A is 50%.\n", + "\n", + "But that is wrong. In fact, the chance of winning if you stick with Door\n", + "A is only 1/3; if you switch, your chances are 2/3.\n", + "\n", + "By applying Bayes’s theorem, we can break this problem into simple\n", + "pieces, and maybe convince ourselves that the correct answer is, in\n", + "fact, correct.\n", + "\n", + "To start, we should make a careful statement of the data. In this case\n", + "$D$ consists of two parts: Monty chooses Door B *and* there\n", + "is no car there.\n", + "\n", + "Next we define three hypotheses: $A$, $B$, and $C$ represent the\n", + "hypothesis that the car is behind Door A, Door B, or Door C. Again,\n", + "let’s apply the table method:\n", + "\n", + "\n", + "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", + "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", + "| A | 1/3 | 1/2 | 1/6 |1/3|\n", + "| B | 1/3 | 0 | 0 | 0 |\n", + "| C | 1/3 | 1 | 1/3 | 2/3 |\n", + "\n", + "Filling in the priors is easy because we are told that the prizes are\n", + "arranged at random, which suggests that the car is equally likely to be\n", + "behind any door.\n", + "\n", + "Figuring out the likelihoods takes some thought, but with reasonable\n", + "care we can be confident that we have it right:\n", + "\n", + "- If the car is actually behind A, Monty could safely open Doors B or\n", + " C. So the probability that he chooses B is 1/2. And since the car is\n", + " actually behind A, the probability that the car is not behind B is\n", + " 1.\n", + "\n", + "- If the car is actually behind B, Monty has to open door C, so the\n", + " probability that he opens door B is 0.\n", + "\n", + "- Finally, if the car is behind Door C, Monty opens B with probability\n", + " 1 and finds no car there with probability 1.\n", + "\n", + "Now the hard part is over; the rest is just arithmetic. The sum of the\n", + "third column is 1/2. Dividing through yields\n", + "${{\\mathrm{p}(A|D)}} = 1/3$ and\n", + "${{\\mathrm{p}(C|D)}} = 2/3$. So you are better off switching.\n", + "\n", + "There are many variations of the Monty Hall problem. One of the\n", + "strengths of the Bayesian approach is that it generalizes to handle\n", + "these variations.\n", + "\n", + "For example, suppose that Monty always chooses B if he can, and only\n", + "chooses C if he has to (because the car is behind B). In that case the\n", + "revised table is:\n", + "\n", + "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", + "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", + "| A | 1/3| 1 |1/3 | 1/2\n", + "| B |1/3 | 0 | 0 | 0 |\n", + "| C |1/3 | 1 | 1/3 | 1/2|\n", + "\n", + "The only change is $\\mathrm{p}(D|A)$. If the car is behind\n", + "$A$, Monty can choose to open B or C. But in this variation he always\n", + "chooses B, so ${{\\mathrm{p}(D|A)}} = 1$.\n", + "\n", + "As a result, the likelihoods are the same for $A$ and $C$, and the\n", + "posteriors are the same:\n", + "${{\\mathrm{p}(A|D)}} = {{\\mathrm{p}(C|D)}} = 1/2$.\n", + "In this case, the fact that Monty chose B reveals no information about\n", + "the location of the car, so it doesn’t matter whether the contestant\n", + "sticks or switches.\n", + "\n", + "On the other hand, if he had opened $C$, we would know\n", + "${{\\mathrm{p}(B|D)}} = 1$.\n", + "\n", + "I included the Monty Hall problem in this chapter because I think it is\n", + "fun, and because Bayes’s theorem makes the complexity of the problem a\n", + "little more manageable. But it is not a typical use of Bayes’s theorem,\n", + "so if you found it confusing, don’t worry!\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "For many problems involving conditional probability, Bayes’s theorem\n", + "provides a divide-and-conquer strategy. If\n", + "$\\mathrm{p}(A|B)$ is hard to compute, or hard to measure\n", + "experimentally, check whether it might be easier to compute the other\n", + "terms in Bayes’s theorem, $\\mathrm{p}(B|A)$,\n", + "$\\mathrm{p}(A)$ and $\\mathrm{p}(B)$.\n", + "\n", + "If the Monty Hall problem is your idea of fun, I have collected a number\n", + "of similar problems in an article called “All your Bayes are belong to\n", + "us,” which you can read at\n", + "." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/02_Computational_Statistics.ipynb b/notebooks/02_Computational_Statistics.ipynb new file mode 100644 index 0000000..a2d0af2 --- /dev/null +++ b/notebooks/02_Computational_Statistics.ipynb @@ -0,0 +1,999 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The autoreload extension is already loaded. To reload it, use:\n", + " %reload_ext autoreload\n" + ] + } + ], + "source": [ + "import sys\n", + "from __future__ import division, print_function\n", + "%load_ext autoreload\n", + "%autoreload 2 \n", + "import sys\n", + "sys.path.insert(0,'../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Computational Statistics {#compstat}\n", + "\n", + "##Distributions\n", + "\n", + "In statistics a **distribution** is a set of values and\n", + "their corresponding probabilities.\n", + "\n", + "For example, if you roll a six-sided die, the set of possible values is\n", + "the numbers 1 to 6, and the probability associated with each value is\n", + "1/6.\n", + "\n", + "As another example, you might be interested in how many times each word\n", + "appears in common English usage. You could build a distribution that\n", + "includes each word and how many times it appears.\n", + "\n", + "To represent a distribution in Python, you could use a dictionary that\n", + "maps from each value to its probability. I have written a class called\n", + "Pmf that uses a Python dictionary in exactly that way, and\n", + "provides a number of useful methods. I called the class Pmf in reference\n", + "to a **probability mass function**, which is a way to\n", + "represent a distribution mathematically.\n", + "\n", + "Pmf is defined in a Python module I wrote to accompany this\n", + "book, thinkbayes.py. You can download it from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "To use Pmf you can import it like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from thinkbayes import Pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following code builds a Pmf to represent the distribution of\n", + "outcomes for a six-sided die:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1 0.16666666666666666\n", + "2 0.16666666666666666\n", + "3 0.16666666666666666\n", + "4 0.16666666666666666\n", + "5 0.16666666666666666\n", + "6 0.16666666666666666\n" + ] + } + ], + "source": [ + "pmf = Pmf()\n", + "for x in [1,2,3,4,5,6]:\n", + " pmf.Set(x, 1/6.0)\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Pmf` creates an empty Pmf with no values. The `Set` method sets the\n", + "probability associated with each value to $1/6$.\n", + "\n", + "Here’s another example that counts the number of times each word appears\n", + "in a sequence:" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "bye 1\n", + "football 1\n", + "hi 2\n", + "sky 1\n", + "the 1\n" + ] + } + ], + "source": [ + "word_list = ['hi', 'the', 'bye', 'hi', 'football', 'sky']\n", + "pmf = Pmf()\n", + "for word in word_list:\n", + " pmf.Incr(word, 1)\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Incr` increases the “probability” associated with each word by 1. If a\n", + "word is not already in the Pmf, it is added.\n", + "\n", + "I put “probability” in quotes because in this example, the probabilities\n", + "are not normalized; that is, they do not add up to 1. So they are not\n", + "true probabilities.\n", + "\n", + "But in this example the word counts are proportional to the\n", + "probabilities. So after we count all the words, we can compute\n", + "probabilities by dividing through by the total number of words.\n", + "Pmf provides a method, `Normalize`, that does exactly that:" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "bye 0.16666666666666666\n", + "football 0.16666666666666666\n", + "hi 0.3333333333333333\n", + "sky 0.16666666666666666\n", + "the 0.16666666666666666\n" + ] + } + ], + "source": [ + "pmf.Normalize()\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once you have a Pmf object, you can ask for the probability associated\n", + "with any value:" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.16666666666666666\n" + ] + } + ], + "source": [ + "print(pmf.Prob('the'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And that would print the frequency of the word “the” as a fraction of\n", + "the words in the list.\n", + "\n", + "Pmf uses a Python dictionary to store the values and their\n", + "probabilities, so the values in the Pmf can be any hashable type. The\n", + "probabilities can be any numerical type, but they are usually\n", + "floating-point numbers (type `float`)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The cookie problem\n", + "\n", + "In the context of Bayes’s theorem, it is natural to use a Pmf to map\n", + "from each hypothesis to its probability. In the cookie problem, the\n", + "hypotheses are $B_1$ and $B_2$. In Python, I represent them with\n", + "strings:" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pmf = Pmf()\n", + "pmf.Set('Bowl 1', 0.5)\n", + "pmf.Set('Bowl 2', 0.5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This distribution, which contains the priors for each hypothesis, is\n", + "called (wait for it) the **prior distribution**.\n", + "\n", + "To update the distribution based on new data (the vanilla cookie), we\n", + "multiply each prior by the corresponding likelihood. The likelihood of\n", + "drawing a vanilla cookie from Bowl 1 is 3/4. The likelihood for Bowl 2\n", + "is 1/2." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pmf.Mult('Bowl 1', 0.75)\n", + "pmf.Mult('Bowl 2', 0.5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Mult` does what you would expect. It gets the probability for the given\n", + "hypothesis and multiplies by the given likelihood.\n", + "\n", + "After this update, the distribution is no longer normalized, but because\n", + "these hypotheses are mutually exclusive and collectively exhaustive, we\n", + "can **renormalize**:" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.625" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pmf.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The result is a distribution that contains the posterior probability for\n", + "each hypothesis, which is called (wait now) the **posterior\n", + "distribution**.\n", + "\n", + "Finally, we can get the posterior probability for Bowl 1:" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.6000000000000001\n" + ] + } + ], + "source": [ + "print(pmf.Prob('Bowl 1'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the answer is 0.6. You can download this example from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The Bayesian framework {#framework}\n", + "\n", + "Before we go on to other problems, I want to rewrite the code from the\n", + "previous section to make it more general. First I’ll define a class to\n", + "encapsulate the code related to this problem:" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "class Cookie(Pmf):\n", + " \"\"\"A map from string bowl ID to probablity.\"\"\"\n", + "\n", + " def __init__(self, hypos):\n", + " \"\"\"Initialize self.\n", + "\n", + " hypos: sequence of string bowl IDs\n", + " \"\"\"\n", + " Pmf.__init__(self)\n", + " for hypo in hypos:\n", + " self.Set(hypo, 1)\n", + " self.Normalize()\n", + "\n", + " def Update(self, data):\n", + " \"\"\"Updates the PMF with new data.\n", + "\n", + " data: string cookie type\n", + " \"\"\"\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " self.Normalize()\n", + "\n", + " mixes = {\n", + " 'Bowl 1':dict(vanilla=0.75, chocolate=0.25),\n", + " 'Bowl 2':dict(vanilla=0.5, chocolate=0.5),\n", + " }\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " \"\"\"The likelihood of the data under the hypothesis.\n", + "\n", + " data: string cookie type\n", + " hypo: string bowl ID\n", + " \"\"\"\n", + " mix = self.mixes[hypo]\n", + " like = mix[data]\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A Cookie object is a Pmf that maps from hypotheses to their\n", + "probabilities. The `__init__` method gives each hypothesis the same\n", + "prior probability. As in the previous section, there are two hypotheses:" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hypos = ['Bowl 1', 'Bowl 2']\n", + "pmf = Cookie(hypos)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Cookie` provides an `Update` method that takes data as a parameter and\n", + "updates the probabilities. `Update` loops through each hypothesis in the suite and multiplies its\n", + "probability by the likelihood of the data under the hypothesis, which is\n", + "computed by `Likelihood`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Likelihood` uses `mixes`, which is a dictionary that maps from the name\n", + "of a bowl to the mix of cookies in the bowl.\n", + "\n", + "Here’s what the update looks like:" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pmf.Update('vanilla')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And then we can print the posterior probability of each hypothesis:" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Bowl 2 0.4\n", + "Bowl 1 0.6000000000000001\n" + ] + } + ], + "source": [ + "for hypo, prob in pmf.Items():\n", + " print(hypo, prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "which is the same as what we got before. This code is more complicated\n", + "than what we saw in the previous section. One advantage is that it\n", + "generalizes to the case where we draw more than one cookie from the same\n", + "bowl (with replacement):" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Bowl 1 0.6549865229110512\n", + "Bowl 2 0.34501347708894875\n" + ] + } + ], + "source": [ + "dataset = ['vanilla', 'chocolate', 'vanilla']\n", + "for data in dataset:\n", + " pmf.Update(data)\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The other advantage is that it provides a framework for solving many\n", + "similar problems. In the next section we’ll solve the Monty Hall problem\n", + "computationally and then see what parts of the framework are the same.\n", + "\n", + "The code in this section is available from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Monty Hall problem\n", + "----------------------\n", + "\n", + "To solve the Monty Hall problem, I’ll define a new class:" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Monty(Pmf):\n", + " \"\"\"Map from string location of car to probability\"\"\"\n", + "\n", + " def __init__(self, hypos):\n", + " \"\"\"Initialize the distribution.\n", + "\n", + " hypos: sequence of hypotheses\n", + " \"\"\"\n", + " Pmf.__init__(self)\n", + " for hypo in hypos:\n", + " self.Set(hypo, 1)\n", + " self.Normalize()\n", + "\n", + " def Update(self, data):\n", + " \"\"\"Updates each hypothesis based on the data.\n", + "\n", + " data: any representation of the data\n", + " \"\"\"\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " self.Normalize()\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " \"\"\"Compute the likelihood of the data under the hypothesis.\n", + "\n", + " hypo: string name of the door where the prize is\n", + " data: string name of the door Monty opened\n", + " \"\"\"\n", + " if hypo == data:\n", + " return 0\n", + " elif hypo == 'A':\n", + " return 0.5\n", + " else:\n", + " return 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So far `Monty` and `Cookie` are nearly the same. And the code that\n", + "creates the Pmf is the same, too, except for the names of the\n", + "hypotheses:" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "hypos = 'ABC'\n", + "pmf = Monty(hypos)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Calling `Update` is pretty much the same:" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "data = 'B'\n", + "pmf.Update(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the implementation of `Update` is exactly the same. The only part that requires some work is `Likelihood`:\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " if hypo == data:\n", + " return 0\n", + " elif hypo == 'A':\n", + " return 0.5\n", + " else:\n", + " return 1\n", + "```\n", + "Finally, printing the results is the same:" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B 0.0\n", + "A 0.3333333333333333\n", + "C 0.6666666666666666\n" + ] + } + ], + "source": [ + "for hypo, prob in pmf.Items():\n", + " print(hypo, prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, writing `Likelihood` is a little complicated, but the\n", + "framework of the Bayesian update is simple. The code in this section is\n", + "available from . For more information\n", + "see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Encapsulating the framework\n", + "---------------------------\n", + "\n", + "Now that we see what elements of the framework are the same, we can\n", + "encapsulate them in an object—a `Suite` is a `Pmf` that provides\n", + "`__init__`, `Update`, and `Print`:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Suite(Pmf):\n", + " \"\"\"Represents a suite of hypotheses and their probabilities.\"\"\"\n", + "\n", + " def __init__(self, hypo=tuple()):\n", + " \"\"\"Initializes the distribution.\"\"\"\n", + "\n", + " def Update(self, data):\n", + " \"\"\"Updates each hypothesis based on the data.\"\"\"\n", + "\n", + " def Print(self):\n", + " \"\"\"Prints the hypotheses and their probabilities.\"\"\"\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The implementation of `Suite` is in `thinkbayes.py`. To use `Suite`, you\n", + "should write a class that inherits from it and provides `Likelihood`.\n", + "For example, here is the solution to the Monty Hall problem rewritten to\n", + "use `Suite`:" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "from thinkbayes import Suite\n", + "\n", + "class Monty(Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " if hypo == data:\n", + " return 0\n", + " elif hypo == 'A':\n", + " return 0.5\n", + " else:\n", + " return 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And here’s the code that uses this class:" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A 0.3333333333333333\n", + "B 0.0\n", + "C 0.6666666666666666\n" + ] + } + ], + "source": [ + "suite = Monty('ABC')\n", + "suite.Update('B')\n", + "suite.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can download this example from .\n", + "For more information see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The M&M problem\n", + "----------------------------\n", + "\n", + "We can use the `Suite` framework to solve the M&M problem.\n", + "Writing the `Likelihood` function is tricky, but everything else is\n", + "straightforward." + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class M_and_M(Suite):\n", + " \"\"\"Map from hypothesis (A or B) to probability.\"\"\"\n", + "\n", + " mix94 = dict(brown=30,\n", + " yellow=20,\n", + " red=20,\n", + " green=10,\n", + " orange=10,\n", + " tan=10)\n", + "\n", + " mix96 = dict(blue=24,\n", + " green=20,\n", + " orange=16,\n", + " yellow=14,\n", + " red=13,\n", + " brown=13)\n", + "\n", + " hypoA = dict(bag1=mix94, bag2=mix96)\n", + " hypoB = dict(bag1=mix96, bag2=mix94)\n", + "\n", + " hypotheses = dict(A=hypoA, B=hypoB)\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " \"\"\"Computes the likelihood of the data under the hypothesis.\n", + "\n", + " hypo: string hypothesis (A or B)\n", + " data: tuple of string bag, string color\n", + " \"\"\"\n", + " bag, color = data\n", + " mix = self.hypotheses[hypo][bag]\n", + " like = mix[color]\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First I need to encode the color mixes from before and after 1995:\n", + "\n", + "```python\n", + "mix94 = dict(brown=30,\n", + " yellow=20,\n", + " ...\n", + "\n", + "mix96 = dict(blue=24,\n", + " green=20,\n", + " ...\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then I have to encode the hypotheses:\n", + "\n", + "```python\n", + "hypoA = dict(bag1=mix94, bag2=mix96)\n", + "hypoB = dict(bag1=mix96, bag2=mix94)\n", + "```\n", + "\n", + "`hypoA` represents the hypothesis that Bag 1 is from 1994 and Bag 2 from 1996. `hypoB` is the other way around.\n", + "\n", + "Next I map from the name of the hypothesis to the representation:\n", + "\n", + "```python\n", + "hypotheses = dict(A=hypoA, B=hypoB)\n", + "```\n", + "\n", + "And finally I can write `Likelihood`. In this case the hypothesis,\n", + "`hypo`, is a string, either `A` or `B`. The data is a tuple that\n", + "specifies a bag and a color.\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " bag, color = data\n", + " mix = self.hypotheses[hypo][bag]\n", + " like = mix[color]\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here’s the code that creates the suite and updates it:" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A 0.7407407407407407\n", + "B 0.2592592592592592\n" + ] + } + ], + "source": [ + "suite = M_and_M('AB')\n", + "\n", + "suite.Update(('bag1', 'yellow'))\n", + "suite.Update(('bag2', 'green'))\n", + "\n", + "suite.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The posterior probability of A is approximately $20/27$, which is what\n", + "we got before.\n", + "\n", + "The code in this section is available from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Discussion\n", + "----------\n", + "\n", + "This chapter presents the Suite class, which encapsulates the Bayesian\n", + "update framework.\n", + "\n", + "Suite is an **abstract type**, which means\n", + "that it defines the interface a Suite is supposed to have, but does not\n", + "provide a complete implementation. The Suite interface\n", + "includes Update and Likelihood, but the\n", + "Suite class only provides an implementation of\n", + "Update, not Likelihood.\n", + "\n", + "A **concrete type** is a class that extends an abstract\n", + "parent class and provides an implementation of the missing methods. For\n", + "example, Monty extends Suite, so it inherits\n", + "Update and provides Likelihood.\n", + "\n", + "If you are familiar with design patterns, you might recognize this as an\n", + "example of the template method pattern. You can read about this pattern\n", + "at .\n", + "\n", + "Most of the examples in the following chapters follow the same pattern;\n", + "for each problem we define a new class that extends Suite,\n", + "inherits Update, and provides Likelihood. In a\n", + "few cases we override Update, usually to improve\n", + "performance." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Exercises\n", + "---------\n", + "\n", + "In Section [framework] I said that the solution to the cookie problem\n", + "generalizes to the case where we draw multiple cookies with replacement.\n", + "\n", + "But in the more likely scenario where we eat the cookies we draw, the\n", + "likelihood of each draw depends on the previous draws.\n", + "\n", + "Modify the solution in this chapter to handle selection without\n", + "replacement. Hint: add instance variables to Cookie to\n", + "represent the hypothetical state of the bowls, and modify\n", + "Likelihood accordingly. You might want to define a\n", + "Bowl object." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/03_Estimation.ipynb b/notebooks/03_Estimation.ipynb new file mode 100644 index 0000000..e35e9e4 --- /dev/null +++ b/notebooks/03_Estimation.ipynb @@ -0,0 +1,495 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Estimation\n", + "==========\n", + "\n", + "The dice problem\n", + "----------------\n", + "\n", + "Suppose I have a box of dice that contains a 4-sided die, a 6-sided die,\n", + "an 8-sided die, a 12-sided die, and a 20-sided die. If you have ever\n", + "played *Dungeons & Dragons*, you know what I am talking\n", + "about.\n", + "\n", + "Suppose I select a die from the box at random, roll it, and get a 6.\n", + "What is the probability that I rolled each die?\n", + "\n", + "Let me suggest a three-step strategy for approaching a problem like\n", + "this.\n", + "\n", + "1. Choose a representation for the hypotheses.\n", + "\n", + "2. Choose a representation for the data.\n", + "\n", + "3. Write the likelihood function.\n", + "\n", + "In previous examples I used strings to represent hypotheses and data,\n", + "but for the die problem I’ll use numbers. Specifically, I’ll use the\n", + "integers 4, 6, 8, 12, and 20 to represent hypotheses:\n", + "\n", + " suite = Dice([4, 6, 8, 12, 20])\n", + "\n", + "And integers from 1 to 20 for the data. These representations make it\n", + "easy to write the likelihood function:\n", + "\n", + " class Dice(Suite):\n", + " def Likelihood(self, data, hypo):\n", + " if hypo < data:\n", + " return 0\n", + " else:\n", + " return 1.0/hypo\n", + "\n", + "Here’s how `Likelihood` works. If `hypohypo\n", + "sides, what is the chance of rolling data?” The answer is\n", + "`1/hypo`, regardless of data.\n", + "\n", + "Here is the statement that does the update (if I roll a 6):\n", + "\n", + " suite.Update(6)\n", + "\n", + "And here is the posterior distribution:\n", + "\n", + " 4 0.0\n", + " 6 0.392156862745\n", + " 8 0.294117647059\n", + " 12 0.196078431373\n", + " 20 0.117647058824\n", + "\n", + "After we roll a 6, the probability for the 4-sided die is 0. The most\n", + "likely alternative is the 6-sided die, but there is still almost a 12%\n", + "chance for the 20-sided die.\n", + "\n", + "What if we roll a few more times and get 6, 8, 7, 7, 5, and 4?\n", + "\n", + " for roll in [6, 8, 7, 7, 5, 4]:\n", + " suite.Update(roll)\n", + "\n", + "With this data the 6-sided die is eliminated, and the 8-sided die seems\n", + "quite likely. Here are the results:\n", + "\n", + " 4 0.0\n", + " 6 0.0\n", + " 8 0.943248453672\n", + " 12 0.0552061280613\n", + " 20 0.0015454182665\n", + "\n", + "Now the probability is 94% that we are rolling the 8-sided die, and less\n", + "than 1% for the 20-sided die.\n", + "\n", + "The dice problem is based on an example I saw in Sanjoy Mahajan’s class\n", + "on Bayesian inference. You can download the code in this section from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "The locomotive problem\n", + "----------------------\n", + "\n", + "I found the locomotive problem in Frederick Mosteller’s, *Fifty\n", + "Challenging Problems in Probability with Solutions* (Dover,\n", + "1987):\n", + "\n", + "> “A railroad numbers its locomotives in order 1..N. One day you see a\n", + "> locomotive with the number 60. Estimate how many locomotives the\n", + "> railroad has.”\n", + "\n", + "Based on this observation, we know the railroad has 60 or more\n", + "locomotives. But how many more? To apply Bayesian reasoning, we can\n", + "break this problem into two steps:\n", + "\n", + "1. What did we know about $N$ before we saw the data?\n", + "\n", + "2. For any given value of $N$, what is the likelihood of seeing the\n", + " data (a locomotive with number 60)?\n", + "\n", + "The answer to the first question is the prior. The answer to the second\n", + "is the likelihood.\n", + "\n", + "![Posterior distribution for the locomotive problem, based on a uniform\n", + "prior.](figs/train1.pdf)\n", + "\n", + "[fig.train1]\n", + "\n", + "We don’t have much basis to choose a prior, but we can start with\n", + "something simple and then consider alternatives. Let’s assume that $N$\n", + "is equally likely to be any value from 1 to 1000.\n", + "\n", + " hypos = xrange(1, 1001)\n", + "\n", + "Now all we need is a likelihood function. In a hypothetical fleet of $N$\n", + "locomotives, what is the probability that we would see number 60? If we\n", + "assume that there is only one train-operating company (or only one we\n", + "care about) and that we are equally likely to see any of its\n", + "locomotives, then the chance of seeing any particular locomotive is\n", + "$1/N$.\n", + "\n", + "Here’s the likelihood function:\n", + "\n", + " class Train(Suite):\n", + " def Likelihood(self, data, hypo):\n", + " if hypo < data:\n", + " return 0\n", + " else:\n", + " return 1.0/hypo\n", + "\n", + "This might look familiar; the likelihood functions for the locomotive\n", + "problem and the dice problem are identical.\n", + "\n", + "Here’s the update:\n", + "\n", + " suite = Train(hypos)\n", + " suite.Update(60)\n", + "\n", + "There are too many hypotheses to print, so I plotted the results in\n", + "Figure [fig.train1]. Not surprisingly, all values of $N$ below 60 have\n", + "been eliminated.\n", + "\n", + "The most likely value, if you had to guess, is 60. That might not seem\n", + "like a very good guess; after all, what are the chances that you just\n", + "happened to see the train with the highest number? Nevertheless, if you\n", + "want to maximize the chance of getting the answer exactly right, you\n", + "should guess 60.\n", + "\n", + "But maybe that’s not the right goal. An alternative is to compute the\n", + "mean of the posterior distribution:\n", + "\n", + " def Mean(suite):\n", + " total = 0\n", + " for hypo, prob in suite.Items():\n", + " total += hypo * prob\n", + " return total\n", + "\n", + " print Mean(suite)\n", + "\n", + "Or you could use the very similar method provided by Pmf:\n", + "\n", + " print suite.Mean()\n", + "\n", + "The mean of the posterior is 333, so that might be a good guess if you\n", + "wanted to minimize error. If you played this guessing game over and\n", + "over, using the mean of the posterior as your estimate would minimize\n", + "the mean squared error over the long run (see\n", + ").\n", + "\n", + "You can download this example from . For\n", + "more information see Section [download].\n", + "\n", + "What about that prior?\n", + "----------------------\n", + "\n", + "To make any progress on the locomotive problem we had to make\n", + "assumptions, and some of them were pretty arbitrary. In particular, we\n", + "chose a uniform prior from 1 to 1000, without much justification for\n", + "choosing 1000, or for choosing a uniform distribution.\n", + "\n", + "It is not crazy to believe that a railroad company might operate 1000\n", + "locomotives, but a reasonable person might guess more or fewer. So we\n", + "might wonder whether the posterior distribution is sensitive to these\n", + "assumptions. With so little data—only one observation—it probably is.\n", + "\n", + "Recall that with a uniform prior from 1 to 1000, the mean of the\n", + "posterior is 333. With an upper bound of 500, we get a posterior mean of\n", + "207, and with an upper bound of 2000, the posterior mean is 552.\n", + "\n", + "So that’s bad. There are two ways to proceed:\n", + "\n", + "- Get more data.\n", + "\n", + "- Get more background information.\n", + "\n", + "With more data, posterior distributions based on different priors tend\n", + "to converge. For example, suppose that in addition to train 60 we also\n", + "see trains 30 and 90. We can update the distribution like this:\n", + "\n", + " for data in [60, 30, 90]:\n", + " suite.Update(data)\n", + "\n", + "With these data, the means of the posteriors are\n", + "\n", + " ------- -----------\n", + " Upper Posterior\n", + " Bound Mean\n", + " 500 152\n", + " 1000 164\n", + " 2000 171\n", + " ------- -----------\n", + "\n", + "So the differences are smaller.\n", + "\n", + "An alternative prior\n", + "--------------------\n", + "\n", + "![Posterior distribution based on a power law prior, compared to a\n", + "uniform prior.](figs/train4.pdf)\n", + "\n", + "[fig.train4]\n", + "\n", + "If more data are not available, another option is to improve the priors\n", + "by gathering more background information. It is probably not reasonable\n", + "to assume that a train-operating company with 1000 locomotives is just\n", + "as likely as a company with only 1.\n", + "\n", + "With some effort, we could probably find a list of companies that\n", + "operate locomotives in the area of observation. Or we could interview an\n", + "expert in rail shipping to gather information about the typical size of\n", + "companies.\n", + "\n", + "But even without getting into the specifics of railroad economics, we\n", + "can make some educated guesses. In most fields, there are many small\n", + "companies, fewer medium-sized companies, and only one or two very large\n", + "companies. In fact, the distribution of company sizes tends to follow a\n", + "power law, as Robert Axtell reports in *Science* (see\n", + ").\n", + "\n", + "This law suggests that if there are 1000 companies with fewer than 10\n", + "locomotives, there might be 100 companies with 100 locomotives, 10\n", + "companies with 1000, and possibly one company with 10,000 locomotives.\n", + "\n", + "Mathematically, a power law means that the number of companies with a\n", + "given size is inversely proportional to size, or\n", + "$${\\mathrm{PMF}}(x) \\propto \\left( \\frac{1}{x} \\right)^{\\alpha}$$ where\n", + "${\\mathrm{PMF}}(x)$ is the probability mass function of $x$ and $\\alpha$\n", + "is a parameter that is often near 1.\n", + "\n", + "We can construct a power law prior like this:\n", + "\n", + " class Train(Dice):\n", + "\n", + " def __init__(self, hypos, alpha=1.0):\n", + " Pmf.__init__(self)\n", + " for hypo in hypos:\n", + " self.Set(hypo, hypo**(-alpha))\n", + " self.Normalize()\n", + "\n", + "And here’s the code that constructs the prior:\n", + "\n", + " hypos = range(1, 1001)\n", + " suite = Train(hypos)\n", + "\n", + "Again, the upper bound is arbitrary, but with a power law prior, the\n", + "posterior is less sensitive to this choice.\n", + "\n", + "Figure [fig.train4] shows the new posterior based on the power law,\n", + "compared to the posterior based on the uniform prior. Using the\n", + "background information represented in the power law prior, we can all\n", + "but eliminate values of $N$ greater than 700.\n", + "\n", + "If we start with this prior and observe trains 30, 60, and 90, the means\n", + "of the posteriors are\n", + "\n", + " ------- -----------\n", + " Upper Posterior\n", + " Bound Mean\n", + " 500 131\n", + " 1000 133\n", + " 2000 134\n", + " ------- -----------\n", + "\n", + "Now the differences are much smaller. In fact, with an arbitrarily large\n", + "upper bound, the mean converges on 134.\n", + "\n", + "So the power law prior is more realistic, because it is based on general\n", + "information about the size of companies, and it behaves better in\n", + "practice.\n", + "\n", + "You can download the examples in this section from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Credible intervals {#credible}\n", + "------------------\n", + "\n", + "Once you have computed a posterior distribution, it is often useful to\n", + "summarize the results with a single point estimate or an interval. For\n", + "point estimates it is common to use the mean, median, or the value with\n", + "maximum likelihood.\n", + "\n", + "For intervals we usually report two values computed so that there is a\n", + "90% chance that the unknown value falls between them (or any other\n", + "probability). These values define a **credible interval**.\n", + "\n", + "A simple way to compute a credible interval is to add up the\n", + "probabilities in the posterior distribution and record the values that\n", + "correspond to probabilities 5% and 95%. In other words, the 5th and 95th\n", + "percentiles.\n", + "\n", + "`thinkbayes` provides a function that computes percentiles:\n", + "\n", + " def Percentile(pmf, percentage):\n", + " p = percentage / 100.0\n", + " total = 0\n", + " for val, prob in pmf.Items():\n", + " total += prob\n", + " if total >= p:\n", + " return val \n", + "\n", + "And here’s the code that uses it:\n", + "\n", + " interval = Percentile(suite, 5), Percentile(suite, 95)\n", + " print interval\n", + "\n", + "For the previous example—the locomotive problem with a power law prior\n", + "and three trains—the 90% credible interval is $(91, 243)$. The width of\n", + "this range suggests, correctly, that we are still quite uncertain about\n", + "how many locomotives there are.\n", + "\n", + "Cumulative distribution functions\n", + "---------------------------------\n", + "\n", + "In the previous section we computed percentiles by iterating through the\n", + "values and probabilities in a Pmf. If we need to compute more than a few\n", + "percentiles, it is more efficient to use a cumulative distribution\n", + "function, or Cdf.\n", + "\n", + "Cdfs and Pmfs are equivalent in the sense that they contain the same\n", + "information about the distribution, and you can always convert from one\n", + "to the other. The advantage of the Cdf is that you can compute\n", + "percentiles more efficiently.\n", + "\n", + "thinkbayes provides a Cdf class that\n", + "represents a cumulative distribution function. Pmf provides\n", + "a method that makes the corresponding Cdf:\n", + "\n", + " cdf = suite.MakeCdf()\n", + "\n", + "And Cdf provides a function named `Percentile`\n", + "\n", + " interval = cdf.Percentile(5), cdf.Percentile(95)\n", + "\n", + "Converting from a Pmf to a Cdf takes time proportional to the number of\n", + "values, len(pmf). The Cdf stores the values and\n", + "probabilities in sorted lists, so looking up a probability to get the\n", + "corresponding value takes “log time”: that is, time proportional to the\n", + "logarithm of the number of values. Looking up a value to get the\n", + "corresponding probability is also logarithmic, so Cdfs are efficient for\n", + "many calculations.\n", + "\n", + "The examples in this section are in .\n", + "For more information see Section [download].\n", + "\n", + "The German tank problem\n", + "-----------------------\n", + "\n", + "During World War II, the Economic Warfare Division of the American\n", + "Embassy in London used statistical analysis to estimate German\n", + "production of tanks and other equipment.[^2]\n", + "\n", + "The Western Allies had captured log books, inventories, and repair\n", + "records that included chassis and engine serial numbers for individual\n", + "tanks.\n", + "\n", + "Analysis of these records indicated that serial numbers were allocated\n", + "by manufacturer and tank type in blocks of 100 numbers, that numbers in\n", + "each block were used sequentially, and that not all numbers in each\n", + "block were used. So the problem of estimating German tank production\n", + "could be reduced, within each block of 100 numbers, to a form of the\n", + "locomotive problem.\n", + "\n", + "Based on this insight, American and British analysts produced estimates\n", + "substantially lower than estimates from other forms of intelligence. And\n", + "after the war, records indicated that they were substantially more\n", + "accurate.\n", + "\n", + "They performed similar analyses for tires, trucks, rockets, and other\n", + "equipment, yielding accurate and actionable economic intelligence.\n", + "\n", + "The German tank problem is historically interesting; it is also a nice\n", + "example of real-world application of statistical estimation. So far many\n", + "of the examples in this book have been toy problems, but it will not be\n", + "long before we start solving real problems. I think it is an advantage\n", + "of Bayesian analysis, especially with the computational approach we are\n", + "taking, that it provides such a short path from a basic introduction to\n", + "the research frontier.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "Among Bayesians, there are two approaches to choosing prior\n", + "distributions. Some recommend choosing the prior that best represents\n", + "background information about the problem; in that case the prior is said\n", + "to be **informative**. The problem with using an\n", + "informative prior is that people might use different background\n", + "information (or interpret it differently). So informative priors often\n", + "seem subjective.\n", + "\n", + "The alternative is a so-called **uninformative prior**,\n", + "which is intended to be as unrestricted as possible, in order to let the\n", + "data speak for themselves. In some cases you can identify a unique prior\n", + "that has some desirable property, like representing minimal prior\n", + "information about the estimated quantity.\n", + "\n", + "Uninformative priors are appealing because they seem more objective. But\n", + "I am generally in favor of using informative priors. Why? First,\n", + "Bayesian analysis is always based on modeling decisions. Choosing the\n", + "prior is one of those decisions, but it is not the only one, and it\n", + "might not even be the most subjective. So even if an uninformative prior\n", + "is more objective, the entire analysis is still subjective.\n", + "\n", + "Also, for most practical problems, you are likely to be in one of two\n", + "regimes: either you have a lot of data or not very much. If you have a\n", + "lot of data, the choice of the prior doesn’t matter very much;\n", + "informative and uninformative priors yield almost the same results.\n", + "We’ll see an example like this in the next chapter.\n", + "\n", + "But if, as in the locomotive problem, you don’t have much data, using\n", + "relevant background information (like the power law distribution) makes\n", + "a big difference.\n", + "\n", + "And if, as in the German tank problem, you have to make life-and-death\n", + "decisions based on your results, you should probably use all of the\n", + "information at your disposal, rather than maintaining the illusion of\n", + "objectivity by pretending to know less than you do.\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "To write a likelihood function for the locomotive problem, we had to\n", + "answer this question: “If the railroad has $N$ locomotives, what is the\n", + "probability that we see number 60?”\n", + "\n", + "The answer depends on what sampling process we use when we observe the\n", + "locomotive. In this chapter, I resolved the ambiguity by specifying that\n", + "there is only one train-operating company (or only one that we care\n", + "about).\n", + "\n", + "But suppose instead that there are many companies with different numbers\n", + "of trains. And suppose that you are equally likely to see any train\n", + "operated by any company. In that case, the likelihood function is\n", + "different because you are more likely to see a train operated by a large\n", + "company.\n", + "\n", + "As an exercise, implement the likelihood function for this variation of\n", + "the locomotive problem, and compare the results.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/04_More_Estimation.ipynb b/notebooks/04_More_Estimation.ipynb new file mode 100644 index 0000000..17b5178 --- /dev/null +++ b/notebooks/04_More_Estimation.ipynb @@ -0,0 +1,839 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The autoreload extension is already loaded. To reload it, use:\n", + " %reload_ext autoreload\n" + ] + } + ], + "source": [ + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "%load_ext autoreload\n", + "%autoreload 2 \n", + "import sys\n", + "sys.path.insert(0,'../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "More Estimation {#more}\n", + "===============\n", + "\n", + "The Euro problem {#euro}\n", + "----------------\n", + "\n", + "In *Information Theory, Inference, and Learning\n", + "Algorithms*, David MacKay poses this problem:\n", + "\n", + "> A statistical statement appeared in \\`\\`The Guardian\" on Friday\n", + "> January 4, 2002:\n", + ">\n", + "> > When spun on edge 250 times, a Belgian one-euro coin came up heads\n", + "> > 140 times and tails 110. ‘It looks very suspicious to me,’ said\n", + "> > Barry Blight, a statistics lecturer at the London School of\n", + "> > Economics. ‘If the coin were unbiased, the chance of getting a\n", + "> > result as extreme as that would be less than 7%.’\n", + ">\n", + "> But do these data give evidence that the coin is biased rather than\n", + "> fair?\n", + "\n", + "To answer that question, we’ll proceed in two steps. The first is to\n", + "estimate the probability that the coin lands face up. The second is to\n", + "evaluate whether the data support the hypothesis that the coin is\n", + "biased.\n", + "\n", + "You can download the code in this section from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Any given coin has some probability, $x$, of landing heads up when spun\n", + "on edge. It seems reasonable to believe that the value of $x$ depends on\n", + "some physical characteristics of the coin, primarily the distribution of\n", + "weight.\n", + "\n", + "If a coin is perfectly balanced, we expect $x$ to be close to 50%, but\n", + "for a lopsided coin, $x$ might be substantially different. We can use\n", + "Bayes’s theorem and the observed data to estimate $x$.\n", + "\n", + "Let’s define 101 hypotheses, where $H_x$ is the hypothesis that the\n", + "probability of heads is $x$%, for values from 0 to 100. I’ll start with\n", + "a uniform prior where the probability of $H_x$ is the same for all $x$.\n", + "We’ll come back later to consider other priors.\n", + "\n", + "[Posterior distribution for the Euro problem on a uniform\n", + "prior.](figs/euro1.pdf)\n", + "\n", + "[fig.euro1]\n", + "\n", + "The likelihood function is relatively easy: If $H_x$ is true, the\n", + "probability of heads is $x/100$ and the probability of tails is $1-\n", + "x/100$." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from thinkbayes import Suite\n", + "class Euro(Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = hypo\n", + " if data == 'H':\n", + " return x/100.0\n", + " else:\n", + " return 1 - x/100.0" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here’s the code that makes the suite and updates it:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "suite = Euro(range(0, 101))\n", + "dataset = 'H' * 140 + 'T' * 110\n", + "\n", + "for data in dataset:\n", + " suite.Update(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The result may be plotted with:" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHCFJREFUeJzt3XmQXPV57vHvw2hjl9i0R8JGFBIGGwoLXS/QuUXuHcsY\n3SpCQI5NjBND5Vo2cSgugbp1PforccouMKWA5SBTBDuRgSQuQVAwWztO4gjJLMFIwhK2QAItICEW\nBaztvX+cHtRqZno9p0/39POpmqL7bP3OKebpn96zKSIwM7PecUTeBZiZWXs5+M3MeoyD38ysxzj4\nzcx6jIPfzKzHOPjNzHpMzeCX1C9pvaQNkm4YYv4Zkn4m6V1J1w0xv0/SU5LuT6toMzNrXtXgl9QH\nLAH6gTnAQkmzKxbbCXwF+OYwm7kWWAv4ggEzsw5Qa8Q/F9gYEZsiYh+wHFhQvkBEvBoRa4B9lStL\nmgbMB+4AlE7JZmbWilrBPxXYXPZ+S2lavW4GrgcONliXmZllpFbwN92ekXQxsCMinsKjfTOzjjGq\nxvyXgell76eTjPrr8THgEknzgXHAcZL+JiKuLF9Iknv/ZmZNiIimBtW1RvxrgFmSZkoaA1wOrBhm\n2cMKiIibImJ6RJwKXAE8Vhn6Zcv6J4Kvf/3rudfQKT/eF94X3hfVf1pRdcQfEfslLQIeAvqAZRGx\nTtI1pflLJU0CVgPHAQclXQvMiYi3KzfXUqVmZpaKWq0eImIlsLJi2tKy19s4vB001DZ+AvykyRrN\nzCxFvnK3gxQKhbxL6BjeF4d4XxzifZEOtdorarkAKfKuwcys20giMjq4a2ZmI4yD38ysxzj4zcx6\njIPfzKzHOPjNzHqMg9/MrMc4+M3MeoyD38ysxzj4zcx6jIPfzKzHOPjNzHqMg9/MrMc4+M3MeoyD\n36xN/vRP4V/+Je8qzOp4EIuZte6tt+C222DHDrjggryrsV7nEb9ZGzzwAJx1FvzTP8G77+ZdjfU6\nB79ZG9xzDyxaBGefDT/+cd7VWK/zE7jMMvbmmzBtGrz0Enz/+7BqFdx9d95VWbfzE7jMOtgDDyR9\n/fHj4dJLk/e/+U3eVVkvc/CbZeyee+Cyy5LXkycnvf6HH863JuttDn6zDL35Jjz+OCxYcGjaZZfB\nvffmV5NZXcEvqV/SekkbJN0wxPwzJP1M0ruSriubPl3S45Kek/QLSV9Ns3izTnf//fDJTyZtnkGX\nXppMd7vH8lIz+CX1AUuAfmAOsFDS7IrFdgJfAb5ZMX0f8LWIOBOYB3x5iHXNRqx774Xf+73Dp02Z\nAmeeCY88kk9NZvWM+OcCGyNiU0TsA5YDC8oXiIhXI2INSdCXT98WEU+XXr8NrAOmpFK5WYd7990k\n3C+55P3zfvd34Uc/an9NZlBf8E8FNpe931Ka1hBJM4FzgFWNrmvWjV56CSZNOrzNM+gjH4H169tf\nkxnUd8uGlk+yl3QMcB9wbWnkf5iBgYH3XhcKBQqFQqsfaZa7TZtgxoyh582YAS++2NZyrMsVi0WK\nxWIq26p5AZekecBARPSX3t8IHIyIbwyx7NeBtyPiW2XTRgMPACsj4pYh1vEFXDYiffe7ycVay5a9\nf96+fXD00bBnD4we3f7arPtlfQHXGmCWpJmSxgCXAyuGq6WiMAHLgLVDhb7ZSPbii8OP+EePTs7p\n37KlvTWZQR3BHxH7gUXAQ8Ba4IcRsU7SNZKuAZA0SdJm4GvA/5X0Uqm983Hgc8BvS3qq9NOf2W9j\n1kE2bYKZM4ef73aP5aWu2zJHxEpgZcW0pWWvtwHTh1j1X/FFYtajXnzRwW+dyaFslpFqB3fBwW/5\ncfCbZWDv3uShK1OrnPg8Y0by5WDWbg5+swxs3pyE/qgqzVSP+C0vDn6zDNRq80DS/3fwWx4c/GYZ\nqHVgF+C3fis5nfPgwbaUZPYeB79ZBmqdygkwblxyO4dt29pRkdkhDn6zDNTT6gEf4LV8OPjNMlBP\nqwd8gNfy4eA3y0A9rR5w8Fs+HPxmKdu/P+nbT5tWe1mf2WN5cPCbpWzLFpg4sb67bnrEb3lw8Jul\nrN42D/jgruXDwW+Wsmq3Y640OOL3IymsnRz8ZilrZMR/3HEwZgzs3JllRWaHc/CbpayRET/4AK+1\nn4PfLGWNjPjBfX5rPwe/WcqaCX6P+K2dHPxmKTpwAF5+GaYP9Ty6YTj4rd0c/GYpeuUVOOkkGDu2\n/nUc/NZuDn6zFNV7j55yDn5rNwe/WYrqvStnuZkzfXDX2svBb5airVthypTG1jnxRHj3XdizJ5ua\nzCo5+M1StH17cp+eRkhwyinJw9nN2qFm8Evql7Re0gZJNwwx/wxJP5P0rqTrGlnXbKRpJvghWWf7\n9vTrMRtK1eCX1AcsAfqBOcBCSbMrFtsJfAX4ZhPrmo0o27Y5+K3z1RrxzwU2RsSmiNgHLAcWlC8Q\nEa9GxBpgX6Prmo0027fDpEmNr+fgt3aqFfxTgc1l77eUptWjlXXNulKzrZ5TTnHwW/uMqjG/lZvF\n1r3uwMDAe68LhQKFQqGFjzXLx4EDsGtXcgFXoyZOhI0b06/JRo5isUixWExlW7WC/2Wg/OLz6SQj\n93rUvW558Jt1q1dfhQkTYFStv6ohTJwI//Zv6ddkI0floHjx4sVNb6tWq2cNMEvSTEljgMuBFcMs\nqxbWNet6zfb3wT1+a6+qY5OI2C9pEfAQ0Acsi4h1kq4pzV8qaRKwGjgOOCjpWmBORLw91LpZ/jJm\neWq2vw8Ofmuvmv8ojYiVwMqKaUvLXm/j8JZO1XXNRqpWgt8XcFk7+cpds5Q0ew4/wAknwFtvwd69\n6dZkNhQHv1lKWunxH3EEnHyyR/3WHg5+s5S00uoB9/mtfRz8Zilx8Fu3cPCbpaSVHj/4AK+1j4Pf\nLCUe8Vu3cPCbpWDwdg0nn9z8Nhz81i4OfrMUtHK7hkEOfmsXB79ZClpt84CD39rHwW+WgjSC3wd3\nrV0c/GYpaOXirUEe8Vu7OPjNUtDqqZyQHBjetSs5UGyWJQe/WQrSaPWMGgXjx8Nrr6VTk9lwHPxm\nKUgj+CHZhvv8ljUHv1kK0ujxg5+9a+3h4DdLQRo9fvABXmsPB79ZCtJs9Tj4LWsOfrMWpXG7hkEO\nfmsHB79Zi157rfXbNQzywV1rBwe/WYvS6u+DD+5aezj4zVqUVn8f3Oqx9nDwm7XIwW/dxsFv1qK0\nzuGHQzdqi0hne2ZDqRn8kvolrZe0QdINwyxza2n+M5LOKZt+o6TnJD0r6W8ljU2zeLNOkGaPf9w4\nOPJI2L07ne2ZDaVq8EvqA5YA/cAcYKGk2RXLzAdOi4hZwNXA7aXpM4EvAedGxFlAH3BFyvWb5S7N\nVg+43WPZqzXinwtsjIhNEbEPWA4sqFjmEuAugIhYBYyXNBF4E9gHHCVpFHAU8HKaxZt1Age/dZta\nwT8V2Fz2fktpWs1lImIX8C3gJeAVYHdEPNJauWadZ8cOB791l1qXnNR7iEnvmyB9EPgTYCbwBnCv\npN+PiB9ULjswMPDe60KhQKFQqPNjzfK3fXtyUDYtEycmz/A1K1csFikWi6lsq1bwvwxML3s/nWRE\nX22ZaaVpBeDfI2IngKR/AD4GVA1+s25y8GBy5W4at2sYdPLJvnrX3q9yULx48eKmt1Wr1bMGmCVp\npqQxwOXAioplVgBXAkiaR9LS2Q48D8yTdKQkARcBa5uu1KwD7doFxx4LY8akt00/e9eyVnXEHxH7\nJS0CHiI5K2dZRKyTdE1p/tKIeFDSfEkbgT3AVaV5T0v6G5Ivj4PAk8B3M/xdzNou7f4+OPgtezVv\nKxURK4GVFdOWVrxfNMy6fwn8ZSsFmnWyHTvS7e+Dg9+y5yt3zVqQ9oFdcPBb9hz8Zi1wq8e6kYPf\nrAVZjPjHj4e334a9e9PdrtkgB79ZC7IY8R9xRHJKp8/lt6w4+M1akMWIH9zusWw5+M1akMWIHxz8\nli0Hv1kLsjidExz8li0Hv1kL3OqxbuTgN2vSnj1w4EByy4a0OfgtSw5+syYNtnn0vnvTts7Bb1ly\n8Js1KasDu+Dgt2w5+M2alNWBXXDwW7Yc/GZNyurALjj4LVsOfrMmZdnqGXwYS9T7DDyzBjj4zZqU\n5Yj/6KOhry+5Z49Z2hz8Zk3KcsQPbvdYdhz8Zk3KcsQPDn7LjoPfrEke8Vu3cvCbNSnL0znBwW/Z\ncfCbNWH/fnj9dTjxxOw+w8FvWXHwmzVh506YMAFGjcruMxz8lhUHv1kTsj6wCw5+y46D36wJWR/Y\nBQe/Zadm8Evql7Re0gZJNwyzzK2l+c9IOqds+nhJ90laJ2mtpHlpFm+WF4/4rZtVDX5JfcASoB+Y\nAyyUNLtimfnAaRExC7gauL1s9reBByNiNnA2sC7F2s1yk/UZPeDgt+zUGvHPBTZGxKaI2AcsBxZU\nLHMJcBdARKwCxkuaKOl44JMR8b3SvP0R8Ua65Zvlox2tnpNOgl27koe9mKWpVvBPBTaXvd9SmlZr\nmWnAqcCrku6U9KSkv5Z0VKsFm3WCdrR6Ro2C449Pwt8sTbVORqv33oCVzyCK0rbPBRZFxGpJtwB/\nBvy/ypUHBgbee10oFCgUCnV+rFk+2jHih0PtnpNPzv6zrLMVi0WKxWIq26oV/C8D08veTycZ0Vdb\nZlppmoAtEbG6NP0+kuB/n/LgN+sG7Rjxw6HgP/PM7D/LOlvloHjx4sVNb6tWq2cNMEvSTEljgMuB\nFRXLrACuBCidtbM7IrZHxDZgs6TTS8tdBDzXdKVmHaTdI36zNFUd8UfEfkmLgIeAPmBZRKyTdE1p\n/tKIeFDSfEkbgT3AVWWb+Arwg9KXxgsV88y6UkT72i8OfstCzQvOI2IlsLJi2tKK94uGWfcZ4KOt\nFGjWad56K3lIytFHZ/9ZDn7Lgq/cNWvQ1q0weXJ7PsvBb1lw8Js16JVXYMqU9nzWKackB5LN0uTg\nN2vQ1q3tC/5Jk2DbtvZ8lvUOB79Zg9o54p8yJfmiMUuTg9+sQe0M/kmTkuA/eLA9n2e9wcFv1qB2\nBv+4cXDsscmDX8zS4uA3a1A7gx/c7rH0OfjNGtTu4J88OflMs7Q4+M0aEJGEcLvO44fkS8bBb2ly\n8Js14I03YPRoOOaY9n3m5Mlu9Vi6HPxmDWh3mwc84rf0OfjNGuDgt5HAwW/WgDyC360eS5uD36wB\nHvHbSODgN2tAHsE/eL8eX71raXHwmzUgj+D31buWNge/WQPyCH5wu8fS5eA3a0Bewe8DvJYmB79Z\nnSLa+/Stch7xW5oc/GZ12rkzuWJ33Lj2f7aD39Lk4DerU15tHnCrx9Ll4DerU57B7xG/pcnBb1an\nvEf8Dn5LS83gl9Qvab2kDZJuGGaZW0vzn5F0TsW8PklPSbo/raLN8pD3iN+tHktL1eCX1AcsAfqB\nOcBCSbMrlpkPnBYRs4CrgdsrNnMtsBaItIo2y0PeI35fvWtpqTXinwtsjIhNEbEPWA4sqFjmEuAu\ngIhYBYyXNBFA0jRgPnAHoDQLN2u3PIN/7Njk6t3XXsvn821kqRX8U4HNZe+3lKbVu8zNwPWAxynW\n9dr95K1KbvdYWkbVmF9ve6ZyNC9JFwM7IuIpSYVqKw8MDLz3ulAoUChUXdwsF3mO+OHQmT0f/nB+\nNVh+isUixWIxlW0pYvhslzQPGIiI/tL7G4GDEfGNsmW+AxQjYnnp/XqgAHwV+DywHxgHHAf8fURc\nWfEZUa0Gs05w4EBy4daePTBmTD41fOELcMEF8MUv5vP51lkkERFNtdBrtXrWALMkzZQ0BrgcWFGx\nzArgylIh84DdEbEtIm6KiOkRcSpwBfBYZeibdYtXX4UJE/ILffC5/Jaeqq2eiNgvaRHwENAHLIuI\ndZKuKc1fGhEPSpovaSOwB7hquM2lWbhZO23dmm+bB5LjC+vW5VuDjQy1evxExEpgZcW0pRXvF9XY\nxk+AnzRToFknyLu/D8nnP/ZYvjXYyOArd83q8PLL+Z7RA271WHoc/GZ12LQJZs7MtwbftsHS4uA3\nq8MLL8AHP5hvDZMnw/btvnrXWufgN6tDJwT/2LEwfnxy6wazVjj4zerQCcEP8IEPJLWYtcLBb1bD\nrl1Je+XEE/OuJPnycfBbqxz8ZjUMjvbVAbcZdPBbGhz8ZjV0SpsHkjp+9au8q7Bu5+A3q6HTgt8j\nfmuVg9+sBge/jTQOfrMaOin4J02C//ovePPNvCuxbubgN6uhk4Jf8imd1joHv1kV77yTPO5w2rS8\nKznE7R5rlYPfrIpf/xpmzIC+vrwrOcTBb61y8JtV0UltnkEOfmuVg9+sCge/jUQOfrMqHPw2Ejn4\nzap44YXkLJpOMmNG8ijIvXvzrsS6lYPfrIpOHPGPHg1TpyYPhzFrhoPfbBgHDsCLL3beiB/c7rHW\nOPjNhrFlS3Ir5iOPzLuS93PwWysc/GbD6MQ2zyAHv7XCwW82DAe/jVR1Bb+kfknrJW2QdMMwy9xa\nmv+MpHNK06ZLelzSc5J+IemraRZvliUHv41UNYNfUh+wBOgH5gALJc2uWGY+cFpEzAKuBm4vzdoH\nfC0izgTmAV+uXNesU3Vy8H/gA8ntJA4ezLsS60b1jPjnAhsjYlNE7AOWAwsqlrkEuAsgIlYB4yVN\njIhtEfF0afrbwDpgSmrVm2Xo+efh9NPzrmJoxx6b/Gzdmncl1o3qCf6pwOay91tK02otc9j9DCXN\nBM4BVjVapFm77dkDGzfChz6UdyXDc7vHmjWqjmWizm1VPor6vfUkHQPcB1xbGvkfZmBg4L3XhUKB\nQqFQ50eaZePJJ+Gss2Ds2LwrGd5pp8GGDXDBBXlXYu1QLBYpFoupbEsR1XNd0jxgICL6S+9vBA5G\nxDfKlvkOUIyI5aX364ELI2K7pNHAA8DKiLhliO1HrRrM2u1b30p66EuW5F3J8G6+OQn+227LuxLL\ngyQionLAXZd6Wj1rgFmSZkoaA1wOrKhYZgVwZamYecDuUugLWAasHSr0zTrV6tUwd27eVVR3/vmw\nyo1Ta0LN4I+I/cAi4CFgLfDDiFgn6RpJ15SWeRD4laSNwFLgf5dW/zjwOeC3JT1V+unP4hcxS9MT\nT3R+8J9zDqxblzwlzKwRNVs9mRfgVo91mNdeSw6cvv46HNHhlziedx58+9vw8Y/nXYm1W9atHrOe\nsnp1EqidHvrgdo81pwv+1zZrryeegI9+NO8q6nP++fAf/5F3FdZtHPxmFbrhwO6gefM84rfGOfjN\nykR0x4HdQbNmwVtvwbZteVdi3cTBb1bmpZegry95wlU3kJIvKY/6rREOfrMyg6N9NXWuRD58gNca\n5eA3K9NNB3YHOfitUQ5+szLddGB30Ny5sGZN8oxgs3o4+M1KDhxIbs523nl5V9KYk06Ck0+G9evz\nrsS6hYPfrGT1apg2DU44Ie9KGufTOq0RDn6zku9/HxYuzLuK5rjPb43wvXrMgL17k1M4n3gCTj01\n72oa95//CZ/5THIr6W641YS1zvfqMWvRP/8znHFGd4Y+wNlnJy2qRx/NuxLrBg5+M+Duu+Hzn8+7\nitb80R/BsmV5V2HdwK0e63m7d8OMGbBpE0yYkHc1zXv99eRfLC+8ACeemHc1ljW3esxacO+98Du/\n092hD0n9F1+cHKQ2q8bBbz1vJLR5Bv3hH8IddyQ3mzMbjoPfetqmTcnjCz/1qbwrSceFFyaPYly9\nOu9KrJM5+K2n3XILXHEFjBmTdyXpOOII+OIXfZDXqvPBXetZjz4KV14JzzyT3PZgpHjlFfjQh2DD\nBh/kHcl8cNesQTt3whe+AHfeObJCH2DKlKTX/9nP+sZtNjSP+K3nRMBll8H06XDzzXlXk439+6G/\nP7nh3F/8Rd7VWBYyHfFL6pe0XtIGSTcMs8ytpfnPSDqnkXXN2u3OO+GXv4Q///O8K8nOqFGwfDn8\n8Idwzz15V2OdpmrwS+oDlgD9wBxgoaTZFcvMB06LiFnA1cDt9a5rhysWi3mX0DGy2Bd79sB118FN\nN8EPfgDjxqX+EZlodl+cdBL84z/CokXw05+mW1Ne/DeSjloj/rnAxojYFBH7gOXAgoplLgHuAoiI\nVcB4SZPqXNfK+H/qQ9LeF48+mtzPZts2ePZZOOusVDefqVb2xUc+kvwL57OfhUsvTU5d7Wb+G0nH\nqBrzpwKby95vAc6vY5mpwJQ61jVL3TvvJAH/5JPwyCNJ6O/dC3/1V/DpT+ddXft9+tNJa2vJkuQ8\n/0IBPvEJOPfc5IvhmGPyrtDarVbw13vUtaVHU3/mM62sPXI8/zz8/Od5V9EZnn8+eZwgHLoKNeLQ\nz8GDyRkr+/fDb36ThP0778COHcl/J02CM8+Eiy6CP/7j5PTGXr5d8ZFHwvXXw5e+lNyi4uc/T27t\n8OyzMHp0cruHCRPgqKOS96NGJT9Sst+k9z+APo8H0vf638jixckXdquqntUjaR4wEBH9pfc3Agcj\n4htly3wHKEbE8tL79cCFwKm11i1N9yk9ZmZNaPasnloj/jXALEkzgVeAy4HKZxStABYBy0tfFLsj\nYruknXWs23ThZmbWnKrBHxH7JS0CHgL6gGURsU7SNaX5SyPiQUnzJW0E9gBXVVs3y1/GzMxqy/0C\nLjMza69cD3f18gVekqZLelzSc5J+IemrpeknSHpY0i8l/VjS+LxrbQdJfZKeknR/6X1P7gcASeMl\n3SdpnaS1ks7vxf0h6cbS38ezkv5W0the2Q+Svidpu6Rny6YN+7uX9tWGUp7+j1rbzy34fYEX+4Cv\nRcSZwDzgy6Xf/8+AhyPidODR0vtecC2wlkNnkvXqfgD4NvBgRMwGzgbW02P7o3Rs8EvAuRFxFkm7\n+Ap6Zz/cSZKN5Yb83SXNITmGOqe0zm2SqmZ7niP+nr7AKyK2RcTTpddvA+tIrn9474K40n//Vz4V\nto+kacB84A4OnRrcc/sBQNLxwCcj4nuQHCuLiDfovf3xJsng6ChJo4CjSE4S6Yn9EBE/BV6vmDzc\n774A+LuI2BcRm4CNJPk6rDyDf7gLv3pOaXRzDrAKmBgR20uztgMTcyqrnW4GrgcOlk3rxf0AyWnQ\nr0q6U9KTkv5a0tH02P6IiF3At4CXSAJ/d0Q8TI/thwrD/e5TSPJzUM0szTP4fVQZkHQM8PfAtRHx\nVvm80m1LR/R+knQxsCMinmKYCwF7YT+UGQWcC9wWEeeSnCl3WDujF/aHpA8CfwLMJAm2YyR9rnyZ\nXtgPw6njd6+6X/IM/peB6WXvp3P4t9aIJ2k0SejfHRE/Kk3eXrrXEZImAzvyqq9NPgZcIunXwN8B\n/13S3fTefhi0BdgSEYMPT7yP5ItgW4/tj/OAf4+InRGxH/gH4L/Re/uh3HB/E5VZOq00bVh5Bv97\nF4dJGkNycGJFjvW0lSQBy4C1EXFL2awVwB+UXv8B8KPKdUeSiLgpIqZHxKkkB+8ei4jP02P7YVBE\nbAM2Szq9NOki4Dngfnprf6wH5kk6svS3chHJwf9e2w/lhvubWAFcIWmMpFOBWcATVbcUEbn9AJ8C\nnic5GHFjnrXk8Lt/gqSn/TTwVOmnHzgBeAT4JfBjYHzetbZxn1wIrCi97uX98GFgNfAMyUj3+F7c\nH8D/IfnSe5bkYOboXtkPJP/6fQXYS3Is9KpqvztwUylH1wP/s9b2fQGXmVmP6eH7FZqZ9SYHv5lZ\nj3Hwm5n1GAe/mVmPcfCbmfUYB7+ZWY9x8JuZ9RgHv5lZj/n/1UdUSlFFFjEAAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt \n", + "xs = list(range(0,100))\n", + "ps = suite.Probs(xs)\n", + "plt.plot(xs, ps);" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Summarizing the posterior\n", + "\n", + "Again, there are several ways to summarize the posterior distribution.\n", + "One option is to find the most likely value in the posterior\n", + "distribution. `thinkbayes` provides a function that does that:\n", + "\n", + "```python\n", + " def MaximumLikelihood(pmf):\n", + " \"\"\"Returns the value with the highest probability.\"\"\"\n", + " prob, val = max((prob, val) for val, prob in pmf.Items())\n", + " return val\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "56" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "suite.MaximumLikelihood()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this case the result is 56, which is also the observed percentage of\n", + "heads, $140/250 = 56\\%$. So that suggests (correctly) that the observed\n", + "percentage is the maximum likelihood estimator for the population.\n", + "\n", + "We might also summarize the posterior by computing the mean and median:" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Mean 55.952380952380956\n", + "Median 56\n" + ] + } + ], + "source": [ + "import thinkbayes\n", + "print('Mean', suite.Mean())\n", + "print('Median', thinkbayes.Percentile(suite, 50))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can compute a credible interval:" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CI (51, 61)\n" + ] + } + ], + "source": [ + "print('CI', thinkbayes.CredibleInterval(suite, 90))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, getting back to the original question, we would like to know\n", + "whether the coin is fair. We observe that the posterior credible\n", + "interval does not include 50%, which suggests that the coin is not fair.\n", + "\n", + "But that is not exactly the question we started with. MacKay asked, “ Do\n", + "these data give evidence that the coin is biased rather than fair?” To\n", + "answer that question, we will have to be more precise about what it\n", + "means to say that data constitute evidence for a hypothesis. And that is\n", + "the subject of the next chapter.\n", + "\n", + "But before we go on, I want to address one possible source of confusion.\n", + "Since we want to know whether the coin is fair, it might be tempting to\n", + "ask for the probability that x is 50%:\n", + "\n", + " print suite.Prob(50)\n", + "\n", + "The result is 0.021, but that value is almost meaningless. The decision\n", + "to evaluate 101 hypotheses was arbitrary; we could have divided the\n", + "range into more or fewer pieces, and if we had, the probability for any\n", + "given hypothesis would be greater or less." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Swamping the priors {#triangle}\n", + "\n", + "We started with a uniform prior, but that might not be a good choice. I\n", + "can believe that if a coin is lopsided, $x$ might deviate substantially\n", + "from 50%, but it seems unlikely that the Belgian Euro coin is so\n", + "imbalanced that $x$ is 10% or 90%.\n", + "\n", + "It might be more reasonable to choose a prior that gives higher\n", + "probability to values of $x$ near 50% and lower probability to extreme\n", + "values.\n", + "\n", + "As an example, I constructed a triangular prior. Here’s the code that constructs the prior:" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def TrianglePrior(suite):\n", + " for x in range(0, 51):\n", + " suite.Set(x, x)\n", + " for x in range(51, 101):\n", + " suite.Set(x, 100-x) \n", + " suite.Normalize()\n", + " return suite" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEACAYAAABcXmojAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X98VPWd7/HXJxhvsRAihkYNKCq0/qi9urYQb0Wo0l2I\ngmItmtQfaLcLCN3W3l4BZRXdPh4WUK56XREosmptQGutICKwaog+3EJxWdsKUX4pSACt8suqbYDP\n/ePMhCEmmQmZyZmZ834+HvPonDPnzHznWOaTc873+32buyMiIgJQEHYDREQke6goiIhIIxUFERFp\npKIgIiKNVBRERKSRioKIiDRKWhTMbIiZ1ZnZejOb0MI2D8Ref8PMzo2t62VmL5vZm2b2JzP754Tt\np5jZe2a2JvYYkr6vJCIiR+qo1l40s07Ag8BgYBvwezNb6O7rErapAPq4e18z6w/MBMqBBuBmd/9v\nM+sCvG5my9y9DnBghrvPyMzXEhGRI5HsTKEfsMHd33H3BmA+cFmTbYYDjwK4+0qg2MxK3X2Hu/93\nbP3HwDqgLGE/S8cXEBGR9ElWFMqArQnL73H4D3tL2/RM3MDMegPnAisTVv8wdrlprpkVt6HNIiKS\nIcmKQqpzYDT9q79xv9ilo18DP4qdMUBwiekU4BxgO3Bvip8jIiIZ1Oo9BYL7CL0SlnsRnAm0tk3P\n2DrMrBB4Gvilu/82voG7vx9/bma/ABY19+FmpomZRETayN2P+PJ8sjOF1UBfM+ttZkcDVwELm2yz\nELgOwMzKgd3uvtPMDJgLrHX3+xJ3MLMTEhZHAH9sqQHuroc7d9xxR+htyIaHjoOOhY5F64/2avVM\nwd33m9l4YCnQCZjr7uvMbHTs9Vnu/ryZVZjZBuAvwA2x3b8JXAP8wczWxNZNcvcXgKlmdg7BZabN\nwOh2fxMREWm3ZJePcPclwJIm62Y1WR7fzH6v0sKZiLtf17ZmiohIR9CI5hwxaNCgsJuQFXQcDtGx\nOETHIn0sHdegMsXMPJvbJyKSbcwMb8eN5qSXj0REjkTQ10QyKRN/NKsoiEjG6Ew/czJVdHVPQURE\nGqkoiIhIIxUFERFppKIgIpKisWPH8rOf/axxeebMmZSWllJUVMSuXbtCbFn6qEuqiGRErGtk2M3I\nmIaGBrp168aqVav46le/2uGf39LxbW+XVJ0piIgcgR07dvDZZ59xxhlntHnfdM1TlAkqCiISKQUF\nBWzatKlxedSoUfzLv/wLADU1NfTs2ZMZM2ZQWlrKiSeeyL//+79/btv169dz+umnA1BcXMzgwYMB\neO211/jGN75BcXEx/fr14z//8z8b9x00aBCTJ0/mm9/8Jl26dGHTpk0UFBQwc+ZM+vbtS1FREbff\nfjsbN27k/PPPp7i4mKuvvpqGhoYOOCqHaJyCiESamR3W53/nzp3s3buX+vp6li1bxpVXXsmIESPo\n1q1b47Z9+/Zl7dq1nHLKKezZs4eCggI++ugjLrnkEh588EEqKyt58sknueSSS9i4cSPHHnssAL/8\n5S9ZsmQJX/nKVzhw4AAAy5YtY82aNWzZsoVzzz2XV199lerqarp37875559PdXU1113XcdPF6UxB\nREIR/4Ft7yMdEi/lFBYWcvvtt9OpUyeGDh1Kly5deOuttz63bdPLP4sXL+YrX/kK3/ve9ygoKODq\nq6/m9NNPZ+HChY3fd9SoUZxxxhkUFBRQWFgIwC233EKXLl0488wzOfvssxk6dCi9e/emqKiIoUOH\nsmbNGjqSzhREJBTZek39uOOOo6Dg0N/LxxxzDB9//HErewTq6+s56aSTDlt38sknU19f37jcq1ev\nprtRWlra+Lxz586fW96xY0eb2t9eOlMQkUg55phj+OSTTxqXt2/fnpYzjrKyMt59993D1r377ruU\nlR2Ktc+F+aBUFEQkUs455xyeeOIJDhw4wAsvvEBtbW3K+7Z2dlNRUcHbb79NdXU1+/fvZ8GCBdTV\n1XHppZemtH9z24RxNqWiICKRcv/997No0SKOPfZYfvWrXzFixIjDXm/tr/mm9zESn3fv3p3nnnuO\ne++9l5KSEu655x6ee+45unfv3uJ7N/dZTd+/o88uNHhNRDIi3wevhU2D10REJONUFEREpJGKgoiI\nNFJREBGRRioKIiLSSEVBREQaqSiIiEgjFQUREWmkoiAi0sSWLVvo2rVrhwy+a5rvEDYVBRGJnN69\ne/PSSy+1+PpJJ53Evn37cmICu3RTURCRyGltCo79+/d3cGuyi4qCiETKtddey5YtWxg2bBhdu3Zl\n+vTpFBQU8Mgjj3DyySczePBg3n33XQoKCjh48CAA8+bN48wzz6SoqIjTTjuN2bNnN75fsgjPDz/8\nkGHDhtGtWzf69evH5MmTGTBgQLNt++tf/8pPf/pTTj75ZI4//njGjh3LZ599ltHj0ZSKgohEyuOP\nP85JJ53Ec889x759+xg5ciQAtbW11NXVsXTp0s+dRZSWlrJ48WL27t3LvHnzuPnmmw9LREuM8Jw7\ndy7jxo1jz549AIwbN46uXbuyc+dOHn30UR577LEWL0tNnDiRDRs28MYbb7Bhwwa2bdvGXXfdlaEj\n0QJ3z9pH0DwRyUXJ/v1Ceh5Honfv3v7iiy+6u/vmzZvdzHzz5s2Nr8fXHThwoNn9L7/8cr///vvd\n3f3ll1/2zp07H7btl770JV+5cqXv37/fCwsL/e233258bfLkyX7BBRc0LpuZb9y40Q8ePOhf/OIX\nfePGjY2vvfbaa37KKac024aWjm9s/RH/7iqOU0RCkW2zajcXlRm3ZMkS7rzzTtavX8/Bgwf55JNP\n+NrXvtb4eksRnh988AH79+8/7L179uzZ7Gd88MEHfPLJJ5x33nmN69y98RJWR9HlIxGJnGThNon+\n+te/8p3vfIdbbrmF999/n127dlFRUZFSd9UePXpw1FFHsXXr1sZ1ic8TlZSU0LlzZ9auXcuuXbvY\ntWsXu3fvZu/evSl+q/RQURCRyCktLWXjxo0pbfu3v/2Nv/3tb5SUlFBQUMCSJUtYtmxZSvt26tSJ\nK664gilTpvDpp59SV1fH448/3mwBKigo4Ac/+AE//vGP+eCDDwDYtm1byp+VLioKIhI5kyZN4mc/\n+xndu3fn6aefbvXMoWvXrjzwwAOMHDmS7t27U11dzWWXXdbsts158MEH2bNnD8cffzzXX389lZWV\nHH300c3uO3XqVPr06UN5eTndunXj29/+Nm+//XZ7v26bJI3jNLMhwH1AJ+AX7j61mW0eAIYCnwCj\n3H2NmfUCHgO+BDgw290fiG3fHVgAnAy8A4x0993NvK+ncoomItlHcZzNmzBhAu+//z7z5s1r1/uE\nEsdpZp2AB4EhwJlApZmd0WSbCqCPu/cF/gmYGXupAbjZ3c8CyoFxZnZ67LWJwHJ3/zLwYmxZRCTv\nvPXWW/zhD3/A3Vm1ahWPPPIII0aMCLtZLUrW+6gfsMHd3wEws/nAZcC6hG2GA48CuPtKMys2s1J3\n3wHsiK3/2MzWAWVAXWyfgbH9HwVqUGEQkTy0b98+Kisrqa+vp7S0lJ/+9KcMHz487Ga1KFlRKAMS\nb5W/B/RPYZuewM74CjPrDZwLrIytKnX3+Os7gdK2NFokTH/+M3TqBMceG3ZLJBd8/etfZ/369WE3\nI2XJikKqFwSbXr9q3M/MugC/Bn7k7h9/7gNigzdaeuMpU6Y0Ph80aBCDBg1KsUki6bd9O1xwAXzh\nC1BbC8cdF3aLJOpqamqoqalJ2/u1eqPZzMqBKe4+JLY8CTiYeLPZzB4Gatx9fmy5Dhjo7jvNrBB4\nDlji7vcl7FMHDHL3HWZ2AvCyu59OE7rRLNnkww9h4ECorIR9++DFF4NHUVHYLctOutGcWaHcaAZW\nA33NrLeZHQ1cBSxsss1C4LpYY8qB3bGCYMBcYG1iQUjY5/rY8+uB3x7pFxDpCPv2QUUFDB0Kt94K\nd98N550Hw4fDp5+G3TqR9EmlS+pQDnVJnevud5vZaAB3nxXbJt5D6S/ADe7+X2Z2AVAL/IFDl5Mm\nufsLsS6pTwInoS6pkuU++ywoCH36wKxZEO9WfuAAXHttUDB+8xsoLAy3ndkmilkEHS0TZwpJi0KY\nVBQkbA0NcOWV0LkzPPFEcIO56esjRgSXkB5//POvi3S0TF8+Eomsgwfhhhtg/3547LHmf/ALC+Gp\np6C+HsaPz75J3kTaSkVBpBnu8MMfwpYtwY9+wqwEn9O5MyxcCKtXw6RJHddGkUzQ1NkizZg8GVau\nhJdegmOOSb59UREsWRL0Tiouhokaiik5SkVBpIl77oFnngnGIbSlu2lJCSxfDgMGBIVhzJjMtVEk\nU1QURBLMmQP/9m/wyivBj3xbnXhiUBguvDAoKFVV6W+jSCapKIjELFgAU6bAihXQQjhWSk49FV54\nAQYPDgrDpZemrYkiGacuqSLA888HPY3+4z/g7LPT856rVgUF4cknQbOzSEdRl1SRdqqthVGj4Nln\n01cQAPr1C84+Ro6E3/8+fe8rkkkqChJpr78eDE6rroby8vS//7e+BXPnwrBh8Oab6X9/kXRTUZDI\nWrsWLrkEZs+Giy/O3OcMGwb33gv/8A+waVPmPkckHXSjWSJp8+bgR3r6dLj88sx/3ve+B3v3Bjef\nX3016KUkko1UFCRytm+Hb38bJkwIJrTrKGPHwp49wWcri0GylXofSaR89FEw6vjqq+G228Jpw8SJ\nymKQzNEsqSIp2rcvuHxz4YUwbdqhKbA7mjvcdBOsWxdMjdG5czjtkPykoiCSgpYyEcKiLAbJFBUF\nkSSSZSKERVkMkgkavCbSingmQkNDy5kIYVEWg2QjFQXJW4mZCL/+deuZCGFRFoNkG3VJlbw1eTL8\n7nepZyKERVkMkk1UFCQvTZsW3MCtrYVu3cJuTXLKYpBsoaIgeWf2bJg5M8hE6NEj7NakTlkMkg1U\nFCSvzJ8Pd97Z/kyEsCRmMXTtGsybJNKR1CVV8sbixXDjjenNRAiLshjkSKlLqgjBvYMbbgh68uR6\nQQBlMUh4VBQk5yVmIvTvH3Zr0kdZDBIGFQXJaWvXBpdZ5szJbCZCWIYNgxkzlMUgHUc3miVnJWYi\nXHZZ2K3JnKqqQ1Nuv/KKshgks1QUJCfFMxEmToRrrgm7NZmnLAbpKOp9JDnnww+D0b9VVXDrrWG3\npmMpi0GS0SypEinxTISBA2Hq1PCnwO5o7sFZQ12dshikeSoKEhnZlokQFmUxSGtUFCQSsjUTISzx\nLIZu3YIshgL1I5QYDV6TvBfPRNi/P/syEcISz2LYtg3GjVMWg6SPioJktcRMhKeeys5MhLAkZjFE\n7Ya7ZI66pEpWy5VMhLAkZjF066YsBmm/pGcKZjbEzOrMbL2ZTWhhmwdir79hZucmrH/EzHaa2R+b\nbD/FzN4zszWxx5D2fxXJN9OnwzPPwNKluZGJEJaSEli2LJgy/OGHw26N5LpWzxTMrBPwIDAY2Ab8\n3swWuvu6hG0qgD7u3tfM+gMzgfLYy/OA/wc81uStHZjh7jPS8zUk3yRmIpSUhN2a7FdWFswOqywG\naa9kZwr9gA3u/o67NwDzgaYTCgwHHgVw95VAsZkdH1t+BdjVwntHtEOhJBPPRFi2LPixk9TEsxh+\n8hNYtCjs1kiuSlYUyoCtCcvvxda1dZvm/DB2uWmumRWnsL1EwOLF8KMfBT9uffqE3Zrc89WvBjef\nv/99qKkJuzWSi5IVhVQ7ujX9qz/ZfjOBU4BzgO3AvSl+juSxFSvyKxMhLMpikPZI1vtoG9ArYbkX\nwZlAa9v0jK1rkbu/H39uZr8AWjzZnTJlSuPzQYMGMUgxVHlp9Wr47neDS0f5lIkQlsQshhdfhLPO\nCrtFkik1NTXUpPG0sNURzWZ2FPAWcDFQD6wCKpu50Tze3SvMrBy4z93LE17vDSxy97MT1p3g7ttj\nz28GvuHun7s1phHN0bB2bZCF8PDD+T0Fdhh+9Su45ZZgZtVTTw27NdIR2juiudUzBXffb2bjgaVA\nJ2Cuu68zs9Gx12e5+/NmVmFmG4C/ADckNK4aGAgcZ2ZbgdvdfR4w1czOIbjMtBkYfaRfQHJbPBNh\n2jQVhExQFoO0leY+ktBs3w4DBsDNNwdTNUjm/PznwRxJymLIf5oQT3JSlDMRwqIshmhQUZCcE/VM\nhLAoiyEaVBQkpygTIVzKYsh/KgqSM5SJkB3iWQxFRcF9Bv13yC/KU5CcoEyE7BHPYqivh/HjlcUg\nh1NRkIyLZyJs3apMhGyhLAZpifIUJOPimQgvv6xMhGySmMVQXAwTmp0YX6JGRUEyatq04IZmba26\nQWajeBbDgAFBYRitYaSRp6IgGZOYidCjR9itkZYkZjF07aoshqhTUZCMiGcirFgBPXuG3RpJJp7F\nMHhwUBiGDQu7RRIWdUmVtFu8GG68MfjrU1Ng55ZVq+DSS+HJJ0ETEucmdUmVrLJiBYwaBc8+q4KQ\ni5TFICoKkjaJmQjl5cm3l+yUmMXw5ptht0Y6moqCpMXatcFlhzlzgmwEyW3DhsGMGcG05ps2hd0a\n6Ui60SztFs9EuOceZSLkk6oq2LtXWQxRo6Ig7bJ9e/CjMXEiXHNN2K2RdBszBnbvDv4bK4shGtT7\nSI7YRx8Fo2ErKzVVQr5TFkPu0CypEgplIkSLshhyh4qCdLjPPoNLLoHTTlMmQpQcOADXXRfcZ1AW\nQ/ZSUZAO1dAQdDv9wheUiRBFDQ1wxRXBqGdlMWQnDV6TDhPPRGhoUCZCVBUWBqOdlcWQv1QUJCXK\nRJC4xCyGSZPCbo2km7qkSkpuuw1WroSXXlImgnw+i2HixLBbJOmioiBJTZ0KzzyjTAQ5XNMshjFj\nwm6RpIOKgrRq1ix4+GF49VVlIsjnJWYxFBUpiyEfqChIi6qr4a67gplPy8rCbo1kK2Ux5Bd1SZVm\nKRNB2kpZDNlBXVIl7VasCLqeLlyogiCpUxZDflBRkMMkZiL07x92ayTXKIsh96koSKPETISLLgq7\nNZKrlMWQ23SjWYBDmQjTpysTQdqvqgr27FEWQy5SURC2bw96jkycCNdeG3ZrJF+MHXuoMCiLIXeo\n91HEffhhMCq1qkqZCJIZymLoWJolVY5YPBPhwgth2jRNgS2ZoSyGjqWiIEfks8+gogL69FEmgmTe\ngQPBpcl9+5TFkGkqCtJmDQ1w5ZXBX2zKRJCO0tAAI0YEl5CUxZA5GR+8ZmZDzKzOzNab2YQWtnkg\n9vobZnZuwvpHzGynmf2xyfbdzWy5mb1tZsvMrPhIv4C0TTwTYf9+ZSJIxyosDKZdr6+HceOUxZCt\nWi0KZtYJeBAYApwJVJrZGU22qQD6uHtf4J+AmQkvz4vt29REYLm7fxl4MbYsGRbPRNiyRZkIEo54\nFsPrr6tjQ7ZKdqbQD9jg7u+4ewMwH2jai3048CiAu68Eis3s+NjyK8CuZt63cZ/Y/15+ZM2Xtpg8\nOchEeO45ZSJIeOJZDAsXws9/HnZrpKlk4xTKgK0Jy+8BTSc/aG6bMmBHK+9b6u47Y893AqXJmyrt\nMX26MhEkeyiLIXslKwqpXvVrelMj5auF7u5mpquLGTRnDjz0UDCytKQk7NaIBBKzGLp1g8rKsFsk\nkLwobAN6JSz3IjgTaG2bnrF1rdlpZse7+w4zOwF4v6UNTX0l22kkMAMYSK9eG8NujEgzzqKq6j+o\nqvpHYHHYjclZd9xxR1rep9UuqWZ2FPAWcDFQD6wCKt19XcI2FcB4d68ws3LgPncvT3i9N7DI3c9O\nWDcN+NDdp5rZRKDY3T93s1ldUtvn+eeDnkbKRJBspyyG9Mlol1R33w+MB5YCa4EF7r7OzEab2ejY\nNs8Dm8xsAzALuCmhcdXAa8CXzWyrmd0Qe+nnwLfN7G3gotiypFFtLYwaBc8+q4Ig2U9ZDNlDg9fy\n0OrVwWjl6mq4+OKwWyOSukWL4Ac/COZJOuussFuTm5S8JoeJZyLMnq2CILlHWQzh09TZeSQxE+Fy\njfyQHFVVBXv3KoshLCoKeWL79uAfkTIRJB+MGaMshrDonkIeiGciVFbCbbeF3RqR9Jk0Keg9pyyG\n1GmW1IiLZyIMHAhTp2oKbMkv7nDTTbBunbIYUqWiEGHKRJAoOHgwuCS6d6+yGFKhohBRykSQKGlo\ngCuugK5dlcWQjLqkRpAyESRqCguD0c719TB+vLIYMklFIcfEMxG2blUmgkRLPIth9ergBrRkhrqk\n5ph4JsJLLykTQaInnsUwcGAws6qKQ/qpKOSQadOCTIQVK9Q9T6KrpASWL4cLLgiyGMaODbtF+UVF\nIUfMng0zZ8Krr0KPHmG3RiRcJ554eBZDVVXYLcofKgo5YMECuOsuqKkJgklEBE49FV54IRin07Vr\nMG+StJ+6pGY5ZSKItC6exbBgAXzrW2G3JnzqkprHVqwIMhEWLlRBEGlJPIvhqquUxZAOKgpZavVq\n+O53Yf586N8/7NaIZLdvfQvmzg0uIb35ZtityW0qClkonokwZw5cdFHYrRHJDcpiSA/daM4yiZkI\nl10WdmtEcouyGNpPRSGLbN8e9KRQJoLIkRszBnbvVhbDkVLvoywRz0SoqoJbbw27NSK5b+LEIIch\nalkMmiU1D8QzES68MBi1rCmwRdrPPRjtXFcXrSwGFYUcp0wEkcw5cCC4FLtvX3SyGFQUcpgyEUQy\nr6EBRowILiFFIYtBg9dylDIRRDpGYWEwzXx9PYwbpyyGZFQUQqBMBJGOFc9ieP11deRIRl1SQ6BM\nBJGO1zSLYeLEsFuUnVQUOlg8E6G2Nlrd5ESyQUkJLFsGAwYEWQxjxoTdouyjotCBEjMRSkrCbo1I\nNJWVHcpiKCpSFkNTKgodZP58uPPOYOZTZSKIhEtZDC1Tl9QOsHgx3HijMhFEsk08i+HJJ2HQoLBb\nkx7qkprlVqwIup4qE0Ek+8SzGEaOVBZDnIpCBsUzEaqrlYkgkq2UxXA4FYUMScxEuPjisFsjIq1R\nFsMhutGcAfFMhHvuUSaCSK6oqoI9e5TFoKKQZtu3B/+nmjgRrrkm7NaISFuMHXuoMEQ1i0G9j9Io\nnolQWQm33RZ2a0TkSOVyFkPGex+Z2RAzqzOz9WY2oYVtHoi9/oaZnZtsXzObYmbvmdma2GPIkX6B\nbLFvXzAFdkWF5lYRyXV33w3nnQfDh8Onn4bdmo7V6pmCmXUC3gIGA9uA3wOV7r4uYZsKYLy7V5hZ\nf+B+dy9vbV8zuwPY5+4zWm1cjpwpKBNBJP/kahZDps8U+gEb3P0dd28A5gNNb50OBx4FcPeVQLGZ\nHZ/Cvnnx09nQAFddBV/6UjCFhQqCSH7o1AkefTR4fv31QZGIgmRFoQzYmrD8XmxdKtucmGTfH8Yu\nN801s+I2tTpLKBNBJL8VFgajnevrYfz4aGQxJOt9lOohaOvfxzOBu2LP/xW4F/h+cxtOmTKl8fmg\nQYMYlCVj0eOZCFu2BHOoKBNBJD/Fsxguvji4X3j33WG36HA1NTXU1NSk7f2S3VMoB6a4+5DY8iTg\noLtPTdjmYaDG3efHluuAgcApyfaNre8NLHL3z00Ckc33FCZPDuZmf+mlYG52Eclvf/5z0Lvwuutg\nQrNdbrJDpu8prAb6mllvMzsauApY2GSbhcB1scaUA7vdfWdr+5rZCQn7jwD+eKRfIAzTp8PTTwdn\nCCoIItFQUgLLlwdT4M+aFXZrMqfVy0fuvt/MxgNLgU7A3FjvodGx12e5+/NmVmFmG4C/ADe0tm/s\nraea2TkEl6c2A6Mz8eUyYc4ceOihYMRjjx5ht0ZEOtKJJwaFIZ7FUFkZdovST4PX2mDBAvjJT4KZ\nT/v0Cbs1IhKWP/0pyGL4xS+COc6ySXsvH6kopOj554OeRspEEBHI3iwG5Sl0gNpaGDUKnn1WBUFE\nAvmaxaCikMTrr8OVVwZxmuXlYbdGRLJJPmYxqCi0Yt26Q5kIF10UdmtEJBslZjFs3Bh2a9pPU2e3\nYPNm+Pu/D7qfKhNBRFpTVQV79wa/GbmexaCi0AxlIohIW40ZA7t3B78dK1YE4xpykXofNRHPRKiq\n0hTYItJ2YWcxqEtqGu3bF/Q9HjgQpk7VjKci0nbucNNNwT3JJUuCuZM6kopCmigTQUTS5eDB4NLz\n3r3wzDMdm8WgopAGDQ1Bt9POneGJJzQFtoi0X0MDjBgBXbvCL3/Zcb8rGrzWTspEEJFMKCyEp54K\nOq6MG5c7WQyRLgqJmQhPPaVMBBFJr3gWw+uvw6RJYbcmNZHukjp5MqxcGWQiHHNM2K0RkXxUVBTc\ncB44EIqLg95J2SyyRWHatOAGUG1tON3GRCQ6Skpg2TIYMCAoDGPGhN2ilkWyKMyeDTNnwquv5u4A\nExHJLWVlwSzL8SyGqqqwW9S8yBWF+fPhzjuDEYdlZWG3RkSi5NRTYenSIO+5a9dg3qRsE6kuqYsX\nw403KhNBRMKVySwGdUlN0YoVQdfThQtVEEQkXNmcxRCJorB6NXz3u1BdDf37h90aEZHszWLI+6Kw\ndu2hTISLLw67NSIihyRmMWzaFHZrAnl9o3nz5uBg33OPMhFEJDtVVcGePcGU29mQxZC3RUGZCCKS\nK8aOPVQYamvhuOPCa0te9j6KZyJUVsJtt2WgYSIiGZCOLAbNktqEMhFEJFe5B2cNdXVHnsWgopBA\nmQgikusOHIBrrw3+wP3Nb9qexaCiEKNMBBHJFw0NcMUVwajnxx9v2++ZBq+hTAQRyS+FhcFo5/p6\nGD++Y7MYcr4oxDMRtm5VJoKI5I94FsPq1XDrrR33uTnfJXXyZPjd7+Dll5WJICL5pWkWw4QJmf/M\nnC4K06cHN2KUiSAi+appFsPo0Zn9vJwtCrNnw0MPBSMAe/QIuzUiIplTVgbLlwdnDEVFwRisTMnJ\nopCYidCzZ9itERHJvNNOgxdeCOZw69Ilc1kMOdclVZkIIhJlybIYItUldcUKGDUKnn1WBUFEoinT\nWQw5UxTimQgLFkB5editEREJTyazGJIWBTMbYmZ1ZrbezJrtEGVmD8Ref8PMzk22r5l1N7PlZva2\nmS0zs+KAp3A1AAAEzklEQVTW2pCYiXDRRW35eiIi+SlTWQytFgUz6wQ8CAwBzgQqzeyMJttUAH3c\nvS/wT8DMFPadCCx39y8DL8aWmxXPRJg+PdqZCDU1NWE3ISvoOByiY3FIVI9FVVUwVmvZsvS9Z7Iz\nhX7ABnd/x90bgPlA05/m4cCjAO6+Eig2s+OT7Nu4T+x/L2+pAfFMhGuvbcO3ykNR/T99UzoOh+hY\nHBLlYzFmTPBIl2RFoQzYmrD8XmxdKtuc2Mq+pe6+M/Z8J1DaUgNuvBHGjUvSShERSYtkRSHV/qqp\ndH+y5t4v1ue0xc+ZNCnFFoiISPu5e4sPoBx4IWF5EjChyTYPA1cnLNcR/OXf4r6xbY6PPT8BqGvh\n810PPfTQQ4+2PVr7XU/2SDaieTXQ18x6A/XAVUDTAdYLgfHAfDMrB3a7+04z+7CVfRcC1wNTY//7\n2+Y+vD0DMEREpO1aLQruvt/MxgNLgU7AXHdfZ2ajY6/PcvfnzazCzDYAfwFuaG3f2Fv/HHjSzL4P\nvAOMzMB3ExGRNsrqaS5ERKRjZeWI5lQGzOUrM+tlZi+b2Ztm9icz++fY+jYN+MsnZtbJzNaY2aLY\nciSPhZkVm9mvzWydma01s/4RPhaTYv9G/mhmvzKz/xGVY2Fmj5jZTjP7Y8K6Fr977Fitj/2m/n2y\n98+6opDKgLk81wDc7O5nEdysHxf7/ikP+MtDPwLWEtxEg+gei/uB5939DOBrBB02IncsYvcpfwD8\nnbufTXB5+mqicyzmEfw+Jmr2u5vZmQT3c8+M7fOQmbX6u591RYHUBszlLXff4e7/HXv+MbCOYHxH\nygP+8omZ9QQqgF9wqOtz5I6FmXUDBrj7IxDcs3P3PUTwWAB7Cf54OsbMjgKOIejMEolj4e6vALua\nrG7pu18GVLt7g7u/A2wg+I1tUTYWhVQGzEVC7C+ic4GVtGHAX575v8D/AQ4mrIvisTgF+MDM5pnZ\nf5nZHDP7IhE8Fu7+EXAvsIWgGOx29+VE8FgkaOm7n0jwGxqX9Pc0G4uC7nwDZtYFeBr4kbvvS3wt\n2YC/fGFmlwLvu/saWhggGZVjQdBT8O+Ah9z97wh6+h12eSQqx8LMTgN+DPQm+NHrYmbXJG4TlWPR\nnBS+e6vHJRuLwjagV8JyLw6vdHnPzAoJCsLj7h4fw7EzNqcUZnYC8H5Y7etA/wsYbmabgWrgIjN7\nnGgei/eA99w9PoP+rwmKxI4IHouvA6+5+4fuvh/4DXA+0TwWcS39m2j6e9oztq5F2VgUGgfMmdnR\nBDdJFobcpg5jZgbMBda6+30JL8UH/EErA/7yibvf6u693P0UghuJL7n7tUTzWOwAtprZl2OrBgNv\nAouI2LEguMFebmadY/9eBhN0RIjisYhr6d/EQuBqMzvazE4B+gKrWnujrBynYGZDgfs4NOjt7pCb\n1GHM7AKgFvgDh07zJhH8h3wSOInYgD933x1GG8NgZgOB/+3uw82sOxE8Fmb2PwluuB8NbCQYKNqJ\naB6LWwh+/A4C/wX8I9CVCBwLM6sGBgIlBPcPbgeepYXvbma3AjcC+wkuRy9t9f2zsSiIiEg4svHy\nkYiIhERFQUREGqkoiIhIIxUFERFppKIgIiKNVBRERKSRioKIiDRSURARkUb/HwGucLN2dHFmAAAA\nAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "xs = list(range(0,100))\n", + "\n", + "ps = thinkbayes.MakeUniformPmf(0, 100, 101).Probs(xs)\n", + "plt.plot(xs, ps, label='uniform', color='k')\n", + "\n", + "suite = TrianglePrior(Euro())\n", + "ps = suite.Probs(xs)\n", + "plt.plot(xs, ps, label='triangle');\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows the result (and the uniform prior for\n", + "comparison). Updating this prior with the same dataset yields the\n", + "posterior distribution shown in Figure [fig.euro3]. Even with\n", + "substantially different priors, the posterior distributions are very\n", + "similar. The medians and the credible intervals are identical; the means\n", + "differ by less than 0.5%." + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQVPWd9/H3dwYQRhhwRAe5yFCCXOIlGGUxajJrTBZR\nISYplUq8ZGujZSSiFaPRUgMp/0lFk2ix65qNEDUG46MxhQrBmDjxcfVBWRGNMMAgd5iB5X4RnGG+\nzx+ne2iameme7tN9uqc/r6ou+ty/fYr5zG9+/TvnmLsjIiKloyzqAkREJL8U/CIiJUbBLyJSYhT8\nIiIlRsEvIlJiFPwiIiUmZfCb2SQzqzez1WZ2TzvLx5jZO2Z2yMx+2M7ycjNbamYvh1W0iIhkrtPg\nN7NyYDYwCRgHTDOzsUmr7QB+ADzcwW5mAMsBXTAgIlIAUrX4JwAN7r7O3ZuB54CpiSu4+3Z3XwI0\nJ29sZkOBycBvAAunZBERyUaq4B8CbEyY3hSbl65fAj8CWrtYl4iI5Eiq4M+4e8bMrgS2uftS1NoX\nESkYPVIs3wwMS5geRtDqT8cXgSlmNhnoDVSa2dPufkPiSmamvn8RkQy4e0aN6lQt/iXAKDOrMbNe\nwLXA/A7WPaYAd7/P3Ye5+wjgOuBvyaGfsK5e7vzkJz+JvIZCeelc6FzoXHT+ykanLX53bzGz6cAi\noBx40t1XmNktseVPmNkg4D2gEmg1sxnAOHffn7y7rCoVEZFQpOrqwd0XAguT5j2R8L6RY7uD2tvH\n34G/Z1ijiIiESFfuFpDa2tqoSygYOhdH6VwcpXMRDsu2ryjrAsw86hpERIqNmeEZfrmbsqtHRKQj\nZhqpnQ9hN44V/CKSFf3Fnlu5+OWqPn4RkRKj4BcRKTEKfhGREqPgFxFJcuutt/LQQw+1TT/++ONU\nV1dTWVnJrl27IqwsHBrOKSIZiw0pjLqMnGpubqZ///68++67nHXWWXk/fkfnOJvhnGrxi4h0orGx\nkUOHDjF2bPIzqFIL4746uaDgF5FuqaysjE8++aRt+qabbuKBBx4AoK6ujqFDh/KLX/yC6upqBg8e\nzG9/+9vj1l29ejVjxowBYMCAAVx22WUAvP3221xwwQUMGDCACRMm8M4777RtW1tby/33389FF11E\n3759+eSTTygrK+Pxxx9n1KhRVFZW8uCDD7JmzRouvPBCBgwYwHXXXUdz83HPssoZjeMXkZJgZseM\niW9qamLv3r1s2bKF1157jW9961tcffXV9O/fv23dUaNGsXz5ckaMGMGePXsoKytj586dXHHFFcye\nPZtp06bx/PPPc8UVV7BmzRpOOukkAH73u9+xcOFCRo8ezZEjRwB47bXXWLp0KRs2bGD8+PG89dZb\nzJs3j6qqKi688ELmzZvHDTe0ewPj0KnFLyI5FQ/RbF5hSex26dmzJw8++CDl5eVcfvnl9O3bl5Ur\nVx63bnJXzauvvsro0aP59re/TVlZGddddx1jxoxh/vz5bZ/3pptuYuzYsZSVldGzZ08A7r77bvr2\n7cu4ceM4++yzufzyy6mpqaGyspLLL7+cpUuXhvY5U1GLX0RyqhD7uAFOPvlkysqOtn0rKirYvz/5\nbvLH27JlC6effvox84YPH86WLVvapocNO/6GxdXV1W3v+/Tpc9x0Y2Njl+rPhlr8ItItVVRUcPDg\nwbbprVu3hvLXw5AhQ1i/fv0x89avX8+QIUcfR17o9zBS8IvkgTtceSW8/nrUlZSOz3/+8zz77LMc\nOXKEP//5z7z55ptpb9vZXymTJ09m1apVzJs3j5aWFv7whz9QX1/PlVdemdb27a2T77+KFPwiefDs\ns7BgQfCS/Hj00Ud5+eWXOemkk/j973/P1Vdffczyzlrlyd8tJL6vqqrilVde4ZFHHmHgwIE8/PDD\nvPLKK1RVVXW47/aOlbz/fP6VoAu4RHJs714YOxZmzIAXX4TFi6OuKDylcAFX1HJxAZeCXyTH7roL\ndu6E2bPhlFNg+3aoqIi6qnAo+HMvF8GvUT0iObR8OTz1FPzjH0HYn302vPcefPnLUVcmpUx9/CI5\n4g633w4PPADxkXsXXQRvvRVtXSIKfpEc2bYN3n8fvv/9o/Muugj++7+jq0kEFPwiOdPQAKNHQ4+E\nDtWLLoJ33oHW1ujqEkkr+M1skpnVm9lqM7unneVjzOwdMztkZj9MmD/MzN4ws4/N7B9mdnuYxYsU\nsoYGOOOMY+dVV8PAgUHfv0hUUga/mZUDs4FJwDhgmpkl3590B/AD4OGk+c3Ane7+OWAicFs724p0\nSw0NMHLk8fPV3SNRS6fFPwFocPd17t4MPAdMTVzB3be7+xKCoE+c3+juH8Te7wdWAINDqVykwHUW\n/PqCV6KUTvAPATYmTG+KzesSM6sBxgPd6PIVkY6tWaMWf6HbsGED/fr1y8u1CMnPB4hSOuP4sz4j\nZtYXeAGYEWv5H2PmzJlt72tra6mtrc32kCKR66jFP2YM7N4NW7fCaaflv65SUlNTw5w5c7j00kvb\nXX766aezb9++PFeVmbq6Ourq6kLZVzrBvxlIvMfoMIJWf1rMrCfwIvA7d/9Te+skBr9Id7BzJxw5\nAieffPyysrKjrf5vfSv/tZWSzq4sbmlpoUeP4rmGNblRPGvWrIz3lU5XzxJglJnVmFkv4Fpgfgfr\nHnP5sAV3HXoSWO7uv8q4SpEiE2/td3TfLXX35N7111/Phg0buOqqq+jXrx8///nPKSsrY86cOQwf\nPpzLLruM9evXU1ZWRmtsfO3cuXMZN24clZWVnHHGGfz6179u21+qxzXu2LGDq666iv79+zNhwgTu\nv/9+LrnkknZrO3z4MHfddRfDhw9n0KBB3HrrrRw6dCin5yNRyuB39xZgOrAIWA78wd1XmNktZnYL\ngJkNMrONwJ3A/Wa2Ida9cxHwHeCfzWxp7DUpZ59GpEB01M0Td845UF+fv3pK0TPPPMPpp5/OK6+8\nwr59+7jmmmsAePPNN6mvr2fRokXH/TVQXV3Nq6++yt69e5k7dy533nnnMU/GSnxc45NPPsltt93G\nnj17ALjtttvo168fTU1NPPXUUzz99NMd3nHzxz/+MQ0NDSxbtoyGhgY2b97MT3/60xydiXbEnwIf\n1SsoQaR7+elP3e+7r+Ply5a5n3VW/urJlXR+foObV2T3ylRNTY3/9a9/dXf3tWvXupn52rVr25bH\n5x05cqTd7b/+9a/7o48+6u7ub7zxhvfp0+eYdU899VRfvHixt7S0eM+ePX3VqlVty+6//36/+OKL\n26bNzNesWeOtra1+4okn+po1a9qWvf322z5ixIh2a+joHMfmZ5S7xdPBJVJEGho6vxHb0KGwKe1v\nyopbod28s73HIsYtXLiQWbNmsXr1alpbWzl48CDnnHNO2/KOHte4fft2Wlpajtn30KFD2z3G9u3b\nOXjwIF/4whfa5rl7W3dTPuiWDSI5kKqr56ST4PBhSOMRr5KFVA9ASXT48GG++c1vcvfdd7Nt2zZ2\n7drF5MmT0xrqecopp9CjRw82bjw68j3xfaKBAwfSp08fli9fzq5du9i1axe7d+9m7969aX6q7Cn4\nRXIgVfCbBa3+zZvzV1Mpqq6uZs2aNWmt+9lnn/HZZ58xcOBAysrKWLhwIa+99lpa25aXl/ONb3yD\nmTNn8umnn1JfX88zzzzT7i+ZsrIyvve973HHHXewfft2ADZv3pz2scKg4BcJ2b59QUs+1Rj9Uuru\nicq9997LQw89RFVVFS+++GKnfwH069ePxx57jGuuuYaqqirmzZvH1KlT2123PbNnz2bPnj0MGjSI\nG2+8kWnTptGrV692t/3Zz37GyJEjmThxIv379+erX/0qq1atyvbjpk1P4BIJ2QcfwPXXw0cfdb7e\nDTfAV74CN96Yn7pyQU/g6tg999zDtm3bmDt3blb7ycUTuNTiFwlZqm6eOLX4u5eVK1fy4Ycf4u68\n++67zJkz57gHvBcKjeoRCVlXgj/VXwVSPPbt28e0adPYsmUL1dXV3HXXXUyZMiXqstql4BcJ2Zo1\ncP75qdcbOhQWLsx9PZIf559/PqtXr466jLSoq0ckZOrqkUKn4BcJWXtP3mqPgl+iolE9IiH69NPg\n4qwDB6C8vPN1W1uhoiK4RXPv3vmpL2wa1ZN7uRjVoz5+kRCtXQs1NalDH4LbMw8eHFzElc5fCIWq\ns7HtUpgU/CIhSrd/Py7e3VOswa/WfnFSH79IiDZsgOHD019f/fwSBQW/SIgaG7v2OEUFv0RBwS8S\noq1bYdCg9NdX8EsUFPwiIWpsVPBL4VPwi4RIwS/FQMEvEiIFvxQDXcAlEpLW1uBCrP37IeE27J06\ncgT69OnaNiKg2zKLFIQdO6CysmsBXl4e/IWwdWvu6hJJpuAXCUlXu3ni1N0j+abgFwmJgl+KRcrg\nN7NJZlZvZqvN7J52lo8xs3fM7JCZ/bAr24p0Jwp+KRadBr+ZlQOzgUnAOGCamY1NWm0H8APg4Qy2\nFek2FPxSLFK1+CcADe6+zt2bgeeAYx477+7b3X0J0NzVbUW6EwW/FItUwT8E2JgwvSk2Lx3ZbCtS\ndBT8UixS3ZY5mwH2aW87c+bMtve1tbXU1tZmcViRaCj4JZfq6uqoq6sLZV+pgn8zMCxhehhByz0d\naW+bGPwixSrT4D/tNGhqCi7mSucBLlKakhvFs2bNynhfqbp6lgCjzKzGzHoB1wLzO1g3+Qqyrmwr\nUvQyDf6ePaF//+ACMJF86LTF7+4tZjYdWASUA0+6+wozuyW2/AkzGwS8B1QCrWY2Axjn7vvb2zaX\nH0YkKocPw759UFWV2fanngrbtgX/iuRaykcvuvtCYGHSvCcS3jdybJdOp9uKdEfx0C7L8JLIePCL\n5IOu3BUJQVcfwJJMwS/5pOAXCUFXH7mYTMEv+aTgFwlBpl/sxin4JZ8U/CIhUPBLMVHwi4RAwS/F\nRMEvEgIFvxQTBb9ICMII/qam8OoR6YyCXyQEavFLMVHwi2TJPQj+6urM91FZCc3NcPBgeHWJdETB\nL5KlffuCm6v17Zv5PsyCVv/27eHVJdIRBb9IlrLt5olTd4/ki4JfJEsKfik2Cn6RLCn4pdgo+EWy\npOCXYqPgF8mSgl+KjYJfJEsKfik2Cn6RLGU7hj9OwS/5ouAXyVJTk1r8UlwU/CJZCutZuQp+yRdz\n92gLMPOoaxDJlDv07g27d0OfPtnt6/Bh6Ncv+NcsnPqk+zIz3D2j/ylq8YtkYe9eOOGE7EMfgv1U\nVMCuXdnvS6QzCn6RLITVzRNXXa3uHsk9Bb9IFsIOfvXzSz6kDH4zm2Rm9Wa22szu6WCdx2LLl5nZ\n+IT595rZx2b2kZn93sxOCLN4kag1NSn4pfh0GvxmVg7MBiYB44BpZjY2aZ3JwEh3HwXcDDwem18D\nfA84z93PBsqB60KuXyRS27aFM4Y/TsEv+ZCqxT8BaHD3de7eDDwHTE1aZwrwFIC7LwYGmFk1sBdo\nBirMrAdQAWwOs3iRqKmrR4pRquAfAmxMmN4Um5dyHXffCTwCbAC2ALvd/fXsyhUpLAp+KUY9UixP\nd4D9cWNJzewM4A6gBtgD/B8z+7a7P5u87syZM9ve19bWUltbm+ZhRaK1bRtcckl4+zv1VHjjjfD2\nJ91HXV0ddXV1oewrVfBvBoYlTA8jaNF3ts7Q2Lxa4G133wFgZn8Evgh0GvwixUQtfsmX5EbxrFmz\nMt5Xqq6eJcAoM6sxs17AtcD8pHXmAzcAmNlEgi6dJmAlMNHM+piZAZcByzOuVKQAKfilGHXa4nf3\nFjObDiwiGJXzpLuvMLNbYsufcPcFZjbZzBqAA8B3Y8s+MLOnCX55tALvA7/O4WcRyTsFvxQj3atH\nJEMtLcGtGg4dgvLycPbZ2hrcuuHAAejVK5x9Sveke/WIROB//xeqqsILfYCyMhg4MNi3SK4o+EUy\nFPZVu3Hq7pFcU/CLZCjsq3bjqquDXyoiuaLgF8lQ2F/sxp16qoJfckvBL5KhXAb/9u3h71ckTsEv\nkqFcBr/6+CWXFPwiGcpV8KuPX3JNwS+SIbX4pVgp+EUypOCXYqXgF8mQunqkWCn4RTKUq+A/5ZRg\n37qTieSKgl8kA/v3B8F84onh77tPn+B+PXv3hr9vEVDwi2QkftWuZXSLrNR0EZfkkoJfJAO56uaJ\nq67WF7ySOwp+kQzkOvg1skdyScEvkoF8BL+6eiRXFPwiGVBXjxQzBb9IBtTVI8VMwS+SAXX1SDFT\n8ItkQF09UswU/CIZUFePFDMFv0gGcvW83Th19UgumUd8QxAz86hrEOmKI0egd2/49FPo0SM3x2ht\nDW7bcOAA9OqVm2NIcTMz3D2ja8dTtvjNbJKZ1ZvZajO7p4N1HostX2Zm4xPmDzCzF8xshZktN7OJ\nmRQpUkh27oTKytyFPkBZWXCzNj2CUXKh0+A3s3JgNjAJGAdMM7OxSetMBka6+yjgZuDxhMWPAgvc\nfSxwDrAixNpFIrF1K5x2Wu6Po+4eyZVULf4JQIO7r3P3ZuA5YGrSOlOApwDcfTEwwMyqzaw/cIm7\nz4kta3H3PeGWL5J/jY35CX6N7JFcSRX8Q4CNCdObYvNSrTMUGAFsN7O5Zva+mf2XmVVkW7BI1LZu\nhUGDcn8ctfglV1L1Uqb7rWvyFwwe2/d5wHR3f8/MfgX8GHgweeOZM2e2va+traW2tjbNw4rkXz67\netTil7i6ujrq6upC2Veq4N8MDEuYHkbQou9snaGxeQZscvf3YvNfIAj+4yQGv0ih27oVampyfxx1\n9Uii5EbxrFmzMt5Xqq6eJcAoM6sxs17AtcD8pHXmAzcAxEbt7Hb3JndvBDaa2Zmx9S4DPs64UpEC\n0diorh4pbp22+N29xcymA4uAcuBJd19hZrfElj/h7gvMbLKZNQAHgO8m7OIHwLOxXxprkpaJFCV1\n9UixSzkS2d0XAguT5j2RND29g22XARdkU6BIoclX8KurR3JFt2wQ6SKN6pFip+AX6YL9+4PbKVRW\n5v5Y8St3dUcTCZuCX6QL4t08ltEdUrqmd2+oqIDdu3N/LCktCn6RLshX/36cunskFxT8Il2Qr6Gc\ncRrZI7mg4Bfpgny3+DWyR3JBwS/SBerqke5AwS/SBfkayhmnrh7JBQW/SBfk65bMcerqkVxQ8It0\ngbp6pDtQ8It0Qb6D/7TTYMuW/B1PSoOCXyRNzc3BxVQDB+bvmEOGwObN+TuelAYFv0iampqC2yiU\nl+fvmIMHB8c9ciR/x5TuT8EvkqZ8d/MA9OoFAwboC14Jl4JfJE35HsoZp+4eCZuCXyRN+R7KGTd0\nqIJfwqXgF0lTFF09oBa/hE/BL5ImdfVId6HgF0lTVF09Cn4Jm4JfJE3q6pHuQsEvkiYFv3QX5hE/\n0NPMPOoaRFJxhxNOgL17g0ci5tOuXVBTA3v25Pe4UtjMDHfP6CGgavGLpGHnTjjxxPyHPgQXcDU3\nw759+T+2dE8pg9/MJplZvZmtNrN7OljnsdjyZWY2PmlZuZktNbOXwypaJN+i6uaB4MHu6u6RMHUa\n/GZWDswGJgHjgGlmNjZpncnASHcfBdwMPJ60mxnAckD9OVK0ohrKGafglzClavFPABrcfZ27NwPP\nAVOT1pkCPAXg7ouBAWZWDWBmQ4HJwG+AjPqiRApBlC1+UPBLuFIF/xBgY8L0pti8dNf5JfAjoDWL\nGkUit3lzcKfMqCj4JUw9UixPt3smuTVvZnYlsM3dl5pZbWcbz5w5s+19bW0ttbWdri6Sd+vWwbnn\nRnf8IUNg9eroji/Rq6uro66uLpR9pQr+zcCwhOlhBC36ztYZGpv3TWBK7DuA3kClmT3t7jckHyQx\n+EUK0bp1MDW5kzOPhgyBkH7mpUglN4pnzZqV8b5SdfUsAUaZWY2Z9QKuBeYnrTMfuAHAzCYCu929\n0d3vc/dh7j4CuA74W3uhL1IM1q2DESOiO77u0Clh6rTF7+4tZjYdWASUA0+6+wozuyW2/Al3X2Bm\nk82sATgAfLej3YVZuEi+tLbC+vUwfHh0NaiPX8KkK3dFUtiyBcaPDx6BGJXm5uACsoMHoUeqDlop\nCbpyVySH1q0LbpkQpZ494eSTo/3lI92Hgl8khaj79+PU3SNhUfCLpLB2bfQtflDwS3gU/CIpFEJX\nDyj4JTwKfpEU1NUj3Y2CXyQFtfilu1Hwi3TiyBHYsAFOPz3qSoLg35R83bxIBhT8Ip3YuhWqqqBP\nn6grUYtfwqPgF+lEofTvw9Hg1/WOki0Fv0gnCqV/H6CyEsrKYPfuqCuRYqfgF+lEoYzhh+ARjCNH\nQkND1JVIsVPwi3SikFr8AKNHw6pVUVchxU7BL9KJQurjBzjzTFi5MuoqpNgp+EU6UWgt/jPPVItf\nsqfgF+nAkSPBuPlCGMMfp+CXMCj4RTqweTOccgqccELUlRw1alQQ/BrSKdlQ8It0oNC6eQAGDAge\nyLJ1a9SVSDFT8It0oBCDH4KRPfqCV7Kh4BfpQCGN4U+kfn7JloJfpAOFNpQzTsEv2VLwi3Rg7VoY\nPjzqKo6n4JdsKfhF2uEOH38M48ZFXcnxFPySLQW/SDuamoLwP+20qCs53hlnwPr10NwcdSVSrNIK\nfjObZGb1ZrbazO7pYJ3HYsuXmdn42LxhZvaGmX1sZv8ws9vDLF4kV5Ytg3PPDW6MVmhOOCG4RfPa\ntVFXIsUqZfCbWTkwG5gEjAOmmdnYpHUmAyPdfRRwM/B4bFEzcKe7fw6YCNyWvK1IIfrwQzjnnKir\n6Ji6eyQb6bT4JwAN7r7O3ZuB54CpSetMAZ4CcPfFwAAzq3b3Rnf/IDZ/P7ACGBxa9SI5ouCX7iyd\n4B8CbEyY3hSbl2qdoYkrmFkNMB5Y3NUiRfJNwS/dWY801kn3riDJvaFt25lZX+AFYEas5X+MmTNn\ntr2vra2ltrY2zUOKhO+zz4JQLcQRPXGjR8OLL0ZdheRTXV0ddXV1oezLPMXdnsxsIjDT3SfFpu8F\nWt39Zwnr/CdQ5+7PxabrgS+7e5OZ9QReARa6+6/a2b+nqkEknz76CK65BlasiLqSjm3YABdeqIev\nlzIzw90zGn6QTlfPEmCUmdWYWS/gWmB+0jrzgRtixUwEdsdC34AngeXthb5IIYqP6ClkQ4fCrl2w\n/7i/n0VSSxn87t4CTAcWAcuBP7j7CjO7xcxuia2zAPjEzBqAJ4Dvxza/CPgO8M9mtjT2mpSLDyIS\nlkLv34fgoesjR8Lq1VFXIsUonT5+3H0hsDBp3hNJ09Pb2e4tdJGYFJkPP4Tpx/1vLjzxxzCOHx91\nJVJsFMoiSYqhxQ9w3nnw3ntRVyHFSMEvkmD7djh4EIYNi7qS1L70JXjzzairkGKk4BdJ8NFHQWu/\nEG/VkOyCC4KRR3v3Rl2JFBsFv0iCDz8s/BE9cSecAOefD2+/HXUlUmwU/CIJli0rjv79OHX3SCYU\n/CIJiuWL3TgFv2Qi5ZW7OS9AV+5KgWhpgcpK2LYN+vaNupr0HDgA1dXBl9J9+kRdjeRTrq/cFSkJ\nH3wQPGqxWEIf4MQT4eyzYbFufShdoOAXiXnpJZiafMPxIqDuHukqBb9IzEsvwTe+EXUVXafgl65S\nH78IwXj4r30teJZtWZE1h3bvDi4427EDevWKuhrJl2z6+Asi+CMtQASAe4FBwIyoC8nQ+wT3Rvx/\nURciORbP7KIP/qhrEDn/fHj4YSjWZwDNmAGDB8M990RdieSLRvWIZGH9+uDBJhdfHHUlmfvSl+D1\n16OuQoqFgl9K3ksvwZQp0COtm5QXpsmTg4vP6uujrkSKgYJfSt4f/1ico3kS9ekDt94Kv/xl1JVI\nMVAfv5S0piYYMwYaG4ObnhWzbduCh7CvXAmnnhp1NZJr6uMXydDzz8OkScUf+hCE/TXXwL//e9SV\nSKFTi19K1rZtwe0OFi4MnmbVHaxcCZdcAuvWQUVF1NVILqnFL5KBO+6Am27qPqEPQVfPhRfC009H\nXYkUMrX4pSQtWAC33x6MhOluLeM334R/+zdYvry4RypJ53QBl0gX7NsHZ50Fc+bAV74SdTXhc4er\nroKqKvjtb4vvFhSSnpx29ZjZJDOrN7PVZtbudYFm9lhs+TIzG9+VbUXyqbUVfvhDuPTS7hn6EDwv\n+PnngwvTbr01+EUgkqjT4DezcmA2MAkYB0wzs7FJ60wGRrr7KOBm4PF0t5Vj1dXVRV1CwcjFudi4\nEb76Vfj4Y3jkkdB3nzOZnIuKCnjlleBRknfe2X3CXz8j4UjV4p8ANLj7OndvBp4Dku9YPgV4CsDd\nFwMDzGxQmttKAv2nPirMc+EO8+bBF74QtPT//vegG6RYZHou+vWDP/856POfMgXeeKP4fwHoZyQc\nqb76GQJsTJjeBPxTGusMAQansa1I6A4ehK1b4b33guBbtCgY475gQXAztlIyYAC89RY88wxMnx70\n9//rv8K55wYjgAYPDrqGpLSkCv502wdZ/de56qpstu4+Vq6E//mfqKsoDCtXwpIlwft4K9X96Ku1\nNXi1tMChQ8Hr4MFgbP6hQ0GgnXVWcHHWAw/AGWdE91miVlEBt9wCN98Mf/tb0P//pz8F53j//uCv\nn379gkdO9u4N5eVHX2bHvhJF8Quj1H9GZs0KZ/hxp6N6zGwiMNPdJ8Wm7wVa3f1nCev8J1Dn7s/F\npuuBLwMjUm0bm1/kf3yKiEQj01E9qVr8S4BRZlYDbAGuBaYlrTMfmA48F/tFsdvdm8xsRxrbZly4\niIhkptPgd/cWM5sOLALKgSfdfYWZ3RJb/oS7LzCzyWbWABwAvtvZtrn8MCIiklrkF3CJiEh+RXpN\nXylf4GVmw8zsDTP72Mz+YWa3x+ZXmdlfzGyVmb1mZgOirjUfzKzczJaa2cux6ZI8DwBmNsDMXjCz\nFWa23Mz+qRTPh5ndG/v5+MjMfm9mJ5TKeTCzOWbWZGYfJczr8LPHztXqWJ5+LdX+Iwt+XeBFM3Cn\nu38OmAjcFvv8Pwb+4u5nAn+NTZeCGcByjo4kK9XzAPAosMDdxwLnAPWU2PmIfTf4PeA8dz+boLv4\nOkrnPMx3MyMeAAAClElEQVQlyMZE7X52MxtH8B3quNg2/2FmnWZ7lC3+kr7Ay90b3f2D2Pv9wAqC\n6x/aLoiL/fv1aCrMHzMbCkwGfsPRocEldx4AzKw/cIm7z4HguzJ330PpnY+9BI2jCjPrAVQQDBIp\nifPg7v8X2JU0u6PPPhWY5+7N7r4OaCDI1w5FGfwdXfhVcmKtm/HAYqDa3Ztii5qA6ojKyqdfAj8C\nWhPmleJ5gGAY9HYzm2tm75vZf5nZiZTY+XD3ncAjwAaCwN/t7n+hxM5Dko4++2CC/IxLmaVRBr++\nVQbMrC/wIjDD3fclLovdtrRbnyczuxLY5u5L6eBCwFI4Dwl6AOcB/+Hu5xGMlDumO6MUzoeZnQHc\nAdQQBFtfM/tO4jqlcB46ksZn7/S8RBn8m4FhCdPDOPa3VrdnZj0JQv8Zd/9TbHZT7F5HmNlpwLao\n6suTLwJTzGwtMA+41MyeofTOQ9wmYJO7vxebfoHgF0FjiZ2P84G33X2Hu7cAfwQupPTOQ6KOfiaS\ns3RobF6Hogz+tovDzKwXwZcT8yOsJ6/MzIAngeXu/quERfOBG2PvbwT+lLxtd+Lu97n7MHcfQfDl\n3d/c/XpK7DzEuXsjsNHMzozNugz4GHiZ0jof9cBEM+sT+1m5jODL/1I7D4k6+pmYD1xnZr3MbAQw\nCni30z25e2Qv4HJgJcGXEfdGWUsEn/1igj7tD4ClsdckoAp4HVgFvAYMiLrWPJ6TLwPzY+9L+Tyc\nC7wHLCNo6fYvxfMB3E3wS+8jgi8ze5bKeSD463cL8BnBd6Hf7eyzA/fFcrQe+JdU+9cFXCIiJUYP\nZRMRKTEKfhGREqPgFxEpMQp+EZESo+AXESkxCn4RkRKj4BcRKTEKfhGREvP/AfApOG5pWjCyAAAA\nAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "for data in dataset:\n", + " suite.Update(data)\n", + " \n", + "ps = thinkbayes.MakeUniformPmf(0, 100, 101).Probs(xs)\n", + "plt.plot(xs, ps, label='uniform', color='k')\n", + "\n", + "ps = suite.Probs(xs)\n", + "plt.plot(xs, ps, label='triangle');\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is an example of **swamping the priors** with enough\n", + "data, people who start with different priors will tend to converge on\n", + "the same posterior." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Optimization\n", + "\n", + "The code I have shown so far is meant to be easy to read, but it is not\n", + "very efficient. In general, I like to develop code that is demonstrably\n", + "correct, then check whether it is fast enough for my purposes. If so,\n", + "there is no need to optimize. For this example, if we care about run\n", + "time, there are several ways we can speed it up.\n", + "\n", + "The first opportunity is to reduce the number of times we normalize the\n", + "suite. In the original code, we call `Update` once for each spin.\n", + "\n", + "```python\n", + " dataset = 'H' * heads + 'T' * tails\n", + "\n", + " for data in dataset:\n", + " suite.Update(data)\n", + "```\n", + "\n", + "And here’s what `Update` looks like:\n", + "\n", + "```python\n", + " def Update(self, data):\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " return self.Normalize()\n", + "```\n", + "\n", + "Each update iterates through the hypotheses, then calls `Normalize`,\n", + "which iterates through the hypotheses again. We can save some time by\n", + "doing all of the updates before normalizing.\n", + "\n", + "`Suite` provides a method called `UpdateSet` that does exactly that.\n", + "Here it is:\n", + "\n", + "```\n", + " def UpdateSet(self, dataset):\n", + " for data in dataset:\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " return self.Normalize()\n", + "```\n", + "\n", + "And here’s how we can invoke it:" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "2.908843447705344e-75" + ] + }, + "execution_count": 52, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "heads, tails = 140, 110\n", + "\n", + "dataset = 'H' * heads + 'T' * tails\n", + "suite.UpdateSet(dataset)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This optimization speeds things up, but the run time is still\n", + "proportional to the amount of data. We can speed things up even more by\n", + "rewriting `Likelihood` to process the entire dataset, rather than one\n", + "spin at a time.\n", + "\n", + "In the original version, `data` is a string that encodes either heads or\n", + "tails:\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " x = hypo / 100.0\n", + " if data == 'H':\n", + " return x\n", + " else:\n", + " return 1-x\n", + "```\n", + "\n", + "As an alternative, we could encode the dataset as a tuple of two\n", + "integers: the number of heads and tails. In that case `Likelihood` looks\n", + "like this:\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " x = hypo / 100.0\n", + " heads, tails = data\n", + " like = x**heads * (1-x)**tails\n", + " return like\n", + "```\n", + "\n", + "And then we can call `Update` like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.4406779172095556" + ] + }, + "execution_count": 53, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "heads, tails = 140, 110\n", + "suite.Update((heads, tails))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since we have replaced repeated multiplication with exponentiation, this\n", + "version takes the same time for any number of spins." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The beta distribution\n", + "\n", + "There is one more optimization that solves this problem even faster.\n", + "\n", + "So far we have used a Pmf object to represent a discrete set of values\n", + "for $x$. Now we will use a continuous distribution,\n", + "specifically the beta distribution (see\n", + ").\n", + "\n", + "The beta distribution is defined on the interval from 0 to 1 (including\n", + "both), so it is a natural choice for describing proportions and\n", + "probabilities. But wait, it gets better.\n", + "\n", + "It turns out that if you do a Bayesian update with a binomial likelihood\n", + "function, as we did in the previous section, the beta distribution is a\n", + "**conjugate prior**. That means that if the prior\n", + "distribution for x is a beta distribution, the posterior is\n", + "also a beta distribution. But wait, it gets even better.\n", + "\n", + "The shape of the beta distribution depends on two parameters, written\n", + "$\\alpha$ and $\\beta$, or alpha and beta. If\n", + "the prior is a beta distribution with parameters alpha and\n", + "beta, and we see data with h heads and\n", + "t tails, the posterior is a beta distribution with\n", + "parameters alpha+h and beta+t. In other words,\n", + "we can do an update with two additions.\n", + "\n", + "So that’s great, but it only works if we can find a beta distribution\n", + "that is a good choice for a prior. Fortunately, for many realistic\n", + "priors there is a beta distribution that is at least a good\n", + "approximation, and for a uniform prior there is a perfect match. The\n", + "beta distribution with alpha=1 and beta=1 is\n", + "uniform from 0 to 1.\n", + "\n", + "Let’s see how we can take advantage of all this.\n", + "thinkbayes.py provides a class that represents a beta\n", + "distribution:\n", + "\n", + "```python\n", + "class Beta(object):\n", + "\n", + " def __init__(self, alpha=1, beta=1):\n", + " self.alpha = alpha\n", + " self.beta = beta\n", + "```\n", + "\n", + "By default `__init__` makes a uniform distribution. Update\n", + "performs a Bayesian update:\n", + "\n", + "```python\n", + "def Update(self, data):\n", + " heads, tails = data\n", + " self.alpha += heads\n", + " self.beta += tails\n", + "```\n", + "\n", + "`data` is a pair of integers representing the number of heads\n", + "and tails.\n", + "\n", + "So we have yet another way to solve the Euro problem:" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.5595238095238095\n" + ] + } + ], + "source": [ + "beta = thinkbayes.Beta()\n", + "beta.Update((140, 110))\n", + "print(beta.Mean())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Beta` provides `Mean,` which computes a simple\n", + "function of `alpha` and `beta`:\n", + "\n", + "```python\n", + "def Mean(self):\n", + " return float(self.alpha) / (self.alpha + self.beta)\n", + "```\n", + "\n", + "For the Euro problem the posterior mean is 56%, which is the same result\n", + "we got using Pmfs.\n", + "\n", + "`Beta` also provides `EvalPdf`, which evaluates\n", + "the probability density function (PDF) of the beta distribution:\n", + "\n", + "```python\n", + "def EvalPdf(self, x):\n", + " return x**(self.alpha-1) * (1-x)**(self.beta-1)\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "111" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "beta.beta" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "3.192662744667652e-75" + ] + }, + "execution_count": 72, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "beta.EvalPdf(.55)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, `Beta` provides `MakePmf`, which uses\n", + "`EvalPdf` to generate a discrete approximation of the beta\n", + "distribution." + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG+NJREFUeJzt3XmwnXWd5/H3h5uEsJpAQkIWE5DYLILCaMi44LGGrrpG\nJVVD0yytFGIL5XSQsSmHhppqLn/M2HZpgVRajJqmaHUmAr1FmkjLctR2MCQCESGhEzSQQBZIWNNg\ntu/88ZxLDie5Z32Wc+75vKpucZ/1fO9TuZ/74/tsigjMzKw/HFJ0AWZmlh+HvplZH3Hom5n1EYe+\nmVkfceibmfURh76ZWR9pGPqSBiWtlbRO0rUHWX6ypIckvSnpmoMsH5D0qKQfpVW0mZm1p27oSxoA\nFgGDwKnAxZJOqVltO3AV8LURdnM18CTgGwLMzArWaKQ/F1gfERsiYjewFFhQvUJEvBARq4DdtRtL\nmgHMB74LKJ2SzcysXY1CfzqwsWp6U2Ves24Cvgzsa7EuMzPLQKPQb7slI+mTwLaIeBSP8s3MusKY\nBsufA2ZWTc8kGe0344PAeZLmA+OBoyX9XURcWr2SJPf6zczaEBEtD6gbjfRXAXMkzZY0DrgQWDbC\num/78Ii4PiJmRsQJwEXAA7WBX7WuvyK44YYbCq+hW758LHwsfCzqf7Wr7kg/IvZIWgjcCwwASyJi\njaQrK8sXS5oKrASOBvZJuho4NSJer91d21WamVkqGrV3iIjlwPKaeYurvt/C21tAB9vHT4Gftlmj\nmZmlxHfkdpFSqVR0CV3Dx2I/H4v9fCw6p056Q6kUIEXRNZiZ9RpJRAYncs3MbBRx6JuZ9RGHvplZ\nH3Hom5n1EYe+mVkfceibmfURh76ZWR9x6JuZ9RGHvplZH3Hom5n1EYe+mVkfceibmfURh76ZWR9x\n6Jvl5M//HH72s6KrsH7X8CUqZta5116Db34Ttm2Dc84puhrrZx7pm+Xg7rvh9NPhX/4F3nyz6Gqs\nnzn0zXJwxx2wcCGccQb8678WXY31M785yyxjr74KM2bAs8/C978PK1bA975XdFXW6/zmLLMudffd\nSR9/wgQ4//xk+ve/L7oq61cOfbOM3XEHXHBB8v3xxye9/Z/8pNiarH859M0y9Oqr8OCDsGDB/nkX\nXAB33llcTdbfmgp9SYOS1kpaJ+nagyw/WdJDkt6UdE3V/JmSHpT0hKTfSPpimsWbdbsf/Qg+8pGk\ntTPs/POT+W7xWBEahr6kAWARMAicClws6ZSa1bYDVwFfq5m/G/hSRJwGzAP+7CDbmo1ad94Jf/zH\nb583bRqcdhrcd18xNVl/a2akPxdYHxEbImI3sBRYUL1CRLwQEatIQr56/paIeKzy/evAGmBaKpWb\ndbk330yC/bzzDlz2R38E//RP+ddk1kzoTwc2Vk1vqsxriaTZwJnAila3NetFzz4LU6e+vbUz7H3v\ng7Vr86/JrJnHMHR8Eb2kI4G7gKsrI/63GRoaeuv7UqlEqVTq9CPNCrdhA8yadfBls2bBM8/kWo71\nuHK5TLlc7ng/DW/OkjQPGIqIwcr0dcC+iPjqQda9AXg9Ir5eNW8scDewPCJuPsg2vjnLRqVvfzu5\nEWvJkgOX7d4NRxwBO3fC2LH512a9L8ubs1YBcyTNljQOuBBYNlIdNUUJWAI8ebDANxvNnnlm5JH+\n2LHJNfubNuVbk1nD0I+IPcBC4F7gSeCHEbFG0pWSrgSQNFXSRuBLwP+U9GylpfMh4NPAxyQ9Wvka\nzOynMesiGzbA7NkjL3eLx4rQ1KOVI2I5sLxm3uKq77cAMw+y6b/hG8CsTz3zjEPfuo8D2Swj9U7k\ngkPfiuHQN8vArl3JC1Om17m4edas5A+DWZ4c+mYZ2LgxCfwxdRqoHulbERz6Zhlo1NqBpN/v0Le8\nOfTNMtDoJC7AO9+ZXLK5b18uJZkBDn2zTDS6XBNg/PjkEQ1btuRRkVnCoW+WgWbaO+CTuZY/h75Z\nBppp74BP5lr+HPpmGWimvQMOfcufQ98sZXv2JH36GTMar+sreCxvDn2zlG3aBFOmNPf0TI/0LW8O\nfbOUNdvaAZ/Itfw59M1SVu+RyrWGR/p+pYTlxaFvlrJWRvpHHw3jxsH27VlWZLafQ98sZa2M9MEn\ncy1fDn2zlLUy0gf39S1fDn2zlLUT+h7pW14c+mYp2rsXnnsOZh7sPXIjcOhbnhz6Zil6/nmYNAkO\nPbT5bRz6lieHvlmKmn3mTjWHvuXJoW+Womafrllt9myfyLX8OPTNUrR5M0yb1to2xx4Lb74JO3dm\nU5NZNYe+WYq2bk2eu9MKCY47LnmRulnWGoa+pEFJayWtk3TtQZafLOkhSW9KuqaVbc1Gm3ZCH5Jt\ntm5Nvx6zWnVDX9IAsAgYBE4FLpZ0Ss1q24GrgK+1sa3ZqLJli0Pfulujkf5cYH1EbIiI3cBSYEH1\nChHxQkSsAna3uq3ZaLN1K0yd2vp2Dn3LS6PQnw5srJreVJnXjE62NetJ7bZ3jjvOoW/5GNNgeScP\nfG1626Ghobe+L5VKlEqlDj7WrBh798KOHcnNWa2aMgXWr0+/Jhs9yuUy5XK54/00Cv3ngOobymeS\njNib0fS21aFv1qteeAEmToQxjX6rDmLKFPjFL9KvyUaP2gHxjTfe2NZ+GrV3VgFzJM2WNA64EFg2\nwrrqYFuzntduPx/c07f81B2TRMQeSQuBe4EBYElErJF0ZWX5YklTgZXA0cA+SVcDp0bE6wfbNssf\nxqxI7fbzwaFv+Wn4P6IRsRxYXjNvcdX3W3h7G6futmajVSeh75uzLC++I9csJe1eow9wzDHw2muw\na1e6NZnVcuibpaSTnv4hh8DkyR7tW/Yc+mYp6aS9A+7rWz4c+mYpcehbL3Dom6Wkk54++GSu5cOh\nb5YSj/StFzj0zVIw/AiGyZPb34dD3/Lg0DdLQSePYBjm0Lc8OPTNUtBpawcc+pYPh75ZCtIIfZ/I\ntTw49M1S0MmNWcM80rc8OPTNUtDp5ZqQnATesSM5KWyWFYe+WQrSaO+MGQMTJsCLL6ZTk9nBOPTN\nUpBG6EOyD/f1LUsOfbMUpNHTB78r17Ln0DdLQRo9ffDJXMueQ98sBWm2dxz6liWHvlmH0ngEwzCH\nvmXNoW/WoRdf7PwRDMN8Itey5tA361Ba/XzwiVzLnkPfrENp9fPB7R3LnkPfrEMOfeslDn2zDqV1\njT7sf+haRDr7M6vVMPQlDUpaK2mdpGtHWOeWyvLVks6smn+dpCckPS7p/0g6NM3izbpBmj398ePh\nsMPg5ZfT2Z9ZrbqhL2kAWAQMAqcCF0s6pWad+cBJETEHuAK4tTJ/NvB54KyIOB0YAC5KuX6zwqXZ\n3gG3eCxbjUb6c4H1EbEhInYDS4EFNeucB9wOEBErgAmSpgCvAruBwyWNAQ4HnkuzeLNu4NC3XtIo\n9KcDG6umN1XmNVwnInYAXweeBZ4HXo6I+zor16z7bNvm0Lfe0eh2kmZPJ+mAGdK7gP8OzAZeAe6U\n9CcR8YPadYeGht76vlQqUSqVmvxYs+Jt3ZqcgE3LlCnJO3fNqpXLZcrlcsf7aRT6zwEzq6Znkozk\n660zozKvBPy/iNgOIOkfgA8CdUPfrJfs25fckZvGIxiGTZ7su3LtQLUD4htvvLGt/TRq76wC5kia\nLWkccCGwrGadZcClAJLmkbRxtgJPAfMkHSZJwLnAk21VadalduyAo46CcePS26fflWtZqjvSj4g9\nkhYC95JcfbMkItZIurKyfHFE3CNpvqT1wE7gs5Vlj0n6O5I/HPuAR4BvZ/izmOUu7X4+OPQtWw0f\nERURy4HlNfMW10wvHGHbvwb+upMCzbrZtm3p9vPBoW/Z8h25Zh1I+yQuOPQtWw59sw64vWO9xqFv\n1oEsRvoTJsDrr8OuXenu1wwc+mYdyWKkf8ghyWWbvlbfsuDQN+tAFiN9cIvHsuPQN+tAFiN9cOhb\ndhz6Zh3I4pJNcOhbdhz6Zh1we8d6jUPfrE07d8LevcljGNLm0LesOPTN2jTc2tEBz5jtnEPfsuLQ\nN2tTVidxwaFv2XHom7Upq5O44NC37Dj0zdqU1UlccOhbdhz6Zm3Ksr0z/CKVaPbddWZNcuibtSnL\nkf4RR8DAQPIMHrM0OfTN2pTlSB/c4rFsOPTN2pTlSB8c+pYNh75ZmzzSt17k0DdrU5aXbIJD37Lh\n0Ddrw5498NJLcOyx2X2GQ9+y4NA3a8P27TBxIowZk91nOPQtCw59szZkfRIXHPqWDYe+WRuyPokL\nDn3LRsPQlzQoaa2kdZKuHWGdWyrLV0s6s2r+BEl3SVoj6UlJ89Is3qwoHulbr6ob+pIGgEXAIHAq\ncLGkU2rWmQ+cFBFzgCuAW6sWfwO4JyJOAc4A1qRYu1lhsr5yBxz6lo1GI/25wPqI2BARu4GlwIKa\ndc4DbgeIiBXABElTJL0D+EhE/G1l2Z6IeCXd8s2KkUd7Z9Ik2LEjeVGLWVoahf50YGPV9KbKvEbr\nzABOAF6QdJukRyR9R9LhnRZs1g3yaO+MGQPveEcS/GZpaXTBWbPP+Kt9d1BU9n0WsDAiVkq6GfgL\n4C9rNx4aGnrr+1KpRKlUavJjzYqRx0gf9rd4Jk/O/rOsu5XLZcrlcsf7aRT6zwEzq6Znkozk660z\nozJPwKaIWFmZfxdJ6B+gOvTNekEeI33YH/qnnZb9Z1l3qx0Q33jjjW3tp1F7ZxUwR9JsSeOAC4Fl\nNessAy4FqFyd83JEbI2ILcBGSe+urHcu8ERbVZp1mbxH+mZpqTvSj4g9khYC9wIDwJKIWCPpysry\nxRFxj6T5ktYDO4HPVu3iKuAHlT8YT9csM+tJEfm1XBz6lraGN5FHxHJgec28xTXTC0fYdjXwgU4K\nNOs2r72WvODkiCOy/yyHvqXNd+SatWjzZjj++Hw+y6FvaXPom7Xo+edh2rR8Puu445KTxmZpceib\ntWjz5vxCf+pU2LIln8+y/uDQN2tRniP9adOSPzJmaXHom7Uoz9CfOjUJ/X378vk8G/0c+mYtyjP0\nx4+Ho45KXtpilgaHvlmL8gx9cIvH0uXQN2tR3qF//PHJZ5qlwaFv1oKIJIDzuk4fkj8wDn1Li0Pf\nrAWvvAJjx8KRR+b3mccf7/aOpcehb9aCvFs74JG+pcuhb9YCh771Ooe+WQuKCH23dyxNDn2zFnik\nb73OoW/WgiJCf/j5O74r19Lg0DdrQRGh77tyLU0OfbMWFBH64BaPpcehb9aCokLfJ3MtLQ59syZF\n5PvWrGoe6VtaHPpmTdq+PbkTd/z4/D/boW9pceibNamo1g64vWPpceibNanI0PdI39Li0DdrUtEj\nfYe+paFh6EsalLRW0jpJ146wzi2V5aslnVmzbEDSo5J+lFbRZkUoeqTv9o6loW7oSxoAFgGDwKnA\nxZJOqVlnPnBSRMwBrgBurdnN1cCTQKRVtFkRih7p+65cS0Ojkf5cYH1EbIiI3cBSYEHNOucBtwNE\nxApggqQpAJJmAPOB7wJKs3CzvBUZ+ocemtyV++KLxXy+jR6NQn86sLFqelNlXrPr3AR8GfD4xHpe\n3m/MquUWj6VhTIPlzbZkakfxkvRJYFtEPCqpVG/joaGht74vlUqUSnVXNytEkSN92H8Fz3vfW1wN\nVpxyuUy5XO54P4oYOdclzQOGImKwMn0dsC8ivlq1zreAckQsrUyvBUrAF4HPAHuA8cDRwN9HxKU1\nnxH1ajDrBnv3Jjdl7dwJ48YVU8Nll8E558Dllxfz+dZdJBERLbfNG7V3VgFzJM2WNA64EFhWs84y\n4NJKEfOAlyNiS0RcHxEzI+IE4CLggdrAN+sVL7wAEycWF/jga/UtHXXbOxGxR9JC4F5gAFgSEWsk\nXVlZvjgi7pE0X9J6YCfw2ZF2l2bhZnnavLnY1g4k5xPWrCm2But9jXr6RMRyYHnNvMU10wsb7OOn\nwE/bKdCsGxTdz4fk8x94oNgarPf5jlyzJjz3XLFX7oDbO5YOh75ZEzZsgNmzi63Bj2KwNDj0zZrw\n9NPwrncVW8Pxx8PWrb4r1zrj0DdrQjeE/qGHwoQJyeMYzNrl0DdrQjeEPsCJJya1mLXLoW/WwI4d\nSUvl2GOLriT5w+PQt0449M0aGB7lqwseGejQt0459M0a6JbWDiR1/Pa3RVdhvcyhb9ZAt4W+R/rW\nCYe+WQMOfRtNHPpmDXRT6E+dCv/xH/Dqq0VXYr3KoW/WQDeFvuTLNq0zDn2zOt54I3lF4YwZRVey\nn1s81gmHvlkdv/sdzJoFAwNFV7KfQ9864dA3q6ObWjvDHPrWCYe+WR0OfRttHPpmdTj0bbRx6JvV\n8fTTydUy3WTWrOT1jbt2FV2J9SKHvlkd3TjSHzsWpk9PXuxi1iqHvtkI9u6FZ57pvpE+uMVj7XPo\nm41g06bkccqHHVZ0JQdy6Fu7HPpmI+jG1s4wh761y6FvNgKHvo1GTYW+pEFJayWtk3TtCOvcUlm+\nWtKZlXkzJT0o6QlJv5H0xTSLN8uSQ99Go4ahL2kAWAQMAqcCF0s6pWad+cBJETEHuAK4tbJoN/Cl\niDgNmAf8We22Zt2qm0P/xBOTR0Ts21d0JdZrmhnpzwXWR8SGiNgNLAUW1KxzHnA7QESsACZImhIR\nWyLiscr814E1wLTUqjfL0FNPwbvfXXQVB3fUUcnX5s1FV2K9ppnQnw5srJreVJnXaJ23PZdQ0mzg\nTGBFq0Wa5W3nTli/Ht7znqIrGZlbPNaOMU2sE03uq/a10W9tJ+lI4C7g6sqI/22Ghobe+r5UKlEq\nlZr8SLNsPPIInH46HHpo0ZWM7KSTYN06OOecoiuxPJTLZcrlcsf7UUT9TJc0DxiKiMHK9HXAvoj4\natU63wLKEbG0Mr0W+GhEbJU0FrgbWB4RNx9k/9GoBrO8ff3rSc980aKiKxnZTTclof/NbxZdiRVB\nEhFRO9huqJn2zipgjqTZksYBFwLLatZZBlxaKWQe8HIl8AUsAZ48WOCbdauVK2Hu3KKrqO/ss2GF\nm6XWooahHxF7gIXAvcCTwA8jYo2kKyVdWVnnHuC3ktYDi4H/Vtn8Q8CngY9JerTyNZjFD2KWpocf\n7v7QP/NMWLMmebuXWbMatncyL8DtHesyL76YnCR96SU4pMtvX3z/++Eb34APfajoSixvWbZ3zPrK\nypVJmHZ74INbPNa6HvhnbZavhx+GD3yg6Cqac/bZ8MtfFl2F9RKHvlmNXjiJO2zePI/0rTUOfbMq\nEb1xEnfYnDnw2muwZUvRlVivcOibVXn2WRgYSN5M1Quk5A+UR/vWLIe+WZXhUb5aviaiOD6Za61w\n6JtV6aWTuMMc+tYKh75ZlV46iTts7lxYtSp5p69ZIw59s4q9e5MHrb3//UVX0ppJk2DyZFi7tuhK\nrBc49M0qVq6EGTPgmGOKrqR1vnTTmuXQN6v4/vfh4ouLrqI97utbs/zsHTNg167kMs2HH4YTTii6\nmtb9+tfwqU8lj4PuhcdHWOf87B2zDvz4x3Dyyb0Z+ABnnJG0pe6/v+hKrNs59M2A730PPvOZoqvo\nzJ/+KSxZUnQV1u3c3rG+9/LLMGsWbNgAEycWXU37Xnop+T+Vp5+GY48tuhrLmts7Zm268074wz/s\n7cCHpP5PfjI5IW02Eoe+9b3R0NoZ9rnPwXe/mzw4zuxgHPrW1zZsSF45+PGPF11JOj760eT1iStX\nFl2JdSuHvvW1m2+Giy6CceOKriQdhxwCl1/uE7o2Mp/Itb51//1w6aWwenXyKIPR4vnn4T3vgXXr\nfEJ3NPOJXLMWbN8Ol10Gt902ugIfYNq0pLd/ySV+CJsdyCN96zsRcMEFMHMm3HRT0dVkY88eGBxM\nHh73V39VdDWWhcxG+pIGJa2VtE7StSOsc0tl+WpJZ7ayrVnebrsN/v3f4StfKbqS7IwZA0uXwg9/\nCHfcUXQ11k3qhr6kAWARMAicClws6ZSadeYDJ0XEHOAK4NZmt7W3K5fLRZfQNbI4Fjt3wjXXwPXX\nww9+AOPHp/4RmWj3WEyaBP/4j7BwIfz85+nWVBT/jnSu0Uh/LrA+IjZExG5gKbCgZp3zgNsBImIF\nMEHS1Ca3tSr+B71f2sfi/vuT59Ns2QKPPw6nn57q7jPVybF43/uS/7O55BI4//zk8tRe5t+Rzo1p\nsHw6sLFqehNwdhPrTAemNbGtWereeCMJ90cegfvuSwJ/1y74m7+BT3yi6Ory94lPJO2sRYuS6/hL\nJfjwh+Gss5I/CkceWXSFlqdGod/sGdaOXiP9qU91svXo8dRT8KtfFV1Fd3jqqeQVgLD/7tKI/V/7\n9iVXpuzZA7//fRL0b7wB27Yl/506FU47Dc49F77wheQSxn5+5PBhh8GXvwyf/3zy2Ilf/Sp5XMPj\nj8PYsckjHCZOhMMPT6bHjEm+pOS4SQe+LL6Il8f7d6Rzda/ekTQPGIqIwcr0dcC+iPhq1TrfAsoR\nsbQyvRb4KHBCo20r833pjplZG9q5eqfRSH8VMEfSbOB54EKg9t1Cy4CFwNLKH4mXI2KrpO1NbNtW\n0WZm1p66oR8ReyQtBO4FBoAlEbFG0pWV5Ysj4h5J8yWtB3YCn623bZY/jJmZ1Vf4zVlmZpaf3E5t\ndXKT12jT6FhI+pPKMfi1pF9IOqOIOvPQ7A18kj4gaY+k/5pnfXlq8nekJOlRSb+RVM65xNw08Tsy\nSdKPJT1WORaXFVBm5iT9raStkh6vs05ruRkRmX+RtHfWA7OBscBjwCk168wH7ql8fzbwyzxqy/ur\nyWPxn4F3VL4f7OdjUbXeA8DdwPlF113gv4sJwBPAjMr0pKLrLvBYDAFfGT4OwHZgTNG1Z3AsPgKc\nCTw+wvKWczOvkX67N3lNyam+PDU8FhHxUES8UplcAczIuca8NHsD31XAXcALeRaXs2aOxSXA30fE\nJoCIeDHnGvPSzLHYDBxd+f5oYHtE7MmxxlxExM+Bl+qs0nJu5hX6I93A1Wid0Rh2zRyLap8D7sm0\nouI0PBaSppP8wt9amTVaT0I18+9iDnCMpAclrZI0St73dYBmjsV3gNMkPQ+sBq7OqbZu03JuNrpk\nMy3t3uQ1Gn/Bm/6ZJH0MuBz4UHblFKqZY3Ez8BcREZJEhzcCdrFmjsVY4CzgvwCHAw9J+mVErMu0\nsvw1cyyuBx6LiJKkdwE/kfTeiHgt49q6UUu5mVfoPwfMrJqeSfIXqd46MyrzRptmjgWVk7ffAQYj\not7/3vWyZo7FfyK5BwSS3u3HJe2OiGX5lJibZo7FRuDFiHgDeEPSz4D3AqMt9Js5Fh8E/hdARDwt\n6XfAH5DcW9RPWs7NvNo7b93kJWkcyY1atb+0y4BL4a07gV+OiK051ZenhsdC0juBfwA+HRHrC6gx\nLw2PRUScGBEnRMQJJH39L4zCwIfmfkf+GfiwpAFJh5OcuHsy5zrz0MyxWAucC1DpYf8B8Ntcq+wO\nLedmLiP96OAmr9GmmWMB/CUwEbi1MsLdHRFzi6o5K00ei77Q5O/IWkk/Bn4N7AO+ExGjLvSb/Hfx\nv4HbJK0mGbz+j4jYUVjRGZH0f0keazNJ0kbgBpI2X9u56ZuzzMz6SB8/d9DMrP849M3M+ohD38ys\njzj0zcz6iEPfzKyPOPTNzPqIQ9/MrI849M3M+sj/B/lIdTt46O5YAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "pmf = beta.MakePmf()\n", + "xs = [i / 100. for i in range(101)]\n", + "ps = pmf.Probs(xs)\n", + "plt.plot(xs, ps);" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Discussion\n", + "\n", + "In this chapter we solved the same problem with two different priors and\n", + "found that with a large dataset, the priors get swamped. If two people\n", + "start with different prior beliefs, they generally find, as they see\n", + "more data, that their posterior distributions converge. At some point\n", + "the difference between their distribution is small enough that it has no\n", + "practical effect.\n", + "\n", + "When this happens, it relieves some of the worry about objectivity that\n", + "I discussed in the previous chapter. And for many real-world problems\n", + "even stark prior beliefs can eventually be reconciled by data.\n", + "\n", + "But that is not always the case. First, remember that all Bayesian\n", + "analysis is based on modeling decisions. If you and I do not choose the\n", + "same model, we might interpret data differently. So even with the same\n", + "data, we would compute different likelihoods, and our posterior beliefs\n", + "might not converge.\n", + "\n", + "Also, notice that in a Bayesian update, we multiply each prior\n", + "probability by a likelihood, so if $\\mathrm{p}(H)$ is 0,\n", + "$\\mathrm{p}(H|D)$ is also 0, regardless of $D$. In the Euro\n", + "problem, if you are convinced that $x$ is less than 50%, and you assign\n", + "probability 0 to all other hypotheses, no amount of data will convince\n", + "you otherwise.\n", + "\n", + "This observation is the basis of **Cromwell’s rule**, which\n", + "is the recommendation that you should avoid giving a prior probability\n", + "of 0 to any hypothesis that is even remotely possible (see\n", + ").\n", + "\n", + "Cromwell’s rule is named after Oliver Cromwell, who wrote, “I beseech\n", + "you, in the bowels of Christ, think it possible that you may be\n", + "mistaken.” For Bayesians, this turns out to be good advice (even if it’s\n", + "a little overwrought)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Exercises\n", + "---------\n", + "\n", + "Suppose that instead of observing coin tosses directly, you measure the\n", + "outcome using an instrument that is not always correct. Specifically,\n", + "suppose there is a probability y that an actual heads is\n", + "reported as tails, or actual tails reported as heads.\n", + "\n", + "Write a class that estimates the bias of a coin given a series of\n", + "outcomes and the value of y.\n", + "\n", + "How does the spread of the posterior distribution depend on\n", + "y?\n", + "\n", + "This exercise is inspired by a question posted by a “redditor” named\n", + "dominosci on Reddit’s statistics “subreddit” at\n", + ".\n", + "\n", + "Reddit is an online forum with many interest groups called subreddits.\n", + "Users, called redditors, post links to online content and other web\n", + "pages. Other redditors vote on the links, giving an “upvote” to\n", + "high-quality links and a “downvote” to links that are bad or irrelevant.\n", + "\n", + "A problem, identified by dominosci, is that some redditors are more\n", + "reliable than others, and Reddit does not take this into account.\n", + "\n", + "The challenge is to devise a system so that when a redditor casts a\n", + "vote, the estimated quality of the link is updated in accordance with\n", + "the reliability of the redditor, and the estimated reliability of the\n", + "redditor is updated in accordance with the quality of the link.\n", + "\n", + "One approach is to model the quality of the link as the probability of\n", + "garnering an upvote, and to model the reliability of the redditor as the\n", + "probability of correctly giving an upvote to a high-quality item.\n", + "\n", + "Write class definitions for redditors and links and an update function\n", + "that updates both objects whenever a redditor casts a vote." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/05_Odds_and_Addends.ipynb b/notebooks/05_Odds_and_Addends.ipynb new file mode 100644 index 0000000..9bc1267 --- /dev/null +++ b/notebooks/05_Odds_and_Addends.ipynb @@ -0,0 +1,527 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Odds and Addends\n", + "================\n", + "\n", + "Odds\n", + "----\n", + "\n", + "One way to represent a probability is with a number between 0 and 1, but\n", + "that’s not the only way. If you have ever bet on a football game or a\n", + "horse race, you have probably encountered another representation of\n", + "probability, called **odds**.\n", + "\n", + "You might have heard expressions like “the odds are three to one,” but\n", + "you might not know what that means. The **odds in favor**\n", + "of an event are the ratio of the probability it will occur to the\n", + "probability that it will not.\n", + "\n", + "So if I think my team has a 75% chance of winning, I would say that the\n", + "odds in their favor are three to one, because the chance of winning is\n", + "three times the chance of losing.\n", + "\n", + "You can write odds in decimal form, but it is most common to write them\n", + "as a ratio of integers. So “three to one” is written $3:1$.\n", + "\n", + "When probabilities are low, it is more common to report the **odds\n", + "against** rather than the odds in favor. For example, if I think\n", + "my horse has a 10% chance of winning, I would say that the odds against\n", + "are $9:1$.\n", + "\n", + "Probabilities and odds are different representations of the same\n", + "information. Given a probability, you can compute the odds like this:\n", + "\n", + " def Odds(p):\n", + " return p / (1-p)\n", + "\n", + "Given the odds in favor, in decimal form, you can convert to probability\n", + "like this:\n", + "\n", + " def Probability(o):\n", + " return o / (o+1)\n", + "\n", + "If you represent odds with a numerator and denominator, you can convert\n", + "to probability like this:\n", + "\n", + " def Probability2(yes, no):\n", + " return yes / (yes + no)\n", + "\n", + "When I work with odds in my head, I find it helpful to picture people at\n", + "the track. If 20% of them think my horse will win, then 80% of them\n", + "don’t, so the odds in favor are $20:80$ or $1:4$.\n", + "\n", + "If the odds are $5:1$ against my horse, then five out of six people\n", + "think she will lose, so the probability of winning is $1/6$.\n", + "\n", + "The odds form of Bayes’s theorem\n", + "--------------------------------\n", + "\n", + "In Chapter [intro] I wrote Bayes’s theorem in the **probability\n", + "form**:\n", + "$${\\ensuremath{\\mathrm{p}(H|D)}} = \\frac{{\\ensuremath{\\mathrm{p}(H)}}~{\\ensuremath{\\mathrm{p}(D|H)}}}{{\\ensuremath{\\mathrm{p}(D)}}}$$\n", + "If we have two hypotheses, $A$ and $B$, we can write the ratio of\n", + "posterior probabilities like this:\n", + "$$\\frac{{\\ensuremath{\\mathrm{p}(A|D)}}}{{\\ensuremath{\\mathrm{p}(B|D)}}} = \\frac{{\\ensuremath{\\mathrm{p}(A)}}~{\\ensuremath{\\mathrm{p}(D|A)}}}\n", + " {{\\ensuremath{\\mathrm{p}(B)}}~{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "Notice that the normalizing constant, $\\mathrm{p}(D)$,\n", + "drops out of this equation.\n", + "\n", + "If $A$ and $B$ are mutually exclusive and collectively exhaustive, that\n", + "means ${\\ensuremath{\\mathrm{p}(B)}} = 1 - {\\ensuremath{\\mathrm{p}(A)}}$,\n", + "so we can rewrite the ratio of the priors, and the ratio of the\n", + "posteriors, as odds.\n", + "\n", + "Writing $\\mathrm{o}(A)$ for odds in favor of $A$, we get:\n", + "$${\\ensuremath{\\mathrm{o}(A|D)}} = {\\ensuremath{\\mathrm{o}(A)}}~\\frac{{\\ensuremath{\\mathrm{p}(D|A)}}}{{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "In words, this says that the posterior odds are the prior odds times the\n", + "likelihood ratio. This is the **odds form** of Bayes’s\n", + "theorem.\n", + "\n", + "This form is most convenient for computing a Bayesian update on paper or\n", + "in your head. For example, let’s go back to the cookie problem:\n", + "\n", + "> Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla\n", + "> cookies and 10 chocolate cookies. Bowl 2 contains 20 of each.\n", + ">\n", + "> Now suppose you choose one of the bowls at random and, without\n", + "> looking, select a cookie at random. The cookie is vanilla. What is the\n", + "> probability that it came from Bowl 1?\n", + "\n", + "The prior probability is 50%, so the prior odds are $1:1$, or just 1.\n", + "The likelihood ratio is $\\frac{3}{4} / \\frac{1}{2}$, or $3/2$. So the\n", + "posterior odds are $3:2$, which corresponds to probability $3/5$.\n", + "\n", + "Oliver’s blood {#oliver}\n", + "--------------\n", + "\n", + "Here is another problem from MacKay’s *Information Theory,\n", + "Inference, and Learning Algorithms*:\n", + "\n", + "> Two people have left traces of their own blood at the scene of a\n", + "> crime. A suspect, Oliver, is tested and found to have type ‘O’ blood.\n", + "> The blood groups of the two traces are found to be of type ‘O’ (a\n", + "> common type in the local population, having frequency 60%) and of type\n", + "> ‘AB’ (a rare type, with frequency 1%). Do these data [the traces found\n", + "> at the scene] give evidence in favor of the proposition that Oliver\n", + "> was one of the people [who left blood at the scene]?\n", + "\n", + "To answer this question, we need to think about what it means for data\n", + "to give evidence in favor of (or against) a hypothesis. Intuitively, we\n", + "might say that data favor a hypothesis if the hypothesis is more likely\n", + "in light of the data than it was before.\n", + "\n", + "In the cookie problem, the prior odds are $1:1$, or probability 50%. The\n", + "posterior odds are $3:2$, or probability 60%. So we could say that the\n", + "vanilla cookie is evidence in favor of Bowl 1.\n", + "\n", + "The odds form of Bayes’s theorem provides a way to make this intuition\n", + "more precise. Again\n", + "$${\\ensuremath{\\mathrm{o}(A|D)}} = {\\ensuremath{\\mathrm{o}(A)}}~\\frac{{\\ensuremath{\\mathrm{p}(D|A)}}}{{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "Or dividing through by $\\mathrm{o}(A)$:\n", + "$$\\frac{{\\ensuremath{\\mathrm{o}(A|D)}}}{{\\ensuremath{\\mathrm{o}(A)}}} = \\frac{{\\ensuremath{\\mathrm{p}(D|A)}}}{{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "The term on the left is the ratio of the posterior and prior odds. The\n", + "term on the right is the likelihood ratio, also called the **Bayes\n", + "factor**.\n", + "\n", + "If the Bayes factor value is greater than 1, that means that the data\n", + "were more likely under $A$ than under $B$. And since the odds ratio is\n", + "also greater than 1, that means that the odds are greater, in light of\n", + "the data, than they were before.\n", + "\n", + "If the Bayes factor is less than 1, that means the data were less likely\n", + "under $A$ than under $B$, so the odds in favor of $A$ go down.\n", + "\n", + "Finally, if the Bayes factor is exactly 1, the data are equally likely\n", + "under either hypothesis, so the odds do not change.\n", + "\n", + "Now we can get back to the Oliver’s blood problem. If Oliver is one of\n", + "the people who left blood at the crime scene, then he accounts for the\n", + "‘O’ sample, so the probability of the data is just the probability that\n", + "a random member of the population has type ‘AB’ blood, which is 1%.\n", + "\n", + "If Oliver did not leave blood at the scene, then we have two samples to\n", + "account for. If we choose two random people from the population, what is\n", + "the chance of finding one with type ‘O’ and one with type ‘AB’? Well,\n", + "there are two ways it might happen: the first person we choose might\n", + "have type ‘O’ and the second ‘AB’, or the other way around. So the total\n", + "probability is $2 (0.6) (0.01) = 1.2\\%$.\n", + "\n", + "The likelihood of the data is slightly higher if Oliver is\n", + "*not* one of the people who left blood at the scene, so the\n", + "blood data is actually evidence against Oliver’s guilt.\n", + "\n", + "This example is a little contrived, but it is an example of the\n", + "counterintuitive result that data *consistent* with a\n", + "hypothesis are not necessarily *in favor of* the\n", + "hypothesis.\n", + "\n", + "If this result is so counterintuitive that it bothers you, this way of\n", + "thinking might help: the data consist of a common event, type ‘O’ blood,\n", + "and a rare event, type ‘AB’ blood. If Oliver accounts for the common\n", + "event, that leaves the rare event still unexplained. If Oliver doesn’t\n", + "account for the ‘O’ blood, then we have two chances to find someone in\n", + "the population with ‘AB’ blood. And that factor of two makes the\n", + "difference.\n", + "\n", + "Addends\n", + "-------\n", + "\n", + "The fundamental operation of Bayesian statistics is Update,\n", + "which takes a prior distribution and a set of data, and produces a\n", + "posterior distribution. But solving real problems usually involves a\n", + "number of other operations, including scaling, addition and other\n", + "arithmetic operations, max and min, and mixtures.\n", + "\n", + "This chapter presents addition and max; I will present other operations\n", + "as we need them.\n", + "\n", + "The first example is based on *Dungeons & Dragons*, a\n", + "role-playing game where the results of players’ decisions are usually\n", + "determined by rolling dice. In fact, before game play starts, players\n", + "generate each attribute of their characters—strength, intelligence,\n", + "wisdom, dexterity, constitution, and charisma—by rolling three 6-sided\n", + "dice and adding them up.\n", + "\n", + "So you might be curious to know the distribution of this sum. There are\n", + "two ways you might compute it:\n", + "\n", + "Simulation:\n", + ": Given a Pmf that represents the distribution for a single die, you\n", + " can draw random samples, add them up, and accumulate the\n", + " distribution of simulated sums.\n", + "\n", + "Enumeration:\n", + ": Given two Pmfs, you can enumerate all possible pairs of values and\n", + " compute the distribution of the sums.\n", + "\n", + "`thinkbayes` provides functions for both. Here’s an example of the first\n", + "approach. First, I’ll define a class to represent a single die as a Pmf:\n", + "\n", + " class Die(thinkbayes.Pmf):\n", + "\n", + " def __init__(self, sides):\n", + " thinkbayes.Pmf.__init__(self)\n", + " for x in xrange(1, sides+1):\n", + " self.Set(x, 1)\n", + " self.Normalize()\n", + "\n", + "Now I can create a 6-sided die:\n", + "\n", + " d6 = Die(6)\n", + "\n", + "And use `thinkbayes.SampleSum` to generate a sample of 1000 rolls.\n", + "\n", + " dice = [d6] * 3\n", + " three = thinkbayes.SampleSum(dice, 1000)\n", + "\n", + "`SampleSum` takes list of distributions (either Pmf or Cdf objects) and\n", + "the sample size, n. It generates n random sums\n", + "and returns their distribution as a Pmf object.\n", + "\n", + " def SampleSum(dists, n):\n", + " pmf = MakePmfFromList(RandomSum(dists) for i in xrange(n))\n", + " return pmf\n", + "\n", + "`SampleSum` uses `RandomSum`, also in `thinkbayes.py`:\n", + "\n", + " def RandomSum(dists):\n", + " total = sum(dist.Random() for dist in dists)\n", + " return total\n", + "\n", + "RandomSum invokes Random on each distribution\n", + "and adds up the results.\n", + "\n", + "The drawback of simulation is that the result is only approximately\n", + "correct. As `n` gets larger, it gets more accurate, but of course the\n", + "run time increases as well.\n", + "\n", + "The other approach is to enumerate all pairs of values and compute the\n", + "sum and probability of each pair. This is implemented in `Pmf.__add__`:\n", + "\n", + " # class Pmf\n", + "\n", + " def __add__(self, other):\n", + " pmf = Pmf()\n", + " for v1, p1 in self.Items():\n", + " for v2, p2 in other.Items():\n", + " pmf.Incr(v1+v2, p1*p2)\n", + " return pmf\n", + "\n", + "self is a Pmf, of course; other can be a Pmf\n", + "or anything else that provides Items. The result is a new\n", + "Pmf. The time to run `__add__` depends on the number of items in\n", + "self and other; it is proportional to\n", + "len(self) \\* len(other).\n", + "\n", + "And here’s how it’s used:\n", + "\n", + " three_exact = d6 + d6 + d6\n", + "\n", + "When you apply the + operator to a Pmf, Python invokes\n", + "`__add__`. In this example, `__add__` is invoked twice.\n", + "\n", + "Figure [fig.dungeons1] shows an approximate result generated by\n", + "simulation and the exact result computed by enumeration.\n", + "\n", + "![Approximate and exact distributions for the sum of three 6-sided\n", + "dice.](figs/dungeons1.pdf)\n", + "\n", + "[fig.dungeons1]\n", + "\n", + "`Pmf.__add__` is based on the assumption that the random selections from\n", + "each Pmf are independent. In the example of rolling several dice, this\n", + "assumption is pretty good. In other cases, we would have to extend this\n", + "method to use conditional probabilities.\n", + "\n", + "The code from this section is available from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Maxima\n", + "------\n", + "\n", + "![Distribution of the maximum of six rolls of three\n", + "dice.](figs/dungeons2.pdf)\n", + "\n", + "[fig.dungeons2]\n", + "\n", + "When you generate a *Dungeons & Dragons* character, you are\n", + "particularly interested in the character’s best attributes, so you might\n", + "like to know the distribution of the maximum attribute.\n", + "\n", + "There are three ways to compute the distribution of a maximum:\n", + "\n", + "Simulation:\n", + ": Given a Pmf that represents the distribution for a single selection,\n", + " you can generate random samples, find the maximum, and accumulate\n", + " the distribution of simulated maxima.\n", + "\n", + "Enumeration:\n", + ": Given two Pmfs, you can enumerate all possible pairs of values and\n", + " compute the distribution of the maximum.\n", + "\n", + "Exponentiation:\n", + ": If we convert a Pmf to a Cdf, there is a simple and efficient\n", + " algorithm for finding the Cdf of the maximum.\n", + "\n", + "The code to simulate maxima is almost identical to the code for\n", + "simulating sums:\n", + "\n", + " def RandomMax(dists):\n", + " total = max(dist.Random() for dist in dists)\n", + " return total\n", + "\n", + " def SampleMax(dists, n):\n", + " pmf = MakePmfFromList(RandomMax(dists) for i in xrange(n))\n", + " return pmf\n", + "\n", + "All I did was replace “sum” with “max”. And the code for enumeration is\n", + "almost identical, too:\n", + "\n", + " def PmfMax(pmf1, pmf2):\n", + " res = thinkbayes.Pmf()\n", + " for v1, p1 in pmf1.Items():\n", + " for v2, p2 in pmf2.Items():\n", + " res.Incr(max(v1, v2), p1*p2)\n", + " return res\n", + "\n", + "In fact, you could generalize this function by taking the appropriate\n", + "operator as a parameter.\n", + "\n", + "The only problem with this algorithm is that if each Pmf has $m$ values,\n", + "the run time is proportional to $m^2$. And if we want the maximum of\n", + "k selections, it takes time proportional to $k m^2$.\n", + "\n", + "If we convert the Pmfs to Cdfs, we can do the same calculation much\n", + "faster! The key is to remember the definition of the cumulative\n", + "distribution function: $$CDF(x) = {\\ensuremath{\\mathrm{p}(X \\le~x)}}$$\n", + "where $X$ is a random variable that means “a value chosen randomly from\n", + "this distribution.” So, for example, $CDF(5)$ is the probability that a\n", + "value from this distribution is less than or equal to 5.\n", + "\n", + "If I draw $X$ from $CDF_1$ and $Y$ from $CDF_2$, and compute the maximum\n", + "$Z = max(X, Y)$, what is the chance that $Z$ is less than or equal to 5?\n", + "Well, in that case both $X$ and $Y$ must be less than or equal to 5.\n", + "\n", + "If the selections of $X$ and $Y$ are independent,\n", + "$$CDF_3(5) = CDF_1(5) CDF_2(5)$$ where $CDF_3$ is the distribution of\n", + "$Z$. I chose the value 5 because I think it makes the formulas easy to\n", + "read, but we can generalize for any value of $z$:\n", + "$$CDF_3(z) = CDF_1(z) CDF_2(z)$$ In the special case where we draw $k$\n", + "values from the same distribution, $$CDF_k(z) = CDF_1(z)^k$$ So to find\n", + "the distribution of the maximum of $k$ values, we can enumerate the\n", + "probabilities in the given Cdf and raise them to the $k$th power. `Cdf`\n", + "provides a method that does just that:\n", + "\n", + " # class Cdf\n", + "\n", + " def Max(self, k):\n", + " cdf = self.Copy()\n", + " cdf.ps = [p**k for p in cdf.ps]\n", + " return cdf\n", + "\n", + "`Max` takes the number of selections, k, and returns a new\n", + "Cdf that represents the distribution of the maximum of k\n", + "selections. The run time for this method is proportional to $m$, the\n", + "number of items in the Cdf.\n", + "\n", + "`Pmf.Max` does the same thing for Pmfs. It has to do a little more work\n", + "to convert the Pmf to a Cdf, so the run time is proportional to\n", + "$m \\log m$, but that’s still better than quadratic.\n", + "\n", + "Finally, here’s an example that computes the distribution of a\n", + "character’s best attribute:\n", + "\n", + " best_attr_cdf = three_exact.Max(6)\n", + " best_attr_pmf = best_attr_cdf.MakePmf()\n", + "\n", + "Where `three_exact` is defined in the previous section. If we print the\n", + "results, we see that the chance of generating a character with at least\n", + "one attribute of 18 is about 3%. Figure [fig.dungeons2] shows the\n", + "distribution.\n", + "\n", + "Mixtures {#mixture}\n", + "--------\n", + "\n", + "![Distribution outcome for random die from a box.](figs/dungeons3.pdf)\n", + "\n", + "[fig.dungeons3]\n", + "\n", + "Let’s do one more example from *Dungeons & Dragons*.\n", + "Suppose I have a box of dice with the following inventory:\n", + "\n", + " 5 4-sided dice\n", + " 4 6-sided dice\n", + " 3 8-sided dice\n", + " 2 12-sided dice\n", + " 1 20-sided die\n", + "\n", + "I choose a die from the box and roll it. What is the distribution of the\n", + "outcome?\n", + "\n", + "If you know which die it is, the answer is easy. A die with\n", + "n sides yields a uniform distribution from 1 to\n", + "n, including both.\n", + "\n", + "But if we don’t know which die it is, the resulting distribution is a\n", + "**mixture** of uniform distributions with different bounds.\n", + "In general, this kind of mixture does not fit any simple mathematical\n", + "model, but it is straightforward to compute the distribution in the form\n", + "of a PMF.\n", + "\n", + "As always, one option is to simulate the scenario, generate a random\n", + "sample, and compute the PMF of the sample. This approach is simple and\n", + "it generates an approximate solution quickly. But if we want an exact\n", + "solution, we need a different approach.\n", + "\n", + "Let’s start with a simple version of the problem where there are only\n", + "two dice, one with 6 sides and one with 8. We can make a Pmf to\n", + "represent each die:\n", + "\n", + " d6 = Die(6)\n", + " d8 = Die(8)\n", + "\n", + "Then we create a Pmf to represent the mixture:\n", + "\n", + " mix = thinkbayes.Pmf()\n", + " for die in [d6, d8]:\n", + " for outcome, prob in die.Items():\n", + " mix.Incr(outcome, prob)\n", + " mix.Normalize()\n", + "\n", + "The first loop enumerates the dice; the second enumerates the outcomes\n", + "and their probabilities. Inside the loop, Pmf.Incr adds up\n", + "the contributions from the two distributions.\n", + "\n", + "This code assumes that the two dice are equally likely. More generally,\n", + "we need to know the probability of each die so we can weight the\n", + "outcomes accordingly.\n", + "\n", + "First we create a Pmf that maps from each die to the probability it is\n", + "selected:\n", + "\n", + " pmf_dice = thinkbayes.Pmf()\n", + " pmf_dice.Set(Die(4), 5)\n", + " pmf_dice.Set(Die(6), 4)\n", + " pmf_dice.Set(Die(8), 3)\n", + " pmf_dice.Set(Die(12), 2)\n", + " pmf_dice.Set(Die(20), 1)\n", + " pmf_dice.Normalize()\n", + "\n", + "Next we need a more general version of the mixture algorithm:\n", + "\n", + " mix = thinkbayes.Pmf()\n", + " for die, weight in pmf_dice.Items():\n", + " for outcome, prob in die.Items():\n", + " mix.Incr(outcome, weight*prob)\n", + "\n", + "Now each die has a weight associated with it (which makes it a weighted\n", + "die, I suppose). When we add each outcome to the mixture, its\n", + "probability is multiplied by weight.\n", + "\n", + "Figure [fig.dungeons3] shows the result. As expected, values 1 through 4\n", + "are the most likely because any die can produce them. Values above 12\n", + "are unlikely because there is only one die in the box that can produce\n", + "them (and it does so less than half the time).\n", + "\n", + "thinkbayes provides a function named\n", + "MakeMixture that encapsulates this algorithm, so we could\n", + "have written:\n", + "\n", + " mix = thinkbayes.MakeMixture(pmf_dice)\n", + "\n", + "We’ll use MakeMixture again in Chapters [prediction] and\n", + " [observer].\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "Other than the odds form of Bayes’s theorem, this chapter is not\n", + "specifically Bayesian. But Bayesian analysis is all about distributions,\n", + "so it is important to understand the concept of a distribution well.\n", + "From a computational point of view, a distribution is any data structure\n", + "that represents a set of values (possible outcomes of a random process)\n", + "and their probabilities.\n", + "\n", + "We have seen two representations of distributions: Pmfs and Cdfs. These\n", + "representations are equivalent in the sense that they contain the same\n", + "information, so you can convert from one to the other. The primary\n", + "difference between them is performance: some operations are faster and\n", + "easier with a Pmf; others are faster with a Cdf.\n", + "\n", + "The other goal of this chapter is to introduce operations that act on\n", + "distributions, like `Pmf.__add__`, Cdf.Max, and\n", + "thinkbayes.MakeMixture. We will use these operations later,\n", + "but I introduce them now to encourage you to think of a distribution as\n", + "a fundamental unit of computation, not just a container for values and\n", + "probabilities.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/06_Decision_Analysis.ipynb b/notebooks/06_Decision_Analysis.ipynb new file mode 100644 index 0000000..e17152d --- /dev/null +++ b/notebooks/06_Decision_Analysis.ipynb @@ -0,0 +1,625 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Decision Analysis {#decisionanalysis}\n", + "=================\n", + "\n", + "The *Price is Right* problem\n", + "-----------------------------------------\n", + "\n", + "On November 1, 2007, contestants named Letia and Nathaniel appeared on\n", + "*The Price is Right*, an American game show. They competed\n", + "in a game called *The Showcase*, where the objective is to\n", + "guess the price of a showcase of prizes. The contestant who comes\n", + "closest to the actual price of the showcase, without going over, wins\n", + "the prizes.\n", + "\n", + "Nathaniel went first. His showcase included a dishwasher, a wine\n", + "cabinet, a laptop computer, and a car. He bid \\$26,000.\n", + "\n", + "Letia’s showcase included a pinball machine, a video arcade game, a pool\n", + "table, and a cruise of the Bahamas. She bid \\$21,500.\n", + "\n", + "The actual price of Nathaniel’s showcase was \\$25,347. His bid was too\n", + "high, so he lost.\n", + "\n", + "The actual price of Letia’s showcase was \\$21,578. She was only off by\n", + "\\$78, so she won her showcase and, because her bid was off by less than\n", + "\\$250, she also won Nathaniel’s showcase.\n", + "\n", + "For a Bayesian thinker, this scenario suggests several questions:\n", + "\n", + "1. Before seeing the prizes, what prior beliefs should the contestant\n", + " have about the price of the showcase?\n", + "\n", + "2. After seeing the prizes, how should the contestant update those\n", + " beliefs?\n", + "\n", + "3. Based on the posterior distribution, what should the contestant bid?\n", + "\n", + "The third question demonstrates a common use of Bayesian analysis:\n", + "decision analysis. Given a posterior distribution, we can choose the bid\n", + "that maximizes the contestant’s expected return.\n", + "\n", + "This problem is inspired by an example in Cameron Davidson-Pilon’s book,\n", + "*Bayesian Methods for Hackers*. The code I wrote for this\n", + "chapter is available from ; it reads\n", + "data files you can download from\n", + " and\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "The prior\n", + "---------\n", + "\n", + "![Distribution of prices for showcases on *The Price is\n", + "Right*, 2011-12.](figs/price1.pdf)\n", + "\n", + "[fig.price1]\n", + "\n", + "To choose a prior distribution of prices, we can take advantage of data\n", + "from previous episodes. Fortunately, fans of the show keep detailed\n", + "records. When I corresponded with Mr. Davidson-Pilon about his book, he\n", + "sent me data collected by Steve Gee at . It\n", + "includes the price of each showcase from the 2011 and 2012 seasons and\n", + "the bids offered by the contestants.\n", + "\n", + "Figure [fig.price1] shows the distribution of prices for these\n", + "showcases. The most common value for both showcases is around \\$28,000,\n", + "but the first showcase has a second mode near \\$50,000, and the second\n", + "showcase is occasionally worth more than \\$70,000.\n", + "\n", + "These distributions are based on actual data, but they have been\n", + "smoothed by Gaussian kernel density estimation (KDE). Before we go on, I\n", + "want to take a detour to talk about probability density functions and\n", + "KDE.\n", + "\n", + "Probability density functions\n", + "-----------------------------\n", + "\n", + "So far we have been working with probability mass functions, or PMFs. A\n", + "PMF is a map from each possible value to its probability. In my\n", + "implementation, a Pmf object provides a method named Prob\n", + "that takes a value and returns a probability, also known as a\n", + "**probability mass**.\n", + "\n", + "A **probability density function**, or PDF, is the\n", + "continuous version of a PMF, where the possible values make up a\n", + "continuous range rather than a discrete set.\n", + "\n", + "In mathematical notation, PDFs are usually written as functions; for\n", + "example, here is the PDF of a Gaussian distribution with mean 0 and\n", + "standard deviation 1: $$f(x) = \\frac{1}{\\sqrt{2 \\pi}} \\exp(-x^2/2)$$ For\n", + "a given value of $x$, this function computes a probability density. A\n", + "density is similar to a probability mass in the sense that a higher\n", + "density indicates that a value is more likely.\n", + "\n", + "But a density is not a probability. A density can be 0 or any positive\n", + "value; it is not bounded, like a probability, between 0 and 1.\n", + "\n", + "If you integrate a density over a continuous range, the result is a\n", + "probability. But for the applications in this book we seldom have to do\n", + "that.\n", + "\n", + "Instead we primarily use probability densities as part of a likelihood\n", + "function. We will see an example soon.\n", + "\n", + "Representing PDFs\n", + "-----------------\n", + "\n", + "To represent PDFs in Python, thinkbayes.py provides a class\n", + "named Pdf. Pdf is an **abstract\n", + "type**, which means that it defines the interface a Pdf is\n", + "supposed to have, but does not provide a complete implementation. The\n", + "Pdf interface includes two methods, Density\n", + "and MakePmf:\n", + "\n", + " class Pdf(object):\n", + "\n", + " def Density(self, x):\n", + " raise UnimplementedMethodException()\n", + "\n", + " def MakePmf(self, xs):\n", + " pmf = Pmf()\n", + " for x in xs:\n", + " pmf.Set(x, self.Density(x))\n", + " pmf.Normalize()\n", + " return pmf\n", + "\n", + "Density takes a value, x, and returns the\n", + "corresponding density. MakePmf makes a discrete\n", + "approximation to the PDF.\n", + "\n", + "Pdf provides an implementation of MakePmf, but\n", + "not Density, which has to be provided by a child class.\n", + "\n", + "A **concrete type** is a child class that extends an\n", + "abstract type and provides an implementation of the missing methods. For\n", + "example, GaussianPdf extends Pdf and provides\n", + "Density:\n", + "\n", + " class GaussianPdf(Pdf):\n", + "\n", + " def __init__(self, mu, sigma):\n", + " self.mu = mu\n", + " self.sigma = sigma\n", + " \n", + " def Density(self, x):\n", + " return scipy.stats.norm.pdf(x, self.mu, self.sigma)\n", + "\n", + "`__init__` takes mu and sigma, which are the\n", + "mean and standard deviation of the distribution, and stores them as\n", + "attributes.\n", + "\n", + "Density uses a function from scipy.stats to\n", + "evaluate the Gaussian PDF. The function is called norm.pdf\n", + "because the Gaussian distribution is also called the “normal”\n", + "distribution.\n", + "\n", + "The Gaussian PDF is defined by a simple mathematical function, so it is\n", + "easy to evaluate. And it is useful because many quantities in the real\n", + "world have distributions that are approximately Gaussian.\n", + "\n", + "But with real data, there is no guarantee that the distribution is\n", + "Gaussian or any other simple mathematical function. In that case we can\n", + "use a sample to estimate the PDF of the whole population.\n", + "\n", + "For example, in *The Price Is Right* data, we have 313\n", + "prices for the first showcase. We can think of these values as a sample\n", + "from the population of all possible showcase prices.\n", + "\n", + "This sample includes the following values (in order):\n", + "$$28800, 28868, 28941, 28957, 28958$$ In the sample, no values appear\n", + "between 28801 and 28867, but there is no reason to think that these\n", + "values are impossible. Based on our background information, we expect\n", + "all values in this range to be equally likely. In other words, we expect\n", + "the PDF to be fairly smooth.\n", + "\n", + "Kernel density estimation (KDE) is an algorithm that takes a sample and\n", + "finds an appropriately smooth PDF that fits the data. You can read\n", + "details at .\n", + "\n", + "scipy provides an implementation of KDE and\n", + "thinkbayes provides a class called\n", + "EstimatedPdf that uses it:\n", + "\n", + " class EstimatedPdf(Pdf):\n", + "\n", + " def __init__(self, sample):\n", + " self.kde = scipy.stats.gaussian_kde(sample)\n", + "\n", + " def Density(self, x):\n", + " return self.kde.evaluate(x)\n", + "\n", + "`__init__` takes a sample and computes a kernel density estimate. The\n", + "result is a `gaussian_kde` object that provides an evaluate\n", + "method.\n", + "\n", + "Density takes a value, calls `gaussian_kde.evaluate`, and\n", + "returns the resulting density.\n", + "\n", + "Finally, here’s an outline of the code I used to generate\n", + "Figure [fig.price1]:\n", + "\n", + " prices = ReadData()\n", + " pdf = thinkbayes.EstimatedPdf(prices)\n", + "\n", + " low, high = 0, 75000\n", + " n = 101\n", + " xs = numpy.linspace(low, high, n) \n", + " pmf = pdf.MakePmf(xs)\n", + "\n", + "pdf is a Pdf object, estimated by KDE.\n", + "pmf is a Pmf object that approximates the Pdf by evaluating\n", + "the density at a sequence of equally spaced values.\n", + "\n", + "linspace stands for “linear space.” It takes a range,\n", + "low and high, and the number of points,\n", + "n, and returns a new numpy array with\n", + "n elements equally spaced between low and\n", + "high, including both.\n", + "\n", + "And now back to *The Price is Right*.\n", + "\n", + "Modeling the contestants\n", + "------------------------\n", + "\n", + "![Cumulative distribution (CDF) of the difference between the\n", + "contestant’s bid and the actual price.](figs/price2.pdf)\n", + "\n", + "[fig.price2]\n", + "\n", + "The PDFs in Figure [fig.price1] estimate the distribution of possible\n", + "prices. If you were a contestant on the show, you could use this\n", + "distribution to quantify your prior belief about the price of each\n", + "showcase (before you see the prizes).\n", + "\n", + "To update these priors, we have to answer these questions:\n", + "\n", + "1. What data should we consider and how should we quantify it?\n", + "\n", + "2. Can we compute a likelihood function; that is, for each hypothetical\n", + " value of price, can we compute the conditional\n", + " likelihood of the data?\n", + "\n", + "To answer these questions, I am going to model the contestant as a\n", + "price-guessing instrument with known error characteristics. In other\n", + "words, when the contestant sees the prizes, he or she guesses the price\n", + "of each prize—ideally without taking into consideration the fact that\n", + "the prize is part of a showcase—and adds up the prices. Let’s call this\n", + "total guess.\n", + "\n", + "Under this model, the question we have to answer is, “If the actual\n", + "price is price, what is the likelihood that the\n", + "contestant’s estimate would be guess?”\n", + "\n", + "Or if we define\n", + "\n", + " error = price - guess\n", + "\n", + "then we could ask, “What is the likelihood that the contestant’s\n", + "estimate is off by error?”\n", + "\n", + "To answer this question, we can use the historical data again.\n", + "Figure [fig.price2] shows the cumulative distribution of\n", + "diff, the difference between the contestant’s bid and the\n", + "actual price of the showcase.\n", + "\n", + "The definition of diff is\n", + "\n", + " diff = price - bid\n", + "\n", + "When diff is negative, the bid is too high. As an aside, we\n", + "can use this distribution to compute the probability that the\n", + "contestants overbid: the first contestant overbids 25% of the time; the\n", + "second contestant overbids 29% of the time.\n", + "\n", + "We can also see that the bids are biased; that is, they are more likely\n", + "to be too low than too high. And that makes sense, given the rules of\n", + "the game.\n", + "\n", + "Finally, we can use this distribution to estimate the reliability of the\n", + "contestants’ guesses. This step is a little tricky because we don’t\n", + "actually know the contestant’s guesses; we only know what they bid.\n", + "\n", + "So we’ll have to make some assumptions. Specifically, I assume that the\n", + "distribution of error is Gaussian with mean 0 and the same\n", + "variance as diff.\n", + "\n", + "The Player class implements this model:\n", + "\n", + " class Player(object):\n", + "\n", + " def __init__(self, prices, bids, diffs):\n", + " self.pdf_price = thinkbayes.EstimatedPdf(prices)\n", + " self.cdf_diff = thinkbayes.MakeCdfFromList(diffs)\n", + "\n", + " mu = 0\n", + " sigma = numpy.std(diffs)\n", + " self.pdf_error = thinkbayes.GaussianPdf(mu, sigma)\n", + "\n", + "prices is a sequence of showcase prices, bids\n", + "is a sequence of bids, and diffs is a sequence of diffs,\n", + "where again diff = price - bid.\n", + "\n", + "`pdf_price` is the smoothed PDF of prices, estimated by KDE. `cdf_diff`\n", + "is the cumulative distribution of diff, which we saw in\n", + "Figure [fig.price2]. And `pdf_error` is the PDF that characterizes the\n", + "distribution of errors; where error = price - guess.\n", + "\n", + "Again, we use the variance of diff to estimate the variance\n", + "of error. This estimate is not perfect because contestants’\n", + "bids are sometimes strategic; for example, if Player 2 thinks that\n", + "Player 1 has overbid, Player 2 might make a very low bid. In that case\n", + "diff does not reflect error. If this happens a\n", + "lot, the observed variance in diff might overestimate the\n", + "variance in error. Nevertheless, I think it is a reasonable\n", + "modeling decision.\n", + "\n", + "As an alternative, someone preparing to appear on the show could\n", + "estimate their own distribution of error by watching\n", + "previous shows and recording their guesses and the actual prices.\n", + "\n", + "Likelihood\n", + "----------\n", + "\n", + "Now we are ready to write the likelihood function. As usual, I define a\n", + "new class that extends thinkbayes.Suite:\n", + "\n", + " class Price(thinkbayes.Suite):\n", + "\n", + " def __init__(self, pmf, player):\n", + " thinkbayes.Suite.__init__(self, pmf)\n", + " self.player = player\n", + "\n", + "pmf represents the prior distribution and\n", + "player is a Player object as described in the previous\n", + "section. Here’s Likelihood:\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " price = hypo\n", + " guess = data\n", + "\n", + " error = price - guess\n", + " like = self.player.ErrorDensity(error)\n", + "\n", + " return like\n", + "\n", + "hypo is the hypothetical price of the showcase.\n", + "data is the contestant’s best guess at the price.\n", + "error is the difference, and like is the\n", + "likelihood of the data, given the hypothesis.\n", + "\n", + "ErrorDensity is defined in Player:\n", + "\n", + " # class Player:\n", + "\n", + " def ErrorDensity(self, error):\n", + " return self.pdf_error.Density(error)\n", + "\n", + "ErrorDensity works by evaluating `pdf_error` at the given\n", + "value of error. The result is a probability density, so it\n", + "is not really a probability. But remember that Likelihood\n", + "doesn’t need to compute a probability; it only has to compute something\n", + "*proportional* to a probability. As long as the constant of\n", + "proportionality is the same for all likelihoods, it gets canceled out\n", + "when we normalize the posterior distribution.\n", + "\n", + "And therefore, a probability density is a perfectly good likelihood.\n", + "\n", + "Update\n", + "------\n", + "\n", + "![Prior and posterior distributions for Player 1, based on a best guess\n", + "of \\$20,000.](figs/price3.pdf)\n", + "\n", + "[fig.price3]\n", + "\n", + "Player provides a method that takes the contestant’s guess\n", + "and computes the posterior distribution:\n", + "\n", + " # class Player\n", + "\n", + " def MakeBeliefs(self, guess):\n", + " pmf = self.PmfPrice()\n", + " self.prior = Price(pmf, self)\n", + " self.posterior = self.prior.Copy()\n", + " self.posterior.Update(guess)\n", + "\n", + "PmfPrice generates a discrete approximation to the PDF of\n", + "price, which we use to construct the prior.\n", + "\n", + "PmfPrice uses MakePmf, which evaluates\n", + "`pdf_price` at a sequence of values:\n", + "\n", + " # class Player\n", + "\n", + " n = 101\n", + " price_xs = numpy.linspace(0, 75000, n)\n", + "\n", + " def PmfPrice(self):\n", + " return self.pdf_price.MakePmf(self.price_xs)\n", + "\n", + "To construct the posterior, we make a copy of the prior and then invoke\n", + "Update, which invokes Likelihood for each\n", + "hypothesis, multiplies the priors by the likelihoods, and renormalizes.\n", + "\n", + "So let’s get back to the original scenario. Suppose you are Player 1 and\n", + "when you see your showcase, your best guess is that the total price of\n", + "the prizes is \\$20,000.\n", + "\n", + "Figure [fig.price3] shows prior and posterior beliefs about the actual\n", + "price. The posterior is shifted to the left because your guess is on the\n", + "low end of the prior range.\n", + "\n", + "On one level, this result makes sense. The most likely value in the\n", + "prior is \\$27,750, your best guess is \\$20,000, and the mean of the\n", + "posterior is somewhere in between: \\$25,096.\n", + "\n", + "On another level, you might find this result bizarre, because it\n", + "suggests that if you *think* the price is \\$20,000, then\n", + "you should *believe* the price is \\$24,000.\n", + "\n", + "To resolve this apparent paradox, remember that you are combining two\n", + "sources of information, historical data about past showcases and guesses\n", + "about the prizes you see.\n", + "\n", + "We are treating the historical data as the prior and updating it based\n", + "on your guesses, but we could equivalently use your guess as a prior and\n", + "update it based on historical data.\n", + "\n", + "If you think of it that way, maybe it is less surprising that the most\n", + "likely value in the posterior is not your original guess.\n", + "\n", + "Optimal bidding\n", + "---------------\n", + "\n", + "Now that we have a posterior distribution, we can use it to compute the\n", + "optimal bid, which I define as the bid that maximizes expected return\n", + "(see ).\n", + "\n", + "I’m going to present the methods in this section top-down, which means I\n", + "will show you how they are used before I show you how they work. If you\n", + "see an unfamiliar method, don’t worry; the definition will be along\n", + "shortly.\n", + "\n", + "To compute optimal bids, I wrote a class called\n", + "GainCalculator:\n", + "\n", + " class GainCalculator(object):\n", + "\n", + " def __init__(self, player, opponent):\n", + " self.player = player\n", + " self.opponent = opponent\n", + "\n", + "player and opponent are Player\n", + "objects.\n", + "\n", + "GainCalculator provides ExpectedGains, which\n", + "computes a sequence of bids and the expected gain for each bid:\n", + "\n", + " def ExpectedGains(self, low=0, high=75000, n=101):\n", + " bids = numpy.linspace(low, high, n)\n", + "\n", + " gains = [self.ExpectedGain(bid) for bid in bids]\n", + "\n", + " return bids, gains\n", + "\n", + "low and high specify the range of possible\n", + "bids; n is the number of bids to try.\n", + "\n", + "ExpectedGains calls ExpectedGain, which\n", + "computes expected gain for a given bid:\n", + "\n", + " def ExpectedGain(self, bid):\n", + " suite = self.player.posterior\n", + " total = 0\n", + " for price, prob in sorted(suite.Items()):\n", + " gain = self.Gain(bid, price)\n", + " total += prob * gain\n", + " return total\n", + "\n", + "ExpectedGain loops through the values in the posterior and\n", + "computes the gain for each bid, given the actual prices of the showcase.\n", + "It weights each gain with the corresponding probability and returns the\n", + "total.\n", + "\n", + "![Expected gain versus bid in a scenario where Player 1’s best guess is\n", + "\\$20,000 and Player 2’s best guess is \\$40,000.](figs/price5.pdf)\n", + "\n", + "[fig.price5]\n", + "\n", + "ExpectedGain invokes Gain, which takes a bid\n", + "and an actual price and returns the expected gain:\n", + "\n", + " def Gain(self, bid, price):\n", + " if bid > price:\n", + " return 0\n", + "\n", + " diff = price - bid\n", + " prob = self.ProbWin(diff)\n", + "\n", + " if diff <= 250:\n", + " return 2 * price * prob\n", + " else:\n", + " return price * prob\n", + "\n", + "If you overbid, you get nothing. Otherwise we compute the difference\n", + "between your bid and the price, which determines your probability of\n", + "winning.\n", + "\n", + "If diff is less than \\$250, you win both showcases. For\n", + "simplicity, I assume that both showcases have the same price. Since this\n", + "outcome is rare, it doesn’t make much difference.\n", + "\n", + "Finally, we have to compute the probability of winning based on\n", + "diff:\n", + "\n", + " def ProbWin(self, diff):\n", + " prob = (self.opponent.ProbOverbid() + \n", + " self.opponent.ProbWorseThan(diff))\n", + " return prob\n", + "\n", + "If your opponent overbids, you win. Otherwise, you have to hope that\n", + "your opponent is off by more than diff. Player\n", + "provides methods to compute both probabilities:\n", + "\n", + " # class Player:\n", + "\n", + " def ProbOverbid(self):\n", + " return self.cdf_diff.Prob(-1)\n", + "\n", + " def ProbWorseThan(self, diff):\n", + " return 1 - self.cdf_diff.Prob(diff)\n", + "\n", + "This code might be confusing because the computation is now from the\n", + "point of view of the opponent, who is computing, “What is the\n", + "probability that I overbid?” and “What is the probability that my bid is\n", + "off by more than diff?”\n", + "\n", + "Both answers are based on the CDF of diff. If the\n", + "opponent’s diff is less than or equal to -1, you win. If\n", + "the opponent’s diff is worse than yours, you win. Otherwise\n", + "you lose.\n", + "\n", + "Finally, here’s the code that computes optimal bids:\n", + "\n", + " # class Player:\n", + "\n", + " def OptimalBid(self, guess, opponent):\n", + " self.MakeBeliefs(guess)\n", + " calc = GainCalculator(self, opponent)\n", + " bids, gains = calc.ExpectedGains()\n", + " gain, bid = max(zip(gains, bids))\n", + " return bid, gain\n", + "\n", + "Given a guess and an opponent, OptimalBid computes the\n", + "posterior distribution, instantiates a GainCalculator,\n", + "computes expected gains for a range of bids and returns the optimal bid\n", + "and expected gain. Whew!\n", + "\n", + "Figure [fig.price5] shows the results for both players, based on a\n", + "scenario where Player 1’s best guess is \\$20,000 and Player 2’s best\n", + "guess is \\$40,000.\n", + "\n", + "For Player 1 the optimal bid is \\$21,000, yielding an expected return of\n", + "almost \\$16,700. This is a case (which turns out to be unusual) where\n", + "the optimal bid is actually higher than the contestant’s best guess.\n", + "\n", + "For Player 2 the optimal bid is \\$31,500, yielding an expected return of\n", + "almost \\$19,400. This is the more typical case where the optimal bid is\n", + "less than the best guess.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "One of the features of Bayesian estimation is that the result comes in\n", + "the form of a posterior distribution. Classical estimation usually\n", + "generates a single point estimate or a confidence interval, which is\n", + "sufficient if estimation is the last step in the process, but if you\n", + "want to use an estimate as an input to a subsequent analysis, point\n", + "estimates and intervals are often not much help.\n", + "\n", + "In this example, we use the posterior distribution to compute an optimal\n", + "bid. The return on a given bid is asymmetric and discontinuous (if you\n", + "overbid, you lose), so it would be hard to solve this problem\n", + "analytically. But it is relatively simple to do computationally.\n", + "\n", + "Newcomers to Bayesian thinking are often tempted to summarize the\n", + "posterior distribution by computing the mean or the maximum likelihood\n", + "estimate. These summaries can be useful, but if that’s all you need,\n", + "then you probably don’t need Bayesian methods in the first place.\n", + "\n", + "Bayesian methods are most useful when you can carry the posterior\n", + "distribution into the next step of the analysis to perform some kind of\n", + "decision analysis, as we did in this chapter, or some kind of\n", + "prediction, as we see in the next chapter.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/07_Prediction.ipynb b/notebooks/07_Prediction.ipynb new file mode 100644 index 0000000..a9bd746 --- /dev/null +++ b/notebooks/07_Prediction.ipynb @@ -0,0 +1,472 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Prediction\n", + "==========\n", + "\n", + "The Boston Bruins problem\n", + "-------------------------\n", + "\n", + "In the 2010-11 National Hockey League (NHL) Finals, my beloved Boston\n", + "Bruins played a best-of-seven championship series against the despised\n", + "Vancouver Canucks. Boston lost the first two games 0-1 and 2-3, then won\n", + "the next two games 8-1 and 4-0. At this point in the series, what is the\n", + "probability that Boston will win the next game, and what is their\n", + "probability of winning the championship?\n", + "\n", + "As always, to answer a question like this, we need to make some\n", + "assumptions. First, it is reasonable to believe that goal scoring in\n", + "hockey is at least approximately a Poisson process, which means that it\n", + "is equally likely for a goal to be scored at any time during a game.\n", + "Second, we can assume that against a particular opponent, each team has\n", + "some long-term average goals per game, denoted $\\lambda$.\n", + "\n", + "Given these assumptions, my strategy for answering this question is\n", + "\n", + "1. Use statistics from previous games to choose a prior distribution\n", + " for $\\lambda$.\n", + "\n", + "2. Use the score from the first four games to estimate $\\lambda$ for\n", + " each team.\n", + "\n", + "3. Use the posterior distributions of $\\lambda$ to compute distribution\n", + " of goals for each team, the distribution of the goal differential,\n", + " and the probability that each team wins the next game.\n", + "\n", + "4. Compute the probability that each team wins the series.\n", + "\n", + "To choose a prior distribution, I got some statistics from\n", + ", specifically the average goals per game for each\n", + "team in the 2010-11 season. The distribution is roughly Gaussian with\n", + "mean 2.8 and standard deviation 0.3.\n", + "\n", + "The Gaussian distribution is continuous, but we’ll approximate it with a\n", + "discrete Pmf. `thinkbayes` provides `MakeGaussianPmf` to do exactly\n", + "that:\n", + "\n", + " def MakeGaussianPmf(mu, sigma, num_sigmas, n=101):\n", + " pmf = Pmf()\n", + " low = mu - num_sigmas*sigma\n", + " high = mu + num_sigmas*sigma\n", + "\n", + " for x in numpy.linspace(low, high, n):\n", + " p = scipy.stats.norm.pdf(mu, sigma, x)\n", + " pmf.Set(x, p)\n", + " pmf.Normalize()\n", + " return pmf\n", + "\n", + "mu and sigma are the mean and standard\n", + "deviation of the Gaussian distribution. `num_sigmas` is the number of\n", + "standard deviations above and below the mean that the Pmf will span, and\n", + "n is the number of values in the Pmf.\n", + "\n", + "Again we use numpy.linspace to make an array of\n", + "n equally spaced values between low and\n", + "high, including both.\n", + "\n", + "`norm.pdf` evaluates the Gaussian probability density function (PDF).\n", + "\n", + "Getting back to the hockey problem, here’s the definition for a suite of\n", + "hypotheses about the value of $\\lambda$.\n", + "\n", + " class Hockey(thinkbayes.Suite):\n", + "\n", + " def __init__(self):\n", + " pmf = thinkbayes.MakeGaussianPmf(2.7, 0.3, 4)\n", + " thinkbayes.Suite.__init__(self, pmf)\n", + "\n", + "So the prior distribution is Gaussian with mean 2.7, standard deviation\n", + "0.3, and it spans 4 sigmas above and below the mean.\n", + "\n", + "As always, we have to decide how to represent each hypothesis; in this\n", + "case I represent the hypothesis that $\\lambda=x$ with the floating-point\n", + "value x.\n", + "\n", + "Poisson processes\n", + "-----------------\n", + "\n", + "In mathematical statistics, a **process** is a stochastic\n", + "model of a physical system (“stochastic” means that the model has some\n", + "kind of randomness in it). For example, a Bernoulli process is a model\n", + "of a sequence of events, called trials, in which each trial has two\n", + "possible outcomes, like success and failure. So a Bernoulli process is a\n", + "natural model for a series of coin flips, or a series of shots on goal.\n", + "\n", + "A Poisson process is the continuous version of a Bernoulli process,\n", + "where an event can occur at any point in time with equal probability.\n", + "Poisson processes can be used to model customers arriving in a store,\n", + "buses arriving at a bus stop, or goals scored in a hockey game.\n", + "\n", + "In many real systems the probability of an event changes over time.\n", + "Customers are more likely to go to a store at certain times of day,\n", + "buses are supposed to arrive at fixed intervals, and goals are more or\n", + "less likely at different times during a game.\n", + "\n", + "But all models are based on simplifications, and in this case modeling a\n", + "hockey game with a Poisson process is a reasonable choice. Heuer, Müller\n", + "and Rubner (2010) analyze scoring in a German soccer league and come to\n", + "the same conclusion; see\n", + ".\n", + "\n", + "The benefit of using this model is that we can compute the distribution\n", + "of goals per game efficiently, as well as the distribution of time\n", + "between goals. Specifically, if the average number of goals in a game is\n", + "lam, the distribution of goals per game is given by the\n", + "Poisson PMF:\n", + "\n", + " def EvalPoissonPmf(k, lam):\n", + " return (lam)**k * math.exp(-lam) / math.factorial(k)\n", + "\n", + "And the distribution of time between goals is given by the exponential\n", + "PDF:\n", + "\n", + " def EvalExponentialPdf(x, lam):\n", + " return lam * math.exp(-lam * x)\n", + "\n", + "I use the variable lam because lambda is a\n", + "reserved keyword in Python. Both of these functions are in\n", + "`thinkbayes.py`.\n", + "\n", + "The posteriors\n", + "--------------\n", + "\n", + "![Posterior distribution of the number of goals per\n", + "game.](figs/hockey1.pdf)\n", + "\n", + "[fig.hockey1]\n", + "\n", + "Now we can compute the likelihood that a team with a hypothetical value\n", + "of lam scores k goals in a game:\n", + "\n", + " # class Hockey\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " lam = hypo\n", + " k = data\n", + " like = thinkbayes.EvalPoissonPmf(k, lam)\n", + " return like\n", + "\n", + "Each hypothesis is a possible value of $\\lambda$; data is\n", + "the observed number of goals, k.\n", + "\n", + "With the likelihood function in place, we can make a suite for each team\n", + "and update them with the scores from the first four games.\n", + "\n", + " suite1 = Hockey('bruins')\n", + " suite1.UpdateSet([0, 2, 8, 4])\n", + " \n", + " suite2 = Hockey('canucks')\n", + " suite2.UpdateSet([1, 3, 1, 0])\n", + "\n", + "Figure [fig.hockey1] shows the resulting posterior distributions for\n", + "lam. Based on the first four games, the most likely values\n", + "for lam are 2.6 for the Canucks and 2.9 for the Bruins.\n", + "\n", + "The distribution of goals\n", + "-------------------------\n", + "\n", + "![Distribution of goals in a single game.](figs/hockey2.pdf)\n", + "\n", + "[fig.hockey2]\n", + "\n", + "To compute the probability that each team wins the next game, we need to\n", + "compute the distribution of goals for each team.\n", + "\n", + "If we knew the value of lam exactly, we could use the\n", + "Poisson distribution again. `thinkbayes` provides a method that computes\n", + "a truncated approximation of a Poisson distribution:\n", + "\n", + " def MakePoissonPmf(lam, high):\n", + " pmf = Pmf()\n", + " for k in xrange(0, high+1):\n", + " p = EvalPoissonPmf(k, lam)\n", + " pmf.Set(k, p)\n", + " pmf.Normalize()\n", + " return pmf\n", + "\n", + "The range of values in the computed Pmf is from 0 to high.\n", + "So if the value of lam were exactly 3.4, we would compute:\n", + "\n", + " lam = 3.4\n", + " goal_dist = thinkbayes.MakePoissonPmf(lam, 10)\n", + "\n", + "I chose the upper bound, 10, because the probability of scoring more\n", + "than 10 goals in a game is quite low.\n", + "\n", + "That’s simple enough so far; the problem is that we don’t know the value\n", + "of lam exactly. Instead, we have a distribution of possible\n", + "values for lam.\n", + "\n", + "For each value of lam, the distribution of goals is\n", + "Poisson. So the overall distribution of goals is a mixture of these\n", + "Poisson distributions, weighted according to the probabilities in the\n", + "distribution of lam.\n", + "\n", + "Given the posterior distribution of lam, here’s the code\n", + "that makes the distribution of goals:\n", + "\n", + " def MakeGoalPmf(suite):\n", + " metapmf = thinkbayes.Pmf()\n", + "\n", + " for lam, prob in suite.Items():\n", + " pmf = thinkbayes.MakePoissonPmf(lam, 10)\n", + " metapmf.Set(pmf, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(metapmf)\n", + " return mix\n", + "\n", + "For each value of lam we make a Poisson Pmf and add it to\n", + "the meta-Pmf. I call it a meta-Pmf because it is a Pmf that contains\n", + "Pmfs as its values.\n", + "\n", + "Then we use `MakeMixture` to compute the mixture (we saw\n", + "MakeMixture in Section [mixture]).\n", + "\n", + "Figure [fig.hockey2] shows the resulting distribution of goals for the\n", + "Bruins and Canucks. The Bruins are less likely to score 3 goals or fewer\n", + "in the next game, and more likely to score 4 or more.\n", + "\n", + "The probability of winning\n", + "--------------------------\n", + "\n", + "![Distribution of time between goals.](figs/hockey3.pdf)\n", + "\n", + "[fig.hockey3]\n", + "\n", + "To get the probability of winning, first we compute the distribution of\n", + "the goal differential:\n", + "\n", + " goal_dist1 = MakeGoalPmf(suite1)\n", + " goal_dist2 = MakeGoalPmf(suite2)\n", + " diff = goal_dist1 - goal_dist2\n", + "\n", + "The subtraction operator invokes `Pmf.__sub__`, which enumerates pairs\n", + "of values and computes the difference. Subtracting two distributions is\n", + "almost the same as adding, which we saw in Section [addends].\n", + "\n", + "If the goal differential is positive, the Bruins win; if negative, the\n", + "Canucks win; if 0, it’s a tie:\n", + "\n", + " p_win = diff.ProbGreater(0)\n", + " p_loss = diff.ProbLess(0)\n", + " p_tie = diff.Prob(0)\n", + "\n", + "With the distributions from the previous section, `p_win` is 46%,\n", + "`p_loss` is 37%, and `p_tie` is 17%.\n", + "\n", + "In the event of a tie at the end of “regulation play,” the teams play\n", + "overtime periods until one team scores. Since the game ends immediately\n", + "when the first goal is scored, this overtime format is known as “sudden\n", + "death.”\n", + "\n", + "Sudden death\n", + "------------\n", + "\n", + "To compute the probability of winning in a sudden death overtime, the\n", + "important statistic is not goals per game, but time until the first\n", + "goal. The assumption that goal-scoring is a Poisson process implies that\n", + "the time between goals is exponentially distributed.\n", + "\n", + "Given lam, we can compute the time between goals like this:\n", + "\n", + " lam = 3.4\n", + " time_dist = thinkbayes.MakeExponentialPmf(lam, high=2, n=101)\n", + "\n", + "high is the upper bound of the distribution. In this case I\n", + "chose 2, because the probability of going more than two games without\n", + "scoring is small. n is the number of values in the Pmf.\n", + "\n", + "If we know lam exactly, that’s all there is to it. But we\n", + "don’t; instead we have a posterior distribution of possible values. So\n", + "as we did with the distribution of goals, we make a meta-Pmf and compute\n", + "a mixture of Pmfs.\n", + "\n", + " def MakeGoalTimePmf(suite):\n", + " metapmf = thinkbayes.Pmf()\n", + "\n", + " for lam, prob in suite.Items():\n", + " pmf = thinkbayes.MakeExponentialPmf(lam, high=2, n=2001)\n", + " metapmf.Set(pmf, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(metapmf)\n", + " return mix\n", + "\n", + "Figure [fig.hockey3] shows the resulting distributions. For time values\n", + "less than one period (one third of a game), the Bruins are more likely\n", + "to score. The time until the Canucks score is more likely to be longer.\n", + "\n", + "I set the number of values, n, fairly high in order to\n", + "minimize the number of ties, since it is not possible for both teams to\n", + "score simultaneously.\n", + "\n", + "Now we compute the probability that the Bruins score first:\n", + "\n", + " time_dist1 = MakeGoalTimePmf(suite1)\n", + " time_dist2 = MakeGoalTimePmf(suite2)\n", + " p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", + "\n", + "For the Bruins, the probability of winning in overtime is 52%.\n", + "\n", + "Finally, the total probability of winning is the chance of winning at\n", + "the end of regulation play plus the probability of winning in overtime.\n", + "\n", + " p_tie = diff.Prob(0)\n", + " p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", + "\n", + " p_win = diff.ProbGreater(0) + p_tie * p_overtime\n", + "\n", + "For the Bruins, the overall chance of winning the next game is 55%.\n", + "\n", + "To win the series, the Bruins can either win the next two games or split\n", + "the next two and win the third. Again, we can compute the total\n", + "probability:\n", + "\n", + " # win the next two\n", + " p_series = p_win**2\n", + "\n", + " # split the next two, win the third\n", + " p_series += 2 * p_win * (1-p_win) * p_win\n", + "\n", + "The Bruins chance of winning the series is 57%. And in 2011, they did.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "As always, the analysis in this chapter is based on modeling decisions,\n", + "and modeling is almost always an iterative process. In general, you want\n", + "to start with something simple that yields an approximate answer,\n", + "identify likely sources of error, and look for opportunities for\n", + "improvement.\n", + "\n", + "In this example, I would consider these options:\n", + "\n", + "- I chose a prior based on the average goals per game for each team.\n", + " But this statistic is averaged across all opponents. Against a\n", + " particular opponent, we might expect more variability. For example,\n", + " if the team with the best offense plays the team with the worst\n", + " defense, the expected goals per game might be several standard\n", + " deviations above the mean.\n", + "\n", + "- For data I used only the first four games of the championship\n", + " series. If the same teams played each other during the regular\n", + " season, I could use the results from those games as well. One\n", + " complication is that the composition of teams changes during the\n", + " season due to trades and injuries. So it might be best to give more\n", + " weight to recent games.\n", + "\n", + "- To take advantage of all available information, we could use results\n", + " from all regular season games to estimate each team’s goal scoring\n", + " rate, possibly adjusted by estimating an additional factor for each\n", + " pairwise match-up. This approach would be more complicated, but it\n", + " is still feasible.\n", + "\n", + "For the first option, we could use the results from the regular season\n", + "to estimate the variability across all pairwise match-ups. Thanks to\n", + "Dirk Hoag at , I was able to get the\n", + "number of goals scored during regulation play (not overtime) for each\n", + "game in the regular season.\n", + "\n", + "Teams in different conferences only play each other one or two times in\n", + "the regular season, so I focused on pairs that played each other 4–6\n", + "times. For each pair, I computed the average goals per game, which is an\n", + "estimate of $\\lambda$, then plotted the distribution of these estimates.\n", + "\n", + "The mean of these estimates is 2.8, again, but the standard deviation is\n", + "0.85, substantially higher than what we got computing one estimate for\n", + "each team.\n", + "\n", + "If we run the analysis again with the higher-variance prior, the\n", + "probability that the Bruins win the series is 80%, substantially higher\n", + "than the result with the low-variance prior, 57%.\n", + "\n", + "So it turns out that the results are sensitive to the prior, which makes\n", + "sense considering how little data we have to work with. Based on the\n", + "difference between the low-variance model and the high-variable model,\n", + "it seems worthwhile to put some effort into getting the prior right.\n", + "\n", + "The code and data for this chapter are available from\n", + " and\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "If buses arrive at a bus stop every 20 minutes, and you arrive at the\n", + "bus stop at a random time, your wait time until the bus arrives is\n", + "uniformly distributed from 0 to 20 minutes.\n", + "\n", + "But in reality, there is variability in the time between buses. Suppose\n", + "you are waiting for a bus, and you know the historical distribution of\n", + "time between buses. Compute your distribution of wait times.\n", + "\n", + "Hint: Suppose that the time between buses is either 5 or 10 minutes with\n", + "equal probability. What is the probability that you arrive during one of\n", + "the 10 minute intervals?\n", + "\n", + "I solve a version of this problem in the next chapter.\n", + "\n", + "Suppose that passengers arriving at the bus stop are well-modeled by a\n", + "Poisson process with parameter $\\lambda$. If you arrive at the stop and\n", + "find 3 people waiting, what is your posterior distribution for the time\n", + "since the last bus arrived.\n", + "\n", + "I solve a version of this problem in the next chapter.\n", + "\n", + "Suppose that you are an ecologist sampling the insect population in a\n", + "new environment. You deploy 100 traps in a test area and come back the\n", + "next day to check on them. You find that 37 traps have been triggered,\n", + "trapping an insect inside. Once a trap triggers, it cannot trap another\n", + "insect until it has been reset.\n", + "\n", + "If you reset the traps and come back in two days, how many traps do you\n", + "expect to find triggered? Compute a posterior predictive distribution\n", + "for the number of traps.\n", + "\n", + "Suppose you are the manager of an apartment building with 100 light\n", + "bulbs in common areas. It is your responsibility to replace light bulbs\n", + "when they break.\n", + "\n", + "On January 1, all 100 bulbs are working. When you inspect them on\n", + "February 1, you find 3 light bulbs out. If you come back on April 1, how\n", + "many light bulbs do you expect to find broken?\n", + "\n", + "In the previous exercise, you could reasonably assume that an event is\n", + "equally likely at any time. For light bulbs, the likelihood of failure\n", + "depends on the age of the bulb. Specifically, old bulbs have an\n", + "increasing failure rate due to evaporation of the filament.\n", + "\n", + "This problem is more open-ended than some; you will have to make\n", + "modeling decisions. You might want to read about the Weibull\n", + "distribution (). Or\n", + "you might want to look around for information about light bulb survival\n", + "curves.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/08_Observer_Bias.ipynb b/notebooks/08_Observer_Bias.ipynb new file mode 100644 index 0000000..84c805a --- /dev/null +++ b/notebooks/08_Observer_Bias.ipynb @@ -0,0 +1,695 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Observer Bias {#observer}\n", + "=============\n", + "\n", + "The Red Line problem\n", + "--------------------\n", + "\n", + "In Massachusetts, the Red Line is a subway that connects Cambridge and\n", + "Boston. When I was working in Cambridge I took the Red Line from Kendall\n", + "Square to South Station and caught the commuter rail to Needham. During\n", + "rush hour Red Line trains run every 7–8 minutes, on average.\n", + "\n", + "When I arrived at the station, I could estimate the time until the next\n", + "train based on the number of passengers on the platform. If there were\n", + "only a few people, I inferred that I just missed a train and expected to\n", + "wait about 7 minutes. If there were more passengers, I expected the\n", + "train to arrive sooner. But if there were a large number of passengers,\n", + "I suspected that trains were not running on schedule, so I would go back\n", + "to the street level and get a taxi.\n", + "\n", + "While I was waiting for trains, I thought about how Bayesian estimation\n", + "could help predict my wait time and decide when I should give up and\n", + "take a taxi. This chapter presents the analysis I came up with.\n", + "\n", + "This chapter is based on a project by Brendan Ritter and Kai Austin, who\n", + "took a class with me at Olin College. The code in this chapter is\n", + "available from . The code I used to\n", + "collect data is in . For more\n", + "information see Section [download].\n", + "\n", + "The model\n", + "---------\n", + "\n", + "![PMF of gaps between trains, based on collected data, smoothed by KDE.\n", + "z is the actual distribution; zb is the biased\n", + "distribution seen by passengers. ](figs/redline0.pdf)\n", + "\n", + "[fig.redline0]\n", + "\n", + "Before we get to the analysis, we have to make some modeling decisions.\n", + "First, I will treat passenger arrivals as a Poisson process, which means\n", + "I assume that passengers are equally likely to arrive at any time, and\n", + "that they arrive at an unknown rate, $\\lambda$, measured in passengers\n", + "per minute. Since I observe passengers during a short period of time,\n", + "and at the same time every day, I assume that $\\lambda$ is constant.\n", + "\n", + "On the other hand, the arrival process for trains is not Poisson. Trains\n", + "to Boston are supposed to leave from the end of the line (Alewife\n", + "station) every 7–8 minutes during peak times, but by the time they get\n", + "to Kendall Square, the time between trains varies between 3 and 12\n", + "minutes.\n", + "\n", + "To gather data on the time between trains, I wrote a script that\n", + "downloads real-time data from\n", + ", selects south-bound\n", + "trains arriving at Kendall square, and records their arrival times in a\n", + "database. I ran the script from 4pm to 6pm every weekday for 5 days, and\n", + "recorded about 15 arrivals per day. Then I computed the time between\n", + "consecutive arrivals; the distribution of these gaps is shown in\n", + "Figure [fig.redline0], labeled z.\n", + "\n", + "If you stood on the platform from 4pm to 6pm and recorded the time\n", + "between trains, this is the distribution you would see. But if you\n", + "arrive at some random time (without regard to the train schedule) you\n", + "would see a different distribution. The average time between trains, as\n", + "seen by a random passenger, is substantially higher than the true\n", + "average.\n", + "\n", + "Why? Because a passenger is more like to arrive during a large interval\n", + "than a small one. Consider a simple example: suppose that the time\n", + "between trains is either 5 minutes or 10 minutes with equal probability.\n", + "In that case the average time between trains is 7.5 minutes.\n", + "\n", + "But a passenger is more likely to arrive during a 10 minute gap than a 5\n", + "minute gap; in fact, twice as likely. If we surveyed arriving\n", + "passengers, we would find that 2/3 of them arrived during a 10 minute\n", + "gap, and only 1/3 during a 5 minute gap. So the average time between\n", + "trains, as seen by an arriving passenger, is 8.33 minutes.\n", + "\n", + "This kind of **observer bias** appears in many contexts.\n", + "Students think that classes are bigger than they are because more of\n", + "them are in the big classes. Airline passengers think that planes are\n", + "fuller than they are because more of them are on full flights.\n", + "\n", + "In each case, values from the actual distribution are oversampled in\n", + "proportion to their value. In the Red Line example, a gap that is twice\n", + "as big is twice as likely to be observed.\n", + "\n", + "So given the actual distribution of gaps, we can compute the\n", + "distribution of gaps as seen by passengers. BiasPmf does\n", + "this computation:\n", + "\n", + " def BiasPmf(pmf):\n", + " new_pmf = pmf.Copy()\n", + "\n", + " for x, p in pmf.Items():\n", + " new_pmf.Mult(x, x)\n", + " \n", + " new_pmf.Normalize()\n", + " return new_pmf\n", + "\n", + "pmf is the actual distribution; `new_pmf` is the biased\n", + "distribution. Inside the loop, we multiply the probability of each\n", + "value, x, by the likelihood it will be observed, which is\n", + "proportional to x. Then we normalize the result.\n", + "\n", + "Figure [fig.redline0] shows the actual distribution of gaps, labeled\n", + "z, and the distribution of gaps seen by passengers, labeled\n", + "zb for “z biased”.\n", + "\n", + "Wait times\n", + "----------\n", + "\n", + "![CDF of z, zb, and the wait time seen by\n", + "passengers, y. ](figs/redline2.pdf)\n", + "\n", + "[fig.redline2]\n", + "\n", + "Wait time, which I call y, is the time between the arrival\n", + "of a passenger and the next arrival of a train. Elapsed time, which I\n", + "call x, is the time between the arrival of the previous\n", + "train and the arrival of a passenger. I chose these definitions so that\n", + "zb = x + y.\n", + "\n", + "Given the distribution of zb, we can compute the\n", + "distribution of y. I’ll start with a simple case and then\n", + "generalize. Suppose, as in the previous example, that zb is\n", + "either 5 minutes with probability 1/3, or 10 minutes with probability\n", + "2/3.\n", + "\n", + "If we arrive at a random time during a 5 minute gap, y is\n", + "uniform from 0 to 5 minutes. If we arrive during a 10 minute gap,\n", + "y is uniform from 0 to 10. So the overall distribution is a\n", + "mixture of uniform distributions weighted according to the probability\n", + "of each gap.\n", + "\n", + "The following function takes the distribution of zb and\n", + "computes the distribution of y:\n", + "\n", + " def PmfOfWaitTime(pmf_zb):\n", + " metapmf = thinkbayes.Pmf()\n", + " for gap, prob in pmf_zb.Items():\n", + " uniform = MakeUniformPmf(0, gap)\n", + " metapmf.Set(uniform, prob)\n", + "\n", + " pmf_y = thinkbayes.MakeMixture(metapmf)\n", + " return pmf_y\n", + "\n", + "PmfOfWaitTime makes a meta-Pmf that maps from each uniform\n", + "distribution to its probability. Then it uses MakeMixture,\n", + "which we saw in Section [mixture], to compute the mixture.\n", + "\n", + "PmfOfWaitTime also uses MakeUniformPmf,\n", + "defined here:\n", + "\n", + " def MakeUniformPmf(low, high):\n", + " pmf = thinkbayes.Pmf()\n", + " for x in MakeRange(low=low, high=high):\n", + " pmf.Set(x, 1)\n", + " pmf.Normalize()\n", + " return pmf\n", + "\n", + "low and high are the range of the uniform\n", + "distribution, (both ends included). Finally, MakeUniformPmf\n", + "uses MakeRange, defined here:\n", + "\n", + " def MakeRange(low, high, skip=10):\n", + " return range(low, high+skip, skip)\n", + "\n", + "MakeRange defines a set of possible values for wait time\n", + "(expressed in seconds). By default it divides the range into 10 second\n", + "intervals.\n", + "\n", + "To encapsulate the process of computing these distributions, I created a\n", + "class called WaitTimeCalculator:\n", + "\n", + " class WaitTimeCalculator(object):\n", + "\n", + " def __init__(self, pmf_z):\n", + " self.pmf_z = pmf_z\n", + " self.pmf_zb = BiasPmf(pmf)\n", + "\n", + " self.pmf_y = self.PmfOfWaitTime(self.pmf_zb)\n", + " self.pmf_x = self.pmf_y\n", + "\n", + "The parameter, `pmf_z`, is the unbiased distribution of z.\n", + "`pmf_zb` is the biased distribution of gap time, as seen by passengers.\n", + "\n", + "`pmf_y` is the distribution of wait time. `pmf_x` is the distribution of\n", + "elapsed time, which is the same as the distribution of wait time. To see\n", + "why, remember that for a particular value of zp, the\n", + "distribution of y is uniform from 0 to zp.\n", + "Also\n", + "\n", + " x = zp - y\n", + "\n", + "So the distribution of x is also uniform from 0 to\n", + "zp.\n", + "\n", + "Figure [fig.redline2] shows the distribution of z,\n", + "zb, and y based on the data I collected from\n", + "the Red Line web site.\n", + "\n", + "To present these distributions, I am switching from Pmfs to Cdfs. Most\n", + "people are more familiar with Pmfs, but I think Cdfs are easier to\n", + "interpret, once you get used to them. And if you want to plot several\n", + "distributions on the same axes, Cdfs are the way to go.\n", + "\n", + "The mean of z is 7.8 minutes. The mean of zb\n", + "is 8.8 minutes, about 13% higher. The mean of y is 4.4,\n", + "half the mean of zb.\n", + "\n", + "As an aside, the Red Line schedule reports that trains run every 9\n", + "minutes during peak times. This is close to the average of\n", + "zb, but higher than the average of z. I\n", + "exchanged email with a representative of the MBTA, who confirmed that\n", + "the reported time between trains is deliberately conservative in order\n", + "to account for variability.\n", + "\n", + "Predicting wait times {#elapsed}\n", + "---------------------\n", + "\n", + "![Prior and posterior of x and predicted y.\n", + "](figs/redline3.pdf)\n", + "\n", + "[fig.redline3]\n", + "\n", + "Let’s get back to the motivating question: suppose that when I arrive at\n", + "the platform I see 10 people waiting. How long should I expect to wait\n", + "until the next train arrives?\n", + "\n", + "As always, let’s start with the easiest version of the problem and work\n", + "our way up. Suppose we are given the actual distribution of\n", + "z, and we know that the passenger arrival rate, $\\lambda$,\n", + "is 2 passengers per minute.\n", + "\n", + "In that case we can:\n", + "\n", + "1. Use the distribution of z to compute the prior\n", + " distribution of zp, the time between trains as seen by\n", + " a passenger.\n", + "\n", + "2. Then we can use the number of passengers to estimate the\n", + " distribution of x, the elapsed time since the last\n", + " train.\n", + "\n", + "3. Finally, we use the relation y = zp - x to get the\n", + " distribution of y.\n", + "\n", + "The first step is to create a WaitTimeCalculator that\n", + "encapsulates the distributions of zp, x, and\n", + "y, prior to taking into account the number of passengers.\n", + "\n", + " wtc = WaitTimeCalculator(pmf_z)\n", + "\n", + "`pmf_z` is the given distribution of gap times.\n", + "\n", + "The next step is to make an ElapsedTimeEstimator (defined\n", + "below), which encapsulates the posterior distribution of x\n", + "and the predictive distribution of y.\n", + "\n", + " ete = ElapsedTimeEstimator(wtc,\n", + " lam=2.0/60,\n", + " num_passengers=15)\n", + "\n", + "The parameters are the WaitTimeCalculator, the passenger\n", + "arrival rate, lam (expressed in passengers per second), and\n", + "the observed number of passengers, let’s say 15.\n", + "\n", + "Here is the definition of ElapsedTimeEstimator:\n", + "\n", + " class ElapsedTimeEstimator(object):\n", + "\n", + " def __init__(self, wtc, lam, num_passengers):\n", + " self.prior_x = Elapsed(wtc.pmf_x)\n", + "\n", + " self.post_x = self.prior_x.Copy()\n", + " self.post_x.Update((lam, num_passengers))\n", + "\n", + " self.pmf_y = PredictWaitTime(wtc.pmf_zb, self.post_x)\n", + "\n", + "`prior_x` and `posterior_x` are the prior and posterior distributions of\n", + "elapsed time. `pmf_y` is the predictive distribution of wait time.\n", + "\n", + "ElapsedTimeEstimator uses Elapsed and\n", + "PredictWaitTime, defined below.\n", + "\n", + "Elapsed is a Suite that represents the hypothetical\n", + "distribution of x. The prior distribution of x\n", + "comes straight from the WaitTimeCalculator. Then we use the\n", + "data, which consists of the arrival rate, lam, and the\n", + "number of passengers on the platform, to compute the posterior\n", + "distribution.\n", + "\n", + "Here’s the definition of Elapsed:\n", + "\n", + " class Elapsed(thinkbayes.Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = hypo\n", + " lam, k = data\n", + " like = thinkbayes.EvalPoissonPmf(k, lam * x)\n", + " return like\n", + "\n", + "As always, Likelihood takes a hypothesis and data, and\n", + "computes the likelihood of the data under the hypothesis. In this case\n", + "hypo is the elapsed time since the last train and\n", + "data is a tuple of lam and the number of\n", + "passengers.\n", + "\n", + "The likelihood of the data is the probability of getting k\n", + "arrivals in x time, given arrival rate lam. We\n", + "compute that using the PMF of the Poisson distribution.\n", + "\n", + "Finally, here’s the definition of PredictWaitTime:\n", + "\n", + " def PredictWaitTime(pmf_zb, pmf_x):\n", + " pmf_y = pmf_zb - pmf_x\n", + " RemoveNegatives(pmf_y)\n", + " return pmf_y\n", + "\n", + "`pmf_zb` is the distribution of gaps between trains; `pmf_x` is the\n", + "distribution of elapsed time, based on the observed number of\n", + "passengers. Since y = zb - x, we can compute\n", + "\n", + " pmf_y = pmf_zb - pmf_x\n", + "\n", + "The subtraction operator invokes `Pmf.__sub__`, which enumerates all\n", + "pairs of zb and x, computes the differences,\n", + "and adds the results to `pmf_y`.\n", + "\n", + "The resulting Pmf includes some negative values, which we know are\n", + "impossible. For example, if you arrive during a gap of 5 minutes, you\n", + "can’t wait more than 5 minutes. RemoveNegatives removes the\n", + "impossible values from the distribution and renormalizes.\n", + "\n", + " def RemoveNegatives(pmf):\n", + " for val in pmf.Values():\n", + " if val < 0:\n", + " pmf.Remove(val)\n", + " pmf.Normalize()\n", + "\n", + "Figure [fig.redline3] shows the results. The prior distribution of\n", + "x is the same as the distribution of y in\n", + "Figure [fig.redline2]. The posterior distribution of x\n", + "shows that, after seeing 15 passengers on the platform, we believe that\n", + "the time since the last train is probably 5-10 minutes. The predictive\n", + "distribution of y indicates that we expect the next train\n", + "in less than 5 minutes, with about 80% confidence.\n", + "\n", + "Estimating the arrival rate\n", + "---------------------------\n", + "\n", + "![Prior and posterior distributions of lam based on five\n", + "days of passenger data. ](figs/redline1.pdf)\n", + "\n", + "[fig.redline1]\n", + "\n", + "The analysis so far has been based on the assumption that we know (1)\n", + "the distribution of gaps and (2) the passenger arrival rate. Now we are\n", + "ready to relax the second assumption.\n", + "\n", + "Suppose that you just moved to Boston, so you don’t know much about the\n", + "passenger arrival rate on the Red Line. After a few days of commuting,\n", + "you could make a guess, at least qualitatively. With a little more\n", + "effort, you could estimate $\\lambda$ quantitatively.\n", + "\n", + "Each day when you arrive at the platform, you should note the time and\n", + "the number of passengers waiting (if the platform is too big, you could\n", + "choose a sample area). Then you should record your wait time and the\n", + "number of new arrivals while you are waiting.\n", + "\n", + "After five days, you might have data like this:\n", + "\n", + " k1 y k2\n", + " -- --- --\n", + " 17 4.6 9\n", + " 22 1.0 0\n", + " 23 1.4 4\n", + " 18 5.4 12\n", + " 4 5.8 11\n", + "\n", + "where k1 is the number of passengers waiting when you\n", + "arrive, y is your wait time in minutes, and k2\n", + "is the number of passengers who arrive while you are waiting.\n", + "\n", + "Over the course of one week, you waited 18 minutes and saw 36 passengers\n", + "arrive, so you would estimate that the arrival rate is 2 passengers per\n", + "minute. For practical purposes that estimate is good enough, but for the\n", + "sake of completeness I will compute a posterior distribution for\n", + "$\\lambda$ and show how to use that distribution in the rest of the\n", + "analysis.\n", + "\n", + "ArrivalRate is a Suite that represents\n", + "hypotheses about $\\lambda$. As always, Likelihood takes a\n", + "hypothesis and data, and computes the likelihood of the data under the\n", + "hypothesis.\n", + "\n", + "In this case the hypothesis is a value of $\\lambda$. The data is a pair,\n", + "y, k, where y is a wait time and\n", + "k is the number of passengers that arrived.\n", + "\n", + " class ArrivalRate(thinkbayes.Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " lam = hypo\n", + " y, k = data\n", + " like = thinkbayes.EvalPoissonPmf(k, lam * y)\n", + " return like\n", + "\n", + "This Likelihood might look familiar; it is almost identical\n", + "to Elapsed.Likelihood in Section [elapsed]. The difference\n", + "is that in Elapsed.Likelihood the hypothesis is\n", + "x, the elapsed time; in ArrivalRate.Likelihood\n", + "the hypothesis is lam, the arrival rate. But in both cases\n", + "the likelihood is the probability of seeing k arrivals in\n", + "some period of time, given lam.\n", + "\n", + "ArrivalRateEstimator encapsulates the process of estimating\n", + "$\\lambda$. The parameter, `passenger_data`, is a list of k1, y,\n", + "k2 tuples, as in the table above.\n", + "\n", + " class ArrivalRateEstimator(object):\n", + "\n", + " def __init__(self, passenger_data):\n", + " low, high = 0, 5\n", + " n = 51\n", + " hypos = numpy.linspace(low, high, n) / 60\n", + "\n", + " self.prior_lam = ArrivalRate(hypos)\n", + "\n", + " self.post_lam = self.prior_lam.Copy()\n", + " for k1, y, k2 in passenger_data:\n", + " self.post_lam.Update((y, k2))\n", + "\n", + "`__init__` builds hypos, which is a sequence of\n", + "hypothetical values for lam, then builds the prior\n", + "distribution, `prior_lam`. The for loop updates the prior\n", + "with data, yielding the posterior distribution, `post_lam`.\n", + "\n", + "Figure [fig.redline1] shows the prior and posterior distributions. As\n", + "expected, the mean and median of the posterior are near the observed\n", + "rate, 2 passengers per minute. But the spread of the posterior\n", + "distribution captures our uncertainty about $\\lambda$ based on a small\n", + "sample.\n", + "\n", + "Incorporating uncertainty\n", + "-------------------------\n", + "\n", + "![Predictive distributions of y for possible values of\n", + "lam. ](figs/redline4.pdf)\n", + "\n", + "[fig.redline4]\n", + "\n", + "Whenever there is uncertainty about one of the inputs to an analysis, we\n", + "can take it into account by a process like this:\n", + "\n", + "1. Implement the analysis based on a deterministic value of the\n", + " uncertain parameter (in this case $\\lambda$).\n", + "\n", + "2. Compute the distribution of the uncertain parameter.\n", + "\n", + "3. Run the analysis for each value of the parameter, and generate a set\n", + " of predictive distributions.\n", + "\n", + "4. Compute a mixture of the predictive distributions, using the weights\n", + " from the distribution of the parameter.\n", + "\n", + "We have already done steps (1) and (2). I wrote a class called\n", + "WaitMixtureEstimator to handle steps (3) and (4).\n", + "\n", + " class WaitMixtureEstimator(object):\n", + "\n", + " def __init__(self, wtc, are, num_passengers=15):\n", + " self.metapmf = thinkbayes.Pmf()\n", + "\n", + " for lam, prob in sorted(are.post_lam.Items()):\n", + " ete = ElapsedTimeEstimator(wtc, lam, num_passengers)\n", + " self.metapmf.Set(ete.pmf_y, prob)\n", + "\n", + " self.mixture = thinkbayes.MakeMixture(self.metapmf)\n", + "\n", + "wtc is the WaitTimeCalculator that contains\n", + "the distribution of zb. are is the\n", + "ArrivalTimeEstimator that contains the distribution of\n", + "lam.\n", + "\n", + "The first line makes a meta-Pmf that maps from each possible\n", + "distribution of y to its probability. For each value of\n", + "lam, we use ElapsedTimeEstimator to compute\n", + "the corresponding distribution of y and store it in the\n", + "Meta-Pmf. Then we use MakeMixture to compute the mixture.\n", + "\n", + "Figure [fig.redline4] shows the results. The shaded lines in the\n", + "background are the distributions of y for each value of\n", + "lam, with line thickness that represents likelihood. The\n", + "dark line is the mixture of these distributions.\n", + "\n", + "In this case we could get a very similar result using a single point\n", + "estimate of lam. So it was not necessary, for practical\n", + "purposes, to include the uncertainty of the estimate.\n", + "\n", + "In general, it is important to include variability if the system\n", + "response is non-linear; that is, if small changes in the input can cause\n", + "big changes in the output. In this case, posterior variability in\n", + "lam is small and the system response is approximately\n", + "linear for small perturbations.\n", + "\n", + "Decision analysis {#decision-analysis}\n", + "-----------------\n", + "\n", + "![Probability that wait time exceeds 15 minutes as a function of the\n", + "number of passengers on the platform. ](figs/redline5.pdf)\n", + "\n", + "[fig.redline5]\n", + "\n", + "At this point we can use the number of passengers on the platform to\n", + "predict the distribution of wait times. Now let’s get to the second part\n", + "of the question: when should I stop waiting for the train and go catch a\n", + "taxi?\n", + "\n", + "Remember that in the original scenario, I am trying to get to South\n", + "Station to catch the commuter rail. Suppose I leave the office with\n", + "enough time that I can wait 15 minutes and still make my connection at\n", + "South Station.\n", + "\n", + "In that case I would like to know the probability that y\n", + "exceeds 15 minutes as a function of `num_passengers`. It is easy enough\n", + "to use the analysis from Section [elapsed] and run it for a range of\n", + "`num_passengers`.\n", + "\n", + "But there’s a problem. The analysis is sensitive to the frequency of\n", + "long delays, and because long delays are rare, it is hard estimate their\n", + "frequency.\n", + "\n", + "I only have data from one week, and the longest delay I observed was 15\n", + "minutes. So I can’t estimate the frequency of longer delays accurately.\n", + "\n", + "However, I can use previous observations to make at least a coarse\n", + "estimate. When I commuted by Red Line for a year, I saw three long\n", + "delays caused by a signaling problem, a power outage, and “police\n", + "activity” at another stop. So I estimate that there are about 3 major\n", + "delays per year.\n", + "\n", + "But remember that my observations are biased. I am more likely to\n", + "observe long delays because they affect a large number of passengers. So\n", + "we should treat my observations as a sample of zb rather\n", + "than z. Here’s how we can do that.\n", + "\n", + "During my year of commuting, I took the Red Line home about 220 times.\n", + "So I take the observed gap times, `gap_times`, generate a sample of 220\n", + "gaps, and compute their Pmf:\n", + "\n", + " n = 220\n", + " cdf_z = thinkbayes.MakeCdfFromList(gap_times)\n", + " sample_z = cdf_z.Sample(n)\n", + " pmf_z = thinkbayes.MakePmfFromList(sample_z)\n", + "\n", + "Next I bias `pmf_z` to get the distribution of zb, draw a\n", + "sample, and then add in delays of 30, 40, and 50 minutes (expressed in\n", + "seconds):\n", + "\n", + " cdf_zp = BiasPmf(pmf_z).MakeCdf()\n", + " sample_zb = cdf_zp.Sample(n) + [1800, 2400, 3000]\n", + "\n", + "Cdf.Sample is more efficient than Pmf.Sample,\n", + "so it is usually faster to convert a Pmf to a Cdf before sampling.\n", + "\n", + "Next I use the sample of zb to estimate a Pdf using KDE,\n", + "and then convert the Pdf to a Pmf:\n", + "\n", + " pdf_zb = thinkbayes.EstimatedPdf(sample_zb)\n", + " xs = MakeRange(low=60)\n", + " pmf_zb = pdf_zb.MakePmf(xs)\n", + "\n", + "Finally I unbias the distribution of zb to get the\n", + "distribution of z, which I use to create the\n", + "WaitTimeCalculator:\n", + "\n", + " pmf_z = UnbiasPmf(pmf_zb)\n", + " wtc = WaitTimeCalculator(pmf_z)\n", + "\n", + "This process is complicated, but all of the steps are operations we have\n", + "seen before. Now we are ready to compute the probability of a long wait.\n", + "\n", + " def ProbLongWait(num_passengers, minutes):\n", + " ete = ElapsedTimeEstimator(wtc, lam, num_passengers)\n", + " cdf_y = ete.pmf_y.MakeCdf()\n", + " prob = 1 - cdf_y.Prob(minutes * 60)\n", + "\n", + "Given the number of passengers on the platform,\n", + "ProbLongWait makes an ElapsedTimeEstimator,\n", + "extracts the distribution of wait time, and computes the probability\n", + "that wait time exceeds minutes.\n", + "\n", + "Figure [fig.redline5] shows the result. When the number of passengers is\n", + "less than 20, we infer that the system is operating normally, so the\n", + "probability of a long delay is small. If there are 30 passengers, we\n", + "estimate that it has been 15 minutes since the last train; that’s longer\n", + "than a normal delay, so we infer that something is wrong and expect\n", + "longer delays.\n", + "\n", + "If we are willing to accept a 10% chance of missing the connection at\n", + "South Station, we should stay and wait as long as there are fewer than\n", + "30 passengers, and take a taxi if there are more.\n", + "\n", + "Or, to take this analysis one step further, we could quantify the cost\n", + "of missing the connection and the cost of taking a taxi, then choose the\n", + "threshold that minimizes expected cost.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "The analysis so far has been based on the assumption that the arrival\n", + "rate of passengers is the same every day. For a commuter train during\n", + "rush hour, that might not be a bad assumption, but there are some\n", + "obvious exceptions. For example, if there is a special event nearby, a\n", + "large number of people might arrive at the same time. In that case, the\n", + "estimate of lam would be too low, so the estimates of\n", + "x and y would be too high.\n", + "\n", + "If special events are as common as major delays, it would be important\n", + "to include them in the model. We could do that by extending the\n", + "distribution of lam to include occasional large values.\n", + "\n", + "We started with the assumption that we know distribution of\n", + "z. As an alternative, a passenger could estimate\n", + "z, but it would not be easy. As a passenger, you only\n", + "observe only your own wait time, y. Unless you skip the\n", + "first train and wait for the second, you don’t observe the gap between\n", + "trains, z.\n", + "\n", + "However, we could make some inferences about zb. If we note\n", + "the number of passengers waiting when we arrive, we can estimate the\n", + "elapsed time since the last train, x. Then we observe\n", + "y. If we add the posterior distribution of x\n", + "to the observed y, we get a distribution that represents\n", + "our posterior belief about the observed value of zb.\n", + "\n", + "We can use this distribution to update our beliefs about the\n", + "distribution of zb. Finally, we can compute the inverse of\n", + "BiasPmf to get from the distribution of zb to\n", + "the distribution of z.\n", + "\n", + "I leave this analysis as an exercise for the reader. One suggestion: you\n", + "should read Chapter [species] first. You can find the outline of a\n", + "solution in . For more information see\n", + "Section [download].\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "This exercise is from MacKay, *Information Theory, Inference, and\n", + "Learning Algorithms*:\n", + "\n", + "> Unstable particles are emitted from a source and decay at a distance\n", + "> $x$, a real number that has an exponential probability distribution\n", + "> with [parameter] $\\lambda$. Decay events can only be observed if they\n", + "> occur in a window extending from $x=1$ cm to $x=20$ cm. $N$ decays are\n", + "> observed at locations $\\{ 1.5, 2, 3, 4, 5, 12 \\}$ cm. What is the\n", + "> posterior distribution of $\\lambda$?\n", + "\n", + "You can download a solution to this exercise from\n", + ".\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/09_Two_Dimensions.ipynb b/notebooks/09_Two_Dimensions.ipynb new file mode 100644 index 0000000..4f1e29e --- /dev/null +++ b/notebooks/09_Two_Dimensions.ipynb @@ -0,0 +1,468 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Two Dimensions {#paintball}\n", + "==============\n", + "\n", + "Paintball\n", + "---------\n", + "\n", + "Paintball is a sport in which competing teams try to shoot each other\n", + "with guns that fire paint-filled pellets that break on impact, leaving a\n", + "colorful mark on the target. It is usually played in an arena decorated\n", + "with barriers and other objects that can be used as cover.\n", + "\n", + "Suppose you are playing paintball in an indoor arena 30 feet wide and 50\n", + "feet long. You are standing near one of the 30 foot walls, and you\n", + "suspect that one of your opponents has taken cover nearby. Along the\n", + "wall, you see several paint spatters, all the same color, that you think\n", + "your opponent fired recently.\n", + "\n", + "The spatters are at 15, 16, 18, and 21 feet, measured from the\n", + "lower-left corner of the room. Based on these data, where do you think\n", + "your opponent is hiding?\n", + "\n", + "Figure [fig.paintball] shows a diagram of the arena. Using the\n", + "lower-left corner of the room as the origin, I denote the unknown\n", + "location of the shooter with coordinates $\\alpha$ and $\\beta$, or\n", + "alpha and beta. The location of a spatter is\n", + "labeled x. The angle the opponent shoots at is $\\theta$ or\n", + "theta.\n", + "\n", + "The Paintball problem is a modified version of the Lighthouse problem, a\n", + "common example of Bayesian analysis. My notation follows the\n", + "presentation of the problem in D.S. Sivia’s, *Data Analysis: a\n", + "Bayesian Tutorial, Second Edition* (Oxford, 2006).\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "The suite\n", + "---------\n", + "\n", + "![Diagram of the layout for the paintball problem.](figs/paintball.pdf)\n", + "\n", + "[fig.paintball]\n", + "\n", + "To get started, we need a Suite that represents a set of hypotheses\n", + "about the location of the opponent. Each hypothesis is a pair of\n", + "coordinates: (alpha, beta).\n", + "\n", + "Here is the definition of the Paintball suite:\n", + "\n", + " class Paintball(thinkbayes.Suite, thinkbayes.Joint):\n", + "\n", + " def __init__(self, alphas, betas, locations):\n", + " self.locations = locations\n", + " pairs = [(alpha, beta) \n", + " for alpha in alphas \n", + " for beta in betas]\n", + " thinkbayes.Suite.__init__(self, pairs)\n", + "\n", + "Paintball inherits from Suite, which we have\n", + "seen before, and Joint, which I will explain soon.\n", + "\n", + "alphas is the list of possible values for\n", + "alpha; betas is the list of values for\n", + "beta. pairs is a list of all (alpha,\n", + "beta) pairs.\n", + "\n", + "locations is a list of possible locations along the wall;\n", + "it is stored for use in Likelihood.\n", + "\n", + "![Posterior CDFs for alpha and beta, given the\n", + "data.](figs/paintball2.pdf)\n", + "\n", + "[fig.paintball2]\n", + "\n", + "The room is 30 feet wide and 50 feet long, so here’s the code that\n", + "creates the suite:\n", + "\n", + " alphas = range(0, 31)\n", + " betas = range(1, 51)\n", + " locations = range(0, 31)\n", + "\n", + " suite = Paintball(alphas, betas, locations)\n", + "\n", + "This prior distribution assumes that all locations in the room are\n", + "equally likely. Given a map of the room, we might choose a more detailed\n", + "prior, but we’ll start simple.\n", + "\n", + "Trigonometry\n", + "------------\n", + "\n", + "Now we need a likelihood function, which means we have to figure out the\n", + "likelihood of hitting any spot along the wall, given the location of the\n", + "opponent.\n", + "\n", + "As a simple model, imagine that the opponent is like a rotating turret,\n", + "equally likely to shoot in any direction. In that case, he is most\n", + "likely to hit the wall at location alpha, and less likely\n", + "to hit the wall far away from alpha.\n", + "\n", + "With a little trigonometry, we can compute the probability of hitting\n", + "any spot along the wall. Imagine that the shooter fires a shot at angle\n", + "$\\theta$; the pellet would hit the wall at location $x$, where\n", + "$$x - \\alpha = \\beta \\tan \\theta$$ Solving this equation for $\\theta$\n", + "yields $$\\theta = tan^{-1} \\left( \\frac{x - \\alpha}{\\beta} \\right)$$ So\n", + "given a location on the wall, we can find $\\theta$.\n", + "\n", + "Taking the derivative of the first equation with respect to $\\theta$\n", + "yields $$\\frac{dx}{d\\theta} = \\frac{\\beta}{\\cos^2 \\theta}$$ This\n", + "derivative is what I’ll call the “strafing speed”, which is the speed of\n", + "the target location along the wall as $\\theta$ increases. The\n", + "probability of hitting a given point on the wall is inversely related to\n", + "strafing speed.\n", + "\n", + "If we know the coordinates of the shooter and a location along the wall,\n", + "we can compute strafing speed:\n", + "\n", + " def StrafingSpeed(alpha, beta, x):\n", + " theta = math.atan2(x - alpha, beta)\n", + " speed = beta / math.cos(theta)**2\n", + " return speed\n", + "\n", + "alpha and beta are the coordinates of the\n", + "shooter; x is the location of a spatter. The result is the\n", + "derivative of x with respect to theta.\n", + "\n", + "![PMF of location given alpha=10, for several values of\n", + "beta.](figs/paintball1.pdf)\n", + "\n", + "[fig.paintball1]\n", + "\n", + "Now we can compute a Pmf that represents the probability of hitting any\n", + "location on the wall. MakeLocationPmf takes\n", + "alpha and beta, the coordinates of the\n", + "shooter, and locations, a list of possible values of\n", + "x.\n", + "\n", + " def MakeLocationPmf(alpha, beta, locations):\n", + " pmf = thinkbayes.Pmf()\n", + " for x in locations:\n", + " prob = 1.0 / StrafingSpeed(alpha, beta, x)\n", + " pmf.Set(x, prob)\n", + " pmf.Normalize()\n", + " return pmf\n", + "\n", + "MakeLocationPmf computes the probability of hitting each\n", + "location, which is inversely related to strafing speed. The result is a\n", + "Pmf of locations and their probabilities.\n", + "\n", + "Figure [fig.paintball1] shows the Pmf of location with alpha =\n", + "10 and a range of values for beta. For all values of\n", + "beta the most likely spatter location is x = 10; as\n", + "beta increases, so does the spread of the Pmf.\n", + "\n", + "Likelihood\n", + "----------\n", + "\n", + "Now all we need is a likelihood function. We can use\n", + "MakeLocationPmf to compute the likelihood of any value of\n", + "x, given the coordinates of the opponent.\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " alpha, beta = hypo\n", + " x = data\n", + " pmf = MakeLocationPmf(alpha, beta, self.locations)\n", + " like = pmf.Prob(x)\n", + " return like\n", + "\n", + "Again, alpha and beta are the hypothetical\n", + "coordinates of the shooter, and x is the location of an\n", + "observed spatter.\n", + "\n", + "pmf contains the probability of each location, given the\n", + "coordinates of the shooter. From this Pmf, we select the probability of\n", + "the observed location.\n", + "\n", + "And we’re done. To update the suite, we can use UpdateSet,\n", + "which is inherited from Suite.\n", + "\n", + " suite.UpdateSet([15, 16, 18, 21])\n", + "\n", + "The result is a distribution that maps each (alpha, beta)\n", + "pair to a posterior probability.\n", + "\n", + "Joint distributions\n", + "-------------------\n", + "\n", + "When each value in a distribution is a tuple of variables, it is called\n", + "a **joint distribution** because it represents the\n", + "distributions of the variables together, that is “jointly”. A joint\n", + "distribution contains the distributions of the variables, as well\n", + "information about the relationships among them.\n", + "\n", + "Given a joint distribution, we can compute the distributions of each\n", + "variable independently, which are called the **marginal\n", + "distributions**.\n", + "\n", + "thinkbayes.Joint provides a method that computes marginal\n", + "distributions:\n", + "\n", + " # class Joint:\n", + "\n", + " def Marginal(self, i):\n", + " pmf = Pmf()\n", + " for vs, prob in self.Items():\n", + " pmf.Incr(vs[i], prob)\n", + " return pmf\n", + "\n", + "i is the index of the variable we want; in this example\n", + "i=0 indicates the distribution of alpha, and\n", + "i=1 indicates the distribution of beta.\n", + "\n", + "Here’s the code that extracts the marginal distributions:\n", + "\n", + " marginal_alpha = suite.Marginal(0)\n", + " marginal_beta = suite.Marginal(1)\n", + "\n", + "Figure [fig.paintball2] shows the results (converted to CDFs). The\n", + "median value for alpha is 18, near the center of mass of\n", + "the observed spatters. For beta, the most likely values are\n", + "close to the wall, but beyond 10 feet the distribution is almost\n", + "uniform, which indicates that the data do not distinguish strongly\n", + "between these possible locations.\n", + "\n", + "Given the posterior marginals, we can compute credible intervals for\n", + "each coordinate independently:\n", + "\n", + " print 'alpha CI', marginal_alpha.CredibleInterval(50)\n", + " print 'beta CI', marginal_beta.CredibleInterval(50)\n", + "\n", + "The 50% credible intervals are (14, 21) for\n", + "alpha and (5, 31) for beta. So\n", + "the data provide evidence that the shooter is in the near side of the\n", + "room. But it is not strong evidence. The 90% credible intervals cover\n", + "most of the room!\n", + "\n", + "Conditional distributions {#conditional}\n", + "-------------------------\n", + "\n", + "![Posterior distributions for alpha conditioned on several\n", + "values of beta.](figs/paintball3.pdf)\n", + "\n", + "[fig.paintball3]\n", + "\n", + "The marginal distributions contain information about the variables\n", + "independently, but they do not capture the dependence between variables,\n", + "if any.\n", + "\n", + "One way to visualize dependence is by computing **conditional\n", + "distributions**. thinkbayes.Joint provides a method\n", + "that does that:\n", + "\n", + " def Conditional(self, i, j, val):\n", + " pmf = Pmf()\n", + " for vs, prob in self.Items():\n", + " if vs[j] != val: continue\n", + " pmf.Incr(vs[i], prob)\n", + "\n", + " pmf.Normalize()\n", + " return pmf\n", + "\n", + "Again, i is the index of the variable we want;\n", + "j is the index of the conditioning variable, and\n", + "val is the conditional value.\n", + "\n", + "The result is the distribution of the $i$th variable under the condition\n", + "that the $j$th variable is val.\n", + "\n", + "For example, the following code computes the conditional distributions\n", + "of alpha for a range of values of beta:\n", + "\n", + " betas = [10, 20, 40]\n", + "\n", + " for beta in betas:\n", + " cond = suite.Conditional(0, 1, beta)\n", + "\n", + "Figure [fig.paintball3] shows the results, which we could fully describe\n", + "as “posterior conditional marginal distributions.” Whew!\n", + "\n", + "If the variables were independent, the conditional distributions would\n", + "all be the same. Since they are all different, we can tell the variables\n", + "are dependent. For example, if we know (somehow) that beta =\n", + "10, the conditional distribution of alpha is fairly\n", + "narrow. For larger values of beta, the distribution of\n", + "alpha is wider.\n", + "\n", + "Credible intervals {#credible-intervals}\n", + "------------------\n", + "\n", + "![Credible intervals for the coordinates of the\n", + "opponent.](figs/paintball5.pdf)\n", + "\n", + "[fig.paintball5]\n", + "\n", + "Another way to visualize the posterior joint distribution is to compute\n", + "credible intervals. When we looked at credible intervals in\n", + "Section [credible], I skipped over a subtle point: for a given\n", + "distribution, there are many intervals with the same level of\n", + "credibility. For example, if you want a 50% credible interval, you could\n", + "choose any set of values whose probability adds up to 50%.\n", + "\n", + "When the values are one-dimensional, it is most common to choose the\n", + "**central credible interval**; for example, the central 50%\n", + "credible interval contains all values between the 25th and 75th\n", + "percentiles.\n", + "\n", + "In multiple dimensions it is less obvious what the right credible\n", + "interval should be. The best choice might depend on context, but one\n", + "common choice is the maximum likelihood credible interval, which\n", + "contains the most likely values that add up to 50% (or some other\n", + "percentage).\n", + "\n", + "thinkbayes.Joint provides a method that computes maximum\n", + "likelihood credible intervals.\n", + "\n", + " # class Joint:\n", + "\n", + " def MaxLikeInterval(self, percentage=90):\n", + " interval = []\n", + " total = 0\n", + "\n", + " t = [(prob, val) for val, prob in self.Items()]\n", + " t.sort(reverse=True)\n", + "\n", + " for prob, val in t:\n", + " interval.append(val)\n", + " total += prob\n", + " if total >= percentage/100.0:\n", + " break\n", + "\n", + " return interval\n", + "\n", + "The first step is to make a list of the values in the suite, sorted in\n", + "descending order by probability. Next we traverse the list, adding each\n", + "value to the interval, until the total probability exceeds\n", + "percentage. The result is a list of values from the suite.\n", + "Notice that this set of values is not necessarily contiguous.\n", + "\n", + "To visualize the intervals, I wrote a function that “colors” each value\n", + "according to how many intervals it appears in:\n", + "\n", + " def MakeCrediblePlot(suite):\n", + " d = dict((pair, 0) for pair in suite.Values())\n", + "\n", + " percentages = [75, 50, 25]\n", + " for p in percentages:\n", + " interval = suite.MaxLikeInterval(p)\n", + " for pair in interval:\n", + " d[pair] += 1\n", + "\n", + " return d\n", + "\n", + "d is a dictionary that maps from each value in the suite to\n", + "the number of intervals it appears in. The loop computes intervals for\n", + "several percentages and modifies d.\n", + "\n", + "Figure [fig.paintball5] shows the result. The 25% credible interval is\n", + "the darkest region near the bottom wall. For higher percentages, the\n", + "credible interval is bigger, of course, and skewed toward the right side\n", + "of the room.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "This chapter shows that the Bayesian framework from the previous\n", + "chapters can be extended to handle a two-dimensional parameter space.\n", + "The only difference is that each hypothesis is represented by a tuple of\n", + "parameters.\n", + "\n", + "I also presented Joint, which is a parent class that\n", + "provides methods that apply to joint distributions:\n", + "Marginal, Conditional, and\n", + "MakeLikeInterval. In object-oriented terms,\n", + "Joint is a mixin (see\n", + ").\n", + "\n", + "There is a lot of new vocabulary in this chapter, so let’s review:\n", + "\n", + "Joint distribution:\n", + ": A distribution that represents all possible values in a\n", + " multidimensional space and their probabilities. The example in this\n", + " chapter is a two-dimensional space made up of the coordinates\n", + " alpha and beta. The joint distribution\n", + " represents the probability of each (alpha,\n", + " beta) pair.\n", + "\n", + "Marginal distribution:\n", + ": The distribution of one parameter in a joint distribution, treating\n", + " the other parameters as unknown. For example,\n", + " Figure [fig.paintball2] shows the distributions of\n", + " alpha and beta independently.\n", + "\n", + "Conditional distribution:\n", + ": The distribution of one parameter in a joint distribution,\n", + " conditioned on one or more of the other parameters.\n", + " Figure [fig.paintball3] several distributions for\n", + " alpha, conditioned on different values of\n", + " beta.\n", + "\n", + "Given the joint distribution, you can compute marginal and conditional\n", + "distributions. With enough conditional distributions, you could\n", + "re-create the joint distribution, at least approximately. But given the\n", + "marginal distributions you cannot re-create the joint distribution\n", + "because you have lost information about the dependence between\n", + "variables.\n", + "\n", + "If there are $n$ possible values for each of two parameters, most\n", + "operations on the joint distribution take time proportional to $n^2$. If\n", + "there are $d$ parameters, run time is proportional to $n^d$, which\n", + "quickly becomes impractical as the number of dimensions increases.\n", + "\n", + "If you can process a million hypotheses in a reasonable amount of time,\n", + "you could handle two dimensions with 1000 values for each parameter, or\n", + "three dimensions with 100 values each, or six dimensions with 10 values\n", + "each.\n", + "\n", + "If you need more dimensions, or more values per dimension, there are\n", + "optimizations you can try. I present an example in Chapter [species].\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "In our simple model, the opponent is equally likely to shoot in any\n", + "direction. As an exercise, let’s consider improvements to this model.\n", + "\n", + "The analysis in this chapter suggests that a shooter is most likely to\n", + "hit the closest wall. But in reality, if the opponent is close to a\n", + "wall, he is unlikely to shoot at the wall because he is unlikely to see\n", + "a target between himself and the wall.\n", + "\n", + "Design an improved model that takes this behavior into account. Try to\n", + "find a model that is more realistic, but not too complicated.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/10_Approximate_Bayesian_Computation.ipynb b/notebooks/10_Approximate_Bayesian_Computation.ipynb new file mode 100644 index 0000000..971b4cb --- /dev/null +++ b/notebooks/10_Approximate_Bayesian_Computation.ipynb @@ -0,0 +1,734 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Approximate Bayesian Computation\n", + "================================\n", + "\n", + "The Variability Hypothesis\n", + "--------------------------\n", + "\n", + "I have a soft spot for crank science. Recently I visited Norumbega\n", + "Tower, which is an enduring monument to the crackpot theories of Eben\n", + "Norton Horsford, inventor of double-acting baking powder and fake\n", + "history. But that’s not what this chapter is about.\n", + "\n", + "This chapter is about the Variability Hypothesis, which\n", + "\n", + "> “originated in the early nineteenth century with Johann Meckel, who\n", + "> argued that males have a greater range of ability than females,\n", + "> especially in intelligence. In other words, he believed that most\n", + "> geniuses and most mentally retarded people are men. Because he\n", + "> considered males to be the ’superior animal,’ Meckel concluded that\n", + "> females’ lack of variation was a sign of inferiority.”\n", + ">\n", + "> From .\n", + "\n", + "I particularly like that last part, because I suspect that if it turns\n", + "out that women are actually more variable, Meckel would take that as a\n", + "sign of inferiority, too. Anyway, you will not be surprised to hear that\n", + "the evidence for the Variability Hypothesis is weak.\n", + "\n", + "Nevertheless, it came up in my class recently when we looked at data\n", + "from the CDC’s Behavioral Risk Factor Surveillance System (BRFSS),\n", + "specifically the self-reported heights of adult American men and women.\n", + "The dataset includes responses from 154407 men and 254722 women. Here’s\n", + "what we found:\n", + "\n", + "- The average height for men is 178 cm; the average height for women\n", + " is 163 cm. So men are taller, on average. No surprise there.\n", + "\n", + "- For men the standard deviation is 7.7 cm; for women it is 7.3 cm. So\n", + " in absolute terms, men’s heights are more variable.\n", + "\n", + "- But to compare variability between groups, it is more meaningful to\n", + " use the coefficient of variation (CV), which is the standard\n", + " deviation divided by the mean. It is a dimensionless measure of\n", + " variability relative to scale. For men CV is 0.0433; for women it is\n", + " 0.0444.\n", + "\n", + "That’s very close, so we could conclude that this dataset provides weak\n", + "evidence against the Variability Hypothesis. But we can use Bayesian\n", + "methods to make that conclusion more precise. And answering this\n", + "question gives me a chance to demonstrate some techniques for working\n", + "with large datasets.\n", + "\n", + "I will proceed in a few steps:\n", + "\n", + "1. We’ll start with the simplest implementation, but it only works for\n", + " datasets smaller than 1000 values.\n", + "\n", + "2. By computing probabilities under a log transform, we can scale up to\n", + " the full size of the dataset, but the computation gets slow.\n", + "\n", + "3. Finally, we speed things up substantially with Approximate Bayesian\n", + " Computation, also known as ABC.\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Mean and standard deviation\n", + "---------------------------\n", + "\n", + "In Chapter [paintball] we estimated two parameters simultaneously using\n", + "a joint distribution. In this chapter we use the same method to estimate\n", + "the parameters of a Gaussian distribution: the mean, mu,\n", + "and the standard deviation, sigma.\n", + "\n", + "For this problem, I define a Suite called Height that\n", + "represents a map from each mu, sigma pair to its\n", + "probability:\n", + "\n", + " class Height(thinkbayes.Suite, thinkbayes.Joint):\n", + "\n", + " def __init__(self, mus, sigmas):\n", + " thinkbayes.Suite.__init__(self)\n", + "\n", + " pairs = [(mu, sigma) \n", + " for mu in mus\n", + " for sigma in sigmas]\n", + "\n", + " thinkbayes.Suite.__init__(self, pairs)\n", + "\n", + "mus is a sequence of possible values for mu;\n", + "sigmas is a sequence of values for sigma. The\n", + "prior distribution is uniform over all mu, sigma pairs.\n", + "\n", + "The likelihood function is easy. Given hypothetical values of\n", + "mu and sigma, we compute the likelihood of a\n", + "particular value, x. That’s what\n", + "EvalGaussianPdf does, so all we have to do is use it:\n", + "\n", + " # class Height\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = data\n", + " mu, sigma = hypo\n", + " like = thinkbayes.EvalGaussianPdf(x, mu, sigma)\n", + " return like\n", + "\n", + "If you have studied statistics from a mathematical perspective, you know\n", + "that when you evaluate a PDF, you get a probability density. In order to\n", + "get a probability, you have to integrate probability densities over some\n", + "range.\n", + "\n", + "But for our purposes, we don’t need a probability; we just need\n", + "something proportional to the probability we want. A probability density\n", + "does that job nicely.\n", + "\n", + "The hardest part of this problem turns out to be choosing appropriate\n", + "ranges for mus and sigmas. If the range is too\n", + "small, we omit some possibilities with non-negligible probability and\n", + "get the wrong answer. If the range is too big, we get the right answer,\n", + "but waste computational power.\n", + "\n", + "So this is an opportunity to use classical estimation to make Bayesian\n", + "techniques more efficient. Specifically, we can use classical estimators\n", + "to find a likely location for mu and sigma,\n", + "and use the standard errors of those estimates to choose a likely\n", + "spread.\n", + "\n", + "If the true parameters of the distribution are $\\mu$ and $\\sigma$, and\n", + "we take a sample of $n$ values, an estimator of $\\mu$ is the sample\n", + "mean, m.\n", + "\n", + "And an estimator of $\\sigma$ is the sample standard variance,\n", + "s.\n", + "\n", + "The standard error of the estimated $\\mu$ is $s / \\sqrt{n}$ and the\n", + "standard error of the estimated $\\sigma$ is $s / \\sqrt{2 (n-1)}$.\n", + "\n", + "Here’s the code to compute all that:\n", + "\n", + " def FindPriorRanges(xs, num_points, num_stderrs=3.0):\n", + "\n", + " # compute m and s\n", + " n = len(xs)\n", + " m = numpy.mean(xs)\n", + " s = numpy.std(xs)\n", + "\n", + " # compute ranges for m and s\n", + " stderr_m = s / math.sqrt(n)\n", + " mus = MakeRange(m, stderr_m, num_stderrs)\n", + "\n", + " stderr_s = s / math.sqrt(2 * (n-1))\n", + " sigmas = MakeRange(s, stderr_s, num_stderrs)\n", + "\n", + " return mus, sigmas\n", + "\n", + "xs is the dataset. `num_points` is the desired number of\n", + "values in the range. `num_stderrs` is the width of the range on each\n", + "side of the estimate, in number of standard errors.\n", + "\n", + "The return value is a pair of sequences, mus and\n", + "sigmas.\n", + "\n", + "Here’s MakeRange:\n", + "\n", + " def MakeRange(estimate, stderr, num_stderrs):\n", + " spread = stderr * num_stderrs\n", + " array = numpy.linspace(estimate-spread,\n", + " estimate+spread,\n", + " num_points)\n", + " return array\n", + "\n", + "numpy.linspace makes an array of equally spaced elements\n", + "between estimate-spread and estimate+spread,\n", + "including both.\n", + "\n", + "Update\n", + "------\n", + "\n", + "Finally here’s the code to make and update the suite:\n", + "\n", + " mus, sigmas = FindPriorRanges(xs, num_points)\n", + " suite = Height(mus, sigmas)\n", + " suite.UpdateSet(xs)\n", + " print suite.MaximumLikelihood() \n", + "\n", + "This process might seem bogus, because we use the data to choose the\n", + "range of the prior distribution, and then use the data again to do the\n", + "update. In general, using the same data twice is, in fact, bogus.\n", + "\n", + "But in this case it is ok. Really. We use the data to choose the range\n", + "for the prior, but only to avoid computing a lot of probabilities that\n", + "would have been very small anyway. With `num_stderrs=4`, the range is\n", + "big enough to cover all values with non-negligible likelihood. After\n", + "that, making it bigger has no effect on the results.\n", + "\n", + "In effect, the prior is uniform over all values of mu and\n", + "sigma, but for computational efficiency we ignore all the\n", + "values that don’t matter.\n", + "\n", + "The posterior distribution of CV\n", + "--------------------------------\n", + "\n", + "Once we have the posterior joint distribution of mu and\n", + "sigma, we can compute the distribution of CV for men and\n", + "women, and then the probability that one exceeds the other.\n", + "\n", + "To compute the distribution of CV, we enumerate pairs of mu\n", + "and sigma:\n", + "\n", + " def CoefVariation(suite):\n", + " pmf = thinkbayes.Pmf()\n", + " for (mu, sigma), p in suite.Items():\n", + " pmf.Incr(sigma/mu, p)\n", + " return pmf\n", + "\n", + "Then we use `thinkbayes.PmfProbGreater` to compute the probability that\n", + "men are more variable.\n", + "\n", + "The analysis itself is simple, but there are two more issues we have to\n", + "deal with:\n", + "\n", + "1. As the size of the dataset increases, we run into a series of\n", + " computational problems due to the limitations of floating-point\n", + " arithmetic.\n", + "\n", + "2. The dataset contains a number of extreme values that are almost\n", + " certainly errors. We will need to make the estimation process robust\n", + " in the presence of these outliers.\n", + "\n", + "The following sections explain these problems and their solutions.\n", + "\n", + "Underflow\n", + "---------\n", + "\n", + "If we select the first 100 values from the BRFSS dataset and run the\n", + "analysis I just described, it runs without errors and we get posterior\n", + "distributions that look reasonable.\n", + "\n", + "If we select the first 1000 values and run the program again, we get an\n", + "error in `Pmf.Normalize`:\n", + "\n", + " ValueError: total probability is zero.\n", + "\n", + "The problem is that we are using probability densities to compute\n", + "likelihoods, and densities from continuous distributions tend to be\n", + "small. And if you take 1000 small values and multiply them together, the\n", + "result is very small. In this case it is so small it can’t be\n", + "represented by a floating-point number, so it gets rounded down to zero,\n", + "which is called **underflow**. And if all probabilities in\n", + "the distribution are 0, it’s not a distribution any more.\n", + "\n", + "A possible solution is to renormalize the Pmf after each update, or\n", + "after each batch of 100. That would work, but it would be slow.\n", + "\n", + "A better alternative is to compute likelihoods under a log transform.\n", + "That way, instead of multiplying small values, we can add up log\n", + "likelihoods. Pmf provides methods Log,\n", + "LogUpdateSet and Exp to make this process\n", + "easy.\n", + "\n", + "Log computes the log of the probabilities in a Pmf:\n", + "\n", + " # class Pmf\n", + "\n", + " def Log(self):\n", + " m = self.MaxLike()\n", + " for x, p in self.d.iteritems():\n", + " if p:\n", + " self.Set(x, math.log(p/m))\n", + " else:\n", + " self.Remove(x)\n", + "\n", + "Before applying the log transform Log uses\n", + "MaxLike to find m, the highest probability in\n", + "the Pmf. It divide all probabilities by m, so the highest\n", + "probability gets normalized to 1, which yields a log of 0. The other log\n", + "probabilities are all negative. If there are any values in the Pmf with\n", + "probability 0, they are removed.\n", + "\n", + "While the Pmf is under a log transform, we can’t use\n", + "Update, UpdateSet, or Normalize.\n", + "The result would be nonsensical; if you try, Pmf raises an exception.\n", + "Instead, we have to use LogUpdate and\n", + "LogUpdateSet.\n", + "\n", + "Here’s the implementation of LogUpdateSet:\n", + "\n", + " # class Suite\n", + "\n", + " def LogUpdateSet(self, dataset):\n", + " for data in dataset:\n", + " self.LogUpdate(data)\n", + "\n", + "LogUpdateSet loops through the data and calls\n", + "LogUpdate:\n", + "\n", + " # class Suite\n", + "\n", + " def LogUpdate(self, data):\n", + " for hypo in self.Values():\n", + " like = self.LogLikelihood(data, hypo)\n", + " self.Incr(hypo, like)\n", + "\n", + "LogUpdate is just like Update except that it\n", + "calls LogLikelihood instead of Likelihood, and\n", + "Incr instead of Mult.\n", + "\n", + "Using log-likelihoods avoids the problem with underflow, but while the\n", + "Pmf is under the log transform, there’s not much we can do with it. We\n", + "have to use Exp to invert the transform:\n", + "\n", + " # class Pmf\n", + "\n", + " def Exp(self):\n", + " m = self.MaxLike()\n", + " for x, p in self.d.iteritems():\n", + " self.Set(x, math.exp(p-m))\n", + "\n", + "If the log-likelihoods are large negative numbers, the resulting\n", + "likelihoods might underflow. So Exp finds the maximum\n", + "log-likelihood, m, and shifts all the likelihoods up by\n", + "m. The resulting distribution has a maximum likelihood of\n", + "1. This process inverts the log transform with minimal loss of\n", + "precision.\n", + "\n", + "Log-likelihood\n", + "--------------\n", + "\n", + "Now all we need is LogLikelihood.\n", + "\n", + " # class Height\n", + "\n", + " def LogLikelihood(self, data, hypo):\n", + " x = data\n", + " mu, sigma = hypo\n", + " loglike = scipy.stats.norm.logpdf(x, mu, sigma)\n", + " return loglike\n", + "\n", + "norm.logpdf computes the log-likelihood of the Gaussian\n", + "PDF.\n", + "\n", + "Here’s what the whole update process looks like:\n", + "\n", + " suite.Log()\n", + " suite.LogUpdateSet(xs)\n", + " suite.Exp()\n", + " suite.Normalize()\n", + "\n", + "To review, Log puts the suite under a log transform.\n", + "LogUpdateSet calls LogUpdate, which calls\n", + "LogLikelihood. LogUpdate uses\n", + "Pmf.Incr, because adding a log-likelihood is the same as\n", + "multiplying by a likelihood.\n", + "\n", + "After the update, the log-likelihoods are large negative numbers, so\n", + "Exp shifts them up before inverting the transform, which is\n", + "how we avoid underflow.\n", + "\n", + "Once the suite is transformed back, the probabilities are “linear”\n", + "again, which means “not logarithmic”, so we can use\n", + "Normalize again.\n", + "\n", + "Using this algorithm, we can process the entire dataset without\n", + "underflow, but it is still slow. On my computer it might take an hour.\n", + "We can do better.\n", + "\n", + "A little optimization\n", + "---------------------\n", + "\n", + "This section uses math and computational optimization to speed things up\n", + "by a factor of 100. But the following section presents an algorithm that\n", + "is even faster. So if you want to get right to the good stuff, feel free\n", + "to skip this section.\n", + "\n", + "Suite.LogUpdateSet calls LogUpdate once for\n", + "each data point. We can speed it up by computing the log-likelihood of\n", + "the entire dataset at once.\n", + "\n", + "We’ll start with the Gaussian PDF:\n", + "$$\\frac{1}{\\sigma \\sqrt{2 \\pi}} \\exp \\left[ -\\frac{1}{2} \\left( \\frac{x-\\mu}{\\sigma} \\right)^2 \\right]$$\n", + "and compute the log (dropping the constant term):\n", + "$$-\\log \\sigma -\\frac{1}{2} \\left( \\frac{x-\\mu}{\\sigma} \\right)^2$$\n", + "Given a sequence of values, $x_i$, the total log-likelihood is\n", + "$$\\sum_i -\\log \\sigma - \\frac{1}{2} \\left( \\frac{x_i-\\mu}{\\sigma} \\right)^2$$\n", + "Pulling out the terms that don’t depend on $i$, we get\n", + "$$-n \\log \\sigma - \\frac{1}{2 \\sigma^2} \\sum_i (x_i - \\mu)^2$$ which we\n", + "can translate into Python:\n", + "\n", + " # class Height\n", + "\n", + " def LogUpdateSetFast(self, data):\n", + " xs = tuple(data)\n", + " n = len(xs)\n", + "\n", + " for hypo in self.Values():\n", + " mu, sigma = hypo\n", + " total = Summation(xs, mu)\n", + " loglike = -n * math.log(sigma) - total / 2 / sigma**2\n", + " self.Incr(hypo, loglike)\n", + "\n", + "By itself, this would be a small improvement, but it creates an\n", + "opportunity for a bigger one. Notice that the summation only depends on\n", + "mu, not sigma, so we only have to compute it\n", + "once for each value of mu.\n", + "\n", + "To avoid recomputing, I factor out a function that computes the\n", + "summation, and **memoize** it so it stores previously\n", + "computed results in a dictionary (see\n", + "):\n", + "\n", + " def Summation(xs, mu, cache={}):\n", + " try:\n", + " return cache[xs, mu]\n", + " except KeyError:\n", + " ds = [(x-mu)**2 for x in xs]\n", + " total = sum(ds)\n", + " cache[xs, mu] = total\n", + " return total\n", + "\n", + "cache stores previously computed sums. The try\n", + "statement returns a result from the cache if possible; otherwise it\n", + "computes the summation, then caches and returns the result.\n", + "\n", + "The only catch is that we can’t use a list as a key in the cache,\n", + "because it is not a hashable type. That’s why\n", + "LogUpdateSetFast converts the dataset to a tuple.\n", + "\n", + "This optimization speeds up the computation by about a factor of 100,\n", + "processing the entire dataset (154 407 men and 254 722 women) in less\n", + "than a minute on my not-very-fast computer.\n", + "\n", + "ABC\n", + "---\n", + "\n", + "But maybe you don’t have that kind of time. In that case, Approximate\n", + "Bayesian Computation (ABC) might be the way to go. The motivation behind\n", + "ABC is that the likelihood of any particular dataset is:\n", + "\n", + "1. Very small, especially for large datasets, which is why we had to\n", + " use the log transform,\n", + "\n", + "2. Expensive to compute, which is why we had to do so much\n", + " optimization, and\n", + "\n", + "3. Not really what we want anyway.\n", + "\n", + "We don’t really care about the likelihood of seeing the exact dataset we\n", + "saw. Especially for continuous variables, we care about the likelihood\n", + "of seeing any dataset like the one we saw.\n", + "\n", + "For example, in the Euro problem, we don’t care about the order of the\n", + "coin flips, only the total number of heads and tails. And in the\n", + "locomotive problem, we don’t care about which particular trains were\n", + "seen, only the number of trains and the maximum of the serial numbers.\n", + "\n", + "Similarly, in the BRFSS sample, we don’t really want to know the\n", + "probability of seeing one particular set of values (especially since\n", + "there are hundreds of thousands of them). It is more relevant to ask,\n", + "“If we sample 100,000 people from a population with hypothetical values\n", + "of $\\mu$ and $\\sigma$, what would be the chance of collecting a sample\n", + "with the observed mean and variance?”\n", + "\n", + "For samples from a Gaussian distribution, we can answer this question\n", + "efficiently because we can find the distribution of the sample\n", + "statistics analytically. In fact, we already did it when we computed the\n", + "range of the prior.\n", + "\n", + "If you draw $n$ values from a Gaussian distribution with parameters\n", + "$\\mu$ and $\\sigma$, and compute the sample mean, $m$, the distribution\n", + "of $m$ is Gaussian with parameters $\\mu$ and $\\sigma / \\sqrt{n}$.\n", + "\n", + "Similarly, the distribution of the sample standard deviation, $s$, is\n", + "Gaussian with parameters $\\sigma$ and $\\sigma / \\sqrt{2 (n-1)}$.\n", + "\n", + "We can use these sample distributions to compute the likelihood of the\n", + "sample statistics, $m$ and $s$, given hypothetical values for $\\mu$ and\n", + "$\\sigma$. Here’s a new version of `LogUpdateSet` that does it:\n", + "\n", + " def LogUpdateSetABC(self, data):\n", + " xs = data\n", + " n = len(xs)\n", + "\n", + " # compute sample statistics\n", + " m = numpy.mean(xs)\n", + " s = numpy.std(xs)\n", + "\n", + " for hypo in sorted(self.Values()):\n", + " mu, sigma = hypo\n", + "\n", + " # compute log likelihood of m, given hypo\n", + " stderr_m = sigma / math.sqrt(n)\n", + " loglike = EvalGaussianLogPdf(m, mu, stderr_m)\n", + "\n", + " #compute log likelihood of s, given hypo\n", + " stderr_s = sigma / math.sqrt(2 * (n-1))\n", + " loglike += EvalGaussianLogPdf(s, sigma, stderr_s)\n", + "\n", + " self.Incr(hypo, loglike)\n", + "\n", + "On my computer this function processes the entire dataset in about a\n", + "second, and the result agrees with the exact result with about 5 digits\n", + "of precision.\n", + "\n", + "Robust estimation\n", + "-----------------\n", + "\n", + "![Contour plot of the posterior joint distribution of mean and standard\n", + "deviation of height for men in the\n", + "U.S.](figs/variability_posterior_male.pdf)\n", + "\n", + "[fig.variability1]\n", + "\n", + "![Contour plot of the posterior joint distribution of mean and standard\n", + "deviation of height for women in the\n", + "U.S.](figs/variability_posterior_female.pdf)\n", + "\n", + "[fig.variability2]\n", + "\n", + "We are almost ready to look at results, but we have one more problem to\n", + "deal with. There are a number of outliers in this dataset that are\n", + "almost certainly errors. For example, there are three adults with\n", + "reported height of 61 cm, which would place them among the shortest\n", + "living adults in the world. At the other end, there are four women with\n", + "reported height 229 cm, just short of the tallest women in the world.\n", + "\n", + "It is not impossible that these values are correct, but it is unlikely,\n", + "which makes it hard to know how to deal with them. And we have to get it\n", + "right, because these extreme values have a disproportionate effect on\n", + "the estimated variability.\n", + "\n", + "Because ABC is based on summary statistics, rather than the entire\n", + "dataset, we can make it more robust by choosing summary statistics that\n", + "are robust in the presence of outliers. For example, rather than use the\n", + "sample mean and standard deviation, we could use the median and\n", + "inter-quartile range (IQR), which is the difference between the 25th and\n", + "75th percentiles.\n", + "\n", + "More generally, we could compute an inter-percentile range (IPR) that\n", + "spans any given fraction of the distribution, p:\n", + "\n", + " def MedianIPR(xs, p):\n", + " cdf = thinkbayes.MakeCdfFromList(xs)\n", + " median = cdf.Percentile(50)\n", + "\n", + " alpha = (1-p) / 2\n", + " ipr = cdf.Value(1-alpha) - cdf.Value(alpha)\n", + " return median, ipr\n", + "\n", + "xs is a sequence of values. p is the desired\n", + "range; for example, p=0.5 yields the inter-quartile range.\n", + "\n", + "MedianIPR works by computing the CDF of xs,\n", + "then extracting the median and the difference between two percentiles.\n", + "\n", + "We can convert from ipr to an estimate of\n", + "sigma using the Gaussian CDF to compute the fraction of the\n", + "distribution covered by a given number of standard deviations. For\n", + "example, it is a well-known rule of thumb that 68% of a Gaussian\n", + "distribution falls within one standard deviation of the mean, which\n", + "leaves 16% in each tail. If we compute the range between the 16th and\n", + "84th percentiles, we expect the result to be 2 \\* sigma. So\n", + "we can estimate sigma by computing the 68% IPR and dividing\n", + "by 2.\n", + "\n", + "More generally we could use any number of sigmas.\n", + "MedianS performs the more general version of this\n", + "computation:\n", + "\n", + " def MedianS(xs, num_sigmas):\n", + " half_p = thinkbayes.StandardGaussianCdf(num_sigmas) - 0.5\n", + "\n", + " median, ipr = MedianIPR(xs, half_p * 2)\n", + " s = ipr / 2 / num_sigmas\n", + "\n", + " return median, s\n", + "\n", + "Again, xs is the sequence of values; `num_sigmas` is the\n", + "number of standard deviations the results should be based on. The result\n", + "is median, which estimates $\\mu$, and s, which\n", + "estimates $\\sigma$.\n", + "\n", + "Finally, in LogUpdateSetABC we can replace the sample mean\n", + "and standard deviation with median and s. And\n", + "that pretty much does it.\n", + "\n", + "It might seem odd that we are using observed percentiles to estimate\n", + "$\\mu$ and $\\sigma$, but it is an example of the flexibility of the\n", + "Bayesian approach. In effect we are asking, “Given hypothetical values\n", + "for $\\mu$ and $\\sigma$, and a sampling process that has some chance of\n", + "introducing errors, what is the likelihood of generating a given set of\n", + "sample statistics?”\n", + "\n", + "We are free to choose any sample statistics we like, up to a point:\n", + "$\\mu$ and $\\sigma$ determine the location and spread of a distribution,\n", + "so we need to choose statistics that capture those characteristics. For\n", + "example, if we chose the 49th and 51st percentiles, we would get very\n", + "little information about spread, so it would leave the estimate of\n", + "$\\sigma$ relatively unconstrained by the data. All values of\n", + "sigma would have nearly the same likelihood of producing\n", + "the observed values, so the posterior distribution of sigma\n", + "would look a lot like the prior.\n", + "\n", + "Who is more variable?\n", + "---------------------\n", + "\n", + "![Posterior distributions of CV for men and women, based on robust\n", + "estimators.](figs/variability_cv.pdf)\n", + "\n", + "[fig.variability3]\n", + "\n", + "Finally we are ready to answer the question we started with: is the\n", + "coefficient of variation greater for men than for women?\n", + "\n", + "Using ABC based on the median and IPR with `num_sigmas=1`, I computed\n", + "posterior joint distributions for mu and\n", + "sigma. Figures [fig.variability1] and  [fig.variability2]\n", + "show the results as a contour plot with mu on the x-axis,\n", + "sigma on the y-axis, and probability on the z-axis.\n", + "\n", + "For each joint distribution, I computed the posterior distribution of\n", + "CV. Figure [fig.variability3] shows these distributions for men and\n", + "women. The mean for men is 0.0410; for women it is 0.0429. Since there\n", + "is no overlap between the distributions, we conclude with near certainty\n", + "that women are more variable in height than men.\n", + "\n", + "So is that the end of the Variability Hypothesis? Sadly, no. It turns\n", + "out that this result depends on the choice of the inter-percentile\n", + "range. With `num_sigmas=1`, we conclude that women are more variable,\n", + "but with `num_sigmas=2` we conclude with equal confidence that men are\n", + "more variable.\n", + "\n", + "The reason for the difference is that there are more men of short\n", + "stature, and their distance from the mean is greater.\n", + "\n", + "So our evaluation of the Variability Hypothesis depends on the\n", + "interpretation of “variability.” With `num_sigmas=1` we focus on people\n", + "near the mean. As we increase `num_sigmas`, we give more weight to the\n", + "extremes.\n", + "\n", + "To decide which emphasis is appropriate, we would need a more precise\n", + "statement of the hypothesis. As it is, the Variability Hypothesis may be\n", + "too vague to evaluate.\n", + "\n", + "Nevertheless, it helped me demonstrate several new ideas and, I hope you\n", + "agree, it makes an interesting example.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "There are two ways you might think of ABC. One interpretation is that it\n", + "is, as the name suggests, an approximation that is faster to compute\n", + "than the exact value.\n", + "\n", + "But remember that Bayesian analysis is always based on modeling\n", + "decisions, which implies that there is no “exact” solution. For any\n", + "interesting physical system there are many possible models, and each\n", + "model yields different results. To interpret the results, we have to\n", + "evaluate the models.\n", + "\n", + "So another interpretation of ABC is that it represents an alternative\n", + "model of the likelihood. When we compute $\\mathrm{p}(D|H)$,\n", + "we are asking “What is the likelihood of the data under a given\n", + "hypothesis?”\n", + "\n", + "For large datasets, the likelihood of the data is very small, which is a\n", + "hint that we might not be asking the right question. What we really want\n", + "to know is the likelihood of any outcome like the data, where the\n", + "definition of “like” is yet another modeling decision.\n", + "\n", + "The underlying idea of ABC is that two datasets are alike if they yield\n", + "the same summary statistics. But in some cases, like the example in this\n", + "chapter, it is not obvious which summary statistics to choose.\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "An “effect size” is a statistic intended to measure the difference\n", + "between two groups (see ).\n", + "\n", + "For example, we could use data from the BRFSS to estimate the difference\n", + "in height between men and women. By sampling values from the posterior\n", + "distributions of $\\mu$ and $\\sigma$, we could generate the posterior\n", + "distribution of this difference.\n", + "\n", + "But it might be better to use a dimensionless measure of effect size,\n", + "rather than a difference measured in cm. One option is to use divide\n", + "through by the standard deviation (similar to what we did with the\n", + "coefficient of variation).\n", + "\n", + "If the parameters for Group 1 are $(\\mu_1, \\sigma_1)$, and the\n", + "parameters for Group 2 are $(\\mu_2, \\sigma_2)$, the dimensionless effect\n", + "size is $$\\frac{\\mu_1 - \\mu_2}{(\\sigma_1 + \\sigma_2)/2}$$ Write a\n", + "function that takes joint distributions of mu and\n", + "sigma for two groups and returns the posterior distribution\n", + "of effect size.\n", + "\n", + "Hint: if enumerating all pairs from the two distributions takes too\n", + "long, consider random sampling.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/11_Hypothesis_Testing.ipynb b/notebooks/11_Hypothesis_Testing.ipynb new file mode 100644 index 0000000..fdeab1f --- /dev/null +++ b/notebooks/11_Hypothesis_Testing.ipynb @@ -0,0 +1,285 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Hypothesis Testing\n", + "==================\n", + "\n", + "Back to the Euro problem\n", + "------------------------\n", + "\n", + "In Section [euro] I presented a problem from MacKay’s *Information\n", + "Theory, Inference, and Learning Algorithms*:\n", + "\n", + "> A statistical statement appeared in \\`\\`The Guardian\" on Friday\n", + "> January 4, 2002:\n", + ">\n", + "> > When spun on edge 250 times, a Belgian one-euro coin came up heads\n", + "> > 140 times and tails 110. ‘It looks very suspicious to me,’ said\n", + "> > Barry Blight, a statistics lecturer at the London School of\n", + "> > Economics. ‘If the coin were unbiased, the chance of getting a\n", + "> > result as extreme as that would be less than 7%.’\n", + ">\n", + "> But do these data give evidence that the coin is biased rather than\n", + "> fair?\n", + "\n", + "We estimated the probability that the coin would land face up, but we\n", + "didn’t really answer MacKay’s question: Do the data give evidence that\n", + "the coin is biased?\n", + "\n", + "In Chapter [more] I proposed that data are in favor of a hypothesis if\n", + "the data are more likely under the hypothesis than under the alternative\n", + "or, equivalently, if the Bayes factor is greater than 1.\n", + "\n", + "In the Euro example, we have two hypotheses to consider: I’ll use $F$\n", + "for the hypothesis that the coin is fair and $B$ for the hypothesis that\n", + "it is biased.\n", + "\n", + "If the coin is fair, it is easy to compute the likelihood of the data,\n", + "$\\mathrm{p}(D|F)$. In fact, we already wrote the function\n", + "that does it.\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = hypo / 100.0\n", + " head, tails = data\n", + " like = x**heads * (1-x)**tails\n", + " return like\n", + "\n", + "To use it we can create a Euro suite and invoke\n", + "Likelihood:\n", + "\n", + " suite = Euro()\n", + " likelihood = suite.Likelihood(data, 50)\n", + "\n", + "$\\mathrm{p}(D|F)$ is $5.5 \\cdot 10^{-76}$, which doesn’t\n", + "tell us much except that the probability of seeing any particular\n", + "dataset is very small. It takes two likelihoods to make a ratio, so we\n", + "also have to compute $\\mathrm{p}(D|B)$.\n", + "\n", + "It is not obvious how to compute the likelihood of $B$, because it’s not\n", + "obvious what “biased” means.\n", + "\n", + "One possibility is to cheat and look at the data before we define the\n", + "hypothesis. In that case we would say that “biased” means that the\n", + "probability of heads is 140/250.\n", + "\n", + " actual_percent = 100.0 * 140 / 250\n", + " likelihood = suite.Likelihood(data, actual_percent)\n", + "\n", + "This version of $B$ I call `B_cheat`; the likelihood of `b_cheat` is\n", + "$34 \\cdot 10^{-76}$ and the likelihood ratio is 6.1. So we would say\n", + "that the data are evidence in favor of this version of $B$.\n", + "\n", + "But using the data to formulate the hypothesis is obviously bogus. By\n", + "that definition, any dataset would be evidence in favor of $B$, unless\n", + "the observed percentage of heads is exactly 50%.\n", + "\n", + "Making a fair comparison {#suitelike}\n", + "------------------------\n", + "\n", + "To make a legitimate comparison, we have to define $B$ without looking\n", + "at the data. So let’s try a different definition. If you inspect a\n", + "Belgian Euro coin, you might notice that the “heads” side is more\n", + "prominent than the “tails” side. You might expect the shape to have some\n", + "effect on $x$, but be unsure whether it makes heads more or less likely.\n", + "So you might say “I think the coin is biased so that $x$ is either 0.6\n", + "or 0.4, but I am not sure which.”\n", + "\n", + "We can think of this version, which I’ll call `B_two` as a hypothesis\n", + "made up of two sub-hypotheses. We can compute the likelihood for each\n", + "sub-hypothesis and then compute the average likelihood.\n", + "\n", + " like40 = suite.Likelihood(data, 40)\n", + " like60 = suite.Likelihood(data, 60)\n", + " likelihood = 0.5 * like40 + 0.5 * like60\n", + "\n", + "The likelihood ratio (or Bayes factor) for `b_two` is 1.3, which means\n", + "the data provide weak evidence in favor of `b_two`.\n", + "\n", + "More generally, suppose you suspect that the coin is biased, but you\n", + "have no clue about the value of $x$. In that case you might build a\n", + "Suite, which I call `b_uniform`, to represent sub-hypotheses from 0 to\n", + "100.\n", + "\n", + " b_uniform = Euro(xrange(0, 101))\n", + " b_uniform.Remove(50)\n", + " b_uniform.Normalize()\n", + "\n", + "I initialize `b_uniform` with values from 0 to 100. I removed the\n", + "sub-hypothesis that $x$ is 50%, because if $x$ is 50% the coin is fair,\n", + "but it has almost no effect on the result whether you remove it or not.\n", + "\n", + "To compute the likelihood of `b_uniform` we compute the likelihood of\n", + "each sub-hypothesis and accumulate a weighted average.\n", + "\n", + " def SuiteLikelihood(suite, data):\n", + " total = 0\n", + " for hypo, prob in suite.Items():\n", + " like = suite.Likelihood(data, hypo)\n", + " total += prob * like\n", + " return total\n", + "\n", + "The likelihood ratio for `b_uniform` is 0.47, which means that the data\n", + "are weak evidence against `b_uniform`, compared to $F$.\n", + "\n", + "If you think about the computation performed by `SuiteLikelihood`, you\n", + "might notice that it is similar to an update. To refresh your memory,\n", + "here’s the Update function:\n", + "\n", + " def Update(self, data):\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " return self.Normalize()\n", + "\n", + "And here’s Normalize:\n", + "\n", + " def Normalize(self):\n", + " total = self.Total()\n", + " \n", + " factor = 1.0 / total\n", + " for x in self.d:\n", + " self.d[x] *= factor\n", + "\n", + " return total\n", + "\n", + "The return value from Normalize is the total of the\n", + "probabilities in the Suite, which is the average of the likelihoods for\n", + "the sub-hypotheses, weighted by the prior probabilities. And\n", + "Update passes this value along, so instead of using\n", + "SuiteLikelihood, we could compute the likelihood of\n", + "`b_uniform` like this:\n", + "\n", + " likelihood = b_uniform.Update(data)\n", + "\n", + "The triangle prior\n", + "------------------\n", + "\n", + "In Chapter [more] we also considered a triangle-shaped prior that gives\n", + "higher probability to values of $x$ near 50%. If we think of this prior\n", + "as a suite of sub-hypotheses, we can compute its likelihood like this:\n", + "\n", + " b_triangle = TrianglePrior()\n", + " likelihood = b_triangle.Update(data)\n", + "\n", + "The likelihood ratio for `b_triangle` is 0.84, compared to $F$, so again\n", + "we would say that the data are weak evidence against $B$.\n", + "\n", + "The following table shows the priors we have considered, the likelihood\n", + "of each, and the likelihood ratio (or Bayes factor) relative to $F$.\n", + "\n", + " -------------- ------------------- --------\n", + " Hypothesis Likelihood Bayes\n", + " $\\times 10^{-76}$ Factor\n", + " $F$ 5.5 –\n", + " `B_cheat` 34 6.1\n", + " `B_two` 7.4 1.3\n", + " `B_uniform` 2.6 0.47\n", + " `B_triangle` 4.6 0.84\n", + " -------------- ------------------- --------\n", + "\n", + "Depending on which definition we choose, the data might provide evidence\n", + "for or against the hypothesis that the coin is biased, but in either\n", + "case it is relatively weak evidence.\n", + "\n", + "In summary, we can use Bayesian hypothesis testing to compare the\n", + "likelihood of $F$ and $B$, but we have to do some work to specify\n", + "precisely what $B$ means. This specification depends on background\n", + "information about coins and their behavior when spun, so people could\n", + "reasonably disagree about the right definition.\n", + "\n", + "My presentation of this example follows David MacKay’s discussion, and\n", + "comes to the same conclusion. You can download the code I used in this\n", + "chapter from . For more information see\n", + "Section [download].\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "The Bayes factor for `B_uniform` is 0.47, which means that the data\n", + "provide evidence against this hypothesis, compared to $F$. In the\n", + "previous section I characterized this evidence as “weak,” but didn’t say\n", + "why.\n", + "\n", + "Part of the answer is historical. Harold Jeffreys, an early proponent of\n", + "Bayesian statistics, suggested a scale for interpreting Bayes factors:\n", + "\n", + " ---------- -------------------------\n", + " Bayes Strength\n", + " Factor \n", + " 1 – 3 Barely worth mentioning\n", + " 3 – 10 Substantial\n", + " 10 – 30 Strong\n", + " 30 – 100 Very strong\n", + " $>$ 100 Decisive\n", + " ---------- -------------------------\n", + "\n", + "In the example, the Bayes factor is 0.47 in favor of `B_uniform`, so it\n", + "is 2.1 in favor is $F$, which Jeffreys would consider “barely worth\n", + "mentioning.” Other authors have suggested variations on the wording. To\n", + "avoid arguing about adjectives, we could think about odds instead.\n", + "\n", + "If your prior odds are 1:1, and you see evidence with Bayes factor 2,\n", + "your posterior odds are 2:1. In terms of probability, the data changed\n", + "your degree of belief from 50% to 66%. For most real world problems,\n", + "that change would be small relative to modeling errors and other sources\n", + "of uncertainty.\n", + "\n", + "On the other hand, if you had seen evidence with Bayes factor 100, your\n", + "posterior odds would be 100:1 or more than 99%. Whether or not you agree\n", + "that such evidence is “decisive,” it is certainly strong.\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "Some people believe in the existence of extra-sensory perception (ESP);\n", + "for example, the ability of some people to guess the value of an unseen\n", + "playing card with probability better than chance.\n", + "\n", + "What is your prior degree of belief in this kind of ESP? Do you think it\n", + "is as likely to exist as not? Or are you more skeptical about it? Write\n", + "down your prior odds.\n", + "\n", + "Now compute the strength of the evidence it would take to convince you\n", + "that ESP is at least 50% likely to exist. What Bayes factor would be\n", + "needed to make you 90% sure that ESP exists?\n", + "\n", + "Suppose that your answer to the previous question is 1000; that is,\n", + "evidence with Bayes factor 1000 in favor of ESP would be sufficient to\n", + "change your mind.\n", + "\n", + "Now suppose that you read a paper in a respectable peer-reviewed\n", + "scientific journal that presents evidence with Bayes factor 1000 in\n", + "favor of ESP. Would that change your mind?\n", + "\n", + "If not, how do you resolve the apparent contradiction? You might find it\n", + "helpful to read about David Hume’s article, “Of Miracles,” at\n", + ".\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/12_Evidence.ipynb b/notebooks/12_Evidence.ipynb new file mode 100644 index 0000000..97bf6fd --- /dev/null +++ b/notebooks/12_Evidence.ipynb @@ -0,0 +1,555 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Evidence\n", + "========\n", + "\n", + "Interpreting SAT scores\n", + "-----------------------\n", + "\n", + "Suppose you are the Dean of Admission at a small engineering college in\n", + "Massachusetts, and you are considering two candidates, Alice and Bob,\n", + "whose qualifications are similar in many ways, with the exception that\n", + "Alice got a higher score on the Math portion of the SAT, a standardized\n", + "test intended to measure preparation for college-level work in\n", + "mathematics.\n", + "\n", + "If Alice got 780 and Bob got a 740 (out of a possible 800), you might\n", + "want to know whether that difference is evidence that Alice is better\n", + "prepared than Bob, and what the strength of that evidence is.\n", + "\n", + "Now in reality, both scores are very good, and both candidates are\n", + "probably well prepared for college math. So the real Dean of Admission\n", + "would probably suggest that we choose the candidate who best\n", + "demonstrates the other skills and attitudes we look for in students. But\n", + "as an example of Bayesian hypothesis testing, let’s stick with a\n", + "narrower question: “How strong is the evidence that Alice is better\n", + "prepared than Bob?”\n", + "\n", + "To answer that question, we need to make some modeling decisions. I’ll\n", + "start with a simplification I know is wrong; then we’ll come back and\n", + "improve the model. I pretend, temporarily, that all SAT questions are\n", + "equally difficult. Actually, the designers of the SAT choose questions\n", + "with a range of difficulty, because that improves the ability to measure\n", + "statistical differences between test-takers.\n", + "\n", + "But if we choose a model where all questions are equally difficult, we\n", + "can define a characteristic, `p_correct`, for each test-taker, which is\n", + "the probability of answering any question correctly. This simplification\n", + "makes it easy to compute the likelihood of a given score.\n", + "\n", + "The scale\n", + "---------\n", + "\n", + "In order to understand SAT scores, we have to understand the scoring and\n", + "scaling process. Each test-taker gets a raw score based on the number of\n", + "correct and incorrect questions. The raw score is converted to a scaled\n", + "score in the range 200–800.\n", + "\n", + "In 2009, there were 54 questions on the math SAT. The raw score for each\n", + "test-taker is the number of questions answered correctly minus a penalty\n", + "of $1/4$ point for each question answered incorrectly.\n", + "\n", + "The College Board, which administers the SAT, publishes the map from raw\n", + "scores to scaled scores. I have downloaded that data and wrapped it in\n", + "an Interpolator object that provides a forward lookup (from raw score to\n", + "scaled) and a reverse lookup (from scaled score to raw).\n", + "\n", + "You can download the code for this example from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "The prior\n", + "---------\n", + "\n", + "The College Board also publishes the distribution of scaled scores for\n", + "all test-takers. If we convert each scaled score to a raw score, and\n", + "divide by the number of questions, the result is an estimate of\n", + "`p_correct`. So we can use the distribution of raw scores to model the\n", + "prior distribution of `p_correct`.\n", + "\n", + "Here is the code that reads and processes the data:\n", + "\n", + " class Exam(object):\n", + "\n", + " def __init__(self):\n", + " self.scale = ReadScale()\n", + " scores = ReadRanks()\n", + " score_pmf = thinkbayes.MakePmfFromDict(dict(scores))\n", + " self.raw = self.ReverseScale(score_pmf)\n", + " self.prior = DivideValues(raw, 54)\n", + "\n", + "Exam encapsulates the information we have about the exam.\n", + "ReadScale and ReadRanks read files and return\n", + "objects that contain the data: self.scale is the\n", + "Interpolator that converts from raw to scaled scores and\n", + "back; scores is a list of (score, frequency) pairs.\n", + "\n", + "`score_pmf` is the Pmf of scaled scores. self.raw is the\n", + "Pmf of raw scores, and self.prior is the Pmf of\n", + "`p_correct`.\n", + "\n", + "![Prior distribution of p\\_correct for SAT\n", + "test-takers.](figs/sat_prior.pdf)\n", + "\n", + "[fig.satprior]\n", + "\n", + "Figure [fig.satprior] shows the prior distribution of `p_correct`. This\n", + "distribution is approximately Gaussian, but it is compressed at the\n", + "extremes. By design, the SAT has the most power to discriminate between\n", + "test-takers within two standard deviations of the mean, and less power\n", + "outside that range.\n", + "\n", + "For each test-taker, I define a Suite called Sat that\n", + "represents the distribution of `p_correct`. Here’s the definition:\n", + "\n", + " class Sat(thinkbayes.Suite):\n", + "\n", + " def __init__(self, exam, score):\n", + " thinkbayes.Suite.__init__(self)\n", + "\n", + " self.exam = exam\n", + " self.score = score\n", + "\n", + " # start with the prior distribution\n", + " for p_correct, prob in exam.prior.Items():\n", + " self.Set(p_correct, prob)\n", + "\n", + " # update based on an exam score\n", + " self.Update(score)\n", + "\n", + "`__init__` takes an Exam object and a scaled score. It makes a copy of\n", + "the prior distribution and then updates itself based on the exam score.\n", + "\n", + "As usual, we inherit Update from Suite and\n", + "provide Likelihood:\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " p_correct = hypo\n", + " score = data\n", + "\n", + " k = self.exam.Reverse(score)\n", + " n = self.exam.max_score\n", + " like = thinkbayes.EvalBinomialPmf(k, n, p_correct)\n", + " return like\n", + "\n", + "hypo is a hypothetical value of `p_correct`, and\n", + "data is a scaled score.\n", + "\n", + "To keep things simple, I interpret the raw score as the number of\n", + "correct answers, ignoring the penalty for wrong answers. With this\n", + "simplification, the likelihood is given by the binomial distribution,\n", + "which computes the probability of $k$ correct responses out of $n$\n", + "questions.\n", + "\n", + "Posterior\n", + "---------\n", + "\n", + "![Posterior distributions of p\\_correct for Alice and\n", + "Bob.](figs/sat_posteriors_p_corr.pdf)\n", + "\n", + "[fig.satposterior1]\n", + "\n", + "Figure [fig.satposterior1] shows the posterior distributions of\n", + "`p_correct` for Alice and Bob based on their exam scores. We can see\n", + "that they overlap, so it is possible that `p_correct` is actually higher\n", + "for Bob, but it seems unlikely.\n", + "\n", + "Which brings us back to the original question, “How strong is the\n", + "evidence that Alice is better prepared than Bob?” We can use the\n", + "posterior distributions of `p_correct` to answer this question.\n", + "\n", + "To formulate the question in terms of Bayesian hypothesis testing, I\n", + "define two hypotheses:\n", + "\n", + "- $A$: `p_correct` is higher for Alice than for Bob.\n", + "\n", + "- $B$: `p_correct` is higher for Bob than for Alice.\n", + "\n", + "To compute the likelihood of $A$, we can enumerate all pairs of values\n", + "from the posterior distributions and add up the total probability of the\n", + "cases where `p_correct` is higher for Alice than for Bob. And we already\n", + "have a function, `thinkbayes.PmfProbGreater`, that does that.\n", + "\n", + "So we can define a Suite that computes the posterior probabilities of\n", + "$A$ and $B$:\n", + "\n", + " class TopLevel(thinkbayes.Suite):\n", + "\n", + " def Update(self, data):\n", + " a_sat, b_sat = data\n", + "\n", + " a_like = thinkbayes.PmfProbGreater(a_sat, b_sat)\n", + " b_like = thinkbayes.PmfProbLess(a_sat, b_sat)\n", + " c_like = thinkbayes.PmfProbEqual(a_sat, b_sat)\n", + "\n", + " a_like += c_like / 2\n", + " b_like += c_like / 2\n", + "\n", + " self.Mult('A', a_like)\n", + " self.Mult('B', b_like)\n", + "\n", + " self.Normalize()\n", + "\n", + "Usually when we define a new Suite, we inherit Update and\n", + "provide Likelihood. In this case I override\n", + "Update, because it is easier to evaluate the likelihood of\n", + "both hypotheses at the same time.\n", + "\n", + "The data passed to Update are Sat objects that represent\n", + "the posterior distributions of `p_correct`.\n", + "\n", + "`a_like` is the total probability that `p_correct` is higher for Alice;\n", + "`b_like` is that probability that it is higher for Bob.\n", + "\n", + "`c_like` is the probability that they are “equal,” but this equality is\n", + "an artifact of the decision to model `p_correct` with a set of discrete\n", + "values. If we use more values, `c_like` is smaller, and in the extreme,\n", + "if `p_correct` is continuous, `c_like` is zero. So I treat `c_like` as a\n", + "kind of round-off error and split it evenly between `a_like` and\n", + "`b_like`.\n", + "\n", + "Here is the code that creates TopLevel and updates it:\n", + "\n", + " exam = Exam()\n", + " a_sat = Sat(exam, 780)\n", + " b_sat = Sat(exam, 740)\n", + "\n", + " top = TopLevel('AB')\n", + " top.Update((a_sat, b_sat))\n", + " top.Print()\n", + "\n", + "The likelihood of $A$ is 0.79 and the likelihood of $B$ is 0.21. The\n", + "likelihood ratio (or Bayes factor) is 3.8, which means that these test\n", + "scores are evidence that Alice is better than Bob at answering SAT\n", + "questions. If we believed, before seeing the test scores, that $A$ and\n", + "$B$ were equally likely, then after seeing the scores we should believe\n", + "that the probability of $A$ is 79%, which means there is still a 21%\n", + "chance that Bob is actually better prepared.\n", + "\n", + "A better model\n", + "--------------\n", + "\n", + "Remember that the analysis we have done so far is based on the\n", + "simplification that all SAT questions are equally difficult. In reality,\n", + "some are easier than others, which means that the difference between\n", + "Alice and Bob might be even smaller.\n", + "\n", + "But how big is the modeling error? If it is small, we conclude that the\n", + "first model—based on the simplification that all questions are equally\n", + "difficult—is good enough. If it’s large, we need a better model.\n", + "\n", + "In the next few sections, I develop a better model and discover (spoiler\n", + "alert!) that the modeling error is small. So if you are satisfied with\n", + "the simple mode, you can skip to the next chapter. If you want to see\n", + "how the more realistic model works, read on...\n", + "\n", + "- Assume that each test-taker has some degree of\n", + " efficacy, which measures their ability to answer SAT\n", + " questions.\n", + "\n", + "- Assume that each question has some level of difficulty.\n", + "\n", + "- Finally, assume that the chance that a test-taker answers a question\n", + " correctly is related to efficacy and\n", + " difficulty according to this function:\n", + "\n", + " def ProbCorrect(efficacy, difficulty, a=1):\n", + " return 1 / (1 + math.exp(-a * (efficacy - difficulty)))\n", + "\n", + "This function is a simplified version of the curve used in **item\n", + "response theory**, which you can read about at\n", + ".\n", + "efficacy and difficulty are considered to be\n", + "on the same scale, and the probability of getting a question right\n", + "depends only on the difference between them.\n", + "\n", + "When efficacy and difficulty are equal, the\n", + "probability of getting the question right is 50%. As\n", + "efficacy increases, this probability approaches 100%. As it\n", + "decreases (or as difficulty increases), the probability\n", + "approaches 0%.\n", + "\n", + "Given the distribution of efficacy across test-takers and\n", + "the distribution of difficulty across questions, we can\n", + "compute the expected distribution of raw scores. We’ll do that in two\n", + "steps. First, for a person with given efficacy, we’ll\n", + "compute the distribution of raw scores.\n", + "\n", + " def PmfCorrect(efficacy, difficulties):\n", + " pmf0 = thinkbayes.Pmf([0])\n", + "\n", + " ps = [ProbCorrect(efficacy, diff) for diff in difficulties]\n", + " pmfs = [BinaryPmf(p) for p in ps]\n", + " dist = sum(pmfs, pmf0)\n", + " return dist\n", + "\n", + "difficulties is a list of difficulties, one for each\n", + "question. ps is a list of probabilities, and\n", + "pmfs is a list of two-valued Pmf objects; here’s the\n", + "function that makes them:\n", + "\n", + " def BinaryPmf(p):\n", + " pmf = thinkbayes.Pmf()\n", + " pmf.Set(1, p)\n", + " pmf.Set(0, 1-p)\n", + " return pmf\n", + "\n", + "dist is the sum of these Pmfs. Remember from\n", + "Section [addends] that when we add up Pmf objects, the result is the\n", + "distribution of the sums. In order to use Python’s sum to\n", + "add up Pmfs, we have to provide pmf0 which is the identity\n", + "for Pmfs, so pmf + pmf0 is always pmf.\n", + "\n", + "If we know a person’s efficacy, we can compute their distribution of raw\n", + "scores. For a group of people with a different efficacies, the resulting\n", + "distribution of raw scores is a mixture. Here’s the code that computes\n", + "the mixture:\n", + "\n", + " # class Exam:\n", + "\n", + " def MakeRawScoreDist(self, efficacies):\n", + " pmfs = thinkbayes.Pmf()\n", + " for efficacy, prob in efficacies.Items():\n", + " scores = PmfCorrect(efficacy, self.difficulties)\n", + " pmfs.Set(scores, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(pmfs)\n", + " return mix\n", + "\n", + "MakeRawScoreDist takes efficacies, which is a\n", + "Pmf that represents the distribution of efficacy across test-takers. I\n", + "assume it is Gaussian with mean 0 and standard deviation 1.5. This\n", + "choice is mostly arbitrary. The probability of getting a question\n", + "correct depends on the difference between efficacy and difficulty, so we\n", + "can choose the units of efficacy and then calibrate the units of\n", + "difficulty accordingly.\n", + "\n", + "pmfs is a meta-Pmf that contains one Pmf for each level of\n", + "efficacy, and maps to the fraction of test-takers at that level.\n", + "MakeMixture takes the meta-pmf and computes the\n", + "distribution of the mixture (see Section [mixture]).\n", + "\n", + "Calibration\n", + "-----------\n", + "\n", + "If we were given the distribution of difficulty, we could use\n", + "`MakeRawScoreDist` to compute the distribution of raw scores. But for us\n", + "the problem is the other way around: we are given the distribution of\n", + "raw scores and we want to infer the distribution of difficulty.\n", + "\n", + "![Actual distribution of raw scores and a model to fit\n", + "it.](figs/sat_calibrate.pdf)\n", + "\n", + "[fig.satcalibrate]\n", + "\n", + "I assume that the distribution of difficulty is uniform with parameters\n", + "center and width.\n", + "MakeDifficulties makes a list of difficulties with these\n", + "parameters.\n", + "\n", + " def MakeDifficulties(center, width, n):\n", + " low, high = center-width, center+width\n", + " return numpy.linspace(low, high, n)\n", + "\n", + "By trying out a few combinations, I found that center=-0.05\n", + "and width=1.8 yield a distribution of raw scores similar to\n", + "the actual data, as shown in Figure [fig.satcalibrate].\n", + "\n", + "So, assuming that the distribution of difficulty is uniform, its range\n", + "is approximately -1.85 to 1.75, given that\n", + "efficacy is Gaussian with mean 0 and standard deviation 1.5.\n", + "\n", + "The following table shows the range of ProbCorrect for\n", + "test-takers at different levels of efficacy:\n", + "\n", + "|r|r|r|r| &\\\n", + "Efficacy & -1.85 & -0.05 & 1.75\\\n", + "3.00 & 0.99 & 0.95 & 0.78\\\n", + "1.50 & 0.97 & 0.82 & 0.44\\\n", + "0.00 & 0.86 & 0.51 & 0.15\\\n", + "-1.50 & 0.59 & 0.19 & 0.04\\\n", + "-3.00 & 0.24 & 0.05 & 0.01\\\n", + "\n", + "Someone with efficacy 3 (two standard deviations above the mean) has a\n", + "99% chance of answering the easiest questions on the exam, and a 78%\n", + "chance of answering the hardest. On the other end of the range, someone\n", + "two standard deviations below the mean has only a 24% chance of\n", + "answering the easiest questions.\n", + "\n", + "Posterior distribution of efficacy\n", + "----------------------------------\n", + "\n", + "![Posterior distributions of efficacy for Alice and\n", + "Bob.](figs/sat_posteriors_eff.pdf)\n", + "\n", + "[fig.satposterior2]\n", + "\n", + "Now that the model is calibrated, we can compute the posterior\n", + "distribution of efficacy for Alice and Bob. Here is a version of the Sat\n", + "class that uses the new model:\n", + "\n", + " class Sat2(thinkbayes.Suite):\n", + "\n", + " def __init__(self, exam, score):\n", + " self.exam = exam\n", + " self.score = score\n", + "\n", + " # start with the Gaussian prior\n", + " efficacies = thinkbayes.MakeGaussianPmf(0, 1.5, 3)\n", + " thinkbayes.Suite.__init__(self, efficacies)\n", + "\n", + " # update based on an exam score\n", + " self.Update(score)\n", + "\n", + "`Update` invokes `Likelihood`, which computes the likelihood of a given\n", + "test score for a hypothetical level of efficacy.\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " efficacy = hypo\n", + " score = data\n", + " raw = self.exam.Reverse(score)\n", + "\n", + " pmf = self.exam.PmfCorrect(efficacy)\n", + " like = pmf.Prob(raw)\n", + " return like\n", + "\n", + "pmf is the distribution of raw scores for a test-taker with\n", + "the given efficacy; like is the probability of the observed\n", + "score.\n", + "\n", + "Figure [fig.satposterior2] shows the posterior distributions of efficacy\n", + "for Alice and Bob. As expected, the location of Alice’s distribution is\n", + "farther to the right, but again there is some overlap.\n", + "\n", + "Using TopLevel again, we compare $A$, the hypothesis that\n", + "Alice’s efficacy is higher, and $B$, the hypothesis that Bob’s is\n", + "higher. The likelihood ratio is 3.4, a bit smaller than what we got from\n", + "the simple model (3.8). So this model indicates that the data are\n", + "evidence in favor of $A$, but a little weaker than the previous\n", + "estimate.\n", + "\n", + "If our prior belief is that $A$ and $B$ are equally likely, then in\n", + "light of this evidence we would give $A$ a posterior probability of 77%,\n", + "leaving a 23% chance that Bob’s efficacy is higher.\n", + "\n", + "Predictive distribution\n", + "-----------------------\n", + "\n", + "The analysis we have done so far generates estimates for Alice and Bob’s\n", + "efficacy, but since efficacy is not directly observable, it is hard to\n", + "validate the results.\n", + "\n", + "To give the model predictive power, we can use it to answer a related\n", + "question: “If Alice and Bob take the math SAT again, what is the chance\n", + "that Alice will do better again?”\n", + "\n", + "We’ll answer this question in two steps:\n", + "\n", + "- We’ll use the posterior distribution of efficacy to generate a\n", + " predictive distribution of raw score for each test-taker.\n", + "\n", + "- We’ll compare the two predictive distributions to compute the\n", + " probability that Alice gets a higher score again.\n", + "\n", + "We already have most of the code we need. To compute the predictive\n", + "distributions, we can use `MakeRawScoreDist` again:\n", + "\n", + " exam = Exam()\n", + " a_sat = Sat(exam, 780)\n", + " b_sat = Sat(exam, 740)\n", + "\n", + " a_pred = exam.MakeRawScoreDist(a_sat)\n", + " b_pred = exam.MakeRawScoreDist(b_sat)\n", + "\n", + "Then we can find the likelihood that Alice does better on the second\n", + "test, Bob does better, or they tie:\n", + "\n", + " a_like = thinkbayes.PmfProbGreater(a_pred, b_pred)\n", + " b_like = thinkbayes.PmfProbLess(a_pred, b_pred)\n", + " c_like = thinkbayes.PmfProbEqual(a_pred, b_pred)\n", + "\n", + "The probability that Alice does better on the second exam is 63%, which\n", + "means that Bob has a 37% chance of doing as well or better.\n", + "\n", + "Notice that we have more confidence about Alice’s efficacy than we do\n", + "about the outcome of the next test. The posterior odds are 3:1 that\n", + "Alice’s efficacy is higher, but only 2:1 that Alice will do better on\n", + "the next exam.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "![Joint posterior distribution of p\\_correct for Alice and\n", + "Bob.](figs/sat_joint.pdf)\n", + "\n", + "[fig.satjoint]\n", + "\n", + "We started this chapter with the question, “How strong is the evidence\n", + "that Alice is better prepared than Bob?” On the face of it, that sounds\n", + "like we want to test two hypotheses: either Alice is more prepared or\n", + "Bob is.\n", + "\n", + "But in order to compute likelihoods for these hypotheses, we have to\n", + "solve an estimation problem. For each test-taker we have to find the\n", + "posterior distribution of either `p_correct` or `efficacy`.\n", + "\n", + "Values like this are called **nuisance parameters** because\n", + "we don’t care what they are, but we have to estimate them to answer the\n", + "question we care about.\n", + "\n", + "One way to visualize the analysis we did in this chapter is to plot the\n", + "space of these parameters. `thinkbayes.MakeJoint` takes two Pmfs,\n", + "computes their joint distribution, and returns a joint pmf of each\n", + "possible pair of values and its probability.\n", + "\n", + " def MakeJoint(pmf1, pmf2):\n", + " joint = Joint()\n", + " for v1, p1 in pmf1.Items():\n", + " for v2, p2 in pmf2.Items():\n", + " joint.Set((v1, v2), p1 * p2)\n", + " return joint\n", + "\n", + "This function assumes that the two distributions are independent.\n", + "\n", + "Figure [fig.satjoint] shows the joint posterior distribution of\n", + "`p_correct` for Alice and Bob. The diagonal line indicates the part of\n", + "the space where `p_correct` is the same for Alice and Bob. To the right\n", + "of this line, Alice is more prepared; to the left, Bob is more prepared.\n", + "\n", + "In TopLevel.Update, when we compute the likelihoods of $A$\n", + "and $B$, we add up the probability mass on each side of this line. For\n", + "the cells that fall on the line, we add up the total mass and split it\n", + "between $A$ and $B$.\n", + "\n", + "The process we used in this chapter—estimating nuisance parameters in\n", + "order to evaluate the likelihood of competing hypotheses—is a common\n", + "Bayesian approach to problems like this.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/13_Simulation.ipynb b/notebooks/13_Simulation.ipynb new file mode 100644 index 0000000..7f220ce --- /dev/null +++ b/notebooks/13_Simulation.ipynb @@ -0,0 +1,570 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\n", + "Simulation\n", + "==========\n", + "\n", + "In this chapter I describe my solution to a problem posed by a patient\n", + "with a kidney tumor. I think the problem is important and relevant to\n", + "patients with these tumors and doctors treating them.\n", + "\n", + "And I think the solution is interesting because, although it is a\n", + "Bayesian approach to the problem, the use of Bayes’s theorem is\n", + "implicit. I present the solution and my code; at the end of the chapter\n", + "I will explain the Bayesian part.\n", + "\n", + "If you want more technical detail than I present here, you can read my\n", + "paper on this work at .\n", + "\n", + "The Kidney Tumor problem\n", + "------------------------\n", + "\n", + "I am a frequent reader and occasional contributor to the online\n", + "statistics forum at . In November 2011,\n", + "I read the following message:\n", + "\n", + "> “I have Stage IV Kidney Cancer and am trying to determine if the\n", + "> cancer formed before I retired from the military. ... Given the dates\n", + "> of retirement and detection is it possible to determine when there was\n", + "> a 50/50 chance that I developed the disease? Is it possible to\n", + "> determine the probability on the retirement date? My tumor was 15.5 cm\n", + "> x 15 cm at detection. Grade II.”\n", + "\n", + "I contacted the author of the message and got more information; I\n", + "learned that veterans get different benefits if it is “more likely than\n", + "not” that a tumor formed while they were in military service (among\n", + "other considerations).\n", + "\n", + "Because renal tumors grow slowly, and often do not cause symptoms, they\n", + "are sometimes left untreated. As a result, doctors can observe the rate\n", + "of growth for untreated tumors by comparing scans from the same patient\n", + "at different times. Several papers have reported these growth rates.\n", + "\n", + "I collected data from a paper by Zhang et al[^3]. I contacted the\n", + "authors to see if I could get raw data, but they refused on grounds of\n", + "medical privacy. Nevertheless, I was able to extract the data I needed\n", + "by printing one of their graphs and measuring it with a ruler.\n", + "\n", + "![CDF of RDT in doublings per year.](figs/kidney2.pdf)\n", + "\n", + "[fig.kidney2]\n", + "\n", + "They report growth rates in reciprocal doubling time (RDT), which is in\n", + "units of doublings per year. So a tumor with $RDT=1$ doubles in volume\n", + "each year; with $RDT=2$ it quadruples in the same time, and with\n", + "$RDT=-1$, it halves. Figure [fig.kidney2] shows the distribution of RDT\n", + "for 53 patients.\n", + "\n", + "The squares are the data points from the paper; the line is a model I\n", + "fit to the data. The positive tail fits an exponential distribution\n", + "well, so I used a mixture of two exponentials.\n", + "\n", + "A simple model\n", + "--------------\n", + "\n", + "It is usually a good idea to start with a simple model before trying\n", + "something more challenging. Sometimes the simple model is sufficient for\n", + "the problem at hand, and if not, you can use it to validate the more\n", + "complex model.\n", + "\n", + "For my simple model, I assume that tumors grow with a constant doubling\n", + "time, and that they are three-dimensional in the sense that if the\n", + "maximum linear measurement doubles, the volume is multiplied by eight.\n", + "\n", + "I learned from my correspondent that the time between his discharge from\n", + "the military and his diagnosis was 3291 days (about 9 years). So my\n", + "first calculation was, “If this tumor grew at the median rate, how big\n", + "would it have been at the date of discharge?”\n", + "\n", + "The median volume doubling time reported by Zhang et al is 811 days.\n", + "Assuming 3-dimensional geometry, the doubling time for a linear measure\n", + "is three times longer.\n", + "\n", + " # time between discharge and diagnosis, in days \n", + " interval = 3291.0\n", + "\n", + " # doubling time in linear measure is doubling time in volume * 3\n", + " dt = 811.0 * 3\n", + "\n", + " # number of doublings since discharge\n", + " doublings = interval / dt\n", + "\n", + " # how big was the tumor at time of discharge (diameter in cm)\n", + " d1 = 15.5\n", + " d0 = d1 / 2.0 ** doublings\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "The result, d0, is about 6 cm. So if this tumor formed\n", + "after the date of discharge, it must have grown substantially faster\n", + "than the median rate. Therefore I concluded that it is “more likely than\n", + "not” that this tumor formed before the date of discharge.\n", + "\n", + "In addition, I computed the growth rate that would be implied if this\n", + "tumor had formed after the date of discharge. If we assume an initial\n", + "size of 0.1 cm, we can compute the number of doublings to get to a final\n", + "size of 15.5 cm:\n", + "\n", + " # assume an initial linear measure of 0.1 cm\n", + " d0 = 0.1\n", + " d1 = 15.5\n", + "\n", + " # how many doublings would it take to get from d0 to d1\n", + " doublings = log2(d1 / d0)\n", + "\n", + " # what linear doubling time does that imply?\n", + " dt = interval / doublings\n", + "\n", + " # compute the volumetric doubling time and RDT\n", + " vdt = dt / 3\n", + " rdt = 365 / vdt\n", + "\n", + "dt is linear doubling time, so vdt is\n", + "volumetric doubling time, and rdt is reciprocal doubling\n", + "time.\n", + "\n", + "The number of doublings, in linear measure, is 7.3, which implies an RDT\n", + "of 2.4. In the data from Zhang et al, only 20% of tumors grew this fast\n", + "during a period of observation. So again, I concluded that is “more\n", + "likely than not” that the tumor formed prior to the date of discharge.\n", + "\n", + "These calculations are sufficient to answer the question as posed, and\n", + "on behalf of my correspondent, I wrote a letter explaining my\n", + "conclusions to the Veterans’ Benefit Administration.\n", + "\n", + "Later I told a friend, who is an oncologist, about my results. He was\n", + "surprised by the growth rates observed by Zhang et al, and by what they\n", + "imply about the ages of these tumors. He suggested that the results\n", + "might be interesting to researchers and doctors.\n", + "\n", + "But in order to make them useful, I wanted a more general model of the\n", + "relationship between age and size.\n", + "\n", + "A more general model\n", + "--------------------\n", + "\n", + "Given the size of a tumor at time of diagnosis, it would be most useful\n", + "to know the probability that the tumor formed before any given date; in\n", + "other words, the distribution of ages.\n", + "\n", + "To find it, I run simulations of tumor growth to get the distribution of\n", + "size conditioned on age. Then we can use a Bayesian approach to get the\n", + "distribution of age conditioned on size.\n", + "\n", + "The simulation starts with a small tumor and runs these steps:\n", + "\n", + "1. Choose a growth rate from the distribution of RDT.\n", + "\n", + "2. Compute the size of the tumor at the end of an interval.\n", + "\n", + "3. Record the size of the tumor at each interval.\n", + "\n", + "4. Repeat until the tumor exceeds the maximum relevant size.\n", + "\n", + "For the initial size I chose 0.3 cm, because carcinomas smaller than\n", + "that are less likely to be invasive and less likely to have the blood\n", + "supply needed for rapid growth (see\n", + ").\n", + "\n", + "I chose an interval of 245 days (about 8 months) because that is the\n", + "median time between measurements in the data source.\n", + "\n", + "For the maximum size I chose 20 cm. In the data source, the range of\n", + "observed sizes is 1.0 to 12.0 cm, so we are extrapolating beyond the\n", + "observed range at each end, but not by far, and not in a way likely to\n", + "have a strong effect on the results.\n", + "\n", + "![Simulations of tumor growth, size vs. time.](figs/kidney4.pdf)\n", + "\n", + "[fig.kidney4]\n", + "\n", + "The simulation is based on one big simplification: the growth rate is\n", + "chosen independently during each interval, so it does not depend on age,\n", + "size, or growth rate during previous intervals.\n", + "\n", + "In Section [serial] I review these assumptions and consider more\n", + "detailed models. But first let’s look at some examples.\n", + "\n", + "Figure [fig.kidney4] shows the size of simulated tumors as a function of\n", + "age. The dashed line at 10 cm shows the range of ages for tumors at that\n", + "size: the fastest-growing tumor gets there in 8 years; the slowest takes\n", + "more than 35.\n", + "\n", + "I am presenting results in terms of linear measurements, but the\n", + "calculations are in terms of volume. To convert from one to the other,\n", + "again, I use the volume of a sphere with the given diameter.\n", + "\n", + "Implementation\n", + "--------------\n", + "\n", + "Here is the kernel of the simulation:\n", + "\n", + " def MakeSequence(rdt_seq, v0=0.01, interval=0.67, vmax=Volume(20.0)):\n", + " seq = v0,\n", + " age = 0\n", + "\n", + " for rdt in rdt_seq:\n", + " age += interval\n", + " final, seq = ExtendSequence(age, seq, rdt, interval)\n", + " if final > vmax:\n", + " break\n", + "\n", + " return seq\n", + "\n", + "`rdt_seq` is an iterator that yields random values from the CDF of\n", + "growth rate. v0 is the initial volume in mL.\n", + "interval is the time step in years. vmax is\n", + "the final volume corresponding to a linear measurement of 20 cm.\n", + "\n", + "Volume converts from linear measurement in cm to volume in\n", + "mL, based on the simplification that the tumor is a sphere:\n", + "\n", + " def Volume(diameter, factor=4*math.pi/3):\n", + " return factor * (diameter/2.0)**3\n", + "\n", + "ExtendSequence computes the volume of the tumor at the end\n", + "of the interval.\n", + "\n", + " def ExtendSequence(age, seq, rdt, interval):\n", + " initial = seq[-1]\n", + " doublings = rdt * interval\n", + " final = initial * 2**doublings\n", + " new_seq = seq + (final,)\n", + " cache.Add(age, new_seq, rdt)\n", + " \n", + " return final, new_seq\n", + "\n", + "age is the age of the tumor at the end of the interval.\n", + "seq is a tuple that contains the volumes so far.\n", + "rdt is the growth rate during the interval, in doublings\n", + "per year. interval is the size of the time step in years.\n", + "\n", + "The return values are final, the volume of the tumor at the\n", + "end of the interval, and `new_seq`, a new tuple containing the volumes\n", + "in seq plus the new volume final.\n", + "\n", + "Cache.Add records the age and size of each tumor at the end\n", + "of each interval, as explained in the next section.\n", + "\n", + "Caching the joint distribution\n", + "------------------------------\n", + "\n", + "![Joint distribution of age and tumor size.](figs/kidney8.pdf)\n", + "\n", + "[fig.kidney8]\n", + "\n", + "Here’s how the cache works.\n", + "\n", + " class Cache(object):\n", + "\n", + " def __init__(self):\n", + " self.joint = thinkbayes.Joint()\n", + "\n", + "joint is a joint Pmf that records the frequency of each\n", + "age-size pair, so it approximates the joint distribution of age and\n", + "size.\n", + "\n", + "At the end of each simulated interval, ExtendSequence calls\n", + "Add:\n", + "\n", + " # class Cache\n", + "\n", + " def Add(self, age, seq):\n", + " final = seq[-1]\n", + " cm = Diameter(final)\n", + " bucket = round(CmToBucket(cm))\n", + " self.joint.Incr((age, bucket))\n", + "\n", + "Again, age is the age of the tumor, and seq is\n", + "the sequence of volumes so far.\n", + "\n", + "![Distributions of age, conditioned on size.](figs/kidney6.pdf)\n", + "\n", + "[fig.kidney6]\n", + "\n", + "Before adding the new data to the joint distribution, we use\n", + "Diameter to convert from volume to diameter in centimeters:\n", + "\n", + " def Diameter(volume, factor=3/math.pi/4, exp=1/3.0):\n", + " return 2 * (factor * volume) ** exp\n", + "\n", + "And CmToBucket to convert from centimeters to a discrete\n", + "bucket number:\n", + "\n", + " def CmToBucket(x, factor=10):\n", + " return factor * math.log(x)\n", + "\n", + "The buckets are equally spaced on a log scale. Using\n", + "factor=10 yields a reasonable number of buckets; for\n", + "example, 1 cm maps to bucket 0 and 10 cm maps to bucket 23.\n", + "\n", + "After running the simulations, we can plot the joint distribution as a\n", + "pseudocolor plot, where each cell represents the number of tumors\n", + "observed at a given size-age pair. Figure [fig.kidney8] shows the joint\n", + "distribution after 1000 simulations.\n", + "\n", + "Conditional distributions {#conditional-distributions}\n", + "-------------------------\n", + "\n", + "![Percentiles of tumor age as a function of size.](figs/kidney7.pdf)\n", + "\n", + "[fig.kidney7]\n", + "\n", + "By taking a vertical slice from the joint distribution, we can get the\n", + "distribution of sizes for any given age. By taking a horizontal slice,\n", + "we can get the distribution of ages conditioned on size.\n", + "\n", + "Here’s the code that reads the joint distribution and builds the\n", + "conditional distribution for a given size.\n", + "\n", + " # class Cache\n", + "\n", + " def ConditionalCdf(self, bucket):\n", + " pmf = self.joint.Conditional(0, 1, bucket)\n", + " cdf = pmf.MakeCdf()\n", + " return cdf\n", + "\n", + "`bucket` is the integer bucket number corresponding to tumor size.\n", + "Joint.Conditional computes the PMF of age conditioned on\n", + "bucket. The result is the CDF of age conditioned on\n", + "bucket.\n", + "\n", + "Figure [fig.kidney6] shows several of these CDFs, for a range of sizes.\n", + "To summarize these distributions, we can compute percentiles as a\n", + "function of size.\n", + "\n", + " percentiles = [95, 75, 50, 25, 5]\n", + "\n", + " for bucket in cache.GetBuckets():\n", + " cdf = ConditionalCdf(bucket) \n", + " ps = [cdf.Percentile(p) for p in percentiles]\n", + "\n", + "Figure [fig.kidney7] shows these percentiles for each size bucket. The\n", + "data points are computed from the estimated joint distribution. In the\n", + "model, size and time are discrete, which contributes numerical errors,\n", + "so I also show a least squares fit for each sequence of percentiles.\n", + "\n", + "Serial Correlation {#serial}\n", + "------------------\n", + "\n", + "The results so far are based on a number of modeling decisions; let’s\n", + "review them and consider which ones are the most likely sources of\n", + "error:\n", + "\n", + "- To convert from linear measure to volume, we assume that tumors are\n", + " approximately spherical. This assumption is probably fine for tumors\n", + " up to a few centimeters, but not for very large tumors.\n", + "\n", + "- The distribution of growth rates in the simulations are based on a\n", + " continuous model we chose to fit the data reported by Zhang et al,\n", + " which is based on 53 patients. The fit is only approximate and, more\n", + " importantly, a larger sample would yield a different distribution.\n", + "\n", + "- The growth model does not take into account tumor subtype or grade;\n", + " this assumption is consistent with the conclusion of Zhang et al:\n", + " “Growth rates in renal tumors of different sizes, subtypes and\n", + " grades represent a wide range and overlap substantially.” But with a\n", + " larger sample, a difference might become apparent.\n", + "\n", + "- The distribution of growth rate does not depend on the size of the\n", + " tumor. This assumption would not be realistic for very small and\n", + " very large tumors, whose growth is limited by blood supply.\n", + "\n", + " But tumors observed by Zhang et al ranged from 1 to 12 cm, and they\n", + " found no statistically significant relationship between size and\n", + " growth rate. So if there is a relationship, it is likely to be weak,\n", + " at least in this size range.\n", + "\n", + "- In the simulations, growth rate during each interval is independent\n", + " of previous growth rates. In reality it is plausible that tumors\n", + " that have grown quickly in the past are more likely to grow quickly.\n", + " In other words, there is probably a serial correlation in growth\n", + " rate.\n", + "\n", + "Of these, the first and last seem the most problematic. I’ll investigate\n", + "serial correlation first, then come back to spherical geometry.\n", + "\n", + "To simulate correlated growth, I wrote a generator[^4] that yields a\n", + "correlated series from a given Cdf. Here’s how the algorithm works:\n", + "\n", + "1. Generate correlated values from a Gaussian distribution. This is\n", + " easy to do because we can compute the distribution of the next value\n", + " conditioned on the previous value.\n", + "\n", + "2. Transform each value to its cumulative probability using the\n", + " Gaussian CDF.\n", + "\n", + "3. Transform each cumulative probability to the corresponding value\n", + " using the given Cdf.\n", + "\n", + "Here’s what that looks like in code:\n", + "\n", + " def CorrelatedGenerator(cdf, rho):\n", + " x = random.gauss(0, 1)\n", + " yield Transform(x)\n", + "\n", + " sigma = math.sqrt(1 - rho**2); \n", + " while True:\n", + " x = random.gauss(x * rho, sigma)\n", + " yield Transform(x)\n", + "\n", + "cdf is the desired Cdf; rho is the desired\n", + "correlation. The values of x are Gaussian;\n", + "Transform converts them to the desired distribution.\n", + "\n", + "The first value of x is Gaussian with mean 0 and standard\n", + "deviation 1. For subsequent values, the mean and standard deviation\n", + "depend on the previous value. Given the previous x, the\n", + "mean of the next value is x \\* rho, and the variance is\n", + "1 - rho\\*\\*2.\n", + "\n", + "Transform maps from each Gaussian value, x, to\n", + "a value from the given Cdf, y.\n", + "\n", + " def Transform(x):\n", + " p = thinkbayes.GaussianCdf(x)\n", + " y = cdf.Value(p)\n", + " return y\n", + "\n", + "GaussianCdf computes the CDF of the standard Gaussian\n", + "distribution at x, returning a cumulative probability.\n", + "Cdf.Value maps from a cumulative probability to the\n", + "corresponding value in cdf.\n", + "\n", + "Depending on the shape of cdf, information can be lost in\n", + "transformation, so the actual correlation might be lower than\n", + "rho. For example, when I generate 10000 values from the\n", + "distribution of growth rates with rho=0.4, the actual\n", + "correlation is 0.37. But since we are guessing at the right correlation\n", + "anyway, that’s close enough.\n", + "\n", + "Remember that MakeSequence takes an iterator as an\n", + "argument. That interface allows it to work with different generators:\n", + "\n", + " iterator = UncorrelatedGenerator(cdf)\n", + " seq1 = MakeSequence(iterator)\n", + "\n", + " iterator = CorrelatedGenerator(cdf, rho)\n", + " seq2 = MakeSequence(iterator)\n", + "\n", + "In this example, seq1 and seq2 are drawn from\n", + "the same distribution, but the values in seq1 are\n", + "uncorrelated and the values in seq2 are correlated with a\n", + "coefficient of approximately rho.\n", + "\n", + "Now we can see what effect serial correlation has on the results; the\n", + "following table shows percentiles of age for a 6 cm tumor, using the\n", + "uncorrelated generator and a correlated generator with target\n", + "$\\rho = 0.4$.\n", + "\n", + "Correlation makes the fastest growing tumors faster and the slowest\n", + "slower, so the range of ages is wider. The difference is modest for low\n", + "percentiles, but for the 95th percentile it is more than 6 years. To\n", + "compute these percentiles precisely, we would need a better estimate of\n", + "the actual serial correlation.\n", + "\n", + "However, this model is sufficient to answer the question we started\n", + "with: given a tumor with a linear dimension of 15.5 cm, what is the\n", + "probability that it formed more than 8 years ago?\n", + "\n", + "Here’s the code:\n", + "\n", + " # class Cache\n", + "\n", + " def ProbOlder(self, cm, age):\n", + " bucket = CmToBucket(cm)\n", + " cdf = self.ConditionalCdf(bucket)\n", + " p = cdf.Prob(age)\n", + " return 1-p\n", + "\n", + "cm is the size of the tumor; age is the age\n", + "threshold in years. ProbOlder converts size to a bucket\n", + "number, gets the Cdf of age conditioned on bucket, and computes the\n", + "probability that age exceeds the given value.\n", + "\n", + "With no serial correlation, the probability that a 15.5 cm tumor is\n", + "older than 8 years is 0.999, or almost certain. With correlation 0.4,\n", + "faster-growing tumors are more likely, but the probability is still\n", + "0.995. Even with correlation 0.8, the probability is 0.978.\n", + "\n", + "Another likely source of error is the assumption that tumors are\n", + "approximately spherical. For a tumor with linear dimensions 15.5 x 15\n", + "cm, this assumption is probably not valid. If, as seems likely, a tumor\n", + "this size is relatively flat, it might have the same volume as a 6 cm\n", + "sphere. With this smaller volume and correlation 0.8, the probability of\n", + "age greater than 8 is still 95%.\n", + "\n", + "So even taking into account modeling errors, it is unlikely that such a\n", + "large tumor could have formed less than 8 years prior to the date of\n", + "diagnosis.\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "Well, we got through a whole chapter without using Bayes’s theorem or\n", + "the Suite class that encapsulates Bayesian updates. What\n", + "happened?\n", + "\n", + "One way to think about Bayes’s theorem is as an algorithm for inverting\n", + "conditional probabilities. Given $\\mathrm{p}(B|A)$, we can\n", + "compute $\\mathrm{p}(A|B)$, provided we know\n", + "$\\mathrm{p}(A)$ and $\\mathrm{p}(B)$. Of course\n", + "this algorithm is only useful if, for some reason, it is easier to\n", + "compute $\\mathrm{p}(B|A)$ than\n", + "$\\mathrm{p}(A|B)$.\n", + "\n", + "In this example, it is. By running simulations, we can estimate the\n", + "distribution of size conditioned on age, or\n", + "$\\mathrm{p}(size|age)$. But it is harder to get the\n", + "distribution of age conditioned on size, or\n", + "$\\mathrm{p}(age|size)$. So this seems like a perfect\n", + "opportunity to use Bayes’s theorem.\n", + "\n", + "The reason I didn’t is computational efficiency. To estimate\n", + "$\\mathrm{p}(size|age)$ for any given size, you have to run\n", + "a lot of simulations. Along the way, you end up computing\n", + "$\\mathrm{p}(size|age)$ for a lot of sizes. In fact, you end\n", + "up computing the entire joint distribution of size and age,\n", + "$\\mathrm{p}(size, age)$.\n", + "\n", + "And once you have the joint distribution, you don’t really need Bayes’s\n", + "theorem, you can extract $\\mathrm{p}(age|size)$ by taking\n", + "slices from the joint distribution, as demonstrated in\n", + "ConditionalCdf.\n", + "\n", + "So we side-stepped Bayes, but he was with us in spirit.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/14_A_Hierarchical_Model.ipynb b/notebooks/14_A_Hierarchical_Model.ipynb new file mode 100644 index 0000000..6aa17c9 --- /dev/null +++ b/notebooks/14_A_Hierarchical_Model.ipynb @@ -0,0 +1,323 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "A Hierarchical Model {#hierarchical}\n", + "====================\n", + "\n", + "The Geiger counter problem\n", + "--------------------------\n", + "\n", + "I got the idea for the following problem from Tom Campbell-Ricketts,\n", + "author of the Maximum Entropy blog at\n", + ". And he got the idea from\n", + "E. T. Jaynes, author of the classic *Probability Theory: The Logic\n", + "of Science*:\n", + "\n", + "> Suppose that a radioactive source emits particles toward a Geiger\n", + "> counter at an average rate of $r$ particles per second, but the\n", + "> counter only registers a fraction, $f$, of the particles that hit it.\n", + "> If $f$ is 10% and the counter registers 15 particles in a one second\n", + "> interval, what is the posterior distribution of $n$, the actual number\n", + "> of particles that hit the counter, and $r$, the average rate particles\n", + "> are emitted?\n", + "\n", + "To get started on a problem like this, think about the chain of\n", + "causation that starts with the parameters of the system and ends with\n", + "the observed data:\n", + "\n", + "1. The source emits particles at an average rate, $r$.\n", + "\n", + "2. During any given second, the source emits $n$ particles toward the\n", + " counter.\n", + "\n", + "3. Out of those $n$ particles, some number, $k$, get counted.\n", + "\n", + "The probability that an atom decays is the same at any point in time, so\n", + "radioactive decay is well modeled by a Poisson process. Given $r$, the\n", + "distribution of $n$ is Poisson distribution with parameter $r$.\n", + "\n", + "And if we assume that the probability of detection for each particle is\n", + "independent of the others, the distribution of $k$ is the binomial\n", + "distribution with parameters $n$ and $f$.\n", + "\n", + "Given the parameters of the system, we can find the distribution of the\n", + "data. So we can solve what is called the **forward\n", + "problem**.\n", + "\n", + "Now we want to go the other way: given the data, we want the\n", + "distribution of the parameters. This is called the **inverse\n", + "problem**. And if you can solve the forward problem, you can use\n", + "Bayesian methods to solve the inverse problem.\n", + "\n", + "Start simple\n", + "------------\n", + "\n", + "![Posterior distribution of $n$ for three values of\n", + "$r$.](figs/jaynes1.pdf)\n", + "\n", + "[fig.jaynes1]\n", + "\n", + "Let’s start with a simple version of the problem where we know the value\n", + "of $r$. We are given the value of $f$, so all we have to do is estimate\n", + "$n$.\n", + "\n", + "I define a Suite called Detector that models the behavior\n", + "of the detector and estimates $n$.\n", + "\n", + " class Detector(thinkbayes.Suite):\n", + "\n", + " def __init__(self, r, f, high=500, step=1):\n", + " pmf = thinkbayes.MakePoissonPmf(r, high, step=step)\n", + " thinkbayes.Suite.__init__(self, pmf, name=r)\n", + " self.r = r\n", + " self.f = f\n", + "\n", + "If the average emission rate is $r$ particles per second, the\n", + "distribution of $n$ is Poisson with parameter $r$. high and\n", + "step determine the upper bound for $n$ and the step size\n", + "between hypothetical values.\n", + "\n", + "Now we need a likelihood function:\n", + "\n", + " # class Detector\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " k = data\n", + " n = hypo\n", + " p = self.f\n", + "\n", + " return thinkbayes.EvalBinomialPmf(k, n, p)\n", + "\n", + "data is the number of particles detected, and\n", + "hypo is the hypothetical number of particles emitted, $n$.\n", + "\n", + "If there are actually $n$ particles, and the probability of detecting\n", + "any one of them is $f$, the probability of detecting $k$ particles is\n", + "given by the binomial distribution.\n", + "\n", + "That’s it for the Detector. We can try it out for a range of values of\n", + "$r$:\n", + "\n", + " f = 0.1\n", + " k = 15\n", + "\n", + " for r in [100, 250, 400]:\n", + " suite = Detector(r, f, step=1)\n", + " suite.Update(k)\n", + " print suite.MaximumLikelihood()\n", + "\n", + "Figure [fig.jaynes1] shows the posterior distribution of $n$ for several\n", + "given values of $r$.\n", + "\n", + "Make it hierarchical\n", + "--------------------\n", + "\n", + "In the previous section, we assume $r$ is known. Now let’s relax that\n", + "assumption. I define another Suite, called Emitter, that\n", + "models the behavior of the emitter and estimates $r$:\n", + "\n", + " class Emitter(thinkbayes.Suite):\n", + "\n", + " def __init__(self, rs, f=0.1):\n", + " detectors = [Detector(r, f) for r in rs]\n", + " thinkbayes.Suite.__init__(self, detectors)\n", + "\n", + "rs is a sequence of hypothetical value for $r$.\n", + "detectors is a sequence of Detector objects, one for each\n", + "value of $r$. The values in the Suite are Detectors, so Emitter is a\n", + "**meta-Suite**; that is, a Suite that contains other Suites\n", + "as values.\n", + "\n", + "To update the Emitter, we have to compute the likelihood of the data\n", + "under each hypothetical value of $r$. But each value of $r$ is\n", + "represented by a Detector that contains a range of values for $n$.\n", + "\n", + "To compute the likelihood of the data for a given Detector, we loop\n", + "through the values of $n$ and add up the total probability of $k$.\n", + "That’s what SuiteLikelihood does:\n", + "\n", + " # class Detector\n", + "\n", + " def SuiteLikelihood(self, data):\n", + " total = 0\n", + " for hypo, prob in self.Items():\n", + " like = self.Likelihood(data, hypo)\n", + " total += prob * like\n", + " return total\n", + "\n", + "Now we can write the Likelihood function for the Emitter:\n", + "\n", + " # class Detector\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " detector = hypo\n", + " like = detector.SuiteLikelihood(data)\n", + " return like\n", + "\n", + "Each hypo is a Detector, so we can invoke\n", + "SuiteLikelihood to get the likelihood of the data under the\n", + "hypothesis.\n", + "\n", + "After we update the Emitter, we have to update each of the Detectors,\n", + "too.\n", + "\n", + " # class Detector\n", + "\n", + " def Update(self, data):\n", + " thinkbayes.Suite.Update(self, data)\n", + " \n", + " for detector in self.Values():\n", + " detector.Update()\n", + "\n", + "A model like this, with multiple levels of Suites, is called\n", + "**hierarchical**.\n", + "\n", + "A little optimization\n", + "---------------------\n", + "\n", + "You might recognize SuiteLikelihood; we saw it in\n", + "Section [suitelike]. At the time, I pointed out that we didn’t really\n", + "need it, because the total probability computed by\n", + "SuiteLikelihood is exactly the normalizing constant\n", + "computed and returned by Update.\n", + "\n", + "So instead of updating the Emitter and then updating the Detectors, we\n", + "can do both steps at the same time, using the result from\n", + "Detector.Update as the likelihood of Emitter.\n", + "\n", + "Here’s the streamlined version of Emitter.Likelihood:\n", + "\n", + " # class Emitter\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " return hypo.Update(data)\n", + "\n", + "And with this version of Likelihood we can use the default\n", + "version of Update. So this version has fewer lines of code,\n", + "and it runs faster because it does not compute the normalizing constant\n", + "twice.\n", + "\n", + "Extracting the posteriors\n", + "-------------------------\n", + "\n", + "![Posterior distributions of $n$ and $r$.](figs/jaynes2.pdf)\n", + "\n", + "[fig.jaynes2]\n", + "\n", + "After we update the Emitter, we can get the posterior distribution of\n", + "$r$ by looping through the Detectors and their probabilities:\n", + "\n", + " # class Emitter\n", + "\n", + " def DistOfR(self):\n", + " items = [(detector.r, prob) for detector, prob in self.Items()]\n", + " return thinkbayes.MakePmfFromItems(items)\n", + "\n", + "items is a list of values of $r$ and their probabilities.\n", + "The result is the Pmf of $r$.\n", + "\n", + "To get the posterior distribution of $n$, we have to compute the mixture\n", + "of the Detectors. We can use thinkbayes.MakeMixture, which\n", + "takes a meta-Pmf that maps from each distribution to its probability.\n", + "And that’s exactly what the Emitter is:\n", + "\n", + " # class Emitter\n", + "\n", + " def DistOfN(self):\n", + " return thinkbayes.MakeMixture(self)\n", + "\n", + "Figure [fig.jaynes2] shows the results. Not surprisingly, the most\n", + "likely value for $n$ is 150. Given $f$ and $n$, the expected count is\n", + "$k = f n$, so given $f$ and $k$, the expected value of $n$ is $k / f$,\n", + "which is 150.\n", + "\n", + "And if 150 particles are emitted in one second, the most likely value of\n", + "$r$ is 150 particles per second. So the posterior distribution of $r$ is\n", + "also centered on 150.\n", + "\n", + "The posterior distributions of $r$ and $n$ are similar; the only\n", + "difference is that we are slightly less certain about $n$. In general,\n", + "we can be more certain about the long-range emission rate, $r$, than\n", + "about the number of particles emitted in any particular second, $n$.\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "The Geiger counter problem demonstrates the connection between causation\n", + "and hierarchical modeling. In the example, the emission rate $r$ has a\n", + "causal effect on the number of particles, $n$, which has a causal effect\n", + "on the particle count, $k$.\n", + "\n", + "The hierarchical model reflects the structure of the system, with causes\n", + "at the top and effects at the bottom.\n", + "\n", + "1. At the top level, we start with a range of hypothetical values for\n", + " $r$.\n", + "\n", + "2. For each value of $r$, we have a range of values for $n$, and the\n", + " prior distribution of $n$ depends on $r$.\n", + "\n", + "3. When we update the model, we go bottom-up. We compute a posterior\n", + " distribution of $n$ for each value of $r$, then compute the\n", + " posterior distribution of $r$.\n", + "\n", + "So causal information flows down the hierarchy, and inference flows up.\n", + "\n", + "Exercises\n", + "---------\n", + "\n", + "This exercise is also inspired by an example in Jaynes,\n", + "*Probability Theory*.\n", + "\n", + "Suppose you buy a mosquito trap that is supposed to reduce the\n", + "population of mosquitoes near your house. Each week, you empty the trap\n", + "and count the number of mosquitoes captured. After the first week, you\n", + "count 30 mosquitoes. After the second week, you count 20 mosquitoes.\n", + "Estimate the percentage change in the number of mosquitoes in your yard.\n", + "\n", + "To answer this question, you have to make some modeling decisions. Here\n", + "are some suggestions:\n", + "\n", + "- Suppose that each week a large number of mosquitoes, $N$, is bred in\n", + " a wetland near your home.\n", + "\n", + "- During the week, some fraction of them, $f_1$, wander into your\n", + " yard, and of those some fraction, $f_2$, are caught in the trap.\n", + "\n", + "- Your solution should take into account your prior belief about how\n", + " much $N$ is likely to change from one week to the next. You can do\n", + " that by adding a level to the hierarchy to model the percent change\n", + " in $N$.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/15_Dealing_with_Dimensions.ipynb b/notebooks/15_Dealing_with_Dimensions.ipynb new file mode 100644 index 0000000..f81908d --- /dev/null +++ b/notebooks/15_Dealing_with_Dimensions.ipynb @@ -0,0 +1,1120 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Dealing with Dimensions {#species}\n", + "=======================\n", + "\n", + "Belly button bacteria\n", + "---------------------\n", + "\n", + "Belly Button Biodiversity 2.0 (BBB2) is a nation-wide citizen science\n", + "project with the goal of identifying bacterial species that can be found\n", + "in human navels (). The project might\n", + "seem whimsical, but it is part of an increasing interest in the human\n", + "microbiome, the set of microorganisms that live on human skin and parts\n", + "of the body.\n", + "\n", + "In their pilot study, BBB2 researchers collected swabs from the navels\n", + "of 60 volunteers, used multiplex pyrosequencing to extract and sequence\n", + "fragments of 16S rDNA, then identified the species or genus the\n", + "fragments came from. Each identified fragment is called a “read.”\n", + "\n", + "We can use these data to answer several related questions:\n", + "\n", + "- Based on the number of species observed, can we estimate the total\n", + " number of species in the environment?\n", + "\n", + "- Can we estimate the prevalence of each species; that is, the\n", + " fraction of the total population belonging to each species?\n", + "\n", + "- If we are planning to collect additional samples, can we predict how\n", + " many new species we are likely to discover?\n", + "\n", + "- How many additional reads are needed to increase the fraction of\n", + " observed species to a given threshold?\n", + "\n", + "These questions make up what is called the **Unseen Species\n", + "problem**.\n", + "\n", + "Lions and tigers and bears\n", + "--------------------------\n", + "\n", + "I’ll start with a simplified version of the problem where we know that\n", + "there are exactly three species. Let’s call them lions, tigers and\n", + "bears. Suppose we visit a wild animal preserve and see 3 lions, 2 tigers\n", + "and one bear.\n", + "\n", + "If we have an equal chance of observing any animal in the preserve, the\n", + "number of each species we see is governed by the multinomial\n", + "distribution. If the prevalence of lions and tigers and bears is\n", + "`p_lion` and `p_tiger` and `p_bear`, the likelihood of seeing 3 lions, 2\n", + "tigers and one bear is\n", + "\n", + " p_lion**3 * p_tiger**2 * p_bear**1\n", + "\n", + "An approach that is tempting, but not correct, is to use beta\n", + "distributions, as in Section [beta], to describe the prevalence of each\n", + "species separately. For example, we saw 3 lions and 3 non-lions; if we\n", + "think of that as 3 “heads” and 3 “tails,” then the posterior\n", + "distribution of `p_lion` is:\n", + "\n", + " beta = thinkbayes.Beta()\n", + " beta.Update((3, 3))\n", + " print beta.MaximumLikelihood()\n", + "\n", + "The maximum likelihood estimate for `p_lion` is the observed rate, 50%.\n", + "Similarly the MLEs for `p_tiger` and `p_bear` are 33% and 17%.\n", + "\n", + "But there are two problems:\n", + "\n", + "1. We have implicitly used a prior for each species that is uniform\n", + " from 0 to 1, but since we know that there are three species, that\n", + " prior is not correct. The right prior should have a mean of 1/3, and\n", + " there should be zero likelihood that any species has a prevalence of\n", + " 100%.\n", + "\n", + "2. The distributions for each species are not independent, because the\n", + " prevalences have to add up to 1. To capture this dependence, we need\n", + " a joint distribution for the three prevalences.\n", + "\n", + "We can use a Dirichlet distribution to solve both of these problems (see\n", + "). In the same way\n", + "we used the beta distribution to describe the distribution of bias for a\n", + "coin, we can use a Dirichlet distribution to describe the joint\n", + "distribution of `p_lion`, `p_tiger` and `p_bear`.\n", + "\n", + "The Dirichlet distribution is the multi-dimensional generalization of\n", + "the beta distribution. Instead of two possible outcomes, like heads and\n", + "tails, the Dirichlet distribution handles any number of outcomes: in\n", + "this example, three species.\n", + "\n", + "If there are n outcomes, the Dirichlet distribution is\n", + "described by n parameters, written $\\alpha_1$ through\n", + "$\\alpha_n$.\n", + "\n", + "Here’s the definition, from thinkbayes.py, of a class that\n", + "represents a Dirichlet distribution:\n", + "\n", + " class Dirichlet(object):\n", + "\n", + " def __init__(self, n):\n", + " self.n = n\n", + " self.params = numpy.ones(n, dtype=numpy.int)\n", + "\n", + "n is the number of dimensions; initially the parameters are\n", + "all 1. I use a numpy array to store the parameters so I can\n", + "take advantage of array operations.\n", + "\n", + "Given a Dirichlet distribution, the marginal distribution for each\n", + "prevalence is a beta distribution, which we can compute like this:\n", + "\n", + " def MarginalBeta(self, i):\n", + " alpha0 = self.params.sum()\n", + " alpha = self.params[i]\n", + " return Beta(alpha, alpha0-alpha)\n", + "\n", + "i is the index of the marginal distribution we want.\n", + "alpha0 is the sum of the parameters; alpha is\n", + "the parameter for the given species.\n", + "\n", + "In the example, the prior marginal distribution for each species is\n", + "Beta(1, 2). We can compute the prior means like this:\n", + "\n", + " dirichlet = thinkbayes.Dirichlet(3)\n", + " for i in range(3):\n", + " beta = dirichlet.MarginalBeta(i)\n", + " print beta.Mean()\n", + "\n", + "As expected, the prior mean prevalence for each species is 1/3.\n", + "\n", + "To update the Dirichlet distribution, we add the observations to the\n", + "parameters like this:\n", + "\n", + " def Update(self, data):\n", + " m = len(data)\n", + " self.params[:m] += data\n", + "\n", + "Here data is a sequence of counts in the same order as\n", + "params, so in this example, it should be the number of\n", + "lions, tigers and bears.\n", + "\n", + "data can be shorter than params; in that case\n", + "there are some species that have not been observed.\n", + "\n", + "Here’s code that updates dirichlet with the observed data\n", + "and computes the posterior marginal distributions.\n", + "\n", + " data = [3, 2, 1]\n", + " dirichlet.Update(data)\n", + "\n", + " for i in range(3):\n", + " beta = dirichlet.MarginalBeta(i)\n", + " pmf = beta.MakePmf()\n", + " print i, pmf.Mean()\n", + "\n", + "![Distribution of prevalences for three species.](figs/species1.pdf)\n", + "\n", + "[fig.species1]\n", + "\n", + "Figure [fig.species1] shows the results. The posterior mean prevalences\n", + "are 44%, 33%, and 22%.\n", + "\n", + "The hierarchical version\n", + "------------------------\n", + "\n", + "We have solved a simplified version of the problem: if we know how many\n", + "species there are, we can estimate the prevalence of each.\n", + "\n", + "Now let’s get back to the original problem, estimating the total number\n", + "of species. To solve this problem I’ll define a meta-Suite, which is a\n", + "Suite that contains other Suites as hypotheses. In this case, the\n", + "top-level Suite contains hypotheses about the number of species; the\n", + "bottom level contains hypotheses about prevalences.\n", + "\n", + "Here’s the class definition:\n", + "\n", + " class Species(thinkbayes.Suite):\n", + "\n", + " def __init__(self, ns):\n", + " hypos = [thinkbayes.Dirichlet(n) for n in ns]\n", + " thinkbayes.Suite.__init__(self, hypos)\n", + "\n", + "`__init__` takes a list of possible values for n and makes\n", + "a list of Dirichlet objects.\n", + "\n", + "Here’s the code that creates the top-level suite:\n", + "\n", + " ns = range(3, 30)\n", + " suite = Species(ns)\n", + "\n", + "ns is the list of possible values for n. We\n", + "have seen 3 species, so there have to be at least that many. I chose an\n", + "upper bound that seems reasonable, but we will check later that the\n", + "probability of exceeding this bound is low. And at least initially we\n", + "assume that any value in this range is equally likely.\n", + "\n", + "To update a hierarchical model, you have to update all levels. Usually\n", + "you have to update the bottom level first and work up, but in this case\n", + "we can update the top level first:\n", + "\n", + " #class Species\n", + "\n", + " def Update(self, data):\n", + " thinkbayes.Suite.Update(self, data)\n", + " for hypo in self.Values():\n", + " hypo.Update(data)\n", + "\n", + "Species.Update invokes Update in the parent\n", + "class, then loops through the sub-hypotheses and updates them.\n", + "\n", + "Now all we need is a likelihood function:\n", + "\n", + " # class Species\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " dirichlet = hypo\n", + " like = 0\n", + " for i in range(1000):\n", + " like += dirichlet.Likelihood(data)\n", + "\n", + " return like\n", + "\n", + "data is a sequence of observed counts; hypo is\n", + "a Dirichlet object. Species.Likelihood calls\n", + "Dirichlet.Likelihood 1000 times and returns the total.\n", + "\n", + "Why call it 1000 times? Because Dirichlet.Likelihood\n", + "doesn’t actually compute the likelihood of the data under the whole\n", + "Dirichlet distribution. Instead, it draws one sample from the\n", + "hypothetical distribution and computes the likelihood of the data under\n", + "the sampled set of prevalences.\n", + "\n", + "Here’s what it looks like:\n", + "\n", + " # class Dirichlet\n", + "\n", + " def Likelihood(self, data):\n", + " m = len(data)\n", + " if self.n < m:\n", + " return 0\n", + "\n", + " x = data\n", + " p = self.Random()\n", + " q = p[:m]**x\n", + " return q.prod()\n", + "\n", + "The length of data is the number of species observed. If we\n", + "see more species than we thought existed, the likelihood is 0.\n", + "\n", + "Otherwise we select a random set of prevalences, p, and\n", + "compute the multinomial PMF, which is\n", + "$$c_x p_1^{x_1} \\cdots p_n^{x_n}$$ $p_i$ is the prevalence of the $i$th\n", + "species, and $x_i$ is the observed number. The first term, $c_x$, is the\n", + "multinomial coefficient; I leave it out of the computation because it is\n", + "a multiplicative factor that depends only on the data, not the\n", + "hypothesis, so it gets normalized away (see\n", + ").\n", + "\n", + "m is the number of observed species. We only need the first\n", + "m elements of p; for the others, $x_i$ is 0,\n", + "so $p_i^{x_i}$ is 1, and we can leave them out of the product.\n", + "\n", + "Random sampling {#randomdir}\n", + "---------------\n", + "\n", + "There are two ways to generate a random sample from a Dirichlet\n", + "distribution. One is to use the marginal beta distributions, but in that\n", + "case you have to select one at a time and scale the rest so they add up\n", + "to 1 (see\n", + ").\n", + "\n", + "A less obvious, but faster, way is to select values from n\n", + "gamma distributions, then normalize by dividing through by the total.\n", + "Here’s the code:\n", + "\n", + " # class Dirichlet\n", + "\n", + " def Random(self):\n", + " p = numpy.random.gamma(self.params)\n", + " return p / p.sum()\n", + "\n", + "Now we’re ready to look at some results. Here is the code that extracts\n", + "the posterior distribution of n:\n", + "\n", + " def DistOfN(self):\n", + " pmf = thinkbayes.Pmf()\n", + " for hypo, prob in self.Items():\n", + " pmf.Set(hypo.n, prob)\n", + " return pmf\n", + "\n", + "DistOfN iterates through the top-level hypotheses and\n", + "accumulates the probability of each n.\n", + "\n", + "![Posterior distribution of n.](figs/species2.pdf)\n", + "\n", + "[fig.species2]\n", + "\n", + "Figure [fig.species2] shows the result. The most likely value is 4.\n", + "Values from 3 to 7 are reasonably likely; after that the probabilities\n", + "drop off quickly. The probability that there are 29 species is low\n", + "enough to be negligible; if we chose a higher bound, we would get nearly\n", + "the same result.\n", + "\n", + "Remember that this result is based on a uniform prior for\n", + "n. If we have background information about the number of\n", + "species in the environment, we might choose a different prior.\n", + "\n", + "Optimization\n", + "------------\n", + "\n", + "I have to admit that I am proud of this example. The Unseen Species\n", + "problem is not easy, and I think this solution is simple and clear, and\n", + "takes surprisingly few lines of code (about 50 so far).\n", + "\n", + "The only problem is that it is slow. It’s good enough for the example\n", + "with only 3 observed species, but not good enough for the belly button\n", + "data, with more than 100 species in some samples.\n", + "\n", + "The next few sections present a series of optimizations we need to make\n", + "this solution scale. Before we get into the details, here’s a road map.\n", + "\n", + "- The first step is to recognize that if we update the Dirichlet\n", + " distributions with the same data, the first m\n", + " parameters are the same for all of them. The only difference is the\n", + " number of hypothetical unseen species. So we don’t really need\n", + " n Dirichlet objects; we can store the parameters in the\n", + " top level of the hierarchy. Species2 implements this\n", + " optimization.\n", + "\n", + "- Species2 also uses the same set of random values for\n", + " all of the hypotheses. This saves time generating random values, but\n", + " it has a second benefit that turns out to be more important: by\n", + " giving all hypotheses the same selection from the sample space, we\n", + " make the comparison between the hypotheses more fair, so it takes\n", + " fewer iterations to converge.\n", + "\n", + "- Even with these changes there is a major performance problem. As the\n", + " number of observed species increases, the array of random\n", + " prevalences gets bigger, and the chance of choosing one that is\n", + " approximately right becomes small. So the vast majority of\n", + " iterations yield small likelihoods that don’t contribute much to the\n", + " total, and don’t discriminate between hypotheses.\n", + "\n", + " The solution is to do the updates one species at a time.\n", + " Species4 is a simple implementation of this strategy\n", + " using Dirichlet objects to represent the sub-hypotheses.\n", + "\n", + "- Finally, Species5 combines the sub-hypotheses into the\n", + " top level and uses numpy array operations to speed\n", + " things up.\n", + "\n", + "If you are not interested in the details, feel free to skip to\n", + "Section [belly] where we look at results from the belly button data.\n", + "\n", + "Collapsing the hierarchy {#collapsing}\n", + "------------------------\n", + "\n", + "All of the bottom-level Dirichlet distributions are updated with the\n", + "same data, so the first m parameters are the same for all\n", + "of them. We can eliminate them and merge the parameters into the\n", + "top-level suite. Species2 implements this optimization:\n", + "\n", + " class Species2(object):\n", + " \n", + " def __init__(self, ns):\n", + " self.ns = ns\n", + " self.probs = numpy.ones(len(ns), dtype=numpy.double)\n", + " self.params = numpy.ones(self.high, dtype=numpy.int)\n", + "\n", + "ns is the list of hypothetical values for n;\n", + "probs is the list of corresponding probabilities. And\n", + "params is the sequence of Dirichlet parameters, initially\n", + "all 1.\n", + "\n", + "Species2.Update updates both levels of the hierarchy: first\n", + "the probability for each value of n, then the Dirichlet\n", + "parameters:\n", + "\n", + " # class Species2\n", + "\n", + " def Update(self, data):\n", + " like = numpy.zeros(len(self.ns), dtype=numpy.double)\n", + " for i in range(1000):\n", + " like += self.SampleLikelihood(data)\n", + "\n", + " self.probs *= like\n", + " self.probs /= self.probs.sum()\n", + "\n", + " m = len(data)\n", + " self.params[:m] += data\n", + "\n", + "SampleLikelihood returns an array of likelihoods, one for\n", + "each value of n. like accumulates the total\n", + "likelihood for 1000 samples. self.probs is multiplied by\n", + "the total likelihood, then normalized. The last two lines, which update\n", + "the parameters, are the same as in Dirichlet.Update.\n", + "\n", + "Now let’s look at SampleLikelihood. There are two\n", + "opportunities for optimization here:\n", + "\n", + "- When the hypothetical number of species, n, exceeds the\n", + " observed number, m, we only need the first\n", + " m terms of the multinomial PMF; the rest are 1.\n", + "\n", + "- If the number of species is large, the likelihood of the data might\n", + " be too small for floating-point (see  [underflow]). So it is safer\n", + " to compute log-likelihoods.\n", + "\n", + "Again, the multinomial PMF is $$c_x p_1^{x_1} \\cdots p_n^{x_n}$$ So the\n", + "log-likelihood is $$\\log c_x + x_1 \\log p_1 + \\cdots + x_n \\log p_n$$\n", + "which is fast and easy to compute. Again, $c_x$ it is the same for all\n", + "hypotheses, so we can drop it. Here’s the code:\n", + "\n", + " # class Species2\n", + "\n", + " def SampleLikelihood(self, data):\n", + " gammas = numpy.random.gamma(self.params)\n", + "\n", + " m = len(data)\n", + " row = gammas[:m]\n", + " col = numpy.cumsum(gammas)\n", + "\n", + " log_likes = []\n", + " for n in self.ns:\n", + " ps = row / col[n-1]\n", + " terms = data * numpy.log(ps)\n", + " log_like = terms.sum()\n", + " log_likes.append(log_like)\n", + "\n", + " log_likes -= numpy.max(log_likes)\n", + " likes = numpy.exp(log_likes)\n", + "\n", + " coefs = [thinkbayes.BinomialCoef(n, m) for n in self.ns]\n", + " likes *= coefs\n", + "\n", + " return likes\n", + "\n", + "gammas is an array of values from a gamma distribution; its\n", + "length is the largest hypothetical value of n.\n", + "row is just the first m elements of\n", + "gammas; since these are the only elements that depend on\n", + "the data, they are the only ones we need.\n", + "\n", + "For each value of n we need to divide row by\n", + "the total of the first n values from gamma.\n", + "cumsum computes these cumulative sums and stores them in\n", + "col.\n", + "\n", + "The loop iterates through the values of n and accumulates a\n", + "list of log-likelihoods.\n", + "\n", + "Inside the loop, ps contains the row of probabilities,\n", + "normalized with the appropriate cumulative sum. terms\n", + "contains the terms of the summation, $x_i \\log p_i$, and `log_like`\n", + "contains their sum.\n", + "\n", + "After the loop, we want to convert the log-likelihoods to linear\n", + "likelihoods, but first it’s a good idea to shift them so the largest\n", + "log-likelihood is 0; that way the linear likelihoods are not too small\n", + "(see  [underflow]).\n", + "\n", + "Finally, before we return the likelihood, we have to apply a correction\n", + "factor, which is the number of ways we could have observed these\n", + "m species, if the total number of species is\n", + "n. BinomialCoefficient computes “n choose m”,\n", + "which is written $\\binom{n}{m}$.\n", + "\n", + "As often happens, the optimized version is less readable and more\n", + "error-prone than the original. But that’s one reason I think it is a\n", + "good idea to start with the simple version; we can use it for regression\n", + "testing. I plotted results from both versions and confirmed that they\n", + "are approximately equal, and that they converge as the number of\n", + "iterations increases.\n", + "\n", + "One more problem\n", + "----------------\n", + "\n", + "There’s more we could do to optimize this code, but there’s another\n", + "problem we need to fix first. As the number of observed species\n", + "increases, this version gets noisier and takes more iterations to\n", + "converge on a good answer.\n", + "\n", + "The problem is that if the prevalences we choose from the Dirichlet\n", + "distribution, the ps, are not at least approximately right,\n", + "the likelihood of the observed data is close to zero and almost equally\n", + "bad for all values of n. So most iterations don’t provide\n", + "any useful contribution to the total likelihood. And as the number of\n", + "observed species, m, gets large, the probability of\n", + "choosing ps with non-negligible likelihood gets small.\n", + "Really small.\n", + "\n", + "Fortunately, there is a solution. Remember that if you observe a set of\n", + "data, you can update the prior distribution with the entire dataset, or\n", + "you can break it up into a series of updates with subsets of the data,\n", + "and the result is the same either way.\n", + "\n", + "For this example, the key is to perform the updates one species at a\n", + "time. That way when we generate a random set of ps, only\n", + "one of them affects the computed likelihood, so the chance of choosing a\n", + "good one is much better.\n", + "\n", + "Here’s a new version that updates one species at a time:\n", + "\n", + " class Species4(Species):\n", + "\n", + " def Update(self, data):\n", + " m = len(data)\n", + "\n", + " for i in range(m):\n", + " one = numpy.zeros(i+1)\n", + " one[i] = data[i] \n", + " Species.Update(self, one)\n", + "\n", + "This version inherits `__init__` from Species, so it\n", + "represents the hypotheses as a list of Dirichlet objects (unlike\n", + "Species2).\n", + "\n", + "Update loops through the observed species and makes an\n", + "array, one, with all zeros and one species count. Then it\n", + "calls Update in the parent class, which computes the\n", + "likelihoods and updates the sub-hypotheses.\n", + "\n", + "So in the running example, we do three updates. The first is something\n", + "like “I have seen three lions.” The second is “I have seen two tigers\n", + "and no additional lions.” And the third is “I have seen one bear and no\n", + "more lions and tigers.”\n", + "\n", + "Here’s the new version of Likelihood:\n", + "\n", + " # class Species4\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " dirichlet = hypo\n", + " like = 0\n", + " for i in range(self.iterations):\n", + " like += dirichlet.Likelihood(data)\n", + "\n", + " # correct for the number of unseen species the new one\n", + " # could have been\n", + " m = len(data)\n", + " num_unseen = dirichlet.n - m + 1\n", + " like *= num_unseen\n", + "\n", + " return like\n", + "\n", + "This is almost the same as Species.Likelihood. The\n", + "difference is the factor, `num_unseen`. This correction is necessary\n", + "because each time we see a species for the first time, we have to\n", + "consider that there were some number of other unseen species that we\n", + "might have seen. For larger values of n there are more\n", + "unseen species that we could have seen, which increases the likelihood\n", + "of the data.\n", + "\n", + "This is a subtle point and I have to admit that I did not get it right\n", + "the first time. But again I was able to validate this version by\n", + "comparing it to the previous versions.\n", + "\n", + "We’re not done yet\n", + "------------------\n", + "\n", + "Performing the updates one species at a time solves one problem, but it\n", + "creates another. Each update takes time proportional to $k m$, where $k$\n", + "is the number of hypotheses and $m$ is the number of observed species.\n", + "So if we do $m$ updates, the total run time is proportional to $k m^2$.\n", + "\n", + "But we can speed things up using the same trick we used in\n", + "Section [collapsing]: we’ll get rid of the Dirichlet objects and\n", + "collapse the two levels of the hierarchy into a single object. So here’s\n", + "yet another version of Species:\n", + "\n", + " class Species5(Species2):\n", + " \n", + " def Update(self, data):\n", + " m = len(data)\n", + " for i in range(m):\n", + " self.UpdateOne(i+1, data[i])\n", + " self.params[i] += data[i]\n", + "\n", + "This version inherits `__init__` from Species2, so it uses\n", + "ns and probs to represent the distribution of\n", + "n, and params to represent the parameters of\n", + "the Dirichlet distribution.\n", + "\n", + "Update is similar to what we saw in the previous section.\n", + "It loops through the observed species and calls UpdateOne:\n", + "\n", + " # class Species5\n", + "\n", + " def UpdateOne(self, i, count):\n", + " likes = numpy.zeros(len(self.ns), dtype=numpy.double)\n", + " for i in range(self.iterations):\n", + " likes += self.SampleLikelihood(i, count)\n", + "\n", + " unseen_species = [n-i+1 for n in self.ns]\n", + " likes *= unseen_species\n", + "\n", + " self.probs *= likes\n", + " self.probs /= self.probs.sum()\n", + "\n", + "This function is similar to Species2.Update, with two\n", + "changes:\n", + "\n", + "- The interface is different. Instead of the whole dataset, we get\n", + " i, the index of the observed species, and\n", + " count, how many of that species we’ve seen.\n", + "\n", + "- We have to apply a correction factor for the number of unseen\n", + " species, as in Species4.Likelihood. The difference here\n", + " is that we update all of the likelihoods at once with array\n", + " multiplication.\n", + "\n", + "Finally, here’s SampleLikelihood:\n", + "\n", + " # class Species5\n", + "\n", + " def SampleLikelihood(self, i, count):\n", + " gammas = numpy.random.gamma(self.params)\n", + "\n", + " sums = numpy.cumsum(gammas)[self.ns[0]-1:]\n", + "\n", + " ps = gammas[i-1] / sums\n", + " log_likes = numpy.log(ps) * count\n", + "\n", + " log_likes -= numpy.max(log_likes)\n", + " likes = numpy.exp(log_likes)\n", + "\n", + " return likes\n", + "\n", + "This is similar to Species2.SampleLikelihood; the\n", + "difference is that each update only includes a single species, so we\n", + "don’t need a loop.\n", + "\n", + "The runtime of this function is proportional to the number of\n", + "hypotheses, $k$. It runs $m$ times, so the run time of the update is\n", + "proportional to $k m$. And the number of iterations we need to get an\n", + "accurate result is usually small.\n", + "\n", + "The belly button data {#belly}\n", + "---------------------\n", + "\n", + "That’s enough about lions and tigers and bears. Let’s get back to belly\n", + "buttons. To get a sense of what the data look like, consider subject\n", + "B1242, whose sample of 400 reads yielded 61 species with the following\n", + "counts:\n", + "\n", + " 92, 53, 47, 38, 15, 14, 12, 10, 8, 7, 7, 5, 5, \n", + " 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2,\n", + " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n", + " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1\n", + "\n", + "There are a few dominant species that make up a large fraction of the\n", + "whole, but many species that yielded only a single read. The number of\n", + "these “singletons” suggests that there are likely to be at least a few\n", + "unseen species.\n", + "\n", + "In the example with lions and tigers, we assume that each animal in the\n", + "preserve is equally likely to be observed. Similarly, for the belly\n", + "button data, we assume that each bacterium is equally likely to yield a\n", + "read.\n", + "\n", + "In reality, each step in the data-collection process might introduce\n", + "biases. Some species might be more likely to be picked up by a swab, or\n", + "to yield identifiable amplicons. So when we talk about the prevalence of\n", + "each species, we should remember this source of error.\n", + "\n", + "I should also acknowledge that I am using the term “species” loosely.\n", + "First, bacterial species are not well defined. Second, some reads\n", + "identify a particular species, others only identify a genus. To be more\n", + "precise, I should say “operational taxonomic unit”, or OTU.\n", + "\n", + "Now let’s process some of the belly button data. I define a class called\n", + "Subject to represent information about each subject in the\n", + "study:\n", + "\n", + " class Subject(object):\n", + "\n", + " def __init__(self, code):\n", + " self.code = code\n", + " self.species = []\n", + "\n", + "Each subject has a string code, like “B1242”, and a list of (count,\n", + "species name) pairs, sorted in increasing order by count.\n", + "Subject provides several methods to make it easy to access\n", + "these counts and species names. You can see the details in\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "![Distribution of n for subject\n", + "B1242.](figs/species-ndist-B1242.pdf)\n", + "\n", + "[species-ndist]\n", + "\n", + "Subject provides a method named Process that\n", + "creates and updates a Species5 suite, which represents the\n", + "distributions of n and the prevalences.\n", + "\n", + "And Suite2 provides DistOfN, which returns the\n", + "posterior distribution of n.\n", + "\n", + " # class Suite2\n", + "\n", + " def DistN(self):\n", + " items = zip(self.ns, self.probs)\n", + " pmf = thinkbayes.MakePmfFromItems(items)\n", + " return pmf\n", + "\n", + "Figure [species-ndist] shows the distribution of n for\n", + "subject B1242. The probability that there are exactly 61 species, and no\n", + "unseen species, is nearly zero. The most likely value is 72, with 90%\n", + "credible interval 66 to 79. At the high end, it is unlikely that there\n", + "are as many as 87 species.\n", + "\n", + "Next we compute the posterior distribution of prevalence for each\n", + "species. Species2 provides DistOfPrevalence:\n", + "\n", + " # class Species2\n", + "\n", + " def DistOfPrevalence(self, index):\n", + " metapmf = thinkbayes.Pmf()\n", + "\n", + " for n, prob in zip(self.ns, self.probs):\n", + " beta = self.MarginalBeta(n, index)\n", + " pmf = beta.MakePmf()\n", + " metapmf.Set(pmf, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(metapmf)\n", + " return metapmf, mix\n", + "\n", + "index indicates which species we want. For each\n", + "n, we have a different posterior distribution of\n", + "prevalence.\n", + "\n", + "![Distribution of prevalences for subject\n", + "B1242.](figs/species-prev-B1242.pdf)\n", + "\n", + "[species-prev]\n", + "\n", + "The loop iterates through the possible values of n and\n", + "their probabilities. For each value of n it gets a Beta\n", + "object representing the marginal distribution for the indicated species.\n", + "Remember that Beta objects contain the parameters alpha and\n", + "beta; they don’t have values and probabilities like a Pmf,\n", + "but they provide MakePmf, which generates a discrete\n", + "approximation to the continuous beta distribution.\n", + "\n", + "metapmf is a meta-Pmf that contains the distributions of\n", + "prevalence, conditioned on n. MakeMixture\n", + "combines the meta-Pmf into mix, which combines the\n", + "conditional distributions into a single distribution of prevalence.\n", + "\n", + "Figure [species-prev] shows results for the five species with the most\n", + "reads. The most prevalent species accounts for 23% of the 400 reads, but\n", + "since there are almost certainly unseen species, the most likely\n", + "estimate for its prevalence is 20%, with 90% credible interval between\n", + "17% and 23%.\n", + "\n", + "Predictive distributions\n", + "------------------------\n", + "\n", + "![Simulated rarefaction curves for subject\n", + "B1242.](figs/species-rare-B1242.pdf)\n", + "\n", + "[species-rare]\n", + "\n", + "I introduced the hidden species problem in the form of four related\n", + "questions. We have answered the first two by computing the posterior\n", + "distribution for n and the prevalence of each species.\n", + "\n", + "The other two questions are:\n", + "\n", + "- If we are planning to collect additional reads, can we predict how\n", + " many new species we are likely to discover?\n", + "\n", + "- How many additional reads are needed to increase the fraction of\n", + " observed species to a given threshold?\n", + "\n", + "To answer predictive questions like this we can use the posterior\n", + "distributions to simulate possible future events and compute predictive\n", + "distributions for the number of species, and fraction of the total, we\n", + "are likely to see.\n", + "\n", + "The kernel of these simulations looks like this:\n", + "\n", + "1. Choose n from its posterior distribution.\n", + "\n", + "2. Choose a prevalence for each species, including possible unseen\n", + " species, using the Dirichlet distribution.\n", + "\n", + "3. Generate a random sequence of future observations.\n", + "\n", + "4. Compute the number of new species, `num_new`, as a function of the\n", + " number of additional reads, k.\n", + "\n", + "5. Repeat the previous steps and accumulate the joint distribution of\n", + " `num_new` and k.\n", + "\n", + "And here’s the code. RunSimulation runs a single\n", + "simulation:\n", + "\n", + " # class Subject\n", + "\n", + " def RunSimulation(self, num_reads):\n", + " m, seen = self.GetSeenSpecies()\n", + " n, observations = self.GenerateObservations(num_reads)\n", + "\n", + " curve = []\n", + " for k, obs in enumerate(observations):\n", + " seen.add(obs)\n", + "\n", + " num_new = len(seen) - m\n", + " curve.append((k+1, num_new))\n", + "\n", + " return curve\n", + "\n", + "`num_reads` is the number of additional reads to simulate.\n", + "m is the number of seen species, and seen is a\n", + "set of strings with a unique name for each species. n is a\n", + "random value from the posterior distribution, and\n", + "observations is a random sequence of species names.\n", + "\n", + "Each time through the loop, we add the new observation to\n", + "seen and record the number of reads and the number of new\n", + "species so far.\n", + "\n", + "The result of RunSimulation is a **rarefaction\n", + "curve**, represented as a list of pairs with the number of reads\n", + "and the number of new species.\n", + "\n", + "Before we see the results, let’s look at GetSeenSpecies and\n", + "GenerateObservations.\n", + "\n", + " #class Subject\n", + "\n", + " def GetSeenSpecies(self):\n", + " names = self.GetNames()\n", + " m = len(names)\n", + " seen = set(SpeciesGenerator(names, m))\n", + " return m, seen\n", + "\n", + "GetNames returns the list of species names that appear in\n", + "the data files, but for many subjects these names are not unique. So I\n", + "use SpeciesGenerator to extend each name with a serial\n", + "number:\n", + "\n", + " def SpeciesGenerator(names, num):\n", + " i = 0\n", + " for name in names:\n", + " yield '%s-%d' % (name, i)\n", + " i += 1\n", + "\n", + " while i < num:\n", + " yield 'unseen-%d' % i\n", + " i += 1\n", + "\n", + "Given a name like Corynebacterium,\n", + "SpeciesGenerator yields Corynebacterium-1.\n", + "When the list of names is exhausted, it yields names like\n", + "unseen-62.\n", + "\n", + "Here is GenerateObservations:\n", + "\n", + " # class Subject\n", + "\n", + " def GenerateObservations(self, num_reads):\n", + " n, prevalences = self.suite.SamplePosterior()\n", + "\n", + " names = self.GetNames()\n", + " name_iter = SpeciesGenerator(names, n)\n", + "\n", + " d = dict(zip(name_iter, prevalences))\n", + " cdf = thinkbayes.MakeCdfFromDict(d)\n", + " observations = cdf.Sample(num_reads)\n", + "\n", + " return n, observations\n", + "\n", + "Again, `num_reads` is the number of additional reads to generate.\n", + "n and prevalences are samples from the\n", + "posterior distribution.\n", + "\n", + "cdf is a Cdf object that maps species names, including the\n", + "unseen, to cumulative probabilities. Using a Cdf makes it efficient to\n", + "generate a random sequence of species names.\n", + "\n", + "Finally, here is Species2.SamplePosterior:\n", + "\n", + " def SamplePosterior(self):\n", + " pmf = self.DistOfN()\n", + " n = pmf.Random()\n", + " prevalences = self.SamplePrevalences(n)\n", + " return n, prevalences\n", + "\n", + "And SamplePrevalences, which generates a sample of\n", + "prevalences conditioned on n:\n", + "\n", + " # class Species2\n", + "\n", + " def SamplePrevalences(self, n):\n", + " params = self.params[:n]\n", + " gammas = numpy.random.gamma(params)\n", + " gammas /= gammas.sum()\n", + " return gammas\n", + "\n", + "We saw this algorithm for generating random values from a Dirichlet\n", + "distribution in Section [randomdir].\n", + "\n", + "Figure [species-rare] shows 100 simulated rarefaction curves for subject\n", + "B1242. The curves are “jittered;” that is, I shifted each curve by a\n", + "random offset so they would not all overlap. By inspection we can\n", + "estimate that after 400 more reads we are likely to find 2–6 new\n", + "species.\n", + "\n", + "Joint posterior\n", + "---------------\n", + "\n", + "![Distributions of the number of new species conditioned on the number\n", + "of additional reads.](figs/species-cond-B1242.pdf)\n", + "\n", + "[species-cond]\n", + "\n", + "We can use these simulations to estimate the joint distribution of\n", + "`num_new` and k, and from that we can get the distribution\n", + "of `num_new` conditioned on any value of k.\n", + "\n", + " def MakeJointPredictive(curves):\n", + " joint = thinkbayes.Joint()\n", + " for curve in curves:\n", + " for k, num_new in curve:\n", + " joint.Incr((k, num_new))\n", + " joint.Normalize()\n", + " return joint\n", + "\n", + "MakeJointPredictive makes a Joint object, which is a Pmf\n", + "whose values are tuples.\n", + "\n", + "curves is a list of rarefaction curves created by\n", + "RunSimulation. Each curve contains a list of pairs of\n", + "k and `num_new`.\n", + "\n", + "The resulting joint distribution is a map from each pair to its\n", + "probability of occurring. Given the joint distribution, we can use\n", + "Joint.Conditional get the distribution of `num_new`\n", + "conditioned on k (see Section [conditional]).\n", + "\n", + "Subject.MakeConditionals takes a list of ks\n", + "and computes the conditional distribution of `num_new` for each\n", + "k. The result is a list of Cdf objects.\n", + "\n", + " def MakeConditionals(curves, ks):\n", + " joint = MakeJointPredictive(curves)\n", + "\n", + " cdfs = []\n", + " for k in ks:\n", + " pmf = joint.Conditional(1, 0, k)\n", + " pmf.name = 'k=%d' % k\n", + " cdf = pmf.MakeCdf()\n", + " cdfs.append(cdf)\n", + "\n", + " return cdfs\n", + "\n", + "Figure [species-cond] shows the results. After 100 reads, the median\n", + "predicted number of new species is 2; the 90% credible interval is 0 to\n", + "5. After 800 reads, we expect to see 3 to 12 new species.\n", + "\n", + "Coverage\n", + "--------\n", + "\n", + "![Complementary CDF of coverage for a range of additional\n", + "reads.](figs/species-frac-B1242.pdf)\n", + "\n", + "[species-frac]\n", + "\n", + "The last question we want to answer is, “How many additional reads are\n", + "needed to increase the fraction of observed species to a given\n", + "threshold?”\n", + "\n", + "To answer this question, we need a version of RunSimulation\n", + "that computes the fraction of observed species rather than the number of\n", + "new species.\n", + "\n", + " # class Subject\n", + "\n", + " def RunSimulation(self, num_reads):\n", + " m, seen = self.GetSeenSpecies()\n", + " n, observations = self.GenerateObservations(num_reads)\n", + "\n", + " curve = []\n", + " for k, obs in enumerate(observations):\n", + " seen.add(obs)\n", + "\n", + " frac_seen = len(seen) / float(n)\n", + " curve.append((k+1, frac_seen))\n", + "\n", + " return curve\n", + "\n", + "Next we loop through each curve and make a dictionary, d,\n", + "that maps from the number of additional reads, k, to a list\n", + "of fracs; that is, a list of values for the coverage\n", + "achieved after k reads.\n", + "\n", + " def MakeFracCdfs(self, curves):\n", + " d = {}\n", + " for curve in curves:\n", + " for k, frac in curve:\n", + " d.setdefault(k, []).append(frac)\n", + "\n", + " cdfs = {}\n", + " for k, fracs in d.iteritems():\n", + " cdf = thinkbayes.MakeCdfFromList(fracs)\n", + " cdfs[k] = cdf\n", + "\n", + " return cdfs\n", + "\n", + "Then for each value of k we make a Cdf of\n", + "fracs; this Cdf represents the distribution of coverage\n", + "after k reads.\n", + "\n", + "Remember that the CDF tells you the probability of falling below a given\n", + "threshold, so the *complementary* CDF tells you the\n", + "probability of exceeding it. Figure [species-frac] shows complementary\n", + "CDFs for a range of values of k.\n", + "\n", + "To read this figure, select the level of coverage you want to achieve\n", + "along the $x$-axis. As an example, choose 90%.\n", + "\n", + "Now you can read up the chart to find the probability of achieving 90%\n", + "coverage after k reads. For example, with 200 reads, you\n", + "have about a 40% chance of getting 90% coverage. With 1000 reads, you\n", + "have a 90% chance of getting 90% coverage.\n", + "\n", + "With that, we have answered the four questions that make up the unseen\n", + "species problem. To validate the algorithms in this chapter with real\n", + "data, I had to deal with a few more details. But this chapter is already\n", + "too long, so I won’t discuss them here.\n", + "\n", + "You can read about the problems, and how I addressed them, at\n", + ".\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Discussion\n", + "----------\n", + "\n", + "The Unseen Species problem is an area of active research, and I believe\n", + "the algorithm in this chapter is a novel contribution. So in fewer than\n", + "200 pages we have made it from the basics of probability to the research\n", + "frontier. I’m very happy about that.\n", + "\n", + "My goal for this book is to present three related ideas:\n", + "\n", + "- **Bayesian thinking**: The foundation of Bayesian\n", + " analysis is the idea of using probability distributions to represent\n", + " uncertain beliefs, using data to update those distributions, and\n", + " using the results to make predictions and inform decisions.\n", + "\n", + "- **A computational approach**: The premise of this book\n", + " is that it is easier to understand Bayesian analysis using\n", + " computation rather than math, and easier to implement Bayesian\n", + " methods with reusable building blocks that can be rearranged to\n", + " solve real-world problems quickly.\n", + "\n", + "- **Iterative modeling**: Most real-world problems\n", + " involve modeling decisions and trade-offs between realism and\n", + " complexity. It is often impossible to know ahead of time what\n", + " factors should be included in the model and which can be abstracted\n", + " away. The best approach is to iterate, starting with simple models\n", + " and adding complexity gradually, using each model to validate the\n", + " others.\n", + "\n", + "These ideas are versatile and powerful; they are applicable to problems\n", + "in every area of science and engineering, from simple examples to topics\n", + "of current research.\n", + "\n", + "If you made it this far, you should be prepared to apply these tools to\n", + "new problems relevant to your work. I hope you find them useful; let me\n", + "know how it goes!\n", + "\n", + "[^1]: Based on an example from\n", + " that is no longer\n", + " there.\n", + "\n", + "[^2]: Ruggles and Brodie, “An Empirical Approach to Economic\n", + " Intelligence in World War II,” *Journal of the American\n", + " Statistical Association*, Vol. 42, No. 237 (March 1947).\n", + "\n", + "[^3]: Zhang et al, Distribution of Renal Tumor Growth Rates Determined\n", + " by Using Serial Volumetric CT Measurements, January 2009\n", + " *Radiology*, 250, 137-144.\n", + "\n", + "[^4]: If you are not familiar with Python generators, see\n", + " .\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/Preface.ipynb b/notebooks/Preface.ipynb new file mode 100644 index 0000000..6075f89 --- /dev/null +++ b/notebooks/Preface.ipynb @@ -0,0 +1,545 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#Preface\n", + "\n", + "## My theory, which is mine\n", + "\n", + "The premise of this book, and the other books in the *Think X* series,\n", + "is that if you know how to program, you can use that skill to learn\n", + "other topics.\n", + "\n", + "Most books on Bayesian statistics use mathematical notation and present\n", + "ideas in terms of mathematical concepts like calculus. This book uses\n", + "Python code instead of math, and discrete approximations instead of\n", + "continuous mathematics. As a result, what would be an integral in a math\n", + "book becomes a summation, and most operations on probability\n", + "distributions are simple loops.\n", + "\n", + "I think this presentation is easier to understand, at least for people\n", + "with programming skills. It is also more general, because when we make\n", + "modeling decisions, we can choose the most appropriate model without\n", + "worrying too much about whether the model lends itself to conventional\n", + "analysis.\n", + "\n", + "Also, it provides a smooth development path from simple examples to\n", + "real-world problems. Chapter [estimation] is a good example. It starts\n", + "with a simple example involving dice, one of the staples of basic\n", + "probability. From there it proceeds in small steps to the locomotive\n", + "problem, which I borrowed from Mosteller’s *Fifty Challenging Problems\n", + "in Probability with Solutions*, and from there to the German tank\n", + "problem, a famously successful application of Bayesian methods during\n", + "World War II.\n", + "\n", + "##Modeling and approximation\n", + "\n", + "Most chapters in this book are motivated by a real-world problem, so\n", + "they involve some degree of modeling. Before we can apply Bayesian\n", + "methods (or any other analysis), we have to make decisions about which\n", + "parts of the real-world system to include in the model and which details\n", + "we can abstract away.\n", + "\n", + "For example, in Chapter [prediction], the motivating problem is to\n", + "predict the winner of a hockey game. I model goal-scoring as a Poisson\n", + "process, which implies that a goal is equally likely at any point in the\n", + "game. That is not exactly true, but it is probably a good enough model\n", + "for most purposes.\n", + "\n", + "In Chapter [evidence] the motivating problem is interpreting SAT scores\n", + "(the SAT is a standardized test used for college admissions in the\n", + "United States). I start with a simple model that assumes that all SAT\n", + "questions are equally difficult, but in fact the designers of the SAT\n", + "deliberately include some questions that are relatively easy and some\n", + "that are relatively hard. I present a second model that accounts for\n", + "this aspect of the design, and show that it doesn’t have a big effect on\n", + "the results after all.\n", + "\n", + "I think it is important to include modeling as an explicit part of\n", + "problem solving because it reminds us to think about modeling errors\n", + "(that is, errors due to simplifications and assumptions of the model).\n", + "\n", + "Many of the methods in this book are based on discrete distributions,\n", + "which makes some people worry about numerical errors. But for real-world\n", + "problems, numerical errors are almost always smaller than modeling\n", + "errors.\n", + "\n", + "Furthermore, the discrete approach often allows better modeling\n", + "decisions, and I would rather have an approximate solution to a good\n", + "model than an exact solution to a bad model.\n", + "\n", + "On the other hand, continuous methods sometimes yield performance\n", + "advantages—for example by replacing a linear- or quadratic-time\n", + "computation with a constant-time solution.\n", + "\n", + "So I recommend a general process with these steps:\n", + "\n", + "1. While you are exploring a problem, start with simple models and\n", + " implement them in code that is clear, readable, and demonstrably\n", + " correct. Focus your attention on good modeling decisions, not\n", + " optimization.\n", + "\n", + "2. Once you have a simple model working, identify the biggest sources of\n", + " error. You might need to increase the number of values in a discrete\n", + " approximation, or increase the number of iterations in a Monte Carlo\n", + " simulation, or add details to the model.\n", + "\n", + "3. If the performance of your solution is good enough for your\n", + " application, you might not have to do any optimization. But if you\n", + " do, there are two approaches to consider. You can review your code\n", + " and look for optimizations; for example, if you cache previously\n", + " computed results you might be able to avoid redundant computation. Or\n", + " you can look for analytic methods that yield computational shortcuts.\n", + "\n", + "One benefit of this process is that Steps 1 and 2 tend to be fast, so\n", + "you can explore several alternative models before investing heavily in\n", + "any of them.\n", + "\n", + "Another benefit is that if you get to Step 3, you will be starting with\n", + "a reference implementation that is likely to be correct, which you can\n", + "use for regression testing (that is, checking that the optimized code\n", + "yields the same results, at least approximately).\n", + "\n", + "Working with the code\n", + "---------------------\n", + "\n", + "Many of the examples in this book use classes and functions defined in\n", + "thinkbayes.py. You can download this module from\n", + "http://thinkbayes.com/thinkbayes.py.\n", + "\n", + "Most chapters contain references to code you can download from\n", + "http://thinkbayes.com. Some of those files have dependencies you will\n", + "also have to download. I suggest you keep all of these files in the same\n", + "directory so they can import each other without changing the Python\n", + "search path.\n", + "\n", + "You can download these files one at a time as you need them, or you can\n", + "download them all at once from\n", + "http://thinkbayes.com/thinkbayes_code.zip. This file also contains the\n", + "data files used by some of the programs. When you unzip it, it creates a\n", + "directory named ``thinkbayes_code`` that contains all the code used in\n", + "this book.\n", + "\n", + "Or, if you are a Git user, you can get all of the files at once by\n", + "forking and cloning this repository:\n", + "https://github.com/AllenDowney/ThinkBayes.\n", + "\n", + "One of the modules I use is thinkplot.py, which provides wrappers for\n", + "some of the functions in pyplot. To use it, you need to install\n", + "matplotlib. If you don’t already have it, check your package manager to\n", + "see if it is available. Otherwise you can get download instructions from\n", + "http://matplotlib.org.\n", + "\n", + "Finally, some programs in this book use NumPy and SciPy, which are\n", + "available from http://numpy.org and http://scipy.org.\n", + "\n", + "Code style\n", + "----------\n", + "\n", + "Experienced Python programmers will notice that the code in this book\n", + "does not comply with PEP 8, which is the most common style guide for\n", + "Python (http://www.python.org/dev/peps/pep-0008/).\n", + "\n", + "Specifically, PEP 8 calls for lowercase function names with underscores\n", + "between words, ``like_this``. In this book and the accompanying code,\n", + "function and method names begin with a capital letter and use camel\n", + "case, ``LikeThis``.\n", + "\n", + "I broke this rule because I developed some of the code while I was a\n", + "Visiting Scientist at Google, so I followed the Google style guide,\n", + "which deviates from PEP 8 in a few places. Once I got used to Google\n", + "style, I found that I liked it. And at this point, it would be too much\n", + "trouble to change.\n", + "\n", + "Also on the topic of style, I write “Bayes’s theorem” with an *s* after\n", + "the apostrophe, which is preferred in some style guides and deprecated\n", + "in others. I don’t have a strong preference. I had to choose one, and\n", + "this is the one I chose.\n", + "\n", + "And finally one typographical note: throughout the book, I use PMF and\n", + "CDF for the mathematical concept of a probability mass function or\n", + "cumulative distribution function, and Pmf and Cdf to refer to the Python\n", + "objects I use to represent them.\n", + "\n", + "Prerequisites\n", + "-------------\n", + "\n", + "There are several excellent modules for doing Bayesian statistics in\n", + "Python, including pymc and OpenBUGS. I chose not to use them for this\n", + "book because you need a fair amount of background knowledge to get\n", + "started with these modules, and I want to keep the prerequisites\n", + "minimal. If you know Python and a little bit about probability, you are\n", + "ready to start this book.\n", + "\n", + "Chapter [intro] is about probability and Bayes’s theorem; it has no\n", + "code. Chapter [compstat] introduces Pmf, a thinly disguised Python\n", + "dictionary I use to represent a probability mass function (PMF). Then\n", + "Chapter [estimation] introduces Suite, a kind of Pmf that provides a\n", + "framework for doing Bayesian updates. And that’s just about all there is\n", + "to it.\n", + "\n", + "Well, almost. In some of the later chapters, I use analytic\n", + "distributions including the Gaussian (normal) distribution, the\n", + "exponential and Poisson distributions, and the beta distribution. In\n", + "Chapter [species] I break out the less-common Dirichlet distribution,\n", + "but I explain it as I go along. If you are not familiar with these\n", + "distributions, you can read about them on Wikipedia. You could also read\n", + "the companion to this book, *Think Stats*, or an introductory statistics\n", + "book (although I’m afraid most of them take a mathematical approach that\n", + "is not particularly helpful for practical purposes).\n", + "\n", + "Contributor List\n", + "----------------\n", + "\n", + "If you have a suggestion or correction, please send email to\n", + "*downey@allendowney.com*. If I make a change based on your feedback, I\n", + "will add you to the contributor list (unless you ask to be omitted).\n", + "\n", + "If you include at least part of the sentence the error appears in, that\n", + "makes it easy for me to search. Page and section numbers are fine, too,\n", + "but not as easy to work with. Thanks!\n", + "\n", + "- First, I have to acknowledge David MacKay’s excellent book,\n", + " *Information Theory, Inference, and Learning Algorithms*, which is\n", + " where I first came to understand Bayesian methods. With his\n", + " permission, I use several problems from his book as examples.\n", + "\n", + "- This book also benefited from my interactions with Sanjoy Mahajan,\n", + " especially in fall 2012, when I audited his class on Bayesian\n", + " Inference at Olin College.\n", + "\n", + "- I wrote parts of this book during project nights with the Boston\n", + " Python User Group, so I would like to thank them for their company\n", + " and pizza.\n", + "\n", + "- Jonathan Edwards sent in the first typo.\n", + "\n", + "- George Purkins found a markup error.\n", + "\n", + "- Olivier Yiptong sent several helpful suggestions.\n", + "\n", + "- Yuriy Pasichnyk found several errors.\n", + "\n", + "- Kristopher Overholt sent a long list of corrections and suggestions.\n", + "\n", + "- Robert Marcus found a misplaced *i*.\n", + "\n", + "- Max Hailperin suggested a clarification in Chapter [intro].\n", + "\n", + "- Markus Dobler pointed out that drawing cookies from a bowl with\n", + " replacement is an unrealistic scenario.\n", + "\n", + "- Tom Pollard and Paul A. Giannaros spotted a version problem with some\n", + " of the numbers in the train example.\n", + "\n", + "- Ram Limbu found a typo and suggested a clarification.\n", + "\n", + "- In spring 2013, students in my class, Computational Bayesian\n", + " Statistics, made many helpful corrections and suggestions: Kai\n", + " Austin, Claire Barnes, Kari Bender, Rachel Boy, Kat Mendoza, Arjun\n", + " Iyer, Ben Kroop, Nathan Lintz, Kyle McConnaughay, Alec Radford,\n", + " Brendan Ritter, and Evan Simpson.\n", + "\n", + "- Greg Marra and Matt Aasted helped me clarify the discussion of *The\n", + " Price is Right* problem.\n", + "\n", + "- Marcus Ogren pointed out that the original statement of the\n", + " locomotive problem was ambiguous.\n", + "\n", + "- Jasmine Kwityn and Dan Fauxsmith at O’Reilly Media proofread the book\n", + " and found many opportunities for improvement.\n", + "\n", + "- James Lawry spotted a math error.\n", + "\n", + "- Ben Kahle found a reference to the wrong figure.\n", + "\n", + "- Jeffrey Law found an inconsistency between the text and the code.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "# Preface\n", + "\n", + "##My theory, which is mine\n", + "\n", + "\n", + "The premise of this book, and the other books in the *Think\n", + "X* series, is that if you know how to program, you can use that\n", + "skill to learn other topics.\n", + "\n", + "Most books on Bayesian statistics use mathematical notation and present\n", + "ideas in terms of mathematical concepts like calculus. This book uses\n", + "Python code instead of math, and discrete approximations instead of\n", + "continuous mathematics. As a result, what would be an integral in a math\n", + "book becomes a summation, and most operations on probability\n", + "distributions are simple loops.\n", + "\n", + "I think this presentation is easier to understand, at least for people\n", + "with programming skills. It is also more general, because when we make\n", + "modeling decisions, we can choose the most appropriate model without\n", + "worrying too much about whether the model lends itself to conventional\n", + "analysis.\n", + "\n", + "Also, it provides a smooth development path from simple examples to\n", + "real-world problems. Chapter [estimation] is a good example. It starts\n", + "with a simple example involving dice, one of the staples of basic\n", + "probability. From there it proceeds in small steps to the locomotive\n", + "problem, which I borrowed from Mosteller’s *Fifty Challenging\n", + "Problems in Probability with Solutions*, and from there to the\n", + "German tank problem, a famously successful application of Bayesian\n", + "methods during World War II.\n", + "\n", + "##Modeling and approximation\n", + "\n", + "Most chapters in this book are motivated by a real-world problem, so\n", + "they involve some degree of modeling. Before we can apply Bayesian\n", + "methods (or any other analysis), we have to make decisions about which\n", + "parts of the real-world system to include in the model and which details\n", + "we can abstract away.\n", + "\n", + "For example, in Chapter [prediction], the motivating problem is to\n", + "predict the winner of a hockey game. I model goal-scoring as a Poisson\n", + "process, which implies that a goal is equally likely at any point in the\n", + "game. That is not exactly true, but it is probably a good enough model\n", + "for most purposes.\n", + "\n", + "In Chapter [evidence] the motivating problem is interpreting SAT scores\n", + "(the SAT is a standardized test used for college admissions in the\n", + "United States). I start with a simple model that assumes that all SAT\n", + "questions are equally difficult, but in fact the designers of the SAT\n", + "deliberately include some questions that are relatively easy and some\n", + "that are relatively hard. I present a second model that accounts for\n", + "this aspect of the design, and show that it doesn’t have a big effect on\n", + "the results after all.\n", + "\n", + "I think it is important to include modeling as an explicit part of\n", + "problem solving because it reminds us to think about modeling errors\n", + "(that is, errors due to simplifications and assumptions of the model).\n", + "\n", + "Many of the methods in this book are based on discrete distributions,\n", + "which makes some people worry about numerical errors. But for real-world\n", + "problems, numerical errors are almost always smaller than modeling\n", + "errors.\n", + "\n", + "Furthermore, the discrete approach often allows better modeling\n", + "decisions, and I would rather have an approximate solution to a good\n", + "model than an exact solution to a bad model.\n", + "\n", + "On the other hand, continuous methods sometimes yield performance\n", + "advantages—for example by replacing a linear- or quadratic-time\n", + "computation with a constant-time solution.\n", + "\n", + "So I recommend a general process with these steps:\n", + "\n", + "1. While you are exploring a problem, start with simple models and\n", + " implement them in code that is clear, readable, and demonstrably\n", + " correct. Focus your attention on good modeling decisions, not\n", + " optimization.\n", + "\n", + "2. Once you have a simple model working, identify the biggest sources\n", + " of error. You might need to increase the number of values in a\n", + " discrete approximation, or increase the number of iterations in a\n", + " Monte Carlo simulation, or add details to the model.\n", + "\n", + "3. If the performance of your solution is good enough for your\n", + " application, you might not have to do any optimization. But if you\n", + " do, there are two approaches to consider. You can review your code\n", + " and look for optimizations; for example, if you cache previously\n", + " computed results you might be able to avoid redundant computation.\n", + " Or you can look for analytic methods that yield computational\n", + " shortcuts.\n", + "\n", + "One benefit of this process is that Steps 1 and 2 tend to be fast, so\n", + "you can explore several alternative models before investing heavily in\n", + "any of them.\n", + "\n", + "Another benefit is that if you get to Step 3, you will be starting with\n", + "a reference implementation that is likely to be correct, which you can\n", + "use for regression testing (that is, checking that the optimized code\n", + "yields the same results, at least approximately).\n", + "\n", + "## Working with the code {#download}\n", + "\n", + "Many of the examples in this book use classes and functions defined in\n", + "`thinkbayes.py`. You can download this module from\n", + ".\n", + "\n", + "Most chapters contain references to code you can download from\n", + ". Some of those files have dependencies you will\n", + "also have to download. I suggest you keep all of these files in the same\n", + "directory so they can import each other without changing the Python\n", + "search path.\n", + "\n", + "You can download these files one at a time as you need them, or you can\n", + "download them all at once from\n", + ". This file also contains the\n", + "data files used by some of the programs. When you unzip it, it creates a\n", + "directory named `thinkbayes_code` that contains all the code used in\n", + "this book.\n", + "\n", + "Or, if you are a Git user, you can get all of the files at once by\n", + "forking and cloning this repository:\n", + ".\n", + "\n", + "One of the modules I use is `thinkplot.py`, which provides\n", + "wrappers for some of the functions in `pyplot`. To use it,\n", + "you need to install `matplotlib`. If you don’t already have\n", + "it, check your package manager to see if it is available. Otherwise you\n", + "can get download instructions from .\n", + "\n", + "Finally, some programs in this book use NumPy and SciPy, which are\n", + "available from and .\n", + "\n", + "Code style\n", + "----------\n", + "\n", + "Experienced Python programmers will notice that the code in this book\n", + "does not comply with PEP 8, which is the most common style guide for\n", + "Python ().\n", + "\n", + "Specifically, PEP 8 calls for lowercase function names with underscores\n", + "between words, `like_this`. In this book and the accompanying code,\n", + "function and method names begin with a capital letter and use camel\n", + "case, `LikeThis`.\n", + "\n", + "I broke this rule because I developed some of the code while I was a\n", + "Visiting Scientist at Google, so I followed the Google style guide,\n", + "which deviates from PEP 8 in a few places. Once I got used to Google\n", + "style, I found that I liked it. And at this point, it would be too much\n", + "trouble to change.\n", + "\n", + "Also on the topic of style, I write “Bayes’s theorem” with an\n", + "*s* after the apostrophe, which is preferred in some style\n", + "guides and deprecated in others. I don’t have a strong preference. I had\n", + "to choose one, and this is the one I chose.\n", + "\n", + "And finally one typographical note: throughout the book, I use PMF and\n", + "CDF for the mathematical concept of a probability mass function or\n", + "cumulative distribution function, and Pmf and Cdf to refer to the Python\n", + "objects I use to represent them.\n", + "\n", + "##Prerequisites\n", + "\n", + "There are several excellent modules for doing Bayesian statistics in\n", + "Python, including `pymc` and OpenBUGS. I chose not to use\n", + "them for this book because you need a fair amount of background\n", + "knowledge to get started with these modules, and I want to keep the\n", + "prerequisites minimal. If you know Python and a little bit about\n", + "probability, you are ready to start this book.\n", + "\n", + "Chapter [intro] is about probability and Bayes’s theorem; it has no\n", + "code. Chapter [compstat] introduces `Pmf`, a thinly disguised\n", + "Python dictionary I use to represent a probability mass function (PMF).\n", + "Then Chapter [estimation] introduces `Suite`, a kind of Pmf\n", + "that provides a framework for doing Bayesian updates. And that’s just\n", + "about all there is to it.\n", + "\n", + "Well, almost. In some of the later chapters, I use analytic\n", + "distributions including the Gaussian (normal) distribution, the\n", + "exponential and Poisson distributions, and the beta distribution. In\n", + "Chapter [species] I break out the less-common Dirichlet distribution,\n", + "but I explain it as I go along. If you are not familiar with these\n", + "distributions, you can read about them on Wikipedia. You could also read\n", + "the companion to this book, *Think Stats*, or an\n", + "introductory statistics book (although I’m afraid most of them take a\n", + "mathematical approach that is not particularly helpful for practical\n", + "purposes).\n", + "\n", + "Contributor List {#contributor-list .unnumbered}\n", + "----------------\n", + "\n", + "If you have a suggestion or correction, please send email to\n", + "*downey@allendowney.com*. If I make a change based on your\n", + "feedback, I will add you to the contributor list (unless you ask to be\n", + "omitted).\n", + "\n", + "If you include at least part of the sentence the error appears in, that\n", + "makes it easy for me to search. Page and section numbers are fine, too,\n", + "but not as easy to work with. Thanks!\n", + "\n", + "- First, I have to acknowledge David MacKay’s excellent book,\n", + " *Information Theory, Inference, and Learning\n", + " Algorithms*, which is where I first came to understand\n", + " Bayesian methods. With his permission, I use several problems from\n", + " his book as examples.\n", + "\n", + "- This book also benefited from my interactions with Sanjoy Mahajan,\n", + " especially in fall 2012, when I audited his class on Bayesian\n", + " Inference at Olin College.\n", + "\n", + "- I wrote parts of this book during project nights with the Boston\n", + " Python User Group, so I would like to thank them for their company\n", + " and pizza.\n", + "\n", + "- Jonathan Edwards sent in the first typo.\n", + "\n", + "- George Purkins found a markup error.\n", + "\n", + "- Olivier Yiptong sent several helpful suggestions.\n", + "\n", + "- Yuriy Pasichnyk found several errors.\n", + "\n", + "- Kristopher Overholt sent a long list of corrections and suggestions.\n", + "\n", + "- Robert Marcus found a misplaced *i*.\n", + "\n", + "- Max Hailperin suggested a clarification in Chapter [intro].\n", + "\n", + "- Markus Dobler pointed out that drawing cookies from a bowl with\n", + " replacement is an unrealistic scenario.\n", + "\n", + "- Tom Pollard and Paul A. Giannaros spotted a version problem with\n", + " some of the numbers in the train example.\n", + "\n", + "- Ram Limbu found a typo and suggested a clarification.\n", + "\n", + "- In spring 2013, students in my class, Computational Bayesian\n", + " Statistics, made many helpful corrections and suggestions: Kai\n", + " Austin, Claire Barnes, Kari Bender, Rachel Boy, Kat Mendoza, Arjun\n", + " Iyer, Ben Kroop, Nathan Lintz, Kyle McConnaughay, Alec Radford,\n", + " Brendan Ritter, and Evan Simpson.\n", + "\n", + "- Greg Marra and Matt Aasted helped me clarify the discussion of\n", + " *The Price is Right* problem.\n", + "\n", + "- Marcus Ogren pointed out that the original statement of the\n", + " locomotive problem was ambiguous.\n", + "\n", + "- Jasmine Kwityn and Dan Fauxsmith at O’Reilly Media proofread the\n", + " book and found many opportunities for improvement.\n", + "\n", + "- James Lawry spotted a math error.\n", + "\n", + "- Ben Kahle found a reference to the wrong figure.\n", + "\n", + "- Jeffrey Law found an inconsistency between the text and the code." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From 7093971ca2424b7f7351641697184f2556dd206d Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Tue, 4 Aug 2015 12:45:05 -0700 Subject: [PATCH 02/10] Make code work in both Python 2 and 3. --- .gitignore | 3 ++ code/thinkbayes.py | 96 +++++++++++++++++++++++++++++----------------- 2 files changed, 64 insertions(+), 35 deletions(-) diff --git a/.gitignore b/.gitignore index d2d6f36..daf010b 100644 --- a/.gitignore +++ b/.gitignore @@ -33,3 +33,6 @@ nosetests.xml .mr.developer.cfg .project .pydevproject + +# Jupiter Notebooks +.ipynb_checkpoints/ \ No newline at end of file diff --git a/code/thinkbayes.py b/code/thinkbayes.py index 0c0208a..3fa4b4e 100644 --- a/code/thinkbayes.py +++ b/code/thinkbayes.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -18,7 +20,6 @@ Pdf: represents a continuous probability density function """ - import bisect import copy import logging @@ -29,6 +30,31 @@ import scipy.stats from scipy.special import erf, erfinv + +try: + dict.iteritems +except AttributeError: + # Python 3 + def listvalues(d): + return list(d.values()) + def listitems(d): + return list(d.items()) + def itervalues(d): + return iter(d.values()) + def iteritems(d): + return iter(d.items()) +else: + # Python 2 + def listvalues(d): + return d.values() + def listitems(d): + return d.items() + # Python 2 + def itervalues(d): + return d.itervalues() + def iteritems(d): + return d.iteritems() + ROOT2 = math.sqrt(2) def RandomSeed(x): @@ -38,7 +64,7 @@ def RandomSeed(x): """ random.seed(x) numpy.random.seed(x) - + def Odds(p): """Computes odds for a given probability. @@ -74,7 +100,7 @@ def Probability2(yes, no): """Computes the probability corresponding to given odds. Example: yes=2, no=1 means 2:1 odds in favor, or 2/3 probability. - + yes, no: int or float odds in favor """ return float(yes) / (yes + no) @@ -159,7 +185,7 @@ def InitMapping(self, values): values: map from value to probability """ - for value, prob in values.iteritems(): + for value, prob in iteritems(values): self.Set(value, prob) def InitPmf(self, values): @@ -216,7 +242,7 @@ def Scale(self, factor): def Log(self, m=None): """Log transforms the probabilities. - + Removes values with probability 0. Normalizes so that the largest logprob is 0. @@ -228,7 +254,7 @@ def Log(self, m=None): if m is None: m = self.MaxLike() - for x, p in self.d.iteritems(): + for x, p in iteritems(self.d): if p: self.Set(x, math.log(p / m)) else: @@ -248,7 +274,7 @@ def Exp(self, m=None): if m is None: m = self.MaxLike() - for x, p in self.d.iteritems(): + for x, p in iteritems(self.d): self.Set(x, math.exp(p - m)) def GetDict(self): @@ -282,8 +308,8 @@ def Render(self): def Print(self): """Prints the values and freqs/probs in ascending order.""" - for val, prob in sorted(self.d.iteritems()): - print val, prob + for val, prob in sorted(iteritems(self.d)): + print(val, prob) def Set(self, x, y=0): """Sets the freq/prob associated with the value x. @@ -324,12 +350,12 @@ def Remove(self, x): def Total(self): """Returns the total of the frequencies/probabilities in the map.""" - total = sum(self.d.itervalues()) + total = sum(itervalues(self.d)) return total def MaxLike(self): """Returns the largest frequency/probability in the map.""" - return max(self.d.itervalues()) + return max(itervalues(self.d)) class Hist(_DictWrapper): @@ -369,7 +395,7 @@ def Subtract(self, other): class Pmf(_DictWrapper): """Represents a probability mass function. - + Values can be any hashable type; probabilities are floating-point. Pmfs are not necessarily normalized. """ @@ -401,7 +427,7 @@ def ProbGreater(self, x): returns: float probability """ - t = [prob for (val, prob) in self.d.iteritems() if val > x] + t = [prob for (val, prob) in iteritems(self.d) if val > x] return sum(t) def ProbLess(self, x): @@ -411,7 +437,7 @@ def ProbLess(self, x): returns: float probability """ - t = [prob for (val, prob) in self.d.iteritems() if val < x] + t = [prob for (val, prob) in iteritems(self.d) if val < x] return sum(t) def __lt__(self, obj): @@ -511,7 +537,7 @@ def Random(self): target = random.random() total = 0.0 - for x, p in self.d.iteritems(): + for x, p in iteritems(self.d): total += p if total >= target: return x @@ -526,7 +552,7 @@ def Mean(self): float mean """ mu = 0.0 - for x, p in self.d.iteritems(): + for x, p in iteritems(self.d): mu += p * x return mu @@ -544,7 +570,7 @@ def Var(self, mu=None): mu = self.Mean() var = 0.0 - for x, p in self.d.iteritems(): + for x, p in iteritems(self.d): var += p * (x - mu) ** 2 return var @@ -978,7 +1004,7 @@ def Random(self): def Sample(self, n): """Generates a random sample from this distribution. - + Args: n: int length of the sample """ @@ -1093,7 +1119,7 @@ def MakeCdfFromDict(d, name=''): Returns: Cdf object """ - return MakeCdfFromItems(d.iteritems(), name) + return MakeCdfFromItems(iteritems(d), name) def MakeCdfFromHist(hist, name=''): @@ -1223,7 +1249,7 @@ def LogLikelihood(self, data, hypo): def Print(self): """Prints the hypotheses and their probabilities.""" for hypo, prob in sorted(self.Items()): - print hypo, prob + print(hypo, prob) def MakeOdds(self): """Transforms from probabilities to odds. @@ -1485,7 +1511,7 @@ def SampleSum(dists, n): returns: new Pmf of sums """ - pmf = MakePmfFromList(RandomSum(dists) for i in xrange(n)) + pmf = MakePmfFromList(RandomSum(dists) for i in range(n)) return pmf @@ -1495,7 +1521,7 @@ def EvalGaussianPdf(x, mu, sigma): x: value mu: mean sigma: standard deviation - + returns: float probability density """ return scipy.stats.norm.pdf(x, mu, sigma) @@ -1503,7 +1529,7 @@ def EvalGaussianPdf(x, mu, sigma): def MakeGaussianPmf(mu, sigma, num_sigmas, n=201): """Makes a PMF discrete approx to a Gaussian distribution. - + mu: float mean sigma: float standard deviation num_sigmas: how many sigmas to extend in each direction @@ -1528,7 +1554,7 @@ def EvalBinomialPmf(k, n, p): Returns the probabily of k successes in n trials with probability p. """ return scipy.stats.binom.pmf(k, n, p) - + def EvalPoissonPmf(k, lam): """Computes the Poisson PMF. @@ -1554,7 +1580,7 @@ def MakePoissonPmf(lam, high, step=1): returns: normalized Pmf """ pmf = Pmf() - for k in xrange(0, high + 1, step): + for k in range(0, high + 1, step): p = EvalPoissonPmf(k, lam) pmf.Set(k, p) pmf.Normalize() @@ -1596,13 +1622,13 @@ def MakeExponentialPmf(lam, high, n=200): def StandardGaussianCdf(x): """Evaluates the CDF of the standard Gaussian distribution. - + See http://en.wikipedia.org/wiki/Normal_distribution #Cumulative_distribution_function Args: x: float - + Returns: float """ @@ -1611,14 +1637,14 @@ def StandardGaussianCdf(x): def GaussianCdf(x, mu=0, sigma=1): """Evaluates the CDF of the gaussian distribution. - + Args: x: float mu: mean parameter - + sigma: standard deviation parameter - + Returns: float """ @@ -1628,15 +1654,15 @@ def GaussianCdf(x, mu=0, sigma=1): def GaussianCdfInverse(p, mu=0, sigma=1): """Evaluates the inverse CDF of the gaussian distribution. - See http://en.wikipedia.org/wiki/Normal_distribution#Quantile_function + See http://en.wikipedia.org/wiki/Normal_distribution#Quantile_function Args: p: float mu: mean parameter - + sigma: standard deviation parameter - + Returns: float """ @@ -1701,14 +1727,14 @@ def MakePmf(self, steps=101, name=''): pmf = cdf.MakePmf() return pmf - xs = [i / (steps - 1.0) for i in xrange(steps)] + xs = [i / (steps - 1.0) for i in range(steps)] probs = [self.EvalPdf(x) for x in xs] pmf = MakePmfFromDict(dict(zip(xs, probs)), name) return pmf def MakeCdf(self, steps=101): """Returns the CDF of this distribution.""" - xs = [i / (steps - 1.0) for i in xrange(steps)] + xs = [i / (steps - 1.0) for i in range(steps)] ps = [scipy.special.betainc(self.alpha, self.beta, x) for x in xs] cdf = Cdf(xs, ps) return cdf From 47a27c9bedabb3e31414f33c57b9db72c00d3021 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Tue, 4 Aug 2015 20:31:08 -0700 Subject: [PATCH 03/10] Converted chapters 4-7. Start on chapter 8. --- .gitignore | 7 +- code/538.json | 47 ++ code/book_format.py | 127 +++ code/columns.py | 4 +- code/custom.css | 226 ++++++ code/dungeons.py | 13 +- code/hockey.py | 34 +- code/price.py | 108 ++- code/python23.py | 30 + code/thinkbayes.py | 29 +- code/thinkplot.py | 41 +- notebooks/04_More_Estimation.ipynb | 65 +- notebooks/05_Odds_and_Addends.ipynb | 871 +++++++++++++++----- notebooks/06_Decision_Analysis.ipynb | 1097 +++++++++++++++++++------- notebooks/07_Prediction.ipynb | 838 ++++++++++++++++---- notebooks/08_Observer_Bias.ipynb | 588 ++++++++++---- 16 files changed, 3221 insertions(+), 904 deletions(-) create mode 100644 code/538.json create mode 100644 code/book_format.py create mode 100644 code/custom.css create mode 100644 code/python23.py diff --git a/.gitignore b/.gitignore index daf010b..2d8b549 100644 --- a/.gitignore +++ b/.gitignore @@ -35,4 +35,9 @@ nosetests.xml .pydevproject # Jupiter Notebooks -.ipynb_checkpoints/ \ No newline at end of file +.ipynb_checkpoints/ + +# generated plots +*.eps +*.pdf +*.png \ No newline at end of file diff --git a/code/538.json b/code/538.json new file mode 100644 index 0000000..5aaa0c7 --- /dev/null +++ b/code/538.json @@ -0,0 +1,47 @@ +{ + "lines.linewidth": 2.0, + "patch.linewidth": 0.5, + "legend.fancybox": true, + "axes.color_cycle": [ + "#6d904f", + "#013afe", + "#202020", + "#fc4f30", + "#e5ae38", + "#A60628", + "#30a2da", + "#008080", + "#7A68A6", + "#CF4457", + "#188487", + "#E24A33" + ], + "axes.facecolor": "#ffffff", + "axes.labelsize": "large", + "axes.axisbelow": true, + "axes.grid": true, + "patch.edgecolor": "#f0f0f0", + "axes.titlesize": "x-large", + "examples.directory": "", + "figure.facecolor": "#ffffff", + "grid.linestyle": "-", + "grid.linewidth": 2.0, + "grid.color": "#cbcbcb", + "axes.edgecolor":"#f0f0f0", + "xtick.major.size": 0, + "xtick.minor.size": 0, + "ytick.major.size": 0, + "ytick.minor.size": 0, + "axes.linewidth": 3.0, + "font.size":14.0, + "lines.linewidth": 3, + "lines.solid_capstyle": "butt", + "savefig.edgecolor": "#f0f0f0", + "savefig.facecolor": "#f0f0f0", + "figure.subplot.left" : 0.08, + "figure.subplot.right" : 0.95, + "figure.subplot.bottom" : 0.07, + "figure.subplot.hspace" : 0.5, + "legend.scatterpoints" : 1 +} + diff --git a/code/book_format.py b/code/book_format.py new file mode 100644 index 0000000..7aa75f1 --- /dev/null +++ b/code/book_format.py @@ -0,0 +1,127 @@ +# -*- coding: utf-8 -*- + +"""Copyright 2015 Roger R Labbe Jr. + + +Code supporting the book + +Kalman and Bayesian Filters in Python +https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python + + +This is licensed under an MIT license. See the LICENSE.txt file +for more information. +""" + +from __future__ import (absolute_import, division, print_function, + unicode_literals) + +from contextlib import contextmanager +from IPython.core.display import HTML +import json +import matplotlib.pylab as pylab +import matplotlib.pyplot as plt +import numpy as np +import os.path +import sys + + +sys.path.insert(0, './code') # allow us to import book_format + +def test_filterpy_version(): + import filterpy + min_version = [0,0,25] + v = filterpy.__version__ + tokens = v.split('.') + for i,v in enumerate(tokens): + if int(v) > min_version[i]: + return + + i = len(tokens) - 1 + if min_version[i] > int(tokens[i]): + raise Exception("Minimum FilterPy version supported is {}.{}.{}.\n" + "Please install a more recent version.\n" + " ex: pip install filterpy --upgrade".format( + *min_version)) + v = int(tokens[0]*1000) + + +# ensure that we have the correct filterpy loaded. This is +# called when this module is imported at the top of each book +# chapter so the reader can see that they need to update FilterPy. +test_filterpy_version() + + +def equal_axis(): + pylab.rcParams['figure.figsize'] = 10,10 + plt.axis('equal') + + +def reset_axis(): + pylab.rcParams['figure.figsize'] = 11, 4 + +def set_figsize(x=11, y=4): + pylab.rcParams['figure.figsize'] = x, y + + +@contextmanager +def figsize(x=11, y=4): + """Temporarily set the figure size using 'with figsize(a,b):'""" + + size = pylab.rcParams['figure.figsize'] + set_figsize(x, y) + yield + pylab.rcParams['figure.figsize'] = size + +@contextmanager +def numpy_precision(precision): + old = np.get_printoptions()['precision'] + np.set_printoptions(precision=precision) + yield + np.set_printoptions(old) + +@contextmanager +def printoptions(*args, **kwargs): + original = np.get_printoptions() + np.set_printoptions(*args, **kwargs) + yield + np.set_printoptions(**original) + +def _decode_list(data): + rv = [] + for item in data: + if isinstance(item, unicode): + item = item.encode('utf-8') + elif isinstance(item, list): + item = _decode_list(item) + elif isinstance(item, dict): + item = _decode_dict(item) + rv.append(item) + return rv + +def _decode_dict(data): + rv = {} + for key, value in data.iteritems(): + if isinstance(key, unicode): + key = key.encode('utf-8') + if isinstance(value, unicode): + value = value.encode('utf-8') + elif isinstance(value, list): + value = _decode_list(value) + elif isinstance(value, dict): + value = _decode_dict(value) + rv[key] = value + return rv + + +def load_style(directory = '.', name='custom.css'): + if sys.version_info[0] >= 3: + s = json.load(open(os.path.join(directory, "538.json"))) + else: + s = json.load(open(directory + "538.json"), object_hook=_decode_dict) + plt.rcParams.update(s) + reset_axis () + np.set_printoptions(suppress=True) + + styles = open(os.path.join(directory, name), 'r').read() + return HTML(styles) diff --git a/code/columns.py b/code/columns.py index d720228..f05eaf7 100644 --- a/code/columns.py +++ b/code/columns.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code related to "Think Stats", by Allen B. Downey, available from greenteapress.com @@ -47,7 +49,7 @@ def print_cols(cols): cols: list of columns """ for i, col in enumerate(cols): - print i, col[0], col[1] + print(i, col[0], col[1]) def make_col_dict(cols, names): diff --git a/code/custom.css b/code/custom.css new file mode 100644 index 0000000..6597f5d --- /dev/null +++ b/code/custom.css @@ -0,0 +1,226 @@ + + diff --git a/code/dungeons.py b/code/dungeons.py index d185976..2a592c6 100644 --- a/code/dungeons.py +++ b/code/dungeons.py @@ -23,10 +23,15 @@ def __init__(self, sides, name=''): name: string """ thinkbayes.Pmf.__init__(self, name=name) - for x in xrange(1, sides+1): + self.__hash__ = thinkbayes.Pmf.__hash__ + for x in range(1, sides+1): self.Set(x, 1) self.Normalize() + def __hash__(self): + return id(self) + + def PmfMax(pmf1, pmf2): """Computes the distribution of the max of values drawn from two Pmfs. @@ -40,7 +45,7 @@ def PmfMax(pmf1, pmf2): for v2, p2 in pmf2.Items(): res.Incr(max(v1, v2), p1*p2) return res - + def main(): pmf_dice = thinkbayes.Pmf() @@ -89,7 +94,7 @@ def main(): thinkplot.Clf() thinkplot.PrePlot(num=1) - + # compute the distribution of the best attribute the hard way best_attr2 = PmfMax(three_exact, three_exact) best_attr4 = PmfMax(best_attr2, best_attr2) @@ -108,7 +113,7 @@ def main(): ylabel='Probability', axis=[2, 19, 0, 0.23], formats=FORMATS) - + if __name__ == '__main__': diff --git a/code/hockey.py b/code/hockey.py index 4a92191..e1bcb0f 100644 --- a/code/hockey.py +++ b/code/hockey.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -34,7 +36,7 @@ def __init__(self, name=''): pmf = thinkbayes.MakeGaussianPmf(mu, sigma, 4) thinkbayes.Suite.__init__(self, pmf, name=name) - + def Likelihood(self, data, hypo): """Computes the likelihood of the data under the hypothesis. @@ -48,6 +50,8 @@ def Likelihood(self, data, hypo): like = thinkbayes.EvalPoissonPmf(k, lam) return like + def __hash__(self): + return id(self) def MakeGoalPmf(suite, high=10): """Makes the distribution of goals scored, given distribution of lam. @@ -150,9 +154,9 @@ def ProcessScoresPairwise(pairs): thinkplot.Show() mu, var = thinkstats.MeanVar(lams) - print 'mu, sig', mu, math.sqrt(var) + print('mu, sig', mu, math.sqrt(var)) - print 'BOS v VAN', pairs['BOS', 'VAN'] + print('BOS v VAN', pairs['BOS', 'VAN']) def ProcessScoresTeamwise(pairs): @@ -181,7 +185,7 @@ def ProcessScoresTeamwise(pairs): thinkplot.Show() mu, var = thinkstats.MeanVar(lams) - print 'mu, sig', mu, math.sqrt(var) + print('mu, sig', mu, math.sqrt(var)) def main(): @@ -227,16 +231,16 @@ def main(): ylabel='Probability', formats=formats) - time_dist1 = MakeGoalTimePmf(suite1) + time_dist1 = MakeGoalTimePmf(suite1) time_dist2 = MakeGoalTimePmf(suite2) - - print 'MLE bruins', suite1.MaximumLikelihood() - print 'MLE canucks', suite2.MaximumLikelihood() - + + print('MLE bruins', suite1.MaximumLikelihood()) + print('MLE canucks', suite2.MaximumLikelihood()) + thinkplot.Clf() thinkplot.PrePlot(num=2) thinkplot.Pmf(time_dist1) - thinkplot.Pmf(time_dist2) + thinkplot.Pmf(time_dist2) thinkplot.Save(root='hockey3', xlabel='Games until goal', ylabel='Probability', @@ -247,16 +251,16 @@ def main(): p_loss = diff.ProbLess(0) p_tie = diff.Prob(0) - print p_win, p_loss, p_tie + print(p_win, p_loss, p_tie) p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2) p_adjust = thinkbayes.PmfProbEqual(time_dist1, time_dist2) p_overtime += p_adjust / 2 - print 'p_overtime', p_overtime + print('p_overtime', p_overtime) - print p_overtime * p_tie + print(p_overtime * p_tie) p_win += p_overtime * p_tie - print 'p_win', p_win + print('p_win', p_win) # win the next two p_series = p_win**2 @@ -264,7 +268,7 @@ def main(): # split the next two, win the third p_series += 2 * p_win * (1-p_win) * p_win - print 'p_series', p_series + print('p_series', p_series) if __name__ == '__main__': diff --git a/code/price.py b/code/price.py index 449397c..ac92a22 100644 --- a/code/price.py +++ b/code/price.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -7,6 +9,7 @@ import csv import numpy +import os import thinkbayes import thinkplot @@ -16,7 +19,7 @@ FORMATS = ['png', 'pdf', 'eps'] -def ReadData(filename='showcases.2011.csv'): +def ReadData(path='.', filename='showcases.2011.csv'): """Reads a CSV file of data. Args: @@ -24,7 +27,9 @@ def ReadData(filename='showcases.2011.csv'): Returns: sequence of (price1 price2 bid1 bid2 diff1 diff2) tuples """ - fp = open(filename) + + fullname = os.path.join(path, filename) + fp = open(fullname) reader = csv.reader(fp) res = [] @@ -39,8 +44,8 @@ def ReadData(filename='showcases.2011.csv'): pass fp.close() - return zip(*res) - + return list(zip(*res)) + class Price(thinkbayes.Suite): """Represents hypotheses about the price of a showcase.""" @@ -90,7 +95,7 @@ def ExpectedGains(self, low=0, high=75000, n=101): n: number of bids to evaluates returns: tuple (sequence of bids, sequence of gains) - + """ bids = numpy.linspace(low, high, n) @@ -135,7 +140,7 @@ def ProbWin(self, diff): diff: how much your bid was off by """ - prob = (self.opponent.ProbOverbid() + + prob = (self.opponent.ProbOverbid() + self.opponent.ProbWorseThan(diff)) return prob @@ -196,7 +201,7 @@ def MakeBeliefs(self, guess): Sets attributes prior and posterior. - guess: what the player thinks the showcase is worth + guess: what the player thinks the showcase is worth """ pmf = self.PmfPrice() self.prior = Price(pmf, self, name='prior') @@ -205,8 +210,8 @@ def MakeBeliefs(self, guess): def OptimalBid(self, guess, opponent): """Computes the bid that maximizes expected return. - - guess: what the player thinks the showcase is worth + + guess: what the player thinks the showcase is worth opponent: Player Returns: (optimal bid, expected gain) @@ -230,6 +235,27 @@ def PlotBeliefs(self, root): ylabel='PMF', formats=FORMATS) +def MakePrice1(player1, player2): + """ plot the prior distribution of price for both players""" + thinkplot.Clf() + thinkplot.PrePlot(num=2) + pmf1 = player1.PmfPrice() + pmf1.name = 'showcase 1' + pmf2 = player2.PmfPrice() + pmf2.name = 'showcase 2' + thinkplot.Pmfs([pmf1, pmf2]) + +def MakePrice2(player1, player2): + """ plot the historical distribution of underness for both players""" + + thinkplot.Clf() + thinkplot.PrePlot(num=2) + cdf1 = player1.CdfDiff() + cdf1.name = 'player 1' + cdf2 = player2.CdfDiff() + cdf2.name = 'player 2' + thinkplot.Cdfs([cdf1, cdf2]) + def MakePlots(player1, player2): """Generates two plots. @@ -239,18 +265,13 @@ def MakePlots(player1, player2): """ # plot the prior distribution of price for both players - thinkplot.Clf() - thinkplot.PrePlot(num=2) - pmf1 = player1.PmfPrice() - pmf1.name = 'showcase 1' - pmf2 = player2.PmfPrice() - pmf2.name = 'showcase 2' - thinkplot.Pmfs([pmf1, pmf2]) + MakePrice1(player1, player2) thinkplot.Save(root='price1', xlabel='price ($)', ylabel='PDF', formats=FORMATS) + # plot the historical distribution of underness for both players thinkplot.Clf() thinkplot.PrePlot(num=2) @@ -259,11 +280,11 @@ def MakePlots(player1, player2): cdf2 = player2.CdfDiff() cdf2.name = 'player 2' - print 'Player median', cdf1.Percentile(50) - print 'Player median', cdf2.Percentile(50) + print('Player median', cdf1.Percentile(50)) + print('Player median', cdf2.Percentile(50)) - print 'Player 1 overbids', player1.ProbOverbid() - print 'Player 2 overbids', player2.ProbOverbid() + print('Player 1 overbids', player1.ProbOverbid()) + print('Player 2 overbids', player2.ProbOverbid()) thinkplot.Cdfs([cdf1, cdf2]) thinkplot.Save(root='price2', @@ -272,10 +293,10 @@ def MakePlots(player1, player2): formats=FORMATS) -def MakePlayers(): +def MakePlayers(path='.'): """Reads data and makes player objects.""" - data = ReadData(filename='showcases.2011.csv') - data += ReadData(filename='showcases.2012.csv') + data = ReadData(path=path, filename='./showcases.2011.csv') + data += ReadData(path=path, filename='showcases.2012.csv') cols = zip(*data) price1, price2, bid1, bid2, diff1, diff2 = cols @@ -289,24 +310,24 @@ def MakePlayers(): return player1, player2 -def PlotExpectedGains(guess1=20000, guess2=40000): +def PlotExpectedGains(guess1=20000, guess2=40000, path='.', save=True): """Plots expected gains as a function of bid. guess1: player1's estimate of the price of showcase 1 guess2: player2's estimate of the price of showcase 2 """ - player1, player2 = MakePlayers() + player1, player2 = MakePlayers(path) MakePlots(player1, player2) player1.MakeBeliefs(guess1) player2.MakeBeliefs(guess2) - print 'Player 1 prior mle', player1.prior.MaximumLikelihood() - print 'Player 2 prior mle', player2.prior.MaximumLikelihood() - print 'Player 1 mean', player1.posterior.Mean() - print 'Player 2 mean', player2.posterior.Mean() - print 'Player 1 mle', player1.posterior.MaximumLikelihood() - print 'Player 2 mle', player2.posterior.MaximumLikelihood() + print('Player 1 prior mle', player1.prior.MaximumLikelihood()) + print('Player 2 prior mle', player2.prior.MaximumLikelihood()) + print('Player 1 mean', player1.posterior.Mean()) + print('Player 2 mean', player2.posterior.Mean()) + print('Player 1 mle', player1.posterior.MaximumLikelihood()) + print('Player 2 mle', player2.posterior.MaximumLikelihood()) player1.PlotBeliefs('price3') player2.PlotBeliefs('price4') @@ -319,22 +340,23 @@ def PlotExpectedGains(guess1=20000, guess2=40000): bids, gains = calc1.ExpectedGains() thinkplot.Plot(bids, gains, label='Player 1') - print 'Player 1 optimal bid', max(zip(gains, bids)) + print('Player 1 optimal bid', max(zip(gains, bids))) bids, gains = calc2.ExpectedGains() thinkplot.Plot(bids, gains, label='Player 2') - print 'Player 2 optimal bid', max(zip(gains, bids)) + print('Player 2 optimal bid', max(zip(gains, bids))) - thinkplot.Save(root='price5', - xlabel='bid ($)', - ylabel='expected gain ($)', - formats=FORMATS) + if save: + thinkplot.Save(root='price5', + xlabel='bid ($)', + ylabel='expected gain ($)', + formats=FORMATS) -def PlotOptimalBid(): +def PlotOptimalBid(path='.'): """Plots optimal bid vs estimated price. """ - player1, player2 = MakePlayers() + player1, player2 = MakePlayers(path) guesses = numpy.linspace(15000, 60000, 21) res = [] @@ -351,7 +373,7 @@ def PlotOptimalBid(): res.append((guess, mean, mle, gain, bid)) guesses, means, _mles, gains, bids = zip(*res) - + thinkplot.PrePlot(num=3) pyplot.plot([15000, 60000], [15000, 60000], color='gray') thinkplot.Plot(guesses, means, label='mean') @@ -370,14 +392,14 @@ def TestCode(calc): """ # test ProbWin for diff in [0, 100, 1000, 10000, 20000]: - print diff, calc.ProbWin(diff) + print(diff, calc.ProbWin(diff)) print # test Return price = 20000 for bid in [17000, 18000, 19000, 19500, 19800, 20001]: - print bid, calc.Gain(bid, price) - print + print(bid, calc.Gain(bid, price)) + print() def main(): diff --git a/code/python23.py b/code/python23.py new file mode 100644 index 0000000..4e0c20e --- /dev/null +++ b/code/python23.py @@ -0,0 +1,30 @@ +# -*- coding: utf-8 -*- +""" +Created on Tue Aug 4 13:26:26 2015 + +@author: RL +""" + +try: + dict.iteritems +except AttributeError: + # Python 3 + def listvalues(d): + return list(d.values()) + def listitems(d): + return list(d.items()) + def itervalues(d): + return iter(d.values()) + def iteritems(d): + return iter(d.items()) +else: + # Python 2 + def listvalues(d): + return d.values() + def listitems(d): + return d.items() + # Python 2 + def itervalues(d): + return d.itervalues() + def iteritems(d): + return d.iteritems() \ No newline at end of file diff --git a/code/thinkbayes.py b/code/thinkbayes.py index 3fa4b4e..1e30f60 100644 --- a/code/thinkbayes.py +++ b/code/thinkbayes.py @@ -20,6 +20,7 @@ Pdf: represents a continuous probability density function """ +from python23 import * import bisect import copy import logging @@ -31,29 +32,6 @@ from scipy.special import erf, erfinv -try: - dict.iteritems -except AttributeError: - # Python 3 - def listvalues(d): - return list(d.values()) - def listitems(d): - return list(d.items()) - def itervalues(d): - return iter(d.values()) - def iteritems(d): - return iter(d.items()) -else: - # Python 2 - def listvalues(d): - return d.values() - def listitems(d): - return d.items() - # Python 2 - def itervalues(d): - return d.itervalues() - def iteritems(d): - return d.iteritems() ROOT2 = math.sqrt(2) @@ -503,6 +481,9 @@ def __ne__(self, obj): """ return 1 - (self == obj) + def __hash__(self): + return id(self) + def Normalize(self, fraction=1.0): """Normalizes this PMF so the sum of all probs is fraction. @@ -1403,7 +1384,7 @@ def Density(self, x): def MakePmf(self, xs, name=''): ps = self.kde.evaluate(xs) - pmf = MakePmfFromItems(zip(xs, ps), name=name) + pmf = MakePmfFromItems(list(zip(xs, ps)), name=name) return pmf diff --git a/code/thinkplot.py b/code/thinkplot.py index 3a10e92..c164dcf 100644 --- a/code/thinkplot.py +++ b/code/thinkplot.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Stats", by Allen B. Downey, available from greenteapress.com @@ -9,6 +11,7 @@ import matplotlib import matplotlib.pyplot as pyplot import numpy as np +from python23 import * # customize some matplotlib attributes #matplotlib.rc('figure', figsize=(4, 3)) @@ -29,7 +32,7 @@ class Brewer(object): Shades of blue that look good in color and can be distinguished in grayscale (up to a point). - + Borrowed from http://colorbrewer2.org/ """ color_iter = None @@ -104,7 +107,7 @@ def PrePlot(num=None, rows=1, cols=1): global SUBPLOT_ROWS, SUBPLOT_COLS SUBPLOT_ROWS = rows SUBPLOT_COLS = cols - + def SubPlot(plot_number): pyplot.subplot(SUBPLOT_ROWS, SUBPLOT_COLS, plot_number) @@ -141,7 +144,7 @@ def Underride(d, **options): if d is None: d = {} - for key, val in options.iteritems(): + for key, val in iteritems(options): d.setdefault(key, val) return d @@ -151,13 +154,13 @@ def Clf(): """Clears the figure and any hints that have been set.""" Brewer.ClearIter() pyplot.clf() - + def Figure(**options): """Sets options for the current figure.""" Underride(options, figsize=(6, 8)) pyplot.figure(**options) - + def Plot(xs, ys, style='', **options): """Plots a line. @@ -172,11 +175,13 @@ def Plot(xs, ys, style='', **options): if color_iter: try: - options = Underride(options, color=color_iter.next()) + #options = Underride(options, color=color_iter.next()) + options = Underride(options, color=next(color_iter)) except StopIteration: - print 'Warning: Brewer ran out of colors.' + print('Warning: Brewer ran out of colors.') Brewer.ClearIter() - + + options = Underride(options, linewidth=3, alpha=0.8) pyplot.plot(xs, ys, style, **options) @@ -188,7 +193,7 @@ def Scatter(xs, ys, **options): ys: y values options: options passed to pyplot.scatter """ - options = Underride(options, color='blue', alpha=0.2, + options = Underride(options, color='blue', alpha=0.2, s=30, edgecolors='none') pyplot.scatter(xs, ys, **options) @@ -211,7 +216,7 @@ def Pmfs(pmfs, **options): Options are passed along for all PMFs. If you want different options for each pmf, make multiple calls to Pmf. - + Args: pmfs: sequence of PMF objects options: keyword args passed to pyplot.plot @@ -234,7 +239,7 @@ def Hist(hist, **options): if hist.name: options = Underride(options, label=hist.name) - options = Underride(options, + options = Underride(options, align='center', linewidth=0, width=width) @@ -319,7 +324,7 @@ def Cdf(cdf, complement=False, transform=None, **options): def Cdfs(cdfs, complement=False, transform=None, **options): """Plots a sequence of CDFs. - + cdfs: sequence of CDF objects complement: boolean, whether to plot the complementary CDF transform: string, one of 'exponential', 'pareto', 'weibull', 'gumbel' @@ -331,7 +336,7 @@ def Cdfs(cdfs, complement=False, transform=None, **options): def Contour(obj, pcolor=False, contour=True, imshow=False, **options): """Makes a contour plot. - + d: map from (x, y) to z, or object that provides GetDict pcolor: boolean, whether to make a pseudocolor plot contour: boolean, whether to make a contour plot @@ -366,11 +371,11 @@ def Contour(obj, pcolor=False, contour=True, imshow=False, **options): if imshow: extent = xs[0], xs[-1], ys[0], ys[-1] pyplot.imshow(Z, extent=extent, **options) - + def Pcolor(xs, ys, zs, pcolor=True, contour=False, **options): """Makes a pseudocolor plot. - + xs: ys: zs: @@ -393,7 +398,7 @@ def Pcolor(xs, ys, zs, pcolor=True, contour=False, **options): if contour: cs = pyplot.contour(X, Y, Z, **options) pyplot.clabel(cs, inline=1, fontsize=10) - + def Config(**options): """Configures the plot. @@ -473,7 +478,7 @@ def SaveFormat(root, fmt='eps'): fmt: string format """ filename = '%s.%s' % (root, fmt) - print 'Writing', filename + print('Writing', filename) pyplot.savefig(filename, format=fmt, dpi=300) @@ -501,7 +506,7 @@ def SaveFormat(root, fmt='eps'): def main(): color_iter = Brewer.ColorGenerator(7) for color in color_iter: - print color + print(color) if __name__ == '__main__': main() diff --git a/notebooks/04_More_Estimation.ipynb b/notebooks/04_More_Estimation.ipynb index 17b5178..520092f 100644 --- a/notebooks/04_More_Estimation.ipynb +++ b/notebooks/04_More_Estimation.ipynb @@ -2,20 +2,11 @@ "cells": [ { "cell_type": "code", - "execution_count": 15, + "execution_count": 1, "metadata": { "collapsed": false }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "The autoreload extension is already loaded. To reload it, use:\n", - " %reload_ext autoreload\n" - ] - } - ], + "outputs": [], "source": [ "%matplotlib inline\n", "import sys\n", @@ -86,7 +77,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 2, "metadata": { "collapsed": false }, @@ -112,7 +103,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 3, "metadata": { "collapsed": true }, @@ -134,7 +125,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 4, "metadata": { "collapsed": false }, @@ -143,7 +134,7 @@ "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHCFJREFUeJzt3XmQXPV57vHvw2hjl9i0R8JGFBIGGwoLXS/QuUXuHcsY\n3SpCQI5NjBND5Vo2cSgugbp1PforccouMKWA5SBTBDuRgSQuQVAwWztO4gjJLMFIwhK2QAItICEW\nBaztvX+cHtRqZno9p0/39POpmqL7bP3OKebpn96zKSIwM7PecUTeBZiZWXs5+M3MeoyD38ysxzj4\nzcx6jIPfzKzHOPjNzHpMzeCX1C9pvaQNkm4YYv4Zkn4m6V1J1w0xv0/SU5LuT6toMzNrXtXgl9QH\nLAH6gTnAQkmzKxbbCXwF+OYwm7kWWAv4ggEzsw5Qa8Q/F9gYEZsiYh+wHFhQvkBEvBoRa4B9lStL\nmgbMB+4AlE7JZmbWilrBPxXYXPZ+S2lavW4GrgcONliXmZllpFbwN92ekXQxsCMinsKjfTOzjjGq\nxvyXgell76eTjPrr8THgEknzgXHAcZL+JiKuLF9Iknv/ZmZNiIimBtW1RvxrgFmSZkoaA1wOrBhm\n2cMKiIibImJ6RJwKXAE8Vhn6Zcv6J4Kvf/3rudfQKT/eF94X3hfVf1pRdcQfEfslLQIeAvqAZRGx\nTtI1pflLJU0CVgPHAQclXQvMiYi3KzfXUqVmZpaKWq0eImIlsLJi2tKy19s4vB001DZ+AvykyRrN\nzCxFvnK3gxQKhbxL6BjeF4d4XxzifZEOtdorarkAKfKuwcys20giMjq4a2ZmI4yD38ysxzj4zcx6\njIPfzKzHOPjNzHqMg9/MrMc4+M3MeoyD38ysxzj4zcx6jIPfzKzHOPjNzHqMg9/MrMc4+M3MeoyD\n36xN/vRP4V/+Je8qzOp4EIuZte6tt+C222DHDrjggryrsV7nEb9ZGzzwAJx1FvzTP8G77+ZdjfU6\nB79ZG9xzDyxaBGefDT/+cd7VWK/zE7jMMvbmmzBtGrz0Enz/+7BqFdx9d95VWbfzE7jMOtgDDyR9\n/fHj4dJLk/e/+U3eVVkvc/CbZeyee+Cyy5LXkycnvf6HH863JuttDn6zDL35Jjz+OCxYcGjaZZfB\nvffmV5NZXcEvqV/SekkbJN0wxPwzJP1M0ruSriubPl3S45Kek/QLSV9Ns3izTnf//fDJTyZtnkGX\nXppMd7vH8lIz+CX1AUuAfmAOsFDS7IrFdgJfAb5ZMX0f8LWIOBOYB3x5iHXNRqx774Xf+73Dp02Z\nAmeeCY88kk9NZvWM+OcCGyNiU0TsA5YDC8oXiIhXI2INSdCXT98WEU+XXr8NrAOmpFK5WYd7990k\n3C+55P3zfvd34Uc/an9NZlBf8E8FNpe931Ka1hBJM4FzgFWNrmvWjV56CSZNOrzNM+gjH4H169tf\nkxnUd8uGlk+yl3QMcB9wbWnkf5iBgYH3XhcKBQqFQqsfaZa7TZtgxoyh582YAS++2NZyrMsVi0WK\nxWIq26p5AZekecBARPSX3t8IHIyIbwyx7NeBtyPiW2XTRgMPACsj4pYh1vEFXDYiffe7ycVay5a9\nf96+fXD00bBnD4we3f7arPtlfQHXGmCWpJmSxgCXAyuGq6WiMAHLgLVDhb7ZSPbii8OP+EePTs7p\n37KlvTWZQR3BHxH7gUXAQ8Ba4IcRsU7SNZKuAZA0SdJm4GvA/5X0Uqm983Hgc8BvS3qq9NOf2W9j\n1kE2bYKZM4ef73aP5aWu2zJHxEpgZcW0pWWvtwHTh1j1X/FFYtajXnzRwW+dyaFslpFqB3fBwW/5\ncfCbZWDv3uShK1OrnPg8Y0by5WDWbg5+swxs3pyE/qgqzVSP+C0vDn6zDNRq80DS/3fwWx4c/GYZ\nqHVgF+C3fis5nfPgwbaUZPYeB79ZBmqdygkwblxyO4dt29pRkdkhDn6zDNTT6gEf4LV8OPjNMlBP\nqwd8gNfy4eA3y0A9rR5w8Fs+HPxmKdu/P+nbT5tWe1mf2WN5cPCbpWzLFpg4sb67bnrEb3lw8Jul\nrN42D/jgruXDwW+Wsmq3Y640OOL3IymsnRz8ZilrZMR/3HEwZgzs3JllRWaHc/CbpayRET/4AK+1\nn4PfLGWNjPjBfX5rPwe/WcqaCX6P+K2dHPxmKTpwAF5+GaYP9Ty6YTj4rd0c/GYpeuUVOOkkGDu2\n/nUc/NZuDn6zFNV7j55yDn5rNwe/WYrqvStnuZkzfXDX2svBb5airVthypTG1jnxRHj3XdizJ5ua\nzCo5+M1StH17cp+eRkhwyinJw9nN2qFm8Evql7Re0gZJNwwx/wxJP5P0rqTrGlnXbKRpJvghWWf7\n9vTrMRtK1eCX1AcsAfqBOcBCSbMrFtsJfAX4ZhPrmo0o27Y5+K3z1RrxzwU2RsSmiNgHLAcWlC8Q\nEa9GxBpgX6Prmo0027fDpEmNr+fgt3aqFfxTgc1l77eUptWjlXXNulKzrZ5TTnHwW/uMqjG/lZvF\n1r3uwMDAe68LhQKFQqGFjzXLx4EDsGtXcgFXoyZOhI0b06/JRo5isUixWExlW7WC/2Wg/OLz6SQj\n93rUvW558Jt1q1dfhQkTYFStv6ohTJwI//Zv6ddkI0floHjx4sVNb6tWq2cNMEvSTEljgMuBFcMs\nqxbWNet6zfb3wT1+a6+qY5OI2C9pEfAQ0Acsi4h1kq4pzV8qaRKwGjgOOCjpWmBORLw91LpZ/jJm\neWq2vw8Ofmuvmv8ojYiVwMqKaUvLXm/j8JZO1XXNRqpWgt8XcFk7+cpds5Q0ew4/wAknwFtvwd69\n6dZkNhQHv1lKWunxH3EEnHyyR/3WHg5+s5S00uoB9/mtfRz8Zilx8Fu3cPCbpaSVHj/4AK+1j4Pf\nLCUe8Vu3cPCbpWDwdg0nn9z8Nhz81i4OfrMUtHK7hkEOfmsXB79ZClpt84CD39rHwW+WgjSC3wd3\nrV0c/GYpaOXirUEe8Vu7OPjNUtDqqZyQHBjetSs5UGyWJQe/WQrSaPWMGgXjx8Nrr6VTk9lwHPxm\nKUgj+CHZhvv8ljUHv1kK0ujxg5+9a+3h4DdLQRo9fvABXmsPB79ZCtJs9Tj4LWsOfrMWpXG7hkEO\nfmsHB79Zi157rfXbNQzywV1rBwe/WYvS6u+DD+5aezj4zVqUVn8f3Oqx9nDwm7XIwW/dxsFv1qK0\nzuGHQzdqi0hne2ZDqRn8kvolrZe0QdINwyxza2n+M5LOKZt+o6TnJD0r6W8ljU2zeLNOkGaPf9w4\nOPJI2L07ne2ZDaVq8EvqA5YA/cAcYKGk2RXLzAdOi4hZwNXA7aXpM4EvAedGxFlAH3BFyvWb5S7N\nVg+43WPZqzXinwtsjIhNEbEPWA4sqFjmEuAugIhYBYyXNBF4E9gHHCVpFHAU8HKaxZt1Age/dZta\nwT8V2Fz2fktpWs1lImIX8C3gJeAVYHdEPNJauWadZ8cOB791l1qXnNR7iEnvmyB9EPgTYCbwBnCv\npN+PiB9ULjswMPDe60KhQKFQqPNjzfK3fXtyUDYtEycmz/A1K1csFikWi6lsq1bwvwxML3s/nWRE\nX22ZaaVpBeDfI2IngKR/AD4GVA1+s25y8GBy5W4at2sYdPLJvnrX3q9yULx48eKmt1Wr1bMGmCVp\npqQxwOXAioplVgBXAkiaR9LS2Q48D8yTdKQkARcBa5uu1KwD7doFxx4LY8akt00/e9eyVnXEHxH7\nJS0CHiI5K2dZRKyTdE1p/tKIeFDSfEkbgT3AVaV5T0v6G5Ivj4PAk8B3M/xdzNou7f4+OPgtezVv\nKxURK4GVFdOWVrxfNMy6fwn8ZSsFmnWyHTvS7e+Dg9+y5yt3zVqQ9oFdcPBb9hz8Zi1wq8e6kYPf\nrAVZjPjHj4e334a9e9PdrtkgB79ZC7IY8R9xRHJKp8/lt6w4+M1akMWIH9zusWw5+M1akMWIHxz8\nli0Hv1kLsjidExz8li0Hv1kL3OqxbuTgN2vSnj1w4EByy4a0OfgtSw5+syYNtnn0vnvTts7Bb1ly\n8Js1KasDu+Dgt2w5+M2alNWBXXDwW7Yc/GZNyurALjj4LVsOfrMmZdnqGXwYS9T7DDyzBjj4zZqU\n5Yj/6KOhry+5Z49Z2hz8Zk3KcsQPbvdYdhz8Zk3KcsQPDn7LjoPfrEke8Vu3cvCbNSnL0znBwW/Z\ncfCbNWH/fnj9dTjxxOw+w8FvWXHwmzVh506YMAFGjcruMxz8lhUHv1kTsj6wCw5+y46D36wJWR/Y\nBQe/Zadm8Evql7Re0gZJNwyzzK2l+c9IOqds+nhJ90laJ2mtpHlpFm+WF4/4rZtVDX5JfcASoB+Y\nAyyUNLtimfnAaRExC7gauL1s9reBByNiNnA2sC7F2s1yk/UZPeDgt+zUGvHPBTZGxKaI2AcsBxZU\nLHMJcBdARKwCxkuaKOl44JMR8b3SvP0R8Ua65Zvlox2tnpNOgl27koe9mKWpVvBPBTaXvd9SmlZr\nmWnAqcCrku6U9KSkv5Z0VKsFm3WCdrR6Ro2C449Pwt8sTbVORqv33oCVzyCK0rbPBRZFxGpJtwB/\nBvy/ypUHBgbee10oFCgUCnV+rFk+2jHih0PtnpNPzv6zrLMVi0WKxWIq26oV/C8D08veTycZ0Vdb\nZlppmoAtEbG6NP0+kuB/n/LgN+sG7Rjxw6HgP/PM7D/LOlvloHjx4sVNb6tWq2cNMEvSTEljgMuB\nFRXLrACuBCidtbM7IrZHxDZgs6TTS8tdBDzXdKVmHaTdI36zNFUd8UfEfkmLgIeAPmBZRKyTdE1p\n/tKIeFDSfEkbgT3AVWWb+Arwg9KXxgsV88y6UkT72i8OfstCzQvOI2IlsLJi2tKK94uGWfcZ4KOt\nFGjWad56K3lIytFHZ/9ZDn7Lgq/cNWvQ1q0weXJ7PsvBb1lw8Js16JVXYMqU9nzWKackB5LN0uTg\nN2vQ1q3tC/5Jk2DbtvZ8lvUOB79Zg9o54p8yJfmiMUuTg9+sQe0M/kmTkuA/eLA9n2e9wcFv1qB2\nBv+4cXDsscmDX8zS4uA3a1A7gx/c7rH0OfjNGtTu4J88OflMs7Q4+M0aEJGEcLvO44fkS8bBb2ly\n8Js14I03YPRoOOaY9n3m5Mlu9Vi6HPxmDWh3mwc84rf0OfjNGuDgt5HAwW/WgDyC360eS5uD36wB\nHvHbSODgN2tAHsE/eL8eX71raXHwmzUgj+D31buWNge/WQPyCH5wu8fS5eA3a0Bewe8DvJYmB79Z\nnSLa+/Stch7xW5oc/GZ12rkzuWJ33Lj2f7aD39Lk4DerU15tHnCrx9Ll4DerU57B7xG/pcnBb1an\nvEf8Dn5LS83gl9Qvab2kDZJuGGaZW0vzn5F0TsW8PklPSbo/raLN8pD3iN+tHktL1eCX1AcsAfqB\nOcBCSbMrlpkPnBYRs4CrgdsrNnMtsBaItIo2y0PeI35fvWtpqTXinwtsjIhNEbEPWA4sqFjmEuAu\ngIhYBYyXNBFA0jRgPnAHoDQLN2u3PIN/7Njk6t3XXsvn821kqRX8U4HNZe+3lKbVu8zNwPWAxynW\n9dr95K1KbvdYWkbVmF9ve6ZyNC9JFwM7IuIpSYVqKw8MDLz3ulAoUChUXdwsF3mO+OHQmT0f/nB+\nNVh+isUixWIxlW0pYvhslzQPGIiI/tL7G4GDEfGNsmW+AxQjYnnp/XqgAHwV+DywHxgHHAf8fURc\nWfEZUa0Gs05w4EBy4daePTBmTD41fOELcMEF8MUv5vP51lkkERFNtdBrtXrWALMkzZQ0BrgcWFGx\nzArgylIh84DdEbEtIm6KiOkRcSpwBfBYZeibdYtXX4UJE/ILffC5/Jaeqq2eiNgvaRHwENAHLIuI\ndZKuKc1fGhEPSpovaSOwB7hquM2lWbhZO23dmm+bB5LjC+vW5VuDjQy1evxExEpgZcW0pRXvF9XY\nxk+AnzRToFknyLu/D8nnP/ZYvjXYyOArd83q8PLL+Z7RA271WHoc/GZ12LQJZs7MtwbftsHS4uA3\nq8MLL8AHP5hvDZMnw/btvnrXWufgN6tDJwT/2LEwfnxy6wazVjj4zerQCcEP8IEPJLWYtcLBb1bD\nrl1Je+XEE/OuJPnycfBbqxz8ZjUMjvbVAbcZdPBbGhz8ZjV0SpsHkjp+9au8q7Bu5+A3q6HTgt8j\nfmuVg9+sBge/jTQOfrMaOin4J02C//ovePPNvCuxbubgN6uhk4Jf8imd1joHv1kV77yTPO5w2rS8\nKznE7R5rlYPfrIpf/xpmzIC+vrwrOcTBb61y8JtV0UltnkEOfmuVg9+sCge/jUQOfrMqHPw2Ejn4\nzap44YXkLJpOMmNG8ijIvXvzrsS6lYPfrIpOHPGPHg1TpyYPhzFrhoPfbBgHDsCLL3beiB/c7rHW\nOPjNhrFlS3Ir5iOPzLuS93PwWysc/GbD6MQ2zyAHv7XCwW82DAe/jVR1Bb+kfknrJW2QdMMwy9xa\nmv+MpHNK06ZLelzSc5J+IemraRZvliUHv41UNYNfUh+wBOgH5gALJc2uWGY+cFpEzAKuBm4vzdoH\nfC0izgTmAV+uXNesU3Vy8H/gA8ntJA4ezLsS60b1jPjnAhsjYlNE7AOWAwsqlrkEuAsgIlYB4yVN\njIhtEfF0afrbwDpgSmrVm2Xo+efh9NPzrmJoxx6b/Gzdmncl1o3qCf6pwOay91tK02otc9j9DCXN\nBM4BVjVapFm77dkDGzfChz6UdyXDc7vHmjWqjmWizm1VPor6vfUkHQPcB1xbGvkfZmBg4L3XhUKB\nQqFQ50eaZePJJ+Gss2Ds2LwrGd5pp8GGDXDBBXlXYu1QLBYpFoupbEsR1XNd0jxgICL6S+9vBA5G\nxDfKlvkOUIyI5aX364ELI2K7pNHAA8DKiLhliO1HrRrM2u1b30p66EuW5F3J8G6+OQn+227LuxLL\ngyQionLAXZd6Wj1rgFmSZkoaA1wOrKhYZgVwZamYecDuUugLWAasHSr0zTrV6tUwd27eVVR3/vmw\nyo1Ta0LN4I+I/cAi4CFgLfDDiFgn6RpJ15SWeRD4laSNwFLgf5dW/zjwOeC3JT1V+unP4hcxS9MT\nT3R+8J9zDqxblzwlzKwRNVs9mRfgVo91mNdeSw6cvv46HNHhlziedx58+9vw8Y/nXYm1W9atHrOe\nsnp1EqidHvrgdo81pwv+1zZrryeegI9+NO8q6nP++fAf/5F3FdZtHPxmFbrhwO6gefM84rfGOfjN\nykR0x4HdQbNmwVtvwbZteVdi3cTBb1bmpZegry95wlU3kJIvKY/6rREOfrMyg6N9NXWuRD58gNca\n5eA3K9NNB3YHOfitUQ5+szLddGB30Ny5sGZN8oxgs3o4+M1KDhxIbs523nl5V9KYk06Ck0+G9evz\nrsS6hYPfrGT1apg2DU44Ie9KGufTOq0RDn6zku9/HxYuzLuK5rjPb43wvXrMgL17k1M4n3gCTj01\n72oa95//CZ/5THIr6W641YS1zvfqMWvRP/8znHFGd4Y+wNlnJy2qRx/NuxLrBg5+M+Duu+Hzn8+7\nitb80R/BsmV5V2HdwK0e63m7d8OMGbBpE0yYkHc1zXv99eRfLC+8ACeemHc1ljW3esxacO+98Du/\n092hD0n9F1+cHKQ2q8bBbz1vJLR5Bv3hH8IddyQ3mzMbjoPfetqmTcnjCz/1qbwrSceFFyaPYly9\nOu9KrJM5+K2n3XILXHEFjBmTdyXpOOII+OIXfZDXqvPBXetZjz4KV14JzzyT3PZgpHjlFfjQh2DD\nBh/kHcl8cNesQTt3whe+AHfeObJCH2DKlKTX/9nP+sZtNjSP+K3nRMBll8H06XDzzXlXk439+6G/\nP7nh3F/8Rd7VWBYyHfFL6pe0XtIGSTcMs8ytpfnPSDqnkXXN2u3OO+GXv4Q///O8K8nOqFGwfDn8\n8Idwzz15V2OdpmrwS+oDlgD9wBxgoaTZFcvMB06LiFnA1cDt9a5rhysWi3mX0DGy2Bd79sB118FN\nN8EPfgDjxqX+EZlodl+cdBL84z/CokXw05+mW1Ne/DeSjloj/rnAxojYFBH7gOXAgoplLgHuAoiI\nVcB4SZPqXNfK+H/qQ9LeF48+mtzPZts2ePZZOOusVDefqVb2xUc+kvwL57OfhUsvTU5d7Wb+G0nH\nqBrzpwKby95vAc6vY5mpwJQ61jVL3TvvJAH/5JPwyCNJ6O/dC3/1V/DpT+ddXft9+tNJa2vJkuQ8\n/0IBPvEJOPfc5IvhmGPyrtDarVbw13vUtaVHU3/mM62sPXI8/zz8/Od5V9EZnn8+eZwgHLoKNeLQ\nz8GDyRkr+/fDb36ThP0778COHcl/J02CM8+Eiy6CP/7j5PTGXr5d8ZFHwvXXw5e+lNyi4uc/T27t\n8OyzMHp0cruHCRPgqKOS96NGJT9Sst+k9z+APo8H0vf638jixckXdquqntUjaR4wEBH9pfc3Agcj\n4htly3wHKEbE8tL79cCFwKm11i1N9yk9ZmZNaPasnloj/jXALEkzgVeAy4HKZxStABYBy0tfFLsj\nYruknXWs23ThZmbWnKrBHxH7JS0CHgL6gGURsU7SNaX5SyPiQUnzJW0E9gBXVVs3y1/GzMxqy/0C\nLjMza69cD3f18gVekqZLelzSc5J+IemrpeknSHpY0i8l/VjS+LxrbQdJfZKeknR/6X1P7gcASeMl\n3SdpnaS1ks7vxf0h6cbS38ezkv5W0the2Q+Svidpu6Rny6YN+7uX9tWGUp7+j1rbzy34fYEX+4Cv\nRcSZwDzgy6Xf/8+AhyPidODR0vtecC2wlkNnkvXqfgD4NvBgRMwGzgbW02P7o3Rs8EvAuRFxFkm7\n+Ap6Zz/cSZKN5Yb83SXNITmGOqe0zm2SqmZ7niP+nr7AKyK2RcTTpddvA+tIrn9474K40n//Vz4V\nto+kacB84A4OnRrcc/sBQNLxwCcj4nuQHCuLiDfovf3xJsng6ChJo4CjSE4S6Yn9EBE/BV6vmDzc\n774A+LuI2BcRm4CNJPk6rDyDf7gLv3pOaXRzDrAKmBgR20uztgMTcyqrnW4GrgcOlk3rxf0AyWnQ\nr0q6U9KTkv5a0tH02P6IiF3At4CXSAJ/d0Q8TI/thwrD/e5TSPJzUM0szTP4fVQZkHQM8PfAtRHx\nVvm80m1LR/R+knQxsCMinmKYCwF7YT+UGQWcC9wWEeeSnCl3WDujF/aHpA8CfwLMJAm2YyR9rnyZ\nXtgPw6njd6+6X/IM/peB6WXvp3P4t9aIJ2k0SejfHRE/Kk3eXrrXEZImAzvyqq9NPgZcIunXwN8B\n/13S3fTefhi0BdgSEYMPT7yP5ItgW4/tj/OAf4+InRGxH/gH4L/Re/uh3HB/E5VZOq00bVh5Bv97\nF4dJGkNycGJFjvW0lSQBy4C1EXFL2awVwB+UXv8B8KPKdUeSiLgpIqZHxKkkB+8ei4jP02P7YVBE\nbAM2Szq9NOki4Dngfnprf6wH5kk6svS3chHJwf9e2w/lhvubWAFcIWmMpFOBWcATVbcUEbn9AJ8C\nnic5GHFjnrXk8Lt/gqSn/TTwVOmnHzgBeAT4JfBjYHzetbZxn1wIrCi97uX98GFgNfAMyUj3+F7c\nH8D/IfnSe5bkYOboXtkPJP/6fQXYS3Is9KpqvztwUylH1wP/s9b2fQGXmVmP6eH7FZqZ9SYHv5lZ\nj3Hwm5n1GAe/mVmPcfCbmfUYB7+ZWY9x8JuZ9RgHv5lZj/n/1UdUSlFFFjEAAAAASUVORK5CYII=\n", "text/plain": [ - "" + "" ] }, "metadata": {}, @@ -177,7 +168,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 5, "metadata": { "collapsed": false }, @@ -188,7 +179,7 @@ "56" ] }, - "execution_count": 23, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -210,7 +201,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 6, "metadata": { "collapsed": false }, @@ -239,7 +230,7 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 7, "metadata": { "collapsed": false }, @@ -302,7 +293,7 @@ }, { "cell_type": "code", - "execution_count": 43, + "execution_count": 8, "metadata": { "collapsed": true }, @@ -319,7 +310,7 @@ }, { "cell_type": "code", - "execution_count": 47, + "execution_count": 9, "metadata": { "collapsed": false }, @@ -328,7 +319,7 @@ "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEACAYAAABcXmojAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X98VPWd7/HXJxhvsRAihkYNKCq0/qi9urYQb0Wo0l2I\ngmItmtQfaLcLCN3W3l4BZRXdPh4WUK56XREosmptQGutICKwaog+3EJxWdsKUX4pSACt8suqbYDP\n/ePMhCEmmQmZyZmZ834+HvPonDPnzHznWOaTc873+32buyMiIgJQEHYDREQke6goiIhIIxUFERFp\npKIgIiKNVBRERKSRioKIiDRKWhTMbIiZ1ZnZejOb0MI2D8Ref8PMzo2t62VmL5vZm2b2JzP754Tt\np5jZe2a2JvYYkr6vJCIiR+qo1l40s07Ag8BgYBvwezNb6O7rErapAPq4e18z6w/MBMqBBuBmd/9v\nM+sCvG5my9y9DnBghrvPyMzXEhGRI5HsTKEfsMHd33H3BmA+cFmTbYYDjwK4+0qg2MxK3X2Hu/93\nbP3HwDqgLGE/S8cXEBGR9ElWFMqArQnL73H4D3tL2/RM3MDMegPnAisTVv8wdrlprpkVt6HNIiKS\nIcmKQqpzYDT9q79xv9ilo18DP4qdMUBwiekU4BxgO3Bvip8jIiIZ1Oo9BYL7CL0SlnsRnAm0tk3P\n2DrMrBB4Gvilu/82voG7vx9/bma/ABY19+FmpomZRETayN2P+PJ8sjOF1UBfM+ttZkcDVwELm2yz\nELgOwMzKgd3uvtPMDJgLrHX3+xJ3MLMTEhZHAH9sqQHuroc7d9xxR+htyIaHjoOOhY5F64/2avVM\nwd33m9l4YCnQCZjr7uvMbHTs9Vnu/ryZVZjZBuAvwA2x3b8JXAP8wczWxNZNcvcXgKlmdg7BZabN\nwOh2fxMREWm3ZJePcPclwJIm62Y1WR7fzH6v0sKZiLtf17ZmiohIR9CI5hwxaNCgsJuQFXQcDtGx\nOETHIn0sHdegMsXMPJvbJyKSbcwMb8eN5qSXj0REjkTQ10QyKRN/NKsoiEjG6Ew/czJVdHVPQURE\nGqkoiIhIIxUFERFppKIgIpKisWPH8rOf/axxeebMmZSWllJUVMSuXbtCbFn6qEuqiGRErGtk2M3I\nmIaGBrp168aqVav46le/2uGf39LxbW+XVJ0piIgcgR07dvDZZ59xxhlntHnfdM1TlAkqCiISKQUF\nBWzatKlxedSoUfzLv/wLADU1NfTs2ZMZM2ZQWlrKiSeeyL//+79/btv169dz+umnA1BcXMzgwYMB\neO211/jGN75BcXEx/fr14z//8z8b9x00aBCTJ0/mm9/8Jl26dGHTpk0UFBQwc+ZM+vbtS1FREbff\nfjsbN27k/PPPp7i4mKuvvpqGhoYOOCqHaJyCiESamR3W53/nzp3s3buX+vp6li1bxpVXXsmIESPo\n1q1b47Z9+/Zl7dq1nHLKKezZs4eCggI++ugjLrnkEh588EEqKyt58sknueSSS9i4cSPHHnssAL/8\n5S9ZsmQJX/nKVzhw4AAAy5YtY82aNWzZsoVzzz2XV199lerqarp37875559PdXU1113XcdPF6UxB\nREIR/4Ft7yMdEi/lFBYWcvvtt9OpUyeGDh1Kly5deOuttz63bdPLP4sXL+YrX/kK3/ve9ygoKODq\nq6/m9NNPZ+HChY3fd9SoUZxxxhkUFBRQWFgIwC233EKXLl0488wzOfvssxk6dCi9e/emqKiIoUOH\nsmbNGjqSzhREJBTZek39uOOOo6Dg0N/LxxxzDB9//HErewTq6+s56aSTDlt38sknU19f37jcq1ev\nprtRWlra+Lxz586fW96xY0eb2t9eOlMQkUg55phj+OSTTxqXt2/fnpYzjrKyMt59993D1r377ruU\nlR2Ktc+F+aBUFEQkUs455xyeeOIJDhw4wAsvvEBtbW3K+7Z2dlNRUcHbb79NdXU1+/fvZ8GCBdTV\n1XHppZemtH9z24RxNqWiICKRcv/997No0SKOPfZYfvWrXzFixIjDXm/tr/mm9zESn3fv3p3nnnuO\ne++9l5KSEu655x6ee+45unfv3uJ7N/dZTd+/o88uNHhNRDIi3wevhU2D10REJONUFEREpJGKgoiI\nNFJREBGRRioKIiLSSEVBREQaqSiIiEgjFQUREWmkoiAi0sSWLVvo2rVrhwy+a5rvEDYVBRGJnN69\ne/PSSy+1+PpJJ53Evn37cmICu3RTURCRyGltCo79+/d3cGuyi4qCiETKtddey5YtWxg2bBhdu3Zl\n+vTpFBQU8Mgjj3DyySczePBg3n33XQoKCjh48CAA8+bN48wzz6SoqIjTTjuN2bNnN75fsgjPDz/8\nkGHDhtGtWzf69evH5MmTGTBgQLNt++tf/8pPf/pTTj75ZI4//njGjh3LZ599ltHj0ZSKgohEyuOP\nP85JJ53Ec889x759+xg5ciQAtbW11NXVsXTp0s+dRZSWlrJ48WL27t3LvHnzuPnmmw9LREuM8Jw7\ndy7jxo1jz549AIwbN46uXbuyc+dOHn30UR577LEWL0tNnDiRDRs28MYbb7Bhwwa2bdvGXXfdlaEj\n0QJ3z9pH0DwRyUXJ/v1Ceh5Honfv3v7iiy+6u/vmzZvdzHzz5s2Nr8fXHThwoNn9L7/8cr///vvd\n3f3ll1/2zp07H7btl770JV+5cqXv37/fCwsL/e233258bfLkyX7BBRc0LpuZb9y40Q8ePOhf/OIX\nfePGjY2vvfbaa37KKac024aWjm9s/RH/7iqOU0RCkW2zajcXlRm3ZMkS7rzzTtavX8/Bgwf55JNP\n+NrXvtb4eksRnh988AH79+8/7L179uzZ7Gd88MEHfPLJJ5x33nmN69y98RJWR9HlIxGJnGThNon+\n+te/8p3vfIdbbrmF999/n127dlFRUZFSd9UePXpw1FFHsXXr1sZ1ic8TlZSU0LlzZ9auXcuuXbvY\ntWsXu3fvZu/evSl+q/RQURCRyCktLWXjxo0pbfu3v/2Nv/3tb5SUlFBQUMCSJUtYtmxZSvt26tSJ\nK664gilTpvDpp59SV1fH448/3mwBKigo4Ac/+AE//vGP+eCDDwDYtm1byp+VLioKIhI5kyZN4mc/\n+xndu3fn6aefbvXMoWvXrjzwwAOMHDmS7t27U11dzWWXXdbsts158MEH2bNnD8cffzzXX389lZWV\nHH300c3uO3XqVPr06UN5eTndunXj29/+Nm+//XZ7v26bJI3jNLMhwH1AJ+AX7j61mW0eAIYCnwCj\n3H2NmfUCHgO+BDgw290fiG3fHVgAnAy8A4x0993NvK+ncoomItlHcZzNmzBhAu+//z7z5s1r1/uE\nEsdpZp2AB4EhwJlApZmd0WSbCqCPu/cF/gmYGXupAbjZ3c8CyoFxZnZ67LWJwHJ3/zLwYmxZRCTv\nvPXWW/zhD3/A3Vm1ahWPPPIII0aMCLtZLUrW+6gfsMHd3wEws/nAZcC6hG2GA48CuPtKMys2s1J3\n3wHsiK3/2MzWAWVAXWyfgbH9HwVqUGEQkTy0b98+Kisrqa+vp7S0lJ/+9KcMHz487Ga1KFlRKAMS\nb5W/B/RPYZuewM74CjPrDZwLrIytKnX3+Os7gdK2NFokTH/+M3TqBMceG3ZLJBd8/etfZ/369WE3\nI2XJikKqFwSbXr9q3M/MugC/Bn7k7h9/7gNigzdaeuMpU6Y0Ph80aBCDBg1KsUki6bd9O1xwAXzh\nC1BbC8cdF3aLJOpqamqoqalJ2/u1eqPZzMqBKe4+JLY8CTiYeLPZzB4Gatx9fmy5Dhjo7jvNrBB4\nDlji7vcl7FMHDHL3HWZ2AvCyu59OE7rRLNnkww9h4ECorIR9++DFF4NHUVHYLctOutGcWaHcaAZW\nA33NrLeZHQ1cBSxsss1C4LpYY8qB3bGCYMBcYG1iQUjY5/rY8+uB3x7pFxDpCPv2QUUFDB0Kt94K\nd98N550Hw4fDp5+G3TqR9EmlS+pQDnVJnevud5vZaAB3nxXbJt5D6S/ADe7+X2Z2AVAL/IFDl5Mm\nufsLsS6pTwInoS6pkuU++ywoCH36wKxZEO9WfuAAXHttUDB+8xsoLAy3ndkmilkEHS0TZwpJi0KY\nVBQkbA0NcOWV0LkzPPFEcIO56esjRgSXkB5//POvi3S0TF8+Eomsgwfhhhtg/3547LHmf/ALC+Gp\np6C+HsaPz75J3kTaSkVBpBnu8MMfwpYtwY9+wqwEn9O5MyxcCKtXw6RJHddGkUzQ1NkizZg8GVau\nhJdegmOOSb59UREsWRL0Tiouhokaiik5SkVBpIl77oFnngnGIbSlu2lJCSxfDgMGBIVhzJjMtVEk\nU1QURBLMmQP/9m/wyivBj3xbnXhiUBguvDAoKFVV6W+jSCapKIjELFgAU6bAihXQQjhWSk49FV54\nAQYPDgrDpZemrYkiGacuqSLA888HPY3+4z/g7LPT856rVgUF4cknQbOzSEdRl1SRdqqthVGj4Nln\n01cQAPr1C84+Ro6E3/8+fe8rkkkqChJpr78eDE6rroby8vS//7e+BXPnwrBh8Oab6X9/kXRTUZDI\nWrsWLrkEZs+Giy/O3OcMGwb33gv/8A+waVPmPkckHXSjWSJp8+bgR3r6dLj88sx/3ve+B3v3Bjef\nX3016KUkko1UFCRytm+Hb38bJkwIJrTrKGPHwp49wWcri0GylXofSaR89FEw6vjqq+G228Jpw8SJ\nymKQzNEsqSIp2rcvuHxz4YUwbdqhKbA7mjvcdBOsWxdMjdG5czjtkPykoiCSgpYyEcKiLAbJFBUF\nkSSSZSKERVkMkgkavCbSingmQkNDy5kIYVEWg2QjFQXJW4mZCL/+deuZCGFRFoNkG3VJlbw1eTL8\n7nepZyKERVkMkk1UFCQvTZsW3MCtrYVu3cJuTXLKYpBsoaIgeWf2bJg5M8hE6NEj7NakTlkMkg1U\nFCSvzJ8Pd97Z/kyEsCRmMXTtGsybJNKR1CVV8sbixXDjjenNRAiLshjkSKlLqgjBvYMbbgh68uR6\nQQBlMUh4VBQk5yVmIvTvH3Zr0kdZDBIGFQXJaWvXBpdZ5szJbCZCWIYNgxkzlMUgHUc3miVnJWYi\nXHZZ2K3JnKqqQ1Nuv/KKshgks1QUJCfFMxEmToRrrgm7NZmnLAbpKOp9JDnnww+D0b9VVXDrrWG3\npmMpi0GS0SypEinxTISBA2Hq1PCnwO5o7sFZQ12dshikeSoKEhnZlokQFmUxSGtUFCQSsjUTISzx\nLIZu3YIshgL1I5QYDV6TvBfPRNi/P/syEcISz2LYtg3GjVMWg6SPioJktcRMhKeeys5MhLAkZjFE\n7Ya7ZI66pEpWy5VMhLAkZjF066YsBmm/pGcKZjbEzOrMbL2ZTWhhmwdir79hZucmrH/EzHaa2R+b\nbD/FzN4zszWxx5D2fxXJN9OnwzPPwNKluZGJEJaSEli2LJgy/OGHw26N5LpWzxTMrBPwIDAY2Ab8\n3swWuvu6hG0qgD7u3tfM+gMzgfLYy/OA/wc81uStHZjh7jPS8zUk3yRmIpSUhN2a7FdWFswOqywG\naa9kZwr9gA3u/o67NwDzgaYTCgwHHgVw95VAsZkdH1t+BdjVwntHtEOhJBPPRFi2LPixk9TEsxh+\n8hNYtCjs1kiuSlYUyoCtCcvvxda1dZvm/DB2uWmumRWnsL1EwOLF8KMfBT9uffqE3Zrc89WvBjef\nv/99qKkJuzWSi5IVhVQ7ujX9qz/ZfjOBU4BzgO3AvSl+juSxFSvyKxMhLMpikPZI1vtoG9ArYbkX\nwZlAa9v0jK1rkbu/H39uZr8AWjzZnTJlSuPzQYMGMUgxVHlp9Wr47neDS0f5lIkQlsQshhdfhLPO\nCrtFkik1NTXUpPG0sNURzWZ2FPAWcDFQD6wCKpu50Tze3SvMrBy4z93LE17vDSxy97MT1p3g7ttj\nz28GvuHun7s1phHN0bB2bZCF8PDD+T0Fdhh+9Su45ZZgZtVTTw27NdIR2juiudUzBXffb2bjgaVA\nJ2Cuu68zs9Gx12e5+/NmVmFmG4C/ADckNK4aGAgcZ2ZbgdvdfR4w1czOIbjMtBkYfaRfQHJbPBNh\n2jQVhExQFoO0leY+ktBs3w4DBsDNNwdTNUjm/PznwRxJymLIf5oQT3JSlDMRwqIshmhQUZCcE/VM\nhLAoiyEaVBQkpygTIVzKYsh/KgqSM5SJkB3iWQxFRcF9Bv13yC/KU5CcoEyE7BHPYqivh/HjlcUg\nh1NRkIyLZyJs3apMhGyhLAZpifIUJOPimQgvv6xMhGySmMVQXAwTmp0YX6JGRUEyatq04IZmba26\nQWajeBbDgAFBYRitYaSRp6IgGZOYidCjR9itkZYkZjF07aoshqhTUZCMiGcirFgBPXuG3RpJJp7F\nMHhwUBiGDQu7RRIWdUmVtFu8GG68MfjrU1Ng55ZVq+DSS+HJJ0ETEucmdUmVrLJiBYwaBc8+q4KQ\ni5TFICoKkjaJmQjl5cm3l+yUmMXw5ptht0Y6moqCpMXatcFlhzlzgmwEyW3DhsGMGcG05ps2hd0a\n6Ui60SztFs9EuOceZSLkk6oq2LtXWQxRo6Ig7bJ9e/CjMXEiXHNN2K2RdBszBnbvDv4bK4shGtT7\nSI7YRx8Fo2ErKzVVQr5TFkPu0CypEgplIkSLshhyh4qCdLjPPoNLLoHTTlMmQpQcOADXXRfcZ1AW\nQ/ZSUZAO1dAQdDv9wheUiRBFDQ1wxRXBqGdlMWQnDV6TDhPPRGhoUCZCVBUWBqOdlcWQv1QUJCXK\nRJC4xCyGSZPCbo2km7qkSkpuuw1WroSXXlImgnw+i2HixLBbJOmioiBJTZ0KzzyjTAQ5XNMshjFj\nwm6RpIOKgrRq1ix4+GF49VVlIsjnJWYxFBUpiyEfqChIi6qr4a67gplPy8rCbo1kK2Ux5Bd1SZVm\nKRNB2kpZDNlBXVIl7VasCLqeLlyogiCpUxZDflBRkMMkZiL07x92ayTXKIsh96koSKPETISLLgq7\nNZKrlMWQ23SjWYBDmQjTpysTQdqvqgr27FEWQy5SURC2bw96jkycCNdeG3ZrJF+MHXuoMCiLIXeo\n91HEffhhMCq1qkqZCJIZymLoWJolVY5YPBPhwgth2jRNgS2ZoSyGjqWiIEfks8+gogL69FEmgmTe\ngQPBpcl9+5TFkGkqCtJmDQ1w5ZXBX2zKRJCO0tAAI0YEl5CUxZA5GR+8ZmZDzKzOzNab2YQWtnkg\n9vobZnZuwvpHzGynmf2xyfbdzWy5mb1tZsvMrPhIv4C0TTwTYf9+ZSJIxyosDKZdr6+HceOUxZCt\nWi0KZtYJeBAYApwJVJrZGU22qQD6uHtf4J+AmQkvz4vt29REYLm7fxl4MbYsGRbPRNiyRZkIEo54\nFsPrr6tjQ7ZKdqbQD9jg7u+4ewMwH2jai3048CiAu68Eis3s+NjyK8CuZt63cZ/Y/15+ZM2Xtpg8\nOchEeO45ZSJIeOJZDAsXws9/HnZrpKlk4xTKgK0Jy+8BTSc/aG6bMmBHK+9b6u47Y893AqXJmyrt\nMX26MhEkeyiLIXslKwqpXvVrelMj5auF7u5mpquLGTRnDjz0UDCytKQk7NaIBBKzGLp1g8rKsFsk\nkLwobAN6JSz3IjgTaG2bnrF1rdlpZse7+w4zOwF4v6UNTX0l22kkMAMYSK9eG8NujEgzzqKq6j+o\nqvpHYHHYjclZd9xxR1rep9UuqWZ2FPAWcDFQD6wCKt19XcI2FcB4d68ws3LgPncvT3i9N7DI3c9O\nWDcN+NDdp5rZRKDY3T93s1ldUtvn+eeDnkbKRJBspyyG9Mlol1R33w+MB5YCa4EF7r7OzEab2ejY\nNs8Dm8xsAzALuCmhcdXAa8CXzWyrmd0Qe+nnwLfN7G3gotiypFFtLYwaBc8+q4Ig2U9ZDNlDg9fy\n0OrVwWjl6mq4+OKwWyOSukWL4Ac/COZJOuussFuTm5S8JoeJZyLMnq2CILlHWQzh09TZeSQxE+Fy\njfyQHFVVBXv3KoshLCoKeWL79uAfkTIRJB+MGaMshrDonkIeiGciVFbCbbeF3RqR9Jk0Keg9pyyG\n1GmW1IiLZyIMHAhTp2oKbMkv7nDTTbBunbIYUqWiEGHKRJAoOHgwuCS6d6+yGFKhohBRykSQKGlo\ngCuugK5dlcWQjLqkRpAyESRqCguD0c719TB+vLIYMklFIcfEMxG2blUmgkRLPIth9ergBrRkhrqk\n5ph4JsJLLykTQaInnsUwcGAws6qKQ/qpKOSQadOCTIQVK9Q9T6KrpASWL4cLLgiyGMaODbtF+UVF\nIUfMng0zZ8Krr0KPHmG3RiRcJ554eBZDVVXYLcofKgo5YMECuOsuqKkJgklEBE49FV54IRin07Vr\nMG+StJ+6pGY5ZSKItC6exbBgAXzrW2G3JnzqkprHVqwIMhEWLlRBEGlJPIvhqquUxZAOKgpZavVq\n+O53Yf586N8/7NaIZLdvfQvmzg0uIb35ZtityW0qClkonokwZw5cdFHYrRHJDcpiSA/daM4yiZkI\nl10WdmtEcouyGNpPRSGLbN8e9KRQJoLIkRszBnbvVhbDkVLvoywRz0SoqoJbbw27NSK5b+LEIIch\nalkMmiU1D8QzES68MBi1rCmwRdrPPRjtXFcXrSwGFYUcp0wEkcw5cCC4FLtvX3SyGFQUcpgyEUQy\nr6EBRowILiFFIYtBg9dylDIRRDpGYWEwzXx9PYwbpyyGZFQUQqBMBJGOFc9ieP11deRIRl1SQ6BM\nBJGO1zSLYeLEsFuUnVQUOlg8E6G2Nlrd5ESyQUkJLFsGAwYEWQxjxoTdouyjotCBEjMRSkrCbo1I\nNJWVHcpiKCpSFkNTKgodZP58uPPOYOZTZSKIhEtZDC1Tl9QOsHgx3HijMhFEsk08i+HJJ2HQoLBb\nkx7qkprlVqwIup4qE0Ek+8SzGEaOVBZDnIpCBsUzEaqrlYkgkq2UxXA4FYUMScxEuPjisFsjIq1R\nFsMhutGcAfFMhHvuUSaCSK6oqoI9e5TFoKKQZtu3B/+nmjgRrrkm7NaISFuMHXuoMEQ1i0G9j9Io\nnolQWQm33RZ2a0TkSOVyFkPGex+Z2RAzqzOz9WY2oYVtHoi9/oaZnZtsXzObYmbvmdma2GPIkX6B\nbLFvXzAFdkWF5lYRyXV33w3nnQfDh8Onn4bdmo7V6pmCmXUC3gIGA9uA3wOV7r4uYZsKYLy7V5hZ\nf+B+dy9vbV8zuwPY5+4zWm1cjpwpKBNBJP/kahZDps8U+gEb3P0dd28A5gNNb50OBx4FcPeVQLGZ\nHZ/Cvnnx09nQAFddBV/6UjCFhQqCSH7o1AkefTR4fv31QZGIgmRFoQzYmrD8XmxdKtucmGTfH8Yu\nN801s+I2tTpLKBNBJL8VFgajnevrYfz4aGQxJOt9lOohaOvfxzOBu2LP/xW4F/h+cxtOmTKl8fmg\nQYMYlCVj0eOZCFu2BHOoKBNBJD/Fsxguvji4X3j33WG36HA1NTXU1NSk7f2S3VMoB6a4+5DY8iTg\noLtPTdjmYaDG3efHluuAgcApyfaNre8NLHL3z00Ckc33FCZPDuZmf+mlYG52Eclvf/5z0Lvwuutg\nQrNdbrJDpu8prAb6mllvMzsauApY2GSbhcB1scaUA7vdfWdr+5rZCQn7jwD+eKRfIAzTp8PTTwdn\nCCoIItFQUgLLlwdT4M+aFXZrMqfVy0fuvt/MxgNLgU7A3FjvodGx12e5+/NmVmFmG4C/ADe0tm/s\nraea2TkEl6c2A6Mz8eUyYc4ceOihYMRjjx5ht0ZEOtKJJwaFIZ7FUFkZdovST4PX2mDBAvjJT4KZ\nT/v0Cbs1IhKWP/0pyGL4xS+COc6ySXsvH6kopOj554OeRspEEBHI3iwG5Sl0gNpaGDUKnn1WBUFE\nAvmaxaCikMTrr8OVVwZxmuXlYbdGRLJJPmYxqCi0Yt26Q5kIF10UdmtEJBslZjFs3Bh2a9pPU2e3\nYPNm+Pu/D7qfKhNBRFpTVQV79wa/GbmexaCi0AxlIohIW40ZA7t3B78dK1YE4xpykXofNRHPRKiq\n0hTYItJ2YWcxqEtqGu3bF/Q9HjgQpk7VjKci0nbucNNNwT3JJUuCuZM6kopCmigTQUTS5eDB4NLz\n3r3wzDMdm8WgopAGDQ1Bt9POneGJJzQFtoi0X0MDjBgBXbvCL3/Zcb8rGrzWTspEEJFMKCyEp54K\nOq6MG5c7WQyRLgqJmQhPPaVMBBFJr3gWw+uvw6RJYbcmNZHukjp5MqxcGWQiHHNM2K0RkXxUVBTc\ncB44EIqLg95J2SyyRWHatOAGUG1tON3GRCQ6Skpg2TIYMCAoDGPGhN2ilkWyKMyeDTNnwquv5u4A\nExHJLWVlwSzL8SyGqqqwW9S8yBWF+fPhzjuDEYdlZWG3RkSi5NRTYenSIO+5a9dg3qRsE6kuqYsX\nw403KhNBRMKVySwGdUlN0YoVQdfThQtVEEQkXNmcxRCJorB6NXz3u1BdDf37h90aEZHszWLI+6Kw\ndu2hTISLLw67NSIihyRmMWzaFHZrAnl9o3nz5uBg33OPMhFEJDtVVcGePcGU29mQxZC3RUGZCCKS\nK8aOPVQYamvhuOPCa0te9j6KZyJUVsJtt2WgYSIiGZCOLAbNktqEMhFEJFe5B2cNdXVHnsWgopBA\nmQgikusOHIBrrw3+wP3Nb9qexaCiEKNMBBHJFw0NcMUVwajnxx9v2++ZBq+hTAQRyS+FhcFo5/p6\nGD++Y7MYcr4oxDMRtm5VJoKI5I94FsPq1XDrrR33uTnfJXXyZPjd7+Dll5WJICL5pWkWw4QJmf/M\nnC4K06cHN2KUiSAi+appFsPo0Zn9vJwtCrNnw0MPBSMAe/QIuzUiIplTVgbLlwdnDEVFwRisTMnJ\nopCYidCzZ9itERHJvNNOgxdeCOZw69Ilc1kMOdclVZkIIhJlybIYItUldcUKGDUKnn1WBUFEoinT\nWQw5UxTimQgLFkB5editEREJTyazGJIWBTMbYmZ1ZrbezJrtEGVmD8Ref8PMzk22r5l1N7PlZva2\nmS0zs+KAp3A1AAAEzklEQVTW2pCYiXDRRW35eiIi+SlTWQytFgUz6wQ8CAwBzgQqzeyMJttUAH3c\nvS/wT8DMFPadCCx39y8DL8aWmxXPRJg+PdqZCDU1NWE3ISvoOByiY3FIVI9FVVUwVmvZsvS9Z7Iz\nhX7ABnd/x90bgPlA05/m4cCjAO6+Eig2s+OT7Nu4T+x/L2+pAfFMhGuvbcO3ykNR/T99UzoOh+hY\nHBLlYzFmTPBIl2RFoQzYmrD8XmxdKtuc2Mq+pe6+M/Z8J1DaUgNuvBHGjUvSShERSYtkRSHV/qqp\ndH+y5t4v1ue0xc+ZNCnFFoiISPu5e4sPoBx4IWF5EjChyTYPA1cnLNcR/OXf4r6xbY6PPT8BqGvh\n810PPfTQQ4+2PVr7XU/2SDaieTXQ18x6A/XAVUDTAdYLgfHAfDMrB3a7+04z+7CVfRcC1wNTY//7\n2+Y+vD0DMEREpO1aLQruvt/MxgNLgU7AXHdfZ2ajY6/PcvfnzazCzDYAfwFuaG3f2Fv/HHjSzL4P\nvAOMzMB3ExGRNsrqaS5ERKRjZeWI5lQGzOUrM+tlZi+b2Ztm9icz++fY+jYN+MsnZtbJzNaY2aLY\nciSPhZkVm9mvzWydma01s/4RPhaTYv9G/mhmvzKz/xGVY2Fmj5jZTjP7Y8K6Fr977Fitj/2m/n2y\n98+6opDKgLk81wDc7O5nEdysHxf7/ikP+MtDPwLWEtxEg+gei/uB5939DOBrBB02IncsYvcpfwD8\nnbufTXB5+mqicyzmEfw+Jmr2u5vZmQT3c8+M7fOQmbX6u591RYHUBszlLXff4e7/HXv+MbCOYHxH\nygP+8omZ9QQqgF9wqOtz5I6FmXUDBrj7IxDcs3P3PUTwWAB7Cf54OsbMjgKOIejMEolj4e6vALua\nrG7pu18GVLt7g7u/A2wg+I1tUTYWhVQGzEVC7C+ic4GVtGHAX575v8D/AQ4mrIvisTgF+MDM5pnZ\nf5nZHDP7IhE8Fu7+EXAvsIWgGOx29+VE8FgkaOm7n0jwGxqX9Pc0G4uC7nwDZtYFeBr4kbvvS3wt\n2YC/fGFmlwLvu/saWhggGZVjQdBT8O+Ah9z97wh6+h12eSQqx8LMTgN+DPQm+NHrYmbXJG4TlWPR\nnBS+e6vHJRuLwjagV8JyLw6vdHnPzAoJCsLj7h4fw7EzNqcUZnYC8H5Y7etA/wsYbmabgWrgIjN7\nnGgei/eA99w9PoP+rwmKxI4IHouvA6+5+4fuvh/4DXA+0TwWcS39m2j6e9oztq5F2VgUGgfMmdnR\nBDdJFobcpg5jZgbMBda6+30JL8UH/EErA/7yibvf6u693P0UghuJL7n7tUTzWOwAtprZl2OrBgNv\nAouI2LEguMFebmadY/9eBhN0RIjisYhr6d/EQuBqMzvazE4B+gKrWnujrBynYGZDgfs4NOjt7pCb\n1GHM7AKgFvgDh07zJhH8h3wSOInYgD933x1GG8NgZgOB/+3uw82sOxE8Fmb2PwluuB8NbCQYKNqJ\naB6LWwh+/A4C/wX8I9CVCBwLM6sGBgIlBPcPbgeepYXvbma3AjcC+wkuRy9t9f2zsSiIiEg4svHy\nkYiIhERFQUREGqkoiIhIIxUFERFppKIgIiKNVBRERKSRioKIiDRSURARkUb/HwGucLN2dHFmAAAA\nAElFTkSuQmCC\n", "text/plain": [ - "" + "" ] }, "metadata": {}, @@ -361,7 +352,7 @@ }, { "cell_type": "code", - "execution_count": 48, + "execution_count": 10, "metadata": { "collapsed": false }, @@ -370,7 +361,7 @@ "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQVPWd9/H3dwYQRhhwRAe5yFCCXOIlGGUxajJrTBZR\nISYplUq8ZGujZSSiFaPRUgMp/0lFk2ix65qNEDUG46MxhQrBmDjxcfVBWRGNMMAgd5iB5X4RnGG+\nzx+ne2iameme7tN9uqc/r6ou+ty/fYr5zG9+/TvnmLsjIiKloyzqAkREJL8U/CIiJUbBLyJSYhT8\nIiIlRsEvIlJiFPwiIiUmZfCb2SQzqzez1WZ2TzvLx5jZO2Z2yMx+2M7ycjNbamYvh1W0iIhkrtPg\nN7NyYDYwCRgHTDOzsUmr7QB+ADzcwW5mAMsBXTAgIlIAUrX4JwAN7r7O3ZuB54CpiSu4+3Z3XwI0\nJ29sZkOBycBvAAunZBERyUaq4B8CbEyY3hSbl65fAj8CWrtYl4iI5Eiq4M+4e8bMrgS2uftS1NoX\nESkYPVIs3wwMS5geRtDqT8cXgSlmNhnoDVSa2dPufkPiSmamvn8RkQy4e0aN6lQt/iXAKDOrMbNe\nwLXA/A7WPaYAd7/P3Ye5+wjgOuBvyaGfsK5e7vzkJz+JvIZCeelc6FzoXHT+ykanLX53bzGz6cAi\noBx40t1XmNktseVPmNkg4D2gEmg1sxnAOHffn7y7rCoVEZFQpOrqwd0XAguT5j2R8L6RY7uD2tvH\n34G/Z1ijiIiESFfuFpDa2tqoSygYOhdH6VwcpXMRDsu2ryjrAsw86hpERIqNmeEZfrmbsqtHRKQj\nZhqpnQ9hN44V/CKSFf3Fnlu5+OWqPn4RkRKj4BcRKTEKfhGREqPgFxFJcuutt/LQQw+1TT/++ONU\nV1dTWVnJrl27IqwsHBrOKSIZiw0pjLqMnGpubqZ///68++67nHXWWXk/fkfnOJvhnGrxi4h0orGx\nkUOHDjF2bPIzqFIL4746uaDgF5FuqaysjE8++aRt+qabbuKBBx4AoK6ujqFDh/KLX/yC6upqBg8e\nzG9/+9vj1l29ejVjxowBYMCAAVx22WUAvP3221xwwQUMGDCACRMm8M4777RtW1tby/33389FF11E\n3759+eSTTygrK+Pxxx9n1KhRVFZW8uCDD7JmzRouvPBCBgwYwHXXXUdz83HPssoZjeMXkZJgZseM\niW9qamLv3r1s2bKF1157jW9961tcffXV9O/fv23dUaNGsXz5ckaMGMGePXsoKytj586dXHHFFcye\nPZtp06bx/PPPc8UVV7BmzRpOOukkAH73u9+xcOFCRo8ezZEjRwB47bXXWLp0KRs2bGD8+PG89dZb\nzJs3j6qqKi688ELmzZvHDTe0ewPj0KnFLyI5FQ/RbF5hSex26dmzJw8++CDl5eVcfvnl9O3bl5Ur\nVx63bnJXzauvvsro0aP59re/TVlZGddddx1jxoxh/vz5bZ/3pptuYuzYsZSVldGzZ08A7r77bvr2\n7cu4ceM4++yzufzyy6mpqaGyspLLL7+cpUuXhvY5U1GLX0RyqhD7uAFOPvlkysqOtn0rKirYvz/5\nbvLH27JlC6effvox84YPH86WLVvapocNO/6GxdXV1W3v+/Tpc9x0Y2Njl+rPhlr8ItItVVRUcPDg\nwbbprVu3hvLXw5AhQ1i/fv0x89avX8+QIUcfR17o9zBS8IvkgTtceSW8/nrUlZSOz3/+8zz77LMc\nOXKEP//5z7z55ptpb9vZXymTJ09m1apVzJs3j5aWFv7whz9QX1/PlVdemdb27a2T77+KFPwiefDs\ns7BgQfCS/Hj00Ud5+eWXOemkk/j973/P1Vdffczyzlrlyd8tJL6vqqrilVde4ZFHHmHgwIE8/PDD\nvPLKK1RVVXW47/aOlbz/fP6VoAu4RHJs714YOxZmzIAXX4TFi6OuKDylcAFX1HJxAZeCXyTH7roL\ndu6E2bPhlFNg+3aoqIi6qnAo+HMvF8GvUT0iObR8OTz1FPzjH0HYn302vPcefPnLUVcmpUx9/CI5\n4g633w4PPADxkXsXXQRvvRVtXSIKfpEc2bYN3n8fvv/9o/Muugj++7+jq0kEFPwiOdPQAKNHQ4+E\nDtWLLoJ33oHW1ujqEkkr+M1skpnVm9lqM7unneVjzOwdMztkZj9MmD/MzN4ws4/N7B9mdnuYxYsU\nsoYGOOOMY+dVV8PAgUHfv0hUUga/mZUDs4FJwDhgmpkl3590B/AD4OGk+c3Ane7+OWAicFs724p0\nSw0NMHLk8fPV3SNRS6fFPwFocPd17t4MPAdMTVzB3be7+xKCoE+c3+juH8Te7wdWAINDqVykwHUW\n/PqCV6KUTvAPATYmTG+KzesSM6sBxgPd6PIVkY6tWaMWf6HbsGED/fr1y8u1CMnPB4hSOuP4sz4j\nZtYXeAGYEWv5H2PmzJlt72tra6mtrc32kCKR66jFP2YM7N4NW7fCaaflv65SUlNTw5w5c7j00kvb\nXX766aezb9++PFeVmbq6Ourq6kLZVzrBvxlIvMfoMIJWf1rMrCfwIvA7d/9Te+skBr9Id7BzJxw5\nAieffPyysrKjrf5vfSv/tZWSzq4sbmlpoUeP4rmGNblRPGvWrIz3lU5XzxJglJnVmFkv4Fpgfgfr\nHnP5sAV3HXoSWO7uv8q4SpEiE2/td3TfLXX35N7111/Phg0buOqqq+jXrx8///nPKSsrY86cOQwf\nPpzLLruM9evXU1ZWRmtsfO3cuXMZN24clZWVnHHGGfz6179u21+qxzXu2LGDq666iv79+zNhwgTu\nv/9+LrnkknZrO3z4MHfddRfDhw9n0KBB3HrrrRw6dCin5yNRyuB39xZgOrAIWA78wd1XmNktZnYL\ngJkNMrONwJ3A/Wa2Ida9cxHwHeCfzWxp7DUpZ59GpEB01M0Td845UF+fv3pK0TPPPMPpp5/OK6+8\nwr59+7jmmmsAePPNN6mvr2fRokXH/TVQXV3Nq6++yt69e5k7dy533nnnMU/GSnxc45NPPsltt93G\nnj17ALjtttvo168fTU1NPPXUUzz99NMd3nHzxz/+MQ0NDSxbtoyGhgY2b97MT3/60xydiXbEnwIf\n1SsoQaR7+elP3e+7r+Ply5a5n3VW/urJlXR+foObV2T3ylRNTY3/9a9/dXf3tWvXupn52rVr25bH\n5x05cqTd7b/+9a/7o48+6u7ub7zxhvfp0+eYdU899VRfvHixt7S0eM+ePX3VqlVty+6//36/+OKL\n26bNzNesWeOtra1+4okn+po1a9qWvf322z5ixIh2a+joHMfmZ5S7xdPBJVJEGho6vxHb0KGwKe1v\nyopbod28s73HIsYtXLiQWbNmsXr1alpbWzl48CDnnHNO2/KOHte4fft2Wlpajtn30KFD2z3G9u3b\nOXjwIF/4whfa5rl7W3dTPuiWDSI5kKqr56ST4PBhSOMRr5KFVA9ASXT48GG++c1vcvfdd7Nt2zZ2\n7drF5MmT0xrqecopp9CjRw82bjw68j3xfaKBAwfSp08fli9fzq5du9i1axe7d+9m7969aX6q7Cn4\nRXIgVfCbBa3+zZvzV1Mpqq6uZs2aNWmt+9lnn/HZZ58xcOBAysrKWLhwIa+99lpa25aXl/ONb3yD\nmTNn8umnn1JfX88zzzzT7i+ZsrIyvve973HHHXewfft2ADZv3pz2scKg4BcJ2b59QUs+1Rj9Uuru\nicq9997LQw89RFVVFS+++GKnfwH069ePxx57jGuuuYaqqirmzZvH1KlT2123PbNnz2bPnj0MGjSI\nG2+8kWnTptGrV692t/3Zz37GyJEjmThxIv379+erX/0qq1atyvbjpk1P4BIJ2QcfwPXXw0cfdb7e\nDTfAV74CN96Yn7pyQU/g6tg999zDtm3bmDt3blb7ycUTuNTiFwlZqm6eOLX4u5eVK1fy4Ycf4u68\n++67zJkz57gHvBcKjeoRCVlXgj/VXwVSPPbt28e0adPYsmUL1dXV3HXXXUyZMiXqstql4BcJ2Zo1\ncP75qdcbOhQWLsx9PZIf559/PqtXr466jLSoq0ckZOrqkUKn4BcJWXtP3mqPgl+iolE9IiH69NPg\n4qwDB6C8vPN1W1uhoiK4RXPv3vmpL2wa1ZN7uRjVoz5+kRCtXQs1NalDH4LbMw8eHFzElc5fCIWq\ns7HtUpgU/CIhSrd/Py7e3VOswa/WfnFSH79IiDZsgOHD019f/fwSBQW/SIgaG7v2OEUFv0RBwS8S\noq1bYdCg9NdX8EsUFPwiIWpsVPBL4VPwi4RIwS/FQMEvEiIFvxQDXcAlEpLW1uBCrP37IeE27J06\ncgT69OnaNiKg2zKLFIQdO6CysmsBXl4e/IWwdWvu6hJJpuAXCUlXu3ni1N0j+abgFwmJgl+KRcrg\nN7NJZlZvZqvN7J52lo8xs3fM7JCZ/bAr24p0Jwp+KRadBr+ZlQOzgUnAOGCamY1NWm0H8APg4Qy2\nFek2FPxSLFK1+CcADe6+zt2bgeeAYx477+7b3X0J0NzVbUW6EwW/FItUwT8E2JgwvSk2Lx3ZbCtS\ndBT8UixS3ZY5mwH2aW87c+bMtve1tbXU1tZmcViRaCj4JZfq6uqoq6sLZV+pgn8zMCxhehhByz0d\naW+bGPwixSrT4D/tNGhqCi7mSucBLlKakhvFs2bNynhfqbp6lgCjzKzGzHoB1wLzO1g3+Qqyrmwr\nUvQyDf6ePaF//+ACMJF86LTF7+4tZjYdWASUA0+6+wozuyW2/AkzGwS8B1QCrWY2Axjn7vvb2zaX\nH0YkKocPw759UFWV2fanngrbtgX/iuRaykcvuvtCYGHSvCcS3jdybJdOp9uKdEfx0C7L8JLIePCL\n5IOu3BUJQVcfwJJMwS/5pOAXCUFXH7mYTMEv+aTgFwlBpl/sxin4JZ8U/CIhUPBLMVHwi4RAwS/F\nRMEvEgIFvxQTBb9ICMII/qam8OoR6YyCXyQEavFLMVHwi2TJPQj+6urM91FZCc3NcPBgeHWJdETB\nL5KlffuCm6v17Zv5PsyCVv/27eHVJdIRBb9IlrLt5olTd4/ki4JfJEsKfik2Cn6RLCn4pdgo+EWy\npOCXYqPgF8mSgl+KjYJfJEsKfik2Cn6RLGU7hj9OwS/5ouAXyVJTk1r8UlwU/CJZCutZuQp+yRdz\n92gLMPOoaxDJlDv07g27d0OfPtnt6/Bh6Ncv+NcsnPqk+zIz3D2j/ylq8YtkYe9eOOGE7EMfgv1U\nVMCuXdnvS6QzCn6RLITVzRNXXa3uHsk9Bb9IFsIOfvXzSz6kDH4zm2Rm9Wa22szu6WCdx2LLl5nZ\n+IT595rZx2b2kZn93sxOCLN4kag1NSn4pfh0GvxmVg7MBiYB44BpZjY2aZ3JwEh3HwXcDDwem18D\nfA84z93PBsqB60KuXyRS27aFM4Y/TsEv+ZCqxT8BaHD3de7eDDwHTE1aZwrwFIC7LwYGmFk1sBdo\nBirMrAdQAWwOs3iRqKmrR4pRquAfAmxMmN4Um5dyHXffCTwCbAC2ALvd/fXsyhUpLAp+KUY9UixP\nd4D9cWNJzewM4A6gBtgD/B8z+7a7P5u87syZM9ve19bWUltbm+ZhRaK1bRtcckl4+zv1VHjjjfD2\nJ91HXV0ddXV1oewrVfBvBoYlTA8jaNF3ts7Q2Lxa4G133wFgZn8Evgh0GvwixUQtfsmX5EbxrFmz\nMt5Xqq6eJcAoM6sxs17AtcD8pHXmAzcAmNlEgi6dJmAlMNHM+piZAZcByzOuVKQAKfilGHXa4nf3\nFjObDiwiGJXzpLuvMLNbYsufcPcFZjbZzBqAA8B3Y8s+MLOnCX55tALvA7/O4WcRyTsFvxQj3atH\nJEMtLcGtGg4dgvLycPbZ2hrcuuHAAejVK5x9Sveke/WIROB//xeqqsILfYCyMhg4MNi3SK4o+EUy\nFPZVu3Hq7pFcU/CLZCjsq3bjqquDXyoiuaLgF8lQ2F/sxp16qoJfckvBL5KhXAb/9u3h71ckTsEv\nkqFcBr/6+CWXFPwiGcpV8KuPX3JNwS+SIbX4pVgp+EUypOCXYqXgF8mQunqkWCn4RTKUq+A/5ZRg\n37qTieSKgl8kA/v3B8F84onh77tPn+B+PXv3hr9vEVDwi2QkftWuZXSLrNR0EZfkkoJfJAO56uaJ\nq67WF7ySOwp+kQzkOvg1skdyScEvkoF8BL+6eiRXFPwiGVBXjxQzBb9IBtTVI8VMwS+SAXX1SDFT\n8ItkQF09UswU/CIZUFePFDMFv0gGcvW83Th19UgumUd8QxAz86hrEOmKI0egd2/49FPo0SM3x2ht\nDW7bcOAA9OqVm2NIcTMz3D2ja8dTtvjNbJKZ1ZvZajO7p4N1HostX2Zm4xPmDzCzF8xshZktN7OJ\nmRQpUkh27oTKytyFPkBZWXCzNj2CUXKh0+A3s3JgNjAJGAdMM7OxSetMBka6+yjgZuDxhMWPAgvc\nfSxwDrAixNpFIrF1K5x2Wu6Po+4eyZVULf4JQIO7r3P3ZuA5YGrSOlOApwDcfTEwwMyqzaw/cIm7\nz4kta3H3PeGWL5J/jY35CX6N7JFcSRX8Q4CNCdObYvNSrTMUGAFsN7O5Zva+mf2XmVVkW7BI1LZu\nhUGDcn8ctfglV1L1Uqb7rWvyFwwe2/d5wHR3f8/MfgX8GHgweeOZM2e2va+traW2tjbNw4rkXz67\netTil7i6ujrq6upC2Veq4N8MDEuYHkbQou9snaGxeQZscvf3YvNfIAj+4yQGv0ih27oVampyfxx1\n9Uii5EbxrFmzMt5Xqq6eJcAoM6sxs17AtcD8pHXmAzcAxEbt7Hb3JndvBDaa2Zmx9S4DPs64UpEC\n0diorh4pbp22+N29xcymA4uAcuBJd19hZrfElj/h7gvMbLKZNQAHgO8m7OIHwLOxXxprkpaJFCV1\n9UixSzkS2d0XAguT5j2RND29g22XARdkU6BIoclX8KurR3JFt2wQ6SKN6pFip+AX6YL9+4PbKVRW\n5v5Y8St3dUcTCZuCX6QL4t08ltEdUrqmd2+oqIDdu3N/LCktCn6RLshX/36cunskFxT8Il2Qr6Gc\ncRrZI7mg4Bfpgny3+DWyR3JBwS/SBerqke5AwS/SBfkayhmnrh7JBQW/SBfk65bMcerqkVxQ8It0\ngbp6pDtQ8It0Qb6D/7TTYMuW/B1PSoOCXyRNzc3BxVQDB+bvmEOGwObN+TuelAYFv0iampqC2yiU\nl+fvmIMHB8c9ciR/x5TuT8EvkqZ8d/MA9OoFAwboC14Jl4JfJE35HsoZp+4eCZuCXyRN+R7KGTd0\nqIJfwqXgF0lTFF09oBa/hE/BL5ImdfVId6HgF0lTVF09Cn4Jm4JfJE3q6pHuQsEvkiYFv3QX5hE/\n0NPMPOoaRFJxhxNOgL17g0ci5tOuXVBTA3v25Pe4UtjMDHfP6CGgavGLpGHnTjjxxPyHPgQXcDU3\nw759+T+2dE8pg9/MJplZvZmtNrN7OljnsdjyZWY2PmlZuZktNbOXwypaJN+i6uaB4MHu6u6RMHUa\n/GZWDswGJgHjgGlmNjZpncnASHcfBdwMPJ60mxnAckD9OVK0ohrKGafglzClavFPABrcfZ27NwPP\nAVOT1pkCPAXg7ouBAWZWDWBmQ4HJwG+AjPqiRApBlC1+UPBLuFIF/xBgY8L0pti8dNf5JfAjoDWL\nGkUit3lzcKfMqCj4JUw9UixPt3smuTVvZnYlsM3dl5pZbWcbz5w5s+19bW0ttbWdri6Sd+vWwbnn\nRnf8IUNg9eroji/Rq6uro66uLpR9pQr+zcCwhOlhBC36ztYZGpv3TWBK7DuA3kClmT3t7jckHyQx\n+EUK0bp1MDW5kzOPhgyBkH7mpUglN4pnzZqV8b5SdfUsAUaZWY2Z9QKuBeYnrTMfuAHAzCYCu929\n0d3vc/dh7j4CuA74W3uhL1IM1q2DESOiO77u0Clh6rTF7+4tZjYdWASUA0+6+wozuyW2/Al3X2Bm\nk82sATgAfLej3YVZuEi+tLbC+vUwfHh0NaiPX8KkK3dFUtiyBcaPDx6BGJXm5uACsoMHoUeqDlop\nCbpyVySH1q0LbpkQpZ494eSTo/3lI92Hgl8khaj79+PU3SNhUfCLpLB2bfQtflDwS3gU/CIpFEJX\nDyj4JTwKfpEU1NUj3Y2CXyQFtfilu1Hwi3TiyBHYsAFOPz3qSoLg35R83bxIBhT8Ip3YuhWqqqBP\nn6grUYtfwqPgF+lEofTvw9Hg1/WOki0Fv0gnCqV/H6CyEsrKYPfuqCuRYqfgF+lEoYzhh+ARjCNH\nQkND1JVIsVPwi3SikFr8AKNHw6pVUVchxU7BL9KJQurjBzjzTFi5MuoqpNgp+EU6UWgt/jPPVItf\nsqfgF+nAkSPBuPlCGMMfp+CXMCj4RTqweTOccgqccELUlRw1alQQ/BrSKdlQ8It0oNC6eQAGDAge\nyLJ1a9SVSDFT8It0oBCDH4KRPfqCV7Kh4BfpQCGN4U+kfn7JloJfpAOFNpQzTsEv2VLwi3Rg7VoY\nPjzqKo6n4JdsKfhF2uEOH38M48ZFXcnxFPySLQW/SDuamoLwP+20qCs53hlnwPr10NwcdSVSrNIK\nfjObZGb1ZrbazO7pYJ3HYsuXmdn42LxhZvaGmX1sZv8ws9vDLF4kV5Ytg3PPDW6MVmhOOCG4RfPa\ntVFXIsUqZfCbWTkwG5gEjAOmmdnYpHUmAyPdfRRwM/B4bFEzcKe7fw6YCNyWvK1IIfrwQzjnnKir\n6Ji6eyQb6bT4JwAN7r7O3ZuB54CpSetMAZ4CcPfFwAAzq3b3Rnf/IDZ/P7ACGBxa9SI5ouCX7iyd\n4B8CbEyY3hSbl2qdoYkrmFkNMB5Y3NUiRfJNwS/dWY801kn3riDJvaFt25lZX+AFYEas5X+MmTNn\ntr2vra2ltrY2zUOKhO+zz4JQLcQRPXGjR8OLL0ZdheRTXV0ddXV1oezLPMXdnsxsIjDT3SfFpu8F\nWt39Zwnr/CdQ5+7PxabrgS+7e5OZ9QReARa6+6/a2b+nqkEknz76CK65BlasiLqSjm3YABdeqIev\nlzIzw90zGn6QTlfPEmCUmdWYWS/gWmB+0jrzgRtixUwEdsdC34AngeXthb5IIYqP6ClkQ4fCrl2w\n/7i/n0VSSxn87t4CTAcWAcuBP7j7CjO7xcxuia2zAPjEzBqAJ4Dvxza/CPgO8M9mtjT2mpSLDyIS\nlkLv34fgoesjR8Lq1VFXIsUonT5+3H0hsDBp3hNJ09Pb2e4tdJGYFJkPP4Tpx/1vLjzxxzCOHx91\nJVJsFMoiSYqhxQ9w3nnw3ntRVyHFSMEvkmD7djh4EIYNi7qS1L70JXjzzairkGKk4BdJ8NFHQWu/\nEG/VkOyCC4KRR3v3Rl2JFBsFv0iCDz8s/BE9cSecAOefD2+/HXUlUmwU/CIJli0rjv79OHX3SCYU\n/CIJiuWL3TgFv2Qi5ZW7OS9AV+5KgWhpgcpK2LYN+vaNupr0HDgA1dXBl9J9+kRdjeRTrq/cFSkJ\nH3wQPGqxWEIf4MQT4eyzYbFufShdoOAXiXnpJZiafMPxIqDuHukqBb9IzEsvwTe+EXUVXafgl65S\nH78IwXj4r30teJZtWZE1h3bvDi4427EDevWKuhrJl2z6+Asi+CMtQASAe4FBwIyoC8nQ+wT3Rvx/\nURciORbP7KIP/qhrEDn/fHj4YSjWZwDNmAGDB8M990RdieSLRvWIZGH9+uDBJhdfHHUlmfvSl+D1\n16OuQoqFgl9K3ksvwZQp0COtm5QXpsmTg4vP6uujrkSKgYJfSt4f/1ico3kS9ekDt94Kv/xl1JVI\nMVAfv5S0piYYMwYaG4ObnhWzbduCh7CvXAmnnhp1NZJr6uMXydDzz8OkScUf+hCE/TXXwL//e9SV\nSKFTi19K1rZtwe0OFi4MnmbVHaxcCZdcAuvWQUVF1NVILqnFL5KBO+6Am27qPqEPQVfPhRfC009H\nXYkUMrX4pSQtWAC33x6MhOluLeM334R/+zdYvry4RypJ53QBl0gX7NsHZ50Fc+bAV74SdTXhc4er\nroKqKvjtb4vvFhSSnpx29ZjZJDOrN7PVZtbudYFm9lhs+TIzG9+VbUXyqbUVfvhDuPTS7hn6EDwv\n+PnngwvTbr01+EUgkqjT4DezcmA2MAkYB0wzs7FJ60wGRrr7KOBm4PF0t5Vj1dXVRV1CwcjFudi4\nEb76Vfj4Y3jkkdB3nzOZnIuKCnjlleBRknfe2X3CXz8j4UjV4p8ANLj7OndvBp4Dku9YPgV4CsDd\nFwMDzGxQmttKAv2nPirMc+EO8+bBF74QtPT//vegG6RYZHou+vWDP/856POfMgXeeKP4fwHoZyQc\nqb76GQJsTJjeBPxTGusMAQansa1I6A4ehK1b4b33guBbtCgY475gQXAztlIyYAC89RY88wxMnx70\n9//rv8K55wYjgAYPDrqGpLSkCv502wdZ/de56qpstu4+Vq6E//mfqKsoDCtXwpIlwft4K9X96Ku1\nNXi1tMChQ8Hr4MFgbP6hQ0GgnXVWcHHWAw/AGWdE91miVlEBt9wCN98Mf/tb0P//pz8F53j//uCv\nn379gkdO9u4N5eVHX2bHvhJF8Quj1H9GZs0KZ/hxp6N6zGwiMNPdJ8Wm7wVa3f1nCev8J1Dn7s/F\npuuBLwMjUm0bm1/kf3yKiEQj01E9qVr8S4BRZlYDbAGuBaYlrTMfmA48F/tFsdvdm8xsRxrbZly4\niIhkptPgd/cWM5sOLALKgSfdfYWZ3RJb/oS7LzCzyWbWABwAvtvZtrn8MCIiklrkF3CJiEh+RXpN\nXylf4GVmw8zsDTP72Mz+YWa3x+ZXmdlfzGyVmb1mZgOirjUfzKzczJaa2cux6ZI8DwBmNsDMXjCz\nFWa23Mz+qRTPh5ndG/v5+MjMfm9mJ5TKeTCzOWbWZGYfJczr8LPHztXqWJ5+LdX+Iwt+XeBFM3Cn\nu38OmAjcFvv8Pwb+4u5nAn+NTZeCGcByjo4kK9XzAPAosMDdxwLnAPWU2PmIfTf4PeA8dz+boLv4\nOkrnPMx3MyMeAAAClElEQVQlyMZE7X52MxtH8B3quNg2/2FmnWZ7lC3+kr7Ay90b3f2D2Pv9wAqC\n6x/aLoiL/fv1aCrMHzMbCkwGfsPRocEldx4AzKw/cIm7z4HguzJ330PpnY+9BI2jCjPrAVQQDBIp\nifPg7v8X2JU0u6PPPhWY5+7N7r4OaCDI1w5FGfwdXfhVcmKtm/HAYqDa3Ztii5qA6ojKyqdfAj8C\nWhPmleJ5gGAY9HYzm2tm75vZf5nZiZTY+XD3ncAjwAaCwN/t7n+hxM5Dko4++2CC/IxLmaVRBr++\nVQbMrC/wIjDD3fclLovdtrRbnyczuxLY5u5L6eBCwFI4Dwl6AOcB/+Hu5xGMlDumO6MUzoeZnQHc\nAdQQBFtfM/tO4jqlcB46ksZn7/S8RBn8m4FhCdPDOPa3VrdnZj0JQv8Zd/9TbHZT7F5HmNlpwLao\n6suTLwJTzGwtMA+41MyeofTOQ9wmYJO7vxebfoHgF0FjiZ2P84G33X2Hu7cAfwQupPTOQ6KOfiaS\ns3RobF6Hogz+tovDzKwXwZcT8yOsJ6/MzIAngeXu/quERfOBG2PvbwT+lLxtd+Lu97n7MHcfQfDl\n3d/c/XpK7DzEuXsjsNHMzozNugz4GHiZ0jof9cBEM+sT+1m5jODL/1I7D4k6+pmYD1xnZr3MbAQw\nCni30z25e2Qv4HJgJcGXEfdGWUsEn/1igj7tD4ClsdckoAp4HVgFvAYMiLrWPJ6TLwPzY+9L+Tyc\nC7wHLCNo6fYvxfMB3E3wS+8jgi8ze5bKeSD463cL8BnBd6Hf7eyzA/fFcrQe+JdU+9cFXCIiJUYP\nZRMRKTEKfhGREqPgFxEpMQp+EZESo+AXESkxCn4RkRKj4BcRKTEKfhGREvP/AfApOG5pWjCyAAAA\nAElFTkSuQmCC\n", "text/plain": [ - "" + "" ] }, "metadata": {}, @@ -451,7 +442,7 @@ }, { "cell_type": "code", - "execution_count": 52, + "execution_count": 11, "metadata": { "collapsed": false }, @@ -459,10 +450,10 @@ { "data": { "text/plain": [ - "2.908843447705344e-75" + "2.38031546401283e-75" ] }, - "execution_count": 52, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -512,7 +503,7 @@ }, { "cell_type": "code", - "execution_count": 53, + "execution_count": 12, "metadata": { "collapsed": false }, @@ -520,10 +511,10 @@ { "data": { "text/plain": [ - "0.4406779172095556" + "0.4413448096175153" ] }, - "execution_count": 53, + "execution_count": 12, "metadata": {}, "output_type": "execute_result" } @@ -609,7 +600,7 @@ }, { "cell_type": "code", - "execution_count": 55, + "execution_count": 13, "metadata": { "collapsed": false }, @@ -654,7 +645,7 @@ }, { "cell_type": "code", - "execution_count": 62, + "execution_count": 14, "metadata": { "collapsed": false }, @@ -665,7 +656,7 @@ "111" ] }, - "execution_count": 62, + "execution_count": 14, "metadata": {}, "output_type": "execute_result" } @@ -676,7 +667,7 @@ }, { "cell_type": "code", - "execution_count": 72, + "execution_count": 15, "metadata": { "collapsed": false }, @@ -687,7 +678,7 @@ "3.192662744667652e-75" ] }, - "execution_count": 72, + "execution_count": 15, "metadata": {}, "output_type": "execute_result" } @@ -707,7 +698,7 @@ }, { "cell_type": "code", - "execution_count": 66, + "execution_count": 16, "metadata": { "collapsed": false }, @@ -716,7 +707,7 @@ "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG+NJREFUeJzt3XmwnXWd5/H3h5uEsJpAQkIWE5DYLILCaMi44LGGrrpG\nJVVD0yytFGIL5XSQsSmHhppqLn/M2HZpgVRajJqmaHUmAr1FmkjLctR2MCQCESGhEzSQQBZIWNNg\ntu/88ZxLDie5Z32Wc+75vKpucZ/1fO9TuZ/74/tsigjMzKw/HFJ0AWZmlh+HvplZH3Hom5n1EYe+\nmVkfceibmfURh76ZWR9pGPqSBiWtlbRO0rUHWX6ypIckvSnpmoMsH5D0qKQfpVW0mZm1p27oSxoA\nFgGDwKnAxZJOqVltO3AV8LURdnM18CTgGwLMzArWaKQ/F1gfERsiYjewFFhQvUJEvBARq4DdtRtL\nmgHMB74LKJ2SzcysXY1CfzqwsWp6U2Ves24Cvgzsa7EuMzPLQKPQb7slI+mTwLaIeBSP8s3MusKY\nBsufA2ZWTc8kGe0344PAeZLmA+OBoyX9XURcWr2SJPf6zczaEBEtD6gbjfRXAXMkzZY0DrgQWDbC\num/78Ii4PiJmRsQJwEXAA7WBX7WuvyK44YYbCq+hW758LHwsfCzqf7Wr7kg/IvZIWgjcCwwASyJi\njaQrK8sXS5oKrASOBvZJuho4NSJer91d21WamVkqGrV3iIjlwPKaeYurvt/C21tAB9vHT4Gftlmj\nmZmlxHfkdpFSqVR0CV3Dx2I/H4v9fCw6p056Q6kUIEXRNZiZ9RpJRAYncs3MbBRx6JuZ9RGHvplZ\nH3Hom5n1EYe+mVkfceibmfURh76ZWR9x6JuZ9RGHvplZH3Hom5n1EYe+mVkfceibmfURh76ZWR9x\n6Jvl5M//HH72s6KrsH7X8CUqZta5116Db34Ttm2Dc84puhrrZx7pm+Xg7rvh9NPhX/4F3nyz6Gqs\nnzn0zXJwxx2wcCGccQb8678WXY31M785yyxjr74KM2bAs8/C978PK1bA975XdFXW6/zmLLMudffd\nSR9/wgQ4//xk+ve/L7oq61cOfbOM3XEHXHBB8v3xxye9/Z/8pNiarH859M0y9Oqr8OCDsGDB/nkX\nXAB33llcTdbfmgp9SYOS1kpaJ+nagyw/WdJDkt6UdE3V/JmSHpT0hKTfSPpimsWbdbsf/Qg+8pGk\ntTPs/POT+W7xWBEahr6kAWARMAicClws6ZSa1bYDVwFfq5m/G/hSRJwGzAP+7CDbmo1ad94Jf/zH\nb583bRqcdhrcd18xNVl/a2akPxdYHxEbImI3sBRYUL1CRLwQEatIQr56/paIeKzy/evAGmBaKpWb\ndbk330yC/bzzDlz2R38E//RP+ddk1kzoTwc2Vk1vqsxriaTZwJnAila3NetFzz4LU6e+vbUz7H3v\ng7Vr86/JrJnHMHR8Eb2kI4G7gKsrI/63GRoaeuv7UqlEqVTq9CPNCrdhA8yadfBls2bBM8/kWo71\nuHK5TLlc7ng/DW/OkjQPGIqIwcr0dcC+iPjqQda9AXg9Ir5eNW8scDewPCJuPsg2vjnLRqVvfzu5\nEWvJkgOX7d4NRxwBO3fC2LH512a9L8ubs1YBcyTNljQOuBBYNlIdNUUJWAI8ebDANxvNnnlm5JH+\n2LHJNfubNuVbk1nD0I+IPcBC4F7gSeCHEbFG0pWSrgSQNFXSRuBLwP+U9GylpfMh4NPAxyQ9Wvka\nzOynMesiGzbA7NkjL3eLx4rQ1KOVI2I5sLxm3uKq77cAMw+y6b/hG8CsTz3zjEPfuo8D2Swj9U7k\ngkPfiuHQN8vArl3JC1Om17m4edas5A+DWZ4c+mYZ2LgxCfwxdRqoHulbERz6Zhlo1NqBpN/v0Le8\nOfTNMtDoJC7AO9+ZXLK5b18uJZkBDn2zTDS6XBNg/PjkEQ1btuRRkVnCoW+WgWbaO+CTuZY/h75Z\nBppp74BP5lr+HPpmGWimvQMOfcufQ98sZXv2JH36GTMar+sreCxvDn2zlG3aBFOmNPf0TI/0LW8O\nfbOUNdvaAZ/Itfw59M1SVu+RyrWGR/p+pYTlxaFvlrJWRvpHHw3jxsH27VlWZLafQ98sZa2M9MEn\ncy1fDn2zlLUy0gf39S1fDn2zlLUT+h7pW14c+mYp2rsXnnsOZh7sPXIjcOhbnhz6Zil6/nmYNAkO\nPbT5bRz6lieHvlmKmn3mTjWHvuXJoW+Womafrllt9myfyLX8OPTNUrR5M0yb1to2xx4Lb74JO3dm\nU5NZNYe+WYq2bk2eu9MKCY47LnmRulnWGoa+pEFJayWtk3TtQZafLOkhSW9KuqaVbc1Gm3ZCH5Jt\ntm5Nvx6zWnVDX9IAsAgYBE4FLpZ0Ss1q24GrgK+1sa3ZqLJli0Pfulujkf5cYH1EbIiI3cBSYEH1\nChHxQkSsAna3uq3ZaLN1K0yd2vp2Dn3LS6PQnw5srJreVJnXjE62NetJ7bZ3jjvOoW/5GNNgeScP\nfG1626Ghobe+L5VKlEqlDj7WrBh798KOHcnNWa2aMgXWr0+/Jhs9yuUy5XK54/00Cv3ngOobymeS\njNib0fS21aFv1qteeAEmToQxjX6rDmLKFPjFL9KvyUaP2gHxjTfe2NZ+GrV3VgFzJM2WNA64EFg2\nwrrqYFuzntduPx/c07f81B2TRMQeSQuBe4EBYElErJF0ZWX5YklTgZXA0cA+SVcDp0bE6wfbNssf\nxqxI7fbzwaFv+Wn4P6IRsRxYXjNvcdX3W3h7G6futmajVSeh75uzLC++I9csJe1eow9wzDHw2muw\na1e6NZnVcuibpaSTnv4hh8DkyR7tW/Yc+mYp6aS9A+7rWz4c+mYpcehbL3Dom6Wkk54++GSu5cOh\nb5YSj/StFzj0zVIw/AiGyZPb34dD3/Lg0DdLQSePYBjm0Lc8OPTNUtBpawcc+pYPh75ZCtIIfZ/I\ntTw49M1S0MmNWcM80rc8OPTNUtDp5ZqQnATesSM5KWyWFYe+WQrSaO+MGQMTJsCLL6ZTk9nBOPTN\nUpBG6EOyD/f1LUsOfbMUpNHTB78r17Ln0DdLQRo9ffDJXMueQ98sBWm2dxz6liWHvlmH0ngEwzCH\nvmXNoW/WoRdf7PwRDMN8Itey5tA361Ba/XzwiVzLnkPfrENp9fPB7R3LnkPfrEMOfeslDn2zDqV1\njT7sf+haRDr7M6vVMPQlDUpaK2mdpGtHWOeWyvLVks6smn+dpCckPS7p/0g6NM3izbpBmj398ePh\nsMPg5ZfT2Z9ZrbqhL2kAWAQMAqcCF0s6pWad+cBJETEHuAK4tTJ/NvB54KyIOB0YAC5KuX6zwqXZ\n3gG3eCxbjUb6c4H1EbEhInYDS4EFNeucB9wOEBErgAmSpgCvAruBwyWNAQ4HnkuzeLNu4NC3XtIo\n9KcDG6umN1XmNVwnInYAXweeBZ4HXo6I+zor16z7bNvm0Lfe0eh2kmZPJ+mAGdK7gP8OzAZeAe6U\n9CcR8YPadYeGht76vlQqUSqVmvxYs+Jt3ZqcgE3LlCnJO3fNqpXLZcrlcsf7aRT6zwEzq6Znkozk\n660zozKvBPy/iNgOIOkfgA8CdUPfrJfs25fckZvGIxiGTZ7su3LtQLUD4htvvLGt/TRq76wC5kia\nLWkccCGwrGadZcClAJLmkbRxtgJPAfMkHSZJwLnAk21VadalduyAo46CcePS26fflWtZqjvSj4g9\nkhYC95JcfbMkItZIurKyfHFE3CNpvqT1wE7gs5Vlj0n6O5I/HPuAR4BvZ/izmOUu7X4+OPQtWw0f\nERURy4HlNfMW10wvHGHbvwb+upMCzbrZtm3p9vPBoW/Z8h25Zh1I+yQuOPQtWw59sw64vWO9xqFv\n1oEsRvoTJsDrr8OuXenu1wwc+mYdyWKkf8ghyWWbvlbfsuDQN+tAFiN9cIvHsuPQN+tAFiN9cOhb\ndhz6Zh3I4pJNcOhbdhz6Zh1we8d6jUPfrE07d8LevcljGNLm0LesOPTN2jTc2tEBz5jtnEPfsuLQ\nN2tTVidxwaFv2XHom7Upq5O44NC37Dj0zdqU1UlccOhbdhz6Zm3Ksr0z/CKVaPbddWZNcuibtSnL\nkf4RR8DAQPIMHrM0OfTN2pTlSB/c4rFsOPTN2pTlSB8c+pYNh75ZmzzSt17k0DdrU5aXbIJD37Lh\n0Ddrw5498NJLcOyx2X2GQ9+y4NA3a8P27TBxIowZk91nOPQtCw59szZkfRIXHPqWDYe+WRuyPokL\nDn3LRsPQlzQoaa2kdZKuHWGdWyrLV0s6s2r+BEl3SVoj6UlJ89Is3qwoHulbr6ob+pIGgEXAIHAq\ncLGkU2rWmQ+cFBFzgCuAW6sWfwO4JyJOAc4A1qRYu1lhsr5yBxz6lo1GI/25wPqI2BARu4GlwIKa\ndc4DbgeIiBXABElTJL0D+EhE/G1l2Z6IeCXd8s2KkUd7Z9Ik2LEjeVGLWVoahf50YGPV9KbKvEbr\nzABOAF6QdJukRyR9R9LhnRZs1g3yaO+MGQPveEcS/GZpaXTBWbPP+Kt9d1BU9n0WsDAiVkq6GfgL\n4C9rNx4aGnrr+1KpRKlUavJjzYqRx0gf9rd4Jk/O/rOsu5XLZcrlcsf7aRT6zwEzq6Znkozk660z\nozJPwKaIWFmZfxdJ6B+gOvTNekEeI33YH/qnnZb9Z1l3qx0Q33jjjW3tp1F7ZxUwR9JsSeOAC4Fl\nNessAy4FqFyd83JEbI2ILcBGSe+urHcu8ERbVZp1mbxH+mZpqTvSj4g9khYC9wIDwJKIWCPpysry\nxRFxj6T5ktYDO4HPVu3iKuAHlT8YT9csM+tJEfm1XBz6lraGN5FHxHJgec28xTXTC0fYdjXwgU4K\nNOs2r72WvODkiCOy/yyHvqXNd+SatWjzZjj++Hw+y6FvaXPom7Xo+edh2rR8Puu445KTxmZpceib\ntWjz5vxCf+pU2LIln8+y/uDQN2tRniP9adOSPzJmaXHom7Uoz9CfOjUJ/X378vk8G/0c+mYtyjP0\nx4+Ho45KXtpilgaHvlmL8gx9cIvH0uXQN2tR3qF//PHJZ5qlwaFv1oKIJIDzuk4fkj8wDn1Li0Pf\nrAWvvAJjx8KRR+b3mccf7/aOpcehb9aCvFs74JG+pcuhb9YCh771Ooe+WQuKCH23dyxNDn2zFnik\nb73OoW/WgiJCf/j5O74r19Lg0DdrQRGh77tyLU0OfbMWFBH64BaPpcehb9aCokLfJ3MtLQ59syZF\n5PvWrGoe6VtaHPpmTdq+PbkTd/z4/D/boW9pceibNamo1g64vWPpceibNanI0PdI39Li0DdrUtEj\nfYe+paFh6EsalLRW0jpJ146wzi2V5aslnVmzbEDSo5J+lFbRZkUoeqTv9o6loW7oSxoAFgGDwKnA\nxZJOqVlnPnBSRMwBrgBurdnN1cCTQKRVtFkRih7p+65cS0Ojkf5cYH1EbIiI3cBSYEHNOucBtwNE\nxApggqQpAJJmAPOB7wJKs3CzvBUZ+ocemtyV++KLxXy+jR6NQn86sLFqelNlXrPr3AR8GfD4xHpe\n3m/MquUWj6VhTIPlzbZkakfxkvRJYFtEPCqpVG/joaGht74vlUqUSnVXNytEkSN92H8Fz3vfW1wN\nVpxyuUy5XO54P4oYOdclzQOGImKwMn0dsC8ivlq1zreAckQsrUyvBUrAF4HPAHuA8cDRwN9HxKU1\nnxH1ajDrBnv3Jjdl7dwJ48YVU8Nll8E558Dllxfz+dZdJBERLbfNG7V3VgFzJM2WNA64EFhWs84y\n4NJKEfOAlyNiS0RcHxEzI+IE4CLggdrAN+sVL7wAEycWF/jga/UtHXXbOxGxR9JC4F5gAFgSEWsk\nXVlZvjgi7pE0X9J6YCfw2ZF2l2bhZnnavLnY1g4k5xPWrCm2But9jXr6RMRyYHnNvMU10wsb7OOn\nwE/bKdCsGxTdz4fk8x94oNgarPf5jlyzJjz3XLFX7oDbO5YOh75ZEzZsgNmzi63Bj2KwNDj0zZrw\n9NPwrncVW8Pxx8PWrb4r1zrj0DdrQjeE/qGHwoQJyeMYzNrl0DdrQjeEPsCJJya1mLXLoW/WwI4d\nSUvl2GOLriT5w+PQt0449M0aGB7lqwseGejQt0459M0a6JbWDiR1/Pa3RVdhvcyhb9ZAt4W+R/rW\nCYe+WQMOfRtNHPpmDXRT6E+dCv/xH/Dqq0VXYr3KoW/WQDeFvuTLNq0zDn2zOt54I3lF4YwZRVey\nn1s81gmHvlkdv/sdzJoFAwNFV7KfQ9864dA3q6ObWjvDHPrWCYe+WR0OfRttHPpmdTj0bbRx6JvV\n8fTTydUy3WTWrOT1jbt2FV2J9SKHvlkd3TjSHzsWpk9PXuxi1iqHvtkI9u6FZ57pvpE+uMVj7XPo\nm41g06bkccqHHVZ0JQdy6Fu7HPpmI+jG1s4wh761y6FvNgKHvo1GTYW+pEFJayWtk3TtCOvcUlm+\nWtKZlXkzJT0o6QlJv5H0xTSLN8uSQ99Go4ahL2kAWAQMAqcCF0s6pWad+cBJETEHuAK4tbJoN/Cl\niDgNmAf8We22Zt2qm0P/xBOTR0Ts21d0JdZrmhnpzwXWR8SGiNgNLAUW1KxzHnA7QESsACZImhIR\nWyLiscr814E1wLTUqjfL0FNPwbvfXXQVB3fUUcnX5s1FV2K9ppnQnw5srJreVJnXaJ23PZdQ0mzg\nTGBFq0Wa5W3nTli/Ht7znqIrGZlbPNaOMU2sE03uq/a10W9tJ+lI4C7g6sqI/22Ghobe+r5UKlEq\nlZr8SLNsPPIInH46HHpo0ZWM7KSTYN06OOecoiuxPJTLZcrlcsf7UUT9TJc0DxiKiMHK9HXAvoj4\natU63wLKEbG0Mr0W+GhEbJU0FrgbWB4RNx9k/9GoBrO8ff3rSc980aKiKxnZTTclof/NbxZdiRVB\nEhFRO9huqJn2zipgjqTZksYBFwLLatZZBlxaKWQe8HIl8AUsAZ48WOCbdauVK2Hu3KKrqO/ss2GF\nm6XWooahHxF7gIXAvcCTwA8jYo2kKyVdWVnnHuC3ktYDi4H/Vtn8Q8CngY9JerTyNZjFD2KWpocf\n7v7QP/NMWLMmebuXWbMatncyL8DtHesyL76YnCR96SU4pMtvX3z/++Eb34APfajoSixvWbZ3zPrK\nypVJmHZ74INbPNa6HvhnbZavhx+GD3yg6Cqac/bZ8MtfFl2F9RKHvlmNXjiJO2zePI/0rTUOfbMq\nEb1xEnfYnDnw2muwZUvRlVivcOibVXn2WRgYSN5M1Quk5A+UR/vWLIe+WZXhUb5aviaiOD6Za61w\n6JtV6aWTuMMc+tYKh75ZlV46iTts7lxYtSp5p69ZIw59s4q9e5MHrb3//UVX0ppJk2DyZFi7tuhK\nrBc49M0qVq6EGTPgmGOKrqR1vnTTmuXQN6v4/vfh4ouLrqI97utbs/zsHTNg167kMs2HH4YTTii6\nmtb9+tfwqU8lj4PuhcdHWOf87B2zDvz4x3Dyyb0Z+ABnnJG0pe6/v+hKrNs59M2A730PPvOZoqvo\nzJ/+KSxZUnQV1u3c3rG+9/LLMGsWbNgAEycWXU37Xnop+T+Vp5+GY48tuhrLmts7Zm268074wz/s\n7cCHpP5PfjI5IW02Eoe+9b3R0NoZ9rnPwXe/mzw4zuxgHPrW1zZsSF45+PGPF11JOj760eT1iStX\nFl2JdSuHvvW1m2+Giy6CceOKriQdhxwCl1/uE7o2Mp/Itb51//1w6aWwenXyKIPR4vnn4T3vgXXr\nfEJ3NPOJXLMWbN8Ol10Gt902ugIfYNq0pLd/ySV+CJsdyCN96zsRcMEFMHMm3HRT0dVkY88eGBxM\nHh73V39VdDWWhcxG+pIGJa2VtE7StSOsc0tl+WpJZ7ayrVnebrsN/v3f4StfKbqS7IwZA0uXwg9/\nCHfcUXQ11k3qhr6kAWARMAicClws6ZSadeYDJ0XEHOAK4NZmt7W3K5fLRZfQNbI4Fjt3wjXXwPXX\nww9+AOPHp/4RmWj3WEyaBP/4j7BwIfz85+nWVBT/jnSu0Uh/LrA+IjZExG5gKbCgZp3zgNsBImIF\nMEHS1Ca3tSr+B71f2sfi/vuT59Ns2QKPPw6nn57q7jPVybF43/uS/7O55BI4//zk8tRe5t+Rzo1p\nsHw6sLFqehNwdhPrTAemNbGtWereeCMJ90cegfvuSwJ/1y74m7+BT3yi6Ory94lPJO2sRYuS6/hL\nJfjwh+Gss5I/CkceWXSFlqdGod/sGdaOXiP9qU91svXo8dRT8KtfFV1Fd3jqqeQVgLD/7tKI/V/7\n9iVXpuzZA7//fRL0b7wB27Yl/506FU47Dc49F77wheQSxn5+5PBhh8GXvwyf/3zy2Ilf/Sp5XMPj\nj8PYsckjHCZOhMMPT6bHjEm+pOS4SQe+LL6Il8f7d6Rzda/ekTQPGIqIwcr0dcC+iPhq1TrfAsoR\nsbQyvRb4KHBCo20r833pjplZG9q5eqfRSH8VMEfSbOB54EKg9t1Cy4CFwNLKH4mXI2KrpO1NbNtW\n0WZm1p66oR8ReyQtBO4FBoAlEbFG0pWV5Ysj4h5J8yWtB3YCn623bZY/jJmZ1Vf4zVlmZpaf3E5t\ndXKT12jT6FhI+pPKMfi1pF9IOqOIOvPQ7A18kj4gaY+k/5pnfXlq8nekJOlRSb+RVM65xNw08Tsy\nSdKPJT1WORaXFVBm5iT9raStkh6vs05ruRkRmX+RtHfWA7OBscBjwCk168wH7ql8fzbwyzxqy/ur\nyWPxn4F3VL4f7OdjUbXeA8DdwPlF113gv4sJwBPAjMr0pKLrLvBYDAFfGT4OwHZgTNG1Z3AsPgKc\nCTw+wvKWczOvkX67N3lNyam+PDU8FhHxUES8UplcAczIuca8NHsD31XAXcALeRaXs2aOxSXA30fE\nJoCIeDHnGvPSzLHYDBxd+f5oYHtE7MmxxlxExM+Bl+qs0nJu5hX6I93A1Wid0Rh2zRyLap8D7sm0\nouI0PBaSppP8wt9amTVaT0I18+9iDnCMpAclrZI0St73dYBmjsV3gNMkPQ+sBq7OqbZu03JuNrpk\nMy3t3uQ1Gn/Bm/6ZJH0MuBz4UHblFKqZY3Ez8BcREZJEhzcCdrFmjsVY4CzgvwCHAw9J+mVErMu0\nsvw1cyyuBx6LiJKkdwE/kfTeiHgt49q6UUu5mVfoPwfMrJqeSfIXqd46MyrzRptmjgWVk7ffAQYj\not7/3vWyZo7FfyK5BwSS3u3HJe2OiGX5lJibZo7FRuDFiHgDeEPSz4D3AqMt9Js5Fh8E/hdARDwt\n6XfAH5DcW9RPWs7NvNo7b93kJWkcyY1atb+0y4BL4a07gV+OiK051ZenhsdC0juBfwA+HRHrC6gx\nLw2PRUScGBEnRMQJJH39L4zCwIfmfkf+GfiwpAFJh5OcuHsy5zrz0MyxWAucC1DpYf8B8Ntcq+wO\nLedmLiP96OAmr9GmmWMB/CUwEbi1MsLdHRFzi6o5K00ei77Q5O/IWkk/Bn4N7AO+ExGjLvSb/Hfx\nv4HbJK0mGbz+j4jYUVjRGZH0f0keazNJ0kbgBpI2X9u56ZuzzMz6SB8/d9DMrP849M3M+ohD38ys\njzj0zcz6iEPfzKyPOPTNzPqIQ9/MrI849M3M+sj/B/lIdTt46O5YAAAAAElFTkSuQmCC\n", "text/plain": [ - "" + "" ] }, "metadata": {}, diff --git a/notebooks/05_Odds_and_Addends.ipynb b/notebooks/05_Odds_and_Addends.ipynb index 9bc1267..68ef1df 100644 --- a/notebooks/05_Odds_and_Addends.ipynb +++ b/notebooks/05_Odds_and_Addends.ipynb @@ -1,23 +1,37 @@ { "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "%load_ext autoreload\n", + "%autoreload 2 \n", + "import sys\n", + "sys.path.insert(0,'../code')" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ + "#Odds and Addends\n", "\n", - "Odds and Addends\n", - "================\n", - "\n", - "Odds\n", - "----\n", + "##Odds\n", "\n", "One way to represent a probability is with a number between 0 and 1, but\n", "that’s not the only way. If you have ever bet on a football game or a\n", "horse race, you have probably encountered another representation of\n", - "probability, called **odds**.\n", + "probability, called **odds**.\n", "\n", "You might have heard expressions like “the odds are three to one,” but\n", - "you might not know what that means. The **odds in favor**\n", + "you might not know what that means. The **odds in favor**\n", "of an event are the ratio of the probability it will occur to the\n", "probability that it will not.\n", "\n", @@ -28,56 +42,190 @@ "You can write odds in decimal form, but it is most common to write them\n", "as a ratio of integers. So “three to one” is written $3:1$.\n", "\n", - "When probabilities are low, it is more common to report the **odds\n", - "against** rather than the odds in favor. For example, if I think\n", + "When probabilities are low, it is more common to report the **odds\n", + "against** rather than the odds in favor. For example, if I think\n", "my horse has a 10% chance of winning, I would say that the odds against\n", "are $9:1$.\n", "\n", "Probabilities and odds are different representations of the same\n", - "information. Given a probability, you can compute the odds like this:\n", - "\n", - " def Odds(p):\n", - " return p / (1-p)\n", + "information. Given a probability, you can compute the odds like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.11111111111111112" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def Odds(p):\n", + " return p / (1-p)\n", "\n", + "Odds(0.1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "Given the odds in favor, in decimal form, you can convert to probability\n", - "like this:\n", - "\n", - " def Probability(o):\n", - " return o / (o+1)\n", + "like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.09999999999999999" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def Probability(o):\n", + " return o / (o+1)\n", "\n", + "Probability(1/9.)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "If you represent odds with a numerator and denominator, you can convert\n", - "to probability like this:\n", - "\n", - " def Probability2(yes, no):\n", - " return yes / (yes + no)\n", + "to probability like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.1" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def Probability2(yes, no):\n", + " return float(yes) / (yes + no)\n", "\n", + "Probability2(1, 9)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "When I work with odds in my head, I find it helpful to picture people at\n", "the track. If 20% of them think my horse will win, then 80% of them\n", - "don’t, so the odds in favor are $20:80$ or $1:4$.\n", - "\n", + "don’t, so the odds in favor are $20:80$ or $1:4$." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.2" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Probability2(20, 80)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "If the odds are $5:1$ against my horse, then five out of six people\n", - "think she will lose, so the probability of winning is $1/6$.\n", - "\n", - "The odds form of Bayes’s theorem\n", - "--------------------------------\n", + "think she will lose, so the probability of winning is $1/6$." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.16666666666666663" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "1 - Probability2(5, 1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The odds form of Bayes’s theorem\n", "\n", "In Chapter [intro] I wrote Bayes’s theorem in the **probability\n", "form**:\n", - "$${\\ensuremath{\\mathrm{p}(H|D)}} = \\frac{{\\ensuremath{\\mathrm{p}(H)}}~{\\ensuremath{\\mathrm{p}(D|H)}}}{{\\ensuremath{\\mathrm{p}(D)}}}$$\n", + "$${{\\mathrm{p}(H|D)}} = \\frac{{{\\mathrm{p}(H)}}~{{\\mathrm{p}(D|H)}}}{{{\\mathrm{p}(D)}}}$$\n", "If we have two hypotheses, $A$ and $B$, we can write the ratio of\n", "posterior probabilities like this:\n", - "$$\\frac{{\\ensuremath{\\mathrm{p}(A|D)}}}{{\\ensuremath{\\mathrm{p}(B|D)}}} = \\frac{{\\ensuremath{\\mathrm{p}(A)}}~{\\ensuremath{\\mathrm{p}(D|A)}}}\n", - " {{\\ensuremath{\\mathrm{p}(B)}}~{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "$$\\frac{{{\\mathrm{p}(A|D)}}}{{{\\mathrm{p}(B|D)}}} = \\frac{{{\\mathrm{p}(A)}}~{{\\mathrm{p}(D|A)}}}\n", + " {{{\\mathrm{p}(B)}}~{{\\mathrm{p}(D|B)}}}$$\n", "Notice that the normalizing constant, $\\mathrm{p}(D)$,\n", "drops out of this equation.\n", "\n", "If $A$ and $B$ are mutually exclusive and collectively exhaustive, that\n", - "means ${\\ensuremath{\\mathrm{p}(B)}} = 1 - {\\ensuremath{\\mathrm{p}(A)}}$,\n", + "means ${{\\mathrm{p}(B)}} = 1 - {{\\mathrm{p}(A)}}$,\n", "so we can rewrite the ratio of the priors, and the ratio of the\n", "posteriors, as odds.\n", "\n", "Writing $\\mathrm{o}(A)$ for odds in favor of $A$, we get:\n", - "$${\\ensuremath{\\mathrm{o}(A|D)}} = {\\ensuremath{\\mathrm{o}(A)}}~\\frac{{\\ensuremath{\\mathrm{p}(D|A)}}}{{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "$${{\\mathrm{o}(A|D)}} = {{\\mathrm{o}(A)}}~\\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", "In words, this says that the posterior odds are the prior odds times the\n", "likelihood ratio. This is the **odds form** of Bayes’s\n", "theorem.\n", @@ -94,13 +242,17 @@ "\n", "The prior probability is 50%, so the prior odds are $1:1$, or just 1.\n", "The likelihood ratio is $\\frac{3}{4} / \\frac{1}{2}$, or $3/2$. So the\n", - "posterior odds are $3:2$, which corresponds to probability $3/5$.\n", - "\n", - "Oliver’s blood {#oliver}\n", - "--------------\n", + "posterior odds are $3:2$, which corresponds to probability $3/5$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Oliver’s blood\n", "\n", - "Here is another problem from MacKay’s *Information Theory,\n", - "Inference, and Learning Algorithms*:\n", + "Here is another problem from MacKay’s *Information Theory,\n", + "Inference, and Learning Algorithms*:\n", "\n", "> Two people have left traces of their own blood at the scene of a\n", "> crime. A suspect, Oliver, is tested and found to have type ‘O’ blood.\n", @@ -121,9 +273,9 @@ "\n", "The odds form of Bayes’s theorem provides a way to make this intuition\n", "more precise. Again\n", - "$${\\ensuremath{\\mathrm{o}(A|D)}} = {\\ensuremath{\\mathrm{o}(A)}}~\\frac{{\\ensuremath{\\mathrm{p}(D|A)}}}{{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "$${{\\mathrm{o}(A|D)}} = {{\\mathrm{o}(A)}}~\\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", "Or dividing through by $\\mathrm{o}(A)$:\n", - "$$\\frac{{\\ensuremath{\\mathrm{o}(A|D)}}}{{\\ensuremath{\\mathrm{o}(A)}}} = \\frac{{\\ensuremath{\\mathrm{p}(D|A)}}}{{\\ensuremath{\\mathrm{p}(D|B)}}}$$\n", + "$$\\frac{{{\\mathrm{o}(A|D)}}}{{{\\mathrm{o}(A)}}} = \\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", "The term on the left is the ratio of the posterior and prior odds. The\n", "term on the right is the likelihood ratio, also called the **Bayes\n", "factor**.\n", @@ -166,10 +318,14 @@ "event, that leaves the rare event still unexplained. If Oliver doesn’t\n", "account for the ‘O’ blood, then we have two chances to find someone in\n", "the population with ‘AB’ blood. And that factor of two makes the\n", - "difference.\n", - "\n", - "Addends\n", - "-------\n", + "difference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Addends\n", "\n", "The fundamental operation of Bayesian statistics is Update,\n", "which takes a prior distribution and a set of data, and produces a\n", @@ -200,40 +356,104 @@ " compute the distribution of the sums.\n", "\n", "`thinkbayes` provides functions for both. Here’s an example of the first\n", - "approach. First, I’ll define a class to represent a single die as a Pmf:\n", - "\n", - " class Die(thinkbayes.Pmf):\n", - "\n", - " def __init__(self, sides):\n", - " thinkbayes.Pmf.__init__(self)\n", - " for x in xrange(1, sides+1):\n", - " self.Set(x, 1)\n", - " self.Normalize()\n", - "\n", - "Now I can create a 6-sided die:\n", - "\n", - " d6 = Die(6)\n", - "\n", - "And use `thinkbayes.SampleSum` to generate a sample of 1000 rolls.\n", - "\n", - " dice = [d6] * 3\n", - " three = thinkbayes.SampleSum(dice, 1000)\n", - "\n", + "approach. First, I’ll define a class to represent a single die as a Pmf:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import thinkbayes\n", + "\n", + "class Die(thinkbayes.Pmf):\n", + "\n", + " def __init__(self, sides):\n", + " thinkbayes.Pmf.__init__(self)\n", + " for x in range(1, sides+1):\n", + " self.Set(x, 1)\n", + " self.Normalize()\n", + " \n", + " # for Python 3 compliance\n", + " def __hash__(self):\n", + " return id(self)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now I can create a 6-sided die:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "d6 = Die(6)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And use `thinkbayes.SampleSum` to generate a sample of 1000 rolls." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "dice = [d6] * 3\n", + "three = thinkbayes.SampleSum(dice, 1000)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "`SampleSum` takes list of distributions (either Pmf or Cdf objects) and\n", "the sample size, n. It generates n random sums\n", - "and returns their distribution as a Pmf object.\n", - "\n", - " def SampleSum(dists, n):\n", - " pmf = MakePmfFromList(RandomSum(dists) for i in xrange(n))\n", - " return pmf\n", - "\n", - "`SampleSum` uses `RandomSum`, also in `thinkbayes.py`:\n", + "and returns their distribution as a Pmf object." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def SampleSum(dists, n):\n", + " pmf = MakePmfFromList(RandomSum(dists) for i in xrange(n))\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "SampleSum` uses `RandomSum`, also in `thinkbayes.py`:\n", "\n", + "```python\n", " def RandomSum(dists):\n", " total = sum(dist.Random() for dist in dists)\n", " return total\n", + "```\n", "\n", - "RandomSum invokes Random on each distribution\n", + "`RandomSum` invokes `Random` on each distribution\n", "and adds up the results.\n", "\n", "The drawback of simulation is that the result is only approximately\n", @@ -243,36 +463,90 @@ "The other approach is to enumerate all pairs of values and compute the\n", "sum and probability of each pair. This is implemented in `Pmf.__add__`:\n", "\n", + "```python\n", " # class Pmf\n", - "\n", " def __add__(self, other):\n", " pmf = Pmf()\n", " for v1, p1 in self.Items():\n", " for v2, p2 in other.Items():\n", " pmf.Incr(v1+v2, p1*p2)\n", " return pmf\n", + "```\n", "\n", - "self is a Pmf, of course; other can be a Pmf\n", - "or anything else that provides Items. The result is a new\n", + "`self` is a Pmf, of course; `other` can be a Pmf\n", + "or anything else that provides `Items<`. The result is a new\n", "Pmf. The time to run `__add__` depends on the number of items in\n", - "self and other; it is proportional to\n", - "len(self) \\* len(other).\n", + "`self` and `other`; it is proportional to\n", + "`len(self) \\* len(other)`.\n", "\n", - "And here’s how it’s used:\n", + "And here’s how it’s used:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "three_exact = d6 + d6 + d6" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When you apply the `+` operator to a Pmf, Python invokes\n", + "`__add__`. In this example, `__add__` is invoked twice.\n", "\n", - " three_exact = d6 + d6 + d6\n", + "This shows an approximate result generated by\n", + "simulation and the exact result computed by enumeration:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX5+PHPk8m+L4SELJAAYQcBAQEFAoIs4lL7q8vX\nale19etSa+vSTbT92r0utYut1rbaqtVqiwqyiAEURJB9D4EEEhIg+77MzPn9McMkhKyQ5E6S5/16\n8WLuvedOnizzzJ17znmOGGNQSinVt/lYHYBSSqnup8leKaX6AU32SinVD2iyV0qpfkCTvVJK9QOa\n7JVSqh9oN9mLyCIROSgimSLycAvHR4nIZhGpFZEHWzhuE5EdIvJOVwWtlFKqc9pM9iJiA54DFgFj\ngFtEZHSzZkXAvcCvWnma+4H9gA7oV0opi7R3ZT8NOGKMyTbGNACvAdc1bWCMOWOM2QY0ND9ZRJKA\nJcALgHRNyEoppTqrvWSfCJxosp3r3tdRTwHfBZydjEsppVQXai/ZX/CtFxFZCpw2xuxAr+qVUspS\nvu0czwOSm2wn47q674iZwLUisgQIBMJF5O/GmNubNhIRvZevlFIXwBjT4Qvp9q7stwFpIpIiIv7A\nTcDyVtqe80WNMd8zxiQbY1KBm4F1zRN9k7Ze/++xxx6zPAaNU+PszXH2hhh7U5yd1eaVvTHGLiL3\nAKsAG/CiMeaAiNzlPv68iMQDW4FwwCki9wNjjDGVzZ+u09EppZTqEu3dxsEYsxJY2Wzf800eF3Du\nrZ6WnmM9sP4CY1RKKXWRdAZtB6Wnp1sdQodonF1L4+w6vSFG6D1xdpZcyL2fLg1AxFgdg1JK9TYi\ngunCDlqllFJ9gCZ7pZTqBzTZK6VUP6DJXiml+gFN9kr1oLLyWhwOLRWlel674+yVUhevrLyWP/1t\nO5u35hIRHsicmUOYNzuFwUkRVoem+gkdeqlUN/v0szx+/5dtlFfUnXcsbWgM82ancPllyYSG+FsQ\nneqtOjv0UpO9Ut2kqrqeF1/ZScZH2e229fezcdmURObNTmX86IH4+GihWNU2TfZKeYHd+07x3Atb\nKSyq9uyLigzi7q9NQYAPNmSzdUcedvv59+9jY0KYOyuFubNSiIsN6cGoVW+iyV4pC9XV2XnlX3t4\nb03mOftnzRjMHbdNJjS08VZNRWUdGzYd58ON2RzNKWnx+caPGci8WalMn5JIQIB2salGmuyVskhm\nVhHP/mkrefnlnn2hIf7c9eVLufyyNmsFcjS7hA83ZrN+Uw6VVfXnHQ8O8uOK6YOZNyuFtGHRiOht\nnv5Ok71SPazB7uDN/x7g3+8cwOls/Fu+9JJBfPNrU4iODOrwc9U3ONi6/SQfbsxmx56CFuuWJyeG\nM3dWKnNmDiEqMrBLvgfV+2iyV6oHHc8t49nnPz3nNkxgoC9fvXUiV85Ovagr8KLiajI+yuGDjcco\nONV8eQiw2XyYPCGeK2encunEQdhsOm2mP9Fkr1QPcDicvLMqk1ff3EuD3eHZP2ZkLPfcMZX4gaFd\n9rWMMRw4XMi6Ddl8/OkJ6urs57WZMimBR+6/XEfx9COa7JXqZgWnK/ntn7Zy4PAZzz4/Xxu3fmEc\nSxeO6NaEW1PbwKYtuazbmH3O1wd4+P7LuezSxG772sq7aLJXqpsYY1iTcYyX/rnznKvroUOiuO+u\naT0+G/ZkQQX/eGMPm7fmAq4JWj97bJ523vYTmuxVv7P/0Bme+eOn2O1OEgaFkjAojMT4MM//A2ND\nOnU/2+E0vH3oFP89dJoHp6cwMT6c4tIafv/CNrbvzgcgZ3AY1SF+REcGER0VxNn8+sSc4UyMDz/v\nOX/wYSa7T1dgE2FqYgTfmJxMdJDfRX/vxaU1fOPb73nG6z/xaDrjRg+86OdV3q+zyV4H7qpezRjD\n83/9jDNFVQCUlNWw7+C5tzd8fX2IHxhKQpM3gIRBYSQlhBEWGnBO29NV9fxmSzZ7Tld49n285QTP\n//Wzc4ZEhgT7E5UQSkCAL04MtHO9YgCHMTiM4eMTJew5XcG9UwYzMznqor7/6Mgg5s5KYc2HRwF4\n692DmuxVizTZq15t975TnMgrb7ON3e4k92Q5uSfPbxcWGkDioDAS4sMoj/BnQ3UtDh/Bz8+GcRr+\n9uZecj7NPeecq69KIzM+mD2F54+Q6ajyOjv/9/FRfnHlSMbGXlxn7vWLR7I24xjGGHbuKeBodglD\nUy7uTUT1PZrsVa/2zvuNM1XnzUpl+tRETuZXkFdQwcn8Ck7mV1JSVtPq+RWVdRzMrOOT0+WcTGhM\nugLEldZxrKDKUwd8QEww994xjfFjBuJwtnwp39rt8ifmDAdg75lKntqSzZnqei5PjmLMgIsvhzAo\nPowZU5PY9OkJAN5+7yAP/u+Mi35e1bdosle9Vl5+ueceOgifv2YUg+LDYOK57aqq65u8AVSSl1/B\nyYIK8gsqqG9wDZsML6+nKMZBXYAN/3oHCSerCK5p7ISdNyuVr9x6CSHBrnIHtk6OuDnb/pK4MH67\ncDQv7znJF8cndFln6ueWjvIk+02f5vI/n69w/SyUcutQsheRRcDTgA14wRjz82bHRwEvAZOA7xtj\nfu3enwz8HRiI67bln4wxz3Zd+Ko/e2/1Ec/jqZMGtZrcQoL9SRsWQ9qwmHP2O52GwuJq1xtBfgV7\n88rYXFxF9JlaSt2JPjIikG98+VKmdeGQxrAAX+6eMrjV43V2JwG+nZsgNSwlikvGxbNrr2vW7X9X\nHuYbX7n0YkNVfUi7yV5EbMBzwHwgD9gqIsuNMQeaNCsC7gWub3Z6A/CAMWaniIQCn4nImmbnKtVp\nlZX1rNuY7dleunBEh891OA02H8HHRxg4IISBA0KYOD6eq5u0qa21U1Ja0+mRPBcrt7yWR9cd5kuX\nJHJlSudq4NywdBS79hYAsG7jMW783JhOlWpQfVtH/oqnAUeMMdnGmAbgNeC6pg2MMWeMMdtwJfem\n+wuMMTvdjyuBA0BCl0Su+rU1649SX++6+h6SHMm40bHtnmOMYcWRM9y76gDVDY422wYG+jIoPqxH\nE73dafjVJ9kU1zbw1JZsfrrpGOUtzJZtzbjRsaQNdX16sdudvLsqs50zVH/Skb/kROBEk+1c975O\nEZEUXLd5tnT2XKWacjicrFzbeAtn6VVp7V4Bl9Y28MTGLH637Tg5ZTX84bMTbba3QlFNPTVN3oQ+\nPlHCPe/vZ2dB26ONzhIRPrd0pGd79bosqqrPr6Cp+qeO3LO/6BlP7ls4bwL3u6/wz7Fs2TLP4/T0\ndNLT0y/2S6o+7JNteZ5FQcLDApg1o/X73wCfnizjmS05lNY1fvDMKqmmusFBsJ+tW2PtjLiQAJ5Z\nOIoXduSxMss1V6CoxvUm9ZdrxhEZ2P4krGmTE0kcFE5efjnVNQ2s+iCLG64Z3d2hqx6QkZFBRkbG\nBZ/f7gxaEZkOLDPGLHJvPwo4m3fSuo89BlSe7aB17/MD3gVWGmOebuEcnUGrOuV7P17HwcxCAG68\nfgw33zCu1bbZpTX87/v7z9l33YiBfGlCYqc7QXvSlrxSnv40h/I6O9+YnMw1Izo+UeqD9cf43Ytb\nAVcH8x9/fTX+/t7zpqa6Rmdn0Hbkr30bkCYiKSLiD9wELG/t6zcLRoAXgf0tJXqlOiszq8iT6G02\nHxbOG95m+5TIIBYNGwBAdKAfP56Txp2Tk7060QNclhjJ7xeN4dZxCSxNa78/oqnZlw8mOsrVMVta\nVsuHHVgDV/V97f7FG2PswD3AKmA/8Lox5oCI3CUidwGISLyInAAeAH4gIsfdt24uB74IzBWRHe5/\ni7rtu1F93rurGzsdr5ie3KHFO74+MYlr0gby3OIxTB50ft0abxUV5Mf/jBvUYn+EMabVzls/XxvX\nLW68d/+fFYdwOM5f61b1L1oITfUaRcXVfOPBFZ7E9csnFjCsSVmAWruTQC+/Yu8q67KL+POOXO6Z\nMpjLW6ivU1PbwF0PvOep5/Ptu6dzxfS2+zZU79Idt3GU8grvf5DlSfRjRsaek+gzsov5yjt7OFJc\nbVV4Paagso4/fnaC8jo7T358lGc+zTlvKGlQoB9LFjTe4nrr3YMtLnGo+g9N9qpXqKuzs9pd2RFg\n6cI0z+M3DhTwy09cY9J/ufkYtfa+fcuirM5+ziii1UcLeXTdYeqafd9LFqTh7+8acJd9vJQduwt6\nNE7lXTTZq15h/abjVFTWATBwQAhTJ7nm5h0srOTl3Sc97RzGUNjHx5aPjAnhtwtHM2dwtGffkZJq\nXtqVd0678LAAFsxJ9Wy//d7BHotReR9N9srrGWN4b/Vhz/bVV6Vhs/lQ3eDgV59k43Dfnhg7IJRn\nF44mKbz9TtveLizAl4dmpvL1iUkABPnaGB4dfF67axaP8MwC3nfwjGckk+p/NNkrr9e0Zn1goC/z\nZqcAkJFTTL77aj/Ez8aDM1K8apJUT7h+5EBuHjOI5xaNZn5qzHnHBw4I4YrpyZ7tt9/Vq/v+SpO9\n8npNa9ZfOSvVU2Z48bABfPuyFIJ8bdx96WDiQgJae4o+S0S4bUIC8aGtf++fu3qU5/HWHSc5nlvW\nE6EpL6PJXnm15jXrm44wERGuTI3hz1ePJT0luuUnUAxOivD0cYBr3L3qfzTZK6/WkZr1UV2wcHdf\n1HSo5eeWNl7db9x8nDOFVVaEpCykyV55rYupWd+fnS3l/PAHh2lwz0sYlTaAMSNdZRccDifL3z/c\n1lOoPkiTvfJazWvWhyWEcqK81uKovJsxhp9tOsbvth1nX2Elr+zN9xy7ocnV/ZqMY5RX1FkRorKI\nJnvllZrXrF84fzg/23SM+1cdYMWRMzobtBUiwugmi5j/+8Apdp+uAGDShHiGJEcCUF9vZ8WaIy0+\nh+qbNNkrr9S8Zv2RYBsnymupczh5YUcuhTUN7TxD/3XtiIFMinMVfDMYfvNJNhV1dkTknKv7FWsy\nqanVn2N/ocleeaX3mlS3HHXFYFYea5wMdOfkZGLdwy/V+XxEeOCyIYQHuEolnKmu5087cgGYOS2J\nuFjXlX9lVT1rM45ZFqfqWZrslddpWrPe+NvY3mQ9tRmJkSwcev7kIXWumGB/7ps6BIDhUcHcOCYe\ncK0BcN2SxvLHy98/TIO97fV4Vd+gyV55naY169MuTeBs1fboQD/unTak3fVmlcuMpEi+d/lQfjV/\nJMlNSkjMm5VKhHu7qLiajZuOWxWi6kGa7JVXKSquZtOnuZ7tuxaP4umrRjE8KphvT08hIqAjyyar\nsy5PjsLPdu7L3N/fxtKrGquGvv3eIZxO7fDu6zTZK6/SUs36wRFBPHXVKCbF955VprzdwiuHEeRe\nwDwvv5ytO062c4bq7TTZK6/RVs16H71102WMMTh8fVh45TDPvrfe0cVN+jpN9sprtFazXnWdijo7\nP990jAfXHOTKeUPxdS/jmHm0iH0Hz1gcnepOmuyVVzhbs74q2JfSCH+WLBjuqcOuuobTGB5Zd5iN\nJ0rIr6zjX0cLmTsrxXP8LS1/3Kfpq0l5hd37TnGsoIK8xFAKksLYFexDRZ29/RNVh/mI8IXR8Z7t\ntceKSJg8yDO6aeeeAo5ml1gVnupmmuyVV1j+/mEK4kOw+/oQFurP/uJqGnSESJdLT4lm7pDGctCv\nZhVyyZREz7YuXdh3abJXlsvLL+fDnGLKw12zYiPCA/nWtCFEa+nibvHNSwcz0D0DuaLejhnZOElt\n06e55BdUWBWa6kbtJnsRWSQiB0UkU0QebuH4KBHZLCK1IvJgZ85VCuBV91U9QEiwH9eOiuOyxEiL\no+q7QvxtfGdGKjYRrkkbyCPz0rhknOv2jjGG/67U8sd9UZvJXkRswHPAImAMcIuIjG7WrAi4F/jV\nBZyr+rnKyno+2pqLX4NrbH3awDC+PimxnbPUxRobG8ofl4zhG5cmE+Drc06BtHUbj1FcWmNhdKo7\ntHdlPw04YozJNsY0AK8B1zVtYIw5Y4zZBjQvn9fuuUqtWX8Un8p6Uo+VMcwpPHHVSAJ9+9ei4VZJ\nCGssoTBudCxp7ppDdruTd1dltnaa6qXaS/aJwIkm27nufR1xMeeqfqBpzXofA9++YigjYkLaOUt1\nBxHhc0sbC6StXpdFVXW9hRGprtZeoZGLGQ7R4XOXLVvmeZyenk56evpFfFnVWzSvWT9rxmCLI+rf\npk1OJC4hnOziKqhpYNUHWdxwjd559RYZGRlkZGRc8PntJfs8ILnJdjKuK/SO6PC5TZO96j+a1qxf\ndOUw/P319o2VDhdXcXJUNCdO+DD0aBnL3z/MovnDCdZRUV6h+YXw448/3qnz27uNsw1IE5EUEfEH\nbgKWt9K2efGSzpyr+pENOcX8cOUB9mW5atbbbD4snDfc4qj6twaHk59tOkZDgA0CfTmZEEpZRR3L\nVx6yOjTVRdpM9sYYO3APsArYD7xujDkgIneJyF0AIhIvIieAB4AfiMhxEQlt7dzu/GaU9ztTXc/v\nth1n5ZEzHEuNoM7fhyumJxMVGdj+yarb+Nl8uH/aEEQgOjKIylA/SqIC+M+Kwzoyp48QqyvdiYix\nOgbVc361+Rhrsgo5nluGb72ToUdL+c2yBQxLibI6NAW8sCOXtw+e4sTJcuy1doYfKWXx7FS++ZUp\nVoemmhERjDEdLgerM2hVjymorGP98RLKK+owBhJOVjI+LVYTvRe5bXwCSeGBxEQF4fQRyiICWJtx\njBN55VaHpi6SJnvVY948cAqHw0l5eR0hVQ2EVNvPqVmvrBfg68NNY+MZmxDOLJsf0cW1GGN45V+7\nrQ5NXSRN9qpHGGMoqqmnoqoeh9MwoLBGa9Z7qfQh0fx24WgeuXGCZ9TF1h0ntd59L6fJXvUIEeGx\n2cMZlVdNTFENwdV2rVnvpXxEEBGGpUQxe8YQz/6/v7ZbV7PqxfSVpnrMkWPFFB0tJu50DQF+NubN\nSrU6JNWOW/7fuHNWs2q6GLzqXTTZqx6zJqNxfdkZ05IIDfW3MBrVEXGxISyZ39iv8sobe2iwOyyM\nSF0oTfaqR1TXNLBx83HP9lVzh7XRWnmTJUvSqIwPpTLEj1OnK1m97mj7Jymvo8ledSuHe7Wpjz45\nQW2ta5nB5MRwRqXFtHWa8hI7C8q5b10mVcMiOD0wCAP86z/7tUhaL6TJXnUbu9Nw98r9/H7bcZav\nb7waXJA+1LPuqfJuKZFB1DmcRIQHYg/1pyrEl4rKOv7znpZR6G002atuk5FTTG5FLW/tKyDD5sQA\nfr425lw+pN1zlXeIDPTjqqExrjIKUUEUxQQB8M77hz0VS1XvoMledQunMbx5oACA8oo6ootrEWDG\n1CTCQgOsDU51yudGxmETISzEH3tUIDWBNuobHLz+9j6rQ1OdoMledYsteWWcKK/FOA3VFXVEldQB\nsGDuUIsjU50VHxrAnCHRIBATHexZGH7dxmxyTpRaHJ3qKE32qssZY3jDfVVfWdVARFENNqchcVA4\nY0YOsDg6dSE+PyqOS+PDefrqMSyMDwdcv+eXX99jcWSqozTZqy5X5zAkhAZgE6GyvI6YoloAFqSn\nasdsL5USGcQT6WlMGBjG7TdewtnlK7bvzmfP/tPWBqc6RJO96nKBvj58Z0YqP5yYTNSxUnwdBl9f\nH9KvSLE6NNUFUgZHMndWkzIKr+/G6dQyCt5Ok73qNrs+PUFEuWs89vQpSYSHacdsX3HLDePw93Mt\nI5l1rJhNn56wOCLVHk32qlvU1dnJ+CjHs70gXTtm+5IBMcFc3aQ89T/e2EN9g5ZR8Gaa7FW32PRp\nLtU1DQDEx4UybnSsxRGprlRc00B1agSViWEAnDpTxaoPsiyOSrVFk73qMqW1DZ7HTYue6YzZviWr\npJqvvbOX5VmFOIZG4vBx/W7f+O9+Kqu0jIK30mSvusTpqnq+vHwPT358lA37T3EwsxAAm82Hudox\n26ekRAQxMMQ11j4g1B/HkAgAKqvqeevdg1aGptqgyV51ibcOnqLBafj4RAm/2dD4cX7a5EQiIwIt\njEx1NZuP8PnRcQCIgEmNwOn+4PbeqkzOFFZZGJ1qjSZ7ddFKaxtYfdR1JW8MmMNFnmNXzdOO2b5o\n7pBoYoPd6xEE+BI4wlXFtMHu4NW3tIyCN9Jkry7aO4fPUOdwAhDuMPgU1gAQNzCU8aMHWhma6iZ+\nNh+uH+n+3QqMmty4lnDGRzkcy9EyCt6m3WQvIotE5KCIZIrIw620edZ9fJeITGqy/1ER2Scie0Tk\nnyKiA637mOoGB+9kNs6gDDpR4VmkesGcVHx8tGO2r1o0bACLh8Xyu0VjWLZ4NFMmnk34hpf/tdvS\n2NT52kz2ImIDngMWAWOAW0RkdLM2S4Dhxpg04E7gD+79KcAdwGRjzHjABtzcxfErizmchquGDiDQ\n14coXxulB84A7o5ZXWO2Twv0tXHP1MGkRLrKHn/xxvGeUVc79xSwa2+BleGpZtq7sp8GHDHGZBtj\nGoDXgOuatbkW+BuAMWYLECkicUA50AAEi4gvEAzkdWXwynphAb58fVISL10zngnVDs9V/dRJCURF\nasdsfzI4KYIrZze+wf/9NS2j4E3aS/aJQNN50Lnufe22McYUA78GjgMngVJjzNqLC1d5q0AR9m5u\n/DPQGbP90803jMXf3xeAY8dL2bA5p50zVE/xbed4R9+Wz7sxKyLDgG8BKUAZ8IaI3GqM+UfztsuW\nLfM8Tk9PJz09vYNfVnmLLZ/leSbUxMaEcMm4OIsjUlaIjgpi6cI03nrnAAD/fHMvM6cm4+9vsziy\n3i8jI4OMjIwLPr+9ZJ8HJDfZTsZ15d5WmyT3vnRgkzGmCEBE3gJmAm0me9U7rf6wcWz9grnaMdsf\nHS+r4c0DpyiJCSA8LIDyijoKi6pZsfYI1y8ZaXV4vV7zC+HHH3+8U+e3dxtnG5AmIiki4g/cBCxv\n1mY5cDuAiEzHdbvmFHAImC4iQeLqtZkP7O9UdMorGWM4VNQ4cSYvv5x9B10dsz4+wjztmO13Smoa\nuPf9A3yQXcT20xXMXtxYJO3fyw9QUVlnYXQK2kn2xhg7cA+wCleift0Yc0BE7hKRu9xtVgBHReQI\n8Dxwt3v/TuDvuN4wzo7D+lO3fBeqR23NL+fbaw7y0NpDbM8vZ23GMc+xSy8ZRHRUkIXRKStEBfkx\nMznSs50fEcCgOFeRtKrqev69XMsoWE2Msba3XESM1TGoznlo7SH2FVYCcM3wWDb/ZYfnyu37D87i\n0ksGWRmeskhWSTX3rXLdqxeEO5Oi+OuftgLg6+vDb3++mLjYECtD7FNEBGNMh++X6gxa1Sl7T1d4\nEr2vCMk1Dk+iHxATzETtmO23hkUFM2WQqyiawXDUTxgxzFVGwW538uqbe60Mr9/TZK865Y0DpzyP\n56XGsPXj457tK2enYrPpn1R/9gV3gTQ/HyHQ18ZtN433HNuwOYes7BKrQuv39JWpOiyrpJpt+WWA\n62P6rJgQdu93JX8ROWdCjeqfxg0M467JyfzlmvF849Jkxo4ayLTJjVNzXn5dyyhYRZO96rBBoQF8\n5ZJEogL9uDw5kr3bTnqOXTpxEANigi2MTnmLa0cMJDrIz7P9xRvHe4bi7t53iuO5ZVaF1q9pslcd\nFuxn4/+NjufFpeP42iUJfLgx23NswRydMatalpQQzmWXNl7dr1hzxMJo+i9N9qrTAnx9OLL/DGXl\ntYBr1uTkS+Itjkp5syULGsfdZ3yco8sXWkCTvbogqz9sXGN2/hztmFVtS06NZHCSa6ROfb39nE+F\nqmfoK1R1WsHpSnbvOzsqRztmVcuMMew+XcGPMjK59/2DLJg3zHNs5QdHtCJmD9Nkr9pUUWfn4xMl\nOJtMfPtgfeOM2ckT4okdoBNl1PkanIZfbjrGZwXlFNbU40wKI8S9lGHBqUp27tF69z1Jk71q07uZ\nZ3jy46N8c8V+tuSVYrc7Wbch23N8wVztmFUt87f5cN3IxmUp38kqJH3WEM+2dtT2LE32qlW1dgfL\n3UsO5lbUUtXgYNvOk5SUudaYjYoI0tIIqk1LhscS6ucqb5xXUcuAcXGcrYi+fXcB+QUVFkbXv2iy\nV616L7OQ8jo7AHEhAcwZHM2ajMaO2SvnpOLrq39CqnXBfjaWpjVe3a/JL2XSJWdLahjeX5fV8omq\ny+krVbWorM7O6/vzPdufHxVHYVE1O3Y3dszOm51iSWyqd7lmRCwBNh9C/WzMT41h0ZXDPcc+WH+M\n2lq7hdH1H+0tXqL6qTVHC6lqcACQGBbIVUNjePM/+zm7eNnE8XHEDwy1MELVW0QG+vG9K4YyIjqE\n8ABfnE7DoLgw8k9VUF3TwPpNOSxsMlJHdQ+9slctumFUHN+dnkpssD9fm5iID7C2ySgcXWNWdcaU\nQRGEB7iuLX18hMXzmwzDXHsELXPe/TTZqxb5iJCeEs3zS8YyLSGC7bsKKCl1dcxGRgQydVKCxRGq\n3mzurBQC3Mn/eG6ZZ6Uz1X002as2Bfj6ICLnrDE7d1aKdsyqixIS7E/65Y3DMFeu1WGY3U1fsapd\nZwqr2L67cQKMFj1TF8tpDOFjY6kJdA3L3PJZHoVF1RZH1bdpslcejlamr3+wIdtzT3XC2Dji47Rj\nVl247NIaHlh9kFeyzlA3KgYDOJ3mnHpLqutpslcAVNbbuWvFPt48UEC9w+nZ73A4+WCDdsyqrhPo\n60OOe2KeMyKQ8nBXCYXVH2ZRX++wMrQ+TZO9AuC1fQXkV9bx0q48lm1ovH+6Y08BRcWuj9fhYQFM\nu1Q7ZtXFiQ8N4LqRrolVIcF+lCaF4RQor6hj09YTFkfXd2myV5ysqOWdw6c924uGDfA8XtPko/W8\nWSn4+dp6NDbVN900Jp7IAD8QCIwMpCgmCNB6Od1Jk73ipV152N335McOCGVWchQARcXVbNvZOIv2\nyjlaylh1jWA/G18c76qrFBYWQHWYPwY4crSYzKwia4Pro9pN9iKySEQOikimiDzcSptn3cd3icik\nJvsjReRNETkgIvtFZHpXBq8u3q5TFWzKLfVsf21SEiKCMYZX3tjj6ZgdN3ogiYPCrQpT9UFXDR3A\npLhw7r/X8ObyAAAfmElEQVRsCLcnRrrLo8EKHYbZLdpM9iJiA54DFgFjgFtEZHSzNkuA4caYNOBO\n4A9NDj8DrDDGjAYmAAe6MHbVBSICfBk/MAyAeSnRjIxx1aZ/d1Um6z/O8bRbPH94i+crdaFsPsJP\n5qaxeHgsVzdZtvDjLScoLau1MLK+qb0r+2nAEWNMtjGmAXgNuK5Zm2uBvwEYY7YAkSISJyIRwCxj\nzF/cx+zGGF1W3sukRAbx07lp/OCKYdw+wbUo9I7dBfz11V2eNnNnpTB9SmJrT6HURRueGs2IYTEA\n2O1O1q7XYZhdrb1knwg07R7Pde9rr00SkAqcEZGXRGS7iPxZRIIvNmDV9USEGUmRxAb7k5dfzq9/\nt9lz+2bk8Bju+vKliEg7z6LUxVmyoPHT4/sfZGG3O9torTqrvaqXHa1O1DwTGPdzTwbuMcZsFZGn\ngUeAHzU/edmyZZ7H6enppKend/DLqq5UWVXPz57eRHVNAwAx0cE8dN/l+PvpCBzV/WZMS+Kvr+6i\nqKKOwtIaPt2ex8xpyVaH5TUyMjLIyMi44PPbS/Z5QNOfdjKuK/e22iS59wmQa4zZ6t7/Jq5kf56m\nyV5Zw+Fw8tQfPiEvvxwAfz8bD98/k6jIQIsjU/2Fj48P8ZclsiW7iJiiWlauPaLJvonmF8KPP/54\np85v7zbONiBNRFJExB+4CVjerM1y4HYA92ibUmPMKWNMAXBCREa4280H9nUqOtUtVhw5Q2F1/Tn7\nXvnXHnY0qX9zzx1TGZ4a3dOhqX5sXXYR23Hi9PWhcEAQuzILyT5e2v6JqkPaTPbGGDtwD7AK2A+8\nbow5ICJ3ichd7jYrgKMicgR4Hri7yVPcC/xDRHbhGo3zZDd8D6oTDhRW8rttx7nzvX38c28+xhg+\n/Cib/6485Gnz+WtGc8X0wRZGqfqjuSkxDIkKJiTYH6ePcCY2iJUf6DDMriJWLxogIsbqGPoLpzF8\nZ+0hDhVVAXBFchQ3xEXwwyc/9HSGTZmUwCP3X46Pj3bIqp63Ja+U7605RF6+ayHyUScqeeUXSwgN\n9bc4Mu/jng/T4ReqzqDtRzbklHgSvZ+PcP2QaH7xzCZPok9OjOBb37hME72yzLSECKYPiSLA3zUo\n4ER0AOs2HmvnLNURmuz7iVq7k7/uzvNsLx0Wy1/+vI0Sd/XB0BB/Hv3W5QQH+VkVolKICHdMSiYy\nPJCgGjuxZ2pY+UEWzlbKb6uO02TfT+w7U0GJe0hlZIAvxVtyyTpWDLjWBP3OPTO0Tr3yCimRQTy3\ndAzjztQSXGPn1OlKtu/Ob/9E1SZN9v3EpYMieG7xGKYmRDC83vDJJ43z4L5660QmjI2zMDqlzjV+\nUATzZzcW3tNlCy+eJvt+JDk8kKXhwex+L9Ozb8HcoVr3RnmlhVcO4+x8zR27CzxzQNSF0WTfj5zI\nK+ep329B3BOjR4+I5eu3TdJSCMorxQ8MZcqkQZ7t9z/IaqO1ao8m+36iorKOnz79ETW1rvv2sTEh\nPHTfDF2MRHm1Je5PnXab8Pf9BRw5U2FxRL2XJvs+rMq9nqfD4eTXv/uEglOVAPj7+/LIty4nIlxL\nISjvNmFsHP4pkRwZHsmpMD+eXH2o/ZNUi9qrjaN6qXqHk/tXH2BoVDBBWSXs3nfKc+z+u6aROiTS\nwuiU6hgfH2H+1CR27j0JwLb8CnbklzFpUITFkfU+emXfRy0/fJr8yjpW7i/gxZwinO7b8jdeP5YZ\nU5OsDU6pTvif+WnEVLluP9Y3OPj1hqM4dNx9p2my74NKaxv41/4CamvtFBZVM6CwBh8DM6YmceP1\nY6wOT6lOCQ7y4wvDY/FxJ/gDBeWsOabr1HaWJvs+6JU9+ZTVNFBwuhL/WgdRJXWkDI7k3jumaSkE\n1St9fsEIYopcSxVWVTdwuECHYXaWJvs+Jru0hpVHzlBwqhKHwzDwdDURYQE88q3LCQzULhrVOyUn\nhjMnLpzw8npSj5URlVdpdUi9jib7PiY22I+oMzXU19kJqWogosbBQ/fNZOCAEKtDU+qiXDt/OEl5\nlQTVOlibcYx692gz1TGa7PuYFSsPU73tJMOyyhiUX8Wdt09mzMhYq8NS6qJdOnEQsTGui5aKyjo+\n3nKinTNUU5rs+5BPtuXy6r/3AuDf4OT69KFcNXeoxVEp1TVsNh8WzR/m2X5vTSa6FkbHabLvI/Ly\ny3nm+U892xPGxPGV/5loYURKdb35c1Lx93PN+j6aXcLhrGK25ZdR73BaHJn302TfBxhjeOM/B6ir\nswMQNzCUB++Zjs2mv17Vt4SFBjBrhmvJTIcP/Oj9Azy2/gh/333S4si8n2aDXq7W7uDe9/azYn/j\nYuEPfOMywkIDLIxKqe6zyF0vpyIsgN2VtTgchrcPnWKnDsdskyb7Xu6FHblsO17C8YQQCuKCGTsq\nlhHDY6wOS6luMywlilFpA4goqyOkooHy8joAfrMlm3L3p1t1Pk32vdgnuaW8l3mG8grXH3tgrZ3r\nloy0OCqlut+SBcMRIOFkJVUlNTgchqKaBn67NUc7bVuhyb6XKq5p4JmtOZRX1OF0GsLL6xkbEsjk\nCYPaP1mpXu6yKYnEDQzF12EYmFtBcYlrLeVjpTWU6dV9izTZ91LPbs2hrNZOWXkdvnYn8QVVXLto\nhJZDUP2Cn6+Nr97qGm0WVtmA7Xg5kyODeXbhaCID/SyOzju1m+xFZJGIHBSRTBF5uJU2z7qP7xKR\nSc2O2URkh4i801VBK7h5TDz+9Q7sdieJeZXEhAQwZ+YQq8NSqsdMnZTAlIkJAMQXVFH3SR6BOgKt\nVW3+ZETEBjwHLALGALeIyOhmbZYAw40xacCdwB+aPc39wH5Ab6R1oZExIQzJLCUpt5KQajtLFgzH\n319XnVL9y1dvnYifrw0Bjhwt5oMNx6wOyWu19zY4DThijMk2xjQArwHXNWtzLfA3AGPMFiBSROIA\nRCQJWAK8wNmVg1WX2L3/NCeOlxJeUY+/vy+LrhzW/klK9THxcaFcf3XjoIRX/rWHiso6CyPyXu0l\n+0SgaQGKXPe+jrZ5CvguoNPbutg7Kw97Hl85O0XH1at+64alo86pmfPPN10lQ+rsTl7ec9KzPGd/\n117N247eeml+1S4ishQ4bYzZISLpbZ28bNkyz+P09HTS09ts3i9V1tsJ9Xf9uo7nlrF9d777iHDN\nwhHWBaaUxQICfPnKrZfwi2c3AbBq3VFGTknk9ePFHC+v4XRVHQ9OT7U4youXkZFBRkbGBZ8vbY1J\nFZHpwDJjzCL39qOA0xjz8yZt/ghkGGNec28fBNKB+4DbADsQCIQD/zbG3N7saxgdF9u2sjo7976/\nn5lJUXzlkkT+/NJnrNvoujc5Y2oS3713psURKmUtYww//tVGdu5xzSQPHzWAM6kRnsvQh2akMmdI\ntIURdj0RwRjT4dvj7d3G2QakiUiKiPgDNwHLm7VZDtzu/uLTgVJjTIEx5nvGmGRjTCpwM7CueaJX\n7TPG8NtPcyiqaeCdzNM8uvYgGZtyPMevXayTqJQSEb72xYmeelDlBwsZ6tOY3n637Tinqvr3vfw2\nk70xxg7cA6zCNaLmdWPMARG5S0TucrdZARwVkSPA88DdrT1d14Xdf6w+WsTmvFLPdkxhLU53hb9R\naQMYqaURlAIgcVA41zW5+CndeJwB7jH3VQ0OfvNJTr9eqLzddeqMMSuBlc32Pd9s+552nmM9sP5C\nAuzPcstreX57Y9/3wpQYtv51p2f72sV6r16ppv7ftaNZvymHouJqqsrruLSknuJgH5zGEBXoS4PT\nic2nfw5R1hkIXuxvu/Ooc1/FJ4cHklxaT1V1PeAacjZ1UoKV4SnldQIDffnyLZd4trdnHOPqxEi+\nfVkKD89MJdC3fyZ60GTv1e6bOoTZg6PwFeHb04awanWm59i1i0ZovXqlWjBzWhITxsQBrj6v4xnZ\nzEuJRqR/T/XRbOHFwgJ8eWhGKr9dNJozWSWcLqxy7Q8NYO4VKdYGp5SXEhG+dtskz8XQwcOFrG8y\nqKG/0mTv5USE5PBA/rvykGff4vnDCAhot7tFqX4rOTGcpVelebZffm0P1TUNFkZkPU32vcCBw4Uc\nOVoMuKr9LbpyuMURKeX9vnD9GKIigwAoKavh9bf3eY6dKK/lhxmZFPejNwBN9l4ks7iKhhYWTv7v\nisar+vQrhhAZEdiTYSnVKwUH+Z3TWfve6kyO55axLruI+1cdYHtBOU9tycbZTyZ1arL3EgWVdXz/\nw0y+s/YQx8tqPPvz8svZuqNxMeVrFulwS6U66orpyYwdFQuA02l44eUdxAT6Ue9wJfjtBeW8c/iM\nlSH2GE32XsDhNPxmSzZVDQ6OlFTz803HPFcby99vLHg2ZVICSQnhVoWpVK8jItxx+2TPoj57D5ym\nIruUG0YN9LT56648sktrWnuKPkOTvRd480AB+85UAmAT4b5pQ/ARobSsloyNjaMIrtPSCEp12uCk\nCJYsaOys/euru/h8WizDo4IBqHc6+eXmY9S3cAu1L9Fkb7FDRVX8c2++Z/vmsYMY6S7X+v4HR2iw\nu8qzDh8azZiRAyyJUane7qbPjfH0dRWX1PDfdw/x4PQU/G0+BNh8WJoWi18fX9JTk73FMnKKsbtv\n2YwZEMpNY+IBqKuzs3JtlqfddYtH9vtJIUpdqJBgf267aYJn+533D+NT1cB3pqfwzMLRLB4e2+df\nX5rsLXbnpCTumTKYqEA/Hpyegs19dZHxcY5nxZ2BA0KYPqX5mjFKqc5Iv3wIo0a4Ph07HE5eeHkH\nM5MiSQ7vH6PbNNlbTERYPDyWF5eOI9692pTD4WR5k5Woli5M09IISl0kEeGO2yZ7ruB37zvFJ9vy\nLI6q52gG8RIBvo2/im0788k/VQG4Pn7Om937V9lRyhukDolk8fzG9Zpf+udO6urs57Ur6IPr2Gqy\n72EdWZWraWmEq+YNJTjIrztDUqpfuflz4wgPc32KLiyq5t/vHDzn+PtZhdz13j42HC+2Irxuo8m+\nB5XX2Xn0w0wOF1W12uZgZiEHDxcCYLP5cHWTIWNKqYsXGurPbTc2dtb+Z8VBTha4Pkmvyy7it1tz\nsBvDrzdns/VkmUVRdj1N9j2kusHBY+uPsOd0Bd//MJM9pytabPdOk0lUs2cOJjoqqKdCVKrfmDsr\nhbShrlXe7HYnf3llJ8YYJseHkxTm6rC1G8OTHx9ldyuv1d5Gk30PqLU7eXxDFoeLXVf0NXZniwWY\nCk5VsnlrY4eRTqJSqnv4+Ah3fGkSZ1ck3747n2078okM9OP/5qYRH+K6zVPvcPLEhiwOtfFpvLfQ\nZN/NGhxOfvrxUfaeabw6+N8pyS2udP/u6sOcXap30oR4BidF9FSYSvU7w1OjuWruUM/2i6/soL7e\nwYBgf/5vbhrR7vVr/WxCXxiBr8m+m+07U8ln+eWe7a9NTGLx8Njz2lVU1rF2fbZn+1q9qleq2936\nhXGEhvgDcLqwirfedXXWxocG8JO5aaREBPHTuSMY4Z7V3ptpsu9mE+PDeWhGCr4i3Dx2EDeMimux\n3ap1WdTXu4aApQyOZMKYgS22U0p1nbDQAG79wnjP9tvvHqTgtKtO1ZCIIH67aDQpkX2j30yTfQ+Y\nPSSaZxaO5ovjBrV4vL7ewYo1Rzzb1y4e0eenbivlLebPSWVYquu2aoPdwUv/2Ok55tOHXoea7HtI\nSmRQqwl8w+YcSstqAYiOCuKKywb3ZGhK9Ws2mw933DbJs711x0kyPs5u85yNx0sob2EyljfrULIX\nkUUiclBEMkXk4VbaPOs+vktEJrn3JYvIhyKyT0T2ish9XRm8Nyqt7dwyZ06naVYaYQS+vvoerFRP\nGjE8hiubzFR/9vlP+cWzmygsqj6v7X8OneJnm46ybMMRqhscPRnmRWk3q4iIDXgOWASMAW4RkdHN\n2iwBhhtj0oA7gT+4DzUADxhjxgLTgf9tfm5fsuLIGe58bx97OzEud/vufHJPujpwgwL9WJCupRGU\nssIXbxzPgJhgz/Yn23K595H3efu9g9jtrlr3x0qreWGHa3j0oaIqfrwxizp776iD35FLyGnAEWNM\ntjGmAXgNuK5Zm2uBvwEYY7YAkSISZ4wpMMbsdO+vBA4ACV0WvRdZl13E77edoKrBwY/WH2G/ezGS\n9jRdX3ZB+lBCgv27K0SlVBsiwgP55ePzmTsrxbOvrs7Oy6/v5sEfrmHfwTOkRgbzjUuTPcd3n67g\np5uOYnd6/zq2HUn2icCJJtu57n3ttUlq2kBEUoBJwJbOBuntNueW8vSWHIx7jHxKZFCHevCPHCtm\n30HX+pc2mw9LF2ppBKWsFBEeyL13TOMn3597zjyXE3ll/PDJD3nm+S1cMTCML1/SmAK3nizjj5+d\naOnpvIpvB9p09C2ree+j5zwRCQXeBO53X+GfY9myZZ7H6enppKend/BLWm9HQTk/33QUh7vAWUpE\nEMtmDyfYz9buuU3v1c+clnTOR0illHXGjIzlV08s4L3Vmbz29j5PZcz1H+ewbUc+t35hHF8YFccb\nB08RHejH0rTz5850tYyMDDIyMi74fGmvCqOITAeWGWMWubcfBZzGmJ83afNHIMMY85p7+yAwxxhz\nSkT8gHeBlcaYp1t4ftORSpDeakNOMb/6JBuHMSSEBvCLK0cS1YEqlacLq7j7Oytwuj/+/fKJBQxL\nierucJVSnVRYVM1L/9zJ5q255+wfmhpNUnoKN05JIiGs5xdAERGMMR0eG9qR2zjbgDQRSRERf+Am\nYHmzNsuB290BTAdK3YlegBeB/S0l+r5g9pBovn/FUBLDAvnJ3LQOJXqA91ZnehL9+DEDNdEr5aUG\nxATz3Xtn8oMHZxE3MNSz/+ixYja8tIN339pPZVW9hRF2TLtX9gAishh4GrABLxpjfioidwEYY553\ntzk7YqcK+IoxZruIXAFsAHbTeFvnUWPM+02e26uv7OsbHPzt1V0cyixqs52Tzk1aOJFX7llM/AcP\nzmLyJS1PuFJKeY/6egdvvXuQt9896Hn9gute/5dumcCcmUN6bEJkZ6/sO5Tsu5M3J3tjDE//cQsb\nNx/vtq+RnBjO008u1BmzSvUi+QUV/PnlHezcU3DO/rGjYrnj9slsr6ol0NfGomEDui0GTfZd6B9v\n7OHf7xzwbDf4Cg1+NoJrumrmnPDoA5czdVKfHI2qVJ9mjOGTbXm8+MoOiktqXPuA4oHB1A2NJDoq\niEdmpjK7hQq3XUGTfRdZtS6L5//6mWd71pxUtkf4UVTbwDfHJjAm6uJHzsREBREZ0T9Wtleqr6qu\naeCN/+znnVWHsRtDdko4tYG++Pr6EBcTzP9dNZLpSV3fJ6fJvgt8tiufnz71kacD9ZKJgygaM4Cs\nUtfU6UBfH15cOo7IQF0bVinlknOilOf/tp29WUXkDAmnLsA1/DosyI+fLxzFvNEtV7y9UJrsL1JW\ndgk/+L8PPeNqE1Oj8JuZxKESV6IXhIdmpHTbRzOlVO9ljCHjoxz+9K/d7BsQQL2/K+EH2p08Oj6R\nz109Cv8OzMHpiO4YetlvnC6s4slff+RJ9LEDQpDLEj2JHuDeqYM10SulWiQizJ2Vwp9+upDbkqLx\ntTvxa3CSmFPO6nVHPTV2rKDJ3q2ysp6f/GojJWWujpbQEH9+8OAs7p42GJt7pMwdk5JY2I2960qp\nviEsNIDvfm0qv79uPHPsQkC9k6/eOpHgDs7D6Q56GwfXWPof/3KDp06Nr68Pjz00h7GjXFOglx8+\nTXJ4IJPiw60MUynVCzkcTj7dfpLpUxLPGWJdWW8n2M92wQuk6D37Tjo7ln7D5uMYAR8D3757OldM\n1wVElFLdw+E0fD8jE18RvnXZEAZcQLVbvWffSf98cy8fbMslZ0gYpwcGc9tNEzTRK6W61VuHTrHn\ndAU7TpVzz/sH+PhESbd/zX6d7Fety+Iv67M4OjSC6mA/7EMiSJoUb3VYSqk+rs7uRNyFgivq7Tz5\n8VGe2pLdrStf9dtk/9Fnefxo7SHyEkNx+gghwX7EDQghr6LO6tCUUn3cF8cn8OS8NGKb3L5Ze6yI\ntcfarsF1MfrlPfus7BK+/PxmzoS7ftAB/jYmDx/Aw5enMmpAaDtnK6VU16ist/OHz06QkVPMuNgw\nnpybhs2nY7fhtYO2HacLq3j08XUUVtRydGgEJsiXL142hPtmpHRowRGllOpq63OKGTUghLiQgA6f\no8m+DZWV9XzvJ+s8C3ybmCC+dPtkrp/UfJVFpZTyDqer6okN9juvMm5nk31HliXs1YwxVNQ7CPQR\nfv7sx55E7+vrw2N3TfeMpVdKKW9TUtPA/asPMC42lHumDiEi4MJTdp9O9iU1DTz9aQ4lNQ0MPlrq\nmTQFcN+d0zTRK6W8ljGGpz7NobzOzqbcUg4WVvHAZSlMHnRhkzv7bLLfklfK0+4fVHFxDZ8dLeFs\nzTkdS6+U8nZ2pyE+pHG0TnFtAz9cn8m1Iwby5Qmdv/Xc54Ze1todPLc1hyc2ZlFeZ6e8vI6SslrA\ntbDAwnnDuH7JSGuDVEqpdvjZfLh7ymAemz2cyIDGmjrvZp4h213DqzP6XLLflFvKyqxCAKqrGyg9\nVcng4+XEna5h6sQEvn7bJF0CUCnVa0xLiOB3i0czNSECgBtHxzMyJqTTz9PnRuMYY/jxxiw2HCum\n6kgxsbkV+DoMQ1Oi+Mn35hIY2GfvXCml+jBjDBuPlzAzOQpfH9HaOCLCzUNjCd5fRHxOOb4Ow8AB\nIXz/27M00Sulei0RYfaQaHw7OOmquT6X/Sor63nuuc345lcArrr0339wFlGRutarUqr/avfKXkQW\nichBEckUkYdbafOs+/guEZnUmXO7Un2D47yx9A/ffznJiVqHXinVv7WZ7EXEBjwHLALGALeIyOhm\nbZYAw40xacCdwB86em5Xe+kfO7ttLH1GRkaXPE930zi7lsbZdXpDjNB74uys9q7spwFHjDHZxpgG\n4DXgumZtrgX+BmCM2QJEikh8B8/tUksXphEX6+ql7uqx9L3lD0Dj7FoaZ9fpDTFC74mzs9q7Z58I\nnGiynQtc1oE2iUBCB87tUomDwvnpj65kw6Ycrlk0oju/lFJK9SrtJfuOjon0moHrkRGBXLtYJ00p\npVRTbY6zF5HpwDJjzCL39qOA0xjz8yZt/ghkGGNec28fBOYAqe2d695v7UB/pZTqpbqy6uU2IE1E\nUoCTwE3ALc3aLAfuAV5zvzmUGmNOiUhRB87tVLBKKaUuTJvJ3hhjF5F7gFWADXjRGHNARO5yH3/e\nGLNCRJaIyBGgCvhKW+d25zejlFKqZZaXS1BKKdX9LCuXICLJIvKhiOwTkb0icp9VsXSEiNhEZIeI\nvGN1LK0RkUgReVNEDojIfvdtNa8iIo+6f+d7ROSfItLxddi6kYj8RUROicieJvuiRWSNiBwWkdUi\nEmlljO6YWorzl+7f+S4ReUtEIqyM0R3TeXE2OfagiDhFJNqK2JrF0mKcInKv+2e6V0R+3tr5PaWV\n3/s0EfnUnZe2isjUtp7Dyto4DcADxpixwHTgf7t70tVFuh/YT8dHKFnhGWCFMWY0MAHwqttm7v6b\nO4DJxpjxuG7v3WxlTE28hGsCYFOPAGuMMSOAD9zbVmspztXAWGPMJcBh4NEej+p8LcWJiCQDC4Cc\nHo+oZefFKSJzcc0fmmCMGQf8yorAmmnp5/kL4IfGmEnAj9zbrbIs2RtjCowxO92PK3ElpgSr4mmL\niCQBS4AX8KJhpk25r+ZmGWP+Aq4+E2NMmcVhNVeO600+WER8gWAgz9qQXIwxG4GSZrs9Ewbd/1/f\no0G1oKU4jTFrjDFO9+YWIKnHA2umlZ8nwG+Ah3o4nFa1Euc3gZ+6J4NijDlz3ok9rJU484Gzn+Ii\naee15BVVL91XfJNw/aF6o6eA7wLO9hpaKBU4IyIvich2EfmziARbHVRTxphi4NfAcVwjtEqNMWut\njapNccaYU+7Hp8Cz2Jk3+yqwwuogWiIi1wG5xpjdVsfSjjRgtoh8IiIZIjLF6oBa8QjwaxE5DvyS\ndj7RWZ7sRSQUeBO4332F71VEZClw2hizAy+9qnfzBSYDvzfGTMY1Msobbjt4iMgw4FtACq5PcaEi\ncqulQXWQe9EFb76Fh4h8H6g3xvzT6liac194fA94rOlui8Jpjy8QZYyZjusi718Wx9OaF4H7jDGD\ngQeAv7TV2NJkLyJ+wL+BV4wx/7EyljbMBK4VkWPAq8A8Efm7xTG1JBfXVdNW9/abuJK/N5kCbDLG\nFBlj7MBbuH6+3uqUu84TIjIIOG1xPK0SkS/jutXorW+ew3C9ye9yv5aSgM9EZKClUbUsF9ffJu7X\nk1NEYqwNqUXTjDFvux+/iaseWausHI0juN6Z9htjnrYqjvYYY75njEk2xqTi6kxcZ4y53eq4mjPG\nFAAnRORsUaD5wD4LQ2rJQWC6iAS5f//zcXV6e6vlwJfcj78EeOUFiYgswnUFep0xptbqeFpijNlj\njIkzxqS6X0u5uDrqvfEN9D/APAD368nfGFNkbUgtOiIic9yP5+HqnG+dMcaSf8AVuO6B7wR2uP8t\nsiqeDsY8B1hudRxtxHcJsBXYhevKJMLqmFqI8SFcb0J7cHV6+lkdkzuuV3H1I9TjKuD3FSAaWOt+\nEa0GIr0wzq8CmbhGt5x9Hf3ei+KsO/vzbHb8KBDtjXECfsDL7r/Rz4B0L4qz6d/nFFz9nDuBzcCk\ntp5DJ1UppVQ/YHkHrVJKqe6nyV4ppfoBTfZKKdUPaLJXSql+QJO9Ukr1A5rslVKqH9Bkr5RS/YAm\ne6WU6gf+PxvSqgR3RsBIAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import random\n", + "import thinkplot\n", "\n", - "When you apply the + operator to a Pmf, Python invokes\n", - "`__add__`. In this example, `__add__` is invoked twice.\n", + "random.seed(17)\n", + "\n", + "d6 = Die(6)\n", "\n", - "Figure [fig.dungeons1] shows an approximate result generated by\n", - "simulation and the exact result computed by enumeration.\n", + "dice = [d6] * 3\n", + "three = thinkbayes.SampleSum(dice, 1000)\n", + "three.name = 'sample'\n", "\n", - "![Approximate and exact distributions for the sum of three 6-sided\n", - "dice.](figs/dungeons1.pdf)\n", "\n", - "[fig.dungeons1]\n", + "three_exact = d6 + d6 + d6\n", + "three_exact.name = 'exact'\n", "\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(three)\n", + "thinkplot.Pmf(three_exact, linestyle='dashed')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "`Pmf.__add__` is based on the assumption that the random selections from\n", "each Pmf are independent. In the example of rolling several dice, this\n", "assumption is pretty good. In other cases, we would have to extend this\n", @@ -280,66 +554,100 @@ "\n", "The code from this section is available from\n", ". For more information see\n", - "Section [download].\n", - "\n", - "Maxima\n", - "------\n", - "\n", - "![Distribution of the maximum of six rolls of three\n", - "dice.](figs/dungeons2.pdf)\n", - "\n", - "[fig.dungeons2]\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Maxima\n", "\n", - "When you generate a *Dungeons & Dragons* character, you are\n", + "When you generate a *Dungeons & Dragons*\n", + "character, you are\n", "particularly interested in the character’s best attributes, so you might\n", "like to know the distribution of the maximum attribute.\n", "\n", "There are three ways to compute the distribution of a maximum:\n", "\n", - "Simulation:\n", - ": Given a Pmf that represents the distribution for a single selection,\n", + "**Simulation**:\n", + "\n", + " Given a Pmf that represents the distribution for a single selection,\n", " you can generate random samples, find the maximum, and accumulate\n", " the distribution of simulated maxima.\n", "\n", - "Enumeration:\n", - ": Given two Pmfs, you can enumerate all possible pairs of values and\n", + "**Enumeration**:\n", + "\n", + " Given two Pmfs, you can enumerate all possible pairs of values and\n", " compute the distribution of the maximum.\n", "\n", - "Exponentiation:\n", - ": If we convert a Pmf to a Cdf, there is a simple and efficient\n", + "**Exponentiation**:\n", + "\n", + " If we convert a Pmf to a Cdf, there is a simple and efficient\n", " algorithm for finding the Cdf of the maximum.\n", "\n", "The code to simulate maxima is almost identical to the code for\n", - "simulating sums:\n", - "\n", - " def RandomMax(dists):\n", - " total = max(dist.Random() for dist in dists)\n", - " return total\n", - "\n", - " def SampleMax(dists, n):\n", - " pmf = MakePmfFromList(RandomMax(dists) for i in xrange(n))\n", - " return pmf\n", + "simulating sums:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def RandomMax(dists):\n", + " total = max(dist.Random() for dist in dists)\n", + " return total\n", "\n", + "def SampleMax(dists, n):\n", + " pmf = MakePmfFromList(RandomMax(dists) for i in xrange(n))\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "All I did was replace “sum” with “max”. And the code for enumeration is\n", - "almost identical, too:\n", - "\n", - " def PmfMax(pmf1, pmf2):\n", - " res = thinkbayes.Pmf()\n", - " for v1, p1 in pmf1.Items():\n", - " for v2, p2 in pmf2.Items():\n", - " res.Incr(max(v1, v2), p1*p2)\n", - " return res\n", - "\n", + "almost identical, too:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def PmfMax(pmf1, pmf2):\n", + " res = thinkbayes.Pmf()\n", + " for v1, p1 in pmf1.Items():\n", + " for v2, p2 in pmf2.Items():\n", + " res.Incr(max(v1, v2), p1*p2)\n", + " return res" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "In fact, you could generalize this function by taking the appropriate\n", "operator as a parameter.\n", "\n", "The only problem with this algorithm is that if each Pmf has $m$ values,\n", "the run time is proportional to $m^2$. And if we want the maximum of\n", - "k selections, it takes time proportional to $k m^2$.\n", + "`k` selections, it takes time proportional to $k m^2$.\n", "\n", "If we convert the Pmfs to Cdfs, we can do the same calculation much\n", "faster! The key is to remember the definition of the cumulative\n", - "distribution function: $$CDF(x) = {\\ensuremath{\\mathrm{p}(X \\le~x)}}$$\n", + "distribution function: \n", + "\n", + "$$CDF(x) = {{\\mathrm{p}(X \\le~x)}}$$\n", + "\n", "where $X$ is a random variable that means “a value chosen randomly from\n", "this distribution.” So, for example, $CDF(5)$ is the probability that a\n", "value from this distribution is less than or equal to 5.\n", @@ -349,21 +657,33 @@ "Well, in that case both $X$ and $Y$ must be less than or equal to 5.\n", "\n", "If the selections of $X$ and $Y$ are independent,\n", - "$$CDF_3(5) = CDF_1(5) CDF_2(5)$$ where $CDF_3$ is the distribution of\n", + "\n", + "$$CDF_3(5) = CDF_1(5) CDF_2(5)$$ \n", + "\n", + "where $CDF_3$ is the distribution of\n", "$Z$. I chose the value 5 because I think it makes the formulas easy to\n", "read, but we can generalize for any value of $z$:\n", - "$$CDF_3(z) = CDF_1(z) CDF_2(z)$$ In the special case where we draw $k$\n", - "values from the same distribution, $$CDF_k(z) = CDF_1(z)^k$$ So to find\n", + "\n", + "$$CDF_3(z) = CDF_1(z) CDF_2(z)$$ \n", + "\n", + "In the special case where we draw $k$\n", + "values from the same distribution,\n", + "\n", + "$$CDF_k(z) = CDF_1(z)^k$$ \n", + "\n", + "So to find\n", "the distribution of the maximum of $k$ values, we can enumerate the\n", "probabilities in the given Cdf and raise them to the $k$th power. `Cdf`\n", "provides a method that does just that:\n", "\n", - " # class Cdf\n", + "```python\n", + "# class Cdf\n", "\n", - " def Max(self, k):\n", - " cdf = self.Copy()\n", - " cdf.ps = [p**k for p in cdf.ps]\n", - " return cdf\n", + " def Max(self, k):\n", + " cdf = self.Copy()\n", + " cdf.ps = [p**k for p in cdf.ps]\n", + " return cdf\n", + "```\n", "\n", "`Max` takes the number of selections, k, and returns a new\n", "Cdf that represents the distribution of the maximum of k\n", @@ -375,24 +695,88 @@ "$m \\log m$, but that’s still better than quadratic.\n", "\n", "Finally, here’s an example that computes the distribution of a\n", - "character’s best attribute:\n", - "\n", - " best_attr_cdf = three_exact.Max(6)\n", - " best_attr_pmf = best_attr_cdf.MakePmf()\n", - "\n", + "character’s best attribute:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3 9.84640042004852e-15\n", + "4 4.0321009720098687e-11\n", + "5 9.806069563928e-09\n", + "6 6.203232264630567e-07\n", + "7 1.7470129795277652e-05\n", + "8 0.0002855723336481055\n", + "9 0.002477241673570361\n", + "10 0.012844085693359392\n", + "11 0.04397964477539067\n", + "12 0.10559054189425404\n", + "13 0.18102225453979948\n", + "14 0.21201352141686403\n", + "15 0.19422452755628616\n", + "16 0.14145216951432227\n", + "17 0.0786340868804698\n", + "18 0.027458253422613543\n" + ] + } + ], + "source": [ + "best_attr_cdf = three_exact.Max(6)\n", + "best_attr_pmf = best_attr_cdf.MakePmf()\n", + "best_attr_pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "Where `three_exact` is defined in the previous section. If we print the\n", "results, we see that the chance of generating a character with at least\n", "one attribute of 18 is about 3%. Figure [fig.dungeons2] shows the\n", - "distribution.\n", - "\n", - "Mixtures {#mixture}\n", - "--------\n", - "\n", - "![Distribution outcome for random die from a box.](figs/dungeons3.pdf)\n", + "distribution." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH2dJREFUeJzt3WmYVNW1xvH/EtAwGAeuAUUMqBBEweCADN7Yzo2JYMyj\nyMUIeGNIIsZgNIoZJB9ykzhEIHEgKkowEYegkETEIbZigkzKoMwKyOQ8ohDAXvfDLqRpu7uqu6tq\nV9V5f8/TDzWcU/V207169zr77GPujoiIlLY9YgcQEZHcU7EXEUkAFXsRkQRQsRcRSQAVexGRBFCx\nFxFJgLTF3szKzWyZma00s6treH6wmS00s0Vm9i8z617luTWpx180sznZDi8iIpmxuubZm1kTYDlw\nGrABmAsMcvelVbbpDSxx9w/MrBwY7e69Us+tBo5193dz+DmIiEga6Ub2PYFV7r7G3bcDk4EBVTdw\n91nu/kHq7mzg4GqvYVlJKiIiDZau2LcD1lW5vz71WG3+F3i0yn0HnjSzeWZ2ScMiiohIYzVN83zG\naymY2cnAxUDfKg/3dfdNZnYA8ISZLXP3mQ3IKSIijZCu2G8A2le5354wut9N6qDsHUC5u7+383F3\n35T69y0ze5jQFppZbV8tziMi0gDunnGbPF0bZx7Qycw6mNmewEBgWtUNzOwQYApwobuvqvJ4CzPb\nO3W7JXAGsLiWwAX/cd1110XPoJzKWawZlTP7H/VV58je3XeY2QhgBtAEuMvdl5rZ8NTz44FfAPsB\nt5kZwHZ37wm0BaakHmsK/NndH693QhERabR0bRzcfTowvdpj46vc/g7wnRr2exX4ahYyiohII+kM\n2gyVlZXFjpAR5cyuYshZDBlBOWOr86SqvAQw89gZRESKjZnhWTxAKyIiJUDFXkQkAVTsRUQSQMVe\nRCQBVOxFRBJAxV5EJAFU7EVEEkDFXkQkAVTsRUQSQMVeRCQBVOxFRBJAxV5EJAFU7EVEEkDFXkQk\nAVTsRUQSQMVeRCQBVOxFRBJAxV5EJAFU7EVEEkDFXkQkAVTsRUQSQMVeRCQBVOxFRBJAxV5EJAFU\n7EVEEkDFXkQkAVTsRUQSQMVeRHLu4Yehc2eYNSt2kuRSsReRnHrlFRg+HC65BM45B26+Gdxjp0oe\n88hfdTPz2BlEJDe2boU+fWDYMLjsMlizBs4/H9q1g7vvhn33jZ2weJkZ7m6Zbq+RvYjkzBVXwGGH\nwYgR4X6HDjBzJrRvD8ceC/PnR42XKCr2IpIT990Hjz8Od94JVmX8uddeMG4c/OY30K8f3H672jr5\noDaOiGTd8uVw4omh2PfoUft2K1bAeefBkUfC+PGw9975y1js1MYRkag++SQU8F/9qu5CD2GGzvPP\nQ8uWcPzx8NJL+cmYRGmLvZmVm9kyM1tpZlfX8PxgM1toZovM7F9m1j3TfUWk9Fx2GXTrFmbfZKJ5\nc7jjDhg1Ck4+GSZOzG2+pKqzjWNmTYDlwGnABmAuMMjdl1bZpjewxN0/MLNyYLS798pk39T+auOI\nlIiJE0Mvfu5caNWq/vu/9FL4q6BPH/j976FFi+xnLBXZbuP0BFa5+xp33w5MBgZU3cDdZ7n7B6m7\ns4GDM91XRErHSy/BlVfCgw82rNADHHVU+EWxdSv06hV6+pId6Yp9O2BdlfvrU4/V5n+BRxu4r4gU\nqc2bw4j8hhtCwW6MVq3g3nvh0kuhb1944IHsZEy6pmmez7i/YmYnAxcDfeu77+jRoz+7XVZWRllZ\nWaa7ikhk7vC970Hv3jB0aHZe0yycdXv88eGXyMyZcOONYdpmUlVUVFBRUdHg/dP17HsRevDlqfuj\ngEp3/2217boDU4Byd19Vz33VsxcpYnfcEebNz56dmx77++/DxRfDunVhlN+xY/bfoxhlu2c/D+hk\nZh3MbE9gIDCt2hseQij0F+4s9JnuKyLFbcECuPba0KfP1cHUffeFv/4VBg8OffxpqiINUmcbx913\nmNkIYAbQBLjL3Zea2fDU8+OBXwD7AbdZOE1uu7v3rG3fHH4uIpJHH34YWixjx0KXLrl9LzP40Y/g\nhBPgggvguefCPP5mzXL7vqVEZ9CKSL25w8CBsP/+YbmDfHr7bbjoovDLZvJkOPjg9PuUIp1BKyI5\nd+utsHIljBmT//f+r/+Cv/8dTj8dvvENrauTKY3sRaRe5s0LC5jNmgWHHx4vh3uY5jluHJx6arwc\nsWhkLyI58957YT36226LW+gh9PFHjoTf/S5ujmKhkb2IZMQdvvlNOOSQMJouBFu2hDXyn3km9weJ\nC41G9iKSE2PGwMaN4SzZQtG8eTihK8axg2Kjkb2IpDVrVrh+7OzZYSRdSN54I4zqV64MB2+TQiN7\nEcmqd94Jc9vvuKPwCj1AmzZw7rnh4idSO43sRaRWlZVw9tnQtWthtW+qW7wYzjgjXNA8KevnaGQv\nIllz/fVhbZr/+7/YSerWrVv4mDw5dpLCpZG9iNTo2WfDNMt584rjLNXHHoNrroEXX9z9AuelSiN7\nEWm0N9+E//kfuOee4ij0AGeeCdu3w9NPx05SmFTsRWQ3lZVhhckhQ6C8PHaazO1cLE0nWdVMbRwR\n2U1FBYwYEZYvbpru8kYFJkknWamNIyKNMnVqWNGy2Ao97DrJauzY2EkKj0b2IvIZdzj00FDwu3eP\nnaZhknKSlUb2ItJgixeH3ne3brGTNFybNmENH51ktTsVexH5zCOPwIABxT91ceRIuOUW+M9/Yicp\nHCr2IvKZqVPDGjjFrls3OPJIuP/+2EkKh4q9iADw2muwdi307Rs7SXZccUWYhqlDgoGKvYgAYVT/\njW8U5yycmpx5JmzbFqaSioq9iKSUSgtnpz320JWsqtLUSxHhvffgy1+GTZugZcvYabJny5bwec2c\nCV/5Suw02aWplyJSb48+CiefXFqFHnSSVVUq9iLy2ZTLUvSDH8B994WLsCSZ2jgiCbd1K7RtG844\nPeCA2GlyY9gw6NQJrr02dpLsURtHROrln/8M89JLtdBDOFD7hz+E2TlJpWIvknClNgunJt276yQr\ntXFEEqyyEtq1C7NVDj88dprcmj49tHFeeKH4l4MAtXFEpB7mzIHWrUu/0EM4yWrr1uSeZKViL5Jg\npTwLp7qdJ1ndfHPsJHGojSOSYF26wKRJcPzxsZPkx86TrJ57Djp3jp2mcdTGEZGMLFsGmzfDscfG\nTpI/zZvD8OEwZkzsJPmnYi+SUFOnQv/+ob2RJJdeGk6yevfd2EnyK2H/zSKyUxKmXNakbdvweSft\nSlbq2Ysk0OuvwxFHhOu17rln7DT5t2gR9OsHq1cX7+ef9Z69mZWb2TIzW2lmV9fwfBczm2VmW83s\nx9WeW2Nmi8zsRTObk2koEcmtv/0NysuLt9A1Vvfu4ZfdAw/ETpI/dRZ7M2sC/AEoB7oCg8zsiGqb\nvQNcBtxYw0s4UObuPdy9ZxbyikgWJGnKZW2SdiWrdCP7nsAqd1/j7tuBycBu3yLu/pa7zwO21/Ia\nJXCumkjp+OijcMZsv36xk8RVXh6mYj7zTOwk+ZGu2LcD1lW5vz71WKYceNLM5pnZJfUNJyLZN2MG\n9O4N++wTO0lce+wBP/pRck6ySlfsG/sHTl937wH0Ay41s/9u5OuJSCMldRZOTb79bZg1C1asiJ0k\n99JdWngD0L7K/faE0X1G3H1T6t+3zOxhQltoZvXtRo8e/dntsrIyysrKMn0LEamH7dvhH/+A3/wm\ndpLC0KJFOMlq7Fi45ZbYaepWUVFBRSMW9qlz6qWZNQWWA6cCG4E5wCB3X1rDtqOBj9z9ptT9FkAT\nd//IzFoCjwO/dPfHq+2nqZciefLUUzBqVFgATYJNm6BrV3jlFdh//9hpMpfVqZfuvgMYAcwAlgD3\nu/tSMxtuZsNTb9jWzNYBI4GfmdlrZtYKaAvMNLMFwGzg79ULvYjkl1o4n3fggWFm0h//GDtJbumk\nKpGEcA+LgE2fHi7kIbssXAhnnVVcJ1lpITQRqdGCBbDXXqFlIbs7+ujSP8lKxV4kIXaeSFUKV2nK\nhZ1r3Zdqo0HFXiQh1K+vW79+8PHH8OyzsZPkhoq9SAKsXg0bN4aTqaRmO0+yGjcudpLcULEXSYBp\n0+Dss6FJk9hJCtugQWF66ttvx06SfSr2IgnwyCNq4WRin33g618PFzcpNSr2IiXunXfghRfgtNNi\nJykOQ4fCxImxU2Sfir1IifvHP+DUU8P1VyW9U04JF3d56aXYSbJLxV6kxGnt+vpp0gQuuqj0Rvc6\ng1akhG3ZEq65+uqr0Lp17DTFY/lyKCuDdeugabrlIiPRGbQi8pknn4QePVTo6+srX4EOHeDxElrN\nS8VepITpRKqGGzIE7rkndorsURtHpER9+ikcdBA8/zx07Bg7TfF5773wdVu9GvbbL3aaz1MbR0SA\nUOTbtlWhb6j99oMzz4TJk2MnyQ4Ve5ESpVk4jVdKc+5V7EVKkLvOms2G00+H116DZctiJ2k8FXuR\nErR0KfznP2EmjjRc06Zw4YWlMbpXsRcpQVOnau36bBkyBCZNCge8i5mKvUgJUgsne448Mlyn9qmn\nYidpHBV7kRKzcSOsXAlf+1rsJKWjFObcq9iLlJhp08LFs5s1i52kdAwaBI8+Ch98EDtJw6nYi5QY\nTbnMvtatw8qhxXxBchV7kRLy4Yfw739DeXnsJKVn6NDibuWo2IuUkOnT4cQTYe+9YycpPeXlsGpV\nOB5SjFTsRUqIFj7LnWbNYPDg4p1zr4XQRErEtm3Qpk04oapt29hpStPCheHC7WvWwB6Rh8paCE0k\noZ55Brp0UaHPpaOPDgdrn346dpL6U7EXKRE6kSo/hgwpzlaO2jgiJcAd2rcPV6bq0iV2mtL25pvQ\nuXO4ZGHMA+Fq44gk0Pz50KqVCn0+fOlLcNJJ8NBDsZPUj4q9SAnQiVT5VYxz7lXsRUqAplzm19e/\nDkuWwKuvxk6SORV7kSL3yivw1ltwwgmxkyTHnnuG9XL+9KfYSTKnYi9S5KZOhf7948/7TpohQ0Kx\nr6yMnSQz+vYQKXLq18dxzDHQsiXMnBk7SWZU7EWK2Ouvw6JFYUVGyS+z4ppzn7bYm1m5mS0zs5Vm\ndnUNz3cxs1lmttXMflyffUWkcSZNgm99C77whdhJkmnwYHj4Yfj449hJ0quz2JtZE+APQDnQFRhk\nZkdU2+wd4DLgxgbsKyIN5A4TJsCwYbGTJNeBB0KfPjBlSuwk6aUb2fcEVrn7GnffDkwGdusOuvtb\n7j4P2F7ffUWk4WbPDhfB7ts3dpJkK5Y59+mKfTtgXZX761OPZaIx+4pIGnffHUb1lvEJ85ILZ58N\nCxbA2rWxk9QtXbFvzKI1WvBGJEc++QQefBAuuih2EvnCF2DgwHD8pJA1TfP8BqB9lfvtCSP0TGS8\n7+jRoz+7XVZWRllZWYZvIZJMU6ZAr17QTn8rF4ShQ8PB2p/+NHd/aVVUVFBRUdHg/etc9dLMmgLL\ngVOBjcAcYJC7L61h29HAR+5+U3321aqXIvV36qnwve/BeefFTiIQDpZ37Qp33pm/Yyj1XfWyzpG9\nu+8wsxHADKAJcJe7LzWz4annx5tZW2Au8EWg0swuB7q6++aa9m3YpyUiO61eHebW9+8fO4nsZLbr\nQG2hHjDXevYiRWb0aHj3XRg3LnYSqWrDBujWDdavhxYtcv9+Ws9epIRVVobRo+bWF5527aBnz7B8\nRSFSsRcpIk8/DfvtBz16xE4iNSnk5RNU7EWKyM659VKYzjkH5s4NrZxCo569SJF4/33o0CGsX9+6\ndew0UpvvfhcOPRSuuSa376OevUiJuv9+OP10FfpCt3NWTqGNYVXsRYqEFj0rDr17hwPpc+bETrI7\nFXuRIrBkSegDn3FG7CSSzs517gttcTT17EWKwFVXQdOm8Otfx04imXjttTBjasOG3F1rQD17kRKz\nfXtYZEstnOJxyCGh2E+bFjvJLir2IgVu+nQ4/HDo3Dl2EqmPQptzr2IvUuB0YLY4nXsu/PvfsGlT\n7CSBir1IAXvzTXjmGTj//NhJpL5atoRvfhP+/OfYSQIVe5ECdu+9MGAA7L137CTSEIU0517FXqRA\n6YLixe/EE8NVxebPj51ExV6kYM2bB1u3wte+FjuJNNQee4QDtXfeGTuJ5tmLFKzvfz8sm/uzn8VO\nIo2xaVO4itWKFXDAAdl73frOs1exFylAW7bAwQfDggXQvn367aWwXXJJ+MVd5XLbjaaTqkRKwCOP\nwHHHqdCXih//GG69NfTvY1GxFylAOjBbWrp0CQukxVwvR20ckQKzdi0ce2xY+CxX66pI/j33XDhY\nu2IFNGnS+NdTG0ekyE2cCAMHqtCXmr59oU0bmDIlzvur2IsUEF1QvHSZhdVLb7ghzklWKvYiBeTZ\nZ6FVq9DGkdLTv3+4vOSzz+b/vVXsRQrIzgOzlnEnVopJkyZhZs4NN+T/vXWAVqRAfPhhWAd95crs\nnnwjhWXLFujYEZ56Co48suGvowO0IkXqgQfglFNU6Etd8+YwYgTceGN+31cje5EC0acPjBoFZ58d\nO4nk2jvvQKdOsHhxOLO2ITSyFylCy5bB6tXQr1/sJJIPrVvDhRfCuHH5e0+N7EUKwNVXh+l4118f\nO4nky5o1YdbV6tXwxS/Wf3+N7EWKzI4duqB4EnXoAGecAX/8Y37eT8VeJLIZM+DLX4YjjoidRPLt\nqqtg7FjYti3376ViLxKZFj1LrmOOgc6dYfLk3L+XevYiEb31VpiVsXYt7LNP7DQSw4wZcOWVsGhR\n/U6mU89epIj8+c9hqqUKfXKdcUa4fOFjj+X2fVTsRSJxh7vvVgsn6czCyD7XSyio2ItE8uKLYYmE\nsrLYSSS2Cy4Iy2TMn5+790hb7M2s3MyWmdlKM7u6lm3GpZ5faGY9qjy+xswWmdmLZjYnm8FFit2E\nCTB0aPgTXpKtWTMYOTK3o/s6D9CaWRNgOXAasAGYCwxy96VVtjkLGOHuZ5nZCcBYd++Vem41cKy7\nv1vHe+gArSTO1q3hguLz54dplyIffRQWSJs7N/ybTrYP0PYEVrn7GnffDkwGBlTbpj8wEcDdZwP7\nmlmbqpkyDSOSFFOnwle/qkIvu+y9N3znO/C73+Xm9dMV+3bAuir316cey3QbB540s3lmdkljgoqU\nEh2YlZr88Idw771hobRsa5rm+Uz7K7WN3k90941mdgDwhJktc/eZ1TcaPXr0Z7fLysoo0xErKWHr\n1sGcOfDww7GTSKE56CA491y49Vb4+c93f66iooKKiooGv3a6nn0vYLS7l6fujwIq3f23Vba5Hahw\n98mp+8uAk9z9jWqvdR2w2d1vqva4evaSKL/6VSj4t98eO4kUoqVL4eSTwwJpzZvXvl22e/bzgE5m\n1sHM9gQGAtOqbTMNuCj15r2A9939DTNrYWZ7px5vCZwBLM40mEgp0tx6SeeII+D442HixOy+bp3F\n3t13ACOAGcAS4H53X2pmw81seGqbR4FXzWwVMB74QWr3tsBMM1sAzAb+7u6PZze+SHGZORP22gt6\n9oydRArZVVfBTTfBp59m7zW1No5IHg0bFq47euWVsZNIIXOH3r3hJz8JPfya1LeNo2IvkierVoUR\n/ZIl0LZt7DRS6P7613CS1axZNS+QpoXQRArQ1q1w/vnwy1+q0EtmzjknTMF87rnsvJ5G9iJ58IMf\nhOWMH3igfsvYSrLddhtMnw7Tqk+LQW0ckYJz//1w7bXwwgtayljqZ8uWcPnCiorPX8lMbRyRArJy\nJYwYAQ8+qEIv9de8OVx6aZiZ01ga2YvkyJYtYUbFd78b2jgiDfH22+HShS+/DAceuOtxtXFECsTw\n4fD+++H6ourTS2OMGBEWSvv1r3c9pmIvUgD+8he47rqwhPEXvxg7jRS7V18N03ZXrw5FH9SzF4lu\n+XK4/PLQp1ehl2w49FA49VS4886Gv4ZG9iJZ9Mkn0KtXOKg2fHjsNFJK5s0LZ9O+8kq4spVG9iIR\nXX45HHVUOCgrkk3HHQeHHRam8jaEir1Iltx7Lzz7LIwfrwOykhtXXRWWUGhIM0TFXiQLli4NF4x+\n8MFdB9BEsq1fP6ishCeeqP++KvYijfTJJ3DeeWFaXPfusdNIKTMLK6becEMD9o19cFQHaKXYXXwx\nbNsGkyapfSO5t21bmJ2zYYMO0IrkzcSJYQna229XoZf82HPPsM59fWlkL9JAL78MZWXw9NNhBo5I\nPmnqpUgefPxxWJ/++utV6KU4aGQvUk/uMHRouH3PPWrfSBz1Hdk3zWUYkVJ0993hbMY5c1TopXho\nZC9SD4sXwymnwDPPQNeusdNIkqlnL5IjmzeHPv1NN6nQS/HRyF4kA+7w7W+HaW8TJsROI6KevUhO\n3HUXLFwIs2fHTiLSMBrZi6SxaFFYS3zmTOjSJXYakUA9e5Es+uijsO7NzTer0Etx08hepBbuMHgw\ntGwJd9wRO43I7tSzF8mS8ePDkgjPPx87iUjjqdiLVFFZCY89Fto2S5fCU09B8+axU4k0noq9CGFN\n+kmTYMyYUNxHjoSBA8NUS5FSoGIvibZxI9xyS+jJ9+4Nt90GJ52kZRCk9Gg2jiTSCy+Ek6SOOgo+\n/BD+9S+YOjUsWaxCL6VIxV4S49NPdxX0AQOgWzd45RX4/e+hU6fY6URyS20cKXmbN4eliMeMgf33\nhyuugG99C5o1i51MJH9U7KVkrVsXRu0TJoTR/MSJ0KeP2jSSTGnbOGZWbmbLzGylmV1dyzbjUs8v\nNLMe9dlXJNvmzIELLoCjj4bt22HuXHjoIejbV4VekqvOYm9mTYA/AOVAV2CQmR1RbZuzgMPdvRPw\nXeC2TPctJhUVFbEjZCSpOXfs2FXQzz8fevaE1avDfPmOHRv+usXw9SyGjKCcsaUb2fcEVrn7Gnff\nDkwGBlTbpj8wEcDdZwP7mlnbDPctGsXyDVBqOd3hgw9g+fJwwZD774exY2HUKBg2DM46C445Btq0\nCYV95EhYtSr05ffZJ385YyqGjKCcsaXr2bcD1lW5vx44IYNt2gEHZbCvJNSOHbB2Lbz+evqPZs2g\nbdtdH23ahH9PPHHXYwceCAcdFPuzEilc6Yp9piuUFUwn9Mknw8gv25Yvh/nzs/+62eQOK1aE66NW\nf7ymbTN9rLIyTFus/lHb47V97Nx+xw7YuhX+9Kfdi3jbtmE65Omn717YW7bMztdHJMnqXPXSzHoB\no929PHV/FFDp7r+tss3tQIW7T07dXwacBHRMt2/qcS15KSLSANlc9XIe0MnMOgAbgYHAoGrbTANG\nAJNTvxzed/c3zOydDPatV1gREWmYOou9u+8wsxHADKAJcJe7LzWz4annx7v7o2Z2lpmtAj4GhtW1\nby4/GRERqVn0i5eIiEjuRVsbx8zam9nTZvaymb1kZj+MlSUdM2tiZi+a2d9iZ6mNme1rZg+Z2VIz\nW5JqqRUcMxuV+j9fbGZ/MbO9YmcCMLMJZvaGmS2u8tj+ZvaEma0ws8fNbN+YGVOZasp5Q+r/faGZ\nTTGzLEw6bZyaclZ57sdmVmlm+8fIVi1LjTnN7LLU1/QlM/ttbfvnSy3/7z3NbE6qNs01s+Preo2Y\nC6FtB0a6+5FAL+DSAj7p6nJgCZnPTophLPCoux8BdAcKrmWWOn5zCXCMu3cjtPcuiJmpirsJJwBW\ndQ3whLt3Bp5K3Y+tppyPA0e6+9HACmBU3lN9Xk05MbP2wOnA2rwnqtnncprZyYTzh7q7+1HAjTGC\nVVPT1/N64Ofu3gP4Rep+raIVe3d/3d0XpG5vJhSngpspbWYHA2cBd1JAU0yrSo3k/tvdJ0A4XuLu\nH0SOVZMPCb/kW5hZU6AFsCFupMDdZwLvVXv4sxMGU/+ek9dQNagpp7s/4e6VqbuzgYPzHqyaWr6e\nAL8DfpLnOLWqJef3gV+nTgbF3d/Ke7Bqasm5Cdj5V9y+pPlZKogljlMjvh6Eb9RCczNwFVCZbsOI\nOgJvmdndZvaCmd1hZi1ih6rO3d8FbgJeI8zQet/dn4ybqk5t3P2N1O03gDYxw2ToYuDR2CFqYmYD\ngPXuvih2ljQ6AV8zs+fNrMLMjosdqBbXADeZ2WvADaT5iy56sTezVsBDwOWpEX7BMLNvAG+6+4sU\n6Kg+pSlwDHCrux9DmBVVCC2H3ZjZYcCPgA6Ev+JamdngqKEy5GEmQyG38TCznwLb3P0vsbNUlxp8\nXAtcV/XhSHHSaQrs5+69CAO9ByLnqc1dwA/d/RBgJDChro2jFnszawb8FbjX3R+JmaUWfYD+ZrYa\nuA84xcz+FDlTTdYTRkxzU/cfIhT/QnMc8G93f8fddwBTCF/jQvVGap0nzOxA4M3IeWplZkMJ7cZC\n/eV5GOGX/MLUz9PBwHwz+1LUVDVbT/jeJPUzVWlmreNGqlFPd384dfshwnpktYo5G8cIv5mWuPuY\nWDnq4u7Xunt7d+9IOJD4T3e/KHau6tz9dWCdmXVOPXQa8HLESLVZBvQys+ap///TCAe+C9U0YEjq\n9hCgEAckmFk5YQQ6wN23xs5TE3df7O5t3L1j6udpPeFAfSH+An0EOAUg9TO1p7u/EzdSjVaZ2Ump\n26cQDs7Xzt2jfAAnEvrgC4AXUx/lsfJkkPckYFrsHHXkOxqYCywkjEr2iZ2plpw/IfwiWkw46Nks\ndqZUrvsIxxG2ERbwGwbsDzyZ+iF6HNi3AHNeDKwkzG7Z+XN0awHl/M/Or2e1518F9i/EnEAzYFLq\ne3Q+UFZAOat+fx5HOM65AJgF9KjrNXRSlYhIAkQ/QCsiIrmnYi8ikgAq9iIiCaBiLyKSACr2IiIJ\noGIvIpIAKvYiIgmgYi8ikgD/D23c8yqBMTUZAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "xs = list(range(3,19))\n", + "ps = best_attr_pmf.Probs(xs)\n", + "plt.plot(xs, ps);" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Mixtures {#mixture}\n", "\n", - "[fig.dungeons3]\n", "\n", - "Let’s do one more example from *Dungeons & Dragons*.\n", + "Let’s do one more example from *Dungeons & Dragons*.\n", "Suppose I have a box of dice with the following inventory:\n", "\n", " 5 4-sided dice\n", @@ -421,21 +805,65 @@ "\n", "Let’s start with a simple version of the problem where there are only\n", "two dice, one with 6 sides and one with 8. We can make a Pmf to\n", - "represent each die:\n", - "\n", - " d6 = Die(6)\n", - " d8 = Die(8)\n", - "\n", - "Then we create a Pmf to represent the mixture:\n", - "\n", - " mix = thinkbayes.Pmf()\n", - " for die in [d6, d8]:\n", - " for outcome, prob in die.Items():\n", - " mix.Incr(outcome, prob)\n", - " mix.Normalize()\n", - "\n", + "represent each die:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "d6 = Die(6)\n", + "d8 = Die(8)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then we create a Pmf to represent the mixture:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1 0.14583333333333334\n", + "2 0.14583333333333334\n", + "3 0.14583333333333334\n", + "4 0.14583333333333334\n", + "5 0.14583333333333334\n", + "6 0.14583333333333334\n", + "7 0.06250000000000001\n", + "8 0.06250000000000001\n" + ] + } + ], + "source": [ + "mix = thinkbayes.Pmf()\n", + "for die in [d6, d8]:\n", + " for outcome, prob in die.Items():\n", + " mix.Incr(outcome, prob)\n", + "mix.Normalize()\n", + "mix.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "The first loop enumerates the dice; the second enumerates the outcomes\n", - "and their probabilities. Inside the loop, Pmf.Incr adds up\n", + "and their probabilities. Inside the loop, `Pmf.Incr` adds up\n", "the contributions from the two distributions.\n", "\n", "This code assumes that the two dice are equally likely. More generally,\n", @@ -443,43 +871,116 @@ "outcomes accordingly.\n", "\n", "First we create a Pmf that maps from each die to the probability it is\n", - "selected:\n", - "\n", - " pmf_dice = thinkbayes.Pmf()\n", - " pmf_dice.Set(Die(4), 5)\n", - " pmf_dice.Set(Die(6), 4)\n", - " pmf_dice.Set(Die(8), 3)\n", - " pmf_dice.Set(Die(12), 2)\n", - " pmf_dice.Set(Die(20), 1)\n", - " pmf_dice.Normalize()\n", - "\n", - "Next we need a more general version of the mixture algorithm:\n", - "\n", - " mix = thinkbayes.Pmf()\n", - " for die, weight in pmf_dice.Items():\n", - " for outcome, prob in die.Items():\n", - " mix.Incr(outcome, weight*prob)\n", - "\n", + "selected:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "15" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pmf_dice = thinkbayes.Pmf()\n", + "pmf_dice.Set(Die(4), 5)\n", + "pmf_dice.Set(Die(6), 4)\n", + "pmf_dice.Set(Die(8), 3)\n", + "pmf_dice.Set(Die(12), 2)\n", + "pmf_dice.Set(Die(20), 1)\n", + "pmf_dice.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we need a more general version of the mixture algorithm:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "mix = thinkbayes.Pmf()\n", + "for die, weight in pmf_dice.Items():\n", + " for outcome, prob in die.Items():\n", + " mix.Incr(outcome, weight*prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "Now each die has a weight associated with it (which makes it a weighted\n", "die, I suppose). When we add each outcome to the mixture, its\n", - "probability is multiplied by weight.\n", - "\n", - "Figure [fig.dungeons3] shows the result. As expected, values 1 through 4\n", + "probability is multiplied by weight." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFARJREFUeJzt3X+s3fV93/Hna5ewljJKp0kwzJ2MiivwlKqw1UNh0Y4i\nNBmrwplW1bEa0fFH4j9qQitUESIt3PvXNJSsFFlCbuNEsNBZE+kyV+LWbUROtqaM2MVAAnaHq1q1\nTU0YKW1hYrLl9/4435jjY997zv3hewmf50Oy+H4/38/nnLe/fPW6H3/u+Z5vqgpJ0gfb31vrAiRJ\nl55hL0kNMOwlqQGGvSQ1wLCXpAYY9pLUgLFhn2RzkiNJXk3ywEWO35Tk2STvJrl/5NiDSV5O8t0k\nv5vk769k8ZKkySwY9kmmgF3AZmAjsD3JzSPd3gTuBb4wMnY98Cng1qr6MDAFfGJFqpYkLcq4mf0m\n4GhVHauq08BeYOtwh6p6o6oOAqdHxv5t13ZFksuAK4CTK1O2JGkxxoX9OuD40P6Jrm2sqvoB8EXg\nL4HXgLeq6htLKVKStDzjwn7J36WQ5KeBXwPWA9cBVyb55aW+niRp6S4bc/wkMD20P81gdj+Jfw78\nSVW9CZDk94CPAE8Od0ril/NI0hJUVSbtO25mfxDYkGR9ksuBbcC+efqOvukR4LYkP54kwB3AK/MU\n7J8qHnrooTWv4f3yx3PhufBcLPxnsRac2VfVmSQ7gf0MPk2zp6oOJ9nRHd+d5FrgAHAVcDbJfcDG\nqnoxyRMMfmCcBZ4HfnvRFUqSlm3cMg5VNQfMjbTtHto+xflLPcP9HgYeXmaNkqRl8g7a95Fer7fW\nJbxveC7e47l4j+di6bKUtZ8VLSCpta5Bkn7UJKFW8Be0kqQPAMNekhpg2EtSAwx7SWqAYS9JDTDs\nJakBhr0kNcCwl6QGGPaS1ICx342z1j429+1F9X/mzttXZcxy3kuSVpsze0lqgGEvSQ0w7CWpAYa9\nJDVgbNgn2ZzkSJJXkzxwkeM3JXk2ybtJ7h85dnWSp5IcTvJKkttWsnhJ0mQW/DROkilgF4Pnx54E\nDiTZV1WHh7q9CdwLfPwiL/FbwNNV9YtJLgN+YmXKliQtxriZ/SbgaFUdq6rTwF5g63CHqnqjqg4C\np4fbk/wk8NGq+nLX70xV/c3KlS5JmtS4sF8HHB/aP9G1TeIG4I0kX0nyfJLfSXLFUoqUJC3PuJuq\nlvO8wMuAW4GdVXUgySPAZ4HPj3acmZk5t93r9XzOpCSN6Pf79Pv9JY8fF/Yngemh/WkGs/tJnABO\nVNWBbv8pBmF/geGwlyRdaHQiPDs7u6jx45ZxDgIbkqxPcjmwDdg3T9/zHnxbVaeA40l+pmu6A3h5\nUdVJklbEgjP7qjqTZCewH5gC9lTV4SQ7uuO7k1wLHACuAs4muQ/YWFVvM/iUzpPdD4o/B+65hH8X\nSdI8xn4RWlXNAXMjbbuHtk9x/lLPcL8XgZ9fZo2SpGXyDlpJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY\n9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgPGhn2SzUmO\nJHk1yQMXOX5TkmeTvJvk/oscn0pyKMnvr1TRkqTFWTDsk0wBu4DNwEZge5KbR7q9yeDxg1+Y52Xu\nA14BanmlSpKWatzMfhNwtKqOVdVpYC+wdbhDVb1RVQeB06ODk1wPbAG+xMgDySVJq2dc2K8Djg/t\nn+jaJvWbwG8AZxdZlyRpBY174PiSl16S/ALw/ao6lKS3UN+ZmZlz271ej15vwe6S1Jx+v0+/31/y\n+HFhfxKYHtqfZjC7n8RHgLuSbAF+DLgqyRNVdfdox+GwlyRdaHQiPDs7u6jx45ZxDgIbkqxPcjmw\nDdg3T9/z1uSr6nNVNV1VNwCfAJ65WNBLki69BWf2VXUmyU5gPzAF7Kmqw0l2dMd3J7kWOABcBZxN\nch+wsareHn25lS9fkjSJccs4VNUcMDfStnto+xTnL/Vc7DW+BXxriTVKkpbJO2glqQGGvSQ1wLCX\npAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lq\ngGEvSQ2YKOyTbE5yJMmrSR64yPGbkjyb5N0k9w+1Tyf5ZpKXk3wvyWdWsnhJ0mTGPqkqyRSwC7iD\nwQPIDyTZV1WHh7q9CdwLfHxk+Gng16vqhSRXAn+a5I9GxkqSLrFJZvabgKNVdayqTgN7ga3DHarq\njao6yCDch9tPVdUL3fbbwGHguhWpXJI0sUnCfh1wfGj/RNe2KEnWA7cAzy12rCRpecYu4wC13Dfp\nlnCeAu7rZvjnmZmZObfd6/Xo9XrLfUtJ+kDp9/v0+/0lj58k7E8C00P70wxm9xNJ8iHga8BXq+rr\nF+szHPaSpAuNToRnZ2cXNX6SZZyDwIYk65NcDmwD9s3TN+ftJAH2AK9U1SOLqkyStGLGzuyr6kyS\nncB+YArYU1WHk+zoju9Oci1wALgKOJvkPmAj8HPAJ4GXkhzqXvLBqvqDS/B3kSTNY5JlHKpqDpgb\nads9tH2K85d6fuiP8cYtSVpzBrEkNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9\nJDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNGBv2STYnOZLk1SQPXOT4TUmeTfJukvsXM1aS\ntDoWDPskU8AuYDODJ09tT3LzSLc3gXuBLyxhrCRpFYyb2W8CjlbVsao6DewFtg53qKo3quogcHqx\nYyVJq2Nc2K8Djg/tn+jaJrGcsZKkFTQu7GsZr72csZKkFTTugeMnOf9B4tMMZuiTmHjszMzMue1e\nr0ev15vwLSSpDf1+n36/v+Tx48L+ILAhyXrgNWAbsH2evlnq2OGwlyRdaHQiPDs7u6jxC4Z9VZ1J\nshPYD0wBe6rqcJId3fHdSa4FDgBXAWeT3AdsrKq3LzZ2UdVJklbEuJk9VTUHzI207R7aPsX5yzUL\njpUkrT7voJWkBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg\n2EtSAwx7SWqAYS9JDTDsJakBhr0kNWDsw0uSbAYeYfC0qS9V1X+8SJ9HgTuB/wv8u6o61LU/CHwS\nOAt8F7inqv7fypX/o+ljc99eVP9n7rx9WeMkacGZfZIpYBewGdgIbE9y80ifLcCNVbUB+DTwWNe+\nHvgUcGtVfZjBD4tPrHD9kqQJjFvG2QQcrapjVXUa2AtsHelzF/A4QFU9B1yd5Brgb4HTwBVJLgOu\nAE6uZPGSpMmMC/t1wPGh/RNd29g+VfUD4IvAXwKvAW9V1TeWV64kaSnGhX1N+Dq5oCH5aeDXgPXA\ndcCVSX55UdVJklbEuF/QngSmh/anGczcF+pzfdfWA/6kqt4ESPJ7wEeAJ0ffZGZm5tx2r9ej1+tN\nUrskNaPf79Pv95c8flzYHwQ2dL9sfQ3YBmwf6bMP2AnsTXIbg+Wa15P8GfDvk/w48C5wB/Cdi73J\ncNhLki40OhGenZ1d1PgFw76qziTZCexn8GmaPVV1OMmO7vjuqno6yZYkR4F3gHu6Yy8keYLBD4yz\nwPPAby+qOknSihj7OfuqmgPmRtp2j+zvnGfsw8DDyylQkrR83kErSQ0w7CWpAYa9JDXAsJekBhj2\nktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWrA2LBP\nsjnJkSSvJnlgnj6PdsdfTHLLUPvVSZ5KcjjJK91jCyVJq2zBsE8yBewCNgMbge1Jbh7pswW4sao2\nAJ8GHhs6/FvA01V1M/CzwOEVrF2SNKFxM/tNwNGqOlZVp4G9wNaRPncBjwNU1XPA1UmuSfKTwEer\n6svdsTNV9TcrW74kaRLjwn4dcHxo/0TXNq7P9cANwBtJvpLk+SS/k+SK5RYsSVq8cQ8crwlfJxcZ\ndxlwK7Czqg4keQT4LPD50cEzMzPntnu9Hr1eb8K3laQ29Pt9+v3+ksePC/uTwPTQ/jSDmftCfa7v\n2gKcqKoDXftTDML+AsNhL0m60OhEeHZ2dlHjxy3jHAQ2JFmf5HJgG7BvpM8+4G6A7tM2b1XV61V1\nCjie5Ge6fncALy+qOknSilhwZl9VZ5LsBPYDU8CeqjqcZEd3fHdVPZ1kS5KjwDvAPUMvcS/wZPeD\n4s9HjkmSVsm4ZRyqag6YG2nbPbK/c56xLwI/v5wCJUnL5x20ktQAw16SGmDYS1IDDHtJaoBhL0kN\nMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGjA37JJuT\nHEnyapIH5unzaHf8xSS3jBybSnIoye+vVNGSpMVZMOyTTAG7gM3ARmB7kptH+mwBbqyqDcCngcdG\nXuY+4BWgVqpoSdLijJvZbwKOVtWxqjoN7AW2jvS5C3gcoKqeA65Ocg1AkuuBLcCXgKxk4ZKkyY0L\n+3XA8aH9E13bpH1+E/gN4OwyapQkLdO4B45PuvQyOmtPkl8Avl9Vh5L0Fho8MzNzbrvX69HrLdhd\nkprT7/fp9/tLHj8u7E8C00P70wxm7gv1ub5r+7fAXd2a/o8BVyV5oqruHn2T4bDXyvrY3LcX1f+Z\nO29f0hhJl9boRHh2dnZR48ct4xwENiRZn+RyYBuwb6TPPuBugCS3AW9V1amq+lxVTVfVDcAngGcu\nFvSSpEtvwZl9VZ1JshPYD0wBe6rqcJId3fHdVfV0ki1JjgLvAPfM93IrWbgkaXLjlnGoqjlgbqRt\n98j+zjGv8S3gW0spUJK0fN5BK0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9J\nDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqwERhn2RzkiNJXk3ywDx9Hu2Ov5jklq5t\nOsk3k7yc5HtJPrOSxUuSJjM27JNMAbuAzcBGYHuSm0f6bAFurKoNwKeBx7pDp4Ffr6p/CtwG/Oro\nWEnSpTfJzH4TcLSqjlXVaWAvsHWkz13A4wBV9RxwdZJrugePv9C1vw0cBq5bseolSROZJOzXAceH\n9k90beP6XD/cIcl64BbgucUWKUlanrEPHAdqwtfKfOOSXAk8BdzXzfDPMzMzc2671+vR6/UmfEtJ\nakO/36ff7y95/CRhfxKYHtqfZjBzX6jP9V0bST4EfA34alV9/WJvMBz2kqQLjU6EZ2dnFzV+kmWc\ng8CGJOuTXA5sA/aN9NkH3A2Q5Dbgrap6PUmAPcArVfXIoiqTJK2YsTP7qjqTZCewH5gC9lTV4SQ7\nuuO7q+rpJFuSHAXeAe7pht8OfBJ4Kcmhru3BqvqDFf+bSJLmNckyDlU1B8yNtO0e2d95kXF/jDdu\nSdKaM4glqQGGvSQ1YKJlHGkSH5v79qL6P3Pn7Usas9rvJX0QOLOXpAYY9pLUAMNekhpg2EtSAwx7\nSWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgPGhn2SzUmOJHk1yQPz9Hm0O/5iklsW\nM1aSdOkt+K2XSaaAXcAdDJ4peyDJvqo6PNRnC3BjVW1I8i+Ax4DbJhmr8/31S8/zUz9761qX8b7w\nfjkX74dv5ZzvXKz2N4D+KJwLzW/czH4TcLSqjlXVaWAvsHWkz13A4wBV9RxwdZJrJxyrIW+9dGh8\np0Z4Lt7juXiP52LpxoX9OuD40P6Jrm2SPtdNMFaStArGhX1N+DpZbiGSpEtn3JOqTgLTQ/vTDGbo\nC/W5vuvzoQnGApCs3M+KpbzSUt/9UrzXsd/9yoq8z1LHvZ/On+fiPZ6L96zkuWjJuLA/CGxIsh54\nDdgGbB/psw/YCexNchvwVlW9nuTNCcZSVf5/kqRLbMGwr6ozSXYC+4EpYE9VHU6yozu+u6qeTrIl\nyVHgHeCehcZeyr+MJOniUjXpsrwk6UfVmt5B601X70lyLMlLSQ4l+c5a17Oaknw5yetJvjvU9g+T\n/FGS/53kD5NcvZY1rpZ5zsVMkhPdtXEoyea1rHG1JJlO8s0kLyf5XpLPdO3NXRsLnIuJr401m9l3\nN139GUM3XQHbW13qSfIXwD+rqh+sdS2rLclHgbeBJ6rqw13bw8D/qaqHu4nAT1XVZ9eyztUwz7l4\nCPi7qvpPa1rcKuvu17m2ql5IciXwp8DHGSwVN3VtLHAufokJr421nNl709WFmvxldVX9T+CvR5rP\n3azX/ffjq1rUGpnnXECD10ZVnaqqF7rtt4HDDO7Vae7aWOBcwITXxlqG/SQ3bLWkgG8kOZjkU2td\nzPvANVX1erf9OnDNWhbzPnBv991Te1pYthjVfarvFuA5Gr82hs7F/+qaJro21jLs/c3w+W6vqluA\nO4Ff7f45L6AGa40tXy+PATcAPwf8FfDFtS1ndXXLFl8D7quqvxs+1tq10Z2Lpxici7dZxLWxlmE/\nyQ1bzaiqv+r++wbw3xgsc7Xs9W6dkiT/GPj+GtezZqrq+9UBvkRD10aSDzEI+v9cVV/vmpu8NobO\nxVd/eC4Wc22sZdifu2EryeUMbrrat4b1rJkkVyT5B932TwD/GvjuwqM+8PYBv9Jt/wrw9QX6fqB1\ngfZD/4ZGro0Mbq3fA7xSVY8MHWru2pjvXCzm2ljTz9knuRN4hPduuvoPa1bMGkpyA4PZPAxudHuy\npXOR5L8A/wr4RwzWYD8P/HfgvwL/BDgG/FJVvbVWNa6Wi5yLh4Aeg3+mF/AXwI6hNesPrCT/Evgf\nwEu8t1TzIPAdGrs25jkXn2PwrQQTXRveVCVJDfCxhJLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakB\nhr0kNcCwl6QG/H+VZF9t35J1SQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "colors = thinkplot.Brewer.Colors()\n", + "thinkplot.Hist(mix, width=0.9, color=colors[4])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows the result. As expected, values 1 through 4\n", "are the most likely because any die can produce them. Values above 12\n", "are unlikely because there is only one die in the box that can produce\n", "them (and it does so less than half the time).\n", "\n", - "thinkbayes provides a function named\n", - "MakeMixture that encapsulates this algorithm, so we could\n", + "`thinkbayes`provides a function named\n", + "`MakeMixture` that encapsulates this algorithm, so we could\n", "have written:\n", "\n", + "```python```\n", " mix = thinkbayes.MakeMixture(pmf_dice)\n", + "```\n", "\n", "We’ll use MakeMixture again in Chapters [prediction] and\n", - " [observer].\n", - "\n", - "Discussion\n", - "----------\n", + " [observer]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#Discussion\n", "\n", "Other than the odds form of Bayes’s theorem, this chapter is not\n", "specifically Bayesian. But Bayesian analysis is all about distributions,\n", @@ -495,11 +996,11 @@ "easier with a Pmf; others are faster with a Cdf.\n", "\n", "The other goal of this chapter is to introduce operations that act on\n", - "distributions, like `Pmf.__add__`, Cdf.Max, and\n", - "thinkbayes.MakeMixture. We will use these operations later,\n", + "distributions, like `Pmf.__add__`, `Cdf.Max`, and\n", + "`thinkbayes.MakeMixture`. We will use these operations later,\n", "but I introduce them now to encourage you to think of a distribution as\n", "a fundamental unit of computation, not just a container for values and\n", - "probabilities.\n" + "probabilities." ] } ], diff --git a/notebooks/06_Decision_Analysis.ipynb b/notebooks/06_Decision_Analysis.ipynb index e17152d..46bb551 100644 --- a/notebooks/06_Decision_Analysis.ipynb +++ b/notebooks/06_Decision_Analysis.ipynb @@ -1,19 +1,275 @@ { "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "%load_ext autoreload\n", + "%autoreload 2 \n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ + "#Decision Analysis\n", "\n", - "Decision Analysis {#decisionanalysis}\n", - "=================\n", - "\n", - "The *Price is Right* problem\n", - "-----------------------------------------\n", + "## The *Price is Right* problem\n", "\n", "On November 1, 2007, contestants named Letia and Nathaniel appeared on\n", - "*The Price is Right*, an American game show. They competed\n", - "in a game called *The Showcase*, where the objective is to\n", + "*The Price is Right*, an American game show. They competed\n", + "in a game called *The Showcase*, where the objective is to\n", "guess the price of a showcase of prizes. The contestant who comes\n", "closest to the actual price of the showcase, without going over, wins\n", "the prizes.\n", @@ -46,29 +302,59 @@ "that maximizes the contestant’s expected return.\n", "\n", "This problem is inspired by an example in Cameron Davidson-Pilon’s book,\n", - "*Bayesian Methods for Hackers*. The code I wrote for this\n", + "*Bayesian Methods for Hackers*. The code I wrote for this\n", "chapter is available from ; it reads\n", "data files you can download from\n", " and\n", ". For more information see\n", - "Section [download].\n", - "\n", - "The prior\n", - "---------\n", - "\n", - "![Distribution of prices for showcases on *The Price is\n", - "Right*, 2011-12.](figs/price1.pdf)\n", - "\n", - "[fig.price1]\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The prior\n", "\n", "To choose a prior distribution of prices, we can take advantage of data\n", "from previous episodes. Fortunately, fans of the show keep detailed\n", "records. When I corresponded with Mr. Davidson-Pilon about his book, he\n", "sent me data collected by Steve Gee at . It\n", "includes the price of each showcase from the 2011 and 2012 seasons and\n", - "the bids offered by the contestants.\n", - "\n", - "Figure [fig.price1] shows the distribution of prices for these\n", + "the bids offered by the contestants." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvYAAAEWCAYAAAANTdM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVOX+B/DPmWEZQFaRfRVBRdBU3FBATUkzLdPrklaa\n5dW0q3kz83cts7r3Zl69aje1xcrKyjTUFk3RDHBLUVEExQVFFAYB2ZeZYeb8/hg8MAGCBgzL5/16\n9YrznOec+c4jy3ee8yyCKIoiiIiIiIioVZMZOwAiIiIiIvrzmNgTEREREbUBTOyJiIiIiNoAJvZE\nRERERG0AE3siIiIiojaAiT0RERERURvAxJ6IiIiIqA1ocGK/YcMG+Pr6wsLCAiEhITh8+PA96ycm\nJiIiIgKWlpbw8PDA22+/bXD+t99+g0wmq/HfpUuXHuydEBERERG1YyYNqbRt2zYsXLgQGzduxJAh\nQ/DBBx9g9OjRSE5OhqenZ436hYWFGDlyJIYOHYr4+HhcuHABM2fOhJWVFRYtWmRQNzk5GQ4ODtKx\no6Pjn3xLRERERETtj9CQnWcHDBiAhx56CB9++KFUFhAQgIkTJ+Jf//pXjfobN27E0qVLkZWVBXNz\ncwDAP//5T2zcuBE3b94EoO+xHz58OLKzs9GxY8fGej9ERERERO1SvUNx1Go1Tp8+jcjISIPyyMhI\nHD16tNZrjh07hrCwMCmpv1s/IyMDaWlpBnVDQkLg5uaGESNG4LfffnuAt0BERERERPUm9jk5OdBq\ntXB2djYod3JyglKprPUapVJZo/7d47vXuLm5YdOmTYiKikJUVBS6du2Khx9+uN6x+0REREREVFOD\nxtjfL0EQ6q0TEBCAgIAA6XjgwIG4fv06Vq1ahSFDhkjlBQUFTREiEREREVGzsLW1bZbXqbfH3tHR\nEXK5HFlZWQblWVlZcHV1rfUaFxeXGr35d693cXGp87X69++Py5cv1xs0EREREREZqjexNzMzQ9++\nfbF//36D8ujoaISGhtZ6zaBBgxAXFweVSmVQ393dHd7e3nW+VkJCAtzc3BoaOxERERERVWrQUJxF\nixbh6aefRv/+/REaGopNmzZBqVRizpw5AIClS5fi5MmTOHDgAADgqaeewooVKzBjxgwsW7YMKSkp\nWLlyJd58803pnmvXroWvry8CAwOhVqvx1VdfYffu3YiKiqozjuZ6jNEexMfHA9BPXqbGw3ZtGmzX\npsF2bRps16bBdm06bNumYYzh5A1K7CdNmoTc3Fy88847yMzMRHBwMPbs2SOtYa9UKpGamirVt7Gx\nQXR0NObNm4eQkBA4ODjglVdewcsvvyzV0Wg0WLx4MW7evAkLCwsEBQVhz549GDVqVCO/RSIiIiKi\ntq/Bk2fnzp2LuXPn1nrus88+q1EWFBSEmJiYOu+3ePFiLF68uKEvT0RERERE91DvGHsiIiIiImr5\nmNgTEREREbUBTOyJiIiIiNqAJtmgioiIiMgYdDod1Gq1scNoVe4uRV5eXm7kSFoPMzMzyGQtr3+c\niT0RERG1CaIoQqVSQaFQQBAEY4fTaigUCmOH0KqIoojy8vIW+X3W8j5qEBERET0AtVoNMzOzFpds\nUdsiCALMzMxa5JMhJvZERETUJoiiCLlcbuwwqB2Qy+UQRdHYYdTAxJ6IiIiIqA1gYk9ERERE1AYw\nsSciIiIiagOY2BMRERG1AUOHDkX37t2NHQYZERN7IiIiojaCKwLdP6VSiddeew0PP/wwbG1tIZPJ\nsG3bNmOH9UCY2BMRERFRu3Xx4kW89957SE9PR+/evQG03g9I3KCKiIiIiNqtkJAQ5Obmwt7eHjEx\nMRg2bJixQ3pg7LEnIiIiauGKi4vxyiuvwNfXFwqFAk5OThg2bBji4uJq1E1OTsbw4cNhZWUFDw8P\nrFq1qkadnJwczJ49Gy4uLrCwsEBwcDA++eQTgzp9+/bFuHHjDMoGDRoEmUyGo0ePSmUHDx6ETCZD\nTEyMVFZQUIDFixejc+fOUCgU8PDwwLRp05CRkQEA0Gg0WL58Ofr16wcHBwdYWlpiwIAB2L17d41Y\nDx48iPDwcDg4OMDKygpdunTBSy+9ZFBHpVJhxYoV8Pf3l15v0aJFKCsrq7dtO3ToAHt7ewBokWvT\n3w/22BMRERG1cHPnzsX27dsxf/589OjRA3fu3MGJEydw7tw5hIWFSfXy8/Px6KOP4sknn8TkyZOx\nfft2LFmyBMHBwRg1ahQAoLy8HMOGDUNKSgrmz58PPz8/7Ny5E7Nnz0Zubi6WLFkCAAgPD8cXX3wh\n3bu0tBSnTp2CXC5HbGwsQkNDAQBxcXEwNzfHwIEDAQAlJSWIiIhAUlISZs6ciZCQEOTk5GDv3r24\nevUq3NzcUFBQgI8++ghTp07FrFmzUFZWhq+//hrjx4/Hnj17pFiTk5MxZswY9OrVCytWrIClpSWu\nXLmC/fv3S3GJoojx48cjNjYWs2fPRmBgIJKTk7FhwwYkJSVh3759TfuP05KILVx+fr70HzWekydP\niidPnjR2GG0O21VPp9OJd/LKRLWmolHux3ZtGmzXpsF2bRoNadeysrJmiqb52dnZiS+99NI960RE\nRIiCIIhffvmlVKZWq0VXV1dx4sSJUtm6detEQRDEL774QirTarXiiBEjRIVCIebm5oqiKIo7duwQ\nBUEQz549K4qiKB48eFCUyWTiX/7yF3H06NHStcOHDxeHDBkiHS9fvlwUBEHcsWNHnbFqtVpRrVYb\nlKnVajEoKEgcMWKEVLZ27VpREAQpptps3bpVlMlkYmxsbI1yQRDE/fv313ntHx06dEgUBEHctm1b\nvXXr+34zRg7LHnsiahRl5RokJt3GqXOZOH1Widw7pTA1kcPP1x7+fg4I8OuIrl06oqODRaudlERE\nbcu3UUn4bldSk9x70hM9MOXJHo12Pzs7Oxw/fhwZGRlwc3Ors56lpSWmT58uHZuamqJ///5ITU2V\nyn7++Wc4OTkZ1JPJZFi4cCEOHjyIAwcOYNKkSRgyZAgAfY98z549ERsbi+DgYDz++OOYN28eRFFE\nRUUFjh8/jgULFkj32rFjB4KCgjBhwoQ645TJZJDJ9CPC1Wo1iouLodPpEBYWhm+//dbgfQPAzp07\nMXPmTOma6r777jsEBAQgMDAQOTk5Unl4eDgEQcChQ4cwcuTIOmNpS5jYE9EDu51Tgt/jb+HU2Uwk\np2SjokJncF5TocXFyzm4eLnqF629nQV6BTlj+l+C4WBv0dwhExG1SqtWrcKzzz4LLy8v9O7dG6NG\njcLTTz+NgIAAg3ru7u41rrWzs8O5c+ek47S0NHTp0qVGJ0u3bt2k8wDg7OyMgIAAxMbGYt68eYiL\ni0N4eDjCw8NRWFiIhIQEqNVqlJWVITw8XLrP1atXMX78+Hrf0yeffIL//ve/uHjxosHY9urJ++TJ\nk7F582a88MILeO211zB8+HA88cQTmDRpEuRyOQDg0qVLSElJQadOnWq8hiAIyM7OrjeWtoKJPRHd\nN7Vaix0/XMDOny9Cq9XVWsfMVA61RlujPC+/DL8dvo5zSVn4v5eHoLOPfVOHS0TU6k2cOBFhYWHY\nvXs39u/fj/Xr1+O9997D559/jqlTp0r17ia7fyQ+4KTQ8PBw/Pjjj1LP/Ny5c+Hp6QkvLy/ExMSg\noqICMpkMgwcPlq5pyFPZrVu3Yvbs2Rg3bhyWLl0KJycnmJiY4NNPP8XXX38t1VMoFIiJiUFsbCz2\n7NmDffv2Ydq0aVizZg3i4uKgUCig0+nQo0cPrFu3rtbXutcTjraGiT0R3Zez55XY9PlpZN0urnHO\n29MOfXq5oG8vV3Tt0hFFxWpcvnoHl67m4tLVXFxOvYPy8goAwJ28Mvzjn4ewaO5A9OvTfn7pElHL\nMeXJxh0u09ScnZ0xe/ZszJ49GwUFBRg4cCCWL19ukNg3hLe3NxISEqDT6Qx6xy9evAgA8PHxkcqG\nDBmCTz75BFu3bjXomQ8PD0dsbCx0Oh169eoFa2tr6Ro/Pz8kJibeM4bt27fDz88Pu3btMijfvHlz\njQ8GgiAgIiICERERWLlyJTZt2oQXX3wRO3fuxNSpU+Hn54fTp09j+PDh99UObRGXuySiBskvKMfa\njb9jxXuxBkm9f+eOmDMzBB/99zH895+ReHpSTwR27QS5XAY7WwX69XHDtL8EY8VrQ/Hlxiew9OUh\nsLQwBQCoVBX499oj+PGXS61+iTEioqai0+lQUFBgUGZrawsfH58a5XWpniyPHTsW2dnZBj3jOp0O\n69atg0KhwIgRI6Tyu4n8u+++i4CAADg5OQEAwsLCEBcXhyNHjhgMwwH0TxeSkpKwY8eOOuMxMTGB\nKIoGv/tTU1Oxc+dOg3p37typce3dTaTy8/MBAFOmTEFWVhY2btxYo65KpUJxcc2OqLaKPfZEdE86\nnYiDsdfw5bZzKC5RS+VWlmaYPikYI4d2hkzWsMmwcrkM/Xq74V+vD8e//3sYWdklAER89nUCMpRF\neP7p3pDL2d9ARFRdYWEh3N3dMXHiRPTs2RM2NjY4cuQI9u3bV2M997o6SaqXv/DCC/joo48wa9Ys\nnDlzBr6+vti1axd+/fVXvPvuu9Ka7oC+997NzQ0pKSl4/vnnpfLw8HDk5uYCgMFymwCwePFifP/9\n95g6dSr279+PPn36ID8/H7/88gveeusthIeHY9y4cYiKisK4ceMwduxY3Lp1Cxs3bkS3bt2QkJAg\n3eutt95CTEwMxowZA29vb+Tl5WHTpk3o0KEDHnvsMQDA9OnTsWPHDsybNw8xMTEYPHgwRFFESkoK\ntm/fjh07dtT48PFH77zzDgDg2rVrAICoqChcunQJALBs2bJ7XtuSMLEnojqVlmmwdtPviD+TYVAe\nNsgLM6Y+BHs7xQPd18vDFv9+42GsXHcEKVf0fxj2/XoVytsleGX+QFhZmv3p2ImI2gorKyvMnz8f\n0dHR+OGHH6DRaNC5c2esXr3aYDUaQRBqHd/+x3Jzc3McOnQIS5cuxdatW5Gfnw9/f398/PHHmDVr\nVo3rw8PDsW3bNoPkuGvXrujUqRNycnJqJPaWlpaIjY3Fm2++iaioKGzZsgXOzs6IiIiQJvs+88wz\nuH37NjZu3IgDBw7A398fa9euxeXLl3H27FnpXk888QTS09OxZcsWZGdno2PHjggNDcUbb7wBT09P\n6f1FRUVh7dq12LJlC3bv3g0LCwv4+flh3rx5CA4OrreN33jjDQiCAFEUIQgCtm/fju+++w6CILSq\nxF4QW/jz7+qPmGxtbY0YSdsSHx8PQL+NMjWettSuWdkl+Pd/D+PGzaqfQWenDvjrs33wULBLo7yG\nWq3F/z45icPHb0hlXTo74J/LhsHUpGoCWFtq15aE7do02K5NoyHtWl5eDoXiwTociO5Xfd9vxshh\n+cybiGpITsnGkjcPGCT1jz0SgHX/eqTRknoAMDOT4+W5AzDpiUCp7ErqHXz/w4VGew0iIqL2gkNx\niMjAobjr2PhZvLQmvVwuw9yZfTE83LdJXk8QBEx5Mgjm5ib4cpt+neUdP1xAvz7u8ONSmERERA3G\nHnsiAgBotTp88e1ZvP/xCSmpt7E2x1tLhzZZUl/duFEB6BbgCEA/YXf9hydqXQefiIiIasfEnoig\nVmvx3vqj2LUnRSrz8rDFyjdHoHtlst3U5HIZXnq+H8zM9A8S028V4Ludyc3y2kRERG0BE3uidk5T\nocWq94/iZLWVb0Iql6R07mTVrLG4uljjmclVqxfs/PkiLlWumkNERET3xsSeqB2rqNBh9f+O49TZ\nTKls3KiuWPK3UGkTqeY26uEu6NGtEwD9usvvf3wSGo3OKLEQERG1JkzsidoprVaH/278HSdO35LK\nJoztjmen9jTqJlEymYB5z/eDQqEfknMrsxAH4m4bLR4iIqLWgok9UTuk1erw/kcncexkulT2+Oiu\neGpiUK2bmzQ3F6cOeHZKL+n4yIkcpN0sMWJERERELR8Te6J2RqcTseHTeMQeS5PKxoz0xzNTeraI\npP6uyGGd0bOHMwBAFIHvf7oFlarCyFERERG1XEzsidoRURTx4ZZTOBR3XSqLHOaH56Y/1KKSekC/\nvv385/tJY/1z89TY+XNKPVcRERG1X0zsidqRr3ecR/ShVOn44XBfzH62T4tL6u9y7GiJZ6oNyflp\n3yUUl6iNGBEREVHLxcSeqJ2IT8jA9z9ekI4jBntjzsy+kMlaZlJ/18PhPnB0MAMAlJZp8NO+S0aO\niIiIqGVqUGK/YcMG+Pr6wsLCAiEhITh8+PA96ycmJiIiIgKWlpbw8PDA22+/XWfdw4cPw8TEBMHB\nwXXWIaI/JzunBOs/PCEdPxTsgvnP9zPq6jcNJZfLMGywk3T8077LKCpWGTEiIqKWaejQoejevbux\nwyAjqvev+rZt27Bw4UIsW7YMCQkJCA0NxejRo5Genl5r/cLCQowcORKurq6Ij4/HunXrsGrVKqxZ\ns6ZG3by8PDzzzDMYMWJEix0KQNTaaSq0WP3BcWkIi4O9BRbOGdAqkvq7egbaolPHql77H3+5bOSI\niIhaJuZT9+/gwYN47rnnEBAQACsrK/j5+eGFF16AUqk0dmj3rd6/7GvWrMHMmTMxa9YsdO3aFevX\nr4erqys2btxYa/2tW7eivLwcW7ZsQWBgICZMmIAlS5bUmtjPmjULM2fOxKBBgyCK4p9/N0RUw1ff\nJeLSVf3urTKZgL/PGwQba3MjR3V/ZDIBw4dU67Xff4m99kRE1CiWLFmC2NhYTJgwAe+//z6mTJmC\n7777Dr179251yb3JvU6q1WqcPn0ar776qkF5ZGQkjh49Wus1x44dQ1hYGMzNzQ3qv/7660hLS4O3\ntzcA/fCe7OxsLFu2DCtWrPiz74OIanE8/iZ+/KVqTPr0ST3RPcCxUe6tLFbhSHo+0gvLUFqhQ6la\ni9IKLUo1WpRpdFBpdXC2MoO/gxUCOloiwMEKnjYKyB9wTH9QN1ucvVCB9FuFKC+vwO49lzB9Eofw\nERHRn7N27VoMGTLEoGzUqFGIiIjA+vXr8a9//ctIkd2/e/bY5+TkQKvVwtnZ2aDcycmpzk8wSqWy\nRv27x3evSUxMxFtvvYWvvvqKj4yImogyqxj/+/ikdBzS2w2Pjw74U/e8XaJG1MUsLIq+iFk/ncen\nZ28i+loujqTn4UxWIVJyS5BeWI6cMjWK1BW4kleKvVezse5EGub9koy/fJ+AVw+k4POzt6C8zx53\nmUzApCd6SMd7DlxGYRF77YmofSguLsYrr7wCX19fKBQKODk5YdiwYYiLi6tRNzk5GcOHD4eVlRU8\nPDywatWqGnVycnIwe/ZsuLi4wMLCAsHBwfjkk08M6vTt2xfjxo0zKBs0aBBkMplBB+/Bgwchk8kQ\nExMjlRUUFGDx4sXo3LkzFAoFPDw8MG3aNGRkZAAANBoNli9fjn79+sHBwQGWlpYYMGAAdu/eXSPW\ngwcPIjw8HA4ODrCyskKXLl3w0ksvGdRRqVRYsWIF/P39pddbtGgRysrK6m3bPyb1ABAWFgYHBwdc\nuHChlitarnv22D+I+hJ1lUqFyZMn4z//+Y/Ue99Q8fHxfyY0qgXbtGkYu101Gh0++ioV2TnlAAB7\nW1MMCTHBqVOn7vte5VoRJ+5ocTZfixuluj8VVymAk4XFOJl2G1+eSkVPWzmGOpnA3aJh4/1NBSU6\nWGqRla1CSQnw/qZoPDLM5U/FRMb/fm2r2K5N417t6u3tDYVC0YzRNJ+5c+di+/btmD9/Pnr06IE7\nd+7gxIkTOHfuHMLCwqR6+fn5ePTRR/Hkk09i8uTJ2L59O5YsWYLg4GCMGjUKAFBeXo5hw4YhJSUF\n8+fPh5+fH3bu3InZs2cjNzcXS5YsAQCEh4fjiy++kO5dWlqKU6dOQS6XIzY2FqGhoQCAuLg4mJub\nY+DAgQCAkpISREREICkpCTNnzkRISAhycnKwd+9eXL16FW5ubigoKMBHH32EqVOnYtasWSgrK8PX\nX3+N8ePHY8+ePVKsycnJGDNmDHr16oUVK1bA0tISV65cwf79+6W4RFHE+PHjERsbi9mzZyMwMBDJ\nycnYsGEDkpKSsG/fvvtu7+LiYhQVFcHRse6n3EVFRTh//nyd5/39/e/7df+seyb2jo6OkMvlyMrK\nMijPysqCq6trrde4uLjU6M2/e72LiwsyMzNx8eJFzJw5EzNnzgQA6HQ6iKIIU1NT7N27FyNGjHjg\nN0REwN5flchQ6pN6uQyY8oQnLC3u/3P8xUItdtzUoEBTcw6MTAACrGXobi2HlYkAhRwwl+n/r5AJ\nkAtAZrkO6aU6pJfpcLNUNLiPTgQS8rVIyNfC31qGoZ1M4N9Bds/Ogbtj7b/ZqZ+8f+xULgb3d0QH\nq0bvoyCidmDr+Qx8fT6zSe79VJArpgW5Ndr9fvrpJ8yePRv/+c9/7llPqVTiiy++wPTp0wEAzz33\nHLy9vbF582YpWf7oo4+QlJSELVu24Omnnwag/+DwyCOP4M0338QLL7wABwcHDBkyBOvWrcO5c+fQ\ns2dPHD9+HFqtFhMmTEBsbCxee+01APrEPiQkRBqGvWrVKpw7dw7bt2/HhAkTpNj+7//+T/rawcEB\nN27cgKmpqVQ2f/589OnTB6tXr5ZijY6Ohlqtxt69e+Hg4CDV/fe//y19/c0332Dfvn347bffDD7k\nhISEYPr06YiOjsbIkSPvo7X1w3M0Gg2mTJlyX9cZ2z3/GpqZmaFv377Yv3+/wT9MdHQ0/vKXv9R6\nzaBBg7BkyRKoVCrpHzg6Ohru7u7w9vZGRUVFjU83H3zwAaKjo7Fr16579uKHhIQ0+I3Rvd3t8WCb\nNq6W0K5JF7NxPiUVVlaWAIBZ03tjTOT99RoUqirw8Zmb+PV2LmBmAkv9gjQwEQT0crHBEE87DHK3\ng7X5/SXUd8o0SM4pxp4r2TibVSSV39IBW7MAP7Ulng52Qz83W4Prqrdrnz4izl6IRlp6PgAgLcMC\nz07tBbp/LeH7tS1iuzaNhrRreXl5c4XT7Ozs7HD8+HFkZGTAza3uDwyWlpZSUg8Apqam6N+/P1JT\nqzYn/Pnnn+Hk5GRQTyaTYeHChTh48CAOHDiASZMmSUNU4uLi0LNnT8TGxiI4OBiPP/445s2bB1EU\nUVFRgePHj2PBggXSvXbs2IGgoCCD3PGPZDIZZDL901q1Wo3i4mLodDqEhYXh22+/NXjfALBz507M\nnDlTuqa67777DgEBAQgMDEROTo5UHh4eDkEQcOjQoftK7GNjY7FixQpMmjQJDz/8cJ31rK2t7/n9\nWFBQ0ODXbCz1Pv9etGgRPv/8c2zevBkXLlzAggULoFQqMWfOHADA0qVLDXrYn3rqKVhaWmLGjBlI\nSkpCVFQUVq5ciUWLFgEATExMEBgYaPBfp06dYG5ujsDAQFhZWTXRWyVq+3Q6EZ99nSAd9+/jjkdH\ndrmvexxJz8OLe5Px6/VcqczG3AQv9vXCl0/0xFsRXRDZ2fG+k3oAcLAwxRBPe/xrWADWRnZDuJc9\nZNV66K/mleLN2Ct4/2QayjTaWu8hkwmYPD5QOt578Cry8tvuH3MiIkDfC56UlAQvLy/069cPr7/+\nOi5dqrlhn7u7e40yOzs75OXlScdpaWno0qVLjSek3bp1k84D+jmSAQEBiI2NBaBP8MPDwxEeHo7C\nwkIkJCTg9OnTKCsrQ3h4uHSfq1evIigoqN739Mknn6BHjx6wsLCAo6MjnJycsGnTJhQWFkp1Jk+e\njCFDhuCFF16As7MzJk+ejG+++QZabdXfiEuXLiElJQWdOnWCk5OT9J+XlxcAIDs7u95Y7rp48SLG\njx+Pnj17YvPmzQ2+rqWo9y/zpEmTkJubi3feeQeZmZkIDg7Gnj174OnpCUD/yKf6p0AbGxtER0dj\n3rx5CAkJgYODA1555RW8/PLLdb6GIAicREvUCGKPpSH1uv6Xt6mJHM9Nf6jBP1t5ZRpsOp2Ow+l5\nBuURXg6Y3ccDdgrTOq58MP4OVlgS2hnPFquwK+U29qfmQKXVj+H/5WoOzmYV4e8DfdDdsUONawf0\ndYevlx2u3ciHWl2BXXsuYuZTDzVqfETU9k0LcmvU4TJNaeLEiQgLC8Pu3buxf/9+rF+/Hu+99x4+\n//xzTJ06Vaonl8trvf5BlxUPDw/Hjz/+KPXMz507F56envDy8kJMTAwqKiogk8kwePBg6ZqG/N3Z\nunUrZs+ejXHjxmHp0qVwcnKCiYkJPv30U3z99ddSPYVCgZiYGMTGxmLPnj3Yt28fpk2bhjVr1iAu\nLg4KhQI6nQ49evTAunXran2tez3hqC49PR2RkZGwt7fHnj17WmVnc4O63ObOnYu5c+fWeu6zzz6r\nURYUFGQwM7o+y5cvx/Llyxtcn4hqUqkqsHV71TC3saMC4OTYsF9K1/PLsOy3y8gr10hlDgpTzO/n\nhQHudo0ea3UuHcwxp68npvRwwaZT6Yir/GCRWazCqwcvYVJ3FwSIIuTV/lAIgoDJ43vg3XVHAAD7\nfk3FxHHdYd2hda3PT0R0P5ydnTF79mzMnj0bBQUFGDhwIJYvX26Q2DeEt7c3EhISoNPpDIa2XLx4\nEQDg4+MjlQ0ZMgSffPIJtm7datAzHx4ejtjYWOh0OvTq1QvW1tbSNX5+fkhMTLxnDNu3b4efnx92\n7dplUL558+YaHwwEQUBERAQiIiKwcuVKbNq0CS+++CJ27tyJqVOnws/PD6dPn8bw4cPvqx2qy83N\nRWRkJDQaDQ4dOlRjhcfWovVsPUlE9/TjvkvIvVMKALC1UeDJsd0adN2NgjL845BhUh/Z2REbHw1s\n8qS+OjuFKZaE+uLvA31gZarvcdKJIr5NzsT/rqhwu9xwRZ5+fdzg46WPT62uwP5fU2vck4ioLdDp\ndDXGa9va2sLHx6fB47irJ8tjx45Fdna2Qc+4TqfDunXroFAoDIZY303k3333XQQEBMDJSb9ZYFhY\nGOLi4nDkyBGDYTiA/ulCUlISduzYUWc8JiYmEEXR4ElCamoqdu7caVDvzp07Na7t3bs3AP0KQAAw\nZcoUZGVyRxmWAAAgAElEQVRl1bp5qkqlQnFxcZ1xAPpVfB599FFkZmZiz5498PPzu2f9loxLSRC1\nAXn55Yj66aJ0POXJHrC0qH/ozM3Ccvzj0GXkq/RJvaWJHEsG+yLE1baeK5uGIAgY7tMRQZ2sseb3\n60i8rZ9ge7NUxNrLKtj75GOQh51Ud+yoALz/0QkAwJ7oKxj3aABMTWp/DE1E1FoVFhbC3d0dEydO\nRM+ePWFjY4MjR45g3759NdZzr2vITfXyF154AR999BFmzZqFM2fOwNfXF7t27cKvv/6Kd999F/b2\n9lJdHx8fuLm5ISUlBc8//7xUHh4ejtxc/Vys6ivRAMDixYvx/fffY+rUqdi/fz/69OmD/Px8/PLL\nL3jrrbcQHh6OcePGISoqCuPGjcPYsWNx69YtbNy4Ed26dUNCQtVcsbfeegsxMTEYM2YMvL29kZeX\nh02bNqFDhw547LHHAADTp0/Hjh07MG/ePMTExGDw4MEQRREpKSnYvn07duzYUePDR3XTpk3DyZMn\n8dxzzyEpKQlJSUnSOWtrazz++ON1XtvSMLEnagO+3Xke5eUVAABPdxuMiPCt95rMIhX+cegy7lT2\n1CtMZFgR0QWBnWqOaW9uTlZm+Ncwf+y8eBtfJt4CAGh0wL+OpGJ+iBce8dOvKzxkoCe+2paIvIIy\n5BWU4fDxdAwb4mPEyImIGp+VlRXmz5+P6Oho/PDDD9BoNOjcuTNWr15tsBpNXXMW/1hubm6OQ4cO\nYenSpdi6dSvy8/Ph7++Pjz/+GLNmzapxfXh4OLZt22aQHHft2hWdOnVCTk5OjcTe0tISsbGxePPN\nNxEVFYUtW7bA2dkZERERCAjQb5T4zDPP4Pbt29i4cSMOHDgAf39/rF27FpcvX8bZs2elez3xxBNI\nT0/Hli1bkJ2djY4dOyI0NBRvvPGGNN9TEARERUVh7dq12LJlC3bv3g0LCwv4+flh3rx5CA6+9y7l\nZ8+ehSAI+PTTT/Hpp58anPPx8WlVib0gPuhsimZS/RGTra1xehHbIi7H1jSM0a43bhbg5X/sl3pj\nlv09DH161b7PxF3KYhVe+/USskvVAABzuT6pD3ayvud1xnAtvxSLfzqNXJUIy8olPJ8JdsOkQBcI\ngoAdPyTj6x36uQU+XnZY/fZITsZvIP4eaBps16bR0OUu2+oGVdTy1Pf9ZowclmPsiVq5z785KyX1\nvYJc0LvnvXdizSpRYWm1pN5MLsPycL8WmdQDgK+dJeb5mcPdoipZ/yIxAx+evgmdKOKR4X4wM9M/\nfLx+Ix+JybeNFSoREZFRMbEnasVOn81EQqJ+p2dBEDBjas979lZnl6rxj0OXcftuUi+T4Y0wP/Ry\ntmmWeB+UtamAOX7meKhanD9evo1Vx67B3MIUw8N8qsp/qbmuMxERUXvAxJ6oldJqddjy7Tnp+OFw\nX3h71r2KTYVOxL+PpCKzWAUAMJUJ+EdYZ/R2adlJ/V0KuYDl4X4I86ya1BV7Iw9vxV7F8OG+APQf\naE6dzcTNjMI67kJERNR2MbEnaqV+jbuO9Fv68Xvm5iaYOuHeu/xtS85ESm4JAMBEELB0cGejrX7z\noMzkMrwa6oux/k5S2ZmsQmy8mIWHelcNQfpxH3vtiYio/WFiT9QKiaKIH/ZWJa9PPtYN9nZ1T+C5\nkFOMb5OU0vH0YLdmXaO+MckEAX/t44Fngqt2EryYWwKljy10laOQYg6noaCw3EgREhERGQcTe6JW\nKDH5Nm5l6oebKBQmGBPpX2fdUo0Wq49fh65ygm1QJ2s82a117qh3lyAImNzDFXP6eEplaSoNSro7\nQgSg1mixjxtWERFRO8PEnqgV+uXgVenroYN97rkZ1Uen06Vx9Vamciwa6A25rG0sBzk2wAlPV+u5\nVztaINPVCiKAvQeuQK3WGi84IiKiZsbEnqiVycktxYnTt6TjUQ/XvfX1kfQ8RF/LlY7n9vWEs5V5\nk8bX3CYHuuCJAP2YeysrMxQ7WuC2kwUKCssRd+yGkaMjoubWwrfnoTaipX6fMbEnamWif0uFTqf/\nhdKjWyd4edQ+ATanVI33T1YlthFeDhjm07FZYmxOgiBgVm8PPOzTEYIA2NqYI7ejBXI6KvDDL5da\n7C9fImp8ZmZmKC8v5889NSlRFFFeXg4zMzNjh1KDibEDIKKG01RoEf1b1djx0SO61FpPJ4pY+3sa\nitQVAIBOlmZ4McSz1rptgUwQsKC/N0o0WhzV5eFOXjluO1niXGYJEhKz6t20i4jaBplMBnNzc6hU\nKmOH0qoUFRUBAKytW+ZGhS2Rubk5ZLKW1z/OxJ6oFfk9/hbyC/SrvdjbWaB/H/da6/14KRtnsvST\nawUIWDTQBx3M2vaPu1wm4NVBvnhTo8WdvDIUFKqQ6WqFj/anYCMTe6J2QyaTQaGoe5Uwqun8+fMA\ngJCQECNHQn9Wy/uoQUR12nugatLsyKGdYWJS80dYWazC52erxuBP6O6Mnk7toxfG3ESG18P80Lva\nRl0xKjWOXLxtxKiIiIiaBxN7olYiLT0fFy5lAwDkchkih3eutd6XiRlQ63QAAD97S0wPcm22GFsC\nS1M53o3sBmcr/dhHnUzAsgMpyCvTGDkyIiKipsXEnqiVqN5bP6CvOxzsLGrUuXynBL+l3ZGO5/Tx\nhKm8/f2Y2ylM8UZ4F8gqJxkri1VYfugy1FqdkSMjIiJqOu3vLz5RK1RSqkbM0TTpuLYlLkVRxKcJ\nVUNwQj3sENipQ7PE1xIN7+WKgaL+V5woAifT7uD9kze4WgYREbVZTOyJWoHfDqdBpdKvcOPpbose\n3TrVqBOfWYhzt/UrG8gFAc/2rH1ibXshCAJmjQyAc1YpAKCwUIWD13Lx/cUsI0dGRETUNJjYE7Vw\noiga7DQ7eoQfBMFw51itTsRn1SbMPuLnCA8brgoxMMQdXQU57PJV0OpEFBWp8PnZDPx+K9/YoRER\nETU6JvZELVxi8m3cytQvXWmhMEV4qHeNOgev5yKtoAwAoDCR4ake7WvCbF3kchnGPuIPF2UJLEs1\nyC9UQRRFrDp2HdfyS40dHhERUaNiYk/UwlXvrR86xBuWFqYG58srdPgqMUM6ntDNBfZ/qNOePRzh\nCytzU3jcLAZKNSgt06CsQot34lJRVDm8iYiIqC1gYk/UguUXlOPE6WpDbIbXnDS7OyULuZVLOToo\nTDG+q1OzxdcaWFqYYsRQX5hoRXimF6GkQL8jpbJEhTW/X4eOk2mJiKiNYGJP1IL9fuoWdJVLNnYL\ncISXh63B+fxyDXZcqJoMOi3YDRam8maNsTUYE+kPQRCgUGlhc+UO1CotAOBERgG2JyuNHB0REVHj\nYGJP1IIdO3lT+npwf88a579NUqK0Qp+ketooMNK3Y7PF1po4OVphUD8PAIBNkQZuxVWbVX2ZmIkz\nykJjhUZERNRomNgTtVBFxSqcv3BbOh4YYrh8ZUZROfZeyZaOZ/Zyh1xmuFoOVRk3KkD6uiQ+E/42\n+g2+RIh479g1ZJeqjRUaERFRo2BiT9RCnTiVIQ3D6dqlIzo6WBqc/yZJiYrK8eFBnazR3822xj2o\nSkCXjugW4AgA0Gl1CCzUwEGhn2RcqKrAv4+kcmdaIiJq1ZjYE7VQ1Yfh3B1Gcld2qRqxaXek4xm9\n3GqsbU81jX2kqtc+7rdr+FsfT8gr2y0ltwSbE27WdSkREVGLx8SeqAUqLlHjbFLVpNiBf0jsf7x0\n26C3vrtjh2aNr7Xq38cNrs7WAICSUjXSEjIxs1fVEKefLmfj0PVcY4VHRET0pzCxJ2qB4hMyoK0c\nFuLn6wAnRyvpXKlGi1+u5kjHT3bj8pYNJZfLMGFsN+n4h72XEOllj8Ge9lLZ+ydvSJt9ERERtSZM\n7IlaoGMn6h6Gs+9qDko0+pVwPKwV6Mex9fclPNQbzp30H5SKS9TYfygVC/t7w8NaAQBQaXV47+g1\nqCo43p6IiFoXJvZELUxpmQYJiVXDcAZVWw2nQidi96WqlXKe6OoEGcfW3xcTExnGP9ZdOt69JwUy\nrYilgzvDTK7/lXi9oAwfn+F4eyIial2Y2BO1MKcSMqGpXJve18sOri7W0rnDN/KkZRltzU0w3Ifr\n1j+IYWHecOyoX2WosEiF6N9S4WNngdm9q56O7L2ajSPpecYKkYiI6L4xsSdqYY7HVxuG078q0RRF\nETtTqnryH/N3grkJf4QfhKmJHE8+VjXWftfPKVCrtRjl52gw3n79iTRklaiMESIREdF9Y1ZA1IKU\nl1fg1FmldDwwpCqxP3e7GFfySgEAZnIZxvh3avb42pLhYb6wt9NvUpVXUIYDMakQBAF/6+cFJ0sz\nAECxRotVx66jonI/ASIiopaMiT1RC3L6XCbU6goAgKe7LTzcbKRzURereusf9ukIW3OTZo+vLTEz\nk2N8tV77qJ8uQq3RooOZCV4N9ZXWt7+QU4yvz2cYK0wiIqIGa3Biv2HDBvj6+sLCwgIhISE4fPjw\nPesnJiYiIiIClpaW8PDwwNtvv21wPiYmBqGhoXB0dISlpSW6d++O1atXP9i7IGoj6tqU6np+GeIz\nCwAAAgQ80ZVLXDaGyKGdYWerXw3nTl4ZDsVdBwB0d+yAp4PdpHrfJWchQVlojBCJiIgarEGJ/bZt\n27Bw4UIsW7YMCQkJCA0NxejRo5Genl5r/cLCQowcORKurq6Ij4/HunXrsGrVKqxZs0aqY21tjYUL\nFyIuLg4XLlzAsmXLsHz5cnzwwQeN886IWhm1WotTZzOl49Bq4+urr4Qz0N0WHjaKZo2trTIzk+Px\n0V2l46gfL6KicpnLCd2d0dtZ/8REhIjVx68jv1xjlDiJiIgaokGJ/Zo1azBz5kzMmjULXbt2xfr1\n6+Hq6oqNGzfWWn/r1q0oLy/Hli1bEBgYiAkTJmDJkiUGiX2fPn0wadIkdO/eHd7e3pg2bRoiIyNx\n9OjRxnlnRK1MQqIS5eX6YThuLtbwdNcnlXllGoPdUJ/s5myU+NqqR4b7wbqDOQAgO7cEMUfSAAAy\nQcDfB/rAztwUAHCnXIP//p4GUeR4eyIiapnqTezVajVOnz6NyMhIg/J7JeHHjh1DWFgYzM3NDepn\nZGQgLS2t1mvOnDmDY8eOYeTIkfcTP1Gb8cdhOELlGO+fLmdDUzl5s2tHK3Svtgst/XkKhQkeHx0g\nHe/48YK066+9hSkWDfSWzsVnFmBvtV1/iYiIWpJ6Z9/l5ORAq9XC2dmwl9DJyQlKpbLWa5RKJby8\nvAzK7l6vVCrh7V31h9LDwwM5OTnQaDR4/fXXMWPGjDpjiY+Pry9cuk9s06Zxv+1aUaHDr7EpKFfp\nE0pri3zEx8dDrRPxTXI5yvTL2iPIsQKnTp1q7HBbjab6fnVy0ELUqVBapkXqtVJ8+OkB9O/tIJ3v\na6lBXLb+acrauIuA8hqcFG1n7QH+HmgabNemwXZtOmzbxuXv79/sr9kkf5mE+9gJ88iRIzh16hQ+\n/PBDrFu3Dhs2bGiKkIhatCvXS6Sk3sHeDK7O+jH05/K1UlLvYCagh23bSSZbEoW5HIP7V232FR2T\nhbJyrXQ82sUEzgr97zWNDvg2XQMth+QQEVELU2+PvaOjI+RyObKysgzKs7Ky4OrqWus1Li4uNXrz\n717v4uJiUH63975Hjx7IysrCqlWr8OKLL9Z635CQkPrCpQa6+6mcbdq4HrRdT56Lh5WVfifU0SO6\nol+/XgCA7w6kwNKqGAAwpac7BgS61HmPtqw5vl979tTi0rW9yMnV7xVw5YY5Zj71kHTe2b8UL++/\niApRRC6AywpXTK+2ck5rxN8DTYPt2jTYrk2Hbds0CgoKmv016+3+MzMzQ9++fbF//36D8ujoaISG\nhtZ6zaBBgxAXFweVSmVQ393d3WAYzh9ptVrodLqGxk7UJoiiiDOJVR+EQ3rrk8W0gjIk5eiTehNB\nwAjfjrVeT43DzEyOZ6f0ko73RF/BrcyqJS4721vi6Z7Vl8BU4mLlvw8REVFL0KDn+osWLcLnn3+O\nzZs348KFC1iwYAGUSiXmzJkDAFi6dClGjBgh1X/qqadgaWmJGTNmICkpCVFRUVi5ciUWLVok1Xn/\n/ffx888/4/Lly7h8+TI2b96M1atX4+mnn27kt0jUst3MKJJ6iS0tTBHgp0/go1OrVsLp724LewtT\no8TXnoT290D3AP2OvlqtDlu+OWdwfnxXZwR1stafF0X85/h1lGm0Ne5DRERkDA3aunLSpEnIzc3F\nO++8g8zMTAQHB2PPnj3w9PQEoJ8Qm5qaKtW3sbFBdHQ05s2bh5CQEDg4OOCVV17Byy+/LNXR6XRY\nsmQJrl+/DhMTE3Tp0gUrV67EX//610Z+i0QtW0K13vrgQCeYmMig1upwsNoSl4/4ORojtHZHEAQ8\nN/0hLH7jAAAR8QkZSEhU4qFg/RAouUzAywO88dIvF1BaoUVmsQqfJNzES/3qfhJJRETUXBq8J/3c\nuXMxd+7cWs999tlnNcqCgoIQExNT5/0WLFiABQsWNPTlidqshPNVif3dBPL4zXwUqvSrsHSyNJM2\nSqKm5+djj4fDfXAw9hoA4LOvE7DmnUjI5foHnC4dzDGnryfW/H4dAPDL1Rz0d7PFAHc7Y4VMREQE\noIlWxSGihlGrtUi6WLUu+t3E/pfUqrLIzo6Qyxq+0hT9eU9NDIaFQj/0Kf1WIfb9etXg/HAfBwz2\ntJeO151I4660RERkdEzsiYzowqVsqNX6nnl3Vxs4d7JCRlE5zmYVAdDvfjqyMyfNNjd7OwUmPt5d\nOv42KglFxVWLAQiCgPkhXnCoTP4LVBXYcCq92eMkIiKqjok9kRGdPldtGE6QfhO3fdUmzYa42qCT\npVmzx0XAmEh/ODt1AAAUl6ixbWeSwXkbcxMsGFA1tv5Ieh5ib9xp1hiJiIiqY2JPZEQJiVX7QzzU\n0wUVOhEHr1Ul9qM4adZozEzlmDG1avnLXw5exY2bhmsSh7ja4pHOVf9GG0+lI6+MQ3KIiMg4mNgT\nGUlObinSb+kTRRMTGXp07YTfb+Ujr3KsdkcLU4S42hozxHavfx83BAc6AQB0OhEff3EaOp3hjrPP\n9/aQnqoUqiqw4dQNiNyVloiIjICJPZGRnD1f1Vsf2LUTFAoTg2E4I305adbYBEHAc9Megqzy3yHp\nYjZ+2nfJoI6lqRwL+1cNyTl6Mx8xaXnNGicRERHAxJ7IaAzG1we7IKtEhdOVO50K4KTZlsLb0w4T\nxlZNpN26/XyNITkPudhgtF8n6XjT6XTc4ZAcIiJqZkzsiYxAq9UhMbmqx753sDP2p+ZChH4IR28X\na7h0MDdWePQHf3k8EJ199Mtbaiq0WLvpd2gqDHecfe4hdzhVDskpUlfgfyfTOCSHiIiaFRN7IiO4\nci0PxSVqAIC9nQXc3WxwIJU7zbZUJiYyLPjrAJiZygEA12/kY1tUskEdS1M5FlZbJef3jAIcSuMq\nOURE1HyY2BMZwRmDYTjOOKUsRE6ZPtG3MzfFADdOmm1pPN1tMH1SsHS88+eLuHApx6BOL2cbPOZf\nNSTnw1PpyC1VN1uMRETUvjGxJzKChMSqxL53sAsOVFvickTnjjCV80ezJXp0pD96Bur3GxBFEes/\nOoGyP+w4O6OnO1wrh1EVa7T4XzxXySEioubB7IGomRUXq3E59e4QDQF+AR1xMqNqMuYIX06abalk\nMgHzX+gHSwv9jrNZt4vx+TdnDepYmMrxt2qr5JzIKOAqOURE1CyY2BM1s7NJWVIPrn9nByTcKYWm\ncm10fwcreNoojBke1cOxoyWef6a3dBx9KBXxCRkGdXo6WWNMl2pDcs6kI7+cq+QQEVHTYmJP1Myq\nD8N5KNgZh65XTbAc7uNgjJDoPkWEemNQPw/peMPmeNzJLzOoM6OXu8HGVR+eTm/WGImIqP1hYk/U\njERRREK1jak8/DsiKacYACAXBIR72RsrNLoPgiBgzsy+sLe1AADkF5TjvXVHoVZXLYFpaSrHS/28\npOPYG3k4fjO/2WMlIqL2g4k9UTNKv1WI3DulAABLC1OkQSed6+NqAzuFqbFCo/tk3cEcf5vTH4Kg\n35X20tVcfLjllMFE2b6utnjYp2rOxAfxN1Csrmj2WImIqH1gYk/UjKoPw+nZwwkxN6omVQ735jCc\n1qZXD2fMmNpLOj4Udx0/779sUOf53h6wr/zAdqdcg80Jt5o1RiIiaj+Y2BM1ozOJVcNwHP07IqNY\nBQCwMpVjgLudscKiP+GxR/wxdIiPdPz5N2dxNqnq39nG3ARz+3pKx/tTc3BGWdicIRIRUTvBxJ6o\nmajVWlxIyZaOc61MpK8He9rD3IQ/jq2RIAiYM6Mv/Dvrh9zodCJW/+8YMpVFUp3BnvYY7Fk1f2L9\niTSUabQ17kVERPRnMJMgaiYXL+dAXZnMubhY41ROiXRuGFfDadXMzORYsiBUmkxbXKLGu+uOoLSs\naonLuX08YW2m/zB3u1SNLecyar0XERHRg2JiT9RMzl+o6q23C+iIospJlJ0szRDUqYOxwqJG4mBv\ngdcWhsLURA5AP1F63abfoavco8DewhQv9K5aIvOny9lIzi42SqxERNQ2MbEnaiaJyVXjrgvtzKWv\nh/s4QFa5sgq1bv5+HTHnub7S8ckzGfhqe6J0PNzHAX1dbQEAIkSsP5kGtVZX4z5EREQPgok9UTMo\nLdPgcqp+IyqtTECapmrJw2HVlkOk1m/YEB+MHRUgHe/6+SJ+qlwpRxAEzAvxhMXdXv3CcmxLVtZ6\nHyIiovvFxJ6oGSSnZEtDMhR+9tBVdtD7O1jB00ZhxMioKTwzuSdCHnKTjj/9KgGHj98AADhbmePZ\nnlXndiQrcf0Pu9YSERE9CCb2RM0gMfm29LXKyUr6ejgnzbZJcrkMf583EF273H0aI2LdhyekZTDH\n+HdCoKN+XkWFqB+So9WJddyNiIioYZjYEzWD85WJvdpUhgJTfXe9XBAQ7mV/r8uoFTM3N8H/LRoC\nDzcbAIBWq8N7644i9XoeZIKAl/p5wVSm/15IyS3Bj5dv3+t2RERE9WJiT9TECotUuHYjHwBQZK+A\nQqFf8rCPqw3sKnckpbbJuoM5Xn8lDA72+mUwy8o1eGd1HJS3i+Fla4EpPVylul+cy4CycsMyIiKi\nB8HEnqiJnb+g74kVAVS4doCsspd2uDeH4bQHnRyt8MbicFhZmgEA8gvK8daqWOQXlGNCN2f4VK59\nr9Lq8L+TNyCKHJJDREQPhok9URO7O76+XCGHzlLfW29pIscAdztjhkXNyMvDFktfHiytca/MKsY7\nq+OgUWvxt/7e0nKnZ7IKEX0t15ihEhFRK8bEnqiJ3U3sC2zNYVE59CbU0w7mJvzxa08Cu3bConkD\nIVQm8anX8/Du2iPwtVbg8QAnqd7mMzeRV23HWiIiooZiZkHUhHJyS5GhLIIIoNjWXBpfP5TDcNql\nAX3d8dcZVRtYnb9wG2s2HMfUQGe4WOk3LSvWaLHxdLqxQiQiolaMiT1REzp/Ud9bX2JlArmVGQQB\nsFeYoqeTtZEjI2OJHNYZ0/8SLB2fOH0Ln32RgPkhnlLZkfQ8HEnPM0Z4RETUijGxJ2pCd4fhFNqY\nw8JC31sf5mUPeeUEWmqfxj/WDY+P7iod/xp3DQmHrmGkb9UuxBtPpaNQVVHb5URERLViYk/URERR\nRGLybegEoNDGDBYchkOVBEHAM1N64uFwX6nsx18uwTGzGB0t9PMw8so1+PjMTWOFSERErRATe6Im\noswqRk5uKYo7mAImMijMTeDawRwBDpbGDo1aAEEQMGdmXwzo6y6VRUUlo5/MRDr+9XouTmYUGCM8\nIiJqhZjYEzWRc9WH4ShMAAGI8HKQVkUhkstleHnuQAQHVq2Kc2h7EvxMqpL7D+JvoFSjNUZ4RETU\nyjCxJ2oi5y/chlYGFFmbSstchnvbGzkqamnMzORYsmAwunS+O0RLxM19VyCr0AEAskvV+DThlvEC\nJCKiVqPBif2GDRvg6+sLCwsLhISE4PDhw/esn5iYiIiICFhaWsLDwwNvv/22wfmoqChERkbCyckJ\nNjY2GDhwIH788ccHexdELYxOpx9fX2RtBlEQYGFhgs52lvCu3GWUqDpLC1O8/koY3F1t9AUqLbSn\nlVCr9D31e69m49ztIiNGSERErUGDEvtt27Zh4cKFWLZsGRISEhAaGorRo0cjPb32tZYLCwsxcuRI\nuLq6Ij4+HuvWrcOqVauwZs0aqU5sbCxGjBiBPXv2ICEhAY8++ijGjx9f7wcGotYg/VYBCotUKLQx\nh1wmwMxUjgj21tM9WHcwx+uLw2Bvp//wZ55ThrLr+aio7LlffyIN5ZVfExER1aZBif2aNWswc+ZM\nzJo1C127dsX69evh6uqKjRs31lp/69atKC8vx5YtWxAYGIgJEyZgyZIlBon92rVr8eqrryIkJASd\nO3fGG2+8gb59+2LXrl2N886IjCgx+TYq5AJKrExhYWEKCEC4F1fDoXtzcrTCG4vDYGlhCgGAfVoB\nbmcWQasVkVmswleJGcYOkYiIWrB6E3u1Wo3Tp08jMjLSoDwyMhJHjx6t9Zpjx44hLCwM5ubmBvUz\nMjKQlpZW52sVFhbCwYHJD7V+icm3UWhjBlEALBQm6NGpA5yszIwdFrUC3p52eG3hYJiYyGBaIcL+\nRhGUWcUQdSJ2pdxGcnaxsUMkIqIWyqS+Cjk5OdBqtXB2djYod3JyglKprPUapVIJLy8vg7K71yuV\nSnh7e9e45oMPPkBGRgaefvrpOmOJj4+vL1y6T2zTxqfTiTh+MhW53g7Q6XQANPCsyGNbN4L21IaP\nDrfHt7vSYVJSCtFShnRdBTram+ONfQl42d8c5vLGW12pPbVrc2K7Ng22a9Nh2zYuf3//Zn/NJlkV\n536X8/v+++/x6quv4uuvv4anp2f9FxC1YBnKMhSJAlQdzCCXCzAzkaGnrdzYYVErE9TNFo+NdIUA\nwFmObQgAACAASURBVDEtD5rSCuQXqJGrErFXqTF2eERE1ALV22Pv6OgIuVyOrKwsg/KsrCy4urrW\neo2Li0uN3vy717u4uBiU79ixA88++yy+/PJLjBkz5p6xhISE1BcuNdDdT+Vs08YVHx+PK9dLUOFi\nC5lMhg5WZhji54qIgV2MHVqr1l6/X0NCAHuHRHz/4wWI2eXIMDWBWiPDmTJzTPTwx0MuNn/q/u21\nXZsa27VpsF2bDtu2aRQUNP8Gg/X22JuZmaFv377Yv3+/QXl0dDRCQ0NrvWbQoEGIi4uDSqUyqO/u\n7m4wDOe7777DM888gy1btuDJJ5980PdA1KKkphWjwFY/v8TCwhRDuRoO/QlPTQzC0CE+sC1Qw7pI\njdy8MpSUaLD2RBo3riIiIgMNGoqzaNEifP7559i8eTMuXLiABQsWQKlUYs6cOQCApUuXYsSIEVL9\np556CpaWlpgxYwaSkpIQFRWFlStXYtGiRVKdb7/9FtOmTcPKlSsxZMgQKJVKKJVK3Llzp5HfIlHz\n0Wh0uJSjhspcP/TGzsoMA93tjBwVtWaCIGDuc30R1K0TXDNLINfqkJVdgpt5Zfj4zE1jh0dERC1I\ngxL7SZMmYe3atXjnnXfQu3dvHD16FHv27JHGwyuVSqSmpkr1bWxsEB0djYyMDISEhOCll17CK6+8\ngpdfflmq8+GHH0Kn02HBggVwc3OT/ps4cWIjv0Wi5pN2qxQFNvp1yM1M5Rji7QALU46vpz/H1ESO\nV/8WCg9HK7goSyGKIjKzirHn0m2czGj+R71ERNQy1TvG/q65c+di7ty5tZ777LPPapQFBQUhJiam\nzvsdOnSooS9N1GpcuVaMEgd9Ym9hYcJNqajRWHcwxz8WDcFrK35FUaEahTZmUGYV47/Hr2PTmB6w\nMW/wr3MiImqjmmRVHKL2KiGzHBVm+h56xw7m6PMnJzcSVefuaoNXXhoE9+wymFTooFJrkZKejw9P\n1b4LOBERtS9M7IkaSVGxCtfEqmE3I/w7wVTOHzH6//buPDyq6vwD+Pfe2SeTfZssZIMQyAKEnQAK\nWlBQaa2KKyp1FxFBq8XlUeterS22gFqtYqsFa6lV4YcEgywSlCWBkAUC2bdJJstkmf3e8/tjJjcZ\nwxZImCzv53nGmXvuuZczx8nMO2fee07fGp8SjoduT0dEbQcAoMPswH9yq7G3stnLLSOEEOJtFHUQ\n0kdy8+vREehKw1Gr5LgqMdTLLSJD1fy5Cbh9VgL8Ta6Zx1pMVvw+8zjqO+xebhkhhBBvosCekD6y\n9WgNRPcIfbiPEmNDfLzcIjKULbk5DQvC/aBwiACAyvp2PJ95HILIvNwyQggh3kKBPSF9ZL+hVXo8\nNz6o1yswE9IbMhmPpx6ajkkOBo4BjAF7ihvw8WHKtyeEkOGKAntC+kBpTSvq3H9NHAfcOGmEdxtE\nhgWNWoFXHp6BSJMrBcfpFLFubyny6lrPcSQhhJChiAJ7QvrAFwcrwdwj9EEcw6hgSsMhl0ZUhB9e\nujENWrMDANBhceDx/x1Dm83p5ZYRQgi51CiwJ6QPfF/etWJyqo5ScMillTFlBO5LjoBMcOXXVzSZ\n8cyWAjBG+faEEDKcUGBPyEWqb7ehzOyejYQBc2LU3m0QGZbuXzwOM1VKaXtHcQM2Ha7yYosIIYRc\nahTYE3KRvjxSI81EorPYkRih8XKLyHAkk/F468HpiLK5ZskRRYY3dp7EyYZ2L7eMEELIpUKBPSEX\naduJeulxkgrgeUrFId7h56vCO7elQ20XAAAWh4CHNubA4hC83DJCCCGXAgX2hFyEcpMFZSYrAIBj\nDNMjlOc4gpD+lTI6FE9MjwPnzq+vbLfh8f8cpXx7QggZBiiwJ+QiZJU2wmJ1zT7i2+bA2Hidl1tE\nCHDHVUmYF9T1WtxZ1oSPfiz3YosIIYRcChTYE3KBGGPYWmiQRkLjeB5BATRiTwaGP9w9BTHuGS8Z\nY1jzQykOVbV4t1GEEEL6FQX2hFygQmMHqlosAACZwHDZyGAvt4iQLmq1HGuXTISPzZVfb7MLWPXV\nMdR32L3cMkIIIf2FAntCLlBWWRMsls40HDvSU/RebhEhnhLjgvDk9DjIna6ZcgwmK1Z9dQw29zYh\nhJChhQJ7Qi6AXRCxu7wJVvfqnv4mO9KSw7zcKkJ6WnzVaFztp5Uupj1c0Yy39pyii2kJIWQIosCe\nkAvwY7UJDa2u2XAUDhGpoTr4+aq83CpCeuI4Ds/fMwWjzK6UHFFk+HduNb7uNk0rIYSQoYECe0Iu\nwI7SRpgtDgBAQIsN41NotJ4MXL46FV6/YyICTDYAgNXqxB92nsTJdprfnhBChhIK7AnpJaPZjsN1\nrVJ+vb/JhnEp4V5uFSFnlzImDI9Oi4PaPT2rsdmC94qsqLFQvj0hhAwVFNgT0ktZZU2wOwTYHQJ8\nOhzwYRzGjg7xdrMIOaebF43FL9Qq6WLauhY73j9lQ127zcstI4QQ0hcosCekFxhjyCwxosPsSsPx\nN9mQlhwGlUru5ZYRcm4yGY/VD0zDmAYreJFBEBiqmux4ftdJmNwXghNCCBm8KLAnpBcKjB2oabfB\nbHaAFxn8Wu2YNCHC280i5LwFB2mx8vZ0jKhsA8cYzBYBRXWteHH3SVidlHNPCCGDGQX2hPRCZokR\noshgsTrgb7KBZ8BkCuzJIDNz2ggsTI9CaGkzwIAGoxn5hja8vq8UgkjTYBJCyGBFgT0h58niELCn\nshkWixOMueaujx0RgNAQH283jZBeu3dJOqKYgOAqE0SRod5oxoFqE/56sILmuCeEkEGKAntCztOe\nymZYnSI6zA6obAI0FiempEd6u1mEXBAfrRI3XRcNf2MHQowWWCwOmFpt2F5ixMdHayi4J4SQQYgC\ne0LO047SRoABZosd/iYbOIDy68mgFjfCB7OnhyC0wYKAFhsam82w2wV8UViHD3OrKbgnhJBBhqby\nIOQ8VLdZkd/QDpvNCdHJENBig5+vCokJQd5uGiEX5crZYWg2mYDyZggyDvWKDkRF+uG/xw2wCSIe\nmjQCPMd5u5mEEELOA43YE3IevittBAB0WBzQtdshFxgmTYgAz1PAQwY3uYzHow9MhUohQ1R1O1RG\nC5qaLQCArScbsOancrqglhBCBgkK7Ak5B0Fk+K60CQBgNrtmwwGAyRMov54MDTHR/lhy8zjwDIiq\nbger6FpZeUdpI97aXwonBfeEEDLgUWBPyDnkGFphtNjhdIpwWhzwbXNALucxPjXc200jpM8snDcK\nE9L04ABE1rQD5SaIgiuY313RjNd/KIFdEL3bSEIIIWdFgT0h57CjxJWG4xqtt4MDkDImDFqNwrsN\nI6QPcRyHR+6dAl+dChwA/5IWBDZbpf3Z1S14eW8JLA5axIoQQgYqCuwJOYtWmxP7q1sAAB1mBwJa\nOtNwaDYcMvQEBWrw0G8mAQA4AJafajBepZT2H6o14bffHYehw+alFhJCCDkbCuwJOYusskY4RAYm\nMnAtVqhtrtFKmuaSDFXTJ0fjysviAbiC+/JtJ3FdXLC0v7TFgpXbi5Df0O6lFhJCCDkTCuwJOQNB\nZPimuAEAYLY44dfkSksYEeUPfZjOm00jpF/95o4JCHe/xi0WByp3lGL55BjI3dNemmxOPJ11AttP\nGb3ZTEIIIT9DgT0hZ3Cw1oTadlfKgdPSbTacdBqtJ0ObRq3AYw9Ok6ZzLTzRAHOhEa9eMRr+Ktfy\nJ07GsOZAOd4/XEnTYRJCyABx3oH9unXrEB8fD41Gg8mTJ2Pv3r1nrZ+Xl4fLL78cWq0W0dHReOml\nlzz219XV4bbbbsPYsWMhl8uxdOnSC3sGhPSTr0+4RuvBAE1dB3h37ELTXJLhIGlUMG5clCxt/2tz\nPtQdDvxp/hgkBGil8v+dqMcLu0+izeb0RjMJIYR0c16B/aZNm/DYY4/h2WefRW5uLjIyMrBgwQJU\nVlaetn5rayvmzZuHiIgIHDx4EGvWrMGbb76Jt99+W6pjs9kQGhqK1atXY9q0aeBoZUMygJSbLMgx\ntAIA7HYBqlpXPrGvToXRI2m1WTI83PTLsRg90pVfLwgi/rz+RwTIZfjDlaORER0g1Ttc14pHvy1E\nAeXdE0KIV51XYP/2229j6dKluOeee5CUlIR33nkHERERWL9+/Wnrf/rpp7BardiwYQOSk5Nxww03\n4KmnnvII7GNjY7FmzRrceeedCAqiQIkMLNJoPQC9wKB0uObvnjheD5mMMtjI8CCT8XjswWlQudNv\nqmtb8f4nh6FRyLB6ZgJuSelKS6s32/FU1gl8eqyGUnMIIcRLzhmh2O12HD58GPPnz/conz9/Pvbt\n23faY7KzszF79myoVCqP+jU1NSgvL7/IJhPSv9psTmSVNUrbiso26TGl4ZDhRh+uw713pEvbO/eU\n4btdpeA5DkvSIvH0zAToFDIAgMgYPjtWi9VZJ2hKTEII8YJzBvZGoxGCICA83HOVzbCwMNTV1Z32\nmLq6uh71O7fPdAwhA8X2kkbY3CtsRmqUaD7VBMA1ejkhjVabJcPPFZfF4fKZsdL23z45jNJy1/oO\nM0cE4q8LkpEa6ivtzze2Y/m2Quwub7rkbSWEkOFM3h8n7a98+YMHD/bLeYcz6lNPAmP4tMgGs92V\nSqBtaoK5wwwAGBnng8KCo+d1HurX/kH92j/Op1+npHE4lCPA0GBDB4BnX96Kh+4eCbXKNVr/az+G\nIKsT2w1OiAwwA3h+Rz4mBcrwyygFNLLhdx0VvV77B/Vr/6G+7VuJiYmX/N8854h9SEgIZDIZDAaD\nR7nBYEBExOmn/dPr9T1G5juP1+v1F9pWQvpdYauIZndQ7yMHWgu7RhyTR/t5q1mEeJ1SyePW62Og\nVLg+NoxNdvx3azUYc/29yDgOV4Yr8PBIFYKUXUH8oWYBbx63Ic8keKXdhBAynJxzxF6pVGLSpEnY\nvn07brjhBqk8MzMTN91002mPmTFjBp566inYbDYpzz4zMxNRUVGIjY097THnY/LkyRd8LPHU+a2c\n+tTTf7JOQOvjyqm/MsIfe76rhY+PFnI5j7tunwNfneqsx1O/9g/q1/5xIf2q8xuBP63fDwAorXSi\nvjkA18zvGpWaDGCBQ8C6gxXY6U7FEQB80QDUqgPx0MQRCNQo+uw5DET0eu0f1K/9h/q2f5hMpkv+\nb57X9B6rVq3Cxx9/jA8//BCFhYVYsWIF6urq8OCDDwIAVq9ejV/84hdS/dtuuw1arRZ333038vPz\nsXnzZrzxxhtYtWqVx3lzc3ORm5sLk8mExsZG5ObmoqCgoA+fHiHnr6zFgqP1rqBexnFQVXdN3Tcl\nPfKcQT0hw8HsGTG46oqR0vbH/zqCEycbPepoFTI8MSMeT89MQJC6K4j/obIZD/1fATJLjNJIPyGE\nkL5zXjn2ixcvRmNjI15++WXU1tYiLS0NW7duxYgRIwC4LogtKSmR6vv5+SEzMxPLli3D5MmTERQU\nhCeeeAIrV670OO/EiRMBuHLyGWP4+uuvERcX53EuQi6Vr07US49nRPnj0OddXzLnzIzzQosIGZiW\n3j4BxSVNKClrhiCIeGttNv740rweX35njgjEuDBf/P1INbaXGAEAbXYn/vxTOXaVN+ORKTHQ0xdm\nQgjpM+c9IfdDDz2E0tJSWK1WHDhwALNmzZL2ffTRRz2C8dTUVOzatQsWiwXV1dV47rnnepxTFEWI\noghBEKTHFNQTb2i1OaW0AQBI4mVobrEAAPz91EgfR9eGENJJqZDhiUdmQOtOqTE2mvHHtfvhcPbM\no/dVybFiaixemZMIvU9XEJ9jaMVD/1eAjfm1sLtnoSKEEHJxaKUdQgB8W2KUgotRgVqU5tRK+y7L\niIFcTn8qhHSnD9Ph0funSttH8w1Y9+HBM6bYTND7Ye2Csbg+KRy8e+Y0uyDiH3k1WL6tELl1rZek\n3YQQMpRRtEKGPZtTxFfHu9Jw5sUG4cDhGmmb0nAIOb2pk6Kw+Fcp0vauH8rxj8/zzlhfLZfh3vRo\nvPWLJIwK1ErlVW1WPPN9Mf6wrxSNZnu/tpkQQoYyCuzJsLflZAOarA4AQLBGAa66HXaHK6UgLiYA\n8bEB3mweIQPazdcnY96cBGn7yy1F2LK9+KzHJAX74O15Y/DgxBHwca9aCwC7Kprw4NYC/O94PQSR\nLq4lhJDeosCeDGtmh4B/F3atuXBzcgR+yC6XtufOivNCqwgZPDiOw/13TcSU9Eip7MN/5uKHHyvP\nepyM53Dd6DC8tzAFc2ODpHKzU8D7OZV49NtC5LlnqSKEEHJ+KLAnw9rXJ+rRanMCAMK0SozzUaHo\nhGv2DpmMx+wZMd5sHiGDgkzGY9XD05E0KthdwrDmvR9xrLD+rMcBQKBGgSdmxOPVuaMR7auWystM\nFvwu6wTezKb0HEIIOV8U2JNhq83mxH+KulZUvjU1Anv2VUjb6eP0CPBXn+5QQsjPqFRyPL1qFqIi\nXCs0O50iXv/zDyivbDmv48eH++KvV4/F3eOjoO52sfr35U14YGsB/lNogJPScwgh5KwosCfD1pfH\n69HhzqWP9lVjbkwQvv+hTNpPaTiE9I6vToVnn5iNQH8NAMBsceD3b+5BRdX5rb6okPG4aawe7y5M\nwWUxgVK5xSng70eq8Mi2Apo9hxBCzoICezIstVgd+PJE12j97akRKDphhLHRDADQ+SgxaUKEt5pH\nyKAVHuqDZ5+YLc1x39xiwdMvZSE3r+4cR3YJ1SrxVEYCXp2biBg/jVRe2eqaPefVH0pg6LD1edsJ\nIWSwo8CeDEtfFBpgdbrmrY/z12BWTCB27i2T9s+eEQNlt9k6CCHnLz42AL97bCbUatfi5maLAy//\ncQ8yv+/dAoTjw/3wl6vH4t4J0dDKu/4ef6hsxkNbXYtb2Zy0uBUhhHSiwJ4MO41mO7acbJC2l6RF\nwmZzIvtAlVQ2Z2asN5pGyJCROjYMrzxzBYICXSPuosiw/u8H8Y/Pj0LsRa68nOdw/ZhwvHdNCq6I\nC5bKbe7FrZZtK8CP1S1nXBiLEEKGEwrsybCzqaBOWmU2McgH06L8sf9ANWzu2XGiI/0wKiHobKcg\nhJyH+NgAvPH8lUiI7cqX/+83RXh73X7Y7UKvzhWkUeDx6XF488okjOy2uFVtuw2/33MKL+w+hapW\na5+1nRBCBiMK7MmwUtduw7enjNL2krQIcByHzF1dKQJzZ8eBcy95Twi5OMFBWrz0zBxMntA1z/2+\nnyrx/Ovfo6nF0uvzJYfq8Kd5Y/DwpBj4KuVS+cFaE5b9XwE+zK2C2dG7Lw2EEDJUUGBPhpVNBXVw\nun+yTwnRYaLeD0fzDdLc9TzP4fIMSsMhpC9p1Ao8tSIDC+eNksqOn2zEo09tw5btxRCE3uXJy3gO\n1ySG4r1rUrBgZCg4uL6IOxnD5iID7t+Sj8wSI0RKzyGEDDMU2JNho6zFgu9KG6XtJeNcI4gb/5sv\nlV0xO17KCSaE9B2ZjMe9SybiN3ekA+5A3Gxx4MN/5uDJ53fg+MnGs5/gNPxVcjwyJQZrrhqDlBCd\nVN5sdeDPP5XjiR3Hcbyxo6+eAiGEDHgU2JNhQRAZ3jlQDsE9gpce7oe0MF/k5hk8Vpq98ZdjvdlM\nQoa8a+cn4sXfXS4tZAUApRUtWP3777Duw4Noa+/9NJYjA7V448rReHJGPEI0Sqn8eGMHVmUW4Y/7\nS9FAq9cSQoYBCuzJsPDf4wZp5E7Bc7g3PRqMMfxr8zGpzrw5CQgL8fFWEwkZNtKSw/DHl+fhthtT\noeyWJ79jVwmW/fb/sGV7McwWR6/OyXEcLo8NwrsLk3FLcgQUfNd1MlllTXhgSz7+mVcDq5Py7wkh\nQxcF9mTIq2y14tO8Wmn71pQIxAVocDCnFidLmgAACrkMN1w3xltNJGTYUSpkuHFRMta8dhWmpHdd\nWNveYceH/8zBPY9+jfUfHURJWXOvzqtRyLBkXCTeXZiCGVEBUrlNEPGv/Frc9w3l3xNChi75uasQ\nMngJIsOan8phF10X540K1OKGsXqIIsPGbqP1V105EsFB2jOdhhDST8JDfbB65SwcOFyDD/+Zg3qj\n65c1m82JzJ0lyNxZgsSEYMyfm4BZ00dApTq/jy29ToVnZ4/EEUMbPsytwqlm16rSTe78+2+KG3BP\nejTGhfn223MjhJBLjQJ7MqR9daIehcZ2AICc4/DYtFjIeQ7ZB6pQWtECAFAq5bj+GhqtJ8SbpkyM\nxLiUMHy3uxTfZp1CZXWrtK+4pBHFJY346LNcjE8NR/o4PdLT9Of1ZXx8uC/+NG8MvitrxD+O1qDJ\n6krxOdlsxuqsE5gU4Y+7xkV6zI1PCCGDFQX2ZMiqbrPik7waaXtxih7xAVoIgoiNm7tmwlk4bxQC\nA9TeaCIhpBuVSo6F8xKx4BejUHjCiO1ZJdh3oBJOp+sXN7PFgewDVdIq0bEjApA+To+J4/QYPSoY\nSoXstOeV8RzmJ4Rg9ohAfFFowObjBmmRukO1JhyqNeHymCAsSYtEhK/q0jxZQgjpBxTYkyFJZAxr\nfiyXPrzjAzRYPFYPANj3UxUqq00AALVajl8tTPJaOwkhPXEch+SkUCQnhWLp7eORtacM23eWwFDf\n7lGvvLIF5ZUt+HJLEeRyHqPigzB6ZDCSEl23oADPqWs78++vGhmCT4/VIKusScq131XRhB8qm3HV\nyBDcmhKBQI3ikj1fQgjpKxTYkyHpm+IG5LtTcGQch5XT4qCQ8RAEEZu6zVt/7fzR8KMROkIGLH8/\nNa6/Zgx+tTAJldWtOHy0DrlH65B/vMFjYSunU0RRsRFFxUZgm6ssNNgHSYnBSIgLxMi4QCTEBcBH\nq0SYjxIrp8Xh+qRw/COvBvurXWl5Tsaw5WQDdpQ2YuGoUPx6TDiCKMAnhAwiFNiTIae2zYaPj1RL\n2zeN1Uv5s7v3VaCmrg0AoNUosOjq0V5pIyGkdziOQ0y0P2Ki/fGrhUmwWB04VtCAnLw6HDlmQK2h\nrccxDY0daGjswN79FVKZPlyHhNhAjIwPRNyIACxLjcQNY8Kw4WgtjjW4zmETRPz3uAFbihswf2Qw\nbhyrR6hW2eP8hBAy0FBgT4YUs0PA6/tKYHOP5MX6a3BziisFx+kU8fn/CqS6ixYkQaejD2tCBiON\nWoEpEyMxZaJrqkxTqxUnTjah6KQRx4sbcbK0GXa7s8dxdYZ21Bnase+nSqnMV6fCiGg/TI7QIZ9n\naGYMCoUMdoj4prgB204acWV8MBYn66HX0S98hJCBiwJ7MmTYBRGv7C3BSfe0djKOw4qpsVDKXMs1\n/OPzo1KOrs5HiWuvSvRaWwkhfcvfT+0R6DudIsoqWnCytAmnSptRUtaMiupWj/SdTm3tNhQUNQBF\nDWAAeJ0ChhANbD4KKBQ8FHIZ/tVkxuajNZiq98OvxoRhWnwwZDJaCoYQMrBQYE+GBJExvL2/DLmG\nrinylk2OQVKwayXZH36sxNfbTkj7Fv8qGVrKnSVkyJLLeYxKCMKohCCpzG4XUF7ZgpLyFpSUN6Oi\n0oTyKhOs1q6RfQ6Ab7sDunYHOnwUMIZo0K7t+qj8qqEDX+XVQmNxIs7JkKRVISxIi5BgLUKCNAgO\n0qK21gx/XwVEkYHvtgIuIYT0NwrsyaDHGMP7h6uwp7Jrhco701wzXwBAZXUr/vrBAWnflPRILJxH\no/WEDDdKpQyJI4ORODJYKhNFhnpjByqrWlFe1YLyShNqattQY2gH1+GAT4cDZq0cxhANOny6BgMs\nGjkKARQ7nQg82YDAA1bIBdcMOx0drl8N139SjeBADcJDdQgP94E+TNd1C/eBRk2DC4SQvkWBPRn0\n/l1owNfF9dL2dYlhWJzsyqs3Wxz4wzv7YLO5RuTCw3RYfv9UGkUjhAAAeJ6Tgu3ONB7ANWDQbLKi\ntq4dtYY21NS1o8jQhlyzDZUcg0N0BfFOOY+GUA2MIWro2h3wN9nBm83gmCsdyNDQAUNDB1DQ898O\nCtQgPiYA8bGBiIvxR3xMAMLDdPT+RAi5YBTYk0Fte4kRG452zYAza0Qg7kuPBsdxYIxh7QcHUV3r\nSs9RKuV46tEM6HzogllCyNlxHIegAA2CAjRIGRPqsa/F6sCWE/X4qqgeDR12OJ0inIIIp5+IpjAR\nzmgttE0WcK0CNBYnzhSmNzVb0NRswaEjtVKZWi1HXEwAxiSGIDkpBGMSQ+g9ixBy3iiwJ4PWj9Ut\n+OuBrmnsxof74vHpcZC5R7u+3nYC2Qe6Zr54cOlExMUEXPJ2EkKGlgC1ArePi8LNqZHYV9nssW4G\nAJg7eCBMA16jgY9SjtEaJfQCA9dihaHe7B7Fb5dW1O3OanWi6IQRRSeM+HILAHCIj/FHclIoxiaF\nIGVMKPz9aKVsQsjpUWBPBh3GGL4pbsCHuVUQ3KtGjgzU4tlZI6UZcPKLGvDJpqPSMVdfOQpzZsZ5\no7mEkCFKznO4LDYIl8UGoabNiqyyJmSVNaHUnWPP8RyanQJ+bLMAAIL8FJg6NgbzIv2RFqpDY4MZ\nZRUtKK1oQWl5C0rLm9HaZvvZv8Jc+ytasCWzGACHhLgATEjVY0JaOJISg6GQyy7tEyeEDFgU2JNB\npcMu4J0D5djb7UJZvY8KL142ClqF68OtqcWCt9fuh+jOgU1MCMbdt433SnsJIcNDpK8ad6RF4rbU\nCPx710841CygVJDB7BSkOk1WB7adMmLbKSNUMh7jw30xLswXGUmjcHuABjwHNJusKD7VhILjDSg4\nbkRJWTOYewDDhaGkzDV95+ZvCqFWy5E6NgwTUsORnqZHhN730j95QsiAQYE9GTRONpnx+r4S1LZ3\njWiNCtRi9cwEBLqnrjxZ0oQ3/5KNZpNrhMxXp8Jvl8+AUkEjWoSQ/sdzHEbqZBipkyF1wjjk1rVi\nf3ULfqoxwWTrmlbTJoj4qcaEn2pMAACdQobUMF+khemQGh+IJemRkPMczBYHjhc3ouB4A/KLjDPw\nKgAAFlNJREFUGnDiVKM0aAG4UncO5tTgYE4NANcEAZ1BfmpyGE3rS8gwQ4E9GfAYY9hy0ogPciql\nmSgAYOGoUNw7IRoqOQ/GGL7NOoW/f5or5a1yHIdVy6YjJFjrraYTQoYxtZzH9OgATI8OgCAyHG/s\nwI81JuyvakFVm9WjbrtDwP7qFuyvbgEAKGU8RgZqMSpQi8QgLWbNH4VbbkiFxeLAscJ65OYZkJtX\nh3pjh8d5DPXt+DarHd9mnYJMxiNpVDDGp4YjZUwoRiUE0SAHIUMcBfZkQGuxOvDe4UrsruhKvdHK\nZVg+JQaXxboWnrFYHVj/90PYu7/rQlqtRoFH75+K8Snhl7zNhBDyczKeQ3KoDsmhOiwdH4XqNiuO\nGtqQV9+OvPo2NFkdHvXtgohCYzsKu12Uq5bziA/QIsZPjZiMaDxw9Sio7CJKjzfgSH498grqpal9\nAUAQRHdKTwMAQKmQISkxGCljwpA6lgJ9QoYiCuzJgFTfYcfmIgO+LTHC3m0J+IQALX43Mx5Rvq5Z\nISqqTHjrr9moqulacTY+JgC/XZ4BfbjukrebEELOR5SvGlG+aiwYFQrGGKrbbMirb8PR+jYUGjvQ\nYLb3OMbq7BnsA4CvUo6YtFBMmhYFrsMBU20bak81ob68GbJuE+/YHQLyClxfAADX6ryxIwIwMi4Q\nCXGBGBUfiBHRfnQxLiGDGAX2ZECparXii8I67CxrgtPjgjFgwchQ3JfuSr2xOwTs3FOGjz47Aru9\na4Rq3pwE3HNHOpRK+mAihAwOHMch2k+NaD9XoA+4fq082WxGcaMZxU0dKG4y9xjV79RmdyLf2I78\nzgIlgLFBEEYHgrcLgNkBS5MFdpMNCocAhUOEwiFC7hRxsrQJp0qbpHN1BvvRkb6I1PsiIlyHCPc9\n5esTMvCdV2C/bt06vPnmm6irq0NKSgr+/Oc/Y9asWWesn5eXh0ceeQQHDhxAUFAQHnjgATz33HMe\ndXbt2oVVq1ahoKAAkZGRePLJJ/HAAw9c3LMhg5JDEFHY2IEtxQ34obIFDJ4B/ahALW5LjcC0qADU\n1bdj+84SfLerFG3dLqJVKuV44O6JmDsr7hK3nhBC+l6AWoHJEf6YHOEvlTVZHCg3Wdw3KypMFlSY\nrB4z73Qnk3GARg5o5NAGa6B0irBanTBZnbBYHXA4RHCMQe5kkDtF6VbbZoGsoAOyvFrIBAaZIEIm\nMATpVIgM8UFwgBqBAWoE+msQFKhxPQ5Qw89XDZ1OQSP+veQUGaxOAVanCKtThE0Q3Y+Fn2137bdL\nNyZtOwQGp8ggMAah271T9PxM5TiAAweOA3gOkPM8TE02yDngG/NJKHgOCp6HRsFDI5dBq5BBq+Ch\nlcugUcjgo5BBp5TBTyWHn0oOjZwHx9FqyQPFOQP7TZs24bHHHsP69esxa9YsrF27FgsWLEBBQQFG\njBjRo35rayvmzZuHOXPm4ODBgygsLMTSpUvh4+ODVatWAQBKS0uxcOFC3Hvvvfjss8+wZ88ePPzw\nwwgNDcWvf/3rvn+WZMCpa7chp64Vh2pbccTQdtoPptRQXyxODsf4UB1yjhrw0r+OIueoAfhZ4B+p\n98Vvl89A7AhafIoQMnQFaRQI0iiQrveTyhhjaDA7UNVqRW27DbXtNtS0WVHdZoOhw+Yx4YBczkOn\nU0Knc61kKwgMdrsTVpsAu12AxeY87aJZnkTwrR3gTO2QCQwcA3iRgRcZOMbAi4BCxkGlcAWBaqUc\nKqUM6m43lUrueqySQ62SQ6Vy1VOr5ZDJuDOu1Fta7/rFoqyw7rz6q/uZOuNO7mfbZ32mDGAMYJ3D\nTQwQGYPIABEMoggIjEllTpHBKYpwiMwVZDMGhzvgtgudATmTgnKbIMLmFHv8Ou0N5g7X//ca9yxN\nvSHnOOiUcviqZAhQK+CvkiNALYe/SuG+lyNA7XocqFbQF4F+ds7A/u2338bSpUtxzz33AADeeecd\nbNu2DevXr8err77ao/6nn34Kq9WKDRs2QKVSITk5GUVFRXj77belwP7dd99FdHQ01qxZAwBISkrC\njz/+iLfeeosC+yGGMQajxYHqViuq2myobLUgt66tx4wQ3Y0P0WGKTg3WYEbWF/lYV9yI5hZLj3oh\nwVrMn5uAhfMS6SdiQsiwxHEcwnyUCPNR9tgniAxGix31HXY0mO1ocN/Xmx1o6LCj0WJHh4yDptv7\npygw2OwCHA4BDqcIh0OA3SHC6RTQGX+KPAeAg3DOgXkGOByuW8e56rrwPAee5yDrvJfx7nsOTocD\nHA/41JVJZTzPu1YbpzjRa5yMocXmQIvNgcrWM3+2d1LKeAR0C/Y7vwz4q+Twd38R8Fcp4K+Ww1cp\ng0pGXwR646yBvd1ux+HDh/Hkk096lM+fPx/79u077THZ2dmYPXs2VCqVR/3nnnsO5eXliI2NRXZ2\nNubPn9/jnBs2bIAgCJDJ6Ge8gYox1896naMO7Q4BrTanx81kc6LJ4kC1e9TI0m00njFAFEQ4BQZB\nEF0/FwoitAwIsotQVrehqPYUis7YAg4Tx+lx1RUjMXG8HjL3SrOEEEI8yXgO4T4qhPuozljH6hTR\nYnWgyeJAs/u+xepEq92JdrsTbTYBbXYn2mxONHXY0Wp1QhBEOAURgsDgdIrubQbR/Z5+MUSRQRQZ\nnKfd5xpVbm3r+Qtv9y8DPM+7g/6f3TgOPO/6MtRZxnGdKSmue4679F8SZBwHtZyHSsZDLeehlst+\nts1DJeehlrn2KWUcVHIeSlnnjYNKxkPB85DLOMg5V1/IOA5yngPvfl6MuX556P4DgeD+TM/NOwaB\nASNHj4RDdH2+W50izA4BFocIs1NwPXaK6LC7XhOdn/k24Vy/8niyCyLqzXbUn+YC8dNR8jx0Shl8\nVXL4KuWYEumHG8fqe/VvDidnDeyNRiMEQUB4uOeUgWFhYairO/1PYXV1dYiJifEo6zy+rq4OsbGx\nMBgMPc4ZHh4Op9MJo9HYY1+na/68++zPZpjpzdtnZ93OFQzNFtcIuGbPLjD3m5jIXPUYABEA41z3\nIgCh243B4z/dH0rnZwCYyNw/ZTL3ras9HGPw6XDCp8OBgHY7lHYRDgCnvzTMtdDUlZfHY/7cBOjD\naLYbQgjpC2o5D71OBb3uzMF/dyJzBX0Whyvws7jzwC1OwTXo4xTRbnWgzexAu9l1b7E5YbE5YbU5\nYbELsNkEWDtTgBwCrDYnbHZXOtDPr7Hqzm53fUIolefzC21XdN79s+x8uL4k8OB5SPc81+2LAwfw\ncH1J4JmrnAMg4wGeuevCFbDzAHgAcg6QwRV08QDkcNdxb58uAalzkNrmvrVK5X3/zcNgMAAAynLO\n9Cncxcd96wytBTDYu7XTzgFWMGnbxgFWADYw2DhXHHExzA0dFNifRZ/PitOfP5d8tnR8v52bDBYC\nTKbe5wBeKomJiQAwoNs4GFG/9g/q1/4xHPqVB6AFoOXhmoVHCbiC6c5wVe21tpELkebtBvTKUP7b\nulhnzWMICQmBTCaTvsl1MhgMiIiIOO0xer2+x2h+5/F6vf6sdeRyOUJCQnr3DAghhBBCCCFnD+yV\nSiUmTZqE7du3e5RnZmYiIyPjtMfMmDEDe/bsgc1m86gfFRWF2NhYqU5mZmaPc06ZMoXy6wkhhBBC\nCLkAHGNnn2fp888/x5IlS7Bu3TpkZGTg3XffxUcffYT8/HyMGDECq1evxoEDB7Bjxw4Arukuk5KS\nMGfOHDz77LM4fvw4li5dihdeeAErV64EAJSVlSE1NRX33Xcf7r//fvzwww9YtmwZNm7ciOuvv77/\nnzUhhBBCCCFDzDlz7BcvXozGxka8/PLLqK2tRVpaGrZu3SrNYV9XV4eSkhKpvp+fHzIzM7Fs2TJM\nnjwZQUFBeOKJJ6SgHgDi4uKwdetWrFy5EuvXr0dUVBT+8pe/UFBPCCGEEELIBTrniD0hhBBCCCFk\n4BvQk4CvW7cO8fHx0Gg0mDx5Mvbu3evtJnnN7t27sWjRIkRHR4PneWzYsKFHnRdeeAFRUVHQarWY\nO3cuCgoKPPbbbDYsX74coaGh0Ol0+OUvf4nq6mqPOs3NzViyZAkCAgIQEBCAO++8s8fV5xUVFbju\nuuug0+kQGhqKFStWwOE49xRZA9Frr72GKVOmwN/fH2FhYVi0aBHy8/N71KO+7Z21a9di/Pjx8Pf3\nh7+/PzIyMrB161aPOtSnF++1114Dz/NYvny5Rzn1be+88MIL4Hne4xYZGdmjDvVp79XW1uKuu+5C\nWFgYNBoNUlJSsHu359TV1Le9ExcX1+P1yvM8rr32WgCuKaapT3vP6XTi6aefRkJCAjQaDRISEvDc\nc89BEDwn6BzwfcsGqI0bNzKFQsE++OADVlRUxJYvX850Oh2rqKjwdtO8YuvWreyZZ55hX3zxBdNq\ntWzDhg0e+19//XXm6+vLNm/ezI4dO8YWL17MIiMjWVtbm1TnwQcfZJGRkWzHjh3s8OHDbM6cOWzC\nhAlMEASpztVXX81SU1PZ/v37WXZ2NktJSWHXXXedtN/pdLLU1FQ2d+5clpOTwzIzM1lkZCRbvnx5\n/3dCP7jqqqvYxx9/zPLz81leXh67/vrrmV6vZ01NTVId6tve+9///se2bdvGTp06xYqLi9kzzzzD\nFAoFy83NZYxRn/aF7OxsFh8fz8aPH+/xfKhve+/5559nY8eOZQaDQboZjUZpP/XphWlubmbx8fHs\nrrvuYgcOHGBlZWUsKyuLFRYWSnWob3vPaDR6vFZzcnIYz/Psk08+YYxRn16oF198kQUFBbFvvvmG\nlZeXs6+++ooFBQWxl156SaozGPp2wAb2U6dOZffff79HWWJiIlu9erWXWjRw6HQ6j8BeFEWm1+vZ\nq6++KpVZLBbm6+vL3nvvPcYYYy0tLUypVLLPPvtMqlNZWcl4nmfffvstY4yxgoICxnEc27dvn1Rn\n7969jOM4duLECcaY6wsGz/OsqqpKqvPPf/6TqdVqjxf2YNXe3s5kMhn75ptvGGPUt30pKCiIvf/+\n+9SnfaClpYWNHDmSff/992zOnDnSmz317YV5/vnnWWpq6mn3UZ9euNWrV7NZs2adcT/1bd94+eWX\nWWBgILNardSnF+Haa69ld999t0fZnXfeya699lrG2OB5vQ7IVBy73Y7Dhw9j/vz5HuXz58/Hvn37\nvNSqgau0tBQGg8Gjv9RqNS677DKpvw4dOgSHw+FRJzo6GmPHjkV2djYAIDs7GzqdDjNmzJDqZGRk\nwMfHRzpPdnY2kpOTERUVJdWZP38+bDYbDh061K/P81JobW2FKIoIDAwEQH3bFwRBwMaNG2G1WnHZ\nZZdRn/aB+++/HzfddBMuv/xyabVngF6vF6OkpARRUVFISEjArbfeitLSUgDUpxfjyy+/xNSpU3Hz\nzTcjPDwc6enpWLt2rbSf+vbiMcbw4Ycf4o477oBKpaI+vQgLFixAVlYWjh8/DgAoKCjAzp07cc01\n1wAYPK/XPl95ti8YjUYIgoDw8HCP8rCwsB4LWxFIfXK6/qqpqZHqyGQyBAcHe9QJDw+Xjq+rq0No\naKjHfo7jPPq9rq6ux7/TuZDZUPh/s2LFCqSnp0t/cNS3Fy4vLw8zZsyAzWaDRqPB559/jqSkJOmN\ni/r0wvztb39DSUkJPvvsMwCeq33T6/XCTJ8+HRs2bMCYMWNgMBjw8ssvIyMjA/n5+dSnF6GkpATr\n1q3DqlWr8PTTTyMnJ0e6HmTZsmXUt30gMzMTZWVluO+++wDQe8DFePjhh1FVVYWxY8dCLpfD6XTi\n2WefxYMPPghg8PTtgAzsSd/p/qF/OuwCJkW6kGMGg1WrVmHfvn3Yu3fvOfsNoL49lzFjxuDo0aMw\nmUz497//jVtuuQU7d+486zHUp2d3/PhxPPPMM9i7d6+0mB9zpVSe81jq2zO7+uqrpcepqamYMWMG\n4uPjsWHDBkybNu2Mx1Gfnp0oipg6dSpeeeUVAMD48eNRXFyMtWvXYtmyZWc9lvr2/Pztb3/D1KlT\nkZaWds661Kdn98477+Cjjz7Cxo0bkZKSgpycHKxYsQJxcXH4zW9+c9ZjB1LfDshUnM5vJQaDwaPc\nYDAgIiLCS60auPR6PQCctr869+n1egiCgMbGxrPWaWho8NjPGEN9fb1HnZ//O52/sHTWGYxWrlyJ\nTZs2ISsrC3FxcVI59e2FUygUSEhIQHp6Ol599VVMnz4da9eulf6GqU97Lzs7G0ajESkpKVAoFFAo\nFNi9ezfWrVsHpVKJkJAQANS3F0ur1SIlJQUnT56k1+tFiIyMRHJyskfZmDFjUFFRAYDeXy9WfX09\nvvrqK2m0HqA+vRivvPIKnn76aSxevBgpKSm44447sGrVKrz22msABk/fDsjAXqlUYtKkSdi+fbtH\neWZmJjIyMrzUqoErPj4eer3eo7+sViv27t0r9dekSZOgUCg86lRVVaGoqEiqM2PGDLS3t0t5YIAr\nkOjo6JDqZGRkoLCw0GPqpszMTKhUKkyaNKlfn2d/WbFihRTUjx492mMf9W3fEQQBoihSn16E66+/\nHseOHcORI0dw5MgR5ObmYvLkybj11luRm5uLxMRE6ts+YLVaUVhYiIiICHq9XoSZM2eiqKjIo+zE\niRPS4An17cX5+OOPoVarceutt0pl1KcXjjEGnvcMi3mel0bOB03fnvXSWi/atGkTUyqV7IMPPmAF\nBQXs0UcfZb6+vsN2usv29naWk5PDcnJymFarZb///e9ZTk6O1B9vvPEG8/f3Z5s3b2Z5eXns5ptv\nZlFRUay9vV06x0MPPcSio6M9pmBKT09noihKdRYsWMDS0tJYdnY227dvH0tNTWWLFi2S9guCwNLS\n0tgVV1whTcEUFRXFHn300UvXGX3o4YcfZn5+fiwrK4vV1tZKt+79Rn3be0899RTbs2cPKy0tZUeP\nHmW/+93vGM/zbPv27Ywx6tO+dPnll7NHHnlE2qa+7b3HH3+c7dq1i5WUlLD9+/eza665hvn7+9P7\n60U6cOAAUygU7JVXXmHFxcXs888/Z/7+/mzdunVSHerbCyOKIktMTOwxeyBj1KcX6r777mPR0dFs\ny5YtrLS0lG3evJmFhoayJ554QqozGPp2wAb2jDG2bt06FhcXx1QqFZs8eTLbs2ePt5vkNTt37mQc\nxzGO4xjP89LjpUuXSnVeeOEFFhERwdRqNZszZw7Lz8/3OIfNZmPLly9nwcHBTKvVskWLFnlMpcSY\na97hO+64g/n5+TE/Pz+2ZMkSZjKZPOpUVFSwa6+9lmm1WhYcHMxWrFjB7HZ7/z35fvTz/uy8vfji\nix71qG975+6772axsbFMpVKxsLAwNm/ePCmo70R92je6T3fZifq2d2655RYWGRnJlEoli4qKYjfe\neKPHXOuMUZ9eqC1btrDx48cztVrNkpKS2F/+8pcedahvey8rK4vxPM8OHDhw2v3Up73X3t7OHn/8\ncRYXF8c0Gg1LSEhgzzzzDLPZbB71BnrfcowNoSsfCCGEEEIIGaYGZI49IYQQQgghpHcosCeEEEII\nIWQIoMCeEEIIIYSQIYACe0IIIYQQQoYACuwJIYQQQggZAiiwJ4QQQgghZAigwJ4QQgghhJAhgAJ7\nQgghhBBChgAK7AkhhBBCCBkC/h9y/thwpWL2tAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from price import *\n", + "import matplotlib.pyplot as plt\n", + "player1, player2 = MakePlayers(path='../code')\n", + "MakePrice1(player1, player2)\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows the distribution of prices for these\n", "showcases. The most common value for both showcases is around \\$28,000,\n", "but the first showcase has a second mode near \\$50,000, and the second\n", "showcase is occasionally worth more than \\$70,000.\n", @@ -76,25 +362,32 @@ "These distributions are based on actual data, but they have been\n", "smoothed by Gaussian kernel density estimation (KDE). Before we go on, I\n", "want to take a detour to talk about probability density functions and\n", - "KDE.\n", - "\n", - "Probability density functions\n", - "-----------------------------\n", + "KDE." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Probability density functions\n", "\n", "So far we have been working with probability mass functions, or PMFs. A\n", "PMF is a map from each possible value to its probability. In my\n", - "implementation, a Pmf object provides a method named Prob\n", + "implementation, a Pmf object provides a method named Prob\n", "that takes a value and returns a probability, also known as a\n", - "**probability mass**.\n", + "**probability mass**.\n", "\n", - "A **probability density function**, or PDF, is the\n", + "A **probability density function**, or PDF, is the\n", "continuous version of a PMF, where the possible values make up a\n", "continuous range rather than a discrete set.\n", "\n", "In mathematical notation, PDFs are usually written as functions; for\n", "example, here is the PDF of a Gaussian distribution with mean 0 and\n", - "standard deviation 1: $$f(x) = \\frac{1}{\\sqrt{2 \\pi}} \\exp(-x^2/2)$$ For\n", - "a given value of $x$, this function computes a probability density. A\n", + "standard deviation 1: \n", + "\n", + "$$f(x) = \\frac{1}{\\sqrt{2 \\pi}} \\exp(-x^2/2)$$\n", + "\n", + "For a given value of $x$, this function computes a probability density. A\n", "density is similar to a probability mass in the sense that a higher\n", "density indicates that a value is more likely.\n", "\n", @@ -106,57 +399,88 @@ "that.\n", "\n", "Instead we primarily use probability densities as part of a likelihood\n", - "function. We will see an example soon.\n", - "\n", - "Representing PDFs\n", - "-----------------\n", + "function. We will see an example soon." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Representing PDFs\n", "\n", - "To represent PDFs in Python, thinkbayes.py provides a class\n", - "named Pdf. Pdf is an **abstract\n", - "type**, which means that it defines the interface a Pdf is\n", + "To represent PDFs in Python, `thinkbayes.py` provides a class\n", + "named `Pdf`. `Pdf` is an **abstract type**, which means that it defines the interface a Pdf is\n", "supposed to have, but does not provide a complete implementation. The\n", - "Pdf interface includes two methods, Density\n", - "and MakePmf:\n", - "\n", - " class Pdf(object):\n", - "\n", - " def Density(self, x):\n", - " raise UnimplementedMethodException()\n", + "`Pdf` interface includes two methods, `Density`\n", + "and `MakePmf`:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Pdf(object):\n", "\n", - " def MakePmf(self, xs):\n", - " pmf = Pmf()\n", - " for x in xs:\n", - " pmf.Set(x, self.Density(x))\n", - " pmf.Normalize()\n", - " return pmf\n", + " def Density(self, x):\n", + " raise UnimplementedMethodException()\n", "\n", - "Density takes a value, x, and returns the\n", - "corresponding density. MakePmf makes a discrete\n", + " def MakePmf(self, xs):\n", + " pmf = Pmf()\n", + " for x in xs:\n", + " pmf.Set(x, self.Density(x))\n", + " pmf.Normalize()\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Density` takes a value, `x`, and returns the\n", + "corresponding density. `MakePmf` makes a discrete\n", "approximation to the PDF.\n", "\n", - "Pdf provides an implementation of MakePmf, but\n", - "not Density, which has to be provided by a child class.\n", + "`Pdf` provides an implementation of `MakePmf`, but\n", + "not `Density`, which has to be provided by a child class.\n", "\n", - "A **concrete type** is a child class that extends an\n", + "A **concrete type** is a child class that extends an\n", "abstract type and provides an implementation of the missing methods. For\n", - "example, GaussianPdf extends Pdf and provides\n", - "Density:\n", - "\n", - " class GaussianPdf(Pdf):\n", + "example, `GaussianPdf` extends `Pdf` and provides\n", + "`Density`:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class GaussianPdf(Pdf):\n", "\n", - " def __init__(self, mu, sigma):\n", - " self.mu = mu\n", - " self.sigma = sigma\n", - " \n", - " def Density(self, x):\n", - " return scipy.stats.norm.pdf(x, self.mu, self.sigma)\n", + " def __init__(self, mu, sigma):\n", + " self.mu = mu\n", + " self.sigma = sigma\n", "\n", - "`__init__` takes mu and sigma, which are the\n", + " def Density(self, x):\n", + " return scipy.stats.norm.pdf(x, self.mu, self.sigma)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`__init__` takes `mu` and `sigma`, which are the\n", "mean and standard deviation of the distribution, and stores them as\n", "attributes.\n", "\n", - "Density uses a function from scipy.stats to\n", - "evaluate the Gaussian PDF. The function is called norm.pdf\n", + "`Density` uses a function from `scipy.stats` to\n", + "evaluate the Gaussian PDF. The function is called `norm.pdf`\n", "because the Gaussian distribution is also called the “normal”\n", "distribution.\n", "\n", @@ -168,12 +492,15 @@ "Gaussian or any other simple mathematical function. In that case we can\n", "use a sample to estimate the PDF of the whole population.\n", "\n", - "For example, in *The Price Is Right* data, we have 313\n", + "For example, in *The Price Is Right* data, we have 313\n", "prices for the first showcase. We can think of these values as a sample\n", "from the population of all possible showcase prices.\n", "\n", "This sample includes the following values (in order):\n", - "$$28800, 28868, 28941, 28957, 28958$$ In the sample, no values appear\n", + "\n", + "$$28800, 28868, 28941, 28957, 28958$$ \n", + "\n", + "In the sample, no values appear\n", "between 28801 and 28867, but there is no reason to think that these\n", "values are impossible. Based on our background information, we expect\n", "all values in this range to be equally likely. In other words, we expect\n", @@ -183,55 +510,105 @@ "finds an appropriately smooth PDF that fits the data. You can read\n", "details at .\n", "\n", - "scipy provides an implementation of KDE and\n", - "thinkbayes provides a class called\n", - "EstimatedPdf that uses it:\n", - "\n", - " class EstimatedPdf(Pdf):\n", - "\n", - " def __init__(self, sample):\n", - " self.kde = scipy.stats.gaussian_kde(sample)\n", + "`scipy` provides an implementation of KDE and\n", + "`thinkbayes` provides a class called\n", + "`EstimatedPdf` that uses it:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class EstimatedPdf(Pdf):\n", "\n", - " def Density(self, x):\n", - " return self.kde.evaluate(x)\n", + " def __init__(self, sample):\n", + " self.kde = scipy.stats.gaussian_kde(sample)\n", "\n", + " def Density(self, x):\n", + " return self.kde.evaluate(x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "`__init__` takes a sample and computes a kernel density estimate. The\n", - "result is a `gaussian_kde` object that provides an evaluate\n", + "result is a `gaussian_kde` object that provides an `evaluate`\n", "method.\n", "\n", - "Density takes a value, calls `gaussian_kde.evaluate`, and\n", + "`Density` takes a value, calls `gaussian_kde.evaluate`, and\n", "returns the resulting density.\n", "\n", "Finally, here’s an outline of the code I used to generate\n", - "Figure [fig.price1]:\n", - "\n", - " prices = ReadData()\n", - " pdf = thinkbayes.EstimatedPdf(prices)\n", - "\n", - " low, high = 0, 75000\n", - " n = 101\n", - " xs = numpy.linspace(low, high, n) \n", - " pmf = pdf.MakePmf(xs)\n", - "\n", - "pdf is a Pdf object, estimated by KDE.\n", - "pmf is a Pmf object that approximates the Pdf by evaluating\n", + "Figure [fig.price1]:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Warning: Brewer ran out of colors.\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvYAAAEWCAYAAAANTdM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl0VGWeP/537amq7GtVVhKyk7AlBBNBQCGK7TJIi9/R\nUWGcaaUZfyw/l8O0jvbAdA/Hbo7aX0C7x5/SLT3SYzPaKiqxkT0KIQQDCWvIRlKVPSFbpZb7+yNw\nSXUCSSCVW1V5v87hUM+t51a96zHET26e57kyQRAEEBERERGRR5NLHYCIiIiIiG4fC3siIiIiIi/A\nwp6IiIiIyAuwsCciIiIi8gIs7ImIiIiIvAALeyIiIiIiL8DCnoiIiIjIC4y4sN+6dSvi4+Oh1WqR\nnZ2NQ4cO3bR/aWkp5s2bB51Oh+joaGzYsMHp+X379kEulw/6c+7cuVv7JEREREREE5hyJJ127tyJ\nNWvWYNu2bZgzZw62bNmCxYsXo6ysDDExMYP6d3R0YNGiRZg/fz6KiopQXl6OFStWQK/XY926dU59\ny8rKEBwcLLZDQ0Nv8yMREREREU08spHceXb27NmYPn063n33XfFYcnIyfvzjH+MXv/jFoP7btm3D\n+vXrYTabodFoAAD/8R//gW3btqG2thZA/xX7u+++G42NjQgJCRmrz0NERERENCENOxWnr68PxcXF\nyM/Pdzqen5+PI0eODHlOYWEh5s6dKxb11/rX1dWhqqrKqW92djYiIyOxcOFC7Nu37xY+AhERERER\nDVvYNzU1wW63IyIiwul4eHg4TCbTkOeYTKZB/a+1r50TGRmJd955B7t27cKuXbuQkpKCe+65Z9i5\n+0RERERENNiI5tiPlkwmG7ZPcnIykpOTxfYdd9yByspKvPHGG5gzZ454vL293RURiYiIiIjGRUBA\nwLi8z7BX7ENDQ6FQKGA2m52Om81mGI3GIc8xGAyDruZfO99gMNzwvXJycnD+/PlhQxMRERERkbNh\nC3u1Wo2srCzs2bPH6XhBQQHy8vKGPCc3NxcHDx6ExWJx6h8VFYW4uLgbvldJSQkiIyNHmp2IiIiI\niK4a0VScdevW4cknn0ROTg7y8vLwzjvvwGQy4bnnngMArF+/HseOHcM333wDAHj88cfx85//HMuX\nL8crr7yCs2fPYtOmTXj99dfF13zzzTcRHx+P9PR09PX14cMPP8Snn36KXbt23TDHeP0aYyIoKioC\n0L94mcYOx9U1OK6uwXF1DY6ra3BcXYdj6xpSTCcfUWG/bNkyNDc3Y+PGjaivr0dmZiZ2794t7mFv\nMplQUVEh9vf390dBQQFWrVqF7OxsBAcH44UXXsDatWvFPlarFS+++CJqa2uh1WqRkZGB3bt34777\n7hvjj0hERERE5P1GvHh25cqVWLly5ZDPvf/++4OOZWRkYP/+/Td8vRdffBEvvvjiSN+eiIiIiIhu\nYtg59kRERERE5P5Y2BMREREReQEW9kREREREXoCFPRERERGRF2BhT0RERETkBVjYExERERF5ARb2\nRERERERegIU9EREREZEXYGFPREREROQFWNgTEREREXkBFvZERERERF6AhT0RERERkRdgYU9ERERE\n5AWUUgcgoonBautDQ+tlmFpqIAgCYsInIywoEnIZry8QERGNBRb2RDTmBEFAQ1sdTM3VqG+uhqml\nFs3tJjgEh1M/rUaPSYZkxBmSEReRDH99oESJiYiIPB8LeyIaM4Ig4GJdGY6U7oG5tXbY/j2WLpRX\nnUB51QkAQIh/BGalzUdmQo6roxIREXkdFvZEdNtGUtDLIENwQDgMwTFwOByoMp9Dd2+nU5/mDjO+\n+n4nWjoaMHfa/ZymQ0RENAos7Inolt2soFcp1EiITIMhJAaG4BhEBEdDo/IRn3cIDjS11aPSdA6V\npnOobayA3W4DABwt/xadPe24L+cxKBT8NkVERDQS/D8mEd2ShtbL2HPsY9Q3VzsdVypUmJ6Yh5y0\n+dBr/W94vlwmR3hQFMKDopCTtgB9Vgu+KNyBC5dPAwDKKovR1duJh+c87dLPQURE5C1Y2BPRqFht\nVnx3ugBHz+yDw2EXj4+0oL8RtUqDh+c8jW+O/y9OXigEAFSZzuGjb7ZgcmA2fFT6MfsMRERE3oiF\nPRGNWE3DRew5+j9oudIoHlPIlZielIuctLvhewsF/UByuQKLspfCTxuAQ6VfAQAa2upgbvwcOQn3\n3tZrExEReTsW9kQ0LIu1F/tLPhevpF8THZaAe3MeRbB/+Ji9l0wmQ27GIvjqArDn6P/AITjQ09eJ\nIxc+R0ZmBkIDDGP2XkRERN6EhT0R3VRF3RnsOfonXOlpF4+pVT6YN+1HmJp4h8t2rslMyIHexx9/\nObwdQDesNgu+/O4jPLHoecjlCpe8JxERkSfjXnJENCSb3Yq9xZ/iz/t/51TUT46aghWLX8T0pDyX\nb0eZEJmKx+5eCZm8/31MLTUoOnvApe9JRETkqXjFnogGaWo34YsjO9DQVice02l8cXfW3yE1djpk\nMtm4ZTGGxCIpYgbO1R8HABwu/RqJURkI9g8btwxERESegIU9EYkEQcDJC4X49sRfYLNbxeMJkWm4\nb/Zj0Pv4SZJrcvhUmNouwYZe2OxW7Dn2Jyy7eyVvYEVERDQA/69IRACAbksnPjn4PgqK/iwW9QqF\nEvdkLcEjdz0jWVEP9O95PzXmLrGQr2mowMkL30mWh4iIyB2xsCci1DfX4PdfbhZvDgUAYYFGPJm/\nBjOT54zr1JsbCdCFYFbaArF9oORzdHS1SpiIiIjIvbCwJ5rgyqtO4KO/bnFaIDszeQ6eWLQaYYFG\nCZMNljtlkbi1Zp/NgoKiP0MQBIlTERERuQcW9kQTlENw4ODJ3fj8yIfi1BuNWotH7vpH3JO1BCql\nSuKEg6mUKtyXswwy9P8GoaKuHGWVxyVORURE5B5Y2BNNQBZrLz49+AG+K/ureCzYPxz/sGg1JkdN\nkTDZ8KLC4jEjeY7Y/rb4U3T1dEiYiIiIyD2wsCeaYNo6m/Hf3/xfp/n08cZU/MOi/8djtpCcO3Ux\nAvTBAICevm7sLf5U4kRERETSY2FPNIHUNVXiwz1vobGtXjyWnToPj9z1j9CotRImGx21SoN7cx4V\n22eqS9DQelnCRERERNJjYU80QdQ31+Djfb9Dj6ULAKCQK7F49v/BghkPQS5XSJxu9OIMyUiOyRTb\n35XtlTANERGR9FjYE00AjW31+PO+38Ji7QXQfxfZx+5+DhkJsyROdnvumLJIfHyu+iSaO8wSpiEi\nIpIWC3siL9fS0YD/+fZd9PR1AwB81Do8uuBZRIXFS5zs9kUERSEhMg0AIEDA96d51Z6IiCYuFvZE\nXqytsxl/2vsOunqvAAA0Kh/8eP5PEB4UKXGysXNH+kLxcXlVMdo6myVMQ0REJJ0RFfZbt25FfHw8\ntFotsrOzcejQoZv2Ly0txbx586DT6RAdHY0NGzbcsO+hQ4egVCqRmZl5wz5ENHodXW3407fvijee\nUinUWDrvn2AMiZE42diKCpuE2IgkAP178x8t/1biRERERNIYtrDfuXMn1qxZg1deeQUlJSXIy8vD\n4sWLUVNTM2T/jo4OLFq0CEajEUVFRXjrrbfwxhtvYPPmzYP6tra24qmnnsLChQvd4pb1RN6iq6cD\n/7PvXbRfvXqtUCix5K4VXjH9Zii5U65ftT9VcQwdXW0SpiEiIpLGsIX95s2bsWLFCjzzzDNISUnB\n22+/DaPRiG3btg3Zf8eOHejt7cX27duRnp6OpUuX4uWXXx6ysH/mmWewYsUK5Obm8rbwRGPEauvD\nx/t+h5aOBgCAXK7Aw3c+jThDssTJXCcmfDKiQicBAOwOG4rO7pM0DxERkRRuWtj39fWhuLgY+fn5\nTsfz8/Nx5MiRIc8pLCzE3LlzodFonPrX1dWhqqpKPLZ161Y0NjbilVdeYVFPNIb2l3yOhrY6AIBM\nJscDuU9gclS6xKlcSyaT4Y4BV+1/uPA970ZLREQTzk0L+6amJtjtdkRERDgdDw8Ph8lkGvIck8k0\nqP+19rVzSktL8e///u/48MMPOQWHaAxV1JXjxPnDYnth1hKkxE6TMNH4iTemIiIoGgBgtfeh6OwB\niRMRERGNL+VYv+BwhbrFYsFjjz2GX/3qV4iLixvVaxcVFd1ONBoCx9Q1pBhXi60HB87+L/qsPQCA\niIA4WNtUXvXfeLjPEqSMxqWucwCAgye+hqo3EGqlz3hE82je9DXiTjiursFxdR2O7dhKSkoa9/e8\n6RX70NBQKBQKmM3ON30xm80wGo1DnmMwGAZdzb92vsFgQH19Pc6cOYMVK1ZApVJBpVJhw4YNOH36\nNFQqFb755pvb+TxEE5IgCCitOSQW9RqVFlNj5ky434gZAibB1ycQAGC321DZVCZxIiIiovFz0yv2\narUaWVlZ2LNnD5YuXSoeLygowKOPPjrkObm5uXj55ZdhsVjEefYFBQWIiopCXFwcbDYbTp065XTO\nli1bUFBQgE8++eSmV/Gzs7NH/MHo5q79VM4xHVtSjesPF7/HlQtN0Ol1AICl8/4ZCZGp45rBlUYz\nrvowBb4o3AEAaLJUY+m0p6BR8ar9UPh9wDU4rq7BcXUdjq1rtLe3j/t7Drsrzrp16/DBBx/gvffe\nQ3l5OVavXg2TyYTnnnsOALB+/XosXHh90drjjz8OnU6H5cuX4/Tp09i1axc2bdqEdevWAQCUSiXS\n09Od/oSFhUGj0SA9PR16vd5FH5XIO7VeacTe4k/E9oykO72qqB+t1NhpCPQNAQBY+nrww8XvJU5E\nREQ0PoadY79s2TI0Nzdj48aNqK+vR2ZmJnbv3o2YmP6b3JhMJlRUVIj9/f39UVBQgFWrViE7OxvB\nwcF44YUXsHbt2hu+h0wmm3BTBojGgsNhxxeFf4TV1gcACPYPx7zpD0icSlpyuQI5aQuw59jHAICS\n84eRlTIXchlvtE1ERN5tRItnV65ciZUrVw753Pvvvz/oWEZGBvbv3z/iEK+99hpee+21Efcnon6F\np79BfXM1gP6C9oHcJ6BSqiVOJb20STOxv+RzWKy9aOtsRmX9uQn9WwwiIpoYeAmLyEPVN1fju9PX\nF5vPybwXEcHREiZyH2qlBhkJOWK7ZMAWoERERN6KhT2RBxIEAd8WfwqH4AAARIclYFbqfGlDuZnp\niXni44q6crR1NkuYhoiIyPVY2BN5oErTWVxuqgQAKORKLL7j/0AuV0gbys0E+4dhkiEFACBAwMkL\nhRInIiIici0W9kQeRhAEHC79WmxPnTxb3AWGnM1IvlN8XFpxFFabVcI0RERErsXCnsjDVNSViQtm\nFQolZqffLXEi95VgTIW/PggA0GPpwtnqEokTERERuQ4LeyIP0n+1fo/Ynp6YBz9doISJ3JtcrnCa\na3+Ci2iJiMiLsbAn8iAXLp+CubUWAKBSqDE7bYHEidxf5uQcKBT9O/uaWmrE33YQERF5Gxb2RB7C\nITic5tZPT8qDXusvYSLPoNP4IjVmutg+cY5X7YmIyDuxsCfyEOdrStHYVg8AUCnVyOHV+hEbuIj2\nTE0Jui2dEqYhIiJyDRb2RB7A4bDj8KnrV+tnJs+FzsdXwkSexRgSC0NwDADAbreh9OJRiRMRERGN\nPRb2RB7gTPVJNLebAQBqlQ9mpc6TOJHnmZF0/ap9yYUjcDjsEqYhIiIaeyzsidycw2FH4anrO+Fk\nJc+FVqOXMJFnSomdLo5bR1crKurPSJyIiIhobLGwJ3Jz5VUn0HKlEQCgUfkgm1frb4lKqUJmQo7Y\nLuHWl0RE5GVY2BO5MYfDjiMDrtZnp86Hj1orYSLPNj0xDzLIAACX6s+irbNZ4kRERERjh4U9kRur\nNJ0Ti08ftQ5ZKXMlTuTZAnyDER+ZKrZPXiiUMA0REdHYYmFP5MZKK67v3pKZkAONykfCNN5hWmKu\n+PhUxTHY7TYJ0xAREY0dFvZEbqq7txMXLp8W2xkJsyRM4z0SjKnw0wUCALotnThXWypxIiIiorHB\nwp7ITZVVHhe3ZIwMiUNogEHiRN5BLldg6uTZYpvTcYiIyFuwsCdyQ4Ig4NSlY2I7Y8BuLnT7MhNy\nIJf1f/urabiI5g6zxImIiIhuHwt7IjdkaqlFY1s9AEClUCM1brrEibyLny4Qk6PSxfYPF76XMA0R\nEdHYYGFP5IZODVg0mxwzlYtmXcBpEe2lY7DarBKmISIiun0s7IncjNVmRXlVsdjmolnXiDMkI8A3\nBADQ29eNczUnJU5ERER0e1jYE7mZ87WlsFh7AQCBviGICZ8scSLvJJfJMW3AItoSLqIlIiIPx8Ke\nyM0MnIaTkZADmUwmYRrvlhE/C3K5AgBQ11SJhtY6iRMRERHdOhb2RG6kvbMFVebzAAAZZJgyKVvi\nRN5Nr/VHcnSm2D558TsJ0xAREd0eFvZEbmTgFpeTjCnw1wdKmGZiGLiItqzyOPqsFgnTEBER3ToW\n9kRuwiE4cKriemGfyb3rx0VM+GQE+4UBAPqsvThTfULiRERERLeGhT2Rm6gxX0BHdysAQKvWOe2z\nTq4jk8mcrtpzES0REXkqFvZEbqJ0wKLZtElZUCpUEqaZWKbEZ0OhUAIAzC21qG+uljgRERHR6LGw\nJ3IDPZZunKstFduZ3Lt+XGk1eqTGXL+774nzhyVMQ0REdGtY2BO5gTPVJbDbbQCAiKBohAdFSZxo\n4pmRfKf4+Ex1Cbp7OyVMQ0RENHos7IncwLmaH8THvNOsNIwhsTCGxAIA7HYbt74kIiKPw8KeSGI9\nlm7UNlwU20kD9lWn8TUj6fpV+5MXCuFw2CVMQ0RENDos7Ikkdqm+HA7BAaD/qrGfLkDiRBNXSuw0\n6DS+AIAr3W24cPm0xImIiIhGjoU9kcQu1F4vHidHTZEwCSkVKkxNvENsF587JGEaIiKi0WFhTyQh\nm92KS/VnxHYiC3vJTZucC7ms/1tjTcNFNLbVS5yIiIhoZFjYE0mo2nwRfTYLACDQNwShAQaJE5G/\nPtBpncMJXrUnIiIPMeLCfuvWrYiPj4dWq0V2djYOHbr5/+xKS0sxb9486HQ6REdHY8OGDU7P79+/\nH3l5eQgNDYVOp0NaWhp+/etf39qnIPJQFy6fEh8nRmVAJpNJmIauGbj1ZVllMXos3RKmISIiGhnl\nSDrt3LkTa9aswbZt2zBnzhxs2bIFixcvRllZGWJiYgb17+jowKJFizB//nwUFRWhvLwcK1asgF6v\nx7p16wAAfn5+WLNmDTIzM6HT6XDo0CE8++yz8PHxwapVq8b2UxK5IYfgwMXLZWJ7clS6hGlooOiw\nBIQFRqKxrQ5Wex9OXTqGWanzpI5FRER0UyO6Yr9582asWLECzzzzDFJSUvD222/DaDRi27ZtQ/bf\nsWMHent7sX37dqSnp2Pp0qV4+eWXsXnzZrHPzJkzsWzZMqSlpSEuLg5PPPEE8vPzceTIkbH5ZERu\nztxSi86edgCAVq1DdFi8xInoGplMhpkDrtqXnD8s7lxERETkroYt7Pv6+lBcXIz8/Hyn4zcrwgsL\nCzF37lxoNBqn/nV1daiqqhrynBMnTqCwsBCLFi0aTX4ijzVwK8WEyHTI5QoJ09DfSoubCR+1DgDQ\n1tmMS3VnhjmDiIhIWsNOxWlqaoLdbkdERITT8fDwcJhMpiHPMZlMiI2NdTp27XyTyYS4uDjxeHR0\nNJqammC1WvHqq69i+fLlN8xSVFQ0XFwaJY6pa4xkXI+dPYjunv652/YuJf9bjMB4j1GgyoiK1lIA\nwFeHd2F2gnfOtefXnmtwXF2D4+o6HNuxlZSUNO7vOaI59qM1mgWAhw8fRmdnJwoLC/Hiiy8iPDwc\nP/3pT10Ri8htdFs6cKWnFQAglysQ5hctcSIaSlxIKioaSwEBaOq4jM7eNvj6BEodi4iIaEjDFvah\noaFQKBQwm81Ox81mM4xG45DnGAyGQVfzr51vMDhv53ft6v2UKVNgNpvxxhtv3LCwz87OHi4ujdC1\nn8o5pmNrpONadGY/dPr+aR4JkWm4Y3auy7N5Mim/XpvtleK0Kbv2CrKzFo57Blfh9wHX4Li6BsfV\ndTi2rtHe3j7u7znsHHu1Wo2srCzs2bPH6XhBQQHy8vKGPCc3NxcHDx6ExWJx6h8VFeU0Dedv2e12\nOBxcoEbeb+D8+sSoDAmT0HBmJM8RH5dWHEWPpUvCNERERDc2ol1x1q1bhw8++ADvvfceysvLsXr1\naphMJjz33HMAgPXr12PhwutXsR5//HHodDosX74cp0+fxq5du7Bp0yZxq0sA+M1vfoMvvvgC58+f\nx/nz5/Hee+/h17/+NZ588skx/ohE7qXH0oXaxksAABlkSOQ2l24tLiIJYYGRAACrrQ8nznPnLiIi\nck8jmmO/bNkyNDc3Y+PGjaivr0dmZiZ2794t7mFvMplQUVEh9vf390dBQQFWrVqF7OxsBAcH44UX\nXsDatWvFPg6HAy+//DIqKyuhVCqRmJiITZs24dlnnx3jj0jkXirqyiFc3TrRGBILvdZf4kR0MzKZ\nDDlpC/BF4Q4AQPG5g8hOvQtqpWaYM4mIiMbXiBfPrly5EitXrhzyuffff3/QsYyMDOzfv/+Gr7d6\n9WqsXr16pG9P5DWc7jYbPUXCJDRSqbHTcOiHL9He1YIeSxdOVRzDzAFTdIiIiNzBiKbiENHYsNqs\nuFR/VmwnRnN+vSeQyxXIHnDn2WNn9sHusEuYiIiIaDAW9kTjqNp8HlZbHwAg2C8MIf4Rw5xB7iIz\nIQdajR4A0NHVijNVJyRORERE5IyFPdE4ctoNh1frPYpKqUZW8lyxfezMPgiCIGEiIiIiZyzsicaJ\nQ3DgYl2Z2E6M4vx6TzM9KQ8qpRoA0NhWj4q6cokTERERXcfCnmicNLbVo6unAwCg1ehhDImVOBGN\nllajx7TJ128m9n35XgnTEBEROWNhTzROqkznxMeTDMmQyxUSpqFblZVyl/jf7nLjJVy+ek8CIiIi\nqbGwJxonlQMK+zhDsoRJ6Hb46wORPilLbH9f/q2EaYiIiK5jYU80Dqw2K2obr9/ELS6Chb0ny0md\nLz6+ePk0GtvqpQtDRER0FQt7onFwuekS7HYbACDYPxz++kCJE9HtCAmIQNKAXY2Ole+TLgwREdFV\nLOyJxsHfzq8nz5eTtkB8XF59Au2dLRKmISIiYmFPNC4qB9xtlvPrvUNk6CTEhE8GADgcdhw5XSBx\nIiIimuhY2BO5WFdPBxra6gAAcrlCLAbJ8+Vl5IuPT18qQnOHWcI0REQ00bGwJ3KxKvMF8XFkSBw0\nKh8J09BYio1IFH8DIwgOHC7dI3EiIiKayFjYE7lYFbe59Gpzpy4WH5+tLoG59bKEaYiIaCJjYU/k\nQoIgOO1fP8mQJGEacgVjSKzTDjmHfvhSwjRERDSRsbAncqHmDjM6e9oBABq1FobgGIkTkSvcmXkf\nZJABACrqynk3WiIikgQLeyIXcpqGE5EIuVwhYRpylbBAI9ImzRDbB3/4EoIgSJiIiIgmIhb2RC5U\n6VTYc369N8vLyBd/cKtpuIhq83mJExER0UTDwp7IRex2G2oaLoptLpz1bkF+YciMnyW2edWeiIjG\nGwt7Ihe53FQJq60PABDoG4Igv1CJE5Gr3TFlERQKJQCgvrkaFy6fljgRERFNJCzsiVykasBUDF6t\nnxj89YGYkXin2D5c+hUcgkPCRERENJGwsCdykSqnbS5Z2E8UOekLoFKqAQCNbfU4U3VC4kRERDRR\nsLAncoEeSxdMLbUAAJlMjpjwRIkT0XjR+/ghK+UusX3why9htVklTERERBMFC3siF6g2X4BwdQqG\nITgaWo1O4kQ0nnJS50Or0QMAOrpacfzsAYkTERHRRMDCnsgFnPav5zScCUej1uLOzPvE9vdlf0Vn\nT4eEiYiIaCJgYU/kApUDFs5yfv3ENG3ybIQGGAAAfTYLDv3wpcSJiIjI27GwJxpjXZYOtHc2AwBU\nSjUiQ+IkTkRSkMsVWDDzYbF9quIYzFfXXRAREbkCC3uiMdZ05bL4ODY8UdzXnCaeSYZkTI5MBwAI\nELC3+FPetIqIiFyGhT3RGGvqrBMfxxmSJExC7mD+jAchlysAALWNFThX84PEiYiIyFuxsCcaQ4Ig\noLmzXmzHRrCwn+iC/cMxI+n6Tav2l3zO7S+JiMglWNgTjaGOnhZYbRYAgM7HV1w8SRNb7pRF0Kr7\ntzxt72pB8Tluf0lERGOPhT3RGGoeOA0nIgkymUzCNOQutBod8jLvFdvfnf4rurj9JRERjTEW9kRj\naOD8+tgI3m2WrpuemIuQgAgA/dtfHuT2l0RENMZY2BONEbvDjpYuk9jm/HoaSC5XYMGMh8T2qYpj\nqG+ukTARERF5Gxb2RGPE1FwNu90GAAjQByPQN0TiRORu4o2pSIhMA9C//eVfj++CQ3BInIqIiLwF\nC3uiMVJlviA+5jQcupG7Zz4Mhbz/3gb1zdU4VXFU4kREROQtWNgTjZFq83nxMafh0I0E+YVhVtp8\nsX3g5G70WLqkC0RERF5jxIX91q1bER8fD61Wi+zsbBw6dOim/UtLSzFv3jzodDpER0djw4YNTs/v\n2rUL+fn5CA8Ph7+/P+644w589tlnt/YpiCRmtfWhrqlKbPOKPd3MHen3wF8fBADosXRxIS0REY2J\nERX2O3fuxJo1a/DKK6+gpKQEeXl5WLx4MWpqhl741dHRgUWLFsFoNKKoqAhvvfUW3njjDWzevFns\nc+DAASxcuBC7d+9GSUkJ7r//fixZsmTYHxiI3FFdUyXsjv759b4+gfDV+kuciNyZSqnG3TMfFts/\nXPgOphYupCUiotszosJ+8+bNWLFiBZ555hmkpKTg7bffhtFoxLZt24bsv2PHDvT29mL79u1IT0/H\n0qVL8fLLLzsV9m+++SZeeuklZGdnIyEhAf/2b/+GrKwsfPLJJ2PzyYjGUdWAaTghvpESJiFPkRiV\n4bSQ9psiLqQlIqLbM2xh39fXh+LiYuTn5zsdz8/Px5EjR4Y8p7CwEHPnzoVGo3HqX1dXh6qqqiHP\nAfqv9Ac+XWJwAAAgAElEQVQHB480O5HbqB6wcDbUzyhhEvIUMplsiIW0xyRORUREnkw5XIempibY\n7XZEREQ4HQ8PD4fJZBryHJPJhNjYWKdj1843mUyIi4sbdM6WLVtQV1eHJ5988oZZioqKhotLo8Qx\nvX1WmwUXa84CggDIgGC9kePqIt44rhG6BFwwlwAAPj/43+hstEKt9BnXDN44ru6A4+oaHFfX4diO\nraSk8d9IwyW74shkslH1//Of/4yXXnoJf/zjHxETE+OKSEQu09Jl6i/qAQRoQ6BWaoY5g+i6xIhp\n8FHrAfT/kHjWdFziRERE5KmGvWIfGhoKhUIBs9nsdNxsNsNoHHrKgcFgGHQ1/9r5BoPB6fjHH3+M\np59+Gn/4wx/wox/96KZZsrOzh4tLI3Ttp3KO6e376/Fa6PQ6AECIPgoAx3WsefvXa5BRj08PfQAA\naO6pQXRCBAzBrr/I4e3jKhWOq2twXF2HY+sa7e3t4/6ew16xV6vVyMrKwp49e5yOFxQUIC8vb8hz\ncnNzcfDgQVgsFqf+UVFRTtNw/vSnP+Gpp57C9u3b8cgjj9zqZyCSVE3DgPn1vpxfT6OXFJ2BeGMq\nAC6kJSKiWzeiqTjr1q3DBx98gPfeew/l5eVYvXo1TCYTnnvuOQDA+vXrsXDhQrH/448/Dp1Oh+XL\nl+P06dPYtWsXNm3ahHXr1ol9PvroIzzxxBPYtGkT5syZA5PJBJPJhJaWljH+iESu09XTgca2egCA\nXK5AkD5imDOIButfSPt3TgtpSy/yjrRERDQ6Iyrsly1bhjfffBMbN27EjBkzcOTIEezevVucD28y\nmVBRUSH29/f3R0FBAerq6pCdnY3nn38eL7zwAtauXSv2effdd+FwOLB69WpERkaKf3784x+P8Uck\ncp3qhovi48iQOCgVKgnTkCcL9ne+I+3Bk1/wjrRERDQqw86xv2blypVYuXLlkM+9//77g45lZGRg\n//79N3y9b7/9dqRvTeS2qgfsXx8bkQhYbtKZaBh3pN+D8spitHe1oKevGwdO7sa9OY9KHYuIiDyE\nS3bFIZooBu5fHxsx/ttakXdRKdW4O+vvxHbpxe9R13Tje38QERENxMKe6Ba1d7agrbMZAKBSqBEZ\nEjvMGUTDS4yagslRUwAMWEjrsEucioiIPAELe6JbVD1gN5zo8HgoFCOe2UZ0U3fPfFhcr2FurUXJ\nhUKJExERkSdgYU90i6pMA+fXcxoOjZ1A3xDckX6P2D70w5fo6umQMBEREXkCFvZEt0AQBKcr9rER\niRKmIW+UnTofQX6hAACLtRf7T34hcSIiInJ3LOyJbkFjW714BVWr1iE8MFLiRORtVEoV7slaIrZP\nXypCbWPFTc4gIqKJjoU90S2oNJ0VH8cakiGXKyRMQ94q3piK5JipYpsLaYmI6GZY2BPdgkrTOfFx\nvDFFwiTk7RbMeBgqpRpA/2+KTpw/LHEiIiJyVyzsiUbJautzmhIRF5EsYRrydv76QOROWSS2D5d+\nzYW0REQ0JBb2RKNU21gBu90GAAjxj4C/PlDiROTtslPuQrBfGAAupCUiohtjYU80SpX116fhTOI0\nHBoHCoUSd//NQtrLjZckTERERO6IhT3RKA1cODvJwMKexke8MQXJMZli+6/HP+FCWiIicsLCnmgU\nrnS3oandBABQyJWIDo+XOBFNJPNnPASVon8hrbm1Ficvfi9xIiIicics7IlGYeBuOFFh8VArNRKm\noYkmQB+M2el3i+1DP3yJbkunhImIiMidsLAnGoUqbnNJEstOnY9A3xAAQG9fNw6e/FLiRERE5C5Y\n2BONkENwOF2xjzNwm0safyqlCnfPfFhsl178HvXN1RImIiIid8HCnmiEGlovo8fSBQDQ+fgiLNAo\ncSKaqCZHTcHkyHQAgAABfz3+v3AIDolTERGR1FjYE43QpXrn3XDkMv7zIeksmPkwFAolAKC+uRqn\nKo5KnIiIiKTGyoRohAbOr5/EaTgksSC/UOSkLhDbB07uRm9fj4SJiIhIaizsiUbAYu3F5aZKsc3C\nntzB7PS74a8LAgD0WLpwuPRriRMREZGUWNgTjUBNw0XxZkDhgZHQa/0lTkQEqJRqzJ/xoNguOX8Y\njW31EiYiIiIpsbAnGgGnaTjc5pLcSHLMVMSGJwLo37lp7/FPIAiCxKmIiEgKLOyJRqDSaeEsp+GQ\n+5DJZLg76+/ExdzVDRdwruYHiVMREZEUWNgTDaO9swUtVxoBACqFGpGh8RInInIWFmjE9KQ7xfa+\nE5/BauuTMBEREUmBhT3RMCpN16/WR4cnQKVUSZiGaGh3Zt4LncYXANDR3Yrvy/ZKnIiIiMYbC3ui\nYVRym0vyAD5qLeZOu19sHz3zLdo6myVMRERE442FPdFNOBx2p4WzcSzsyY1lJMyCITgGAGC327Dv\nxF8kTkREROOJhT3RTdQ318Bi7QUA+GkDEBpgkDgR0Y3JZXLck/V3Yvt87SmnOyYTEZF3Y2FPdBMX\n68rEx3HGFMhkMgnTEA0vMnQSpsRni+29xZ/AbrdJmIiIiMYLC3uiGxAEAecHbBuYGDVFwjREIzdv\n2o+gVvkAAFo6GnD83EGJExER0XhgYU90A80dZqdtLicZeGMq8gx6rT/uzMgX20dO7UFHV5uEiYiI\naDywsCe6gfM1peLj+MhUbnNJHmVG8hxxTYjV1of9JZ9JnIiIiFyNhT3RDZyvPSU+To6ZKmESotFT\nyBW4J2uJ2D5TXeK0wxMREXkfFvZEQ2jrbIa5tRYAoJArkRCZJnEiotGLjUhEWtxMsf3X41xIS0Tk\nzVjYEw1h4NX6OEMSNFcXIhJ5mnnTH4BaqQHQv26EC2mJiLwXC3uiIQzcDScpOlPCJES3x08XgLzM\n6wtpC08V4Ep3u4SJiIjIVUZc2G/duhXx8fHQarXIzs7GoUOHbtq/tLQU8+bNg06nQ3R0NDZs2OD0\nvMlkwuOPP460tDQolUqsWLHi1j4B0Rjr7OnA5aZKAIBMJuc2l+TxZibPRUhABACgz2bhQloiIi81\nosJ+586dWLNmDV555RWUlJQgLy8PixcvRk1NzZD9Ozo6sGjRIhiNRhQVFeGtt97CG2+8gc2bN4t9\nLBYLwsLCsH79esyePZs3/iG3MXAaTkxYAnQ+vhKmIbp9CrkCC7MeEdvlVSdQbb4gYSIiInKFERX2\nmzdvxooVK/DMM88gJSUFb7/9NoxGI7Zt2zZk/x07dqC3txfbt29Heno6li5dipdfftmpsI+Li8Nb\nb72Fp556CsHBwWPzaYjGwPna69tcJsVwGg55h/6FtDPE9jfHd8EhOCRMREREY23Ywr6vrw/FxcXI\nz893Op6fn48jR44MeU5hYSHmzp0LjUbj1L+urg5VVVW3GZnIdXosXagZcCUzMSpDwjREY8tpIW27\nGZcaTw1zBhEReZJhC/umpibY7XZEREQ4HQ8PD4fJZBryHJPJNKj/tfaNziFyBxcvl4lXMY0hsfDX\nB0qciGjs+OkCnRbSnjMVo8vChbRERN5C6YoXddV8+aKiIpe87kTGMXVWdOkbdHd1AwBUAX63PD4c\nV9fguN4+QdBBKWjR0d0MACitOQzdMX+uc3IBfr26BsfVdTi2YyspKWnc33PYK/ahoaFQKBQwm81O\nx81mM4xG45DnGAyGQVfmr51vMBhuNSuRS9nsVjReuSy2IwLiJExD5BpymRxTo+eIhXxzZz1qWnhH\nWiIibzDsFXu1Wo2srCzs2bMHS5cuFY8XFBTg0UcfHfKc3NxcvPzyy7BYLOI8+4KCAkRFRSEu7taL\npezs7Fs+l5xd+6mcY3rd2eqT8Kno/3oNCzRi/p33jPo1OK6uwXEde0p/G/YVfQEAqL1ShkVzH4Cf\nLkDiVN6BX6+uwXF1HY6ta7S3j/9UxxHtirNu3Tp88MEHeO+991BeXo7Vq1fDZDLhueeeAwCsX78e\nCxcuFPs//vjj0Ol0WL58OU6fPo1du3Zh06ZNWLdundPrlpSUoKSkBO3t7WhubkZJSQnKysrG8OMR\njZzTbji8KRV5ubyMe6HT+AMALNZe/PX4/0qciIiIbteI5tgvW7YMzc3N2LhxI+rr65GZmYndu3cj\nJiYGQP+C2IqKCrG/v78/CgoKsGrVKmRnZyM4OBgvvPAC1q5d6/S6M2fOBNA/J18QBHz22WeYNGmS\n02sRjQerzYqLl6//UMnCnrydSqnC1Jg5+O7CbgD9P9ieq/kByTFTJU5GRES3asSLZ1euXImVK1cO\n+dz7778/6FhGRgb2799/09d0OLiHMrmHavN59NksAIBA3xCEBQ69foTIm4T4GhETkoLm3v6bDX5z\n/H8RE54IrUYncTIiIroVI5qKQ+TtBk7DSY6Zyh1CaMJIM86CXts/JaerpwP7Sz6TOBEREd0qFvY0\n4fVZLThX84PYTormTalo4lApNViY9YjYLq04iirTeQkTERHRrWJhTxNeeVUxLNZeAECQXxgMIbES\nJyIaX8kxmU5z6786uhM9lm4JExER0a1gYU8TmiAIKLlQKLanJ+ZCLuM/C5p4FmYtgUatBQB0dLXi\n66M7IQiCxKmIiGg0WMHQhFbXVIWG1v6bUqkUakyJnyVxIiJp6LX+uC/nMbF9vvYUis8dlDARERGN\nFgt7mtBKLhwRH6fGTeduIDShJcdkIitlrtjeV/I56purJUxERESjwcKeJqzu3k6crT4ptmck3Slh\nGiL3MG/aAzAE99+jxOGw47PDf+B8eyIiD8HCnias0oqjsDtsAABjSCwigqMlTkQkPYVCiQfvfBIa\nlQ8AoL2rhfPtiYg8BAt7mpAcDjtOOi2azZMwDZF7CfQNwX2zOd+eiMjTsLCnCelS/Vm0d7UAALRq\nHVJip0uciMi9JMdMxcxkzrcnIvIkLOxpQhq4aDYjIQcqpUrCNETuad70Hw2ab99t6ZQ4FRER3QgL\ne5pw2jqbcanujNielpgrYRoi96VUqAbNt//4299xMS0RkZtiYU8TzskLhRDQvxAw3piKIL9QiRMR\nua9A3xAsvuPvIYMMAGBurcWf9/9OvFszERG5Dxb2NKFYbVaUVhwV29OTuGiWaDhJ0Rm4d/YysV3f\nXI1d+99Dn80iYSoiIvpbLOxpQjlXcxI9li4AgL8uCAnGVIkTEXmGzIQcLMx+RGzXNlbgk4MfwGqz\nSpiKiIgGYmFPE0rJgC0upyXmQi5XSJiGyLPMSLoT82c8JLarTOfwl8PbYbfbJExFRETXsLCnCaOu\nqQp1TZUAALlcgczJOdIGIvJAs1LnYc7UxWK7oq4cnxfugMNhlzAVEREBLOxpgnAIDuwt/kRsp8RM\ng97HT8JERJ4rd8pC3DFlodg+V/MDdh34/9DVe0XCVERExMKeJoTTl4rEm+so5ErcmZkvcSIizzYn\n8z5kp8wT25fqz+CDL3+FigFbyRIR0fhiYU9ez9LXgwMnvxDbs1LnIcgvTMJERJ5PJpNh/owHMTv9\nHvFYd28n/rz/d9hb/Clsdi6qJSIabyzsyesdOVWA7t7+u2X66QIxe8o9w5xBRCMhk8lw17T78eiC\nZ6HX+ovHj589gB173kZTu0nCdEREEw8Le/JqTe0mFJ8/JLbnTX8AaqVGwkRE3meSIRnL7/t/MTky\nXTzW0FaHP3z9Jo6fPcAtMYmIxgkLe/JagiBg7/FPxN06YsInIzV2usSpiLyTzscXS+76R9yTtQQK\nhRIAYLNbsbf4U/z2LxtxuPRrdPV0SJySiMi7KaUOQOQq52tLUWU+DwCQy+S4J2sJZDKZxKmIvJdM\nJsPM5DmICZ+MLwp3oLGtHgDQbenEkVN78H35XqTHzURWyl0ICzRKnJaIyPuwsCevZLX14dviv4jt\n6Ul5LCSIxklYoBFPLFqNkguHUXz2EDq6WwEAdrsNpRVHUVpxFHGGZCRGTUFcRBKC/cP5QzcR0Rhg\nYU9e6Wj5t2IxodXokZdxr8SJiCYWlVKFWanzkZU8F2drfsDxswfELWeB/rvWVpnOAQD0Wn/ERSQi\nNiIJsRGJCNAHSxWbiMijsbAnr9Pe2YKj5d+K7blT74dWo5MwEdHEJZcrkBY3A6mx01HXVImiswdw\nvvYUBMEh9unq6UBZZTHKKosBAHofP4QEGBAaEIHQAANCAwwICTDAR62V6mMQEXkEFvbkVaw2K3Z/\n99/iHtoRwdHITJglcSoikslkiAqLR1RYPNo7W3CxrgzV5vOobrgIS1+PU9+u3ivo6r2C6qtrZK7R\na/0R7BeKQN/+P0F+oQj0C0Wgbwg0Kp/x/Dg3ZHfY0dPXiR5rF85UlaCr9wr6bBbY7FZYbX2w2fpg\ntffBarNCIVdAo9bCR62Fj1oHjcoHPmodtBo9QvzDofPx4xQlIhoVFvbkNewOOz47/HvUNlaIx+6Z\nuQRyuULCVET0twJ8gzEzeQ5mJs+Bw2GHubUONQ0XUG0+j9qGS7Da+4Y8r6unA109HahpqBj0nFaj\nh78+CAH6YATog8XHfrpA6H18odXob/t7gUNwoKvnCq50t+FKdxs6ulvR0dUmtq/0tKO75wq6uroA\nACfrbu83hVqNHqEBBoQFGsXfXIQFRXLLXiK6IRb25BUcggNfff8RLtaVicfmz3gQUWGTpAtFRMOS\nyxUwhsTAGBKDnLQFcDjsaO9qQVO7GU3tJjRf/bulowF2h+2Gr9Nj6UKPpQvmltohn5dBBh+NDjqN\nL3RXC32lQgWZTAa5TA65XHH1bzlsdhssfT2wWHvQ29cDS1//373WHnH73PHQY+lCTcNF1DRcFI/1\nj1csYsMTERuRCGNIHFRK1bhlIiL3xsKePJ4gCPi2+FNxfi4AzE6/B7NS50sXiohuiVyuQJBfGIL8\nwpAUnSEe7y/4W9HW2YTWK01o62zu/3P18c2KfgAQIIjFf3OH2WX5ZZBBo9LCR6XHpOgE6H38oVZp\noFKqoVKqoVSooVaqoVSoYHfY0dvXDYu1t//vqz9MXOluR3O7GX02y6DXdzjsuNx4CZcbL6HwdAEU\nCiWiQiYhNiIR8ZGpCA+KglzGW9QQTVQs7MnjHTm1B8Xnrt9ddlpiLuZOXSxhIiIaa/0Ff/+8+vi/\n2bn22hSZjq4WtHe1oL2rtf9xZws6ezrQbelEr6UbAoTbzqHV6OGvC4KfLgD++iD46QKv/gmAny4Q\nvj7+OHGiBACQnZ19y+/jEBy40tWGxvZ6NLWZ0NRuQkPbZTS3O/9QYrfbUN1wAdUNF3Co9Cvotf5I\nMKYiITIdcYYkt1l7QETjg4U9ebTjZw/iyKk9Yjs1djoW8kZURBOKXCa/WlgHICosfsg+DocdPZYu\ndFu60N17Bd2WLjgcdjgEBwTBAYfDIT6WyeTwUevgo9b2X32/usBVrdKO27QXuUyOAN9gBPgGIzFq\nini8u7cTNQ0XUW3uL+ZbOhqczuvq6RDvFSCXKxATPhmTI9OQEJmOIL/QcclORNJhYU8e6/SlIuwt\n/kRsxxtTcf8df8/FskQ0iFyugF7rD73WH4Dn3qxO5+OLlNhpSImdBgC40t2OmoaLqKw/i4r6cvRY\nusS+DoddvF/A3uJPEewfjsmR6UiITENUWDwU/F5J5HVY2JPH6eq9gv0nPsfpyiLxWFToJDw05yko\nFPySJqKJw08XgPRJM5E+aSYcDjvqm2tQUV+OirpyNLRedurb0tGAlo4GHDuzDxq1FvGGFEwypiAu\nIhn++kCJPgERjSVWQeQxHIIDpReP4sDJL9Db1y0eDws0Ysldz3ALOCKa0ORyBaLCJiEqbBLmTl2M\njq42XKovx8W6clSbzjttI2rp68GZ6hKcqe5fDxASEIFJhhRMMqQgOjye30+JPBQLe/IIjW31KCj6\nMy43XnI6nnJ1Tj3vLEtE5MxfH4hpibmYlpgLq82KmoYLuHi5DBV15ejobnXq29xuRnO7GcfPHoBC\nroQxJBZRYfGIDouHMSSO32OJPMSICvutW7fijTfegMlkwpQpU/Dmm29izpw5N+xfWlqKf/mXf8Gx\nY8cQHByMZ599Fq+++qpTn/3792PdunUoKytDZGQkXnrpJTz77LO392nI63R0teHE+UMoOnvAaf/o\nAN8QLMx6BAmRqRKmIyLyDCqlCgmRaUiITIMgCGhsq0el6SwqTedQ21gBu/36dqF2hw21jRWobazA\n9+jfwjM00ICo0HhEhsYhPCgKIf7hXM9E5IaGLex37tyJNWvWYNu2bZgzZw62bNmCxYsXo6ysDDEx\nMYP6d3R0YNGiRZg/fz6KiopQXl6OFStWQK/XY926dQCAS5cu4f7778c//dM/4Y9//CMOHjyIn/70\npwgLC8Mjjzwy9p+SPErrlUacrz2FczU/oL652uk5uVyBWanzkTtlIVRKtUQJiYg8l0wmQ3hQJMKD\nIpGTtgBWWx9qGytQWX8OVeZzaGyrd+ovoP8Hgca2epRcOAIAUCpUCAs0IiwwEuFBUQgPjESwfziv\n7BNJbNjCfvPmzVixYgWeeeYZAMDbb7+Nr776Ctu2bcMvfvGLQf137NiB3t5ebN++HRqNBunp6Thz\n5gw2b94sFvbvvPMOoqOj8dZbbwEAUlJS8P333+NXv/oVC/sJRhAEdPVeQUtHA2obK3CuphSNbXVD\n9o0Ki8ei7KUIC/TcHS2IiNyNSqlGvDEV8cb+34B29XTgclMlLjdW4nLTJZhbauEQHE7n2OxW1DdX\nD7r4otXoEeQbiiC/MAReve9AgD4YvtoA6LV+3ImHyMVuWtj39fWhuLgYL730ktPx/Px8HDlyZMhz\nCgsLMXfuXGg0Gqf+r776KqqqqhAXF4fCwkLk5+cPes3t27fDbrdDoeA/fG9gt9vQ29eNXmsPei09\nV++u2IW2zha0XmlAS0cjWq40os/ae8PXuLYPc0Z8NlLjZvCOikRELqbX+iM5ZiqSY6YCAPpsFtQ3\nVeNy0yWYWmrR0HoZV7rbhjz32t1965qrBj0ngwxaHz18tQHw1fpD7+MHH7UOGrV20H0D1CoNVAo1\nlEpV/9/c8YxoRG76L6WpqQl2ux0RERFOx8PDw2EymYY8x2QyITY21unYtfNNJhPi4uJgNpsHvWZE\nRARsNhuampoGPXfNrv3v3fzTkGi4Oyyazf13L6zqKnE+TxD6zxacX0cQBAiCo/9vCGLb4XDA7rDB\n7rCLfzvsNtjsNqcdGEZDoVBikiEFydGZmByVDq1Gf0uvQ0REt0+t1CDOkIQ4Q5J4rNvSicbWOphb\nL6OhtQ5N7fVovdIEm916w9cRIKC7txPdvZ2DtuIciZ6eXihkCnxf/RnkcgXkcjkUcgXkMkV/WyaH\nTCa7+qf/sVwmQ/+PFDJAJoMMuPq3DP33Mbx+M8OxuLGhDJ55c8RrNUF110mJkwwvPjINM5LypI7h\ntsb8R2BX3vHznuk/dtlrk3vq67Whr7dd6hgjlpTU/z++9nbPyewJOK6uwXF1jYkyroHaCARqI5AS\nOVPqKDTBePu/rdtx03kNoaGhUCgU4k9y15jNZhiNQ89zNhgMg67mXzvfYDDctI9SqURoKG95TURE\nREQ0Wjct7NVqNbKysrBnzx6n4wUFBcjLG/rXILm5uTh48CAsFotT/6ioKMTFxYl9CgoKBr3mrFmz\nOL+eiIiIiOgWyARBuOlk7D/96U948sknsXXrVuTl5eGdd97B+++/j9OnTyMmJgbr16/HsWPH8M03\n3wDo3+4yJSUF8+fPxyuvvIKzZ89ixYoVeP3117F27VoAQGVlJTIyMvDP//zP+MlPfoLDhw9j1apV\n+Oijj7BkyRLXf2oiIiIiIi8z7Bz7ZcuWobm5GRs3bkR9fT0yMzOxe/ducQ97k8mEiooKsb+/vz8K\nCgqwatUqZGdnIzg4GC+88IJY1APApEmTsHv3bqxduxbbtm1DVFQUfvOb37CoJyIiIiK6RcNesSci\nIiIiIvfn1puCb926FfHx8dBqtcjOzsahQ4ekjiSZAwcO4KGHHkJ0dDTkcjm2b98+qM/rr7+OqKgo\n6HQ6LFiwAGVlZU7PWywWPP/88wgLC4Ovry8efvhhXL7svOVYa2srnnzySQQGBiIwMBBPPfXUoNXn\n1dXVePDBB+Hr64uwsDCsXr0aVuuNtzhzZ7/85S8xa9YsBAQEIDw8HA899BBOnz49qB/HdnS2bNmC\nadOmISAgAAEBAcjLy8Pu3bud+nBMb98vf/lLyOVyPP/8807HObaj8/rrr0Mulzv9iYyMHNSHYzp6\n9fX1ePrppxEeHg6tVospU6bgwIEDTn04tqMzadKkQV+vcrkcDzzwAID+7ak5pqNns9nwr//6r0hI\nSIBWq0VCQgJeffVV2O12p35uP7aCm/roo48ElUol/Nd//Zdw5swZ4fnnnxd8fX2F6upqqaNJYvfu\n3cLPfvYz4eOPPxZ0Op2wfft2p+f/8z//U/Dz8xN27dolnDp1Sli2bJkQGRkpXLlyRezz3HPPCZGR\nkcI333wjFBcXC/PnzxemT58u2O12sc99990nZGRkCN99951QWFgoTJkyRXjwwQfF5202m5CRkSEs\nWLBAOHHihFBQUCBERkYKzz//vOsHwQXuvfde4YMPPhBOnz4tlJaWCkuWLBEMBoPQ0tIi9uHYjt6n\nn34qfPXVV8LFixeF8+fPCz/72c8ElUollJSUCILAMR0LhYWFQnx8vDBt2jSnz8OxHb3XXntNSEtL\nE8xms/inqalJfJ5jemtaW1uF+Ph44emnnxaOHTsmVFZWCnv37hXKy8vFPhzb0WtqanL6Wj1x4oQg\nl8uF3//+94IgcExv1c9//nMhODhY+Pzzz4WqqirhL3/5ixAcHCxs2LBB7OMJY+u2hX1OTo7wk5/8\nxOlYUlKSsH79eokSuQ9fX1+nwt7hcAgGg0H4xS9+IR7r6ekR/Pz8hHfffVcQBEFoa2sT1Gq18Mc/\n/lHsU1NTI8jlcuHrr78WBEEQysrKBJlMJhw5ckTsc+jQIUEmkwnnzp0TBKH/Bwy5XC7U1taKfT78\n8DcsW+UAAAfKSURBVEPBx8fH6QvbU3V2dgoKhUL4/PPPBUHg2I6l4OBg4be//S3HdAy0tbUJkydP\nFvbt2yfMnz9f/GbPsb01r732mpCRkTHkcxzTW7d+/Xphzpw5N3yeYzs2Nm7cKAQFBQm9vb0c09vw\nwAMPCMuXL3c69tRTTwkPPPCAIAie8/XqllNx+vr6UFxcjPz8fKfj+fn5OHLkiESp3NelS5dgNpud\nxsvHxwd33XWXOF7Hjx+H1Wp16hMdHY20tDQUFhYCAAoLC+Hr64vc3FyxT15eHvR6vfg6hYWFSE9P\nR1RUlNgnPz8fFosFx48fd+nnHA8dHR1wOBwICgoCwLEdC3a7HR999BF6e3tx1113cUzHwE9+8hM8\n+uijmDdv3tW7Rffj2N66iooKREVFISEhAX//93+PS5cuAeCY3o5PPvkEOTk5eOyxxxAREYEZM2Zg\ny5Yt4vMc29snCALee+89/MM//AM0Gg3H9DYsXrwYe/fuxdmzZwEAZWVl+Pbbb/GjH/0IgOd8vY75\nnWfHQlNTE+x2OyIiIpyOh4eHD7qxFeH/b+/+QprqwziAf7fanIPmxbKcE5qCuZpDZGLOLoogKAoh\niFIwqEAoLUczKLSLilZ0W7gbg/Im0sCLwBsHk3Jo4MX8k2kZGiKlVhfCAo22570ID53+2Otmr3Pv\n9wODec7z25+v29njPOd3lEx+lde7d++Umg0bNsBsNqtqtm7dqoyfmZlBZmamar1Go1HlPjMz89P9\nLJ3ILBV+Nx6PB8XFxcobjtnGb3h4GG63G4uLi0hPT0d7ezsKCgqUDRczjU9LSwsmJibw8OFDAOqz\nffP1Gp+ysjK0trbCbrdjdnYWN27cQHl5OUZGRphpAiYmJuD3++H1etHY2IhwOKwcD1JXV8dsV0Eg\nEMDbt29RU1MDgNuARNTW1mJ6eho7duzAxo0b8fXrV1y5cgVnzpwBsH6yTcrGnlbP9x/6vyJxTIoU\nz5j1wOv1ore3F6FQ6I+5Acz2T+x2O4aGhjA/P4/Hjx+jsrIS3d3dy45hpst79eoVmpqaEAqFlJP5\nybddKv84ltn+3oEDB5TrhYWFcLvdyM3NRWtrK3bt2vXbccx0ebFYDKWlpfD5fACAoqIijI+Po7m5\nGXV1dcuOZbb/TktLC0pLS+F0Ov9Yy0yXd+fOHdy/fx+PHj2Cw+FAOByGx+OBzWbD6dOnlx2bTNkm\n5a44S3+VzM7OqpbPzs7CYrGs0aNKXllZWQDwy7yW1mVlZSEajeLTp0/L1nz48EG1XkQwNzenqvnx\nfpb+w7JUsx5duHABbW1tCAaDsNlsynJmGz+dToe8vDwUFxfj5s2bKCsrQ3Nzs/IeZqYr19fXh48f\nP8LhcECn00Gn0+HZs2fw+/3Q6/XYvHkzAGabKKPRCIfDgTdv3vD1moDs7Gzs3LlTtcxut2NqagoA\nt6+Jmpubw5MnT5Rv6wFmmgifz4fGxkYcO3YMDocD1dXV8Hq9uHXrFoD1k21SNvZ6vR4ulwtdXV2q\n5YFAAOXl5Wv0qJJXbm4usrKyVHktLCwgFAopeblcLuh0OlXN9PQ0xsbGlBq3241IJKLsBwZ8ayQ+\nf/6s1JSXl2N0dFQ1dVMgEEBaWhpcLtdffZ5/i8fjUZr67du3q9Yx29UTjUYRi8WYaQKOHDmCFy9e\nYHBwEIODgxgYGEBJSQmqqqowMDCA/Px8ZrsKFhYWMDo6CovFwtdrAnbv3o2xsTHVstevXytfnjDb\nxDx48AAGgwFVVVXKMmYaPxGBVqtui7VarfLN+brJdtlDa9dQW1ub6PV6uXfvnrx8+VLq6+tl06ZN\n/9vpLiORiITDYQmHw2I0GuX69esSDoeVPG7fvi0ZGRnS0dEhw8PDcvz4cbFarRKJRJTbOHv2rOTk\n5KimYCouLpZYLKbUHDx4UJxOp/T19Ulvb68UFhZKRUWFsj4ajYrT6ZR9+/YpUzBZrVapr6//78JY\nRbW1tWIymSQYDMr79++Vy/e5MduVu3TpkvT09Mjk5KQMDQ3J5cuXRavVSldXl4gw09W0Z88eOXfu\nnPIzs125hoYGefr0qUxMTMjz58/l0KFDkpGRwe1rgvr7+0Wn04nP55Px8XFpb2+XjIwM8fv9Sg2z\njU8sFpP8/PyfZg8UYabxqqmpkZycHOns7JTJyUnp6OiQzMxMuXjxolKzHrJN2sZeRMTv94vNZpO0\ntDQpKSmRnp6etX5Ia6a7u1s0Go1oNBrRarXK9VOnTik1V69eFYvFIgaDQfbu3SsjIyOq21hcXJTz\n58+L2WwWo9EoFRUVqqmURL7NO1xdXS0mk0lMJpOcOHFC5ufnVTVTU1Ny+PBhMRqNYjabxePxyJcv\nX/7ek/+Lfsxz6XLt2jVVHbNdmZMnT8q2bdskLS1NtmzZIvv371ea+iXMdHV8P93lEma7MpWVlZKd\nnS16vV6sVqscPXpUNde6CDONV2dnpxQVFYnBYJCCggK5e/fuTzXMduWCwaBotVrp7+//5XpmunKR\nSEQaGhrEZrNJenq65OXlSVNTkywuLqrqkj1bjUgKHflARERERPQ/lZT72BMRERER0cqwsSciIiIi\nSgFs7ImIiIiIUgAbeyIiIiKiFMDGnoiIiIgoBbCxJyIiIiJKAWzsiYiIiIhSABt7IiIiIqIUwMae\niIiIiCgF/AM1hIn1h8z1WwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "data = ReadData(path='../code')\n", + "cols = zip(*data)\n", + "price1, price2, bid1, bid2, diff1, diff2 = cols\n", + "\n", + "pdf = thinkbayes.EstimatedPdf(price1)\n", + "low, high = 0, 75000\n", + "n = 101\n", + "xs = numpy.linspace(low, high, n) \n", + "pdf.kde.evaluate([3, 3])\n", + "\n", + "pmf = pdf.MakePmf(xs)\n", + "thinkplot.Pmfs([pmf])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pdf` is a `Pdf` object, estimated by KDE.\n", + "`pmf` is a Pmf object that approximates the Pdf by evaluating\n", "the density at a sequence of equally spaced values.\n", "\n", - "linspace stands for “linear space.” It takes a range,\n", - "low and high, and the number of points,\n", - "n, and returns a new numpy array with\n", - "n elements equally spaced between low and\n", - "high, including both.\n", - "\n", - "And now back to *The Price is Right*.\n", - "\n", - "Modeling the contestants\n", - "------------------------\n", + "`linspace` stands for “linear space.” It takes a range,\n", + "`low` and `high`, and the number of points,\n", + "`n`, and returns a new `numpy` array with\n", + "`n` elements equally spaced between `low` and\n", + "`high`, including both.\n", "\n", - "![Cumulative distribution (CDF) of the difference between the\n", - "contestant’s bid and the actual price.](figs/price2.pdf)\n", - "\n", - "[fig.price2]\n", + "And now back to *The Price is Right*." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Modeling the contestants\n", "\n", "The PDFs in Figure [fig.price1] estimate the distribution of possible\n", "prices. If you were a contestant on the show, you could use this\n", @@ -243,7 +620,7 @@ "1. What data should we consider and how should we quantify it?\n", "\n", "2. Can we compute a likelihood function; that is, for each hypothetical\n", - " value of price, can we compute the conditional\n", + " value of `price`, can we compute the conditional\n", " likelihood of the data?\n", "\n", "To answer these questions, I am going to model the contestant as a\n", @@ -251,29 +628,32 @@ "words, when the contestant sees the prizes, he or she guesses the price\n", "of each prize—ideally without taking into consideration the fact that\n", "the prize is part of a showcase—and adds up the prices. Let’s call this\n", - "total guess.\n", + "total `guess`.\n", "\n", "Under this model, the question we have to answer is, “If the actual\n", - "price is price, what is the likelihood that the\n", - "contestant’s estimate would be guess?”\n", + "price is `price`, what is the likelihood that the\n", + "contestant’s estimate would be `guess`?”\n", "\n", "Or if we define\n", "\n", - " error = price - guess\n", - "\n", + "```python\n", + "error = price - guess\n", + "```\n", "then we could ask, “What is the likelihood that the contestant’s\n", - "estimate is off by error?”\n", + "estimate is off by `error`?”\n", "\n", "To answer this question, we can use the historical data again.\n", "Figure [fig.price2] shows the cumulative distribution of\n", - "diff, the difference between the contestant’s bid and the\n", + "`diff`, the difference between the contestant’s bid and the\n", "actual price of the showcase.\n", "\n", "The definition of diff is\n", "\n", + "```python\n", " diff = price - bid\n", + "```\n", "\n", - "When diff is negative, the bid is too high. As an aside, we\n", + "When `diff` is negative, the bid is too high. As an aside, we\n", "can use this distribution to compute the probability that the\n", "contestants overbid: the first contestant overbids 25% of the time; the\n", "second contestant overbids 29% of the time.\n", @@ -287,125 +667,185 @@ "actually know the contestant’s guesses; we only know what they bid.\n", "\n", "So we’ll have to make some assumptions. Specifically, I assume that the\n", - "distribution of error is Gaussian with mean 0 and the same\n", - "variance as diff.\n", - "\n", - "The Player class implements this model:\n", + "distribution of `error` is Gaussian with mean 0 and the same\n", + "variance as `diff`.\n", "\n", - " class Player(object):\n", - "\n", - " def __init__(self, prices, bids, diffs):\n", - " self.pdf_price = thinkbayes.EstimatedPdf(prices)\n", - " self.cdf_diff = thinkbayes.MakeCdfFromList(diffs)\n", + "The `Player` class implements this model:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Player(object):\n", "\n", - " mu = 0\n", - " sigma = numpy.std(diffs)\n", - " self.pdf_error = thinkbayes.GaussianPdf(mu, sigma)\n", + " def __init__(self, prices, bids, diffs):\n", + " self.pdf_price = thinkbayes.EstimatedPdf(prices)\n", + " self.cdf_diff = thinkbayes.MakeCdfFromList(diffs)\n", "\n", - "prices is a sequence of showcase prices, bids\n", - "is a sequence of bids, and diffs is a sequence of diffs,\n", - "where again diff = price - bid.\n", + " mu = 0\n", + " sigma = numpy.std(diffs)\n", + " self.pdf_error = thinkbayes.GaussianPdf(mu, sigma)\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`prices` is a sequence of showcase prices, `bids`\n", + "is a sequence of bids, and `diffs` is a sequence of diffs,\n", + "where again `diff = price - bid`.\n", "\n", "`pdf_price` is the smoothed PDF of prices, estimated by KDE. `cdf_diff`\n", - "is the cumulative distribution of diff, which we saw in\n", + "is the cumulative distribution of `diff`, which we saw in\n", "Figure [fig.price2]. And `pdf_error` is the PDF that characterizes the\n", - "distribution of errors; where error = price - guess.\n", - "\n", - "Again, we use the variance of diff to estimate the variance\n", - "of error. This estimate is not perfect because contestants’\n", + "distribution of errors; where `error = price - guess`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu8AAAEWCAYAAADW9nkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X901NWd//FXJiGZBAiQQEJCEEFZUKBgiRZSFLSaLVaR\nIgKlolBaFlpBpJ66rHRVwF91l2N1BcVaZKus8WCWbGHWAqsFEdqCEAuBpGj8IpTMSBISBBIwmfv9\nI84wk0ySGTKTmUmej3NyZD5zP8yd60Beubw/70+MMcYIAAAAQMSzhHsCAAAAAPxDeAcAAACiBOEd\nAAAAiBKEdwAAACBKEN4BAACAKEF4BwAAAKIE4R0AAACIEn6H9507d2rSpEnKysqSxWLR+vXrWz3n\n4MGDGj9+vJKSkpSVlaUVK1a0abIAAABAZ+Z3eD937py+8Y1v6Ne//rUSExMVExPT4vgzZ87otttu\nU0ZGhvbt26df//rXeu6557Rq1ao2TxoAAADojGIu5w6r3bt310svvaT77ruv2TFr1qzR0qVL5XA4\nlJCQIEl68skntWbNGp04ceLyZwwAAAB0UiGred+zZ49uvPFGd3CXpNzcXJ08eVLHjh0L1csCAAAA\nHVbIwrvdbld6errXMddju90eqpcFAAAAOqy4UP3GrdXEe6qurg7VNAAAAICQ69GjR7u8Tsh23vv2\n7dtkh93hcLifAwAAABCYkIX3sWPH6oMPPtCFCxfcx7Zt26Z+/fppwIABoXpZAAAAoMPyu2zm3Llz\nOnr0qCTJ6XTq2LFjKiwsVGpqqvr376+lS5dq79692r59uyRp5syZeuKJJzR79mwtW7ZMJSUlevbZ\nZ/X444+3+Drt9U8OncG+ffskSdnZ2WGeScfCuoYG6xo6rG1otLauBbYS5W0qUm1tnSSpIsWqU30S\n5bT4X1baGTmdTkmSxeL//qLFEqNePa3q2cMaqmlFvfPnzkuSkromheX1E+NiNXN4hqYMTW99cBQJ\nR+m33+F97969uuWWWyQ11LM/9thjeuyxxzR79mz99re/ld1uV2lpqXt8cnKytm3bpp/97GfKzs5W\nSkqKHn74YT300EPBfxcAAARBfrFDGw6VqaauvtkxVdW1Ol1Vq7qvx1i2bW/+NxyY7NfrWpxGQ0tO\ny2qN0/TJw3TX7UMCmndHwg+boXFpXUeHeSZoK7/D+4QJE9w/Dfuybt26JseGDx+uHTt2XN7MAAAI\nkD/hOxCuoO50BnxLlCaa2x3uqDuSAEIjZN1mAABoT/nFDr1W2LabAAYzrEtNAztBHUBbEd4BAFGl\nrbvrrQV0i9Mo/VSNUitrm/09rNY43TAySeO+1ZvyDgDtivAOAIgq/gT3uaOyfO5uF9hKtN72qQLZ\n926uDt1VQwwA7YnwDgCIOJe7u+6rLKVx15fWcNEogEhGeAcARIRAA3tiXKw2Th3lfuwK6W/8z9/0\nhh/n3z9jJAEdQNQhvAMAwqatO+yB7qpL7KwDiG6EdwBAuwkkrDfXmcVzh91fBHYAHQXhHQAQcv6G\n9tZaKRbYSrT+rY+bPZ+QDqCjI7wDAELGn9DuT+/zlspjCOwAOhPCOwAgZHwFd39vVNRaPTsXnALo\njAjvAICQ8QzuLYX2QC48ZacdQGdGeAcABE1LZTKebR0l+q8DwOUgvAMA2qy12vbEuFivx61deOpC\nYAcAb4R3AECbtRbch9ZLM+flt7jLTlAHgNYR3gEAl6W53XZXbXtsaZW7LOZwC78PF54CgP8s4Z4A\nACD65Bc79FrhCZ/BfePUUYotrdL6tz5udaed4A4AgWHnHQAQEFdwb8y14y5JeZuKmjxPWQwAtB3h\nHQDgl+bKZK6LiVXp/5WqtrZOb/zP3/RGo/PYXQeA4CG8AwBa1dxu+3UxsTq85W/Nnme1xhHcASCI\nCO8AgGa1dFHq0Hq1GtynTx4W6ikCQKdCeAcAeGmtZ/vcUVnuC1I9UR4DAKFHeAcAeGkuuLt22zeu\n+rBJFxmCOwC0D8I7AMBL4+B+/swFxR6tVILjvM9+7QR3AGg/hHcA6KRaK4+pqq7VgI8cSmimVzut\nHwGg/RHeAaATaq57jNQQ2k9X1Up1TqX7CO6EdgAIH8I7AHQi/uy2V1TWyOI06nOqxn2cwA4AkYHw\nDgCdRHO77XNHZWnK0HRJ0sx5+V677YR2AIgshHcA6CQ2HCrzepwYF6uZwzPcwb3AVuLVRYYLUQEg\n8hDeAaADyy92aO2hGl2ol5K6JrmPu3bbC2wlmumj9SN3RgWAyER4B4AObMOhMl1oVN5+/swFbVz1\nod5opouMJO6MCgARivAOAB2Y54WpVdW1qq6sUarjfJOddhdq3AEgshHeAaCDyi92SJLOnqvTmbNf\naVhxldJ9jCOwA0D0ILwDQAfk2VnmzNmvpDrj9TyBHQCiE+EdADqQxn3cq6prZYyUUnZGEqEdAKId\n4R0AOpDGN2A6XVWrlBPV6vHFOVlTk7Vh7ZQwzg4A0FaEdwDoIPKLHV7B/fyZC+pTdk7WL85JooMM\nAHQEhHcA6CA8b8KUGBerHvvsqq2t0zlJCfEWSmUAoAOwBHrC6tWrNXDgQCUmJio7O1u7du1qcbzN\nZtOYMWOUnJysPn36aPLkyTp69OhlTxgA0JTnrntVda2q/nzCqx3kLePSwjU1AEAQBRTe8/LytHjx\nYi1btkyFhYXKycnRxIkTdfz4cZ/jP/nkE02ePFkTJkxQYWGhtm/frtraWt1+++1BmTwAoIHnrnt1\nZY2SHefdjxPiLRr3rd7hmBYAIMgCCu+rVq3SnDlzNHfuXA0ZMkQvvPCCMjIytGbNGp/jCwsL5XQ6\n9fTTT2vQoEEaOXKkHnnkEX366aeqrKwMyhsAgM7OteteVV2rz45VKdUjuFutcey6A0AH4nd4v3jx\novbv36/c3Fyv47m5udq9e7fPc7797W+rW7duevXVV1VfX68vv/xSr7/+um644QalpKS0beYAAOUX\nO/TvOz7RZ8eqVFFZI9U5lVpZK6khuG9YO4VddwDoQPwO7+Xl5aqvr1d6uvf9+dLS0mS3232ek5GR\nIZvNpmXLlslqtapnz54qKirS73//+7bNGgDgDu4VlTVyOhtuwtTnVI2kS/3cAQAdS4wxxrQ+TDp5\n8qSysrK0c+dOjRs3zn18+fLl2rBhg4qLi5ucU1paqjFjxmjOnDmaOXOmzpw5o3/913+VJL333nuK\niYmRJFVXV7vP4WJWAGjdrj+X65XTMar/+u9RSUo5Ua20qhrdMi6N3XYAaAeDBw92/7pHjx7t8pp+\nt4rs3bu3YmNj5XA4vI47HA5lZGT4POeVV15R//799eyzz7qPvfHGG+rfv7/27NmjnJycy5w2AHRO\nu/5crvd2faEveiaqPuvSN4qUE9WaMby7xn1rYBhnBwAINb/De3x8vEaPHq2tW7fq7rvvdh/ftm2b\n7rnnHp/nGGNksXhX5rgeO51On+dkZ2f7OyW0Yt++fZJY02BjXUODdW1dga1EO/9cqrguVp3J6imL\npWHXPU7S0ju/0Wwfd9Y2NFjX0GBdQ4N1DQ3P6pH2ElC3mSVLluj111/Xa6+9piNHjujBBx+U3W7X\n/PnzJUlLly7Vrbfe6h4/adIk7d+/XytWrNDRo0e1f/9+zZkzR1dccYVGjx4d3HcCAB1c3qYiSVJF\nilVOS4wslhilpiRq+fdHcAMmAOgkArrD6rRp01RRUaGVK1eqrKxMI0aMkM1mU//+/SVJdrtdpaWl\n7vHjxo1TXl6ennnmGf3qV79SUlKSxo4dq3fffVeJiYnBfScA0IEV2Er096Q4nRrQXc6vQ3vPHlYl\nxsVqytD01n8DAECHEFB4l6QFCxZowYIFPp9bt25dk2NTp07V1KlTA58ZAEBSQ3Bf/9bHOjWkl3vH\nvWcPqyRp5nDf1xwBADqmgMM7AKB9ucplnF/XuPfq2bDjPnN4BrvuANDJEN4BIIIV2EpUW1unipSG\nnXZXuczGqaPCPDMAQDgEdMEqAKD9uMplJOlUn0R3uUxiXGyYZwYACBfCOwBEqMbdZXr1pM4dADo7\nwjsARChXuYwjPYnuMgAASdS8A0BEWpZXqOKvu8tIorsMAEAS4R0AIs6yvEJtLC2Xvg7urjupzh2V\nxa47AHRyhHcAiBD5xQ69uPNTlVWc9zqe3iuR4A4AkER4B4Cwyy92aMOhMtXU1ctxusbruamDemvl\ndNpCAgAacMEqAISZK7hXVdfK6TSSJIvTENwBAE0Q3gEgjPKLHaqpq5ckna6qlcVplO44r1HHviS4\nAwCaoGwGAMIkv9ih1wpPSJKqqmulOqeGlpyWJE2fMTKcUwMARCjCOwC0M88ad6khuFdU1ij9VEO9\nu9Uap7tuHxLOKQIAIhRlMwDQzjyDu9RQLpPuOK/UylpJ0vTJw8I1NQBAhGPnHQDameeOe3Vljfp4\nBPf7Z4xk1x0A0CzCOwC0s6rqWp2uaugsc+2RSvdxymUAAK0hvANAOyqwlaiisqbJcas1jnIZAECr\nCO8A0I7yNhVJA5Pdj12hnR13AIA/CO8A0A6W5RVqU2m56jyCe2pKojYsvTWMswIARBu6zQBAiBXY\nSpT/ySnVeRyzWGKUkdo1bHMCAEQnwjsAhNiL7x+V0xLjfmyxxCi9V6JmDs8I46wAANGIshkACLGT\nPRPcv85ITdL783LCOBsAQDRj5x0AQsxz133hTVeFcSYAgGjHzjsAhEiBrUQvvn9USrG6j00Zmh7G\nGQEAoh3hHQCCrMBWorxNRfp7Upwc6Unu4/yFCwBoK76XAECQvfj+UZ0c0L3JRaqTr0wN46wAAB0B\n4R0AgqjAVqKTPRPcwd1iiVGvnlb9fPzVlMwAANqMC1YBIEgKbCVa/9bHXsH92qtSCe4AgKBh5x0A\ngiRvU5EqPC5O7dXTqo1TR4VxRgCAjoaddwAIktraOp3qkyhJSk1J5A6qAICgI7wDQBC5SmZ69rBy\nB1UAQNAR3gEgCApsJV4lMxI93QEAwUd4B4AgePH9o+6e7hZLjBLjYsM8IwBAR0R4B4A2crWHdOnV\nk5IZAEBoEN4BoA3yix169OPjXu0haQ0JAAgVWkUCwGXIL3Zow6EylVWcU53H8fReiQR3AEDIsPMO\nAJdhw6Ey1dTV63RVrftYnKSFN10VvkkBADq8gMP76tWrNXDgQCUmJio7O1u7du1q9Zznn39eQ4cO\nldVqVWZmppYuXXpZkwWASJBf7FBNXb0kyek0sjiN0h3n9eTI/uy6AwBCKqCymby8PC1evFhr1qzR\nuHHj9NJLL2nixIk6fPiw+vfv7/OcJUuWaMuWLfq3f/s3jRgxQtXV1SorKwvK5AGgveUXO/Ra4QlJ\nUlV1rSxOo6ElpyVJd90+JJxTAwB0AgGF91WrVmnOnDmaO3euJOmFF17Qu+++qzVr1uipp55qMr6k\npET/8R//oYMHD2rIkEvf1EaOHNnGaQNA+2sc3Csqa5R+qkaSZLVyCREAIPT8Lpu5ePGi9u/fr9zc\nXK/jubm52r17t89zCgoKNGjQINlsNg0aNEgDBw7U7NmzderUqbbNGgDCYMOhS/9qeLqqVumO80qt\nbKh5nz55WLimBQDoRPzeKiovL1d9fb3S073rOdPS0mS3232eU1paqmPHjuntt9/Wf/7nf0qSHn74\nYd15553as2ePYmJimpyzb9++QOYPP7CmocG6hkYkr2tFdcMu+9lzdUr+f5WyfnFO5yRNvKWv+qV9\nGdFzlyJ7baMZ6xoarGtosK7BNXjw4HZ/zZD+O6/T6dSFCxf0u9/9TldffbUk6Xe/+52GDBmiffv2\n6frrrw/lywNASJw5+5Wu/OKcJCkh3qJx3+od5hkBADoLv8N77969FRsbK4fD4XXc4XAoI8P3nQQz\nMjIUFxfnDu6SdPXVVys2Nlaff/65z/CenZ3t75TQCtdP16xpcLGuoRHp65pf7FBS1xOqqq5VTIxF\nXbsmSZLunzFS2dmRfaFqpK9ttGJdQ4N1DQ3WNTSqq6vb/TX9rnmPj4/X6NGjtXXrVq/j27ZtU05O\njs9zxo0bp7q6OpWWlrqPlZaWqr6+XgMGDLjMKQNA+8kvdmjqxkL3haqnqxo6zEgNF6nSYQYA0J4C\n6vO+ZMkSvf7663rttdd05MgRPfjgg7Lb7Zo/f74kaenSpbr11lvd42+99VZ985vf1I9+9CMVFhbq\nwIED+tGPfqQxY8bwkx+AqOC6GZPU0GHG6TTq83WHGS5SBQC0t4Bq3qdNm6aKigqtXLlSZWVlGjFi\nhGw2m7vHu91u99plj4mJ0ebNm7Vo0SLddNNNSkxMVG5urlatWhXcdwEAIeB5MyZJqq6scXeYYdcd\nABAOAV+wumDBAi1YsMDnc+vWrWtyrG/fvnr77bcDnxkAhJlna8jzZy7oH45Uuh+z6w4ACIeAymYA\noLPw3HWvqq6V83C5+zl23QEA4UJ4BwAfPHfdqytr3Ddjkth1BwCED+EdABppXOue6jjv/vX9M0ay\n6w4ACJuQ3qQJAKKR5657YlysEjx23QnuAIBwYucdADw03nWfOdz3TegAAAgHwjsAfC2/2OG+GZPU\nsOseW1oVxhkBAOCN8A4AX/Msl5Eadt3zNhW5H1utVBoCAMKL8A4AX/Msl5k7KkuxpVWqra1zH6PL\nDAAg3AjvAKCGkhlPU4amN9l152JVAEC4Ed4BdHq+at0lsesOAIg4hHcAnZ6vWvfG2HUHAEQCwjuA\nTq1xa8i5o7I0ZWh6GGcEAEDzaJ0AoFNrfEOmKUPTVWAr8ap3BwAgUrDzDqDTau6GTHmbirzq3WkR\nCQCIFIR3AJ2Wr113SU2COxerAgAiBdtJADotX7vuBbYSrzEb1k5p1zkBANASdt4BdEq++rpL4o6q\nAICIRngH0Ok019ddorc7ACCyEd4BdCqNg7vUfMkMvd0BAJGG8A6gU2l8QybPvu6UzAAAIh3hHUCn\n0tINmSiZAQBEOsI7gE6juYtUfaFkBgAQiQjvADqFli5SlZrWuwMAEIko6gTQ4bV2kSp3VAUARAt2\n3gF0eK1dpOoZ3CXq3QEAkYvtJQAdXnMXqRbYSprsuE+fPIx6dwBAxCK8A+hUPC9SbdwacsPaKeGY\nEgAAfqNsBkCnRWtIAEC0IbwD6NAat4dsDqUyAIBoQHgH0KF5XqzauD0kAADRhpp3AB1WfrHD62LV\nxu0hAQCINoR3AB1W41332NIqzVz1YZPWkPR1BwBEC8pmAHRIvnbdffV0d7WHBAAgGrDdBKBDarzr\nPmVout6gpzsAIMoR3gF0SJ677kPrpZnz8r2ep6c7ACAaUTYDoMMr/b/SJndSBQAgGvEdDECHkl/s\n8CqZkdQkuFPjDgCIVgHtvK9evVoDBw5UYmKisrOztWvXLr/OO3r0qLp3767u3btf1iQBwF8bDpV5\nlcycP3PB+/m1U6hzBwBELb/De15enhYvXqxly5apsLBQOTk5mjhxoo4fP97ieRcvXtSMGTM0fvx4\nxcTEtHnCANCcxh1mzp+5IOfhcvdjymUAANHO7/C+atUqzZkzR3PnztWQIUP0wgsvKCMjQ2vWrGnx\nvEceeUSjRo3SPffcI2NMmycMAL7kFzv0WuEJ9+PEuFhl7LMrtbLWfYxyGQBAtPMrvF+8eFH79+9X\nbm6u1/Hc3Fzt3r272fO2bNmiLVu26MUXXyS4AwipxnXuM4dneNW63z9jJOUyAICo59e/IZeXl6u+\nvl7p6elex9PS0mS3232ec/LkSc2bN0+bNm1SUlKS3xPat2+f32PhH9Y0NFjX0Ah0XXecqtM2x1e6\ncKlaRndkdtEVZ4/r3Lnz7mP90r7s9P/POvv7DxXWNTRY19BgXYNr8ODB7f6aIWsVOWvWLC1YsEDX\nX399qF4CAJoE94RYaXwfatsBAB2TX9/hevfurdjYWDkcDq/jDodDGRkZPs95//33tXPnTj3xxBOS\nJGOMnE6nunTpojVr1ujHP/6xz/Oys7MDmT9a4PrpmjUNLtY1NC53XWM/+Uiuf9tLjIvVzOEZyh7a\n8K+EXbuWusd15v9ffGZDg3UNDdY1NFjX0Kiurm731/QrvMfHx2v06NHaunWr7r77bvfxbdu26Z57\n7vF5zqFDh7web9q0SU8++aT27t2rzMzMNkwZABrkF3tvKGycOsr96wJbSXtPBwCAkPP735aXLFmi\nWbNm6YYbblBOTo5efvll2e12zZ8/X5K0dOlS7d27V9u3b5ckXXvttV7n/+Uvf5HFYmlyHAAul+dF\nqolxsV7P5W0qcv+aFpEAgI7C7+9o06ZNU0VFhVauXKmysjKNGDFCNptN/fv3lyTZ7XaVlpa2+HvQ\n5x1AMHn2dJ853LuEz7PTDC0iAQAdRUDbUQsWLNCCBQt8Prdu3boWz509e7Zmz54dyMsBQLMal8xM\nGXqpG1bjkhlaRAIAOoqQdZsBgFCiZAYA0BkR3gFEnfxiByUzAIBOiS0pAFGn8a67q2SmwFbitesu\nUTIDAOhYCO8AokZ+sUMbDpX53HUvsJVo/Vsfe42nZAYA0NFQNgMgajQO7p677o133K3WOEpmAAAd\nDttSAKJC4zp3191UpYZdd8869/tnjKRcBgDQIRHeAUS8/GKHXis84X6cGBerjVNHqcBWopmrPvQK\n7lZrHMEdANBhEd4BRDzPC1Slhjp3XzXuEt1lAAAdGzXvACKeZ7nM3FFZmjI03WeNO+UyAICOjp13\nABHL1V3Gk+sCVWrcAQCdETvvACKWr+4yvhDcAQCdBeEdQERqqbsMAACdFWUzACJS47uobpw6SpLv\nu6gCANBZEN4BRCTuogoAQFOUzQCIOPnFDq/HU4amNxvcaQ0JAOhM2LICEHEal8z4Cu50mAEAdEbs\nvAOIKI0vVJ05PKNJjTvBHQDQWRHeAUSUxrvuU4am09MdAICvUTYDIGLsOFWnmroY92NfrSEJ7gCA\nzoyddwARYcepOm0++ZX7cWJcrGJLqzRzXn4YZwUAQGQhvAOICNscX3k9dtW6e5bM0BYSANDZEd4B\nhF1+sUMXLl2jqrmjsprUutMWEgAAat4BhFl+sUOvFZ5wP3ZdpFpgK/Eat2HtlPaeGgAAEYeddwBh\n5dldRrp0kapne0jKZQAAaEB4BxA2jXu635HZxb3r7lkyQ7kMAAANCO8AwsZz1z0hVhrfp2GHvfGu\nO+0hAQBoQHgHEDaeu+63pXdx/5pddwAAfCO8A4gIrl33xth1BwDgEq4CA9Du8osdTS5UlaRdfy7X\nqrXclAkAgOaw8w6g3W04VOZVMpMYFytJem/XF9yUCQCAFhDeAbS7xsF95vAM7fpzuS5cdLqPc1Mm\nAACaYlsLQLvKL3Z4Pd44dZQkadITX7iPWa1x3JQJAAAf2HkH0K48a91d5TIFthKvXXd23AEA8I3w\nDqBdeZbMNHc3VTrMAADgG+EdQLtpXDLD3VQBAAgMNe8AQs7VGtJz1/38mQuaOS/fK7gnxFvYdQcA\noAUB77yvXr1aAwcOVGJiorKzs7Vr165mx/7xj3/UXXfdpczMTHXt2lUjR47UunXr2jRhANGncXCX\npNijlV7BXZJuGZfWntMCACDqBBTe8/LytHjxYi1btkyFhYXKycnRxIkTdfz4cZ/j9+zZo5EjR+qd\nd95RUVGRFixYoHnz5um//uu/gjJ5ANGhcWvIuaOylOw47z5mtcZp4i19Ne5bvcMxPQAAokZA4X3V\nqlWaM2eO5s6dqyFDhuiFF15QRkaG1qxZ43P80qVLtXz5co0dO1ZXXnml5s+frylTpuidd94JyuQB\nRD5frSFjS6u8jm1YO4XgDgCAH/wO7xcvXtT+/fuVm5vrdTw3N1e7d+/2+wWrq6uVkpLi/wwBRK38\nYodeKzzhfuxqDdm4uwwAAPCP3981y8vLVV9fr/T0dK/jaWlpstvtfv0emzdv1nvvvRdQ2AcQnRoH\nd6mhNSTdZQAAuHzttuX14Ycf6oc//KFefPFFZWdnNztu37597TWlToM1DQ3WtXk7TtVp88mvvI7d\nkdlFV5w9ruW/O+y+IVNCvEX90r70WkvWNXRY29BgXUODdQ0N1jW4Bg8e3O6v6XfZTO/evRUbGyuH\nw7t+1eFwKCMjo8Vzd+3apdtvv10rVqzQP/3TP13eTAFEheaC+/g+DXsFnndSpbsMAACB8XvnPT4+\nXqNHj9bWrVt19913u49v27ZN99xzT7Pn7dy5U3fccYeWL1+uRYsWtfo6Le3KIzCun65Z0+BiXVv2\nzMZCJXXt4n48d1SWpgy9VG7XtWup+9eLf/Zd969Z19BhbUODdQ0N1jU0WNfQqK6ubvfXDKhsZsmS\nJZo1a5ZuuOEG5eTk6OWXX5bdbtf8+fMlNXSX2bt3r7Zv3y6poc/79773PT3wwAP6wQ9+4K6Nj42N\nVZ8+fYL8VgCEW36xw6stZOPgXmArCce0AADoMAIK79OmTVNFRYVWrlypsrIyjRgxQjabTf3795ck\n2e12lZZe2lVbv369amtr9dxzz+m5555zH7/yyiu9xgGIfr46y7iCe4GtRHmbirwuVKXLDAAAgQv4\nu+eCBQu0YMECn881vnvqunXruKMq0ElsOFTm9Xjm8EvXwjQO7hJdZgAAuBxsfQEIiubKZRq3hrRa\n4zR98jDddfuQdp8jAADRjvAOoE3yix1Ndt2nDE1vtlRmw9op7T1FAAA6DMI7gDbZcKjMa9c9MS5W\nBbYSrX/r4yZjKZUBAKBt/O7zDgCNNe4ukxgXq5nDM5S3qchrnNUap/tnjKRUBgCANmLnHUBAXGUy\nnqFdagjus5ISlbfqQ69SGUI7AADBQ3gHEBBfwV2ShtarSamM1RpHcAcAIIgI7wAC0ji4nz9zQbFH\nK3XYcd7ruKurDAAACB7COwC/5Rc7vB5vmTFaM+flN+nhTqkMAAChQXgH4DfPlpCurjL0cAcAoP0Q\n3gG0qLkLVBvXuNPDHQCA0CO8A2hR4+BeVV2r6soaJRyp9BpHfTsAAKFHeAfQrMZ93Kuqa3W6/Lz6\nnKrxGkeNOwAA7YPwDsCn/GKHXis84X58/swFpX/4d6V7jKHGHQCA9kV4B+CT58WpVdW1ijtS4fU8\nu+0AALSzIp89AAARp0lEQVQ/wjsAN8+LU6uqa3W6qlZOp1G647xSK2vd4wjuAACEB+EdgApsJXrx\n/aM6kWJt8pzFaQjuAABECEu4JwAgvApsJVr/1sc62TOhyXMWp3FfnGq1xhHcAQAIM3begU5qWV6h\nNpWWq06Srknxes5VJsMFqQAARBbCO9BJuYO7h9SURGWkdtXGpaPCMicAANAywjvQCeUXO7yCu8US\no149rcpI7aqZwzPCNi8AANAywjvQSeQXO/Tizk/lOF0jp9O4j1ucRoeX3hrGmQEAAH9xwSrQCbhu\nuNQ4uEtSZtWFMM0KAAAEip13oINzBfeq6tomO+6ZVRe08ObBYZwdAAAIBOEd6MCW5RUq//9VeIX2\ndMd59Ttfpw1rp4RxZgAA4HIQ3oEOqLmbLrlaQE6fMTJMMwMAAG1BeAeiVIGtRHmbilRb27jho1SR\nYpUjPcnrmGvHfTo3WgIAIGoR3oEo5Lorqi+Ng7vFEqMpV6ZqJb3bAQCIeoR3IEK1tLPuS0WKVaf6\nJMppiZF0qXf7z8dfrSlD00M5VQAA0E4I70CE8je43z9jpOoH9dRrhSfUrdFzc0dlEdwBAOhA6PMO\nRKACW0mrwd1qjdP9X9evbzhU5vVcYlwswR0AgA6InXcggvgqlbFa45pt65hf7NDUjYWqqat3HyO0\nAwDQcRHegQjQUn379MnDJDUE9Q2HyryCemOJcbEEdwAAOjDCOxABfAV3qzVO0ycPU/2gnk12131J\njIvVzOEZoZwmAAAIM8I7EEL+doypSLHq1IDuclpi3F1ievawau2Zs1Lh2RbPdYV2dtwBAOj4CO9A\nCATS5tGzL7vFEqOBA3o2O5agDgBA50Z4B9og0F7sjTUO7r16Wn2OI7QDAACJ8A60yFc4P3fuvCSp\na9fSVs+vSLGqIj1JPVIS1bNH02De7esvFzrFAACAlgTc53316tUaOHCgEhMTlZ2drV27drU4/uDB\ngxo/frySkpKUlZWlFStWXPZkgfZUYCvR+rc+9rv0pXhILx2+JsX9VTwsVXXXpGrAwF4+g3tjBHcA\nANCagHbe8/LytHjxYq1Zs0bjxo3TSy+9pIkTJ+rw4cPq379/k/FnzpzRbbfdpgkTJmjfvn06cuSI\n5syZo65du2rJkiVBexNAKORtKmpyrCLFKvvVPWRiY2SxNPzs6yp3GehHQPeFkhgAAOCvgML7qlWr\nNGfOHM2dO1eS9MILL+jdd9/VmjVr9NRTTzUZ/+abb6q2tlbr169XQkKCrr32WhUXF2vVqlWEd4RN\na3XqFSlWneqTKOfAZPex1K/LXrpJyvy6bCapa5Jfr0c4BwAAweJ3eL948aL279+vX/ziF17Hc3Nz\ntXv3bp/n7NmzRzfeeKMSEhK8xv/yl7/UsWPHNGDAgMucNjqjQDu4nOqTKKclxvcAj2DeGoslxq+y\nF0I6AAAINb/De3l5uerr65We7h1M0tLSZLfbfZ5jt9t1xRVXeB1znW+32yM2vPtzJ8tocP5cjSQp\n6ZOPwjyTtqmqrtXpqlo5nSag0B0MvjrAJMRKt6V30c8njm7XuQAAAIS020xMTDO7ni2orq4OwUwC\n850Mq76TMTDc00CEi4TPakcyePBgSaxrKLC2ocG6hgbrGhqsa8fhd7eZ3r17KzY2Vg6Hw+u4w+FQ\nRobvW7L37du3ya686/y+ffsGOlcAAACgU/M7vMfHx2v06NHaunWr1/Ft27YpJyfH5zljx47VBx98\noAsXLniN79evX8SWzAAAAACRKsYYY/wd/Pbbb2vWrFlavXq1cnJy9PLLL2vdunUqKipS//79tXTp\nUu3du1fbt2+X1NAqcsiQIZowYYKWLVumkpISzZkzR48//rgeeuihkL0pAAAAoCMKqOZ92rRpqqio\n0MqVK1VWVqYRI0bIZrO5e7zb7XaVll6662RycrK2bdumn/3sZ8rOzlZKSooefvhhgjsAAABwGQLa\neQcAAAAQPn7XvLfmJz/5ia6++molJSUpLS1NkydP1pEjR7zGnD59WrNmzVLPnj3Vs2dP3XfffU2u\nev7888915513qlu3burTp48efPBBffXVV15jDh48qPHjxyspKUlZWVlasWJFk/ns2LFDo0ePVmJi\noq666iq98sorwXqr7eb06dNauHChrrnmGiUlJemKK67QT3/6U1VWVjYZx7oGZu3atbr55pvVs2dP\nWSwWff75503GsK7ta/Xq1Ro4cKASExOVnZ2tXbt2hXtKYbNz505NmjRJWVlZslgsWr9+fZMxjz/+\nuPr166ekpCTdfPPNOnz4sNfzFy5c0MKFC9WnTx9169ZNd911l/7+9797jQnWZzwaPP3007r++uvV\no0cPpaWladKkSSoqanoXZdY1cC+99JJGjhypHj16qEePHsrJyZHNZvMaw7q23dNPPy2LxaKFCxd6\nHWdtA/P444/LYrF4fWVmZjYZE9FraoLklVdeMbt27TLHjh0z+/fvN5MmTTKZmZnmq6++co/57ne/\na4YPH27+9Kc/mT179phhw4aZO++80/18XV2dGT58uLn55pvNgQMHzLZt20xmZqZZuHChe0x1dbVJ\nT08306dPN0VFRWbjxo2me/fu5t///d/dY0pLS01SUpJZtGiRKS4uNq+++qrp0qWLeeedd4L1dtvF\noUOHzJQpU8zvf/978+mnn5odO3aYYcOGmdzcXK9xrGvgnn/+efPMM8+Y559/3sTExJhjx441GcO6\ntp+33nrLdOnSxfzmN78xxcXFZuHChaZbt27m888/D/fUwsJms5lHH33UbNy40SQlJZn169d7Pf/M\nM8+Y7t27m/z8fHPo0CEzbdo0k5mZab788kv3mPnz55vMzEyzfft2s3//fjNhwgQzatQoU19f7x4T\njM94tPjHf/xH8/rrr5uioiJz8OBB8/3vf9/07dvXVFZWusewrpenoKDAvPvuu+bTTz81R48eNY8+\n+qjp0qWLKSwsNMawrsGwZ88eM3DgQDNy5Eiv98PaBu6xxx4z11xzjXE4HO6v8vJy9/PRsKZBC++N\nffzxxyYmJsb87W9/M8YYc/jwYRMTE2N2797tHrNr1y6vMTabzVgsFnPixAn3mDfeeMNYrVb3oq1e\nvdr06NHD1NbWusesXLnS9OvXz/34F7/4hfmHf/gHr/n8+Mc/NmPHjg3+G21nrjVyrQfr2jZ79+71\nGd5Z1/Z1ww03mHnz5nkdGzx4sFm6dGmYZhQ5unXr5hXenU6n6du3r3nqqafcx2pqakz37t3NK6+8\nYowxpqqqysTHx5sNGza4xxw/ftxYLBbzhz/8wRgTvM94tDp79qyJjY01mzdvNsawrsGWkpJi1q5d\ny7oGQVVVlbnqqqvMH//4RzNhwgR3uGNtL89jjz1mhg8f7vO5aFnToJXNeDp37pzWrVunwYMHa+DA\nhpsd7dmzR926ddPYsWPd43JyctS1a1ft3r3bPebaa69Vv3793GNyc3N14cIFffTRR+4xN954oxIS\nErzGnDx5UseOHXOPyc3N9ZpTbm6u9u3bp/r66L5ranV1tRISEpSUlCSJdQ0V1rX9XLx4Ufv37/e5\nBq61xiWfffaZHA6H13pZrVbddNNN7vX66KOP9NVXX3mNycrK0jXXXKM9e/ZICt5nPFqdOXNGTqdT\nvXr1ksS6Bkt9fb3eeust1dbW6qabbmJdg2DevHm65557NH78eBmPyxRZ28tXWlqqfv36adCgQfrB\nD36gzz77TFL0rGlQw/vq1avVvXt3de/eXZs3b9aWLVsUF9fQ0MZut6tPnz5e42NiYpSWlua+kZPd\nbld6errXGNfNoVoa43rsGuNwOHyOqaurU3l5eZDebfurqqrSL3/5S82bN08WS8P/OtY1NFjX9lNe\nXq76+voma+C51rjEtSYtrZfdbldsbKxSU1O9xqSnp3uNCcZnPFo9+OCDuu6669zfXFnXtjl48KC6\ndesmq9WqefPm6e2339aQIUNY1zZ69dVXVVpaqpUrV0ryvnM9a3t5xowZo/Xr1+sPf/iDXn31Vdnt\nduXk5KiysjJq1rTF8L5s2bImRf2Nv3bu3Okef++996qwsFA7duzQtddeq4kTJ+rLL79scQKNmVaa\n33h+cKNVoOsqSWfPntWdd96p/v3761e/+lXAr8m6+l7XtuoM64ro0drnrbXPa7DOiXRLlizR7t27\n9c477/j1Z5R1bd3QoUP117/+VX/5y1/0wAMPaMaMGdq3b1+L57CuLSspKdGjjz6qN998U7GxsZIa\n3p8/75G1bd53v/tdTZ06VcOHD9d3vvMdbdmyRU6n02djAE+RtKYthveHHnpIxcXFLX5df/317vHJ\nycm66qqrdOONN2rjxo0qKyvTf//3f0uS+vbtq1OnTjWZ9BdffKG+ffu6xzgcDq8xrl05zzGNfyJx\nndPamLi4OPXu3du/lQmhQNf17Nmzuv3222WxWLR582bFx8e7n2NdLwl0XVvCurYf105D47V0OBzK\nyMgI06wil+tz42u9PD9T9fX1qqioaHFMMD7j0eahhx5SXl6e3nvvPV155ZXu46xr23Tp0kWDBg3S\nddddp6eeekpjxozRSy+95P4zzLoGbs+ePSovL9ewYcPUpUsXdenSRTt37tTq1asVHx/v/v7A2rZN\nUlKShg0bpk8++SR6Pq8tVsS3QW1trUlKSjK//e1vjTG+i/c//PBDr+L9//3f/21SvP/mm296Fe+v\nWbPGJCcne10A+OSTT5qsrCz340ceeaTJBYA/+clPTE5OTvDfaIidOXPGfPvb3zbjxo0zZ8+ebfI8\n69o2gVywyrqGzre+9S2fF6z+y7/8S5hmFDl8XbCakZHR5IKq5ORks3btWmNMyxdUbd261RgTvM94\nNFm0aJHJyMgwxcXFTZ5jXYPr5ptvNvfdd58xxrCul6mqqsoUFRW5vw4dOmSuv/5688Mf/tAUFRXx\nmQ2Smpoa07dvX7NixQpjTHR8XoMS3j/55BPzzDPPmI8++sgcO3bMfPjhh+bOO+80KSkp5osvvnCP\nmzhxohkxYoTZs2eP2b17txk+fLiZNGmS+/n6+nozYsQIc8stt7jb5vTr188sWrTIPaa6utr07dvX\nzJgxwxw6dMi88847Jjk52axatco95rPPPjNdu3Y1ixcvNocPHzavvvqqiY+PN/n5+cF4u+3mzJkz\nZsyYMWbYsGHm6NGjpqyszP118eJF9zjWNXBlZWXmwIED5s033zQxMTHGZrOZAwcOeLWNY13bT15e\nnomPjze/+c1vzOHDh82iRYtM9+7dO22ryLNnz5oDBw6YAwcOmKSkJLN8+XJz4MAB93o8++yzpkeP\nHiY/P98cPHjQTJ8+3fTr18/rB/wFCxaYrKwsr1Zm1113nXE6ne4xwfiMR4uf/vSnJjk52bz33nte\nf5d6rhnrenkeeeQR88EHH5jPPvvM/PWvfzX//M//7BVkWNfgGT9+vHnggQfcj1nbwP385z83O3bs\nMKWlpeZPf/qT+d73vmd69OgRVX+/BiW8Hz9+3EycONGkpaWZ+Ph4079/f3PvvfeakpISr3GnT582\n9957r0lOTjbJyclm1qxZprq62mvM559/bu644w6TlJRkUlNTzYMPPugVVI0x5uDBg+amm24yVqvV\nZGZmmuXLlzeZ044dO8w3v/lNk5CQYAYNGuRu8RNN3n//fRMTE2MsFouJiYlxf1ksFrNjxw73ONY1\ncI899pjXerr+67nDybq2r9WrV5srr7zSJCQkmOzsbPPBBx+Ee0ph4/qz3/jP/5w5c9xjHn/8cZOR\nkWGsVquZMGGCKSoq8vo9Lly4YBYuXGhSU1NNUlKSmTRpktcOjzHB+4xHA19/l8bExJgnnnjCaxzr\nGrjZs2ebAQMGmISEBJOWlmZuu+02d3B3YV2Dw7NVpAtrG5gZM2aYzMxMEx8fb/r162emTp1qjhw5\n4jUm0tc0xpgOcgUCAAAA0MGFpM87AAAAgOAjvAMAAABRgvAOAAAARAnCOwAAABAlCO8AAABAlCC8\nAwAAAFGC8A4AAABECcI7AAAAECUI7wAAAECU+P++rJzJDw3WfgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "MakePrice2(player1, player2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, we use the variance of `diff` to estimate the variance\n", + "of `error`. This estimate is not perfect because contestants’\n", "bids are sometimes strategic; for example, if Player 2 thinks that\n", "Player 1 has overbid, Player 2 might make a very low bid. In that case\n", - "diff does not reflect error. If this happens a\n", - "lot, the observed variance in diff might overestimate the\n", - "variance in error. Nevertheless, I think it is a reasonable\n", + "`diff` does not reflect `error`. If this happens a\n", + "lot, the observed variance in `diff` might overestimate the\n", + "variance in `error`. Nevertheless, I think it is a reasonable\n", "modeling decision.\n", "\n", "As an alternative, someone preparing to appear on the show could\n", - "estimate their own distribution of error by watching\n", - "previous shows and recording their guesses and the actual prices.\n", - "\n", - "Likelihood\n", - "----------\n", + "estimate their own distribution of `error` by watching\n", + "previous shows and recording their guesses and the actual prices." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Likelihood\n", "\n", "Now we are ready to write the likelihood function. As usual, I define a\n", - "new class that extends thinkbayes.Suite:\n", - "\n", - " class Price(thinkbayes.Suite):\n", - "\n", - " def __init__(self, pmf, player):\n", - " thinkbayes.Suite.__init__(self, pmf)\n", - " self.player = player\n", - "\n", - "pmf represents the prior distribution and\n", - "player is a Player object as described in the previous\n", - "section. Here’s Likelihood:\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " price = hypo\n", - " guess = data\n", + "new class that extends `thinkbayes.Suite`:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Price(thinkbayes.Suite):\n", "\n", - " error = price - guess\n", - " like = self.player.ErrorDensity(error)\n", + " def __init__(self, pmf, player):\n", + " thinkbayes.Suite.__init__(self, pmf)\n", + " self.player = player\n", + " \n", + " def Likelihood(self, data, hypo):\n", + " price = hypo\n", + " guess = data\n", "\n", - " return like\n", + " error = price - guess\n", + " like = self.player.ErrorDensity(error)\n", "\n", - "hypo is the hypothetical price of the showcase.\n", - "data is the contestant’s best guess at the price.\n", - "error is the difference, and like is the\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pmf` represents the prior distribution and\n", + "`player` is a Player object as described in the previous\n", + "section. In `Likelihood` `hypo` is the hypothetical price of the showcase.\n", + "`data` is the contestant’s best guess at the price.\n", + "`error` is the difference, and `like` is the\n", "likelihood of the data, given the hypothesis.\n", "\n", - "ErrorDensity is defined in Player:\n", + "`ErrorDensity` is defined in `Player`:\n", "\n", - " # class Player:\n", + "```python\n", + "# class Player:\n", "\n", - " def ErrorDensity(self, error):\n", - " return self.pdf_error.Density(error)\n", + " def ErrorDensity(self, error):\n", + " return self.pdf_error.Density(error)\n", + "```\n", "\n", - "ErrorDensity works by evaluating `pdf_error` at the given\n", - "value of error. The result is a probability density, so it\n", - "is not really a probability. But remember that Likelihood\n", + "`ErrorDensity` works by evaluating `pdf_error` at the given\n", + "value of `error`. The result is a probability density, so it\n", + "is not really a probability. But remember that `Likelihood`\n", "doesn’t need to compute a probability; it only has to compute something\n", - "*proportional* to a probability. As long as the constant of\n", + "*proportional* to a probability. As long as the constant of\n", "proportionality is the same for all likelihoods, it gets canceled out\n", "when we normalize the posterior distribution.\n", "\n", - "And therefore, a probability density is a perfectly good likelihood.\n", - "\n", - "Update\n", - "------\n", - "\n", - "![Prior and posterior distributions for Player 1, based on a best guess\n", - "of \\$20,000.](figs/price3.pdf)\n", - "\n", - "[fig.price3]\n", + "And therefore, a probability density is a perfectly good likelihood." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Update\n", "\n", - "Player provides a method that takes the contestant’s guess\n", + "`Player` provides a method that takes the contestant’s guess\n", "and computes the posterior distribution:\n", "\n", - " # class Player\n", + "```python\n", + "# class Player\n", "\n", - " def MakeBeliefs(self, guess):\n", - " pmf = self.PmfPrice()\n", - " self.prior = Price(pmf, self)\n", - " self.posterior = self.prior.Copy()\n", - " self.posterior.Update(guess)\n", + " def MakeBeliefs(self, guess):\n", + " pmf = self.PmfPrice()\n", + " self.prior = Price(pmf, self)\n", + " self.posterior = self.prior.Copy()\n", + " self.posterior.Update(guess)\n", + "```\n", "\n", - "PmfPrice generates a discrete approximation to the PDF of\n", + "`PmfPrice` generates a discrete approximation to the PDF of\n", "price, which we use to construct the prior.\n", "\n", - "PmfPrice uses MakePmf, which evaluates\n", + "`PmfPrice` uses `MakePmf`, which evaluates\n", "`pdf_price` at a sequence of values:\n", "\n", - " # class Player\n", + "```python\n", + "# class Player\n", "\n", - " n = 101\n", - " price_xs = numpy.linspace(0, 75000, n)\n", + " n = 101\n", + " price_xs = numpy.linspace(0, 75000, n)\n", "\n", - " def PmfPrice(self):\n", - " return self.pdf_price.MakePmf(self.price_xs)\n", + " def PmfPrice(self):\n", + " return self.pdf_price.MakePmf(self.price_xs)\n", + "```\n", "\n", "To construct the posterior, we make a copy of the prior and then invoke\n", - "Update, which invokes Likelihood for each\n", + "`Update`, which invokes `Likelihood` for each\n", "hypothesis, multiplies the priors by the likelihoods, and renormalizes.\n", "\n", "So let’s get back to the original scenario. Suppose you are Player 1 and\n", @@ -421,8 +861,8 @@ "posterior is somewhere in between: \\$25,096.\n", "\n", "On another level, you might find this result bizarre, because it\n", - "suggests that if you *think* the price is \\$20,000, then\n", - "you should *believe* the price is \\$24,000.\n", + "suggests that if you *think* the price is \\$20,000, then\n", + "you should *believe* the price is \\$24,000.\n", "\n", "To resolve this apparent paradox, remember that you are combining two\n", "sources of information, historical data about past showcases and guesses\n", @@ -433,10 +873,14 @@ "update it based on historical data.\n", "\n", "If you think of it that way, maybe it is less surprising that the most\n", - "likely value in the posterior is not your original guess.\n", - "\n", - "Optimal bidding\n", - "---------------\n", + "likely value in the posterior is not your original guess." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Optimal bidding\n", "\n", "Now that we have a posterior distribution, we can use it to compute the\n", "optimal bid, which I define as the bid that maximizes expected return\n", @@ -448,86 +892,180 @@ "shortly.\n", "\n", "To compute optimal bids, I wrote a class called\n", - "GainCalculator:\n", - "\n", - " class GainCalculator(object):\n", - "\n", - " def __init__(self, player, opponent):\n", - " self.player = player\n", - " self.opponent = opponent\n", - "\n", - "player and opponent are Player\n", + "`GainCalculator`:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class GainCalculator(object):\n", + "\n", + " def __init__(self, player, opponent):\n", + " self.player = player\n", + " self.opponent = opponent\n", + " \n", + " def ExpectedGains(self, low=0, high=75000, n=101):\n", + " bids = numpy.linspace(low, high, n)\n", + "\n", + " gains = [self.ExpectedGain(bid) for bid in bids]\n", + "\n", + " return bids, gains \n", + "\n", + " def ExpectedGain(self, bid):\n", + " suite = self.player.posterior\n", + " total = 0\n", + " for price, prob in sorted(suite.Items()):\n", + " gain = self.Gain(bid, price)\n", + " total += prob * gain\n", + " return total\n", + " \n", + " def Gain(self, bid, price):\n", + " # if you overbid, you get nothing\n", + " if bid > price:\n", + " return 0\n", + "\n", + " # otherwise compute the probability of winning\n", + " diff = price - bid\n", + " prob = self.ProbWin(diff)\n", + "\n", + " # if you are within 250 dollars, you win both showcases\n", + " if diff <= 250:\n", + " return 2 * price * prob\n", + " else:\n", + " return price * prob\n", + " \n", + " def ProbWin(self, diff):\n", + " prob = (self.opponent.ProbOverbid() +\n", + " self.opponent.ProbWorseThan(diff))\n", + " return prob" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`player` and `opponent` are `Player`\n", "objects.\n", "\n", - "GainCalculator provides ExpectedGains, which\n", - "computes a sequence of bids and the expected gain for each bid:\n", - "\n", - " def ExpectedGains(self, low=0, high=75000, n=101):\n", - " bids = numpy.linspace(low, high, n)\n", - "\n", - " gains = [self.ExpectedGain(bid) for bid in bids]\n", - "\n", - " return bids, gains\n", - "\n", - "low and high specify the range of possible\n", - "bids; n is the number of bids to try.\n", - "\n", - "ExpectedGains calls ExpectedGain, which\n", - "computes expected gain for a given bid:\n", - "\n", - " def ExpectedGain(self, bid):\n", - " suite = self.player.posterior\n", - " total = 0\n", - " for price, prob in sorted(suite.Items()):\n", - " gain = self.Gain(bid, price)\n", - " total += prob * gain\n", - " return total\n", + "`GainCalculator` provides `ExpectedGains`, which\n", + "computes a sequence of bids and the expected gain for each bid:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`low` and `high` specify the range of possible\n", + "bids; `n` is the number of bids to try.\n", "\n", - "ExpectedGain loops through the values in the posterior and\n", + "`ExpectedGains` calls `ExpectedGain`, which\n", + "computes expected gain for a given bid:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`ExpectedGain` loops through the values in the posterior and\n", "computes the gain for each bid, given the actual prices of the showcase.\n", "It weights each gain with the corresponding probability and returns the\n", "total.\n", "\n", - "![Expected gain versus bid in a scenario where Player 1’s best guess is\n", - "\\$20,000 and Player 2’s best guess is \\$40,000.](figs/price5.pdf)\n", - "\n", - "[fig.price5]\n", - "\n", - "ExpectedGain invokes Gain, which takes a bid\n", - "and an actual price and returns the expected gain:\n", - "\n", - " def Gain(self, bid, price):\n", - " if bid > price:\n", - " return 0\n", - "\n", - " diff = price - bid\n", - " prob = self.ProbWin(diff)\n", - "\n", - " if diff <= 250:\n", - " return 2 * price * prob\n", - " else:\n", - " return price * prob\n", - "\n", + "`ExpectedGain` invokes `Gain`, which takes a bid\n", + "and an actual price and returns the expected gain." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Player 1 optimal bid (16668.781463564384, 21000.0)\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwMAAAEWCAYAAAA6pksiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VFX6wPHvzCSZ9ARCChBqCL1ICZLQEUIHxUIRIiIL\nK8qCZVl3LYi7soqrKLIr6E8EQUBFBJEWigghoQUiLfQSWkIS0ntm7u+PSyYZaiBlZpL38zzzZM69\n5868uYHkvnPPeY9GURQFIYQQQgghRLWjtXQAQgghhBBCCMuQZEAIIYQQQohqSpIBIYQQQgghqilJ\nBoQQQgghhKimJBkQQgghhBCimpJkQAghhBBCiGpKkgEhhBBCCCGqqXsmA//+978JCgrCw8MDHx8f\nhg0bxrFjx27r9+6771K3bl2cnZ3p3bs3x48fN9ufl5fH1KlT8fb2xtXVleHDh3PlyhWzPikpKYwb\nNw5PT088PT0JCwsjLS3NrE9cXBxDhw7F1dUVb29vpk2bRkFBwcN+70IIIYQQQlRr90wGfv/9d15+\n+WWioqLYvn07dnZ29O3bl5SUFFOfDz/8kE8++YT58+ezf/9+fHx86NevH5mZmaY+06dPZ/Xq1axc\nuZJdu3aRnp7OkCFDMBqNpj5jxowhJiaGzZs3s2nTJg4ePMi4ceNM+w0GA4MHDyYrK4uIiAhWrFjB\nqlWreO2118rzfAghhBBCCFFtaB5kBeKsrCw8PDxYu3YtgwcPRlEU6tSpw1/+8hf+/ve/A5Cbm4uP\njw//+c9/mDRpEmlpafj4+LB48WJGjx4NwOXLl2nQoAEbN24kNDSU2NhYWrVqxe7duwkODgZg9+7d\ndO/enZMnTxIYGMjGjRsZMmQIcXFx1K1bF4DvvvuOiRMnkpiYiKura3mfGyGEEEIIIaq0B5ozkJ6e\njtFopEaNGgCcP3+ehIQEQkNDTX0cHR3p0aMHkZGRAERHR1NQUGDWx9/fnxYtWhAVFQVAVFQUrq6u\npkQAICQkBBcXF9PrREVF0bJlS1MiABAaGkpeXh7R0dEP+n0LIYQQQghR7T1QMjBt2jTat29vumiP\nj48HwNfX16yfj4+PaV98fDw6nQ4vLy+zPr6+vmZ9vL29zfZrNJrbXufW96lVqxY6nc7URwghhBBC\nCFF6dqXt+OqrrxIZGUlERAQajea+/e/X5wFGJz3wMbdOPBZCCCGEEMKWeHh4VMr7lOrOwCuvvML3\n33/P9u3badiwoWm7n58fAAkJCWb9ExISTPv8/PwwGAwkJyffs09iYqLZfkVRuH79ulmfW98nKSkJ\ng8Fg6iOEEEIIIYQovfsmA9OmTTMlAk2bNjXb16hRI/z8/AgPDzdty83NJSIigpCQEAA6duyIvb29\nWZ/Lly9z4sQJU5/g4GAyMzNNcwhAnSOQlZVl6hMSEkJsbKxZSdItW7ag1+vp2LHjw3zvQgghhBBC\nVGv3rCb00ksvsWzZMtasWUOLFi1M293c3HBxcQFgzpw5zJ49m2+++YbAwED+9a9/ERERwcmTJ019\npkyZwrp161i8eDE1a9bk1VdfJS0tjejoaNNwokGDBnH58mW+/PJLFEVh0qRJNG7cmLVr1wJgNBp5\n5JFH8Pb25uOPPyYpKYnx48fz5JNP8tlnn5nFXXKYUGXdYqkODhw4AECnTp0sHEnVIue1Ysh5rRhy\nXiuGnNeKIee1Ysh5rTiWuIa955yBL774Ao1Gw2OPPWa2/d133+Wdd94BYMaMGeTk5PDSSy+RkpJC\nly5dCA8PNyUCAJ9++il2dnaMHDmSnJwc+vbty7Jly8zmFSxfvpypU6fSv39/AIYPH878+fNN+7Va\nLevXr2fKlCl07doVJycnxo4dy0cffVT2syCEEEIIIUQ1dM9koOSiYPcyc+ZMZs6cedf9Dg4OzJs3\nj3nz5t21j6enJ0uXLr3n+9SrV49169aVKiYhhBBCCCHEvT1QaVEhhKiqDAaF+GSFtMwHr3QmhBBC\n2KpSlxYVQoiqIC1T4dApuHQdriTC5etwNQmuJUGhQe3jW1MhsB40qw9N66tf69S6f8lkIYQQwtZI\nMiCEqBZOX1L4fhts3gu5effum3BDfUT8UbzN3QUe76Hw3CBwd5GkQAghRNUgyYAQosoyGGF7tJoE\nHDxx//5eHpCZA3n5t+9Lz4JvN8LPv8NzgxRGPgaOekkKhBAPx2g0kp9/h182NqBBgwaAWk5elJ6D\ngwNarfWN0JdkQAhR5eQXKKzfV5Oth2qQa7h9f9P60C4Q/L3B3wfq1oI63uCk12AwKFy4Bqcuwck4\n9eupOEjLVI/NyIb5q+D7bTBxqMKwbmBnJ0mBEKL0FEUhLy8PR0dHmxx+6OjoaOkQbI6iKOTm5lrl\nz1ySASFElXLiosKsr+GPUz4AuDir23U66N0BRj6mJgJ3+2Ws02kI8IcAfxgYrG4zGhW27ocFa+DS\nzYXQE1Pg39/Cd5th6tMKvTpY1y93IYT1ys/Px8HBweouCkXF0Wg0ODg4kJ+fj16vt3Q4ZiQZEEJU\nCYWFCt9sgEXriicCA3i6wYheMKIn+NZ8uD+8Wq2G0EehT0eFXyLgq18gKVXdF5cAf50PLwxVmPy4\nTDIWQtyfoijodDpLhyEqmU6no6CgwNJh3EaSASGEzTt3ReHdryH2QvE2vZ3CU92v8/rzjdE7lM8F\nup2dhhG9YFCwOg9hyQZ12BDA1+vgegr8I0yRYUNCCCFshiQDQgibZTAoLN8CC36G/BIftrQNhCc6\nnsOvZkG5JQIlOeo1PDdIrS709lcQdUTdvi4CktLggxcVnB0lIRBCCGH9rG9KsxBClEJmtsJfPoV5\nPxQnAvZ2MPVp+HIG+NWs+FuxHq4aPpkKQ7sVb4s6An+eA8lpsniZEEII6yfJgBDC5lxPUZj0Iew7\nVryteQNY+g6EDdSg01Xep/J2dhrefh4mDiveFnsBXpgNF+MlIRBCiDvZsWMHWq2WnTt3WjqUak+S\nASGETTl7WeGF2XD6UvG254fAN29CgL9lhuZoNBomP67hH89BUQnpK4kw8d9w7JwkBEKI6mPx4sVo\ntVrTw97ennr16jFhwgSuXr1q6fDKzb59+3jppZcICgpCr9ej1Wq5fv26pcN6KJIMCCFsxsGTCn/6\nAOKT1badDt6ZAFNGaKxi0u4TPTX852VwvFk1LjUDXvtcvZMhhBDVyaxZs1i2bBkLFy6kX79+fPvt\nt3Tv3p2cnBxLh1YuNmzYwFdffYXRaKRZs2Y2XUlOkgEhhE3Yul/h5Y+Lq/c4O8In02BoN+v6Bdz9\nEQ0L/goermo7OQ1mzIe8fEkIhBDVR//+/RkzZgwTJkxg0aJFTJ8+nfPnz7N27VpLh1Yq+fn5GAx3\nWLXypilTppCRkUF0dDQjRoxAUWz3d7wkA0IIq7c8XOEfC6CgUG17ecDCv0Fwa+tKBIq0aqzhwynq\nQmcAx87Dv5di038shBCiLHr37g3AhQsX7trn8OHDPP/88wQEBODk5IS3tzejR4/m0qXicaGnT59G\nq9Uyd+7cOx6v1WpZuHChadu1a9eYOHEifn5+ODo60rJlSxYsWGB2XNH8heXLl/Puu+9Sv359nJ2d\nuXLlyl1j9fHxsbrFwx6WlBYVQli1Rb8qfLG6uN2wNnw6Hep6W2ciUKRjcw2vjFT4z3K1vX43NKsP\no/tZNi4hhLCEs2fPAuDl5XXXPlu3buXUqVOMHz+eOnXqcObMGRYsWMC+ffs4evQoTk5OBAYGEhwc\nzLJly3jllVfMjl+2bBl6vZ6RI0cCcP36dbp06YKiKLz88sv4+PiwdetWpkyZQnJyMm+++abZ8bNn\nz0an0/HKK6+gKAouLi7lfBaskyQDQgirtTzcPBFoGwifTFVLetqCZx6DE3Hwa4Ta/uwHaOKvENTC\nNuIXQoiHlZqaSlJSErm5uezevZv33nsPZ2dnhgwZctdjXnzxRV599VWzbcOGDaNr166sXr2aZ599\nFoCwsDBefPFFYmNjadGiBQBGo5EVK1YwePBgPD09AXjrrbcoKCjgyJEjpiRk0qRJTJo0idmzZ/Py\nyy/j4eFheq/MzExiY2NxcnIq13Nh7SQZEEJYpdU7FOauLG53agGfTqNCFhGrKBqNhjfGKly4CkfP\ngcEA/1gAi99SrP7OhhDCeuw+spnIo+EV8tohrUPp2qZ/ub/ugAEDzNqtWrVi3rx51K5d+67HlLwI\nz8zMJC8vj8DAQDw9PTl48KApGRg5ciTTp09n6dKlzJ49G1CH+ly5coVx48YB6rDMVatW8eSTT6Io\nCklJSabX7tevH//3f//H3r17CQ0NNW0PCwurdokAyJwBIYQV2hCp8MHS4nbbQPh4qm0lAkX0Dhrm\nvAS11A+qSM2Av86HnDyZPyCEqLo+//xztm7dSkREBHFxcRw5csQ0b+BuUlJSmDx5Ml5eXri7u+Pt\n7Y2Pjw+pqamkpaWZ+nl6ejJs2DCWL19u2rZs2TK8vLwYPHgwAImJiaSmpvL111/j4+Nj9hg5ciQa\njYbExESz9w8ICCjHM2A75M6AEMKqbDugMGsRFM21bdEQPpsGzo62lwgU8a6h4cMpCpM/hEKDukbC\ne4tg9p8Vmy5HJ4QQdxMUFETnzp0f6JhnnnmGyMhIXn/9ddq3b4+bmxsAo0aNwmg0mvUNCwvjxx9/\nZNeuXQQFBfHTTz8xduxY7OzUS9ui/kUVje6kZcuWZu3qeFcAJBkQQliRiD8U3loIRb/zm/jD56+C\nq7PtXzC3baLhb+MU3l+strfuh+7tYFCIJaMSQtiCrm36V8hQHmuSkpLCtm3bmDVrFm+//bZpe25u\nLjdu3Lit/4ABA/Dx8eHbb7/l2rVrZGRkmIYIAXh7e+Pm5kZBQQF9+vSplO/BVskwISGEVdgfq/C3\n/6mfnAM08IP5r9nOZOHSeLyHhhG9ittzv4eUdBkuJIQQupu1mG+9AzB37tw7lmXW6XQ8++yzrFq1\nikWLFhEYGEiXLl3M9j/11FOsWbOGw4cP33b8rUOEqjO5MyCEsLhTcQp/nQ/5BWq7rjf893Xw8qg6\niUCRac9A5BF1FeXUDDUheO9Plo5KCCEsy93dnV69ejFnzhzy8/OpX78+ERER7Ny5Ey8vrzsmBGFh\nYcydO5fw8HBmzZp12/4PPviAHTt2EBwczJ/+9CdatmxJSkoKMTExrFmzpkyrIV+8eJGlS9XJbTt3\n7gTUxMXFxYWGDRsyduzYh37tyibJgBDCoq6nKLzyGWTd/J3sU0NNBHxrVr1EANS5D2+MU5j+qdre\nGAUDgxWrXUBNCCEeVGnnQt3ab/ny5UybNo2FCxdSUFBAz5492b59O3379r3ja7Zr1442bdpw9OjR\nO158e3t7s3fvXv75z3+yZs0avvjiC2rWrEnLli355JNPHirmIufPn+edd94xHavRaPjwww8B6NWr\nl00lAxqlCi6JWXLGecn6saJsDhw4AECnTp0sHEnVUp3Pa1aOwqQP4VSc2nZ1hq/egCb+Zb8wtvbz\n+uZChfC96vM6tWDlP8FJb/0JgbWfV1sl57ViWOt5zc3NxdHR0dJhVAlBQUE4Ojqya9cuS4dSKvf7\n2VviGlbmDAghLKKwUOGNL4oTATsdfDilfBIBW/DaaPBwVZ9fTYIFP1s2HiGEsDUxMTFER0fz3HPP\nWToUmybJgBCi0imKwoffwZ6jxdv+HgadW1aPRACgpruGac8Ut1duhePnq9yNWiGEKHdHjx5lyZIl\njB8/Hl9fX5sakmONJBkQQlS6JRtgze/F7ReGwrDu1ScRKDKkKwTdLHNtNMLsJeodEyGEEHf3008/\nMWHCBPLy8li5cqUMuSojSQaEEJVq816F//5U3B4UApMft1w8lqTRaPhHGDjq1fbJOFi+xbIxCSGE\ntZs5cyYGg4HY2Fh69uxp6XBsniQDQohK88dphVlfF7c7Noe3xj94FYeqxN9Hw5+GFbe/XAuXEuTu\ngBBCiMohyYAQolLEJyvM+C8UFKrtRnVgzktgb1d9E4Eiz4ZC8wbq87x8+GSlZeMRQghRfUgyIISo\ncDl5Cq99DjfS1banG8ydBu4ukggA6HQa/vEcFN0gifgDDp2SuwNCCCEqniQDQogKZTQqvPu1eQnR\nOS9BXW9JBEpq0VDDwODi9ueruOOKm0IIIUR5khWIhbBiN9Kvs+f4NuISzmCns8PRwdn0cNKrX2t7\n1adR7eZWO+7+/9bB9gPF7b+Ng/ZNrTNWS5s8HLbsU4dSHTkDvx+CXh0sHZUQQoiqTJIBIaxQUlo8\ne45t40RcDIpivG//xnVaEBr0NG7O1rXi9tb9Cl+tLW6P7AuP95BE4G7qeGt4uo/C8nC1/d+foHs7\nBZ1OzpkQQoiKIcOEhLAiianX+GX3UhZv+A+xFw+WKhEAOHc1lm82fsSx8wesZmjJyYsKsxYVtzu3\ngldGWi4eW/H8YHB1Vp9fuAa/Rlo2HiGEEFWb3BkQwgrkF+byS8S3nLz0x237GtdpQecWfXBxdCUn\nL5vc/OLH9dRrHD23D4C8/Bw27FnBqUuHCQ16Chcn98r+NkyS0xRenw+5eWq7vi/8+8/IJ9yl4Omm\nYdwAhS9Wq+0v10D/zgqOejl3Qgghyp8kA0JYWG5BNnvPbsKoyzPbHlC3FcGt+lLbq/49j2/VsCOb\n9v1AWmYyAGeuHONK0gUe6/gEzes/UulzCfILFGb8D+LVcHB1ho//IpWDHsTofvDjdkhKhesp8P02\neG6QpaMSQojys2PHDvr06cOOHTvo0aOHpcOp1mSYkBAWlJZ5g6gzv5KZm2LaFujfmrD+rzCix4T7\nJgIA9X2bMH7AazzSJMS0LScvi18jl7Hj0LoKiftuFEXho+Vw+LTa1mrh/cnQsLYkAg/CSW++ENni\nDZCWaR3Dv4QQ4l4WL16MVqs1Pezt7alXrx4TJkzg6tWrlg6vXCiKwuLFixk2bBj169fH1dWVNm3a\n8P7775OXl3f/F7AycmdACAu5kX6dH35bSHZeBgBajZZBwWNo0aD9A7+Wg72efkFP0rReGzbt/YH0\nbDW5OHDyd2q6+9CuSZdyjf1uftoBa34vbr/8FIS0kUTgYQzrBsvD4WI8ZGarCcG0ZywdlRBClM6s\nWbMICAggNzeXiIgIvv32W37//XeOHj2Kk5OTpcMrk6ysLCZMmEBwcDAvvvgiPj4+REZGMnPmTLZu\n3cpvv/1m6RAfiCQDQljA9ZQr/Pjbl2TnZQKg0WoZ3n08Teq2KtPrNvBryvhBr7MhajlnrhwDYGv0\namp5+FLXu1GZ476XgycVPl5e3B7QBcb2r9C3rNLs7DS8OELhjf+p7R+2wcjHFPy8JLkSQli//v37\n07lzZwAmTJhAzZo1+eSTT1i7di2jRo2ycHT3l5+fj06nQ6fT3bZPr9cTGRlJly7FH7S98MILNGzY\nkJkzZxIeHk5oaGhlhlsmMkxIiEp2NekCK7d/YUoEtFodnRuFljkRKKK3d2RwyLP4eNYBwGg0sHb3\nt2Rkp5bL699JfLJ60VpoUNstGsJb47HatQ9sRZ+O0Lqx+jy/AL5ce+/+QghhrXr37g3AhQsX7trn\n8OHDPP/88wQEBODk5IS3tzejR4/m0qVLpj6nT59Gq9Uyd+7cOx6v1WpZuHChadu1a9eYOHEifn5+\nODo60rJlSxYsWGB23I4dO9BqtSxfvpx3332X+vXr4+zszJUrV+4Yp729vVkiUOTxxx8HIDY29u4n\nwgpJMiBEJYpLOMMPvy0kLz8HUC/cHw0YSC23uuX6Pg52eh7v/jxOehcAsnLSWRuxhILCgnJ9H4Dc\nPIW/zocUdbQTNd3ho5dB7yCJQFlpNBpefqq4/etuuBgvcweEELbn7NmzAHh5ed21z9atWzl16hTj\nx49n/vz5TJw4kU2bNtGrVy9yctS/m4GBgQQHB7Ns2bLbjl+2bBl6vZ6RI9U61tevX6dLly6Eh4fz\n8ssvM2/ePFq3bs2UKVN4//33bzt+9uzZ/Pzzz7zyyivMmTMHFxeXB/oe4+PjAahVq9YDHWdpMkxI\niEqSnJ7Az7u+oaAwHwAnvQtP957MpbPXKuT9PFxrMqxrGD/+thCjYuRachxbDqxi4KOjyu0Te0VR\n+OdiOHFRbdvp4IMp4FtTEoHy0rG5hi6tFfYcBUWBJRvgnQmWjkoIUZm+XGu+gGN5+tNwmDS8/H9n\np6amkpSURG5uLrt37+a9997D2dmZIUOG3PWYF198kVdffdVs27Bhw+jatSurV6/m2WefBSAsLIwX\nX3yR2NhYWrRoAYDRaGTFihUMHjwYT09PAN566y0KCgo4cuSIKQmZNGkSkyZNYvbs2bz88st4eBQv\n1pmZmUlsbOxDz2mYM2cO7u7uDBpkW+Xf5M6AEJUgLz+HNbsWk1+QC4Crkwej+76Eb43yvSNwq/q+\nTejVvrgszbHzBzh4KqLcXn/pJgjfW9z+67PQvqkkAuXthRJ/OzdGwbUkuTsghLBuAwYMwMfHh/r1\n6zN69Ghq167NunXrqF279l2PKXkRnpmZSXJyMoGBgXh6enLw4EHTvpEjR6LX61m6dKlp244dO7hy\n5Qrjxo0D1A+rVq1axeDBg1EUhaSkJNOjX79+5OTksHdviT9gqEnGwyYCs2fPZtu2bXzwwQfUqFHj\noV7DUiQZEKKCGRUj6/es4Eb6dQDsdQ6M6PkCXu6+lfL+HZp2o3WjIFN7x6FfuBh/usyvG3lE4b8/\nFbdH9IIRvSQRqAiPNNXQvpn6vNAAyzZbNh4hhLifzz//nK1btxIREUFcXBxHjhwxzRu4m5SUFCZP\nnoyXlxfu7u54e3vj4+NDamoqaWlppn6enp4MGzaM5cuLq1YsW7YMLy8vBg8eDEBiYiKpqal8/fXX\n+Pj4mD1GjhyJRqMhMTHR7P0DAgIe6nv9/vvvefvtt5k4cSJ//vOfH+o1LEmGCQlRwaKObuHszco+\nAKGdn67wOwIlaTQa+nZ6kuT0BK4lx2FUjKyLXMpzA17Dzdnj/i9wBxfjFd5cCEaj2m4XCK+PKceg\nxW0mDIapJ9Xna3fBhCEKXh6SfAlRHUwarmHScEtH8WCCgoJM1YRK65lnniEyMpLXX3+d9u3b4+bm\nBsCoUaMwFv3BuSksLIwff/yRXbt2ERQUxE8//cTYsWOxs1MvbYv6jxkzhgkT7jy2smXLlmbth7kr\nsGXLFsLCwhgyZMhtE5NthSQDQlSgM1eOEXk03NTu1KwnLRt2qPQ47O3sGd5tPEvDPyUrJ52cvCx2\n/rGewcEPfgWfma3w+udq7XsA35rw4RSwt5ML04r0aCu1SlPsBcjLV9cgmPq0paMSQojykZKSwrZt\n25g1axZvv/22aXtubi43bty4rX/RMKRvv/2Wa9eukZGRYRoiBODt7Y2bmxsFBQX06dOnQmLeu3cv\nTzzxBJ07d+aHH35Aq7XNATe2GbUQNiA5PYH1UcW3MBv4BtLzkcEWi8fN2YPBXUab2scvRHMl8fwD\nvYbBoPDml3Dh5pxnRz38ZyryCXUl0Gg0PF/in8+q3yA9S+YOCCGqhqJ6/rfeAZg7dy6KcvvvOp1O\nx7PPPsuqVatYtGgRgYGBZuU+dTodTz31FGvWrOHw4cO3HX/rEKEHFRsby+DBg2ncuDG//vorer2+\nTK9nSfdNBnbu3MmwYcPw9/dHq9WyZMkSs/3jx483W3Zaq9USEhJi1icvL4+pU6fi7e2Nq6srw4cP\nv612a0pKCuPGjcPT0xNPT0/CwsLMxocBxMXFMXToUFxdXfH29mbatGkUFJR/qUQhyurWCcPuLjUY\n0nUsWu3ti5dUpgZ+TWlar62pvS16DUajodTH/281RJb4nfr2eGjeQBKBytKzPTS+OcIsOxe+32bZ\neIQQory4u7vTq1cv5syZw5tvvsnChQsZN24cCxYswMvL644JQdG1Ynh4OGPHjr1t/wcffIC/vz/B\nwcFMnz6dL7/8kg8//JDRo0dTv379h441IyOD/v37k5qaytixY1m3bh3Lli0zPfbs2fPQr20J900G\nsrKyaNu2LZ999hlOTk63lSTUaDT069eP+Ph402PDhg1mfaZPn87q1atZuXIlu3btIj09nSFDhphl\nf2PGjCEmJobNmzezadMmDh48aHa7x2AwMHjwYLKysoiIiGDFihWsWrWK1157raznQIhydacJw493\nfx5nvauFI1P1aj8UO509AAkplzlybn+pjtu0R+HbjcXt8YMh9FFJBCqTVqvhuYHF7e+3Qnau3B0Q\nQliX0pavvrXf8uXLGTJkCAsXLmTGjBmkpaWxfft2XF1d7/ia7dq1o02bNmg0mjsmA97e3uzdu5eJ\nEyeyZs0apk6dyqeffsr169f55JNPHipmgOTkZC5fvoyiKLzxxhuEhYWZPb788stSv5Y10Ch3SrXu\nws3Njf/+97+EhYWZto0fP57k5GTWrVt3x2PS0tLw8fFh8eLFjB6tDlG4fPkyDRo0YOPGjYSGhhIb\nG0urVq3YvXs3wcHBAOzevZvu3btz8uRJAgMD2bhxI0OGDCEuLo66ddWPxr777jsmTpxIYmIirq6u\nZu9ZpGT9WFE2Bw4cAKBTp04WjsS67Yv9jd9jfjW1Bwc/e895ApY4r7uPbDbNZXDSu/DC4Ddw0jvf\ntf/x8wqTPlTHqgN0bQsfTwWdznqTgar679VgUHjqTbis5ppMGwlj+1fez6GqnldLk/NaMaz1vObm\n5uLo6GjpMKqEoKAgHB0d2bVrl6VDKZX7/ewtcQ1b5jkDGo2GiIgIfH19adasGZMmTTIbhxUdHU1B\nQQGhoaGmbf7+/rRo0YKoqCgAoqKicHV1NSUCACEhIbi4uBAZGWnq07JlS1MiABAaGkpeXh7R0dFl\n/TaEKBcJNy6z63Dxx+eWmjB8P51b9MHdRa2DnJOXReTRu9eqTE5TVxguSgQa1oZ/TbLuRKAq0+k0\njBtQ3P5uM+Tly90BIUT1ExMTQ3R0NM8995ylQ7FpZU4GBgwYwNKlS9m+fTsff/wx+/bto0+fPuTn\nq1cO8fHx6HS625af9vX1NS3bHB8fj7e3t9l+jUaDj4+PWR9fX/O67LVq1UKn05n6CGFJ+YV5/Bq5\nzDQGv7Y0qp5fAAAgAElEQVRXfXpYcMLwvdjb2dO7xGJkMacjSUy9fSXkvHw1EbieorbdnNU7Aq7O\nkghY0pCu4H1zTZukVPh1t2XjEUKIynT06FGWLFnC+PHj8fX1veMQIVF6ZS4tOnLkSNPzVq1a0bFj\nRxo0aMD69et54okn7nrcA4xOKtMxRbcIRfmRc3pnRy5FcDn5IgA6nR31XFpz6OChUh9f2edVURSc\ntJ4kZ1wF4PtNX/FowEDTuElFgS/W12HPCXcAtBp4ceAlrl/O4vrlSg21TKrqv9eQwBos36F+QDJv\nZQF1nc9iV4nz06vqebU0Oa8Vw9rOa4MGDWSYUBn89NNPvPfeezRt2pSVK1fa1LnMyMjg6NGjd90f\nGBhYidGoyr20aO3atfH39+fMmTMA+Pn5YTAYSE5ONuuXkJCAn5+fqc+tJZ4UReH69etmfRISEsz6\nJCUlYTAYTH2EsJT41AvEJZ80tVvVCcZFb93zVTQaDa3qPGq6+E/OvEZ82gXT/rV7vEyJAMDoXgm0\nbZRV2WGKu+jVNhU3J/UuVGKavdnPSgghqrKZM2diMBiIjY2lZ8+elg7H5pX7omOJiYlcuXKF2rVr\nA9CxY0fs7e0JDw83m0B84sQJUwnS4OBgMjMziYqKMs0biIqKIisry9QnJCSE999/nytXrpjmDWzZ\nsgW9Xk/Hjh3vGo+1TRqyZdY6EcvSMrJT2bNxLc4u6gTcZvXaMbTr6FJXJrD0eVVcsok+qU68upp1\nkgG9hrPjoD2bD8HNb4kRveCNcY0fqNqCpVn6vFaGSckKC35Wn+87F8i0sAeriPEwqsN5tQQ5rxXD\nWs9rbm6upUMQFuLm5nbPf4+3ltWvDKUqLRoTE0NMTAxGo5GLFy8SExPDpUuXyMrK4vXXX2fPnj1c\nuHCBHTt2MGzYMHx9fU1DhDw8PHjhhReYMWMG27Zt49ChQ4wbN4527drRt29fAFq0aMGAAQOYPHky\ne/bsISoqismTJzN06FDT7ZLQ0FBatWpFWFgYMTExbN26lRkzZjBp0iSzSkJCVCajYmTjnpXk5KvL\n8bo5e9Iv6CmbumgOad0fJ70LAOnZKXy/bR+zFhXvD2oJfx1T8ReZ4sE93Udd+A3g9CXYH2vZeIQQ\nQtie+yYD+/fvp0OHDnTo0IHc3FxmzpxJhw4dmDlzJjqdjqNHjzJ8+HCaNWvG+PHjTVWCXFxcTK/x\n6aef8sQTTzBy5Ei6deuGu7s769atM7u4WL58Oe3ataN///4MGDCA9u3bs3Tp0uJAtVrWr1+Ps7Mz\nXbt2ZdSoUTz11FP85z//KedTIkTpHTjxOxcTTgOgQcOgLqPvWaLTGjk6ONGj3SAAMrJdmb3Em5w8\ndQ2QBn7wwYtgZyeJgDVyd9EwtFtx+7twy8UihBDCNt13mFCvXr1uWxq6pE2bNt33TRwcHJg3bx7z\n5s27ax9PT0+zi/87qVev3l3XMxCist1aRrRzyz7U921iwYgeXutGQew+EsmK8J5kZDuhKBnUqeXB\nJ39RLziF9Rr1GKzark74jjwMF64pNKwtPzMhrJ2iKHLHtZp5mEI4laHcJxALUR0UFObza9R3pjKi\nfjXr0bV16H2Osl6KomXfsTEkpqglfnPz0/nX5Hzq+8kfKmtX309D90eK2yu2WC4WIUTpODg4kJub\na7UXh6L8KYpCbm4uDg4Olg7lNuU+gViI6mDnH+u5ka4uAWtv58Dg4GfR6Wzzv5OiKMz9Ho6e9cFO\nF0+hoZCeHbai0TQAHrN0eKIUng2FnTer2K6PhBefUPB0k0ROCGul1WrR6/Xk5eVZOpSHkpGRAaiT\nYUXp6fV6tFrr+xzeNq9ehLCg89dOcPBUhKndu/1warp73+MI6/ZdOHy/FUCDi5M7jetupVXjYxw4\neYEOzbrhYKe3dIjiPto3hRYNIfaCulL0TzvghaEWDkoIcU9ardam6uOXVFQn39qqNImHY33piRBW\nLCcvi417vze1m9RtRduARy0YUdmE71X4rPjbYUiIMwND1F/yOXlZ/HEmykKRiQeh0WgY3a+4/eN2\nyC+Q4QdCCCHuT5IBIUpJURTC968iKycdAGdHV0I7P22zE8CiTyi8+3Vx+5GmMGuihuCWfUzb9sfu\noKCwwALRiQfVNwh8aqjPk9Ng817LxiOEEMI2SDIgRCkdO3+AU5cOm9oDOj+Di6Ntjpc8e1nhr/Oh\noFBtN6oD/3kZ9A4aWjcOws1JXT05KzeDI+fkqtIW2NtpeKbEFI/l4dZbuUIIIYT1kGRAiFJIzUxm\n28E1pna7gC4E1G1lwYgeXmKKwvTPIENdJ41anvDZdPBwVe9w2OnsCWrR29R/3/HtFBrk7oAteKJn\n8SJkZy7LImRCCCHuT5IBIe7DaDSwYc8K8gvU5eNruNWiV/thFo7q4WTlqIlAfLLadnaET6dB7Vrm\nQ53aBnQx3fXIyEnj2PkDlR2qeAjuLhqGySJkQgghHoAkA0Lcx/4TO7iSeB4ArUbLoC5jcLC3vQo7\nefkKr34Op+LUtp0OPpwCzRrcPufB3s6eTs17mtp7j2/HcHNNBWHdRvWFomkskYfh/FUZKiSEEOLu\nJBkQ4h4Sblwm4shmU7tLq77UqdXAghE9nIJChb/9Dw6eKN72j+egS+u7T35+pEkITnoXANKybnD8\nQnRFhynKQT1fDT3aF7dlETIhhBD3IsmAEHeRX5DHushlplWGa3vVp0urvhaO6sEZDArvfAW7i+c+\nM/VpGNrt3lWQHOz1dGzWw9Tee3yb6VwI6zamRJnRDVGQmiF3B4QQQtyZJANC3MW26J9JyUgEwMFO\nz6AuY9BpdRaO6sEoisK/l8LW/cXbJgyFsIGlK4faoWk39A5OAKRkJHH68tGKCFOUs6JFyEBdhGzN\nTouGI4QQwopJMiDEHcRePMTR88VX0H07jbC5VYYVReHT72FtiQvBZx6DPz9e+tfQ2zvSPrCrqR19\nalc5RigqikajYWSJm1irflPvEAkhhBC3kmRAiFukZiazZf8qU7tlww60amR7S65/9Ytaa77I4K7w\n2mgeeJG09oFd0d68I3Il8TzXkuPKM0xRQfoFQU139XnCDdhxyLLxCCGEsE6SDAhRgsFo4NfIZeTd\nLCPq6epF305PWjiqB7c8XOGrtcXtPp3g7fGg1T74asmuTu60qF88IzX6pNwdsAUO9hqeKC4Ixcqt\nlotFCCGE9ZJkQIgSIo9sNn3yrdXqGBIyFr29o4WjejDLwxXmrixuP9oK/vkn0OkePBEo0rFZd9Pz\nk5f+ICM7tSwhikryZC+1hCxAzCk4eVGGCgkhhDAnyYAQN12MP83e49tN7W5tBlDbq74FI3pw3240\nTwTaBcJHL6ufEpeFb01/6vk0BtRF2A6djizT64nK4V1DQ58SI9x+2H73vkIIIaonSQaEALLzMtmw\nZwUK6ienDfyaEtSil2WDekCLflX4/MfidrtA+Gw6OOnLlggU6dC0uMzo4TNRFBTml8vrioo18rHi\n55v2SJlRIYQQ5iQZENWeUTGycc9KMnPSAHDSuzCoy2i0Gtv476EoCl+uVfhidfG2Ds1h3ivg4lQ+\niQBAk7ot8XD1AiAnP1sWIbMRbQKgZSP1eX4B/CxlRoUQQpRgG1c7QlSgPce2ce5qrKk98NFRuDq5\nWzCi0lMUhS9+xmyycOdW8Nk0cHYsv0QA1DkUHUqUGT1wcidGxViu7yHKn0ajMbs78NNvUFgodweE\nEEKoJBkQ1dr5ayeJPLLZ1A5q3ouAui0tGFHpKYrCvB/hm1+LtwW3gU+mgmM5DQ26VZuAR3G4OaH6\nRvp1Llw7VSHvI8pXXykzKoQQ4i4kGRDVVlrWDdZHfWeaJ1DPJ4Ae7QZZOKrSMRgUPlgKyzYVb+vW\nDj56CfQOFZMIgLoIWZvGnU3tg6dkzIktuLXM6PfbLBeLEEII6yLJgKiWCg0FrNu9lJy8LABcnNwZ\nGjLWtLiWNcvNU5jxP1i9o3hbzw4wp4ITgSIdmnZDc3M+xflrJ0lKi6/w9xRlJ2VGhRBC3IkkA6Ja\n2n5wrdl6AsO7huFiA/ME0jIVpnwMO0sM8xjQBT74M9jbVXwiAOpCbE3qtjK1D8oiZDbh1jKjcndA\nCCEESDIgqqGj5/bzx5koU7vXI0Op693IghGVztVEhYn/hiNnireNGwizJoJdJSUCRUouQnbsQjTZ\neZmV+v7i4YzqW/x8815ISZe7A0IIUd1JMiCqlYSUK2w58JOp3aJBezo07WbBiErnVJzCC/+GC9fU\ntkYDr46GvzytQaut3EQAwN+7Mb41/AF1yNUfZ/ZUegziwbVuDK2kzKgQQogSJBkQ1UZOXha/RCyh\n0FAAgJeHL6FBT6PRVP7F9IPYH6sweQ4kpaptezt4fzKM7me5uDUajdndgT/ORGE0GiwWjygdjUbD\nMyXKjK7eIWVGhRCiupNkQFQLBkMhayOWkJqZDICDvSOPdxuPg73ewpHdnaIo/LBNYeonkJmtbnN1\nhs9fhX6dLZ/ANKvfDme9KwAZ2amcvXrcwhGJ0pAyo0IIIUqSZEBUeYqisHn/j1y6fta0beCjo6jp\n7mPBqO4tv0DhX4vho+/AcPMDd+8a8NUb0LG55RMBADudPW0DHjW1D52OtGA0orSkzKgQQoiSJBkQ\nVd7e49s5dv6Aqd2j3WCa1mtjwYjuLTFFYfKH8EuJIj0tG8E3b0ITf+tIBIq0bdLFVGb0YvwpbqRf\nt3BEojSkzKgQQogikgyIKu1k3B/sOrzB1G7TuDOdW/S2YET3duSsQtg/4ei54m2DQuDLv4FvTetK\nBAA8XGoSUKeFqS0TiW3DrWVGf9huuViEEEJYliQDosq6lhzHhj0rTO16PgH06/Sk1U4Y/mWXekeg\naKKwTqdWDHr3hcpZTOxhPRIYYnp+5Pw+CgrzLRiNKK1n+hQ/37xXXcNCCCFE9SPJgKiS0rNS+Hnn\nIlPloBpu3gzv9hw6nZ2FI7tdTp6WLzfW5p/fQEGhus3DFea9olYMstbkpUgDv6Z4unoBkJefQ+xF\nmZFqC9o2geYN1Od5+bBGyowKIUS1JMmAqHLyCnJZvfNrsnIzAHBycGZEjxdw0rtYOLLbHT+v8M7S\nhkQc8zBtC6wHS96Gzi2tOwkootVoze4OxJyORFHkU2Zrp9FoGFliEbJVv4HBID83IYSobiQZEFVK\noaGAtRFLSExVV+fSanUM6/YcNd29LRyZOaNR4duNCi/MhoRUB9P2QSHw9T+grrdtJAJFWjcKwk5n\nD0BCymWuJcdZOCJRGv2CwNNNfR6fDL/HWDYeIYQQlU+SAVFlGI0Gfo38jovxp0zbQoOeor5vEwtG\ndbukVIWpc+HzH6HwZtlQJwcj7/0JZk3U4KS3rUQAwEnvQvMG7U3tGCkzahP0DhpG9CpuS5lRIYSo\nfiQZEFWCUTGyad8PnL58xLSta5sBtGnc2YJR3W5njMKYmbDvWPG2gNo5vBd2noHBtpcElPRIk+Kh\nQicuxZCdl2nBaERpjeipTlYHOHgCTl+SoUJCCFGdSDIgbJ6iKPx2cK3ZWgKdmvUkuFXfexxVuW6k\nK/xjgcJr8yBFncqARgPPDYI3R13E17PAsgGWg9pe9ajtVR9QV3w+em6/hSMSpeFbU0OvDsVtKTMq\nhBDViyQDwuZFHg3n4KkIU7tN4870aj/UKqrwKIrC+kiFZ96CLfuKt9fyhPmvwctPaUyLP1UF7ZoE\nm57HnInCaDRYMBpRWqMeK36+aY+UGRVCiOpEkgFh0w6c+J3Io+GmdrN67QgNesoqEoGriQpTP4F3\n/w/SSoyYGRQCK9+znWpBD6J5/fY4OjgDkJaZzIUS8zeE9WoXCM3Umzrk5sHaXffuL4QQouqQZEDY\nrCPn9vHboV9M7Ua1mzM4eAxarWU/ai8sVFgerjDyHdhbYm5A7Vow71V1krCHa9VLBADs7exp3TjI\n1D50ercFoxGlpdFoeKbE3QEpMyqEENWHJAPCJh0+u5fNe38wtet6N7KKRcUijyiMeRfmrlQ/YQV1\nbsDoUPVuQHDrqpkElPRIiaFC56+eIDUz2YLRiNLq/2hxmdFrSVJmVAghqgtJBoTNOXgqgs37fkBB\n/eTSt4Y/T/Z4AXs7h/scWXHOX1WY9qnCtLlw/mrx9oC6sOhNeHWUBmfHqp8IgLrac6PazQBQUDhy\ndq+FIxKlcWuZ0ZVbLRaKEEKISiTJgLAp+2J/Y1v0z6a2bw1/nur9J/QOThaJJy1T4aPvFEbPhMjD\nxdtdnOClJ2HpTGjduHokASWVnEh85Nw+DDKR2CY81QvThPZDJ+HERRkqJIQQVZ0kA8ImKIrC7iOb\n+T3mV9O2OrUa8kyfP+Osd630eHLy1BWER/wdftgGhpvXuhoNPN4TfpoN4wdrsLerfokAQECdlrg6\neQCQlZvBmctHLRyRKA3vGhoeK57ywYotlotFCCFE5ZBkQFg9RVHY+cd6s6pB9XwCeLrXJBwr+Y5A\nURIwfIa6gnB6VvG+Ds1h6Tvw5nMavDyqZxJQRKvVmS349sfZPRaMRjyI0SWW5wjfC8lpcndACCGq\nMkkGhFUzKka2Rf/MvtjfTNsa1W7Gkz0n4mCvr7Q4cvIUlm4qTgKKFg4D8PeBj16GBX+FZg2qdxJQ\nUtuAR9Fo1F8xF+NPkZKRaOGIRGm0aqyhbaD6vNCgVhYSQghRdUkyIKxWQWEBG6KWm5WnDPRvzePd\nn6+0ycIZWeqdgMf/BvN+ME8C/Lzg72Hww7+gVweNVaxtYE3cXWrQuHZzU/uwTCS2GSXvDvz0G+Tl\ny90BIYSoqixbh1GIu8jOzWTNrm+4knTBtK1Fg/YM7DIaXSWsI3ApQWHlVlgXATl55vv8vOD5wTC0\nG9V2TkBptWvShbNXjwPqROKubfpjp7O3cFTifnp3UP+dxyerCfDmvTCsu6WjEkIIURHue2dg586d\nDBs2DH9/f7RaLUuWLLmtz7vvvkvdunVxdnamd+/eHD9+3Gx/Xl4eU6dOxdvbG1dXV4YPH86VK1fM\n+qSkpDBu3Dg8PT3x9PQkLCyMtLQ0sz5xcXEMHToUV1dXvL29mTZtGgUFBQ/zfQsrlpyewHdb5pkl\nAo80CWFQBScCiqJwIFbhtc8VnvyHOjG4ZCLg5wVvjFMnB4/oVX0nBz+IRrWb4+5cA4CcvCxOy0Ri\nm6DTaXi6T3F75Vb1/4cQQoiq577JQFZWFm3btuWzzz7DycnptqEQH374IZ988gnz589n//79+Pj4\n0K9fPzIzM019pk+fzurVq1m5ciW7du0iPT2dIUOGYDQaTX3GjBlDTEwMmzdvZtOmTRw8eJBx48aZ\n9hsMBgYPHkxWVhYRERGsWLGCVatW8dprr5XHeRBW4mL8ab7b8rlpoSoNGnq3H0bfTiMqbGXhjCyF\nn35TGDsLXvwIdh6Cktc9AXXhzfFqEvBkbw0O9pIElJZWq6NNwKOm9h9noiwYjXgQj/cAx5vTck5f\ngugTlo1HCCFExbjvMKGBAwcycOBAAMaPH2+2T1EUPv30U/7+97/zxBNPALBkyRJ8fHxYvnw5kyZN\nIi0tjUWLFrF48WIee0xd737p0qU0aNCArVu3EhoaSmxsLJs3b2b37t08+qh64bBw4UK6d+/O6dOn\nCQwMJDw8nOPHjxMXF0fdunUBmDNnDhMnTmT27Nm4ulZ+eUlRvo6c20f4/lUYb9akt9c5MKTrWJrU\nbVXu72U0Khw8Cb9EwPZoyMu/vU9IWxjTDzq3ROYDlEGbxp2JOhqOUTFy6fpZktMT8HL3tXRY4j7c\nXTQM6aqwarvaXrEVOrWwbExCCCHKX5kmEJ8/f56EhARCQ0NN2xwdHenRoweRkZEAREdHU1BQYNbH\n39+fFi1aEBWlfkoYFRWFq6srwcHFCxWFhITg4uJiep2oqChatmxpSgQAQkNDycvLIzo6uizfhrAw\no9HA7zG/smnv96ZEwMXJnVF9p5R7IhCfrPD1OnUY0IsfwcYo80TAUQ8jeqmTgj+bruHRVjIxuKzc\nnD0IKPFzPHxGJhLbilElJhLvilHn0gghhKhayjSBOD4+HgBfX/NP+Xx8fLh69aqpj06nw8vLy6yP\nr6+v6fj4+Hi8vb3N9ms0Gnx8fMz63Po+tWrVQqfTmfrcyYEDBx7iOxP3Up7nNLcgm0MXf+NGZvHP\n0M2pBm18enL5XAKXzyWU+T2S0u04cMqdfSfdOHPtzusS1PfOo2ebVIJbpOHqZCT5KiRfLfNbP5Cq\n/G9VX+BJdlY2AFGHt+Fc6INOWzn1C6ryea0MTXz9+eO8euf14yUpjL05l0DOa8WQ81ox5LxWDDmv\n5S8wMLDS37PC/hrf79PUh5mMJhPYqpakjKscittBfkGOaZu3uz/tG/TGXle20qHJ6XbsP+XGvpPu\nd00AXPRGglum0aN1Kg198+7YR5SPWm51cdK7kZOXQUFhPtdSL+Bfs4mlwxKlENoxxZQM7DrqwZNd\nE3HSG+9zlBBCCFtRpmTAz88PgISEBPz9/U3bExISTPv8/PwwGAwkJyeb3R1ISEigZ8+epj6JieYL\nEimKwvXr181ep2jIUJGkpCQMBoOpz5106tSpDN+hKKnoE4CynlOj0cCe49s5emYXdg4a7Byc0aAh\npE1/urTs81AThQ0GhSNnYfcRiDwCp+KK97k4Fz+306njnod2hZ7tQe/gDtQr0/dTVuV1Xq2d0SWd\nnX9sACBbm0SnTqMq9P2qy3mtaB07KmyMgfM375T9fsSTAZ1uyHktZ/LvtWLIea0Ycl4rzq2VNCtD\nmeYMNGrUCD8/P8LDw03bcnNziYiIICQkBICOHTtib29v1ufy5cucOHHC1Cc4OJjMzEzTHAJQ5whk\nZWWZ+oSEhBAbG2tWknTLli3o9Xo6duxYlm9DVKKs3AxW/f5/7D6yCUVRP110dnTl6d6TCGnd74ES\ngaRUhfWRCv9YoNBvGvzpA1i83jwRADUB6NIa3noeNs2Fz1/VEPqoBr2DzAWoTK0bBZl+vlcSz5OY\nes3CEYnS0Gg0jO5X3N4cXYNCg+XiEUIIUb7ue2cgKyuL06dPA2A0Grl48SIxMTF4eXlRr149pk+f\nzuzZs2nevDmBgYH861//ws3NjTFjxgDg4eHBCy+8wIwZM/Dx8aFmzZq8+uqrtGvXjr591dlpLVq0\nYMCAAUyePJkvv/wSRVGYPHkyQ4cONY2dCg0NpVWrVoSFhfHxxx+TlJTEjBkzmDRpklQSshHnr51k\n097vycwpznrr+QQwJGQsrk7u9z3+eopC9Ak4eAoOnoC4e0wnKLoD0DcIerUHD1e58Lc0Fyd3Auu2\n5uSlPwA4fHYPj3V8wsJRidIYFAwLfoYb6ZCcYc/eE+50efT+xwkhhLB+900G9u/fT58+6owxjUbD\nzJkzmTlzJuPHj2fRokXMmDGDnJwcXnrpJVJSUujSpQvh4eG4uLiYXuPTTz/Fzs6OkSNHkpOTQ9++\nfVm2bJnZvILly5czdepU+vfvD8Dw4cOZP3++ab9Wq2X9+vVMmTKFrl274uTkxNixY/noo4/K7WSI\nipGXn8Nvh37hyLl9Ztu7tOpL19ahd7wbYDAonL0CR8/BsfNw6BRcus9cYt+aENIGuraFTs3BxUkS\nAGvTrkkXUzJw7PwBerQbjL1d2eaHiIqnd9DwzGMKC35W2xsPePGXMEUqbQkhRBVw32SgV69eZouD\n3UlRgnA3Dg4OzJs3j3nz5t21j6enJ0uXLr3n+9SrV49169bdO2BhVc5djSV8349klLgb4KR3YVCX\nMTSu0xxQa/5fvg4n44ov/k/GQe595vTqHaB1YzUBCGmjLg4mFyfWrZ5vE2q41SIlI4m8glxOxMXQ\npnFnS4clSuGp3rB4A2RlQ1yinr3H1OF3QgghbFvl1PYT1U5OXja/HVrLsfPmZcfqeXeivu8wDsQ6\ns2KLwplLcObK/S/8Qb34bxMAHZtBx+bQqhGyGrCN0Wq0tA3owu8xvwLqisSSDNgGD1cNw7opfHPz\n85ilmyUZEEKIqkCSAVGuFEXh+IUj/LxrG1cT7UlJ70BKRg0ysn1AaUp+gVupX8vPC1o2Ui/62wao\nz+Xi3/a1bhRExOFNGIyFXEuO43rKFXxq1L3/gcLixoTCkl/BqMC+Y3DyokKzBvJ/UgghbJkkA+KB\n5eUrJKVBwg24lgxXEuFqEpy7ks2x84kkpnqgKE+a+jvpnXFz9kSruXulIC8PCPBXL/yLHrU85SKj\nKnJ2dKVpvTbEXjwEwB9n9tAv6Mn7HCWsQV1vDZ2bprPnpDrhf+lm+NckCwclhBCiTCQZECY5eQrJ\naZCcdrNqSPrNr2lwPQVOnG3IjQx7jLdcoxsVA5nZaWTnZ4NSXK1Wq9Xh7lIDR/viRb/0DtDAD5r4\nQ2A9CPRXn3t5yIV/ddI2oIspGTh+8SA9HxmCg73ewlGJ0hgYdMOUDGzdDy+NUKhdS/7/CiGErZJk\noIpRFIWcPMjIhsxsSM9Wn2dkQ3qW+jUtE1JvPtJLPL/fuP2sbEegeBEvBYXs3EyyctIxKsWTzDUa\nqFPLnrYBXjSua08DPzUBaOinDv3RauXCobqr5xNATXcfbqRfJ78glxNxh2gb0MXSYYlSaOSXS4t6\n2cQlu2AwwIqt8GrFrh8nhBCiAkkyYCGKolBogLx8yCuA3Hz1Yjyn6OvNR26++jX7Zjsr5+bX3OKv\n2TmQefORnQv3Kf5UZlqtgpM+jUJDHO4uSbi7pOHhmoabSzqtG9ViWLfHqFPr7qtCC6HRaGgb0IUd\nh34B1KFCkgzYjsGdk/liozcAa36HiUMV3F0kyRdCCFtU5ZOBwkL1ottghELDHR6Ft28rKLz5uLm/\n4Oa2/AJ1f36B+baCQvWCvqidX/LrzYv9/AL1a9HFf14BGKxsFU8He6jpDl7uUNPj5nMPqOkGPjUg\n4co5cgtPk6c7RXp2ktmxNd286dV+GI3rtJDynqJUWjfqxK7DGzAYCom/cYmEG5fxrelv6bBEKbRp\nmPbr1wwAACAASURBVEUTfzhzWf1Q4qcd8PxgS0clhBDiYVT5ZCC4Gk5uc9SDqxO4OYOrs/rV3Rnc\nXdS2hwt4uoKnm/rVw1X96ux45zr9BkMhxy4cYP/x1eTkZeBcNE4I0Ns7EtI6lPaBXdHpqvw/J1GO\nnPQuNKvXluMXDgJqmdHQzk9bOCpRGhoNPNsfZn2ttn/YBs+GKlLtSwghbJBcvVmQnU6dUOvoAHp7\n9SLeyQH0enWbU4mvznr1Yr3o4eSgfnVxUi/8XW4+d3EEO7vy+YOcm5/D0XP7iD65i/TsFHLysk37\nHB2c6disBx2adsPRwekeryLE3bUNCDYlA7EXD9Gz/VD09o4WjkqURv9H4YvVanGBpFTYuAeGd7d0\nVEIIIR5UlU8G7HSg0938qlUfdnZq+7aHHdiX2Ff03OHm9qKHg93Nvjp1aI2Dvbqt6Ku9nXqRry+x\nz1Ff3Kfo4r+8LtrL2/WUq8Sc3s3xCwcpMOSb7bO309Ot7UA6NO0mF22izPy9G+Hl7ktyegL5hXmc\nuHiIdk2CLR2WKAV7Ow0j+yp8/qPaXrYJhoQo6HTW+XtNCCHEnVX5ZCDqK/nDVBoGo4FTlw4TczqS\ny4nnbtvvrHelvkcr6ns1J7hViAUiFFWRRqOhXZMubD+4FiieSCzzTmzDiJ7wzXq1ctmFa7A9GvrJ\ngtJCCGFTqnwyIO5OURSuJccRe/EgJ+JiyM7NvK2Pt2dt2gd2pUXDDhyOOWKBKEVV17JhJ3b+sYFC\nQwEJKZeJv3GJ2l71LR2WKAVXZw3PPKawaJ3aXvQrPNZJkfLBQghhQyQZqIZupF8n9uKh/2/vzsOb\nqvL/gb9v0iRNuqT7jrSlhZYWCxQKrcgiyiKIw+iwOICgP3cZFB0ddOYRx5Vn/DqODrjghjoMoOIG\njICyVsralq0FWtpSWtqUlu57k/P7I3BLZJG2Kbdp36/nuU/uufck+eTTEPLJufdcZJ5KRXl16SX7\nVSo1+oYMwMDImxDiG8ZfaalT6XUG9OsVh6N5+wFYTyRmMeA4Zt4K/HeTdVah7AJg50Fg1CCloyIi\nomvFYqAHEELgbEURTp7JQHbBERSfO33Zfi56d8T1GY4b+wyHm8F4naOkniwuYrhcDBw7lY7Rg6bw\nxHQH4eEm4a4xAl/8aG1/vA4YOVDwRwQiIgfBYqCbam5pxumSkzh5JgM5hRmoqiu/bD+txhl9Qwag\nf+hg9PLrA5VKfZ0jJQKCfELh6xGIsxVFaDY3ISPvAAb3HaF0WHSNZo23Ti/a1Axk5AK7jwKJsUpH\nRURE14LFQDdhsZhhKi/E6ZKT1sV08pKZgC5QqdToExSN6N6DER7UHxonzXWOlsiW9UTiRPy0fy0A\nID1rFwZF3sRflx2Et1HC70YKrPnZ2v7oB2B4DEcHiIgcAYsBB9Xc0oyS8kIUnD2J0yU5KDybi6aW\nxiv212mcERYYhfCgaIQHRUOvc7mO0RL9tv6h8diRvh5NLY0oqzLhdMlJ3OAfoXRYdI3mTATWbrNe\npf1gFpB6HIiPUjoqIiL6LSwGHIDZ3ILSymIUnzuN4nMFMJ0rwNnKIlgs5qvez8vdD+FB0egT1B/B\nvmFQ8xAg6sJ0GmdEhw7GwewUAEB6dgqLAQfi7yVh8giBb7db2x/+wGKAiMgRsBjoQiwWMyprz6Gs\nqgSlFcUoqypGWWUJSiuLYba0/Ob93Q2eCPELRy+/Pujl1weebj7XIWoi+xkYkSQXA1kFh1FbXwUX\nvbvCUdG1mns78EMyYDYD+zOBg1kCcZE8VIiIqCtjMXCdWSxmVNdVoqKmFBU1ZTbLuaoStJibr/mx\nvNx8EejTW/7yb3Tx4jG65ND8PIMQ7BuGwrO5sFjMOJSzF4kxtyodFl2jYF8J44cJbNhlbX+8HvjX\nE8rGREREV8diwI7M5hbUNtSgrrEa1XWVqK6rOL+cX6+33v7W4T2XY3TxQoBXLwR494K/ZwgCvEKg\n49SL1A0NjEhE4dlcAMCh7N0YFj2Gs1w5kHmTgP+lAEIAuw4BmXkC0aH8kYKIqKtiMXAFQgg0m5vQ\n0FiPhqZa1DfWob6xFvUXrzfWoLahBrUNVairr0Z9U12Hn9fF2Q3e7v7w8QiAt7s/vI3+8DEG8IRf\n6jH69roRW1K/Q31jLarqypFTdAwRwTFKh0XXKDRQwq1DBTbvtbY/Xgf843FlYyIioivr9sWAqbwQ\nzS2NaGpust62NKKxqd5629yApuYLt/VoaKpHY5P1tqG5vl2/4F8Lg7MrPFx94OHqbbN4uvvCoHPt\nlOckchROag0GhCdgb+ZWANZpRlkMOJZ5kyAXA9tSgewCgYgQjg4QEXVF3b4Y+OzHN6/bc0mSCgad\nC1yc3eCid4ebweP8YoS7wQOuBiPc9B7QanTXLSYiRxQXkYh9mdsgIJBbdAzl1aU8Id6BRPaSMHKQ\nwI40a/u9b4E3ODpARNQldftioCOc1BrotHo4a/Uw6Fyh17lArzVYb88vBmc365d/Z+t+HttM1HEe\nrt4IC4pCzplMAMChk7sxauBkhaOitnhwCuRiYHsqcDRHICacowNERF1Nty8GfD2CoHXSQuOkg1aj\nhUatg07rDK3GGbrzi1bjDK2TDs5a/fkv/wboNHpemZdIQQMjEuVi4HDOXiTFjue/SQfSr7eE2xJa\nzx1Y9g2w9CllYyIiokt1+2Jg7kT+70PkiMICo+Du4omq2nLUN9bixOmDiAkbonRY1AYP/Q74eT9g\nsQB7jwIHjgnER3F0gIioK1EpHQAR0eWoVGrE9UmU2+nnL0ZGjqN3gITJN7W2l31jnamNiIi6DhYD\nRNRlDeiTIJ+Hc6Y0DyXlhQpHRG31/+4AnM6fSnUoC/jlkLLxEBGRLRYDRNRluTi7oV+vG+V2WtYv\nCkZD7RHoI+H3o1vb730DWCwcHSAi6ipYDBBRlzYwMklez8hLRV1jjYLRUHvcNxlwPj+j8vF863kE\nRETUNbAYIKIuLdgnDP6eIQCAFnMzDmXvUTgiaitvo4TpY1vbH3wHmM0cHSAi6gpYDBBRlyZJEuL7\n3Sy307N+gbmTrg5OnWf2BMDVYF3PKwLW71I2HiIismIxQERdXr8b4uDi7AYAqK6vxInTPAvV0Rhd\nJcwa39r+8AegqZmjA0RESmMxQERdnpNag7iI1mlGU4/vVDAaaq8ZtwKe1poORaXAtzuUjYeIiFgM\nEJGDGBiRCLXKep3EM2WncKb0lMIRUVu56CXMndTa/vB7oLaeowNEREpiMUBEDsFF746o3gPlduoJ\njg44ortGAwHe1vXyauDjdYqGQ0TU47EYICKHEd9vpLx+/PQhVNdVKBgNtYdOK+Hxu1vb/90MFJ7l\n6AARkVJYDBCRw/D3DEaIbzgAwGIxIz07ReGIqD3GJQCx1j8jmluApV8rGw8RUU/GYoCIHMrF04we\nzE5Bc0uzgtFQe0iShCemt7Y37wUOZXN0gIhICSwGiMihRATHwN3FEwBQ31iLzFOpCkdE7REXKeHW\noa3tf64ChGBBQER0vbEYICKHolKpMSjyJrmdemInv0Q6qMfvBjTWCaJwJMc6QkBERNcXiwEicjgD\nwodBo9YCAM5WFCHflK1wRNQewb4SZt7W2n7nK6CxiYUdEdH1xGKAiByOXmdATNgQuc1pRh3XvEmt\nFyIrLrPOLkRERNcPiwEickiD+42Q108WZqCs0qRgNNRergYJD/2utf3pBqCskqMDRETXC4sBInJI\n3u7+6BPUHwAgILAnc4vCEVF7/W4kEB5sXa+tBz74Ttl4iIh6EhYDROSwhvUfK69n5qWisuacgtFQ\ne6nVEhZMa21/uwM4kc/RASKi64HFABE5rGDfUPTy6wMAsAgL9h3bpmxA1G5JAyQMj7WuWyzA618A\nFgsLAiKiztbhYmDx4sVQqVQ2S1BQ0CV9goODYTAYMGbMGGRkZNjsb2xsxPz58+Hr6wtXV1fceeed\nKCwstOlTXl6O2bNnw8PDAx4eHpgzZw4qKys7Gj4RObjhMbfK64dy9qC2vkrBaKgjnr4HcFJb1w9n\nAz/8omw8REQ9gV1GBqKiolBcXCwvhw8flvctWbIEb775Jv79739j37598PPzw2233Yaamhq5zxNP\nPIG1a9di1apV2LlzJ6qqqjB58mRYLBa5zz333IP09HRs3LgRP/74I1JTUzF79mx7hE9EDqy3fyQC\nvHoBAMzmFuw/vkPhiKi9egdImHN7a/udL4GKao4OEBF1JrsUA2q1Gn5+fvLi7e0NwHo1ybfeeguL\nFi3C1KlTERMTgxUrVqC6uhorV64EAFRWVuLjjz/GG2+8gbFjx2LQoEH4/PPPcejQIfz0008AgMzM\nTGzcuBEffPABhg0bhuHDh+P999/HunXrcOLECXu8BCJyUJIk2Zw7kJ6dgvrGOgUjoo64bxIQ7Gtd\nr6yxXnuAiIg6j12KgZycHAQHByM8PBwzZ85Ebm4uACA3Nxcmkwnjxo2T+zo7O2PkyJHYtWsXAODA\ngQNobm626RMSEoLo6GikpKQAAFJSUuDq6orExES5T1JSElxcXOQ+RNRzRYTEwNvoDwBoam5AWhaP\nL3FUOq2Ep2a2tr/fCRzM4ugAEVFn6XAxMHz4cKxYsQIbN27E8uXLUVxcjKSkJJw7dw7FxcUAAH9/\nf5v7+Pn5yfuKi4uhVqvl0YQL/P39bfr4+vra7JckyeZxiKjnUkkqDIu+RW6nHt+BFnOzghFRR9w8\nUMKowa3t1z8HzGYWBEREncGpow8wYcIEeT02NhaJiYkICwvDihUrMGzYsCveT5Kkqz6uEPb54N+/\nf79dHodaMaedg3ntGIuwQDSrUN9Ug7raOuQbjiPcN5Z57SSdnddb+zvh59190Ngi4eAJ4LXlJkwY\nUt6pz9kV8P3aOZjXzsG82l9kZOR1f067Ty1qMBgQExOD7OxsBAYGAgBMJtsrg5pMJgQEBAAAAgIC\nYDabUVZWdtU+Z8+etdkvhEBJSYnch4h6NpWkQh+/G+V2TslhmC1mBSOijvBxb8Hvklo/99fu8kVZ\nVYd/vyIiol+x+ydrQ0MDMjMzccsttyAsLAwBAQHYtGkT4uPj5f3Jycl44403AADx8fHQaDTYtGkT\nZs60HihaUFCAY8eOISkpCQCQmJiImpoapKSkyOcNpKSkoLa2Vu5zJUOGDLH3S+yxLvwCwJzaF/Nq\nP3EtcTCty0FtfRXqautQWJ6F34+fpXRY3cr1fL/GDRTIKAJyzs80vfV4HF5/5Oqjyo6KnwOdg3nt\nHMxr51Fi2vwOjww8/fTT2LFjB3Jzc7Fnzx7cfffdqK+vx7333gvAOm3okiVL8M033+DIkSOYO3cu\n3NzccM899wAAjEYj7r//fjzzzDP4+eefkZaWhtmzZyMuLg633mqdPzw6OhoTJkzAQw89hN27dyMl\nJQUPPfQQ7rjjDkWGU4ioa9I4aTCk30i5fbLkECwcHXBYGicJz15Uy/28D0g5wnMHiIjsqcPFQGFh\nIWbOnImoqCjcdddd0Ov12L17N3r1ss77/cwzz+DJJ5/EY489hqFDh8JkMmHTpk1wcXGRH+Ott97C\n1KlTMX36dIwYMQLu7u744YcfbM4rWLlyJeLi4jB+/HhMmDBBnoKUiOhicRGJcNYaAAB1jdXIPJWm\ncETUEYP7Sbj9ogHg1z4DautZEBAR2Ysk7HWmbhdy8RCL0WhUMJLuhcOCnYN5tb9fDm/ET3u+BQAE\n+AXjvtufhcZJo3BU3YMS79dzVQLT/mq97gAA/H40sGhO9zpciJ8DnYN57RzMa+dR4jus3U8gJiJS\n2pB+I6Fx0gEAqmrLkZ69S+GIqCO83CU8fU9re+02YG9Gt/sdi4hIESwGiKjb0Wn1iPQfJLd3H/2J\nVyV2cOOHwebaAy99wsOFiIjsgcUAEXVLvb2jYNC5AQAamuqwN3OLwhFRR0iShEWzAaOrtV1cBvxr\njbIxERF1BywGiKhbUqnU6BfYejzrgRM7UVXb/S9a1Z15GyU888fW9jfbgT1HOTpARNQRLAaIqNsK\nNIYh0PsGAIDZ3ILkQz8qHBF11G0JwJj41vbLn/JwISKijmAxQETdliRJGBk3SW5n5B2AqbxQwYio\noyTJeu0BD+sRYCguA/65WtmYiIgcGYsBIurWbvCPQJ+g/gAAAYEd6esVjog6ytso4c8XzS703Q5e\njIyIqL1YDBBRt3dz3O2QJOvHXV7xceQVn1A4Iuqo2xKAWy6a4vyVT4HKGhYERERtxWKAiLo9X49A\nDAgfKre3p6+DRVgUjIg66teHC5nOAS99CnTD62gSEXUqFgNE1CMkxY6HRq0FAJSUFyIzL1XhiKij\nvNwlPH9va3t7KrDmZ+XiISJyRCwGiKhHcDMYER81Um7vPLgBjU31CkZE9jB6sIRpY1vb/1oDZOZx\ndICI6FqxGCCiHiMhegwMztarVlXXV2L7QZ5M3B0smAZE9bauN7cAz73H6UaJiK4ViwEi6jF0GmeM\nHTxVbh/MTkG+KVvBiMgetBoJrz4MuOit7YIS4NXPeP4AEdG1YDFARD1KvxviEBEcI7c37l2D5pYm\nBSMie+jlL+G5Oa3tTXuA73YqFw8RkaNgMUBEPYokSbh1yF3QaZwBABU1ZUg+zCsTdwfjhkmYOqq1\n/cZK4GQBRweIiK6GxQAR9ThuBiPGDL5Tbh84vhNFZfkKRkT28tRMICLEut7YBCx6D6hvZEFARHQl\nLAaIqEeKDRuK3gF9AQBCWPDjntUwm1sUjoo6Sqe1nj/grLO2c88AL33C8weIiK6ExQAR9UiSJGHc\n0Lvlaw+UVhZjdwYnqe8OwoKsFyS7YPNeYPn3ysVDRNSVsRggoh7Lw9UbN8fdLrd3Z/yMsxVFCkZE\n9jL5Jgl/uOj6A8u/Azbt4egAEdGvsRggoh5tUGQSgnxCAQAWixkb966BxWJWNiiyi6dmAMNaJ47C\nix8DR3JYEBARXYzFABH1aCqVGhMSpkGtcgIAFJXlY+eh/ykcFdmDWi3htYeB0EBru6kZePodoLiM\nBQER0QUsBoiox/M2+iMpdpzc3pu5FZmn0hSMiOzFzUXCPxcAHm7Wdlkl8NQ7QF0DCwIiIoDFABER\nACCh/xj0CeovtzfuWQNTeaGCEZG9hPhJWPIo4KS2tk/kA39bDpjNLAiIiFgMEBEBUEkqTEq8B17u\nfgCAZnMTvt35CeoaaxSOjOxhcD8Jiy66QvGONOCdr5SLh4ioq2AxQER0nk6rx+9ungvt+asTV9WW\n44fkz3lCcTcx5WYJsye2tv+zEfjwe44OEFHPxmKAiOgi3u7+mJR4j9zOL8nGtvR1CkZE9vTY74FR\ng1vb738LrNjAgoCIei4WA0REvxIRHIObBkyQ2weO78DR3P0KRkT2olZLeOVBIOGiKUf//RWwchML\nAiLqmVgMEBFdxvCYsYgMGSC3N+77EkVl+QpGRPai00r4v8eB+KjWbf9cBaz5mQUBEfU8LAaIiC5D\nJakwcfgMeBv9AQBmcwu+3rYcpnMFCkdG9uCss045OrBv67Z//Af4ZjsLAiLqWVgMEBFdgU7jjKk3\nz4NOqwcA1DfVYfXW9zhC0E3odRLeWgAMiGjd9tpnwA/JLAiIqOdgMUBEdBWebr74w+iH5IKgsake\nX259H4Vn85QNjOzCRS/h7SeA/mHWthDAS58AX29lQUBEPQOLASKi3xDo3QvTb3kEep0LAKCxuQFf\nbfsABWdzFI6M7MHVIOGdhUC/G6xtIYDXPwfeWi14YTIi6vZYDBARXQN/z2BMG/MwDDpXAEBTSyO+\n2roc+aZshSMje3B3kbD0aSA6tHXbfzYCz74L1DeyICCi7ovFABHRNfLzDML0sY/AxdkNgPUqxWu3\nf4RTxScUjozswegq4f1nba9DsD0VeGgJUFrBgoCIuicWA0REbeBjDMCMsY/CVW8EYC0Ivt7+EdKy\nfoEQ/MLo6PQ6CUseAf44vnVbZh4w7xUg6zT/vkTU/bAYICJqIy93P8wY+wjcDZ4AALOlBT/tX4t1\nu75AY3ODwtFRR6nVEp6YLuEvswG12rqtuAx44HUg+SALAiLqXlgMEBG1g6ebL2aMfRT+niHytmP5\n6fhi41soKT+jYGRkL3eNsV6LwMU6kRRq64En/wX84z8CDTyPgIi6CRYDRETtZHT1wsxbH0dcRKK8\n7Vz1Wfxn89s4dHIPDxvqBhJjJXy4CAjwbt225mdg9t+BjFz+fYnI8bEYICLqAI2TBuOG3o1JiX+E\nxkkLAGgxN2Pj3jX43+5VPGyoG4gIkbDib8DIQa3b8oqA+18FPvqB048SkWNjMUBEZAf9Qwdj9vgn\n4GMMkLcdzduPj9YvwdHc/bAIi4LRUUd5uUt443Hgr/MAvc66rcUMvPeN9VyC0yYWBETkmFgMEBHZ\nibe7P2aNW4DYsKHyttr6KmzY/V/8d/O/UVSWr2B01FGSJOHOmyWsfBG4MbJ1++GTwD2Lgfe/Fait\nZ1FARI6FxQARkR1pnLSYOHwGJiX+ES56d3n7mbJT+GLTv/C/PatRW1+lYITUUSF+Ej54Bnj0LsDp\n/GxDDY3Ah98Ddz0HfL1VoKWFRQEROQYWA0REnaB/6GDcP+lZJESPgVrlJG8/krMXH65fgpSjP6G+\nsU7BCKkj1GoJ8yZJ+OSvQN8bWreXVQKvfw7MfAHYlip4EjkRdXksBoiIOolO44xRAydj3u1PIyI4\nRt7e1NyA5EP/w/vfv4SfD3yD8upSBaOkjojqLeGzvwEv3A/4e7VuzysC/vxv6/kEezNYFBBR1+X0\n212IiKgjPN18MXXkfcgtOoYtqd/hXFUJAKC5pQmpJ5KRduIXRITEYmjUKAT5hEKSJIUjprZQqyVM\nvgm4bajAqp+ATzcANecHfQ5mAY+9AYQGAnePEZh8E+Ci59+XiLoOFgNERNdJWGAU5k6MREbeARw4\nvgNnK4oAAAICWQWHkVVwGIHeN6B/aDz69brR5pwD6vp0Wgn33g7cebPAR+uAr7ZYZxwCrCMFb6wE\nlq0FJiYK/GEM0CeERQERKY/FABHRdaRWqTEgPAGxYUNxypSF/ce2I7fomLy/qCwfRWX52JL6HXr5\n9UHUDXHo2+tG6HUuCkZNbeHhJuGpmcD0sQL/3Qys32W9ejEA1DUAX2+1LnGRArfEA6MHXf3xiIg6\nE4sBIiIFSJKE0IC+CA3oi7MVRThwfAcyTqXCbG4BAAhhQb4pC/mmLPx04Bv09o9En+AY9PaPgKeb\nLw8lcgAhfhL+/Efg0d8L/G838OUWIKewdf/BLOvyz1WAj2soBkfUwOgvEBEC/n2J6LphMUBEpDBf\nj0BMGDYdI+Mm4fjpQzien46CkhwIWE86tVjMyC06Jo8guBk8cINfBHoHRKKXXwTcXTyUDJ9+g4te\nwt1jgLtGC6QeB9ZsAbanAWZza59TJc44VeKMTelAkA8QHyUQFwnERQC9A1gcEFHncchiYNmyZfjH\nP/6B4uJixMTE4K233sKIESOUDouIqEMMzq4YFJmEQZFJqK6rxInTB3Es/yDOlObZ9Kuuq8DRvP04\nmrcfAODh6o0Ar17w9wqBn2cw/D2DeVhRFyRJEuKjgPgooKxSYEc6sC0N2Jdh2+9MKXAmGfgh2do2\nugID+gjERQD9w4A+wYCXOwsEIrIPhysGVq9ejSeeeALvvvsuRowYgaVLl2LixInIyMhAr169lA6P\niMgu3AxGxPcbifh+I1FZew7ZBUdwypSNgpKTaGxusOlbUVOGipoyHMtPl7e5Gzzh5xkEb6M/PN18\n4eHqAy83Hxic3fglsgvwNkqYOgqYOgqorRf49OtCHMh2Q26pizwT0QWVNUDyQetygbsLEBYkEB4E\nhAUBYYFAoA8Q4GU9kZmI6Fo5XDHw5ptvYt68ebj//vsBAG+//TZ+/PFHvPvuu3j11VcVjo6IyP6M\nLl5yYWCxmFF8rgD5pmzkl2Sj8GwuWszNl9ynqq4cVXXlyC48arNdq3GGp6sPjK5ecNUb4ap3P78Y\n4WYwwkXvDq2TjgXDdeSilzAsqhrDoqpxY1wgMnKBg9nW5fBJoKL60vtU1baec/BrXu4CAd6wLl6A\nt9E6kuDpZl0urLNoICLAwYqBpqYmpKam4plnnrHZPm7cOOzatUuhqIiIrh+VSo0gn94I8umN4TFj\n0WJuxtmKYpSUF8BUXoiS8kKUVJyRT0T+tabmBpjKC2AqL7jic6hVTnDWGaDXGqDXGeCsdYFeZ4BO\no8eZ4iI4qbXQZLdAq9FB66SDxkkLjZMWapUTnNROcFJr4aR2glrtBLVKDZVKDZXEa1xeC61GwsC+\nwMC+1rYQAqeKgUPni4OcQiC3qHV2oss5V2VdMnKv/lx6nYCrAXDVo/X2/KLXWRdn7flbHaDXAjot\noNUAWidA4wToNNZbrQZwUl9+Uat5SBNRV+ZQxUBpaSnMZjP8/f1ttvv5+aG4uFihqIiIlOOk1iDQ\nuxcCvVsPkzRbzCirNOFsxRmcqz6LypoynKs+i4rq0ksOMbocs6UFtfVVqK2vumRfXa31GJZTFYfb\nFKdKUlmLApW6tUCABElSQVKpoJJUkCTp/K11XYIESJALCWtbggScv5Vg/Y4pte6X91lvf03uc8lu\n277X+8tricl6IbpTtenno7n0+WPCrYsQQGWNM4rPuaOk3BWmc24orXBFebUeVbV6WMS1xV5eA6DM\nbi/hqlSSgCQJqFXWW5UkoLpo3fp3tkCSAEkSkIDWfYC8/eJ1uY0L7dbnk2B9zIZ6J0iSgP7z1ul7\nL/STIC6+g+19cdm3z/nt4qK+v955+StN2/vddF3enld4LQBQV2s9+92wpm2fA0q5dYgWC6b1UzqM\nLsuhioH2qKysVDqEbiMyMhIAc2pvzGvn6Ol51alcEOIViRCvSKVDISJSXE/9v+BaONS4rY+PD9Rq\nNUwmk812k8mEwMBAhaIiIiIiInJMDlUMaLVaxMfHY9OmTTbbN2/ejKSkJIWiIiIiIiJyTA53PqsA\nSAAACwxJREFUmNDChQsxe/ZsJCQkICkpCe+99x6Ki4vx8MMPy32MRqOCERIREREROQaHKwamTZuG\nsrIyvPzyyygqKsKAAQOwYcMGXmOAiIiIiKiNJCHElU8XJyIiIiKibsuhzhm4FsuWLUNYWBj0ej2G\nDBmC5ORkpUNSzI4dOzBlyhSEhIRApVJhxYoVl/RZvHgxgoODYTAYMGbMGGRkZNjsb2xsxPz58+Hr\n6wtXV1fceeedKCwstOlTXl6O2bNnw8PDAx4eHpgzZ84lZ+3n5+fjjjvugKurK3x9fbFgwQI0N196\noSRH8Nprr2Ho0KEwGo3w8/PDlClTcPTo0Uv6Mbdts3TpUsTFxcFoNMJoNCIpKQkbNmyw6cOcdtxr\nr70GlUqF+fPn22xnbttm8eLFUKlUNktQUNAlfZjTtisqKsK9994LPz8/6PV6xMTEYMeOHTZ9mNu2\nCQ0NveT9qlKpMHnyZADW61kwp23X0tKC5557DuHh4dDr9QgPD8ff/vY3mM1mm35dPreiG1m1apXQ\naDTiww8/FMeOHRPz588Xrq6uIj8/X+nQFLFhwwbx/PPPi6+++koYDAaxYsUKm/2vv/66cHNzE2vX\nrhVHjhwR06ZNE0FBQaK6ulru8/DDD4ugoCDx008/idTUVDF69GgxcOBAYTab5T4TJkwQsbGxYvfu\n3SIlJUXExMSIO+64Q97f0tIiYmNjxZgxY0RaWprYvHmzCAoKEvPnz+/8JHSC8ePHi08//VQcPXpU\nHD58WEydOlUEBASIc+fOyX2Y27b77rvvxI8//ihOnjwpsrKyxPPPPy80Go1IT08XQjCn9pCSkiLC\nwsJEXFyczethbtvuhRdeENHR0cJkMslLaWmpvJ85bZ/y8nIRFhYm7r33XrFv3z6Rl5cntmzZIjIz\nM+U+zG3blZaW2rxX09LShEqlEp999pkQgjltrxdffFF4eXmJdevWiVOnTonvv/9eeHl5iZdeeknu\n4wi57VbFQEJCgnjwwQdttkVGRopFixYpFFHX4erqalMMWCwWERAQIF599VV5W319vXBzcxPvv/++\nEEKIiooKodVqxcqVK+U+p0+fFiqVSmzcuFEIIURGRoaQJEns2rVL7pOcnCwkSRInTpwQQliLEpVK\nJQoKCuQ+X3zxhXB2drb5x+CoampqhFqtFuvWrRNCMLf25OXlJT744APm1A4qKipEnz59xLZt28To\n0aPl/yCY2/Z54YUXRGxs7GX3Maftt2jRIjFixIgr7mdu7ePll18Wnp6eoqGhgTntgMmTJ4u5c+fa\nbJszZ46YPHmyEMJx3q/d5jChpqYmpKamYty4cTbbx40bh127dikUVdeVm5sLk8lkky9nZ2eMHDlS\nzteBAwfQ3Nxs0yckJATR0dFISUkBAKSkpMDV1RWJiYlyn6SkJLi4uMiPk5KSgv79+yM4OFjuM27c\nODQ2NuLAgQOd+jqvh6qqKlgsFnh6egJgbu3BbDZj1apVaGhowMiRI5lTO3jwwQfxhz/8AaNGjYK4\n6FQx5rb9cnJyEBwcjPDwcMycORO5ubkAmNOO+Pbbb5GQkIDp06fD398fgwYNwtKlS+X9zG3HCSHw\n0UcfYdasWdDpdMxpB0ycOBFbtmzB8ePHAQAZGRnYunUrJk2aBMBx3q8ON5vQlZSWlsJsNsPf399m\nu5+fH4qLixWKquu6kJPL5evMmTNyH7VaDW9vb5s+/v7+8v2Li4vh6+trs1+SJJu8FxcXX/I8Fy4g\n1x3+NgsWLMCgQYPkf6TMbfsdPnwYiYmJaGxshF6vx5o1a9CvXz/5w445bZ/ly5cjJycHK1euBGB9\nzRfw/do+w4cPx4oVKxAVFQWTyYSXX34ZSUlJOHr0KHPaATk5OVi2bBkWLlyI5557DmlpafL5LY89\n9hhzawebN29GXl4eHnjgAQD8DOiIRx99FAUFBYiOjoaTkxNaWlrw17/+VZ7u3lFy222KAbKfi78o\nXI5oxwRU7bmPI1i4cCF27dqF5OTk38wbwNz+lqioKBw6dAiVlZX48ssvMWPGDGzduvWq92FOr+74\n8eN4/vnnkZycDLVaDcD6+q7lNTK3VzZhwgR5PTY2FomJiQgLC8OKFSswbNiwK96POb06i8WChIQE\nvPLKKwCAuLg4ZGVlYenSpXjssceuel/m9tosX74cCQkJGDBgwG/2ZU6v7u2338Ynn3yCVatWISYm\nBmlpaViwYAFCQ0Nx3333XfW+XSm33eYwoQvVj8lkstluMpkQGBioUFRdV0BAAABcNl8X9gUEBMBs\nNqOsrOyqfc6ePWuzXwiBkpISmz6/fp4LIzkX+jiiJ598EqtXr8aWLVsQGhoqb2du20+j0SA8PByD\nBg3Cq6++iuHDh2Pp0qXyv2HmtO1SUlJQWlqKmJgYaDQaaDQa7NixA8uWLYNWq4WPjw8A5rajDAYD\nYmJikJ2dzfdrBwQFBaF///4226KiopCfnw+An68dVVJSgu+//14eFQCY04545ZVX8Nxzz2HatGmI\niYnBrFmzsHDhQrz22msAHCe33aYY0Gq1iI+Px6ZNm2y2b968GUlJSQpF1XWFhYUhICDAJl8NDQ1I\nTk6W8xUfHw+NRmPTp6CgAMeOHZP7JCYmoqamRj6uDbB++aitrZX7JCUlITMz02aarM2bN0On0yE+\nPr5TX2dnWbBggVwI9O3b12Yfc2s/ZrMZFouFOe2AqVOn4siRIzh48CAOHjyI9PR0DBkyBDNnzkR6\nejoiIyOZWztoaGhAZmYmAgMD+X7tgJtuugnHjh2z2XbixAn5BxfmtmM+/fRTODs7Y+bMmfI25rT9\nhBBQqWy/SqtUKvkXeofJ7VVPL3Ywq1evFlqtVnz44YciIyND/OlPfxJubm49dmrRmpoakZaWJtLS\n0oTBYBB///vfRVpampyPJUuWCKPRKNauXSsOHz4spk+fLoKDg0VNTY38GI888ogICQmxme5q0KBB\nwmKxyH0mTpwoBgwYIFJSUsSuXbtEbGysmDJlirzfbDaLAQMGiFtuuUWe7io4OFj86U9/un7JsKNH\nH31UuLu7iy1btoiioiJ5uThvzG3bPfvss2Lnzp0iNzdXHDp0SPzlL38RKpVKbNq0SQjBnNrTqFGj\nxOOPPy63mdu2e+qpp8T27dtFTk6O2L17t5g0aZIwGo38fO2gffv2CY1GI1555RWRlZUl1qxZI4xG\no1i2bJnch7ltH4vFIiIjIy+ZdVEI5rS9HnjgARESEiLWr18vcnNzxdq1a4Wvr694+umn5T6OkNtu\nVQwIIcSyZctEaGio0Ol0YsiQIWLnzp1Kh6SYrVu3CkmShCRJQqVSyevz5s2T+yxevFgEBgYKZ2dn\nMXr0aHH06FGbx2hsbBTz588X3t7ewmAwiClTpthMWyWEdV7oWbNmCXd3d+Hu7i5mz54tKisrbfrk\n5+eLyZMnC4PBILy9vcWCBQtEU1NT5734TvTrfF5YXnzxRZt+zG3bzJ07V/Tu3VvodDrh5+cnbrvt\nNrkQuIA5tY+Lpxa9gLltmxkzZoigoCCh1WpFcHCwuPvuu23mwheCOW2v9evXi7i4OOHs7Cz69esn\n3nnnnUv6MLdtt2XLFqFSqcS+ffsuu585bbuamhrx1FNPidDQUKHX60V4eLh4/vnnRWNjo02/rp5b\nSYhudCYHERERERFds25zzgAREREREbUNiwEiIiIioh6KxQARERERUQ/FYoCIiIiIqIdiMUBERERE\n1EOxGCAiIiIi6qFYDBARERER9VAsBoiIiIiIeigWA0REREREPdT/B/d1uO30WoozAAAAAElFTkSu\nQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "player1.MakeBeliefs(20000)\n", + "player2.MakeBeliefs(40000)\n", + " \n", + "calc1 = GainCalculator(player1, player2)\n", + "calc2 = GainCalculator(player2, player1)\n", + "\n", + "bids, gains = calc1.ExpectedGains()\n", + "thinkplot.Plot(bids, gains, label='Player 1')\n", + "print('Player 1 optimal bid', max(zip(gains, bids)))\n", + "\n", + "bids, gains = calc2.ExpectedGains()\n", + "thinkplot.Plot(bids, gains, label='Player 2')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 6.4: Expected gain versus bid in a scenario where Player 1’s best guess is\n", + "\\$20,000 and Player 2’s best guess is \\$40,000." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "If you overbid, you get nothing. Otherwise we compute the difference\n", "between your bid and the price, which determines your probability of\n", "winning.\n", "\n", - "If diff is less than \\$250, you win both showcases. For\n", + "If `diff` is less than \\$250, you win both showcases. For\n", "simplicity, I assume that both showcases have the same price. Since this\n", "outcome is rare, it doesn’t make much difference.\n", "\n", "Finally, we have to compute the probability of winning based on\n", - "diff:\n", - "\n", - " def ProbWin(self, diff):\n", - " prob = (self.opponent.ProbOverbid() + \n", - " self.opponent.ProbWorseThan(diff))\n", - " return prob\n", - "\n", + "`diff`:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "def ProbWin(self, diff):\n", + " prob = (self.opponent.ProbOverbid() + \n", + " self.opponent.ProbWorseThan(diff))\n", + " return prob\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "If your opponent overbids, you win. Otherwise, you have to hope that\n", - "your opponent is off by more than diff. Player\n", + "your opponent is off by more than `diff`. `Player`\n", "provides methods to compute both probabilities:\n", "\n", + "```python\n", " # class Player:\n", "\n", " def ProbOverbid(self):\n", @@ -535,19 +1073,21 @@ "\n", " def ProbWorseThan(self, diff):\n", " return 1 - self.cdf_diff.Prob(diff)\n", + "```\n", "\n", "This code might be confusing because the computation is now from the\n", "point of view of the opponent, who is computing, “What is the\n", "probability that I overbid?” and “What is the probability that my bid is\n", - "off by more than diff?”\n", + "off by more than `diff`?”\n", "\n", - "Both answers are based on the CDF of diff. If the\n", - "opponent’s diff is less than or equal to -1, you win. If\n", - "the opponent’s diff is worse than yours, you win. Otherwise\n", + "Both answers are based on the CDF of `diff`. If the\n", + "opponent’s `diff` is less than or equal to -1, you win. If\n", + "the opponent’s `diff` is worse than yours, you win. Otherwise\n", "you lose.\n", "\n", "Finally, here’s the code that computes optimal bids:\n", "\n", + "```python\n", " # class Player:\n", "\n", " def OptimalBid(self, guess, opponent):\n", @@ -556,9 +1096,10 @@ " bids, gains = calc.ExpectedGains()\n", " gain, bid = max(zip(gains, bids))\n", " return bid, gain\n", + "```\n", "\n", - "Given a guess and an opponent, OptimalBid computes the\n", - "posterior distribution, instantiates a GainCalculator,\n", + "Given a guess and an opponent, `OptimalBid` computes the\n", + "posterior distribution, instantiates a `GainCalculator`,\n", "computes expected gains for a range of bids and returns the optimal bid\n", "and expected gain. Whew!\n", "\n", @@ -572,10 +1113,14 @@ "\n", "For Player 2 the optimal bid is \\$31,500, yielding an expected return of\n", "almost \\$19,400. This is the more typical case where the optimal bid is\n", - "less than the best guess.\n", - "\n", - "Discussion\n", - "----------\n", + "less than the best guess." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Discussion\n", "\n", "One of the features of Bayesian estimation is that the result comes in\n", "the form of a posterior distribution. Classical estimation usually\n", @@ -597,7 +1142,7 @@ "Bayesian methods are most useful when you can carry the posterior\n", "distribution into the next step of the analysis to perform some kind of\n", "decision analysis, as we did in this chapter, or some kind of\n", - "prediction, as we see in the next chapter.\n" + "prediction, as we see in the next chapter." ] } ], diff --git a/notebooks/07_Prediction.ipynb b/notebooks/07_Prediction.ipynb index a9bd746..d12706a 100644 --- a/notebooks/07_Prediction.ipynb +++ b/notebooks/07_Prediction.ipynb @@ -1,15 +1,271 @@ { "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "%load_ext autoreload\n", + "%autoreload 2 \n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ + "#Prediction\n", "\n", - "Prediction\n", - "==========\n", - "\n", - "The Boston Bruins problem\n", - "-------------------------\n", + "##The Boston Bruins problem\n", "\n", "In the 2010-11 National Hockey League (NHL) Finals, my beloved Boston\n", "Bruins played a best-of-seven championship series against the despised\n", @@ -59,37 +315,53 @@ " pmf.Normalize()\n", " return pmf\n", "\n", - "mu and sigma are the mean and standard\n", + "`mu` and `sigma` are the mean and standard\n", "deviation of the Gaussian distribution. `num_sigmas` is the number of\n", "standard deviations above and below the mean that the Pmf will span, and\n", - "n is the number of values in the Pmf.\n", + "`n` is the number of values in the Pmf.\n", "\n", - "Again we use numpy.linspace to make an array of\n", - "n equally spaced values between low and\n", - "high, including both.\n", + "Again we use `numpy.linspace` to make an array of\n", + "`n` equally spaced values between `low` and\n", + "`high`, including both.\n", "\n", "`norm.pdf` evaluates the Gaussian probability density function (PDF).\n", "\n", "Getting back to the hockey problem, here’s the definition for a suite of\n", - "hypotheses about the value of $\\lambda$.\n", - "\n", - " class Hockey(thinkbayes.Suite):\n", - "\n", - " def __init__(self):\n", - " pmf = thinkbayes.MakeGaussianPmf(2.7, 0.3, 4)\n", - " thinkbayes.Suite.__init__(self, pmf)\n", + "hypotheses about the value of $\\lambda$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Hockey(thinkbayes.Suite):\n", "\n", + " def __init__(self):\n", + " pmf = thinkbayes.MakeGaussianPmf(mu=2.7, sigma=0.3, num_sigmas=4)\n", + " thinkbayes.Suite.__init__(self, pmf)\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "So the prior distribution is Gaussian with mean 2.7, standard deviation\n", "0.3, and it spans 4 sigmas above and below the mean.\n", "\n", "As always, we have to decide how to represent each hypothesis; in this\n", "case I represent the hypothesis that $\\lambda=x$ with the floating-point\n", - "value x.\n", - "\n", - "Poisson processes\n", - "-----------------\n", + "value `x`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Poisson processes\n", "\n", - "In mathematical statistics, a **process** is a stochastic\n", + "In mathematical statistics, a **process** is a stochastic\n", "model of a physical system (“stochastic” means that the model has some\n", "kind of randomness in it). For example, a Bernoulli process is a model\n", "of a sequence of events, called trials, in which each trial has two\n", @@ -115,24 +387,56 @@ "The benefit of using this model is that we can compute the distribution\n", "of goals per game efficiently, as well as the distribution of time\n", "between goals. Specifically, if the average number of goals in a game is\n", - "lam, the distribution of goals per game is given by the\n", - "Poisson PMF:\n", - "\n", - " def EvalPoissonPmf(k, lam):\n", - " return (lam)**k * math.exp(-lam) / math.factorial(k)\n", - "\n", + "`lam`, the distribution of goals per game is given by the\n", + "Poisson PMF:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def EvalPoissonPmf(k, lam):\n", + " return (lam)**k * math.exp(-lam) / math.factorial(k)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "And the distribution of time between goals is given by the exponential\n", - "PDF:\n", - "\n", - " def EvalExponentialPdf(x, lam):\n", - " return lam * math.exp(-lam * x)\n", - "\n", - "I use the variable lam because lambda is a\n", + "PDF:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def EvalExponentialPdf(x, lam):\n", + " return lam * math.exp(-lam * x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "I use the variable `lam` because `lambda` is a\n", "reserved keyword in Python. Both of these functions are in\n", - "`thinkbayes.py`.\n", - "\n", - "The posteriors\n", - "--------------\n", + "`thinkbayes.py`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The posteriors\n", "\n", "![Posterior distribution of the number of goals per\n", "game.](figs/hockey1.pdf)\n", @@ -140,7 +444,7 @@ "[fig.hockey1]\n", "\n", "Now we can compute the likelihood that a team with a hypothetical value\n", - "of lam scores k goals in a game:\n", + "of `lam` scores `k` goals in a game:\n", "\n", " # class Hockey\n", "\n", @@ -150,36 +454,72 @@ " like = thinkbayes.EvalPoissonPmf(k, lam)\n", " return like\n", "\n", - "Each hypothesis is a possible value of $\\lambda$; data is\n", - "the observed number of goals, k.\n", + "Each hypothesis is a possible value of $\\lambda$; `data` is\n", + "the observed number of goals, `k`.\n", "\n", "With the likelihood function in place, we can make a suite for each team\n", - "and update them with the scores from the first four games.\n", - "\n", - " suite1 = Hockey('bruins')\n", - " suite1.UpdateSet([0, 2, 8, 4])\n", - " \n", - " suite2 = Hockey('canucks')\n", - " suite2.UpdateSet([1, 3, 1, 0])\n", + "and update them with the scores from the first four games." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvQAAAEWCAYAAAAJuAMCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcU1feP/BPFgKEfSeyo4KAiEAURa1dlJZaO7aoVRyt\nVjujta3L428cWrv4VDu1Tn1qF9E6rqOtWpdO7VCVVutKVaQgm4KyCwFB9iWB5P7+CNwQiYAK3EC+\n79freb2ecznn5hvnNvnm3nO+h8cwDANCCCGEEEJIv8TnOgBCCCGEEELIo6OEnhBCCCGEkH6MEnpC\nCCGEEEL6MUroCSGEEEII6ccooSeEEEIIIaQfo4SeEEIIIYSQfowSekIIIYQQQvqxbiX0W7ZsgZeX\nF0xNTSGVSnHhwoVO+6empmLixIkQi8VwdXXFRx99pPV3mUyG6Oho+Pn5QSgUYsGCBTrP89lnn8HX\n1xdisRhubm548803UV9f3823RgghhBBCyMDXZUJ/8OBBLF++HGvWrEFycjLCw8MRGRmJwsJCnf1r\namowefJkSCQSJCYmYvPmzdi4cSM2bdrE9pHL5XBwcEBMTAzCwsLA4/E6nGfv3r1499138f777+PG\njRvYu3cv4uLisGzZssd4u4QQQgghhAwsvK52ig0LC8PIkSOxbds29piPjw+mT5+Ojz/+uEP/2NhY\nxMTEoLS0FMbGxgCA9evXIzY2FkVFRR36T506FQ4ODti5c6fW8TfffBNpaWn47bff2GMffPABjh49\nitTU1Id6k4QQQgghhAxUnd6hVygUSEpKQkREhNbxiIgIXLp0SeeYhIQETJgwgU3m2/oXFxcjPz+/\n24FFRkYiJSUFly9fBgAUFBTgxx9/xJQpU7p9DkIIIYQQQga6ThP68vJyKJVKODk5aR13dHSETCbT\nOUYmk3Xo39Z+0BhdpkyZgvXr12PChAkQiUTw9PREUFAQPvnkk26fgxBCCCGEkIFO2NMn1DUf/lEc\nO3YM77zzDrZu3YqwsDBkZ2dj2bJl+OCDD7B27Vq2X3V1dY+8HiGEEEIIIVyxsrJ65LGdJvT29vYQ\nCAQoLS3VOl5aWgqJRKJzjLOzc4c78W3jnZ2dux3YJ598goULF+K1114DAAQEBKC+vh6LFi3CBx98\nAD6fKm4SQgghhBDSaVYsEokQGhqKU6dOaR2Pj49HeHi4zjFjx47F+fPnIZfLtfq7uLjAw8Oj24Ex\nDNMhaefz+ehiDS8hhBBCCCEGpcspNytXrsTcuXMxevRohIeHY+vWrZDJZFi8eDEAICYmBlevXsUv\nv/wCAIiOjsbatWsxf/58rFmzBjdv3sSGDRvw4Ycfap03OTkZgHrKDJ/PR3JyMkQiEfz9/QEA06ZN\nw4YNGyCVSjF69GjcunUL7733HqZOnfrAu/OP86iCDGyJiYkAAKlUynEkRJ/RdUK6g64T0h10nZDu\n6Kmp410m9DNnzkRFRQXWrVuHkpISBAYGIi4uDm5ubgDUC11zcnLY/paWloiPj8fSpUshlUpha2uL\nVatWYcWKFVrnDQkJAaCec88wDI4fPw5PT0/2XKtXrwbDMHjvvfdQVFQEBwcHTJ06FevXr++RN04I\nIYQQQshA0GUden3X/pcN3aEnD0J3Skh30HVCuoOuE9IddJ2Q7uipPJZWlhJCCCGEENKPUUJPCCGE\nEEJIP0YJPSGEEEIIIf0YJfSEEEIIIYT0Yz2+UywhhBDSU+obFCi724Dyew1QqdQ1HHgAxGIjODqY\nwc7GFAIB3ZsihBg2SugJIYToBUWzErdy7iEt8y7Sb5QhJ68K9Q2KTscIBHw4OZjBz8cew/0cEejv\nCFsb0z6KmBBC9AMl9IQQQjjDMAxuZFfgzPk8XLxciMam5ocar1SqUCyrRbGsFr+eywUADPW2w9NP\neGJcmBvMzUS9ETYhhOgVSugJIYT0ueYWJU6fy8PxE1koltU+sJ/ISAAHezPY24khMlJPrWEAVFfL\ncbeiHlXVTR3GZOdUIDunAjv3JSM8zBVRU/3gOsiyt94KIYRwjhJ6QgghfaYtkT9yPBPlFQ0d/u5o\nb4YRAU4Y7ucAf18H2NmagsfjPfB8cnkLcvKrkJZZhtSMMmRmlUOpVLGvdfZiPs5dKsCEMe6Y/idK\n7AkhAxMl9IQQQvpESnoptu2+BllpndZxUxMjjB/jhifHe2LYULtOE/j7GRsL4edjDz8fe8z4kz9q\n6+Q4d6kAZ87nISe/EoB6Ws+5hHyc/70AkZMGI3p6IMSmRj363gghhEuU0BNCCOlVlVVN2P1dMs4n\nFGgdt7QwxrQpw/Ds094wNemZBNvC3BhTIoZiSsRQZN2qwKEfMpB0vQSAOrGPi7+FhCt38NqfRyJ8\ntOtD/XgghBB9RQk9IYSQXpNwtQixOxNRV6+pViM2NULUi36IfGYITEx672vIZ4gd1qyagKxbFfj2\ncBquZ5QCACqrG/HZ1wk4n+CCpYuksDA37rUYCCGkL1BCTwghpMcpFErsOZCCn3+5pXV8wlh3zJ89\nEjbWJn0Wi88QO3yw+glcvFyIXftTUFndCAC4knQHOe9VYvniMPj7OvRZPIQQ0tMooSeEkD7EMAxK\n6xXIrWpERWMzKhubUdnUjBYVAwGfBwGPBxMhH45mIjiZiSAxN4aLhQkE/P4zNaREVouNXyUgr6CK\nPeZgZ4Y3FoYiaLgzJzHxeDyMH+OOkYHO2HcoFafO3AYAlFc04L2Pf8PsqABETfWjKTiEkH6JEnpC\nCOllxbVNuFRUhdSyOmTdq0eNvOWhxpsZCeBnb47hDuYIdrbEYJvOK79wKf3GXWzYfFFris0YqSve\neE0Kc3Pua8Kbm4mweEEoQoMk+HL7FdTVK8AwDL49nIb8wmq89fpoiEQCrsMkhJCHQgk9IYT0gooG\nBU7mVOBiYSXyWqd4PKr6ZiUSS6qRWFKN3dfvwNXCBBM9bPGkhw0GWfTd1JWunLmQhy07EtmykUZC\nAeZHB+G5Zwbr3Q+QUSGDsGldBDbF/o4bWeUAgIuXC3G3vAGrl43r0ylBhBDyuPjd6bRlyxZ4eXnB\n1NQUUqkUFy5c6LR/amoqJk6cCLFYDFdXV3z00Udaf5fJZIiOjoafnx+EQiEWLFig8zw1NTV4++23\n4eLiAhMTEwwdOhTff/99N98aIYT0LYZhcKO8Dp9eysVrx9OwP61YZzJvbiTASCdLPDfYHrMDJHgj\n1B3LR3vg7VEeWCp1x7zAQXjW2x4jnSxhq6P6S1FtE/anFeP1/6Zj7blbSCurBcMwffEWdWIYBgeO\npuHLb66wyby1lQnWvfskIicN0btkvo29nRgfxTyJ554Zwh7Lul2Bv6/9FUXFNRxGRgghD6fLO/QH\nDx7E8uXLERsbi/Hjx+Prr79GZGQkMjIy4Obm1qF/TU0NJk+ejCeffBKJiYnIzMzEggULYGZmhpUr\nVwIA5HI5HBwcEBMTg23btun8sG9ubsbkyZNhb2+P77//Hq6urigqKoJIxP0jW0IIud/NinrsuX4H\nKaUddz0V8fkIkVgizMUK/vbmGGRhDH43k1yGYSCrUyDtbi2SS2tx+U41GluU7N+vFFfjSnE1htmZ\n4c+BgxDs3LcbJzEMg537k/HfU9nsMXdXK7yzcjwc7c36NJZHIRDw8fq8YAySWGDX/mQwDIO7FfVY\ns/4M3v9/T8Db04brEAkhpEs8povbOmFhYRg5ciS2bdvGHvPx8cH06dPx8ccfd+gfGxuLmJgYlJaW\nwthYXQps/fr1iI2NRVFRUYf+U6dOhYODA3bu3Kl1/JtvvsGnn36KGzduQCh88O+O6upq9v+3srLq\n7K0QA5aYmAgAkEqlHEdC9NmjXCfFtU3Ye70Y5wsrO/wtwMEcU4Y4YPQgK5ga9cy87KYWFa4UV+F0\n3j1cLa7u8HepxAqLgl3hZtn7U0ZUKgbb9lxD/Jkc9tjIQGesenNsv9y4KTG5GJ99/TvkrWsczMQi\nvLdqAnyG2Gn3o88T0g10nZDu6Kk8ttMpNwqFAklJSYiIiNA6HhERgUuXLukck5CQgAkTJrDJfFv/\n4uJi5OfndzuwH374AeHh4Vi6dCkkEgkCAgKwdu1atLQ83GIyQgjpDS0qBgfTS7D050ytZF7A4+EZ\nTzt88awfPn3GFxM9bHssmQcAEyEfT7jb4sMnhmDr8wF41tseRu0q4CSWVGPpzxnYnlSIpnZ38nua\nUqnCl9uvaCXzY0e5ImbFuH6ZzAOAdOQgfPC3J9j46xsU+PDTs0jLLOM4MkII6VynU27Ky8uhVCrh\n5OSkddzR0REymUznGJlMBnd3d61jbeNlMhk8PDy6FVhOTg7OnDmDOXPmIC4uDrm5uVi6dCnq6uqw\nceNGnWPafg0T8iB0jZDu6Oo6KWxQ4XCRAsWN2g84A60EeM5ZCEdhOSpvl6MvrrZwPuDvyuBUaTOu\n3tMk8N/+UY+TGfmY6WoEb/OerdrCMAyO/VyMaymaHzIjh1tjYpgRUpL/6NHX4sLMqfbYdSAPDY1K\n1NcDMWtPYMFsT7i7iLX60ecJ6Q66Tkhnhg4d2iPn6dai2IfRU4ufVCoVnJycsH37dgQHB+Pll1/G\n//7v/yI2NrZHzk8IIQ9LxTD4tbQZX92SayXzrmIe3hxijHmeIjia9PjHapesRTzMdBNhuY8xvM01\nr18hZxB7W4Hjxc1oVvXMolmGYfDzaZlWMj9qpA2ipriA349q5XdmkLMpFs3xgoW5+p6XolmFvYfy\nUFLWxHFkhBCiW6d36O3t7SEQCFBaWqp1vLS0FBKJROcYZ2fnDnfv28Y7O3d/Q5FBgwZBJBJp/UAY\nNmwYGhoaUFFRATs7uw5jaJ4aeRCay0i6o7PrpKqpGf9MyMMftTUwEasX5xsL+JgzfBCm+TrqxcZP\nUgAvMgx+ya3A9j+KUN+svmOf2ABUVYoRM84bzubGnZ+kC4d+yEByeiPMzNR3q58c74k3F40aMMl8\neyNGjMCa9WdQWycHAByNu4d17z6JkjtZAOjzhHSOvndId7SfQ/84Or2VJBKJEBoailOnTmkdj4+P\nR3h4uM4xY8eOxfnz5yGXy7X6u7i4dHu6DQCMGzcO2dnZWqXYsrKyYGZmpjOZJ4SQ3pJ+tw5vncjE\nH6WaUoYB9ub4+jl/RPk56UUy34bH42Gytz2+jvSHVKJZYHWrsgHLT93AFR0LabvrxK+3cOBoGtse\nI3XF0oXSAZnMA4Cbi6XWnPrqmias/fQcqmuaOY6MEEK0dflseOXKldi9ezd27NiBzMxMLFu2DDKZ\nDIsXLwYAxMTEYNKkSWz/6OhoiMVizJ8/H+np6Th69Cg2bNjAlqxsk5ycjOTkZFRXV6OiogLJycnI\nyMhg/75kyRLcu3cPy5Ytw82bN3Hy5El8+OGHeOONN3rqvRNCSJfic8rxzuks3GvSJHEz/Zzxj6d9\nILF4vLvdvclBLMKHTwzG4hA3CFufdNYqWrD23C18l1by0HXrr6WUYPtezfz4EQFOWL4kDAJB308x\n6kvenjZ4938mwNhY/UC7vKIB/z6cjyZ57y04JoSQh9VlHfqZM2eioqIC69atQ0lJCQIDAxEXF8fW\noJfJZMjJ0VQ5sLS0RHx8PJYuXQqpVApbW1usWrUKK1as0DpvSEgIAPXdJIZhcPz4cXh6erLncnV1\nxalTp7By5UoEBwfD2dkZCxcuxJo1a3rszRNCyIOoGAZ7rxfj+0zNFEJLYyFWjfFEqKR/lMjl8XiY\n6uMIHzsz/ONiDu42KAAA+9KKUVInx1uj3GHUjYQ8J68Sn32dwP4IGOJti78vGwdRD1bv0Wd+Pvb4\n+7JwrPvsApRKFUpKm3DwP4UYO2bUgP9BQwjpH7qsQ6/vqA496Q6ay0i6o+06CRwZgs8u5+Fiu3KU\nnlameP+JwXAy09+78p2plrfg00u5SG43bSjIyQLvjPOGuejB93bKKxqweu2vqKxS73jrYGeGTz54\nBjbWvV/nXt/8ejYXX++4ivr6BgDASy8E4q/zQ/R2J1zCLfreId3RJ3XoCSHE0MiVDP73/G2tZH7U\nICtsnOTbb5N5ALAyFmLtxCGI8LZnj6WU1uJvv2bhXqPuOeEKhRKffH6RTebFpkZ493/GG2QyDwDP\nTPTC9Bf92fapM7fx08nsTkYQQkjfoISeEEJaNbQw+CZHoXUX+0UfR7w3fjDEA2B6iZDPw9uj3DEv\ncBB7LL+6EX8/ncVOx2nDMAxidyYiJ1/9w0Yg4OP/vR0Od1fDfhI6OyoAQf6af4M9B1KQkl7ayQhC\nCOl9lNATQgjUZSm35shR0KBij80LHIS/BLvqVRWbx8Xj8fBKgAT/M8YTgtapIndqm/C3X26ipFZT\nney/p7Jx9pJmd+9Fc4MRFODU4XyGhsfj4eUpLnBzMQUAqFQMNn39O2RldRxHRggxZJTQE0IMXo28\nBWt+y0ZJu82iFoe44ZUAyYCdH/20px1ixnmzFXDKGhRYffom7tQ2IS2zDLu/S2H7TprojYinvLkK\nVe8IhXxEv+QOGyt1Ul9bJ8enX1xCU1MLx5ERQgwVJfSEEINWp2jBe79lI7d1njiPB6wM88RUH0eO\nI+t9Y12t8d4TgyFqrdRS0diMv526gfWxv0PVurPsUG87LJoXPGB/2DwqSwsj/L+3x7JVbvIKqhC7\nK/Ghy4ESQkhPoISeEGKwGpqVeP/sLdyqbGCPzXQ1wjNehrN5nVRihbUTh8BYwAcYIL2gCul2xmgW\n8mBtZYK/vT3WYMpTPqxhQ+3xl1dD2Pb5hAKcOpPTyQhCCOkdlNATQgySQqnCR+dv42ZFPXtsuqsR\npLZdbs8x4IxwtMB7EwajuqoJTU0tUIgEKHC3xOuvj4KdrZjr8PTa5Ce9MWmiZjrSzn3JyMmr7GQE\nIYT0PEroCSEGR8Uw2PR7Hq6X1bLHloS6I8zO8JL5Nry7DTBNvQte65QRM2dzHJJVo6mFdkTtyqK5\nwfBwswYANLco8c+vE9DwgFKghBDSGyihJ4QYFIZh8E1SEc63qzP/6ggXvDDUgcOouHWvqhGfb70M\nizoFBhXXQ2xqBBsrE2Tdq8cnl3KhVNG88M6IRAKsenMMTEzUPwhlpXXYsoPm0xNC+g4l9IQQg3Lk\nRimOZ5ex7alDHTHDz3DLMapUDL7YdgU1rSUrPXkC/P2pIUDrGtirxdXYcq2AktMuuEgs8cZrmh1B\nL10pxOlzedwFRAgxKJTQE0IMxsXCSuxKucO2x7vZ4PVgV4Ou4BIXn43r7MZIPCxbPBozg1ww08+Z\n7XPidjkOpMu4CbAfGT/GHRFPDWbb/9r3B4pltZ2MIISQnkEJPSHEIGRV1OOz3/PYdqCjhXpzpQG0\nadTDKiiqxr8PprLtaVN8MaJ186h5IwbhaU9NtZ99acU4l3+vz2PsbxZEB8FFYgkAkMtb8PnWy2hp\nUXUxihBCHg8l9ISQAa+8QYF1F25DrlQnVi4WJnhnnDdbf90QKZqV+L/Yy2huXfTq5W6NWS8HsH/n\n8Xh4e5Q7gp0s2WP/dyUfWe2qApGOjI2FWLEkjK1PfyvnHg4eS+c4KkLIQGe432aEEIPQ1KIuT1nR\nWnXE3EiA9ycMhqWx4Va0AYDvDqchv7AKACAyEmDZ4rAO9eaNBHysDveCq4UJAHWpz3UXbqOiQdHn\n8fYn3p42mDMjkG0fOX4DGTfvchgRIWSgo4SeEDJgMQyDr67msxtHCXg8xIzzhqulCceRcSszqxz/\n+TmLbc+bNQLurlY6+1oYC/HehMEwb032Kxqbse5CDuQ0jaRTLz7ngxH+bYutGXy5/Soam6iUJSGk\nd3Qrod+yZQu8vLxgamoKqVSKCxcudNo/NTUVEydOhFgshqurKz766COtv8tkMkRHR8PPzw9CoRAL\nFizo9Hzfffcd+Hw+pk6d2p1wCSEEAHA8+y7OtJv3/dcQN4x0tuxkxMAnl7fgq+1XAair1owIcELk\npCGdjnG1NMHfx3lD0Lp4OOtePWKp8k2n+Hwe3nx9FMSmRgCA0rI67D1wneOoCCEDVZcJ/cGDB7F8\n+XKsWbMGycnJCA8PR2RkJAoLC3X2r6mpweTJkyGRSJCYmIjNmzdj48aN2LRpE9tHLpfDwcEBMTEx\nCAsL67TCRE5ODv72t79hwoQJBl2JghDycNLKarHjjyK2HeFtj+eH2HMYkX7YfzgNJaXqyitiUyO8\nuWhUtz5bg50tsSjYlW3H51bgxO3yXotzILC3E2PRvGC2ffL0baSkUbUgQkjP6zKh37RpExYsWICF\nCxfC19cXX3zxBSQSCWJjY3X2379/P5qamrBnzx74+/sjKioKq1ev1kroPTw8sHnzZsybNw+2trYP\nfO3m5mbMnj0bH3/8Mby9veluECGkW8obFPjkUi5aWj8zfGzNsDjEzeBvCmTcvIufTmaz7fnRQbC3\nE3d7/NShDlqVb7YlFeImLZLt1MRwD4wOcWHbX/0rEfW0BoEQ0sM6TegVCgWSkpIQERGhdTwiIgKX\nLl3SOSYhIQETJkyAsbGxVv/i4mLk5+c/VHDvvvsuvL29MXfuXErmCSHdolQx+PRSLipb5ytbGQvx\nznhvGAsNe8mQXN6Cr/6lmWoTPMIZzzzh9VDn4PF4WCp1h7e1+kdAs4rBPy7moIrmhj8Qj8fDX+eH\nwsJc/Z1Yca8Bu79N4TgqQshA0+k3XHl5OZRKJZyctHdRdHR0hEym+7GhTCbr0L+t/aAxupw6dQqH\nDx/Gtm3bAKg/FA397hohpGv7UouRXl4HAODzeFgd7g0HsYjjqLh38Fg6ZKXqfxexqRGWLJA+0meq\niZCPd8Z5s4tk7zYo8M+EPKjopssD2Vib4K/zQ9j2r+dy223mRQghj6/H67b1RNJ99+5dzJ8/HwcO\nHIClpXoBG8MwXd6lT0xMfOzXJgMbXSMD240aJXbnaqYzPOcsRHPhTSTqXvLzQAPtOrlT0ohvD99G\n20doxMRByMvNQF7uo5/zBRsldrb+W1/MacDGhnI842TUA9H2Hw9znYj4gJebEGk3agAA//i/X/H2\nwiEwMjLsJ0eGYKB9npCeNXTo0B45T6cJvb29PQQCAUpLte8klJaWQiKR6Bzj7Ozc4U5823hnZ2dd\nQzpIT0+HTCbDM888wx5TqdQl0oyMjJCRkdFj/wCEkIGhSsHgQKEmmfex4OMpR8OuNQ8ASiWDY3F3\n2GTe28MMoSNsHvu8fpYCPO0oxOmyFgDAydIWeJnx4W0u6GKk4ZoyWYJbuXVokqtwr1KB0xfL8OyT\n3fteJISQznT6bScSiRAaGopTp04hKiqKPR4fH48ZM2boHDN27FisXr0acrmcnUcfHx8PFxcXeHh4\ndCuo0aNHIy0tjW0zDIM1a9agqqoKX3/9NTw9PXWOk0ql3To/MTxtd0joGhmYlCoG75zJBmPMh9gY\nsDM1woZn/WBt8nB3jAfidXL0eCZq6vkwMxNDZCTA+3+LgMTZokfOHaxiUHs6i53i9N9qEb4Y6wer\nAb5p1+NcJy2ME7buUo//I60Rs6cPhrfn4//AIvpnIH6ekJ5XXV3dI+fp8lnfypUrsXv3buzYsQOZ\nmZlYtmwZZDIZFi9eDACIiYnBpEmT2P7R0dEQi8WYP38+0tPTcfToUWzYsAErV67UOm9ycjKSk5NR\nXV2NiooKJCcnIyMjAwAgFovh7+/P/l9AQACsrKxgbm4Of39/GBkZ1mNdQkjnjt4oRdpddSlGPo+H\nv431euhkfiAqltXi4LEMtv3KywE9lswDgIDPw/8L94KFSJ3Alzcq8H+X86iIQScmTfRCwDAHAIBK\nxWDLzkQolbRJFyHk8XR5G2XmzJmoqKjAunXrUFJSgsDAQMTFxcHNzQ2AeqFrTk4O29/S0hLx8fFY\nunQppFIpbG1tsWrVKqxYsULrvCEh6gVCPB4PDMPg+PHj8PT01DpXe7QolhCiy617DdiXWsy2ZwU4\nY7hjzyWt/RXDMPhmTxKaW5QAAG8PG7z4nE+Pv46DWISVYzyx9twtAMDV4mrE3SrHlKEOPf5aAwGf\nz8PiBaFY+W48mluUyMmrxH9PZePFSF+uQyOE9GPdei66ZMkSLFmyROffdu3a1eHY8OHDcfbs2U7P\n2TYnvrt0vQ4hxLA1taiwMUFTb36YnRlm+ete32NoLl4uZCup8Hg8LFkohUDQOwswRw+ywku+Tjh2\nU/16O5KLEORkAVdLk155vf7ORWKJGdP88O1h9dTSb4+kY3SoC5wdzTmOjBDSX9HyekJIv7UrpQhF\ntU0A1OUUV43xgoBPT/IaGpuxa7+m1nnkpMEY3MvztOcGDoKnlSkAQK5U4Z+/56FFRVNvHmTa88Pg\n7moFAFAoWrBtdxJNVSKEPDJK6Akh/dLV4mr8lH2Xbf81xA0SC+NORhiOA0fTUVndCACwtjLB7Kjh\nvf6axkI+/meMJ4xaf1Bl36vHd2klvf66/ZVQyMfShZq9AFLSZDh76eE2XySEkDaU0BNC+p2qpmZs\nvqJJfsa6WGOylx2HEemPtjnZbebPDoJZH22s5W0jxtxAF7Z9KFOGjLt1ffLa/dHQwXaYEqEpwbxr\nfwqqa5o4jIgQ0l9RQk8I6VcYhsGXVwtQ2dQMALA1McJboz1o0TzUVVO279VM3Rju54gJY937NIaX\nhjkisHVRsophsOlyHhqalX0aQ38yOyoADnZmAIDaOjn2HUrlOCJCSH9ECT0hpF85lVOB3+9Use1l\nYR4Dvu55d50+n4ubtyoAAAIBH6/PC+nzHzp8Hg8rwzxhZqTeYKqkTo7tfxT1aQz9iamJEf4yP4Rt\n/3pO878hIYR0FyX0hJB+405tE7YlFbLtqUMdIZVYcRiR/qiplWPvgets+0+RvnBzseQkFkczEd4I\n1TwZOJVTjouFlZzE0h+EBkkwKngQ296+J4lq0xNCHgol9ISQfkHFMNh8OR/y1kTHzdIE84Ncuhhl\nOPYdSkVdvQIA4GBnhhl/8uM0nic9bTHR3ZZtf5VYgHuNzRxGpN9e+3MwRK0bdOXkV+LUGd17shBC\niC6U0BNC+oX/Zt9Ferl6gaWAx8OqMV4wEdJHGADcyC7HL2c1CeCiucEw1oNpSEtC3eDQuiC3Rt6C\nr67mU2lG1mF4AAAgAElEQVTGB3ByMMP0F4ex7W8Pp6GqmhbIEkK6h74NCSF6T1Ynx+7rd9j2DD9n\nDLEVcxiR/lAqVfhmTxLbHhU8CKNCBnUyou9YGAuxIsyDbV8ursa5App68yAvRvrC2Um9uVR9gwL/\nPnS9ixGEEKJGCT0hRK+1VbVpalFPtXG3NMUrAc4cR6U/fjmbi7wC9SJhkUiI1/4czHFE2oKcLPH8\nEAe2vTWpEJU09UYnkZEAi9r973fmfB4ys8o5jIgQ0l9QQk8I0WsncyqQXFoDQF1BZXmYB0QC+ugC\ngLo6Bb49nMa2X35hGJwczDiMSLcFQS5aU2+2tlvYTLSFBEkwRurKtrfvpQWyhJCu0bciIURv3W1Q\nYEe7kofTfB3ha6d/CStXDh5LR22dHADgaG+Gac/7chyRbmIjAd4apal6c6GwkqredGJBdBC7QDav\noAonfr3NcUSEEH1HCT0hRC8xDIOvrhagoUW9KZGLhQn+PFw/5obrg4Kiavz86y22/ersIIhEAg4j\n6lyoxAoR3vZsO/ZaIWrkLRxGpL8c7M0w40VNlaLvjqShsooWyBJCHowSekKIXjqTfw+JJdUAAB54\nWDbaA8ZU1QaA+sfOzv3JUKnUFWMChjlgjFT/S3guHOkCWxMjAEBlUzO2/0FTbx5kaqQPBjmrd9xt\naGzG3oMpHEdECNFn9O1ICNE7lY3N+CZJM9XmhaEOCHAw5zAi/ZL4Rwmup5cCAHg8Hhb+ObjPd4R9\nFOYiId5sN/XmdN49XG636y/REBkJsGiuZoHs2Yv5tIMsIeSBKKEnhOgVhmGw5VoBahXq6RjOZsZ4\ndQRNtWmjaFZi57fJbDviKW94ultzGNHDCXOxxlMemg2nvk4sQJ2Cpt7oMjLQWWuB7M59mqcyhBDS\nXrcT+i1btsDLywumpqaQSqW4cOFCp/1TU1MxceJEiMViuLq64qOPPtL6u0wmQ3R0NPz8/CAUCrFg\nwYIO59i+fTsmTJgAW1tb2NjY4Omnn8bFixe7GzIhpB9KKKrCpSLNXdu3RrnD1Eh/54b3tZ9OZqO0\nTL3BlrmZCNHTh3Mc0cP7S4gbrI3VU28qGpuxM/lOFyMM17xZI2AkVF//2TkVOHsxn+OICCH6qFsJ\n/cGDB7F8+XKsWbMGycnJCA8PR2RkJAoLdc9/rKmpweTJkyGRSJCYmIjNmzdj48aN2LRpE9tHLpfD\nwcEBMTExCAsL0/m4+OzZs5g9ezbOnDmDy5cvw9fXF88++yxu3brVoS8hpP9raFZiW7upNs8NtsdI\nZ0sOI9Iv96oacfjHDLb9yksBsDA35jCiR2NpLMQSqRvbPplTjpTW0qREm7OjOf7UrnrRvkOpaKA6\n/oSQ+3Qrod+0aRMWLFiAhQsXwtfXF1988QUkEgliY2N19t+/fz+ampqwZ88e+Pv7IyoqCqtXr9ZK\n6D08PLB582bMmzcPtra2Os+zb98+vPHGGwgKCoKPjw9iY2NhYWGBkydPPsJbJYTou73Xi1HeqAAA\n2JgYYf4I/V/o2Zf2HUpFU5N6eoqbiyWefXowxxE9uvFuNhjnZsO2v04shLyF6q3r8tILvrCxNgUA\nVFY34sjxTI4jIoTomy4TeoVCgaSkJERERGgdj4iIwKVLl3SOSUhIwIQJE2BsbKzVv7i4GPn5j/64\nUC6Xo6mpCTY2Nl13JoT0Kzcr6vFT9l22/ZdgV1gYCzmMSL9k367Abxfy2PZrfw6GsJ9X/Vkc4gaz\n1ulUd2qbcChDxnFE+snUxAhzXwlk28dPZEFWWsdhRIQQfdPlt2V5eTmUSiWcnJy0jjs6OkIm0/3h\nK5PJ4O7urnWsbbxMJoOHh8cjBbtmzRpYWFjgxRdf1Pn3xMTERzovMRx0jegnJcPgi2w56hvVC/58\nLfgwLbuNxLs5nMSjb9cJwzDYvi8X9fUNAAC/oRZobixEYmL/L/s4wbwFR4vUU0h2J96GTU0RnE36\nxw+VvrxOxEYMbK0ZFN5pBABs+Pwk5kQ92ncp6Vv69nlC9MvQoUN75Dy98qnZG+XTNm/ejG+++QZH\njx6FuTmVryNkIDl3twXFrcm8ER94ycWoX5Rh7CvpN2uQX6RO5gV8IPIZZ44j6jlhtgJ4mqm/ilQM\ncKSoGSqGKrncj8/n4YVJEradkVWLW3l0l54QotblHXp7e3sIBAKUlpZqHS8tLYVEItE5xtnZucPd\n+7bxzs4P/0X0+eef4/3338eJEycglUof2K+zvxHD1naHhK4R/SOrkyMhNwNiM/X86QVBLnjWj5uE\nVR+vE0WzEjsOnISZmRgAMPU5Hzw7eSTHUfUsx6GNWHYyEy0MgzIAFTbuiBziwHVYD8TVdSIFUCAz\nYadeJVyTY8ZLT0Ag6B9PNAyNPn6eEP1TXV3dI+fp8lNAJBIhNDQUp06d0joeHx+P8PBwnWPGjh2L\n8+fPQy6Xa/V3cXF56Ok2mzZtwvvvv4+4uLgHvh4hpH9iGAax1wohV6qTeS9rU0zzdepilGE5+ett\nrTKVM1705ziinudpbYooP83/7rtS7uAeVXLR6c8zA2Fior4XV3inGidP3+Y4IkKIPujWz/qVK1di\n9+7d2LFjBzIzM7Fs2TLIZDIsXrwYABATE4NJkyax/aOjoyEWizF//nykp6fj6NGj2LBhA1auXKl1\n3uTkZCQnJ6O6uhoVFRVITk5GRoamJNvGjRsRExODHTt2YMiQIZDJZJDJZKipofJmhAwE5wsrkVii\nvjvBAw9vjfKAkE9TbdrU1snx/X80n4kz/uQPc3MRhxH1nlf8JRjUWoKzvlmJb5L6//qA3mBrbYrp\nU/3Y9oGj6aitk3cyghBiCLpVQmLmzJmoqKjAunXrUFJSgsDAQMTFxcHNTV1HWCaTISdHs3jN0tIS\n8fHxWLp0KaRSKWxtbbFq1SqsWLFC67whISEA1HPuGYbB8ePH4enpyZ5ry5YtaGlpwSuvvKI1bv78\n+di5c+ejv2tCCOfqFC3Y3q7m/JSh9vC1M+MwIv1z+MdM1NWry3g6OZrjuUn9t0xlV4yFfCyVuuPd\n37IBqH/sPVNcjVGDrDiOTP+88KwP4s/morSsDnX1Chw8lo5Fc0O4DosQwqFu14RbsmQJlixZovNv\nu3bt6nBs+PDhOHv2bKfnVKk6rzmcm5vb3fAIIf3M7pRi3GtST6uwMzXCvECqOd9eiawWcfGaTfTm\nvaLZMXSgGulsiac97XA6rwIAsCWxAFsi/Wmn4PuIRALMnx2EDZvVO6ef+PU2nn16CNxcaBM2QgwV\nraQhhPS5mxX1OHG7nG3/NcQNZiJK2trb930qlK1rC4b52GOM1DB+8CwKdoVl6/4DZQ0K7E8r4Tgi\n/TQ6ZBCG+zkCAFQqBnu+S+E4IkIIlyihJ4T0KaVKvRCWgbo0oVRihXBXa46j0i+ZWeVIuKqZjjR/\nVpDBlPG0MhZi4UhXtv1jVhlyqxo4jEg/8Xg8LIgeCUB9XSRdL0FSCv34IcRQUUJPCOlTJ3PKkX2v\nHgAg4vOxOMTNYJLV7mAYBru/1dxtHT/GHT5D7DiMqO8942mLQEcLAOpNx7YkFlJteh28PKwxaaIX\n2979XQr7VIcQYlgooSeE9JmqpmbsvV7Mtqf7O0FiYcxhRPrn4uVCZOeo55ALhXxETx/OcUR9j8fj\n4Y1QNwhbf+hllNfhl9wKjqPST7OjhsPUxAgAUFRcQ2UsCTFQlNATQvrMnuvFqFW0AAAk5saYPmzg\n7HjaExQKJfYdSmXbL0QMhbOjYe6M7W5lipeHademr5G3cBiRfrKxNkHUi5oylgePZaCuTsFhRIQQ\nLlBCTwjpExl363AqR3shrLGQPoLai/vlFsrK1dORLMyN8XK7euOG6JUACRzF6rr7NfIW7E65w3FE\n+umFiKFwtFeXfK2tk+P7HzO6GEEIGWjo25QQ0uuUKgZb220UNNbFmuqL36emVo7D7TaRmjnNH+Zm\nA3MTqe4yEfKxONSNbZ/MKUdmeR2HEeknkUiAebNGsO24+Fu4U0IbMBJiSCihJ4T0uv/euovblepK\nJcYCPl4Pce1ihOH5/ocMNDSq6/JLnCwQ8bQ3xxHphzAXa4xx0VRB2pJYCKWKFsjeb+woV/j5OAAA\nlEoV9h64znFEhJC+RAk9IaRXVTY2Y1+qZiHsK/7OcDKjhbDt3SmpwYl2ixnnzRr4m0g9jNeDXWEs\nUH9d5VQ14Hh2GccR6R8ej4cFc4LQVsby6h/FSEkv5TYoQkifoYSeENKrdqbcQX2zEgDgamGCl9ot\ndCRq+w5pNpHy83HA6JBBHEekX5zNjTE7QMK296WWoLyBFn7eb4iXLZ6a4MG2d+1PpjKWhBgISugJ\nIb0mtawWp/M05QYXh7pBJKCPnfbSb9zF5WuaxZ7zZ4+guvw6TPN1hJulCQCgsUWJf/1R1MUIwzRn\neiCMW3faLSiqxi9nczmOiBDSF+iblRDSK1pad4RtM97NBsHOlhxGpH9UKgZ7Dmg2kZow1h1DBxvW\nJlLdZSTg4w2pO9s+X1iJJFr42YGtjSlefmEY2/7uSBrq6WkGIQMeJfSEkF5xPKsM+dWNANTVShYF\n00LY+134vQC3cu4BAIyEAsyZEchxRPpthKMFnva0Zdux1wogb6EpJff7U6Qv7O3EANTVk478mMlx\nRISQ3kYJPSGkx5U3KLA/rYRtzw6QwEFs2CUY76dQKLHve80mUlOe1dQSJw/2WpArzIzUC4aL6+Q4\ncoMWft5PJBJg7kxNGcufTmVDVkrlPgkZyCihJ4T0uB3JRWhsUS+Edbc0xTRfWgh7v59OZaO8Ql3K\n08LcGFFTh3UxggCAjakR5o1wYdvfZ8hQXNvEYUT6afwYN/i0Tt9qaVFh70EqY0nIQNathH7Lli3w\n8vKCqakppFIpLly40Gn/1NRUTJw4EWKxGK6urvjoo4+0/i6TyRAdHQ0/Pz8IhUIsWLBA53mOHDkC\nf39/mJiYICAgAD/88EM33xYhhCvJshqcK6hk20ukbhDyaZFne9U1TTh6XDMN4pWX/GFGTzC6LXKw\nPYbaqp9mKFQqbL1WCIah2vTt8Xg8vDZnJNv+PbEIaZlU7pOQgarLhP7gwYNYvnw51qxZg+TkZISH\nhyMyMhKFhYU6+9fU1GDy5MmQSCRITEzE5s2bsXHjRmzatIntI5fL4eDggJiYGISFhems6JCQkIBZ\ns2Zh7ty5SElJwZw5czBjxgxcuXLlMd4uIaQ3NStVWgthn/SwxQhHCw4j0k+H2m0i5SKxRMRTgzmO\nqH8R8Hl4I9QNvNaa69dkNbhYVMVxVPrHZ4gdnhirKWO5+9sUKmNJyADVZUK/adMmLFiwAAsXLoSv\nry+++OILSCQSxMbG6uy/f/9+NDU1Yc+ePfD390dUVBRWr16tldB7eHhg8+bNmDdvHmxtbXWe5/PP\nP8fTTz+NmJgY+Pr64p133sGTTz6Jzz///BHfKiGkt/1wswxFrdMfxEIBFo6khbD3u1NSg5PtN5F6\nZQSEQpr9+LB87Mzw/BB7tr09qQgNrfsdEI0/zwyESKQuY5mTX4nfLuRzHBEhpDd0+i2iUCiQlJSE\niIgIreMRERG4dOmSzjEJCQmYMGECjI2NtfoXFxcjP7/7HyS///77Q70uIYRbZfUKfJeuWQg7J1AC\nW1MjDiPST3sPXIdKpZ4eEjDMAdJgSRcjyIPMGzEI1sbqa6y8Ufv6I2r2dmJMe96Xbe8/nMo+HSKE\nDBydJvTl5eVQKpVwctJe0Obo6AiZTKZzjEwm69C/rf2gMQ9znoc5ByGk72z/oxDy1sf5XtammDrU\nkeOI9E9aZhmu/lHMtl+dHUSbSD0Gc5EQC4M1C2T/c7MMeVWNHEakn6Y97wsba1MAQFV1E479dIPj\niAghPU3Y0yfk8sspMTGRs9cm/QNdI73jRo0Sv+RqNq+ZKGnBH0nXOIzo8fTGdaJSMYjdcxv19eop\nScHDrVFVkYPEipwefy1DYsEwcOYpkFOn/jG59kQS3hgs6pPvov70eTImxBSHj6t3bf72cBIcbWph\nY00LsftCf7pOSN8bOnRoj5yn0zv09vb2EAgEKC3VrvNbWloKiUT3Y2JnZ+cOd9Hbxjs7O3c7sAed\n52HOQQjpfc0qBj8Uax7hh9oI4GUm4DAi/ZSSUY1imTqZNxLyMHkilfLsCTweDy+5GKGtkFJevQrX\nKmku/f2C/K3gKlHfpW9pYXDyN6rfT8hA0ukdepFIhNDQUJw6dQpRUVHs8fj4eMyYMUPnmLFjx2L1\n6tWQy+XsPPr4+Hi4uLjAw8ND55gHnSc+Ph6rVq3Set1x48Y9cIxUKu32+YlhabtDQtdIz/surQSN\nwmKIhYC5kQBrng+AtUn/nDvfW9eJQqHEN/t/hpmZevfOqKl+eOZp2hW2J921uIPDmeqbQOfqhZgz\nMQAWxj3+EBpA//08sbD2wrvrTgMAcgqaYWbpCT8f+y5GkUfVX68T0reqq6t75DxdllZYuXIldu/e\njR07diAzMxPLli2DTCbD4sWLAQAxMTGYNGkS2z86OhpisRjz589Heno6jh49ig0bNmDlypVa501O\nTkZycjKqq6tRUVGB5ORkZGRksH9ftmwZTp8+jQ0bNuDGjRv4xz/+gd9++w3Lly/vkTdOCHl8sjo5\nDmVqnqTNHeHSb5P53vTjiZvsJlJWliZ46QXaRKqnzfJ3Zncjrpa3YG9qcRcjDI+fjz3Ghbmx7Z37\nk9kF2oSQ/q3L2xczZ85ERUUF1q1bh5KSEgQGBiIuLg5ubuoPBZlMhpwczRxQS0tLxMfHY+nSpZBK\npbC1tcWqVauwYsUKrfOGhIQAUD8uZRgGx48fh6enJ3uusWPH4sCBA1izZg3ef/99DBkyBIcOHcKo\nUaN67M0TQh7PtqRCKFoXwg6xESNyMN3tu19lVROOtluEOOvlAIip+k+PMzUS4C/Brlh/Uf0d8vOt\nckzysoOvnRnHkemXua+MwJVrxWhuUeJ27j2cS8jHk+M8uQ6LEPKYuvU8csmSJViyZInOv+3atavD\nseHDh+Ps2bOdnlOl6npzi6ioKK2pPoQQ/XH5ThWuFKsfFfLAwxtSdwhoR9gODhxLQ1NTCwDAzcUS\nkyZ6cRzRwDXW1RpSiRUSS6rBgMGWxAJsmjyMrst2HO3N8GKkD4607lT874OpGBPqChOT3pmeRAjp\nG7SbCSHkoclbVNiWVMS2nx1Md0J1KSiqxi+/5bLtV2cFQSCgj93ewuPxsCTUDaLWf+NblQ34+XY5\nx1Hpn5deGAYbK/UC2cqqRvwQR2UsCenv6JuFEPLQDmXIUFovBwBYiISYN8KlixGGafd3KWAY9Rzl\noOHOCB5BVbp6m7O5MWb4af6d916/g0raSEmL2NQI0dOHs+0f4rLYNR6EkP6JEnpCyEMpqmnCkRua\nhbDzg1xg1UvVRPqzpJQSJKeq/514PB7mzx5Bm0j1kahhThhkrq6yVt+sxM6UOxxHpH+eHO8Bbw8b\nAIBC0YJ9h1I5jogQ8jgooSeEdBvDMNh6rRDNrZUxfO3MEOFtx3FU+kepVGHPgetse9JEL3i4WXMY\nkWExFvKxJNSdbZ/Oq8D1sloOI9I/AgEf86OD2Pa5hHxk3argMCJCyOOghJ4Q0m0XCqvwR2kNAIDP\n42Gp1B18uuvcwS9nc1F4R71g2MREiFkvD+9iBOlpIRJLjHezYduxiYVooRKNWob7OWKM1JVt79yf\nzE4RI4T0L5TQE0K6paFZie1/FLLtKUMcMNhGzGFE+qmhsRkHjqaz7ZdfGAYbaxMOIzJcrwe7wlSo\n3rW4oKYRP9yk3VHvN++VERAK1alA1u0KXPi9sIsRhBB9RAk9IaRbvk0rQUXr4kIbEyPMDRzEcUT6\n6dhPN1Bd0wQAsLMVY+qzPhxHZLjsxSLMGS5h29+mleBug4LDiPSPs5M5Xmh3jf770HXI5S0cRkQI\neRSU0BNCupRX1Ygfs8rY9sKRLjATCTiMSD/dLa/H8RNZbHvOjOEwpgXDnJrq4wjP1hKNcqUK3yTR\nHej7TX/RD5YW6kXE5RUN+LHdNUwI6R8ooSeEdIph1Bv0KFvn1gY6WuBJD1uOo9JP+79Pg6JZCQAY\n7GWLJ8Z6cBwREfLVm561uVRUhcSSag4j0j9iUyPMjtKs8zhy/AbuVTZyGBEh5GFRQk8I6dSvefeQ\nXl4HABC2btxD5Rc7yr5dgXMJ+Wx7/uwg8GmHUr0Q4GCOSV6aakxbrxVC3tL1buWGpH0lJoWiBfu+\npzKWhPQnlNATQh6oVt6CncmaHWGnDXOCR+v0BaLBMAx2fZfCtsNCXRAwzIHDiMj9FgS5wEKknv5U\nUifH4XZ7KZCOZSx/u5CHW7n3OIyIEPIwKKEnhDzQ3tRiVLcukHMQizDLn3Y61eX3xDu4kVUOQJ0Y\nzX1lBMcRkftZmxjh1RGahdyHM0pxp7aJw4j0T1CAE0YFa/6NqIwlIf0HJfSEEJ2yKurx861ytv16\nsCtMjWgh7P2aW5TYe1CziVTkpCEY5GzBYUTkQSK87eFrZwYAUKhU2HqtkBLW+8ybNQICgTo1uJFV\njoSrRV2MIIToA0roCSEdKFUMtlwrBAN1shMqsUK4K+10qsuJX26jtEy9xsDcTIQZf/LjOCLyIAK+\neg1I22ZoSbIanC+o5Dgq/eIiscTzk4ew7b0HrkOhUHIYESGkOyihJ4R0cDKnHNn36gEAIj4fi0Nc\naSGsDrV1chz6IYNtz/iTPyzMjTmMiHRlqK0ZpgzRrG/45o8i1FLddS0zXtRcx2Xl9fjpJJWxJETf\nUUJPCNFS1dSMPSl32PZ0fycMsqCdTnX57kg66ls3KnJ2MsdzkwZzHBHpjnkjBsHO1AgAUNnUjD3X\nizmOSL+Ym4vwykv+bPvI8RuorKL1BoTos24l9Fu2bIGXlxdMTU0hlUpx4cKFTvunpqZi4sSJEIvF\ncHV1xUcffdShz9mzZxEaGgpTU1MMHjwY27Zt69Dns88+g6+vL8RiMdzc3PDmm2+ivr6+m2+NEPIo\ndqXcQV1rLXWJuTGmD6OFsLrkFVTh5OnbbHvB7JEwEtIag/5AbCTAX0Pc2PbPt+8i424dhxHpn2ef\nHgw3F0sAQGNTM747ksZxRISQznSZ0B88eBDLly/HmjVrkJycjPDwcERGRqKwUPduezU1NZg8eTIk\nEgkSExOxefNmbNy4EZs2bWL75Obm4vnnn8f48eORnJyMmJgYvPXWWzh69CjbZ+/evXj33Xfx/vvv\n48aNG9i7dy/i4uKwbNmyHnjbhBBd0spq8UtuBdteHOoGYyE9yLsfwzBaFUBGBDhBGizhOCryMMJd\nrRE2yIptf51YgBYVLZBtIxDwMX/2SLb9y9lc5OTRegNC9FWX39SbNm3CggULsHDhQvj6+uKLL76A\nRCJBbGyszv779+9HU1MT9uzZA39/f0RFRWH16tVaCf3WrVvh6uqKzZs3w9fXF4sWLcKrr76Kf/7z\nn2yfK1euYMyYMZgzZw7c3d3x1FNPYe7cubh8+XIPvG1CyP0UShW+vFrAtsNdrSGVWHUywnD9nngH\naZllAAA+n4fX5oykNQb9DI/Hw+JQd5i0/mDNq27EsRulHEelX4JHOCNkRNsPVQa7vqUyloToq04T\neoVCgaSkJERERGgdj4iIwKVLl3SOSUhIwIQJE2BsbKzVv7i4GPn5+WwfXedMTEyEUql+1B8ZGYmU\nlBQ2gS8oKMCPP/6IKVOmPORbJIR0x5HMUhS11uU2FQrwl3ZTEoiGQqHEnnabSEU+MwTurvTDpz9y\nNBPhz8M1dde/TS+BrE7OYUT659V2Ox6n37iLi5d1P50nhHCr04S+vLwcSqUSTk5OWscdHR0hk+ne\nZU8mk3Xo39ZuG1NaWqqzT0tLC8rL1XWvp0yZgvXr12PChAkQiUTw9PREUFAQPvnkk4d4e4SQ7iiq\nacKhDM1/0/NGDIKDWMRhRPrrxxM3UVauXstjYW6MV14K4Dgi8jhe9HHEYBsxAPVTqi2JBXQXuh03\nF0tETtKUsdz1bQoaGps5jIgQoouwp0/YU4+djx07hnfeeQdbt25FWFgYsrOzsWzZMnzwwQdYu3at\nzjGJiYk98tpk4KJrpCOGYfBNjgJVdSoAgKuYD6fqfCQmFnQxcuB60HVSXdOM3d9moblZnfBNGm+N\nGzeu6+xL+o8nTVVILVLfmT9/uwE7Wiow0rrrr0dD+Tzx9VQiDgrU1begvr4Bm748ieeepsXy3WUo\n1wl5NEOHDu2R83R6h97e3h4CgQClpdrzCktLSyGR6F4A5uzs3OHufdt4Z2fnTvsIhULY29sDAD75\n5BMsXLgQr732GgICAjBt2jR8/PHH+PTTT6FSqR7iLRJCOnOtUolbrck8nwdMdzWCgOaD63TyNxmb\nzDs7mkA60objiEhPcBfzMc5eU6Hox+IWNLTQXfo2JiYCrQT+0tVylN6lMpaE6JNOb0GIRCKEhobi\n1KlTiIqKYo/Hx8djxowZOseMHTsWq1evhlwuZ+fRx8fHw8XFBR4eHmyfY8eOaY2Lj4/HqFGjIBCo\nP1QZhgGfr/17g8/nd/ooVCqVdvZ2iAFru0NC14i2ankLPotLh9hMXZP7JV8n/CnYleOouNPZdZKZ\nVY5beTkwM1NPz1j11pMI9Hfs0/hI7/EPUmJxXDoqGpuhBJDMt8fbUg+dfQ3x8yQ0lEFe0W/IzLoL\nALiS0oIPV4fSYvBOGOJ1Qh5edXV1j5ynyyo3K1euxO7du7Fjxw5kZmZi2bJlkMlkWLx4MQAgJiYG\nkyZNYvtHR0dDLBZj/vz5SE9Px9GjR7FhwwasXLmS7bN48WLcuXMHK1asQGZmJv71r39hz549WLVq\nFdtn2rRp+Oabb3Dw4EHk5uYiPj4e7733HqZOndoh0SeEPJpdyUWoad0l00EswpzhVHpRF5VKXaay\nzdhRrpTMDzD316Y/mVOOlNJaDiPSLzweD6/PC2YXyKZmlNECWUL0SJeTBGfOnImKigqsW7cOJSUl\nCBg8PkQAACAASURBVAwMRFxcHNzc1B98MpkMOTk5bH9LS0vEx8dj6dKlkEqlsLW1xapVq7BixQq2\nj6enJ+Li4rBixQrExsbCxcUFX375JV566SW2z+rVq8EwDN577z0UFRXBwcEBU6dOxfr163vy/RNi\nsK6X1SK+Xc35JaFuMDWijZF0OXM+D7dz7wEAjIQCzJsVxHFEpDeMc7NBuKs1LhVVAQC+vJqPr57z\nZ0tbGjpPd2s8P3kofjqZBUC9QDYkSAJx6667hBDu8Jh+vpy//aMKKysqHUd0o0ef2hRKFd46kcmW\nqRznZoN3xnlzHBX3dF0ndfUKvLX6BKpr1P9W01/0R/T04ZzER3rfvcZmLI5LR33rbskvD3PCwpHa\n09AM+fOkvkH930NVtfq/h2nP+9IP3Acw5OuEdF9P5bF024EQA/R9poxN5sVCAf5qwPPmu/LdkTQ2\nmbezFePlF4ZxHBHpTbamRljULoH/4WYZsirqOYxIv5iJRZg3awTbPn4yGwVFPTMHmBDy6CihJ8TA\n5FU14lC6ds15O6o5r9PtvEqc+PU2235tzkiYmPR4tV+iZyZ72yHIyQIAoGIYfHE1Hy2qfv0wu0dN\nDPeAn48DAECpVGHHvj+odj8hHKOEnhADolS1JietX77D7Mzw/BAHjqPSTyoVg292J7GJStBwZ4yR\nunAcFekLPB4Pb4/ygLFA/RWZW9WII5m6N1M0RDweD395lRbIEqJPKKEnxID8mFWGm63TB4z4PLw9\n2gMCPpWd0+X0+Vxk56gXDQuFfLw+L5hK9BkQZ3NjzA0cxLa/Sy9BQXUjhxHpFw839QLZNrv2p6Cu\nXsFhRIQYNkroCTEQJbVy/Du1mG3PCpDAw8qUw4j0V22dHP8+mMq2pz0/DIOcLTiMiHDhRR9H+Nia\nAQCaVQy+uloAFU0tYc16OQA21urPkMrqRuw7lNrFCEJIb6GEnhADwLTOA5Yr1TvCelmbYrofbd3+\nIPu+T0VtnRwA4GBnhqiptBDWEAn4PCwb7QFh65OZ9PI6/Df7LsdR6Q+xqREWzQ1m26fO3EZmVjmH\nERFiuCihJ8QAnLhdjutl6k1y+LzWJIWm2uhUVNyA+DO5bHvR3GAYG9NCWEPlaW2K6f6aH797rhfj\nnkLFYUT6ZYzUBdJgzdSkrbsS0dyi5DAiQgwTJfSEDHB3GxTYlXKHbb88zAlDW6cREG0qFYMfTxYD\nUE+rCA2SQBpMu+caulf8neFmaQIAaGxR4lBhM029acXj8fB6ux+9hXdq8J+4mxxHRYjhoYSekAGM\nYRhsSSxgN8lxsTBBdAAlqA+SmFyJOzJ1zXmRkQAL59JCWAKIBHysCPMEv/VauF2nQkIF3YVu42Bv\nhugozWZr3/+QiWJZLYcREWJ4KKEnZAA7V1CJK8WaTV/eGuUOY9rGXqfqmiacOlvKtl96YRicHc05\njIjoE187M0QNc2Lb/y1pRnHr5mwEeH7yEHh72gAAmluU2Lb7GtWmJ6QP0Tc7IQNUVVMztiZpakM/\nP8QBgY5UqeVBdu1PQWOT+q6rk6M5XppCC2GJtujhEni2VoZqVgGfX86HkjacAgAIBHwseU3KPtFK\nzSjDbxfzOY6KEMNBCT0hAxDDMNhyrRA18hYAgINYhAVBtCnSg/xxXYZzCZrk4/W5wRCJBBxGRPSR\nZuqNup1eXof/ZJVxG5QeGexpgxee1dSm3/1tCmpq5RxGRIjhoISekAHot/x7uFhYybbfHOUOsREl\nqLo0NbVg2+5rbDvI3wohQbTOgOg2xFaMZxw1VY/+fb0YhTU09abNrJcDYG8nBqDez2H3dykcR0SI\nYaCE/v+3d9/hUVX5/8Dfd2oyk14nvRECoYYkQAIRlKKw6iI21FXBigUpiyJLWZWmoBRdQBRXWH+4\nsgqW7y4KUUoICSVAgBQSIJ1khkzapE2/vz8muWFIBQIzmfm8nmeeyT1z7uRMOJz7mXNPIcTGKJu0\n+Px021CbByK8EOfnasESWbd/783CNaVp91yJIx9TJ1IwT7o2wVeAAEdTN73WaMSGE0U09KaFo4MQ\nLz83gjs+nFqEc1lyC5aIEPtAAT0hNsS0gVQJGlpWtZFJxXhxeKCFS2W9LhdU47/7L3HHUybI4CSl\nNedJ1/gMgyeDRNyGU3lVjdhzUdHNWfYjPsYfCfFB3PHmrzLQ1KyzYIkIsX0U0BNiQ367osTpCtOq\nNgwYzBsVQkNtOqHXG7H1nxncShxDB/kiZrCbhUtF+go/Rx6eHtx2N2fXhXIU1TZbsETW5aVnY+Ds\nJAYAKKua8K/vzlu4RITYth4F9Fu2bEFYWBgcHR0RFxeH1NTULvNfuHAB48aNg0QiQWBgIFasWNEu\nz5EjRxAbGwtHR0dERERg27Zt7fKoVCq89dZbCAgIgIODAyIjI/H999/38KMRYl8q6jXYnlnGHU+L\n8qFVbbrwy2/5KCypBQCIRALMnhVLa86Tm/LYQBmiPE2btOlZFh8fL4TWQLvIAoC7mwNefi6GOz5w\n6ArOZdNdDELulG4D+t27d2PevHlYunQpMjMzkZiYiClTpqC0tLTD/CqVCpMmTYKfnx8yMjKwadMm\nrFu3DuvXr+fyFBYWYurUqRg7diwyMzOxePFizJkzB3v37uXy6HQ6TJo0CVeuXMH333+P/Px87Ny5\nE2FhYb3wsQmxLQYjiw0niqDWm4KJIBcHPDvEv5uz7FeFvB7/+TGbO37ykWhac57cND6PwbyRIRDx\nTZfSwtpm/Ot8uYVLZT3GjArCqNi21bW2bKehN4TcKd0G9OvXr8esWbPw4osvIioqCp9++in8/Pyw\ndevWDvPv2rULarUaO3fuRHR0NB599FEsWrTILKD//PPPERgYiE2bNiEqKgovvfQSnn/+eXz88cdc\nnq+//hpVVVX4+eefkZiYiODgYCQmJiIuLq4XPjYhtuXn/GvIVjYAMI3vXTAqlDaQ6gTLsti24wy0\nLfMMwkPc8fAD/S1cKtJXBbs64oXrloT9MU+BMxUqC5bIejAMg1eej4WTVAQAqKxqpKE3hNwhXV7x\ntVotzpw5g8mTJ5ulT548GWlpaR2ek56ejqSkJIjFYrP85eXlKC4u5vJ09J4ZGRkwGEwX2Z9++gmJ\niYl444034Ofnh0GDBuH999+HXq+/+U9JiA0rqm3GN9f1Cj4RLUP/lmEApL1DR4twPsd0659hGMx+\nIRZ8Pn35IbfuwUhvs5WkNpwoQq2aeqKB1qE3bave0NAbQu6MLpdzUCqVMBgM8PX1NUv38fGBXN7x\nMlRyuRzBwcFmaa3ny+VyhISEQKFQtHtPX19f6PV6KJVK+Pr6oqCgAIcOHcIzzzyDffv2obCwEG+8\n8QYaGhqwbt26Dn93RkZG15+W2D1bqyN6I4t/XNagttk0sTPAkUGk+ioyMui2f0dU9Tp8uv0ytyPs\n2JGeqK0qQEZVgVk+W6sn5M64vp6MF7E4o1GjQQ80NQJL/u8kZoaKaF4GAAcBi5AAPnLy6wEAqz/5\nHXNe7AcHsX1M2Kf2hHQlMjKy+0w90OvdUr3VeBmNRvj6+uLLL79ETEwMpk+fjg8++KDToT6E2KP9\nCj2utgTzAh4wI1gEPgUQHWJZFj/+epUL5j3chJiQ5NvNWYT0jLPQtJRlqxyVEcerDRYskfVgGAYP\n3+8PiaMpgK+t0+HAYeqlJ6Q3ddlD7+XlBT6fD4XC/D+eQqGAn1/Hm6/IZLJ2vfet58tksi7zCAQC\neHl5AQD8/f0hEpn3bgwYMABNTU2oqqqCp6dnu99N4+tJZ1p7SGypjpypUOHk5UuQtIxPfTkmEA9G\nUYDamd+PFOCqvBBSqWkXyyULx2PwQB+zPLZYT0jv66yexAFoPFOKX/KvAQAOqXj48+gBCHZ1vNtF\ntEp8kT82fn4CAJCVp8Zj04IwdJDttlnUnpCeqKur65X36bKHXiQSITY2FgcOHDBLT05ORmJiYofn\nJCQk4OjRo9BoNGb5AwICEBISwuVJTk5u957x8fHg803f4MeMGYNLly5xa0QDQH5+PqRSaYfBPCH2\npFatw/oTRdxxnJ8rHu7v0/kJdk5R2Yh/7srkjh+8v3+7YJ6Q3jBzaABCWwJ4jcGIdelFtJRli6SE\nYMTHtK2+9Y/tp9DQqLVgiQixHd0OuVmwYAF27NiBr776Crm5uZg7dy7kcjlmz54NAFi8eDEmTpzI\n5X/66achkUgwc+ZMZGdnY+/evfjoo4+wYMECLs/s2bNx9epVzJ8/H7m5udi+fTt27tyJhQsXcnle\ne+01VFdXY+7cucjLy8P+/fvx3nvv4fXXX+/Nz09In2NkWaw/UYyalkl37g5CzB8VAh4NtemQ0chi\n8/ZTUKtNE+oD/Fzwl8eHWLhUxFaJBTy8nRAGEc90eS2obaKlLFswDINXZ7WteqOsasIXO8+YddwR\nQm5NtwH9E088gY0bN2LlypWIiYlBWloa9u3bh6Ag07bOcrkcBQVtE8pcXFyQnJyM8vJyxMXFYc6c\nOVi4cCHmz5/P5QkNDcW+ffuQkpKCmJgYrFmzBp999hkeeeQRLk9gYCAOHDiA06dPIyYmBq+99hpe\nfPFFrFq1qjc/PyF9zi/517jdYAFgwahQuDkILVgi6/br75eRlWsaAsEwDOa8HA+RyD4m4xHLCHVz\nxKzh5ktZZlT0zm31vs7DzRGvvdA2BCX1eAmOpBVbsESE2AaG7eNfja8fe+Tq6tpFTmLPbGUs4+Xq\nJvw1+SL0Lf9tHx3gixeGB1q4VNbraoUKf132O7RaU+/8ow8NxDNd9M7bSj0hd1ZP6gnLsngv5QoX\nyLuIBfj0/oHwlog6PceebN5+Cn+kFAIAHB2E+GTlJJvb3I3aE9ITvRXH0uLLhPQRzToD1qUXcsF8\npIeUdoPtgsFgxGdfnOKC+ZAgNzw+LdrCpSL2gmEYzB8VAk9H090zlUaPj9IKoTf26T60XvPCX4ZD\n5msK4JvVOmz6/AQMNNeAkFtGAT0hfcQXZ8tQVq8GADgK+Hg7IRRC2hCpUz//mo/8K1UAAD6fZxpq\nI6ShNuTucXMQYlFiOLeUbK6yATvOXbVwqayDo4MQ82eP4jZ1y7tche9/zrVwqQjpuygaIKQPSCmu\nxoECJXf8WmwQApwdLFgi61ZcWovv9mZxx09Mi0Z4qLsFS0Ts1SBvJzw3tO1O2o95ChwrrbFgiaxH\nZIQnZjwyiDv+/ucc5OYruziDENIZCugJsXLFdc3YdKpt0ti9IR64L9TDgiWybhqNHuu3nIBeb7p9\n3y/cA9MfHGDhUhF7Nn2AL0b5t42N3XSyGBX1mi7OsB/T/hSF6ChvAKZ5B5s+P4HGJlrKkpCbRQE9\nIVasSWfA6tQCqFuCU38nMV6PC6bt5Lvwz28zUXrVNMlIJBJgzssjudv6hFgCj2Ewf1QofKViAECj\nzoAP0wqg0dOYcT6fh7mvjoS0ZbLwNWUjtv/rrIVLRUjfQ1c5QqwUy7LYeLKYGzcv5vOwZGwEJDQO\nvFPHTpQi+VDbMrovPDMcQQEuFiwRISbOYgHeTQyDkGf6Mn65pglfni21cKmsg7eXFLNnxXLHR9KK\ncSi1yHIFIqQPooCeECv1U941s7G2c+KDEepGW8h3RlHZiK3/zOCOE0cGYdL4MAuWiBBz/T2leCkm\niDv+9YoSB4uqLFgi6zFmVBDGjw3ljrftOIPi0lrLFYiQPoYCekKsUNa1enx93WoYf+rnjXtDPS1Y\nIuum1xuxYctxNDWbds/18ZLitRdiaWgSsTp/6ueFccFtc2D+caoEV2qaLFgi6/HSszEI8DPdUdNq\n9Vj7aTr3f5oQ0jUK6AmxMtXNOnyUVghDy3rzUZ5SvBRDm0d15d97s8yWqFzw+mhuTC4h1oRhGLwZ\nH4zAllWqNAYjVhy9ghoKXCFxFOLtOQkQiwUAgApFPTZvP4U+vv8lIXcFBfSEWBG9kcWHaQWoVpsu\n7i5iARaPCYeIJnV26lyWHD/+9yJ3/PRjg9G/H93NINZLIuRjWVIEpC3zYSqbtFh9rABa2lgJwYGu\neP2Ftp1V00+V4f9+y7dgiQjpGyhKIMSK7Dh3FdmVDQBMK2MsSgijreK7UFOrxqZtJ7njYYNl+POU\nKAuWiJCeCXRxwDuJYeC1DAvLUTZg6+lS6o0GkJQQjCkT+3HH/9p9Hjl5lRYsESHWjwJ6QqzEwaIq\n/Jin4I6fHeKP4TJaoaUzRiOLT784ido60ypAbq4OeOuVkeDxaNw86Rvi/Fwxa1gAd3ygQIn/u0SB\nKwA8/9QwRIab7rQZjSw++cdx1NSqLVwqQqwXBfSEWIGcygZ8erJt86jRAW54bKCvBUtk/b7/OQfn\nsuTc8dxXR8LdjXbPJX3LI1E+uO+6Ce9fni3DWbnKgiWyDiIhHwvfHA1nJ9Pa/TV1zdiw9TgMNCyJ\nkA5RQE+IhckbNFiZegU6o+lWe4irIxaMCuVuxZP2zpyrwO4fc7jjRx4cgGGDZRYsESG3hmEYvBkX\njChPKQDAyLL4KK0QV+upN9rbS4r5r40CYGoLs3Kv4ds9WZYtFCFWigJ6QiyoSWfAB0evoE6jB2Ca\nBLs8KQJSEW0e1Rn5tQZs2HoCgOkL0JBoHzz96GDLFoqQ2yAW8LBkTDg8HYUAgHqtHh+kXEGj1mDh\nklne8CEyPDEtmjv+8b8XcewEbchFyI16FNBv2bIFYWFhcHR0RFxcHFJTU7vMf+HCBYwbNw4SiQSB\ngYFYsWJFuzxHjhxBbGwsHB0dERERgW3btnX6fv/+97/B4/Hw0EMP9aS4hPQJBqOpJ664rhkAIOQx\nWDY2ArKWW8ykPa3WgHWfpqGxSQsA8PSQYP5ro8GnVYBIH+cpEWHJ2AiIeKa6XFavxppjBdDREBM8\nMS0aMUPb7sB99uUpFBTVdHEGIfan26vg7t27MW/ePCxduhSZmZlITEzElClTUFra8TdklUqFSZMm\nwc/PDxkZGdi0aRPWrVuH9evXc3kKCwsxdepUjB07FpmZmVi8eDHmzJmDvXv3tnu/goICvPPOO0hK\nSqJNYohN+SqzDBkVddzxWyNDEO3tZMESWTeWZbHlqwwUlph2j+TzeXj7zQS4udK4eWIbojylmDsy\nhDs+q1Dh01PFMNr5yjc8HoP5s0fDX+YMwLTp1IebjtEkWUKu021Av379esyaNQsvvvgioqKi8Omn\nn8LPzw9bt27tMP+uXbugVquxc+dOREdH49FHH8WiRYvMAvrPP/8cgYGB2LRpE6KiovDSSy/h+eef\nx8cff2z2XjqdDk899RRWr16N8PBwWs6L2IxfL1fi5/xr3PGT0TKziXGkvZ/25SElvW3i8EvPxtB6\n88TmjA/1wDOD/bnjg0XV2Hm+3IIlsg5OTiIsnj8GkpZhScqqJqz7LA1aHQ1LIgToJqDXarU4c+YM\nJk+ebJY+efJkpKWldXhOeno6kpKSIBaLzfKXl5ejuLiYy9PRe2ZkZMBgaPvPuWTJEoSHh+PZZ5+l\nYJ7YjEy5CltPt93hGhPkjr8M8e/iDHL6XAW+2X2BO540PhyT7w23YIkIuXOeGiTDAxFe3PEPuXL8\ncl0HgL0K8HPBgtdHc3frL15SYsv2DIoPCEE3Ab1SqYTBYICvr/nyeT4+PpDL5R2eI5fL2+VvPW49\nR6FQdJhHr9dDqVQCAA4cOIAffviBG1vPMAwNuSF9XkFNE1YfK4Ch5QLUz11CK9p0o/SqChu2HEfr\nJNgB/b3w0nMx1B4Qm8UwDF6PDcYof1cu7YszZThWSuPGRwzzw3NPDuWOU9KLsfvHbAuWiBDrIOjt\nN+yNi2xlZSVmzpyJ7777Di4upo11WJbt9lt4RkbGbf9uYtssWUeqtUb847IW9TpTPXYRMnjI2Yis\nzDMWK5O1a2zS4/N/FaC6xjQJ1s1FiMlJEpzLPHtHfy+1JaQn7nQ9mejA4gqrRUmTaWLse79n4eUw\nEcKd7HsVLH9vFtGRYpzKNH3B+XpXBurrKhAzxN3CJesYtSekK5GRkb3yPl320Ht5eYHP50OhUJil\nKxQK+Pn5dXiOTCZr13vfer5MJusyj0AggJeXF7KzsyGXyzFhwgQIhUIIhUJ888032LdvH4RCIS5d\nunRzn5IQC2vUs9he0BbMO/CBl8JEcBNRL3NndDojdu0p4YJ5oZDBM48Gw0na6/0QhFglEY/BC2Ei\neItN7YTeCOwo0kKutu+VbxiGwcP3+yMyvG0RgR/3XUVBcYMFS0WIZXV5ZRSJRIiNjcWBAwfw6KOP\ncunJycl4/PHHOzwnISEBixYtgkaj4cbRJycnIyAgACEhIVyeH3/80ey85ORkxMfHg8/nY+TIkcjK\nats8gmVZLF26FLW1tdi8eTNCQ0M7/N1xcXHdf2Jil1p7SCxRR5p1Biw5fAmNAh4kAtPylB+Mj8RQ\nH+e7Xpa+wmhksfHzE1DWAFKpBACDd95KwOi4wDv6ey1ZT0jfcbfrSdRgDRb+nocatQ4A8EO1CGsn\nRMFHKrorv99aDR4yHH9bcRAlZabVwn45UIMVfxuOsBA3C5fMhNoT0hN1dXXdZ+qBble5WbBgAXbs\n2IGvvvoKubm5mDt3LuRyOWbPng0AWLx4MSZOnMjlf/rppyGRSDBz5kxkZ2dj7969+Oijj7BgwQIu\nz+zZs3H16lXMnz8fubm52L59O3bu3ImFCxcCACQSCaKjo7nHoEGD4OrqCicnJ0RHR0MoFPbKhyfk\nTtPojViZWoC8qkYAAAMGCxPCKJjvxnd7s5F6vIQ7nvnU0DsezBNirWROYrw/rh8cBaahNpVNWiw5\nlI/qZp2FS2ZZEkchlv41Ce5ujgCApmYdVn58FPJr1FNP7E+3Af0TTzyBjRs3YuXKlYiJiUFaWhr2\n7duHoKAgAKaJrgUFBVx+FxcXJCcno7y8HHFxcZgzZw4WLlyI+fPnc3lCQ0Oxb98+pKSkICYmBmvW\nrMFnn32GRx55pNNy0KRY0tfojSzWphciU6Hi0l4dEYixQdY5ztNa7D94BT/8ksMd339fBB56oL8F\nS0SI5UW4S7B0bDiEPNN1sLxBgyWHLnG7TNsrL08Jlr+dxC1nWVPXjBXrjqJORWvUE/vCsH18vafr\nb1W4urp2kZPYs7t969PIsthwoggHi6q5tOeG+OPJQR3PPSEmxzPKsO6zdG4CfMxQGf42f+xd2wmW\nbpGTnrBkPTlxtRarUs1Xylp1byScRPY9tyT7YiU+WJsCnd609HV4qDvef3ccpBLLDUui9oT0RG/F\nsbRfOiG9zMiy+MepErNg/rGBMjwRLeviLJJ9sRIbtpzggvl+4R5Y+GbCXQvmCekLRgW4YeHoUDAw\n9dRfrmnCssOX0aC17576QQO8Mf/1Udyd/IKiGqxanwq12r7/LsR+0JWSkF5kZFlsySjF/gIllzYl\nwhszh/rTkLEuFBbXYs2GVK53zc/XGUsWjIWjA82XIeRG94R44K2RwdxxfnUjlh6+jHo7H34zOi4Q\ns2fFcscX85X4cNMxaLW0myyxfRTQE9JLWJbF1tOl+PVKJZc2IdQTr8UGUTDfhdKrKry/9giaWib4\nubs6Yvk798DVxcHCJSPEek0O98KbcW1B/aXqRiw7QkH9pPHhmPX0cO74fLYCn2xOh15v30t9EttH\nAT0hvcDYEszvu9wWzN8X6om5I0PA51Ew3xm5ogHvfXQEqnoNAEAqEWHpwiT4ekstXDJCrN+Uft7t\ngvqlhy+hVm3fq9889EB/PPXoYO741NlyfLI5nbsDSIgtooCekNtkMLLYdLIY/7sumB8f4oF5FMx3\nSVnVhL9/eAQ1tc0AAAcHAZYuTLKaNaQJ6QtuDOov1zTh3YP5UDZpLVgqy3vs4YGY9qcB3PGJ01fx\nyT+OU1BPbBYF9ITcBr2RxcfHC/F7YRWXNi7YAwtGhVIw3wVlVROWrTmMypb1+UVCPv42fyyi+nla\nuGSE9D1T+nlj7sgQ8FqG9pWq1Fj0Rz7kDRoLl8xyGIbBs08MwZ+nRHFpJ89cxcefUU89sU0U0BNy\nizR6I9YcK0BKSQ2XNjncC38dTcF8VyqVjVi25jAULZu/8Pk8vPNWIgYP9LFwyQjpuyaHe+HthFAI\nWoJ6eaMG7/yRh6KWO2D2iGEYPDdjKKZNbQvqT50tx0cb06Cx87kGxPZQQE/ILWjQ6rH8yGUcv1rL\npT0Y6Y058cEUzHfhmrIRy28M5uckYsQwWp+fkNt1T7AHliRFQMQzXdqrmnV45488nL9Wb+GSWQ7D\nMHj2yaFmw2/OnK/Aio+PchPxCbEFFNATcpOqm3V492A+sirbLpKPDvDF7BFB3C1v0l65vB7LVh2G\notI0zEYgMPXMx4/wt3DJCLEdI/1d8f74fpAK+QCARp0Byw9fQmppTTdn2q7W4TePPRzNpeXkVeLv\nH7ZNyCekr6OAnpCbUKZS4+3f81B43W3sF4YF4oXhgbQ0ZRcKi2uxdNUhbsy8QMDDorljEB9DwTwh\nvW2ojzPW3NcfHi37OOiMLD48VogfLyrQxzeHv2UMw+DpxwbjuRnDuLQrhdVYsvIQ18lASF9GAT0h\nPXT+Wj0W/p4HeaOpR4fPMFgwKhSPDvS1cMmsW26+EsvXHEZtnRoAIBIJsHjeWMTSMBtC7pgIdwnW\nTYxCoLNpPwcWLLZnlmFzRin0RvsM6gFg2tQovDozFmjZafdqhQp/++AgCors9w4GsQ0U0BPSAweL\nqrDs0CXUt2yvLubzsDQpAhPCaFWWrmRkluP9tSlobFlCT+IoxPK3kxAzVGbhkhFi+2ROYqydGIWB\nXk5c2q9XKvHekcto0NrvpND774vAX98YDYHAFALV1DVj6epDOJclt3DJCLl1FNAT0gUjy+Kb8+X4\n5HgR9C23qj0chPhoQn+M9He1cOms2/6DV7BmwzFoWwIHF2cxPlg8HtFR3hYuGSH2w1UswKrxkRgX\n7MGlnVWo8NfkPJSq1BYsmWWNGRWE5W/fA4mjaViSWq3Hyk9Ssf/gFQuXjJBbQwE9IZ1o1Bqw+MDn\ndwAAIABJREFU4ugVfJdTwaWFujrik0kDEOlBO5l2xmhk8f/+cwHbdpzmxuv6eEmxaum9CA91t3Dp\nCLE/YgEPbyeE4unBbcPcyurVWHDgIk5ct1KXvRk80Aerl90HTw8JAMBgMGLbjtP46v+dhcFgtHDp\nCLk5FNAT0oEylRoLki/iZHkdlxbr54p1E6PgIxVZsGTWTaPRY/2W49j731wuLTzUHWuWT0CAn4sF\nS0aIfWMYBs8M9seihDCI+aZLf5PegA+OXsGurHIY7HRcfXCgKz5cfh/CQ9o6G/534BJWrU9FQ6N9\n77ZL+hYK6Am5wbHSGixIvoiy+rbb0Y8NlOHvSRGQtCwFR9pTVjVhycpDSDtZyqXFDvPDir+Nh7ub\ngwVLRghpdU+IB9ZNjIKvVMylfZtVgb8fuYxatX2uy+7pIcHKJfdidFwgl5Z5QY533vsdxaX2eweD\n9C09Dui3bNmCsLAwODo6Ii4uDqmpqV3mv3DhAsaNGweJRILAwECsWLGiXZ4jR44gNjYWjo6OiIiI\nwLZt28xe//LLL5GUlAQPDw+4u7vjvvvuw7Fjx3paZEJuit7IYvvZMqw+VoBGnWlrcBGfh7dHh2HW\nsADaMKoLFy8psei9P1BQ3LZSxAMT+uHdeWPg2LJ0HiHEOkS4S7Bh8gAM83Xm0s4qVHhrfy6y7HQT\nKgcHARa+mWC2Vr1c0YBF7x/E0fQSC5aMkJ7pUUC/e/duzJs3D0uXLkVmZiYSExMxZcoUlJaWdphf\npVJh0qRJ8PPzQ0ZGBjZt2oR169Zh/fr1XJ7CwkJMnToVY8eORWZmJhYvXow5c+Zg7969XJ4jR47g\nqaeewqFDh3DixAlERUXh/vvvx+XLl2/zYxNiTt6gwbsH8/FjnoJL85WKsXZCf4wP9ejiTPvGsiz2\nJV/CstWHUVNnWpufz+fh1ZmxeOX5EeDz6SYgIdbIVSzAinGRmBHdNq6+qlmHxYcu2e0QHB7PtFb9\nX99IgFgsAABotXps2Hoc2785A63WYOESEtI5hu3BLhOjRo3C8OHDzXrQ+/fvj8ceewyrV69ul3/r\n1q1YvHgxFAoFxGLTbb1Vq1Zh69atKCsrAwAsWrQIP/30E/Ly8rjzXn75ZWRnZyMtLa3Tsvj5+WHp\n0qV44403AAB1dW1jnF1dadUR0rGMjAwAQFxcXLvXUoqr8Y+MEq5XHgBGB7hh3sgQOLc06qS9ZrUO\nW/95GqnH23qvnKQivPNWIgYP9LFgyW5dV/WEkFa2Vk8yKurwyfEiqDRtS1kO9HLCwtGhkDmJuzjT\ndpWU1WHtp2kol7fdsQgLdsOCN0b3eD6QrdUTcmf0VhzbbfeZVqvFmTNnMHnyZLP0yZMndxp4p6en\nIykpiQvmW/OXl5ejuLiYy9PRe2ZkZMBg6PhbsEajgVqthrs7rZRBbl+TzoBPjhfho/RCLpjnMwxm\nDQvA0rHhFMx3obi0Fu++/4dZMB8e4o6170/ss8E8IfYqzs8Vn94/EIO924bg5CobMOe3XBy4orTL\n3WWDA13x0XsTMCo2gEsrLKnFwuW/49DRIrv8mxDr1m3EolQqYTAY4Otrvhumj48P5PKON2GQy+UI\nDg42S2s9Xy6XIyQkBAqFot17+vr6Qq/XQ6lUtnsNAJYuXQpnZ2c8/PDDHf7e1m/DhHSmtY5cqjfg\nP2U61GrbGmVPMYOngkQIaSzD6dNlliqiVWNZFifOVOPXg3Lo9W1/u/jh7vjTRFeUlVxEmQ0MN6W2\nhPSErdWT6S4sPNR6HFDoYWSBJgBrDudi72keHgsUwU1kf/OI7ksUQergin1/yGEwsGhsBD7ceBj/\n96sLHr7fH07S7jt+bK2ekN4VGRnZK+9zR7ogGab3/9Nv2rQJX3zxBf744w84OTl1fwIhHdAYWPyv\nQof0KvO7QLHufEwLEMKBb38XrJ5qaNTjx31XcfFy2y1ooZDBQ5P9ETuU7poR0tfxGQYTfIWIdObj\n3yVaKDWmL+159UZ8kq/Gg35CxHvwwbsD13hrxTAMRsd6IjhAgt0/l0JZbVrKMjtPheKyJkyb4o+B\nkbQkL7G8bgN6Ly8v8Pl8KBQKs3SFQgE/P78Oz5HJZO1671vPl8lkXeYRCATw8vIyS9+4cSOWL1+O\n3377rcuxaDROjXQmIyMDuSoD/miQoFLNg6RlXyhnkQCvxwbhnhCa+NqVYydK8fXuM6hvMEAqNW3C\nEhbshvmvj0agv+1czGjMK+kJW68ncQCm6o345vxV/JxfCRamwH5fDVAscMIbccEIcXW0bCEtYNJE\nHXb++zwOHDLtJssC+PFXJapqpZj1zDC4upgvz2vr9YT0juvH0N+ObsfQi0QixMbG4sCBA2bpycnJ\nSExM7PCchIQEHD16FBqNxix/QEAAQkJCuDzJycnt3jM+Ph58ftta3+vXr8fy5cuxb9++Tn8fIV2p\nbtbhm2It/lmoRWVT20YhowPcsHVKNAXzXahTqfHxP9LxyeZ01De0/X9+8P7+WPP3CTYVzBNC2jgI\neHh5RBA+vC8S/tdNjM2ubMBbv+Xim/PlUOvta9UXRwchZs+KxdK/JsH9ui80KenFeOvd/Th8jMbW\nE8vhv/fee+91l8nFxQV///vf4e/vD0dHR6xcuRKpqan4+uuv4erqisWLF+PDDz/Ec889B8C0As62\nbduQmZmJgQMHIjU1FW+//TYWL16MhIQEAEC/fv3w0UcfobKyEiEhIfj555+xevVqrF+/HgMHDgQA\nrFu3DkuWLMHOnTsRExODhoYGNDQ0wGg0chNur//S4OBAm9eQNjqDEb/kX8OHaQUorDVtEiUUCeEi\nFuDN+GA8P9QfjrRRVIdYlsXBlCJ8uCkNVwqruXQvTwkWvpmAKRP72eSSlOXl5QAAf39/C5eEWDN7\nqic+UjEmh3sBLJBX1QgjACOArMoGHCqqhoeDEMGuDndkqK218pM54757QlFdrUZxmal3Vas14MTp\nq8jNVyIi1B2uLg52VU/IreutOLZHy1YCpqUo165di4qKCgwZMgQbNmzA2LFjAQCzZs3CkSNHUFBQ\nwOXPysrCG2+8gZMnT8LDwwOzZ8/GsmXLzN4zJSUF8+fPR3Z2NgICArBo0SK88sor3OthYWEoKSlp\n94135syZ+Oc//wmAlq0kHTtTocK2M6Xcbq9NjU0AgIcGBeHFmEC40go2nSorV2HbjtPIvlhplj7h\nnjDMfHoYpBKRhUp259EtctIT9lpPimqbsTmjBDnKBrP0wd7OeGF4AKI8pRYqmeWcOVeBz3echrKq\niUvj83l4cHIkIoLVcBDz7a6ekJvTW3FsjwN6a0UBPbleUW0zvj53FRkV5mPSpPpmPBIgxFP3jrJQ\nyaxfY5MW//kpB/uSL8NgMHLp3p5SvDpzBEYM63jOjC2x10CN3Bx7ridGlsXvhVXYea4ctRqd2WsJ\nAW74yxB/hLrZ1/j6ZrUO3/6QhX3Jl806IBloMfEeH7wya6JN3tEkvYMC+hYU0BMAuNaoxa6scvxR\nWM1N4AIAiYCPpwf7wU9VAgGPscsLcHcMBiP+SCnCtz9cgKq+7dYfj8fg4SlReOLP0XBwsI87GvYc\nqJGeo3oCNGoN+Hd2Bf4v/xr0ZkEsg3tDPfDMYD+725SqqKQWX/7rLHLzTXc3G1vuDA/o74fnZgzF\niKEyuxqaRHqGAvoWFNDbt8omLb7PkWP/FaXZRYXHMJgY5onnhvjD3VFIF+AOsCyLk2fKsev7Cygr\nV5m9NqC/F15+dgTCQtwsVDrLoHpCeoLqSZsylRq7ssqRUlJjli5gGEyO8MKMaBk8bXiY3o1YlkVK\nWgm++c95lJZVAQC3MtiASC889ehgDImmzfdIGwroW1BAb58q6jX4KV+BA1eqoDUazV6L93fFzKEB\nZrd96QLchmVZXMi5hn/vyULe5Sqz17w8JXjuyaEYMyrILnuSqJ6QnqB60t6Vmib863x5u+GOQh6D\ne0M9MS3Kx66WutRo9Phs2wGkpCshEJpPdBw80AePPjwQQ6N97LKdJeZ6K461j/voxGbkVTViz0UF\n0stqYbzhu2i0lxP+MsQfw3ydOznbvrEsi3NZCvzn5xxczFeaveboIMSfp0bhz1P6Q0wThgkhNynC\nXYL3x/VD1rV6/Ot8ObJbJs7qjCwOFChxoECJeH9XTB/giyHeTjYfyIrFAoxP9EHsUHdcLhHjwKEC\nbm5SVu41ZOVeQ3iIO/48NQqJIwNpjD25bXTlJlbPyLI4ebUOey8quIvE9aI8pfjLYH/EyJxt/iJx\nKwwGI45nXMXPv+bhckG12WsCAQ8PTOiHRx8a0G5TFEIIuVmDfZzx0YT+OC1X4dusCuRVNXKvnSqv\nw6nyOkR6SDE9ygeJQe4Q8Gy7zXZ2EuLl50bgz1Oj8MPPuTh4tBBGo6kzqqC4Bhu2Hsf/+48UDz0Q\niQnjwuDoILRwiUlfRUNuiNWqVetwqKgav15R4mrL8pPXi5W54JEBvhju230gb4+3yBsatThyrBj/\n3Z8PRWWj2Wt8Pg8T7gnD9IcGwMfL/paa64w91hNy86ie9AzLsshRNmLvRQVOXK0zW7AAANwdhJgQ\n5olJYZ4ItMEOhY7qiVzRgF9+y8fBo0XQavVm+Z2kIkwcH45J48LgJ6M7zfaChtwQm2QwsjgjVyG5\nsAonymrNJroCpolW40I88MgAH4S5SSxUSuvFsizyr1Qj+VABUk+UtrtgCAV8TBgXhukPDoCXJ/39\nCCF3DsMwGOTthEHeTihTqfFT3jX8UVQFbcvQkxq1Dj/kyvFDrhyDvJwwKdwTScHucBDY7oZ/Ml8n\nvPL8CMyYPgj7ki/j198vc7twNzRq8dP/LuKn/13EoAHemDAuDInxQRCJbPfvQXoP9dATq1Ber8bv\nhVX4o7AaymZtu9elQj6m9vPGQ5Het7Rigq33qDU0apGSVozkw4UoLq1t97qTVIQHJvTDlIn94O5m\nez1hvcXW6wnpHVRPbl2tWof/Xa7E/itKVDXr2r0uEfCRFOyOpGB3DPFx7tNDcnpSTzQaPQ6lFuHn\nX/OhuNZ+SKnEUYh7EkMwYVwYIkLd71hZieVQDz3p80pVaqSV1uBYWS2u1DR1mGeglxMmh5l6bRyF\n1EtxPa3WgMwLchw7WYoTp8vb9cYDQHCgK+6/LwL3jg21m7XkCSHWy81BiGcG+2NGtB8yKuqQXFCF\nU+V13N3YJr0B+wuU2F+ghItYgNEBbhgT5Ibhvi59OrjvjFgswAMT+mHS+HCczqzA7ymFOJ1ZwW1Q\n1dSsw29/XMZvf1xGoL8LRscFIiE+EKHBrjRnjJihKzy5a1iWRXGdGqmlNUgrq0VxXXOH+dzEQtwX\n5oFJYZ4ItqNlznpCqzXgXJYCx06U4tTZcjSr2/dwiUQCJI0OwqTx4YiM8KBGnxBidfg8BqMC3DAq\nwA01zTr8UVSF5IIqlF03X0ql0XMr5DiLBBgV4IpR/q4Y5usCqY0NQ+HzeRgZG4CRsQGormnGoaNF\n+D2l0KzXvqxchR9+ycEPv+RA5uuE0XGBSIwPRESYO7XzhIbckDtLpdHjnKIeZ+UqnJGrUNnUfjgN\nYFqreITMBRPDPDEywK3Xe2L68i3yOpUa57IUOHNO3mkQDwAhQW6YfG847kkMhtSONnLpTX25npC7\nh+rJncGyLHKVjUgtrcGx0toOh18CAJ9hMMBTihF+LoiRuSDSQwKeFQa0t1tPjEYWOXmV+P1wIY6f\nvtrhXVjAtH/I8MEyDBvsi6GDfOBsZzv09nU05IZYJY3eiPzqRmTK63FWoUJ+VVO7lQ1aifk8xPq5\nYEygO0YGuEJCQ2oAADq9AXmXqpB5QYHMC3IUFNd0mlfm64QxI4OQEB+IsBA36qUhhPRZDMMg2tsJ\n0d5OeCkmEHlVjThWWovU0hqzziADyyJb2YBsZQO+uVAOF7EAw32dMcjbGdFeUoS4OoJvA8NzeDwG\ngwf6YPBAH2g0epw5L8fxjDJknK0w69hRVjXh9yMF+P1IAQAGkeEeGDbYF8MG+6J/Pw8IbXiSMWlD\nAT25LdXNOuQqG5CrbESOsgFXqpvarUxzPYmAjzh/F4wJckecn4tNr2bQU2q1HpcKqpF3WYmL+VXI\nya+EWt1xTwxgCuIT44OQOJKCeEKIbeIxDAZ6OWGglxNeHB6A/OomHC+rxRm5CpdvmHOl0uiRUlKD\nlBJT54dUyMcATymivU3n9/eQ9Pk5WGKxAAnxpvHzrUMv0zPKcOpMORrN7nyzuFRQhUsFVfjhlxwI\nBXxEhLtjQD9PREV6oX+EJy2MYKMooCc9wrIsatR6FNQ2obCmGQW1TcivaoK8UdPleTyGQaSHBDEy\nF4yQuSDKU2qTE5t6ymAwQn6tAQVFtVwAX1Ray2000hEej0H/fp6IGSJD7DA/CuIJIXaFYRhEeUoR\n5SnF88MCUKvWtQzjNA3nrLlhGGKjzoDTchVOy1Wm88EgwFmMCHcJwt0d0c9dgnB3CVz66K7YIhEf\n8SP8ET/CH3q9EflXqnA+W4HMCwpcKqjG9SOpdXoDLuYrW3YHzwMA+Po4ISrCE2EhbggNcUNYsBtc\nnGmYTl/XN2szuaNUGj2u1qtRqlKjTKVGYW0zCmqaUavpeOz2jQKdHTDI2wkj/Fww1Me5zzaat6up\nWYfi0joUldSiqLQWRcV1KC6r63Qc5PV8vKQYPkSG4UN8MSTah8bEE0JICzcHIe4N9cS9oZ4wsiwK\na5tx4Vo9cpSNyK1sQPUNAT4LFmX1apTVq3GkpC3dWyJCsKsjgl0cEOAsRpCLAwJdHODWh3ZrFQh4\niI7yRnSUN2ZMH4yGBi0u5F7DuSwFzmUrOlwKU3GtAYprDUhJL+bSPNwdERrkhrAQNwQHusLfzxn+\nMifaubYP6VGktWXLFqxbtw5yuRyDBg3Cxo0bMXbs2E7zX7hwAW+++SZOnToFDw8PvPrqq1i2bJlZ\nniNHjmDBggXIycmBv78/3nnnHbz66qtmefbs2YNly5ahoKAAERERWLVqFaZNm3YLH5Ncj2VZ1Kr1\nuNakhaJBA0WTFuX1GlxVqVFar4ZK033A2UrE56G/hxQDvUyPAV5OcLWjAF6rNaCyqhHlFQ24Kq9H\nhbwe5fJ6lFc0oKaTVXzaYxAc6IKoSE8M6OeFAZGekPk6US88IYR0g8cwiHCXIMJdgmlRpuubolGL\nHGUDcipNw0FLVGoYOxgKWtmkRWWTFqcr6szSXcQCBDo7wFcqgq9UDF8nEWROYvhKRfByFFn1+Hwn\nJxE3NAcAamrVyL9ShbxLSly8XIUrBTXQ6Q3tzquuaUZ1TTPOnK8wS3d3c0SAnzP8Zc4I8HOGj7cU\n3p4S+HhJIZUK6TplRbqNvHbv3o158+Zh69atGDt2LDZv3owpU6YgJycHQUFB7fKrVCpMmjQJ48eP\nR0ZGBnJzczFr1ixIpVIsWLAAAFBYWIipU6fipZdewrfffoujR4/i9ddfh7e3N6ZPnw4ASE9Px4wZ\nM/DBBx9g+vTp2LNnDx5//HEcO3YMI0eO7OU/g+1Q6w2obtahRq1HjVqHmmad6VmtR1WTFopGLa41\nabmd+m6Gg4CHcDcJwtwcEe7e8uzmCCGfdwc+iWWxLItmtR51dWrUqjSorVOjqroJlVVNUFa1PdfW\nqbt/sxu4uTogJMgN/SM8ENXPE/37ecJJSj3whBByuxiGgcxJDJmTGPeFegIA1HojiuuacaWmqeXR\njOLaZmiNHV8HVRo9cjQNyFG2f03AMHB3FMLDUQjPlmcPB+ENaSK4WMmymu5uDhgVG4BRsQEATENw\nCopqcaWwGkUldSgqrUVJaR20uvZBPgDU1DajprYZWbnX2r3m6CCEt5cE3p4SeHlK4ObqADdXB7i3\nPLu5OcDNxYF2ur1Lul22ctSoURg+fDi2bdvGpfXv3x+PPfYYVq9e3S7/1q1bsXjxYigUCojFpjFZ\nq1atwtatW1FWVgYAWLRoEX766Sfk5eVx57388svIzs5GWloaAODJJ59EbW0t9u/fz+WZNGkSvL29\n8e2333JptrhsJcuy0BpYNOsNaNAa0KDVo0FnQIPGgHqtHo06U3q9Vm961ui5oL25g2/eN0vE5yHA\nWYwAZwcEOjsg1M0R4e6O8HMSW+XSYN0xGIw4lnYSGo0REf0GoqlZh6YmHRqbdGhs0kJVr0VtnRq1\nKjXq6jSoU5mC+J4MjekKn8+Dv8wZYcFuCAl2NT0HudGEJCtGyxGSnqB60vfpjSzKVKZhOGWqtiGm\npSo1NLfQ4XUjAcOA1TRBwgeCZd5wFgngIhbAScSHs0gAZxEfzmLTs4OAD4mQD4mAB0chHw4C3l29\n1hoMRpTLG1BUUovCklpcLVehXN4A+bUGGHrhbyGViFqCfTFcnMVwkooglYrgJBFCKhVBKhGa0lqe\nnZxEcHQQQiCwvc7CjtyVZSu1Wi3OnDmDd955xyx98uTJXOB9o/T0dCQlJXHBfGv+ZcuWobi4GCEh\nIUhPT8fkyZPbvefOnTthMBjA5/Nx/PhxvPXWW+3ybN68+aY+4J1QqlJDZzBCZ2ShN7LczzqD0XTc\n+rjuWGswQq03PTQGA9R6I5p1RmgMRjTrTccavRHNLXk6W+qxtzgJ+fB1EsNbIoKv1HQ7McBZjEAX\nB3hLRL3SmLAsC6ORhZFlwRrB/Ww0mJ4NBhYGgxE6nQE6vRH6lofpZ1OaTmdsyWOEVmeAwWCERmuA\nRmOAVquHWmOARquHWt16rIdWa4Rao0ez2hS4NzXr0NhoWhVBKi277c91PR6PgYe7I/xlzi0PJ9Oz\nnzO8PSXg2+DdC0II6esEPAahbo4IdTPfvNDIsqhs0kLeoIGiQQt5owbyBi0UjRooGrXtJuB2Rs+y\naNKxqNcB9ZXtx7F3x1HAh6OAB0lLgO8o5EPM50HEZyDi81oeDIQ8HpcuvC5dzOdByGMg4PMgYBjw\neQz4jOlz83mM6ZkxPfMYBo7uDhji6YfhI/y5PDCyqKpqwlV5Pcor6lEub0BlVSMqlU24pmzqcadX\nY5MWjU1aXK3oPu/1+HweHMQCiMV8OIgFcBALILruZ7GYD7HI9LODAx8ikQBCAQ+ClodQyINQwDf9\nLOCB3/JsysOHUGjKx+cx4PEY8Hg88PmmvwePb/obMC1/Cx6PsfrhRV0G9EqlEgaDAb6+vmbpPj4+\nkMvlHZ4jl8sRHBxsltZ6vlwuR0hICBQKRbv39PX1hV6vh1KphK+vL+RyeYd5Ovu9d9MjX59AF4uS\nAED7cJzt9JWend/D0xkAYiMLEdvybATELAtRy8+ORhYORkDY8j61LItaABdb355tfW77RUajKTA3\nGFmwxrZgnL3u+frXDS35+/ieZRAJ+XB1Md06dHURw8PdET5epluLXi1jCN3dHChoJ4QQG8FjGNO4\neakY8G3/ulpvRE2zDtVqHaqatdwQ1+pmLaqb9VxaYydDWHqqWW9As97QboLv3caAAY8xxRY8KQ+M\nkzN4oS6mNAMLfWuHnM4IQ0tnnKGlM671NbAsmBvCAaaT+ODGfOYMgNYAaLVAfWd5u447un7/7vMy\njKkjz/SzKfhHy9+H4Zn+VlxGtKS3/sy91pLe8sPe2bE9L1QXen32oiW/wVx/2+JO+mPm8Lvye4i1\n06GhwbKNLeldkZGRAO5eW0L6Jqon9k0CQCICAkV8wJXGhxPr0GXXopeXF/h8PhQKhVm6QqGAn59f\nh+fIZLJ2veit58tksi7zCAQCeHl5dZmn9T0IIYQQQggh3QT0IpEIsbGxOHDggFl6cnIyEhMTOzwn\nISEBR48ehUajMcsfEBCAkJAQLk9ycnK794yPjwefz+8yz5gxY3r40QghhBBCCLEDbDd2797NikQi\ndvv27WxOTg771ltvsc7OzmxJSQnLsiz77rvvshMmTODy19XVsTKZjJ0xYwablZXF7tmzh3VxcWHX\nr1/P5SksLGSlUik7b948Nicnh/3yyy9ZkUjE7t27l8uTlpbGCgQC9sMPP2Rzc3PZ1atXs0KhkD15\n8mR3RSaEEEIIIcRudLtsJWBainLt2rWoqKjAkCFDsGHDBm5jqVmzZuHIkSMoKCjg8mdlZeGNN97A\nyZMn4eHhgdmzZ7fbWColJQXz589HdnY2AgICsGjRIrzyyitmefbs2YOlS5eioKAA/fr1o42lCCGE\nEEIIuUGPAnpCCCGEEEKIdbL69fZSUlLw8MMPIzAwEDweDzt37uwyf1FREXg8XrvHjfMAiO1Ys2YN\n4uPj4erqCh8fHzz88MPIzs7u9rwLFy5g3LhxkEgkCAwMxIoVK+5CaYml3Eo9ofbE/mzevBnDhg2D\nq6srXF1dkZiYiH379nV5DrUl9udm6wm1JWTNmjXg8XiYM2dOl/lutT3p9WUre1tjYyOGDh2K559/\nHs8991yPl8Xcv38/hg0bxh27u7vfqSISCzty5AjefPNNxMfHw2g0Yvny5Zg4cSJycnI6/XdXqVSY\nNGkSxo8fj4yMDOTm5mLWrFmQSqVYsGDBXf4E5G64lXrSitoT+xEUFIS1a9ciMjISRqMRO3bswLRp\n03Dq1CmzOtCK2hL7dLP1pBW1Jfbp+PHj+PLLLzF06NAu49jbak8sO4T/5jg5ObE7d+7sMk9hYSHL\nMAybkZFxl0pFrE1DQwPL5/PZ//73v53m2bJlC+vq6sqq1WoubeXKlWxAQMDdKCKxAj2pJ9SeEJZl\nWQ8PD/aLL77o8DVqS0irruoJtSX2q7a2lo2IiGAPHz7Mjh8/np0zZ06neW+nPbH6ITe3avr06fD1\n9cXYsWOxZ88eSxeH3EUqlQpGo7HLno/09HQkJSVBLBZzaZMnT0Z5eTmKi4vvRjGJhfWknrSi9sQ+\nGQwGfPfdd1Cr1bjnnns6zENtCelJPWlFbYn9eeWVV/D4449j3LhxYLuZtno77YnNBfTOzs745JNP\n8P333+PXX3/FhAkT8OSTT2LXrl2WLhq5S+bOnYuYmBgkJCR0mkcul8PX13xf79bjGzeAZVRBAAAD\neklEQVQ0I7apJ/WE2hP7dOHCBTg5OcHBwQGvvPIK/vOf/yAqKqrDvNSW2K+bqSfUltinL7/8EgUF\nBVi5ciUAdDts/HbaE6sfQ3+zPD09MX/+fO54xIgRqKqqwtq1a/HMM89YsGTkbliwYAHS0tKQmpra\n5X+cns7FILapp/WE2hP7NGDAAJw/fx51dXX4/vvvMWPGDBw6dAhxcXHt8lJbYr9upp5QW2J/8vLy\nsGTJEqSmpnKbprIs22Uv/e20JzbXQ9+R+Ph4XLp0ydLFIHfY/PnzsXv3bhw8eBChoaFd5pXJZO2+\n7SoUCu41Yrtupp50hNoT2ycUChEeHo6YmBisXr0ao0ePxubNmzvMS22J/bqZetIRaktsW3p6OpRK\nJQYNGgShUAihUIiUlBRs2bIFIpEIOp2u3Tm3057YRUCfmZkJf39/SxeD3EFz587lgrT+/ft3mz8h\nIQFHjx6FRqPh0pKTkxEQEICQkJA7WVRiQTdbTzpC7Yn9MRgMMBqNHb5GbQlp1VU96Qi1JbbtkUce\nQVZWFs6dO4dz584hMzMTcXFxeOqpp5CZmQmhUNjunNtqT25//u6d1dDQwJ49e5Y9e/YsK5FI2A8+\n+IA9e/YsW1JSwrIsy7777rvshAkTuPw7duxgv/32WzYnJ4e9ePEiu27dOlYkErEbN2601Ecgd9jr\nr7/Ouri4sAcPHmQrKiq4R0NDA5fnxnpSV1fHymQydsaMGWxWVha7Z88e1sXFhV2/fr0lPgK5C26l\nnlB7Yn8WLVrEHj16lC0sLGTPnz/PvvvuuyyPx2MPHDjAsiy1JcTkZusJtSWEZVl23Lhx7Jtvvskd\n92Z7YvUB/aFDh1iGYViGYVgej8f9PGvWLJZlWXbmzJlsWFgYl3/nzp1sdHQ0K5VKWRcXFzY+Pp7d\ntWuXpYpP7oIb60br4/333+fy3FhPWJZlL1y4wN5zzz2sg4MD6+/vz37wwQd3u+jkLrqVekLtif2Z\nOXMmGxISworFYtbHx4edNGkSF6S1vk5tCbnZekJtCWFZtt2ylb3ZnjAs280aOoQQQgghhBCrZRdj\n6AkhhBBCCLFVFNATQgghhBDSh1FATwghhBBCSB9GAT0hhBBCCCF9GAX0hBBCCCGE9GEU0BNCCCGE\nENKHUUBPCCGEEEJIH0YBPSGEEEIIIX0YBfSEEEIIIYT0Yf8f47Ibu4i/EXkAAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from hockey import *\n", + "import thinkplot\n", "\n", - "Figure [fig.hockey1] shows the resulting posterior distributions for\n", - "lam. Based on the first four games, the most likely values\n", - "for lam are 2.6 for the Canucks and 2.9 for the Bruins.\n", + "suite1 = Hockey('bruins')\n", + "suite1.UpdateSet([0, 2, 8, 4])\n", "\n", - "The distribution of goals\n", - "-------------------------\n", + "suite2 = Hockey('canucks')\n", + "suite2.UpdateSet([1, 3, 1, 0])\n", "\n", - "![Distribution of goals in a single game.](figs/hockey2.pdf)\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(suite1)\n", + "thinkplot.Pmf(suite2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.1: Posterior distribution of the number of goals per\n", + "game.\n", "\n", - "[fig.hockey2]\n", + "Figure 7.1 shows the resulting posterior distributions for\n", + "`lam`. Based on the first four games, the most likely values\n", + "for `lam` are 2.6 for the Canucks and 2.9 for the Bruins." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The distribution of goals\n", "\n", "To compute the probability that each team wins the next game, we need to\n", "compute the distribution of goals for each team.\n", "\n", - "If we knew the value of lam exactly, we could use the\n", + "If we knew the value of `lam` exactly, we could use the\n", "Poisson distribution again. `thinkbayes` provides a method that computes\n", "a truncated approximation of a Poisson distribution:\n", "\n", + "```python\n", " def MakePoissonPmf(lam, high):\n", " pmf = Pmf()\n", " for k in xrange(0, high+1):\n", @@ -187,154 +527,366 @@ " pmf.Set(k, p)\n", " pmf.Normalize()\n", " return pmf\n", + "```\n", "\n", - "The range of values in the computed Pmf is from 0 to high.\n", - "So if the value of lam were exactly 3.4, we would compute:\n", - "\n", - " lam = 3.4\n", - " goal_dist = thinkbayes.MakePoissonPmf(lam, 10)\n", - "\n", + "The range of values in the computed Pmf is from 0 to `high`.\n", + "So if the value of `lam` were exactly 3.4, we would compute:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "lam = 3.4\n", + "goal_dist = thinkbayes.MakePoissonPmf(lam, 10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "I chose the upper bound, 10, because the probability of scoring more\n", "than 10 goals in a game is quite low.\n", "\n", "That’s simple enough so far; the problem is that we don’t know the value\n", - "of lam exactly. Instead, we have a distribution of possible\n", - "values for lam.\n", + "of `lam` exactly. Instead, we have a distribution of possible\n", + "values for `lam`.\n", "\n", - "For each value of lam, the distribution of goals is\n", + "For each value of `lam`, the distribution of goals is\n", "Poisson. So the overall distribution of goals is a mixture of these\n", "Poisson distributions, weighted according to the probabilities in the\n", - "distribution of lam.\n", + "distribution of `lam`.\n", "\n", - "Given the posterior distribution of lam, here’s the code\n", + "Given the posterior distribution of `lam`, here’s the code\n", "that makes the distribution of goals:\n", "\n", - " def MakeGoalPmf(suite):\n", - " metapmf = thinkbayes.Pmf()\n", + "```python\n", + "def MakeGoalPmf(suite):\n", + " metapmf = thinkbayes.Pmf()\n", "\n", - " for lam, prob in suite.Items():\n", - " pmf = thinkbayes.MakePoissonPmf(lam, 10)\n", - " metapmf.Set(pmf, prob)\n", + " for lam, prob in suite.Items():\n", + " pmf = thinkbayes.MakePoissonPmf(lam, 10)\n", + " metapmf.Set(pmf, prob)\n", "\n", - " mix = thinkbayes.MakeMixture(metapmf)\n", - " return mix\n", + " mix = thinkbayes.MakeMixture(metapmf)\n", + " return mix\n", + "```\n", "\n", - "For each value of lam we make a Poisson Pmf and add it to\n", + "For each value of `lam` we make a Poisson Pmf and add it to\n", "the meta-Pmf. I call it a meta-Pmf because it is a Pmf that contains\n", "Pmfs as its values.\n", "\n", "Then we use `MakeMixture` to compute the mixture (we saw\n", - "MakeMixture in Section [mixture]).\n", - "\n", - "Figure [fig.hockey2] shows the resulting distribution of goals for the\n", - "Bruins and Canucks. The Bruins are less likely to score 3 goals or fewer\n", - "in the next game, and more likely to score 4 or more.\n", - "\n", - "The probability of winning\n", - "--------------------------\n", + "`MakeMixture` in Section [mixture])." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAugAAAEWCAYAAAA0KmJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdUVNfaBvDnzAxD770KKKCIogIWrKgxUXOTm8S0m+RL\nM2qK6cbYu8aYm+a9liQmpsfEmG4SKxY0KiI2QEBA6gy9l2HK98eQg1wVZmQQhOe3lmt59py9zx44\nwDv77P1uQafT6UBERERERF2CpLM7QEREREREzRigExERERF1IQzQiYiIiIi6EAboRERERERdCAN0\nIiIiIqIuhAE6EREREVEXwgCdiIiIiKgLMShA37BhAwICAmBpaYnIyEgcPnz4mufGxsbizjvvhJeX\nF6ytrREeHo5PPvnkinMkEskV/1JTU9v3boiIiIiIbnJtBujbtm3Diy++iIULFyIxMRHR0dGYPHky\ncnJyrnr+0aNHER4eju+//x7nz5/H008/jRkzZuDrr7++4tykpCQoFArxX58+fdr/joiIiIiIbmJC\nWzuJDhs2DIMGDcLmzZvFsuDgYEybNg2rV6826CL3338/NBoNtm/fDkA/gj5+/HgUFRXB2dm5Hd0n\nIiIiIupeWh1BV6lUSEhIwKRJk1qUT5o0CUeOHDH4IhUVFXBycrqiPDIyEl5eXpg4cSJiY2MNbo+I\niIiIqLuStfZicXExNBoN3N3dW5S7ublBoVAYdIFff/0V+/btaxHQe3l5YdOmTYiKikJDQwM+//xz\nTJgwAQcOHMCoUaOu420QEREREXUPrQbo7RUXF4eHHnoI69evR2RkpFgeHByM4OBg8Xj48OHIysrC\nunXrWgToFRUVHdk9IiIiIqIOZW9vb3SdVqe4uLi4QCqVQqlUtihXKpXw9PRsteHDhw9jypQpWLFi\nBWbOnNlmR4YOHYq0tDQDukxERERE1H21GqDL5XJERERg165dLcp3796N6Ojoa9Y7ePAgpkyZgmXL\nluH55583qCOJiYnw8vIy6FwiIiIiou6qzSkuL7/8Mh555BEMHToU0dHR2LRpExQKBWbNmgUAmDdv\nHk6cOIE9e/YA0GdomTp1Kp577jk8+OCD4lx1qVQKV1dXAMC7776LgIAAhIaGQqVS4YsvvsBPP/2E\nHTt2XLMf1/N4gLovRXUD3jiSibTSGtTW1AIArKytxNcjPe1xTz93DHC1gSAIndVN6kLi4+MBoMV0\nO6L/xfuEDMH7hNrS3mnabQbo9913H0pKSrBy5UoUFBRgwIAB2LlzJ3x9fQEACoUCGRkZ4vmffvop\n6uvrsW7dOqxbt04s9/f3F89rbGzEnDlzkJubC0tLS4SFhWHnzp247bbb2vVmqGf4K7cc7xzLQnWj\nRixzNhdQd9k58QUViC+oQIizNe7p647h3g6QShioExERUdfXZh70znT5pw+OoJNaq8PW03n44ULz\nmgiZICDGXoWRzlLYBYbi+xQljuSUQ4eWt7WPrQXu6uuOmF5OMJcZtIEudTMc8SJD8D4hQ/A+oba0\nN4bt0CwuRKZSVKvC2iOZSC6uFsvcrOR4fWQgqjKTAQAhztaYPzIQeVX1+CFFib2ZpVBptQCA3Kp6\nrD9xCV+ezccdwW6Y3McFNnLe/kRERNT1cCiRurwT+RWY/Udyi+B8mJc93ru1H0Kcra8439vWAs9F\n9cLH/wjDff08YG0mFV8rrW/E1jN5ePznc9iSmIviWtUNeQ9EREREhuIQInVZGq0On5/Nx3fJzZti\nSQUBjw70xt193dpc/OloaYZHw71xb6gH/rhYjJ8uFKK4Th+Q16o12JGixM8XChHj74S7+7rDz96y\nQ98PERERkSEYoFOXVFKrwptHs3CuqEosc7GUY250AEJdbYxqy8pMirv7uuMfQa6IvVSKHSmFyK7U\nLylV63TYnVmC3ZklGO7tgHv6uhvdPhEREZEpMUCnLueUohLrjmaiokEtlkV42OHl4f5wsDC77nbN\npBLcEuiCCQHOOJFfge+TlTh/2bSZv/LK8VdeOfq72OCefu6I8rKHhCkaiYiI6AZjgE5dhkarw9fn\nC/DNeYWYhUUiCHg4zBP3hnqYLFiWCAKGeTtgmLcDkoqq8X2KEn/llYuvny+uxvlD1fC1s8DdTZlf\nzKRcrkFEREQ3BgN06hLK6hrx1l9ZSFRWimVOFmaYEx2AgW62HXbdUFcbhLraIKeyHjtSlNiXWQJ1\nU+bRnMp6vHf8Er48WyBmfrG6bMEpERERUUdggE6d7kxhFdYdyURpfaNYFu5uiznDA+Boef1TWozh\na2eBF4b2wsNhnvgptRC/pxejVq3fCKm4ToWPT+diW1IBpvRxxZ3BbjesX0RERNTzMECnTqPV6fBd\nkgJfnCuAtmnUWoCAB8M88ECoZ6fs/OlsJccTg3xwf6gndl4sws8XCsUPDjWNGnyXrMCPF5SYEOCM\nu0Lc4WNnccP7SERERN0bA3TqFBUNavz7aCZOKpqntNibyzBnRAAGe9h1Ys/0rOVS3NvPA3cGu2F/\nVil2pCiRW1UPAGjU6vDHxWL8ebEEI3zsMa2fx1XzsRMRERFdDwbodMMlFVVj7ZFMMSc5APR3tcHc\nEQFwtpJ3Ys+uJJdKcGtvF9wS6Iy/csuxPUWJCyU1AAAddDiSW44jueUY4GaLaf3cEeFh12Z+diIi\nIqLWMECnG0ar02FHihKfncmHpmlKCwDc188DDw/wuq4pLbV1jfj6h2zk5NehT+9KeHvYwsvTFt4e\ntvD0sIWHuzXMZO1f2CkRBET7OmKEjwPONWV+OZFfIb5+trAKZwur4G9viXv6uWOMnxNknTBFh4iI\niG5+DNDphqhsUOOdY1k4fllQayuX4ZXh/ojysr+uNquqG7DirUM4l6KfJpOSWoyU1OIW50gkAtxc\nrcXA3cvDFt6e+v872lsYPdotCAIGuNligJstssrr8H2KEgcvlYqZX7Iq6vDvv7Lw2Zl83BXijkmB\nzrBk5hciIiIyAgN06nAXSmqwJi4DRbXNU1r6udhgbnQAXK9zSktpeR2Wv3kQ2bkVrZ6n1eqgUFZD\noazGydMFLV6ztDCDp4eNPmD/O3D30P+zsGj7R8PfwRKvDPfHIwO88HNqIX6/WIR6tRYAUFSrwgen\ncvD1+QJMDXLFP4Jc27XJEhEREfUcDNCpw+h0OvycWoSPE3PFEWYAuCvEHY+Fe1/3FBBlUQ2Wrj0A\nZaF+F1BBAKZM8MCYUYOQV1CFAmUVcvOrkK+oQnFJ7TXbqatvREZWGTKyyq54zcnR8srA3dMWrs5W\nkP7PpkVu1nJMH+yD+0I98Ht6EX5KLRR3Qa1SqfHN+QLsSFFiYoAz7g5xh6et+XW9byIiIuoZGKBT\nh6hWqfHe8Us4ktu8Q6eNmRQvDfPHcB+H6243N78Sy948iJJSfeAtkQi49w4fhIc6YNAADwwa4NHi\n/IYGNQqU1cgr0Afs+QVVyFdUI6+gErV1jVe7BACgtKwOpWV1OJtU2KLcTCaFh7sNvDxsxLnuXp76\nIN7Oxhz39/fEP0PcsTerBDtSlCiobgAAqDRa7Ewvwh8XizHSxwH39HNHkBMzvxAREdGVGKCTyaWX\n1uKNIxlicAoAQU7WeD06AB421z96nJFVhuXrDqKySt+umUyKV58bAUGbf8065uYy+Ps5wN+v5YcC\nnU6H8ooGFCir9MF7QRXyFFUoUFRDUVgNjUZ71fYa1Rrk5FUgJ+/KqTW2NuaXTZOxwRMe9ijwEhBb\nWImM8joA+oWyh3LKcCinDIPc7TCtnzsGudsy8wsRERGJGKCTyeh0OuxML8aHp3LQqG2e0vKPIDc8\nMcgb8v+ZGmKM5NRirPr3IXHU29xchnkvjsTA/u6Ij792gH4tgiDA0cECjg4WCA1xbfGaWq2Fsqga\n+QXV+lF3RZU4Al9eUX/NNquqG5CS1oCUtJYLVSEIsPO2RbmrJUrlUsjNJDAzkyI+rxyJykr0cbTC\nPX3dMdLXsVM2ZyIiIqKuhQE6mURtowbrT1zCwezm+dxWMimeH9oLo/0c29X26XMKrHn3CFQq/bxu\nays5Fr46GiF9nNvV7rXIZBJ4e9rB2/PKDZNqalXiaPvlAXy+olrs3xV0OjTkVsIytxLO5lKUOFug\n2M4cOkH/QSE3rxJxF4rgYmGGCd72mBriBn9ve1hZclEpERFRT8QAndots7wWb8RlijttAkBvRyu8\nHh0AL1uLdrV97GQe/v3fo1A3ZUext7PAktfGXDFl5UaxtpIjqLczgnq3/HCg1epQUlqLfGU18vIr\nxaA9v6AKhcW1APRPFCwaNPDOr4FrUR1KnSxQ7mCOBp0ODSoNqmtU2FJSg62ncuFU2oAALeDv3jI1\npJe7Ddxcra9YqEpERETdBwN0um46nQ67M0qwMSEHqsvmbE/u7YqnBvvAXNa+IDI2Lgv/+fAEtE3T\nZVycrbDktTFXHdnubBKJAFcXa7i6WCO8v3uL11QqDQqUzYtT9XPe9aPvFcV1KHO0QKmTOTRNQbdG\nKkGRqyVKtDqkl1bDKb0Y8sbmr69UKoGnu42YWebyhaq2NnLOZyciIrrJMUCn61Kv1uC/8TnYl1Ui\nllnIJJgd2Qvj/J3a3f7ve9Lx4WcJ4rGnuy2Wzh0DV5ebL/OJXC5FL18H9PK9cqFqZVUD8gqqkJVf\ngf3Z5firvAbljVqo1RpoJQJKnSxQ5mgBu8oGOJfUw6JBA41Gi9z8SuTmV15xLRtruRi49wlwxIQx\nATA35485ERHRzYR/uclolyrq8EZcJrIr68Qyf3tLvD4yEL527ZvSAgA7fknGF9+dFY97+Tpg8Zwx\ncHRof9tdiSAIsLezgL2dfqHqFAAarQ6HssuwPVmB1OIaNDZqoGrUoNHOHAXu1rCqaoRNfhWsatW4\n2jh5dY0KqRdLkHqxBLGHs7DvUBbmvzQKTo6WN/rtERER0XVigE5G2ZdVgv+cyEbDZVNabglwxqwI\nP1i0c0qLTqfDF9+dxQ+/pohlQYHOWPTqaNjYXN+OozcbqUTAOH8njO3liARFJb5PUeK0sqr5BE9A\n28cRzhZmGGJlDoeaRhQoqlGgqEaeogoNDS0XqmZklWHusr2Y/9IoBPTqnHn7REREZBwG6GSQBrUW\nmxJysCujOYWguVSCpyN8cUugS7vb12p1+OjzU/hjb7pYNiDUDXNfGNkjs5kIgoAIT3tEeNrjQkkN\ndqQocSS3HFqdDhKJgAKVGr+p1PCyMcfdt/bBc/7OkEsFlJTWIU9RheQLxdj+c5K4eHXBqn14+Znh\niBzk1dlvjYiIiNrAVBDUptzKeryyJ6VFcO5ja4G3b+lrkuBco9Fi/YfHWwTnkYO9sODl0T0yOP9f\nIc7WmDcyEJunhGJyb1fIJc0/tvnVDfhPfDae+OUcvk1SwNxGjvD+7njg7v5Y+Erz16++Xo0178Th\nt11pnfU2iIiIyEAM0KlVBy+V4sVdKcgsb55vPq6XE96d1Bf+Du2f16xq1OCt/xzFgbhLYtmo4X54\nbXY05HJpu9vvTrxsLfBclB8+/kcYHgj1hI1Z89envKERn53Nx+O/nMNHp3JRVKvCoAEeWL1oPFyd\n9QtrdTodtnxxCh99nnDNnVKJiIio8zFAp6tqUGvx3/hsrD2aiTq1BgAgl0jwXKQfXh3uD0uz9gfP\n9fVqrH77MI6dzBPLbokJxAszh0LWzvns3ZmjpRkeGeiFT+4YgKcG+8DVqnl+fp1agx8uKDH9l3N4\n51gW7F2t8caSCQgKbM7bvnN3Ota8EyfuykpERERdC6MgukJBVQNe23sBO9OLxDIvG3O8dUsIJvdx\nNUme7eoaFZavO4gz55Vi2T+nhGDWYxHchMdAVmZS/DPEHR/dHoaXh/mjl33zEw21Toc9mSWYvy8V\nEksZVswfhxFRvuLrCWcKsGDlfhQV13RG14mIiKgVjISohbicMrywKxnpZbVi2ShfR7x3az/0drQy\nyTXKK+qxZE0sUtKa57T/a1oYHrl/IDfZuQ4yiYAJAc747239sGRMH4S52oqvZVXUYVFsGup1Orzy\n7HBMuyNUfO1STjleX7YP6RmlndFtIiIiugYG6AQAaNRo8UFCDlbHZaCmUT+lRSYImDXEF69HB8DK\nBFNaAKC4pBaLVsciM7tcLHvi4cGYdkcog/N2EgQBQ73ssXZCMF4Z7g+hKVP6xbJaLIpNQ61ag39N\nC8Psp4aKTynKKuqwcHUs/orP7cSeExER0eUYoBOUNQ2Yuy8VP6UWimXu1uZYNzEE/wh2M1ngXKCo\nwoKV+5FXoN8BUxAEPDc9CrdPCjJJ+9RsvL8zXhjqJwbp6WW1WHwgHbWNGsSM9seS18bAxlo/d12l\nUuPN94/ih99SoNPpOrPbREREBAboPd6xvHK88GcKLpQ0z0Ue7u2A9yb1RXBT9g9TyM6twMJVsShq\nuo5UKsErzw7H+DEBJrsGtXRLoAtmR/mJxxdKarCkKUgP6+eGNYvHw8PdpulVHT7fdgYbPz4JtZoZ\nXoiIiDoTA/QeSq3V4ePEXCw/dBFVKv3ukzJBwFODfbBwVCBszU23h1XaxRIsXLUfZRX6VI1yuQzz\nXhyJ6KG+bdSk9rq1twueiWgO0pOKq7H0YDrq1Rp4e9rhjcUT0C/YVXx9z4EMrHzrEKprVJ3RXSIi\nIgID9B6pqFaFeftS8X1KcwYVVys53pgQjH+GuJt0Lvi55EIsWXtADPgsLcyweM5oDAn3NNk1qHVT\ng1wxc0jzh6HzRdVYdvAi6tVa2NmaY8ncMRg7spf4+pkkJeYt3wdFYXVndJeIiKjHY4Dew8QXVOD5\nP5ORVNwcfEV62uP9W/uhn4tNKzWNd/J0AVa+dQj19foRelsbcyybNxahIa5t1CRTuyPYDU8O8hGP\nzxRWYeWhi2hQayE3k+L5GUPx4D1h4ut5BZWYu3QvklOLr9YcERERdSAG6D2ERqvD52fyseRAOiob\n9AGzVBDwWLg3lozpDTsTTmkBgLhjOXjj3TiomjLCONpbYsX8cegT4GTS65Dh7u7rjscGeovHp5SV\nWB2XAZVGC0EQcO+doXjp6eEwk+kz9lRVN2DpGwdw6Gh2Z3WZiIioRzIoQN+wYQMCAgJgaWmJyMhI\nHD58+JrnxsbG4s4774SXlxesra0RHh6OTz755IrzDhw4gIiICFhaWqJ3797YvHnz9b8LalVpXSMW\nxKbhm6QCsczJwgyrYoJwbz8PSEyc3nDPgQy8veEvcTt5d1drrFoUAz8fe5Neh4x3b6gHHg7zEo/j\nCyrwRlwGGpu+V6NH+GHp62NhZ2sOAGhUa/DOxr/w7Y9JzPBCRER0g7QZoG/btg0vvvgiFi5ciMTE\nRERHR2Py5MnIycm56vlHjx5FeHg4vv/+e5w/fx5PP/00ZsyYga+//lo8JzMzE1OmTMGoUaOQmJiI\nefPmYfbs2dixY4fp3hkBAE4rKzH7j2ScLawSywa722H9bf0wwM22lZrX55c/UrFhS7wYzPl42WHl\nghh4uJl2+gxdvwfDPPFA/+Y1AMfyK/Dm0UyotfrvWb9gF7yxZAJ8vOzEc77ZcQ7vf3BcfCJCRERE\nHUfQtTEsNmzYMAwaNKjFCHdwcDCmTZuG1atXG3SR+++/HxqNBtu3bwcAzJ07Fz/++CMuXLggnvPU\nU0/h/PnzOHLkiFhWUVEh/t/enqOvxtBoddiWpMBX5wqgg/5bLEDAQ2GeuC/UA1KJaUfNdTodtv2Q\nhG9/PC+WBfZyxOLXxoijsR0lPj4eABAZGdmh1+lOdDodPj2Tj++SFWLZKF9HvDYiQLw3qmtUeGv9\nUZxJal5M3C/YFXNfiO7w72lH4H1ChuB9QobgfUJtaW8M2+oIukqlQkJCAiZNmtSifNKkSS0C6bZU\nVFTAyal57vHRo0ev2mZ8fDw0Go7QtVd5fSOWHEjHl+fyxeDcwdwMK8f1wYNhnh0SnG/9+nSL4Lxv\nsAuWzRt7UwZyPYEgCHh0oBfuCnEXyw7nlOHtY1nQNI2k21jLsfDV0Zg4NlA8Jzm1CHOX7RU3myIi\nIiLTazVALy4uhkajgbu7e4tyNzc3KBSKa9Rq6ddff8W+ffswY8YMsUypVF7Rpru7O9RqNYqLmTWi\nPc4WVmH2H8k4pWwOoAa42WL9bf0wyMOulZrXR6PRYuPHJ/HLH6li2aABHlgyZwysreQmvx6ZjiAI\neHKQN+4MdhPLYi+V4t3jl8QgXSaT4OknIvB/D4QDTbuSKgurMW/5PpxLLrxas0RERNROpk3d8T/i\n4uLw0EMPYf369e1+DPT34yS6Oq1Oh9giNf5UqKG9bNLSBHcZbrGtQMb508gw8TXVGi22/5KHs8nN\nj3HC+tph0mgLnD2baOKrtY33yPUZrNMhy6IRR0v0T69+PV8LZUE+pvmYiQuIfdyAO291wne/5KKx\nUYeamlq8tngn7pzsjYiBjp3ZfaPxPiFD8D4hQ/A+oWsJCgpqV/1WA3QXFxdIpVIolcoW5UqlEp6e\nrW80c/jwYUydOhUrVqzAzJkzW7zm4eFxxQi8UqmETCaDi4uLMf0nADVqHb7JUSGlsnmLdmsZ8KCf\nHCG20g65ZmOjFl/9kI3Ui8351IcMcMBdU7whMfEUGupYgiDgn95m0OiA46X6IP1EqQZSAbjb20zc\nuKp/iD3s7czwxfZsVFWrodECO37LQ2mZChNGu/H7TkREZCKtBuhyuRwRERHYtWsX7rnnHrF89+7d\nuPfee69Z7+DBg7j99tuxfPlyPP/881e8PmLECPzwww8tynbv3o2oqChIpVcPKLkQ4+qSiqrxn6OZ\nKNJIYGWtL+vvYoM50QFw7aApJrV1jVjzzmHkKbSwtrYCAEy5pQ+eeGhwpwRpXKxjGpGROrx3/BL2\nZJYAAM7UA70kbpg5xEcM0iMBjIquxaq3D+NSTjkA4MTpGsjkarwwcyjk8o75QGgKvE/IELxPyBC8\nT6gtly8SvR5tpll8+eWXsXXrVmzZsgXJycl44YUXoFAoMGvWLADAvHnzMHHiRPH82NhYTJ48GU8/\n/TQefPBBKBQKKBQKFBUViefMmjULeXl5eOmll5CcnIyPPvoIn376KV599dV2vZmeRKfT4YcUJebt\nS0VRrUosn9bPA6vHB3dYcF5V3YClaw/gfErz93PaHaF48uHOCc7JdCSCgOejemG8f/OC7l/SCrEl\nMa9FDnQXZyusWhiDIQObn6IdPZGDRatjUVZefyO7TERE1C21GaDfd999ePfdd7Fy5UoMHjwYR44c\nwc6dO+Hr6wsAUCgUyMhont386aefor6+HuvWrYOnpye8vLzg5eWFYcOGief4+/tj586dOHjwIAYP\nHow1a9Zg/fr1uOuuuzrgLXY/VQ1qrDycgY8Sc6FuCpxs5TIsHt0bj4d7Q9ZBgXJpeR0WrY5Fekap\nWPbI/QPxr2lh4ggr3dykEgEvDvXHWL/mIP2HC0psPZPfIki3sjTDvJdGYsotfcSytIwSvL5sL7Jz\n2zdqQERE1NO1mQe9MzEP+pXSSmuwJi4TypoGsSzE2RpzowPgbt1xKQ2VRTVYuvYAlIV/zzkXMPOx\nIbh1fO8Ou6ah+KjR9DRaHdYezURcTplY9kCoJx4e4HnFh7HfdqXh4y8TxQDeytIMrzw7AoMHetzQ\nPreF9wkZgvcJGYL3CbWlQ/OgU9dSWKPCgv1pLYLzfwa7Ye344A4NznPzK7Fw1X4xOJdIBLw4a2iX\nCM6pY0glAl4bEYDh3g5i2TdJBfjqfMEV506dFIR5L42EhYV+SUttXSNWvX0If+67eMP6S0RE1J0w\nQL+JfJCQg5qmrdatzaSYPzIQTw3xhZm0476NGVllWLhqP0pKawEAZjIp5j4/EmOie3XYNalrkEkE\nzI0OQJRX8yf/r84VYNtVgvTIQV5YtWA8nJ30i4a1Wh02bz2JT75KhEajveJ8IiIiujYG6DeJY3nl\nOJpXLh4vHt0bI307Nv90cmoxFq+JRWWVfsTe3FyG+a+MQtQQrw69LnUdcqkE86IDMeSyTa4+O5uP\n7clXblQW0MsBa5dOQKB/8335yx+pePP9I6irb7wh/SUiIuoOGKDfBOrVGmw8mSMeTwp0QZibbYde\n8/Q5BZa9eRC1dfrAytpKjqVzxyK8v3sbNam7MZdJsHBUbwxybw7SPzmdhx8vKK8418nBEivnx2BY\nhLdYduJUPhatihWfwhAREVHrGKDfBL4+rxBTKdqZy/B4uHcbNdrn2Mk8rHr7MFQqNQDA3s4Cy+eN\nQ0gf5w69LnVd5jIJFo3ujQGXfTD88FQufkktvOJcCwsZ5syOxj+nhIhlGZfK8NrSvcjIKrvifCIi\nImqJAXoXl1Vehx9TmkcqHw/3hp15q/tLtUtsXBbWrT8CtVo/b9jF2QorF4xDQC+HNmpSd2chk2DJ\n6N7o72Ijlm1KyMHv6UVXnCuRCPi/B8Ix6/FIMT9+WXkdFqzajxMJ+Tesz0RERDcjBuhdmFanw4b4\nbDHXeX9XG0wM6LhR7N/3pOP9zceh1eqv5+lui1ULYuDtaddGTeopLM2kWDq2D/o6W4tl/4nPxq6M\n4quePykmEIvmjIGVpRkAoKFBjTXvxuGXP1LRhTO8EhERdSoG6F3YnswSnC/WpzaUCQKeifCDpIM2\nBNrxSzI+/CxBPPbzscfKBTFwdbFupRb1RFZmUiwfG4Rgp+Z74/3j2dibWXLV88P7u2PN4glwd/37\nfB0++SoRH352ihleiIiIroIBehdV0aDGx4l54vFdfd3h72Bp8uvodDp8/u0ZfPHdWbEsKNAZK+aP\ng6ODhcmvR92DtVyK5WP7oI+jPq2iDjq8e/wSYrNKr3q+r7cd1iye0GIdwx9707Hq7cPiQmQiIiLS\nY4DeRX2SmIuqpkWa7tbmeKC/p8mvodXq8OFnp/DDryliWVg/NyyZOwa2Nh238RF1D7bmMqwYF4SA\npg+OWp0O/z6WhUPZV18I6mBvgWWvj8Oo4X5iWeJZBeav2IfC4pob0mciIqKbAQP0LuhcYRV2XzZd\nYFaELyxkpv1WaTRarP/wOP7Ymy6WRQ72wsJXRovzhYnaYmcuw8pxQehl3xykrzuaiSM5Vw/S5XIp\nXpw1DPdY9IoyAAAgAElEQVTeGSqWZedW4PVle5GafvUpMkRERD0NA/QuplGjxYbLcp5H+zhg6GU7\nOZqCqlGDt/5zFAfiLollo4b74bXZ0ZDLpSa9FnV/DhZmWDUuCD62+ilRGp0Oa49k4thlG2tdTiIR\n8OA9YXh+5lBIm3bBLa+ox+I1sThyPOeqdYiIiHoSBuhdzI+phbhUUQdAn9ZuxhBfk7ZfX6/G6rcP\n49jJ5vntt8QE4oWZQyEz8Sg99RyOlmZYMz4Y3k1Bulqnw5q4DMQXVFyzzriR/lg6dyxsrOUAmj84\n7vglmRleiIioR2NE1oUoqhvw9bkC8fjhMC+4WslN1n51jQrL1x3EmfPNedXvnByCWY9FiCOZRNfL\nydIMq2KC4Nm0fqFRq8OqQxk4pai8Zp3+fV3xxpIJ8HRv3gDpi+/OYsOWeDSqNR3eZyIioq6IUVkX\nodPpsCkhBw1NaecCHaxwR7Cbydovr6jHkjWxSElrzlf94D1h+L8HBkLooNSN1PO4WsmxOiYY7tb6\nIF2l1WL5oYs4ray6Zh0vD1u8sWQ8+vd1Fcv2HszEinWHUF2t6vA+ExERdTUM0LuIo7nlOJGvnw4g\nQMCzkb6QSkwTOBeX1GLR6lhkZjfPCX7i4cG4985QBudkcm7WcqyOCRKf/qg0Wiw7mI6zhdcO0m1t\nzLH4tTEYN8pfLDuXXIjXl+9FgeLa9YiIiLojBuhdQF2jBh+cyhWPb+vtgr6XbafeHgWKKixYuR95\nBfppBoIg4Nkno3D7pCCTtE90NR425lgTEwznv3cQ1Wix9GA6koqqr1nHTCbF7Kei8K9pYWJZvqIK\nry/fh6QLRR3eZyIioq6CAXoX8OW5AhTV6h/l25vL8OhAL5O0m51bgYWrYlFUos8xLZVK8MqzwzFh\nbIBJ2idqjaetOVbHBMPJQh+k16u1WHIgHRdKrp3zXBAETLsjFK88OwJmMn1GoarqBixde6BF1iEi\nIqLujAF6J8soq8XPqYXi8fTBPrA1l7W73bSLJVi4aj/KmjLCyOUyzHtxJKKHmjYrDFFrfOwssCom\nCA7m+iC9Vq3Botg0pJW2vjHRyGG+WD5/HOztmrLCqLV4b/MxfLPjPDO8EBFRt8cAvRNpdTr8Nz4b\nmqaAY6CbLWJ6ObW73XPJhViy9gCqa/Sj8pYWZlg8ZzSGhJt+N1KitvjZW2JVTBDsmj541jRqsCg2\nHRfLalutF9LHGW8smQBfbzux7Nsfz+OdjcegUjHDCxERdV8M0DvRnxeLkdL0uF8mCHgm0q/dizZP\nni7AyrcOob5eDUC/+G7Z62MRGuLaRk2ijuPvYIlV44JgK9cH6VUqNRbGpiGrvK7Veu6u1li9aDwG\n9ncXyw7/lY2lbx5ARWV9h/aZiIioszBA7yTl9Y3Yerp5s6Bp/Tzg2/Q4/3rFHcvBG+/GQdWoH110\ntLfEivnj0Cew/aPyRO0V6GiFleOCYGOmn1te2aDG/P2p4sZc12JtJcfCV0ZjUkxvsSwltRivL9+H\n3Pxr51gnIiK6WTFA7yRbEvNQ3RRIe9qY475Qj3a1t+dABt7e8Bc0TXnU3V2tsWpRDPx87NvdVyJT\n6eNkheXjgmDdFKRXNKixYH8actsYDZfJJJj52BA89mA4AP1TJmVhNeYt34fTl228RURE1B0wQO8E\np5VV2JdVIh7PivCFuez6vxW//JGKDVvixcVzPl52WLkgBh5upknVSGRKIc7WWDa2DyybsrSU1Tdi\n/v5U5Fe1HqQLgoA7Jofg9ReiIW+aKlNTq8LKtw5hz4GMDu83ERHRjcIA/QZTabTYEJ8tHo/2dUSk\n5/WNcut0Onyz4zw++SpRLAvs5YgV88fB2cmq3X0l6ij9XGywbGwfmEv1v4JK6hoxf38aFNUNbdYd\nGuGNVQtj4OhgCQDQaLTYsCUen207A62WGV6IiOjmxwD9BvshRYncppFCK5kUTw32ua52dDodtn59\nGt/+eF4s6xvsgmXzxoqp6Yi6sv6uNlh6WZBeVKvCvH2pUNa0HaT39nfE2iUT4O/nIJb9+FsK/v3f\no2hoUHdYn4mIiG4EBug3UEFVA75JUojHjwz0gnPTdujG0Gi02PjxSfzyR6pYNmiAB5bMGQPr62iP\nqLMMdLPF4tG9IW8K0gtrVViwP03cuKs1Ls5WWLUwBpGDmjf2OnoiF4vWxKK0jewwREREXRkD9BtE\np9NhY0IOVE2LOPs4WmFqH+NTHzaqNXh307EWc25HRPng9RdHwtwEGxwR3WiDPOywcFQgzCT6xZ8F\n1Q2Yvy8VJQYE6ZYWZpj7QjRuvzVYLEvPKMXry/biUk55h/WZiIioIzFAv0EO55TjZEEFAECAPue5\nVGJcznOtVoe31h9F3LEcsWzcKH+8/MxwyJuyYhDdjCI87TF/VG/ImvYByK9uwPz9aSira2yzrlQq\nwRMPDcJT/zdE3EeguKQW81fsR8Lpgg7tNxERUUdggH4D1DZq8EFCc1A9NcgFIc7WRrfz2640nDiV\nLx5PuaUPnpseBamU30a6+Q31sse8kYFikJ5bVY/5+9NQXt92kA4Akyf2wYKXR8HSwgwAUFffiFVv\nH8bO3Wkd1mciIqKOwMjuBvj8bD5Km4IMJwsz/N8Ab6PbyM6twBffnhWPb781GE8+PBgSI0fhibqy\n4T4OeC06ANKmID27sg4L9qehwsCFn0PCPbF6UQxcnPVZjHQ6HT76/BS2fHGKGV6IiOimwQC9g6WX\n1uLXtCLx+KnBPrCWGzcdRa3W4v0PjqNRrd/YKLCXIx65f4D4OJ+oOxnp64hXh/tD0nR/Z1XUYVFs\nGioNDNJ7+Tpg7ZKJLXbQ/W1XGr7Yno36Bk2H9JmIiMiUGKB3II1Wh//GZ0PbtIHQYHc7jPZzNLqd\n7T8nIyOrDIB+R8XnZw6FmYxzzqn7GtPLCS8N6wWhadfQi2W1WBSbhmqVYUG6o4MFVswbhxFRzWlM\nL1yswkdfZKKktLZD+kxERGQqDNA70O8Xi5FaWgMAkEskeDrS1+hR7/SMUmz/OUk8/te0AfDzub6N\njYhuJuP9nfH8UD/xOL2sFosPpKO20bBRcHNzGV55dgTuur2vWFZQWI/5K/YjX1Fl8v4SERGZCgP0\nDlJW14jPzuSJx9NC3eFta9wGQiqVBu9tPi7One0X7Ip/3Bpk0n4SdWWTAl0wO6qXeHyhpAZLD6Sj\nzsAgXSIR8Mh9A/Hsk1H4ey11UUkNFqzcLz6VIiIi6moYoHeQD0/loqYpiPC2tcC0vh5Gt/Hl9rPI\nK6gEoB8NnD2DGVuo57mttwuejmgeST9fXI2lBy+iXm34fPIJYwPw0LReMDPTP8GqqKzH4jWxOJdc\naPL+EhERtRejvQ5wSlGJA9ml4vEzEb4wlxn3pT6XXNhip9DHHgyHh5uNyfpIdDO5PcgVM4f4isfn\niqqw/NBF1Ku1BrcR0tsWjz/gL+62W1vXiBXrDuFEQn4bNYmIiG4sg6LGDRs2ICAgAJaWloiMjMTh\nw4eveW5DQwMee+wxhIeHQy6XIyYm5opzYmNjIZFIrviXmpp6lRZvLg1qLTbEZ4vH43o5YZCHnVFt\n1NY1Yv0HJ8TjIQM9MSkm0GR9JLoZ3RHshicHNS/6PK2swspDF9FgRJDey8caKxeMg6O9JQD9zrxr\n34/D/kNZpu4uERHRdWszQN+2bRtefPFFLFy4EImJiYiOjsbkyZORk5Nz1fM1Gg0sLS0xe/ZsTJ06\ntdVFkUlJSVAoFOK/Pn36XP876SK2pyiQX90AALA2k2L6ZQGFoT75MhFFJfrFpTbWcjzzZCRTKhIB\nuLuvOx4b2LyPwCllJVbHZUClMSJI93XAqkUxcG96IqXV6rD+w+P4+fcLJu8vERHR9WgzQH/77bfx\n+OOP48knn0RISAjef/99eHp6YuPGjVc938rKChs3bsT06dPh7e0Nne7am4O4urrCzc1N/CeR3Nwz\nbvKq6vFdkkI8fnSgNxwtzYxq40RCPvYezBSPZzw6BE6OlibrI9HN7t5QDzwc5iUexxdU4I24DDQa\nEaR7uNlg9cLx8PdzEMu2fn0aX3x7ttXfWURERDdCqxGxSqVCQkICJk2a1KJ80qRJOHLkSLsvHhkZ\nCS8vL0ycOBGxsbHtbq8z6XQ6bIzPQWNTxpVgJ2vc1tvFqDYqqxqw8ZN48XjkMF+MGu7XSg2inunB\nME88EOopHh/Lr8CbRzOhNmK3UEcHCyyfNw59g5t/Tnf8moxNn5yExohgn4iIyNRkrb1YXFwMjUYD\nd3f3FuVubm5QKBTXqNU2Ly8vbNq0CVFRUWhoaMDnn3+OCRMm4MCBAxg1atRV68THx1+1vKs4VaZG\nXHYjAEAQgLFeGpxKOGlwfZ1Oh29+zEFevj5ri62NDFEDhS7/vrsSfq16lr46HYZaqxFbqN+8aM+F\nWhQplXjQzwzSVqaE/e99cvsEG1RXluHCRX1u9B9/O4f0i9m49x8+kBm5uJu6D/4+IUPwPqFrCQpq\nX1rsVgP0jhIcHIzg4GDxePjw4cjKysK6deuuGaB3ZXUaHX4paN7hcKSzDD5Wxv1hP5NUgXMpleLx\nXVO8YWXZKd8eopuCIAiY4iGDVgccLNL//J0u10AiAPf7th6kX05uJsFD9/hhx848JJ4rBwCcS6lE\nXf0l/OtuP1iYc9deIiK6sVqNAF1cXCCVSqFUKluUK5VKeHp6XqPW9Rk6dCi2bdt2zdcjIyNNej1T\n2hCfDY28CFZywNnSDPOm9IeVmeF/1EtKa/HeR7tgbW0FALhlXCAeur/rvt+u5u8RjK58j1DHidTp\n8MGpXPycqs9pfkEFxGmc8eKwXpBcFqS3dZ9ERUXik68S8duuNACAokiHn3dXYeEro2Fna97B74K6\nCv4+IUPwPqG2VFRUtKt+q8O8crkcERER2LVrV4vy3bt3Izo6ul0X/l+JiYnw8vJq+8Qu5kJJDXam\nF4vHM4b4GhWc63Q6/OejeNTUqgAA7q7WePTBcJP3k6i7EgQBMwb7YEofV7Fsb1YJ1p/IhtaIBZ8S\niYAnHhqEf00LE8vSM0qxYOV+FJfUmrTPRERErWlzDsXLL7+MRx55BEOHDkV0dDQ2bdoEhUKBWbNm\nAQDmzZuHEydOYM+ePWKdpKQkqFQqFBcXo7q6GqdPn4ZOp8OgQYMAAO+++y4CAgIQGhoKlUqFL774\nAj/99BN27NjRQW+zY2i0OmyIz4YO+iAg0tMeI30c2qjV0p/7LuL0ub/n8wuYPWMorIzM/ELU0wmC\ngKcjfKHR6vBnhv4D866MYkgFAc9G+hqcplQQBEy7IxS2NubYvDUBgA55BZWYv3IfFs8ZAx8v4/Y0\nICIiuh5tBuj33XcfSkpKsHLlShQUFGDAgAHYuXMnfH31u/opFApkZGS0qDN16lRcunQJgP4P3uDB\ngyEIAjQa/dbcjY2NmDNnDnJzc2FpaYmwsDDs3LkTt912m6nfX4f6Lb0I6WX6kTW5VIKnIwwPBACg\nQFGFrV+fEY/vnByM0BDXVmoQ0bVIBAHPRflBo9NhT2YJAOD3i0WQSQTMHGLcfgS3ju8NG2s53t10\nDBqNFsUltViwcj8WvToafQKdOqL7REREIkHXhZP+Xj5/x97evhN7cqXiWhWe3pmEWrX+Q8f/DfDC\n/f0Nn5ev0WixaHUsUtL0o32+3vZYt2wi5HIuSDMW5wLS5TRaHd45loX9l0rFsrtC3BGuLoAgCEbd\nJ4lnFXjjvSNQqfSLUC0sZHj9hZEY2N+9jZp0s+LvEzIE7xNqS3tjWOYQu04fnsoVg3MfWwvc1de4\nP9g//Z4qBudSqQTPzxzK4JzIBKQSAS8N88cYP0ex7IcLSuxUqI3ehGjQAA8se30sbKzlAID6ejVW\n/vsQjp7INWmfiYiILscA/TrEF1TgcE6ZePxslB/kUsO/lFnZ5fhmxznx+N47Q9Hb37GVGkRkDKlE\nwCvDAzDSt/nnKrZQjT+V6lZqXV1IH2esXBADRwf9jr5qtRZv/eco9hzIaKMmERHR9WGAbqQGtRab\nTuaIx+P9nTHQzdbg+o1qDd7/4DjUav1OhUGBzrjnH31N3k+ink4mETBnuD+Gezcv3N6rVOOT03lG\nj6T7+dhj9aLx8HTX/6zrdDps2BKPH35LMWmfiYiIAAboRvs2SYGC6gYAgK1chicHeRtX/4ckZGXr\nN0ORm0kxe0YUpEaMvhOR4cykEsyNDkCkZ/P8v+3JCrx3/BI0WuOCdHdXa6xaGIPAXs2j8p9vO4PP\nvjltdMBPRETUGkaGRsiprMf2ZIV4/Fi4NxwsDE+JmJJWjB2/No+4PXzfAKZtI+pgcqkE80cGItSu\n+dfd7swSrI7LQEPTkyxDOdhbYPn8cejftznb0o87L2DDlnhoNMa1RUREdC0M0A2k0+nw3/hsqJtG\nyvq52GBSoLPB9evr1Vj/wQlxpC2snxum3BLUIX0lopbMZRL8n78cUU7NC7H/yivHogNpqFYZNy/d\nytIMi14dg6jBzRur7T2YiXXrj0Kl0pisz0RE1HMxQDfQ/kulOFtYBQDi5icSI3Kef/7tGRQo9fUt\nLczw3FNRkEgMr09E7SMVBNzrY4Zp/TzEsvNF1Xh9XypK6xqNaksul+K156MRM9pfLDuekIcVbx1E\nrZFtERER/S8G6AaobFDjo1PNadXuCHZDgIOVwfVPn1Pg9z3p4vGTDw+Cm4u1SftIRG0TBAGPh3vj\nyUHNGxdlltfhtb0XUFDVYFRbUqkEz02Pwh23hYhl51OKsHh1LMor6k3VZSIi6oEYoBvgszN5qGjQ\nPwZ3tZLjoTDDNySqrlFh/YcnxOOowV4tRt2I6Ma7u687XhrmD2nTU7CC6gbM2XsBF5t2BjaUIAh4\n9MGBePi+gWJZxqUyLFy1H4XFNSbtMxER9RwM0NuQXFyN3y8Wi8czh/jC0szwDYW2fHEKpWV1AABb\nG3PMejwSghFTY4ioY0wMcMaCUYHiHgZl9Y2Yty8VZ5qmshlKEATcfXvfFj/b+YoqzF+xD9m5FW3U\nJiIiuhID9FZotPqFoX8b5mWP4d6Gb9d69EQuDsRdEo9nPR4BRwcLk/aRiK7fMG8HrBwXBJumD901\njRosiU3H0dxyo9uaFBOIV58bAZlM/2u1tKwOC1ftR2p6iUn7TERE3R8D9Fb8nFqIzHL96Le5VIKZ\nEb4Gj36Xlddj0ycnxeOxI3thRJRPKzWIqDP0d7XB2gkhcGpKmarSarE6LgO7LntyZqgRUT5Y8Mpo\nWFjIAOinuC1ZewCnzynaqElERNSMAfo1FNWq8MW5fPH4X2GecLc2N6iuTqfDpk/iUdW0oZGzkxWe\nfHhwh/STiNrP38ES6yaGwMtG/zOu1enw3olL+C5JYfQmROH93bFs7ljYNrXV0KDGyn8fRtyxnDZq\nEhER6TFAv4YPEnJQ37SJSS97S/wzxN3guvsPZeHEqebg/rnpkbCxlpu8j0RkOh425lg3MQR9HJsz\nNG09k4ctiXnQGhmkB/V2xqqFMXB20rel0Wjx7//+hT/3XTRpn4mIqHtigH4Vx/LKceSyOajPRvpB\nZmDO8sLiGnz8ZaJ4PHliH4SHebRSg4i6CgcLM6wZH4yBbrZi2Q8XlHjn2CWotcYF6T5edli9MAbe\nnn/vFqzD5q0nsf3nJKNH5YmIqGdhgP4/6tUabDrZ/Ch6UqAL+rvaGFRXq9XhPx+eEDcq8XC3wSOX\npV8joq7PykyKpWP6YKSvo1i2L6sEKw5dFJ+qGcrVxRorF4xD7wAnseyr7eew9evT0BoZ8BMRUc/B\nAP1/fH1egcJaFQDAzlyGx8K9Da67c086ziUXAtCnXnt+xlBxsRgR3TzMZRLMHRGAyb1dxbL4ggos\nik1DZdOeCIayt7PAstfHYkCom1j2yx+pWP/hcaiNDPiJiKhnYIB+mazyOvyYohSPHw/3hr25YQF2\nbn4lvth2Rjy+a2pf9A1yMXkfiejGkEoEPBvpiwdCmzcmSyquxuv7UlHc9CHeUFaWZljwymgMj2zO\n5HQg7hLeXH8EKpXGZH0mIqLugQF6E61Ohw3x2VA3zQ3t72qDiQHOBtXVaLR4/4PjUDXq/9D6+zng\n/rtDO6yvRHRjCIKARwZ6YeYQX7HsUkUdXt1zAbmV9Ua1JTeT4pVnh2Pi2ECxLP5UPpa9eRA1Rgb8\nRETUvTFAb7InswTni6sBADJBwDMRfpAYmPP8+19SkJ5RCgCQSiV4fsZQmMkM322UiLq2O4LdMGd4\nAGRNvxOKalWYs/cCUktqjGpHKpXg6ScicNftfcWy5NQiLFodi7Jy4wJ+IiLqvhigA6hoUOOT03ni\n8V193eHvYGlQ3YtZZfjupyTx+MF7wuDv52DyPhJR5xrn74RFY3rDXKr/tVnZoMb8/WlIVFQa1Y4g\nCHjkvoF49IFwsSwruxzzV+6DorDapH0mIqKbEwN0AFtP54kLv9ys5Higv2cbNfRUKg3e33wMGo1+\noVffIBfcOTm4w/pJRJ0r0tMea8YHw1auX5tSp9Zg6cF0HMouM7qtO6eE4LnpUZA0pXBVFlZjwYr9\nuJRT3kZNIiLq7np8gH6usAq7Mpq39J4V4QsLmWFflq+/P4ecPP3omVwuw+ynoiCV9vgvKVG3FuJs\njbUTguFqpd98rFGrw9ojmfgtrcjotsaPCcBrs6PFKXFlFXVYuCoWyanFbdQkIqLurEdHk2qtDhsu\ny3ke7eOAYd6GTU9JulCEn35PFY8fe3AgPD1sW6lBRN1FL3tLvDkhBD62FgAAHXTYcDIbX50rMHoT\noqER3lg0ZzQsLcwAADW1Kix78yASTheYvN9ERHRz6NEB+o8XlLhUUQcAsJBJMOOyTA2tqa1rxPoP\njgPQ/yEeNMADt47v3VHdJKIuyM1ajjcnhiDYyVos+/JcPjYn5EJrZJAe1s8Ny+ePg52tOQBApVJj\nzbtxOHQ026R9JiKim0OPDdAV1Q346lzzCNXDYV7iI+u2fPrNaSiL9NkbrK3kePbJSAgGZnwhou7D\n3lyG1TFBGOxuJ5b9klaIt45moVFj3CZEvf0dsWphDFycrQDo07e+s/EYdu5OM2WXiYjoJtAjA3Sd\nTofNCTloaPoDGuhghTuC3dqopXfydAF2788Qj6c/MhjOTlYd0k8i6voszaRYMqY3xvg5imUHskux\n/NBF1DUatwmRt6cdVi8cD1/vvwN+HT76/BS+2XHe6KkzRER08+qRAfpfeRU4nl8BABCg3y1QKml7\nBLyqugEbPooXj0dE+WBMtF+H9ZOIbg5mUgnmjAjA7UGuYlmCohILYtNQ0ZQhylAuzlZYuSAGQYHN\nG6V9++N5bPniFLRaBulERD1BjwvQ6xo12JzQvDD0tt7O6OtiY1DdDz5NQFnTnHUHewvMfCyCU1uI\nCAAgEQTMGuKLh8K8xLILJTV4bc8FFNYYt1OorY05lr4+BuFhHmLZzt3peHfTMajVxk2dISKim0+P\nC9C/PFeAoqZtte3NZfi/gd4G1Tv8VzbijjUH9k8/Hiku6CIiAvSbEP0rzBPPRPhBgP7De25VPV7b\newHZTR/uDWVpYYb5L49E9NDmxeuH/8rGG+/GocHIUXkiIrq59KgAPbO8Fj+nForH0wf7wM5c1ma9\n0rI6fPBpgng8YUwAooZ4tVKDiHqyqUGumBsdAFnTE7aiWhVe25uKCyU1RrVjJpPipaeHYVJMc5ao\nhDMFWPbmQVRXGzcqT0REN48eE6BrdTr8Nz4HmqaFVgPcbBHTy6nNejqdDhu2xKO66RG1q7M1Hn9o\nUIf2lYhufqP9HLFsbB9YNm1CVKVSY96+VJwsqDCqHalUgpmPDcG9d4aKZSlpxVi4ej9Ky40blSci\noptDjwnQd2WUILm4GgAgEwQ8G+ln0PzxXfszkHCmOR3j7BlRsLI067B+ElH3McjDDqtjgsQndQ0a\nLZYfvIgDl0qNakcQBDx4TxieuGxwIDu3AgtW7INCWW3SPhMRUefrEQF6eX0jtp7OE4+n9fOAr51F\nm/UUhdXY+vVp8fgftwUjrJ9h6RiJiAAg2Nka6yaEiPssqHU6vHk0s8V0O0Pdfmswnp85FJKmrFPK\nohrMX7kPmZfKTdpnIiLqXD0iQN+SmIcqlX5RlaeNOe4L9Wijhn6TkPc/OC4uxvLxssND0wZ0aD+J\nqHvysbPAWxND0MveUizbnJCDz8/mG53ffNxIf8x9YSTkZvqpM+UV9Vi0ej+SLhSZtM9ERNR5un2A\nflpZhX1ZJeLxrAhfmMvaftu//JmGlNRiAIBEIuD5GUMhl0s7rJ9E1L25WMnxxvhg9Lssres35wv0\na2OMzG8eNdgLi18bI063q61rxPI3D+LEqXyT9pmIiDpHtw7QGzVabIjPFo9H+Toi0tO+zXrZuRX4\navtZ8XjaHaHoE9j2glIiotbYmcuwclxQi99Dv18swtqjmVBpjMtvHhriihXzY+Bgr5+up2rUYO17\ncYiNyzJll4mIqBMYFKBv2LABAQEBsLS0RGRkJA4fPnzNcxsaGvDYY48hPDwccrkcMTExVz3vwIED\niIiIgKWlJXr37o3Nmzdf3ztoxY4UJXKr6gEAVjIpnhrs02adRrUG728+Lm4GEujviGl39DN534io\nZ7KQSbBodG+M92/+0B+XU4alB9NR26gxqq2AXg5YvXA83F2tAQBarQ7vbz6OX/9MNWmfiYjoxmoz\nQN+2bRtefPFFLFy4EImJiYiOjsbkyZORk5Nz1fM1Gg0sLS0xe/ZsTJ069aqZUjIzMzFlyhSMGjUK\niYmJmDdvHmbPno0dO3a0/x01UVQ34JskhXj8yEAvuDQt0mrN9p+SkXGpDIA+B/HzM4ZCZsCUGCIi\nQ8kkAl4a5o9/BjcvOj+trMK8fakor280qi0PdxusWjQefj7No/Iff5mIr7afM3p+OxERdQ1tRp5v\nv/02Hn/8cTz55JMICQnB+++/D09PT2zcuPGq51tZWWHjxo2YPn06vL29r/oHYtOmTfDx8cF7772H\nkNI7AwwAACAASURBVJAQTJ8+HY8++ijeeuut9r8jNOUuP5kjPjLu42iFqX1c26yXdrEE3/+SLB4/\ndG9Yiz96RESmIhEETB/sg8cu2804vawWr+1NhaK6wai2nBwssXJ+DPoGuYhl239OwuatCdAYOXWG\niIg6X6sBukqlQkJCAiZNmtSifNKkSThy5Mh1X/To0aNXbTM+Ph4ajXGPeK8mLrdc3AxEgIBnIv0g\nlbSe87yhQY33PzgBbdNirf59XXH7rcHt7gsR0bUIgoB7Qz3wfFQvSJqeNuZV1WPOngvIMnITIhsb\nORbPGYMhAz3Fsl37L+KdjcfQqG7/71UiIrpxWt3nvri4GBqNBu7u7i3K3dzcoFAorlGrbUql8oo2\n3d3doVarUVxcfMVrABAfH29Q2/UaHdZdaEBtoz7QjnaRoiozGfGZrdf7dXcBUtP12V7M5RKMHmqO\nhISTBl2TugZD7xHq2brifeIM4B4XDb7MVkGtBWprgGd+jMfjAXIEWBuXPeqW0RaorDDD6ST9IMXu\n/ReQmZWHf93tC3NmojJYV7xPqOvhfULXEhQU1K763W5y9Z8KNSqbgnNbMwG3ebS96+fFrGocjW9O\nxThlogecHNqer05EZCph9lJMD5DDvCmGrtMAH2SokFRp3Oi3VCpg2j98MDyieRFqemY1Pvk6C7V1\nahP2mIiIOkqrI+guLi6QSqVQKpUtypVKJTw9Pa9Rq20eHh5XjMArlUrIZDK4uLhctU5kZGSb7aaX\n1iLxYgqsrPXB9SsjAjC6V+vpEWtqVfjgy12wtrYCAESEe2LmE6OuuriVuqa/RzAMuUeo57oZ7pNI\nAJFltVgcm47yBv1i0e1FAl4I6IUJAc5GtRUVpcO3PyZh2w/nAQClFcD3O8uxeM4YODtZmbrr3cbN\ncJ9Q5+N9Qm2pqKhoV/1WR9DlcjkiIiKwa9euFuW7d+9GdHT0dV90xIgR2L179xVtRkVFQSq9vkew\nGq0O/43PhrZpUepgdzuM8XNss97HXyaiuKQWAGBrY45nnoxicE5Enaa3oxXWTQyGh7U5AECj0+Ht\nY1n4IUXZRs2WBEHA/Xf1x/RHBgPQ/07LyavE/BX7kVdQaepuExGRCbU5xeXll1/G1q1bsWXLFiQn\nJ+OFF16AQqHArFmzgP9v787jo6ru/4+/7mzJZJuE7HtYEvY1YUsQUBZFvvp1pa2/qrUqWlFZ6lIQ\nLVYEl9YKKLjW2sWvWGvVFqxE2ReBAAFNIAQCIetkIWSfTGb5/THJJDEhCQmQCXyej0cezD1z750z\neh+Td8587jnA4sWLmT59eotj0tPTSU1NpaSkhKqqKg4fPkxqaqrz+Ycffpi8vDwWLlzI0aNHee+9\n9/jwww954oknuvxG/nuyhONnqwHQqVT8KiGyw6C970AeW3acdm7PvXcMfr7uXe6DEEJcDGHe7rw6\nfSB9ffXOtvdSc/ngcN4FT51444xYFv5qPGq14+O+uLSaZ5Zv4eTpsovaZyGEEBdPuyUuAHPmzKG0\ntJTly5dTUFDA8OHD2bhxI5GRkQAUFhaSlZXV4pjZs2eTnZ0NOEZxRo8ejaIozhlaYmJi2LhxIwsX\nLmTdunWEh4ezZs0abr311i69ibLaej48kufcvmNIMOHe7Qft8goT6z5ougn0molRJI2P7NLrCyHE\nxdZHr+Wl6+L43Y6TpBVXAfDp0UIq6iw82omZqZq7ZmIUnh5aXlmzB7PZQkVlHc+t2MrihUkMGxzU\n4fFCCCEuL8XuwitZNK/fMRjOPx/5q3tOsTX7LADh3u6suX4wbu0sLmS323ll9W72HnCEej9fPatW\nXI+Xl9wY2htJLaDojN56ndRZbLyy5xTf5Z1ztk0I9+WpiX3b/Zxry9HjJax4bSfVNWbAsRjbr+dN\nYFx8eAdHXj1663UiLi+5TkRHOpthz6fXz+KSWljhDOcAj8RHdvhLa9vubGc4B5h3f4KEcyGES3LT\nqFiS1I/pzW4S/S7vHM9tO0G1+cJmeBkcF8DyZ6biZ3CUztRbrLy8ejd/WX8E8wWeSwghxKXTqwN6\nncXG2gM5zu0pUX0YFeLT7jElpTW895dDzu2Z1/ZnzMiuz0gjhBCXmlqlsGBcNHcMDnG2/VBcyW82\nH6estv6CzhUd6cuLz15LcJAX4PhG8fMNx1i0dBNHj5dc1H4LIYToml4d0D89VkhepQkAT62aB0dH\ntLu/zWbnjff2U9PwCy04yIt7fzbikvdTCCG6S1EU7hsZzi9HNn3OZZ2r4clvMyiorLugc4UEebFi\n6XUMH9JUf55fWMkzy7fw/l8PYTLJfOlCCNGTem1Az6s08Wl607Rj94wIx0/f/qJE//32BEfSGo9R\neHzuOPTuHS9kJIQQruL2wcEsHB+DumGWqoKqOp78NoOTZTUXdB4/X3eWPT2Fh34R3+xz0M6G5EwW\nPvN1s89KIYQQl1uvDOh2u511KTmYbTYA4vp4Mqt/2wscNcorqOAv6793bt8yeyCD49o/RgghXNH0\nvv48M6kfuoapE8tM9SzefJwjRZUXdB5FUbj+uv6sWnk9Y0Y0lfoZi6tZ9vI21n2Q4ryhVAghxOXT\nKwP6jjNlHDI6FtpQKQrzOphyzGq1sead/ZjNjq9toyIM/PS2oZelr0IIcSmMD/dl+dRYPLWOxd2q\n6638dusJ9uSe6+DI1gL8PXjm15N4/KFxeHk23TCfvCWLBUs2ceBwwUXrtxBCiI71uoBeZbbw7qFc\n5/ZNsYEM6GDZ6n9tOMbxk6UAqNUqHp87Dp22ayuWCiGEqxga6MXL0+Lo01CiYrbZWLEri01ZF36z\np6IoTE2KYdXKGxjfbNrF0rM1vPiHHax6ey+VVRdW6y6EEKJrel1A/+v3+Zw1OW7y9Ndr+fnwsHb3\nzzpdxvp/pTu3f3LrEPrF+F3SPgohxOXS19eDV6cPJMzLDQCb3c6qfdn842jhBa86Co7a9KceT+TX\n8ybi4+3mbN+2K5v5i7/mu5Tcdo4WQghxMfSqgJ5RWs2GzKaRobljIvFoZyTcXG9l9Tv7sFobatX7\n+3Pr7EGXvJ9CCHE5hXi58er0gQzwa/o28c+H83g/NQ9bF0K6oigkjY9k9Us3cM3EKGf7uXITr6ze\nzatr9nCu3HRR+i6EEKK1XhPQrTY7a1POYMfxyyY+1EBShG+7x3z8WRpnch0rOel0Gh6bOxa1ute8\nZSGE6DRfdy0rr4tjRJC3s+1fGUb+uDcbi61rC0b7eLux8FcTWLxgknNxI4A9+3OYv/hrtu/O7tIo\nvRBCiPb1mrS64UQxJxqmEdOpVfxqTCSKcv4bQ48eL+GLjRnO7Xt+Mpzw0PYXMRJCiN7MQ6tm2eQB\nJDYbvNh8upTlO09isti6fN6xY8JY9dL1TJvc19lWWVXH62/tZeXruyg9e2FTPAohhGhfrwnofz2S\n73z80yEhhDarjfyxWlM9q9/Z5xzZGTEkmBumDbjkfRRCiJ7mplHxm8R+Laae3Z9fzrNbM6ms6/oC\nRF6eOuY9MJbnnpxMgH9TKU3KoXzmL/6ab7edktF0IYS4SHpNQK+xWAGI8Hbn1kHB7e77l4+PYCyq\nAsBDr+XRB8eiamcaRiGEuJKoVY7pZ386pGlu8/SSKn6z+Tgl3ZzXfNTwEF5fcX2LQY+a2nrefH8/\nz7+ynaKS6m6dXwghRC8K6I3mjY1yLs7RlkNHCvl680nn9v13j24x2iOEEFcDRVG4e0QYD42JdLad\nLq/lyW8yyK3o3g2eHnotc+8dw+8WTyU4yMvZfiTNyIIlX/PVNyewdbHuXQghRC8L6NfF+Le4AerH\nqqrMvPHefuf2+PhwpiZFX46uCSGES7o5LognJ/RF03DPTlGNmSe/zeB4afdHuocNDuL1F2dy0w1x\ngOP8JpOFd/9ykOdWbqWg8MJWNhVCCOHQawK6t07D/aPC293n3b8epOxcLeCYfeDh++LbvZFUCCGu\nBlNj+vDs5P64NXz7WFFnYcmWTFILK7p9bjc3DffdNYqVz11HRFjTjfjpGcUsXJrMl19lOKe6FUII\n0Tm9JqDfOyIM34bV8tqya28OO/accW4/8ssEDD7ul6NrQgjh8hJCDay4NhZvnQaAWouVZdtPsDOn\n7KKcf+AAf37/wgzuuHmI854fs9nCn//vMEte2OKc8lYIIUTHek1Av77ZjAQ/VnbOxDsfHnRuT50U\nw7j49kfbhRDiajMowIuXp8UR6KEDoN5mZ+WuLF777jSl3bx5FECnVXPXHcN4Zdl0YqKapnrMzCrl\nieeS+fTLdCzdmO5RCCGuFr0moKvOU6pit9tZ96cUKqvqAAjw9+D+n4+6nF0TQoheI9qg55VpA4nw\nbvqG8dvTpczdmMb6tALqLkKA7hfjxyvLpvOz24c5F4ezWGx89OkPPL3sG7JOX5xReyGEuFL1moB+\nPt9uP0VKatMc6Y8+MBbPhtEhIYQQrQV56nhl+kAmhDeNcpssNv7yfT6/+iqdnTll3Z7TXKNRcef/\nDuEPL8xgQL8+zvZTZ87x9PPf8tGnP2Cut3brNYQQ4krVqwO6sbiaP/091bk9e0YsI4a2P0e6EEII\nMLhpePaa/rw4NZZog97ZbqyuY+WuLH6zOZOTZd1fITQqwsDKZ6/jnp+ORKdVA2C12vj0y3SefC6Z\n4ydKu/0aQghxpem1Ad1ms/PGu/swmRwr44WFePPzOcN7uFdCCNG7jArxYc31g3kkPgofN42z/Yfi\nSuZ/fYxV+7Ipq63v1muo1SpuuXEgf1g+g0FxTfcT5eRVsPiFzXz4f4cxm2U0XQghGvXagL5hUyZp\nx4oBx4Icj88dh1uzXy5CCCE6R61SmB0byDs3DuXWgcHOOdPt2NmUVcLcDWn842gh5m5Olxge6sPy\nJdfywN2jnZ/XdrudL77KYOEzm0jPKO72exFCiCtBrwzoZ3LL+dsn3zu3b79pMHED/HuwR0II0ft5\nu2l4YHQEb84awtgwg7O9xmLlz4fzeOSrdHZ3sz5dpVK4cUYsr6+4vkVJYoGxkqUvbuG9vx6k1tS9\nEXshhOjtel1At1hsrH5nH/UWx9eh/aL9uPOWwT3cKyGEuHJE+LizbPIAfjdlAJHN1pMoqKrjxV1Z\nLNmSSVY369ODAz357VOTeeT+BDz0TWtcbEw+wYLFmzicZuzW+YUQojfrdQH90y+POqfo0mhUPP7Q\nOLQadQ/3SgghrjzxoQbeuGEID4+JdC5wBHCkqJL5m47xxv5sznVjtFtRFKZP6cfrK64nYVSYs724\ntJrnX97G2vdTqL4I87MLIURv06sC+omss3z6Zbpz+2e3DyMqwtDOEUIIIbpDo1K4KS6Id2YP5abY\nINQN9ek2u52vTjrq0/951Eh9N+rTA/w9WLwwifkPjcfLs2ma3G+2ZTF/8dfsP5TfztFCCHHl6TUB\n3Wy2svqdfdhsjtrHQXEB3HxDXA/3Sgghrg4+bhoejo9kzQ2DiQ/xcbZX11v50+FcHvkqnb1557pc\nn64oClOSolm18gYmjo1wtp8tq2XlH3fy+rq9zgXphBDiStdrAvrfP/2e3PwKANzcNDw+d5xzhToh\nhBCXR7RBz/NTBrBs8oAWq5HmV9Xxux0neXbrCU6fq+3y+f183XnysUSeeHQihmb179v3ZPP4b75m\n976cbvVfCCF6g16TcP/930zn41/8bCQhQV492BshhLh6KYrC2DADb84awtzRkXhpm+4DOmSs4LGv\nj/JmyhnK6yxdfo3EcZGsful6piRGO9vKK0z8/o09vLpmN2XnTN16D0II4cp6TUAHx9emY0aEMvPa\nfj3cFyGEEBqVwv8ODOKd/xnG7AGBqJrVp288Uczc//zA5xldr0/39nJj/sPjWbJoEn38mlY73bM/\nl/mL/8vWXae7NeWjEEK4ql4U0MHLU8cj9yegNPwSEEII0fMMbhoeSYhizfWDGR3cVJ9eVW/l3UO5\nPPrfo+zLL+9ymE4YFcaqldczY2rT4ExVtZnVb+9jxWs7KSnt3pSPQgjhanpVQJ9775gWoyhCCCFc\nR4yvnhemDuC5a/oT5uXmbM+tNPH89hP8dtsJssu7Vp/u6aHjV79M4LdPTyEowNPZfuBwAQuWfE3y\n1iwZTRdCXDF6TUBPHBfJpAlRPd0NIYQQ7VAUhfHhvqydNYT7R0Xg2aw+/UBhBY/99yhvHcihoov1\n6SOHBvPHFTO5ccYAZ1tNbT3r/pTC8y9vp7CoqtvvQQghelqvCegP/WJMT3dBCCFEJ2nVKm4bFMw7\ns4cyq39TfbrVbuffmUXM3ZDGl8eLsNgufNRb767lgbvHsPyZawkN9na2H0k3svCZTWzYlOmcklcI\nIXqjXhPQvZt9XSqEEKJ38HXX8ujYKFZfP4gRQU1hutJs4e2DOTz633RSCsq7dO4hAwP544szueXG\ngc57k+rqLLz/t0MsXbGFvIKKi/IehBDicutUQF+7di19+/ZFr9eTkJDAzp07293/+++/Z8qUKXh4\neBAREcELL7zQ4vmtW7eiUqla/Rw/frzr70QIIYTL6uvrwYprY3kmqR+hzQZccipM/Haboz49t+LC\np07U6dTc89ORvPTcdUSGN60sfex4Cb9emsy/NhzD2o1VToUQoid0GNDXr1/PggULWLp0KampqSQm\nJjJr1ixyctpeLKKiooIZM2YQGhpKSkoKq1at4tVXX+W1115rtW96ejqFhYXOnwEDBrRxRiGEEFcC\nRVFIjPRj7awh3DcyHA9NU316SkE5875K592DOVR2oT49tr8/r/5uOnNuGeJcxM5cb+Wv64+w+IXN\nnMnt2ii9EEL0hA4D+muvvcZ9993H/fffz8CBA1m9ejWhoaGsW7euzf3//ve/YzKZ+PDDDxkyZAi3\n3347Tz/9dJsBPTAwkKCgIOePStVrKm6EEEJ0kU6t4o7BIbwzeygz+wWg4ChPsdjtfH68iLkb0/hP\nZjHWC6wj12nV/PS2Ybz822n0i/Zztp/IOsuvn03mk8/TsVhkNF0I4fraTcRms5mDBw8yc+bMFu0z\nZ85k9+7dbR6zZ88errnmGtzc3Frsn5+fT3Z2dot9ExISCAsLY/r06WzdurWLb0EIIURv5KfXMn9c\nNKuuH8SwwKb69Io6C+sOnOHxr4+SWnjhdeT9Yvx46bfT+H93DEejcfyas1ptfPzZDzy17BtOni67\naO9BCCEuhXYDeklJCVarleDg4BbtQUFBFBYWtnlMYWFhq/0btxuPCQsL46233uKzzz7js88+Y+DA\ngUybNq3D2nYhhBBXnv5+Hrx0XSyLk/oR7Nk0uHO6vJZntmbyu+0nyKu8sPp0jUbF7TcP5g8vzCSu\nv3/TOc+c4+ll3/D3f3yP2Wy9aO9BCCEuJs3FPmFnVvmMi4sjLi7OuT1hwgROnz7Nq6++yqRJk9o8\nJiUl5aL1UVyZ5BoRnSHXietyBx4MtrO92MLmIgvmhmqULZk1bDtRwKQADdODNejVF7aa9G2zfNiT\nUk/ydiP19Y6ymb98fICvvvmB224MJyrco9Uxcp2IzpDrRJxPbGxst45vdwQ9ICAAtVqN0Whs0W40\nGgkNDW3zmJCQkFaj643Hh4SEnPe1xo0bR2ZmZqc6LYQQ4sqkVSlMC9by9CB3Evo03URqs8P2Ygsv\nHzPxXakF6wWsGqpSKSSNC+CxXw6gb1TTKqTFJXW889csNn5bgLleatOFEK6j3RF0nU5HfHw8mzZt\n4vbbb3e2Jycnc+edd7Z5zMSJE3n66aepq6tz1qEnJycTHh5OdHT0eV8rNTWVsLCw8z6fkJDQ7hsR\nV6/GEQy5RkR75Drpfa4DjpdW896hXNJKHCuE2oGvyuCYXc+DoyMZ2Wyhos6YMd3Opi0n+cv6I5hM\njtliUtNqKSguYd79CZiqzwBynYj2yeeJ6Eh5efdmjupw2pRFixbx5z//mffff5+jR48yf/58CgsL\nefjhhwFYvHgx06dPd+5/11134eHhwS9+8QvS0tL47LPPePnll1m0aJFzn9dff50vvviCzMxM0tLS\nWLx4MV988QWPPvpot96MEEKIK0ucvycvT4vj6Yl9CfTQOdtPnatlyZbjvLjzJAWVdZ0+n0qlcMO0\nAby+4npGDmv6VtdYVMVzK7fy5df5mOqkNl0I0bM6rEGfM2cOpaWlLF++nIKCAoYPH87GjRuJjIwE\nHDd+ZmVlOff38fEhOTmZefPmkZCQQJ8+fXjiiSdYuHChc5/6+nqefPJJcnNz0ev1DBs2jI0bN3LD\nDTdcgrcohBCiN1MUhcnRfRgf7stnx4z842ghdQ2LD+3OPcf+/HL+d2AwPxkSgodW3cHZHIICPHnu\nyWvYvP00H3yUSk1tPQB7D54lLaOczGw3piRG0zfat1P3VgkhxMWk2O0XUMh3mTX/esBgMLSzp7ia\nyVeNojPkOrlylNSY+fBIPptPl7Zo93XTcs+IMKb39Uet6nyoPltWy9t/PsD+Q/lUV9cA4OnpuHE0\nMtzAlKRoJk+MIsC/9c2k4uoknyeiI93NsOply5Ytu4j9uajq6pq+tnR3d+/BnghXlp+fD9DuPQxC\nyHVy5fDQqkmM8CUh1EB2eS0lDaPfJquNvfnl7M8vJ8LHvcWUje3R67VMmhBJRJgPqUdyqDPb0Om0\nAFRU1nEkzci/v84kPaMYu81OcJAX2k6O1Isrk3yeiI50N8NKQBe9nnxQis6Q6+TKE+ChY2Y/f8K9\n3ck8W0NNvaN2/Kypnm9OlZJdbiK2jwdeuo5nFFYUhagIAzHhdURFeBAYEEiBsRqrtWl2l6LiavYd\nzOc/m05wJrccnU5NcKAnqgsYrRdXBvk8ER3pboa96POgCyGEEJeLoihMjenDhAgD/zxm5J9Hjc76\n9F05ZezPK+eWgUHc2cn6dJVKIa6fN3fNSWCuqZ69B/LYujOb79OLaKwINZst7PzuDDu/O4PBx51J\nEyKZkhRN/xg/qVcXQlwUEtCFEEL0eu4aNf9vWBgz+wXw58N5bM0+C4DZZuOTo4V8c6qUe0aEMa2v\nP6pOhmi9u5apSTFMTYrhbFkt2/ecYduubLJzzjn3Ka8wsWFTJhs2ZRIR5sOUxGgmJ0YRGODZzpmF\nEKJ9UuIiej35qlF0hlwnVwdPrZqkSD/GhPhw+pyJ0ob69FqLje/yHPXpUQY9QZ66No8/33Wi12sZ\nFBvADdP6Mz4+HHc3DUUlNc651MFRr/59ehH/+TqTH44WYbPZCQ7yRCf16lcc+TwRHZEadHHVkw9K\n0RlynVxdAhvq00O93MgorabW4ih7OWuqJ/lUKbkVbdend+Y68TW4M2p4CP8zM5YhgwIBKCxqWa9e\nXFLD/kP5bPg6k9M55Wi1Uq9+JZHPE9ERqUEXQggh2qBSFKb19ScxwpdPjxr5LMOIuSFEbz9Txnd5\n5dw2MJg7hwTjrrnwUW61WsXIocGMHBrMQ/da2Hsgj227szn8g7GpXr3eyu59Oezel4OPt5ujXj0x\nmgH9+ki9uhDivCSgCyGEuKLptWruHhHGzP7+fJCax46cMgDMVhsfpxeQfKqEX4wMZ2p0ny6/hru7\nhilJ0UxJiubsuVp2NtSrnzrTVK9eUVnHxuQTbEw+QXioD5MTo5icGE1woNSrCyFakoWKRK8nC0aI\nzpDrRDRKK67inYM5nCiradE+0N+TJF0F0Z6qi3adnMktZ+uubLbvzuZsWW2b+wyOC2RKUjSJ4yLw\nOk9tvHAt8nkiOiILFYmrntQCis6Q60Q0CvLUcX3/AII8dWSW1jjr00tr69llNJFXa8PNwxM/dy36\nbt7gafBxZ+SwYGbPjGXooEAUlUJhURUWS1O9eklpDSmp+fzn60xOnzmHVqMiKMgTtUrVrdcWl458\nnoiOSA26EEIIcYFUisLMfgFMivTjH+mFfJ5RhNnmCM3pFTZO7z0NwAA/D+JDfYgPNTDI3xN1F2/y\nVKtVjBgazIihwcy9Zwz7DuazbVc2qT8UYrM5vsiut1jZsz+XPftz8fZyI2m8Y371uP5Sry7E1UZK\nXESvJ181is6Q60S0p7Cqjj8dziP5WB4AHp4erfbx0qoZFeJDfKgPCaEG+ui13X7dsnMmdu09w9Zd\n2WSdLmtzn9BgbyYnRjElKZqQIK9uv6boPvk8ER3pboaVgC56PfmgFJ0h14nojP/s3MfRCiul7v4c\nLa7C0s6vyP5+HsQ3BPbBAV5dHl1vdCa3nO27z7B9TzYlpTVt7jMoLoApiY56dW8vt269nug6+TwR\nHeluhpUSFyGEEKJBiLuKEHcVCQlxVJutpBorOFDg+CmpNbfY92RZDSfLavjkaCGeDaPrCSE+jAn1\nIcDjwm/2jIow8PM5w7nrjmGkZxSzbVc2e/bnUtOw2BLAseMlHDtewvt/O0T8yFCmJEUzZmSoLIYk\nxBVGAroQQgjRBk+dY1XSpEg/7HY72eUmUgrKSSmoaDW6Xl1vZVdOGbsapnDs66snPtRAfKgPQwK8\n0FzA6LpKpTBscBDDBgfx4D1j2Hcwj+27z3Do+0LnYkgWi429B/LYeyAPL08dieMc9eqDYv2lXl2I\nK4AEdCGEEKIDiqIQ46snxlfPHYNDqKm3cthYyYGGwF5c03J0/dS5Wk6dq+XThtH1kcHeJIQaGBPq\nQ+AFjK7rdGomTYhi0oQoyitM7Pwuh227szmRdda5T1W1mU1bTrJpy0mCg7yYkhjFlMRoQkO8L9r7\nF0JcXhLQhRBCiAvkoVUzMcKXiRG+ztH1AwUVHCgsJ62o9ej67txz7M51LFoUY9A7Z4YZEuCJVt25\n6RQNPu7MnhnL7Jmx5OZXsH23YzGk4tJq5z7Goio++TydTz5PJ66/P1OSopk0IVLq1YXoZSSgCyGE\nEN3QfHT99sHB1NRbOWKsJKWgggMF5RT9aHT9dHktp8tr+ecxI3qNmlHB3o6ZYcIMnR5djwjz4a47\nhvHT24Zy9HgJ23Zls3tfTot69eMnSzl+spQ//T2VMSNCmDophvhRUq8uRG8gAV0IIYS4iDy0FsT7\newAAE5lJREFUaiZE+DKhYXQ9p8LkvNH0h+JK6m1No+u1Fit78s6xJ88xuh7lo2+YxtGHIYFe6DoY\nXVepFIYOCmTooEAeuHs0KamO+dUPHmmqV7dabew/lM/+Q/l4euhIHBfRUK8egKqbM88IIS4NCehC\nCCHEJaIoClEGPVEGPbcOCqa23sqRosbR9QqM1XUt9j9TUcuZilr+lWHEXaNiVLBPQzmMD8Ge7Zep\n6HRqEsdFkjgukorKOnZ+d4Ztu86QmVXq3Ke6xkzy1iySt2YRFODJ5MRopiRFER7qc0nevxCiaySg\nCyGEEJeJXqtmfLgv48Mdo+u5lXUcKCh3jK4XVTlXMwUwWWx8l3eO7xpG1yN93EkINXRqdN3H240b\nZ8Ry44xY8goc9erbd2djLG6qVy8qqebTL9P59Mt0Yvv5MyUpiqTxkRh8LnxZciHExSUBXQghhOgB\niqIQ6eNOpI87twwMxmSxcqSoyhnYC6pajq7nVJjIqTDxrwwjbmoVI4O9iW8I7CHt3AQaHurDz253\n1Ksfyyxl265sdu3NobpZbXxmVimZWaV88NFhRg8PYUpSNGNHh6HTSb26ED1BAroQQgjhAtw1asaF\nGRgXZsBut5NXWcfBwgpSCir4vqgSs7VpdL3OamNffjn78h2rFUZ4u5PQMDPM0EAv3DStR9cVRWFw\nXACD4wL45c9HcSC1gG27sjlwuKBFvXpKaj4pqfl46LVMHOuoVx8yMFDq1YW4jCSgCyGEEC5GURQi\nfNyJ8HHn5rggTBYbPxRXkpLvmBkm/0ej67mVJnIrTXx+vAg3tYrhQd4khPqQEGog1Lv16LpOq2bi\n2Agmjo2gsqqOXXtz2LYrm4wTTfXqNbX1fLv9FN9uP4XeXUtEmA+R4S1/Avw9ZGEkIS4BCehCCCGE\ni3PXqBrqzw1AJPmVJueNpkfaGF13rHhaDuQQ7u3uvNF0eKB3q9F1by83bpg2gBumDaDQWMXWXdls\n252NsajKuU+tqd5ZBtNcW8E9KsKAfx+9BHchukECuhBCCNHLhHm7c7O3Y3S9rmF0vXEqx9xKU4t9\n8ypN5FWa+PJ4ETq1ihFB3s7AHu7d8obQkGAvfnrbUH5y6xAyTjjq1ffsz6WisuWIfaOOgntUhI8z\nwEtwF6LzJKALIYQQvZibRkV8qIH4UAMAhVV17M8v50BhBUeMldQ1G103txhdh1AvNxJCDY7R9SBv\n3BtG1xVFYVBsAINiA5h77xjOlddxJq+cnLwKcvMqOJNXTm5eBVXV5tYdonPBPTLcp+GxBHchfkwC\nuhBCCHEFCfFy46a4IG5qGF1PK67iYGEF+/PLW42uF1TV8e/MIv6d6RhdHx7o1RD2fQj3dkNRFBRF\nwc/XHT9fd0YODXYea7fbKSs3kZNX0fDTFOAvNLh76H9cKmMgMtxHgru4aklAF0IIIa5QbhoVY0J9\nGBPqwwOjIyisqmsohSkntY3R9QOFFRworIBDjtH1MSGOUpiB/p4Y3DQtwrKiKPTx1dPHV9/t4F5T\nW8/xk6UcPynBXQiQgC6EEEJcNUK83JgdG8js2EDMVhvpxVWkFFSQUlBOTkXr0fUNJ4rZcKIYADe1\nikAPHYEeOoI8m/3rqSPIQ4e/XotWrbrg4J6TW9FiTvbmJLiLq5UEdCGEEOIqpFOrGBXiw6gQx+i6\nsbqOgwWOedcPGyuptVhb7F9ntTmnc2yLgkIfvaZFgP9xmPczuF+24B4VYXA+luAuehsJ6EIIIYQg\n2NONWQMCmTUgkHqrjfQSx+j6EWMl+ZV11PwosP+YHTultfWU1tZzrLS6zX08NGqCPHUENIT34MYg\n76kjrK8fwwYHoW5YEKlFcM8tJye/olvBvflNqY2PJbgLVyUBXQghhBAtaNUqRgb7MDLYx9lWbbZS\nVFNHUbWZomozJbX1FFebKaoxU1xtprS2Hjv2ds9bY7FyuryW0+W1bT6vVhT89Vpn2Uzj6HvIiGCG\nT4gkyEOHu0bVpeCecaK0xUJM0BTcG0tkGn/6+ElwFz1LAroQQgghOuSpU9NX50FfX482n6+32iit\nrXcG9uIax09joC+uMbe4KbUtVrudohpH6E87zz7eOk1TCU2AnqAoA8M8dQR6aNHW26gorSGvsVwm\nX4K76J0koAshhBCi27RqFSFeboR4ubX5vN1up8JsdYy6V9dRXFPvCPANgb6o2sy5uvoOX6fSbKHS\nbOFkWU3b/VApjhKaGB+ChgYwxEOLh13BWlVHzdlaKozVFORXcia3nJratl+vo+Cuogofbw0FJcfx\n8tTh5anD01Pr+NfD8a9Op5YwL7pMAroQQgghLjlFUTC4aTC4aRjQp+1R+DqLjZJac1PpTE3T6HtR\ntZmSGjP1tvbLaOptdgqq6iioanv1UzxV+A4LIGh8KN5qNeo6C7bqeurKTFQWV1OaW0FdtZm2onVj\ncK+udvxxsPdQ27X2ABqNCk8PXYvQ3hTidXh5ah3/ejU937iP+4+mtBRXHwnoQgghhHAJbhoV4d7u\nhHu7t/m8zW6n3GRxlMFUtwzwjY8rzZYOX+dcXX3r0XofDfgYoL8BtR08AI3ZBrX1mM/VUVlcja3a\njLbehl0Bpf2/E7BYbJRXmCivaHvWm/ao1So89Nrzjs57ejYL+I3tXo7HencJ91eCTgX0tWvX8uqr\nr1JYWMjQoUN5/fXXmTRp0nn3//7773n00UfZv38/ffr04aGHHuLZZ59tsc+2bdtYtGgR6enphIWF\n8dRTT/HQQw91790IIYQQ4oqlUhT89Fr89FoG+nu2uU9tvdU5+t68Ft5Y3XQzq9Xefrq2KlAJ4KYC\nNzfwdUMV44PdYqOm3kpVVS0qmx2zVgdWO1htYLFhq7dhNVux1lmxW22obPYWP4qdZo/tqGz86HlH\nm9Vqo7KqjsrzfQvQDkVR8PTQNoT41qPzLUbwf7SPh16LSiXh3hV0GNDXr1/PggULWLduHZMmTeLN\nN99k1qxZpKenExkZ2Wr/iooKZsyYwdSpU0lJSeHo0aPcd999eHp6smjRIgBOnTrFjTfeyAMPPMBH\nH33Ejh07eOSRRwgMDOS22267+O9SCCGEEFcFvVZNtEFPtEHf5vNWm52zpnpn3XtjHXxjCU1Rtfm8\nU0qqNSr0GhV2m2P03cOz7dcAsNvBZrVhtdmxNfxYbXbqrfaGNluLdpuz3Y5itbUK8I7tttpa71Np\ns6OqNKEqr211XPvxW2kI681G5z2bjdo3C/4/3sdDr0WtVnXy/5LoiGK3t/9n5Pjx4xk1ahRvv/22\nsy0uLo477riDFStWtNp/3bp1LF68GKPRiJub40aRF198kXXr1pGbmwvA008/zeeff05GRobzuAcf\nfJC0tDR2797tbCsvL3c+NhgMXXyL4kqXkpICQEJCQg/3RLgyuU5EZ8h1IgCqzJYWM9CU1NY7b2wt\nqq4jp8SRTzw8266l7y67HUdwt/4oxDcP+208Z7Xa6SDWdRj8m7YdbT/ebm8fjUqFRqNCp3H8q9Go\nUKtVaJttazQqNOqmx9qGfdRqBa1G3cF+ChqNutljFVqN2vm4+fkaHzva1Wg0irM/GrXqkn9T0N0M\n2+4Iutls5uDBgzz11FMt2mfOnNkiSDe3Z88errnmGmc4b9z/2WefJTs7m+joaPbs2cPMmTNbnfPD\nDz/EarWiVqsv+I0IIYQQQlwMXjoNXjrNeaeU3Lt/P2YbDBk+HJPVhslio87i+NdksWKyNt+2Ndu2\nOrfPd0yd1YaigFqtdCkPNYZ7m82G1WrvVMA3N2u/GBz1+XZnnX7Ttg3FZgMbKOa2nj//ttLwh0fH\nzzm2W56r9T5qRUGtKGjUCmqVgkaloFGrWjzWOp9TodU42jRqBW1DyNc2/PGh0yho1Wo0agVdwx8N\n/zM5rFv/DdsN6CUlJVitVoKDg1u0BwUFUVhY2OYxhYWFREVFtWhrPL6wsJDo6GiMRmOrcwYHB2Ox\nWCgpKWn1nBBCCCGEq1ArCno1+HvoLvq5bXa7M7jXWZsF+BbbTe0/buvomHYXk7LTLMjbnKPy7QV8\naxvh3q4AKA3/Nuptte12x48dsFqh/YV0W7mkAb0rLtWdw82/KhCiudjYWECuEdE+uU5EZ8h1Ijrj\nclwnKkAP6BVA2/CDAqgbfsSVrN1q/oCAANRqNUajsUW70WgkNDS0zWNCQkJaja43Hh8SEtLuPhqN\nhoCAgAt7B0IIIYQQQlxB2g3oOp2O+Ph4Nm3a1KI9OTmZxMTENo+ZOHEiO3bsoK6ursX+4eHhREdH\nO/dJTk5udc6xY8dK/bkQQgghhLiqdTiLyyeffMLdd9/N2rVrSUxM5K233uKDDz4gLS2NyMhIFi9e\nzP79+/nmm28AxzSLAwcOZOrUqSxdupSMjAzuu+8+li1bxsKFCwE4ffo0w4YN48EHH2Tu3Lns2rWL\nefPm8fHHH3Prrbde+ncthBBCCCGEi+qwBn3OnDmUlpayfPlyCgoKGD58OBs3bnTOgV5YWEhWVpZz\nfx8fH5KTk5k3bx4JCQn06dOHJ554whnOAWJiYti4cSMLFy5k3bp1hIeHs2bNGgnnQgghhBDiqtfh\nCLoQQgghhBDi8nHpJZ/Wrl1L37590ev1JCQksHPnzp7uknAhK1euZOzYsRgMBoKCgrj55ptJS0vr\n6W4JF7Zy5UpUKhWPPfZYT3dFuJiCggLuvfdegoKC0Ov1DB06lO3bt/d0t4QLsVgsLFmyhH79+qHX\n6+nXrx/PPvssVusFzr8nrijbt2/n5ptvJiIiApVKxYcffthqn2XLlhEeHo6HhwfXXnst6enpHZ7X\nZQP6+vXrWbBgAUuXLiU1NZXExERmzZpFTk5OT3dNuIht27bx6KOPsmfPHjZv3oxGo2H69OmUlZX1\ndNeEC/ruu+949913GTFixCWbDlb0TufOnSMpKQlFUdi4cSPHjh3jjTfeICgoqKe7JlzIihUrePvt\nt1mzZg0ZGRmsWrWKtWvXsnLlyp7umuhB1dXVjBgxglWrVqHX61v9fnn55Zd57bXXeOONN9i/fz9B\nQUHMmDGDqqqqds/rsiUu48ePZ9SoUbz99tvOtri4OO644w5WrFjRgz0Trqq6uhqDwcAXX3zB7Nmz\ne7o7woWUl5cTHx/P+++/z7Jlyxg+fDirV6/u6W4JF7FkyRJ27NjBjh07erorwoXddNNNBAQE8MEH\nHzjb7r33XsrKyvjyyy97sGfCVXh7e/Pmm29yzz33AGC32wkLC+Pxxx9n8eLFAJhMJoKCgvj973/P\n3Llzz3sulxxBN5vNHDx4kJkzZ7ZonzlzJrt37+6hXglXV1FRgc1mw8/Pr6e7IlzM3LlzufPOO5ky\nZQouOiYhetDnn3/OuHHj+MlPfkJwcDCjR4/mzTff7OluCRcza9YsNm/eTEZGBgDp6els2bKFG2+8\nsYd7JlzVqVOnMBqNLfKsu7s7kydP7jDPXvSVRC+GkpISrFYrwcHBLdqDgoJaLXAkRKP58+czevRo\nJk6c2NNdES7k3XffJSsri48++gi4dKsdi94rKyuLtWvXsmjRIpYsWcKhQ4ec9ynMmzevh3snXMUj\njzxCbm4ugwcPRqPRYLFYWLp0KQ8//HBPd024qMbM2laezc/Pb/dYlwzoQlyoRYsWsXv3bnbu3CkB\nTDhlZGTwzDPPsHPnTuciaHa7XUbRRQs2m41x48bx4osvAjBy5EgyMzN58803JaALp9WrV/PBBx/w\n8ccfM3ToUA4dOsT8+fOJiYnhl7/8ZU93T/QyHWUVlwzoAQEBqNVqjEZji3aj0UhoaGgP9Uq4qoUL\nF/LJJ5+wZcsWYmJiero7woXs2bOHkpIShg4d6myzWq3s2LGDt99+m+rqarRabQ/2ULiCsLAwhgwZ\n0qJt0KBBnDlzpod6JFzRiy++yNKlS5kzZw4AQ4cOJTs7m5UrV0pAF20KCQkBHPk1IiLC2W40Gp3P\nnY9L1qDrdDri4+PZtGlTi/bk5GQSExN7qFfCFc2fP5/169ezefNm4uLiero7wsXceuut/PDDDxw+\nfJjDhw+TmppKQkICP/vZz0hNTZVwLgBISkri2LFjLdqOHz8uf/CLFux2OypVy9ikUqnkGzlxXn37\n9iUkJKRFnjWZTOzcubPDPOuSI+jgKFm4++67GTduHImJibz11lsUFhZKrZdwmjdvHn/729/4/PPP\nMRgMzlovb29vPD09e7h3whUYDAYMBkOLNg8PD/z8/FqNmIqr18KFC0lMTGTFihXMmTOHQ4cOsWbN\nGpk+T7Rwyy238NJLL9G3b1+GDBnCoUOH+OMf/8i9997b010TPai6uprMzEzAUS6XnZ1Namoq/v7+\nREZGsmDBAlasWMGgQYOIjY1l+fLleHt7c9ddd7V/YrsLW7t2rT0mJsbu5uZmT0hIsO/YsaOnuyRc\niKIodpVKZVcUpcXP888/39NdEy5s6tSp9scee6ynuyFczIYNG+wjR460u7u72wcOHGhfs2ZNT3dJ\nuJiqqir7r3/9a3tMTIxdr9fb+/XrZ3/mmWfsdXV1Pd010YO2bNnizB/NM8l9993n3GfZsmX20NBQ\nu7u7u33q1Kn2tLS0Ds/rsvOgCyGEEEIIcTVyyRp0IYQQQgghrlYS0IUQQgghhHAhEtCFEEIIIYRw\nIRLQhRBCCCGEcCES0IUQQgghhHAhEtCFEEIIIYRwIRLQhRBCCCGEcCES0IUQQgghhHAhEtCFEEII\nIYRwIf8fuzynxGGpCUQAAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "goal_dist1 = MakeGoalPmf(suite1)\n", + "goal_dist2 = MakeGoalPmf(suite2)\n", "\n", - "![Distribution of time between goals.](figs/hockey3.pdf)\n", + "thinkplot.Clf()\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(goal_dist1)\n", + "thinkplot.Pmf(goal_dist2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.2: Distribution of goals in a single game.\n", "\n", - "[fig.hockey3]\n", + "Figure 7.2 shows the resulting distribution of goals for the\n", + "Bruins and Canucks. The Bruins are less likely to score 3 goals or fewer\n", + "in the next game, and more likely to score 4 or more." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The probability of winning\n", "\n", "To get the probability of winning, first we compute the distribution of\n", - "the goal differential:\n", - "\n", - " goal_dist1 = MakeGoalPmf(suite1)\n", - " goal_dist2 = MakeGoalPmf(suite2)\n", - " diff = goal_dist1 - goal_dist2\n", - "\n", + "the goal differential:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "goal_dist1 = MakeGoalPmf(suite1)\n", + "goal_dist2 = MakeGoalPmf(suite2)\n", + "diff = goal_dist1 - goal_dist2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "The subtraction operator invokes `Pmf.__sub__`, which enumerates pairs\n", "of values and computes the difference. Subtracting two distributions is\n", "almost the same as adding, which we saw in Section [addends].\n", "\n", "If the goal differential is positive, the Bruins win; if negative, the\n", - "Canucks win; if 0, it’s a tie:\n", - "\n", - " p_win = diff.ProbGreater(0)\n", - " p_loss = diff.ProbLess(0)\n", - " p_tie = diff.Prob(0)\n", - "\n", + "Canucks win; if 0, it’s a tie:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.457996492072\n", + "0.370293408955\n", + "0.171710098973\n" + ] + } + ], + "source": [ + "p_win = diff.ProbGreater(0)\n", + "p_loss = diff.ProbLess(0)\n", + "p_tie = diff.Prob(0)\n", + "print(p_win)\n", + "print(p_loss)\n", + "print(p_tie)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "With the distributions from the previous section, `p_win` is 46%,\n", "`p_loss` is 37%, and `p_tie` is 17%.\n", "\n", "In the event of a tie at the end of “regulation play,” the teams play\n", "overtime periods until one team scores. Since the game ends immediately\n", "when the first goal is scored, this overtime format is known as “sudden\n", - "death.”\n", - "\n", - "Sudden death\n", - "------------\n", + "death.”" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Sudden death\n", "\n", "To compute the probability of winning in a sudden death overtime, the\n", "important statistic is not goals per game, but time until the first\n", "goal. The assumption that goal-scoring is a Poisson process implies that\n", "the time between goals is exponentially distributed.\n", "\n", - "Given lam, we can compute the time between goals like this:\n", - "\n", - " lam = 3.4\n", - " time_dist = thinkbayes.MakeExponentialPmf(lam, high=2, n=101)\n", - "\n", - "high is the upper bound of the distribution. In this case I\n", + "Given `lam`, we can compute the time between goals like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "lam = 3.4\n", + "time_dist = thinkbayes.MakeExponentialPmf(lam, high=2, n=101)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`high` is the upper bound of the distribution. In this case I\n", "chose 2, because the probability of going more than two games without\n", - "scoring is small. n is the number of values in the Pmf.\n", + "scoring is small. `n` is the number of values in the Pmf.\n", "\n", - "If we know lam exactly, that’s all there is to it. But we\n", + "If we know `lam` exactly, that’s all there is to it. But we\n", "don’t; instead we have a posterior distribution of possible values. So\n", "as we did with the distribution of goals, we make a meta-Pmf and compute\n", - "a mixture of Pmfs.\n", - "\n", - " def MakeGoalTimePmf(suite):\n", - " metapmf = thinkbayes.Pmf()\n", - "\n", - " for lam, prob in suite.Items():\n", - " pmf = thinkbayes.MakeExponentialPmf(lam, high=2, n=2001)\n", - " metapmf.Set(pmf, prob)\n", + "a mixture of Pmfs." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeGoalTimePmf(suite):\n", + " metapmf = thinkbayes.Pmf()\n", "\n", - " mix = thinkbayes.MakeMixture(metapmf)\n", - " return mix\n", + " for lam, prob in suite.Items():\n", + " pmf = thinkbayes.MakeExponentialPmf(lam, high=2, n=2001)\n", + " metapmf.Set(pmf, prob)\n", "\n", - "Figure [fig.hockey3] shows the resulting distributions. For time values\n", + " mix = thinkbayes.MakeMixture(metapmf, name=suite.name)\n", + " return mix" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.3 below shows the resulting distributions. For time values\n", "less than one period (one third of a game), the Bruins are more likely\n", "to score. The time until the Canucks score is more likely to be longer.\n", "\n", - "I set the number of values, n, fairly high in order to\n", + "I set the number of values, `n`, fairly high in order to\n", "minimize the number of ties, since it is not possible for both teams to\n", "score simultaneously.\n", "\n", - "Now we compute the probability that the Bruins score first:\n", - "\n", - " time_dist1 = MakeGoalTimePmf(suite1)\n", - " time_dist2 = MakeGoalTimePmf(suite2)\n", - " p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", - "\n", + "Now we compute the probability that the Bruins score first:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "time_dist1 = MakeGoalTimePmf(suite1)\n", + "time_dist2 = MakeGoalTimePmf(suite2)\n", + "p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "For the Bruins, the probability of winning in overtime is 52%.\n", "\n", "Finally, the total probability of winning is the chance of winning at\n", - "the end of regulation play plus the probability of winning in overtime.\n", + "the end of regulation play plus the probability of winning in overtime." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "p_tie = diff.Prob(0)\n", + "p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", "\n", - " p_tie = diff.Prob(0)\n", - " p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", + "p_win = diff.ProbGreater(0) + p_tie * p_overtime" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvwAAAEWCAYAAAAab0P2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VNX9x/H3zCSTTMi+hwSyAQEBBQkCEWSPgCuCCLZW\nXBurKFBafqnUYgEVUQpiQVCsKLZFLWorKMSNXTRiMEAEgYCQZAIJkBDIPvP7IzAQE8JiYLJ8Xs+T\n5/Ge+71nvpNc9TtnzjnXYLfb7YiIiIiISJNkdHYCIiIiIiJy+ajgFxERERFpwlTwi4iIiIg0YSr4\nRURERESaMBX8IiIiIiJNmAp+EREREZEmTAW/iIiIiEgTdkEF//z584mOjsZisRAfH8/69evrjE9P\nT6dv3754eHgQERHBtGnTasSsWbOGbt26YbFYiI2NZeHChdXOv/vuu8THx+Pn54enpyddu3blzTff\n/MW5iYiIiIg0J+ct+JctW8b48eOZMmUKaWlpJCQkMHToUA4cOFBrfGFhIYMHDyYsLIzU1FTmzp3L\nrFmzmD17tiMmMzOTYcOG0bt3b9LS0khOTmbcuHEsX77cERMYGMhTTz3F5s2bSU9P57777uOBBx5g\nxYoVl5ybiIiIiEhzYzjfk3Z79OhBly5dqo3At2vXjpEjR/LMM8/UiF+wYAHJycnk5ubi5uYGwIwZ\nM1iwYAEHDx4EYPLkyXzwwQfs3LnTcd1DDz3E9u3b2bhx4zlz6datG0OGDGHGjBmXlJuIiIiISHNT\n5wh/WVkZW7ZsITExsVp7YmLiOQvzTZs20adPH0exfzo+Ozub/fv3O2Jq6zM1NZXKysoafdrtdj77\n7DN27drFwIEDLzk3EREREZHmxqWuk3l5eVRWVhISElKtPTg4GKvVWus1VquV1q1bV2s7fb3VaiUy\nMpLc3NwafYaEhFBRUUFeXp7jXEFBAeHh4ZSVlWEwGPj73//OgAEDLjk3EREREZHmps6C/1IYDIZ6\n68vb25vvv/+eoqIiPv30U5544gmCg4O59dZbL6qfgoKCestJRERERORK8/HxueRr6yz4AwMDMZlM\n5ObmVmvPzc0lLCys1mtCQ0NrjLCfvj40NLTOGBcXFwIDAx1tBoOBmJgYAK6++moyMjL429/+xq23\n3npJuYmIiIiINDd1zuE3m81069aN1atXV2tPSUkhISGh1mt69erFunXrKC0trRYfHh5OZGSkIyYl\nJaVGn927d8dkMp0zn8rKSmw22yXnJiIiIiLS3Jx3Ss/EiRO55557uO6660hISOCVV17BarWSlJQE\nQHJyMt988w2ffvopAHfffTdPP/00Y8eOZcqUKezcuZOZM2cydepUR59JSUm8/PLLTJgwgYcffpgN\nGzawZMkS/v3vfztiZsyYQc+ePYmOjqa0tJSVK1eydOlS5s+ff8G51eaXfB0iciWkpqYCEB8f7+RM\nRC6M7llpTHS/SmNSX9PSz1vwjxo1ivz8fKZPn05OTg6dO3dm5cqVtGrVCqhaiLt3715HvLe3Nykp\nKTz66KPEx8fj7+/PpEmTmDBhgiMmKiqKlStXMmHCBBYsWEB4eDjz5s1j+PDhjpgTJ07wyCOPcPDg\nQSwWCx06dOCtt97irrvuuuDcRERERESau/Puw98UnP3pSCP80tBp9EkaG92z0pjofpXGpL5q2PM+\naVdERERERBovFfwiIiIiIk2YCn4RERERkSas3h+81dDZ7fZ6fTiYiIiISH2w2WyUlZU5Ow25QoxG\nI66urlekLm12BX9qWg7du7Z0dhoiIiIiDna7ndLSUtzd3TUw2QzY7XZsNhslJSVX5G/e7Kb0/PO9\nbdhsTX5jIhEREWlEysrKMJvNKvabCYPBgMlkwmw2U15eftlfr9kV/PsPHGPD5gPOTkNERETEwW63\nYzKZnJ2GXGFGoxGbzXb5X+eyv0ID9M//bKOi4vL/ckVEREREzuVKfaPTLAv+3ENFfLY209lpiIiI\niIhcds2y4Ad454MdlJVVOjsNEREREZHLqtkV/H4+FgCOHitmRcqPTs5GREREpOmaOnUqRqORQ4cO\nXfbX+vLLLzEajaxdu/ayv1Zj0+wK/jtv7+D45/c/+oETJ7XfrYiIiEhToF2OatfsCv6BfaMJCfYE\noOhEGR+u3OXkjERERETkl+rbty/FxcX06dPH2ak0OM2u4Hd1MTF6eEfH8f9W7eLosRInZiQiIiIi\nP1dcXHxR8QaDQc8yOIdmV/AD9OnVmtYRPgCUllaw7P3tTs5IREREpOnKz8/n7rvvxtfXF39/f5KS\nkjhx4oTjfFRUFEOHDuWzzz6jR48eWCwWZs2aBVTtVf/000/X6DMqKor77rvPcVzbHP6xY8disVjI\nzs7m9ttvx8vLi+DgYP7whz/U2P/+nXfeoXv37vj4+ODt7c1VV13F9OnT6/tX4RTNsuA3Gg385q6r\nHcefrtnLgaxCJ2YkIiIi0nSNHj2agoICnn32WUaOHMmiRYsYNWqU47zBYGD37t3ceeed9O/fn3nz\n5tGzZ89q53/OYDBc0Gi+zWZjyJAhBAUF8eKLL9K3b19efPFFFi1a5Ij59NNPGT16NH5+fjz33HO8\n8MILDBkyhA0bNvzCd94wuDg7AWfpenUoV18Vwvc7crHZ7Lz1zvf8aUJvZ6clIiIi0uRERESwYsUK\nx3FYWBjTpk3js88+Y+DAgdjtdvbs2cN///tfbr755np97fLyckaNGsWUKVMAePjhh+nWrRuLFy8m\nKSkJgBUrVuDj48OqVaua5JSgZlvwGwwGfjP6aiY99SlgJ/W7bLZlHKJTh2BnpyYiIiJSp38v3847\nH1yeKcmjbu/I6Ds6nj/wIjz22GPVjh9//HGmTZvGihUrGDhwIACtWrWq92L/tIceeqjace/evVm6\ndKnj2NfXl6KiIlatWsWQIUMuSw7O1Cyn9JwWE+VH3+tbO47f+NdWbDa7EzMSERERaXratm1b7Tgg\nIABfX1/27dvnaIuJibksr202mwkJCanW5ufnx9GjRx3Hv/vd74iLi2PYsGFEREQwduxY/ve//12W\nfJyhWRf8AL8a2RmzqwmAvfuOsm7TT07OSERERKT5sVgsFxVfUVFxQXEXMkUnKCiI7777jhUrVnDH\nHXewceNGbrvtNm699daLyqmharZTek4LDPDg5hvbsfyjDADefi+dXt0jMJtNTs5MREREpHaj76j/\naTeX065du2jTpo3jOC8vj2PHjhEVFXXea/38/Dh27Fi1trKyMnJycuo1R1dXV4YOHcrQoUMBSE5O\nZubMmWzcuJGEhIR6fa0rrdmP8AMMvzkOL083APLyT7Ii5UcnZyQiIiLSdLz88svVjl966SUAbrrp\npvNeGxsby5o1a6q1LVq0qMa2mudyISP8R44cqdHWpUsXAAoKCi7odRqyZj/CD9DCw8xdw6/itbe+\nA+A//81g4A3ReHu5OTkzERERkcYvOzubYcOGcdNNN7F161Zee+01brzxRseC3bo8+OCDJCUlMXLk\nSAYNGsTWrVtZvXo1gYGB2O3nX3t5ITEPPPAA+fn5DBw4kIiICLKysnj55Zdp2bIlN9xwwwW9x4ZM\nI/ynJPaPpWWoFwAni8sv28p3ERERkebi9F75//rXv/Dz8+PJJ5/kvffe46GHHuLdd9+tFncuDz30\nEJMnT2bt2rVMmjSJ/fv3k5KSQosWLWpcV9vxhezhf8899+Dp6ckrr7zCo48+yuuvv87NN9/Mhg0b\naNGixaW+/QbDYL+Qjz2N3NlfxRjdW+DlVvsXG5u/zWLm3KoHLBiNBmZPT3Q8kVfkSklNTQUgPj7e\nyZmIXBjds9KYNNT7taSkBHd3d2enIU5Q19/+7BrWx+fSa9JmN8L/+tasc5677tqWdGwfBIDNZuf1\nt9Mu6GsgEREREZGGqtkV/Kv35vH9oeO1njMYDDzw666Or3i+355L6nf1uwJcRERERORKanYFP8DL\n3/xEaUXtK7ujWvuS2P/Mgx9e/2caZeWVVyo1EREREZF61SwL/qzjJSzbce6R+zEjOtHCwwxA7qEi\nVqzWNp0iIiIi0jhdUME/f/58oqOjsVgsxMfHs379+jrj09PT6du3Lx4eHkRERDBt2rQaMWvWrKFb\nt25YLBZiY2NZuHBhtfOvvvoqffr0wd/fHz8/PwYMGMCGDRuqxUydOhWj0Vjtp2XLlhfylvhPRi6Z\nx07Wes7by427znqYxbsf7uDosZIL6ldEREREpCE5b8G/bNkyxo8fz5QpU0hLSyMhIYGhQ4dy4MCB\nWuMLCwsZPHgwYWFhpKamMnfuXGbNmsXs2bMdMZmZmQwbNozevXuTlpZGcnIy48aNY/ny5Y6YNWvW\nMGbMGL744gs2b95MXFwcN954I7t37672eu3bt8dqtTp+0tPT63w/HQM9Aaiw25n3zU9U2mpflDtk\nQCytwr0BKCmp4O136+5XRERERKQhOm/BP3v2bO677z4eeOAB4uLieOmllwgLC2PBggW1xr/99tuU\nlJSwZMkSrrrqKkaMGMHkyZOrFfyvvPIKERERzJ07l7i4OB588EHuvfdeXnjhBUfM0qVL+d3vfsc1\n11xDu3btWLBgAV5eXqxatara65lMJoKDgx0/AQEBdb6fx7q3xtVYtSh3Z/4JVuw+XGuci4uR+3/V\nxXH8+bpMftyTX/cvS0RERESkgamz4C8rK2PLli0kJiZWa09MTGTjxo21XrNp0yb69OmDm5tbtfjs\n7Gz279/viKmtz9TUVCora18gW1paSklJCX5+ftXa9+7dS3h4ODExMYwZM4bMzMy63hKtfSyMuirM\ncfzm99kcOlFWa+w1nULp3vXMFKHFS9OwneMbARERERGRhqjOgj8vL4/KykpCQkKqtQcHB2O1Wmu9\nxmq11og/fXz6mtzc3FpjKioqyMvLq7XfKVOm4OXlxa233upo69mzJ0uWLGHVqlW8+uqrWK1WEhIS\nOHLkSF1vizs7hNDa2wJAcUUlL3+z/5z77Y8dcw0mU9WvadeefNZs2F9n3yIiIiIiDUntj5z9Bep6\nNPKlmjt3LosWLeKzzz7D09PT0T5kyBDHP3fq1IlevXoRHR3NkiVLmDBhQq19nX7C3oAWlczPqRrZ\nX7fnJAvK8rnOv/ZfR5er3Fn7VdUHkXmL1mHCioel3n91ItWcvldFGgvds9KYNLT7NTIyUk/abaaO\nHz/Otm3baj3Xtm3benmNOkf4AwMDMZlM5ObmVmvPzc0lLCys1mtCQ0NrjP6fvj40NLTOGBcXFwID\nA6u1z5kzhz//+c98/PHH530MtoeHBx07dqyxsLc20S1M3BB0pmj/X3Y5x8pqH+Xvd30Qvt6uAJw4\nWcmnaw+dt38RERERkYagzmFqs9lMt27dWL16NSNGjHC0p6SkcOedd9Z6Ta9evZg8eTKlpaWOefwp\nKSmEh4cTGRnpiHn//ferXZeSkkL37t0xmUyOttmzZzN16lRWrlxJQkLCed9MSUkJGRkZDBgw4Jwx\nZ39o6FRhI2dVBlnHq7bcXFvuw9O9Ymv9lsLoGs6seVXrFrbtLOU3Y2JoE+N/3pxELtbpUafzfcAV\naSh0z0pj0lDv15ISbf/dXHl5eZ3zfiwoKKiX1zjvLj0TJ07kjTfeYPHixWRkZPDEE09gtVpJSkoC\nIDk5mUGDBjni7777bjw8PBg7dizbt29n+fLlzJw5k4kTJzpikpKSyMrKYsKECWRkZPDaa6+xZMkS\nJk2a5IiZNWsWycnJLF68mDZt2ji23SwsLHTETJo0ibVr15KZmcnmzZsZOXIkxcXF3HvvvRf05t1d\njIy/LhIDVQX+tzkFpGTWvhNPz/hwrr369LcadhYu2UJlZe1P6xURERGRpsNoNPLII484O41Ldt6C\nf9SoUcyZM4fp06fTtWtXNm7cyMqVK2nVqhVQtRB37969jnhvb29SUlLIzs4mPj6ecePGMWnSpGpz\n6qOioli5ciVr166la9euPPvss8ybN4/hw4c7YubPn09FRQV33XUXLVu2dPyMHz/eEZOVlcWYMWNo\n3749I0aMwGKx8NVXXzlyuxBXBXlye1yw4/i17w5y+GTNXXsMBgMP3tMVV5eqbyD2ZB7h0zV17wgk\nIiIiIk3D5VineqUY7OfanqYJOfvrEB8fnxrnSypsPH7W1J74MB+m3lD71J5/L9/OOx9sB8CzhZmX\nnhuCr48W2Uj9aahfN4uci+5ZaUwa6v1aUlKiRbsNmNFoJCkpifnz59d733X97c9Xw16o847wNwfu\nLkaeOGtqT2odU3vuuLk9IcFVOwUVnSjjrXe+v2J5ioiIiIhcLBX8p3QM8uS2dkGO43NN7TGbTTx0\nT1fH8Rfr9rFjZ+1P6xURERERyMnJ4be//S0RERG4u7sTHR3Nww8/TFFREUePHuUPf/gDV199Nd7e\n3nh5edG/f3/Wr19frY99+/ZhNBqZOXMmr776KrGxsbi7u3PdddfV2Ga1X79+9O/fv0YeY8eOJTo6\nulqb3W7n73//O126dMHDw4OgoCASExNrvP7P/e1vf8NoNPLss8862ubPn0/nzp3x9PTE19eXrl27\nsmjRoov9ddU7bSZ/lnuuDufr7AKyi0o5UV7JnM37mdavDcafTe259powesZH8FXqQQAWvrGFF6YN\ncszvFxEREZEqVquV6667jiNHjvDwww/TsWNHsrKy+OCDDzhy5AiHDh3iP//5D3fddRcxMTEcPXqU\nxYsXM2jQIL755hs6d+5crb9ly5ZRVFTkWET7/PPPc8cdd7B3715cXKpKW4PBcM459z9vf/jhh1m8\neDFDhgzh/vvvx2azsXHjRtatW0fv3r1r7WPGjBn8+c9/Zs6cOTz++OMALF68mMcee4w777yTxx9/\nnPLycrZt28amTZt4+OGHf9Hv8JdSwX8WdxcjE3pEMfnzXdjsdtJyC/nfrsPcdtai3tPu/1UXvku3\nUlpawYGsAj5YsZM7b7vKCVmLiIhIc/P2tmz+uS3nsvR9d6cwftWpZb3193//939YrVY2btxI9+7d\nHe1/+ctfgKrnM529AQxUFeHt27fnpZde4tVXX6127uDBg/z444+OOe1xcXHcdtttrFq1iptuugmo\nGrU/V8F/9vLVL7/8ksWLF/Poo48yb948R/vZm8T83J/+9CdmzpzJK6+8Uq2Q/+ijj+jUqRPLli2r\n8/fhDJrS8zNXBXkysn2I4/iNrVnsO1ZcIy4wwIO7R3RyHL/74Q6ycgprxImIiIg0Vzabjffff5+h\nQ4dWK/bPZjabHf9cUlJCfn4+FRUVxMfH8+2339aIHzFiRLUFrKdH4TMzL373xPfeew+Ap59++ryx\ndrudCRMmMGvWLJYsWVJj1N7X15cDBw40uKc4gwr+Wt3dKYw2fh4AlNlszN68j/Ja9twfNriN4+Fb\nFRU2Frz+LTZbk9/0SEREROSCHD58mOPHj9OpU6dzxtjtdp577jliYmIcc+iDg4NZuXJltecvnda6\ndetqx35+fgAcPXr0ovPbs2cPoaGh+Puf/2Gqb7/9NnPnzmXu3Ln8+te/rnF+8uTJeHl5cd1119Gm\nTRseeeQRvvzyy4vO6XLQlJ5auJqMTOwZxfhVP1Bms7Hn6Ene3pbD2GvCq8WZTEYeuS+eP/wlBZvN\nzo6dh/l0TSaJ/WOclLmIiIg0B7/q1LJep9040zPPPMOf//xnxo4dS2JiIgEBAY7FsD+f6gNgMtW+\nZvLsqTrnms5TWVlZ53V1SUhIID09nZdffplRo0YRGBhY7Xz79u3ZuXMnK1euZNWqVXz00UcsXLiQ\n3/3ud7z88ssX9BqXi0b4zyHSx1KtwH8vI5fth4tqxEVH+nLbsDjH8Zv/3sqRWqYAiYiIiDQ3QUFB\neHt7k56efs6Yd999l/79+/P6668zevRoBg8ezMCBAykuvvR6ys/Pr9YR//3791f7MBAbG0tubi55\neXnn7TMmJoZPP/2UvLw8EhMTq+2Rf5rFYmHEiBEsWrSIffv28atf/Yr58+eTk3N51ltcKBX8dbil\nXRBdQ7wBsGPnxa/2cbK85ifDu27v6Nib/2RxOYvfSruieYqIiIg0REajkeHDh/Pxxx/z9ddf1zhv\nt9txcXHBZqs+dXrjxo1s2rTpkl+3TZs2/PDDD9UK+a1bt7Jhw4ZqcXfeeScAU6dOvaB+O3TowKpV\nq8jMzGTo0KGcOHHCcS4/v/oznEwmk2Mq07Fjxy7lbdQbTempg9Fg4IkekTz28Q6KyivJPVHKoi0H\nGN8jqlqc2Wwi6b5uPD1zDQCbvjnAN1si6X5t0/iqTURERORSPfvss6SkpNCvXz9++9vf0qFDB3Jz\nc3n//fd5//33ufXWW5k6dSr33nsvvXv35scff+TVV1+lY8eOFBXVnF1xIe6//35mz57NjTfeyP33\n38+hQ4dYuHAhnTp1qrYuoG/fvowdO5b58+ezZ88ehgwZAsCmTZu45pprSE5OrtF3165dWbFiBYmJ\nidx2222sWLECNzc3EhMTCQkJ4frrryc0NJTdu3fz8ssvc80119ChQ4dL++XVE43wn0eQh5lH488s\nDknJzGftT0dqxF3TMYT+faIcx4ve3MLJ4vIrkaKIiIhIgxUaGsrmzZsZPXo0//rXv3j88cd54403\n6NGjB4GBgSQnJ/PHP/6Rzz//nCeeeII1a9awbNky4uPjzzkX/3zat2/Pm2++SUFBAb///e/56KOP\nWLp0Kddee22NPhcvXszs2bM5cOAAkydPZvr06Rw7dox+/fqds/+EhAQ+/PBDNmzYwJ133klFRQVJ\nSUkUFxczd+5cHn30Ud555x0eeOABUlJSLuk91CeD/UJXKjRiZ8+xOnsbp4sxa1MmX+6vKvRbuJp4\n6cYOhHq6VYs5XlTKuMmfUHi8FIDB/WN45L74S8xamqvT23nFx+vekcZB96w0Jg31fi0pKcHd3d3Z\naYgT1PW3r48aFjTCf8F+1601oS2qCvwT5ZW88NU+Kn62BaeXpxsP3tPVcZzyxV62brNe0TxFRERE\nRM6mgv8CtTCb+GNCNC6nvgbKyCvin9uya8Rd36MVPeMjHMd/X5yqqT0iIiIi4jQq+C9CXEALfn31\nmYW47+zIZWtu9QdCGAwGHvrNtXidmu6Tl3+SJf/eekXzFBERERE5TQX/RRrRPqTaVp0vbNrHsZLq\nI/h+vu489BtN7RERERER51PBf5GMBgMTe0bh41a1o+mRknLmfL2/xlPaNLVHRERERBoCFfyXwN/i\nysSeUY7jb7IL+HDXoWoxmtojIiIiIg2BCv5LFB/mw/C4EMfxP9Ky2Jl/olqMpvaIiIiIiLOp4P8F\n7r26JW38PACosNt5dsNeCksrqsX8fGrPy6+lUlRUdkXzFBERkYavGTwaSX7mSv3NVfD/Aq4mI8nX\nx9DC1QTA4ZNlvPjVPmxn/fF+PrUn/8hJFr25xSn5ioiISMNkNpspKSlR0d+MVFZWUlJSgtlsvuyv\n5XLZX6GJC/V04/c9o/jruj0ApOYU8O4OK3d1DHPE+Pm688j93Xj+pY0ArP/qJ7p3bUmfXq2dkrOI\niIg0LEajETc3N0pLS52dilwhBoMBd3d3DKee8XQ5qeCvBz3CfRnRPoT//JALwNJtObQP9OSaEC9H\nTM/4CAb0iebzdZkALHzjWzq0CyQwwMMpOYuIiEjDYjQacXd3d3Ya0gRpSk89+c3V4XQM8gTAZrfz\n/MZM8k9Wn6t//6+7EBLUAoCTxeW8tOhrbDZ9dSciIiIil48K/nriYjQwuVc0vm6uABwrLWfmpkwq\nzyroPSyuPP7bHo6vbrZlHOKjVbuckq+IiIiINA8q+OtRgIeZPyZEYTxV0G8/XMSb6dnVYjq0C+SO\nm9s7jpe+m85PBwuuaJ4iIiIi0nyo4K9n14R48+tOZxbsvpdhZcOBo9ViRg2/iphIPwAqKmz8bcFm\nysorr2ieIiIiItI8qOC/DO68KpT4MB/H8ezN+9h3rNhx7Opi4omkHphPbee5/8Ax3vr391c8TxER\nERFp+lTwXwZGg4Hf94wi7NTe+yUVNmas38Pxsx7K1Srcm9+MvtpxvCLlR77Zkl2jLxERERGRX+KC\nCv758+cTHR2NxWIhPj6e9evX1xmfnp5O37598fDwICIigmnTptWIWbNmDd26dcNisRAbG8vChQur\nnX/11Vfp06cP/v7++Pn5MWDAADZs2PCLc7tSvN1cmNI7FjdT1a84u6iUF77aV20R79BBbejetaXj\neN6rX5OXf/KK5yoiIiIiTdd5C/5ly5Yxfvx4pkyZQlpaGgkJCQwdOpQDBw7UGl9YWMjgwYMJCwsj\nNTWVuXPnMmvWLGbPnu2IyczMZNiwYfTu3Zu0tDSSk5MZN24cy5cvd8SsWbOGMWPG8MUXX7B582bi\n4uK48cYb2b179yXndqVF+VqY0CPKcZyaU8Db286M4hsMBh57qDsB/lV78RedKGPOK5uprLRd6VRF\nREREpIky2M/zDOcePXrQpUuXaiPw7dq1Y+TIkTzzzDM14hcsWEBycjK5ubm4uVVNaZkxYwYLFizg\n4MGDAEyePJkPPviAnTt3Oq576KGH2L59Oxs3bjxnLmFhYTz55JM89thjF5VbQcGZXXB8fM7Mrb9S\n3tiaxbsZVsfxn66P4fpWfo7jHTsP8+dnvnQ8TnvU7R0ZfUfHK52mNBCpqakAxMfHOzkTkQuje1Ya\nE92v0pjUVw1b5wh/WVkZW7ZsITExsVp7YmLiOQvzTZs20adPH0exfzo+Ozub/fv3O2Jq6zM1NZXK\nytp3qyktLaWkpAR/f/9Lzs1Z7unckm51LOK9Ki6IUbdf5Th+98MdbMs4dEVzFBEREZGmyaWuk3l5\neVRWVhISElKtPTg4GKvVWus1VquV1q1bV2s7fb3VaiUyMpLc3NwafYaEhFBRUUFeXl6NcwBTpkzB\ny8uLW2+99ZJzgzOf7K+0QW52dlSUkl9q5yQw6aNvGdfGjRYuVXv2R4fbCQ4wkPnTCQCenrmacQ+0\nwcNS559ImjBn3asil0r3rDQmul+lMWjbtm299FPvu/ScfopsfZo7dy6LFi1i+fLleHp61nv/V4KH\ni4GxUWYOCiORAAAgAElEQVTMp37j+aV2lu4vo/LUNB6j0cCdt0TgYanaqrPweAX/WZHFeWZciYiI\niIjUqc7h48DAQEwmE7m5udXac3NzCQsLq/Wa0NDQGiPsp68PDQ2tM8bFxYXAwMBq7XPmzOGpp57i\nk08+qTbf7lJyA+fP2fOLPMozG/YCkG2Hbwjk0W6tHR+UfPwjeWZ21U5DB7IrOXjIi+E3tT9nf9L0\naH6pNDa6Z6Ux0f0qjcnZc/h/iTpH+M1mM926dWP16tXV2lNSUkhISKj1ml69erFu3TpKS0urxYeH\nhxMZGemISUlJqdFn9+7dMZlMjrbZs2fz1FNPsXLlyhqvdym5NQTXt/Ljns5ntuL8eE8eH+46M18/\nvktLbhnSznG89J10zecXERERkUt23ik9EydO5I033mDx4sVkZGTwxBNPYLVaSUpKAiA5OZlBgwY5\n4u+++248PDwYO3Ys27dvZ/ny5cycOZOJEyc6YpKSksjKymLChAlkZGTw2muvsWTJEiZNmuSImTVr\nFsnJySxevJg2bdpgtVqxWq0UFhZecG4N1V1XhdI/0t9xvDgti6+zz3yCu2fU1bRvW/VNh91uZ/b8\nrzhy1iJfEREREZELdd6Cf9SoUcyZM4fp06fTtWtXNm7cyMqVK2nVqhVQtRB37969jnhvb29SUlLI\nzs4mPj6ecePGMWnSJCZMmOCIiYqKYuXKlaxdu5auXbvy7LPPMm/ePIYPH+6ImT9/PhUVFdx11120\nbNnS8TN+/PgLzq2hMhgMjOseSYfAqvUINrud5zdmOnbucXEx8vtHe+LtVbXT0bGCEmb//Svtzy8i\nIiIiF+28+/A3Bc7eh/9cjhaXMzHlBw6dLAMg2MPM7MHt8bO4ArB1ey5/fX6tY+Hu7cPi+M3oa5yW\nr1wZml8qjY3uWWlMdL9KY3JF9uGXy8vP4spfbmiDxaVq3cKhk2XM2LCX0oqqkfxrOoYwZsSZB3B9\nsHInm7/NckquIiIiItI4qeB3sihfC39MiMZ4apeejLwi5ny9D9upUf07bu5AfJczi3znLfqaHOtx\np+QqIiIiIo2PCv4G4LqWPjzQJdxxvPano/xja9VIvtFo4PGHryM4sAUAJ4vLmfnSRkpKKpySq4iI\niIg0Lir4G4jb2gVzU5sgx/HyH3L576ntOj09zUwa1wsXl6o/108HC5j36jd6KJeIiIiInJcK/gbC\nYDDw22tb0TPc19G2aMtBNhw4CkCbaH9+e283x7lN3xzgP//LuOJ5ioiIiEjjooK/ATEZDfyhVzTt\nA6qm79ix88JX+9hxuAiAgX2jGTa4jSP+n+9t55vvsp2Sq4iIiIg0Dir4Gxh3FyNP3dCGcC93AMoq\nbfx13R4OFpYAMHZMFzq2Pz31x86cBZs5mF14jt5EREREpLlTwd8A+bi58PQNbfBxcwHgeFkFT63Z\nzdHiclxcjPxhXAJBp74FKC4p57k5Gyg6UebMlEVERESkgVLB30CFebnxlxva4Gaq+hPlnihl6trd\nnCyvxNvLjclPJGA2V30gyLYeZ86CzXoSr4iIiIjUoIK/AYsLaMHks/bo3330JH9dt4fSChsxUX48\n9uCZpwRu+T6Ht9/b5qxURURERKSBUsHfwPUI92VcfGvHcfqh4zy/KZNKm53ePVtzx80dHOc+WPED\nn63JdEaaIiIiItJAqeBvBBJjA7nvmjMP5voq6xhzvt6PzW5nzIiO1Z7E+8ob35K+45Az0hQRERGR\nBkgFfyMxskMoIzuEOo4/35fP4u8OYjQaGP9ID6JaV+3fX1lpY9a8jWTlaOceEREREVHB36iMvbol\nN8YEOo4/2HWIZTuseFhc+dOE3vj5WAAoOlHG9BfXU3i81FmpioiIiEgDoYK/ETEYDDwa35rrW/k5\n2t5Kz+ajHw8TGOBB8sTrHTv35B4qYubcjZSVVzorXRERERFpAFTwNzImo4FJPaPoGuLtaFvw7U+k\n7M2jTbQ/45OuA6p29cnYdZgFr6dit9udlK2IiIiIOJsK/kbIbDLyZO8Y4k49fAtg7tc/8eW+I/SM\nj+Ceuzo72tds2M87H+xwRpoiIiIi0gCo4G+kLK4mnr6hDbF+HgDYsTN78z7WHzjK7cPiGHhDtCN2\n2fvb+XTNXmelKiIiIiJOpIK/EfNyc2Fav7ZEnlqsW2m3M2tjJl9nF/Dbsd24umOII/aVf3xLalq2\ns1IVERERESdRwd/I+bi5MKNfWyK83AGosNt5dsNe0g4f54+PJxATWbXA12az88LLX7Frd74z0xUR\nERGRK0wFfxPgZ3FlRv+2hHm6AVBuszNj3V52FRTz5O/7EBJUNde/rKyCGbPXa49+ERERkWZEBX8T\nEehh5pn+7QhpUVX0l9ls/HXdHn4qLWPKpD54nfowcLyolGmz1nH0WIkz0xURERGRK0QFfxMS3MLM\nM/3bEuRhBqC00sbUtXvIBZ6c2NuxR/+hvBPMeHEdJ4vLnZitiIiIiFwJKvibmFBPN2b0b0ugparo\nL6u0MW3dHgo8XJj0WE+Mxqo9+vfuP8qzf1tPWZkezCUiIiLSlKngb4LCvdx5bkA7x0h/mc3GjPV7\nqAzyIOm+bo647T8c5oWXN1FRYXNWqiIiIiJymangb6LCvNx4bsCZOf3lNjvPrN+LJcaPe+662hGX\nmpbNS4u+prJSRb+IiIhIU6SCvwkL9XRj5sB2jt17Kux2Zm7MJKBzMHfc3MERt/6rn3j1ze+w2+3O\nSlVERERELhMV/E1ckIeZ5wa0c+zTX2m3M2vTPsK6t2TIwDaOuNVf7OGtd9JV9IuIiIg0MSr4m4FA\nDzPPDmhHK++qot9mt/O3zfsJ6xVB34RIR9wHK37gP//LcFaaIiIiInIZXFDBP3/+fKKjo7FYLMTH\nx7N+/fo649PT0+nbty8eHh5EREQwbdq0GjFr1qyhW7duWCwWYmNjWbhwYbXz27dvZ+TIkcTGxmI0\nGnn66adr9DF16lSMRmO1n5YtW17IW2p2/C2uPDegHVE+FgDs2HllywGCrm9FfNcwR9w/39vG/z7Z\n5aw0RURERKSenbfgX7ZsGePHj2fKlCmkpaWRkJDA0KFDOXDgQK3xhYWFDB48mLCwMFJTU5k7dy6z\nZs1i9uzZjpjMzEyGDRtG7969SUtLIzk5mXHjxrF8+XJHTHFxMTExMUyfPp3o6GgMBkOtr9e+fXus\nVqvjJz09/WJ/B82Gr7srzw5oR1xAC0fb29tz8O8dSeerQhxt//hnGh+t/tEZKYqIiIhIPTtvwT97\n9mzuu+8+HnjgAeLi4njppZcICwtjwYIFtca//fbblJSUsGTJEq666ipGjBjB5MmTqxX8r7zyChER\nEcydO5e4uDgefPBB7r33Xl544QVHTHx8PM8//zxjxozBw8PjnPmZTCaCg4MdPwEBARfz/psdbzcX\nZvRrS9cQb0fb//YcxrNPK9q1C3S0vb70O1amqOgXERERaezqLPjLysrYsmULiYmJ1doTExPZuHFj\nrdds2rSJPn364ObmVi0+Ozub/fv3O2Jq6zM1NZXKyot7ENTevXsJDw8nJiaGMWPGkJmZeVHXN0cW\nVxNP3RBLQoSvo+3LA0ex9G5F27Zniv7X3vqOVZ/vcUaKIiIiIlJPXOo6mZeXR2VlJSEhIdXag4OD\nsVqttV5jtVpp3bp1tbbT11utViIjI8nNza3RZ0hICBUVFeTl5dU4dy49e/ZkyZIltG/fntzcXKZP\nn05CQgLbt2/H39+/1mtSU1MvqO/mYJCbnQK3cr45UvUh68s9J4luY8Y7D3IOngRg9t/XsWfPHq7r\nWvvvUy4f3avS2OielcZE96s0Bm3btq2Xfup9l55zzbW/HIYMGcLIkSPp1KkTAwcOZMWKFdhsNpYs\nWXLFcmjMTAYDIyNcuSHozOe+zJN2SuPDCIk4M43qw0+ySd16xBkpioiIiMgvVOcIf2BgICaTidzc\n3Grtubm5hIWF1XpNaGhojdH/09eHhobWGePi4kJgYCCXysPDg44dO7J79+5zxsTHx19y/01Vd7ud\nZTusvJWeDUAhENCvPbHfH8K6p6rQX72mgNjYNgzuF+PETJuH06NOulelsdA9K42J7ldpTAoKCuql\nnzpH+M1mM926dWP16tXV2lNSUkhISKj1ml69erFu3TpKS0urxYeHhxMZGemISUlJqdFn9+7dMZlM\nl/RGAEpKSsjIyDjnhxGpncFgYHTHMB7vHonp1Dc0+aXlHL4qAP+Y01N57Cx4PVW794iIiIg0Mued\n0jNx4kTeeOMNFi9eTEZGBk888QRWq5WkpCQAkpOTGTRokCP+7rvvxsPDg7Fjx7J9+3aWL1/OzJkz\nmThxoiMmKSmJrKwsJkyYQEZGBq+99hpLlixh0qRJjpjy8nLS0tJIS0ujuLiYnJwc0tLSqo3eT5o0\nibVr15KZmcnmzZsZOXIkxcXF3HvvvfXyy2lubowN5M99YnEzVd0WJyttHGrvR4u2Z+bvv770O5br\n4VwiIiIijUadU3oARo0aRX5+PtOnTycnJ4fOnTuzcuVKWrVqBVQtxN27d68j3tvbm5SUFB599FHi\n4+Px9/dn0qRJTJgwwRETFRXFypUrmTBhAgsWLCA8PJx58+YxfPhwR0xWVhbXXnstUDUCvXDhQhYu\nXEi/fv34/PPPHTFjxowhLy+PoKAgevXqxVdffeXITS5e95Y+PDugHU+v3U1BaQWVQH6sL76uJkp2\nHAZg6bvplJZVMvqOjld0zYaIiIiIXDyD3W63OzuJy+3s+U8+Pj5OzKTxyDleyl/W7ibreAkANpsd\nj6wibOmHOF3i3zKkHWPHXKOiv55pfqk0NrpnpTHR/SqNSX3VsPW+S480DWFebjw/sB3tTz2V12g0\nUBzhRVnXECpP3TX/+2QXC9/Ygs3W5D8zioiIiDRaKvjlnHzdXZnRvx09w6se0GUwgD3Uk6KuoZS7\nVN06q7/Yw7xFX1NRYXNmqiIiIiJyDir4pU7uLkb+dH0Md7SvehiawQDuwS041jWYYveqHZXWbNzP\nc3M2UFJS4cxURURERKQWKvjlvExGAw90ieCJ7pG4GAxgAO/AFhy7OpgCbzMAW77PYerMNRwvKj1P\nbyIiIiJyJanglwuWGBvItP5t8TK7gAH8Az042TGQw4EW7MCuPfk8Of0L8vJPOjtVERERETlFBb9c\nlKuDvXhxcBwRXu4A+PtZsLcPICvck0ojHMwuJHna5xzIKnRypiIiIiICKvjlEoR7ufPCoDi6hHgD\n4OPthiXWj/3RPpS5Gsk/cpInp3/Ozt35Ts5URERERFTwyyXxcnPh6b5tuKVtMACenmb8W/uwL9aX\nohauFJ0o4y/PreGr1INOzlRERESkeVPBL5fMxWggqVsrxl8XidloxGJxJTTci6wob/IC3Cktq+D5\nlzbx34930gye7yYiIiLSIKngl19scEwgzw1sR5CHGTc3F8JbenM03PPUvH47b/xrK6+++R2Vldqr\nX0RERORKU8Ev9SIuoAV/G9yeTkFeuLoaCQ/zpizIg31RPpSajXzy2W6em7OB4pJyZ6cqIiIi0qyo\n4Jd642dxZUb/ttzSNhiTyUDLMC9c/S1kRvtQ4G3m2605TJn+BflHtG2niIiIyJWigl/q1el5/RN6\nROHmYiQkqAU+fhaywj3JCfVgz4FjTH76M3ZnHnF2qiIiIiLNggp+uSwGRQfw4qA4wr3d8fe3EBTo\nwVE/d/ZFeWMtKmXK9C9Y/9VPzk5TREREpMlTwS+XTYyfB3MS29O7lR/eXm60DPWkzMOVvTE+5Lkb\nmT3/K5a+k47Nph18RERERC4XFfxyWXm4mvi/hGiSrm2Fl4eZiJZemNxcOBjhhTXEg/dWZPDc3A2c\nLNZiXhEREZHLQQW/XHYGg4Fb2gXz/KA4Inw9iGjphYfFlSP+VVN8Nmy3kvzXz8ixHnd2qiIiIiJN\njgp+uWLiAlowN7E9vSJ8CQvxxNfHnRJ3FzKjffj+RCl/fPoz0tKtzk5TREREpElRwS9XlJebC3/u\nE8tvu7UiLLAFwUEtsJsM5IS1YKevmadmr+fdD3doXr+IiIhIPVHBL1ecwWDg1nbBzE6Mo2NLH8LD\nvDCZjBR6m8mM9ua1j3/gmdnrOV5U6uxURURERBo9FfziNNG+Vbv43NExjFbh3ljcXSh3NbI/0ptP\nco7x+6c+Zc++o85OU0RERKRRU8EvTuXmYuR38a15ul8b2kf64evjDkBeoIWvvUxMeO5LPl2z18lZ\nioiIiDReKvilQegR7svLQzowqEMIocGeGI0GStxd+LG1J3/9YBsvLfqa0tIKZ6cpIiIi0uio4JcG\nI8DDzF/7tWFC7xiiI3wwm03YDQYOBXvwj4NHeGzaZ+w/cMzZaYqIiIg0Kir4pUExGgzcFhfMwls6\n0b9jKF6eZgCKLS5ssBh5YM46Pv5sN3a7dvERERERuRAq+KVBivB2Z3Zie34/oC2hQS0wGAzYjAYO\nBlqY8sWPTJ23kaKiMmenKSIiItLgqeCXBstkNDC6Yxivj7iGXu0CMZtNAJxo4cq7x08yeuYXbN91\n2MlZioiIiDRsKvilwYvx82DhbZ2ZMKAtvt5uANiMBnZ7mPj121uY/973VFbanJyliIiISMOkgl8a\nBVeTkfu6RvDm6Gvp0toPo9EAQLG7iXm7crlz1pfsyypwcpYiIiIiDc8FFfzz588nOjoai8VCfHw8\n69evrzM+PT2dvn374uHhQUREBNOmTasRs2bNGrp164bFYiE2NpaFCxdWO799+3ZGjhxJbGwsRqOR\np59+ul5yk8atXUALlo65lon92uDh5gKA3WBgh93G7a9vZtGKDC3oFRERETnLeQv+ZcuWMX78eKZM\nmUJaWhoJCQkMHTqUAwcO1BpfWFjI4MGDCQsLIzU1lblz5zJr1ixmz57tiMnMzGTYsGH07t2btLQ0\nkpOTGTduHMuXL3fEFBcXExMTw/Tp04mOjsZgMPzi3KRpcDEaeLBHJO/fdx2dQ7w4fWuUuBiZ/X0W\nI/62jn2Hjjs3SREREZEGwmA/z3Bojx496NKlS7UR+Hbt2jFy5EieeeaZGvELFiwgOTmZ3Nxc3Nyq\n5lvPmDGDBQsWcPDgQQAmT57MBx98wM6dOx3XPfTQQ2zfvp2NGzfW6LNz587ceeedPPXUU5eUW0HB\nmakePj4+db1daWRsdjtLNu3npQ17Ka44M4/fDNzfJYLHh8RhrOXDYkOWmpoKQHx8vJMzEbkwumel\nMdH9Ko1JfdWwdY7wl5WVsWXLFhITE6u1JyYm1lqYA2zatIk+ffo4iv3T8dnZ2ezfv98RU1ufqamp\nVFZWXlDil5KbND1Gg4H7EqJY8VAvrglo4WgvA15JO8jQl9eTduCo8xIUERERcTKXuk7m5eVRWVlJ\nSEhItfbg4GCsVmut11itVlq3bl2t7fT1VquVyMhIcnNza/QZEhJCRUUFeXl5Nc7VV25w5pO9ND1/\nuNaVT3e58K89JRQbqz7LZhYWM/qtb7g+0IV7r/bEw6XxrFPXvSqNje5ZaUx0v0pj0LZt23rpp96r\nn9rm2otcKYPaeTK7ry+djRUYTs1Ws9lh3eEKJq4tYF1WqRb1ioiISLNS5wh/YGAgJpOJ3Nzcau25\nubmEhYXVek1oaGiNEfbT14eGhtYZ4+LiQmBg4AUlfim5gebsNRcDb4BPNv/EtNU7yXep+hBaVAmv\n/lDC14Vmpg5tT1ygp5OzrJ3ml0pjo3tWGhPdr9KYnD2H/5eoc4TfbDbTrVs3Vq9eXa09JSWFhISE\nWq/p1asX69ato7S0tFp8eHg4kZGRjpiUlJQafXbv3h2TyXRBiV9KbtK8DOnRmtW/78ttgV64nFrQ\na7PZ+e5gAaOXfstfP9tF3skyJ2cpIiIicnmdd0rPxIkTeeONN1i8eDEZGRk88cQTWK1WkpKSAEhO\nTmbQoEGO+LvvvhsPDw/Gjh3L9u3bWb58OTNnzmTixImOmKSkJLKyspgwYQIZGRm89tprLFmyhEmT\nJjliysvLSUtLIy0tjeLiYnJyckhLS2P37t0XnJtICw8zMx/qwRvDryaq1OaY5lNcXM6/vvmJO976\nhre/z6K0Qk/qFRERkaapzik9AKNGjSI/P5/p06eTk5ND586dWblyJa1atQKqFuLu3bvXEe/t7U1K\nSgqPPvoo8fHx+Pv7M2nSJCZMmOCIiYqKYuXKlUyYMIEFCxYQHh7OvHnzGD58uCMmKyuLa6+9Fqha\nF7Bw4UIWLlxIv379+Pzzzy8oN5HT4juF8mHbQF55J52lGVaOe7lit8OhI8U8t3oX72+z8njvaPq0\n8tM6FBEREWlSzrsPf1OgffjlbD/uyefZpd/xrb2SUrczU8i8PM30jAkk6brWdAxy3vx+zS+Vxkb3\nrDQmul+lMbki+/CLNEVtYwN4dcoApveKJjK/BFNl1XSe40VlfLYth6T3v+eva3ezv6DYyZmKiIiI\n/HIq+KVZMpmM3HJjO/715EB+5eOJ/5ESDHY7Npudw/kn+c83B3jgg3Tmfr2fw1rYKyIiIo2YCn5p\n1vz9LPxpXAJ/vyeengUV+BRU7S5VWlbJwexC3tq8n/s/SOcfW7MoKqtwcrYiIiIiF08FvwjQ9epQ\nFk5LZMJ1kbT7qQjPonIAiorK2P3TMV7dtI/7/7uNd3dYKS6vdG6yIiIiIhdBBb/IKWazidF3dOLV\nvw7mrlAfWv9UiHtJBXa7nSNHi9mReYS/b9rHAx9tY/kPuZRoK08RERFpBFTwi/xMSFAL/jAugdmP\n96ZfhZHwrCLMZZVUVNiwHioiY99R5m/ex4MfbePDnYe0h7+IiIg0aCr4Rc6hU4dgXvzrIP54R2eu\nyS0hLOcEruU2iksqOJh9nB9+Osbfv97PQyu28dGPhymrVOEvIiIiDc95H7wl0pyZTEZuHBDL9de1\nYtn721n5+R7yvV3JC7RQdKKMEyfLKCh04+WiMt7LsDKifQiJMYG4ueiztIiIiDQMKvhFLoCnp5kH\n7unK0MFtePvddDakHuSYrxt5gRYKCks5XlRGoY87C4pKWbbDyu1xwQxrE4SHq+n8nYuIiIhcRir4\nRS5Cy1Av/jAugVt35/Pmv79n+4+HOernTl6gO0eOFlNQWEqhrzuvF5fzXkYuN7cN4tZ2wXi76V81\nERERcQ5VISKXIK5NANOf7Efqdzm89c73/PTjMY76uZMf4E5e/kmOFZTg5+POP0sreH9nLkNjg7ij\nfQj+Fldnpy4iIiLNjAp+kUtkMBjofm1Lrr0mlM/X7ePfy7fjt/sYBT5u5Ae4c7jCVlX4+1p4/4dc\nVvx4mAHR/tweF0Irb3dnpy8iIiLNhAp+kV/IZDIyuF8MfRMi+eSz3Sz/6Ad89xRQ6G0mL9DCoQob\nRwuK8fe18MnuPD7Zk0f3lj4Mjwvh6mBPDAaDs9+CiIiINGEq+EXqidls4tahcQzuH8PHn+7hgxU/\n4L23gCLPql19csttHDlWNdXn66wCvskuoI2fB7fHBdOntT8uRhX+IiIiUv9U8IvUM4u7K3fc3J4h\nA2P5aNWP/PfjnXjuK+SkhwtH/N05VF7JkWPF+Pq486MdXvhqH29szeaWdkEMiQ10dvoiIiLSxKjg\nF7lMPCyujLr9KoYNbsP/PtnFR6t+pMXBIkrNRo74u5NfXsnRYyX4+rhhs9n5x9Ys/rkthzjXMhIC\n9K+miIiI1A9VFSKXmWcLM2NGdOKWIe34+NPdfLTqR9ysJwk6XMxRP3eOllZw9FgJPt7u+Hq78VVh\nJV/lV7K+bBc3tw2iV4SvpvuIiIjIJVPBL3KFeLYwc+dtV3HLje1I+XIvH6zciUteMQH5xRR6u5Ef\nUMGxghLc3cCzhQvbDh9n2+HjBFhcGRpbNd3HT9t6ioiIyEVSwS9yhbm7u3DLkHbcODCWL9fv5/0V\nP2A8VIRPQSknPVw45Gki19edk8Xg6+NOPrB0Wzb/3p7D9a38GNImkM5B2t1HRERELowKfhEnMbua\nSOwfw8Abotj49UGWf/QD+w8cI/hwIRWuRspa+pHrV4pLCzM+3m54tjCz5qcjrPnpCBFe7gyOCWBQ\ndAC+7hr1FxERkXNTwS/iZCaTkT69WtO7Zyu+33GI15asY9eeInxOTfc57mWueoqvd1Xh7+3pxsHj\nJfxjaxZvfZ9NjwhfbowJoGuoN0aN+ouIiMjPqOAXaSAMBgPXdAzh3lFR5B4uYX+2O2s27MdYWIZP\nYRklbiaO+rlxwNcdi5cbPt5u4GZiw4GjbDhwlGAPM4kxgQyKCSDIw+zstyMiIiINhAp+kQYoJMid\nm4bGM2ZEJ1Z9vpuPP90DRaWEWU8SknuSQm8zR3zdsfm5431qus+hk2Us3ZbNP7fn0CXEi4FRAfSM\n8MXdxejstyMiIiJOpIJfpAHz83Vn9B2duOPmDqzb9BMff7qbvfuP4ltQhm9BGSXWExzzdSPL3x0P\nn6ri38XFyBZrIVushVhcTPRu5cuA6AA6BXlqyo+IiEgzpIJfpBEwm00M7BvNgBui2LXnCB+n7Gbj\nNwdwL60kNPcktkMnOe5lJs/XDXugB95eZjw8zBRXVJKSmU9KZj5BHmYGRPkzICqACG93Z78lERER\nuUJU8Is0IgaDgbg2AcS1CWDs3dfw6Zq9rP5iL3n5J/E5Nde/LOcEhd5mcgMsuPpb8PI0YzabOHyy\njGU7rCzbYSUuoAX9Iv3p3coPf+3tLyIi0qSp4BdppHx93Bl561UMv6k9qWk5fPLZHrZuy8VcbiMw\nv4SA/BJK3As55utGZagn7l5utGjhitFoYGf+CXbmn2DRloN0Cvakb2s/Elr54eOm/ySIiIg0Nfq/\nu0gjZzIZ6dEtnB7dwsk9fIIv1mXy+bp95OWfxFJSicV6ElvuSYo8XTnmb8EW3IIWnmYs7i7YDXbS\nDx0n/dBxFnx7gC4hXvRu7UdChC+eZv3nQUREpCm4oO075s+fT3R0NBaLhfj4eNavX19nfHp6On37\n9qL1kD8AACAASURBVMXDw+P/27v34LjK++Dj33Pdm3Yly7pZsi0bX+ILt4AFyDEkhdgNdMqbuFBM\n+07eMG9oZigvMbw0xDOQUgdoWlomtAMeD+k0/gNP3FYNnWHSgt6hdWxsSoRNfJExGMt371qypJX2\nfnveP87ualeS5QuyLVm/z8xyznnO75w9K585PL/zPM85zJw5kx//+McjYrZu3cqtt96Kx+Nh3rx5\nbNy4cURMW1sbS5Yswe12s3TpUt56662y9c8//zy6rpd9GhsbL+QnCXFNqq/1sWb19Wz4m/t49v/e\nSWvLTAxDR1cQGEwz4+gA9buCqN1B+j7vpbc3TiqVBSCrFB8FB3j1w6P8z7f2sP7Xh3jvyFkiqcxV\n/lVCCCGE+CLOewtvy5YtrF27lg0bNrBixQpee+017r33Xjo7O5k1a9aI+IGBAVauXMnXvvY1Ojo6\nOHDgAI888gg+n4+nnnoKgK6uLu677z6++93vsnnzZrZt28Zjjz1GbW0tq1evBmDnzp2sWbOG9evX\ns3r1atra2njwwQd5//33ue2224rft2jRIv7rv/6ruGwYxhf9mwgx6RmGzi03zeCWm2bQH06w9f2j\n/L+tXZw8PYCZVUzrT0J/krTZz6DfJtXoJ1Plwl9hYxg66Zziv0+F+e9TYUxN46Z6P60zq7ijqYpp\n0udfCCGEmFQ0pZQaK+D222/n5ptvLrsDv3DhQh544AFeeumlEfEbNmxg3bp1hEIhXC4XAC+++CIb\nNmzgxIkTADzzzDO89dZbHDx4sLjdo48+yv79+9mxYwcADz30EP39/bzzzjvFmJUrV1JbW8vmzZsB\n5w5/W1sbe/fuHfNHhsPh4nxlZeWYsUJcbR0dHQAsW7ZsXPerlOKzz3vZuuMo2z84zmAkWbY+beoM\nVLowZgfIVNjF/v6lNDSW1Pi4Y2YVy2dW0VDhGtdjFJPT5Tpnhbgc5HwVk8l41WHH7NKTSqXYtWsX\nq1atKitftWpVsWI+3M6dO7nzzjuLlf1C/KlTpzh69GgxZrR9dnR0kM063Qs++OCDC/rew4cP09TU\nxHXXXcfDDz9MV1fXWD9JiClL0zQWzp/Oo9++hX/4u99n3doVtLbMwjKdVjErk2P62ThVu0MEPjhJ\n7KMgiWCUwUiKXM65L6BQ7O+J8A8fn+B/v72P//MfB3hz3yk+74txnnsHQgghhLhKxuzS09PTQzab\npb6+vqy8rq6OYDA46jbBYJDZs2eXlRW2DwaDNDc3EwqFRuyzvr6eTCZDT08P9fX1BIPBUWNKv/eO\nO+5g06ZNLFq0iFAoxAsvvMDy5cvZv38/1dXV5/npQkxdpqnTcksjLbc0Eo2l2PmbE2x9/yj7P+kG\nwE7nmN4dg+4YaVMnWuXCnl1JosLC4x2683+4P8bh/hib952mxmPT0higpbGSm+oD8oZfIYQQYoIY\n98dwaFfwTZ7f+MY3ivPXX389ra2tzJ07l02bNvHkk0+Ouk2hKU+Iie5KnqtVPvgfq/x89XYXew6E\n2ffJACdPx4vrrXAEdfQspqkTqXZjzgqQCLiwXUax8n8sGuNYTz9te8DUYX6FzmK/waKATrUtlf+p\nQK6vYjKR81VMBgsWLBiX/YxZ4a+pqcEwDEKhUFl5KBRixowZo27T0NAw4u5/YfuGhoYxY0zTpKam\nZsyYwj5G4/V6Wbp0KYcOHRrrZwkhzqGq0uauO2q5645aevtT7DsQZt8nYU4GEwAYmRzeMzE4E8PU\nNRJVLvRZAVLT3OguE9NwKv+ZHHwykOOTgRychAa3xqKAwSK/TrNXx9Sv3I0BIYQQYqobs8Jv2za3\n3nor7777Ln/wB39QLG9vb+fBBx8cdZvW1laeeeYZkslksR9/e3s7TU1NNDc3F2N++ctflm3X3t5O\nS0tL8Sk7ra2ttLe38/TTT5fFfOUrXznn8SYSCQ4cOMDdd999zhgZpCMmuok0oGzV151pMBRhx4fH\n2fHhCQ4f7Suu9yeBQxEUEeIeE3t2AFXnI24buFxDT8waAD6MOh+3qXNDnZ9bGgJ8uSHATL/rirYM\nivE3kc5ZIc5HzlcxmZQO2v0ijOeff/75sQICgQB//ud/TmNjIx6PhxdeeIHt27fzj//4j1RWVrJu\n3Tp+8pOf8O1vfxtwnuCzceNGPv74YxYvXsz27dv5sz/7M9atW0draysA8+fP56/+6q/o7u6mubmZ\nf/u3f+Oll17ilVdeYfHixQA0NTXxox/9CJfLRU1NDW+88QY///nPeeONN2hqagLg6aefxu12k8vl\n+PTTT3n88cc5fPgwGzduLBvJnEwOPY3E7XaPyx9OiMvl1KlTABPqnRIVFTaLv1TLqrvncWfrbKZX\ne0kmM5ztde78aziDfvWzcYxjA2gnBjDiWdwugxig61qxUp/JKU4NJvno9ABvf9ZNe9dZjoYTpLI5\nprktXNL3f9KZiOesEOci56uYTMarDnvePvx/+Id/yNmzZ3nhhRc4ffo0N9xwA7/61a+Kz+APBoMc\nPny4GB8IBGhvb+dP//RPWbZsGdXV1Tz99NNlfernzJnDr371K5588kk2bNhAU1MTf//3f8+3vvWt\nYkxrayu/+MUvePbZZ/nRj37E/Pnz+ad/+idaWlqKMSdPnuThhx+mp6eH2tpaWltb+eCDD0Z9P4AQ\nYnw0Nvj51u8t4lu/t4i+/gS/2X2KDz86yZ7OEJlMDnAG/XIsTO5YGJ8GyUoXnjlVpKrcJC0dyxqq\n1HfHUrx7uId3D/egoTG/2svN9X5urPOzpNaH25R3awghhBBfxHmfw38tkOfwi8lksjY3x+JpPt4b\n5MOPTvHRb08TjaVGjUtZOlZTALvJz6DbANvgXD16TE1j4XQfN9RVcGOdn0U1FfL0nwlosp6zYmqS\n81VMJuNVhx33p/QIIaYmr8di+W2zWH7bLDKZHJ0Hu9m9J8iuPac5fnKgGGenc3Ckn/SRflxAJuDC\nP7eKdLWbsKFhWYbTRwjIKEVnT4TOnghbOoOYmsaXpvu4oc7PjfUVLJpeIV2AhBBCiPOQCr8QYtyZ\nps6NS+u5cWk9/+vhmzjTE2X3niAf/fY0e/afIZXKAPm+/wNJEr91nuTl0TVUjRff7EpSlTaDulbW\n/SejnBd/7e+J8ItOsHSNhdU+ltRWsLjGx+KaCgIuuawJIYQQpeT/jEKIy66uxsfv3j2P3717HqlU\nlgOfdrNrT5Bdvw1y8vTQ3X8jp+BMlOSZKABuQ8Oe4cc7K0DcbzOoKQxjKAFI54YSgIJZATdLaytY\nXFPBkhofMyrkKUBCCCGmNqnwCyGuKNs2uOn6Bm66voFH/gjO9ETZu/8Mv90fYm/nGcIDiWKsmVXk\nTgwQOeEkBbah4W4K4GryE/NZRHSnNaHU8YEExwcS/MfnPQBMc1ssrvGxpKaCJbUVzK3yYBvSDUgI\nIcTUIRV+IcRVVVfj456vzuWer85FKcWxE2H27D/Db/eF2H+wm2QyU4w1s4rMsTCZY84gJpehYTVU\n4JsZIOW3CeugD0sA+hJpdpzoZ8eJfmcfmsZ107wsnO7lS9N9LKz20eh3oUsrgBBCiGuUVPiFEBOG\npmk0z6qieVYVv/+NhaQzWT77vJc9+0PsO9DNZ5/3ks5ki/FmVqFODhI5OQiASwOz1ot/dhXZKhe9\nOuQMvewpQBml+LQ3yqe9Ud7+rBsAn2WwsNrnJADTvSys9jHNY13R3y6EEEJcLlLhF0JMWJZpsORL\ntSz5Ui0AqVSWQ129dB7sZv8nPXzyWU9ZC4CuIHcmRvhMzNkeyHpNKmZVYdR5ibkNItrIbkDRdJbd\noQF2h4bGE9R6bRZUe5k/zct105ypJAFCCCEmI6nwCyEmDdseSgAeuB8ymRxdR/vYf7CHzoPdHDjY\nU/b8fw0wYxkSB3vgoFPmMjSsWh/+mQFylS7CpkZaA00v79LTHUvRHUsVuwIBVLutYuX/umke5k3z\nUu+zZVCwEEKICU0q/EKIScs0dRbMm86CedP55n1fIpdzxgAcPHSWTz7r4dNDvZwODZZvk1WoYISB\noPNkHx0wbIOKpgB2vY+kzyKsg2ZoI5KA3kSa3tNhOk4PvQilwjK4bpqXefkkYG6Vh5l+N5YMDBZC\nCDFBSIVfCHHN0HWNObOrmDO7it+9ex4Ag5EkBw+d5dNDvXx66CyfHj5LIjHUDUgDrFSWZFcfya4+\nANxAymPia/TjqvWRqbAIa4Cpj3grcCSdZc+ZQfacGUosTE2jKeBmTqWH5ko3c6o8NFd6pDVACCHE\nVSEVfiHENc1f4WLZzY0su7kRgGw2x/GTAxw8dJaDh87yeVcfJ04NoJQqbqMBrniGzOd9ZD53kgAX\nkPWY+Gb4MWs8ZCpsBnSNrKFhGOWV+IxSHA3HORqOl5V7TYPmKg9zKt00V3ry8x55WZgQQojLSv4v\nI4SYUgxDH9EKkEhk6DrWz+ddvXze1cfnR/o4cWoQKE8CzHiG5OE+koedJMAG0paOUe3BXV+BVuki\nbuvENLAsY0RrQCyT5UBPhAMlLwoD510BMwNuZvrdzK50prMq3dR4LGkREEII8YVJhV8IMeW53SaL\nF9aweGFNsSwWT9N1tJ/DR/o4lE8ETgXLxwNogJ3OQShKOuS8HdgAvDpkPBYVjX7MaR7SXpOIoZHR\nNSxTdzYs0ZdI05dIs/dM+f7dpl6WBMwMuJkVcDOjwiVjBIQQQlwwqfALIcQovB6LpYtqWbqotlgW\ni6c5ejzM0eP9HDmWnx4Plz0aFMDIgRFNk/6sl3S+zAUYpkbWa+Op96FXuZ1EQNfQTQ3LNEYkAolM\njkN9MQ71xcrKTU2jvsLFjAoXTX5n2uh30Vjhps5nY+jSKiCEEGKIVPiFEOICeT3WiJaAXE4ROhPh\nyPEwR4715z9hus9Gy7bVACujsAaSqIEkWZwnBPlxugVlPCbeOh9mlYeM1yRmaGTyXYP0UcYInBxM\ncHIwQcfp8mM0NY2GChcNw5KBJr+bWq8kA0IIMRVJhV8IIb4AXdeY0eBnRoOf1paZxfJoLMXR42GO\nnxzgxKkBjp90Pn395QN5C92C7HQKBlJkcMYHeICsoZF0GWgBN/Z0D6rCImkbJHQnEbBMfcSjQzNK\ncWIwwYkxkoF6n02dz0V9hU29z6be55RVukwZMyCEENcgqfALIcRl4PPaZW8JLohEU8UEoDQRONtb\n3m1Hw3lngBnLQCwC+fcGeAG3BinbIGXr2NUe7EoPOZ9JwtRJ6s4bii1TH7VloJAMjMZt6jT4XNTl\nk4CGCmda57Np8Lnw2ca4/X2EEEJcOVLhF0KIK6jCZ7NoQQ2LFtSUlcfiaU7kk4DToQingoOcCkY4\nHYqQSpWPEdAVuJNZ3MksDKaBAXSgAvAWkwEDKiysKjf4LKdM1zBNHcsyMEd5p0Aik+NIOM6RYY8T\nLfDbJjVei1qvTY3Xzk8tuiNZqiyNdDYng4mFEGICkgq/EEJMAF6PxcL501k4f3pZeS6n6O2LczI4\nyOl8EnDq9CCnQoOc6Y6Sy6my+PJkIAWnnbEEHiCrQ9oySFs6MdvAqnRh+F1k3QZJU0flE4LCRx/W\nXWgwlWEwlaFrWLekWDQFwN8d3800t1VMBAqJQel8tduScQRCCHGFSYVfCCEmMF3XqJnupWa6l5uW\n1petS2eynOmOcup0hFBPlNCZCKHuKKEzUULd0REtA0YOjEIyQBp6h7r2VOCMGUhbOmnLIGrr4LUw\nKmxyHpOUpaMbellCYBrnfsTop72j/x4NjSq3yXSPxTSPRbXbotpjMd3jTKe5nfkqSQyEEGLcSIVf\nCCEmKcs0aJoRoGlGYMQ6pRT94SSh7kgxATjTHXWWu6P0nI0z4sViWYWZzeJJZJ3Cs0MJgQIypkbG\n1ElbOol8S4Hhs0gZOXIuk2gcTEPHMDVnmk8QDF0rJgYKVUwK8uOTR6VrGlUuk2rPUEJQSBCq3BZV\nbpMqlzP1WDK2QAghxiIVfiGEuAZpmsa0KjfTqtwjxgsApNJZunui9PTG6emJ0X02Rk/h0+tMU+ns\n0P7IP1Y0U0gI0sV10agz4Njr8zqPGM0nBSlLJ2bqZFwGmtdCuQxyloFh5BOCfDJgGDqmoaEbQ+MK\nckrRm0jTe57EAMBl6CVJgEmV22kpqMwvV+bXTXNbVNgGujyJSAgxxUiFXwghpiDbOnfrADgtBAOD\nybJEYGg+Ts/ZGH3hBMNbCZxHjOZg9HG/5DTIGjoZUysmBwnTmWZMHTymkxiYTguBYRQSgsK8li8f\nSg6S2RyhaJJQNHne321qGoF8UhBwmQRcBn7bzM+bBPLzfpdBwGVS6TJxGbo8rlQIMalJhV8IIcQI\nmqZRGXBTGXAzf271qDGZTI7+cIJt2zsYiKSprZtNb1+C3v44vX1x+vrj9PYliMZSxW10BXomh5Vh\nqOvQKAqJgZMU5LsSmTpx01nOGjrKZaBsAy2fDOh6SVKgO4OOhy9nKGk5uEC2rpclAcUEoZgoGPhd\nJhWWQYVtUmEb+CxDnlgkhJgwpMIvhBDikpimTs10L7OavAAsW7Zg1LhEIuNU/vsT9PY5yUBvf5y+\n/gThgQT94ST94QSDkRSFFoOhxCA35jEoIKdDxtSLLQeF+WQ+MciYQ+Wa6Qw+dpIAp7WgkBjo+lBi\nUFqWIkdPPEXPOVotzsVl6PgLCYDttCR4LQN/yXIhOaiwTXz5dRW2icuUZEEIMX6kwi+EEOKycrvN\n4tuIx5LN5hgYTNEfzicC+WQgHHbmwwP5+XCS8GCCXE6hkX/6UCoHjJ0cQHmXoqyhkzW0/McpS45S\nprRCMuBMdUMfmi8rL8w7SUMuq0hkchedKIDTquCzDbyWgcfMz5sGHstJEHyWgcfS8RbnDbyms1z6\nMeVJR0IIpMIvhBBigjAMvTjQ+HxyOUUk6iQHkWiKgcEkA4MpIhFnOhBJMpgvG4wmGRxMEY2lyroU\nwbm7FJV9V1mSUJoo5Mci5OdzulZMFnK6jsrXtTWNYnKga0OJwlhlcU1jMF5edinDCFyGji9f+fdY\nBl7LWXabTiLhsQzcho7H0otlbtOZd5t6WYzb1KWbkhCTlFT4hRBCTDq6rhHwuwj4XRe8TSaTYzCS\nYjCSZDBSSBKSRCJOghCJpohEUkRjaSJRZxqNpUgkMiVJwoUpdDUaLREYakHQyOka6XwCURqnRqnd\nFxMHTUMrm5JPCLSSKcX1vfnl8vXOdtpFtgBYulZMBpyEYCgxcJtOq4PHNHDl17uMko85NLUNrazM\nberYhi5PUBLiMpEKvxBCiCnBNC+8BaFUJpMjGks5SUA0XUwEItHRy6JRJ2GIxdPE4mly6fN3NSql\nAKVRTAIKiYAzr5+jXCM7fNm4sMrziCRBG1p25kuThfx8oVwvmS/bpnz+Quvxtl6eGIxIEIxR1ufn\nLV3DNnSsfKylO/O24Wxv6U5SMZhWmDqksjms/G8X4lp3QRX+119/nZdffplgMMjSpUv56U9/yooV\nK84Zv3fvXh5//HF+85vfUF1dzfe+9z2ee+65spitW7fy1FNP0dnZSWNjIz/4wQ/43ve+VxbT1tbG\nc889x+HDh5k3bx4vvvgi3/zmN7/QsQkhhBAXwzT14hOLLpZSilQqSyyeIRZPEY9niMbTxONp4vEM\nsXiaeCLtrI85CUI87izHE+liWSyeJpdT5//C0u/GaWUoJABDCYI+onzoU1KujVyvvkDduDRhcJKG\n8oTiUqb68HI0tGGJhqZpznvf8sceizovlPN27QZwEoJisuAkDoUEwjL0YeudxMFlOOssQ8PWdUxd\nK34K+ygu6857JkzNWbaN0eMNTcMyNGnlEJfFeSv8W7ZsYe3atWzYsIEVK1bw2muvce+999LZ2cms\nWbNGxA8MDLBy5Uq+9rWv0dHRwYEDB3jkkUfw+Xw89dRTAHR1dXHffffx3e9+l82bN7Nt2zYee+wx\namtrWb16NQA7d+5kzZo1rF+/ntWrV9PW1saDDz7I+++/z2233XZJxyaEEEJcSZqm4XKZuFzmRbcs\nlFJKkU7niMbSJJMZ4okMiWSGRGFaMj9ifSJDMpkdsU0ylil7udp5jwGn5WF4AlDogjRa8jBUPjTv\nTJ3l7LB1l1Oh8p/LZQEN00wPJQn5dZQtl5aVJxul+yvsg5JtyuOB0n0WY8v3U5h3EoGh5MIyhiUP\nuuYkFEY+riR5MPJJhVGybOhOsmHoGoZG2brSWDPfilOILawrxuWnev4Yy/dbHm/IYPEJR1NKjXnL\n4Pbbb+fmm29m48aNxbKFCxfywAMP8NJLL42I37BhA+vWrSMUCuFyOX0rX3zxRTZs2MCJEycAeOaZ\nZ3jrrbc4ePBgcbtHH32U/fv3s2PHDgAeeugh+vv7eeedd4oxK1eupLa2ls2bN1/UsYXD4eJ8ZWXl\nBfxZhLh6Ojo6AFi2bNlVPhIhLoycs5NXNpvLJwlZ4ok0iWSWVCpLKuWUpdJZksksyVSGZMpZl8yv\nT6VyJFOZkrLSdeVl2ez5uzWVJhSlScG5EobR5pVG2baFskJyojSNjMqhNNBMY9SxEhNNWcLgzJYl\nGKOtK/yscyUVlMWMsW7Y941WVvwLFsry+ylNBgydYiKgayVl+WWzUF6SaBTKjZJPIRnSdTB0vZjA\naAwlI7o2lNxohe8qjH/B+d7CMegaJcfkHKMTN1Re2GehRUqnfNnIJ3ZG/o9mFI6DoVYoPb++2CpV\nEqcx8jtKjVcddsw7/KlUil27dvGDH/ygrHzVqlXFivlwO3fu5M477yxW9gvxzz33HEePHqW5uZmd\nO3eyatWqEfvctGkT2WwWwzD44IMPeOKJJ0bEvPbaa5d8bEIIIYQYYhg6Pq+NzwvguWzfk8nkypKH\nVMpJJtLpHOl0lnQmP03nSKWyZLLONJ0pTLPOPlLZ4r6K26ZzpDLly+lRlgui0RgAPp93WJIxlCQM\nJQwXs2606VjrRo8Zzrktqyi/PXtx3buuVUN/rvJxIlr5f86xLr9dfrnsL1+SvAybFIMLy2X/ZJo2\nyn5KV5f/+5YdVyHpKnzyScPbf3Q942HMCn9PTw/ZbJb6+vqy8rq6OoLB4KjbBINBZs+eXVZW2D4Y\nDNLc3EwoFBqxz/r6ejKZDD09PdTX1xMMBkeNKXzvpRybEEIIIa4809QxTR2vx7oq35/LqWKi0NGx\ni2xWsfT6G8hmc2QyOTJZZ70znxuaL/0MK09ncvn9OonJ8Lhsfp/OuizZrHLm0/n1uSy5rCKTdfaT\nzc9nsoqsypHKAaiy5AKGEoTSxKMwhfIkojS+MP5ieKJxvpjz7XO08itlKAkanhANlQvHuD+lZ6KP\ndi9tGhFiIlqwwHlbqZyrYrKQc1ZMJl++eXF+rnDXv3BPVd4xIK5dY57dNTU1GIZBKBQqKw+FQsyY\nMWPUbRoaGkbcYS9s39DQMGaMaZrU1NSMGVPYx6UcmxBCCCGEEFPNmBV+27a59dZbeffdd8vK29vb\nWb58+ajbtLa2sm3bNpLJZFl8U1MTzc3NxZj29vYR+2xpacEwjDFjvvKVr1zysQkhhBBCCDHlqPPY\nsmWLsm1b/exnP1OdnZ3qiSeeUH6/Xx07dkwppdQPf/hDdc899xTjw+GwamhoUGvWrFH79u1TbW1t\nKhAIqFdeeaUY09XVpXw+n1q7dq3q7OxUb7zxhrJtW/3rv/5rMWbHjh3KNE31k5/8RB04cEC99NJL\nyrIs9eGHH17wsQkhhBBCCDHVnbfCr5RSr7/+upozZ45yuVxq2bJlatu2bcV13/nOd9TcuXPL4vfu\n3avuuusu5Xa7VWNjo1q/fv2IfW7dulXdcsstyuVyqeuuu05t3LhxRMy//Mu/qEWLFinbttWSJUvU\nL3/5y4s6NiGEEEIIIaa68z6HXwghhBBCCDF5XTND0l9//XXmzp2Lx+Nh2bJlbN++fcz4vXv38tWv\nfhWv18vMmTP58Y9/fIWOVIiLO1+PHDmCrusjPsPHrwhxOfz617/m/vvvZ+bMmei6zqZNm867jVxf\nxdVyseerXF/F1fSXf/mXtLS0UFlZSV1dHffffz/79+8/73aXco29Jir8W7ZsYe3atTz77LN8/PHH\nLF++nHvvvZfjx4+PGj8wMMDKlSuZMWMGHR0dvPrqq7z88su88sorV/jIxVR0sedrwTvvvEMwGCx+\nfud3fucKHbGYyqLRKDfeeCOvvvoqHo/nvI9eluuruJou9nwtkOuruBq2bt3K448/zs6dO3nvvfcw\nTZOvf/3r9PX1nXObS77GXu0+RePhtttuU3/yJ39SVrZgwQK1bt26UeNff/11VVlZqRKJRLHshRde\nUE1NTZf1OIVQ6uLP166uLqVpmuro6LgShyfEOVVUVKhNmzaNGSPXVzFRXMj5KtdXMZFEIhFlGIZ6\n++23zxlzqdfYSX+HP5VKsWvXLlatWlVWvmrVKnbs2DHqNjt37uTOO+/E5XKVxZ86dYqjR49e1uMV\nU9ulnK8Fq1evpr6+nhUrVtDW1nY5D1OISybXVzEZyfVVTAQDAwPkcjmmTZt2zphLvcZO+gp/T08P\n2WyW+vr6svK6uroRL+4qCAaDI+ILy+faRojxcCnnq9/v52//9m/553/+Z/793/+de+65h4ceeog3\n33zzShyyEBdFrq9iMpHrq5hIvv/97/PlL3+Z1tbWc8Zc6jXWHJ9DnFwutE+fEBPB9OnTefLJJ4vL\nt9xyC2fPnuWv//qv+eM//uOreGRCjCTXVzGZyPVVTBRPPfUUO3bsYPv27WNeRy/1Gjvp7/DX1NRg\nGAahUKisPBQKMWPGjFG3aWhoGJEFFbZvaGi4PAcqBJd2vo6mpaWFzz77bLwPT4gvTK6vYrKT66u4\n0p588km2bNnCe++9x5w5c8aMvdRr7KSv8Nu2za233jriEVrt7e0sX7581G1aW1vZtm0byWSyLL6p\nqYnm5ubLerxiaruU83U0H3/8MY2NjeN9eEJ8YXJ9FZOdXF/FlfT973+/WNlfuHDheeMv+Ro7GBwf\ntwAAAatJREFUbkOLr6ItW7Yo27bVz372M9XZ2ameeOIJ5ff71bFjx5RSSv3whz9U99xzTzE+HA6r\nhoYGtWbNGrVv3z7V1tamAoGAeuWVV67WTxBTyMWerz//+c/V5s2bVWdnp/rkk0/Uyy+/rGzbVj/9\n6U+v1k8QU0gkElG7d+9Wu3fvVl6vV61fv17t3r1brq9iQrrY81Wur+Jqeuyxx1QgEFDvvfeeOn36\ndPETiUSKMeN1jb0mKvxKOY8pmjNnjnK5XGrZsmVq27ZtxXXf+c531Ny5c8vi9+7dq+666y7ldrtV\nY2OjWr9+/ZU+ZDGFXcz5umnTJrVkyRLl8/lUIBBQLS0t6s0337wahy2moP/8z/9UmqYpTdOUruvF\n+UceeUQpJddXMbFc7Pkq11dxNQ0/Twufv/iLvyjGjNc1VlNKqfFokhBCCCGEEEJMPJO+D78QQggh\nhBDi3KTCL4QQQgghxDVMKvxCCCGEEEJcw6TCL4QQQgghxDVMKvxCCCGEEEJcw6TCL4QQQgghxDVM\nKvxCCCGEEEJcw6TCL4QQQgghxDVMKvxCCCGEEEJcw/4//RuYnBMuD30AAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", "\n", - " p_win = diff.ProbGreater(0) + p_tie * p_overtime\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(time_dist1)\n", + "thinkplot.Pmf(time_dist2)\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.3: Distribution of time between goals.\n", "\n", "For the Bruins, the overall chance of winning the next game is 55%.\n", "\n", "To win the series, the Bruins can either win the next two games or split\n", "the next two and win the third. Again, we can compute the total\n", - "probability:\n", - "\n", - " # win the next two\n", - " p_series = p_win**2\n", - "\n", - " # split the next two, win the third\n", - " p_series += 2 * p_win * (1-p_win) * p_win\n", - "\n", - "The Bruins chance of winning the series is 57%. And in 2011, they did.\n", + "probability:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.57142746956373802" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# win the next two\n", + "p_series = p_win**2\n", "\n", - "Discussion\n", - "----------\n", + "# split the next two, win the third\n", + "p_series += 2 * p_win * (1-p_win) * p_win\n", + "p_series" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Bruins chance of winning the series is 57%. And in 2011, they did." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Discussion\n", "\n", "As always, the analysis in this chapter is based on modeling decisions,\n", "and modeling is almost always an iterative process. In general, you want\n", @@ -444,7 +996,7 @@ "modeling decisions. You might want to read about the Weibull\n", "distribution (). Or\n", "you might want to look around for information about light bulb survival\n", - "curves.\n" + "curves." ] } ], diff --git a/notebooks/08_Observer_Bias.ipynb b/notebooks/08_Observer_Bias.ipynb index 84c805a..b6cc953 100644 --- a/notebooks/08_Observer_Bias.ipynb +++ b/notebooks/08_Observer_Bias.ipynb @@ -1,15 +1,271 @@ { "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "%load_ext autoreload\n", + "%autoreload 2 \n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ + "#Observer Bias\n", "\n", - "Observer Bias {#observer}\n", - "=============\n", - "\n", - "The Red Line problem\n", - "--------------------\n", + "## The Red Line problem\n", "\n", "In Massachusetts, the Red Line is a subway that connects Cambridge and\n", "Boston. When I was working in Cambridge I took the Red Line from Kendall\n", @@ -32,13 +288,17 @@ "took a class with me at Olin College. The code in this chapter is\n", "available from . The code I used to\n", "collect data is in . For more\n", - "information see Section [download].\n", - "\n", - "The model\n", - "---------\n", + "information see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The model\n", "\n", "![PMF of gaps between trains, based on collected data, smoothed by KDE.\n", - "z is the actual distribution; zb is the biased\n", + "`z` is the actual distribution; `zb` is the biased\n", "distribution seen by passengers. ](figs/redline0.pdf)\n", "\n", "[fig.redline0]\n", @@ -63,7 +323,7 @@ "database. I ran the script from 4pm to 6pm every weekday for 5 days, and\n", "recorded about 15 arrivals per day. Then I computed the time between\n", "consecutive arrivals; the distribution of these gaps is shown in\n", - "Figure [fig.redline0], labeled z.\n", + "Figure [fig.redline0], labeled `z`.\n", "\n", "If you stood on the platform from 4pm to 6pm and recorded the time\n", "between trains, this is the distribution you would see. But if you\n", @@ -83,7 +343,7 @@ "gap, and only 1/3 during a 5 minute gap. So the average time between\n", "trains, as seen by an arriving passenger, is 8.33 minutes.\n", "\n", - "This kind of **observer bias** appears in many contexts.\n", + "This kind of **observer bias** appears in many contexts.\n", "Students think that classes are bigger than they are because more of\n", "them are in the big classes. Airline passengers think that planes are\n", "fuller than they are because more of them are on full flights.\n", @@ -93,55 +353,69 @@ "as big is twice as likely to be observed.\n", "\n", "So given the actual distribution of gaps, we can compute the\n", - "distribution of gaps as seen by passengers. BiasPmf does\n", - "this computation:\n", - "\n", - " def BiasPmf(pmf):\n", - " new_pmf = pmf.Copy()\n", + "distribution of gaps as seen by passengers. `BiasPmf` does\n", + "this computation:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def BiasPmf(pmf):\n", + " new_pmf = pmf.Copy()\n", "\n", - " for x, p in pmf.Items():\n", - " new_pmf.Mult(x, x)\n", - " \n", - " new_pmf.Normalize()\n", - " return new_pmf\n", + " for x, p in pmf.Items():\n", + " new_pmf.Mult(x, x)\n", "\n", - "pmf is the actual distribution; `new_pmf` is the biased\n", + " new_pmf.Normalize()\n", + " return new_pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pmf` is the actual distribution; `new_pmf` is the biased\n", "distribution. Inside the loop, we multiply the probability of each\n", - "value, x, by the likelihood it will be observed, which is\n", - "proportional to x. Then we normalize the result.\n", + "value, `x`, by the likelihood it will be observed, which is\n", + "proportional to `x`. Then we normalize the result.\n", "\n", "Figure [fig.redline0] shows the actual distribution of gaps, labeled\n", - "z, and the distribution of gaps seen by passengers, labeled\n", - "zb for “z biased”.\n", + "`z`, and the distribution of gaps seen by passengers, labeled\n", + "`zb` for “z biased”.\n", "\n", "Wait times\n", "----------\n", "\n", - "![CDF of z, zb, and the wait time seen by\n", - "passengers, y. ](figs/redline2.pdf)\n", + "![CDF of `z`, `zb`, and the wait time seen by\n", + "passengers, `y`. ](figs/redline2.pdf)\n", "\n", "[fig.redline2]\n", "\n", - "Wait time, which I call y, is the time between the arrival\n", + "Wait time, which I call `y`, is the time between the arrival\n", "of a passenger and the next arrival of a train. Elapsed time, which I\n", - "call x, is the time between the arrival of the previous\n", + "call `x`, is the time between the arrival of the previous\n", "train and the arrival of a passenger. I chose these definitions so that\n", - "zb = x + y.\n", + "`zb = x + y`.\n", "\n", - "Given the distribution of zb, we can compute the\n", - "distribution of y. I’ll start with a simple case and then\n", - "generalize. Suppose, as in the previous example, that zb is\n", + "Given the distribution of `zb`, we can compute the\n", + "distribution of `y`. I’ll start with a simple case and then\n", + "generalize. Suppose, as in the previous example, that `zb` is\n", "either 5 minutes with probability 1/3, or 10 minutes with probability\n", "2/3.\n", "\n", - "If we arrive at a random time during a 5 minute gap, y is\n", + "If we arrive at a random time during a 5 minute gap, `y` is\n", "uniform from 0 to 5 minutes. If we arrive during a 10 minute gap,\n", - "y is uniform from 0 to 10. So the overall distribution is a\n", + "`y` is uniform from 0 to 10. So the overall distribution is a\n", "mixture of uniform distributions weighted according to the probability\n", "of each gap.\n", "\n", - "The following function takes the distribution of zb and\n", - "computes the distribution of y:\n", + "The following function takes the distribution of `zb` and\n", + "computes the distribution of `y`:\n", "\n", " def PmfOfWaitTime(pmf_zb):\n", " metapmf = thinkbayes.Pmf()\n", @@ -152,11 +426,11 @@ " pmf_y = thinkbayes.MakeMixture(metapmf)\n", " return pmf_y\n", "\n", - "PmfOfWaitTime makes a meta-Pmf that maps from each uniform\n", - "distribution to its probability. Then it uses MakeMixture,\n", + "`PmfOfWaitTime` makes a meta-Pmf that maps from each uniform\n", + "distribution to its probability. Then it uses `MakeMixture`,\n", "which we saw in Section [mixture], to compute the mixture.\n", "\n", - "PmfOfWaitTime also uses MakeUniformPmf,\n", + "`PmfOfWaitTime` also uses `MakeUniformPmf`,\n", "defined here:\n", "\n", " def MakeUniformPmf(low, high):\n", @@ -166,19 +440,19 @@ " pmf.Normalize()\n", " return pmf\n", "\n", - "low and high are the range of the uniform\n", - "distribution, (both ends included). Finally, MakeUniformPmf\n", - "uses MakeRange, defined here:\n", + "`low` and `high` are the range of the uniform\n", + "distribution, (both ends included). Finally, `MakeUniformPmf`\n", + "uses `MakeRange`, defined here:\n", "\n", " def MakeRange(low, high, skip=10):\n", " return range(low, high+skip, skip)\n", "\n", - "MakeRange defines a set of possible values for wait time\n", + "`MakeRange` defines a set of possible values for wait time\n", "(expressed in seconds). By default it divides the range into 10 second\n", "intervals.\n", "\n", "To encapsulate the process of computing these distributions, I created a\n", - "class called WaitTimeCalculator:\n", + "class called `WaitTimeCalculator`:\n", "\n", " class WaitTimeCalculator(object):\n", "\n", @@ -189,22 +463,22 @@ " self.pmf_y = self.PmfOfWaitTime(self.pmf_zb)\n", " self.pmf_x = self.pmf_y\n", "\n", - "The parameter, `pmf_z`, is the unbiased distribution of z.\n", + "The parameter, `pmf_z`, is the unbiased distribution of `z`.\n", "`pmf_zb` is the biased distribution of gap time, as seen by passengers.\n", "\n", "`pmf_y` is the distribution of wait time. `pmf_x` is the distribution of\n", "elapsed time, which is the same as the distribution of wait time. To see\n", - "why, remember that for a particular value of zp, the\n", - "distribution of y is uniform from 0 to zp.\n", + "why, remember that for a particular value of `zp`, the\n", + "distribution of `y` is uniform from 0 to `zp`.\n", "Also\n", "\n", " x = zp - y\n", "\n", - "So the distribution of x is also uniform from 0 to\n", - "zp.\n", + "So the distribution of `x` is also uniform from 0 to\n", + "`zp`.\n", "\n", - "Figure [fig.redline2] shows the distribution of z,\n", - "zb, and y based on the data I collected from\n", + "Figure [fig.redline2] shows the distribution of `z`,\n", + "`zb`, and `y` based on the data I collected from\n", "the Red Line web site.\n", "\n", "To present these distributions, I am switching from Pmfs to Cdfs. Most\n", @@ -212,13 +486,13 @@ "interpret, once you get used to them. And if you want to plot several\n", "distributions on the same axes, Cdfs are the way to go.\n", "\n", - "The mean of z is 7.8 minutes. The mean of zb\n", - "is 8.8 minutes, about 13% higher. The mean of y is 4.4,\n", - "half the mean of zb.\n", + "The mean of `z` is 7.8 minutes. The mean of `zb`\n", + "is 8.8 minutes, about 13% higher. The mean of `y` is 4.4,\n", + "half the mean of `zb`.\n", "\n", "As an aside, the Red Line schedule reports that trains run every 9\n", "minutes during peak times. This is close to the average of\n", - "zb, but higher than the average of z. I\n", + "`zb`, but higher than the average of `z`. I\n", "exchanged email with a representative of the MBTA, who confirmed that\n", "the reported time between trains is deliberately conservative in order\n", "to account for variability.\n", @@ -226,7 +500,7 @@ "Predicting wait times {#elapsed}\n", "---------------------\n", "\n", - "![Prior and posterior of x and predicted y.\n", + "![Prior and posterior of `x` and predicted `y`.\n", "](figs/redline3.pdf)\n", "\n", "[fig.redline3]\n", @@ -237,43 +511,43 @@ "\n", "As always, let’s start with the easiest version of the problem and work\n", "our way up. Suppose we are given the actual distribution of\n", - "z, and we know that the passenger arrival rate, $\\lambda$,\n", + "`z`, and we know that the passenger arrival rate, $\\lambda$,\n", "is 2 passengers per minute.\n", "\n", "In that case we can:\n", "\n", - "1. Use the distribution of z to compute the prior\n", - " distribution of zp, the time between trains as seen by\n", + "1. Use the distribution of `z` to compute the prior\n", + " distribution of `zp`, the time between trains as seen by\n", " a passenger.\n", "\n", "2. Then we can use the number of passengers to estimate the\n", - " distribution of x, the elapsed time since the last\n", + " distribution of `x`, the elapsed time since the last\n", " train.\n", "\n", - "3. Finally, we use the relation y = zp - x to get the\n", - " distribution of y.\n", + "3. Finally, we use the relation `y = zp - x` to get the\n", + " distribution of `y`.\n", "\n", - "The first step is to create a WaitTimeCalculator that\n", - "encapsulates the distributions of zp, x, and\n", - "y, prior to taking into account the number of passengers.\n", + "The first step is to create a `WaitTimeCalculator` that\n", + "encapsulates the distributions of `zp`, `x`, and\n", + "`y`, prior to taking into account the number of passengers.\n", "\n", " wtc = WaitTimeCalculator(pmf_z)\n", "\n", "`pmf_z` is the given distribution of gap times.\n", "\n", - "The next step is to make an ElapsedTimeEstimator (defined\n", - "below), which encapsulates the posterior distribution of x\n", - "and the predictive distribution of y.\n", + "The next step is to make an `ElapsedTimeEstimator` (defined\n", + "below), which encapsulates the posterior distribution of `x`\n", + "and the predictive distribution of `y`.\n", "\n", " ete = ElapsedTimeEstimator(wtc,\n", " lam=2.0/60,\n", " num_passengers=15)\n", "\n", - "The parameters are the WaitTimeCalculator, the passenger\n", - "arrival rate, lam (expressed in passengers per second), and\n", + "The parameters are the `WaitTimeCalculator`, the passenger\n", + "arrival rate, `lam` (expressed in passengers per second), and\n", "the observed number of passengers, let’s say 15.\n", "\n", - "Here is the definition of ElapsedTimeEstimator:\n", + "Here is the definition of `ElapsedTimeEstimator`:\n", "\n", " class ElapsedTimeEstimator(object):\n", "\n", @@ -288,17 +562,17 @@ "`prior_x` and `posterior_x` are the prior and posterior distributions of\n", "elapsed time. `pmf_y` is the predictive distribution of wait time.\n", "\n", - "ElapsedTimeEstimator uses Elapsed and\n", - "PredictWaitTime, defined below.\n", + "`ElapsedTimeEstimator` uses `Elapsed` and\n", + "`PredictWaitTime`, defined below.\n", "\n", - "Elapsed is a Suite that represents the hypothetical\n", - "distribution of x. The prior distribution of x\n", - "comes straight from the WaitTimeCalculator. Then we use the\n", - "data, which consists of the arrival rate, lam, and the\n", + "`Elapsed` is a Suite that represents the hypothetical\n", + "distribution of `x`. The prior distribution of `x`\n", + "comes straight from the `WaitTimeCalculator`. Then we use the\n", + "data, which consists of the arrival rate, `lam`, and the\n", "number of passengers on the platform, to compute the posterior\n", "distribution.\n", "\n", - "Here’s the definition of Elapsed:\n", + "Here’s the definition of `Elapsed`:\n", "\n", " class Elapsed(thinkbayes.Suite):\n", "\n", @@ -308,17 +582,17 @@ " like = thinkbayes.EvalPoissonPmf(k, lam * x)\n", " return like\n", "\n", - "As always, Likelihood takes a hypothesis and data, and\n", + "As always, `Likelihood` takes a hypothesis and data, and\n", "computes the likelihood of the data under the hypothesis. In this case\n", - "hypo is the elapsed time since the last train and\n", - "data is a tuple of lam and the number of\n", + "`hypo` is the elapsed time since the last train and\n", + "`data` is a tuple of `lam` and the number of\n", "passengers.\n", "\n", - "The likelihood of the data is the probability of getting k\n", - "arrivals in x time, given arrival rate lam. We\n", + "The likelihood of the data is the probability of getting `k`\n", + "arrivals in `x` time, given arrival rate `lam`. We\n", "compute that using the PMF of the Poisson distribution.\n", "\n", - "Finally, here’s the definition of PredictWaitTime:\n", + "Finally, here’s the definition of `PredictWaitTime`:\n", "\n", " def PredictWaitTime(pmf_zb, pmf_x):\n", " pmf_y = pmf_zb - pmf_x\n", @@ -327,17 +601,17 @@ "\n", "`pmf_zb` is the distribution of gaps between trains; `pmf_x` is the\n", "distribution of elapsed time, based on the observed number of\n", - "passengers. Since y = zb - x, we can compute\n", + "passengers. Since `y = zb - x`, we can compute\n", "\n", " pmf_y = pmf_zb - pmf_x\n", "\n", "The subtraction operator invokes `Pmf.__sub__`, which enumerates all\n", - "pairs of zb and x, computes the differences,\n", + "pairs of `zb` and `x`, computes the differences,\n", "and adds the results to `pmf_y`.\n", "\n", "The resulting Pmf includes some negative values, which we know are\n", "impossible. For example, if you arrive during a gap of 5 minutes, you\n", - "can’t wait more than 5 minutes. RemoveNegatives removes the\n", + "can’t wait more than 5 minutes. `RemoveNegatives` removes the\n", "impossible values from the distribution and renormalizes.\n", "\n", " def RemoveNegatives(pmf):\n", @@ -347,17 +621,17 @@ " pmf.Normalize()\n", "\n", "Figure [fig.redline3] shows the results. The prior distribution of\n", - "x is the same as the distribution of y in\n", - "Figure [fig.redline2]. The posterior distribution of x\n", + "`x` is the same as the distribution of `y` in\n", + "Figure [fig.redline2]. The posterior distribution of `x`\n", "shows that, after seeing 15 passengers on the platform, we believe that\n", "the time since the last train is probably 5-10 minutes. The predictive\n", - "distribution of y indicates that we expect the next train\n", + "distribution of `y` indicates that we expect the next train\n", "in less than 5 minutes, with about 80% confidence.\n", "\n", "Estimating the arrival rate\n", "---------------------------\n", "\n", - "![Prior and posterior distributions of lam based on five\n", + "![Prior and posterior distributions of `lam` based on five\n", "days of passenger data. ](figs/redline1.pdf)\n", "\n", "[fig.redline1]\n", @@ -386,8 +660,8 @@ " 18 5.4 12\n", " 4 5.8 11\n", "\n", - "where k1 is the number of passengers waiting when you\n", - "arrive, y is your wait time in minutes, and k2\n", + "where `k1` is the number of passengers waiting when you\n", + "arrive, `y` is your wait time in minutes, and `k2`\n", "is the number of passengers who arrive while you are waiting.\n", "\n", "Over the course of one week, you waited 18 minutes and saw 36 passengers\n", @@ -397,14 +671,14 @@ "$\\lambda$ and show how to use that distribution in the rest of the\n", "analysis.\n", "\n", - "ArrivalRate is a Suite that represents\n", - "hypotheses about $\\lambda$. As always, Likelihood takes a\n", + "`ArrivalRate` is a `Suite` that represents\n", + "hypotheses about $\\lambda$. As always, `Likelihood` takes a\n", "hypothesis and data, and computes the likelihood of the data under the\n", "hypothesis.\n", "\n", "In this case the hypothesis is a value of $\\lambda$. The data is a pair,\n", - "y, k, where y is a wait time and\n", - "k is the number of passengers that arrived.\n", + "`y, k`, where `y` is a wait time and\n", + "`k` is the number of passengers that arrived.\n", "\n", " class ArrivalRate(thinkbayes.Suite):\n", "\n", @@ -414,17 +688,17 @@ " like = thinkbayes.EvalPoissonPmf(k, lam * y)\n", " return like\n", "\n", - "This Likelihood might look familiar; it is almost identical\n", - "to Elapsed.Likelihood in Section [elapsed]. The difference\n", - "is that in Elapsed.Likelihood the hypothesis is\n", - "x, the elapsed time; in ArrivalRate.Likelihood\n", - "the hypothesis is lam, the arrival rate. But in both cases\n", - "the likelihood is the probability of seeing k arrivals in\n", - "some period of time, given lam.\n", + "This `Likelihood` might look familiar; it is almost identical\n", + "to `Elapsed.Likelihood` in Section [elapsed]. The difference\n", + "is that in `Elapsed.Likelihood` the hypothesis is\n", + "`x`, the elapsed time; in `ArrivalRate.Likelihood`\n", + "the hypothesis is `lam`, the arrival rate. But in both cases\n", + "the likelihood is the probability of seeing `k` arrivals in\n", + "some period of time, given `lam`.\n", "\n", - "ArrivalRateEstimator encapsulates the process of estimating\n", - "$\\lambda$. The parameter, `passenger_data`, is a list of k1, y,\n", - "k2 tuples, as in the table above.\n", + "`ArrivalRateEstimator` encapsulates the process of estimating\n", + "$\\lambda$. The parameter, `passenger_data`, is a list of `k1, y,\n", + "k2` tuples, as in the table above.\n", "\n", " class ArrivalRateEstimator(object):\n", "\n", @@ -439,9 +713,9 @@ " for k1, y, k2 in passenger_data:\n", " self.post_lam.Update((y, k2))\n", "\n", - "`__init__` builds hypos, which is a sequence of\n", - "hypothetical values for lam, then builds the prior\n", - "distribution, `prior_lam`. The for loop updates the prior\n", + "`__init__` builds `hypos`, which is a sequence of\n", + "hypothetical values for `lam`, then builds the prior\n", + "distribution, `prior_lam`. The `for` loop updates the prior\n", "with data, yielding the posterior distribution, `post_lam`.\n", "\n", "Figure [fig.redline1] shows the prior and posterior distributions. As\n", @@ -453,8 +727,8 @@ "Incorporating uncertainty\n", "-------------------------\n", "\n", - "![Predictive distributions of y for possible values of\n", - "lam. ](figs/redline4.pdf)\n", + "![Predictive distributions of `y` for possible values of\n", + "`lam`. ](figs/redline4.pdf)\n", "\n", "[fig.redline4]\n", "\n", @@ -473,7 +747,7 @@ " from the distribution of the parameter.\n", "\n", "We have already done steps (1) and (2). I wrote a class called\n", - "WaitMixtureEstimator to handle steps (3) and (4).\n", + "`WaitMixtureEstimator` to handle steps (3) and (4).\n", "\n", " class WaitMixtureEstimator(object):\n", "\n", @@ -486,30 +760,30 @@ "\n", " self.mixture = thinkbayes.MakeMixture(self.metapmf)\n", "\n", - "wtc is the WaitTimeCalculator that contains\n", - "the distribution of zb. are is the\n", - "ArrivalTimeEstimator that contains the distribution of\n", - "lam.\n", + "`wtc` is the `WaitTimeCalculator` that contains\n", + "the distribution of `zb`. `are` is the\n", + "`ArrivalTimeEstimator` that contains the distribution of\n", + "`lam`.\n", "\n", "The first line makes a meta-Pmf that maps from each possible\n", - "distribution of y to its probability. For each value of\n", - "lam, we use ElapsedTimeEstimator to compute\n", - "the corresponding distribution of y and store it in the\n", - "Meta-Pmf. Then we use MakeMixture to compute the mixture.\n", + "distribution of `y` to its probability. For each value of\n", + "`lam`, we use `ElapsedTimeEstimator` to compute\n", + "the corresponding distribution of `y` and store it in the\n", + "Meta-Pmf. Then we use `MakeMixture` to compute the mixture.\n", "\n", "Figure [fig.redline4] shows the results. The shaded lines in the\n", - "background are the distributions of y for each value of\n", - "lam, with line thickness that represents likelihood. The\n", + "background are the distributions of `y` for each value of\n", + "`lam`, with line thickness that represents likelihood. The\n", "dark line is the mixture of these distributions.\n", "\n", "In this case we could get a very similar result using a single point\n", - "estimate of lam. So it was not necessary, for practical\n", + "estimate of `lam`. So it was not necessary, for practical\n", "purposes, to include the uncertainty of the estimate.\n", "\n", "In general, it is important to include variability if the system\n", "response is non-linear; that is, if small changes in the input can cause\n", "big changes in the output. In this case, posterior variability in\n", - "lam is small and the system response is approximately\n", + "`lam` is small and the system response is approximately\n", "linear for small perturbations.\n", "\n", "Decision analysis {#decision-analysis}\n", @@ -530,7 +804,7 @@ "enough time that I can wait 15 minutes and still make my connection at\n", "South Station.\n", "\n", - "In that case I would like to know the probability that y\n", + "In that case I would like to know the probability that `y`\n", "exceeds 15 minutes as a function of `num_passengers`. It is easy enough\n", "to use the analysis from Section [elapsed] and run it for a range of\n", "`num_passengers`.\n", @@ -550,8 +824,8 @@ "\n", "But remember that my observations are biased. I am more likely to\n", "observe long delays because they affect a large number of passengers. So\n", - "we should treat my observations as a sample of zb rather\n", - "than z. Here’s how we can do that.\n", + "we should treat my observations as a sample of `zb` rather\n", + "than `z`. Here’s how we can do that.\n", "\n", "During my year of commuting, I took the Red Line home about 220 times.\n", "So I take the observed gap times, `gap_times`, generate a sample of 220\n", @@ -562,26 +836,26 @@ " sample_z = cdf_z.Sample(n)\n", " pmf_z = thinkbayes.MakePmfFromList(sample_z)\n", "\n", - "Next I bias `pmf_z` to get the distribution of zb, draw a\n", + "Next I bias `pmf_z` to get the distribution of `zb`, draw a\n", "sample, and then add in delays of 30, 40, and 50 minutes (expressed in\n", "seconds):\n", "\n", " cdf_zp = BiasPmf(pmf_z).MakeCdf()\n", " sample_zb = cdf_zp.Sample(n) + [1800, 2400, 3000]\n", "\n", - "Cdf.Sample is more efficient than Pmf.Sample,\n", + "`Cdf.Sample` is more efficient than `Pmf.Sample`,\n", "so it is usually faster to convert a Pmf to a Cdf before sampling.\n", "\n", - "Next I use the sample of zb to estimate a Pdf using KDE,\n", + "Next I use the sample of `zb` to estimate a Pdf using KDE,\n", "and then convert the Pdf to a Pmf:\n", "\n", " pdf_zb = thinkbayes.EstimatedPdf(sample_zb)\n", " xs = MakeRange(low=60)\n", " pmf_zb = pdf_zb.MakePmf(xs)\n", "\n", - "Finally I unbias the distribution of zb to get the\n", - "distribution of z, which I use to create the\n", - "WaitTimeCalculator:\n", + "Finally I unbias the distribution of `zb` to get the\n", + "distribution of `z`, which I use to create the\n", + "`WaitTimeCalculator`:\n", "\n", " pmf_z = UnbiasPmf(pmf_zb)\n", " wtc = WaitTimeCalculator(pmf_z)\n", @@ -595,9 +869,9 @@ " prob = 1 - cdf_y.Prob(minutes * 60)\n", "\n", "Given the number of passengers on the platform,\n", - "ProbLongWait makes an ElapsedTimeEstimator,\n", + "`ProbLongWait` makes an `ElapsedTimeEstimator`,\n", "extracts the distribution of wait time, and computes the probability\n", - "that wait time exceeds minutes.\n", + "that wait time exceeds `minutes`.\n", "\n", "Figure [fig.redline5] shows the result. When the number of passengers is\n", "less than 20, we infer that the system is operating normally, so the\n", @@ -622,31 +896,31 @@ "rush hour, that might not be a bad assumption, but there are some\n", "obvious exceptions. For example, if there is a special event nearby, a\n", "large number of people might arrive at the same time. In that case, the\n", - "estimate of lam would be too low, so the estimates of\n", - "x and y would be too high.\n", + "estimate of `lam` would be too low, so the estimates of\n", + "`x` and `y` would be too high.\n", "\n", "If special events are as common as major delays, it would be important\n", "to include them in the model. We could do that by extending the\n", - "distribution of lam to include occasional large values.\n", + "distribution of `lam` to include occasional large values.\n", "\n", "We started with the assumption that we know distribution of\n", - "z. As an alternative, a passenger could estimate\n", - "z, but it would not be easy. As a passenger, you only\n", - "observe only your own wait time, y. Unless you skip the\n", + "`z`. As an alternative, a passenger could estimate\n", + "`z`, but it would not be easy. As a passenger, you only\n", + "observe only your own wait time, `y`. Unless you skip the\n", "first train and wait for the second, you don’t observe the gap between\n", - "trains, z.\n", + "trains, `z`.\n", "\n", - "However, we could make some inferences about zb. If we note\n", + "However, we could make some inferences about `zb`. If we note\n", "the number of passengers waiting when we arrive, we can estimate the\n", - "elapsed time since the last train, x. Then we observe\n", - "y. If we add the posterior distribution of x\n", - "to the observed y, we get a distribution that represents\n", - "our posterior belief about the observed value of zb.\n", + "elapsed time since the last train, `x`. Then we observe\n", + "`y`. If we add the posterior distribution of `x`\n", + "to the observed `y`, we get a distribution that represents\n", + "our posterior belief about the observed value of `zb`.\n", "\n", "We can use this distribution to update our beliefs about the\n", - "distribution of zb. Finally, we can compute the inverse of\n", - "BiasPmf to get from the distribution of zb to\n", - "the distribution of z.\n", + "distribution of `zb`. Finally, we can compute the inverse of\n", + "`BiasPmf` to get from the distribution of `zb` to\n", + "the distribution of `z`.\n", "\n", "I leave this analysis as an exercise for the reader. One suggestion: you\n", "should read Chapter [species] first. You can find the outline of a\n", @@ -656,8 +930,8 @@ "Exercises\n", "---------\n", "\n", - "This exercise is from MacKay, *Information Theory, Inference, and\n", - "Learning Algorithms*:\n", + "This exercise is from MacKay, *Information Theory, Inference, and\n", + "Learning Algorithms*:\n", "\n", "> Unstable particles are emitted from a source and decay at a distance\n", "> $x$, a real number that has an exponential probability distribution\n", @@ -667,7 +941,7 @@ "> posterior distribution of $\\lambda$?\n", "\n", "You can download a solution to this exercise from\n", - ".\n" + "." ] } ], From 32094b37e407db4f9a19fd8f304dcfe3a0bd63c4 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Sun, 14 Feb 2016 12:45:39 -0800 Subject: [PATCH 04/10] Got it basically working A lot of things are non-ideal, but this is Alan's work, not mine, and I don't want to mae decisions for him. --- code/book_format.py | 5 + code/dice.py | 6 +- code/euro.py | 16 +- code/jaynes.py | 200 + code/kidney.py | 45 +- code/sat.py | 20 +- code/species.py | 98 +- code/train.py | 4 +- code/train3.py | 8 +- notebooks/01_Bayes_Theorem.ipynb | 1250 ++- notebooks/02_Computational_Statistics.ipynb | 2223 ++-- notebooks/03_Estimation.ipynb | 1582 ++- notebooks/04_More_Estimation.ipynb | 1900 ++-- notebooks/05_Odds_and_Addends.ipynb | 2305 ++-- notebooks/06_Decision_Analysis.ipynb | 2325 ++-- notebooks/07_Prediction.ipynb | 2043 ++-- notebooks/08_Observer_Bias.ipynb | 2243 ++-- notebooks/09_Two_Dimensions.ipynb | 1410 ++- .../10_Approximate_Bayesian_Computation.ipynb | 1928 ++-- notebooks/11_Hypothesis_Testing.ipynb | 1001 +- notebooks/12_Evidence.ipynb | 1551 ++- notebooks/13_Simulation.ipynb | 1591 ++- notebooks/14_A_Hierarchical_Model.ipynb | 972 +- notebooks/15_Dealing_with_Dimensions.ipynb | 2764 +++-- notebooks/Preface.ipynb | 825 +- notebooks/hockey_data.csv | 9841 +++++++++++++++++ notebooks/journal.pone.0047712.s001.csv | 3968 +++++++ notebooks/sat_ranks.csv | 67 + notebooks/sat_scale.csv | 18 + notebooks/showcases.2011.csv | 11 + notebooks/showcases.2012.csv | 10 + 31 files changed, 30549 insertions(+), 11681 deletions(-) create mode 100644 code/jaynes.py create mode 100644 notebooks/hockey_data.csv create mode 100644 notebooks/journal.pone.0047712.s001.csv create mode 100644 notebooks/sat_ranks.csv create mode 100644 notebooks/sat_scale.csv create mode 100644 notebooks/showcases.2011.csv create mode 100644 notebooks/showcases.2012.csv diff --git a/code/book_format.py b/code/book_format.py index 7aa75f1..57e5d89 100644 --- a/code/book_format.py +++ b/code/book_format.py @@ -24,6 +24,11 @@ import numpy as np import os.path import sys +import warnings + +# recent matplotlibs are raising deprecation warnings that +# we don't worry about. +warnings.filterwarnings("ignore") sys.path.insert(0, './code') # allow us to import book_format diff --git a/code/dice.py b/code/dice.py index 6984b43..0bc1edd 100644 --- a/code/dice.py +++ b/code/dice.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -27,13 +29,13 @@ def main(): suite = Dice([4, 6, 8, 12, 20]) suite.Update(6) - print 'After one 6' + print('After one 6') suite.Print() for roll in [4, 8, 7, 7, 2]: suite.Update(roll) - print 'After more rolls' + print('After more rolls') suite.Print() diff --git a/code/euro.py b/code/euro.py index 629a62b..46746ab 100644 --- a/code/euro.py +++ b/code/euro.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -89,17 +91,17 @@ def RunUpdate(suite, heads=140, tails=110): def Summarize(suite): """Prints summary statistics for the suite.""" - print suite.Prob(50) + print(suite.Prob(50)) - print 'MLE', suite.MaximumLikelihood() + print('MLE', suite.MaximumLikelihood()) - print 'Mean', suite.Mean() - print 'Median', thinkbayes.Percentile(suite, 50) + print('Mean', suite.Mean()) + print('Median', thinkbayes.Percentile(suite, 50)) - print '5th %ile', thinkbayes.Percentile(suite, 5) - print '95th %ile', thinkbayes.Percentile(suite, 95) + print('5th %ile', thinkbayes.Percentile(suite, 5)) + print('95th %ile', thinkbayes.Percentile(suite, 95)) - print 'CI', thinkbayes.CredibleInterval(suite, 90) + print('CI', thinkbayes.CredibleInterval(suite, 90)) def PlotSuites(suites, root): diff --git a/code/jaynes.py b/code/jaynes.py new file mode 100644 index 0000000..7f0833f --- /dev/null +++ b/code/jaynes.py @@ -0,0 +1,200 @@ +from __future__ import print_function + +"""This file contains code used in "Think Stats", +by Allen B. Downey, available from greenteapress.com + +Copyright 2013 Allen B. Downey +License: GNU GPLv3 http://www.gnu.org/licenses/gpl.html +""" + +import thinkbayes +import thinkplot + +from math import exp + +"""This file contains a solution to an exercise from Think Bayes, +by Allen B. Downey + +I got the idea from Tom Campbell-Ricketts author of the Maximum +Entropy blog at + +http://maximum-entropy-blog.blogspot.com + +And he got the idea from E.T. Jaynes, author of the classic +_Probability Theory: The Logic of Science_. + +Here's the version from Think Bayes: + +Radioactive decay is well modeled by a Poisson process; the +probability that an atom decays is the same at any point in time. + +Suppose that a radioactive source emits particles toward a Geiger +counter at an average rate of $r$ particles per second, but the +counter only registers a fraction, $f$, of the particles that hit it. +If $f$ is 10\% and the counter registers 15 particles in a one second +interval, what is the posterior distribution of $n$, the actual number +of particles that hit the counter, and $p$, the average rate particles +are emitted? + +""" + +FORMATS = ['pdf', 'eps', 'png'] + +class Emitter(thinkbayes.Suite): + """Represents hypotheses about r.""" + + def __init__(self, rs, f=0.1): + """Initializes the Suite. + + rs: sequence of hypothetical emission rates + f: fraction of particles registered + """ + detectors = [Detector(r, f) for r in rs] + thinkbayes.Suite.__init__(self, detectors) + + def Update(self, data): + """Updates the Suite based on data. + + data: number of particles counted + """ + thinkbayes.Suite.Update(self, data) + + for detector in self.Values(): + detector.Update() + + def Likelihood(self, data, hypo): + """Likelihood of the data given the hypothesis. + + Args: + data: number of particles counted + hypo: emission rate, r + + Returns: + probability density of the data under the hypothesis + """ + detector = hypo + like = detector.SuiteLikelihood(data) + return like + + def DistOfR(self, name=''): + """Returns the PMF of r.""" + items = [(detector.r, prob) for detector, prob in self.Items()] + return thinkbayes.MakePmfFromItems(items, name=name) + + def DistOfN(self, name=''): + """Returns the PMF of n.""" + return thinkbayes.MakeMixture(self, name=name) + + +class Emitter2(thinkbayes.Suite): + """Represents hypotheses about r.""" + + def __init__(self, rs, f=0.1): + """Initializes the Suite. + + rs: sequence of hypothetical emission rates + f: fraction of particles registered + """ + detectors = [Detector(r, f) for r in rs] + thinkbayes.Suite.__init__(self, detectors) + + def Likelihood(self, data, hypo): + """Likelihood of the data given the hypothesis. + + Args: + data: number of counted per unit time + hypo: emission rate, r + + Returns: + probability density of the data under the hypothesis + """ + return hypo.Update(data) + + def DistOfR(self, name=''): + """Returns the PMF of r.""" + items = [(detector.r, prob) for detector, prob in self.Items()] + return thinkbayes.MakePmfFromItems(items, name=name) + + def DistOfN(self, name=''): + """Returns the PMF of n.""" + return thinkbayes.MakeMixture(self, name=name) + + +class Detector(thinkbayes.Suite): + """Represents hypotheses about n.""" + + def __init__(self, r, f, high=500, step=5): + """Initializes the suite. + + r: known emission rate, r + f: fraction of particles registered + high: maximum number of particles, n + step: step size between hypothetical values of n + """ + pmf = thinkbayes.MakePoissonPmf(r, high, step=step) + thinkbayes.Suite.__init__(self, pmf, name=r) + self.r = r + self.f = f + + def Likelihood(self, data, hypo): + """Likelihood of the data given the hypothesis. + + data: number of particles counted + hypo: number of particles hitting the counter, n + """ + k = data + n = hypo + p = self.f + + return thinkbayes.EvalBinomialPmf(k, n, p) + + def SuiteLikelihood(self, data): + """Adds up the total probability of the data under the suite. + + data: number of particles counted + """ + total = 0 + for hypo, prob in self.Items(): + like = self.Likelihood(data, hypo) + total += prob * like + return total + + +def main(): + k = 15 + f = 0.1 + + # plot Detector suites for a range of hypothetical r + thinkplot.PrePlot(num=3) + for r in [100, 250, 400]: + suite = Detector(r, f, step=1) + suite.Update(k) + thinkplot.Pmf(suite) + print(suite.MaximumLikelihood()) + + thinkplot.Save(root='jaynes1', + xlabel='Number of particles (n)', + ylabel='PMF', + formats=FORMATS) + + # plot the posterior distributions of r and n + hypos = range(1, 501, 5) + suite = Emitter2(hypos, f=f) + suite.Update(k) + + thinkplot.PrePlot(num=2) + post_r = suite.DistOfR(name='posterior r') + post_n = suite.DistOfN(name='posterior n') + + thinkplot.Pmf(post_r) + thinkplot.Pmf(post_n) + + thinkplot.Save(root='jaynes2', + xlabel='Emission rate', + ylabel='PMF', + formats=FORMATS) + + + +if __name__ == '__main__': + main() diff --git a/code/kidney.py b/code/kidney.py index 6419cbd..733395f 100644 --- a/code/kidney.py +++ b/code/kidney.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -44,12 +46,12 @@ def SimpleModel(): d1 = 15.5 d0 = d1 / 2.0 ** doublings - print 'interval (days)', interval - print 'interval (years)', interval / 365 - print 'dt', dt - print 'doublings', doublings - print 'd1', d1 - print 'd0', d0 + print('interval (days)', interval) + print('interval (years)', interval / 365) + print('dt', dt) + print('doublings', doublings) + print('d1', d1) + print('d0', d0) # assume an initial linear measure of 0.1 cm d0 = 0.1 @@ -61,19 +63,19 @@ def SimpleModel(): # what linear doubling time does that imply? dt = interval / doublings - print 'doublings', doublings - print 'dt', dt + print('doublings', doublings) + print('dt', dt) # compute the volumetric doubling time and RDT vdt = dt / 3 rdt = 365 / vdt - print 'vdt', vdt - print 'rdt', rdt + print('vdt', vdt) + print('rdt', rdt) cdf = MakeCdf() p = cdf.Prob(rdt) - print 'Prob{RDT > 2.4}', 1-p + print('Prob{RDT > 2.4}', 1-p) def MakeCdf(): @@ -321,7 +323,10 @@ def __init__(self): def GetBuckets(self): """Returns an iterator for the keys in the cache.""" - return self.sequences.iterkeys() + try: # python 2 + return self.sequences.iterkeys() + except: # python 3 + return self.sequences.keys() def GetSequence(self, bucket): """Looks up a bucket in the cache.""" @@ -389,7 +394,7 @@ def Print(self): for bucket in sorted(self.GetBuckets()): ss = self.GetSequence(bucket) diameter = BucketToCm(bucket) - print diameter, len(ss) + print(diameter, len(ss)) def Correlation(self): """Computes the correlation between log volumes and rdts.""" @@ -421,7 +426,7 @@ def MakeSequences(self, n, rho, cdf): sequences.append(seq) if i % 100 == 0: - print i + print(i) return sequences @@ -665,7 +670,7 @@ def PrintTable(fp, xs, ts): fp.write(r'\hline' '\n') for i, (cm, ps) in enumerate(zip(xs, ts)): - #print cm, ps + #print(cm, ps) if i % 3 == 0: PrintCI(fp, cm, ps) @@ -727,7 +732,7 @@ def TestCorrelation(cdf): xs = [rdt_seq.next() for _ in range(n)] rho2 = correlation.SerialCorr(xs) - print rho, rho2 + print(rho, rho2) cdf2 = thinkbayes.MakeCdfFromList(xs) thinkplot.Cdfs([cdf, cdf2]) @@ -737,7 +742,7 @@ def TestCorrelation(cdf): def main(script): for size in [1, 5, 10]: bucket = CmToBucket(size) - print 'Size, bucket', size, bucket + print('Size, bucket', size, bucket) SimpleModel() @@ -761,10 +766,10 @@ def main(script): calc.PlotBuckets() _ = calc.MakeSequences(1900, rho, fit) - print 'V0-RDT correlation', calc.cache.Correlation() + print('V0-RDT correlation', calc.cache.Correlation()) - print '15.5 Probability age > 8 year', calc.cache.ProbOlder(15.5, 8) - print '6.0 Probability age > 8 year', calc.cache.ProbOlder(6.0, 8) + print('15.5 Probability age > 8 year', calc.cache.ProbOlder(15.5, 8)) + print('6.0 Probability age > 8 year', calc.cache.ProbOlder(6.0, 8)) calc.PlotConditionalCdfs() diff --git a/code/sat.py b/code/sat.py index 9a9af93..b267f47 100644 --- a/code/sat.py +++ b/code/sat.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code used in "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -130,10 +132,10 @@ def CompareScores(self, a_score, b_score, constructor): ratio = top.Prob('A') / top.Prob('B') - print 'Likelihood ratio', ratio + print('Likelihood ratio', ratio) posterior = ratio / (ratio + 1) - print 'Posterior', posterior + print('Posterior', posterior) if constructor is Sat2: ComparePosteriorPredictive(a_sat, b_sat) @@ -341,10 +343,10 @@ def ComparePosteriorPredictive(a_sat, b_sat): b_like = thinkbayes.PmfProbLess(a_pred, b_pred) c_like = thinkbayes.PmfProbEqual(a_pred, b_pred) - print 'Posterior predictive' - print 'A', a_like - print 'B', b_like - print 'C', c_like + print('Posterior predictive') + print('A', a_like) + print('B', b_like) + print('C', c_like) def PlotPriorDist(pmf): @@ -441,11 +443,11 @@ def ProbCorrectTable(): difficulties = [-1.85, -0.05, 1.75] for eff in efficacies: - print '%0.2f & ' % eff, + print('%0.2f & ' % eff, end=' ') for diff in difficulties: p = ProbCorrect(eff, diff) - print '%0.2f & ' % p, - print r'\\' + print('%0.2f & ' % p, end=' ') + print(r'\\') def main(script): diff --git a/code/species.py b/code/species.py index 576cb50..fc8ae36 100644 --- a/code/species.py +++ b/code/species.py @@ -1,3 +1,5 @@ +#from __future__ import print_function + """This file contains code used in "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -107,7 +109,7 @@ def prob_bogus(k, r): p = (1-q) ** k return p - print self.code, clean_param + print(self.code, clean_param) counts = self.GetCounts() r = 1.0 * sum(counts) @@ -145,7 +147,7 @@ def GetNames(self): def PrintCounts(self): """Prints the counts and species names.""" for count, name in reversed(self.species): - print count, name + print(count, name) def GetSpecies(self, index): """Gets the count and name of the indicated species. @@ -191,7 +193,7 @@ def Process(self, low=None, high=500, conc=1, iters=100): self.suite.Update(counts) #end = time.time() - #print 'Processing time' end-start + #print('Processing time' end-start)) def MakePrediction(self, num_sims=100): """Make predictions for the given subject. @@ -243,7 +245,7 @@ def MakeFigures(self): def PlotDistN(self): """Plots distribution of n.""" pmf = self.suite.DistN() - print '90% CI for N:', pmf.CredibleInterval(90) + print('90% CI for N:', pmf.CredibleInterval(90)) pmf.name = self.code thinkplot.Clf() @@ -290,8 +292,8 @@ def PlotPrevalence(self, rank=1, cdf_flag=True): count, _ = self.GetSpecies(index) mix.name = '%d (%d)' % (rank, count) - print '90%% CI for prevalence of species %d:' % rank, - print mix.CredibleInterval(90) + print('90%% CI for prevalence of species %d:' % rank, end=' ') + print(mix.CredibleInterval(90)) if cdf_flag: cdf = mix.MakeCdf() @@ -307,8 +309,8 @@ def PlotMixture(self, rank=1): # convert rank to index index = self.GetM() - rank - print self.GetSpecies(index) - print self.GetCounts()[index] + print(self.GetSpecies(index)) + print(self.GetCounts()[index]) metapmf, mix = self.suite.DistOfPrevalence(index) @@ -354,7 +356,7 @@ def GenerateObservations(self, num_reads): observations = cdf.Sample(num_reads) #for ob in observations: - # print ob + # print(ob return n, observations @@ -478,8 +480,8 @@ def MakeConditionals(curves, ks): pmf.name = 'k=%d' % k cdf = pmf.MakeCdf() cdfs.append(cdf) - print '90%% credible interval for %d' % k, - print cdf.CredibleInterval(90) + print('90%% credible interval for %d' % k, end=' ') + print(cdf.CredibleInterval(90)) return cdfs @@ -620,7 +622,7 @@ def ReadCompleteDataset(filename='BBB_data_from_Rob.csv', clean_param=0): counts = [int(x) for x in t[1:-1]] - # print otu_code, species + # print(otu_code, species for code, count in zip(subject_codes, counts): if count > 0: @@ -825,12 +827,12 @@ def Preload(self, data): m = len(data) singletons = data.count(1) num = m - singletons - print m, singletons, num + print(m, singletons, num) addend = numpy.ones(num, dtype=numpy.float) * 1 - print len(addend) - print len(self.params[singletons:m]) + print(len(addend)) + print(len(self.params[singletons:m])) self.params[singletons:m] += addend - print 'Preload', num + print('Preload', num) def Update(self, data): """Updates the distribution based on data. @@ -972,7 +974,7 @@ def SamplePosterior(self): n = self.RandomN() prevalences = self.SamplePrevalences(n) - #print 'Peeking at n_cheat' + #print('Peeking at n_cheat') #n = n_cheat return n, prevalences @@ -1011,7 +1013,7 @@ def Unbias(self, n, m, q_desired): x = sum(params[:m]) y = sum(params[m:]) a = x + y - #print x, y, a, x/a, y/a + #print(x, y, a, x/a, y/a g = q_desired * a / y f = (a - g * y) / x @@ -1207,11 +1209,11 @@ def MakePosterior(constructor, data, ns, conc=1, iters=1000): """ suite = constructor(ns, conc=conc, iters=iters) - # print constructor.__name__ + # print(constructor.__name__ start = time.time() suite.Update(data) end = time.time() - print 'Processing time', end-start + print('Processing time', end-start) return suite @@ -1264,12 +1266,12 @@ def SimpleDirichletExample(): dirichlet = thinkbayes.Dirichlet(3) for i in range(3): beta = dirichlet.MarginalBeta(i) - print 'mean', names[i], beta.Mean() + print('mean', names[i], beta.Mean()) dirichlet.Update(data) for i in range(3): beta = dirichlet.MarginalBeta(i) - print 'mean', names[i], beta.Mean() + print('mean', names[i], beta.Mean()) pmf = beta.MakePmf(name=names[i]) thinkplot.Pmf(pmf) @@ -1341,8 +1343,8 @@ def ProcessSubjects(codes): pmfs.append(pmf) - print 'ProbGreater', thinkbayes.PmfProbGreater(pmfs[0], pmfs[1]) - print 'ProbLess', thinkbayes.PmfProbLess(pmfs[0], pmfs[1]) + print('ProbGreater', thinkbayes.PmfProbGreater(pmfs[0], pmfs[1])) + print('ProbLess', thinkbayes.PmfProbLess(pmfs[0], pmfs[1])) thinkplot.Save(root='species4', xlabel='Number of species', @@ -1390,24 +1392,24 @@ def RunSubject(code, conc=1, high=500): def PrintSummary(subject): - """Print a summary of a subject. + """print(a summary of a subject. subject: Subject """ - print subject.code - print 'found %d species in %d reads' % (subject.num_species, - subject.num_reads) + print(subject.code) + print('found %d species in %d reads' % (subject.num_species, + subject.num_reads)) - print 'total %d species in %d reads' % (subject.total_species, - subject.total_reads) + print('total %d species in %d reads' % (subject.total_species, + subject.total_reads)) cdf = subject.suite.DistN().MakeCdf() - print 'n' + print('n') PrintPrediction(cdf, 'unknown') def PrintPrediction(cdf, actual): - """Print a summary of a prediction. + """print(a summary of a prediction. cdf: predictive distribution actual: actual value @@ -1415,8 +1417,8 @@ def PrintPrediction(cdf, actual): median = cdf.Percentile(50) low, high = cdf.CredibleInterval(75) - print 'predicted %0.2f (%0.2f %0.2f)' % (median, low, high) - print 'actual', actual + print('predicted %0.2f (%0.2f %0.2f)' % (median, low, high)) + print('actual', actual) def RandomSeed(x): @@ -1490,7 +1492,7 @@ def PlotActualPrevalences(): continue actual_max = max(prevalences) - print code, m, actual_max + print(code, m, actual_max) # incr the PMFs if m > 50: @@ -1593,7 +1595,7 @@ def Validate(self, num_runs=100, clean_param=0): continue num_reads = 100 - print 'Validate', match.code + print('Validate', match.code) subject = match.Resample(num_reads) subject.Match(match) @@ -1647,9 +1649,9 @@ def PlotL(self, root='species-n'): def PlotCalibrationCurves(self, root='species5'): """Plots calibration curves""" - print self.total_n - print self.total_q - print self.total_l + print(self.total_n) + print(self.total_q) + print(self.total_l) thinkplot.Plot([0, 100], [0, 100], color='gray', alpha=0.2) @@ -1689,14 +1691,14 @@ def RunCalibration(self, seed, n_low, n_high, r, tr): self.conc) l_actual = len(hist) - len(subhist) - print 'Run low, high, conc', n_low, n_high, self.conc - print 'Run r, tr', r, tr - print 'Run n, q, l', n_actual, q_actual, l_actual + print('Run low, high, conc', n_low, n_high, self.conc) + print('Run r, tr', r, tr) + print('Run n, q, l', n_actual, q_actual, l_actual) # extract the data data = [count for species, count in subhist.Items()] data.sort() - print 'data', data + print('data', data) # make a Subject and process subject = Subject('simulated') @@ -1726,17 +1728,17 @@ def RunSubject(self, subject, n_actual, q_actual, l_actual): # check the distribution of n pmf_n = suite.DistN() - print 'n' + print('n') self.total_n += self.CheckDistribution(pmf_n, n_actual, self.n_seq) # check the distribution of q pmf_q = suite.DistQ() - print 'q' + print('q') self.total_q += self.CheckDistribution(pmf_q, q_actual, self.q_seq) # check the distribution of additional species pmf_l = subject.DistL() - print 'l' + print('l') self.total_l += self.CheckDistribution(pmf_l, l_actual, self.l_seq) def CheckDistribution(self, pmf, actual, seq): @@ -1847,12 +1849,12 @@ def PlotSubjectCdf(code=None, clean_param=0): else: subject = subject_map[code] - print subject.code + print(subject.code) m = subject.GetM() subject.Process(high=m, conc=0.1, iters=0) - print subject.suite.params[:m] + print(subject.suite.params[:m]) # plot the cdf options = dict(linewidth=3, color='blue', alpha=0.5) diff --git a/code/train.py b/code/train.py index e6ea01d..cc9038a 100644 --- a/code/train.py +++ b/code/train.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -22,7 +24,7 @@ def main(): suite = Train(hypos) suite.Update(60) - print suite.Mean() + print(suite.Mean()) thinkplot.PrePlot(1) thinkplot.Pmf(suite) diff --git a/code/train3.py b/code/train3.py index 73cd534..bafb8a6 100644 --- a/code/train3.py +++ b/code/train3.py @@ -1,3 +1,5 @@ +from __future__ import print_function + """This file contains code for use with "Think Bayes", by Allen B. Downey, available from greenteapress.com @@ -80,18 +82,18 @@ def main(): for high in [500, 1000, 2000]: suite = MakePosterior(high, dataset, Train2) - print high, suite.Mean() + print(high, suite.Mean()) thinkplot.Save(root='train3', xlabel='Number of trains', ylabel='Probability') interval = Percentile(suite, 5), Percentile(suite, 95) - print interval + print(interval) cdf = thinkbayes.MakeCdfFromPmf(suite) interval = cdf.Percentile(5), cdf.Percentile(95) - print interval + print(interval) if __name__ == '__main__': diff --git a/notebooks/01_Bayes_Theorem.ipynb b/notebooks/01_Bayes_Theorem.ipynb index 27a0eeb..a140099 100644 --- a/notebooks/01_Bayes_Theorem.ipynb +++ b/notebooks/01_Bayes_Theorem.ipynb @@ -1,476 +1,774 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Bayes’s Theorem {#intro}\n", - "\n", - "##Conditional probability\n", - "\n", - "The fundamental idea behind all Bayesian statistics is Bayes’s theorem,\n", - "which is surprisingly easy to derive, provided that you understand\n", - "conditional probability. So we’ll start with probability, then\n", - "conditional probability, then Bayes’s theorem, and on to Bayesian\n", - "statistics.\n", - "\n", - "A probability is a number between 0 and 1 (including both) that\n", - "represents a degree of belief in a fact or prediction. The value 1\n", - "represents certainty that a fact is true, or that a prediction will come\n", - "true. The value 0 represents certainty that the fact is false.\n", - "\n", - "Intermediate values represent degrees of certainty. The value 0.5, often\n", - "written as 50%, means that a predicted outcome is as likely to happen as\n", - "not. For example, the probability that a tossed coin lands face up is\n", - "very close to 50%.\n", - "\n", - "A conditional probability is a probability based on some background\n", - "information. For example, I want to know the probability that I will\n", - "have a heart attack in the next year. According to the CDC, “Every year\n", - "about 785,000 Americans have a first coronary attack.\n", - "()”\n", - "\n", - "The U.S. population is about 311 million, so the probability that a\n", - "randomly chosen American will have a heart attack in the next year is\n", - "roughly 0.3%.\n", - "\n", - "But I am not a randomly chosen American. Epidemiologists have identified\n", - "many factors that affect the risk of heart attacks; depending on those\n", - "factors, my risk might be higher or lower than average.\n", - "\n", - "I am male, 45 years old, and I have borderline high cholesterol. Those\n", - "factors increase my chances. However, I have low blood pressure and I\n", - "don’t smoke, and those factors decrease my chances.\n", - "\n", - "Plugging everything into the online calculator at\n", - ", I find that my risk of a\n", - "heart attack in the next year is about 0.2%, less than the national\n", - "average. That value is a conditional probability, because it is based on\n", - "a number of factors that make up my “condition.”\n", - "\n", - "The usual notation for conditional probability is\n", - "$\\mathrm{p}(A|B)$, which is the probability of $A$ given\n", - "that $B$ is true. In this example, $A$ represents the prediction that I\n", - "will have a heart attack in the next year, and $B$ is the set of\n", - "conditions I listed.\n", - "\n", - "##Conjoint probability\n", - "\n", - "**Conjoint probability** is a fancy way to say the\n", - "probability that two things are true. I write\n", - "$\\mathrm{p}(A {~\\mathrm{and}~}B)$ to mean the probability\n", - "that $A$ and $B$ are both true.\n", - "\n", - "If you learned about probability in the context of coin tosses and dice,\n", - "you might have learned the following formula:\n", - "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B)}} \\quad\\quad\\mbox{WARNING: not always true}$$\n", - "For example, if I toss two coins, and $A$ means the first coin lands\n", - "face up, and $B$ means the second coin lands face up, then\n", - "${{\\mathrm{p}(A)}} =\n", - "{{\\mathrm{p}(B)}} = 0.5$, and sure enough,\n", - "${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B)}} = 0.25$.\n", - "\n", - "But this formula only works because in this case $A$ and $B$ are\n", - "independent; that is, knowing the outcome of the first event does not\n", - "change the probability of the second. Or, more formally,\n", - "$\\mathrm{p}(B|A)$ = $\\mathrm{p}(B)$.\n", - "\n", - "Here is a different example where the events are not independent.\n", - "Suppose that $A$ means that it rains today and $B$ means that it rains\n", - "tomorrow. If I know that it rained today, it is more likely that it will\n", - "rain tomorrow, so\n", - "${{\\mathrm{p}(B|A)}} > {{\\mathrm{p}(B)}}$.\n", - "\n", - "In general, the probability of a conjunction is\n", - "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", - "for any $A$ and $B$. So if the chance of rain on any given day is 0.5,\n", - "the chance of rain on two consecutive days is not 0.25, but probably a\n", - "bit higher.\n", - "\n", - "##The cookie problem\n", - "\n", - "We’ll get to Bayes’s theorem soon, but I want to motivate it with an\n", - "example called the cookie problem.[^1] Suppose there are two bowls of\n", - "cookies. Bowl 1 contains 30 vanilla cookies and 10 chocolate cookies.\n", - "Bowl 2 contains 20 of each.\n", - "\n", - "Now suppose you choose one of the bowls at random and, without looking,\n", - "select a cookie at random. The cookie is vanilla. What is the\n", - "probability that it came from Bowl 1?\n", - "\n", - "This is a conditional probability; we want\n", - "$\\mathrm{p}({\\mbox{Bowl 1}} |\n", - " {\\mbox{vanilla}})$, but it is not obvious how to compute it. If\n", - "I asked a different question—the probability of a vanilla cookie given\n", - "Bowl 1—it would be easy:\n", - "$${{\\mathrm{p}({\\mbox{vanilla}} | {\\mbox{Bowl 1}})}} = 3/4$$\n", - "Sadly, $\\mathrm{p}(A|B)$ is *not* the same as\n", - "$\\mathrm{p}(B|A)$, but there is a way to get from one to\n", - "the other: Bayes’s theorem.\n", - "\n", - "##Bayes’s theorem {#bayess-theorem}\n", - "\n", - "At this point we have everything we need to derive Bayes’s theorem.\n", - "We’ll start with the observation that conjunction is commutative; that\n", - "is\n", - "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(B {~\\mathrm{and}~}A)}}$$\n", - "for any events $A$ and $B$.\n", - "\n", - "Next, we write the probability of a conjunction:\n", - "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", - "Since we have not said anything about what $A$ and $B$ mean, they are\n", - "interchangeable. Interchanging them yields\n", - "$${{\\mathrm{p}(B {~\\mathrm{and}~}A)}} = {{\\mathrm{p}(B)}}~{{\\mathrm{p}(A|B)}}$$\n", - "That’s all we need. Pulling those pieces together, we get\n", - "$${{\\mathrm{p}(B)}}~{{\\mathrm{p}(A|B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", - "Which means there are two ways to compute the conjunction. If you have\n", - "$\\mathrm{p}(A)$, you multiply by the conditional\n", - "probability $\\mathrm{p}(B|A)$. Or you can do it the other\n", - "way around; if you know $\\mathrm{p}(B)$, you multiply by\n", - "$\\mathrm{p}(A|B)$. Either way you should get the same\n", - "thing.\n", - "\n", - "Finally we can divide through by $\\mathrm{p}(B)$:\n", - "$${{\\mathrm{p}(A|B)}} = \\frac{{{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}}{{{\\mathrm{p}(B)}}}$$\n", - "And that’s Bayes’s theorem! It might not look like much, but it turns\n", - "out to be surprisingly powerful.\n", - "\n", - "For example, we can use it to solve the cookie problem. I’ll write $B_1$\n", - "for the hypothesis that the cookie came from Bowl 1 and $V$ for the\n", - "vanilla cookie. Plugging in Bayes’s theorem we get\n", - "$${{\\mathrm{p}(B_1|V)}} = \\frac{{{\\mathrm{p}(B_1)}}~{{\\mathrm{p}(V|B_1)}}}{{{\\mathrm{p}(V)}}}$$\n", - "The term on the left is what we want: the probability of Bowl 1, given\n", - "that we chose a vanilla cookie. The terms on the right are:\n", - "\n", - "- ${{\\mathrm{p}(B_1)}}$: This is the probability that we\n", - " chose Bowl 1, unconditioned by what kind of cookie we got. Since the\n", - " problem says we chose a bowl at random, we can assume\n", - " ${{\\mathrm{p}(B_1)}} = 1/2$.\n", - "\n", - "- ${{\\mathrm{p}(V|B_1)}}$: This is the probability of\n", - " getting a vanilla cookie from Bowl 1, which is 3/4.\n", - "\n", - "- $\\mathrm{p}(V)$: This is the probability of drawing a\n", - " vanilla cookie from either bowl. Since we had an equal chance of\n", - " choosing either bowl and the bowls contain the same number of\n", - " cookies, we had the same chance of choosing any cookie. Between the\n", - " two bowls there are 50 vanilla and 30 chocolate cookies, so\n", - " $\\mathrm{p}(V)$ = 5/8.\n", - "\n", - "Putting it together, we have\n", - "$${{\\mathrm{p}(B_1|V)}} = \\frac{(1/2)~(3/4)}{5/8}$$ which\n", - "reduces to 3/5. So the vanilla cookie is evidence in favor of the\n", - "hypothesis that we chose Bowl 1, because vanilla cookies are more likely\n", - "to come from Bowl 1.\n", - "\n", - "This example demonstrates one use of Bayes’s theorem: it provides a\n", - "strategy to get from $\\mathrm{p}(B|A)$ to\n", - "$\\mathrm{p}(A|B)$. This strategy is useful in cases, like\n", - "the cookie problem, where it is easier to compute the terms on the right\n", - "side of Bayes’s theorem than the term on the left.\n", - "\n", - "##The diachronic interpretation\n", - "\n", - "There is another way to think of Bayes’s theorem: it gives us a way to\n", - "update the probability of a hypothesis, $H$, in light of some body of\n", - "data, $D$.\n", - "\n", - "This way of thinking about Bayes’s theorem is called the\n", - "**diachronic interpretation**. “Diachronic” means that\n", - "something is happening over time; in this case the probability of the\n", - "hypotheses changes, over time, as we see new data.\n", - "\n", - "Rewriting Bayes’s theorem with $H$ and $D$ yields:\n", - "$${{\\mathrm{p}(H|D)}} = \\frac{{{\\mathrm{p}(H)}}~{{\\mathrm{p}(D|H)}}}{{{\\mathrm{p}(D)}}}$$\n", - "In this interpretation, each term has a name:\n", - "\n", - "- $\\mathrm{p}(H)$ is the probability of the hypothesis\n", - " before we see the data, called the prior probability, or just\n", - " **prior**.\n", - "\n", - "- $\\mathrm{p}(H|D)$ is what we want to compute, the\n", - " probability of the hypothesis after we see the data, called the\n", - " **posterior**.\n", - "\n", - "- $\\mathrm{p}(D|H)$ is the probability of the data under\n", - " the hypothesis, called the **likelihood**.\n", - "\n", - "- $\\mathrm{p}(D)$ is the probability of the data under\n", - " any hypothesis, called the **normalizing constant**.\n", - "\n", - "Sometimes we can compute the prior based on background information. For\n", - "example, the cookie problem specifies that we choose a bowl at random\n", - "with equal probability.\n", - "\n", - "In other cases the prior is subjective; that is, reasonable people might\n", - "disagree, either because they use different background information or\n", - "because they interpret the same information differently.\n", - "\n", - "The likelihood is usually the easiest part to compute. In the cookie\n", - "problem, if we know which bowl the cookie came from, we find the\n", - "probability of a vanilla cookie by counting.\n", - "\n", - "The normalizing constant can be tricky. It is supposed to be the\n", - "probability of seeing the data under any hypothesis at all, but in the\n", - "most general case it is hard to nail down what that means.\n", - "\n", - "Most often we simplify things by specifying a set of hypotheses that are\n", - "\n", - "Mutually exclusive:\n", - ": At most one hypothesis in the set can be true, and\n", - "\n", - "Collectively exhaustive:\n", - ": There are no other possibilities; at least one of the hypotheses has\n", - " to be true.\n", - "\n", - "I use the word **suite** for a set of hypotheses that has\n", - "these properties.\n", - "\n", - "In the cookie problem, there are only two hypotheses—the cookie came\n", - "from Bowl 1 or Bowl 2—and they are mutually exclusive and collectively\n", - "exhaustive.\n", - "\n", - "In that case we can compute $\\mathrm{p}(D)$ using the law\n", - "of total probability, which says that if there are two exclusive ways\n", - "that something might happen, you can add up the probabilities like this:\n", - "$${{\\mathrm{p}(D)}} = {{\\mathrm{p}(B_1)}}~{{\\mathrm{p}(D|B_1)}} + {{\\mathrm{p}(B_2)}}~{{\\mathrm{p}(D|B_2)}}$$\n", - "Plugging in the values from the cookie problem, we have\n", - "$${{\\mathrm{p}(D)}} = (1/2)~(3/4) + (1/2)~(1/2) = 5/8$$ which\n", - "is what we computed earlier by mentally combining the two bowls." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The M&M problem\n", - "\n", - "M&M’s are small candy-coated chocolates that come in a\n", - "variety of colors. Mars, Inc., which makes M&M’s, changes\n", - "the mixture of colors from time to time.\n", - "\n", - "In 1995, they introduced blue M&M’s. Before then, the color\n", - "mix in a bag of plain M&M’s was 30% Brown, 20% Yellow, 20%\n", - "Red, 10% Green, 10% Orange, 10% Tan. Afterward it was 24% Blue , 20%\n", - "Green, 16% Orange, 14% Yellow, 13% Red, 13% Brown.\n", - "\n", - "Suppose a friend of mine has two bags of M&M’s, and he\n", - "tells me that one is from 1994 and one from 1996. He won’t tell me which\n", - "is which, but he gives me one M&M from each bag. One is\n", - "yellow and one is green. What is the probability that the yellow one\n", - "came from the 1994 bag?\n", - "\n", - "This problem is similar to the cookie problem, with the twist that I\n", - "draw one sample from each bowl/bag. This problem also gives me a chance\n", - "to demonstrate the table method, which is useful for solving problems\n", - "like this on paper. In the next chapter we will solve them\n", - "computationally.\n", - "\n", - "The first step is to enumerate the hypotheses. The bag the yellow\n", - "M&M came from I’ll call Bag 1; I’ll call the other Bag 2.\n", - "So the hypotheses are:\n", - "\n", - "- A: Bag 1 is from 1994, which implies that Bag 2 is from 1996.\n", - "\n", - "- B: Bag 1 is from 1996 and Bag 2 from 1994.\n", - "\n", - "Now we construct a table with a row for each hypothesis and a column for\n", - "each term in Bayes’s theorem:\n", - "\n", - "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", - "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", - "| A | 1/2 | (20)(20) | 200 | 20/27|\n", - "| B | 1/2 | (14)(10) | 70 | 7/27|\n", - "\n", - "\n", - "The first column has the priors. Based on the statement of the problem,\n", - "it is reasonable to choose\n", - "${{\\mathrm{p}(A)}} = {{\\mathrm{p}(B)}} = 1/2$.\n", - "\n", - "The second column has the likelihoods, which follow from the information\n", - "in the problem. For example, if $A$ is true, the yellow\n", - "M&M came from the 1994 bag with probability 20%, and the\n", - "green came from the 1996 bag with probability 20%. If $B$ is true, the\n", - "yellow M&M came from the 1996 bag with probability 14%, and\n", - "the green came from the 1994 bag with probability 10%. Because the\n", - "selections are independent, we get the conjoint probability by\n", - "multiplying.\n", - "\n", - "The third column is just the product of the previous two. The sum of\n", - "this column, 270, is the normalizing constant. To get the last column,\n", - "which contains the posteriors, we divide the third column by the\n", - "normalizing constant.\n", - "\n", - "That’s it. Simple, right?\n", - "\n", - "Well, you might be bothered by one detail. I write\n", - "$\\mathrm{p}(D|H)$ in terms of percentages, not\n", - "probabilities, which means it is off by a factor of 10,000. But that\n", - "cancels out when we divide through by the normalizing constant, so it\n", - "doesn’t affect the result.\n", - "\n", - "When the set of hypotheses is mutually exclusive and collectively\n", - "exhaustive, you can multiply the likelihoods by any factor, if it is\n", - "convenient, as long as you apply the same factor to the entire column." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The Monty Hall problem\n", - "----------------------\n", - "\n", - "The Monty Hall problem might be the most contentious question in the\n", - "history of probability. The scenario is simple, but the correct answer\n", - "is so counterintuitive that many people just can’t accept it, and many\n", - "smart people have embarrassed themselves not just by getting it wrong\n", - "but by arguing the wrong side, aggressively, in public.\n", - "\n", - "Monty Hall was the original host of the game show *Let’s Make a\n", - "Deal*. The Monty Hall problem is based on one of the regular\n", - "games on the show. If you are on the show, here’s what happens:\n", - "\n", - "- Monty shows you three closed doors and tells you that there is a\n", - " prize behind each door: one prize is a car, the other two are less\n", - " valuable prizes like peanut butter and fake finger nails. The prizes\n", - " are arranged at random.\n", - "\n", - "- The object of the game is to guess which door has the car. If you\n", - " guess right, you get to keep the car.\n", - "\n", - "- You pick a door, which we will call Door A. We’ll call the other\n", - " doors B and C.\n", - "\n", - "- Before opening the door you chose, Monty increases the suspense by\n", - " opening either Door B or C, whichever does not have the car. (If the\n", - " car is actually behind Door A, Monty can safely open B or C, so he\n", - " chooses one at random.)\n", - "\n", - "- Then Monty offers you the option to stick with your original choice\n", - " or switch to the one remaining unopened door.\n", - "\n", - "The question is, should you “stick” or “switch” or does it make no\n", - "difference?\n", - "\n", - "Most people have the strong intuition that it makes no difference. There\n", - "are two doors left, they reason, so the chance that the car is behind\n", - "Door A is 50%.\n", - "\n", - "But that is wrong. In fact, the chance of winning if you stick with Door\n", - "A is only 1/3; if you switch, your chances are 2/3.\n", - "\n", - "By applying Bayes’s theorem, we can break this problem into simple\n", - "pieces, and maybe convince ourselves that the correct answer is, in\n", - "fact, correct.\n", - "\n", - "To start, we should make a careful statement of the data. In this case\n", - "$D$ consists of two parts: Monty chooses Door B *and* there\n", - "is no car there.\n", - "\n", - "Next we define three hypotheses: $A$, $B$, and $C$ represent the\n", - "hypothesis that the car is behind Door A, Door B, or Door C. Again,\n", - "let’s apply the table method:\n", - "\n", - "\n", - "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", - "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", - "| A | 1/3 | 1/2 | 1/6 |1/3|\n", - "| B | 1/3 | 0 | 0 | 0 |\n", - "| C | 1/3 | 1 | 1/3 | 2/3 |\n", - "\n", - "Filling in the priors is easy because we are told that the prizes are\n", - "arranged at random, which suggests that the car is equally likely to be\n", - "behind any door.\n", - "\n", - "Figuring out the likelihoods takes some thought, but with reasonable\n", - "care we can be confident that we have it right:\n", - "\n", - "- If the car is actually behind A, Monty could safely open Doors B or\n", - " C. So the probability that he chooses B is 1/2. And since the car is\n", - " actually behind A, the probability that the car is not behind B is\n", - " 1.\n", - "\n", - "- If the car is actually behind B, Monty has to open door C, so the\n", - " probability that he opens door B is 0.\n", - "\n", - "- Finally, if the car is behind Door C, Monty opens B with probability\n", - " 1 and finds no car there with probability 1.\n", - "\n", - "Now the hard part is over; the rest is just arithmetic. The sum of the\n", - "third column is 1/2. Dividing through yields\n", - "${{\\mathrm{p}(A|D)}} = 1/3$ and\n", - "${{\\mathrm{p}(C|D)}} = 2/3$. So you are better off switching.\n", - "\n", - "There are many variations of the Monty Hall problem. One of the\n", - "strengths of the Bayesian approach is that it generalizes to handle\n", - "these variations.\n", - "\n", - "For example, suppose that Monty always chooses B if he can, and only\n", - "chooses C if he has to (because the car is behind B). In that case the\n", - "revised table is:\n", - "\n", - "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", - "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", - "| A | 1/3| 1 |1/3 | 1/2\n", - "| B |1/3 | 0 | 0 | 0 |\n", - "| C |1/3 | 1 | 1/3 | 1/2|\n", - "\n", - "The only change is $\\mathrm{p}(D|A)$. If the car is behind\n", - "$A$, Monty can choose to open B or C. But in this variation he always\n", - "chooses B, so ${{\\mathrm{p}(D|A)}} = 1$.\n", - "\n", - "As a result, the likelihoods are the same for $A$ and $C$, and the\n", - "posteriors are the same:\n", - "${{\\mathrm{p}(A|D)}} = {{\\mathrm{p}(C|D)}} = 1/2$.\n", - "In this case, the fact that Monty chose B reveals no information about\n", - "the location of the car, so it doesn’t matter whether the contestant\n", - "sticks or switches.\n", - "\n", - "On the other hand, if he had opened $C$, we would know\n", - "${{\\mathrm{p}(B|D)}} = 1$.\n", - "\n", - "I included the Monty Hall problem in this chapter because I think it is\n", - "fun, and because Bayes’s theorem makes the complexity of the problem a\n", - "little more manageable. But it is not a typical use of Bayes’s theorem,\n", - "so if you found it confusing, don’t worry!\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "For many problems involving conditional probability, Bayes’s theorem\n", - "provides a divide-and-conquer strategy. If\n", - "$\\mathrm{p}(A|B)$ is hard to compute, or hard to measure\n", - "experimentally, check whether it might be easier to compute the other\n", - "terms in Bayes’s theorem, $\\mathrm{p}(B|A)$,\n", - "$\\mathrm{p}(A)$ and $\\mathrm{p}(B)$.\n", - "\n", - "If the Monty Hall problem is your idea of fun, I have collected a number\n", - "of similar problems in an article called “All your Bayes are belong to\n", - "us,” which you can read at\n", - "." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Bayes’s Theorem\n", + "\n", + "## Conditional probability\n", + "\n", + "The fundamental idea behind all Bayesian statistics is Bayes’s theorem,\n", + "which is surprisingly easy to derive, provided that you understand\n", + "conditional probability. So we’ll start with probability, then\n", + "conditional probability, then Bayes’s theorem, and on to Bayesian\n", + "statistics.\n", + "\n", + "A probability is a number between 0 and 1 (including both) that\n", + "represents a degree of belief in a fact or prediction. The value 1\n", + "represents certainty that a fact is true, or that a prediction will come\n", + "true. The value 0 represents certainty that the fact is false.\n", + "\n", + "Intermediate values represent degrees of certainty. The value 0.5, often\n", + "written as 50%, means that a predicted outcome is as likely to happen as\n", + "not. For example, the probability that a tossed coin lands face up is\n", + "very close to 50%.\n", + "\n", + "A conditional probability is a probability based on some background\n", + "information. For example, I want to know the probability that I will\n", + "have a heart attack in the next year. According to the CDC, “Every year\n", + "about 785,000 Americans have a first coronary attack.\n", + "()”\n", + "\n", + "The U.S. population is about 311 million, so the probability that a\n", + "randomly chosen American will have a heart attack in the next year is\n", + "roughly 0.3%.\n", + "\n", + "But I am not a randomly chosen American. Epidemiologists have identified\n", + "many factors that affect the risk of heart attacks; depending on those\n", + "factors, my risk might be higher or lower than average.\n", + "\n", + "I am male, 45 years old, and I have borderline high cholesterol. Those\n", + "factors increase my chances. However, I have low blood pressure and I\n", + "don’t smoke, and those factors decrease my chances.\n", + "\n", + "Plugging everything into the online calculator at\n", + ", I find that my risk of a\n", + "heart attack in the next year is about 0.2%, less than the national\n", + "average. That value is a conditional probability, because it is based on\n", + "a number of factors that make up my “condition.”\n", + "\n", + "The usual notation for conditional probability is\n", + "$\\mathrm{p}(A|B)$, which is the probability of $A$ given\n", + "that $B$ is true. In this example, $A$ represents the prediction that I\n", + "will have a heart attack in the next year, and $B$ is the set of\n", + "conditions I listed." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conjoint probability\n", + "\n", + "**Conjoint probability** is a fancy way to say the\n", + "probability that two things are true. I write\n", + "$\\mathrm{p}(A {~\\mathrm{and}~}B)$ to mean the probability\n", + "that $A$ and $B$ are both true.\n", + "\n", + "If you learned about probability in the context of coin tosses and dice,\n", + "you might have learned the following formula:\n", + "\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B)}} \\quad\\quad\\mbox{WARNING: not always true}$$\n", + "\n", + "For example, if I toss two coins, and $A$ means the first coin lands\n", + "face up, and $B$ means the second coin lands face up, then\n", + "\n", + "${{\\mathrm{p}(A)}} =\n", + "{{\\mathrm{p}(B)}} = 0.5$, and sure enough,\n", + "${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B)}} = 0.25$.\n", + "\n", + "But this formula only works because in this case $A$ and $B$ are\n", + "independent; that is, knowing the outcome of the first event does not\n", + "change the probability of the second. Or, more formally,\n", + "$\\mathrm{p}(B|A)$ = $\\mathrm{p}(B)$.\n", + "\n", + "Here is a different example where the events are not independent.\n", + "Suppose that $A$ means that it rains today and $B$ means that it rains\n", + "tomorrow. If I know that it rained today, it is more likely that it will\n", + "rain tomorrow, so\n", + "\n", + "${{\\mathrm{p}(B|A)}} > {{\\mathrm{p}(B)}}$.\n", + "\n", + "In general, the probability of a conjunction is\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", + "for any $A$ and $B$. So if the chance of rain on any given day is 0.5,\n", + "the chance of rain on two consecutive days is not 0.25, but probably a\n", + "bit higher." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The cookie problem\n", + "\n", + "We’ll get to Bayes’s theorem soon, but I want to motivate it with an\n", + "example called the cookie problem.[^1] Suppose there are two bowls of\n", + "cookies. Bowl 1 contains 30 vanilla cookies and 10 chocolate cookies.\n", + "Bowl 2 contains 20 of each.\n", + "\n", + "Now suppose you choose one of the bowls at random and, without looking,\n", + "select a cookie at random. The cookie is vanilla. What is the\n", + "probability that it came from Bowl 1?\n", + "\n", + "This is a conditional probability; we want\n", + "$\\mathrm{p}({\\mbox{Bowl 1}} |\n", + " {\\mbox{vanilla}})$, but it is not obvious how to compute it. If\n", + "I asked a different question—the probability of a vanilla cookie given\n", + "Bowl 1—it would be easy:\n", + "\n", + "$${{\\mathrm{p}({\\mbox{vanilla}} | {\\mbox{Bowl 1}})}} = 3/4$$\n", + "\n", + "Sadly, $\\mathrm{p}(A|B)$ is *not* the same as\n", + "$\\mathrm{p}(B|A)$, but there is a way to get from one to\n", + "the other: Bayes’s theorem." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Bayes’s theorem\n", + "\n", + "At this point we have everything we need to derive Bayes’s theorem.\n", + "We’ll start with the observation that conjunction is commutative; that\n", + "is\n", + "\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(B {~\\mathrm{and}~}A)}}$$\n", + "for any events $A$ and $B$.\n", + "\n", + "Next, we write the probability of a conjunction:\n", + "\n", + "$${{\\mathrm{p}(A {~\\mathrm{and}~}B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", + "\n", + "Since we have not said anything about what $A$ and $B$ mean, they are\n", + "interchangeable. Interchanging them yields\n", + "\n", + "$${{\\mathrm{p}(B {~\\mathrm{and}~}A)}} = {{\\mathrm{p}(B)}}~{{\\mathrm{p}(A|B)}}$$\n", + "\n", + "That’s all we need. Pulling those pieces together, we get\n", + "\n", + "$${{\\mathrm{p}(B)}}~{{\\mathrm{p}(A|B)}} = {{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}$$\n", + "\n", + "Which means there are two ways to compute the conjunction. If you have\n", + "$\\mathrm{p}(A)$, you multiply by the conditional\n", + "probability $\\mathrm{p}(B|A)$. Or you can do it the other\n", + "way around; if you know $\\mathrm{p}(B)$, you multiply by\n", + "$\\mathrm{p}(A|B)$. Either way you should get the same\n", + "thing.\n", + "\n", + "Finally we can divide through by $\\mathrm{p}(B)$:\n", + "$${{\\mathrm{p}(A|B)}} = \\frac{{{\\mathrm{p}(A)}}~{{\\mathrm{p}(B|A)}}}{{{\\mathrm{p}(B)}}}$$\n", + "And that’s Bayes’s theorem! It might not look like much, but it turns\n", + "out to be surprisingly powerful.\n", + "\n", + "For example, we can use it to solve the cookie problem. I’ll write $B_1$\n", + "for the hypothesis that the cookie came from Bowl 1 and $V$ for the\n", + "vanilla cookie. Plugging in Bayes’s theorem we get\n", + "\n", + "$${{\\mathrm{p}(B_1|V)}} = \\frac{{{\\mathrm{p}(B_1)}}~{{\\mathrm{p}(V|B_1)}}}{{{\\mathrm{p}(V)}}}$$\n", + "\n", + "The term on the left is what we want: the probability of Bowl 1, given\n", + "that we chose a vanilla cookie. The terms on the right are:\n", + "\n", + "- ${{\\mathrm{p}(B_1)}}$: This is the probability that we\n", + " chose Bowl 1, unconditioned by what kind of cookie we got. Since the\n", + " problem says we chose a bowl at random, we can assume\n", + " ${{\\mathrm{p}(B_1)}} = 1/2$.\n", + "\n", + "- ${{\\mathrm{p}(V|B_1)}}$: This is the probability of\n", + " getting a vanilla cookie from Bowl 1, which is 3/4.\n", + "\n", + "- $\\mathrm{p}(V)$: This is the probability of drawing a\n", + " vanilla cookie from either bowl. Since we had an equal chance of\n", + " choosing either bowl and the bowls contain the same number of\n", + " cookies, we had the same chance of choosing any cookie. Between the\n", + " two bowls there are 50 vanilla and 30 chocolate cookies, so\n", + " $\\mathrm{p}(V)$ = 5/8.\n", + "\n", + "Putting it together, we have\n", + "\n", + "$${{\\mathrm{p}(B_1|V)}} = \\frac{(1/2)~(3/4)}{5/8}$$ \n", + "\n", + "which\n", + "reduces to 3/5. So the vanilla cookie is evidence in favor of the\n", + "hypothesis that we chose Bowl 1, because vanilla cookies are more likely\n", + "to come from Bowl 1.\n", + "\n", + "This example demonstrates one use of Bayes’s theorem: it provides a\n", + "strategy to get from $\\mathrm{p}(B|A)$ to\n", + "$\\mathrm{p}(A|B)$. This strategy is useful in cases, like\n", + "the cookie problem, where it is easier to compute the terms on the right\n", + "side of Bayes’s theorem than the term on the left." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The diachronic interpretation\n", + "\n", + "There is another way to think of Bayes’s theorem: it gives us a way to\n", + "update the probability of a hypothesis, $H$, in light of some body of\n", + "data, $D$.\n", + "\n", + "This way of thinking about Bayes’s theorem is called the\n", + "**diachronic interpretation**. “Diachronic” means that\n", + "something is happening over time; in this case the probability of the\n", + "hypotheses changes, over time, as we see new data.\n", + "\n", + "Rewriting Bayes’s theorem with $H$ and $D$ yields:\n", + "$${{\\mathrm{p}(H|D)}} = \\frac{{{\\mathrm{p}(H)}}~{{\\mathrm{p}(D|H)}}}{{{\\mathrm{p}(D)}}}$$\n", + "In this interpretation, each term has a name:\n", + "\n", + "- $\\mathrm{p}(H)$ is the probability of the hypothesis\n", + " before we see the data, called the prior probability, or just\n", + " **prior**.\n", + "\n", + "- $\\mathrm{p}(H|D)$ is what we want to compute, the\n", + " probability of the hypothesis after we see the data, called the\n", + " **posterior**.\n", + "\n", + "- $\\mathrm{p}(D|H)$ is the probability of the data under\n", + " the hypothesis, called the **likelihood**.\n", + "\n", + "- $\\mathrm{p}(D)$ is the probability of the data under\n", + " any hypothesis, called the **normalizing constant**.\n", + "\n", + "Sometimes we can compute the prior based on background information. For\n", + "example, the cookie problem specifies that we choose a bowl at random\n", + "with equal probability.\n", + "\n", + "In other cases the prior is subjective; that is, reasonable people might\n", + "disagree, either because they use different background information or\n", + "because they interpret the same information differently.\n", + "\n", + "The likelihood is usually the easiest part to compute. In the cookie\n", + "problem, if we know which bowl the cookie came from, we find the\n", + "probability of a vanilla cookie by counting.\n", + "\n", + "The normalizing constant can be tricky. It is supposed to be the\n", + "probability of seeing the data under any hypothesis at all, but in the\n", + "most general case it is hard to nail down what that means.\n", + "\n", + "Most often we simplify things by specifying a set of hypotheses that are\n", + "\n", + "Mutually exclusive:\n", + ": At most one hypothesis in the set can be true, and\n", + "\n", + "Collectively exhaustive:\n", + ": There are no other possibilities; at least one of the hypotheses has\n", + " to be true.\n", + "\n", + "I use the word **suite** for a set of hypotheses that has\n", + "these properties.\n", + "\n", + "In the cookie problem, there are only two hypotheses—the cookie came\n", + "from Bowl 1 or Bowl 2—and they are mutually exclusive and collectively\n", + "exhaustive.\n", + "\n", + "In that case we can compute $\\mathrm{p}(D)$ using the law\n", + "of total probability, which says that if there are two exclusive ways\n", + "that something might happen, you can add up the probabilities like this:\n", + "\n", + "$${{\\mathrm{p}(D)}} = {{\\mathrm{p}(B_1)}}~{{\\mathrm{p}(D|B_1)}} + {{\\mathrm{p}(B_2)}}~{{\\mathrm{p}(D|B_2)}}$$\n", + "\n", + "Plugging in the values from the cookie problem, we have\n", + "\n", + "$${{\\mathrm{p}(D)}} = (1/2)~(3/4) + (1/2)~(1/2) = 5/8$$ \n", + "\n", + "which is what we computed earlier by mentally combining the two bowls." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The M&M problem\n", + "\n", + "M&M’s are small candy-coated chocolates that come in a\n", + "variety of colors. Mars, Inc., which makes M&M’s, changes\n", + "the mixture of colors from time to time.\n", + "\n", + "In 1995, they introduced blue M&M’s. Before then, the color\n", + "mix in a bag of plain M&M’s was 30% Brown, 20% Yellow, 20%\n", + "Red, 10% Green, 10% Orange, 10% Tan. Afterward it was 24% Blue , 20%\n", + "Green, 16% Orange, 14% Yellow, 13% Red, 13% Brown.\n", + "\n", + "Suppose a friend of mine has two bags of M&M’s, and he\n", + "tells me that one is from 1994 and one from 1996. He won’t tell me which\n", + "is which, but he gives me one M&M from each bag. One is\n", + "yellow and one is green. What is the probability that the yellow one\n", + "came from the 1994 bag?\n", + "\n", + "This problem is similar to the cookie problem, with the twist that I\n", + "draw one sample from each bowl/bag. This problem also gives me a chance\n", + "to demonstrate the table method, which is useful for solving problems\n", + "like this on paper. In the next chapter we will solve them\n", + "computationally.\n", + "\n", + "The first step is to enumerate the hypotheses. The bag the yellow\n", + "M&M came from I’ll call Bag 1; I’ll call the other Bag 2.\n", + "So the hypotheses are:\n", + "\n", + "- A: Bag 1 is from 1994, which implies that Bag 2 is from 1996.\n", + "\n", + "- B: Bag 1 is from 1996 and Bag 2 from 1994.\n", + "\n", + "Now we construct a table with a row for each hypothesis and a column for\n", + "each term in Bayes’s theorem:\n", + "\n", + "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", + "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", + "| A | 1/2 | (20)(20) | 200 | 20/27|\n", + "| B | 1/2 | (14)(10) | 70 | 7/27|\n", + "\n", + "\n", + "The first column has the priors. Based on the statement of the problem,\n", + "it is reasonable to choose\n", + "${{\\mathrm{p}(A)}} = {{\\mathrm{p}(B)}} = 1/2$.\n", + "\n", + "The second column has the likelihoods, which follow from the information\n", + "in the problem. For example, if $A$ is true, the yellow\n", + "M&M came from the 1994 bag with probability 20%, and the\n", + "green came from the 1996 bag with probability 20%. If $B$ is true, the\n", + "yellow M&M came from the 1996 bag with probability 14%, and\n", + "the green came from the 1994 bag with probability 10%. Because the\n", + "selections are independent, we get the conjoint probability by\n", + "multiplying.\n", + "\n", + "The third column is just the product of the previous two. The sum of\n", + "this column, 270, is the normalizing constant. To get the last column,\n", + "which contains the posteriors, we divide the third column by the\n", + "normalizing constant.\n", + "\n", + "That’s it. Simple, right?\n", + "\n", + "Well, you might be bothered by one detail. I write\n", + "$\\mathrm{p}(D|H)$ in terms of percentages, not\n", + "probabilities, which means it is off by a factor of 10,000. But that\n", + "cancels out when we divide through by the normalizing constant, so it\n", + "doesn’t affect the result.\n", + "\n", + "When the set of hypotheses is mutually exclusive and collectively\n", + "exhaustive, you can multiply the likelihoods by any factor, if it is\n", + "convenient, as long as you apply the same factor to the entire column." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The Monty Hall problem\n", + "\n", + "The Monty Hall problem might be the most contentious question in the\n", + "history of probability. The scenario is simple, but the correct answer\n", + "is so counterintuitive that many people just can’t accept it, and many\n", + "smart people have embarrassed themselves not just by getting it wrong\n", + "but by arguing the wrong side, aggressively, in public.\n", + "\n", + "Monty Hall was the original host of the game show *Let’s Make a\n", + "Deal*. The Monty Hall problem is based on one of the regular\n", + "games on the show. If you are on the show, here’s what happens:\n", + "\n", + "- Monty shows you three closed doors and tells you that there is a\n", + " prize behind each door: one prize is a car, the other two are less\n", + " valuable prizes like peanut butter and fake finger nails. The prizes\n", + " are arranged at random.\n", + "\n", + "- The object of the game is to guess which door has the car. If you\n", + " guess right, you get to keep the car.\n", + "\n", + "- You pick a door, which we will call Door A. We’ll call the other\n", + " doors B and C.\n", + "\n", + "- Before opening the door you chose, Monty increases the suspense by\n", + " opening either Door B or C, whichever does not have the car. (If the\n", + " car is actually behind Door A, Monty can safely open B or C, so he\n", + " chooses one at random.)\n", + "\n", + "- Then Monty offers you the option to stick with your original choice\n", + " or switch to the one remaining unopened door.\n", + "\n", + "The question is, should you “stick” or “switch” or does it make no\n", + "difference?\n", + "\n", + "Most people have the strong intuition that it makes no difference. There\n", + "are two doors left, they reason, so the chance that the car is behind\n", + "Door A is 50%.\n", + "\n", + "But that is wrong. In fact, the chance of winning if you stick with Door\n", + "A is only 1/3; if you switch, your chances are 2/3.\n", + "\n", + "By applying Bayes’s theorem, we can break this problem into simple\n", + "pieces, and maybe convince ourselves that the correct answer is, in\n", + "fact, correct.\n", + "\n", + "To start, we should make a careful statement of the data. In this case\n", + "$D$ consists of two parts: Monty chooses Door B *and* there\n", + "is no car there.\n", + "\n", + "Next we define three hypotheses: $A$, $B$, and $C$ represent the\n", + "hypothesis that the car is behind Door A, Door B, or Door C. Again,\n", + "let’s apply the table method:\n", + "\n", + "\n", + "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", + "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", + "| A | 1/3 | 1/2 | 1/6 |1/3|\n", + "| B | 1/3 | 0 | 0 | 0 |\n", + "| C | 1/3 | 1 | 1/3 | 2/3 |\n", + "\n", + "Filling in the priors is easy because we are told that the prizes are\n", + "arranged at random, which suggests that the car is equally likely to be\n", + "behind any door.\n", + "\n", + "Figuring out the likelihoods takes some thought, but with reasonable\n", + "care we can be confident that we have it right:\n", + "\n", + "- If the car is actually behind A, Monty could safely open Doors B or\n", + " C. So the probability that he chooses B is 1/2. And since the car is\n", + " actually behind A, the probability that the car is not behind B is\n", + " 1.\n", + "\n", + "- If the car is actually behind B, Monty has to open door C, so the\n", + " probability that he opens door B is 0.\n", + "\n", + "- Finally, if the car is behind Door C, Monty opens B with probability\n", + " 1 and finds no car there with probability 1.\n", + "\n", + "Now the hard part is over; the rest is just arithmetic. The sum of the\n", + "third column is 1/2. Dividing through yields\n", + "${{\\mathrm{p}(A|D)}} = 1/3$ and\n", + "${{\\mathrm{p}(C|D)}} = 2/3$. So you are better off switching.\n", + "\n", + "There are many variations of the Monty Hall problem. One of the\n", + "strengths of the Bayesian approach is that it generalizes to handle\n", + "these variations.\n", + "\n", + "For example, suppose that Monty always chooses B if he can, and only\n", + "chooses C if he has to (because the car is behind B). In that case the\n", + "revised table is:\n", + "\n", + "| |Prior $\\mathrm{p}(H)$ | Likelihood $\\mathrm{p}(D\\vert H)$ | $\\mathrm{p}(H) \\mathrm{p}(D\\vert H)$ | Posterior $\\mathrm{p}(H\\vert D)$|\n", + "| --- | ---------------- | --------------------------- | ----------- | ------ |\n", + "| A | 1/3| 1 |1/3 | 1/2\n", + "| B |1/3 | 0 | 0 | 0 |\n", + "| C |1/3 | 1 | 1/3 | 1/2|\n", + "\n", + "The only change is $\\mathrm{p}(D|A)$. If the car is behind\n", + "$A$, Monty can choose to open B or C. But in this variation he always\n", + "chooses B, so ${{\\mathrm{p}(D|A)}} = 1$.\n", + "\n", + "As a result, the likelihoods are the same for $A$ and $C$, and the\n", + "posteriors are the same:\n", + "${{\\mathrm{p}(A|D)}} = {{\\mathrm{p}(C|D)}} = 1/2$.\n", + "In this case, the fact that Monty chose B reveals no information about\n", + "the location of the car, so it doesn’t matter whether the contestant\n", + "sticks or switches.\n", + "\n", + "On the other hand, if he had opened $C$, we would know\n", + "${{\\mathrm{p}(B|D)}} = 1$.\n", + "\n", + "I included the Monty Hall problem in this chapter because I think it is\n", + "fun, and because Bayes’s theorem makes the complexity of the problem a\n", + "little more manageable. But it is not a typical use of Bayes’s theorem,\n", + "so if you found it confusing, don’t worry!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "\n", + "For many problems involving conditional probability, Bayes’s theorem\n", + "provides a divide-and-conquer strategy. If\n", + "$\\mathrm{p}(A|B)$ is hard to compute, or hard to measure\n", + "experimentally, check whether it might be easier to compute the other\n", + "terms in Bayes’s theorem, $\\mathrm{p}(B|A)$,\n", + "$\\mathrm{p}(A)$ and $\\mathrm{p}(B)$.\n", + "\n", + "If the Monty Hall problem is your idea of fun, I have collected a number\n", + "of similar problems in an article called “All your Bayes are belong to\n", + "us,” which you can read at\n", + "." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/02_Computational_Statistics.ipynb b/notebooks/02_Computational_Statistics.ipynb index a2d0af2..095ca68 100644 --- a/notebooks/02_Computational_Statistics.ipynb +++ b/notebooks/02_Computational_Statistics.ipynb @@ -1,999 +1,1224 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "The autoreload extension is already loaded. To reload it, use:\n", - " %reload_ext autoreload\n" - ] - } - ], - "source": [ - "import sys\n", - "from __future__ import division, print_function\n", - "%load_ext autoreload\n", - "%autoreload 2 \n", - "import sys\n", - "sys.path.insert(0,'../code')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Computational Statistics {#compstat}\n", - "\n", - "##Distributions\n", - "\n", - "In statistics a **distribution** is a set of values and\n", - "their corresponding probabilities.\n", - "\n", - "For example, if you roll a six-sided die, the set of possible values is\n", - "the numbers 1 to 6, and the probability associated with each value is\n", - "1/6.\n", - "\n", - "As another example, you might be interested in how many times each word\n", - "appears in common English usage. You could build a distribution that\n", - "includes each word and how many times it appears.\n", - "\n", - "To represent a distribution in Python, you could use a dictionary that\n", - "maps from each value to its probability. I have written a class called\n", - "Pmf that uses a Python dictionary in exactly that way, and\n", - "provides a number of useful methods. I called the class Pmf in reference\n", - "to a **probability mass function**, which is a way to\n", - "represent a distribution mathematically.\n", - "\n", - "Pmf is defined in a Python module I wrote to accompany this\n", - "book, thinkbayes.py. You can download it from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "To use Pmf you can import it like this:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "from thinkbayes import Pmf" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The following code builds a Pmf to represent the distribution of\n", - "outcomes for a six-sided die:" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1 0.16666666666666666\n", - "2 0.16666666666666666\n", - "3 0.16666666666666666\n", - "4 0.16666666666666666\n", - "5 0.16666666666666666\n", - "6 0.16666666666666666\n" - ] - } - ], - "source": [ - "pmf = Pmf()\n", - "for x in [1,2,3,4,5,6]:\n", - " pmf.Set(x, 1/6.0)\n", - "pmf.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Pmf` creates an empty Pmf with no values. The `Set` method sets the\n", - "probability associated with each value to $1/6$.\n", - "\n", - "Here’s another example that counts the number of times each word appears\n", - "in a sequence:" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "bye 1\n", - "football 1\n", - "hi 2\n", - "sky 1\n", - "the 1\n" - ] - } - ], - "source": [ - "word_list = ['hi', 'the', 'bye', 'hi', 'football', 'sky']\n", - "pmf = Pmf()\n", - "for word in word_list:\n", - " pmf.Incr(word, 1)\n", - "pmf.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Incr` increases the “probability” associated with each word by 1. If a\n", - "word is not already in the Pmf, it is added.\n", - "\n", - "I put “probability” in quotes because in this example, the probabilities\n", - "are not normalized; that is, they do not add up to 1. So they are not\n", - "true probabilities.\n", - "\n", - "But in this example the word counts are proportional to the\n", - "probabilities. So after we count all the words, we can compute\n", - "probabilities by dividing through by the total number of words.\n", - "Pmf provides a method, `Normalize`, that does exactly that:" - ] - }, - { - "cell_type": "code", - "execution_count": 26, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "bye 0.16666666666666666\n", - "football 0.16666666666666666\n", - "hi 0.3333333333333333\n", - "sky 0.16666666666666666\n", - "the 0.16666666666666666\n" - ] - } - ], - "source": [ - "pmf.Normalize()\n", - "pmf.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Once you have a Pmf object, you can ask for the probability associated\n", - "with any value:" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.16666666666666666\n" - ] - } - ], - "source": [ - "print(pmf.Prob('the'))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And that would print the frequency of the word “the” as a fraction of\n", - "the words in the list.\n", - "\n", - "Pmf uses a Python dictionary to store the values and their\n", - "probabilities, so the values in the Pmf can be any hashable type. The\n", - "probabilities can be any numerical type, but they are usually\n", - "floating-point numbers (type `float`)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## The cookie problem\n", - "\n", - "In the context of Bayes’s theorem, it is natural to use a Pmf to map\n", - "from each hypothesis to its probability. In the cookie problem, the\n", - "hypotheses are $B_1$ and $B_2$. In Python, I represent them with\n", - "strings:" - ] - }, - { - "cell_type": "code", - "execution_count": 28, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "pmf = Pmf()\n", - "pmf.Set('Bowl 1', 0.5)\n", - "pmf.Set('Bowl 2', 0.5)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This distribution, which contains the priors for each hypothesis, is\n", - "called (wait for it) the **prior distribution**.\n", - "\n", - "To update the distribution based on new data (the vanilla cookie), we\n", - "multiply each prior by the corresponding likelihood. The likelihood of\n", - "drawing a vanilla cookie from Bowl 1 is 3/4. The likelihood for Bowl 2\n", - "is 1/2." - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "pmf.Mult('Bowl 1', 0.75)\n", - "pmf.Mult('Bowl 2', 0.5)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Mult` does what you would expect. It gets the probability for the given\n", - "hypothesis and multiplies by the given likelihood.\n", - "\n", - "After this update, the distribution is no longer normalized, but because\n", - "these hypotheses are mutually exclusive and collectively exhaustive, we\n", - "can **renormalize**:" - ] - }, - { - "cell_type": "code", - "execution_count": 30, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.625" - ] - }, - "execution_count": 30, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "pmf.Normalize()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The result is a distribution that contains the posterior probability for\n", - "each hypothesis, which is called (wait now) the **posterior\n", - "distribution**.\n", - "\n", - "Finally, we can get the posterior probability for Bowl 1:" - ] - }, - { - "cell_type": "code", - "execution_count": 31, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.6000000000000001\n" - ] - } - ], - "source": [ - "print(pmf.Prob('Bowl 1'))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And the answer is 0.6. You can download this example from\n", - ". For more information see\n", - "Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The Bayesian framework {#framework}\n", - "\n", - "Before we go on to other problems, I want to rewrite the code from the\n", - "previous section to make it more general. First I’ll define a class to\n", - "encapsulate the code related to this problem:" - ] - }, - { - "cell_type": "code", - "execution_count": 37, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "class Cookie(Pmf):\n", - " \"\"\"A map from string bowl ID to probablity.\"\"\"\n", - "\n", - " def __init__(self, hypos):\n", - " \"\"\"Initialize self.\n", - "\n", - " hypos: sequence of string bowl IDs\n", - " \"\"\"\n", - " Pmf.__init__(self)\n", - " for hypo in hypos:\n", - " self.Set(hypo, 1)\n", - " self.Normalize()\n", - "\n", - " def Update(self, data):\n", - " \"\"\"Updates the PMF with new data.\n", - "\n", - " data: string cookie type\n", - " \"\"\"\n", - " for hypo in self.Values():\n", - " like = self.Likelihood(data, hypo)\n", - " self.Mult(hypo, like)\n", - " self.Normalize()\n", - "\n", - " mixes = {\n", - " 'Bowl 1':dict(vanilla=0.75, chocolate=0.25),\n", - " 'Bowl 2':dict(vanilla=0.5, chocolate=0.5),\n", - " }\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " \"\"\"The likelihood of the data under the hypothesis.\n", - "\n", - " data: string cookie type\n", - " hypo: string bowl ID\n", - " \"\"\"\n", - " mix = self.mixes[hypo]\n", - " like = mix[data]\n", - " return like" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "A Cookie object is a Pmf that maps from hypotheses to their\n", - "probabilities. The `__init__` method gives each hypothesis the same\n", - "prior probability. As in the previous section, there are two hypotheses:" - ] - }, - { - "cell_type": "code", - "execution_count": 38, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "hypos = ['Bowl 1', 'Bowl 2']\n", - "pmf = Cookie(hypos)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Cookie` provides an `Update` method that takes data as a parameter and\n", - "updates the probabilities. `Update` loops through each hypothesis in the suite and multiplies its\n", - "probability by the likelihood of the data under the hypothesis, which is\n", - "computed by `Likelihood`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Likelihood` uses `mixes`, which is a dictionary that maps from the name\n", - "of a bowl to the mix of cookies in the bowl.\n", - "\n", - "Here’s what the update looks like:" - ] - }, - { - "cell_type": "code", - "execution_count": 40, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "pmf.Update('vanilla')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And then we can print the posterior probability of each hypothesis:" - ] - }, - { - "cell_type": "code", - "execution_count": 41, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Bowl 2 0.4\n", - "Bowl 1 0.6000000000000001\n" - ] - } - ], - "source": [ - "for hypo, prob in pmf.Items():\n", - " print(hypo, prob)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "which is the same as what we got before. This code is more complicated\n", - "than what we saw in the previous section. One advantage is that it\n", - "generalizes to the case where we draw more than one cookie from the same\n", - "bowl (with replacement):" - ] - }, - { - "cell_type": "code", - "execution_count": 43, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Bowl 1 0.6549865229110512\n", - "Bowl 2 0.34501347708894875\n" - ] - } - ], - "source": [ - "dataset = ['vanilla', 'chocolate', 'vanilla']\n", - "for data in dataset:\n", - " pmf.Update(data)\n", - "pmf.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The other advantage is that it provides a framework for solving many\n", - "similar problems. In the next section we’ll solve the Monty Hall problem\n", - "computationally and then see what parts of the framework are the same.\n", - "\n", - "The code in this section is available from\n", - ". For more information see\n", - "Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The Monty Hall problem\n", - "----------------------\n", - "\n", - "To solve the Monty Hall problem, I’ll define a new class:" - ] - }, - { - "cell_type": "code", - "execution_count": 44, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class Monty(Pmf):\n", - " \"\"\"Map from string location of car to probability\"\"\"\n", - "\n", - " def __init__(self, hypos):\n", - " \"\"\"Initialize the distribution.\n", - "\n", - " hypos: sequence of hypotheses\n", - " \"\"\"\n", - " Pmf.__init__(self)\n", - " for hypo in hypos:\n", - " self.Set(hypo, 1)\n", - " self.Normalize()\n", - "\n", - " def Update(self, data):\n", - " \"\"\"Updates each hypothesis based on the data.\n", - "\n", - " data: any representation of the data\n", - " \"\"\"\n", - " for hypo in self.Values():\n", - " like = self.Likelihood(data, hypo)\n", - " self.Mult(hypo, like)\n", - " self.Normalize()\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " \"\"\"Compute the likelihood of the data under the hypothesis.\n", - "\n", - " hypo: string name of the door where the prize is\n", - " data: string name of the door Monty opened\n", - " \"\"\"\n", - " if hypo == data:\n", - " return 0\n", - " elif hypo == 'A':\n", - " return 0.5\n", - " else:\n", - " return 1" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So far `Monty` and `Cookie` are nearly the same. And the code that\n", - "creates the Pmf is the same, too, except for the names of the\n", - "hypotheses:" - ] - }, - { - "cell_type": "code", - "execution_count": 45, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "hypos = 'ABC'\n", - "pmf = Monty(hypos)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Calling `Update` is pretty much the same:" - ] - }, - { - "cell_type": "code", - "execution_count": 46, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "data = 'B'\n", - "pmf.Update(data)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And the implementation of `Update` is exactly the same. The only part that requires some work is `Likelihood`:\n", - "\n", - "```python\n", - "def Likelihood(self, data, hypo):\n", - " if hypo == data:\n", - " return 0\n", - " elif hypo == 'A':\n", - " return 0.5\n", - " else:\n", - " return 1\n", - "```\n", - "Finally, printing the results is the same:" - ] - }, - { - "cell_type": "code", - "execution_count": 48, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "B 0.0\n", - "A 0.3333333333333333\n", - "C 0.6666666666666666\n" - ] - } - ], - "source": [ - "for hypo, prob in pmf.Items():\n", - " print(hypo, prob)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this example, writing `Likelihood` is a little complicated, but the\n", - "framework of the Bayesian update is simple. The code in this section is\n", - "available from . For more information\n", - "see Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Encapsulating the framework\n", - "---------------------------\n", - "\n", - "Now that we see what elements of the framework are the same, we can\n", - "encapsulate them in an object—a `Suite` is a `Pmf` that provides\n", - "`__init__`, `Update`, and `Print`:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```python\n", - "class Suite(Pmf):\n", - " \"\"\"Represents a suite of hypotheses and their probabilities.\"\"\"\n", - "\n", - " def __init__(self, hypo=tuple()):\n", - " \"\"\"Initializes the distribution.\"\"\"\n", - "\n", - " def Update(self, data):\n", - " \"\"\"Updates each hypothesis based on the data.\"\"\"\n", - "\n", - " def Print(self):\n", - " \"\"\"Prints the hypotheses and their probabilities.\"\"\"\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The implementation of `Suite` is in `thinkbayes.py`. To use `Suite`, you\n", - "should write a class that inherits from it and provides `Likelihood`.\n", - "For example, here is the solution to the Monty Hall problem rewritten to\n", - "use `Suite`:" - ] - }, - { - "cell_type": "code", - "execution_count": 49, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "from thinkbayes import Suite\n", - "\n", - "class Monty(Suite):\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " if hypo == data:\n", - " return 0\n", - " elif hypo == 'A':\n", - " return 0.5\n", - " else:\n", - " return 1" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And here’s the code that uses this class:" - ] - }, - { - "cell_type": "code", - "execution_count": 50, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "A 0.3333333333333333\n", - "B 0.0\n", - "C 0.6666666666666666\n" - ] - } - ], - "source": [ - "suite = Monty('ABC')\n", - "suite.Update('B')\n", - "suite.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You can download this example from .\n", - "For more information see Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The M&M problem\n", - "----------------------------\n", - "\n", - "We can use the `Suite` framework to solve the M&M problem.\n", - "Writing the `Likelihood` function is tricky, but everything else is\n", - "straightforward." - ] - }, - { - "cell_type": "code", - "execution_count": 54, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class M_and_M(Suite):\n", - " \"\"\"Map from hypothesis (A or B) to probability.\"\"\"\n", - "\n", - " mix94 = dict(brown=30,\n", - " yellow=20,\n", - " red=20,\n", - " green=10,\n", - " orange=10,\n", - " tan=10)\n", - "\n", - " mix96 = dict(blue=24,\n", - " green=20,\n", - " orange=16,\n", - " yellow=14,\n", - " red=13,\n", - " brown=13)\n", - "\n", - " hypoA = dict(bag1=mix94, bag2=mix96)\n", - " hypoB = dict(bag1=mix96, bag2=mix94)\n", - "\n", - " hypotheses = dict(A=hypoA, B=hypoB)\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " \"\"\"Computes the likelihood of the data under the hypothesis.\n", - "\n", - " hypo: string hypothesis (A or B)\n", - " data: tuple of string bag, string color\n", - " \"\"\"\n", - " bag, color = data\n", - " mix = self.hypotheses[hypo][bag]\n", - " like = mix[color]\n", - " return like" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "First I need to encode the color mixes from before and after 1995:\n", - "\n", - "```python\n", - "mix94 = dict(brown=30,\n", - " yellow=20,\n", - " ...\n", - "\n", - "mix96 = dict(blue=24,\n", - " green=20,\n", - " ...\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Then I have to encode the hypotheses:\n", - "\n", - "```python\n", - "hypoA = dict(bag1=mix94, bag2=mix96)\n", - "hypoB = dict(bag1=mix96, bag2=mix94)\n", - "```\n", - "\n", - "`hypoA` represents the hypothesis that Bag 1 is from 1994 and Bag 2 from 1996. `hypoB` is the other way around.\n", - "\n", - "Next I map from the name of the hypothesis to the representation:\n", - "\n", - "```python\n", - "hypotheses = dict(A=hypoA, B=hypoB)\n", - "```\n", - "\n", - "And finally I can write `Likelihood`. In this case the hypothesis,\n", - "`hypo`, is a string, either `A` or `B`. The data is a tuple that\n", - "specifies a bag and a color.\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " bag, color = data\n", - " mix = self.hypotheses[hypo][bag]\n", - " like = mix[color]\n", - " return like" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Here’s the code that creates the suite and updates it:" - ] - }, - { - "cell_type": "code", - "execution_count": 55, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "A 0.7407407407407407\n", - "B 0.2592592592592592\n" - ] - } - ], - "source": [ - "suite = M_and_M('AB')\n", - "\n", - "suite.Update(('bag1', 'yellow'))\n", - "suite.Update(('bag2', 'green'))\n", - "\n", - "suite.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The posterior probability of A is approximately $20/27$, which is what\n", - "we got before.\n", - "\n", - "The code in this section is available from\n", - ". For more information see\n", - "Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Discussion\n", - "----------\n", - "\n", - "This chapter presents the Suite class, which encapsulates the Bayesian\n", - "update framework.\n", - "\n", - "Suite is an **abstract type**, which means\n", - "that it defines the interface a Suite is supposed to have, but does not\n", - "provide a complete implementation. The Suite interface\n", - "includes Update and Likelihood, but the\n", - "Suite class only provides an implementation of\n", - "Update, not Likelihood.\n", - "\n", - "A **concrete type** is a class that extends an abstract\n", - "parent class and provides an implementation of the missing methods. For\n", - "example, Monty extends Suite, so it inherits\n", - "Update and provides Likelihood.\n", - "\n", - "If you are familiar with design patterns, you might recognize this as an\n", - "example of the template method pattern. You can read about this pattern\n", - "at .\n", - "\n", - "Most of the examples in the following chapters follow the same pattern;\n", - "for each problem we define a new class that extends Suite,\n", - "inherits Update, and provides Likelihood. In a\n", - "few cases we override Update, usually to improve\n", - "performance." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Exercises\n", - "---------\n", - "\n", - "In Section [framework] I said that the solution to the cookie problem\n", - "generalizes to the case where we draw multiple cookies with replacement.\n", - "\n", - "But in the more likely scenario where we eat the cookies we draw, the\n", - "likelihood of each draw depends on the previous draws.\n", - "\n", - "Modify the solution in this chapter to handle selection without\n", - "replacement. Hint: add instance variables to Cookie to\n", - "represent the hypothetical state of the bowls, and modify\n", - "Likelihood accordingly. You might want to define a\n", - "Bowl object." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Computational Statistics\n", + "\n", + "## Distributions\n", + "\n", + "In statistics a **distribution** is a set of values and\n", + "their corresponding probabilities.\n", + "\n", + "For example, if you roll a six-sided die, the set of possible values is\n", + "the numbers 1 to 6, and the probability associated with each value is\n", + "1/6.\n", + "\n", + "As another example, you might be interested in how many times each word\n", + "appears in common English usage. You could build a distribution that\n", + "includes each word and how many times it appears.\n", + "\n", + "To represent a distribution in Python, you could use a dictionary that\n", + "maps from each value to its probability. I have written a class called\n", + "Pmf that uses a Python dictionary in exactly that way, and\n", + "provides a number of useful methods. I called the class Pmf in reference\n", + "to a **probability mass function**, which is a way to\n", + "represent a distribution mathematically.\n", + "\n", + "Pmf is defined in a Python module I wrote to accompany this\n", + "book, thinkbayes.py. You can download it from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "To use Pmf you can import it like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from thinkbayes import Pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following code builds a Pmf to represent the distribution of\n", + "outcomes for a six-sided die:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1 0.16666666666666666\n", + "2 0.16666666666666666\n", + "3 0.16666666666666666\n", + "4 0.16666666666666666\n", + "5 0.16666666666666666\n", + "6 0.16666666666666666\n" + ] + } + ], + "source": [ + "pmf = Pmf()\n", + "for x in [1,2,3,4,5,6]:\n", + " pmf.Set(x, 1/6.0)\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Pmf` creates an empty Pmf with no values. The `Set` method sets the\n", + "probability associated with each value to $1/6$.\n", + "\n", + "Here’s another example that counts the number of times each word appears\n", + "in a sequence:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "bye 1\n", + "football 1\n", + "hi 2\n", + "sky 1\n", + "the 1\n" + ] + } + ], + "source": [ + "word_list = ['hi', 'the', 'bye', 'hi', 'football', 'sky']\n", + "pmf = Pmf()\n", + "for word in word_list:\n", + " pmf.Incr(word, 1)\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Incr` increases the “probability” associated with each word by 1. If a\n", + "word is not already in the Pmf, it is added.\n", + "\n", + "I put “probability” in quotes because in this example, the probabilities\n", + "are not normalized; that is, they do not add up to 1. So they are not\n", + "true probabilities.\n", + "\n", + "But in this example the word counts are proportional to the\n", + "probabilities. So after we count all the words, we can compute\n", + "probabilities by dividing through by the total number of words.\n", + "Pmf provides a method, `Normalize`, that does exactly that:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "bye 0.16666666666666666\n", + "football 0.16666666666666666\n", + "hi 0.3333333333333333\n", + "sky 0.16666666666666666\n", + "the 0.16666666666666666\n" + ] + } + ], + "source": [ + "pmf.Normalize()\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once you have a Pmf object, you can ask for the probability associated\n", + "with any value:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.16666666666666666\n" + ] + } + ], + "source": [ + "print(pmf.Prob('the'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And that would print the frequency of the word “the” as a fraction of\n", + "the words in the list.\n", + "\n", + "Pmf uses a Python dictionary to store the values and their\n", + "probabilities, so the values in the Pmf can be any hashable type. The\n", + "probabilities can be any numerical type, but they are usually\n", + "floating-point numbers (type `float`)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The cookie problem\n", + "\n", + "In the context of Bayes’s theorem, it is natural to use a Pmf to map\n", + "from each hypothesis to its probability. In the cookie problem, the\n", + "hypotheses are $B_1$ and $B_2$. In Python, I represent them with\n", + "strings:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pmf = Pmf()\n", + "pmf.Set('Bowl 1', 0.5)\n", + "pmf.Set('Bowl 2', 0.5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This distribution, which contains the priors for each hypothesis, is\n", + "called (wait for it) the **prior distribution**.\n", + "\n", + "To update the distribution based on new data (the vanilla cookie), we\n", + "multiply each prior by the corresponding likelihood. The likelihood of\n", + "drawing a vanilla cookie from Bowl 1 is 3/4. The likelihood for Bowl 2\n", + "is 1/2." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pmf.Mult('Bowl 1', 0.75)\n", + "pmf.Mult('Bowl 2', 0.5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Mult` does what you would expect. It gets the probability for the given\n", + "hypothesis and multiplies by the given likelihood.\n", + "\n", + "After this update, the distribution is no longer normalized, but because\n", + "these hypotheses are mutually exclusive and collectively exhaustive, we\n", + "can **renormalize**:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.625" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pmf.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The result is a distribution that contains the posterior probability for\n", + "each hypothesis, which is called (wait now) the **posterior\n", + "distribution**.\n", + "\n", + "Finally, we can get the posterior probability for Bowl 1:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.6000000000000001\n" + ] + } + ], + "source": [ + "print(pmf.Prob('Bowl 1'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the answer is 0.6. You can download this example from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The Bayesian framework\n", + "\n", + "Before we go on to other problems, I want to rewrite the code from the\n", + "previous section to make it more general. First I’ll define a class to\n", + "encapsulate the code related to this problem:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "class Cookie(Pmf):\n", + " \"\"\"A map from string bowl ID to probablity.\"\"\"\n", + "\n", + " def __init__(self, hypos):\n", + " \"\"\"Initialize self.\n", + "\n", + " hypos: sequence of string bowl IDs\n", + " \"\"\"\n", + " Pmf.__init__(self)\n", + " for hypo in hypos:\n", + " self.Set(hypo, 1)\n", + " self.Normalize()\n", + "\n", + " def Update(self, data):\n", + " \"\"\"Updates the PMF with new data.\n", + "\n", + " data: string cookie type\n", + " \"\"\"\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " self.Normalize()\n", + "\n", + " mixes = {\n", + " 'Bowl 1':dict(vanilla=0.75, chocolate=0.25),\n", + " 'Bowl 2':dict(vanilla=0.5, chocolate=0.5),\n", + " }\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " \"\"\"The likelihood of the data under the hypothesis.\n", + "\n", + " data: string cookie type\n", + " hypo: string bowl ID\n", + " \"\"\"\n", + " mix = self.mixes[hypo]\n", + " like = mix[data]\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A Cookie object is a Pmf that maps from hypotheses to their\n", + "probabilities. The `__init__` method gives each hypothesis the same\n", + "prior probability. As in the previous section, there are two hypotheses:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "hypos = ['Bowl 1', 'Bowl 2']\n", + "pmf = Cookie(hypos)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Cookie` provides an `Update` method that takes data as a parameter and\n", + "updates the probabilities. `Update` loops through each hypothesis in the suite and multiplies its\n", + "probability by the likelihood of the data under the hypothesis, which is\n", + "computed by `Likelihood`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Likelihood` uses `mixes`, which is a dictionary that maps from the name\n", + "of a bowl to the mix of cookies in the bowl.\n", + "\n", + "Here’s what the update looks like:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "pmf.Update('vanilla')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And then we can print the posterior probability of each hypothesis:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Bowl 1 0.6000000000000001\n", + "Bowl 2 0.4\n" + ] + } + ], + "source": [ + "for hypo, prob in pmf.Items():\n", + " print(hypo, prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "which is the same as what we got before. This code is more complicated\n", + "than what we saw in the previous section. One advantage is that it\n", + "generalizes to the case where we draw more than one cookie from the same\n", + "bowl (with replacement):" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Bowl 1 0.627906976744186\n", + "Bowl 2 0.37209302325581395\n" + ] + } + ], + "source": [ + "dataset = ['vanilla', 'chocolate', 'vanilla']\n", + "for data in dataset:\n", + " pmf.Update(data)\n", + "pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The other advantage is that it provides a framework for solving many\n", + "similar problems. In the next section we’ll solve the Monty Hall problem\n", + "computationally and then see what parts of the framework are the same.\n", + "\n", + "The code in this section is available from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The Monty Hall problem\n", + "\n", + "To solve the Monty Hall problem, I’ll define a new class:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Monty(Pmf):\n", + " \"\"\"Map from string location of car to probability\"\"\"\n", + "\n", + " def __init__(self, hypos):\n", + " \"\"\"Initialize the distribution.\n", + "\n", + " hypos: sequence of hypotheses\n", + " \"\"\"\n", + " Pmf.__init__(self)\n", + " for hypo in hypos:\n", + " self.Set(hypo, 1)\n", + " self.Normalize()\n", + "\n", + " def Update(self, data):\n", + " \"\"\"Updates each hypothesis based on the data.\n", + "\n", + " data: any representation of the data\n", + " \"\"\"\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " self.Normalize()\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " \"\"\"Compute the likelihood of the data under the hypothesis.\n", + "\n", + " hypo: string name of the door where the prize is\n", + " data: string name of the door Monty opened\n", + " \"\"\"\n", + " if hypo == data:\n", + " return 0\n", + " elif hypo == 'A':\n", + " return 0.5\n", + " else:\n", + " return 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So far `Monty` and `Cookie` are nearly the same. And the code that\n", + "creates the Pmf is the same, too, except for the names of the\n", + "hypotheses:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "hypos = 'ABC'\n", + "pmf = Monty(hypos)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Calling `Update` is pretty much the same:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "data = 'B'\n", + "pmf.Update(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the implementation of `Update` is exactly the same. The only part that requires some work is `Likelihood`:\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " if hypo == data:\n", + " return 0\n", + " elif hypo == 'A':\n", + " return 0.5\n", + " else:\n", + " return 1\n", + "```\n", + "Finally, printing the results is the same:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B 0.0\n", + "A 0.3333333333333333\n", + "C 0.6666666666666666\n" + ] + } + ], + "source": [ + "for hypo, prob in pmf.Items():\n", + " print(hypo, prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, writing `Likelihood` is a little complicated, but the\n", + "framework of the Bayesian update is simple. The code in this section is\n", + "available from . For more information\n", + "see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Encapsulating the framework\n", + "\n", + "Now that we see what elements of the framework are the same, we can\n", + "encapsulate them in an object—a `Suite` is a `Pmf` that provides\n", + "`__init__`, `Update`, and `Print`:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Suite(Pmf):\n", + " \"\"\"Represents a suite of hypotheses and their probabilities.\"\"\"\n", + "\n", + " def __init__(self, hypo=tuple()):\n", + " \"\"\"Initializes the distribution.\"\"\"\n", + "\n", + " def Update(self, data):\n", + " \"\"\"Updates each hypothesis based on the data.\"\"\"\n", + "\n", + " def Print(self):\n", + " \"\"\"Prints the hypotheses and their probabilities.\"\"\"\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The implementation of `Suite` is in `thinkbayes.py`. To use `Suite`, you\n", + "should write a class that inherits from it and provides `Likelihood`.\n", + "For example, here is the solution to the Monty Hall problem rewritten to\n", + "use `Suite`:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "from thinkbayes import Suite\n", + "\n", + "class Monty(Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " if hypo == data:\n", + " return 0\n", + " elif hypo == 'A':\n", + " return 0.5\n", + " else:\n", + " return 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And here’s the code that uses this class:" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A 0.3333333333333333\n", + "B 0.0\n", + "C 0.6666666666666666\n" + ] + } + ], + "source": [ + "suite = Monty('ABC')\n", + "suite.Update('B')\n", + "suite.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can download this example from .\n", + "For more information see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The M&M problem\n", + "\n", + "We can use the `Suite` framework to solve the M&M problem.\n", + "Writing the `Likelihood` function is tricky, but everything else is\n", + "straightforward." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class M_and_M(Suite):\n", + " \"\"\"Map from hypothesis (A or B) to probability.\"\"\"\n", + "\n", + " mix94 = dict(brown=30,\n", + " yellow=20,\n", + " red=20,\n", + " green=10,\n", + " orange=10,\n", + " tan=10)\n", + "\n", + " mix96 = dict(blue=24,\n", + " green=20,\n", + " orange=16,\n", + " yellow=14,\n", + " red=13,\n", + " brown=13)\n", + "\n", + " hypoA = dict(bag1=mix94, bag2=mix96)\n", + " hypoB = dict(bag1=mix96, bag2=mix94)\n", + "\n", + " hypotheses = dict(A=hypoA, B=hypoB)\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " \"\"\"Computes the likelihood of the data under the hypothesis.\n", + "\n", + " hypo: string hypothesis (A or B)\n", + " data: tuple of string bag, string color\n", + " \"\"\"\n", + " bag, color = data\n", + " mix = self.hypotheses[hypo][bag]\n", + " like = mix[color]\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First I need to encode the color mixes from before and after 1995:\n", + "\n", + "```python\n", + "mix94 = dict(brown=30,\n", + " yellow=20,\n", + " ...\n", + "\n", + "mix96 = dict(blue=24,\n", + " green=20,\n", + " ...\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then I have to encode the hypotheses:\n", + "\n", + "```python\n", + "hypoA = dict(bag1=mix94, bag2=mix96)\n", + "hypoB = dict(bag1=mix96, bag2=mix94)\n", + "```\n", + "\n", + "`hypoA` represents the hypothesis that Bag 1 is from 1994 and Bag 2 from 1996. `hypoB` is the other way around.\n", + "\n", + "Next I map from the name of the hypothesis to the representation:\n", + "\n", + "```python\n", + "hypotheses = dict(A=hypoA, B=hypoB)\n", + "```\n", + "\n", + "And finally I can write `Likelihood`. In this case the hypothesis,\n", + "`hypo`, is a string, either `A` or `B`. The data is a tuple that\n", + "specifies a bag and a color.\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " bag, color = data\n", + " mix = self.hypotheses[hypo][bag]\n", + " like = mix[color]\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here’s the code that creates the suite and updates it:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A 0.7407407407407407\n", + "B 0.2592592592592592\n" + ] + } + ], + "source": [ + "suite = M_and_M('AB')\n", + "\n", + "suite.Update(('bag1', 'yellow'))\n", + "suite.Update(('bag2', 'green'))\n", + "\n", + "suite.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The posterior probability of A is approximately $20/27$, which is what\n", + "we got before.\n", + "\n", + "The code in this section is available from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "This chapter presents the Suite class, which encapsulates the Bayesian\n", + "update framework.\n", + "\n", + "Suite is an **abstract type**, which means\n", + "that it defines the interface a Suite is supposed to have, but does not\n", + "provide a complete implementation. The Suite interface\n", + "includes Update and Likelihood, but the\n", + "Suite class only provides an implementation of\n", + "Update, not Likelihood.\n", + "\n", + "A **concrete type** is a class that extends an abstract\n", + "parent class and provides an implementation of the missing methods. For\n", + "example, Monty extends Suite, so it inherits\n", + "Update and provides Likelihood.\n", + "\n", + "If you are familiar with design patterns, you might recognize this as an\n", + "example of the template method pattern. You can read about this pattern\n", + "at .\n", + "\n", + "Most of the examples in the following chapters follow the same pattern;\n", + "for each problem we define a new class that extends Suite,\n", + "inherits Update, and provides Likelihood. In a\n", + "few cases we override Update, usually to improve\n", + "performance." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "In Section [framework] I said that the solution to the cookie problem\n", + "generalizes to the case where we draw multiple cookies with replacement.\n", + "\n", + "But in the more likely scenario where we eat the cookies we draw, the\n", + "likelihood of each draw depends on the previous draws.\n", + "\n", + "Modify the solution in this chapter to handle selection without\n", + "replacement. Hint: add instance variables to Cookie to\n", + "represent the hypothetical state of the bowls, and modify\n", + "Likelihood accordingly. You might want to define a\n", + "Bowl object." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/03_Estimation.ipynb b/notebooks/03_Estimation.ipynb index e35e9e4..1402e00 100644 --- a/notebooks/03_Estimation.ipynb +++ b/notebooks/03_Estimation.ipynb @@ -1,495 +1,1087 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "Estimation\n", - "==========\n", - "\n", - "The dice problem\n", - "----------------\n", - "\n", - "Suppose I have a box of dice that contains a 4-sided die, a 6-sided die,\n", - "an 8-sided die, a 12-sided die, and a 20-sided die. If you have ever\n", - "played *Dungeons & Dragons*, you know what I am talking\n", - "about.\n", - "\n", - "Suppose I select a die from the box at random, roll it, and get a 6.\n", - "What is the probability that I rolled each die?\n", - "\n", - "Let me suggest a three-step strategy for approaching a problem like\n", - "this.\n", - "\n", - "1. Choose a representation for the hypotheses.\n", - "\n", - "2. Choose a representation for the data.\n", - "\n", - "3. Write the likelihood function.\n", - "\n", - "In previous examples I used strings to represent hypotheses and data,\n", - "but for the die problem I’ll use numbers. Specifically, I’ll use the\n", - "integers 4, 6, 8, 12, and 20 to represent hypotheses:\n", - "\n", - " suite = Dice([4, 6, 8, 12, 20])\n", - "\n", - "And integers from 1 to 20 for the data. These representations make it\n", - "easy to write the likelihood function:\n", - "\n", - " class Dice(Suite):\n", - " def Likelihood(self, data, hypo):\n", - " if hypo < data:\n", - " return 0\n", - " else:\n", - " return 1.0/hypo\n", - "\n", - "Here’s how `Likelihood` works. If `hypohypo\n", - "sides, what is the chance of rolling data?” The answer is\n", - "`1/hypo`, regardless of data.\n", - "\n", - "Here is the statement that does the update (if I roll a 6):\n", - "\n", - " suite.Update(6)\n", - "\n", - "And here is the posterior distribution:\n", - "\n", - " 4 0.0\n", - " 6 0.392156862745\n", - " 8 0.294117647059\n", - " 12 0.196078431373\n", - " 20 0.117647058824\n", - "\n", - "After we roll a 6, the probability for the 4-sided die is 0. The most\n", - "likely alternative is the 6-sided die, but there is still almost a 12%\n", - "chance for the 20-sided die.\n", - "\n", - "What if we roll a few more times and get 6, 8, 7, 7, 5, and 4?\n", - "\n", - " for roll in [6, 8, 7, 7, 5, 4]:\n", - " suite.Update(roll)\n", - "\n", - "With this data the 6-sided die is eliminated, and the 8-sided die seems\n", - "quite likely. Here are the results:\n", - "\n", - " 4 0.0\n", - " 6 0.0\n", - " 8 0.943248453672\n", - " 12 0.0552061280613\n", - " 20 0.0015454182665\n", - "\n", - "Now the probability is 94% that we are rolling the 8-sided die, and less\n", - "than 1% for the 20-sided die.\n", - "\n", - "The dice problem is based on an example I saw in Sanjoy Mahajan’s class\n", - "on Bayesian inference. You can download the code in this section from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "The locomotive problem\n", - "----------------------\n", - "\n", - "I found the locomotive problem in Frederick Mosteller’s, *Fifty\n", - "Challenging Problems in Probability with Solutions* (Dover,\n", - "1987):\n", - "\n", - "> “A railroad numbers its locomotives in order 1..N. One day you see a\n", - "> locomotive with the number 60. Estimate how many locomotives the\n", - "> railroad has.”\n", - "\n", - "Based on this observation, we know the railroad has 60 or more\n", - "locomotives. But how many more? To apply Bayesian reasoning, we can\n", - "break this problem into two steps:\n", - "\n", - "1. What did we know about $N$ before we saw the data?\n", - "\n", - "2. For any given value of $N$, what is the likelihood of seeing the\n", - " data (a locomotive with number 60)?\n", - "\n", - "The answer to the first question is the prior. The answer to the second\n", - "is the likelihood.\n", - "\n", - "![Posterior distribution for the locomotive problem, based on a uniform\n", - "prior.](figs/train1.pdf)\n", - "\n", - "[fig.train1]\n", - "\n", - "We don’t have much basis to choose a prior, but we can start with\n", - "something simple and then consider alternatives. Let’s assume that $N$\n", - "is equally likely to be any value from 1 to 1000.\n", - "\n", - " hypos = xrange(1, 1001)\n", - "\n", - "Now all we need is a likelihood function. In a hypothetical fleet of $N$\n", - "locomotives, what is the probability that we would see number 60? If we\n", - "assume that there is only one train-operating company (or only one we\n", - "care about) and that we are equally likely to see any of its\n", - "locomotives, then the chance of seeing any particular locomotive is\n", - "$1/N$.\n", - "\n", - "Here’s the likelihood function:\n", - "\n", - " class Train(Suite):\n", - " def Likelihood(self, data, hypo):\n", - " if hypo < data:\n", - " return 0\n", - " else:\n", - " return 1.0/hypo\n", - "\n", - "This might look familiar; the likelihood functions for the locomotive\n", - "problem and the dice problem are identical.\n", - "\n", - "Here’s the update:\n", - "\n", - " suite = Train(hypos)\n", - " suite.Update(60)\n", - "\n", - "There are too many hypotheses to print, so I plotted the results in\n", - "Figure [fig.train1]. Not surprisingly, all values of $N$ below 60 have\n", - "been eliminated.\n", - "\n", - "The most likely value, if you had to guess, is 60. That might not seem\n", - "like a very good guess; after all, what are the chances that you just\n", - "happened to see the train with the highest number? Nevertheless, if you\n", - "want to maximize the chance of getting the answer exactly right, you\n", - "should guess 60.\n", - "\n", - "But maybe that’s not the right goal. An alternative is to compute the\n", - "mean of the posterior distribution:\n", - "\n", - " def Mean(suite):\n", - " total = 0\n", - " for hypo, prob in suite.Items():\n", - " total += hypo * prob\n", - " return total\n", - "\n", - " print Mean(suite)\n", - "\n", - "Or you could use the very similar method provided by Pmf:\n", - "\n", - " print suite.Mean()\n", - "\n", - "The mean of the posterior is 333, so that might be a good guess if you\n", - "wanted to minimize error. If you played this guessing game over and\n", - "over, using the mean of the posterior as your estimate would minimize\n", - "the mean squared error over the long run (see\n", - ").\n", - "\n", - "You can download this example from . For\n", - "more information see Section [download].\n", - "\n", - "What about that prior?\n", - "----------------------\n", - "\n", - "To make any progress on the locomotive problem we had to make\n", - "assumptions, and some of them were pretty arbitrary. In particular, we\n", - "chose a uniform prior from 1 to 1000, without much justification for\n", - "choosing 1000, or for choosing a uniform distribution.\n", - "\n", - "It is not crazy to believe that a railroad company might operate 1000\n", - "locomotives, but a reasonable person might guess more or fewer. So we\n", - "might wonder whether the posterior distribution is sensitive to these\n", - "assumptions. With so little data—only one observation—it probably is.\n", - "\n", - "Recall that with a uniform prior from 1 to 1000, the mean of the\n", - "posterior is 333. With an upper bound of 500, we get a posterior mean of\n", - "207, and with an upper bound of 2000, the posterior mean is 552.\n", - "\n", - "So that’s bad. There are two ways to proceed:\n", - "\n", - "- Get more data.\n", - "\n", - "- Get more background information.\n", - "\n", - "With more data, posterior distributions based on different priors tend\n", - "to converge. For example, suppose that in addition to train 60 we also\n", - "see trains 30 and 90. We can update the distribution like this:\n", - "\n", - " for data in [60, 30, 90]:\n", - " suite.Update(data)\n", - "\n", - "With these data, the means of the posteriors are\n", - "\n", - " ------- -----------\n", - " Upper Posterior\n", - " Bound Mean\n", - " 500 152\n", - " 1000 164\n", - " 2000 171\n", - " ------- -----------\n", - "\n", - "So the differences are smaller.\n", - "\n", - "An alternative prior\n", - "--------------------\n", - "\n", - "![Posterior distribution based on a power law prior, compared to a\n", - "uniform prior.](figs/train4.pdf)\n", - "\n", - "[fig.train4]\n", - "\n", - "If more data are not available, another option is to improve the priors\n", - "by gathering more background information. It is probably not reasonable\n", - "to assume that a train-operating company with 1000 locomotives is just\n", - "as likely as a company with only 1.\n", - "\n", - "With some effort, we could probably find a list of companies that\n", - "operate locomotives in the area of observation. Or we could interview an\n", - "expert in rail shipping to gather information about the typical size of\n", - "companies.\n", - "\n", - "But even without getting into the specifics of railroad economics, we\n", - "can make some educated guesses. In most fields, there are many small\n", - "companies, fewer medium-sized companies, and only one or two very large\n", - "companies. In fact, the distribution of company sizes tends to follow a\n", - "power law, as Robert Axtell reports in *Science* (see\n", - ").\n", - "\n", - "This law suggests that if there are 1000 companies with fewer than 10\n", - "locomotives, there might be 100 companies with 100 locomotives, 10\n", - "companies with 1000, and possibly one company with 10,000 locomotives.\n", - "\n", - "Mathematically, a power law means that the number of companies with a\n", - "given size is inversely proportional to size, or\n", - "$${\\mathrm{PMF}}(x) \\propto \\left( \\frac{1}{x} \\right)^{\\alpha}$$ where\n", - "${\\mathrm{PMF}}(x)$ is the probability mass function of $x$ and $\\alpha$\n", - "is a parameter that is often near 1.\n", - "\n", - "We can construct a power law prior like this:\n", - "\n", - " class Train(Dice):\n", - "\n", - " def __init__(self, hypos, alpha=1.0):\n", - " Pmf.__init__(self)\n", - " for hypo in hypos:\n", - " self.Set(hypo, hypo**(-alpha))\n", - " self.Normalize()\n", - "\n", - "And here’s the code that constructs the prior:\n", - "\n", - " hypos = range(1, 1001)\n", - " suite = Train(hypos)\n", - "\n", - "Again, the upper bound is arbitrary, but with a power law prior, the\n", - "posterior is less sensitive to this choice.\n", - "\n", - "Figure [fig.train4] shows the new posterior based on the power law,\n", - "compared to the posterior based on the uniform prior. Using the\n", - "background information represented in the power law prior, we can all\n", - "but eliminate values of $N$ greater than 700.\n", - "\n", - "If we start with this prior and observe trains 30, 60, and 90, the means\n", - "of the posteriors are\n", - "\n", - " ------- -----------\n", - " Upper Posterior\n", - " Bound Mean\n", - " 500 131\n", - " 1000 133\n", - " 2000 134\n", - " ------- -----------\n", - "\n", - "Now the differences are much smaller. In fact, with an arbitrarily large\n", - "upper bound, the mean converges on 134.\n", - "\n", - "So the power law prior is more realistic, because it is based on general\n", - "information about the size of companies, and it behaves better in\n", - "practice.\n", - "\n", - "You can download the examples in this section from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Credible intervals {#credible}\n", - "------------------\n", - "\n", - "Once you have computed a posterior distribution, it is often useful to\n", - "summarize the results with a single point estimate or an interval. For\n", - "point estimates it is common to use the mean, median, or the value with\n", - "maximum likelihood.\n", - "\n", - "For intervals we usually report two values computed so that there is a\n", - "90% chance that the unknown value falls between them (or any other\n", - "probability). These values define a **credible interval**.\n", - "\n", - "A simple way to compute a credible interval is to add up the\n", - "probabilities in the posterior distribution and record the values that\n", - "correspond to probabilities 5% and 95%. In other words, the 5th and 95th\n", - "percentiles.\n", - "\n", - "`thinkbayes` provides a function that computes percentiles:\n", - "\n", - " def Percentile(pmf, percentage):\n", - " p = percentage / 100.0\n", - " total = 0\n", - " for val, prob in pmf.Items():\n", - " total += prob\n", - " if total >= p:\n", - " return val \n", - "\n", - "And here’s the code that uses it:\n", - "\n", - " interval = Percentile(suite, 5), Percentile(suite, 95)\n", - " print interval\n", - "\n", - "For the previous example—the locomotive problem with a power law prior\n", - "and three trains—the 90% credible interval is $(91, 243)$. The width of\n", - "this range suggests, correctly, that we are still quite uncertain about\n", - "how many locomotives there are.\n", - "\n", - "Cumulative distribution functions\n", - "---------------------------------\n", - "\n", - "In the previous section we computed percentiles by iterating through the\n", - "values and probabilities in a Pmf. If we need to compute more than a few\n", - "percentiles, it is more efficient to use a cumulative distribution\n", - "function, or Cdf.\n", - "\n", - "Cdfs and Pmfs are equivalent in the sense that they contain the same\n", - "information about the distribution, and you can always convert from one\n", - "to the other. The advantage of the Cdf is that you can compute\n", - "percentiles more efficiently.\n", - "\n", - "thinkbayes provides a Cdf class that\n", - "represents a cumulative distribution function. Pmf provides\n", - "a method that makes the corresponding Cdf:\n", - "\n", - " cdf = suite.MakeCdf()\n", - "\n", - "And Cdf provides a function named `Percentile`\n", - "\n", - " interval = cdf.Percentile(5), cdf.Percentile(95)\n", - "\n", - "Converting from a Pmf to a Cdf takes time proportional to the number of\n", - "values, len(pmf). The Cdf stores the values and\n", - "probabilities in sorted lists, so looking up a probability to get the\n", - "corresponding value takes “log time”: that is, time proportional to the\n", - "logarithm of the number of values. Looking up a value to get the\n", - "corresponding probability is also logarithmic, so Cdfs are efficient for\n", - "many calculations.\n", - "\n", - "The examples in this section are in .\n", - "For more information see Section [download].\n", - "\n", - "The German tank problem\n", - "-----------------------\n", - "\n", - "During World War II, the Economic Warfare Division of the American\n", - "Embassy in London used statistical analysis to estimate German\n", - "production of tanks and other equipment.[^2]\n", - "\n", - "The Western Allies had captured log books, inventories, and repair\n", - "records that included chassis and engine serial numbers for individual\n", - "tanks.\n", - "\n", - "Analysis of these records indicated that serial numbers were allocated\n", - "by manufacturer and tank type in blocks of 100 numbers, that numbers in\n", - "each block were used sequentially, and that not all numbers in each\n", - "block were used. So the problem of estimating German tank production\n", - "could be reduced, within each block of 100 numbers, to a form of the\n", - "locomotive problem.\n", - "\n", - "Based on this insight, American and British analysts produced estimates\n", - "substantially lower than estimates from other forms of intelligence. And\n", - "after the war, records indicated that they were substantially more\n", - "accurate.\n", - "\n", - "They performed similar analyses for tires, trucks, rockets, and other\n", - "equipment, yielding accurate and actionable economic intelligence.\n", - "\n", - "The German tank problem is historically interesting; it is also a nice\n", - "example of real-world application of statistical estimation. So far many\n", - "of the examples in this book have been toy problems, but it will not be\n", - "long before we start solving real problems. I think it is an advantage\n", - "of Bayesian analysis, especially with the computational approach we are\n", - "taking, that it provides such a short path from a basic introduction to\n", - "the research frontier.\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "Among Bayesians, there are two approaches to choosing prior\n", - "distributions. Some recommend choosing the prior that best represents\n", - "background information about the problem; in that case the prior is said\n", - "to be **informative**. The problem with using an\n", - "informative prior is that people might use different background\n", - "information (or interpret it differently). So informative priors often\n", - "seem subjective.\n", - "\n", - "The alternative is a so-called **uninformative prior**,\n", - "which is intended to be as unrestricted as possible, in order to let the\n", - "data speak for themselves. In some cases you can identify a unique prior\n", - "that has some desirable property, like representing minimal prior\n", - "information about the estimated quantity.\n", - "\n", - "Uninformative priors are appealing because they seem more objective. But\n", - "I am generally in favor of using informative priors. Why? First,\n", - "Bayesian analysis is always based on modeling decisions. Choosing the\n", - "prior is one of those decisions, but it is not the only one, and it\n", - "might not even be the most subjective. So even if an uninformative prior\n", - "is more objective, the entire analysis is still subjective.\n", - "\n", - "Also, for most practical problems, you are likely to be in one of two\n", - "regimes: either you have a lot of data or not very much. If you have a\n", - "lot of data, the choice of the prior doesn’t matter very much;\n", - "informative and uninformative priors yield almost the same results.\n", - "We’ll see an example like this in the next chapter.\n", - "\n", - "But if, as in the locomotive problem, you don’t have much data, using\n", - "relevant background information (like the power law distribution) makes\n", - "a big difference.\n", - "\n", - "And if, as in the German tank problem, you have to make life-and-death\n", - "decisions based on your results, you should probably use all of the\n", - "information at your disposal, rather than maintaining the illusion of\n", - "objectivity by pretending to know less than you do.\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "To write a likelihood function for the locomotive problem, we had to\n", - "answer this question: “If the railroad has $N$ locomotives, what is the\n", - "probability that we see number 60?”\n", - "\n", - "The answer depends on what sampling process we use when we observe the\n", - "locomotive. In this chapter, I resolved the ambiguity by specifying that\n", - "there is only one train-operating company (or only one that we care\n", - "about).\n", - "\n", - "But suppose instead that there are many companies with different numbers\n", - "of trains. And suppose that you are equally likely to see any train\n", - "operated by any company. In that case, the likelihood function is\n", - "different because you are more likely to see a train operated by a large\n", - "company.\n", - "\n", - "As an exercise, implement the likelihood function for this variation of\n", - "the locomotive problem, and compare the results.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Estimation\n", + "\n", + "## The dice problem\n", + "\n", + "Suppose I have a box of dice that contains a 4-sided die, a 6-sided die,\n", + "an 8-sided die, a 12-sided die, and a 20-sided die. If you have ever\n", + "played *Dungeons & Dragons*, you know what I am talking\n", + "about.\n", + "\n", + "Suppose I select a die from the box at random, roll it, and get a 6.\n", + "What is the probability that I rolled each die?\n", + "\n", + "Let me suggest a three-step strategy for approaching a problem like\n", + "this.\n", + "\n", + "1. Choose a representation for the hypotheses.\n", + "\n", + "2. Choose a representation for the data.\n", + "\n", + "3. Write the likelihood function.\n", + "\n", + "In previous examples I used strings to represent hypotheses and data,\n", + "but for the die problem I’ll use numbers. Specifically, I’ll use the\n", + "integers 4, 6, 8, 12, and 20 to represent hypotheses:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from dice import Dice\n", + "\n", + "suite = Dice([4, 6, 8, 12, 20])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And integers from 1 to 20 for the data. These representations make it\n", + "easy to write the likelihood function:\n", + "\n", + "```python\n", + "class Dice(Suite):\n", + " def Likelihood(self, data, hypo):\n", + " if hypo < data:\n", + " return 0\n", + " else:\n", + " return 1.0/hypo\n", + "```\n", + "\n", + "Here’s how `Likelihood` works. If `hypo. For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The locomotive problem\n", + "\n", + "I found the locomotive problem in Frederick Mosteller’s, *Fifty\n", + "Challenging Problems in Probability with Solutions* (Dover,\n", + "1987):\n", + "\n", + "> “A railroad numbers its locomotives in order 1..N. One day you see a\n", + "> locomotive with the number 60. Estimate how many locomotives the\n", + "> railroad has.”\n", + "\n", + "Based on this observation, we know the railroad has 60 or more\n", + "locomotives. But how many more? To apply Bayesian reasoning, we can\n", + "break this problem into two steps:\n", + "\n", + "1. What did we know about $N$ before we saw the data?\n", + "\n", + "2. For any given value of $N$, what is the likelihood of seeing the\n", + " data (a locomotive with number 60)?\n", + "\n", + "The answer to the first question is the prior. The answer to the second\n", + "is the likelihood.\n", + "\n", + "![Posterior distribution for the locomotive problem, based on a uniform\n", + "prior.](figs/train1.pdf)\n", + "\n", + "[fig.train1]\n", + "\n", + "We don’t have much basis to choose a prior, but we can start with\n", + "something simple and then consider alternatives. Let’s assume that $N$\n", + "is equally likely to be any value from 1 to 1000." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "hypos = range(1, 1001)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now all we need is a likelihood function. In a hypothetical fleet of $N$\n", + "locomotives, what is the probability that we would see number 60? If we\n", + "assume that there is only one train-operating company (or only one we\n", + "care about) and that we are equally likely to see any of its\n", + "locomotives, then the chance of seeing any particular locomotive is\n", + "$1/N$.\n", + "\n", + "Here’s the likelihood function:\n", + "\n", + "```python\n", + "class Train(Suite):\n", + " def Likelihood(self, data, hypo):\n", + " if hypo < data:\n", + " return 0\n", + " else:\n", + " return 1.0/hypo\n", + "```\n", + "\n", + "This might look familiar; the likelihood functions for the locomotive\n", + "problem and the dice problem are identical.\n", + "\n", + "Here’s the update:" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxQAAAEtCAYAAACRaSbdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XtcVHX+P/DXzMDADPeLDFdBRUXBEhlMRA0vQK5t62XV\nci3NtWKzC9jPymptK+m7tq7Xr4K7mmm6ZV9zrXbVQE1DjRQMEgGvgHIZLnK/CQzz+wM5MgJyZ2B4\nPR8PHp458zln3odOyovP53M+Io1GowEREREREVEniHVdABERERER9V8MFERERERE1GkMFERERERE\n1GkMFERERERE1GkMFERERERE1GkMFERERERE1GkMFERERERE1Gl9IlBs374dQ4cOhUwmg1KpxJkz\nZx7aPikpCQEBAZDL5XBxccFHH33UrM3p06ehVCohk8ng7u6OHTt2NGtTVlaG1157DU5OTjA2NsaI\nESNw8ODBbrsuIiIiIiJ9Z6DrAg4cOIDQ0FBERkbC398f27Ztw8yZM5GSkgJnZ+dm7cvKyhAYGIiA\ngADEx8cjJSUFS5cuhampKcLCwgAA6enpmDVrFpYvX479+/cjJiYGL7/8Muzs7DBnzhwAQF1dHWbM\nmAFbW1scPHgQTk5OyMzMhJGRUa9ePxERERFRfybS9UrZEyZMwNixYxEZGSnsGzFiBObPn4/w8PBm\n7SMiIrB69Wrk5eVBKpUCAMLDwxEZGYnbt28DAN566y0cPnwYV65cEY574YUXkJycjLNnzwIA/vGP\nf+CTTz5BamoqDAx0nquIiIiIiPolnQ55qq2tRXx8PAIDA7X2BwUF4dy5cy0eExsbi8mTJwthAgCC\ng4ORnZ2NjIwMoU1QUJDWccHBwYiLi4NarQYAfPPNN/D398crr7wCBwcHeHp64oMPPkBdXV13XiIR\nERERkV7TaaAoKCiAWq2GQqHQ2q9QKKBSqVo8RqVStdheo9EIx7TWpq6uDgUFBQCAmzdv4quvvkJd\nXR2OHDmCtWvXIjIyEu+88053XR4RERERkd4bsGN96uvroVAo8M9//hMikQje3t4oKCjAypUr8ckn\nn2i1LSkp0VGVRERERERdZ2Fh0WPn1mmgsLW1hUQiQW5urtb+3Nxc2Nvbt3iMvb19i+1FIpFwTGtt\nDAwMYGtrCwBwcHCAVCqFSCQS2owaNQqVlZW4c+cObGxsunx9RERERET6TqdDngwNDeHj44Po6Git\n/dHR0fD392/xGD8/P8TExKCmpkbYFxUVBUdHR7i6ugptHjxnVFQUlEolJBIJAMDf3x/Xr1/XanPl\nyhXI5XKGCSIiIiKidtL5U56++uorPPfcc9i2bRv8/f0RERGB3bt3Izk5Gc7Ozli9ejUuXLiA48eP\nAwBKS0vh4eGBgIAAvPvuu7hy5Qqef/55fPDBBwgNDQXQ8NjYMWPGYPny5XjppZdw5swZvPLKK/jy\nyy8xe/ZsAEBmZia8vLzw3HPP4ZVXXkFaWhqWL1+OP/zhD/jrX/+qVWPTIU892V3UFWcvfY/oCwfh\n/8hMBCrn6rqcASkuLg4AoFQqdVwJ9WW8T6g9eJ9QW3iPUHv01s+wOp9DsWDBAhQWFiI8PBw5OTnw\n8vLC0aNHhTUoVCoV0tLShPbm5uaIjo7GihUr4OvrCysrK6xatUoIEwDg5uaGI0eOICwsDJGRkXB0\ndMTWrVuFMAEAzs7OiIqKwsqVK+Ht7Q17e3ssX74c7777bu9dfDc6cDICAPDd2b2YMHo6zOR9M/gQ\nERERkX7ReaAAgJCQEISEhLT43u7du5vt8/T0xKlTpx56zsmTJwvpvTXjx49vc1Xu/uh23g2Mdhun\n6zKIiIiIaADQ6RwK6hlZBem6LoGIiIiIBggGCj2UzUBBRERERL2EgUIPMVAQERERUW9hoNBDOXdu\nQV2v1nUZRERERDQAMFDoCYlYe359XlGWjiohIiIiooGEgUJPaKC9nEhWflorLYmIiIiIug8Dhb54\nYH3CrAIGCiIiIiLqeQwUeuLB5c6z8tN1UQYRERERDTAMFHqKa1EQERERUW9goNAXDwx5KqssRmlF\nkY6KISIiIqKBgoFCTzw4KRsAbufd0EElRERERDSQMFDosVu513VdAhERERHpOQYKPXYrj4GCiIiI\niHoWA4Ue0GiaD3cCgNu5N1p9j4iIiIioOzBQ6IEH508YGRoDAEori1BSUaiLkoiIiIhogGCg0AdN\neiFEEMHFbpjwmvMoiIiIiKgnMVDoG5EIgxXuwsvbnEdBRERERD2IgUIPPDhLwsXufqC4lctHxxIR\nERFRz2Gg0AdaQ56g1UNxK+86J2YTERERUY9hoNADWpOyRSLYWthDJpUDACqqSlFYlqejyoiIiIhI\n3zFQ6BkRRBCJRBisGC7sS8+5qsOKiIiIiEifMVDogZZGNLk5jBC201VXerEaIiIiIhpIGCj0QtMh\nTw1/uNmPFHal5zBQEBEREVHPYKDQA03nUIjuJQo3h/uBIjM/DbV1Nb1eFxERERHpPwYKPdMYKEyM\nzWBn5QQAUNfX4XYeHx9LRERERN2PgUIftPJU2CFNhj2lcdgTEREREfUABgo9oGlhDgWgPewpPSe1\nFysiIiIiooGCgUIfaJrPoQC0J2anqa5wgTsiIiIi6nYMFPpGdD9QONi4wEgqAwCUVhShqCxfV1UR\nERERkZ7qE4Fi+/btGDp0KGQyGZRKJc6cOfPQ9klJSQgICIBcLoeLiws++uijZm1Onz4NpVIJmUwG\nd3d37NixQ+v9PXv2QCwWQyKRQCwWC9s1Nf3vaUit9TuIxRK4Ke6vR3EjO6V3CiIiIiKiAUPngeLA\ngQMIDQ3Fe++9h4SEBEycOBEzZ85EZmZmi+3LysoQGBgIBwcHxMfHY/Pmzfjb3/6GjRs3Cm3S09Mx\na9YsTJo0CQkJCXj77bfx6quv4t///rfWuUxMTKBSqYSvnJwcSKXSHr3enqDRGvKkbajTaGH7RlZS\nL1VERERERAOFga4L2LhxI5YtW4Zly5YBALZs2YJjx44hIiIC4eHhzdrv27cPVVVV2LNnD6RSKUaN\nGoWUlBRs2LABYWFhAICIiAg4OTlh06ZNAICRI0fi559/xvr16zFnzhzhXCKRCIMGDeqFq+xpTSdl\na0cKdydPYft65uXeKoiIiIiIBgid9lDU1tYiPj4egYGBWvuDgoJw7ty5Fo+JjY3F5MmTtXoSgoOD\nkZ2djYyMDKFNUFCQ1nHBwcGIi4uDWq0W9lVVVcHNzQ0uLi747W9/i4SEhO66NJ0RPdBH4WY/AgYS\nQwBAXnE2SioKdVEWEREREekpnQaKgoICqNVqKBQKrf0KhQIqlarFY1QqVYvtNRqNcExrberq6lBQ\nUACgodfi008/xbfffosvv/wSxsbG8Pf3x40b/W8BOE2rsygAQwMpXO2bzKPISu6NkoiIiIhogND5\nkCddmTBhAiZMmCC89vPzg7e3N7Zu3SoMlWpJXFxcb5TXIXfrqoRttVrdrEYTkZWwfe6XH1Bfatxr\ntQ1EffEeob6H9wm1B+8TagvvEXqY4cOH98rn6LSHwtbWFhKJBLm5uVr7c3NzYW9v3+Ix9vb2LbYX\niUTCMa21MTAwgK2tbYvnFYvF8PHxwbVr1zp7Obqjta7dg9OyAYWFq7CdW5LRGxURERER0QCh0x4K\nQ0ND+Pj4IDo6GvPmzRP2R0dHY/78+S0e4+fnh7fffhs1NTXCPIqoqCg4OjrC1dVVaHP48GGt46Ki\noqBUKiGRSFqtJzExEd7e3g+tWalUtuvaelNFdRkOnG/YNjAwaFZjTe0YnEw5AHV9HUqqCjBytDvM\n5JY6qFS/Nf6WqC/eI9R38D6h9uB9Qm3hPULtUVJS0iufo/PHxq5cuRKfffYZdu3ahdTUVLz++uvI\nyclBSEgIAGD16tWYMWOG0H7RokWQy+VYunQpLl++jEOHDmHdunV44403hDYhISHIyspCWFgYUlNT\nsXPnTuzduxerVq0S2nz44YeIiopCWloaEhMTsWzZMiQlJeFPf/pT7118d2ljBWypoREGK9yF19c5\nj4KIiIiIuonO51AsWLAAhYWFCA8PR05ODry8vHD06FE4OzsDaJhgnZaWJrQ3NzdHdHQ0VqxYAV9f\nX1hZWWHVqlUIDQ0V2ri5ueHIkSMICwtDZGQkHB0dsXXrVsyePVtoU1xcjJdeegkqlQoWFhbw9vZG\nTEwMfHx8eu/iu4lWnBA1H/IENDw+Ni0nFUDDehTewyf2fGFEREREpPd0HiiAhh6Fxh6JB+3evbvZ\nPk9PT5w6deqh55w8efJDJypt2LABGzZs6FCdfdXDFrZr5O7shei4rwFwPQoiIiIi6j46H/JE3ayV\nHoqhDh4Qixr+c2ffyUBZZXFvVkVEREREeoqBQi88fA4FABhJZXCzHym8vnIrsScLIiIiIqIBgoFC\nD2jaeGxsIw/XscJ26q3+vyo4EREREekeA4VeaHsOBQB4uN5/JG7qrQStuRdERERERJ3BQKFvWplD\nAQCD7YZBbmQKACitKELOnVu9VRURERER6SkGCj2gacccCgAQiyUYMfgR4TWHPRERERFRVzFQ6IN2\nzqEAAI/B2sOeiIiIiIi6goFCD2i0E8VDeQx+VNi+kXkZtXU1PVQVEREREQ0EDBR6pq0eCmtzO9hZ\nOQEAatU1uJGV3BtlEREREZGeYqDQAx19WtMoVw57IiIiIqLuwUChF5oOeWpjzBMAj8H316NITo/v\niYKIiIiIaIBgoNADmvZPoQAADHceA0OJFACgKryN/OKcnimMiIiIiPQeA4WeaWsOBQBIDY0wssnk\n7KS0Cz1ZEhERERHpMQYKvdDxFa+9ho4XtpNuMlAQERERUecwUOgBTQfnUACA1xClsH0j6zIqq8u7\nuywiIiIiGgAYKPRBB+dQAIC5iRVc7UcAAOo19ZycTURERESdwkChb9rZQwEAXkN8hW3OoyAiIiKi\nzmCg0AOaTsyhAIAxQ+8HiuT0i6hT13ZXSUREREQ0QDBQ6IGmC9u15ylPjRxsXGFtbgcAqK6p5KrZ\nRERERNRhDBT6pv15AiKRCGOaPO0p4fpPPVAQEREREekzBgo905EeCgB4ZNgEYfvX6z9BXa/u7pKI\niIiISI8xUOiBpkOeOmqY4yiYy60AAGVVJbiRdbm7yiIiIiKiAYCBQi90bg4FAIjFEowd7ie8/uXq\n2W6rioiIiIj0HwOFHtDqn+jAY2MbeQ+fJGwn3OCwJyIiIiJqPwYKPdPxOAEMcfSAhYk1AKCiqhTX\nM5O6tygiIiIi0lsMFPqg6RyKTvRQiEVijB0+UXj9y7Uz3VEVEREREQ0ADBR6oLML2zXVdNhT4vVY\nqNV1XT4nEREREek/Bgo9oNVB0alBT4CbwwhYmtoAACqqy3DldmJ3lEZEREREeo6BQt90Lk9ALBJj\n3IjJwuvzKae6px4iIiIi0mt9IlBs374dQ4cOhUwmg1KpxJkzDx/Dn5SUhICAAMjlcri4uOCjjz5q\n1ub06dNQKpWQyWRwd3fHjh07Wj3fF198AbFYjKeeeqrL16IbnX9sbFPjRwUI25du/IyquxVdKYqI\niIiIBgCdB4oDBw4gNDQU7733HhISEjBx4kTMnDkTmZmZLbYvKytDYGAgHBwcEB8fj82bN+Nvf/sb\nNm7cKLRJT0/HrFmzMGnSJCQkJODtt9/Gq6++in//+9/Nznfz5k28+eabmDJlSo9dY0/rysJ2TTna\nusFp0BAAQK26Br9cO9ct5yUiIiIi/aXzQLFx40YsW7YMy5Ytw8iRI7FlyxY4ODggIiKixfb79u1D\nVVUV9uzZg1GjRmHu3Ll46623sGHDBqFNREQEnJycsGnTJowcORLLly/HkiVLsH79eq1z1dXVYdGi\nRfj4448xZMiQHr3OntWkh6ITT3lqavyoqcL2+ZSTXToXEREREem/DgeK7vptOADU1tYiPj4egYGB\nWvuDgoJw7lzLvx2PjY3F5MmTIZVKhX3BwcHIzs5GRkaG0CYoKEjruODgYMTFxUGtvr9o2zvvvIOh\nQ4fi2Wef7a5L6vd8RkyBWNRwW9zMTkF+cY6OKyIiIiKivsygowe4uLhg8eLFWLx4Mby8vLr04QUF\nBVCr1VAoFFr7FQoFTpw40eIxKpUKLi4uzdprNBqoVCq4urpCpVI1CykKhQJ1dXUoKCiAQqFAVFQU\nDh48iMTEjj3NKC4urkPte0NRRa6wXVVV3eUaHS2HIbPoGgDgm5P/wtjBj3fpfANNX7xHqO/hfULt\nwfuE2sJ7hB5m+PDhvfI5He6hGD9+PDZv3oxHH30U3t7e2LhxI1QqVU/U1mMKCgrw/PPPY8+ePTAz\nM9N1OV3WfX1GDYbajRG2b+Zd6tZeKSIiIiLSLx3uoTh06BBKSkpw4MAB7N+/H//v//0/vPnmm5g+\nfTqWLFmC2bNnQyaTtetctra2kEgkyM3N1dqfm5sLe3v7Fo+xt7dvsb1IJBKOaa2NgYEBbG1tcebM\nGahUKkyfPl34Ybm+vh4AIJVKcfny5VYTnVKpbNe19aas/DT8J6FhWy6Xd7nGR+sewYX071F1twLl\nd4thZmcID9ex3VCpfmv8LVFfvEeo7+B9Qu3B+4TawnuE2qOkpKRXPqdTk7ItLCzw4osv4vTp07h5\n8yY++OADZGZmYvHixVAoFHj++edx8mTbE3oNDQ3h4+OD6Ohorf3R0dHw9/dv8Rg/Pz/ExMSgpqZG\n2BcVFQVHR0e4uroKbR48Z1RUFJRKJSQSCXx9fXHp0iUkJCQgMTERiYmJeOqppzBlyhQkJib2uwna\nGq3HxnadoYFUa3L22aTvu+GsRERERKSPuvyUJ1dXV7zzzjs4duwY5s+fj/LycuzZsweBgYFwdXXF\npk2btCZCP2jlypX47LPPsGvXLqSmpuL1119HTk4OQkJCAACrV6/GjBkzhPaLFi2CXC7H0qVLcfny\nZRw6dAjr1q3DG2+8IbQJCQlBVlYWwsLCkJqaip07d2Lv3r1YtWoVgIbf4o8ePVrry9LSEmZmZhg1\nahQMDDrccaNTWiOSuviUp0YTvYKF7Us3fkZJRWG3nJeIiIiI9EuXfnIuKyvDwYMHsW/fPpw+fRoS\niQS/+93vsGTJEkilUvzjH//AypUrkZSUhJ07d7Z4jgULFqCwsBDh4eHIycmBl5cXjh49CmdnZwAN\nk7DT0tKE9ubm5oiOjsaKFSvg6+sLKysrrFq1CqGhoUIbNzc3HDlyBGFhYYiMjISjoyO2bt2K2bNn\nd+Vy+7Dun+PgYOOCYY6jcSM7GfWaesRePoHg8fO7/XOIiIiIqH/rcKBQq9U4evQo9u3bh++++w5V\nVVXw8fHBpk2b8Mwzz8DGxkZo+5vf/AbvvfcetmzZ0mqgABp6FBp7JB60e/fuZvs8PT1x6tSph9Y5\nefLkDj35oKXP6Y+6slL2gyaOCcaN7GQAwE9JUQhUzoVYLOm28xMRERFR/9fhQGFvb4/CwkI4ODjg\n1VdfxZIlSzBq1KhW23t5eaG8vLxLRdLDaT2FqfvyBMa6++HQ6Z2oqC5DYVk+UjJ+gecQTv4iIiIi\novs6PIciODgYx44dw+3bt/HXv/71oWECAJ5++mnhCUrU87qzh8LQQIrHRk8TXp+5dKzbzk1ERERE\n+qHDgeLFF1/E2LFjIWpl8m9BQQF+/PHHLhdG7deT60Q0nZydnBaPvKLsHvssIiIiIup/Ohwopk6d\n2uyRrE2dOHECU6dObfV96lnd2UMBAHZWjhjt5gOg4fG0Pyb+p1vPT0RERET9W4cDRVu/Da+pqYFY\n3OWn0VKHNJ1D0b2BAgCmej8lbMcmn0TlXc6JISIiIqIG7ZqUXVpaiuLiYuH1nTt3cOvWrWbtioqK\n8K9//QtOTk7dVyG1SWsZih44/wiXR+BgMxg5d26hprYaPyUdx3QffX0ELxERERF1RLu6EjZu3Igh\nQ4ZgyJAhEIlECA0NFV43/Ro3bhy+//57vPzyyz1dNzXRk3MoAEAkEiFg7G+F1z8m/hfq+tYXKyQi\nIiKigaNdPRRBQUEwNTWFRqPBm2++iWeeeQbjxo3TaiMSiWBiYgKlUgkfH58eKZbaoQeGPAGA0uNx\nfHvuc1RUlaKoLB+J13/CuBGTeuSziIiIiKj/aFeg8PPzg5+fHwCgoqICc+fOxZgxY3q0MOqI+z0U\n3T0pu5GhgRSTxjyB789/BQA4HncI3sP9W33aFxERERENDB2ePf3+++8zTPQxmp6eRHHPlEd/A0MD\nKQAgM/8mUjIu9tyHEREREVG/0GYPxd69ewEAzz77LEQikfC6Lc8991zXKqMO6Nk5FI3M5JaY6BWE\n0wkNj46NOn8Qo1zHsZeCiIiIaABrM1AsXboUIpEITz/9NKRSKZYuXdrmSUUiEQOFjvTUkKdG08b9\nDmd+PQZ1fR1u5qTgRnYy3J08e/QziYiIiKjvajNQpKWlAQCkUqnWa+o7mj7lqacDhZXZIPiOCkDs\n5eMAgKjz/wf3OQwURERERANVm4HC1dX1oa+pL2i6sF3Pf9oMn7n4OfkkNJp6pN5KQIbqKlztR/T8\nBxMRERFRn8MlrfVA78yguM/OyhHjhvsLr//70796uQIiIiIi6iva7KFYtmxZh08qEomwa9euThVE\nXdPTQ54aPfHYQly8dlbopbiedZlzKYiIiIgGoDYDxcmTJzv8FB8+9ad3aa2U3Uvfe4W1M3w9Hsf5\nlB8AAP89tx+v/T6c/+2JiIiIBpg2A0V6enovlEFd03RSdu+Z+djTiL8SA3V9HW5kJyP1VgJGuXr3\nYgVEREREpGucQ6EHNL09ieIeGwsF/DxnCK//e24/6jX1uimGiIiIiHSCgULf9PKQo6Dx82EoaXik\n8K286/jl6ple/XwiIiIi0q02hzwNGTIEYrEYqampMDQ0xJAhQ9ocJy8SiXDjxo1uK5La0nvrUDzI\n0tQGj499EsfjDwEAvj37OcYMewxSA6NerYOIiIiIdKPNQPH4449DJBJBLBZrvaa+Q3thu94X6DsP\nscknUF5VgqKyfJy6+C2Cxs/XQSVERERE1NvaDBSfffbZQ18TyYxM8JsJz+CrHyIBANFxX2OC5wyY\nm1jpuDIiIiIi6mmcQ6FvdNR75OcVCAebwQCAu7XVOBLLxe6IiIiIBoJOBYq6ujp8+umnmDt3LsaN\nG4dx48Zh7ty5+PTTT1FXV9fdNVIbtIc86SZQSMQSzJ78vPD6p8snkJWfppNaiIiIiKj3dDhQ5OXl\nwdfXF8uXL8epU6cgEokgEolw6tQpLF++HEqlErm5uT1RK7VCg6YL2+mujlGu3hjlOg4AoNHU46sf\ndvAxskRERER6rsOB4tVXX0VycjJ27dqF/Px8xMfHIz4+Hvn5+di5cydSUlLw2muv9USt1A/MmfI8\nJOKGqTlpOamIvXxCxxURERERUU/qcKA4evQoXnvtNTz//POQSCTCfolEgmXLluGVV17B0aNHu7VI\naj9dDXlqZG/tguk+c4TX357Zg7LKEh1WREREREQ9qcOBwsjICK6urq2+P2TIEBgbG3epKOqYpnMo\ndDUpu6mg8b+HjYUCAFB5txzfnPlMtwURERERUY/pcKBYuHAhvvjiC9TW1jZ7r6amBl988QUWLlzY\noXNu374dQ4cOhUwmg1KpxJkzD19tOSkpCQEBAZDL5XBxccFHH33UrM3p06ehVCohk8ng7u6OHTt2\naL1/8OBB+Pr6wsrKCqampvD29sbevXs7VHffoftJ2U1JDYwwP+Al4fX5lB9wLfOSDisiIiIiop7S\n5joU58+f13o9f/58xMTEwNfXFyEhIXB3dwcAXLt2Tfih/fe//327Czhw4ABCQ0MRGRkJf39/bNu2\nDTNnzkRKSgqcnZ2btS8rK0NgYCACAgIQHx+PlJQULF26FKampggLCwMApKenY9asWVi+fDn279+P\nmJgYvPzyy7Czs8OcOQ3DcWxtbfHnP/8ZHh4eMDQ0xHfffYc//vGPsLOzwxNPPNHu+vsCrR6KPmK0\n2ziMHT4RCdfOAQAOnIjAm3/YyBW0iYiIiPSMSNPGT6NisbjZythajym9996D+9RqdbsKmDBhAsaO\nHYvIyEhh34gRIzB//nyEh4c3ax8REYHVq1cjLy8PUqkUABAeHo7IyEjcvn0bAPDWW2/h8OHDuHLl\ninDcCy+8gOTkZJw9e7bVWnx8fPDEE080+9ySkvtzACwsLNp1Xb3pclocdny7FgAw2nUcQmav0XFF\nDUrKC7H28xW4W1MFAJjq/RTmTFmm46p6TlxcHABAqVTquBLqy3ifUHvwPqG28B6h9uitn2Hb7KHY\nvXt3j314bW0t4uPjsWrVKq39QUFBOHfuXIvHxMbGYvLkyUKYAIDg4GCsWbMGGRkZcHV1RWxsLIKC\ngrSOCw4Oxt69e6FWq7Umkzc6ceIErl69iv/5n//phivToT4wh6KRhak15kx+Hl+e2A4AOPXLd3hk\n2GMY5uSp48qIiIiIqLu0GSiWLFnSYx9eUFAAtVoNhUKhtV+hUODEiZYfN6pSqeDi4tKsvUajgUql\ngqurK1QqFQIDA5u1qaurQ0FBgfB5paWlcHJywt27d2FgYIBt27Y1CyIPavyNQF+SWXhN2C4pKelT\nNRpqrOBoORTZxTehgQaf/mc9nhz7Agwl0rYP7qf60vef+i7eJ9QevE+oLbxH6GGGDx/eK5/TqZWy\n9YWZmRkSExMRFxeH8PBwhIWF4YcfftB1WR2mtbBdHyMSieDn/iQMJQ1zJ8qqi3Ax46SOqyIiIiKi\n7tJmD0VLqqurcejQIcTHx6OkpAT19dqrIYtEIuzatavN89ja2kIikTRbWTs3Nxf29vYtHmNvb99i\ne5FIJBzTWhsDAwPY2tpq1Tl06FAAwCOPPILk5GR8/PHHmDp1aqs198WxikY36/FDSsO2paVln6zR\n2EqD/dFbAQBXcuIw/bEn4eE6VsdVdS+OZ6X24H1C7cH7hNrCe4Tao+kcip7U4UBx+/ZtTJ06FTdv\n3oSlpSVKSkpgbW2NoqIi1NfXw9bWFqampu06l6GhIXx8fBAdHY158+YJ+6OjozF//vwWj/Hz88Pb\nb7+Nmpr+Fyo7AAAgAElEQVQaYR5FVFQUHB0dhfUx/Pz8cPjwYa3joqKioFQqW5w/0ai+vh53795t\nV+19VV94bGxLxo+ahoTrP+FyWsNfgPuiNuPNRRthbmKp48qIiIiIqCs6POTpzTffxJ07d3Du3Dlc\nvXoVGo0GBw4cQEVFBcLDwyGXy3H8+PF2n2/lypX47LPPsGvXLqSmpuL1119HTk4OQkJCAACrV6/G\njBkzhPaLFi2CXC7H0qVLcfnyZRw6dAjr1q3DG2+8IbQJCQlBVlYWwsLCkJqaip07d2Lv3r1ak78/\n/vhjnDhxAmlpaUhNTcXf//537Nu3D88++2xHvyU619cWtmuJSCTCM9NXwEzW8ISB0soi7IvejHpN\nfRtHEhEREVFf1uEeiuPHj+NPf/oTJkyYgMLCQgANP9AaGRlh9erVSElJQVhYGL755pt2nW/BggUo\nLCxEeHg4cnJy4OXlhaNHjwprUKhUKqSlpQntzc3NER0djRUrVggL061atQqhoaFCGzc3Nxw5cgRh\nYWGIjIyEo6Mjtm7ditmzZwttysvL8fLLLyMzMxMymQweHh74/PPPsWDBgo5+S/qAvjuHoilzEyss\nDg5FxOEPAACpGb/gh4vfYrrP7DaOJCIiIqK+qsOBory8HMOGDQMAGBndm2hbVia8P2nSJLz55psd\nOmdISIjQI/Gglh5b6+npiVOnTj30nJMnT37okw8+/vhjfPzxxx2qsz/om/0T941y9cZ0nzk4Ef9v\nAMB35z6Hu9NouNqP0HFlRERERNQZHR7y5OTkhOzsbACAiYkJrK2tkZCQILyfkZEBQ0PD7quQ2qQ9\n4qmvRwrgSb8/CAGivl6N3Uf+hvKqUh1XRURERESd0eFAMWXKFHz//ffC63nz5mH9+vVYu3YtPvzw\nQ2zevBnTp0/v1iKpLU2HPPX9QCGRGGDpE29AJpUDAArL8vHZ0fVQ17dvdXUiIiIi6js6HCjCwsIw\nd+5cVFdXAwDWrVsHPz8/rFmzBn/5y18wfvx4bN68udsLpdZpTcruJ2wsFHg2OEx4ffX2r/jPuc91\nWBERERERdUaH51CMGTMGY8aMEV5bWloiOjoaxcXFkEgkMDMz69YCqWP6fv/EfV5DffHEYwtx7OcD\nAIAT8YfhYueOcSMm6bgyIiIiImqvblsp29LSkmFCR7T6J/rBHIqmnnhsITyH3F+U51/RW3E776YO\nKyIiIiKijuhUoCguLsaaNWswbtw4WFtbw9raGuPGjcOaNWtQVFTU3TVSm+5Hir66sF1rxCIxng0O\nxSALBwBATd1d/OPbtSgqK9BxZURERETUHh0OFNevX8cjjzyCtWvXora2FlOnTsXUqVNRW1uLtWvX\n4pFHHsG1a9d6olZqRX+cQ9GU3MgUy3/7jjBJu6SiEP/4Lhx3a6p0XBkRERERtaXDgeKVV15BSUkJ\nTpw4gUuXLuHrr7/G119/jUuXLuH48eMoLS3Fa6+91hO1Unv0rw4KgYONC5bNegtisQQAkJWfhs+O\n/R31fPITERERUZ/W4UARExOD0NBQTJ06tdl706ZNw+uvv44ff/yxW4qjjutvQ56aGjn4USyYen+B\nw8tpcfi/U//s9z0wRERERPqsw4HC0tISVlZWrb5vZWUFS0vLLhVFHdP0B+7+sLDdw0z0CsQMn7nC\n67OXjuG/P/1LhxURERER0cN0OFAsX74cu3btQllZWbP3SkpKsGvXLrzwwgvdUhy1l379Bv9J/8Xw\nGTlFeB114f9w8uI3OqyIiIiIiFrT5joUX331ldZrDw8PiMVijBgxAkuWLIG7uzsA4Nq1a9i7dy8U\nCgVGjhzZM9VSO/TvHgqg4clPiwNfQ9XdCiSnxwMADsfshtzIFBM8uQo7ERERUV/SZqB4+umnIRKJ\nhGE1Tbc/+eSTZu1zc3OxePFiPPPMM91cKrVGa8iTDuvoThKJAZb95k1sP/wX3MxOAQB8cWIbjKUy\njB0+UcfVEREREVGjNgPFDz/80Bt1UBf054XtHkZqaIQXn3oXW7/+M7Ly06DR1OOzY3/HUoChgoiI\niKiPaDNQPP74471RB3WJfs2haEpuZIo//e59bDn4DvKKs1Ffr8ZnR9fjOc1KjBsxSdflEREREQ14\nbQaK1qjValy8eBHp6ekAADc3N/j4+EAs7tTi29RN+vNjY1tjbmKJV+etxdav32sIFZp67D22ARqN\nBj4jJ+u6PCIiIqIBrVM//R84cACDBw/GhAkTsHDhQixcuBATJkyAi4tLs0nc1PO01mnQvzwBALAw\ntcarv18LhZUzADSEiu834kLqaR1XRkRERDSwdThQfPPNN1i0aBEsLS2xZcsWHD9+HMePH8eWLVtg\nZWWFRYsW4dtvv+2JWqkd9LGHopGFiTVenbcW9tYuAACNph6ff78RpxP+o+PKiIiIiAauDg95Cg8P\nh4+PD3788UcYGxsL+6dNm4bly5dj0qRJWLt2LZ566qluLZRaN5BWkm4Y/vQR/vfQGuTcuQUA+Pr0\nTpRVFmOW3x/6/cJ+RERERP1Nh3sokpKSsHjxYq0w0cjIyAjPPvsskpKSuqU46riB8AO1mdwSr/0+\nHG4O99c7ibpwEF+e2A51vVqHlRERERENPB0OFHK5HAUFBa2+n5+fD7lc3qWiqKMGTg9FIxNjM7wy\n50OMdvMR9v10ORq7/rsOd2urdVgZERER0cDS4UAxffp0bN68GTExMc3eO3v2LLZs2YIZM2Z0S3HU\nPtpzsvW/h6KR1NAILzy5GuNHTRX2Jd08j83/9w6KyloPvURERETUfTocKD755BOYmJggICAAvr6+\nWLx4MRYvXgxfX19MmTIFpqamWLduXU/USq3QDMAeikYSiQEWBb6K6T6zhX2Z+Tfx9wOrcCv3ug4r\nIyIiIhoYOhwoXF1dkZiYiNdffx1lZWX4+uuv8fXXX6OsrAxhYWFISEiAq6trT9RK7TEA5lA8SCwS\n43eTluLp6S9DLJYAAEorirD54DtIuHZOx9URERER6bdOLWw3aNAgbNiwARs2bOjueqgzmox5Gnhx\n4r6JXkGwtbDHrv+uQ9XdCtTW1eDTI59ghnIeZvktguRe2CAiIiKi7tOhHorKykpIJBKEh4f3VD3U\nCVoDngZgD0VTI1wewRsLP8EgS0dh3/G4rxHx77+grLJYh5URERER6acOBQq5XI5BgwbBwsKip+qh\nThm4cyhaYmflhJUL12G06zhh39XMS/jkizeQlpOqw8qIiIiI9E+H51AsWLAAX331Ferr63uiHuqi\ngfSUp4cxMTbDi797DzMnPCN8T0rK72DzwXdx6pfvBtRigEREREQ9qcNzKObMmYOTJ09i0qRJeOGF\nFzB06FDIZLJm7caPH98tBVLbmv5wPMBHPGkRi8SY+dhCuCqGY+/3G1FZXYb6ejUO/bgLqRm/YFHg\nazA3sdR1mURERET9WqfWoUhOTkZsbCz++Mc/Ytq0afDz8xO+JkyYAD8/vw6dc/v27UIwUSqVOHPm\nzEPbJyUlISAgAHK5HC4uLvjoo4+atTl9+jSUSiVkMhnc3d2xY8cOrfd37tyJKVOmwNraGlZWVpg2\nbRrOnj3bobr7JiaKB412G4dVz6yHi90wYV9yxkX8df/ruJwWp8PKiIiIiPq/DvdQ7N69u1sLOHDg\nAEJDQxEZGQl/f39s27YNM2fOREpKCpydnZu1LysrQ2BgIAICAhAfH4+UlBQsXboUpqamCAsLAwCk\np6dj1qxZWL58Ofbv34+YmBi8/PLLsLOzw5w5cwA0BI6nn34a/v7+kMvl2LBhA4KDg5GYmIhhw4Y1\n+9y+jMN32mZjrkDo/L/ivz/tw8mL3wAAyqtKsOPbtZjy6G/w1KQlkBoY6bhKIiIiov6n3YGiuroa\n33zzDXJycmBjY4Mnn3wSDg4OXS5g48aNWLZsGZYtWwYA2LJlC44dO4aIiIgWnya1b98+VFVVYc+e\nPZBKpRg1ahRSUlKwYcMGIVBERETAyckJmzZtAgCMHDkSP//8M9avXy8Eis8//1zrvBERETh8+DCO\nHTuGFStWdPm6dIVzKFpnaGCI2ZOfh8dgb+yL3ozSiiIAwI+JR5B6KxGLZryKoY4eOq6SiIiIqH9p\n15Cn7OxseHl5YdGiRXjnnXfw0ksvYcSIETh16lSXPry2thbx8fEIDAzU2h8UFIRz51pekCw2NhaT\nJ0+GVCoV9gUHByM7OxsZGRlCm6CgIK3jgoODERcXB7Va3eJ57969i+rqalhZWXXlknRCa6VsTqJo\nk4frWLz9h83wGnp/nk9eURY2/99qHDq9CzW1d3VYHREREVH/0q4eivfeew/p6ekICwvDtGnTcP36\ndXz00Ud47bXX8Ouvv3b6wwsKCqBWq6FQKLT2KxQKnDhxosVjVCoVXFxcmrXXaDRQqVRwdXWFSqVq\nFlIUCgXq6upQUFDQ7PMar9HMzAxPPfXUQ2uOi+t7Y+4zcjKE7fz8/D5ZY1/kbR8IU5Et4tKOo66+\nBhpocCrhO8SnnoGf+5Owt+jciu/8/lN78D6h9uB9Qm3hPUIPM3z48F75nHYFiujoaCxZsgTr168X\n9ikUCixatAiZmZktznXoTzZv3ox//vOfOHHiBExNTXVdDvUSkUiEEfbj4Gg1DD9d/y9yim8CAMqq\nixCV9Dnc7cZinNs0GBvKdVwpERERUd/VrkChUqng7++vtW/SpEnQaDS4detWpwOFra0tJBIJcnNz\ntfbn5ubC3t6+xWPs7e1bbC8SiYRjWmtjYGAAW1tbrf2bNm3C+++/j2PHjsHHx6fNmpVKZZtteltl\nYh7ON/wsDDs7uz5ZY1/3+MRp+Dn5JP794y5U1VQCAK7nJSC79AZ+O3Ex/LwCIRY9fIRg42+J+P2n\nh+F9Qu3B+4TawnuE2qOkpKRXPqddcyjUajWMjY219jW+rq6u7vSHGxoawsfHB9HR0Vr7o6OjmwWY\nRn5+foiJiUFNTY2wLyoqCo6OjnB1dRXaPHjOqKgoKJVKSCQSYd+GDRvw/vvv48iRIx1+1G3f0mQd\nCk7K7hSRSIQJntPxzrP/izFN5lZUVpfhwMkIbDzwFm7lXtdhhURERER9U7uf8nTz5k2cP39eeN2Y\neFJTU1scJtTehe1WrlyJ5557Dr6+vvD390dERARycnIQEhICAFi9ejUuXLiA48ePAwAWLVqEDz/8\nEEuXLsW7776LK1euYN26dfjggw+Ec4aEhGDbtm0ICwvDSy+9hDNnzmDv3r348ssvhTZ/+9vf8N57\n72H//v1wd3cXejRkMhnMzc3b+23pE7iwXfexMLXGC799B0k3L+Dr0ztxp7ThvsjIvYa/f7kKE8cE\n4zcTnoaZnAviEREREQEdCBTvv/8+3n///Wb7X331Va3XGo0GIpGo1acpPWjBggUoLCxEeHg4cnJy\n4OXlhaNHjwrDqFQqFdLS0oT25ubmiI6OxooVK+Dr6wsrKyusWrUKoaGhQhs3NzccOXIEYWFhiIyM\nhKOjI7Zu3YrZs2cLbbZv3466ujosXLhQq54lS5bg008/bVftpL+8hvpixOBHcPzCIUTHfw21ug4a\naHD20jHEXTmNQOU8BHj/lmtXEBER0YDXrkDR3YvZPSgkJETokWjPZ3t6erb5yNrJkyc/9MkHTUOK\nfmEXRXeRGhjhN37PwHdUAA6e+idSMi4CAO7WVOE/5/bh7K/H8KT/YviMnNLm/AoiIiIifdWuQLFk\nyZKeroO6QHvIEwNFdxtk6YCQ3/0Zl9Pi8M2ZPcgtygQAFJUX4PPvN+HUL9/htxOfFXrniIiIiAaS\ndg95or5La2E76hEikQheQ30xym0cfkqKxpHYL1Be1TCP6HbeDWw//BfYmbtg7OAAAHziBhEREQ0c\nHKehD5gneo1ELMGkR57An5dEIMj39zCU3F+xPa/0NqKSPsf/HlqDtJxUHVZJRERE1HvYQ6Fn+NjY\n3iEzkuPJiYvhP+YJRF04iJ8uR6O+vuFBBFdv/4qrt3/FKNdxCPKdh2FOnjquloiIiKjnMFDoAa0h\nTxzD36uszGyxcFoIZvjMwb4j23Ez71fhv0dKxkWkZFzEUIdRmKGcC88hSs6xICIiIr3DQKEXmi5s\nR7pgY6GA//DfYoyzPzIrLiP+yo9CsLiZk4J/fBcORxtXzFDOhfeISZCIJW2ckYiIiKh/4BwKPaDh\nHIo+w1xmjeeeCMO7z/0vJnjOgER8P7Nn38nA3u834qM9f8LJi9+g8m65DislIiIi6h4MFHqGQ2r6\nBjsrJyya8QrWLI3EVO+nIDU0Ft4rLM3D4ZjdWLNrOf7vh38gtyhLh5USERERdQ2HPOmFpl0UDBR9\niZWZLeZMWYYg39/jx8QjOJ34X1RWlwEAamqrEfPrEcT8egSjXcdhytgn4eE6lovkERERUb/CQKEH\ntBe202Eh1CoTmTlmTnga033mIO7KaZxO+A9y7twS3k/OuIjkjIuwMVfAzysQj42eBgsTax1WTERE\nRNQ+DBR6gFMo+g+poREmegXBzzMQV2//itMJ/8HltDhhAved0lz859w+HPnpX/AaOh4TvYLgMfhR\niDmJm4iIiPooBgq9wy6K/kAkEmHk4EcxcvCjyC/OQcyvR3E++aQwUbteU49fb8Ti1xuxsDYbhAme\nM+A7KgA25godV05ERESkjYFCHzQd8sRA0e8MsnTA3CnL8OTEPyDxeizOJUXhRtZl4f3Csnwcif0C\nR2K/wDDH0fAdFYCxwydCbmSqw6qJiIiIGjBQ6AEubKcfpAZG8PV4HL4ejyO3MBPnkqJwPuUHVNyb\nxA0AN7KTcSM7GQdP/RNeQ33h6xGA0a7jIJHwf2UiIiLSDf4Uog80XNhO3yisnTFnyjI8OfFZ/Hoj\nFhdSfkDqrQTUa+oBAHXqWiRcO4eEa+dgIjOHt/tEjB0+EcOcPLloHhEREfUqBgqiPszQwBA+IyfD\nZ+RklFYUI/7qj7iQegqZeTeFNhVVpThz6RjOXDoGU5kFHh02AWOHT4S7sxfDBREREfU4Bgo9oLUK\nBYc86S1zE0tM9X4KU72fQs6d27iQegpxqadQXH5HaFNeVYKzSd/jbNL3MJVZ4JFhj2Gs+0QMd/bi\nsCgiIiLqEfwJQy9wYbuBxsHGBU/5P4snJ/4BN7NTkHDtLBKu/4TSiiKhTXlVCc4lReFcUhRkUjlG\nufnAa4gSo918IDfmhG4iIiLqHgwUeoAL2w1cYpEY7k6ecHfyxNzHlyMtOwW/XDuHhOvntMJFVU0l\nLl6NwcWrMRCLxBjqNBpjhoyH11BfDLJ00OEVEBERUX/HQEGkJ8QiMYY5eWKYkyfmPv5HpGWnIuH6\nOVy68TMKy/KFdvWaelzPTML1zCT8O+ZTKKydMdp1HDxcvTHMaTSkBkY6vAoiIiLqbxgo9ID2Stns\noqDGcDEaw5xGY+6UPyK7IANJaReQdPM8MnKvabXNLcxEbmEmfvjlWxhKpBjm7AmPwWMxytUb9tYu\nnJdDRERED8VAoQ+4sB09hEgkgtMgNzgNckPw+PkoqSjE5bR4JN08jyu3ElGrrhHa1qprkJrxC1Iz\nfsHhmN2wNLWBx+Cx8HD1xnDnMTCTW+jwSoiIiKgvYqDQA9oL2+muDuofLEysMdErEBO9AlFTexfX\nMi8h9VYCUjJ+QV5Rllbb4vI7iE0+gdjkEwAAB5vBcHfygruzF9ydPBkwiIiIiIGCaCCTGhrBc4gS\nnkOUAIDC0jykZPyC1FsJuHorEVU1lVrtc+7cQs6dW4j59QiAhoAx3NlLCBmmMvNevwYiIiLSLQYK\nfaDVQcEuCuo8a3M7+I8Jhv+YYKjr1chQXUPqrV9w5VYibuVeh7q+Tqt9Y8D4MfF+wBji4CF8DbJ0\n4BwMIiIiPcdAoQe0hzzxhzfqHhKxBEMdPTDU0QO/mfAM7tZWIz3nCq7de0JUeu5V1NertY5pDBjn\nkqIAACYycyFcDHUYCReFO58iRUREpGcYKPRC00nZRD3DyNAYIwc/ipGDHwUA3K2tRlp2Kq5nJeFa\nZhIycq81CxgVVaVIunkeSTfPAwAkYgM4DxqCIQ4ecLUfjsGK4bC1sGcvBhERUT/GQEFEnWJkaAwP\n17HwcB0LoCFgZKiuIS0nFWk5qUjPuYLKu+Vax6jr65CRe03r0bUyIxMMtnOHi8Idrgp3uNi5w8rM\nliGDiIion+gTgWL79u1Yv349cnJy4OnpiU2bNmHSpEmttk9KSsIrr7yC8+fPw8bGBi+++CL+/Oc/\na7U5ffo03njjDVy+fBlOTk5YtWoVXnrpJeH95ORkvP/++7h48SLS0tLwl7/8BWvWrOmxa+xJGq0R\nT/whjHTDyNAYI1zGYITLGAANC+jlFWUhLTv1Xsi4gtyizGbHVd2twJXbibhyO1HYZyazwGDFcLgo\nhsFVMRxOg4bAwsSa9zcREVEfpPNAceDAAYSGhiIyMhL+/v7Ytm0bZs6ciZSUFDg7OzdrX1ZWhsDA\nQAQEBCA+Ph4pKSlYunQpTE1NERYWBgBIT0/HrFmzsHz5cuzfvx8xMTF4+eWXYWdnhzlz5gAAKisr\nMWTIEMybNw/vvfder15z92u6tB1/4KK+QSwSw97aBfbWLvDzCgTQMAQqXXUVaTlXcCvvOm7lXkdl\ndVmzY8uqSnA5PQ6X0+OEfSYyczjbDrm3psYQONm6QWHlDIlE53+NERERDWg6/5d448aNWLZsGZYt\nWwYA2LJlC44dO4aIiAiEh4c3a79v3z5UVVVhz549kEqlGDVqFFJSUrBhwwYhUERERMDJyQmbNm0C\nAIwcORI///wz1q9fLwQKpVIJpbLhUZktfU5/otFwDgX1DyYyc63H1Go0GhSW5iEj9xpu511HRu51\n3M67gbs1Vc2OragqbdaTIZEYwMF6sBAwnAa5wcl2COTGpr12TURERAOdTgNFbW0t4uPjsWrVKq39\nQUFBOHfuXIvHxMbGYvLkyZBKpcK+4OBgrFmzBhkZGXB1dUVsbCyCgoK0jgsODsbevXuhVqshkUi6\n/2KIqMNEIhFsLBSwsVBg3IiGYY71mnrkF2XfCxk3cDv3BrLupLcYMtTqOmTm30Rm/k2t/eYmVnCw\nHgx7Gxc42AyGvfVg2Ns4Q27EoEFERNTddBooCgoKoFaroVAotPYrFAqcOHGixWNUKhVcXFyatddo\nNFCpVHB1dYVKpUJgYGCzNnV1dSgoKGj2ef1d0wFPfGws9XdikRgKa2corJ0xftRUAA0h405JLrIL\n0pGZn4asgnRk5aehqCy/xXOUVhShtKJIqzcDaFgl3N7G5V7YGAx7axc42LhAZmTS49dFRESkr3Q+\n5Km/iYuLa7tRL8vJyRG2s7KyECfqezUOJH3xHtEfhlAYjoDCYQTGOQB366pQVJGLooo8FFbkoqgi\nF8WV+ajXqFs8uqSiECUVhbhySztoyAxNYS6zhrnMBuYyG1jIbGAus4apkSXE4p7p0eR9Qu3B+4Ta\nwnuEHmb48OG98jk6DRS2traQSCTIzc3V2p+bmwt7e/sWj7G3t2+xvUgkEo5prY2BgQFsbW278Qr6\niqZzKNhDQQOHkYEM9hZusLdwE/bVa+pRXl2E4soCFFfmo7gyHyWV+SiputNq0KiqLUdVbTlyS29p\n7ReJxDAzshSChrnM+l7YsIGxoQmfOkVERAQdBwpDQ0P4+PggOjoa8+bNE/ZHR0dj/vz5LR7j5+eH\nt99+GzU1NcI8iqioKDg6OsLV1VVoc/jwYa3joqKioFQquzx/onEid1+SVX0Zl7Matp2cnftkjQNB\n42+J+P3vm9T1ahSUqKC6t5q3qvA2VHduI7c4C2p1XYvHaDT1KK0uRGl1IVB0Tes9Y6kcthb2sLFQ\nwNbCXviysVDAymwQJK30bPA+ofbgfUJt4T1C7VFSUtIrn6PzIU8rV67Ec889B19fX/j7+yMiIgI5\nOTkICQkBAKxevRoXLlzA8ePHAQCLFi3Chx9+iKVLl+Ldd9/FlStXsG7dOnzwwQfCOUNCQrBt2zaE\nhYXhpZdewpkzZ7B37158+eWXQpva2lokJydDo9GguroaKpUKiYmJMDU1xbBhw3r3m9BlfMoTUVsk\nYgkUVk5QWDnhUXc/Yb+6Xo3C0jzkFWUhrzgbeUXZyC/KQm5xNkrK77R6vuqayhYnhAOAWCyBtdmg\ne2HDAbb3QoeNhQK1dXdhaGDUI9dIRESkCzoPFAsWLEBhYSHCw8ORk5MDLy8vHD16VFiDQqVSIS0t\nTWhvbm6O6OhorFixAr6+vrCyssKqVasQGhoqtHFzc8ORI0cQFhaGyMhIODo6YuvWrZg9e7bQJjs7\nG97e3sKQhR07dmDHjh14/PHHcfLkyV66+u7Bhe2IOk8ilmCQpQMGWTrA84H37tZUIa84B/nF2cgt\nykJ+UTbyirKQW5zV4lOnGtXf6w0pKFHhChKbvS81kOHkVQdYmw+CldkgWJvZ3d82t4OJsRn/XyYi\non5DpGm6iAG1qGl3kYWFhQ4radnhmN04efEbAMDvJi3FdJ/ZbRxBPYHdzwOHRqNBWWUJ7pSq7gWH\nXNwpUQkhorSiqEvnlxoaw9psEKzNBsHK3K5h23wQrO4FD3N5z00Wp76Bf59QW3iPUHv01s+wOu+h\nICLqb0QiEcxNLGFuYokhDh7N3q+pvYs7pblCwLhzL3QUlKhQUKxqdXL4/eOrG+Z4FN5u8X2xSAxz\nEytYmtrC0tQGFqbWwrblvW1zE2sYGhh2y/USERE9DAOFHtBaKZujJIh0TmpoBAebwXCwGdzsvQsX\nLqCqphwuQx1QWJqPwrJ8FJXmNfxZlo/C0jzcra1+6PnrNfUoLr+D4ofM8QAAU5nFvZBhA4t7fzZ+\nmZtYw8LECjIjPq2KiIi6hoFCD2iPWeMPBkR9mUgkgtzIDEMcPFrs3dBoNKi8W47C0nwUleU1Cx2F\nZfmoqCpt12eVV5WgvKqkxYnjjQwkhjCXW8LcxLqh10VuBXOTe19Nts3klq0+uYqIiAY2Bgp9oOE6\nFNgYB+gAAB3PSURBVET6QiQSwcTYDCbGZnCxG9pim9q6GpRUFDb0UpQVaG0X39surSiCRlPf5ufV\nqWuFoPLQuiCCicz8XtCwvBc0rIVtU5kFzOQWMJVZwMTYlHM8iIgGEAYKIqJ+xtBAKqx70Rp1vRpl\nlcUPhI4CFJcXorisAKUVRSitLGpzeFUjDTRCj0d2G21FIjFMjM2EgNH4p6nMHGZyyyb7zGEqt4BM\nymFXRET9GQOFHtA0HfTEf5OJCA2Pw22cLwH7Ea22u1tThdLKYpRWFKKkoghllcUNf1YUoaSyCKX3\ntsurSrX/rnkIjaZeCB/tq9UApnILmMnuhwxTY3OYyMxgIvx570tmDhNjMxgaSNt1biIi6nkMFHqG\nQ56IqCOMpDIMksowyNLhoe3U6jqUV5WipKLwXu9GQwgprSwWAkdZVQnKK0tQebe8QzWo6+tQUn7n\noQsJPkhqYKQVMExkZpAbmwlBRG58P4SYyswhNzaDsVTGnhAioh7AQKEHtJ/yxH8siaj7SSQGsDC1\nhoWpdZtt1eo6lFeXoryyBGWVJULQKK+6v930z5p2DrtqqqbuLmrK76KovKD91yA2gNzYtOHLyBQy\nIxPIjUwhNzaBzMhUe9vYFHKj+9tSAyP+/UpE1AoGCiIi6lYSiQEsTKxhYdJ2+AAa1u0or2oIH41/\nVlSXoqKqDBXVTb6qSoXt+vqHr+XREnV9Hcoqi1FWWdzhYyVig4aAcS+MNG63FkqMpSaQGckhk8ph\nLJVDIuE/t0Skv/g3nF7gYudE1H9JDY1gbWgHa3O7drXXaDSorqlqFjoqq8tQfi90VFaX3X/v3r6a\nurudrlFdX4eyez0snWFoIIVMagLjxpBh1BA0GrZNmu3LLs6EVGL8/9u786iozvsN4M+9szCggAur\ngAbcooJapTEuP0XPcYtGY6rWpHUXtwgJaQyuDdQYjbGN2qBpEjUYc1watdYlJ+4LblUbNeKCxn1B\niwdEBGZ9f38MXOc6AyIBxuDzOWfOzH3vd+597/CK83A33M2+CUPRPJ1Gz70kRPRMYqCoBoTjOdn8\nz4aIqjlJkux//ffwKvVKV48zW0xFYSMPBcY85BsfIr8wD/nGPBQUPkS+8dHrAqN62mw1/aI+my0m\nmC0m5OZnP9X7tp5aprzWyFrXgUTvBQ+9Jww6T/uz3hMeqtcGeOiKXhfN413UiagiMVBUA+orrzBQ\nEBG5otPqH1356imZLSZ7wCh8aA8jxUHEMZQYHyLf+BAFhXkoMOWj0PgQhaYCFJjyy3RPkCex2iz2\nvS1lvLFhaTSytiiEGOwhwzGQOIQRvc6gBBTnoOIJD50H9DoDtBod/6BF9BxjoCAiInoCnVYPX23Z\nzwtxJISAyVyIAlM+Coz5KDQVBQ3jQxSW0HY3KxMmixGyFkWh5CGsVkuFbY/VZkF+0aFhFUGWZOh1\nBuh1HvDQGqDXG+zPRYHDQ2coevaAXmuAh94AvdbFPIfp4mXx/BOiZx//lVYHqjtlExHRs0SSJGUv\nQFn3jhw7dgwAEB0drbSZLSYlfNifHR8FMJoKUGguhNFUAKO5wKHN/uz42lYBe0wc2YRN6UtF02i0\nrsNJUSDR6zyKwokHdFo9dFr1tF7rAZ320evH2/U6D2h4Z3eiX4SBohpQnZLNXc5ERNWS/cuyHj41\nav2i5QghYLaaYDQVwlgUMApLCiHmAhSaCl0HE3MhTOZCGM2F5brqVllZrRbkW/Oe+v4mT0OWNfaw\nURw8dEVhQwkoeuh0jtPOAUYdWOy1Wq0eOo0OOq0eWo0eOq0OGlnLw8Oo2mGgqA5Ueyj4S4qIiEom\nSZLy5dkbvhWyTIvVDJPZqAoZJoux6LURJnNB0XNhUV2B/dlin1beUzzfUjxtrJDzT57EZrNW2h6W\nx0mSDJ1GZw8bDoFDp9FDq9VBp7G3O762BxJ13e3M29DIWsgZhU7zdVpdUYBRr0PmnhiqJAwURERE\n9ItoNTpoNTp4GWpW6HKFEEVhxR5MlNBhKXQIMEaYLUaYLCaYLEaYzUaYrSZ7ULEYYXZsL3pdPG0q\nqquK0PJom2xKHyrCgQubylwry5qicOGhCjX2n59W+Tk+emiVoOJ6vuP7SqjROixH82hdDDfVCwNF\nNeB4lSfuRiUioupCkiTlr+w1PCtnHUIIWG2WR2GjKKCop02PwsnjAcZisk87BhhzIcxWMywWE8xW\ns73GaoLFYobVVnEn1z8tm80Ko80KYznuTl/RZEl2HUzKEXA0shYajRZaWQtNUZ1G1irPGk3x68fm\nFS1DNS0z7JQHA0W1wBvbERERlYckScoXU0+PGpW+PqvNCktR2DBZTLAUBw6LCRarCWaLGWZr8fSj\nefZgYoSlaP6t2zdhtVngW8tHFVgcX5usJlWoqco9MU9iq+A9NRVJkuSicOIYVLSPtenU8xzDyxOD\njVYdhB4PNsXrkTWqZcqy5tF8WVNU82yck8NAUQ3wxnZERES/DvYviRp46Az4JfHF1ZXASiOEgM1m\nVYKJ2WIuCjD24GJ/WJymH712bCuhxuKwHKXG9TKepXDzOCFs9lD3C29oWVXkojHlGES0sj1sxA+Y\nWyV9YKAgIiIiquYkSVL+um7QV9LxY0/BcU+N69DxKIw8Cjmuw4nVVvRstcBisz9bbRaHNrN6nsPr\nR21mWIre8yyHHVdsNqs9LMJ9AYiBohrgnbKJiIjo18RxT82zxmazPgocReHkUfgwK23FgUV5bbMH\nHlfBxrndOQQ9WqcZVptVWa/VZoVVaXv02mKzVOolm58GA0V1wBvbEREREVUIWdZAL2sArYe7u/JE\nxYeyWWwWdQApel1VGCiqAd7YjoiIiOj543gomyv379+vkn7IVbIWIiIiIiKqlhgoqgPeKZuIiIiI\n3ISBohpQ39jOjR0hIiIioucOA0W1w0RBRERERFWHgYKIiIiIiMrtmQgUixcvRkREBDw9PREdHY20\ntLRS60+fPo2YmBh4eXkhLCwMs2bNcqrZu3cvoqOj4enpiUaNGuEf//iHU826devQokULGAwGREZG\n4l//+leFbVNVEo7nUPCYJyIiIiKqQm4PFGvWrME777yDGTNm4MSJE+jQoQN69+6NGzduuKx/8OAB\nunfvjuDgYBw/fhwLFy7EJ598gk8//VSpuXLlCvr06YNOnTrhxIkTmDJlCuLi4rBhwwal5tChQxgy\nZAiGDh2KkydP4s0338SgQYNw9OjRSt/miqa+sR0RERERUdVxe6D49NNPMWrUKIwaNQpNmzbFokWL\nEBwcjCVLlrisX7lyJQoKCpCamopmzZrh9ddfR2JiIv72t78pNUuWLEFISAgWLFiApk2bYsyYMRg+\nfDjmz5+v1CxcuBDdunXDlClT0LRpU0ybNg0xMTFYsGBBpW9zhXPIE9xDQURERERVya2Bwmw24/jx\n4+jevbuqvUePHjh48KDL9xw+fBj/93//B71er7T17NkTt27dwtWrV5WaHj16qN7Xs2dPHDt2DFar\n/Rblhw4dcllT0nqJiIiIiMiZW++UnZWVBavVisDAQFV7YGAgdu7c6fI9mZmZCAsLc6oXQiAzMxMN\nGjRAZmamU0gJDAyExWJBVlYWAgMDkZmZ6XK9mZmZpfb5H//+sKybV2Vu3L3kMMU9FERERERUddwa\nKH6NhnSJc3cXnqiqbrNOao0bNwbAz59Kx3FCZcFxQk/CMULPErce8uTn5weNRoM7d+6o2u/cuYOg\noCCX7wkKCnJZL0mS8p6SarRaLfz8/EqtKWm9RERERETkzK2BQqfToW3btti+fbuqffv27ejYsaPL\n97Rv3x779++HyWRS2rZt24Z69eqhQYMGSs3jy9y2bRuio6Oh0WhKrNm+fTs6dOjwi7eLiIiIiOh5\nIQnHmxi4wdq1azFs2DCkpKSgY8eOWLJkCZYvX44zZ84gNDQUU6dOxdGjR7Fjxw4AQG5uLl588UXE\nxMRg+vTpOH/+PEaOHInk5GS88847AOyXjY2KisKYMWMwbtw4pKWlYdKkSVi9ejVee+01APaTsrt0\n6YJZs2bhtddew/r165GUlIQDBw4gOjrabZ8HEREREdGvidsDBQB8/vnnmDdvHm7fvo3IyEgsWLBA\n2UMxcuRI7Nu3Dz///LNSn56ejrfeegv/+c9/ULt2bUyYMAEzZsxQLXP//v1ISEhAeno66tWrhylT\npiA2NlZVs379esyYMQOXLl1Cw4YN8dFHH6F///6Vv8FERERERNXEMxEoiIiIiIjo18ntN7b7NVi8\neDEiIiLg6emJ6OhopKWlubtLVEXmzJmDl156Cb6+vggICEC/fv2Qnp7uVJeUlISQkBB4eXmha9eu\nOHPmjGq+yWRCXFwc/P39UbNmTfTv3x83b96sqs2gKjRnzhzIsoz4+HhVO8cIZWZmYsSIEQgICICn\npyciIyOxf/9+VQ3HyfPNZrNh5syZyneOiIgIzJw5EzabTVXHcfJ82b9/P/r374/Q0FDIsowVK1Y4\n1VTEmMjJycHQoUNRq1Yt1KpVC8OGDSv7VcQElWr16tVCp9OJpUuXinPnzom4uDhRs2ZNcf36dXd3\njapAr169RGpqqkhPTxenT58WAwYMEEFBQSI7O1upmTt3rvDx8REbNmwQ6enpYvDgwaJevXoiLy9P\nqRk/frwICQkRO3fuFD/++KOIiYkRrVu3FjabzR2bRZXk0KFDIjw8XLRu3VrExcUp7RwjlJOTIyIi\nIsSIESPEsWPHxJUrV8SuXbvEuXPnlBqOE5o9e7aoW7eu2LJli7h69arYtGmTqFOnjvjwww+VGo6T\n58/WrVvF9OnTxbp160SNGjVEamqqan5FjYlevXqJyMhIceTIEXH48GHRokUL0a9fvzL1kYHiCdq1\nayfGjRunamvcuLGYNm2am3pE7pSXlyc0Go3YvHmz0hYcHCzmzJmjTBcUFAhvb2/xxRdfCCGEuH//\nvtDr9WLVqlVKzfXr14Usy2Lbtm1V13mqVDk5OaJhw4Ziz549IiYmRhUoOEZo6tSpolOnTqXWcJxQ\n3759xYgRI1Rtw4cPF6+++qoyzXHyfKtZs6ZToKiIMXHmzBkhSZI4dOiQUpOWliYkSRIZGRlP7BcP\neSqF2WzG8ePHne663aNHDxw8eNBNvSJ3ys3Nhc1mQ+3atQEAly9fdrozu8FgQOfOnZUxcuzYMVgs\nFlVNaGgomjVrxnFUjYwdOxaDBw9Gly5dVO0cIwQAGzduRLt27TBkyBAEBgbiN7/5DVJSUpT5HCcE\nAJ06dcLu3btx/vx5AMCZM2ewa9cu9OnTBwDHCTmrqDFx+PBheHt74+WXX1ZqOnbsiBo1apRp3PBO\n2aXIysqC1WpFYGCgqj0wMBA7d+50U6/Ind5++220adMG7du3B2A/JlqSJJdj5NatWwDsN0zUaDSo\nW7euU01mZmbVdJwq1ZdffolLly5h1apVTvM4RggALl26hMWLFyMhIQFTp07FiRMnMGnSJEiShIkT\nJ3KcEAAgMTERDx48QPPmzaHRaGC1WjF9+nSMGzcOAH+fkLOKGhOZmZnw9/d3Wn5AQECZxg0DBVEZ\nvfvuuzh48CAOHDgASZLc3R16RmRkZGD69Ok4cOAAZJk7fck1m82Gl156CbNnzwYAtGrVChkZGUhJ\nScHEiRPd3Dt6VqxevRrffPMNVq9ejebNm+PEiROIj49HeHg4Ro4c6e7uEZWI//uVws/PDxqNBnfu\n3FG137lzB0FBQW7qFblDQkIC1qxZg927dyt3ZAeAoKAgCCFKHSNBQUGwWq24d+9eiTX063Xo0CHc\nu3cPzZs3h06ng06nw969e5GSkgK9Xo+6detyjBCCg4PRrFkzVVuzZs1w7do1APxdQnbvv/8+Jk+e\njEGDBqFFixb4wx/+gHfffRdz5swBwHFCzipqTAQFBeF///uf0/Lv3r1bpnHDQFEKnU6Htm3bYvv2\n7ar27du3Kzfeo+rv7bffVsJE48aNVfPCw8MRFBSkGiOFhYXYv3+/Mkbatm0LrVarqrlx4wbOnj3L\ncVQNDBgwAD/99BNOnjypPKKjo/HGG2/g5MmTaNKkCccIoWPHjspx8cXOnz+v/IGCv0sIAPLz8532\ndMqyrFw2luOEHldRY6J9+/bIy8vD4cOHlZqDBw8iPz8fHTp0eHJHynuW+fNizZo1wsPDQ3z11Vfi\n7NmzIj4+Xnh7e4tr1665u2tUBSZOnCh8fHzE7t27RWZmpvJwvBTbxx9/LGrVqiXWr18vfvrpJ/H7\n3/9ehISEqGomTJggwsLCxI4dO8R///tf0bVrV9GmTRtewq+aevwqTxwjdPToUaHX68Xs2bPFxYsX\nxdq1a4Wvr69YsmSJUsNxQiNGjBBhYWFiy5Yt4sqVK2L9+vXC399fTJ48WanhOHn+5OXliRMnTogf\nf/xReHl5iVmzZokTJ04o30Urakz07t1btGzZUhw6dEgcPHhQREVFif79+5epjwwUZbBkyRIRHh4u\nDAaDiI6OFmlpae7uElURSZKELMtOj+TkZFVdcnKyqFevnvD09BQxMTEiPT1dNd9kMon4+Hjh5+cn\natSoIfr37y9u3LhRlZtCVahr166qQCEExwjZryXfqlUr4enpKZo2bSo+++wzpxqOk+dbXl6eSEhI\nEC+88ILw8vISDRs2FDNmzBBGo1FVx3HyfNmzZ4/L7yMjR45UaipiTOTk5IihQ4cKX19f4evrK4YN\nGybu379fpj5KQghRQXtdiIiIiIjoOcNzKIiIiIiIqNwYKIiIiIiIqNwYKIiIiIiIqNwYKIiIiIiI\nqNwYKIiIiIiIqNwYKIiIiIiIqNwYKIiIiIiIqNwYKIiICACwd+9eyLKMtWvXursrZbZw4UI0atQI\nWq0Wbdq0cVs/RowYgfDwcLetn4jInRgoiIiqUGpqKmRZhsFgwI0bN5zm9+3bFxEREW7omZ0kSW5b\n99M6cOAAEhIS8PLLL2P58uX46KOPSqz9/vvvkZycXGl9kSQJssz/Uono+cTffkREbmA2m11+AXb3\nF3ohhFvX/zT27NkDSZLw+eefY+jQoejVq1eJtVu3bsVf/vKXSuvLV199hXPnzlXa8omInmUMFERE\nbtC6dWssX77c5V6K6i4/P79ClnPnzh0AQM2aNZ9Y+7RBqbCw8KnqNRoNdDrdU72HiKi6YKAgIqpi\nkiRh6tSpEEJg9uzZpdZevXoVsixjxYoVTvNkWVb91T0pKQmyLOPcuXP44x//iFq1asHf3x/Tp08H\nANy8eRMDBgyAr68vgoKCMH/+fJd9s1qt+OCDDxASEoIaNWqgV69euHjxolPthQsXMHjwYPj5+cHT\n0xNt2rTBunXrVDXFh3jt3r0b8fHxCAoKgre3d6nbbLPZMHv2bDRu3BgGgwENGjRAYmKi6ku+LMtI\nSUlRXms0GpefEQCMHDkSixcvVmqL669du6a0TZw4EWvXrkXLli3h4eGhnEfy9ddfo3v37ggODobB\nYECTJk0wd+5cp4Di6hyK4uVu3LgRUVFRMBgMiIyMxA8//KCqe/jwId577z1ERETAYDAgICAAXbt2\nRVpaWqmfExHRs0Lr7g4QET2P6tevj1GjRmHZsmWYPn06QkNDf/Eyiw+XeuONN9CsWTN8/PHH2Lp1\nK+bOnYs6depg2bJl6NKlC+bNm4dvv/0WiYmJiI6ORkxMjLIMIQTmzp0Lm82GyZMnIzs7GwsWLEC3\nbt1w6tQp1KpVCwBw9uxZdOzYEcHBwUhMTETNmjWxYcMGDBo0CCtXrsSbb76p6ltcXBzq1KmDGTNm\n4P79+6Vux9ixY7Fs2TIMHDgQf/rTn3Ds2DF88sknSE9Px+bNmwEAK1euRGpqKnbs2IFvv/0WQgh0\n6NDB5fLGjx+PW7duqWoBwN/fX6nZu3cvvvvuO0yaNAlBQUF48cUXAQCLFy9G8+bN0adPHxgMBuzc\nuRPTpk1Dbm6u6pA1SZJcHq528OBBbNq0CRMmTIC3tzcWLVqEgQMH4tq1a6hdu7bSv+J1N2/eHNnZ\n2Thy5AhOnjyJTp06lfpZERE9EwQREVWZr7/+WsiyLI4cOSKuX78uPDw8xPjx45X5ffv2FeHh4cr0\nlStXhCRJIjU11WlZkiSJ5ORkZTopKUlIkiRiY2OVNqvVKsLCwoRGoxFz5sxR2nNycoSXl5cYOnSo\n0rZnzx4hSZIICgoSubm5SvuuXbuEJEli5syZSlv37t1FVFSUMBqNqj716NFDhIWFqbZXkiTRvn17\nYbVan/j5nDp1SkiSJEaPHq1qT0pKErIsiy1btihtkyZNErIsP3GZT6qVJEloNBpx6tQpp3kFBQVO\nbWPHjhXe3t7CZDIpbSNGjFD93IqX6+HhIS5duuS0fSkpKUpb7dq1RVxcXJm2g4joWcRDnoiI3CQ0\nNBSjRo3C8uXLcf369QpZpiRJGD16tDItyzKio6MhhMCoUaOUdl9fXzRt2hSXLl1yWsbw4cNVhyV1\n7doVLVq0UPYOZGdnY+fOnRg0aBAePHiAe/fuKY+ePXvi5s2buHDhgmqZsbGxZboK0pYtWyBJEhIS\nElTtCQkJkGUZW7ZsKdsH8ZQ6dOiAqKgop3aDwQDAfhhWTk4O7t27h86dO+Phw4dlOgm7W7duqkOh\noqKi4OPjo/rcfX19ceTIEdy6dasCtoSIqOoxUBARudG0adMA4InnUjyN+vXrq6Z9fX2h0+kQEBDg\n1J6dne30/kaNGjm1NWnSBFeuXAEAXLx4EUIIJCUlwd/fX/V47733AAB3795V3itJUpkvhXv16lVI\nkoQmTZqo2n18fBAcHKz0oaI1bNjQZXtaWho6d+4MLy8v1KlTB/7+/hg6dCgAPPHQLQAICwtzaqtd\nu7bqcy8+nKt+/fr47W9/i5kzZyIjI6OcW0JEVPV4DgURkRuFhoZi9OjRWLp0qRIuHJV0GVmbzVbi\nMjUajVNbSXsHRDkuE1u87oSEBLzyyisuayIjI1XTnp6eT72equSqf5cvX0b37t3RtGlTLFy4EGFh\nYTAYDDh+/DimTJlS6s+gmKufBaD+3AcOHIjOnTtj48aN2LZtG/7+979j3rx5SE1NxZAhQ8q/UURE\nVYSBgojIzaZNm4alS5fiww8/dJpXfOJuTk6Oqv3q1auV1p/HD1cCgIyMDLzwwgsAoOxt0Gq16Nat\nW4Wuu0GDBhBCICMjAy1atFDaHzx4gNu3b+PVV18t13LLc3+Pf//73zCZTNi8ebPqpPmff/65XH0o\nTUBAAGJjYxEbG4vc3Fy0a9cOH3zwAQMFEf0q8JAnIiI3CwkJwZgxY5CamuoUFLy9veHn54d9+/ap\n2lNSUirtJngrVqxQHc6za9cupKeno2/fvgDsV0fq2rUrvvzyS5fH/WdlZZV73X369IEQAgsWLFC1\nL1iwADabrdyBokaNGgDKdphSseK9C457IoxGIz777LNy9cEVm82G3NxcVZuPjw/Cw8OdQiQR0bOK\neyiIiKqYq8OMpk6diqVLl+L06dPKnoBiY8aMwdy5cxEbG4vo6Gjs27cPFy5cqLS7WgcGBqJjx44Y\nPXo0srOzsXDhQoSEhKhOlF6yZAk6deqEli1bIjY2Fg0bNsTdu3dx5MgRnD17VnUOwNP0MyoqCqNH\nj8ayZcuQk5ODbt264fjx41i+fDl69+5d6t2wS1N8Yvpbb72F3r17Q6vVol+/fqUeitWzZ0/odDr0\n6dMH48aNQ2FhIVauXAmttuL+63zw4AFCQkLwu9/9Dq1atYKPjw/S0tLwww8/IC4ursLWQ0RUmRgo\niIiqmKs9C8V7KRYvXuw0/89//jOysrLw3Xff4Z///CdeeeUVfP/99wgICCjzXoqS6h5vlyQJiYmJ\nOH/+PObPn4+cnBx07twZixYtUg6/AuwnaR87dgzJycn45ptvkJWVBX9/f7Rq1crp0K2n3ZPyxRdf\nICIiAsuWLcOmTZsQGBiI999/H0lJSWXerse9/vrrSEhIwKpVq7B69WoIIXD58mXUr1+/xHtING7c\nGBs3bsS0adOQmJgIPz8/DB8+HF26dEGPHj2e2JeSluvY7uXlhUmTJmH79u3YtGkTzGYzwsPD8de/\n/hXx8fFl2jYiIneTRGX9iYuIiIiIiKo9nkNBRERERETlxkBBRERERETlxkBBRERERETlxkBBRERE\nRETlxkBBRERERETlxkBBRERERETlxkBBRERERETlxkBBRERERETlxkBBRERERETlxkBBRERERETl\n9v9cnNVbKxM1TwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from train import Train\n", + "import matplotlib.pyplot as plt\n", + "\n", + "suite = Train(hypos)\n", + "suite.Update(60)\n", + "\n", + "hs, ps = [], []\n", + "for hypo, prob in sorted(suite.Items()):\n", + " hs.append(hypo)\n", + " ps.append(prob)\n", + " \n", + "plt.plot(hs, ps)\n", + "plt.xlabel('Number of trains')\n", + "plt.ylabel('Probability');" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are too many hypotheses to print, so I plotted the results in\n", + "Figure [fig.train1]. Not surprisingly, all values of $N$ below 60 have\n", + "been eliminated.\n", + "\n", + "The most likely value, if you had to guess, is 60. That might not seem\n", + "like a very good guess; after all, what are the chances that you just\n", + "happened to see the train with the highest number? Nevertheless, if you\n", + "want to maximize the chance of getting the answer exactly right, you\n", + "should guess 60.\n", + "\n", + "But maybe that’s not the right goal. An alternative is to compute the\n", + "mean of the posterior distribution:" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "333.41989326371095\n" + ] + } + ], + "source": [ + "def Mean(suite):\n", + " total = 0\n", + " for hypo, prob in suite.Items():\n", + " total += hypo * prob\n", + " return total\n", + "\n", + "print(Mean(suite))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Or you could use the very similar method provided by `Pmf`:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "333.41989326371095\n" + ] + } + ], + "source": [ + "print(suite.Mean())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The mean of the posterior is 333, so that might be a good guess if you\n", + "wanted to minimize error. If you played this guessing game over and\n", + "over, using the mean of the posterior as your estimate would minimize\n", + "the mean squared error over the long run (see\n", + ").\n", + "\n", + "You can download this example from . For\n", + "more information see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## What about that prior?\n", + "\n", + "To make any progress on the locomotive problem we had to make\n", + "assumptions, and some of them were pretty arbitrary. In particular, we\n", + "chose a uniform prior from 1 to 1000, without much justification for\n", + "choosing 1000, or for choosing a uniform distribution.\n", + "\n", + "It is not crazy to believe that a railroad company might operate 1000\n", + "locomotives, but a reasonable person might guess more or fewer. So we\n", + "might wonder whether the posterior distribution is sensitive to these\n", + "assumptions. With so little data—only one observation—it probably is.\n", + "\n", + "Recall that with a uniform prior from 1 to 1000, the mean of the\n", + "posterior is 333. With an upper bound of 500, we get a posterior mean of\n", + "207, and with an upper bound of 2000, the posterior mean is 552.\n", + "\n", + "So that’s bad. There are two ways to proceed:\n", + "\n", + "- Get more data.\n", + "\n", + "- Get more background information.\n", + "\n", + "With more data, posterior distributions based on different priors tend\n", + "to converge. For example, suppose that in addition to train 60 we also\n", + "see trains 30 and 90. We can update the distribution like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "for data in [60, 30, 90]:\n", + " suite.Update(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With these data, the means of the posteriors are\n", + "\n", + " ------- -----------\n", + " Upper Posterior\n", + " Bound Mean\n", + " 500 152\n", + " 1000 164\n", + " 2000 171\n", + " ------- -----------\n", + "\n", + "So the differences are smaller." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## An alternative prior\n", + "\n", + "![Posterior distribution based on a power law prior, compared to a\n", + "uniform prior.](figs/train4.pdf)\n", + "\n", + "[fig.train4]\n", + "\n", + "If more data are not available, another option is to improve the priors\n", + "by gathering more background information. It is probably not reasonable\n", + "to assume that a train-operating company with 1000 locomotives is just\n", + "as likely as a company with only 1.\n", + "\n", + "With some effort, we could probably find a list of companies that\n", + "operate locomotives in the area of observation. Or we could interview an\n", + "expert in rail shipping to gather information about the typical size of\n", + "companies.\n", + "\n", + "But even without getting into the specifics of railroad economics, we\n", + "can make some educated guesses. In most fields, there are many small\n", + "companies, fewer medium-sized companies, and only one or two very large\n", + "companies. In fact, the distribution of company sizes tends to follow a\n", + "power law, as Robert Axtell reports in *Science* (see\n", + ").\n", + "\n", + "This law suggests that if there are 1000 companies with fewer than 10\n", + "locomotives, there might be 100 companies with 100 locomotives, 10\n", + "companies with 1000, and possibly one company with 10,000 locomotives.\n", + "\n", + "Mathematically, a power law means that the number of companies with a\n", + "given size is inversely proportional to size, or\n", + "\n", + "$${\\mathrm{PMF}}(x) \\propto \\left( \\frac{1}{x} \\right)^{\\alpha}$$ \n", + "\n", + "where\n", + "${\\mathrm{PMF}}(x)$ is the probability mass function of $x$ and $\\alpha$\n", + "is a parameter that is often near 1.\n", + "\n", + "We can construct a power law prior like this:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Train(Dice):\n", + "\n", + " def __init__(self, hypos, alpha=1.0):\n", + " Pmf.__init__(self)\n", + " for hypo in hypos:\n", + " self.Set(hypo, hypo**(-alpha))\n", + " self.Normalize()\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And here’s the code that constructs the prior:" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from train3 import Train\n", + "\n", + "hypos = range(1, 1001)\n", + "suite = Train(hypos)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, the upper bound is arbitrary, but with a power law prior, the\n", + "posterior is less sensitive to this choice.\n", + "\n", + "Figure [fig.train4] shows the new posterior based on the power law,\n", + "compared to the posterior based on the uniform prior. Using the\n", + "background information represented in the power law prior, we can all\n", + "but eliminate values of $N$ greater than 700.\n", + "\n", + "If we start with this prior and observe trains 30, 60, and 90, the means\n", + "of the posteriors are\n", + "\n", + " ------- -----------\n", + " Upper Posterior\n", + " Bound Mean\n", + " 500 131\n", + " 1000 133\n", + " 2000 134\n", + " ------- -----------\n", + "\n", + "Now the differences are much smaller. In fact, with an arbitrarily large\n", + "upper bound, the mean converges on 134.\n", + "\n", + "So the power law prior is more realistic, because it is based on general\n", + "information about the size of companies, and it behaves better in\n", + "practice.\n", + "\n", + "You can download the examples in this section from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Credible intervals\n", + "\n", + "Once you have computed a posterior distribution, it is often useful to\n", + "summarize the results with a single point estimate or an interval. For\n", + "point estimates it is common to use the mean, median, or the value with\n", + "maximum likelihood.\n", + "\n", + "For intervals we usually report two values computed so that there is a\n", + "90% chance that the unknown value falls between them (or any other\n", + "probability). These values define a **credible interval**.\n", + "\n", + "A simple way to compute a credible interval is to add up the\n", + "probabilities in the posterior distribution and record the values that\n", + "correspond to probabilities 5% and 95%. In other words, the 5th and 95th\n", + "percentiles.\n", + "\n", + "`thinkbayes` provides a function that computes percentiles:\n", + "\n", + "```python\n", + "def Percentile(pmf, percentage):\n", + " p = percentage / 100.0\n", + " total = 0\n", + " for val, prob in pmf.Items():\n", + " total += prob\n", + " if total >= p:\n", + " return val \n", + "```\n", + "\n", + "And here’s the code that uses it:" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(50, 950)\n" + ] + } + ], + "source": [ + "from thinkbayes import Percentile\n", + "\n", + "interval = Percentile(suite, 5), Percentile(suite, 95)\n", + "print(interval)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the previous example—the locomotive problem with a power law prior\n", + "and three trains—the 90% credible interval is $(91, 243)$. The width of\n", + "this range suggests, correctly, that we are still quite uncertain about\n", + "how many locomotives there are." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cumulative distribution functions\n", + "\n", + "In the previous section we computed percentiles by iterating through the\n", + "values and probabilities in a Pmf. If we need to compute more than a few\n", + "percentiles, it is more efficient to use a cumulative distribution\n", + "function, or Cdf.\n", + "\n", + "Cdfs and Pmfs are equivalent in the sense that they contain the same\n", + "information about the distribution, and you can always convert from one\n", + "to the other. The advantage of the Cdf is that you can compute\n", + "percentiles more efficiently.\n", + "\n", + "`thinkbayes` provides a `Cdf` class that\n", + "represents a cumulative distribution function. `Pmf` provides\n", + "a method that makes the corresponding Cdf:" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "cdf = suite.MakeCdf()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And `Cdf` provides a function named `Percentile`" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(50, 950)\n" + ] + } + ], + "source": [ + "interval = cdf.Percentile(5), cdf.Percentile(95)\n", + "print(interval)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Converting from a Pmf to a Cdf takes time proportional to the number of\n", + "values, `len(pmf)`. The Cdf stores the values and\n", + "probabilities in sorted lists, so looking up a probability to get the\n", + "corresponding value takes “log time”: that is, time proportional to the\n", + "logarithm of the number of values. Looking up a value to get the\n", + "corresponding probability is also logarithmic, so Cdfs are efficient for\n", + "many calculations.\n", + "\n", + "The examples in this section are in .\n", + "For more information see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The German tank problem\n", + "\n", + "During World War II, the Economic Warfare Division of the American\n", + "Embassy in London used statistical analysis to estimate German\n", + "production of tanks and other equipment.[^2]\n", + "\n", + "The Western Allies had captured log books, inventories, and repair\n", + "records that included chassis and engine serial numbers for individual\n", + "tanks.\n", + "\n", + "Analysis of these records indicated that serial numbers were allocated\n", + "by manufacturer and tank type in blocks of 100 numbers, that numbers in\n", + "each block were used sequentially, and that not all numbers in each\n", + "block were used. So the problem of estimating German tank production\n", + "could be reduced, within each block of 100 numbers, to a form of the\n", + "locomotive problem.\n", + "\n", + "Based on this insight, American and British analysts produced estimates\n", + "substantially lower than estimates from other forms of intelligence. And\n", + "after the war, records indicated that they were substantially more\n", + "accurate.\n", + "\n", + "They performed similar analyses for tires, trucks, rockets, and other\n", + "equipment, yielding accurate and actionable economic intelligence.\n", + "\n", + "The German tank problem is historically interesting; it is also a nice\n", + "example of real-world application of statistical estimation. So far many\n", + "of the examples in this book have been toy problems, but it will not be\n", + "long before we start solving real problems. I think it is an advantage\n", + "of Bayesian analysis, especially with the computational approach we are\n", + "taking, that it provides such a short path from a basic introduction to\n", + "the research frontier." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "Among Bayesians, there are two approaches to choosing prior\n", + "distributions. Some recommend choosing the prior that best represents\n", + "background information about the problem; in that case the prior is said\n", + "to be **informative**. The problem with using an\n", + "informative prior is that people might use different background\n", + "information (or interpret it differently). So informative priors often\n", + "seem subjective.\n", + "\n", + "The alternative is a so-called **uninformative prior**,\n", + "which is intended to be as unrestricted as possible, in order to let the\n", + "data speak for themselves. In some cases you can identify a unique prior\n", + "that has some desirable property, like representing minimal prior\n", + "information about the estimated quantity.\n", + "\n", + "Uninformative priors are appealing because they seem more objective. But\n", + "I am generally in favor of using informative priors. Why? First,\n", + "Bayesian analysis is always based on modeling decisions. Choosing the\n", + "prior is one of those decisions, but it is not the only one, and it\n", + "might not even be the most subjective. So even if an uninformative prior\n", + "is more objective, the entire analysis is still subjective.\n", + "\n", + "Also, for most practical problems, you are likely to be in one of two\n", + "regimes: either you have a lot of data or not very much. If you have a\n", + "lot of data, the choice of the prior doesn’t matter very much;\n", + "informative and uninformative priors yield almost the same results.\n", + "We’ll see an example like this in the next chapter.\n", + "\n", + "But if, as in the locomotive problem, you don’t have much data, using\n", + "relevant background information (like the power law distribution) makes\n", + "a big difference.\n", + "\n", + "And if, as in the German tank problem, you have to make life-and-death\n", + "decisions based on your results, you should probably use all of the\n", + "information at your disposal, rather than maintaining the illusion of\n", + "objectivity by pretending to know less than you do." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "To write a likelihood function for the locomotive problem, we had to\n", + "answer this question: “If the railroad has $N$ locomotives, what is the\n", + "probability that we see number 60?”\n", + "\n", + "The answer depends on what sampling process we use when we observe the\n", + "locomotive. In this chapter, I resolved the ambiguity by specifying that\n", + "there is only one train-operating company (or only one that we care\n", + "about).\n", + "\n", + "But suppose instead that there are many companies with different numbers\n", + "of trains. And suppose that you are equally likely to see any train\n", + "operated by any company. In that case, the likelihood function is\n", + "different because you are more likely to see a train operated by a large\n", + "company.\n", + "\n", + "As an exercise, implement the likelihood function for this variation of\n", + "the locomotive problem, and compare the results." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/04_More_Estimation.ipynb b/notebooks/04_More_Estimation.ipynb index 520092f..a91a839 100644 --- a/notebooks/04_More_Estimation.ipynb +++ b/notebooks/04_More_Estimation.ipynb @@ -1,830 +1,1070 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "%matplotlib inline\n", - "import sys\n", - "from __future__ import division, print_function\n", - "%load_ext autoreload\n", - "%autoreload 2 \n", - "import sys\n", - "sys.path.insert(0,'../code')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "More Estimation {#more}\n", - "===============\n", - "\n", - "The Euro problem {#euro}\n", - "----------------\n", - "\n", - "In *Information Theory, Inference, and Learning\n", - "Algorithms*, David MacKay poses this problem:\n", - "\n", - "> A statistical statement appeared in \\`\\`The Guardian\" on Friday\n", - "> January 4, 2002:\n", - ">\n", - "> > When spun on edge 250 times, a Belgian one-euro coin came up heads\n", - "> > 140 times and tails 110. ‘It looks very suspicious to me,’ said\n", - "> > Barry Blight, a statistics lecturer at the London School of\n", - "> > Economics. ‘If the coin were unbiased, the chance of getting a\n", - "> > result as extreme as that would be less than 7%.’\n", - ">\n", - "> But do these data give evidence that the coin is biased rather than\n", - "> fair?\n", - "\n", - "To answer that question, we’ll proceed in two steps. The first is to\n", - "estimate the probability that the coin lands face up. The second is to\n", - "evaluate whether the data support the hypothesis that the coin is\n", - "biased.\n", - "\n", - "You can download the code in this section from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Any given coin has some probability, $x$, of landing heads up when spun\n", - "on edge. It seems reasonable to believe that the value of $x$ depends on\n", - "some physical characteristics of the coin, primarily the distribution of\n", - "weight.\n", - "\n", - "If a coin is perfectly balanced, we expect $x$ to be close to 50%, but\n", - "for a lopsided coin, $x$ might be substantially different. We can use\n", - "Bayes’s theorem and the observed data to estimate $x$.\n", - "\n", - "Let’s define 101 hypotheses, where $H_x$ is the hypothesis that the\n", - "probability of heads is $x$%, for values from 0 to 100. I’ll start with\n", - "a uniform prior where the probability of $H_x$ is the same for all $x$.\n", - "We’ll come back later to consider other priors.\n", - "\n", - "[Posterior distribution for the Euro problem on a uniform\n", - "prior.](figs/euro1.pdf)\n", - "\n", - "[fig.euro1]\n", - "\n", - "The likelihood function is relatively easy: If $H_x$ is true, the\n", - "probability of heads is $x/100$ and the probability of tails is $1-\n", - "x/100$." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "from thinkbayes import Suite\n", - "class Euro(Suite):\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " x = hypo\n", - " if data == 'H':\n", - " return x/100.0\n", - " else:\n", - " return 1 - x/100.0" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Here’s the code that makes the suite and updates it:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "suite = Euro(range(0, 101))\n", - "dataset = 'H' * 140 + 'T' * 110\n", - "\n", - "for data in dataset:\n", - " suite.Update(data)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The result may be plotted with:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHCFJREFUeJzt3XmQXPV57vHvw2hjl9i0R8JGFBIGGwoLXS/QuUXuHcsY\n3SpCQI5NjBND5Vo2cSgugbp1PforccouMKWA5SBTBDuRgSQuQVAwWztO4gjJLMFIwhK2QAItICEW\nBaztvX+cHtRqZno9p0/39POpmqL7bP3OKebpn96zKSIwM7PecUTeBZiZWXs5+M3MeoyD38ysxzj4\nzcx6jIPfzKzHOPjNzHpMzeCX1C9pvaQNkm4YYv4Zkn4m6V1J1w0xv0/SU5LuT6toMzNrXtXgl9QH\nLAH6gTnAQkmzKxbbCXwF+OYwm7kWWAv4ggEzsw5Qa8Q/F9gYEZsiYh+wHFhQvkBEvBoRa4B9lStL\nmgbMB+4AlE7JZmbWilrBPxXYXPZ+S2lavW4GrgcONliXmZllpFbwN92ekXQxsCMinsKjfTOzjjGq\nxvyXgell76eTjPrr8THgEknzgXHAcZL+JiKuLF9Iknv/ZmZNiIimBtW1RvxrgFmSZkoaA1wOrBhm\n2cMKiIibImJ6RJwKXAE8Vhn6Zcv6J4Kvf/3rudfQKT/eF94X3hfVf1pRdcQfEfslLQIeAvqAZRGx\nTtI1pflLJU0CVgPHAQclXQvMiYi3KzfXUqVmZpaKWq0eImIlsLJi2tKy19s4vB001DZ+AvykyRrN\nzCxFvnK3gxQKhbxL6BjeF4d4XxzifZEOtdorarkAKfKuwcys20giMjq4a2ZmI4yD38ysxzj4zcx6\njIPfzKzHOPjNzHqMg9/MrMc4+M3MeoyD38ysxzj4zcx6jIPfzKzHOPjNzHqMg9/MrMc4+M3MeoyD\n36xN/vRP4V/+Je8qzOp4EIuZte6tt+C222DHDrjggryrsV7nEb9ZGzzwAJx1FvzTP8G77+ZdjfU6\nB79ZG9xzDyxaBGefDT/+cd7VWK/zE7jMMvbmmzBtGrz0Enz/+7BqFdx9d95VWbfzE7jMOtgDDyR9\n/fHj4dJLk/e/+U3eVVkvc/CbZeyee+Cyy5LXkycnvf6HH863JuttDn6zDL35Jjz+OCxYcGjaZZfB\nvffmV5NZXcEvqV/SekkbJN0wxPwzJP1M0ruSriubPl3S45Kek/QLSV9Ns3izTnf//fDJTyZtnkGX\nXppMd7vH8lIz+CX1AUuAfmAOsFDS7IrFdgJfAb5ZMX0f8LWIOBOYB3x5iHXNRqx774Xf+73Dp02Z\nAmeeCY88kk9NZvWM+OcCGyNiU0TsA5YDC8oXiIhXI2INSdCXT98WEU+XXr8NrAOmpFK5WYd7990k\n3C+55P3zfvd34Uc/an9NZlBf8E8FNpe931Ka1hBJM4FzgFWNrmvWjV56CSZNOrzNM+gjH4H169tf\nkxnUd8uGlk+yl3QMcB9wbWnkf5iBgYH3XhcKBQqFQqsfaZa7TZtgxoyh582YAS++2NZyrMsVi0WK\nxWIq26p5AZekecBARPSX3t8IHIyIbwyx7NeBtyPiW2XTRgMPACsj4pYh1vEFXDYiffe7ycVay5a9\nf96+fXD00bBnD4we3f7arPtlfQHXGmCWpJmSxgCXAyuGq6WiMAHLgLVDhb7ZSPbii8OP+EePTs7p\n37KlvTWZQR3BHxH7gUXAQ8Ba4IcRsU7SNZKuAZA0SdJm4GvA/5X0Uqm983Hgc8BvS3qq9NOf2W9j\n1kE2bYKZM4ef73aP5aWu2zJHxEpgZcW0pWWvtwHTh1j1X/FFYtajXnzRwW+dyaFslpFqB3fBwW/5\ncfCbZWDv3uShK1OrnPg8Y0by5WDWbg5+swxs3pyE/qgqzVSP+C0vDn6zDNRq80DS/3fwWx4c/GYZ\nqHVgF+C3fis5nfPgwbaUZPYeB79ZBmqdygkwblxyO4dt29pRkdkhDn6zDNTT6gEf4LV8OPjNMlBP\nqwd8gNfy4eA3y0A9rR5w8Fs+HPxmKdu/P+nbT5tWe1mf2WN5cPCbpWzLFpg4sb67bnrEb3lw8Jul\nrN42D/jgruXDwW+Wsmq3Y640OOL3IymsnRz8ZilrZMR/3HEwZgzs3JllRWaHc/CbpayRET/4AK+1\nn4PfLGWNjPjBfX5rPwe/WcqaCX6P+K2dHPxmKTpwAF5+GaYP9Ty6YTj4rd0c/GYpeuUVOOkkGDu2\n/nUc/NZuDn6zFNV7j55yDn5rNwe/WYrqvStnuZkzfXDX2svBb5airVthypTG1jnxRHj3XdizJ5ua\nzCo5+M1StH17cp+eRkhwyinJw9nN2qFm8Evql7Re0gZJNwwx/wxJP5P0rqTrGlnXbKRpJvghWWf7\n9vTrMRtK1eCX1AcsAfqBOcBCSbMrFtsJfAX4ZhPrmo0o27Y5+K3z1RrxzwU2RsSmiNgHLAcWlC8Q\nEa9GxBpgX6Prmo0027fDpEmNr+fgt3aqFfxTgc1l77eUptWjlXXNulKzrZ5TTnHwW/uMqjG/lZvF\n1r3uwMDAe68LhQKFQqGFjzXLx4EDsGtXcgFXoyZOhI0b06/JRo5isUixWExlW7WC/2Wg/OLz6SQj\n93rUvW558Jt1q1dfhQkTYFStv6ohTJwI//Zv6ddkI0floHjx4sVNb6tWq2cNMEvSTEljgMuBFcMs\nqxbWNet6zfb3wT1+a6+qY5OI2C9pEfAQ0Acsi4h1kq4pzV8qaRKwGjgOOCjpWmBORLw91LpZ/jJm\neWq2vw8Ofmuvmv8ojYiVwMqKaUvLXm/j8JZO1XXNRqpWgt8XcFk7+cpds5Q0ew4/wAknwFtvwd69\n6dZkNhQHv1lKWunxH3EEnHyyR/3WHg5+s5S00uoB9/mtfRz8Zilx8Fu3cPCbpaSVHj/4AK+1j4Pf\nLCUe8Vu3cPCbpWDwdg0nn9z8Nhz81i4OfrMUtHK7hkEOfmsXB79ZClpt84CD39rHwW+WgjSC3wd3\nrV0c/GYpaOXirUEe8Vu7OPjNUtDqqZyQHBjetSs5UGyWJQe/WQrSaPWMGgXjx8Nrr6VTk9lwHPxm\nKUgj+CHZhvv8ljUHv1kK0ujxg5+9a+3h4DdLQRo9fvABXmsPB79ZCtJs9Tj4LWsOfrMWpXG7hkEO\nfmsHB79Zi157rfXbNQzywV1rBwe/WYvS6u+DD+5aezj4zVqUVn8f3Oqx9nDwm7XIwW/dxsFv1qK0\nzuGHQzdqi0hne2ZDqRn8kvolrZe0QdINwyxza2n+M5LOKZt+o6TnJD0r6W8ljU2zeLNOkGaPf9w4\nOPJI2L07ne2ZDaVq8EvqA5YA/cAcYKGk2RXLzAdOi4hZwNXA7aXpM4EvAedGxFlAH3BFyvWb5S7N\nVg+43WPZqzXinwtsjIhNEbEPWA4sqFjmEuAugIhYBYyXNBF4E9gHHCVpFHAU8HKaxZt1Age/dZta\nwT8V2Fz2fktpWs1lImIX8C3gJeAVYHdEPNJauWadZ8cOB791l1qXnNR7iEnvmyB9EPgTYCbwBnCv\npN+PiB9ULjswMPDe60KhQKFQqPNjzfK3fXtyUDYtEycmz/A1K1csFikWi6lsq1bwvwxML3s/nWRE\nX22ZaaVpBeDfI2IngKR/AD4GVA1+s25y8GBy5W4at2sYdPLJvnrX3q9yULx48eKmt1Wr1bMGmCVp\npqQxwOXAioplVgBXAkiaR9LS2Q48D8yTdKQkARcBa5uu1KwD7doFxx4LY8akt00/e9eyVnXEHxH7\nJS0CHiI5K2dZRKyTdE1p/tKIeFDSfEkbgT3AVaV5T0v6G5Ivj4PAk8B3M/xdzNou7f4+OPgtezVv\nKxURK4GVFdOWVrxfNMy6fwn8ZSsFmnWyHTvS7e+Dg9+y5yt3zVqQ9oFdcPBb9hz8Zi1wq8e6kYPf\nrAVZjPjHj4e334a9e9PdrtkgB79ZC7IY8R9xRHJKp8/lt6w4+M1akMWIH9zusWw5+M1akMWIHxz8\nli0Hv1kLsjidExz8li0Hv1kL3OqxbuTgN2vSnj1w4EByy4a0OfgtSw5+syYNtnn0vnvTts7Bb1ly\n8Js1KasDu+Dgt2w5+M2alNWBXXDwW7Yc/GZNyurALjj4LVsOfrMmZdnqGXwYS9T7DDyzBjj4zZqU\n5Yj/6KOhry+5Z49Z2hz8Zk3KcsQPbvdYdhz8Zk3KcsQPDn7LjoPfrEke8Vu3cvCbNSnL0znBwW/Z\ncfCbNWH/fnj9dTjxxOw+w8FvWXHwmzVh506YMAFGjcruMxz8lhUHv1kTsj6wCw5+y46D36wJWR/Y\nBQe/Zadm8Evql7Re0gZJNwyzzK2l+c9IOqds+nhJ90laJ2mtpHlpFm+WF4/4rZtVDX5JfcASoB+Y\nAyyUNLtimfnAaRExC7gauL1s9reBByNiNnA2sC7F2s1yk/UZPeDgt+zUGvHPBTZGxKaI2AcsBxZU\nLHMJcBdARKwCxkuaKOl44JMR8b3SvP0R8Ua65Zvlox2tnpNOgl27koe9mKWpVvBPBTaXvd9SmlZr\nmWnAqcCrku6U9KSkv5Z0VKsFm3WCdrR6Ro2C449Pwt8sTbVORqv33oCVzyCK0rbPBRZFxGpJtwB/\nBvy/ypUHBgbee10oFCgUCnV+rFk+2jHih0PtnpNPzv6zrLMVi0WKxWIq26oV/C8D08veTycZ0Vdb\nZlppmoAtEbG6NP0+kuB/n/LgN+sG7Rjxw6HgP/PM7D/LOlvloHjx4sVNb6tWq2cNMEvSTEljgMuB\nFRXLrACuBCidtbM7IrZHxDZgs6TTS8tdBDzXdKVmHaTdI36zNFUd8UfEfkmLgIeAPmBZRKyTdE1p\n/tKIeFDSfEkbgT3AVWWb+Arwg9KXxgsV88y6UkT72i8OfstCzQvOI2IlsLJi2tKK94uGWfcZ4KOt\nFGjWad56K3lIytFHZ/9ZDn7Lgq/cNWvQ1q0weXJ7PsvBb1lw8Js16JVXYMqU9nzWKackB5LN0uTg\nN2vQ1q3tC/5Jk2DbtvZ8lvUOB79Zg9o54p8yJfmiMUuTg9+sQe0M/kmTkuA/eLA9n2e9wcFv1qB2\nBv+4cXDsscmDX8zS4uA3a1A7gx/c7rH0OfjNGtTu4J88OflMs7Q4+M0aEJGEcLvO44fkS8bBb2ly\n8Js14I03YPRoOOaY9n3m5Mlu9Vi6HPxmDWh3mwc84rf0OfjNGuDgt5HAwW/WgDyC360eS5uD36wB\nHvHbSODgN2tAHsE/eL8eX71raXHwmzUgj+D31buWNge/WQPyCH5wu8fS5eA3a0Bewe8DvJYmB79Z\nnSLa+/Stch7xW5oc/GZ12rkzuWJ33Lj2f7aD39Lk4DerU15tHnCrx9Ll4DerU57B7xG/pcnBb1an\nvEf8Dn5LS83gl9Qvab2kDZJuGGaZW0vzn5F0TsW8PklPSbo/raLN8pD3iN+tHktL1eCX1AcsAfqB\nOcBCSbMrlpkPnBYRs4CrgdsrNnMtsBaItIo2y0PeI35fvWtpqTXinwtsjIhNEbEPWA4sqFjmEuAu\ngIhYBYyXNBFA0jRgPnAHoDQLN2u3PIN/7Njk6t3XXsvn821kqRX8U4HNZe+3lKbVu8zNwPWAxynW\n9dr95K1KbvdYWkbVmF9ve6ZyNC9JFwM7IuIpSYVqKw8MDLz3ulAoUChUXdwsF3mO+OHQmT0f/nB+\nNVh+isUixWIxlW0pYvhslzQPGIiI/tL7G4GDEfGNsmW+AxQjYnnp/XqgAHwV+DywHxgHHAf8fURc\nWfEZUa0Gs05w4EBy4daePTBmTD41fOELcMEF8MUv5vP51lkkERFNtdBrtXrWALMkzZQ0BrgcWFGx\nzArgylIh84DdEbEtIm6KiOkRcSpwBfBYZeibdYtXX4UJE/ILffC5/Jaeqq2eiNgvaRHwENAHLIuI\ndZKuKc1fGhEPSpovaSOwB7hquM2lWbhZO23dmm+bB5LjC+vW5VuDjQy1evxExEpgZcW0pRXvF9XY\nxk+AnzRToFknyLu/D8nnP/ZYvjXYyOArd83q8PLL+Z7RA271WHoc/GZ12LQJZs7MtwbftsHS4uA3\nq8MLL8AHP5hvDZMnw/btvnrXWufgN6tDJwT/2LEwfnxy6wazVjj4zerQCcEP8IEPJLWYtcLBb1bD\nrl1Je+XEE/OuJPnycfBbqxz8ZjUMjvbVAbcZdPBbGhz8ZjV0SpsHkjp+9au8q7Bu5+A3q6HTgt8j\nfmuVg9+sBge/jTQOfrMaOin4J02C//ovePPNvCuxbubgN6uhk4Jf8imd1joHv1kV77yTPO5w2rS8\nKznE7R5rlYPfrIpf/xpmzIC+vrwrOcTBb61y8JtV0UltnkEOfmuVg9+sCge/jUQOfrMqHPw2Ejn4\nzap44YXkLJpOMmNG8ijIvXvzrsS6lYPfrIpOHPGPHg1TpyYPhzFrhoPfbBgHDsCLL3beiB/c7rHW\nOPjNhrFlS3Ir5iOPzLuS93PwWysc/GbD6MQ2zyAHv7XCwW82DAe/jVR1Bb+kfknrJW2QdMMwy9xa\nmv+MpHNK06ZLelzSc5J+IemraRZvliUHv41UNYNfUh+wBOgH5gALJc2uWGY+cFpEzAKuBm4vzdoH\nfC0izgTmAV+uXNesU3Vy8H/gA8ntJA4ezLsS60b1jPjnAhsjYlNE7AOWAwsqlrkEuAsgIlYB4yVN\njIhtEfF0afrbwDpgSmrVm2Xo+efh9NPzrmJoxx6b/Gzdmncl1o3qCf6pwOay91tK02otc9j9DCXN\nBM4BVjVapFm77dkDGzfChz6UdyXDc7vHmjWqjmWizm1VPor6vfUkHQPcB1xbGvkfZmBg4L3XhUKB\nQqFQ50eaZePJJ+Gss2Ds2LwrGd5pp8GGDXDBBXlXYu1QLBYpFoupbEsR1XNd0jxgICL6S+9vBA5G\nxDfKlvkOUIyI5aX364ELI2K7pNHAA8DKiLhliO1HrRrM2u1b30p66EuW5F3J8G6+OQn+227LuxLL\ngyQionLAXZd6Wj1rgFmSZkoaA1wOrKhYZgVwZamYecDuUugLWAasHSr0zTrV6tUwd27eVVR3/vmw\nyo1Ta0LN4I+I/cAi4CFgLfDDiFgn6RpJ15SWeRD4laSNwFLgf5dW/zjwOeC3JT1V+unP4hcxS9MT\nT3R+8J9zDqxblzwlzKwRNVs9mRfgVo91mNdeSw6cvv46HNHhlziedx58+9vw8Y/nXYm1W9atHrOe\nsnp1EqidHvrgdo81pwv+1zZrryeegI9+NO8q6nP++fAf/5F3FdZtHPxmFbrhwO6gefM84rfGOfjN\nykR0x4HdQbNmwVtvwbZteVdi3cTBb1bmpZegry95wlU3kJIvKY/6rREOfrMyg6N9NXWuRD58gNca\n5eA3K9NNB3YHOfitUQ5+szLddGB30Ny5sGZN8oxgs3o4+M1KDhxIbs523nl5V9KYk06Ck0+G9evz\nrsS6hYPfrGT1apg2DU44Ie9KGufTOq0RDn6zku9/HxYuzLuK5rjPb43wvXrMgL17k1M4n3gCTj01\n72oa95//CZ/5THIr6W641YS1zvfqMWvRP/8znHFGd4Y+wNlnJy2qRx/NuxLrBg5+M+Duu+Hzn8+7\nitb80R/BsmV5V2HdwK0e63m7d8OMGbBpE0yYkHc1zXv99eRfLC+8ACeemHc1ljW3esxacO+98Du/\n092hD0n9F1+cHKQ2q8bBbz1vJLR5Bv3hH8IddyQ3mzMbjoPfetqmTcnjCz/1qbwrSceFFyaPYly9\nOu9KrJM5+K2n3XILXHEFjBmTdyXpOOII+OIXfZDXqvPBXetZjz4KV14JzzyT3PZgpHjlFfjQh2DD\nBh/kHcl8cNesQTt3whe+AHfeObJCH2DKlKTX/9nP+sZtNjSP+K3nRMBll8H06XDzzXlXk439+6G/\nP7nh3F/8Rd7VWBYyHfFL6pe0XtIGSTcMs8ytpfnPSDqnkXXN2u3OO+GXv4Q///O8K8nOqFGwfDn8\n8Idwzz15V2OdpmrwS+oDlgD9wBxgoaTZFcvMB06LiFnA1cDt9a5rhysWi3mX0DGy2Bd79sB118FN\nN8EPfgDjxqX+EZlodl+cdBL84z/CokXw05+mW1Ne/DeSjloj/rnAxojYFBH7gOXAgoplLgHuAoiI\nVcB4SZPqXNfK+H/qQ9LeF48+mtzPZts2ePZZOOusVDefqVb2xUc+kvwL57OfhUsvTU5d7Wb+G0nH\nqBrzpwKby95vAc6vY5mpwJQ61jVL3TvvJAH/5JPwyCNJ6O/dC3/1V/DpT+ddXft9+tNJa2vJkuQ8\n/0IBPvEJOPfc5IvhmGPyrtDarVbw13vUtaVHU3/mM62sPXI8/zz8/Od5V9EZnn8+eZwgHLoKNeLQ\nz8GDyRkr+/fDb36ThP0778COHcl/J02CM8+Eiy6CP/7j5PTGXr5d8ZFHwvXXw5e+lNyi4uc/T27t\n8OyzMHp0cruHCRPgqKOS96NGJT9Sst+k9z+APo8H0vf638jixckXdquqntUjaR4wEBH9pfc3Agcj\n4htly3wHKEbE8tL79cCFwKm11i1N9yk9ZmZNaPasnloj/jXALEkzgVeAy4HKZxStABYBy0tfFLsj\nYruknXWs23ThZmbWnKrBHxH7JS0CHgL6gGURsU7SNaX5SyPiQUnzJW0E9gBXVVs3y1/GzMxqy/0C\nLjMza69cD3f18gVekqZLelzSc5J+IemrpeknSHpY0i8l/VjS+LxrbQdJfZKeknR/6X1P7gcASeMl\n3SdpnaS1ks7vxf0h6cbS38ezkv5W0the2Q+Svidpu6Rny6YN+7uX9tWGUp7+j1rbzy34fYEX+4Cv\nRcSZwDzgy6Xf/8+AhyPidODR0vtecC2wlkNnkvXqfgD4NvBgRMwGzgbW02P7o3Rs8EvAuRFxFkm7\n+Ap6Zz/cSZKN5Yb83SXNITmGOqe0zm2SqmZ7niP+nr7AKyK2RcTTpddvA+tIrn9474K40n//Vz4V\nto+kacB84A4OnRrcc/sBQNLxwCcj4nuQHCuLiDfovf3xJsng6ChJo4CjSE4S6Yn9EBE/BV6vmDzc\n774A+LuI2BcRm4CNJPk6rDyDf7gLv3pOaXRzDrAKmBgR20uztgMTcyqrnW4GrgcOlk3rxf0AyWnQ\nr0q6U9KTkv5a0tH02P6IiF3At4CXSAJ/d0Q8TI/thwrD/e5TSPJzUM0szTP4fVQZkHQM8PfAtRHx\nVvm80m1LR/R+knQxsCMinmKYCwF7YT+UGQWcC9wWEeeSnCl3WDujF/aHpA8CfwLMJAm2YyR9rnyZ\nXtgPw6njd6+6X/IM/peB6WXvp3P4t9aIJ2k0SejfHRE/Kk3eXrrXEZImAzvyqq9NPgZcIunXwN8B\n/13S3fTefhi0BdgSEYMPT7yP5ItgW4/tj/OAf4+InRGxH/gH4L/Re/uh3HB/E5VZOq00bVh5Bv97\nF4dJGkNycGJFjvW0lSQBy4C1EXFL2awVwB+UXv8B8KPKdUeSiLgpIqZHxKkkB+8ei4jP02P7YVBE\nbAM2Szq9NOki4Dngfnprf6wH5kk6svS3chHJwf9e2w/lhvubWAFcIWmMpFOBWcATVbcUEbn9AJ8C\nnic5GHFjnrXk8Lt/gqSn/TTwVOmnHzgBeAT4JfBjYHzetbZxn1wIrCi97uX98GFgNfAMyUj3+F7c\nH8D/IfnSe5bkYOboXtkPJP/6fQXYS3Is9KpqvztwUylH1wP/s9b2fQGXmVmP6eH7FZqZ9SYHv5lZ\nj3Hwm5n1GAe/mVmPcfCbmfUYB7+ZWY9x8JuZ9RgHv5lZj/n/1UdUSlFFFjEAAAAASUVORK5CYII=\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import matplotlib.pyplot as plt \n", - "xs = list(range(0,100))\n", - "ps = suite.Probs(xs)\n", - "plt.plot(xs, ps);" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Summarizing the posterior\n", - "\n", - "Again, there are several ways to summarize the posterior distribution.\n", - "One option is to find the most likely value in the posterior\n", - "distribution. `thinkbayes` provides a function that does that:\n", - "\n", - "```python\n", - " def MaximumLikelihood(pmf):\n", - " \"\"\"Returns the value with the highest probability.\"\"\"\n", - " prob, val = max((prob, val) for val, prob in pmf.Items())\n", - " return val\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "56" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "suite.MaximumLikelihood()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this case the result is 56, which is also the observed percentage of\n", - "heads, $140/250 = 56\\%$. So that suggests (correctly) that the observed\n", - "percentage is the maximum likelihood estimator for the population.\n", - "\n", - "We might also summarize the posterior by computing the mean and median:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Mean 55.952380952380956\n", - "Median 56\n" - ] - } - ], - "source": [ - "import thinkbayes\n", - "print('Mean', suite.Mean())\n", - "print('Median', thinkbayes.Percentile(suite, 50))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, we can compute a credible interval:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "CI (51, 61)\n" - ] - } - ], - "source": [ - "print('CI', thinkbayes.CredibleInterval(suite, 90))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, getting back to the original question, we would like to know\n", - "whether the coin is fair. We observe that the posterior credible\n", - "interval does not include 50%, which suggests that the coin is not fair.\n", - "\n", - "But that is not exactly the question we started with. MacKay asked, “ Do\n", - "these data give evidence that the coin is biased rather than fair?” To\n", - "answer that question, we will have to be more precise about what it\n", - "means to say that data constitute evidence for a hypothesis. And that is\n", - "the subject of the next chapter.\n", - "\n", - "But before we go on, I want to address one possible source of confusion.\n", - "Since we want to know whether the coin is fair, it might be tempting to\n", - "ask for the probability that x is 50%:\n", - "\n", - " print suite.Prob(50)\n", - "\n", - "The result is 0.021, but that value is almost meaningless. The decision\n", - "to evaluate 101 hypotheses was arbitrary; we could have divided the\n", - "range into more or fewer pieces, and if we had, the probability for any\n", - "given hypothesis would be greater or less." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Swamping the priors {#triangle}\n", - "\n", - "We started with a uniform prior, but that might not be a good choice. I\n", - "can believe that if a coin is lopsided, $x$ might deviate substantially\n", - "from 50%, but it seems unlikely that the Belgian Euro coin is so\n", - "imbalanced that $x$ is 10% or 90%.\n", - "\n", - "It might be more reasonable to choose a prior that gives higher\n", - "probability to values of $x$ near 50% and lower probability to extreme\n", - "values.\n", - "\n", - "As an example, I constructed a triangular prior. Here’s the code that constructs the prior:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def TrianglePrior(suite):\n", - " for x in range(0, 51):\n", - " suite.Set(x, x)\n", - " for x in range(51, 101):\n", - " suite.Set(x, 100-x) \n", - " suite.Normalize()\n", - " return suite" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEACAYAAABcXmojAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X98VPWd7/HXJxhvsRAihkYNKCq0/qi9urYQb0Wo0l2I\ngmItmtQfaLcLCN3W3l4BZRXdPh4WUK56XREosmptQGutICKwaog+3EJxWdsKUX4pSACt8suqbYDP\n/ePMhCEmmQmZyZmZ834+HvPonDPnzHznWOaTc873+32buyMiIgJQEHYDREQke6goiIhIIxUFERFp\npKIgIiKNVBRERKSRioKIiDRKWhTMbIiZ1ZnZejOb0MI2D8Ref8PMzo2t62VmL5vZm2b2JzP754Tt\np5jZe2a2JvYYkr6vJCIiR+qo1l40s07Ag8BgYBvwezNb6O7rErapAPq4e18z6w/MBMqBBuBmd/9v\nM+sCvG5my9y9DnBghrvPyMzXEhGRI5HsTKEfsMHd33H3BmA+cFmTbYYDjwK4+0qg2MxK3X2Hu/93\nbP3HwDqgLGE/S8cXEBGR9ElWFMqArQnL73H4D3tL2/RM3MDMegPnAisTVv8wdrlprpkVt6HNIiKS\nIcmKQqpzYDT9q79xv9ilo18DP4qdMUBwiekU4BxgO3Bvip8jIiIZ1Oo9BYL7CL0SlnsRnAm0tk3P\n2DrMrBB4Gvilu/82voG7vx9/bma/ABY19+FmpomZRETayN2P+PJ8sjOF1UBfM+ttZkcDVwELm2yz\nELgOwMzKgd3uvtPMDJgLrHX3+xJ3MLMTEhZHAH9sqQHuroc7d9xxR+htyIaHjoOOhY5F64/2avVM\nwd33m9l4YCnQCZjr7uvMbHTs9Vnu/ryZVZjZBuAvwA2x3b8JXAP8wczWxNZNcvcXgKlmdg7BZabN\nwOh2fxMREWm3ZJePcPclwJIm62Y1WR7fzH6v0sKZiLtf17ZmiohIR9CI5hwxaNCgsJuQFXQcDtGx\nOETHIn0sHdegMsXMPJvbJyKSbcwMb8eN5qSXj0REjkTQ10QyKRN/NKsoiEjG6Ew/czJVdHVPQURE\nGqkoiIhIIxUFERFppKIgIpKisWPH8rOf/axxeebMmZSWllJUVMSuXbtCbFn6qEuqiGRErGtk2M3I\nmIaGBrp168aqVav46le/2uGf39LxbW+XVJ0piIgcgR07dvDZZ59xxhlntHnfdM1TlAkqCiISKQUF\nBWzatKlxedSoUfzLv/wLADU1NfTs2ZMZM2ZQWlrKiSeeyL//+79/btv169dz+umnA1BcXMzgwYMB\neO211/jGN75BcXEx/fr14z//8z8b9x00aBCTJ0/mm9/8Jl26dGHTpk0UFBQwc+ZM+vbtS1FREbff\nfjsbN27k/PPPp7i4mKuvvpqGhoYOOCqHaJyCiESamR3W53/nzp3s3buX+vp6li1bxpVXXsmIESPo\n1q1b47Z9+/Zl7dq1nHLKKezZs4eCggI++ugjLrnkEh588EEqKyt58sknueSSS9i4cSPHHnssAL/8\n5S9ZsmQJX/nKVzhw4AAAy5YtY82aNWzZsoVzzz2XV199lerqarp37875559PdXU1113XcdPF6UxB\nREIR/4Ft7yMdEi/lFBYWcvvtt9OpUyeGDh1Kly5deOuttz63bdPLP4sXL+YrX/kK3/ve9ygoKODq\nq6/m9NNPZ+HChY3fd9SoUZxxxhkUFBRQWFgIwC233EKXLl0488wzOfvssxk6dCi9e/emqKiIoUOH\nsmbNGjqSzhREJBTZek39uOOOo6Dg0N/LxxxzDB9//HErewTq6+s56aSTDlt38sknU19f37jcq1ev\nprtRWlra+Lxz586fW96xY0eb2t9eOlMQkUg55phj+OSTTxqXt2/fnpYzjrKyMt59993D1r377ruU\nlR2Ktc+F+aBUFEQkUs455xyeeOIJDhw4wAsvvEBtbW3K+7Z2dlNRUcHbb79NdXU1+/fvZ8GCBdTV\n1XHppZemtH9z24RxNqWiICKRcv/997No0SKOPfZYfvWrXzFixIjDXm/tr/mm9zESn3fv3p3nnnuO\ne++9l5KSEu655x6ee+45unfv3uJ7N/dZTd+/o88uNHhNRDIi3wevhU2D10REJONUFEREpJGKgoiI\nNFJREBGRRioKIiLSSEVBREQaqSiIiEgjFQUREWmkoiAi0sSWLVvo2rVrhwy+a5rvEDYVBRGJnN69\ne/PSSy+1+PpJJ53Evn37cmICu3RTURCRyGltCo79+/d3cGuyi4qCiETKtddey5YtWxg2bBhdu3Zl\n+vTpFBQU8Mgjj3DyySczePBg3n33XQoKCjh48CAA8+bN48wzz6SoqIjTTjuN2bNnN75fsgjPDz/8\nkGHDhtGtWzf69evH5MmTGTBgQLNt++tf/8pPf/pTTj75ZI4//njGjh3LZ599ltHj0ZSKgohEyuOP\nP85JJ53Ec889x759+xg5ciQAtbW11NXVsXTp0s+dRZSWlrJ48WL27t3LvHnzuPnmmw9LREuM8Jw7\ndy7jxo1jz549AIwbN46uXbuyc+dOHn30UR577LEWL0tNnDiRDRs28MYbb7Bhwwa2bdvGXXfdlaEj\n0QJ3z9pH0DwRyUXJ/v1Ceh5Honfv3v7iiy+6u/vmzZvdzHzz5s2Nr8fXHThwoNn9L7/8cr///vvd\n3f3ll1/2zp07H7btl770JV+5cqXv37/fCwsL/e233258bfLkyX7BBRc0LpuZb9y40Q8ePOhf/OIX\nfePGjY2vvfbaa37KKac024aWjm9s/RH/7iqOU0RCkW2zajcXlRm3ZMkS7rzzTtavX8/Bgwf55JNP\n+NrXvtb4eksRnh988AH79+8/7L179uzZ7Gd88MEHfPLJJ5x33nmN69y98RJWR9HlIxGJnGThNon+\n+te/8p3vfIdbbrmF999/n127dlFRUZFSd9UePXpw1FFHsXXr1sZ1ic8TlZSU0LlzZ9auXcuuXbvY\ntWsXu3fvZu/evSl+q/RQURCRyCktLWXjxo0pbfu3v/2Nv/3tb5SUlFBQUMCSJUtYtmxZSvt26tSJ\nK664gilTpvDpp59SV1fH448/3mwBKigo4Ac/+AE//vGP+eCDDwDYtm1byp+VLioKIhI5kyZN4mc/\n+xndu3fn6aefbvXMoWvXrjzwwAOMHDmS7t27U11dzWWXXdbsts158MEH2bNnD8cffzzXX389lZWV\nHH300c3uO3XqVPr06UN5eTndunXj29/+Nm+//XZ7v26bJI3jNLMhwH1AJ+AX7j61mW0eAIYCnwCj\n3H2NmfUCHgO+BDgw290fiG3fHVgAnAy8A4x0993NvK+ncoomItlHcZzNmzBhAu+//z7z5s1r1/uE\nEsdpZp2AB4EhwJlApZmd0WSbCqCPu/cF/gmYGXupAbjZ3c8CyoFxZnZ67LWJwHJ3/zLwYmxZRCTv\nvPXWW/zhD3/A3Vm1ahWPPPIII0aMCLtZLUrW+6gfsMHd3wEws/nAZcC6hG2GA48CuPtKMys2s1J3\n3wHsiK3/2MzWAWVAXWyfgbH9HwVqUGEQkTy0b98+Kisrqa+vp7S0lJ/+9KcMHz487Ga1KFlRKAMS\nb5W/B/RPYZuewM74CjPrDZwLrIytKnX3+Os7gdK2NFokTH/+M3TqBMceG3ZLJBd8/etfZ/369WE3\nI2XJikKqFwSbXr9q3M/MugC/Bn7k7h9/7gNigzdaeuMpU6Y0Ph80aBCDBg1KsUki6bd9O1xwAXzh\nC1BbC8cdF3aLJOpqamqoqalJ2/u1eqPZzMqBKe4+JLY8CTiYeLPZzB4Gatx9fmy5Dhjo7jvNrBB4\nDlji7vcl7FMHDHL3HWZ2AvCyu59OE7rRLNnkww9h4ECorIR9++DFF4NHUVHYLctOutGcWaHcaAZW\nA33NrLeZHQ1cBSxsss1C4LpYY8qB3bGCYMBcYG1iQUjY5/rY8+uB3x7pFxDpCPv2QUUFDB0Kt94K\nd98N550Hw4fDp5+G3TqR9EmlS+pQDnVJnevud5vZaAB3nxXbJt5D6S/ADe7+X2Z2AVAL/IFDl5Mm\nufsLsS6pTwInoS6pkuU++ywoCH36wKxZEO9WfuAAXHttUDB+8xsoLAy3ndkmilkEHS0TZwpJi0KY\nVBQkbA0NcOWV0LkzPPFEcIO56esjRgSXkB5//POvi3S0TF8+Eomsgwfhhhtg/3547LHmf/ALC+Gp\np6C+HsaPz75J3kTaSkVBpBnu8MMfwpYtwY9+wqwEn9O5MyxcCKtXw6RJHddGkUzQ1NkizZg8GVau\nhJdegmOOSb59UREsWRL0Tiouhokaiik5SkVBpIl77oFnngnGIbSlu2lJCSxfDgMGBIVhzJjMtVEk\nU1QURBLMmQP/9m/wyivBj3xbnXhiUBguvDAoKFVV6W+jSCapKIjELFgAU6bAihXQQjhWSk49FV54\nAQYPDgrDpZemrYkiGacuqSLA888HPY3+4z/g7LPT856rVgUF4cknQbOzSEdRl1SRdqqthVGj4Nln\n01cQAPr1C84+Ro6E3/8+fe8rkkkqChJpr78eDE6rroby8vS//7e+BXPnwrBh8Oab6X9/kXRTUZDI\nWrsWLrkEZs+Giy/O3OcMGwb33gv/8A+waVPmPkckHXSjWSJp8+bgR3r6dLj88sx/3ve+B3v3Bjef\nX3016KUkko1UFCRytm+Hb38bJkwIJrTrKGPHwp49wWcri0GylXofSaR89FEw6vjqq+G228Jpw8SJ\nymKQzNEsqSIp2rcvuHxz4YUwbdqhKbA7mjvcdBOsWxdMjdG5czjtkPykoiCSgpYyEcKiLAbJFBUF\nkSSSZSKERVkMkgkavCbSingmQkNDy5kIYVEWg2QjFQXJW4mZCL/+deuZCGFRFoNkG3VJlbw1eTL8\n7nepZyKERVkMkk1UFCQvTZsW3MCtrYVu3cJuTXLKYpBsoaIgeWf2bJg5M8hE6NEj7NakTlkMkg1U\nFCSvzJ8Pd97Z/kyEsCRmMXTtGsybJNKR1CVV8sbixXDjjenNRAiLshjkSKlLqgjBvYMbbgh68uR6\nQQBlMUh4VBQk5yVmIvTvH3Zr0kdZDBIGFQXJaWvXBpdZ5szJbCZCWIYNgxkzlMUgHUc3miVnJWYi\nXHZZ2K3JnKqqQ1Nuv/KKshgks1QUJCfFMxEmToRrrgm7NZmnLAbpKOp9JDnnww+D0b9VVXDrrWG3\npmMpi0GS0SypEinxTISBA2Hq1PCnwO5o7sFZQ12dshikeSoKEhnZlokQFmUxSGtUFCQSsjUTISzx\nLIZu3YIshgL1I5QYDV6TvBfPRNi/P/syEcISz2LYtg3GjVMWg6SPioJktcRMhKeeys5MhLAkZjFE\n7Ya7ZI66pEpWy5VMhLAkZjF066YsBmm/pGcKZjbEzOrMbL2ZTWhhmwdir79hZucmrH/EzHaa2R+b\nbD/FzN4zszWxx5D2fxXJN9OnwzPPwNKluZGJEJaSEli2LJgy/OGHw26N5LpWzxTMrBPwIDAY2Ab8\n3swWuvu6hG0qgD7u3tfM+gMzgfLYy/OA/wc81uStHZjh7jPS8zUk3yRmIpSUhN2a7FdWFswOqywG\naa9kZwr9gA3u/o67NwDzgaYTCgwHHgVw95VAsZkdH1t+BdjVwntHtEOhJBPPRFi2LPixk9TEsxh+\n8hNYtCjs1kiuSlYUyoCtCcvvxda1dZvm/DB2uWmumRWnsL1EwOLF8KMfBT9uffqE3Zrc89WvBjef\nv/99qKkJuzWSi5IVhVQ7ujX9qz/ZfjOBU4BzgO3AvSl+juSxFSvyKxMhLMpikPZI1vtoG9ArYbkX\nwZlAa9v0jK1rkbu/H39uZr8AWjzZnTJlSuPzQYMGMUgxVHlp9Wr47neDS0f5lIkQlsQshhdfhLPO\nCrtFkik1NTXUpPG0sNURzWZ2FPAWcDFQD6wCKpu50Tze3SvMrBy4z93LE17vDSxy97MT1p3g7ttj\nz28GvuHun7s1phHN0bB2bZCF8PDD+T0Fdhh+9Su45ZZgZtVTTw27NdIR2juiudUzBXffb2bjgaVA\nJ2Cuu68zs9Gx12e5+/NmVmFmG4C/ADckNK4aGAgcZ2ZbgdvdfR4w1czOIbjMtBkYfaRfQHJbPBNh\n2jQVhExQFoO0leY+ktBs3w4DBsDNNwdTNUjm/PznwRxJymLIf5oQT3JSlDMRwqIshmhQUZCcE/VM\nhLAoiyEaVBQkpygTIVzKYsh/KgqSM5SJkB3iWQxFRcF9Bv13yC/KU5CcoEyE7BHPYqivh/HjlcUg\nh1NRkIyLZyJs3apMhGyhLAZpifIUJOPimQgvv6xMhGySmMVQXAwTmp0YX6JGRUEyatq04IZmba26\nQWajeBbDgAFBYRitYaSRp6IgGZOYidCjR9itkZYkZjF07aoshqhTUZCMiGcirFgBPXuG3RpJJp7F\nMHhwUBiGDQu7RRIWdUmVtFu8GG68MfjrU1Ng55ZVq+DSS+HJJ0ETEucmdUmVrLJiBYwaBc8+q4KQ\ni5TFICoKkjaJmQjl5cm3l+yUmMXw5ptht0Y6moqCpMXatcFlhzlzgmwEyW3DhsGMGcG05ps2hd0a\n6Ui60SztFs9EuOceZSLkk6oq2LtXWQxRo6Ig7bJ9e/CjMXEiXHNN2K2RdBszBnbvDv4bK4shGtT7\nSI7YRx8Fo2ErKzVVQr5TFkPu0CypEgplIkSLshhyh4qCdLjPPoNLLoHTTlMmQpQcOADXXRfcZ1AW\nQ/ZSUZAO1dAQdDv9wheUiRBFDQ1wxRXBqGdlMWQnDV6TDhPPRGhoUCZCVBUWBqOdlcWQv1QUJCXK\nRJC4xCyGSZPCbo2km7qkSkpuuw1WroSXXlImgnw+i2HixLBbJOmioiBJTZ0KzzyjTAQ5XNMshjFj\nwm6RpIOKgrRq1ix4+GF49VVlIsjnJWYxFBUpiyEfqChIi6qr4a67gplPy8rCbo1kK2Ux5Bd1SZVm\nKRNB2kpZDNlBXVIl7VasCLqeLlyogiCpUxZDflBRkMMkZiL07x92ayTXKIsh96koSKPETISLLgq7\nNZKrlMWQ23SjWYBDmQjTpysTQdqvqgr27FEWQy5SURC2bw96jkycCNdeG3ZrJF+MHXuoMCiLIXeo\n91HEffhhMCq1qkqZCJIZymLoWJolVY5YPBPhwgth2jRNgS2ZoSyGjqWiIEfks8+gogL69FEmgmTe\ngQPBpcl9+5TFkGkqCtJmDQ1w5ZXBX2zKRJCO0tAAI0YEl5CUxZA5GR+8ZmZDzKzOzNab2YQWtnkg\n9vobZnZuwvpHzGynmf2xyfbdzWy5mb1tZsvMrPhIv4C0TTwTYf9+ZSJIxyosDKZdr6+HceOUxZCt\nWi0KZtYJeBAYApwJVJrZGU22qQD6uHtf4J+AmQkvz4vt29REYLm7fxl4MbYsGRbPRNiyRZkIEo54\nFsPrr6tjQ7ZKdqbQD9jg7u+4ewMwH2jai3048CiAu68Eis3s+NjyK8CuZt63cZ/Y/15+ZM2Xtpg8\nOchEeO45ZSJIeOJZDAsXws9/HnZrpKlk4xTKgK0Jy+8BTSc/aG6bMmBHK+9b6u47Y893AqXJmyrt\nMX26MhEkeyiLIXslKwqpXvVrelMj5auF7u5mpquLGTRnDjz0UDCytKQk7NaIBBKzGLp1g8rKsFsk\nkLwobAN6JSz3IjgTaG2bnrF1rdlpZse7+w4zOwF4v6UNTX0l22kkMAMYSK9eG8NujEgzzqKq6j+o\nqvpHYHHYjclZd9xxR1rep9UuqWZ2FPAWcDFQD6wCKt19XcI2FcB4d68ws3LgPncvT3i9N7DI3c9O\nWDcN+NDdp5rZRKDY3T93s1ldUtvn+eeDnkbKRJBspyyG9Mlol1R33w+MB5YCa4EF7r7OzEab2ejY\nNs8Dm8xsAzALuCmhcdXAa8CXzWyrmd0Qe+nnwLfN7G3gotiypFFtLYwaBc8+q4Ig2U9ZDNlDg9fy\n0OrVwWjl6mq4+OKwWyOSukWL4Ac/COZJOuussFuTm5S8JoeJZyLMnq2CILlHWQzh09TZeSQxE+Fy\njfyQHFVVBXv3KoshLCoKeWL79uAfkTIRJB+MGaMshrDonkIeiGciVFbCbbeF3RqR9Jk0Keg9pyyG\n1GmW1IiLZyIMHAhTp2oKbMkv7nDTTbBunbIYUqWiEGHKRJAoOHgwuCS6d6+yGFKhohBRykSQKGlo\ngCuugK5dlcWQjLqkRpAyESRqCguD0c719TB+vLIYMklFIcfEMxG2blUmgkRLPIth9ergBrRkhrqk\n5ph4JsJLLykTQaInnsUwcGAws6qKQ/qpKOSQadOCTIQVK9Q9T6KrpASWL4cLLgiyGMaODbtF+UVF\nIUfMng0zZ8Krr0KPHmG3RiRcJ554eBZDVVXYLcofKgo5YMECuOsuqKkJgklEBE49FV54IRin07Vr\nMG+StJ+6pGY5ZSKItC6exbBgAXzrW2G3JnzqkprHVqwIMhEWLlRBEGlJPIvhqquUxZAOKgpZavVq\n+O53Yf586N8/7NaIZLdvfQvmzg0uIb35ZtityW0qClkonokwZw5cdFHYrRHJDcpiSA/daM4yiZkI\nl10WdmtEcouyGNpPRSGLbN8e9KRQJoLIkRszBnbvVhbDkVLvoywRz0SoqoJbbw27NSK5b+LEIIch\nalkMmiU1D8QzES68MBi1rCmwRdrPPRjtXFcXrSwGFYUcp0wEkcw5cCC4FLtvX3SyGFQUcpgyEUQy\nr6EBRowILiFFIYtBg9dylDIRRDpGYWEwzXx9PYwbpyyGZFQUQqBMBJGOFc9ieP11deRIRl1SQ6BM\nBJGO1zSLYeLEsFuUnVQUOlg8E6G2Nlrd5ESyQUkJLFsGAwYEWQxjxoTdouyjotCBEjMRSkrCbo1I\nNJWVHcpiKCpSFkNTKgodZP58uPPOYOZTZSKIhEtZDC1Tl9QOsHgx3HijMhFEsk08i+HJJ2HQoLBb\nkx7qkprlVqwIup4qE0Ek+8SzGEaOVBZDnIpCBsUzEaqrlYkgkq2UxXA4FYUMScxEuPjisFsjIq1R\nFsMhutGcAfFMhHvuUSaCSK6oqoI9e5TFoKKQZtu3B/+nmjgRrrkm7NaISFuMHXuoMEQ1i0G9j9Io\nnolQWQm33RZ2a0TkSOVyFkPGex+Z2RAzqzOz9WY2oYVtHoi9/oaZnZtsXzObYmbvmdma2GPIkX6B\nbLFvXzAFdkWF5lYRyXV33w3nnQfDh8Onn4bdmo7V6pmCmXUC3gIGA9uA3wOV7r4uYZsKYLy7V5hZ\nf+B+dy9vbV8zuwPY5+4zWm1cjpwpKBNBJP/kahZDps8U+gEb3P0dd28A5gNNb50OBx4FcPeVQLGZ\nHZ/Cvnnx09nQAFddBV/6UjCFhQqCSH7o1AkefTR4fv31QZGIgmRFoQzYmrD8XmxdKtucmGTfH8Yu\nN801s+I2tTpLKBNBJL8VFgajnevrYfz4aGQxJOt9lOohaOvfxzOBu2LP/xW4F/h+cxtOmTKl8fmg\nQYMYlCVj0eOZCFu2BHOoKBNBJD/Fsxguvji4X3j33WG36HA1NTXU1NSk7f2S3VMoB6a4+5DY8iTg\noLtPTdjmYaDG3efHluuAgcApyfaNre8NLHL3z00Ckc33FCZPDuZmf+mlYG52Eclvf/5z0Lvwuutg\nQrNdbrJDpu8prAb6mllvMzsauApY2GSbhcB1scaUA7vdfWdr+5rZCQn7jwD+eKRfIAzTp8PTTwdn\nCCoIItFQUgLLlwdT4M+aFXZrMqfVy0fuvt/MxgNLgU7A3FjvodGx12e5+/NmVmFmG4C/ADe0tm/s\nraea2TkEl6c2A6Mz8eUyYc4ceOihYMRjjx5ht0ZEOtKJJwaFIZ7FUFkZdovST4PX2mDBAvjJT4KZ\nT/v0Cbs1IhKWP/0pyGL4xS+COc6ySXsvH6kopOj554OeRspEEBHI3iwG5Sl0gNpaGDUKnn1WBUFE\nAvmaxaCikMTrr8OVVwZxmuXlYbdGRLJJPmYxqCi0Yt26Q5kIF10UdmtEJBslZjFs3Bh2a9pPU2e3\nYPNm+Pu/D7qfKhNBRFpTVQV79wa/GbmexaCi0AxlIohIW40ZA7t3B78dK1YE4xpykXofNRHPRKiq\n0hTYItJ2YWcxqEtqGu3bF/Q9HjgQpk7VjKci0nbucNNNwT3JJUuCuZM6kopCmigTQUTS5eDB4NLz\n3r3wzDMdm8WgopAGDQ1Bt9POneGJJzQFtoi0X0MDjBgBXbvCL3/Zcb8rGrzWTspEEJFMKCyEp54K\nOq6MG5c7WQyRLgqJmQhPPaVMBBFJr3gWw+uvw6RJYbcmNZHukjp5MqxcGWQiHHNM2K0RkXxUVBTc\ncB44EIqLg95J2SyyRWHatOAGUG1tON3GRCQ6Skpg2TIYMCAoDGPGhN2ilkWyKMyeDTNnwquv5u4A\nExHJLWVlwSzL8SyGqqqwW9S8yBWF+fPhzjuDEYdlZWG3RkSi5NRTYenSIO+5a9dg3qRsE6kuqYsX\nw403KhNBRMKVySwGdUlN0YoVQdfThQtVEEQkXNmcxRCJorB6NXz3u1BdDf37h90aEZHszWLI+6Kw\ndu2hTISLLw67NSIihyRmMWzaFHZrAnl9o3nz5uBg33OPMhFEJDtVVcGePcGU29mQxZC3RUGZCCKS\nK8aOPVQYamvhuOPCa0te9j6KZyJUVsJtt2WgYSIiGZCOLAbNktqEMhFEJFe5B2cNdXVHnsWgopBA\nmQgikusOHIBrrw3+wP3Nb9qexaCiEKNMBBHJFw0NcMUVwajnxx9v2++ZBq+hTAQRyS+FhcFo5/p6\nGD++Y7MYcr4oxDMRtm5VJoKI5I94FsPq1XDrrR33uTnfJXXyZPjd7+Dll5WJICL5pWkWw4QJmf/M\nnC4K06cHN2KUiSAi+appFsPo0Zn9vJwtCrNnw0MPBSMAe/QIuzUiIplTVgbLlwdnDEVFwRisTMnJ\nopCYidCzZ9itERHJvNNOgxdeCOZw69Ilc1kMOdclVZkIIhJlybIYItUldcUKGDUKnn1WBUFEoinT\nWQw5UxTimQgLFkB5editEREJTyazGJIWBTMbYmZ1ZrbezJrtEGVmD8Ref8PMzk22r5l1N7PlZva2\nmS0zs+KAp3A1AAAEzklEQVTW2pCYiXDRRW35eiIi+SlTWQytFgUz6wQ8CAwBzgQqzeyMJttUAH3c\nvS/wT8DMFPadCCx39y8DL8aWmxXPRJg+PdqZCDU1NWE3ISvoOByiY3FIVI9FVVUwVmvZsvS9Z7Iz\nhX7ABnd/x90bgPlA05/m4cCjAO6+Eig2s+OT7Nu4T+x/L2+pAfFMhGuvbcO3ykNR/T99UzoOh+hY\nHBLlYzFmTPBIl2RFoQzYmrD8XmxdKtuc2Mq+pe6+M/Z8J1DaUgNuvBHGjUvSShERSYtkRSHV/qqp\ndH+y5t4v1ue0xc+ZNCnFFoiISPu5e4sPoBx4IWF5EjChyTYPA1cnLNcR/OXf4r6xbY6PPT8BqGvh\n810PPfTQQ4+2PVr7XU/2SDaieTXQ18x6A/XAVUDTAdYLgfHAfDMrB3a7+04z+7CVfRcC1wNTY//7\n2+Y+vD0DMEREpO1aLQruvt/MxgNLgU7AXHdfZ2ajY6/PcvfnzazCzDYAfwFuaG3f2Fv/HHjSzL4P\nvAOMzMB3ExGRNsrqaS5ERKRjZeWI5lQGzOUrM+tlZi+b2Ztm9icz++fY+jYN+MsnZtbJzNaY2aLY\nciSPhZkVm9mvzWydma01s/4RPhaTYv9G/mhmvzKz/xGVY2Fmj5jZTjP7Y8K6Fr977Fitj/2m/n2y\n98+6opDKgLk81wDc7O5nEdysHxf7/ikP+MtDPwLWEtxEg+gei/uB5939DOBrBB02IncsYvcpfwD8\nnbufTXB5+mqicyzmEfw+Jmr2u5vZmQT3c8+M7fOQmbX6u591RYHUBszlLXff4e7/HXv+MbCOYHxH\nygP+8omZ9QQqgF9wqOtz5I6FmXUDBrj7IxDcs3P3PUTwWAB7Cf54OsbMjgKOIejMEolj4e6vALua\nrG7pu18GVLt7g7u/A2wg+I1tUTYWhVQGzEVC7C+ic4GVtGHAX575v8D/AQ4mrIvisTgF+MDM5pnZ\nf5nZHDP7IhE8Fu7+EXAvsIWgGOx29+VE8FgkaOm7n0jwGxqX9Pc0G4uC7nwDZtYFeBr4kbvvS3wt\n2YC/fGFmlwLvu/saWhggGZVjQdBT8O+Ah9z97wh6+h12eSQqx8LMTgN+DPQm+NHrYmbXJG4TlWPR\nnBS+e6vHJRuLwjagV8JyLw6vdHnPzAoJCsLj7h4fw7EzNqcUZnYC8H5Y7etA/wsYbmabgWrgIjN7\nnGgei/eA99w9PoP+rwmKxI4IHouvA6+5+4fuvh/4DXA+0TwWcS39m2j6e9oztq5F2VgUGgfMmdnR\nBDdJFobcpg5jZgbMBda6+30JL8UH/EErA/7yibvf6u693P0UghuJL7n7tUTzWOwAtprZl2OrBgNv\nAouI2LEguMFebmadY/9eBhN0RIjisYhr6d/EQuBqMzvazE4B+gKrWnujrBynYGZDgfs4NOjt7pCb\n1GHM7AKgFvgDh07zJhH8h3wSOInYgD933x1GG8NgZgOB/+3uw82sOxE8Fmb2PwluuB8NbCQYKNqJ\naB6LWwh+/A4C/wX8I9CVCBwLM6sGBgIlBPcPbgeepYXvbma3AjcC+wkuRy9t9f2zsSiIiEg4svHy\nkYiIhERFQUREGqkoiIhIIxUFERFppKIgIiKNVBRERKSRioKIiDRSURARkUb/HwGucLN2dHFmAAAA\nAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "xs = list(range(0,100))\n", - "\n", - "ps = thinkbayes.MakeUniformPmf(0, 100, 101).Probs(xs)\n", - "plt.plot(xs, ps, label='uniform', color='k')\n", - "\n", - "suite = TrianglePrior(Euro())\n", - "ps = suite.Probs(xs)\n", - "plt.plot(xs, ps, label='triangle');\n", - "plt.legend();" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This shows the result (and the uniform prior for\n", - "comparison). Updating this prior with the same dataset yields the\n", - "posterior distribution shown in Figure [fig.euro3]. Even with\n", - "substantially different priors, the posterior distributions are very\n", - "similar. The medians and the credible intervals are identical; the means\n", - "differ by less than 0.5%." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEACAYAAAC08h1NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQVPWd9/H3dwYQRhhwRAe5yFCCXOIlGGUxajJrTBZR\nISYplUq8ZGujZSSiFaPRUgMp/0lFk2ix65qNEDUG46MxhQrBmDjxcfVBWRGNMMAgd5iB5X4RnGG+\nzx+ne2iameme7tN9uqc/r6ou+ty/fYr5zG9+/TvnmLsjIiKloyzqAkREJL8U/CIiJUbBLyJSYhT8\nIiIlRsEvIlJiFPwiIiUmZfCb2SQzqzez1WZ2TzvLx5jZO2Z2yMx+2M7ycjNbamYvh1W0iIhkrtPg\nN7NyYDYwCRgHTDOzsUmr7QB+ADzcwW5mAMsBXTAgIlIAUrX4JwAN7r7O3ZuB54CpiSu4+3Z3XwI0\nJ29sZkOBycBvAAunZBERyUaq4B8CbEyY3hSbl65fAj8CWrtYl4iI5Eiq4M+4e8bMrgS2uftS1NoX\nESkYPVIs3wwMS5geRtDqT8cXgSlmNhnoDVSa2dPufkPiSmamvn8RkQy4e0aN6lQt/iXAKDOrMbNe\nwLXA/A7WPaYAd7/P3Ye5+wjgOuBvyaGfsK5e7vzkJz+JvIZCeelc6FzoXHT+ykanLX53bzGz6cAi\noBx40t1XmNktseVPmNkg4D2gEmg1sxnAOHffn7y7rCoVEZFQpOrqwd0XAguT5j2R8L6RY7uD2tvH\n34G/Z1ijiIiESFfuFpDa2tqoSygYOhdH6VwcpXMRDsu2ryjrAsw86hpERIqNmeEZfrmbsqtHRKQj\nZhqpnQ9hN44V/CKSFf3Fnlu5+OWqPn4RkRKj4BcRKTEKfhGREqPgFxFJcuutt/LQQw+1TT/++ONU\nV1dTWVnJrl27IqwsHBrOKSIZiw0pjLqMnGpubqZ///68++67nHXWWXk/fkfnOJvhnGrxi4h0orGx\nkUOHDjF2bPIzqFIL4746uaDgF5FuqaysjE8++aRt+qabbuKBBx4AoK6ujqFDh/KLX/yC6upqBg8e\nzG9/+9vj1l29ejVjxowBYMCAAVx22WUAvP3221xwwQUMGDCACRMm8M4777RtW1tby/33389FF11E\n3759+eSTTygrK+Pxxx9n1KhRVFZW8uCDD7JmzRouvPBCBgwYwHXXXUdz83HPssoZjeMXkZJgZseM\niW9qamLv3r1s2bKF1157jW9961tcffXV9O/fv23dUaNGsXz5ckaMGMGePXsoKytj586dXHHFFcye\nPZtp06bx/PPPc8UVV7BmzRpOOukkAH73u9+xcOFCRo8ezZEjRwB47bXXWLp0KRs2bGD8+PG89dZb\nzJs3j6qqKi688ELmzZvHDTe0ewPj0KnFLyI5FQ/RbF5hSex26dmzJw8++CDl5eVcfvnl9O3bl5Ur\nVx63bnJXzauvvsro0aP59re/TVlZGddddx1jxoxh/vz5bZ/3pptuYuzYsZSVldGzZ08A7r77bvr2\n7cu4ceM4++yzufzyy6mpqaGyspLLL7+cpUuXhvY5U1GLX0RyqhD7uAFOPvlkysqOtn0rKirYvz/5\nbvLH27JlC6effvox84YPH86WLVvapocNO/6GxdXV1W3v+/Tpc9x0Y2Njl+rPhlr8ItItVVRUcPDg\nwbbprVu3hvLXw5AhQ1i/fv0x89avX8+QIUcfR17o9zBS8IvkgTtceSW8/nrUlZSOz3/+8zz77LMc\nOXKEP//5z7z55ptpb9vZXymTJ09m1apVzJs3j5aWFv7whz9QX1/PlVdemdb27a2T77+KFPwiefDs\ns7BgQfCS/Hj00Ud5+eWXOemkk/j973/P1Vdffczyzlrlyd8tJL6vqqrilVde4ZFHHmHgwIE8/PDD\nvPLKK1RVVXW47/aOlbz/fP6VoAu4RHJs714YOxZmzIAXX4TFi6OuKDylcAFX1HJxAZeCXyTH7roL\ndu6E2bPhlFNg+3aoqIi6qnAo+HMvF8GvUT0iObR8OTz1FPzjH0HYn302vPcefPnLUVcmpUx9/CI5\n4g633w4PPADxkXsXXQRvvRVtXSIKfpEc2bYN3n8fvv/9o/Muugj++7+jq0kEFPwiOdPQAKNHQ4+E\nDtWLLoJ33oHW1ujqEkkr+M1skpnVm9lqM7unneVjzOwdMztkZj9MmD/MzN4ws4/N7B9mdnuYxYsU\nsoYGOOOMY+dVV8PAgUHfv0hUUga/mZUDs4FJwDhgmpkl3590B/AD4OGk+c3Ane7+OWAicFs724p0\nSw0NMHLk8fPV3SNRS6fFPwFocPd17t4MPAdMTVzB3be7+xKCoE+c3+juH8Te7wdWAINDqVykwHUW\n/PqCV6KUTvAPATYmTG+KzesSM6sBxgPd6PIVkY6tWaMWf6HbsGED/fr1y8u1CMnPB4hSOuP4sz4j\nZtYXeAGYEWv5H2PmzJlt72tra6mtrc32kCKR66jFP2YM7N4NW7fCaaflv65SUlNTw5w5c7j00kvb\nXX766aezb9++PFeVmbq6Ourq6kLZVzrBvxlIvMfoMIJWf1rMrCfwIvA7d/9Te+skBr9Id7BzJxw5\nAieffPyysrKjrf5vfSv/tZWSzq4sbmlpoUeP4rmGNblRPGvWrIz3lU5XzxJglJnVmFkv4Fpgfgfr\nHnP5sAV3HXoSWO7uv8q4SpEiE2/td3TfLXX35N7111/Phg0buOqqq+jXrx8///nPKSsrY86cOQwf\nPpzLLruM9evXU1ZWRmtsfO3cuXMZN24clZWVnHHGGfz6179u21+qxzXu2LGDq666iv79+zNhwgTu\nv/9+LrnkknZrO3z4MHfddRfDhw9n0KBB3HrrrRw6dCin5yNRyuB39xZgOrAIWA78wd1XmNktZnYL\ngJkNMrONwJ3A/Wa2Ida9cxHwHeCfzWxp7DUpZ59GpEB01M0Td845UF+fv3pK0TPPPMPpp5/OK6+8\nwr59+7jmmmsAePPNN6mvr2fRokXH/TVQXV3Nq6++yt69e5k7dy533nnnMU/GSnxc45NPPsltt93G\nnj17ALjtttvo168fTU1NPPXUUzz99NMd3nHzxz/+MQ0NDSxbtoyGhgY2b97MT3/60xydiXbEnwIf\n1SsoQaR7+elP3e+7r+Ply5a5n3VW/urJlXR+foObV2T3ylRNTY3/9a9/dXf3tWvXupn52rVr25bH\n5x05cqTd7b/+9a/7o48+6u7ub7zxhvfp0+eYdU899VRfvHixt7S0eM+ePX3VqlVty+6//36/+OKL\n26bNzNesWeOtra1+4okn+po1a9qWvf322z5ixIh2a+joHMfmZ5S7xdPBJVJEGho6vxHb0KGwKe1v\nyopbod28s73HIsYtXLiQWbNmsXr1alpbWzl48CDnnHNO2/KOHte4fft2Wlpajtn30KFD2z3G9u3b\nOXjwIF/4whfa5rl7W3dTPuiWDSI5kKqr56ST4PBhSOMRr5KFVA9ASXT48GG++c1vcvfdd7Nt2zZ2\n7drF5MmT0xrqecopp9CjRw82bjw68j3xfaKBAwfSp08fli9fzq5du9i1axe7d+9m7969aX6q7Cn4\nRXIgVfCbBa3+zZvzV1Mpqq6uZs2aNWmt+9lnn/HZZ58xcOBAysrKWLhwIa+99lpa25aXl/ONb3yD\nmTNn8umnn1JfX88zzzzT7i+ZsrIyvve973HHHXewfft2ADZv3pz2scKg4BcJ2b59QUs+1Rj9Uuru\nicq9997LQw89RFVVFS+++GKnfwH069ePxx57jGuuuYaqqirmzZvH1KlT2123PbNnz2bPnj0MGjSI\nG2+8kWnTptGrV692t/3Zz37GyJEjmThxIv379+erX/0qq1atyvbjpk1P4BIJ2QcfwPXXw0cfdb7e\nDTfAV74CN96Yn7pyQU/g6tg999zDtm3bmDt3blb7ycUTuNTiFwlZqm6eOLX4u5eVK1fy4Ycf4u68\n++67zJkz57gHvBcKjeoRCVlXgj/VXwVSPPbt28e0adPYsmUL1dXV3HXXXUyZMiXqstql4BcJ2Zo1\ncP75qdcbOhQWLsx9PZIf559/PqtXr466jLSoq0ckZOrqkUKn4BcJWXtP3mqPgl+iolE9IiH69NPg\n4qwDB6C8vPN1W1uhoiK4RXPv3vmpL2wa1ZN7uRjVoz5+kRCtXQs1NalDH4LbMw8eHFzElc5fCIWq\ns7HtUpgU/CIhSrd/Py7e3VOswa/WfnFSH79IiDZsgOHD019f/fwSBQW/SIgaG7v2OEUFv0RBwS8S\noq1bYdCg9NdX8EsUFPwiIWpsVPBL4VPwi4RIwS/FQMEvEiIFvxQDXcAlEpLW1uBCrP37IeE27J06\ncgT69OnaNiKg2zKLFIQdO6CysmsBXl4e/IWwdWvu6hJJpuAXCUlXu3ni1N0j+abgFwmJgl+KRcrg\nN7NJZlZvZqvN7J52lo8xs3fM7JCZ/bAr24p0Jwp+KRadBr+ZlQOzgUnAOGCamY1NWm0H8APg4Qy2\nFek2FPxSLFK1+CcADe6+zt2bgeeAYx477+7b3X0J0NzVbUW6EwW/FItUwT8E2JgwvSk2Lx3ZbCtS\ndBT8UixS3ZY5mwH2aW87c+bMtve1tbXU1tZmcViRaCj4JZfq6uqoq6sLZV+pgn8zMCxhehhByz0d\naW+bGPwixSrT4D/tNGhqCi7mSucBLlKakhvFs2bNynhfqbp6lgCjzKzGzHoB1wLzO1g3+Qqyrmwr\nUvQyDf6ePaF//+ACMJF86LTF7+4tZjYdWASUA0+6+wozuyW2/AkzGwS8B1QCrWY2Axjn7vvb2zaX\nH0YkKocPw759UFWV2fanngrbtgX/iuRaykcvuvtCYGHSvCcS3jdybJdOp9uKdEfx0C7L8JLIePCL\n5IOu3BUJQVcfwJJMwS/5pOAXCUFXH7mYTMEv+aTgFwlBpl/sxin4JZ8U/CIhUPBLMVHwi4RAwS/F\nRMEvEgIFvxQTBb9ICMII/qam8OoR6YyCXyQEavFLMVHwi2TJPQj+6urM91FZCc3NcPBgeHWJdETB\nL5KlffuCm6v17Zv5PsyCVv/27eHVJdIRBb9IlrLt5olTd4/ki4JfJEsKfik2Cn6RLCn4pdgo+EWy\npOCXYqPgF8mSgl+KjYJfJEsKfik2Cn6RLGU7hj9OwS/5ouAXyVJTk1r8UlwU/CJZCutZuQp+yRdz\n92gLMPOoaxDJlDv07g27d0OfPtnt6/Bh6Ncv+NcsnPqk+zIz3D2j/ylq8YtkYe9eOOGE7EMfgv1U\nVMCuXdnvS6QzCn6RLITVzRNXXa3uHsk9Bb9IFsIOfvXzSz6kDH4zm2Rm9Wa22szu6WCdx2LLl5nZ\n+IT595rZx2b2kZn93sxOCLN4kag1NSn4pfh0GvxmVg7MBiYB44BpZjY2aZ3JwEh3HwXcDDwem18D\nfA84z93PBsqB60KuXyRS27aFM4Y/TsEv+ZCqxT8BaHD3de7eDDwHTE1aZwrwFIC7LwYGmFk1sBdo\nBirMrAdQAWwOs3iRqKmrR4pRquAfAmxMmN4Um5dyHXffCTwCbAC2ALvd/fXsyhUpLAp+KUY9UixP\nd4D9cWNJzewM4A6gBtgD/B8z+7a7P5u87syZM9ve19bWUltbm+ZhRaK1bRtcckl4+zv1VHjjjfD2\nJ91HXV0ddXV1oewrVfBvBoYlTA8jaNF3ts7Q2Lxa4G133wFgZn8Evgh0GvwixUQtfsmX5EbxrFmz\nMt5Xqq6eJcAoM6sxs17AtcD8pHXmAzcAmNlEgi6dJmAlMNHM+piZAZcByzOuVKQAKfilGHXa4nf3\nFjObDiwiGJXzpLuvMLNbYsufcPcFZjbZzBqAA8B3Y8s+MLOnCX55tALvA7/O4WcRyTsFvxQj3atH\nJEMtLcGtGg4dgvLycPbZ2hrcuuHAAejVK5x9Sveke/WIROB//xeqqsILfYCyMhg4MNi3SK4o+EUy\nFPZVu3Hq7pFcU/CLZCjsq3bjqquDXyoiuaLgF8lQ2F/sxp16qoJfckvBL5KhXAb/9u3h71ckTsEv\nkqFcBr/6+CWXFPwiGcpV8KuPX3JNwS+SIbX4pVgp+EUypOCXYqXgF8mQunqkWCn4RTKUq+A/5ZRg\n37qTieSKgl8kA/v3B8F84onh77tPn+B+PXv3hr9vEVDwi2QkftWuZXSLrNR0EZfkkoJfJAO56uaJ\nq67WF7ySOwp+kQzkOvg1skdyScEvkoF8BL+6eiRXFPwiGVBXjxQzBb9IBtTVI8VMwS+SAXX1SDFT\n8ItkQF09UswU/CIZUFePFDMFv0gGcvW83Th19UgumUd8QxAz86hrEOmKI0egd2/49FPo0SM3x2ht\nDW7bcOAA9OqVm2NIcTMz3D2ja8dTtvjNbJKZ1ZvZajO7p4N1HostX2Zm4xPmDzCzF8xshZktN7OJ\nmRQpUkh27oTKytyFPkBZWXCzNj2CUXKh0+A3s3JgNjAJGAdMM7OxSetMBka6+yjgZuDxhMWPAgvc\nfSxwDrAixNpFIrF1K5x2Wu6Po+4eyZVULf4JQIO7r3P3ZuA5YGrSOlOApwDcfTEwwMyqzaw/cIm7\nz4kta3H3PeGWL5J/jY35CX6N7JFcSRX8Q4CNCdObYvNSrTMUGAFsN7O5Zva+mf2XmVVkW7BI1LZu\nhUGDcn8ctfglV1L1Uqb7rWvyFwwe2/d5wHR3f8/MfgX8GHgweeOZM2e2va+traW2tjbNw4rkXz67\netTil7i6ujrq6upC2Veq4N8MDEuYHkbQou9snaGxeQZscvf3YvNfIAj+4yQGv0ih27oVampyfxx1\n9Uii5EbxrFmzMt5Xqq6eJcAoM6sxs17AtcD8pHXmAzcAxEbt7Hb3JndvBDaa2Zmx9S4DPs64UpEC\n0diorh4pbp22+N29xcymA4uAcuBJd19hZrfElj/h7gvMbLKZNQAHgO8m7OIHwLOxXxprkpaJFCV1\n9UixSzkS2d0XAguT5j2RND29g22XARdkU6BIoclX8KurR3JFt2wQ6SKN6pFip+AX6YL9+4PbKVRW\n5v5Y8St3dUcTCZuCX6QL4t08ltEdUrqmd2+oqIDdu3N/LCktCn6RLshX/36cunskFxT8Il2Qr6Gc\ncRrZI7mg4Bfpgny3+DWyR3JBwS/SBerqke5AwS/SBfkayhmnrh7JBQW/SBfk65bMcerqkVxQ8It0\ngbp6pDtQ8It0Qb6D/7TTYMuW/B1PSoOCXyRNzc3BxVQDB+bvmEOGwObN+TuelAYFv0iampqC2yiU\nl+fvmIMHB8c9ciR/x5TuT8EvkqZ8d/MA9OoFAwboC14Jl4JfJE35HsoZp+4eCZuCXyRN+R7KGTd0\nqIJfwqXgF0lTFF09oBa/hE/BL5ImdfVId6HgF0lTVF09Cn4Jm4JfJE3q6pHuQsEvkiYFv3QX5hE/\n0NPMPOoaRFJxhxNOgL17g0ci5tOuXVBTA3v25Pe4UtjMDHfP6CGgavGLpGHnTjjxxPyHPgQXcDU3\nw759+T+2dE8pg9/MJplZvZmtNrN7OljnsdjyZWY2PmlZuZktNbOXwypaJN+i6uaB4MHu6u6RMHUa\n/GZWDswGJgHjgGlmNjZpncnASHcfBdwMPJ60mxnAckD9OVK0ohrKGafglzClavFPABrcfZ27NwPP\nAVOT1pkCPAXg7ouBAWZWDWBmQ4HJwG+AjPqiRApBlC1+UPBLuFIF/xBgY8L0pti8dNf5JfAjoDWL\nGkUit3lzcKfMqCj4JUw9UixPt3smuTVvZnYlsM3dl5pZbWcbz5w5s+19bW0ttbWdri6Sd+vWwbnn\nRnf8IUNg9eroji/Rq6uro66uLpR9pQr+zcCwhOlhBC36ztYZGpv3TWBK7DuA3kClmT3t7jckHyQx\n+EUK0bp1MDW5kzOPhgyBkH7mpUglN4pnzZqV8b5SdfUsAUaZWY2Z9QKuBeYnrTMfuAHAzCYCu929\n0d3vc/dh7j4CuA74W3uhL1IM1q2DESOiO77u0Clh6rTF7+4tZjYdWASUA0+6+wozuyW2/Al3X2Bm\nk82sATgAfLej3YVZuEi+tLbC+vUwfHh0NaiPX8KkK3dFUtiyBcaPDx6BGJXm5uACsoMHoUeqDlop\nCbpyVySH1q0LbpkQpZ494eSTo/3lI92Hgl8khaj79+PU3SNhUfCLpLB2bfQtflDwS3gU/CIpFEJX\nDyj4JTwKfpEU1NUj3Y2CXyQFtfilu1Hwi3TiyBHYsAFOPz3qSoLg35R83bxIBhT8Ip3YuhWqqqBP\nn6grUYtfwqPgF+lEofTvw9Hg1/WOki0Fv0gnCqV/H6CyEsrKYPfuqCuRYqfgF+lEoYzhh+ARjCNH\nQkND1JVIsVPwi3SikFr8AKNHw6pVUVchxU7BL9KJQurjBzjzTFi5MuoqpNgp+EU6UWgt/jPPVItf\nsqfgF+nAkSPBuPlCGMMfp+CXMCj4RTqweTOccgqccELUlRw1alQQ/BrSKdlQ8It0oNC6eQAGDAge\nyLJ1a9SVSDFT8It0oBCDH4KRPfqCV7Kh4BfpQCGN4U+kfn7JloJfpAOFNpQzTsEv2VLwi3Rg7VoY\nPjzqKo6n4JdsKfhF2uEOH38M48ZFXcnxFPySLQW/SDuamoLwP+20qCs53hlnwPr10NwcdSVSrNIK\nfjObZGb1ZrbazO7pYJ3HYsuXmdn42LxhZvaGmX1sZv8ws9vDLF4kV5Ytg3PPDW6MVmhOOCG4RfPa\ntVFXIsUqZfCbWTkwG5gEjAOmmdnYpHUmAyPdfRRwM/B4bFEzcKe7fw6YCNyWvK1IIfrwQzjnnKir\n6Ji6eyQb6bT4JwAN7r7O3ZuB54CpSetMAZ4CcPfFwAAzq3b3Rnf/IDZ/P7ACGBxa9SI5ouCX7iyd\n4B8CbEyY3hSbl2qdoYkrmFkNMB5Y3NUiRfJNwS/dWY801kn3riDJvaFt25lZX+AFYEas5X+MmTNn\ntr2vra2ltrY2zUOKhO+zz4JQLcQRPXGjR8OLL0ZdheRTXV0ddXV1oezLPMXdnsxsIjDT3SfFpu8F\nWt39Zwnr/CdQ5+7PxabrgS+7e5OZ9QReARa6+6/a2b+nqkEknz76CK65BlasiLqSjm3YABdeqIev\nlzIzw90zGn6QTlfPEmCUmdWYWS/gWmB+0jrzgRtixUwEdsdC34AngeXthb5IIYqP6ClkQ4fCrl2w\n/7i/n0VSSxn87t4CTAcWAcuBP7j7CjO7xcxuia2zAPjEzBqAJ4Dvxza/CPgO8M9mtjT2mpSLDyIS\nlkLv34fgoesjR8Lq1VFXIsUonT5+3H0hsDBp3hNJ09Pb2e4tdJGYFJkPP4Tpx/1vLjzxxzCOHx91\nJVJsFMoiSYqhxQ9w3nnw3ntRVyHFSMEvkmD7djh4EIYNi7qS1L70JXjzzairkGKk4BdJ8NFHQWu/\nEG/VkOyCC4KRR3v3Rl2JFBsFv0iCDz8s/BE9cSecAOefD2+/HXUlUmwU/CIJli0rjv79OHX3SCYU\n/CIJiuWL3TgFv2Qi5ZW7OS9AV+5KgWhpgcpK2LYN+vaNupr0HDgA1dXBl9J9+kRdjeRTrq/cFSkJ\nH3wQPGqxWEIf4MQT4eyzYbFufShdoOAXiXnpJZiafMPxIqDuHukqBb9IzEsvwTe+EXUVXafgl65S\nH78IwXj4r30teJZtWZE1h3bvDi4427EDevWKuhrJl2z6+Asi+CMtQASAe4FBwIyoC8nQ+wT3Rvx/\nURciORbP7KIP/qhrEDn/fHj4YSjWZwDNmAGDB8M990RdieSLRvWIZGH9+uDBJhdfHHUlmfvSl+D1\n16OuQoqFgl9K3ksvwZQp0COtm5QXpsmTg4vP6uujrkSKgYJfSt4f/1ico3kS9ekDt94Kv/xl1JVI\nMVAfv5S0piYYMwYaG4ObnhWzbduCh7CvXAmnnhp1NZJr6uMXydDzz8OkScUf+hCE/TXXwL//e9SV\nSKFTi19K1rZtwe0OFi4MnmbVHaxcCZdcAuvWQUVF1NVILqnFL5KBO+6Am27qPqEPQVfPhRfC009H\nXYkUMrX4pSQtWAC33x6MhOluLeM334R/+zdYvry4RypJ53QBl0gX7NsHZ50Fc+bAV74SdTXhc4er\nroKqKvjtb4vvFhSSnpx29ZjZJDOrN7PVZtbudYFm9lhs+TIzG9+VbUXyqbUVfvhDuPTS7hn6EDwv\n+PnngwvTbr01+EUgkqjT4DezcmA2MAkYB0wzs7FJ60wGRrr7KOBm4PF0t5Vj1dXVRV1CwcjFudi4\nEb76Vfj4Y3jkkdB3nzOZnIuKCnjlleBRknfe2X3CXz8j4UjV4p8ANLj7OndvBp4Dku9YPgV4CsDd\nFwMDzGxQmttKAv2nPirMc+EO8+bBF74QtPT//vegG6RYZHou+vWDP/856POfMgXeeKP4fwHoZyQc\nqb76GQJsTJjeBPxTGusMAQansa1I6A4ehK1b4b33guBbtCgY475gQXAztlIyYAC89RY88wxMnx70\n9//rv8K55wYjgAYPDrqGpLSkCv502wdZ/de56qpstu4+Vq6E//mfqKsoDCtXwpIlwft4K9X96Ku1\nNXi1tMChQ8Hr4MFgbP6hQ0GgnXVWcHHWAw/AGWdE91miVlEBt9wCN98Mf/tb0P//pz8F53j//uCv\nn379gkdO9u4N5eVHX2bHvhJF8Quj1H9GZs0KZ/hxp6N6zGwiMNPdJ8Wm7wVa3f1nCev8J1Dn7s/F\npuuBLwMjUm0bm1/kf3yKiEQj01E9qVr8S4BRZlYDbAGuBaYlrTMfmA48F/tFsdvdm8xsRxrbZly4\niIhkptPgd/cWM5sOLALKgSfdfYWZ3RJb/oS7LzCzyWbWABwAvtvZtrn8MCIiklrkF3CJiEh+RXpN\nXylf4GVmw8zsDTP72Mz+YWa3x+ZXmdlfzGyVmb1mZgOirjUfzKzczJaa2cux6ZI8DwBmNsDMXjCz\nFWa23Mz+qRTPh5ndG/v5+MjMfm9mJ5TKeTCzOWbWZGYfJczr8LPHztXqWJ5+LdX+Iwt+XeBFM3Cn\nu38OmAjcFvv8Pwb+4u5nAn+NTZeCGcByjo4kK9XzAPAosMDdxwLnAPWU2PmIfTf4PeA8dz+boLv4\nOkrnPMx3MyMeAAAClElEQVQlyMZE7X52MxtH8B3quNg2/2FmnWZ7lC3+kr7Ay90b3f2D2Pv9wAqC\n6x/aLoiL/fv1aCrMHzMbCkwGfsPRocEldx4AzKw/cIm7z4HguzJ330PpnY+9BI2jCjPrAVQQDBIp\nifPg7v8X2JU0u6PPPhWY5+7N7r4OaCDI1w5FGfwdXfhVcmKtm/HAYqDa3Ztii5qA6ojKyqdfAj8C\nWhPmleJ5gGAY9HYzm2tm75vZf5nZiZTY+XD3ncAjwAaCwN/t7n+hxM5Dko4++2CC/IxLmaVRBr++\nVQbMrC/wIjDD3fclLovdtrRbnyczuxLY5u5L6eBCwFI4Dwl6AOcB/+Hu5xGMlDumO6MUzoeZnQHc\nAdQQBFtfM/tO4jqlcB46ksZn7/S8RBn8m4FhCdPDOPa3VrdnZj0JQv8Zd/9TbHZT7F5HmNlpwLao\n6suTLwJTzGwtMA+41MyeofTOQ9wmYJO7vxebfoHgF0FjiZ2P84G33X2Hu7cAfwQupPTOQ6KOfiaS\ns3RobF6Hogz+tovDzKwXwZcT8yOsJ6/MzIAngeXu/quERfOBG2PvbwT+lLxtd+Lu97n7MHcfQfDl\n3d/c/XpK7DzEuXsjsNHMzozNugz4GHiZ0jof9cBEM+sT+1m5jODL/1I7D4k6+pmYD1xnZr3MbAQw\nCni30z25e2Qv4HJgJcGXEfdGWUsEn/1igj7tD4ClsdckoAp4HVgFvAYMiLrWPJ6TLwPzY+9L+Tyc\nC7wHLCNo6fYvxfMB3E3wS+8jgi8ze5bKeSD463cL8BnBd6Hf7eyzA/fFcrQe+JdU+9cFXCIiJUYP\nZRMRKTEKfhGREqPgFxEpMQp+EZESo+AXESkxCn4RkRKj4BcRKTEKfhGREvP/AfApOG5pWjCyAAAA\nAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "for data in dataset:\n", - " suite.Update(data)\n", - " \n", - "ps = thinkbayes.MakeUniformPmf(0, 100, 101).Probs(xs)\n", - "plt.plot(xs, ps, label='uniform', color='k')\n", - "\n", - "ps = suite.Probs(xs)\n", - "plt.plot(xs, ps, label='triangle');\n", - "plt.legend();" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This is an example of **swamping the priors** with enough\n", - "data, people who start with different priors will tend to converge on\n", - "the same posterior." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Optimization\n", - "\n", - "The code I have shown so far is meant to be easy to read, but it is not\n", - "very efficient. In general, I like to develop code that is demonstrably\n", - "correct, then check whether it is fast enough for my purposes. If so,\n", - "there is no need to optimize. For this example, if we care about run\n", - "time, there are several ways we can speed it up.\n", - "\n", - "The first opportunity is to reduce the number of times we normalize the\n", - "suite. In the original code, we call `Update` once for each spin.\n", - "\n", - "```python\n", - " dataset = 'H' * heads + 'T' * tails\n", - "\n", - " for data in dataset:\n", - " suite.Update(data)\n", - "```\n", - "\n", - "And here’s what `Update` looks like:\n", - "\n", - "```python\n", - " def Update(self, data):\n", - " for hypo in self.Values():\n", - " like = self.Likelihood(data, hypo)\n", - " self.Mult(hypo, like)\n", - " return self.Normalize()\n", - "```\n", - "\n", - "Each update iterates through the hypotheses, then calls `Normalize`,\n", - "which iterates through the hypotheses again. We can save some time by\n", - "doing all of the updates before normalizing.\n", - "\n", - "`Suite` provides a method called `UpdateSet` that does exactly that.\n", - "Here it is:\n", - "\n", - "```\n", - " def UpdateSet(self, dataset):\n", - " for data in dataset:\n", - " for hypo in self.Values():\n", - " like = self.Likelihood(data, hypo)\n", - " self.Mult(hypo, like)\n", - " return self.Normalize()\n", - "```\n", - "\n", - "And here’s how we can invoke it:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "2.38031546401283e-75" - ] - }, - "execution_count": 11, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "heads, tails = 140, 110\n", - "\n", - "dataset = 'H' * heads + 'T' * tails\n", - "suite.UpdateSet(dataset)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This optimization speeds things up, but the run time is still\n", - "proportional to the amount of data. We can speed things up even more by\n", - "rewriting `Likelihood` to process the entire dataset, rather than one\n", - "spin at a time.\n", - "\n", - "In the original version, `data` is a string that encodes either heads or\n", - "tails:\n", - "\n", - "```python\n", - "def Likelihood(self, data, hypo):\n", - " x = hypo / 100.0\n", - " if data == 'H':\n", - " return x\n", - " else:\n", - " return 1-x\n", - "```\n", - "\n", - "As an alternative, we could encode the dataset as a tuple of two\n", - "integers: the number of heads and tails. In that case `Likelihood` looks\n", - "like this:\n", - "\n", - "```python\n", - "def Likelihood(self, data, hypo):\n", - " x = hypo / 100.0\n", - " heads, tails = data\n", - " like = x**heads * (1-x)**tails\n", - " return like\n", - "```\n", - "\n", - "And then we can call `Update` like this:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.4413448096175153" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "heads, tails = 140, 110\n", - "suite.Update((heads, tails))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Since we have replaced repeated multiplication with exponentiation, this\n", - "version takes the same time for any number of spins." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## The beta distribution\n", - "\n", - "There is one more optimization that solves this problem even faster.\n", - "\n", - "So far we have used a Pmf object to represent a discrete set of values\n", - "for $x$. Now we will use a continuous distribution,\n", - "specifically the beta distribution (see\n", - ").\n", - "\n", - "The beta distribution is defined on the interval from 0 to 1 (including\n", - "both), so it is a natural choice for describing proportions and\n", - "probabilities. But wait, it gets better.\n", - "\n", - "It turns out that if you do a Bayesian update with a binomial likelihood\n", - "function, as we did in the previous section, the beta distribution is a\n", - "**conjugate prior**. That means that if the prior\n", - "distribution for x is a beta distribution, the posterior is\n", - "also a beta distribution. But wait, it gets even better.\n", - "\n", - "The shape of the beta distribution depends on two parameters, written\n", - "$\\alpha$ and $\\beta$, or alpha and beta. If\n", - "the prior is a beta distribution with parameters alpha and\n", - "beta, and we see data with h heads and\n", - "t tails, the posterior is a beta distribution with\n", - "parameters alpha+h and beta+t. In other words,\n", - "we can do an update with two additions.\n", - "\n", - "So that’s great, but it only works if we can find a beta distribution\n", - "that is a good choice for a prior. Fortunately, for many realistic\n", - "priors there is a beta distribution that is at least a good\n", - "approximation, and for a uniform prior there is a perfect match. The\n", - "beta distribution with alpha=1 and beta=1 is\n", - "uniform from 0 to 1.\n", - "\n", - "Let’s see how we can take advantage of all this.\n", - "thinkbayes.py provides a class that represents a beta\n", - "distribution:\n", - "\n", - "```python\n", - "class Beta(object):\n", - "\n", - " def __init__(self, alpha=1, beta=1):\n", - " self.alpha = alpha\n", - " self.beta = beta\n", - "```\n", - "\n", - "By default `__init__` makes a uniform distribution. Update\n", - "performs a Bayesian update:\n", - "\n", - "```python\n", - "def Update(self, data):\n", - " heads, tails = data\n", - " self.alpha += heads\n", - " self.beta += tails\n", - "```\n", - "\n", - "`data` is a pair of integers representing the number of heads\n", - "and tails.\n", - "\n", - "So we have yet another way to solve the Euro problem:" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.5595238095238095\n" - ] - } - ], - "source": [ - "beta = thinkbayes.Beta()\n", - "beta.Update((140, 110))\n", - "print(beta.Mean())" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Beta` provides `Mean,` which computes a simple\n", - "function of `alpha` and `beta`:\n", - "\n", - "```python\n", - "def Mean(self):\n", - " return float(self.alpha) / (self.alpha + self.beta)\n", - "```\n", - "\n", - "For the Euro problem the posterior mean is 56%, which is the same result\n", - "we got using Pmfs.\n", - "\n", - "`Beta` also provides `EvalPdf`, which evaluates\n", - "the probability density function (PDF) of the beta distribution:\n", - "\n", - "```python\n", - "def EvalPdf(self, x):\n", - " return x**(self.alpha-1) * (1-x)**(self.beta-1)\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "111" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "beta.beta" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "3.192662744667652e-75" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "beta.EvalPdf(.55)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, `Beta` provides `MakePmf`, which uses\n", - "`EvalPdf` to generate a discrete approximation of the beta\n", - "distribution." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG+NJREFUeJzt3XmwnXWd5/H3h5uEsJpAQkIWE5DYLILCaMi44LGGrrpG\nJVVD0yytFGIL5XSQsSmHhppqLn/M2HZpgVRajJqmaHUmAr1FmkjLctR2MCQCESGhEzSQQBZIWNNg\ntu/88ZxLDie5Z32Wc+75vKpucZ/1fO9TuZ/74/tsigjMzKw/HFJ0AWZmlh+HvplZH3Hom5n1EYe+\nmVkfceibmfURh76ZWR9pGPqSBiWtlbRO0rUHWX6ypIckvSnpmoMsH5D0qKQfpVW0mZm1p27oSxoA\nFgGDwKnAxZJOqVltO3AV8LURdnM18CTgGwLMzArWaKQ/F1gfERsiYjewFFhQvUJEvBARq4DdtRtL\nmgHMB74LKJ2SzcysXY1CfzqwsWp6U2Ves24Cvgzsa7EuMzPLQKPQb7slI+mTwLaIeBSP8s3MusKY\nBsufA2ZWTc8kGe0344PAeZLmA+OBoyX9XURcWr2SJPf6zczaEBEtD6gbjfRXAXMkzZY0DrgQWDbC\num/78Ii4PiJmRsQJwEXAA7WBX7WuvyK44YYbCq+hW758LHwsfCzqf7Wr7kg/IvZIWgjcCwwASyJi\njaQrK8sXS5oKrASOBvZJuho4NSJer91d21WamVkqGrV3iIjlwPKaeYurvt/C21tAB9vHT4Gftlmj\nmZmlxHfkdpFSqVR0CV3Dx2I/H4v9fCw6p056Q6kUIEXRNZiZ9RpJRAYncs3MbBRx6JuZ9RGHvplZ\nH3Hom5n1EYe+mVkfceibmfURh76ZWR9x6JuZ9RGHvplZH3Hom5n1EYe+mVkfceibmfURh76ZWR9x\n6Jvl5M//HH72s6KrsH7X8CUqZta5116Db34Ttm2Dc84puhrrZx7pm+Xg7rvh9NPhX/4F3nyz6Gqs\nnzn0zXJwxx2wcCGccQb8678WXY31M785yyxjr74KM2bAs8/C978PK1bA975XdFXW6/zmLLMudffd\nSR9/wgQ4//xk+ve/L7oq61cOfbOM3XEHXHBB8v3xxye9/Z/8pNiarH859M0y9Oqr8OCDsGDB/nkX\nXAB33llcTdbfmgp9SYOS1kpaJ+nagyw/WdJDkt6UdE3V/JmSHpT0hKTfSPpimsWbdbsf/Qg+8pGk\ntTPs/POT+W7xWBEahr6kAWARMAicClws6ZSa1bYDVwFfq5m/G/hSRJwGzAP+7CDbmo1ad94Jf/zH\nb583bRqcdhrcd18xNVl/a2akPxdYHxEbImI3sBRYUL1CRLwQEatIQr56/paIeKzy/evAGmBaKpWb\ndbk330yC/bzzDlz2R38E//RP+ddk1kzoTwc2Vk1vqsxriaTZwJnAila3NetFzz4LU6e+vbUz7H3v\ng7Vr86/JrJnHMHR8Eb2kI4G7gKsrI/63GRoaeuv7UqlEqVTq9CPNCrdhA8yadfBls2bBM8/kWo71\nuHK5TLlc7ng/DW/OkjQPGIqIwcr0dcC+iPjqQda9AXg9Ir5eNW8scDewPCJuPsg2vjnLRqVvfzu5\nEWvJkgOX7d4NRxwBO3fC2LH512a9L8ubs1YBcyTNljQOuBBYNlIdNUUJWAI8ebDANxvNnnlm5JH+\n2LHJNfubNuVbk1nD0I+IPcBC4F7gSeCHEbFG0pWSrgSQNFXSRuBLwP+U9GylpfMh4NPAxyQ9Wvka\nzOynMesiGzbA7NkjL3eLx4rQ1KOVI2I5sLxm3uKq77cAMw+y6b/hG8CsTz3zjEPfuo8D2Swj9U7k\ngkPfiuHQN8vArl3JC1Om17m4edas5A+DWZ4c+mYZ2LgxCfwxdRqoHulbERz6Zhlo1NqBpN/v0Le8\nOfTNMtDoJC7AO9+ZXLK5b18uJZkBDn2zTDS6XBNg/PjkEQ1btuRRkVnCoW+WgWbaO+CTuZY/h75Z\nBppp74BP5lr+HPpmGWimvQMOfcufQ98sZXv2JH36GTMar+sreCxvDn2zlG3aBFOmNPf0TI/0LW8O\nfbOUNdvaAZ/Itfw59M1SVu+RyrWGR/p+pYTlxaFvlrJWRvpHHw3jxsH27VlWZLafQ98sZa2M9MEn\ncy1fDn2zlLUy0gf39S1fDn2zlLUT+h7pW14c+mYp2rsXnnsOZh7sPXIjcOhbnhz6Zil6/nmYNAkO\nPbT5bRz6lieHvlmKmn3mTjWHvuXJoW+Womafrllt9myfyLX8OPTNUrR5M0yb1to2xx4Lb74JO3dm\nU5NZNYe+WYq2bk2eu9MKCY47LnmRulnWGoa+pEFJayWtk3TtQZafLOkhSW9KuqaVbc1Gm3ZCH5Jt\ntm5Nvx6zWnVDX9IAsAgYBE4FLpZ0Ss1q24GrgK+1sa3ZqLJli0Pfulujkf5cYH1EbIiI3cBSYEH1\nChHxQkSsAna3uq3ZaLN1K0yd2vp2Dn3LS6PQnw5srJreVJnXjE62NetJ7bZ3jjvOoW/5GNNgeScP\nfG1626Ghobe+L5VKlEqlDj7WrBh798KOHcnNWa2aMgXWr0+/Jhs9yuUy5XK54/00Cv3ngOobymeS\njNib0fS21aFv1qteeAEmToQxjX6rDmLKFPjFL9KvyUaP2gHxjTfe2NZ+GrV3VgFzJM2WNA64EFg2\nwrrqYFuzntduPx/c07f81B2TRMQeSQuBe4EBYElErJF0ZWX5YklTgZXA0cA+SVcDp0bE6wfbNssf\nxqxI7fbzwaFv+Wn4P6IRsRxYXjNvcdX3W3h7G6futmajVSeh75uzLC++I9csJe1eow9wzDHw2muw\na1e6NZnVcuibpaSTnv4hh8DkyR7tW/Yc+mYp6aS9A+7rWz4c+mYpcehbL3Dom6Wkk54++GSu5cOh\nb5YSj/StFzj0zVIw/AiGyZPb34dD3/Lg0DdLQSePYBjm0Lc8OPTNUtBpawcc+pYPh75ZCtIIfZ/I\ntTw49M1S0MmNWcM80rc8OPTNUtDp5ZqQnATesSM5KWyWFYe+WQrSaO+MGQMTJsCLL6ZTk9nBOPTN\nUpBG6EOyD/f1LUsOfbMUpNHTB78r17Ln0DdLQRo9ffDJXMueQ98sBWm2dxz6liWHvlmH0ngEwzCH\nvmXNoW/WoRdf7PwRDMN8Itey5tA361Ba/XzwiVzLnkPfrENp9fPB7R3LnkPfrEMOfeslDn2zDqV1\njT7sf+haRDr7M6vVMPQlDUpaK2mdpGtHWOeWyvLVks6smn+dpCckPS7p/0g6NM3izbpBmj398ePh\nsMPg5ZfT2Z9ZrbqhL2kAWAQMAqcCF0s6pWad+cBJETEHuAK4tTJ/NvB54KyIOB0YAC5KuX6zwqXZ\n3gG3eCxbjUb6c4H1EbEhInYDS4EFNeucB9wOEBErgAmSpgCvAruBwyWNAQ4HnkuzeLNu4NC3XtIo\n9KcDG6umN1XmNVwnInYAXweeBZ4HXo6I+zor16z7bNvm0Lfe0eh2kmZPJ+mAGdK7gP8OzAZeAe6U\n9CcR8YPadYeGht76vlQqUSqVmvxYs+Jt3ZqcgE3LlCnJO3fNqpXLZcrlcsf7aRT6zwEzq6Znkozk\n660zozKvBPy/iNgOIOkfgA8CdUPfrJfs25fckZvGIxiGTZ7su3LtQLUD4htvvLGt/TRq76wC5kia\nLWkccCGwrGadZcClAJLmkbRxtgJPAfMkHSZJwLnAk21VadalduyAo46CcePS26fflWtZqjvSj4g9\nkhYC95JcfbMkItZIurKyfHFE3CNpvqT1wE7gs5Vlj0n6O5I/HPuAR4BvZ/izmOUu7X4+OPQtWw0f\nERURy4HlNfMW10wvHGHbvwb+upMCzbrZtm3p9vPBoW/Z8h25Zh1I+yQuOPQtWw59sw64vWO9xqFv\n1oEsRvoTJsDrr8OuXenu1wwc+mYdyWKkf8ghyWWbvlbfsuDQN+tAFiN9cIvHsuPQN+tAFiN9cOhb\ndhz6Zh3I4pJNcOhbdhz6Zh1we8d6jUPfrE07d8LevcljGNLm0LesOPTN2jTc2tEBz5jtnEPfsuLQ\nN2tTVidxwaFv2XHom7Upq5O44NC37Dj0zdqU1UlccOhbdhz6Zm3Ksr0z/CKVaPbddWZNcuibtSnL\nkf4RR8DAQPIMHrM0OfTN2pTlSB/c4rFsOPTN2pTlSB8c+pYNh75ZmzzSt17k0DdrU5aXbIJD37Lh\n0Ddrw5498NJLcOyx2X2GQ9+y4NA3a8P27TBxIowZk91nOPQtCw59szZkfRIXHPqWDYe+WRuyPokL\nDn3LRsPQlzQoaa2kdZKuHWGdWyrLV0s6s2r+BEl3SVoj6UlJ89Is3qwoHulbr6ob+pIGgEXAIHAq\ncLGkU2rWmQ+cFBFzgCuAW6sWfwO4JyJOAc4A1qRYu1lhsr5yBxz6lo1GI/25wPqI2BARu4GlwIKa\ndc4DbgeIiBXABElTJL0D+EhE/G1l2Z6IeCXd8s2KkUd7Z9Ik2LEjeVGLWVoahf50YGPV9KbKvEbr\nzABOAF6QdJukRyR9R9LhnRZs1g3yaO+MGQPveEcS/GZpaXTBWbPP+Kt9d1BU9n0WsDAiVkq6GfgL\n4C9rNx4aGnrr+1KpRKlUavJjzYqRx0gf9rd4Jk/O/rOsu5XLZcrlcsf7aRT6zwEzq6Znkozk660z\nozJPwKaIWFmZfxdJ6B+gOvTNekEeI33YH/qnnZb9Z1l3qx0Q33jjjW3tp1F7ZxUwR9JsSeOAC4Fl\nNessAy4FqFyd83JEbI2ILcBGSe+urHcu8ERbVZp1mbxH+mZpqTvSj4g9khYC9wIDwJKIWCPpysry\nxRFxj6T5ktYDO4HPVu3iKuAHlT8YT9csM+tJEfm1XBz6lraGN5FHxHJgec28xTXTC0fYdjXwgU4K\nNOs2r72WvODkiCOy/yyHvqXNd+SatWjzZjj++Hw+y6FvaXPom7Xo+edh2rR8Puu445KTxmZpceib\ntWjz5vxCf+pU2LIln8+y/uDQN2tRniP9adOSPzJmaXHom7Uoz9CfOjUJ/X378vk8G/0c+mYtyjP0\nx4+Ho45KXtpilgaHvlmL8gx9cIvH0uXQN2tR3qF//PHJZ5qlwaFv1oKIJIDzuk4fkj8wDn1Li0Pf\nrAWvvAJjx8KRR+b3mccf7/aOpcehb9aCvFs74JG+pcuhb9YCh771Ooe+WQuKCH23dyxNDn2zFnik\nb73OoW/WgiJCf/j5O74r19Lg0DdrQRGh77tyLU0OfbMWFBH64BaPpcehb9aCokLfJ3MtLQ59syZF\n5PvWrGoe6VtaHPpmTdq+PbkTd/z4/D/boW9pceibNamo1g64vWPpceibNanI0PdI39Li0DdrUtEj\nfYe+paFh6EsalLRW0jpJ146wzi2V5aslnVmzbEDSo5J+lFbRZkUoeqTv9o6loW7oSxoAFgGDwKnA\nxZJOqVlnPnBSRMwBrgBurdnN1cCTQKRVtFkRih7p+65cS0Ojkf5cYH1EbIiI3cBSYEHNOucBtwNE\nxApggqQpAJJmAPOB7wJKs3CzvBUZ+ocemtyV++KLxXy+jR6NQn86sLFqelNlXrPr3AR8GfD4xHpe\n3m/MquUWj6VhTIPlzbZkakfxkvRJYFtEPCqpVG/joaGht74vlUqUSnVXNytEkSN92H8Fz3vfW1wN\nVpxyuUy5XO54P4oYOdclzQOGImKwMn0dsC8ivlq1zreAckQsrUyvBUrAF4HPAHuA8cDRwN9HxKU1\nnxH1ajDrBnv3Jjdl7dwJ48YVU8Nll8E558Dllxfz+dZdJBERLbfNG7V3VgFzJM2WNA64EFhWs84y\n4NJKEfOAlyNiS0RcHxEzI+IE4CLggdrAN+sVL7wAEycWF/jga/UtHXXbOxGxR9JC4F5gAFgSEWsk\nXVlZvjgi7pE0X9J6YCfw2ZF2l2bhZnnavLnY1g4k5xPWrCm2But9jXr6RMRyYHnNvMU10wsb7OOn\nwE/bKdCsGxTdz4fk8x94oNgarPf5jlyzJjz3XLFX7oDbO5YOh75ZEzZsgNmzi63Bj2KwNDj0zZrw\n9NPwrncVW8Pxx8PWrb4r1zrj0DdrQjeE/qGHwoQJyeMYzNrl0DdrQjeEPsCJJya1mLXLoW/WwI4d\nSUvl2GOLriT5w+PQt0449M0aGB7lqwseGejQt0459M0a6JbWDiR1/Pa3RVdhvcyhb9ZAt4W+R/rW\nCYe+WQMOfRtNHPpmDXRT6E+dCv/xH/Dqq0VXYr3KoW/WQDeFvuTLNq0zDn2zOt54I3lF4YwZRVey\nn1s81gmHvlkdv/sdzJoFAwNFV7KfQ9864dA3q6ObWjvDHPrWCYe+WR0OfRttHPpmdTj0bbRx6JvV\n8fTTydUy3WTWrOT1jbt2FV2J9SKHvlkd3TjSHzsWpk9PXuxi1iqHvtkI9u6FZ57pvpE+uMVj7XPo\nm41g06bkccqHHVZ0JQdy6Fu7HPpmI+jG1s4wh761y6FvNgKHvo1GTYW+pEFJayWtk3TtCOvcUlm+\nWtKZlXkzJT0o6QlJv5H0xTSLN8uSQ99Go4ahL2kAWAQMAqcCF0s6pWad+cBJETEHuAK4tbJoN/Cl\niDgNmAf8We22Zt2qm0P/xBOTR0Ts21d0JdZrmhnpzwXWR8SGiNgNLAUW1KxzHnA7QESsACZImhIR\nWyLiscr814E1wLTUqjfL0FNPwbvfXXQVB3fUUcnX5s1FV2K9ppnQnw5srJreVJnXaJ23PZdQ0mzg\nTGBFq0Wa5W3nTli/Ht7znqIrGZlbPNaOMU2sE03uq/a10W9tJ+lI4C7g6sqI/22Ghobe+r5UKlEq\nlZr8SLNsPPIInH46HHpo0ZWM7KSTYN06OOecoiuxPJTLZcrlcsf7UUT9TJc0DxiKiMHK9HXAvoj4\natU63wLKEbG0Mr0W+GhEbJU0FrgbWB4RNx9k/9GoBrO8ff3rSc980aKiKxnZTTclof/NbxZdiRVB\nEhFRO9huqJn2zipgjqTZksYBFwLLatZZBlxaKWQe8HIl8AUsAZ48WOCbdauVK2Hu3KKrqO/ss2GF\nm6XWooahHxF7gIXAvcCTwA8jYo2kKyVdWVnnHuC3ktYDi4H/Vtn8Q8CngY9JerTyNZjFD2KWpocf\n7v7QP/NMWLMmebuXWbMatncyL8DtHesyL76YnCR96SU4pMtvX3z/++Eb34APfajoSixvWbZ3zPrK\nypVJmHZ74INbPNa6HvhnbZavhx+GD3yg6Cqac/bZ8MtfFl2F9RKHvlmNXjiJO2zePI/0rTUOfbMq\nEb1xEnfYnDnw2muwZUvRlVivcOibVXn2WRgYSN5M1Quk5A+UR/vWLIe+WZXhUb5aviaiOD6Za61w\n6JtV6aWTuMMc+tYKh75ZlV46iTts7lxYtSp5p69ZIw59s4q9e5MHrb3//UVX0ppJk2DyZFi7tuhK\nrBc49M0qVq6EGTPgmGOKrqR1vnTTmuXQN6v4/vfh4ouLrqI97utbs/zsHTNg167kMs2HH4YTTii6\nmtb9+tfwqU8lj4PuhcdHWOf87B2zDvz4x3Dyyb0Z+ABnnJG0pe6/v+hKrNs59M2A730PPvOZoqvo\nzJ/+KSxZUnQV1u3c3rG+9/LLMGsWbNgAEycWXU37Xnop+T+Vp5+GY48tuhrLmts7Zm268074wz/s\n7cCHpP5PfjI5IW02Eoe+9b3R0NoZ9rnPwXe/mzw4zuxgHPrW1zZsSF45+PGPF11JOj760eT1iStX\nFl2JdSuHvvW1m2+Giy6CceOKriQdhxwCl1/uE7o2Mp/Itb51//1w6aWwenXyKIPR4vnn4T3vgXXr\nfEJ3NPOJXLMWbN8Ol10Gt902ugIfYNq0pLd/ySV+CJsdyCN96zsRcMEFMHMm3HRT0dVkY88eGBxM\nHh73V39VdDWWhcxG+pIGJa2VtE7StSOsc0tl+WpJZ7ayrVnebrsN/v3f4StfKbqS7IwZA0uXwg9/\nCHfcUXQ11k3qhr6kAWARMAicClws6ZSadeYDJ0XEHOAK4NZmt7W3K5fLRZfQNbI4Fjt3wjXXwPXX\nww9+AOPHp/4RmWj3WEyaBP/4j7BwIfz85+nWVBT/jnSu0Uh/LrA+IjZExG5gKbCgZp3zgNsBImIF\nMEHS1Ca3tSr+B71f2sfi/vuT59Ns2QKPPw6nn57q7jPVybF43/uS/7O55BI4//zk8tRe5t+Rzo1p\nsHw6sLFqehNwdhPrTAemNbGtWereeCMJ90cegfvuSwJ/1y74m7+BT3yi6Ory94lPJO2sRYuS6/hL\nJfjwh+Gss5I/CkceWXSFlqdGod/sGdaOXiP9qU91svXo8dRT8KtfFV1Fd3jqqeQVgLD/7tKI/V/7\n9iVXpuzZA7//fRL0b7wB27Yl/506FU47Dc49F77wheQSxn5+5PBhh8GXvwyf/3zy2Ilf/Sp5XMPj\nj8PYsckjHCZOhMMPT6bHjEm+pOS4SQe+LL6Il8f7d6Rzda/ekTQPGIqIwcr0dcC+iPhq1TrfAsoR\nsbQyvRb4KHBCo20r833pjplZG9q5eqfRSH8VMEfSbOB54EKg9t1Cy4CFwNLKH4mXI2KrpO1NbNtW\n0WZm1p66oR8ReyQtBO4FBoAlEbFG0pWV5Ysj4h5J8yWtB3YCn623bZY/jJmZ1Vf4zVlmZpaf3E5t\ndXKT12jT6FhI+pPKMfi1pF9IOqOIOvPQ7A18kj4gaY+k/5pnfXlq8nekJOlRSb+RVM65xNw08Tsy\nSdKPJT1WORaXFVBm5iT9raStkh6vs05ruRkRmX+RtHfWA7OBscBjwCk168wH7ql8fzbwyzxqy/ur\nyWPxn4F3VL4f7OdjUbXeA8DdwPlF113gv4sJwBPAjMr0pKLrLvBYDAFfGT4OwHZgTNG1Z3AsPgKc\nCTw+wvKWczOvkX67N3lNyam+PDU8FhHxUES8UplcAczIuca8NHsD31XAXcALeRaXs2aOxSXA30fE\nJoCIeDHnGvPSzLHYDBxd+f5oYHtE7MmxxlxExM+Bl+qs0nJu5hX6I93A1Wid0Rh2zRyLap8D7sm0\nouI0PBaSppP8wt9amTVaT0I18+9iDnCMpAclrZI0St73dYBmjsV3gNMkPQ+sBq7OqbZu03JuNrpk\nMy3t3uQ1Gn/Bm/6ZJH0MuBz4UHblFKqZY3Ez8BcREZJEhzcCdrFmjsVY4CzgvwCHAw9J+mVErMu0\nsvw1cyyuBx6LiJKkdwE/kfTeiHgt49q6UUu5mVfoPwfMrJqeSfIXqd46MyrzRptmjgWVk7ffAQYj\not7/3vWyZo7FfyK5BwSS3u3HJe2OiGX5lJibZo7FRuDFiHgDeEPSz4D3AqMt9Js5Fh8E/hdARDwt\n6XfAH5DcW9RPWs7NvNo7b93kJWkcyY1atb+0y4BL4a07gV+OiK051ZenhsdC0juBfwA+HRHrC6gx\nLw2PRUScGBEnRMQJJH39L4zCwIfmfkf+GfiwpAFJh5OcuHsy5zrz0MyxWAucC1DpYf8B8Ntcq+wO\nLedmLiP96OAmr9GmmWMB/CUwEbi1MsLdHRFzi6o5K00ei77Q5O/IWkk/Bn4N7AO+ExGjLvSb/Hfx\nv4HbJK0mGbz+j4jYUVjRGZH0f0keazNJ0kbgBpI2X9u56ZuzzMz6SB8/d9DMrP849M3M+ohD38ys\njzj0zcz6iEPfzKyPOPTNzPqIQ9/MrI849M3M+sj/B/lIdTt46O5YAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "pmf = beta.MakePmf()\n", - "xs = [i / 100. for i in range(101)]\n", - "ps = pmf.Probs(xs)\n", - "plt.plot(xs, ps);" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Discussion\n", - "\n", - "In this chapter we solved the same problem with two different priors and\n", - "found that with a large dataset, the priors get swamped. If two people\n", - "start with different prior beliefs, they generally find, as they see\n", - "more data, that their posterior distributions converge. At some point\n", - "the difference between their distribution is small enough that it has no\n", - "practical effect.\n", - "\n", - "When this happens, it relieves some of the worry about objectivity that\n", - "I discussed in the previous chapter. And for many real-world problems\n", - "even stark prior beliefs can eventually be reconciled by data.\n", - "\n", - "But that is not always the case. First, remember that all Bayesian\n", - "analysis is based on modeling decisions. If you and I do not choose the\n", - "same model, we might interpret data differently. So even with the same\n", - "data, we would compute different likelihoods, and our posterior beliefs\n", - "might not converge.\n", - "\n", - "Also, notice that in a Bayesian update, we multiply each prior\n", - "probability by a likelihood, so if $\\mathrm{p}(H)$ is 0,\n", - "$\\mathrm{p}(H|D)$ is also 0, regardless of $D$. In the Euro\n", - "problem, if you are convinced that $x$ is less than 50%, and you assign\n", - "probability 0 to all other hypotheses, no amount of data will convince\n", - "you otherwise.\n", - "\n", - "This observation is the basis of **Cromwell’s rule**, which\n", - "is the recommendation that you should avoid giving a prior probability\n", - "of 0 to any hypothesis that is even remotely possible (see\n", - ").\n", - "\n", - "Cromwell’s rule is named after Oliver Cromwell, who wrote, “I beseech\n", - "you, in the bowels of Christ, think it possible that you may be\n", - "mistaken.” For Bayesians, this turns out to be good advice (even if it’s\n", - "a little overwrought)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Exercises\n", - "---------\n", - "\n", - "Suppose that instead of observing coin tosses directly, you measure the\n", - "outcome using an instrument that is not always correct. Specifically,\n", - "suppose there is a probability y that an actual heads is\n", - "reported as tails, or actual tails reported as heads.\n", - "\n", - "Write a class that estimates the bias of a coin given a series of\n", - "outcomes and the value of y.\n", - "\n", - "How does the spread of the posterior distribution depend on\n", - "y?\n", - "\n", - "This exercise is inspired by a question posted by a “redditor” named\n", - "dominosci on Reddit’s statistics “subreddit” at\n", - ".\n", - "\n", - "Reddit is an online forum with many interest groups called subreddits.\n", - "Users, called redditors, post links to online content and other web\n", - "pages. Other redditors vote on the links, giving an “upvote” to\n", - "high-quality links and a “downvote” to links that are bad or irrelevant.\n", - "\n", - "A problem, identified by dominosci, is that some redditors are more\n", - "reliable than others, and Reddit does not take this into account.\n", - "\n", - "The challenge is to devise a system so that when a redditor casts a\n", - "vote, the estimated quality of the link is updated in accordance with\n", - "the reliability of the redditor, and the estimated reliability of the\n", - "redditor is updated in accordance with the quality of the link.\n", - "\n", - "One approach is to model the quality of the link as the probability of\n", - "garnering an upvote, and to model the reliability of the redditor as the\n", - "probability of correctly giving an upvote to a high-quality item.\n", - "\n", - "Write class definitions for redditors and links and an update function\n", - "that updates both objects whenever a redditor casts a vote." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# More Estimation \n", + "\n", + "\n", + "## The Euro problem\n", + "\n", + "In *Information Theory, Inference, and Learning\n", + "Algorithms*, David MacKay poses this problem:\n", + "\n", + "> A statistical statement appeared in \\`\\`The Guardian\" on Friday\n", + "> January 4, 2002:\n", + ">\n", + "> > When spun on edge 250 times, a Belgian one-euro coin came up heads\n", + "> > 140 times and tails 110. ‘It looks very suspicious to me,’ said\n", + "> > Barry Blight, a statistics lecturer at the London School of\n", + "> > Economics. ‘If the coin were unbiased, the chance of getting a\n", + "> > result as extreme as that would be less than 7%.’\n", + ">\n", + "> But do these data give evidence that the coin is biased rather than\n", + "> fair?\n", + "\n", + "To answer that question, we’ll proceed in two steps. The first is to\n", + "estimate the probability that the coin lands face up. The second is to\n", + "evaluate whether the data support the hypothesis that the coin is\n", + "biased.\n", + "\n", + "You can download the code in this section from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "Any given coin has some probability, $x$, of landing heads up when spun\n", + "on edge. It seems reasonable to believe that the value of $x$ depends on\n", + "some physical characteristics of the coin, primarily the distribution of\n", + "weight.\n", + "\n", + "If a coin is perfectly balanced, we expect $x$ to be close to 50%, but\n", + "for a lopsided coin, $x$ might be substantially different. We can use\n", + "Bayes’s theorem and the observed data to estimate $x$.\n", + "\n", + "Let’s define 101 hypotheses, where $H_x$ is the hypothesis that the\n", + "probability of heads is $x$%, for values from 0 to 100. I’ll start with\n", + "a uniform prior where the probability of $H_x$ is the same for all $x$.\n", + "We’ll come back later to consider other priors.\n", + "\n", + "[Posterior distribution for the Euro problem on a uniform\n", + "prior.](figs/euro1.pdf)\n", + "\n", + "[fig.euro1]\n", + "\n", + "The likelihood function is relatively easy: If $H_x$ is true, the\n", + "probability of heads is $x/100$ and the probability of tails is $1-\n", + "x/100$." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from thinkbayes import Suite\n", + "class Euro(Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = hypo\n", + " if data == 'H':\n", + " return x/100.0\n", + " else:\n", + " return 1 - x/100.0" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here’s the code that makes the suite and updates it:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "suite = Euro(range(0, 101))\n", + "dataset = 'H' * 140 + 'T' * 110\n", + "\n", + "for data in dataset:\n", + " suite.Update(data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The result may be plotted with:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu8AAAEWCAYAAADW9nkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VNW99/HvZHIhARICIZncCAQDclFEggqBipZAQXuh\nvuBY6wWop80pIgEPtVSq9fDwqKc9olQCnpaCtyo+rZdTD1gigiUgbUkNCoRL5RoyMxBCQkhCLjPz\n/BEzzJCQDCHJnkk+79eLl7N31t77t+lq+GZl7bVNLpfLJQAAAAB+L8joAgAAAAD4hvAOAAAABAjC\nOwAAABAgCO8AAABAgCC8AwAAAAGC8A4AAAAECMI7AAAAECB8Du85OTlKTU1VeHi40tPTlZeXd8W2\nNTU1mjNnjkaNGqXQ0FDdeeedLZ47Ly9PISEhuvHGG32vHAAAAOhmfArvGzZsUHZ2tpYuXaqCggKN\nHz9e06ZNU1FRUbPtHQ6HwsPDNX/+fN19990tnrusrEwPPfSQJk+efPXVAwAAAN2IyZc3rN522226\n6aabtGbNGve+IUOGaObMmVq+fHmLx86fP1/79u3Txx9/3OzX77nnHt10001yOp364x//qM8///wq\nbwEAAADoHlodea+rq1N+fr4yMzO99k+ZMkU7d+68povn5OTo9OnTWrp06TWdBwAAAOgOWg3vJSUl\ncjgciouL89ofFxcnm83W5gt/8cUXWrZsmd544w2ZTKY2nwcAAADoLoKNuGhtba3uvfde/epXv9KA\nAQMkSS3N3ikvL++s0gAAAIB2FxUV1S7naTW8x8TEyGw2y263e+232+2yWCxtuqjValVhYaHmzJmj\n2bNnS5KcTqdcLpdCQ0O1ceNGHmAFAAAALtPqtJmQkBCNGTNGubm5Xvtzc3OVkZHRposmJiZq7969\nKigo0J49e7Rnzx5lZWUpLS1Ne/bs0fjx49t0XgAAAKAr82nazKJFi/Tggw9q7NixysjI0OrVq2W1\nWpWVlSVJWrJkif7+97/ro48+ch9TWFiompoalZSU6MKFC9qzZ48kadSoUQoODtbw4cO9rhEbG6uw\nsDANGzasxVra61cO6Hp2794tSUpPTze4Evgz+gl8QT+BL+gnaE1HTP32KbzPmjVLpaWlWr58uaxW\nq0aOHKlNmzYpKSlJkmSz2XT06FGvY6ZPn64TJ064t0ePHi2TySSHw9GO5QMAAADdh0/rvBvN86cW\nRt5xJYyAwBf0E/iCfgJf0E/Qmo7IsD69YRUAAACA8QjvAAAAQIAgvAMAAAABgvAOAAAABAjCOwAA\nABAgCO8AAABAgCC8AwAAAAGC8A4AAAAECMI7AAAAECAI7wAAAECAILwDAAAAAYLwDgAAAAQIwjsA\nAAAQIAjvAAAAQIAgvAMA/N65ihKdPP2l0WUAgOGCjS4AAICWnDpzTC/+4We6WFulabd9T9Nu/Rej\nSwIAwzDyDgDwW06nQ29+9JIu1lZJkv781w06efqIwVUBgHEI7wAAv7Wt4AOdOP1P97bT5dRbW1bJ\n4XQYWBUAGIfwDgDwSyXlNv3vp2802X/y9Jf6pOADAyoCAOMR3gEAfsflcmnDx6tVV18rSUrol6Jv\neMx13/jp73W23G5UeQBgGMI7AMDv/K1wqw6e2CNJMpmC9L3J8zR17Ewl9EuRJNXW12jDx6vlcrmM\nLBMAOh3hHQDgV85Xlundv/zOvX37TXcrxTJEZnOw7p08TyaZJEkHThRo98FPjCoTAAxBeAcA+JV3\n/rJWVTUXJEl9I2N117j73F8baBmir91016W2n6xVRVV5p9cIAEbxObzn5OQoNTVV4eHhSk9PV15e\n3hXb1tTUaM6cORo1apRCQ0N15513Nmnz7rvvaurUqYqNjVVkZKRuu+02/elPf2rbXQAAuoS9R/6u\nfxza7t7+lzv/TWEhPbza3DXu+4ru3V+SVHmxQu9u/50AoLvwKbxv2LBB2dnZWrp0qQoKCjR+/HhN\nmzZNRUVFzbZ3OBwKDw/X/Pnzdffddzfb5pNPPtHXv/51bdy4UQUFBZo+fbpmzJihHTt2tP1uAAAB\nq7qmSm9vXePevmXYHRqWMrpJux6h4Zp1x4/c27sPfKLC4591So0AYDSfwvuKFSs0d+5czZ07V0OH\nDtXKlSsVHx+v1atXN9s+IiJCOTk5evjhh5WYmNhsmxdeeEE/+clPlJ6ertTUVD355JMaM2aM3nvv\nvbbfDQAgYH2w83WVXTgrSeoVHqUZE+dcse2IQem6echE9/aGj1erpu5ih9cIAEZrNbzX1dUpPz9f\nmZmZXvunTJminTt3tmsxFRUVio6ObtdzAgD839nzduV9vsm9fc/tP1DP8MgWj7nn9h8ookdvSVLp\n+dP6y56NHVojAPiDVsN7SUmJHA6H4uLivPbHxcXJZrO1WyGrVq3SqVOn9MADD7TbOQEAgeHQic/l\nUsOyj4MTR3iNql9J74g+Xg+zHmTqDIBuINjoAiTpj3/8ox5//HG9/fbbSk5ObrHt7t27O6kqBCr6\nCHxBP/Evfzt86SHVqOA45efn+3ScsybM/flI8QH97W9/VVCQud3qop/AF/QTXElaWlq7n7PVkfeY\nmBiZzWbZ7d5vsrPb7bJYLNdcwB/+8Ac9+OCDeu211zR9+vRrPh8AIPDYz59wf46NbHkQx1PPsEj1\nDIuSJNU761RayVtXAXRtrY68h4SEaMyYMcrNzdU999zj3p+bm6uZM2de08XffvttzZkzR6+++qpm\nzJjh0zHp6enXdE10XY0jH/QRtIR+4n/KL5Tqwo4ySVKIOVRTbr9bweYQn4/fX3KT+2VNYVFS+s3X\n/r8t/QS+oJ+gNeXl7f8eCp9Wm1m0aJHWr1+vtWvX6sCBA1qwYIGsVquysrIkSUuWLNHkyZO9jiks\nLFRBQYFKSkp04cIF7dmzR3v27HF//a233tL999+vZ599VhMmTJDdbpfdbte5c+fa8fYAAP7uy+L9\n7s8plrSrCu6SlJowzP35iMe5AKAr8mnO+6xZs1RaWqrly5fLarVq5MiR2rRpk5KSkiRJNptNR48e\n9Tpm+vTpOnHi0q9BR48eLZPJJIfDIUl6+eWX5XA4lJ2drezsbHe722+/XR9//PE13xgAIDB4Bu7U\nhOFXfbx3eC+Uy+WSyWRql9oAwN/4/MBqVlaWe6T9cuvWrWuy7/Iwf7mtW7f6emkAQBf25alL4X1w\n4tWHd0u/ZIWH9VR1TaUqqst1psyq2OiE9iwRAPyGT9NmAADoCFU1F1RcclySZDIFaaBl6FWfI8gU\npEHx17u3j1oL260+APA3hHcAgGGOWQ+613dP7D9Q4WERbTqP59SZL4sJ7wC6LsI7AMAwXlNm2jDf\n/dKx3vPeAaCrIrwDAAzz5TU+rNpoQFyazOaGx7hOnzuliqr2X54NAPwB4R0AYIi6+lodtx92b1/L\nyHtIcKgGxF7n3mbeO4CuivAOADDECfs/5XDUS5L690lQZM8+13S+y5eMBICuiPAOADCE55QZzznr\nbcVDqwC6A8I7AMAQR061z3x39zk8los8efpL1dbVXPM5AcDfEN4BAJ3O6XToqPWAe7stL2e6XM/w\nSFn6JrvPf9x+6JrPCQD+hvAOAOh01rMnVF1bJUmKjIhWTJSlXc7LvHcAXR3hHQDQ6byWiEwcJpPJ\n1C7nZd47gK6O8A4A6HTt9XKmy3me65j1oJxOR7udGwD8AeEdANCpXC6X15SW9nhYtVHfyFhF9oyW\nJF2srZL17Il2OzcA+APCOwCgU509b1d5ZakkqUdohBJjUtrt3CaTiakzALo0wjsAoFN5TpkZFH+9\ngoLM7Xp+z6kzPLQKoKshvAMAOlV7v5zpct4j7/vlcrna/RoAYBTCOwCgU3mOhrfH+u6XS4gZqLCQ\nHpKk8gtnda7iTLtfAwCMQngHAHSaiqoynT53SpJkNgdrQFxau1/DHGTWwPih7m3mvQPoSgjvAIBO\n4znqnhKbppDg0A65Tirz3gF0UYR3AECn8XxYNbUDpsw0Guz1ptX9LbQEgMBCeAcAdBqv+e4d8LBq\noxTLEAWZGv6Js549oaqLFzrsWgDQmQjvAIBOUVN3UUVnjkiSTDJpUML1HXatsJAeSood7N4+aj3Q\nYdcCgM5EeAcAdArr2RNyupySpNi+iYoI69Wh10uNv/TDwXHb4Q69FgB0Fp/De05OjlJTUxUeHq70\n9HTl5eVdsW1NTY3mzJmjUaNGKTQ0VHfeeWez7T755BOlp6crPDxc1113nV5++eWrvwMAQEAoLjnu\n/pwYM7DDr5fYf5D7s7X0RIdfDwA6g0/hfcOGDcrOztbSpUtVUFCg8ePHa9q0aSoqKmq2vcPhUHh4\nuObPn6+777672TbHjh3TXXfdpQkTJqigoEA//elPNX/+fL377rttvxsAgN+ynr0U3uP7pXT49eL7\nDfC4NuEdQNfgU3hfsWKF5s6dq7lz52ro0KFauXKl4uPjtXr16mbbR0REKCcnRw8//LASExObbbN6\n9WolJibqhRde0NChQ/Xwww/roYce0q9+9au23w0AwG9ZSzzD+4AWWrYPS99kmWSSJJ0ps6quvrbD\nrwkAHa3V8F5XV6f8/HxlZmZ67Z8yZYp27tzZ5gvv2rVLU6ZM8do3depU7d69Ww6Ho83nBQD4p2KP\n0e+EmI4feQ8NCVO/qDhJksvllP1c878tBoBA0mp4LykpkcPhUFxcnNf+uLg42Wy2Nl/YZrM1e876\n+nqVlJS0+bwAAP9TUVWmC9XlkqTQkB7qGxnbKdf1njpzslOuCQAdKdjoAq7W7t27jS4Bfo4+Al/Q\nTzqXteyo+3NkWF/9I/8fnXPh2hD3x8/2/lWmCz2v6nD6CXxBP8GVpKWltfs5Wx15j4mJkdlslt1u\n99pvt9tlsVjafGGLxdLsOYODgxUTE9Pm8wIA/M+5qtPuz30iOmfU/fJrlVWd6bTrAkBHaXXkPSQk\nRGPGjFFubq7uuece9/7c3FzNnDmzzRceN26c3nvvPa99mzdvVnp6usxm8xWPS09Pb/M10bU1jnzQ\nR9AS+okxDn20y/35hutvVvrozvn7TyiJ0fZDDauYVTnKff7fnX4CX9BP0Jry8vJ2P6dPq80sWrRI\n69ev19q1a3XgwAEtWLBAVqtVWVlZkqQlS5Zo8uTJXscUFhaqoKBAJSUlunDhgvbs2aM9e/a4v56V\nlaVTp05p4cKFOnDggH7729/q1Vdf1eLFi9vx9gAA/sBzpZmETlgmslFsdKKCghoGhErPn1ZNbXWn\nXRsAOoJPc95nzZql0tJSLV++XFarVSNHjtSmTZuUlJQkqeHh06NHj3odM336dJ04cWllgdGjR8tk\nMrlXkhk4cKA2btyohQsXas2aNUpISNCvf/1rfec732mvewMA+AGnyylr6aWHRTtjpZlGweYQxfZJ\nkO2r69tKTyrFMqTTrg8A7c3nB1azsrLcI+2XW7duXZN9l4f55kycOJGHPACgiys9f1q1dRclSb3C\no9Q7ok+nXj++3wB3eC8+e4LwDiCg+TRtBgCAtir2mjLT8S9nuhxvWgXQlRDeAQAdyjMwx3filBn3\nNT3m2FvPHm+hJQD4P8I7AKBDeQbm+E58WPXSNZM9amHkHUBgI7wDADqU17QZA0beY6IsCjY3vKzp\nfOU5VV6s6PQaAKC9EN4BAB2mrr5Op8uK3duWvskttO4YQUFmr+vaGH0HEMAI7wCADnP63Ck5nQ1L\nBPeLjFOP0HBD6vB8aLWY8A4ggBHeAQAdpthzvrsBU2bc12bFGQBdBOEdANBhPIOyEctENiK8A+gq\nCO8AgA5jLTF2pZlL174U3m1nT8jlchlWCwBcC8I7AKDDeE2bMXDkPbp3f4WF9JAkVV6sUEVVmWG1\nAMC1ILwDADpEdU2VzlWckSSZg4IVF51oWC0mk+mylzUxdQZAYCK8AwA6hGdAjotOlNkcbGA1l684\nw5tWAQQmwjsAoENY/WTKTCMLb1oF0AUQ3gEAHcLqJ8tENkrwmDZjO3vSwEoAoO0I7wCADlHssdJM\ngoErzTTyXi7yOCvOAAhIhHcAQLtzuVxeU1PiY4yfNtM7oo969ugtSaqpu+h+mBYAAgnhHQDQ7s5X\nnVPlxQpJUlhouPr2jjW4osYVZ3hZE4DARngHALS74hLvh1VNJpOB1Vxi8VpxhvAOIPAQ3gEA7c5z\nVDvBD1aaaXT5m1YBINAQ3gEA7c7qNfJu/MOqjRKYNgMgwBHeAQDtrvisf4Z3z2kzttKTcjodBlYD\nAFeP8A4AaFdOp0O20kvrqCf4wRrvjXr26K2onn0lSfWOOpWU2wyuCACuDuEdANCuSsrtqquvlSRF\nRkSrV3ikwRV5Y8UZAIGM8A4AaFdeb1b1o4dVG1kI7wACmM/hPScnR6mpqQoPD1d6erry8vJabL93\n715NmjRJERERSk5O1rJly5q0+f3vf6/Ro0erZ8+eio+P1wMPPCC73X71dwEA8BvFXi9n8p8pM40Y\neQcQyHwK7xs2bFB2draWLl2qgoICjR8/XtOmTVNRUVGz7SsqKpSZman4+Hjl5+frxRdf1C9/+Uut\nWLHC3WbHjh168MEHNWfOHO3fv1/vv/++CgsLdf/997fPnQEADOG50kyCHz2s2ogVZwAEMp/C+4oV\nKzR37lzNnTtXQ4cO1cqVKxUfH6/Vq1c32/71119XdXW1XnnlFQ0bNkzf/e539fjjj+v55593t9m1\na5eSk5P16KOPKiUlRbfccoseeeQR/fWvf22fOwMAGMIzEPvltJm+ye7Pp8uKVVdfZ2A1AHB1Wg3v\ndXV1ys/PV2Zmptf+KVOmaOfOnc0es2vXLk2cOFGhoaHufVOnTlVxcbGOH28YkcnIyJDVatUHH3wg\nSSopKdFbb72lu+66q803AwAwVl19rc6UFUuSTDLJ0i+5lSM6X1houPpFxklqWBnnTNkpgysCAN8F\nt9agpKREDodDcXFxXvvj4uK0ZcuWZo+x2WxKTk5u0t7lcslmsyklJUW33Xab3nzzTX3/+99XdXW1\n6uvrNWXKFK1fv77Fenbv3t1ayejm6CPwBf2kY5RW2uV0OSVJPXtE6Ys9ew2uqHnh5khJDc9Y7fj7\nNg3qP7LZdvQT+IJ+gitJS0tr93MattrM/v37NX/+fD311FP6xz/+oT//+c+yWq364Q9/aFRJAIBr\nVFZ1xv25T3h/AytpWVTEpdrKqkoMrAQArk6rI+8xMTEym81NVoGx2+2yWCzNHmOxWJptbzKZ3Mc8\n++yzuvXWW7Vo0SJJ0siRI5WTk6OJEyfqmWeeUUJCQrPnTk9Pb/2u0C01jnzQR9AS+knHsu084P58\n/eAb/fbv2dmzQvtONUz9DAqrb1In/QS+oJ+gNeXl5e1+zlZH3kNCQjRmzBjl5uZ67c/NzVVGRkaz\nx4wbN07bt29XbW2te9/mzZuVkJCglJSGlQeqqqpkNpu9iwkKkslkktPpvOobAQAYz/thVf+b797I\n86FVW2nzK6cBgD/yadrMokWLtH79eq1du1YHDhzQggULZLValZWVJUlasmSJJk+e7G5/3333KSIi\nQrNnz9a+ffv0zjvv6LnnntNjjz3mbvPNb35T77//vtasWaOjR49qx44dWrBggcaMGaOkpKR2vk0A\nQGewnT3p/mzp638rzTSK63vp35kzZcWqd7DiDIDA0Oq0GUmaNWuWSktLtXz5clmtVo0cOVKbNm1y\nh2ybzaajR4+620dGRio3N1fz5s3T2LFjFR0drcWLFys7O9vd5qGHHtKFCxe0atUq/fu//7v69Omj\nO++8U88++2w73yIAoDPU1teopNwmSTKZghTXN9Hgiq4sLKSH+kbGqvT8aTldTp0+V6wEP3yhFABc\nzqfwLklZWVnukfbLrVu3rsm+ESNGaNu2bS2ec968eZo3b56vJQAA/Ji99JRcckmSYqIsCg0OM7ii\nlsX3HaDS86clSbbSk4R3AAHBsNVmAABdi/XspTer+vN890aWfpemzthKT7bQEgD8B+EdANAuAmW+\neyPvh1YJ7wACA+EdANAurKWBsdJMI6/wfpbwDiAwEN4BAO0i0Ebe4zzC++myYjkc9QZWAwC+IbwD\nAK5ZTd1FnT3f8HK+IFOQYqP9d6WZRj1CwxXdu+FNq06nQ2fKrQZXBACtI7wDAK6Z3eNFRzF94hUS\nHGJgNb5j6gyAQEN4BwBcM683q/b1//nujSweL2uy8tAqgABAeAcAXDObx8Oqln7+P9+9kWetdsI7\ngABAeAcAXDOrx5ST+EAK70ybARBgCO8AgGtm85w2E1Dh/dK0GXvZKTmcDgOrAYDWEd4BANfkYm21\nSivOSJKCgszq3yfe4Ip8Fx7WU1G9+kmSHI56lZTbDK4IAFpGeAcAXBPPt5PG9klQsDkwVpppFM/U\nGQABhPAOALgmnivNWALgzaqX85r3zkOrAPwc4R0AcE285rsHwJtVL+f5AwfhHYC/I7wDAK6J5/ro\ngbRMZCPvFWdOtNASAIxHeAcAXBPvlWYCe9qM/dwpOVlxBoAfI7wDANqsuqZSZRfOSpLMQcHqHxU4\nK800iujRS5E9oyVJ9Y46nT1/2uCKAODKCO8AgDbzWmkmOkFmc7CB1bQdD60CCBSEdwBAm3m/WTXF\nwEqujWd4tzLvHYAfI7wDANrMeva4+3Mgzndv5PlWWEbeAfgzwjsAoM08X2pkCcBlIhtZ+ia5PxPe\nAfgzwjsAoM2spYG90kwjrxVnSovkcrkMrAYArozwDgBok6qLF3S+8pwkKdgcopgoi8EVtV3P8Ej1\nDo+SJNXV1+pCTZnBFQFA83wO7zk5OUpNTVV4eLjS09OVl5fXYvu9e/dq0qRJioiIUHJyspYtW9ak\nTV1dnZ588kmlpqaqR48eGjhwoF566aWrvwsAQKfzfLAzLjpRQUFmA6u5dp4vmCqvKjGwEgC4Mp/W\n9NqwYYOys7O1Zs0aZWRkaNWqVZo2bZoKCwuVlJTUpH1FRYUyMzM1adIk5efnq7CwULNnz1avXr20\ncOFCd7t/+Zd/UXFxsX7729/quuuuk91uV3V1dfvdHQCgw9gC/M2ql7P0Tdbhoi8kSWVVZ5TUN83g\nigCgKZ/C+4oVKzR37lzNnTtXkrRy5Up9+OGHWr16tZYvX96k/euvv67q6mq98sorCg0N1bBhw1RY\nWKjnn3/eHd43b96srVu36ssvv1Tfvn0lSQMGBP43fwDoLjxH3uP7Bu5890aeD62WVZ0xsBIAuLJW\np83U1dUpPz9fmZmZXvunTJminTt3NnvMrl27NHHiRIWGhrr3TZ06VcXFxTp+vGFZsffff19jx47V\nf/3Xfyk5OVlDhgzRggULVFlZeS33AwDoJDaP8N4lRt49Hrgtr2baDAD/1OrIe0lJiRwOh+Li4rz2\nx8XFacuWLc0eY7PZlJyc3KS9y+WSzWZTSkqKjhw5ou3btyssLEzvvPOOysrK9Mgjj8hqtertt9++\nYj27d+/25b7QjdFH4Av6ybU7YT/i/lxSXK7d5wL77/Ri3aXBo/KqErlcLvoJfEI/wZWkpbX/9DvD\n3mPtdDoVFBSkN998U7169ZIkvfTSS/rGN76hM2fOqH///kaVBgBoxcW6KnfYNQcFq1ePPgZXdO16\nhPRUWHCEauqrVO+sU2VNeZe4LwBdS6vhPSYmRmazWXa73Wu/3W6XxdL8smAWi6XZ9iaTyX1MfHy8\nEhMT3cFdkoYNGyaXy6UTJ05cMbynp6e3VjK6qcaRD/oIWkI/aR+Hi/ZKf2v4HB8zQLeMvcXYgtrJ\njmOD9OWpfZKksqoSTZow2eCK4M/4foLWlJeXt/s5W53zHhISojFjxig3N9drf25urjIyMpo9Zty4\ncdq+fbtqa2vd+zZv3qyEhASlpKRIkjIyMlRcXKyqqip3m4MHD8pkMrnbAAD8k83rYdXAn+/eyPNl\nTeXVPLQKwP/4tM77okWLtH79eq1du1YHDhzQggULZLValZWVJUlasmSJJk++NDpx3333KSIiQrNn\nz9a+ffv0zjvv6LnnntNjjz3m1aZfv36aM2eO9u/frx07dig7O1szZ85UTExMO98mAKA9WbvYMpGN\nPN8SW8Za7wD8kE9z3mfNmqXS0lItX75cVqtVI0eO1KZNm9xrvNtsNh09etTdPjIyUrm5uZo3b57G\njh2r6OhoLV68WNnZ2e42PXv21EcffaT58+frlltuUXR0tGbMmKFnnnmmnW8RANDebF1smchGXiPv\nhHcAfsjnB1azsrLcI+2XW7duXZN9I0aM0LZt21o8Z1pamj788ENfSwAA+AGXy6Viz/DehUbeL582\n43K5ZDKZDKwIALz5NG0GAIBG5yrOqOpihSQpPDRCfSNjDa6o/fSO6KOIsIaFFOoctSq7wOg7AP9C\neAcAXJWTpy+t754UO7hLjUybTCavlzUVlxw3sBoAaIrwDgC4KkVnvnR/To5NNbCSjpEYM8j9uejM\n0RZaAkDnI7wDAK6K18h7/64X3pP6e4b3Iy20BIDOR3gHAFyVIo/wnhw72MBKOkaSxz0R3gH4G8I7\nAMBn5ZWlOl91TpIUGtJD/fvEG1xR+4vvl6wgU8M/j2fL7aqquWBwRQBwCeEdAOAzz1H3pJhBCgoy\nG1hNxwg2hygqor97+xTz3gH4EcI7AMBnntNIkrrgw6qN+va0uD8XnSa8A/AfhHcAgM9Oes1377rh\nvV8vj/DOvHcAfoTwDgDwWdHpS8tEJvXveg+rNvIaeSe8A/AjhHcAgE8qq8+rtOKMpIZ54Za+SQZX\n1HGie8a5P9tLi1RbX2NgNQBwCeEdAOATzxcWJfRLkdkcbGA1HSvEHKrIHn0lSU6XU1betArATxDe\nAQA+Oek5ZaYLz3dv1Ndr3jsPrQLwD4R3AIBPPOd+d8WXM13Oc9675w8uAGAkwjsAwCeeK80k9Wfk\nHQCMQHgHALSquqZKZ8qKJUlBpiAlxKQYXFHH8xx5t5Ycl8PpMLAaAGhAeAcAtOpUyaWRZ0u/AQoJ\nDjWwms7RIyRC0b1iJEl1jlrZS4sMrggACO8AAB94zvlO7gZTZholejyYy3rvAPwB4R0A0Koiz/nu\n3WClmUbnTx7nAAAcmUlEQVRJ/Qe5P3v+HQCAUQjvAIBWdbeVZhp5PpjLyDsAf0B4BwC0qLauRrav\n5nubZFJizEBjC+pEyR6/ZTh15qhcLpeB1QAA4R0A0Iris8flcjklSbHRiQoLDTe4os7Tp1eMevbo\nLUmqrq3S2fN2gysC0N0R3gEALepub1b1ZDKZvKbOnGTeOwCD+Rzec3JylJqaqvDwcKWnpysvL6/F\n9nv37tWkSZMUERGh5ORkLVu27Ipt8/LyFBISohtvvNH3ygEAncLzQc3kbhbeJSkp9tJDq6eY9w7A\nYD6F9w0bNig7O1tLly5VQUGBxo8fr2nTpqmoqPk1bysqKpSZman4+Hjl5+frxRdf1C9/+UutWLGi\nSduysjI99NBDmjx58rXdCQCgQ5w84zHy3r/7PKzayPOeWXEGgNF8Cu8rVqzQ3LlzNXfuXA0dOlQr\nV65UfHy8Vq9e3Wz7119/XdXV1XrllVc0bNgwffe739Xjjz+u559/vknbH/zgB5o9e7Zuu+22a7sT\nAEC7q3fUyVpywr3tOQrdXSR5rfV+tIWWANDxWg3vdXV1ys/PV2Zmptf+KVOmaOfOnc0es2vXLk2c\nOFGhoZfewDd16lQVFxfr+PHj7n05OTk6ffq0li5d2tb6AQAdyHr2pBzOeklSv8g4RYT1Mriizte/\nT7xCQ3pIks5XnVN5ZanBFQHozloN7yUlJXI4HIqLi/PaHxcXJ5vN1uwxNput2fYul8t9zBdffKFl\ny5bpjTfekMlkamv9AIAOVNSNH1ZtFGQK8loek6kzAIwUbMRFa2trde+99+pXv/qVBgwYIEk+r527\ne/fujiwNXQB9BL6gn/jmH1/ucn8Oqg3rdn9vjfcbqku/cfhrQZ6qzxpVEfxRd/v/BXyXlpbW7uds\nNbzHxMTIbDbLbvde29Zut8tisTR7jMViaba9yWSSxWKR1WpVYWGh5syZo9mzZ0uSnE6nXC6XQkND\ntXHjRh5gBQA/cLby0m9Y+/aKN7ASY/Xteenfu9LK5n/rDACdodXwHhISojFjxig3N1f33HOPe39u\nbq5mzpzZ7DHjxo3TT3/6U9XW1rrnvW/evFkJCQlKSUlRfX299u7d63XMqlWr9NFHH+m9995TSkrK\nFetJT0/36cbQ/TSOfNBH0BL6ie8cTofe/Ot/urfvGD9VkT37GFhR57m8n8Sd7qtP//mBJKmy7hz9\nB5L4foLWlZeXt/s5fVptZtGiRVq/fr3Wrl2rAwcOaMGCBbJarcrKypIkLVmyxGuk/L777lNERIRm\nz56tffv26Z133tFzzz2nxx57TJIUHBys4cOHe/2JjY1VWFiYhg0bpoiIiHa/UQDA1Tl97pTq6msl\nSVG9+nWb4N6c+H7JMgc1jHedPW9XVc0FgysC0F35NOd91qxZKi0t1fLly2W1WjVy5Eht2rRJSUlJ\nkhoeUD169NLyWZGRkcrNzdW8efM0duxYRUdHa/HixcrOzu6YuwAAtDvPN6sm9++eD6s2CjaHKL7f\nABV99ZKmU2eOKi3pBoOrAtAd+fzAalZWlnuk/XLr1q1rsm/EiBHatm2bz4U89dRTeuqpp3xuDwDo\nWJ5rmnfXlWY8JfUf5A7vRacJ7wCM4dO0GQBA93PCdtj9OTm2+71Z9XKeP8B4vnUWADoT4R0A0ERN\nbbWO2Q+5twdahhhYjX9I8pg6dIo3rQIwCOEdANDEl8X75XQ6JEkJ/VLUO6L7PqzaKDFmoExqeKmg\nrbRItXU1BlcEoDsivAMAmjh08nP35yEDRhlYif8ICw1XbN9ESZLL5dQx26FWjgCA9kd4BwA0cdAj\nvA9NvtHASvxLWuJI92fPH3AAoLMQ3gEAXiqqyt1zuoOCzBqcOMLgivzHEI8fZAjvAIxAeAcAeDlc\n9IX7c0pcmnqEhhtYjX9JS77BPe/9hP2wqmsqDa4IQHdDeAcAePGa786UGS89e/RWYuwgSZLT5dQ/\nT+0zuCIA3Q3hHQDghfDesqHJlx7gZeoMgM5GeAcAuJWeP62ScpskKSQ4VAMtQw2uyP8w7x2AkQjv\nAAA3z1VmBieOUEhwiIHV+KfBCcNlNgdLkqxnT+h85TmDKwLQnRDeAQBuh07scX9micjmhYaEaZDH\nbyQOMvoOoBMR3gEAkiSXy6VDHivNMN/9ypg6A8AohHcAgKSGKSAVVWWSpIgevZXYf5DBFfmvoQO8\nH1p1uVwGVgOgOyG8AwAkeY8gpyWNVJCJfyKuZEBcmsK+Wv/+XMUZ90O+ANDR+M4MAJDEEpFXwxxk\n1nUeb5496PGsAAB0JMI7AEAOp0OHT+11b3uuZY7mMe8dgBEI7wAAnbAfVk1ttSQpuleM+veJN7gi\n/+f5A87hoi/kdDkNrAZAd0F4BwA0mTJjMpkMrCYwxPcboN4RfSRJlRcrdOrMMWMLAtAtEN4BAF5r\nlQ8ZwHx3X5hMJg1JusG9fegk894BdDzCOwB0c7V1NTpqPeDeHpJEePeV57x3XtYEoDMQ3gGgmztS\nXCiHo16SFNc3SVG9+hpcUeDwXO/9yKn9qnfUGVgNgO6A8A4A3ZznfPehLBF5VfpGxiomyiJJqq2v\n0THbIYMrAtDV+Rzec3JylJqaqvDwcKWnpysvL6/F9nv37tWkSZMUERGh5ORkLVu2zOvr7777rqZO\nnarY2FhFRkbqtttu05/+9Ke23QUAoM0OeszVHsISkVfNa+oM670D6GA+hfcNGzYoOztbS5cuVUFB\ngcaPH69p06apqKio2fYVFRXKzMxUfHy88vPz9eKLL+qXv/ylVqxY4W7zySef6Otf/7o2btyogoIC\nTZ8+XTNmzNCOHTva584AAK2qvFihotNHJEkmU5CuSxrRyhG4HOu9A+hMwb40WrFihebOnau5c+dK\nklauXKkPP/xQq1ev1vLly5u0f/3111VdXa1XXnlFoaGhGjZsmAoLC/X8889r4cKFkqQXXnjB65gn\nn3xS//u//6v33ntPGRkZ13pfAAAf/LNor1xySZIGxA5WRFgvgysKPGkeK84ctx/Wxdpq9QgNN7Ai\nAF1ZqyPvdXV1ys/PV2Zmptf+KVOmaOfOnc0es2vXLk2cOFGhoaHufVOnTlVxcbGOHz9+xWtVVFQo\nOjra19oBANeo8Phn7s9DmO/eJr0jopTYf5Akyel06MtT+wyuCEBX1mp4LykpkcPhUFxcnNf+uLg4\n2Wy2Zo+x2WzNtne5XFc8ZtWqVTp16pQeeOABX2sHAFyDmrqL+sehS88vDRt4s4HVBLahLBkJoJP4\nNG2mo/3xj3/U448/rrffflvJyckttt29e3cnVYVARR+BL+gn0mH7Z7pYWyVJiuzRV+eKq7Tbyt+L\nJ1/7ienipWkyBQd3aUAEv8XoTvh+gitJS0tr93O2OvIeExMjs9ksu93utd9ut8tisTR7jMViaba9\nyWRqcswf/vAHPfjgg3rttdc0ffr0q60fANBGh2yXpsykWW6WyWQysJrAFhs5QEGmhn9Sy6pOq6yq\nxOCKAHRVrY68h4SEaMyYMcrNzdU999zj3p+bm6uZM2c2e8y4ceP005/+VLW1te5575s3b1ZCQoJS\nUlLc7d5++23NmTNHr776qmbMmOFTwenp6T61Q/fTOPJBH0FL6CcNTp4+orM7iiVJZnOw7pnyoHqF\nRxpclf9oSz/Zf+YW7flylySpzHlSk9O/0SG1wX/w/QStKS8vb/dz+rRU5KJFi7R+/XqtXbtWBw4c\n0IIFC2S1WpWVlSVJWrJkiSZPnuxuf9999ykiIkKzZ8/Wvn379M477+i5557TY4895m7z1ltv6f77\n79ezzz6rCRMmyG63y26369y5c+18iwCAy+384s/uzzddN57g3g4m3DjN/flvhVtVU3fRwGoAdFU+\nhfdZs2bphRde0PLlyzV69Gjt3LlTmzZtUlJSkqSGB1SPHj3qbh8ZGanc3FwVFxdr7Nixmj9/vhYv\nXqzs7Gx3m5dfflkOh0PZ2dlKSEhw//Ec3QcAtL+LtdXaffAT93bGDVMNrKbrGJJ8o2KjEyVJF2ur\nlH/wLwZXBKAr8vmB1aysLPdI++XWrVvXZN+IESO0bdu2K55v69atvl4aANCO8g/+xT0qHNc3SYMT\nhhtcUddgMpmUccNUvfuX30mStn++SeNGZPIsAYB25dPIOwCg69ix99KUmfEjpxAu29Gtw+5USHDD\ns16nzhzVMdtBgysC0NUQ3gGgGzlh/6eKTh+RJAWbQ3TLsDsMrqhriejRS2OGfs29vf3zTQZWA6Ar\nIrwDQDeyw+NB1dFpGerZo7eB1XRNEz0eXP3s8A5VVLX/ahMAui/COwB0E9U1Vco/tN29zYOqHSM5\ndrBSLEMkSQ5HvXbt32JwRQC6EsI7AHQTuw9+otqvHlSN7zdAg+KvN7iirstz9H3HFx/K6XQYWA2A\nroTwDgDdgMvl8lrbnQdVO9botAxFfDUlqfT8aRUe/6yVIwDAN4R3AOgGjtsP61TJMUlSSHCoxg6b\nZGg9XV1IcKjGjfi6ezvv8w8NrAZAV0J4B4BuYIdHeLw5bYIiwnoZWE33MH7kpWcK9h/L19lyu4HV\nAOgqCO8A0MVV1VzQPw7nubfH86Bqp+jfJ17DUm6WJLnk8lrpBwDaivAOAF3c9j2bVFdfK0lKiBmo\ngV+thIKON+HGb7g/f7r/I/f/DgDQVoR3AOjCTp05qg//usG9PeGGb/CgaicaMXCMonv3lyRVVp9X\nwT93GlwRgEBHeAeALqquvlav/nmFHM56SdKAuDSNGzHZ4Kq6l6Ags9d6+h/nv6d6R52BFQEIdIR3\nAOiiPtj5uqxnT0hqWP3kganZMpuDDa6q+xk3YrKCzSGSpFMlx/Q/O14zuCIAgYzwDgBd0KGTn2vr\nZ//j3v7OxDmKi040sKLuq3dEH909/n739rbP/kdfHPmbgRUBCGSEdwDoYqpqLuj1zS+6t4en3KwJ\nN3yjhSPQ0e4Y/S2NHDTWvf3G5pUqPX/GwIoABCrCOwB0Mf9v63+r7MJZSVLPHr31vcxHeEjVYCaT\nSd+f8qiie8VIavgBa/2Hv5LDUW9wZQACDeEdALqQ/IPblX/wL+7te78+T1E9+xpYERr17NFbD037\ndwWZGv7pPWY9qA8+fcPgqgAEGsI7AHQR5ypK9PbWNe7tW4d/XaOuu83AinC51ITrvea/b8l/V/uO\n7jawIgCBhvAOAF2A0+XUG7krVV1TKUnqGxmr737tBwZXhebcOeY7Gv7Vm1cl6fXNL+pcRYmBFQEI\nJIR3AAhw5yvPafW7T+vQyc8lSSZTkB6Ykq3wsAiDK0NzgkxBun9qtqJ69ZMkVV6s0Csf/pccTofB\nlQEIBIR3AAhghcc/03NvZOvgyT3ufZPHzNDgxOEGVoXW9AqP1OxvLHLPfz9SXKhXP3xeldXnDa4M\ngL8jvANAAKp31On9vPVa/d7TqqgulySZZNKUsTN117j7DK4OvhicOELTb/uee/uzwzv0f1+br88O\n75TL5TKwMgD+jFftAUCAKSm36ZVN/6Xj9sPufZER0XpgaraGDhhlYGW4WpPH3qOScpt27d8iSaqo\nLte6jf+pGwffppl3/JCVggA04fPIe05OjlJTUxUeHq709HTl5eW12H7v3r2aNGmSIiIilJycrGXL\nljVp88knnyg9PV3h4eG67rrr9PLLL1/9HQBAN1FbV6Nd+7boP3+/yCu4D0u5WY9/fwXBPQAFmYJ0\nX+Z8/es3f+YV1D//cpf+72vztWvfFkbhAXjxaeR9w4YNys7O1po1a5SRkaFVq1Zp2rRpKiwsVFJS\nUpP2FRUVyszM1KRJk5Sfn6/CwkLNnj1bvXr10sKFCyVJx44d01133aWHH35Yb7zxhrZv364f//jH\nio2N1YwZM9r3LgEgQNXW16jw2Gf67PAO7T36d9XWXXR/LSjIrG+Of0B33Pwt99xpBKYbUm/RdYkj\n9H7eeu3cmytJqq6p1O8/+rXyD/5F42+YoqHJoxTRo5fBlQIwmsnlw4/0t912m2666SatWXNp/eAh\nQ4Zo5syZWr58eZP2q1ev1pIlS3T69GmFhoZKkpYvX641a9bo5MmTkqTHH39c7733ng4ePOg+7l//\n9V+1f/9+7dixw+t85eXl7s9RUVFXeYvoLnbvblgrOT093eBK4M8CoZ9U11TqcNHehsB+5G+q8Qjs\njfpFxmn2tMeUYhliQIVdn5H95NDJz/XmllU6W2732m8yBSklLk3Xp9ykYSmjNSAuTeYgc6fXh0sC\n4fsJjNURGbbVkfe6ujrl5+dr8eLFXvunTJminTt3NnvMrl27NHHiRHdwl6SpU6fqySef1PHjx5WS\nkqJdu3ZpypQpXsdNnTpVr776qhwOh8xmviEB6FocTodq6qpVU1uti7UXVXXxvErKbQ1/ymzuz5UX\nK654jtjoRN08ZILuGP0thYf17MTq0VmGJN+oJd9fqf/99A1tK/hALpdTkuRyOXXMdlDHbAf14V83\nKDysp1Ljh6lP7xhF9YxWVK9+iurZV3169VVUz76K6NFbJpPJ4LsB0N5aDe8lJSVyOByKi4vz2h8X\nF6ctW7Y0e4zNZlNycnKT9i6XSzabTSkpKbLZbMrMzGzSpr6+XiUlJU2u1+jl//k/rZWMbqq8rOGn\n2/ziDw2uBP6srKxMkpR/qmk/cemyX0R6/GLS1fh1l+ur/zZsu77adjmdcjgdcjodcri++q/TIYez\nXjW1F1VTV626+to21dy/T4JGp2VodFqGEmJSCGTdQGhImGZ8ba5uHX6nPju8UweOf6YT9n969dHq\nmkrtO9by21mDzSEKCQ5ViDlUIcGhCg5u2DYHBctkMinIFCSTKeir/zZs66v+ZVLjf9VkX7O6Ybfk\n3x00umXYHRqdltEp1wq41WbuvX2+0SUAgGHOn2cd8M6QlpYmyftX3kboGRKtCcPv0oThdxlaB4DW\nddb3i1afcIqJiZHZbJbd7j33zm63y2KxNHuMxWJptr3JZHIfc6U2wcHBiomJuaqbAAAAALqDVsN7\nSEiIxowZo9zcXK/9ubm5ysho/tcD48aN0/bt21Vbe+lXxJs3b1ZCQoJSUlLcbS4/5+bNm5Wens58\ndwAAAKAZPq028/bbb+vBBx/UqlWrlJGRodWrV2vdunXav3+/kpKStGTJEv3973/XRx99JKnh17rX\nX3+9Jk2apCeeeEIHDx7UnDlz9PTTTys7O1tSw1KRN9xwgx5++GH96Ec/Ul5enh555BG99dZb+s53\nvtOxdw0AAAAEIJ/mvM+aNUulpaVavny5rFarRo4cqU2bNrnXeLfZbDp69Ki7fWRkpHJzczVv3jyN\nHTtW0dHRWrx4sTu4S9LAgQO1ceNGLVy4UGvWrFFCQoJ+/etfE9wBAACAK/Bp5B0AAACA8QLilXw5\nOTlKTU1VeHi40tPTlZeXZ3RJMMgzzzyjW265RVFRUYqNjdW3vvUt7du3r0m7X/ziF0pMTFRERITu\nuOMO7d+/34Bq4S+eeeYZBQUF6dFHH/XaTz+BzWbT7NmzFRsbq/DwcI0cOVLbt2/3akM/6d6cTqd+\n/vOfu3NIamqqfv7zn8vpdHq1o590L9u3b9e3v/1tJSUlKSgoSK+++mqTNq31idraWs2fP1/9+/dX\nr1699O1vf1unTp1q9dp+H943bNig7OxsLV26VAUFBRo/frymTZumoqIio0uDAf7yl7/okUce0aef\nfqqtW7cqODhYkydPdq/dLUnPPfecVqxYoVWrVmn37t2KjY1VZmamKisrDawcRtm1a5d+85vfaNSo\nUV776ScoLy9XRkaGTCaTNm3apAMHDujXv/61YmNj3W3oJ3j22We1evVqvfTSSzp48KBWrlypnJwc\nPfPMM+429JPu58KFC7rhhhu0cuVKRURENPm6L31iwYIFevfdd7Vhwwbl5eXp/Pnzuvvuu9XqpBiX\nn7v11ltdP/rRj7z2paWluX72s58ZVBH8yYULF1xms9n1wQcfuPfFx8e7nnnmGfd2dXW1q3fv3q7/\n/u//NqJEGKisrMw1ePBg17Zt21yTJk1yzZ8/3/01+gmWLFnimjBhQott6Ce4++67XbNnz/ba99BD\nD7m++c1vurfpJ91br169XK+88orXvtb6RHl5uSs0NNT15ptvutucPHnSFRQU5Nq8eXOL1/Prkfe6\nujrl5+c3eRPrlClTtHPnToOqgj85f/68nE6noqOjJUlHjx5t8vbeHj166Gtf+xp9phv64Q9/qFmz\nZun222/32k8/gSS9//77uvXWW3XvvfcqLi5Oo0eP1qpVq9xfp59AkiZMmKCtW7fq4MGDkqT9+/fr\n448/1l13Nbw4i36Cy/nSJ3bv3q36+nqvNklJSRo2bFir/cav37BaUlIih8OhuLg4r/1xcXHasmWL\nQVXBnyxYsEA333yzxo0bJ6lh/qrJZGq2zxQXFxtRIgzym9/8RkeOHNGbb77Z5Gv0E0jSkSNHlJOT\no4ULF2rJkiUqKCjQI488IpPJpB//+Mf0E0iSHn/8cVVUVGj48OEym81yOBx64okn9KMf/UgS30/Q\nlC99wm63y2w2q1+/fk3a2Gy2Fs/v1+EdaMmiRYu0c+dO7dixQyaTyehy4EcOHTqkJ554Qjt27FBQ\nkF//ghEGcjqduuWWW7R8+XJJ0qhRo3To0CGtWrVKP/7xjw2uDv7irbfe0muvvaa33npLw4cPV0FB\ngR599FENGjRIc+bMMbo8dEN+/a9aTEyMzGaz7Ha713673S6LxWJQVfAHCxcu1IYNG7R161b3W3sl\nyWKxyOVy0We6uU8//VRnz57V8OHDFRISopCQEH3yySdatWqVQkND1a9fP/oJFB8fr2HDhnntGzZs\nmE6cOCGJ7ydo8JOf/ESLFy/WzJkzNWLECH3/+9/XokWL3A+s0k9wOV/6hMVikcPh0NmzZ6/Y5kr8\nOryHhIRozJgxys3N9dqfm5urjIwMg6qC0RYsWOAO7mlpaV5fGzRokCwWi1efuXjxorZv306f6UZm\nzJihL774Qnv27HH/SU9P1/e+9z3t2bNHQ4YMoZ9AGRkZ7nnMjQ4ePOgeEOD7CSSpqqqqyW/wgoKC\n3EtF0k9wOV/6xJgxYxQcHOzVpqioSIWFha32G/MvfvGLX3RI5e0kMjJSTz31lOLj4xUREaFly5Zp\n+/bt+t3vfqeoqCijy0Mnmzdvnl599VX94Q9/UFJSkiorK1VZWSmTyaTQ0FBJksPh0LPPPquhQ4fK\n4XBo0aJFstvtevnll91t0LWFhYWpf//+Xn9+//vfa+DAgXrwwQcl0U8gpaSk6D/+4z9kNpuVkJCg\nLVu2aOnSpfrZz36m9PR0SfQTSIWFhXrttdc0dOhQhYaGauvWrXriiSf0ve99z/2wIf2k+6msrFRh\nYaFsNpvWrl2rG2+8UVFRUaqrq1NUVFSrfSIsLExWq1WrVq3SjTfeqPLycv3bv/2boqOj9eyzz7Y8\nHbh9FsnpWKtXr3YNGjTI1aNHD1d6erorLy/P6JJgEJPJ5AoKCmry5+mnn/Zq9/TTT7sSEhJc4eHh\nrkmTJrn27dtnUMXwF3fccYfXUpEuF/0ELtfGjRtdo0aNcoWHh7uGDh3qeumll5q0oZ90bxcuXHAt\nXLjQNXDgQFdERIRr8ODBrqVLl7pqamq82tFPupdt27Y1m0nmzJnjbtNan6itrXU9+uijrpiYGFfP\nnj1d3/72t11FRUWtXtvkcrW2EjwAAAAAf+DXc94BAAAAXEJ4BwAAAAIE4R0AAAAIEIR3AAAAIEAQ\n3gEAAIAAQXgHAAAAAgThHQAAAAgQhHcAAAAgQBDeAQAAgADx/wF03Eun5ILTOAAAAABJRU5ErkJg\ngg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt \n", + "xs = list(range(0,100))\n", + "ps = suite.Probs(xs)\n", + "plt.plot(xs, ps);" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Summarizing the posterior\n", + "\n", + "Again, there are several ways to summarize the posterior distribution.\n", + "One option is to find the most likely value in the posterior\n", + "distribution. `thinkbayes` provides a function that does that:\n", + "\n", + "```python\n", + "def MaximumLikelihood(pmf):\n", + " \"\"\"Returns the value with the highest probability.\"\"\"\n", + " prob, val = max((prob, val) for val, prob in pmf.Items())\n", + " return val\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "56" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "suite.MaximumLikelihood()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this case the result is 56, which is also the observed percentage of\n", + "heads, $140/250 = 56\\%$. So that suggests (correctly) that the observed\n", + "percentage is the maximum likelihood estimator for the population.\n", + "\n", + "We might also summarize the posterior by computing the mean and median:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Mean 55.952380952380956\n", + "Median 56\n" + ] + } + ], + "source": [ + "import thinkbayes\n", + "print('Mean', suite.Mean())\n", + "print('Median', thinkbayes.Percentile(suite, 50))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we can compute a credible interval:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CI (51, 61)\n" + ] + } + ], + "source": [ + "print('CI', thinkbayes.CredibleInterval(suite, 90))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, getting back to the original question, we would like to know\n", + "whether the coin is fair. We observe that the posterior credible\n", + "interval does not include 50%, which suggests that the coin is not fair.\n", + "\n", + "But that is not exactly the question we started with. MacKay asked, “ Do\n", + "these data give evidence that the coin is biased rather than fair?” To\n", + "answer that question, we will have to be more precise about what it\n", + "means to say that data constitute evidence for a hypothesis. And that is\n", + "the subject of the next chapter.\n", + "\n", + "But before we go on, I want to address one possible source of confusion.\n", + "Since we want to know whether the coin is fair, it might be tempting to\n", + "ask for the probability that x is 50%:\n", + "\n", + "```python\n", + "print(suite.Prob(50))\n", + "```\n", + "\n", + "The result is 0.021, but that value is almost meaningless. The decision\n", + "to evaluate 101 hypotheses was arbitrary; we could have divided the\n", + "range into more or fewer pieces, and if we had, the probability for any\n", + "given hypothesis would be greater or less." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Swamping the priors\n", + "\n", + "We started with a uniform prior, but that might not be a good choice. I\n", + "can believe that if a coin is lopsided, $x$ might deviate substantially\n", + "from 50%, but it seems unlikely that the Belgian Euro coin is so\n", + "imbalanced that $x$ is 10% or 90%.\n", + "\n", + "It might be more reasonable to choose a prior that gives higher\n", + "probability to values of $x$ near 50% and lower probability to extreme\n", + "values.\n", + "\n", + "As an example, I constructed a triangular prior. Here’s the code that constructs the prior:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def TrianglePrior(suite):\n", + " for x in range(0, 51):\n", + " suite.Set(x, x)\n", + " for x in range(51, 101):\n", + " suite.Set(x, 100-x) \n", + " suite.Normalize()\n", + " return suite" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvgAAAEWCAYAAAAThOOMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlYU2faBvA7CYRNobgByqYIiKKAgIRNUAQURdAWq1gR\nnXbq1JnWttPFqf2m1TqtYxdtx6XtjPterSCugCurAgqKIqIoboAbAiqy5vsDTY2ARAUSwv27rl6X\nffOec57UU3hy8t7nCKRSqRRERERERKQWhMougIiIiIiIWg4bfCIiIiIiNcIGn4iIiIhIjbDBJyIi\nIiJSI2zwiYiIiIjUCBt8IiIiIiI1wgafiIiIiEiNKNzgL126FH369IGOjg5cXFyQmJj4zPnZ2dnw\n9fWFrq4uzMzMMG/ePLnXt2/fjsDAQPTo0QP6+vqQSCSIiYmRm7N69WoIhUKIRCIIhULZn6uqqp7j\nLRIRERERdRwKNfibN2/GrFmzMGfOHGRmZsLDwwOjRo3C1atXG51fXl4Of39/mJiYICMjA4sXL8bC\nhQvxww8/yOYcPnwYfn5+2L17NzIzMxEUFIRx48YhKSlJbl96enooKiqS/VNYWAixWPwSb5mIiIiI\nSH0JFHmSrUQigaOjI5YvXy4bs7GxQVhYGObPn99g/rJlyzB79mzcuHFD1ozPnz8fy5cvx5UrV5o8\njpubG4YOHYqFCxcCqL+C/7e//Q1lZWXP/caIiIiIiDqiZq/gV1dXIyMjA/7+/nLjAQEBSE5ObnSb\n1NRUeHt7y11pDwwMxPXr11FQUNDkscrLy2FoaCg3VlFRAUtLS5iZmSE4OBiZmZnNlUxERERE1GE1\n2+DfunULtbW1MDIykhs3MjJCUVFRo9sUFRU1Ol8qlTa5zZIlS3Dt2jVMmTJFNmZra4sVK1Zgx44d\n2LRpE7S1teHp6YkLFy40+8aIiIiIiDoiDWUXAADbtm3DJ598gi1btsDMzEw2LpFIIJFIZP/u7u4O\nJycn/PTTT1i0aJHcPkpLS9usXiIiIiKilmZgYNAi+2n2Cn63bt0gEolQXFwsN15cXAxjY+NGtzE2\nNm50vkAgaLDN1q1bERERgbVr1yIoKOjZxQqFcHZ2Rl5eXnNlExERERF1SM02+JqamnB2dkZcXJzc\neFxcHDw9PRvdxt3dHQkJCXK3s4yNjUXPnj1hYWEhG9uyZQumTp2KNWvWYNy4cQoVnJWVBRMTE4Xm\nEhERERF1NAot0fnggw8QEREBV1dXeHp6YtmyZSgsLMSMGTMAALNnz0ZaWhri4+MBAOHh4Zg7dy4i\nIyPx2WefITc3FwsWLMCXX34p2+emTZsQERGB7777Dl5eXrIr/mKxWBa0nTt3LiQSCaytrVFWVobF\nixcjOzsbv/zyyzPrbamvN0j9pKenAwBcXFyUXAmpMp4npAieJ9QcniOkiNZYZq5Qgz9hwgTcuXMH\n8+fPR2FhIezt7bFnzx6YmpoCqA/VXrx4UTZfX18fcXFxmDlzJlxdXWFoaIiPPvoIs2bNks35+eef\nUVtbi1mzZsmN+/j44MCBAwCAu3fv4u2330ZRUREMDAzg5OSEhIQEODs7t8ibJyIiIiJSNwrdB789\nePLTD6/gU1N4NYUUwfOEFMHzhJrDc4QU0Ro9rEJPsiUiIiIiovaBDT4RERERkRphg09EREREpEZU\n4kFXRERERB2FVCpFdXU16urqlF0KtTKhUAhNTU0IBII2PS4bfCIiIqI2IpVK8fDhQ4jFYqU0ftR2\npFIp6urq8PDhQ2hra7fp3zWX6BARERG1kerqaojFYohEIjb3ak4gEEAkEkEsFqO6urpNj80Gn4iI\niKiN1NXVQShk+9WRCIXCNl+OxTOMiIiIqA3xyn3Hooy/bzb4RERERERqhA0+EREREZEaYYNPRERE\nRCpn1apVEAqFuHz5stz45s2b0b9/f2hpaaFLly5Kqk618TaZRERERKRyBAJBg/Xrly5dwpQpUzB8\n+HB88skn0NPTU1J1qo0NPhERERGpnIiICEyaNAlisVg2lpSUhNraWnz77bewt7dXYnWqjUt0iIiI\niEjlCAQCueYeAIqLiwEA+vr6LXacioqKFtuXqmCDT0REREQvLTIyEr17924w/sUXX8jd+18oFOKd\nd95BdHQ0Bg4cCG1tbdjb22Pfvn1y2z29Br937974+9//DgCwtLSEUCjE3LlzZfOXL1+OgQMHQkdH\nByYmJpgxYwZKSkrk9unr64v+/fsjKysLw4YNQ6dOnTBz5ky517Kzs+Hr6ws9PT1YWVlhy5YtAIDE\nxES4u7tDV1cX/fr1Q2xsbAv8V2sdbPCJiIiI6KU1tma+qfHk5GT89a9/xaRJk7Bw4UJUVlbitdde\nk2vIn95u8eLFmDRpkuzP69atw/jx4wEAX331Fd555x2YmJjg22+/xaRJk7BixQr4+fnJPUVWIBCg\npKQEI0eOxIABA7Bo0SIEBQXJXrt79y7GjBmDIUOGYOHChdDT08Mbb7yBzZs3IywsDEFBQViwYAEe\nPHiACRMmoLy8vOX+A7YgrsEnIiIiojZ19uxZ5OTkyK74+/r6wsHBARs3bsQ777zT6DZjx45FXl4e\nNm3ahJCQEJibmwMAbt26ha+++gr+/v7Yu3ev7EOBg4MDpk2bhl9//VVunzdu3MCPP/4ou3L/pOLi\nYqxduxbh4eEAgBEjRqBfv36YPHkyEhMTIZFIAAD9+vVDYGAgfvvtN0yfPr3l/sO0EF7BJyIiIlJR\nX3zxhexKdmv888UXXyjlfQ0fPlxuOc/AgQOhr6+P/Pz8595XfHw8qqur8d5778ld8Z8yZQqMjIyw\na9cuufkaGhp46623Gt2Xjo6OrLkHABsbG7zyyiuwsbGRNfcA4ObmBgAvVG9bYINPRERERG3KzMys\nwZihoWGDNfOKKCgoAFDfjD9JKBTC2toaly5dkhvv2bNng/DuY7169WowZmBg0KDexyHfF6m3LbDB\nJyIiIqKX1tj6ewCora1tMCYSiRqdK5VKW7Smxujo6DT5WlN1KbPeF8EGn4iIiEhFffHFF5BKpa32\nT0su0TE0NMTdu3cbjD99Bb2lWVhYQCqVIjc3V25cKpUiLy8PlpaWrXp8VcQGn4iIiIhempWVFUpL\nS5GdnS0bKywsRFRUVKse19/fH2KxGD/++KPcFfV169ahuLgYwcHBrXp8VcQGn4iIiIhe2sSJE6Gr\nq4vQ0FD8+OOP+PrrryGRSGBra9uqx+3atSs+//xzxMfHIyAgAEuWLMEHH3yAN998E05OTvjTn/7U\nqsdXRWzwiYiIiOildenSBVFRUdDT08Mnn3yCtWvX4ptvvsGYMWPk5j3P/fIV9dlnn2HZsmUoKirC\n3//+d2zcuBHTp09HfHw8NDU1GxynKc9T18vU29oEUlVNBzyn0tJS2Z8NDAyUWAmpsvT0dACAi4uL\nkishVcbzhBTB84Sa09g58vDhQ2hrayurJFKSZ/29t0YPyyv4RERERERqhA0+EREREZEaYYNPRERE\nRKRG2OATEREREakRNvhERERERGqEDT4RERERkRphg09EREREpEbY4BMRERERqRE2+EREREREaoQN\nPhERERGRGmGDT0RERESkRtjgExERERGpEYUb/KVLl6JPnz7Q0dGBi4sLEhMTnzk/Ozsbvr6+0NXV\nhZmZGebNmyf3+vbt2xEYGIgePXpAX18fEokEMTExDfazbds2DBgwANra2rC3t0dUVJSiJRMRERGR\nivP19cWwYcOUXYaMr68vhg8fruwyXopCDf7mzZsxa9YszJkzB5mZmfDw8MCoUaNw9erVRueXl5fD\n398fJiYmyMjIwOLFi7Fw4UL88MMPsjmHDx+Gn58fdu/ejczMTAQFBWHcuHFISkqSzUlJScHEiRMx\nZcoUZGVlITw8HGFhYUhLS3vJt01ERERELS0lJQVffvklysrKFN5GIBBAKFSdRSUCgUDZJbw0gVQq\nlTY3SSKRwNHREcuXL5eN2djYICwsDPPnz28wf9myZZg9ezZu3LgBsVgMAJg/fz6WL1+OK1euNHkc\nNzc3DB06FAsXLgQATJw4ESUlJdi3b59sjr+/P3r06IH169fLbVtaWir7s4GBQXNviTqo9PR0AICL\ni4uSKyFVxvOEFMHzhJrT2Dny8OFDaGtrK6ukVvfdd9/h448/xsWLF2Fubq7QNjU1NQAADQ2N1ixN\nYcOGDYNAIMCBAwdabJ/P+ntvjR622Y9L1dXVyMjIgL+/v9x4QEAAkpOTG90mNTUV3t7esuYeAAID\nA3H9+nUUFBQ0eazy8nIYGhrK/j0lJQUBAQFycwIDA5s8LhEREREpjwLXjWUqKioA1Df2qtLcq4tm\nG/xbt26htrYWRkZGcuNGRkYoKipqdJuioqJG50ul0ia3WbJkCa5du4YpU6Y0u5+m9kFERC2rqqYS\nsWlbMXfVDPwa8y9cv9X0RRoi6ti+/PJLfPzxxwAAS0tLCIVCiEQiHD58GJaWlggKCsKBAwcgkUig\no6MjW7HR2Jr37777Dt7e3ujevTt0dHQwaNAg/O9//2twzMf7TUpKgpubG3R0dGBlZYW1a9c2mHvy\n5En4+PjI8qHz58/HihUrIBQKcfny5Wbf308//YRBgwZBR0cHRkZGePPNN3H79u0X+U/V6lTi49K2\nbdvwySefYMuWLTAzM3vp/T3+SoyoKTxHSBEd+TyRSqXIv3kKJwoO4UFV/VraW6VFyM5Pg5WRAxzN\nfaAr7qzkKlVDRz5PSDFPniMWFhZqu0Tn1Vdfxblz57Bp0yYsXrwYXbt2hUAggJ2dHQQCAfLy8hAW\nFoa33noLb775pmwJT2Nr3hctWoTg4GC8/vrrEAgEiI6OxltvvYXa2lr8+c9/ls0TCATIz89HWFgY\n/vSnPyEyMhIrVqzAtGnT4OLiAjs7OwDA9evXMWzYMAiFQvzjH/+Anp4e/vvf/0JTU1OhNfczZszA\nypUrERkZib/97W+4cuUKfvzxR6SlpSEtLU1u1UpjysvLkZ2d3ehr1tbWzR7/eTXb4Hfr1g0ikQjF\nxcVy48XFxTA2Nm50G2Nj40bnCwSCBtts3boVU6dOxbp16xAUFKTQfpo6LhERvbyiu5eQfiked+43\n/LZUCinOF2fi0s3TGNDLHf17SaApevYvNiJ6cbtTN2Lv0c2ttv+Rbq8jSDKpRfZlb2+PwYMHY9Om\nTQgJCWmwBj8/Px87duzA6NGjm91XXl6e3AehmTNnIjAwEN9++61cg/947pEjR+Dp6QkACAsLg5mZ\nGVauXIl///vfAIBvvvkGd+/eRXp6OpycnAAA06ZNQ9++fZutJTk5Gb/88gvWrl2LyZMny8ZHjhwJ\nLy8vrFmzBm+++Waz+2lLzTb4mpqacHZ2RlxcHF599VXZeFxcHMLCwhrdxt3dHZ9++imqqqpkn2hi\nY2PRs2dPWFhYyOZt2bIF06ZNw5o1azBu3LhG9xMXF4cPP/xQ7rgeHh7PrJmBJ2oKQ3GkiI56nhTd\nuYIdiWuQfVH+TmWddQww3Hkczl05iZyC4wCAmrpqZF05got3TmG0JBxu/YdDKBQpo2yl6ajnCSmu\nqZBtR2VqaqpQcw9A1tzX1NSgvLwcdXV18PX1RXx8PMrLy9G58x/fINrY2Miae6D+4rStrS3y8/Nl\nY/v27cOQIUNkzT0AvPLKK5g8eTL+85//PLOWLVu2oHPnzggICJBbkmNjYwMjIyMcPHiw2Qa/c+fO\nTf6seDJk21IUWqLzwQcfICIiAq6urvD09MSyZctQWFiIGTNmAABmz56NtLQ0xMfHAwDCw8Mxd+5c\nREZG4rPPPkNubi4WLFiAL7/8UrbPTZs2ISIiAt999x28vLxkV+rFYrEsaPvee+/Bx8cHCxYsQGho\nKH7//XccOnRI7laaRET0csof3MWe1E1Izo5FnbRONq6pIcbwwSHwcx4PbbEO/JxDkVNwAtEJq3D9\ndv1a/LL7Jdi4fwkOZ+5EiHck7CycmjoMEXVwffr0UXhudHQ0vvrqK2RmZqK2tlY2LhAIUFpaKtfg\nN3a3HkNDQ5SUlMj+vaCgAEOGDGkwT5Er+Hl5eSgvL2+QC31cz40bN5rdR1tTqMGfMGEC7ty5g/nz\n56OwsBD29vbYs2cPTE1NAdSHYS9evCibr6+vj7i4OMycOROurq4wNDTERx99hFmzZsnm/Pzzz6it\nrcWsWbPkxn18fGS3JXJ3d8emTZswZ84c/POf/4SVlRW2bNnCqyVERC2gqroSh07sQFzG76isqpCN\nCyCAq50vRrtPhmHnbnLb2Fk4wdZsEI7mHMSulPUou1//C/T67QIsi/oS/cwdEeIViV7dLdvyrRCp\nrSDJpBZbQqNsOjo6Cs1LTEzE+PHjMXToUPz888/o2bMnxGIxdu3ahUWLFqGurk5uvkjU+LeHz3NH\nn2epq6tDt27dsHnz5kb3+eQdIFWFwiHbGTNmyK7YP23lypUNxgYMGIBDhw41ub+DBw8qdNzx48dj\n/PjxCs0lIqLm1UnrkH72MHYmr8Pde/J3gLAxHYgQ72kw69H0lTahUAT3ASMw2MYLB45HY3/GdlRV\n1y87OHs5E7kb3odb/+EY7T4ZBp26tOp7ISLV0hIPidq2bRt0dHQQGxsLTU1N2fj+/ftfeJ8WFhY4\nf/58g/G8vLxmt7WyskJ8fDzc3Nygq6v7wjW0JdV5bBgREbW6c1dOYuHGD7EudrFcc2/cxQxvj52D\nmePnPrO5f5KWpjZGub2Oz6cuhYe9PwSC+l8pUkiRemY/5q3+C3anbpT7doCI1Juenh4AyC2PeV4i\nkQgCgUBuaU5JSUmjF5QVFRgYiGPHjuH48eOysTt37mDDhg3Nbvv666+jtrYWc+fObfBaXV0d7t69\n+8J1tRaVuE0mERG1rsLbV7AjcTVOX5K/pWNn3VcQJJkEyYAREL1gSNZArwsm+s3EUIcx2JG4Gmce\nBXGraiqx9+hmJJ+KRZB7OCQdMIhL1NG4uLhAKpXi008/RXh4OMRicYN73DcnODgY33//PUaMGIEp\nU6bg9u3b+O9//wsTE5MGd1dU1Mcff4x169YhMDAQ7777ruw2mebm5igpKXnmNw/e3t6YOXMmFi5c\niKysLAQGBkJLSwt5eXnYtm0b5s2bh4iIiBeqq7WwwSciUmNl9+9iT+pGJJ+Og7RBgDYUfs7joC1W\nbF1sc3p2s8CM0P/D2YJMRCWuwvVbl+preFCCTfuX4HBmDEK8ItHfcnCLHI+IVI+zszO++eYbLF26\nFNOnT0ddXR0OHjwIgUDwzCb6ydd8fHywZs0afP3113j//fdhamqK9957DwYGBvjTn/7UYLum9vvk\nuKmpKQ4dOoR3330XX3/9Nbp164a//OUv6NSpE2bNmtXg2QRP7/Onn36Cs7Mzli9fjjlz5kBDQwPm\n5uaYOHHic3+AaQsCaUslEJTsyVsMGRgYKLESUmW8rR0pQh3Ok6rqShw8EY349N9RWf3HbfkEEGBI\n/+EY7R6OVzp1bbXj19XVIu3sIexMXo/S+3fkXrM1d0CoVyR6de/dasdvC+pwnlDrauo2mer6oKv2\naNasWfj1119x7969FskPNOVZf++t0cPyCj4RkRqpb6wPY2fKepQ+HaA1G4RQ70iYdlf8VnUvSigU\nwa2/HxytPXHweDTinwji5l7Owr83fNAmHzSIiB57usm+ffs21q1bB29v71Zt7pWBDT4RkZrIvZyF\nqMRVuHbzoty4cRczhHhNRX9L5zb/JaalqY2Rbq/Dwz4Au1M3IOX0fkildZBCiqNn9uPEucRHS4VC\nodVCS4WIiBrj7u4OX19f2NnZoaioCCtWrEB5eTk+//xzZZfW4tjgExG1c4W3LyM6cTXOXMqQG++s\n+wpGu4fDrb/fCwdoW4q+niEm+s2Ej2OwXK1VNZXYe2wzkrL3YbR7OCT9/RjEJaJWMXr0aGzduhW/\n/vorBAIBnJ2dsXLlSrmn4KoLrsGnDoVrZkkR7eU8Kbtfgt2pG5FyOr7RAO0I53Eqe1W8qW8bTLqa\nI8RrKuwsBqv8V+bt5Twh5eEafHqMa/CJiOiZKqsfNljXDtQHaN36D0dQO1jXbmvugI8mftsgL1B4\n+zKWR89TmyAuEZEysMEnImon6upqcSznEHalqMedaeqDuMPhZO2Jgyd2ID59m+yOPwziEhG9ODb4\nRETtQO7lLEQlrMS1R/eWf6x+SUsk7CycVH5JS1PEmloIHBIG9wEjsCd1k+ye/Y+DuMfPJbT4PfuJ\niNQZG3wiIhV2/VaB3NNhH9PXNUSQezjc+g9XeoC2pejrGeJ1v79gqOMYuafuVtdUYd+xLUjOjn3p\np+4SqQKpVNpuP5DT81NG3JUNPhGRCiq9fwd7UjfKbiv5mFhDC8OdQ+E3WH1vK2nS1Qxvh8xpEMQt\nf3AXmw8sw+HMnUq77SfRyxKLxXj48CHEYjFEIn5QVXe1tbWoqqqClpZWmx6XDT4RkQqprH6IA8ej\nsb+xAO0AP4yWhMOgUxclVth2bM0d8NGk75B+9jBiktfJgrhFd67g5x1ftemDu4hailAohLa2Nqqq\nqlBdXa3scqiVCQQCaGtrt/nFCDb4REQqoK6uFkdzDmJXynqU3S+Re62fuSNCvCLRq7ulcopTIqFA\niCF2w+DY1wOHTuxA3BNB3HNXTmLhhg8xxG4YgtzDYdi5m5KrJVKMQCBo8yu61LGwwSciUrKcghOI\nTliF67cL5MZNupoj1Hsa7CyclFSZ6hBraiFgSBgkA/yx9+gmJGfHou5xEDfnAI7nJWL44BD4OY9n\nEJeIOjw2+ERESnL91iVEJa7G2YITcuP6uoaPnkA7nE91fYq+3iuYMHwGvB1GY0fSapy++GQQ9zck\nn4pFkHs4g7hE1KGxwSciamOl9+9gV8oGHD1zoEGA1s95HIYPDlHbAG1LMelqhrfHzsG5KycRlbAK\nV2/mAwDKK0qx+cAyHMqMQahXJIO4RNQhscEnImojldUPcSAjqj5AW1MpGxcIhHB79EAnA72OEaBt\nKTZmg/D3Sd8i/exh7Exeh7uPgrjFd67WB3FNByLEexrMejCIS0QdBxt8IqJWVldXi6NnDmBX6oYG\nAVo7i8EI8YpAz26WyilODciCuNYeOHQipj6IW1UBADh39RS+3fghXO18Mdo9HIaduyu5WiKi1scG\nn4ioFeUUnEBUwkoU3r4sN96zqwVCvCMZoG1BYg0tBLi+Vv9E3KObkXxqnyyIeyznIE6cS8KwwSHw\ncx4HHS1dZZdLRNRq2OATEbWCazcvITpxFc5ezpQb19czxGj3yXCzG8YAbSvprPsKJgx7Gz4OoxGd\ntAbZ+ccAANW1VYhN+w0p2bEYJZkEd3t/BnGJSC2xwSciakGl9+5gV8r6+gAt/ng8uVhT+48Araa2\nEivsOIy6mOLPwf9A3tVT2J6wEldv/BHE3XJwueyJuAN6uzCIS0RqhQ0+EVELqKyqwP7jUTiQEdUg\nQCvp74cg90kM0CqJtelA/H3it8jIPYKdSetQcu8WAKC45Cp+iZkPa9OBCPWOhFkPKyVXSkTUMtjg\nExG9hLq6WqSeOYDdKRtQ9kA+QNvfYjDGek1Fz24WSqqOHhMKhHDt5wuHvu44fGInYtO3yoK4eVdP\nYeHGD+HazxdjPCYziEtE7R4bfCKiF3Tm0nFEJ65qGKDtZolQr0j0s3BUUmXUFLGGFvxdX4VkwAjs\nPboZSaf2ou7RswjSzh5CZl4yhg0eCz/n8QziElG7xQafiOg5NRWgNdDrgjEek+Haz5cBWhXXWdcA\nYcP+jKGOo7EjcTVOyQVxtyI5Ow6jJBPhYR/AIC4RtTts8ImIFPSsAO0I53EYxgBtu2Nk2AtvBf8D\neVezEZWwElduXAAA3KsoxW8Hf8aRzF0Y6xUB+96uDOISUbvBBp+IqBmVVRXYnxGFA8cbBmjdB4xA\nkGQS9PUMlVghvSxrU3t8OHEhMnITsDN5HUrKbwKoD+L+GvMv9DW1R6hXJMyN+iq5UiKi5rHBJyJq\nQm1dLY6e2Y9dKRtQ/uCu3Gv9LZ0R4jUVJl3NlVQdtbT6IK4PHPpKcDhzF+LStuJh1QMAwPmr2fh2\n09/h0s8HY9zfQBd9BnGJSHWxwScieopUKsX1uxcQt2FNgwBtr26WCPWeBltzByVVR61NrKEFf5fx\nkPT3axDETT97GJl5yfB1Gotuot4Qa3BJFhGpHjb4RERPuHbzIuJPb0Bh6UW5cYNOXTHGfTJc+/kw\nQNtBNBXEramtRnz6Nmhp6MLBfCicah0hEvHXKRGpDv5EIiICcPfebexK2YBjTwVotTS1McLlVQxz\nGguxppYSKyRleTKIG52wCpdvnAcAVNY8wLH8vbi0/hRCvKYyiEtEKoMNPhF1aA+rKrA/YzsOHI9C\ndU2VbFwAATzsAzBKMpEBWgJQH8T9YOK/cTw3ATFPBHFvlFyrD+L2GoBQ72kM4hKR0rHBJ6IOqbau\nFqmn47E7dWODAG0vw75wtvTDiKGjlFQdqSqhQAiXfj5w6OuO9TuX49TVJFTX1t9Z6fy10/VBXFsf\njPGYjC76PZRcLRF1VGzwiahDkUqlyCk4jujE1Q0DtN17I9QrEuU3qpVUHbUXmhpi2Jt6oK+RI4oq\nc5F4ai/q6moBAOm5h5F5Phm+jsHwd30VOlp6Sq6WiDoaNvhE1GFcvZmP6ITVyL2SJTdu0Kkrgj3e\ngEs/HwgFQqTfSFdShdTeaGvq4jX3tzDUIQg7ktbg5IWjAB4FcTN+R8qZeIxymwhP+wAGcYmozQgV\nnbh06VL06dMHOjo6cHFxQWJi4jPnZ2dnw9fXF7q6ujAzM8O8efPkXi8qKsLkyZNhZ2cHDQ0NTJ8+\nvcE+Vq9eDaFQCJFIBKFQKPtzVVVVg7lERE25e+821sf+iIUbPpRr7rU0tTHGfTI+j1iKIXbDIBQo\n/CORSE4Pw154c8xsvPvafJgbWcvG71eUYeuhX/D1undx8sJRSKXSZ+yFiKhlKHQ5YfPmzZg1axaW\nL18OT09PLFmyBKNGjUJOTg5MTU0bzC8vL4e/vz98fX2RkZGBnJwcREZGolOnTnj//fcBAJWVleje\nvTtmz57QlE3YAAAgAElEQVSNX375pclj6+npIT8/X+6Holgsft73SUQdUH2A9nccOB4tF6AVCoRw\ntw/AKLeJ0Nd7RYkVkrrp22sAPnh9AU6cS0RM0lrceRzEvXsd/935Nax6DUCoVyQsjK2b2RMR0YtT\nqMH/4YcfMH36dNlV9h9//BF79+7FsmXLMH/+/Abz161bh4qKCqxevRpisRh2dnbIycnB999/L2vw\nLSwssGjRIgDAb7/91uSxBQIBunfnEwOJSHGyAG3KBpRXlMq9NsDSBWO9psKkq5mSqiN1JxQI4Ww7\nFIOsJDiStQuxx35DxaMn4l64dhrfbf4IzrZDEezxBoO4RNQqmv0+urq6GhkZGfD395cbDwgIQHJy\ncqPbpKamwtvbW+5Ke2BgIK5fv46CgoLnKrCiogKWlpYwMzNDcHAwMjMzn2t7Iuo4pFIpTl9Mxzfr\n38PmA8vkmnvT7n3w1/Fz8XbIHDb31CY0NcTwcx6HzyOXw8dxjNwD0jJyj+CrNTMRnbgaDyrvKbFK\nIlJHzV7Bv3XrFmpra2FkZCQ3bmRkhP379ze6TVFREczMzBrMl0qlKCoqgoWFhULF2draYsWKFXBw\ncEB5eTkWLVoET09PnDx5ElZWVk1ul57OgBw9G88R9XP7XhEyLsWjqPSS3LiuuDOcLIahT/eBKCuu\nQnqx4n/3PE9IEYqcJxZ6jjB0NMfxggO4fPssgPog7v6M7UjM2otBZkNhazyYT0lWU/xZQs9ibd3y\nS/ZUOtIvkUggkUhk/+7u7g4nJyf89NNPsuU9RNSx3a8sQ+blQ7hw46TcuKZIDHtTT9iZDIGGSFNJ\n1RH9QV+nC3z7vYbissvIuBiPW/euAwAqayqQdnEfzhamwdlyOMy62PKJuET0Uppt8Lt16waRSITi\n4mK58eLiYhgbGze6jbGxcaPzBQJBk9soQigUwtnZGXl5ec+c5+Li8sLHIPX2+CoKz5H272FVBeLT\nf8fBE9GorpUP0D5+Am1n3RcL0PI8IUW8+HnigiDpOJzIS8KOpDW4U3YDAFD+8A4Ond0Kq579Eeod\nCQtjmxaumNoaf5aQIkpLS5uf9JyaXYOvqakJZ2dnxMXFyY3HxcXB09Oz0W3c3d2RkJAgdzvL2NhY\n9OzZU+HlOU3JysqCiYnJS+2DiNqv2rpaJJ7ci3mrZiA27Te55t6+tys+fWMxJgyf8cLNPVFbEAgE\nGGzjhc+m/AchXpHQEevKXrtw/Qy+2/wxVu/5DrfLip+xFyKixim0ROeDDz5AREQEXF1d4enpiWXL\nlqGwsBAzZswAAMyePRtpaWmIj48HAISHh2Pu3LmIjIzEZ599htzcXCxYsABffvml3H6zsrIglUpR\nVlYGkUiErKws2V13AGDu3LmQSCSwtrZGWVkZFi9ejOzs7GfeVpOI1NPjAG100moU37kq95ppjz4I\n9ZoGG7OBSqqO6MXUB3FDIek/HHuPbUHCyT2yJ+JmnEtA5oUU+DqOgb/ra9DV6qTkaomovVCowZ8w\nYQLu3LmD+fPno7CwEPb29tizZ4/sHvhFRUW4ePGibL6+vj7i4uIwc+ZMuLq6wtDQEB999BFmzZol\nt18nJye5dYYxMTGwsLBAfn4+AODu3bt4++23UVRUBAMDAzg5OSEhIQHOzs4v/caJqP24cuMCohJW\nIe/qKblxw07dMMbzDTjbDuVDqqhd09PRx6s+b8J7UBBiktci63wKAKC2tgb7M6KQeno/Rrq9Ds+B\ngcyUEFGzBFI1eazek+uXDAwMlFgJqTKuh2xfSspvYmfyeqSdPSQ3riXWQYDLa/BxGgOxhlaLH5fn\nCSmiNc+T/Os52J6wEgVF5+TGuxuYYKxXBAZZSRjEbQf4s4QU0Ro9rErfRYeIOqaKygfYn/E7Dh7f\n0SBA6zlwJEa6TeAae1JrfXra4YMJC3AiLwkxSWtla/Fvlhbif7sWoI+JHUKHToMlg7hE1Ag2+ESk\nMmpra5B8Og57Ujfh3lNPoB3YZwjGek2FkWEvJVVH1LYeB3EH9nFDwsld2HfsN1RU3gcA5Bfm4PvN\nH2OwjTeCPd5AVwOjZvZGRB0JG3wiUjqpVIrsi2nYkbgGxSXyAVqzHlYI9Z4Ga1N7JVVHpFyaGpoY\nPjgUbnbDse/Yb0g4uQe1dTUAgOPnEpB1IQU+DmMQ4PoadLUZxCUiNvhEpGSXi88jKnEVzl/Nlhs3\n7NwdYzwmM0BL9Iiejj7G+/wJ3g5BiElai8zzyQDqv/k6cDwKqWf2Y+SQCfAaNJJBXKIOjg0+ESnF\nnbKb2JmyDulnD8uNa4t14e/6GnwcR7dKgJaovev+igmmj/4Y+dfPIiphJS4V5QIAHjwsx+9H/ocj\nWbsw1jMCDn3dGcQl6qDY4BNRm6qovI+49N9x6MQO1NRWy8aFAiG8Bo1E4JDX0VmXd8Iiak6fnv3w\n/oRvkHk+GTsS18iCuLdKi7Bi97/R26QfQr2nobeJrZIrJaK2xgafiNpEbW0NkrJjsefoJtyvKJN7\njQFaohcjEAjgZO0J+95DkHhyD/Yd24IHlfcAABcLz+KHLZ/AydoTwZ5T0M3AWMnVElFbYYNPRK3q\ncYA2OnE1bpRck3vNvEdfhHhHMkBL9JI0NTQxbPBYDOk/DLHHfsORrN2yIO6JvCScvHAUQx2CEDAk\nDHranZVcLRG1Njb4RNRqLhefR1TCSpy/dlpu3LBzdwR7vIHBtt4M0BK1ID3tzhg3dDq8Bo1CTPJa\nZOY9CuLW1eDgiR04euYAAodMgNegUdDUYBCXSF2xwSeiFnen7AZ2Jq9Hem7DAG2A62vwcRwDTQ2x\nkqojUn/dXzHB9KCPcbHwLLYnrMSlwkdB3Mp72J6wAkdO7sJYz6lwZBCXSC2xwSeiFlNReR9xadtw\nKDNGPkArFMFr4EiMdHsdnXT0lVghUcfS26Qf3g/7BpnnU7AjaTVul9YHcW+XFmPl7n/D0sQW47yn\nobdJPyVXSkQtiQ0+Eb20+gDtPuw5urlBgHaQlQRjPaegBwO0REpRH8T1gH1vVySe2oN9R/8I4l4q\nzMUPWz6Fo7UHgj2moPsrJkqulohaAht8InphUqkUp/KPYUfiaty4e13uNXMja4zzjoRVrwFKqo6I\nnqSpoYlhTmPhZjccsWm/4XDmLlkQNzMvGacuHIO3QxACGcQlavfY4BPRCykoykNU4ipceCpA26Vz\ndwR7RsDJxpMBWiIVpKvdCaHe0+qDuElrcSIvCUB9EPfQiR04xiAuUbvHBp+InsudshuISV6HjNwj\ncuM6Yl0EDAnDUIfRDNAStQPdDIwxLegj+BaORVTCSlwsPAvg6SBuBBz7ejCIS9TOsMEnIoVUVN5H\nbNpWHM7c2SBA6z1oFAKHTGCAlqgd6m1ii1lhXyPrfAp2JK3BrdIiAI+DuAthaWKLUK9p6NOTQVyi\n9oINPhE9kyxAm7oJ9x+Wy71WH6CNQA/DnkqqjohagkAggKO1B+z7uCLx5F7sPbYFDx79/36pMBeL\nfvsUjn09EOzJIC5Re8AGn4gaVR+gPYroxDW4+VSA1sLIGqHe02DVq7+SqiOi1qAh0oSvUzCG2A2r\n/8Yuaydqax8Fcc8n41T+sfpv7NwmMIhLpMLY4BNRAwVF5xCVsAoXrp+RG++i3wNjPSPgZO3JNblE\naqw+iBsJ70GjEJO8DsfPJQB4FMTNjMHRnPogrvegIAZxiVQQG3wikrldVoydSeuQ8eiX+WP1AdoJ\njwK0/GVO1FF0NTBC5KgP4esUjKgjK5FfmAOgPpMTlbASCVm7Eew5hR/6iVQMG3wiwoPKe4hL24pD\nmX98HQ/8EaAdOWQC9BigJeqwLI1t8F7Yv3DyQip2JK7BzdJCAPUXBVbt+RaHTsQg1DsSfXraKblS\nIgLY4BN1aDW11Ug6Vf8E2gdPBWgd+rpjrGcEA3VEBKA+iOvQ1x0Ders0+LlxqSgXi36bzZ8bRCqC\nDT5RBySVShtciXvMwtgG47yn8UocETVKQ6QJH8cxcLXzbfDNX9b5FGTnp8Fr0Eh+80ekRGzwiTqY\nS0XnEJWwEvnXc+TGu+obcS0tESlMV6sTQrwi4TVolFx2p7auBoczd+LYmQPM7hApCRt8og7idmmx\n3N0wHtPR0kPgkDB4D+IvYSJ6fl31jTD1cRD3ibtvVVQ9QHTiKiSc3M27bxG1MTb4RGruwcN7De5n\nDQAioQa8HYIQOCSM97MmopdmYWyDd1+b3+D5GXfKbmDVnm9x8MQOLv8jaiNs8InUVE1tdYMnUj7G\nJ1ISUWsQCAQYZCXBAEuXBk/ALig6Vx/EtZIgmE/AJmpVbPCJ1IxUKkXW+RTsSFqDW6VFcq9ZGtsi\n1Hsa+vTsp6TqiKgjEIk0MNRhNFz6+SAubRsOZ+5ETW01ACDrQipOXUzjLXiJWhEbfCI1crEwF1EJ\nK3Gx8KzcOAO0RKQM9UHcqbIn4mbkHgEA1NXVPhXEDYKmhljJ1RKpDzb4RGrgVmkRYpLW4kRekty4\nrlYnBA6ZAK9BoxigJSKl6aLfA1NHfgBfx2BEJaxsNIgb7DEFg228eBGCqAWwwSdqxx48vId9x7bg\nSNZu1NYxQEtEqs3C2LrJIO7qvd/h0IkdCPWOhFWvAUqulKh9Y4NP1A5V11Qj8eQe7Du2BQ8q78m9\n5mjtgWAPBmiJSDU9M4hbnIfFWz/DICsJxnpOQQ/DXkqulqh9YoNP1I5IpVJknk/BjqTVuF1aLPea\npYktQr0YoCWi9uFxENe1ny/i0rbhUGaMLIh78kIqsi+mwWvgSIx0ex2dGMQlei5s8InaiYuFZ7E9\nYSUuFebKjXc1MMJYzwg49vXg2lUiand0tPQw1isCXoNGNgjiHsnahbScg/B3fQ0+jmMYxCVSEBt8\nIhV3824hYpLXIjMvWW6cAVoiUiePg7jDnMZie8JKXLh2GkB9EHdH0hokntxTfzcwGy8IBUIlV0uk\n2tjgE6mo+w/Lse/Yb0hoJEA71CEIAQzQEpEaMjfqi3df/QrZF9MQnbAKNx4HcctvYvXe73HwRAxC\nvSPRl0FcoiaxwSdSMdU11Ug4uRv7jm1BReV9udecrD0R7DkF3QyMlVQdEVHrEwgEGNhnCPpbDEZS\ndiz2HN2E+xVlAIDLxXn4cetnGNhnCEK8pjKIS9QIhb/jWrp0Kfr06QMdHR24uLggMTHxmfOzs7Ph\n6+sLXV1dmJmZYd68eXKvFxUVYfLkybCzs4OGhgamT5/e6H62bduGAQMGQFtbG/b29oiKilK0ZKJ2\nRSqV4kReEv617q+ISlgp19z3NumH9yd8g2lBH7G5J6IOoz6IG4T/m7oMI1xehYboj+WIp/KP4V/r\n3sXWQ7+g/EGpEqskUj0KNfibN2/GrFmzMGfOHGRmZsLDwwOjRo3C1atXG51fXl4Of39/mJiYICMj\nA4sXL8bChQvxww8/yOZUVlaie/fumD17NiQSSaP7SUlJwcSJEzFlyhRkZWUhPDwcYWFhSEtLe4G3\nSqS68q+fxQ9bPsXK3Qvl7o7TzcAY04M+xqywr9HbhHfHIaKOSUdLD2M9p2BOxBK49PORjdcHcXdj\n3uq/IC79d1TVVCqxSiLVIZBKpdLmJkkkEjg6OmL58uWyMRsbG4SFhWH+/PkN5i9btgyzZ8/GjRs3\nIBbXJ97nz5+P5cuX48qVKw3mBwcHo3v37lixYoXc+MSJE1FSUoJ9+/bJxvz9/dGjRw+sX79ebm5p\n6R+f3g0MDJp7S9RBpaenAwBcXFyUXEm9m3cLEZO0FpnnGwnQuk2A96BRclesqG2o2nlCqonnifJc\nLj6PqISVOP8oiPuYYefuGOPxBpxtvVUiiMtzhBTRGj1ss2d/dXU1MjIy4O/vLzceEBCA5OTkRrdJ\nTU2Ft7e3rLkHgMDAQFy/fh0FBQUKF5eSkoKAgAC5scDAwCaPS9Re3K8ow++H/4d/rf2bXHMvEmlg\n+OAQ/F/kcgxzGsvmnoioEeZGffG3V7/CW8H/kFuDX1J+E2v3/YDvNn2EvKvZSqyQSLmaDdneunUL\ntbW1MDIykhs3MjLC/v37G92mqKgIZmZmDeZLpVIUFRXBwsJCoeKKiooaPW5RUdEzt3v8iZmoKco6\nR2rranC2MB2nriSiqvah3GuW3frDyWIYOmsb4kz2WaXUR/L4s4QUwfNEmYQI6BeBc8UnkHX5CCpr\nHgAArty4gJ+2zYFpFxs4W/jBQLerUqvkOULPYm1t3eL7VMu76Li6uiq7BKIGrJ16wn10fxh005Mb\nv55/G4nRp1FcEA3ga+UUR0TUzom1NeDsZw1HHytoiEUAgKt3zuHyzbPITi7AsX1nUXGvSslVUkf2\n1ltv4c9//nObHKvZBr9bt24QiUQoLi6WGy8uLoaxceN38zA2Nm50vkAgaHKb59nP8+yDSNlMeneB\nV8gAGFt2kRu/e/MekmPO4MLJQiVVRkSkPqoe1iBlVw5OJV2C+2g79HOtX0kgFAkxyLs3+rmaIj0+\nD5mHL6C2uk7J1VJH1LNnz0bzGE+uwW8pza7B19TUhLOzM+Li4uTG4+Li4Onp2eg27u7uSEhIQFXV\nH5+UY2Nj0bNnT4WX5zzeT2PH9fDwUHgfRMpi0E0Po6a54rX3vOWa+4r7VTj8+yms/+YAm3siohZ2\n724F4tYfx6ZvD+Fq3k3ZuFhbEx5j+mPKP/xg62wKCJRYJFErU2iJzgcffICIiAi4urrC09MTy5Yt\nQ2FhIWbMmAEAmD17NtLS0hAfHw8ACA8Px9y5cxEZGYnPPvsMubm5WLBgAb788ku5/WZlZUEqlaKs\nrAwikQhZWVkQi8Wws7MDALz33nvw8fHBggULEBoait9//x2HDh1CUlLSM+tV4MZA1EG1xR0N7leU\nYe+xLUg4uQd1dbWycZFIA76OY+Dv+hp0/9Gp1Y5PL493viBF8DxRfVKpFNkX07AjcQ2KS+pv7d3Z\nUBcBU5wx/cMwjPOeBmvTga12fJ4jpCwKNfgTJkzAnTt3MH/+fBQWFsLe3h579uyBqakpgPow7MWL\nF2Xz9fX1ERcXh5kzZ8LV1RWGhob46KOPMGvWLLn9Ojk5QSD44yN0TEwMLCwskJ+fD6D+Cv6mTZsw\nZ84c/POf/4SVlRW2bNnC/1FIJVXXVONI1i7EHtuCiqoHcq8523hjjMcb6Gpg1MTWRETU0mRPxLV0\nRnJ2LPakbsK9ivrlEFdv5OOnbZ/DvrcrQrymwqiLqZKrJWo5Ct0Hvz3gffBJEa1xNeXxE2h3JK3B\nnbIbcq/16WmHUO9psDS2abHjUevjVTdSBM+T9qei8gHi07fh0IkYVNf+sYxYKBDCY2AgRrm9js66\nr7TY8XiOkCJao4dVy7voELWV/Os52J6wEgVF5+TGuxuYYKzXVAyycpP7loqIiJRHR0sXwZ5T4DVo\nJHalbMCxnIMAgDppHRJP7kHa2UPwd3kVvk7BEGtoKblaohfHBp/oBdy8W4gdSWuQdT5FblxPuzNG\nur0Oz4GBfEgVEZGKMuzcHW8EvAcfxzGISliFvKunAACVVRXYmbwOSSf3YrTHZLj081GJJ+ISPS82\n+ETPoakArYZIEz6OY+Dv+ip0tRigJSJqD8x6WOGv4+fizKUMRCWuQvGd+iBuyb1bWBe7GIcyYxDq\nNQ02Zq0XxCVqDWzwiRRQXVOFI1m7Gw/Q2g7FGI/J6KrPAC0RUXsjEAgwoLcL+lk4ISU7DntSN6L8\niSDuf36vD+KO9YqAcRczJVdLpBg2+ETPIJVKcfxcImKS1zYI0Fr17I9Q72mwMG75R0wTEVHbEglF\n8Bo0Es62Q7E/YzsOHo+WBXGzL6bhzKUMeNgHYJRkYosGcYlaAxt8oiZcuHYaUQmrUFCcJzfe45We\nGOs1FQP7DGGAlohIzeho6WKMx2R4DgzErpT1SMs5BCmk9UHcU3uRlnsY/s7j4es0FmJNBnFJNbHB\nJ3rKjZJr2JG0FicvpMqN62l3xijJRHjaB0Ik4v86RETqzLBzt0dB3GBEJ6zEuSeDuCnrkXhqL8Z4\nvMEgLqkkdilEj9yrKMPeo5uReGpvgwCtr2Mw/F1fhY6WnhIrJCKitmbWow9mPgriRieuRtGdKwCA\nu/du1wdxT8Qg1DsSNmaDlFwp0R/Y4FOHVx+g3YXYY781CNC62PpgjMdkdNHvoaTqiIhI2Z4M4qae\njsfulA1/BHFv5uM/v/8fBvR2QYjXVAZxSSWwwacOq05ahxPnEhGTtBZ3ym/Kvda31wCEek+DuVFf\nJVVHRESqRiQUwXNgoCyIe+B4FKpr6oO4py+mI+fScbjbB2CU20To6zGIS8rDBp86pPOPArSXGwnQ\nhnhHwr63KwO0RETUKG2xDka7h9cHcZPX41jOQVkQN+nUXqSfPYQRLq/CQGrKhx6SUrDBpw6lrOI2\nMi4dwJWkXLlxPR19BLlNhId9AAO0RESkkFc6dcXkgHfh6xSMqIRVyL2SBQCorH6IXSnroSvuDCcL\nXwyWDmYQl9oUOxnqEMoflGLfsc1IOLkXUmmdbFxTJIavUzBGuIxngJaIiF5Ir+698c64L5BTcALR\niatQePsyAOBBVTmS8mJw6W42Qr0iYWvuoORKqaNgg09qrbqmCocydyIubSsePhWgde3ni9Huk9FF\nv7uSqiMiInUhEAjQ33IwbM0dcPTMfuxK2YDyB3cBANduXsSS7f9Ef0tnhHhNhUlXcyVXS+qODT6p\npTppHTJyE7AzeR1KngrQGumbY0rQuwzQEhFRixMJRfCwD4CzjTfW7lyK09dSUFtXAwA4cykDOQUn\n4D5gBIIkk6CvZ6jkakldscEntXP+2mlEHVmJyzfOy433MOyFAcaeMDW0ZnNPREStSkusA0dzH9gY\nOeHqg9M4euYApJBCKq1DcnYs0nOPYITzOAwfHMon4lKLY4NPaqO45Bp2JK7GqfxjcuOddAwwSjIR\nHgP8ceJEppKqIyKijkhXSx/hnn+Dj2MwohJXIvdyfRC3qvohdqduRNKpfRjjMRmu/XwhFIqUXC2p\nCzb41O6VPyjF3qObkXRqL+qeCtAOGzwWfs7joaOlq8QKiYioo+vV3RIzx32JnIITiEpYKQvilt6/\ng/VxPz16Iu40BnGpRbDBp3arqqYSh0/sRGz6VlRWVci95trPF2M8JsOwMwO0RESkOuwsnGBrNgip\nZw5gd8oGlD0oAQBcu3WpPohrMRhjvaaiZzcLJVdK7RkbfGp36gO0R7AzaR1K7t2Se62vqT3GeU+D\nWQ8rJVVHRET0bEKhCB72/nC28cL+41E4kBGFqppKAMCZguPIuZwJ9wF+CJKEM4hLL4QNPrUreVdP\nYXvCSly9kS83bmRoihCvqRjQ24VPoCUionZBS6yDIMkkeNoHYlfqBhw9vf+JIG4c0nMT4Oc8DsMH\nh0BLU1vZ5VI7wgaf2oXiO1cRnbga2RfT5MZlAVr7AIgYTiIionbIoFMXhI/4K3wcxjQI4u5J3Yik\nU3sx2n0y3OyGMYhLCmGDTyqt/MFd7Dm6Gcmn9jUI0NY/gfZVBmiJiEgtPA7inrl0XO6JuGX3S7Ax\n/j84nLkToV6R6GfhqORKSdWxwSeVVFVTiUMnYhCXvq1BgHaI3TCMdg9ngJaIiNRSf8vB6GfugKNn\nDmDXE0Hc67cuYWnUF+hn4YRQr6no2c1SuYWSymKDTyqlTlqH9LOHsTN5He7euy33mrXpQIR6RzJA\nS0REak8oFMHd3h+Dbbxw4Hg09mdslwVxzxacwILLWXDrPxyj3cNhoNdFydWSqmGDTyrj3JVTiEpY\nias3GaAlIiIC6oO4j7NmTwdxU0/H4/jjIK5zKIO4JMMGn5Su6M4VRCeuxumL6XLjnXUMMEoyCe72\n/gzQEhFRh/Y4iOvrOAbRiWuQU3AcQP2S1j1HNyEpex9GS8Lh1n84g7jEBp+Up+z+Xew5ugkp2bGN\nPIE2BH7O4xigJSIiekLPbpb4S+j/IafgBKITVuH67QIAj4K4+5fgcOZOhHhHws7CScmVkjKxwac2\nV1VdiYMndiA+fRsqqx/KxgUQwNXOF6PdJ8OwczclVkhERKTaHj8R92jOQexKWY+y+4+CuLcLsCzq\nS/Qzd0SIVyR6dbdUbqGkFGzwqc08K0BrYzoQId7TYNajj5KqIyIial+EQhHcB4yQD+I+unB29nIm\ncje8/yiIOxkGnRjE7UjY4FObOHflJLYnrMS1mxflxo27mCHEayr6WzozQEtERPQCtDS1McrtdXjY\n+2NP6kaknN4PqbQOUkiRemY/jp9LxHDnUPgNDoWWWEfZ5VIbYINPrarw9hXsSFyN05caBmiD3MMh\nGTCCAVoiIqIWYKDXBRP9ZmKowxjsSFyNM08Ecfce3YzkU7H1v3sZxFV7bPCpVZTdv4s9qRuRfDoO\n0icDtBpiDB8cAj/n8dDmVQQiIqIW17ObBWaE/h/OFmQiKnEVrt+6BAAoe1CCTfuX4HBmDEK8ItHf\ncrByC6VWwwafWhQDtERERKqhn4UjPjb7DsdyDmFXynqU3r8DACi8fRnLo+cyiKvG2OBTi6iT1iEt\n5xB2pqxH6dMBWrNBCPWOhGl3BmiJiIjaklAogmSAH5xsPHHweDTiGcTtENjg00vLvZyFqMRVjQZo\nQ70jYWcxmAFaIiIiJdLS1MZIt9fhYR+A3akbkXI6vmEQd3Ao/JwZxFUHQkUnLl26FH369IGOjg5c\nXFyQmJj4zPnZ2dnw9fWFrq4uzMzMMG/evAZzDh8+DBcXF+jo6KBv3774+eef5V5fvXo1hEIhRCIR\nhEKh7M9VVVWKlk2tqPD2FSyPnocl2/8p19x31n0FE/3ewSeTF/HuOERERCpEX88QE/3ewaePfkc/\nVlVTib3HNmPe6neQnB2LurpaJVZJL0uhK/ibN2/GrFmzsHz5cnh6emLJkiUYNWoUcnJyYGpq2mB+\neXk5/P394evri4yMDOTk5CAyMhKdOnXC+++/DwC4dOkSRo8ejTfffBPr169HQkIC3nnnHfTo0QPj\nxnp2J3oAABtySURBVI2T7UtPTw/5+fmQSqWyMbFY/LLvm15C2f0S7End1ESANhR+zuMYoCUiIlJh\nJl3NMSPk8/pv4RNW4ppcEHdp/RNxvabyW/h2SqEG/4cffsD06dMxffp0AMCPP/6IvXv3YtmyZZg/\nf36D+evWrUNFRQVWr14NsVgMOzs75OTk4Pvvv5c1+MuWLUOvXr2waNEiAICtrS2OHj2Kb7/9Vq7B\nFwgE6N69+0u/UXp59QHaaMSn/94gQOvWfziC3MPxSqeuSqyQiIiInoetuQM+mvQd0s4elsvR1Qdx\n58HWzAGh3pHo1b23kiul59HsEp3q6mpkZGTA399fbjwgIADJycmNbpOamgpvb2+5K+2BgYG4fv06\nCgoKZHMCAgLktgsMDER6ejpqa//4WqiiogKWlpYwMzNDcHAwMjMzFX931CLq6mpx9Mx+zFv9F+xK\n2SDX3NuaO+Dj8O8R7v83NvdERETtkFAoglv/4fg8YilGu0+Glqa27LXcK1n494YPsD7upwZPoSfV\n1ewV/Fu3bqG2thZGRkZy40ZGRti/f3+j2xQVFcHMzKzBfKlUiqKiIlhYWKCoqKjBhwYjIyPU1NTg\n1q1bMDIygq2tLVasWAEHBweUl5dj0aJF8PT0xMmTJ2FlZfW875VewNNf3T1m0tUcIV6RsLNw4ld3\nREREakCsqYXAIWFwHzBCbimuFFIcPbMfx88lcCluO6HSd9GRSCSQSCSyf3d3d4eTkxN++ukn2dKe\nxqSnpzf5Gimm5P4NHC/Yj2slF+TGdTQ7wcF8KPoaOeLBrTpk3MpQUoUvh+cIKYLnCSmC5wk1pz2e\nI1YGrujqaImMS/txreQ8AKC6pgr7jm3B4RO74Gjug75GjhAKFL5fCzXB2tq6xff5/+3deViU9doH\n8O/MMIMgiyQCIh4R3FgUlUF2hRTNRMFMw/RV7JSmloody/U9lcfUTuftXBWKqeWSC+WCuSWEiMOW\ngkKyuuACCriCiig4PO8f6OgjKFTKwPD9XJd/+Jt7nBuv+7K7mec7T70Lvrm5OWQyGUpKSkTnJSUl\nsLKyqvM5VlZWddZLJBLNc55Wo6enB3Pzum+EJJVK4erqilOnTtXXNv1JFZW3kX4hHqdL0iHgUbBZ\nTyqHYwcPOHXwhFzGkDMREZGua2PYDgMdQ1BUehap537FjfKave1uVTlSzuxDbtFRuNoOhHUbe36a\n38TUu+DL5XK4uroiJiYGo0aN0pzHxMRg9OjRdT7H09MTc+fORWVlpeY6/OjoaFhbW6NTp06amqio\nKNHzoqOjoVQqIZPJntpPRkYG+vTp88yelUplfT8WPeFe1V0cPLYLsemPboABPAjQOg3EMI83deIG\nGA/fReGM0LNwTqghOCdUH92ZESWGCaOQmhuP3Uk/aIK4pXeuIDZ7K7p3dEGQ70Te0PJPKisre+5/\nZoM+V5k9ezbWrVuHtWvXIjc3FzNnzkRRURHeffddAMC8efMwaNAgTf2bb74JQ0NDhIaGIisrCzt2\n7MDy5cvxwQcfaGreffddXLx4EWFhYcjNzcWaNWuwYcMGzJkzR1Pz6aefIjo6GmfPnkVGRgbeeust\nZGZmYurUqc/r52/xqqvVSMmKxb/WT8P+lC2i5b7H33rjwze/xJuD3tOJ5Z6IiIj+HKlEin4O/lg0\nYQUC6wji/nvzB9gU/RWDuE1Eg67BHzNmDK5fv44lS5agqKgIzs7O2L9/v+Y78IuLi3H27KMbHZmY\nmCAmJgbTp0+Hm5sbzMzMMGfOHMyaNUtTY2tri3379iEsLAwRERGwtrbG119/jeDgYE1NaWkppkyZ\nguLiYpiamqJPnz5QqVRwdX10Ywb683LPpyMqYR0uPRGgtW7bCUG+NQFaIiIioocUcn0M7jcaHk4B\n+OW3rTU3xXoYxM05iGOnEvBy3yAMdH2NQVwtkgiP30GqGXv84w1TU1MtdtL0Xbp6HrsS1iPn/DHR\nuUlrMwzzHAd3B39IpU+/TKo5052PS+lF4pxQQ3BOqD4tYUaKrxfg54QNyDx7VHRubGCKVz3fhIfT\nIMh0dKd4Xl7EDtukv0WHnq+y8uvYl7wFKdmxojvQKvT0MdB1JF7uGwR9/t82ERERNZDVSx0xecQC\nnCz4HVGqdSi8kg8AuFVRhsiDKzV3xHW0dWUQtxFxwW8B7lXdxcG0KMQeixIHaCVSeDgOxKueY2Ha\nmtfYExER0Z/TrWMv/GPsF0jNjceepB801+IXXy/Aqp//hW4deyHIJxQdLRjEbQxc8HVYzR1oD2Jv\nymbcLL8hesyhU18E+UyAtbmtdpojIiIinfIwiNu7qxcOHd+NmNTtuFdZAQA4WfA7vtjyAdwc/DDM\ncxzMjOv+SnR6Prjg66ic88exS7UOl66dF51bm9siyGciA7RERET0Qij09DHY7fWaO+L+FomkEwc0\nQdwjOXE4fjIR/n2DMEjJIO6LwgVfx1y6eg5RCeuRe/646NyktRkCPcejn4OfzgZoiYiIqOkwNmyD\nMf5TMMBlGHYlbkBm/hEAQJW6EtFHf0JyZjSGeoyFp3MAg7jPGRd8HVF2+zr2pmzGb9kHxQFaeatH\nAdrHvrOWiIiIqDFYvmSDycPn41ThCUSp1qHg8hkANUHcH+MiEJ+xB0HeE+HUWckg7nPCBb+Zu1dZ\ngdhjUTiYFoXK+/c05xKJFJ5OAzHUgwFaIiIi0r6uNj3xQci/kZZ3GHsSf8CN21cBACXXC/Ht7iXo\natMTwb6h6Ghhr+VOmz8u+M1UdbUaKdkHsS95M27eEQdoHTv1xQifibA276Sl7oiIiIhqk0qkcOvh\nB5cunog/vgfRqds0QdxThSfw7y0fwK2HHwK9xsHMuJ2Wu22+uOA3Q9nnjmFXwjoUXbsgOrc2t0Ww\nTyh6dOqtpc6IiIiI6qfQ00eA2yh4OA3EL7/9iMQTv6D6wSXGR3MPIf1UEvz7jsBA19dgoG+o5W6b\nHy74zcjFK2cRlbAOeRcyROcM0BIREVFzZGzYBqP9J6O/y6t1BHG3ISkzBkM9QuDlFACZjGtrQ/Fv\nqhkovX0Ne5M340j2QQgQNOcKeSsMch0JfwZoiYiIqBl7WhD3dkUZfopbhcPpezHCZwKcO7sxiNsA\nXPCbsHuVFYhNi8LBY3UFaAfhVY+xMGltpsUOiYiIiJ6fR0FcFfYkbnwUxL1RiNW7P0MXG2cE+4Ti\nb5ZdtNxp08YFvwlSV6vxW3Ys9iZvxq07paLHGKAlIiIiXVYTxB0Aly4eiE/fi5ij23C38g4A4HRh\nJr7Y+g8oewxAoOd4vGTCIG5duOA3IYIgIOf8MexKWF8rQNvB3BbBvpPQ/W8uWuqOiIiIqPEo9PQR\noHwNHo4D8ctvkaIgbmpuPNJPJcGvzwgEKF+DgX5rLXfbtHDBbyIKr+Rjl2o98grEAVrT1i8h0Gsc\n3HowQEtEREQtj7GhaU0Qt/cw/JywHiceBHHvq6vwa+p2JGfFYKh7CLydBzOI+wD/FrSs9PY17E3a\nhCM5caIArb68FQYpX4N/nyAo5Ppa7JCIiIhI+yzNOuCd4fNxqjATu1TrcOHyaQBAecVNbDv0LQ5n\n7EWQz0QGccEFX2vuVlYgNm0HDh7bhar7lZpziUQKL6cADPUIYYCWiIiI6AldbZwxO+RzHMtTYXfS\nD7hx6woA4PKNizVB3A5OCPad1KKDuFzwG5m6Wo2UrF+xL2VLrQCtk60SI3wmon3bjlrqjoiIiKjp\nk0qkUPYYAJcunjiUvkccxL2YVRPE7T4AgV7j8JKJhZa7bXxc8BuJIAjIPpeGXQnrUXy9QPRYh3ad\nEewTygAtERER0R8g11NogrgHjkQi4cQBVFerAQCpefFIP50Ev97DEeA2qkUFcbngN4LCK/mIUq3D\nyYLfReemRm0x3Gs8lD0GQCqRaqk7IiIioubN2NAUr/tNhq/LMOxO3IDfz/wG4EEQN20HkrN/xVD3\nN+DtPKRFBHF1/yfUohu3rmJv8iYczTlUR4B2FPz7jGCAloiIiOg5sTTrgLcD5+H0xSxEqdbhQskp\nAA+DuKsf3BF3Inra9dPpIC4X/BfgbmUFfk3dgbjj4gCtVCKFp/NgDHUPgUnrNlrskIiIiEh3deng\nhNlvLMexPBX2JP2A6w+DuKWXsGbPUth3cMJIHQ7icsF/jjQB2uTNuFVRJnrMqbMSI7wZoCUiIiJq\nDI8HcQ9n7EX0kZ9Q8SCIe+ZBENe1e38M9xqvc0FcLvjPwbMCtDbt7BDsG4puHXtpqTsiIiKilkuu\np8BA15FwdxyIA0d+hOr3/ZogblreYWScTta5IC4X/L/oaQHaNkZtEcgALREREVGTYGRgglED3oZv\nr1exO3EDMs6kAHgsiJsVg6EeIToRxG3e3WvRUwO0CgMEKEfBr89wKPQYoCUiIiJqSizMrPH3wLk4\nczELOx8P4t69hW2HViM+fS+CfCagp517sw3icsH/g54VoPXqOQRD3d+AsSEDtERERERNmf2DIO7x\nk4nYnbhBE8S9UnoJa/Ysg721I4J9J6GTVVctd/rHccFvIHW1GsmZMdifsqVWgNa5sxuCfCbC8iUb\nLXVHRERERH+UVCKFa3df9LJ3x+GMfYg+8uOjIO6lbPwncg5cu/ki0Hs82ppYarnbhuOCXw9BEJB1\nNhW7Etej5Hqh6DEbCzuM9J2ErjY9tdQdEREREf1VNUHcYHg4voxfngzinlQh/Uwy/HoHIsDtdRjq\nG2m52/pxwX+Ggsv52KX6HicLT4jOzYzMEeg9Hq7d+zNAS0RERKQjWj8exE3aiIzTyQAAtfo+YtOi\nkJIVi1fc34B3zyHQk8m13O3TccGvw41bV7A3eXOdAdrBytcxoE8gA7REREREOsrCzBp/H/YR8i/l\nYKfqe5wvPgmgJoi7PX7NgzviTkAve48mGcTlgv+Yint3EJu2A3HHfkaVuq4AbQiMDU212CERERER\nNRY7awfMHrMcx08lYnfiRly7WQIAuFJWhLV7l8PO2gHBvpNga9VNy52KccFHTYA2KTMa+1O24vYT\nAdqedv0wwmciLM06aKk7IiIiItIWiUSCvt180NPOHarf9+LAkZ9Qca8cAJB/KQf/F/kh+nbzxXCv\n8Whr2jSCuC16wRcEAZlnj+LnhA0ouSEO0Ha0sEew7yR0tXHWUndERERE1FTI9eR4uW8w3B1exoEj\nP0H1+36oq+8DAI6dVCHjTDIGuARisNvrMGyl3SBui13wCy6fQZRqHU49GaA1bodAr/Fw7e7LAC0R\nERERibQ2MMFrA/4OX5dXsTtxI9JPJwGoCeIePBaFlOxYvNJvDHx6vaK1IG6LW/Cv37yCPck/IDU3\nXnTeSmGIAOUoBmiJiIiIqF7t2rTHW8M+RP6lXESpvse54jwAwJ27t7Dj8FocztiLEd4T4NLFs9GD\nuC1mwa+4V46Y1B04dPxn3FdXac6lEil8er2CIf3eYICWiIiIiP4QO+seCBuzrFYQ92pZMb7b9zns\n2jsguH/jBnEbfA3KihUrYGdnBwMDAyiVSiQkJDyzPjMzE35+fjA0NETHjh2xePHiWjXx8fFQKpUw\nMDBAly5dsGrVqlo127dvh5OTE1q1agVnZ2dERUU1tGUANR+XHM7Yh0/XT8WvqdtFy31Pu36Y9z9f\n43W/yVzuiYiIiOhPeRjEnf8/32Ck71uim2HlF9UEcdfv/w+qhepG6adB7+BHRkZi1qxZiIiIgLe3\nN8LDwzF06FDk5OTAxsamVv2tW7cQEBAAPz8/pKWlIScnB6GhoTAyMkJYWBgA4Ny5cxg2bBjefvtt\nbNq0CSqVCtOmTYOFhQVGjhwJAEhOTkZISAgWL16MkSNHYvv27Rg9ejSSkpLg5ub2zJ4FQcCJ/CP4\nOXEDLt+4KHrsbxZdENx/Erp0cGrQXxIRERERUX3kenL49x2Bfo7+iD7yEw5n7NMEceV6ikbLd0oE\nQRDqK/Lw8EDv3r0RERGhOevWrRtGjx6NJUuW1KpfuXIl5s2bh8uXL0OhUAAAlixZgoiICBQUFAAA\nPvroI0RFRSEvL0/zvHfeeQfZ2dlITEwEAISEhODGjRs4cOCApiYgIAAWFhbYtGmT6DXLyh59vWXZ\n3SvYqfoeZy5miWrMjNthuNd49GWAtsVKTU0FACiVSi13Qk0Z54QagnNC9eGM0JXSIuxO2ojsc8ew\naMIKmBq9VKvm8R3W1PT5XFFS75ZbVVWFtLQ0BAQEiM4HDx6MpKSkOp+TkpICX19fzXIPAEOGDMGl\nS5dw/vx5Tc3gwYNFzxsyZAhSU1OhVqsB1LyDX1fN0173oS+2/kO03LdSGGKE9wQsnBAOZY8BXO6J\niIiI6IVr16Y93nr1QyyaWPdy/6LUu+levXoVarUalpbiL+63tLREcXFxnc8pLi6us14QBM1znlZz\n//59XL169Zk1T3vdJ0mlMvR3GYb/DY3AIOVrkOsp6n8SEREREdFzZNq68ZZ7QEe/RWdx6HrR79WV\nAsoqy55STS1J165dAYg/DiN6EueEGoJzQvXhjJC21PsOvrm5OWQyGUpKSkTnJSUlsLKyqvM5VlZW\nddZLJBLNc55Wo6enB3Nz82fWPO11iYiIiIhaunoXfLlcDldXV8TExIjOY2Ji4O3tXedzPD09oVKp\nUFlZqTmLjo6GtbU1OnXqpKl58s+Mjo6GUqmETCZ7ak1MTAy8vLwa8KMREREREbVAQgNERkYK+vr6\nwpo1a4ScnBxhxowZgrGxsVBQUCAIgiDMnTtXGDhwoKa+rKxMaN++vTB27FghMzNT2L59u2BiYiJ8\n+eWXmpqzZ88KRkZGwqxZs4ScnBxh9erVgr6+vrBz505NTVJSkiCXy4Vly5YJubm5wmeffSYoFArh\n6NGjDWmbiIiIiKjFadDXZAJAREQEPv/8cxQVFcHZ2Rn//e9/Ne/gT5o0CYcPH8aZM2c09VlZWZg+\nfTqOHDkCMzMzTJ06FQsXLhT9mSqVCmFhYcjKyoK1tTXmzp2Ld955R1SzY8cOLFy4EPn5+bC3t8dn\nn32GoKCgv/r/NUREREREOqnBCz4RERERETV9OvOF8CtWrICdnR0MDAygVCqRkJCg7ZZIS5YuXYp+\n/frB1NQUFhYWGDFiBLKysmrVffzxx+jQoQMMDQ3h7++P7OxsLXRLTcXSpUshlUoxY8YM0TnnhIqL\nixEaGgoLCwsYGBjA2dkZKpVKVMM5admqq6uxaNEizR5iZ2eHRYsWobq6WlTHOWk5VCoVgoKCYGNj\nA6lUig0bNtSqqW8eKisr8f7776Ndu3YwMjJCUFAQLl682KDX14kFPzIyErNmzcLChQuRnp4OLy8v\nDB06FIWFhdpujbTg8OHDeO+995CcnIy4uDjo6elh0KBBKC0t1dQsX74cX375JcLDw5GamgoLCwsE\nBASgvLxci52TtqSkpGD16tVwcXERnXNOqKysDN7e3pBIJNi/fz9yc3Px9ddfw8LCQlPDOaFly5Zh\n5cqV+Oabb5CXl4evvvoKK1aswNKlSzU1nJOW5fbt2+jZsye++uorGBoa1nq8IfMwc+ZM7Ny5E5GR\nkUhISMDNmzcRGBiIBl18o9UEwHPi7u4uTJkyRXTWtWtXYf78+VrqiJqS27dvCzKZTNizZ4/mrH37\n9sLSpUs1v6+oqBCMjY2Fb7/9VhstkhaVlpYK9vb2wqFDhwQ/Pz/h/fff1zzGOaF58+YJPj4+z6zh\nnFBgYKAQGhoqOps4caIwfPhwze85Jy2XkZGRsH79etFZffNQVlYmKBQKYcuWLZqagoICQSqVCtHR\n0fW+ZrN/B7+qqgppaWkICAgQnQ8ePBhJSUla6oqakps3b6K6uhpmZmYAgLNnz6K4uFg0M61atUL/\n/v05My3Q5MmTMWbMGAwYMEB0zjkhANi1axfc3d0REhICS0tL9OnTB+Hh4ZrHOScEAD4+PoiLi0Ne\nXh4AIDs7GwcPHsSwYcMAcE5IrCHzkJqaivv374tqbGxs4ODg0KCZafZ3sr169SrUajUsLS1F55aW\nloiNjdVSV9SUzJw5E3379oWnpyeAmutpJRJJnTNz6dIlbbRIWrJ69Wrk5+djy5YttR7jnBAA5Ofn\nY8WKFQgLC8O8efOQnp6O9957DxKJBNOmTeOcEADgo48+wq1bt+Do6AiZTAa1Wo0FCxZgypQpAPjv\nCYk1ZB5KSkogk8nQtm3bWjXFxcX1vkazX/CJnmX27NlISkpCYmIiJBKJttuhJuTkyZNYsGABEhMT\nIZU2+w8z6QWprq5Gv379sGTJEgCAi4sLTp48ifDwcEybNk3L3VFTsXXrVmzcuBFbt26Fo6Mj0tPT\nMWPGDHTu3BmTJk3SdnvUAjX7/6qZm5tDJpOhpKREdF5SUgIrKystdUVNQVhYGCIjIxEXF6e5gzIA\nWFlZQRAEzkwLl5ycjGvXrsHR0RFyuRxyuRzx8fEIDw+HQqFA27ZtOSeE9u3bw8HBQXTm4OCACxcu\nAOC/J1Tjww8/xJw5czB69Gg4OTlh3LhxmD17tiZkyzmhxzVkHqysrKBWq3Ht2rWn1jxLs1/w5XI5\nXF1dERMTIzqPiYnR3IiLWp6ZM2dqlvuuXbuKHuvcuTOsrKxEM3P37l2oVCrOTAsycuRInDhxAhkZ\nGZpfSqUSY8eORUZGBrp168Y5IXh7e2uuq34oLy9P86YB/z0hALhz506tTwKlUqnmazI5J/S4hsyD\nq6sr9PT0RDWFhYXIyclp0MzIPv7444+fe+eNzMTEBP/85z/Rvn17GBoaYvHixVCpVPjuu+9gamqq\n7faokU2fPh0bNmzAtm3bYGNjg/LycpSXl0MikUChUAAA1Go1li1bhu7du0OtVmP27NkoKSnBqlWr\nNDWk2/T19dGuXTvRr82bN8PW1hYTJkwAwDkhoFOnTvj0008hk8lgbW2N2NhYLFy4EPPnz4dSqQTA\nOSEgJycHGzduRPfu3aFQKBAXF4cFCxZg7NixmpAk56RlKS8vR05ODoqLi7F27Vr06tULpqamqKqq\ngqmpab3zoK+vj6KiIoSHh6NXr14oKyvD1KlTYWZmhmXLltV/2fFf//KfpmHlypVC586dhVatWglK\npVJISEjQdkukJRKJRJBKpbV+ffLJJ6K6Tz75RLC2thYMDAwEPz8/ISsrS0sdU1Ph7+8v+ppMQeCc\nkCDs27dPcHFxEQwMDITu3bsL33zzTa0azknLdvv2bSEsLEywtbUVDA0NBXt7e2HhwoXCvXv3RHWc\nk5bj0KFDde4jkyZN0tTUNw+VlZXCjBkzBHNzc6F169ZCUFCQUFhY2KDXlwhCQ74tn4iIiIiImoNm\nfw0+ERERERE9wgWfiIiIiEiHcMEnIiIiItIhXPCJiIiIiHQIF3wiIiIiIh3CBZ+IiIiISIdwwSci\nIiIi0iFc8ImIiIiIdAgXfCIiIiIiHfL/6vyVi+YfKfwAAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "xs = list(range(0,100))\n", + "\n", + "ps = thinkbayes.MakeUniformPmf(0, 100, 101).Probs(xs)\n", + "plt.plot(xs, ps, label='uniform', color='k')\n", + "\n", + "suite = TrianglePrior(Euro())\n", + "ps = suite.Probs(xs)\n", + "plt.plot(xs, ps, label='triangle');\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows the result (and the uniform prior for\n", + "comparison). Updating this prior with the same dataset yields the\n", + "posterior distribution shown in Figure [fig.euro3]. Even with\n", + "substantially different priors, the posterior distributions are very\n", + "similar. The medians and the credible intervals are identical; the means\n", + "differ by less than 0.5%." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu8AAAEWCAYAAADW9nkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcU1f6P/BPEhIIIMgiAWQTRUWxasEVrdQRrEv30e6t\nUDvlV+uIdrFO7Wr5ar92qrUj6nSsVu0ofqdWp622otVWpM4UWmxV3KqiQBJkFWRP8vuDck1kCxi4\nCfm8X6++5uZw7r1P8Ex4OJz7HInBYDCAiIiIiIisnlTsAIiIiIiIyDxM3omIiIiIbASTdyIiIiIi\nG8HknYiIiIjIRjB5JyIiIiKyEUzeiYiIiIhsBJN3IiIiIiIbYXbynpKSgtDQUCiVSkRFRSE9Pb3V\nvrW1tYiPj8fw4cOhUCgwefLkNq+dnp4OuVyO2267zfzIiYiIiIjsjFnJe2pqKpKSkrB06VJkZ2dj\n/PjxmDZtGvLy8lrsr9PpoFQqMX/+fMycObPNa5eVleGpp57ClClTOh49EREREZEdkZizw+rYsWMx\nYsQIrF+/XmgbOHAgZs2aheTk5DbPnT9/Pk6ePIlvv/22xa8/+OCDGDFiBPR6PT777DP88ssvHXwL\nRERERET2od2Z9/r6emRlZSE2NtakPS4uDhkZGbd085SUFBQWFmLp0qW3dB0iIiIiInvQbvJeVFQE\nnU4HlUpl0q5SqaDRaDp9419//RXLli3Dp59+ColE0unrEBERERHZCwcxblpXV4eHH34Y7733HoKC\nggAAba3eKS8v767QiIiIiIgszt3d3SLXaTd59/b2hkwmg1arNWnXarXw9fXt1E3VajVycnIQHx+P\nOXPmAAD0ej0MBgMUCgX27t3LB1iJiIiIiG7S7rIZuVyOyMhIpKWlmbSnpaUhOjq6Uzft27cvTpw4\ngezsbBw/fhzHjx9HYmIiwsLCcPz4cYwfP75T1yUiIiIi6snMWjazaNEiPPnkkxg1ahSio6Oxbt06\nqNVqJCYmAgCWLFmCH3/8EQcOHBDOycnJQW1tLYqKilBZWYnjx48DAIYPHw4HBwcMGTLE5B4+Pj5w\ndHREeHh4m7FY6k8O1PNkZmYCAKKiokSOhKwZxwmZg+OEzMFxQu3piqXfZiXvs2fPRklJCZKTk6FW\nqxEREYF9+/YhICAAAKDRaHDx4kWTc6ZPn47Lly8Lr0eOHAmJRAKdTmfB8ImIiIiI7IdZdd7FZvxb\nC2feqTWcASFzcJyQOThOyBwcJ9SershhzdphlYiIiIiIxMfknYiIiIjIRjB5JyIiIiKyEaJs0kRE\nRETUExkMBtTX10Ov14sdCnUDhUIBqbR758KZvBMRERFZgMFgQE1NDRQKBeRyOSQSidghURdq+vd2\ncnLq1n9rLpshIiIisoD6+nooFArIZDIm7nZAIpFAoVCgrq6uW+/L5J2IiIjIAvR6fbcvoSBxyWQy\ndHfVdY4wIiIiIgvhjDt1NSbvREREREQ2gsk7EREREZGNYPJORERERN1u8+bNkEqluHz5skl7amoq\nhgwZAkdHR3h6eooUnfViqUgiIiIi6nYSiaTZMwKXLl3CE088gcmTJ2Px4sVwcXERKTrrxeSdiIiI\niLrdk08+iUceeQQKhUJoO3r0KHQ6Hd577z1ERESIGJ314rIZIiIiIup2TXXSjWm1WgCAm5ubxe5T\nXV1tsWtZAybvRERk9QqKLuHXC/+FXq8TOxQiuzVnzhz069evWfubb75pUt9eKpXiueeew549ezBs\n2DA4OTkhIiIC33zzjcl5N69579evH1588UUAQEhICKRSKd5++22h//r16zFs2DAolUr4+fkhMTER\npaWlJteMiYnBkCFDcPz4cdx5551wdXXFvHnzTL524sQJxMTEwMXFBf3798fOnTsBAOnp6Rg3bhyc\nnZ0xePBg7N+/3wLfNctj8k5ERFbt8M9f4N1PF+KjL/4He49tFzscIrvV0hr11tozMjLw/PPP45FH\nHsHKlStRW1uLP/7xjybJ9s3nffDBB3jkkUeE423btuGBBx4AALzzzjt47rnn4Ofnh/feew+PPPII\nPv74Y/zhD39AfX29yTVLS0tx1113YejQoVi9ejWmT58ufK2srAwzZ87E6NGjsXLlSri4uODxxx9H\namoqZs2ahenTp+Pdd99FVVUVZs+ejYqKCst9Ay2Ea96JiMgq6fU67D6yGYezvxDaDmd/icmR98HZ\n0VXEyIioPadPn0ZOTo4wUx8TE4Phw4dj+/bteO6551o855577sG5c+ewY8cO3HvvvQgKCgIAFBUV\n4Z133kFsbCy+/vprIeEfPnw44uPj8dFHH5lcs7CwEGvWrBFm3I1ptVps3boVjz76KABgypQpGDx4\nMB577DGkp6dj7NixAIDBgwdj6tSp+L//+z8kJCRY7htjAZx5JyIiq1PXUItNe1eaJO4AUFdfgx9O\npIkUFZFlvfnmm8Lsc1f89+abb4r23iZPnmyyxGbYsGFwc3PDhQsXOnytAwcOoL6+HgsWLDCZqX/i\niSegUqnw1VdfmfR3cHDAM8880+K1lEqlkLgDwMCBA9G7d28MHDhQSNwBYMyYMQDQqXi7GpN3IiKy\nKpXV1/C3Xa/j+G/HhDYfj77C8XfZX0KnaxAjNCIyU2BgYLM2Dw+PZmvUzZGbmwugMdE2JpVKERYW\nhkuXLpm0+/v7N3sQtknfvn2btbm7uzeLt+mB2c7E29WYvBMRkdW4WqbGqtTFuKQ+I7TFjLwHLz/y\nPnop3QEAZZXFyD7/g1ghEtmtlta7A4BO1/xBcplM1mJfg8Fg0ZhaolQqW/1aa3GJGW9HMXknIiKr\ncFF9Bu/vXIyr5WoAgAQSPHDH03jgjgQo5I6YcNs0oe+hn/ZY5Q9Voo548803YTAYuuw/Sy+b8fDw\nQFlZWbP2m2e+LS04OBgGgwFnzpwxaTcYDDh37hxCQkK69P7Whsk7ERGJ7npNBTbsWYbr1dcAAHKZ\nAgkzXkbMyLuFPhNuuwsOMjkA4HLheVwoyBElViJ71b9/f5SXl+PEiRNCm1qtxu7du7v0vrGxsVAo\nFFizZo3JL+3btm2DVqvF3Xff3cbZPQ+TdyIiEt3Ji5moqq0EALg49cLzD76N4QPGmfTp5dwbowbH\nCK8P/fzv7gyRyO49/PDDcHZ2xn333Yc1a9Zg+fLlGDt2LAYNGtSl9/Xy8sJrr72GAwcOIC4uDmvX\nrsWiRYswd+5cjBw5Ek8//XSX3t/amJ28p6SkIDQ0FEqlElFRUUhPT2+1b21tLeLj4zF8+HAoFApM\nnjy5WZ/PP/8cU6dOhY+PD9zc3DB27Fh88cUXLVyNiIh6urNXfhGOY0beg35+g1vsZzwT/+tv/8HV\nMnWXx0ZEjTw9PbF79264uLhg8eLF2Lp1K1asWIGZM2ea9OtIPXhzvfrqq1i3bh00Gg1efPFFbN++\nHQkJCThw4ADkcnmz+7SmI3HdSrxdSWIwY9FgamoqnnjiCaxfvx7R0dFYu3YtNm3ahJycHAQEBDTr\nX1VVhRdffBG333479u7di7KyMnz77bcmfZKSkuDv74/JkyfD09MT27Ztw9tvv43vvvsO0dHRJn3L\ny8uFY3d3986+V+rhMjMzAQBRUVEiR0LWjOPE+hgMBry2MQHXrjdWdXjhof9FsO/AVvun7H4Lp3N/\nBgBMGjETD06aa/GYOE7IHDePk5qaGjg5OYkZEomgrX/3rshhzZp5X7VqFRISEpCQkIBBgwZhzZo1\n8PPzw7p161rs7+zsjJSUFMydO7fFkjwAsHr1arz88suIiopCaGgoXn/9dURGRnb5uikiIrIumpIr\nQuKudHRBoE//NvvfOfIe4fiHkweE5TZERPag3eS9vr4eWVlZiI2NNWmPi4tDRkaGRYOpqKiAh4eH\nRa9JRETW7fTlbOF4YMAwSKUtl2xrMjhoBPy8Gnde5KZNRGRv2k3ei4qKoNPpoFKpTNpVKhU0Go3F\nAlm7di3y8/PxxBNPWOyaRERk/c5evrHefVDQiHb7SyQSxIy4sfb9++yvuGkTEdkNB7EDAIDPPvsM\nixcvxs6dO1vckctY0/oyotZwjJA5OE6sg06vw5nLx4XX9eVSs/5tpPpecJI7o6a+CqWVRdi1/1P0\n6zPU4vFxnJA5msZJcHAw17zboYqKCpPymcbCwsIsfr92Z969vb0hk8mg1WpN2rVaLXx9fW85gH/9\n61948sknsXXrVkyfPv2Wr0dERLajqCIPDfp6AICrY2/0UnqadZ5M6oCBvpHC61MFx7hpExHZhXZn\n3uVyOSIjI5GWloYHH3xQaE9LS8OsWbNu6eY7d+5EfHw8tmzZgvvvv9+sc/jkP7WG1SHIHBwn1uWr\nH27smHhb2KgO/bsMDB+AU5uOoUFXj+JKNTz9XdC/7xCLxMVxQuZoqdoM2Z9evXq1+llhXG3GUsyq\nNrNo0SJs3rwZGzduxOnTp7FgwQKo1WokJiYCAJYsWYIpU6aYnJOTk4Ps7GwUFRWhsrISx48fx/Hj\nN/40umPHDjz++ONYsWIFJkyYAK1WC61Wi9LSUgu+PSIismZnOrje3ZibS29EDZ4kvM48873F4iIi\nslZmrXmfPXs2SkpKkJycDLVajYiICOzbt0+o8a7RaHDx4kWTc6ZPn47Lly8Lr0eOHAmJRAKdTgcA\n2LBhA3Q6HZKSkpCUlCT0mzRpUrOa8ERE1PNU1VYiV3sOACCBBAMDhnX4GreHTcCxkwcAALnasxaN\nj4jIGpn9wGpiYqIw036zTZs2NWu7OZm/2aFDh8y9NRER9UDnrpyAwaAHAAT69IeL0q3D1whSDRCO\nC4pyUd9QB7mDwmIxEhFZG7OWzRAREVnaGeP67kHDO3UNZydX+PT2BwDo9TrkXW174oiIyNYxeSci\nIlGcuXJjvfvgTibvABCkulGK7fLvy3CIiHoqJu9ERNTtSq4V4mpZAQBA7qBAP7/wTl8r2PdG8p6r\nYfJORD0bk3ciIup2p402ZurfdyjkDvJOX8t45j2XM+9EViUmJgZ33nmn2GEIYmJiMHnyZLHDuCVM\n3omIqNudvXIjeR8U2PklMwAQ0KcfpFIZAOBqWQGqaipv6XpE1LIffvgBb731Fq5du2b2ORKJBFKp\n9aSbEolE7BBumfV8N4mIyC7oDXqLrXcHGpfd9PUOEV5f1p6/pesRUcsyMjLw9ttvo6yszOxz0tLS\n8M0333RhVPaHyTsREXWr/KsXcb26ceaul9Idft7Bt3xNLp0h6noGg8HsvtXV1QAABwcHODiYXZmc\nzMDknYiIupXxeveBQcMhldz6j6JgJu9EXeqtt97Cyy+/DAAICQmBVCqFTCbDd999h5CQEEyfPh3f\nfvstxo4dC6VSiZUrVwJoeY35X//6V0ycOBF9+vSBUqnEbbfdho0bNza7Z9N1jx49ijFjxkCpVKJ/\n//7YunVrs76//PILJk2aBGdnZwQGBiI5ORkff/wxpFKpyaahrfnwww9x2223QalUQqVSYe7cuSgu\nLu7Mt6rL8VchIiLqVmcvW269exPjijOXNedgMBh6xNpWImvx4IMP4uzZs9ixYwc++OADeHl5QSKR\nIDw8HBKJBOfOncOsWbPwzDPPYO7cuQgKCgLQ8hrz1atX4+6778ZDDz0EiUSCPXv24JlnnoFOp8Of\n/vQnoZ9EIsGFCxcwa9YsPP3005gzZw4+/vhjxMfHIyoqCuHhjVWqCgoKcOedd0IqleIvf/kLXFxc\n8I9//ANyudysz4HExERs2rQJc+bMwfz583HlyhWsWbMGP/74I3788UcoFNa18RuTdyIi6jZ1DbX4\nreCU8HrQLa53b6Ly6AtHuRNq62twraoUZZXF8OjlbZFrE3WVvce24+v/pHbZ9e8a8xCmj33EIteK\niIjA7bffjh07duDee+8VkvMmFy5cwL///W/MmDGj3WudO3cOTk5Owut58+Zh6tSpeO+990yS96a+\n33//PaKjowEAs2bNQmBgIDZt2oT//d//BQCsWLECZWVlyMzMxMiRIwEA8fHxGDBgANqTkZGBv//9\n79i6dSsee+wxof2uu+7ChAkTsGXLFsydO7fd63QnLpshIqJucyE/Bw26egCAyiPAYgm2VCpDoE9/\n4TU3ayLqXgEBAWYl7gCExL2hoQGlpaUoLi5GTEwMfvvtN1RUVJj0HThwoJC4A4C3tzcGDRqECxcu\nCG3ffPMNRo8eLSTuANC7d2+TZLw1O3fuRK9evRAXF4fi4mLhv4EDB0KlUuHQoUNmvafuxJl3IiLq\nNmeuZAvHlpp1bxLsG4bz+ScBNG7WNHzAOIten4haFxoaanbfPXv24J133kF2djZ0Op3QLpFIUF5e\njl69egltN8/wA4CHhwdKS0uF17m5uRg9enSzfubMvJ87dw4VFRVQqVTNviaRSFBYWNjuNbobk3ci\nIuo2Zy7fKBFp6eQ9SDVQOObMO9mC6WMfsdiyFrEplUqz+qWnp+OBBx7AHXfcgQ0bNsDf3x8KhQJf\nffUVVq9eDb1eb9JfJpO1eJ2OVL5pi16vh7e3N1JTU1u8poeHh0XuY0lM3omIqFtU1VQi72rjn7ql\nEikG9I2w6PVNKs4UnofeoLdIJRsiamSJh8A/++wzKJVK7N+/H3L5jZ2VDx482OlrBgcH4/z55vs7\nnDvX/i/x/fv3x4EDBzBmzBg4Ozt3OobuxE81IiLqFvlFF4Vj/z4hUDpa9gelRy9v9FK6AwBq66pR\nWJpv0esT2TsXFxcAMFmy0lEymQwSicRkuUxpaSk2bdrU6WtOnToV//3vf/HTTz8JbSUlJfjnP//Z\n7rkPPfQQdDod3n777WZf0+v1HdqQqrsweSciom5RUJQrHPf17mfx60skEgQZlYzM1XDpDJElRUVF\nwWAw4JVXXsHWrVuRmpqKq1evdugad999N65fv44pU6Zgw4YN+J//+R9ERkbCz8+v03G9/PLLcHd3\nx9SpU7Fs2TK8//77mDBhQpvlKptMnDgR8+bNw8qVKzFt2jSsXr0a69atw6JFi9CvXz/8+9//7nRc\nXYXJOxERdYv8okvCsb8FdlVtifHSmcva5n9GJ6LOi4yMxIoVK5CTk4OEhAQ8+uijyMnJgUQiaTNB\nNv7apEmTsGXLFpSWlmLhwoXYvHkzFixYgPnz57d4XmvXNW4PCAjA4cOHMXToUCxfvhyrV6/GE088\ngfj4eAAwKUt587lA4wZNGzduRGlpKZYuXYolS5bgwIEDePjhh5ttMGUNJAZLrfjvQuXl5cKxu7u7\niJGQNcvMzATQODNA1BqOE/G8t/1FXC5sTKiff+BtDAy8zeL3yMn9Get2vwUACFKF4cWHV3bqOhwn\nZI6bx0lNTU2zRJHEk5SUhI8++giVlZVdumlbW//uXZHDcuadiIi6nF6vg7r4xhblfl5dM/MeZFTr\nPf/qRdQ31HfJfYjIutTU1Ji8Li4uxrZt2zBx4sQet9syq80QEVGXu1quQb2uDgDg5uKBXs5d81dU\nF6UbvN19UVSugU7fgIKiiwj2Hdj+iURk08aNG4eYmBiEh4dDo9Hg448/RkVFBV577TWxQ7M4Ju9E\nRNTlCozWu3fFw6rGglVhKCrXAAByteeZvBPZgRkzZuBf//oXPvroI0gkEkRGRmLTpk0mu7P2FEze\niYioyxV0w8OqTYJ8w5B19ggAbtZEZC/eeecdvPPOO2KH0S245p2IiLpc/tVLwrG/d0iX3ivYaKdV\nloskop7G7OQ9JSUFoaGhUCqViIqKQnp6eqt9a2trER8fj+HDh0OhULRaZue7775DVFQUlEolBgwY\ngA0bNnT8HRARkdUzXTbTtTPvAX36CTurakvzUF17vUvvR0TUncxK3lNTU5GUlISlS5ciOzsb48eP\nx7Rp05CXl9dif51OB6VSifnz52PmzJkt9rl06RJmzJiBCRMmIDs7G6+88grmz5+Pzz//vPPvhoiI\nrE517XWUVDRu5CKTOsDHo2+X3k8hd4Sf0S8IVwp/69L7ERF1J7OS91WrViEhIQEJCQkYNGgQ1qxZ\nAz8/P6xbt67F/s7OzkhJScHcuXPRt2/LH9Lr1q1D3759sXr1agwaNAhz587FU089hffee6/z74aI\niKyO8c6qvp4BcJDJu/yexps1cekMdScb2D6HLEiMf+92k/f6+npkZWUhNjbWpD0uLg4ZGRmdvvGx\nY8cQFxdn0jZ16lRkZmZCp9N1+rpERGRdTB9WDemWe5ok73xolbqJQqFATU0N8xg7YTAYUFNTA4VC\n0a33bbfaTFFREXQ6HVQqlUm7SqXCwYMHO31jjUbT7BcClUqFhoYGFBUVNbsfERHZpnwRkvcgJu8k\nAqlUCicnJ9TV1aG+nhuE2QNHR0dIpd1b/8XmSkU2bUVM1BqOETIHx0n3OZt7Uji+XlLbLd97vUEP\nB6kcDfp6lFcW4/ujh+Ds2KvD1+E4IXNwnFBrwsLC2u/UQe3+quDt7Q2ZTAatVmvSrtVq4evr2+kb\n+/r6tnhNBwcHeHt7d/q6RERkPQwGA8quFwqvPVy656+qUokUnq43fkYVVRZ0y32JiLpauzPvcrkc\nkZGRSEtLw4MPPii0p6WlYdasWZ2+8bhx47B7926Ttv379yMqKgoymazV86Kiojp9T+rZmmY+OEao\nLRwn3etqmRoNGY3LB1yV7pgwbhIkEkm33PtK1S8o/PkKAMDJTdqhf3OOEzIHxwm1p7y83OLXNGuR\nzqJFi7B582Zs3LgRp0+fxoIFC6BWq5GYmAgAWLJkCaZMmWJyTk5ODrKzs1FUVITKykocP34cx48f\nF76emJiI/Px8LFy4EKdPn8Y//vEPbNmyBS+99JIF3x4REYnJuNJMX++QbkvcAdOdXDUlV7rtvkRE\nXcmsNe+zZ89GSUkJkpOToVarERERgX379iEgIABA48OnFy9eNDln+vTpuHz5svB65MiRkEgkwhPY\nISEh2Lt3LxYuXIj169fD398fH374Ie677z5LvTciIhKZaaWZrt2c6Wa+nkHCsbrkchs9iYhsh9kP\nrCYmJgoz7TfbtGlTs7abk/mWTJw4kQ95EBH1YGJUmmmi8gwQjq+WqdGgq++WGvNERF2pe2vbEBGR\nXRGjxnsTJ4USHr36AAD0eh2ulmm69f5ERF2ByTsREXWJ2rpqFJU3JsxSiRS+RjPh3cXPM1A41nDp\nDBH1AEzeiYioSxQU30iWVZ4BkDt07y6EAODrZZS8F/OhVSKyfUzeiYioS5gsmfHq3odVm/ChVSLq\naZi8ExFRlxDzYdUmxjPv2pI8UWIgIrIkJu9ERNQlxCwT2cTXaM17YWkBdLoGUeIgIrIUJu9ERGRx\nBoPBZIMmsWbenRRKeLh6AwB0+gZcLVeLEgcRkaUweSciIosrrbiKmroqAICzUy/0dvUSLRaV0dIZ\nNR9aJSIbx+SdiIgsLv+mJTMSiUS0WEzLRTJ5JyLbxuSdiIgszni9e1+Rlsw0MV73rmXyTkQ2jsk7\nERFZXL4VlIls4utlVC6ymOUiici2MXknIiKLs4aHVZsY7+xaWFoAnV4nYjRERLeGyTsREVlUXX0t\nrpY1VnWRSKTwM5r5FoPS0UV4YFanb0BRGSvOEJHtYvJOREQWpSm5AoNBDwDo09sPCrmjyBGZrnvn\nQ6tEZMuYvBMRkUXlX70oHIu1OdPNjJN3rnsnIlvG5J2IiCyqoPjGenexK800MX5oVVOSJ2IkRES3\nhsk7ERFZlGmN9xDR4jBmsmyGM+9EZMOYvBMRkcUYDIabKs1YybIZrxsVZ7Rl+aw4Q0Q2i8k7ERFZ\nzLWqUlTVVAAAHBVKePbyETmiRs6OrnB38QQA6HQNKC7XiBwREVHnMHknIiKL0RTfqOTi6xkIiUQi\nYjSmfL2MH1plxRkisk1M3omIyGKMyzAarzO3BiwXSUQ9AZN3IiKyGOOZdz8v60rejTeL4kOrRGSr\nmLwTEZHFcOadiKhrMXknIiKLMBgMUJsk70Ft9O5+xsm7tjQfelacISIbZHbynpKSgtDQUCiVSkRF\nRSE9Pb3N/idOnEBMTAycnZ0RGBiIZcuWNevzz3/+EyNHjoSLiwv8/PzwxBNPQKvVdvxdEBGR6Cqq\nym9UmpE7waOXt8gRmXJ2coWbiwcAoEFXj6Jy/rwhIttjVvKempqKpKQkLF26FNnZ2Rg/fjymTZuG\nvLyWd6mrqKhAbGws/Pz8kJWVhQ8++AArV67EqlWrhD5Hjx7Fk08+ifj4eJw6dQp79uxBTk4OHn/8\nccu8MyIi6laakhvryK2t0kwT06UzXPdORLbHrOR91apVSEhIQEJCAgYNGoQ1a9bAz88P69ata7H/\ntm3bUF1djU8++QTh4eF44IEHsHjxYrz//vtCn2PHjiEwMBB//vOfERwcjNGjR+P555/Hf/7zH8u8\nMyIi6lbWvN69ielDq1z3TkS2p93kvb6+HllZWYiNjTVpj4uLQ0ZGRovnHDt2DBMnToRCoRDapk6d\nioKCAuTmNu68Fx0dDbVajS+//BIAUFRUhB07dmDGjBmdfjNERCQe49rpvl7Wtd69ienMe8t/PSYi\nsmYO7XUoKiqCTqeDSqUyaVepVDh48GCL52g0GgQGBjbrbzAYoNFoEBwcjLFjx2L79u147LHHUF1d\njYaGBsTFxWHz5s1txpOZmdleyGTnOEbIHBwnlnc+95RwXFFcY5Xf47Jr14Xj3/JOtxujNb4Hsj4c\nJ9SasLAwi19TtGozp06dwvz58/HGG2/gp59+wjfffAO1Wo0//elPYoVERESdZDAYUFZ1VXjd29m6\nHlZt4q68Ede16mLoDXoRoyEi6rh2Z969vb0hk8maVYHRarXw9fVt8RxfX98W+0skEuGcFStWYMyY\nMVi0aBEAICIiAikpKZg4cSKWL18Of3//Fq8dFRXV/rsiu9Q088ExQm3hOOkaFVVlqM2oBgAo5E64\nI3oypBLrrEa878THqKgqg07fgH5hgejT269ZH44TMgfHCbWnvLzc4tds95NVLpcjMjISaWlpJu1p\naWmIjo5u8Zxx48bhyJEjqKurE9r2798Pf39/BAcHAwCqqqogk8lMg5FKIZFIoNdzJoSIyJaYrHf3\nDLTaxB1EbqhGAAAgAElEQVQA/LhZExHZMLM+XRctWoTNmzdj48aNOH36NBYsWAC1Wo3ExEQAwJIl\nSzBlyhSh/6OPPgpnZ2fMmTMHJ0+exK5du/Duu+/ihRdeEPrcfffd2LNnD9avX4+LFy/i6NGjWLBg\nASIjIxEQEGDht0lERF3JtNKMdX+G+3rdSN7VxSwXSUS2pd1lMwAwe/ZslJSUIDk5GWq1GhEREdi3\nb5+QZGs0Gly8eFHo7+bmhrS0NMybNw+jRo2Ch4cHXnrpJSQlJQl9nnrqKVRWVmLt2rV48cUX0bt3\nb0yePBkrVqyw8FskIqKupjFKgv2stNJME+OdXznzTkS2xqzkHQASExOFmfabbdq0qVnb0KFDcfjw\n4TavOW/ePMybN8/cEIiIyErZQo33JsYz76z1TkS2xnoXJRIRkc1QGyfvXtadvBuvedeW5EGv14kY\nDRFRxzB5JyKiW1JRVY7r1dcAAAoHR3j06iNyRG1zUbqhl9IdAFCvq0NJxdV2ziAish5M3omI6JZo\nSm6sd7f2SjNNVHxolYhslPV/whIRkVUzXjdu7UtmmvgZP7TKde9EZEOYvBMR0S1R29DDqk1MykWW\ncOadiGwHk3ciIroltlRppolxOUvOvBORLWHyTkREt0RrlPxae433Jqw4Q0S2isk7ERF1WmX1NVRU\nlwP4vdKMm3VXmmlyc8WZ4muFIkdERGQeJu9ERNRpxktmVJ4BNlFppomvF3daJSLbYzufskREZHWM\nyyzaynr3Jn4sF0lENojJOxERdZrWZGdV21jv3sSX5SKJyAYxeSciok5TG9d49wwQMZKO82O5SCKy\nQUzeiYio04zXittKpZkmxn8pKCzJZ8UZIrIJTN6JiKhTrldfQ0VVGQBA7qCAp5uPyBF1jItTL7g5\newBorDhTVK4VOSIiovYxeSciok6x5UozTYx3WtVw6QwR2QDb+6QlIiKrYLre3bYqzTQxXuqj5kOr\nRGQDmLwTEVGnmKx397St9e5NjH/p0LBcJBHZACbvRETUKcbJrvHyE1tiMvPOjZqIyAYweSciok7R\nlOQJx7a6bMb4l47C0nzoWHGGiKwck3ciIuqw6zUVuFZVCgCQyxTwsrFKM02cHV3h7uIJAGjQ1aO4\nXCNyREREbWPyTkREHWa8I6nKMwBSqUzEaG6N8V8N+NAqEVk7Ju9ERNRhxg+r2uqSmSYsF0lEtoTJ\nOxERdZhp8h4gYiS3juUiiciWmJ28p6SkIDQ0FEqlElFRUUhPT2+z/4kTJxATEwNnZ2cEBgZi2bJl\nzfrU19fj9ddfR2hoKJycnBASEoK//e1vHX8XRETUrdQmlWZss0xkE1+jMpcsF0lE1s7BnE6pqalI\nSkrC+vXrER0djbVr12LatGnIyclBQEDzGZeKigrExsYiJiYGWVlZyMnJwZw5c+Dq6oqFCxcK/R56\n6CEUFBTgH//4BwYMGACtVovq6mrLvTsiIuoSJsm7zS+bufFzTFvGijNEZN3MSt5XrVqFhIQEJCQk\nAADWrFmDr7/+GuvWrUNycnKz/tu2bUN1dTU++eQTKBQKhIeHIycnB++//76QvO/fvx+HDh3Cb7/9\nBk/Pxif9g4Jse/aGiMgeXLtehoqqMgCAQu4Eb3eVyBHdGmdHV7i7eqG8shg6XQOKytRih0RE1Kp2\nl83U19cjKysLsbGxJu1xcXHIyMho8Zxjx45h4sSJUCgUQtvUqVNRUFCA3NxcAMCePXswatQo/PWv\nf0VgYCAGDhyIBQsW4Pr167fyfoiIqIvlF10Ujv29gm260kwTP5OKM1w6Q0TWq92Z96KiIuh0OqhU\npjMrKpUKBw8ebPEcjUaDwMDAZv0NBgM0Gg2Cg4Nx4cIFHDlyBI6Ojti1axfKysrw/PPPQ61WY+fO\nna3Gk5mZac77IjvGMULm4DjpvBN5NyZuFHDpEd9LSYOjcJz16zEMD7oDAMcJmYfjhFoTFhZm8Wua\ntWymK+j1ekilUmzfvh2urq4AgL/97W+46667cPXqVfTp00es0IiIqA2l17XCsYezbS+ZadLb2Vs4\nLqsqEjESIqK2tZu8e3t7QyaTQavVmrRrtVr4+vq2eI6vr2+L/SUSiXCOn58f+vbtKyTuABAeHg6D\nwYDLly+3mrxHRUW1FzLZqaaZD44RagvHya3bn/OJcDw+Kgb9/AaJGI1leKl74YfzXwEA6lAptHOc\nUFv4eULtKS8vt/g1213zLpfLERkZibS0NJP2tLQ0REdHt3jOuHHjcOTIEdTV1Qlt+/fvh7+/P4KD\ngwEA0dHRKCgoQFVVldDnzJkzkEgkQh8iIrIudQ21KCzNBwBIIIG/d8/4vDaumFNYWgA9K84QkZUy\nq877okWLsHnzZmzcuBGnT5/GggULoFarkZiYCABYsmQJpkyZIvR/9NFH4ezsjDlz5uDkyZPYtWsX\n3n33Xbzwwgsmfby8vBAfH49Tp07h6NGjSEpKwqxZs+Dt7d0sBiIiEp+m+Ar0Bj0AoE9vPzjKnUSO\nyDKUjs7wcG382aPTN+BaTanIERERtcysNe+zZ89GSUkJkpOToVarERERgX379gk13jUaDS5evFF9\nwM3NDWlpaZg3bx5GjRoFDw8PvPTSS0hKShL6uLi44MCBA5g/fz5Gjx4NDw8P3H///Vi+fLmF3yIR\nEVlK/lWjSjN9QsQLpAv4egWhtLJxvXt51VWTdfBERNbC7AdWExMThZn2m23atKlZ29ChQ3H48OE2\nrxkWFoavv/7a3BCIiEhkxmUiA7z7iRiJ5fl6BiAn9ycAQFnVVQQjXOSIiIiaM2vZDBEREQDkX70k\nHPft08OSd68bGwWWVV0VMRIiotYxeSciIrMYDAbkF10SXve05N2PyTsR2QAm70REZJaSa4WoqWus\nEObi1AvuLp4iR2RZxhVnrtWUQMeKM0RkhZi8ExGRWfKMHlbt6x0CiUQiYjSW56RQwqNX4x4jBoMe\nFTUlIkdERNQck3ciIjKL8cOqPW3JTBM/o9l3Lp0hImvE5J2IiMxiXCaypybvfGiViKwdk3ciIjKL\nycOqPaxMZBM/rxsz7+VVRSJGQkTUMibvRETUrqraSpRcKwQAyKQOUHn2FTmiruHryZl3IrJuTN6J\niKhdBUW5wrGvVyAcZHIRo+k6vp4BwvG1mhI06OpFjIaIqDkm70RE1K78myrN9FSOCiU83XwANFac\nKSwtEDkiIiJTTN6JiKhd9vCwahN/r2DhOO/qBREjISJqjsk7ERG1K8+4TGQPfVi1SaBPf+H4SuFv\nIkZCRNQck3ciImqTTtcATfEV4XXfPiHiBdMNglQDhOPL2vMiRkJE1ByTdyIiapO2NF94cNPD1Rsu\nTr1EjqhrGc+85129AJ1eJ2I0RESmmLwTEVGbTOq79/D17gDg5uIBZ0XjLyj1DXXQllxp5wwiou7D\n5J2IiNpUYLzevYcvmWni5eovHF/Wct07EVkPJu9ERNSmvKv287BqEy9XP+H4ciHXvROR9WDyTkRE\nrTIYDMi/ekl4bQ/LZgDT5P0KH1olIivC5J2IiFp1raoUldXlAABHuRO83FUiR9Q9jJP3/KJL3GmV\niKwGk3ciImqV8ay7v3cIpBL7+LHhJHeGq6M7AKBBVw91MR9aJSLrYB+fwkRE1Cn2tLPqzYwfWr3C\nde9EZCWYvBMRUatMykR6h4gWhxhMHlrluncishJM3omIqFXGM+8BdjfzzoozRGR9zE7eU1JSEBoa\nCqVSiaioKKSnp7fZ/8SJE4iJiYGzszMCAwOxbNmyVvump6dDLpfjtttuMz9yIiLqUnX1tSgsKwAA\nSCRS+HkFixxR9/J09RWO1UWXUd9QJ2I0RESNzEreU1NTkZSUhKVLlyI7Oxvjx4/HtGnTkJeX12L/\niooKxMbGws/PD1lZWfjggw+wcuVKrFq1qlnfsrIyPPXUU5gyZcqtvRMiIrIodXEuDAY9AMCntz8U\nckeRI+pejg5K9HFvnH3X6RtQUJQrckRERGYm76tWrUJCQgISEhIwaNAgrFmzBn5+fli3bl2L/bdt\n24bq6mp88sknCA8PxwMPPIDFixfj/fffb9b36aefxpw5czB27NhbeydERGRRJuvd7WRn1ZsFqgYI\nx1w6Q0TWoN3kvb6+HllZWYiNjTVpj4uLQ0ZGRovnHDt2DBMnToRCoRDapk6dioKCAuTm3pi5SElJ\nQWFhIZYuXdrZ+ImIqIvY486qNwtS9ReOuVkTEVmDdpP3oqIi6HQ6qFSmG3OoVCpoNJoWz9FoNC32\nNxgMwjm//vorli1bhk8//RQSiaSz8RMRURe5UvibcGy3M+8+xjPvv7XRk4ioeziIcdO6ujo8/PDD\neO+99xAUFASgcQtuc2RmZnZlaNQDcIyQOThO2lavqzMpj1iivo7MIvv7nl3NKxeO1UW5OPafH+Ag\nk4sYEVkjfp5Qa8LCwix+zXZn3r29vSGTyaDVak3atVotfH19WzzH19e3xf4SiQS+vr5Qq9XIyclB\nfHw85HI55HI5li1bhhMnTkChUODAgQO38JaIiOhWFVXkCw+r9nbuAye5s8gRiUPh4Ah3pRcAwAAD\nSq9r2zmDiKhrtTvzLpfLERkZibS0NDz44INCe1paGmbNmtXiOePGjcMrr7yCuro6Yd37/v374e/v\nj+DgYDQ0NODEiRMm56xduxYHDhzA7t27ERzcejmyqKgos94Y2Z+mmQ+OEWoLx4l59v5wTjiOGBBl\nd98v43FyqjgCmae/AwA4ezogaoR9fS+odfw8ofaUl5e336mDzKo2s2jRImzevBkbN27E6dOnsWDB\nAqjVaiQmJgIAlixZYlLq8dFHH4WzszPmzJmDkydPYteuXXj33XfxwgsvAAAcHBwwZMgQk/98fHzg\n6OiI8PBwODvb5wwPEZG1OF9wUjge0HeoiJGIL8ho3fsVrnsnIpGZteZ99uzZKCkpQXJyMtRqNSIi\nIrBv3z4EBAQAaHxA9eLFG1UJ3NzckJaWhnnz5mHUqFHw8PDASy+9hKSkpK55F0REZDH1DfXIVZ8V\nXvf3HyJiNOILMi4XyYozRCQysx9YTUxMFGbab7Zp06ZmbUOHDsXhw4fNDuSNN97AG2+8YXZ/IiLq\nGlcKz6Ne17ibaB93P7i7eoockbj69ukHiUQKg0EPbUkeauuq4ahQih0WEdkps5bNEBGR/Tiff2PJ\nTP++9j3rDgCOcif4ejb+pdkAA/KuXhA5IiKyZ0zeiYjIxG/5p4Tj/na+3r2J8br3y1queyci8TB5\nJyIigU6vwwV1jvDa3h9WbRJovO69kOveiUg8TN6JiEiQf/UiauuqAQC9Xb3g6eYjckTWwfih1St8\naJWIRMTknYiIBDcvmZFIJCJGYz38vYMhlcoAAIVlBaiuvS5yRERkr5i8ExGR4DfWd2+RwsERfl5B\nwusrhXxolYjEweSdiIgAAHqD/qaZd1aaMWa6WROXzhCROJi8ExERAEBbkofrNRUAABelG1QeASJH\nZF24WRMRWQMm70REBMC0vvsA/yFc736TQJ/+wjErzhCRWJi8ExERANZ3b4+fVzBkssaNyYvLtais\nviZyRERkj5i8ExERDAYDfjPZWZXJ+83kDnIE9rkx+37m8nERoyEie8XknYiIUFSuQfn1EgCAk8IZ\nfb2DRY7IOoWH3C4c5+T+JGIkRGSvmLwTEZHJkplQ/3ChpjmZGhJ8I3k/dekn6A16EaMhInvE5J2I\niLhkxkyBqv5wVboDACqry5HHeu9E1M2YvBMREc6bbM7E+u6tkUqkCA8eKbw+eSlLxGiIyB4xeSci\nsnOlFUUoLtcCAOQOCpOSiNTcEON175e47p2IuheTdyIiO3eh4MZ6936+g+Agk4sYjfUbHDQCEknj\nj89czVmWjCSibsXknYjIzp1nffcOcVG6Idg3DABggAGnc38WOSIisidM3omI7BwfVu24oSGRwvEp\nlowkom7E5J2IyI5VVJVDU3IFACCTOiDEd6DIEdmG8GDjeu8/s2QkEXUbJu9ERHbMeL17kGoAFHJH\nEaOxHQE+oejl3BsAcL36Gq5oz4scERHZCybvRER27FzeCeGYS2bMd3PJyFOsOkNE3YTJOxGRndIb\n9Pjlt2PC67CACBGjsT1DjNe9s947EXUTs5P3lJQUhIaGQqlUIioqCunp6W32P3HiBGJiYuDs7IzA\nwEAsW7bM5Ouff/45pk6dCh8fH7i5uWHs2LH44osvOvcuiIiowy4W5KCsshgA4OLUCwMDhokckW0x\nLhl5WXseFVXlIkdERPbArOQ9NTUVSUlJWLp0KbKzszF+/HhMmzYNeXl5LfavqKhAbGws/Pz8kJWV\nhQ8++AArV67EqlWrhD7fffcd/vCHP2Dv3r3Izs7G9OnTcf/99+Po0aOWeWdERNSmrDNHhOMRA8ZD\nJnMQMRrb4+zkin5+gwD8XjLyMktGElHXMyt5X7VqFRISEpCQkIBBgwZhzZo18PPzw7p161rsv23b\nNlRXV+OTTz5BeHg4HnjgASxevBjvv/++0Gf16tV4+eWXERUVhdDQULz++uuIjIzE7t27LfPOiIio\nVTq9Dj+fzxBe3z5ooojR2K4hRlVnuO6diLpDu8l7fX09srKyEBsba9IeFxeHjIyMFs85duwYJk6c\nCIVCIbRNnToVBQUFyM3NbfVeFRUV8PDwMDd2IiLqpLNXfsH133cGdXfxRH//cJEjsk1D+t1Y9346\n92fo9ToRoyEie9Bu8l5UVASdTgeVSmXSrlKpoNFoWjxHo9G02N9gMLR6ztq1a5Gfn48nnnjC3NiJ\niKiTss58LxyPHDgBUqlMxGhsV1/vfnBzaZx0ul5TgVyWjCSiLmYVCxw/++wzLF68GDt37kRgYGCb\nfTMzM7spKrJVHCNkDnseJzp9A34+e+Mvp846b7v+frTFnO+Lj0sQrl0vBQAc/OFLjAiq6OqwyMrw\n/z/UmrCwMItfs92Zd29vb8hkMmi1WpN2rVYLX1/fFs/x9fVtsb9EIml2zr/+9S88+eST2Lp1K6ZP\nn97R+ImIqIPyS8+jXlcLAOjl5AEvVz+RI7JtfT0GCMf5pZx5J6Ku1e7Mu1wuR2RkJNLS0vDggw8K\n7WlpaZg1a1aL54wbNw6vvPIK6urqhHXv+/fvh7+/P4KDg4V+O3fuRHx8PLZs2YL777/frICjoqLM\n6kf2p2nmg2OE2sJxAvyy91vheNywKRg1apSI0VinjoyTobXhOHL2c+gNehRXqjEwfADcXHp3dYhk\nBfh5Qu0pL7d8CVmzqs0sWrQImzdvxsaNG3H69GksWLAAarUaiYmJAIAlS5ZgypQpQv9HH30Uzs7O\nmDNnDk6ePIldu3bh3XffxQsvvCD02bFjBx5//HGsWLECEyZMgFarhVarRWlpqYXfIhERNampq8bJ\nCzf+xB/JKjO3TOnogn5GD/yyZCQRdSWzkvfZs2dj9erVSE5OxsiRI5GRkYF9+/YhICAAQOMDqhcv\nXhT6u7m5IS0tDQUFBRg1ahTmz5+Pl156CUlJSUKfDRs2QKfTISkpCf7+/sJ/xrP7RERkWb9e+C/q\ndXUAAH+vYPh5BYkcUc/AkpFE1F3MfmA1MTFRmGm/2aZNm5q1DR06FIcPH271eocOHTL31kREZCE/\nGW3MdPvACSJG0rMMCYnEFxlbATSWjKxrqIXCwVHkqIioJzJr5p2IiGzf9epryDFa0sGNmSzH3zsY\nnm4+AICq2kr89xQnqIioazB5JyKyE8d/OyZsIhTsOxDe7i1XDKOOk0gkmDRipvD64E+fQ8cNm4io\nCzB5JyKyE1lcMtOlxg+NhbNTLwBAcbkWx8//IHJERNQTMXknIrID5ZUlOJ93AgAggQS3hzF5tzRH\nhRJ33HZjv5IDmbtgMBhEjIiIeiIm70REduDnc0dhQGMiOSAgAu6uniJH1DPdMWIG5A6N+5vkXb2A\n05ezRY6IiHoaJu9ERHYg6+yNJTOs7d51XJVuGDc0Vnh9MHOXiNEQUU/E5J2IqIcrKtcgV3MWACCV\nyjC8/1iRI+rZ7rz9HkgljT9ez+b9ilzNOZEjIqKehMk7EVEPd/TXb4Tj8KCRcFG6iRhNz+flpjIp\nw3kgi7PvRGQ5TN6JiHowTckVHP75C+H1qPAY8YKxI1Mi7xeOfzl/DNrSfBGjIaKehMk7EVEPZTAY\nsPPb9dDpGwAA/fwGY0TYeJGjsg/+3iEYGhIFADDAgG+zdoscERH1FEzeiYh6qB9PH8b5/JMAAKlE\niocmJwprsanrTYm6Mfv+39OHUF5ZImI0RNRTOIgdQEdJJBKxQyAisnqOznI8vuQPcO7lCADIPHgG\nfRf0Ezkq+/PHBRPh188TOl0Dpj86ERlfnBI7JCLqAm+88QbefPPNbrkXp2CIiHqg8TOHCIl7RWkV\n/vv1GZEjsk9ZB25UmhkWHQKF0ubmzIjIyjB5JyLqYXxDPBAxPkR4/f2uX1FfpxMvIDt28ZQGxepr\nAACFkxzDovnXDyK6NTY3BcCtpqk1mZmZAICoqCiRIyFr1tPHiU6vw3vbX0B+0SUAQES/Ufjg+Odc\ncthBlhwn/zn1LT5NWwMAiLlvJHZt24sg1YBbvi6Jr6d/npB14sw7EVEP8n32V0LirnBwxB9jnmHi\nLrLIQRPh09sfAFDXUIu//zsZJdcKRY6KiGwVk3cioh6itKIIe4/9U3g9dcxD8HTzETEiAgAHmRzP\n3P0XODu6AgCuVZViw7/fQVVtpciREZEtYvJORNRD7Pp+I2rrawAAfl5BmDzyHpEjoiYqzwDMvXsJ\nZLLG1arq4sv4+Mt30aCrFzkyIrI1TN6JiGyc3qDHvmM7cPz8D0Lb7DufFRJFsg4D+g7FY1PmC6/P\n5v2K1IPr+CwXEXUIk3ciIhtWW1eNTV/9L/b9Z4fQNiZ8Mvr3HSpiVNSaqMGTMGPcY8Lr/+R8i6//\nu1PEiIjI1jB5JyKyUUXlGry/czGO/3ZMaBsYeBsemDRXxKioPXGj/oixQ/4gvN53bDv+m3NIxIiI\nyJbwb6pERDbo7JVf8PHelaiqqRDaYkbcjXsnzoFMKhMxMmqPRCLBQ5P/H0orinDmynEAwPYDayGR\nSBE16A5WByKiNnHmnYjIhhgMBnyX/SVSPn9TSNxlMgc8FjsfD0x6mom7jZDJHJAw42X4eQUBAHT6\nBmz9ZhXeT30Z5/JOiBwdEVkzs5P3lJQUhIaGQqlUIioqCunp6W32P3HiBGJiYuDs7IzAwEAsW7as\nWZ/vvvsOUVFRUCqVGDBgADZs2NDxd0BEZAcMBgMuac7ik6/fx2ff/QN6gx4A4ObsgT8/mIwxRssw\nyDYoHV3w7D2vwcPVW2jL1Z7Dh58txYY976Dg93r9RETGzFo2k5qaiqSkJKxfvx7R0dFYu3Ytpk2b\nhpycHAQEBDTrX1FRgdjYWMTExCArKws5OTmYM2cOXF1dsXDhQgDApUuXMGPGDMydOxeffvopjhw5\ngueeew4+Pj64//77LfsuiYhsVPE1LTJPf4cfcw6jsKzA5GtBqjDMnfkKert6iRQd3SpPtz54+bFV\nSPvxM3x//CuhdOTJS5k4dSkLo8PvxPRxj8CjVx+RIyUiayExmFGjauzYsRgxYgTWr18vtA0cOBCz\nZs1CcnJys/7r1q3DkiVLUFhYCIVCAQBITk7G+vXrceXKFQDA4sWLsXv3bpw5c0Y475lnnsGpU6dw\n9OhRk+uVl5cLx+7u7h18i2QvuE01mcPax4neoEdZRRFOX87GjzmH8VvBqRb7jRocg4f/8BzkDopu\njtA+iDFOSq5dxd5j/8SPOYdhwI0fzTKpAwJ8QtHPdxBC/Aahn98gJvNWwto/T0h8XZHDtjvzXl9f\nj6ysLLz00ksm7XFxccjIyGjxnGPHjmHixIlC4g4AU6dOxeuvv47c3FwEBwfj2LFjiIuLMzlv6tSp\n2LJlC3Q6HWQyrtskop5Fp2tATX01auuqUVNXjes113C1TI2rZQW//68aRWUa1OvqWjzfUe6EEQPG\nY/SQyRjQdygfbOxhPN364PG4Bbhz5L344ugWnMr9CUDjevhczVnkas4C2V8AANxdvdDPdxAC+vRD\nL+fecFG6wVXpDlelG1yd3aBUuHB8EPVQ7SbvRUVF0Ol0UKlUJu0qlQoHDx5s8RyNRoPAwMBm/Q0G\nAzQaDYKDg6HRaBAbG9usT0NDA4qKiprdr8mGf7/TXshkp8rLGn+7zSr4WuRIyJqVlZUBALLym48T\n49nOxgaD0dd+/7rB8Pv/Nr42/P7aoNdDp9dBr9dBZ/j9f/U66PQNqK2rQW1ddatJeVukEikGB43A\nqPAYDAsdA4XcscPXINvSt08IEu97HWev/Iovf9iGS+ozzfqUVxYj+3wGss+3PIkmlcrgpHCGg8wB\nDlIHyGRyOMgcIJM5wEEmh1QihUQihUQigRQSQCKBRCJpbMONpF84+v0XAeOvNWOHvyvw5w41GR1+\nJ0aGRXfLvWyuVOTDk+a334mIqAeqrqpBNWrEDsMuhIWFATD9k3d3U7kF4empfxHt/kTUMd31edFu\ntRlvb2/IZDJotVqTdq1WC19f3xbP8fX1bbG/RCIRzmmtj4ODA7y9vUFERERERKbaTd7lcjkiIyOR\nlpZm0p6Wlobo6Jb/PDBu3DgcOXIEdXU3/kS8f/9++Pv7Izg4WOhz8zX379+PqKgorncnIiIiImqB\nWdVmdu7ciSeffBJr165FdHQ01q1bh02bNuHUqVMICAjAkiVL8OOPP+LAgQMAgGvXrmHw4MGIiYnB\nq6++ijNnziA+Ph5vvfUWkpKSADSWihw2bBjmzp2LZ599Funp6Xj++eexY8cO3HfffV37romIiIiI\nbJBZa95nz56NkpISJCcnQ61WIyIiAvv27RNqvGs0Gly8eFHo7+bmhrS0NMybNw+jRo2Ch4cHXnrp\nJSFxB4CQkBDs3bsXCxcuxPr16+Hv748PP/yQiTsRERERUSvMmnknIiIiIiLxtbvm3RqkpKQgNDQU\nSielf84AAAcASURBVKUSUVFRSE9PFzskEsny5csxevRouLu7w8fHB/fccw9OnjzZrN+bb76Jvn37\nwtnZGXfeeSdOnWp5oxuyD8uXL4dUKsWf//xnk3aOE9JoNJgzZw58fHygVCoRERGBI0eOmPThOLFv\ner0er732mpCHhIaG4rXXXoNerzfpx3FiX44cOYJ7770XAQEBkEql2LJlS7M+7Y2Juro6zJ8/H336\n9IGrqyvuvfde5Ofnt3tvq0/eU1NTkZSUhKVLlyI7Oxvjx4/HtGnTkJeXJ3ZoJILvv/8ezz//PH74\n4QccOnQIDg4OmDJlilC7GwDeffddrFq1CmvXrkVmZiZ8fHwQGxuL69evixg5ieXYsWP46KOPMHz4\ncJN2jhMqLy9HdHQ0JBIJ9u3bh9OnT+PDDz+Ej4+P0IfjhFasWIF169bhb3/7G86cOYM1a9YgJSUF\ny5cvF/pwnNifyspKDBs2DGvWrIGzs3Ozr5szJhYsWIDPP/8cqampSE9Px7Vr1zBz5ky0uyjGYOXG\njBljePbZZ03awsLCDH/5y19EioisSWVlpUEmkxm+/PJLoc3Pz8+wfPly4XV1dbWhV69ehr///e9i\nhEgiKisrM/Tv399w+PBhQ0xMjGH+/PnC1zhOaMmSJYYJEya02YfjhGbOnGmYM2eOSdtTTz1luPvu\nu4XXHCf2zdXV1fDJJ5+YtLU3JsrLyw0KhcKwfft2oc+VK1cMUqnUsH///jbvZ9Uz7/X19cjKymq2\nE2tcXBwyMlreVY7sy7Vr16DX6+Hh4QEAuHjxYrPde52cnHDHHXdwzNihP/3pT5g9ezYmTZpk0s5x\nQgCwZ88ejBkzBg8//DBUKhVGjhyJtWvXCl/nOCEAmDBhAg4dOoQzZxp3uj116hS+/fZbzJgxAwDH\nCTVnzpjIzMxEQ0ODSZ+AgACEh4e3O26seofVoqIi6HQ6qFQqk3aVSoWDBw+KFBVZkwULFuD222/H\nuHHjADSuX5VIJC2OmYKCAjFCJJF89NFHuHDhArZv397saxwnBAAXLlxASkoKFi5ciCVLliA7OxvP\nP/88JBIJnnvuOY4TAgAsXrwYFRUVGDJkCGQyGXQ6HV599VU8++yzAPh5Qs2ZMya0Wi1kMhm8vLya\n9dFoNG1e36qTd6K2LFq0CBkZGTh69CgkEonY4ZAVOXv2LF599VUcPXoUUqlV/4GRRKTX6zF69Ggk\nJycDAIYPH46zZ89i7dq1eO6550SOjqzFjh07sHXrVuzYsQNDhgxBdnY2/vznP6Nfv36Ij48XOzyy\nQ1b9U83b2xsymQxardakXavVwtfXV6SoyBosXLgQqampOHTokLBrLwD4+vrCYDBwzNi5H374AcXF\nxRgyZAjkcjnk8v/fzv20pLbFYRx/tNI0SKIClSAlSiSwgc6c1CuIBg0kCBpGUCTUoISyJr6AJBrU\noKBR4yYSFfYGJEKahlANGgTt6A+y7uBy98XTOcczOOeat+8H1mT7A9fgYfOge682nZ+fK5/Py+Vy\nqbu7m5xAgUBA0Wi05lo0GtXNzY0k7if42/LyspaWljQ5Oanh4WFNTU0pnU7bL6ySE3zrVzLh9/tV\nrVb18PDww5kf+dTlva2tTfF4XIVCoeZ6oVBQMpls0K7QaAsLC3ZxHxwcrPksHA7L7/fXZObl5UXF\nYpHMfCETExO6vLxUqVSyVyKRUCqVUqlU0tDQEDmBksmk/RzzP66vr+0fBLifQJKen58//IPndDrt\noyLJCb71K5mIx+NqbW2tmalUKiqXy3Vz07K+vr7+R3b+m3R2dmptbU2BQEBer1ebm5sqFova29uT\nz+dr9PbwH5ubm9P+/r6Ojo7U19cny7JkWZYcDodcLpckqVqtKpfLKRKJqFqtKp1O6/7+Xjs7O/YM\n/t/cbrd6e3tr1uHhoUKhkKanpyWRE0j9/f3a2NhQS0uLgsGgTk5OlMlktLKyokQiIYmcQCqXyzo4\nOFAkEpHL5dLp6alWV1eVSqXslw3JyddjWZbK5bLu7u60u7urWCwmn8+n9/d3+Xy+uplwu926vb1V\nPp9XLBbT4+OjZmdn1dXVpVwu9/PHgX/PITl/1vb2tgmHw6a9vd0kEglzcXHR6C2hQRwOh3E6nR9W\nNputmctmsyYYDBqPx2NGR0fN1dVVg3aMz2JsbKzmqEhjyAmMOT4+NiMjI8bj8ZhIJGK2trY+zJCT\nr+3p6cksLi6aUChkvF6vGRgYMJlMxry+vtbMkZOv5ezs7LudZGZmxp6pl4m3tzczPz9venp6TEdH\nhxkfHzeVSqXudzuMqXcSPAAAAIDP4FM/8w4AAADgX5R3AAAAoElQ3gEAAIAmQXkHAAAAmgTlHQAA\nAGgSlHcAAACgSVDeAQAAgCZBeQcAAACaBOUdAAAAaBJ/ARoxJsge8C+LAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "for data in dataset:\n", + " suite.Update(data)\n", + " \n", + "ps = thinkbayes.MakeUniformPmf(0, 100, 101).Probs(xs)\n", + "plt.plot(xs, ps, label='uniform', color='k')\n", + "\n", + "ps = suite.Probs(xs)\n", + "plt.plot(xs, ps, label='triangle');\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This is an example of **swamping the priors** with enough\n", + "data, people who start with different priors will tend to converge on\n", + "the same posterior." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Optimization\n", + "\n", + "The code I have shown so far is meant to be easy to read, but it is not\n", + "very efficient. In general, I like to develop code that is demonstrably\n", + "correct, then check whether it is fast enough for my purposes. If so,\n", + "there is no need to optimize. For this example, if we care about run\n", + "time, there are several ways we can speed it up.\n", + "\n", + "The first opportunity is to reduce the number of times we normalize the\n", + "suite. In the original code, we call `Update` once for each spin.\n", + "\n", + "```python\n", + "dataset = 'H' * heads + 'T' * tails\n", + "\n", + "for data in dataset:\n", + " suite.Update(data)\n", + "```\n", + "\n", + "And here’s what `Update` looks like:\n", + "\n", + "```python\n", + "def Update(self, data):\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " return self.Normalize()\n", + "```\n", + "\n", + "Each update iterates through the hypotheses, then calls `Normalize`,\n", + "which iterates through the hypotheses again. We can save some time by\n", + "doing all of the updates before normalizing.\n", + "\n", + "`Suite` provides a method called `UpdateSet` that does exactly that.\n", + "Here it is:\n", + "\n", + "```python\n", + "def UpdateSet(self, dataset):\n", + " for data in dataset:\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " return self.Normalize()\n", + "```\n", + "\n", + "And here’s how we can invoke it:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "2.38031546401283e-75" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "heads, tails = 140, 110\n", + "\n", + "dataset = 'H' * heads + 'T' * tails\n", + "suite.UpdateSet(dataset)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This optimization speeds things up, but the run time is still\n", + "proportional to the amount of data. We can speed things up even more by\n", + "rewriting `Likelihood` to process the entire dataset, rather than one\n", + "spin at a time.\n", + "\n", + "In the original version, `data` is a string that encodes either heads or\n", + "tails:\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " x = hypo / 100.0\n", + " if data == 'H':\n", + " return x\n", + " else:\n", + " return 1-x\n", + "```\n", + "\n", + "As an alternative, we could encode the dataset as a tuple of two\n", + "integers: the number of heads and tails. In that case `Likelihood` looks\n", + "like this:\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " x = hypo / 100.0\n", + " heads, tails = data\n", + " like = x**heads * (1-x)**tails\n", + " return like\n", + "```\n", + "\n", + "And then we can call `Update` like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.4413448096175153" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "heads, tails = 140, 110\n", + "suite.Update((heads, tails))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since we have replaced repeated multiplication with exponentiation, this\n", + "version takes the same time for any number of spins." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The beta distribution\n", + "\n", + "There is one more optimization that solves this problem even faster.\n", + "\n", + "So far we have used a Pmf object to represent a discrete set of values\n", + "for $x$. Now we will use a continuous distribution,\n", + "specifically the beta distribution (see\n", + ").\n", + "\n", + "The beta distribution is defined on the interval from 0 to 1 (including\n", + "both), so it is a natural choice for describing proportions and\n", + "probabilities. But wait, it gets better.\n", + "\n", + "It turns out that if you do a Bayesian update with a binomial likelihood\n", + "function, as we did in the previous section, the beta distribution is a\n", + "**conjugate prior**. That means that if the prior\n", + "distribution for x is a beta distribution, the posterior is\n", + "also a beta distribution. But wait, it gets even better.\n", + "\n", + "The shape of the beta distribution depends on two parameters, written\n", + "$\\alpha$ and $\\beta$, or alpha and beta. If\n", + "the prior is a beta distribution with parameters alpha and\n", + "beta, and we see data with h heads and\n", + "t tails, the posterior is a beta distribution with\n", + "parameters alpha+h and beta+t. In other words,\n", + "we can do an update with two additions.\n", + "\n", + "So that’s great, but it only works if we can find a beta distribution\n", + "that is a good choice for a prior. Fortunately, for many realistic\n", + "priors there is a beta distribution that is at least a good\n", + "approximation, and for a uniform prior there is a perfect match. The\n", + "beta distribution with alpha=1 and beta=1 is\n", + "uniform from 0 to 1.\n", + "\n", + "Let’s see how we can take advantage of all this.\n", + "thinkbayes.py provides a class that represents a beta\n", + "distribution:\n", + "\n", + "```python\n", + "class Beta(object):\n", + "\n", + " def __init__(self, alpha=1, beta=1):\n", + " self.alpha = alpha\n", + " self.beta = beta\n", + "```\n", + "\n", + "By default `__init__` makes a uniform distribution. Update\n", + "performs a Bayesian update:\n", + "\n", + "```python\n", + "def Update(self, data):\n", + " heads, tails = data\n", + " self.alpha += heads\n", + " self.beta += tails\n", + "```\n", + "\n", + "`data` is a pair of integers representing the number of heads\n", + "and tails.\n", + "\n", + "So we have yet another way to solve the Euro problem:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.5595238095238095\n" + ] + } + ], + "source": [ + "beta = thinkbayes.Beta()\n", + "beta.Update((140, 110))\n", + "print(beta.Mean())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Beta` provides `Mean,` which computes a simple\n", + "function of `alpha` and `beta`:\n", + "\n", + "```python\n", + "def Mean(self):\n", + " return float(self.alpha) / (self.alpha + self.beta)\n", + "```\n", + "\n", + "For the Euro problem the posterior mean is 56%, which is the same result\n", + "we got using Pmfs.\n", + "\n", + "`Beta` also provides `EvalPdf`, which evaluates\n", + "the probability density function (PDF) of the beta distribution:\n", + "\n", + "```python\n", + "def EvalPdf(self, x):\n", + " return x**(self.alpha-1) * (1-x)**(self.beta-1)\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "111" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "beta.beta" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "3.192662744667652e-75" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "beta.EvalPdf(.55)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, `Beta` provides `MakePmf`, which uses\n", + "`EvalPdf` to generate a discrete approximation of the beta\n", + "distribution." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu0AAAEWCAYAAADSA6kgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4VNW9x//PZHIPBBJC7iEQDIjghSZYIXKkyqWgp9V6\n4FjrBainza+WEvFYpeXYp6Wc6mmPeCkBf48cULGKv3qrFixREQkUK9GgQECK4ZqZgRASAgm5zOzf\nH5hhBkIySSazJ8n79Tw8zt6svfd309Xkk5W117YYhmEIAAAAQNAKMbsAAAAAAG0jtAMAAABBjtAO\nAAAABDlCOwAAABDkCO0AAABAkCO0AwAAAEGO0A4AAAAEOZ9De2FhobKyshQVFaXc3FwVFxdfsm1D\nQ4PmzJmjq6++WuHh4brxxhvbPHdxcbHCwsJ01VVX+V45AAAA0Ef4FNrXrl2rgoICLVq0SKWlpZow\nYYKmT5+uI0eOtNre6XQqKipK8+bN0y233NLmuaurq3Xvvfdq8uTJHa8eAAAA6AMsvrwR9brrrtM1\n11yjFStWuPeNGDFCM2fO1JIlS9o8dt68edq1a5c++OCDVv/+9ttv1zXXXCOXy6XXXntNn3/+eQdv\nAQAAAOjd2h1pb2pqUklJiaZMmeK1f+rUqdq6dWuXLl5YWKhjx45p0aJFXToPAAAA0Ju1G9orKyvl\ndDqVlJTktT8pKUl2u73TF/7iiy+0ePFivfTSS7JYLJ0+DwAAANDbhZpx0cbGRt1xxx36wx/+oCFD\nhkiS2pqlU1NTE6jSAAAAAL8bMGBAl45vN7QnJCTIarXK4XB47Xc4HEpOTu7URW02m8rKyjRnzhzN\nnj1bkuRyuWQYhsLDw7Vu3ToeTAUAAAC+1u70mLCwMOXk5KioqMhrf1FRkfLy8jp10bS0NO3cuVOl\npaXasWOHduzYofz8fGVnZ2vHjh2aMGFCp84LAAAA9EY+TY9ZsGCB7rnnHo0bN055eXlavny5bDab\n8vPzJUkLFy7UJ598ovfee899TFlZmRoaGlRZWanTp09rx44dkqSrr75aoaGhuuKKK7yukZiYqIiI\nCI0aNarNWrr6qwX0Xtu3b5ck5ebmmlwJghn9BL6gn8AX9BO0x59TvH0K7bNmzVJVVZWWLFkim82m\nMWPGaP369UpPT5ck2e12lZeXex0zY8YMHTp0yL09duxYWSwWOZ1OvxUPAAAA9AU+rdNuNs+fUhhp\nx6Uw4gFf0E/gC/oJfEE/QXv8mWF9eiMqAAAAAPMQ2gEAAIAgR2gHAAAAghyhHQAAAAhyhHYAAAAg\nyBHaAQAAgCBHaAcAAACCHKEdAAAACHKEdgAAACDIEdoBAACAIEdoBwAAAIIcoR0AAAAIcoR2AAAA\nIMgR2gEAAIAgR2gHAAS9k7WVOnxsv9llAIBpQs0uAACAthw9fkBP/fkXOttYp+nXfV/Tv/nvZpcE\nAAHHSDsAIGi5XE69/N4fdbaxTpL0t4/X6vCxr0yuCgACj9AOAAhaH5a+o0PH/unedhkuvfL+Mjld\nThOrAoDAI7QDAIJSZY1df/37SxftP3xsvzaVvmNCRQBgHkI7ACDoGIahtR8sV1NzoyQpdVCmvu0x\nl33d3/+kEzUOs8oDgIAjtAMAgs4/yjZq76EdkiSLJUTfn3y/po2bqdRBmZKkxuYGrf1guQzDMLNM\nAAgYQjsAIKicOlOtNz76P/f2DdfcoszkEbJaQ3XH5PtlkUWStOdQqbbv3WRWmQAQUIR2AEBQef2j\nlaprOC1Jio9N1M3j73T/3dDkEfqXa24+33bTStXW1QS8RgAINJ9De2FhobKyshQVFaXc3FwVFxdf\nsm1DQ4PmzJmjq6++WuHh4brxxhsvavPGG29o2rRpSkxMVGxsrK677jq9/fbbnbsLAECvsPOrT/Tp\nl5vd2/9+4/+jiLBIrzY3j/+B4voPliSdOVurNzb/nwCgt/MptK9du1YFBQVatGiRSktLNWHCBE2f\nPl1Hjhxptb3T6VRUVJTmzZunW265pdU2mzZt0k033aR169aptLRUM2bM0G233aYtW7Z0/m4AAD1W\nfUOdXt24wr197ahvaVTm2IvaRYZHada3fuze3r5nk8oOfhaQGgHALD6F9qVLl2ru3LmaO3euRo4c\nqaefflopKSlavnx5q+2jo6NVWFio++67T2lpaa22efLJJ/Xzn/9cubm5ysrK0qOPPqqcnBy9+eab\nnb8bAECP9c7WNao+fUKS1C9qgG6bOOeSbUcPy9U3Rkx0b6/9YLkams52e40AYJZ2Q3tTU5NKSko0\nZcoUr/1Tp07V1q1b/VpMbW2t4uLi/HpOAEDwO3HKoeLP17u3b7/hh4qJim3zmNtv+KGiI/tLkqpO\nHdNHO9Z1a40AYKZ2Q3tlZaWcTqeSkpK89iclJclut/utkGXLluno0aO6++67/XZOAEDP8OWhz2Xo\n3PKNw9NGe42iX0r/6IFeD6nuZYoMgF4s1OwCJOm1117Tww8/rFdffVUZGRlttt2+fXuAqkJPRR+B\nL+gnweUf+84/fDogNEklJSU+HedqiHB//qpij/7xj48VEmL1W130E/iCfoJLyc7O9tu52h1pT0hI\nkNVqlcPh/eY5h8Oh5OTkLhfw5z//Wffcc49efPFFzZgxo8vnAwD0PI5Th9yfE2PbHrzxFBMRq5iI\nAZKkZleTqs7wllQAvVO7I+1hYWHKyclRUVGRbr/9dvf+oqIizZw5s0sXf/XVVzVnzhy98MILuu22\n23w6Jjc3t0vXRO/VMtJBH0Fb6CfBp+Z0lU5vqZYkhVnDNfWGWxRqDfP5+N2V17hfshQxQMr9Rtf/\nt6WfwBf0E7SnpsZ/75HwafWYBQsWaPXq1Vq5cqX27Nmj+fPny2azKT8/X5K0cOFCTZ482euYsrIy\nlZaWqrKyUqdPn9aOHTu0Y8cO99+/8soruuuuu/TYY4/p+uuvl8PhkMPh0MmTJ/12cwCA4Le/Yrf7\nc2ZydocCuyRlpY5yf/7K41wA0Jv4NKd91qxZqqqq0pIlS2Sz2TRmzBitX79e6enpkiS73a7y8nKv\nY2bMmKFDh87/unPs2LGyWCxyOp2SpGeffVZOp1MFBQUqKChwt7vhhhv0wQcfdPnGAAA9g2fQzkq9\nosPHe4f2MhmGIYvF4pfaACBY+Pwgan5+vntk/UKrVq26aN+FIf5CGzdu9PXSAIBebP/R86F9eFrH\nQ3vyoAxFRcSovuGMautrdLzapsS4VH+WCACm82l6DAAA3aGu4bQqKg9KkiyWEA1NHtnhc4RYQjQs\n5XL3drmtzG/1AUCwILQDAExzwLbXvT572uChioqI7tR5PKfI7K8gtAPofQjtAADTeE2N6cR89vPH\nes9rB4DehtAOADDN/i4+hNpiSFK2rNZzj2kdO3lUtXX+W2YNAIIBoR0AYIqm5kYddOxzb3dlpD0s\nNFxDEi9zbzOvHUBvQ2gHAJjikOOfcjqbJUmDB6YqNmZgl8534dKPANCbENoBAKbwnBrjOSe9s3gY\nFUBvRmgHAJjiq6P+mc/uPofHso+Hj+1XY1NDl88JAMGC0A4ACDiXy6ly2x73dmdeqnShmKhYJcdn\nuM9/0PFll88JAMGC0A4ACDjbiUOqb6yTJMVGxylhQLJfzsu8dgC9FaEdABBwXks9po2SxWLxy3mZ\n1w6gtyK0AwACzl8vVbqQ57kO2PbK5XL67dwAYCZCOwAgoAzD8Jq64o+HUFvExyYqNiZOknS2sU62\nE4f8dm4AMBOhHQAQUCdOOVRzpkqSFBkerbSETL+d22KxMEUGQK9EaAcABJTn1JhhKZcrJMTq1/N7\nTpHhYVQAvQWhHQAQUP5+qdKFvEfad8swDL9fAwACjdAOAAgoz9Fvf6zPfqHUhKGKCIuUJNWcPqGT\ntcf9fg0ACDRCOwAgYGrrqnXs5FFJktUaqiFJ2X6/hjXEqqEpI93bzGsH0BsQ2gEAAeM5yp6ZmK2w\n0PBuuU4W89oB9DKEdgBAwHg+hJrVDVNjWgz3ejPq7jZaAkDPQGgHAASM13z2bngItUVm8giFWM59\ni7OdOKS6s6e77VoAEAiEdgBAQDQ0ndWR419JkiyyaFjq5d12rYiwSKUnDndvl9v2dNu1ACAQCO0A\ngICwnTgkl+GSJCXGpyk6ol+3Xi8r5fwPBQft+7r1WgDQ3XwO7YWFhcrKylJUVJRyc3NVXFx8ybYN\nDQ2aM2eOrr76aoWHh+vGG29std2mTZuUm5urqKgoXXbZZXr22Wc7fgcAgB6hovKg+3NawtBuv17a\n4GHuz7aqQ91+PQDoTj6F9rVr16qgoECLFi1SaWmpJkyYoOnTp+vIkSOttnc6nYqKitK8efN0yy23\ntNrmwIEDuvnmm3X99dertLRUjzzyiObNm6c33nij83cDAAhathPnQ3vKoMxuv17KoCEe1ya0A+jZ\nfArtS5cu1dy5czV37lyNHDlSTz/9tFJSUrR8+fJW20dHR6uwsFD33Xef0tLSWm2zfPlypaWl6ckn\nn9TIkSN133336d5779Uf/vCHzt8NACBo2So9Q/uQNlr6R3J8hiyySJKOV9vU1NzY7dcEgO7Sbmhv\nampSSUmJpkyZ4rV/6tSp2rp1a6cvvG3bNk2dOtVr37Rp07R9+3Y5nc5OnxcAEJwqPEa7UxO6f6Q9\nPCxCgwYkSZIMwyXHydZ/OwwAPUG7ob2yslJOp1NJSUle+5OSkmS32zt9Ybvd3uo5m5ubVVlZ2enz\nAgCCT21dtU7X10iSwsMiFR+bGJDrek+RORyQawJAdwg1u4CO2r59u9klIMjRR+AL+klg2arL3Z9j\nI+L1acmngblwY5j742c7P5bldEyHDqefwBf0E1xKdna2387V7kh7QkKCrFarHA6H136Hw6Hk5ORO\nXzg5ObnVc4aGhiohIaHT5wUABJ+TdcfcnwdGB2aU/cJrVdcdD9h1AcDf2h1pDwsLU05OjoqKinT7\n7be79xcVFWnmzJmdvvD48eP15ptveu3bsGGDcnNzZbVaL3lcbm5up6+J3q1lpIM+grbQT8zx5Xvb\n3J+vvPwbyh0bmH//1MoEbf7y3Kpkdc4an/93p5/AF/QTtKempsZv5/Jp9ZgFCxZo9erVWrlypfbs\n2aP58+fLZrMpPz9fkrRw4UJNnjzZ65iysjKVlpaqsrJSp0+f1o4dO7Rjxw733+fn5+vo0aN64IEH\ntGfPHj333HN64YUX9NBDD/nt5gAAwcFz5ZjUACz32CIxLk0hIecGgqpOHVNDY33Arg0A/uTTnPZZ\ns2apqqpKS5Yskc1m05gxY7R+/Xqlp6dLOvdQaXl5udcxM2bM0KFD51cKGDt2rCwWi3tlmKFDh2rd\nunV64IEHtGLFCqWmpuqZZ57Rrbfe6q97AwAEAZfhkq3q/EOggVg5pkWoNUyJA1Nl//r69qrDykwe\nEbDrA4C/+Pwgan5+vntk/UKrVq26aN+FIb41EydO5OENAOjlqk4dU2PTWUlSv6gB6h89MKDXTxk0\nxB3aK04cIrQD6JF8mh4DAEBnVXhNjen+lypdiDejAugNCO0AgG7lGZRTAjg1xn1Njzn0thMH22gJ\nAMGL0A4A6FaeQTklgA+hnr9mhkctjLQD6JkI7QCAbuU1PcaEkfaEAckKtZ57ydKpMyd15mxtwGsA\ngK4itAMAuk1Tc5OOVVe4t5PjM9po3T1CQqxe17Uz2g6gByK0AwC6zbGTR+VynVvqd1BskiLDo0yp\nw/Nh1ApCO4AeiNAOAOg2FZ7z2U2YGuO+NivIAOjhCO0AgG7jGZDNWO6xBaEdQE9HaAcAdBtbpbkr\nx5y/9vnQbj9xSIZhmFYLAHQGoR0A0G28pseYONIe13+wIsIiJUlnztaqtq7atFoAoDMI7QCAblHf\nUKeTtcclSdaQUCXFpZlWi8ViueAlS0yRAdCzENoBAN3CMxgnxaXJag01sZoLV5DhzagAehZCOwCg\nW9iCZGpMi2TejAqgByO0AwC6hS1IlntskeoxPcZ+4rCJlQBAxxHaAQDdosJj5ZhUE1eOaeG97ONB\nVpAB0KMQ2gEAfmcYhtcUlJQE86fH9I8eqJjI/pKkhqaz7odkAaAnILQDAPzuVN1JnTlbK0mKCI9S\nfP9EkytqWUGGlywB6JkI7QAAv6uo9H4I1WKxmFjNecleK8gQ2gH0HIR2AIDfeY5ipwbByjEtLnwz\nKgD0FIR2AIDf2bxG2s1/CLVFKtNjAPRQhHYAgN9VnAjO0O45PcZedVgul9PEagDAd4R2AIBfuVxO\n2avOr4OeGgRrtLeIieyvATHxkqRmZ5Mqa+wmVwQAviG0AwD8qrLGoabmRklSbHSc+kXFmlyRN1aQ\nAdATEdoBAH7l9SbUIHoItUUyoR1AD+RzaC8sLFRWVpaioqKUm5ur4uLiNtvv3LlTkyZNUnR0tDIy\nMrR48eKL2vzpT3/S2LFjFRMTo5SUFN19991yOBwdvwsAQNCo8HqpUvBMjWnBSDuAnsin0L527VoV\nFBRo0aJFKi0t1YQJEzR9+nQdOXKk1fa1tbWaMmWKUlJSVFJSoqeeekq///3vtXTpUnebLVu26J57\n7tGcOXO0e/duvfXWWyorK9Ndd93lnzsDAJjCc+WY1CB6CLUFK8gA6Il8Cu1Lly7V3LlzNXfuXI0c\nOVJPP/20UlJStHz58lbbr1mzRvX19Xr++ec1atQofe9739PDDz+sJ554wt1m27ZtysjI0M9+9jNl\nZmbq2muv1U9/+lN9/PHH/rkzAIApPINwUE6Pic9wfz5WXaGm5iYTqwEA37Qb2puamlRSUqIpU6Z4\n7Z86daq2bt3a6jHbtm3TxIkTFR4e7t43bdo0VVRU6ODBcyMweXl5stlseueddyRJlZWVeuWVV3Tz\nzTd3+mYAAOZqam7U8eoKSZJFFiUPymjniMCLCI/SoNgkSedWujlefdTkigCgfaHtNaisrJTT6VRS\nUpLX/qSkJL3//vutHmO325WRkXFRe8MwZLfblZmZqeuuu04vv/yyfvCDH6i+vl7Nzc2aOnWqVq9e\n3WY927dvb69k9HH0EfiCftI9qs445DJckqSYyAH6YsdOkytqXZQ1VtK5Z6i2fPKhhg0e02o7+gl8\nQT/BpWRnZ/vtXKatHrN7927NmzdPv/rVr/Tpp5/qb3/7m2w2m370ox+ZVRIAoIuq6467Pw+MGmxi\nJW0bEH2+tuq6ShMrAQDftDvSnpCQIKvVetGqLg6HQ8nJya0ek5yc3Gp7i8XiPuaxxx7TN7/5TS1Y\nsECSNGbMGBUWFmrixIn63e9+p9TU1FbPnZub2/5doU9qGemgj6At9JPuZd+6x/358uFXBe2/syum\nVruOnpviGRLRfFGd9BP4gn6C9tTU1PjtXO2OtIeFhSknJ0dFRUVe+4uKipSXl9fqMePHj9fmzZvV\n2Njo3rdhwwalpqYqM/PcSgJ1dXWyWq3exYSEyGKxyOVydfhGAADm834INfjms7fwfBjVXtX6SmgA\nEEx8mh6zYMECrV69WitXrtSePXs0f/582Ww25efnS5IWLlyoyZMnu9vfeeedio6O1uzZs7Vr1y69\n/vrrevzxx/Xggw+62/zrv/6r3nrrLa1YsULl5eXasmWL5s+fr5ycHKWnp/v5NgEAgWA/cdj9OTk+\n+FaOaZEUf/77zPHqCjU7WUEGQHBrd3qMJM2aNUtVVVVasmSJbDabxowZo/Xr17vDtd1uV3l5ubt9\nbGysioqKdP/992vcuHGKi4vTQw89pIKCAnebe++9V6dPn9ayZcv0n//5nxo4cKBuvPFGPfbYY36+\nRQBAIDQ2N6iyxi5JslhClBSfZnJFlxYRFqn42ERVnToml+HSsZMVSg3CF0EBQAufQrsk5efnu0fW\nL7Rq1aqL9o0ePVoffvhhm+e8//77df/99/taAgAgiDmqjsqQIUlKGJCs8NAIkytqW0r8EFWdOiZJ\nslcdJrQDCGqmrR4DAOhdbCfOvwk1mOezt0gedH6KjL3qcBstAcB8hHYAgF/0lPnsLbwfRiW0Awhu\nhHYAgF/YqnrGyjEtvEL7CUI7gOBGaAcA+EVPG2lP8gjtx6or5HQ2m1gNALSN0A4A6LKGprM6cerc\nS/VCLCFKjAvelWNaRIZHKa7/uTejulxOHa+xmVwRAFwaoR0A0GUOjxcUJQxMUVhomInV+I4pMgB6\nCkI7AKDLvN6EGh/889lbJHu8ZMnGw6gAghihHQDQZXaPh1CTBwX/fPYWnrU6CO0AghihHQDQZTaP\nqSUpPSm0Mz0GQA9BaAcAdJndc3pMjwrt56fHOKqPyulymlgNAFwaoR0A0CVnG+tVVXtckhQSYtXg\ngSkmV+S7qIgYDeg3SJLkdDarssZuckUA0DpCOwCgSzzfJpo4MFWh1p6xckyLFKbIAOgBCO0AgC7x\nXDkmuQe8CfVCXvPaeRgVQJAitAMAusRrPnsPeBPqhTx/0CC0AwhWhHYAQJd4rm/ek5Z7bOG9gsyh\nNloCgHkI7QCALvFeOaZnT49xnDwqFyvIAAhChHYAQKfVN5xR9ekTkiRrSKgGD+g5K8e0iI7sp9iY\nOElSs7NJJ04dM7kiALgYoR0A0GleK8fEpcpqDTWxms7jYVQAwY7QDgDoNO83oWaaWEnXeIZ2G/Pa\nAQQhQjsAoNNsJw66P/fE+ewtPN/iykg7gGBEaAcAdJrny4iSe+Byjy2S49PdnwntAIIRoR0A0Gm2\nqp69ckwLrxVkqo7IMAwTqwGAixHaAQCdUnf2tE6dOSlJCrWGKWFAsskVdV5MVKz6Rw2QJDU1N+p0\nQ7XJFQGAN59De2FhobKyshQVFaXc3FwVFxe32X7nzp2aNGmSoqOjlZGRocWLF1/UpqmpSY8++qiy\nsrIUGRmpoUOH6o9//GPH7wIAEHCeD2wmxaUpJMRqYjVd5/liqJq6ShMrAYCL+bQ219q1a1VQUKAV\nK1YoLy9Py5Yt0/Tp01VWVqb09PSL2tfW1mrKlCmaNGmSSkpKVFZWptmzZ6tfv3564IEH3O3+/d//\nXRUVFXruued02WWXyeFwqL6+3n93BwDoNvYe/ibUCyXHZ2jfkS8kSdV1x5Uen21yRQBwnk+hfenS\npZo7d67mzp0rSXr66af17rvvavny5VqyZMlF7desWaP6+no9//zzCg8P16hRo1RWVqYnnnjCHdo3\nbNigjRs3av/+/YqPj5ckDRnS87/oA0Bf4TnSnhLfc+ezt/B8GLW67riJlQDAxdqdHtPU1KSSkhJN\nmTLFa//UqVO1devWVo/Ztm2bJk6cqPDwcPe+adOmqaKiQgcPnlse7K233tK4ceP0v//7v8rIyNCI\nESM0f/58nTlzpiv3AwAIELtHaO8VI+0eD9LW1DM9BkBwaXekvbKyUk6nU0lJSV77k5KS9P7777d6\njN1uV0ZGxkXtDcOQ3W5XZmamvvrqK23evFkRERF6/fXXVV1drZ/+9Key2Wx69dVXL1nP9u3bfbkv\n9GH0EfiCftJ1hxxfuT9XVtRo+8me/W96tun8oFFNXaUMw6CfwCf0E1xKdrb/ptmZ9r5pl8ulkJAQ\nvfzyy+rXr58k6Y9//KO+/e1v6/jx4xo8eLBZpQEA2nG2qc4dcq0hoeoXOdDkirouMixGEaHRamiu\nU7OrSWcaanrFfQHoHdoN7QkJCbJarXI4HF77HQ6HkpNbX94rOTm51fYWi8V9TEpKitLS0tyBXZJG\njRolwzB06NChS4b23Nzc9kpGH9Uy0kEfQVvoJ/6x78hO6R/nPqckDNG14641tyA/2XJgmPYf3SVJ\nqq6r1KTrJ5tcEYIZX0/QnpqaGr+dq9057WFhYcrJyVFRUZHX/qKiIuXl5bV6zPjx47V582Y1Nja6\n923YsEGpqanKzMyUJOXl5amiokJ1dXXuNnv37pXFYnG3AQAEJ7vXQ6g9fz57C8+XLNXU8zAqgODh\n0zrtCxYs0OrVq7Vy5Urt2bNH8+fPl81mU35+viRp4cKFmjz5/GjEnXfeqejoaM2ePVu7du3S66+/\nrscff1wPPvigV5tBgwZpzpw52r17t7Zs2aKCggLNnDlTCQkJfr5NAIA/2XrZco8tPN/qWs1a7QCC\niE9z2mfNmqWqqiotWbJENptNY8aM0fr1691rtNvtdpWXl7vbx8bGqqioSPfff7/GjRunuLg4PfTQ\nQyooKHC3iYmJ0Xvvvad58+bp2muvVVxcnG677Tb97ne/8/MtAgD8zd7Llnts4TXSTmgHEER8fhA1\nPz/fPbJ+oVWrVl20b/To0frwww/bPGd2drbeffddX0sAAAQBwzBU4Rnae9FI+4XTYwzDkMViMbEi\nADjHp+kxAAC0OFl7XHVnayVJUeHRio9NNLki/+kfPVDREecWSGhyNqr6NKPtAIIDoR0A0CGHj51f\nnz09cXivGom2WCxeL1mqqDxoYjUAcB6hHQDQIUeO73d/zkjMMrGS7pGWMMz9+cjx8jZaAkDgENoB\nAB3iNdI+uPeF9vTBnqH9qzZaAkDgENoBAB1yxCO0ZyQON7GS7pHucU+EdgDBgtAOAPBZzZkqnao7\nKUkKD4vU4IEpJlfkfymDMhRiOfft8USNQ3UNp02uCAAI7QCADvAcZU9PGKaQEKuJ1XSPUGuYBkQP\ndm8fZV47gCBAaAcA+Mxzukh6L3wItUV8TLL785FjhHYA5iO0AwB8dthrPnvvDe2D+nmEdua1AwgC\nhHYAgM+OHDu/3GP64N73EGoLr5F2QjuAIEBoBwD45Ez9KVXVHpd0bt53cny6yRV1n7iYJPdnR9UR\nNTY3mFgNABDaAQA+8nzRUOqgTFmtoSZW073CrOGKjYyXJLkMl2y8GRWAyQjtAACfHPacGtOL57O3\niPea187DqADMRWgHAPjEc253b3yp0oU857V7/sACAGYgtAMAfOK5ckz6YEbaASCQCO0AgHbVN9Tp\neHWFJCnEEqLUhEyTK+p+niPttsqDcrqcJlYDoK8jtAMA2nW08vxIc/KgIQoLDTexmsCIDItWXL8E\nSVKTs1GOqiMmVwSgLyO0AwDa5TmnO6MPTI1pkebxwC3rtQMwE6EdANCuI57z2fvAyjEt0gcPc3/2\n/DcAgED6vEovAAAcUklEQVQjtAMA2tXXVo5p4fnALSPtAMxEaAcAtKmxqUH2r+dzW2RRWsJQcwsK\noAyP3yocPV4uwzBMrAZAX0ZoBwC0qeLEQRmGS5KUGJemiPAokysKnIH9EhQT2V+SVN9YpxOnHCZX\nBKCvIrQDANrU196E6slisXhNkTnMvHYAJvE5tBcWFiorK0tRUVHKzc1VcXFxm+137typSZMmKTo6\nWhkZGVq8ePEl2xYXFyssLExXXXWV75UDAALC8wHMjD4W2iUpPfH8w6hHmdcOwCQ+hfa1a9eqoKBA\nixYtUmlpqSZMmKDp06fryJHW16ytra3VlClTlJKSopKSEj311FP6/e9/r6VLl17Utrq6Wvfee68m\nT57ctTsBAHSLw8c9RtoH952HUFt43jMryAAwi0+hfenSpZo7d67mzp2rkSNH6umnn1ZKSoqWL1/e\navs1a9aovr5ezz//vEaNGqXvfe97evjhh/XEE09c1PaHP/yhZs+ereuuu65rdwIA8LtmZ5NslYfc\n256jzn1Futda7eVttASA7tNuaG9qalJJSYmmTJnitX/q1KnaunVrq8ds27ZNEydOVHj4+TfmTZs2\nTRUVFTp48KB7X2FhoY4dO6ZFixZ1tn4AQDeynTgsp6tZkjQoNknREf1MrijwBg9MUXhYpCTpVN1J\n1ZypMrkiAH1Ru6G9srJSTqdTSUlJXvuTkpJkt9tbPcZut7fa3jAM9zFffPGFFi9erJdeekkWi6Wz\n9QMAutGRPvwQaosQS4jXMpdMkQFghlAzLtrY2Kg77rhDf/jDHzRkyBBJ8nnt2+3bt3dnaegF6CPw\nBf3EN5/u3+b+HNIY0ef+3VruN1znf8PwcWmx6k+YVRGCUV/7/wV8l52d7bdztRvaExISZLVa5XB4\nr03rcDiUnJzc6jHJycmttrdYLEpOTpbNZlNZWZnmzJmj2bNnS5JcLpcMw1B4eLjWrVvHg6kAEARO\nnDn/G9X4fikmVmKu+Jjz3++qzrT+W2YA6E7thvawsDDl5OSoqKhIt99+u3t/UVGRZs6c2eox48eP\n1yOPPKLGxkb3vPYNGzYoNTVVmZmZam5u1s6dO72OWbZsmd577z29+eabyszMvGQ9ubm5Pt0Y+p6W\nkQ76CNpCP/Gd0+XUyx//j3v7WxOmKTZmoIkVBc6F/STpWLz+/s93JElnmk7SfyCJrydoX01Njd/O\n5dPqMQsWLNDq1au1cuVK7dmzR/Pnz5fNZlN+fr4kaeHChV4j43feeaeio6M1e/Zs7dq1S6+//roe\nf/xxPfjgg5Kk0NBQXXHFFV5/EhMTFRERoVGjRik6OtpvNwgA6JxjJ4+qqblRkjSg36A+E9hbkzIo\nQ9aQc+NcJ045VNdw2uSKAPQ1Ps1pnzVrlqqqqrRkyRLZbDaNGTNG69evV3p6uqRzD56Wl59fBis2\nNlZFRUW6//77NW7cOMXFxemhhx5SQUFB99wFAMDvPN+EmjG4bz6E2iLUGqaUQUN05OuXKx09Xq7s\n9CtNrgpAX+Lzg6j5+fnukfULrVq16qJ9o0eP1ocffuhzIb/61a/0q1/9yuf2AIDu5bkmeV9dOcZT\n+uBh7tB+5BihHUBg+TQ9BgDQ9xyy73N/zkjse29CvZDnDy6eb4kFgEAgtAMALtLQWK8Dji/d20OT\nR5hYTXBI95gidJQ3owIIMEI7AOAi+yt2y+VySpJSB2Wqf3TffQi1RVrCUFl07mWA9qojamxqMLki\nAH0JoR0AcJEvD3/u/jxiyNUmVhI8IsKjlBifJkkyDJcO2L9s5wgA8B9COwDgIns9QvvIjKtMrCS4\nZKeNcX/2/MEGALoboR0A4KW2rsY9ZzskxKrhaaNNrih4jPD4AYbQDiCQCO0AAC/7jnzh/pyZlK3I\n8CgTqwku2RlXuue1H3LsU33DGZMrAtBXENoBAF685rMzNcZLTGR/pSUOkyS5DJf+eXSXyRUB6CsI\n7QAAL4T2to3MOP9gLlNkAAQKoR0A4FZ16pgqa+ySpLDQcA1NHmlyRcGHee0AzEBoBwC4ea4aMzxt\ntMJCw0ysJjgNT71CVmuoJMl24pBOnTlpckUA+gJCOwDA7ctDO9yfWeqxdeFhERrm8RuIvYy2AwgA\nQjsAQJJkGIa+9Fg5hvnsl8YUGQCBRmgHAEg6N9Wjtq5akhQd2V9pg4eZXFHwGjnE+2FUwzBMrAZA\nX0BoBwBI8h4xzk4foxAL3yIuZUhStiK+Xr/+ZO1x98O7ANBd+IoMAJDEUo8dYQ2x6jKPN8Xu9XgW\nAAC6A6EdACCny6l9R3e6tz3XIkfrmNcOIJAI7QAAHXLsU0NjvSQprl+CBg9MMbmi4Of5g82+I1/I\nZbhMrAZAb0doBwBcNDXGYrGYWE3PkDJoiPpHD5QknTlbq6PHD5hbEIBejdAOAPBaa3zEEOaz+8Ji\nsWhE+pXu7S8PM68dQPchtANAH9fY1KBy2x739oh0QruvPOe185IlAN2J0A4AfdxXFWVyOpslSUnx\n6RrQL97kinoOz/Xavzq6W83OJhOrAdCbEdoBoI/znM8+kqUeOyQ+NlEJA5IlSY3NDTpg/9LkigD0\nVj6H9sLCQmVlZSkqKkq5ubkqLi5us/3OnTs1adIkRUdHKyMjQ4sXL/b6+zfeeEPTpk1TYmKiYmNj\ndd111+ntt9/u3F0AADptr8dc7BEs9dhhXlNkWK8dQDfxKbSvXbtWBQUFWrRokUpLSzVhwgRNnz5d\nR44cabV9bW2tpkyZopSUFJWUlOipp57S73//ey1dutTdZtOmTbrpppu0bt06lZaWasaMGbrtttu0\nZcsW/9wZAKBdZ87W6sixryRJFkuILksf3c4RuBDrtQMIhFBfGi1dulRz587V3LlzJUlPP/203n33\nXS1fvlxLliy5qP2aNWtUX1+v559/XuHh4Ro1apTKysr0xBNP6IEHHpAkPfnkk17HPProo/rrX/+q\nN998U3l5eV29LwCAD/55ZKcMGZKkIYnDFR3Rz+SKep5sjxVkDjr26WxjvSLDo0ysCEBv1O5Ie1NT\nk0pKSjRlyhSv/VOnTtXWrVtbPWbbtm2aOHGiwsPD3fumTZumiooKHTx48JLXqq2tVVxcnK+1AwC6\nqOzgZ+7PI5jP3in9owcobfAwSZLL5dT+o7tMrghAb9RuaK+srJTT6VRSUpLX/qSkJNnt9laPsdvt\nrbY3DOOSxyxbtkxHjx7V3Xff7WvtAIAuaGg6q0+/PP980qih3zCxmp5tJEs/AuhmPk2P6W6vvfaa\nHn74Yb366qvKyMhos+327dsDVBV6KvoIfEE/kfY5PtPZxjpJUmxkvE5W1Gm7jX8XT772E8vZ89Nh\nSvdu05BofmvRl/D1BJeSnZ3tt3O1O9KekJAgq9Uqh8Phtd/hcCg5ObnVY5KTk1ttb7FYLjrmz3/+\ns+655x69+OKLmjFjRkfrBwB00pf281NjspO/IYvFYmI1PVti7BCFWM59S62uO6bqukqTKwLQ27Q7\n0h4WFqacnBwVFRXp9ttvd+8vKirSzJkzWz1m/PjxeuSRR9TY2Oie175hwwalpqYqMzPT3e7VV1/V\nnDlz9MILL+i2227zqeDc3Fyf2qHvaRnpoI+gLfSTcw4f+0ontlRIkqzWUN0+9R71i4o1uarg0Zl+\nsvv4tdqxf5skqdp1WJNzv90ttSF48PUE7ampqfHbuXxa8nHBggVavXq1Vq5cqT179mj+/Pmy2WzK\nz8+XJC1cuFCTJ092t7/zzjsVHR2t2bNna9euXXr99df1+OOP68EHH3S3eeWVV3TXXXfpscce0/XX\nXy+HwyGHw6GTJ0/67eYAAK3b+sXf3J+vuWwCgd0Prr9quvvzP8o2qqHprInVAOhtfArts2bN0pNP\nPqklS5Zo7Nix2rp1q9avX6/09HRJ5x48LS8vd7ePjY1VUVGRKioqNG7cOM2bN08PPfSQCgoK3G2e\nffZZOZ1OFRQUKDU11f3HczQfAOB/ZxvrtX3vJvd23pXTTKym9xiRcZUS49IkSWcb61Sy9yOTKwLQ\nm/j8IGp+fr57ZP1Cq1atumjf6NGj9eGHH17yfBs3bvT10gAAPyrZ+5F7FDgpPl3DU68wuaLewWKx\nKO/KaXrjo/+TJG3+fL3Gj57CswIA/MKnkXYAQO+xZef5qTETxkwlVPrRN0fdqLDQc89yHT1ergP2\nvSZXBKC3ILQDQB9yyPFPHTn2lSQp1Bqma0d9y+SKepfoyH7KGfkv7u3Nn683sRoAvQmhHQD6kC0e\nD6COzc5TTGR/E6vpnSZ6PJD62b4tqq3z3+oRAPouQjsA9BH1DXUq+XKze5sHULtHRuJwZSaPkCQ5\nnc3atvt9kysC0BsQ2gGgj9i+d5Mav34ANWXQEA1Ludzkinovz9H2LV+8K5fLaWI1AHoDQjsA9AGG\nYXitzc4DqN1rbHaeor+eelR16pjKDn7WzhEA0DZCOwD0AQcd+3S08oAkKSw0XONGTTK1nt4uLDRc\n40ff5N4u/vxdE6sB0BsQ2gGgD9jiERq/kX29oiP6mVhN3zBhzPlnBnYfKNGJGoeJ1QDo6QjtANDL\n1TWc1qf7it3bE3gANSAGD0zRqMxvSJIMGV4r9wBARxHaAaCX27xjvZqaGyVJqQlDNfTrlU3Q/a6/\n6tvuz3/f/Z77fwcA6ChCOwD0YkePl+vdj9e6t6+/8ts8gBpAo4fmKK7/YEnSmfpTKv3nVpMrAtBT\nEdoBoJdqam7UC39bKqerWZI0JClb40dPNrmqviUkxOq1Hv4HJW+q2dlkYkUAeipCOwD0Uu9sXSPb\niUOSzq1mcve0AlmtoSZX1feMHz1ZodYwSdLRygP6y5YXTa4IQE9EaAeAXujLw59r42d/cW/fOnGO\nkuLSTKyo7+ofPVC3TLjLvf3hZ3/RF1/9w8SKAPREhHYA6GXqGk5rzYan3NtXZH5D11/57TaOQHf7\n1tjvaMywce7tlzY8rapTx02sCEBPQ2gHgF7m/9v4/6r69AlJUkxkf31/yk95+NRkFotFP5j6M8X1\nS5B07ger1e/+QU5ns8mVAegpCO0A0IuU7N2skr0fubfvuOl+DYiJN7EitIiJ7K97p/+nQiznvvUe\nsO3VO39/yeSqAPQUhHYA6CVO1lbq1Y0r3NvfvOImXX3ZdSZWhAtlpV7uNb/9/ZI3tKt8u4kVAegp\nCO0A0Au4DJdeKnpa9Q1nJEnxsYn63r/80OSq0Jobc27VFV+/KVWS1mx4SidrK02sCEBPQGgHgB7u\n1JmTWv7Gr/Xl4c8lSRZLiO6eWqCoiGiTK0NrQiwhumtagQb0GyRJOnO2Vs+/+79yupwmVwYgmBHa\nAaAHKzv4mR5/qUB7D+9w75ucc5uGp11hYlVoT7+oWM3+9gL3/PavKsr0wrtP6Ez9KZMrAxCsCO0A\n0AM1O5v0VvFqLX/z16qtr5EkWWTR1HEzdfP4O02uDr4YnjZaM677vnv7s31b9N8vztNn+7bKMAwT\nKwMQjHg1HgD0MJU1dj2//n910LHPvS82Ok53TyvQyCFXm1gZOmryuNtVWWPXtt3vS5Jq62u0at3/\n6Krh12nmt37Eyj8A3HweaS8sLFRWVpaioqKUm5ur4uLiNtvv3LlTkyZNUnR0tDIyMrR48eKL2mza\ntEm5ubmKiorSZZddpmeffbbjdwAAfURjU4O27Xpf//OnBV6BfVTmN/TwD5YS2HugEEuI7pwyT//x\nr7/wCuif79+m/35xnrbtep9RdwCSfBxpX7t2rQoKCrRixQrl5eVp2bJlmj59usrKypSenn5R+9ra\nWk2ZMkWTJk1SSUmJysrKNHv2bPXr108PPPCAJOnAgQO6+eabdd999+mll17S5s2b9ZOf/ESJiYm6\n7bbb/HuXANBDNTY3qOzAZ/ps3xbtLP9EjU1n3X8XEmLVv064W9/6xnfcc6PRM12Zda0uSxutt4pX\na+vOIklSfcMZ/em9Z1Sy9yNNuHKqRmZcrejIfiZXCsAsFsOHH+Gvu+46XXPNNVqx4vz6vyNGjNDM\nmTO1ZMmSi9ovX75cCxcu1LFjxxQeHi5JWrJkiVasWKHDhw9Lkh5++GG9+eab2rt3r/u4//iP/9Du\n3bu1ZcsWr/PV1NS4Pw8YMKCDt4i+Yvv2c2sd5+bmmlwJgllP6Cf1DWe078jOc0H9q3+owSOotxgU\nm6TZ0x9UZvIIEyrs/czsJ18e/lwvv79MJ2ocXvstlhBlJmXr8sxrNCpzrIYkZcsaYg14fTivJ3w9\ngbn8mWHbHWlvampSSUmJHnroIa/9U6dO1datW1s9Ztu2bZo4caI7sEvStGnT9Oijj+rgwYPKzMzU\ntm3bNHXqVK/jpk2bphdeeEFOp1NWK1+IAPQuTpdTDU31amis19nGs6o7e0qVNfZzf6rt7s9nztZe\n8hyJcWn6xojr9a2x31FUREwAq0egjMi4Sgt/8LT++veX9GHpOzIMlyTJMFw6YN+rA/a9evfjtYqK\niFFWyigN7J+gATFxGtBvkAbExGtgv3gNiIlXdGR/WSwWk+8GgL+0G9orKyvldDqVlJTktT8pKUnv\nv/9+q8fY7XZlZGRc1N4wDNntdmVmZsput2vKlCkXtWlublZlZeVF12vx7F9+217J6KNqqs/9NFtS\n8a7JlSCYVVdXS5JKjl7cTwxd8ItHj19EGi1/bxhf//fctvH1tuFyyelyyuVyyml8/V+XU05Xsxoa\nz6qhqV5NzY2dqnnwwFSNzc7T2Ow8pSZkEsT6gPCwCN32L3P1zStu1Gf7tmrPwc90yPFPrz5a33BG\nuw60/TbVUGuYwkLDFWYNV1houEJDz21bQ0JlsVgUYgmRxRLy9X/Pbevr/mVRy3910b5W9cFuyfcd\ntOeOG+b57Vw9bvUYf948APQ0p06xjncgZGdnS/L+1bYZYsLidP0VN+v6K242tQ4A5mv3yaWEhARZ\nrVY5HN5z6xwOh5KTk1s9Jjk5udX2FovFfcyl2oSGhiohIaFDNwEAAAD0Zu2G9rCwMOXk5KioqMhr\nf1FRkfLy8lo9Zvz48dq8ebMaG8//KnjDhg1KTU1VZmamu82F59ywYYNyc3OZzw4AAAB48Gn1mFdf\nfVX33HOPli1bpry8PC1fvlyrVq3S7t27lZ6eroULF+qTTz7Re++9J+ncr28vv/xyTZo0Sb/85S+1\nd+9ezZkzR7/+9a9VUFAg6dySj1deeaXuu+8+/fjHP1ZxcbF++tOf6pVXXtGtt97avXcNAAAA9CA+\nzWmfNWuWqqqqtGTJEtlsNo0ZM0br1693r9Fut9tVXl7ubh8bG6uioiLdf//9GjdunOLi4vTQQw+5\nA7skDR06VOvWrdMDDzygFStWKDU1Vc888wyBHQAAALiATyPtAAAAAMwTNK/QKywsVFZWlqKiopSb\nm6vi4uI22+/cuVOTJk1SdHS0MjIytHjx4gBVCjN1pJ9s2rRJt956q1JTUxUTE6Orr75aq1atCmC1\nMEtHv5602Ldvn/r376/Y2NhurhBm60wfefLJJzVq1ChFRkYqLS1Nv/jFLwJQKczU0X7yt7/9TRMm\nTFBsbKwGDx6sW2+9Vfv27QtQtTDD5s2b9d3vflfp6ekKCQnRCy+80O4xnc6wRhB45ZVXjLCwMGPl\nypXGnj17jHnz5hn9+vUzDh8+3Gr7U6dOGcnJycYdd9xh7N6923jttdeM/v37G0888USAK0cgdbSf\n/Pd//7fxX//1X8bWrVuN8vJyY/ny5UZoaKjx8ssvB7hyBFJH+0mLxsZGIycnx7jllluM/v37B6ha\nmKEzfeSBBx4wRo4cabz99ttGeXm5UVpaaqxfvz6AVSPQOtpPysvLjcjISOORRx4x9u/fb+zYscOY\nNm2akZ2dHeDKEUjr1q0zfvnLXxqvvfaaERMTYzz//PNttu9Khg2K0P7Nb37T+PGPf+y1Lzs72/jF\nL37RavvCwkJjwIABRkNDg3vfb3/7WyM9Pb1b64S5OtpPWjNr1izj3/7t3/xdGoJIZ/tJQUGBMXfu\nXGP16tWE9l6uo31kz549RlhYmLF3795AlIcg0dF+8uc//9kIDQ01XC6Xe9/GjRuNkJAQ48SJE91a\nK4JDv3792g3tXcmwpk+PaWpqUklJyUVvR506daq2bt3a6jHbtm3TxIkTFR4e7t43bdo0VVRU6ODB\ng91aL8zRmX7SmlOnTikuLs7f5SFIdLaf/PWvf9W6dev0zDPPdHeJMFln+shf/vIXDR8+XOvWrdPw\n4cM1bNgwzZ49W8ePHw9EyTBBZ/rJuHHjFBYWpueee04ul0u1tbVavXq1rr32WsXHxweibPQAXcmw\npof2yspKOZ1OJSUlee1PSkqS3W5v9Ri73d5qe8MwLnkMerbO9JMLvfPOO/rggw/04x//uDtKRBDo\nTD+pqKjQj370I7300kuKjo4ORJkwUWf6yFdffaUDBw5o7dq1euGFF7RmzRrt2bNH3/nOdwJRMkzQ\nmX4yZMgQbdiwQY8++qgiIiI0cOBA7dq1S2+//XYgSkYP0ZUMa3poBwJhy5Yt+sEPfqBnnnlGOTk5\nZpeDIHL33XfrJz/5iXJzcyVJBgtq4QIul0uNjY1as2aN8vLylJeXpxdffFEff/yxPvnkE7PLQ5Bw\nOBz64Q9/qHvvvVfbt2/Xpk2b1L9/f82cOdPs0tBLmB7aExISZLVa5XA4vPY7HA4lJye3ekxycnKr\n7S0WyyWPQc/WmX7Sori4WDNmzNBvf/tb/ehHP+rOMmGyzvSTjRs36te//rXCwsIUFham++67T6dP\nn1Z4eLiee+65QJSNAOpMH0lJSVFoaKiGDx/u3pednS2r1apDhw51a70wR2f6ybJly9SvXz899thj\nuvrqq3X99dfrxRdf1KZNmzo0jRO9W1cyrOmhPSwsTDk5OSoqKvLaX1RUpLy8vFaPGT9+vDZv3qzG\nxkb3vg0bNig1NVWZmZndWi/M0Zl+IkkfffSRZsyYod/85jeaN29ed5cJk3Wmn+zcuVOlpaXasWOH\nduzYod/85jeKjo7Wjh07GCHrhTrTR/Ly8tTc3Oz1EsH9+/fL6XTyPaeX6kw/qaurk9Vq9doXEnIu\nZrlcru4pFD1OlzJs156T9Y+1a9caERERxnPPPWeUlZUZP/vZz4z+/fu7l1V65JFHjJtuusndvqam\nxkhJSTG+//3vGzt37jRee+01IzY21li6dKlZt4AA6Gg/2bhxoxETE2P8/Oc/N+x2u/vP8ePHzboF\nBEBH+8mFWD2m9+toH3G5XEZubq4xadIk47PPPjM+/fRT44YbbjAmTJhg1i0gADraTz744APDarUa\nv/nNb4x9+/YZJSUlxrRp04zMzEyjrq7OrNtANzt9+rRRWlpqfPbZZ0Z0dLSxePFio7S01Dh06JBh\nGP7NsEER2g3DMJYvX24MGzbMiIyMNHJzc43i4mL3382ePdvIysryar9z507jhhtuMKKioozU1FRj\n8eLFgS4ZJuhIP5k9e7YREhJy0Z9hw4aZUToCqKNfTzwR2vuGjvYRu91uzJo1y4iNjTWSkpKMu+++\n2zh27Figy0aAdbSfrF271sjJyTH69+9vJCUlGd/97neNsrKyQJeNAPrwww8Ni8VyUdaYM2eOYRj+\nzbAWw+CpKwAAACCYmT6nHQAAAEDbCO0AAABAkCO0AwAAAEGO0A4AAAAEOUI7AAAAEOQI7QAAAECQ\nI7QDAAAAQY7QDgAAAAQ5QjsAAAAQ5P5/Sa9sV+qpDpwAAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "pmf = beta.MakePmf()\n", + "xs = [i / 100. for i in range(101)]\n", + "ps = pmf.Probs(xs)\n", + "plt.plot(xs, ps);" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "In this chapter we solved the same problem with two different priors and\n", + "found that with a large dataset, the priors get swamped. If two people\n", + "start with different prior beliefs, they generally find, as they see\n", + "more data, that their posterior distributions converge. At some point\n", + "the difference between their distribution is small enough that it has no\n", + "practical effect.\n", + "\n", + "When this happens, it relieves some of the worry about objectivity that\n", + "I discussed in the previous chapter. And for many real-world problems\n", + "even stark prior beliefs can eventually be reconciled by data.\n", + "\n", + "But that is not always the case. First, remember that all Bayesian\n", + "analysis is based on modeling decisions. If you and I do not choose the\n", + "same model, we might interpret data differently. So even with the same\n", + "data, we would compute different likelihoods, and our posterior beliefs\n", + "might not converge.\n", + "\n", + "Also, notice that in a Bayesian update, we multiply each prior\n", + "probability by a likelihood, so if $\\mathrm{p}(H)$ is 0,\n", + "$\\mathrm{p}(H|D)$ is also 0, regardless of $D$. In the Euro\n", + "problem, if you are convinced that $x$ is less than 50%, and you assign\n", + "probability 0 to all other hypotheses, no amount of data will convince\n", + "you otherwise.\n", + "\n", + "This observation is the basis of **Cromwell’s rule**, which\n", + "is the recommendation that you should avoid giving a prior probability\n", + "of 0 to any hypothesis that is even remotely possible (see\n", + ").\n", + "\n", + "Cromwell’s rule is named after Oliver Cromwell, who wrote, “I beseech\n", + "you, in the bowels of Christ, think it possible that you may be\n", + "mistaken.” For Bayesians, this turns out to be good advice (even if it’s\n", + "a little overwrought)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "Suppose that instead of observing coin tosses directly, you measure the\n", + "outcome using an instrument that is not always correct. Specifically,\n", + "suppose there is a probability y that an actual heads is\n", + "reported as tails, or actual tails reported as heads.\n", + "\n", + "Write a class that estimates the bias of a coin given a series of\n", + "outcomes and the value of y.\n", + "\n", + "How does the spread of the posterior distribution depend on\n", + "y?\n", + "\n", + "This exercise is inspired by a question posted by a “redditor” named\n", + "dominosci on Reddit’s statistics “subreddit” at\n", + ".\n", + "\n", + "Reddit is an online forum with many interest groups called subreddits.\n", + "Users, called redditors, post links to online content and other web\n", + "pages. Other redditors vote on the links, giving an “upvote” to\n", + "high-quality links and a “downvote” to links that are bad or irrelevant.\n", + "\n", + "A problem, identified by dominosci, is that some redditors are more\n", + "reliable than others, and Reddit does not take this into account.\n", + "\n", + "The challenge is to devise a system so that when a redditor casts a\n", + "vote, the estimated quality of the link is updated in accordance with\n", + "the reliability of the redditor, and the estimated reliability of the\n", + "redditor is updated in accordance with the quality of the link.\n", + "\n", + "One approach is to model the quality of the link as the probability of\n", + "garnering an upvote, and to model the reliability of the redditor as the\n", + "probability of correctly giving an upvote to a high-quality item.\n", + "\n", + "Write class definitions for redditors and links and an update function\n", + "that updates both objects whenever a redditor casts a vote." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/05_Odds_and_Addends.ipynb b/notebooks/05_Odds_and_Addends.ipynb index 68ef1df..98861ee 100644 --- a/notebooks/05_Odds_and_Addends.ipynb +++ b/notebooks/05_Odds_and_Addends.ipynb @@ -1,1028 +1,1277 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "%matplotlib inline\n", - "import sys\n", - "from __future__ import division, print_function\n", - "%load_ext autoreload\n", - "%autoreload 2 \n", - "import sys\n", - "sys.path.insert(0,'../code')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#Odds and Addends\n", - "\n", - "##Odds\n", - "\n", - "One way to represent a probability is with a number between 0 and 1, but\n", - "that’s not the only way. If you have ever bet on a football game or a\n", - "horse race, you have probably encountered another representation of\n", - "probability, called **odds**.\n", - "\n", - "You might have heard expressions like “the odds are three to one,” but\n", - "you might not know what that means. The **odds in favor**\n", - "of an event are the ratio of the probability it will occur to the\n", - "probability that it will not.\n", - "\n", - "So if I think my team has a 75% chance of winning, I would say that the\n", - "odds in their favor are three to one, because the chance of winning is\n", - "three times the chance of losing.\n", - "\n", - "You can write odds in decimal form, but it is most common to write them\n", - "as a ratio of integers. So “three to one” is written $3:1$.\n", - "\n", - "When probabilities are low, it is more common to report the **odds\n", - "against** rather than the odds in favor. For example, if I think\n", - "my horse has a 10% chance of winning, I would say that the odds against\n", - "are $9:1$.\n", - "\n", - "Probabilities and odds are different representations of the same\n", - "information. Given a probability, you can compute the odds like this:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.11111111111111112" - ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "def Odds(p):\n", - " return p / (1-p)\n", - "\n", - "Odds(0.1)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Given the odds in favor, in decimal form, you can convert to probability\n", - "like this:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.09999999999999999" - ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "def Probability(o):\n", - " return o / (o+1)\n", - "\n", - "Probability(1/9.)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you represent odds with a numerator and denominator, you can convert\n", - "to probability like this:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.1" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "def Probability2(yes, no):\n", - " return float(yes) / (yes + no)\n", - "\n", - "Probability2(1, 9)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When I work with odds in my head, I find it helpful to picture people at\n", - "the track. If 20% of them think my horse will win, then 80% of them\n", - "don’t, so the odds in favor are $20:80$ or $1:4$." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.2" - ] - }, - "execution_count": 5, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "Probability2(20, 80)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If the odds are $5:1$ against my horse, then five out of six people\n", - "think she will lose, so the probability of winning is $1/6$." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.16666666666666663" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "1 - Probability2(5, 1)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The odds form of Bayes’s theorem\n", - "\n", - "In Chapter [intro] I wrote Bayes’s theorem in the **probability\n", - "form**:\n", - "$${{\\mathrm{p}(H|D)}} = \\frac{{{\\mathrm{p}(H)}}~{{\\mathrm{p}(D|H)}}}{{{\\mathrm{p}(D)}}}$$\n", - "If we have two hypotheses, $A$ and $B$, we can write the ratio of\n", - "posterior probabilities like this:\n", - "$$\\frac{{{\\mathrm{p}(A|D)}}}{{{\\mathrm{p}(B|D)}}} = \\frac{{{\\mathrm{p}(A)}}~{{\\mathrm{p}(D|A)}}}\n", - " {{{\\mathrm{p}(B)}}~{{\\mathrm{p}(D|B)}}}$$\n", - "Notice that the normalizing constant, $\\mathrm{p}(D)$,\n", - "drops out of this equation.\n", - "\n", - "If $A$ and $B$ are mutually exclusive and collectively exhaustive, that\n", - "means ${{\\mathrm{p}(B)}} = 1 - {{\\mathrm{p}(A)}}$,\n", - "so we can rewrite the ratio of the priors, and the ratio of the\n", - "posteriors, as odds.\n", - "\n", - "Writing $\\mathrm{o}(A)$ for odds in favor of $A$, we get:\n", - "$${{\\mathrm{o}(A|D)}} = {{\\mathrm{o}(A)}}~\\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", - "In words, this says that the posterior odds are the prior odds times the\n", - "likelihood ratio. This is the **odds form** of Bayes’s\n", - "theorem.\n", - "\n", - "This form is most convenient for computing a Bayesian update on paper or\n", - "in your head. For example, let’s go back to the cookie problem:\n", - "\n", - "> Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla\n", - "> cookies and 10 chocolate cookies. Bowl 2 contains 20 of each.\n", - ">\n", - "> Now suppose you choose one of the bowls at random and, without\n", - "> looking, select a cookie at random. The cookie is vanilla. What is the\n", - "> probability that it came from Bowl 1?\n", - "\n", - "The prior probability is 50%, so the prior odds are $1:1$, or just 1.\n", - "The likelihood ratio is $\\frac{3}{4} / \\frac{1}{2}$, or $3/2$. So the\n", - "posterior odds are $3:2$, which corresponds to probability $3/5$." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Oliver’s blood\n", - "\n", - "Here is another problem from MacKay’s *Information Theory,\n", - "Inference, and Learning Algorithms*:\n", - "\n", - "> Two people have left traces of their own blood at the scene of a\n", - "> crime. A suspect, Oliver, is tested and found to have type ‘O’ blood.\n", - "> The blood groups of the two traces are found to be of type ‘O’ (a\n", - "> common type in the local population, having frequency 60%) and of type\n", - "> ‘AB’ (a rare type, with frequency 1%). Do these data [the traces found\n", - "> at the scene] give evidence in favor of the proposition that Oliver\n", - "> was one of the people [who left blood at the scene]?\n", - "\n", - "To answer this question, we need to think about what it means for data\n", - "to give evidence in favor of (or against) a hypothesis. Intuitively, we\n", - "might say that data favor a hypothesis if the hypothesis is more likely\n", - "in light of the data than it was before.\n", - "\n", - "In the cookie problem, the prior odds are $1:1$, or probability 50%. The\n", - "posterior odds are $3:2$, or probability 60%. So we could say that the\n", - "vanilla cookie is evidence in favor of Bowl 1.\n", - "\n", - "The odds form of Bayes’s theorem provides a way to make this intuition\n", - "more precise. Again\n", - "$${{\\mathrm{o}(A|D)}} = {{\\mathrm{o}(A)}}~\\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", - "Or dividing through by $\\mathrm{o}(A)$:\n", - "$$\\frac{{{\\mathrm{o}(A|D)}}}{{{\\mathrm{o}(A)}}} = \\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", - "The term on the left is the ratio of the posterior and prior odds. The\n", - "term on the right is the likelihood ratio, also called the **Bayes\n", - "factor**.\n", - "\n", - "If the Bayes factor value is greater than 1, that means that the data\n", - "were more likely under $A$ than under $B$. And since the odds ratio is\n", - "also greater than 1, that means that the odds are greater, in light of\n", - "the data, than they were before.\n", - "\n", - "If the Bayes factor is less than 1, that means the data were less likely\n", - "under $A$ than under $B$, so the odds in favor of $A$ go down.\n", - "\n", - "Finally, if the Bayes factor is exactly 1, the data are equally likely\n", - "under either hypothesis, so the odds do not change.\n", - "\n", - "Now we can get back to the Oliver’s blood problem. If Oliver is one of\n", - "the people who left blood at the crime scene, then he accounts for the\n", - "‘O’ sample, so the probability of the data is just the probability that\n", - "a random member of the population has type ‘AB’ blood, which is 1%.\n", - "\n", - "If Oliver did not leave blood at the scene, then we have two samples to\n", - "account for. If we choose two random people from the population, what is\n", - "the chance of finding one with type ‘O’ and one with type ‘AB’? Well,\n", - "there are two ways it might happen: the first person we choose might\n", - "have type ‘O’ and the second ‘AB’, or the other way around. So the total\n", - "probability is $2 (0.6) (0.01) = 1.2\\%$.\n", - "\n", - "The likelihood of the data is slightly higher if Oliver is\n", - "*not* one of the people who left blood at the scene, so the\n", - "blood data is actually evidence against Oliver’s guilt.\n", - "\n", - "This example is a little contrived, but it is an example of the\n", - "counterintuitive result that data *consistent* with a\n", - "hypothesis are not necessarily *in favor of* the\n", - "hypothesis.\n", - "\n", - "If this result is so counterintuitive that it bothers you, this way of\n", - "thinking might help: the data consist of a common event, type ‘O’ blood,\n", - "and a rare event, type ‘AB’ blood. If Oliver accounts for the common\n", - "event, that leaves the rare event still unexplained. If Oliver doesn’t\n", - "account for the ‘O’ blood, then we have two chances to find someone in\n", - "the population with ‘AB’ blood. And that factor of two makes the\n", - "difference." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Addends\n", - "\n", - "The fundamental operation of Bayesian statistics is Update,\n", - "which takes a prior distribution and a set of data, and produces a\n", - "posterior distribution. But solving real problems usually involves a\n", - "number of other operations, including scaling, addition and other\n", - "arithmetic operations, max and min, and mixtures.\n", - "\n", - "This chapter presents addition and max; I will present other operations\n", - "as we need them.\n", - "\n", - "The first example is based on *Dungeons & Dragons*, a\n", - "role-playing game where the results of players’ decisions are usually\n", - "determined by rolling dice. In fact, before game play starts, players\n", - "generate each attribute of their characters—strength, intelligence,\n", - "wisdom, dexterity, constitution, and charisma—by rolling three 6-sided\n", - "dice and adding them up.\n", - "\n", - "So you might be curious to know the distribution of this sum. There are\n", - "two ways you might compute it:\n", - "\n", - "Simulation:\n", - ": Given a Pmf that represents the distribution for a single die, you\n", - " can draw random samples, add them up, and accumulate the\n", - " distribution of simulated sums.\n", - "\n", - "Enumeration:\n", - ": Given two Pmfs, you can enumerate all possible pairs of values and\n", - " compute the distribution of the sums.\n", - "\n", - "`thinkbayes` provides functions for both. Here’s an example of the first\n", - "approach. First, I’ll define a class to represent a single die as a Pmf:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "import thinkbayes\n", - "\n", - "class Die(thinkbayes.Pmf):\n", - "\n", - " def __init__(self, sides):\n", - " thinkbayes.Pmf.__init__(self)\n", - " for x in range(1, sides+1):\n", - " self.Set(x, 1)\n", - " self.Normalize()\n", - " \n", - " # for Python 3 compliance\n", - " def __hash__(self):\n", - " return id(self)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now I can create a 6-sided die:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "d6 = Die(6)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And use `thinkbayes.SampleSum` to generate a sample of 1000 rolls." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "dice = [d6] * 3\n", - "three = thinkbayes.SampleSum(dice, 1000)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`SampleSum` takes list of distributions (either Pmf or Cdf objects) and\n", - "the sample size, n. It generates n random sums\n", - "and returns their distribution as a Pmf object." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def SampleSum(dists, n):\n", - " pmf = MakePmfFromList(RandomSum(dists) for i in xrange(n))\n", - " return pmf" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "SampleSum` uses `RandomSum`, also in `thinkbayes.py`:\n", - "\n", - "```python\n", - " def RandomSum(dists):\n", - " total = sum(dist.Random() for dist in dists)\n", - " return total\n", - "```\n", - "\n", - "`RandomSum` invokes `Random` on each distribution\n", - "and adds up the results.\n", - "\n", - "The drawback of simulation is that the result is only approximately\n", - "correct. As `n` gets larger, it gets more accurate, but of course the\n", - "run time increases as well.\n", - "\n", - "The other approach is to enumerate all pairs of values and compute the\n", - "sum and probability of each pair. This is implemented in `Pmf.__add__`:\n", - "\n", - "```python\n", - " # class Pmf\n", - " def __add__(self, other):\n", - " pmf = Pmf()\n", - " for v1, p1 in self.Items():\n", - " for v2, p2 in other.Items():\n", - " pmf.Incr(v1+v2, p1*p2)\n", - " return pmf\n", - "```\n", - "\n", - "`self` is a Pmf, of course; `other` can be a Pmf\n", - "or anything else that provides `Items<`. The result is a new\n", - "Pmf. The time to run `__add__` depends on the number of items in\n", - "`self` and `other`; it is proportional to\n", - "`len(self) \\* len(other)`.\n", - "\n", - "And here’s how it’s used:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "three_exact = d6 + d6 + d6" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When you apply the `+` operator to a Pmf, Python invokes\n", - "`__add__`. In this example, `__add__` is invoked twice.\n", - "\n", - "This shows an approximate result generated by\n", - "simulation and the exact result computed by enumeration:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX5+PHPk8m+L4SELJAAYQcBAQEFAoIs4lL7q8vX\nale19etSa+vSTbT92r0utYut1rbaqtVqiwqyiAEURJB9D4EEEhIg+77MzPn9McMkhKyQ5E6S5/16\n8WLuvedOnizzzJ17znmOGGNQSinVt/lYHYBSSqnup8leKaX6AU32SinVD2iyV0qpfkCTvVJK9QOa\n7JVSqh9oN9mLyCIROSgimSLycAvHR4nIZhGpFZEHWzhuE5EdIvJOVwWtlFKqc9pM9iJiA54DFgFj\ngFtEZHSzZkXAvcCvWnma+4H9gA7oV0opi7R3ZT8NOGKMyTbGNACvAdc1bWCMOWOM2QY0ND9ZRJKA\nJcALgHRNyEoppTqrvWSfCJxosp3r3tdRTwHfBZydjEsppVQXai/ZX/CtFxFZCpw2xuxAr+qVUspS\nvu0czwOSm2wn47q674iZwLUisgQIBMJF5O/GmNubNhIRvZevlFIXwBjT4Qvp9q7stwFpIpIiIv7A\nTcDyVtqe80WNMd8zxiQbY1KBm4F1zRN9k7Ze/++xxx6zPAaNU+PszXH2hhh7U5yd1eaVvTHGLiL3\nAKsAG/CiMeaAiNzlPv68iMQDW4FwwCki9wNjjDGVzZ+u09EppZTqEu3dxsEYsxJY2Wzf800eF3Du\nrZ6WnmM9sP4CY1RKKXWRdAZtB6Wnp1sdQodonF1L4+w6vSFG6D1xdpZcyL2fLg1AxFgdg1JK9TYi\ngunCDlqllFJ9gCZ7pZTqBzTZK6VUP6DJXiml+gFN9kr1oLLyWhwOLRWlel674+yVUhevrLyWP/1t\nO5u35hIRHsicmUOYNzuFwUkRVoem+gkdeqlUN/v0szx+/5dtlFfUnXcsbWgM82ancPllyYSG+FsQ\nneqtOjv0UpO9Ut2kqrqeF1/ZScZH2e229fezcdmURObNTmX86IH4+GihWNU2TfZKeYHd+07x3Atb\nKSyq9uyLigzi7q9NQYAPNmSzdUcedvv59+9jY0KYOyuFubNSiIsN6cGoVW+iyV4pC9XV2XnlX3t4\nb03mOftnzRjMHbdNJjS08VZNRWUdGzYd58ON2RzNKWnx+caPGci8WalMn5JIQIB2salGmuyVskhm\nVhHP/mkrefnlnn2hIf7c9eVLufyyNmsFcjS7hA83ZrN+Uw6VVfXnHQ8O8uOK6YOZNyuFtGHRiOht\nnv5Ok71SPazB7uDN/x7g3+8cwOls/Fu+9JJBfPNrU4iODOrwc9U3ONi6/SQfbsxmx56CFuuWJyeG\nM3dWKnNmDiEqMrBLvgfV+2iyV6oHHc8t49nnPz3nNkxgoC9fvXUiV85Ovagr8KLiajI+yuGDjcco\nONV8eQiw2XyYPCGeK2encunEQdhsOm2mP9Fkr1QPcDicvLMqk1ff3EuD3eHZP2ZkLPfcMZX4gaFd\n9rWMMRw4XMi6Ddl8/OkJ6urs57WZMimBR+6/XEfx9COa7JXqZgWnK/ntn7Zy4PAZzz4/Xxu3fmEc\nSxeO6NaEW1PbwKYtuazbmH3O1wd4+P7LuezSxG772sq7aLJXqpsYY1iTcYyX/rnznKvroUOiuO+u\naT0+G/ZkQQX/eGMPm7fmAq4JWj97bJ523vYTmuxVv7P/0Bme+eOn2O1OEgaFkjAojMT4MM//A2ND\nOnU/2+E0vH3oFP89dJoHp6cwMT6c4tIafv/CNrbvzgcgZ3AY1SF+REcGER0VxNn8+sSc4UyMDz/v\nOX/wYSa7T1dgE2FqYgTfmJxMdJDfRX/vxaU1fOPb73nG6z/xaDrjRg+86OdV3q+zyV4H7qpezRjD\n83/9jDNFVQCUlNWw7+C5tzd8fX2IHxhKQpM3gIRBYSQlhBEWGnBO29NV9fxmSzZ7Tld49n285QTP\n//Wzc4ZEhgT7E5UQSkCAL04MtHO9YgCHMTiM4eMTJew5XcG9UwYzMznqor7/6Mgg5s5KYc2HRwF4\n692DmuxVizTZq15t975TnMgrb7ON3e4k92Q5uSfPbxcWGkDioDAS4sMoj/BnQ3UtDh/Bz8+GcRr+\n9uZecj7NPeecq69KIzM+mD2F54+Q6ajyOjv/9/FRfnHlSMbGXlxn7vWLR7I24xjGGHbuKeBodglD\nUy7uTUT1PZrsVa/2zvuNM1XnzUpl+tRETuZXkFdQwcn8Ck7mV1JSVtPq+RWVdRzMrOOT0+WcTGhM\nugLEldZxrKDKUwd8QEww994xjfFjBuJwtnwp39rt8ifmDAdg75lKntqSzZnqei5PjmLMgIsvhzAo\nPowZU5PY9OkJAN5+7yAP/u+Mi35e1bdosle9Vl5+ueceOgifv2YUg+LDYOK57aqq65u8AVSSl1/B\nyYIK8gsqqG9wDZsML6+nKMZBXYAN/3oHCSerCK5p7ISdNyuVr9x6CSHBrnIHtk6OuDnb/pK4MH67\ncDQv7znJF8cndFln6ueWjvIk+02f5vI/n69w/SyUcutQsheRRcDTgA14wRjz82bHRwEvAZOA7xtj\nfu3enwz8HRiI67bln4wxz3Zd+Ko/e2/1Ec/jqZMGtZrcQoL9SRsWQ9qwmHP2O52GwuJq1xtBfgV7\n88rYXFxF9JlaSt2JPjIikG98+VKmdeGQxrAAX+6eMrjV43V2JwG+nZsgNSwlikvGxbNrr2vW7X9X\nHuYbX7n0YkNVfUi7yV5EbMBzwHwgD9gqIsuNMQeaNCsC7gWub3Z6A/CAMWaniIQCn4nImmbnKtVp\nlZX1rNuY7dleunBEh891OA02H8HHRxg4IISBA0KYOD6eq5u0qa21U1Ja0+mRPBcrt7yWR9cd5kuX\nJHJlSudq4NywdBS79hYAsG7jMW783JhOlWpQfVtH/oqnAUeMMdnGmAbgNeC6pg2MMWeMMdtwJfem\n+wuMMTvdjyuBA0BCl0Su+rU1649SX++6+h6SHMm40bHtnmOMYcWRM9y76gDVDY422wYG+jIoPqxH\nE73dafjVJ9kU1zbw1JZsfrrpGOUtzJZtzbjRsaQNdX16sdudvLsqs50zVH/Skb/kROBEk+1c975O\nEZEUXLd5tnT2XKWacjicrFzbeAtn6VVp7V4Bl9Y28MTGLH637Tg5ZTX84bMTbba3QlFNPTVN3oQ+\nPlHCPe/vZ2dB26ONzhIRPrd0pGd79bosqqrPr6Cp+qeO3LO/6BlP7ls4bwL3u6/wz7Fs2TLP4/T0\ndNLT0y/2S6o+7JNteZ5FQcLDApg1o/X73wCfnizjmS05lNY1fvDMKqmmusFBsJ+tW2PtjLiQAJ5Z\nOIoXduSxMss1V6CoxvUm9ZdrxhEZ2P4krGmTE0kcFE5efjnVNQ2s+iCLG64Z3d2hqx6QkZFBRkbG\nBZ/f7gxaEZkOLDPGLHJvPwo4m3fSuo89BlSe7aB17/MD3gVWGmOebuEcnUGrOuV7P17HwcxCAG68\nfgw33zCu1bbZpTX87/v7z9l33YiBfGlCYqc7QXvSlrxSnv40h/I6O9+YnMw1Izo+UeqD9cf43Ytb\nAVcH8x9/fTX+/t7zpqa6Rmdn0Hbkr30bkCYiKSLiD9wELG/t6zcLRoAXgf0tJXqlOiszq8iT6G02\nHxbOG95m+5TIIBYNGwBAdKAfP56Txp2Tk7060QNclhjJ7xeN4dZxCSxNa78/oqnZlw8mOsrVMVta\nVsuHHVgDV/V97f7FG2PswD3AKmA/8Lox5oCI3CUidwGISLyInAAeAH4gIsfdt24uB74IzBWRHe5/\ni7rtu1F93rurGzsdr5ie3KHFO74+MYlr0gby3OIxTB50ft0abxUV5Mf/jBvUYn+EMabVzls/XxvX\nLW68d/+fFYdwOM5f61b1L1oITfUaRcXVfOPBFZ7E9csnFjCsSVmAWruTQC+/Yu8q67KL+POOXO6Z\nMpjLW6ivU1PbwF0PvOep5/Ptu6dzxfS2+zZU79Idt3GU8grvf5DlSfRjRsaek+gzsov5yjt7OFJc\nbVV4Paagso4/fnaC8jo7T358lGc+zTlvKGlQoB9LFjTe4nrr3YMtLnGo+g9N9qpXqKuzs9pd2RFg\n6cI0z+M3DhTwy09cY9J/ufkYtfa+fcuirM5+ziii1UcLeXTdYeqafd9LFqTh7+8acJd9vJQduwt6\nNE7lXTTZq15h/abjVFTWATBwQAhTJ7nm5h0srOTl3Sc97RzGUNjHx5aPjAnhtwtHM2dwtGffkZJq\nXtqVd0678LAAFsxJ9Wy//d7BHotReR9N9srrGWN4b/Vhz/bVV6Vhs/lQ3eDgV59k43Dfnhg7IJRn\nF44mKbz9TtveLizAl4dmpvL1iUkABPnaGB4dfF67axaP8MwC3nfwjGckk+p/NNkrr9e0Zn1goC/z\nZqcAkJFTTL77aj/Ez8aDM1K8apJUT7h+5EBuHjOI5xaNZn5qzHnHBw4I4YrpyZ7tt9/Vq/v+SpO9\n8npNa9ZfOSvVU2Z48bABfPuyFIJ8bdx96WDiQgJae4o+S0S4bUIC8aGtf++fu3qU5/HWHSc5nlvW\nE6EpL6PJXnm15jXrm44wERGuTI3hz1ePJT0luuUnUAxOivD0cYBr3L3qfzTZK6/WkZr1UV2wcHdf\n1HSo5eeWNl7db9x8nDOFVVaEpCykyV55rYupWd+fnS3l/PAHh2lwz0sYlTaAMSNdZRccDifL3z/c\n1lOoPkiTvfJazWvWhyWEcqK81uKovJsxhp9tOsbvth1nX2Elr+zN9xy7ocnV/ZqMY5RX1FkRorKI\nJnvllZrXrF84fzg/23SM+1cdYMWRMzobtBUiwugmi5j/+8Apdp+uAGDShHiGJEcCUF9vZ8WaIy0+\nh+qbNNkrr9S8Zv2RYBsnymupczh5YUcuhTUN7TxD/3XtiIFMinMVfDMYfvNJNhV1dkTknKv7FWsy\nqanVn2N/ocleeaX3mlS3HHXFYFYea5wMdOfkZGLdwy/V+XxEeOCyIYQHuEolnKmu5087cgGYOS2J\nuFjXlX9lVT1rM45ZFqfqWZrslddpWrPe+NvY3mQ9tRmJkSwcev7kIXWumGB/7ps6BIDhUcHcOCYe\ncK0BcN2SxvLHy98/TIO97fV4Vd+gyV55naY169MuTeBs1fboQD/unTak3fVmlcuMpEi+d/lQfjV/\nJMlNSkjMm5VKhHu7qLiajZuOWxWi6kGa7JVXKSquZtOnuZ7tuxaP4umrRjE8KphvT08hIqAjyyar\nsy5PjsLPdu7L3N/fxtKrGquGvv3eIZxO7fDu6zTZK6/SUs36wRFBPHXVKCbF955VprzdwiuHEeRe\nwDwvv5ytO062c4bq7TTZK6/RVs16H71102WMMTh8fVh45TDPvrfe0cVN+jpN9sprtFazXnWdijo7\nP990jAfXHOTKeUPxdS/jmHm0iH0Hz1gcnepOmuyVVzhbs74q2JfSCH+WLBjuqcOuuobTGB5Zd5iN\nJ0rIr6zjX0cLmTsrxXP8LS1/3Kfpq0l5hd37TnGsoIK8xFAKksLYFexDRZ29/RNVh/mI8IXR8Z7t\ntceKSJg8yDO6aeeeAo5ml1gVnupmmuyVV1j+/mEK4kOw+/oQFurP/uJqGnSESJdLT4lm7pDGctCv\nZhVyyZREz7YuXdh3abJXlsvLL+fDnGLKw12zYiPCA/nWtCFEa+nibvHNSwcz0D0DuaLejhnZOElt\n06e55BdUWBWa6kbtJnsRWSQiB0UkU0QebuH4KBHZLCK1IvJgZ85VCuBV91U9QEiwH9eOiuOyxEiL\no+q7QvxtfGdGKjYRrkkbyCPz0rhknOv2jjGG/67U8sd9UZvJXkRswHPAImAMcIuIjG7WrAi4F/jV\nBZyr+rnKyno+2pqLX4NrbH3awDC+PimxnbPUxRobG8ofl4zhG5cmE+Drc06BtHUbj1FcWmNhdKo7\ntHdlPw04YozJNsY0AK8B1zVtYIw5Y4zZBjQvn9fuuUqtWX8Un8p6Uo+VMcwpPHHVSAJ9+9ei4VZJ\nCGssoTBudCxp7ppDdruTd1dltnaa6qXaS/aJwIkm27nufR1xMeeqfqBpzXofA9++YigjYkLaOUt1\nBxHhc0sbC6StXpdFVXW9hRGprtZeoZGLGQ7R4XOXLVvmeZyenk56evpFfFnVWzSvWT9rxmCLI+rf\npk1OJC4hnOziKqhpYNUHWdxwjd559RYZGRlkZGRc8PntJfs8ILnJdjKuK/SO6PC5TZO96j+a1qxf\ndOUw/P319o2VDhdXcXJUNCdO+DD0aBnL3z/MovnDCdZRUV6h+YXw448/3qnz27uNsw1IE5EUEfEH\nbgKWt9K2efGSzpyr+pENOcX8cOUB9mW5atbbbD4snDfc4qj6twaHk59tOkZDgA0CfTmZEEpZRR3L\nVx6yOjTVRdpM9sYYO3APsArYD7xujDkgIneJyF0AIhIvIieAB4AfiMhxEQlt7dzu/GaU9ztTXc/v\nth1n5ZEzHEuNoM7fhyumJxMVGdj+yarb+Nl8uH/aEEQgOjKIylA/SqIC+M+Kwzoyp48QqyvdiYix\nOgbVc361+Rhrsgo5nluGb72ToUdL+c2yBQxLibI6NAW8sCOXtw+e4sTJcuy1doYfKWXx7FS++ZUp\nVoemmhERjDEdLgerM2hVjymorGP98RLKK+owBhJOVjI+LVYTvRe5bXwCSeGBxEQF4fQRyiICWJtx\njBN55VaHpi6SJnvVY948cAqHw0l5eR0hVQ2EVNvPqVmvrBfg68NNY+MZmxDOLJsf0cW1GGN45V+7\nrQ5NXSRN9qpHGGMoqqmnoqoeh9MwoLBGa9Z7qfQh0fx24WgeuXGCZ9TF1h0ntd59L6fJXvUIEeGx\n2cMZlVdNTFENwdV2rVnvpXxEEBGGpUQxe8YQz/6/v7ZbV7PqxfSVpnrMkWPFFB0tJu50DQF+NubN\nSrU6JNWOW/7fuHNWs2q6GLzqXTTZqx6zJqNxfdkZ05IIDfW3MBrVEXGxISyZ39iv8sobe2iwOyyM\nSF0oTfaqR1TXNLBx83HP9lVzh7XRWnmTJUvSqIwPpTLEj1OnK1m97mj7Jymvo8ledSuHe7Wpjz45\nQW2ta5nB5MRwRqXFtHWa8hI7C8q5b10mVcMiOD0wCAP86z/7tUhaL6TJXnUbu9Nw98r9/H7bcZav\nb7waXJA+1LPuqfJuKZFB1DmcRIQHYg/1pyrEl4rKOv7znpZR6G002atuk5FTTG5FLW/tKyDD5sQA\nfr425lw+pN1zlXeIDPTjqqExrjIKUUEUxQQB8M77hz0VS1XvoMledQunMbx5oACA8oo6ootrEWDG\n1CTCQgOsDU51yudGxmETISzEH3tUIDWBNuobHLz+9j6rQ1OdoMledYsteWWcKK/FOA3VFXVEldQB\nsGDuUIsjU50VHxrAnCHRIBATHexZGH7dxmxyTpRaHJ3qKE32qssZY3jDfVVfWdVARFENNqchcVA4\nY0YOsDg6dSE+PyqOS+PDefrqMSyMDwdcv+eXX99jcWSqozTZqy5X5zAkhAZgE6GyvI6YoloAFqSn\nasdsL5USGcQT6WlMGBjG7TdewtnlK7bvzmfP/tPWBqc6RJO96nKBvj58Z0YqP5yYTNSxUnwdBl9f\nH9KvSLE6NNUFUgZHMndWkzIKr+/G6dQyCt5Ok73qNrs+PUFEuWs89vQpSYSHacdsX3HLDePw93Mt\nI5l1rJhNn56wOCLVHk32qlvU1dnJ+CjHs70gXTtm+5IBMcFc3aQ89T/e2EN9g5ZR8Gaa7FW32PRp\nLtU1DQDEx4UybnSsxRGprlRc00B1agSViWEAnDpTxaoPsiyOSrVFk73qMqW1DZ7HTYue6YzZviWr\npJqvvbOX5VmFOIZG4vBx/W7f+O9+Kqu0jIK30mSvusTpqnq+vHwPT358lA37T3EwsxAAm82Hudox\n26ekRAQxMMQ11j4g1B/HkAgAKqvqeevdg1aGptqgyV51ibcOnqLBafj4RAm/2dD4cX7a5EQiIwIt\njEx1NZuP8PnRcQCIgEmNwOn+4PbeqkzOFFZZGJ1qjSZ7ddFKaxtYfdR1JW8MmMNFnmNXzdOO2b5o\n7pBoYoPd6xEE+BI4wlXFtMHu4NW3tIyCN9Jkry7aO4fPUOdwAhDuMPgU1gAQNzCU8aMHWhma6iZ+\nNh+uH+n+3QqMmty4lnDGRzkcy9EyCt6m3WQvIotE5KCIZIrIw620edZ9fJeITGqy/1ER2Scie0Tk\nnyKiA637mOoGB+9kNs6gDDpR4VmkesGcVHx8tGO2r1o0bACLh8Xyu0VjWLZ4NFMmnk34hpf/tdvS\n2NT52kz2ImIDngMWAWOAW0RkdLM2S4Dhxpg04E7gD+79KcAdwGRjzHjABtzcxfErizmchquGDiDQ\n14coXxulB84A7o5ZXWO2Twv0tXHP1MGkRLrKHn/xxvGeUVc79xSwa2+BleGpZtq7sp8GHDHGZBtj\nGoDXgOuatbkW+BuAMWYLECkicUA50AAEi4gvEAzkdWXwynphAb58fVISL10zngnVDs9V/dRJCURF\nasdsfzI4KYIrZze+wf/9NS2j4E3aS/aJQNN50Lnufe22McYUA78GjgMngVJjzNqLC1d5q0AR9m5u\n/DPQGbP90803jMXf3xeAY8dL2bA5p50zVE/xbed4R9+Wz7sxKyLDgG8BKUAZ8IaI3GqM+UfztsuW\nLfM8Tk9PJz09vYNfVnmLLZ/leSbUxMaEcMm4OIsjUlaIjgpi6cI03nrnAAD/fHMvM6cm4+9vsziy\n3i8jI4OMjIwLPr+9ZJ8HJDfZTsZ15d5WmyT3vnRgkzGmCEBE3gJmAm0me9U7rf6wcWz9grnaMdsf\nHS+r4c0DpyiJCSA8LIDyijoKi6pZsfYI1y8ZaXV4vV7zC+HHH3+8U+e3dxtnG5AmIiki4g/cBCxv\n1mY5cDuAiEzHdbvmFHAImC4iQeLqtZkP7O9UdMorGWM4VNQ4cSYvv5x9B10dsz4+wjztmO13Smoa\nuPf9A3yQXcT20xXMXtxYJO3fyw9QUVlnYXQK2kn2xhg7cA+wCleift0Yc0BE7hKRu9xtVgBHReQI\n8Dxwt3v/TuDvuN4wzo7D+lO3fBeqR23NL+fbaw7y0NpDbM8vZ23GMc+xSy8ZRHRUkIXRKStEBfkx\nMznSs50fEcCgOFeRtKrqev69XMsoWE2Msba3XESM1TGoznlo7SH2FVYCcM3wWDb/ZYfnyu37D87i\n0ksGWRmeskhWSTX3rXLdqxeEO5Oi+OuftgLg6+vDb3++mLjYECtD7FNEBGNMh++X6gxa1Sl7T1d4\nEr2vCMk1Dk+iHxATzETtmO23hkUFM2WQqyiawXDUTxgxzFVGwW538uqbe60Mr9/TZK865Y0DpzyP\n56XGsPXj457tK2enYrPpn1R/9gV3gTQ/HyHQ18ZtN433HNuwOYes7BKrQuv39JWpOiyrpJpt+WWA\n62P6rJgQdu93JX8ROWdCjeqfxg0M467JyfzlmvF849Jkxo4ayLTJjVNzXn5dyyhYRZO96rBBoQF8\n5ZJEogL9uDw5kr3bTnqOXTpxEANigi2MTnmLa0cMJDrIz7P9xRvHe4bi7t53iuO5ZVaF1q9pslcd\nFuxn4/+NjufFpeP42iUJfLgx23NswRydMatalpQQzmWXNl7dr1hzxMJo+i9N9qrTAnx9OLL/DGXl\ntYBr1uTkS+Itjkp5syULGsfdZ3yco8sXWkCTvbogqz9sXGN2/hztmFVtS06NZHCSa6ROfb39nE+F\nqmfoK1R1WsHpSnbvOzsqRztmVcuMMew+XcGPMjK59/2DLJg3zHNs5QdHtCJmD9Nkr9pUUWfn4xMl\nOJtMfPtgfeOM2ckT4okdoBNl1PkanIZfbjrGZwXlFNbU40wKI8S9lGHBqUp27tF69z1Jk71q07uZ\nZ3jy46N8c8V+tuSVYrc7Wbch23N8wVztmFUt87f5cN3IxmUp38kqJH3WEM+2dtT2LE32qlW1dgfL\n3UsO5lbUUtXgYNvOk5SUudaYjYoI0tIIqk1LhscS6ucqb5xXUcuAcXGcrYi+fXcB+QUVFkbXv2iy\nV616L7OQ8jo7AHEhAcwZHM2ajMaO2SvnpOLrq39CqnXBfjaWpjVe3a/JL2XSJWdLahjeX5fV8omq\ny+krVbWorM7O6/vzPdufHxVHYVE1O3Y3dszOm51iSWyqd7lmRCwBNh9C/WzMT41h0ZXDPcc+WH+M\n2lq7hdH1H+0tXqL6qTVHC6lqcACQGBbIVUNjePM/+zm7eNnE8XHEDwy1MELVW0QG+vG9K4YyIjqE\n8ABfnE7DoLgw8k9VUF3TwPpNOSxsMlJHdQ+9slctumFUHN+dnkpssD9fm5iID7C2ySgcXWNWdcaU\nQRGEB7iuLX18hMXzmwzDXHsELXPe/TTZqxb5iJCeEs3zS8YyLSGC7bsKKCl1dcxGRgQydVKCxRGq\n3mzurBQC3Mn/eG6ZZ6Uz1X002as2Bfj6ICLnrDE7d1aKdsyqixIS7E/65Y3DMFeu1WGY3U1fsapd\nZwqr2L67cQKMFj1TF8tpDOFjY6kJdA3L3PJZHoVF1RZH1bdpslcejlamr3+wIdtzT3XC2Dji47Rj\nVl247NIaHlh9kFeyzlA3KgYDOJ3mnHpLqutpslcAVNbbuWvFPt48UEC9w+nZ73A4+WCDdsyqrhPo\n60OOe2KeMyKQ8nBXCYXVH2ZRX++wMrQ+TZO9AuC1fQXkV9bx0q48lm1ovH+6Y08BRcWuj9fhYQFM\nu1Q7ZtXFiQ8N4LqRrolVIcF+lCaF4RQor6hj09YTFkfXd2myV5ysqOWdw6c924uGDfA8XtPko/W8\nWSn4+dp6NDbVN900Jp7IAD8QCIwMpCgmCNB6Od1Jk73ipV152N335McOCGVWchQARcXVbNvZOIv2\nyjlaylh1jWA/G18c76qrFBYWQHWYPwY4crSYzKwia4Pro9pN9iKySEQOikimiDzcSptn3cd3icik\nJvsjReRNETkgIvtFZHpXBq8u3q5TFWzKLfVsf21SEiKCMYZX3tjj6ZgdN3ogiYPCrQpT9UFXDR3A\npLhw7r/X8ObyAAAfmElEQVRsCLcnRrrLo8EKHYbZLdpM9iJiA54DFgFjgFtEZHSzNkuA4caYNOBO\n4A9NDj8DrDDGjAYmAAe6MHbVBSICfBk/MAyAeSnRjIxx1aZ/d1Um6z/O8bRbPH94i+crdaFsPsJP\n5qaxeHgsVzdZtvDjLScoLau1MLK+qb0r+2nAEWNMtjGmAXgNuK5Zm2uBvwEYY7YAkSISJyIRwCxj\nzF/cx+zGGF1W3sukRAbx07lp/OCKYdw+wbUo9I7dBfz11V2eNnNnpTB9SmJrT6HURRueGs2IYTEA\n2O1O1q7XYZhdrb1knwg07R7Pde9rr00SkAqcEZGXRGS7iPxZRIIvNmDV9USEGUmRxAb7k5dfzq9/\nt9lz+2bk8Bju+vKliEg7z6LUxVmyoPHT4/sfZGG3O9torTqrvaqXHa1O1DwTGPdzTwbuMcZsFZGn\ngUeAHzU/edmyZZ7H6enppKend/DLqq5UWVXPz57eRHVNAwAx0cE8dN/l+PvpCBzV/WZMS+Kvr+6i\nqKKOwtIaPt2ex8xpyVaH5TUyMjLIyMi44PPbS/Z5QNOfdjKuK/e22iS59wmQa4zZ6t7/Jq5kf56m\nyV5Zw+Fw8tQfPiEvvxwAfz8bD98/k6jIQIsjU/2Fj48P8ZclsiW7iJiiWlauPaLJvonmF8KPP/54\np85v7zbONiBNRFJExB+4CVjerM1y4HYA92ibUmPMKWNMAXBCREa4280H9nUqOtUtVhw5Q2F1/Tn7\nXvnXHnY0qX9zzx1TGZ4a3dOhqX5sXXYR23Hi9PWhcEAQuzILyT5e2v6JqkPaTPbGGDtwD7AK2A+8\nbow5ICJ3ichd7jYrgKMicgR4Hri7yVPcC/xDRHbhGo3zZDd8D6oTDhRW8rttx7nzvX38c28+xhg+\n/Cib/6485Gnz+WtGc8X0wRZGqfqjuSkxDIkKJiTYH6ePcCY2iJUf6DDMriJWLxogIsbqGPoLpzF8\nZ+0hDhVVAXBFchQ3xEXwwyc/9HSGTZmUwCP3X46Pj3bIqp63Ja+U7605RF6+ayHyUScqeeUXSwgN\n9bc4Mu/jng/T4ReqzqDtRzbklHgSvZ+PcP2QaH7xzCZPok9OjOBb37hME72yzLSECKYPiSLA3zUo\n4ER0AOs2HmvnLNURmuz7iVq7k7/uzvNsLx0Wy1/+vI0Sd/XB0BB/Hv3W5QQH+VkVolKICHdMSiYy\nPJCgGjuxZ2pY+UEWzlbKb6uO02TfT+w7U0GJe0hlZIAvxVtyyTpWDLjWBP3OPTO0Tr3yCimRQTy3\ndAzjztQSXGPn1OlKtu/Ob/9E1SZN9v3EpYMieG7xGKYmRDC83vDJJ43z4L5660QmjI2zMDqlzjV+\nUATzZzcW3tNlCy+eJvt+JDk8kKXhwex+L9Ozb8HcoVr3RnmlhVcO4+x8zR27CzxzQNSF0WTfj5zI\nK+ep329B3BOjR4+I5eu3TdJSCMorxQ8MZcqkQZ7t9z/IaqO1ao8m+36iorKOnz79ETW1rvv2sTEh\nPHTfDF2MRHm1Je5PnXab8Pf9BRw5U2FxRL2XJvs+rMq9nqfD4eTXv/uEglOVAPj7+/LIty4nIlxL\nISjvNmFsHP4pkRwZHsmpMD+eXH2o/ZNUi9qrjaN6qXqHk/tXH2BoVDBBWSXs3nfKc+z+u6aROiTS\nwuiU6hgfH2H+1CR27j0JwLb8CnbklzFpUITFkfU+emXfRy0/fJr8yjpW7i/gxZwinO7b8jdeP5YZ\nU5OsDU6pTvif+WnEVLluP9Y3OPj1hqM4dNx9p2my74NKaxv41/4CamvtFBZVM6CwBh8DM6YmceP1\nY6wOT6lOCQ7y4wvDY/FxJ/gDBeWsOabr1HaWJvs+6JU9+ZTVNFBwuhL/WgdRJXWkDI7k3jumaSkE\n1St9fsEIYopcSxVWVTdwuECHYXaWJvs+Jru0hpVHzlBwqhKHwzDwdDURYQE88q3LCQzULhrVOyUn\nhjMnLpzw8npSj5URlVdpdUi9jib7PiY22I+oMzXU19kJqWogosbBQ/fNZOCAEKtDU+qiXDt/OEl5\nlQTVOlibcYx692gz1TGa7PuYFSsPU73tJMOyyhiUX8Wdt09mzMhYq8NS6qJdOnEQsTGui5aKyjo+\n3nKinTNUU5rs+5BPtuXy6r/3AuDf4OT69KFcNXeoxVEp1TVsNh8WzR/m2X5vTSa6FkbHabLvI/Ly\ny3nm+U892xPGxPGV/5loYURKdb35c1Lx93PN+j6aXcLhrGK25ZdR73BaHJn302TfBxhjeOM/B6ir\nswMQNzCUB++Zjs2mv17Vt4SFBjBrhmvJTIcP/Oj9Azy2/gh/333S4si8n2aDXq7W7uDe9/azYn/j\nYuEPfOMywkIDLIxKqe6zyF0vpyIsgN2VtTgchrcPnWKnDsdskyb7Xu6FHblsO17C8YQQCuKCGTsq\nlhHDY6wOS6luMywlilFpA4goqyOkooHy8joAfrMlm3L3p1t1Pk32vdgnuaW8l3mG8grXH3tgrZ3r\nloy0OCqlut+SBcMRIOFkJVUlNTgchqKaBn67NUc7bVuhyb6XKq5p4JmtOZRX1OF0GsLL6xkbEsjk\nCYPaP1mpXu6yKYnEDQzF12EYmFtBcYlrLeVjpTWU6dV9izTZ91LPbs2hrNZOWXkdvnYn8QVVXLto\nhJZDUP2Cn6+Nr97qGm0WVtmA7Xg5kyODeXbhaCID/SyOzju1m+xFZJGIHBSRTBF5uJU2z7qP7xKR\nSc2O2URkh4i801VBK7h5TDz+9Q7sdieJeZXEhAQwZ+YQq8NSqsdMnZTAlIkJAMQXVFH3SR6BOgKt\nVW3+ZETEBjwHLALGALeIyOhmbZYAw40xacCdwB+aPc39wH5Ab6R1oZExIQzJLCUpt5KQajtLFgzH\n319XnVL9y1dvnYifrw0Bjhwt5oMNx6wOyWu19zY4DThijMk2xjQArwHXNWtzLfA3AGPMFiBSROIA\nRCQJWAK8wNmVg1WX2L3/NCeOlxJeUY+/vy+LrhzW/klK9THxcaFcf3XjoIRX/rWHiso6CyPyXu0l\n+0SgaQGKXPe+jrZ5CvguoNPbutg7Kw97Hl85O0XH1at+64alo86pmfPPN10lQ+rsTl7ec9KzPGd/\n117N247eeml+1S4ishQ4bYzZISLpbZ28bNkyz+P09HTS09ts3i9V1tsJ9Xf9uo7nlrF9d777iHDN\nwhHWBaaUxQICfPnKrZfwi2c3AbBq3VFGTknk9ePFHC+v4XRVHQ9OT7U4youXkZFBRkbGBZ8vbY1J\nFZHpwDJjzCL39qOA0xjz8yZt/ghkGGNec28fBNKB+4DbADsQCIQD/zbG3N7saxgdF9u2sjo7976/\nn5lJUXzlkkT+/NJnrNvoujc5Y2oS3713psURKmUtYww//tVGdu5xzSQPHzWAM6kRnsvQh2akMmdI\ntIURdj0RwRjT4dvj7d3G2QakiUiKiPgDNwHLm7VZDtzu/uLTgVJjTIEx5nvGmGRjTCpwM7CueaJX\n7TPG8NtPcyiqaeCdzNM8uvYgGZtyPMevXayTqJQSEb72xYmeelDlBwsZ6tOY3n637Tinqvr3vfw2\nk70xxg7cA6zCNaLmdWPMARG5S0TucrdZARwVkSPA88DdrT1d14Xdf6w+WsTmvFLPdkxhLU53hb9R\naQMYqaURlAIgcVA41zW5+CndeJwB7jH3VQ0OfvNJTr9eqLzddeqMMSuBlc32Pd9s+552nmM9sP5C\nAuzPcstreX57Y9/3wpQYtv51p2f72sV6r16ppv7ftaNZvymHouJqqsrruLSknuJgH5zGEBXoS4PT\nic2nfw5R1hkIXuxvu/Ooc1/FJ4cHklxaT1V1PeAacjZ1UoKV4SnldQIDffnyLZd4trdnHOPqxEi+\nfVkKD89MJdC3fyZ60GTv1e6bOoTZg6PwFeHb04awanWm59i1i0ZovXqlWjBzWhITxsQBrj6v4xnZ\nzEuJRqR/T/XRbOHFwgJ8eWhGKr9dNJozWSWcLqxy7Q8NYO4VKdYGp5SXEhG+dtskz8XQwcOFrG8y\nqKG/0mTv5USE5PBA/rvykGff4vnDCAhot7tFqX4rOTGcpVelebZffm0P1TUNFkZkPU32vcCBw4Uc\nOVoMuKr9LbpyuMURKeX9vnD9GKIigwAoKavh9bf3eY6dKK/lhxmZFPejNwBN9l4ks7iKhhYWTv7v\nisar+vQrhhAZEdiTYSnVKwUH+Z3TWfve6kyO55axLruI+1cdYHtBOU9tycbZTyZ1arL3EgWVdXz/\nw0y+s/YQx8tqPPvz8svZuqNxMeVrFulwS6U66orpyYwdFQuA02l44eUdxAT6Ue9wJfjtBeW8c/iM\nlSH2GE32XsDhNPxmSzZVDQ6OlFTz803HPFcby99vLHg2ZVICSQnhVoWpVK8jItxx+2TPoj57D5ym\nIruUG0YN9LT56648sktrWnuKPkOTvRd480AB+85UAmAT4b5pQ/ARobSsloyNjaMIrtPSCEp12uCk\nCJYsaOys/euru/h8WizDo4IBqHc6+eXmY9S3cAu1L9Fkb7FDRVX8c2++Z/vmsYMY6S7X+v4HR2iw\nu8qzDh8azZiRAyyJUane7qbPjfH0dRWX1PDfdw/x4PQU/G0+BNh8WJoWi18fX9JTk73FMnKKsbtv\n2YwZEMpNY+IBqKuzs3JtlqfddYtH9vtJIUpdqJBgf267aYJn+533D+NT1cB3pqfwzMLRLB4e2+df\nX5rsLXbnpCTumTKYqEA/Hpyegs19dZHxcY5nxZ2BA0KYPqX5mjFKqc5Iv3wIo0a4Ph07HE5eeHkH\nM5MiSQ7vH6PbNNlbTERYPDyWF5eOI9692pTD4WR5k5Woli5M09IISl0kEeGO2yZ7ruB37zvFJ9vy\nLI6q52gG8RIBvo2/im0788k/VQG4Pn7Om937V9lRyhukDolk8fzG9Zpf+udO6urs57Ur6IPr2Gqy\n72EdWZWraWmEq+YNJTjIrztDUqpfuflz4wgPc32KLiyq5t/vHDzn+PtZhdz13j42HC+2Irxuo8m+\nB5XX2Xn0w0wOF1W12uZgZiEHDxcCYLP5cHWTIWNKqYsXGurPbTc2dtb+Z8VBTha4Pkmvyy7it1tz\nsBvDrzdns/VkmUVRdj1N9j2kusHBY+uPsOd0Bd//MJM9pytabPdOk0lUs2cOJjoqqKdCVKrfmDsr\nhbShrlXe7HYnf3llJ8YYJseHkxTm6rC1G8OTHx9ldyuv1d5Gk30PqLU7eXxDFoeLXVf0NXZniwWY\nCk5VsnlrY4eRTqJSqnv4+Ah3fGkSZ1ck3747n2078okM9OP/5qYRH+K6zVPvcPLEhiwOtfFpvLfQ\nZN/NGhxOfvrxUfaeabw6+N8pyS2udP/u6sOcXap30oR4BidF9FSYSvU7w1OjuWruUM/2i6/soL7e\nwYBgf/5vbhrR7vVr/WxCXxiBr8m+m+07U8ln+eWe7a9NTGLx8Njz2lVU1rF2fbZn+1q9qleq2936\nhXGEhvgDcLqwirfedXXWxocG8JO5aaREBPHTuSMY4Z7V3ptpsu9mE+PDeWhGCr4i3Dx2EDeMimux\n3ap1WdTXu4aApQyOZMKYgS22U0p1nbDQAG79wnjP9tvvHqTgtKtO1ZCIIH67aDQpkX2j30yTfQ+Y\nPSSaZxaO5ovjBrV4vL7ewYo1Rzzb1y4e0eenbivlLebPSWVYquu2aoPdwUv/2Ok55tOHXoea7HtI\nSmRQqwl8w+YcSstqAYiOCuKKywb3ZGhK9Ws2mw933DbJs711x0kyPs5u85yNx0sob2EyljfrULIX\nkUUiclBEMkXk4VbaPOs+vktEJrn3JYvIhyKyT0T2ish9XRm8Nyqt7dwyZ06naVYaYQS+vvoerFRP\nGjE8hiubzFR/9vlP+cWzmygsqj6v7X8OneJnm46ybMMRqhscPRnmRWk3q4iIDXgOWASMAW4RkdHN\n2iwBhhtj0oA7gT+4DzUADxhjxgLTgf9tfm5fsuLIGe58bx97OzEud/vufHJPujpwgwL9WJCupRGU\nssIXbxzPgJhgz/Yn23K595H3efu9g9jtrlr3x0qreWGHa3j0oaIqfrwxizp776iD35FLyGnAEWNM\ntjGmAXgNuK5Zm2uBvwEYY7YAkSISZ4wpMMbsdO+vBA4ACV0WvRdZl13E77edoKrBwY/WH2G/ezGS\n9jRdX3ZB+lBCgv27K0SlVBsiwgP55ePzmTsrxbOvrs7Oy6/v5sEfrmHfwTOkRgbzjUuTPcd3n67g\np5uOYnd6/zq2HUn2icCJJtu57n3ttUlq2kBEUoBJwJbOBuntNueW8vSWHIx7jHxKZFCHevCPHCtm\n30HX+pc2mw9LF2ppBKWsFBEeyL13TOMn3597zjyXE3ll/PDJD3nm+S1cMTCML1/SmAK3nizjj5+d\naOnpvIpvB9p09C2ree+j5zwRCQXeBO53X+GfY9myZZ7H6enppKend/BLWm9HQTk/33QUh7vAWUpE\nEMtmDyfYz9buuU3v1c+clnTOR0illHXGjIzlV08s4L3Vmbz29j5PZcz1H+ewbUc+t35hHF8YFccb\nB08RHejH0rTz5850tYyMDDIyMi74fGmvCqOITAeWGWMWubcfBZzGmJ83afNHIMMY85p7+yAwxxhz\nSkT8gHeBlcaYp1t4ftORSpDeakNOMb/6JBuHMSSEBvCLK0cS1YEqlacLq7j7Oytwuj/+/fKJBQxL\nierucJVSnVRYVM1L/9zJ5q255+wfmhpNUnoKN05JIiGs5xdAERGMMR0eG9qR2zjbgDQRSRERf+Am\nYHmzNsuB290BTAdK3YlegBeB/S0l+r5g9pBovn/FUBLDAvnJ3LQOJXqA91ZnehL9+DEDNdEr5aUG\nxATz3Xtn8oMHZxE3MNSz/+ixYja8tIN339pPZVW9hRF2TLtX9gAishh4GrABLxpjfioidwEYY553\ntzk7YqcK+IoxZruIXAFsAHbTeFvnUWPM+02e26uv7OsbHPzt1V0cyixqs52Tzk1aOJFX7llM/AcP\nzmLyJS1PuFJKeY/6egdvvXuQt9896Hn9gute/5dumcCcmUN6bEJkZ6/sO5Tsu5M3J3tjDE//cQsb\nNx/vtq+RnBjO008u1BmzSvUi+QUV/PnlHezcU3DO/rGjYrnj9slsr6ol0NfGomEDui0GTfZd6B9v\n7OHf7xzwbDf4Cg1+NoJrumrmnPDoA5czdVKfHI2qVJ9mjOGTbXm8+MoOiktqXPuA4oHB1A2NJDoq\niEdmpjK7hQq3XUGTfRdZtS6L5//6mWd71pxUtkf4UVTbwDfHJjAm6uJHzsREBREZ0T9Wtleqr6qu\naeCN/+znnVWHsRtDdko4tYG++Pr6EBcTzP9dNZLpSV3fJ6fJvgt8tiufnz71kacD9ZKJgygaM4Cs\nUtfU6UBfH15cOo7IQF0bVinlknOilOf/tp29WUXkDAmnLsA1/DosyI+fLxzFvNEtV7y9UJrsL1JW\ndgk/+L8PPeNqE1Oj8JuZxKESV6IXhIdmpHTbRzOlVO9ljCHjoxz+9K/d7BsQQL2/K+EH2p08Oj6R\nz109Cv8OzMHpiO4YetlvnC6s4slff+RJ9LEDQpDLEj2JHuDeqYM10SulWiQizJ2Vwp9+upDbkqLx\ntTvxa3CSmFPO6nVHPTV2rKDJ3q2ysp6f/GojJWWujpbQEH9+8OAs7p42GJt7pMwdk5JY2I2960qp\nviEsNIDvfm0qv79uPHPsQkC9k6/eOpHgDs7D6Q56GwfXWPof/3KDp06Nr68Pjz00h7GjXFOglx8+\nTXJ4IJPiw60MUynVCzkcTj7dfpLpUxLPGWJdWW8n2M92wQuk6D37Tjo7ln7D5uMYAR8D3757OldM\n1wVElFLdw+E0fD8jE18RvnXZEAZcQLVbvWffSf98cy8fbMslZ0gYpwcGc9tNEzTRK6W61VuHTrHn\ndAU7TpVzz/sH+PhESbd/zX6d7Fety+Iv67M4OjSC6mA/7EMiSJoUb3VYSqk+rs7uRNyFgivq7Tz5\n8VGe2pLdrStf9dtk/9Fnefxo7SHyEkNx+gghwX7EDQghr6LO6tCUUn3cF8cn8OS8NGKb3L5Ze6yI\ntcfarsF1MfrlPfus7BK+/PxmzoS7ftAB/jYmDx/Aw5enMmpAaDtnK6VU16ist/OHz06QkVPMuNgw\nnpybhs2nY7fhtYO2HacLq3j08XUUVtRydGgEJsiXL142hPtmpHRowRGllOpq63OKGTUghLiQgA6f\no8m+DZWV9XzvJ+s8C3ybmCC+dPtkrp/UfJVFpZTyDqer6okN9juvMm5nk31HliXs1YwxVNQ7CPQR\nfv7sx55E7+vrw2N3TfeMpVdKKW9TUtPA/asPMC42lHumDiEi4MJTdp9O9iU1DTz9aQ4lNQ0MPlrq\nmTQFcN+d0zTRK6W8ljGGpz7NobzOzqbcUg4WVvHAZSlMHnRhkzv7bLLfklfK0+4fVHFxDZ8dLeFs\nzTkdS6+U8nZ2pyE+pHG0TnFtAz9cn8m1Iwby5Qmdv/Xc54Ze1todPLc1hyc2ZlFeZ6e8vI6SslrA\ntbDAwnnDuH7JSGuDVEqpdvjZfLh7ymAemz2cyIDGmjrvZp4h213DqzP6XLLflFvKyqxCAKqrGyg9\nVcng4+XEna5h6sQEvn7bJF0CUCnVa0xLiOB3i0czNSECgBtHxzMyJqTTz9PnRuMYY/jxxiw2HCum\n6kgxsbkV+DoMQ1Oi+Mn35hIY2GfvXCml+jBjDBuPlzAzOQpfH9HaOCLCzUNjCd5fRHxOOb4Ow8AB\nIXz/27M00Sulei0RYfaQaHw7OOmquT6X/Sor63nuuc345lcArrr0339wFlGRutarUqr/avfKXkQW\nichBEckUkYdbafOs+/guEZnUmXO7Un2D47yx9A/ffznJiVqHXinVv7WZ7EXEBjwHLALGALeIyOhm\nbZYAw40xacCdwB86em5Xe+kfO7ttLH1GRkaXPE930zi7lsbZdXpDjNB74uys9q7spwFHjDHZxpgG\n4DXgumZtrgX+BmCM2QJEikh8B8/tUksXphEX6+ql7uqx9L3lD0Dj7FoaZ9fpDTFC74mzs9q7Z58I\nnGiynQtc1oE2iUBCB87tUomDwvnpj65kw6Ycrlk0oju/lFJK9SrtJfuOjon0moHrkRGBXLtYJ00p\npVRTbY6zF5HpwDJjzCL39qOA0xjz8yZt/ghkGGNec28fBOYAqe2d695v7UB/pZTqpbqy6uU2IE1E\nUoCTwE3ALc3aLAfuAV5zvzmUGmNOiUhRB87tVLBKKaUuTJvJ3hhjF5F7gFWADXjRGHNARO5yH3/e\nGLNCRJaIyBGgCvhKW+d25zejlFKqZZaXS1BKKdX9LCuXICLJIvKhiOwTkb0icp9VsXSEiNhEZIeI\nvGN1LK0RkUgReVNEDojIfvdtNa8iIo+6f+d7ROSfItLxddi6kYj8RUROicieJvuiRWSNiBwWkdUi\nEmlljO6YWorzl+7f+S4ReUtEIqyM0R3TeXE2OfagiDhFJNqK2JrF0mKcInKv+2e6V0R+3tr5PaWV\n3/s0EfnUnZe2isjUtp7Dyto4DcADxpixwHTgf7t70tVFuh/YT8dHKFnhGWCFMWY0MAHwqttm7v6b\nO4DJxpjxuG7v3WxlTE28hGsCYFOPAGuMMSOAD9zbVmspztXAWGPMJcBh4NEej+p8LcWJiCQDC4Cc\nHo+oZefFKSJzcc0fmmCMGQf8yorAmmnp5/kL4IfGmEnAj9zbrbIs2RtjCowxO92PK3ElpgSr4mmL\niCQBS4AX8KJhpk25r+ZmGWP+Aq4+E2NMmcVhNVeO600+WER8gWAgz9qQXIwxG4GSZrs9Ewbd/1/f\no0G1oKU4jTFrjDFO9+YWIKnHA2umlZ8nwG+Ah3o4nFa1Euc3gZ+6J4NijDlz3ok9rJU484Gzn+Ii\naee15BVVL91XfJNw/aF6o6eA7wLO9hpaKBU4IyIvich2EfmziARbHVRTxphi4NfAcVwjtEqNMWut\njapNccaYU+7Hp8Cz2Jk3+yqwwuogWiIi1wG5xpjdVsfSjjRgtoh8IiIZIjLF6oBa8QjwaxE5DvyS\ndj7RWZ7sRSQUeBO4332F71VEZClw2hizAy+9qnfzBSYDvzfGTMY1Msobbjt4iMgw4FtACq5PcaEi\ncqulQXWQe9EFb76Fh4h8H6g3xvzT6liac194fA94rOlui8Jpjy8QZYyZjusi718Wx9OaF4H7jDGD\ngQeAv7TV2NJkLyJ+wL+BV4wx/7EyljbMBK4VkWPAq8A8Efm7xTG1JBfXVdNW9/abuJK/N5kCbDLG\nFBlj7MBbuH6+3uqUu84TIjIIOG1xPK0SkS/jutXorW+ew3C9ye9yv5aSgM9EZKClUbUsF9ffJu7X\nk1NEYqwNqUXTjDFvux+/iaseWausHI0juN6Z9htjnrYqjvYYY75njEk2xqTi6kxcZ4y53eq4mjPG\nFAAnRORsUaD5wD4LQ2rJQWC6iAS5f//zcXV6e6vlwJfcj78EeOUFiYgswnUFep0xptbqeFpijNlj\njIkzxqS6X0u5uDrqvfEN9D/APAD368nfGFNkbUgtOiIic9yP5+HqnG+dMcaSf8AVuO6B7wR2uP8t\nsiqeDsY8B1hudRxtxHcJsBXYhevKJMLqmFqI8SFcb0J7cHV6+lkdkzuuV3H1I9TjKuD3FSAaWOt+\nEa0GIr0wzq8CmbhGt5x9Hf3ei+KsO/vzbHb8KBDtjXECfsDL7r/Rz4B0L4qz6d/nFFz9nDuBzcCk\ntp5DJ1UppVQ/YHkHrVJKqe6nyV4ppfoBTfZKKdUPaLJXSql+QJO9Ukr1A5rslVKqH9Bkr5RS/YAm\ne6WU6gf+PxvSqgR3RsBIAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import random\n", - "import thinkplot\n", - "\n", - "random.seed(17)\n", - "\n", - "d6 = Die(6)\n", - "\n", - "dice = [d6] * 3\n", - "three = thinkbayes.SampleSum(dice, 1000)\n", - "three.name = 'sample'\n", - "\n", - "\n", - "three_exact = d6 + d6 + d6\n", - "three_exact.name = 'exact'\n", - "\n", - "thinkplot.PrePlot(num=2)\n", - "thinkplot.Pmf(three)\n", - "thinkplot.Pmf(three_exact, linestyle='dashed')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Pmf.__add__` is based on the assumption that the random selections from\n", - "each Pmf are independent. In the example of rolling several dice, this\n", - "assumption is pretty good. In other cases, we would have to extend this\n", - "method to use conditional probabilities.\n", - "\n", - "The code from this section is available from\n", - ". For more information see\n", - "Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Maxima\n", - "\n", - "When you generate a *Dungeons & Dragons*\n", - "character, you are\n", - "particularly interested in the character’s best attributes, so you might\n", - "like to know the distribution of the maximum attribute.\n", - "\n", - "There are three ways to compute the distribution of a maximum:\n", - "\n", - "**Simulation**:\n", - "\n", - " Given a Pmf that represents the distribution for a single selection,\n", - " you can generate random samples, find the maximum, and accumulate\n", - " the distribution of simulated maxima.\n", - "\n", - "**Enumeration**:\n", - "\n", - " Given two Pmfs, you can enumerate all possible pairs of values and\n", - " compute the distribution of the maximum.\n", - "\n", - "**Exponentiation**:\n", - "\n", - " If we convert a Pmf to a Cdf, there is a simple and efficient\n", - " algorithm for finding the Cdf of the maximum.\n", - "\n", - "The code to simulate maxima is almost identical to the code for\n", - "simulating sums:" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def RandomMax(dists):\n", - " total = max(dist.Random() for dist in dists)\n", - " return total\n", - "\n", - "def SampleMax(dists, n):\n", - " pmf = MakePmfFromList(RandomMax(dists) for i in xrange(n))\n", - " return pmf" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "All I did was replace “sum” with “max”. And the code for enumeration is\n", - "almost identical, too:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def PmfMax(pmf1, pmf2):\n", - " res = thinkbayes.Pmf()\n", - " for v1, p1 in pmf1.Items():\n", - " for v2, p2 in pmf2.Items():\n", - " res.Incr(max(v1, v2), p1*p2)\n", - " return res" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In fact, you could generalize this function by taking the appropriate\n", - "operator as a parameter.\n", - "\n", - "The only problem with this algorithm is that if each Pmf has $m$ values,\n", - "the run time is proportional to $m^2$. And if we want the maximum of\n", - "`k` selections, it takes time proportional to $k m^2$.\n", - "\n", - "If we convert the Pmfs to Cdfs, we can do the same calculation much\n", - "faster! The key is to remember the definition of the cumulative\n", - "distribution function: \n", - "\n", - "$$CDF(x) = {{\\mathrm{p}(X \\le~x)}}$$\n", - "\n", - "where $X$ is a random variable that means “a value chosen randomly from\n", - "this distribution.” So, for example, $CDF(5)$ is the probability that a\n", - "value from this distribution is less than or equal to 5.\n", - "\n", - "If I draw $X$ from $CDF_1$ and $Y$ from $CDF_2$, and compute the maximum\n", - "$Z = max(X, Y)$, what is the chance that $Z$ is less than or equal to 5?\n", - "Well, in that case both $X$ and $Y$ must be less than or equal to 5.\n", - "\n", - "If the selections of $X$ and $Y$ are independent,\n", - "\n", - "$$CDF_3(5) = CDF_1(5) CDF_2(5)$$ \n", - "\n", - "where $CDF_3$ is the distribution of\n", - "$Z$. I chose the value 5 because I think it makes the formulas easy to\n", - "read, but we can generalize for any value of $z$:\n", - "\n", - "$$CDF_3(z) = CDF_1(z) CDF_2(z)$$ \n", - "\n", - "In the special case where we draw $k$\n", - "values from the same distribution,\n", - "\n", - "$$CDF_k(z) = CDF_1(z)^k$$ \n", - "\n", - "So to find\n", - "the distribution of the maximum of $k$ values, we can enumerate the\n", - "probabilities in the given Cdf and raise them to the $k$th power. `Cdf`\n", - "provides a method that does just that:\n", - "\n", - "```python\n", - "# class Cdf\n", - "\n", - " def Max(self, k):\n", - " cdf = self.Copy()\n", - " cdf.ps = [p**k for p in cdf.ps]\n", - " return cdf\n", - "```\n", - "\n", - "`Max` takes the number of selections, k, and returns a new\n", - "Cdf that represents the distribution of the maximum of k\n", - "selections. The run time for this method is proportional to $m$, the\n", - "number of items in the Cdf.\n", - "\n", - "`Pmf.Max` does the same thing for Pmfs. It has to do a little more work\n", - "to convert the Pmf to a Cdf, so the run time is proportional to\n", - "$m \\log m$, but that’s still better than quadratic.\n", - "\n", - "Finally, here’s an example that computes the distribution of a\n", - "character’s best attribute:" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "3 9.84640042004852e-15\n", - "4 4.0321009720098687e-11\n", - "5 9.806069563928e-09\n", - "6 6.203232264630567e-07\n", - "7 1.7470129795277652e-05\n", - "8 0.0002855723336481055\n", - "9 0.002477241673570361\n", - "10 0.012844085693359392\n", - "11 0.04397964477539067\n", - "12 0.10559054189425404\n", - "13 0.18102225453979948\n", - "14 0.21201352141686403\n", - "15 0.19422452755628616\n", - "16 0.14145216951432227\n", - "17 0.0786340868804698\n", - "18 0.027458253422613543\n" - ] - } - ], - "source": [ - "best_attr_cdf = three_exact.Max(6)\n", - "best_attr_pmf = best_attr_cdf.MakePmf()\n", - "best_attr_pmf.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Where `three_exact` is defined in the previous section. If we print the\n", - "results, we see that the chance of generating a character with at least\n", - "one attribute of 18 is about 3%. Figure [fig.dungeons2] shows the\n", - "distribution." - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH2dJREFUeJzt3WmYVNW1xvH/EtAwGAeuAUUMqBBEweCADN7Yzo2JYMyj\nyMUIeGNIIsZgNIoZJB9ykzhEIHEgKkowEYegkETEIbZigkzKoMwKyOQ8ohDAXvfDLqRpu7uqu6tq\nV9V5f8/TDzWcU/V207169zr77GPujoiIlLY9YgcQEZHcU7EXEUkAFXsRkQRQsRcRSQAVexGRBFCx\nFxFJgLTF3szKzWyZma00s6treH6wmS00s0Vm9i8z617luTWpx180sznZDi8iIpmxuubZm1kTYDlw\nGrABmAsMcvelVbbpDSxx9w/MrBwY7e69Us+tBo5193dz+DmIiEga6Ub2PYFV7r7G3bcDk4EBVTdw\n91nu/kHq7mzg4GqvYVlJKiIiDZau2LcD1lW5vz71WG3+F3i0yn0HnjSzeWZ2ScMiiohIYzVN83zG\naymY2cnAxUDfKg/3dfdNZnYA8ISZLXP3mQ3IKSIijZCu2G8A2le5354wut9N6qDsHUC5u7+383F3\n35T69y0ze5jQFppZbV8tziMi0gDunnGbPF0bZx7Qycw6mNmewEBgWtUNzOwQYApwobuvqvJ4CzPb\nO3W7JXAGsLiWwAX/cd1110XPoJzKWawZlTP7H/VV58je3XeY2QhgBtAEuMvdl5rZ8NTz44FfAPsB\nt5kZwHZ37wm0BaakHmsK/NndH693QhERabR0bRzcfTowvdpj46vc/g7wnRr2exX4ahYyiohII+kM\n2gyVlZXFjpAR5cyuYshZDBlBOWOr86SqvAQw89gZRESKjZnhWTxAKyIiJUDFXkQkAVTsRUQSQMVe\nRCQBVOxFRBJAxV5EJAFU7EVEEkDFXkQkAVTsRUQSQMVeRCQBVOxFRBJAxV5EJAFU7EVEEkDFXkQk\nAVTsRUQSQMVeRCQBVOxFRBJAxV5EJAFU7EVEEkDFXkQkAVTsRUQSQMVeRCQBVOxFRBJAxV5EJAFU\n7EVEEkDFXkQkAVTsRUQSQMVeRHLu4Yehc2eYNSt2kuRSsReRnHrlFRg+HC65BM45B26+Gdxjp0oe\n88hfdTPz2BlEJDe2boU+fWDYMLjsMlizBs4/H9q1g7vvhn33jZ2weJkZ7m6Zbq+RvYjkzBVXwGGH\nwYgR4X6HDjBzJrRvD8ceC/PnR42XKCr2IpIT990Hjz8Od94JVmX8uddeMG4c/OY30K8f3H672jr5\noDaOiGTd8uVw4omh2PfoUft2K1bAeefBkUfC+PGw9975y1js1MYRkag++SQU8F/9qu5CD2GGzvPP\nQ8uWcPzx8NJL+cmYRGmLvZmVm9kyM1tpZlfX8PxgM1toZovM7F9m1j3TfUWk9Fx2GXTrFmbfZKJ5\nc7jjDhg1Ck4+GSZOzG2+pKqzjWNmTYDlwGnABmAuMMjdl1bZpjewxN0/MLNyYLS798pk39T+auOI\nlIiJE0Mvfu5caNWq/vu/9FL4q6BPH/j976FFi+xnLBXZbuP0BFa5+xp33w5MBgZU3cDdZ7n7B6m7\ns4GDM91XRErHSy/BlVfCgw82rNADHHVU+EWxdSv06hV6+pId6Yp9O2BdlfvrU4/V5n+BRxu4r4gU\nqc2bw4j8hhtCwW6MVq3g3nvh0kuhb1944IHsZEy6pmmez7i/YmYnAxcDfeu77+jRoz+7XVZWRllZ\nWaa7ikhk7vC970Hv3jB0aHZe0yycdXv88eGXyMyZcOONYdpmUlVUVFBRUdHg/dP17HsRevDlqfuj\ngEp3/2217boDU4Byd19Vz33VsxcpYnfcEebNz56dmx77++/DxRfDunVhlN+xY/bfoxhlu2c/D+hk\nZh3MbE9gIDCt2hseQij0F+4s9JnuKyLFbcECuPba0KfP1cHUffeFv/4VBg8OffxpqiINUmcbx913\nmNkIYAbQBLjL3Zea2fDU8+OBXwD7AbdZOE1uu7v3rG3fHH4uIpJHH34YWixjx0KXLrl9LzP40Y/g\nhBPgggvguefCPP5mzXL7vqVEZ9CKSL25w8CBsP/+YbmDfHr7bbjoovDLZvJkOPjg9PuUIp1BKyI5\nd+utsHIljBmT//f+r/+Cv/8dTj8dvvENrauTKY3sRaRe5s0LC5jNmgWHHx4vh3uY5jluHJx6arwc\nsWhkLyI58957YT36226LW+gh9PFHjoTf/S5ujmKhkb2IZMQdvvlNOOSQMJouBFu2hDXyn3km9weJ\nC41G9iKSE2PGwMaN4SzZQtG8eTihK8axg2Kjkb2IpDVrVrh+7OzZYSRdSN54I4zqV64MB2+TQiN7\nEcmqd94Jc9vvuKPwCj1AmzZw7rnh4idSO43sRaRWlZVw9tnQtWthtW+qW7wYzjgjXNA8KevnaGQv\nIllz/fVhbZr/+7/YSerWrVv4mDw5dpLCpZG9iNTo2WfDNMt584rjLNXHHoNrroEXX9z9AuelSiN7\nEWm0N9+E//kfuOee4ij0AGeeCdu3w9NPx05SmFTsRWQ3lZVhhckhQ6C8PHaazO1cLE0nWdVMbRwR\n2U1FBYwYEZYvbpru8kYFJkknWamNIyKNMnVqWNGy2Ao97DrJauzY2EkKj0b2IvIZdzj00FDwu3eP\nnaZhknKSlUb2ItJgixeH3ne3brGTNFybNmENH51ktTsVexH5zCOPwIABxT91ceRIuOUW+M9/Yicp\nHCr2IvKZqVPDGjjFrls3OPJIuP/+2EkKh4q9iADw2muwdi307Rs7SXZccUWYhqlDgoGKvYgAYVT/\njW8U5yycmpx5JmzbFqaSioq9iKSUSgtnpz320JWsqtLUSxHhvffgy1+GTZugZcvYabJny5bwec2c\nCV/5Suw02aWplyJSb48+CiefXFqFHnSSVVUq9iLy2ZTLUvSDH8B994WLsCSZ2jgiCbd1K7RtG844\nPeCA2GlyY9gw6NQJrr02dpLsURtHROrln/8M89JLtdBDOFD7hz+E2TlJpWIvknClNgunJt276yQr\ntXFEEqyyEtq1C7NVDj88dprcmj49tHFeeKH4l4MAtXFEpB7mzIHWrUu/0EM4yWrr1uSeZKViL5Jg\npTwLp7qdJ1ndfHPsJHGojSOSYF26wKRJcPzxsZPkx86TrJ57Djp3jp2mcdTGEZGMLFsGmzfDscfG\nTpI/zZvD8OEwZkzsJPmnYi+SUFOnQv/+ob2RJJdeGk6yevfd2EnyK2H/zSKyUxKmXNakbdvweSft\nSlbq2Ysk0OuvwxFHhOu17rln7DT5t2gR9OsHq1cX7+ef9Z69mZWb2TIzW2lmV9fwfBczm2VmW83s\nx9WeW2Nmi8zsRTObk2koEcmtv/0NysuLt9A1Vvfu4ZfdAw/ETpI/dRZ7M2sC/AEoB7oCg8zsiGqb\nvQNcBtxYw0s4UObuPdy9ZxbyikgWJGnKZW2SdiWrdCP7nsAqd1/j7tuBycBu3yLu/pa7zwO21/Ia\nJXCumkjp+OijcMZsv36xk8RVXh6mYj7zTOwk+ZGu2LcD1lW5vz71WKYceNLM5pnZJfUNJyLZN2MG\n9O4N++wTO0lce+wBP/pRck6ySlfsG/sHTl937wH0Ay41s/9u5OuJSCMldRZOTb79bZg1C1asiJ0k\n99JdWngD0L7K/faE0X1G3H1T6t+3zOxhQltoZvXtRo8e/dntsrIyysrKMn0LEamH7dvhH/+A3/wm\ndpLC0KJFOMlq7Fi45ZbYaepWUVFBRSMW9qlz6qWZNQWWA6cCG4E5wCB3X1rDtqOBj9z9ptT9FkAT\nd//IzFoCjwO/dPfHq+2nqZciefLUUzBqVFgATYJNm6BrV3jlFdh//9hpMpfVqZfuvgMYAcwAlgD3\nu/tSMxtuZsNTb9jWzNYBI4GfmdlrZtYKaAvMNLMFwGzg79ULvYjkl1o4n3fggWFm0h//GDtJbumk\nKpGEcA+LgE2fHi7kIbssXAhnnVVcJ1lpITQRqdGCBbDXXqFlIbs7+ujSP8lKxV4kIXaeSFUKV2nK\nhZ1r3Zdqo0HFXiQh1K+vW79+8PHH8OyzsZPkhoq9SAKsXg0bN4aTqaRmO0+yGjcudpLcULEXSYBp\n0+Dss6FJk9hJCtugQWF66ttvx06SfSr2IgnwyCNq4WRin33g618PFzcpNSr2IiXunXfghRfgtNNi\nJykOQ4fCxImxU2Sfir1IifvHP+DUU8P1VyW9U04JF3d56aXYSbJLxV6kxGnt+vpp0gQuuqj0Rvc6\ng1akhG3ZEq65+uqr0Lp17DTFY/lyKCuDdeugabrlIiPRGbQi8pknn4QePVTo6+srX4EOHeDxElrN\nS8VepITpRKqGGzIE7rkndorsURtHpER9+ikcdBA8/zx07Bg7TfF5773wdVu9GvbbL3aaz1MbR0SA\nUOTbtlWhb6j99oMzz4TJk2MnyQ4Ve5ESpVk4jVdKc+5V7EVKkLvOms2G00+H116DZctiJ2k8FXuR\nErR0KfznP2EmjjRc06Zw4YWlMbpXsRcpQVOnau36bBkyBCZNCge8i5mKvUgJUgsne448Mlyn9qmn\nYidpHBV7kRKzcSOsXAlf+1rsJKWjFObcq9iLlJhp08LFs5s1i52kdAwaBI8+Ch98EDtJw6nYi5QY\nTbnMvtatw8qhxXxBchV7kRLy4Yfw739DeXnsJKVn6NDibuWo2IuUkOnT4cQTYe+9YycpPeXlsGpV\nOB5SjFTsRUqIFj7LnWbNYPDg4p1zr4XQRErEtm3Qpk04oapt29hpStPCheHC7WvWwB6Rh8paCE0k\noZ55Brp0UaHPpaOPDgdrn346dpL6U7EXKRE6kSo/hgwpzlaO2jgiJcAd2rcPV6bq0iV2mtL25pvQ\nuXO4ZGHMA+Fq44gk0Pz50KqVCn0+fOlLcNJJ8NBDsZPUj4q9SAnQiVT5VYxz7lXsRUqAplzm19e/\nDkuWwKuvxk6SORV7kSL3yivw1ltwwgmxkyTHnnuG9XL+9KfYSTKnYi9S5KZOhf7948/7TpohQ0Kx\nr6yMnSQz+vYQKXLq18dxzDHQsiXMnBk7SWZU7EWK2Ouvw6JFYUVGyS+z4ppzn7bYm1m5mS0zs5Vm\ndnUNz3cxs1lmttXMflyffUWkcSZNgm99C77whdhJkmnwYHj4Yfj449hJ0quz2JtZE+APQDnQFRhk\nZkdU2+wd4DLgxgbsKyIN5A4TJsCwYbGTJNeBB0KfPjBlSuwk6aUb2fcEVrn7GnffDkwGdusOuvtb\n7j4P2F7ffUWk4WbPDhfB7ts3dpJkK5Y59+mKfTtgXZX761OPZaIx+4pIGnffHUb1lvEJ85ILZ58N\nCxbA2rWxk9QtXbFvzKI1WvBGJEc++QQefBAuuih2EvnCF2DgwHD8pJA1TfP8BqB9lfvtCSP0TGS8\n7+jRoz+7XVZWRllZWYZvIZJMU6ZAr17QTn8rF4ShQ8PB2p/+NHd/aVVUVFBRUdHg/etc9dLMmgLL\ngVOBjcAcYJC7L61h29HAR+5+U3321aqXIvV36qnwve/BeefFTiIQDpZ37Qp33pm/Yyj1XfWyzpG9\nu+8wsxHADKAJcJe7LzWz4annx5tZW2Au8EWg0swuB7q6++aa9m3YpyUiO61eHebW9+8fO4nsZLbr\nQG2hHjDXevYiRWb0aHj3XRg3LnYSqWrDBujWDdavhxYtcv9+Ws9epIRVVobRo+bWF5527aBnz7B8\nRSFSsRcpIk8/DfvtBz16xE4iNSnk5RNU7EWKyM659VKYzjkH5s4NrZxCo569SJF4/33o0CGsX9+6\ndew0UpvvfhcOPRSuuSa376OevUiJuv9+OP10FfpCt3NWTqGNYVXsRYqEFj0rDr17hwPpc+bETrI7\nFXuRIrBkSegDn3FG7CSSzs517gttcTT17EWKwFVXQdOm8Otfx04imXjttTBjasOG3F1rQD17kRKz\nfXtYZEstnOJxyCGh2E+bFjvJLir2IgVu+nQ4/HDo3Dl2EqmPQptzr2IvUuB0YLY4nXsu/PvfsGlT\n7CSBir1IAXvzTXjmGTj//NhJpL5atoRvfhP+/OfYSQIVe5ECdu+9MGAA7L137CTSEIU0517FXqRA\n6YLixe/EE8NVxebPj51ExV6kYM2bB1u3wte+FjuJNNQee4QDtXfeGTuJ5tmLFKzvfz8sm/uzn8VO\nIo2xaVO4itWKFXDAAdl73frOs1exFylAW7bAwQfDggXQvn367aWwXXJJ+MVd5XLbjaaTqkRKwCOP\nwHHHqdCXih//GG69NfTvY1GxFylAOjBbWrp0CQukxVwvR20ckQKzdi0ce2xY+CxX66pI/j33XDhY\nu2IFNGnS+NdTG0ekyE2cCAMHqtCXmr59oU0bmDIlzvur2IsUEF1QvHSZhdVLb7ghzklWKvYiBeTZ\nZ6FVq9DGkdLTv3+4vOSzz+b/vVXsRQrIzgOzlnEnVopJkyZhZs4NN+T/vXWAVqRAfPhhWAd95crs\nnnwjhWXLFujYEZ56Co48suGvowO0IkXqgQfglFNU6Etd8+YwYgTceGN+31cje5EC0acPjBoFZ58d\nO4nk2jvvQKdOsHhxOLO2ITSyFylCy5bB6tXQr1/sJJIPrVvDhRfCuHH5e0+N7EUKwNVXh+l4118f\nO4nky5o1YdbV6tXwxS/Wf3+N7EWKzI4duqB4EnXoAGecAX/8Y37eT8VeJLIZM+DLX4YjjoidRPLt\nqqtg7FjYti3376ViLxKZFj1LrmOOgc6dYfLk3L+XevYiEb31VpiVsXYt7LNP7DQSw4wZcOWVsGhR\n/U6mU89epIj8+c9hqqUKfXKdcUa4fOFjj+X2fVTsRSJxh7vvVgsn6czCyD7XSyio2ItE8uKLYYmE\nsrLYSSS2Cy4Iy2TMn5+790hb7M2s3MyWmdlKM7u6lm3GpZ5faGY9qjy+xswWmdmLZjYnm8FFit2E\nCTB0aPgTXpKtWTMYOTK3o/s6D9CaWRNgOXAasAGYCwxy96VVtjkLGOHuZ5nZCcBYd++Vem41cKy7\nv1vHe+gArSTO1q3hguLz54dplyIffRQWSJs7N/ybTrYP0PYEVrn7GnffDkwGBlTbpj8wEcDdZwP7\nmlmbqpkyDSOSFFOnwle/qkIvu+y9N3znO/C73+Xm9dMV+3bAuir316cey3QbB540s3lmdkljgoqU\nEh2YlZr88Idw771hobRsa5rm+Uz7K7WN3k90941mdgDwhJktc/eZ1TcaPXr0Z7fLysoo0xErKWHr\n1sGcOfDww7GTSKE56CA491y49Vb4+c93f66iooKKiooGv3a6nn0vYLS7l6fujwIq3f23Vba5Hahw\n98mp+8uAk9z9jWqvdR2w2d1vqva4evaSKL/6VSj4t98eO4kUoqVL4eSTwwJpzZvXvl22e/bzgE5m\n1sHM9gQGAtOqbTMNuCj15r2A9939DTNrYWZ7px5vCZwBLM40mEgp0tx6SeeII+D442HixOy+bp3F\n3t13ACOAGcAS4H53X2pmw81seGqbR4FXzWwVMB74QWr3tsBMM1sAzAb+7u6PZze+SHGZORP22gt6\n9oydRArZVVfBTTfBp59m7zW1No5IHg0bFq47euWVsZNIIXOH3r3hJz8JPfya1LeNo2IvkierVoUR\n/ZIl0LZt7DRS6P7613CS1axZNS+QpoXQRArQ1q1w/vnwy1+q0EtmzjknTMF87rnsvJ5G9iJ58IMf\nhOWMH3igfsvYSrLddhtMnw7Tqk+LQW0ckYJz//1w7bXwwgtayljqZ8uWcPnCiorPX8lMbRyRArJy\nJYwYAQ8+qEIv9de8OVx6aZiZ01ga2YvkyJYtYUbFd78b2jgiDfH22+HShS+/DAceuOtxtXFECsTw\n4fD+++H6ourTS2OMGBEWSvv1r3c9pmIvUgD+8he47rqwhPEXvxg7jRS7V18N03ZXrw5FH9SzF4lu\n+XK4/PLQp1ehl2w49FA49VS4886Gv4ZG9iJZ9Mkn0KtXOKg2fHjsNFJK5s0LZ9O+8kq4spVG9iIR\nXX45HHVUOCgrkk3HHQeHHRam8jaEir1Iltx7Lzz7LIwfrwOykhtXXRWWUGhIM0TFXiQLli4NF4x+\n8MFdB9BEsq1fP6ishCeeqP++KvYijfTJJ3DeeWFaXPfusdNIKTMLK6becEMD9o19cFQHaKXYXXwx\nbNsGkyapfSO5t21bmJ2zYYMO0IrkzcSJYQna229XoZf82HPPsM59fWlkL9JAL78MZWXw9NNhBo5I\nPmnqpUgefPxxWJ/++utV6KU4aGQvUk/uMHRouH3PPWrfSBz1Hdk3zWUYkVJ0993hbMY5c1TopXho\nZC9SD4sXwymnwDPPQNeusdNIkqlnL5IjmzeHPv1NN6nQS/HRyF4kA+7w7W+HaW8TJsROI6KevUhO\n3HUXLFwIs2fHTiLSMBrZi6SxaFFYS3zmTOjSJXYakUA9e5Es+uijsO7NzTer0Etx08hepBbuMHgw\ntGwJd9wRO43I7tSzF8mS8ePDkgjPPx87iUjjqdiLVFFZCY89Fto2S5fCU09B8+axU4k0noq9CGFN\n+kmTYMyYUNxHjoSBA8NUS5FSoGIvibZxI9xyS+jJ9+4Nt90GJ52kZRCk9Gg2jiTSCy+Ek6SOOgo+\n/BD+9S+YOjUsWaxCL6VIxV4S49NPdxX0AQOgWzd45RX4/e+hU6fY6URyS20cKXmbN4eliMeMgf33\nhyuugG99C5o1i51MJH9U7KVkrVsXRu0TJoTR/MSJ0KeP2jSSTGnbOGZWbmbLzGylmV1dyzbjUs8v\nNLMe9dlXJNvmzIELLoCjj4bt22HuXHjoIejbV4VekqvOYm9mTYA/AOVAV2CQmR1RbZuzgMPdvRPw\nXeC2TPctJhUVFbEjZCSpOXfs2FXQzz8fevaE1avDfPmOHRv+usXw9SyGjKCcsaUb2fcEVrn7Gnff\nDkwGBlTbpj8wEcDdZwP7mlnbDPctGsXyDVBqOd3hgw9g+fJwwZD774exY2HUKBg2DM46C445Btq0\nCYV95EhYtSr05ffZJ385YyqGjKCcsaXr2bcD1lW5vx44IYNt2gEHZbCvJNSOHbB2Lbz+evqPZs2g\nbdtdH23ahH9PPHHXYwceCAcdFPuzEilc6Yp9piuUFUwn9Mknw8gv25Yvh/nzs/+62eQOK1aE66NW\nf7ymbTN9rLIyTFus/lHb47V97Nx+xw7YuhX+9Kfdi3jbtmE65Omn717YW7bMztdHJMnqXPXSzHoB\no929PHV/FFDp7r+tss3tQIW7T07dXwacBHRMt2/qcS15KSLSANlc9XIe0MnMOgAbgYHAoGrbTANG\nAJNTvxzed/c3zOydDPatV1gREWmYOou9u+8wsxHADKAJcJe7LzWz4annx7v7o2Z2lpmtAj4GhtW1\nby4/GRERqVn0i5eIiEjuRVsbx8zam9nTZvaymb1kZj+MlSUdM2tiZi+a2d9iZ6mNme1rZg+Z2VIz\nW5JqqRUcMxuV+j9fbGZ/MbO9YmcCMLMJZvaGmS2u8tj+ZvaEma0ws8fNbN+YGVOZasp5Q+r/faGZ\nTTGzLEw6bZyaclZ57sdmVmlm+8fIVi1LjTnN7LLU1/QlM/ttbfvnSy3/7z3NbE6qNs01s+Preo2Y\nC6FtB0a6+5FAL+DSAj7p6nJgCZnPTophLPCoux8BdAcKrmWWOn5zCXCMu3cjtPcuiJmpirsJJwBW\ndQ3whLt3Bp5K3Y+tppyPA0e6+9HACmBU3lN9Xk05MbP2wOnA2rwnqtnncprZyYTzh7q7+1HAjTGC\nVVPT1/N64Ofu3gP4Rep+raIVe3d/3d0XpG5vJhSngpspbWYHA2cBd1JAU0yrSo3k/tvdJ0A4XuLu\nH0SOVZMPCb/kW5hZU6AFsCFupMDdZwLvVXv4sxMGU/+ek9dQNagpp7s/4e6VqbuzgYPzHqyaWr6e\nAL8DfpLnOLWqJef3gV+nTgbF3d/Ke7Bqasm5Cdj5V9y+pPlZKogljlMjvh6Eb9RCczNwFVCZbsOI\nOgJvmdndZvaCmd1hZi1ih6rO3d8FbgJeI8zQet/dn4ybqk5t3P2N1O03gDYxw2ToYuDR2CFqYmYD\ngPXuvih2ljQ6AV8zs+fNrMLMjosdqBbXADeZ2WvADaT5iy56sTezVsBDwOWpEX7BMLNvAG+6+4sU\n6Kg+pSlwDHCrux9DmBVVCC2H3ZjZYcCPgA6Ev+JamdngqKEy5GEmQyG38TCznwLb3P0vsbNUlxp8\nXAtcV/XhSHHSaQrs5+69CAO9ByLnqc1dwA/d/RBgJDChro2jFnszawb8FbjX3R+JmaUWfYD+ZrYa\nuA84xcz+FDlTTdYTRkxzU/cfIhT/QnMc8G93f8fddwBTCF/jQvVGap0nzOxA4M3IeWplZkMJ7cZC\n/eV5GOGX/MLUz9PBwHwz+1LUVDVbT/jeJPUzVWlmreNGqlFPd384dfshwnpktYo5G8cIv5mWuPuY\nWDnq4u7Xunt7d+9IOJD4T3e/KHau6tz9dWCdmXVOPXQa8HLESLVZBvQys+ap///TCAe+C9U0YEjq\n9hCgEAckmFk5YQQ6wN23xs5TE3df7O5t3L1j6udpPeFAfSH+An0EOAUg9TO1p7u/EzdSjVaZ2Ump\n26cQDs7Xzt2jfAAnEvrgC4AXUx/lsfJkkPckYFrsHHXkOxqYCywkjEr2iZ2plpw/IfwiWkw46Nks\ndqZUrvsIxxG2ERbwGwbsDzyZ+iF6HNi3AHNeDKwkzG7Z+XN0awHl/M/Or2e1518F9i/EnEAzYFLq\ne3Q+UFZAOat+fx5HOM65AJgF9KjrNXRSlYhIAkQ/QCsiIrmnYi8ikgAq9iIiCaBiLyKSACr2IiIJ\noGIvIpIAKvYiIgmgYi8ikgD/D23c8yqBMTUZAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import matplotlib.pyplot as plt\n", - "xs = list(range(3,19))\n", - "ps = best_attr_pmf.Probs(xs)\n", - "plt.plot(xs, ps);" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Mixtures {#mixture}\n", - "\n", - "\n", - "Let’s do one more example from *Dungeons & Dragons*.\n", - "Suppose I have a box of dice with the following inventory:\n", - "\n", - " 5 4-sided dice\n", - " 4 6-sided dice\n", - " 3 8-sided dice\n", - " 2 12-sided dice\n", - " 1 20-sided die\n", - "\n", - "I choose a die from the box and roll it. What is the distribution of the\n", - "outcome?\n", - "\n", - "If you know which die it is, the answer is easy. A die with\n", - "n sides yields a uniform distribution from 1 to\n", - "n, including both.\n", - "\n", - "But if we don’t know which die it is, the resulting distribution is a\n", - "**mixture** of uniform distributions with different bounds.\n", - "In general, this kind of mixture does not fit any simple mathematical\n", - "model, but it is straightforward to compute the distribution in the form\n", - "of a PMF.\n", - "\n", - "As always, one option is to simulate the scenario, generate a random\n", - "sample, and compute the PMF of the sample. This approach is simple and\n", - "it generates an approximate solution quickly. But if we want an exact\n", - "solution, we need a different approach.\n", - "\n", - "Let’s start with a simple version of the problem where there are only\n", - "two dice, one with 6 sides and one with 8. We can make a Pmf to\n", - "represent each die:" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "d6 = Die(6)\n", - "d8 = Die(8)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Then we create a Pmf to represent the mixture:" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1 0.14583333333333334\n", - "2 0.14583333333333334\n", - "3 0.14583333333333334\n", - "4 0.14583333333333334\n", - "5 0.14583333333333334\n", - "6 0.14583333333333334\n", - "7 0.06250000000000001\n", - "8 0.06250000000000001\n" - ] - } - ], - "source": [ - "mix = thinkbayes.Pmf()\n", - "for die in [d6, d8]:\n", - " for outcome, prob in die.Items():\n", - " mix.Incr(outcome, prob)\n", - "mix.Normalize()\n", - "mix.Print()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The first loop enumerates the dice; the second enumerates the outcomes\n", - "and their probabilities. Inside the loop, `Pmf.Incr` adds up\n", - "the contributions from the two distributions.\n", - "\n", - "This code assumes that the two dice are equally likely. More generally,\n", - "we need to know the probability of each die so we can weight the\n", - "outcomes accordingly.\n", - "\n", - "First we create a Pmf that maps from each die to the probability it is\n", - "selected:" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "15" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "pmf_dice = thinkbayes.Pmf()\n", - "pmf_dice.Set(Die(4), 5)\n", - "pmf_dice.Set(Die(6), 4)\n", - "pmf_dice.Set(Die(8), 3)\n", - "pmf_dice.Set(Die(12), 2)\n", - "pmf_dice.Set(Die(20), 1)\n", - "pmf_dice.Normalize()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next we need a more general version of the mixture algorithm:" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "mix = thinkbayes.Pmf()\n", - "for die, weight in pmf_dice.Items():\n", - " for outcome, prob in die.Items():\n", - " mix.Incr(outcome, weight*prob)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now each die has a weight associated with it (which makes it a weighted\n", - "die, I suppose). When we add each outcome to the mixture, its\n", - "probability is multiplied by weight." - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFARJREFUeJzt3X+s3fV93/Hna5ewljJKp0kwzJ2MiivwlKqw1UNh0Y4i\nNBmrwplW1bEa0fFH4j9qQitUESIt3PvXNJSsFFlCbuNEsNBZE+kyV+LWbUROtqaM2MVAAnaHq1q1\nTU0YKW1hYrLl9/4435jjY997zv3hewmf50Oy+H4/38/nnLe/fPW6H3/u+Z5vqgpJ0gfb31vrAiRJ\nl55hL0kNMOwlqQGGvSQ1wLCXpAYY9pLUgLFhn2RzkiNJXk3ywEWO35Tk2STvJrl/5NiDSV5O8t0k\nv5vk769k8ZKkySwY9kmmgF3AZmAjsD3JzSPd3gTuBb4wMnY98Cng1qr6MDAFfGJFqpYkLcq4mf0m\n4GhVHauq08BeYOtwh6p6o6oOAqdHxv5t13ZFksuAK4CTK1O2JGkxxoX9OuD40P6Jrm2sqvoB8EXg\nL4HXgLeq6htLKVKStDzjwn7J36WQ5KeBXwPWA9cBVyb55aW+niRp6S4bc/wkMD20P81gdj+Jfw78\nSVW9CZDk94CPAE8Od0ril/NI0hJUVSbtO25mfxDYkGR9ksuBbcC+efqOvukR4LYkP54kwB3AK/MU\n7J8qHnrooTWv4f3yx3PhufBcLPxnsRac2VfVmSQ7gf0MPk2zp6oOJ9nRHd+d5FrgAHAVcDbJfcDG\nqnoxyRMMfmCcBZ4HfnvRFUqSlm3cMg5VNQfMjbTtHto+xflLPcP9HgYeXmaNkqRl8g7a95Fer7fW\nJbxveC7e47l4j+di6bKUtZ8VLSCpta5Bkn7UJKFW8Be0kqQPAMNekhpg2EtSAwx7SWqAYS9JDTDs\nJakBhr0kNcCwl6QGGPaS1ICx342z1j429+1F9X/mzttXZcxy3kuSVpsze0lqgGEvSQ0w7CWpAYa9\nJDVgbNgn2ZzkSJJXkzxwkeM3JXk2ybtJ7h85dnWSp5IcTvJKkttWsnhJ0mQW/DROkilgF4Pnx54E\nDiTZV1WHh7q9CdwLfPwiL/FbwNNV9YtJLgN+YmXKliQtxriZ/SbgaFUdq6rTwF5g63CHqnqjqg4C\np4fbk/wk8NGq+nLX70xV/c3KlS5JmtS4sF8HHB/aP9G1TeIG4I0kX0nyfJLfSXLFUoqUJC3PuJuq\nlvO8wMuAW4GdVXUgySPAZ4HPj3acmZk5t93r9XzOpCSN6Pf79Pv9JY8fF/Yngemh/WkGs/tJnABO\nVNWBbv8pBmF/geGwlyRdaHQiPDs7u6jx45ZxDgIbkqxPcjmwDdg3T9/zHnxbVaeA40l+pmu6A3h5\nUdVJklbEgjP7qjqTZCewH5gC9lTV4SQ7uuO7k1wLHACuAs4muQ/YWFVvM/iUzpPdD4o/B+65hH8X\nSdI8xn4RWlXNAXMjbbuHtk9x/lLPcL8XgZ9fZo2SpGXyDlpJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY\n9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgPGhn2SzUmO\nJHk1yQMXOX5TkmeTvJvk/oscn0pyKMnvr1TRkqTFWTDsk0wBu4DNwEZge5KbR7q9yeDxg1+Y52Xu\nA14BanmlSpKWatzMfhNwtKqOVdVpYC+wdbhDVb1RVQeB06ODk1wPbAG+xMgDySVJq2dc2K8Djg/t\nn+jaJvWbwG8AZxdZlyRpBY174PiSl16S/ALw/ao6lKS3UN+ZmZlz271ej15vwe6S1Jx+v0+/31/y\n+HFhfxKYHtqfZjC7n8RHgLuSbAF+DLgqyRNVdfdox+GwlyRdaHQiPDs7u6jx45ZxDgIbkqxPcjmw\nDdg3T9/z1uSr6nNVNV1VNwCfAJ65WNBLki69BWf2VXUmyU5gPzAF7Kmqw0l2dMd3J7kWOABcBZxN\nch+wsareHn25lS9fkjSJccs4VNUcMDfStnto+xTnL/Vc7DW+BXxriTVKkpbJO2glqQGGvSQ1wLCX\npAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lq\ngGEvSQ2YKOyTbE5yJMmrSR64yPGbkjyb5N0k9w+1Tyf5ZpKXk3wvyWdWsnhJ0mTGPqkqyRSwC7iD\nwQPIDyTZV1WHh7q9CdwLfHxk+Gng16vqhSRXAn+a5I9GxkqSLrFJZvabgKNVdayqTgN7ga3DHarq\njao6yCDch9tPVdUL3fbbwGHguhWpXJI0sUnCfh1wfGj/RNe2KEnWA7cAzy12rCRpecYu4wC13Dfp\nlnCeAu7rZvjnmZmZObfd6/Xo9XrLfUtJ+kDp9/v0+/0lj58k7E8C00P70wxm9xNJ8iHga8BXq+rr\nF+szHPaSpAuNToRnZ2cXNX6SZZyDwIYk65NcDmwD9s3TN+ftJAH2AK9U1SOLqkyStGLGzuyr6kyS\nncB+YArYU1WHk+zoju9Oci1wALgKOJvkPmAj8HPAJ4GXkhzqXvLBqvqDS/B3kSTNY5JlHKpqDpgb\nads9tH2K85d6fuiP8cYtSVpzBrEkNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9\nJDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNGBv2STYnOZLk1SQPXOT4TUmeTfJukvsXM1aS\ntDoWDPskU8AuYDODJ09tT3LzSLc3gXuBLyxhrCRpFYyb2W8CjlbVsao6DewFtg53qKo3quogcHqx\nYyVJq2Nc2K8Djg/tn+jaJrGcsZKkFTQu7GsZr72csZKkFTTugeMnOf9B4tMMZuiTmHjszMzMue1e\nr0ev15vwLSSpDf1+n36/v+Tx48L+ILAhyXrgNWAbsH2evlnq2OGwlyRdaHQiPDs7u6jxC4Z9VZ1J\nshPYD0wBe6rqcJId3fHdSa4FDgBXAWeT3AdsrKq3LzZ2UdVJklbEuJk9VTUHzI207R7aPsX5yzUL\njpUkrT7voJWkBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg\n2EtSAwx7SWqAYS9JDTDsJakBhr0kNWDsw0uSbAYeYfC0qS9V1X+8SJ9HgTuB/wv8u6o61LU/CHwS\nOAt8F7inqv7fypX/o+ljc99eVP9n7rx9WeMkacGZfZIpYBewGdgIbE9y80ifLcCNVbUB+DTwWNe+\nHvgUcGtVfZjBD4tPrHD9kqQJjFvG2QQcrapjVXUa2AtsHelzF/A4QFU9B1yd5Brgb4HTwBVJLgOu\nAE6uZPGSpMmMC/t1wPGh/RNd29g+VfUD4IvAXwKvAW9V1TeWV64kaSnGhX1N+Dq5oCH5aeDXgPXA\ndcCVSX55UdVJklbEuF/QngSmh/anGczcF+pzfdfWA/6kqt4ESPJ7wEeAJ0ffZGZm5tx2r9ej1+tN\nUrskNaPf79Pv95c8flzYHwQ2dL9sfQ3YBmwf6bMP2AnsTXIbg+Wa15P8GfDvk/w48C5wB/Cdi73J\ncNhLki40OhGenZ1d1PgFw76qziTZCexn8GmaPVV1OMmO7vjuqno6yZYkR4F3gHu6Yy8keYLBD4yz\nwPPAby+qOknSihj7OfuqmgPmRtp2j+zvnGfsw8DDyylQkrR83kErSQ0w7CWpAYa9JDXAsJekBhj2\nktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWrA2LBP\nsjnJkSSvJnlgnj6PdsdfTHLLUPvVSZ5KcjjJK91jCyVJq2zBsE8yBewCNgMbge1Jbh7pswW4sao2\nAJ8GHhs6/FvA01V1M/CzwOEVrF2SNKFxM/tNwNGqOlZVp4G9wNaRPncBjwNU1XPA1UmuSfKTwEer\n6svdsTNV9TcrW74kaRLjwn4dcHxo/0TXNq7P9cANwBtJvpLk+SS/k+SK5RYsSVq8cQ8crwlfJxcZ\ndxlwK7Czqg4keQT4LPD50cEzMzPntnu9Hr1eb8K3laQ29Pt9+v3+ksePC/uTwPTQ/jSDmftCfa7v\n2gKcqKoDXftTDML+AsNhL0m60OhEeHZ2dlHjxy3jHAQ2JFmf5HJgG7BvpM8+4G6A7tM2b1XV61V1\nCjie5Ge6fncALy+qOknSilhwZl9VZ5LsBPYDU8CeqjqcZEd3fHdVPZ1kS5KjwDvAPUMvcS/wZPeD\n4s9HjkmSVsm4ZRyqag6YG2nbPbK/c56xLwI/v5wCJUnL5x20ktQAw16SGmDYS1IDDHtJaoBhL0kN\nMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGjA37JJuT\nHEnyapIH5unzaHf8xSS3jBybSnIoye+vVNGSpMVZMOyTTAG7gM3ARmB7kptH+mwBbqyqDcCngcdG\nXuY+4BWgVqpoSdLijJvZbwKOVtWxqjoN7AW2jvS5C3gcoKqeA65Ocg1AkuuBLcCXgKxk4ZKkyY0L\n+3XA8aH9E13bpH1+E/gN4OwyapQkLdO4B45PuvQyOmtPkl8Avl9Vh5L0Fho8MzNzbrvX69HrLdhd\nkprT7/fp9/tLHj8u7E8C00P70wxm7gv1ub5r+7fAXd2a/o8BVyV5oqruHn2T4bDXyvrY3LcX1f+Z\nO29f0hhJl9boRHh2dnZR48ct4xwENiRZn+RyYBuwb6TPPuBugCS3AW9V1amq+lxVTVfVDcAngGcu\nFvSSpEtvwZl9VZ1JshPYD0wBe6rqcJId3fHdVfV0ki1JjgLvAPfM93IrWbgkaXLjlnGoqjlgbqRt\n98j+zjGv8S3gW0spUJK0fN5BK0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9J\nDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqwERhn2RzkiNJXk3ywDx9Hu2Ov5jklq5t\nOsk3k7yc5HtJPrOSxUuSJjM27JNMAbuAzcBGYHuSm0f6bAFurKoNwKeBx7pDp4Ffr6p/CtwG/Oro\nWEnSpTfJzH4TcLSqjlXVaWAvsHWkz13A4wBV9RxwdZJrugePv9C1vw0cBq5bseolSROZJOzXAceH\n9k90beP6XD/cIcl64BbgucUWKUlanrEPHAdqwtfKfOOSXAk8BdzXzfDPMzMzc2671+vR6/UmfEtJ\nakO/36ff7y95/CRhfxKYHtqfZjBzX6jP9V0bST4EfA34alV9/WJvMBz2kqQLjU6EZ2dnFzV+kmWc\ng8CGJOuTXA5sA/aN9NkH3A2Q5Dbgrap6PUmAPcArVfXIoiqTJK2YsTP7qjqTZCewH5gC9lTV4SQ7\nuuO7q+rpJFuSHAXeAe7pht8OfBJ4Kcmhru3BqvqDFf+bSJLmNckyDlU1B8yNtO0e2d95kXF/jDdu\nSdKaM4glqQGGvSQ1YKJlHGkSH5v79qL6P3Pn7Usas9rvJX0QOLOXpAYY9pLUAMNekhpg2EtSAwx7\nSWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgPGhn2SzUmOJHk1yQPz9Hm0O/5iklsW\nM1aSdOkt+K2XSaaAXcAdDJ4peyDJvqo6PNRnC3BjVW1I8i+Ax4DbJhmr8/31S8/zUz9761qX8b7w\nfjkX74dv5ZzvXKz2N4D+KJwLzW/czH4TcLSqjlXVaWAvsHWkz13A4wBV9RxwdZJrJxyrIW+9dGh8\np0Z4Lt7juXiP52LpxoX9OuD40P6Jrm2SPtdNMFaStArGhX1N+DpZbiGSpEtn3JOqTgLTQ/vTDGbo\nC/W5vuvzoQnGApCs3M+KpbzSUt/9UrzXsd/9yoq8z1LHvZ/On+fiPZ6L96zkuWjJuLA/CGxIsh54\nDdgGbB/psw/YCexNchvwVlW9nuTNCcZSVf5/kqRLbMGwr6ozSXYC+4EpYE9VHU6yozu+u6qeTrIl\nyVHgHeCehcZeyr+MJOniUjXpsrwk6UfVmt5B601X70lyLMlLSQ4l+c5a17Oaknw5yetJvjvU9g+T\n/FGS/53kD5NcvZY1rpZ5zsVMkhPdtXEoyea1rHG1JJlO8s0kLyf5XpLPdO3NXRsLnIuJr401m9l3\nN139GUM3XQHbW13qSfIXwD+rqh+sdS2rLclHgbeBJ6rqw13bw8D/qaqHu4nAT1XVZ9eyztUwz7l4\nCPi7qvpPa1rcKuvu17m2ql5IciXwp8DHGSwVN3VtLHAufokJr421nNl709WFmvxldVX9T+CvR5rP\n3azX/ffjq1rUGpnnXECD10ZVnaqqF7rtt4HDDO7Vae7aWOBcwITXxlqG/SQ3bLWkgG8kOZjkU2td\nzPvANVX1erf9OnDNWhbzPnBv991Te1pYthjVfarvFuA5Gr82hs7F/+qaJro21jLs/c3w+W6vqluA\nO4Ff7f45L6AGa40tXy+PATcAPwf8FfDFtS1ndXXLFl8D7quqvxs+1tq10Z2Lpxici7dZxLWxlmE/\nyQ1bzaiqv+r++wbw3xgsc7Xs9W6dkiT/GPj+GtezZqrq+9UBvkRD10aSDzEI+v9cVV/vmpu8NobO\nxVd/eC4Wc22sZdifu2EryeUMbrrat4b1rJkkVyT5B932TwD/GvjuwqM+8PYBv9Jt/wrw9QX6fqB1\ngfZD/4ZGro0Mbq3fA7xSVY8MHWru2pjvXCzm2ljTz9knuRN4hPduuvoPa1bMGkpyA4PZPAxudHuy\npXOR5L8A/wr4RwzWYD8P/HfgvwL/BDgG/FJVvbVWNa6Wi5yLh4Aeg3+mF/AXwI6hNesPrCT/Evgf\nwEu8t1TzIPAdGrs25jkXn2PwrQQTXRveVCVJDfCxhJLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakB\nhr0kNcCwl6QG/H+VZF9t35J1SQAAAABJRU5ErkJggg==\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "colors = thinkplot.Brewer.Colors()\n", - "thinkplot.Hist(mix, width=0.9, color=colors[4])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This shows the result. As expected, values 1 through 4\n", - "are the most likely because any die can produce them. Values above 12\n", - "are unlikely because there is only one die in the box that can produce\n", - "them (and it does so less than half the time).\n", - "\n", - "`thinkbayes`provides a function named\n", - "`MakeMixture` that encapsulates this algorithm, so we could\n", - "have written:\n", - "\n", - "```python```\n", - " mix = thinkbayes.MakeMixture(pmf_dice)\n", - "```\n", - "\n", - "We’ll use MakeMixture again in Chapters [prediction] and\n", - " [observer]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#Discussion\n", - "\n", - "Other than the odds form of Bayes’s theorem, this chapter is not\n", - "specifically Bayesian. But Bayesian analysis is all about distributions,\n", - "so it is important to understand the concept of a distribution well.\n", - "From a computational point of view, a distribution is any data structure\n", - "that represents a set of values (possible outcomes of a random process)\n", - "and their probabilities.\n", - "\n", - "We have seen two representations of distributions: Pmfs and Cdfs. These\n", - "representations are equivalent in the sense that they contain the same\n", - "information, so you can convert from one to the other. The primary\n", - "difference between them is performance: some operations are faster and\n", - "easier with a Pmf; others are faster with a Cdf.\n", - "\n", - "The other goal of this chapter is to introduce operations that act on\n", - "distributions, like `Pmf.__add__`, `Cdf.Max`, and\n", - "`thinkbayes.MakeMixture`. We will use these operations later,\n", - "but I introduce them now to encourage you to think of a distribution as\n", - "a fundamental unit of computation, not just a container for values and\n", - "probabilities." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Odds and Addends\n", + "\n", + "## Odds\n", + "\n", + "One way to represent a probability is with a number between 0 and 1, but\n", + "that’s not the only way. If you have ever bet on a football game or a\n", + "horse race, you have probably encountered another representation of\n", + "probability, called **odds**.\n", + "\n", + "You might have heard expressions like “the odds are three to one,” but\n", + "you might not know what that means. The **odds in favor**\n", + "of an event are the ratio of the probability it will occur to the\n", + "probability that it will not.\n", + "\n", + "So if I think my team has a 75% chance of winning, I would say that the\n", + "odds in their favor are three to one, because the chance of winning is\n", + "three times the chance of losing.\n", + "\n", + "You can write odds in decimal form, but it is most common to write them\n", + "as a ratio of integers. So “three to one” is written $3:1$.\n", + "\n", + "When probabilities are low, it is more common to report the **odds\n", + "against** rather than the odds in favor. For example, if I think\n", + "my horse has a 10% chance of winning, I would say that the odds against\n", + "are $9:1$.\n", + "\n", + "Probabilities and odds are different representations of the same\n", + "information. Given a probability, you can compute the odds like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.11111111111111112" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def Odds(p):\n", + " return p / (1-p)\n", + "\n", + "Odds(0.1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given the odds in favor, in decimal form, you can convert to probability\n", + "like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.09999999999999999" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def Probability(o):\n", + " return o / (o+1)\n", + "\n", + "Probability(1/9.)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you represent odds with a numerator and denominator, you can convert\n", + "to probability like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.1" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def Probability2(yes, no):\n", + " return float(yes) / (yes + no)\n", + "\n", + "Probability2(1, 9)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When I work with odds in my head, I find it helpful to picture people at\n", + "the track. If 20% of them think my horse will win, then 80% of them\n", + "don’t, so the odds in favor are $20:80$ or $1:4$." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.2" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Probability2(20, 80)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If the odds are $5:1$ against my horse, then five out of six people\n", + "think she will lose, so the probability of winning is $1/6$." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.16666666666666663" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "1 - Probability2(5, 1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##The odds form of Bayes’s theorem\n", + "\n", + "In Chapter [intro] I wrote Bayes’s theorem in the **probability\n", + "form**:\n", + "\n", + "$${{\\mathrm{p}(H|D)}} = \\frac{{{\\mathrm{p}(H)}}~{{\\mathrm{p}(D|H)}}}{{{\\mathrm{p}(D)}}}$$\n", + "\n", + "If we have two hypotheses, $A$ and $B$, we can write the ratio of\n", + "posterior probabilities like this:\n", + "\n", + "$$\\frac{{{\\mathrm{p}(A|D)}}}{{{\\mathrm{p}(B|D)}}} = \\frac{{{\\mathrm{p}(A)}}~{{\\mathrm{p}(D|A)}}}\n", + "{{{\\mathrm{p}(B)}}~{{\\mathrm{p}(D|B)}}}$$\n", + " \n", + "Notice that the normalizing constant, $\\mathrm{p}(D)$,\n", + "drops out of this equation.\n", + "\n", + "If $A$ and $B$ are mutually exclusive and collectively exhaustive, that\n", + "means ${{\\mathrm{p}(B)}} = 1 - {{\\mathrm{p}(A)}}$,\n", + "so we can rewrite the ratio of the priors, and the ratio of the\n", + "posteriors, as odds.\n", + "\n", + "Writing $\\mathrm{o}(A)$ for odds in favor of $A$, we get:\n", + "\n", + "$${{\\mathrm{o}(A|D)}} = {{\\mathrm{o}(A)}}~\\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", + "\n", + "In words, this says that the posterior odds are the prior odds times the\n", + "likelihood ratio. This is the **odds form** of Bayes’s\n", + "theorem.\n", + "\n", + "This form is most convenient for computing a Bayesian update on paper or\n", + "in your head. For example, let’s go back to the cookie problem:\n", + "\n", + "> Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla\n", + "> cookies and 10 chocolate cookies. Bowl 2 contains 20 of each.\n", + ">\n", + "> Now suppose you choose one of the bowls at random and, without\n", + "> looking, select a cookie at random. The cookie is vanilla. What is the\n", + "> probability that it came from Bowl 1?\n", + "\n", + "The prior probability is 50%, so the prior odds are $1:1$, or just 1.\n", + "The likelihood ratio is $\\frac{3}{4} / \\frac{1}{2}$, or $3/2$. So the\n", + "posterior odds are $3:2$, which corresponds to probability $3/5$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Oliver’s blood\n", + "\n", + "Here is another problem from MacKay’s *Information Theory,\n", + "Inference, and Learning Algorithms*:\n", + "\n", + "> Two people have left traces of their own blood at the scene of a\n", + "> crime. A suspect, Oliver, is tested and found to have type ‘O’ blood.\n", + "> The blood groups of the two traces are found to be of type ‘O’ (a\n", + "> common type in the local population, having frequency 60%) and of type\n", + "> ‘AB’ (a rare type, with frequency 1%). Do these data [the traces found\n", + "> at the scene] give evidence in favor of the proposition that Oliver\n", + "> was one of the people [who left blood at the scene]?\n", + "\n", + "To answer this question, we need to think about what it means for data\n", + "to give evidence in favor of (or against) a hypothesis. Intuitively, we\n", + "might say that data favor a hypothesis if the hypothesis is more likely\n", + "in light of the data than it was before.\n", + "\n", + "In the cookie problem, the prior odds are $1:1$, or probability 50%. The\n", + "posterior odds are $3:2$, or probability 60%. So we could say that the\n", + "vanilla cookie is evidence in favor of Bowl 1.\n", + "\n", + "The odds form of Bayes’s theorem provides a way to make this intuition\n", + "more precise. Again\n", + "\n", + "$${{\\mathrm{o}(A|D)}} = {{\\mathrm{o}(A)}}~\\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", + "\n", + "Or dividing through by $\\mathrm{o}(A)$:\n", + "\n", + "$$\\frac{{{\\mathrm{o}(A|D)}}}{{{\\mathrm{o}(A)}}} = \\frac{{{\\mathrm{p}(D|A)}}}{{{\\mathrm{p}(D|B)}}}$$\n", + "\n", + "The term on the left is the ratio of the posterior and prior odds. The\n", + "term on the right is the likelihood ratio, also called the **Bayes\n", + "factor**.\n", + "\n", + "If the Bayes factor value is greater than 1, that means that the data\n", + "were more likely under $A$ than under $B$. And since the odds ratio is\n", + "also greater than 1, that means that the odds are greater, in light of\n", + "the data, than they were before.\n", + "\n", + "If the Bayes factor is less than 1, that means the data were less likely\n", + "under $A$ than under $B$, so the odds in favor of $A$ go down.\n", + "\n", + "Finally, if the Bayes factor is exactly 1, the data are equally likely\n", + "under either hypothesis, so the odds do not change.\n", + "\n", + "Now we can get back to the Oliver’s blood problem. If Oliver is one of\n", + "the people who left blood at the crime scene, then he accounts for the\n", + "‘O’ sample, so the probability of the data is just the probability that\n", + "a random member of the population has type ‘AB’ blood, which is 1%.\n", + "\n", + "If Oliver did not leave blood at the scene, then we have two samples to\n", + "account for. If we choose two random people from the population, what is\n", + "the chance of finding one with type ‘O’ and one with type ‘AB’? Well,\n", + "there are two ways it might happen: the first person we choose might\n", + "have type ‘O’ and the second ‘AB’, or the other way around. So the total\n", + "probability is $2 (0.6) (0.01) = 1.2\\%$.\n", + "\n", + "The likelihood of the data is slightly higher if Oliver is\n", + "*not* one of the people who left blood at the scene, so the\n", + "blood data is actually evidence against Oliver’s guilt.\n", + "\n", + "This example is a little contrived, but it is an example of the\n", + "counterintuitive result that data *consistent* with a\n", + "hypothesis are not necessarily *in favor of* the\n", + "hypothesis.\n", + "\n", + "If this result is so counterintuitive that it bothers you, this way of\n", + "thinking might help: the data consist of a common event, type ‘O’ blood,\n", + "and a rare event, type ‘AB’ blood. If Oliver accounts for the common\n", + "event, that leaves the rare event still unexplained. If Oliver doesn’t\n", + "account for the ‘O’ blood, then we have two chances to find someone in\n", + "the population with ‘AB’ blood. And that factor of two makes the\n", + "difference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Addends\n", + "\n", + "The fundamental operation of Bayesian statistics is Update,\n", + "which takes a prior distribution and a set of data, and produces a\n", + "posterior distribution. But solving real problems usually involves a\n", + "number of other operations, including scaling, addition and other\n", + "arithmetic operations, max and min, and mixtures.\n", + "\n", + "This chapter presents addition and max; I will present other operations\n", + "as we need them.\n", + "\n", + "The first example is based on *Dungeons & Dragons*, a\n", + "role-playing game where the results of players’ decisions are usually\n", + "determined by rolling dice. In fact, before game play starts, players\n", + "generate each attribute of their characters—strength, intelligence,\n", + "wisdom, dexterity, constitution, and charisma—by rolling three 6-sided\n", + "dice and adding them up.\n", + "\n", + "So you might be curious to know the distribution of this sum. There are\n", + "two ways you might compute it:\n", + "\n", + "Simulation:\n", + ": Given a Pmf that represents the distribution for a single die, you\n", + " can draw random samples, add them up, and accumulate the\n", + " distribution of simulated sums.\n", + "\n", + "Enumeration:\n", + ": Given two Pmfs, you can enumerate all possible pairs of values and\n", + " compute the distribution of the sums.\n", + "\n", + "`thinkbayes` provides functions for both. Here’s an example of the first\n", + "approach. First, I’ll define a class to represent a single die as a Pmf:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import thinkbayes\n", + "\n", + "class Die(thinkbayes.Pmf):\n", + "\n", + " def __init__(self, sides):\n", + " thinkbayes.Pmf.__init__(self)\n", + " for x in range(1, sides+1):\n", + " self.Set(x, 1)\n", + " self.Normalize()\n", + " \n", + " # for Python 3 compliance\n", + " def __hash__(self):\n", + " return id(self)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now I can create a 6-sided die:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "d6 = Die(6)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And use `thinkbayes.SampleSum` to generate a sample of 1000 rolls." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "dice = [d6] * 3\n", + "three = thinkbayes.SampleSum(dice, 1000)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`SampleSum` takes list of distributions (either Pmf or Cdf objects) and\n", + "the sample size, n. It generates n random sums\n", + "and returns their distribution as a Pmf object." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def SampleSum(dists, n):\n", + " pmf = MakePmfFromList(RandomSum(dists) for i in xrange(n))\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "SampleSum` uses `RandomSum`, also in `thinkbayes.py`:\n", + "\n", + "```python\n", + "def RandomSum(dists):\n", + " total = sum(dist.Random() for dist in dists)\n", + " return total\n", + "```\n", + "\n", + "`RandomSum` invokes `Random` on each distribution\n", + "and adds up the results.\n", + "\n", + "The drawback of simulation is that the result is only approximately\n", + "correct. As `n` gets larger, it gets more accurate, but of course the\n", + "run time increases as well.\n", + "\n", + "The other approach is to enumerate all pairs of values and compute the\n", + "sum and probability of each pair. This is implemented in `Pmf.__add__`:\n", + "\n", + "```python\n", + "# class Pmf\n", + " def __add__(self, other):\n", + " pmf = Pmf()\n", + " for v1, p1 in self.Items():\n", + " for v2, p2 in other.Items():\n", + " pmf.Incr(v1+v2, p1*p2)\n", + " return pmf\n", + "```\n", + "\n", + "`self` is a Pmf, of course; `other` can be a Pmf\n", + "or anything else that provides `Items<`. The result is a new\n", + "Pmf. The time to run `__add__` depends on the number of items in\n", + "`self` and `other`; it is proportional to\n", + "`len(self) \\* len(other)`.\n", + "\n", + "And here’s how it’s used:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "three_exact = d6 + d6 + d6" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When you apply the `+` operator to a Pmf, Python invokes\n", + "`__add__`. In this example, `__add__` is invoked twice.\n", + "\n", + "This shows an approximate result generated by\n", + "simulation and the exact result computed by enumeration:" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAEWCAYAAADfHdlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VFX6wPHv1Ex6hTRSCL1IDTU0EVBQd3VdcddVFNbC\nriKIovITZBVRFBREBHQt6GLDvmsl0gRCh4BA6OmN9GTSpt3fHwMTQgIESDIp7+d5fB7umXPvvHNM\nZt7cOee8KkVRFIQQQgghhBBNjtrZAQghhBBCCCFqJ8m6EEIIIYQQTZQk60IIIYQQQjRRkqwLIYQQ\nQgjRREmyLoQQQgghRBMlyboQQgghhBBNlCTrQgghhBBCNFF1TtZXrFhBVFQUrq6uREdHs3Xr1ov2\nraysZPLkyfTu3Ru9Xs/o0aMvee2tW7ei0+no1atX3SMXQgghhBCihatTsv75558zY8YM5syZQ3x8\nPEOHDmX8+PGkpaXV2t9qteLq6sq0adO45ZZbLnntwsJC7rvvPsaMGXPl0QshhBBCCNGCqepSwXTw\n4MH06dOHVatWOdo6d+7MnXfeyYIFCy557rRp0zh8+DAbNmyo9fE77riDPn36YLPZ+Oqrrzh48OAV\nvgQhhBBCCCFapsveWTebzezdu5exY8dWax83bhxxcXHX9OQrVqzgzJkzzJkz55quI4QQQgghREt0\n2WQ9NzcXq9VKYGBgtfbAwECysrKu+ol///135s+fz8cff4xKpbrq6wghhBBCCNFSaZ3xpCaTib/8\n5S8sXryY8PBwAC41G6eoqKixQhNCCCGEEKLeeXt7X9V5l03WAwIC0Gg0ZGdnV2vPzs4mKCjoqp40\nMzOThIQEJk+ezP333w+AzWZDURT0ej0//vijLDgVQgghhBCt3mWnweh0Ovr3709sbGy19tjYWGJi\nYq7qSUNDQzl06BDx8fEcOHCAAwcOMHXqVDp16sSBAwcYOnToVV1XCCGEEEKIlqRO02BmzpzJpEmT\nGDBgADExMaxcuZLMzEymTp0KwOzZs9m9eze//vqr45yEhAQqKyvJzc3FaDRy4MABAHr37o1Wq6V7\n9+7VnqNt27a4uLjQrVu3S8ZytV8hiJr27NkDQHR0tJMjaXlkbBuGjGvDkHFtGDKuDUPGtWHIuDaM\n+pjKXadkfeLEieTn57NgwQIyMzPp2bMnP/30E+3atQMgKyuLxMTEaudMmDCBlJQUx3Hfvn1RqVRY\nrdZrDloIIYQQQojWoM4LTKdOneq4k36hDz74oEbbhcn75cybN4958+Zd0TlCCCGEEEK0ZHWqYCqE\nEEIIIYRofJKsCyGEEEII0URJsi6EEEIIIUQTJcm6EEIIIYQQTZQk60IIIYQQQjRRkqwLIYQQQgjR\nREmyLoQQotmw2RSsVpuzwxBCiEZT533WhRBCCGcwma3sO5DJb3Ep7I3PREEhvJ03Hdr7EhXhS4dI\nX8LDvNHrNM4OVQgh6p0k60IIIZocm00h4Xguv8UlE7crjdIyU7XHTycVcDqpwHGs0agJD/UiKtKX\nDu3tCXxEuI8k8EKIZk+SdSGEEE1Gcmohm+NS2LI9hbz8sjqfZ7XaSEwpJDGlkPW/2Stoq1Qq+x34\nSF+iIn3o0N6PyDBvXFzko08I0XzIO5YQQginys0rY8uOFH6LSyE5tbDWPm0D3Bk+JJwRQyPw9TZw\nKrmAU4n2u+unkgrIPmOscY6iKCSnFpKcWsiGLfY2lUpFuxAvx933qEhf2of7YDDIx6EQommSdych\nhBCNzlhqYvvuNH6LS+bw0VxAqdHHw11PzKAwRsZE0KWjPyqVyvFY7x6B9O4RWO16p5MKOJ1cyKnE\nfE4nFZKZXVLjmoqikJpeRGp6EZu2JgHnEnhPoiJ8HdNo2kf44GrQ1fvrFkKIKyXJuhBCiEZhMlvZ\nG5/Jlu0p7InPwGKpuauLTqthYP8QRgyJoE+vQHTaus0593DX06tHIL3OS+BLy0wkpRRxKjGfU0n2\nO/EZWUYu/MPAnsAXk5pezOa45LOtKkKDPe3TZyJ96dDej/YRPri5SgIvhGhckqwLIYRoMDabwpFj\nOfwWl0LcrlTKys01+qhUKq7r3paRMREMjg6ttzva7m56enRtQ4+ubRxt5RVmEpMLHVNoTicXkJZR\ngqJceGdfIT2zmPTMYrZsT3G0Bgd6Oua/R0X4EBXpi4e7vl7iFUKI2kiyLoQQot4lpRTyW1wyv21P\nIb+gvNY+UZG+jBwaQczgMPx8XBslLleDju5d2tC9S1UCX1FhITGlkFNnd5g5lVhAWkZxLQk8ZGaX\nkJldwradqY62wLYejvnvHdr70iHCFw8PSeCFEPVDknUhhBD1Iie3lC07Utm8LZnU9KJa+7QNcGdk\nTATDh4TTLsSrkSOsncGgpVvnALp1DnC0VVbaE3j7FpH2RD4lrajWBD77jJHsM0bidlUl8G0D3HEz\nVNK1kyf9+imo1aoa5wkhRF1Isi6EEOKqGY0m4s4uFD1yLKfWPh7ueoYNDmfE0PAaC0WbKhcXLV07\nBdC1U1UCbzJbST57B/5UYoEjgbfZaibwZ3JLKS0t4/CxYlDv58FJfZvF6xZCND2SrAshhLgi5xaK\nbt6WzL6DmbUuFNXrNAzoF8LImAj69AxCq1U7IdL6pddp6NTBn04d/B1tJrOVlNQi+xSas9tJpqQV\nVRuTn9efxNfHwJ1/7O6MsIUQzZwk60II0QCMpSZsNgUPd32zmQJRarJSYrLUaHfTafDQaTh8NIfN\n25LZsSeNsnIzVrUKq0YFOnsirlKp6NY5gFGDw7h+YFiNhaKXur5XLYWKLtbf16DDpYkk/3qdho5R\nfnSM8nO0mS1WUtKKWfHvjRw6WgzAp18dwsfbwNhRUc4KVQjRTEmyLoQQ9ezH2BO8tyb+7PxmFR7u\nOry9DHh66vHycMHL89x/ejw9XfA+e+x5ts3gonXKlInYxFz+vT+tWpup0kp7i4Lt4JkaC0ULfVzI\nDnTDxUWDp7sLHh56DmtUdPTQ1bqjS23XB/hj57Y81C+szv0D3V1475YeTXZaiU6roUOkL3f+oR3l\nFclk5dinyaz6YC8+XgYG9AtxcoRCiOZEknUhhKhHu/dl8O5/4qnay1vBWGrCWGqCzLpdQ6fV2BN7\nTxe8PFyq/l3jP3uy7+XhUqdpJjZF4WB2Ca46DV383S/az2KxYTSaKDGaMJmtlOZXEHRBoh7Yxp1O\nfYPYr1jR6eq2F3p9GRLq3WQT9fNpNWru/lM43/xURGJKIYqisHj5dp6fPbLaXHghhLgUSdaFEKKe\nJCYX8vrKHZxL1LVada3zuS/HbLGSX1B+0S0Pa+PmqnMk7t5eLlV38b1cqNBpOG4yc7CknBKLjYEh\n3swf3alawltWbuFYWg4F6SWUlJoc7TpAY7W/Hk8PF4YNDmPE0Ag6d/AjNjGPnMNZNWLx0Nf+0eKm\n0xDo7nJN/SstNka396+1/9bUAnLLTIyK8MOniVQfNbhomDtrBLNfWE92Tilmi5WXXt/KgjmjCQtt\nGrvhCCGaNknWhRCiHhQUVvDSkq1UVtrnWLcNcOeVf92Au5ueEqOJEmMlxSWVFBVXnv23iZKSSopK\n7O3n/l1SYsJssV7x85eVmykrN5N9xuhoM+nUpLXzoMJQ/a0+Ja2I3z8+iJ+bHi9PPaWlRWRklmNw\ndSMICDqvr16vZVD/EEbcF0HvHoHV7uCPiwpgXFTd7xDXV//atk8E+Cohm+P5pbwXn06/IC9uiPRj\nUKiP0+e3+3gbeG7WCGbP30BxSSXGUhMvLPqNhc+Nxt/PzamxCSGavjon6ytWrGDx4sVkZmbSo0cP\nli5dyrBhw2rtW1lZydSpU9m3bx8JCQkMGzaMDRs2VOvzzTffsGrVKvbv309FRQXdu3fn2Wef5dZb\nb722VySEEI3MZLKycOk28vLLAHvhnf+bOQxvLwMAvj4GfH0MdbqWoihUVlopNlZSXGxP5IuN9iT+\n/H9XJf2VlBhNtSawWosN8wVTVDRWG95FJmwoFBVXUFRcQWlp9Tv4KpWKPtcFMnxIOIP6119F0fpS\n2xSY1OIKjueXAvbpPnsyi9iTWYSbVsMbN3YlxLNu499QgoM8mfPEcOa+vInKSgt5+WW8sGgLC569\nXgooCSEuqU7J+ueff86MGTNYtWoVMTExvPXWW4wfP56EhATatWtXo7/VasXV1ZVp06bx448/UlhY\nWKPP5s2bueGGG1iwYAF+fn6sWbOG22+/nc2bNxMTE3Ptr0wIIRqBoigsf3c3J07nAfZE8olHBhPe\nzvuqrqdSqTAYtBgMWtoGXHxe+fnPfzjHSBudBkuFheJzSf3Zu/X/yy7iSLmJthYF/1IL+nwzRqOJ\nCkvN5L5jlB8jhkYwbFAYPt7OTW6vVICrjukDIliflM+hnBJHu6eLliCPmlNvnKFjlB9PPzaUF1/b\ngs2mkJpexMtLtzHvqRHo9Y07718I0XzUKVlfsmQJU6ZMYcqUKQAsW7aMn3/+mZUrV7JgwYIa/d3c\n3FixYgUABw4cqDVZX7p0abXj5557jh9++IFvv/1WknUhRLOx9tsjbN2R4jie8rc+9Osd3ODPm2Ws\nZH1SHhuT8sk0VjJ9YATjogIIveCpR5Wb0alVeF6wNaLJbMVotN+h37U7Hl8fPePGDG3wuBuKq07D\nuA4BjOsQQJaxko1J+WxIymNEhB/qWu7E55aZ2JVRxPAw3xpj05D6XBfEtAcH8sbbOwFIOJ7DkpU7\nmDVtaLPZ4lMI0bgu+w5lNpvZu3cvs2bNqtY+btw44uLi6jWYkpISfH196/WaQgjRULbuSOHzbw47\njm8c3YEJYzs26HMeyC7h498zOJxrrNa+PjG/1vndfq61T2HR6zT4+bri5+tKXo5Hg8TqLEEeLvy1\nZzB/6RGEuZbqogAbkvL58GA67+xLZWCoDzdE+tE/2BttIyTMI2MiKCyq4MPPDgCwc28673y4j4fv\n79csdrkRQjSuy666yc3NxWq1EhgYWK09MDCQrKyauwBcrbfeeov09HTuvffeerumEEI0lBOn8njz\nnd2O417dA/n7PQ1fUt5is9VI1N11GsK8XLBdZOFla6VSqdBran7MKYrC+kT7tCWzTWFbagEvbDnF\npO8OEpda0Cix/XFCF269qbPjeN3GU6z99kijPLcQonlpErvBfPXVVzz99NOsXbuWsLCahTHOt2fP\nnkaKqvWQMW04MrYNw9njWlRsZsXqkxhL7bu2BPjpuWGYgfj4ffVyfUVRyDcp+LvUTDStioLGVInR\notDVU01/Xy3dvWzoVDns25tzTc/r7HFtLFZFoZvGSpliJa2samvNslLITTrBnuz63T3mYuPao6NC\nQqSOA4eLAHh/zW7y89IZ0Mev1v6iutby89rYZFzrV6dOna75GpdN1gMCAtBoNGRnZ1drz87OJigo\n6CJn1d2XX37Jfffdx5o1a5gwYcI1X08IIRpSpcnKf75IdiTqrgYNk+6MwNVw7QsEi8wK+wss7C2w\nkl2pMLebAU9d9Tv1GpWKe8J1tHFR13hM1I1GpWJYgJZhAVqyKmzsK7Cyr8CKqwaCDbWP6eEiKx09\n1Lho6m/M1WoVf7o5lNJSCyeT7DvZfPdzBh5uWrp1lj3YhRB2l03WdTod/fv3JzY2ljvuuMPRHhsb\ny5133nlNT7527VomT57MRx99xO23316nc6Kjo6/pOUWVc389y5jWPxnbhuHscbXZFBa+sY3iUjXu\n7m5oNGr+9fRIenRtc03XjUst4KdTucRnl2BT1KABVzco9gvj+i5ta/Sv71fv7HF1tluwb/eYX24m\nwK3mNoopReXM++kIhgI1Q9v5cEOkP70CPWtduHq+uo5r7959mfvSJk4n2afgfP9rAdHRvejWWaqc\n1qa1/7w2FBnXhlFUVHTN16jTd30zZ85k9erVvPfeexw9epTp06eTmZnJ1KlTAZg9ezZjxoypdk5C\nQgLx8fHk5uZiNBo5cOAABw4ccDz+2Wefcc8997Bw4UKGDRtGdnY22dnZFBQ0znxBIYS4UmvWHmTP\n/gzH8dT7+19zog6wL6uEfVnF1eac6zVqjGbLNV9b1I1apao1UQfYmJQPQIXFxoakfJ7ddILJ/z3E\nt8eya+1/pVwNOuY8MZzAtvaFvmaLlZeXbCUl7do/5IUQzV+d5qxPnDiR/Px8FixYQGZmJj179uSn\nn35y7LGelZVFYmJitXMmTJhASkrVdmZ9+9oXXlmt9q+O3377baxWKzNmzGDGjBmOfiNHjqxRQEkI\nIZxt/eZEvv3xmOP4j+O7cMPI9ld0jQqLDUMt1TRvaO/HT6dyUKHiurYejI70Y1iYL6462Xu7KQj0\ncCHMy0BqcYWjLbfcRKn5yivNXoyPt4F5Z6ucFhVXnFfl9AYC/KXKqRCtWZ0XmE6dOtVxJ/1CH3zw\nQY22C5P3C23cuLGuTy2EEE51+GgOq1bvdRxH9w3h3rt61enc4koLm5Pz2ZCUj16j5pUbOtfo09Xf\nnYf7hTE41Ie27lLNsqm5qUMAN0b5c7KgjA1J+WxKzqe40sLoCP9a+6cVV1zV/8egQA/mPDmcuS9t\npKLCQn5BOfMX/8aCZ0dLlVMhWrEmsRuMEEI0VZlZJbzyxjasVvuuIZHhPjz+j0GXLGBjUxR2phex\nPjGP3RlFWM6b3pJlrKxRUVOlUvGHzjXnpoumQ6VS0cnPnU5+7vy9TzsSco0Ee9asjKooCvO3nKKw\nwkx7jYlBftorWmPQIdL3bJXTrVitNlLTi3lp6Vb+9dRIqXIqRCtVv/tTCSFEC2IsNfHSkm0YS00A\neHsZmD0jBldD7YWGzkksLOelbafZnl5YLVHXqlQczy9t0JhFw9OqVVzX1rPWx07kl5FWUoHRbGVn\nnpVlJyr5975UTFZbrf1r07tnEI89NNBxfPR4Lq+9td3xB6MQonWRZF0IIWphtdpYvHw76ZnFAOi0\nGmbPiKFNgPtlz+3g68ZfewQ7jrv6u/PP/uH857ZejAiXPbRbsqJKC4Hu1e+4f3v8DM9sOH5FRauG\nDwln8t19HMe792fwzkf7UKTwlRCtjkyDEUKICyiKwrv/2c/Bw1W7fUx7aACdO9Y+R7k2f+kRhJtO\nzYAQb0I9DQ0RpmiCBoR40z/YiyM5Rt7afJCjxfa74THtfC+71eOFbr2pM/kF5Xz3k31hc+zG0/h6\nu/KXP/Wo97iFEE2XJOtCCHGBH2NP8suGU47jibf1YNjg8Fr7Gk0WPPQ130rVKhW3dQlssBhF06VW\nqejZ1pMpkXq25Vkpcffh9q5Xtybh3rt6UVhUwea4ZADWfnsYXx8DN47uUJ8hCyGaMJkGI4QQ59l3\nIJP3P453HMcMCuOu27vX2ve35Hym/O8QezNlP2xRk+pspdQ5w6Ku+K76OWq1ikceGECf66oqhr/z\n4T527k2vrzCFEE2cJOtCCHFWSloRr721wzEvuFOUP9MeHIjqgkSrwmJj2a5kXtmeSKnZyms7ksgv\nNzsjZNEMXPjzc86OtEL+d/zMZeeha7VqZk0bQof29vUOiqLw+ls7OHIsp95jFUI0PZKsCyEEUFRc\nwUuvb6W8wp50+/u58cyMmBrb5SUVljNjXQK/nM51tLlqNRRXSrVRUXdnSk0s3ZXMqn2pLNh6mpLL\n/Py4GnQ8O3MYQYHnVzndJlVOhWgFJFkXQrR6JrOVV96I40yufVtFFxct//f4MHx9qi8MtdoUXtx6\nqlolyxHhvrx5UzcifVwbNWbRvK09kkmJyZ6gb08vZNovCRzOMV7yHB9vA8/NGoG3l/3nsrTMXuU0\nJ1e2AxWiJZNkXQjRqimKwsr393D0xLk75Spm/mMw7SN8avTVqFU8NjACFSr0GjXTB0bw1JD2uOmk\nWI24Mg/1C6tWCCunzMQzG47zw4lLT20JauvB3CeHYzDYFzXbq5xuwWg0NWi8QgjnkWRdCNGqffP9\nUTZvS3YcT/pLLwb0C7lo/15tPXkkOow3xnVlXFTARecjC3Epeo2ah/uF8dzwDniet5tQB9/Lf0MT\nFenL09Nj0GjsH+FpGcUseH0LlTIVS4gWSZJ1IUSrtX13Gmu++N1xPHp4e/44vjMANkXBaqt94d/4\njm0I95ZpL+LaDQr14c2butGjjQeTeoXQNcCjTuf17hHI9IerqpweO5nHayt2SJVTIVogSdaFEK3S\nqaQC3nh7l+O4e5c2PDy5HyqVioJyM89tOsnHhzKcGKFoLdq46Xn5+s7c0fXK9uUfNjicKX+rqnK6\nZ38Gb6+WKqdCtDSSrAshWp38gnJeen0rprML/ALbevD09KHotBr2ZRbz6M8J7M8uZu2RbOKzip0c\nrWgNNGrVRfdi/+/xM2SWVNb62C03dua2CV0cx79uPs1nXx9ukBiFEM4hyboQolWprLTw0pKtFBSW\nA+Dmat8Sz9VNzwcH0pm7+QSFlVV7pp8sKHNWqEKwM72Qt/elMu2XBDYn59fa5967ejEyJsJx/MV3\nR/h5/cnGClEI0cAkWRdCtBo2m8Kyd3ZxOqkAsBermfXYUNqFePHRwXS+TMhy9PVx0TF/VEf+3C3o\nYpcTokGZrDaW704BoNxi5dXtiSzdmUSFxVqtn0ql4pG/D6Bvr/OrnO5n++60Ro1XCNEwJFkXQrQa\nn319uFoC8+CkvvTuYZ8n/KeugfgadAD0C/LirfHd6Bvk5ZQ4hQD7jjFzh3cg2MPF0RabmMf0X46S\nXlJRra9Wq+bJR6uqnILC0pU7OXxUqpwK0dxJsi6EaBU2bUviy/8ecRzfPLYTN93Q0XHsY9Dx5OBI\n7u8dyvMjO+JzNnEXwpk6+7uz7MZujIrwc7RVWm14nbfd4zmuBh1znhhGcKAncK7K6VaSUwsbLV4h\nRP2TZF0I0eIdPZHLivf2OI779Ark/rt71+jXJ8iLO7sFXXShnxDO4KbT8OTgSKYPjMBNq+GpIe3x\ndKmZrAN4exl47qkR+Hjbq5yWlZt5YdEWR3VeIUTzI8m6EKJFy84pZeHSbVgsNhRAE+VDeZ8gkIRc\nNCMqlYpxUQF88IeedG9z6b3YA9u4M/fJEY4qpwWF5cxftIUSY+07ygghmjZJ1oUQLVZZuZmXXt9K\ncUklVrWKnEhvijv6cjDXyCeHM50dnhBXzKOW6S8AZquNX07lYju7x3r7CB9mzxiGVmv/mE/PLGbB\na1ulyqkQzZAk60KIFslmU3j9rR2kphdRbtCQ1MEblygfR/KyPa0Qk1R7FC3EhwczWLY7mbmbTlJQ\nbt969LrubZn+8CDA/i3S8VN5vPaWVDkVormRZF0I0SKt/vQA+w5m2hP1SG+8gz0d0wJu6hDA62O7\notfIW6Bo/n4/U8I3x7IBiM+2F/Xam1kEQMygMP5+z3lVTuMzWPn+XqlyKkQzUudPqhUrVhAVFYWr\nqyvR0dFs3br1on0rKyuZPHkyvXv3Rq/XM3r06Fr7bd68mejoaFxdXenYsSNvv/32lb8CIYS4wC8b\nTvH9L8cBMFRY6dnGA08PPW5aDU8Pac+0AREYtJKoi5ahe4AHd3UPQnX2DnphpZnnNp/k/fg0rDaF\nm8d14vZbujr6b9iSyCdfHnJWuEKIK1SnT6vPP/+cGTNmMGfOHOLj4xk6dCjjx48nLa32ggtWqxVX\nV1emTZvGLbfcUmufpKQkbr75ZoYNG0Z8fDzPPPMM06ZN45tvvrn6VyOEaPUOHM7m3x/tcxwPiW7H\nqrv7MTDEmzdv6saI87bAE6Il0KhVTOoVyoujOjpqBQCkFVegPruO+p47r+P64ZGOx776XwI/xp5o\n5EiFEFejTsn6kiVLmDJlClOmTKFLly4sW7aM4OBgVq5cWWt/Nzc3VqxYwQMPPEBoaGitfVauXElo\naChLly6lS5cuPPDAA9x3330sXrz46l+NEKJVS88sZvGb27HZ7F/xR0X48thDA/F31zNvREeCzisu\nI0RL0yfIi+U3daN/kBcBrnpmDIpEdXbXI5VKxT8mR9OvV7Cj/7v/iSduV6qzwhVC1NFlk3Wz2cze\nvXsZO3ZstfZx48YRFxd31U+8Y8cOxo0bV63txhtvZM+ePVit1oucJYQQtUs8Y+Rv7++i0GRfXOfr\n48rsx2Mc89SFaA18DDr+NbIji8Z0weuCvdjPVTntFOV/tkVh6aqdHEo40/iBCiHq7LLJem5uLlar\nlcDAwGrtgYGBZGVlXfUTZ2Vl1XpNi8VCbm7uVV9XCNH6bE3O566P95ClhvRQD3Q6DbMfj8Hfz83Z\noQnR6NQqFW3d9bU+VmixMumhaEeVU4vFxstLtpGUIlVOhWiqmt0tpz179ly+k7giMqYNR8a2YZwb\nV7NN4fsMMz+cLqes3P6NnNFFQ5/hPhTmnWZP3mlnhtnsyM9rw2gq42pVFFacNJFRYWPMEE+yvsvD\naLRQWgpPzv2eh++Nwten9iS/KWoq49rSyLjWr06dOl3zNS57Zz0gIACNRkN2dna19uzsbIKCgq76\niYOCgmq9plarJSAg4KqvK4RoHayKwopTlaxLqXQk6lqTlT95WRnb09vJ0QnR9KzLspBSZsNig58L\nFHxvCEfrqgGgxGhh9edJGEulaJIQTc1l76zrdDr69+9PbGwsd9xxh6M9NjaWO++886qfeMiQIXz7\n7bfV2tatW0d0dDQajeai50VHR1/1c4rqzv31LGNa/2RsG8aF47ohN4HdSemo1Wo8S0zcGebHrKmD\nHIvqRN3Iz2vDaGrj2qaonMy4RJKLygHIBNwmdKd8awp6o5nySvj6p0LmPTWiSU8ha2rj2lLIuDaM\noqKia75GnXaDmTlzJqtXr+a9997j6NGjTJ8+nczMTKZOnQrA7NmzGTNmTLVzEhISiI+PJzc3F6PR\nyIEDBzhw4IDj8alTp5Kens7jjz/O0aNHeffdd/noo4+YNWvWNb8oIUTLl5RSyMFvEvAwmgnKKuUG\ngwszHhggiboQFxHh7cqSsV0Z36GNo61MDSNu6eL4vUnLKGbOgo1k55Q6K0whxAXqNGd94sSJ5Ofn\ns2DBAjIzM+nZsyc//fQT7dq1A+yLRRMTE6udM2HCBFJSUhzHffv2RaVSOXZ6iYyM5Mcff+Txxx9n\n1apVhITP3Qv7AAAgAElEQVSE8Oabb3LbbbfV12sTQrRQhUUVvLRkK5UVFsJSSwgMcOeZ6THodRf/\nVk4IAS5aNY8OCKdPkCfLdiXTvY0Hs4Z3IM7fkyUrd2CzKWTnlPLsixt4/pmRhAZ7OTtkIVq9Oi8w\nnTp1quNO+oU++OCDGm0XJu+1GT58uCxkEEJcVpaxEu3Z6i4Wi42FS7eRm1cGgKtBy//NHIaPt8GZ\nIQrRrAwL86WjrxsGrRqVSkXMoDD0eg2L39yO2WIlv6CcOQs2Me+pEUSG+zg7XCFaNam3LYRo0gor\nzDy78QRP/nqM7HIrX/2QzvFTeYC90MsTjwwhvJ0sKBXiSgV5uOBzXsXTAX1D+L8nhuFydn/2ouIK\nnnt5EyfO/r4JIZxDknUhRJNVZrby3OaTZJVWklNmYuE+IwcSqhbrTL67N/17B1/iCkKIK9G7RyBP\nPh5DSZA7CmAsNTHvlc0cPprj7NCEaLUkWRdCNEkmq40Xt57iVIF9uktZqRkO5KBS7I+PvT6Km8dd\n+/61QogqZ0pNvJOYi9KjDcXt7IWTKiosvPDqb+w7kOnk6IRonSRZF0I0OTZF4bUdSRzILgHAbLai\nO5yDW1EFAL26B/Lgvf1k5xch6tn/TpwhraQCFxcN6q7+VIbaE3azxcrLS7exfXeakyMUovWRZF0I\n0eSogPCzC0YVm4JbYhEeufa9of199Tz56BC0Wnn7EqK+3dcrlAEh9jUger0GS/cAaGffEcZqtbF4\n+XY2bUtyYoRCtD7yaSeEaHJUKhV/6xnCP/uH4V9QiepkAQBarYq/3h6Gh0fzKYkuRHOiVat4ZmgU\nXf3d7cc6NRU9AvAMtSfsiqKw7O1d/LLhlDPDFKJVkWRdCNFkaVNLKN2ZzrnJLn8YF0JwoKtTYxKi\npTNo1cwb0ZEwL/u3W3/sFsiSp0dW23Xp7dV7+fbHY84KUYhWRZJ1IUSTdCqpgPfXxDuORw9vT//e\nvk6MSIjWw8tFy/xRnXi4XxgP9W2Hn48rL/7f9XRo7+fo89FnB/js68MoiuLESIVo+SRZF0I43ZEc\nI1nGSsex0Whi0bI4zBZ7xeOIMB8enNTXWeEJ0Sq1cdPzh85tHQu5PTz0PP/MSLp3aePos/bbw6z+\n9IAk7EI0IEnWhRBOdbqgjHmbT/JE7DFOF5ShKApv/nsXZ3JLATAYtMyaNsRRqEUI4TxurjrmPjmc\nPtcFOdr+9/NxVq3ei80mCbsQDUE+/YQQTpNlrOS5zScps1gps1h5JS6RG9Gwe3+Go8+0BwcSEuTp\nxCiFEOcrtSl4jwgnWq9mz17772rsxtOYKq08+uAANBq5DyhEfZLfKCGEUxRWmJm76QQFFWYA3LQa\n/hzszcdrf3f0ueXGzgwZ0M5ZIQohLpBeUsGTvx5jXWIe1r5BjBgS7nhsc1wyi5dvx2S2OjFCIVoe\nSdaFEI3ObLXxr99OkXF2nrperebxfmF89sF+x9zXLh39ufeu65wZphDiAnszi8kutf/e7sgoQtM/\nmDGj2jse37k3nYVLt1FZaXFWiEK0OJKsCyEanU6jZlSEfWcXtUrFk4Mj+WXt7xQU2QsfebjreeKR\nIei0GmeGKYS4wK2d2nB7l0DH8c+nc/EaEMIfburiaIv/PYv5i7dQVm52RohCtDiSrAshnOK2LoE8\nOTiSR6PDSd6Vzu9Hzpx9RMXj/xhMgL+bU+MTQtSkUqmY0ieU0ZFVWzh+ejiLvqPbM/G2Ho62I8dy\nmLdwM0ajyRlhCtGiSLIuhHCa6yP9aWM08+V/jzjaJt7Wnb69gi5xlhDCmdQqFdMHRtI/2F4k6S89\ngukV6Mlf/tSD+/7S29HvVGI+c17aSGFRhbNCFaJFkGRdCOE0ObmlLF2103Hcq3sgE2/r7sSIhBB1\noVWr+L+Y9jw9pD33Xhfi2Iv9jxO68PD9/eFs3eGUtCKefXEjuXllToxWiOZNknUhRIPblJRP8tn5\n6OeYLVYWLd+OsdT+NbmfryuP/3MQarXKGSEKIa6QQathRIRfjfYbR3fgsYcHOBL4zOwSnn1xI1nZ\nxsYOUYgWQZJ1IUSD2pFWyGs7k3h6/XEScqs+rD/89AAnT+cDoFareOKRIXh7GZwVphCiHo2KieTJ\nR4c49lzPySvl2Rc3kpJW5OTIhGh+JFkXQjSYQ2dKeCUuEZuiUGKy8EF8OoqisG1nKj/GnnT0m3RX\nL7p1DnBipEKI+nKu2FnXnoHMnhHj2NWpoKicuS9t4lRSgXMDFKKZkWRdCNEgEgvLeGHLKUw2GwDB\nHi7837AoMrJKeOu93Y5+g/qHcutNnZ0VphCiHiUWljHr12PszSyyJ+zd2/LcUyMwGOwF00uMlcx7\neRNHT+Q6OVIhmg9J1oUQ9a7cbGXe5pOUnq1k6OOi48VRnXBVqXh12XYqKuwFUwLbevDog1VzW4UQ\nzduZUhOFZwsinS4sY/6WU3Tq5M+/nhqJu5segLJyM8+/+hsHD2c7M1Qhmg1J1oUQ9c5Vp+GeniGo\nVSrctBrmj+pIoLueVav3kppun7Oq02p4atpQxwe4EKL5GxTqw7QB4Y7jg2dKWLwjiY4d/Hhh9ii8\nPF0AqKy0sOC1rezen+GsUIVoNiRZF0I0iHEdApgzLIq5IzoQ5evGr5sT2bwt2fH4g/f1pX2EjxMj\nFEI0hHFRAdzfK9RxvC21gB1phbSP8GHBnOvx83UF7DtCvfLGNrbtTHVWqEI0C3VO1lesWEFUVBSu\nrq5ER0ezdevWS/Y/dOgQo0aNws3NjbCwMObPn1+jzyeffELfvn1xd3cnODiYe++9l+xs+VpMiJZi\nUKgPvdp6cjqpgHc/2u9ov354JDeMaO/EyIQQDenP3QK5rXNbAKb0bsfQMF8AQoO9WDBnNIFt3AGw\n2RReX7GD9ZsTnRarEE1dnZL1zz//nBkzZjBnzhzi4+MZOnQo48ePJy0trdb+JSUljB07luDgYPbu\n3csbb7zBokWLWLJkiaPPtm3bmDRpEpMnT+bIkSN89913JCQkcM8999TPKxNCNAnGUhOLlm/HbLHP\nXw9v581Dk/rJPHUhWjCVSsXf+7Zj4ejO3NEtsNpjgW3cefHZ6wkN9gJAURTeem83P8aecEaoQjR5\ndUrWlyxZwpQpU5gyZQpdunRh2bJlBAcHs3Llylr7r1mzhvLycj788EO6devGn/70J55++mlef/11\nR58dO3YQFhbGY489RkREBAMHDuTRRx9l586dtV5TCNE0KYrCx4cyOJFfWutjy/+9m+wz9v3VDQYt\nTz02FBcXbWOHKYRoZGqViuvaetb6mL+fGy8+O4rI8KqpcO/+Zz9f/y+hscITotm4bLJuNpvZu3cv\nY8eOrdY+btw44uLiaj1nx44dDB8+HL2+auHYjTfeSEZGBsnJ9jmrMTExZGZm8v333wOQm5vLZ599\nxs0333zVL0YI0fi+PJrNJ4cymb3hBPFZxdUe+9/Px9m1L91x/MjfBxASVPuHtxCi9bApCt5eBl6Y\nPYrOHfwd7Wu++J2Pv/gdRVGcGJ0QTctlb2/l5uZitVoJDLzga6zAQNavX1/rOVlZWYSFhdXorygK\nWVlZREREMHjwYD799FP+9re/UV5ejsViYdy4caxevfqS8ezZs+dyIYsrJGPacFr62O7Kt/BFqhmA\nMuDDbYewRNj/SE9KLeXdjxM595k7JNofF002e/Zc+7qUlj6uziLj2jBkXKs7U2HjP8km/hquJ8RV\nza1jPVhTlM/pZPu3cx99tpcTJ5OYcEMQavXFp8vJuDYMGdf61alTp2u+htN2gzly5AjTpk1j3rx5\n7Nu3j19++YXMzEweeughZ4UkhLgCh4usfJlmdhxHeaiZGKYDwFhq4fNvUx2JeliIKzeNDqztMkKI\nViSlzMZbpyrJqlB4N9FEXqUNF72GSXdG0KVD1bdu2/fk8e1P6dhscoddiMveWQ8ICECj0dTYpSU7\nO5ugoKBazwkKCqq1v0qlcpyzcOFCBg0axMyZMwHo2bMnK1asYPjw4bz88suEhITUeu3o6OjLvypR\nJ+f+epYxrX8tfWwLK8y8/P0hXN3sbyFRPm4sHN0Zd70Gm03hhVd/w6rocXfX4+Gu5+V5Y2kT4H7N\nz9vSx9VZZFwbhoxrTf6FZbjnHAezFSvwZYELi8Z0wcegI3pAf5au3MX23fatHI+eMrFlt4XpDw9C\nq626tyjj2jBkXBtGUVHRNV/jsnfWdTod/fv3JzY2tlp7bGwsMTExtZ4zZMgQtmzZgslkcrStW7eO\nkJAQIiIiACgrK0Oj0VQPRq1GpVJhO1ueXAjRNPkYdEwfEIFWpSLQ3YUXRnbEXW//fV777WEOHjn3\nx7qKGVMH1UuiLoRo/tr7uPHc8A7o1fb0I8NYybzNJyk3W9FpNcz85yBGDYt09N+2M5VX34zDdLYa\nshCtUZ2mwcycOZPVq1fz3nvvcfToUaZPn05mZiZTp04FYPbs2YwZM8bR/+6778bNzY3777+fw4cP\n8/XXX/PKK6/wxBNPOPrceuutfPfdd6xatYrExES2bdvG9OnT6d+/P+3atavnlymEqG8jIvx4fmRH\nXhzVEV9X+/SX/QezWPtt1W4Od/6xG/16BzsrRCFEE9SzrSdPD22P+uz2rScLytiRXgiARqPm0QcG\nMH5MR0f/PfszeOm1rVRUWJwSrxDOVqf90yZOnEh+fj4LFiwgMzOTnj178tNPPzmS6qysLBITqwoa\neHl5ERsbyyOPPMKAAQPw9fVl1qxZzJgxw9Hnvvvuw2g08tZbb/Hkk0/i4+PD6NGjWbhwYT2/RCFE\nQ+kT5OX4d25eGUtW7gDsc0x7dQ/krtt7OCkyIURTNridD49Gh7N8Twr/7B/G9ZFVO8Ko1SoeuLcv\nLi5avv3hKAAHj2TzwqLfePaJYc4KWQinUSnNYH+k8+f7eHt7OzGSlkXmpzWc1ja2ZouVuQs2cfxU\nHgC+Pq68Nn8sPt6Gen2e1jaujUXGtWHIuF5eekkFoZ61v08oisJX/0vgky8POdqiIny5dawn7m5a\nGdd6Jj+vDaM+clin7QYjhGgeLDaFV+MSOXSm5KJ9PvrsoCNRV6lUPPHI4HpP1IUQLc/FEnWwv5f8\n+Q/dmXx3H0fb6eQC3v04kWKj+aLnCdHSSLIuhLgom6Lwxq4kNqfkM3fTSXakFdboE7crlR/WVZUJ\nn3RXL7p3adOYYQohWpgKS9VGE7fe1Jl/TIkG7HPcz+RW8u81ieTmlTkpOiEalyTrQoiLej8+nQ1J\n+QCYbDZOFlT/cMzIKmH5u7sdxwP7hfKH8Z0bNUYhRMtyuqCMh384zPbzbg6MHRXF4/8YhOrsotT8\nAhMvL9lKZaUsOhUtnyTrQohafZWQzTfHquoljO8QwN96Vu3sUllp4dVlcY4dGgLbuPPogwMcH6ZC\nCHGlEnKNPLX+OLnlJl6NS+T386bfDR8SztOPDeXsro8kphSy/N3dNIOld0JcE0nWhRA1ZBkr+fBg\nuuN4aDsf/hkdXi0R//dH+0lJsy+c0Wk1zJo2FA93faPHKoRoOYI9XPAx2DeqM9lszN9yiqTCcsfj\nA/uHcuu4qqKJ23am8s33Rxs9TiEakyTrQogagjxcmDO8A3qNmp5tPJk1pGpPZID1mxPZsKVqu9a/\n39uXqEhfZ4QqhGhBfAw6XhzVCV+DvXZDqdnK3E0nyDJWOvoM7OvHwL5V7zdrvjjEnviMRo9ViMYi\nyboQolYDQ7x5ZXRn5g6PQq+peqtITC7knQ/3OY5HxkQwdlR7Z4QohGiBgjzsVZHdtPaqyPkV5mrT\nYQBuHht83kJ2hSUrdpKWUdzIkQrROCRZF0JcVGd/dzz0VbXTSstMLFoeh9liL/0dFurNw/f1l3nq\nQoh6FeXrxtwRHXDVapg1uD1jowKqPa7VqHny0SEE+LsBUF5h5uUl2zCWmpwRrhANSpJ1IUSdFmgp\nisJb7+4hK9sIgMGg5anHhmAw1KkQshBCXJFebT15/9aejIr0q/VxH28Ds2cMQ6+z34HPzC5hycod\n2Gyy4FS0LJKsC9HKlVRamBl7jD2ZRZfs9/0vJ9ixJ81x/M8p0YQGezV0eEKIVszL5dI3A9pH+DDt\noYGO4/0Hs1iz9mBDhyVEo5JkXYhWrMJi4/ktpzieX8oLv51iY1Jerf2Onsjlw88OOI4njO3IsMHh\njRWmEEJUU2JWHN8IxgwK445buzke+/bHY/wWl+ys0ISod5KsC9FKlZutzN9yioRc+7QWq6KgqWXu\neVFxBYuXb3d8tdwxyo/7/tq7UWMVQohzkkptLDpWweqDGY6E/a939CS6T9WWjm+9u4eTp/OdFaIQ\n9UqSdSFaoaJKC7M3niA+u2r3hIf7hTEiovrcUJtNYcnKneQX2Pc59nDX8+SjQ9Cd3aVBCCEa07G8\nUt4+XUm5Fb5MyGLZ7hSsNgW1WsX0qQMdU/PMFisL39hGQWGFkyMW4tpJsi5EK5RXZiKtuOpDbNJ1\nIfyhc9sa/dZ+e4SDh6uqmM6YOoi2Ae6NEqMQQlyovY8rnT2qUpd1p3NZGHcak9WGu5ue2Y/H4O5m\nL86WX1DOq8u2YTJbnRWuEPVCknUhWqEoXzfmni169Gh0OHf1CK7RJ/73LNZ+e8RxfMet3ejXu2Y/\nIYRoLHqNmkmRevr7Vn27F5dWyIKtp1EUhZAgT2b+c7BjO9ljJ/N496P9ddrxSoimSpJ1IVqp3oGe\nvH9LT8Z3bFPjsbz8Mpas3AnYP+Cu696Wv97Rs5EjFEKImjQqFRPDdNzeJRAAtUrFhI4BjgS9b68g\n7r2rl6P/r5tP89OvJ50SqxD1QTZIFqIV83XV1WizWGwsXr6dkrPlvX29XXn8H4NRq6XwkRCiaVCr\nVDzQtx0+Bi2+Bh2DQn2qPf7H8Z1JTilk89ldYd5bE094O296dqs53U+Ipk7urAvRwv2WnM+607l1\n7v/R5wc5dtK+haNKpeKJRwfj421oqPCEEOKq/blbEDe096/RrlKp+MeUaDq0ty+aVxSFV5fFkZ1T\n2tghCnHNJFkXogX7/kQOr25P4s3dKcSlFly2//bdaXz/y3HH8T0Tr6N7l5rTZIQQoqkzofDM9KF4\ne9lvNhhLTSxcuo3yCrOTIxPiykiyLkQLpCgKHx/KYOXeFBQUbIrCZ0eysF1ikVVGVgnL/73bcTyg\nbwi3TejSGOEKIUS92pdZzJT/HeJ0hZmnpw9Fq7WnO8mphbz5zm5ZcCqaFUnWhWhhbIrCqn2pfHIo\n09HWxd+dF0d1Ql1L0SMAk8nKoje3O+44tQ1wZ9pDAx0LtoQQork4llfKgm2nKD1b+C1Tp+Lh+/o7\nHt+xJ40v/5vgxAiFuDKSrAvRwuSUmdicXDXlpV+QFwtGdcLL5eLryf/90T6SUwsB0Gk1zHpsKB7u\n+gaPVQgh6puHXoO3i33xvFVRWLwjCWOQOxPGdnT0+fSrQ+zam+6sEIW4IpKsC9HCBLq78K8RHXDR\nqBkR7stzwzvgqrt4xdH1mxNZ/1ui43jKPX3oEOnbGKEKIUS9C/U0sOiGzkR4uzra3tmfilufoGq7\nwSx9eycpaUXOCFGIK1LnZH3FihVERUXh6upKdHQ0W7duvWT/Q4cOMWrUKNzc3AgLC2P+/Pk1+pjN\nZp577jmioqIwGAxERkayfPnyK38VQohqugZ48PrYrswa0h6d5uK/5qeTCnjnw32O45FDIxh3fVRj\nhCiEEA3G303PK6M70y3AAwCdWkWPtp7MmjbEUYW5osLCy0u3YTSanBmqEJdVp2T9888/Z8aMGcyZ\nM4f4+HiGDh3K+PHjSUtLq7V/SUkJY8eOJTg4mL179/LGG2+waNEilixZUq3fXXfdxbp163j33Xc5\nfvw4X3zxBb169ar1mkKIKxPp43rROepmi5Uv/3uE2S9swGyxl+IOC/Xi4fv7yzx1IUSL4Omi5cVR\nnRgU4s2sIe3pHeiJp4cLsx+PweXstMDsM0YWv7Udq9Xm5GiFuLg6FUVasmQJU6ZMYcqUKQAsW7aM\nn3/+mZUrV7JgwYIa/desWUN5eTkffvgher2ebt26kZCQwOuvv87jjz8OwLp169i4cSOnTp3Cz8++\nD2p4eHh9vS4hWoW04goO5xi5sUNAnc85djKPFe/tITW96utfg0HLk48OxWCQOmlCiJbDoFUzd3iH\najchIsJ8eOyhgSx6Mw6Ag4ez+fCzg0z5Wx9nhSnEJV32zrrZbGbv3r2MHTu2Wvu4ceOIi4ur9Zwd\nO3YwfPhw9PqqBWo33ngjGRkZJCfbq4l99913DBgwgNdee42wsDA6d+7M9OnTKS2VggVC1MXxvFJm\nrT/Gst3JrE/Mu2z/8goz//5oH7Nf2FAtUY+K9GXBs6MJC/VqyHCFEMIpavu2cMiAdky8rTvnNnD8\n/pfjbNyS1KhxCVFXl72Nlpubi9VqJTAwsFp7YGAg69evr/WcrKwswsLCavRXFIWsrCwiIiI4ffo0\nW7ZswcXFha+//prCwkIeffRRMjMzWbt27UXj2bNnT11el7gCMqYNp6HG9kSJlQ+TTVTaZ7CwePMR\ndNkG3LS1T2FJOFHMf3/JoLjE4mjT6VSMGRHI0Ggf8nJOkpfTIKE2CPmZbRgyrg1DxrVhXOu4lnpa\nyOrhg9v+bLRmK4uXb6YwP4WwULd6irB5kp/X+tWpU6drvobTvvO22Wyo1Wo+/fRTPDzsC0CWL1/O\nTTfdRE5ODm3aSNVEIWpzsNDKJykmrGdvCblq4O/tXWpN1IuNZr5fl8nhY8XV2jtHefCHG0Pw9ZHt\nGYUQrc9Jo5VPU83o/Q3kXdcW3yM5UGHh469T+Of9HfDy1Dk7RCEcLpusBwQEoNFoyM7OrtaenZ1N\nUFBQrecEBQXV2l+lUjnOCQ4OJjQ01JGoA3Tr1g1FUUhJSblosh4dHX25kEUdnfvrWca0/jXU2FZY\nbCz74RAubvZf3QBXPS+M6lhtizIAm00hdtNp/vP5QcrKLbi72+8UeXm68Pd7+jJscFizXEgqP7MN\nQ8a1Yci4Noz6GFdTagEeZxIx2xRcXFzJUmsISSrGVm7hl81lzH92FPpLbHnbEsnPa8MoKrr27UEv\nO2ddp9PRv39/YmNjq7XHxsYSExNT6zlDhgxhy5YtmExV2yGtW7eOkJAQIiIiAIiJiSEjI4OysjJH\nn2PHjqFSqRx9hBDVGbRq/jWiI+46De08DSwa06VGop6aXsyclzby9uq9lJWbHe2jh7dn+SvjGT4k\nvFkm6kIIUV+Ghvny/MiOuGo16HRqAoI8SYnwxOiu48TpPFZ9sBdFUS5/ISEaQZ22bpw5cyarV6/m\nvffe4+jRo0yfPp3MzEymTp0KwOzZsxkzZoyj/913342bmxv3338/hw8f5uuvv+aVV17hiSeeqNbH\n39+fyZMnc+TIEbZt28aMGTO48847CQio+84WQrQ2Ub5uzB/ViVdu6Ezb86qMmsxWPvv6ME/MXcfR\n47mO9qBAD55/ZhSPPjgADw+Z9iKEEAC9A7146Xp7dWdXVy2BbdxR2+wJ+qatSfyw7oSTIxTCrk5z\n1idOnEh+fj4LFiwgMzOTnj178tNPP9GuXTvAvqA0MbGqAqKXlxexsbE88sgjDBgwAF9fX2bNmsWM\nGTMcfdzd3fn111+ZNm0aAwcOxNfXl9tvv52XX365nl+iEC1PF3/3asdHjuWw8v29pGdWzU1Xq1Xc\nfnNX7vxjd/T61vV1rhBC1EVnf3cW3dCF5387ycMjOrIdrWNXmA8+OUBYO2969wi89EWEaGB1XmA6\ndepUx530C33wwQc12nr06MGmTZsuec1OnTrx888/1zUEIVqVCouVHWlFjIr0u2gfY6mJ/6w9SOzG\n09XaO0X588+/9ycizKehwxRCiGatnZeBlRN6oFWr6HV/f9LSSzhxOg9FUXht+XZe/dcYggI9Ln8h\nIRpInabBCCEaV0mlhTmbTrJoRyLfn6i5p6KiKMTtSuWxZ36ulqgbDFoeuLcvLz83WhJ1IYSoI63a\nvo5Hr9Pw9Iyh+PrY1wIZS028tHQr5RXmS50uRIOSZF2IJia3zMRT64+TkGsEYNXeVJIKy6sezyvj\n5aXbWLx8O4VFFY726L4hLHv5JiaM7YRaLQtIhRDiavj5uPLM9KHotBoKvfXswMbrq3Zis8mCU+Ec\nUltciCYkvaSCORtPcKasaielh/q1I9LHFZtN4ef1J1nzxe9UVFQVN/LxNvDgpH4Mjg6VXV6EEKIe\ndOrgz/UTe7B0fxoAX+eWEPb1ISb9+TonRyZaI0nWhWgiFEVh0fYkR6KuUamYOSiSUZF+JKcWsuK9\nvZw4nVftnLHXR3HvxF54uMsuL0IIUZ90QR74eBsoLKqg1F3HkoPphIR6MWaIbC8tGpck60I0ESqV\nillDInny12NUWGw8OyyKXv4efPzF73zzw9FqX8GGBnvxjyn96d5FKv0KIURDmNI7FBe1itc2nKS8\n3Ey5q5ZZvx7jPX83+nWW917ReCRZF6IJCfU08MLIjlhsCubsUma8vo2sbKPjca1WzZ9u6cafbu3a\n6qrrCSFEY1KpVNzTKxRXlZp//ZyA2WKjUq3i9Xd3s2LuDXh5ujg7RNFKSLIuRBMTpNfy4acH2bAl\nsVp7184B/GNyNGGhXk6KTAghWp87rgsGk5V5647SLqWEsnILi5dv57lZI9BqZZ8O0fAkWRfCCRRF\n4eCZEnq19XQsClUUha07UnlvzX6KSyodfd1cddx7Vy/GjoqSXV6EEMIJ7ujfjgCrwhvLtwNwKOEM\nqz+N54F7+zk5MtEaSLIuRCOzKQrvx6fzzbFs7u8dyp3dgsjOKeWdD/ey/2BWtb5DBrTj7/f2xe/s\nnr9CCCGcY+TAMM7cUcKnXx0C4MfYk0SG+zBmZJSTIxMtnSTrQjQii03hjV3JbEiy7+qyOj6d1IQc\ndv14ApOpajtGP19XHr6vPwP6hTgrVCGEEBf48x+6kZRSyPbd9i0d3169j9MqhdBgL27uJItORcOQ\nZHBFT2kAACAASURBVF2IRlJhsbEw7jS7M4oAqKy0oso08ltCLmrHRi8qJoztwN1/vg43V53TYhVC\nCFGTSqVi2oMDycgykpxaSL67ljd2JNEuxIuiSgt/7REk9S5EvZOVEUI0khV7UtidUYRiU8jLL6f0\nSA4eR6oS9fB23ix8bjQP3NtPEnUhhGiiDAYtz8yIwd1dT4GvAatVIeuMkTW/Z/D2vjRsilQ6FfVL\nknUhGsnfrgtGb1FISS9Ge6qAoMxSVIBOq+Fvf76OxS+MpXNHf2eHKYQQ4jIC27jz9GNDiUgrwb3U\nTGWllZzcUv534gyv7UjCbLU5O0TRgsg0GCEaQVFxBZ99cgDTnjQC3LX4Fdh3e7mue1sevr8/IUGe\nTo5QCCHElejZrS0P/a0vb/9nHxkhHhQDer2G/VnF5JWbCfKQfdhF/ZBkXYgGpCgKm7Ym88En8RhL\nTbgALiYrHu567v9rb64fHinzG4UQopm66YYOJKUUsm7TaTRWNwpsCk9dFyqJuqhXkqwL0QDOVNhQ\nl5p4/pXfOHgku9pjw4eEM/nuPvh4G5wUnRBCiPqgUql4YFJfUtOL4UQu/nkVfJGynwFR/vKNqag3\nMmddiHq2OSmP5/cZ+dePZ6ol6m383ZnzxHAe/8dgSdSFEKKF0Gk1PPXYUPx9XdGbbZSVm3l5yTZK\ny0zODk20EJKsC1FPTFYby3cl8/i3h/6/vTuPi7paHzj+GWCAYUfZN8ENwQUX3AITNTVbvFlpZZta\nt7pZuV1LUyuv+y21NJeWe8uym3a7bfrLEsVdMUExWdxF2QZE9p2Z+f7+wMZIFFJwGHzer5evnDPn\nO/PMkZhnzpzzHPKLdeR5OJDvYoNKpWLk3cG8t2g4PcO8TR2mEEKIRubibMuMyRGorSwByMgq4t01\nBzEYairDrIm/wJbTF1GkUoy4AbIMRohGcKGwnH/uP8fB05coK68GwKpSR4iLhqlTBtA+qJWJIxRC\nCNGU2ge1YuKz4by79iAA8Uez+PJ/ifj18WXzqYsAJGQX83LvABysJf0SDSc/LUI0gv+maDl8Pp/i\nkpoqL3YF5Yx0t+TNaXdhaSlfYAkhxO3gzjvakHqhgO9+PAHA15tScKyuMq5j2JuWz6m8Ml7tH0gn\nNwcTRirMiWQRQjSC0CqF4pxSVIqCd1Ypd9sr3D/IUxJ1IYS4zTwxphs9unkBoAIqYlLp7WJvvD+7\ntJJXt5/kl8unWQtRH8kkhLhJx5Jz+Pcnh/FLLyboXBEDfVx4YISPlGQUQojbkIWFiil/64e3Z001\nGF2VnuNfJ/OghzP26po17V4ONnR1l5l10TCSrAvxJxgUhaJKnfH2hfRClry3D73egKZCTwc3B6a/\n0h8rmVEXQojbloO9Na9PjcDB3hqAqiodP647wig7DV3cHXm1fxCay4m7EPVpcEaxevVq2rZti0aj\nITw8nL179163f2JiIlFRUdjZ2eHv78+8efOu2Xfv3r2o1Wq6devW8MiFuMUKKqqZu/sMc3aeokpv\nIL+gggVL9xo3lLo6a5g9LRJ7O2sTRyqEEMLUfL2dWDhnMB5uNUtgFEXhv18eJSS7jLYuGhNHJ8xJ\ng5L1jRs3MnnyZGbPnk1CQgJ33HEHI0aMID09vc7+xcXFDB06FG9vb+Lj43nvvfd4++23Wb58+VV9\nCwoKePrpp7nrrrtu7pUI0YR+zSnmlZ9TiMsq5HR+GR/Hp7Fg2R4uXioFwMbGilnTInF3s6/nkYQQ\nQtwu/HycWPzmENr9riLYDz+dZOmqWKqq9bX6VuoMLD+YivZyoQIhftOgZH358uVMmDCBCRMmEBwc\nzIoVK/D29mbNmjV19l+/fj3l5eWsW7eOkJAQHnzwQV577TWWLVt2Vd9nnnmGcePG0a9fv5t7JUI0\nAb1BYf2xTF6POcWlyzPoKLB7/3nOpOYDNSfY/f2l/rQNdDVhpEIIIZojF2db5s2MIryHj7HtwKE0\n3lq8y1hBDOCjI2lsO3eJV35OYV9avilCFc1Uvcl6dXU18fHxDB06tFb7sGHD2L9/f53XxMbGMmDA\nAKytrywHGD58OJmZmZw/f97Ytnr1anJycpg9e/aNxi9Ek9qXns+XSVko1Bxk4WRtRa9yPVWHtfy2\nffS5p3vSSw47EkIIcQ22tlbMmBTBPUPbG9uOn8plxtwYtNklZBRXsPXsJQBKq/Us3HeW9w9doFJn\nMFXIohmpN1nPzc1Fr9fj6elZq93T0xOtVlvnNVqtts7+iqIYrzl27Bjz5s3jiy++kKoZotka4O9K\nP18XALp6OHKfWk3izlTj/Q/cE8zwwe1MFJ0QQghzYWGh4pknejB+bHe4PN2TlV3MjH9spyS7lCVD\nOuLxuz1PW85cZGr0cUqqdNd4RHG7MMmhSFVVVTz66KO88847BAQEADT4CN64uLimDO22JGN6fQPV\nCnbOelplpPH592n89qPaNcSZTm2rrjt+MrZNQ8a1aci4Ng0Z16ZhruPq7QZ/GdaKrzalo9MplJaW\nMXXWZsaM9GNCeyf+m1bNscKa9ew21pWkHD1ySyc1zXVcm6sOHTrc9GPUO7Pu5uaGpaUl2dnZtdqz\ns7Px8vKq8xovL686+6tUKry8vMjKyiIlJYXx48ejVqtRq9XMmzePxMRErK2t2bZt2028JCFuTLWh\n7g+MdlYqgiqr+HpzujFRb+Nnx8P3+WJhId8KCSGE+HM6d3LmmceCsNPUlG/U6RS+/DaNw/GXeLKN\nmof81HjaqnjQVy2rD0T9M+tqtZpevXoRHR3NQw89ZGyPjo5m9OjRdV7Tv39/ZsyYQVVVlXHd+tat\nW/Hx8aFNmzbodDoSExNrXbNq1Sq2bdvGd999R5s2ba4ZT3h4eINemKjfb5+eZUwhJvUSnx7N4J9D\ngvFysKl1X5a2mJX/jsHGRoONDXh7OrL4zcE4/qHf78nYNg0Z16Yh49o0ZFybRksZ1/Bw6NevmPlL\n96DNLgFgV2wRDk6evDg2nL8BlrdwQqiljGtzU1h48yfVNqgazNSpU/n000/517/+xfHjx5k0aRJZ\nWVm88MILAMycObNW6cWxY8diZ2fHuHHjSEpK4ptvvmHJkiVMmzYNACsrK0JDQ2v98fDwwMbGhpCQ\nEOzs7G76hQnREOXVepbGprI0NpVL5dUs3n+Oav2VDT3FJZXMX7rXuGPf0cGG2dMir5uoCyGEEA3h\n4+XI4jeGENy+tbHt/7ae4u2V+9H9obTjby4UlvPt8WwMDVw+LMxfg5L1MWPG8O6777JgwQJ69OjB\n/v372bJlC35+fkDNhtJz584Z+zs5OREdHU1mZia9e/fm5ZdfZvr06UyePLlpXoUQN+BMfhmTth4n\nJvWSsa2sWk9eRU2JxqpqPYuW7yMruxgAtZUlM6dE4O3laJJ4hRBCtDxOjjbMnRFFv3A/Y9vB+Aze\nWLSLgsKKWn0rdQb+eeAcHyek89buMxRcfr8SLZtKaejOThP6/VcIzs7OJoykZbmdv/IqrtQxflMi\n5borMxdDAlvzt17+aNSWGAwKy1bHsv+XtMv3qpj+cn/69/ar+wH/4HYe26Yk49o0ZFybhoxr02ip\n42owKHy24Vd++OmEsc3T3Z450+/E5/Ik0X9TtHx6NMN4fytbNX/vH0iYp9NNP39LHVdTa4wctkEz\n60K0NI42VjzZteaAClsrC6b1C2Rqv0A06prNPuv/e+x3iTqMe6xbgxN1IYQQ4s+ysFAxbmwYzz7Z\nw7ipNPtiKTPmbif5xEUAHujowUOdrpTGzquoZtaO02xIyjJJzOLWkGRd3LZGdnTnoU6erBgewuDA\nK+sFf445w3f/d9x4e8Rd7bn/7o6mCFEIIcRt5p6hHZgxKQLry5NHJaVVzF2ym30H01BbWjChux9z\nB7bH2aamRoiCgout2pQhiyYmybpo8a610kulUjGhux++jrbGtvijWXy47rDxdngPH555ooeUzhJC\nCHHL9O7pw7zXo3B2qnl/qtbpWbrqAN/+33EURSHc25n37w4lzNOROwNcGd62dT2PKMyZJOuiRcst\nq2LmjlPsuZBfb9+zqfm88/4BY3LfNtCVqX/rJ7XUhRBC3HId2rVm8ZtDjOvVAT7f+CsfrjuMXm+g\nlUbN/KgOTOoTKBNKLZwk66LF+iWzkJd/TuFYTjErD51He7n8Yl0u5payYOleKitrjnV2b23PrKkD\nsLU1ySG/QgghBJ7u9ix+Ywihwe7Gtp9jzrDkvf1UVOiwUKmwtao7lTuYUUB6UUWd9wnzIsm6aHGq\n9QY+PpLO3N2nKbqcfJfrDBzLKa6zf2lZFfOX7iW/sBwAO42aWdMicXWxrbO/EEIIcas4OFjzxqt3\nEtkvwNgWl5DJ7IU7yC+oOxnPKK7gnwfOMWlrCtvOXbrmclBhHiRZFy3O0thUvj2RbbztprFm4aAO\nDG3rdlXfap2et1ccIC2jprSSpaUFr02KIMBPSoQKIYRoHqzVlkx+oS8P3hdibDubms+Mudu5kF77\nhExFUVh+8DwVOgMVOgPLD6ay7GAq5dc4ZEk0f5KsixbnwU6eWF1ev9fHx5kVd4fQ1ePqg4wURWHt\nJ/H8mnwlsZ/4bDhdQz1uWaxCCCFEQ1hYqHhiTFeeH9fLuEb94qVSZs3fQdLxi8Z+KpWKieH++P2u\neEJMah6v/JzC2fyyWx63uHmSrIsWp2Nre57p4cdzPfx5Y0A7Y3mrP/rqu2R27Ek13n70wS5ERQTe\nmiCFEEKIGzB8cDtmTY3E5vJ7W2lZFW8t2cWufeeNfYJc7HhveCeGBl2pEpNbVo2FbEQ1S5KsixZp\nZEcP/hLscc0d8jv2prLx2yTj7cEDghj9l5A6+wohhBDNSc8wb+bPGoSrswYAvd7Aex8c5Osfko3r\n022tLJncN5Dp/YLQWFny155+BLpoTBm2uEGSrAuzpCgKm09d5OMj6X/62l+Tsln9rzjj7W6hnrww\nvpeUvhJCCGE22gW6svjNwfj7Ohnb/vN1Imv+HY9OZzC2RQW24oN7QhnR7up9W8I8SLIuzE5xpY6F\n+86yJv4C357IZn9a/TXUf3MhvZB/rtiPXl/ziyzAz5npr/TH6hqlr4QQQojmyt3NngWzB9faa7Vt\n11kWLt9LeUW1sa21nfU1J6QOpBdgkGoxzZpkKMKsJF8s4eWfU9ifXmBs23TqYoPKUuUXVDB/6R7K\nymt+gbk6a5g9bQD2dtZNFq8QQgjRlBzsrZn99wEMjGhjbEs4pmXW/B1cyrv+htIdqZeYv/cMM2NO\nUVAlCXtzJcm6MBsH0gt4LeYkF8uqjG33dXBn7sD29S5hKa+oZsGyPeReqvnFZWtrxaxpkbi1tmvS\nmIUQQoimpray5JXn+jDmgVBjW+qFAl6bu53zaQV1XpNdWsmquDQAEi8Ws/xUBclFUt6xOZJkXZiN\nrh4OuF+eBXe0tmJ2ZDv+1isAa8vr/xjr9QaWrz7I2dSa5TIqlYq/T+xP20DXJo9ZCCGEuBVUKhWP\nPtiFic/0xsKiZgIrL7+c1+ft4GhS9lX93e2seaiTp7FCTJkOPjlXxT/3n6NCJ0l7cyLJujAbDtZW\nvNo/kK4ejqwYHkJ/P5d6r1EUhX+tTyAuIdPY9tzTPekZ5t2UoQohhBAmMWRgEHP+PgBb25rSjuUV\n1cx7ezcxu8/V6mehUvFYF28WDuqAm+bKctCcsips6pkEE7eW/GuIZqegoprkiyV13tfJzYFFgzrg\nYd+wdeY/bDnJT9tPG28/cG8nhg9u1yhxCiGEEM1RWBcvFs4eTCvXmlKNBoPC+x8fYsM3SVft8erq\n4cjKu0Po5mIJwDPdfaU6WjMjybpoNip0BjYmZfHs5iQW7Tt7za/hGvpLZP8vaazbcNR4O6KvP0+M\n7toosQohhBDNWWCAC0veHEJgwJVvob/6LomVHx2i+g/vr042VjzZxpq/B9sQ4uZQ5+NdKCxvUDEH\n0fgkWRcmZ1AUtp7N5fn/S+KzY5mU6/TkVVTzzfGcG37M46dyeW/tL8bbnTq68fJzfYzr+IQQQoiW\nrnUrO+bPGkT3rl7Gtp17U1nwzl5Kf1es4TeetnWnhQUV1UyNPsGU6BMcyylusnhF3SRZFya3/GAq\n7/1yntzyK784/J1s6XiDlVqytMUsWr7POHPg7enIzMkRWKstGyVeIYQQwlzYadS8PiWSIXcGGdt+\nTc7m9Xk7jBXS6vOfxCzKdXpO5ZUyI+Yk/9h9mrSiiqYKWfyBJOvC5IYEtjb+3cVGzUvhAay6O5Rw\nb+c//VhFxZXMW7qH4pJKABwdbJjz9wE4Otg0WrxCCCGEObGysuDFZ8IZ+3AXY1taRiGvvbXdWCnt\nehysLbG2uJIyHsws5MUtyWw+dbFJ4hW1WZk6ACG6ezkR6e9KgLMtDwZ7ornBGfCqaj2L392HNrtm\nc6raypLXp0bi5Vn3+jshhBDidqFSqXh4ZChure1Y9XEcer2B/MJyZi/cwd8n9r/utU9182V4OzfW\nH8skJjUPAEWBzu72tyL0257MrItboqxaz/pjmRRX6uq8f8YdQTzexeeGE3WDQeG9tb9w/FTu5RYV\nk//Wl+D2ra97nRBCCHE7iYoI5I3pd2KnUQNQUaFjwbK9HErIu+51nvY2TOsXxHvDQgjzdOSuoFYE\nucjBgrdCg5P11atX07ZtWzQaDeHh4ezdu/e6/RMTE4mKisLOzg5/f3/mzZtX6/5vv/2W4cOH4+Hh\ngZOTE/369WPTpk039ipEs6UzKGw6mcMzmxP5MimLjcnaOvvdbJmo9V/9yoFDacbb48eG0b+33009\nphBCCNESdQ31YOGcwbi3rpkZVxSF77ZksnWnFr3ecN1r27eyY0FUB14MD6jz/pzSKuKyCqVyTCNq\nULK+ceNGJk+ezOzZs0lISOCOO+5gxIgRpKen19m/uLiYoUOH4u3tTXx8PO+99x5vv/02y5cvN/bZ\ntWsXQ4YM4ccffyQhIYF77rmHUaNGsW/fvsZ5ZcKkFEVhX1o+L25JZu3hNIouz6hvPpVDTunVO9Bv\nxs8xZ/juxxPG2yPuas99wzs06nMIIYQQLUmAnzOL3xxC2zZXTvPedSCX8S/9wMqPfuHQ4Uyqqq5d\nQvlap4d/fiyDN3edZvbO05zJb9gGVnF9KqUBH3369etH9+7dWbt2rbGtY8eOjB49mgULFlzVf82a\nNcycOZOcnBysrWsOr1mwYAFr164lLS3tqv6/6du3L3feeSdvv/12rfbCwkLj352d//ymQ1G3uLg4\nAMLDwxv9sVMLypn4U3KtNg87a57u5sudbVyNxxvfrLiETBYt32f8BB/ew4cZkyJMXqKxKcf2dibj\n2jRkXJuGjGvTkHFtXOUV1SxbFcvu/WcAsLe/srTFxsaKnt286NvLl17dvbG3u/6BhKfzypi0NcV4\nW4WKQYGuPNnVt8GHGbY0jZHD1juzXl1dTXx8PEOHDq3VPmzYMPbv31/nNbGxsQwYMMCYqAMMHz6c\nzMxMzp8/f83nKi4uxtXV9Zr3C/MR6KLhzoCaf0sHtSUTwvz44N7ORAW2arRE/UxqPktXxRoT9XZB\nrZj6t34mT9SFEEIIc6GxVTNjcgSDIz1wcqxdd6SyUseBQ+m8u/Yg4yb+wD/e3s3PMWcoKKy7bKOb\nnZoR7dyN7/MKCjGpeby4JZnSa8zSi/rVWw0mNzcXvV6Pp6dnrXZPT0+2b99e5zVarRZ/f/+r+iuK\nglarpU2bNldds2rVKjIyMnjyySf/TPyiGXu6my+tNdaMCfXCyaZxCw9dzC1lwdI9VF5eXuPe2p7X\np0RiaysFjoQQQog/w9LSgiEDPBgU4Y5r67YciMvgYFwGWdlXDkDS6w0kHNOScEzLB58eplOH1vTt\n5UvfcF+8PGqqrrnYqnmpdwAjO7rz6dEMDmbWzCoPbdsae2s56+RGNYvM5n//+x+vvfYaX3311VVJ\n/h/99vWXaDw3OqbVBoU9uTpKdXC/j7rOPt2Bk8fq3lR6o8or9Hz4+VlycmtqqWtsLfjLcB/OnE5s\n1OdpDPLz2jRkXJuGjGvTkHFtGjKujc/CQkVh/jlC20FIWzdych1JOVVE0okiMrW1Z9PjE0qJT7jA\n6n+Bl4ctoR2dCA12wsvdBpVKxT12EOylZ1u2jk5VWuLisk30qkyrQ4eb30NXb7Lu5uaGpaUl2dm1\nBzk7OxsvL686r/Hy8qqzv0qluuqar7/+mqeffpr169dzzz33/Nn4hQkYFIXD+Xp+0uoorFZQqaB3\nK0u8rnFMcWPS6Q3855sLxkTd0gLGPtgGDzfbJn9uIYQQ4nahUqnwdLfF092WqDs8yC+sIuVkEUkn\nizifVsbvdzxqcyrQ5lQQszeHVi5qY+Ie5GPH8+2ufSjhlqxqOjlZEGQvs+7XU2+yrlar6dWrF9HR\n0Tz00EPG9ujoaEaPHl3nNf3792fGjBlUVVUZ161v3boVHx+fWktgvvrqK8aPH89nn33GqFGjGhSw\nbChpPDeySSdBW8S/EjI4W1AG1mp+22uSauPOfdco49RYFEVh5UeHyM5VjBtgXnm+D1ERgU36vDdC\nNkA1DRnXpiHj2jRkXJuGjGvTaMi4Dh1S89/CogrijmRxMD6DhEQtOt2Vco+V1XAkqZwjSeW4ONvS\nu6cP/Xr50SXUHbXVlaT8WE4xsadPElsKd/i5MC7MF1/Hljfx9vsNpjeqQctgpk6dylNPPUXv3r2J\niIhgzZo1ZGVl8cILLwAwc+ZMDh06xLZt2wAYO3Ys//jHPxg3bhyzZs3ixIkTLFmyhLlz5xofc8OG\nDTz11FMsXbqUyMhI40y8tbW1bDJtxvam5dck6pe52KgZ28Wb4e3cmvy5N36bzM69qcbbjz3UpVkm\n6kIIIURL5uxky5CBQQwZGER5RTWHj2qJjUsn/mgWFRVXDj8sKKwgesdZonecxU6jpld3b/r28qVH\nNy8+P5Zp7Lc/vYCDGYWMaO/GY529cbGte2nt7apByfqYMWPIy8tjwYIFZGVl0aVLF7Zs2YKfX82h\nM1qtlnPnzhn7Ozk5ER0dzcSJE+nduzeurq5Mnz6dyZMnG/t88MEH6PV6Jk+eXKt94MCBxMTENNbr\nE9dgMCgcSsjj+KliNm8vbfB1FUCONShAOz0E6eHQyXwONVmkNfR6A0nHLxpvDx4QxMMjQ5r4WYUQ\nQghxPRpbNRF9/Yno609VtZ7E5Bxi4zL45XAGRcWVxn5l5dXsOXCBPQcuoLaypH1XD9r4OHBOp8fC\nUoVeUdh86iJONlY83sXHhK+o+WnwBtMXXnjBOJP+R5988slVbZ07d2bnzp3XfLwdO3Y09KlFIzt+\nKpeP1h0mMaXmU629/dXllAwqsLhGBX4XBzW2FTqqdQrJdXdpUt06e/LC+F43feqpEEIIIRqPtdqS\nnmHe9Azz5oXxvTh+KrcmcY/PICf3ysRgtU5PypEsOAIqjRWVHVtR4WiNj6uGB4M9r/MMt6dmUQ1G\n3BoFhRV8tvHXWktJ/kgB8l1tyHXT4J9WjKbi6kTesaS66YKsR2CAC9Nf7o+VVdNvZhVCCCHEjbGw\nUBEa7E5osDvjx4aReqGQ2Lh0YuMySMu4so7btlyHzdEcVA5q8lQq3ki8RL/LJSH9fJyAmj1rt/ME\nnSTrtwG93sCWbaf58n9JlFdcSbStrFTc2c+d4Xf1RFEUkgrK2ZSRj0VFNR6AbycPJgZ7Npv/Qays\nLOjYrjXWUqtVCCGEMBsqlYqgNi4EtXHhsYe6kKUtJja+Zsb9xOlLqLgyEXimOI8z5/L44utj+Ho7\n0S/cF02gC4dLKnimhy9BLnbXf7IWSJL1Fi7p+EU+XHe41qdYgL69fOnZWYWrizVeQa4sO5hK0sUS\nUIFGU7Oxo9raEv/2rXGr53hhIYQQQoiG8vZyZNS9nRh1byfy8sv55XAGsXEZJKbkYDBcWYObkVXE\n15uKONvWGb29mm3JWga1acXkge3wbIGVY65FkvUW6lJeGes2/Mre2Au12r09HXn2yR706OZlLNNk\nr7bkwu+ODrazsmRMqBcjO3pgI8tNhBBCCNFEWrlquHtIe+4e0p6SkirijmbyS3wmh49mUVWtp8zO\nikobS9AZKCiq5NtjWWxK1DK8TSveebR7s/n2vylJst7CVOv0bP75FF99l0xl5ZXySTY2Vox5IJT7\nhneoVecUwNHGikdCvfjkaAb3tHfn0c5eUjZJCCGEELeUg4M1URGBREUEUlmpIyExm9i4dPYmZXPO\nUU2JQ01uolMU0quqb4tEHSRZb1GOJmr56LMjZGqLa7X36+NHl0FtibtUgv7URR4Oufrk2Xs7uNPH\n17lFHkgghBBCCPNiY2NF316+9O3ly0SdgaQTF/k29gI/agsoslAxqlvd5R0rdHpsLC1aVCIvyXoL\nkJNbyqf/OUpsXLqxTQGc2zjj39+f+PIqdh3LACC9uJKHOl29adTa0kISdSGEEEI0O1ZWFoR19iSs\nsyez9Qa+OZzOiNCrJx4BPvs1k9iMQgYHtmJwYCt8WkBuI8m6Gauq1vPd/53gm00pVFVfKbGosVUz\n5L5gvi4t52JRea1rMoorOF9YQaCL5laHK4QQQghxU6wsLRjTO6DO+3QGhZ3n8yis1PFlUhZfJmUR\n4ubAkMBWRLVphUZtntXkJFk3U4eOZPLv9UfIvlj79NGoyECeHNMNVxdbjvyUwtmCMgA87W0YEtSK\nQW1axqdMIYQQQojfSy+qQG+ofaJjSm4Jx3NL6e3jLMm6uDW02SX8+4sEDiVkUuKgptDXAfeL5QR7\nOfLXp3oS0tHN2Pee9m6cyitjSFBrQt3sW9T6LSGEEEKI3wt00fD5A934JbOQmHOXiMsqQq8odPd0\nrLMMtaLUJPbNPT+SZN1MVFbq+PqHFDbEnOaSg5qiDi7orCywsFAR1cOHxQ+HYWFR+4dtRHt3Rpgo\nXiGEEEKIW83a0oJIf1ci/V0pqKhm94V8fBxt6uz7a04Ja+PTGHx55UFzPVdGkvVmTlEUYuMygJu2\n7gAAD75JREFU+OQ/CZw06NEGOBrvc3K0oZWrhmwbS2jeHwqFEEIIIW4pF1s1Izt6XPP+mNRLXCgq\n59OjGaw7mkmYpyNDglrR39elWS2ZkWS9GUvPLOLjz4/wa1I2AA7qmgOKbGwscW9tj7ezLYMCWzE4\nsDUWzfwrHCGEEEKI5sKgKBzOKjLeVlBIyC4iIbuIieEB3NPe3YTR1SbJejNiUBQSL5Zw4EI+tqfy\n2PTzqVrH7ra2UeMe4ErntjUJencvR0nShRBCCCH+JAuVig/v7cyBjAK2n8vjaHYxCgpqCxUD/F1N\nHV4tkqw3A+lFFcSkXiImNY9zOSXk5pUTcDofjeHKxoe7h7TjsQe7YG+vbvYbIYQQQgghmjuN2pLB\nga0ZHNia3LIqdpzPo6hSh6PN1elxtd7A3N1n6OPrzJ0Brrf0pHdJ1k1s+cFUtp27RFWVntxLZZRX\n6AAocLFBoy2jU0c3/vpkT4LauJg4UiGEEEKIlsnNzprRdZzw/pvD2iKOZNf8+fhIOuHeTgwJak1v\nH2esLS2aNDZJ1k3MW6Mm91IZhUWVAFjqFZyKKgnQwd+e78PAO9rITLoQQgghhAntSM0z/l2vKBzM\nLORgZiF3+LkwK7Jdkz63JOtNTFEUTuaVkV9eTT+/K7PjBoPCzr2pbP46kSIvWxxLqnEurMSpVMfI\nYR0ZMyoUO82t+4pFCCGEEELUbWJ4AGGejmxPzSMlt8TYHuHX9OvbJVlvItmllexMzWN7ah4ZxRW4\naazp4+uMhUrFmdR8Pv7sMCdOXwKgY1E5lgboGurBM0/0IMDP2cTRCyGEEEKI3zjaWNWcX9Penczi\nCmJS84jNKKg1Eft7X6docbK2oq/bzU+8SrLeyKr0Bt7cdZpfc4prteeWV3EgNY9jO1PZuuMscKXK\ni7uzhvFju3NHHz9Z8iKEEEII0Yz5ONryRFcfnujqU+f9FTo9G5K0lOv0/GdE+5t+PknWG5m1pQV6\nRanVprGywEsH7y7fh76gwthuaWnBX0YE8/DIEGxt5Z9CCCGEEMLcHUgvoFynb7THkwyxCQwJbE1K\nbik9vJzoaG1F/JZTnE/Nr9WnRzcvnnmiBz5ejtd4FCGEEEIIYW56eDnx1x5+bD+XV3/nBpBkvQnc\nGeBKBwcbfvzhON/sOVfrPg83e555ogfhPbxlyYsQQgghRAvjYqvmgWBPHgj2pLCw8KYfr8GFIVev\nXk3btm3RaDSEh4ezd+/e6/ZPTEwkKioKOzs7/P39mTdv3lV9du3aRXh4OBqNhvbt2/PBBx/8+VfQ\nzOj1BmJ2nOWNN7YR87tEXW1lyZgHOrNi8d307ukjiboQQgghhKhXg2bWN27cyOTJk1m7di0RERGs\nWrWKESNGkJKSgp+f31X9i4uLGTp0KFFRUcTHx5OSksK4ceNwcHBgypQpAKSmpnLvvffy7LPP8sUX\nX7Bnzx5efPFFPDw8GDVqVOO+ylso+2Ipn/wnAYPhyrr1Pj19Gf94dzzd7U0YmRBCCCGEMDcNStaX\nL1/OhAkTmDBhAgArVqzgp59+Ys2aNSxYsOCq/uvXr6e8vJx169ZhbW1NSEgIKSkpLFu2zJisr1mz\nBl9fX959910AgoODOXjwIO+8845ZJ+s+Xo7cP7wj3285gbenI8880Z2eYd6mDksIIYQQQpihepfB\nVFdXEx8fz9ChQ2u1Dxs2jP3799d5TWxsLAMGDMDa2trYNnz4cDIzMzl//ryxz7Bhw2pdN3z4cOLi\n4tDrG28HrSmMGRXKuMfCWL5wmCTqQgghhBDihtWbrOfm5qLX6/H09KzV7unpiVarrfMarVZbZ39F\nUYzXXKuPTqcjNzf3T72I5kZjq2bkiGCs1ZamDkUIIYQQQpgxs6sG0xi7akWNDh06ADKmTUHGtmnI\nuDYNGdemIePaNGRcm4aMa/NV78y6m5sblpaWZGdn12rPzs7Gy8urzmu8vLzq7K9SqYzXXKuPlZUV\nbm5uf+pFCCGEEEII0RLVm6yr1Wp69epFdHR0rfbo6GgiIiLqvKZ///7s2bOHqqoqY9vWrVvx8fGh\nTZs2xj5/fMytW7cSHh6OpaUsHxFCCCGEEEKlKIpSX6evvvqKp556ilWrVhEREcGaNWv45JNPSE5O\nxs/Pj5kzZ3Lo0CG2bdsGQFFREZ06dSIqKopZs2Zx4sQJxo8fz9y5c5k8eTJQU7qxa9euPPvsszz/\n/PPs3buXl156iQ0bNvDAAw807asWQgghhBDCDDRozfqYMWPIy8tjwYIFZGVl0aVLF7Zs2WKssa7V\najl37soBQE5OTkRHRzNx4kR69+6Nq6sr06dPNybqAIGBgfz4449MmTKFtWvX4uPjw8qVKyVRF0II\nIYQQ4rIGzawLIYQQQgghbr1616yb0qJFi+jTpw/Ozs54eHgwcuRIkpKSTB1Wi7No0SIsLCx45ZVX\nTB2K2dNqtYwbNw4PDw80Gg1dunRhz549pg7LrBkMBubMmUPbtm3RaDS0bduWOXPmYDAYTB2a2dmz\nZw9/+ctf8PPzw8LCgs8+++yqPm+99Ra+vr7Y2dkxaNAgkpOTTRCpebneuOp0Ol577TXCwsJwcHDA\nx8eHxx9/nLS0NBNGbB4a8vP6m+effx4LCwuWLVt2CyM0Tw0Z15MnT/LQQw/h6uqKvb094eHhnDhx\nwgTRmo/6xrW0tJSXX34Zf39/7Ozs6NSpk/Fg0Po062R99+7dvPTSSxw4cIAdO3ZgZWXFXXfdRUFB\ngalDazFiY2P56KOPCAsLM3UoZq+wsJCIiAhUKhVbtmzh+PHjrFy5Eg8PD1OHZtYWL17MmjVreP/9\n9zlx4gQrVqxg9erVLFq0yNShmZ2SkhK6du3KihUrsLOzu+r+JUuWsHz5clatWkVcXBweHh4MHTqU\n0tJSE0RrPq43rmVlZSQkJDBnzhyOHDnCDz/8QFpaGiNGjJAPnPWo7+f1N19//TWHDh3C19f3FkZn\nvuob19TUVCIjI2nXrh07d+4kKSmJ+fPn4+DgYIJozUd94zplyhS2bNnCF198wfHjx5k9ezYzZszg\niy++qP/BFTNSUlKiWFpaKps3bzZ1KC1CQUGB0q5dO2Xnzp1KVFSU8vLLL5s6JLM2c+ZMJTIy0tRh\ntDj33XefMm7cuFptTz/9tHL//febKKKWwcHBQVm3bl2tNm9vb2XRokXG2+Xl5Yqjo6Py4Ycf3urw\nzFZd4/pHycnJikqlUhITE29RVObvWuOampqq+Pn5KcePH1cCAwOVpUuXmiA681XXuI4dO1Z54okn\nTBRRy1DXuHbp0kV56623arUNHDiwQblXs55Z/6OioiIMBgOurq6mDqVFeO655xgzZgwDBw40dSgt\nwvfff0/fvn159NFH8fT0pEePHqxatcrUYZm9yMhIduzYYfwKNjk5mZiYGO69914TR9aynDt3Dq1W\ny9ChQ41ttra23Hnnnezfv9+EkbU8hYWFqFQqeS+7SXq9nrFjxzJnzhyCg4NNHU6LoCgKmzZtIjQ0\nlBEjRuDh4UGfPn346quvTB2a2YuMjGTTpk2kp6cDsH//fo4ePcqIESPqvdasTjCdNGkSPXv2pH//\n/qYOxex99NFHnD17li+//NLUobQYZ8+eZfXq1UyZMoWZM2eSkJDASy+9hEql4sUXXzR1eGbrtdde\no7i4mNDQUCwtLdHr9cyaNYvnn3/e1KG1KFqtFpVKhaenZ612T09PMjMzTRRVy1NdXc20adMYOXIk\nPj4+pg7HrL3xxht4eHjw3HPPmTqUFiMnJ4eSkhIWLlzI/PnzWbJkCdu3b+fxxx/H0dGxQYmlqNuK\nFSt4/vnnCQgIwMrKCpVKxcqVK1tWsj516lT279/Pvn37UKlUpg7HrJ08eZJZs2axb98+LCzM6suV\nZs1gMNCnTx8WLFgAQFhYGCdPnmTVqlWSrN+EDRs28Pnnn7NhwwZCQ0NJSEjglVdeISgoiPHjx5s6\nPCEaTK/X8/jjj1NUVMTmzZtNHY5Z27lzJ+vWrePo0aOmDqVF+W0fxQMPPMCkSZMA6NatG3Fxcbz/\n/vuSrN+EFStWcODAATZv3kxAQAC7d+9m2rRpBAYGMmzYsOteaxaZ2pQpU9i4cSM7duwwnoAqbtyB\nAwe4dOkSoaGhqNVq1Go1u3btYtWqVVhbW1NdXW3qEM2St7c3ISEhtdpCQkK4cOGCiSJqGV599VWm\nT5/O6NGj6dy5M48//jhTp06VDaaNzMvLC0VRyM7OrtWenZ2Nl5eXiaJqOfR6PY8++iiJiYnExMTI\nEpibtGvXLrRaLV5eXsb3sfPnz/Pqq68SEBBg6vDMlpubG1ZWVvJe1sgqKip4/fXXefvtt7nnnnvo\n0qULL774Io8++ijvvPNOvdc3+5n1SZMm8d///pedO3fSoUMHU4fTIowaNYrevXvXahs3bhwdO3Zk\n1qxZqNVqE0Vm3iIiIq4qbXXixAn5gHmTysrKrvoGyMLCQippNLKgoCC8vLyIjo6mV69eQM0bzJ49\ne1i6dKmJozNvOp2ORx55hOTkZHbt2oW7u7upQzJ7EydOZPTo0bXahg0bxtixY/nrX/9qoqjMn1qt\npnfv3le9l508eVLey25CdXU11dXVV72XWVpaNui9rFkn6xMnTmT9+vV8//33ODs7G2d8HBwcsLe3\nN3F05svJyYnQ0NBabfb29rRq1eqqT9Oi4aZMmUJERAQLFy7kkUce4fDhw6xcuZLFixebOjSzdv/9\n97N48WICAwPp3Lkzhw8fZvny5YwbN87UoZmd0tJSTp8+jaIoGAwGLly4wNGjR2nVqhX+/v5MnjyZ\nRYsWERwcTIcOHZg/fz6Ojo489thjpg69WbveuPr4+PDwww8THx/Ppk2ban174ezsjK2trYmjb77q\n+3l1c3Or1V+tVuPl5SUTe/Wob1xfffVVHnnkESIjIxk8eDAxMTFs3LiR77//3tShN2v1jevAgQOZ\nMWMG9vb2tGnThp07d/LZZ581aGa9WZduVKlUioWFxVV/5s6da+rQWpxBgwZJ6cZG8OOPPyphYWGK\nRqNRgoODlffff9/UIZm9kpISZcqUKUpgYKBiZ2entGvXTpk9e7ZSWVlp6tDMzs6dO+v8vTp+/Hhj\nn7lz5yo+Pj6KRqNRoqKilKSkJBNGbB6uN66pqanXfC+rr8Tj7a4hP6+/FxQUJKUbG6Ah47pu3Tql\nY8eOip2dnRIWFqZs3LjRhBGbh/rGNTs7W5kwYYLi5+en2NnZKSEhIcqyZcsa9NgqRVGUpv+8IYQQ\nQgghhPizzGKDqRBCCCGEELcjSdaFEEIIIYRopiRZF0IIIYQQopmSZF0IIYQQQohmSpJ1IYQQQggh\nmilJ1oUQQgghhGimJFkXQgghhBCimZJkXQghhBBCiGZKknUhhBBCCCGaqf8HPQdVMh11H50AAAAA\nSUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import random\n", + "import thinkplot\n", + "\n", + "random.seed(17)\n", + "\n", + "d6 = Die(6)\n", + "\n", + "dice = [d6] * 3\n", + "three = thinkbayes.SampleSum(dice, 1000)\n", + "three.name = 'sample'\n", + "\n", + "\n", + "three_exact = d6 + d6 + d6\n", + "three_exact.name = 'exact'\n", + "\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(three)\n", + "thinkplot.Pmf(three_exact, linestyle='dashed')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Pmf.__add__` is based on the assumption that the random selections from\n", + "each Pmf are independent. In the example of rolling several dice, this\n", + "assumption is pretty good. In other cases, we would have to extend this\n", + "method to use conditional probabilities.\n", + "\n", + "The code from this section is available from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Maxima\n", + "\n", + "When you generate a *Dungeons & Dragons*\n", + "character, you are\n", + "particularly interested in the character’s best attributes, so you might\n", + "like to know the distribution of the maximum attribute.\n", + "\n", + "There are three ways to compute the distribution of a maximum:\n", + "\n", + "**Simulation**:\n", + "\n", + " Given a Pmf that represents the distribution for a single selection,\n", + " you can generate random samples, find the maximum, and accumulate\n", + " the distribution of simulated maxima.\n", + "\n", + "**Enumeration**:\n", + "\n", + " Given two Pmfs, you can enumerate all possible pairs of values and\n", + " compute the distribution of the maximum.\n", + "\n", + "**Exponentiation**:\n", + "\n", + " If we convert a Pmf to a Cdf, there is a simple and efficient\n", + " algorithm for finding the Cdf of the maximum.\n", + "\n", + "The code to simulate maxima is almost identical to the code for\n", + "simulating sums:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def RandomMax(dists):\n", + " total = max(dist.Random() for dist in dists)\n", + " return total\n", + "\n", + "def SampleMax(dists, n):\n", + " pmf = MakePmfFromList(RandomMax(dists) for i in xrange(n))\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "All I did was replace “sum” with “max”. And the code for enumeration is\n", + "almost identical, too:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def PmfMax(pmf1, pmf2):\n", + " res = thinkbayes.Pmf()\n", + " for v1, p1 in pmf1.Items():\n", + " for v2, p2 in pmf2.Items():\n", + " res.Incr(max(v1, v2), p1*p2)\n", + " return res" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In fact, you could generalize this function by taking the appropriate\n", + "operator as a parameter.\n", + "\n", + "The only problem with this algorithm is that if each Pmf has $m$ values,\n", + "the run time is proportional to $m^2$. And if we want the maximum of\n", + "`k` selections, it takes time proportional to $k m^2$.\n", + "\n", + "If we convert the Pmfs to Cdfs, we can do the same calculation much\n", + "faster! The key is to remember the definition of the cumulative\n", + "distribution function: \n", + "\n", + "$$CDF(x) = {{\\mathrm{p}(X \\le~x)}}$$\n", + "\n", + "where $X$ is a random variable that means “a value chosen randomly from\n", + "this distribution.” So, for example, $CDF(5)$ is the probability that a\n", + "value from this distribution is less than or equal to 5.\n", + "\n", + "If I draw $X$ from $CDF_1$ and $Y$ from $CDF_2$, and compute the maximum\n", + "$Z = max(X, Y)$, what is the chance that $Z$ is less than or equal to 5?\n", + "Well, in that case both $X$ and $Y$ must be less than or equal to 5.\n", + "\n", + "If the selections of $X$ and $Y$ are independent,\n", + "\n", + "$$CDF_3(5) = CDF_1(5) CDF_2(5)$$ \n", + "\n", + "where $CDF_3$ is the distribution of\n", + "$Z$. I chose the value 5 because I think it makes the formulas easy to\n", + "read, but we can generalize for any value of $z$:\n", + "\n", + "$$CDF_3(z) = CDF_1(z) CDF_2(z)$$ \n", + "\n", + "In the special case where we draw $k$\n", + "values from the same distribution,\n", + "\n", + "$$CDF_k(z) = CDF_1(z)^k$$ \n", + "\n", + "So to find\n", + "the distribution of the maximum of $k$ values, we can enumerate the\n", + "probabilities in the given Cdf and raise them to the $k$th power. `Cdf`\n", + "provides a method that does just that:\n", + "\n", + "```python\n", + "# class Cdf\n", + "\n", + " def Max(self, k):\n", + " cdf = self.Copy()\n", + " cdf.ps = [p**k for p in cdf.ps]\n", + " return cdf\n", + "```\n", + "\n", + "`Max` takes the number of selections, k, and returns a new\n", + "Cdf that represents the distribution of the maximum of k\n", + "selections. The run time for this method is proportional to $m$, the\n", + "number of items in the Cdf.\n", + "\n", + "`Pmf.Max` does the same thing for Pmfs. It has to do a little more work\n", + "to convert the Pmf to a Cdf, so the run time is proportional to\n", + "$m \\log m$, but that’s still better than quadratic.\n", + "\n", + "Finally, here’s an example that computes the distribution of a\n", + "character’s best attribute:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3 9.84640042004852e-15\n", + "4 4.0321009720098687e-11\n", + "5 9.806069563928e-09\n", + "6 6.203232264630567e-07\n", + "7 1.7470129795277652e-05\n", + "8 0.0002855723336481055\n", + "9 0.002477241673570361\n", + "10 0.012844085693359392\n", + "11 0.04397964477539067\n", + "12 0.10559054189425404\n", + "13 0.18102225453979948\n", + "14 0.21201352141686403\n", + "15 0.19422452755628616\n", + "16 0.14145216951432227\n", + "17 0.0786340868804698\n", + "18 0.027458253422613543\n" + ] + } + ], + "source": [ + "best_attr_cdf = three_exact.Max(6)\n", + "best_attr_pmf = best_attr_cdf.MakePmf()\n", + "best_attr_pmf.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Where `three_exact` is defined in the previous section. If we print the\n", + "results, we see that the chance of generating a character with at least\n", + "one attribute of 18 is about 3%. Figure [fig.dungeons2] shows the\n", + "distribution." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAEWCAYAAADfHdlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVmX+//HXzQ0oKCqKrBqIipK4Ja64K6C5ZJm2WGlm\n6kw5aY2V38n5zUz5bfpmaZvaNm7ZZGWZmqio4YbmkloquOMKKKIoiiw39+8P6hS5QAqcG3g/Hw8f\ndl1c55z3uSL63IdzrmOx2+12RERERETE4TiZHUBERERERK5PxbqIiIiIiINSsS4iIiIi4qBUrIuI\niIiIOCgV6yIiIiIiDkrFuoiIiIiIg1KxLiIiIiLioIpdrM+YMYPg4GDc3NwIDw9n48aNNxy7bt06\nBg0ahL+/P9WqVaNly5bMnj37mjFOTk6F/litVg4cOHDrZyMiIiIiUoEUq1hfuHAh48eP56WXXmLX\nrl106tSJvn37cvLkyeuOj4+Pp0WLFixatIi9e/fypz/9idGjR/PZZ58VGmexWEhISCAlJYWUlBSS\nk5Np3Ljx7Z+ViIiIiEgFYCnOG0w7dOhAq1atmDVrltEXEhLCkCFDmDJlSrEO9MADD5Cfn88XX3wB\nFFxZ79mzJ2fPnqV27dq3GF9EREREpOIq8sp6bm4uO3bsIDIyslB/VFQU8fHxxT7QxYsX8fT0LNRn\nt9sJDw/H39+f3r17ExcXV+z9iYiIiIhUdEUW62lpadhsNnx8fAr1+/j4kJKSUqyDLFu2jLVr1zJm\nzBijz8/Pj1mzZrFo0SK+/vprmjRpQq9evdi0adMfPAURERERkYrJubQPsGnTJoYNG8Y777xDmzZt\njP6QkBBCQkKMdvv27UlKSuL1118nIiKi0D4yMjJKO6aIiIiISKmpWbPmLW1X5JV1Ly8vrFYrqamp\nhfpTU1Px9fW96bYbN27k7rvv5pVXXmH06NFFhmnfvj0HDx4scpyIiIiISGVQZLHu4uJCmzZtiI2N\nLdQfGxt7zRXw31q/fj133303//rXvxg3blyxwuzcuRM/P79ijRURERERqeiKdRvMs88+y2OPPUbb\ntm2JiIhg5syZJCcnM3bsWAAmTZrEtm3bWL16NQBxcXH079+fp556igcffNC4Km+1WvHy8gLgrbfe\nIigoiGbNmpGTk8P8+fNZsmQJX3311U2z3OqvEORa27dvByA8PNzkJBWP5rZ0aF5Lh+a1dGheS4fm\ntXRoXktHSdzKXaxifejQoaSnpzNlyhSSk5MJCwsjJiaGevXqAZCSksLRo0eN8XPnziUrK4upU6cy\ndepUoz8wMJAjR44AkJOTw/PPP8/Jkydxc3OjWbNmLF++nOjo6Ns+KRERERGRiqDYD5iOHTvWuJL+\ne79/O+ns2bOv6fu9iRMnMnHixOIeXkRERESk0inWG0xFRERERKTsqVgXEREREXFQKtZFRERERByU\ninUREREREQelYl1ERERExEGpWBcRERERcVAq1kVEREREHJSKdRERERERB6ViXURERETEQalYFxER\nERFxUCrWRUREREQclIp1EREREREHpWJdRERERMRBqVgXEREREXFQKtZFRERERByUinUREREREQel\nYl1ERERExEGpWBcRERERcVAq1kVEREREHJSKdRERERERB6ViXURERETEQalYFxERERFxUCrWRURE\nREQclIp1EREREREHpWJdRERERMRBqVgXEREREXFQKtZFRERERByUinUREREREQelYl1ERERExEGp\nWBcREZFyIzXjOMfOJZKde9XsKCJlwtnsACIiIiJFuXI1ky/jPmT7/nUAfH9kOeFNutIxLIr63sEm\npxMpPSrWRURExKElHtvFgtXvkJF5zui7mnOFjT+tYONPK6jv3ZBOYVHcFdIFtyruJiYVKXkq1kVE\nRMQhZedeZcnGeWz4cXmhfjeX6mTlZhrtE2cOs3DtTL5e/x/uCulMx7AognxDsFgsZR1ZpMSpWBcR\nERGHczR5P5+seouzF04bfdXcahAeGMUdtZtQy8+NzXti2XUonjxbLgA5edls2beGLfvW4FfnDjqF\nRdG2aXfcq1Y36zREbpuKdREREXEYebZcVm79nFXbFmG35xv9YQ3a8mCvpziQcAiAxvXCaFwvjMFX\nR7E9cR3xe1aRfO64MT753HEWrfuIJRvn0bJxRzqFRdHQ/05dbZdyR8W6iIiIOITkc8eZv3I6J88e\nMfqquLoxuOso2t/Z87qFdrWqHnRr1Z+uLfuRlHKAzXtW8cOBjeTkZQOQa8the+I6tieuw9szgI7N\nImkX2gMP95pldl4it0PFuoiIiJgqP99G3K6lLItfYNzSAtAooBnDov5CnRo+Re7DYrHQwK8JDfya\ncG/XJ/jhwAY27VnJyTO/Fv5nzp/im41zWBb/Cc0btqNTsyhC7miBk0UrWYvjKvZ354wZMwgODsbN\nzY3w8HA2btx4w7Hr1q1j0KBB+Pv7U61aNVq2bMns2bOvOy48PBw3NzcaNWrE+++/f2tnISIiIuXS\nuYupvPPV31m8YY5RqDtbXRjU5XGeHvxysQr133Or4k5E82ief+hNJj70BhHN+1DF1c34ui0/j10H\n45mx+B+8POdPrNz6BRmZ6SV2TiIlqVhX1hcuXMj48eOZNWsWERERvPfee/Tt25eEhATq1at3zfj4\n+HhatGjBCy+8gJ+fHytWrGD06NG4ubnx4IMPApCUlES/fv0YNWoUCxYsYMOGDfz5z3/G29ube++9\nt2TPUkRERByK3W5ny741fLXuo0IvOKpXN5hHo8fjV+eOEjlOfe+GPNCzIYO6jGDngU3E711FUvJ+\n4+vnLqby7eYFxGz5L80ahNMpLIrQwNY4OVlL5Pgit8tit9vtRQ3q0KEDrVq1YtasWUZfSEgIQ4YM\nYcqUKcU60AMPPEB+fj5ffPEFAC+88AKLFy9m//5f/4N58skn2bdvH5s2bSq0bUZGhvHPNWvqHrOS\nsn37dgDCw8NNTlLxaG5Lh+a1dGheS4fm9cYuXr7AZ2veY8/RbUafk8WJyLb3E91uCM5WlxtuWxLz\nejrtGJv3xrItIY4r2ZnXfL1W9Tp0uLM3HZr1pnaNurd8nPJE36+loyRq2CKvrOfm5rJjxw4mTpxY\nqD8qKor4+PhiH+jixYvUr1/faG/ZsoWoqKhCY6Kjo5k3bx42mw2rVZ9oRUREKprdhzbz2dqZXM66\naPTVreXPo9HjCfINKZMM/l6BDO42ioERj7Hr0GY271nFoVN7ja9fyDzHiq0LWbn1c5oGtqZTWCRh\nDdpitepRPyl7RX7XpaWlYbPZ8PEpfM+Yj48Pa9asKdZBli1bxtq1awsV9ykpKURGRl6zz7y8PNLS\n0q45noiIiJRfV7IzWRT3EdsS4wr1d215NwMjhuPqUqXMM7k4u9K2aTfaNu3GmfOn2Lw3lu/3fUdm\nVsHVUDt2Eo79QMKxH/Bwr0X70J50DIukbi2/Ms8qlVepf0TctGkTw4YN45133qFNmza3vb9ffk0j\nJUdzWno0t6VD81o6NK+lQ/MKyReOsungUq7k/Ho13d3Vg06NB+BfPZgfd//0h/dZGvMaUDWMe1qF\ncjL9AAdSd5J84deVZC5ducDqHV+xesdX+NYMorFPa+6o0wSrU8W62q7v15LVuHHj295Hkd9hXl5e\nWK1WUlNTC/Wnpqbi6+t70203btxIv379eOWVVxg9enShr/n6+l53n87Oznh5eRU3v4iIiDioPFsu\nPxxbS2LytkL9wXXDaBscTRVntxtsaR6rk5VAr1ACvUK5dPU8h1J3c+jMbrJyLhljUjKSSMlIooqz\nG8HeLWjs05pa7qpdpHQUWay7uLjQpk0bYmNjGTx4sNEfGxvLkCFDbrjd+vXr6d+/Py+//DLjxo27\n5usdO3Zk8eLFhfpWrVpFeHj4Te9X14MPJUcPk5QezW3p0LyWDs1r6ajs83os5SDzV03nzPlTRl+1\nqh4M7fknWjfudMv7Let57UEktnwb+5J2EL9nFfuSfjDerJqdl0XC6e9JOP09wX6hdAyLpHXjCFNu\n6bldlf37tbT89gHTW1Ws3908++yzPPbYY7Rt25aIiAhmzpxJcnIyY8eOBWDSpEls27aN1atXAxAX\nF0f//v156qmnePDBB40r6Far1bhqPnbsWN577z0mTJjAmDFj2LhxI/PmzeOzzz677ZMSERERc9hs\neazc+gWrtn1B/s9FLUCzoHAe7P1nalarbWK6W2N1stI8uB3Ng9tx/lIa3+9bw5a9q0m/dNYYcyQ5\ngSPJCXy17iPaNO1Gp7BI6tUNNjG1VBTFKtaHDh1Keno6U6ZMITk5mbCwMGJiYow11lNSUjh69Kgx\nfu7cuWRlZTF16lSmTp1q9AcGBnLkSMH9X0FBQSxfvpwJEyYwa9Ys/P39eeeddxg0aFBJnp+IiIiU\nkZT0E8xfOZ0TZw4bfVVcqnJv1yfo2Kw3FovFxHQlw9PDiz7tHyCq7f3sP/Ej8XtW8dORreTn2wDI\nyrnCxh9j2PhjDHd4NyKiRR/ah/bQuu1yy4r9VMTYsWONK+m/9/u3k86ePfu6byz9vS5duuhBBhER\nkXIu357Pul3LWLppvvEWUoBg/1AeiXoGr5o3f8atPHJyshIa2JrQwNZcvHyBrQlr2bwnlrMZycaY\n42cOcXz1uxw6uYdHop6pEB9WpOxVrEeYRUREpEylXzzDJ7Fvc+jkHqPPanWmf8dh9Gg9sFJcUa5R\nrRa9w++jV5t7OXhyD5v3rGLX4c3YbHkAbEuMw6uWH33bP2ByUimPVKyLiIjIH2a329masJYv131E\ndk6W0R9QtwGPRj2Dv1eQeeFMYrFYCKnfnJD6zbk/6yJfb5jN1oTvAIjZ8l/q1vQlvGk3k1NKeaNi\nXURERP6QS1cusHDtTH48/L3RZ7E4ERk+mD7th+JsdTExnWOo5laDh3o9RUZmOvtP7AZgwep3qF3D\nm2D/UJPTSXniZHYAERERKT9+PPw9r37yTKFCvW5NP8YP+V/6dxqmQv03rFZnHu83Ed/a9YGClXI+\nXPYqZy8kF7GlyK9UrIuIiEiRsrIvs2DV23y07FUys35dO7pzi748P2waDfyampjOcblXqc6YgS9R\n3a0mAJezLvL+kle4cjXT5GRSXqhYFxERkZs6cOIn/r1gPN8nrDX6alarzZ8G/T+G9hhDFZeqJqZz\nfHVq+vDkgEnGbx3OnD/Fx9++VmjlHJEbUbEuIiIi15WTl81X6//Du19N5vxvXgDUpklXJj3yNqGB\nrU1MV7408GvKI1HPGO2DJ3/i87WzsNvtJqaS8kAPmIqIiMg1jqceYv6q6aSmnzT63Kt6MLTHGO4K\n6WxisvLrrpDOnL2QzLebFwCwZd8a6noGEBl+n8nJxJGpWBcRERGDzZbHqu2LWLn1c+OtnAB3Bt7F\nQ72fpmb12iamK/+i2t7P2QunjSUdl26ah1dNX1o37mRyMnFUKtZFREQEgNTzp5i/cjrHUw8afa4u\nVbm3y+N0CovSGzhLgMVi4cFefyb94hkOndoLwCcrp1Pbw4tA3xCT04kj0j3rIiIilVy+PZ91u5bx\nfwsmFCrUg/1CeeHhaUQ0j1ahXoKcrS480f9F6tbyByDXlsMHS/+X9ItnTE4mjkjFuoiISCV2NSeL\nmYv/yaJ1H5FrywHA6uTMgIjH+Mv9r1C3lp/JCSumalU9GDPwJdyregAFL5p6f8krZGVfNjmZOBoV\n6yIiIpXYkk3z2H98t9H29wrirw9OJTL8PpycrCYmq/i8Pf15sv+LWJ0K7kpOPnec2TFTsf3mWQER\nFesiIiKV1IkzR9j000qj3fOuQTz3wOsE1A0yL1Ql0zCgGQ9HPm20E4/t5Mu4D7WkoxhUrIuIiFRC\n+fZ8voh7H7s9H4Cmd7Tins7DcXF2MTlZ5dO2aXf6tHvAaG/6aQVxO5eamEgciYp1ERGRSmhbQhxJ\nyfuBgnvU7+/+pB4iNVHfDg/SJqSL0V68YTY/HdlqYiJxFCrWRUREKpkr2Zks2TjXaPe46x68PQNM\nTCQWi4WHI8fRwK8pAHbszI15gxNnDpucTMymYl1ERKSSidnyGZeyMgCoVb0O0e2GmJxIAFycXRnV\nfxJ1avoAkJOXzQdLpnD+UprJycRMKtZFREQqkVNnk1i/e7nRvrfrSKq4VDUxkfyWh3tNxg6cjJur\nOwAZl9P5YOkUsnOyTE4mZlGxLiIiUknY7fZCD5WG1G9Bq0Z6zb2j8aldjyf6v2gsnXnq7FHmrHiD\nfC3pWCmpWBcREakktu9fx5HTCQA4OVn1UKkDC6nfggd6jDXae49uZ/GGOeYFEtOoWBcREakEsrKv\n8M2GXx8q7d5qAL6165uYSIrSMSyS3m3uM9pxu5ay4Te3MEnloGJdRESkEljx/WdcvHIegJrVatOn\n/QNFbCGOoH/EI7Rs1NFof7nuI/Yl/WBiIilrKtZFREQquORzx1m3a5nRHtRlBFVd3UxMJMXlZHHi\n0ajxBPo0BsBuz2d2zOucTksyN5iUGRXrIiIiFVjBQ6UfkP/zQ6WNAppx129eviOOz9WlCk8O+B88\nPeoCkJ2TxfvfvMLFy+dNTiZlQcW6iIhIBfbDgY0cOrkHKLhKe3/30XqotByqUc2TMQP/RpWffyNy\nPjOND5b+Lzm52SYnk9KmYl1ERKSCupqTxeINs41211b98fcKNDGR3A5/ryAe7zsRJ0tB+XY89SDz\nV04zfmsiFZOKdRERkQpq5daFZFxOB8DDvRZ99VBpuXdn0F0M7v6k0d59eAtLN803MZGUNhXrIiIi\nFVBK+gm+27nUaN/TeThuVaqZmEhKSpcWfeneaoDRXrPja+L3xJqYSEqTinUREZEKxm63syjuI+ON\nl8H+obRt2t3cUFKiBnUZQViDtkb78+9msf/4bhMTSWlRsS4iIlLB7Dq0mf0nCgo3i8WJIXqotMJx\ncrIyvM+z1KsbDEB+vo3/fPsaKeknTE4mJU3FuoiISAWSnXuVxev/Y7S7tOhLQN0GJiaS0lLF1Y3R\nA/9GzWq1AcjKucL737zCpSsXTE4mJUnFuoiISAWyausXnM9MA8DDrSZ3d3zI5ERSmmpVr8PogS/h\n6lIVgHMXU/lw2avk5uWYnExKiop1ERGRCuLM+VOs/eEboz2w82O4V6luYiIpC/W9gxne51ksFNzq\nlJS8nwWx72hJxwpCxbqIiEgFYLfb+XLdR9jy8wAI8mtC29AeJqeSstI8uB2Duj5utH84sIGYLZ+Z\nmEhKiop1ERGRCuDHw9+TeGwnABYsDOk+2nh5jlQO3VsNoHOLvkZ75dbP2ZrwnYmJpCTov2IREZFy\nLic3m6/Wf2y0I5pHU9+7oYmJxAwWi4XB3UYRGniX0fff1e9x6NReE1PJ7Sp2sT5jxgyCg4Nxc3Mj\nPDycjRs33nBsdnY2jz/+OC1btsTV1ZWePXteM2bdunU4OTkV+mO1Wjlw4MCtnYmIiEglFbt9Eecv\nnQWgWlUP+nUaZnIiMYvVycqIvn/Fr84dANjy8/ho2b85c/60ycnkVhWrWF+4cCHjx4/npZdeYteu\nXXTq1Im+ffty8uTJ64632Wy4ubkxbtw4+vfvf8P9WiwWEhISSElJISUlheTkZBo3bnxrZyIiIlIJ\nnb2QzJodXxvtARGPUa2qh4mJxGxuVdwZM/AlPNxrAXDl6iXeX/IKl7MumpxMbkWxivVp06YxcuRI\nRo4cSZMmTXj77bfx8/Nj5syZ1x3v7u7OjBkzGDVqFAEBATfdd926dfH29jb+6KUNIiIixffVuo/J\ns+UCEOjTmA7NepmcSBxB7RrejB7wN1ycXQE4e+E0H337Grl5uSYnkz+qyGI9NzeXHTt2EBkZWag/\nKiqK+Pj42zq43W4nPDwcf39/evfuTVxc3G3tT0REpDL56chW9iZtB35+qLTHGD1UKoZA38Y8GjXe\naB8+tZeFa2dgt9tNTCV/lHNRA9LS0rDZbPj4+BTq9/HxYc2aNbd8YD8/P2bNmkXbtm3Jyclh3rx5\n9OrVi/Xr1xMREXHD7bZv337Lx5Tr05yWHs1t6dC8lg7Na+korXnNs+WyZOf7RruRTyvOnLjAmROV\n49+jvl+Ly5W7Anvyw7G1AGxN+I6cy/m0qN/luqM1ryWrJG7vLrJYLy0hISGEhIQY7fbt25OUlMTr\nr79+02JdREREYO+pzWRmF7xW3tXZjdaBWlNdrq9ZQEcuZqVz6MwuAHYdX4dH1do0qNvM5GRSHEUW\n615eXlitVlJTUwv1p6am4uvrW6Jh2rdvz8KFC286Jjw8vESPWZn98ulZc1ryNLelQ/NaOjSvpaM0\n5/VcRir/3bLFaA/qMpzOLbqW+HEckb5fb81dd7Vm5uJ/cuDkTwBsPryMNi3bE+zfFNC8lpaMjIzb\n3keRN7a5uLjQpk0bYmNjC/XHxsaW+BXwnTt34ufnV6L7FBERqWi+Wv8xubYcAOp5B9MpLLKILaSy\ns1qdGdnvBXw86wEFt1F9tOxV0jJSTE4mRSnWUyjPPvssc+bM4eOPPyYxMZFnnnmG5ORkxo4dC8Ck\nSZPo3bt3oW0SEhLYtWsXaWlpZGZmsnv3bnbv3m18/a233uKbb77h0KFD7Nu3j0mTJrFkyRLGjRtX\ngqcnIiJSsexL2sFPR7Ya7SHdx+DkZDUxkZQX7lWrM+ael6jmVgOAzKwM3v/mFa5kZ5qcTG6mWPes\nDx06lPT0dKZMmUJycjJhYWHExMRQr17Bp7OUlBSOHj1aaJu7776b48ePG+3WrVtjsViw2WwA5OTk\n8Pzzz3Py5Enc3Nxo1qwZy5cvJzo6uqTOTUREpELJzctlUdxHRrvDnb1o4NfExERS3njV9OXJ/v/D\nu19NJs+WS+r5k/zn2/+jXb1++tDnoIr9gOnYsWONK+m/N3v27Gv6fl+8/97EiROZOHFicQ8vIiJS\n6a39YTFnM5IBcKtSjQERj5mcSMqjYP+mDIscx9wVbwJw4MSP5Oc40bFhP5OTyfVoMVYREZFyIP3i\nGVZt+8Jo9+s4DA/3miYmkvKsTZOu3N3hIaN9KHUXe09tuckWYhYV6yIiIuXA1+v/Q25ewUOlAV5B\nRDTXbaNye6LbDaVt0+5G+4dja/jx8PfmBZLrUrEuIiLi4BKO7WT34V+veg7pMQar7i+W22SxWHiw\n11M09L/T6FsQ+zbnL501MZX8nop1ERERB1bwUOmHRrtdaA+C/UNNTCQViYuzC6P6v0i1KgW3VGVl\nX2beyunk59tMTia/ULEuIiLiwOJ2LeXMhdMAVHV1Z2DEcJMTSUVTza0GXUIGYcECwOFTe1m9/SuT\nU8kvVKyLiIg4qPOX0li59XOjfXeHh6hRrZaJiaSi8q5Rnxb1uxjt5Vv+S1LKARMTyS9UrIuIiDio\nxRtmk5N7FQD/OoF0aXm3yYmkImtevzPBfgW3WOXb85m74g2ysq+YnEpUrIuIiDig/cd3s/PgJqN9\nf4/ReqhUSpWTxYlH+4ynqqs7AOcyUlm07sMitpLSpmJdRETEweTZcvnyN0VSmyZdaRTQzMREUlnU\nqeHDAz1/fQnm1oTv2LF/vYmJRMW6iIiIg1m361tS008CUMWlKoM6jzA3kFQqbZp0pV1oD6O9cO0s\nzl1MNTFR5aZiXURExIFkZKaz4vvPjHbfDg9Ss3ptExNJZXR/99HUqekDwNWcK8xbMQ2blnM0hYp1\nERERB7J44xyyf36o1Ld2fbq17G9yIqmMqrq6MbzPczhZCkrFo8mJrNr6hcmpKicV6yIiIg7i4Mk9\nhe4Pvr/7aKxWZxMTSWUW5BvC3R0eMtortn7OkdMJJiaqnFSsi4iIOACbLY8v4z4w2neFdCakfnMT\nE4lA7/D7jIeb7fZ85q14k6zsyyanqlxUrIuIiDiA9T8uJ/nccQBcXapyjx4qFQfg5GTl0ejxuFWp\nBkD6pbN8vnYWdrvd5GSVh4p1ERERk128fJ6YLb8+VBrdbiieHl4mJhL5ladHXR7s9ZTR3nFgA9sS\n48wLVMmoWBcRETHZNxvncjWn4E2R3p4B9Gg9wOREIoW1btyJDs16G+0vvnufsxeSTUxUeahYFxER\nMdHhU/sKXaW8v9uTOFtdzAskcgODuz5B3Vr+AGTnXmXeijex2fJMTlXxqVgXERExiS3fxhe/eai0\nZaOONA1sZWIikRur4urG8D7PYnUqWKHoWOpBYr5faHKqik/FuoiIiEk2/bSC02lJALg6V+HeLiPN\nDSRShDt8GtGv48NGO3bblxw8ucfERBWfinURERETXLpygW/jFxjtqLb3U7tGXRMTiRRPzzaDCKlX\nsKyoHTvzV07jytVMk1NVXCrWRURETLBk03yyfn6otG5NP3rcNcjkRCLF42Rx4pHo8bhX9QDgQuY5\nPlszQ8s5lhIV6yIiImXsaHIi3+9bY7QHdx+Fi7MeKpXyo1b1Ojzc+9flHHcdimfLb76npeSoWBcR\nESlD+b97qLR5cDvuDGpjYiKRW9OiYQcimvcx2oviPuTM+VMmJqqYVKyLiIiUoU17VnHyzBEAXKyu\n3NftCZMTidy6e7s8jk/tegDk5GUzd8Wb5NlyTU5VsahYFxERKSOZWRcLPVTau+1g6tTwMTGRyO1x\ndalSsJyjtWA5xxNnDvPt5k9NTlWxqFgXEREpI8vi53Mlu2DVjDo1fejd5l6TE4ncvnp1gxnY6TGj\nvWbH1+w/vtvERBWLinUREZEycCzlIJv3rDbag7uOwsXZ1cREIiWnW+v+NA1sbbQ/WfUWmVkXTUxU\ncahYFxERKWX59ny+iPsAOwVL2zVrEE5YcFuTU4mUHCeLE49E/oXqbjUByLiczn9Xv6vlHEuAinUR\nEZFStmXvao6nHgTA2erCfV31UKlUPDWqeTIscpzR/unIVuL3rDIxUcWgYl1ERKQUXb56iaWb5hvt\nXm3upW4tPxMTiZSeZg3C6drybqP91fqPSUk/YWKi8k/FuoiISClaFr+Ay1cvAVDboy6R4YNNTiRS\nugZ2Ho5fnTsAyM3LYW7MG+TmaTnHW6ViXUREpJScy0wm/qeVRvu+bk/g6lLFxEQipc/VuQrD+zyH\ns7XgrbyRTGAZAAAgAElEQVSn0pJYGj+/iK3kRlSsi4iIlAK73c73h1cYD5WGBt5F8+D2JqcSKRv+\nXoEM6jLCaMftXELCsZ3mBSrHVKyLiIiUgsNndpOWWfDqdavVmcHdRmGxWExOJVJ2urS4m2ZB4Ub7\nk1VvcenKBRMTlU8q1kVERErYlauZ/HBsrdHuddcgvD39TUwkUvYsFgsPRz6Nh3stAC5ducCnsVrO\n8Y9SsS4iIlKC7HY7X8Z9yNXcKwB4Vvcisu39JqcSMYeHey0eiXrGaO9N2s6GH5ebmKj8KXaxPmPG\nDIKDg3FzcyM8PJyNGzfecGx2djaPP/44LVu2xNXVlZ49e1533Lp16wgPD8fNzY1GjRrx/vvv//Ez\nEBERcSBbE75j+/51Rvu+bk9QxaWqiYlEzBUa2JrurQca7cUb5nA6Lcm8QOVMsYr1hQsXMn78eF56\n6SV27dpFp06d6Nu3LydPnrzueJvNhpubG+PGjaN///7XHZOUlES/fv3o3Lkzu3bt4sUXX2TcuHF8\n/fXXt342IiIiJjpz/hRfxH1gtBt6t6Rlo44mJhJxDAM6PUqAVxAAebZc5q54k5y8bHNDlRPFKtan\nTZvGyJEjGTlyJE2aNOHtt9/Gz8+PmTNnXne8u7s7M2bMYNSoUQQEBFx3zMyZMwkICGD69Ok0adKE\nUaNGMXz4cKZOnXrrZyMiImKS3LxcZsdMJSf3KgA13OrQLjja5FQijsHF2YXhfZ/DxdkVgORzx1my\nca7JqcqHIov13NxcduzYQWRkZKH+qKgo4uPjb/nAW7ZsISoqqlBfdHQ027dvx2az3fJ+RUREzLBk\n01xOnT0KFKz+0jXkXlysrianEnEcvrXrc1/XJ4z2+t3L2Xt0u4mJygfnogakpaVhs9nw8fEp1O/j\n48OaNWtu+cApKSnXfADw8fEhLy+PtLS0a473i+3b9S+1pGlOS4/mtnRoXkuH5vXWnUg/wLqEZUb7\nrjt6Ubu6L6B5LS2a19JR2vPqaq9N/dohnEg/AMCc5W8ysPVo3Fyrl+pxzdK4cePb3odWgxEREbkN\nV7IvEn9wqdGuVzuEpn7hN9lCpPKyWCx0bNQfN1cPALLzrrDx4BIt53gTRV5Z9/Lywmq1kpqaWqg/\nNTUVX1/fWz6wr6/vdffp7OyMl5fXDbcLD9cPwJLyy6dnzWnJ09yWDs1r6dC83rr8fBvvfvV3svOy\nAKhZvQ5P3f8S1dxqaF5Liea1dJT1vNYNqMmMr/+BHTvJF46QaU2mx10Di96wnMnIyLjtfRR5Zd3F\nxYU2bdoQGxtbqD82NpaIiIhbPnDHjh2v2eeqVasIDw/HarXe8n5FRETKyqptX3Lo1F4ALBYnHoue\nQDW3GianEnF8Te5oSc82g4z2kvh5nDx7xMREjqtYt8E8++yzzJkzh48//pjExESeeeYZkpOTGTt2\nLACTJk2id+/ehbZJSEhg165dpKWlkZmZye7du9m9e7fx9bFjx3Lq1CkmTJhAYmIiH330EfPmzWPi\nxIkleHoiIiKl4/CpfcR8v9BoR7cbQuN6YSYmEilf+nV8mPreDQGw2fKYG/MmOblazvH3irwNBmDo\n0KGkp6czZcoUkpOTCQsLIyYmhnr16gEFD4sePXq00DZ33303x48fN9qtW7fGYrEYK70EBQWxfPly\nJkyYwKxZs/D39+edd95h0KBBiIiIOLLLVy8xb8Wb2O35ADT0v5PodkNNTiVSvjhbXRje51n+79Nn\nycnLJvX8Sb5e/x8e6PUns6M5lGIV61BwJfyXK+m/N3v27Gv6fl+8X0+XLl30NLeIiJQrdrud/65+\nl/OZaQC4V6nOY30mYHXSLZwif5S3ZwCDuz/Jf1e/C8CmPSsJDWpNi4YdTE7mOLQajIiIyB+w8ccY\nfjz8vdF+OHIcnh51TUwkUr51uLMXrRp1Mtqfrn6PC5nnTEzkWFSsi4iIFNOps0l8veHX3yZ3aXE3\nLRq2NzGRSPlnsVh4sNef8axesBrglauX+GTldPJ/vs2sslOxLiIiUgzZuVeZs2IqebZcAPy9ghjU\nZYS5oUQqCPeq1XkkejwWLAAcOPkTa3csNjmVY1CxLiIiUgxfrfuY1PSTALg6V2FE3+dwcXY1OZVI\nxdG4XhiRbe832ss2L+B46iETEzkGFesiIiJF+OHARjbv/fXdIIO7P4lv7fomJhKpmPq2f4BA3xCg\n4KVjc1e8SXZOlsmpzKViXURE5CbOZaTy2ZoZRvuukC50uLOXiYlEKi6r1ZnHoidQxaUqAGcvnGbR\n+o9NTmUuFesiIiI3YLPlMWfFG1zNuQJAnRo+PNBzLBaLxeRkIhVX3Vp+DOkxxmhv2buanQfjTUxk\nLhXrIiIiN/Dt5k85lnIAACcnKyP6PodblWompxKp+No27U6bkC5G+7M173H+0lkTE5lHxbqIiMh1\nJB7bxeodXxnt/h2HGffSikjpslgsDO05lto1vAHIyr7MvJXTyc+3mZys7KlYFxER+Z2Lly8wf9V0\no930jlb0bDPIxEQilY9blWo8Fj0Bi6WgXD18ai+x278qYquKR8W6iIjIb+Tb8/kk9i0uXbkAgId7\nLR6JGo+TRf/LFClrwf6h9Gk31GjHbPkvST/fmlZZ6CePiIjIb3z3wxISj+002o9EPUONarVMTCRS\nuUW1G0KwXyhQ8GF67oo3yMq+YnKqsqNiXURE5GfHUg6yNH6+0e7V5l5CA1ubmEhErE5WHu0znqqu\n7kDBcqqL1n1ocqqyo2JdREQEyMq+wpwVU40H2AJ9GtO/4zCTU4kI/Lps6i+2JnzHtsQ48wKVIRXr\nIiJS6dntdj5fO5NzGakAVHV1Z3jf57BanU1OJiK/aNOkK+1Cexjt/65+j8On9pmYqGyoWBcRkUrv\n+31r2XFgg9F+oOef8Krpa2IiEbme+7uPxtszAIA8Wy4fLnuV1POnTE5VulSsi4hIpZaafpIv4z4w\n2h3u7EWbJl1usoWImKWqqxtj75mMh1tNAK5cvcTMxf/k4uXzJicrPSrWRUSk0srNy2FOzFRy8rIB\n8PGsx+DuT5qcSkRuxqumL2PumYyrcxUA0i+e4f0lr5Cdk2VystKhYl1ERCqtbzbO5VRaEgDOVhdG\n9P0rVVyqmhtKRIp0h08jRvT9q/HCpBNnDjM7Ziq2CviGUxXrIiJSKf10ZCvrd39rtAd1eZyAukHm\nBRKRPyQsuC1De4wx2vuSdvDFd7Ow2+0mpip5KtZFRKTSOX8pjQWx7xjtFg3b06VFXxMTicitiGge\nTVTb+412/J5YVm370sREJU/FuoiIVCr5+TbmrZzGlauXAKhVvQ4P9X4ai8VicjIRuRX9Og4jvGk3\no/3t5gVsTfjOxEQlS8W6iIhUKiu3fsHhU3sBsFicGN7nWapV9TA5lYjcKovFwsO9nyakfguj79PV\n77L/+G4TU5UcFesiIlJpHDq1lxVbPzfafdoNpWFAMxMTiUhJcLa68ES/F/CvEwgU/Abt429f49TZ\nJHODlQAV6yIiUilczrrIvBVvYrfnA9AooBnR7YaYnEpESopblWqMuWcyNavXAeBqzhVmLXmZ85fS\nTE52e1Ssi4hIhWe32/l09btcyDwHgHtVDx6NnoCTk9XkZCJSkjw9vBg7cDJVXd0ByMg8x6xv/kVW\n9mWTk906FesiIlLhbfgxhp+ObDXawyLH4enhZWIiESktAXWDeKLfC8aH8eRzx/l42b/Js+WanOzW\nqFgXEZEK7dTZoyzeMNtod23Zj+bB7UxMJCKlrckdLXm499NG+8DJn/h09bvlcg12FesiIlJhZede\nZXbMVOOKWoBXEPd0Hm5yKhEpC+1Ce9Cv4zCjvT1xHd9uXmBiolujYl1ERCqsRXEfcub8KQBcnasw\n4u6JuDi7mpxKRMpKVNv76RQWZbRXbfuSTT+tNDHRH6diXUREKqQd+zewZd8aoz2kx2h8PANMTCQi\nZc1isTCkxxiaBYUbfZ9/9z57jmwzMdUfo2JdREQqnLSMFD5bO8Not2nSlXahPU1MJCJmsTpZGdH3\nOep7NwTAbs9nTsxUjqUcNDlZ8ahYFxGRCsVmy2NuzBtk52QB4FXTl6E9xmKxWExOJiJmqeLqxpiB\nk6lTwweAnLxsPljyCmkZKSYnK5qKdRERqVCWbV7AsdSCK2ZOTlaG93kOtyruJqcSEbPVqFaLsYP+\njntVDwAuZWUwa/G/uJx10eRkN6diXUREKoyEYztZs+Nroz2g06ME+jY2MZGIOBIfzwBGD/gfnK0u\nAJy5cJoPlv4vOXnZJie7MRXrIiJSIVy8fJ5PVk432k0DW9PjroEmJhIRRxTsH8qj0ROwUHBr3NHk\nROavnE5+vs3kZNdX7GJ9xowZBAcH4+bmRnh4OBs3brzp+D179tC9e3fc3d2pX78+L7/8cqGvr1u3\nDicnp0J/rFYrBw4cuLUzERGRSivfns/8VdO5lJUBgId7LR6JfAYni65Jici1WjfuxKCujxvt3Yc2\ns3jDHPMC3USxfootXLiQ8ePH89JLL7Fr1y46depE3759OXny5HXHX7p0icjISPz8/NixYwdvvfUW\nr7/+OtOmTSs0zmKxkJCQQEpKCikpKSQnJ9O4sX5dKSIif8zaHYvZf3w3ABYsPBY9gRrVapmcSkQc\nWY/WA+neaoDRjtu1lO9+WGJiousrVrE+bdo0Ro4cyciRI2nSpAlvv/02fn5+zJw587rjP/nkE7Ky\nspg7dy6hoaHcd999vPDCC7z55pvXjK1bty7e3t7GHz2tLyIif0RSygGW/eathL3C76PJHS1NTCQi\n5cWgro/TslFHo714w2x2How3MdG1iizWc3Nz2bFjB5GRkYX6o6KiiI+//sls2bKFLl264Or661vi\noqOjOX36NMeOHTP67HY74eHh+Pv707t3b+Li4m7xNEREpDLKyr7M3Jg3jHtNg3yb0K/DQyanEpHy\nwsnixKPR4wn2CwXAjp35K6dx+NQ+k5P9qshiPS0tDZvNho+PT6F+Hx8fUlKuvzZlSkrKdcfb7XZj\nGz8/P2bNmsWiRYv4+uuvadKkCb169WLTpk23ei4iIlKJ2O12Fq6dxbmLqQBUdXVneJ9nsVqdTU4m\nIuWJq3MVnhwwCe+f33CcZ8vlw6X/S2r69W/3Lmum/UQLCQkhJCTEaLdv356kpCRef/11IiIibrjd\n9u3byyJepaI5LT2a29KheS0d5W1eD6bu4odDG4x2uwZ9OHrwBEc5YWKqa5W3eS0vNK+lozLPa0Tw\nIGJ+nMPV3Mtcyc7krc//Rt8Wj+PmWv2W91kSz2IWeWXdy8sLq9VKampqof7U1FR8fX2vu42vr+91\nx1sslhtuAwUF+8GD5ePVryIiYp4LV9LYdmSl0W7k04ogrztNTCQi5Z1HVU96hj6As1PBGuyZ2Rms\n3beQXFuOqbmKvLLu4uJCmzZtiI2NZfDgwUZ/bGwsQ4YMue42HTt25MUXXyQnJ8e4b33VqlX4+/sT\nGBh4w2Pt3LkTPz+/m+YJDw8vKrIU0y+fnjWnJU9zWzo0r6WjvM1rbl4Obyx8nrz8XAB8atdj7OBJ\nuLpUMTlZYeVtXssLzWvp0Lz+qn6QPx8s/V/s9nzOXU5md8oanhzwP1idrH94XxkZGbedp1irwTz7\n7LPMmTOHjz/+mMTERJ555hmSk5MZO3YsAJMmTaJ3797G+Icffhh3d3dGjBjB3r17+eqrr3jttdd4\n7rnnjDFvvfUW33zzDYcOHWLfvn1MmjSJJUuWMG7cuNs+KRERqbgWb5jD6bQkAJytLjze968OV6iL\nSPnVrEE4Q3uMMdr7knbwxXezsNvtpuQp1j3rQ4cOJT09nSlTppCcnExYWBgxMTHUq1cPKHig9OjR\no8b4GjVqEBsby1NPPUXbtm3x9PRk4sSJjB8/3hiTk5PD888/z8mTJ3Fzc6NZs2YsX76c6OjoEj5F\nERGpKH48vIUNPy432vd2HYm/V5B5gUSkQopoHs35S2dZte1LAOL3xOLp4U10u+vfVVKaiv2A6dix\nY40r6b83e/bsa/qaNWt206UYJ06cyMSJE4t7eBERqeRSz5/i09h3jXaLhh3o3LyPiYlEpCLr13EY\n5y+lsS0xDoBvNy/A08OLdqE9yjSH1rcSERGHZrfb2fhjDN9snEtOXjYAntW9eKj3U3qRnoiUGovF\nwkO9nyLjcjoHTvwIwKer36Vmtdpl+uK1Yt2zLiIiYoaMzHRmfvMvvoj7wCjUna0uDO/7HNWqepic\nTkQqOmerC0/0ewH/OgULpOTn2/jo239z6mxSmWVQsS4iIg7phwMbefWTv5B4bKfR51fnDp594DWC\n/UNNTCYilYlblWqMuWcyNavXASA7J4tZS17m/KWzZXJ8FesiIuJQrlzNZG7MG8yJmcqV7EwALFjo\nedcg/vrgVOrVDTY5oYhUNp4eXvzpnslUdXUHICPzHLO+edn4GVWaVKyLiIjDSDy2i1cXPMOOA7++\nmbS2R12eHvwyg7qMwMXZ1cR0IlKZ+XsF8US/F7A6FTzymXzuOB8ve408W26pHlfFuoiImC4nN5sv\n4z5gxuJ/kJF5zuhvf2cvXhj2Fo3rhZmYTkSkQJM7WvJQ76eM9sGTP/Fp7Luluga7VoMRERFTHUs5\nyPxV0zlz/pTRV92tJg/2+jMtGrY3MZmIyLXahfbgwqU0lm1eAMD2/evw9PBiQMSjpXI8FesiImIK\nmy2Pldu+YNXWL8i35xv9YcHteKjXn/Fwr2ViOhGRG4tsez/pl84Sv2cVALHbF1G7hjcRzUv+5Z4q\n1kVEpMylpp9k/srpHD9zyOir4lKV+7qNosOdvbR+uog4NIvFwpAeY8jITGdv0nYAPv/ufWpWq01Y\ncNsSPZbuWRcRkTKTb89n3a5l/N+nzxYq1Bv638mLw96iY7PeKtRFpFywOlkZcfdfucO7EQB2ez5z\nYqZyLOVgiR5HxbqIiJSJ85fSmPn1P1m07iNybTkAWK3O3NN5OOMGv0ydmj4mJxQR+WOquFRl9MCX\nqFOj4OdXTl42Hyx5hbSMlBI7hop1EREpVXa7nW2J6/j3J39h/4ndRr+/VxB/fWAqvdrci5OT1cSE\nIiK3rka1Wowd9Hfcf36r8qWsDGYu/heZWRdLZP8q1kVEpNRczrrI7JjXmb9yGlk5V4CCFxz1bnMf\nzz3wOgF1g8wNKCJSAnw8Axg94H9wtroAcPbCaT5YOqVE9q0HTEVEpFTsS/qBT1e/w8XL542+OjV8\neCTqGRoG3GliMhGRkhfsH8pj0ROYvfx17NhJSt5fIvtVsS4iIiUqO/cqizfMYdNPKwr1d2wWyb1d\nR1LV1c2kZCIipatV404M6vo4X6//T4ntU8W6iIiUmKPJ+/lk5XTOZiQbfR5uNXmw91M0D25nYjIR\nkbLRo/VAzl9KI27nkhLZn4p1ERG5bXm2XFZ8/zmx2xdh/80Ljlo07MADPf+Eh3tNE9OJiJStQV1G\ncOFSWonsS8W6iIjcluRzJ5i/ahonzxwx+qq4unF/tydpF9pD66aLSKXjZHHi0ejxXLmcddv7UrEu\nIiK3JN+ez7qdy1gaP588W67R36heGI9E/oXaNbxNTCciYi4XZ1dAxbqIiJgg/eJZFsS+zcGTPxl9\nzlYX+nd6hO6tB+Bk0crAIiIlQcW6iIgUW8ELjuL4Mu5Drv68bjpAQN0GPBo1Hn+vQBPTiYhUPCrW\nRUSkWDKzLrJwzQx2H95i9FksTkSG30ef9g8YLwMREZGSo2JdRESKtOfINv675j0uXblg9HnV9OWR\nqPEE+zc1MZmISMWmYl1ERG7oak4WX6//D5v3xhbqjwiLZlCXEVTRC45EREqVinUREbmuI6cTmL9q\nOucyUo2+Gu6ePNT7KZo1CDcxmYhI5aFiXURECsnNyyXm+89Ys+PrQi84atWoE0N7jqW6Ww0T04mI\nVC4q1kVExHA6LYl5K6dzOi3J6HNzdef+HqMJb9JNLzgSESljKtZFRIR8ez5rdixm2eZPsNnyjP6Q\nes0ZFvUXPD3qmphORKTyUrEuIlLJXbp6nviDS0m9eNzoc7G6MiDiUbq26qcXHImImEjFuohIJZOd\ne5VjKQc5mpzAkdOJHDzxE3n5ucbX63s35NHo8fjWrm9iShERARXrIiIV3oXMcxw5ncDR5ESOnE7g\n1Nmj5P/mwdFfOFmciGx7P33aDcVq1f8eREQcgX4ai4hUIPn5NpLPHefI6QSOJCdy9HQC6ZfOFrld\nLfe6jBwwkSDfkDJIKSIixaViXUSkHMvOySIp5YBRmCelHOBqzpUit/OrcwcN/JoS7B/K5bRcqlf1\nVKEuIuKAVKyLiJQj5y+lGbezHElO4PTZpOve0vJbLs6uBPo0Jtg/lAZ+TWng1xT3qtWNr2/fvr20\nY4uIyC1SsS4i4qDy822cSjvG0eQEjp5O5EhyIueLcUtLjWqeBPuF0sC/KcF+odSr20D3oIuIlFP6\n6S0i4iCu5mSRlLyfIz8X50kp+8nOvXrTbSxYCm5p8Q8l+OfivHYNb728SESkglCxLiJikvSLZ43l\nE48kJ3A67Rj2Im5pcXWuQqBvCMH+TWngF0qQXwjuVarfdBsRESm/il2sz5gxg6lTp5KcnEyzZs2Y\nPn06nTt3vuH4PXv28PTTT7N161bq1KnD6NGjmTx5cqEx69at47nnnmPv3r0EBAQwceJExowZc+tn\nIyLioGz5Nk6nJRVaQvFC5rkit6tZvQ7BPz8I2sCvKQFeQbqlRUSkEinWT/yFCxcyfvx4Zs2aRURE\nBO+99x59+/YlISGBevXqXTP+0qVLREZG0r17d3bs2EFCQgIjRoygevXqTJgwAYCkpCT69evHqFGj\nWLBgARs2bODPf/4z3t7e3HvvvSV7liIiZSg/30ZW9mWOnzlcUJyfTiAp9SA5xbilxd8rsOCWlp8L\ndE+PurqlRUSkEitWsT5t2jRGjhzJyJEjAXj77bdZsWIFM2fOZMqUKdeM/+STT8jKymLu3Lm4uroS\nGhpKQkICb775plGsz5w5k4CAAKZPnw5AkyZN+P7775k6daqKdREpM3a7nVxbDtk5WWTnXjX+vlqo\nXfhrxpjcLHJyCv7O/s0/5+blFOvYri5VCfINMR4GDfJtglsV91I+YxERKU+KLNZzc3PZsWMHEydO\nLNQfFRVFfHz8dbfZsmULXbp0wdXV1eiLjo7m73//O8eOHSMwMJAtW7YQFRVVaLvo6GjmzZuHzWbD\narXeyvmISAWXb8/nSnZmEcX19YvswsX1r/1F3SdeUmpVr2PczhLsH4q/VxBWJ/2sExGRGyuyWE9L\nS8Nms+Hj41Oo38fHhzVr1lx3m5SUFOrXr3/NeLvdTkpKCoGBgaSkpBAZGXnNmLy8PNLS0q45Xnmy\nbtcyEo/vMjtGkTIuZACw4/QKk5NUPBcuXABgx6mi59aOvfg7thdv7B/YY8Hx7Xbsdjt2Cv99s/58\n8oscc+Nt7QUFsh3s9nzsFP77lzG/30d+vq1gTfHrXydwGBYsuLpWpW5NP+NB0GD/pnh61DU7moiI\nlDPl7imljIwMsyMUqVWDLrRq0MXsGCLiSPId9+dX48aNAcfNV15pXkuH5rV0aF4dl1NRA7y8vLBa\nraSmphbqT01NxdfX97rb+Pr6Xne8xWIxtrnRGGdnZ7y8vP7QSYiIiIiIVERFFusuLi60adOG2NjY\nQv2xsbFERERcd5uOHTuyYcMGcnJ+fchq1apV+Pv7ExgYaIz5/T5XrVpFeHi47lcXEREREQEsdnvR\nN8F+/vnnPPbYY7z33ntEREQwc+ZMZs+ezb59+6hXrx6TJk1i27ZtrF69GoCLFy/StGlTunfvzt/+\n9jf279/P448/zj//+U/Gjx8PFCzd2Lx5c0aNGsWYMWPYuHEjTz/9NJ999hmDBg0q3bMWERERESkH\ninXP+tChQ0lPT2fKlCkkJycTFhZGTEyMscZ6SkoKR48eNcbXqFGD2NhYnnrqKdq2bYunpycTJ040\nCnWAoKAgli9fzoQJE5g1axb+/v688847KtRFRERERH5WrCvrIiIiIiJS9oq8Z91Mr776Ku3ataNm\nzZp4e3szcOBA9u7da3asCufVV1/FycmJv/zlL2ZHKfdSUlIYMWIE3t7euLm5ERYWxoYNG8yOVa7l\n5+czefJkgoODcXNzIzg4mMmTJ5OfXzZro1ckGzZs4J577qFevXo4OTkxb968a8b84x//ICAgAHd3\nd3r06MG+fftMSFq+3Gxe8/LyeOGFF2jZsiXVq1fH39+fYcOGceLECRMTlw/F+X79xZgxY3BycuLN\nN98sw4TlU3Hm9cCBAwwePBhPT0+qVatGeHg4+/fvNyFt+VHUvF6+fJlx48ZRv3593N3dadq0qfFi\n0KI4dLG+fv16nn76aTZv3sx3332Hs7MzvXv3Ntawltu3ZcsWPvzwQ1q2bGl2lHIvIyODiIgILBYL\nMTExJCYm8s477+Dt7W12tHLt3//+NzNnzuTdd99l//79vP3228yYMYNXX33V7GjlTmZmJs2bN+ft\nt9/G3f3aN6W+9tprTJs2jffee4/t27fj7e1NZGQkly9fNiFt+XGzeb1y5Qq7du1i8uTJ7Ny5kyVL\nlnDixAn69u2rD5xFKOr79Rdffvkl27ZtIyAgoAzTlV9FzWtSUhKdO3emYcOGxMXFsXfvXl555RWq\nV69uQtryo6h5nTBhAjExMSxYsIDExEReeuklXnzxRRYsWFD0zu3lSGZmpt1qtdqXLVtmdpQK4cKF\nC/aGDRva4+Li7N27d7ePGzfO7Ejl2qRJk+ydO3c2O0aF079/f/uIESMK9Q0fPtw+YMAAkxJVDNWr\nV7fPnTu3UJ+fn5/91VdfNdpZWVl2Dw8P+wcffFDW8cqt683r7+3bt89usVjse/bsKaNU5d+N5jUp\nKcler149e2Jioj0oKMj+xhtvmJCu/LrevD788MP2Rx55xKREFcP15jUsLMz+j3/8o1Bft27dilV7\nOfsXs74AAAYkSURBVPSV9d+7ePEi+fn5eHp6mh2lQhg9ejRDhw6lW7duZkepEL755hvat2/Pgw8+\niI+PD61bt+a9994zO1a517lzZ7777jvjV7D79u1j7dq19OvXz+RkFcvRo0evebN01apV6dq1K/Hx\nDv7K2HImIyMDi8Wi/5fdJpvNxsMPP8zkyZNp0qSJ2XEqBLvdztKlS/9/e/cX0lQbxwH8e6aL3DJB\nTE+j1kao7BgNCoVQ6M+FYFAUJLN2o0EJGq0prT+LQDAz+gf+6aYrK8jVlSjdRLYV6k2JgwoaISUR\nLuimLIo1n/fi5ZV32Nv2vvbynLO+H9jNwznjy8Nhv985Z+c50DQNdXV1KC4uRlVVFe7cuSM7muHV\n1NRgeHgYb9++BQCMj48jGo2irq4u7b6GeoOpz+fDpk2bsGXLFtlRDO/69euYnp7G7du3ZUfJGtPT\n07h27Rr8fj9OnTqFqakpHDlyBIqioKWlRXY8wzpx4gQ+ffoETdOQk5ODZDKJYDCI5uZm2dGyyuzs\nLBRFQUlJScp4SUkJ3r17JylV9kkkEmhvb8fu3bths9lkxzG0s2fPori4GIcPH5YdJWu8f/8ec3Nz\n6OrqQmdnJy5cuIAHDx7A6/UiPz8/o8aSfqynpwfNzc2w2+3Izc2Foijo7e3Nrma9ra0N4+PjGBsb\ng6IosuMYWiwWQzAYxNjYGEwmQ91c0bX5+XlUVVXh3LlzAAC3241YLIb+/n4260swODiImzdvYnBw\nEJqmYWpqCkePHoXT6URTU5PseEQZSyaT8Hq9+PjxI0ZGRmTHMbRwOIyBgQFEo1HZUbLKX89R7Nmz\nBz6fDwCwceNGPHnyBH19fWzWl6CnpwcTExMYGRmB3W7Ho0eP0N7eDofDgdra2p/ua4hOze/3IxQK\n4eHDhwtvQKX/bmJiAh8+fICmaTCbzTCbzYhEIujv78eyZcuQSCRkRzSk1atXw+VypYy5XC7MzMxI\nSpQdAoEAjh8/jvr6elRUVMDr9aKtrY0PmP5iqqpCCIF4PJ4yHo/HoaqqpFTZI5lMoqGhAc+ePcPo\n6Cj/ArNEkUgEs7OzUFV1oY69efMGgUAAdrtddjzDKioqQm5uLmvZL/b161ecPn0aFy9exM6dO7Fh\nwwa0tLSgoaEBly5dSru/7q+s+3w+3L17F+FwGKWlpbLjZIW9e/eisrIyZayxsRFlZWUIBoMwm82S\nkhlbdXX1oqWtXr58yRPMJfry5cuiO0Amk4krafxiTqcTqqri/v372Lx5M4A/C8zjx49x+fJlyemM\n7fv37/B4PHjx4gUikQhWrVolO5Lhtba2or6+PmWstrYWBw4cwKFDhySlMj6z2YzKyspFtSwWi7GW\nLUEikUAikVhUy3JycjKqZbpu1ltbW3Hr1i0MDQ2hoKBg4YrPihUrYLVaJaczrpUrV0LTtJQxq9WK\nwsLCRWfTlDm/34/q6mp0dXXB4/FgcnISvb296O7ulh3N0Hbt2oXu7m44HA5UVFRgcnISV69eRWNj\no+xohvP582e8evUKQgjMz89jZmYG0WgUhYWFWLt2LY4dO4bz58+jvLwcpaWl6OzsRH5+Pvbv3y87\nuq79bF5tNhv27duHp0+fYnh4OOXuRUFBAZYvXy45vX6lO16LiopStjebzVBVlRf20kg3r4FAAB6P\nBzU1NdixYwdGR0cRCoUwNDQkO7qupZvXrVu34uTJk7BarVi3bh3C4TBu3LiR0ZV1XS/dqCiKMJlM\niz4dHR2yo2Wd7du3c+nGX+DevXvC7XaLvLw8UV5eLvr6+mRHMry5uTnh9/uFw+EQFotFrF+/Xpw5\nc0Z8+/ZNdjTDCYfDP/xdbWpqWtimo6ND2Gw2kZeXJ7Zt2yaeP38uMbEx/GxeX79+/Y+1LN0Sj7+7\nTI7Xv3M6nVy6MQOZzOvAwIAoKysTFotFuN1uEQqFJCY2hnTzGo/HxcGDB8WaNWuExWIRLpdLXLly\nJaPvVoQQ4v8/3yAiIiIion/LEA+YEhERERH9jtisExERERHpFJt1IiIiIiKdYrNORERERKRTbNaJ\niIiIiHSKzToRERERkU6xWSciIiIi0ik260REREREOsVmnYiIiIhIp/4AW0E3FgNNfSIAAAAASUVO\nRK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "xs = list(range(3,19))\n", + "ps = best_attr_pmf.Probs(xs)\n", + "plt.plot(xs, ps);" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Mixtures\n", + "\n", + "Let’s do one more example from *Dungeons & Dragons*.\n", + "Suppose I have a box of dice with the following inventory:\n", + "\n", + " 5 4-sided dice\n", + " 4 6-sided dice\n", + " 3 8-sided dice\n", + " 2 12-sided dice\n", + " 1 20-sided die\n", + "\n", + "I choose a die from the box and roll it. What is the distribution of the\n", + "outcome?\n", + "\n", + "If you know which die it is, the answer is easy. A die with\n", + "n sides yields a uniform distribution from 1 to\n", + "n, including both.\n", + "\n", + "But if we don’t know which die it is, the resulting distribution is a\n", + "**mixture** of uniform distributions with different bounds.\n", + "In general, this kind of mixture does not fit any simple mathematical\n", + "model, but it is straightforward to compute the distribution in the form\n", + "of a PMF.\n", + "\n", + "As always, one option is to simulate the scenario, generate a random\n", + "sample, and compute the PMF of the sample. This approach is simple and\n", + "it generates an approximate solution quickly. But if we want an exact\n", + "solution, we need a different approach.\n", + "\n", + "Let’s start with a simple version of the problem where there are only\n", + "two dice, one with 6 sides and one with 8. We can make a Pmf to\n", + "represent each die:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "d6 = Die(6)\n", + "d8 = Die(8)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then we create a Pmf to represent the mixture:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1 0.14583333333333334\n", + "2 0.14583333333333334\n", + "3 0.14583333333333334\n", + "4 0.14583333333333334\n", + "5 0.14583333333333334\n", + "6 0.14583333333333334\n", + "7 0.06250000000000001\n", + "8 0.06250000000000001\n" + ] + } + ], + "source": [ + "mix = thinkbayes.Pmf()\n", + "for die in [d6, d8]:\n", + " for outcome, prob in die.Items():\n", + " mix.Incr(outcome, prob)\n", + "mix.Normalize()\n", + "mix.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The first loop enumerates the dice; the second enumerates the outcomes\n", + "and their probabilities. Inside the loop, `Pmf.Incr` adds up\n", + "the contributions from the two distributions.\n", + "\n", + "This code assumes that the two dice are equally likely. More generally,\n", + "we need to know the probability of each die so we can weight the\n", + "outcomes accordingly.\n", + "\n", + "First we create a Pmf that maps from each die to the probability it is\n", + "selected:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "15" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pmf_dice = thinkbayes.Pmf()\n", + "pmf_dice.Set(Die(4), 5)\n", + "pmf_dice.Set(Die(6), 4)\n", + "pmf_dice.Set(Die(8), 3)\n", + "pmf_dice.Set(Die(12), 2)\n", + "pmf_dice.Set(Die(20), 1)\n", + "pmf_dice.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we need a more general version of the mixture algorithm:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "mix = thinkbayes.Pmf()\n", + "for die, weight in pmf_dice.Items():\n", + " for outcome, prob in die.Items():\n", + " mix.Incr(outcome, weight*prob)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now each die has a weight associated with it (which makes it a weighted\n", + "die, I suppose). When we add each outcome to the mixture, its\n", + "probability is multiplied by weight." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAEWCAYAAADfHdlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X9Qleed///XEUEhLpaGekAgGCbEWklcyzFR0a1rPFBM\nsluTlWZtokBdy6xSjiR+jVs2JuMw0W+MGCqgO7r+qEnESY1OE3E9Rk0F6jYwgxMVszsJ1TWccxrG\natRYJcfz+aOfnM8eAc8NKOf2+HzMOFMu39d9Xdd4zT2vXL3vG4vP5/MJAAAAgOkMCvUEAAAAAHSP\nsA4AAACYFGEdAAAAMCnCOgAAAGBShHUAAADApAjrAAAAgEkR1gEAAACTMhzWq6urlZaWpujoaNls\nNtXX1/dYe/XqVRUUFGjcuHGKiorS9OnTu6176623NH78eN1zzz1KTEzUc889J4/H0/tVAAAAAGHI\nUFivra2Vw+FQWVmZWlpaNHnyZOXm5urs2bPd1nu9XkVHR6u4uFhPPPFEtzUNDQ2aO3euCgoKdPLk\nSe3Zs0etra169tln+74aAAAAIIxYjPwG04kTJ+qv//qvtX79en/bgw8+qNmzZ6u8vPymfYuLi3Xi\nxAkdPHgwoP3111/XunXr1NbW5m/bsmWLfv7zn+vLL7/s7ToAAACAsBP0ZL2zs1PNzc2y2+0B7dnZ\n2WpsbOzzwFlZWXK5XHrvvfckSR0dHdqxY4cef/zxPl8TAAAACCdBw3pHR4e8Xq+sVmtAu9Vqldvt\n7vPAEydO1Ntvv62f/OQnioqK0ogRIyT95XQdAAAAgDQ4VAOfPHlSxcXFWr58ubKzs+VyufTCCy9o\nwYIF2rp1a0DthQsXQjRLAAAAoP+GDx/ep35BT9bj4+MVERHR5SstHo9HCQkJfRpUklauXKlHH31U\npaWlysjIkN1uV3V1tX71q1+pvb29z9cFAAAAwkXQsB4ZGanMzEw5nc6AdqfTqaysrD4P/NVXXyki\nIiJwMoMGyWKx6Pr1632+LgAAABAuDD0GU1paqrlz52rChAnKyspSTU2NXC6XioqKJEnLli3TRx99\npAMHDvj7tLa26urVq+ro6NClS5d07NgxSdK4ceMkSU8++aQWLFig9evXKycnR+3t7Vq8eLEyMzOV\nnJzc41z6+n8hIPw1NTVJkmw2W4hnAjNjn8AI9gmMYJ8gmFvxKLehsJ6Xl6dz586pvLxcLpdLGRkZ\nqqur84dqt9sd8AlGSZo5c6bOnDnj/3n8+PGyWCzyer2SpHnz5unSpUuqqqrSCy+8oG9961uaPn26\nVq5c2e9FAQAAAOHA0HfWQ+1//1cJJ+voCSccMIJ9AiPYJzCCfYJgbkWGNfQbTAEAAAAMPMI6AAAA\nYFKEdQAAAMCkCOsAAACASRHWAQAAAJMirAMAAAAmRVgHAAAATIqwDgAAAJgUYR0AAAAwKcI6AAAA\nYFKEdQAAAMCkCOsAAACASQ0O9QQGwvS6hts+xsHcrLAdL5RrAwAAuJtxsg4AAACYFGEdAAAAMCnC\nOgAAAGBShsN6dXW10tLSFB0dLZvNpvr6+h5rr169qoKCAo0bN05RUVGaPn16t3WdnZ166aWXlJaW\npqFDh2rUqFFat25d71cBAAAAhCFDL5jW1tbK4XBo/fr1ysrKUlVVlXJzc9Xa2qrk5OQu9V6vV9HR\n0SouLtbevXt1/vz5bq/74x//WO3t7dq4caMeeOABeTweXblypX8rAgAAAMKEobBeUVGhwsJCFRYW\nSpIqKyu1b98+1dTUqLy8vEt9TEyMqqurJUnHjh3rNqzv379fhw4d0qeffqpvf/vbkqT77ruvzwsB\nAAAAwk3Qx2A6OzvV3Nwsu90e0J6dna3GxsY+D7xnzx5NmDBBr7/+ulJSUvTggw+qpKREly9f7vM1\nAQAAgHAS9GS9o6NDXq9XVqs1oN1qteqDDz7o88CfffaZjhw5oiFDhmjXrl06f/68Fi1aJJfLpZ07\nd/bYr6mpqc9j3k4DPa+BHO9OW5tZ9wjMhX0CI9gnMIJ9gp6kp6f3+xoh+6VI169f16BBg/T2229r\n2LBhkqR169bphz/8ob744gt95zvfCdXUAAAAAFMIGtbj4+MVEREhj8cT0O7xeJSQkNDngRMTE5WU\nlOQP6pI0ZswY+Xw+nTlzpsewbrPZej/YAPzWzYB5hdl4IV1bL3xzstHX/rg7sE9gBPsERrBPEMyF\nCxf6fY2gz6xHRkYqMzNTTqczoN3pdCorq++/Fj4rK0vt7e366quv/G2ffPKJLBaLUlNT+3xdAAAA\nIFwY+s56aWmptmzZok2bNunUqVMqKSmRy+VSUVGRJGnZsmWaMWNGQJ/W1la1tLSoo6NDly5d0rFj\nx3Ts2DH/38+ZM0f33nuvCgoKdPLkSTU0NMjhcGj27NmKj4+/hUsEAAAA7kyGnlnPy8vTuXPnVF5e\nLpfLpYyMDNXV1fm/se52u9XW1hbQZ+bMmTpz5oz/5/Hjx8tiscjr9UqS7rnnHh04cEDFxcV65JFH\nFBcXp1mzZunVV1+9VWsDAAAA7miGXzAtKiryn6TfaPPmzV3abgzv3UlPT9e+ffuMTgEAAAC4qxh6\nDAYAAADAwCOsAwAAACZFWAcAAABMirAOAAAAmBRhHQAAADApwjoAAABgUoR1AAAAwKQI6wAAAIBJ\nEdYBAAAAkyKsAwAAACZFWAcAAABMirAOAAAAmBRhHQAAADApwjoAAABgUoR1AAAAwKQMh/Xq6mql\npaUpOjpaNptN9fX1PdZevXpVBQUFGjdunKKiojR9+vSbXru+vl6RkZF6+OGHjc8cAAAACHOGwnpt\nba0cDofKysrU0tKiyZMnKzc3V2fPnu223uv1Kjo6WsXFxXriiSdueu3z589r3rx5mjFjRu9nDwAA\nAIQxQ2G9oqJChYWFKiws1OjRo1VZWanExETV1NR0Wx8TE6Pq6mrNnz9fSUlJN732T3/6U+Xn52vi\nxIm9nz0AAAAQxoKG9c7OTjU3N8tutwe0Z2dnq7GxsV+DV1dX649//KPKysr6dR0AAAAgHAUN6x0d\nHfJ6vbJarQHtVqtVbre7zwN//PHHWrFihd58801ZLJY+XwcAAAAIV4NDMei1a9f0zDPPaPXq1brv\nvvskST6fz1Dfpqam2zm1PhvoeQ3keHfa2sy6R2Au7BMYwT6BEewT9CQ9Pb3f1wga1uPj4xURESGP\nxxPQ7vF4lJCQ0KdBXS6XWltbVVBQoPz8fEnS9evX5fP5FBUVpb179/LCKQAAAO56QcN6ZGSkMjMz\n5XQ69fTTT/vbnU6nZs+e3adBk5KSdPz48YC2qqoqHThwQLt371ZqamqPfW02W+8HrGvofZ9eCphX\nmI0X0rX1wjcnG33tj7sD+wRGsE9gBPsEwVy4cKHf1zD0GExpaanmzp2rCRMmKCsrSzU1NXK5XCoq\nKpIkLVu2TB999JEOHDjg79Pa2qqrV6+qo6NDly5d0rFjxyRJ48aN0+DBg/W9730vYIwRI0ZoyJAh\nGjNmTL8XBQAAAIQDQ2E9Ly9P586dU3l5uVwulzIyMlRXV6fk5GRJktvtVltbW0CfmTNn6syZM/6f\nx48fL4vFIq/XewunDwAAAIQvwy+YFhUV+U/Sb7R58+YubTeG92CWL1+u5cuX96oPAAAAEM4M/VIk\nAAAAAAOPsA4AAACYFGEdAAAAMCnCOgAAAGBShHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJkVY\nBwAAAEyKsA4AAACYFGEdAAAAMCnCOgAAAGBShHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJmU4\nrFdXVystLU3R0dGy2Wyqr6/vsfbq1asqKCjQuHHjFBUVpenTp3epeffdd5WTk6MRI0YoNjZWEydO\n1G9+85u+rQIAAAAIQ4bCem1trRwOh8rKytTS0qLJkycrNzdXZ8+e7bbe6/UqOjpaxcXFeuKJJ7qt\n+fDDD/XYY49p7969amlp0cyZMzVr1iw1NDT0fTUAAABAGBlspKiiokKFhYUqLCyUJFVWVmrfvn2q\nqalReXl5l/qYmBhVV1dLko4dO6bz5893qVm7dm3Azy+99JLef/997d69W1lZWb1eCAAAABBugp6s\nd3Z2qrm5WXa7PaA9OztbjY2Nt3QyFy9eVFxc3C29JgAAAHCnCnqy3tHRIa/XK6vVGtButVr1wQcf\n3LKJVFVV6fPPP9dzzz1307qmpqZbNuatNNDzGsjx7rS1mXWPwFzYJzCCfQIj2CfoSXp6er+vYegx\nmNvt17/+tZYuXaqdO3cqJSUl1NMBAAAATCFoWI+Pj1dERIQ8Hk9Au8fjUUJCQr8n8M4772jevHna\nvn27Zs6cGbTeZrP1fpC62//SasC8wmy8kK6tF7452ehrf9wd2Ccwgn0CI9gnCObChQv9vkbQZ9Yj\nIyOVmZkpp9MZ0O50Ovv9IujOnTs1b948bdu2TbNmzerXtQAAAIBwY+gxmNLSUs2dO1cTJkxQVlaW\nampq5HK5VFRUJElatmyZPvroIx04cMDfp7W1VVevXlVHR4cuXbqkY8eOSZLGjRsnSdqxY4fmzp2r\n119/XVOmTPGf3EdFRfGSKQAAACCDYT0vL0/nzp1TeXm5XC6XMjIyVFdXp+TkZEmS2+1WW1tbQJ+Z\nM2fqzJkz/p/Hjx8vi8Uir9crSdqwYYO8Xq8cDoccDoe/7gc/+IEOHjzY74UBAAAAdzrDL5gWFRX5\nT9JvtHnz5i5tN4b3Gx06dMjo0AAAAMBdydBvMAUAAAAw8AjrAAAAgEkR1gEAAACTIqwDAAAAJkVY\nBwAAAEyKsA4AAACYFGEdAAAAMCnCOgAAAGBShHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJkVY\nBwAAAEyKsA4AAACYFGEdAAAAMCnDYb26ulppaWmKjo6WzWZTfX19j7VXr15VQUGBxo0bp6ioKE2f\nPr3bug8//FA2m03R0dF64IEHtGHDht6vAAAAAAhThsJ6bW2tHA6HysrK1NLSosmTJys3N1dnz57t\ntt7r9So6OlrFxcV64oknuq35wx/+oMcff1xTpkxRS0uLXnzxRRUXF+vdd9/t+2oAAACAMGIorFdU\nVKiwsFCFhYUaPXq0KisrlZiYqJqamm7rY2JiVF1drfnz5yspKanbmpqaGiUlJWnt2rUaPXq05s+f\nr3nz5mn16tV9Xw0AAAAQRoKG9c7OTjU3N8tutwe0Z2dnq7Gxsc8DHz16VNnZ2QFtOTk5ampqktfr\n7fN1AQAAgHARNKx3dHTI6/XKarUGtFutVrnd7j4P7Ha7u73m119/rY6Ojj5fFwAAAAgXg0M9gd5q\namoK9RS6NdDzGsjx7rS1mXWPwFzYJzCCfQIj2CfoSXp6er+vEfRkPT4+XhEREfJ4PAHtHo9HCQkJ\nfR44ISGh22sOHjxY8fHxfb4uAAAAEC6CnqxHRkYqMzNTTqdTTz/9tL/d6XRq9uzZfR540qRJ2r17\nd0Db/v37ZbPZFBER0WM/m83W+8HqGnrfp5cC5hVm44V0bb3wzclGX/vj7sA+gRHsExjBPkEwFy5c\n6Pc1DH0NprS0VFu2bNGmTZt06tQplZSUyOVyqaioSJK0bNkyzZgxI6BPa2urWlpa1NHRoUuXLunY\nsWM6duyY/++Lior0+eefa/HixTp16pQ2btyobdu2acmSJf1eFAAAABAODD2znpeXp3Pnzqm8vFwu\nl0sZGRmqq6tTcnKypL+8LNrW1hbQZ+bMmTpz5oz/5/Hjx8tisfi/9DJq1Cjt3btXixcv1vr16zVy\n5Ej98pe/1I9+9KNbtTYAAADgjmb4BdOioiL/SfqNNm/e3KXtxvDenalTp/JSBgAAANADQ4/BAAAA\nABh4hHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJkVYBwAAAEyKsA4AAACYFGEdAAAAMCnCOgAA\nAGBShHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJkVYBwAAAEyKsA4AAACYFGEdAAAAMKnBRgur\nq6u1evVquVwujR07VmvXrtWUKVN6rD9+/LgWLVqk3//+97r33nu1YMEC/eu//mtAzVtvvaXXXntN\n//Vf/6XY2FjNmDFDq1evltVq7fuKEFam1zX0vlMv+xzMzer7WL30zVgAAABGGDpZr62tlcPhUFlZ\nmVpaWjR58mTl5ubq7Nmz3dZfvHhRdrtdiYmJam5u1htvvKHXXntNFRUV/pqGhgbNnTtXBQUFOnny\npPbs2aPW1lY9++yzt2ZlAAAAwB3OUFivqKhQYWGhCgsLNXr0aFVWVioxMVE1NTXd1m/fvl1XrlzR\n1q1bNWbMGD311FNaunSp1qxZ4685evSoUlJS9POf/1ypqal65JFHtGjRIv3nf/7nrVkZAAAAcIcL\nGtY7OzvV3Nwsu90e0J6dna3GxsZu+xw9elRTp05VVFSUvy0nJ0ft7e06ffq0JCkrK0sul0vvvfee\nJKmjo0M7duzQ448/3ufFAAAAAOEkaFjv6OiQ1+vt8hy51WqV2+3uto/b7e623ufz+ftMnDhRb7/9\ntn7yk58oKipKI0aMkCRt2bKlL+sAAAAAwo7hF0xvtZMnT6q4uFjLly9Xdna2XC6XXnjhBS1YsEBb\nt27tsV9TU9MAztK4gZ7XQI4Xzmsb6PHMun/vRvxbwAj2CYxgn6An6enp/b5G0JP1+Ph4RUREyOPx\nBLR7PB4lJCR02ychIaHbeovF4u+zcuVKPfrooyotLVVGRobsdruqq6v1q1/9Su3t7X1dDwAAABA2\ngp6sR0ZGKjMzU06nU08//bS/3el0avbs2d32mTRpkl588UVdu3bN/9z6/v37NXLkSKWmpkqSvvrq\nK0VERAT0GzRokCwWi65fv97jfGw2W/BV3WgAPskXMK8wGy+c1xYw3kCvDSHxzQkY/xa4GfYJjGCf\nIJgLFy70+xqGvgZTWlqqLVu2aNOmTTp16pRKSkrkcrlUVFQkSVq2bJlmzJjhr58zZ45iYmKUn5+v\nEydOaNeuXVq1apWef/55f82TTz6pPXv2aP369Wpra1NDQ4NKSkqUmZmp5OTkfi8MAAAAuNMZemY9\nLy9P586dU3l5uVwulzIyMlRXV+cP1W63W21tbf762NhYOZ1OLVy4UBMmTFBcXJyWLFkih8Phr5k3\nb54uXbqkqqoqvfDCC/rWt76l6dOna+XKlbd4iQAAAMCdyfALpkVFRf6T9Btt3ry5S9vYsWN1+PDh\nm15z4cKFWrhwodEpAAAAAHcVQ4/BAAAAABh4hHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJkVY\nBwAAAEyKsA4AAACYFGEdAAAAMCnCOgAAAGBShHUAAADApAjrAAAAgEkR1gEAAACTIqwDAAAAJkVY\nBwAAAEyKsA4AAACYFGEdAAAAMCnDYb26ulppaWmKjo6WzWZTfX39TeuPHz+uadOmKSYmRikpKVqx\nYkWXms7OTr300ktKS0vT0KFDNWrUKK1bt673qwAAAADC0GAjRbW1tXI4HFq/fr2ysrJUVVWl3Nxc\ntba2Kjk5uUv9xYsXZbfbNW3aNDU3N6u1tVX5+fkaNmyYFi9e7K/78Y9/rPb2dm3cuFEPPPCAPB6P\nrly5cutWBwAAANzBDIX1iooKFRYWqrCwUJJUWVmpffv2qaamRuXl5V3qt2/fritXrmjr1q2KiorS\nmDFj1NraqjVr1vjD+v79+3Xo0CF9+umn+va3vy1Juu+++27VugAAAIA7XtDHYDo7O9Xc3Cy73R7Q\nnp2drcbGxm77HD16VFOnTlVUVJS/LScnR+3t7Tp9+rQkac+ePZowYYJef/11paSk6MEHH1RJSYku\nX77cn/UAAAAAYSPoyXpHR4e8Xq+sVmtAu9Vq1QcffNBtH7fbrZSUlC71Pp9Pbrdbqamp+uyzz3Tk\nyBENGTJEu3bt0vnz57Vo0SK5XC7t3Lmzx/k0NTUZWdeAG+h5DeR44by2gR7PrPv3bsS/BYxgn8AI\n9gl6kp6e3u9rGHoM5na4fv26Bg0apLffflvDhg2TJK1bt04//OEP9cUXX+g73/lOqKYGAAAAmELQ\nsB4fH6+IiAh5PJ6Ado/Ho4SEhG77JCQkdFtvsVj8fRITE5WUlOQP6pI0ZswY+Xw+nTlzpsewbrPZ\ngk25q7qG3vfppYB5hdl44by2gPEGem0IiW9OwPi3wM2wT2AE+wTBXLhwod/XCPrMemRkpDIzM+V0\nOgPanU6nsrKyuu0zadIkHTlyRNeuXfO37d+/XyNHjlRqaqokKSsrS+3t7frqq6/8NZ988oksFou/\nBgAAALibGfrOemlpqbZs2aJNmzbp1KlTKikpkcvlUlFRkSRp2bJlmjFjhr9+zpw5iomJUX5+vk6c\nOKFdu3Zp1apVev755wNq7r33XhUUFOjkyZNqaGiQw+HQ7NmzFR8ff4uXCQAAANx5DD2znpeXp3Pn\nzqm8vFwul0sZGRmqq6vzf2Pd7Xarra3NXx8bGyun06mFCxdqwoQJiouL05IlS+RwOPw199xzjw4c\nOKDi4mI98sgjiouL06xZs/Tqq6/e4iUCAAAAdybDL5gWFRX5T9JvtHnz5i5tY8eO1eHDh296zfT0\ndO3bt8/oFAAAAIC7iqHHYAAAAAAMPMI6AAAAYFKEdQAAAMCkCOsAAACASRHWAQAAAJMirAMAAAAm\nRVgHAAAATIqwDgAAAJgUYR0AAAAwKcI6AAAAYFKEdQAAAMCkCOsAAACASRHWAQAAAJMirAMAAAAm\nRVgHAAAATMpwWK+urlZaWpqio6Nls9lUX19/0/rjx49r2rRpiomJUUpKilasWNFjbX19vSIjI/Xw\nww8bnzkAAAAQ5gyF9draWjkcDpWVlamlpUWTJ09Wbm6uzp492239xYsXZbfblZiYqObmZr3xxht6\n7bXXVFFR0aX2/PnzmjdvnmbMmNG/lQAAAABhxlBYr6ioUGFhoQoLCzV69GhVVlYqMTFRNTU13dZv\n375dV65c0datWzVmzBg99dRTWrp0qdasWdOl9qc//any8/M1ceLE/q0EAAAACDNBw3pnZ6eam5tl\nt9sD2rOzs9XY2Nhtn6NHj2rq1KmKioryt+Xk5Ki9vV2nT5/2t1VXV+uPf/yjysrK+jp/AAAAIGwF\nDesdHR3yer2yWq0B7VarVW63u9s+bre723qfz+fv8/HHH2vFihV68803ZbFY+jp/AAAAIGwNDsWg\n165d0zPPPKPVq1frvvvukyT5fD5DfZuamm7n1PpsoOc1kOOF89oGejyz7t+7Ef8WMIJ9AiPYJ+hJ\nenp6v68RNKzHx8crIiJCHo8noN3j8SghIaHbPgkJCd3WWywWJSQkyOVyqbW1VQUFBcrPz5ckXb9+\nXT6fT1FRUdq7dy8vnCLs/X9fXL3tY/z/3xky4GMBAIBbJ2hYj4yMVGZmppxOp55++ml/u9Pp1OzZ\ns7vtM2nSJL344ou6du2a/7n1/fv3a+TIkUpNTdXXX3+t48ePB/SpqqrSgQMHtHv3bqWmpvY4H5vN\nZmhhAeoaet+nlwLmFWbjhfPaAsZjbbdmLJP75gTsTpkvQoN9AiPYJwjmwoUL/b6Goa/BlJaWasuW\nLdq0aZNOnTqlkpISuVwuFRUVSZKWLVsWcBI+Z84cxcTEKD8/XydOnNCuXbu0atUqPf/885KkwYMH\n63vf+17AnxEjRmjIkCEaM2aMYmJi+r0wAAAA4E5n6Jn1vLw8nTt3TuXl5XK5XMrIyFBdXZ2Sk5Ml\n/eWF0ra2Nn99bGysnE6nFi5cqAkTJiguLk5LliyRw+G4PasAAAAAwpDhF0yLior8J+k32rx5c5e2\nsWPH6vDhw4Ynsnz5ci1fvtxwPQAAABDuDD0GAwAAAGDgEdYBAAAAkyKsAwAAACZFWAcAAABMirAO\nAAAAmBRhHQAAADApwjoAAABgUoR1AAAAwKQI6wAAAIBJEdYBAAAAkyKsAwAAACZFWAcAAABMirAO\nAAAAmBRhHQAAADApwjoAAABgUoR1AAAAwKQMh/Xq6mqlpaUpOjpaNptN9fX1N60/fvy4pk2bppiY\nGKWkpGjFihUBf//uu+8qJydHI0aMUGxsrCZOnKjf/OY3fVsFAAAAEIYMhfXa2lo5HA6VlZWppaVF\nkydPVm5urs6ePdtt/cWLF2W325WYmKjm5ma98cYbeu2111RRUeGv+fDDD/XYY49p7969amlp0cyZ\nMzVr1iw1NDTcmpUBAAAAd7jBRooqKipUWFiowsJCSVJlZaX27dunmpoalZeXd6nfvn27rly5oq1b\ntyoqKkpjxoxRa2ur1qxZo8WLF0uS1q5dG9DnpZde0vvvv6/du3crKyurv+sCAAAA7nhBT9Y7OzvV\n3Nwsu90e0J6dna3GxsZu+xw9elRTp05VVFSUvy0nJ0ft7e06ffp0j2NdvHhRcXFxRucOAAAAhLWg\nJ+sdHR3yer2yWq0B7VarVR988EG3fdxut1JSUrrU+3w+ud1upaamdulTVVWlzz//XM8999xN59PU\n1BRsyiEx0PMayPHCeW0DPR5rM487bb4IDfYJjGCfoCfp6en9voahx2But1//+tdaunSpdu7c2SXk\nAwAAAHeroGE9Pj5eERER8ng8Ae0ej0cJCQnd9klISOi23mKxdOnzzjvvaN68edq+fbtmzpwZdMI2\nmy1oTRd1t/+l1YB5hdl44by2gPFY260Zy+S+OQG7U+aL0GCfwAj2CYK5cOFCv68R9Jn1yMhIZWZm\nyul0BrQ7nc4eXwSdNGmSjhw5omvXrvnb9u/fr5EjRwY8ArNz507NmzdP27Zt06xZs/q6BgAAACAs\nGfp0Y2lpqbZs2aJNmzbp1KlTKikpkcvlUlFRkSRp2bJlmjFjhr9+zpw5iomJUX5+vk6cOKFdu3Zp\n1apVev755/01O3bs0LPPPquVK1dqypQp8ng88ng8+tOf/nSLlwgAAADcmQw9s56Xl6dz586pvLxc\nLpdLGRkZqqurU3JysqS/vFDa1tbmr4+NjZXT6dTChQs1YcIExcXFacmSJXI4HP6aDRs2yOv1yuFw\nBLT/4AcqDRRAAAAJaklEQVQ/0MGDB2/V+gAAAIA7luEXTIuKivwn6TfavHlzl7axY8fq8OHDPV7v\n0KFDRocGAAAA7kqGHoMBAAAAMPAI6wAAAIBJmeI76wDCy/QB+FTkwdz/9zWqPo3Xiz79HquXBnK8\n/z0WAMB8OFkHAAAATIqwDgAAAJgUYR0AAAAwKcI6AAAAYFKEdQAAAMCkCOsAAACASRHWAQAAAJMi\nrAMAAAAmRVgHAAAATIqwDgAAAJgUYR0AAAAwKcI6AAAAYFKGw3p1dbXS0tIUHR0tm82m+vr6m9Yf\nP35c06ZNU0xMjFJSUrRixYouNR9++KFsNpuio6P1wAMPaMOGDb1fAQAAABCmBhspqq2tlcPh0Pr1\n65WVlaWqqirl5uaqtbVVycnJXeovXrwou92uadOmqbm5Wa2trcrPz9ewYcO0ePFiSdIf/vAHPf74\n45o/f77efPNNHTlyRP/8z/+sESNGaNasWbd2lQCAoKbXNdz2MQ7mZoVkvD6P1Yt+d9za+jBWOI53\nS8bq4z4BjDB0sl5RUaHCwkIVFhZq9OjRqqysVGJiompqarqt3759u65cuaKtW7dqzJgxeuqpp7R0\n6VKtWbPGX1NTU6OkpCStXbtWo0eP1vz58zVv3jytXr361qwMAAAAuMMFDeudnZ1qbm6W3W4PaM/O\nzlZjY2O3fY4ePaqpU6cqKirK35aTk6P29nadPn3aX5OdnR3QLycnR01NTfJ6vb1eCAAAABBugob1\njo4Oeb1eWa3WgHar1Sq3291tH7fb3W29z+fz9+mp5uuvv1ZHR0evFgEAAACEI4vP5/PdrMDlcikp\nKUm//e1vNWXKFH/7ihUr9NZbb6m1tbVLn5ycHKWkpGjjxo3+tv/5n/9Ramqqfve73+nRRx/V6NGj\n9dxzz6msrMxfc+TIEU2bNk3t7e0BQf7ChQv9WiQAAAAQSsOHD+9Tv6An6/Hx8YqIiJDH4wlo93g8\nSkhI6LZPQkJCt/UWi8Xfp6eawYMHKz4+vleLAAAAAMJR0LAeGRmpzMxMOZ3OgHan06msrO7faJ40\naZKOHDmia9eu+dv279+vkSNHKjU11V9z4zX3798vm82miIiIXi8EAAAACDdBH4ORpJ07d2ru3Lmq\nqqpSVlaWampqtHnzZp08eVLJyclatmyZPvroIx04cECS9OWXX+q73/2upk2bpl/84hf65JNPVFBQ\noFdeeUUOh0PSXz7d+NBDD2n+/Pn62c9+pvr6ei1atEg7duzQj370o9u7agAAAOAOYOg763l5eTp3\n7pzKy8vlcrmUkZGhuro6/zfW3W632tra/PWxsbFyOp1auHChJkyYoLi4OC1ZssQf1CVp1KhR2rt3\nrxYvXqz169dr5MiR+uUvf0lQBwAAAP4vQyfrAAAAAAaeoV+KFGrV1dVKS0tTdHS0bDab6uvrQz0l\nmMgrr7yiQYMGBfwZOXJkqKeFEDty5Ij+/u//XsnJyRo0aJC2bdvWpebll19WUlKSYmJi9Ld/+7c6\nefJkCGaKUAm2RwoKCrrcWyZPnhyi2SJUXn31VT3yyCMaPny4RowYob/7u7/TiRMnutRxP7l7Gdkj\n/bmfmD6s19bWyuFwqKysTC0tLZo8ebJyc3N19uzZUE8NJvLd735XHo9HbrdbbrdbH3/8cainhBC7\ndOmSHnroIVVWViomJqbL369atUoVFRWqqqpSU1OTRowYIbvdrsuXL4dgtgiFYHtEkux2e8C9Ze/e\nvQM8S4Tab3/7Wy1atEi/+93vdOjQIQ0ePFgzZszQ+fPn/TXcT+5uRvaI1I/7ic/kHn30Ud/Pfvaz\ngLb09HTfv/zLv4RoRjCbl19+2ffQQw+FehowsWHDhvm2bt0a0JaYmOh79dVX/T9fuXLF91d/9Ve+\nf/u3fxvo6cEEutsj+fn5vieffDJEM4JZXbp0yRcREeF77733/G3cT/C/dbdH+nM/MfXJemdnp5qb\nm2W32wPas7Oz1djYGKJZwYw+++wzJSUlKS0tTf/4j/8Y8MIzcKO2tja53e6Ae8vQoUP1N3/zN9xb\nEKC+vl5Wq1WjR4/WggUL9MUXX4R6SgixL7/8UtevX1dcXJwk7ifo6sY98o2+3k9MHdY7Ojrk9XoD\nfpupJFmtVrnd7hDNCmYzceJEbdmyRf/xH/+hjRs3yu12a/LkyfrTn/4U6qnBpNxutywWC/cW3FRu\nbq62bdumgwcPas2aNfr973+vxx57TJ2dnaGeGkKopKRE3//+9zVp0iRJ3E/Q1Y17ROrf/cTQpxsB\nM8vJyQn4eeLEibr//vu1devWgM+FAkBv5OXl+f/32LFj9f3vf1+pqal6//33+czwXaq0tFSNjY1q\naGiQxWIJ9XRgQj3tkf7cT0x9sh4fH6+IiAh5PJ6Ado/Ho4SEhBDNCmYXExOjsWPH6r//+79DPRWY\nVEJCgnw+H/cW9EpiYqKSk5O5t9ylFi9erNraWh06dMj/29gl7if4f3raI93pzf3E1GE9MjJSmZmZ\ncjqdAe1Op1NZWVkhmhXM7s9//rNOnTqlxMTEUE8FJnX//fcrISEh4N7y5z//WUeOHOHegh598cUX\n+vzzz7m33IVKSkr8ISw9PT3g77ifQLr5HulOb+4nES+//PLLt2COt01sbKyWL1+uxMRExcTEaMWK\nFTpy5Ij+/d//XcOHDw/19GACS5Ys0dChQ+Xz+fTJJ59o0aJF+vTTT7Vhwwb2yF3s8uXLam1tldvt\n1qZNm/Twww9r+PDh6uzs1PDhw+X1erVy5UqNHj1aXq9XpaWl8ng82rBhg6KiokI9fQyAm+2RwYMH\n6xe/+IViY2Pl9XrV0tKif/qnf5LP59O6devYI3eRhQsXatu2bXrnnXeUnJysy5cv6/Lly7JYLP59\nwP3k7hZsj1y+fLl/95Nb8IWa266mpsZ3//33+4YOHeqz2Wy++vr6UE8JJvLMM8/4kpKSfEOGDPEl\nJyf7/uEf/sHX2toa6mkhxA4fPuyzWCy+QYMGBfwpKCjw17zyyiu+kSNH+qKjo33Tpk3znThxIoQz\nxkC72R65cuWKLycnx2e1Wn1DhgzxjRo1yldYWOg7e/ZsqKeNAdbdHhk0aJDvlVdeCajjfnL3CrZH\n+ns/sfh8Pt/A/HcHAAAAgN4w9TPrAAAAwN2MsA4AAACYFGEdAAAAMCnCOgAAAGBShHUAAADApAjr\nAAAAgEkR1gEAAACTIqwDAAAAJkVYBwAAAEzq/wDtMvOielNHwgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "colors = thinkplot.Brewer.Colors()\n", + "thinkplot.Hist(mix, width=0.9, color=colors[4])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows the result. As expected, values 1 through 4\n", + "are the most likely because any die can produce them. Values above 12\n", + "are unlikely because there is only one die in the box that can produce\n", + "them (and it does so less than half the time).\n", + "\n", + "`thinkbayes`provides a function named\n", + "`MakeMixture` that encapsulates this algorithm, so we could\n", + "have written:\n", + "\n", + "```python```\n", + " mix = thinkbayes.MakeMixture(pmf_dice)\n", + "```\n", + "\n", + "We’ll use MakeMixture again in Chapters [prediction] and\n", + " [observer]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "Other than the odds form of Bayes’s theorem, this chapter is not\n", + "specifically Bayesian. But Bayesian analysis is all about distributions,\n", + "so it is important to understand the concept of a distribution well.\n", + "From a computational point of view, a distribution is any data structure\n", + "that represents a set of values (possible outcomes of a random process)\n", + "and their probabilities.\n", + "\n", + "We have seen two representations of distributions: Pmfs and Cdfs. These\n", + "representations are equivalent in the sense that they contain the same\n", + "information, so you can convert from one to the other. The primary\n", + "difference between them is performance: some operations are faster and\n", + "easier with a Pmf; others are faster with a Cdf.\n", + "\n", + "The other goal of this chapter is to introduce operations that act on\n", + "distributions, like `Pmf.__add__`, `Cdf.Max`, and\n", + "`thinkbayes.MakeMixture`. We will use these operations later,\n", + "but I introduce them now to encourage you to think of a distribution as\n", + "a fundamental unit of computation, not just a container for values and\n", + "probabilities." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/06_Decision_Analysis.ipynb b/notebooks/06_Decision_Analysis.ipynb index 46bb551..c5de0ac 100644 --- a/notebooks/06_Decision_Analysis.ipynb +++ b/notebooks/06_Decision_Analysis.ipynb @@ -1,1170 +1,1155 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 1, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# format the book\n", - "%matplotlib inline\n", - "import sys\n", - "from __future__ import division, print_function\n", - "%load_ext autoreload\n", - "%autoreload 2 \n", - "import sys\n", - "sys.path.insert(0,'../code')\n", - "import book_format\n", - "book_format.load_style('../code')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#Decision Analysis\n", - "\n", - "## The *Price is Right* problem\n", - "\n", - "On November 1, 2007, contestants named Letia and Nathaniel appeared on\n", - "*The Price is Right*, an American game show. They competed\n", - "in a game called *The Showcase*, where the objective is to\n", - "guess the price of a showcase of prizes. The contestant who comes\n", - "closest to the actual price of the showcase, without going over, wins\n", - "the prizes.\n", - "\n", - "Nathaniel went first. His showcase included a dishwasher, a wine\n", - "cabinet, a laptop computer, and a car. He bid \\$26,000.\n", - "\n", - "Letia’s showcase included a pinball machine, a video arcade game, a pool\n", - "table, and a cruise of the Bahamas. She bid \\$21,500.\n", - "\n", - "The actual price of Nathaniel’s showcase was \\$25,347. His bid was too\n", - "high, so he lost.\n", - "\n", - "The actual price of Letia’s showcase was \\$21,578. She was only off by\n", - "\\$78, so she won her showcase and, because her bid was off by less than\n", - "\\$250, she also won Nathaniel’s showcase.\n", - "\n", - "For a Bayesian thinker, this scenario suggests several questions:\n", - "\n", - "1. Before seeing the prizes, what prior beliefs should the contestant\n", - " have about the price of the showcase?\n", - "\n", - "2. After seeing the prizes, how should the contestant update those\n", - " beliefs?\n", - "\n", - "3. Based on the posterior distribution, what should the contestant bid?\n", - "\n", - "The third question demonstrates a common use of Bayesian analysis:\n", - "decision analysis. Given a posterior distribution, we can choose the bid\n", - "that maximizes the contestant’s expected return.\n", - "\n", - "This problem is inspired by an example in Cameron Davidson-Pilon’s book,\n", - "*Bayesian Methods for Hackers*. The code I wrote for this\n", - "chapter is available from ; it reads\n", - "data files you can download from\n", - " and\n", - ". For more information see\n", - "Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The prior\n", - "\n", - "To choose a prior distribution of prices, we can take advantage of data\n", - "from previous episodes. Fortunately, fans of the show keep detailed\n", - "records. When I corresponded with Mr. Davidson-Pilon about his book, he\n", - "sent me data collected by Steve Gee at . It\n", - "includes the price of each showcase from the 2011 and 2012 seasons and\n", - "the bids offered by the contestants." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvYAAAEWCAYAAAANTdM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVOX+B/DPmWEZQFaRfRVBRdBU3FBATUkzLdPrklaa\n5dW0q3kz83cts7r3Zl69aje1xcrKyjTUFk3RDHBLUVEExQVFFAYB2ZeZYeb8/hg8MAGCBgzL5/16\n9YrznOec+c4jy3ee8yyCKIoiiIiIiIioVZMZOwAiIiIiIvrzmNgTEREREbUBTOyJiIiIiNoAJvZE\nRERERG0AE3siIiIiojaAiT0RERERURvAxJ6IiIiIqA1ocGK/YcMG+Pr6wsLCAiEhITh8+PA96ycm\nJiIiIgKWlpbw8PDA22+/bXD+t99+g0wmq/HfpUuXHuydEBERERG1YyYNqbRt2zYsXLgQGzduxJAh\nQ/DBBx9g9OjRSE5OhqenZ436hYWFGDlyJIYOHYr4+HhcuHABM2fOhJWVFRYtWmRQNzk5GQ4ODtKx\no6Pjn3xLRERERETtj9CQnWcHDBiAhx56CB9++KFUFhAQgIkTJ+Jf//pXjfobN27E0qVLkZWVBXNz\ncwDAP//5T2zcuBE3b94EoO+xHz58OLKzs9GxY8fGej9ERERERO1SvUNx1Go1Tp8+jcjISIPyyMhI\nHD16tNZrjh07hrCwMCmpv1s/IyMDaWlpBnVDQkLg5uaGESNG4LfffnuAt0BERERERPUm9jk5OdBq\ntXB2djYod3JyglKprPUapVJZo/7d47vXuLm5YdOmTYiKikJUVBS6du2Khx9+uN6x+0REREREVFOD\nxtjfL0EQ6q0TEBCAgIAA6XjgwIG4fv06Vq1ahSFDhkjlBQUFTREiEREREVGzsLW1bZbXqbfH3tHR\nEXK5HFlZWQblWVlZcHV1rfUaFxeXGr35d693cXGp87X69++Py5cv1xs0EREREREZqjexNzMzQ9++\nfbF//36D8ujoaISGhtZ6zaBBgxAXFweVSmVQ393dHd7e3nW+VkJCAtzc3BoaOxERERERVWrQUJxF\nixbh6aefRv/+/REaGopNmzZBqVRizpw5AIClS5fi5MmTOHDgAADgqaeewooVKzBjxgwsW7YMKSkp\nWLlyJd58803pnmvXroWvry8CAwOhVqvx1VdfYffu3YiKiqozjuZ6jNEexMfHA9BPXqbGw3ZtGmzX\npsF2bRps16bBdm06bNumYYzh5A1K7CdNmoTc3Fy88847yMzMRHBwMPbs2SOtYa9UKpGamirVt7Gx\nQXR0NObNm4eQkBA4ODjglVdewcsvvyzV0Wg0WLx4MW7evAkLCwsEBQVhz549GDVqVCO/RSIiIiKi\ntq/Bk2fnzp2LuXPn1nrus88+q1EWFBSEmJiYOu+3ePFiLF68uKEvT0RERERE91DvGHsiIiIiImr5\nmNgTEREREbUBTOyJiIiIiNqAJtmgioiIiMgYdDod1Gq1scNoVe4uRV5eXm7kSFoPMzMzyGQtr3+c\niT0RERG1CaIoQqVSQaFQQBAEY4fTaigUCmOH0KqIoojy8vIW+X3W8j5qEBERET0AtVoNMzOzFpds\nUdsiCALMzMxa5JMhJvZERETUJoiiCLlcbuwwqB2Qy+UQRdHYYdTAxJ6IiIiIqA1gYk9ERERE1AYw\nsSciIiIiagOY2BMRERG1AUOHDkX37t2NHQYZERN7IiIiojaCKwLdP6VSiddeew0PP/wwbG1tIZPJ\nsG3bNmOH9UCY2BMRERFRu3Xx4kW89957SE9PR+/evQG03g9I3KCKiIiIiNqtkJAQ5Obmwt7eHjEx\nMRg2bJixQ3pg7LEnIiIiauGKi4vxyiuvwNfXFwqFAk5OThg2bBji4uJq1E1OTsbw4cNhZWUFDw8P\nrFq1qkadnJwczJ49Gy4uLrCwsEBwcDA++eQTgzp9+/bFuHHjDMoGDRoEmUyGo0ePSmUHDx6ETCZD\nTEyMVFZQUIDFixejc+fOUCgU8PDwwLRp05CRkQEA0Gg0WL58Ofr16wcHBwdYWlpiwIAB2L17d41Y\nDx48iPDwcDg4OMDKygpdunTBSy+9ZFBHpVJhxYoV8Pf3l15v0aJFKCsrq7dtO3ToAHt7ewBokWvT\n3w/22BMRERG1cHPnzsX27dsxf/589OjRA3fu3MGJEydw7tw5hIWFSfXy8/Px6KOP4sknn8TkyZOx\nfft2LFmyBMHBwRg1ahQAoLy8HMOGDUNKSgrmz58PPz8/7Ny5E7Nnz0Zubi6WLFkCAAgPD8cXX3wh\n3bu0tBSnTp2CXC5HbGwsQkNDAQBxcXEwNzfHwIEDAQAlJSWIiIhAUlISZs6ciZCQEOTk5GDv3r24\nevUq3NzcUFBQgI8++ghTp07FrFmzUFZWhq+//hrjx4/Hnj17pFiTk5MxZswY9OrVCytWrIClpSWu\nXLmC/fv3S3GJoojx48cjNjYWs2fPRmBgIJKTk7FhwwYkJSVh3759TfuP05KILVx+fr70HzWekydP\niidPnjR2GG0O21VPp9OJd/LKRLWmolHux3ZtGmzXpsF2bRoNadeysrJmiqb52dnZiS+99NI960RE\nRIiCIIhffvmlVKZWq0VXV1dx4sSJUtm6detEQRDEL774QirTarXiiBEjRIVCIebm5oqiKIo7duwQ\nBUEQz549K4qiKB48eFCUyWTiX/7yF3H06NHStcOHDxeHDBkiHS9fvlwUBEHcsWNHnbFqtVpRrVYb\nlKnVajEoKEgcMWKEVLZ27VpREAQpptps3bpVlMlkYmxsbI1yQRDE/fv313ntHx06dEgUBEHctm1b\nvXXr+34zRg7LHnsiahRl5RokJt3GqXOZOH1Widw7pTA1kcPP1x7+fg4I8OuIrl06oqODRaudlERE\nbcu3UUn4bldSk9x70hM9MOXJHo12Pzs7Oxw/fhwZGRlwc3Ors56lpSWmT58uHZuamqJ///5ITU2V\nyn7++Wc4OTkZ1JPJZFi4cCEOHjyIAwcOYNKkSRgyZAgAfY98z549ERsbi+DgYDz++OOYN28eRFFE\nRUUFjh8/jgULFkj32rFjB4KCgjBhwoQ645TJZJDJ9CPC1Wo1iouLodPpEBYWhm+//dbgfQPAzp07\nMXPmTOma6r777jsEBAQgMDAQOTk5Unl4eDgEQcChQ4cwcuTIOmNpS5jYE9EDu51Tgt/jb+HU2Uwk\np2SjokJncF5TocXFyzm4eLnqF629nQV6BTlj+l+C4WBv0dwhExG1SqtWrcKzzz4LLy8v9O7dG6NG\njcLTTz+NgIAAg3ru7u41rrWzs8O5c+ek47S0NHTp0qVGJ0u3bt2k8wDg7OyMgIAAxMbGYt68eYiL\ni0N4eDjCw8NRWFiIhIQEqNVqlJWVITw8XLrP1atXMX78+Hrf0yeffIL//ve/uHjxosHY9urJ++TJ\nk7F582a88MILeO211zB8+HA88cQTmDRpEuRyOQDg0qVLSElJQadOnWq8hiAIyM7OrjeWtoKJPRHd\nN7Vaix0/XMDOny9Cq9XVWsfMVA61RlujPC+/DL8dvo5zSVn4v5eHoLOPfVOHS0TU6k2cOBFhYWHY\nvXs39u/fj/Xr1+O9997D559/jqlTp0r17ia7fyQ+4KTQ8PBw/Pjjj1LP/Ny5c+Hp6QkvLy/ExMSg\noqICMpkMgwcPlq5pyFPZrVu3Yvbs2Rg3bhyWLl0KJycnmJiY4NNPP8XXX38t1VMoFIiJiUFsbCz2\n7NmDffv2Ydq0aVizZg3i4uKgUCig0+nQo0cPrFu3rtbXutcTjraGiT0R3Zez55XY9PlpZN0urnHO\n29MOfXq5oG8vV3Tt0hFFxWpcvnoHl67m4tLVXFxOvYPy8goAwJ28Mvzjn4ewaO5A9OvTfn7pElHL\nMeXJxh0u09ScnZ0xe/ZszJ49GwUFBRg4cCCWL19ukNg3hLe3NxISEqDT6Qx6xy9evAgA8PHxkcqG\nDBmCTz75BFu3bjXomQ8PD0dsbCx0Oh169eoFa2tr6Ro/Pz8kJibeM4bt27fDz88Pu3btMijfvHlz\njQ8GgiAgIiICERERWLlyJTZt2oQXX3wRO3fuxNSpU+Hn54fTp09j+PDh99UObRGXuySiBskvKMfa\njb9jxXuxBkm9f+eOmDMzBB/99zH895+ReHpSTwR27QS5XAY7WwX69XHDtL8EY8VrQ/Hlxiew9OUh\nsLQwBQCoVBX499oj+PGXS61+iTEioqai0+lQUFBgUGZrawsfH58a5XWpniyPHTsW2dnZBj3jOp0O\n69atg0KhwIgRI6Tyu4n8u+++i4CAADg5OQEAwsLCEBcXhyNHjhgMwwH0TxeSkpKwY8eOOuMxMTGB\nKIoGv/tTU1Oxc+dOg3p37typce3dTaTy8/MBAFOmTEFWVhY2btxYo65KpUJxcc2OqLaKPfZEdE86\nnYiDsdfw5bZzKC5RS+VWlmaYPikYI4d2hkzWsMmwcrkM/Xq74V+vD8e//3sYWdklAER89nUCMpRF\neP7p3pDL2d9ARFRdYWEh3N3dMXHiRPTs2RM2NjY4cuQI9u3bV2M997o6SaqXv/DCC/joo48wa9Ys\nnDlzBr6+vti1axd+/fVXvPvuu9Ka7oC+997NzQ0pKSl4/vnnpfLw8HDk5uYCgMFymwCwePFifP/9\n95g6dSr279+PPn36ID8/H7/88gveeusthIeHY9y4cYiKisK4ceMwduxY3Lp1Cxs3bkS3bt2QkJAg\n3eutt95CTEwMxowZA29vb+Tl5WHTpk3o0KEDHnvsMQDA9OnTsWPHDsybNw8xMTEYPHgwRFFESkoK\ntm/fjh07dtT48PFH77zzDgDg2rVrAICoqChcunQJALBs2bJ7XtuSMLEnojqVlmmwdtPviD+TYVAe\nNsgLM6Y+BHs7xQPd18vDFv9+42GsXHcEKVf0fxj2/XoVytsleGX+QFhZmv3p2ImI2gorKyvMnz8f\n0dHR+OGHH6DRaNC5c2esXr3aYDUaQRBqHd/+x3Jzc3McOnQIS5cuxdatW5Gfnw9/f398/PHHmDVr\nVo3rw8PDsW3bNoPkuGvXrujUqRNycnJqJPaWlpaIjY3Fm2++iaioKGzZsgXOzs6IiIiQJvs+88wz\nuH37NjZu3IgDBw7A398fa9euxeXLl3H27FnpXk888QTS09OxZcsWZGdno2PHjggNDcUbb7wBT09P\n6f1FRUVh7dq12LJlC3bv3g0LCwv4+flh3rx5CA4OrreN33jjDQiCAFEUIQgCtm/fju+++w6CILSq\nxF4QW/jz7+qPmGxtbY0YSdsSHx8PQL+NMjWettSuWdkl+Pd/D+PGzaqfQWenDvjrs33wULBLo7yG\nWq3F/z45icPHb0hlXTo74J/LhsHUpGoCWFtq15aE7do02K5NoyHtWl5eDoXiwTociO5Xfd9vxshh\n+cybiGpITsnGkjcPGCT1jz0SgHX/eqTRknoAMDOT4+W5AzDpiUCp7ErqHXz/w4VGew0iIqL2gkNx\niMjAobjr2PhZvLQmvVwuw9yZfTE83LdJXk8QBEx5Mgjm5ib4cpt+neUdP1xAvz7u8ONSmERERA3G\nHnsiAgBotTp88e1ZvP/xCSmpt7E2x1tLhzZZUl/duFEB6BbgCEA/YXf9hydqXQefiIiIasfEnoig\nVmvx3vqj2LUnRSrz8rDFyjdHoHtlst3U5HIZXnq+H8zM9A8S028V4Ludyc3y2kRERG0BE3uidk5T\nocWq94/iZLWVb0Iql6R07mTVrLG4uljjmclVqxfs/PkiLlWumkNERET3xsSeqB2rqNBh9f+O49TZ\nTKls3KiuWPK3UGkTqeY26uEu6NGtEwD9usvvf3wSGo3OKLEQERG1JkzsidoprVaH/278HSdO35LK\nJoztjmen9jTqJlEymYB5z/eDQqEfknMrsxAH4m4bLR4iIqLWgok9UTuk1erw/kcncexkulT2+Oiu\neGpiUK2bmzQ3F6cOeHZKL+n4yIkcpN0sMWJERERELR8Te6J2RqcTseHTeMQeS5PKxoz0xzNTeraI\npP6uyGGd0bOHMwBAFIHvf7oFlarCyFERERG1XEzsidoRURTx4ZZTOBR3XSqLHOaH56Y/1KKSekC/\nvv385/tJY/1z89TY+XNKPVcRERG1X0zsidqRr3ecR/ShVOn44XBfzH62T4tL6u9y7GiJZ6oNyflp\n3yUUl6iNGBEREVHLxcSeqJ2IT8jA9z9ekI4jBntjzsy+kMlaZlJ/18PhPnB0MAMAlJZp8NO+S0aO\niIiIqGVqUGK/YcMG+Pr6wsLCAiEhITh8+PA96ycmJiIiIgKWlpbw8PDA22+/XWfdw4cPw8TEBMHB\nwXXWIaI/JzunBOs/PCEdPxTsgvnP9zPq6jcNJZfLMGywk3T8077LKCpWGTEiIqKWaejQoejevbux\nwyAjqvev+rZt27Bw4UIsW7YMCQkJCA0NxejRo5Genl5r/cLCQowcORKurq6Ij4/HunXrsGrVKqxZ\ns6ZG3by8PDzzzDMYMWJEix0KQNTaaSq0WP3BcWkIi4O9BRbOGdAqkvq7egbaolPHql77H3+5bOSI\niIhaJuZT9+/gwYN47rnnEBAQACsrK/j5+eGFF16AUqk0dmj3rd6/7GvWrMHMmTMxa9YsdO3aFevX\nr4erqys2btxYa/2tW7eivLwcW7ZsQWBgICZMmIAlS5bUmtjPmjULM2fOxKBBgyCK4p9/N0RUw1ff\nJeLSVf3urTKZgL/PGwQba3MjR3V/ZDIBw4dU67Xff4m99kRE1CiWLFmC2NhYTJgwAe+//z6mTJmC\n7777Dr179251yb3JvU6q1WqcPn0ar776qkF5ZGQkjh49Wus1x44dQ1hYGMzNzQ3qv/7660hLS4O3\ntzcA/fCe7OxsLFu2DCtWrPiz74OIanE8/iZ+/KVqTPr0ST3RPcCxUe6tLFbhSHo+0gvLUFqhQ6la\ni9IKLUo1WpRpdFBpdXC2MoO/gxUCOloiwMEKnjYKyB9wTH9QN1ucvVCB9FuFKC+vwO49lzB9Eofw\nERHRn7N27VoMGTLEoGzUqFGIiIjA+vXr8a9//ctIkd2/e/bY5+TkQKvVwtnZ2aDcycmpzk8wSqWy\nRv27x3evSUxMxFtvvYWvvvqKj4yImogyqxj/+/ikdBzS2w2Pjw74U/e8XaJG1MUsLIq+iFk/ncen\nZ28i+loujqTn4UxWIVJyS5BeWI6cMjWK1BW4kleKvVezse5EGub9koy/fJ+AVw+k4POzt6C8zx53\nmUzApCd6SMd7DlxGYRF77YmofSguLsYrr7wCX19fKBQKODk5YdiwYYiLi6tRNzk5GcOHD4eVlRU8\nPDywatWqGnVycnIwe/ZsuLi4wMLCAsHBwfjkk08M6vTt2xfjxo0zKBs0aBBkMplBB+/Bgwchk8kQ\nExMjlRUUFGDx4sXo3LkzFAoFPDw8MG3aNGRkZAAANBoNli9fjn79+sHBwQGWlpYYMGAAdu/eXSPW\ngwcPIjw8HA4ODrCyskKXLl3w0ksvGdRRqVRYsWIF/P39pddbtGgRysrK6m3bPyb1ABAWFgYHBwdc\nuHChlitarnv22D+I+hJ1lUqFyZMn4z//+Y/Ue99Q8fHxfyY0qgXbtGkYu101Gh0++ioV2TnlAAB7\nW1MMCTHBqVOn7vte5VoRJ+5ocTZfixuluj8VVymAk4XFOJl2G1+eSkVPWzmGOpnA3aJh4/1NBSU6\nWGqRla1CSQnw/qZoPDLM5U/FRMb/fm2r2K5N417t6u3tDYVC0YzRNJ+5c+di+/btmD9/Pnr06IE7\nd+7gxIkTOHfuHMLCwqR6+fn5ePTRR/Hkk09i8uTJ2L59O5YsWYLg4GCMGjUKAFBeXo5hw4YhJSUF\n8+fPh5+fH3bu3InZs2cjNzcXS5YsAQCEh4fjiy++kO5dWlqKU6dOQS6XIzY2FqGhoQCAuLg4mJub\nY+DAgQCAkpISREREICkpCTNnzkRISAhycnKwd+9eXL16FW5ubigoKMBHH32EqVOnYtasWSgrK8PX\nX3+N8ePHY8+ePVKsycnJGDNmDHr16oUVK1bA0tISV65cwf79+6W4RFHE+PHjERsbi9mzZyMwMBDJ\nycnYsGEDkpKSsG/fvvtu7+LiYhQVFcHRse6n3EVFRTh//nyd5/39/e/7df+seyb2jo6OkMvlyMrK\nMijPysqCq6trrde4uLjU6M2/e72LiwsyMzNx8eJFzJw5EzNnzgQA6HQ6iKIIU1NT7N27FyNGjHjg\nN0REwN5flchQ6pN6uQyY8oQnLC3u/3P8xUItdtzUoEBTcw6MTAACrGXobi2HlYkAhRwwl+n/r5AJ\nkAtAZrkO6aU6pJfpcLNUNLiPTgQS8rVIyNfC31qGoZ1M4N9Bds/Ogbtj7b/ZqZ+8f+xULgb3d0QH\nq0bvoyCidmDr+Qx8fT6zSe79VJArpgW5Ndr9fvrpJ8yePRv/+c9/7llPqVTiiy++wPTp0wEAzz33\nHLy9vbF582YpWf7oo4+QlJSELVu24Omnnwag/+DwyCOP4M0338QLL7wABwcHDBkyBOvWrcO5c+fQ\ns2dPHD9+HFqtFhMmTEBsbCxee+01APrEPiQkRBqGvWrVKpw7dw7bt2/HhAkTpNj+7//+T/rawcEB\nN27cgKmpqVQ2f/589OnTB6tXr5ZijY6Ohlqtxt69e+Hg4CDV/fe//y19/c0332Dfvn347bffDD7k\nhISEYPr06YiOjsbIkSPvo7X1w3M0Gg2mTJlyX9cZ2z3/GpqZmaFv377Yv3+/wT9MdHQ0/vKXv9R6\nzaBBg7BkyRKoVCrpHzg6Ohru7u7w9vZGRUVFjU83H3zwAaKjo7Fr16579uKHhIQ0+I3Rvd3t8WCb\nNq6W0K5JF7NxPiUVVlaWAIBZ03tjTOT99RoUqirw8Zmb+PV2LmBmAkv9gjQwEQT0crHBEE87DHK3\ng7X5/SXUd8o0SM4pxp4r2TibVSSV39IBW7MAP7Ulng52Qz83W4Prqrdrnz4izl6IRlp6PgAgLcMC\nz07tBbp/LeH7tS1iuzaNhrRreXl5c4XT7Ozs7HD8+HFkZGTAza3uDwyWlpZSUg8Apqam6N+/P1JT\nqzYn/Pnnn+Hk5GRQTyaTYeHChTh48CAOHDiASZMmSUNU4uLi0LNnT8TGxiI4OBiPP/445s2bB1EU\nUVFRgePHj2PBggXSvXbs2IGgoCCD3PGPZDIZZDL901q1Wo3i4mLodDqEhYXh22+/NXjfALBz507M\nnDlTuqa67777DgEBAQgMDEROTo5UHh4eDkEQcOjQoftK7GNjY7FixQpMmjQJDz/8cJ31rK2t7/n9\nWFBQ0ODXbCz1Pv9etGgRPv/8c2zevBkXLlzAggULoFQqMWfOHADA0qVLDXrYn3rqKVhaWmLGjBlI\nSkpCVFQUVq5ciUWLFgEATExMEBgYaPBfp06dYG5ujsDAQFhZWTXRWyVq+3Q6EZ99nSAd9+/jjkdH\ndrmvexxJz8OLe5Px6/VcqczG3AQv9vXCl0/0xFsRXRDZ2fG+k3oAcLAwxRBPe/xrWADWRnZDuJc9\nZNV66K/mleLN2Ct4/2QayjTaWu8hkwmYPD5QOt578Cry8tvuH3MiIkDfC56UlAQvLy/069cPr7/+\nOi5dqrlhn7u7e40yOzs75OXlScdpaWno0qVLjSek3bp1k84D+jmSAQEBiI2NBaBP8MPDwxEeHo7C\nwkIkJCTg9OnTKCsrQ3h4uHSfq1evIigoqN739Mknn6BHjx6wsLCAo6MjnJycsGnTJhQWFkp1Jk+e\njCFDhuCFF16As7MzJk+ejG+++QZabdXfiEuXLiElJQWdOnWCk5OT9J+XlxcAIDs7u95Y7rp48SLG\njx+Pnj17YvPmzQ2+rqWo9y/zpEmTkJubi3feeQeZmZkIDg7Gnj174OnpCUD/yKf6p0AbGxtER0dj\n3rx5CAkJgYODA1555RW8/PLLdb6GIAicREvUCGKPpSH1uv6Xt6mJHM9Nf6jBP1t5ZRpsOp2Ow+l5\nBuURXg6Y3ccDdgrTOq58MP4OVlgS2hnPFquwK+U29qfmQKXVj+H/5WoOzmYV4e8DfdDdsUONawf0\ndYevlx2u3ciHWl2BXXsuYuZTDzVqfETU9k0LcmvU4TJNaeLEiQgLC8Pu3buxf/9+rF+/Hu+99x4+\n//xzTJ06Vaonl8trvf5BlxUPDw/Hjz/+KPXMz507F56envDy8kJMTAwqKiogk8kwePBg6ZqG/N3Z\nunUrZs+ejXHjxmHp0qVwcnKCiYkJPv30U3z99ddSPYVCgZiYGMTGxmLPnj3Yt28fpk2bhjVr1iAu\nLg4KhQI6nQ49evTAunXran2tez3hqC49PR2RkZGwt7fHnj17WmVnc4O63ObOnYu5c+fWeu6zzz6r\nURYUFGQwM7o+y5cvx/Llyxtcn4hqUqkqsHV71TC3saMC4OTYsF9K1/PLsOy3y8gr10hlDgpTzO/n\nhQHudo0ea3UuHcwxp68npvRwwaZT6Yir/GCRWazCqwcvYVJ3FwSIIuTV/lAIgoDJ43vg3XVHAAD7\nfk3FxHHdYd2hda3PT0R0P5ydnTF79mzMnj0bBQUFGDhwIJYvX26Q2DeEt7c3EhISoNPpDIa2XLx4\nEQDg4+MjlQ0ZMgSffPIJtm7datAzHx4ejtjYWOh0OvTq1QvW1tbSNX5+fkhMTLxnDNu3b4efnx92\n7dplUL558+YaHwwEQUBERAQiIiKwcuVKbNq0CS+++CJ27tyJqVOnws/PD6dPn8bw4cPvqx2qy83N\nRWRkJDQaDQ4dOlRjhcfWovVsPUlE9/TjvkvIvVMKALC1UeDJsd0adN2NgjL845BhUh/Z2REbHw1s\n8qS+OjuFKZaE+uLvA31gZarvcdKJIr5NzsT/rqhwu9xwRZ5+fdzg46WPT62uwP5fU2vck4ioLdDp\ndDXGa9va2sLHx6fB47irJ8tjx45Fdna2Qc+4TqfDunXroFAoDIZY303k3333XQQEBMDJSb9ZYFhY\nGOLi4nDkyBGDYTiA/ulCUlISduzYUWc8JiYmEEXR4ElCamoqdu7caVDvzp07Na7t3bs3AP0KQAAw\nZcoUZGVyRxmWAAAgAElEQVRl1bp5qkqlQnFxcZ1xAPpVfB599FFkZmZiz5498PPzu2f9loxLSRC1\nAXn55Yj66aJ0POXJHrC0qH/ozM3Ccvzj0GXkq/RJvaWJHEsG+yLE1baeK5uGIAgY7tMRQZ2sseb3\n60i8rZ9ge7NUxNrLKtj75GOQh51Ud+yoALz/0QkAwJ7oKxj3aABMTWp/DE1E1FoVFhbC3d0dEydO\nRM+ePWFjY4MjR45g3759NdZzr2vITfXyF154AR999BFmzZqFM2fOwNfXF7t27cKvv/6Kd999F/b2\n9lJdHx8fuLm5ISUlBc8//7xUHh4ejtxc/Vys6ivRAMDixYvx/fffY+rUqdi/fz/69OmD/Px8/PLL\nL3jrrbcQHh6OcePGISoqCuPGjcPYsWNx69YtbNy4Ed26dUNCQtVcsbfeegsxMTEYM2YMvL29kZeX\nh02bNqFDhw547LHHAADTp0/Hjh07MG/ePMTExGDw4MEQRREpKSnYvn07duzYUePDR3XTpk3DyZMn\n8dxzzyEpKQlJSUnSOWtrazz++ON1XtvSMLEnagO+3Xke5eUVAABPdxuMiPCt95rMIhX+cegy7lT2\n1CtMZFgR0QWBnWqOaW9uTlZm+Ncwf+y8eBtfJt4CAGh0wL+OpGJ+iBce8dOvKzxkoCe+2paIvIIy\n5BWU4fDxdAwb4mPEyImIGp+VlRXmz5+P6Oho/PDDD9BoNOjcuTNWr15tsBpNXXMW/1hubm6OQ4cO\nYenSpdi6dSvy8/Ph7++Pjz/+GLNmzapxfXh4OLZt22aQHHft2hWdOnVCTk5OjcTe0tISsbGxePPN\nNxEVFYUtW7bA2dkZERERCAjQb5T4zDPP4Pbt29i4cSMOHDgAf39/rF27FpcvX8bZs2elez3xxBNI\nT0/Hli1bkJ2djY4dOyI0NBRvvPGGNN9TEARERUVh7dq12LJlC3bv3g0LCwv4+flh3rx5CA6+9y7l\nZ8+ehSAI+PTTT/Hpp58anPPx8WlVib0gPuhsimZS/RGTra1xehHbIi7H1jSM0a43bhbg5X/sl3pj\nlv09DH161b7PxF3KYhVe+/USskvVAABzuT6pD3ayvud1xnAtvxSLfzqNXJUIy8olPJ8JdsOkQBcI\ngoAdPyTj6x36uQU+XnZY/fZITsZvIP4eaBps16bR0OUu2+oGVdTy1Pf9ZowclmPsiVq5z785KyX1\nvYJc0LvnvXdizSpRYWm1pN5MLsPycL8WmdQDgK+dJeb5mcPdoipZ/yIxAx+evgmdKOKR4X4wM9M/\nfLx+Ix+JybeNFSoREZFRMbEnasVOn81EQqJ+p2dBEDBjas979lZnl6rxj0OXcftuUi+T4Y0wP/Ry\ntmmWeB+UtamAOX7meKhanD9evo1Vx67B3MIUw8N8qsp/qbmuMxERUXvAxJ6oldJqddjy7Tnp+OFw\nX3h71r2KTYVOxL+PpCKzWAUAMJUJ+EdYZ/R2adlJ/V0KuYDl4X4I86ya1BV7Iw9vxV7F8OG+APQf\naE6dzcTNjMI67kJERNR2MbEnaqV+jbuO9Fv68Xvm5iaYOuHeu/xtS85ESm4JAMBEELB0cGejrX7z\noMzkMrwa6oux/k5S2ZmsQmy8mIWHelcNQfpxH3vtiYio/WFiT9QKiaKIH/ZWJa9PPtYN9nZ1T+C5\nkFOMb5OU0vH0YLdmXaO+MckEAX/t44Fngqt2EryYWwKljy10laOQYg6noaCw3EgREhERGQcTe6JW\nKDH5Nm5l6oebKBQmGBPpX2fdUo0Wq49fh65ygm1QJ2s82a117qh3lyAImNzDFXP6eEplaSoNSro7\nQgSg1mixjxtWERFRO8PEnqgV+uXgVenroYN97rkZ1Uen06Vx9Vamciwa6A25rG0sBzk2wAlPV+u5\nVztaINPVCiKAvQeuQK3WGi84IiKiZsbEnqiVycktxYnTt6TjUQ/XvfX1kfQ8RF/LlY7n9vWEs5V5\nk8bX3CYHuuCJAP2YeysrMxQ7WuC2kwUKCssRd+yGkaMjoubWwrfnoTaipX6fMbEnamWif0uFTqf/\nhdKjWyd4edQ+ATanVI33T1YlthFeDhjm07FZYmxOgiBgVm8PPOzTEYIA2NqYI7ejBXI6KvDDL5da\n7C9fImp8ZmZmKC8v5889NSlRFFFeXg4zMzNjh1KDibEDIKKG01RoEf1b1djx0SO61FpPJ4pY+3sa\nitQVAIBOlmZ4McSz1rptgUwQsKC/N0o0WhzV5eFOXjluO1niXGYJEhKz6t20i4jaBplMBnNzc6hU\nKmOH0qoUFRUBAKytW+ZGhS2Rubk5ZLKW1z/OxJ6oFfk9/hbyC/SrvdjbWaB/H/da6/14KRtnsvST\nawUIWDTQBx3M2vaPu1wm4NVBvnhTo8WdvDIUFKqQ6WqFj/anYCMTe6J2QyaTQaGoe5Uwqun8+fMA\ngJCQECNHQn9Wy/uoQUR12nugatLsyKGdYWJS80dYWazC52erxuBP6O6Mnk7toxfG3ESG18P80Lva\nRl0xKjWOXLxtxKiIiIiaBxN7olYiLT0fFy5lAwDkchkih3eutd6XiRlQ63QAAD97S0wPcm22GFsC\nS1M53o3sBmcr/dhHnUzAsgMpyCvTGDkyIiKipsXEnqiVqN5bP6CvOxzsLGrUuXynBL+l3ZGO5/Tx\nhKm8/f2Y2ylM8UZ4F8gqJxkri1VYfugy1FqdkSMjIiJqOu3vLz5RK1RSqkbM0TTpuLYlLkVRxKcJ\nVUNwQj3sENipQ7PE1xIN7+WKgaL+V5woAifT7uD9kze4WgYREbVZTOyJWoHfDqdBpdKvcOPpbose\n3TrVqBOfWYhzt/UrG8gFAc/2rH1ibXshCAJmjQyAc1YpAKCwUIWD13Lx/cUsI0dGRETUNJjYE7Vw\noiga7DQ7eoQfBMFw51itTsRn1SbMPuLnCA8brgoxMMQdXQU57PJV0OpEFBWp8PnZDPx+K9/YoRER\nETU6JvZELVxi8m3cytQvXWmhMEV4qHeNOgev5yKtoAwAoDCR4ake7WvCbF3kchnGPuIPF2UJLEs1\nyC9UQRRFrDp2HdfyS40dHhERUaNiYk/UwlXvrR86xBuWFqYG58srdPgqMUM6ntDNBfZ/qNOePRzh\nCytzU3jcLAZKNSgt06CsQot34lJRVDm8iYiIqC1gYk/UguUXlOPE6WpDbIbXnDS7OyULuZVLOToo\nTDG+q1OzxdcaWFqYYsRQX5hoRXimF6GkQL8jpbJEhTW/X4eOk2mJiKiNYGJP1IL9fuoWdJVLNnYL\ncISXh63B+fxyDXZcqJoMOi3YDRam8maNsTUYE+kPQRCgUGlhc+UO1CotAOBERgG2JyuNHB0REVHj\nYGJP1IIdO3lT+npwf88a579NUqK0Qp+ketooMNK3Y7PF1po4OVphUD8PAIBNkQZuxVWbVX2ZmIkz\nykJjhUZERNRomNgTtVBFxSqcv3BbOh4YYrh8ZUZROfZeyZaOZ/Zyh1xmuFoOVRk3KkD6uiQ+E/42\n+g2+RIh479g1ZJeqjRUaERFRo2BiT9RCnTiVIQ3D6dqlIzo6WBqc/yZJiYrK8eFBnazR3822xj2o\nSkCXjugW4AgA0Gl1CCzUwEGhn2RcqKrAv4+kcmdaIiJq1ZjYE7VQ1Yfh3B1Gcld2qRqxaXek4xm9\n3GqsbU81jX2kqtc+7rdr+FsfT8gr2y0ltwSbE27WdSkREVGLx8SeqAUqLlHjbFLVpNiBf0jsf7x0\n26C3vrtjh2aNr7Xq38cNrs7WAICSUjXSEjIxs1fVEKefLmfj0PVcY4VHRET0pzCxJ2qB4hMyoK0c\nFuLn6wAnRyvpXKlGi1+u5kjHT3bj8pYNJZfLMGFsN+n4h72XEOllj8Ge9lLZ+ydvSJt9ERERtSZM\n7IlaoGMn6h6Gs+9qDko0+pVwPKwV6Mex9fclPNQbzp30H5SKS9TYfygVC/t7w8NaAQBQaXV47+g1\nqCo43p6IiFoXJvZELUxpmQYJiVXDcAZVWw2nQidi96WqlXKe6OoEGcfW3xcTExnGP9ZdOt69JwUy\nrYilgzvDTK7/lXi9oAwfn+F4eyIial2Y2BO1MKcSMqGpXJve18sOri7W0rnDN/KkZRltzU0w3Ifr\n1j+IYWHecOyoX2WosEiF6N9S4WNngdm9q56O7L2ajSPpecYKkYiI6L4xsSdqYY7HVxuG078q0RRF\nETtTqnryH/N3grkJf4QfhKmJHE8+VjXWftfPKVCrtRjl52gw3n79iTRklaiMESIREdF9Y1ZA1IKU\nl1fg1FmldDwwpCqxP3e7GFfySgEAZnIZxvh3avb42pLhYb6wt9NvUpVXUIYDMakQBAF/6+cFJ0sz\nAECxRotVx66jonI/ASIiopaMiT1RC3L6XCbU6goAgKe7LTzcbKRzURereusf9ukIW3OTZo+vLTEz\nk2N8tV77qJ8uQq3RooOZCV4N9ZXWt7+QU4yvz2cYK0wiIqIGa3Biv2HDBvj6+sLCwgIhISE4fPjw\nPesnJiYiIiIClpaW8PDwwNtvv21wPiYmBqGhoXB0dISlpSW6d++O1atXP9i7IGoj6tqU6np+GeIz\nCwAAAgQ80ZVLXDaGyKGdYWerXw3nTl4ZDsVdBwB0d+yAp4PdpHrfJWchQVlojBCJiIgarEGJ/bZt\n27Bw4UIsW7YMCQkJCA0NxejRo5Genl5r/cLCQowcORKurq6Ij4/HunXrsGrVKqxZs0aqY21tjYUL\nFyIuLg4XLlzAsmXLsHz5cnzwwQeN886IWhm1WotTZzOl49Bq4+urr4Qz0N0WHjaKZo2trTIzk+Px\n0V2l46gfL6KicpnLCd2d0dtZ/8REhIjVx68jv1xjlDiJiIgaokGJ/Zo1azBz5kzMmjULXbt2xfr1\n6+Hq6oqNGzfWWn/r1q0oLy/Hli1bEBgYiAkTJmDJkiUGiX2fPn0wadIkdO/eHd7e3pg2bRoiIyNx\n9OjRxnlnRK1MQqIS5eX6YThuLtbwdNcnlXllGoPdUJ/s5myU+NqqR4b7wbqDOQAgO7cEMUfSAAAy\nQcDfB/rAztwUAHCnXIP//p4GUeR4eyIiapnqTezVajVOnz6NyMhIg/J7JeHHjh1DWFgYzM3NDepn\nZGQgLS2t1mvOnDmDY8eOYeTIkfcTP1Gb8cdhOELlGO+fLmdDUzl5s2tHK3Svtgst/XkKhQkeHx0g\nHe/48YK066+9hSkWDfSWzsVnFmBvtV1/iYiIWpJ6Z9/l5ORAq9XC2dmwl9DJyQlKpbLWa5RKJby8\nvAzK7l6vVCrh7V31h9LDwwM5OTnQaDR4/fXXMWPGjDpjiY+Pry9cuk9s06Zxv+1aUaHDr7EpKFfp\nE0pri3zEx8dDrRPxTXI5yvTL2iPIsQKnTp1q7HBbjab6fnVy0ELUqVBapkXqtVJ8+OkB9O/tIJ3v\na6lBXLb+acrauIuA8hqcFG1n7QH+HmgabNemwXZtOmzbxuXv79/sr9kkf5mE+9gJ88iRIzh16hQ+\n/PBDrFu3Dhs2bGiKkIhatCvXS6Sk3sHeDK7O+jH05/K1UlLvYCagh23bSSZbEoW5HIP7V232FR2T\nhbJyrXQ82sUEzgr97zWNDvg2XQMth+QQEVELU2+PvaOjI+RyObKysgzKs7Ky4OrqWus1Li4uNXrz\n717v4uJiUH63975Hjx7IysrCqlWr8OKLL9Z635CQkPrCpQa6+6mcbdq4HrRdT56Lh5WVfifU0SO6\nol+/XgCA7w6kwNKqGAAwpac7BgS61HmPtqw5vl979tTi0rW9yMnV7xVw5YY5Zj71kHTe2b8UL++/\niApRRC6AywpXTK+2ck5rxN8DTYPt2jTYrk2Hbds0CgoKmv016+3+MzMzQ9++fbF//36D8ujoaISG\nhtZ6zaBBgxAXFweVSmVQ393d3WAYzh9ptVrodLqGxk7UJoiiiDOJVR+EQ3rrk8W0gjIk5eiTehNB\nwAjfjrVeT43DzEyOZ6f0ko73RF/BrcyqJS4721vi6Z7Vl8BU4mLlvw8REVFL0KDn+osWLcLnn3+O\nzZs348KFC1iwYAGUSiXmzJkDAFi6dClGjBgh1X/qqadgaWmJGTNmICkpCVFRUVi5ciUWLVok1Xn/\n/ffx888/4/Lly7h8+TI2b96M1atX4+mnn27kt0jUst3MKJJ6iS0tTBHgp0/go1OrVsLp724LewtT\no8TXnoT290D3AP2OvlqtDlu+OWdwfnxXZwR1stafF0X85/h1lGm0Ne5DRERkDA3aunLSpEnIzc3F\nO++8g8zMTAQHB2PPnj3w9PQEoJ8Qm5qaKtW3sbFBdHQ05s2bh5CQEDg4OOCVV17Byy+/LNXR6XRY\nsmQJrl+/DhMTE3Tp0gUrV67EX//610Z+i0QtW0K13vrgQCeYmMig1upwsNoSl4/4ORojtHZHEAQ8\nN/0hLH7jAAAR8QkZSEhU4qFg/RAouUzAywO88dIvF1BaoUVmsQqfJNzES/3qfhJJRETUXBq8J/3c\nuXMxd+7cWs999tlnNcqCgoIQExNT5/0WLFiABQsWNPTlidqshPNVif3dBPL4zXwUqvSrsHSyNJM2\nSqKm5+djj4fDfXAw9hoA4LOvE7DmnUjI5foHnC4dzDGnryfW/H4dAPDL1Rz0d7PFAHc7Y4VMREQE\noIlWxSGihlGrtUi6WLUu+t3E/pfUqrLIzo6Qyxq+0hT9eU9NDIaFQj/0Kf1WIfb9etXg/HAfBwz2\ntJeO151I4660RERkdEzsiYzowqVsqNX6nnl3Vxs4d7JCRlE5zmYVAdDvfjqyMyfNNjd7OwUmPt5d\nOv42KglFxVWLAQiCgPkhXnCoTP4LVBXYcCq92eMkIiKqjok9kRGdPldtGE6QfhO3fdUmzYa42qCT\npVmzx0XAmEh/ODt1AAAUl6ixbWeSwXkbcxMsGFA1tv5Ieh5ib9xp1hiJiIiqY2JPZEQJiVX7QzzU\n0wUVOhEHr1Ul9qM4adZozEzlmDG1avnLXw5exY2bhmsSh7ja4pHOVf9GG0+lI6+MQ3KIiMg4mNgT\nGUlObinSb+kTRRMTGXp07YTfb+Ujr3KsdkcLU4S42hozxHavfx83BAc6AQB0OhEff3EaOp3hjrPP\n9/aQnqoUqiqw4dQNiNyVloiIjICJPZGRnD1f1Vsf2LUTFAoTg2E4I305adbYBEHAc9Megqzy3yHp\nYjZ+2nfJoI6lqRwL+1cNyTl6Mx8xaXnNGicRERHAxJ7IaAzG1we7IKtEhdOVO50K4KTZlsLb0w4T\nxlZNpN26/XyNITkPudhgtF8n6XjT6XTc4ZAcIiJqZkzsiYxAq9UhMbmqx753sDP2p+ZChH4IR28X\na7h0MDdWePQHf3k8EJ199Mtbaiq0WLvpd2gqDHecfe4hdzhVDskpUlfgfyfTOCSHiIiaFRN7IiO4\nci0PxSVqAIC9nQXc3WxwIJU7zbZUJiYyLPjrAJiZygEA12/kY1tUskEdS1M5FlZbJef3jAIcSuMq\nOURE1HyY2BMZwRmDYTjOOKUsRE6ZPtG3MzfFADdOmm1pPN1tMH1SsHS88+eLuHApx6BOL2cbPOZf\nNSTnw1PpyC1VN1uMRETUvjGxJzKChMSqxL53sAsOVFvickTnjjCV80ezJXp0pD96Bur3GxBFEes/\nOoGyP+w4O6OnO1wrh1EVa7T4XzxXySEioubB7IGomRUXq3E59e4QDQF+AR1xMqNqMuYIX06abalk\nMgHzX+gHSwv9jrNZt4vx+TdnDepYmMrxt2qr5JzIKOAqOURE1CyY2BM1s7NJWVIPrn9nByTcKYWm\ncm10fwcreNoojBke1cOxoyWef6a3dBx9KBXxCRkGdXo6WWNMl2pDcs6kI7+cq+QQEVHTYmJP1Myq\nD8N5KNgZh65XTbAc7uNgjJDoPkWEemNQPw/peMPmeNzJLzOoM6OXu8HGVR+eTm/WGImIqP1hYk/U\njERRREK1jak8/DsiKacYACAXBIR72RsrNLoPgiBgzsy+sLe1AADkF5TjvXVHoVZXLYFpaSrHS/28\npOPYG3k4fjO/2WMlIqL2g4k9UTNKv1WI3DulAABLC1OkQSed6+NqAzuFqbFCo/tk3cEcf5vTH4Kg\n35X20tVcfLjllMFE2b6utnjYp2rOxAfxN1Csrmj2WImIqH1gYk/UjKoPw+nZwwkxN6omVQ735jCc\n1qZXD2fMmNpLOj4Udx0/779sUOf53h6wr/zAdqdcg80Jt5o1RiIiaj+Y2BM1ozOJVcNwHP07IqNY\nBQCwMpVjgLudscKiP+GxR/wxdIiPdPz5N2dxNqnq39nG3ARz+3pKx/tTc3BGWdicIRIRUTvBxJ6o\nmajVWlxIyZaOc61MpK8He9rD3IQ/jq2RIAiYM6Mv/Dvrh9zodCJW/+8YMpVFUp3BnvYY7Fk1f2L9\niTSUabQ17kVERPRnMJMgaiYXL+dAXZnMubhY41ROiXRuGFfDadXMzORYsiBUmkxbXKLGu+uOoLSs\naonLuX08YW2m/zB3u1SNLecyar0XERHRg2JiT9RMzl+o6q23C+iIospJlJ0szRDUqYOxwqJG4mBv\ngdcWhsLURA5AP1F63abfoavco8DewhQv9K5aIvOny9lIzi42SqxERNQ2MbEnaiaJyVXjrgvtzKWv\nh/s4QFa5sgq1bv5+HTHnub7S8ckzGfhqe6J0PNzHAX1dbQEAIkSsP5kGtVZX4z5EREQPgok9UTMo\nLdPgcqp+IyqtTECapmrJw2HVlkOk1m/YEB+MHRUgHe/6+SJ+qlwpRxAEzAvxhMXdXv3CcmxLVtZ6\nHyIiovvFxJ6oGSSnZEtDMhR+9tBVdtD7O1jB00ZhxMioKTwzuSdCHnKTjj/9KgGHj98AADhbmePZ\nnlXndiQrcf0Pu9YSERE9CCb2RM0gMfm29LXKyUr6ejgnzbZJcrkMf583EF273H0aI2LdhyekZTDH\n+HdCoKN+XkWFqB+So9WJddyNiIioYZjYEzWD85WJvdpUhgJTfXe9XBAQ7mV/r8uoFTM3N8H/LRoC\nDzcbAIBWq8N7644i9XoeZIKAl/p5wVSm/15IyS3Bj5dv3+t2RERE9WJiT9TECotUuHYjHwBQZK+A\nQqFf8rCPqw3sKnckpbbJuoM5Xn8lDA72+mUwy8o1eGd1HJS3i+Fla4EpPVylul+cy4CycsMyIiKi\nB8HEnqiJnb+g74kVAVS4doCsspd2uDeH4bQHnRyt8MbicFhZmgEA8gvK8daqWOQXlGNCN2f4VK59\nr9Lq8L+TNyCKHJJDREQPhok9URO7O76+XCGHzlLfW29pIscAdztjhkXNyMvDFktfHiytca/MKsY7\nq+OgUWvxt/7e0nKnZ7IKEX0t15ihEhFRK8bEnqiJ3U3sC2zNYVE59CbU0w7mJvzxa08Cu3bConkD\nIVQm8anX8/Du2iPwtVbg8QAnqd7mMzeRV23HWiIiooZiZkHUhHJyS5GhLIIIoNjWXBpfP5TDcNql\nAX3d8dcZVRtYnb9wG2s2HMfUQGe4WOk3LSvWaLHxdLqxQiQiolaMiT1REzp/Ud9bX2JlArmVGQQB\nsFeYoqeTtZEjI2OJHNYZ0/8SLB2fOH0Ln32RgPkhnlLZkfQ8HEnPM0Z4RETUijGxJ2pCd4fhFNqY\nw8JC31sf5mUPeeUEWmqfxj/WDY+P7iod/xp3DQmHrmGkb9UuxBtPpaNQVVHb5URERLViYk/URERR\nRGLybegEoNDGDBYchkOVBEHAM1N64uFwX6nsx18uwTGzGB0t9PMw8so1+PjMTWOFSERErRATe6Im\noswqRk5uKYo7mAImMijMTeDawRwBDpbGDo1aAEEQMGdmXwzo6y6VRUUlo5/MRDr+9XouTmYUGCM8\nIiJqhZjYEzWRc9WH4ShMAAGI8HKQVkUhkstleHnuQAQHVq2Kc2h7EvxMqpL7D+JvoFSjNUZ4RETU\nyjCxJ2oi5y/chlYGFFmbSstchnvbGzkqamnMzORYsmAwunS+O0RLxM19VyCr0AEAskvV+DThlvEC\nJCKiVqPBif2GDRvg6+sLCwsLhISE4PDhw/esn5iYiIiICFhaWsLDwwNvv/22wfmoqChERkbCyckJ\nNjY2GDhwIH788ccHexdELYxOpx9fX2RtBlEQYGFhgs52lvCu3GWUqDpLC1O8/koY3F1t9AUqLbSn\nlVCr9D31e69m49ztIiNGSERErUGDEvtt27Zh4cKFWLZsGRISEhAaGorRo0cjPb32tZYLCwsxcuRI\nuLq6Ij4+HuvWrcOqVauwZs0aqU5sbCxGjBiBPXv2ICEhAY8++ijGjx9f7wcGotYg/VYBCotUKLQx\nh1wmwMxUjgj21tM9WHcwx+uLw2Bvp//wZ55ThrLr+aio7LlffyIN5ZVfExER1aZBif2aNWswc+ZM\nzJo1C127dsX69evh6uqKjRs31lp/69atKC8vx5YtWxAYGIgJEyZgyZIlBon92rVr8eqrryIkJASd\nO3fGG2+8gb59+2LXrl2N886IjCgx+TYq5AJKrExhYWEKCEC4F1fDoXtzcrTCG4vDYGlhCgGAfVoB\nbmcWQasVkVmswleJGcYOkYiIWrB6E3u1Wo3Tp08jMjLSoDwyMhJHjx6t9Zpjx44hLCwM5ubmBvUz\nMjKQlpZW52sVFhbCwYHJD7V+icm3UWhjBlEALBQm6NGpA5yszIwdFrUC3p52eG3hYJiYyGBaIcL+\nRhGUWcUQdSJ2pdxGcnaxsUMkIqIWyqS+Cjk5OdBqtXB2djYod3JyglKprPUapVIJLy8vg7K71yuV\nSnh7e9e45oMPPkBGRgaefvrpOmOJj4+vL1y6T2zTxqfTiTh+MhW53g7Q6XQANPCsyGNbN4L21IaP\nDrfHt7vSYVJSCtFShnRdBTram+ONfQl42d8c5vLGW12pPbVrc2K7Ng22a9Nh2zYuf3//Zn/NJlkV\n536X8/v+++/x6quv4uuvv4anp2f9FxC1YBnKMhSJAlQdzCCXCzAzkaGnrdzYYVErE9TNFo+NdIUA\nwFmObQgAACAASURBVDEtD5rSCuQXqJGrErFXqTF2eERE1ALV22Pv6OgIuVyOrKwsg/KsrCy4urrW\neo2Li0uN3vy717u4uBiU79ixA88++yy+/PJLjBkz5p6xhISE1BcuNdDdT+Vs08YVHx+PK9dLUOFi\nC5lMhg5WZhji54qIgV2MHVqr1l6/X0NCAHuHRHz/4wWI2eXIMDWBWiPDmTJzTPTwx0MuNn/q/u21\nXZsa27VpsF2bDtu2aRQUNP8Gg/X22JuZmaFv377Yv3+/QXl0dDRCQ0NrvWbQoEGIi4uDSqUyqO/u\n7m4wDOe7777DM888gy1btuDJJ5980PdA1KKkphWjwFY/v8TCwhRDuRoO/QlPTQzC0CE+sC1Qw7pI\njdy8MpSUaLD2RBo3riIiIgMNGoqzaNEifP7559i8eTMuXLiABQsWQKlUYs6cOQCApUuXYsSIEVL9\np556CpaWlpgxYwaSkpIQFRWFlStXYtGiRVKdb7/9FtOmTcPKlSsxZMgQKJVKKJVK3Llzp5HfIlHz\n0Wh0uJSjhspcP/TGzsoMA93tjBwVtWaCIGDuc30R1K0TXDNLINfqkJVdgpt5Zfj4zE1jh0dERC1I\ngxL7SZMmYe3atXjnnXfQu3dvHD16FHv27JHGwyuVSqSmpkr1bWxsEB0djYyMDISEhOCll17CK6+8\ngpdfflmq8+GHH0Kn02HBggVwc3OT/ps4cWIjv0Wi5pN2qxQFNvp1yM1M5Rji7QALU46vpz/H1ESO\nV/8WCg9HK7goSyGKIjKzirHn0m2czGj+R71ERNQy1TvG/q65c+di7ty5tZ777LPPapQFBQUhJiam\nzvsdOnSooS9N1GpcuVaMEgd9Ym9hYcJNqajRWHcwxz8WDcFrK35FUaEahTZmUGYV47/Hr2PTmB6w\nMW/wr3MiImqjmmRVHKL2KiGzHBVm+h56xw7m6PMnJzcSVefuaoNXXhoE9+wymFTooFJrkZKejw9P\n1b4LOBERtS9M7IkaSVGxCtfEqmE3I/w7wVTOHzH6//buPDyq6vwD+Pfe2SeTfZssZIMQyAKEnQAK\nWlBQaa2KKyp1FxFBq8XlUeterS22gFqtYqsFa6lV4YcEgywSlCWBkAUC2bdJJstkmf3e8/tjJjcZ\nwxZImCzv53nGmXvuuZczx8nMO2fee07fGp8SjoduT0dEbQcAoMPswH9yq7G3stnLLSOEEOJtFHUQ\n0kdy8+vREehKw1Gr5LgqMdTLLSJD1fy5Cbh9VgL8Ta6Zx1pMVvw+8zjqO+xebhkhhBBvosCekD6y\n9WgNRPcIfbiPEmNDfLzcIjKULbk5DQvC/aBwiACAyvp2PJ95HILIvNwyQggh3kKBPSF9ZL+hVXo8\nNz6o1yswE9IbMhmPpx6ajkkOBo4BjAF7ihvw8WHKtyeEkOGKAntC+kBpTSvq3H9NHAfcOGmEdxtE\nhgWNWoFXHp6BSJMrBcfpFLFubyny6lrPcSQhhJChiAJ7QvrAFwcrwdwj9EEcw6hgSsMhl0ZUhB9e\nujENWrMDANBhceDx/x1Dm83p5ZYRQgi51CiwJ6QPfF/etWJyqo5ScMillTFlBO5LjoBMcOXXVzSZ\n8cyWAjBG+faEEDKcUGBPyEWqb7ehzOyejYQBc2LU3m0QGZbuXzwOM1VKaXtHcQM2Ha7yYosIIYRc\nahTYE3KRvjxSI81EorPYkRih8XKLyHAkk/F468HpiLK5ZskRRYY3dp7EyYZ2L7eMEELIpUKBPSEX\naduJeulxkgrgeUrFId7h56vCO7elQ20XAAAWh4CHNubA4hC83DJCCCGXAgX2hFyEcpMFZSYrAIBj\nDNMjlOc4gpD+lTI6FE9MjwPnzq+vbLfh8f8cpXx7QggZBiiwJ+QiZJU2wmJ1zT7i2+bA2Hidl1tE\nCHDHVUmYF9T1WtxZ1oSPfiz3YosIIYRcChTYE3KBGGPYWmiQRkLjeB5BATRiTwaGP9w9BTHuGS8Z\nY1jzQykOVbV4t1GEEEL6FQX2hFygQmMHqlosAACZwHDZyGAvt4iQLmq1HGuXTISPzZVfb7MLWPXV\nMdR32L3cMkIIIf2FAntCLlBWWRMsls40HDvSU/RebhEhnhLjgvDk9DjIna6ZcgwmK1Z9dQw29zYh\nhJChhQJ7Qi6AXRCxu7wJVvfqnv4mO9KSw7zcKkJ6WnzVaFztp5Uupj1c0Yy39pyii2kJIWQIosCe\nkAvwY7UJDa2u2XAUDhGpoTr4+aq83CpCeuI4Ds/fMwWjzK6UHFFk+HduNb7uNk0rIYSQoYECe0Iu\nwI7SRpgtDgBAQIsN41NotJ4MXL46FV6/YyICTDYAgNXqxB92nsTJdprfnhBChhIK7AnpJaPZjsN1\nrVJ+vb/JhnEp4V5uFSFnlzImDI9Oi4PaPT2rsdmC94qsqLFQvj0hhAwVFNgT0ktZZU2wOwTYHQJ8\nOhzwYRzGjg7xdrMIOaebF43FL9Qq6WLauhY73j9lQ127zcstI4QQ0hcosCekFxhjyCwxosPsSsPx\nN9mQlhwGlUru5ZYRcm4yGY/VD0zDmAYreJFBEBiqmux4ftdJmNwXghNCCBm8KLAnpBcKjB2oabfB\nbHaAFxn8Wu2YNCHC280i5LwFB2mx8vZ0jKhsA8cYzBYBRXWteHH3SVidlHNPCCGDGQX2hPRCZokR\noshgsTrgb7KBZ8BkCuzJIDNz2ggsTI9CaGkzwIAGoxn5hja8vq8UgkjTYBJCyGBFgT0h58niELCn\nshkWixOMueaujx0RgNAQH283jZBeu3dJOqKYgOAqE0SRod5oxoFqE/56sILmuCeEkEGKAntCztOe\nymZYnSI6zA6obAI0FiempEd6u1mEXBAfrRI3XRcNf2MHQowWWCwOmFpt2F5ixMdHayi4J4SQQYgC\ne0LO047SRoABZosd/iYbOIDy68mgFjfCB7OnhyC0wYKAFhsam82w2wV8UViHD3OrKbgnhJBBhqby\nIOQ8VLdZkd/QDpvNCdHJENBig5+vCokJQd5uGiEX5crZYWg2mYDyZggyDvWKDkRF+uG/xw2wCSIe\nmjQCPMd5u5mEEELOA43YE3IevittBAB0WBzQtdshFxgmTYgAz1PAQwY3uYzHow9MhUohQ1R1O1RG\nC5qaLQCArScbsOancrqglhBCBgkK7Ak5B0Fk+K60CQBgNrtmwwGAyRMov54MDTHR/lhy8zjwDIiq\nbger6FpZeUdpI97aXwonBfeEEDLgUWBPyDnkGFphtNjhdIpwWhzwbXNALucxPjXc200jpM8snDcK\nE9L04ABE1rQD5SaIgiuY313RjNd/KIFdEL3bSEIIIWdFgT0h57CjxJWG4xqtt4MDkDImDFqNwrsN\nI6QPcRyHR+6dAl+dChwA/5IWBDZbpf3Z1S14eW8JLA5axIoQQgYqCuwJOYtWmxP7q1sAAB1mBwJa\nOtNwaDYcMvQEBWrw0G8mAQA4AJafajBepZT2H6o14bffHYehw+alFhJCCDkbCuwJOYusskY4RAYm\nMnAtVqhtrtFKmuaSDFXTJ0fjysviAbiC+/JtJ3FdXLC0v7TFgpXbi5Df0O6lFhJCCDkTCuwJOQNB\nZPimuAEAYLY44dfkSksYEeUPfZjOm00jpF/95o4JCHe/xi0WByp3lGL55BjI3dNemmxOPJ11AttP\nGb3ZTEIIIT9DgT0hZ3Cw1oTadlfKgdPSbTacdBqtJ0ObRq3AYw9Ok6ZzLTzRAHOhEa9eMRr+Ktfy\nJ07GsOZAOd4/XEnTYRJCyABx3oH9unXrEB8fD41Gg8mTJ2Pv3r1nrZ+Xl4fLL78cWq0W0dHReOml\nlzz219XV4bbbbsPYsWMhl8uxdOnSC3sGhPSTr0+4RuvBAE1dB3h37ELTXJLhIGlUMG5clCxt/2tz\nPtQdDvxp/hgkBGil8v+dqMcLu0+izeb0RjMJIYR0c16B/aZNm/DYY4/h2WefRW5uLjIyMrBgwQJU\nVlaetn5rayvmzZuHiIgIHDx4EGvWrMGbb76Jt99+W6pjs9kQGhqK1atXY9q0aeBoZUMygJSbLMgx\ntAIA7HYBqlpXPrGvToXRI2m1WTI83PTLsRg90pVfLwgi/rz+RwTIZfjDlaORER0g1Ttc14pHvy1E\nAeXdE0KIV51XYP/2229j6dKluOeee5CUlIR33nkHERERWL9+/Wnrf/rpp7BardiwYQOSk5Nxww03\n4KmnnvII7GNjY7FmzRrceeedCAqiQIkMLNJoPQC9wKB0uObvnjheD5mMMtjI8CCT8XjswWlQudNv\nqmtb8f4nh6FRyLB6ZgJuSelKS6s32/FU1gl8eqyGUnMIIcRLzhmh2O12HD58GPPnz/conz9/Pvbt\n23faY7KzszF79myoVCqP+jU1NSgvL7/IJhPSv9psTmSVNUrbiso26TGl4ZDhRh+uw713pEvbO/eU\n4btdpeA5DkvSIvH0zAToFDIAgMgYPjtWi9VZJ2hKTEII8YJzBvZGoxGCICA83HOVzbCwMNTV1Z32\nmLq6uh71O7fPdAwhA8X2kkbY3CtsRmqUaD7VBMA1ejkhjVabJcPPFZfF4fKZsdL23z45jNJy1/oO\nM0cE4q8LkpEa6ivtzze2Y/m2Quwub7rkbSWEkOFM3h8n7a98+YMHD/bLeYcz6lNPAmP4tMgGs92V\nSqBtaoK5wwwAGBnng8KCo+d1HurX/kH92j/Op1+npHE4lCPA0GBDB4BnX96Kh+4eCbXKNVr/az+G\nIKsT2w1OiAwwA3h+Rz4mBcrwyygFNLLhdx0VvV77B/Vr/6G+7VuJiYmX/N8854h9SEgIZDIZDAaD\nR7nBYEBExOmn/dPr9T1G5juP1+v1F9pWQvpdYauIZndQ7yMHWgu7RhyTR/t5q1mEeJ1SyePW62Og\nVLg+NoxNdvx3azUYc/29yDgOV4Yr8PBIFYKUXUH8oWYBbx63Ic8keKXdhBAynJxzxF6pVGLSpEnY\nvn07brjhBqk8MzMTN91002mPmTFjBp566inYbDYpzz4zMxNRUVGIjY097THnY/LkyRd8LPHU+a2c\n+tTTf7JOQOvjyqm/MsIfe76rhY+PFnI5j7tunwNfneqsx1O/9g/q1/5xIf2q8xuBP63fDwAorXSi\nvjkA18zvGpWaDGCBQ8C6gxXY6U7FEQB80QDUqgPx0MQRCNQo+uw5DET0eu0f1K/9h/q2f5hMpkv+\nb57X9B6rVq3Cxx9/jA8//BCFhYVYsWIF6urq8OCDDwIAVq9ejV/84hdS/dtuuw1arRZ333038vPz\nsXnzZrzxxhtYtWqVx3lzc3ORm5sLk8mExsZG5ObmoqCgoA+fHiHnr6zFgqP1rqBexnFQVXdN3Tcl\nPfKcQT0hw8HsGTG46oqR0vbH/zqCEycbPepoFTI8MSMeT89MQJC6K4j/obIZD/1fATJLjNJIPyGE\nkL5zXjn2ixcvRmNjI15++WXU1tYiLS0NW7duxYgRIwC4LogtKSmR6vv5+SEzMxPLli3D5MmTERQU\nhCeeeAIrV670OO/EiRMBuHLyGWP4+uuvERcX53EuQi6Vr07US49nRPnj0OddXzLnzIzzQosIGZiW\n3j4BxSVNKClrhiCIeGttNv740rweX35njgjEuDBf/P1INbaXGAEAbXYn/vxTOXaVN+ORKTHQ0xdm\nQgjpM+c9IfdDDz2E0tJSWK1WHDhwALNmzZL2ffTRRz2C8dTUVOzatQsWiwXV1dV47rnnepxTFEWI\noghBEKTHFNQTb2i1OaW0AQBI4mVobrEAAPz91EgfR9eGENJJqZDhiUdmQOtOqTE2mvHHtfvhcPbM\no/dVybFiaixemZMIvU9XEJ9jaMVD/1eAjfm1sLtnoSKEEHJxaKUdQgB8W2KUgotRgVqU5tRK+y7L\niIFcTn8qhHSnD9Ph0funSttH8w1Y9+HBM6bYTND7Ye2Csbg+KRy8e+Y0uyDiH3k1WL6tELl1rZek\n3YQQMpRRtEKGPZtTxFfHu9Jw5sUG4cDhGmmb0nAIOb2pk6Kw+Fcp0vauH8rxj8/zzlhfLZfh3vRo\nvPWLJIwK1ErlVW1WPPN9Mf6wrxSNZnu/tpkQQoYyCuzJsLflZAOarA4AQLBGAa66HXaHK6UgLiYA\n8bEB3mweIQPazdcnY96cBGn7yy1F2LK9+KzHJAX74O15Y/DgxBHwca9aCwC7Kprw4NYC/O94PQSR\nLq4lhJDeosCeDGtmh4B/F3atuXBzcgR+yC6XtufOivNCqwgZPDiOw/13TcSU9Eip7MN/5uKHHyvP\nepyM53Dd6DC8tzAFc2ODpHKzU8D7OZV49NtC5LlnqSKEEHJ+KLAnw9rXJ+rRanMCAMK0SozzUaHo\nhGv2DpmMx+wZMd5sHiGDgkzGY9XD05E0KthdwrDmvR9xrLD+rMcBQKBGgSdmxOPVuaMR7auWystM\nFvwu6wTezKb0HEIIOV8U2JNhq83mxH+KulZUvjU1Anv2VUjb6eP0CPBXn+5QQsjPqFRyPL1qFqIi\nXCs0O50iXv/zDyivbDmv48eH++KvV4/F3eOjoO52sfr35U14YGsB/lNogJPScwgh5KwosCfD1pfH\n69HhzqWP9lVjbkwQvv+hTNpPaTiE9I6vToVnn5iNQH8NAMBsceD3b+5BRdX5rb6okPG4aawe7y5M\nwWUxgVK5xSng70eq8Mi2Apo9hxBCzoICezIstVgd+PJE12j97akRKDphhLHRDADQ+SgxaUKEt5pH\nyKAVHuqDZ5+YLc1x39xiwdMvZSE3r+4cR3YJ1SrxVEYCXp2biBg/jVRe2eqaPefVH0pg6LD1edsJ\nIWSwo8CeDEtfFBpgdbrmrY/z12BWTCB27i2T9s+eEQNlt9k6CCHnLz42AL97bCbUatfi5maLAy//\ncQ8yv+/dAoTjw/3wl6vH4t4J0dDKu/4ef6hsxkNbXYtb2Zy0uBUhhHSiwJ4MO41mO7acbJC2l6RF\nwmZzIvtAlVQ2Z2asN5pGyJCROjYMrzxzBYICXSPuosiw/u8H8Y/Pj0LsRa68nOdw/ZhwvHdNCq6I\nC5bKbe7FrZZtK8CP1S1nXBiLEEKGEwrsybCzqaBOWmU2McgH06L8sf9ANWzu2XGiI/0wKiHobKcg\nhJyH+NgAvPH8lUiI7cqX/+83RXh73X7Y7UKvzhWkUeDx6XF488okjOy2uFVtuw2/33MKL+w+hapW\na5+1nRBCBiMK7MmwUtduw7enjNL2krQIcByHzF1dKQJzZ8eBcy95Twi5OMFBWrz0zBxMntA1z/2+\nnyrx/Ovfo6nF0uvzJYfq8Kd5Y/DwpBj4KuVS+cFaE5b9XwE+zK2C2dG7Lw2EEDJUUGBPhpVNBXVw\nun+yTwnRYaLeD0fzDdLc9TzP4fIMSsMhpC9p1Ao8tSIDC+eNksqOn2zEo09tw5btxRCE3uXJy3gO\n1ySG4r1rUrBgZCg4uL6IOxnD5iID7t+Sj8wSI0RKzyGEDDMU2JNho6zFgu9KG6XtJeNcI4gb/5sv\nlV0xO17KCSaE9B2ZjMe9SybiN3ekA+5A3Gxx4MN/5uDJ53fg+MnGs5/gNPxVcjwyJQZrrhqDlBCd\nVN5sdeDPP5XjiR3Hcbyxo6+eAiGEDHgU2JNhQRAZ3jlQDsE9gpce7oe0MF/k5hk8Vpq98ZdjvdlM\nQoa8a+cn4sXfXS4tZAUApRUtWP3777Duw4Noa+/9NJYjA7V448rReHJGPEI0Sqn8eGMHVmUW4Y/7\nS9FAq9cSQoYBCuzJsPDf4wZp5E7Bc7g3PRqMMfxr8zGpzrw5CQgL8fFWEwkZNtKSw/DHl+fhthtT\noeyWJ79jVwmW/fb/sGV7McwWR6/OyXEcLo8NwrsLk3FLcgQUfNd1MlllTXhgSz7+mVcDq5Py7wkh\nQxcF9mTIq2y14tO8Wmn71pQIxAVocDCnFidLmgAACrkMN1w3xltNJGTYUSpkuHFRMta8dhWmpHdd\nWNveYceH/8zBPY9+jfUfHURJWXOvzqtRyLBkXCTeXZiCGVEBUrlNEPGv/Frc9w3l3xNChi75uasQ\nMngJIsOan8phF10X540K1OKGsXqIIsPGbqP1V105EsFB2jOdhhDST8JDfbB65SwcOFyDD/+Zg3qj\n65c1m82JzJ0lyNxZgsSEYMyfm4BZ00dApTq/jy29ToVnZ4/EEUMbPsytwqlm16rSTe78+2+KG3BP\nejTGhfn223MjhJBLjQJ7MqR9daIehcZ2AICc4/DYtFjIeQ7ZB6pQWtECAFAq5bj+GhqtJ8SbpkyM\nxLiUMHy3uxTfZp1CZXWrtK+4pBHFJY346LNcjE8NR/o4PdLT9Of1ZXx8uC/+NG8MvitrxD+O1qDJ\n6krxOdlsxuqsE5gU4Y+7xkV6zI1PCCGDFQX2ZMiqbrPik7waaXtxih7xAVoIgoiNm7tmwlk4bxQC\nA9TeaCIhpBuVSo6F8xKx4BejUHjCiO1ZJdh3oBJOp+sXN7PFgewDVdIq0bEjApA+To+J4/QYPSoY\nSoXstOeV8RzmJ4Rg9ohAfFFowObjBmmRukO1JhyqNeHymCAsSYtEhK/q0jxZQgjpBxTYkyFJZAxr\nfiyXPrzjAzRYPFYPANj3UxUqq00AALVajl8tTPJaOwkhPXEch+SkUCQnhWLp7eORtacM23eWwFDf\n7lGvvLIF5ZUt+HJLEeRyHqPigzB6ZDCSEl23oADPqWs78++vGhmCT4/VIKusScq131XRhB8qm3HV\nyBDcmhKBQI3ikj1fQgjpKxTYkyHpm+IG5LtTcGQch5XT4qCQ8RAEEZu6zVt/7fzR8KMROkIGLH8/\nNa6/Zgx+tTAJldWtOHy0DrlH65B/vMFjYSunU0RRsRFFxUZgm6ssNNgHSYnBSIgLxMi4QCTEBcBH\nq0SYjxIrp8Xh+qRw/COvBvurXWl5Tsaw5WQDdpQ2YuGoUPx6TDiCKMAnhAwiFNiTIae2zYaPj1RL\n2zeN1Uv5s7v3VaCmrg0AoNUosOjq0V5pIyGkdziOQ0y0P2Ki/fGrhUmwWB04VtCAnLw6HDlmQK2h\nrccxDY0daGjswN79FVKZPlyHhNhAjIwPRNyIACxLjcQNY8Kw4WgtjjW4zmETRPz3uAFbihswf2Qw\nbhyrR6hW2eP8hBAy0FBgT4YUs0PA6/tKYHOP5MX6a3BziisFx+kU8fn/CqS6ixYkQaejD2tCBiON\nWoEpEyMxZaJrqkxTqxUnTjah6KQRx4sbcbK0GXa7s8dxdYZ21Bnase+nSqnMV6fCiGg/TI7QIZ9n\naGYMCoUMdoj4prgB204acWV8MBYn66HX0S98hJCBiwJ7MmTYBRGv7C3BSfe0djKOw4qpsVDKXMs1\n/OPzo1KOrs5HiWuvSvRaWwkhfcvfT+0R6DudIsoqWnCytAmnSptRUtaMiupWj/SdTm3tNhQUNQBF\nDWAAeJ0ChhANbD4KKBQ8FHIZ/tVkxuajNZiq98OvxoRhWnwwZDJaCoYQMrBQYE+GBJExvL2/DLmG\nrinylk2OQVKwayXZH36sxNfbTkj7Fv8qGVrKnSVkyJLLeYxKCMKohCCpzG4XUF7ZgpLyFpSUN6Oi\n0oTyKhOs1q6RfQ6Ab7sDunYHOnwUMIZo0K7t+qj8qqEDX+XVQmNxIs7JkKRVISxIi5BgLUKCNAgO\n0qK21gx/XwVEkYHvtgIuIYT0NwrsyaDHGMP7h6uwp7Jrhco701wzXwBAZXUr/vrBAWnflPRILJxH\no/WEDDdKpQyJI4ORODJYKhNFhnpjByqrWlFe1YLyShNqattQY2gH1+GAT4cDZq0cxhANOny6BgMs\nGjkKARQ7nQg82YDAA1bIBdcMOx0drl8N139SjeBADcJDdQgP94E+TNd1C/eBRk2DC4SQvkWBPRn0\n/l1owNfF9dL2dYlhWJzsyqs3Wxz4wzv7YLO5RuTCw3RYfv9UGkUjhAAAeJ6Tgu3ONB7ANWDQbLKi\ntq4dtYY21NS1o8jQhlyzDZUcg0N0BfFOOY+GUA2MIWro2h3wN9nBm83gmCsdyNDQAUNDB1DQ898O\nCtQgPiYA8bGBiIvxR3xMAMLDdPT+RAi5YBTYk0Fte4kRG452zYAza0Qg7kuPBsdxYIxh7QcHUV3r\nSs9RKuV46tEM6HzogllCyNlxHIegAA2CAjRIGRPqsa/F6sCWE/X4qqgeDR12OJ0inIIIp5+IpjAR\nzmgttE0WcK0CNBYnzhSmNzVb0NRswaEjtVKZWi1HXEwAxiSGIDkpBGMSQ+g9ixBy3iiwJ4PWj9Ut\n+OuBrmnsxof74vHpcZC5R7u+3nYC2Qe6Zr54cOlExMUEXPJ2EkKGlgC1ArePi8LNqZHYV9nssW4G\nAJg7eCBMA16jgY9SjtEaJfQCA9dihaHe7B7Fb5dW1O3OanWi6IQRRSeM+HILAHCIj/FHclIoxiaF\nIGVMKPz9aKVsQsjpUWBPBh3GGL4pbsCHuVUQ3KtGjgzU4tlZI6UZcPKLGvDJpqPSMVdfOQpzZsZ5\no7mEkCFKznO4LDYIl8UGoabNiqyyJmSVNaHUnWPP8RyanQJ+bLMAAIL8FJg6NgbzIv2RFqpDY4MZ\nZRUtKK1oQWl5C0rLm9HaZvvZv8Jc+ytasCWzGACHhLgATEjVY0JaOJISg6GQyy7tEyeEDFgU2JNB\npcMu4J0D5djb7UJZvY8KL142ClqF68OtqcWCt9fuh+jOgU1MCMbdt433SnsJIcNDpK8ad6RF4rbU\nCPx710841CygVJDB7BSkOk1WB7adMmLbKSNUMh7jw30xLswXGUmjcHuABjwHNJusKD7VhILjDSg4\nbkRJWTOYewDDhaGkzDV95+ZvCqFWy5E6NgwTUsORnqZHhN730j95QsiAQYE9GTRONpnx+r4S1LZ3\njWiNCtRi9cwEBLqnrjxZ0oQ3/5KNZpNrhMxXp8Jvl8+AUkEjWoSQ/sdzHEbqZBipkyF1wjjk1rVi\nf3ULfqoxwWTrmlbTJoj4qcaEn2pMAACdQobUMF+khemQGh+IJemRkPMczBYHjhc3ouB4A/KLjDPw\nKgAAFlNJREFUGnDiVKM0aAG4UncO5tTgYE4NANcEAZ1BfmpyGE3rS8gwQ4E9GfAYY9hy0ogPciql\nmSgAYOGoUNw7IRoqOQ/GGL7NOoW/f5or5a1yHIdVy6YjJFjrraYTQoYxtZzH9OgATI8OgCAyHG/s\nwI81JuyvakFVm9WjbrtDwP7qFuyvbgEAKGU8RgZqMSpQi8QgLWbNH4VbbkiFxeLAscJ65OYZkJtX\nh3pjh8d5DPXt+DarHd9mnYJMxiNpVDDGp4YjZUwoRiUE0SAHIUMcBfZkQGuxOvDe4UrsruhKvdHK\nZVg+JQaXxboWnrFYHVj/90PYu7/rQlqtRoFH75+K8Snhl7zNhBDyczKeQ3KoDsmhOiwdH4XqNiuO\nGtqQV9+OvPo2NFkdHvXtgohCYzsKu12Uq5bziA/QIsZPjZiMaDxw9Sio7CJKjzfgSH498grqpal9\nAUAQRHdKTwMAQKmQISkxGCljwpA6lgJ9QoYiCuzJgFTfYcfmIgO+LTHC3m0J+IQALX43Mx5Rvq5Z\nISqqTHjrr9moqulacTY+JgC/XZ4BfbjukrebEELOR5SvGlG+aiwYFQrGGKrbbMirb8PR+jYUGjvQ\nYLb3OMbq7BnsA4CvUo6YtFBMmhYFrsMBU20bak81ob68GbJuE+/YHQLyClxfAADX6ryxIwIwMi4Q\nCXGBGBUfiBHRfnQxLiGDGAX2ZECparXii8I67CxrgtPjgjFgwchQ3JfuSr2xOwTs3FOGjz47Aru9\na4Rq3pwE3HNHOpRK+mAihAwOHMch2k+NaD9XoA+4fq082WxGcaMZxU0dKG4y9xjV79RmdyLf2I78\nzgIlgLFBEEYHgrcLgNkBS5MFdpMNCocAhUOEwiFC7hRxsrQJp0qbpHN1BvvRkb6I1PsiIlyHCPc9\n5esTMvCdV2C/bt06vPnmm6irq0NKSgr+/Oc/Y9asWWesn5eXh0ceeQQHDhxAUFAQHnjgATz33HMe\ndXbt2oVVq1ahoKAAkZGRePLJJ/HAAw9c3LMhg5JDEFHY2IEtxQ34obIFDJ4B/ahALW5LjcC0qADU\n1bdj+84SfLerFG3dLqJVKuV44O6JmDsr7hK3nhBC+l6AWoHJEf6YHOEvlTVZHCg3Wdw3KypMFlSY\nrB4z73Qnk3GARg5o5NAGa6B0irBanTBZnbBYHXA4RHCMQe5kkDtF6VbbZoGsoAOyvFrIBAaZIEIm\nMATpVIgM8UFwgBqBAWoE+msQFKhxPQ5Qw89XDZ1OQSP+veQUGaxOAVanCKtThE0Q3Y+Fn2137bdL\nNyZtOwQGp8ggMAah271T9PxM5TiAAweOA3gOkPM8TE02yDngG/NJKHgOCp6HRsFDI5dBq5BBq+Ch\nlcugUcjgo5BBp5TBTyWHn0oOjZwHx9FqyQPFOQP7TZs24bHHHsP69esxa9YsrF27FgsWLEBBQQFG\njBjRo35rayvmzZuHOXPm4ODBgygsLMTSpUvh4+ODVatWAQBKS0uxcOFC3Hvvvfjss8+wZ88ePPzw\nwwgNDcWvf/3rvn+WZMCpa7chp64Vh2pbccTQdtoPptRQXyxODsf4UB1yjhrw0r+OIueoAfhZ4B+p\n98Vvl89A7AhafIoQMnQFaRQI0iiQrveTyhhjaDA7UNVqRW27DbXtNtS0WVHdZoOhw+Yx4YBczkOn\nU0Knc61kKwgMdrsTVpsAu12AxeY87aJZnkTwrR3gTO2QCQwcA3iRgRcZOMbAi4BCxkGlcAWBaqUc\nKqUM6m43lUrueqySQ62SQ6Vy1VOr5ZDJuDOu1Fta7/rFoqyw7rz6q/uZOuNO7mfbZ32mDGAMYJ3D\nTQwQGYPIABEMoggIjEllTpHBKYpwiMwVZDMGhzvgtgudATmTgnKbIMLmFHv8Ou0N5g7X//ca9yxN\nvSHnOOiUcviqZAhQK+CvkiNALYe/SuG+lyNA7XocqFbQF4F+ds7A/u2338bSpUtxzz33AADeeecd\nbNu2DevXr8err77ao/6nn34Kq9WKDRs2QKVSITk5GUVFRXj77belwP7dd99FdHQ01qxZAwBISkrC\njz/+iLfeeosC+yGGMQajxYHqViuq2myobLUgt66tx4wQ3Y0P0WGKTg3WYEbWF/lYV9yI5hZLj3oh\nwVrMn5uAhfMS6SdiQsiwxHEcwnyUCPNR9tgniAxGix31HXY0mO1ocN/Xmx1o6LCj0WJHh4yDptv7\npygw2OwCHA4BDqcIh0OA3SHC6RTQGX+KPAeAg3DOgXkGOByuW8e56rrwPAee5yDrvJfx7nsOTocD\nHA/41JVJZTzPu1YbpzjRa5yMocXmQIvNgcrWM3+2d1LKeAR0C/Y7vwz4q+Twd38R8Fcp4K+Ww1cp\ng0pGXwR646yBvd1ux+HDh/Hkk096lM+fPx/79u077THZ2dmYPXs2VCqVR/3nnnsO5eXliI2NRXZ2\nNubPn9/jnBs2bIAgCJDJ6Ge8gYox1896naMO7Q4BrTanx81kc6LJ4kC1e9TI0m00njFAFEQ4BQZB\nEF0/FwoitAwIsotQVrehqPYUis7YAg4Tx+lx1RUjMXG8HjL3SrOEEEI8yXgO4T4qhPuozljH6hTR\nYnWgyeJAs/u+xepEq92JdrsTbTYBbXYn2mxONHXY0Wp1QhBEOAURgsDgdIrubQbR/Z5+MUSRQRQZ\nnKfd5xpVbm3r+Qtv9y8DPM+7g/6f3TgOPO/6MtRZxnGdKSmue4679F8SZBwHtZyHSsZDLeehlst+\nts1DJeehlrn2KWUcVHIeSlnnjYNKxkPB85DLOMg5V1/IOA5yngPvfl6MuX556P4DgeD+TM/NOwaB\nASNHj4RDdH2+W50izA4BFocIs1NwPXaK6LC7XhOdn/k24Vy/8niyCyLqzXbUn+YC8dNR8jx0Shl8\nVXL4KuWYEumHG8fqe/VvDidnDeyNRiMEQUB4uOeUgWFhYairO/1PYXV1dYiJifEo6zy+rq4OsbGx\nMBgMPc4ZHh4Op9MJo9HYY1+na/68++zPZpjpzdtnZ93OFQzNFtcIuGbPLjD3m5jIXPUYABEA41z3\nIgCh243B4z/dH0rnZwCYyNw/ZTL3ras9HGPw6XDCp8OBgHY7lHYRDgCnvzTMtdDUlZfHY/7cBOjD\naLYbQgjpC2o5D71OBb3uzMF/dyJzBX0Whyvws7jzwC1OwTXo4xTRbnWgzexAu9l1b7E5YbE5YbU5\nYbELsNkEWDtTgBwCrDYnbHZXOtDPr7Hqzm53fUIolefzC21XdN79s+x8uL4k8OB5SPc81+2LAwfw\ncH1J4JmrnAMg4wGeuevCFbDzAHgAcg6QwRV08QDkcNdxb58uAalzkNrmvrVK5X3/zcNgMAAAynLO\n9Cncxcd96wytBTDYu7XTzgFWMGnbxgFWADYw2DhXHHExzA0dFNifRZ/PitOfP5d8tnR8v52bDBYC\nTKbe5wBeKomJiQAwoNs4GFG/9g/q1/4xHPqVB6AFoOXhmoVHCbiC6c5wVe21tpELkebtBvTKUP7b\nulhnzWMICQmBTCaTvsl1MhgMiIiIOO0xer2+x2h+5/F6vf6sdeRyOUJCQnr3DAghhBBCCCFnD+yV\nSiUmTZqE7du3e5RnZmYiIyPjtMfMmDEDe/bsgc1m86gfFRWF2NhYqU5mZmaPc06ZMoXy6wkhhBBC\nCLkAHGNnn2fp888/x5IlS7Bu3TpkZGTg3XffxUcffYT8/HyMGDECq1evxoEDB7Bjxw4Arukuk5KS\nMGfOHDz77LM4fvw4li5dihdeeAErV64EAJSVlSE1NRX33Xcf7r//fvzwww9YtmwZNm7ciOuvv77/\nnzUhhBBCCCFDzDlz7BcvXozGxka8/PLLqK2tRVpaGrZu3SrNYV9XV4eSkhKpvp+fHzIzM7Fs2TJM\nnjwZQUFBeOKJJ6SgHgDi4uKwdetWrFy5EuvXr0dUVBT+8pe/UFBPCCGEEELIBTrniD0hhBBCCCFk\n4BvQk4CvW7cO8fHx0Gg0mDx5Mvbu3evtJnnN7t27sWjRIkRHR4PneWzYsKFHnRdeeAFRUVHQarWY\nO3cuCgoKPPbbbDYsX74coaGh0Ol0+OUvf4nq6mqPOs3NzViyZAkCAgIQEBCAO++8s8fV5xUVFbju\nuuug0+kQGhqKFStWwOE49xRZA9Frr72GKVOmwN/fH2FhYVi0aBHy8/N71KO+7Z21a9di/Pjx8Pf3\nh7+/PzIyMrB161aPOtSnF++1114Dz/NYvny5Rzn1be+88MIL4Hne4xYZGdmjDvVp79XW1uKuu+5C\nWFgYNBoNUlJSsHu359TV1Le9ExcX1+P1yvM8rr32WgCuKaapT3vP6XTi6aefRkJCAjQaDRISEvDc\nc89BEDwn6BzwfcsGqI0bNzKFQsE++OADVlRUxJYvX850Oh2rqKjwdtO8YuvWreyZZ55hX3zxBdNq\ntWzDhg0e+19//XXm6+vLNm/ezI4dO8YWL17MIiMjWVtbm1TnwQcfZJGRkWzHjh3s8OHDbM6cOWzC\nhAlMEASpztVXX81SU1PZ/v37WXZ2NktJSWHXXXedtN/pdLLU1FQ2d+5clpOTwzIzM1lkZCRbvnx5\n/3dCP7jqqqvYxx9/zPLz81leXh67/vrrmV6vZ01NTVId6tve+9///se2bdvGTp06xYqLi9kzzzzD\nFAoFy83NZYxRn/aF7OxsFh8fz8aPH+/xfKhve+/5559nY8eOZQaDQboZjUZpP/XphWlubmbx8fHs\nrrvuYgcOHGBlZWUsKyuLFRYWSnWob3vPaDR6vFZzcnIYz/Psk08+YYxRn16oF198kQUFBbFvvvmG\nlZeXs6+++ooFBQWxl156SaozGPp2wAb2U6dOZffff79HWWJiIlu9erWXWjRw6HQ6j8BeFEWm1+vZ\nq6++KpVZLBbm6+vL3nvvPcYYYy0tLUypVLLPPvtMqlNZWcl4nmfffvstY4yxgoICxnEc27dvn1Rn\n7969jOM4duLECcaY6wsGz/OsqqpKqvPPf/6TqdVqjxf2YNXe3s5kMhn75ptvGGPUt30pKCiIvf/+\n+9SnfaClpYWNHDmSff/992zOnDnSmz317YV5/vnnWWpq6mn3UZ9euNWrV7NZs2adcT/1bd94+eWX\nWWBgILNardSnF+Haa69ld999t0fZnXfeya699lrG2OB5vQ7IVBy73Y7Dhw9j/vz5HuXz58/Hvn37\nvNSqgau0tBQGg8Gjv9RqNS677DKpvw4dOgSHw+FRJzo6GmPHjkV2djYAIDs7GzqdDjNmzJDqZGRk\nwMfHRzpPdnY2kpOTERUVJdWZP38+bDYbDh061K/P81JobW2FKIoIDAwEQH3bFwRBwMaNG2G1WnHZ\nZZdRn/aB+++/HzfddBMuv/xyabVngF6vF6OkpARRUVFISEjArbfeitLSUgDUpxfjyy+/xNSpU3Hz\nzTcjPDwc6enpWLt2rbSf+vbiMcbw4Ycf4o477oBKpaI+vQgLFixAVlYWjh8/DgAoKCjAzp07cc01\n1wAYPK/XPl95ti8YjUYIgoDw8HCP8rCwsB4LWxFIfXK6/qqpqZHqyGQyBAcHe9QJDw+Xjq+rq0No\naKjHfo7jPPq9rq6ux7/TuZDZUPh/s2LFCqSnp0t/cNS3Fy4vLw8zZsyAzWaDRqPB559/jqSkJOmN\ni/r0wvztb39DSUkJPvvsMwCeq33T6/XCTJ8+HRs2bMCYMWNgMBjw8ssvIyMjA/n5+dSnF6GkpATr\n1q3DqlWr8PTTTyMnJ0e6HmTZsmXUt30gMzMTZWVluO+++wDQe8DFePjhh1FVVYWxY8dCLpfD6XTi\n2WefxYMPPghg8PTtgAzsSd/p/qF/OuwCJkW6kGMGg1WrVmHfvn3Yu3fvOfsNoL49lzFjxuDo0aMw\nmUz497//jVtuuQU7d+486zHUp2d3/PhxPPPMM9i7d6+0mB9zpVSe81jq2zO7+uqrpcepqamYMWMG\n4uPjsWHDBkybNu2Mx1Gfnp0oipg6dSpeeeUVAMD48eNRXFyMtWvXYtmyZWc9lvr2/Pztb3/D1KlT\nkZaWds661Kdn98477+Cjjz7Cxo0bkZKSgpycHKxYsQJxcXH4zW9+c9ZjB1LfDshUnM5vJQaDwaPc\nYDAgIiLCS60auPR6PQCctr869+n1egiCgMbGxrPWaWho8NjPGEN9fb1HnZ//O52/sHTWGYxWrlyJ\nTZs2ISsrC3FxcVI59e2FUygUSEhIQHp6Ol599VVMnz4da9eulf6GqU97Lzs7G0ajESkpKVAoFFAo\nFNi9ezfWrVsHpVKJkJAQANS3F0ur1SIlJQUnT56k1+tFiIyMRHJyskfZmDFjUFFRAYDeXy9WfX09\nvvrqK2m0HqA+vRivvPIKnn76aSxevBgpKSm44447sGrVKrz22msABk/fDsjAXqlUYtKkSdi+fbtH\neWZmJjIyMrzUqoErPj4eer3eo7+sViv27t0r9dekSZOgUCg86lRVVaGoqEiqM2PGDLS3t0t5YIAr\nkOjo6JDqZGRkoLCw0GPqpszMTKhUKkyaNKlfn2d/WbFihRTUjx492mMf9W3fEQQBoihSn16E66+/\nHseOHcORI0dw5MgR5ObmYvLkybj11luRm5uLxMRE6ts+YLVaUVhYiIiICHq9XoSZM2eiqKjIo+zE\niRPS4An17cX5+OOPoVarceutt0pl1KcXjjEGnvcMi3mel0bOB03fnvXSWi/atGkTUyqV7IMPPmAF\nBQXs0UcfZb6+vsN2usv29naWk5PDcnJymFarZb///e9ZTk6O1B9vvPEG8/f3Z5s3b2Z5eXns5ptv\nZlFRUay9vV06x0MPPcSio6M9pmBKT09noihKdRYsWMDS0tJYdnY227dvH0tNTWWLFi2S9guCwNLS\n0tgVV1whTcEUFRXFHn300UvXGX3o4YcfZn5+fiwrK4vV1tZKt+79Rn3be0899RTbs2cPKy0tZUeP\nHmW/+93vGM/zbPv27Ywx6tO+dPnll7NHHnlE2qa+7b3HH3+c7dq1i5WUlLD9+/eza665hvn7+9P7\n60U6cOAAUygU7JVXXmHFxcXs888/Z/7+/mzdunVSHerbCyOKIktMTOwxeyBj1KcX6r777mPR0dFs\ny5YtrLS0lG3evJmFhoayJ554QqozGPp2wAb2jDG2bt06FhcXx1QqFZs8eTLbs2ePt5vkNTt37mQc\nxzGO4xjP89LjpUuXSnVeeOEFFhERwdRqNZszZw7Lz8/3OIfNZmPLly9nwcHBTKvVskWLFnlMpcSY\na97hO+64g/n5+TE/Pz+2ZMkSZjKZPOpUVFSwa6+9lmm1WhYcHMxWrFjB7HZ7/z35fvTz/uy8vfji\nix71qG975+6772axsbFMpVKxsLAwNm/ePCmo70R92je6T3fZifq2d2655RYWGRnJlEoli4qKYjfe\neKPHXOuMUZ9eqC1btrDx48cztVrNkpKS2F/+8pcedahvey8rK4vxPM8OHDhw2v3Up73X3t7OHn/8\ncRYXF8c0Gg1LSEhgzzzzDLPZbB71BnrfcowNoSsfCCGEEEIIGaYGZI49IYQQQgghpHcosCeEEEII\nIWQIoMCeEEIIIYSQIYACe0IIIYQQQoYACuwJIYQQQggZAiiwJ4QQQgghZAigwJ4QQgghhJAhgAJ7\nQgghhBBChgAK7AkhhBBCCBkC/h9y/thwpWL2tAAAAABJRU5ErkJggg==\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "from price import *\n", - "import matplotlib.pyplot as plt\n", - "player1, player2 = MakePlayers(path='../code')\n", - "MakePrice1(player1, player2)\n", - "plt.legend();" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This shows the distribution of prices for these\n", - "showcases. The most common value for both showcases is around \\$28,000,\n", - "but the first showcase has a second mode near \\$50,000, and the second\n", - "showcase is occasionally worth more than \\$70,000.\n", - "\n", - "These distributions are based on actual data, but they have been\n", - "smoothed by Gaussian kernel density estimation (KDE). Before we go on, I\n", - "want to take a detour to talk about probability density functions and\n", - "KDE." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Probability density functions\n", - "\n", - "So far we have been working with probability mass functions, or PMFs. A\n", - "PMF is a map from each possible value to its probability. In my\n", - "implementation, a Pmf object provides a method named Prob\n", - "that takes a value and returns a probability, also known as a\n", - "**probability mass**.\n", - "\n", - "A **probability density function**, or PDF, is the\n", - "continuous version of a PMF, where the possible values make up a\n", - "continuous range rather than a discrete set.\n", - "\n", - "In mathematical notation, PDFs are usually written as functions; for\n", - "example, here is the PDF of a Gaussian distribution with mean 0 and\n", - "standard deviation 1: \n", - "\n", - "$$f(x) = \\frac{1}{\\sqrt{2 \\pi}} \\exp(-x^2/2)$$\n", - "\n", - "For a given value of $x$, this function computes a probability density. A\n", - "density is similar to a probability mass in the sense that a higher\n", - "density indicates that a value is more likely.\n", - "\n", - "But a density is not a probability. A density can be 0 or any positive\n", - "value; it is not bounded, like a probability, between 0 and 1.\n", - "\n", - "If you integrate a density over a continuous range, the result is a\n", - "probability. But for the applications in this book we seldom have to do\n", - "that.\n", - "\n", - "Instead we primarily use probability densities as part of a likelihood\n", - "function. We will see an example soon." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Representing PDFs\n", - "\n", - "To represent PDFs in Python, `thinkbayes.py` provides a class\n", - "named `Pdf`. `Pdf` is an **abstract type**, which means that it defines the interface a Pdf is\n", - "supposed to have, but does not provide a complete implementation. The\n", - "`Pdf` interface includes two methods, `Density`\n", - "and `MakePmf`:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class Pdf(object):\n", - "\n", - " def Density(self, x):\n", - " raise UnimplementedMethodException()\n", - "\n", - " def MakePmf(self, xs):\n", - " pmf = Pmf()\n", - " for x in xs:\n", - " pmf.Set(x, self.Density(x))\n", - " pmf.Normalize()\n", - " return pmf" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`Density` takes a value, `x`, and returns the\n", - "corresponding density. `MakePmf` makes a discrete\n", - "approximation to the PDF.\n", - "\n", - "`Pdf` provides an implementation of `MakePmf`, but\n", - "not `Density`, which has to be provided by a child class.\n", - "\n", - "A **concrete type** is a child class that extends an\n", - "abstract type and provides an implementation of the missing methods. For\n", - "example, `GaussianPdf` extends `Pdf` and provides\n", - "`Density`:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class GaussianPdf(Pdf):\n", - "\n", - " def __init__(self, mu, sigma):\n", - " self.mu = mu\n", - " self.sigma = sigma\n", - "\n", - " def Density(self, x):\n", - " return scipy.stats.norm.pdf(x, self.mu, self.sigma)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`__init__` takes `mu` and `sigma`, which are the\n", - "mean and standard deviation of the distribution, and stores them as\n", - "attributes.\n", - "\n", - "`Density` uses a function from `scipy.stats` to\n", - "evaluate the Gaussian PDF. The function is called `norm.pdf`\n", - "because the Gaussian distribution is also called the “normal”\n", - "distribution.\n", - "\n", - "The Gaussian PDF is defined by a simple mathematical function, so it is\n", - "easy to evaluate. And it is useful because many quantities in the real\n", - "world have distributions that are approximately Gaussian.\n", - "\n", - "But with real data, there is no guarantee that the distribution is\n", - "Gaussian or any other simple mathematical function. In that case we can\n", - "use a sample to estimate the PDF of the whole population.\n", - "\n", - "For example, in *The Price Is Right* data, we have 313\n", - "prices for the first showcase. We can think of these values as a sample\n", - "from the population of all possible showcase prices.\n", - "\n", - "This sample includes the following values (in order):\n", - "\n", - "$$28800, 28868, 28941, 28957, 28958$$ \n", - "\n", - "In the sample, no values appear\n", - "between 28801 and 28867, but there is no reason to think that these\n", - "values are impossible. Based on our background information, we expect\n", - "all values in this range to be equally likely. In other words, we expect\n", - "the PDF to be fairly smooth.\n", - "\n", - "Kernel density estimation (KDE) is an algorithm that takes a sample and\n", - "finds an appropriately smooth PDF that fits the data. You can read\n", - "details at .\n", - "\n", - "`scipy` provides an implementation of KDE and\n", - "`thinkbayes` provides a class called\n", - "`EstimatedPdf` that uses it:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class EstimatedPdf(Pdf):\n", - "\n", - " def __init__(self, sample):\n", - " self.kde = scipy.stats.gaussian_kde(sample)\n", - "\n", - " def Density(self, x):\n", - " return self.kde.evaluate(x)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`__init__` takes a sample and computes a kernel density estimate. The\n", - "result is a `gaussian_kde` object that provides an `evaluate`\n", - "method.\n", - "\n", - "`Density` takes a value, calls `gaussian_kde.evaluate`, and\n", - "returns the resulting density.\n", - "\n", - "Finally, here’s an outline of the code I used to generate\n", - "Figure [fig.price1]:" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Warning: Brewer ran out of colors.\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvYAAAEWCAYAAAANTdM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl0VGWeP/537amq7GtVVhKyk7AlBBNBQCGK7TJIi9/R\nUWGcaaUZfyw/l8O0jvbAdA/Hbo7aX0C7x5/SLT3SYzPaKiqxkT0KIQQDCWvIRlKVPSFbpZb7+yNw\nSXUCSSCVW1V5v87hUM+t51a96zHET26e57kyQRAEEBERERGRR5NLHYCIiIiIiG4fC3siIiIiIi/A\nwp6IiIiIyAuwsCciIiIi8gIs7ImIiIiIvAALeyIiIiIiL8DCnoiIiIjIC4y4sN+6dSvi4+Oh1WqR\nnZ2NQ4cO3bR/aWkp5s2bB51Oh+joaGzYsMHp+X379kEulw/6c+7cuVv7JEREREREE5hyJJ127tyJ\nNWvWYNu2bZgzZw62bNmCxYsXo6ysDDExMYP6d3R0YNGiRZg/fz6KiopQXl6OFStWQK/XY926dU59\ny8rKEBwcLLZDQ0Nv8yMREREREU08spHceXb27NmYPn063n33XfFYcnIyfvzjH+MXv/jFoP7btm3D\n+vXrYTabodFoAAD/8R//gW3btqG2thZA/xX7u+++G42NjQgJCRmrz0NERERENCENOxWnr68PxcXF\nyM/Pdzqen5+PI0eODHlOYWEh5s6dKxb11/rX1dWhqqrKqW92djYiIyOxcOFC7Nu37xY+AhERERER\nDVvYNzU1wW63IyIiwul4eHg4TCbTkOeYTKZB/a+1r50TGRmJd955B7t27cKuXbuQkpKCe+65Z9i5\n+0RERERENNiI5tiPlkwmG7ZPcnIykpOTxfYdd9yByspKvPHGG5gzZ454vL293RURiYiIiIjGRUBA\nwLi8z7BX7ENDQ6FQKGA2m52Om81mGI3GIc8xGAyDruZfO99gMNzwvXJycnD+/PlhQxMRERERkbNh\nC3u1Wo2srCzs2bPH6XhBQQHy8vKGPCc3NxcHDx6ExWJx6h8VFYW4uLgbvldJSQkiIyNHmp2IiIiI\niK4a0VScdevW4cknn0ROTg7y8vLwzjvvwGQy4bnnngMArF+/HseOHcM333wDAHj88cfx85//HMuX\nL8crr7yCs2fPYtOmTXj99dfF13zzzTcRHx+P9PR09PX14cMPP8Snn36KXbt23TDHeP0aYyIoKioC\n0L94mcYOx9U1OK6uwXF1DY6ra3BcXYdj6xpSTCcfUWG/bNkyNDc3Y+PGjaivr0dmZiZ2794t7mFv\nMplQUVEh9vf390dBQQFWrVqF7OxsBAcH44UXXsDatWvFPlarFS+++CJqa2uh1WqRkZGB3bt34777\n7hvjj0hERERE5P1GvHh25cqVWLly5ZDPvf/++4OOZWRkYP/+/Td8vRdffBEvvvjiSN+eiIiIiIhu\nYtg59kRERERE5P5Y2BMREREReQEW9kREREREXoCFPRERERGRF2BhT0RERETkBVjYExERERF5ARb2\nRERERERegIU9EREREZEXYGFPREREROQFWNgTEREREXkBFvZERERERF6AhT0RERERkRdgYU9ERERE\n5AWUUgcgoonBautDQ+tlmFpqIAgCYsInIywoEnIZry8QERGNBRb2RDTmBEFAQ1sdTM3VqG+uhqml\nFs3tJjgEh1M/rUaPSYZkxBmSEReRDH99oESJiYiIPB8LeyIaM4Ig4GJdGY6U7oG5tXbY/j2WLpRX\nnUB51QkAQIh/BGalzUdmQo6roxIREXkdFvZEdNtGUtDLIENwQDgMwTFwOByoMp9Dd2+nU5/mDjO+\n+n4nWjoaMHfa/ZymQ0RENAos7Inolt2soFcp1EiITIMhJAaG4BhEBEdDo/IRn3cIDjS11aPSdA6V\npnOobayA3W4DABwt/xadPe24L+cxKBT8NkVERDQS/D8mEd2ShtbL2HPsY9Q3VzsdVypUmJ6Yh5y0\n+dBr/W94vlwmR3hQFMKDopCTtgB9Vgu+KNyBC5dPAwDKKovR1duJh+c87dLPQURE5C1Y2BPRqFht\nVnx3ugBHz+yDw2EXj4+0oL8RtUqDh+c8jW+O/y9OXigEAFSZzuGjb7ZgcmA2fFT6MfsMRERE3oiF\nPRGNWE3DRew5+j9oudIoHlPIlZielIuctLvhewsF/UByuQKLspfCTxuAQ6VfAQAa2upgbvwcOQn3\n3tZrExEReTsW9kQ0LIu1F/tLPhevpF8THZaAe3MeRbB/+Ji9l0wmQ27GIvjqArDn6P/AITjQ09eJ\nIxc+R0ZmBkIDDGP2XkRERN6EhT0R3VRF3RnsOfonXOlpF4+pVT6YN+1HmJp4h8t2rslMyIHexx9/\nObwdQDesNgu+/O4jPLHoecjlCpe8JxERkSfjXnJENCSb3Yq9xZ/iz/t/51TUT46aghWLX8T0pDyX\nb0eZEJmKx+5eCZm8/31MLTUoOnvApe9JRETkqXjFnogGaWo34YsjO9DQVice02l8cXfW3yE1djpk\nMtm4ZTGGxCIpYgbO1R8HABwu/RqJURkI9g8btwxERESegIU9EYkEQcDJC4X49sRfYLNbxeMJkWm4\nb/Zj0Pv4SZJrcvhUmNouwYZe2OxW7Dn2Jyy7eyVvYEVERDQA/69IRACAbksnPjn4PgqK/iwW9QqF\nEvdkLcEjdz0jWVEP9O95PzXmLrGQr2mowMkL30mWh4iIyB2xsCci1DfX4PdfbhZvDgUAYYFGPJm/\nBjOT54zr1JsbCdCFYFbaArF9oORzdHS1SpiIiIjIvbCwJ5rgyqtO4KO/bnFaIDszeQ6eWLQaYYFG\nCZMNljtlkbi1Zp/NgoKiP0MQBIlTERERuQcW9kQTlENw4ODJ3fj8yIfi1BuNWotH7vpH3JO1BCql\nSuKEg6mUKtyXswwy9P8GoaKuHGWVxyVORURE5B5Y2BNNQBZrLz49+AG+K/ureCzYPxz/sGg1JkdN\nkTDZ8KLC4jEjeY7Y/rb4U3T1dEiYiIiIyD2wsCeaYNo6m/Hf3/xfp/n08cZU/MOi/8djtpCcO3Ux\nAvTBAICevm7sLf5U4kRERETSY2FPNIHUNVXiwz1vobGtXjyWnToPj9z1j9CotRImGx21SoN7cx4V\n22eqS9DQelnCRERERNJjYU80QdQ31+Djfb9Dj6ULAKCQK7F49v/BghkPQS5XSJxu9OIMyUiOyRTb\n35XtlTANERGR9FjYE00AjW31+PO+38Ji7QXQfxfZx+5+DhkJsyROdnvumLJIfHyu+iSaO8wSpiEi\nIpIWC3siL9fS0YD/+fZd9PR1AwB81Do8uuBZRIXFS5zs9kUERSEhMg0AIEDA96d51Z6IiCYuFvZE\nXqytsxl/2vsOunqvAAA0Kh/8eP5PEB4UKXGysXNH+kLxcXlVMdo6myVMQ0REJJ0RFfZbt25FfHw8\ntFotsrOzcejQoZv2Ly0txbx586DT6RAdHY0NGzbcsO+hQ4egVCqRmZl5wz5ENHodXW3407fvijee\nUinUWDrvn2AMiZE42diKCpuE2IgkAP178x8t/1biRERERNIYtrDfuXMn1qxZg1deeQUlJSXIy8vD\n4sWLUVNTM2T/jo4OLFq0CEajEUVFRXjrrbfwxhtvYPPmzYP6tra24qmnnsLChQvd4pb1RN6iq6cD\n/7PvXbRfvXqtUCix5K4VXjH9Zii5U65ftT9VcQwdXW0SpiEiIpLGsIX95s2bsWLFCjzzzDNISUnB\n22+/DaPRiG3btg3Zf8eOHejt7cX27duRnp6OpUuX4uWXXx6ysH/mmWewYsUK5Obm8rbwRGPEauvD\nx/t+h5aOBgCAXK7Aw3c+jThDssTJXCcmfDKiQicBAOwOG4rO7pM0DxERkRRuWtj39fWhuLgY+fn5\nTsfz8/Nx5MiRIc8pLCzE3LlzodFonPrX1dWhqqpKPLZ161Y0NjbilVdeYVFPNIb2l3yOhrY6AIBM\nJscDuU9gclS6xKlcSyaT4Y4BV+1/uPA970ZLREQTzk0L+6amJtjtdkRERDgdDw8Ph8lkGvIck8k0\nqP+19rVzSktL8e///u/48MMPOQWHaAxV1JXjxPnDYnth1hKkxE6TMNH4iTemIiIoGgBgtfeh6OwB\niRMRERGNL+VYv+BwhbrFYsFjjz2GX/3qV4iLixvVaxcVFd1ONBoCx9Q1pBhXi60HB87+L/qsPQCA\niIA4WNtUXvXfeLjPEqSMxqWucwCAgye+hqo3EGqlz3hE82je9DXiTjiursFxdR2O7dhKSkoa9/e8\n6RX70NBQKBQKmM3ON30xm80wGo1DnmMwGAZdzb92vsFgQH19Pc6cOYMVK1ZApVJBpVJhw4YNOH36\nNFQqFb755pvb+TxEE5IgCCitOSQW9RqVFlNj5ky434gZAibB1ycQAGC321DZVCZxIiIiovFz0yv2\narUaWVlZ2LNnD5YuXSoeLygowKOPPjrkObm5uXj55ZdhsVjEefYFBQWIiopCXFwcbDYbTp065XTO\nli1bUFBQgE8++eSmV/Gzs7NH/MHo5q79VM4xHVtSjesPF7/HlQtN0Ol1AICl8/4ZCZGp45rBlUYz\nrvowBb4o3AEAaLJUY+m0p6BR8ar9UPh9wDU4rq7BcXUdjq1rtLe3j/t7Drsrzrp16/DBBx/gvffe\nQ3l5OVavXg2TyYTnnnsOALB+/XosXHh90drjjz8OnU6H5cuX4/Tp09i1axc2bdqEdevWAQCUSiXS\n09Od/oSFhUGj0SA9PR16vd5FH5XIO7VeacTe4k/E9oykO72qqB+t1NhpCPQNAQBY+nrww8XvJU5E\nREQ0PoadY79s2TI0Nzdj48aNqK+vR2ZmJnbv3o2YmP6b3JhMJlRUVIj9/f39UVBQgFWrViE7OxvB\nwcF44YUXsHbt2hu+h0wmm3BTBojGgsNhxxeFf4TV1gcACPYPx7zpD0icSlpyuQI5aQuw59jHAICS\n84eRlTIXchlvtE1ERN5tRItnV65ciZUrVw753Pvvvz/oWEZGBvbv3z/iEK+99hpee+21Efcnon6F\np79BfXM1gP6C9oHcJ6BSqiVOJb20STOxv+RzWKy9aOtsRmX9uQn9WwwiIpoYeAmLyEPVN1fju9PX\nF5vPybwXEcHREiZyH2qlBhkJOWK7ZMAWoERERN6KhT2RBxIEAd8WfwqH4AAARIclYFbqfGlDuZnp\niXni44q6crR1NkuYhoiIyPVY2BN5oErTWVxuqgQAKORKLL7j/0AuV0gbys0E+4dhkiEFACBAwMkL\nhRInIiIici0W9kQeRhAEHC79WmxPnTxb3AWGnM1IvlN8XFpxFFabVcI0RERErsXCnsjDVNSViQtm\nFQolZqffLXEi95VgTIW/PggA0GPpwtnqEokTERERuQ4LeyIP0n+1fo/Ynp6YBz9doISJ3JtcrnCa\na3+Ci2iJiMiLsbAn8iAXLp+CubUWAKBSqDE7bYHEidxf5uQcKBT9O/uaWmrE33YQERF5Gxb2RB7C\nITic5tZPT8qDXusvYSLPoNP4IjVmutg+cY5X7YmIyDuxsCfyEOdrStHYVg8AUCnVyOHV+hEbuIj2\nTE0Jui2dEqYhIiJyDRb2RB7A4bDj8KnrV+tnJs+FzsdXwkSexRgSC0NwDADAbreh9OJRiRMRERGN\nPRb2RB7gTPVJNLebAQBqlQ9mpc6TOJHnmZF0/ap9yYUjcDjsEqYhIiIaeyzsidycw2FH4anrO+Fk\nJc+FVqOXMJFnSomdLo5bR1crKurPSJyIiIhobLGwJ3Jz5VUn0HKlEQCgUfkgm1frb4lKqUJmQo7Y\nLuHWl0RE5GVY2BO5MYfDjiMDrtZnp86Hj1orYSLPNj0xDzLIAACX6s+irbNZ4kRERERjh4U9kRur\nNJ0Ti08ftQ5ZKXMlTuTZAnyDER+ZKrZPXiiUMA0REdHYYmFP5MZKK67v3pKZkAONykfCNN5hWmKu\n+PhUxTHY7TYJ0xAREY0dFvZEbqq7txMXLp8W2xkJsyRM4z0SjKnw0wUCALotnThXWypxIiIiorHB\nwp7ITZVVHhe3ZIwMiUNogEHiRN5BLldg6uTZYpvTcYiIyFuwsCdyQ4Ig4NSlY2I7Y8BuLnT7MhNy\nIJf1f/urabiI5g6zxImIiIhuHwt7IjdkaqlFY1s9AEClUCM1brrEibyLny4Qk6PSxfYPF76XMA0R\nEdHYYGFP5IZODVg0mxwzlYtmXcBpEe2lY7DarBKmISIiun0s7IncjNVmRXlVsdjmolnXiDMkI8A3\nBADQ29eNczUnJU5ERER0e1jYE7mZ87WlsFh7AQCBviGICZ8scSLvJJfJMW3AItoSLqIlIiIPx8Ke\nyM0MnIaTkZADmUwmYRrvlhE/C3K5AgBQ11SJhtY6iRMRERHdOhb2RG6kvbMFVebzAAAZZJgyKVvi\nRN5Nr/VHcnSm2D558TsJ0xAREd0eFvZEbmTgFpeTjCnw1wdKmGZiGLiItqzyOPqsFgnTEBER3ToW\n9kRuwiE4cKriemGfyb3rx0VM+GQE+4UBAPqsvThTfULiRERERLeGhT2Rm6gxX0BHdysAQKvWOe2z\nTq4jk8mcrtpzES0REXkqFvZEbqJ0wKLZtElZUCpUEqaZWKbEZ0OhUAIAzC21qG+uljgRERHR6LGw\nJ3IDPZZunKstFduZ3Lt+XGk1eqTGXL+774nzhyVMQ0REdGtY2BO5gTPVJbDbbQCAiKBohAdFSZxo\n4pmRfKf4+Ex1Cbp7OyVMQ0RENHos7IncwLmaH8THvNOsNIwhsTCGxAIA7HYbt74kIiKPw8KeSGI9\nlm7UNlwU20kD9lWn8TUj6fpV+5MXCuFw2CVMQ0RENDos7Ikkdqm+HA7BAaD/qrGfLkDiRBNXSuw0\n6DS+AIAr3W24cPm0xImIiIhGjoU9kcQu1F4vHidHTZEwCSkVKkxNvENsF587JGEaIiKi0WFhTyQh\nm92KS/VnxHYiC3vJTZucC7ms/1tjTcNFNLbVS5yIiIhoZFjYE0mo2nwRfTYLACDQNwShAQaJE5G/\nPtBpncMJXrUnIiIPMeLCfuvWrYiPj4dWq0V2djYOHbr5/+xKS0sxb9486HQ6REdHY8OGDU7P79+/\nH3l5eQgNDYVOp0NaWhp+/etf39qnIPJQFy6fEh8nRmVAJpNJmIauGbj1ZVllMXos3RKmISIiGhnl\nSDrt3LkTa9aswbZt2zBnzhxs2bIFixcvRllZGWJiYgb17+jowKJFizB//nwUFRWhvLwcK1asgF6v\nx7p16wAAfn5+WLNmDTIzM6HT6XDo0CE8++yz8PHxwapVq8b2UxK5IYfgwMXLZWJ7clS6hGlooOiw\nBIQFRqKxrQ5Wex9OXTqGWanzpI5FRER0UyO6Yr9582asWLECzzzzDFJSUvD222/DaDRi27ZtQ/bf\nsWMHent7sX37dqSnp2Pp0qV4+eWXsXnzZrHPzJkzsWzZMqSlpSEuLg5PPPEE8vPzceTIkbH5ZERu\nztxSi86edgCAVq1DdFi8xInoGplMhpkDrtqXnD8s7lxERETkroYt7Pv6+lBcXIz8/Hyn4zcrwgsL\nCzF37lxoNBqn/nV1daiqqhrynBMnTqCwsBCLFi0aTX4ijzVwK8WEyHTI5QoJ09DfSoubCR+1DgDQ\n1tmMS3VnhjmDiIhIWsNOxWlqaoLdbkdERITT8fDwcJhMpiHPMZlMiI2NdTp27XyTyYS4uDjxeHR0\nNJqammC1WvHqq69i+fLlN8xSVFQ0XFwaJY6pa4xkXI+dPYjunv652/YuJf9bjMB4j1GgyoiK1lIA\nwFeHd2F2gnfOtefXnmtwXF2D4+o6HNuxlZSUNO7vOaI59qM1mgWAhw8fRmdnJwoLC/Hiiy8iPDwc\nP/3pT10Ri8htdFs6cKWnFQAglysQ5hctcSIaSlxIKioaSwEBaOq4jM7eNvj6BEodi4iIaEjDFvah\noaFQKBQwm81Ox81mM4xG45DnGAyGQVfzr51vMDhv53ft6v2UKVNgNpvxxhtv3LCwz87OHi4ujdC1\nn8o5pmNrpONadGY/dPr+aR4JkWm4Y3auy7N5Mim/XpvtleK0Kbv2CrKzFo57Blfh9wHX4Li6BsfV\ndTi2rtHe3j7u7znsHHu1Wo2srCzs2bPH6XhBQQHy8vKGPCc3NxcHDx6ExWJx6h8VFeU0Dedv2e12\nOBxcoEbeb+D8+sSoDAmT0HBmJM8RH5dWHEWPpUvCNERERDc2ol1x1q1bhw8++ADvvfceysvLsXr1\naphMJjz33HMAgPXr12PhwutXsR5//HHodDosX74cp0+fxq5du7Bp0yZxq0sA+M1vfoMvvvgC58+f\nx/nz5/Hee+/h17/+NZ588skx/ohE7qXH0oXaxksAABlkSOQ2l24tLiIJYYGRAACrrQ8nznPnLiIi\nck8jmmO/bNkyNDc3Y+PGjaivr0dmZiZ2794t7mFvMplQUVEh9vf390dBQQFWrVqF7OxsBAcH44UX\nXsDatWvFPg6HAy+//DIqKyuhVCqRmJiITZs24dlnnx3jj0jkXirqyiFc3TrRGBILvdZf4kR0MzKZ\nDDlpC/BF4Q4AQPG5g8hOvQtqpWaYM4mIiMbXiBfPrly5EitXrhzyuffff3/QsYyMDOzfv/+Gr7d6\n9WqsXr16pG9P5DWc7jYbPUXCJDRSqbHTcOiHL9He1YIeSxdOVRzDzAFTdIiIiNzBiKbiENHYsNqs\nuFR/VmwnRnN+vSeQyxXIHnDn2WNn9sHusEuYiIiIaDAW9kTjqNp8HlZbHwAg2C8MIf4Rw5xB7iIz\nIQdajR4A0NHVijNVJyRORERE5IyFPdE4ctoNh1frPYpKqUZW8lyxfezMPgiCIGEiIiIiZyzsicaJ\nQ3DgYl2Z2E6M4vx6TzM9KQ8qpRoA0NhWj4q6cokTERERXcfCnmicNLbVo6unAwCg1ehhDImVOBGN\nllajx7TJ128m9n35XgnTEBEROWNhTzROqkznxMeTDMmQyxUSpqFblZVyl/jf7nLjJVy+ek8CIiIi\nqbGwJxonlQMK+zhDsoRJ6Hb46wORPilLbH9f/q2EaYiIiK5jYU80Dqw2K2obr9/ELS6Chb0ny0md\nLz6+ePk0GtvqpQtDRER0FQt7onFwuekS7HYbACDYPxz++kCJE9HtCAmIQNKAXY2Ole+TLgwREdFV\nLOyJxsHfzq8nz5eTtkB8XF59Au2dLRKmISIiYmFPNC4qB9xtlvPrvUNk6CTEhE8GADgcdhw5XSBx\nIiIimuhY2BO5WFdPBxra6gAAcrlCLAbJ8+Vl5IuPT18qQnOHWcI0REQ00bGwJ3KxKvMF8XFkSBw0\nKh8J09BYio1IFH8DIwgOHC7dI3EiIiKayFjYE7lYFbe59Gpzpy4WH5+tLoG59bKEaYiIaCJjYU/k\nQoIgOO1fP8mQJGEacgVjSKzTDjmHfvhSwjRERDSRsbAncqHmDjM6e9oBABq1FobgGIkTkSvcmXkf\nZJABACrqynk3WiIikgQLeyIXcpqGE5EIuVwhYRpylbBAI9ImzRDbB3/4EoIgSJiIiIgmIhb2RC5U\n6VTYc369N8vLyBd/cKtpuIhq83mJExER0UTDwp7IRex2G2oaLoptLpz1bkF+YciMnyW2edWeiIjG\nGwt7Ihe53FQJq60PABDoG4Igv1CJE5Gr3TFlERQKJQCgvrkaFy6fljgRERFNJCzsiVykasBUDF6t\nnxj89YGYkXin2D5c+hUcgkPCRERENJGwsCdykSqnbS5Z2E8UOekLoFKqAQCNbfU4U3VC4kRERDRR\nsLAncoEeSxdMLbUAAJlMjpjwRIkT0XjR+/ghK+UusX3why9htVklTERERBMFC3siF6g2X4BwdQqG\nITgaWo1O4kQ0nnJS50Or0QMAOrpacfzsAYkTERHRRMDCnsgFnPav5zScCUej1uLOzPvE9vdlf0Vn\nT4eEiYiIaCJgYU/kApUDFs5yfv3ENG3ybIQGGAAAfTYLDv3wpcSJiIjI27GwJxpjXZYOtHc2AwBU\nSjUiQ+IkTkRSkMsVWDDzYbF9quIYzFfXXRAREbkCC3uiMdZ05bL4ODY8UdzXnCaeSYZkTI5MBwAI\nELC3+FPetIqIiFyGhT3RGGvqrBMfxxmSJExC7mD+jAchlysAALWNFThX84PEiYiIyFuxsCcaQ4Ig\noLmzXmzHRrCwn+iC/cMxI+n6Tav2l3zO7S+JiMglWNgTjaGOnhZYbRYAgM7HV1w8SRNb7pRF0Kr7\ntzxt72pB8Tluf0lERGOPhT3RGGoeOA0nIgkymUzCNOQutBod8jLvFdvfnf4rurj9JRERjTEW9kRj\naOD8+tgI3m2WrpuemIuQgAgA/dtfHuT2l0RENMZY2BONEbvDjpYuk9jm/HoaSC5XYMGMh8T2qYpj\nqG+ukTARERF5Gxb2RGPE1FwNu90GAAjQByPQN0TiRORu4o2pSIhMA9C//eVfj++CQ3BInIqIiLwF\nC3uiMVJlviA+5jQcupG7Zz4Mhbz/3gb1zdU4VXFU4kREROQtWNgTjZFq83nxMafh0I0E+YVhVtp8\nsX3g5G70WLqkC0RERF5jxIX91q1bER8fD61Wi+zsbBw6dOim/UtLSzFv3jzodDpER0djw4YNTs/v\n2rUL+fn5CA8Ph7+/P+644w589tlnt/YpiCRmtfWhrqlKbPOKPd3MHen3wF8fBADosXRxIS0REY2J\nERX2O3fuxJo1a/DKK6+gpKQEeXl5WLx4MWpqhl741dHRgUWLFsFoNKKoqAhvvfUW3njjDWzevFns\nc+DAASxcuBC7d+9GSUkJ7r//fixZsmTYHxiI3FFdUyXsjv759b4+gfDV+kuciNyZSqnG3TMfFts/\nXPgOphYupCUiotszosJ+8+bNWLFiBZ555hmkpKTg7bffhtFoxLZt24bsv2PHDvT29mL79u1IT0/H\n0qVL8fLLLzsV9m+++SZeeuklZGdnIyEhAf/2b/+GrKwsfPLJJ2PzyYjGUdWAaTghvpESJiFPkRiV\n4bSQ9psiLqQlIqLbM2xh39fXh+LiYuTn5zsdz8/Px5EjR4Y8p7CwEHPnzoVGo3HqX1dXh6qqqiHP\nAfqv9Ac+XWJwAAAgAElEQVQHB480O5HbqB6wcDbUzyhhEvIUMplsiIW0xyRORUREnkw5XIempibY\n7XZEREQ4HQ8PD4fJZBryHJPJhNjYWKdj1843mUyIi4sbdM6WLVtQV1eHJ5988oZZioqKhotLo8Qx\nvX1WmwUXa84CggDIgGC9kePqIt44rhG6BFwwlwAAPj/43+hstEKt9BnXDN44ru6A4+oaHFfX4diO\nraSk8d9IwyW74shkslH1//Of/4yXXnoJf/zjHxETE+OKSEQu09Jl6i/qAQRoQ6BWaoY5g+i6xIhp\n8FHrAfT/kHjWdFziRERE5KmGvWIfGhoKhUIBs9nsdNxsNsNoHHrKgcFgGHQ1/9r5BoPB6fjHH3+M\np59+Gn/4wx/wox/96KZZsrOzh4tLI3Ttp3KO6e376/Fa6PQ6AECIPgoAx3WsefvXa5BRj08PfQAA\naO6pQXRCBAzBrr/I4e3jKhWOq2twXF2HY+sa7e3t4/6ew16xV6vVyMrKwp49e5yOFxQUIC8vb8hz\ncnNzcfDgQVgsFqf+UVFRTtNw/vSnP+Gpp57C9u3b8cgjj9zqZyCSVE3DgPn1vpxfT6OXFJ2BeGMq\nAC6kJSKiWzeiqTjr1q3DBx98gPfeew/l5eVYvXo1TCYTnnvuOQDA+vXrsXDhQrH/448/Dp1Oh+XL\nl+P06dPYtWsXNm3ahHXr1ol9PvroIzzxxBPYtGkT5syZA5PJBJPJhJaWljH+iESu09XTgca2egCA\nXK5AkD5imDOIButfSPt3TgtpSy/yjrRERDQ6Iyrsly1bhjfffBMbN27EjBkzcOTIEezevVucD28y\nmVBRUSH29/f3R0FBAerq6pCdnY3nn38eL7zwAtauXSv2effdd+FwOLB69WpERkaKf3784x+P8Uck\ncp3qhovi48iQOCgVKgnTkCcL9ne+I+3Bk1/wjrRERDQqw86xv2blypVYuXLlkM+9//77g45lZGRg\n//79N3y9b7/9dqRvTeS2qgfsXx8bkQhYbtKZaBh3pN+D8spitHe1oKevGwdO7sa9OY9KHYuIiDyE\nS3bFIZooBu5fHxsx/ttakXdRKdW4O+vvxHbpxe9R13Tje38QERENxMKe6Ba1d7agrbMZAKBSqBEZ\nEjvMGUTDS4yagslRUwAMWEjrsEucioiIPAELe6JbVD1gN5zo8HgoFCOe2UZ0U3fPfFhcr2FurUXJ\nhUKJExERkSdgYU90i6pMA+fXcxoOjZ1A3xDckX6P2D70w5fo6umQMBEREXkCFvZEt0AQBKcr9rER\niRKmIW+UnTofQX6hAACLtRf7T34hcSIiInJ3LOyJbkFjW714BVWr1iE8MFLiRORtVEoV7slaIrZP\nXypCbWPFTc4gIqKJjoU90S2oNJ0VH8cakiGXKyRMQ94q3piK5JipYpsLaYmI6GZY2BPdgkrTOfFx\nvDFFwiTk7RbMeBgqpRpA/2+KTpw/LHEiIiJyVyzsiUbJautzmhIRF5EsYRrydv76QOROWSS2D5d+\nzYW0REQ0JBb2RKNU21gBu90GAAjxj4C/PlDiROTtslPuQrBfGAAupCUiohtjYU80SpX116fhTOI0\nHBoHCoUSd//NQtrLjZckTERERO6IhT3RKA1cODvJwMKexke8MQXJMZli+6/HP+FCWiIicsLCnmgU\nrnS3oandBABQyJWIDo+XOBFNJPNnPASVon8hrbm1Ficvfi9xIiIicics7IlGYeBuOFFh8VArNRKm\noYkmQB+M2el3i+1DP3yJbkunhImIiMidsLAnGoUqbnNJEstOnY9A3xAAQG9fNw6e/FLiRERE5C5Y\n2BONkENwOF2xjzNwm0safyqlCnfPfFhsl178HvXN1RImIiIid8HCnmiEGlovo8fSBQDQ+fgiLNAo\ncSKaqCZHTcHkyHQAgAABfz3+v3AIDolTERGR1FjYE43QpXrn3XDkMv7zIeksmPkwFAolAKC+uRqn\nKo5KnIiIiKTGyoRohAbOr5/EaTgksSC/UOSkLhDbB07uRm9fj4SJiIhIaizsiUbAYu3F5aZKsc3C\nntzB7PS74a8LAgD0WLpwuPRriRMREZGUWNgTjUBNw0XxZkDhgZHQa/0lTkQEqJRqzJ/xoNguOX8Y\njW31EiYiIiIpsbAnGgGnaTjc5pLcSHLMVMSGJwLo37lp7/FPIAiCxKmIiEgKLOyJRqDSaeEsp+GQ\n+5DJZLg76+/ExdzVDRdwruYHiVMREZEUWNgTDaO9swUtVxoBACqFGpGh8RInInIWFmjE9KQ7xfa+\nE5/BauuTMBEREUmBhT3RMCpN16/WR4cnQKVUSZiGaGh3Zt4LncYXANDR3Yrvy/ZKnIiIiMYbC3ui\nYVRym0vyAD5qLeZOu19sHz3zLdo6myVMRERE442FPdFNOBx2p4WzcSzsyY1lJMyCITgGAGC327Dv\nxF8kTkREROOJhT3RTdQ318Bi7QUA+GkDEBpgkDgR0Y3JZXLck/V3Yvt87SmnOyYTEZF3Y2FPdBMX\n68rEx3HGFMhkMgnTEA0vMnQSpsRni+29xZ/AbrdJmIiIiMYLC3uiGxAEAecHbBuYGDVFwjREIzdv\n2o+gVvkAAFo6GnD83EGJExER0XhgYU90A80dZqdtLicZeGMq8gx6rT/uzMgX20dO7UFHV5uEiYiI\naDywsCe6gfM1peLj+MhUbnNJHmVG8hxxTYjV1of9JZ9JnIiIiFyNhT3RDZyvPSU+To6ZKmESotFT\nyBW4J2uJ2D5TXeK0wxMREXkfFvZEQ2jrbIa5tRYAoJArkRCZJnEiotGLjUhEWtxMsf3X41xIS0Tk\nzVjYEw1h4NX6OEMSNFcXIhJ5mnnTH4BaqQHQv26EC2mJiLwXC3uiIQzcDScpOlPCJES3x08XgLzM\n6wtpC08V4Ep3u4SJiIjIVUZc2G/duhXx8fHQarXIzs7GoUOHbtq/tLQU8+bNg06nQ3R0NDZs2OD0\nvMlkwuOPP460tDQolUqsWLHi1j4B0Rjr7OnA5aZKAIBMJuc2l+TxZibPRUhABACgz2bhQloiIi81\nosJ+586dWLNmDV555RWUlJQgLy8PixcvRk1NzZD9Ozo6sGjRIhiNRhQVFeGtt97CG2+8gc2bN4t9\nLBYLwsLCsH79esyePZs3/iG3MXAaTkxYAnQ+vhKmIbp9CrkCC7MeEdvlVSdQbb4gYSIiInKFERX2\nmzdvxooVK/DMM88gJSUFb7/9NoxGI7Zt2zZk/x07dqC3txfbt29Heno6li5dipdfftmpsI+Li8Nb\nb72Fp556CsHBwWPzaYjGwPna69tcJsVwGg55h/6FtDPE9jfHd8EhOCRMREREY23Ywr6vrw/FxcXI\nz893Op6fn48jR44MeU5hYSHmzp0LjUbj1L+urg5VVVW3GZnIdXosXagZcCUzMSpDwjREY8tpIW27\nGZcaTw1zBhEReZJhC/umpibY7XZEREQ4HQ8PD4fJZBryHJPJNKj/tfaNziFyBxcvl4lXMY0hsfDX\nB0qciGjs+OkCnRbSnjMVo8vChbRERN5C6YoXddV8+aKiIpe87kTGMXVWdOkbdHd1AwBUAX63PD4c\nV9fguN4+QdBBKWjR0d0MACitOQzdMX+uc3IBfr26BsfVdTi2YyspKWnc33PYK/ahoaFQKBQwm81O\nx81mM4xG45DnGAyGQVfmr51vMBhuNSuRS9nsVjReuSy2IwLiJExD5BpymRxTo+eIhXxzZz1qWnhH\nWiIibzDsFXu1Wo2srCzs2bMHS5cuFY8XFBTg0UcfHfKc3NxcvPzyy7BYLOI8+4KCAkRFRSEu7taL\npezs7Fs+l5xd+6mcY3rd2eqT8Kno/3oNCzRi/p33jPo1OK6uwXEde0p/G/YVfQEAqL1ShkVzH4Cf\nLkDiVN6BX6+uwXF1HY6ta7S3j/9UxxHtirNu3Tp88MEHeO+991BeXo7Vq1fDZDLhueeeAwCsX78e\nCxcuFPs//vjj0Ol0WL58OU6fPo1du3Zh06ZNWLdundPrlpSUoKSkBO3t7WhubkZJSQnKysrG8OMR\njZzTbji8KRV5ubyMe6HT+AMALNZe/PX4/0qciIiIbteI5tgvW7YMzc3N2LhxI+rr65GZmYndu3cj\nJiYGQP+C2IqKCrG/v78/CgoKsGrVKmRnZyM4OBgvvPAC1q5d6/S6M2fOBNA/J18QBHz22WeYNGmS\n02sRjQerzYqLl6//UMnCnrydSqnC1Jg5+O7CbgD9P9ieq/kByTFTJU5GRES3asSLZ1euXImVK1cO\n+dz7778/6FhGRgb2799/09d0OLiHMrmHavN59NksAIBA3xCEBQ69foTIm4T4GhETkoLm3v6bDX5z\n/H8RE54IrUYncTIiIroVI5qKQ+TtBk7DSY6Zyh1CaMJIM86CXts/JaerpwP7Sz6TOBEREd0qFvY0\n4fVZLThX84PYTormTalo4lApNViY9YjYLq04iirTeQkTERHRrWJhTxNeeVUxLNZeAECQXxgMIbES\nJyIaX8kxmU5z6786uhM9lm4JExER0a1gYU8TmiAIKLlQKLanJ+ZCLuM/C5p4FmYtgUatBQB0dLXi\n66M7IQiCxKmIiGg0WMHQhFbXVIWG1v6bUqkUakyJnyVxIiJp6LX+uC/nMbF9vvYUis8dlDARERGN\nFgt7mtBKLhwRH6fGTeduIDShJcdkIitlrtjeV/I56purJUxERESjwcKeJqzu3k6crT4ptmck3Slh\nGiL3MG/aAzAE99+jxOGw47PDf+B8eyIiD8HCnias0oqjsDtsAABjSCwigqMlTkQkPYVCiQfvfBIa\nlQ8AoL2rhfPtiYg8BAt7mpAcDjtOOi2azZMwDZF7CfQNwX2zOd+eiMjTsLCnCelS/Vm0d7UAALRq\nHVJip0uciMi9JMdMxcxkzrcnIvIkLOxpQhq4aDYjIQcqpUrCNETuad70Hw2ab99t6ZQ4FRER3QgL\ne5pw2jqbcanujNielpgrYRoi96VUqAbNt//4299xMS0RkZtiYU8TzskLhRDQvxAw3piKIL9QiRMR\nua9A3xAsvuPvIYMMAGBurcWf9/9OvFszERG5Dxb2NKFYbVaUVhwV29OTuGiWaDhJ0Rm4d/YysV3f\nXI1d+99Dn80iYSoiIvpbLOxpQjlXcxI9li4AgL8uCAnGVIkTEXmGzIQcLMx+RGzXNlbgk4MfwGqz\nSpiKiIgGYmFPE0rJgC0upyXmQi5XSJiGyLPMSLoT82c8JLarTOfwl8PbYbfbJExFRETXsLCnCaOu\nqQp1TZUAALlcgczJOdIGIvJAs1LnYc7UxWK7oq4cnxfugMNhlzAVEREBLOxpgnAIDuwt/kRsp8RM\ng97HT8JERJ4rd8pC3DFlodg+V/MDdh34/9DVe0XCVERExMKeJoTTl4rEm+so5ErcmZkvcSIizzYn\n8z5kp8wT25fqz+CDL3+FigFbyRIR0fhiYU9ez9LXgwMnvxDbs1LnIcgvTMJERJ5PJpNh/owHMTv9\nHvFYd28n/rz/d9hb/Clsdi6qJSIabyzsyesdOVWA7t7+u2X66QIxe8o9w5xBRCMhk8lw17T78eiC\nZ6HX+ovHj589gB173kZTu0nCdEREEw8Le/JqTe0mFJ8/JLbnTX8AaqVGwkRE3meSIRnL7/t/MTky\nXTzW0FaHP3z9Jo6fPcAtMYmIxgkLe/JagiBg7/FPxN06YsInIzV2usSpiLyTzscXS+76R9yTtQQK\nhRIAYLNbsbf4U/z2LxtxuPRrdPV0SJySiMi7KaUOQOQq52tLUWU+DwCQy+S4J2sJZDKZxKmIvJdM\nJsPM5DmICZ+MLwp3oLGtHgDQbenEkVN78H35XqTHzURWyl0ICzRKnJaIyPuwsCevZLX14dviv4jt\n6Ul5LCSIxklYoBFPLFqNkguHUXz2EDq6WwEAdrsNpRVHUVpxFHGGZCRGTUFcRBKC/cP5QzcR0Rhg\nYU9e6Wj5t2IxodXokZdxr8SJiCYWlVKFWanzkZU8F2drfsDxswfELWeB/rvWVpnOAQD0Wn/ERSQi\nNiIJsRGJCNAHSxWbiMijsbAnr9Pe2YKj5d+K7blT74dWo5MwEdHEJZcrkBY3A6mx01HXVImiswdw\nvvYUBMEh9unq6UBZZTHKKosBAHofP4QEGBAaEIHQAANCAwwICTDAR62V6mMQEXkEFvbkVaw2K3Z/\n99/iHtoRwdHITJglcSoikslkiAqLR1RYPNo7W3CxrgzV5vOobrgIS1+PU9+u3ivo6r2C6qtrZK7R\na/0R7BeKQN/+P0F+oQj0C0Wgbwg0Kp/x/Dg3ZHfY0dPXiR5rF85UlaCr9wr6bBbY7FZYbX2w2fpg\ntffBarNCIVdAo9bCR62Fj1oHjcoHPmodtBo9QvzDofPx4xQlIhoVFvbkNewOOz47/HvUNlaIx+6Z\nuQRyuULCVET0twJ8gzEzeQ5mJs+Bw2GHubUONQ0XUG0+j9qGS7Da+4Y8r6unA109HahpqBj0nFaj\nh78+CAH6YATog8XHfrpA6H18odXob/t7gUNwoKvnCq50t+FKdxs6ulvR0dUmtq/0tKO75wq6uroA\nACfrbu83hVqNHqEBBoQFGsXfXIQFRXLLXiK6IRb25BUcggNfff8RLtaVicfmz3gQUWGTpAtFRMOS\nyxUwhsTAGBKDnLQFcDjsaO9qQVO7GU3tJjRf/bulowF2h+2Gr9Nj6UKPpQvmltohn5dBBh+NDjqN\nL3RXC32lQgWZTAa5TA65XHH1bzlsdhssfT2wWHvQ29cDS1//373WHnH73PHQY+lCTcNF1DRcFI/1\nj1csYsMTERuRCGNIHFRK1bhlIiL3xsKePJ4gCPi2+FNxfi4AzE6/B7NS50sXiohuiVyuQJBfGIL8\nwpAUnSEe7y/4W9HW2YTWK01o62zu/3P18c2KfgAQIIjFf3OH2WX5ZZBBo9LCR6XHpOgE6H38oVZp\noFKqoVKqoVSooVaqoVSoYHfY0dvXDYu1t//vqz9MXOluR3O7GX02y6DXdzjsuNx4CZcbL6HwdAEU\nCiWiQiYhNiIR8ZGpCA+KglzGW9QQTVQs7MnjHTm1B8Xnrt9ddlpiLuZOXSxhIiIaa/0Ff/+8+vi/\n2bn22hSZjq4WtHe1oL2rtf9xZws6ezrQbelEr6UbAoTbzqHV6OGvC4KfLgD++iD46QKv/gmAny4Q\nvj7+OHGiBACQnZ19y+/jEBy40tWGxvZ6NLWZ0NRuQkPbZTS3O/9QYrfbUN1wAdUNF3Co9Cvotf5I\nMKYiITIdcYYkt1l7QETjg4U9ebTjZw/iyKk9Yjs1djoW8kZURBOKXCa/WlgHICosfsg+DocdPZYu\ndFu60N17Bd2WLjgcdjgEBwTBAYfDIT6WyeTwUevgo9b2X32/usBVrdKO27QXuUyOAN9gBPgGIzFq\nini8u7cTNQ0XUW3uL+ZbOhqczuvq6RDvFSCXKxATPhmTI9OQEJmOIL/QcclORNJhYU8e6/SlIuwt\n/kRsxxtTcf8df8/FskQ0iFyugF7rD73WH4Dn3qxO5+OLlNhpSImdBgC40t2OmoaLqKw/i4r6cvRY\nusS+DoddvF/A3uJPEewfjsmR6UiITENUWDwU/F5J5HVY2JPH6eq9gv0nPsfpyiLxWFToJDw05yko\nFPySJqKJw08XgPRJM5E+aSYcDjvqm2tQUV+OirpyNLRedurb0tGAlo4GHDuzDxq1FvGGFEwypiAu\nIhn++kCJPgERjSVWQeQxHIIDpReP4sDJL9Db1y0eDws0Ysldz3ALOCKa0ORyBaLCJiEqbBLmTl2M\njq42XKovx8W6clSbzjttI2rp68GZ6hKcqe5fDxASEIFJhhRMMqQgOjye30+JPBQLe/IIjW31KCj6\nMy43XnI6nnJ1Tj3vLEtE5MxfH4hpibmYlpgLq82KmoYLuHi5DBV15ejobnXq29xuRnO7GcfPHoBC\nroQxJBZRYfGIDouHMSSO32OJPMSICvutW7fijTfegMlkwpQpU/Dmm29izpw5N+xfWlqKf/mXf8Gx\nY8cQHByMZ599Fq+++qpTn/3792PdunUoKytDZGQkXnrpJTz77LO392nI63R0teHE+UMoOnvAaf/o\nAN8QLMx6BAmRqRKmIyLyDCqlCgmRaUiITIMgCGhsq0el6SwqTedQ21gBu/36dqF2hw21jRWobazA\n9+jfwjM00ICo0HhEhsYhPCgKIf7hXM9E5IaGLex37tyJNWvWYNu2bZgzZw62bNmCxYsXo6ysDDEx\nMYP6d3R0YNGiRZg/fz6KiopQXl6OFStWQK/XY926dQCAS5cu4f7778c//dM/4Y9//CMOHjyIn/70\npwgLC8Mjjzwy9p+SPErrlUacrz2FczU/oL652uk5uVyBWanzkTtlIVRKtUQJiYg8l0wmQ3hQJMKD\nIpGTtgBWWx9qGytQWX8OVeZzaGyrd+ovoP8Hgca2epRcOAIAUCpUCAs0IiwwEuFBUQgPjESwfziv\n7BNJbNjCfvPmzVixYgWeeeYZAMDbb7+Nr776Ctu2bcMvfvGLQf137NiB3t5ebN++HRqNBunp6Thz\n5gw2b94sFvbvvPMOoqOj8dZbbwEAUlJS8P333+NXv/oVC/sJRhAEdPVeQUtHA2obK3CuphSNbXVD\n9o0Ki8ei7KUIC/TcHS2IiNyNSqlGvDEV8cb+34B29XTgclMlLjdW4nLTJZhbauEQHE7n2OxW1DdX\nD7r4otXoEeQbiiC/MAReve9AgD4YvtoA6LV+3ImHyMVuWtj39fWhuLgYL730ktPx/Px8HDlyZMhz\nCgsLMXfuXGg0Gqf+r776KqqqqhAXF4fCwkLk5+cPes3t27fDbrdDoeA/fG9gt9vQ29eNXmsPei09\nV++u2IW2zha0XmlAS0cjWq40os/ae8PXuLYPc0Z8NlLjZvCOikRELqbX+iM5ZiqSY6YCAPpsFtQ3\nVeNy0yWYWmrR0HoZV7rbhjz32t1965qrBj0ngwxaHz18tQHw1fpD7+MHH7UOGrV20H0D1CoNVAo1\nlEpV/9/c8YxoRG76L6WpqQl2ux0RERFOx8PDw2EymYY8x2QyITY21unYtfNNJhPi4uJgNpsHvWZE\nRARsNhuampoGPXfNrv3v3fzTkGi4Oyyazf13L6zqKnE+TxD6zxacX0cQBAiCo/9vCGLb4XDA7rDB\n7rCLfzvsNtjsNqcdGEZDoVBikiEFydGZmByVDq1Gf0uvQ0REt0+t1CDOkIQ4Q5J4rNvSicbWOphb\nL6OhtQ5N7fVovdIEm916w9cRIKC7txPdvZ2DtuIciZ6eXihkCnxf/RnkcgXkcjkUcgXkMkV/WyaH\nTCa7+qf/sVwmQ/+PFDJAJoMMuPq3DP33Mbx+M8OxuLGhDJ55c8RrNUF110mJkwwvPjINM5LypI7h\ntsb8R2BX3vHznuk/dtlrk3vq67Whr7dd6hgjlpTU/z++9nbPyewJOK6uwXF1jYkyroHaCARqI5AS\nOVPqKDTBePu/rdtx03kNoaGhUCgU4k9y15jNZhiNQ89zNhgMg67mXzvfYDDctI9SqURoKG95TURE\nREQ0Wjct7NVqNbKysrBnzx6n4wUFBcjLG/rXILm5uTh48CAsFotT/6ioKMTFxYl9CgoKBr3mrFmz\nOL+eiIiIiOgWyARBuOlk7D/96U948sknsXXrVuTl5eGdd97B+++/j9OnTyMmJgbr16/HsWPH8M03\n3wDo3+4yJSUF8+fPxyuvvIKzZ89ixYoVeP3117F27VoAQGVlJTIyMvDP//zP+MlPfoLDhw9j1apV\n+Oijj7BkyRLXf2oiIiIiIi8z7Bz7ZcuWobm5GRs3bkR9fT0yMzOxe/ducQ97k8mEiooKsb+/vz8K\nCgqwatUqZGdnIzg4GC+88IJY1APApEmTsHv3bqxduxbbtm1DVFQUfvOb37CoJyIiIiK6RcNesSci\nIiIiIvfn1puCb926FfHx8dBqtcjOzsahQ4ekjiSZAwcO4KGHHkJ0dDTkcjm2b98+qM/rr7+OqKgo\n6HQ6LFiwAGVlZU7PWywWPP/88wgLC4Ovry8efvhhXL7svOVYa2srnnzySQQGBiIwMBBPPfXUoNXn\n1dXVePDBB+Hr64uwsDCsXr0aVuuNtzhzZ7/85S8xa9YsBAQEIDw8HA899BBOnz49qB/HdnS2bNmC\nadOmISAgAAEBAcjLy8Pu3bud+nBMb98vf/lLyOVyPP/8807HObaj8/rrr0Mulzv9iYyMHNSHYzp6\n9fX1ePrppxEeHg6tVospU6bgwIEDTn04tqMzadKkQV+vcrkcDzzwAID+7ak5pqNns9nwr//6r0hI\nSIBWq0VCQgJeffVV2O12p35uP7aCm/roo48ElUol/Nd//Zdw5swZ4fnnnxd8fX2F6upqqaNJYvfu\n3cLPfvYz4eOPPxZ0Op2wfft2p+f/8z//U/Dz8xN27dolnDp1Sli2bJkQGRkpXLlyRezz3HPPCZGR\nkcI333wjFBcXC/PnzxemT58u2O12sc99990nZGRkCN99951QWFgoTJkyRXjwwQfF5202m5CRkSEs\nWLBAOHHihFBQUCBERkYKzz//vOsHwQXuvfde4YMPPhBOnz4tlJaWCkuWLBEMBoPQ0tIi9uHYjt6n\nn34qfPXVV8LFixeF8+fPCz/72c8ElUollJSUCILAMR0LhYWFQnx8vDBt2jSnz8OxHb3XXntNSEtL\nE8xms/inqalJfJ5jemtaW1uF+Ph44emnnxaOHTsmVFZWCnv37hXKy8vFPhzb0WtqanL6Wj1x4oQg\nl8uF3//+94IgcExv1c9//nMhODhY+Pzzz4WqqirhL3/5ixAcHCxs2LBB7OMJY+u2hX1OTo7wk5/8\nxOlYUlKSsH79eokSuQ9fX1+nwt7hcAgGg0H4xS9+IR7r6ekR/Pz8hHfffVcQBEFoa2sT1Gq18Mc/\n/lHsU1NTI8jlcuHrr78WBEEQysrKBJlMJhw5ckTsc+jQIUEmkwnnzp0TBKH/Bwy5XC7U1taKfT78\n8DcsW+UAAAfKSURBVEPBx8fH6QvbU3V2dgoKhUL4/PPPBUHg2I6l4OBg4be//S3HdAy0tbUJkydP\nFvbt2yfMnz9f/GbPsb01r732mpCRkTHkcxzTW7d+/Xphzpw5N3yeYzs2Nm7cKAQFBQm9vb0c09vw\nwAMPCMuXL3c69tRTTwkPPPCAIAie8/XqllNx+vr6UFxcjPz8fKfj+fn5OHLkiESp3NelS5dgNpud\nxsvHxwd33XWXOF7Hjx+H1Wp16hMdHY20tDQUFhYCAAoLC+Hr64vc3FyxT15eHvR6vfg6hYWFSE9P\nR1RUlNgnPz8fFosFx48fd+nnHA8dHR1wOBwICgoCwLEdC3a7HR999BF6e3tx1113cUzHwE9+8hM8\n+uijmDdv3tW7Rffj2N66iooKREVFISEhAX//93+PS5cuAeCY3o5PPvkEOTk5eOyxxxAREYEZM2Zg\ny5Yt4vMc29snCALee+89/MM//AM0Gg3H9DYsXrwYe/fuxdmzZwEAZWVl+Pbbb/GjH/0IgOd8vY75\nnWfHQlNTE+x2OyIiIpyOh4eHD7qxFeH/b+/+QprqwziAf7fanIPmxbKcE5qCuZpDZGLOLoogKAoh\niFIwqEAoLUczKLSLilZ0W7gbg/Im0sCLwBsHk3Jo4MX8k2kZGiKlVhfCAo22570ID53+2Otmr3Pv\n9wODec7z25+v29njPOd3lEx+lde7d++Umg0bNsBsNqtqtm7dqoyfmZlBZmamar1Go1HlPjMz89P9\nLJ3ILBV+Nx6PB8XFxcobjtnGb3h4GG63G4uLi0hPT0d7ezsKCgqUDRczjU9LSwsmJibw8OFDAOqz\nffP1Gp+ysjK0trbCbrdjdnYWN27cQHl5OUZGRphpAiYmJuD3++H1etHY2IhwOKwcD1JXV8dsV0Eg\nEMDbt29RU1MDgNuARNTW1mJ6eho7duzAxo0b8fXrV1y5cgVnzpwBsH6yTcrGnlbP9x/6vyJxTIoU\nz5j1wOv1ore3F6FQ6I+5Acz2T+x2O4aGhjA/P4/Hjx+jsrIS3d3dy45hpst79eoVmpqaEAqFlJP5\nybddKv84ltn+3oEDB5TrhYWFcLvdyM3NRWtrK3bt2vXbccx0ebFYDKWlpfD5fACAoqIijI+Po7m5\nGXV1dcuOZbb/TktLC0pLS+F0Ov9Yy0yXd+fOHdy/fx+PHj2Cw+FAOByGx+OBzWbD6dOnlx2bTNkm\n5a44S3+VzM7OqpbPzs7CYrGs0aNKXllZWQDwy7yW1mVlZSEajeLTp0/L1nz48EG1XkQwNzenqvnx\nfpb+w7JUsx5duHABbW1tCAaDsNlsynJmGz+dToe8vDwUFxfj5s2bKCsrQ3Nzs/IeZqYr19fXh48f\nP8LhcECn00Gn0+HZs2fw+/3Q6/XYvHkzAGabKKPRCIfDgTdv3vD1moDs7Gzs3LlTtcxut2NqagoA\nt6+Jmpubw5MnT5Rv6wFmmgifz4fGxkYcO3YMDocD1dXV8Hq9uHXrFoD1k21SNvZ6vR4ulwtdXV2q\n5YFAAOXl5Wv0qJJXbm4usrKyVHktLCwgFAopeblcLuh0OlXN9PQ0xsbGlBq3241IJKLsBwZ8ayQ+\nf/6s1JSXl2N0dFQ1dVMgEEBaWhpcLtdffZ5/i8fjUZr67du3q9Yx29UTjUYRi8WYaQKOHDmCFy9e\nYHBwEIODgxgYGEBJSQmqqqowMDCA/Px8ZrsKFhYWMDo6CovFwtdrAnbv3o2xsTHVstevXytfnjDb\nxDx48AAGgwFVVVXKMmYaPxGBVqtui7VarfLN+brJdtlDa9dQW1ub6PV6uXfvnrx8+VLq6+tl06ZN\n/9vpLiORiITDYQmHw2I0GuX69esSDoeVPG7fvi0ZGRnS0dEhw8PDcvz4cbFarRKJRJTbOHv2rOTk\n5KimYCouLpZYLKbUHDx4UJxOp/T19Ulvb68UFhZKRUWFsj4ajYrT6ZR9+/YpUzBZrVapr6//78JY\nRbW1tWIymSQYDMr79++Vy/e5MduVu3TpkvT09Mjk5KQMDQ3J5cuXRavVSldXl4gw09W0Z88eOXfu\nnPIzs125hoYGefr0qUxMTMjz58/l0KFDkpGRwe1rgvr7+0Wn04nP55Px8XFpb2+XjIwM8fv9Sg2z\njU8sFpP8/PyfZg8UYabxqqmpkZycHOns7JTJyUnp6OiQzMxMuXjxolKzHrJN2sZeRMTv94vNZpO0\ntDQpKSmRnp6etX5Ia6a7u1s0Go1oNBrRarXK9VOnTik1V69eFYvFIgaDQfbu3SsjIyOq21hcXJTz\n58+L2WwWo9EoFRUVqqmURL7NO1xdXS0mk0lMJpOcOHFC5ufnVTVTU1Ny+PBhMRqNYjabxePxyJcv\nX/7ek/+Lfsxz6XLt2jVVHbNdmZMnT8q2bdskLS1NtmzZIvv371ea+iXMdHV8P93lEma7MpWVlZKd\nnS16vV6sVqscPXpUNde6CDONV2dnpxQVFYnBYJCCggK5e/fuTzXMduWCwaBotVrp7+//5XpmunKR\nSEQaGhrEZrNJenq65OXlSVNTkywuLqrqkj1bjUgKHflARERERPQ/lZT72BMRERER0cqwsSciIiIi\nSgFs7ImIiIiIUgAbeyIiIiKiFMDGnoiIiIgoBbCxJyIiIiJKAWzsiYiIiIhSABt7IiIiIqIUwMae\niIiIiCgF/AM1hIn1h8z1WwAAAABJRU5ErkJggg==\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "data = ReadData(path='../code')\n", - "cols = zip(*data)\n", - "price1, price2, bid1, bid2, diff1, diff2 = cols\n", - "\n", - "pdf = thinkbayes.EstimatedPdf(price1)\n", - "low, high = 0, 75000\n", - "n = 101\n", - "xs = numpy.linspace(low, high, n) \n", - "pdf.kde.evaluate([3, 3])\n", - "\n", - "pmf = pdf.MakePmf(xs)\n", - "thinkplot.Pmfs([pmf])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`pdf` is a `Pdf` object, estimated by KDE.\n", - "`pmf` is a Pmf object that approximates the Pdf by evaluating\n", - "the density at a sequence of equally spaced values.\n", - "\n", - "`linspace` stands for “linear space.” It takes a range,\n", - "`low` and `high`, and the number of points,\n", - "`n`, and returns a new `numpy` array with\n", - "`n` elements equally spaced between `low` and\n", - "`high`, including both.\n", - "\n", - "And now back to *The Price is Right*." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Modeling the contestants\n", - "\n", - "The PDFs in Figure [fig.price1] estimate the distribution of possible\n", - "prices. If you were a contestant on the show, you could use this\n", - "distribution to quantify your prior belief about the price of each\n", - "showcase (before you see the prizes).\n", - "\n", - "To update these priors, we have to answer these questions:\n", - "\n", - "1. What data should we consider and how should we quantify it?\n", - "\n", - "2. Can we compute a likelihood function; that is, for each hypothetical\n", - " value of `price`, can we compute the conditional\n", - " likelihood of the data?\n", - "\n", - "To answer these questions, I am going to model the contestant as a\n", - "price-guessing instrument with known error characteristics. In other\n", - "words, when the contestant sees the prizes, he or she guesses the price\n", - "of each prize—ideally without taking into consideration the fact that\n", - "the prize is part of a showcase—and adds up the prices. Let’s call this\n", - "total `guess`.\n", - "\n", - "Under this model, the question we have to answer is, “If the actual\n", - "price is `price`, what is the likelihood that the\n", - "contestant’s estimate would be `guess`?”\n", - "\n", - "Or if we define\n", - "\n", - "```python\n", - "error = price - guess\n", - "```\n", - "then we could ask, “What is the likelihood that the contestant’s\n", - "estimate is off by `error`?”\n", - "\n", - "To answer this question, we can use the historical data again.\n", - "Figure [fig.price2] shows the cumulative distribution of\n", - "`diff`, the difference between the contestant’s bid and the\n", - "actual price of the showcase.\n", - "\n", - "The definition of diff is\n", - "\n", - "```python\n", - " diff = price - bid\n", - "```\n", - "\n", - "When `diff` is negative, the bid is too high. As an aside, we\n", - "can use this distribution to compute the probability that the\n", - "contestants overbid: the first contestant overbids 25% of the time; the\n", - "second contestant overbids 29% of the time.\n", - "\n", - "We can also see that the bids are biased; that is, they are more likely\n", - "to be too low than too high. And that makes sense, given the rules of\n", - "the game.\n", - "\n", - "Finally, we can use this distribution to estimate the reliability of the\n", - "contestants’ guesses. This step is a little tricky because we don’t\n", - "actually know the contestant’s guesses; we only know what they bid.\n", - "\n", - "So we’ll have to make some assumptions. Specifically, I assume that the\n", - "distribution of `error` is Gaussian with mean 0 and the same\n", - "variance as `diff`.\n", - "\n", - "The `Player` class implements this model:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```python\n", - "class Player(object):\n", - "\n", - " def __init__(self, prices, bids, diffs):\n", - " self.pdf_price = thinkbayes.EstimatedPdf(prices)\n", - " self.cdf_diff = thinkbayes.MakeCdfFromList(diffs)\n", - "\n", - " mu = 0\n", - " sigma = numpy.std(diffs)\n", - " self.pdf_error = thinkbayes.GaussianPdf(mu, sigma)\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`prices` is a sequence of showcase prices, `bids`\n", - "is a sequence of bids, and `diffs` is a sequence of diffs,\n", - "where again `diff = price - bid`.\n", - "\n", - "`pdf_price` is the smoothed PDF of prices, estimated by KDE. `cdf_diff`\n", - "is the cumulative distribution of `diff`, which we saw in\n", - "Figure [fig.price2]. And `pdf_error` is the PDF that characterizes the\n", - "distribution of errors; where `error = price - guess`." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu8AAAEWCAYAAADW9nkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X901NWd//FXJiGZBAiQQEJCEEFZUKBgiRZSFLSaLVaR\nIgKlolBaFlpBpJ66rHRVwF91l2N1BcVaZKus8WCWbGHWAqsFEdqCEAuBpGj8IpTMSBISBBIwmfv9\nI84wk0ySGTKTmUmej3NyZD5zP8yd60Beubw/70+MMcYIAAAAQMSzhHsCAAAAAPxDeAcAAACiBOEd\nAAAAiBKEdwAAACBKEN4BAACAKEF4BwAAAKIE4R0AAACIEn6H9507d2rSpEnKysqSxWLR+vXrWz3n\n4MGDGj9+vJKSkpSVlaUVK1a0abIAAABAZ+Z3eD937py+8Y1v6Ne//rUSExMVExPT4vgzZ87otttu\nU0ZGhvbt26df//rXeu6557Rq1ao2TxoAAADojGIu5w6r3bt310svvaT77ruv2TFr1qzR0qVL5XA4\nlJCQIEl68skntWbNGp04ceLyZwwAAAB0UiGred+zZ49uvPFGd3CXpNzcXJ08eVLHjh0L1csCAAAA\nHVbIwrvdbld6errXMddju90eqpcFAAAAOqy4UP3GrdXEe6qurg7VNAAAAICQ69GjR7u8Tsh23vv2\n7dtkh93hcLifAwAAABCYkIX3sWPH6oMPPtCFCxfcx7Zt26Z+/fppwIABoXpZAAAAoMPyu2zm3Llz\nOnr0qCTJ6XTq2LFjKiwsVGpqqvr376+lS5dq79692r59uyRp5syZeuKJJzR79mwtW7ZMJSUlevbZ\nZ/X444+3+Drt9U8OncG+ffskSdnZ2WGeScfCuoYG6xo6rG1otLauBbYS5W0qUm1tnSSpIsWqU30S\n5bT4X1baGTmdTkmSxeL//qLFEqNePa3q2cMaqmlFvfPnzkuSkromheX1E+NiNXN4hqYMTW99cBQJ\nR+m33+F97969uuWWWyQ11LM/9thjeuyxxzR79mz99re/ld1uV2lpqXt8cnKytm3bpp/97GfKzs5W\nSkqKHn74YT300EPBfxcAAARBfrFDGw6VqaauvtkxVdW1Ol1Vq7qvx1i2bW/+NxyY7NfrWpxGQ0tO\ny2qN0/TJw3TX7UMCmndHwg+boXFpXUeHeSZoK7/D+4QJE9w/Dfuybt26JseGDx+uHTt2XN7MAAAI\nkD/hOxCuoO50BnxLlCaa2x3uqDuSAEIjZN1mAABoT/nFDr1W2LabAAYzrEtNAztBHUBbEd4BAFGl\nrbvrrQV0i9Mo/VSNUitrm/09rNY43TAySeO+1ZvyDgDtivAOAIgq/gT3uaOyfO5uF9hKtN72qQLZ\n926uDt1VQwwA7YnwDgCIOJe7u+6rLKVx15fWcNEogEhGeAcARIRAA3tiXKw2Th3lfuwK6W/8z9/0\nhh/n3z9jJAEdQNQhvAMAwqatO+yB7qpL7KwDiG6EdwBAuwkkrDfXmcVzh91fBHYAHQXhHQAQcv6G\n9tZaKRbYSrT+rY+bPZ+QDqCjI7wDAELGn9DuT+/zlspjCOwAOhPCOwAgZHwFd39vVNRaPTsXnALo\njAjvAICQ8QzuLYX2QC48ZacdQGdGeAcABE1LZTKebR0l+q8DwOUgvAMA2qy12vbEuFivx61deOpC\nYAcAb4R3AECbtRbch9ZLM+flt7jLTlAHgNYR3gEAl6W53XZXbXtsaZW7LOZwC78PF54CgP8s4Z4A\nACD65Bc79FrhCZ/BfePUUYotrdL6tz5udaed4A4AgWHnHQAQEFdwb8y14y5JeZuKmjxPWQwAtB3h\nHQDgl+bKZK6LiVXp/5WqtrZOb/zP3/RGo/PYXQeA4CG8AwBa1dxu+3UxsTq85W/Nnme1xhHcASCI\nCO8AgGa1dFHq0Hq1GtynTx4W6ikCQKdCeAcAeGmtZ/vcUVnuC1I9UR4DAKFHeAcAeGkuuLt22zeu\n+rBJFxmCOwC0D8I7AMBL4+B+/swFxR6tVILjvM9+7QR3AGg/hHcA6KRaK4+pqq7VgI8cSmimVzut\nHwGg/RHeAaATaq57jNQQ2k9X1Up1TqX7CO6EdgAIH8I7AHQi/uy2V1TWyOI06nOqxn2cwA4AkYHw\nDgCdRHO77XNHZWnK0HRJ0sx5+V677YR2AIgshHcA6CQ2HCrzepwYF6uZwzPcwb3AVuLVRYYLUQEg\n8hDeAaADyy92aO2hGl2ol5K6JrmPu3bbC2wlmumj9SN3RgWAyER4B4AObMOhMl1oVN5+/swFbVz1\nod5opouMJO6MCgARivAOAB2Y54WpVdW1qq6sUarjfJOddhdq3AEgshHeAaCDyi92SJLOnqvTmbNf\naVhxldJ9jCOwA0D0ILwDQAfk2VnmzNmvpDrj9TyBHQCiE+EdADqQxn3cq6prZYyUUnZGEqEdAKId\n4R0AOpDGN2A6XVWrlBPV6vHFOVlTk7Vh7ZQwzg4A0FaEdwDoIPKLHV7B/fyZC+pTdk7WL85JooMM\nAHQEhHcA6CA8b8KUGBerHvvsqq2t0zlJCfEWSmUAoAOwBHrC6tWrNXDgQCUmJio7O1u7du1qcbzN\nZtOYMWOUnJysPn36aPLkyTp69OhlTxgA0JTnrntVda2q/nzCqx3kLePSwjU1AEAQBRTe8/LytHjx\nYi1btkyFhYXKycnRxIkTdfz4cZ/jP/nkE02ePFkTJkxQYWGhtm/frtraWt1+++1BmTwAoIHnrnt1\nZY2SHefdjxPiLRr3rd7hmBYAIMgCCu+rVq3SnDlzNHfuXA0ZMkQvvPCCMjIytGbNGp/jCwsL5XQ6\n9fTTT2vQoEEaOXKkHnnkEX366aeqrKwMyhsAgM7OteteVV2rz45VKdUjuFutcey6A0AH4nd4v3jx\novbv36/c3Fyv47m5udq9e7fPc7797W+rW7duevXVV1VfX68vv/xSr7/+um644QalpKS0beYAAOUX\nO/TvOz7RZ8eqVFFZI9U5lVpZK6khuG9YO4VddwDoQPwO7+Xl5aqvr1d6uvf9+dLS0mS3232ek5GR\nIZvNpmXLlslqtapnz54qKirS73//+7bNGgDgDu4VlTVyOhtuwtTnVI2kS/3cAQAdS4wxxrQ+TDp5\n8qSysrK0c+dOjRs3zn18+fLl2rBhg4qLi5ucU1paqjFjxmjOnDmaOXOmzpw5o3/913+VJL333nuK\niYmRJFVXV7vP4WJWAGjdrj+X65XTMar/+u9RSUo5Ua20qhrdMi6N3XYAaAeDBw92/7pHjx7t8pp+\nt4rs3bu3YmNj5XA4vI47HA5lZGT4POeVV15R//799eyzz7qPvfHGG+rfv7/27NmjnJycy5w2AHRO\nu/5crvd2faEveiaqPuvSN4qUE9WaMby7xn1rYBhnBwAINb/De3x8vEaPHq2tW7fq7rvvdh/ftm2b\n7rnnHp/nGGNksXhX5rgeO51On+dkZ2f7OyW0Yt++fZJY02BjXUODdW1dga1EO/9cqrguVp3J6imL\npWHXPU7S0ju/0Wwfd9Y2NFjX0GBdQ4N1DQ3P6pH2ElC3mSVLluj111/Xa6+9piNHjujBBx+U3W7X\n/PnzJUlLly7Vrbfe6h4/adIk7d+/XytWrNDRo0e1f/9+zZkzR1dccYVGjx4d3HcCAB1c3qYiSVJF\nilVOS4wslhilpiRq+fdHcAMmAOgkArrD6rRp01RRUaGVK1eqrKxMI0aMkM1mU//+/SVJdrtdpaWl\n7vHjxo1TXl6ennnmGf3qV79SUlKSxo4dq3fffVeJiYnBfScA0IEV2Er096Q4nRrQXc6vQ3vPHlYl\nxsVqytD01n8DAECHEFB4l6QFCxZowYIFPp9bt25dk2NTp07V1KlTA58ZAEBSQ3Bf/9bHOjWkl3vH\nvWcPqyRp5nDf1xwBADqmgMM7AKB9ucplnF/XuPfq2bDjPnN4BrvuANDJEN4BIIIV2EpUW1unipSG\nnXZXuczGqaPCPDMAQDgEdMEqAKD9uMplJOlUn0R3uUxiXGyYZwYACBfCOwBEqMbdZXr1pM4dADo7\nwjsARChXuYwjPYnuMgAASdS8A0BEWpZXqOKvu8tIorsMAEAS4R0AIs6yvEJtLC2Xvg7urjupzh2V\nxa47AHRyhHcAiBD5xQ69uPNTlVWc9zqe3iuR4A4AkER4B4Cwyy92aMOhMtXU1ctxusbruamDemvl\ndNpCAgAacMEqAISZK7hXVdfK6TSSJIvTENwBAE0Q3gEgjPKLHaqpq5ckna6qlcVplO44r1HHviS4\nAwCaoGwGAMIkv9ih1wpPSJKqqmulOqeGlpyWJE2fMTKcUwMARCjCOwC0M88ad6khuFdU1ij9VEO9\nu9Uap7tuHxLOKQIAIhRlMwDQzjyDu9RQLpPuOK/UylpJ0vTJw8I1NQBAhGPnHQDameeOe3Vljfp4\nBPf7Z4xk1x0A0CzCOwC0s6rqWp2uaugsc+2RSvdxymUAAK0hvANAOyqwlaiisqbJcas1jnIZAECr\nCO8A0I7yNhVJA5Pdj12hnR13AIA/CO8A0A6W5RVqU2m56jyCe2pKojYsvTWMswIARBu6zQBAiBXY\nSpT/ySnVeRyzWGKUkdo1bHMCAEQnwjsAhNiL7x+V0xLjfmyxxCi9V6JmDs8I46wAANGIshkACLGT\nPRPcv85ITdL783LCOBsAQDRj5x0AQsxz133hTVeFcSYAgGjHzjsAhEiBrUQvvn9USrG6j00Zmh7G\nGQEAoh3hHQCCrMBWorxNRfp7Upwc6Unu4/yFCwBoK76XAECQvfj+UZ0c0L3JRaqTr0wN46wAAB0B\n4R0AgqjAVqKTPRPcwd1iiVGvnlb9fPzVlMwAANqMC1YBIEgKbCVa/9bHXsH92qtSCe4AgKBh5x0A\ngiRvU5EqPC5O7dXTqo1TR4VxRgCAjoaddwAIktraOp3qkyhJSk1J5A6qAICgI7wDQBC5SmZ69rBy\nB1UAQNAR3gEgCApsJV4lMxI93QEAwUd4B4AgePH9o+6e7hZLjBLjYsM8IwBAR0R4B4A2crWHdOnV\nk5IZAEBoEN4BoA3yix169OPjXu0haQ0JAAgVWkUCwGXIL3Zow6EylVWcU53H8fReiQR3AEDIsPMO\nAJdhw6Ey1dTV63RVrftYnKSFN10VvkkBADq8gMP76tWrNXDgQCUmJio7O1u7du1q9Zznn39eQ4cO\nldVqVWZmppYuXXpZkwWASJBf7FBNXb0kyek0sjiN0h3n9eTI/uy6AwBCKqCymby8PC1evFhr1qzR\nuHHj9NJLL2nixIk6fPiw+vfv7/OcJUuWaMuWLfq3f/s3jRgxQtXV1SorKwvK5AGgveUXO/Ra4QlJ\nUlV1rSxOo6ElpyVJd90+JJxTAwB0AgGF91WrVmnOnDmaO3euJOmFF17Qu+++qzVr1uipp55qMr6k\npET/8R//oYMHD2rIkEvf1EaOHNnGaQNA+2sc3Csqa5R+qkaSZLVyCREAIPT8Lpu5ePGi9u/fr9zc\nXK/jubm52r17t89zCgoKNGjQINlsNg0aNEgDBw7U7NmzderUqbbNGgDCYMOhS/9qeLqqVumO80qt\nbKh5nz55WLimBQDoRPzeKiovL1d9fb3S073rOdPS0mS3232eU1paqmPHjuntt9/Wf/7nf0qSHn74\nYd15553as2ePYmJimpyzb9++QOYPP7CmocG6hkYkr2tFdcMu+9lzdUr+f5WyfnFO5yRNvKWv+qV9\nGdFzlyJ7baMZ6xoarGtosK7BNXjw4HZ/zZD+O6/T6dSFCxf0u9/9TldffbUk6Xe/+52GDBmiffv2\n6frrrw/lywNASJw5+5Wu/OKcJCkh3qJx3+od5hkBADoLv8N77969FRsbK4fD4XXc4XAoI8P3nQQz\nMjIUFxfnDu6SdPXVVys2Nlaff/65z/CenZ3t75TQCtdP16xpcLGuoRHp65pf7FBS1xOqqq5VTIxF\nXbsmSZLunzFS2dmRfaFqpK9ttGJdQ4N1DQ3WNTSqq6vb/TX9rnmPj4/X6NGjtXXrVq/j27ZtU05O\njs9zxo0bp7q6OpWWlrqPlZaWqr6+XgMGDLjMKQNA+8kvdmjqxkL3haqnqxo6zEgNF6nSYQYA0J4C\n6vO+ZMkSvf7663rttdd05MgRPfjgg7Lb7Zo/f74kaenSpbr11lvd42+99VZ985vf1I9+9CMVFhbq\nwIED+tGPfqQxY8bwkx+AqOC6GZPU0GHG6TTq83WHGS5SBQC0t4Bq3qdNm6aKigqtXLlSZWVlGjFi\nhGw2m7vHu91u99plj4mJ0ebNm7Vo0SLddNNNSkxMVG5urlatWhXcdwEAIeB5MyZJqq6scXeYYdcd\nABAOAV+wumDBAi1YsMDnc+vWrWtyrG/fvnr77bcDnxkAhJlna8jzZy7oH45Uuh+z6w4ACIeAymYA\noLPw3HWvqq6V83C5+zl23QEA4UJ4BwAfPHfdqytr3Ddjkth1BwCED+EdABppXOue6jjv/vX9M0ay\n6w4ACJuQ3qQJAKKR5657YlysEjx23QnuAIBwYucdADw03nWfOdz3TegAAAgHwjsAfC2/2OG+GZPU\nsOseW1oVxhkBAOCN8A4AX/Msl5Eadt3zNhW5H1utVBoCAMKL8A4AX/Msl5k7KkuxpVWqra1zH6PL\nDAAg3AjvAKCGkhlPU4amN9l152JVAEC4Ed4BdHq+at0lsesOAIg4hHcAnZ6vWvfG2HUHAEQCwjuA\nTq1xa8i5o7I0ZWh6GGcEAEDzaJ0AoFNrfEOmKUPTVWAr8ap3BwAgUrDzDqDTau6GTHmbirzq3WkR\nCQCIFIR3AJ2Wr113SU2COxerAgAiBdtJADotX7vuBbYSrzEb1k5p1zkBANASdt4BdEq++rpL4o6q\nAICIRngH0Ok019ddorc7ACCyEd4BdCqNg7vUfMkMvd0BAJGG8A6gU2l8QybPvu6UzAAAIh3hHUCn\n0tINmSiZAQBEOsI7gE6juYtUfaFkBgAQiQjvADqFli5SlZrWuwMAEIko6gTQ4bV2kSp3VAUARAt2\n3gF0eK1dpOoZ3CXq3QEAkYvtJQAdXnMXqRbYSprsuE+fPIx6dwBAxCK8A+hUPC9SbdwacsPaKeGY\nEgAAfqNsBkCnRWtIAEC0IbwD6NAat4dsDqUyAIBoQHgH0KF5XqzauD0kAADRhpp3AB1WfrHD62LV\nxu0hAQCINoR3AB1W41332NIqzVz1YZPWkPR1BwBEC8pmAHRIvnbdffV0d7WHBAAgGrDdBKBDarzr\nPmVout6gpzsAIMoR3gF0SJ677kPrpZnz8r2ep6c7ACAaUTYDoMMr/b/SJndSBQAgGvEdDECHkl/s\n8CqZkdQkuFPjDgCIVgHtvK9evVoDBw5UYmKisrOztWvXLr/OO3r0qLp3767u3btf1iQBwF8bDpV5\nlcycP3PB+/m1U6hzBwBELb/De15enhYvXqxly5apsLBQOTk5mjhxoo4fP97ieRcvXtSMGTM0fvx4\nxcTEtHnCANCcxh1mzp+5IOfhcvdjymUAANHO7/C+atUqzZkzR3PnztWQIUP0wgsvKCMjQ2vWrGnx\nvEceeUSjRo3SPffcI2NMmycMAL7kFzv0WuEJ9+PEuFhl7LMrtbLWfYxyGQBAtPMrvF+8eFH79+9X\nbm6u1/Hc3Fzt3r272fO2bNmiLVu26MUXXyS4AwipxnXuM4dneNW63z9jJOUyAICo59e/IZeXl6u+\nvl7p6elex9PS0mS3232ec/LkSc2bN0+bNm1SUlKS3xPat2+f32PhH9Y0NFjX0Ah0XXecqtM2x1e6\ncKlaRndkdtEVZ4/r3Lnz7mP90r7s9P/POvv7DxXWNTRY19BgXYNr8ODB7f6aIWsVOWvWLC1YsEDX\nX399qF4CAJoE94RYaXwfatsBAB2TX9/hevfurdjYWDkcDq/jDodDGRkZPs95//33tXPnTj3xxBOS\nJGOMnE6nunTpojVr1ujHP/6xz/Oys7MDmT9a4PrpmjUNLtY1NC53XWM/+Uiuf9tLjIvVzOEZyh7a\n8K+EXbuWusd15v9ffGZDg3UNDdY1NFjX0Kiurm731/QrvMfHx2v06NHaunWr7r77bvfxbdu26Z57\n7vF5zqFDh7web9q0SU8++aT27t2rzMzMNkwZABrkF3tvKGycOsr96wJbSXtPBwCAkPP735aXLFmi\nWbNm6YYbblBOTo5efvll2e12zZ8/X5K0dOlS7d27V9u3b5ckXXvttV7n/+Uvf5HFYmlyHAAul+dF\nqolxsV7P5W0qcv+aFpEAgI7C7+9o06ZNU0VFhVauXKmysjKNGDFCNptN/fv3lyTZ7XaVlpa2+HvQ\n5x1AMHn2dJ853LuEz7PTDC0iAQAdRUDbUQsWLNCCBQt8Prdu3boWz509e7Zmz54dyMsBQLMal8xM\nGXqpG1bjkhlaRAIAOoqQdZsBgFCiZAYA0BkR3gFEnfxiByUzAIBOiS0pAFGn8a67q2SmwFbitesu\nUTIDAOhYCO8AokZ+sUMbDpX53HUvsJVo/Vsfe42nZAYA0NFQNgMgajQO7p677o133K3WOEpmAAAd\nDttSAKJC4zp3191UpYZdd8869/tnjKRcBgDQIRHeAUS8/GKHXis84X6cGBerjVNHqcBWopmrPvQK\n7lZrHMEdANBhEd4BRDzPC1Slhjp3XzXuEt1lAAAdGzXvACKeZ7nM3FFZmjI03WeNO+UyAICOjp13\nABHL1V3Gk+sCVWrcAQCdETvvACKWr+4yvhDcAQCdBeEdQERqqbsMAACdFWUzACJS47uobpw6SpLv\nu6gCANBZEN4BRCTuogoAQFOUzQCIOPnFDq/HU4amNxvcaQ0JAOhM2LICEHEal8z4Cu50mAEAdEbs\nvAOIKI0vVJ05PKNJjTvBHQDQWRHeAUSUxrvuU4am09MdAICvUTYDIGLsOFWnmroY92NfrSEJ7gCA\nzoyddwARYcepOm0++ZX7cWJcrGJLqzRzXn4YZwUAQGQhvAOICNscX3k9dtW6e5bM0BYSANDZEd4B\nhF1+sUMXLl2jqrmjsprUutMWEgAAat4BhFl+sUOvFZ5wP3ZdpFpgK/Eat2HtlPaeGgAAEYeddwBh\n5dldRrp0kapne0jKZQAAaEB4BxA2jXu635HZxb3r7lkyQ7kMAAANCO8AwsZz1z0hVhrfp2GHvfGu\nO+0hAQBoQHgHEDaeu+63pXdx/5pddwAAfCO8A4gIrl33xth1BwDgEq4CA9Du8osdTS5UlaRdfy7X\nqrXclAkAgOaw8w6g3W04VOZVMpMYFytJem/XF9yUCQCAFhDeAbS7xsF95vAM7fpzuS5cdLqPc1Mm\nAACaYlsLQLvKL3Z4Pd44dZQkadITX7iPWa1x3JQJAAAf2HkH0K48a91d5TIFthKvXXd23AEA8I3w\nDqBdeZbMNHc3VTrMAADgG+EdQLtpXDLD3VQBAAgMNe8AQs7VGtJz1/38mQuaOS/fK7gnxFvYdQcA\noAUB77yvXr1aAwcOVGJiorKzs7Vr165mx/7xj3/UXXfdpczMTHXt2lUjR47UunXr2jRhANGncXCX\npNijlV7BXZJuGZfWntMCACDqBBTe8/LytHjxYi1btkyFhYXKycnRxIkTdfz4cZ/j9+zZo5EjR+qd\nd95RUVGRFixYoHnz5um//uu/gjJ5ANGhcWvIuaOylOw47z5mtcZp4i19Ne5bvcMxPQAAokZA4X3V\nqlWaM2eO5s6dqyFDhuiFF15QRkaG1qxZ43P80qVLtXz5co0dO1ZXXnml5s+frylTpuidd94JyuQB\nRD5frSFjS6u8jm1YO4XgDgCAH/wO7xcvXtT+/fuVm5vrdTw3N1e7d+/2+wWrq6uVkpLi/wwBRK38\nYodeKzzhfuxqDdm4uwwAAPCP3981y8vLVV9fr/T0dK/jaWlpstvtfv0emzdv1nvvvRdQ2AcQnRoH\nd6mhNSTdZQAAuHzttuX14Ycf6oc//KFefPFFZWdnNztu37597TWlToM1DQ3WtXk7TtVp88mvvI7d\nkdlFV5w9ruW/O+y+IVNCvEX90r70WkvWNXRY29BgXUODdQ0N1jW4Bg8e3O6v6XfZTO/evRUbGyuH\nw7t+1eFwKCMjo8Vzd+3apdtvv10rVqzQP/3TP13eTAFEheaC+/g+DXsFnndSpbsMAACB8XvnPT4+\nXqNHj9bWrVt19913u49v27ZN99xzT7Pn7dy5U3fccYeWL1+uRYsWtfo6Le3KIzCun65Z0+BiXVv2\nzMZCJXXt4n48d1SWpgy9VG7XtWup+9eLf/Zd969Z19BhbUODdQ0N1jU0WNfQqK6ubvfXDKhsZsmS\nJZo1a5ZuuOEG5eTk6OWXX5bdbtf8+fMlNXSX2bt3r7Zv3y6poc/79773PT3wwAP6wQ9+4K6Nj42N\nVZ8+fYL8VgCEW36xw6stZOPgXmArCce0AADoMAIK79OmTVNFRYVWrlypsrIyjRgxQjabTf3795ck\n2e12lZZe2lVbv369amtr9dxzz+m5555zH7/yyiu9xgGIfr46y7iCe4GtRHmbirwuVKXLDAAAgQv4\nu+eCBQu0YMECn881vnvqunXruKMq0ElsOFTm9Xjm8EvXwjQO7hJdZgAAuBxsfQEIiubKZRq3hrRa\n4zR98jDddfuQdp8jAADRjvAOoE3yix1Ndt2nDE1vtlRmw9op7T1FAAA6DMI7gDbZcKjMa9c9MS5W\nBbYSrX/r4yZjKZUBAKBt/O7zDgCNNe4ukxgXq5nDM5S3qchrnNUap/tnjKRUBgCANmLnHUBAXGUy\nnqFdagjus5ISlbfqQ69SGUI7AADBQ3gHEBBfwV2ShtarSamM1RpHcAcAIIgI7wAC0ji4nz9zQbFH\nK3XYcd7ruKurDAAACB7COwC/5Rc7vB5vmTFaM+flN+nhTqkMAAChQXgH4DfPlpCurjL0cAcAoP0Q\n3gG0qLkLVBvXuNPDHQCA0CO8A2hR4+BeVV2r6soaJRyp9BpHfTsAAKFHeAfQrMZ93Kuqa3W6/Lz6\nnKrxGkeNOwAA7YPwDsCn/GKHXis84X58/swFpX/4d6V7jKHGHQCA9kV4B+CT58WpVdW1ijtS4fU8\nu+0AALSzIp89AAARp0lEQVQ/wjsAN8+LU6uqa3W6qlZOp1G647xSK2vd4wjuAACEB+EdgApsJXrx\n/aM6kWJt8pzFaQjuAABECEu4JwAgvApsJVr/1sc62TOhyXMWp3FfnGq1xhHcAQAIM3begU5qWV6h\nNpWWq06Srknxes5VJsMFqQAARBbCO9BJuYO7h9SURGWkdtXGpaPCMicAANAywjvQCeUXO7yCu8US\no149rcpI7aqZwzPCNi8AANAywjvQSeQXO/Tizk/lOF0jp9O4j1ucRoeX3hrGmQEAAH9xwSrQCbhu\nuNQ4uEtSZtWFMM0KAAAEip13oINzBfeq6tomO+6ZVRe08ObBYZwdAAAIBOEd6MCW5RUq//9VeIX2\ndMd59Ttfpw1rp4RxZgAA4HIQ3oEOqLmbLrlaQE6fMTJMMwMAAG1BeAeiVIGtRHmbilRb27jho1SR\nYpUjPcnrmGvHfTo3WgIAIGoR3oEo5Lorqi+Ng7vFEqMpV6ZqJb3bAQCIeoR3IEK1tLPuS0WKVaf6\nJMppiZF0qXf7z8dfrSlD00M5VQAA0E4I70CE8je43z9jpOoH9dRrhSfUrdFzc0dlEdwBAOhA6PMO\nRKACW0mrwd1qjdP9X9evbzhU5vVcYlwswR0AgA6InXcggvgqlbFa45pt65hf7NDUjYWqqat3HyO0\nAwDQcRHegQjQUn379MnDJDUE9Q2HyryCemOJcbEEdwAAOjDCOxABfAV3qzVO0ycPU/2gnk12131J\njIvVzOEZoZwmAAAIM8I7EEL+doypSLHq1IDuclpi3F1ievawau2Zs1Lh2RbPdYV2dtwBAOj4CO9A\nCATS5tGzL7vFEqOBA3o2O5agDgBA50Z4B9og0F7sjTUO7r16Wn2OI7QDAACJ8A60yFc4P3fuvCSp\na9fSVs+vSLGqIj1JPVIS1bNH02De7esvFzrFAACAlgTc53316tUaOHCgEhMTlZ2drV27drU4/uDB\ngxo/frySkpKUlZWlFStWXPZkgfZUYCvR+rc+9rv0pXhILx2+JsX9VTwsVXXXpGrAwF4+g3tjBHcA\nANCagHbe8/LytHjxYq1Zs0bjxo3TSy+9pIkTJ+rw4cPq379/k/FnzpzRbbfdpgkTJmjfvn06cuSI\n5syZo65du2rJkiVBexNAKORtKmpyrCLFKvvVPWRiY2SxNPzs6yp3GehHQPeFkhgAAOCvgML7qlWr\nNGfOHM2dO1eS9MILL+jdd9/VmjVr9NRTTzUZ/+abb6q2tlbr169XQkKCrr32WhUXF2vVqlWEd4RN\na3XqFSlWneqTKOfAZPex1K/LXrpJyvy6bCapa5Jfr0c4BwAAweJ3eL948aL279+vX/ziF17Hc3Nz\ntXv3bp/n7NmzRzfeeKMSEhK8xv/yl7/UsWPHNGDAgMucNjqjQDu4nOqTKKclxvcAj2DeGoslxq+y\nF0I6AAAINb/De3l5uerr65We7h1M0tLSZLfbfZ5jt9t1xRVXeB1znW+32yM2vPtzJ8tocP5cjSQp\n6ZOPwjyTtqmqrtXpqlo5nSag0B0MvjrAJMRKt6V30c8njm7XuQAAAIS020xMTDO7ni2orq4OwUwC\n850Mq76TMTDc00CEi4TPakcyePBgSaxrKLC2ocG6hgbrGhqsa8fhd7eZ3r17KzY2Vg6Hw+u4w+FQ\nRobvW7L37du3ya686/y+ffsGOlcAAACgU/M7vMfHx2v06NHaunWr1/Ft27YpJyfH5zljx47VBx98\noAsXLniN79evX8SWzAAAAACRKsYYY/wd/Pbbb2vWrFlavXq1cnJy9PLLL2vdunUqKipS//79tXTp\nUu3du1fbt2+X1NAqcsiQIZowYYKWLVumkpISzZkzR48//rgeeuihkL0pAAAAoCMKqOZ92rRpqqio\n0MqVK1VWVqYRI0bIZrO5e7zb7XaVll6662RycrK2bdumn/3sZ8rOzlZKSooefvhhgjsAAABwGQLa\neQcAAAAQPn7XvLfmJz/5ia6++molJSUpLS1NkydP1pEjR7zGnD59WrNmzVLPnj3Vs2dP3XfffU2u\nev7888915513qlu3burTp48efPBBffXVV15jDh48qPHjxyspKUlZWVlasWJFk/ns2LFDo0ePVmJi\noq666iq98sorwXqr7eb06dNauHChrrnmGiUlJemKK67QT3/6U1VWVjYZx7oGZu3atbr55pvVs2dP\nWSwWff75503GsK7ta/Xq1Ro4cKASExOVnZ2tXbt2hXtKYbNz505NmjRJWVlZslgsWr9+fZMxjz/+\nuPr166ekpCTdfPPNOnz4sNfzFy5c0MKFC9WnTx9169ZNd911l/7+9797jQnWZzwaPP3007r++uvV\no0cPpaWladKkSSoqanoXZdY1cC+99JJGjhypHj16qEePHsrJyZHNZvMaw7q23dNPPy2LxaKFCxd6\nHWdtA/P444/LYrF4fWVmZjYZE9FraoLklVdeMbt27TLHjh0z+/fvN5MmTTKZmZnmq6++co/57ne/\na4YPH27+9Kc/mT179phhw4aZO++80/18XV2dGT58uLn55pvNgQMHzLZt20xmZqZZuHChe0x1dbVJ\nT08306dPN0VFRWbjxo2me/fu5t///d/dY0pLS01SUpJZtGiRKS4uNq+++qrp0qWLeeedd4L1dtvF\noUOHzJQpU8zvf/978+mnn5odO3aYYcOGmdzcXK9xrGvgnn/+efPMM8+Y559/3sTExJhjx441GcO6\ntp+33nrLdOnSxfzmN78xxcXFZuHChaZbt27m888/D/fUwsJms5lHH33UbNy40SQlJZn169d7Pf/M\nM8+Y7t27m/z8fHPo0CEzbdo0k5mZab788kv3mPnz55vMzEyzfft2s3//fjNhwgQzatQoU19f7x4T\njM94tPjHf/xH8/rrr5uioiJz8OBB8/3vf9/07dvXVFZWusewrpenoKDAvPvuu+bTTz81R48eNY8+\n+qjp0qWLKSwsNMawrsGwZ88eM3DgQDNy5Eiv98PaBu6xxx4z11xzjXE4HO6v8vJy9/PRsKZBC++N\nffzxxyYmJsb87W9/M8YYc/jwYRMTE2N2797tHrNr1y6vMTabzVgsFnPixAn3mDfeeMNYrVb3oq1e\nvdr06NHD1NbWusesXLnS9OvXz/34F7/4hfmHf/gHr/n8+Mc/NmPHjg3+G21nrjVyrQfr2jZ79+71\nGd5Z1/Z1ww03mHnz5nkdGzx4sFm6dGmYZhQ5unXr5hXenU6n6du3r3nqqafcx2pqakz37t3NK6+8\nYowxpqqqysTHx5sNGza4xxw/ftxYLBbzhz/8wRgTvM94tDp79qyJjY01mzdvNsawrsGWkpJi1q5d\ny7oGQVVVlbnqqqvMH//4RzNhwgR3uGNtL89jjz1mhg8f7vO5aFnToJXNeDp37pzWrVunwYMHa+DA\nhpsd7dmzR926ddPYsWPd43JyctS1a1ft3r3bPebaa69Vv3793GNyc3N14cIFffTRR+4xN954oxIS\nErzGnDx5UseOHXOPyc3N9ZpTbm6u9u3bp/r66L5ranV1tRISEpSUlCSJdQ0V1rX9XLx4Ufv37/e5\nBq61xiWfffaZHA6H13pZrVbddNNN7vX66KOP9NVXX3mNycrK0jXXXKM9e/ZICt5nPFqdOXNGTqdT\nvXr1ksS6Bkt9fb3eeust1dbW6qabbmJdg2DevHm65557NH78eBmPyxRZ28tXWlqqfv36adCgQfrB\nD36gzz77TFL0rGlQw/vq1avVvXt3de/eXZs3b9aWLVsUF9fQ0MZut6tPnz5e42NiYpSWlua+kZPd\nbld6errXGNfNoVoa43rsGuNwOHyOqaurU3l5eZDebfurqqrSL3/5S82bN08WS8P/OtY1NFjX9lNe\nXq76+voma+C51rjEtSYtrZfdbldsbKxSU1O9xqSnp3uNCcZnPFo9+OCDuu6669zfXFnXtjl48KC6\ndesmq9WqefPm6e2339aQIUNY1zZ69dVXVVpaqpUrV0ryvnM9a3t5xowZo/Xr1+sPf/iDXn31Vdnt\nduXk5KiysjJq1rTF8L5s2bImRf2Nv3bu3Okef++996qwsFA7duzQtddeq4kTJ+rLL79scQKNmVaa\n33h+cKNVoOsqSWfPntWdd96p/v3761e/+lXAr8m6+l7XtuoM64ro0drnrbXPa7DOiXRLlizR7t27\n9c477/j1Z5R1bd3QoUP117/+VX/5y1/0wAMPaMaMGdq3b1+L57CuLSspKdGjjz6qN998U7GxsZIa\n3p8/75G1bd53v/tdTZ06VcOHD9d3vvMdbdmyRU6n02djAE+RtKYthveHHnpIxcXFLX5df/317vHJ\nycm66qqrdOONN2rjxo0qKyvTf//3f0uS+vbtq1OnTjWZ9BdffKG+ffu6xzgcDq8xrl05zzGNfyJx\nndPamLi4OPXu3du/lQmhQNf17Nmzuv3222WxWLR582bFx8e7n2NdLwl0XVvCurYf105D47V0OBzK\nyMgI06wil+tz42u9PD9T9fX1qqioaHFMMD7j0eahhx5SXl6e3nvvPV155ZXu46xr23Tp0kWDBg3S\nddddp6eeekpjxozRSy+95P4zzLoGbs+ePSovL9ewYcPUpUsXdenSRTt37tTq1asVHx/v/v7A2rZN\nUlKShg0bpk8++SR6Pq8tVsS3QW1trUlKSjK//e1vjTG+i/c//PBDr+L9//3f/21SvP/mm296Fe+v\nWbPGJCcne10A+OSTT5qsrCz340ceeaTJBYA/+clPTE5OTvDfaIidOXPGfPvb3zbjxo0zZ8+ebfI8\n69o2gVywyrqGzre+9S2fF6z+y7/8S5hmFDl8XbCakZHR5IKq5ORks3btWmNMyxdUbd261RgTvM94\nNFm0aJHJyMgwxcXFTZ5jXYPr5ptvNvfdd58xxrCul6mqqsoUFRW5vw4dOmSuv/5688Mf/tAUFRXx\nmQ2Smpoa07dvX7NixQpjTHR8XoMS3j/55BPzzDPPmI8++sgcO3bMfPjhh+bOO+80KSkp5osvvnCP\nmzhxohkxYoTZs2eP2b17txk+fLiZNGmS+/n6+nozYsQIc8stt7jb5vTr188sWrTIPaa6utr07dvX\nzJgxwxw6dMi88847Jjk52axatco95rPPPjNdu3Y1ixcvNocPHzavvvqqiY+PN/n5+cF4u+3mzJkz\nZsyYMWbYsGHm6NGjpqyszP118eJF9zjWNXBlZWXmwIED5s033zQxMTHGZrOZAwcOeLWNY13bT15e\nnomPjze/+c1vzOHDh82iRYtM9+7dO22ryLNnz5oDBw6YAwcOmKSkJLN8+XJz4MAB93o8++yzpkeP\nHiY/P98cPHjQTJ8+3fTr18/rB/wFCxaYrKwsr1Zm1113nXE6ne4xwfiMR4uf/vSnJjk52bz33nte\nf5d6rhnrenkeeeQR88EHH5jPPvvM/PWvfzX//M//7BVkWNfgGT9+vHnggQfcj1nbwP385z83O3bs\nMKWlpeZPf/qT+d73vmd69OgRVX+/BiW8Hz9+3EycONGkpaWZ+Ph4079/f3PvvfeakpISr3GnT582\n9957r0lOTjbJyclm1qxZprq62mvM559/bu644w6TlJRkUlNTzYMPPugVVI0x5uDBg+amm24yVqvV\nZGZmmuXLlzeZ044dO8w3v/lNk5CQYAYNGuRu8RNN3n//fRMTE2MsFouJiYlxf1ksFrNjxw73ONY1\ncI899pjXerr+67nDybq2r9WrV5srr7zSJCQkmOzsbPPBBx+Ee0ph4/qz3/jP/5w5c9xjHn/8cZOR\nkWGsVquZMGGCKSoq8vo9Lly4YBYuXGhSU1NNUlKSmTRpktcOjzHB+4xHA19/l8bExJgnnnjCaxzr\nGrjZs2ebAQMGmISEBJOWlmZuu+02d3B3YV2Dw7NVpAtrG5gZM2aYzMxMEx8fb/r162emTp1qjhw5\n4jUm0tc0xpgOcgUCAAAA0MGFpM87AAAAgOAjvAMAAABRgvAOAAAARAnCOwAAABAlCO8AAABAlCC8\nAwAAAFGC8A4AAABECcI7AAAAECUI7wAAAECU+P++rJzJDw3WfgAAAABJRU5ErkJggg==\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "MakePrice2(player1, player2)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Again, we use the variance of `diff` to estimate the variance\n", - "of `error`. This estimate is not perfect because contestants’\n", - "bids are sometimes strategic; for example, if Player 2 thinks that\n", - "Player 1 has overbid, Player 2 might make a very low bid. In that case\n", - "`diff` does not reflect `error`. If this happens a\n", - "lot, the observed variance in `diff` might overestimate the\n", - "variance in `error`. Nevertheless, I think it is a reasonable\n", - "modeling decision.\n", - "\n", - "As an alternative, someone preparing to appear on the show could\n", - "estimate their own distribution of `error` by watching\n", - "previous shows and recording their guesses and the actual prices." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Likelihood\n", - "\n", - "Now we are ready to write the likelihood function. As usual, I define a\n", - "new class that extends `thinkbayes.Suite`:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class Price(thinkbayes.Suite):\n", - "\n", - " def __init__(self, pmf, player):\n", - " thinkbayes.Suite.__init__(self, pmf)\n", - " self.player = player\n", - " \n", - " def Likelihood(self, data, hypo):\n", - " price = hypo\n", - " guess = data\n", - "\n", - " error = price - guess\n", - " like = self.player.ErrorDensity(error)\n", - "\n", - " return like" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`pmf` represents the prior distribution and\n", - "`player` is a Player object as described in the previous\n", - "section. In `Likelihood` `hypo` is the hypothetical price of the showcase.\n", - "`data` is the contestant’s best guess at the price.\n", - "`error` is the difference, and `like` is the\n", - "likelihood of the data, given the hypothesis.\n", - "\n", - "`ErrorDensity` is defined in `Player`:\n", - "\n", - "```python\n", - "# class Player:\n", - "\n", - " def ErrorDensity(self, error):\n", - " return self.pdf_error.Density(error)\n", - "```\n", - "\n", - "`ErrorDensity` works by evaluating `pdf_error` at the given\n", - "value of `error`. The result is a probability density, so it\n", - "is not really a probability. But remember that `Likelihood`\n", - "doesn’t need to compute a probability; it only has to compute something\n", - "*proportional* to a probability. As long as the constant of\n", - "proportionality is the same for all likelihoods, it gets canceled out\n", - "when we normalize the posterior distribution.\n", - "\n", - "And therefore, a probability density is a perfectly good likelihood." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Update\n", - "\n", - "`Player` provides a method that takes the contestant’s guess\n", - "and computes the posterior distribution:\n", - "\n", - "```python\n", - "# class Player\n", - "\n", - " def MakeBeliefs(self, guess):\n", - " pmf = self.PmfPrice()\n", - " self.prior = Price(pmf, self)\n", - " self.posterior = self.prior.Copy()\n", - " self.posterior.Update(guess)\n", - "```\n", - "\n", - "`PmfPrice` generates a discrete approximation to the PDF of\n", - "price, which we use to construct the prior.\n", - "\n", - "`PmfPrice` uses `MakePmf`, which evaluates\n", - "`pdf_price` at a sequence of values:\n", - "\n", - "```python\n", - "# class Player\n", - "\n", - " n = 101\n", - " price_xs = numpy.linspace(0, 75000, n)\n", - "\n", - " def PmfPrice(self):\n", - " return self.pdf_price.MakePmf(self.price_xs)\n", - "```\n", - "\n", - "To construct the posterior, we make a copy of the prior and then invoke\n", - "`Update`, which invokes `Likelihood` for each\n", - "hypothesis, multiplies the priors by the likelihoods, and renormalizes.\n", - "\n", - "So let’s get back to the original scenario. Suppose you are Player 1 and\n", - "when you see your showcase, your best guess is that the total price of\n", - "the prizes is \\$20,000.\n", - "\n", - "Figure [fig.price3] shows prior and posterior beliefs about the actual\n", - "price. The posterior is shifted to the left because your guess is on the\n", - "low end of the prior range.\n", - "\n", - "On one level, this result makes sense. The most likely value in the\n", - "prior is \\$27,750, your best guess is \\$20,000, and the mean of the\n", - "posterior is somewhere in between: \\$25,096.\n", - "\n", - "On another level, you might find this result bizarre, because it\n", - "suggests that if you *think* the price is \\$20,000, then\n", - "you should *believe* the price is \\$24,000.\n", - "\n", - "To resolve this apparent paradox, remember that you are combining two\n", - "sources of information, historical data about past showcases and guesses\n", - "about the prizes you see.\n", - "\n", - "We are treating the historical data as the prior and updating it based\n", - "on your guesses, but we could equivalently use your guess as a prior and\n", - "update it based on historical data.\n", - "\n", - "If you think of it that way, maybe it is less surprising that the most\n", - "likely value in the posterior is not your original guess." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Optimal bidding\n", - "\n", - "Now that we have a posterior distribution, we can use it to compute the\n", - "optimal bid, which I define as the bid that maximizes expected return\n", - "(see ).\n", - "\n", - "I’m going to present the methods in this section top-down, which means I\n", - "will show you how they are used before I show you how they work. If you\n", - "see an unfamiliar method, don’t worry; the definition will be along\n", - "shortly.\n", - "\n", - "To compute optimal bids, I wrote a class called\n", - "`GainCalculator`:" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "class GainCalculator(object):\n", - "\n", - " def __init__(self, player, opponent):\n", - " self.player = player\n", - " self.opponent = opponent\n", - " \n", - " def ExpectedGains(self, low=0, high=75000, n=101):\n", - " bids = numpy.linspace(low, high, n)\n", - "\n", - " gains = [self.ExpectedGain(bid) for bid in bids]\n", - "\n", - " return bids, gains \n", - "\n", - " def ExpectedGain(self, bid):\n", - " suite = self.player.posterior\n", - " total = 0\n", - " for price, prob in sorted(suite.Items()):\n", - " gain = self.Gain(bid, price)\n", - " total += prob * gain\n", - " return total\n", - " \n", - " def Gain(self, bid, price):\n", - " # if you overbid, you get nothing\n", - " if bid > price:\n", - " return 0\n", - "\n", - " # otherwise compute the probability of winning\n", - " diff = price - bid\n", - " prob = self.ProbWin(diff)\n", - "\n", - " # if you are within 250 dollars, you win both showcases\n", - " if diff <= 250:\n", - " return 2 * price * prob\n", - " else:\n", - " return price * prob\n", - " \n", - " def ProbWin(self, diff):\n", - " prob = (self.opponent.ProbOverbid() +\n", - " self.opponent.ProbWorseThan(diff))\n", - " return prob" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`player` and `opponent` are `Player`\n", - "objects.\n", - "\n", - "`GainCalculator` provides `ExpectedGains`, which\n", - "computes a sequence of bids and the expected gain for each bid:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`low` and `high` specify the range of possible\n", - "bids; `n` is the number of bids to try.\n", - "\n", - "`ExpectedGains` calls `ExpectedGain`, which\n", - "computes expected gain for a given bid:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`ExpectedGain` loops through the values in the posterior and\n", - "computes the gain for each bid, given the actual prices of the showcase.\n", - "It weights each gain with the corresponding probability and returns the\n", - "total.\n", - "\n", - "`ExpectedGain` invokes `Gain`, which takes a bid\n", - "and an actual price and returns the expected gain." - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Player 1 optimal bid (16668.781463564384, 21000.0)\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwMAAAEWCAYAAAA6pksiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VFX6wPHvzCSZ9ARCChBqCL1ICZLQEUIHxUIRIiIL\nK8qCZVl3LYi7soqrKLIr6E8EQUBFBJEWigghoQUiLfQSWkIS0ntm7u+PSyYZaiBlZpL38zzzZM69\n5868uYHkvnPPeY9GURQFIYQQQgghRLWjtXQAQgghhBBCCMuQZEAIIYQQQohqSpIBIYQQQgghqilJ\nBoQQQgghhKimJBkQQgghhBCimpJkQAghhBBCiGpKkgEhhBBCCCGqqXsmA//+978JCgrCw8MDHx8f\nhg0bxrFjx27r9+6771K3bl2cnZ3p3bs3x48fN9ufl5fH1KlT8fb2xtXVleHDh3PlyhWzPikpKYwb\nNw5PT088PT0JCwsjLS3NrE9cXBxDhw7F1dUVb29vpk2bRkFBwcN+70IIIYQQQlRr90wGfv/9d15+\n+WWioqLYvn07dnZ29O3bl5SUFFOfDz/8kE8++YT58+ezf/9+fHx86NevH5mZmaY+06dPZ/Xq1axc\nuZJdu3aRnp7OkCFDMBqNpj5jxowhJiaGzZs3s2nTJg4ePMi4ceNM+w0GA4MHDyYrK4uIiAhWrFjB\nqlWreO2118rzfAghhBBCCFFtaB5kBeKsrCw8PDxYu3YtgwcPRlEU6tSpw1/+8hf+/ve/A5Cbm4uP\njw//+c9/mDRpEmlpafj4+LB48WJGjx4NwOXLl2nQoAEbN24kNDSU2NhYWrVqxe7duwkODgZg9+7d\ndO/enZMnTxIYGMjGjRsZMmQIcXFx1K1bF4DvvvuOiRMnkpiYiKura3mfGyGEEEIIIaq0B5ozkJ6e\njtFopEaNGgCcP3+ehIQEQkNDTX0cHR3p0aMHkZGRAERHR1NQUGDWx9/fnxYtWhAVFQVAVFQUrq6u\npkQAICQkBBcXF9PrREVF0bJlS1MiABAaGkpeXh7R0dEP+n0LIYQQQghR7T1QMjBt2jTat29vumiP\nj48HwNfX16yfj4+PaV98fDw6nQ4vLy+zPr6+vmZ9vL29zfZrNJrbXufW96lVqxY6nc7URwghhBBC\nCFF6dqXt+OqrrxIZGUlERAQajea+/e/X5wFGJz3wMbdOPBZCCCGEEMKWeHh4VMr7lOrOwCuvvML3\n33/P9u3badiwoWm7n58fAAkJCWb9ExISTPv8/PwwGAwkJyffs09iYqLZfkVRuH79ulmfW98nKSkJ\ng8Fg6iOEEEIIIYQovfsmA9OmTTMlAk2bNjXb16hRI/z8/AgPDzdty83NJSIigpCQEAA6duyIvb29\nWZ/Lly9z4sQJU5/g4GAyMzNNcwhAnSOQlZVl6hMSEkJsbKxZSdItW7ag1+vp2LHjw3zvQgghhBBC\nVGv3rCb00ksvsWzZMtasWUOLFi1M293c3HBxcQFgzpw5zJ49m2+++YbAwED+9a9/ERERwcmTJ019\npkyZwrp161i8eDE1a9bk1VdfJS0tjejoaNNwokGDBnH58mW+/PJLFEVh0qRJNG7cmLVr1wJgNBp5\n5JFH8Pb25uOPPyYpKYnx48fz5JNP8tlnn5nFXXKYUGXdYqkODhw4AECnTp0sHEnVIue1Ysh5rRhy\nXiuGnNeKIee1Ysh5rTiWuIa955yBL774Ao1Gw2OPPWa2/d133+Wdd94BYMaMGeTk5PDSSy+RkpJC\nly5dCA8PNyUCAJ9++il2dnaMHDmSnJwc+vbty7Jly8zmFSxfvpypU6fSv39/AIYPH878+fNN+7Va\nLevXr2fKlCl07doVJycnxo4dy0cffVT2syCEEEIIIUQ1dM9koOSiYPcyc+ZMZs6cedf9Dg4OzJs3\nj3nz5t21j6enJ0uXLr3n+9SrV49169aVKiYhhBBCCCHEvT1QaVEhhKiqDAaF+GSFtMwHr3QmhBBC\n2KpSlxYVQoiqIC1T4dApuHQdriTC5etwNQmuJUGhQe3jW1MhsB40qw9N66tf69S6f8lkIYQQwtZI\nMiCEqBZOX1L4fhts3gu5effum3BDfUT8UbzN3QUe76Hw3CBwd5GkQAghRNUgyYAQosoyGGF7tJoE\nHDxx//5eHpCZA3n5t+9Lz4JvN8LPv8NzgxRGPgaOekkKhBAPx2g0kp9/h182NqBBgwaAWk5elJ6D\ngwNarfWN0JdkQAhR5eQXKKzfV5Oth2qQa7h9f9P60C4Q/L3B3wfq1oI63uCk12AwKFy4Bqcuwck4\n9eupOEjLVI/NyIb5q+D7bTBxqMKwbmBnJ0mBEKL0FEUhLy8PR0dHmxx+6OjoaOkQbI6iKOTm5lrl\nz1ySASFElXLiosKsr+GPUz4AuDir23U66N0BRj6mJgJ3+2Ws02kI8IcAfxgYrG4zGhW27ocFa+DS\nzYXQE1Pg39/Cd5th6tMKvTpY1y93IYT1ys/Px8HBweouCkXF0Wg0ODg4kJ+fj16vt3Q4ZiQZEEJU\nCYWFCt9sgEXriicCA3i6wYheMKIn+NZ8uD+8Wq2G0EehT0eFXyLgq18gKVXdF5cAf50PLwxVmPy4\nTDIWQtyfoijodDpLhyEqmU6no6CgwNJh3EaSASGEzTt3ReHdryH2QvE2vZ3CU92v8/rzjdE7lM8F\nup2dhhG9YFCwOg9hyQZ12BDA1+vgegr8I0yRYUNCCCFshiQDQgibZTAoLN8CC36G/BIftrQNhCc6\nnsOvZkG5JQIlOeo1PDdIrS709lcQdUTdvi4CktLggxcVnB0lIRBCCGH9rG9KsxBClEJmtsJfPoV5\nPxQnAvZ2MPVp+HIG+NWs+FuxHq4aPpkKQ7sVb4s6An+eA8lpsniZEEII6yfJgBDC5lxPUZj0Iew7\nVryteQNY+g6EDdSg01Xep/J2dhrefh4mDiveFnsBXpgNF+MlIRBCiDvZsWMHWq2WnTt3WjqUak+S\nASGETTl7WeGF2XD6UvG254fAN29CgL9lhuZoNBomP67hH89BUQnpK4kw8d9w7JwkBEKI6mPx4sVo\ntVrTw97ennr16jFhwgSuXr1q6fDKzb59+3jppZcICgpCr9ej1Wq5fv26pcN6KJIMCCFsxsGTCn/6\nAOKT1badDt6ZAFNGaKxi0u4TPTX852VwvFk1LjUDXvtcvZMhhBDVyaxZs1i2bBkLFy6kX79+fPvt\nt3Tv3p2cnBxLh1YuNmzYwFdffYXRaKRZs2Y2XUlOkgEhhE3Yul/h5Y+Lq/c4O8In02BoN+v6Bdz9\nEQ0L/goermo7OQ1mzIe8fEkIhBDVR//+/RkzZgwTJkxg0aJFTJ8+nfPnz7N27VpLh1Yq+fn5GAx3\nWLXypilTppCRkUF0dDQjRoxAUWz3d7wkA0IIq7c8XOEfC6CgUG17ecDCv0Fwa+tKBIq0aqzhwynq\nQmcAx87Dv5di038shBCiLHr37g3AhQsX7trn8OHDPP/88wQEBODk5IS3tzejR4/m0qXicaGnT59G\nq9Uyd+7cOx6v1WpZuHChadu1a9eYOHEifn5+ODo60rJlSxYsWGB2XNH8heXLl/Puu+9Sv359nJ2d\nuXLlyl1j9fHxsbrFwx6WlBYVQli1Rb8qfLG6uN2wNnw6Hep6W2ciUKRjcw2vjFT4z3K1vX43NKsP\no/tZNi4hhLCEs2fPAuDl5XXXPlu3buXUqVOMHz+eOnXqcObMGRYsWMC+ffs4evQoTk5OBAYGEhwc\nzLJly3jllVfMjl+2bBl6vZ6RI0cCcP36dbp06YKiKLz88sv4+PiwdetWpkyZQnJyMm+++abZ8bNn\nz0an0/HKK6+gKAouLi7lfBaskyQDQgirtTzcPBFoGwifTFVLetqCZx6DE3Hwa4Ta/uwHaOKvENTC\nNuIXQoiHlZqaSlJSErm5uezevZv33nsPZ2dnhgwZctdjXnzxRV599VWzbcOGDaNr166sXr2aZ599\nFoCwsDBefPFFYmNjadGiBQBGo5EVK1YwePBgPD09AXjrrbcoKCjgyJEjpiRk0qRJTJo0idmzZ/Py\nyy/j4eFheq/MzExiY2NxcnIq13Nh7SQZEEJYpdU7FOauLG53agGfTqNCFhGrKBqNhjfGKly4CkfP\ngcEA/1gAi99SrP7OhhDCeuw+spnIo+EV8tohrUPp2qZ/ub/ugAEDzNqtWrVi3rx51K5d+67HlLwI\nz8zMJC8vj8DAQDw9PTl48KApGRg5ciTTp09n6dKlzJ49G1CH+ly5coVx48YB6rDMVatW8eSTT6Io\nCklJSabX7tevH//3f//H3r17CQ0NNW0PCwurdokAyJwBIYQV2hCp8MHS4nbbQPh4qm0lAkX0Dhrm\nvAS11A+qSM2Av86HnDyZPyCEqLo+//xztm7dSkREBHFxcRw5csQ0b+BuUlJSmDx5Ml5eXri7u+Pt\n7Y2Pjw+pqamkpaWZ+nl6ejJs2DCWL19u2rZs2TK8vLwYPHgwAImJiaSmpvL111/j4+Nj9hg5ciQa\njYbExESz9w8ICCjHM2A75M6AEMKqbDugMGsRFM21bdEQPpsGzo62lwgU8a6h4cMpCpM/hEKDukbC\ne4tg9p8Vmy5HJ4QQdxMUFETnzp0f6JhnnnmGyMhIXn/9ddq3b4+bmxsAo0aNwmg0mvUNCwvjxx9/\nZNeuXQQFBfHTTz8xduxY7OzUS9ui/kUVje6kZcuWZu3qeFcAJBkQQliRiD8U3loIRb/zm/jD56+C\nq7PtXzC3baLhb+MU3l+strfuh+7tYFCIJaMSQtiCrm36V8hQHmuSkpLCtm3bmDVrFm+//bZpe25u\nLjdu3Lit/4ABA/Dx8eHbb7/l2rVrZGRkmIYIAXh7e+Pm5kZBQQF9+vSplO/BVskwISGEVdgfq/C3\n/6mfnAM08IP5r9nOZOHSeLyHhhG9ittzv4eUdBkuJIQQupu1mG+9AzB37tw7lmXW6XQ8++yzrFq1\nikWLFhEYGEiXLl3M9j/11FOsWbOGw4cP33b8rUOEqjO5MyCEsLhTcQp/nQ/5BWq7rjf893Xw8qg6\niUCRac9A5BF1FeXUDDUheO9Plo5KCCEsy93dnV69ejFnzhzy8/OpX78+ERER7Ny5Ey8vrzsmBGFh\nYcydO5fw8HBmzZp12/4PPviAHTt2EBwczJ/+9CdatmxJSkoKMTExrFmzpkyrIV+8eJGlS9XJbTt3\n7gTUxMXFxYWGDRsyduzYh37tyibJgBDCoq6nKLzyGWTd/J3sU0NNBHxrVr1EANS5D2+MU5j+qdre\nGAUDgxWrXUBNCCEeVGnnQt3ab/ny5UybNo2FCxdSUFBAz5492b59O3379r3ja7Zr1442bdpw9OjR\nO158e3t7s3fvXv75z3+yZs0avvjiC2rWrEnLli355JNPHirmIufPn+edd94xHavRaPjwww8B6NWr\nl00lAxqlCi6JWXLGecn6saJsDhw4AECnTp0sHEnVUp3Pa1aOwqQP4VSc2nZ1hq/egCb+Zb8wtvbz\n+uZChfC96vM6tWDlP8FJb/0JgbWfV1sl57ViWOt5zc3NxdHR0dJhVAlBQUE4Ojqya9cuS4dSKvf7\n2VviGlbmDAghLKKwUOGNL4oTATsdfDilfBIBW/DaaPBwVZ9fTYIFP1s2HiGEsDUxMTFER0fz3HPP\nWToUmybJgBCi0imKwoffwZ6jxdv+HgadW1aPRACgpruGac8Ut1duhePnq9yNWiGEKHdHjx5lyZIl\njB8/Hl9fX5sakmONJBkQQlS6JRtgze/F7ReGwrDu1ScRKDKkKwTdLHNtNMLsJeodEyGEEHf3008/\nMWHCBPLy8li5cqUMuSojSQaEEJVq816F//5U3B4UApMft1w8lqTRaPhHGDjq1fbJOFi+xbIxCSGE\ntZs5cyYGg4HY2Fh69uxp6XBsniQDQohK88dphVlfF7c7Noe3xj94FYeqxN9Hw5+GFbe/XAuXEuTu\ngBBCiMohyYAQolLEJyvM+C8UFKrtRnVgzktgb1d9E4Eiz4ZC8wbq87x8+GSlZeMRQghRfUgyIISo\ncDl5Cq99DjfS1banG8ydBu4ukggA6HQa/vEcFN0gifgDDp2SuwNCCCEqniQDQogKZTQqvPu1eQnR\nOS9BXW9JBEpq0VDDwODi9ueruOOKm0IIIUR5khWIhbBiN9Kvs+f4NuISzmCns8PRwdn0cNKrX2t7\n1adR7eZWO+7+/9bB9gPF7b+Ng/ZNrTNWS5s8HLbsU4dSHTkDvx+CXh0sHZUQQoiqTJIBIaxQUlo8\ne45t40RcDIpivG//xnVaEBr0NG7O1rXi9tb9Cl+tLW6P7AuP95BE4G7qeGt4uo/C8nC1/d+foHs7\nBZ1OzpkQQoiKIcOEhLAiianX+GX3UhZv+A+xFw+WKhEAOHc1lm82fsSx8wesZmjJyYsKsxYVtzu3\ngldGWi4eW/H8YHB1Vp9fuAa/Rlo2HiGEEFWb3BkQwgrkF+byS8S3nLz0x237GtdpQecWfXBxdCUn\nL5vc/OLH9dRrHD23D4C8/Bw27FnBqUuHCQ16Chcn98r+NkyS0xRenw+5eWq7vi/8+8/IJ9yl4Omm\nYdwAhS9Wq+0v10D/zgqOejl3Qgghyp8kA0JYWG5BNnvPbsKoyzPbHlC3FcGt+lLbq/49j2/VsCOb\n9v1AWmYyAGeuHONK0gUe6/gEzes/UulzCfILFGb8D+LVcHB1ho//IpWDHsTofvDjdkhKhesp8P02\neG6QpaMSQojys2PHDvr06cOOHTvo0aOHpcOp1mSYkBAWlJZ5g6gzv5KZm2LaFujfmrD+rzCix4T7\nJgIA9X2bMH7AazzSJMS0LScvi18jl7Hj0LoKiftuFEXho+Vw+LTa1mrh/cnQsLYkAg/CSW++ENni\nDZCWaR3Dv4QQ4l4WL16MVqs1Pezt7alXrx4TJkzg6tWrlg6vXCiKwuLFixk2bBj169fH1dWVNm3a\n8P7775OXl3f/F7AycmdACAu5kX6dH35bSHZeBgBajZZBwWNo0aD9A7+Wg72efkFP0rReGzbt/YH0\nbDW5OHDyd2q6+9CuSZdyjf1uftoBa34vbr/8FIS0kUTgYQzrBsvD4WI8ZGarCcG0ZywdlRBClM6s\nWbMICAggNzeXiIgIvv32W37//XeOHj2Kk5OTpcMrk6ysLCZMmEBwcDAvvvgiPj4+REZGMnPmTLZu\n3cpvv/1m6RAfiCQDQljA9ZQr/Pjbl2TnZQKg0WoZ3n08Teq2KtPrNvBryvhBr7MhajlnrhwDYGv0\namp5+FLXu1GZ476XgycVPl5e3B7QBcb2r9C3rNLs7DS8OELhjf+p7R+2wcjHFPy8JLkSQli//v37\n07lzZwAmTJhAzZo1+eSTT1i7di2jRo2ycHT3l5+fj06nQ6fT3bZPr9cTGRlJly7FH7S98MILNGzY\nkJkzZxIeHk5oaGhlhlsmMkxIiEp2NekCK7d/YUoEtFodnRuFljkRKKK3d2RwyLP4eNYBwGg0sHb3\nt2Rkp5bL699JfLJ60VpoUNstGsJb47HatQ9sRZ+O0Lqx+jy/AL5ce+/+QghhrXr37g3AhQsX7trn\n8OHDPP/88wQEBODk5IS3tzejR4/m0qVLpj6nT59Gq9Uyd+7cOx6v1WpZuHChadu1a9eYOHEifn5+\nODo60rJlSxYsWGB23I4dO9BqtSxfvpx3332X+vXr4+zszJUrV+4Yp729vVkiUOTxxx8HIDY29u4n\nwgpJMiBEJYpLOMMPvy0kLz8HUC/cHw0YSC23uuX6Pg52eh7v/jxOehcAsnLSWRuxhILCgnJ9H4Dc\nPIW/zocUdbQTNd3ho5dB7yCJQFlpNBpefqq4/etuuBgvcweEELbn7NmzAHh5ed21z9atWzl16hTj\nx49n/vz5TJw4kU2bNtGrVy9yctS/m4GBgQQHB7Ns2bLbjl+2bBl6vZ6RI9U61tevX6dLly6Eh4fz\n8ssvM2/ePFq3bs2UKVN4//33bzt+9uzZ/Pzzz7zyyivMmTMHFxeXB/oe4+PjAahVq9YDHWdpMkxI\niEqSnJ7Az7u+oaAwHwAnvQtP957MpbPXKuT9PFxrMqxrGD/+thCjYuRachxbDqxi4KOjyu0Te0VR\n+OdiOHFRbdvp4IMp4FtTEoHy0rG5hi6tFfYcBUWBJRvgnQmWjkoIUZm+XGu+gGN5+tNwmDS8/H9n\np6amkpSURG5uLrt37+a9997D2dmZIUOG3PWYF198kVdffdVs27Bhw+jatSurV6/m2WefBSAsLIwX\nX3yR2NhYWrRoAYDRaGTFihUMHjwYT09PAN566y0KCgo4cuSIKQmZNGkSkyZNYvbs2bz88st4eBQv\n1pmZmUlsbOxDz2mYM2cO7u7uDBpkW+Xf5M6AEJUgLz+HNbsWk1+QC4Crkwej+76Eb43yvSNwq/q+\nTejVvrgszbHzBzh4KqLcXn/pJgjfW9z+67PQvqkkAuXthRJ/OzdGwbUkuTsghLBuAwYMwMfHh/r1\n6zN69Ghq167NunXrqF279l2PKXkRnpmZSXJyMoGBgXh6enLw4EHTvpEjR6LX61m6dKlp244dO7hy\n5Qrjxo0D1A+rVq1axeDBg1EUhaSkJNOjX79+5OTksHdviT9gqEnGwyYCs2fPZtu2bXzwwQfUqFHj\noV7DUiQZEKKCGRUj6/es4Eb6dQDsdQ6M6PkCXu6+lfL+HZp2o3WjIFN7x6FfuBh/usyvG3lE4b8/\nFbdH9IIRvSQRqAiPNNXQvpn6vNAAyzZbNh4hhLifzz//nK1btxIREUFcXBxHjhwxzRu4m5SUFCZP\nnoyXlxfu7u54e3vj4+NDamoqaWlppn6enp4MGzaM5cuLq1YsW7YMLy8vBg8eDEBiYiKpqal8/fXX\n+Pj4mD1GjhyJRqMhMTHR7P0DAgIe6nv9/vvvefvtt5k4cSJ//vOfH+o1LEmGCQlRwaKObuHszco+\nAKGdn67wOwIlaTQa+nZ6kuT0BK4lx2FUjKyLXMpzA17Dzdnj/i9wBxfjFd5cCEaj2m4XCK+PKceg\nxW0mDIapJ9Xna3fBhCEKXh6SfAlRHUwarmHScEtH8WCCgoJM1YRK65lnniEyMpLXX3+d9u3b4+bm\nBsCoUaMwFv3BuSksLIwff/yRXbt2ERQUxE8//cTYsWOxs1MvbYv6jxkzhgkT7jy2smXLlmbth7kr\nsGXLFsLCwhgyZMhtE5NthSQDQlSgM1eOEXk03NTu1KwnLRt2qPQ47O3sGd5tPEvDPyUrJ52cvCx2\n/rGewcEPfgWfma3w+udq7XsA35rw4RSwt5ML04r0aCu1SlPsBcjLV9cgmPq0paMSQojykZKSwrZt\n25g1axZvv/22aXtubi43bty4rX/RMKRvv/2Wa9eukZGRYRoiBODt7Y2bmxsFBQX06dOnQmLeu3cv\nTzzxBJ07d+aHH35Aq7XNATe2GbUQNiA5PYH1UcW3MBv4BtLzkcEWi8fN2YPBXUab2scvRHMl8fwD\nvYbBoPDml3Dh5pxnRz38ZyryCXUl0Gg0PF/in8+q3yA9S+YOCCGqhqJ6/rfeAZg7dy6KcvvvOp1O\nx7PPPsuqVatYtGgRgYGBZuU+dTodTz31FGvWrOHw4cO3HX/rEKEHFRsby+DBg2ncuDG//vorer2+\nTK9nSfdNBnbu3MmwYcPw9/dHq9WyZMkSs/3jx483W3Zaq9USEhJi1icvL4+pU6fi7e2Nq6srw4cP\nv612a0pKCuPGjcPT0xNPT0/CwsLMxocBxMXFMXToUFxdXfH29mbatGkUFJR/qUQhyurWCcPuLjUY\n0nUsWu3ti5dUpgZ+TWlar62pvS16DUajodTH/281RJb4nfr2eGjeQBKBytKzPTS+OcIsOxe+32bZ\neIQQory4u7vTq1cv5syZw5tvvsnChQsZN24cCxYswMvL644JQdG1Ynh4OGPHjr1t/wcffIC/vz/B\nwcFMnz6dL7/8kg8//JDRo0dTv379h441IyOD/v37k5qaytixY1m3bh3Lli0zPfbs2fPQr20J900G\nsrKyaNu2LZ999hlOTk63lSTUaDT069eP+Ph402PDhg1mfaZPn87q1atZuXIlu3btIj09nSFDhphl\nf2PGjCEmJobNmzezadMmDh48aHa7x2AwMHjwYLKysoiIiGDFihWsWrWK1157raznQIhydacJw493\nfx5nvauFI1P1aj8UO509AAkplzlybn+pjtu0R+HbjcXt8YMh9FFJBCqTVqvhuYHF7e+3Qnau3B0Q\nQliX0pavvrXf8uXLGTJkCAsXLmTGjBmkpaWxfft2XF1d7/ia7dq1o02bNmg0mjsmA97e3uzdu5eJ\nEyeyZs0apk6dyqeffsr169f55JNPHipmgOTkZC5fvoyiKLzxxhuEhYWZPb788stSv5Y10Ch3SrXu\nws3Njf/+97+EhYWZto0fP57k5GTWrVt3x2PS0tLw8fFh8eLFjB6tDlG4fPkyDRo0YOPGjYSGhhIb\nG0urVq3YvXs3wcHBAOzevZvu3btz8uRJAgMD2bhxI0OGDCEuLo66ddWPxr777jsmTpxIYmIirq6u\nZu9ZpGT9WFE2Bw4cAKBTp04WjsS67Yv9jd9jfjW1Bwc/e895ApY4r7uPbDbNZXDSu/DC4Ddw0jvf\ntf/x8wqTPlTHqgN0bQsfTwWdznqTgar679VgUHjqTbis5ppMGwlj+1fez6GqnldLk/NaMaz1vObm\n5uLo6GjpMKqEoKAgHB0d2bVrl6VDKZX7/ewtcQ1b5jkDGo2GiIgIfH19adasGZMmTTIbhxUdHU1B\nQQGhoaGmbf7+/rRo0YKoqCgAoqKicHV1NSUCACEhIbi4uBAZGWnq07JlS1MiABAaGkpeXh7R0dFl\n/TaEKBcJNy6z63Dxx+eWmjB8P51b9MHdRa2DnJOXReTRu9eqTE5TVxguSgQa1oZ/TbLuRKAq0+k0\njBtQ3P5uM+Tly90BIUT1ExMTQ3R0NM8995ylQ7FpZU4GBgwYwNKlS9m+fTsff/wx+/bto0+fPuTn\nq1cO8fHx6HS625af9vX1NS3bHB8fj7e3t9l+jUaDj4+PWR9fX/O67LVq1UKn05n6CGFJ+YV5/Bq5\nzDQGv7Y0qp5fAAAgAElEQVRXfXpYcMLwvdjb2dO7xGJkMacjSUy9fSXkvHw1EbieorbdnNU7Aq7O\nkghY0pCu4H1zTZukVPh1t2XjEUKIynT06FGWLFnC+PHj8fX1veMQIVF6ZS4tOnLkSNPzVq1a0bFj\nRxo0aMD69et54okn7nrcA4xOKtMxRbcIRfmRc3pnRy5FcDn5IgA6nR31XFpz6OChUh9f2edVURSc\ntJ4kZ1wF4PtNX/FowEDTuElFgS/W12HPCXcAtBp4ceAlrl/O4vrlSg21TKrqv9eQwBos36F+QDJv\nZQF1nc9iV4nz06vqebU0Oa8Vw9rOa4MGDWSYUBn89NNPvPfeezRt2pSVK1fa1LnMyMjg6NGjd90f\nGBhYidGoyr20aO3atfH39+fMmTMA+Pn5YTAYSE5ONuuXkJCAn5+fqc+tJZ4UReH69etmfRISEsz6\nJCUlYTAYTH2EsJT41AvEJZ80tVvVCcZFb93zVTQaDa3qPGq6+E/OvEZ82gXT/rV7vEyJAMDoXgm0\nbZRV2WGKu+jVNhU3J/UuVGKavdnPSgghqrKZM2diMBiIjY2lZ8+elg7H5pX7omOJiYlcuXKF2rVr\nA9CxY0fs7e0JDw83m0B84sQJUwnS4OBgMjMziYqKMs0biIqKIisry9QnJCSE999/nytXrpjmDWzZ\nsgW9Xk/Hjh3vGo+1TRqyZdY6EcvSMrJT2bNxLc4u6gTcZvXaMbTr6FJXJrD0eVVcsok+qU68upp1\nkgG9hrPjoD2bD8HNb4kRveCNcY0fqNqCpVn6vFaGSckKC35Wn+87F8i0sAeriPEwqsN5tQQ5rxXD\nWs9rbm6upUMQFuLm5nbPf4+3ltWvDKUqLRoTE0NMTAxGo5GLFy8SExPDpUuXyMrK4vXXX2fPnj1c\nuHCBHTt2MGzYMHx9fU1DhDw8PHjhhReYMWMG27Zt49ChQ4wbN4527drRt29fAFq0aMGAAQOYPHky\ne/bsISoqismTJzN06FDT7ZLQ0FBatWpFWFgYMTExbN26lRkzZjBp0iSzSkJCVCajYmTjnpXk5KvL\n8bo5e9Iv6CmbumgOad0fJ70LAOnZKXy/bR+zFhXvD2oJfx1T8ReZ4sE93Udd+A3g9CXYH2vZeIQQ\nQtie+yYD+/fvp0OHDnTo0IHc3FxmzpxJhw4dmDlzJjqdjqNHjzJ8+HCaNWvG+PHjTVWCXFxcTK/x\n6aef8sQTTzBy5Ei6deuGu7s769atM7u4WL58Oe3ataN///4MGDCA9u3bs3Tp0uJAtVrWr1+Ps7Mz\nXbt2ZdSoUTz11FP85z//KedTIkTpHTjxOxcTTgOgQcOgLqPvWaLTGjk6ONGj3SAAMrJdmb3Em5w8\ndQ2QBn7wwYtgZyeJgDVyd9EwtFtx+7twy8UihBDCNt13mFCvXr1uWxq6pE2bNt33TRwcHJg3bx7z\n5s27ax9PT0+zi/87qVev3l3XMxCist1aRrRzyz7U921iwYgeXutGQew+EsmK8J5kZDuhKBnUqeXB\nJ39RLziF9Rr1GKzark74jjwMF64pNKwtPzMhrJ2iKHLHtZp5mEI4laHcJxALUR0UFObza9R3pjKi\nfjXr0bV16H2Osl6KomXfsTEkpqglfnPz0/nX5Hzq+8kfKmtX309D90eK2yu2WC4WIUTpODg4kJub\na7UXh6L8KYpCbm4uDg4Olg7lNuU+gViI6mDnH+u5ka4uAWtv58Dg4GfR6Wzzv5OiKMz9Ho6e9cFO\nF0+hoZCeHbai0TQAHrN0eKIUng2FnTer2K6PhBefUPB0k0ROCGul1WrR6/Xk5eVZOpSHkpGRAaiT\nYUXp6fV6tFrr+xzeNq9ehLCg89dOcPBUhKndu/1warp73+MI6/ZdOHy/FUCDi5M7jetupVXjYxw4\neYEOzbrhYKe3dIjiPto3hRYNIfaCulL0TzvghaEWDkoIcU9ardam6uOXVFQn39qqNImHY33piRBW\nLCcvi417vze1m9RtRduARy0YUdmE71X4rPjbYUiIMwND1F/yOXlZ/HEmykKRiQeh0WgY3a+4/eN2\nyC+Q4QdCCCHuT5IBIUpJURTC968iKycdAGdHV0I7P22zE8CiTyi8+3Vx+5GmMGuihuCWfUzb9sfu\noKCwwALRiQfVNwh8aqjPk9Ng817LxiOEEMI2SDIgRCkdO3+AU5cOm9oDOj+Di6Ntjpc8e1nhr/Oh\noFBtN6oD/3kZ9A4aWjcOws1JXT05KzeDI+fkqtIW2NtpeKbEFI/l4dZbuUIIIYT1kGRAiFJIzUxm\n28E1pna7gC4E1G1lwYgeXmKKwvTPIENdJ41anvDZdPBwVe9w2OnsCWrR29R/3/HtFBrk7oAteKJn\n8SJkZy7LImRCCCHuT5IBIe7DaDSwYc8K8gvU5eNruNWiV/thFo7q4WTlqIlAfLLadnaET6dB7Vrm\nQ53aBnQx3fXIyEnj2PkDlR2qeAjuLhqGySJkQgghHoAkA0Lcx/4TO7iSeB4ArUbLoC5jcLC3vQo7\nefkKr34Op+LUtp0OPpwCzRrcPufB3s6eTs17mtp7j2/HcHNNBWHdRvWFomkskYfh/FUZKiSEEOLu\nJBkQ4h4Sblwm4shmU7tLq77UqdXAghE9nIJChb/9Dw6eKN72j+egS+u7T35+pEkITnoXANKybnD8\nQnRFhynKQT1fDT3aF7dlETIhhBD3IsmAEHeRX5DHushlplWGa3vVp0urvhaO6sEZDArvfAW7i+c+\nM/VpGNrt3lWQHOz1dGzWw9Tee3yb6VwI6zamRJnRDVGQmiF3B4QQQtyZJANC3MW26J9JyUgEwMFO\nz6AuY9BpdRaO6sEoisK/l8LW/cXbJgyFsIGlK4faoWk39A5OAKRkJHH68tGKCFOUs6JFyEBdhGzN\nTouGI4QQwopJMiDEHcRePMTR88VX0H07jbC5VYYVReHT72FtiQvBZx6DPz9e+tfQ2zvSPrCrqR19\nalc5RigqikajYWSJm1irflPvEAkhhBC3kmRAiFukZiazZf8qU7tlww60amR7S65/9Ytaa77I4K7w\n2mgeeJG09oFd0d68I3Il8TzXkuPKM0xRQfoFQU139XnCDdhxyLLxCCGEsE6SDAhRgsFo4NfIZeTd\nLCPq6epF305PWjiqB7c8XOGrtcXtPp3g7fGg1T74asmuTu60qF88IzX6pNwdsAUO9hqeKC4Ixcqt\nlotFCCGE9ZJkQIgSIo9sNn3yrdXqGBIyFr29o4WjejDLwxXmrixuP9oK/vkn0OkePBEo0rFZd9Pz\nk5f+ICM7tSwhikryZC+1hCxAzCk4eVGGCgkhhDAnyYAQN12MP83e49tN7W5tBlDbq74FI3pw3240\nTwTaBcJHL6ufEpeFb01/6vk0BtRF2A6djizT64nK4V1DQ58SI9x+2H73vkIIIaonSQaEALLzMtmw\nZwUK6ienDfyaEtSil2WDekCLflX4/MfidrtA+Gw6OOnLlggU6dC0uMzo4TNRFBTml8vrioo18rHi\n55v2SJlRIYQQ5iQZENWeUTGycc9KMnPSAHDSuzCoy2i0Gtv476EoCl+uVfhidfG2Ds1h3ivg4lQ+\niQBAk7ot8XD1AiAnP1sWIbMRbQKgZSP1eX4B/CxlRoUQQpRgG1c7QlSgPce2ce5qrKk98NFRuDq5\nWzCi0lMUhS9+xmyycOdW8Nk0cHYsv0QA1DkUHUqUGT1wcidGxViu7yHKn0ajMbs78NNvUFgodweE\nEEKoJBkQ1dr5ayeJPLLZ1A5q3ouAui0tGFHpKYrCvB/hm1+LtwW3gU+mgmM5DQ26VZuAR3G4OaH6\nRvp1Llw7VSHvI8pXXykzKoQQ4i4kGRDVVlrWDdZHfWeaJ1DPJ4Ae7QZZOKrSMRgUPlgKyzYVb+vW\nDj56CfQOFZMIgLoIWZvGnU3tg6dkzIktuLXM6PfbLBeLEEII6yLJgKiWCg0FrNu9lJy8LABcnNwZ\nGjLWtLiWNcvNU5jxP1i9o3hbzw4wp4ITgSIdmnZDc3M+xflrJ0lKi6/w9xRlJ2VGhRBC3IkkA6Ja\n2n5wrdl6AsO7huFiA/ME0jIVpnwMO0sM8xjQBT74M9jbVXwiAOpCbE3qtjK1D8oiZDbh1jKjcndA\nCCEESDIgqqGj5/bzx5koU7vXI0Op693IghGVztVEhYn/hiNnireNGwizJoJdJSUCRUouQnbsQjTZ\neZmV+v7i4YzqW/x8815ISZe7A0IIUd1JMiCqlYSUK2w58JOp3aJBezo07WbBiErnVJzCC/+GC9fU\ntkYDr46GvzytQaut3EQAwN+7Mb41/AF1yNUfZ/ZUegziwbVuDK2kzKgQQogSJBkQ1UZOXha/RCyh\n0FAAgJeHL6FBT6PRVP7F9IPYH6sweQ4kpaptezt4fzKM7me5uDUajdndgT/ORGE0GiwWjygdjUbD\nMyXKjK7eIWVGhRCiupNkQFQLBkMhayOWkJqZDICDvSOPdxuPg73ewpHdnaIo/LBNYeonkJmtbnN1\nhs9fhX6dLZ/ANKvfDme9KwAZ2amcvXrcwhGJ0pAyo0IIIUqSZEBUeYqisHn/j1y6fta0beCjo6jp\n7mPBqO4tv0DhX4vho+/AcPMDd+8a8NUb0LG55RMBADudPW0DHjW1D52OtGA0orSkzKgQQoiSJBkQ\nVd7e49s5dv6Aqd2j3WCa1mtjwYjuLTFFYfKH8EuJIj0tG8E3b0ITf+tIBIq0bdLFVGb0YvwpbqRf\nt3BEojSkzKgQQogikgyIKu1k3B/sOrzB1G7TuDOdW/S2YET3duSsQtg/4ei54m2DQuDLv4FvTetK\nBAA8XGoSUKeFqS0TiW3DrWVGf9huuViEEEJYliQDosq6lhzHhj0rTO16PgH06/Sk1U4Y/mWXekeg\naKKwTqdWDHr3hcpZTOxhPRIYYnp+5Pw+CgrzLRiNKK1n+hQ/37xXXcNCCCFE9SPJgKiS0rNS+Hnn\nIlPloBpu3gzv9hw6nZ2FI7tdTp6WLzfW5p/fQEGhus3DFea9olYMstbkpUgDv6Z4unoBkJefQ+xF\nmZFqC9o2geYN1Od5+bBGyowKIUS1JMmAqHLyCnJZvfNrsnIzAHBycGZEjxdw0rtYOLLbHT+v8M7S\nhkQc8zBtC6wHS96Gzi2tOwkootVoze4OxJyORFHkU2Zrp9FoGFliEbJVv4HBID83IYSobiQZEFVK\noaGAtRFLSExVV+fSanUM6/YcNd29LRyZOaNR4duNCi/MhoRUB9P2QSHw9T+grrdtJAJFWjcKwk5n\nD0BCymWuJcdZOCJRGv2CwNNNfR6fDL/HWDYeIYQQlU+SAVFlGI0Gfo38jovxp0zbQoOeor5vEwtG\ndbukVIWpc+HzH6HwZtlQJwcj7/0JZk3U4KS3rUQAwEnvQvMG7U3tGCkzahP0DhpG9CpuS5lRIYSo\nfiQZEFWCUTGyad8PnL58xLSta5sBtGnc2YJR3W5njMKYmbDvWPG2gNo5vBd2noHBtpcElPRIk+Kh\nQicuxZCdl2nBaERpjeipTlYHOHgCTl+SoUJCCFGdSDIgbJ6iKPx2cK3ZWgKdmvUkuFXfexxVuW6k\nK/xjgcJr8yBFncqARgPPDYI3R13E17PAsgGWg9pe9ajtVR9QV3w+em6/hSMSpeFbU0OvDsVtKTMq\nhBDViyQDwuZFHg3n4KkIU7tN4870aj/UKqrwKIrC+kiFZ96CLfuKt9fyhPmvwctPaUyLP1UF7ZoE\nm57HnInCaDRYMBpRWqMeK36+aY+UGRVCiOpEkgFh0w6c+J3Io+GmdrN67QgNesoqEoGriQpTP4F3\n/w/SSoyYGRQCK9+znWpBD6J5/fY4OjgDkJaZzIUS8zeE9WoXCM3Umzrk5sHaXffuL4QQouqQZEDY\nrCPn9vHboV9M7Ua1mzM4eAxarWU/ai8sVFgerjDyHdhbYm5A7Vow71V1krCHa9VLBADs7exp3TjI\n1D50ercFoxGlpdFoeKbE3QEpMyqEENWHJAPCJh0+u5fNe38wtet6N7KKRcUijyiMeRfmrlQ/YQV1\nbsDoUPVuQHDrqpkElPRIiaFC56+eIDUz2YLRiNLq/2hxmdFrSVJmVAghqgtJBoTNOXgqgs37fkBB\n/eTSt4Y/T/Z4AXs7h/scWXHOX1WY9qnCtLlw/mrx9oC6sOhNeHWUBmfHqp8IgLrac6PazQBQUDhy\ndq+FIxKlcWuZ0ZVbLRaKEEKISiTJgLAp+2J/Y1v0z6a2bw1/nur9J/QOThaJJy1T4aPvFEbPhMjD\nxdtdnOClJ2HpTGjduHokASWVnEh85Nw+DDKR2CY81QvThPZDJ+HERRkqJIQQVZ0kA8ImKIrC7iOb\n+T3mV9O2OrUa8kyfP+Osd630eHLy1BWER/wdftgGhpvXuhoNPN4TfpoN4wdrsLerfokAQECdlrg6\neQCQlZvBmctHLRyRKA3vGhoeK57ywYotlotFCCFE5ZBkQFg9RVHY+cd6s6pB9XwCeLrXJBwr+Y5A\nURIwfIa6gnB6VvG+Ds1h6Tvw5nMavDyqZxJQRKvVmS349sfZPRaMRjyI0SWW5wjfC8lpcndACCGq\nMkkGhFUzKka2Rf/MvtjfTNsa1W7Gkz0n4mCvr7Q4cvIUlm4qTgKKFg4D8PeBj16GBX+FZg2qdxJQ\nUtuAR9Fo1F8xF+NPkZKRaOGIRGm0aqyhbaD6vNCgVhYSQghRdUkyIKxWQWEBG6KWm5WnDPRvzePd\nn6+0ycIZWeqdgMf/BvN+ME8C/Lzg72Hww7+gVweNVaxtYE3cXWrQuHZzU/uwTCS2GSXvDvz0G+Tl\ny90BIYSoqixbh1GIu8jOzWTNrm+4knTBtK1Fg/YM7DIaXSWsI3ApQWHlVlgXATl55vv8vOD5wTC0\nG9V2TkBptWvShbNXjwPqROKubfpjp7O3cFTifnp3UP+dxyerCfDmvTCsu6WjEkIIURHue2dg586d\nDBs2DH9/f7RaLUuWLLmtz7vvvkvdunVxdnamd+/eHD9+3Gx/Xl4eU6dOxdvbG1dXV4YPH86VK1fM\n+qSkpDBu3Dg8PT3x9PQkLCyMtLQ0sz5xcXEMHToUV1dXvL29mTZtGgUFBQ/zfQsrlpyewHdb5pkl\nAo80CWFQBScCiqJwIFbhtc8VnvyHOjG4ZCLg5wVvjFMnB4/oVX0nBz+IRrWb4+5cA4CcvCxOy0Ri\nm6DTaXi6T3F75Vb1/4cQQoiq577JQFZWFm3btuWzzz7DycnptqEQH374IZ988gnz589n//79+Pj4\n0K9fPzIzM019pk+fzurVq1m5ciW7du0iPT2dIUOGYDQaTX3GjBlDTEwMmzdvZtOmTRw8eJBx48aZ\n9hsMBgYPHkxWVhYRERGsWLGCVatW8dprr5XHeRBW4mL8ab7b8rlpoSoNGnq3H0bfTiMqbGXhjCyF\nn35TGDsLXvwIdh6Cktc9AXXhzfFqEvBkbw0O9pIElJZWq6NNwKOm9h9noiwYjXgQj/cAx5vTck5f\ngugTlo1HCCFExbjvMKGBAwcycOBAAMaPH2+2T1EUPv30U/7+97/zxBNPALBkyRJ8fHxYvnw5kyZN\nIi0tjUWLFrF48WIee0xd737p0qU0aNCArVu3EhoaSmxsLJs3b2b37t08+qh64bBw4UK6d+/O6dOn\nCQwMJDw8nOPHjxMXF0fdunUBmDNnDhMnTmT27Nm4ulZ+eUlRvo6c20f4/lUYb9akt9c5MKTrWJrU\nbVXu72U0Khw8Cb9EwPZoyMu/vU9IWxjTDzq3ROYDlEGbxp2JOhqOUTFy6fpZktMT8HL3tXRY4j7c\nXTQM6aqwarvaXrEVOrWwbExCCCHKX5kmEJ8/f56EhARCQ0NN2xwdHenRoweRkZEAREdHU1BQYNbH\n39+fFi1aEBWlfkoYFRWFq6srwcHFCxWFhITg4uJiep2oqChatmxpSgQAQkNDycvLIzo6uizfhrAw\no9HA7zG/smnv96ZEwMXJnVF9p5R7IhCfrPD1OnUY0IsfwcYo80TAUQ8jeqmTgj+bruHRVjIxuKzc\nnD0IKPFzPHxGJhLbilElJhLvilHn0gghhKhayjSBOD4+HgBfX/NP+Xx8fLh69aqpj06nw8vLy6yP\nr6+v6fj4+Hi8vb3N9ms0Gnx8fMz63Po+tWrVQqfTmfrcyYEDBx7iOxP3Up7nNLcgm0MXf+NGZvHP\n0M2pBm18enL5XAKXzyWU+T2S0u04cMqdfSfdOHPtzusS1PfOo2ebVIJbpOHqZCT5KiRfLfNbP5Cq\n/G9VX+BJdlY2AFGHt+Fc6INOWzn1C6ryea0MTXz9+eO8euf14yUpjL05l0DOa8WQ81ox5LxWDDmv\n5S8wMLDS37PC/hrf79PUh5mMJhPYqpakjKscittBfkGOaZu3uz/tG/TGXle20qHJ6XbsP+XGvpPu\nd00AXPRGglum0aN1Kg198+7YR5SPWm51cdK7kZOXQUFhPtdSL+Bfs4mlwxKlENoxxZQM7DrqwZNd\nE3HSG+9zlBBCCFtRpmTAz88PgISEBPz9/U3bExISTPv8/PwwGAwkJyeb3R1ISEigZ8+epj6JieYL\nEimKwvXr181ep2jIUJGkpCQMBoOpz5106tSpDN+hKKnoE4CynlOj0cCe49s5emYXdg4a7Byc0aAh\npE1/urTs81AThQ0GhSNnYfcRiDwCp+KK97k4Fz+306njnod2hZ7tQe/gDtQr0/dTVuV1Xq2d0SWd\nnX9sACBbm0SnTqMq9P2qy3mtaB07KmyMgfM375T9fsSTAZ1uyHktZ/LvtWLIea0Ycl4rzq2VNCtD\nmeYMNGrUCD8/P8LDw03bcnNziYiIICQkBICOHTtib29v1ufy5cucOHHC1Cc4OJjMzEzTHAJQ5whk\nZWWZ+oSEhBAbG2tWknTLli3o9Xo6duxYlm9DVKKs3AxW/f5/7D6yCUVRP110dnTl6d6TCGnd74ES\ngaRUhfWRCv9YoNBvGvzpA1i83jwRADUB6NIa3noeNs2Fz1/VEPqoBr2DzAWoTK0bBZl+vlcSz5OY\nes3CEYnS0Gg0jO5X3N4cXYNCg+XiEUIIUb7ue2cgKyuL06dPA2A0Grl48SIxMTF4eXlRr149pk+f\nzuzZs2nevDmBgYH861//ws3NjTFjxgDg4eHBCy+8wIwZM/Dx8aFmzZq8+uqrtGvXjr591dlpLVq0\nYMCAAUyePJkvv/wSRVGYPHkyQ4cONY2dCg0NpVWrVoSFhfHxxx+TlJTEjBkzmDRpklQSshHnr51k\n097vycwpznrr+QQwJGQsrk7u9z3+eopC9Ak4eAoOnoC4e0wnKLoD0DcIerUHD1e58Lc0Fyd3Auu2\n5uSlPwA4fHYPj3V8wsJRidIYFAwLfoYb6ZCcYc/eE+50efT+xwkhhLB+900G9u/fT58+6owxjUbD\nzJkzmTlzJuPHj2fRokXMmDGDnJwcXnrpJVJSUujSpQvh4eG4uLiYXuPTTz/Fzs6OkSNHkpOTQ9++\nfVm2bJnZvILly5czdepU+vfvD8Dw4cOZP3++ab9Wq2X9+vVMmTKFrl274uTkxNixY/noo4/K7WSI\nipGXn8Nvh37hyLl9Ztu7tOpL19ahd7wbYDAonL0CR8/BsfNw6BRcus9cYt+aENIGuraFTs3BxUkS\nAGvTrkkXUzJw7PwBerQbjL1d2eaHiIqnd9DwzGMKC35W2xsPePGXMEUqbQkhRBVw32SgV69eZouD\n3UlRgnA3Dg4OzJs3j3nz5t21j6enJ0uXLr3n+9SrV49169bdO2BhVc5djSV8349klLgb4KR3YVCX\nMTSu0xxQa/5fvg4n44ov/k/GQe595vTqHaB1YzUBCGmjLg4mFyfWrZ5vE2q41SIlI4m8glxOxMXQ\npnFnS4clSuGp3rB4A2RlQ1yinr3H1OF3QgghbFvl1PYT1U5OXja/HVrLsfPmZcfqeXeivu8wDsQ6\ns2KLwplLcObK/S/8Qb34bxMAHZtBx+bQqhGyGrCN0Wq0tA3owu8xvwLqisSSDNgGD1cNw7opfHPz\n85ilmyUZEEKIqkCSAVGuFEXh+IUj/LxrG1cT7UlJ70BKRg0ysn1AaUp+gVupX8vPC1o2Ui/62wao\nz+Xi3/a1bhRExOFNGIyFXEuO43rKFXxq1L3/gcLixoTCkl/BqMC+Y3DyokKzBvJ/UgghbJkkA+KB\n5eUrJKVBwg24lgxXEuFqEpy7ks2x84kkpnqgKE+a+jvpnXFz9kSruXulIC8PCPBXL/yLHrU85SKj\nKnJ2dKVpvTbEXjwEwB9n9tAv6Mn7HCWsQV1vDZ2bprPnpDrhf+lm+NckCwclhBCiTCQZECY5eQrJ\naZCcdrNqSPrNr2lwPQVOnG3IjQx7jLdcoxsVA5nZaWTnZ4NSXK1Wq9Xh7lIDR/viRb/0DtDAD5r4\nQ2A9CPRXn3t5yIV/ddI2oIspGTh+8SA9HxmCg73ewlGJ0hgYdMOUDGzdDy+NUKhdS/7/CiGErZJk\noIpRFIWcPMjIhsxsSM9Wn2dkQ3qW+jUtE1JvPtJLPL/fuP2sbEegeBEvBYXs3EyyctIxKsWTzDUa\nqFPLnrYBXjSua08DPzUBaOinDv3RauXCobqr5xNATXcfbqRfJ78glxNxh2gb0MXSYYlSaOSXS4t6\n2cQlu2AwwIqt8GrFrh8nhBCiAkkyYCGKolBogLx8yCuA3Hz1Yjyn6OvNR26++jX7Zjsr5+bX3OKv\n2TmQefORnQv3Kf5UZlqtgpM+jUJDHO4uSbi7pOHhmoabSzqtG9ViWLfHqFPr7qtCC6HRaGgb0IUd\nh34B1KFCkgzYjsGdk/liozcAa36HiUMV3F0kyRdCCFtU5ZOBwkL1ottghELDHR6Ft28rKLz5uLm/\n4Oa2/AJ1f36B+baCQvWCvqidX/LrzYv9/AL1a9HFf14BGKxsFU8He6jpDl7uUNPj5nMPqOkGPjUg\n4co5cgtPk6c7RXp2ktmxNd286dV+GI3rtJDynqJUWjfqxK7DGzAYCom/cYmEG5fxrelv6bBEKbRp\nmPbr1wwAACAASURBVEUTfzhzWf1Q4qcd8PxgS0clhBDiYVT5ZCC4Gk5uc9SDqxO4OYOrs/rV3Rnc\nXdS2hwt4uoKnm/rVw1X96ux45zr9BkMhxy4cYP/x1eTkZeBcNE4I0Ns7EtI6lPaBXdHpqvw/J1GO\nnPQuNKvXluMXDgJqmdHQzk9bOCpRGhoNPNsfZn2ttn/YBs+GKlLtSwghbJBcvVmQnU6dUOvoAHp7\n9SLeyQH0enWbU4mvznr1Yr3o4eSgfnVxUi/8XW4+d3EEO7vy+YOcm5/D0XP7iD65i/TsFHLysk37\nHB2c6disBx2adsPRwekeryLE3bUNCDYlA7EXD9Gz/VD09o4WjkqURv9H4YvVanGBpFTYuAeGd7d0\nVEIIIR5UlU8G7HSg0938qlUfdnZq+7aHHdiX2Ff03OHm9qKHg93Nvjp1aI2Dvbqt6Ku9nXqRry+x\nz1Ff3Kfo4r+8LtrL2/WUq8Sc3s3xCwcpMOSb7bO309Ot7UA6NO0mF22izPy9G+Hl7ktyegL5hXmc\nuHiIdk2CLR2WKAV7Ow0j+yp8/qPaXrYJhoQo6HTW+XtNCCHEnVX5ZCDqK/nDVBoGo4FTlw4TczqS\ny4nnbtvvrHelvkcr6ns1J7hViAUiFFWRRqOhXZMubD+4FiieSCzzTmzDiJ7wzXq1ctmFa7A9GvrJ\ngtJCCGFTqnwyIO5OURSuJccRe/EgJ+JiyM7NvK2Pt2dt2gd2pUXDDhyOOWKBKEVV17JhJ3b+sYFC\nQwEJKZeJv3GJ2l71LR2WKAVXZw3PPKawaJ3aXvQrPNZJkfLBQghhQyQZqIZupF8n9uKh/2/vzsOb\nqvL/gb9v0iRNuqT7jrSlhZYWCxQKrcgiyiKIw+iwOICgP3cZFB0ddOYRx5Vn/DqODrjghjoMoOIG\njICyVsralq0FWtpSWtqUlu57k/P7I3BLZJG2Kbdp36/nuU/uufck+eTTEPLJufdcZJ5KRXl16SX7\nVSo1+oYMwMDImxDiG8ZfaalT6XUG9OsVh6N5+wFYTyRmMeA4Zt4K/HeTdVah7AJg50Fg1CCloyIi\nomvFYqAHEELgbEURTp7JQHbBERSfO33Zfi56d8T1GY4b+wyHm8F4naOkniwuYrhcDBw7lY7Rg6bw\nxHQH4eEm4a4xAl/8aG1/vA4YOVDwRwQiIgfBYqCbam5pxumSkzh5JgM5hRmoqiu/bD+txhl9Qwag\nf+hg9PLrA5VKfZ0jJQKCfELh6xGIsxVFaDY3ISPvAAb3HaF0WHSNZo23Ti/a1Axk5AK7jwKJsUpH\nRURE14LFQDdhsZhhKi/E6ZKT1sV08pKZgC5QqdToExSN6N6DER7UHxonzXWOlsiW9UTiRPy0fy0A\nID1rFwZF3sRflx2Et1HC70YKrPnZ2v7oB2B4DEcHiIgcAYsBB9Xc0oyS8kIUnD2J0yU5KDybi6aW\nxiv212mcERYYhfCgaIQHRUOvc7mO0RL9tv6h8diRvh5NLY0oqzLhdMlJ3OAfoXRYdI3mTATWbrNe\npf1gFpB6HIiPUjoqIiL6LSwGHIDZ3ILSymIUnzuN4nMFMJ0rwNnKIlgs5qvez8vdD+FB0egT1B/B\nvmFQ8xAg6sJ0GmdEhw7GwewUAEB6dgqLAQfi7yVh8giBb7db2x/+wGKAiMgRsBjoQiwWMyprz6Gs\nqgSlFcUoqypGWWUJSiuLYba0/Ob93Q2eCPELRy+/Pujl1weebj7XIWoi+xkYkSQXA1kFh1FbXwUX\nvbvCUdG1mns78EMyYDYD+zOBg1kCcZE8VIiIqCtjMXCdWSxmVNdVoqKmFBU1ZTbLuaoStJibr/mx\nvNx8EejTW/7yb3Tx4jG65ND8PIMQ7BuGwrO5sFjMOJSzF4kxtyodFl2jYF8J44cJbNhlbX+8HvjX\nE8rGREREV8diwI7M5hbUNtSgrrEa1XWVqK6rOL+cX6+33v7W4T2XY3TxQoBXLwR494K/ZwgCvEKg\n49SL1A0NjEhE4dlcAMCh7N0YFj2Gs1w5kHmTgP+lAEIAuw4BmXkC0aH8kYKIqKtiMXAFQgg0m5vQ\n0FiPhqZa1DfWob6xFvUXrzfWoLahBrUNVairr0Z9U12Hn9fF2Q3e7v7w8QiAt7s/vI3+8DEG8IRf\n6jH69roRW1K/Q31jLarqypFTdAwRwTFKh0XXKDRQwq1DBTbvtbY/Xgf843FlYyIioivr9sWAqbwQ\nzS2NaGpust62NKKxqd5629yApuYLt/VoaKpHY5P1tqG5vl2/4F8Lg7MrPFx94OHqbbN4uvvCoHPt\nlOckchROag0GhCdgb+ZWANZpRlkMOJZ5kyAXA9tSgewCgYgQjg4QEXVF3b4Y+OzHN6/bc0mSCgad\nC1yc3eCid4ebweP8YoS7wQOuBiPc9B7QanTXLSYiRxQXkYh9mdsgIJBbdAzl1aU8Id6BRPaSMHKQ\nwI40a/u9b4E3ODpARNQldftioCOc1BrotHo4a/Uw6Fyh17lArzVYb88vBmc365d/Z+t+HttM1HEe\nrt4IC4pCzplMAMChk7sxauBkhaOitnhwCuRiYHsqcDRHICacowNERF1Nty8GfD2CoHXSQuOkg1aj\nhUatg07rDK3GGbrzi1bjDK2TDs5a/fkv/wboNHpemZdIQQMjEuVi4HDOXiTFjue/SQfSr7eE2xJa\nzx1Y9g2w9CllYyIiokt1+2Jg7kT+70PkiMICo+Du4omq2nLUN9bixOmDiAkbonRY1AYP/Q74eT9g\nsQB7jwIHjgnER3F0gIioK1EpHQAR0eWoVGrE9UmU2+nnL0ZGjqN3gITJN7W2l31jnamNiIi6DhYD\nRNRlDeiTIJ+Hc6Y0DyXlhQpHRG31/+4AnM6fSnUoC/jlkLLxEBGRLRYDRNRluTi7oV+vG+V2WtYv\nCkZD7RHoI+H3o1vb730DWCwcHSAi6ipYDBBRlzYwMklez8hLRV1jjYLRUHvcNxlwPj+j8vF863kE\nRETUNbAYIKIuLdgnDP6eIQCAFnMzDmXvUTgiaitvo4TpY1vbH3wHmM0cHSAi6gpYDBBRlyZJEuL7\n3Sy307N+gbmTrg5OnWf2BMDVYF3PKwLW71I2HiIismIxQERdXr8b4uDi7AYAqK6vxInTPAvV0Rhd\nJcwa39r+8AegqZmjA0RESmMxQERdnpNag7iI1mlGU4/vVDAaaq8ZtwKe1poORaXAtzuUjYeIiFgM\nEJGDGBiRCLXKep3EM2WncKb0lMIRUVu56CXMndTa/vB7oLaeowNEREpiMUBEDsFF746o3gPlduoJ\njg44ortGAwHe1vXyauDjdYqGQ0TU47EYICKHEd9vpLx+/PQhVNdVKBgNtYdOK+Hxu1vb/90MFJ7l\n6AARkVJYDBCRw/D3DEaIbzgAwGIxIz07ReGIqD3GJQCx1j8jmluApV8rGw8RUU/GYoCIHMrF04we\nzE5Bc0uzgtFQe0iShCemt7Y37wUOZXN0gIhICSwGiMihRATHwN3FEwBQ31iLzFOpCkdE7REXKeHW\noa3tf64ChGBBQER0vbEYICKHolKpMSjyJrmdemInv0Q6qMfvBjTWCaJwJMc6QkBERNcXiwEicjgD\nwodBo9YCAM5WFCHflK1wRNQewb4SZt7W2n7nK6CxiYUdEdH1xGKAiByOXmdATNgQuc1pRh3XvEmt\nFyIrLrPOLkRERNcPiwEickiD+42Q108WZqCs0qRgNNRergYJD/2utf3pBqCskqMDRETXC4sBInJI\n3u7+6BPUHwAgILAnc4vCEVF7/W4kEB5sXa+tBz74Ttl4iIh6EhYDROSwhvUfK69n5qWisuacgtFQ\ne6nVEhZMa21/uwM4kc/RASKi64HFABE5rGDfUPTy6wMAsAgL9h3bpmxA1G5JAyQMj7WuWyzA618A\nFgsLAiKiztbhYmDx4sVQqVQ2S1BQ0CV9goODYTAYMGbMGGRkZNjsb2xsxPz58+Hr6wtXV1fceeed\nKCwstOlTXl6O2bNnw8PDAx4eHpgzZw4qKys7Gj4RObjhMbfK64dy9qC2vkrBaKgjnr4HcFJb1w9n\nAz/8omw8REQ9gV1GBqKiolBcXCwvhw8flvctWbIEb775Jv79739j37598PPzw2233Yaamhq5zxNP\nPIG1a9di1apV2LlzJ6qqqjB58mRYLBa5zz333IP09HRs3LgRP/74I1JTUzF79mx7hE9EDqy3fyQC\nvHoBAMzmFuw/vkPhiKi9egdImHN7a/udL4GKao4OEBF1JrsUA2q1Gn5+fvLi7e0NwHo1ybfeeguL\nFi3C1KlTERMTgxUrVqC6uhorV64EAFRWVuLjjz/GG2+8gbFjx2LQoEH4/PPPcejQIfz0008AgMzM\nTGzcuBEffPABhg0bhuHDh+P999/HunXrcOLECXu8BCJyUJIk2Zw7kJ6dgvrGOgUjoo64bxIQ7Gtd\nr6yxXnuAiIg6j12KgZycHAQHByM8PBwzZ85Ebm4uACA3Nxcmkwnjxo2T+zo7O2PkyJHYtWsXAODA\ngQNobm626RMSEoLo6GikpKQAAFJSUuDq6orExES5T1JSElxcXOQ+RNRzRYTEwNvoDwBoam5AWhaP\nL3FUOq2Ep2a2tr/fCRzM4ugAEVFn6XAxMHz4cKxYsQIbN27E8uXLUVxcjKSkJJw7dw7FxcUAAH9/\nf5v7+Pn5yfuKi4uhVqvl0YQL/P39bfr4+vra7JckyeZxiKjnUkkqDIu+RW6nHt+BFnOzghFRR9w8\nUMKowa3t1z8HzGYWBEREncGpow8wYcIEeT02NhaJiYkICwvDihUrMGzYsCveT5Kkqz6uEPb54N+/\nf79dHodaMaedg3ntGIuwQDSrUN9Ug7raOuQbjiPcN5Z57SSdnddb+zvh59190Ngi4eAJ4LXlJkwY\nUt6pz9kV8P3aOZjXzsG82l9kZOR1f067Ty1qMBgQExOD7OxsBAYGAgBMJtsrg5pMJgQEBAAAAgIC\nYDabUVZWdtU+Z8+etdkvhEBJSYnch4h6NpWkQh+/G+V2TslhmC1mBSOijvBxb8Hvklo/99fu8kVZ\nVYd/vyIiol+x+ydrQ0MDMjMzccsttyAsLAwBAQHYtGkT4uPj5f3Jycl44403AADx8fHQaDTYtGkT\nZs60HihaUFCAY8eOISkpCQCQmJiImpoapKSkyOcNpKSkoLa2Vu5zJUOGDLH3S+yxLvwCwJzaF/Nq\nP3EtcTCty0FtfRXqautQWJ6F34+fpXRY3cr1fL/GDRTIKAJyzs80vfV4HF5/5Oqjyo6KnwOdg3nt\nHMxr51Fi2vwOjww8/fTT2LFjB3Jzc7Fnzx7cfffdqK+vx7333gvAOm3okiVL8M033+DIkSOYO3cu\n3NzccM899wAAjEYj7r//fjzzzDP4+eefkZaWhtmzZyMuLg633mqdPzw6OhoTJkzAQw89hN27dyMl\nJQUPPfQQ7rjjDkWGU4ioa9I4aTCk30i5fbLkECwcHXBYGicJz15Uy/28D0g5wnMHiIjsqcPFQGFh\nIWbOnImoqCjcdddd0Ov12L17N3r1ss77/cwzz+DJJ5/EY489hqFDh8JkMmHTpk1wcXGRH+Ott97C\n1KlTMX36dIwYMQLu7u744YcfbM4rWLlyJeLi4jB+/HhMmDBBnoKUiOhicRGJcNYaAAB1jdXIPJWm\ncETUEYP7Sbj9ogHg1z4DautZEBAR2Ysk7HWmbhdy8RCL0WhUMJLuhcOCnYN5tb9fDm/ET3u+BQAE\n+AXjvtufhcZJo3BU3YMS79dzVQLT/mq97gAA/H40sGhO9zpciJ8DnYN57RzMa+dR4jus3U8gJiJS\n2pB+I6Fx0gEAqmrLkZ69S+GIqCO83CU8fU9re+02YG9Gt/sdi4hIESwGiKjb0Wn1iPQfJLd3H/2J\nVyV2cOOHwebaAy99wsOFiIjsgcUAEXVLvb2jYNC5AQAamuqwN3OLwhFRR0iShEWzAaOrtV1cBvxr\njbIxERF1BywGiKhbUqnU6BfYejzrgRM7UVXb/S9a1Z15GyU888fW9jfbgT1HOTpARNQRLAaIqNsK\nNIYh0PsGAIDZ3ILkQz8qHBF11G0JwJj41vbLn/JwISKijmAxQETdliRJGBk3SW5n5B2AqbxQwYio\noyTJeu0BD+sRYCguA/65WtmYiIgcGYsBIurWbvCPQJ+g/gAAAYEd6esVjog6ytso4c8XzS703Q5e\njIyIqL1YDBBRt3dz3O2QJOvHXV7xceQVn1A4Iuqo2xKAWy6a4vyVT4HKGhYERERtxWKAiLo9X49A\nDAgfKre3p6+DRVgUjIg66teHC5nOAS99CnTD62gSEXUqFgNE1CMkxY6HRq0FAJSUFyIzL1XhiKij\nvNwlPH9va3t7KrDmZ+XiISJyRCwGiKhHcDMYER81Um7vPLgBjU31CkZE9jB6sIRpY1vb/1oDZOZx\ndICI6FqxGCCiHiMhegwMztarVlXXV2L7QZ5M3B0smAZE9bauN7cAz73H6UaJiK4ViwEi6jF0GmeM\nHTxVbh/MTkG+KVvBiMgetBoJrz4MuOit7YIS4NXPeP4AEdG1YDFARD1KvxviEBEcI7c37l2D5pYm\nBSMie+jlL+G5Oa3tTXuA73YqFw8RkaNgMUBEPYokSbh1yF3QaZwBABU1ZUg+zCsTdwfjhkmYOqq1\n/cZK4GQBRweIiK6GxQAR9ThuBiPGDL5Tbh84vhNFZfkKRkT28tRMICLEut7YBCx6D6hvZEFARHQl\nLAaIqEeKDRuK3gF9AQBCWPDjntUwm1sUjoo6Sqe1nj/grLO2c88AL33C8weIiK6ExQAR9UiSJGHc\n0Lvlaw+UVhZjdwYnqe8OwoKsFyS7YPNeYPn3ysVDRNSVsRggoh7Lw9UbN8fdLrd3Z/yMsxVFCkZE\n9jL5Jgl/uOj6A8u/Azbt4egAEdGvsRggoh5tUGQSgnxCAQAWixkb966BxWJWNiiyi6dmAMNaJ47C\nix8DR3JYEBARXYzFABH1aCqVGhMSpkGtcgIAFJXlY+eh/ykcFdmDWi3htYeB0EBru6kZePodoLiM\nBQER0QUsBoiox/M2+iMpdpzc3pu5FZmn0hSMiOzFzUXCPxcAHm7Wdlkl8NQ7QF0DCwIiIoDFABER\nACCh/xj0CeovtzfuWQNTeaGCEZG9hPhJWPIo4KS2tk/kA39bDpjNLAiIiFgMEBEBUEkqTEq8B17u\nfgCAZnMTvt35CeoaaxSOjOxhcD8Jiy66QvGONOCdr5SLh4ioq2AxQER0nk6rx+9ungvt+asTV9WW\n44fkz3lCcTcx5WYJsye2tv+zEfjwe44OEFHPxmKAiOgi3u7+mJR4j9zOL8nGtvR1CkZE9vTY74FR\ng1vb738LrNjAgoCIei4WA0REvxIRHIObBkyQ2weO78DR3P0KRkT2olZLeOVBIOGiKUf//RWwchML\nAiLqmVgMEBFdxvCYsYgMGSC3N+77EkVl+QpGRPai00r4v8eB+KjWbf9cBaz5mQUBEfU8LAaIiC5D\nJakwcfgMeBv9AQBmcwu+3rYcpnMFCkdG9uCss045OrBv67Z//Af4ZjsLAiLqWVgMEBFdgU7jjKk3\nz4NOqwcA1DfVYfXW9zhC0E3odRLeWgAMiGjd9tpnwA/JLAiIqOdgMUBEdBWebr74w+iH5IKgsake\nX259H4Vn85QNjOzCRS/h7SeA/mHWthDAS58AX29lQUBEPQOLASKi3xDo3QvTb3kEep0LAKCxuQFf\nbfsABWdzFI6M7MHVIOGdhUC/G6xtIYDXPwfeWi14YTIi6vZYDBARXQN/z2BMG/MwDDpXAEBTSyO+\n2roc+aZshSMje3B3kbD0aSA6tHXbfzYCz74L1DeyICCi7ovFABHRNfLzDML0sY/AxdkNgPUqxWu3\nf4RTxScUjozswegq4f1nba9DsD0VeGgJUFrBgoCIuicWA0REbeBjDMCMsY/CVW8EYC0Ivt7+EdKy\nfoEQ/MLo6PQ6CUseAf44vnVbZh4w7xUg6zT/vkTU/bAYICJqIy93P8wY+wjcDZ4AALOlBT/tX4t1\nu75AY3ODwtFRR6nVEp6YLuEvswG12rqtuAx44HUg+SALAiLqXlgMEBG1g6ebL2aMfRT+niHytmP5\n6fhi41soKT+jYGRkL3eNsV6LwMU6kRRq64En/wX84z8CDTyPgIi6CRYDRETtZHT1wsxbH0dcRKK8\n7Vz1Wfxn89s4dHIPDxvqBhJjJXy4CAjwbt225mdg9t+BjFz+fYnI8bEYICLqAI2TBuOG3o1JiX+E\nxkkLAGgxN2Pj3jX43+5VPGyoG4gIkbDib8DIQa3b8oqA+18FPvqB048SkWNjMUBEZAf9Qwdj9vgn\n4GMMkLcdzduPj9YvwdHc/bAIi4LRUUd5uUt443Hgr/MAvc66rcUMvPeN9VyC0yYWBETkmFgMEBHZ\nibe7P2aNW4DYsKHyttr6KmzY/V/8d/O/UVSWr2B01FGSJOHOmyWsfBG4MbJ1++GTwD2Lgfe/Fait\nZ1FARI6FxQARkR1pnLSYOHwGJiX+ES56d3n7mbJT+GLTv/C/PatRW1+lYITUUSF+Ej54Bnj0LsDp\n/GxDDY3Ah98Ddz0HfL1VoKWFRQEROQYWA0REnaB/6GDcP+lZJESPgVrlJG8/krMXH65fgpSjP6G+\nsU7BCKkj1GoJ8yZJ+OSvQN8bWreXVQKvfw7MfAHYlip4EjkRdXksBoiIOolO44xRAydj3u1PIyI4\nRt7e1NyA5EP/w/vfv4SfD3yD8upSBaOkjojqLeGzvwEv3A/4e7VuzysC/vxv6/kEezNYFBBR1+X0\n212IiKgjPN18MXXkfcgtOoYtqd/hXFUJAKC5pQmpJ5KRduIXRITEYmjUKAT5hEKSJIUjprZQqyVM\nvgm4bajAqp+ATzcANecHfQ5mAY+9AYQGAnePEZh8E+Ci59+XiLoOFgNERNdJWGAU5k6MREbeARw4\nvgNnK4oAAAICWQWHkVVwGIHeN6B/aDz69brR5pwD6vp0Wgn33g7cebPAR+uAr7ZYZxwCrCMFb6wE\nlq0FJiYK/GEM0CeERQERKY/FABHRdaRWqTEgPAGxYUNxypSF/ce2I7fomLy/qCwfRWX52JL6HXr5\n9UHUDXHo2+tG6HUuCkZNbeHhJuGpmcD0sQL/3Qys32W9ejEA1DUAX2+1LnGRArfEA6MHXf3xiIg6\nE4sBIiIFSJKE0IC+CA3oi7MVRThwfAcyTqXCbG4BAAhhQb4pC/mmLPx04Bv09o9En+AY9PaPgKeb\nLw8lcgAhfhL+/Efg0d8L/G838OUWIKewdf/BLOvyz1WAj2soBkfUwOgvEBEC/n2J6LphMUBEpDBf\nj0BMGDYdI+Mm4fjpQzien46CkhwIWE86tVjMyC06Jo8guBk8cINfBHoHRKKXXwTcXTyUDJ9+g4te\nwt1jgLtGC6QeB9ZsAbanAWZza59TJc44VeKMTelAkA8QHyUQFwnERQC9A1gcEFHncchiYNmyZfjH\nP/6B4uJixMTE4K233sKIESOUDouIqEMMzq4YFJmEQZFJqK6rxInTB3Es/yDOlObZ9Kuuq8DRvP04\nmrcfAODh6o0Ar17w9wqBn2cw/D2DeVhRFyRJEuKjgPgooKxSYEc6sC0N2Jdh2+9MKXAmGfgh2do2\nugID+gjERQD9w4A+wYCXOwsEIrIPhysGVq9ejSeeeALvvvsuRowYgaVLl2LixInIyMhAr169lA6P\niMgu3AxGxPcbifh+I1FZew7ZBUdwypSNgpKTaGxusOlbUVOGipoyHMtPl7e5Gzzh5xkEb6M/PN18\n4eHqAy83Hxic3fglsgvwNkqYOgqYOgqorRf49OtCHMh2Q26pizwT0QWVNUDyQetygbsLEBYkEB4E\nhAUBYYFAoA8Q4GU9kZmI6Fo5XDHw5ptvYt68ebj//vsBAG+//TZ+/PFHvPvuu3j11VcVjo6IyP6M\nLl5yYWCxmFF8rgD5pmzkl2Sj8GwuWszNl9ynqq4cVXXlyC48arNdq3GGp6sPjK5ecNUb4ap3P78Y\n4WYwwkXvDq2TjgXDdeSilzAsqhrDoqpxY1wgMnKBg9nW5fBJoKL60vtU1baec/BrXu4CAd6wLl6A\nt9E6kuDpZl0urLNoICLAwYqBpqYmpKam4plnnrHZPm7cOOzatUuhqIiIrh+VSo0gn94I8umN4TFj\n0WJuxtmKYpSUF8BUXoiS8kKUVJyRT0T+tabmBpjKC2AqL7jic6hVTnDWGaDXGqDXGeCsdYFeZ4BO\no8eZ4iI4qbXQZLdAq9FB66SDxkkLjZMWapUTnNROcFJr4aR2glrtBLVKDZVKDZXEa1xeC61GwsC+\nwMC+1rYQAqeKgUPni4OcQiC3qHV2oss5V2VdMnKv/lx6nYCrAXDVo/X2/KLXWRdn7flbHaDXAjot\noNUAWidA4wToNNZbrQZwUl9+Uat5SBNRV+ZQxUBpaSnMZjP8/f1ttvv5+aG4uFihqIiIlOOk1iDQ\nuxcCvVsPkzRbzCirNOFsxRmcqz6LypoynKs+i4rq0ksOMbocs6UFtfVVqK2vumRfXa31GJZTFYfb\nFKdKUlmLApW6tUCABElSQVKpoJJUkCTp/K11XYIESJALCWtbggScv5Vg/Y4pte6X91lvf03uc8lu\n277X+8tricl6IbpTtenno7n0+WPCrYsQQGWNM4rPuaOk3BWmc24orXBFebUeVbV6WMS1xV5eA6DM\nbi/hqlSSgCQJqFXWW5UkoLpo3fp3tkCSAEkSkIDWfYC8/eJ1uY0L7dbnk2B9zIZ6J0iSgP7z1ul7\nL/STIC6+g+19cdm3z/nt4qK+v955+StN2/vddF3enld4LQBQV2s9+92wpm2fA0q5dYgWC6b1UzqM\nLsuhioH2qKysVDqEbiMyMhIAc2pvzGvn6Ol51alcEOIViRCvSKVDISJSXE/9v+BaONS4rY+PD9Rq\nNUwmk812k8mEwMBAhaIiIiIiInJMDlUMaLVaxMfHY9OmTTbbN2/ejKSkJIWiIiIiIiJyTA53PqsA\nSAAACwxJREFUmNDChQsxe/ZsJCQkICkpCe+99x6Ki4vx8MMPy32MRqOCERIREREROQaHKwamTZuG\nsrIyvPzyyygqKsKAAQOwYcMGXmOAiIiIiKiNJCHElU8XJyIiIiKibsuhzhm4FsuWLUNYWBj0ej2G\nDBmC5ORkpUNSzI4dOzBlyhSEhIRApVJhxYoVl/RZvHgxgoODYTAYMGbMGGRkZNjsb2xsxPz58+Hr\n6wtXV1fceeedKCwstOlTXl6O2bNnw8PDAx4eHpgzZ84lZ+3n5+fjjjvugKurK3x9fbFgwQI0N196\noSRH8Nprr2Ho0KEwGo3w8/PDlClTcPTo0Uv6Mbdts3TpUsTFxcFoNMJoNCIpKQkbNmyw6cOcdtxr\nr70GlUqF+fPn22xnbttm8eLFUKlUNktQUNAlfZjTtisqKsK9994LPz8/6PV6xMTEYMeOHTZ9mNu2\nCQ0NveT9qlKpMHnyZADW61kwp23X0tKC5557DuHh4dDr9QgPD8ff/vY3mM1mm35dPreiG1m1apXQ\naDTiww8/FMeOHRPz588Xrq6uIj8/X+nQFLFhwwbx/PPPi6+++koYDAaxYsUKm/2vv/66cHNzE2vX\nrhVHjhwR06ZNE0FBQaK6ulru8/DDD4ugoCDx008/idTUVDF69GgxcOBAYTab5T4TJkwQsbGxYvfu\n3SIlJUXExMSIO+64Q97f0tIiYmNjxZgxY0RaWprYvHmzCAoKEvPnz+/8JHSC8ePHi08//VQcPXpU\nHD58WEydOlUEBASIc+fOyX2Y27b77rvvxI8//ihOnjwpsrKyxPPPPy80Go1IT08XQjCn9pCSkiLC\nwsJEXFyczethbtvuhRdeENHR0cJkMslLaWmpvJ85bZ/y8nIRFhYm7r33XrFv3z6Rl5cntmzZIjIz\nM+U+zG3blZaW2rxX09LShEqlEp999pkQgjltrxdffFF4eXmJdevWiVOnTonvv/9eeHl5iZdeeknu\n4wi57VbFQEJCgnjwwQdttkVGRopFixYpFFHX4erqalMMWCwWERAQIF599VV5W319vXBzcxPvv/++\nEEKIiooKodVqxcqVK+U+p0+fFiqVSmzcuFEIIURGRoaQJEns2rVL7pOcnCwkSRInTpwQQliLEpVK\nJQoKCuQ+X3zxhXB2drb5x+CoampqhFqtFuvWrRNCMLf25OXlJT744APm1A4qKipEnz59xLZt28To\n0aPl/yCY2/Z54YUXRGxs7GX3Maftt2jRIjFixIgr7mdu7ePll18Wnp6eoqGhgTntgMmTJ4u5c+fa\nbJszZ46YPHmyEMJx3q/d5jChpqYmpKamYty4cTbbx40bh127dikUVdeVm5sLk8lkky9nZ2eMHDlS\nzteBAwfQ3Nxs0yckJATR0dFISUkBAKSkpMDV1RWJiYlyn6SkJLi4uMiPk5KSgv79+yM4OFjuM27c\nODQ2NuLAgQOd+jqvh6qqKlgsFnh6egJgbu3BbDZj1apVaGhowMiRI5lTO3jwwQfxhz/8AaNGjYK4\n6FQx5rb9cnJyEBwcjPDwcMycORO5ubkAmNOO+Pbbb5GQkIDp06fD398fgwYNwtKlS+X9zG3HCSHw\n0UcfYdasWdDpdMxpB0ycOBFbtmzB8ePHAQAZGRnYunUrJk2aBMBx3q8ON5vQlZSWlsJsNsPf399m\nu5+fH4qLixWKquu6kJPL5evMmTNyH7VaDW9vb5s+/v7+8v2Li4vh6+trs1+SJJu8FxcXX/I8Fy4g\n1x3+NgsWLMCgQYPkf6TMbfsdPnwYiYmJaGxshF6vx5o1a9CvXz/5w445bZ/ly5cjJycHK1euBGB9\nzRfw/do+w4cPx4oVKxAVFQWTyYSXX34ZSUlJOHr0KHPaATk5OVi2bBkWLlyI5557DmlpafL5LY89\n9hhzawebN29GXl4eHnjgAQD8DOiIRx99FAUFBYiOjoaTkxNaWlrw17/+VZ7u3lFy222KAbKfi78o\nXI5oxwRU7bmPI1i4cCF27dqF5OTk38wbwNz+lqioKBw6dAiVlZX48ssvMWPGDGzduvWq92FOr+74\n8eN4/vnnkZycDLVaDcD6+q7lNTK3VzZhwgR5PTY2FomJiQgLC8OKFSswbNiwK96POb06i8WChIQE\nvPLKKwCAuLg4ZGVlYenSpXjssceuel/m9tosX74cCQkJGDBgwG/2ZU6v7u2338Ynn3yCVatWISYm\nBmlpaViwYAFCQ0Nx3333XfW+XSm33eYwoQvVj8lkstluMpkQGBioUFRdV0BAAABcNl8X9gUEBMBs\nNqOsrOyqfc6ePWuzXwiBkpISmz6/fp4LIzkX+jiiJ598EqtXr8aWLVsQGhoqb2du20+j0SA8PByD\nBg3Cq6++iuHDh2Pp0qXyv2HmtO1SUlJQWlqKmJgYaDQaaDQa7NixA8uWLYNWq4WPjw8A5rajDAYD\nYmJikJ2dzfdrBwQFBaF///4226KiopCfnw+An68dVVJSgu+//14eFQCY04545ZVX8Nxzz2HatGmI\niYnBrFmzsHDhQrz22msAHCe33aYY0Gq1iI+Px6ZNm2y2b968GUlJSQpF1XWFhYUhICDAJl8NDQ1I\nTk6W8xUfHw+NRmPTp6CgAMeOHZP7JCYmoqamRj6uDbB++aitrZX7JCUlITMz02aarM2bN0On0yE+\nPr5TX2dnWbBggVwI9O3b12Yfc2s/ZrMZFouFOe2AqVOn4siRIzh48CAOHjyI9PR0DBkyBDNnzkR6\nejoiIyOZWztoaGhAZmYmAgMD+X7tgJtuugnHjh2z2XbixAn5BxfmtmM+/fRTODs7Y+bMmfI25rT9\nhBBQqWy/SqtUKvkXeofJ7VVPL3Ywq1evFlqtVnz44YciIyND/OlPfxJubm49dmrRmpoakZaWJtLS\n0oTBYBB///vfRVpampyPJUuWCKPRKNauXSsOHz4spk+fLoKDg0VNTY38GI888ogICQmxme5q0KBB\nwmKxyH0mTpwoBgwYIFJSUsSuXbtEbGysmDJlirzfbDaLAQMGiFtuuUWe7io4OFj86U9/un7JsKNH\nH31UuLu7iy1btoiioiJ5uThvzG3bPfvss2Lnzp0iNzdXHDp0SPzlL38RKpVKbNq0SQjBnNrTqFGj\nxOOPPy63mdu2e+qpp8T27dtFTk6O2L17t5g0aZIwGo38fO2gffv2CY1GI1555RWRlZUl1qxZI4xG\no1i2bJnch7ltH4vFIiIjIy+ZdVEI5rS9HnjgARESEiLWr18vcnNzxdq1a4Wvr694+umn5T6OkNtu\nVQwIIcSyZctEaGio0Ol0YsiQIWLnzp1Kh6SYrVu3CkmShCRJQqVSyevz5s2T+yxevFgEBgYKZ2dn\nMXr0aHH06FGbx2hsbBTz588X3t7ewmAwiClTpthMWyWEdV7oWbNmCXd3d+Hu7i5mz54tKisrbfrk\n5+eLyZMnC4PBILy9vcWCBQtEU1NT5734TvTrfF5YXnzxRZt+zG3bzJ07V/Tu3VvodDrh5+cnbrvt\nNrkQuIA5tY+Lpxa9gLltmxkzZoigoCCh1WpFcHCwuPvuu23mwheCOW2v9evXi7i4OOHs7Cz69esn\n3nnnnUv6MLdtt2XLFqFSqcS+ffsuu585bbuamhrx1FNPidDQUKHX60V4eLh4/vnnRWNjo02/rp5b\nSYhudCYHERERERFds25zzgAREREREbUNiwEiIiIioh6KxQARERERUQ/FYoCIiIiIqIdiMUBERERE\n1EOxGCAiIiIi6qFYDBARERER9VAsBoiIiIiIeigWA0REREREPdT/B/d1uO30WoozAAAAAElFTkSu\nQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "player1.MakeBeliefs(20000)\n", - "player2.MakeBeliefs(40000)\n", - " \n", - "calc1 = GainCalculator(player1, player2)\n", - "calc2 = GainCalculator(player2, player1)\n", - "\n", - "bids, gains = calc1.ExpectedGains()\n", - "thinkplot.Plot(bids, gains, label='Player 1')\n", - "print('Player 1 optimal bid', max(zip(gains, bids)))\n", - "\n", - "bids, gains = calc2.ExpectedGains()\n", - "thinkplot.Plot(bids, gains, label='Player 2')\n", - "plt.legend();" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Figure 6.4: Expected gain versus bid in a scenario where Player 1’s best guess is\n", - "\\$20,000 and Player 2’s best guess is \\$40,000." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you overbid, you get nothing. Otherwise we compute the difference\n", - "between your bid and the price, which determines your probability of\n", - "winning.\n", - "\n", - "If `diff` is less than \\$250, you win both showcases. For\n", - "simplicity, I assume that both showcases have the same price. Since this\n", - "outcome is rare, it doesn’t make much difference.\n", - "\n", - "Finally, we have to compute the probability of winning based on\n", - "`diff`:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```python\n", - "def ProbWin(self, diff):\n", - " prob = (self.opponent.ProbOverbid() + \n", - " self.opponent.ProbWorseThan(diff))\n", - " return prob\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If your opponent overbids, you win. Otherwise, you have to hope that\n", - "your opponent is off by more than `diff`. `Player`\n", - "provides methods to compute both probabilities:\n", - "\n", - "```python\n", - " # class Player:\n", - "\n", - " def ProbOverbid(self):\n", - " return self.cdf_diff.Prob(-1)\n", - "\n", - " def ProbWorseThan(self, diff):\n", - " return 1 - self.cdf_diff.Prob(diff)\n", - "```\n", - "\n", - "This code might be confusing because the computation is now from the\n", - "point of view of the opponent, who is computing, “What is the\n", - "probability that I overbid?” and “What is the probability that my bid is\n", - "off by more than `diff`?”\n", - "\n", - "Both answers are based on the CDF of `diff`. If the\n", - "opponent’s `diff` is less than or equal to -1, you win. If\n", - "the opponent’s `diff` is worse than yours, you win. Otherwise\n", - "you lose.\n", - "\n", - "Finally, here’s the code that computes optimal bids:\n", - "\n", - "```python\n", - " # class Player:\n", - "\n", - " def OptimalBid(self, guess, opponent):\n", - " self.MakeBeliefs(guess)\n", - " calc = GainCalculator(self, opponent)\n", - " bids, gains = calc.ExpectedGains()\n", - " gain, bid = max(zip(gains, bids))\n", - " return bid, gain\n", - "```\n", - "\n", - "Given a guess and an opponent, `OptimalBid` computes the\n", - "posterior distribution, instantiates a `GainCalculator`,\n", - "computes expected gains for a range of bids and returns the optimal bid\n", - "and expected gain. Whew!\n", - "\n", - "Figure [fig.price5] shows the results for both players, based on a\n", - "scenario where Player 1’s best guess is \\$20,000 and Player 2’s best\n", - "guess is \\$40,000.\n", - "\n", - "For Player 1 the optimal bid is \\$21,000, yielding an expected return of\n", - "almost \\$16,700. This is a case (which turns out to be unusual) where\n", - "the optimal bid is actually higher than the contestant’s best guess.\n", - "\n", - "For Player 2 the optimal bid is \\$31,500, yielding an expected return of\n", - "almost \\$19,400. This is the more typical case where the optimal bid is\n", - "less than the best guess." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Discussion\n", - "\n", - "One of the features of Bayesian estimation is that the result comes in\n", - "the form of a posterior distribution. Classical estimation usually\n", - "generates a single point estimate or a confidence interval, which is\n", - "sufficient if estimation is the last step in the process, but if you\n", - "want to use an estimate as an input to a subsequent analysis, point\n", - "estimates and intervals are often not much help.\n", - "\n", - "In this example, we use the posterior distribution to compute an optimal\n", - "bid. The return on a given bid is asymmetric and discontinuous (if you\n", - "overbid, you lose), so it would be hard to solve this problem\n", - "analytically. But it is relatively simple to do computationally.\n", - "\n", - "Newcomers to Bayesian thinking are often tempted to summarize the\n", - "posterior distribution by computing the mean or the maximum likelihood\n", - "estimate. These summaries can be useful, but if that’s all you need,\n", - "then you probably don’t need Bayesian methods in the first place.\n", - "\n", - "Bayesian methods are most useful when you can carry the posterior\n", - "distribution into the next step of the analysis to perform some kind of\n", - "decision analysis, as we did in this chapter, or some kind of\n", - "prediction, as we see in the next chapter." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Decision Analysis\n", + "\n", + "## The *Price is Right* problem\n", + "\n", + "On November 1, 2007, contestants named Letia and Nathaniel appeared on\n", + "*The Price is Right*, an American game show. They competed\n", + "in a game called *The Showcase*, where the objective is to\n", + "guess the price of a showcase of prizes. The contestant who comes\n", + "closest to the actual price of the showcase, without going over, wins\n", + "the prizes.\n", + "\n", + "Nathaniel went first. His showcase included a dishwasher, a wine\n", + "cabinet, a laptop computer, and a car. He bid \\$26,000.\n", + "\n", + "Letia’s showcase included a pinball machine, a video arcade game, a pool\n", + "table, and a cruise of the Bahamas. She bid \\$21,500.\n", + "\n", + "The actual price of Nathaniel’s showcase was \\$25,347. His bid was too\n", + "high, so he lost.\n", + "\n", + "The actual price of Letia’s showcase was \\$21,578. She was only off by\n", + "\\$78, so she won her showcase and, because her bid was off by less than\n", + "\\$250, she also won Nathaniel’s showcase.\n", + "\n", + "For a Bayesian thinker, this scenario suggests several questions:\n", + "\n", + "1. Before seeing the prizes, what prior beliefs should the contestant\n", + " have about the price of the showcase?\n", + "\n", + "2. After seeing the prizes, how should the contestant update those\n", + " beliefs?\n", + "\n", + "3. Based on the posterior distribution, what should the contestant bid?\n", + "\n", + "The third question demonstrates a common use of Bayesian analysis:\n", + "decision analysis. Given a posterior distribution, we can choose the bid\n", + "that maximizes the contestant’s expected return.\n", + "\n", + "This problem is inspired by an example in Cameron Davidson-Pilon’s book,\n", + "*Bayesian Methods for Hackers*. The code I wrote for this\n", + "chapter is available from ; it reads\n", + "data files you can download from\n", + " and\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The prior\n", + "\n", + "To choose a prior distribution of prices, we can take advantage of data\n", + "from previous episodes. Fortunately, fans of the show keep detailed\n", + "records. When I corresponded with Mr. Davidson-Pilon about his book, he\n", + "sent me data collected by Steve Gee at . It\n", + "includes the price of each showcase from the 2011 and 2012 seasons and\n", + "the bids offered by the contestants." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvgAAAEWCAYAAAAThOOMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVFX/B/DPzMAwDIuC7IsgIIooauAuiOaSZW6l5lIZ\npY+lpY9l6q/FynwyfcqlTJ/SSlPLVNQWUnFhMcUdF8RdEWSRfYcZZu7vj8ELIyhowAB+3q9Xr7hn\nzp353iuO3znzPedIBEEQQEREREREzYLU0AEQEREREVHdYYJPRERERNSMMMEnIiIiImpGmOATERER\nETUjTPCJiIiIiJoRJvhERERERM0IE3wiIiIiomak1gn+N998Aw8PD5iamiIgIACHDh16YP/z588j\nODgYSqUSrq6uWLhwod7jkZGRkEqlev/JZDJcvnz50a6EiIiIiIhgVJtOW7ZswaxZs7BmzRr06dMH\nq1atwtChQxEfHw8XF5cq/fPz8zFo0CAEBwfj5MmTiI+Px+TJk2Fubo5///vfYj+JRIILFy7AyspK\nbLO1ta2DyyIiIiIiejxJarOTbc+ePdGlSxesWbNGbPP29saYMWOwaNGiKv1Xr16N+fPn486dO5DL\n5QCARYsWYc2aNUhMTASgG8EfMGAA0tPTYW1tXVfXQ0RERET0WKuxREetVuPkyZMYNGiQXvvgwYNx\n+PDhas+JiYlBYGCgmNwDwJAhQ5CcnIyEhASxTRAEBAQEwMnJCQMHDkRERMQjXgYREREREQG1SPAz\nMjKg0Whgb2+v125vb4/U1NRqz0lNTa22vyAI4jmOjo5Ys2YNtm/fjh07dqBdu3Z48skn8ffffz/q\ntRARERERPfZqVYNfH7y9veHt7S0e9+jRAzdv3sTSpUvRp08fvb65ubkNHR4RERERUZ1p0aJFg71W\njSP4NjY2kMlkSEtL02tPS0uDg4NDtec4ODhU218ikdz3HECX5F+5cqU2cRMRERERUTVqTPCNjY3h\n7++P8PBwvfbw8PAqI+139erVC9HR0VCpVGLb3r174eTkBDc3t/u+1unTp+Ho6Fjb2ImIiIiI6B61\nKtGZPXs2XnrpJXTr1g19+vTB6tWrkZKSgmnTpgEA5s+fj+PHj2Pfvn0AgAkTJuCTTz7B5MmT8d57\n7+HSpUv4/PPP8fHHH4vPuWLFCri7u8PX1xcqlQo//fQTfvvtN4SGhj4wlob8eqO5O3HiBAAgICDA\nwJE0P7y39YP3tX7wvtYP3tf6wftaP3hf64ehysxrleCPHTsWWVlZWLRoEVJSUtCxY0f89ddf4hr4\nqampuHHjhtjf0tIS4eHhmD59Orp16wYrKyvMmTMHs2bNEvuoVCq8++67SEpKgqmpKXx9fREWFoYh\nQ4bU8SUSERERET0+aj3Jdtq0aeKI/b1++OGHKm2+vr4PXPZyzpw5mDNnTm1fnoiIiIiIaqHGGnwi\nIiIiImo6mOATERERETUjTPCJiIiIiJoRg210RURERFTXtFqt3jLdVDt3lzEvKSkxcCRNi1wuh1Ta\n+MbLmeATERFRsyAIAkpLS6FQKCCRSAwdTpOiUCgMHUKTIwgCSkpKGuXvW+P7yEFERET0CFQqFeRy\neaNLtqh5kkgkkMvljfIbIyb4RERE1CwIggCZTGboMOgxIpPJIAiCocOoggk+EREREVEzwgSfiIiI\niKgZYYJPRERERNSMMMEnIiIiagaCg4PRoUMHQ4dBjQATfCIiIqJmgKsHPZrU1FTMmzcPAwcORMuW\nLSGVSvHrr78aOqx/hAk+ERERET22Ll26hCVLluDWrVvo2rVrs/igxI2uiIiIiOixFRAQgMzMTFhZ\nWSEyMhL9+/c3dEj/GEfwiYiIiBq5wsJCvPPOO/Dw8IBCoYCdnR369++PQ4cOVekbHx+PAQMGwMzM\nDC4uLli6dGmVPpmZmZg6dSocHR1hamqKTp06Ye3atXp9AgICMGLECL22Xr16QSqV4siRI2Lb/v37\nIZVKERkZKbbl5eVhzpw58PT0hEKhgIuLCyZOnIiUlBQAgFqtxoIFC9C9e3dYW1tDqVSiR48e2LVr\nV5VY9+/fj379+sHa2hpmZmbw8vLCm2++qddHpVLh448/hre3t/h6s2fPRnFxcY331szMDFZWVjX2\na0o4gk9ERETUyE2bNg3btm3DjBkz0KFDB2RnZ+Po0aM4c+YM+vbtK/bLycnB008/jVGjRmHcuHHY\ntm0b5s2bBz8/PwwZMgQAUFpaiuDgYFy+fBkzZsyAh4cHdu7cialTpyIrKwvvvvsuACAwMBA//fST\n+NzFxcU4deoUZDIZoqKi0KtXLwDAoUOHYGJigp49ewIAioqKEBQUhAsXLuCVV16Bv78/MjMzERYW\nhqtXr8LR0RF5eXn47rvv8MILLyAkJAQlJSXYvHkzRo8ejbCwMDHW+Ph4DBs2DH5+fvj444+hVCpx\n7do17NmzR+/+jBw5EtHR0Zg6dSp8fHwQHx+PVatW4cKFC9i9e3f9/cE0VkITkJOTI/5Hdef48ePC\n8ePHDR1Gs8R7q6NSlwlZ2cWCVqutk+fjfa0fvK/1g/e1fjzovhYXFzdwNA3HyspKePPNNx/YJzg4\nWJBKpcLGjRvFNpVKJTg6OgpjxowR21asWCFIpVLhp59+Etu0Wq0wcOBAwdTUVMjKyhIEQRC2b98u\nSKVS4ezZs4IgCMKBAwcEqVQqjB07Vnj66afFc5988kmhb9++4vGCBQsEqVQqbN++/b6xarVaQaVS\n6bWp1WqhY8eOwqBBg6rEejem6mzatEmQyWRCdHS0XvvmzZsFqVQqhIeH3/fce0VERAgSiUTYsmVL\nrc950O+doXJYjuATUZ3KzCrCqbOpOHUmBWfi0lBSUgZzMzm8PVvB26sV2nlZw8vDGmZKuaFDJaLH\n3C+hcfh1Z1y9PPfYkb54YbRvnT1fixYtcPToUSQnJ8PJyem+/UxNTTFx4kTx2NjYGN27d8f169fF\ntrCwMNja2ur1k0gkmDVrFg4cOIB9+/ZhzJgx6Nu3LwRBQFRUFDp16iT+f8SIEXjjjTcgCAI0Gg1i\nYmLw1ltvic+1fft2+Pr6YvTo0feNUyKRwNjYGICuXKegoAAajQZBQUH45Zdf9K4bAEJDQxESElLt\nBNitW7fC29sbPj4+yMzMFNsDAwMBAAcPHsTAgQPvG0tzxASfiP6xy1czcexUMk6dTcHNWzlVHi8o\nVOHU2RScOptS3iKBq7MFenVzxfPDfWBkxOlAREQPsnTpUkyePBmtW7dG165d8dRTT+HFF1+Et7e3\nXj9nZ+cq51pZWeHcuXPicUJCAry8vKokyz4+PhAEATdv3gQA2NnZwdvbG1FRUZg+fTqio6MRFBSE\nwMBA5OXlITY2Fmq1WizJuevatWsYOXJkjde0du1aLF++HPHx8RAEQWyXSiv+TRg3bhy+//57TJ06\nFfPmzcOAAQMwcuRIjB07FjKZDABw+fJlXLp0Cba2tlVeQyKR4M6dOzXG0twwwSeiR5aeUYjvfjqN\nE6eT79vH2EgGdZnmnlYBibfzkHg7DhcvZ+CdN3vB3Iwj+kRE9/P8888jKCgIu3btwt69e/HVV19h\nyZIlWL9+PV544QWx392k916VE+iHERQUhD/++ANlZWWIiYnB66+/DldXV7Ru3RpRUVEoKyuDTCZD\nnz59Hup5N23ahKlTp2LEiBGYN28e7OzsYGRkhO+//x4///yz2E+hUCAyMhJRUVEICwvDnj17MHHi\nRHz55Zdi7b9Wq0WHDh2wcuXKaq/zQd94NFdM8InooWk0WoSFX8Xm7edRWlqm95hMJoVvO1s80dkB\nT3R2hLOjBVLSCnD5WiYuX83C5auZuHErR3wTPnshDfM/OYD33u4LBztzQ1wOET2mXhhdt2U09c3O\nzg5TpkzBlClTkJeXhx49emDBggV6CX5tuLm5ITY2FoIg6I3ix8fHAwDc3d3FtsDAQKxbtw6bN29G\ncXGxWPYSFBSEyMhIaLVa+Pn5wcLCQjzH09MT58+ff2AM27Ztg6enJ3bs2KHXvm7dumr7BwUFISgo\nCIsXL8aaNWswffp0hIaGYvz48fD09MSpU6eaxfKWdYXfixPRQ7l6PQvvLtiHHzbH6iX3gb1aY+7M\nPtiwegQ+mtcPw4e2g4uTJSQSCZwcLBDcxx1TX34C/104CJu/HYVxoyr+Ub2dkoe5H+1H/OUMQ1wS\nEVGjptVqkZeXp9dmaWmJNm3aICenallkTYYNG4b09HRs3rxZbBMEAStWrIBCodCrVw8MDIQgCPjs\ns8/Qtm1b2NnZie3R0dE4fPiwXnkOoPu2IS4uDtu3b79vDNV903D9+nXs3LlTry0rK6tKv65du0IQ\nBPHax40bh9TUVKxevbpKX5VKhYKCgvvG0VxxBJ+IaqWoWI2ft51H2L6rel+BujpbYtorAfDxtqn1\nc5mYGGHcKF84O1rgq2+PQ12mQX5BKRYsjsCM17ohqLdbfVwCEVGTlJ+fD2dnZzz33HPo3LkzLC0t\ncejQIezZs6fKevC1MWXKFHz77bd49dVXcerUKXh4eGDHjh04ePAgFi9erLcmvLu7O5ydnXHp0iW8\n9tprYntQUBAyMzMhkUjEUf275syZg+3bt2P8+PGYPHky/P39kZ2djd27d2PhwoUIDAzE8OHDERoa\nimeffRbDhw9HUlISVq9ejfbt2yM2NlZ8roULFyIiIgLPPPMM3N3dkZ2djTVr1sDc3BzDhg0DAEya\nNElcQjQyMlKcHHzx4kVs3boV27Ztq/Ih5F6ffvopJBKJOBk5NDQUV65cAQC89957D32PDY0JPhHV\n6FZSLj5bdghp6YVim7GRDGNHdsDwp71hbFR9zWdN+vZsDZtWSixe/jfy8ktRVqbF8jVHkZxagHGj\nOjSL7cKJiP4ppVKJGTNmIDw8HL///jvUajXatGmDL774Qm/1GgD3fd+s3G5iYoKIiAjMnz8fmzdv\nRk5ODry8vPDdd98hJCSkyrmBgYHYsmUL+vXrJ7a1a9cOtra2yMzMrJLgK5VKREdH46OPPkJoaCg2\nbNgAOzs7BAcHo23btgCAl156Cenp6Vi9ejX2798PLy8vLF++HFeuXNFL8EeOHInExERs2LAB6enp\naNWqFXr37o0PPvgArq6u4rXt2LEDy5cvx/r16/Hbb7/B1NQUHh4emDFjBvz8/Gq8xx9++KF4jyQS\nCbZu3YqtW7dCIpE0yQRfIjzqrIsGlJubK/58d7kk+udOnDgBQLdTHdWt5nRvT51JwRerYlBcohbb\nunRywNSXn6izmvm09EL858toJN6u+Ap61LD2eHGs/ptyc7qvjQnva/3gfa0fD7qvJSUlUCgUDR0S\nPeYe9HtnqByWNfhEVC1BEPD77stY9OUhMbk3MTHCzH/1wAfvBNbphFh7WzMsen8A/HztxbYdf1zC\nhUvpdfYaREREjwsm+ERUhbpMg9Xfn8QPm2PFevtW1kr85/0B6NfHrV5KZ8zN5Hj/7cBKSb6Ar9ce\nR0lJ2QPPIyIiIn1M8IlIT15+KT7+PAr7Iit2PWzn1QpLPx6INm4t6/W1jYykmPFaN5gqdLsbpqYV\nYOPWs/X6mkRERM0NE3wiEqWk5mPuR/v0SmP69XbDx/OD0bJFw9S12rRSImRiF/E4LPwqzsc/frsQ\nEhERPSom+EQEQDfR9cPFkZVWypFg0phOeOtf3SE3frRVch7VgCB3+Hd2FI+//u643iRfIiIiuj8m\n+ESEjMwiLPgsAplZRQB0S2DOndkbo5/1MchSlRKJBK+HBMBMKQcA3MkoxPqfWapDRERUG0zwiR5z\nWdnF+OCzCNzJ0I3cGxvJ8H+z+6CHv7NB47K2MsWrL1aU6uw9eA1Xbzx+uxESERE9LCb4RI+xnNwS\nfPhZBNLu6BJnmUyKd2f2RueODgaOTKdfbzd0f6Lig0Zo2G2UlGgMGBEREVHjxwSf6DGVm1eCBYsj\nkJyaDwCQSiWYM6OXXu27oUkkEkx7xR/mZrpSndw8NcIOpBo4KiIiosaNCT7RY6igQIWPl0SJO8dK\nJBLMfqMnuhu4LKc6LVso8K/J/uLxyTPZuHw104ARERERNW5M8IkeM+oyDT79Iho3b+UA0CX3M//V\nHb27uxo4svvr08MVPQNcxONfdsQZMBoiIqLGjQk+0WNmw89ncfna3RFwCWa81g1Bvd0MGlNtTBzT\nEXcX9Ik9l4qLVzIMGxAREVEjVesE/5tvvoGHhwdMTU0REBCAQ4cOPbD/+fPnERwcDKVSCVdXVyxc\nuPC+fQ8dOgRjY2P4+fnVPnIiemiHjyXiz/Ar4vFLL/ihf6C74QJ6CM6OlujsW7GT7i+hHMUnIqos\nODgYHTp0MHQY1AjUKsHfsmULZs2ahffffx+xsbHo3bs3hg4diqSkpGr75+fnY9CgQXB0dMTJkyex\nYsUKLF26FMuWLavSNycnBy+//DIGDhz4z66EiB4oJTUfq9aeEI+7P+GMEUO9DRjRw+vfx1YcxT8b\nl6a34y4R0ePOEPuWNAcHDhzAq6++inbt2sHMzAyenp6YMmUKUlOb7qIOtUrwly1bhpCQEISEhKBd\nu3ZYuXIlHB0dsXr16mr7b9y4EcXFxVi/fj18fHwwevRozJ07F19++WWVvq+++iomT56Mnj17/rMr\nIaL7Uqk1WPr1EXE3WDsbM8x4rVuT+8fAxtoEXTpWjOJvYS0+ERH9Q3PnzkVkZCRGjx6Nr776CuPH\nj8evv/6KJ554AmlpaYYO75EY1dRBrVbj5MmTmDNnjl774MGDcfjw4WrPiYmJQWBgIORyudg2ZMgQ\nfPjhh0hISICbm67e95tvvsGdO3fw/vvv45NPPvkn10FED7Bu42lxUq2RkRTvzOgFc3N5DWfVTBAE\nXM4qQkxSDjKKVShSa1Gk1qBIrUFxmRaFKg0ECHBvaYq21mZoa61EW2sl7JTyR/5w0b+PLa7cSIYg\nCDh34Q7iLqbDt73tP74WIiJ6PC1btgx9+/bVaxsyZAj69euHlStXYtGiRQaK7NHVOIKfkZEBjUYD\ne3t7vXZ7e/v7fnWRmppabX9BEMRzzp07h4ULF2LTpk1NbhSRqCmJ/DsB4Qevi8evTOgCLw/rR34+\nQRBwObMQ62KT8Mrv5zE7/CJ+jU/FgZtZiLmdg7N38nE1uwi380uQU6pGbmkZzqTlY1t8Kj77+zpC\nfj+PSTvPYUHkVfx2+Q6K1Q+3cVUrKxP07+suHv8Sev6Rr4WIqKkoLCzEO++8Aw8PDygUCtjZ2aF/\n//7VzomMj4/HgAEDYGZmBhcXFyxdurRKn8zMTEydOhWOjo4wNTVFp06dsHbtWr0+AQEBGDFihF5b\nr169IJVKceTIEbFt//79kEqliIyMFNvy8vIwZ84ceHp6QqFQwMXFBRMnTkRKSgoA3QDyggUL0L17\nd1hbW0OpVKJHjx7YtWtXlVj379+Pfv36wdraGmZmZvDy8sKbb76p10elUuHjjz+Gt7e3+HqzZ89G\ncXFxjff23uQeAAIDA2FtbY0LFy7UeH5jVOMIfn1QqVR44YUX8N///hetW7cGoEsaauPEiRM1d6KH\nwntafwx9b+9klGD1j9ehUmsBAJ18WsCmRfYjxZVRqsXRLA3O5GiQrard39f7KSoEkrNyEXU1BWsO\nA71aGaGvjREsjGv3Yd+7jRp/7i2CVgscO5mAX7dHwMPN/B/FRIb/fW2ueF/rR3X31c3NDQqFwgDR\n1L9p06Zh27ZtmDFjBjp06IDs7GwcPXoUZ86c0UtQc3Jy8PTTT2PUqFEYN24ctm3bhnnz5sHPzw9D\nhgwBAJSWliI4OBiXL1/GjBkz4OHhgZ07d2Lq1KnIysrCu+++C0CX5P7000/icxcXF+PUqVOQyWSI\niopCr169AOgWSzExMRHLrYuKihAUFIQLFy7glVdegb+/PzIzMxEWFoarV6/C0dEReXl5+O677/DC\nCy8gJCQEJSUl2Lx5M0aPHo2wsDAx1vj4eAwbNgx+fn74+OOPoVQqce3aNezZs0fv/owcORLR0dGY\nOnUqfHx8EB8fj1WrVuHChQvYvXv3Q9/vwsJCFBQUwMbGpsa++fn5OH+++sGmtm3bPvRr14UaE3wb\nGxvIZLIqNUhpaWlwcKh+O3sHB4dq+0skEjg4OCAlJQXx8fF45ZVXMHnyZACAVquFIAiQy+UICwvj\npFuif0il1uLnHYlicm9jLcfIoU4P/Y2ZRhAQmV6G8LQylGmrPm4qAzq2kMHdTApTmQQKKaCQSWAi\nBUxkEmgEAUlFAhKLtUgs0iKpWIvSSoP2xRrgwJ0yRGWUwd9KhiAbI9gpHvzlonVLObp2ssLJM9kA\ngP3Rd9CmtRm/DSSih7LpfDI2n0+pl+ee0NEREzs61dnz/fnnn5gyZUq1o/GVpaWlYcOGDZg4cSIA\nICQkBG5ubli3bp2YNP/vf//DhQsXsH79ekyaNAkA8MYbb2Dw4MH46KOPMGXKFFhZWSEwMBArV67E\nuXPn0KlTJ8TExKCsrAzPP/88oqKiMHfuXABAdHQ0AgICYGJiAgBYsmQJzp07h61bt2L06NFibPPn\nzxd/tra2RkJCAoyNjcW2GTNmoGvXrvjiiy/EWMPDw6FSqbB7925YWVmJff/zn/+IP2/evBl79+5F\nRESE3oedgIAATJo0Cfv27XvovHLZsmVQq9UYP378Q53XWNSY4BsbG8Pf3x/h4eF47rnnxPbw8HCM\nGTOm2nN69eqFefPmQaVSiXX4e/fuhZOTE9zc3FBWVlblk86qVauwb98+7Ny5U6zRr05AQECtLoxq\ndnf0g/e07jWGe7tp6zkUFstgZqaEsZEMn77/JNq4taz5xEpu5BRh+dEEXM0rgtxUjrtV+2bGMvRy\naYnA1lbobGcBY1ntt9TQCgJu55ciNjUPuy7fQUpBqfjYuRLgfJIEvVxaYHJnZzhb6I/EVb6vrm4+\nmD4nDFqtgPQsQK5sjc6++qWBVDuN4fe1OeJ9rR8Puq8lJSUNHU6DadGiBY4ePYrk5GQ4Od3/g4Op\nqamY3AO6PK579+64fr2iVDMsLAy2trZ6/SQSCWbNmoUDBw5g3759GDNmDPr27QtBEBAVFYVOnTqJ\n/x8xYgTeeOMNCIIAjUaDmJgYvPXWW+Jzbd++Hb6+vnrJ/b0kEomY3KvVahQUFECj0SAoKAi//PKL\n3nUDQGhoKEJCQqodyNm6dSu8vb3h4+ODzMyKnc4DAwMBAAcPHnyoBD8qKgqffPIJxo4diwEDBtTY\n38LC4r5/z3Nzc2v9unWpVv8qz549Gz/++CPWrVuHixcvYubMmUhJScG0adMA6D6RVb5xEyZMgFKp\nxOTJkxEXF4fQ0FB8/vnnePvttwEARkZG6NChg95/dnZ2MDExgY+PD5RKZT1cKtHjIz2jEL/9dVk8\nDpnU5aGSe7VGi43nkjFzz0VczS4S272slPgw0BObRvrh3z3cEeDY4qGSewCQSiRwtVTgWW87fPuM\nL/6vjwe8rc3ExwUIOJyUgzd3xyPsavp9y/fsbc3wZL824vGW0Lhal/oRETU1S5cuRVxcHFq3bo1u\n3brhgw8+wOXLl6v0c3Z2rtJmZWWF7Oxs8TghIQFeXl5VkmUfHx8IgoCbN28CAOzs7ODt7Y2oqCgA\nupH6oKAgBAYGIi8vD7GxsTh16pRYknPXtWvX0LFjxxqvae3atejYsSMUCgVatWoFOzs7rF69Wi8p\nHjduHPr27YupU6fCzs4O48aNw88//wyNpuKr4MuXL+PSpUuwtbXV++9uGfidO3dqjOWuixcvYvTo\n0fDz86syJ6EpqVUN/tixY5GVlYVFixYhJSUFHTt2xF9//QUXF93W8ampqbhx44bY39LSEuHh4Zg+\nfTq6desGKysrzJkzB7NmzaqfqyAiPZu2noe6TPfm5+FuhUHBHrU+93JmIZYfS0BCbsXEJLlUigkd\nHTG6vT1k0rorg5FKJOjjaoXeLi1xPr0A2y+m4Xiy7o29VKPFqhO3cPR2LmZ1d4OVqXGV859/1gcH\nom5Co9Hi4pUMnI1LQ+eO1ZcOEhHda2JHpzoto6lPzz//PIKCgrBr1y7s3bsXX331FZYsWYL169fj\nhRdeEPvJZLJqz3/UAZCgoCD88ccfKCsrQ0xMDF5//XW4urqidevWiIqKQllZGWQyGfr06fNQz7tp\n0yZMnToVI0aMwLx582BnZwcjIyN8//33+Pnnn8V+CoUCkZGRiIqKQlhYGPbs2YOJEyfiyy+/FGv/\ntVotOnTogJUrV1Z7nQ/6xqOyxMREDB48GFZWVvjzzz9hZmZW80mNVK0n2U6bNk0csb/XDz/8UKXN\n19cXERERtQ5kwYIFWLBgQa37E1H1rl7PQtSRBPH4lQldIK1lUr7/RiaWH0uAttIbpI+NOWZ2d4Or\nZf1NXJNIJOhkZ4FOdha4mlWEL2Ju4lae7gPGiZRcvLH7At7q5oZ7U3xbGzM8GdQGew9eAwBs3RXP\nBJ+Imi07OztMmTIFU6ZMQV5eHnr06IEFCxboJfi14ebmhtjYWAiCoDeKHx8fDwBwd3cX2wIDA7Fu\n3Tps3rwZxcXFYtlLUFAQIiMjodVq4efnBwsLC/EcT0/P+046vWvbtm3w9PTEjh079NrXrVtXbf+g\noCAEBQVh8eLFWLNmDaZPn47Q0FCMHz8enp6eOHXqFPr37/9Q96GyrKwsDB48GGVlZYiIiKiyGmRT\n83DfrRNRoyYIAr7fHCsed3/CudZrxB+4mYllRyuSexOZFP96whVLnvSu1+T+Xl7WSqwY0h4jve3E\ntrzSMnx66Bp+TVShRKM/OvPcs+3Ff6AuXErHtZvZICJqTrRaLfLy8vTaLC0t0aZNG+Tk5Dz08w0b\nNgzp6enYvHmz2CYIAlasWAGFQqFXdh0YGAhBEPDZZ5+hbdu2sLOzE9ujo6Nx+PBhvfIcQPdtQ1xc\nHLZv337fGKr7puH69evYuXOnXltWVlaVfl27doUgCOK1jxs3DqmpqdVuwKpSqVBQUHDfOADdqj9D\nhw5FSkoKwsLC4OFR+2+9GyuDLJNJRPUj5sRtXLycAQCQSiV46QW/Wp0XlZCFZUcTIECXPLdpaYr3\n+3rCwdz5uoB6AAAgAElEQVSk3mJ9ELlMiilPuKKbUwssO5qAjGIVAOB4lgbXC7Xw7FAKRwtdbLY2\nZujTwxWHYm4BAP7YfRkzp/UwSNxERPUhPz8fzs7OeO6559C5c2dYWlri0KFD2LNnT5X14GtjypQp\n+Pbbb/Hqq6/i1KlT8PDwwI4dO3Dw4EEsXrxYb7Uad3d3ODs749KlS3jttdfE9qCgIGRmZkIikYij\n+nfNmTMH27dvx/jx4zF58mT4+/sjOzsbu3fvxsKFCxEYGIjhw4cjNDQUzz77LIYPH46kpCSsXr0a\n7du3R2xsxUDVwoULERERgWeeeQbu7u7Izs7GmjVrYG5ujmHDhgEAJk2aJC4hGhkZKU4OvnjxIrZu\n3Ypt27ZV+RBS2YQJE3D8+HG8+uqriIuLQ1xcxS7p5ubmVfYCaAqY4BM1E+oyDTZsOSseDx3oBScH\niwecoXMoMRtLY26KI/fuLUyxqL83WpgY/u2hi4Mlvn7KB6tPJiLylm4UJ7NUwDv7LuGTYC94Wukm\n5A9/yltM8KNjbmHS2E5oZc3J+kTUPCiVSsyYMQPh4eH4/fffoVar0aZNG3zxxRd6q9cAuO9ywZXb\nTUxMEBERgfnz52Pz5s3IycmBl5cXvvvuO4SEhFQ5NzAwEFu2bEG/fv3Etnbt2sHW1haZmZlVEnyl\nUono6Gh89NFHCA0NxYYNG2BnZ4fg4GBxXfiXXnoJ6enpWL16Nfbv3w8vLy8sX74cV65c0UvwR44c\nicTERGzYsAHp6elo1aoVevfujQ8++ACurq7ite3YsQPLly/H+vXr8dtvv8HU1BQeHh6YMWMG/Pwe\nPNh15swZSCQSfP/99/j+++/1HnNzc2uSCb5EaALLTlSeTX13uST657iEW/0xxL39ffdl/FBenmOm\nlGP1f5+Gubn8geccScrBZ39fh6b8baC1pSk+G9AWLRVVJ7QaWsTNLCw6GIcyLaA0U8LUSIYPAj3R\n2V73Iea9Tw8i/nI6AGDkM+3x0rjafXtBfC+oL7yv9aOmZTKb60ZX1Hg96PfOUDksa/CJmoGCAhW2\n7qrYTnvsyA41JvdHb+dgcaXk3sVCgf/0b5zJPQAEu1tjShs5TMvLNovLNFgQeQWHEnU19yOGeot9\nww9eR3GJ2hBhEhERGRwTfKJm4NddF1BQqKtTt7czx1MDPR/Y/0RKLj77+zrKypN75/LkvrqlKBsT\nD3MZXvc0QavyONVaAYv/voGwq+kI6OoEB3tzAEBhkQoHo28aMFIiIiLDYYJP1MQlp+YjLPyKePzS\nOD8YG1W/DjIApBaUYvHfN6DW6pJ7BzMT/Kd/W7RSPnjEv7FwNJVi6cB2cCnf5VaAgFUnbuHnCyl4\nZpCX2O/3PVeg1Tb6CkQiIqI6xwSfqInb+Os5MZFt722DngFVdzG8S6MV8N+Ymygu3wTLVinHfwa0\nhU0TSe7vsjczwZKB7fR2wN18PgVJVgqYmemuJe1OAY6dum2oEImIiAyGCT5RE5Z6pwAxJyqS2JAJ\nXe67ggIA/HohFfEZuvWApRIJ/q+PB+zNDLMU5j/VwsQI/+nfFv4OlmLbb9fSYdPDpeJ4d9Vt3ImI\niJo7JvhETdjeA9eA8rXru/o5wMvD+r59L2UWYnNcing8saMjvFs13W24AcDUWLeSTi/nlmJbvJGA\nXGtd+c7Fyxm4ci3TUOEREREZBBN8oiZKpdJgX+QN8fipAV737Vus1uC/R26Ia913sDHH2A4O9R5j\nQzCWSfFu7zbicpkyIynyPFsi11JXqsNRfCIietwwwSdqov4+miiunGPbygz+XRzv23dtbBKSC0oB\nAKZGMrzd0x3SB5TyNDVymRQf9PVE2/Ka/JaWCiQ7maPAzBiHjyUhPaPQwBESUUNpAtv7UDPSWH/f\nmOATNVFh+66KPw950hNSafUJ+5GkHOy+liEev+7vCgfzpll3/yCmxjJ83M8LrpYKyE1kUJgaIdHV\nHIUKGf7ce7XmJyCiJk8ul6OkpKTRJl3UvAiCgJKSEsjljW+hCsPvRU9ED+3q9Sxcu5EFADA2kmFg\nvzbV9ssqVmPFsQTxONDVCgPc71+n39S1MDHCp8FtMWffJRS1UCClpAy3Wltg1+GbGDuqA5SNfJ1/\nIvpnpFIpTExMUFpaauhQmpz8/HwAgIWFhYEjaVpMTEwglTa+8XIm+ERN0F/7K0ake3d3gaVF1RF5\nQRCw/FgC8lVlAAAbUzmmB7R+4Co7zYGNUo5P+7fFu/suISOrGGq1BlfsTbFj/1VMHOZj6PCIqJ5J\npVIoFApDh9HknD9/HgAQEBBg4EioLjS+jxxE9ED5BaU4dCRRPB46sPrJtftuZOJkSq54PLunGyxM\nHo/P9M4WCiwMbgv7lrp/5MuMpFh59jYKS9UGjoyIiKj+McEnamL2Rd6AunyjKg93K7T1rFpyU1Km\nxcZzFUtijmpnj872llX6NWceVkosGdoBsvK5CbkSYM5vceJKQkRERM0VE3yiJkSrFbBn/zXxeOiT\nXtWW3Px2+Q4yinUr7FgpjDGx4/1X2GnOnnBugZGV1sg/nJiDjeeSDRgRERFR/WOCT9SEnD6Xijvl\nSz6am8nRt6drlT45JWr8eiFVPJ7Y0RGmxrIGi7GxeWeYL2xzdBPuSkrLsP5UEiJuZhk4KiIiovrD\nBJ+oCfkrvGJy7YDANjCppqb+l7hUFJeX8LhYKDDYw6bB4muMrFoqMMbTFuYFuvr73LxSrDiWgEuZ\nXBufiIiaJyb4RE1E6p0CnDp7d2Regqee9KzSJzm/BGFX08XjkC7OYg3642zEUG843y6ASakGBYUq\nFKnKsDD6GtKLVIYOjYiIqM4xwSdqIvYeuAZAN0G0q589HOzNq/RZfzYZmvJJpL625uju1KIhQ2y0\n3Fxb4okO9nBNzIdMIyAntwTZJWosjL6GkvJvO4iIiJoLJvhETYBKpcG+yBvi8VNPVl0a81JmIQ4l\nZovHr3ZxafZr3j+M4U95Q67WwiUpH/n5pdBqBVzLLsKKYwnc9ZKIiJoVJvhETUDMiSQUFOrKSexs\nzODfWX9VHEEQsC42STzu62qFdq3MGjTGxq5LJ3u4OlvCrKgM9smFyMvXTbyNupWNP66k13A2ERFR\n08EEn6gJOHK8InkfGNwG0nvq6o8l5yIuvQAAYCSRYLKfc4PG1xRIJBI8+5Q3AMAqpxSKlELcHbhf\nezoJ8RkFBoyOiIio7jDBJ2rkikvUOHWmYtnL3t31l8bUaAV8H3tbPH66rS0cLUwaLL6mJKi3GyzL\n743FjRy00Ooy/DJBwOK/byCnhDvdEhFR08cEn6iRO3UmVdy5trVLCzg5WOg9vvd6BpLySwAASiMZ\nxnVwaPAYmwq5sUycvyAVgFZXcmAu1+0RkFGswpIjN7jTLRERNXlM8IkaucPHKspzenVz0XtMrdHi\n57gU8XhMBwe0VBg3WGxN0dCBnjA20iX1ydez8LyTFSTQlTydScvHT2e50y0RETVtTPCJGrHS0jKc\nPFORwN+b4EfdykZmsa6sxEphjOHedg0aX1PUwlKB4L5u4vGlQ7cwvmPFtx6/xqfi6O0cQ4RGRERU\nJ5jgEzVip8+lQqUqAwC4OFmitUvFuvaCICD0Ypp4PNzbDgoj/pWujeFDvYHyUftTZ1PQzdQE/g6W\n4uNfxNxESvkqO0RERE0NswGiRqzy6jk9A/RH70+n5uNmbjEAQGEkxVBPmwaNrSlzdrREnx4V93P7\nb/F4p1cb2CrlAIBCtQb/+fs6VBqtoUIkIiJ6ZEzwiRoplVqD46cr6sHvLc+pPHo/qI0NLEyMGiy2\n5mDMiA7iz8dPJyMztQD/18cDxuVLkF7PKdLbW4CIiKipYIJP1EidOZeGkhJdeY6DvTncW1eU51zP\nLsLptDwAgFQiwch2rL1/WK1dWuh9K7J11wV4tzLDa10rliH940o6jiSxHp+IiJoWJvhEjdTh44ni\nz726uUAiqdjcaselitH73i4t4WDOde8fReVR/JgTSbiVlItnvGzQ26Wl2L7iWALSi1SGCI+IiOiR\nMMEnaoTUZRqcOF396jkZRSpEJmSLx6Pb2zdobM1JG7eWCOjqJB5v3XUBEokEb3VzE+vx81VlWHr4\nBjRaro9PRERNAxN8okboXNwdFJaPGtvZmMHT3Up87LfLd6Ap34zJ19Yc7VqZGSTG5mJspVH8v48m\nISk5DxYmRpjTqw2k5d+axGUU6O03QERE1JgxwSdqhO5dPedueU6RWoO/rmaIj41ux9H7f8rLwxpd\n/e6ugy9g+2/xAHQfniZ2dBT7/RKXijNp+QaIkIiI6OHUOsH/5ptv4OHhAVNTUwQEBODQoUMP7H/+\n/HkEBwdDqVTC1dUVCxcu1Hs8KioKffr0gY2NDZRKJXx8fPDFF1882lUQNSMajRZHT94Wj3t1ryjP\n2Xs9A0VlGgCAs4UC3Z1bVDmfHl7lWvyoI7eQkqpL5Md2cICfnQUAQICAL2JuILe0zCAxEhER1Vat\nEvwtW7Zg1qxZeP/99xEbG4vevXtj6NChSEqqfgm5/Px8DBo0CI6Ojjh58iRWrFiBpUuXYtmyZWIf\nc3NzzJw5E9HR0YiPj8cHH3yABQsWYM2aNXVzZURN1PmL6Sgo1JXnWFuZoq2HNQBAoxWw89Idsd+o\ndnZiCQn9M+3b2sCvg+7bEEEQsP33iwB0KxS909MdluVLkGYWq7Hs6E0IAuvxiYio8apVgr9s2TKE\nhIQgJCQE7dq1w8qVK+Ho6IjVq1dX23/jxo0oLi7G+vXr4ePjg9GjR2Pu3Ln48ssvxT5PPPEExo4d\nCx8fH7i5uWHChAkYMmQIoqOj6+bKiJqoI8cqPjj3CnCBtHxd9r8Ts8XVXCxNjDDAvZVB4muuxo6q\nGMWP+Psm0tILAQCtlHL8u4e7+Njx5Fz8fiW9ocMjIiKqtRoTfLVajZMnT2LQoEF67YMHD8bhw4er\nPScmJgaBgYGQy+Vi25AhQ5CcnIyEhIRqzzl9+jSOHDmC4ODghwifqHnRaoVqy3MEQcD2ShtbDWtr\nCxMjTqGpSx3a2aJDO1sAuj+H0D/ixce6O7XASO+KvQZ+iL2NhPJdhImIiBqbGre+zMjIgEajgb29\n/mQ+e3t77N+/v9pzUlNT4erqWqW/IAhITU2Fm5ub2O7q6or09HRoNBosWLAAU6ZMeWA8J06cqClk\neki8p/XnYe/tjVuFSE7JAgCYmxmhIPcmTpxIwLUCDc4m6UbvjaSAU/5tnDiR/KCnatbq63e2UzsJ\njp8qAgD8HnYebZxLYWOt22Ogo1ZAhLYUycUCigD8358n8aaXCYykzadMiu8F9YP3tX7wvtYP3te6\n1bZtW4O8rsGHAA8dOoSTJ09izZo1WLZsGTZt2mTokIgM5vzFXPFn33aWYnlOTKZGbPe3ksHCuPkk\nlY2Jh5sZ3F2VAACNFvjrQKr4mLFUgvGt5bj7xUlysYDwNE64JSKixqfGEXwbGxvIZDKkpaXptael\npcHBwaHacxwcHKrtL5FIqpxzdzTf19cXqamp+OijjzBx4sT7xhMQEFBTyFRLdz+l857WvUe5t4Ig\n4NtNqTAz0yWYz4/sAT9fe+SVluHG9bNQmukmdk7t5wMPK2XdB90ENMTvrFUrT7z70T4AQGKyBsYK\nF3TuWPG+VWabhu9O6+ZJHCuSYLSbN3xtzestnobA94L6wftaP3hf6wfva/3Izc2tuVM9qHEE39jY\nGP7+/ggPD9drDw8PR58+fao9p1evXoiOjoZKVbG9+969e+Hk5KRXnnMvjUaD0tLS2sZO1KwkJecj\nI1NXHqI0NYZve109+IGbmVCX76La1trssU3uG4qXhzUGBLYRj7/fFAuNRiseD/e2Qxd7SwCAVhDw\nRcxNFKk1VZ6HiIjIUGpVojN79mz8+OOPWLduHS5evIiZM2ciJSUF06ZNAwDMnz8fAwcOFPtPmDAB\nSqUSkydPRlxcHEJDQ/H555/j7bffFvt8/fXX+PPPP3H16lVcvXoV69atwxdffIEXX3yxji+RqGmI\nPVdRDuLnaw+ZTApBELD7WsXGVk952hgitMfOxDGdoFDovuBMvJ2HPQeuiY9JJRL8u4cbzI1lAIC0\nwlL871SiQeIkIiKqTo0lOgAwduxYZGVlYdGiRUhJSUHHjh3x119/wcVFt8JHamoqbty4Ifa3tLRE\neHg4pk+fjm7dusHKygpz5szBrFmzxD4ajQZz585FQkICjIyM4OnpiSVLluBf//pXHV8iUdNwulKC\n37mjblJ7fEYhEvNKAAAKIyn6tbYySGyPG6uWCjz/rA82bj0HAPglNA5Bvdxgbq5bGcxGKccbAa2x\n5IjufW/fjUx0d2qBPq788yEiIsOrVYIPANOmTRNH7O/1ww8/VGnz9fVFRETEfZ9v5syZmDlzZm1f\nnqhZU6k0iIuvWFu9q5+u5rvy6H2/1tYwLR81pvo37Clv7D14HXcyClFQqMKWHXF49cWu4uP93Kxx\n9HYuIm/pVj366vgt+NiYw9rU2FAhExERAWgEq+gQEXDhUjrUZbo6bmdHS9jZmKFAVYboxGyxzxCW\n5zQoubEML4/vLB6H7buKxNt5en1e93eFjaluVD9fVYblxxK4yy0RERkcE3yiRqByeU6XTrrynIiE\nbKjKJ3d6tFTC25qTaxtazwBndPTRbXAlCAJ+2Byrl8BbmBhhds+KhQNOpuQi/Hpmg8dJRERUGRN8\nokag8gTbrp0cIAgC9lQqzxns0QoSCde+b2gSiQQhE7uI9z72XCpOnknR69PZ3lJvl9vvTifhTqEK\nREREhsIEn8jAMjKLxNIPYyMZfNvb4kpWEa7n6JbMlMuk6O9ubcgQH2vurVtiYHDFspk/bDojllPd\n9aKfM5wtFACAojINVrJUh4iIDIgJPpGBxZ6vGL338baBiYkR9lyvGL3v62oFc3mt58NTPZjwXEco\nyyfPpqTlY1fYJb3HFUZS/LuHGyTQjfSfTsvTmyBNRETUkJjgExnY6bMVuz539XNAsVqDyISKybVD\nObnW4FpYKjBulK94vGXHBdxIyNHr42NjjlHtK0p11sXeRmoBN+4jIqKGxwSfyIA0Gi3Oxukn+FG3\nslFcXgLiaqmAj42ZocKjSp4e5AVvz1YAdH9uy9fEQHXPDraTOjrBpbxUp7hMg5XHE6BlqQ4RETUw\nJvhEBnTlehYKi3QTMq2tTOHqbIm/7tm5lpNrGweZTIqZ/+oOubxih9tN5Rth3WVSXqojLf8zO5OW\nj7+uslSHiIgaFhN8IgOKPVcxet+lowNu5BTjSlYhAMBYKkF/91aGCo2q4ehggVcmVKyN//vuKzgf\nf0evT3sbc4xuby8ef38miaU6RETUoJjgExlQ7D3r31eeXNvbxQotTDi5trEZ3N9D3GkYELDy22Pi\ntzB3TezoiNaWpgCAkjItlh1lqQ4RETUcJvhEBlJQoMKV61kAdOut+/jY6k2u5c61jZNEIsH017rB\n3Ey3g21GZhHWbYzV6yOX6ZfqnE/Px59X0hs8ViIiejwxwScykDNxaeJa6V5trHExrwT5qjIAgL2Z\nCTrZmRsyPHoA65am+Ndkf/E44tBNxJxI0uvj3coMY3wqSnXWn01GWiFLdYiIqP4xwScykNNnK+1e\n62ePgzezxOP+btbi6C81Tn16uCKol5t4vPr7k8jOKdHr84KvI1wtK1bV+fr4LW6ARURE9Y4JPpEB\nCIKgt8FV2/a2OJ6cKx4PaMOda5uC117qCmsrXa19fkEpvl57DFptRQIvl0kxs3vFBlinUvOw70am\nQWIlIqLHBxN8IgNIvJ2HrOxiAIDS1BjJMqCsfGTX29oMzuVrqVPjZm4mx5tTu4vHp8+mYvO283p9\nfGzMMcLbVjxeezoJWcXqBouRiIgeP0zwiQygcnlO5472iEys2BV1gDtH75uSzr72GPlMe/E49I94\nHIq5pdfnRT8nOJiZAAAK1BqsPslSHSIiqj9M8IkM4HSl5TFbt7NBfEYBAEAmkSDIjQl+UzNpTKdK\nS2cCX393HNduVqyIpDCS4a3urcXjw0k5OFTpQx0REVFdYoJP1MBKS8sQf6livfscS7n4c4CjJde+\nb4KkUgn+/XpPODlYAABUag0WL/8bObkVk24721viqUpLn645mYi80rIGj5WIiJo/JvhEDezCpQyo\nyzQAAGcnSxwvH70HwJ1rmzBzMznm/7sPTBXGAIDMrCIsWXlY/LMGgJDOLmhlqns8p1SN704nGiRW\nIiJq3pjgEzWwcxfSxJ8d29kgpUC3NrqZsQw9nFsYKiyqA86Olpg9vSdQvmrOxSsZWLvhtFhvbyaX\nYXpARanOgZtZeqsnERER1QUm+EQN7OyFO+LP+dYVq+UEtraCXMa/kk2df2dHvDiuk3gcHnEdu/df\nE497OLdEv9YV8yy+Pn4LRWoNiIiI6gqzCaIGVFCgwvWbusmVglSCq6qK5RL7c3JtszHy6XYI7FUx\nUr/2p9M4U2nfg6lPuMCyfK5FRrEKP5653eAxEhFR88UEn6gBnb94B4CuXMPSyxpFGi0AwN7MBB1s\nzQ0YGdUliUSC6a92g4e7FQDdxmafrzyMqzd0uxW3VBhj2hOuYv8/r6YjLr2g2uciIiJ6WEzwiRrQ\nuUrlOWoHM/Hn/m7WkEokhgiJ6olcLsP8WX3EnW5LSsrw6X+jkZKaDwAIam2Fbk4Vcy5WHkuAqvwD\nHxER0T/BBJ+oAZ2P1yX4ZTIJ0o0qEvoBbVie0xy1slbiwzlBMDfTLYWal1+KT5ZGITunRDfKH9Aa\npkYyAEBSfgl+iUsxZLhERNRMMMEnaiDZOSVIvJ0HAChsaQJjuS6x87Y2g7OF4kGnUhPW2qUF/m92\nXxiXJ/Jp6YX49L9RKCpWw1Ypx+TOzmLfbfFpuJFTZKhQiYiomWCCT9RAKi+PqXWxhESqG8Ef4M7R\n++aufVsbvDOjFyTlZVg3buVgyYrDUKk1eNrLBr42uvkXGkHAimO3oNEKhgyXiIiaOCb4RA3kbv29\nyliKUjPdZkcyiQRBXD3nsdDtCSdMe8VfPD57IQ1ffXsMEIC3urtBLtW9HV/JKsTOS3fu9zREREQ1\nYoJP1EDuJvi5LUxgqtAtkRjgaIkW5cslUvM3KNgD45/rKB7/fTQRP2yKhbOFCV7wdRDbN55PRkp+\nqSFCJCKiZoAJPlEDSEsvxJ2MQggACqxMYFKe1Adz9P6x8/xwHwwd6CUe/xl+BVt3xeM5Hwd4tFQC\nAFQaLb46kSDugEtERPQwmOATNYC79felJjJILEwgkQAKIym6O7c0cGTU0CQSCV6d1BW9urmIbb+E\nnkf4gWt4s1trcbnUM2n52Hs901BhEhFRE8YEn6gBVJTnyGFqqhu97+3SEgoj/hV8HEmlEsyc1gN+\nvvZi23cbTiPtciZGtbMT29aeTkJGkcoQIRIRURPG7IKongmCgHMX7kDA3fp73QTbfizPeazJjWWY\nO7M3vD1blbcIWPG/o/ARJHAyNwEAFJVpsOrELZbqEBHRQ2GCT1TPEm/nISe3BEVKI2jlMpjIZbA0\nMUIXe0tDh0YGZqowxvtvB8LVWfe7oNUKWP5VDIZV+t04lpyLyIRsQ4VIRERNEBN8onp2d/faPEs5\nTE2NAQkQ6GoFI6mkhjPpcWBuLseCd/vBzsYMAKAu02Dr2pPobWMh9llzKhE5JWpDhUhERE0ME3yi\nenbuwh1oJUCeZcXymCzPocqsrUyx4N0gtLDU7WhcVKzGpV0XYSnT7X6bryrD/04lGjJEIiJqQpjg\nE9UjrVZA3MV0FJoZQyOTwFRhBFulHD7lo7VEdzk6WODDOUFQmurmaBTklkA4nQqNRld/H3UrG0eS\ncgwZIhERNRG1TvC/+eYbeHh4wNTUFAEBATh06NAD+58/fx7BwcFQKpVwdXXFwoUL9R7fsWMHhgwZ\nAjs7O1haWqJnz574/fffH+0qiBqp6wnZKChUIbeFHDKZFHK5DP3crMWlEIkqa+PWEu+9HQhjI93I\nvSopD5JbudBqdUn+qhO3kF9aZsgQiYioCahVgr9lyxbMmjUL77//PmJjY9G7d28MHToUSUlJ1fbP\nz8/HoEGD4OjoiJMnT2LFihVYunQpli1bJvaJjIzEk08+ibCwMMTGxuLpp5/GqFGj8Pfff9fNlRE1\nAucu3IFGChSYy6E0vbu5lZWBo6LGzMfbBm9P7wlJ+YdA+aUs5KQVQhCA7BI11sVW/75LRER0V60S\n/GXLliEkJAQhISFo164dVq5cCUdHR6xevbra/hs3bkRxcTHWr18PHx8fjB49GnPnzsWXX34p9lm+\nfDneffddBAQEwMPDAx9++CH8/f2xc+fOurkyokbgfPwdFJjLoZVKYKowRmtLU7i3MDV0WNTIdfd3\nxtSXnwAAyLQCzK5mIT2jEBCA8BuZOJWSZ+AIiYioMasxwVer1Th58iQGDRqk1z548GAcPny42nNi\nYmIQGBgIuVwutg0ZMgTJyclISEi472vl5+fDyoqjm9Q8lGm0uHApA7ktdH8PTE2NEOxmJY7MEj3I\nkAGeGDvSFwBgma+GJLkAmdnFAICVxxNQqNIYMjwiImrEjGrqkJGRAY1GA3t7e712e3t77N+/v9pz\nUlNT4erqWqW/IAhITU2Fm5tblXNWrVqF27dv48UXX3xgPCdOnKgpZHpIvKf1Iym5GOm5BchzM4NM\nIkBVWgKr3EScOHHb0KE1eY/L76ynq4AObU1wPDYbltdLkKSQQlOmRpGZET7+8yjGusprfpKH8Ljc\n14bG+1o/eF/rB+9r3Wrbtq1BXrdRrKKzfft2zJ07Fz///HOVDwZETdX1hEIUWikAiQQmcilaK6Ww\nljeKv3LUREgkEgwf4gSfthaQlWlhcysHuflqFJWU4XiWBhfyOIpPRERV1TiCb2NjA5lMhrS0NL32\ntLQ0ODg4VHuOg4NDtf0lEkmVc7Zt24aXX34ZGzduxNNPP11jwAEBATX2odq5+ymd97TunThxAtcS\nCmhU3FAAACAASURBVFBqZwGpVIoWlko85++JAG87Q4fWpD2uv7NdujyBBYsjcOlqJlQFZciTSWFu\npsTePFOM7NMBliY1vpU/0ON6X+sb72v94H2tH7yv9SM3N9cgr1vjcKKxsTH8/f0RHh6u1x4eHo4+\nffpUe06vXr0QHR0NlUoltu3duxdOTk565Tm//vorXn75ZWzYsAGjRo161GsganRUKi1upJWgSKlb\n09zM1BhBrTm/hB6NXC7De7MD4exoCYfUQsjUWqTeKUB6QSnWnOQGWEREpK9W9QKzZ8/Gjz/+iHXr\n1uHixYuYOXMmUlJSMG3aNADA/PnzMXDgQLH/hAkToFQqMXnyZMTFxSE0NBSff/453n77bbHPL7/8\ngkmTJmHx4sXo27cv0tLSkJaWhuzs7Dq+RKKGdzOxELktlQB0yZm/cwu0VBgbOCpqyszN5Xhvdl+0\nVBjDMaUQGo2AlNQCHLyZiUOJfN8kIqIKtUrwx44di+XLl2PRokXo2rUrDh8+jL/++gsuLi4AdJNq\nb9y4Ifa3tLREeHg4kpOT0a1bN7z55puYM2cOZs2aJfb53//+B41Gg1mzZsHJyUn877nnnqvjSyRq\neNduFqDQSrccpqnCGP3crA0cETUHDvbmmDerD6xKNGiZUwqVWoO0O4VYdfwWckrUhg6PiIgaiVoX\nbk6bNk0csb/XDz/8UKXN19cXERER932+gwcP1valiZqcc7dLoLJvASkASzNj9HZpaeiQqJno0M4W\nr4cEYPna4ygwN0ZRsRrXU/Lw9fFbeK+vB5dhJSKixrGKDlFzkpNbghtCxV+toDatoDSWGTAiam76\n93XH2Gfawym5EACQm1eK3fFpOJiQZeDIiIioMWCCT1THzl5IQ6G1rv5eoTDCIE8bA0dEzdH45zpi\nUAd7WGWXAAAyMovw38hrSC9S1XAmERE1d0zwierYvnOpKJPrRuytlHL4O1oaOCJqjqRSCd6a2h09\nTRUwVmsBADdT8/HJgcvQCoKBoyMi+v/27jw+ivr+H/hrZo/sJpv72lzkIpCTKyBHEEEFvNB61qMi\n8lNBRA4tVUpp9avUWlu1tkCr9UDtV/FrtV6ohLNcQYgJhCScuY/d3JtN9p75/P6YZMIajiQENsf7\n+XisuzPzmc1nPi6775l5z3uIJ1GAT0g/Yoxhf01XzdurRwRCpaB/ZuTy8PJSYu3KLKSapQtsRZFh\n9/F6fHiE7pZMCCHDGUUehPSjakMbapXSRY4cB9w5LsrDPSJDXVCAFr9/Yhr0jVKqjsMp4K//LUFh\nndnDPSOEEOIpFOAT0o8+O1wJQSEF+H5KDhl6Xw/3iAwHiXGB+M28VHhbpCP55nYHVn1ViDaHy8M9\nI4QQ4gkU4BPSj3aUNsqv07wBnkoWkivk+msScH9cCBSClH9fUteG574/Dkb5+IQQMuxQgE9IP2mz\nu3D6rAomM2M0HuwNGY6WzZ+AiegqyfptsRGfHa3xYI8IIYR4AgX4hPSTz/Oq4eo4WurtcCEtkgJ8\ncmWpVQq8umQK9BYpNUcQGH6//RRKm9o93DNCCCFXEgX4hPSTLceN8uskNaM7ihKPCA7yxp/vGQcv\nhwAAaLe7sGRzPhyC6OGeEUIIuVIowCekHzRaHDjRYpWnp4arPdgbMtxlZuixZEwUuI4zSqUtVqz5\nT4GHe0UIIeRKoQCfkH6wo7QRVptUwcTb4sTYBJ2He0SGu8d+lobpOq08/fWpBnyaV+XBHhFCCLlS\nKMAnpB98VWhAZ7GSOE4Bfz+VZztEhj2O4/CXR69ChJSpA8YY1u04hWKqj08IIUMeBfiEXKLKVhtO\nNUoXMXKMYVZCsId7RIjEW6vG+nvHQ+OU8u+tDgFL/30UJjvVxyeEkKGMAnxCLtGusiZYrFJ6jq7N\niUnp4R7uESFdUkcG46nMGPCidIqpusWKX355DIJI9fEJIWSoogCfkEvAGMO2Mw2w26U8iIBWB9KT\nwzzcK0LcPXhTMq7z7irbuq+kERtySj3YI0IIIZcTBfiEXIKihnaUN1kAALzIMCZEB52OKuiQgYXj\nOPz+0asQ1y6l5ogiwzsHK7CrrPEiaxJCCBmMKMAn5BJklzTI1XP8Wh0Yn0rpOWRg8tV54ZUHJsC3\nTfq8Wm0u/Pb74yhtsXi4Z4QQQvobBfiE9JHVKWBvZYucfx/QYseYNErPIQNXRkoYlo6PgbrjJlh1\nTVb86ttimJ2Uj08IIUMJBfiE9NHeymaYbU44nSK87AL8nAzJo0I83S1CLugXt6diplItX3RbXGXC\nm6dtsAsU5BNCyFBBAT4hfbSttAkWi3T03t9kR+roEKhVCg/3ipALUyh4/HbxZIxstAMAXC4RRXUO\nvF/ugFMQPdw7Qggh/YECfEL6oMZsw7F6M9rPCvAnjo/0cK8I6ZmQYG+seWA8Imql+zdYbQLy6x14\n41A5REZH8gkhZLCjAJ+QPthW2ggmMlhtTujaHFC5GCaOi/B0twjpscmZUbgzIxKh9VYAQIvJia2n\nG/DekWoP94wQQsilogCfkF4SGcP20iZYbS4wJl1cGxPlB32YztNdI6RX/t+D45DG8fBtaAdjQF19\nO/5dbMR/Thg93TVCCCGXgAJ8Qnopz9CKBqsD7RYnFIIIXZsTmWPp6D0ZfLQaFVYsnoyQShO8W2yw\n2lxoabXhrbwq/Le8ydPdI4QQ0kcU4BPSS9kl0s2B2i1O+Jsc4Bko/54MWslJIZg5NRShpU3QWl1o\narbCYRfw55wyHK41ebp7hBBC+oACfEJ6wWx3Iae6BXa7AEEQEdBih85HjdEjgz3dNUL67NrpYYjR\nazGiwgy1TYCxvh1OkeHFPWdwsLrF090jhBDSSxTgE9ILu8qb4BQZ2i1OaK0uaOwCMsdGQKGgf0pk\n8FIoONw9LxpaBY8RFWYwixONTVY4RYZ1e0uwp6LZ010khBDSCxSVENIL20ql9ByL1QF/k1RHPJOq\n55AhIDTYCw8/MA4ql4jY8lZYG9phtbogMIY/HijFjrJGT3eREEJID1GAT0gPlTRbcLrZAsElwmFz\nSfn3PIdxGXpPd42QfjFnVgIyx0ZA7ZSC/NZaM0SBQWQMr+aU49vT9Z7uIiGEkB6gAJ+QHuo8et9u\ndcLX7IRCZEgdHQqdj9rDPSOkf3AchycemQQ/Xy+oXAyRp5rhaLQAABgY/na4Al+cqPNwLwkhhFwM\nBfiE9IBTELGjTCobaLE4EdBiAwBMouo5ZIgJ8Nfgif83CQCgFBh0Pxrh6+y6u+2beZXYXFgLRne8\nJYSQAYsCfEJ64IcaE8wO6cZWTrMDPu0uAJR/T4amSRMiMXtWAgBAITKwfRWI8/GSl79fUINXD5bD\nIYie6iIhhJALoACfkB747kwDAMBqdcKv2QYOQKTeF5F6X892jJDL5OH7xiEiXPp8Oy0u+OTXYUxo\n192ad5Q1YvWOk2i2Oj3VRUIIIedBAT4hF1HZasOPhlYAnek5UvWcieMoPYcMXRqNEisfnwye5wAA\np081YIzZhdnxXfd8ON7YjhVbj+N0k8VT3SSEEHIOFOATchFfney6qFDdZIXaKaUlTJpAAT4Z2kYm\nBOHeO9Lk6U8/L8JNwTo8Oj4aPCcF/g1WB1ZtP0G18gkhZADpcYC/YcMGJCQkQKvVYuLEidi7d+8F\n2x87dgwzZ86Et7c3YmJi8MILL7gtNxgMeOCBB5CSkgKlUomFCxf2bQsIuYzaHC5s76j/7bAL8K5p\nAwD4eNPda8nwcMctKUhOCgEAMMbwl38cwg2xwXhuRiJ8VAoAgEMQ8Yf9JfiwoAYiXXxLCCEe16MA\nf/PmzVixYgV+85vfID8/H9OmTcONN96Iqqqqc7Y3m82YPXs2IiIikJubi7/85S945ZVX8Nprr8lt\n7HY7QkNDsXr1akyZMqV/toaQfrattBE2l3TEXuMU4W2RLq4dP0YPpZJOgJGhj+c5LF88GRqNEgBQ\nazTj3Y/ykRnhj1dnJyPKVyO3/aiwFr/ddRpNlJdPCCEe1aMI5bXXXsPChQuxcOFCjB49Gm+88QYi\nIiKwcePGc7b/8MMPYbVasWnTJqSkpOCOO+7AM888g1dffVVuExsbi9dffx3z589HYGBg/2wNIf1I\nZAxfney6sY+vsR1cx+uJVD2HDCPhoT54dP4EeTp7Zwn25lQg2k+DP18/GuPC/eRlecZWPPFdEQ5W\nt3iiq4QQQtCDAN/pdCI3NxezZ892mz9nzhzs37//nOvk5OTg6quvhlrddQOguXPnoqamBuXl5ZfY\nZUKujEM1JhjapQtqNTwPy0kpVYfjOEwYQwE+GV5mZsVi2lUx8vSGdw6jurYVvl5K/M81I3F3ih5c\nxy5wq92F/9lzBhtzK2B3USlNQgi50i4a4Dc0NEAQBISHh7vNDw8Ph8FgOOc6BoPhnO0ZY+ddh5CB\n5suzjt4n8QrwHanFKaNCoNPR3WvJ8MJxHB5fmAl9uFQq02Zz4Y9vHIDd7oKC57BgbBTWzRqJYK1K\nXufrU/VYmX0cZS1WT3WbEEKGJaWnO9Bbhw8f9nQXhhwa0+4MNhH7S6Sj9xwHNJ+qR3u7VAowOMDW\n4zGjsb08aFwvj56M640zffH39+vhcjEcP2nBcy99jTtviZaX/79whv+rcqDQJB25L2634NHPG3GT\nXoWsEIVcfWc4oc/r5UHjennQuPavpKQkj/zdix7BDwkJgUKhgNFodJtvNBqh1+vPuY5erz9ne47j\nzrsOIQPJ3gaX/HqUlkP5qVZ5OnWU37lWIWRYiAjXYt6crhS1HwtakHukq0Smj5LDQ7Fq3BmtQud1\n6C4R+LLGiQ1nHDDYKGWHEEIut4sewVepVMjMzER2djbuvPNOeX52djbuvvvuc64zdepUPPvss3A4\nHHIe/tatWxEZGYnY2NhL6vDEiRMvaX3SpXMvncbUndnuwonSAnj7SIFIhkaDOm9vAEDyqBDMnZ11\n0fegsb08aFwvj96Oa2Ymg931A3bvk66p2rGvFXOun4T42AC5zSQA80xWvHKgFKUdKTr1AP5Zw+He\ntAjcmRwOlWJoV6Kiz+vlQeN6edC4Xh4mk8kjf7dH365PPfUU3nvvPbz99ts4fvw4li9fjtraWixe\nvBgAsHr1alx//fVy+/vvvx/e3t5YsGABCgsL8dlnn+Hll1/G008/7fa+R44cQX5+PlpbW9HU1IQj\nR46guLi4HzePkN7bWtIIhyAF9wkB3jh1qFpeNmt6nId6RcjAwXEcFj2UiZgo6WyW0yXgT387AMtP\nymPG+mvx6uxk3J8eAWVHao5TZPigoAYrth7Hycb2K953QggZDnqUg3/PPfegqakJ69atQ21tLdLT\n0/Htt98iOlrKuzQYDCgtLZXb+/n5ITs7G0888QQmTZqEwMBArFq1CitWrHB73/Hjx4M7Kx/zq6++\nQmxsLEpKSvpj2wjpNUFk+PpU151rJwd447NKqdyfSqnAtKuiz7cqIcOKRqPEqienYdXvtsFud6HW\naMb6fx7GL5dOcfteVyt4PJAeiazoQPzlh3KcbJKC+jKTFU9vO4HbRoXhFxkR0CgVntoUQggZcnp8\nke3ixYvlI/Y/9e6773abl5aWhl27dl3wPUWRcjHJwHKwugV1FgcAwM9LCcvpJnnZlElR8PGm6jmE\ndIqO9MOShRPx2sYcAMCBQ5X4/JtA3HFLcre2cQFa/Hn2aHx5sg6bjtbAIYgQGcPnJ4zYW9mMRRNi\nMCXK323ngBBCSN8M7QRIQnrpy1NdpTHnxAdj/4EKeZrScwjp7uqpIzD32kR5+sNPjmLXvrJztuU5\nDj8bHY6NN6ZibLivPL/e4sCLe8/g+T1nYGizX+4uE0LIkEcBPiEdTjS2o6DODEAKRGJsIlrNUrAR\nFKjFmLTwC61OyLD18APjkDo6VJ7+21uHkF9w/nue6HVeWDczCSsnx8HPq+tE8qEaEx7fUoTNhbXy\ndTCEEEJ6jwJ8Qjq8f7RGfj1jRCB+/KGqa3paLHieUgcIORe1SoFnV2RhRLQ/AEAUGf741/04U9Z8\n3nU4jsP18cF486Y03JgYIt8F1yGKeL+gBku/K0a+ofW86xNCCDk/CvAJAXC0zox8oxRM8ByHW+OD\ncSivK+Cn9BxCLkzno8baX16N4CCppKzN5sKLf9oDQ13bBdfz9VJi6aRY/Hn2aCQGesvzq802rNl1\nCr/fV4K6dsdl7TshhAw1FOCTYY8xhg/POnp/XVwQzhQYIXSkCIxMCJLLARJCzi84yBtrf3k1dD7S\nxeimVhv+55X/osVku+i6o4N98PqcZCyeEAPvsyrq7KtsxuIthZS2QwghvUABPhn2cg2tKGyQjjIq\nOekmPDv3lsvL6eg9IT03Itofz67IgqojSDcY27Du1b2w2pwXWVM6ezZvVBjevDkN18YFyfPtgpS2\n88S3RThc65mbxhBCyGBCAT4Z1hhj+OCso/c3jAyBo8WGM6VSeUylksfVU0Z4qnuEDEqpo0Oxcslk\nueTlmdImvPz6/m43wjqfQK0KT0+Jxx+vG42EgK60nZo2O363+zReoGo7hBByQRTgk2Etp9qE080W\nAICa53FPih679pbJyyeNj4ROR7XvCemtKROj8dhDE+Tpo0VG/PqFHWhotPT4PdJCdXLajk7VlbaT\nU92Cx7cU4cOCGthclLZDCCE/RQE+GbZExvBBQdfR+5uTQhGoUWH3/q7a9zMpPYeQPpt7bSLuvytd\nnq6oMuGZ57ZfsLrOTyl4KW3nHzenYU5CiDzfIYr4qLAWi7cUYl9lMxhj/dp3QggZzCjAJ8PWnopm\nlJusAACtUoG7UsKRX2BAc4s0z99Pg/EZek92kZBB765bU7H0kUlymdlmkxW/WbfTrUpVTwRoVFh+\nVSxenZ2MpCAfeX69xYHf7yvB2l2nUdl68Yt5CSFkOKAAnwxLgsjwr2O18vRto8IQoFFh51npOVdP\nHQGlkv6JEHKprp0Rj9+umgEfbyndzW534Q+v78M3W0/1+r1GB/vg1dmjsWxSrNtNsvKMrXji2yL8\nM68KFqfQb30nhJDBiKIXMixtL2tEtVk62qdTKXB7chhazXb8kNt1VPHaq+M81DtChp4xaeH4/dpr\nERYiHX1njOHtD/Pw9gd5cLp6F5DzHIe5iSF486Y03JIUCr7jYl6BMXx+wohHvy7E1pIGiJS2QwgZ\npijAJ8OOUxDx0VlH7+9IDodOrcR/vjkuBxrxIwIQNyLAU10kZEiKifLDH353HZISguV532SfwlNr\nslFQVNfr9/P1UuLxzBH4y5xkpIXq5Pktdif+8kM5nso+gRON7f3Sd0IIGUwowCfDzpbTDaizSHfG\n9PNS4tZRYWgx2bAl+7Tc5q7bUj3VPUKGtAB/DV749UxMnRQtz6uubcXv/rALr27IQVPHNTC9kRDo\njZevHYVnpsYjRNtV9epUUzueyj6OP+eUoamHJToJIWQooACfDCuGNjveL6iWp+9O0UOrUuCzr4/D\n0ZG3GzciAJMzozzVRUKGPLVagaefmIqFvxgPjaYrj35vTgWW/upbfP39SflO0j3FcRxmxAbhHzen\n4t7UCKj5rp+3HWWNeOwb6W64diqrSQgZBijAJ8MGYwxvHCqX62aP8NPilqRQNDZZ8P32M3K7++5I\nlyt+EEIuD57ncMucJPztjzdi+lk3k7PZXHjnX/lY9bttOJRXA1HsXR69RqnAg2MisfGmVEyL7kqz\ns7oEvF9Qg0VbCvHf8iYqq0kIGdIowCfDxndnGnDEaAYgXaS3YnIs1Aoe//6qK/d+ZEIQJo6P8GQ3\nCRlWggK0eGrJFDz3zDWIivCT55dVtOCl1/Zi8dPf4NMvi3qduqPXeWHN9ESsm5mEEX5aeX69xYGX\nD5Til9tO4HhDW79tByGEDCQU4JNhwdhux9v5Xak5t48Ow+hgH9Q1tCN7V4k8/7470sFxdPSekCtt\nTFo4Xl03G7+4OwNqdVfaTkOjBf/76TE8tuJr/PGN/ThSaOzVUf1xej/87YYUPDFxhFtZzeON7Xh6\n2wn8cX8pDG32ft0WQgjxNOXFmxAyuDHG8NcfKmDtOEof7avBA+mRAIBPvyiWc32Tk0IwLiPcY/0k\nZLhTKRW4Y14KZkyLxZbsU9i2uxRt7dIF8aLIkHO4CjmHqxAS7I0JYyKQOTYC6amh0GpUF3xfBc/h\nppGhuGZEEDYX1eLLk3Vwduwk7K5owv6qZtw0MhT3pOoRcJH3IoSQwYACfDLkZZc0Is/YCgDgIKXm\neCl5GIxt2P7fUrndfXfS0XtCBoKQYG/Mv3cs7r0zHTmHqvD9jhIUn6yXlzc0WrB15xls3XkGCgWP\n1NEhGJehR+bYCERH+p33GhoftQILx0XjxsRQvHOkCvurWgAATpHhi5N12FrSiNuTw3DH6HBoVYor\nsq2EEHI5UIBPhrQGiwNv5VXJ07eNCkVKiFQve/N/CuUL7dJTwpCRGuaRPhJCzk2tUmDGtFjMmBaL\niioTtu48g117y2E5q+SlIIgoKKpDQVEdPth8FDofNUYlBmN0UjBGjwxGUmJQtyP8Eb5Sfv6xOjPe\nO1qD4o5cfKtLwP8eq8U3p+pxb1oEbkgMgVpBmayEkMGHAnwyZElVcypg6UjNidR54cExUvnLqppW\n/Hd/hdz2vjvTPdJHQkjPjIj2xyMPTsBD943FiVON+PGoAXlHDSivbHFr19buwI9Ha/HjUelmdhzH\nITbGH0mJQUiIDURifCBGRPtDrVIgPcwXr1w3Cj/UmPDekRpUtEoX8prsLvzjx0p8dtyIu1L0mJMQ\nTIE+IWRQoQCfDFnby5qQW2sCIKXmLL8qFhql9CO9+fOuo/fjMvRIGRXisX4SQnpOpVQgPSUM6Slh\nmP/zMWhqtiLvqAE/HjXgWHEdzD+5YJYxhrKKFpRVdO0IKBQ8YqP9kRgfiIS4QMTG+OOVmSORYzTj\nw4Ia1HfcCK/e4sDG3ApsLqzFnSnhuCExVP4OIYSQgYwCfDIknWm24O+5lfL0LUmhSA/zBQCUV7Zg\n38GutJ377ki74v0jhPSPoEAtrrsmHtddEw/GGAzGNhw/3YgTpxpx4nQjKqpM3WreC4KIkvJmlJQ3\nu80PDfZBYowfQoO1KGQCnBwHpYpHk82Jt/Kq8EmRAbcnh+OWkaGUo08IGdAowCdDTq3Zjt/uOi1X\nzdH7eGHBWKlqjsMh4K9vHgIg/eBPHB+JpMRgT3WVENKPOI5DhN4XEXpfzJoeBwCwWJ04VdKEM6XN\nOFPajJKyJhjr28+5fn1jO+obpWVqDmgP1KA2WAPmpYRaxaNepcDrdW1452A5ro0JxB1pEUgI19HF\n+YSQAYcCfDKktNicWLv7FFrs0kV4OpUCa69OhEYpHW17c9OP8lE7hYLH/ZR7T8iQ5q1VYWxaOMam\ndZXAbWtz4Ex5M0rKmlFa3oLyShOqa1vd6uvzDAhusiGw2YaWAC80BmthVUnpOY0A3q414+0fKhDQ\n5sRIxiHRT4vQYG+EBGsRHOSN4EAtjPU2+PuqwBijnQBCyBVFAT4ZMqxOAb/bfRq1HTm4ap7H2qsT\nERcg3cUye1cJduzpKou58IFxiBsRcM73IoQMXTqdulvQ73QJqKk1o7zShPIqEyoqTagxtMFY34ag\nZjsCWuww+XuhMVgDh7orPadFp8JhAMesVgQdbYZfqwN8x35Ce7sFABD8fjVCg70RHqaDPkwHfZiP\n9ByuQ2iIN1RKSvchhPQvCvDJkOAQRLy4twSnm6UfVJ7j8My0eDnv/nRpE/75fp7c/pppsbjhukSP\n9JUQMvColArExgQgNsZ9p18QRNTVt6PW2IYaYxtqaluR39COQqcT9Rzko/42jRI1kToYwxn8Wu3w\nNznA2gEOgM3mQmV1KyqrW7v9XY7jEBXhi7gRAYiPDUBcjPQc4K+5EptNCBmiKMAng57IGF47WIZ8\nY9eP59KJIzAlWvqhNrfZ8cobB+DsyMkfEe2PxQ9n0ilzQshFKRS8nNc/4SfLylqs+HdhLXaUNsHi\ncMHlEuESRLj8vWCMEIE2b2jr2qGyMKid4jnfnzGGqppWVNW0Ym9OV+neAH8NEuMCkTI6FKmjQ5AY\nH0hH+gkhPUYBPhnUGGN4K68K/63oqobxYEYk5iZKZS9FkeG1jQflC+e8tSo8s2wavLzoo08IuTRx\nAVo8nZWARyeOwNaSBmw53QBje1eZTku7Aoj0gaDRIlznhXilEiF2AdYmKwx1bTDUtaOxyYrOi/7P\n1mKyIfdILXKPSPX81SoFRo0MRmpHwJ+cFAK1mgJ+Qsi5UZRDBi2bS8D6w5XYUdYoz7slKRQ/T9XL\n05/8pxD5BQZ5evmiyYjQ+17RfhJChjY/LyXuStHjjuRwFNW3YUdZE/ZUNMPSsZxXcKiwOlABqb5+\nbJQPplwViV9EBSDGW42q6laUVrSgtFyq119WaYLD4XL7Gw6ngGPFdThWXAdASilKTQ7F+IxwjMvQ\nIybKj85KEkJkFOCTQancZMVL+0pQ2WqT52XFBGLRhBj5Ry73SC0++U+RvPyOW1IwaULkFe8rIWR4\n4DkO6WG+SA/zxaIJMfhg1w843CSgmnEQz6rFX26yotxkxeYiA4I0KozX+yEjzh+3T45GuI8XRJGh\n1mjG8ZONKDpZj6Lj9d1KezpdAo4cM+DIMQPw0REEBWoxLl2PcRnhGJseDl+d15XefELIAEIBPhl0\nsksasCG3Eg6hK6f1+vhgPDFxBHiOA2MM23aXul1UOyY1HPffRSUxCSFXhpeSx7gAJcYFKJGYloGc\n6hYcrDHhiMEMh9j13dVkc2J7WSO2d5yJDPfxQkaYDhlhvkjPjMS1M+LAcRwamywoOtGAohP1KCiq\nQ43B7Pb3mpqt2LGntKNSGIeRCYEYn6HHuAw9RiUGQaGgO/ASMpxQgE8GDZtLwIbDlfIPIQCoFTyW\nZMZgdoKUc2+3u/CPTT9i194yuU1QoBYrl0wGz9Ppa0LIlReoVeHGkaG4cWQorE4BeYZWHKwxyA12\nXQAAFe1JREFU4YcaE1rt7qk4xnY7jKV2bCuVvud81UqMCvLGyCBvjIrxw51j9XhMq0J9owX5BUYc\nOWbA0cI6tFscZ70Lw+mSJpwuacL/fVEEb60KY9LCkZEahrTkUErnIWQYoACfDAqnmyz4c04ZKlqt\n8rwYPw1WZyUg1l+qc19d24o/vnEAldWmrjZR/nhm+TT4+1HJOUKI52lVCkyLCcS0mECIjOFkYzsK\n6tpQUGdGYUMbbC73ajtmhwu5hlbkGrqqhAVqVIjz1yLWX4OxN43CTT/PgKPRihPF9ThyzICTZ5rA\nzkoJslidyDlchZzDVQAAP18vpCWHIi05FOkpYRTwEzIEUYBPBrTihjZ8XGjA4VqT2/xr44KwJHME\ntCqpisS+g5VY//Yh2GxdR8NmTo/DoocmUMUcQsiAxHMckkN0SA7R4e5UPVwiw+mmdhztCPhPNraj\nzSl0W6/Z5kSzzYk8o3td/VBvNWJmjMCM6xNgb7ahqcqEypMNaG+wyDffAoBWsx0HDlXhwCEp4Nf5\nqJEQG4iEuAAkxgchMS4Q4WE+FPQTMohR5EMGHMYY8gxmbC4y4Fi9e56pmufx+MQYzI4PBsdxaDHZ\n8H9fFOHbbaflNiqlAo8+NB7XzYinHyhCyKCh5LsC/ntS9WCMobbNjtNNFpxssuBUkwVnmi2wuroH\n/QBQb3Gg/uxUHX8VMCkCLqcIhUOA2OZAe4MFzOKEyil2PAQIVgeOFhlxtMgor+rjrUZ8bAAi9b6I\nivCFPlyHSL0OYaE+VI+fkEGgxwH+hg0b8Kc//Qm1tbVIS0vD66+/junTp5+3/bFjx7B06VL88MMP\nCA4OxmOPPYa1a9e6tdm9ezeefvppFBYWIioqCqtWrcKiRYv6vjVkUGtzuJBb24rPjhvlO9J24sBh\nWkwAfpEegRg/DYpONOC77aeRc7gawlkX24aH6fCrJ6chPjbgp29PCCGDCsdxiPTVINJXgxmxQQCk\nG/vVmO0oN1lRYbKhotWKcpMN1a02uFj3evoAoFTxgIqHwkcFvzAfOJwCrDYXWm0uWK1OiCIDLzIo\nXSKUrs5nEeV1rVDUmsDnMigEaZlCBCICNQgP9kFwgBaBARoEBUrPgR3Tvjo1fLzVdN1TLzDG4BAY\nbIIIm0uAzSXC7hJhEzqeXWfNF6RphyC9dggMDqFzmsEpiBAYgyAyCAwQRAaXyCB0fD46/69wnPTb\nCkg7l6ZmO1Q8EGE+BbWCg1LBw0vBQaNUwFulgLeSh1bV9dpHrYSvWgE/LyV81Up4KelC7oGkRwH+\n5s2bsWLFCvz9739HVlYW1q9fjxtvvBHFxcWIjo7u1t5sNmP27NmYOXMmcnNzUVxcjAULFkCn02Hl\nypUAgLKyMtx888145JFH8K9//Qt79uzBkiVLEBYWhttvv71/t5IMSCJjONNswY+1rcitbUVxY7tb\nKTkAUHAcZsUF4a4UPYKUPHbvK8efd5w55y3fJ2dGYemjk+Djrb5Sm0AIIVcUz3GI9tMg2k+DrJiu\n+S6RodpsQ63ZjmqzHbVtNtS2OVBttqHB4gTrvJkWB6jVCqjVCvj7SaU0XS4RdrsLNrsAu0NAm90F\nUTz3zgIAnJLWAt/UCr7BBL5jB0F6ABxj4BjgpeLhpVRAo1ZAq1bAS62ARq2E5uzXXgpoNB3zvJTQ\nqJXw8lKA5zmcb/egtM4JACgrNpynRZefvkvnSV3uJ9Pnw5h0GzLpmXW9ZlLwLDIG8ezXIoOLMTgF\nBqfI4BJFuEQpeHeJrCMgl4L2ztcOQZr/09+/K81ikQ6WGVj339eeUCt4+KmV8PNSwt9LiQBN57MK\n/me9DtAoEahRQU2VnS6rHgX4r732GhYuXIiFCxcCAN544w1899132LhxI9atW9et/Ycffgir1YpN\nmzZBrVYjJSUFxcXFePXVV+UAf+PGjYiKisLrr78OABg9ejQOHjyIP/3pTxTgD0EukcHQZke12YbK\nVhtKW6zIM7TC9JMKEp1UPI+scF+kKZVoKjfjnd0VOH6qAfZztB89Mhg3zU7C9CkxlJJDCBmWlDyH\nWH+tXHTgbA5BhLHdgfp2KYWnrt2BOkvX62arE0olDx+frnVcLhEOhwCHU4DTKcLpFOB0iXCddRGw\nyHMQe3KUnomAXQTszh5vD89zUPAceAUHBc93PHPgeR4ulwM8B3gbyqBQ8FJbBUdnDDzMIYhosDrQ\nYHVcvDEAb6UCgVoVAryUHTsAKnmnoHPav2NnQadWQEU7BL1y0QDf6XQiNzcXq1atcps/Z84c7N+/\n/5zr5OTk4Oqrr4Za3XUkde7cufjtb3+L8vJyxMbGIicnB3PmzHFbb+7cuXj//fchCAIUCsrxG8gY\nk45OdB6FMDtcaLW7YLK70GoX0GqXpuvaO48m2eXTgzjrSIfgEqXTh4IIQWAIAgffdidQ0oIDZjsO\nnOfve3kpMWPaCNxw7UhKxyGEkAtQK3jE+GkQc55qYowxtDsFNFmdaLa50Gx1yhfytjkEmB0umB0C\n2hwumKwuNLU7YHG4IAid390iXIL0fe4SRIgiu+AZgJ6Q38MFAMJPlkk7GSZz92sROgN9BS8F/p0P\nRccz1zmP63wGOJ4Dx3HgeemIvzR9Sd3vExUvpcN4KXholNLDS8l3zJOWaZS8vNxLyUPN81ArOOm1\nouPRsb1KnoOCk153PgPS/+/O/zudP8sukSG/4BgExjBy9Eg4BQaHKJ1dsDoFWF0iLE4BFqcAq0tA\nu0NAu1NEq90lfT7srvOmiJ2PxSXAYhZQbb54WwDQKhXQnZUSdHdKOMbp/Xr1N4eTiwb4DQ0NEAQB\n4eHhbvPDw8Oxffv2c65jMBgQExPTrT1jDAaDAbGxsTAYDJg9e3a3Ni6XCw0NDd3+XqebX//vxbo8\n7PTmn1TXP2oGq9UKBkC7ZzcY1zlfaiPirGeu67UA6ftWRNcXQ+e7so7/MDD5D7GOYJ4xJp/WPPs7\nQOkS4dPuhK7NCZ92J5QCg/0C/Y+J8sPcaxMxc3ocvLWqXmw5IYSQc+E4Djq1Ejq1EiP8e7aOIDLY\nXCKsLgF2QYTV2fXaJTDYXALaLE6YLQ6YLU60WZ2wdqQBWR3Ss80uwO50wW4XYHO44LCLsDlccJ7n\nIuJODod0JkCtvthvgHuUfvZvWU8oeB4KhRT8SzsIUoqUgu/aOeAhPSs4aaeBA6DgAB7SPAU48IDU\nBoCyY5kS0nTnQwkO/HmSkjp3Nmwdj675/bsXYjRKF1mf+fHiZ1qUAPw7HoD0u+8C4ABg73zmABuk\n33Q7J823AbCDwc71LnY5l7HeagrwL2DQVdH534fHeroLZABwOiww9ewsoEckJSUBAEwm00Vakt6g\ncb08aFwvj+EwrqqOh64zUgUgBdZKIFAJoHvKEBmoMjzdgV4byv+2LtVFE5pCQkKgUCjkPbtORqMR\ner3+nOvo9fpztuc4Tl7nfG2USiVCQkJ6tRGEEEIIIYQQyUUDfJVKhczMTGRnZ7vNz87ORlZW1jnX\nmTp1Kvbs2QOHo+sQ69atWxEZGYnY2Fi5zU/fc+vWrZg4cSLl3xNCCCGEENJHHGMXvyrik08+wfz5\n87F+/XpkZWVh48aNePfdd1FUVITo6GisXr0ahw4dwrZt2wAAra2tSE5OxsyZM7FmzRqcOHECDz/8\nMJ5//nmsWLECgFQmMyMjA4888ggWLVqEvXv3YunSpfj444/xs5/97PJuNSGEEEIIIUNUj3Lw77nn\nHjQ1NWHdunWora1Feno6vv32W7kGvsFgQGlpqdzez88P2dnZeOKJJzBp0iQEBgZi1apVcnAPAHFx\ncdiyZQtWrlyJv//974iMjMRf//pXCu4JIYQQQgi5BD06gk8IIYQQQggZHAbFXQM2bNiAhIQEaLVa\nTJw4EXv37vV0lzxmz549uO222xAdHQ2e5/H+++93a/Pcc88hKioK3t7emDVrFoqKityWOxwOPPnk\nkwgNDYVOp8Ntt92G6upqtzYtLS148MEHERAQgICAAMyfP7/b1eqVlZWYN28edDodQkNDsXz5crhc\n575x1UD20ksv4aqrroK/vz/CwsJw6623orCwsFs7Gtfe27BhA8aOHQt/f3/4+/tj2rRp2LJli1sb\nGtdL89JLL4HneSxbtsxtPo1r7z3//PPged7tERkZ6daGxrVvDAYDFixYgLCwMGi1WqSnp2PPnj1u\nbWhseyc+Pr7b55XnecybN09uQ2Pae6IoYu3atXLcmZCQgLVr18r3X+g04MeWDXAff/wxU6lU7O23\n32bHjx9nTz75JNPpdKyystLTXfOILVu2sDVr1rB///vfzMfHh23atMlt+R/+8Afm5+fHPv/8c1ZY\nWMjuueceFhkZydra2uQ2ixcvZlFRUWz79u0sLy+PzZw5k40bN46Joii3ueGGG1h6ejo7ePAgy8nJ\nYWlpaezWW2+VlwuCwNLT09msWbNYfn4+27ZtG4uMjGTLli27/IPQz2644Qa2adMmVlhYyI4dO8Zu\nv/12ptfrWXNzs9yGxrVvvvzyS/bdd9+xM2fOsFOnTrE1a9YwlUrFCgoKGGM0rpfqwIEDLD4+no0b\nN449+eST8nwa17557rnnWEpKCqurq2NGo5EZjUbW0NAgL6dx7ZuWlhaWkJDAFixYwA4fPszKysrY\njh072PHjx+U2NLa919DQIH9OjUYjy8vLYzzPsw8++IAxRmPaV+vWrWPBwcHsm2++YeXl5eyrr75i\nQUFB7MUXX5TbDIaxHfAB/uTJk9miRYvc5iUlJbFf//rXHurRwKHT6boF+BEREeyll16Sp61WK/P1\n9WVvvvkmY4wxk8nE1Go1++ijj+Q2lZWVjOd5tnXrVsYYY0VFRYzjOHbgwAG5zd69exnHcezkyZOM\nMWlHQ6FQsOrqarnNhx9+yLRaLTObzf2/sVdQW1sbUygU7Ouvv5bn0bj2n6CgIHncaFz7rqWlhSUm\nJrJdu3axmTNnugX4NK5989xzz7GMjIzzLqdx7ZvVq1ez6dOnX7ANje2le/HFF1lgYCCz2WyMMRrT\nvrrlllvYggUL3OY99NBDbN68efL0YBjbAZ2i43Q6kZub2+2Ot3PmzMH+/fs91KuBq7S0tNsdgjUa\nDWbMmCGP1+HDh+FyudzaREdHIyUlRW6Tk5MDX19fTJkyRW6TlZUFHx8ftzYpKSlup6/nzp0Lm82G\n3Nzcy7qdl1traytEUURgYCAAGtf+IooiPv74Y7S3tyMrK4vG9RI99thjuOeee3DNNde4zadxvTQl\nJSWIiopCQkIC7rvvPrmABI1r333xxReYPHky7r33XoSHh2P8+PFYv369vJzGtn+88847ePDBB+Hl\n5UVjegmmT5+OnTt34sSJEwCAoqIi7NixAzfffDOAwfN5HdB3sm1oaIAgCAgPD3ebHx4eju3bt3uo\nVwOXwWAAx3HnHK+amhoA0s3EFAoFgoODu7UxGAzy+4SGhnZ7/7CwMLc2P/07nTdF62wzWC1fvhwT\nJkzA1KlTAdC4Xqpjx45h6tSpsNls8PX1xeeff47U1FQcOHCAxrWP3nrrLZSUlOCjjz7qtow+r303\nZcoUvPfee0hOTkZdXR1eeOEFZGVlobCwkMb1EpSUlGDDhg1YuXIlVq9ejfz8fCxduhQcx2HJkiU0\ntv1g69atKCsrw6OPPgqAvgcuxTPPPAOz2YzU1FQoFAoIgoA1a9Zg0aJFAAbP2A7oAJ+QK+2pp57C\n/v37sW/fPnAc5+nuDAnJyck4cuQITCYTPv30U8yfPx+7d+/2dLcGrZMnT2LNmjXYt28feH5An4Qd\ndObOnes2PWXKFMTHx2PTpk2YPHmyh3o1+ImiiKuuugrr1q0DAIwdOxYnT57E+vXrsWTJEg/3bmh4\n6623MGnSJKSnp3u6K4Pexx9/jA8++AAff/wxUlNTkZ+fj2XLliE+Ph4PP/ywp7vXYwP616FzL8Vo\nNLrNNxqN0Ov1HurVwKXX68EYu+B46fV6CIKAxsbGC7apr6/v9v51dXVubX76dzrPuAzW/zcrV67E\n5s2bsXPnTvmOywCN66VSKpVISEjA+PHjsW7dOowbNw6vvfYajWsfHThwAI2NjUhNTYVKpYJKpcLu\n3buxfv16qNVqBAcH07j2E29vb6SlpeHUqVP0eb0EERERSElJcZuXkpKCiooKAPQde6nq6+vx5Zdf\n4rHHHpPn0Zj23a9+9SusWrUKd999N9LS0vDAAw/gqaeewksvvQRg8IztgA7wVSoVMjMzkZ2d7TY/\nOzsbWVlZHurVwBUfHw+9Xu82XjabDXv27JHHKzMzE0ql0q1NVVUViouL5TZTp05FW1sbcnJy5Db7\n9++HxWLBtGnT5DbFxcXy6ShAOkWo0WiQmZl5Wbfzcli+fLkc3CclJbkto3HtX6Iowm6307j20e23\n346CggIcOXJEfkycOBH33Xcfjhw5glGjRtG49hObzYbjx48jMjKSPq+XICsrS85n7nTixAn5QAqN\n7aV59913odFocO+998rzaEz7zmKxdDs7yvO8XCZz0IxtT68q9pTNmzczLy8v9s9//pMVFxezZcuW\nMV9fX1ZRUeHprnlEW1sby8/PZ3l5eczb25u98MILLD8/Xx6Pl19+mQUEBLDPPvuMFRQUsJ///Ocs\nKirKrXTT448/zmJiYti2bdvYjz/+yGbNmsUmTJjgVrrpxhtvZGPGjGEHDhxg+/fvZxkZGey2226T\nlwuCwMaMGcOuu+46lpeXx7Kzs1lUVBRbvnz5lRuMfrJkyRLm5+fHdu7cyQwGg/w4e8xoXPvm2Wef\nZXv27GFlZWWsoKCAPfvss0yhULDvv/+eMUbj2l9+WkWHxrVvfvnLX7Ldu3ez0tJSlpOTw26++Wbm\n7+9P36+X6NChQ0ytVrN169ax06dPs08++YT5+/uzjRs3ym1obPtu1KhR3aoNMkZj2lcLFixgMTEx\n7JtvvmFlZWXss88+Y6GhoWzVqlVym8EwtgM+wGeMsY0bN7L4+Him0WjYxIkT2d69ez3dJY/ZtWsX\n4ziO8Tzv9nj44YflNs8//zyLjIxkWq2WzZw5kxUWFrq9h8PhYMuWLWMhISHMx8eH3Xbbbayqqsqt\nTUtLC3vwwQeZv78/8/f3Z/Pnz2cmk8mtTWVlJZs3bx7z8fFhISEhbMWKFczhcFy+jb9MzjWePM+z\n559/3q0djWvvLViwgMXFxTGNRsPCw8PZ7NmzWXZ2tlsbGtdLN2vWLLcAnzEa17649957WVRUFPPy\n8mLR0dHsrrvuYsXFxW5taFz7ZsuWLWzs2LFMq9Wy0aNHs7/97W/d2tDY9t7OnTsZz/Ps8OHD51xO\nY9p7bW1tbOXKlSwuLo55e3uzxMRE9pvf/IbZ7Xa3dgN9bDnGGOvDGQxCCCGEEELIADSgc/AJIYQQ\nQgghvUMBPiGEEEIIIUMIBfiEEEIIIYQMIRTgE0IIIYQQMoRQgE8IIYQQQsgQQgE+IYQQQgghQwgF\n+IQQQgghhAwhFOATQgghhBAyhFCATwghhBBCyBDy/wFeCY7fzdUj4QAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from price import *\n", + "import matplotlib.pyplot as plt\n", + "player1, player2 = MakePlayers(path='../code')\n", + "MakePrice1(player1, player2)\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This shows the distribution of prices for these\n", + "showcases. The most common value for both showcases is around \\$28,000,\n", + "but the first showcase has a second mode near \\$50,000, and the second\n", + "showcase is occasionally worth more than \\$70,000.\n", + "\n", + "These distributions are based on actual data, but they have been\n", + "smoothed by Gaussian kernel density estimation (KDE). Before we go on, I\n", + "want to take a detour to talk about probability density functions and\n", + "KDE." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Probability density functions\n", + "\n", + "So far we have been working with probability mass functions, or PMFs. A\n", + "PMF is a map from each possible value to its probability. In my\n", + "implementation, a Pmf object provides a method named Prob\n", + "that takes a value and returns a probability, also known as a\n", + "**probability mass**.\n", + "\n", + "A **probability density function**, or PDF, is the\n", + "continuous version of a PMF, where the possible values make up a\n", + "continuous range rather than a discrete set.\n", + "\n", + "In mathematical notation, PDFs are usually written as functions; for\n", + "example, here is the PDF of a Gaussian distribution with mean 0 and\n", + "standard deviation 1: \n", + "\n", + "$$f(x) = \\frac{1}{\\sqrt{2 \\pi}} \\exp(-x^2/2)$$\n", + "\n", + "For a given value of $x$, this function computes a probability density. A\n", + "density is similar to a probability mass in the sense that a higher\n", + "density indicates that a value is more likely.\n", + "\n", + "But a density is not a probability. A density can be 0 or any positive\n", + "value; it is not bounded, like a probability, between 0 and 1.\n", + "\n", + "If you integrate a density over a continuous range, the result is a\n", + "probability. But for the applications in this book we seldom have to do\n", + "that.\n", + "\n", + "Instead we primarily use probability densities as part of a likelihood\n", + "function. We will see an example soon." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Representing PDFs\n", + "\n", + "To represent PDFs in Python, `thinkbayes.py` provides a class\n", + "named `Pdf`. `Pdf` is an **abstract type**, which means that it defines the interface a Pdf is\n", + "supposed to have, but does not provide a complete implementation. The\n", + "`Pdf` interface includes two methods, `Density`\n", + "and `MakePmf`:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Pdf(object):\n", + "\n", + " def Density(self, x):\n", + " raise UnimplementedMethodException()\n", + "\n", + " def MakePmf(self, xs):\n", + " pmf = Pmf()\n", + " for x in xs:\n", + " pmf.Set(x, self.Density(x))\n", + " pmf.Normalize()\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Density` takes a value, `x`, and returns the\n", + "corresponding density. `MakePmf` makes a discrete\n", + "approximation to the PDF.\n", + "\n", + "`Pdf` provides an implementation of `MakePmf`, but\n", + "not `Density`, which has to be provided by a child class.\n", + "\n", + "A **concrete type** is a child class that extends an\n", + "abstract type and provides an implementation of the missing methods. For\n", + "example, `GaussianPdf` extends `Pdf` and provides\n", + "`Density`:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class GaussianPdf(Pdf):\n", + "\n", + " def __init__(self, mu, sigma):\n", + " self.mu = mu\n", + " self.sigma = sigma\n", + "\n", + " def Density(self, x):\n", + " return scipy.stats.norm.pdf(x, self.mu, self.sigma)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`__init__` takes `mu` and `sigma`, which are the\n", + "mean and standard deviation of the distribution, and stores them as\n", + "attributes.\n", + "\n", + "`Density` uses a function from `scipy.stats` to\n", + "evaluate the Gaussian PDF. The function is called `norm.pdf`\n", + "because the Gaussian distribution is also called the “normal”\n", + "distribution.\n", + "\n", + "The Gaussian PDF is defined by a simple mathematical function, so it is\n", + "easy to evaluate. And it is useful because many quantities in the real\n", + "world have distributions that are approximately Gaussian.\n", + "\n", + "But with real data, there is no guarantee that the distribution is\n", + "Gaussian or any other simple mathematical function. In that case we can\n", + "use a sample to estimate the PDF of the whole population.\n", + "\n", + "For example, in *The Price Is Right* data, we have 313\n", + "prices for the first showcase. We can think of these values as a sample\n", + "from the population of all possible showcase prices.\n", + "\n", + "This sample includes the following values (in order):\n", + "\n", + "$$28800, 28868, 28941, 28957, 28958$$ \n", + "\n", + "In the sample, no values appear\n", + "between 28801 and 28867, but there is no reason to think that these\n", + "values are impossible. Based on our background information, we expect\n", + "all values in this range to be equally likely. In other words, we expect\n", + "the PDF to be fairly smooth.\n", + "\n", + "Kernel density estimation (KDE) is an algorithm that takes a sample and\n", + "finds an appropriately smooth PDF that fits the data. You can read\n", + "details at .\n", + "\n", + "`scipy` provides an implementation of KDE and\n", + "`thinkbayes` provides a class called\n", + "`EstimatedPdf` that uses it:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class EstimatedPdf(Pdf):\n", + "\n", + " def __init__(self, sample):\n", + " self.kde = scipy.stats.gaussian_kde(sample)\n", + "\n", + " def Density(self, x):\n", + " return self.kde.evaluate(x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`__init__` takes a sample and computes a kernel density estimate. The\n", + "result is a `gaussian_kde` object that provides an `evaluate`\n", + "method.\n", + "\n", + "`Density` takes a value, calls `gaussian_kde.evaluate`, and\n", + "returns the resulting density.\n", + "\n", + "Finally, here’s an outline of the code I used to generate\n", + "Figure [fig.price1]:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Warning: Brewer ran out of colors.\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvgAAAEWCAYAAAAThOOMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XtUVOe9P/73nmEGZrhfZ7grCIiAV4wXNMEoEtPc843f\nHtsk1tOT+DtpqkmWTT1J2/RYV5uTJiY5J2rP+boSW3Ma08akSROtaDQREQ0oKjcBuchtBpD7dWBm\n//7AbJmAgsiwZ4b3ay1W5tk8m3nvJ4ofNs9+HkEURRFEREREROQUFHIHICIiIiKiicMCn4iIiIjI\nibDAJyIiIiJyIizwiYiIiIicCAt8IiIiIiInwgKfiIiIiMiJsMAnIiIiInIiYy7wd+7ciaioKGg0\nGiQnJyMzM/Om/fPz85GamgqtVovw8HBs27bN6vNfffUVFAqF1YdSqURJScn4roSIiIiIiOAylk77\n9+/H5s2bsXv3bqSkpOCdd97BmjVrUFRUhLCwsGH9Ozo6kJaWhtTUVOTm5qKoqAjr16+Hh4cHnnvu\nOamfIAgoLCyEr6+vdCwwMHACLouIiIiIaGoSxrKT7eLFizF37lzs3r1bOhYbG4vHHnsM27dvH9Z/\n165d2Lp1KxoaGqBWqwEA27dvx+7du1FdXQ1g8A7+3XffjcbGRvj5+U3U9RARERERTWmjTtHp7+9H\nbm4u0tLSrI6vXr0aWVlZI56TnZ2N5cuXS8U9AKSnp6Ourg5VVVXSMVEUkZycjJCQEKxatQrHjx8f\n52UQEREREREwhgK/qakJZrMZOp3O6rhOp4PBYBjxHIPBMGJ/URSlc4KDg7F792589NFH+PjjjxEX\nF4eVK1fi5MmT470WIiIiIqIpb0xz8G0hNjYWsbGxUnvRokWorKzEa6+9hpSUFKu+bW1tkx2PiIiI\niGjCeHt7T9p7jXoHPyAgAEqlEkaj0eq40WiEXq8f8Ry9Xj9if0EQbngOMFjkl5aWjiU3ERERERGN\nYNQCX6VSYcGCBcjIyLA6npGRMexO+7eWLFmCEydOwGQySccOHz6MkJAQREZG3vC9zp07h+Dg4LFm\nJyIiIiKi7xjTFJ3nn38eTzzxBBYuXIiUlBTs2rUL9fX12LhxIwBg69at+Oabb3DkyBEAwLp16/Dv\n//7vWL9+PV566SVcunQJr776Kn79619LX/Ott97CtGnTkJCQAJPJhD/96U/49NNPceDAgZtmmcxf\nbzi7nJwcAEBycrLMSZwPx9Y2OK62wXG1DY6rbXBcbYPjahtyTTMfU4G/du1aNDc3Y/v27aivr0di\nYiIOHjworYFvMBhQUVEh9ffy8kJGRgaeeeYZLFy4EL6+vtiyZQs2b94s9TGZTPjZz36GmpoaaDQa\nJCQk4IsvvkB6evoEXyIRERER0dQx5odsN27cKN2x/65333132LGEhISbLnu5ZcsWbNmyZaxvT0RE\nREREYzDqHHwiIiIiInIcLPCJiIiIiJwIC3wiIiIiIifCAp+IiIiIyImwwCciIiIiciIs8ImIiIiI\nnAgLfCIiIiIiJ8ICn4iIiIjIibDAJyIiIiJyIizwiYiIiIicCAt8IiIiIiInwgKfiIiIiMiJsMAn\nIiIiInIiLnIHIKKppbu3E4bmajS21kPj6o5IfQy83f3kjkVEROQ0WOATkc0MmPtR11QFQ3M16q9e\ngaG5Gu1dLcP6+XoGYpo+FtOC4xAeFA1XlZsMaYmIiJwDC3wimnD9AybklZ3CmaIv0d3bOWr/lo5G\ntHQ04lzpSSgEBUIDp+Ouufcj2D98EtISERE5Fxb4RDRhxlLYK5Uu0PmGIcg3FG2dTahuKMeAuV/6\nvEW0oLrhMj44+g4eSHkc0aEJkxWfiIjIKbDAJ6Lb1j/Qj/NlWTg9QmHvqfHG9JB46P3CEewfDn9v\nPZQKpfT5AXM/ahsrUGkoQaWhBA0ttdLxj0+8h7TkRzFnxuJJvR4iIiJHxgKfiMZNFEUUX8nDsbN/\nQ1dvh9XnPDXeWJSwCklRC+GiVN3wa7goVYjUxyJSH4u7AFxtN+Kj4/8PbV3NEEULDn/zF3T2tGFp\n4mobXw0REZFzYIFPROPS3tWKIzkf4XJdodXxwcJ+JZKi7rhpYX8j/l46/GD1T/HRV/8PxuYaAEBW\n/mG0d7fCXzEdCoGr+xIREd0MC3wiuiUW0YLzZafwdd7nMA30ScfdNV5YPGslZkcvGldhP5S7mye+\nf/e/4tOTe1FRfwkAkF9+Bu7KMsyPvPu2vjYREZGzY4FPRGN2td2If5z+ELVNlVbH58ak4M45907o\n8pZqlSsevvOf8Y8zH6KgIgcA0Nheg9PlB5GcnHzbP0QQERE5Kxb4RDQqi2jBN0XHcPLiYZgtA9Jx\nP89ApC9ai7DAKJu8r1KhxJpF34enxhvZhUcBAK1djcjKz8Cdc+61yXsSERE5Ohb4RHRT7V2t+CL7\nfVQ3lEvHFIICd8SvwJLENJvfSRcEAcvn3Au1yg2Hsv4CADhTdAxxEXOg8w216XsTERE5Ij6tRkQ3\ndOnKeew99LpVca/3C8fj6c9h+Zx7J3WazML4VPh56AEAomjBoewPYLaYJ+39iYiIHAULfCIaxjTQ\nh0On9+PTk39Er6kbACAICixNXI0fpD2LIN+QSc+kEBRICl8GxbU19Bta63Cm6Nik5yAiIrJ3nKJD\nRFaMzTX4LGsfWjoapWNe7r743pJ1NptrP1Yert6I1c/HlbYCAMCp/AzEhiXB31snay4iIiJ7wjv4\nRCQ5X5aNfRlvWxX3MyPm4sl7XpC9uP/W9MBE6P3CAQBmywAOnfkQFtEicyoiIiL7wQKfiGCxmHEk\n5wAOf/MXWK7Na1e7uOLexf+E+5b+EG5qjcwJr1MICtyz6P9KU3XqmipxruSkzKmIiIjsBwt8oimu\np68bfz3+PzhXer1IDvIJwRP3PIeE6ckQBEHGdCML9AnG4lkrpfaJ81+grbNZxkRERET2gwU+0RR2\ntc2I9zPeQpWxVDoWFz4H/5T2E/h6BsqYbHSLZ61EgPfgqjr9ZhP+ceZDiKIocyoiIiL5scAnmqLK\n64quzbdvko6lJKXj/pTHoXZxlTHZ2CiVLrhn0fchCIPfxqqMpciv+EbmVERERPJjgU80BeVeOoED\nX+2Bqb8XAKBSqvHgsiexNHG1XU7JuZFg/3Akx90ptbMuHuba+ERENOWxwCeaYs6WZOLLs59AxOB0\nFi+tL9al/QSx4bNlTjY+S5NWQ+vqAQBo725BYWWuzImIiIjkxQKfaAq5WH4GR3M/ltohAdPww/RN\nCPINlTHV7VG7uCJ55vW7+KcLj3LZTCIimtJY4BNNEUVV5/CP0x9K7RD/SDyW+hTc3TxlTDUx5sak\nwPXaUp4tHU24dOW8zImIiIjkwwKfaAooqb6Iz0/9rzQtJ8g3FI+m/gvUKvt/mHYsXFVuWBC7XGpn\nFxzhXXwiIpqyxlzg79y5E1FRUdBoNEhOTkZmZuZN++fn5yM1NRVarRbh4eHYtm3bDftmZmZCpVJh\n9mzHnANMZM/K64rxWdafIF4reP29dXhsxVN2tXnVRJgfu0xa/aepzYDLtQUyJyIiIpLHmAr8/fv3\nY/PmzXj55ZeRl5eHpUuXYs2aNaipqRmxf0dHB9LS0hAcHIzc3Fy89dZbeO2117Bjx45hfVtbW/Hk\nk09i1apVt3clRDTMFWMZPsl8V9qd1tczEGtXbJQeSnUmGld3zI1ZKrVPFRzhuvhERDQljanA37Fj\nBzZs2IANGzYgLi4Ob7/9NoKDg7Fr164R++/btw89PT3Yu3cv4uPj8cgjj+DFF1/EG2+8MazvP//z\nP2P9+vVYvHjx7V0JEVlpaKnDga/3wGweAAB4ufti7YqN8NB4yZzMdpJn3gWl0gUAYGyuQUX9JZkT\nERERTb5RC/z+/n7k5uYiLS3N6vjq1auRlZU14jnZ2dlYvnw51Gq1dCw9PR11dXWoqqqSju3cuRMN\nDQ14+eWXx5ufiEbQP9CPv5/ah/4BEwDAQ+ONtSs2wsvdR+ZktuXu5ok50ddvFpwqyOBdfCIimnJG\nLfCbmppgNpuh0+msjut0OhgMhhHPMRgMI/YXRVE65+LFi9i2bRvef/99h9pYh8gRnLjwOa62GQEM\nbmL12Iqn4OsZIHOqybFw5gooFEoAQF1TJaobLsuciIiIaHK5yPGmJpMJ3//+9/H73/8eERERADDm\nu2w5OTm2jDYlcUxtR46xbeyowZnL/5DaieHzUFlag0qM/MyMIxptXH3VIai+Ojg959Pjf8bi6DWT\nEcvh8XuBbXBcbYPjahsc14kVExMjy/uOegc/ICAASqUSRqPR6rjRaIRerx/xHL1eP2J/QRCg1+tR\nX1+PoqIi/OhHP4JKpYJKpcK2bduQn58PtVqNI0eO3MYlEU1dpoFenL/ytdQO8gpHhF+cjInkMSNo\nNnDtN4NXO+rQ0mUc5QwiIiLnMeodfJVKhQULFiAjIwOPPvqodDwjIwOPPfbYiOcsWbIEP//5z2Ey\nmaR5+IcPH0ZISAgiIyMxMDCA/Px8q3PeeecdHDlyBJ988gkiIyNvmCc5OXlMF0aj+/andI7pxJNj\nbEVRxKcn90KpFqBVa6F188ATa551io2svnUr49qhMKCwMhcA0I56pCV/z6bZHBm/F9gGx9U2OK62\nwXG1jba2Nlned0yr6Dz//PN47733sGfPHhQXF2PTpk2or6/Hxo0bAQBbt261WuZy3bp10Gq1WL9+\nPQoKCnDgwAG8+uqreOGFFwAALi4umDVrltVHUFAQXF1dER8fD61Wa4NLJXJuBRU5KKm+KLXvuWOt\nUxX3t2rxrJUQMHgXv7yuCE1tIz8zRERE5GzGNAd/7dq1aG5uxvbt21FfX4/ExEQcPHgQYWFhAAYf\nqq2oqJD6e3l5ISMjA8888wwWLlwIX19fbNmyBZs3b7bNVRBNca2dV3E092OpPSd6MaJDE2RMJD9/\nbx1mhCWitGbwh55zpVlIS35E5lRERES2N+aHbDdu3Cjdsf+ud999d9ixhIQEHD9+fMxBfvWrX+FX\nv/rVmPsT0SCLaMEXp/4XpoE+AIObWaXOf0DmVPZhXkyKVOAXVuTgzjn3wlXlJnMqIiIi2xrTFB0i\nsl9nCo+htqkSACAICnxvyQ+gdnGVN5SdiNDNgL/X4JK9poE+FFRwdQgiInJ+LPCJHFhXTztOFWRI\n7ZTE1Qj2D5cxkX0RBAFzY5ZK7bzSLG58RURETo8FPpEDO110DAPmfgBAkE8IFs26W+ZE9idherL0\nG42r7UZufEVERE6PBT6Rg+robkVeWZbUXjb7HmkHV7rOVeWGWdMWSO1zpZkypiEiIrI9FvhEDup0\n4ZcwmwcAAMH+EYgKmSVzIvs1LzZFel1aU4CO7lYZ0xAREdkWC3wiB9Te1YILl09L7ZSkdAjXdm6l\n4QK89QgPigYAiKIFeWWnZE5ERERkOyzwiRxQdsERmC2Dd+9DAqZhmj5O5kT2b17M9bv4Fy5nS7/9\nICIicjYs8IkcTFtnMy6Wn5Hay3j3fkxmhCXCQ+MNAOju7URJzcVRziAiInJMLPCJHExWQQYsogUA\nEB4UhQhdjMyJHINSocScGYul9rkSPmxLRETOiQU+kQNp6Wi02qwpJeke3r2/BbOjF0MhDH7bq22q\nRENLrcyJiIiIJh4LfCIHkpWfAfHa3ftIXYz04CiNjYfGC7Hhs6X2udKsm/QmIiJyTCzwiRzE1XYj\niqrOSe2UpHtkTOO4hi6ZWVR5Fr2mHhnTEBERTTwW+EQO4tSQu/fTg2ciNHCavIEcVGjAdAT6hAAA\n+s0m5A95YJmIiMgZsMAncgBNbQYUV+VJ7ZSkdBnTODZBEDAvZqnUPl92CqIoypiIiIhoYrHAJ3IA\n58tOQcRgERodMgvB/hEyJ3Js8ZHzoXZxBQA0dzSiuuGyzImIiIgmDgt8Ijs3YO5HYeVZqb0gbrmM\naZyDWuWKWdOTpfZ57mxLREROhAU+kZ0rqylAr6kbAODl7otw3QyZEzmHOdHX18QvqbmIrt4OGdMQ\nERFNHBb4RHYuv+L6Q6CJ0xdK67jT7QnyDUFIwDQAgMViRn75N/IGIiIimiCsFIjsWHtXKyrrSwAA\nAgQkRi2UOZFzmTtjifT6/OVsaYdgIiIiR8YCn8iOFVR8Iz1cG6GbAW93P5kTOZfY8DlwU2sBAG2d\nV1FlKJU5ERER0e1jgU9kpyyiBRcrrk8bSYpeJGMa56RyUSGBD9sSEZGTYYFPZKdqGsrR1nkVAOCq\n1mBGaKLMiZzTnBnXH7Ytqy1AR3erjGmIiIhuHwt8Ijt1ccgOq/GR86FyUcmYxnn5e+kQHhQFABBF\ni9W4ExEROSIW+ER2qM/Ug5LqC1I7iQ/X2tScGdd3tr1Qlg2LxSxjGiIiotvDAp/IDhVfycOAuR8A\nEOgTAp1vmMyJnFtMWCK0rh4AgI6eNpTXF8uciIiIaPxY4BPZoaHTRJKiFkIQBBnTOD8XpcpqCVI+\nbEtERI6MBT6RnWlqM6D+6hUAgEKhRPy0+TInmhpmD9nZtqKuGG1dzTKmISIiGj8W+ER2Zujd+xmh\nCdLUEbItX88ATNPHAQBEiLhQli1zIiIiovFhgU9kR8wWMworcqV2UhTXvp9MQ5fMvFB+GmbzgIxp\niIiIxocFPpEdKa8rRHdfJwDAU+ONacGxMieaWqJDE+Cu8QIAdPd2oqTmosyJiIiIbh0LfCI7crH8\n+s61CdOToRD4V3QyKRVKzJ2xRGqfLcmUMQ0REdH4sHogshOm/j5U1l+S2gnTufa9HOZEL4ZCoQQA\n1DVVwthcI3MiIiKiW8MCn8hOVNQXw2wZnPMd6BMCP69AmRNNTe4aL8SFz5HaZ0tPypiGiIjo1rHA\nJ7ITZbUF0usZoQkyJqH5sSnS66Kqs9JzEURERI6ABT6RHTBbzCivK5LaM8JY4Msp2D8SOr/B3YPN\n5gFcvHxmlDOIiIjsBwt8IjtQ21iBXlM3gMHVc3S+YTInmtoEQcD8mGVSO680CxaLWcZEREREY8cC\nn8gOlNXmS6+jwxIhCIKMaQgAZkbOhcbVHQDQ3t2CstpCmRMRERGNzZgL/J07dyIqKgoajQbJycnI\nzLz58nH5+flITU2FVqtFeHg4tm3bZvX5r7/+GikpKQgICIBWq0V8fDxef/318V0FkQMTRRFlNZx/\nb29clCrMjr6+8dU5PmxLREQOwmUsnfbv34/Nmzdj9+7dSElJwTvvvIM1a9agqKgIYWHDpxJ0dHQg\nLS0NqampyM3NRVFREdavXw8PDw8899xzAAAPDw9s2rQJSUlJ0Gq1OHnyJJ566im4u7tj48aNE3uV\nRHassbUebV3NAAC1yg0RQdEyJ6JvzZ2xFGeKjkEULbhiLEVTmwEB3nq5YxEREd3UmO7g79ixAxs2\nbMCGDRsQFxeHt99+G8HBwdi1a9eI/fft24eenh7s3bsX8fHxeOSRR/Diiy/ijTfekPrMnz8fa9eu\nRXx8PCIjI7Fu3Tqkp6fjxIkTE3NlRA5i6Oo5UcEzoVSO6edumgRe7j6IGfLA87kS3sUnIiL7N2qB\n39/fj9zcXKSlpVkdX716NbKyskY8Jzs7G8uXL4darZaOpaeno66uDlVVVSOec+7cOZw6dQqpqam3\nEJ/I8Q2dfz8jLFHGJDSS+bHLpdcFlTnoM/XImIaIiGh0o94qbGpqgtlshk6nszqu0+lw9OjREc8x\nGAwIDw8f1l8URRgMBkRGRkrHw8PD0djYCLPZjF/96lf4l3/5l5vmycnJGS0y3SKOqe2MNrY9pi5U\nVJcAGFy5pbW+GzmN/P8xmsn8MyuKIhRmV3T2tgDoxt+O7kdUoHP+IMbvBbbBcbUNjqttcFwnVkxM\njCzvK/sqOpmZmcjNzcXu3buxY8cOvP/++3JHIpo0xvbrv9Hy8wiGysVVxjQ0EkEQMC1wltSuaiqC\nKIoyJiIiIrq5Ue/gBwQEQKlUwmg0Wh03Go3Q60d+2Eyv14/YXxCEYed8ezc/ISEBBoMBr7zyCn7w\ngx/cME9ycvJokWmMvv0pnWM68cY6thXHzkLrrgUALJ23AvNj+f/iZuT6Mzt7IAm1fyu+Nj1nAP6h\nHogKiZ/UDLbE7wW2wXG1DY6rbXBcbaOtrU2W9x31Dr5KpcKCBQuQkZFhdTwjIwMpKSkjnrNkyRKc\nOHECJpNJOnb48GGEhIRYTc/5LrPZjL6+vrFmJ3JofaYeXGkok9pcHtN+qV1ckTT9Dqn9TdFx+cIQ\nERGNYkxTdJ5//nm899572LNnD4qLi7Fp0ybU19dLy1lu3boVq1atkvqvW7cOWq0W69evR0FBAQ4c\nOIBXX30VL7zwgtTnv/7rv/D555+jrKwMZWVl2LNnD15//XU8/vjjE3yJRPapvL5Y2h01yDcUXu6+\nMieim5kftwyCMPgt80pDGeqvXpE5ERER0cjGtB7f2rVr0dzcjO3bt6O+vh6JiYk4ePCgtAa+wWBA\nRUWF1N/LywsZGRl45plnsHDhQvj6+mLLli3YvHmz1MdsNuPFF19EVVUVXFxcEB0djf/4j//A008/\nPcGXSGSfLtdycytH4u3uh/jIeSiszAUAnCn6Eg8uWy9vKCIiohGMecHtjRs33nADqnfffXfYsYSE\nBBw/fvyGX2/Tpk3YtGnTWN+eyKmYLWaU1xVJ7Rguj+kQ7ohfIRX4pdX5aG5vgJ9XkMypiIiIrMm+\nig7RVFTTcBl9/b0AAC93XwT6hMiciMYi0CcY0SGDK+qIEHGm6JjMiYiIiIZjgU8kg9KaIZtbhSZC\nEAQZ09CtuGPWCul1QWUuOrpbZUxDREQ0HAt8okkmiiLKOP/eYYUFRiE0YBoAwGIxI/fSCXkDERER\nfQcLfKJJ1thaJ931dVVrEBYUJXMiulV3zLpben2+7BR6+rplTENERGSNBT7RJKuoL5ZeRwXPhFKh\nlDENjUdUSDz8vXUAANNAH/LKsmROREREdB0LfKJJVmkolV5P08fJmITGSyEosCj++l38s5dOoH+g\nX8ZERERE17HAJ5pE/QMm1DZe3zMiUh8jYxq6HTMj58FT6wMA6O7rRH7FGZkTERERDWKBTzSJqhvK\nYbYMAAD8vXVSgUiOR6lQYuHMu6T2N0XHpZ2JiYiI5MQCn2gSVRlKpNecnuP4kqIXwU2tBQC0dTXj\nUvUFmRMRERGxwCeaVFXGoQV+rIxJaCKoXVwxP3aZ1D5deBQW0SJjIiIiIhb4RJOms6cdja31AACF\nQsnlMZ3E/NhlUCnVAIDG1nqUXOFdfCIikhcLfKJJMnR6TmjANKhdXGVMQxNF4+qO+XHX7+JnXjzE\nufhERCQrFvhEk6RySIEfyek5TmXhzBVQq9wAAC0djSiozJU5ERERTWUs8IkmgSiKfMDWiWlctbhj\nZqrUzso/jAEz18UnIiJ5sMAnmgRNbQZ09XYAANzUWuj8QmVORBNtQdyd0Li6AwDau1pw4fJpmRMR\nEdFUxQKfaBJYTc/RxUAh8K+es1GrXLF41kqpfaogA6aBPhkTERHRVMUqg2gSVHH+/ZQwN2YpPDXe\nAIDu3k6cKzkpcyIiIpqKWOAT2diAuR/VDZelNte/d14uShWWJKZJ7TNFx9Bn6pExERERTUUs8Ils\nrLaxUnrg0tczAN4efjInIltKjLoDPh7+AIBeUze+Kf5K5kRERDTVsMAnsrGhu9dGcvUcp6dUKJGS\nlC61cy99je7eThkTERHRVMMCn8jGKq2Wx4yRMQlNlpmR8+DvrQMAmAb6cLroS5kTERHRVMICn8iG\nevq60NBcCwAQBAUigmbInIgmg0JQYFnSGql9rvQkOrpbZUxERERTCQt8IhuqMpRChAgACPaPgKta\nI3MimiwxYYnQ+4UDAMzmAWReOCRzIiIimipY4BPZUKXhkvSaq+dMLYIg4M4590rt/IpvYGiuljER\nERFNFSzwiWxEFEVUGUqlNgv8qSdSH4vo0ASp/eXZv0EURRkTERHRVMACn8hGukztaO9uAQCoXVyh\n94+QORHJIXXu/VAolACA2sYKlFRfkDkRERE5Oxb4RDbS1FErvQ7XzYDyWpFHU4ufVyDmxyyT2sfz\nPpP2RSAiIrIFFvhENjK0wOf0nKltSWIaNGotAKC9q4WbXxERkU2xwCeyAYtowdXOeqnNAn9qc1Nr\nkDL7+rKZpwuPorOnXcZERETkzFjgE9lAe89VaRqGp8Ybvp6BMiciuc2JXiRtftU/YMKJCwdlTkRE\nRM6KBT6RDTR11EmvI3QxEARBxjRkDxQKJe6e/5DULijnsplERGQbLPCJbKCpc2iBz91radA0fSyi\nQ2YBAESIOHb2Uy6bSUREE44FPtEEGzD3o6XLKLUjdDEypiF7kzrvfiiEwW+9NY3lXDaTiIgmHAt8\noglW11QFi8UMAPD1DISXu4/Micie+HkFYV6s9bKZ/QMmGRMREZGzYYFPNMGuGMuk15yeQyNZkmC9\nbObpwi9lTkRERM6EBT7RBKsylkqvIzk9h0agcdXizrnfk9pnio6hpaNRxkRERORMWOATTSBTfx8M\nV69I7XBdtIxpyJ4lRt2BYP8IAIDZMoCjuZ/wgVsiIpoQYy7wd+7ciaioKGg0GiQnJyMzM/Om/fPz\n85GamgqtVovw8HBs27bN6vMff/wx0tPTERQUBC8vLyxevBifffbZ+K6CyE7UNJbDIloAAJ4aP2hd\nPWRORPZKISiwcsEjEDC4hGpFfTHKagtkTkVERM5gTAX+/v37sXnzZrz88svIy8vD0qVLsWbNGtTU\n1IzYv6OjA2lpaQgODkZubi7eeustvPbaa9ixY4fU56uvvsLKlSvxxRdfIC8vD/feey8efvhhnDx5\ncmKujEgGQ6fnBHiEyJiEHEGwfzhmRy+S2l+e/YQP3BIR0W0bU4G/Y8cObNiwARs2bEBcXBzefvtt\nBAcHY9euXSP237dvH3p6erB3717Ex8fjkUcewYsvvog33nhD6vPmm2/iZz/7GZKTkxEVFYVf/vKX\nWLBgAT755JOJuTIiGVwxXC/w/T2DZUxCjmL5nHv5wC0REU2oUQv8/v5+5ObmIi0tzer46tWrkZWV\nNeI52dmedn03AAAgAElEQVTZWL58OdRqtXQsPT0ddXV1qKqquuF7dXR0wNfXd6zZiexKT18XGlqv\nbXAlCPB3Z4FPo9O4umP5nCEP3BYfQ0tHk4yJiIjI0bmM1qGpqQlmsxk6nc7quE6nw9GjR0c8x2Aw\nIDw8fFh/URRhMBgQGRk57Jx33nkHtbW1ePzxx2+aJycnZ7TIdIs4phOjvrUC3V3dAAAf9yC4KFUc\nWxtxtnEVRSVUohZt3YOF/Z8P/gELp6+GIAiTmsPZxtVecFxtg+NqGxzXiRUTI89qenaxis5HH32E\nF198EX/+85+H/WBA5CiaOuuk1/4evHtPYycIAhLDluLa87ZobK9BQ/uVm59ERER0A6PewQ8ICIBS\nqYTRaLQ6bjQaodfrRzxHr9eP2F8QhGHn/PWvf8WTTz6Jffv24d577x01cHJy8qh9aGy+/SmdYzox\n8gxHoHUfnEv97QO2HNuJ5ex/Zi1unTh/ORsAUNtVjNV3PQCVi8rm7+vs4yoXjqttcFxtg+NqG21t\nbbK876h38FUqFRYsWICMjAyr4xkZGUhJSRnxnCVLluDEiRMwma6vBnH48GGEhIRYTc/58MMP8eST\nT+KPf/wjHn744fFeA5HsOrpbpY2KlEoX+LrrRjmDaLjlc+6F27UHbtu6mnG6cORpkERERDczpik6\nzz//PN577z3s2bMHxcXF2LRpE+rr67Fx40YAwNatW7Fq1Sqp/7p166DVarF+/XoUFBTgwIEDePXV\nV/HCCy9IfT744AP88Ic/xO9+9zssW7YMRqMRRqMRLS0tE3yJRLZ3xVgmvQ4LmA6lQiljGnJUGld3\n3DnHeofb5nbucEtERLdmTAX+2rVr8eabb2L79u2YN28esrKycPDgQYSFhQEYfKi2oqJC6u/l5YWM\njAzU1dVh4cKFePbZZ7FlyxZs3rxZ6vOHP/wBZrMZmzdvRkhIiPTx6KOPTvAlEtne0PXvw3UzZExC\nji4p2nqH2y/PfswdbomI6JaMOgf/Wxs3bpTu2H/Xu+++O+xYQkICjh8/fsOvd+zYsbG+NZFdE0XR\nav37SF0M6ip515XGRyEosCr5Eez7x1sQIaKi/hJKay4iNny23NGIiMhB2MUqOkSOrKWjCR09gw/R\nqFVu0PuFyZyIHJ3eLxxzYpZK7S/P/g2mgT4ZExERkSNhgU90m64MnZ4TFA0F59/TBFiWdA+0rh4A\nBh/iPpWfMcoZREREg1jgE92moQ/YRgRFy5iEnInGVYu75t4ntXOKv8LVNuNNziAiIhrEAp/oNlhE\ni9Ud/Eh9rIxpyNkkTE9GaOB0AIN/1o7kHOADt0RENCoW+ES3oam1Hj2mbgCA1tUD/t5c/54mjiAI\nWLXgEQjC4LfqKw1lKL6SJ3MqIiKydyzwiW5DpaFEeh2hmwGFwL9SNLGCfEMwP3aZ1D527lP09ffK\nmIiIiOwdqxGi21BZf0l6PS04TsYk5MxSktLhrvECAHT1tOPkxX/InIiIiOwZC3yiceofMKGm8foG\nb9M4/55sxFXlhhXzHpDaZ0sy0dhaL2MiIiKyZyzwicapuqEcZssAAMDfWwdPrY/MiciZzYyYi4ig\nwV2SRdGCo7nc4ZaIiEbGAp9onCoNQ6bn6Dk9h2xLEASsTH5YeuC2uuEyH7glIqIRscAnGqehBf50\nzr+nSRDgrbd64Pb4uc9g6ucOt0REZI0FPtE4tHe1SpsOKZUuCAuMkjkRTRUpSenQug3ucNvZ04ZT\nBdzhloiIrLHAJxqHKuP15THDAqZD5aKWMQ1NJa4qN6TOvV9q51z6GlfbucMtERFdxwKfaBy4PCbJ\nada0Bdd3uLWYcTT3Ez5wS0REEhb4RLfIIlqsNrjiA7Y02QZ3uL3+wG2VoQSlNRdlTkVERPaCBT7R\nLTI216LX1A0A0Lp5INAnWOZENBUF+YZi7owlUvvY2U/RP2CSMREREdkLFvhEt+i7y2MKgiBjGprK\nUpLugcbVHQDQ3t2C7MKjMiciIiJ7wAKf6BYNnX/P5TFJThpXLe6c8z2p/U3RcbR0NMqYiIiI7AEL\nfKJb0Nffi7qmSqkdqY+VLwwRgMSohQj2jwAAmC0DOHbuM5kTERGR3FjgE92C6obLsIgWAECQTwjc\n3TxlTkRTnUJQYOWCRyBgcKrY5doClNcVy5yKiIjkxAKf6BZUcHlMskPB/uFIjFootb88+wnM5gEZ\nExERkZxY4BPdgqrvPGBLZC+Wz7kXrio3AEBLRyNyS07InIiIiOTCAp9ojFo7r6KlowkAoFKqERo4\nTd5AREO4u3kiJSldap/Kz0BHd5uMiYiISC4s8InGaOjqOeG6aLgoVTKmIRpubkwK/L11AADTQB++\nPv+5zImIiEgOLPCJxsh691qunkP2R6lQYuX8h6V2YWUuahsrZExERERyYIFPNAYWixlVxlKpzfn3\nZK8i9TGIDZ8ttY/kHJBWfiIioqmBBT7RGNRfvQJTfy8AwFPjDT+vIJkTEd1Y6rz7pSlkDa11uHj5\ntMyJiIhoMrHAJxoDq+k5wXEQBEHGNEQ35+3uh0Wz7pbaJ85/gZ6+bhkTERHRZGKBTzQGZbUF0mtO\nzyFHsHDmCni7+wEAekzdyLx4SOZEREQ0WVjgE42irbMZDS21AAClwgXTQ2bKnIhodCoXFVLnPSC1\nz5edgvHan2MiInJuLPCJRlFSc1F6HamPkTYTIrJ3MWGJ0m+cRNHCB26JiKYIFvhEoyitviC9jglL\nkjEJ0a0RBAErFzwMhUIJAKhrqkRBRY7MqYiIyNZY4BPdRGdPO+qaqgAAgqDAjNAEmRMR3Ro/r0As\nnJkqtb/O+5wP3BIROTkW+EQ3UVaTDxEiACAscDq0bh4yJyK6dYsTVsJT6wMA6O7rxEk+cEtE5NRY\n4BPdxND590M3DyJyJGoXV6wY8sBtXtkp6cFxIiJyPizwiW6gp68b1cYyqT0jNFHGNES3JzZ8NiJ1\nMQCuP3AriqLMqYiIyBbGXODv3LkTUVFR0Gg0SE5ORmZm5k375+fnIzU1FVqtFuHh4di2bZvV5w0G\nA37wgx8gPj4eLi4u2LBhw/iugMhGLtcWSCuOBPtHwMvdR+ZEROMnCAJWJl9/4La2qRKFlbkypyIi\nIlsYU4G/f/9+bN68GS+//DLy8vKwdOlSrFmzBjU1NSP27+joQFpaGoKDg5Gbm4u33noLr732Gnbs\n2CH16evrQ2BgILZu3YrFixdPzNUQTSBOzyFn4++lQ3LcnVL7eN5n6DX1yJiIiIhsYUwF/o4dO7Bh\nwwZs2LABcXFxePvttxEcHIxdu3aN2H/fvn3o6enB3r17ER8fj0ceeQQvvvgi3njjDalPZGQk3nzz\nTTzxxBPw9fWdmKshmiCm/j5UGi5JbU7PIWexJCENnhpvAEB3bydOXvyHzImIiGiijVrg9/f3Izc3\nF2lpaVbHV69ejaysrBHPyc7OxvLly6FWq6Vj6enpqKurQ1VV1W1GJrK9ivpimM0DAIBAn2D4eQXK\nnIhoYqhVrlgx/0Gpfa70JNq6m2RMREREE23UAr+pqQlmsxk6nc7quE6ng8FgGPEcg8EwYn9RFG94\nDpE9KeHmVuTEvvvA7fnqE9zhlojIibjIHeBW5eRwF8aJxjG1ZraYcb7kjHQHv69VGPcYcWxtg+N6\n+3SuMbjUkw+LxQygGxWNF6HI4cJqtsA/r7bBcbUNjuvEiomJkeV9R/1uHhAQAKVSCaPRaHXcaDRC\nr9ePeI5erx+xvyAINzyHyF40ddZKxb3W1ROebnxGhJyPu6s3YvXzpXaJ4Rw6+9pkTERERBNl1Dv4\nKpUKCxYsQEZGBh599FHpeEZGBh577LERz1myZAl+/vOfw2QySfPwDx8+jJCQEERGRt5W4OTk5Ns6\nn6779qd0jqm1g6cvQ+uuBQDcEX8nFs5deMtfg2NrGxzXiTXfMg/7DrehoqYEFosZht5irF36/0Eh\n8E7+ROCfV9vguNoGx9U22trkuXEypu/izz//PN577z3s2bMHxcXF2LRpE+rr67Fx40YAwNatW7Fq\n1Sqp/7p166DVarF+/XoUFBTgwIEDePXVV/HCCy9Yfd3z588jLy8P7e3taG5uxvnz51FUVDSBl0d0\naywWMy7X5EvtmDCunkPOS6FQ4p5FawFBAABUN5Tj4uXTMqciIqLbNaY5+GvXrkVzczO2b9+O+vp6\nJCYm4uDBgwgLCwMw+FBtRUWF1N/LywsZGRl45plnsHDhQvj6+mLLli3YvHmz1dedN28ehGv/sADA\nZ599hsjISJSXl0/EtRHdsprGCvSYugEAHhpv6P0jZE5EZFtBvqGIDkrCZePgg+XH8/6OqJB4eGq5\nsRsRkaMa80O2GzdulO7Yf9e777477FhCQgKOHz9+069psXDVBrIv1qvnJHKqAk0JMbp5MLRWQsQA\nTP29OJJzAA8t/5HVDRgiInIcrF6IrrGIFpRZTc/h8pg0NSgVLkgKXya1y2oLrH7YJSIix8ICn+ia\namMZOnoGH4bRqLUID4qSORHR5PH3CMac6MVS+0juAfT0dcuYiIiIxosFPtE150qv78w8M3IeFAql\njGmIJt9dc++Dh8YbANDd24kjOR9BFEWZUxER0a1igU8EoKO7FWW1BVJ7XkyKjGmI5OGq1mBV8iNS\nu/hKHi5czpYxERERjQcLfCIAFy6fhigOPvQdETQD/t46mRMRySMmLNFqqs7Rs5+goaVWxkRERHSr\nWODTlGe2mHG+7JTUnhuzVMY0RPJbMf8hBPmEAADM5gF8evJP6OvvlTkVERGNFQt8mvLKavLR1dsB\nAHDXeGEGN7eiKU7losL9KU9A7eIKAGjpaMThM3/hfHwiIgfBAp+mvHOlJ6XXc6IXQ8mHa4ng5xWI\n1Xc8JrU5H5+IyHGwwKcpranNgOqGywAAQVBgdvQimRMR2Y/4yHmcj09E5IBY4NOUljdkacyYsAR4\nan1kTENkf1bMfwiB35mPb+rvkzkVERHdDAt8mrJM/X0oqMyV2nNn8OFaou9SuajwQMrjVvPxD535\nEJZrq04REZH9YYFPU1ZR1VmYrq0M4ucZiAhdjMyJiOyTn1eQ1Xz8S1fycDTnYz50S0Rkp1jg05Qk\niqLVzrVzY5ZCEAQZExHZt/jIeZg7ZAO4vLIsHD/3GYt8IiI7xAKfpqS6pio0ttYBAFyUKiRMXyhz\nIiL7t3LBQ4iPnC+1cy59hcwLB2VMREREI2GBT1NSXtn1pTFnTZsPN7VGxjREjkEhKHDv4u8jNjxJ\nOpZdeBSnCo7ImIqIiL6LBT5NOV29Hbh05YLU5sO1RGOnUChx35IfIiokXjqWeeEgcoq/kjEVEREN\nxQKfppy80iyYLQMAgBD/SOj8wmRORORYlEoXPLjsSUQOeTD92LlPrZ5rISIi+bDApymlvasV3xQd\nl9pzY3j3nmg8XJQqPHTnjxAWGCUdO5LzEU5cOAizxSxjMiIiYoFPU8pXeZ+h32wCAAT6hCA+cp7M\niYgcl9rFFY/e9WME+0dIx7ILjuCDI++gtfOqjMmIiKY2Fvg0ZVwxlqH4Sp7UXrngISgUShkTETk+\ntcoV/yf1Kat9JOquVmHvwddROGQjOSIimjws8GlKsFjM+PLsJ1I7PnIewoOiZUxE5Dzc1Bo8tuIp\n3Dnne1AIg/+smAb68Pmp/8Xnp/4Xfdc2lCMiosnBAp+mhLyyU2hsrQcAqFzUuGvufTInInIuCkGB\nRbPuxrq0n8LXM0A6XliZi70HX0dFfTE3xSIimiQs8Mnpdfd2IvPiIam9JGEVPLU+MiYicl7B/uF4\nIv15JEbdIR1r62rGX4//D/Yeeh0Xy89gwNwvY0IiIufHAp+c3okLX6DP1AMA8PUMxIK4O2VOROTc\n1CpXrFn0f3Hf0h/CVeUmHW9srceh0/vxh09/g6z8DHT3dsqYkojIebnIHYDIluqvVuPi5TNS++75\nD8JFqZIxEdHUER85D6EB03Gm6Evkl38jrWDV3duJkxcPIbvwCGZGzMX04JmI0M2Au5unzImJiJwD\nC3xyWhbRgi9zP4aIwXm/USHxVrtvEpHtebn7YFXyI0hJugcXLmfjXEkmOnraAABm8wAKKnJQUJED\nAPD31iFSF4sI3QyEB0XDTa2RMzoRkcNigU9Oq7AiF3VXqwAASoUL7p7/oMyJiKYujasWi2bdjeSZ\nd+HSlfPIufQVjM01Vn2uthlxtc2IsyUnIECAt4cf/L31CPTWw99bjwBvHfy8gvhbOCKiUbDAJ6d0\ntd2I4+c+ldrJM++Cr2egjImICACUCiVmTZuP+Mh5qL96BRX1xbhiLEPd1SpYhuyAK0JEa+dVtHZe\nxeXaAum4ICjg4+EHH48A+Hj4w9czAD4eAfD1DIC3ux+USvv4Z8000IfOvjb0mrpQUJGD7r4u9A+Y\nMGA2oX/g+ofZMgCViytcVW5wU2vhqtbA7dqHu5sXArz1UKtc5b4cInIw9vGdkGgCtXe14C/H/hs9\npm4AgKfGG4tnrZQ5FRENJQgCQgIiERIQiZSkdJgG+lDbWIkrxlJcMZbB2FILUbQMO08ULWjpaEJL\nR9PwrwkBHhoveHn4wdv92oeHH7y0vvDUekPj6gE3tQaCINxWdrN5AB09bWjvakFHdyvau1vR0dWC\n9u5WdPa0oaO7Db2mbnR3DX4PumjQ3tb7ebv7IcBbjwCfYAR46xHoEwx/b5205wAR0XexwCen0t3b\nib8c+wM6ulsBDK55/8CyJ3kHjMjOqV1cMT04DtOD4wAAA+Z+NLc3oKnNgKY247X/GtDe2Sw9V/Nd\nIkR09LSho6cNtY0VI/ZRCApo3TyhdXWH1s0TbmoNFAolFIICCoUCgqCQXpv6+9DX34NeUy/6TD3o\nNXWjr3/w9Y0y2EJbVzPauppxua5QOuaqckNYUDQigqIRoZuBAJ9gFvxEJGGBT06jr78Xfz3+P2ju\naAQAKBRKPLT8RwgJiJQ5GRHdKhelCkG+oQjyDbU63j9gQmvnVbR0NKG1swmtHU1o6byK1s4mdHS1\njlp4W0QLOnva0HntQV9bUSiU0Kg94KZyx/SIaGjdPKF2UcPFRQ2VUgWVixoqFzWUChf0D5ikHx56\nTT2DP0z096C1swkt7Y2wjPCbjL7+XlyuLZCmL2nUWoTrohGhi0V0SDy83H1ten1EZN9Y4JNT6B/o\nx8df74GxZfChPUFQ4L4lP8A0fazMyYhoIqlc1Aj0CUagT/Cwz5nNA2jvbkX7tTvebZ3NaO9uQVtn\nM7p6O9Dd2wnTQN9tZxAEBTzcPOHp7gsvrQ88tT7wch+cBuR5ra1xdcfZ3LMAgOTk5HG/l9k8gOaO\nxsHfYLTWo7HNAENzNbp62q369Zi6UVJ9ESXVF3EEQIC3HtGhsxAVEo8Q/0goFMrbuWQicjAs8Mnh\nmS1mfJb1J1Q3lEvHVi98FHERc2RMRUSTTal0ga/n4AO3NzJg7kd3bxe6+zrQ3duFXlM3RNECi2iB\nxTL4X1G0wGIxQ+Wilh58dVVp4KZ2G3wIVqWZtIJZqXS5/gNN5DwAgCiKaOloQnVDGaqMZag2lqG7\nz3rTsG+nNJ0u/BJuai2mB89EdOgsTNPHQeN6e88EEJH9Y4FPDs0iWnDo9H6rVTbumnsfZkcvljEV\nEdkrF6UKXu4+8HL3kTvKuAmCAD+vQPh5BWLOjCUQRRFX242oMpQOrkrUUAazeUDq32vqRlHVWRRV\nnYUgKBAWOA1RIYN39/29dLf90DER2R8W+OSwDM3VOPzNX63W0r4jfgXuiF8hYyoiosklCMLgKjve\neiyIWw7TQB+uGMtQXluIy3VFVs8biKIF1Q3lqG4ox1d5f4e3ux+iQuIxLTgOEUEzuCABkZNggU8O\np6+/F5kXDuFc6UmrZfRmRy/GnXO+J2MyIiL5qV1cMSM0ATNCEyCKIhpb63C5thCX6wphuFpt9SBy\nW1czzpWexLnSk1AICoQGTkOkPg7T9HHQ+YVyZR4iB8UCnxyGKIooqb6AL8/+zeqOlFLpgiUJaVg0\n627+qpmIaAhBEKTViJYkpqGrtwPldUUorytEZX2J1UPHliF39zMvHISbWovQwOkIC5yO0IDp0PmF\nchdhIgcx5gJ/586d+P3vf4/6+nokJCTgzTffxLJly27YPz8/Hz/5yU9w5swZ+Pv746mnnsIvfvEL\nqz5fffUVXnjhBRQUFCA0NBRbtmzB008/Pf6rIackiiIMzTXIyv8HyuuKrD4XqY9FWvIj3KWWiGgM\n3N08kRR1B5Ki7oDZPICaxnJUGkpQWX8JDa11Vn17Td1WS3EqlS4I8Y9ASMB0hAREIsgnBJ5aH95Y\nIbJDYyrw9+/fj82bN2P37t1ISUnBO++8gzVr1qCoqAhhYWHD+nd0dCAtLQ2pqanIzc1FUVER1q9f\nDw8PDzz33HMAgMrKSnzve9/Dj3/8Y7z//vs4ceIE/vVf/xVBQUF4+OGHJ/YqyeFYRAvqm6pwqfoC\nSmsuor2rxerzWjcP3D3/IcyMmMt/XIiIxkGpdEGkPhaR+ljcNfc+dPV2oMpQgsr6ElQaS4YtxWk2\nD0h3+L/lptZe+w1BCIJ8QhDkGwpfzwDe6SeS2ZgK/B07dmDDhg3YsGEDAODtt9/GoUOHsGvXLmzf\nvn1Y/3379qGnpwd79+6FWq1GfHw8ioqK8MYbb0gF/q5duxAaGoo333wTABAXF4fTp0/j97//PQv8\nKchiMaOtqxlX2xtQUX8JpTUXh/3jAgxuRT97xuBceze1RoakRETOyd3NE7OmLcCsaQuuLcXZiJrG\nCtQ2VqC2qQItHU3Dzuk1deOKsRRXjKXSMQECPLTe8PUMhK+HP3y9AuHrGQgvrQ88NN5wc9Vybj+R\njY1a4Pf39yM3NxdbtmyxOr569WpkZWWNeE52djaWL18OtVotHUtPT8cvf/lLVFVVITIyEtnZ2Vi9\nerXVeenp6fjjH/8Is9kMpZKbcjgDURTRbzZd2+Z98KOvvwc9fV1obm9AS0cjmtsb0dLZBIvFfMOv\n46pyQ3RoAubFpHBnWiIiGxtcijMIfl5BmB29CADQ1dOO2qZK1DRWoKGlBg0tdejr7x12rggRHd2t\n6OhutSr8v6UQFHDXeMFD4wUPjTe0bh5wU2vhJu03MPjhqtZA7eIKF6UaKhcVXJQqKJV8dJBoLEb9\nm9LU1ASz2QydTmd1XKfT4ejRoyOeYzAYEB4ePqy/KIowGAyIjIyEwWBAWlrasD4DAwNoamoa9n7f\nOvDVntEi0zWjbdluNBoBAFVdedfPEYeeI0IUh3wdUYQoihBhgSgOTqOBKF7bIMYMs2iG2TwAs8UM\ns2Xwv/0DppsW7jejdfXAjLBExIQlIVI3g9/YiYhk5K7xQmz4bMSGzwYw+O9Fe1cLjC21aLj20dRm\nQHt3q9UKZ99lES3SDwDj0dvTB6VCidPVn0EpKKFQfPuhgFJQQlAoIECAIAgQBAUEQYBCECBAAQgC\npEmdgnCtHwCMPNVzPFNAhRt8LXv3bU1wpeu8zEnGZkliGoL9I+SOYbccrmJaOff/yB2BZNDZ2SV3\nhFsSExMDAGhraxulJ90KjqttcFxtY2qMqxI6rwjovCIA/nKVJplz/926PaNOggsICIBSqZR+svuW\n0WiEXq8f8Ry9Xj9if0EQpHNu1MfFxQUBATfeZpyIiIiIiG5s1AJfpVJhwYIFyMjIsDqekZGBlJSU\nEc9ZsmQJTpw4AZPJJB07fPgwQkJCEBkZKfX57tc8fPgwkpOTOf+eiIiIiGicBNF60vWIPvzwQzzx\nxBN45513kJKSgl27duHdd99FYWEhwsLCsHXrVnzzzTc4cuQIAKC9vR0zZ85EamoqXnrpJVy6dAk/\n+tGP8Otf/xqbN28GMLhMZlJSEn784x/j6aefRmZmJn7yk5/ggw8+wEMPPWTbqyYiIiIiclJjmoO/\ndu1aNDc3Y/v27aivr0diYiIOHjworYFvMBhQUVEh9ffy8kJGRgaeeeYZLFy4EL6+vtiyZYtU3APA\ntGnT8MUXX+C5557D7t27ERISgv/8z/9kcU9EREREdBvGdAefiIiIiIgcg0PsNLFz505ERUVBo9Eg\nOTkZmZmZckeSzYkTJ/Dggw8iLCwMCoUCf/zjH4f1eeWVVxAaGgqtVosVK1agsLDQ6vMmkwnPPvss\nAgMD4eHhgQcffBC1tbVWfVpbW/H444/Dx8cHPj4+eOKJJ4Y9rV5dXY37778fHh4eCAwMxKZNmzAw\nMDDxF21jv/3tb3HHHXfA29sbQUFBeOCBB1BQUDCsH8f11u3cuRNz5syBt7c3vL29sXTpUnzxxRdW\nfTiut+e3v/0tFAoFfvrTn1od57jeul//+tdQKBRWHyEhIVZ9OK7jYzAYsH79egQFBUGj0SAxMREn\nTpyw6sOxvTXTp08f9udVoVDg/vvvl/pwTG+dxWLBL37xC6nujIqKwi9+8QtYLNZLv9r92Ip27oMP\nPhBVKpW4Z88esbi4WHz22WdFDw8Psbq6Wu5osvjiiy/El156Sfzoo49Ed3d3ce/evVaf/93vfid6\neXmJH3/8sVhQUCCuXbtWDAkJETs7O6U+GzduFENDQ8WjR4+K586dE1NTU8W5c+eKFotF6nPPPfeI\niYmJ4unTp8Xs7GwxISFBfOCBB6TPm81mMTExUVyxYoWYl5cnHjlyRAwJCRF/+tOf2n4QJtg999wj\n7t27VywoKBDz8/PFhx9+WNTr9WJLS4vUh+M6Pp9++ql46NAh8fLly2Jpaan40ksviSqVSrx48aIo\nihzX23Xq1Clx+vTp4ty5c8Vnn31WOs5xHZ9XXnlFjI+PFxsaGkSj0SgajUaxqalJ+jzHdXxaW1vF\nqKgocf369WJOTo5YWVkpfvnll2JxcbHUh2N765qamqQ/p0ajUTx37pyoUCjEP/3pT6IockzHa/v2\n7aK/v7/4+eefi1VVVeJnn30m+vn5ib/5zW+kPo4wtnZf4C9atEh8+umnrY7FxMSI//Zv/yZTIvvh\n4dN+FCkAAAgJSURBVOExrMAPDg4Wf/vb30rtnp4e0dPTU/zv//5vURRFsa2tTVSr1eKf//xnqU91\ndbWoUCjEw4cPi6IoioWFhaIgCOKpU6ekPpmZmaIgCGJJSYkoioM/aCiVSrG2tlbqs2/fPlGj0Ygd\nHR0Tf7GTqLOzU1QqleLf//536RjHdeL4+flJ48ZxHb/W1lYxOjpaPH78uJiammpV4HNcx+eVV14R\nk5KSbvh5juv4bN26VVy2bNlN+3Bsb99vfvMb0dfXV+zt7RVFkWM6Xvfdd5+4fv16q2NPPvmkeP/9\n90ttRxhbu56i09/fj9zc3GE73q5evRpZWVkypbJfFRUVw3YIdnNzw5133imNV05ODgYGBqz6hIWF\nIT4+XuqTnZ0NT09PLF68WOqTkpICd3d3qz7x8fFWv75OT09Hb28vcnNzbXqdttbe3g6LxQJfX18A\nHNeJYrFY8MEHH6CrqwspKSkc19v01FNPYe3atbjrrrusjnNcb095eTlCQ0MRFRWFf/qnf5IWkPj/\n27u3kKjWMAzA74ynUbEJ1GbSBKfIdCzNQ6nZRRIRISFBkRGKXlQkoSlZht2ISHTVTeaF0YEu0gij\noKCmmgbzAEWOmE1amJiEh4QkERXGb19sXOyVsdnN2J4D7wPrwn99rPnndbH89B/XYq6ue/DgAbKy\nslBYWAiDwYC0tDQ0NjYq+5ntyrh+/TqKiooQEhLCTN2wc+dOWK1WDAwMAADev3+PFy9eID8/H4Dv\nnK9e/STbb9++wel0wmAwqMYNBgOeP3/uoVl5r7GxMWg0ml/m9fXrVwB/P0wsICAAkZGRy2rGxsaU\n40RHRy87/po1a1Q1P7/O0kPRlmp8VUVFBdLT05GTkwOAubrr3bt3yMnJwdzcHCIiInD//n2YzWZ0\ndXUxVxc1NzdjaGgId+7cWbaP56vrsrOzcfPmTSQmJmJiYgL19fXIzc1Ff38/c3XD0NAQrl69isrK\nSpw/fx52ux2nTp2CRqNBWVkZs10BT58+xfDwMI4dOwaA1wF3nDt3Dj9+/IDZbEZAQACcTidqa2tx\n4sQJAL6TrVc3+ET/t6qqKnR2dqKjowMajcbT0/ELiYmJ6O3txfT0NO7du4fi4mLYbDZPT8tnDQ4O\nora2Fh0dHdBqvXoR1ufs3btX9XV2djZMJhNu3bqFrKwsD83K9y0uLmL79u1oaGgAAKSmpmJwcBCN\njY0oKyvz8Oz8Q3NzM7Zt24bNmzd7eio+r6WlBbdv30ZLSwvMZjPsdjvKy8thMplQWlrq6en9Z179\n02Hpt5Tx8XHV+Pj4OIxGo4dm5b2MRiNE5F/zMhqNcDqdmJqa+teaycnJZcefmJhQ1fz8OksrLr76\nvamsrERrayusVqvyxGWAuborMDAQ69evR1paGhoaGrB161ZcvnyZubqoq6sLU1NTMJvNCAoKQlBQ\nEGw2GxobGxEcHIzIyEjmukLCwsKQnJyMjx8/8nx1w9q1a5GUlKQaS0pKwsjICABeY901OTmJhw8f\n4vjx48oYM3Xd2bNnUV1djUOHDiE5ORlHjx5FVVUVLl68CMB3svXqBj8oKAgZGRmwWCyqcYvFgtzc\nXA/NynuZTCYYjUZVXnNzc2hvb1fyysjIQGBgoKpmdHQUDodDqcnJycHMzAy6u7uVms7OTszOzmLH\njh1KjcPhUJajgL+XCHU6HTIyMv7o+/wTKioqlOZ+48aNqn3MdWUtLi5ifn6eubrowIED6OvrQ29v\nr7JlZmbiyJEj6O3tRUJCAnNdIXNzc/jw4QNiYmJ4vrohNzdX+TzzkoGBAeUPKczWPTdu3IBOp0Nh\nYaEyxkxdNzs7u2x1VKvVKrfJ9Jls/+t/FXtKa2urhISEyLVr18ThcEh5eblERETIyMiIp6fmETMz\nM2K326Wnp0fCwsKkvr5e7Ha7kselS5dk9erV0tbWJn19fXL48GGJjY1V3brp5MmTEhcXJ8+ePZO3\nb99KXl6epKenq27dtG/fPklJSZGuri7p7OyULVu2SEFBgbLf6XRKSkqK7N69W3p6esRisUhsbKxU\nVFT8f2GskLKyMlm1apVYrVYZGxtTtn9mxlxdU1NTI+3t7TI8PCx9fX1SU1MjAQEB8uTJExFhrivl\n57voMFfXnDlzRmw2m3z+/Fm6u7slPz9f9Ho9r69uev36tQQHB0tDQ4N8+vRJ7t69K3q9XpqampQa\nZuu6hISEZXcbFGGmriopKZG4uDh59OiRDA8PS1tbm0RHR0t1dbVS4wvZen2DLyLS1NQkJpNJdDqd\nZGZmyqtXrzw9JY95+fKlaDQa0Wq1qq20tFSpqaurk5iYGAkNDZVdu3ZJf3+/6hgLCwtSXl4uUVFR\nEh4eLgUFBTI6Oqqq+f79uxQVFYlerxe9Xi/FxcUyPT2tqvny5Yvs379fwsPDJSoqSk6fPi0LCwt/\n7s3/Ib/KU6vVSl1dnaqOuf6+kpISiY+PF51OJwaDQfbs2SMWi0VVw1zdl5eXp2rwRZirKwoLCyU2\nNlZCQkJk3bp1cvDgQXE4HKoa5uqax48fS2pqqoSGhsqmTZvkypUry2qY7e+zWq2i1WrlzZs3v9zP\nTH/fzMyMVFZWSnx8vISFhcmGDRvkwoULMj8/r6rz9mw1IiIurGAQEREREZEX8urP4BMRERER0e9h\ng09ERERE5EfY4BMRERER+RE2+EREREREfoQNPhERERGRH2GDT0RERETkR9jgExERERH5ETb4RERE\nRER+hA0+EREREZEf+QvTWZxfquj5nwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "data = ReadData(path='../code')\n", + "cols = zip(*data)\n", + "price1, price2, bid1, bid2, diff1, diff2 = cols\n", + "\n", + "pdf = thinkbayes.EstimatedPdf(price1)\n", + "low, high = 0, 75000\n", + "n = 101\n", + "xs = numpy.linspace(low, high, n) \n", + "pdf.kde.evaluate([3, 3])\n", + "\n", + "pmf = pdf.MakePmf(xs)\n", + "thinkplot.Pmfs([pmf])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pdf` is a `Pdf` object, estimated by KDE.\n", + "`pmf` is a Pmf object that approximates the Pdf by evaluating\n", + "the density at a sequence of equally spaced values.\n", + "\n", + "`linspace` stands for “linear space.” It takes a range,\n", + "`low` and `high`, and the number of points,\n", + "`n`, and returns a new `numpy` array with\n", + "`n` elements equally spaced between `low` and\n", + "`high`, including both.\n", + "\n", + "And now back to *The Price is Right*." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Modeling the contestants\n", + "\n", + "The PDFs in Figure [fig.price1] estimate the distribution of possible\n", + "prices. If you were a contestant on the show, you could use this\n", + "distribution to quantify your prior belief about the price of each\n", + "showcase (before you see the prizes).\n", + "\n", + "To update these priors, we have to answer these questions:\n", + "\n", + "1. What data should we consider and how should we quantify it?\n", + "\n", + "2. Can we compute a likelihood function; that is, for each hypothetical\n", + " value of `price`, can we compute the conditional\n", + " likelihood of the data?\n", + "\n", + "To answer these questions, I am going to model the contestant as a\n", + "price-guessing instrument with known error characteristics. In other\n", + "words, when the contestant sees the prizes, he or she guesses the price\n", + "of each prize—ideally without taking into consideration the fact that\n", + "the prize is part of a showcase—and adds up the prices. Let’s call this\n", + "total `guess`.\n", + "\n", + "Under this model, the question we have to answer is, “If the actual\n", + "price is `price`, what is the likelihood that the\n", + "contestant’s estimate would be `guess`?”\n", + "\n", + "Or if we define\n", + "\n", + "```python\n", + "error = price - guess\n", + "```\n", + "then we could ask, “What is the likelihood that the contestant’s\n", + "estimate is off by `error`?”\n", + "\n", + "To answer this question, we can use the historical data again.\n", + "Figure [fig.price2] shows the cumulative distribution of\n", + "`diff`, the difference between the contestant’s bid and the\n", + "actual price of the showcase.\n", + "\n", + "The definition of diff is\n", + "\n", + "```python\n", + "diff = price - bid\n", + "```\n", + "\n", + "When `diff` is negative, the bid is too high. As an aside, we\n", + "can use this distribution to compute the probability that the\n", + "contestants overbid: the first contestant overbids 25% of the time; the\n", + "second contestant overbids 29% of the time.\n", + "\n", + "We can also see that the bids are biased; that is, they are more likely\n", + "to be too low than too high. And that makes sense, given the rules of\n", + "the game.\n", + "\n", + "Finally, we can use this distribution to estimate the reliability of the\n", + "contestants’ guesses. This step is a little tricky because we don’t\n", + "actually know the contestant’s guesses; we only know what they bid.\n", + "\n", + "So we’ll have to make some assumptions. Specifically, I assume that the\n", + "distribution of `error` is Gaussian with mean 0 and the same\n", + "variance as `diff`.\n", + "\n", + "The `Player` class implements this model:\n", + "\n", + "```python\n", + "class Player(object):\n", + "\n", + " def __init__(self, prices, bids, diffs):\n", + " self.pdf_price = thinkbayes.EstimatedPdf(prices)\n", + " self.cdf_diff = thinkbayes.MakeCdfFromList(diffs)\n", + "\n", + " mu = 0\n", + " sigma = numpy.std(diffs)\n", + " self.pdf_error = thinkbayes.GaussianPdf(mu, sigma)\n", + "```\n", + "\n", + "`prices` is a sequence of showcase prices, `bids`\n", + "is a sequence of bids, and `diffs` is a sequence of diffs,\n", + "where again `diff = price - bid`.\n", + "\n", + "`pdf_price` is the smoothed PDF of prices, estimated by KDE. `cdf_diff`\n", + "is the cumulative distribution of `diff`, which we saw in\n", + "Figure [fig.price2]. And `pdf_error` is the PDF that characterizes the\n", + "distribution of errors; where `error = price - guess`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvEAAAEWCAYAAADvkchGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X101NWdx/FPZhKYCQ/hITgJBBUsAhKFSqgKKGAFSn2A\nIiJiRcUui3oEZNeuCGu1QNWtlRZX4qIUqZQ1PRLF2qAGKw8BbAGNa4BEBFTQZARCwlMiJLn7R5wf\nM8kkZMJMZiZ5v87JOZnf3F/mziUn+fDN93d/McYYIwAAAABRwxbuCQAAAAAIDCEeAAAAiDKEeAAA\nACDKEOIBAACAKEOIBwAAAKIMIR4AAACIMoR4AAAAIMoEFOI3bdqksWPHKiUlRTabTX/605/OeU5e\nXp6GDx+u+Ph4de/eXfPnz2/0ZAEAAAAEGOJPnDihyy+/XIsXL1Z8fPw5xx8/flwjR45UcnKyduzY\noT/84Q/67W9/q0WLFjV6wgAAAEBLF9PYO7a2a9dOL7zwgqZMmVLnmPT0dM2ZM0fffvutWrVqJUla\nuHChXnzxRR04cKBxMwYAAABauJD2xH/44Ye69tprrQAvSaNHj9Y333yjL7/8MpQvDQAAADRbIQ3x\nRUVFcrlcPsdcLpeMMSoqKgrlSwMAAADNVmy4J+BRWloa7ikAAAAAjZaQkNBkrxXSSnxSUpLcbrfP\nMbfbrZiYGCUlJYXypQEAAIBmK6Qh/pprrtGmTZt0+vRp69h7772nrl276qKLLgrlSwMAAADNVkDt\nNCdPntTnn38uY4yqqqr01Vdf6ZNPPlGnTp3UvXt3zZkzR9u2bdO6deskSZMnT9avf/1r3XPPPZo7\nd64KCgr0zDPP6Mknn6z3dZryTxHN3fbt2yVJaWlpYZ5J88PahgbrGhqsa2ica13XZBUo482dKi+v\nsI4d6eTQoS5OVdlimmSO0aiqqkqSZLM1vNZos8WoYweHOiQ4QjWtqHfq5ClJUnybc28THgrOWLsm\npyZrfB/XuQdHkXC1hAcU4rdv364RI0YoJqb6B8+vfvUr/epXv9Ldd9+tP/7xjyoqKtL+/fut8e3b\nt1d2drYefPBBDRo0SB07dtQjjzyiWbNmBfddAAAQRJn5bq3KK1RZRaXf50tKy3W0pFwV3z9vy15X\n9xfr0b5Br2mrMupTcFQOR6xuH9dPY3/aO+B5Nxf8pzM0zq7rwDDPBMEQUIgfNmyY9b9jf5YvX17r\nWL9+/bR+/fqAJwYAQGOcK4AHyhPYq6oadVsVH3VVi5trhRJA6ETM7jQAAJyvzHy3luUePO+vE+rg\nTmgHcL4I8QCAqHO+1faGhHRblZHrUJk6F5f7fd7hiNWP+sdr6FWJtH0AaHKEeABA1GlIgL9vQIrf\nSvearAKtyNqrQGrgdfWpe3qMAaCpEeIBABGrMRV3f60q/naJORcuMAUQyQjxAICIEWhod8ba9fqE\nAdZjT1hf+dZnWtmA8++e1J+QDiAqEeIBAGHV2P52T8WdKjuAlogQDwBocoEE97p2clmTVaCM5zY3\nOLwT3AE0J4R4AECTaGhwb8j2i2uyCrTitU/8PkdYB9ASEOIBACHVkPDe0H3T62qdIbgDaGkI8QCA\nkKorwDe04n6ufncuTgXQEhHiAQAh5R3g6wvugV6gSvUdQEtGiAcABFV97TPe20FK7N8OAI1FiAcA\nBE1mvlvLcg/6fc4Za/d5XN/Fqd4I7QBQGyEeABAU5wrwDdnTncAOAA1DiAcANFp9rTP3DUiRfV+J\nFdpXvvVZnV+Hi1MBIDC2cE8AABCdPJX3ugL8+D6uc/a7OxyxBHgAaAQq8QCAgNXVOuO9+8yarAK/\nAZ6WGQA4f4R4AECD1dU+4906s/Ktz7SyxnkOR6xWLR3fdBMFgGaOEA8AaJC6qu8/jLHr9ec219s2\nc/u4fqGcGgC0OPTEAwDOyV+Ad8badd+AFO17f1+9u83Q8w4AwUclHgDgV4N2nqlRgaffHQCaBiEe\nAFBLfXu+19U+Q987ADQdQjwAoJZVeYU+j0tKy1VaXKbO7lPaVVxea7ynAg8AaBqEeABoweprmZGq\nw7vz86NyuU/J5ed52mcAIDwI8QDQQtXXMiNVB/ijh0/J5T5V6znCOwCEFyEeAFqg+gJ8SWm5jpaU\nSxVV6nKozDpOcAeAyEGIB4AWxl+Av29Aisb3qW6YmTwtU64aF62yTSQARBZCPAC0IOcK8JLYMhIA\nogAhHgCaucx8t5bmlem7Sim+Te0Ab99Xosl13HGVLSMBIDIR4gGgmVuVV6jvamw+49l1ZmXW3jrP\nczj4FQEAkYqf0ADQzHlvH+m933t7P/u9e7DvOwBENkI8ALQAJ05W6NiJM+qXX8J+7wDQDBDiAaCZ\n8tzISZKOnTgjY3yfJ7gDQPQixANAM1PzLqwlpeUyRoqprE7xhHcAiH6EeABoZrwDvCQdLSlXTKVR\nx8Jjcjhi2XEGAJoBQjwANCOZ+W6fAO+MtatL4Uk5DhRLkm6/75pwTQ0AEES2cE8AABA8nh54qTrA\n3xXvVGevXWhooQGA5iHgEL9kyRL17NlTTqdTaWlpysnJqXf8u+++q8GDB6t9+/bq0qWLxo0bpz17\n9jR6wgAA/7yr8CWl5Sr5x0GteO0T6/nWrajbAEBzEdBP9IyMDM2aNUvz5s1Tbm6uBg8erDFjxujg\nwYN+x3/xxRcaN26chg0bptzcXL3//vsqLy/XjTfeGJTJAwDO8q7ClxaXqb37lM/z1w+9oKmnBAAI\nkYBC/KJFizR16lRNnTpVvXv31uLFi5WcnKz09HS/43fs2KGKigr95je/Uc+ePXXFFVfo0Ucf1d69\ne1VcXByUNwAAqF2F7+wV4B2OWI25PklDr0oM1/QAAEHW4BB/5swZ7dixQyNHjvQ5PmrUKG3ZssXv\nOYMGDVJcXJxefvllVVVV6fjx43rllVf0ox/9SJ06dTq/mQMAJFUH+N9t+Fz7vyzR3v1HdfTwKasP\n3rMbDQEeAJqXBof4w4cPq7KyUi6X773+XC6XioqK/J5z4YUX6r333tPjjz+u1q1bq0OHDtq5c6f+\n+te/nt+sAQCW5zfu1ZHiMlVVVe8D3+VQmfXc7eP6hWtaAIAQijGm5j38/CssLFS3bt20ceNGDR06\n1Do+f/58rVq1Srt37651jtvt1nXXXaef/exnuuOOO3T8+HE9/vjjMsbogw8+8BlbWlpqfc6FrwBw\nbjn/OKy/53yr/NQk61ing6VK+PakWrey6fqhF1CBB4AQ69Wrl/V5QkJCk71ug/eJT0xMlN1ul9vt\n9jnudruVlJTk95wXXnhBbdu21dNPP20de/XVV9W9e3dt2bJFgwcPbuS0AaDl8oT3705XqfSCNj7P\nTUptp6FX9QjTzAAATaXBIT4uLk4DBw5Udna2br31Vut4dna2brvtNr/nnDp1Sna73eeYzVbdwVNV\nVVXna6WlpTV0WjiH7du3S2JNQ4G1DQ3W9dyeW5qp2DiHSl0OlbjiZZNks8XI1dGpWdP8F0dY19Bg\nXUODdQ0N1jU0vLtJmlJAu9PMnj1br7zyipYtW6b8/HzNnDlThYWFmj59uiRpzpw5uuGGG6zxN954\noz766CPNnz9fn3/+uT766CPde++9uvDCCzVw4MDgvhMAaAHWZBWovLxCknSoi1M2W4w6d3Kqx0Ud\n9NB1l4R5dgCAptLgSrwkTZw4UcXFxVq4cKEKCwuVmpqqtWvXKiUlRZJUVFSk/fv3W+NHjBihVatW\n6b/+67/029/+VvHx8br66qv1zjvvyOl0BvedAEAztyarQCte+0RHOjl0qItTirWpx0UdJEn3DUjR\n+D6uc3wFAEBzEVCIl6Tp06dblfeali9fXuvYxIkTNXHixMBnBgCweAd4tytektS5g0OS5Iy1E+AB\noIXhHtwAEAUy3twpqbqFRpI6d3KqQ4JDzli7Jqcmh3NqAIAwCLgSDwBoWp4++COdHKr6vge+Q4KD\nFhoAaMGoxANAhPOuwttsMVYFngAPAC0XIR4AIljNKnzH7/vgaaEBgJaNEA8AESzjzZ3WxaxU4QEA\nHvTEA0CEmpeRq9yL2qnKFiNJVOEBABZCPABEoHkZuXp932Hp+wDvqcJzMSsAQCLEA0BEycx3a1Ve\noXZ9ccTnuKujkwAPALAQ4gEgQmTmu7Us96BKSstVVWWs4xN6JmrB7QPCODMAQKThwlYAiBCr8gpV\nUlquI8VlkiRblVFKcTkBHgBQC5V4AAgzTwtNWUWljpaUS5Jc7lPqXFyuuyf1D/PsAACRiBAPAGHk\naaGRZLXR2KqMFeDH/rR3mGcIAIhEtNMAQJjUDPBHistkqzLqcqhMDkcsAR4AUCcq8QAQBt4BXpKO\nlpRbLTSSdDttNACAehDiASAMVuUVSqquwB8tKVeXwpNWgKeNBgBwLoR4AAiDwiMndbSkugfeuwJP\nGw0AoCHoiQeAMPAEeEk+Af72cf3COS0AQJSgEg8ATWxNVoHPzZw84Z0KPACgoQjxANBE5mXk6s19\nh1Xhdcxmi9GqpePDNicAQHSinQYAmkjNAC9Jro7OsMwFABDdqMQDQBOYl5FbqwLv6ujUQ9ddErY5\nAQCiFyEeAJrAm/sOW5/HSsr7jx+HbzIAgKhHOw0ANAHvKvy4nolhmwcAoHkgxANAiM3LyPV5vOD2\nAWGaCQCguaCdBgBCYE1WgTLe3Kmv42PldsVbx/mhCwAIBn6fAEAIPP/BHn1zUTtV2WJ8jtNKAwAI\nBkI8AITANx1a+wR4my1G4y/uTCsNACAoCPEAEGRrsgp8Anxqr0RNTk3W+D6uMM4KANCcEOIBIMgy\n3twp9WgvqboC//oEqu8AgOBidxoACKI1WQX6Ov5sfaRjB0cYZwMAaK4I8QAQRBlv7tShLk5J1VX4\n5M5twjwjAEBzRIgHgCDxVOE9/fAdOzg0OTU5zLMCADRHhHgACBJ/VXguZgUAhAIhHgCChCo8AKCp\nsDsNAJynzHy3nt+41+fOrFThAQChRIgHgEbKzHdrVV6hyioq5T5aZh232WKowgMAQop2GgBoJE+A\nl6SqKiNJslUZjb+4M1V4AEBIBRzilyxZop49e8rpdCotLU05OTnnPOf3v/+9+vbtK4fDoW7duumx\nxx5r1GQBIFJk5rutAF9SWi5blZHLfUp9Co5qwe3c3AkAEFoBtdNkZGRo1qxZevHFFzVkyBC98MIL\nGjNmjHbv3q2UlBS/58yePVtZWVl69tlnlZqaqtLSUhUWFgZl8gAQDpn5bi3LPSipOsAfPVwd3iXJ\n4aBLEQAQegH9tlm0aJGmTp2qqVOnSpIWL16sd955R+np6Vq4cGGt8QUFBfrv//5v5eXl6dJLL7WO\n9+/f/zynDQDhsyqvuhBRUlquI8Vlch062w9/+7h+4ZoWAKAFaXA7zZkzZ7Rjxw6NHDnS5/ioUaO0\nZcsWv+e89dZbuuSSS5SVlaVLLrlEPXr00D333KNDhw6d36wBIEy822iOlpTL5T6lzsXlkqS7J/XX\n2J/2Duf0AAAtRIwxxjRkYGFhobp166aNGzdq6NCh1vH58+dr1apV2r17d61z7r//fr3yyisaMGCA\nnn32WUnSv/3bvykmJkZbt271GVtaWmp9vmfPnka9GQAItXl5ZfquOsPrm69P6eJPqqvyY65P0tCr\nEsM4MwBAOPTq1cv6PCEhocleN6TNm1VVVTp9+rRWrlypSy65RJL06quvqnfv3tq2bZsGDRoUypcH\ngKDzBPgTJyvUsfCYdZwADwBoSg0O8YmJibLb7XK73T7H3W63kpKS/J6TnJys2NhYK8BL1f9bsdvt\n+uqrr+oM8WlpaQ2dFs5h+/btkljTUGBtQyOS1zUz3634NgdVUlqu4ycrddlJI7WJl8MRG5Hz9RbJ\n6xrNWNfQYF1Dg3UNDe9ukqbU4J74uLg4DRw4UNnZ2T7Hs7OzNWTIEL/nDBkyRBUVFdq/f791bO/e\nvaqsrNRFF13UyCkDQNPKzHdrwuu51o40R0uqt5T04GJWAEBTC2if+NmzZ+uVV17RsmXLlJ+fr5kz\nZ6qwsFDTp0+XJM2ZM0c33HCDNf6GG27QlVdeqalTpyo3N1cff/yx7rvvPl1zzTX8LxBA1PC+qVNJ\nabmqqoy6fL8jDRezAgDCIaCe+IkTJ6q4uFgLFy5UYWGhUlNTtXbtWmuP+KKiIp+qe0xMjN5++23N\nmDFDw4YNk9Pp1KhRo/S73/0uuO8CAELIE+AlqbS4zNqRxuGIJcADAMIi4Atbp0+fblXea1q+fHmt\nYy6XSxkZGYHPDAAiQGb+2euASkrLdenuYusxbTQAgHAJqJ0GAFoa7xs7HT18yjpOFR4AEE6EeACo\nh/eNnbpwZ1YAQIQI6T7xABCtMvPdVhVekqqqDHdmBQBEDCrxAOCH9440kny2lCTAAwDCjRAPADVk\n5rt9Arwz1u7TSgMAQLgR4gHAS2a+27qpk1Qd4O+Kd1qtNAAARAJCPAB48e6Dl6TJqcnKeHOn9djh\n4FIiAED4EeIBwIt3G819A1I0vo9L5eUV1jF2pQEARAJCPAB8z/vGTpI0vo9La7IKfI5xUSsAIBIQ\n4gFA/nvhJdFKAwCISIR4AJD/XnhJtNIAACISIR4A5L8XviZaaQAAkYK/DQNo8erqhfdupQEAIJJQ\niQfQ4nm30nj3wnu30tAPDwCIJIR4AC1azbuz+uuFdzhi6YcHAEQUSksAWrSaVXh/20quWjq+qacF\nAEC9qMQDaLHqqsKzrSQAINIR4gG0WP6q8BLbSgIAIh8hHkCLVFcVnju0AgCiASEeQIvj7+6snio8\nrTQAgGhAiAfQ4tR1d1aJVhoAQHQgxANocRpyd1aJVhoAQOTib8UAWozMfHetKnxdAR4AgEhGiAfQ\nYqzKK/Spwnvuzromq6DWHVoBAIhktNMAaBFq7kbjjLX77AtfM8BzUSsAIJLxWwpAi1BzT/jXJwyw\nHvsL8FzUCgCIZIR4AC2Cvz3hpdr7wmf+aWKTzQkAgMainQZAi+N9MSv7wgMAohEhHkCzl5nvrvM5\n9oUHAEQjQjyAZq9mP3xd2BceABAt+NsxgGbPXz+8Z1tJAACiESEeQLNWs5XGvq9Ek5/bzJaSAICo\nRjsNgGatZitNXXvC0w8PAIgmlJ4ANGs1W2lWvvWZ9dgT3umFBwBEG0I8gBahpLRcrz+32efYqqXj\nwzQbAADOD+00AJot7374oyXlPm009MADAKIZIR5As5SZ79ay3INnD1RUWZ/SAw8AiHYBh/glS5ao\nZ8+ecjqdSktLU05OToPO27Nnj9q1a6f27dsHPEkACETNAF9SWq4uh8qsx6uWjqcPHgAQ1QIK8RkZ\nGZo1a5bmzZun3NxcDR48WGPGjNHBgwfrPe/MmTO64447NHz48POZKwCcU60KvCTn50fVubhcEm00\nAIDmIaAQv2jRIk2dOlVTp05V7969tXjxYiUnJys9Pb3e8375y1+qf//+mjBhwnlNFgDq4y/A3zcg\nRe3dp6zHtNEAAJqDBof4M2fOaMeOHRo5cqTP8VGjRmnLli11nve3v/1NWVlZev755xs/SwBoAO89\n4aXqAG/fV+JzjDYaAEBz0OC/Kx8+fFiVlZVyuVw+x10ul95//32/53zzzTeaNm2a1qxZo/j4+AZP\navv27Q0ei4ZhTUOHtQ2NQNd1w6EKHSk9Yz2+qWucLjxxQL9+dZe+O119UWvrVrYW/+/V0t9/qLCu\nocG6hgbrGly9evUKy+uGdHeau+66Sw888IDS0tIkScaYUL4cgBYs2302wLe2S8O6VNcoPAFekq4f\nekGTzwsAgFBocCU+MTFRdrtdbrfb57jb7VZSUpLfcz744ANt2rRJTzzxhKTqEF9VVaVWrVppyZIl\n+sUvfuH3PE/ox/nz/G+bNQ0+1jY0GrOumflu2R0H5fl7330DUpTWp/qvhm3a7LPGzXrwJ0GbZ7Th\n+zU0WNfQYF1Dg3UNjdLS0rC8boNDfFxcnAYOHKjs7Gzdeuut1vHs7Gzddtttfs/Jy8vzefzmm2/q\nN7/5jbZt26auXbs2csoA4Mu7F94Za9f47wP8mqyCcE0JAICQCmivtdmzZ2vKlCkaNGiQhgwZovT0\ndBUWFmr69OmSpDlz5mjbtm1at26dJOmyyy7zOX/btm2y2Wzq27dvkKYPoKXLzHerrKLSejw5Ndn6\nPOPNndbnbC0JAGhOAvqtNnHiRBUXF2vhwoUqLCxUamqq1q5dq5SUFElSUVGR9u/fH5KJAoA/dVXh\nJam8vML6nK0lAQDNScClqenTp1uV95qWL19e77l333237r777kBfEgD8qq8KXxNbSwIAmpOQ7k4D\nAKFUXxWefngAQHNGiAcQteqrwtMPDwBozgjxAKJSZr7vdrc1q/D0wwMAmjPKUwCiTma+W8tyD1qP\nnbF2SdXhPePNnT4B3uGIpR8eANDsUIkHEFVqBnjpbCtNzQAvUYUHADRPVOIBRA1/Af6+ASka38dV\nq4XG4YjV7eP6UYUHADRLhHgAUaGuAG/fV6LJz22uFeBXLR3f1FMEAKDJ0E4DICp4bycpna3A00ID\nAGiJqMQDiAre20nSQgMAaOkI8QAiXl3bSdbcC54WGgBAS0E7DYCIVtd2kpLYCx4A0GIR4gFEtJq9\n8DXvzOpBCw0AoCWhnQZAxMrMd9fZC+/dSgMAQEtDiAcQsbyr8M5YuxXgV7z2ic84h4MfZQCAloV2\nGgARqWYV3vuurN48O9IAANCSUL4CEJHqqsJ7X8x696T+9MIDAFokKvEAIlLNKnzNNhqHI5YADwBo\nsQjxACLOhkO+d2D13JnVGy00AICWjBAPIOJku89Yn3v2haeNBgCAswjxACLKhkMV+u5sJ436VEqT\np2X6jCHAAwBaOkI8gIiRme/W29/4VuH3vb/PpwrPdpIAABDiAUSIzHy3luUe9Dk2OTW5VoCnFx4A\nALaYBBAB/AV4z91ZV3odW7V0fNNODACACEUlHkDYee8JL0k3dY2z9oUHAAC1EeIBhFXNO7Pe1DVO\nw7pU/5HQe1tJeuEBADiL34oAwqrmnVmHdTGSVOvurPTCAwBwFpV4AGFV886sHjWr8GwrCQDAWYR4\nABFjfB+X9TlVeAAA6kY7DYCwycx31zqW84/D+nvOt4qNc1jHqMIDAOCLSjyAsKi5raQz1i5J+nvO\nt/rudJV1nAtaAQCojRAPICxqbis5OTVZa7IKagV4WmkAAKiNEheAsPC+oNVzY6fJz222jjkcsdzc\nCQCAOlCJB9DkavbCe27sxMWsAAA0DCEeQJOruTe85LulZOtWNi5mBQCgHoR4AE3O397w3lX464de\n0ORzAgAgmtATD6DJZOa7farwJaXlev25zVrpFeAlaehViU09NQAAokrAlfglS5aoZ8+ecjqdSktL\nU05OTp1jN2zYoHHjxqlr165q06aN+vfvr+XLl5/XhAFEJ8+Wkt5V+NLiMp8KvFTdSgMAAOoX0G/L\njIwMzZo1S/PmzVNubq4GDx6sMWPG6ODBg37Hb9myRVdccYVWr16tnTt36v7779e0adP02muvBWXy\nAKJHzS0lnbF2dXaf8jnmcMTSSgMAQAME1E6zaNEiTZ06VVOnTpUkLV68WO+8847S09O1cOHCWuPn\nzJnj83j69On64IMPtHr1ak2aNOk8pg0gmmTmu2ttKWnfV6IVxeVnx/xpoiRp+/btTT4/AACiTYMr\n8WfOnNGOHTs0cuRIn+OjRo3Sli1bGvyCx44dU8eOHRs+QwBRzd+dWcf3cfnsRsNdWQEACEyDf3Me\nPnxYlZWVcrlcPsddLpfef//9Bn2Nt99+W3//+98DCv0Aoltdd2ZlT3gAABqvycpfmzdv1p133qnn\nn39eAwcOrHcsf04PPtY0dFjbum04VKEjpWesxzd1jdOFJw7o16/u0nenqyRVX8ja7YLjtdaRdQ0N\n1jU0WNfQYF1Dg3UNrl69eoXldRsc4hMTE2W32+V2+95p0e12Kykpqd5zc3JydOONN2rBggWaNm1a\n42YKIGpsOFShbPcZfXe2DV6t7dKwLtU/cjwBXmJPeAAAGqPBIT4uLk4DBw5Udna2br31Vut4dna2\nbrvttjrP27hxo2666SbNnz9fDz30UINeKy0traHTwjl4/rfNmgYfa1u3p1/Pld0Rp3ivY/cNSFFa\nn+p2vDZt9lnHZz34E59zWdfQYF1Dg3UNDdY1NFjX0CgtLQ3L6wbUTjN79mxNmTJFgwYN0pAhQ5Se\nnq7CwkJNnz5dUvVuNNu2bdO6deskSevXr9dNN92kBx98UJMmTbKq+Ha7XYmJ3MwFaK68d6Jxxto1\nOTVZ478P8GuyCsI1LQAAmo2AQvzEiRNVXFyshQsXqrCwUKmpqVq7dq1SUlIkSUVFRdq/f781fsWK\nFSorK9Ozzz6rZ5991jp+0UUXad++fbW+PoDoVvOOrJL0+oQB1udrsgq04rVPrMfsSgMAQOME/Bt0\n+vTpVuW9ppp3Y12+fDl3aAVakFV5hbWq8B41A7zErjQAADQW9zcHEDT+2mg8vPeFl6S7J/XX2J/2\nbrK5AQDQnPC3bABBkZnvu3PV6xMGaE1WgSY/t9lnT3iJAA8AwPkixAM4b/7uyuqvfUaq7oMnwAMA\ncH5opwFwXmoGeKn6rqw122ek6gBPHzwAAOePSjyAgHl2ofHugff4YYxdr9dooaF9BgCA4CLEAwhY\nfQF+198+8zlG+wwAAMFHiAcQsJoB/tSx72TfU6xd7lM+x2mfAQAgNAjxAAJScxeav00aqMnTMtmB\nBgCAJkSIBxAQ7zuyem7m5B3gPdV3AjwAAKFDiAdwTnVdyDo5NVlrsgp8jq1aOr4ppwYAQItEiAdw\nTjUDfElpuUqLy7Qya6/POIeDHykAADQFfuMCOKeaAf7o4VPqcqis1jguYgUAoGkQ4gHUy/tC1pLS\ncrk2fy1XjTH0wQMA0LQI8QDq5H03Vk8F3jvAswMNAADhQYgH4MP7ItaS0nIdLSlXVZWRJLm8WmgI\n8AAAhA8hHoAkaU1WgZ7/YI8OdnL4fd7lPqXOxeWSCPAAAISbLdwTABB+a7IKtOK1T/RNh9a1nrNV\nGSvAOxysSbNHAAARNElEQVSxBHgAACIAlXigBZuXkas39x1WhST17eTznHdw56JVAAAiCyEeaMGs\nAO+lcyenkju30etzBoRlTgAA4NwI8UALlZnv9gnwNluMOnZwKLlzG01OTQ7bvAAAwLkR4oEWxqeF\n5nu2KqNdc24I25wAAEBgCPFAC+HZOnLXF0dUVeO5riXfhWVOAACgcQjxQAvgfdMmz57vUnUFvmvJ\nd3poRK9wTQ0AADQCIR5o5uZl5CrziyO1wnuXQ2Xa8PtbwjgzAADQWIR4oBmbl5Gr1/cd9jnmvXUk\nAACITvwWB6LYmqwCZby5U+XlNTeKlI50csjtivc5VnPvdwAAEJ0I8UAUqyvAS9KhLk6fxxN6JmoB\ne78DANAsEOKBCFZfpd2fI50cOtTFqSpbjHXMZovR+Is7a8HtBHgAAJoLQjwQwRoa4B2OWE2YPUTL\ncg+qbY3nnLF2LZhAgAcAoDmxhXsCAPxbk1XQ4AB/+7h+WpVXWOs5Z6ydu68CANAMUYkHIoy/FhqH\nI1arlo73Oz4z361X8wpVVlFpHbtvQIrG93GFfK4AACA8CPFABFmTVaAVr31S67hnJxnPXVe9A3tN\nzlg7AR4AgGaOEA9ECH8B3tMqM/anvX3uuloX2mcAAGgZCPFAiDV0h5kjnRw61LujtbNM505OdUhw\naOmxE1r62o56z/WEdyrwAAC0DIR4IITqao+pqeaNmTwBvi70vAMA0LIR4oHzFOhe7jV5B3ibLUYd\nOzjqDPBU3AEAgESIB86prpB+8uQpSVKbNvH+TrN4bsDUMTHebzhv+/2HB1V2AABwLgHvE79kyRL1\n7NlTTqdTaWlpysnJqXd8Xl6ehg8frvj4eHXv3l3z589v9GSBcGholf1IJ4fye3fUrr6dfD4OJbep\nM8DXRIAHAAANEVAlPiMjQ7NmzdKLL76oIUOG6IUXXtCYMWO0e/dupaSk1Bp//PhxjRw5UsOHD9eO\nHTu0e/du3XPPPWrbtq0efvjhoL0JIFT83XDJU1mvUAdJks1ms9pgejQgqPtDmwwAAAhEQCF+0aJF\nmjp1qqZOnSpJWrx4sd555x2lp6dr4cKFtcavXLlSZWVlWrFihVq1aqW+fftq9+7deu655wjxCKtz\n9bF7gnqVLUbq20lSdb96j4s6WO0vp75vp4k/RzuNREgHAADB1eAQf+bMGe3YsUOPPPKIz/FRo0Zp\ny5Ytfs/58MMPde2116pVq1bWsdGjR+vxxx/Xl19+qYsuuqiR00ZLFchFpD5B3J8e7QN67Y4dzl1l\nJ6wDAICm0OAQf/jwYVVWVsrl8g0nLpdL77//vt9zioqK1L1791rjjTEqKiqK6BDfkDtjRoNTJ8sk\nSfGf17/PeKQrKS3X0ZJyVVWZgMP3+fK3Y4wz1q7ru8ZpWJdYpaUNbNL5AAAAROTuNKWlpeGegn6c\n7NCPk3uEexqIApHw/dqc9OrVSxLrGmysa2iwrqHBuoYG69q8NHh3msTERNntdrndbp/jbrdbSUlJ\nfs9JSkryOz4mJqbOcwAAAADUr8EhPi4uTgMHDlR2drbP8ezsbA0ZMsTvOddcc402bdqk06dPW8fe\ne+89de3aNaJbaQAAAIBIFmOMMQ0d/Je//EVTpkzRCy+8oCFDhig9PV3Lly/Xrl27lJKSojlz5mjb\ntm1at26dJOnYsWPq06ePhg8frrlz56qgoED33nuvnnzySc2aNStkbwoAAABozgLqiZ84caKKi4u1\ncOFCFRYWKjU1VWvXrrX2iC8qKtL+/fut8e3bt1d2drYefPBBDRo0SB07dtQjjzxCgAcAAADOQ0CV\neAAAAADh1+Ce+IaaNm2afvCDHyg+Pl4XXHCBxo0bp/z8fJ8xJSUluuuuu9ShQwd16NBBU6ZMqXWl\n9IEDB3TzzTerbdu26tKli2bOnKmKCt+9wfPy8jR8+HDFx8ere/fumj9/fq35bNiwQWlpaXI6nfrB\nD36g//mf/wn2Ww65o0ePasaMGerbt6/i4+N14YUX6oEHHlBxcbHPONY1cC+99JKuv/56dezYUTab\nTV999VWtMaxr01myZIl69uwpp9OptLQ05eTkhHtKYbNp0yaNHTtWKSkpstls+tOf/lRrzBNPPKFu\n3bopPj5eI0aM0K5du3yeP336tB566CF16dJFbdu21dixY/X111/7jAnW93e0eOqpp/SjH/1ICQkJ\nuuCCC3TLLbdo586dtcaxtoFZsmSJ+vfvr4SEBCUkJGjw4MHKysryGcOanr+nnnpKNptNM2bM8DnO\n2gbmySef/P5u62c/unbt6jMmKtbUBNnSpUtNTk6O+fLLL83HH39sbrnlFpOSkmIqKiqsMT/5yU9M\namqq+cc//mE+/PBD069fP3PLLbdYz1dWVprU1FQzYsQIk5uba9atW2e6du1qZsyYYY05duyYSUpK\nMpMmTTK7du0yq1evNu3atTPPPfecNWb//v2mTZs2ZubMmSY/P9+89NJLJi4uzmRmZgb7bYdUXl6e\nufXWW83bb79t9u7dazZu3Gj69etnRo8e7TOOdQ3c73//e/P000+bP/zhD8Zms5kvv/yy1hjWtWm8\n9tprJi4uzixbtszk5+ebhx56yLRt29YcOHAg3FMLi6ysLDN37lyzevVq06ZNG7NixQqf559++mnT\nvn1788Ybb5idO3eaiRMnmq5du5oTJ05YY6ZPn266detm3n//ffPxxx+b4cOHmwEDBpiqqiprTDC+\nv6PJT37yE7NixQqzc+dOk5eXZ372s5+ZpKQkc/ToUWsMaxu4t956y7zzzjtm7969Zs+ePWbu3Lkm\nLi7OfPrpp8YY1jQYtm7danr06GEGDBhgHnroIes4axu4J554wvTt29d8++23xu12G7fbbQ4fPmw9\nHy1rGvQQX9P//d//mZiYGPPZZ58ZY4zZtWuXiYmJMVu3brXG5OTk+IzJysoydrvdfP3119aYlStX\nGqfTaY4fP26MMWbJkiUmISHBfPfdd9aYBQsWmJSUFOvxL3/5S3PppZf6zOcXv/iFGTx4cPDfaBPz\nrJFnPXbv3s26noft27f7DfGsa9O56qqrzL/+67/6HOvVq5d57LHHwjSjyNG2bdtaIT45Odk89dRT\n1uOysjLTrl07s3TpUmOMMaWlpaZVq1bmf//3f60xBw4cMDabzbz33nvGmOD9PI5mJ06cMHa73bz9\n9tvWMdY2ODp16mStGWt6fkpKSswll1xi1q9fb4YPH+4T4lnbwD3xxBPm8ssvr/P5aFnToLfTeDt5\n8qT++Mc/6uKLL9bFF18sSfrwww/Vrl07XX311da4IUOGqE2bNtqyZYs1pm/fvj5/2hg9erTKy8u1\nY8cOa8y1116rVq1a+Yz55ptv9OWXX1pjRo0a5TOn0aNHa/v27aqsjO47sZaWlqp169aKj4+XJG3d\nupV1DQHWtWmcOXNGO3bs0MiRI32Ojxo1ylpnnLV//34VFRX5rJfD4dB1111nrdf27dtVUVHhMyYl\nJUV9+/b1+d4Nxvd3NDt27JiqqqrUsWNHSaxtMFRVVem1117TyZMnNWTIENY0CKZNm6aJEydq2LBh\nPsdZ28bbt2+funXrpp49e+qOO+6wNmaJpjUNSYhPT09Xu3bt1K5dO7377rtat26d4uLiJFXvYNOl\nS5da51xwwQUqKiqyxrhcLp/nPTebqm+My+WSMeacYyoqKnT48OHgvNkwKCkp0eOPP65p06bJZqv+\nJ2RdQ4N1bRqHDx9WZWWl3/fvWR+cVVRUpJiYmHrXy+12y263q3PnznWOCdb3dzSbOXOmrrzySl1z\nzTWSWNvzkZeXp3bt2ql169Z64IEH9MYbb+iyyy5jTc/TSy+9pH379mnBggW1nmNtG+fqq6/WK6+8\nonfffVcvv/yyioqKNGTIEB09ejSq1rRBIf4///M/a10A4P1ht9u1ceNGa/zPf/5z5ebmauPGjbr0\n0ks1YcIElZeXN+SlWpRA11Wq/uvGzTffrO7du+uZZ54J08wjW2PWFUDLM3v2bG3ZskWrV69WTExM\nuKcT9fr06aNPPvlE//znP3X//fdrypQptS4GRGA+++wzzZ07V6tWrbKKdjh/o0eP1oQJE5Samqrr\nr79ef/vb31RZWakVK1aEe2oBadA+8Q8//LDuuuuuesdceOGF1ueeKvwll1yiq666Sh07dtTq1at1\n5513KikpSYcOHap1/rfffqukpCRJUlJSUq0/oXsqdcnJydYYt9vtM8btdismJsbn6/gbExsbq8TE\nxIa89ZAKdF1PnjypMWPGyG63669//atPawbrelag61of1rVpeCoP/t6/Z31wVlJSkowxcrvd1n06\nJN/1SkpKUmVlpY4cOeJTLXK73bruuuusMefz/R3N/zYPP/yw/vKXv2j9+vU+dxBnbRsvNjZWPXv2\nlCT98Ic/1D//+U8tWrRIjz32GGvaSFu3btWRI0d02WWXWccqKyu1ceNGvfjii8rLy2NtgyA+Pl79\n+vXTnj17NHbs2OhZ03N2zZ+n8vJyEx8fb5YtW2aMqb5Q0Gaz+TT6b9682dhsNqvRf+3atbUa/f/8\n5z/7NPqnp6fXulBw4cKFPhcK/sd//Ifp3bu3z3z+5V/+JSovFDx+/LgZOnSoufbaa83JkydrPc+6\nnp/6LmxlXZuGvwtbL730UjN37twwzShyNPTC1vbt25uXXnrJGFP/hVfZ2dnGmOB9f0ebGTNmmOTk\nZFNQUOD3edY2OK6//npz1113GWNY08YqLS01O3fu9PkYNGiQufPOO82uXbuMMaxtMJSVlZnk5GSz\nYMECY0z0rGlQQ/znn39unnnmGbNjxw7z1Vdfmc2bN5ubb77ZdOrUybjdbmvcmDFjzBVXXGG2bt1q\ntmzZYi6//HIzduxY6/nKykpzxRVXmB//+Mfm448/NtnZ2aZbt25m5syZ1pjS0lKTnJxs7rjjDpOX\nl2dWr15t2rdvbxYtWmSN2b9/v2nbtq2ZNWuW2b17t3nppZdM69atzRtvvBHMtx1yx48fN1dffbVJ\nTU01n3/+uSkqKrI+Tp8+bY1jXQNXVFRkcnNzzZ///GcTExNjsrKyTG5urikuLrbGsK5NIyMjw7Ru\n3dq8/PLLZvfu3WbGjBmmXbt25quvvgr31MLixIkTJjc313z88ccmPj7ezJ8/3+Tm5lrr8cwzz5gO\nHTqYzMxM8+mnn5rbb7/ddOvWzWcLtPvvv990797drFu3znz00UdmxIgR5sorr/TZAi0Y39/R5IEH\nHjDt27c3H3zwgc/PUu91Y20D9+ijj5pNmzaZL774wnz66afm0UcfNXa73bz77rvGGNY0mGruTsPa\nBu7f//3fzYYNG8z+/fvNhx9+aG688UaTkJAQdT9fgxriDxw4YMaMGWNcLpdp3bq1ufDCC83Pf/7z\nWtWOkpISc9ddd5mEhASTkJBgpkyZYkpLS2t9rZtvvtm0adPGJCYmmlmzZvkEVmOq908fNmyYcTqd\npmvXrmb+/Pm15rRx40YzcOBA43A4TM+ePa3tgaLJ+vXrjc1m8/mIiYkxNpvNbNiwwRrHugbuiSee\nsNbS+8O76sm6Np309HTTo0cP43A4TFpamsnJyQn3lMJm/fr1fr837733XmvMk08+abp27WqcTqcZ\nPny42blzp8/XOH36tJkxY4ZJTEw0bdq0MWPHjjUHDx70GROs7+9o4W9NbTabefLJJ33GsbaBueee\ne8zFF19sHA6HcblcZuTIkVZF0oM1DY4RI0b4hHhjWNtATZo0yXTr1s20bt3apKSkmAkTJpjdu3f7\njImGNY0xxpiAm4cAAAAAhA2XOgMAAABRhhAPAAAARBlCPAAAABBlCPEAAABAlCHEAwAAAFGGEA8A\nAABEGUI8AAAAEGUI8QAAAECUIcQDAAAAUeb/AX/0TB86WcqNAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "MakePrice2(player1, player2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, we use the variance of `diff` to estimate the variance\n", + "of `error`. This estimate is not perfect because contestants’\n", + "bids are sometimes strategic; for example, if Player 2 thinks that\n", + "Player 1 has overbid, Player 2 might make a very low bid. In that case\n", + "`diff` does not reflect `error`. If this happens a\n", + "lot, the observed variance in `diff` might overestimate the\n", + "variance in `error`. Nevertheless, I think it is a reasonable\n", + "modeling decision.\n", + "\n", + "As an alternative, someone preparing to appear on the show could\n", + "estimate their own distribution of `error` by watching\n", + "previous shows and recording their guesses and the actual prices." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Likelihood\n", + "\n", + "Now we are ready to write the likelihood function. As usual, I define a\n", + "new class that extends `thinkbayes.Suite`:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Price(thinkbayes.Suite):\n", + "\n", + " def __init__(self, pmf, player):\n", + " thinkbayes.Suite.__init__(self, pmf)\n", + " self.player = player\n", + " \n", + " def Likelihood(self, data, hypo):\n", + " price = hypo\n", + " guess = data\n", + "\n", + " error = price - guess\n", + " like = self.player.ErrorDensity(error)\n", + "\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pmf` represents the prior distribution and\n", + "`player` is a Player object as described in the previous\n", + "section. In `Likelihood` `hypo` is the hypothetical price of the showcase.\n", + "`data` is the contestant’s best guess at the price.\n", + "`error` is the difference, and `like` is the\n", + "likelihood of the data, given the hypothesis.\n", + "\n", + "`ErrorDensity` is defined in `Player`:\n", + "\n", + "```python\n", + "# class Player:\n", + "\n", + " def ErrorDensity(self, error):\n", + " return self.pdf_error.Density(error)\n", + "```\n", + "\n", + "`ErrorDensity` works by evaluating `pdf_error` at the given\n", + "value of `error`. The result is a probability density, so it\n", + "is not really a probability. But remember that `Likelihood`\n", + "doesn’t need to compute a probability; it only has to compute something\n", + "*proportional* to a probability. As long as the constant of\n", + "proportionality is the same for all likelihoods, it gets canceled out\n", + "when we normalize the posterior distribution.\n", + "\n", + "And therefore, a probability density is a perfectly good likelihood." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Update\n", + "\n", + "`Player` provides a method that takes the contestant’s guess\n", + "and computes the posterior distribution:\n", + "\n", + "```python\n", + "# class Player\n", + " def MakeBeliefs(self, guess):\n", + " pmf = self.PmfPrice()\n", + " self.prior = Price(pmf, self)\n", + " self.posterior = self.prior.Copy()\n", + " self.posterior.Update(guess)\n", + "```\n", + "\n", + "`PmfPrice` generates a discrete approximation to the PDF of\n", + "price, which we use to construct the prior.\n", + "\n", + "`PmfPrice` uses `MakePmf`, which evaluates\n", + "`pdf_price` at a sequence of values:\n", + "\n", + "```python\n", + "# class Player\n", + " n = 101\n", + " price_xs = numpy.linspace(0, 75000, n)\n", + "\n", + " def PmfPrice(self):\n", + " return self.pdf_price.MakePmf(self.price_xs)\n", + "```\n", + "\n", + "To construct the posterior, we make a copy of the prior and then invoke\n", + "`Update`, which invokes `Likelihood` for each\n", + "hypothesis, multiplies the priors by the likelihoods, and renormalizes.\n", + "\n", + "So let’s get back to the original scenario. Suppose you are Player 1 and\n", + "when you see your showcase, your best guess is that the total price of\n", + "the prizes is \\$20,000.\n", + "\n", + "Figure [fig.price3] shows prior and posterior beliefs about the actual\n", + "price. The posterior is shifted to the left because your guess is on the\n", + "low end of the prior range.\n", + "\n", + "On one level, this result makes sense. The most likely value in the\n", + "prior is \\$27,750, your best guess is \\$20,000, and the mean of the\n", + "posterior is somewhere in between: \\$25,096.\n", + "\n", + "On another level, you might find this result bizarre, because it\n", + "suggests that if you *think* the price is \\$20,000, then\n", + "you should *believe* the price is \\$24,000.\n", + "\n", + "To resolve this apparent paradox, remember that you are combining two\n", + "sources of information, historical data about past showcases and guesses\n", + "about the prizes you see.\n", + "\n", + "We are treating the historical data as the prior and updating it based\n", + "on your guesses, but we could equivalently use your guess as a prior and\n", + "update it based on historical data.\n", + "\n", + "If you think of it that way, maybe it is less surprising that the most\n", + "likely value in the posterior is not your original guess." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Optimal bidding\n", + "\n", + "Now that we have a posterior distribution, we can use it to compute the\n", + "optimal bid, which I define as the bid that maximizes expected return\n", + "(see ).\n", + "\n", + "I’m going to present the methods in this section top-down, which means I\n", + "will show you how they are used before I show you how they work. If you\n", + "see an unfamiliar method, don’t worry; the definition will be along\n", + "shortly.\n", + "\n", + "To compute optimal bids, I wrote a class called\n", + "`GainCalculator`:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class GainCalculator(object):\n", + "\n", + " def __init__(self, player, opponent):\n", + " self.player = player\n", + " self.opponent = opponent\n", + " \n", + " def ExpectedGains(self, low=0, high=75000, n=101):\n", + " bids = numpy.linspace(low, high, n)\n", + "\n", + " gains = [self.ExpectedGain(bid) for bid in bids]\n", + "\n", + " return bids, gains \n", + "\n", + " def ExpectedGain(self, bid):\n", + " suite = self.player.posterior\n", + " total = 0\n", + " for price, prob in sorted(suite.Items()):\n", + " gain = self.Gain(bid, price)\n", + " total += prob * gain\n", + " return total\n", + " \n", + " def Gain(self, bid, price):\n", + " # if you overbid, you get nothing\n", + " if bid > price:\n", + " return 0\n", + "\n", + " # otherwise compute the probability of winning\n", + " diff = price - bid\n", + " prob = self.ProbWin(diff)\n", + "\n", + " # if you are within 250 dollars, you win both showcases\n", + " if diff <= 250:\n", + " return 2 * price * prob\n", + " else:\n", + " return price * prob\n", + " \n", + " def ProbWin(self, diff):\n", + " prob = (self.opponent.ProbOverbid() +\n", + " self.opponent.ProbWorseThan(diff))\n", + " return prob" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`player` and `opponent` are `Player`\n", + "objects.\n", + "\n", + "`GainCalculator` provides `ExpectedGains`, which\n", + "computes a sequence of bids and the expected gain for each bid:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`low` and `high` specify the range of possible\n", + "bids; `n` is the number of bids to try.\n", + "\n", + "`ExpectedGains` calls `ExpectedGain`, which\n", + "computes expected gain for a given bid:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`ExpectedGain` loops through the values in the posterior and\n", + "computes the gain for each bid, given the actual prices of the showcase.\n", + "It weights each gain with the corresponding probability and returns the\n", + "total.\n", + "\n", + "`ExpectedGain` invokes `Gain`, which takes a bid\n", + "and an actual price and returns the expected gain." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Warning: Brewer ran out of colors.\n", + "Player 1 optimal bid (16668.781463564381, 21000.0)\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwYAAAEWCAYAAADcj4BmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdYVNf28PHvGXpHZBAVJSJW7AXF3tDYjYmxRDGaqPca\njd4Uf7nJvTG9mJsYTfImpltiSUyMRqNii4qIir1giwUboEgRcGhz3j+OzIBYQIEZYH2eZ56Zfc6e\nmTUHlLNmn722oqqqihBCCCGEEKJS01k6ACGEEEIIIYTlSWIghBBCCCGEkMRACCGEEEIIIYmBEEII\nIYQQAkkMhBBCCCGEEEhiIIQQQgghhEASAyGEEEIIIQRFSAzef/99goOD8fDwwMfHh0GDBnH06NFC\n/d544w1q1qyJs7Mz3bt359ixYwX2Z2VlMXXqVPR6Pa6urgwePJhLly4V6JOcnMyYMWPw9PTE09OT\nsLAwUlJSCvS5cOECAwcOxNXVFb1ez7Rp08jJyXmQzy6EEEIIIYS45b6JwbZt25gyZQo7d+5ky5Yt\n2Nra0qtXL5KTk019PvzwQ2bPns0XX3xBdHQ0Pj4+hIaGkp6ebuozbdo0VqxYwbJly4iIiCA1NZUB\nAwaQf321kSNHcuDAAcLDw1m/fj379u0jLCzMtN9oNNKvXz/S09PZsWMHS5cuZfny5bz44osldTyE\nEEIIIYSolJTirnycnp6Oh4cHK1eupH///gDUqFGD559/nldeeQUAg8GAj48PH3/8MRMmTCA1NRW9\nXs/8+fMZMWIEABcvXsTf359169YRGhpKTEwMQUFBREZG0r59ewB27NhB586dOXHiBPXq1WPt2rUM\nHDiQ2NhYatSoAcBPP/3EhAkTSEhIwNXVtcQOjBBCCCGEEJVJsecYpKamYjQaqVKlCgBnz54lLi6O\n0NBQUx9HR0e6dOlCZGQkANHR0eTk5BTo4+fnR6NGjUx9oqKicHNzMyUFAB07dsTFxaVAn0aNGpmS\nAoA+ffpgMBjYu3dvcT+KEEIIIYQQ4pZiJwbTpk2jVatWhISEABAXF4eiKFSrVq1Av2rVqhEXFwdA\nfHw8NjY2VK1a9a594uLi0Ov1hd7Px8enQJ/b38fb2xsbGxtTHyGEEEIIIUTx2Ran8wsvvEBkZCQ7\nduxAUZTSiqnE3D5xWQghhBBCiPLEw8OjzN6ryCMG//rXv1i2bBlbtmzB39/ftN3X1xdVVYmPjy/Q\nPz4+Hl9fX1Of3NxcEhMT79nn6tWrhd43ISGhQJ/b3+fatWvk5uaa+gghhBBCCCGKr0iJwbRp00xJ\nQb169Qrsq1OnDr6+vmzYsMG0zWAwsH37djp27AhA69atsbW1LdDn4sWLxMTEmPqEhISQlpZGVFSU\nqU9kZCQZGRl06NDB1CcmJobLly+b+oSHh+Po6Ejr1q2L+9mFEEIIIYQQt9y3KtFzzz3HokWLWLly\nJY0aNTJtd3V1xcXFBYBZs2bx/vvv8/3331OvXj3eeecdIiIiOHHihKnP5MmTWb16NT/88ANeXl68\n+OKLpKSkEB0dbbosqV+/fly6dIl58+ahqiqTJk0iICCA33//HdDKlbZs2RK9Xs///vc/rl27xtNP\nP80TTzzBp59+Wij2/JcSleUwTEUXHR0NQJs2bSwcScUix7V0yHEtHXJcS4cc19Ihx7V0yHEtPZY6\nh73vHIMvv/wSRVHo2bNnge0zZ87k9ddfB2DGjBkYDAamTJlCUlIS7dq1Izw83JQUAMyZMwc7OztG\njBjBzZs36dWrFwsXLiwwV2HJkiVMnTqVRx99FIDBgwfz2WefmfbrdDrWrFnD5MmT6dSpE05OTowe\nPZpZs2Y93FEQQgghhBCikrtvYmA0Gov0Qq+//ropUbgTOzs75syZw5w5c+7ax8PDgwULFtzzffz8\n/Fi1alWRYhJCCCGEEEIUTbHLlQohREWWm6sSl6iSklastR+FEEKIcq9Y5UqFEKKiSE1X2XcCLiTA\npatwMQEuX9NuublaH58qKvVrQ4PamO5reFMuyjULIYQQxSWJgRCiUjl1QWXZJlgXBZlZ9+6bkKTd\nIg6at7m7wJAuKmP7gbuLJAhCCCEqDkkMhBAVXq4RNu9VWboR9p+4f38vd0i7CVnZhfelpsOCtbBi\nK4ztpzK8Jzg6SIIghCgeVVXJzs4u8lxOa5S3rpXBYLBwJOWLvb09Op11Xs0viYEQosLKylZZs9uL\njfu9MOQW3l+/NjSvB3568POBmt5QQw9ODgq5uSrn4+BErHY7eQFOnNcSA4AbGfD5cli2CZ4dqDKo\nE9jaSoIghLg/VVUxGAzY29tjZ2dXbi9PdHR0tHQI5U7ez97R0dEqf+6SGAghKqTj51Xe+BYOnfIB\nwMVZ267TQY/WMLynlhTc7T9mGxuFgJoQUBP6hmjbjEaVjXvgq9/hwq1F2K8mwfsL4Kf1MHWYSrdW\n1vcfvRDCumRnZ2Nvb4+NjY2lQxFlTFEU7O3tycrKwsHBwdLhFCKJgRCiQsnJUfnhT/juD/MkYgBP\nN3isKzzRHXyqPNjJu06n0Lsd9GitsioCvlkF15K1fbHx8PLn8MxAlUlDZIKyEOLujEYjdnZ2lg5D\nWIiNjQ3Z2Xe4VtUKSGIghKgwzlxSeeM7iDln3uZgq/JE5wReHh+AvV3JnKzb2ioM7Qb9QrSJzPP/\n1C4tAi0hSUiCV8NUubRICHFX8uWBsEaSGAghyj2jUeWncPhqRcEJw00DYWibM/h6ZZdYUpCfo4PC\n2H7wWFeV/3wNOw9r2/+IgGsp8ME/VZwd5Y+/EEKI8sE6p0QLIUQRpWWoTJ0Nc382JwV2tjB1GHz7\nCvh6lf5wrbuLwidTYWAn87adh+Efs+B6qiyUJoQQonyQxEAIUW4lJKlM/BB2HzVva+gPi2ZCWF8F\nna7svq23tVX47zh4dpB5W8w5GPcOxMZJciCEEPlt3boVnU7Htm3bLB2KyEcSAyFEuXTmksr4d+HU\nBfO2cQPgh9cgoKZlLt9RFIVJQxReHatVPwJtJeXx78Gxs5IcCCEqvvnz56PT6Uw3Ozs7atWqxfjx\n47l8+XKBvuV5nsWePXt47rnnCA4OxtHREZ1OR0JCgqXDemiSGAghyp19J1SefR/ir2ttGxt4fTxM\nHqpYxYTfx7oq/G8KON6qRJeSBv+ao41wCCFERacoCm+99RaLFi1i3rx59O7dmwULFtClS5cKsxja\nn3/+ybfffktubi4NGjQo10lOfpIYCCHKlY17VKZ8bK4C5OwIs6fBwE7W9Z9y5xYKX70M7i5a+3qq\nVs40K1uSAyFExde7d29GjRrF+PHj+e6775g+fTpnz55l5cqVlg6tSLKzs8nNvcPKmLdMnjyZ1NRU\n9u7dy9ChQ8swstIliYEQotxYHK7y6leQnaO1vdxh3v9BSBPrSgryBAUozHrOfFnRsbPw3gJt5Ush\nhKhMevTogaqqnD179q59Dh8+zPjx4wkMDMTJyQm9Xs/IkSO5cMF8zejp06fR6XR8+umnhZ5/6NAh\ndDod8+bNM22Li4vj2WefpXr16jg6OtK4cWO++uqrAs/Lm++wePFi3njjDfz9/XF2dubSpUt3jVWv\n11vlAmUPS8qVCiHKhe9Xq3z5m7nt7wtz/wU19NaZFORp3VDhhREq/1ustdfsgAa1YWSoZeMSQoiy\ndPr0aQCqVq161z4bNmzg5MmTjB07lho1avD333/z5ZdfsmfPHo4cOYKjoyOBgYGEhISwaNEipk+f\nXuD5ixYtwsHBgeHDhwNw9epV2rVrh6qqPPfcc/j4+LBp0yYmT57M9evXefXVVws8/7333sPGxobp\n06ejqiqurq4lfBSsnyQGQgirtzi8YFLQNBBmPw8ertadFOR5siccj4XVEVr702UQ6KfStlH5iF8I\nIYorJSWFxMREDAYDERERvP322zg7OzNgwIC7Pmfy5Mm88MILBbYNGjSIDh068NtvvzFq1CgAwsLC\nmDx5MsePH6dhw4aANhK7dOlSBgwYgKenJwCvvfYa2dnZHDlyBC8vLwAmTpyIh4cH7733HlOmTMHd\n3d30XmlpaRw/fhxHR8cSPRbliSQGQgir9ttfKrOXmtttGsGn08DBvvycVCuKwr/HqJy9BEfPgtEI\n//4SFvxXtfoRDyGE5e04vJ7II+Gl9vodmvSmY9M+JfZ6qqrSp4/59RRFISgoiLlz51K9evW7Pi//\nCXl6ejqZmZkEBgbi6enJ3r17TYnB8OHDmT59OgsXLuTdd98FYMuWLVy8eJHPP//c9Bq//vorQ4cO\nRVVVEhMTTdtDQ0P59ttv2bVrF6Gh5uHbsWPHVuqkAGSOgRDCiv0ZqfLBQnO7aSB8PLV8JQV57O0U\nPpoCVT20dkoavPQ53MyU+QZCiIpFURQ+//xzNm7cSEREBOfPn+fQoUN069btns9LTk5m0qRJeHt7\n4+bmhre3Nz4+PqSkpJCSkmLq5+npyaBBg1i8eLFp26JFi6hatSr9+vUDtMuIkpKS+P7779Hr9QVu\nw4cPR1GUQuVFAwICSu4glFMyYiCEsEqbolXe/B7y5uk2egTmTgdnx/KXFOTRV1GY9ZzKpA8hJ1db\ng+Gt7+G9f6gVptSdEEIAtGnThuDg4GI9Z9iwYezcuZOXXnqJFi1a4ObmhqIoDB8+HKPRWKBvWFgY\ny5cvZ/v27QQHB/Pbb78xevRobG21U9u8/iNHjmT8+PF3fL+goKACbScnp2LFWxFJYiCEsDoRB1Ve\nm6ddcgNQtyZ89gK4Opf/k+dmgQr/N0bl3R+19sY90Lk59OtgyaiEENasY9M+JXqpjzVKTk5m06ZN\nvPXWW/znP/8xbc/MzCQpKalQ/0cffRS9Xs/ChQuJi4vjxo0bjBkzxrRfr9fj5uZGTk4OPXr0KJPP\nUBHIpURCCKuyJ0bl//4f5JWPrl0N/t/L5WeicVEM6aLwWFdz+5OlkJQqlxQJISovGxsbgEIjA598\n8kmhbXn9n3rqKX755Re+++47AgMDadeunWm/TqfjiSee4Pfff+fQoUOFnn/t2rUS/gQVg4wYCCGs\nxslYlZc+g6xsrV3DG76cAV7uFScpyDN9OOw8AnGJ2nyD2cvgrQmWjkoIIR5eUddqyd/Pzc2Nbt26\nMWvWLDIzM/H39yciIoJt27bh7e19x+eHhYUxe/ZsNmzYwJtvvllo/wcffMDWrVsJCQlhwoQJBAUF\nkZSUxP79+1m5ciUZGRkP9gGB2NhYFi7UJsFt27YNVVWZPXs2rq6u+Pv7M3r06Ad+bUuSxEAIYRUS\nklT+NQcyDFrbpwp8+TL4VKl4SQFocyVeGaMy/dYaPWt3Qt8Q1WoXaxNCiKIq6pyp2/stWbKEadOm\n8fXXX5OdnU3Xrl3ZvHkzvXr1uuNrNm/enGbNmnH48GGeeuqpQvv1ej27du3i7bffZuXKlXz11Vd4\neXnRqFEjPvnkkweKOc/Zs2f573//a3qeoijMmjULgK5du5bbxEBRK/ASnPlnsHt4eFgwkoolOjoa\n0CYWiZJTmY9r+k2ViR/CyVit7eIE3/4bAv0e/iTZ2o/ra/NUwndpj2t4w9K3wcnB+pMDaz+u5ZUc\n19JhbcfVYDBU+rKYJSU4OBgHBwe2b99u6VCK5X6/A5Y6h5U5BkIIi8rJUXnlS3NSYGMDs54rmaSg\nPHhxJLi7aI8vX4OvVlg2HiGEKC8OHDhAdHQ0Tz/9tKVDqTAkMRBCWIyqqnz4E0QdMW97NQyCG1eO\npAC0+RPTh5vbSzfCsbMVdiBXCCEe2tGjR5k/fz7jxo3D19f3jpcRiQcjiYEQwmLm/wm/bzW3nxkI\ngzpXnqQgz4CO0Lax9thohHfnayMpQgghClu+fDnPPPMMmZmZLF26VC7LKkGSGAghLGL9LpUvfjW3\n+4bApCGWi8eSFEXh1TBwsNfaJ2Nh8QbLxiSEENZq5syZ5OTkcOzYMbp06WLpcCoUSQyEEGXu0GmV\nN78zt1s3hP+OK35ViIrEz0dh4mBze97vcCFeRg2EEEKUHUkMhBBlKi5R5eXPITtHa9epoU02trOt\nvElBnqd6Q4Pa2uOsbG3hMyGEEKKsSGIghCgzNzNVXvwMrqdqbU83+HQauLtIUgBgY6Pw2tOQN3AS\ncRD2n5RRAyGEEGVDEgMhRJkwGlVmfmsuS2p7qyxpDb0kBfk1ekTh0fbm9mfLi76KqBBCCPEwZOVj\nIcqB66kJRB3bRGzcKWxt7XC0d853c8LJwYXqVWtTp3pDq71O/9s/YMtec/v/xkDL+tYZq6X9Ywhs\n3KNdbnX4NGzdD91aWToqIYQQFZ0kBkJYsWspcUQd3cTx2AOoqvG+/QNqNKJ322G4OVvXSt8b96h8\ns9LcHt4LhnSRpOBuaugVnuihsiRca3/xK3RurmJjI8dMCCFE6ZFLiYSwQleTr7Bqx0J+/PN/xJzf\nV6SkAODM5Rh+WPsRR89GW83lJyfOq7yRrwJRcBD8a/jd+wvN+P7g4qQ9PncFVkdaNh4hhBAVn4wY\nCGFFsnIMrIpYwIkLBwvtq1O9Ie0a98TF0Y2bmekYsjJMt4TkKxw5sxuAzKyb/Bm1hJMXDtE7eBgu\njm5l/TFMElO0ycaZWVq7djV4/x/IN99F4OmmENZX5cvftPbXv0OfYBVHBzl2QgghSockBkJYCUN2\nBrv+XovRJqvA9ro1gwgJCqV61Vr5tuoLPb9JnTasjVpKSvp1AE5fOsqlNWfp2WYoDWu3KPO5B1nZ\nKi9/AfFaOLg6w8fPSwWi4hgZCj9vgsQUSEiCZZtgbD9LRyWEEA9v69atdO/enb/++ksWKbMicimR\nEFYgJe06O0+vJs2QbNpWz68JYX3+xdAu429LCu6slk9dnu77Ei0CO5i23czKYHXkIrbsX1Uqcd+N\nqqp8tFibOAug08G7k+CR6pIUFIeTQ8FFz378E1LTreMSMSGEuJP58+ej0+lMNzs7O2rVqsX48eO5\nfPlygb7WWizjflRV5ccff2Tw4MHUrl0bV1dXmjZtyrvvvktmZqalw3soMmIghIVdT03g581fkZF5\nAwBF0dE/ZBSN/FsW+7Xs7RwIbfs49Ws1Zd2un0nNSAJg74ltVHWvRvPA9vd5hZLx61/w+1Zze8oT\n0KFp+fwDYGmDOsFP6yE2HtIy4Ic1MO1JS0clhBB3pygKb775JgEBARgMBnbs2MH8+fPZtm0bR44c\nwdHR0dIhPpSMjAzGjx9PSEgI//znP/Hx8WHnzp3MnDmTTZs2sXnzZkuH+MAkMRDCghKSLvHLlq/J\nyEwDQNHpGNL5aQJrBj3U6/r71ufpfi+xZudi/r50FICNe3/D28OXmvpHHjbse9p3QuV/i83tR9vD\n6D6l+pYVmq2twuTHVV75f1r7500wvKeKb1VJtIQQ1qt3794EBwcDMH78eKpUqcLs2bNZuXIlw4db\nfwWK7OxsdDodNjY2hfbZ29sTGRlJ+/bmL9ueeeYZ/P39eeONN9iwYQOhoaFlGW6JkUuJhLCQy9fO\nsXTzl6akQKezIbhO74dOCvI42DkyoMNT+HjWAMBozGVlxI/cyEi+zzMfXFyidgKbm6u1Gz0C/3m6\n/A4XW4serSGojvY4Kxu+Xnnv/kIIYW169OiBqqqcPXv2rn0OHz7M+PHjCQwMxMnJCb1ez8iRI7lw\n4YKpz+nTp9HpdHz66aeFnn/o0CF0Oh3z5s0zbYuLi+PZZ5+levXqODo60rhxY7766qsCz9u6dSs6\nnY7Fixfzxhtv4O/vj7OzM5cuXbpjnHZ2dgWSgjyPPfYYqqpy7Nix+x4PayWJgRAWEBt/mp+3zCMz\n6yagncS3r9sXb7eaJfo+9rYODOk8DicHFwDSDTf4ffuP5ORml+j7ABgyVV76DJK0K6LwcoePpoCD\nvSQFD0tRFKYOM7dX74DzcTLXQAhRfpw+rU06q1q16l37bNiwgZMnTzJ27Fg+//xzJkyYwLp16+je\nvTsGgwGAwMBAQkJCWLRoUaHnL1q0CAcHB9OIxNWrV2nXrh3h4eE899xzzJ07l6ZNmzJ58mTee++9\nQs9/7733WLFiBdOnT+fDDz/E1dW1WJ/xypUrAHh7exfredZELiUSoowlpsazYtv3ZOdo1YecHFwY\n1n0SF/6+Uirv5+HqxaCOYfy8ZR6qaiTu+gXC9yynb7sRJfZNvqqqvP0jnIjV2jY28MFkqOYlSUFJ\nad1QoX0TlagjoKow/094fbyloxJClIWvVxZcJLKkTRgMEweX7P/XKSkpJCYmYjAYiIiI4O2338bZ\n2ZkBAwbc9TmTJ0/mhRdeKLBt0KBBdOjQgd9++41Ro0YBEBYWxuTJkzl+/DgNGzYEtL9DS5cuZcCA\nAXh6egLw2muvkZ2dzZEjR/Dy8gJg4sSJeHh48N577zFlyhTc3d1N75WWlsbx48cfeA7ErFmzcHd3\np1+/8ls+TkYMhChDmVk3+X3bD2TlaFULXJ08GNnrOapVKdmRgtvVrhZI95aDTO2jZ6PZdzKixF5/\n4ToI32Vuz3gKWtaXpKCkPZPv7+mfO+HKNRk1EEJYH1VV6dOnD3q9nlq1ajFq1CiqV6/O6tWrqV69\n+l2fl/+EPD09nevXrxMYGIinpyd79+417Rs+fDj29vYsXLjQtG3Lli1cvHiRMWPGmLb9+uuv9O/f\nH1VVSUxMNN1CQ0PJyMhg1658f7iAsWPHPnBS8N5777F582Y+/PBDqlSp8kCvYQ0kMRCijBhVI2ui\nlnD9xlUAbG3seLzrM1R1r1Ym79+qfiea1Glram/Zv4rzcace+nUjD6t88au5/VhXGNpNkoLS0KK+\nQssG2uPcXFi43rLxCCHEnSiKwueff87GjRuJiIjg/PnzHDp0iG7dut3zecnJyUyaNAlvb2/c3Nzw\n9vbGx8eHlJQUUlJSTP08PT0ZNGgQixebK10sWrSIqlWrmr6tv3r1KklJSXz//ffo9foCt+HDh6Mo\nCgkJCQXePyAg4IE+77Jly/jvf//Ls88+y6RJkx7oNayFXEokRBnZeWSDqUIQQJ/gJ/Ep5ZGC/BRF\nIbTt4ySmxnMlMRZVNfLHjgWM7fsSbs4eD/Sa5+NUXpsHRqPWbl4PXn6qBIMWhYzvD1NPaI9XboNn\nB6p4uUsiJkRFNnFwwTVNyoM2bdqYqhIV1bBhw9i5cycvvfQSLVq0wM3NDUVRGD58OMa8PzS3hIWF\nsXz5crZv305wcDC//fYbo0ePxtZWO7XN6z9y5EjGj7/zdZdBQQWLfTg5ORUrXtDmRYwdO5aBAwfy\n5ZdfFvv51kYSAyHKwOlLR4k8Em5qt2nQlcaPtCrzOGxt7Bjc6WkWrp9NuuEGN7My2HZwDf1DRhX7\ntdIyVF6cq9XWB/CpArOeAztbOUktTe2CtGpPMee0CkWL1sPzw+73LCGEsG7Jycls2rSJt956i//8\n5z+m7ZmZmSQlJRXq/+ijj6LX61m4cCFxcXHcuHGjwGVEer0eNzc3cnJy6NGjR6nEvGvXLoYOHUpw\ncDDLli1Dpyv/F+KU/08ghJVLTI1nTeRPprZ/tXp0bdHfYvG4OXsUSASOndvLpat3Lx93J0ajyqvz\n4Hyc1nawh4+fR765LgOKojAu36/Pr1tkNWQhRPmXt17A7SMDn3zySaFtef2feuopfvnlF7777jsC\nAwNp166dab9Op+OJJ57g999/59ChQ4Wef+3atYeKNyYmhgEDBhAQEMAff/yBg4PDQ72etShSYrB9\n+3YGDx6Mn58fOp2OBQsWFNg/bty4Astf63Q6OnToUKBPVlYWU6dORa/X4+rqyuDBgwvVh01OTmbM\nmDF4enri6elJWFhYgWvKAC5cuMDAgQNxdXVFr9czbdo0cnJyHuSzC1Hqbp9s7O5ShQEdR6PTFV4w\npSz5+9anfq1mpvamvb9jVAv/x3s3X/wKOw+b26+Pg4b+khSUla4tIeDWVWgZBli2ybLxCCFEfqpa\ntC8r8vdzc3OjW7duzJo1i9dee42vv/6asLAw5s2bd9fyn3nniRs2bCgwWpDngw8+wM/Pj5CQEKZP\nn84333zDrFmzGDlyJLVr136wD4dWvahPnz4kJyczevRoVq9ezU8//WS6RUVFPfBrW1qREoO0tDSa\nNm3K3LlzcXZ2vmOf0NBQ4uPjiYuLIy4ujj///LPA/mnTprFixQqWLVtGREQEqampDBgwoMAvxciR\nIzlw4ADh4eGsX7+effv2ERYWZtpvNBrp168f6enp7Nixg6VLl7J8+XJefPHFB/nsQpSqO002fqzz\nOJwdilcXubR0azkQGxvtasL4pIsc/nt3kZ63LkplwVpz++n+0LudJAVlSadTGNvX3F66ATIMMmog\nhLAORS2FfXu/JUuWMGDAAL7++mtmzJhBSkoKmzdvxtXV9Y6v2bx5c5o1077keuqpwhPc9Ho9u3bt\nYsKECaxcuZKpU6fy6aefkpCQwCeffPJAMQMkJiZy6dIljEYjr7zyCmFhYQVuX3/9dZFfy9ooalHT\nulvc3Nz44osvCpywjxs3jsTERFatWnXH56SmpqLX65k/fz4jRowA4OLFi/j7+7Nu3TpCQ0OJiYkh\nKCiowBLTO3bsoHPnzpw4cYJ69eqxdu1aBg4cSGxsLDVqaKu5/vTTT0yYMIGEhIRCC1HkH23w8Hiw\nyZWisOjoaECbWCTubnfMFrYeWG1q9w956p7zCixxXHccXm+a++Dk4MIz/V/ByeHOyT/AsbMqEz7Q\nrm0H6NgMPnleO1G1VhX19zU3V+XxV+GSlncybTiM7lN2P4eKelwtTY5r6bC242owGB64LKYoKDg4\nGAcHB7Zv327pUIrlfr8DljqHLbE5BhEREVSrVo0GDRowceJErl69atq3d+9ecnJyCA0NNW3z8/Oj\nUaNGREZGAhAVFYWbm1uBJaY7duyIi4tLgT6NGjUyJQUAffr0wWAwFKhvK4SlxV+/yPZD5q/VLTXZ\n+H6CG/XA3UWrt3wzM53II3evf5mYovLy5+ak4JHq8M5E604KKjIbG4WwfKMGi9ZBVraMGgghKo8D\nBw4QHR3N008/belQKowSSQz69u3LggUL2Lx5M5988gm7d++mR48eZGdrZxBxcXHY2NgUWga7WrVq\nxMXFmfq7dudXAAAgAElEQVTo9fpCr+3j41OgT7VqBWu+e3t7Y2NjY+ojhKVl5WTyR+QijMZcAKpX\nrU0XC042vhc7W7sCC5/tPxXJ1eTCKzBnZWtJQcKtwhBuzvDxVHB1lqTAkgZ0BP2tdXQSU+CPkluz\nTgghrNbRo0eZP38+48aNw9fX946XEYkHUyLlSp988knT46CgIFq1aoW/vz9r1qxhyJAhJfEWDy1v\nGFGUHDmmd3b4QgSXEs8DYGNjSy2XJuzft7/Izy/r46qqKk46TxJvXAZg2bpvaFe3r+l6S1WFL9fU\nIOq4tmy8ToF/9r1AwsV0Ei6WaagPpaL+vnaoV4XFf2lfmMxdmk1N57+xLcO57RX1uFqaHNfSYS3H\n1d/fXy4legjLly/n7bffpn79+ixdurRcHssbN25w5MiRu+6vV69eGUZjVirlSqtXr46fnx+nTmmr\nqvr6+pKbm0tiYmKBfvHx8fj6+pr65L/8KE9CQkKBPvHx8QX2X7t2jdzcXFMfISwpLvkcsYknTO2g\nGiG4OFj3/BZFUQiq2R5uJQKJaVeISzln2r8yqqopKQAY2S2eZnXSyzpMcRfdmiXj6qiNTl1NsSvw\nsxJCiIpo5syZ5OTkcOzYMbp06WLpcCqUUlng7OrVq1y6dInq1asD0Lp1a2xtbdmwYUOByccxMTF0\n7NgRgJCQENLS0oiKijLNM4iMjCQjI8NU+jQkJIR3332Xy5cvm+YZhIeH4+joSOvWre8Zk7VMOKoI\nrG0Sl7W4kZHMzrUrcXbRJu82qNWcgR1HFrnSgaWPq+qczt4T2uSty+kneLTbYP7aZ8f6/XDrIzG0\nG7wyJqBY1RsszdLHtSxMSlT5aoX2ePeZekwLK16FjQdRGY6rJchxLR3WdlwNBoOlQxAW5ubmds/f\nx9vL9ZeVIo0YpKenc/DgQQ4cOIDRaCQ2NpaDBw9y4cIF0tPTefnll4mKiuL8+fP89ddfDB48GF9f\nXx577DEA3N3deeaZZ5gxYwabNm1i//79hIWF0aJFC3r27AlAw4YN6dOnD5MmTSIqKoqdO3fyj3/8\ng4EDB5qGU3r37k1QUBBhYWEcOHCAjRs3MmPGDCZOnFioIpEQZcmoGlkbtRRDlrYMsJuTB6FtnyhX\nJ9AdmvTBycEFgNSMJJZt2s0b35n3t20ML48q/RNOUXzDeoDjrbV1Tl2APTGWjUcIIUT5VKTEIDo6\nmpYtW9K6dWsMBgMzZ86kVatWzJw5ExsbGw4fPsyQIUNo0KAB48aNM1UbcnFxMb3GnDlzeOyxxxgx\nYgSdO3fG3d2dVatWFTjJWLJkCc2bN+fRRx+lb9++tGzZssBiajqdjjVr1uDs7EynTp0YOXIkw4YN\n46OPPirBQyJE8UUf38r5eO3SOQWF/h2eumfZT2vkaO9El+b9ALiR4cq7P/pgyNIWPatdDT74J9ja\nSlJgjdxdFAZ2Mrd/CrdcLEIIIcqvIl1K1LVr1zsuR51n3bp1930NOzs75syZw5w5c+7ax8PDo9Cq\nyrfz8/O763oJQljC7aVJgxv3oJZPXQtG9OCaBASz43AkS9Z3Je2mI3CDGt4ezJ6mnXwK6zWiJyzf\nrE0WjzwE566oPFJdfmZCWCtVVWUEtpIq5hJiZapUJh8LUVlk52SxeudPptKkvl616Nikt4Wjegiq\nwu6jo7iarJUOvpmVyjuTsqjtK3+8rF1tX4XOLcztxTJqIITVsre3x2AwkJuba+lQRBlTVRWDwYC9\nvb2lQ7mjUpl8LERlsfXAaq6nJgBgZ2tP/5CnsLEpn/+sVFXlk6Vw5G8fbG3iyMnNoVurjSiKP9DT\n0uGJIniqN2y7VRl3TSRMHqri6SZJnRDWRqfT4ejoSFZWlmnNp/Loxo0bgDaRVhSdg4MDOp11fjdf\nPs9ghLACZ68cZ/+pHaZ295aD8XIvvEhfefFTOCzbCKDg4uROQM2NBAUcJfrEOVo16IS9rYOlQxT3\n0bI+NHoEYs5pK1T/+hc8M9DCQQkh7khRFBwcyvf/q3l1+K2l2pN4eNaZrghh5W5mprN21zJTu27N\nIJrVbWfBiB5O+C6VOeaPw4AOzvTtoP2HfzMznYOnd1ooMlEciqIwMtTc/nmTtmq1EEIIURSSGAhR\nTKqqEr7nF9JvpgLg7OhKn+Bh5XYS2b4TaoGypC3qw1sTFEIa9zBt2x2zheyc8jvcXZn0ags+VbTH\n11Nh/S7LxiOEEKL8kMRAiGI6ejaakxcOm9qPBj+Ji2P5vL7yzCWVlz6D7BytXacGfDwV7O0UmgS0\nxc1JW7U5w5DG4TNyhlke2NkqPJlvSsjicOuugCGEEMJ6SGIgRDGkpF1n094Vpnbzuu2pWzPIghE9\nuKtJKs/PhhvammxU9YA5081lSW1t7GjbqLup/+5jm8nJlVGD8uCxruYFz05flAXPhBBCFI0kBkIU\nkVE1siZqMVk5mQBUcfOmW8tBFo7qwaTfVJk+B+Kva21nRy0pqO5d8HKoZnXb4+yorSp+42YKR89G\nl3Wo4gHcvuDZovWWi0UIIUT5IYmBEEW0J2YLl66eBUBRdPRrPwp7u/JXUSIzS+Vfc+FkrNa2sYEP\nJ0MD/8JzJOxs7WjbsJupvevYZnKNUne7PBjZC/Kmvew8DGcvy+VEQggh7k0SAyGKIP76RSIOm792\nDQnqRQ1vfwtG9GCyc1T+7//B/hPmba+NhfZN7j5xukVgB5zsnQFISb/OsXN7SztMUQJqVVPo0tLc\nXrLBcrEIIYQoHyQxEOI+srIz+SNykWl14+pVa9M+qJeFoyo+o1Hl9W9gxyHztqnDYGCne1dTsrdz\noHXDrqb2rmObTMdCWLdR+UqXromE5BsyaiCEEOLuJDEQ4j427V1B0o2rANjbOtCv/ShsdDYWjqp4\nVFXlvQWwcY9527gBENa3aCVWW9XvhIO9EwBJN65x6uKR0ghTlLC8Bc9AW/Ds920WDUcIIYSVk8RA\niHuIOb+fI2fNZ9O92gwtd6sbq6rK7GWwMt9J4ZM94Z+PFf01HOwcaVmvo6m994ScYZYHiqIwPN/g\n1i+bITdXRg2EEELcmSQGQtxFcloi4XuWm9qNH2lFUJ3yt+z7N6tgSbi53b8jvDiSYi/I1rJeR3S3\nRkouXTvHlcQLJRmmKCWhbcHLXXuckAR/7bdsPEIIIayXJAZC3EGuMZfVkYvIyjYA4OFalV5tHrdw\nVMW3OFzlm5XmdvfW8N+nQacr/irNrk7uNKzdwtSWUYPywd5O4THzFBGWbrRcLEIIIaybJAZC3EHk\n4fVcSdTqeeoUHQM7jMbBztHCURXP4nCV2UvN7XZB8M5EsLEpflKQp02DLqbHJ2IPcCMj+WFCFGXk\n8W5aWVqAAyfhZKxcTiSEEKIwSQyEuM35uFPsOrbZ1O7UrC/Vq9a2YETFt2BtwaSgeT34aIr27fHD\nqOblRy2fAEBb8G3/qciHej1RNvRVFHq0Nrdl1EAIIcSdSGIgRD4ZmWn8GbUEFe0bVf9q9WjbqJtl\ngyqm71erfPaLud28Hsz9Fzg5PFxSkKdVffOowcHTO8nOySqR1xWla0S+Scjrd0npUiGEEIVJYiDE\nLUbVyNqopaTdTAHAycGFfiGj0Cnl45+Jqqp8vVLly9/M21o11JICZ8eSSQoAAv2C8HDxAsCQlSEL\nnpUTTetC4zra46xsWCFTRIQQQtymfJzxCFEGoo5u4szlGFO7b7sRuDq5WzCiolNVlS9XUGCicdvG\nMGdaySYFoM25aFW/k6kdfWIbRtVYou8hSp6iKAzvaW4v3ww5OTJqIIQQwkwSAyGAs1dOEHl4vand\ntmE36tZsbMGIik5VVeb+Aj+sNm8LaQqznwfHErp86HZN67bD3tYBgOupCZy7crJU3keUrF5SulQI\nIcQ9SGIgKr2U9OusiVxkmldQyyeALs37WTiqosnNVXl/ASxaZ97WqTn8bwo42JdOUgDagmdN67Yz\ntfedlOtSyoPbS5cu22S5WIQQQlgfSQxEpZaTm82qiAXczMoAwMXJnYEdxpgW8rJmhkyVl7+AFVvN\n27q2glnPPXz1oaJoXb8zyq35F2evnOBaSlypv6d4eLeXLj1xXi4nEkIIoZHEQFRqm/f+Ttx1bQVf\nnaJjcMcwXMrBvIKUNJXJH8P2A+Ztj7aHD/4BdralnxQAeLh6EVgzyNTed2J7mbyveDj6Kgo98y3g\nLaMGQggh8khiICqtI2f2cPDvKFO7W8tB1NTXsWBERXP5qsoz78Hh0+ZtY/rCm8+CbRklBXlaN+hs\nenzkXDQZmWll+v7iwdxeujQpVUYNhBBCSGIgKqmEpEuERy83tRv5tyxQacdanYxVeeZ9OH/rqh1F\ngRdGwvPDFHS6sk0KAPz0AVSr4gdAbm4OB09H3ecZwho0CZDSpUIIIQqTxEBUOjcz0/k9Yj65uTkA\nVPWoRu+2w1CUsj+xLo49MSoTP4RryVrbzhbenQQjQy0Xt6IoBUYNDp6KxGjMtVg8omhuL1366xYp\nXSqEEEISA1HJ5ObmsDLiR1LSEgGwt3VgSKensbdzsHBkd6eqKj9vUpnyMaTf1La5OsNnL0BosOWT\nmQa1m+Ps4ArAjZsp/H35mIUjEkUhpUuFEELcThIDUWmoqsr6Pb9wIeGMaVvf9iPxcvexYFT3lpWt\n8vYP8NFPYLy1hpi+CnzzCrRuaPmkAMDWxo5m+UqX7j8VacFoRFFJ6VIhhBC3k8RAVBq7jm3m6Nlo\nU7tL8/7Ur9XUghHd29UklUkfwh8R5m2N68CP/4FAP+tICvI0C2xvKl16Pu4k11OvWjgiURS3ly49\nGSuXEwkhRGUmiYGoFE7EHmT7oT9N7SYBwQQ36m7BiO7t8N8qYW/DEfPgBv06aCMFPlWsKykA8HDx\nIqBGI1P7wGkZNSgP9FUUerQ2t5dutFwsQgghLE8SA1HhXUmMZU3UYlO7lk9derd53GonG6/aro0U\n5E0y1um0ykNvPFM2C5c9qJb1OpgeHzm7h+ycLAtGI4rq9tKlKWkyaiCEEJWVJAaiQktNT+K3bd+Z\nKhBVcdMzuNPT2NjYWjiywgxZCl+vrc7bP0C2Fi7uLtok45GhitUmMnn8fevj6VoVgMysm8Scl9ms\n5UHTutDQX3uclQ2/S+lSIYSotCQxEBVWZraB37Z9R4ZBW3TL0d6ZoV2ewcnB2cKRFXbsrMp/F9Qh\n4qiHaVu9WrDwdQhubN0JQR6doqNFvlGD/ad2oKry7bO1UxSF4flGDX7ZDLm58nMTQojKSBIDUSHl\n5GazcvuPXE2+AoBOZ8OQzk/j5a63cGQFGY0qC9ZqKxnHJ9ubtvcNge9ehRr68pEU5GlSJ9g0GpOQ\ndIkribEWjkgURe9g8HTTHsdfh60HLBuPEEIIy5DEQFQ4RmMuqyN/4nz8KdO23m2foJZPXQtGVdi1\nZJWps+GzXyDn1ppgjvZG3poAb01QcHIoX0kBgJODM438W5naB6R0abkgpUuFEEKAJAaigjGqRtbt\n/plTFw+btnVs+ihNA4ItGFVh2w6ojJoJu4+atwX4Gng77Cx9Q8pfQpBfi0Dz5UTHLxwgIzPNgtGI\nonqiuzbRHWDfcTh1QS4nEkKIykYSA1FhqKrKln0rC6xV0KZBV0KCet3jWWXreqrKq1+pvDgXkm5o\n2xQFxvaD/4w8RzXPbMsGWAKqV62Fr1ctQFtp+siZPRaOSBSFTxWF7vlKl/682XKxCCGEsAxJDESF\nEXkknH0nzauBNQ0IplvLgVZRzUdVVdZEqgx7DTbsNm/39oTPX4QpTyjY2lguvpKWfxLygVORGFWj\nBaMRRTWip/nx2p2Qmi6jBkIIUZlIYiAqhOjjW4k8Em5qN6jVnN7Bw6wiKbh8VWXqJ/DGt5Cabt7e\nrwMsfav8VB0qjoa1W+Jor1V/Skm/zrkrJywckSiK5vWgfm3tcWaWlC4VQojKRhIDUe4dPrObLftX\nmdp1qjekf8godIplf71zclQWh6sMfx125ZtLUN0b5r4Abz6r4OFa8ZICADtbO5oEtDW195/aYcFo\nRFEpisLwfKMGy7dI6VIhhKhMJDEQ5dqhv3exftfPpnZNfR0Gdxpr8QXMIg+rjHoDZi8FQ6a2TVFg\nZG9tlCCkScVMCPJrERhienz28nFS0q5bMBpRVH3agYer9vjKNSldKoQQlYkkBqLc2ncygvW7f0ZF\n+0bTp0pNHu/yDHa29vd5Zuk5e1nl+dkq02bD2cvm7QE14fvX4IURCs6OFT8pAG2V6TrVGwCgonLo\n7ygLRySKwsFeYWg3c3vpRouFIoQQooxJYiDKpd0xW9i0d4WpXa2KH8O6T8TB3ski8aSkqXz0k8qI\n12GnuVIqzo7w3OOwaCY0CagcCUF+zfONGhw6s4tcY64FoxFFNaw72NyaDL//BBw/L5cTCSFEZSCJ\ngShXVFVlx+H1bD2w2rSthvcjPNnjHzg7uJZ5PDcztZWLh/4bft4ExlvFdxQFhnSFFR/A0/0V7Gwr\nX1IAULdGY1ydPADIMKRx+uIRC0ckikJfRaGXeYoISzZYLhYhhBBlRxIDUW6oqsq2g2sKVB+q5VOX\nYd0m4ljGIwV5CcHgGdrKxfmrDbVqqI0QvDZWwcu9ciYEeXQ6mwKLyx08vdOC0YjiGJlv+Y/wXZCY\nIqMGQghR0UliIMoFo2pk094V7I7ZYtpWp3oDHu/6LPZ2DmUWx81MlYXrzAlB3iJlAH4+8NEU+Opl\nqF+7cicE+TWr2w7lVoWo8/GnSLpx1cIRiaIIClBoGqg9zsnVKhQJIYSo2CQxEFYvJzebNZE/FSh5\nWc+vCUM6jyuzicY30rURgiH/B3N/LpgQ+FaFf4fBz+9At1aKVaydYE3cXaoQUL2hqX3o710WjEYU\nx6hQ8+Nft0BWtowaCCFERWbZmo5C3EeGIY0V23/g8rVzpm0Na7egX8gobHSlv1TwhXiVpRvhjwi4\nmVlwXzUvGD8ABnai0s4hKKrmge35+/IxQFt3omPTPtja2Fk4KnE/3Vtpv+fx17VkeF0UDOps6aiE\nEEKUliKNGGzfvp3Bgwfj5+eHTqdjwYIFhfq88cYb1KxZE2dnZ7p3786xY8cK7M/KymLq1Kno9Xpc\nXV0ZPHgwly5dKtAnOTmZMWPG4OnpiaenJ2FhYaSkpBToc+HCBQYOHIirqyt6vZ5p06aRk5NT3M8t\nyoHE1HgWbZhbICloEdiB/h2eKtWkQFVVomNUXpir8vir2qTi/ElBNS94ZQz89j4M7VZ5JxYXR50a\njXBz9gTgZmY6p2QScrlgY6PwZL4Fz5Zu1P59CCGEqJiKlBikpaXRtGlT5s6di7Ozc6H9H374IbNn\nz+aLL74gOjoaHx8fQkNDSU83z8icNm0aK1asYNmyZURERJCamsqAAQMK/JEZOXIkBw4cIDw8nPXr\n17Nv3z7CwsJM+41GI/369SM9PZ0dO3awdOlSli9fzosvvvgwx0BYofNxp/hpw2ekpCUCoKDQveUg\nerUZWmorGt9IV/l1i8roN+GfH8H2A5D/HCigJrz2tJYQPN5dwd5OEoKi0ik6mtVtb2ofPB1pwWhE\ncQzpAo63pvGcugB7j1s2HiGEEKWnSJcS9e3bl759+wIwduzYQvvnzJnDv//9b4YMGQLA/Pnz8fHx\nYfHixUyYMIHU1FS+//575s+fT48ePQBYuHAh/v7+bNy4kdDQUGJiYli/fj2RkZEEB2tVTObNm0fn\nzp05deoU9erVY/369aZ+NWrUAGDWrFlMmDCBd999F1fXsi9XKUre4TO7Cd/9C0ZVq/1pZ2PPgI6j\nCawZVOLvZTSq7DsBqyJgUzRkZRfuE9IUnuoNwY2R+QMPoWlAMJFHwlFVIxcSzpCYGk9V92qWDkvc\nh7uLwoCOKss3a+3FG6BNI8vGJIQQonQ89FevZ8+eJS4ujtBQ8yw1R0dHunTpQmSk9q1gdHQ0OTk5\nBfr4+fnRqFEjU5+oqCjc3Nxo3978rWLHjh1xcXEp0KdRo0ampACgT58+GAwG9u7d+7AfRViY0ZjL\n1gOrWbdrmSkpcHFyZ0SvySWeFMQlqnz3h7b+wD8/grU7CyYFDvYwtJs2oXjuvxTaBcmk4ofl5uxR\n4Od48LSshFxejMhXujTioDb3RgghRMXz0JOP4+LiUBSFatUKfvNXrVo1Ll++DEB8fDw2NjZUrVq1\nUJ+4uDjT6+j1+kKv7+PjU6DP7e/j7e2NjY2Nqc/dREdHF++DifsqyWNqyM5g//ktXE8z/xzdnKrQ\n1KcrF8/Ec/FM/EO/x7VUW6JPurP7hBunr9x53YPa+ky6Nk0mpHEKro5GEi9D4uWHfutiqci/qw7Z\nnmSkZwAQdWgzLjnVsNGVTQ2Einxcy0JgNT8OntVGZT+en8RobfBXjmspkeNaOuS4lg45riWvXr16\nFnlfqUokLO7ajcvsj91CVrbBtE3v7kcr/x4PXbkmMdWWPSfd2H3C/a7JgIuDkZDGKXRpkswj1TLv\n2EeUDG+3mjjZu3IzK43snCyuJJ/DzyvQ0mGJIujdOsmUGGw74sHjHa/i5GC0cFRCCCFK0kMnBr6+\nvqiqSnx8PH5+fqbt8fHx+Pr6mvrk5uaSmJhYYNQgPj6eLl26mPpcvVp44aOEhIQCr5N3WVGea9eu\nkZuba+pzN23atHmwDygKyftm4GGPqVE1EnV0E0dOb8fWXoetvTMKCh2a9qF9UM8HmmScm6ty+G/Y\ncRgiD8PJWPM+l3zz5m1soG0jGNgRurUCezt3oNZDfZ6HVVLH1doZXVLZdvBPADKUq7RpM6JU36+y\nHNfS1rq1ytoDcPbWCNrWw5482ua6HNcSJr+vpUOOa+mQ41p6bq/KWVYeeo5BnTp18PX1ZcOGDaZt\nBoOB7du307FjRwBat26Nra1tgT4XL14kJibG1CckJIS0tDSioszXHUdGRpKRkUGHDh1MfWJiYkyX\nKAGEh4fj6OhI69atH/ajiDKUbrjB8r++Ycfhdai35hM4O7oyrPtEOjQJLVZScC1ZZU2kyqtfqYRO\ngwkfwI9rCiYFoCUD7ZvAf8bB+tnw2QsKvdtJdaGy1iQg2PTzvXTtHFeTr1g4IlEUiqIwMt+CZ+v3\nepGTa7l4hBBClLwijRikp6dz+vRpVFXFaDQSGxvLwYMH8fLyolatWkyfPp3333+fBg0aUK9ePd55\n5x3c3NwYOXIkAO7u7jzzzDPMmDEDvV6Pl5cXL774Ii1atKBnT61IdsOGDenTpw+TJk1i3rx5qKrK\nP/7xDwYOHGi6zqp3794EBQURFhbG//73P65du8aMGTOYOHGiVCQqR85eOcG6XctIu2nOhmv5BDCg\nwxhcndzv+/yEJJW9x2HfSdh3HGLvMf3A1karoNKrLXRrCR6ukgRYmoujG/X8mnLiwkEADp7eSa82\nQy0clSiK/h3gqxVwPRUSb9iy67g77dtZOiohhBAlpUiJQXR0NN27dzdVZZk5cyYzZ85k7NixfP/9\n98yYMQODwcCUKVNISkqiXbt2hIeH4+LiYnqNOXPmYGdnx4gRI7h58ya9evVi4cKFBSq9LFmyhKlT\np/Loo48CMHjwYD777DPTfp1Ox5o1a5g8eTKdOnXCycmJ0aNHM2vWrBI5GKJ0ZWbdZMv+VRw+s7vA\n9vaNe9KxaR90d1i0LDdX5e9LcOSMdjtwCi7cZx6yTxXo2Ey7tW0Ezo6SDFib5oHtTYnBsXN76dpi\nAHa29haOStyPvZ3Ckz1VvlqhtddGV+X5MFUqdgkhRAVRpMSga9euGI33nmT2+uuv8/rrr991v52d\nHXPmzGHOnDl37ePh4XHHVZXz8/PzY9WqVfcOWFidM5djCN/9CzfyjRI4ObjQr/0oAmo0BLQ1BS4m\nwIlYLQk4ehaOn4fMrHu/tr0dNK0LHZpqt7o1Zb0Ba1erWiBV3LxJunGNzGwDx2MP0DQg2NJhiSJ4\nojv8+CekZ0DsVQd2HdUu0RNCCFH+SVUiUapuZmawZf9Kjp4tWMqslr4NtasNIjrGmSUbVE5fgNOX\nwFCEokD2dtAsEFo3gNYNIagOMk+gnMlbCXnrgdWAdjmRJAblg4erwqBOKj/8obUXrJPEQAghKgpJ\nDESpUFWVY+cOs2L7Ji5ftSMptRVJN6pwI8MH1PpkZbsV+bWqeUFQgJYANKsLjSURqBCa1GlLxKF1\n5BpzuJIYS0LSJXyq1LR0WKIInuoN81eDUYU9x+DEeZUG/vJvUgghyjtJDMQDy8pWuZoM8dfhSiJc\nugqXr8GZSxkcOXOVaykeqOrjpv6O9s64u3iiUwrPJcjj5Q6BtbQkIO/m7SknHBWRs6Mr9fyacDz2\nAKCthBza9vH7PEtYgxp6heD6qUSd0IoFLFwP70y0cFBCCCEemiQGopCbmSqJKbcqj6RAYqr2+Foy\nXE2GmNN1SEqzxXjb+bpRzSUtI4WMrAxQzeVGdTob3F2q4GhnXmDM3g4eqQ6BflCvFtS7de/lLklA\nZdI8MMSUGORNQra3c7BwVKIo+ra9bkoMNuyG54aqVPeWf79CCFGeSWJQQamqys1MSLsJN9IhNQNu\n3Lqlpmv3yWmQfANS0iHlhtZOSb//df7pGdqJW96CYSoqGYY00m6mmtYkAFAUqO5tR/O63gTUtMXf\nF/x94RFf8K0KOp2cRFR2tXzq4uWm5/qNq2TlZHI8dj/N6ra3dFiiCOr4GmhUK4PYRBeMRliyEV4o\n3bXqhBBClDJJDCxMVVVycrXKO5nZYMjSTsxv5t3fuhmytPuMW+30m7fuDZgSgIyb2n26ATIMcJ9C\nUg9N0ak42aeSazyPu8s13F1S8HBNwc0llSZ1vBnUqSc1vO+9IrWo3BRFoVlgCH/t1yqNHTwdJYlB\nOdI/OJEv1+oB+H0rPDtQxd1FEn4hhCivKk1ikJOjnYDnGrVbTu6tW06+x7du2Tnm++wcyL7VL/vW\ntqxsbX9WdsFtWXn32be23WpnZkNWltbOzHeflwiU9gl8cdnbadf6e7lDVY98j921NQLiL53BkHOK\nTDq20V4AACAASURBVNsTpKYnFnhuFTc93VsOIqBGIykZKoqkSZ02bD/0J7m5OcRdv0D89YtU8/Kz\ndFiiCJo+kk7dmvD3Je0Lil//gnH9LR2VEEKIB1VpEoOQSjgxztEBXBzB3QVcncHNGdydwc1Fe+zh\nAp6u4Ol26/7WYyeHO68DkJubw9Fz0ew5+is3s9JwzruWCHCwc6RDk960rNcRG5tK82slSoCTgwsN\najXj2Ll9gFa6tHfwMAtHJYpCUWD0o/Dmd1p72UZ4qrcqVcOEEKKckjM4K2BjAw524Gh/6+ag3TvY\nayfpTrfaTg7g7ADOjuabk7127+IErk5aIuDqrPWztS2ZP86GrJscObObvSe2k5qRxM2sDNM+R3tn\nWjfoQqv6nXC0d7rHqwhxd83qhpgSg5jz++nWcpBMQi4n+rSDL3+DhCStWMHaKBjc2dJRCSGEeBCV\nJjGwsQEbHdjaaDcbHdjamtsFbvm229lqN1sbsM+7tzNvt7MFu1vb7O20Pnn3drbayb3DrX0Odlrb\nPt92B7uSO4EvaQlJlzlwagfHzu0jO7fg8sN2tvZ0ataXVvU74WDnaKEIRUXhp69DVfdqJKbGk5WT\nScz5fTQPDLF0WKII7GwVhvdS+ewXrb1wLQzsqEpxASGEKIcqTWIQ9Y38kSqKXGMuJy8c4sCpSC5e\nPVNov7ODK7U9gvCv2oj2QXLiJkqGoig0D2zP5n0rAThweifN6raXeSrlxNCu8P1qrSjC+TjYFA2h\nspC1EEKUO5UmMRB3p6oqVxJjiTm/j+OxB8gwpBXqo/esTst6HWn0SCsOHThsgSjF/2/v3uObqtK9\ngf920iRNeknvLb0ILRRKC7RQLoUqFxGBAUSODIIOiM6ZQR25OeLoMHNeeZVBz3FeRo+gjnhBnBEc\nFQVEoVwtlIKFtkBboNCWXiDp/X5P1vtHICXCKG1TdtP+vp/P/mSvtVebJ09C6JOVvXZPF9lvJA6n\nfwOTqRXFFUUwlBegj/ddcodFt8FVJ2HeZIEPd1naH+wCJo/krAERkaNhYdCLlVeXIOvyKWRdPoWK\nmtKbjiskBQaGDENMeDyCfUP56S11Ka1Gh4iQGGTkpQCwzBqwMHAcj0wBtiZYVie6WAgkpgMThssd\nFRERtQcLg15ECIGSyqu4dCUTFwvPwlBecMtxLlp3RPePw7D+cXDT6e9wlNSbRQ8Yay0Mzl9Ow6Th\nD/Ckdgfh4SbhoUkCn3xnaX+wCxgfI/iBAhGRA2Fh0MO1mlqQb7yES1cykVOUier6iluOUztpMDBk\nGCL7jUCI/wAoJMUdjpQICPTpC1+PPiipvIoWUzMy805ixMC75Q6LbtOvpgKf7bdcvyUzF0jOAMYO\nkTsqIiK6XSwMehiz2QRjRREKii9ZNuOlm1YUuk6hUKJ/4GAM7jsCYYGRUDmp7nC0RLYsJyGPxb6U\nLwEAadlJGB4ez0+dHYS3XsLs8QL/2m9pb9oJxEVx1oCIyFGwMHBwraYWGMuLUFhyCQXFOSgqyUVz\na9O/Ha9ROSO0TwTCAgcjLDASWo3u344lkkNkv1h8n/YNmlubUFZtREHxJdzlP0DusOg2PTYd2H7I\ncnX409nAqfNAbITcURER0e1gYeBATKZWlFYZYCgvgKG8EIbyApRWXoVZmH/y57zcfBEWFIn+gZEI\n8g2FUqG8QxETtZ9G5YzB/UYg/eIxAJaTkFkYOA5/Lwkz7xb46rClvWknCwMiIkfBwqAbMgszqmrL\nUFZdjNJKA8qqDSitMqKsygiTufVnf95N54EQv/7WzdPN5w5ETWQ/MQPGWQuD7ILTqGuohovWXeao\n6HYt/gWwIxEwm4GULOD0RYFhA/h1IiKi7o6FgUzMwoyaukpU1paisrbs2laKypoylNUUw2T6+QLg\nOk83XwT69EWIX3/c5TcA7i6e/E4vOTQ/z0AE+fRDUWkezMKM0zknMDbqPrnDotsU5CthWpzA7iRL\n+/1dwBsr5I2JiIh+HguDLmAytaK+qRZ1jTWoqa9CTX3lte2G/YYqmM2mdv9uvYsXArxCEOAdAn/P\nYAR4BUPD5RypB4oJH4ei0jwAQPrFYxgTeS9Xy3Igj88Avj0GCAEknQay8gQG9+MHFkRE3RkLg58h\nhECLqRlNzQ1oaKpDQ1O95bb5hv2mWtQ11qKusRp1DTVobK7v9P3qnF3h4x4Ab70/fPRtt1qNix0e\nFVH3NzBkGA6c+hoNTXWoqa9EzpUsDAiKkjssuk39+ki4b5RAwglL+4NdwP88I29MRET003pNYVBc\nUYTm1iY0tzSjpbUJza1NaGpusNy2NKK55fptAxqbLVtTcwMaWxo69Mn+7dA5u8LD1Qcert43bD7w\ndPeBTuPaJfdJ5CiclCoMDRuNE1kHAViWLmVh4FgenwFrYXDoFHCxUGBAMGcNiIi6q15TGGz+7v/d\nsfuSJAW0Gh1cnd3honWHm87j2qaHu84Drjo93LQeUKs0dywmIkcUM2Acfsg6BAGB3KvnUFFTypPp\nHUh4iITxwwW+T7W03/kKeJ2zBkRE3VavKQw6Q6l0grNaB2e1FjqNC7QaF2jV126vbTpnN7g4u8FF\n6watxoXfhSayA72rF0IDI5BzJQsAcPpSMibEzJQ5KmqPJbNhLQwOnwIycgSiwjhrQETUHfWawsDX\nIxAqJzXUThqoVWqolBpo1M5QO2mgUWuhUTlDrbK0ndVaaNRaazHgpOQVgYnkEjNgrLUwOHPpOOKH\nTuW/SQcy8C4JU0a3nWuwcTuw4ffyxkRERLfWawqDxdP5PxGRIwoNHAx3F09U11Wgobke5/PTERU6\nUu6wqB2WPAjsT7Fc1+BEBnDynEBsBGcNiIi6G37fhYi6NYWkQHT/sdZ22rULn5Hj6BsgYca4tvaG\nLy0rvhERUffCwoCIur2h/UdDoVACAK6U5qG4okjmiKi9fvMA4GR5CnHmInD0tLzxEBHRzVgYEFG3\n5+LshoHBQ63t1OyjMkZDHdHHR8J/TGxrv70dMJs5a0BE1J2wMCAihzB8YLx1PyPvJOqbamWMhjri\niZmARm3Zv5BvOe+AiIi6DxYGROQQgnxC4e8ZDAAwmVpx+uJxmSOi9vLWS5h/X1v73a8Ak4mzBkRE\n3QULAyJyCJIkIXbQPdZ2WvZRmLroquTUdRZNB1y0lv3LBuCbJHnjISKiNiwMiMhhDLorGjpnVwBA\nTUMVLhTwDFZH4+4iYeG0tvamnUBzC2cNiIi6AxYGROQwnJQqxAxoW/fy1PlEGaOhjlowBfB0s+xf\nLQW++l7eeIiIyIKFARE5lJjwcVAqLNdmvFJ2GVdKL8scEbWXzlnC4hlt7U07gLoGzhoQEcmNhQER\nORQXZzdE9I2xtk9d4KyBI5o7CQjwtuxX1AAf7JI3HiIiYmFARA4odtB46/75/HTU1FfKGA11hFol\n4Zm5be1PE4ArJZw1ICKSEwsDInI4/p5BCPYNAwCYhRlp2VzaxhHdPxoYYnka0dIKvPWFvPEQEfV2\nLAyIyCHduHRp+sVjaGltkTEa6ghJkrByfls74QRw+iJnDYiI5MLCgIgc0oCgKLi7eAIAGprrkXX5\nlMwRUUcMGyDhvlFt7fVbASFYHBARyYGFARE5JIVCieHh8db2qQuJ/IPSQS2dC6gsC03hbI5l5oCI\niO48FgZE5LCGho2BSqkGAJRUXkW+8aLMEVFHBPpKmD+lrf2/nwNNzSzyiIjuNBYGROSwtBodokJH\nWtsnuXSpw3piRttFzwxlllWKiIjozmJhQEQObcSgu637OUWZKKsyyhgNdZSrTsKSB9vaH34DlFdz\n1oCI6E5iYUBEDs3b3R/9AyMBAAICx7MOyBwRddSD44GwIMt+fSPwznZ54yEi6m1YGBCRwxsTOdm6\nn5l3ClW15TJGQx2lVEpYPq+t/XUicCGfswZERHcKCwMicnhBvv0Q4tcfACCEGT+cOyRvQNRh44ZK\niBti2TebgVc/AcxmFgdERHeCXQqDNWvWQKFQ2GyBgYE2Y1566SUEBQVBp9Nh0qRJyMzMtDne3NyM\npUuXwtfXF66urpg9ezaKiopsxlRWVmLhwoXw8PCAh4cHFi1ahKqqKns8BCJycHFR91n3T+ccR11D\ntYzRUGc89wjgpLTsn7kI7DwqbzxERL2F3WYMIiIiYDQaYTAYYDAYcObMGeux1157DevXr8eGDRuQ\nkpICPz8/TJkyBXV1ddYxy5cvx/bt27Ft2zYcOXIE1dXVmDlzps265AsWLEBaWhr27t2LPXv24NSp\nU1i0aJG9HgIRObC+/uEI8AoBAJhMrUg5/73MEVFH9Q2QsHB6W/vNz4DKGs4aEBF1NbsVBk5OTvD1\n9YWfnx/8/Pzg7e1tPfbGG2/gxRdfxIMPPojIyEhs3rwZNTU1+Oc//wkAqK6uxgcffIDXX38d9957\nL2JiYrBlyxacPn0a+/btAwBkZWVhz549eO+99zB69GiMGTMG7777Lnbu3Ins7Gx7PQwiclCSJNmc\na5CWnYSGpnoZI6LO+PVMINDHsl9dZ7m2ARERdS27FQY5OTkICgpCWFgYFixYgNzcXABAbm4uDAYD\npkxpu3qNs7Mzxo8fj6SkJABASkoKWltbbcYEBwdj8ODB1jHJyclwc3NDXFycdUx8fDxcXFysY4io\ndxsQHAVvvT8AoLm1CanZ/A6Ko9KoJTz3SFt7RyKQns1ZAyKirmSXwiAuLg4fffQR9uzZg02bNsFg\nMCA+Ph4VFRUwGAyQJAn+/v42P+Pv7w+DwQAAMBqNUCqVNrMMPx5jMBjg6+t70337+flZxxBR76aQ\nFBgz+F5r++T579FqapExIuqMe2IkTBjR1n51C2AysTggIuoqTvb4JVOnTrVpx8XFITQ0FJs3b8aY\nMWPscRedlpKSIncIPQ5z2jWY184xCzNEiwINzbWor6tHvu48wnyHMK9dpKvzel+kE/Yn90dTq4T0\nC8C694yYNrKiS++zO+DrtWswr12DebW/8PBwWe63S5Yr1el0iIqKQnZ2NgICAiCEgNFoezVSo9GI\ngIAAAEBAQABMJhPKysp+ckxJSclN91VcXGwdQ0SkkBTo7zfM2s4pPgOT2SRjRNQZPu6teHBc23v/\nl0m+KK+xy2daRET0I13y7trY2Ihz585h8uTJCA0NRUBAABISEhAbG2s9npiYiL/+9a8AgNjYWDg5\nOSEhIQHz588HABQWFiIrKwvx8fEAgLFjx6K2thbJycnW8wySkpJQX1+PcePG/WxMI0eO7IqH2itd\n/2SAObUv5tV+YkzRMO64hLrGGtTX1aOoIhv/MfVXcofVo9zJ12t0jEDmVSDn2grWB85F49WnpC6/\nXznwfaBrMK9dg3ntOnItx2+XGYNVq1bh+++/R15eHo4fP465c+eivr7eupToihUr8Nprr2H79u04\ne/YsFi9eDDc3NyxYsAAA4O7ujl//+td4/vnnsX//fqSmpmLRokWIiYnB5MmWVUYiIiIwdepULFmy\nBMnJyTh27BiefPJJzJo1S7bpFiLqnpyUKoyMmGBtXyo+DTNnDRyWyknCH26o6/b/ABw7y3MNiIjs\nzS6FQWFhIR555BFERERg7ty50Gq1SE5ORkiIZU3x559/HitXrsQzzzyD0aNHw2g0Yu/evXBxcbH+\njjfeeANz5szB/Pnzcc8998Dd3R07duyAJLV9KvTpp58iOjoa06ZNw/Tp0zF8+HB8/PHH9ngIRNTD\nxAwYB2e1DgBQ31SDrMupMkdEnTFikIRf3DA5vO5joL6RxQERkT1J4sYriPUwN07D6PV6GSPpWTh1\n2DWYV/s7emYP9h3/CgAQ4BeEX8/4A5yUKpmj6hnkeL2WVwv8crXlugYAMGcC8MfHetZXivg+0DWY\n167BvHYduf6G7ZKTj4mIuoORg8ZD5aQBAFTXVSA1m9c8cWRe7hJWPdrW3n4YOJHZYz/bIiK641gY\nEFGPpVFrEe4/3NpOztjHqyE7uKljYHNtg5c/BOoaWBwQEdkDCwMi6tH6ekdAp3EDADQ21+NE1gGZ\nI6LOkCQJLy4E3K+domYoA974TN6YiIh6ChYGRNSjKRRKDOrT9v3XkxcSUV3X8y+Q1ZN5621XKdp+\nGDiewVkDIqLOYmFARD1eH30oArwsq6SZTK04cvo7mSOizpoyGpgU29bmV4qIiDqPhQER9XiSJGHi\n8FnWdmbeSRRXFMkYEXWWJEl4YSGgd7W0jeXA+m3yxkRE5OhYGBBRrxDi1x/9AyMBAAICh9O+kTki\n6iwvdwnP37BK0dff88JnRESdwcKAiHqN8TEzIEmWt708w3nkGS7IHBF11o+/UvTKh0B1HYsDIqKO\nYGFARL2Gjz4AQ8NGWduHU3fCLMwyRkSddf0rRR6WhadQXAH83w+BHnztTiKiLsPCgIh6lXFDplqv\nflxceQVZeadkjog6y8tdwurH2tqHTwGf7ZcvHiIiR8XCgIh6FTedHiMHjbe2E9N3o6m5QcaIyB4m\njpAwb3Jb+43PgKw8zhoQEbUHCwMi6nVGR94LnbNlOZuahiocTtslc0RkD8vnARF9LfstrcAf3+ES\npkRE7cHCgIh6HY3KGZNHzLG20y8lI994UcaIyB7UKgl/eRLQOVvahcXA2s0834CI6HaxMCCiXmnQ\nXdEYEBRlbe858RlaWptljIjsIcTf9nyDhBPA14nyxUNE5EhYGBBRryRJEu4b+RA0KsvHy5W1ZThy\nhldE7gnuHyNhzoS29v/8A7hUyFkDIqKfw8KAiHotN50ek0bMtrZPnk/E1bJ8GSMie/n9AqB/kGW/\nuQV48R2goYnFARHRT2FhQES92pDQUejrHw4AEMKMb49vhcnUKnNU1FkatYR1TwHOGks79wrwMq9v\nQET0k1gYEFGvJkkSpo6eB5VSDQAoqzIiOZOL4PcEoYES/vCrtnbCCeC9HfLFQ0TU3bEwIKJeT+/q\nhXuif2FtJ2fuR0nlVRkjInuZGS/hlzdc3+C9r4G9xzlrQER0KywMiIgADB8Yj0CffgAAs9mE745v\ng9lskjcosovfzwfGtC1AhTUfAGdzWBwQEf0YCwMiIgAKSYFpo+dBqXACABjKC5B4+luZoyJ7UCol\nrHsS6NfH0m5uAX7/JmAoY3FARHQjFgZERNd46/0xbsj91vaJrIPIupwqY0RkL24uEtYvB/SWC16j\nvBr4/f8C9Y0sDoiIrmNhQER0g9GRk9A/MNLa/u74NhRXFMkYEdlLsJ+E//4d4KS0tC/kA3/6O2A2\nszggIgJYGBAR2VBICswY+wi83HwBAK2mFmxP/BD1TbUyR0b2MGKQhBcXtbUT04A3/yVfPERE3QkL\nAyKiH9GotXhw/ONQO1kWwa+uq8DOI1t4MnIP8cA9EhZOb2v/Yw+waQdnDYiIWBgQEd2Ct7s/Zox7\n1NrOL76IQ2m7ZIyI7OmZh4AJI9ra734FbN7N4oCIejcWBkRE/8aAoCjED51mbZ88/z0yclNkjIjs\nRaGQ8JclwOgbljF963Pgn3tZHBBR78XCgIjoJ8RFTUZ48FBre88P/8LVsgIZIyJ7Uask/PUZIDai\nrW/9VuCz/SwOiKh3YmFARPQTFJICv4hbAG+9PwDAZGrF54f+DmN5ocyRkT04ayzLmMYMbOv7n38A\n2w+zOCCi3oeFARHRz1CrNJhzz+PQqLUAgMbmemw7+A5nDnoIrUbCGyuAoQPa+tZ9DOw8wuKAiHoX\nFgZERLfB080Xv5y4xFocNDU34LOD76CoJE/ewMgudM4S3lwBRIZa2kIAL38IfHGQxQER9R4sDIiI\nblMf7xA8fO9T0Kp1AIDmlkZ8fujvKCzJkTkysgdXnYT/fRYYeJelLQTw6hbgb9sEL4JGRL0CCwMi\nonbw9wzCvHufgk7jCgBobm3C5wffQ77xosyRkT24u0jY+BwwuF9b3z/2AM9vBBqaWBwQUc/GwoCI\nqJ38PAPx8OSnoHO2FActpmZ8cXgTLhsuyBwZ2YPeVcK7f7C9zsHhU8CS14DSShYHRNRzsTAgIuoA\nH30AFkz+HVy1egBAq6kFXxx+H6nZRyEE/3h0dFqNhP9+Gnh0altfVh6w+BUgu4DPLxH1TCwMiIg6\nyMvdD/MnPwU3nQcAwGRuxb6UL7Er6RM0tzTJHB11lkIhYcXDEl5YCCiu/W9pLAf+cx1wJJ3FARH1\nPCwMiIg6wdPNFwsm/w5+nkHWvnP5afh4z3oUV1yRMTKyl4cmSfjbCkDnbGnXNwIr3wD++x8CjTzv\ngIh6EBYGRESdpHf1wqNTliJ6wFhrX0VNCT5JeAOnLx3nV4t6gLFDJLz/R8Dfq63vX/uBX60BMnP5\n/BJRz8DCgIjIDpyUKtw/ai5mjH0UKic1AMtVkvec+AzfJm/lV4t6gAHBEj7+L+CemLa+ywbgib8A\n7+8UMJlYIBCRY2NhQERkR5H9RmDh1BXw1vtb+zLyUrDpm1eRkZsCszDLGB11lpe7hL8uBf70OKDV\nWPpMJuCd7cBvXgUKjCwOiMhxsTAgIrIzb3d/LLx/BYaEjrL21TVUY3fyp/g04S1cLSuQMTrqLEmS\nMPseCf9cAwwd0NZ/5hLwyEvAu18J1DeyQCAix8PCgIioC6ic1JgeNx8zxj4KF2c3a/+Vssv4ZO/f\n8O3xbahrqJYxQuqsYD8Jm14Ann4IUCotfY1NwKYdwJwXgC8OCrS2skAgIsfBwoCIqAtF9huB/5z5\nIkYPngSlwsnafzbnBDbtehXHMvahoalexgipMxQKCY/PkPDRn4CBd7X1l1cDr24B5v8XcOiU4Ano\nROQQWBgQEXUxtUqDCTEz8fgvnkP/oChrf3NrE46c/hbv7ngZ+09uR0VNqYxRUmdE9JWw5b+A//Nr\nwM+zrf+yAVj1luX8gxOZLBCIqHtz+vkhRERkD55uvviP8U8g9+o5HDj1NcqriwEALa3NOHXhCFIv\nHMWA4CEYFTEBgT79IEmSzBFTeygUEmbGA1NGCWzdB3z4DVDXYDmWng387nWgXx9g7iSBmfGAi5bP\nLxF1LywMiIjusNA+EVg8PRyZeSdx8vz3KKm8CgAQEMguPIPswjMI8ApBVOhIDAoZBhetu8wRU3to\n1BIe+wUw+x6B93cBnx8AWk2WY3lXgdf/CWz8Epg+VmDevUBYEAsEIuoeWBgQEclAqVBiaNhoDAkd\nhcvGbKScO4zcq+esxw3lBTCUF+DAqa8R4tcfEXdFY2DIMGg1LjJGTe3h4Sbh9wuAhycLfJoA7Dpq\nuWoyYLn94qBliw4XuDcWmDhc3niJiFgYEBHJSJIk9AsYiH4BA1FaZUDKucPIvHwKJlMrAEAIM/KN\n2cg3ZmNfypfoGzAQ/YOi0Nc/HJ5uPvy6kQMI9pOw6lHgdw8J7D4G/OsAkFPUdjw927Kt3wp4u4Qi\nNrwGen+BAcHg80tEdxQLAyKibsJHH4BpYx7G+OgZOF9wGufzU1FYnAsBywmrZmFG7tVz1pkFN60e\nd/mHo29AOO7yHwA3nYec4dPP0DlLmDsJeGiiwKnzwLb9wOFUwHzDNe/ySzTIL9FgbxoQ6APERghE\nhwPRA4C+ASwUiKhrOWxhsHHjRrz++uu4evUqoqKi8Le//Q1333233GEREXWaztkVw8PHYXj4ONTU\nV+FCQTrO5afjSmmezbiahipk5KUgIy8FAODh6o0ArxD4ewXDzzMI/p5B/OpRNyRJEmIjgNgIoLxa\n4HAqcCgV+CHTdtyVUuDKEWDnEUvb3QUYNkAgegAQGQr0DwK83FksEJH9OGRhsG3bNqxYsQLvvPMO\n4uPjsWHDBkyfPh1ZWVkIDg6WOzwiIrtx0+kRO2g8YgeNR1VdOS4WnsVl40UUFF9Cc0ujzdjK2jJU\n1pbhXH6atc9d5wk/z0B46/3h6eYLD1cfeLn5QOfsxj8ouwEvdwlzJgBzJgB1DQIffVGEkxfdkFPi\nYl3R6LrqOuBIumW7zt0F6Bco0D8QCA0EQvtYZhoCvAG1is8vEbWPQxYG69evxxNPPIEnnngCAPDm\nm2/iu+++w9tvv421a9fKHB0RUdfQu3hZiwSz2QRDeSHyjReRb8xGYWmu9byEG1XXV6C6vgIXizJs\n+tVOGni4+cDD1QuuWg+4at2vbXq46fRw1eqhclKzeLiDXLQSxkTUYExEDYZF90FmLpB+0bKdvghU\n1d78M9V1wOlsy/ZjXu4CAd6Av5elUPDRW2YYPN0s2/V9jZrPMRFZOFxh0NLSgpMnT2LVqlU2/fff\nfz+SkpJkioqI6M5SKJQI9OmLQJ++iIuajFZTC0oqDSiuKISxvBDGiiKUVF29ZbEAWC6uVlxRhOKK\nolseBwClwgnOGh2c1VroNC5wVrtAq9FBo9LiiuEqnJRqqC62Qq3SQO2kgVqlgZNSBaXCCU5K1bXN\nsq9QKKBQKKGQeF3N26FWSYgZCMQMtLSFELhssBQI6RctJy/nXGlb5ehWyqstW2buT9+XViPgqgNc\ntWi7vbZpNZbNWX3tVgNo1YBGDahVgNoJUDkBGpXlVq0CnJS33pRKfu2JqLtzuMKgtLQUJpMJ/v7+\nNv3+/v7Yv3+/TFEREcnLSalCH+8Q9PEOsfaZzCaUVRlRUnkF5TUlqKwpRUVtKSpqSm/6GtKtmMyt\nqGuoRl1DNcp+dKy+rh4AcLnyTLvilCQFFAoFlJISSoXSWixIkgKSJEGhUEKSJMu+pIRk+SFLH9pu\nce0PzLZ+6z1Yjl//Ods7x42jbtEN4Pb+cO2qP3CLjZaL3l2uS7sWzc33ExVm2YQAqmqdYSh3R3GF\nK4xl7iitckFFjRbVdVqYxe3FWFEL3PQEdxGFJCBJAgpJQKEQbW2F5VaS2sZIwLU+S7+Ea7eS5WT8\n63241md5rbSNszQtxxsanCAB0G1pWxL4xqdQunaCP27R9++e6utx/OjHrnW0/wrXHXlF3ZE66yce\nS32d5QIdus/a9z4gl/tGqrF83iC5w+jWHK4w6Kiqqiq5Q+gxwsPDATCn9sa8do3enleNwgXB3xjS\nLgAACzpJREFUXuEI9gqXOxQiItn11v8LbpfDzen6+PhAqVTCaDTa9BuNRgQEBMgUFRERERGRY3O4\nwkClUiE2NhYJCQk2/QkJCYiPj5cpKiIiIiIix+aQXyV69tlnsWjRIowaNQrx8fF4++23cfXqVSxZ\nssRmnF6vlylCIiIiIiLH4pCFwbx581BeXo61a9fi6tWrGDJkCL799luEhIT8/A8TEREREdFNJCFE\n+0+dJyIiIiKiHsXhzjG4XRs3bkRYWBi0Wi1GjhyJI0eOyB2SbBITEzF79mwEBwdDoVDg448/vmnM\nSy+9hKCgIOh0OkyaNAmZmZk2x5ubm7F06VL4+vrC1dUVs2fPRlGR7frnlZWVWLhwITw8PODh4YFF\nixbddPZ/QUEBZs2aBVdXV/j6+mL58uVobb31Ouvd3bp16zB69Gjo9Xr4+fnhgQceQEZGxk3jmNv2\n2bhxI6Kjo6HX66HX6zFu3Djs3r3bZgxz2nnr1q2DQqHAsmXLbPqZ2/ZZs2bNtWs0tG2BgYE2Y5jT\njjEYDFi8eDH8/Pyg1WoxZMgQJCYm2oxhbtsnNDT0pterQqHArFmzrGOY0/Yzm83485//bP27Myws\nDH/+859hNpttxjlEbkUPtHXrVqFSqcT7778vzp07J5YuXSpcXV1FQUGB3KHJYvfu3WL16tXiiy++\nEC4uLmLz5s02x1999VXh7u4utm/fLjIyMsS8efNEYGCgqK2ttY558sknRVBQkNi/f79ITU0VEydO\nFDExMcJsNlvHTJs2TQwZMkQcP35cJCcni6ioKPHAAw9Yj5tMJjFkyBAxadIkkZaWJvbt2ycCAwPF\nsmXLuj4JXWDatGli8+bNIiMjQ5w9e1bMmTNHBAQEiIqKCusY5rb9duzYIb777jtx6dIlkZ2dLVav\nXi1UKpU4c+aMEII5tYdjx46J0NBQERMTI5YuXWrtZ27b76WXXhKDBw8WxcXFwmg0CqPRKEpLS63H\nmdOOqaysFGFhYWLx4sUiJSVF5OXliQMHDohz585ZxzC37VdaWmp9nRqNRpGamioUCoXYsmWLEII5\n7ai1a9cKb29v8c0334jLly+LnTt3Ci8vL/HKK69YxzhKbntkYTBmzBixZMkSm77w8HDxxz/+UaaI\nug9XV9ebCoM+ffqIdevWWdsNDQ3Czc1N/P3vfxdCCFFVVSXUarX49NNPrWMKCgqEQqEQe/fuFUII\nkZmZKSRJEseOHbOOOXLkiJAkSVy4cEEIYSlQlEqlKCoqso755JNPhFarFTU1NfZ/sHdYbW2tUCqV\nYteuXdY+5tY+vLy8rDljTjunsrJS9O/fXxw6dEhMnDjRpjBgbtvvpZdeEkOHDv23x5nTjnnxxRfF\n3Xff/ZNjmNvOe+WVV4Snp6dobGwUQjCnHTVz5kyxePFim77HHntMzJo1y9p2lNz2uK8StbS04OTJ\nk5gyZYpN//3334+kpCSZouq+cnNzYTAYbPLl7OyM8ePHW/OVkpKC1tZWmzHBwcEYPHiwdUxycjLc\n3NwQFxdnHRMfHw8XFxebMYMHD7aZZp86dSoaGxtx8uTJLn2cd0J1dTXMZjM8PT0BMLf2YDabsXXr\nVtTV1SE+Pp45tYPf/va3mDdvHiZMmGDTz9x2XE5ODoKCghAWFoYFCxYgNzcXAHPaGV9//TXGjBmD\n+fPnw9/fH8OHD8eGDRusx5lb+/jggw+wcOFCaDQa5rQT7r77bhw8eBDnz58HAGRmZuLAgQOYMWMG\nAMd6vTrkqkQ/pbS0FCaTCf7+/jb9/v7+2L9/v0xRdV8GgwGSJN0yX1euXAFguXicUqmEt7f3TWMM\nBoP19/j6+t70+/38/GzG/Ph+rl+w7voYR7Z8+XKMGDECY8eOBcDcdsbZs2cxduxYNDY2ws3NDdu3\nb0dkZCSOHTvGnHbCe++9h5ycHHz66ac3HePrtWPi4uLw0UcfISIiAsXFxXj55ZcRHx+PjIwM5rQT\ncnJysHHjRqxcuRIvvvgi0tLS8Mwzz0CSJDz99NPMrR3s3bsXeXl5+M1vfgOA7wGd8Yc//AE1NTWI\njIyEUqmEyWTC6tWrrcvoO1Jue1xhQCSHZ599FklJSTh69CgkSZI7HIcXERGB9PR0VFVV4fPPP8ei\nRYtw+PBhucNyaBcuXMDq1atx9OhRKBQ9brJYNlOnTrVpx8XFITQ0FJs3b8aYMWNkisrxmc1mjB49\nGmvXrgUAREdH48KFC9iwYQOefvppmaPrGd577z2MGjUKQ4YMkTsUh7d161Zs2bIFW7duRWRkJNLS\n0rBs2TKEhobi8ccflzu8dulx/ztcr4qMRqNNv9FoREBAgExRdV8BAQEQQvxkvgICAmAymVBWVvaT\nY0pKSm76/cXFxTZjfnw/12d4HPm5WblyJbZt24aDBw+ib9++1n7mtuOcnJwQFhaG4cOHY+3atYiJ\nicH69euZ0044duwYysrKEBkZCZVKBZVKhcOHD2PDhg1Qq9Xw9vZmbu1Ap9MhKioK2dnZfL12Qp8+\nfTB48GCbvsGDByM/Px8A3187q6SkBDt27MBvf/tbax9z2nHPP/88Vq1ahV/+8peIiorCo48+imef\nfRbr1q0D4Fi57XGFgUqlQmxsLBISEmz6ExISEB8fL1NU3VdoaCgCAgJs8tXY2IjExERrvmJjY+Hk\n5GQzprCwEFlZWdYxY8eORW1tLZKTk61jkpKSUF9fj3HjxlnHZGVlWafNAMtUprOzM2JjY7v0cXaV\n5cuXW4uC8PBwm2PMrf2YzWY0NTUxp50wZ84cnDlzBunp6dZt5MiRWLBgAdLT0zFw4EDm1g4aGxtx\n7tw5BAYG8vXaCfHx8dbva193/vx564cvzG3nfPjhh3B2dsb8+fOtfcxpx9XX1980E6tQKKzLlTpU\nbm/nbGtHs23bNqHRaMSmTZtEVlaWWLZsmXBzcxP5+flyhyaL2tpakZaWJlJTU4VOpxMvv/yySEtL\ns+bjtddeEx4eHuLLL78UZ86cEQ8//LAICgqyWULrqaeeEiEhIWLfvn3i1KlTYtKkSWLEiBE2S2hN\nnz5dDBs2TBw7dkwkJSWJoUOHitmzZ1uPm0wmMWzYMDF58mSRmpoqEhISRFBQkFi+fPmdS4YdPf30\n08Ld3V0cPHhQGAwG63Zj3pjb9nvhhRdEYmKiyMvLE2fOnBEvvPCCUCqVYs+ePUII5tSefrwqEXPb\nfs8995w4fPiwyM3NFcnJyWLGjBlCr9fz/bWTfvjhB6FWq8XatWvFxYsXxWeffSb0er14++23rWOY\n244bOHDgTas3CsGcdtTixYtFSEiI+Oabb0ReXp748ssvha+vr1i1apV1jKPktkcWBkII8fbbb4vQ\n0FDh7OwsRo4cKY4cOSJ3SLI5dOiQkCRJKBQKm+3xxx+3jlmzZo0IDAwUWq1WTJw4UWRkZNj8jubm\nZrFs2TLh4+MjXFxcxOzZs0VhYaHNmMrKSrFw4UKh1+uFXq8XixYtElVVVTZjCgoKxKxZs4SLi4vw\n8fERK1asEM3NzV334LvQrXKqUCjEmjVrbMYxt+2zePFi0a9fP+Hs7Cz8/f3FlClTREJCgs0Y5tQ+\nJk2aZFMYCMHcttf8+fNFUFCQ0Gg0Ijg4WMydO1dkZWXZjGFOO2b37t0iOjpaaLVaMWjQIPHWW2/d\nNIa5bb+DBw8KhUIhUlJSbnmcOW2/2tpasXLlStGvXz+h0+lE//79xZ/+9CfR1NRkM84RcisJIUQ7\nZ0yIiIiIiKiH6XHnGBARERERUfuxMCAiIiIiIhYGRERERETEwoCIiIiIiMDCgIiIiIiIwMKAiIiI\niIjAwoCIiIiIiMDCgIiIiIiIwMKAiIiIiIgA/H8jWKKVHajyCgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "player1.MakeBeliefs(20000)\n", + "player2.MakeBeliefs(40000)\n", + " \n", + "calc1 = GainCalculator(player1, player2)\n", + "calc2 = GainCalculator(player2, player1)\n", + "\n", + "bids, gains = calc1.ExpectedGains()\n", + "thinkplot.Plot(bids, gains, label='Player 1')\n", + "print('Player 1 optimal bid', max(zip(gains, bids)))\n", + "\n", + "bids, gains = calc2.ExpectedGains()\n", + "thinkplot.Plot(bids, gains, label='Player 2')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 6.4: Expected gain versus bid in a scenario where Player 1’s best guess is\n", + "\\$20,000 and Player 2’s best guess is \\$40,000." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you overbid, you get nothing. Otherwise we compute the difference\n", + "between your bid and the price, which determines your probability of\n", + "winning.\n", + "\n", + "If `diff` is less than \\$250, you win both showcases. For\n", + "simplicity, I assume that both showcases have the same price. Since this\n", + "outcome is rare, it doesn’t make much difference.\n", + "\n", + "Finally, we have to compute the probability of winning based on\n", + "`diff`:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "def ProbWin(self, diff):\n", + " prob = (self.opponent.ProbOverbid() + \n", + " self.opponent.ProbWorseThan(diff))\n", + " return prob\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If your opponent overbids, you win. Otherwise, you have to hope that\n", + "your opponent is off by more than `diff`. `Player`\n", + "provides methods to compute both probabilities:\n", + "\n", + "```python\n", + "# class Player:\n", + " def ProbOverbid(self):\n", + " return self.cdf_diff.Prob(-1)\n", + "\n", + " def ProbWorseThan(self, diff):\n", + " return 1 - self.cdf_diff.Prob(diff)\n", + "```\n", + "\n", + "This code might be confusing because the computation is now from the\n", + "point of view of the opponent, who is computing, “What is the\n", + "probability that I overbid?” and “What is the probability that my bid is\n", + "off by more than `diff`?”\n", + "\n", + "Both answers are based on the CDF of `diff`. If the\n", + "opponent’s `diff` is less than or equal to -1, you win. If\n", + "the opponent’s `diff` is worse than yours, you win. Otherwise\n", + "you lose.\n", + "\n", + "Finally, here’s the code that computes optimal bids:\n", + "\n", + "```python\n", + "# class Player:\n", + " def OptimalBid(self, guess, opponent):\n", + " self.MakeBeliefs(guess)\n", + " calc = GainCalculator(self, opponent)\n", + " bids, gains = calc.ExpectedGains()\n", + " gain, bid = max(zip(gains, bids))\n", + " return bid, gain\n", + "```\n", + "\n", + "Given a guess and an opponent, `OptimalBid` computes the\n", + "posterior distribution, instantiates a `GainCalculator`,\n", + "computes expected gains for a range of bids and returns the optimal bid\n", + "and expected gain. Whew!\n", + "\n", + "Figure [fig.price5] shows the results for both players, based on a\n", + "scenario where Player 1’s best guess is \\$20,000 and Player 2’s best\n", + "guess is \\$40,000.\n", + "\n", + "For Player 1 the optimal bid is \\$21,000, yielding an expected return of\n", + "almost \\$16,700. This is a case (which turns out to be unusual) where\n", + "the optimal bid is actually higher than the contestant’s best guess.\n", + "\n", + "For Player 2 the optimal bid is \\$31,500, yielding an expected return of\n", + "almost \\$19,400. This is the more typical case where the optimal bid is\n", + "less than the best guess." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "One of the features of Bayesian estimation is that the result comes in\n", + "the form of a posterior distribution. Classical estimation usually\n", + "generates a single point estimate or a confidence interval, which is\n", + "sufficient if estimation is the last step in the process, but if you\n", + "want to use an estimate as an input to a subsequent analysis, point\n", + "estimates and intervals are often not much help.\n", + "\n", + "In this example, we use the posterior distribution to compute an optimal\n", + "bid. The return on a given bid is asymmetric and discontinuous (if you\n", + "overbid, you lose), so it would be hard to solve this problem\n", + "analytically. But it is relatively simple to do computationally.\n", + "\n", + "Newcomers to Bayesian thinking are often tempted to summarize the\n", + "posterior distribution by computing the mean or the maximum likelihood\n", + "estimate. These summaries can be useful, but if that’s all you need,\n", + "then you probably don’t need Bayesian methods in the first place.\n", + "\n", + "Bayesian methods are most useful when you can carry the posterior\n", + "distribution into the next step of the analysis to perform some kind of\n", + "decision analysis, as we did in this chapter, or some kind of\n", + "prediction, as we see in the next chapter." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/07_Prediction.ipynb b/notebooks/07_Prediction.ipynb index d12706a..98082d6 100644 --- a/notebooks/07_Prediction.ipynb +++ b/notebooks/07_Prediction.ipynb @@ -1,1024 +1,1019 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 1, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# format the book\n", - "%matplotlib inline\n", - "import sys\n", - "from __future__ import division, print_function\n", - "%load_ext autoreload\n", - "%autoreload 2 \n", - "import sys\n", - "sys.path.insert(0,'../code')\n", - "import book_format\n", - "book_format.load_style('../code')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#Prediction\n", - "\n", - "##The Boston Bruins problem\n", - "\n", - "In the 2010-11 National Hockey League (NHL) Finals, my beloved Boston\n", - "Bruins played a best-of-seven championship series against the despised\n", - "Vancouver Canucks. Boston lost the first two games 0-1 and 2-3, then won\n", - "the next two games 8-1 and 4-0. At this point in the series, what is the\n", - "probability that Boston will win the next game, and what is their\n", - "probability of winning the championship?\n", - "\n", - "As always, to answer a question like this, we need to make some\n", - "assumptions. First, it is reasonable to believe that goal scoring in\n", - "hockey is at least approximately a Poisson process, which means that it\n", - "is equally likely for a goal to be scored at any time during a game.\n", - "Second, we can assume that against a particular opponent, each team has\n", - "some long-term average goals per game, denoted $\\lambda$.\n", - "\n", - "Given these assumptions, my strategy for answering this question is\n", - "\n", - "1. Use statistics from previous games to choose a prior distribution\n", - " for $\\lambda$.\n", - "\n", - "2. Use the score from the first four games to estimate $\\lambda$ for\n", - " each team.\n", - "\n", - "3. Use the posterior distributions of $\\lambda$ to compute distribution\n", - " of goals for each team, the distribution of the goal differential,\n", - " and the probability that each team wins the next game.\n", - "\n", - "4. Compute the probability that each team wins the series.\n", - "\n", - "To choose a prior distribution, I got some statistics from\n", - ", specifically the average goals per game for each\n", - "team in the 2010-11 season. The distribution is roughly Gaussian with\n", - "mean 2.8 and standard deviation 0.3.\n", - "\n", - "The Gaussian distribution is continuous, but we’ll approximate it with a\n", - "discrete Pmf. `thinkbayes` provides `MakeGaussianPmf` to do exactly\n", - "that:\n", - "\n", - " def MakeGaussianPmf(mu, sigma, num_sigmas, n=101):\n", - " pmf = Pmf()\n", - " low = mu - num_sigmas*sigma\n", - " high = mu + num_sigmas*sigma\n", - "\n", - " for x in numpy.linspace(low, high, n):\n", - " p = scipy.stats.norm.pdf(mu, sigma, x)\n", - " pmf.Set(x, p)\n", - " pmf.Normalize()\n", - " return pmf\n", - "\n", - "`mu` and `sigma` are the mean and standard\n", - "deviation of the Gaussian distribution. `num_sigmas` is the number of\n", - "standard deviations above and below the mean that the Pmf will span, and\n", - "`n` is the number of values in the Pmf.\n", - "\n", - "Again we use `numpy.linspace` to make an array of\n", - "`n` equally spaced values between `low` and\n", - "`high`, including both.\n", - "\n", - "`norm.pdf` evaluates the Gaussian probability density function (PDF).\n", - "\n", - "Getting back to the hockey problem, here’s the definition for a suite of\n", - "hypotheses about the value of $\\lambda$." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "```python\n", - "class Hockey(thinkbayes.Suite):\n", - "\n", - " def __init__(self):\n", - " pmf = thinkbayes.MakeGaussianPmf(mu=2.7, sigma=0.3, num_sigmas=4)\n", - " thinkbayes.Suite.__init__(self, pmf)\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "So the prior distribution is Gaussian with mean 2.7, standard deviation\n", - "0.3, and it spans 4 sigmas above and below the mean.\n", - "\n", - "As always, we have to decide how to represent each hypothesis; in this\n", - "case I represent the hypothesis that $\\lambda=x$ with the floating-point\n", - "value `x`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Poisson processes\n", - "\n", - "In mathematical statistics, a **process** is a stochastic\n", - "model of a physical system (“stochastic” means that the model has some\n", - "kind of randomness in it). For example, a Bernoulli process is a model\n", - "of a sequence of events, called trials, in which each trial has two\n", - "possible outcomes, like success and failure. So a Bernoulli process is a\n", - "natural model for a series of coin flips, or a series of shots on goal.\n", - "\n", - "A Poisson process is the continuous version of a Bernoulli process,\n", - "where an event can occur at any point in time with equal probability.\n", - "Poisson processes can be used to model customers arriving in a store,\n", - "buses arriving at a bus stop, or goals scored in a hockey game.\n", - "\n", - "In many real systems the probability of an event changes over time.\n", - "Customers are more likely to go to a store at certain times of day,\n", - "buses are supposed to arrive at fixed intervals, and goals are more or\n", - "less likely at different times during a game.\n", - "\n", - "But all models are based on simplifications, and in this case modeling a\n", - "hockey game with a Poisson process is a reasonable choice. Heuer, Müller\n", - "and Rubner (2010) analyze scoring in a German soccer league and come to\n", - "the same conclusion; see\n", - ".\n", - "\n", - "The benefit of using this model is that we can compute the distribution\n", - "of goals per game efficiently, as well as the distribution of time\n", - "between goals. Specifically, if the average number of goals in a game is\n", - "`lam`, the distribution of goals per game is given by the\n", - "Poisson PMF:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def EvalPoissonPmf(k, lam):\n", - " return (lam)**k * math.exp(-lam) / math.factorial(k)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "And the distribution of time between goals is given by the exponential\n", - "PDF:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def EvalExponentialPdf(x, lam):\n", - " return lam * math.exp(-lam * x)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "I use the variable `lam` because `lambda` is a\n", - "reserved keyword in Python. Both of these functions are in\n", - "`thinkbayes.py`." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The posteriors\n", - "\n", - "![Posterior distribution of the number of goals per\n", - "game.](figs/hockey1.pdf)\n", - "\n", - "[fig.hockey1]\n", - "\n", - "Now we can compute the likelihood that a team with a hypothetical value\n", - "of `lam` scores `k` goals in a game:\n", - "\n", - " # class Hockey\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " lam = hypo\n", - " k = data\n", - " like = thinkbayes.EvalPoissonPmf(k, lam)\n", - " return like\n", - "\n", - "Each hypothesis is a possible value of $\\lambda$; `data` is\n", - "the observed number of goals, `k`.\n", - "\n", - "With the likelihood function in place, we can make a suite for each team\n", - "and update them with the scores from the first four games." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvQAAAEWCAYAAAAJuAMCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcU1feP/BPFgKEfSeyo4KAiEAURa1dlJZaO7aoVRyt\nVjujta3L428cWrv4VDu1Tn1qF9E6rqOtWpdO7VCVVutKVaQgm4KyCwFB9iWB5P7+CNwQiYAK3EC+\n79freb2ecznn5hvnNvnm3nO+h8cwDANCCCGEEEJIv8TnOgBCCCGEEELIo6OEnhBCCCGEkH6MEnpC\nCCGEEEL6MUroCSGEEEII6ccooSeEEEIIIaQfo4SeEEIIIYSQfowSekIIIYQQQvqxbiX0W7ZsgZeX\nF0xNTSGVSnHhwoVO+6empmLixIkQi8VwdXXFRx99pPV3mUyG6Oho+Pn5QSgUYsGCBTrP89lnn8HX\n1xdisRhubm548803UV9f3823RgghhBBCyMDXZUJ/8OBBLF++HGvWrEFycjLCw8MRGRmJwsJCnf1r\namowefJkSCQSJCYmYvPmzdi4cSM2bdrE9pHL5XBwcEBMTAzCwsLA4/E6nGfv3r1499138f777+PG\njRvYu3cv4uLisGzZssd4u4QQQgghhAwsvK52ig0LC8PIkSOxbds29piPjw+mT5+Ojz/+uEP/2NhY\nxMTEoLS0FMbGxgCA9evXIzY2FkVFRR36T506FQ4ODti5c6fW8TfffBNpaWn47bff2GMffPABjh49\nitTU1Id6k4QQQgghhAxUnd6hVygUSEpKQkREhNbxiIgIXLp0SeeYhIQETJgwgU3m2/oXFxcjPz+/\n24FFRkYiJSUFly9fBgAUFBTgxx9/xJQpU7p9DkIIIYQQQga6ThP68vJyKJVKODk5aR13dHSETCbT\nOUYmk3Xo39Z+0BhdpkyZgvXr12PChAkQiUTw9PREUFAQPvnkk26fgxBCCCGEkIFO2NMn1DUf/lEc\nO3YM77zzDrZu3YqwsDBkZ2dj2bJl+OCDD7B27Vq2X3V1dY+8HiGEEEIIIVyxsrJ65LGdJvT29vYQ\nCAQoLS3VOl5aWgqJRKJzjLOzc4c78W3jnZ2dux3YJ598goULF+K1114DAAQEBKC+vh6LFi3CBx98\nAD6fKm4SQgghhBDSaVYsEokQGhqKU6dOaR2Pj49HeHi4zjFjx47F+fPnIZfLtfq7uLjAw8Oj24Ex\nDNMhaefz+ehiDS8hhBBCCCEGpcspNytXrsTcuXMxevRohIeHY+vWrZDJZFi8eDEAICYmBlevXsUv\nv/wCAIiOjsbatWsxf/58rFmzBjdv3sSGDRvw4Ycfap03OTkZgHrKDJ/PR3JyMkQiEfz9/QEA06ZN\nw4YNGyCVSjF69GjcunUL7733HqZOnfrAu/OP86iCDGyJiYkAAKlUynEkRJ/RdUK6g64T0h10nZDu\n6Kmp410m9DNnzkRFRQXWrVuHkpISBAYGIi4uDm5ubgDUC11zcnLY/paWloiPj8fSpUshlUpha2uL\nVatWYcWKFVrnDQkJAaCec88wDI4fPw5PT0/2XKtXrwbDMHjvvfdQVFQEBwcHTJ06FevXr++RN04I\nIYQQQshA0GUden3X/pcN3aEnD0J3Skh30HVCuoOuE9IddJ2Q7uipPJZWlhJCCCGEENKPUUJPCCGE\nEEJIP0YJPSGEEEIIIf0YJfSEEEIIIYT0Yz2+UywhhBDSU+obFCi724Dyew1QqdQ1HHgAxGIjODqY\nwc7GFAIB3ZsihBg2SugJIYToBUWzErdy7iEt8y7Sb5QhJ68K9Q2KTscIBHw4OZjBz8cew/0cEejv\nCFsb0z6KmBBC9AMl9IQQQjjDMAxuZFfgzPk8XLxciMam5ocar1SqUCyrRbGsFr+eywUADPW2w9NP\neGJcmBvMzUS9ETYhhOgVSugJIYT0ueYWJU6fy8PxE1koltU+sJ/ISAAHezPY24khMlJPrWEAVFfL\ncbeiHlXVTR3GZOdUIDunAjv3JSM8zBVRU/3gOsiyt94KIYRwjhJ6QgghfaYtkT9yPBPlFQ0d/u5o\nb4YRAU4Y7ucAf18H2NmagsfjPfB8cnkLcvKrkJZZhtSMMmRmlUOpVLGvdfZiPs5dKsCEMe6Y/idK\n7AkhAxMl9IQQQvpESnoptu2+BllpndZxUxMjjB/jhifHe2LYULtOE/j7GRsL4edjDz8fe8z4kz9q\n6+Q4d6kAZ87nISe/EoB6Ws+5hHyc/70AkZMGI3p6IMSmRj363gghhEuU0BNCCOlVlVVN2P1dMs4n\nFGgdt7QwxrQpw/Ds094wNemZBNvC3BhTIoZiSsRQZN2qwKEfMpB0vQSAOrGPi7+FhCt38NqfRyJ8\ntOtD/XgghBB9RQk9IYSQXpNwtQixOxNRV6+pViM2NULUi36IfGYITEx672vIZ4gd1qyagKxbFfj2\ncBquZ5QCACqrG/HZ1wk4n+CCpYuksDA37rUYCCGkL1BCTwghpMcpFErsOZCCn3+5pXV8wlh3zJ89\nEjbWJn0Wi88QO3yw+glcvFyIXftTUFndCAC4knQHOe9VYvniMPj7OvRZPIQQ0tMooSeEkD7EMAxK\n6xXIrWpERWMzKhubUdnUjBYVAwGfBwGPBxMhH45mIjiZiSAxN4aLhQkE/P4zNaREVouNXyUgr6CK\nPeZgZ4Y3FoYiaLgzJzHxeDyMH+OOkYHO2HcoFafO3AYAlFc04L2Pf8PsqABETfWjKTiEkH6JEnpC\nCOllxbVNuFRUhdSyOmTdq0eNvOWhxpsZCeBnb47hDuYIdrbEYJvOK79wKf3GXWzYfFFris0YqSve\neE0Kc3Pua8Kbm4mweEEoQoMk+HL7FdTVK8AwDL49nIb8wmq89fpoiEQCrsMkhJCHQgk9IYT0gooG\nBU7mVOBiYSXyWqd4PKr6ZiUSS6qRWFKN3dfvwNXCBBM9bPGkhw0GWfTd1JWunLmQhy07EtmykUZC\nAeZHB+G5Zwbr3Q+QUSGDsGldBDbF/o4bWeUAgIuXC3G3vAGrl43r0ylBhBDyuPjd6bRlyxZ4eXnB\n1NQUUqkUFy5c6LR/amoqJk6cCLFYDFdXV3z00Udaf5fJZIiOjoafnx+EQiEWLFig8zw1NTV4++23\n4eLiAhMTEwwdOhTff/99N98aIYT0LYZhcKO8Dp9eysVrx9OwP61YZzJvbiTASCdLPDfYHrMDJHgj\n1B3LR3vg7VEeWCp1x7zAQXjW2x4jnSxhq6P6S1FtE/anFeP1/6Zj7blbSCurBcMwffEWdWIYBgeO\npuHLb66wyby1lQnWvfskIicN0btkvo29nRgfxTyJ554Zwh7Lul2Bv6/9FUXFNRxGRgghD6fLO/QH\nDx7E8uXLERsbi/Hjx+Prr79GZGQkMjIy4Obm1qF/TU0NJk+ejCeffBKJiYnIzMzEggULYGZmhpUr\nVwIA5HI5HBwcEBMTg23btun8sG9ubsbkyZNhb2+P77//Hq6urigqKoJIxP0jW0IIud/NinrsuX4H\nKaUddz0V8fkIkVgizMUK/vbmGGRhDH43k1yGYSCrUyDtbi2SS2tx+U41GluU7N+vFFfjSnE1htmZ\n4c+BgxDs3LcbJzEMg537k/HfU9nsMXdXK7yzcjwc7c36NJZHIRDw8fq8YAySWGDX/mQwDIO7FfVY\ns/4M3v9/T8Db04brEAkhpEs8povbOmFhYRg5ciS2bdvGHvPx8cH06dPx8ccfd+gfGxuLmJgYlJaW\nwthYXQps/fr1iI2NRVFRUYf+U6dOhYODA3bu3Kl1/JtvvsGnn36KGzduQCh88O+O6upq9v+3srLq\n7K0QA5aYmAgAkEqlHEdC9NmjXCfFtU3Ye70Y5wsrO/wtwMEcU4Y4YPQgK5ga9cy87KYWFa4UV+F0\n3j1cLa7u8HepxAqLgl3hZtn7U0ZUKgbb9lxD/Jkc9tjIQGesenNsv9y4KTG5GJ99/TvkrWsczMQi\nvLdqAnyG2Gn3o88T0g10nZDu6Kk8ttMpNwqFAklJSYiIiNA6HhERgUuXLukck5CQgAkTJrDJfFv/\n4uJi5OfndzuwH374AeHh4Vi6dCkkEgkCAgKwdu1atLQ83GIyQgjpDS0qBgfTS7D050ytZF7A4+EZ\nTzt88awfPn3GFxM9bHssmQcAEyEfT7jb4sMnhmDr8wF41tseRu0q4CSWVGPpzxnYnlSIpnZ38nua\nUqnCl9uvaCXzY0e5ImbFuH6ZzAOAdOQgfPC3J9j46xsU+PDTs0jLLOM4MkII6VynU27Ky8uhVCrh\n5OSkddzR0REymUznGJlMBnd3d61jbeNlMhk8PDy6FVhOTg7OnDmDOXPmIC4uDrm5uVi6dCnq6uqw\nceNGnWPafg0T8iB0jZDu6Oo6KWxQ4XCRAsWN2g84A60EeM5ZCEdhOSpvl6MvrrZwPuDvyuBUaTOu\n3tMk8N/+UY+TGfmY6WoEb/OerdrCMAyO/VyMaymaHzIjh1tjYpgRUpL/6NHX4sLMqfbYdSAPDY1K\n1NcDMWtPYMFsT7i7iLX60ecJ6Q66Tkhnhg4d2iPn6dai2IfRU4ufVCoVnJycsH37dgQHB+Pll1/G\n//7v/yI2NrZHzk8IIQ9LxTD4tbQZX92SayXzrmIe3hxijHmeIjia9PjHapesRTzMdBNhuY8xvM01\nr18hZxB7W4Hjxc1oVvXMolmGYfDzaZlWMj9qpA2ipriA349q5XdmkLMpFs3xgoW5+p6XolmFvYfy\nUFLWxHFkhBCiW6d36O3t7SEQCFBaWqp1vLS0FBKJROcYZ2fnDnfv28Y7O3d/Q5FBgwZBJBJp/UAY\nNmwYGhoaUFFRATs7uw5jaJ4aeRCay0i6o7PrpKqpGf9MyMMftTUwEasX5xsL+JgzfBCm+TrqxcZP\nUgAvMgx+ya3A9j+KUN+svmOf2ABUVYoRM84bzubGnZ+kC4d+yEByeiPMzNR3q58c74k3F40aMMl8\neyNGjMCa9WdQWycHAByNu4d17z6JkjtZAOjzhHSOvndId7SfQ/84Or2VJBKJEBoailOnTmkdj4+P\nR3h4uM4xY8eOxfnz5yGXy7X6u7i4dHu6DQCMGzcO2dnZWqXYsrKyYGZmpjOZJ4SQ3pJ+tw5vncjE\nH6WaUoYB9ub4+jl/RPk56UUy34bH42Gytz2+jvSHVKJZYHWrsgHLT93AFR0LabvrxK+3cOBoGtse\nI3XF0oXSAZnMA4Cbi6XWnPrqmias/fQcqmuaOY6MEEK0dflseOXKldi9ezd27NiBzMxMLFu2DDKZ\nDIsXLwYAxMTEYNKkSWz/6OhoiMVizJ8/H+np6Th69Cg2bNjAlqxsk5ycjOTkZFRXV6OiogLJycnI\nyMhg/75kyRLcu3cPy5Ytw82bN3Hy5El8+OGHeOONN3rqvRNCSJfic8rxzuks3GvSJHEz/Zzxj6d9\nILF4vLvdvclBLMKHTwzG4hA3CFufdNYqWrD23C18l1by0HXrr6WUYPtezfz4EQFOWL4kDAJB308x\n6kvenjZ4938mwNhY/UC7vKIB/z6cjyZ57y04JoSQh9VlHfqZM2eioqIC69atQ0lJCQIDAxEXF8fW\noJfJZMjJ0VQ5sLS0RHx8PJYuXQqpVApbW1usWrUKK1as0DpvSEgIAPXdJIZhcPz4cXh6erLncnV1\nxalTp7By5UoEBwfD2dkZCxcuxJo1a3rszRNCyIOoGAZ7rxfj+0zNFEJLYyFWjfFEqKR/lMjl8XiY\n6uMIHzsz/ONiDu42KAAA+9KKUVInx1uj3GHUjYQ8J68Sn32dwP4IGOJti78vGwdRD1bv0Wd+Pvb4\n+7JwrPvsApRKFUpKm3DwP4UYO2bUgP9BQwjpH7qsQ6/vqA496Q6ay0i6o+06CRwZgs8u5+Fiu3KU\nnlameP+JwXAy09+78p2plrfg00u5SG43bSjIyQLvjPOGuejB93bKKxqweu2vqKxS73jrYGeGTz54\nBjbWvV/nXt/8ejYXX++4ivr6BgDASy8E4q/zQ/R2J1zCLfreId3RJ3XoCSHE0MiVDP73/G2tZH7U\nICtsnOTbb5N5ALAyFmLtxCGI8LZnj6WU1uJvv2bhXqPuOeEKhRKffH6RTebFpkZ493/GG2QyDwDP\nTPTC9Bf92fapM7fx08nsTkYQQkjfoISeEEJaNbQw+CZHoXUX+0UfR7w3fjDEA2B6iZDPw9uj3DEv\ncBB7LL+6EX8/ncVOx2nDMAxidyYiJ1/9w0Yg4OP/vR0Od1fDfhI6OyoAQf6af4M9B1KQkl7ayQhC\nCOl9lNATQgjUZSm35shR0KBij80LHIS/BLvqVRWbx8Xj8fBKgAT/M8YTgtapIndqm/C3X26ipFZT\nney/p7Jx9pJmd+9Fc4MRFODU4XyGhsfj4eUpLnBzMQUAqFQMNn39O2RldRxHRggxZJTQE0IMXo28\nBWt+y0ZJu82iFoe44ZUAyYCdH/20px1ixnmzFXDKGhRYffom7tQ2IS2zDLu/S2H7TprojYinvLkK\nVe8IhXxEv+QOGyt1Ul9bJ8enX1xCU1MLx5ERQgwVJfSEEINWp2jBe79lI7d1njiPB6wM88RUH0eO\nI+t9Y12t8d4TgyFqrdRS0diMv526gfWxv0PVurPsUG87LJoXPGB/2DwqSwsj/L+3x7JVbvIKqhC7\nK/Ghy4ESQkhPoISeEGKwGpqVeP/sLdyqbGCPzXQ1wjNehrN5nVRihbUTh8BYwAcYIL2gCul2xmgW\n8mBtZYK/vT3WYMpTPqxhQ+3xl1dD2Pb5hAKcOpPTyQhCCOkdlNATQgySQqnCR+dv42ZFPXtsuqsR\npLZdbs8x4IxwtMB7EwajuqoJTU0tUIgEKHC3xOuvj4KdrZjr8PTa5Ce9MWmiZjrSzn3JyMmr7GQE\nIYT0PEroCSEGR8Uw2PR7Hq6X1bLHloS6I8zO8JL5Nry7DTBNvQte65QRM2dzHJJVo6mFdkTtyqK5\nwfBwswYANLco8c+vE9DwgFKghBDSGyihJ4QYFIZh8E1SEc63qzP/6ggXvDDUgcOouHWvqhGfb70M\nizoFBhXXQ2xqBBsrE2Tdq8cnl3KhVNG88M6IRAKsenMMTEzUPwhlpXXYsoPm0xNC+g4l9IQQg3Lk\nRimOZ5ex7alDHTHDz3DLMapUDL7YdgU1rSUrPXkC/P2pIUDrGtirxdXYcq2AktMuuEgs8cZrmh1B\nL10pxOlzedwFRAgxKJTQE0IMxsXCSuxKucO2x7vZ4PVgV4Ou4BIXn43r7MZIPCxbPBozg1ww08+Z\n7XPidjkOpMu4CbAfGT/GHRFPDWbb/9r3B4pltZ2MIISQnkEJPSHEIGRV1OOz3/PYdqCjhXpzpQG0\nadTDKiiqxr8PprLtaVN8MaJ186h5IwbhaU9NtZ99acU4l3+vz2PsbxZEB8FFYgkAkMtb8PnWy2hp\nUXUxihBCHg8l9ISQAa+8QYF1F25DrlQnVi4WJnhnnDdbf90QKZqV+L/Yy2huXfTq5W6NWS8HsH/n\n8Xh4e5Q7gp0s2WP/dyUfWe2qApGOjI2FWLEkjK1PfyvnHg4eS+c4KkLIQGe432aEEIPQ1KIuT1nR\nWnXE3EiA9ycMhqWx4Va0AYDvDqchv7AKACAyEmDZ4rAO9eaNBHysDveCq4UJAHWpz3UXbqOiQdHn\n8fYn3p42mDMjkG0fOX4DGTfvchgRIWSgo4SeEDJgMQyDr67msxtHCXg8xIzzhqulCceRcSszqxz/\n+TmLbc+bNQLurlY6+1oYC/HehMEwb032Kxqbse5CDuQ0jaRTLz7ngxH+bYutGXy5/Soam6iUJSGk\nd3Qrod+yZQu8vLxgamoKqVSKCxcudNo/NTUVEydOhFgshqurKz766COtv8tkMkRHR8PPzw9CoRAL\nFizo9Hzfffcd+Hw+pk6d2p1wCSEEAHA8+y7OtJv3/dcQN4x0tuxkxMAnl7fgq+1XAair1owIcELk\npCGdjnG1NMHfx3lD0Lp4OOtePWKp8k2n+Hwe3nx9FMSmRgCA0rI67D1wneOoCCEDVZcJ/cGDB7F8\n+XKsWbMGycnJCA8PR2RkJAoLC3X2r6mpweTJkyGRSJCYmIjNmzdj48aN2LRpE9tHLpfDwcEBMTEx\nCAsL67TCRE5ODv72t79hwoQJBl2JghDycNLKarHjjyK2HeFtj+eH2HMYkX7YfzgNJaXqyitiUyO8\nuWhUtz5bg50tsSjYlW3H51bgxO3yXotzILC3E2PRvGC2ffL0baSkUbUgQkjP6zKh37RpExYsWICF\nCxfC19cXX3zxBSQSCWJjY3X2379/P5qamrBnzx74+/sjKioKq1ev1kroPTw8sHnzZsybNw+2trYP\nfO3m5mbMnj0bH3/8Mby9veluECGkW8obFPjkUi5aWj8zfGzNsDjEzeBvCmTcvIufTmaz7fnRQbC3\nE3d7/NShDlqVb7YlFeImLZLt1MRwD4wOcWHbX/0rEfW0BoEQ0sM6TegVCgWSkpIQERGhdTwiIgKX\nLl3SOSYhIQETJkyAsbGxVv/i4mLk5+c/VHDvvvsuvL29MXfuXErmCSHdolQx+PRSLipb5ytbGQvx\nznhvGAsNe8mQXN6Cr/6lmWoTPMIZzzzh9VDn4PF4WCp1h7e1+kdAs4rBPy7moIrmhj8Qj8fDX+eH\nwsJc/Z1Yca8Bu79N4TgqQshA0+k3XHl5OZRKJZyctHdRdHR0hEym+7GhTCbr0L+t/aAxupw6dQqH\nDx/Gtm3bAKg/FA397hohpGv7UouRXl4HAODzeFgd7g0HsYjjqLh38Fg6ZKXqfxexqRGWLJA+0meq\niZCPd8Z5s4tk7zYo8M+EPKjopssD2Vib4K/zQ9j2r+dy223mRQghj6/H67b1RNJ99+5dzJ8/HwcO\nHIClpXoBG8MwXd6lT0xMfOzXJgMbXSMD240aJXbnaqYzPOcsRHPhTSTqXvLzQAPtOrlT0ohvD99G\n20doxMRByMvNQF7uo5/zBRsldrb+W1/MacDGhnI842TUA9H2Hw9znYj4gJebEGk3agAA//i/X/H2\nwiEwMjLsJ0eGYKB9npCeNXTo0B45T6cJvb29PQQCAUpLte8klJaWQiKR6Bzj7Ozc4U5823hnZ2dd\nQzpIT0+HTCbDM888wx5TqdQl0oyMjJCRkdFj/wCEkIGhSsHgQKEmmfex4OMpR8OuNQ8ASiWDY3F3\n2GTe28MMoSNsHvu8fpYCPO0oxOmyFgDAydIWeJnx4W0u6GKk4ZoyWYJbuXVokqtwr1KB0xfL8OyT\n3fteJISQznT6bScSiRAaGopTp04hKiqKPR4fH48ZM2boHDN27FisXr0acrmcnUcfHx8PFxcXeHh4\ndCuo0aNHIy0tjW0zDIM1a9agqqoKX3/9NTw9PXWOk0ql3To/MTxtd0joGhmYlCoG75zJBmPMh9gY\nsDM1woZn/WBt8nB3jAfidXL0eCZq6vkwMxNDZCTA+3+LgMTZokfOHaxiUHs6i53i9N9qEb4Y6wer\nAb5p1+NcJy2ME7buUo//I60Rs6cPhrfn4//AIvpnIH6ekJ5XXV3dI+fp8lnfypUrsXv3buzYsQOZ\nmZlYtmwZZDIZFi9eDACIiYnBpEmT2P7R0dEQi8WYP38+0tPTcfToUWzYsAErV67UOm9ycjKSk5NR\nXV2NiooKJCcnIyMjAwAgFovh7+/P/l9AQACsrKxgbm4Of39/GBkZ1mNdQkjnjt4oRdpddSlGPo+H\nv431euhkfiAqltXi4LEMtv3KywE9lswDgIDPw/8L94KFSJ3Alzcq8H+X86iIQScmTfRCwDAHAIBK\nxWDLzkQolbRJFyHk8XR5G2XmzJmoqKjAunXrUFJSgsDAQMTFxcHNzQ2AeqFrTk4O29/S0hLx8fFY\nunQppFIpbG1tsWrVKqxYsULrvCEh6gVCPB4PDMPg+PHj8PT01DpXe7QolhCiy617DdiXWsy2ZwU4\nY7hjzyWt/RXDMPhmTxKaW5QAAG8PG7z4nE+Pv46DWISVYzyx9twtAMDV4mrE3SrHlKEOPf5aAwGf\nz8PiBaFY+W48mluUyMmrxH9PZePFSF+uQyOE9GPdei66ZMkSLFmyROffdu3a1eHY8OHDcfbs2U7P\n2TYnvrt0vQ4hxLA1taiwMUFTb36YnRlm+ete32NoLl4uZCup8Hg8LFkohUDQOwswRw+ywku+Tjh2\nU/16O5KLEORkAVdLk155vf7ORWKJGdP88O1h9dTSb4+kY3SoC5wdzTmOjBDSX9HyekJIv7UrpQhF\ntU0A1OUUV43xgoBPT/IaGpuxa7+m1nnkpMEY3MvztOcGDoKnlSkAQK5U4Z+/56FFRVNvHmTa88Pg\n7moFAFAoWrBtdxJNVSKEPDJK6Akh/dLV4mr8lH2Xbf81xA0SC+NORhiOA0fTUVndCACwtjLB7Kjh\nvf6axkI+/meMJ4xaf1Bl36vHd2klvf66/ZVQyMfShZq9AFLSZDh76eE2XySEkDaU0BNC+p2qpmZs\nvqJJfsa6WGOylx2HEemPtjnZbebPDoJZH22s5W0jxtxAF7Z9KFOGjLt1ffLa/dHQwXaYEqEpwbxr\nfwqqa5o4jIgQ0l9RQk8I6VcYhsGXVwtQ2dQMALA1McJboz1o0TzUVVO279VM3Rju54gJY937NIaX\nhjkisHVRsophsOlyHhqalX0aQ38yOyoADnZmAIDaOjn2HUrlOCJCSH9ECT0hpF85lVOB3+9Use1l\nYR4Dvu55d50+n4ubtyoAAAIBH6/PC+nzHzp8Hg8rwzxhZqTeYKqkTo7tfxT1aQz9iamJEf4yP4Rt\n/3pO878hIYR0FyX0hJB+405tE7YlFbLtqUMdIZVYcRiR/qiplWPvgets+0+RvnBzseQkFkczEd4I\n1TwZOJVTjouFlZzE0h+EBkkwKngQ296+J4lq0xNCHgol9ISQfkHFMNh8OR/y1kTHzdIE84Ncuhhl\nOPYdSkVdvQIA4GBnhhl/8uM0nic9bTHR3ZZtf5VYgHuNzRxGpN9e+3MwRK0bdOXkV+LUGd17shBC\niC6U0BNC+oX/Zt9Ferl6gaWAx8OqMV4wEdJHGADcyC7HL2c1CeCiucEw1oNpSEtC3eDQuiC3Rt6C\nr67mU2lG1mF4AAAgAElEQVTGB3ByMMP0F4ex7W8Pp6GqmhbIEkK6h74NCSF6T1Ynx+7rd9j2DD9n\nDLEVcxiR/lAqVfhmTxLbHhU8CKNCBnUyou9YGAuxIsyDbV8ursa5App68yAvRvrC2Um9uVR9gwL/\nPnS9ixGEEKJGCT0hRK+1VbVpalFPtXG3NMUrAc4cR6U/fjmbi7wC9SJhkUiI1/4czHFE2oKcLPH8\nEAe2vTWpEJU09UYnkZEAi9r973fmfB4ys8o5jIgQ0l9QQk8I0WsncyqQXFoDQF1BZXmYB0QC+ugC\ngLo6Bb49nMa2X35hGJwczDiMSLcFQS5aU2+2tlvYTLSFBEkwRurKtrfvpQWyhJCu0bciIURv3W1Q\nYEe7kofTfB3ha6d/CStXDh5LR22dHADgaG+Gac/7chyRbmIjAd4apal6c6GwkqredGJBdBC7QDav\noAonfr3NcUSEEH1HCT0hRC8xDIOvrhagoUW9KZGLhQn+PFw/5obrg4Kiavz86y22/ersIIhEAg4j\n6lyoxAoR3vZsO/ZaIWrkLRxGpL8c7M0w40VNlaLvjqShsooWyBJCHowSekKIXjqTfw+JJdUAAB54\nWDbaA8ZU1QaA+sfOzv3JUKnUFWMChjlgjFT/S3guHOkCWxMjAEBlUzO2/0FTbx5kaqQPBjmrd9xt\naGzG3oMpHEdECNFn9O1ICNE7lY3N+CZJM9XmhaEOCHAw5zAi/ZL4Rwmup5cCAHg8Hhb+ObjPd4R9\nFOYiId5sN/XmdN49XG636y/REBkJsGiuZoHs2Yv5tIMsIeSBKKEnhOgVhmGw5VoBahXq6RjOZsZ4\ndQRNtWmjaFZi57fJbDviKW94ultzGNHDCXOxxlMemg2nvk4sQJ2Cpt7oMjLQWWuB7M59mqcyhBDS\nXrcT+i1btsDLywumpqaQSqW4cOFCp/1TU1MxceJEiMViuLq64qOPPtL6u0wmQ3R0NPz8/CAUCrFg\nwYIO59i+fTsmTJgAW1tb2NjY4Omnn8bFixe7GzIhpB9KKKrCpSLNXdu3RrnD1Eh/54b3tZ9OZqO0\nTL3BlrmZCNHTh3Mc0cP7S4gbrI3VU28qGpuxM/lOFyMM17xZI2AkVF//2TkVOHsxn+OICCH6qFsJ\n/cGDB7F8+XKsWbMGycnJCA8PR2RkJAoLdc9/rKmpweTJkyGRSJCYmIjNmzdj48aN2LRpE9tHLpfD\nwcEBMTExCAsL0/m4+OzZs5g9ezbOnDmDy5cvw9fXF88++yxu3brVoS8hpP9raFZiW7upNs8NtsdI\nZ0sOI9Iv96oacfjHDLb9yksBsDA35jCiR2NpLMQSqRvbPplTjpTW0qREm7OjOf7UrnrRvkOpaKA6\n/oSQ+3Qrod+0aRMWLFiAhQsXwtfXF1988QUkEgliY2N19t+/fz+ampqwZ88e+Pv7IyoqCqtXr9ZK\n6D08PLB582bMmzcPtra2Os+zb98+vPHGGwgKCoKPjw9iY2NhYWGBkydPPsJbJYTou73Xi1HeqAAA\n2JgYYf4I/V/o2Zf2HUpFU5N6eoqbiyWefXowxxE9uvFuNhjnZsO2v04shLyF6q3r8tILvrCxNgUA\nVFY34sjxTI4jIoTomy4TeoVCgaSkJERERGgdj4iIwKVLl3SOSUhIwIQJE2BsbKzVv7i4GPn5j/64\nUC6Xo6mpCTY2Nl13JoT0Kzcr6vFT9l22/ZdgV1gYCzmMSL9k367Abxfy2PZrfw6GsJ9X/Vkc4gaz\n1ulUd2qbcChDxnFE+snUxAhzXwlk28dPZEFWWsdhRIQQfdPlt2V5eTmUSiWcnJy0jjs6OkIm0/3h\nK5PJ4O7urnWsbbxMJoOHh8cjBbtmzRpYWFjgxRdf1Pn3xMTERzovMRx0jegnJcPgi2w56hvVC/58\nLfgwLbuNxLs5nMSjb9cJwzDYvi8X9fUNAAC/oRZobixEYmL/L/s4wbwFR4vUU0h2J96GTU0RnE36\nxw+VvrxOxEYMbK0ZFN5pBABs+Pwk5kQ92ncp6Vv69nlC9MvQoUN75Dy98qnZG+XTNm/ejG+++QZH\njx6FuTmVryNkIDl3twXFrcm8ER94ycWoX5Rh7CvpN2uQX6RO5gV8IPIZZ44j6jlhtgJ4mqm/ilQM\ncKSoGSqGKrncj8/n4YVJEradkVWLW3l0l54QotblHXp7e3sIBAKUlpZqHS8tLYVEItE5xtnZucPd\n+7bxzs4P/0X0+eef4/3338eJEycglUof2K+zvxHD1naHhK4R/SOrkyMhNwNiM/X86QVBLnjWj5uE\nVR+vE0WzEjsOnISZmRgAMPU5Hzw7eSTHUfUsx6GNWHYyEy0MgzIAFTbuiBziwHVYD8TVdSIFUCAz\nYadeJVyTY8ZLT0Ag6B9PNAyNPn6eEP1TXV3dI+fp8lNAJBIhNDQUp06d0joeHx+P8PBwnWPGjh2L\n8+fPQy6Xa/V3cXF56Ok2mzZtwvvvv4+4uLgHvh4hpH9iGAax1wohV6qTeS9rU0zzdepilGE5+ett\nrTKVM1705ziinudpbYooP83/7rtS7uAeVXLR6c8zA2Fior4XV3inGidP3+Y4IkKIPujWz/qVK1di\n9+7d2LFjBzIzM7Fs2TLIZDIsXrwYABATE4NJkyax/aOjoyEWizF//nykp6fj6NGj2LBhA1auXKl1\n3uTkZCQnJ6O6uhoVFRVITk5GRoamJNvGjRsRExODHTt2YMiQIZDJZJDJZKipofJmhAwE5wsrkVii\nvjvBAw9vjfKAkE9TbdrU1snx/X80n4kz/uQPc3MRhxH1nlf8JRjUWoKzvlmJb5L6//qA3mBrbYrp\nU/3Y9oGj6aitk3cyghBiCLpVQmLmzJmoqKjAunXrUFJSgsDAQMTFxcHNTV1HWCaTISdHs3jN0tIS\n8fHxWLp0KaRSKWxtbbFq1SqsWLFC67whISEA1HPuGYbB8ePH4enpyZ5ry5YtaGlpwSuvvKI1bv78\n+di5c+ejv2tCCOfqFC3Y3q7m/JSh9vC1M+MwIv1z+MdM1NWry3g6OZrjuUn9t0xlV4yFfCyVuuPd\n37IBqH/sPVNcjVGDrDiOTP+88KwP4s/morSsDnX1Chw8lo5Fc0O4DosQwqFu14RbsmQJlixZovNv\nu3bt6nBs+PDhOHv2bKfnVKk6rzmcm5vb3fAIIf3M7pRi3GtST6uwMzXCvECqOd9eiawWcfGaTfTm\nvaLZMXSgGulsiac97XA6rwIAsCWxAFsi/Wmn4PuIRALMnx2EDZvVO6ef+PU2nn16CNxcaBM2QgwV\nraQhhPS5mxX1OHG7nG3/NcQNZiJK2trb930qlK1rC4b52GOM1DB+8CwKdoVl6/4DZQ0K7E8r4Tgi\n/TQ6ZBCG+zkCAFQqBnu+S+E4IkIIlyihJ4T0KaVKvRCWgbo0oVRihXBXa46j0i+ZWeVIuKqZjjR/\nVpDBlPG0MhZi4UhXtv1jVhlyqxo4jEg/8Xg8LIgeCUB9XSRdL0FSCv34IcRQUUJPCOlTJ3PKkX2v\nHgAg4vOxOMTNYJLV7mAYBru/1dxtHT/GHT5D7DiMqO8942mLQEcLAOpNx7YkFlJteh28PKwxaaIX\n2979XQr7VIcQYlgooSeE9JmqpmbsvV7Mtqf7O0FiYcxhRPrn4uVCZOeo55ALhXxETx/OcUR9j8fj\n4Y1QNwhbf+hllNfhl9wKjqPST7OjhsPUxAgAUFRcQ2UsCTFQlNATQvrMnuvFqFW0AAAk5saYPmzg\n7HjaExQKJfYdSmXbL0QMhbOjYe6M7W5lipeHademr5G3cBiRfrKxNkHUi5oylgePZaCuTsFhRIQQ\nLlBCTwjpExl363AqR3shrLGQPoLai/vlFsrK1dORLMyN8XK7euOG6JUACRzF6rr7NfIW7E65w3FE\n+umFiKFwtFeXfK2tk+P7HzO6GEEIGWjo25QQ0uuUKgZb220UNNbFmuqL36emVo7D7TaRmjnNH+Zm\nA3MTqe4yEfKxONSNbZ/MKUdmeR2HEeknkUiAebNGsO24+Fu4U0IbMBJiSCihJ4T0uv/euovblepK\nJcYCPl4Pce1ihOH5/ocMNDSq6/JLnCwQ8bQ3xxHphzAXa4xx0VRB2pJYCKWKFsjeb+woV/j5OAAA\nlEoV9h64znFEhJC+RAk9IaRXVTY2Y1+qZiHsK/7OcDKjhbDt3SmpwYl2ixnnzRr4m0g9jNeDXWEs\nUH9d5VQ14Hh2GccR6R8ej4cFc4LQVsby6h/FSEkv5TYoQkifoYSeENKrdqbcQX2zEgDgamGCl9ot\ndCRq+w5pNpHy83HA6JBBHEekX5zNjTE7QMK296WWoLyBFn7eb4iXLZ6a4MG2d+1PpjKWhBgISugJ\nIb0mtawWp/M05QYXh7pBJKCPnfbSb9zF5WuaxZ7zZ4+guvw6TPN1hJulCQCgsUWJf/1R1MUIwzRn\neiCMW3faLSiqxi9nczmOiBDSF+iblRDSK1pad4RtM97NBsHOlhxGpH9UKgZ7Dmg2kZow1h1DBxvW\nJlLdZSTg4w2pO9s+X1iJJFr42YGtjSlefmEY2/7uSBrq6WkGIQMeJfSEkF5xPKsM+dWNANTVShYF\n00LY+134vQC3cu4BAIyEAsyZEchxRPpthKMFnva0Zdux1wogb6EpJff7U6Qv7O3EANTVk478mMlx\nRISQ3kYJPSGkx5U3KLA/rYRtzw6QwEFs2CUY76dQKLHve80mUlOe1dQSJw/2WpArzIzUC4aL6+Q4\ncoMWft5PJBJg7kxNGcufTmVDVkrlPgkZyCihJ4T0uB3JRWhsUS+Edbc0xTRfWgh7v59OZaO8Ql3K\n08LcGFFTh3UxggCAjakR5o1wYdvfZ8hQXNvEYUT6afwYN/i0Tt9qaVFh70EqY0nIQNathH7Lli3w\n8vKCqakppFIpLly40Gn/1NRUTJw4EWKxGK6urvjoo4+0/i6TyRAdHQ0/Pz8IhUIsWLBA53mOHDkC\nf39/mJiYICAgAD/88EM33xYhhCvJshqcK6hk20ukbhDyaZFne9U1TTh6XDMN4pWX/GFGTzC6LXKw\nPYbaqp9mKFQqbL1WCIah2vTt8Xg8vDZnJNv+PbEIaZlU7pOQgarLhP7gwYNYvnw51qxZg+TkZISH\nhyMyMhKFhYU6+9fU1GDy5MmQSCRITEzE5s2bsXHjRmzatIntI5fL4eDggJiYGISFhems6JCQkIBZ\ns2Zh7ty5SElJwZw5czBjxgxcuXLlMd4uIaQ3NStVWgthn/SwxQhHCw4j0k+H2m0i5SKxRMRTgzmO\nqH8R8Hl4I9QNvNaa69dkNbhYVMVxVPrHZ4gdnhirKWO5+9sUKmNJyADVZUK/adMmLFiwAAsXLoSv\nry+++OILSCQSxMbG6uy/f/9+NDU1Yc+ePfD390dUVBRWr16tldB7eHhg8+bNmDdvHmxtbXWe5/PP\nP8fTTz+NmJgY+Pr64p133sGTTz6Jzz///BHfKiGkt/1wswxFrdMfxEIBFo6khbD3u1NSg5PtN5F6\nZQSEQpr9+LB87Mzw/BB7tr09qQgNrfsdEI0/zwyESKQuY5mTX4nfLuRzHBEhpDd0+i2iUCiQlJSE\niIgIreMRERG4dOmSzjEJCQmYMGECjI2NtfoXFxcjP7/7HyS///77Q70uIYRbZfUKfJeuWQg7J1AC\nW1MjDiPST3sPXIdKpZ4eEjDMAdJgSRcjyIPMGzEI1sbqa6y8Ufv6I2r2dmJMe96Xbe8/nMo+HSKE\nDBydJvTl5eVQKpVwctJe0Obo6AiZTKZzjEwm69C/rf2gMQ9znoc5ByGk72z/oxDy1sf5XtammDrU\nkeOI9E9aZhmu/lHMtl+dHUSbSD0Gc5EQC4M1C2T/c7MMeVWNHEakn6Y97wsba1MAQFV1E479dIPj\niAghPU3Y0yfk8sspMTGRs9cm/QNdI73jRo0Sv+RqNq+ZKGnBH0nXOIzo8fTGdaJSMYjdcxv19eop\nScHDrVFVkYPEipwefy1DYsEwcOYpkFOn/jG59kQS3hgs6pPvov70eTImxBSHj6t3bf72cBIcbWph\nY00LsftCf7pOSN8bOnRoj5yn0zv09vb2EAgEKC3VrvNbWloKiUT3Y2JnZ+cOd9Hbxjs7O3c7sAed\n52HOQQjpfc0qBj8Uax7hh9oI4GUm4DAi/ZSSUY1imTqZNxLyMHkilfLsCTweDy+5GKGtkFJevQrX\nKmku/f2C/K3gKlHfpW9pYXDyN6rfT8hA0ukdepFIhNDQUJw6dQpRUVHs8fj4eMyYMUPnmLFjx2L1\n6tWQy+XsPPr4+Hi4uLjAw8ND55gHnSc+Ph6rVq3Set1x48Y9cIxUKu32+YlhabtDQtdIz/surQSN\nwmKIhYC5kQBrng+AtUn/nDvfW9eJQqHEN/t/hpmZevfOqKl+eOZp2hW2J921uIPDmeqbQOfqhZgz\nMQAWxj3+EBpA//08sbD2wrvrTgMAcgqaYWbpCT8f+y5GkUfVX68T0reqq6t75DxdllZYuXIldu/e\njR07diAzMxPLli2DTCbD4sWLAQAxMTGYNGkS2z86OhpisRjz589Heno6jh49ig0bNmDlypVa501O\nTkZycjKqq6tRUVGB5ORkZGRksH9ftmwZTp8+jQ0bNuDGjRv4xz/+gd9++w3Lly/vkTdOCHl8sjo5\nDmVqnqTNHeHSb5P53vTjiZvsJlJWliZ46QXaRKqnzfJ3Zncjrpa3YG9qcRcjDI+fjz3Ghbmx7Z37\nk9kF2oSQ/q3L2xczZ85ERUUF1q1bh5KSEgQGBiIuLg5ubuoPBZlMhpwczRxQS0tLxMfHY+nSpZBK\npbC1tcWqVauwYsUKrfOGhIQAUD8uZRgGx48fh6enJ3uusWPH4sCBA1izZg3ef/99DBkyBIcOHcKo\nUaN67M0TQh7PtqRCKFoXwg6xESNyMN3tu19lVROOtluEOOvlAIip+k+PMzUS4C/Brlh/Uf0d8vOt\nckzysoOvnRnHkemXua+MwJVrxWhuUeJ27j2cS8jHk+M8uQ6LEPKYuvU8csmSJViyZInOv+3atavD\nseHDh+Ps2bOdnlOl6npzi6ioKK2pPoQQ/XH5ThWuFKsfFfLAwxtSdwhoR9gODhxLQ1NTCwDAzcUS\nkyZ6cRzRwDXW1RpSiRUSS6rBgMGWxAJsmjyMrst2HO3N8GKkD4607lT874OpGBPqChOT3pmeRAjp\nG7SbCSHkoclbVNiWVMS2nx1Md0J1KSiqxi+/5bLtV2cFQSCgj93ewuPxsCTUDaLWf+NblQ34+XY5\nx1Hpn5deGAYbK/UC2cqqRvwQR2UsCenv6JuFEPLQDmXIUFovBwBYiISYN8KlixGGafd3KWAY9Rzl\noOHOCB5BVbp6m7O5MWb4af6d916/g0raSEmL2NQI0dOHs+0f4rLYNR6EkP6JEnpCyEMpqmnCkRua\nhbDzg1xg1UvVRPqzpJQSJKeq/514PB7mzx5Bm0j1kahhThhkrq6yVt+sxM6UOxxHpH+eHO8Bbw8b\nAIBC0YJ9h1I5jogQ8jgooSeEdBvDMNh6rRDNrZUxfO3MEOFtx3FU+kepVGHPgetse9JEL3i4WXMY\nkWExFvKxJNSdbZ/Oq8D1sloOI9I/AgEf86OD2Pa5hHxk3argMCJCyOOghJ4Q0m0XCqvwR2kNAIDP\n42Gp1B18uuvcwS9nc1F4R71g2MREiFkvD+9iBOlpIRJLjHezYduxiYVooRKNWob7OWKM1JVt79yf\nzE4RI4T0L5TQE0K6paFZie1/FLLtKUMcMNhGzGFE+qmhsRkHjqaz7ZdfGAYbaxMOIzJcrwe7wlSo\n3rW4oKYRP9yk3VHvN++VERAK1alA1u0KXPi9sIsRhBB9RAk9IaRbvk0rQUXr4kIbEyPMDRzEcUT6\n6dhPN1Bd0wQAsLMVY+qzPhxHZLjsxSLMGS5h29+mleBug4LDiPSPs5M5Xmh3jf770HXI5S0cRkQI\neRSU0BNCupRX1Ygfs8rY9sKRLjATCTiMSD/dLa/H8RNZbHvOjOEwpgXDnJrq4wjP1hKNcqUK3yTR\nHej7TX/RD5YW6kXE5RUN+LHdNUwI6R8ooSeEdIph1Bv0KFvn1gY6WuBJD1uOo9JP+79Pg6JZCQAY\n7GWLJ8Z6cBwREfLVm561uVRUhcSSag4j0j9iUyPMjtKs8zhy/AbuVTZyGBEh5GFRQk8I6dSvefeQ\nXl4HABC2btxD5Rc7yr5dgXMJ+Wx7/uwg8GmHUr0Q4GCOSV6aakxbrxVC3tL1buWGpH0lJoWiBfu+\npzKWhPQnlNATQh6oVt6CncmaHWGnDXOCR+v0BaLBMAx2fZfCtsNCXRAwzIHDiMj9FgS5wEKknv5U\nUifH4XZ7KZCOZSx/u5CHW7n3OIyIEPIwKKEnhDzQ3tRiVLcukHMQizDLn3Y61eX3xDu4kVUOQJ0Y\nzX1lBMcRkftZmxjh1RGahdyHM0pxp7aJw4j0T1CAE0YFa/6NqIwlIf0HJfSEEJ2yKurx861ytv16\nsCtMjWgh7P2aW5TYe1CziVTkpCEY5GzBYUTkQSK87eFrZwYAUKhU2HqtkBLW+8ybNQICgTo1uJFV\njoSrRV2MIIToA0roCSEdKFUMtlwrBAN1shMqsUK4K+10qsuJX26jtEy9xsDcTIQZf/LjOCLyIAK+\neg1I22ZoSbIanC+o5Dgq/eIiscTzk4ew7b0HrkOhUHIYESGkOyihJ4R0cDKnHNn36gEAIj4fi0Nc\naSGsDrV1chz6IYNtz/iTPyzMjTmMiHRlqK0ZpgzRrG/45o8i1FLddS0zXtRcx2Xl9fjpJJWxJETf\nUUJPCNFS1dSMPSl32PZ0fycMsqCdTnX57kg66ls3KnJ2MsdzkwZzHBHpjnkjBsHO1AgAUNnUjD3X\nizmOSL+Ym4vwykv+bPvI8RuorKL1BoTos24l9Fu2bIGXlxdMTU0hlUpx4cKFTvunpqZi4sSJEIvF\ncHV1xUcffdShz9mzZxEaGgpTU1MMHjwY27Zt69Dns88+g6+vL8RiMdzc3PDmm2+ivr6+m2+NEPIo\ndqXcQV1rLXWJuTGmD6OFsLrkFVTh5OnbbHvB7JEwEtIag/5AbCTAX0Pc2PbPt+8i424dhxHpn2ef\nHgw3F0sAQGNTM747ksZxRISQznSZ0B88eBDLly/HmjVrkJycjPDwcERGRqKwUPduezU1NZg8eTIk\nEgkSExOxefNmbNy4EZs2bWL75Obm4vnnn8f48eORnJyMmJgYvPXWWzh69CjbZ+/evXj33Xfx/vvv\n48aNG9i7dy/i4uKwbNmyHnjbhBBd0spq8UtuBdteHOoGYyE9yLsfwzBaFUBGBDhBGizhOCryMMJd\nrRE2yIptf51YgBYVLZBtIxDwMX/2SLb9y9lc5OTRegNC9FWX39SbNm3CggULsHDhQvj6+uKLL76A\nRCJBbGyszv779+9HU1MT9uzZA39/f0RFRWH16tVaCf3WrVvh6uqKzZs3w9fXF4sWLcKrr76Kf/7z\nn2yfK1euYMyYMZgzZw7c3d3x1FNPYe7cubh8+XIPvG1CyP0UShW+vFrAtsNdrSGVWHUywnD9nngH\naZllAAA+n4fX5oykNQb9DI/Hw+JQd5i0/mDNq27EsRulHEelX4JHOCNkRNsPVQa7vqUyloToq04T\neoVCgaSkJERERGgdj4iIwKVLl3SOSUhIwIQJE2BsbKzVv7i4GPn5+WwfXedMTEyEUql+1B8ZGYmU\nlBQ2gS8oKMCPP/6IKVOmPORbJIR0x5HMUhS11uU2FQrwl3ZTEoiGQqHEnnabSEU+MwTurvTDpz9y\nNBPhz8M1dde/TS+BrE7OYUT659V2Ox6n37iLi5d1P50nhHCr04S+vLwcSqUSTk5OWscdHR0hk+ne\nZU8mk3Xo39ZuG1NaWqqzT0tLC8rL1XWvp0yZgvXr12PChAkQiUTw9PREUFAQPvnkk4d4e4SQ7iiq\nacKhDM1/0/NGDIKDWMRhRPrrxxM3UVauXstjYW6MV14K4Dgi8jhe9HHEYBsxAPVTqi2JBXQXuh03\nF0tETtKUsdz1bQoaGps5jIgQoouwp0/YU4+djx07hnfeeQdbt25FWFgYsrOzsWzZMnzwwQdYu3at\nzjGJiYk98tpk4KJrpCOGYfBNjgJVdSoAgKuYD6fqfCQmFnQxcuB60HVSXdOM3d9moblZnfBNGm+N\nGzeu6+xL+o8nTVVILVLfmT9/uwE7Wiow0rrrr0dD+Tzx9VQiDgrU1begvr4Bm748ieeepsXy3WUo\n1wl5NEOHDu2R83R6h97e3h4CgQClpdrzCktLSyGR6F4A5uzs3OHufdt4Z2fnTvsIhULY29sDAD75\n5BMsXLgQr732GgICAjBt2jR8/PHH+PTTT6FSqR7iLRJCOnOtUolbrck8nwdMdzWCgOaD63TyNxmb\nzDs7mkA60objiEhPcBfzMc5eU6Hox+IWNLTQXfo2JiYCrQT+0tVylN6lMpaE6JNOb0GIRCKEhobi\n1KlTiIqKYo/Hx8djxowZOseMHTsWq1evhlwuZ+fRx8fHw8XFBR4eHmyfY8eOaY2Lj4/HqFGjIBCo\nP1QZhgGfr/17g8/nd/ooVCqVdvZ2iAFru0NC14i2ankLPotLh9hMXZP7JV8n/CnYleOouNPZdZKZ\nVY5beTkwM1NPz1j11pMI9Hfs0/hI7/EPUmJxXDoqGpuhBJDMt8fbUg+dfQ3x8yQ0lEFe0W/IzLoL\nALiS0oIPV4fSYvBOGOJ1Qh5edXV1j5ynyyo3K1euxO7du7Fjxw5kZmZi2bJlkMlkWLx4MQAgJiYG\nkyZNYvtHR0dDLBZj/vz5SE9Px9GjR7FhwwasXLmS7bN48WLcuXMHK1asQGZmJv71r39hz549WLVq\nFdtn2rRp+Oabb3Dw4EHk5uYiPj4e7733HqZOndoh0SeEPJpdyUWoad0l00EswpzhVHpRF5VKXaay\nzdhRrpTMDzD316Y/mVOOlNJaDiPSLzweD6/PC2YXyKZmlNECWUL0SJeTBGfOnImKigqsW7cOJSUl\nCBg8PkQAACAASURBVAwMRFxcHNzc1B98MpkMOTk5bH9LS0vEx8dj6dKlkEqlsLW1xapVq7BixQq2\nj6enJ+Li4rBixQrExsbCxcUFX375JV566SW2z+rVq8EwDN577z0UFRXBwcEBU6dOxfr163vy/RNi\nsK6X1SK+Xc35JaFuMDWijZF0OXM+D7dz7wEAjIQCzJsVxHFEpDeMc7NBuKs1LhVVAQC+vJqPr57z\nZ0tbGjpPd2s8P3kofjqZBUC9QDYkSAJx6667hBDu8Jh+vpy//aMKKysqHUd0o0ef2hRKFd46kcmW\nqRznZoN3xnlzHBX3dF0ndfUKvLX6BKpr1P9W01/0R/T04ZzER3rfvcZmLI5LR33rbskvD3PCwpHa\n09AM+fOkvkH930NVtfq/h2nP+9IP3Acw5OuEdF9P5bF024EQA/R9poxN5sVCAf5qwPPmu/LdkTQ2\nmbezFePlF4ZxHBHpTbamRljULoH/4WYZsirqOYxIv5iJRZg3awTbPn4yGwVFPTMHmBDy6CihJ8TA\n5FU14lC6ds15O6o5r9PtvEqc+PU2235tzkiYmPR4tV+iZyZ72yHIyQIAoGIYfHE1Hy2qfv0wu0dN\nDPeAn48DAECpVGHHvj+odj8hHKOEnhADolS1JietX77D7Mzw/BAHjqPSTyoVg292J7GJStBwZ4yR\nunAcFekLPB4Pb4/ygLFA/RWZW9WII5m6N1M0RDweD395lRbIEqJPKKEnxID8mFWGm63TB4z4PLw9\n2gMCPpWd0+X0+Vxk56gXDQuFfLw+L5hK9BkQZ3NjzA0cxLa/Sy9BQXUjhxHpFw839QLZNrv2p6Cu\nXsFhRIQYNkroCTEQJbVy/Du1mG3PCpDAw8qUw4j0V22dHP8+mMq2pz0/DIOcLTiMiHDhRR9H+Nia\nAQCaVQy+uloAFU0tYc16OQA21urPkMrqRuw7lNrFCEJIb6GEnhADwLTOA5Yr1TvCelmbYrofbd3+\nIPu+T0VtnRwA4GBnhqiptBDWEAn4PCwb7QFh65OZ9PI6/Df7LsdR6Q+xqREWzQ1m26fO3EZmVjmH\nERFiuCihJ8QAnLhdjutl6k1y+LzWJIWm2uhUVNyA+DO5bHvR3GAYG9NCWEPlaW2K6f6aH797rhfj\nnkLFYUT6ZYzUBdJgzdSkrbsS0dyi5DAiQgwTJfSEDHB3GxTYlXKHbb88zAlDW6cREG0qFYMfTxYD\nUE+rCA2SQBpMu+caulf8neFmaQIAaGxR4lBhM029acXj8fB6ux+9hXdq8J+4mxxHRYjhoYSekAGM\nYRhsSSxgN8lxsTBBdAAlqA+SmFyJOzJ1zXmRkQAL59JCWAKIBHysCPMEv/VauF2nQkIF3YVu42Bv\nhugozWZr3/+QiWJZLYcREWJ4KKEnZAA7V1CJK8WaTV/eGuUOY9rGXqfqmiacOlvKtl96YRicHc05\njIjoE187M0QNc2Lb/y1pRnHr5mwEeH7yEHh72gAAmluU2Lb7GtWmJ6QP0Tc7IQNUVVMztiZpakM/\nP8QBgY5UqeVBdu1PQWOT+q6rk6M5XppCC2GJtujhEni2VoZqVgGfX86HkjacAgAIBHwseU3KPtFK\nzSjDbxfzOY6KEMNBCT0hAxDDMNhyrRA18hYAgINYhAVBtCnSg/xxXYZzCZrk4/W5wRCJBBxGRPSR\nZuqNup1eXof/ZJVxG5QeGexpgxee1dSm3/1tCmpq5RxGRIjhoISekAHot/x7uFhYybbfHOUOsREl\nqLo0NbVg2+5rbDvI3wohQbTOgOg2xFaMZxw1VY/+fb0YhTU09abNrJcDYG8nBqDez2H3dykcR0SI\nYaCE/v+3d9/hUVX5/8Dfd2oyk14nvRECoYYkQAIRlKKw6iI21FXBigUpiyJLWZWmoBRdQBRXWH+4\nsgqW7y4KUUoICSVAgBQSIJ1khkzapE2/vz8muWFIBQIzmfm8nmeeyT1z7uRMOJz7mXNPIcTGKJu0\n+Px021CbByK8EOfnasESWbd/783CNaVp91yJIx9TJ1IwT7o2wVeAAEdTN73WaMSGE0U09KaFo4MQ\nLz83gjs+nFqEc1lyC5aIEPtAAT0hNsS0gVQJGlpWtZFJxXhxeKCFS2W9LhdU47/7L3HHUybI4CSl\nNedJ1/gMgyeDRNyGU3lVjdhzUdHNWfYjPsYfCfFB3PHmrzLQ1KyzYIkIsX0U0BNiQ367osTpCtOq\nNgwYzBsVQkNtOqHXG7H1nxncShxDB/kiZrCbhUtF+go/Rx6eHtx2N2fXhXIU1TZbsETW5aVnY+Ds\nJAYAKKua8K/vzlu4RITYth4F9Fu2bEFYWBgcHR0RFxeH1NTULvNfuHAB48aNg0QiQWBgIFasWNEu\nz5EjRxAbGwtHR0dERERg27Zt7fKoVCq89dZbCAgIgIODAyIjI/H999/38KMRYl8q6jXYnlnGHU+L\n8qFVbbrwy2/5KCypBQCIRALMnhVLa86Tm/LYQBmiPE2btOlZFh8fL4TWQLvIAoC7mwNefi6GOz5w\n6ArOZdNdDELulG4D+t27d2PevHlYunQpMjMzkZiYiClTpqC0tLTD/CqVCpMmTYKfnx8yMjKwadMm\nrFu3DuvXr+fyFBYWYurUqRg7diwyMzOxePFizJkzB3v37uXy6HQ6TJo0CVeuXMH333+P/Px87Ny5\nE2FhYb3wsQmxLQYjiw0niqDWm4KJIBcHPDvEv5uz7FeFvB7/+TGbO37ykWhac57cND6PwbyRIRDx\nTZfSwtpm/Ot8uYVLZT3GjArCqNi21bW2bKehN4TcKd0G9OvXr8esWbPw4osvIioqCp9++in8/Pyw\ndevWDvPv2rULarUaO3fuRHR0NB599FEsWrTILKD//PPPERgYiE2bNiEqKgovvfQSnn/+eXz88cdc\nnq+//hpVVVX4+eefkZiYiODgYCQmJiIuLq4XPjYhtuXn/GvIVjYAMI3vXTAqlDaQ6gTLsti24wy0\nLfMMwkPc8fAD/S1cKtJXBbs64oXrloT9MU+BMxUqC5bIejAMg1eej4WTVAQAqKxqpKE3hNwhXV7x\ntVotzpw5g8mTJ5ulT548GWlpaR2ek56ejqSkJIjFYrP85eXlKC4u5vJ09J4ZGRkwGEwX2Z9++gmJ\niYl444034Ofnh0GDBuH999+HXq+/+U9JiA0rqm3GN9f1Cj4RLUP/lmEApL1DR4twPsd0659hGMx+\nIRZ8Pn35IbfuwUhvs5WkNpwoQq2aeqKB1qE3bave0NAbQu6MLpdzUCqVMBgM8PX1NUv38fGBXN7x\nMlRyuRzBwcFmaa3ny+VyhISEQKFQtHtPX19f6PV6KJVK+Pr6oqCgAIcOHcIzzzyDffv2obCwEG+8\n8QYaGhqwbt26Dn93RkZG15+W2D1bqyN6I4t/XNagttk0sTPAkUGk+ioyMui2f0dU9Tp8uv0ytyPs\n2JGeqK0qQEZVgVk+W6sn5M64vp6MF7E4o1GjQQ80NQJL/u8kZoaKaF4GAAcBi5AAPnLy6wEAqz/5\nHXNe7AcHsX1M2Kf2hHQlMjKy+0w90OvdUr3VeBmNRvj6+uLLL79ETEwMpk+fjg8++KDToT6E2KP9\nCj2utgTzAh4wI1gEPgUQHWJZFj/+epUL5j3chJiQ5NvNWYT0jLPQtJRlqxyVEcerDRYskfVgGAYP\n3+8PiaMpgK+t0+HAYeqlJ6Q3ddlD7+XlBT6fD4XC/D+eQqGAn1/Hm6/IZLJ2vfet58tksi7zCAQC\neHl5AQD8/f0hEpn3bgwYMABNTU2oqqqCp6dnu99N4+tJZ1p7SGypjpypUOHk5UuQtIxPfTkmEA9G\nUYDamd+PFOCqvBBSqWkXyyULx2PwQB+zPLZYT0jv66yexAFoPFOKX/KvAQAOqXj48+gBCHZ1vNtF\ntEp8kT82fn4CAJCVp8Zj04IwdJDttlnUnpCeqKur65X36bKHXiQSITY2FgcOHDBLT05ORmJiYofn\nJCQk4OjRo9BoNGb5AwICEBISwuVJTk5u957x8fHg803f4MeMGYNLly5xa0QDQH5+PqRSaYfBPCH2\npFatw/oTRdxxnJ8rHu7v0/kJdk5R2Yh/7srkjh+8v3+7YJ6Q3jBzaABCWwJ4jcGIdelFtJRli6SE\nYMTHtK2+9Y/tp9DQqLVgiQixHd0OuVmwYAF27NiBr776Crm5uZg7dy7kcjlmz54NAFi8eDEmTpzI\n5X/66achkUgwc+ZMZGdnY+/evfjoo4+wYMECLs/s2bNx9epVzJ8/H7m5udi+fTt27tyJhQsXcnle\ne+01VFdXY+7cucjLy8P+/fvx3nvv4fXXX+/Nz09In2NkWaw/UYyalkl37g5CzB8VAh4NtemQ0chi\n8/ZTUKtNE+oD/Fzwl8eHWLhUxFaJBTy8nRAGEc90eS2obaKlLFswDINXZ7WteqOsasIXO8+YddwR\nQm5NtwH9E088gY0bN2LlypWIiYlBWloa9u3bh6Ag07bOcrkcBQVtE8pcXFyQnJyM8vJyxMXFYc6c\nOVi4cCHmz5/P5QkNDcW+ffuQkpKCmJgYrFmzBp999hkeeeQRLk9gYCAOHDiA06dPIyYmBq+99hpe\nfPFFrFq1qjc/PyF9zi/517jdYAFgwahQuDkILVgi6/br75eRlWsaAsEwDOa8HA+RyD4m4xHLCHVz\nxKzh5ktZZlT0zm31vs7DzRGvvdA2BCX1eAmOpBVbsESE2AaG7eNfja8fe+Tq6tpFTmLPbGUs4+Xq\nJvw1+SL0Lf9tHx3gixeGB1q4VNbraoUKf132O7RaU+/8ow8NxDNd9M7bSj0hd1ZP6gnLsngv5QoX\nyLuIBfj0/oHwlog6PceebN5+Cn+kFAIAHB2E+GTlJJvb3I3aE9ITvRXH0uLLhPQRzToD1qUXcsF8\npIeUdoPtgsFgxGdfnOKC+ZAgNzw+LdrCpSL2gmEYzB8VAk9H090zlUaPj9IKoTf26T60XvPCX4ZD\n5msK4JvVOmz6/AQMNNeAkFtGAT0hfcQXZ8tQVq8GADgK+Hg7IRRC2hCpUz//mo/8K1UAAD6fZxpq\nI6ShNuTucXMQYlFiOLeUbK6yATvOXbVwqayDo4MQ82eP4jZ1y7tche9/zrVwqQjpuygaIKQPSCmu\nxoECJXf8WmwQApwdLFgi61ZcWovv9mZxx09Mi0Z4qLsFS0Ts1SBvJzw3tO1O2o95ChwrrbFgiaxH\nZIQnZjwyiDv+/ucc5OYruziDENIZCugJsXLFdc3YdKpt0ti9IR64L9TDgiWybhqNHuu3nIBeb7p9\n3y/cA9MfHGDhUhF7Nn2AL0b5t42N3XSyGBX1mi7OsB/T/hSF6ChvAKZ5B5s+P4HGJlrKkpCbRQE9\nIVasSWfA6tQCqFuCU38nMV6PC6bt5Lvwz28zUXrVNMlIJBJgzssjudv6hFgCj2Ewf1QofKViAECj\nzoAP0wqg0dOYcT6fh7mvjoS0ZbLwNWUjtv/rrIVLRUjfQ1c5QqwUy7LYeLKYGzcv5vOwZGwEJDQO\nvFPHTpQi+VDbMrovPDMcQQEuFiwRISbOYgHeTQyDkGf6Mn65pglfni21cKmsg7eXFLNnxXLHR9KK\ncSi1yHIFIqQPooCeECv1U941s7G2c+KDEepGW8h3RlHZiK3/zOCOE0cGYdL4MAuWiBBz/T2leCkm\niDv+9YoSB4uqLFgi6zFmVBDGjw3ljrftOIPi0lrLFYiQPoYCekKsUNa1enx93WoYf+rnjXtDPS1Y\nIuum1xuxYctxNDWbds/18ZLitRdiaWgSsTp/6ueFccFtc2D+caoEV2qaLFgi6/HSszEI8DPdUdNq\n9Vj7aTr3f5oQ0jUK6AmxMtXNOnyUVghDy3rzUZ5SvBRDm0d15d97s8yWqFzw+mhuTC4h1oRhGLwZ\nH4zAllWqNAYjVhy9ghoKXCFxFOLtOQkQiwUAgApFPTZvP4U+vv8lIXcFBfSEWBG9kcWHaQWoVpsu\n7i5iARaPCYeIJnV26lyWHD/+9yJ3/PRjg9G/H93NINZLIuRjWVIEpC3zYSqbtFh9rABa2lgJwYGu\neP2Ftp1V00+V4f9+y7dgiQjpGyhKIMSK7Dh3FdmVDQBMK2MsSgijreK7UFOrxqZtJ7njYYNl+POU\nKAuWiJCeCXRxwDuJYeC1DAvLUTZg6+lS6o0GkJQQjCkT+3HH/9p9Hjl5lRYsESHWjwJ6QqzEwaIq\n/Jin4I6fHeKP4TJaoaUzRiOLT784ido60ypAbq4OeOuVkeDxaNw86Rvi/Fwxa1gAd3ygQIn/u0SB\nKwA8/9QwRIab7rQZjSw++cdx1NSqLVwqQqwXBfSEWIGcygZ8erJt86jRAW54bKCvBUtk/b7/OQfn\nsuTc8dxXR8LdjXbPJX3LI1E+uO+6Ce9fni3DWbnKgiWyDiIhHwvfHA1nJ9Pa/TV1zdiw9TgMNCyJ\nkA5RQE+IhckbNFiZegU6o+lWe4irIxaMCuVuxZP2zpyrwO4fc7jjRx4cgGGDZRYsESG3hmEYvBkX\njChPKQDAyLL4KK0QV+upN9rbS4r5r40CYGoLs3Kv4ds9WZYtFCFWigJ6QiyoSWfAB0evoE6jB2Ca\nBLs8KQJSEW0e1Rn5tQZs2HoCgOkL0JBoHzz96GDLFoqQ2yAW8LBkTDg8HYUAgHqtHh+kXEGj1mDh\nklne8CEyPDEtmjv+8b8XcewEbchFyI16FNBv2bIFYWFhcHR0RFxcHFJTU7vMf+HCBYwbNw4SiQSB\ngYFYsWJFuzxHjhxBbGwsHB0dERERgW3btnX6fv/+97/B4/Hw0EMP9aS4hPQJBqOpJ664rhkAIOQx\nWDY2ArKWW8ykPa3WgHWfpqGxSQsA8PSQYP5ro8GnVYBIH+cpEWHJ2AiIeKa6XFavxppjBdDREBM8\nMS0aMUPb7sB99uUpFBTVdHEGIfan26vg7t27MW/ePCxduhSZmZlITEzElClTUFra8TdklUqFSZMm\nwc/PDxkZGdi0aRPWrVuH9evXc3kKCwsxdepUjB07FpmZmVi8eDHmzJmDvXv3tnu/goICvPPOO0hK\nSqJNYohN+SqzDBkVddzxWyNDEO3tZMESWTeWZbHlqwwUlph2j+TzeXj7zQS4udK4eWIbojylmDsy\nhDs+q1Dh01PFMNr5yjc8HoP5s0fDX+YMwLTp1IebjtEkWUKu021Av379esyaNQsvvvgioqKi8Omn\nn8LPzw9bt27tMP+uXbugVquxc+dOREdH49FHH8WiRYvMAvrPP/8cgYGB2LRpE6KiovDSSy/h+eef\nx8cff2z2XjqdDk899RRWr16N8PBwWs6L2IxfL1fi5/xr3PGT0TKziXGkvZ/25SElvW3i8EvPxtB6\n88TmjA/1wDOD/bnjg0XV2Hm+3IIlsg5OTiIsnj8GkpZhScqqJqz7LA1aHQ1LIgToJqDXarU4c+YM\nJk+ebJY+efJkpKWldXhOeno6kpKSIBaLzfKXl5ejuLiYy9PRe2ZkZMBgaPvPuWTJEoSHh+PZZ5+l\nYJ7YjEy5CltPt93hGhPkjr8M8e/iDHL6XAW+2X2BO540PhyT7w23YIkIuXOeGiTDAxFe3PEPuXL8\ncl0HgL0K8HPBgtdHc3frL15SYsv2DIoPCEE3Ab1SqYTBYICvr/nyeT4+PpDL5R2eI5fL2+VvPW49\nR6FQdJhHr9dDqVQCAA4cOIAffviBG1vPMAwNuSF9XkFNE1YfK4Ch5QLUz11CK9p0o/SqChu2HEfr\nJNgB/b3w0nMx1B4Qm8UwDF6PDcYof1cu7YszZThWSuPGRwzzw3NPDuWOU9KLsfvHbAuWiBDrIOjt\nN+yNi2xlZSVmzpyJ7777Di4upo11WJbt9lt4RkbGbf9uYtssWUeqtUb847IW9TpTPXYRMnjI2Yis\nzDMWK5O1a2zS4/N/FaC6xjQJ1s1FiMlJEpzLPHtHfy+1JaQn7nQ9mejA4gqrRUmTaWLse79n4eUw\nEcKd7HsVLH9vFtGRYpzKNH3B+XpXBurrKhAzxN3CJesYtSekK5GRkb3yPl320Ht5eYHP50OhUJil\nKxQK+Pn5dXiOTCZr13vfer5MJusyj0AggJeXF7KzsyGXyzFhwgQIhUIIhUJ888032LdvH4RCIS5d\nunRzn5IQC2vUs9he0BbMO/CBl8JEcBNRL3NndDojdu0p4YJ5oZDBM48Gw0na6/0QhFglEY/BC2Ei\neItN7YTeCOwo0kKutu+VbxiGwcP3+yMyvG0RgR/3XUVBcYMFS0WIZXV5ZRSJRIiNjcWBAwfw6KOP\ncunJycl4/PHHOzwnISEBixYtgkaj4cbRJycnIyAgACEhIVyeH3/80ey85ORkxMfHg8/nY+TIkcjK\nats8gmVZLF26FLW1tdi8eTNCQ0M7/N1xcXHdf2Jil1p7SCxRR5p1Biw5fAmNAh4kAtPylB+Mj8RQ\nH+e7Xpa+wmhksfHzE1DWAFKpBACDd95KwOi4wDv6ey1ZT0jfcbfrSdRgDRb+nocatQ4A8EO1CGsn\nRMFHKrorv99aDR4yHH9bcRAlZabVwn45UIMVfxuOsBA3C5fMhNoT0hN1dXXdZ+qBble5WbBgAXbs\n2IGvvvoKubm5mDt3LuRyOWbPng0AWLx4MSZOnMjlf/rppyGRSDBz5kxkZ2dj7969+Oijj7BgwQIu\nz+zZs3H16lXMnz8fubm52L59O3bu3ImFCxcCACQSCaKjo7nHoEGD4OrqCicnJ0RHR0MoFPbKhyfk\nTtPojViZWoC8qkYAAAMGCxPCKJjvxnd7s5F6vIQ7nvnU0DsezBNirWROYrw/rh8cBaahNpVNWiw5\nlI/qZp2FS2ZZEkchlv41Ce5ujgCApmYdVn58FPJr1FNP7E+3Af0TTzyBjRs3YuXKlYiJiUFaWhr2\n7duHoKAgAKaJrgUFBVx+FxcXJCcno7y8HHFxcZgzZw4WLlyI+fPnc3lCQ0Oxb98+pKSkICYmBmvW\nrMFnn32GRx55pNNy0KRY0tfojSzWphciU6Hi0l4dEYixQdY5ztNa7D94BT/8ksMd339fBB56oL8F\nS0SI5UW4S7B0bDiEPNN1sLxBgyWHLnG7TNsrL08Jlr+dxC1nWVPXjBXrjqJORWvUE/vCsH18vafr\nb1W4urp2kZPYs7t969PIsthwoggHi6q5tOeG+OPJQR3PPSEmxzPKsO6zdG4CfMxQGf42f+xd2wmW\nbpGTnrBkPTlxtRarUs1Xylp1byScRPY9tyT7YiU+WJsCnd609HV4qDvef3ccpBLLDUui9oT0RG/F\nsbRfOiG9zMiy+MepErNg/rGBMjwRLeviLJJ9sRIbtpzggvl+4R5Y+GbCXQvmCekLRgW4YeHoUDAw\n9dRfrmnCssOX0aC17576QQO8Mf/1Udyd/IKiGqxanwq12r7/LsR+0JWSkF5kZFlsySjF/gIllzYl\nwhszh/rTkLEuFBbXYs2GVK53zc/XGUsWjIWjA82XIeRG94R44K2RwdxxfnUjlh6+jHo7H34zOi4Q\ns2fFcscX85X4cNMxaLW0myyxfRTQE9JLWJbF1tOl+PVKJZc2IdQTr8UGUTDfhdKrKry/9giaWib4\nubs6Yvk798DVxcHCJSPEek0O98KbcW1B/aXqRiw7QkH9pPHhmPX0cO74fLYCn2xOh15v30t9EttH\nAT0hvcDYEszvu9wWzN8X6om5I0PA51Ew3xm5ogHvfXQEqnoNAEAqEWHpwiT4ekstXDJCrN+Uft7t\ngvqlhy+hVm3fq9889EB/PPXoYO741NlyfLI5nbsDSIgtooCekNtkMLLYdLIY/7sumB8f4oF5FMx3\nSVnVhL9/eAQ1tc0AAAcHAZYuTLKaNaQJ6QtuDOov1zTh3YP5UDZpLVgqy3vs4YGY9qcB3PGJ01fx\nyT+OU1BPbBYF9ITcBr2RxcfHC/F7YRWXNi7YAwtGhVIw3wVlVROWrTmMypb1+UVCPv42fyyi+nla\nuGSE9D1T+nlj7sgQ8FqG9pWq1Fj0Rz7kDRoLl8xyGIbBs08MwZ+nRHFpJ89cxcefUU89sU0U0BNy\nizR6I9YcK0BKSQ2XNjncC38dTcF8VyqVjVi25jAULZu/8Pk8vPNWIgYP9LFwyQjpuyaHe+HthFAI\nWoJ6eaMG7/yRh6KWO2D2iGEYPDdjKKZNbQvqT50tx0cb06Cx87kGxPZQQE/ILWjQ6rH8yGUcv1rL\npT0Y6Y058cEUzHfhmrIRy28M5uckYsQwWp+fkNt1T7AHliRFQMQzXdqrmnV45488nL9Wb+GSWQ7D\nMHj2yaFmw2/OnK/Aio+PchPxCbEFFNATcpOqm3V492A+sirbLpKPDvDF7BFB3C1v0l65vB7LVh2G\notI0zEYgMPXMx4/wt3DJCLEdI/1d8f74fpAK+QCARp0Byw9fQmppTTdn2q7W4TePPRzNpeXkVeLv\nH7ZNyCekr6OAnpCbUKZS4+3f81B43W3sF4YF4oXhgbQ0ZRcKi2uxdNUhbsy8QMDDorljEB9DwTwh\nvW2ojzPW3NcfHi37OOiMLD48VogfLyrQxzeHv2UMw+DpxwbjuRnDuLQrhdVYsvIQ18lASF9GAT0h\nPXT+Wj0W/p4HeaOpR4fPMFgwKhSPDvS1cMmsW26+EsvXHEZtnRoAIBIJsHjeWMTSMBtC7pgIdwnW\nTYxCoLNpPwcWLLZnlmFzRin0RvsM6gFg2tQovDozFmjZafdqhQp/++AgCors9w4GsQ0U0BPSAweL\nqrDs0CXUt2yvLubzsDQpAhPCaFWWrmRkluP9tSlobFlCT+IoxPK3kxAzVGbhkhFi+2ROYqydGIWB\nXk5c2q9XKvHekcto0NrvpND774vAX98YDYHAFALV1DVj6epDOJclt3DJCLl1FNAT0gUjy+Kb8+X4\n5HgR9C23qj0chPhoQn+M9He1cOms2/6DV7BmwzFoWwIHF2cxPlg8HtFR3hYuGSH2w1UswKrxkRgX\n7MGlnVWo8NfkPJSq1BYsmWWNGRWE5W/fA4mjaViSWq3Hyk9Ssf/gFQuXjJBbQwE9IZ1o1Bqw+MDn\ndwAAIABJREFU4ugVfJdTwaWFujrik0kDEOlBO5l2xmhk8f/+cwHbdpzmxuv6eEmxaum9CA91t3Dp\nCLE/YgEPbyeE4unBbcPcyurVWHDgIk5ct1KXvRk80Aerl90HTw8JAMBgMGLbjtP46v+dhcFgtHDp\nCLk5FNAT0oEylRoLki/iZHkdlxbr54p1E6PgIxVZsGTWTaPRY/2W49j731wuLTzUHWuWT0CAn4sF\nS0aIfWMYBs8M9seihDCI+aZLf5PegA+OXsGurHIY7HRcfXCgKz5cfh/CQ9o6G/534BJWrU9FQ6N9\n77ZL+hYK6Am5wbHSGixIvoiy+rbb0Y8NlOHvSRGQtCwFR9pTVjVhycpDSDtZyqXFDvPDir+Nh7ub\ngwVLRghpdU+IB9ZNjIKvVMylfZtVgb8fuYxatX2uy+7pIcHKJfdidFwgl5Z5QY533vsdxaX2eweD\n9C09Dui3bNmCsLAwODo6Ii4uDqmpqV3mv3DhAsaNGweJRILAwECsWLGiXZ4jR44gNjYWjo6OiIiI\nwLZt28xe//LLL5GUlAQPDw+4u7vjvvvuw7Fjx3paZEJuit7IYvvZMqw+VoBGnWlrcBGfh7dHh2HW\nsADaMKoLFy8psei9P1BQ3LZSxAMT+uHdeWPg2LJ0HiHEOkS4S7Bh8gAM83Xm0s4qVHhrfy6y7HQT\nKgcHARa+mWC2Vr1c0YBF7x/E0fQSC5aMkJ7pUUC/e/duzJs3D0uXLkVmZiYSExMxZcoUlJaWdphf\npVJh0qRJ8PPzQ0ZGBjZt2oR169Zh/fr1XJ7CwkJMnToVY8eORWZmJhYvXow5c+Zg7969XJ4jR47g\nqaeewqFDh3DixAlERUXh/vvvx+XLl2/zYxNiTt6gwbsH8/FjnoJL85WKsXZCf4wP9ejiTPvGsiz2\nJV/CstWHUVNnWpufz+fh1ZmxeOX5EeDz6SYgIdbIVSzAinGRmBHdNq6+qlmHxYcu2e0QHB7PtFb9\nX99IgFgsAABotXps2Hoc2785A63WYOESEtI5hu3BLhOjRo3C8OHDzXrQ+/fvj8ceewyrV69ul3/r\n1q1YvHgxFAoFxGLTbb1Vq1Zh69atKCsrAwAsWrQIP/30E/Ly8rjzXn75ZWRnZyMtLa3Tsvj5+WHp\n0qV44403AAB1dW1jnF1dadUR0rGMjAwAQFxcXLvXUoqr8Y+MEq5XHgBGB7hh3sgQOLc06qS9ZrUO\nW/95GqnH23qvnKQivPNWIgYP9LFgyW5dV/WEkFa2Vk8yKurwyfEiqDRtS1kO9HLCwtGhkDmJuzjT\ndpWU1WHtp2kol7fdsQgLdsOCN0b3eD6QrdUTcmf0VhzbbfeZVqvFmTNnMHnyZLP0yZMndxp4p6en\nIykpiQvmW/OXl5ejuLiYy9PRe2ZkZMBg6PhbsEajgVqthrs7rZRBbl+TzoBPjhfho/RCLpjnMwxm\nDQvA0rHhFMx3obi0Fu++/4dZMB8e4o6170/ss8E8IfYqzs8Vn94/EIO924bg5CobMOe3XBy4orTL\n3WWDA13x0XsTMCo2gEsrLKnFwuW/49DRIrv8mxDr1m3EolQqYTAY4Otrvhumj48P5PKON2GQy+UI\nDg42S2s9Xy6XIyQkBAqFot17+vr6Qq/XQ6lUtnsNAJYuXQpnZ2c8/PDDHf7e1m/DhHSmtY5cqjfg\nP2U61GrbGmVPMYOngkQIaSzD6dNlliqiVWNZFifOVOPXg3Lo9W1/u/jh7vjTRFeUlVxEmQ0MN6W2\nhPSErdWT6S4sPNR6HFDoYWSBJgBrDudi72keHgsUwU1kf/OI7ksUQergin1/yGEwsGhsBD7ceBj/\n96sLHr7fH07S7jt+bK2ekN4VGRnZK+9zR7ogGab3/9Nv2rQJX3zxBf744w84OTl1fwIhHdAYWPyv\nQof0KvO7QLHufEwLEMKBb38XrJ5qaNTjx31XcfFy2y1ooZDBQ5P9ETuU7poR0tfxGQYTfIWIdObj\n3yVaKDWmL+159UZ8kq/Gg35CxHvwwbsD13hrxTAMRsd6IjhAgt0/l0JZbVrKMjtPheKyJkyb4o+B\nkbQkL7G8bgN6Ly8v8Pl8KBQKs3SFQgE/P78Oz5HJZO1671vPl8lkXeYRCATw8vIyS9+4cSOWL1+O\n3377rcuxaDROjXQmIyMDuSoD/miQoFLNg6RlXyhnkQCvxwbhnhCa+NqVYydK8fXuM6hvMEAqNW3C\nEhbshvmvj0agv+1czGjMK+kJW68ncQCm6o345vxV/JxfCRamwH5fDVAscMIbccEIcXW0bCEtYNJE\nHXb++zwOHDLtJssC+PFXJapqpZj1zDC4upgvz2vr9YT0juvH0N+ObsfQi0QixMbG4sCBA2bpycnJ\nSExM7PCchIQEHD16FBqNxix/QEAAQkJCuDzJycnt3jM+Ph58ftta3+vXr8fy5cuxb9++Tn8fIV2p\nbtbhm2It/lmoRWVT20YhowPcsHVKNAXzXahTqfHxP9LxyeZ01De0/X9+8P7+WPP3CTYVzBNC2jgI\neHh5RBA+vC8S/tdNjM2ubMBbv+Xim/PlUOvta9UXRwchZs+KxdK/JsH9ui80KenFeOvd/Th8jMbW\nE8vhv/fee+91l8nFxQV///vf4e/vD0dHR6xcuRKpqan4+uuv4erqisWLF+PDDz/Ec889B8C0As62\nbduQmZmJgQMHIjU1FW+//TYWL16MhIQEAEC/fv3w0UcfobKyEiEhIfj555+xevVqrF+/HgMHDgQA\nrFu3DkuWLMHOnTsRExODhoYGNDQ0wGg0chNur//S4OBAm9eQNjqDEb/kX8OHaQUorDVtEiUUCeEi\nFuDN+GA8P9QfjrRRVIdYlsXBlCJ8uCkNVwqruXQvTwkWvpmAKRP72eSSlOXl5QAAf39/C5eEWDN7\nqic+UjEmh3sBLJBX1QgjACOArMoGHCqqhoeDEMGuDndkqK218pM54757QlFdrUZxmal3Vas14MTp\nq8jNVyIi1B2uLg52VU/IreutOLZHy1YCpqUo165di4qKCgwZMgQbNmzA2LFjAQCzZs3CkSNHUFBQ\nwOXPysrCG2+8gZMnT8LDwwOzZ8/GsmXLzN4zJSUF8+fPR3Z2NgICArBo0SK88sor3OthYWEoKSlp\n94135syZ+Oc//wmAlq0kHTtTocK2M6Xcbq9NjU0AgIcGBeHFmEC40go2nSorV2HbjtPIvlhplj7h\nnjDMfHoYpBKRhUp259EtctIT9lpPimqbsTmjBDnKBrP0wd7OeGF4AKI8pRYqmeWcOVeBz3echrKq\niUvj83l4cHIkIoLVcBDz7a6ekJvTW3FsjwN6a0UBPbleUW0zvj53FRkV5mPSpPpmPBIgxFP3jrJQ\nyaxfY5MW//kpB/uSL8NgMHLp3p5SvDpzBEYM63jOjC2x10CN3Bx7ridGlsXvhVXYea4ctRqd2WsJ\nAW74yxB/hLrZ1/j6ZrUO3/6QhX3Jl806IBloMfEeH7wya6JN3tEkvYMC+hYU0BMAuNaoxa6scvxR\nWM1N4AIAiYCPpwf7wU9VAgGPscsLcHcMBiP+SCnCtz9cgKq+7dYfj8fg4SlReOLP0XBwsI87GvYc\nqJGeo3oCNGoN+Hd2Bf4v/xr0ZkEsg3tDPfDMYD+725SqqKQWX/7rLHLzTXc3G1vuDA/o74fnZgzF\niKEyuxqaRHqGAvoWFNDbt8omLb7PkWP/FaXZRYXHMJgY5onnhvjD3VFIF+AOsCyLk2fKsev7Cygr\nV5m9NqC/F15+dgTCQtwsVDrLoHpCeoLqSZsylRq7ssqRUlJjli5gGEyO8MKMaBk8bXiY3o1YlkVK\nWgm++c95lJZVAQC3MtiASC889ehgDImmzfdIGwroW1BAb58q6jX4KV+BA1eqoDUazV6L93fFzKEB\nZrd96QLchmVZXMi5hn/vyULe5Sqz17w8JXjuyaEYMyrILnuSqJ6QnqB60t6Vmib863x5u+GOQh6D\ne0M9MS3Kx66WutRo9Phs2wGkpCshEJpPdBw80AePPjwQQ6N97LKdJeZ6K461j/voxGbkVTViz0UF\n0stqYbzhu2i0lxP+MsQfw3ydOznbvrEsi3NZCvzn5xxczFeaveboIMSfp0bhz1P6Q0wThgkhNynC\nXYL3x/VD1rV6/Ot8ObJbJs7qjCwOFChxoECJeH9XTB/giyHeTjYfyIrFAoxP9EHsUHdcLhHjwKEC\nbm5SVu41ZOVeQ3iIO/48NQqJIwNpjD25bXTlJlbPyLI4ebUOey8quIvE9aI8pfjLYH/EyJxt/iJx\nKwwGI45nXMXPv+bhckG12WsCAQ8PTOiHRx8a0G5TFEIIuVmDfZzx0YT+OC1X4dusCuRVNXKvnSqv\nw6nyOkR6SDE9ygeJQe4Q8Gy7zXZ2EuLl50bgz1Oj8MPPuTh4tBBGo6kzqqC4Bhu2Hsf/+48UDz0Q\niQnjwuDoILRwiUlfRUNuiNWqVetwqKgav15R4mrL8pPXi5W54JEBvhju230gb4+3yBsatThyrBj/\n3Z8PRWWj2Wt8Pg8T7gnD9IcGwMfL/paa64w91hNy86ie9AzLsshRNmLvRQVOXK0zW7AAANwdhJgQ\n5olJYZ4ItMEOhY7qiVzRgF9+y8fBo0XQavVm+Z2kIkwcH45J48LgJ6M7zfaChtwQm2QwsjgjVyG5\nsAonymrNJroCpolW40I88MgAH4S5SSxUSuvFsizyr1Qj+VABUk+UtrtgCAV8TBgXhukPDoCXJ/39\nCCF3DsMwGOTthEHeTihTqfFT3jX8UVQFbcvQkxq1Dj/kyvFDrhyDvJwwKdwTScHucBDY7oZ/Ml8n\nvPL8CMyYPgj7ki/j198vc7twNzRq8dP/LuKn/13EoAHemDAuDInxQRCJbPfvQXoP9dATq1Ber8bv\nhVX4o7AaymZtu9elQj6m9vPGQ5Het7Rigq33qDU0apGSVozkw4UoLq1t97qTVIQHJvTDlIn94O5m\nez1hvcXW6wnpHVRPbl2tWof/Xa7E/itKVDXr2r0uEfCRFOyOpGB3DPFx7tNDcnpSTzQaPQ6lFuHn\nX/OhuNZ+SKnEUYh7EkMwYVwYIkLd71hZieVQDz3p80pVaqSV1uBYWS2u1DR1mGeglxMmh5l6bRyF\n1EtxPa3WgMwLchw7WYoTp8vb9cYDQHCgK+6/LwL3jg21m7XkCSHWy81BiGcG+2NGtB8yKuqQXFCF\nU+V13N3YJr0B+wuU2F+ghItYgNEBbhgT5Ibhvi59OrjvjFgswAMT+mHS+HCczqzA7ymFOJ1ZwW1Q\n1dSsw29/XMZvf1xGoL8LRscFIiE+EKHBrjRnjJihKzy5a1iWRXGdGqmlNUgrq0VxXXOH+dzEQtwX\n5oFJYZ4ItqNlznpCqzXgXJYCx06U4tTZcjSr2/dwiUQCJI0OwqTx4YiM8KBGnxBidfg8BqMC3DAq\nwA01zTr8UVSF5IIqlF03X0ql0XMr5DiLBBgV4IpR/q4Y5usCqY0NQ+HzeRgZG4CRsQGormnGoaNF\n+D2l0KzXvqxchR9+ycEPv+RA5uuE0XGBSIwPRESYO7XzhIbckDtLpdHjnKIeZ+UqnJGrUNnUfjgN\nYFqreITMBRPDPDEywK3Xe2L68i3yOpUa57IUOHNO3mkQDwAhQW6YfG847kkMhtSONnLpTX25npC7\nh+rJncGyLHKVjUgtrcGx0toOh18CAJ9hMMBTihF+LoiRuSDSQwKeFQa0t1tPjEYWOXmV+P1wIY6f\nvtrhXVjAtH/I8MEyDBvsi6GDfOBsZzv09nU05IZYJY3eiPzqRmTK63FWoUJ+VVO7lQ1aifk8xPq5\nYEygO0YGuEJCQ2oAADq9AXmXqpB5QYHMC3IUFNd0mlfm64QxI4OQEB+IsBA36qUhhPRZDMMg2tsJ\n0d5OeCkmEHlVjThWWovU0hqzziADyyJb2YBsZQO+uVAOF7EAw32dMcjbGdFeUoS4OoJvA8NzeDwG\ngwf6YPBAH2g0epw5L8fxjDJknK0w69hRVjXh9yMF+P1IAQAGkeEeGDbYF8MG+6J/Pw8IbXiSMWlD\nAT25LdXNOuQqG5CrbESOsgFXqpvarUxzPYmAjzh/F4wJckecn4tNr2bQU2q1HpcKqpF3WYmL+VXI\nya+EWt1xTwxgCuIT44OQOJKCeEKIbeIxDAZ6OWGglxNeHB6A/OomHC+rxRm5CpdvmHOl0uiRUlKD\nlBJT54dUyMcATymivU3n9/eQ9Pk5WGKxAAnxpvHzrUMv0zPKcOpMORrN7nyzuFRQhUsFVfjhlxwI\nBXxEhLtjQD9PREV6oX+EJy2MYKMooCc9wrIsatR6FNQ2obCmGQW1TcivaoK8UdPleTyGQaSHBDEy\nF4yQuSDKU2qTE5t6ymAwQn6tAQVFtVwAX1Ray2000hEej0H/fp6IGSJD7DA/CuIJIXaFYRhEeUoR\n5SnF88MCUKvWtQzjNA3nrLlhGGKjzoDTchVOy1Wm88EgwFmMCHcJwt0d0c9dgnB3CVz66K7YIhEf\n8SP8ET/CH3q9EflXqnA+W4HMCwpcKqjG9SOpdXoDLuYrW3YHzwMA+Po4ISrCE2EhbggNcUNYsBtc\nnGmYTl/XN2szuaNUGj2u1qtRqlKjTKVGYW0zCmqaUavpeOz2jQKdHTDI2wkj/Fww1Me5zzaat6up\nWYfi0joUldSiqLQWRcV1KC6r63Qc5PV8vKQYPkSG4UN8MSTah8bEE0JICzcHIe4N9cS9oZ4wsiwK\na5tx4Vo9cpSNyK1sQPUNAT4LFmX1apTVq3GkpC3dWyJCsKsjgl0cEOAsRpCLAwJdHODWh3ZrFQh4\niI7yRnSUN2ZMH4yGBi0u5F7DuSwFzmUrOlwKU3GtAYprDUhJL+bSPNwdERrkhrAQNwQHusLfzxn+\nMifaubYP6VGktWXLFqxbtw5yuRyDBg3Cxo0bMXbs2E7zX7hwAW+++SZOnToFDw8PvPrqq1i2bJlZ\nniNHjmDBggXIycmBv78/3nnnHbz66qtmefbs2YNly5ahoKAAERERWLVqFaZNm3YLH5Ncj2VZ1Kr1\nuNakhaJBA0WTFuX1GlxVqVFar4ZK033A2UrE56G/hxQDvUyPAV5OcLWjAF6rNaCyqhHlFQ24Kq9H\nhbwe5fJ6lFc0oKaTVXzaYxAc6IKoSE8M6OeFAZGekPk6US88IYR0g8cwiHCXIMJdgmlRpuubolGL\nHGUDcipNw0FLVGoYOxgKWtmkRWWTFqcr6szSXcQCBDo7wFcqgq9UDF8nEWROYvhKRfByFFn1+Hwn\nJxE3NAcAamrVyL9ShbxLSly8XIUrBTXQ6Q3tzquuaUZ1TTPOnK8wS3d3c0SAnzP8Zc4I8HOGj7cU\n3p4S+HhJIZUK6TplRbqNvHbv3o158+Zh69atGDt2LDZv3owpU6YgJycHQUFB7fKrVCpMmjQJ48eP\nR0ZGBnJzczFr1ixIpVIsWLAAAFBYWIipU6fipZdewrfffoujR4/i9ddfh7e3N6ZPnw4ASE9Px4wZ\nM/DBBx9g+vTp2LNnDx5//HEcO3YMI0eO7OU/g+1Q6w2obtahRq1HjVqHmmad6VmtR1WTFopGLa41\nabmd+m6Gg4CHcDcJwtwcEe7e8uzmCCGfdwc+iWWxLItmtR51dWrUqjSorVOjqroJlVVNUFa1PdfW\nqbt/sxu4uTogJMgN/SM8ENXPE/37ecJJSj3whBByuxiGgcxJDJmTGPeFegIA1HojiuuacaWmqeXR\njOLaZmiNHV8HVRo9cjQNyFG2f03AMHB3FMLDUQjPlmcPB+ENaSK4WMmymu5uDhgVG4BRsQEATENw\nCopqcaWwGkUldSgqrUVJaR20uvZBPgDU1DajprYZWbnX2r3m6CCEt5cE3p4SeHlK4ObqADdXB7i3\nPLu5OcDNxYF2ur1Lul22ctSoURg+fDi2bdvGpfXv3x+PPfYYVq9e3S7/1q1bsXjxYigUCojFpjFZ\nq1atwtatW1FWVgYAWLRoEX766Sfk5eVx57388svIzs5GWloaAODJJ59EbW0t9u/fz+WZNGkSvL29\n8e2333JptrhsJcuy0BpYNOsNaNAa0KDVo0FnQIPGgHqtHo06U3q9Vm961ui5oL25g2/eN0vE5yHA\nWYwAZwcEOjsg1M0R4e6O8HMSW+XSYN0xGIw4lnYSGo0REf0GoqlZh6YmHRqbdGhs0kJVr0VtnRq1\nKjXq6jSoU5mC+J4MjekKn8+Dv8wZYcFuCAl2NT0HudGEJCtGyxGSnqB60vfpjSzKVKZhOGWqtiGm\npSo1NLfQ4XUjAcOA1TRBwgeCZd5wFgngIhbAScSHs0gAZxEfzmLTs4OAD4mQD4mAB0chHw4C3l29\n1hoMRpTLG1BUUovCklpcLVehXN4A+bUGGHrhbyGViFqCfTFcnMVwkooglYrgJBFCKhVBKhGa0lqe\nnZxEcHQQQiCwvc7CjtyVZSu1Wi3OnDmDd955xyx98uTJXOB9o/T0dCQlJXHBfGv+ZcuWobi4GCEh\nIUhPT8fkyZPbvefOnTthMBjA5/Nx/PhxvPXWW+3ybN68+aY+4J1QqlJDZzBCZ2ShN7LczzqD0XTc\n+rjuWGswQq03PTQGA9R6I5p1RmgMRjTrTccavRHNLXk6W+qxtzgJ+fB1EsNbIoKv1HQ7McBZjEAX\nB3hLRL3SmLAsC6ORhZFlwRrB/Ww0mJ4NBhYGgxE6nQE6vRH6lofpZ1OaTmdsyWOEVmeAwWCERmuA\nRmOAVquHWmOARquHWt16rIdWa4Rao0ez2hS4NzXr0NhoWhVBKi277c91PR6PgYe7I/xlzi0PJ9Oz\nnzO8PSXg2+DdC0II6esEPAahbo4IdTPfvNDIsqhs0kLeoIGiQQt5owbyBi0UjRooGrXtJuB2Rs+y\naNKxqNcB9ZXtx7F3x1HAh6OAB0lLgO8o5EPM50HEZyDi81oeDIQ8HpcuvC5dzOdByGMg4PMgYBjw\neQz4jOlz83mM6ZkxPfMYBo7uDhji6YfhI/y5PDCyqKpqwlV5Pcor6lEub0BlVSMqlU24pmzqcadX\nY5MWjU1aXK3oPu/1+HweHMQCiMV8OIgFcBALILruZ7GYD7HI9LODAx8ikQBCAQ+ClodQyINQwDf9\nLOCB3/JsysOHUGjKx+cx4PEY8Hg88PmmvwePb/obMC1/Cx6PsfrhRV0G9EqlEgaDAb6+vmbpPj4+\nkMvlHZ4jl8sRHBxsltZ6vlwuR0hICBQKRbv39PX1hV6vh1KphK+vL+RyeYd5Ovu9d9MjX59AF4uS\nAED7cJzt9JWend/D0xkAYiMLEdvybATELAtRy8+ORhYORkDY8j61LItaABdb355tfW77RUajKTA3\nGFmwxrZgnL3u+frXDS35+/ieZRAJ+XB1Md06dHURw8PdET5epluLXi1jCN3dHChoJ4QQG8FjGNO4\neakY8G3/ulpvRE2zDtVqHaqatdwQ1+pmLaqb9VxaYydDWHqqWW9As97QboLv3caAAY8xxRY8KQ+M\nkzN4oS6mNAMLfWuHnM4IQ0tnnKGlM671NbAsmBvCAaaT+ODGfOYMgNYAaLVAfWd5u447un7/7vMy\njKkjz/SzKfhHy9+H4Zn+VlxGtKS3/sy91pLe8sPe2bE9L1QXen32oiW/wVx/2+JO+mPm8Lvye4i1\n06GhwbKNLeldkZGRAO5eW0L6Jqon9k0CQCICAkV8wJXGhxPr0GXXopeXF/h8PhQKhVm6QqGAn59f\nh+fIZLJ2veit58tksi7zCAQCeHl5dZmn9T0IIYQQQggh3QT0IpEIsbGxOHDggFl6cnIyEhMTOzwn\nISEBR48ehUajMcsfEBCAkJAQLk9ycnK794yPjwefz+8yz5gxY3r40QghhBBCCLEDbDd2797NikQi\ndvv27WxOTg771ltvsc7OzmxJSQnLsiz77rvvshMmTODy19XVsTKZjJ0xYwablZXF7tmzh3VxcWHX\nr1/P5SksLGSlUik7b948Nicnh/3yyy9ZkUjE7t27l8uTlpbGCgQC9sMPP2Rzc3PZ1atXs0KhkD15\n8mR3RSaEEEIIIcRudLtsJWBainLt2rWoqKjAkCFDsGHDBm5jqVmzZuHIkSMoKCjg8mdlZeGNN97A\nyZMn4eHhgdmzZ7fbWColJQXz589HdnY2AgICsGjRIrzyyitmefbs2YOlS5eioKAA/fr1o42lCCGE\nEEIIuUGPAnpCCCGEEEKIdbL69fZSUlLw8MMPIzAwEDweDzt37uwyf1FREXg8XrvHjfMAiO1Ys2YN\n4uPj4erqCh8fHzz88MPIzs7u9rwLFy5g3LhxkEgkCAwMxIoVK+5CaYml3Eo9ofbE/mzevBnDhg2D\nq6srXF1dkZiYiH379nV5DrUl9udm6wm1JWTNmjXg8XiYM2dOl/lutT3p9WUre1tjYyOGDh2K559/\nHs8991yPl8Xcv38/hg0bxh27u7vfqSISCzty5AjefPNNxMfHw2g0Yvny5Zg4cSJycnI6/XdXqVSY\nNGkSxo8fj4yMDOTm5mLWrFmQSqVYsGDBXf4E5G64lXrSitoT+xEUFIS1a9ciMjISRqMRO3bswLRp\n03Dq1CmzOtCK2hL7dLP1pBW1Jfbp+PHj+PLLLzF06NAu49jbak8sO4T/5jg5ObE7d+7sMk9hYSHL\nMAybkZFxl0pFrE1DQwPL5/PZ//73v53m2bJlC+vq6sqq1WoubeXKlWxAQMDdKCKxAj2pJ9SeEJZl\nWQ8PD/aLL77o8DVqS0irruoJtSX2q7a2lo2IiGAPHz7Mjh8/np0zZ06neW+nPbH6ITe3avr06fD1\n9cXYsWOxZ88eSxeH3EUqlQpGo7HLno/09HQkJSVBLBZzaZMnT0Z5eTmKi4vvRjGJhfWknrSi9sQ+\nGQwGfPfdd1Cr1bjnnns6zENtCelJPWlFbYn9eeWVV/D4449j3LhxYLuZtno77YnNBfTOzs745JNP\n8P333+PXX3/FhAkT8OSTT2LXrl2WLhq5S+bOnYuYmBgkJCR0mkcul8PX13xf79bjGzeAZVRBAAAD\neklEQVQ0I7apJ/WE2hP7dOHCBTg5OcHBwQGvvPIK/vOf/yAqKqrDvNSW2K+bqSfUltinL7/8EgUF\nBVi5ciUAdDts/HbaE6sfQ3+zPD09MX/+fO54xIgRqKqqwtq1a/HMM89YsGTkbliwYAHS0tKQmpra\n5X+cns7FILapp/WE2hP7NGDAAJw/fx51dXX4/vvvMWPGDBw6dAhxcXHt8lJbYr9upp5QW2J/8vLy\nsGTJEqSmpnKbprIs22Uv/e20JzbXQ9+R+Ph4XLp0ydLFIHfY/PnzsXv3bhw8eBChoaFd5pXJZO2+\n7SoUCu41Yrtupp50hNoT2ycUChEeHo6YmBisXr0ao0ePxubNmzvMS22J/bqZetIRaktsW3p6OpRK\nJQYNGgShUAihUIiUlBRs2bIFIpEIOp2u3Tm3057YRUCfmZkJf39/SxeD3EFz587lgrT+/ft3mz8h\nIQFHjx6FRqPh0pKTkxEQEICQkJA7WVRiQTdbTzpC7Yn9MRgMMBqNHb5GbQlp1VU96Qi1JbbtkUce\nQVZWFs6dO4dz584hMzMTcXFxeOqpp5CZmQmhUNjunNtqT25//u6d1dDQwJ49e5Y9e/YsK5FI2A8+\n+IA9e/YsW1JSwrIsy7777rvshAkTuPw7duxgv/32WzYnJ4e9ePEiu27dOlYkErEbN2601Ecgd9jr\nr7/Ouri4sAcPHmQrKiq4R0NDA5fnxnpSV1fHymQydsaMGWxWVha7Z88e1sXFhV2/fr0lPgK5C26l\nnlB7Yn8WLVrEHj16lC0sLGTPnz/PvvvuuyyPx2MPHDjAsiy1JcTkZusJtSWEZVl23Lhx7Jtvvskd\n92Z7YvUB/aFDh1iGYViGYVgej8f9PGvWLJZlWXbmzJlsWFgYl3/nzp1sdHQ0K5VKWRcXFzY+Pp7d\ntWuXpYpP7oIb60br4/333+fy3FhPWJZlL1y4wN5zzz2sg4MD6+/vz37wwQd3u+jkLrqVekLtif2Z\nOXMmGxISworFYtbHx4edNGkSF6S1vk5tCbnZekJtCWFZtt2ylb3ZnjAs280aOoQQQgghhBCrZRdj\n6AkhhBBCCLFVFNATQgghhBDSh1FATwghhBBCSB9GAT0hhBBCCCF9GAX0hBBCCCGE9GEU0BNCCCGE\nENKHUUBPCCGEEEJIH0YBPSGEEEIIIX0YBfSEEEIIIYT0Yf8f47Ibu4i/EXkAAAAASUVORK5CYII=\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "from hockey import *\n", - "import thinkplot\n", - "\n", - "suite1 = Hockey('bruins')\n", - "suite1.UpdateSet([0, 2, 8, 4])\n", - "\n", - "suite2 = Hockey('canucks')\n", - "suite2.UpdateSet([1, 3, 1, 0])\n", - "\n", - "thinkplot.PrePlot(num=2)\n", - "thinkplot.Pmf(suite1)\n", - "thinkplot.Pmf(suite2)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Figure 7.1: Posterior distribution of the number of goals per\n", - "game.\n", - "\n", - "Figure 7.1 shows the resulting posterior distributions for\n", - "`lam`. Based on the first four games, the most likely values\n", - "for `lam` are 2.6 for the Canucks and 2.9 for the Bruins." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The distribution of goals\n", - "\n", - "To compute the probability that each team wins the next game, we need to\n", - "compute the distribution of goals for each team.\n", - "\n", - "If we knew the value of `lam` exactly, we could use the\n", - "Poisson distribution again. `thinkbayes` provides a method that computes\n", - "a truncated approximation of a Poisson distribution:\n", - "\n", - "```python\n", - " def MakePoissonPmf(lam, high):\n", - " pmf = Pmf()\n", - " for k in xrange(0, high+1):\n", - " p = EvalPoissonPmf(k, lam)\n", - " pmf.Set(k, p)\n", - " pmf.Normalize()\n", - " return pmf\n", - "```\n", - "\n", - "The range of values in the computed Pmf is from 0 to `high`.\n", - "So if the value of `lam` were exactly 3.4, we would compute:" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "lam = 3.4\n", - "goal_dist = thinkbayes.MakePoissonPmf(lam, 10)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "I chose the upper bound, 10, because the probability of scoring more\n", - "than 10 goals in a game is quite low.\n", - "\n", - "That’s simple enough so far; the problem is that we don’t know the value\n", - "of `lam` exactly. Instead, we have a distribution of possible\n", - "values for `lam`.\n", - "\n", - "For each value of `lam`, the distribution of goals is\n", - "Poisson. So the overall distribution of goals is a mixture of these\n", - "Poisson distributions, weighted according to the probabilities in the\n", - "distribution of `lam`.\n", - "\n", - "Given the posterior distribution of `lam`, here’s the code\n", - "that makes the distribution of goals:\n", - "\n", - "```python\n", - "def MakeGoalPmf(suite):\n", - " metapmf = thinkbayes.Pmf()\n", - "\n", - " for lam, prob in suite.Items():\n", - " pmf = thinkbayes.MakePoissonPmf(lam, 10)\n", - " metapmf.Set(pmf, prob)\n", - "\n", - " mix = thinkbayes.MakeMixture(metapmf)\n", - " return mix\n", - "```\n", - "\n", - "For each value of `lam` we make a Poisson Pmf and add it to\n", - "the meta-Pmf. I call it a meta-Pmf because it is a Pmf that contains\n", - "Pmfs as its values.\n", - "\n", - "Then we use `MakeMixture` to compute the mixture (we saw\n", - "`MakeMixture` in Section [mixture])." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAugAAAEWCAYAAAA0KmJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdUVNfaBvDnzAxD770KKKCIogIWrKgxUXOTm8S0m+RL\nM2qK6cbYu8aYm+a9liQmpsfEmG4SKxY0KiI2QEBA6gy9l2HK98eQg1wVZmQQhOe3lmt59py9zx44\nwDv77P1uQafT6UBERERERF2CpLM7QEREREREzRigExERERF1IQzQiYiIiIi6EAboRERERERdCAN0\nIiIiIqIuhAE6EREREVEXwgCdiIiIiKgLMShA37BhAwICAmBpaYnIyEgcPnz4mufGxsbizjvvhJeX\nF6ytrREeHo5PPvnkinMkEskV/1JTU9v3boiIiIiIbnJtBujbtm3Diy++iIULFyIxMRHR0dGYPHky\ncnJyrnr+0aNHER4eju+//x7nz5/H008/jRkzZuDrr7++4tykpCQoFArxX58+fdr/joiIiIiIbmJC\nWzuJDhs2DIMGDcLmzZvFsuDgYEybNg2rV6826CL3338/NBoNtm/fDkA/gj5+/HgUFRXB2dm5Hd0n\nIiIiIupeWh1BV6lUSEhIwKRJk1qUT5o0CUeOHDH4IhUVFXBycrqiPDIyEl5eXpg4cSJiY2MNbo+I\niIiIqLuStfZicXExNBoN3N3dW5S7ublBoVAYdIFff/0V+/btaxHQe3l5YdOmTYiKikJDQwM+//xz\nTJgwAQcOHMCoUaOu420QEREREXUPrQbo7RUXF4eHHnoI69evR2RkpFgeHByM4OBg8Xj48OHIysrC\nunXrWgToFRUVHdk9IiIiIqIOZW9vb3SdVqe4uLi4QCqVQqlUtihXKpXw9PRsteHDhw9jypQpWLFi\nBWbOnNlmR4YOHYq0tDQDukxERERE1H21GqDL5XJERERg165dLcp3796N6Ojoa9Y7ePAgpkyZgmXL\nluH55583qCOJiYnw8vIy6FwiIiIiou6qzSkuL7/8Mh555BEMHToU0dHR2LRpExQKBWbNmgUAmDdv\nHk6cOIE9e/YA0GdomTp1Kp577jk8+OCD4lx1qVQKV1dXAMC7776LgIAAhIaGQqVS4YsvvsBPP/2E\nHTt2XLMf1/N4gLovRXUD3jiSibTSGtTW1AIArKytxNcjPe1xTz93DHC1gSAIndVN6kLi4+MBoMV0\nO6L/xfuEDMH7hNrS3mnabQbo9913H0pKSrBy5UoUFBRgwIAB2LlzJ3x9fQEACoUCGRkZ4vmffvop\n6uvrsW7dOqxbt04s9/f3F89rbGzEnDlzkJubC0tLS4SFhWHnzp247bbb2vVmqGf4K7cc7xzLQnWj\nRixzNhdQd9k58QUViC+oQIizNe7p647h3g6QShioExERUdfXZh70znT5pw+OoJNaq8PW03n44ULz\nmgiZICDGXoWRzlLYBYbi+xQljuSUQ4eWt7WPrQXu6uuOmF5OMJcZtIEudTMc8SJD8D4hQ/A+oba0\nN4bt0CwuRKZSVKvC2iOZSC6uFsvcrOR4fWQgqjKTAQAhztaYPzIQeVX1+CFFib2ZpVBptQCA3Kp6\nrD9xCV+ezccdwW6Y3McFNnLe/kRERNT1cCiRurwT+RWY/Udyi+B8mJc93ru1H0Kcra8439vWAs9F\n9cLH/wjDff08YG0mFV8rrW/E1jN5ePznc9iSmIviWtUNeQ9EREREhuIQInVZGq0On5/Nx3fJzZti\nSQUBjw70xt193dpc/OloaYZHw71xb6gH/rhYjJ8uFKK4Th+Q16o12JGixM8XChHj74S7+7rDz96y\nQ98PERERkSEYoFOXVFKrwptHs3CuqEosc7GUY250AEJdbYxqy8pMirv7uuMfQa6IvVSKHSmFyK7U\nLylV63TYnVmC3ZklGO7tgHv6uhvdPhEREZEpMUCnLueUohLrjmaiokEtlkV42OHl4f5wsDC77nbN\npBLcEuiCCQHOOJFfge+TlTh/2bSZv/LK8VdeOfq72OCefu6I8rKHhCkaiYiI6AZjgE5dhkarw9fn\nC/DNeYWYhUUiCHg4zBP3hnqYLFiWCAKGeTtgmLcDkoqq8X2KEn/llYuvny+uxvlD1fC1s8DdTZlf\nzKRcrkFEREQ3BgN06hLK6hrx1l9ZSFRWimVOFmaYEx2AgW62HXbdUFcbhLraIKeyHjtSlNiXWQJ1\nU+bRnMp6vHf8Er48WyBmfrG6bMEpERERUUdggE6d7kxhFdYdyURpfaNYFu5uiznDA+Boef1TWozh\na2eBF4b2wsNhnvgptRC/pxejVq3fCKm4ToWPT+diW1IBpvRxxZ3BbjesX0RERNTzMECnTqPV6fBd\nkgJfnCuAtmnUWoCAB8M88ECoZ6fs/OlsJccTg3xwf6gndl4sws8XCsUPDjWNGnyXrMCPF5SYEOCM\nu0Lc4WNnccP7SERERN0bA3TqFBUNavz7aCZOKpqntNibyzBnRAAGe9h1Ys/0rOVS3NvPA3cGu2F/\nVil2pCiRW1UPAGjU6vDHxWL8ebEEI3zsMa2fx1XzsRMRERFdDwbodMMlFVVj7ZFMMSc5APR3tcHc\nEQFwtpJ3Ys+uJJdKcGtvF9wS6Iy/csuxPUWJCyU1AAAddDiSW44jueUY4GaLaf3cEeFh12Z+diIi\nIqLWMECnG0ar02FHihKfncmHpmlKCwDc188DDw/wuq4pLbV1jfj6h2zk5NehT+9KeHvYwsvTFt4e\ntvD0sIWHuzXMZO1f2CkRBET7OmKEjwPONWV+OZFfIb5+trAKZwur4G9viXv6uWOMnxNknTBFh4iI\niG5+DNDphqhsUOOdY1k4fllQayuX4ZXh/ojysr+uNquqG7DirUM4l6KfJpOSWoyU1OIW50gkAtxc\nrcXA3cvDFt6e+v872lsYPdotCAIGuNligJstssrr8H2KEgcvlYqZX7Iq6vDvv7Lw2Zl83BXijkmB\nzrBk5hciIiIyAgN06nAXSmqwJi4DRbXNU1r6udhgbnQAXK9zSktpeR2Wv3kQ2bkVrZ6n1eqgUFZD\noazGydMFLV6ztDCDp4eNPmD/O3D30P+zsGj7R8PfwRKvDPfHIwO88HNqIX6/WIR6tRYAUFSrwgen\ncvD1+QJMDXLFP4Jc27XJEhEREfUcDNCpw+h0OvycWoSPE3PFEWYAuCvEHY+Fe1/3FBBlUQ2Wrj0A\nZaF+F1BBAKZM8MCYUYOQV1CFAmUVcvOrkK+oQnFJ7TXbqatvREZWGTKyyq54zcnR8srA3dMWrs5W\nkP7PpkVu1nJMH+yD+0I98Ht6EX5KLRR3Qa1SqfHN+QLsSFFiYoAz7g5xh6et+XW9byIiIuoZGKBT\nh6hWqfHe8Us4ktu8Q6eNmRQvDfPHcB+H6243N78Sy948iJJSfeAtkQi49w4fhIc6YNAADwwa4NHi\n/IYGNQqU1cgr0Afs+QVVyFdUI6+gErV1jVe7BACgtKwOpWV1OJtU2KLcTCaFh7sNvDxsxLnuXp76\nIN7Oxhz39/fEP0PcsTerBDtSlCiobgAAqDRa7Ewvwh8XizHSxwH39HNHkBMzvxAREdGVGKCTyaWX\n1uKNIxlicAoAQU7WeD06AB421z96nJFVhuXrDqKySt+umUyKV58bAUGbf8065uYy+Ps5wN+v5YcC\nnU6H8ooGFCir9MF7QRXyFFUoUFRDUVgNjUZ71fYa1Rrk5FUgJ+/KqTW2NuaXTZOxwRMe9ijwEhBb\nWImM8joA+oWyh3LKcCinDIPc7TCtnzsGudsy8wsRERGJGKCTyeh0OuxML8aHp3LQqG2e0vKPIDc8\nMcgb8v+ZGmKM5NRirPr3IXHU29xchnkvjsTA/u6Ij792gH4tgiDA0cECjg4WCA1xbfGaWq2Fsqga\n+QXV+lF3RZU4Al9eUX/NNquqG5CS1oCUtJYLVSEIsPO2RbmrJUrlUsjNJDAzkyI+rxyJykr0cbTC\nPX3dMdLXsVM2ZyIiIqKuhQE6mURtowbrT1zCwezm+dxWMimeH9oLo/0c29X26XMKrHn3CFQq/bxu\nays5Fr46GiF9nNvV7rXIZBJ4e9rB2/PKDZNqalXiaPvlAXy+olrs3xV0OjTkVsIytxLO5lKUOFug\n2M4cOkH/QSE3rxJxF4rgYmGGCd72mBriBn9ve1hZclEpERFRT8QAndots7wWb8RlijttAkBvRyu8\nHh0AL1uLdrV97GQe/v3fo1A3ZUext7PAktfGXDFl5UaxtpIjqLczgnq3/HCg1epQUlqLfGU18vIr\nxaA9v6AKhcW1APRPFCwaNPDOr4FrUR1KnSxQ7mCOBp0ODSoNqmtU2FJSg62ncuFU2oAALeDv3jI1\npJe7Ddxcra9YqEpERETdBwN0um46nQ67M0qwMSEHqsvmbE/u7YqnBvvAXNa+IDI2Lgv/+fAEtE3T\nZVycrbDktTFXHdnubBKJAFcXa7i6WCO8v3uL11QqDQqUzYtT9XPe9aPvFcV1KHO0QKmTOTRNQbdG\nKkGRqyVKtDqkl1bDKb0Y8sbmr69UKoGnu42YWebyhaq2NnLOZyciIrrJMUCn61Kv1uC/8TnYl1Ui\nllnIJJgd2Qvj/J3a3f7ve9Lx4WcJ4rGnuy2Wzh0DV5ebL/OJXC5FL18H9PK9cqFqZVUD8gqqkJVf\ngf3Z5firvAbljVqo1RpoJQJKnSxQ5mgBu8oGOJfUw6JBA41Gi9z8SuTmV15xLRtruRi49wlwxIQx\nATA35485ERHRzYR/uclolyrq8EZcJrIr68Qyf3tLvD4yEL527ZvSAgA7fknGF9+dFY97+Tpg8Zwx\ncHRof9tdiSAIsLezgL2dfqHqFAAarQ6HssuwPVmB1OIaNDZqoGrUoNHOHAXu1rCqaoRNfhWsatW4\n2jh5dY0KqRdLkHqxBLGHs7DvUBbmvzQKTo6WN/rtERER0XVigE5G2ZdVgv+cyEbDZVNabglwxqwI\nP1i0c0qLTqfDF9+dxQ+/pohlQYHOWPTqaNjYXN+OozcbqUTAOH8njO3liARFJb5PUeK0sqr5BE9A\n28cRzhZmGGJlDoeaRhQoqlGgqEaeogoNDS0XqmZklWHusr2Y/9IoBPTqnHn7REREZBwG6GSQBrUW\nmxJysCujOYWguVSCpyN8cUugS7vb12p1+OjzU/hjb7pYNiDUDXNfGNkjs5kIgoAIT3tEeNrjQkkN\ndqQocSS3HFqdDhKJgAKVGr+p1PCyMcfdt/bBc/7OkEsFlJTWIU9RheQLxdj+c5K4eHXBqn14+Znh\niBzk1dlvjYiIiNrAVBDUptzKeryyJ6VFcO5ja4G3b+lrkuBco9Fi/YfHWwTnkYO9sODl0T0yOP9f\nIc7WmDcyEJunhGJyb1fIJc0/tvnVDfhPfDae+OUcvk1SwNxGjvD+7njg7v5Y+Erz16++Xo0178Th\nt11pnfU2iIiIyEAM0KlVBy+V4sVdKcgsb55vPq6XE96d1Bf+Du2f16xq1OCt/xzFgbhLYtmo4X54\nbXY05HJpu9vvTrxsLfBclB8+/kcYHgj1hI1Z89envKERn53Nx+O/nMNHp3JRVKvCoAEeWL1oPFyd\n9QtrdTodtnxxCh99nnDNnVKJiIio8zFAp6tqUGvx3/hsrD2aiTq1BgAgl0jwXKQfXh3uD0uz9gfP\n9fVqrH77MI6dzBPLbokJxAszh0LWzvns3ZmjpRkeGeiFT+4YgKcG+8DVqnl+fp1agx8uKDH9l3N4\n51gW7F2t8caSCQgKbM7bvnN3Ota8EyfuykpERERdC6MgukJBVQNe23sBO9OLxDIvG3O8dUsIJvdx\nNUme7eoaFZavO4gz55Vi2T+nhGDWYxHchMdAVmZS/DPEHR/dHoaXh/mjl33zEw21Toc9mSWYvy8V\nEksZVswfhxFRvuLrCWcKsGDlfhQV13RG14mIiKgVjISohbicMrywKxnpZbVi2ShfR7x3az/0drQy\nyTXKK+qxZE0sUtKa57T/a1oYHrl/IDfZuQ4yiYAJAc747239sGRMH4S52oqvZVXUYVFsGup1Orzy\n7HBMuyNUfO1STjleX7YP6RmlndFtIiIiugYG6AQAaNRo8UFCDlbHZaCmUT+lRSYImDXEF69HB8DK\nBFNaAKC4pBaLVsciM7tcLHvi4cGYdkcog/N2EgQBQ73ssXZCMF4Z7g+hKVP6xbJaLIpNQ61ag39N\nC8Psp4aKTynKKuqwcHUs/orP7cSeExER0eUYoBOUNQ2Yuy8VP6UWimXu1uZYNzEE/wh2M1ngXKCo\nwoKV+5FXoN8BUxAEPDc9CrdPCjJJ+9RsvL8zXhjqJwbp6WW1WHwgHbWNGsSM9seS18bAxlo/d12l\nUuPN94/ih99SoNPpOrPbREREBAboPd6xvHK88GcKLpQ0z0Ue7u2A9yb1RXBT9g9TyM6twMJVsShq\nuo5UKsErzw7H+DEBJrsGtXRLoAtmR/mJxxdKarCkKUgP6+eGNYvHw8PdpulVHT7fdgYbPz4JtZoZ\nXoiIiDoTA/QeSq3V4ePEXCw/dBFVKv3ukzJBwFODfbBwVCBszU23h1XaxRIsXLUfZRX6VI1yuQzz\nXhyJ6KG+bdSk9rq1twueiWgO0pOKq7H0YDrq1Rp4e9rhjcUT0C/YVXx9z4EMrHzrEKprVJ3RXSIi\nIgID9B6pqFaFeftS8X1KcwYVVys53pgQjH+GuJt0Lvi55EIsWXtADPgsLcyweM5oDAn3NNk1qHVT\ng1wxc0jzh6HzRdVYdvAi6tVa2NmaY8ncMRg7spf4+pkkJeYt3wdFYXVndJeIiKjHY4Dew8QXVOD5\nP5ORVNwcfEV62uP9W/uhn4tNKzWNd/J0AVa+dQj19foRelsbcyybNxahIa5t1CRTuyPYDU8O8hGP\nzxRWYeWhi2hQayE3k+L5GUPx4D1h4ut5BZWYu3QvklOLr9YcERERdSAG6D2ERqvD52fyseRAOiob\n9AGzVBDwWLg3lozpDTsTTmkBgLhjOXjj3TiomjLCONpbYsX8cegT4GTS65Dh7u7rjscGeovHp5SV\nWB2XAZVGC0EQcO+doXjp6eEwk+kz9lRVN2DpGwdw6Gh2Z3WZiIioRzIoQN+wYQMCAgJgaWmJyMhI\nHD58+JrnxsbG4s4774SXlxesra0RHh6OTz755IrzDhw4gIiICFhaWqJ3797YvHnz9b8LalVpXSMW\nxKbhm6QCsczJwgyrYoJwbz8PSEyc3nDPgQy8veEvcTt5d1drrFoUAz8fe5Neh4x3b6gHHg7zEo/j\nCyrwRlwGGpu+V6NH+GHp62NhZ2sOAGhUa/DOxr/w7Y9JzPBCRER0g7QZoG/btg0vvvgiFi5ciMTE\nRERHR2Py5MnIycm56vlHjx5FeHg4vv/+e5w/fx5PP/00ZsyYga+//lo8JzMzE1OmTMGoUaOQmJiI\nefPmYfbs2dixY4fp3hkBAE4rKzH7j2ScLawSywa722H9bf0wwM22lZrX55c/UrFhS7wYzPl42WHl\nghh4uJl2+gxdvwfDPPFA/+Y1AMfyK/Dm0UyotfrvWb9gF7yxZAJ8vOzEc77ZcQ7vf3BcfCJCRERE\nHUfQtTEsNmzYMAwaNKjFCHdwcDCmTZuG1atXG3SR+++/HxqNBtu3bwcAzJ07Fz/++CMuXLggnvPU\nU0/h/PnzOHLkiFhWUVEh/t/enqOvxtBoddiWpMBX5wqgg/5bLEDAQ2GeuC/UA1KJaUfNdTodtv2Q\nhG9/PC+WBfZyxOLXxoijsR0lPj4eABAZGdmh1+lOdDodPj2Tj++SFWLZKF9HvDYiQLw3qmtUeGv9\nUZxJal5M3C/YFXNfiO7w72lH4H1ChuB9QobgfUJtaW8M2+oIukqlQkJCAiZNmtSifNKkSS0C6bZU\nVFTAyal57vHRo0ev2mZ8fDw0Go7QtVd5fSOWHEjHl+fyxeDcwdwMK8f1wYNhnh0SnG/9+nSL4Lxv\nsAuWzRt7UwZyPYEgCHh0oBfuCnEXyw7nlOHtY1nQNI2k21jLsfDV0Zg4NlA8Jzm1CHOX7RU3myIi\nIiLTazVALy4uhkajgbu7e4tyNzc3KBSKa9Rq6ddff8W+ffswY8YMsUypVF7Rpru7O9RqNYqLmTWi\nPc4WVmH2H8k4pWwOoAa42WL9bf0wyMOulZrXR6PRYuPHJ/HLH6li2aABHlgyZwysreQmvx6ZjiAI\neHKQN+4MdhPLYi+V4t3jl8QgXSaT4OknIvB/D4QDTbuSKgurMW/5PpxLLrxas0RERNROpk3d8T/i\n4uLw0EMPYf369e1+DPT34yS6Oq1Oh9giNf5UqKG9bNLSBHcZbrGtQMb508gw8TXVGi22/5KHs8nN\nj3HC+tph0mgLnD2baOKrtY33yPUZrNMhy6IRR0v0T69+PV8LZUE+pvmYiQuIfdyAO291wne/5KKx\nUYeamlq8tngn7pzsjYiBjp3ZfaPxPiFD8D4hQ/A+oWsJCgpqV/1WA3QXFxdIpVIolcoW5UqlEp6e\nrW80c/jwYUydOhUrVqzAzJkzW7zm4eFxxQi8UqmETCaDi4uLMf0nADVqHb7JUSGlsnmLdmsZ8KCf\nHCG20g65ZmOjFl/9kI3Ui8351IcMcMBdU7whMfEUGupYgiDgn95m0OiA46X6IP1EqQZSAbjb20zc\nuKp/iD3s7czwxfZsVFWrodECO37LQ2mZChNGu/H7TkREZCKtBuhyuRwRERHYtWsX7rnnHrF89+7d\nuPfee69Z7+DBg7j99tuxfPlyPP/881e8PmLECPzwww8tynbv3o2oqChIpVcPKLkQ4+qSiqrxn6OZ\nKNJIYGWtL+vvYoM50QFw7aApJrV1jVjzzmHkKbSwtrYCAEy5pQ+eeGhwpwRpXKxjGpGROrx3/BL2\nZJYAAM7UA70kbpg5xEcM0iMBjIquxaq3D+NSTjkA4MTpGsjkarwwcyjk8o75QGgKvE/IELxPyBC8\nT6gtly8SvR5tpll8+eWXsXXrVmzZsgXJycl44YUXoFAoMGvWLADAvHnzMHHiRPH82NhYTJ48GU8/\n/TQefPBBKBQKKBQKFBUViefMmjULeXl5eOmll5CcnIyPPvoIn376KV599dV2vZmeRKfT4YcUJebt\nS0VRrUosn9bPA6vHB3dYcF5V3YClaw/gfErz93PaHaF48uHOCc7JdCSCgOejemG8f/OC7l/SCrEl\nMa9FDnQXZyusWhiDIQObn6IdPZGDRatjUVZefyO7TERE1C21GaDfd999ePfdd7Fy5UoMHjwYR44c\nwc6dO+Hr6wsAUCgUyMhont386aefor6+HuvWrYOnpye8vLzg5eWFYcOGief4+/tj586dOHjwIAYP\nHow1a9Zg/fr1uOuuuzrgLXY/VQ1qrDycgY8Sc6FuCpxs5TIsHt0bj4d7Q9ZBgXJpeR0WrY5Fekap\nWPbI/QPxr2lh4ggr3dykEgEvDvXHWL/mIP2HC0psPZPfIki3sjTDvJdGYsotfcSytIwSvL5sL7Jz\n2zdqQERE1NO1mQe9MzEP+pXSSmuwJi4TypoGsSzE2RpzowPgbt1xKQ2VRTVYuvYAlIV/zzkXMPOx\nIbh1fO8Ou6ah+KjR9DRaHdYezURcTplY9kCoJx4e4HnFh7HfdqXh4y8TxQDeytIMrzw7AoMHetzQ\nPreF9wkZgvcJGYL3CbWlQ/OgU9dSWKPCgv1pLYLzfwa7Ye344A4NznPzK7Fw1X4xOJdIBLw4a2iX\nCM6pY0glAl4bEYDh3g5i2TdJBfjqfMEV506dFIR5L42EhYV+SUttXSNWvX0If+67eMP6S0RE1J0w\nQL+JfJCQg5qmrdatzaSYPzIQTw3xhZm0476NGVllWLhqP0pKawEAZjIp5j4/EmOie3XYNalrkEkE\nzI0OQJRX8yf/r84VYNtVgvTIQV5YtWA8nJ30i4a1Wh02bz2JT75KhEajveJ8IiIiujYG6DeJY3nl\nOJpXLh4vHt0bI307Nv90cmoxFq+JRWWVfsTe3FyG+a+MQtQQrw69LnUdcqkE86IDMeSyTa4+O5uP\n7clXblQW0MsBa5dOQKB/8335yx+pePP9I6irb7wh/SUiIuoOGKDfBOrVGmw8mSMeTwp0QZibbYde\n8/Q5BZa9eRC1dfrAytpKjqVzxyK8v3sbNam7MZdJsHBUbwxybw7SPzmdhx8vKK8418nBEivnx2BY\nhLdYduJUPhatihWfwhAREVHrGKDfBL4+rxBTKdqZy/B4uHcbNdrn2Mk8rHr7MFQqNQDA3s4Cy+eN\nQ0gf5w69LnVd5jIJFo3ujQGXfTD88FQufkktvOJcCwsZ5syOxj+nhIhlGZfK8NrSvcjIKrvifCIi\nImqJAXoXl1Vehx9TmkcqHw/3hp15q/tLtUtsXBbWrT8CtVo/b9jF2QorF4xDQC+HNmpSd2chk2DJ\n6N7o72Ijlm1KyMHv6UVXnCuRCPi/B8Ix6/FIMT9+WXkdFqzajxMJ+Tesz0RERDcjBuhdmFanw4b4\nbDHXeX9XG0wM6LhR7N/3pOP9zceh1eqv5+lui1ULYuDtaddGTeopLM2kWDq2D/o6W4tl/4nPxq6M\n4quePykmEIvmjIGVpRkAoKFBjTXvxuGXP1LRhTO8EhERdSoG6F3YnswSnC/WpzaUCQKeifCDpIM2\nBNrxSzI+/CxBPPbzscfKBTFwdbFupRb1RFZmUiwfG4Rgp+Z74/3j2dibWXLV88P7u2PN4glwd/37\nfB0++SoRH352ihleiIiIroIBehdV0aDGx4l54vFdfd3h72Bp8uvodDp8/u0ZfPHdWbEsKNAZK+aP\ng6ODhcmvR92DtVyK5WP7oI+jPq2iDjq8e/wSYrNKr3q+r7cd1iye0GIdwx9707Hq7cPiQmQiIiLS\nY4DeRX2SmIuqpkWa7tbmeKC/p8mvodXq8OFnp/DDryliWVg/NyyZOwa2Nh238RF1D7bmMqwYF4SA\npg+OWp0O/z6WhUPZV18I6mBvgWWvj8Oo4X5iWeJZBeav2IfC4pob0mciIqKbAQP0LuhcYRV2XzZd\nYFaELyxkpv1WaTRarP/wOP7Ymy6WRQ72wsJXRovzhYnaYmcuw8pxQehl3xykrzuaiSM5Vw/S5XIp\nXpw1DPdY9IoyAAAgAElEQVTeGSqWZedW4PVle5GafvUpMkRERD0NA/QuplGjxYbLcp5H+zhg6GU7\nOZqCqlGDt/5zFAfiLollo4b74bXZ0ZDLpSa9FnV/DhZmWDUuCD62+ilRGp0Oa49k4thlG2tdTiIR\n8OA9YXh+5lBIm3bBLa+ox+I1sThyPOeqdYiIiHoSBuhdzI+phbhUUQdAn9ZuxhBfk7ZfX6/G6rcP\n49jJ5vntt8QE4oWZQyEz8Sg99RyOlmZYMz4Y3k1Bulqnw5q4DMQXVFyzzriR/lg6dyxsrOUAmj84\n7vglmRleiIioR2NE1oUoqhvw9bkC8fjhMC+4WslN1n51jQrL1x3EmfPNedXvnByCWY9FiCOZRNfL\nydIMq2KC4Nm0fqFRq8OqQxk4pai8Zp3+fV3xxpIJ8HRv3gDpi+/OYsOWeDSqNR3eZyIioq6IUVkX\nodPpsCkhBw1NaecCHaxwR7Cbydovr6jHkjWxSElrzlf94D1h+L8HBkLooNSN1PO4WsmxOiYY7tb6\nIF2l1WL5oYs4ray6Zh0vD1u8sWQ8+vd1Fcv2HszEinWHUF2t6vA+ExERdTUM0LuIo7nlOJGvnw4g\nQMCzkb6QSkwTOBeX1GLR6lhkZjfPCX7i4cG4985QBudkcm7WcqyOCRKf/qg0Wiw7mI6zhdcO0m1t\nzLH4tTEYN8pfLDuXXIjXl+9FgeLa9YiIiLojBuhdQF2jBh+cyhWPb+vtgr6XbafeHgWKKixYuR95\nBfppBoIg4Nkno3D7pCCTtE90NR425lgTEwznv3cQ1Wix9GA6koqqr1nHTCbF7Kei8K9pYWJZvqIK\nry/fh6QLRR3eZyIioq6CAXoX8OW5AhTV6h/l25vL8OhAL5O0m51bgYWrYlFUos8xLZVK8MqzwzFh\nbIBJ2idqjaetOVbHBMPJQh+k16u1WHIgHRdKrp3zXBAETLsjFK88OwJmMn1GoarqBixde6BF1iEi\nIqLujAF6J8soq8XPqYXi8fTBPrA1l7W73bSLJVi4aj/KmjLCyOUyzHtxJKKHmjYrDFFrfOwssCom\nCA7m+iC9Vq3Botg0pJW2vjHRyGG+WD5/HOztmrLCqLV4b/MxfLPjPDO8EBFRt8cAvRNpdTr8Nz4b\nmqaAY6CbLWJ6ObW73XPJhViy9gCqa/Sj8pYWZlg8ZzSGhJt+N1KitvjZW2JVTBDsmj541jRqsCg2\nHRfLalutF9LHGW8smQBfbzux7Nsfz+OdjcegUjHDCxERdV8M0DvRnxeLkdL0uF8mCHgm0q/dizZP\nni7AyrcOob5eDUC/+G7Z62MRGuLaRk2ijuPvYIlV44JgK9cH6VUqNRbGpiGrvK7Veu6u1li9aDwG\n9ncXyw7/lY2lbx5ARWV9h/aZiIioszBA7yTl9Y3Yerp5s6Bp/Tzg2/Q4/3rFHcvBG+/GQdWoH110\ntLfEivnj0Cew/aPyRO0V6GiFleOCYGOmn1te2aDG/P2p4sZc12JtJcfCV0ZjUkxvsSwltRivL9+H\n3Pxr51gnIiK6WTFA7yRbEvNQ3RRIe9qY475Qj3a1t+dABt7e8Bc0TXnU3V2tsWpRDPx87NvdVyJT\n6eNkheXjgmDdFKRXNKixYH8actsYDZfJJJj52BA89mA4AP1TJmVhNeYt34fTl228RURE1B0wQO8E\np5VV2JdVIh7PivCFuez6vxW//JGKDVvixcVzPl52WLkgBh5upknVSGRKIc7WWDa2DyybsrSU1Tdi\n/v5U5Fe1HqQLgoA7Jofg9ReiIW+aKlNTq8LKtw5hz4GMDu83ERHRjcIA/QZTabTYEJ8tHo/2dUSk\n5/WNcut0Onyz4zw++SpRLAvs5YgV88fB2cmq3X0l6ij9XGywbGwfmEv1v4JK6hoxf38aFNUNbdYd\nGuGNVQtj4OhgCQDQaLTYsCUen207A62WGV6IiOjmxwD9BvshRYncppFCK5kUTw32ua52dDodtn59\nGt/+eF4s6xvsgmXzxoqp6Yi6sv6uNlh6WZBeVKvCvH2pUNa0HaT39nfE2iUT4O/nIJb9+FsK/v3f\no2hoUHdYn4mIiG4EBug3UEFVA75JUojHjwz0gnPTdujG0Gi02PjxSfzyR6pYNmiAB5bMGQPr62iP\nqLMMdLPF4tG9IW8K0gtrVViwP03cuKs1Ls5WWLUwBpGDmjf2OnoiF4vWxKK0jewwREREXRkD9BtE\np9NhY0IOVE2LOPs4WmFqH+NTHzaqNXh307EWc25HRPng9RdHwtwEGxwR3WiDPOywcFQgzCT6xZ8F\n1Q2Yvy8VJQYE6ZYWZpj7QjRuvzVYLEvPKMXry/biUk55h/WZiIioIzFAv0EO55TjZEEFAECAPue5\nVGJcznOtVoe31h9F3LEcsWzcKH+8/MxwyJuyYhDdjCI87TF/VG/ImvYByK9uwPz9aSira2yzrlQq\nwRMPDcJT/zdE3EeguKQW81fsR8Lpgg7tNxERUUdggH4D1DZq8EFCc1A9NcgFIc7WRrfz2640nDiV\nLx5PuaUPnpseBamU30a6+Q31sse8kYFikJ5bVY/5+9NQXt92kA4Akyf2wYKXR8HSwgwAUFffiFVv\nH8bO3Wkd1mciIqKOwMjuBvj8bD5Km4IMJwsz/N8Ab6PbyM6twBffnhWPb781GE8+PBgSI0fhibqy\n4T4OeC06ANKmID27sg4L9qehwsCFn0PCPbF6UQxcnPVZjHQ6HT76/BS2fHGKGV6IiOimwQC9g6WX\n1uLXtCLx+KnBPrCWGzcdRa3W4v0PjqNRrd/YKLCXIx65f4D4OJ+oOxnp64hXh/tD0nR/Z1XUYVFs\nGioNDNJ7+Tpg7ZKJLXbQ/W1XGr7Yno36Bk2H9JmIiMiUGKB3II1Wh//GZ0PbtIHQYHc7jPZzNLqd\n7T8nIyOrDIB+R8XnZw6FmYxzzqn7GtPLCS8N6wWhadfQi2W1WBSbhmqVYUG6o4MFVswbhxFRzWlM\nL1yswkdfZKKktLZD+kxERGQqDNA70O8Xi5FaWgMAkEskeDrS1+hR7/SMUmz/OUk8/te0AfDzub6N\njYhuJuP9nfH8UD/xOL2sFosPpKO20bBRcHNzGV55dgTuur2vWFZQWI/5K/YjX1Fl8v4SERGZCgP0\nDlJW14jPzuSJx9NC3eFta9wGQiqVBu9tPi7One0X7Ip/3Bpk0n4SdWWTAl0wO6qXeHyhpAZLD6Sj\nzsAgXSIR8Mh9A/Hsk1H4ey11UUkNFqzcLz6VIiIi6moYoHeQD0/loqYpiPC2tcC0vh5Gt/Hl9rPI\nK6gEoB8NnD2DGVuo57mttwuejmgeST9fXI2lBy+iXm34fPIJYwPw0LReMDPTP8GqqKzH4jWxOJdc\naPL+EhERtRejvQ5wSlGJA9ml4vEzEb4wlxn3pT6XXNhip9DHHgyHh5uNyfpIdDO5PcgVM4f4isfn\niqqw/NBF1Ku1BrcR0tsWjz/gL+62W1vXiBXrDuFEQn4bNYmIiG4sg6LGDRs2ICAgAJaWloiMjMTh\nw4eveW5DQwMee+wxhIeHQy6XIyYm5opzYmNjIZFIrviXmpp6lRZvLg1qLTbEZ4vH43o5YZCHnVFt\n1NY1Yv0HJ8TjIQM9MSkm0GR9JLoZ3RHshicHNS/6PK2swspDF9FgRJDey8caKxeMg6O9JQD9zrxr\n34/D/kNZpu4uERHRdWszQN+2bRtefPFFLFy4EImJiYiOjsbkyZORk5Nz1fM1Gg0sLS0xe/ZsTJ06\ntdVFkUlJSVAoFOK/Pn36XP876SK2pyiQX90AALA2k2L6ZQGFoT75MhFFJfrFpTbWcjzzZCRTKhIB\nuLuvOx4b2LyPwCllJVbHZUClMSJI93XAqkUxcG96IqXV6rD+w+P4+fcLJu8vERHR9WgzQH/77bfx\n+OOP48knn0RISAjef/99eHp6YuPGjVc938rKChs3bsT06dPh7e0Nne7am4O4urrCzc1N/CeR3Nwz\nbvKq6vFdkkI8fnSgNxwtzYxq40RCPvYezBSPZzw6BE6OlibrI9HN7t5QDzwc5iUexxdU4I24DDQa\nEaR7uNlg9cLx8PdzEMu2fn0aX3x7ttXfWURERDdCqxGxSqVCQkICJk2a1KJ80qRJOHLkSLsvHhkZ\nCS8vL0ycOBGxsbHtbq8z6XQ6bIzPQWNTxpVgJ2vc1tvFqDYqqxqw8ZN48XjkMF+MGu7XSg2inunB\nME88EOopHh/Lr8CbRzOhNmK3UEcHCyyfNw59g5t/Tnf8moxNn5yExohgn4iIyNRkrb1YXFwMjUYD\nd3f3FuVubm5QKBTXqNU2Ly8vbNq0CVFRUWhoaMDnn3+OCRMm4MCBAxg1atRV68THx1+1vKs4VaZG\nXHYjAEAQgLFeGpxKOGlwfZ1Oh29+zEFevj5ri62NDFEDhS7/vrsSfq16lr46HYZaqxFbqN+8aM+F\nWhQplXjQzwzSVqaE/e99cvsEG1RXluHCRX1u9B9/O4f0i9m49x8+kBm5uJu6D/4+IUPwPqFrCQpq\nX1rsVgP0jhIcHIzg4GDxePjw4cjKysK6deuuGaB3ZXUaHX4paN7hcKSzDD5Wxv1hP5NUgXMpleLx\nXVO8YWXZKd8eopuCIAiY4iGDVgccLNL//J0u10AiAPf7th6kX05uJsFD9/hhx848JJ4rBwCcS6lE\nXf0l/OtuP1iYc9deIiK6sVqNAF1cXCCVSqFUKluUK5VKeHp6XqPW9Rk6dCi2bdt2zdcjIyNNej1T\n2hCfDY28CFZywNnSDPOm9IeVmeF/1EtKa/HeR7tgbW0FALhlXCAeur/rvt+u5u8RjK58j1DHidTp\n8MGpXPycqs9pfkEFxGmc8eKwXpBcFqS3dZ9ERUXik68S8duuNACAokiHn3dXYeEro2Fna97B74K6\nCv4+IUPwPqG2VFRUtKt+q8O8crkcERER2LVrV4vy3bt3Izo6ul0X/l+JiYnw8vJq+8Qu5kJJDXam\nF4vHM4b4GhWc63Q6/OejeNTUqgAA7q7WePTBcJP3k6i7EgQBMwb7YEofV7Fsb1YJ1p/IhtaIBZ8S\niYAnHhqEf00LE8vSM0qxYOV+FJfUmrTPRERErWlzDsXLL7+MRx55BEOHDkV0dDQ2bdoEhUKBWbNm\nAQDmzZuHEydOYM+ePWKdpKQkqFQqFBcXo7q6GqdPn4ZOp8OgQYMAAO+++y4CAgIQGhoKlUqFL774\nAj/99BN27NjRQW+zY2i0OmyIz4YO+iAg0tMeI30c2qjV0p/7LuL0ub/n8wuYPWMorIzM/ELU0wmC\ngKcjfKHR6vBnhv4D866MYkgFAc9G+hqcplQQBEy7IxS2NubYvDUBgA55BZWYv3IfFs8ZAx8v4/Y0\nICIiuh5tBuj33XcfSkpKsHLlShQUFGDAgAHYuXMnfH31u/opFApkZGS0qDN16lRcunQJgP4P3uDB\ngyEIAjQa/dbcjY2NmDNnDnJzc2FpaYmwsDDs3LkTt912m6nfX4f6Lb0I6WX6kTW5VIKnIwwPBACg\nQFGFrV+fEY/vnByM0BDXVmoQ0bVIBAHPRflBo9NhT2YJAOD3i0WQSQTMHGLcfgS3ju8NG2s53t10\nDBqNFsUltViwcj8WvToafQKdOqL7REREIkHXhZP+Xj5/x97evhN7cqXiWhWe3pmEWrX+Q8f/DfDC\n/f0Nn5ev0WixaHUsUtL0o32+3vZYt2wi5HIuSDMW5wLS5TRaHd45loX9l0rFsrtC3BGuLoAgCEbd\nJ4lnFXjjvSNQqfSLUC0sZHj9hZEY2N+9jZp0s+LvEzIE7xNqS3tjWOYQu04fnsoVg3MfWwvc1de4\nP9g//Z4qBudSqQTPzxzK4JzIBKQSAS8N88cYP0ex7IcLSuxUqI3ehGjQAA8se30sbKzlAID6ejVW\n/vsQjp7INWmfiYiILscA/TrEF1TgcE6ZePxslB/kUsO/lFnZ5fhmxznx+N47Q9Hb37GVGkRkDKlE\nwCvDAzDSt/nnKrZQjT+V6lZqXV1IH2esXBADRwf9jr5qtRZv/eco9hzIaKMmERHR9WGAbqQGtRab\nTuaIx+P9nTHQzdbg+o1qDd7/4DjUav1OhUGBzrjnH31N3k+ink4mETBnuD+Gezcv3N6rVOOT03lG\nj6T7+dhj9aLx8HTX/6zrdDps2BKPH35LMWmfiYiIAAboRvs2SYGC6gYAgK1chicHeRtX/4ckZGXr\nN0ORm0kxe0YUpEaMvhOR4cykEsyNDkCkZ/P8v+3JCrx3/BI0WuOCdHdXa6xaGIPAXs2j8p9vO4PP\nvjltdMBPRETUGkaGRsiprMf2ZIV4/Fi4NxwsDE+JmJJWjB2/No+4PXzfAKZtI+pgcqkE80cGItSu\n+dfd7swSrI7LQEPTkyxDOdhbYPn8cejftznb0o87L2DDlnhoNMa1RUREdC0M0A2k0+nw3/hsqJtG\nyvq52GBSoLPB9evr1Vj/wQlxpC2snxum3BLUIX0lopbMZRL8n78cUU7NC7H/yivHogNpqFYZNy/d\nytIMi14dg6jBzRur7T2YiXXrj0Kl0pisz0RE1HMxQDfQ/kulOFtYBQDi5icSI3Kef/7tGRQo9fUt\nLczw3FNRkEgMr09E7SMVBNzrY4Zp/TzEsvNF1Xh9XypK6xqNaksul+K156MRM9pfLDuekIcVbx1E\nrZFtERER/S8G6AaobFDjo1PNadXuCHZDgIOVwfVPn1Pg9z3p4vGTDw+Cm4u1SftIRG0TBAGPh3vj\nyUHNGxdlltfhtb0XUFDVYFRbUqkEz02Pwh23hYhl51OKsHh1LMor6k3VZSIi6oEYoBvgszN5qGjQ\nPwZ3tZLjoTDDNySqrlFh/YcnxOOowV4tRt2I6Ma7u687XhrmD2nTU7CC6gbM2XsBF5t2BjaUIAh4\n9MGBePi+gWJZxqUyLFy1H4XFNSbtMxER9RwM0NuQXFyN3y8Wi8czh/jC0szwDYW2fHEKpWV1AABb\nG3PMejwSghFTY4ioY0wMcMaCUYHiHgZl9Y2Yty8VZ5qmshlKEATcfXvfFj/b+YoqzF+xD9m5FW3U\nJiIiuhID9FZotPqFoX8b5mWP4d6Gb9d69EQuDsRdEo9nPR4BRwcLk/aRiK7fMG8HrBwXBJumD901\njRosiU3H0dxyo9uaFBOIV58bAZlM/2u1tKwOC1ftR2p6iUn7TERE3R8D9Fb8nFqIzHL96Le5VIKZ\nEb4Gj36Xlddj0ycnxeOxI3thRJRPKzWIqDP0d7XB2gkhcGpKmarSarE6LgO7LntyZqgRUT5Y8Mpo\nWFjIAOinuC1ZewCnzynaqElERNSMAfo1FNWq8MW5fPH4X2GecLc2N6iuTqfDpk/iUdW0oZGzkxWe\nfHhwh/STiNrP38ES6yaGwMtG/zOu1enw3olL+C5JYfQmROH93bFs7ljYNrXV0KDGyn8fRtyxnDZq\nEhER6TFAv4YPEnJQ37SJSS97S/wzxN3guvsPZeHEqebg/rnpkbCxlpu8j0RkOh425lg3MQR9HJsz\nNG09k4ctiXnQGhmkB/V2xqqFMXB20rel0Wjx7//+hT/3XTRpn4mIqHtigH4Vx/LKceSyOajPRvpB\nZmDO8sLiGnz8ZaJ4PHliH4SHebRSg4i6CgcLM6wZH4yBbrZi2Q8XlHjn2CWotcYF6T5edli9MAbe\nnn/vFqzD5q0nsf3nJKNH5YmIqGdhgP4/6tUabDrZ/Ch6UqAL+rvaGFRXq9XhPx+eEDcq8XC3wSOX\npV8joq7PykyKpWP6YKSvo1i2L6sEKw5dFJ+qGcrVxRorF4xD7wAnseyr7eew9evT0BoZ8BMRUc/B\nAP1/fH1egcJaFQDAzlyGx8K9Da67c086ziUXAtCnXnt+xlBxsRgR3TzMZRLMHRGAyb1dxbL4ggos\nik1DZdOeCIayt7PAstfHYkCom1j2yx+pWP/hcaiNDPiJiKhnYIB+mazyOvyYohSPHw/3hr25YQF2\nbn4lvth2Rjy+a2pf9A1yMXkfiejGkEoEPBvpiwdCmzcmSyquxuv7UlHc9CHeUFaWZljwymgMj2zO\n5HQg7hLeXH8EKpXGZH0mIqLugQF6E61Ohw3x2VA3zQ3t72qDiQHOBtXVaLR4/4PjUDXq/9D6+zng\n/rtDO6yvRHRjCIKARwZ6YeYQX7HsUkUdXt1zAbmV9Ua1JTeT4pVnh2Pi2ECxLP5UPpa9eRA1Rgb8\nRETUvTFAb7InswTni6sBADJBwDMRfpAYmPP8+19SkJ5RCgCQSiV4fsZQmMkM322UiLq2O4LdMGd4\nAGRNvxOKalWYs/cCUktqjGpHKpXg6ScicNftfcWy5NQiLFodi7Jy4wJ+IiLqvhigA6hoUOOT03ni\n8V193eHvYGlQ3YtZZfjupyTx+MF7wuDv52DyPhJR5xrn74RFY3rDXKr/tVnZoMb8/WlIVFQa1Y4g\nCHjkvoF49IFwsSwruxzzV+6DorDapH0mIqKbEwN0AFtP54kLv9ys5Higv2cbNfRUKg3e33wMGo1+\noVffIBfcOTm4w/pJRJ0r0tMea8YHw1auX5tSp9Zg6cF0HMouM7qtO6eE4LnpUZA0pXBVFlZjwYr9\nuJRT3kZNIiLq7np8gH6usAq7Mpq39J4V4QsLmWFflq+/P4ecPP3omVwuw+ynoiCV9vgvKVG3FuJs\njbUTguFqpd98rFGrw9ojmfgtrcjotsaPCcBrs6PFKXFlFXVYuCoWyanFbdQkIqLurEdHk2qtDhsu\ny3ke7eOAYd6GTU9JulCEn35PFY8fe3AgPD1sW6lBRN1FL3tLvDkhBD62FgAAHXTYcDIbX50rMHoT\noqER3lg0ZzQsLcwAADW1Kix78yASTheYvN9ERHRz6NEB+o8XlLhUUQcAsJBJMOOyTA2tqa1rxPoP\njgPQ/yEeNMADt47v3VHdJKIuyM1ajjcnhiDYyVos+/JcPjYn5EJrZJAe1s8Ny+ePg52tOQBApVJj\nzbtxOHQ026R9JiKim0OPDdAV1Q346lzzCNXDYV7iI+u2fPrNaSiL9NkbrK3kePbJSAgGZnwhou7D\n3lyG1TFBGOxuJ5b9klaIt45moVFj3CZEvf0dsWphDFycrQDo07e+s/EYdu5OM2WXiYjoJtAjA3Sd\nTofNCTloaPoDGuhghTuC3dqopXfydAF2788Qj6c/MhjOTlYd0k8i6voszaRYMqY3xvg5imUHskux\n/NBF1DUatwmRt6cdVi8cD1/vvwN+HT76/BS+2XHe6KkzRER08+qRAfpfeRU4nl8BABCg3y1QKml7\nBLyqugEbPooXj0dE+WBMtF+H9ZOIbg5mUgnmjAjA7UGuYlmCohILYtNQ0ZQhylAuzlZYuSAGQYHN\nG6V9++N5bPniFLRaBulERD1BjwvQ6xo12JzQvDD0tt7O6OtiY1DdDz5NQFnTnHUHewvMfCyCU1uI\nCAAgEQTMGuKLh8K8xLILJTV4bc8FFNYYt1OorY05lr4+BuFhHmLZzt3peHfTMajVxk2dISKim0+P\nC9C/PFeAoqZtte3NZfi/gd4G1Tv8VzbijjUH9k8/Hiku6CIiAvSbEP0rzBPPRPhBgP7De25VPV7b\newHZTR/uDWVpYYb5L49E9NDmxeuH/8rGG+/GocHIUXkiIrq59KgAPbO8Fj+nForH0wf7wM5c1ma9\n0rI6fPBpgng8YUwAooZ4tVKDiHqyqUGumBsdAFnTE7aiWhVe25uKCyU1RrVjJpPipaeHYVJMc5ao\nhDMFWPbmQVRXGzcqT0REN48eE6BrdTr8Nz4HmqaFVgPcbBHTy6nNejqdDhu2xKO66RG1q7M1Hn9o\nUIf2lYhufqP9HLFsbB9YNm1CVKVSY96+VJwsqDCqHalUgpmPDcG9d4aKZSlpxVi4ej9Ky40blSci\noptDjwnQd2WUILm4GgAgEwQ8G+ln0PzxXfszkHCmOR3j7BlRsLI067B+ElH3McjDDqtjgsQndQ0a\nLZYfvIgDl0qNakcQBDx4TxieuGxwIDu3AgtW7INCWW3SPhMRUefrEQF6eX0jtp7OE4+n9fOAr51F\nm/UUhdXY+vVp8fgftwUjrJ9h6RiJiAAg2Nka6yaEiPssqHU6vHk0s8V0O0Pdfmswnp85FJKmrFPK\nohrMX7kPmZfKTdpnIiLqXD0iQN+SmIcqlX5RlaeNOe4L9Wijhn6TkPc/OC4uxvLxssND0wZ0aD+J\nqHvysbPAWxND0MveUizbnJCDz8/mG53ffNxIf8x9YSTkZvqpM+UV9Vi0ej+SLhSZtM9ERNR5un2A\nflpZhX1ZJeLxrAhfmMvaftu//JmGlNRiAIBEIuD5GUMhl0s7rJ9E1L25WMnxxvhg9Lssres35wv0\na2OMzG8eNdgLi18bI063q61rxPI3D+LEqXyT9pmIiDpHtw7QGzVabIjPFo9H+Toi0tO+zXrZuRX4\navtZ8XjaHaHoE9j2glIiotbYmcuwclxQi99Dv18swtqjmVBpjMtvHhriihXzY+Bgr5+up2rUYO17\ncYiNyzJll4mIqBMYFKBv2LABAQEBsLS0RGRkJA4fPnzNcxsaGvDYY48hPDwccrkcMTExVz3vwIED\niIiIgKWlJXr37o3Nmzdf3ztoxY4UJXKr6gEAVjIpnhrs02adRrUG728+Lm4GEujviGl39DN534io\nZ7KQSbBodG+M92/+0B+XU4alB9NR26gxqq2AXg5YvXA83F2tAQBarQ7vbz6OX/9MNWmfiYjoxmoz\nQN+2bRtefPFFLFy4EImJiYiOjsbkyZORk5Nz1fM1Gg0sLS0xe/ZsTJ069aqZUjIzMzFlyhSMGjUK\niYmJmDdvHmbPno0dO3a0/x01UVQ34JskhXj8yEAvuDQt0mrN9p+SkXGpDIA+B/HzM4ZCZsCUGCIi\nQ8kkAl4a5o9/BjcvOj+trMK8fakor280qi0PdxusWjQefj7No/Iff5mIr7afM3p+OxERdQ1tRp5v\nv/02Hn/8cTz55JMICQnB+++/D09PT2zcuPGq51tZWWHjxo2YPn06vL29r/oHYtOmTfDx8cF7772H\nkNI7AwwAACAASURBVJAQTJ8+HY8++ijeeuut9r8jNOUuP5kjPjLu42iFqX1c26yXdrEE3/+SLB4/\ndG9Yiz96RESmIhEETB/sg8cu2804vawWr+1NhaK6wai2nBwssXJ+DPoGuYhl239OwuatCdAYOXWG\niIg6X6sBukqlQkJCAiZNmtSifNKkSThy5Mh1X/To0aNXbTM+Ph4ajXGPeK8mLrdc3AxEgIBnIv0g\nlbSe87yhQY33PzgBbdNirf59XXH7rcHt7gsR0bUIgoB7Qz3wfFQvSJqeNuZV1WPOngvIMnITIhsb\nORbPGYMhAz3Fsl37L+KdjcfQqG7/71UiIrpxWt3nvri4GBqNBu7u7i3K3dzcoFAorlGrbUql8oo2\n3d3doVarUVxcfMVrABAfH29Q2/UaHdZdaEBtoz7QjnaRoiozGfGZrdf7dXcBUtP12V7M5RKMHmqO\nhISTBl2TugZD7xHq2brifeIM4B4XDb7MVkGtBWprgGd+jMfjAXIEWBuXPeqW0RaorDDD6ST9IMXu\n/ReQmZWHf93tC3NmojJYV7xPqOvhfULXEhQU1K763W5y9Z8KNSqbgnNbMwG3ebS96+fFrGocjW9O\nxThlogecHNqer05EZCph9lJMD5DDvCmGrtMAH2SokFRp3Oi3VCpg2j98MDyieRFqemY1Pvk6C7V1\nahP2mIiIOkqrI+guLi6QSqVQKpUtypVKJTw9Pa9Rq20eHh5XjMArlUrIZDK4uLhctU5kZGSb7aaX\n1iLxYgqsrPXB9SsjAjC6V+vpEWtqVfjgy12wtrYCAESEe2LmE6OuuriVuqa/RzAMuUeo57oZ7pNI\nAJFltVgcm47yBv1i0e1FAl4I6IUJAc5GtRUVpcO3PyZh2w/nAQClFcD3O8uxeM4YODtZmbrr3cbN\ncJ9Q5+N9Qm2pqKhoV/1WR9DlcjkiIiKwa9euFuW7d+9GdHT0dV90xIgR2L179xVtRkVFQSq9vkew\nGq0O/43PhrZpUepgdzuM8XNss97HXyaiuKQWAGBrY45nnoxicE5Enaa3oxXWTQyGh7U5AECj0+Ht\nY1n4IUXZRs2WBEHA/Xf1x/RHBgPQ/07LyavE/BX7kVdQaepuExGRCbU5xeXll1/G1q1bsWXLFiQn\nJ+OFF16AQqHArFmzgP9v787jo6ru/4+/7mzJZJuE7HtYEvY1YUsQUBZFvvp1pa2/qrUqWlFZ6lIQ\nLVYEl9YKKLjW2sWvWGvVFqxE2ReBAAFNIAQCIetkIWSfTGb5/THJJDEhCQmQCXyej0cezD1z750z\neh+Td8587jnA4sWLmT59eotj0tPTSU1NpaSkhKqqKg4fPkxqaqrz+Ycffpi8vDwWLlzI0aNHee+9\n9/jwww954oknuvxG/nuyhONnqwHQqVT8KiGyw6C970AeW3acdm7PvXcMfr7uXe6DEEJcDGHe7rw6\nfSB9ffXOtvdSc/ngcN4FT51444xYFv5qPGq14+O+uLSaZ5Zv4eTpsovaZyGEEBdPuyUuAHPmzKG0\ntJTly5dTUFDA8OHD2bhxI5GRkQAUFhaSlZXV4pjZs2eTnZ0NOEZxRo8ejaIozhlaYmJi2LhxIwsX\nLmTdunWEh4ezZs0abr311i69ibLaej48kufcvmNIMOHe7Qft8goT6z5ougn0molRJI2P7NLrCyHE\nxdZHr+Wl6+L43Y6TpBVXAfDp0UIq6iw82omZqZq7ZmIUnh5aXlmzB7PZQkVlHc+t2MrihUkMGxzU\n4fFCCCEuL8XuwitZNK/fMRjOPx/5q3tOsTX7LADh3u6suX4wbu0sLmS323ll9W72HnCEej9fPatW\nXI+Xl9wY2htJLaDojN56ndRZbLyy5xTf5Z1ztk0I9+WpiX3b/Zxry9HjJax4bSfVNWbAsRjbr+dN\nYFx8eAdHXj1663UiLi+5TkRHOpthz6fXz+KSWljhDOcAj8RHdvhLa9vubGc4B5h3f4KEcyGES3LT\nqFiS1I/pzW4S/S7vHM9tO0G1+cJmeBkcF8DyZ6biZ3CUztRbrLy8ejd/WX8E8wWeSwghxKXTqwN6\nncXG2gM5zu0pUX0YFeLT7jElpTW895dDzu2Z1/ZnzMiuz0gjhBCXmlqlsGBcNHcMDnG2/VBcyW82\nH6estv6CzhUd6cuLz15LcJAX4PhG8fMNx1i0dBNHj5dc1H4LIYToml4d0D89VkhepQkAT62aB0dH\ntLu/zWbnjff2U9PwCy04yIt7fzbikvdTCCG6S1EU7hsZzi9HNn3OZZ2r4clvMyiorLugc4UEebFi\n6XUMH9JUf55fWMkzy7fw/l8PYTLJfOlCCNGTem1Az6s08Wl607Rj94wIx0/f/qJE//32BEfSGo9R\neHzuOPTuHS9kJIQQruL2wcEsHB+DumGWqoKqOp78NoOTZTUXdB4/X3eWPT2Fh34R3+xz0M6G5EwW\nPvN1s89KIYQQl1uvDOh2u511KTmYbTYA4vp4Mqt/2wscNcorqOAv6793bt8yeyCD49o/RgghXNH0\nvv48M6kfuoapE8tM9SzefJwjRZUXdB5FUbj+uv6sWnk9Y0Y0lfoZi6tZ9vI21n2Q4ryhVAghxOXT\nKwP6jjNlHDI6FtpQKQrzOphyzGq1sead/ZjNjq9toyIM/PS2oZelr0IIcSmMD/dl+dRYPLWOxd2q\n6638dusJ9uSe6+DI1gL8PXjm15N4/KFxeHk23TCfvCWLBUs2ceBwwUXrtxBCiI71uoBeZbbw7qFc\n5/ZNsYEM6GDZ6n9tOMbxk6UAqNUqHp87Dp22ayuWCiGEqxga6MXL0+Lo01CiYrbZWLEri01ZF36z\np6IoTE2KYdXKGxjfbNrF0rM1vPiHHax6ey+VVRdW6y6EEKJrel1A/+v3+Zw1OW7y9Ndr+fnwsHb3\nzzpdxvp/pTu3f3LrEPrF+F3SPgohxOXS19eDV6cPJMzLDQCb3c6qfdn842jhBa86Co7a9KceT+TX\n8ybi4+3mbN+2K5v5i7/mu5Tcdo4WQghxMfSqgJ5RWs2GzKaRobljIvFoZyTcXG9l9Tv7sFobatX7\n+3Pr7EGXvJ9CCHE5hXi58er0gQzwa/o28c+H83g/NQ9bF0K6oigkjY9k9Us3cM3EKGf7uXITr6ze\nzatr9nCu3HRR+i6EEKK1XhPQrTY7a1POYMfxyyY+1EBShG+7x3z8WRpnch0rOel0Gh6bOxa1ute8\nZSGE6DRfdy0rr4tjRJC3s+1fGUb+uDcbi61rC0b7eLux8FcTWLxgknNxI4A9+3OYv/hrtu/O7tIo\nvRBCiPb1mrS64UQxJxqmEdOpVfxqTCSKcv4bQ48eL+GLjRnO7Xt+Mpzw0PYXMRJCiN7MQ6tm2eQB\nJDYbvNh8upTlO09isti6fN6xY8JY9dL1TJvc19lWWVXH62/tZeXruyg9e2FTPAohhGhfrwnofz2S\n73z80yEhhDarjfyxWlM9q9/Z5xzZGTEkmBumDbjkfRRCiJ7mplHxm8R+Laae3Z9fzrNbM6ms6/oC\nRF6eOuY9MJbnnpxMgH9TKU3KoXzmL/6ab7edktF0IYS4SHpNQK+xWAGI8Hbn1kHB7e77l4+PYCyq\nAsBDr+XRB8eiamcaRiGEuJKoVY7pZ386pGlu8/SSKn6z+Tgl3ZzXfNTwEF5fcX2LQY+a2nrefH8/\nz7+ynaKS6m6dXwghRC8K6I3mjY1yLs7RlkNHCvl680nn9v13j24x2iOEEFcDRVG4e0QYD42JdLad\nLq/lyW8yyK3o3g2eHnotc+8dw+8WTyU4yMvZfiTNyIIlX/PVNyewdbHuXQghRC8L6NfF+Le4AerH\nqqrMvPHefuf2+PhwpiZFX46uCSGES7o5LognJ/RF03DPTlGNmSe/zeB4afdHuocNDuL1F2dy0w1x\ngOP8JpOFd/9ykOdWbqWg8MJWNhVCCOHQawK6t07D/aPC293n3b8epOxcLeCYfeDh++LbvZFUCCGu\nBlNj+vDs5P64NXz7WFFnYcmWTFILK7p9bjc3DffdNYqVz11HRFjTjfjpGcUsXJrMl19lOKe6FUII\n0Tm9JqDfOyIM34bV8tqya28OO/accW4/8ssEDD7ul6NrQgjh8hJCDay4NhZvnQaAWouVZdtPsDOn\n7KKcf+AAf37/wgzuuHmI854fs9nCn//vMEte2OKc8lYIIUTHek1Av77ZjAQ/VnbOxDsfHnRuT50U\nw7j49kfbhRDiajMowIuXp8UR6KEDoN5mZ+WuLF777jSl3bx5FECnVXPXHcN4Zdl0YqKapnrMzCrl\nieeS+fTLdCzdmO5RCCGuFr0moKvOU6pit9tZ96cUKqvqAAjw9+D+n4+6nF0TQoheI9qg55VpA4nw\nbvqG8dvTpczdmMb6tALqLkKA7hfjxyvLpvOz24c5F4ezWGx89OkPPL3sG7JOX5xReyGEuFL1moB+\nPt9uP0VKatMc6Y8+MBbPhtEhIYQQrQV56nhl+kAmhDeNcpssNv7yfT6/+iqdnTll3Z7TXKNRcef/\nDuEPL8xgQL8+zvZTZ87x9PPf8tGnP2Cut3brNYQQ4krVqwO6sbiaP/091bk9e0YsI4a2P0e6EEII\nMLhpePaa/rw4NZZog97ZbqyuY+WuLH6zOZOTZd1fITQqwsDKZ6/jnp+ORKdVA2C12vj0y3SefC6Z\n4ydKu/0aQghxpem1Ad1ms/PGu/swmRwr44WFePPzOcN7uFdCCNG7jArxYc31g3kkPgofN42z/Yfi\nSuZ/fYxV+7Ipq63v1muo1SpuuXEgf1g+g0FxTfcT5eRVsPiFzXz4f4cxm2U0XQghGvXagL5hUyZp\nx4oBx4Icj88dh1uzXy5CCCE6R61SmB0byDs3DuXWgcHOOdPt2NmUVcLcDWn842gh5m5Olxge6sPy\nJdfywN2jnZ/XdrudL77KYOEzm0jPKO72exFCiCtBrwzoZ3LL+dsn3zu3b79pMHED/HuwR0II0ft5\nu2l4YHQEb84awtgwg7O9xmLlz4fzeOSrdHZ3sz5dpVK4cUYsr6+4vkVJYoGxkqUvbuG9vx6k1tS9\nEXshhOjtel1At1hsrH5nH/UWx9eh/aL9uPOWwT3cKyGEuHJE+LizbPIAfjdlAJHN1pMoqKrjxV1Z\nLNmSSVY369ODAz357VOTeeT+BDz0TWtcbEw+wYLFmzicZuzW+YUQojfrdQH90y+POqfo0mhUPP7Q\nOLQadQ/3SgghrjzxoQbeuGEID4+JdC5wBHCkqJL5m47xxv5sznVjtFtRFKZP6cfrK64nYVSYs724\ntJrnX97G2vdTqL4I87MLIURv06sC+omss3z6Zbpz+2e3DyMqwtDOEUIIIbpDo1K4KS6Id2YP5abY\nINQN9ek2u52vTjrq0/951Eh9N+rTA/w9WLwwifkPjcfLs2ma3G+2ZTF/8dfsP5TfztFCCHHl6TUB\n3Wy2svqdfdhsjtrHQXEB3HxDXA/3Sgghrg4+bhoejo9kzQ2DiQ/xcbZX11v50+FcHvkqnb1557pc\nn64oClOSolm18gYmjo1wtp8tq2XlH3fy+rq9zgXphBDiStdrAvrfP/2e3PwKANzcNDw+d5xzhToh\nhBCXR7RBz/NTBrBs8oAWq5HmV9Xxux0neXbrCU6fq+3y+f183XnysUSeeHQihmb179v3ZPP4b75m\n976cbvVfCCF6g16TcP/930zn41/8bCQhQV492BshhLh6KYrC2DADb84awtzRkXhpm+4DOmSs4LGv\nj/JmyhnK6yxdfo3EcZGsful6piRGO9vKK0z8/o09vLpmN2XnTN16D0II4cp6TUAHx9emY0aEMvPa\nfj3cFyGEEBqVwv8ODOKd/xnG7AGBqJrVp288Uczc//zA5xldr0/39nJj/sPjWbJoEn38mlY73bM/\nl/mL/8vWXae7NeWjEEK4ql4U0MHLU8cj9yegNPwSEEII0fMMbhoeSYhizfWDGR3cVJ9eVW/l3UO5\nPPrfo+zLL+9ymE4YFcaqldczY2rT4ExVtZnVb+9jxWs7KSnt3pSPQgjhanpVQJ9775gWoyhCCCFc\nR4yvnhemDuC5a/oT5uXmbM+tNPH89hP8dtsJssu7Vp/u6aHjV79M4LdPTyEowNPZfuBwAQuWfE3y\n1iwZTRdCXDF6TUBPHBfJpAlRPd0NIYQQ7VAUhfHhvqydNYT7R0Xg2aw+/UBhBY/99yhvHcihoov1\n6SOHBvPHFTO5ccYAZ1tNbT3r/pTC8y9vp7CoqtvvQQghelqvCegP/WJMT3dBCCFEJ2nVKm4bFMw7\ns4cyq39TfbrVbuffmUXM3ZDGl8eLsNgufNRb767lgbvHsPyZawkN9na2H0k3svCZTWzYlOmcklcI\nIXqjXhPQvZt9XSqEEKJ38HXX8ujYKFZfP4gRQU1hutJs4e2DOTz633RSCsq7dO4hAwP544szueXG\ngc57k+rqLLz/t0MsXbGFvIKKi/IehBDicutUQF+7di19+/ZFr9eTkJDAzp07293/+++/Z8qUKXh4\neBAREcELL7zQ4vmtW7eiUqla/Rw/frzr70QIIYTL6uvrwYprY3kmqR+hzQZccipM/Haboz49t+LC\np07U6dTc89ORvPTcdUSGN60sfex4Cb9emsy/NhzD2o1VToUQoid0GNDXr1/PggULWLp0KampqSQm\nJjJr1ixyctpeLKKiooIZM2YQGhpKSkoKq1at4tVXX+W1115rtW96ejqFhYXOnwEDBrRxRiGEEFcC\nRVFIjPRj7awh3DcyHA9NU316SkE5875K592DOVR2oT49tr8/r/5uOnNuGeJcxM5cb+Wv64+w+IXN\nnMnt2ii9EEL0hA4D+muvvcZ9993H/fffz8CBA1m9ejWhoaGsW7euzf3//ve/YzKZ+PDDDxkyZAi3\n3347Tz/9dJsBPTAwkKCgIOePStVrKm6EEEJ0kU6t4o7BIbwzeygz+wWg4ChPsdjtfH68iLkb0/hP\nZjHWC6wj12nV/PS2Ybz822n0i/Zztp/IOsuvn03mk8/TsVhkNF0I4fraTcRms5mDBw8yc+bMFu0z\nZ85k9+7dbR6zZ88errnmGtzc3Frsn5+fT3Z2dot9ExISCAsLY/r06WzdurWLb0EIIURv5KfXMn9c\nNKuuH8SwwKb69Io6C+sOnOHxr4+SWnjhdeT9Yvx46bfT+H93DEejcfyas1ptfPzZDzy17BtOni67\naO9BCCEuhXYDeklJCVarleDg4BbtQUFBFBYWtnlMYWFhq/0btxuPCQsL46233uKzzz7js88+Y+DA\ngUybNq3D2nYhhBBXnv5+Hrx0XSyLk/oR7Nk0uHO6vJZntmbyu+0nyKu8sPp0jUbF7TcP5g8vzCSu\nv3/TOc+c4+ll3/D3f3yP2Wy9aO9BCCEuJs3FPmFnVvmMi4sjLi7OuT1hwgROnz7Nq6++yqRJk9o8\nJiUl5aL1UVyZ5BoRnSHXietyBx4MtrO92MLmIgvmhmqULZk1bDtRwKQADdODNejVF7aa9G2zfNiT\nUk/ydiP19Y6ymb98fICvvvmB224MJyrco9Uxcp2IzpDrRJxPbGxst45vdwQ9ICAAtVqN0Whs0W40\nGgkNDW3zmJCQkFaj643Hh4SEnPe1xo0bR2ZmZqc6LYQQ4sqkVSlMC9by9CB3Evo03URqs8P2Ygsv\nHzPxXakF6wWsGqpSKSSNC+CxXw6gb1TTKqTFJXW889csNn5bgLleatOFEK6j3RF0nU5HfHw8mzZt\n4vbbb3e2Jycnc+edd7Z5zMSJE3n66aepq6tz1qEnJycTHh5OdHT0eV8rNTWVsLCw8z6fkJDQ7hsR\nV6/GEQy5RkR75Drpfa4DjpdW896hXNJKHCuE2oGvyuCYXc+DoyMZ2Wyhos6YMd3Opi0n+cv6I5hM\njtliUtNqKSguYd79CZiqzwBynYj2yeeJ6Eh5efdmjupw2pRFixbx5z//mffff5+jR48yf/58CgsL\nefjhhwFYvHgx06dPd+5/11134eHhwS9+8QvS0tL47LPPePnll1m0aJFzn9dff50vvviCzMxM0tLS\nWLx4MV988QWPPvpot96MEEKIK0ucvycvT4vj6Yl9CfTQOdtPnatlyZbjvLjzJAWVdZ0+n0qlcMO0\nAby+4npGDmv6VtdYVMVzK7fy5df5mOqkNl0I0bM6rEGfM2cOpaWlLF++nIKCAoYPH87GjRuJjIwE\nHDd+ZmVlOff38fEhOTmZefPmkZCQQJ8+fXjiiSdYuHChc5/6+nqefPJJcnNz0ev1DBs2jI0bN3LD\nDTdcgrcohBCiN1MUhcnRfRgf7stnx4z842ghdQ2LD+3OPcf+/HL+d2AwPxkSgodW3cHZHIICPHnu\nyWvYvP00H3yUSk1tPQB7D54lLaOczGw3piRG0zfat1P3VgkhxMWk2O0XUMh3mTX/esBgMLSzp7ia\nyVeNojPkOrlylNSY+fBIPptPl7Zo93XTcs+IMKb39Uet6nyoPltWy9t/PsD+Q/lUV9cA4OnpuHE0\nMtzAlKRoJk+MIsC/9c2k4uoknyeiI93NsOply5Ytu4j9uajq6pq+tnR3d+/BnghXlp+fD9DuPQxC\nyHVy5fDQqkmM8CUh1EB2eS0lDaPfJquNvfnl7M8vJ8LHvcWUje3R67VMmhBJRJgPqUdyqDPb0Om0\nAFRU1nEkzci/v84kPaMYu81OcJAX2k6O1Isrk3yeiI50N8NKQBe9nnxQis6Q6+TKE+ChY2Y/f8K9\n3ck8W0NNvaN2/Kypnm9OlZJdbiK2jwdeuo5nFFYUhagIAzHhdURFeBAYEEiBsRqrtWl2l6LiavYd\nzOc/m05wJrccnU5NcKAnqgsYrRdXBvk8ER3pboa96POgCyGEEJeLoihMjenDhAgD/zxm5J9Hjc76\n9F05ZezPK+eWgUHc2cn6dJVKIa6fN3fNSWCuqZ69B/LYujOb79OLaKwINZst7PzuDDu/O4PBx51J\nEyKZkhRN/xg/qVcXQlwUEtCFEEL0eu4aNf9vWBgz+wXw58N5bM0+C4DZZuOTo4V8c6qUe0aEMa2v\nP6pOhmi9u5apSTFMTYrhbFkt2/ecYduubLJzzjn3Ka8wsWFTJhs2ZRIR5sOUxGgmJ0YRGODZzpmF\nEKJ9UuIiej35qlF0hlwnVwdPrZqkSD/GhPhw+pyJ0ob69FqLje/yHPXpUQY9QZ66No8/33Wi12sZ\nFBvADdP6Mz4+HHc3DUUlNc651MFRr/59ehH/+TqTH44WYbPZCQ7yRCf16lcc+TwRHZEadHHVkw9K\n0RlynVxdAhvq00O93MgorabW4ih7OWuqJ/lUKbkVbdend+Y68TW4M2p4CP8zM5YhgwIBKCxqWa9e\nXFLD/kP5bPg6k9M55Wi1Uq9+JZHPE9ERqUEXQggh2qBSFKb19ScxwpdPjxr5LMOIuSFEbz9Txnd5\n5dw2MJg7hwTjrrnwUW61WsXIocGMHBrMQ/da2Hsgj227szn8g7GpXr3eyu59Oezel4OPt5ujXj0x\nmgH9+ki9uhDivCSgCyGEuKLptWruHhHGzP7+fJCax46cMgDMVhsfpxeQfKqEX4wMZ2p0ny6/hru7\nhilJ0UxJiubsuVp2NtSrnzrTVK9eUVnHxuQTbEw+QXioD5MTo5icGE1woNSrCyFakoWKRK8nC0aI\nzpDrRDRKK67inYM5nCiradE+0N+TJF0F0Z6qi3adnMktZ+uubLbvzuZsWW2b+wyOC2RKUjSJ4yLw\nOk9tvHAt8nkiOiILFYmrntQCis6Q60Q0CvLUcX3/AII8dWSW1jjr00tr69llNJFXa8PNwxM/dy36\nbt7gafBxZ+SwYGbPjGXooEAUlUJhURUWS1O9eklpDSmp+fzn60xOnzmHVqMiKMgTtUrVrdcWl458\nnoiOSA26EEIIcYFUisLMfgFMivTjH+mFfJ5RhNnmCM3pFTZO7z0NwAA/D+JDfYgPNTDI3xN1F2/y\nVKtVjBgazIihwcy9Zwz7DuazbVc2qT8UYrM5vsiut1jZsz+XPftz8fZyI2m8Y371uP5Sry7E1UZK\nXESvJ181is6Q60S0p7Cqjj8dziP5WB4AHp4erfbx0qoZFeJDfKgPCaEG+ui13X7dsnMmdu09w9Zd\n2WSdLmtzn9BgbyYnRjElKZqQIK9uv6boPvk8ER3pboaVgC56PfmgFJ0h14nojP/s3MfRCiul7v4c\nLa7C0s6vyP5+HsQ3BPbBAV5dHl1vdCa3nO27z7B9TzYlpTVt7jMoLoApiY56dW8vt269nug6+TwR\nHeluhpUSFyGEEKJBiLuKEHcVCQlxVJutpBorOFDg+CmpNbfY92RZDSfLavjkaCGeDaPrCSE+jAn1\nIcDjwm/2jIow8PM5w7nrjmGkZxSzbVc2e/bnUtOw2BLAseMlHDtewvt/O0T8yFCmJEUzZmSoLIYk\nxBVGAroQQgjRBk+dY1XSpEg/7HY72eUmUgrKSSmoaDW6Xl1vZVdOGbsapnDs66snPtRAfKgPQwK8\n0FzA6LpKpTBscBDDBgfx4D1j2Hcwj+27z3Do+0LnYkgWi429B/LYeyAPL08dieMc9eqDYv2lXl2I\nK4AEdCGEEKIDiqIQ46snxlfPHYNDqKm3cthYyYGGwF5c03J0/dS5Wk6dq+XThtH1kcHeJIQaGBPq\nQ+AFjK7rdGomTYhi0oQoyitM7Pwuh227szmRdda5T1W1mU1bTrJpy0mCg7yYkhjFlMRoQkO8L9r7\nF0JcXhLQhRBCiAvkoVUzMcKXiRG+ztH1AwUVHCgsJ62o9ej67txz7M51LFoUY9A7Z4YZEuCJVt25\n6RQNPu7MnhnL7Jmx5OZXsH23YzGk4tJq5z7Goio++TydTz5PJ66/P1OSopk0IVLq1YXoZSSgCyGE\nEN3QfHT99sHB1NRbOWKsJKWgggMF5RT9aHT9dHktp8tr+ecxI3qNmlHB3o6ZYcIMnR5djwjz4a47\nhvHT24Zy9HgJ23Zls3tfTot69eMnSzl+spQ//T2VMSNCmDophvhRUq8uRG8gAV0IIYS4iDy0FsT7\newAAE5lJREFUaiZE+DKhYXQ9p8LkvNH0h+JK6m1No+u1Fit78s6xJ88xuh7lo2+YxtGHIYFe6DoY\nXVepFIYOCmTooEAeuHs0KamO+dUPHmmqV7dabew/lM/+Q/l4euhIHBfRUK8egKqbM88IIS4NCehC\nCCHEJaIoClEGPVEGPbcOCqa23sqRosbR9QqM1XUt9j9TUcuZilr+lWHEXaNiVLBPQzmMD8Ge7Zep\n6HRqEsdFkjgukorKOnZ+d4Ztu86QmVXq3Ke6xkzy1iySt2YRFODJ5MRopiRFER7qc0nevxCiaySg\nCyGEEJeJXqtmfLgv48Mdo+u5lXUcKCh3jK4XVTlXMwUwWWx8l3eO7xpG1yN93EkINXRqdN3H240b\nZ8Ry44xY8goc9erbd2djLG6qVy8qqebTL9P59Mt0Yvv5MyUpiqTxkRh8LnxZciHExSUBXQghhOgB\niqIQ6eNOpI87twwMxmSxcqSoyhnYC6pajq7nVJjIqTDxrwwjbmoVI4O9iW8I7CHt3AQaHurDz253\n1Ksfyyxl265sdu3NobpZbXxmVimZWaV88NFhRg8PYUpSNGNHh6HTSb26ED1BAroQQgjhAtw1asaF\nGRgXZsBut5NXWcfBwgpSCir4vqgSs7VpdL3OamNffjn78h2rFUZ4u5PQMDPM0EAv3DStR9cVRWFw\nXACD4wL45c9HcSC1gG27sjlwuKBFvXpKaj4pqfl46LVMHOuoVx8yMFDq1YW4jCSgCyGEEC5GURQi\nfNyJ8HHn5rggTBYbPxRXkpLvmBkm/0ej67mVJnIrTXx+vAg3tYrhQd4khPqQEGog1Lv16LpOq2bi\n2Agmjo2gsqqOXXtz2LYrm4wTTfXqNbX1fLv9FN9uP4XeXUtEmA+R4S1/Avw9ZGEkIS4BCehCCCGE\ni3PXqBrqzw1AJPmVJueNpkfaGF13rHhaDuQQ7u3uvNF0eKB3q9F1by83bpg2gBumDaDQWMXWXdls\n252NsajKuU+tqd5ZBtNcW8E9KsKAfx+9BHchukECuhBCCNHLhHm7c7O3Y3S9rmF0vXEqx9xKU4t9\n8ypN5FWa+PJ4ETq1ihFB3s7AHu7d8obQkGAvfnrbUH5y6xAyTjjq1ffsz6WisuWIfaOOgntUhI8z\nwEtwF6LzJKALIYQQvZibRkV8qIH4UAMAhVV17M8v50BhBUeMldQ1G103txhdh1AvNxJCDY7R9SBv\n3BtG1xVFYVBsAINiA5h77xjOlddxJq+cnLwKcvMqOJNXTm5eBVXV5tYdonPBPTLcp+GxBHchfkwC\nuhBCCHEFCfFy46a4IG5qGF1PK67iYGEF+/PLW42uF1TV8e/MIv6d6RhdHx7o1RD2fQj3dkNRFBRF\nwc/XHT9fd0YODXYea7fbKSs3kZNX0fDTFOAvNLh76H9cKmMgMtxHgru4aklAF0IIIa5QbhoVY0J9\nGBPqwwOjIyisqmsohSkntY3R9QOFFRworIBDjtH1MSGOUpiB/p4Y3DQtwrKiKPTx1dPHV9/t4F5T\nW8/xk6UcPynBXQiQgC6EEEJcNUK83JgdG8js2EDMVhvpxVWkFFSQUlBOTkXr0fUNJ4rZcKIYADe1\nikAPHYEeOoI8m/3rqSPIQ4e/XotWrbrg4J6TW9FiTvbmJLiLq5UEdCGEEOIqpFOrGBXiw6gQx+i6\nsbqOgwWOedcPGyuptVhb7F9ntTmnc2yLgkIfvaZFgP9xmPczuF+24B4VYXA+luAuehsJ6EIIIYQg\n2NONWQMCmTUgkHqrjfQSx+j6EWMl+ZV11PwosP+YHTultfWU1tZzrLS6zX08NGqCPHUENIT34MYg\n76kjrK8fwwYHoW5YEKlFcM8tJye/olvBvflNqY2PJbgLVyUBXQghhBAtaNUqRgb7MDLYx9lWbbZS\nVFNHUbWZomozJbX1FFebKaoxU1xtprS2Hjv2ds9bY7FyuryW0+W1bT6vVhT89Vpn2Uzj6HvIiGCG\nT4gkyEOHu0bVpeCecaK0xUJM0BTcG0tkGn/6+ElwFz1LAroQQgghOuSpU9NX50FfX482n6+32iit\nrXcG9uIax09joC+uMbe4KbUtVrudohpH6E87zz7eOk1TCU2AnqAoA8M8dQR6aNHW26gorSGvsVwm\nX4K76J0koAshhBCi27RqFSFeboR4ubX5vN1up8JsdYy6V9dRXFPvCPANgb6o2sy5uvoOX6fSbKHS\nbOFkWU3b/VApjhKaGB+ChgYwxEOLh13BWlVHzdlaKozVFORXcia3nJratl+vo+Cuogofbw0FJcfx\n8tTh5anD01Pr+NfD8a9Op5YwL7pMAroQQgghLjlFUTC4aTC4aRjQp+1R+DqLjZJac1PpTE3T6HtR\ntZmSGjP1tvbLaOptdgqq6iioanv1UzxV+A4LIGh8KN5qNeo6C7bqeurKTFQWV1OaW0FdtZm2onVj\ncK+udvxxsPdQ27X2ABqNCk8PXYvQ3hTidXh5ah3/ejU937iP+4+mtBRXHwnoQgghhHAJbhoV4d7u\nhHu7t/m8zW6n3GRxlMFUtwzwjY8rzZYOX+dcXX3r0XofDfgYoL8BtR08AI3ZBrX1mM/VUVlcja3a\njLbehl0Bpf2/E7BYbJRXmCivaHvWm/ao1So89Nrzjs57ejYL+I3tXo7HencJ91eCTgX0tWvX8uqr\nr1JYWMjQoUN5/fXXmTRp0nn3//7773n00UfZv38/ffr04aGHHuLZZ59tsc+2bdtYtGgR6enphIWF\n8dRTT/HQQw91790IIYQQ4oqlUhT89Fr89FoG+nu2uU9tvdU5+t68Ft5Y3XQzq9Xefrq2KlAJ4KYC\nNzfwdUMV44PdYqOm3kpVVS0qmx2zVgdWO1htYLFhq7dhNVux1lmxW22obPYWP4qdZo/tqGz86HlH\nm9Vqo7KqjsrzfQvQDkVR8PTQNoT41qPzLUbwf7SPh16LSiXh3hV0GNDXr1/PggULWLduHZMmTeLN\nN99k1qxZpKenExkZ2Wr/iooKZsyYwdSpU0lJSeHo0aPcd999eHp6smjRIgBOnTrFjTfeyAMPPMBH\nH33Ejh07eOSRRwgMDOS22267+O9SCCGEEFcFvVZNtEFPtEHf5vNWm52zpnpn3XtjHXxjCU1Rtfm8\nU0qqNSr0GhV2m2P03cOz7dcAsNvBZrVhtdmxNfxYbXbqrfaGNluLdpuz3Y5itbUK8I7tttpa71Np\ns6OqNKEqr211XPvxW2kI681G5z2bjdo3C/4/3sdDr0WtVnXy/5LoiGK3t/9n5Pjx4xk1ahRvv/22\nsy0uLo477riDFStWtNp/3bp1LF68GKPRiJub40aRF198kXXr1pGbmwvA008/zeeff05GRobzuAcf\nfJC0tDR2797tbCsvL3c+NhgMXXyL4kqXkpICQEJCQg/3RLgyuU5EZ8h1IgCqzJYWM9CU1NY7b2wt\nqq4jp8SRTzw8266l7y67HUdwt/4oxDcP+208Z7Xa6SDWdRj8m7YdbT/ebm8fjUqFRqNCp3H8q9Go\nUKtVaJttazQqNOqmx9qGfdRqBa1G3cF+ChqNutljFVqN2vm4+fkaHzva1Wg0irM/GrXqkn9T0N0M\n2+4Iutls5uDBgzz11FMt2mfOnNkiSDe3Z88errnmGmc4b9z/2WefJTs7m+joaPbs2cPMmTNbnfPD\nDz/EarWiVqsv+I0IIYQQQlwMXjoNXjrNeaeU3Lt/P2YbDBk+HJPVhslio87i+NdksWKyNt+2Ndu2\nOrfPd0yd1YaigFqtdCkPNYZ7m82G1WrvVMA3N2u/GBz1+XZnnX7Ttg3FZgMbKOa2nj//ttLwh0fH\nzzm2W56r9T5qRUGtKGjUCmqVgkaloFGrWjzWOp9TodU42jRqBW1DyNc2/PGh0yho1Wo0agVdwx8N\n/zM5rFv/DdsN6CUlJVitVoKDg1u0BwUFUVhY2OYxhYWFREVFtWhrPL6wsJDo6GiMRmOrcwYHB2Ox\nWCgpKWn1nBBCCCGEq1ArCno1+HvoLvq5bXa7M7jXWZsF+BbbTe0/buvomHYXk7LTLMjbnKPy7QV8\naxvh3q4AKA3/Nuptte12x48dsFqh/YV0W7mkAb0rLtWdw82/KhCiudjYWECuEdE+uU5EZ8h1Ijrj\nclwnKkAP6BVA2/CDAqgbfsSVrN1q/oCAANRqNUajsUW70WgkNDS0zWNCQkJaja43Hh8SEtLuPhqN\nhoCAgAt7B0IIIYQQQlxB2g3oOp2O+Ph4Nm3a1KI9OTmZxMTENo+ZOHEiO3bsoK6ursX+4eHhREdH\nO/dJTk5udc6xY8dK/bkQQgghhLiqdTiLyyeffMLdd9/N2rVrSUxM5K233uKDDz4gLS2NyMhIFi9e\nzP79+/nmm28AxzSLAwcOZOrUqSxdupSMjAzuu+8+li1bxsKFCwE4ffo0w4YN48EHH2Tu3Lns2rWL\nefPm8fHHH3Prrbde+ncthBBCCCGEi+qwBn3OnDmUlpayfPlyCgoKGD58OBs3bnTOgV5YWEhWVpZz\nfx8fH5KTk5k3bx4JCQn06dOHJ554whnOAWJiYti4cSMLFy5k3bp1hIeHs2bNGgnnQgghhBDiqtfh\nCLoQQgghhBDi8nHpJZ/Wrl1L37590ev1JCQksHPnzp7uknAhK1euZOzYsRgMBoKCgrj55ptJS0vr\n6W4JF7Zy5UpUKhWPPfZYT3dFuJiCggLuvfdegoKC0Ov1DB06lO3bt/d0t4QLsVgsLFmyhH79+qHX\n6+nXrx/PPvssVusFzr8nrijbt2/n5ptvJiIiApVKxYcffthqn2XLlhEeHo6HhwfXXnst6enpHZ7X\nZQP6+vXrWbBgAUuXLiU1NZXExERmzZpFTk5OT3dNuIht27bx6KOPsmfPHjZv3oxGo2H69OmUlZX1\ndNeEC/ruu+949913GTFixCWbDlb0TufOnSMpKQlFUdi4cSPHjh3jjTfeICgoqKe7JlzIihUrePvt\nt1mzZg0ZGRmsWrWKtWvXsnLlyp7umuhB1dXVjBgxglWrVqHX61v9fnn55Zd57bXXeOONN9i/fz9B\nQUHMmDGDqqqqds/rsiUu48ePZ9SoUbz99tvOtri4OO644w5WrFjRgz0Trqq6uhqDwcAXX3zB7Nmz\ne7o7woWUl5cTHx/P+++/z7Jlyxg+fDirV6/u6W4JF7FkyRJ27NjBjh07erorwoXddNNNBAQE8MEH\nHzjb7r33XsrKyvjyyy97sGfCVXh7e/Pmm29yzz33AGC32wkLC+Pxxx9n8eLFAJhMJoKCgvj973/P\n3Llzz3sulxxBN5vNHDx4kJkzZ7ZonzlzJrt37+6hXglXV1FRgc1mw8/Pr6e7IlzM3LlzufPOO5ky\nZQouOiYhetDnn3/OuHHj+MlPfkJwcDCjR4/mzTff7OluCRcza9YsNm/eTEZGBgDp6els2bKFG2+8\nsYd7JlzVqVOnMBqNLfKsu7s7kydP7jDPXvSVRC+GkpISrFYrwcHBLdqDgoJaLXAkRKP58+czevRo\nJk6c2NNdES7k3XffJSsri48++gi4dKsdi94rKyuLtWvXsmjRIpYsWcKhQ4ec9ynMmzevh3snXMUj\njzxCbm4ugwcPRqPRYLFYWLp0KQ8//HBPd024qMbM2laezc/Pb/dYlwzoQlyoRYsWsXv3bnbu3CkB\nTDhlZGTwzDPPsHPnTuciaHa7XUbRRQs2m41x48bx4osvAjBy5EgyMzN58803JaALp9WrV/PBBx/w\n8ccfM3ToUA4dOsT8+fOJiYnhl7/8ZU93T/QyHWUVlwzoAQEBqNVqjEZji3aj0UhoaGgP9Uq4qoUL\nF/LJJ5+wZcsWYmJiero7woXs2bOHkpIShg4d6myzWq3s2LGDt99+m+rqarRabQ/2ULiCsLAwhgwZ\n0qJt0KBBnDlzpod6JFzRiy++yNKlS5kzZw4AQ4cOJTs7m5UrV0pAF20KCQkBHPk1IiLC2W40Gp3P\nnY9L1qDrdDri4+PZtGlTi/bk5GQSExN7qFfCFc2fP5/169ezefNm4uLiero7wsXceuut/PDDDxw+\nfJjDhw+TmppKQkICP/vZz0hNTZVwLgBISkri2LFjLdqOHz8uf/CLFux2OypVy9ikUqnkGzlxXn37\n9iUkJKRFnjWZTOzcubPDPOuSI+jgKFm4++67GTduHImJibz11lsUFhZKrZdwmjdvHn/729/4/PPP\nMRgMzlovb29vPD09e7h3whUYDAYMBkOLNg8PD/z8/FqNmIqr18KFC0lMTGTFihXMmTOHQ4cOsWbN\nGpk+T7Rwyy238NJLL9G3b1+GDBnCoUOH+OMf/8i9997b010TPai6uprMzEzAUS6XnZ1Namoq/v7+\nREZGsmDBAlasWMGgQYOIjY1l+fLleHt7c9ddd7V/YrsLW7t2rT0mJsbu5uZmT0hIsO/YsaOnuyRc\niKIodpVKZVcUpcXP888/39NdEy5s6tSp9scee6ynuyFczIYNG+wjR460u7u72wcOHGhfs2ZNT3dJ\nuJiqqir7r3/9a3tMTIxdr9fb+/XrZ3/mmWfsdXV1Pd010YO2bNnizB/NM8l9993n3GfZsmX20NBQ\nu7u7u33q1Kn2tLS0Ds/rsvOgCyGEEEIIcTVyyRp0IYQQQgghrlYS0IUQQgghhHAhEtCFEEIIIYRw\nIRLQhRBCCCGEcCES0IUQQgghhHAhEtCFEEIIIYRwIRLQhRBCCCGEcCES0IUQQgghhHAhEtCFEEII\nIYRwIf8fuzynxGGpCUQAAAAASUVORK5CYII=\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "goal_dist1 = MakeGoalPmf(suite1)\n", - "goal_dist2 = MakeGoalPmf(suite2)\n", - "\n", - "thinkplot.Clf()\n", - "thinkplot.PrePlot(num=2)\n", - "thinkplot.Pmf(goal_dist1)\n", - "thinkplot.Pmf(goal_dist2)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Figure 7.2: Distribution of goals in a single game.\n", - "\n", - "Figure 7.2 shows the resulting distribution of goals for the\n", - "Bruins and Canucks. The Bruins are less likely to score 3 goals or fewer\n", - "in the next game, and more likely to score 4 or more." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The probability of winning\n", - "\n", - "To get the probability of winning, first we compute the distribution of\n", - "the goal differential:" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "goal_dist1 = MakeGoalPmf(suite1)\n", - "goal_dist2 = MakeGoalPmf(suite2)\n", - "diff = goal_dist1 - goal_dist2" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The subtraction operator invokes `Pmf.__sub__`, which enumerates pairs\n", - "of values and computes the difference. Subtracting two distributions is\n", - "almost the same as adding, which we saw in Section [addends].\n", - "\n", - "If the goal differential is positive, the Bruins win; if negative, the\n", - "Canucks win; if 0, it’s a tie:" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0.457996492072\n", - "0.370293408955\n", - "0.171710098973\n" - ] - } - ], - "source": [ - "p_win = diff.ProbGreater(0)\n", - "p_loss = diff.ProbLess(0)\n", - "p_tie = diff.Prob(0)\n", - "print(p_win)\n", - "print(p_loss)\n", - "print(p_tie)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "With the distributions from the previous section, `p_win` is 46%,\n", - "`p_loss` is 37%, and `p_tie` is 17%.\n", - "\n", - "In the event of a tie at the end of “regulation play,” the teams play\n", - "overtime periods until one team scores. Since the game ends immediately\n", - "when the first goal is scored, this overtime format is known as “sudden\n", - "death.”" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Sudden death\n", - "\n", - "To compute the probability of winning in a sudden death overtime, the\n", - "important statistic is not goals per game, but time until the first\n", - "goal. The assumption that goal-scoring is a Poisson process implies that\n", - "the time between goals is exponentially distributed.\n", - "\n", - "Given `lam`, we can compute the time between goals like this:" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "lam = 3.4\n", - "time_dist = thinkbayes.MakeExponentialPmf(lam, high=2, n=101)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`high` is the upper bound of the distribution. In this case I\n", - "chose 2, because the probability of going more than two games without\n", - "scoring is small. `n` is the number of values in the Pmf.\n", - "\n", - "If we know `lam` exactly, that’s all there is to it. But we\n", - "don’t; instead we have a posterior distribution of possible values. So\n", - "as we did with the distribution of goals, we make a meta-Pmf and compute\n", - "a mixture of Pmfs." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def MakeGoalTimePmf(suite):\n", - " metapmf = thinkbayes.Pmf()\n", - "\n", - " for lam, prob in suite.Items():\n", - " pmf = thinkbayes.MakeExponentialPmf(lam, high=2, n=2001)\n", - " metapmf.Set(pmf, prob)\n", - "\n", - " mix = thinkbayes.MakeMixture(metapmf, name=suite.name)\n", - " return mix" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Figure 7.3 below shows the resulting distributions. For time values\n", - "less than one period (one third of a game), the Bruins are more likely\n", - "to score. The time until the Canucks score is more likely to be longer.\n", - "\n", - "I set the number of values, `n`, fairly high in order to\n", - "minimize the number of ties, since it is not possible for both teams to\n", - "score simultaneously.\n", - "\n", - "Now we compute the probability that the Bruins score first:" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "time_dist1 = MakeGoalTimePmf(suite1)\n", - "time_dist2 = MakeGoalTimePmf(suite2)\n", - "p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For the Bruins, the probability of winning in overtime is 52%.\n", - "\n", - "Finally, the total probability of winning is the chance of winning at\n", - "the end of regulation play plus the probability of winning in overtime." - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "collapsed": false - }, - "outputs": [], - "source": [ - "p_tie = diff.Prob(0)\n", - "p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", - "\n", - "p_win = diff.ProbGreater(0) + p_tie * p_overtime" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvwAAAEWCAYAAAAab0P2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VNX9x/H3zCSTTMi+hwSyAQEBBQkCEWSPgCuCCLZW\nXBurKFBafqnUYgEVUQpiQVCsKLZFLWorKMSNXTRiMEAEgYCQZAIJkBDIPvP7IzAQE8JiYLJ8Xs+T\n5/Ge+71nvpNc9TtnzjnXYLfb7YiIiIiISJNkdHYCIiIiIiJy+ajgFxERERFpwlTwi4iIiIg0YSr4\nRURERESaMBX8IiIiIiJNmAp+EREREZEmTAW/iIiIiEgTdkEF//z584mOjsZisRAfH8/69evrjE9P\nT6dv3754eHgQERHBtGnTasSsWbOGbt26YbFYiI2NZeHChdXOv/vuu8THx+Pn54enpyddu3blzTff\n/MW5iYiIiIg0J+ct+JctW8b48eOZMmUKaWlpJCQkMHToUA4cOFBrfGFhIYMHDyYsLIzU1FTmzp3L\nrFmzmD17tiMmMzOTYcOG0bt3b9LS0khOTmbcuHEsX77cERMYGMhTTz3F5s2bSU9P57777uOBBx5g\nxYoVl5ybiIiIiEhzYzjfk3Z79OhBly5dqo3At2vXjpEjR/LMM8/UiF+wYAHJycnk5ubi5uYGwIwZ\nM1iwYAEHDx4EYPLkyXzwwQfs3LnTcd1DDz3E9u3b2bhx4zlz6datG0OGDGHGjBmXlJuIiIiISHNT\n5wh/WVkZW7ZsITExsVp7YmLiOQvzTZs20adPH0exfzo+Ozub/fv3O2Jq6zM1NZXKysoafdrtdj77\n7DN27drFwIEDLzk3EREREZHmxqWuk3l5eVRWVhISElKtPTg4GKvVWus1VquV1q1bV2s7fb3VaiUy\nMpLc3NwafYaEhFBRUUFeXp7jXEFBAeHh4ZSVlWEwGPj73//OgAEDLjk3EREREZHmps6C/1IYDIZ6\n68vb25vvv/+eoqIiPv30U5544gmCg4O59dZbL6qfgoKCestJRERERORK8/HxueRr6yz4AwMDMZlM\n5ObmVmvPzc0lLCys1mtCQ0NrjLCfvj40NLTOGBcXFwIDAx1tBoOBmJgYAK6++moyMjL429/+xq23\n3npJuYmIiIiINDd1zuE3m81069aN1atXV2tPSUkhISGh1mt69erFunXrKC0trRYfHh5OZGSkIyYl\nJaVGn927d8dkMp0zn8rKSmw22yXnJiIiIiLS3Jx3Ss/EiRO55557uO6660hISOCVV17BarWSlJQE\nQHJyMt988w2ffvopAHfffTdPP/00Y8eOZcqUKezcuZOZM2cydepUR59JSUm8/PLLTJgwgYcffpgN\nGzawZMkS/v3vfztiZsyYQc+ePYmOjqa0tJSVK1eydOlS5s+ff8G51eaXfB0iciWkpqYCEB8f7+RM\nRC6M7llpTHS/SmNSX9PSz1vwjxo1ivz8fKZPn05OTg6dO3dm5cqVtGrVCqhaiLt3715HvLe3Nykp\nKTz66KPEx8fj7+/PpEmTmDBhgiMmKiqKlStXMmHCBBYsWEB4eDjz5s1j+PDhjpgTJ07wyCOPcPDg\nQSwWCx06dOCtt97irrvuuuDcRERERESau/Puw98UnP3pSCP80tBp9EkaG92z0pjofpXGpL5q2PM+\naVdERERERBovFfwiIiIiIk2YCn4RERERkSas3h+81dDZ7fZ6fTiYiIiISH2w2WyUlZU5Ow25QoxG\nI66urlekLm12BX9qWg7du7Z0dhoiIiIiDna7ndLSUtzd3TUw2QzY7XZsNhslJSVX5G/e7Kb0/PO9\nbdhsTX5jIhEREWlEysrKMJvNKvabCYPBgMlkwmw2U15eftlfr9kV/PsPHGPD5gPOTkNERETEwW63\nYzKZnJ2GXGFGoxGbzXb5X+eyv0ID9M//bKOi4vL/ckVEREREzuVKfaPTLAv+3ENFfLY209lpiIiI\niIhcds2y4Ad454MdlJVVOjsNEREREZHLqtkV/H4+FgCOHitmRcqPTs5GREREpOmaOnUqRqORQ4cO\nXfbX+vLLLzEajaxdu/ayv1Zj0+wK/jtv7+D45/c/+oETJ7XfrYiIiEhToF2OatfsCv6BfaMJCfYE\noOhEGR+u3OXkjERERETkl+rbty/FxcX06dPH2ak0OM2u4Hd1MTF6eEfH8f9W7eLosRInZiQiIiIi\nP1dcXHxR8QaDQc8yOIdmV/AD9OnVmtYRPgCUllaw7P3tTs5IREREpOnKz8/n7rvvxtfXF39/f5KS\nkjhx4oTjfFRUFEOHDuWzzz6jR48eWCwWZs2aBVTtVf/000/X6DMqKor77rvPcVzbHP6xY8disVjI\nzs7m9ttvx8vLi+DgYP7whz/U2P/+nXfeoXv37vj4+ODt7c1VV13F9OnT6/tX4RTNsuA3Gg385q6r\nHcefrtnLgaxCJ2YkIiIi0nSNHj2agoICnn32WUaOHMmiRYsYNWqU47zBYGD37t3ceeed9O/fn3nz\n5tGzZ89q53/OYDBc0Gi+zWZjyJAhBAUF8eKLL9K3b19efPFFFi1a5Ij59NNPGT16NH5+fjz33HO8\n8MILDBkyhA0bNvzCd94wuDg7AWfpenUoV18Vwvc7crHZ7Lz1zvf8aUJvZ6clIiIi0uRERESwYsUK\nx3FYWBjTpk3js88+Y+DAgdjtdvbs2cN///tfbr755np97fLyckaNGsWUKVMAePjhh+nWrRuLFy8m\nKSkJgBUrVuDj48OqVaua5JSgZlvwGwwGfjP6aiY99SlgJ/W7bLZlHKJTh2BnpyYiIiJSp38v3847\nH1yeKcmjbu/I6Ds6nj/wIjz22GPVjh9//HGmTZvGihUrGDhwIACtWrWq92L/tIceeqjace/evVm6\ndKnj2NfXl6KiIlatWsWQIUMuSw7O1Cyn9JwWE+VH3+tbO47f+NdWbDa7EzMSERERaXratm1b7Tgg\nIABfX1/27dvnaIuJibksr202mwkJCanW5ufnx9GjRx3Hv/vd74iLi2PYsGFEREQwduxY/ve//12W\nfJyhWRf8AL8a2RmzqwmAvfuOsm7TT07OSERERKT5sVgsFxVfUVFxQXEXMkUnKCiI7777jhUrVnDH\nHXewceNGbrvtNm699daLyqmharZTek4LDPDg5hvbsfyjDADefi+dXt0jMJtNTs5MREREpHaj76j/\naTeX065du2jTpo3jOC8vj2PHjhEVFXXea/38/Dh27Fi1trKyMnJycuo1R1dXV4YOHcrQoUMBSE5O\nZubMmWzcuJGEhIR6fa0rrdmP8AMMvzkOL083APLyT7Ii5UcnZyQiIiLSdLz88svVjl966SUAbrrp\npvNeGxsby5o1a6q1LVq0qMa2mudyISP8R44cqdHWpUsXAAoKCi7odRqyZj/CD9DCw8xdw6/itbe+\nA+A//81g4A3ReHu5OTkzERERkcYvOzubYcOGcdNNN7F161Zee+01brzxRseC3bo8+OCDJCUlMXLk\nSAYNGsTWrVtZvXo1gYGB2O3nX3t5ITEPPPAA+fn5DBw4kIiICLKysnj55Zdp2bIlN9xwwwW9x4ZM\nI/ynJPaPpWWoFwAni8sv28p3ERERkebi9F75//rXv/Dz8+PJJ5/kvffe46GHHuLdd9+tFncuDz30\nEJMnT2bt2rVMmjSJ/fv3k5KSQosWLWpcV9vxhezhf8899+Dp6ckrr7zCo48+yuuvv87NN9/Mhg0b\naNGixaW+/QbDYL+Qjz2N3NlfxRjdW+DlVvsXG5u/zWLm3KoHLBiNBmZPT3Q8kVfkSklNTQUgPj7e\nyZmIXBjds9KYNNT7taSkBHd3d2enIU5Q19/+7BrWx+fSa9JmN8L/+tasc5677tqWdGwfBIDNZuf1\nt9Mu6GsgEREREZGGqtkV/Kv35vH9oeO1njMYDDzw666Or3i+355L6nf1uwJcRERERORKanYFP8DL\n3/xEaUXtK7ujWvuS2P/Mgx9e/2caZeWVVyo1EREREZF61SwL/qzjJSzbce6R+zEjOtHCwwxA7qEi\nVqzWNp0iIiIi0jhdUME/f/58oqOjsVgsxMfHs379+jrj09PT6du3Lx4eHkRERDBt2rQaMWvWrKFb\nt25YLBZiY2NZuHBhtfOvvvoqffr0wd/fHz8/PwYMGMCGDRuqxUydOhWj0Vjtp2XLlhfylvhPRi6Z\nx07Wes7by427znqYxbsf7uDosZIL6ldEREREpCE5b8G/bNkyxo8fz5QpU0hLSyMhIYGhQ4dy4MCB\nWuMLCwsZPHgwYWFhpKamMnfuXGbNmsXs2bMdMZmZmQwbNozevXuTlpZGcnIy48aNY/ny5Y6YNWvW\nMGbMGL744gs2b95MXFwcN954I7t37672eu3bt8dqtTp+0tPT63w/HQM9Aaiw25n3zU9U2mpflDtk\nQCytwr0BKCmp4O136+5XRERERKQhOm/BP3v2bO677z4eeOAB4uLieOmllwgLC2PBggW1xr/99tuU\nlJSwZMkSrrrqKkaMGMHkyZOrFfyvvPIKERERzJ07l7i4OB588EHuvfdeXnjhBUfM0qVL+d3vfsc1\n11xDu3btWLBgAV5eXqxatara65lMJoKDgx0/AQEBdb6fx7q3xtVYtSh3Z/4JVuw+XGuci4uR+3/V\nxXH8+bpMftyTX/cvS0RERESkgamz4C8rK2PLli0kJiZWa09MTGTjxo21XrNp0yb69OmDm5tbtfjs\n7Gz279/viKmtz9TUVCora18gW1paSklJCX5+ftXa9+7dS3h4ODExMYwZM4bMzMy63hKtfSyMuirM\ncfzm99kcOlFWa+w1nULp3vXMFKHFS9OwneMbARERERGRhqjOgj8vL4/KykpCQkKqtQcHB2O1Wmu9\nxmq11og/fXz6mtzc3FpjKioqyMvLq7XfKVOm4OXlxa233upo69mzJ0uWLGHVqlW8+uqrWK1WEhIS\nOHLkSF1vizs7hNDa2wJAcUUlL3+z/5z77Y8dcw0mU9WvadeefNZs2F9n3yIiIiIiDUntj5z9Bep6\nNPKlmjt3LosWLeKzzz7D09PT0T5kyBDHP3fq1IlevXoRHR3NkiVLmDBhQq19nX7C3oAWlczPqRrZ\nX7fnJAvK8rnOv/ZfR5er3Fn7VdUHkXmL1mHCioel3n91ItWcvldFGgvds9KYNLT7NTIyUk/abaaO\nHz/Otm3baj3Xtm3benmNOkf4AwMDMZlM5ObmVmvPzc0lLCys1mtCQ0NrjP6fvj40NLTOGBcXFwID\nA6u1z5kzhz//+c98/PHH530MtoeHBx07dqyxsLc20S1M3BB0pmj/X3Y5x8pqH+Xvd30Qvt6uAJw4\nWcmnaw+dt38RERERkYagzmFqs9lMt27dWL16NSNGjHC0p6SkcOedd9Z6Ta9evZg8eTKlpaWOefwp\nKSmEh4cTGRnpiHn//ferXZeSkkL37t0xmUyOttmzZzN16lRWrlxJQkLCed9MSUkJGRkZDBgw4Jwx\nZ39o6FRhI2dVBlnHq7bcXFvuw9O9Ymv9lsLoGs6seVXrFrbtLOU3Y2JoE+N/3pxELtbpUafzfcAV\naSh0z0pj0lDv15ISbf/dXHl5eZ3zfiwoKKiX1zjvLj0TJ07kjTfeYPHixWRkZPDEE09gtVpJSkoC\nIDk5mUGDBjni7777bjw8PBg7dizbt29n+fLlzJw5k4kTJzpikpKSyMrKYsKECWRkZPDaa6+xZMkS\nJk2a5IiZNWsWycnJLF68mDZt2ji23SwsLHTETJo0ibVr15KZmcnmzZsZOXIkxcXF3HvvvRf05t1d\njIy/LhIDVQX+tzkFpGTWvhNPz/hwrr369LcadhYu2UJlZe1P6xURERGRpsNoNPLII484O41Ldt6C\nf9SoUcyZM4fp06fTtWtXNm7cyMqVK2nVqhVQtRB37969jnhvb29SUlLIzs4mPj6ecePGMWnSpGpz\n6qOioli5ciVr166la9euPPvss8ybN4/hw4c7YubPn09FRQV33XUXLVu2dPyMHz/eEZOVlcWYMWNo\n3749I0aMwGKx8NVXXzlyuxBXBXlye1yw4/i17w5y+GTNXXsMBgMP3tMVV5eqbyD2ZB7h0zV17wgk\nIiIiIk3D5VineqUY7OfanqYJOfvrEB8fnxrnSypsPH7W1J74MB+m3lD71J5/L9/OOx9sB8CzhZmX\nnhuCr48W2Uj9aahfN4uci+5ZaUwa6v1aUlKiRbsNmNFoJCkpifnz59d733X97c9Xw16o847wNwfu\nLkaeOGtqT2odU3vuuLk9IcFVOwUVnSjjrXe+v2J5ioiIiIhcLBX8p3QM8uS2dkGO43NN7TGbTTx0\nT1fH8Rfr9rFjZ+1P6xURERERyMnJ4be//S0RERG4u7sTHR3Nww8/TFFREUePHuUPf/gDV199Nd7e\n3nh5edG/f3/Wr19frY99+/ZhNBqZOXMmr776KrGxsbi7u3PdddfV2Ga1X79+9O/fv0YeY8eOJTo6\nulqb3W7n73//O126dMHDw4OgoCASExNrvP7P/e1vf8NoNPLss8862ubPn0/nzp3x9PTE19eXrl27\nsmjRoov9ddU7bSZ/lnuuDufr7AKyi0o5UV7JnM37mdavDcafTe259powesZH8FXqQQAWvrGFF6YN\ncszvFxEREZEqVquV6667jiNHjvDwww/TsWNHsrKy+OCDDzhy5AiHDh3iP//5D3fddRcxMTEcPXqU\nxYsXM2jQIL755hs6d+5crb9ly5ZRVFTkWET7/PPPc8cdd7B3715cXKpKW4PBcM459z9vf/jhh1m8\neDFDhgzh/vvvx2azsXHjRtatW0fv3r1r7WPGjBn8+c9/Zs6cOTz++OMALF68mMcee4w777yTxx9/\nnPLycrZt28amTZt4+OGHf9Hv8JdSwX8WdxcjE3pEMfnzXdjsdtJyC/nfrsPcdtai3tPu/1UXvku3\nUlpawYGsAj5YsZM7b7vKCVmLiIhIc/P2tmz+uS3nsvR9d6cwftWpZb3193//939YrVY2btxI9+7d\nHe1/+ctfgKrnM529AQxUFeHt27fnpZde4tVXX6127uDBg/z444+OOe1xcXHcdtttrFq1iptuugmo\nGrU/V8F/9vLVL7/8ksWLF/Poo48yb948R/vZm8T83J/+9CdmzpzJK6+8Uq2Q/+ijj+jUqRPLli2r\n8/fhDJrS8zNXBXkysn2I4/iNrVnsO1ZcIy4wwIO7R3RyHL/74Q6ycgprxImIiIg0Vzabjffff5+h\nQ4dWK/bPZjabHf9cUlJCfn4+FRUVxMfH8+2339aIHzFiRLUFrKdH4TMzL373xPfeew+Ap59++ryx\ndrudCRMmMGvWLJYsWVJj1N7X15cDBw40uKc4gwr+Wt3dKYw2fh4AlNlszN68j/Ja9twfNriN4+Fb\nFRU2Frz+LTZbk9/0SEREROSCHD58mOPHj9OpU6dzxtjtdp577jliYmIcc+iDg4NZuXJltecvnda6\ndetqx35+fgAcPXr0ovPbs2cPoaGh+Puf/2Gqb7/9NnPnzmXu3Ln8+te/rnF+8uTJeHl5cd1119Gm\nTRseeeQRvvzyy4vO6XLQlJ5auJqMTOwZxfhVP1Bms7Hn6Ene3pbD2GvCq8WZTEYeuS+eP/wlBZvN\nzo6dh/l0TSaJ/WOclLmIiIg0B7/q1LJep9040zPPPMOf//xnxo4dS2JiIgEBAY7FsD+f6gNgMtW+\nZvLsqTrnms5TWVlZ53V1SUhIID09nZdffplRo0YRGBhY7Xz79u3ZuXMnK1euZNWqVXz00UcsXLiQ\n3/3ud7z88ssX9BqXi0b4zyHSx1KtwH8vI5fth4tqxEVH+nLbsDjH8Zv/3sqRWqYAiYiIiDQ3QUFB\neHt7k56efs6Yd999l/79+/P6668zevRoBg8ezMCBAykuvvR6ys/Pr9YR//3791f7MBAbG0tubi55\neXnn7TMmJoZPP/2UvLw8EhMTq+2Rf5rFYmHEiBEsWrSIffv28atf/Yr58+eTk3N51ltcKBX8dbil\nXRBdQ7wBsGPnxa/2cbK85ifDu27v6Nib/2RxOYvfSruieYqIiIg0REajkeHDh/Pxxx/z9ddf1zhv\nt9txcXHBZqs+dXrjxo1s2rTpkl+3TZs2/PDDD9UK+a1bt7Jhw4ZqcXfeeScAU6dOvaB+O3TowKpV\nq8jMzGTo0KGcOHHCcS4/v/oznEwmk2Mq07Fjxy7lbdQbTempg9Fg4IkekTz28Q6KyivJPVHKoi0H\nGN8jqlqc2Wwi6b5uPD1zDQCbvjnAN1si6X5t0/iqTURERORSPfvss6SkpNCvXz9++9vf0qFDB3Jz\nc3n//fd5//33ufXWW5k6dSr33nsvvXv35scff+TVV1+lY8eOFBXVnF1xIe6//35mz57NjTfeyP33\n38+hQ4dYuHAhnTp1qrYuoG/fvowdO5b58+ezZ88ehgwZAsCmTZu45pprSE5OrtF3165dWbFiBYmJ\nidx2222sWLECNzc3EhMTCQkJ4frrryc0NJTdu3fz8ssvc80119ChQ4dL++XVE43wn0eQh5lH488s\nDknJzGftT0dqxF3TMYT+faIcx4ve3MLJ4vIrkaKIiIhIgxUaGsrmzZsZPXo0//rXv3j88cd54403\n6NGjB4GBgSQnJ/PHP/6Rzz//nCeeeII1a9awbNky4uPjzzkX/3zat2/Pm2++SUFBAb///e/56KOP\nWLp0Kddee22NPhcvXszs2bM5cOAAkydPZvr06Rw7dox+/fqds/+EhAQ+/PBDNmzYwJ133klFRQVJ\nSUkUFxczd+5cHn30Ud555x0eeOABUlJSLuk91CeD/UJXKjRiZ8+xOnsbp4sxa1MmX+6vKvRbuJp4\n6cYOhHq6VYs5XlTKuMmfUHi8FIDB/WN45L74S8xamqvT23nFx+vekcZB96w0Jg31fi0pKcHd3d3Z\naYgT1PW3r48aFjTCf8F+1601oS2qCvwT5ZW88NU+Kn62BaeXpxsP3tPVcZzyxV62brNe0TxFRERE\nRM6mgv8CtTCb+GNCNC6nvgbKyCvin9uya8Rd36MVPeMjHMd/X5yqqT0iIiIi4jQq+C9CXEALfn31\nmYW47+zIZWtu9QdCGAwGHvrNtXidmu6Tl3+SJf/eekXzFBERERE5TQX/RRrRPqTaVp0vbNrHsZLq\nI/h+vu489BtN7RERERER51PBf5GMBgMTe0bh41a1o+mRknLmfL2/xlPaNLVHRERERBoCFfyXwN/i\nysSeUY7jb7IL+HDXoWoxmtojIiIiIg2BCv5LFB/mw/C4EMfxP9Ky2Jl/olqMpvaIiIiIiLOp4P8F\n7r26JW38PACosNt5dsNeCksrqsX8fGrPy6+lUlRUdkXzFBERkYavGTwaSX7mSv3NVfD/Aq4mI8nX\nx9DC1QTA4ZNlvPjVPmxn/fF+PrUn/8hJFr25xSn5ioiISMNkNpspKSlR0d+MVFZWUlJSgtlsvuyv\n5XLZX6GJC/V04/c9o/jruj0ApOYU8O4OK3d1DHPE+Pm688j93Xj+pY0ArP/qJ7p3bUmfXq2dkrOI\niIg0LEajETc3N0pLS52dilwhBoMBd3d3DKee8XQ5qeCvBz3CfRnRPoT//JALwNJtObQP9OSaEC9H\nTM/4CAb0iebzdZkALHzjWzq0CyQwwMMpOYuIiEjDYjQacXd3d3Ya0gRpSk89+c3V4XQM8gTAZrfz\n/MZM8k9Wn6t//6+7EBLUAoCTxeW8tOhrbDZ9dSciIiIil48K/nriYjQwuVc0vm6uABwrLWfmpkwq\nzyroPSyuPP7bHo6vbrZlHOKjVbuckq+IiIiINA8q+OtRgIeZPyZEYTxV0G8/XMSb6dnVYjq0C+SO\nm9s7jpe+m85PBwuuaJ4iIiIi0nyo4K9n14R48+tOZxbsvpdhZcOBo9ViRg2/iphIPwAqKmz8bcFm\nysorr2ieIiIiItI8qOC/DO68KpT4MB/H8ezN+9h3rNhx7Opi4omkHphPbee5/8Ax3vr391c8TxER\nERFp+lTwXwZGg4Hf94wi7NTe+yUVNmas38Pxsx7K1Srcm9+MvtpxvCLlR77Zkl2jLxERERGRX+KC\nCv758+cTHR2NxWIhPj6e9evX1xmfnp5O37598fDwICIigmnTptWIWbNmDd26dcNisRAbG8vChQur\nnX/11Vfp06cP/v7++Pn5MWDAADZs2PCLc7tSvN1cmNI7FjdT1a84u6iUF77aV20R79BBbejetaXj\neN6rX5OXf/KK5yoiIiIiTdd5C/5ly5Yxfvx4pkyZQlpaGgkJCQwdOpQDBw7UGl9YWMjgwYMJCwsj\nNTWVuXPnMmvWLGbPnu2IyczMZNiwYfTu3Zu0tDSSk5MZN24cy5cvd8SsWbOGMWPG8MUXX7B582bi\n4uK48cYb2b179yXndqVF+VqY0CPKcZyaU8Db286M4hsMBh57qDsB/lV78RedKGPOK5uprLRd6VRF\nREREpIky2M/zDOcePXrQpUuXaiPw7dq1Y+TIkTzzzDM14hcsWEBycjK5ubm4uVVNaZkxYwYLFizg\n4MGDAEyePJkPPviAnTt3Oq576KGH2L59Oxs3bjxnLmFhYTz55JM89thjF5VbQcGZXXB8fM7Mrb9S\n3tiaxbsZVsfxn66P4fpWfo7jHTsP8+dnvnQ8TnvU7R0ZfUfHK52mNBCpqakAxMfHOzkTkQuje1Ya\nE92v0pjUVw1b5wh/WVkZW7ZsITExsVp7YmLiOQvzTZs20adPH0exfzo+Ozub/fv3O2Jq6zM1NZXK\nytp3qyktLaWkpAR/f/9Lzs1Z7unckm51LOK9Ki6IUbdf5Th+98MdbMs4dEVzFBEREZGmyaWuk3l5\neVRWVhISElKtPTg4GKvVWus1VquV1q1bV2s7fb3VaiUyMpLc3NwafYaEhFBRUUFeXl6NcwBTpkzB\ny8uLW2+99ZJzgzOf7K+0QW52dlSUkl9q5yQw6aNvGdfGjRYuVXv2R4fbCQ4wkPnTCQCenrmacQ+0\nwcNS559ImjBn3asil0r3rDQmul+lMWjbtm299FPvu/ScfopsfZo7dy6LFi1i+fLleHp61nv/V4KH\ni4GxUWYOCiORAAAgAElEQVTMp37j+aV2lu4vo/LUNB6j0cCdt0TgYanaqrPweAX/WZHFeWZciYiI\niIjUqc7h48DAQEwmE7m5udXac3NzCQsLq/Wa0NDQGiPsp68PDQ2tM8bFxYXAwMBq7XPmzOGpp57i\nk08+qTbf7lJyA+fP2fOLPMozG/YCkG2Hbwjk0W6tHR+UfPwjeWZ21U5DB7IrOXjIi+E3tT9nf9L0\naH6pNDa6Z6Ux0f0qjcnZc/h/iTpH+M1mM926dWP16tXV2lNSUkhISKj1ml69erFu3TpKS0urxYeH\nhxMZGemISUlJqdFn9+7dMZlMjrbZs2fz1FNPsXLlyhqvdym5NQTXt/Ljns5ntuL8eE8eH+46M18/\nvktLbhnSznG89J10zecXERERkUt23ik9EydO5I033mDx4sVkZGTwxBNPYLVaSUpKAiA5OZlBgwY5\n4u+++248PDwYO3Ys27dvZ/ny5cycOZOJEyc6YpKSksjKymLChAlkZGTw2muvsWTJEiZNmuSImTVr\nFsnJySxevJg2bdpgtVqxWq0UFhZecG4N1V1XhdI/0t9xvDgti6+zz3yCu2fU1bRvW/VNh91uZ/b8\nrzhy1iJfEREREZELdd6Cf9SoUcyZM4fp06fTtWtXNm7cyMqVK2nVqhVQtRB37969jnhvb29SUlLI\nzs4mPj6ecePGMWnSJCZMmOCIiYqKYuXKlaxdu5auXbvy7LPPMm/ePIYPH+6ImT9/PhUVFdx11120\nbNnS8TN+/PgLzq2hMhgMjOseSYfAqvUINrud5zdmOnbucXEx8vtHe+LtVbXT0bGCEmb//Svtzy8i\nIiIiF+28+/A3Bc7eh/9cjhaXMzHlBw6dLAMg2MPM7MHt8bO4ArB1ey5/fX6tY+Hu7cPi+M3oa5yW\nr1wZml8qjY3uWWlMdL9KY3JF9uGXy8vP4spfbmiDxaVq3cKhk2XM2LCX0oqqkfxrOoYwZsSZB3B9\nsHInm7/NckquIiIiItI4qeB3sihfC39MiMZ4apeejLwi5ny9D9upUf07bu5AfJczi3znLfqaHOtx\np+QqIiIiIo2PCv4G4LqWPjzQJdxxvPano/xja9VIvtFo4PGHryM4sAUAJ4vLmfnSRkpKKpySq4iI\niIg0Lir4G4jb2gVzU5sgx/HyH3L576ntOj09zUwa1wsXl6o/108HC5j36jd6KJeIiIiInJcK/gbC\nYDDw22tb0TPc19G2aMtBNhw4CkCbaH9+e283x7lN3xzgP//LuOJ5ioiIiEjjooK/ATEZDfyhVzTt\nA6qm79ix88JX+9hxuAiAgX2jGTa4jSP+n+9t55vvsp2Sq4iIiIg0Dir4Gxh3FyNP3dCGcC93AMoq\nbfx13R4OFpYAMHZMFzq2Pz31x86cBZs5mF14jt5EREREpLlTwd8A+bi58PQNbfBxcwHgeFkFT63Z\nzdHiclxcjPxhXAJBp74FKC4p57k5Gyg6UebMlEVERESkgVLB30CFebnxlxva4Gaq+hPlnihl6trd\nnCyvxNvLjclPJGA2V30gyLYeZ86CzXoSr4iIiIjUoIK/AYsLaMHks/bo3330JH9dt4fSChsxUX48\n9uCZpwRu+T6Ht9/b5qxURURERKSBUsHfwPUI92VcfGvHcfqh4zy/KZNKm53ePVtzx80dHOc+WPED\nn63JdEaaIiIiItJAqeBvBBJjA7nvmjMP5voq6xhzvt6PzW5nzIiO1Z7E+8ob35K+45Az0hQRERGR\nBkgFfyMxskMoIzuEOo4/35fP4u8OYjQaGP9ID6JaV+3fX1lpY9a8jWTlaOceEREREVHB36iMvbol\nN8YEOo4/2HWIZTuseFhc+dOE3vj5WAAoOlHG9BfXU3i81FmpioiIiEgDoYK/ETEYDDwa35rrW/k5\n2t5Kz+ajHw8TGOBB8sTrHTv35B4qYubcjZSVVzorXRERERFpAFTwNzImo4FJPaPoGuLtaFvw7U+k\n7M2jTbQ/45OuA6p29cnYdZgFr6dit9udlK2IiIiIOJsK/kbIbDLyZO8Y4k49fAtg7tc/8eW+I/SM\nj+Ceuzo72tds2M87H+xwRpoiIiIi0gCo4G+kLK4mnr6hDbF+HgDYsTN78z7WHzjK7cPiGHhDtCN2\n2fvb+XTNXmelKiIiIiJOpIK/EfNyc2Fav7ZEnlqsW2m3M2tjJl9nF/Dbsd24umOII/aVf3xLalq2\ns1IVERERESdRwd/I+bi5MKNfWyK83AGosNt5dsNe0g4f54+PJxATWbXA12az88LLX7Frd74z0xUR\nERGRK0wFfxPgZ3FlRv+2hHm6AVBuszNj3V52FRTz5O/7EBJUNde/rKyCGbPXa49+ERERkWZEBX8T\nEehh5pn+7QhpUVX0l9ls/HXdHn4qLWPKpD54nfowcLyolGmz1nH0WIkz0xURERGRK0QFfxMS3MLM\nM/3bEuRhBqC00sbUtXvIBZ6c2NuxR/+hvBPMeHEdJ4vLnZitiIiIiFwJKvibmFBPN2b0b0ugparo\nL6u0MW3dHgo8XJj0WE+Mxqo9+vfuP8qzf1tPWZkezCUiIiLSlKngb4LCvdx5bkA7x0h/mc3GjPV7\nqAzyIOm+bo647T8c5oWXN1FRYXNWqiIiIiJymangb6LCvNx4bsCZOf3lNjvPrN+LJcaPe+662hGX\nmpbNS4u+prJSRb+IiIhIU6SCvwkL9XRj5sB2jt17Kux2Zm7MJKBzMHfc3MERt/6rn3j1ze+w2+3O\nSlVERERELhMV/E1ckIeZ5wa0c+zTX2m3M2vTPsK6t2TIwDaOuNVf7OGtd9JV9IuIiIg0MSr4m4FA\nDzPPDmhHK++qot9mt/O3zfsJ6xVB34RIR9wHK37gP//LcFaaIiIiInIZXFDBP3/+fKKjo7FYLMTH\nx7N+/fo649PT0+nbty8eHh5EREQwbdq0GjFr1qyhW7duWCwWYmNjWbhwYbXz27dvZ+TIkcTGxmI0\nGnn66adr9DF16lSMRmO1n5YtW17IW2p2/C2uPDegHVE+FgDs2HllywGCrm9FfNcwR9w/39vG/z7Z\n5aw0RURERKSenbfgX7ZsGePHj2fKlCmkpaWRkJDA0KFDOXDgQK3xhYWFDB48mLCwMFJTU5k7dy6z\nZs1i9uzZjpjMzEyGDRtG7969SUtLIzk5mXHjxrF8+XJHTHFxMTExMUyfPp3o6GgMBkOtr9e+fXus\nVqvjJz09/WJ/B82Gr7srzw5oR1xAC0fb29tz8O8dSeerQhxt//hnGh+t/tEZKYqIiIhIPTtvwT97\n9mzuu+8+HnjgAeLi4njppZcICwtjwYIFtca//fbblJSUsGTJEq666ipGjBjB5MmTqxX8r7zyChER\nEcydO5e4uDgefPBB7r33Xl544QVHTHx8PM8//zxjxozBw8PjnPmZTCaCg4MdPwEBARfz/psdbzcX\nZvRrS9cQb0fb//YcxrNPK9q1C3S0vb70O1amqOgXERERaezqLPjLysrYsmULiYmJ1doTExPZuHFj\nrdds2rSJPn364ObmVi0+Ozub/fv3O2Jq6zM1NZXKyot7ENTevXsJDw8nJiaGMWPGkJmZeVHXN0cW\nVxNP3RBLQoSvo+3LA0ex9G5F27Zniv7X3vqOVZ/vcUaKIiIiIlJPXOo6mZeXR2VlJSEhIdXag4OD\nsVqttV5jtVpp3bp1tbbT11utViIjI8nNza3RZ0hICBUVFeTl5dU4dy49e/ZkyZIltG/fntzcXKZP\nn05CQgLbt2/H39+/1mtSU1MvqO/mYJCbnQK3cr45UvUh68s9J4luY8Y7D3IOngRg9t/XsWfPHq7r\nWvvvUy4f3avS2OielcZE96s0Bm3btq2Xfup9l55zzbW/HIYMGcLIkSPp1KkTAwcOZMWKFdhsNpYs\nWXLFcmjMTAYDIyNcuSHozOe+zJN2SuPDCIk4M43qw0+ySd16xBkpioiIiMgvVOcIf2BgICaTidzc\n3Grtubm5hIWF1XpNaGhojdH/09eHhobWGePi4kJgYCCXysPDg44dO7J79+5zxsTHx19y/01Vd7ud\nZTusvJWeDUAhENCvPbHfH8K6p6rQX72mgNjYNgzuF+PETJuH06NOulelsdA9K42J7ldpTAoKCuql\nnzpH+M1mM926dWP16tXV2lNSUkhISKj1ml69erFu3TpKS0urxYeHhxMZGemISUlJqdFn9+7dMZlM\nl/RGAEpKSsjIyDjnhxGpncFgYHTHMB7vHonp1Dc0+aXlHL4qAP+Y01N57Cx4PVW794iIiIg0Mued\n0jNx4kTeeOMNFi9eTEZGBk888QRWq5WkpCQAkpOTGTRokCP+7rvvxsPDg7Fjx7J9+3aWL1/OzJkz\nmThxoiMmKSmJrKwsJkyYQEZGBq+99hpLlixh0qRJjpjy8nLS0tJIS0ujuLiYnJwc0tLSqo3eT5o0\nibVr15KZmcnmzZsZOXIkxcXF3HvvvfXyy2lubowN5M99YnEzVd0WJyttHGrvR4u2Z+bvv770O5br\n4VwiIiIijUadU3oARo0aRX5+PtOnTycnJ4fOnTuzcuVKWrVqBVQtxN27d68j3tvbm5SUFB599FHi\n4+Px9/dn0qRJTJgwwRETFRXFypUrmTBhAgsWLCA8PJx58+YxfPhwR0xWVhbXXnstUDUCvXDhQhYu\nXEi/fv34/PPPHTFjxowhLy+PoKAgevXqxVdffeXITS5e95Y+PDugHU+v3U1BaQWVQH6sL76uJkp2\nHAZg6bvplJZVMvqOjld0zYaIiIiIXDyD3W63OzuJy+3s+U8+Pj5OzKTxyDleyl/W7ibreAkANpsd\nj6wibOmHOF3i3zKkHWPHXKOiv55pfqk0NrpnpTHR/SqNSX3VsPW+S480DWFebjw/sB3tTz2V12g0\nUBzhRVnXECpP3TX/+2QXC9/Ygs3W5D8zioiIiDRaKvjlnHzdXZnRvx09w6se0GUwgD3Uk6KuoZS7\nVN06q7/Yw7xFX1NRYXNmqiIiIiJyDir4pU7uLkb+dH0Md7SvehiawQDuwS041jWYYveqHZXWbNzP\nc3M2UFJS4cxURURERKQWKvjlvExGAw90ieCJ7pG4GAxgAO/AFhy7OpgCbzMAW77PYerMNRwvKj1P\nbyIiIiJyJanglwuWGBvItP5t8TK7gAH8Az042TGQw4EW7MCuPfk8Of0L8vJPOjtVERERETlFBb9c\nlKuDvXhxcBwRXu4A+PtZsLcPICvck0ojHMwuJHna5xzIKnRypiIiIiICKvjlEoR7ufPCoDi6hHgD\n4OPthiXWj/3RPpS5Gsk/cpInp3/Ozt35Ts5URERERFTwyyXxcnPh6b5tuKVtMACenmb8W/uwL9aX\nohauFJ0o4y/PreGr1INOzlRERESkeVPBL5fMxWggqVsrxl8XidloxGJxJTTci6wob/IC3Cktq+D5\nlzbx34930gye7yYiIiLSIKngl19scEwgzw1sR5CHGTc3F8JbenM03PPUvH47b/xrK6+++R2Vldqr\nX0RERORKU8Ev9SIuoAV/G9yeTkFeuLoaCQ/zpizIg31RPpSajXzy2W6em7OB4pJyZ6cqIiIi0qyo\n4Jd642dxZUb/ttzSNhiTyUDLMC9c/S1kRvtQ4G3m2605TJn+BflHtG2niIiIyJWigl/q1el5/RN6\nROHmYiQkqAU+fhaywj3JCfVgz4FjTH76M3ZnHnF2qiIiIiLNggp+uSwGRQfw4qA4wr3d8fe3EBTo\nwVE/d/ZFeWMtKmXK9C9Y/9VPzk5TREREpMlTwS+XTYyfB3MS29O7lR/eXm60DPWkzMOVvTE+5Lkb\nmT3/K5a+k47Nph18RERERC4XFfxyWXm4mvi/hGiSrm2Fl4eZiJZemNxcOBjhhTXEg/dWZPDc3A2c\nLNZiXhEREZHLQQW/XHYGg4Fb2gXz/KA4Inw9iGjphYfFlSP+VVN8Nmy3kvzXz8ixHnd2qiIiIiJN\njgp+uWLiAlowN7E9vSJ8CQvxxNfHnRJ3FzKjffj+RCl/fPoz0tKtzk5TREREpElRwS9XlJebC3/u\nE8tvu7UiLLAFwUEtsJsM5IS1YKevmadmr+fdD3doXr+IiIhIPVHBL1ecwWDg1nbBzE6Mo2NLH8LD\nvDCZjBR6m8mM9ua1j3/gmdnrOV5U6uxURURERBo9FfziNNG+Vbv43NExjFbh3ljcXSh3NbI/0ptP\nco7x+6c+Zc++o85OU0RERKRRU8EvTuXmYuR38a15ul8b2kf64evjDkBeoIWvvUxMeO5LPl2z18lZ\nioiIiDReKvilQegR7svLQzowqEMIocGeGI0GStxd+LG1J3/9YBsvLfqa0tIKZ6cpIiIi0uio4JcG\nI8DDzF/7tWFC7xiiI3wwm03YDQYOBXvwj4NHeGzaZ+w/cMzZaYqIiIg0Kir4pUExGgzcFhfMwls6\n0b9jKF6eZgCKLS5ssBh5YM46Pv5sN3a7dvERERERuRAq+KVBivB2Z3Zie34/oC2hQS0wGAzYjAYO\nBlqY8sWPTJ23kaKiMmenKSIiItLgqeCXBstkNDC6Yxivj7iGXu0CMZtNAJxo4cq7x08yeuYXbN91\n2MlZioiIiDRsKvilwYvx82DhbZ2ZMKAtvt5uANiMBnZ7mPj121uY/973VFbanJyliIiISMOkgl8a\nBVeTkfu6RvDm6Gvp0toPo9EAQLG7iXm7crlz1pfsyypwcpYiIiIiDc8FFfzz588nOjoai8VCfHw8\n69evrzM+PT2dvn374uHhQUREBNOmTasRs2bNGrp164bFYiE2NpaFCxdWO799+3ZGjhxJbGwsRqOR\np59+ul5yk8atXUALlo65lon92uDh5gKA3WBgh93G7a9vZtGKDC3oFRERETnLeQv+ZcuWMX78eKZM\nmUJaWhoJCQkMHTqUAwcO1BpfWFjI4MGDCQsLIzU1lblz5zJr1ixmz57tiMnMzGTYsGH07t2btLQ0\nkpOTGTduHMuXL3fEFBcXExMTw/Tp04mOjsZgMPzi3KRpcDEaeLBHJO/fdx2dQ7w4fWuUuBiZ/X0W\nI/62jn2Hjjs3SREREZEGwmA/z3Bojx496NKlS7UR+Hbt2jFy5EieeeaZGvELFiwgOTmZ3Nxc3Nyq\n5lvPmDGDBQsWcPDgQQAmT57MBx98wM6dOx3XPfTQQ2zfvp2NGzfW6LNz587ceeedPPXUU5eUW0HB\nmakePj4+db1daWRsdjtLNu3npQ17Ka44M4/fDNzfJYLHh8RhrOXDYkOWmpoKQHx8vJMzEbkwumel\nMdH9Ko1JfdWwdY7wl5WVsWXLFhITE6u1JyYm1lqYA2zatIk+ffo4iv3T8dnZ2ezfv98RU1ufqamp\nVFZWXlDil5KbND1Gg4H7EqJY8VAvrglo4WgvA15JO8jQl9eTduCo8xIUERERcTKXuk7m5eVRWVlJ\nSEhItfbg4GCsVmut11itVlq3bl2t7fT1VquVyMhIcnNza/QZEhJCRUUFeXl5Nc7VV25w5pO9ND1/\nuNaVT3e58K89JRQbqz7LZhYWM/qtb7g+0IV7r/bEw6XxrFPXvSqNje5ZaUx0v0pj0LZt23rpp96r\nn9rm2otcKYPaeTK7ry+djRUYTs1Ws9lh3eEKJq4tYF1WqRb1ioiISLNS5wh/YGAgJpOJ3Nzcau25\nubmEhYXVek1oaGiNEfbT14eGhtYZ4+LiQmBg4AUlfim5gebsNRcDb4BPNv/EtNU7yXep+hBaVAmv\n/lDC14Vmpg5tT1ygp5OzrJ3ml0pjo3tWGhPdr9KYnD2H/5eoc4TfbDbTrVs3Vq9eXa09JSWFhISE\nWq/p1asX69ato7S0tFp8eHg4kZGRjpiUlJQafXbv3h2TyXRBiV9KbtK8DOnRmtW/78ttgV64nFrQ\na7PZ+e5gAaOXfstfP9tF3skyJ2cpIiIicnmdd0rPxIkTeeONN1i8eDEZGRk88cQTWK1WkpKSAEhO\nTmbQoEGO+LvvvhsPDw/Gjh3L9u3bWb58OTNnzmTixImOmKSkJLKyspgwYQIZGRm89tprLFmyhEmT\nJjliysvLSUtLIy0tjeLiYnJyckhLS2P37t0XnJtICw8zMx/qwRvDryaq1OaY5lNcXM6/vvmJO976\nhre/z6K0Qk/qFRERkaapzik9AKNGjSI/P5/p06eTk5ND586dWblyJa1atQKqFuLu3bvXEe/t7U1K\nSgqPPvoo8fHx+Pv7M2nSJCZMmOCIiYqKYuXKlUyYMIEFCxYQHh7OvHnzGD58uCMmKyuLa6+9Fqha\nF7Bw4UIWLlxIv379+Pzzzy8oN5HT4juF8mHbQF55J52lGVaOe7lit8OhI8U8t3oX72+z8njvaPq0\n8tM6FBEREWlSzrsPf1OgffjlbD/uyefZpd/xrb2SUrczU8i8PM30jAkk6brWdAxy3vx+zS+Vxkb3\nrDQmul+lMbki+/CLNEVtYwN4dcoApveKJjK/BFNl1XSe40VlfLYth6T3v+eva3ezv6DYyZmKiIiI\n/HIq+KVZMpmM3HJjO/715EB+5eOJ/5ESDHY7Npudw/kn+c83B3jgg3Tmfr2fw1rYKyIiIo2YCn5p\n1vz9LPxpXAJ/vyeengUV+BRU7S5VWlbJwexC3tq8n/s/SOcfW7MoKqtwcrYiIiIiF08FvwjQ9epQ\nFk5LZMJ1kbT7qQjPonIAiorK2P3TMV7dtI/7/7uNd3dYKS6vdG6yIiIiIhdBBb/IKWazidF3dOLV\nvw7mrlAfWv9UiHtJBXa7nSNHi9mReYS/b9rHAx9tY/kPuZRoK08RERFpBFTwi/xMSFAL/jAugdmP\n96ZfhZHwrCLMZZVUVNiwHioiY99R5m/ex4MfbePDnYe0h7+IiIg0aCr4Rc6hU4dgXvzrIP54R2eu\nyS0hLOcEruU2iksqOJh9nB9+Osbfv97PQyu28dGPhymrVOEvIiIiDc95H7wl0pyZTEZuHBDL9de1\nYtn721n5+R7yvV3JC7RQdKKMEyfLKCh04+WiMt7LsDKifQiJMYG4ueiztIiIiDQMKvhFLoCnp5kH\n7unK0MFtePvddDakHuSYrxt5gRYKCks5XlRGoY87C4pKWbbDyu1xwQxrE4SHq+n8nYuIiIhcRir4\nRS5Cy1Av/jAugVt35/Pmv79n+4+HOernTl6gO0eOFlNQWEqhrzuvF5fzXkYuN7cN4tZ2wXi76V81\nERERcQ5VISKXIK5NANOf7Efqdzm89c73/PTjMY76uZMf4E5e/kmOFZTg5+POP0sreH9nLkNjg7ij\nfQj+Fldnpy4iIiLNjAp+kUtkMBjofm1Lrr0mlM/X7ePfy7fjt/sYBT5u5Ae4c7jCVlX4+1p4/4dc\nVvx4mAHR/tweF0Irb3dnpy8iIiLNhAp+kV/IZDIyuF8MfRMi+eSz3Sz/6Ad89xRQ6G0mL9DCoQob\nRwuK8fe18MnuPD7Zk0f3lj4Mjwvh6mBPDAaDs9+CiIiINGEq+EXqidls4tahcQzuH8PHn+7hgxU/\n4L23gCLPql19csttHDlWNdXn66wCvskuoI2fB7fHBdOntT8uRhX+IiIiUv9U8IvUM4u7K3fc3J4h\nA2P5aNWP/PfjnXjuK+SkhwtH/N05VF7JkWPF+Pq486MdXvhqH29szeaWdkEMiQ10dvoiIiLSxKjg\nF7lMPCyujLr9KoYNbsP/PtnFR6t+pMXBIkrNRo74u5NfXsnRYyX4+rhhs9n5x9Ys/rkthzjXMhIC\n9K+miIiI1A9VFSKXmWcLM2NGdOKWIe34+NPdfLTqR9ysJwk6XMxRP3eOllZw9FgJPt7u+Hq78VVh\nJV/lV7K+bBc3tw2iV4SvpvuIiIjIJVPBL3KFeLYwc+dtV3HLje1I+XIvH6zciUteMQH5xRR6u5Ef\nUMGxghLc3cCzhQvbDh9n2+HjBFhcGRpbNd3HT9t6ioiIyEVSwS9yhbm7u3DLkHbcODCWL9fv5/0V\nP2A8VIRPQSknPVw45Gki19edk8Xg6+NOPrB0Wzb/3p7D9a38GNImkM5B2t1HRERELowKfhEnMbua\nSOwfw8Abotj49UGWf/QD+w8cI/hwIRWuRspa+pHrV4pLCzM+3m54tjCz5qcjrPnpCBFe7gyOCWBQ\ndAC+7hr1FxERkXNTwS/iZCaTkT69WtO7Zyu+33GI15asY9eeInxOTfc57mWueoqvd1Xh7+3pxsHj\nJfxjaxZvfZ9NjwhfbowJoGuoN0aN+ouIiMjPqOAXaSAMBgPXdAzh3lFR5B4uYX+2O2s27MdYWIZP\nYRklbiaO+rlxwNcdi5cbPt5u4GZiw4GjbDhwlGAPM4kxgQyKCSDIw+zstyMiIiINhAp+kQYoJMid\nm4bGM2ZEJ1Z9vpuPP90DRaWEWU8SknuSQm8zR3zdsfm5431qus+hk2Us3ZbNP7fn0CXEi4FRAfSM\n8MXdxejstyMiIiJOpIJfpAHz83Vn9B2duOPmDqzb9BMff7qbvfuP4ltQhm9BGSXWExzzdSPL3x0P\nn6ri38XFyBZrIVushVhcTPRu5cuA6AA6BXlqyo+IiEgzpIJfpBEwm00M7BvNgBui2LXnCB+n7Gbj\nNwdwL60kNPcktkMnOe5lJs/XDXugB95eZjw8zBRXVJKSmU9KZj5BHmYGRPkzICqACG93Z78lERER\nuUJU8Is0IgaDgbg2AcS1CWDs3dfw6Zq9rP5iL3n5J/E5Nde/LOcEhd5mcgMsuPpb8PI0YzabOHyy\njGU7rCzbYSUuoAX9Iv3p3coPf+3tLyIi0qSp4BdppHx93Bl561UMv6k9qWk5fPLZHrZuy8VcbiMw\nv4SA/BJK3As55utGZagn7l5utGjhitFoYGf+CXbmn2DRloN0Cvakb2s/Elr54eOm/ySIiIg0Nfq/\nu0gjZzIZ6dEtnB7dwsk9fIIv1mXy+bp95OWfxFJSicV6ElvuSYo8XTnmb8EW3IIWnmYs7i7YDXbS\nDx0n/dBxFnx7gC4hXvRu7UdChC+eZv3nQUREpCm4oO075s+fT3R0NBaLhfj4eNavX19nfHp6On37\n9qL1kD8AACAASURBVMXDw+P/27v34LjK++Dj33Pdm3Yly7pZsi0bX+ILt4AFyDEkhdgNdMqbuFBM\n+07eMG9oZigvMbw0xDOQUgdoWlomtAMeD+k0/gNP3FYNnWHSgt6hdWxsSoRNfJExGMt371qypJX2\nfnveP87ualeS5QuyLVm/z8xyznnO75w9K585PL/zPM85zJw5kx//+McjYrZu3cqtt96Kx+Nh3rx5\nbNy4cURMW1sbS5Yswe12s3TpUt56662y9c8//zy6rpd9GhsbL+QnCXFNqq/1sWb19Wz4m/t49v/e\nSWvLTAxDR1cQGEwz4+gA9buCqN1B+j7vpbc3TiqVBSCrFB8FB3j1w6P8z7f2sP7Xh3jvyFkiqcxV\n/lVCCCGE+CLOewtvy5YtrF27lg0bNrBixQpee+017r33Xjo7O5k1a9aI+IGBAVauXMnXvvY1Ojo6\nOHDgAI888gg+n4+nnnoKgK6uLu677z6++93vsnnzZrZt28Zjjz1GbW0tq1evBmDnzp2sWbOG9evX\ns3r1atra2njwwQd5//33ue2224rft2jRIv7rv/6ruGwYxhf9mwgx6RmGzi03zeCWm2bQH06w9f2j\n/L+tXZw8PYCZVUzrT0J/krTZz6DfJtXoJ1Plwl9hYxg66Zziv0+F+e9TYUxN46Z6P60zq7ijqYpp\n0udfCCGEmFQ0pZQaK+D222/n5ptvLrsDv3DhQh544AFeeumlEfEbNmxg3bp1hEIhXC4XAC+++CIb\nNmzgxIkTADzzzDO89dZbHDx4sLjdo48+yv79+9mxYwcADz30EP39/bzzzjvFmJUrV1JbW8vmzZsB\n5w5/W1sbe/fuHfNHhsPh4nxlZeWYsUJcbR0dHQAsW7ZsXPerlOKzz3vZuuMo2z84zmAkWbY+beoM\nVLowZgfIVNjF/v6lNDSW1Pi4Y2YVy2dW0VDhGtdjFJPT5Tpnhbgc5HwVk8l41WHH7NKTSqXYtWsX\nq1atKitftWpVsWI+3M6dO7nzzjuLlf1C/KlTpzh69GgxZrR9dnR0kM063Qs++OCDC/rew4cP09TU\nxHXXXcfDDz9MV1fXWD9JiClL0zQWzp/Oo9++hX/4u99n3doVtLbMwjKdVjErk2P62ThVu0MEPjhJ\n7KMgiWCUwUiKXM65L6BQ7O+J8A8fn+B/v72P//MfB3hz3yk+74txnnsHQgghhLhKxuzS09PTQzab\npb6+vqy8rq6OYDA46jbBYJDZs2eXlRW2DwaDNDc3EwqFRuyzvr6eTCZDT08P9fX1BIPBUWNKv/eO\nO+5g06ZNLFq0iFAoxAsvvMDy5cvZv38/1dXV5/npQkxdpqnTcksjLbc0Eo2l2PmbE2x9/yj7P+kG\nwE7nmN4dg+4YaVMnWuXCnl1JosLC4x2683+4P8bh/hib952mxmPT0higpbGSm+oD8oZfIYQQYoIY\n98dwaFfwTZ7f+MY3ivPXX389ra2tzJ07l02bNvHkk0+Ouk2hKU+Iie5KnqtVPvgfq/x89XYXew6E\n2ffJACdPx4vrrXAEdfQspqkTqXZjzgqQCLiwXUax8n8sGuNYTz9te8DUYX6FzmK/waKATrUtlf+p\nQK6vYjKR81VMBgsWLBiX/YxZ4a+pqcEwDEKhUFl5KBRixowZo27T0NAw4u5/YfuGhoYxY0zTpKam\nZsyYwj5G4/V6Wbp0KYcOHRrrZwkhzqGq0uauO2q5645aevtT7DsQZt8nYU4GEwAYmRzeMzE4E8PU\nNRJVLvRZAVLT3OguE9NwKv+ZHHwykOOTgRychAa3xqKAwSK/TrNXx9Sv3I0BIYQQYqobs8Jv2za3\n3nor7777Ln/wB39QLG9vb+fBBx8cdZvW1laeeeYZkslksR9/e3s7TU1NNDc3F2N++ctflm3X3t5O\nS0tL8Sk7ra2ttLe38/TTT5fFfOUrXznn8SYSCQ4cOMDdd999zhgZpCMmuok0oGzV151pMBRhx4fH\n2fHhCQ4f7Suu9yeBQxEUEeIeE3t2AFXnI24buFxDT8waAD6MOh+3qXNDnZ9bGgJ8uSHATL/rirYM\nivE3kc5ZIc5HzlcxmZQO2v0ijOeff/75sQICgQB//ud/TmNjIx6PhxdeeIHt27fzj//4j1RWVrJu\n3Tp+8pOf8O1vfxtwnuCzceNGPv74YxYvXsz27dv5sz/7M9atW0draysA8+fP56/+6q/o7u6mubmZ\nf/u3f+Oll17ilVdeYfHixQA0NTXxox/9CJfLRU1NDW+88QY///nPeeONN2hqagLg6aefxu12k8vl\n+PTTT3n88cc5fPgwGzduLBvJnEwOPY3E7XaPyx9OiMvl1KlTABPqnRIVFTaLv1TLqrvncWfrbKZX\ne0kmM5ztde78aziDfvWzcYxjA2gnBjDiWdwugxig61qxUp/JKU4NJvno9ABvf9ZNe9dZjoYTpLI5\nprktXNL3f9KZiOesEOci56uYTMarDnvePvx/+Id/yNmzZ3nhhRc4ffo0N9xwA7/61a+Kz+APBoMc\nPny4GB8IBGhvb+dP//RPWbZsGdXV1Tz99NNlfernzJnDr371K5588kk2bNhAU1MTf//3f8+3vvWt\nYkxrayu/+MUvePbZZ/nRj37E/Pnz+ad/+idaWlqKMSdPnuThhx+mp6eH2tpaWltb+eCDD0Z9P4AQ\nYnw0Nvj51u8t4lu/t4i+/gS/2X2KDz86yZ7OEJlMDnAG/XIsTO5YGJ8GyUoXnjlVpKrcJC0dyxqq\n1HfHUrx7uId3D/egoTG/2svN9X5urPOzpNaH25R3awghhBBfxHmfw38tkOfwi8lksjY3x+JpPt4b\n5MOPTvHRb08TjaVGjUtZOlZTALvJz6DbANvgXD16TE1j4XQfN9RVcGOdn0U1FfL0nwlosp6zYmqS\n81VMJuNVhx33p/QIIaYmr8di+W2zWH7bLDKZHJ0Hu9m9J8iuPac5fnKgGGenc3Ckn/SRflxAJuDC\nP7eKdLWbsKFhWYbTRwjIKEVnT4TOnghbOoOYmsaXpvu4oc7PjfUVLJpeIV2AhBBCiPOQCr8QYtyZ\nps6NS+u5cWk9/+vhmzjTE2X3niAf/fY0e/afIZXKAPm+/wNJEr91nuTl0TVUjRff7EpSlTaDulbW\n/SejnBd/7e+J8ItOsHSNhdU+ltRWsLjGx+KaCgIuuawJIYQQpeT/jEKIy66uxsfv3j2P3717HqlU\nlgOfdrNrT5Bdvw1y8vTQ3X8jp+BMlOSZKABuQ8Oe4cc7K0DcbzOoKQxjKAFI54YSgIJZATdLaytY\nXFPBkhofMyrkKUBCCCGmNqnwCyGuKNs2uOn6Bm66voFH/gjO9ETZu/8Mv90fYm/nGcIDiWKsmVXk\nTgwQOeEkBbah4W4K4GryE/NZRHSnNaHU8YEExwcS/MfnPQBMc1ssrvGxpKaCJbUVzK3yYBvSDUgI\nIcTUIRV+IcRVVVfj456vzuWer85FKcWxE2H27D/Db/eF2H+wm2QyU4w1s4rMsTCZY84gJpehYTVU\n4JsZIOW3CeugD0sA+hJpdpzoZ8eJfmcfmsZ107wsnO7lS9N9LKz20eh3oUsrgBBCiGuUVPiFEBOG\npmk0z6qieVYVv/+NhaQzWT77vJc9+0PsO9DNZ5/3ks5ki/FmVqFODhI5OQiASwOz1ot/dhXZKhe9\nOuQMvewpQBml+LQ3yqe9Ud7+rBsAn2WwsNrnJADTvSys9jHNY13R3y6EEEJcLlLhF0JMWJZpsORL\ntSz5Ui0AqVSWQ129dB7sZv8nPXzyWU9ZC4CuIHcmRvhMzNkeyHpNKmZVYdR5ibkNItrIbkDRdJbd\noQF2h4bGE9R6bRZUe5k/zct105ypJAFCCCEmI6nwCyEmDdseSgAeuB8ymRxdR/vYf7CHzoPdHDjY\nU/b8fw0wYxkSB3vgoFPmMjSsWh/+mQFylS7CpkZaA00v79LTHUvRHUsVuwIBVLutYuX/umke5k3z\nUu+zZVCwEEKICU0q/EKIScs0dRbMm86CedP55n1fIpdzxgAcPHSWTz7r4dNDvZwODZZvk1WoYISB\noPNkHx0wbIOKpgB2vY+kzyKsg2ZoI5KA3kSa3tNhOk4PvQilwjK4bpqXefkkYG6Vh5l+N5YMDBZC\nCDFBSIVfCHHN0HWNObOrmDO7it+9ex4Ag5EkBw+d5dNDvXx66CyfHj5LIjHUDUgDrFSWZFcfya4+\nANxAymPia/TjqvWRqbAIa4Cpj3grcCSdZc+ZQfacGUosTE2jKeBmTqWH5ko3c6o8NFd6pDVACCHE\nVSEVfiHENc1f4WLZzY0su7kRgGw2x/GTAxw8dJaDh87yeVcfJ04NoJQqbqMBrniGzOd9ZD53kgAX\nkPWY+Gb4MWs8ZCpsBnSNrKFhGOWV+IxSHA3HORqOl5V7TYPmKg9zKt00V3ry8x55WZgQQojLSv4v\nI4SYUgxDH9EKkEhk6DrWz+ddvXze1cfnR/o4cWoQKE8CzHiG5OE+koedJMAG0paOUe3BXV+BVuki\nbuvENLAsY0RrQCyT5UBPhAMlLwoD510BMwNuZvrdzK50prMq3dR4LGkREEII8YVJhV8IMeW53SaL\nF9aweGFNsSwWT9N1tJ/DR/o4lE8ETgXLxwNogJ3OQShKOuS8HdgAvDpkPBYVjX7MaR7SXpOIoZHR\nNSxTdzYs0ZdI05dIs/dM+f7dpl6WBMwMuJkVcDOjwiVjBIQQQlwwqfALIcQovB6LpYtqWbqotlgW\ni6c5ejzM0eP9HDmWnx4Plz0aFMDIgRFNk/6sl3S+zAUYpkbWa+Op96FXuZ1EQNfQTQ3LNEYkAolM\njkN9MQ71xcrKTU2jvsLFjAoXTX5n2uh30Vjhps5nY+jSKiCEEGKIVPiFEOICeT3WiJaAXE4ROhPh\nyPEwR4715z9hus9Gy7bVACujsAaSqIEkWZwnBPlxugVlPCbeOh9mlYeM1yRmaGTyXYP0UcYInBxM\ncHIwQcfp8mM0NY2GChcNw5KBJr+bWq8kA0IIMRVJhV8IIb4AXdeY0eBnRoOf1paZxfJoLMXR42GO\nnxzgxKkBjp90Pn395QN5C92C7HQKBlJkcMYHeICsoZF0GWgBN/Z0D6rCImkbJHQnEbBMfcSjQzNK\ncWIwwYkxkoF6n02dz0V9hU29z6be55RVukwZMyCEENcgqfALIcRl4PPaZW8JLohEU8UEoDQRONtb\n3m1Hw3lngBnLQCwC+fcGeAG3BinbIGXr2NUe7EoPOZ9JwtRJ6s4bii1TH7VloJAMjMZt6jT4XNTl\nk4CGCmda57Np8Lnw2ca4/X2EEEJcOVLhF0KIK6jCZ7NoQQ2LFtSUlcfiaU7kk4DToQingoOcCkY4\nHYqQSpWPEdAVuJNZ3MksDKaBAXSgAvAWkwEDKiysKjf4LKdM1zBNHcsyMEd5p0Aik+NIOM6RYY8T\nLfDbJjVei1qvTY3Xzk8tuiNZqiyNdDYng4mFEGICkgq/EEJMAF6PxcL501k4f3pZeS6n6O2LczI4\nyOl8EnDq9CCnQoOc6Y6Sy6my+PJkIAWnnbEEHiCrQ9oySFs6MdvAqnRh+F1k3QZJU0flE4LCRx/W\nXWgwlWEwlaFrWLekWDQFwN8d3800t1VMBAqJQel8tduScQRCCHGFSYVfCCEmMF3XqJnupWa6l5uW\n1petS2eynOmOcup0hFBPlNCZCKHuKKEzUULd0REtA0YOjEIyQBp6h7r2VOCMGUhbOmnLIGrr4LUw\nKmxyHpOUpaMbellCYBrnfsTop72j/x4NjSq3yXSPxTSPRbXbotpjMd3jTKe5nfkqSQyEEGLcSIVf\nCCEmKcs0aJoRoGlGYMQ6pRT94SSh7kgxATjTHXWWu6P0nI0z4sViWYWZzeJJZJ3Cs0MJgQIypkbG\n1ElbOol8S4Hhs0gZOXIuk2gcTEPHMDVnmk8QDF0rJgYKVUwK8uOTR6VrGlUuk2rPUEJQSBCq3BZV\nbpMqlzP1WDK2QAghxiIVfiGEuAZpmsa0KjfTqtwjxgsApNJZunui9PTG6emJ0X02Rk/h0+tMU+ns\n0P7IP1Y0U0gI0sV10agz4Njr8zqPGM0nBSlLJ2bqZFwGmtdCuQxyloFh5BOCfDJgGDqmoaEbQ+MK\nckrRm0jTe57EAMBl6CVJgEmV22kpqMwvV+bXTXNbVNgGujyJSAgxxUiFXwghpiDbOnfrADgtBAOD\nybJEYGg+Ts/ZGH3hBMNbCZxHjOZg9HG/5DTIGjoZUysmBwnTmWZMHTymkxiYTguBYRQSgsK8li8f\nSg6S2RyhaJJQNHne321qGoF8UhBwmQRcBn7bzM+bBPLzfpdBwGVS6TJxGbo8rlQIMalJhV8IIcQI\nmqZRGXBTGXAzf271qDGZTI7+cIJt2zsYiKSprZtNb1+C3v44vX1x+vrj9PYliMZSxW10BXomh5Vh\nqOvQKAqJgZMU5LsSmTpx01nOGjrKZaBsAy2fDOh6SVKgO4OOhy9nKGk5uEC2rpclAcUEoZgoGPhd\nJhWWQYVtUmEb+CxDnlgkhJgwpMIvhBDikpimTs10L7OavAAsW7Zg1LhEIuNU/vsT9PY5yUBvf5y+\n/gThgQT94ST94QSDkRSFFoOhxCA35jEoIKdDxtSLLQeF+WQ+MciYQ+Wa6Qw+dpIAp7WgkBjo+lBi\nUFqWIkdPPEXPOVotzsVl6PgLCYDttCR4LQN/yXIhOaiwTXz5dRW2icuUZEEIMX6kwi+EEOKycrvN\n4tuIx5LN5hgYTNEfzicC+WQgHHbmwwP5+XCS8GCCXE6hkX/6UCoHjJ0cQHmXoqyhkzW0/McpS45S\nprRCMuBMdUMfmi8rL8w7SUMuq0hkchedKIDTquCzDbyWgcfMz5sGHstJEHyWgcfS8RbnDbyms1z6\nMeVJR0IIpMIvhBBigjAMvTjQ+HxyOUUk6iQHkWiKgcEkA4MpIhFnOhBJMpgvG4wmGRxMEY2lyroU\nwbm7FJV9V1mSUJoo5Mci5OdzulZMFnK6jsrXtTWNYnKga0OJwlhlcU1jMF5edinDCFyGji9f+fdY\nBl7LWXabTiLhsQzcho7H0otlbtOZd5t6WYzb1KWbkhCTlFT4hRBCTDq6rhHwuwj4XRe8TSaTYzCS\nYjCSZDBSSBKSRCJOghCJpohEUkRjaSJRZxqNpUgkMiVJwoUpdDUaLREYakHQyOka6XwCURqnRqnd\nFxMHTUMrm5JPCLSSKcX1vfnl8vXOdtpFtgBYulZMBpyEYCgxcJtOq4PHNHDl17uMko85NLUNrazM\nberYhi5PUBLiMpEKvxBCiCnBNC+8BaFUJpMjGks5SUA0XUwEItHRy6JRJ2GIxdPE4mly6fN3NSql\nAKVRTAIKiYAzr5+jXCM7fNm4sMrziCRBG1p25kuThfx8oVwvmS/bpnz+Quvxtl6eGIxIEIxR1ufn\nLV3DNnSsfKylO/O24Wxv6U5SMZhWmDqksjms/G8X4lp3QRX+119/nZdffplgMMjSpUv56U9/yooV\nK84Zv3fvXh5//HF+85vfUF1dzfe+9z2ee+65spitW7fy1FNP0dnZSWNjIz/4wQ/43ve+VxbT1tbG\nc889x+HDh5k3bx4vvvgi3/zmN7/QsQkhhBAXwzT14hOLLpZSilQqSyyeIRZPEY9niMbTxONp4vEM\nsXiaeCLtrI85CUI87izHE+liWSyeJpdT5//C0u/GaWUoJABDCYI+onzoU1KujVyvvkDduDRhcJKG\n8oTiUqb68HI0tGGJhqZpznvf8sceizovlPN27QZwEoJisuAkDoUEwjL0YeudxMFlOOssQ8PWdUxd\nK34K+ygu6857JkzNWbaN0eMNTcMyNGnlEJfFeSv8W7ZsYe3atWzYsIEVK1bw2muvce+999LZ2cms\nWbNGxA8MDLBy5Uq+9rWv0dHRwYEDB3jkkUfw+Xw89dRTAHR1dXHffffx3e9+l82bN7Nt2zYee+wx\namtrWb16NQA7d+5kzZo1rF+/ntWrV9PW1saDDz7I+++/z2233XZJxyaEEEJcSZqm4XKZuFzmRbcs\nlFJKkU7niMbSJJMZ4okMiWSGRGFaMj9ifSJDMpkdsU0ylil7udp5jwGn5WF4AlDogjRa8jBUPjTv\nTJ3l7LB1l1Oh8p/LZQEN00wPJQn5dZQtl5aVJxul+yvsg5JtyuOB0n0WY8v3U5h3EoGh5MIyhiUP\nuuYkFEY+riR5MPJJhVGybOhOsmHoGoZG2brSWDPfilOILawrxuWnev4Yy/dbHm/IYPEJR1NKjXnL\n4Pbbb+fmm29m48aNxbKFCxfywAMP8NJLL42I37BhA+vWrSMUCuFyOX0rX3zxRTZs2MCJEycAeOaZ\nZ3jrrbc4ePBgcbtHH32U/fv3s2PHDgAeeugh+vv7eeedd4oxK1eupLa2ls2bN1/UsYXD4eJ8ZWXl\nBfxZhLh6Ojo6AFi2bNlVPhIhLoycs5NXNpvLJwlZ4ok0iWSWVCpLKuWUpdJZksksyVSGZMpZl8yv\nT6VyJFOZkrLSdeVl2ez5uzWVJhSlScG5EobR5pVG2baFskJyojSNjMqhNNBMY9SxEhNNWcLgzJYl\nGKOtK/yscyUVlMWMsW7Y941WVvwLFsry+ylNBgydYiKgayVl+WWzUF6SaBTKjZJPIRnSdTB0vZjA\naAwlI7o2lNxohe8qjH/B+d7CMegaJcfkHKMTN1Re2GehRUqnfNnIJ3ZG/o9mFI6DoVYoPb++2CpV\nEqcx8jtKjVcddsw7/KlUil27dvGDH/ygrHzVqlXFivlwO3fu5M477yxW9gvxzz33HEePHqW5uZmd\nO3eyatWqEfvctGkT2WwWwzD44IMPeOKJJ0bEvPbaa5d8bEIIIYQYYhg6Pq+NzwvguWzfk8nkypKH\nVMpJJtLpHOl0lnQmP03nSKWyZLLONJ0pTLPOPlLZ4r6K26ZzpDLly+lRlgui0RgAPp93WJIxlCQM\nJQwXs2606VjrRo8Zzrktqyi/PXtx3buuVUN/rvJxIlr5f86xLr9dfrnsL1+SvAybFIMLy2X/ZJo2\nyn5KV5f/+5YdVyHpKnzyScPbf3Q942HMCn9PTw/ZbJb6+vqy8rq6OoLB4KjbBINBZs+eXVZW2D4Y\nDNLc3EwoFBqxz/r6ejKZDD09PdTX1xMMBkeNKXzvpRybEEIIIa4809QxTR2vx7oq35/LqWKi0NGx\ni2xWsfT6G8hmc2QyOTJZZ70znxuaL/0MK09ncvn9OonJ8Lhsfp/OuizZrHLm0/n1uSy5rCKTdfaT\nzc9nsoqsypHKAaiy5AKGEoTSxKMwhfIkojS+MP5ieKJxvpjz7XO08itlKAkanhANlQvHuD+lZ6KP\ndi9tGhFiIlqwwHlbqZyrYrKQc1ZMJl++eXF+rnDXv3BPVd4xIK5dY57dNTU1GIZBKBQqKw+FQsyY\nMWPUbRoaGkbcYS9s39DQMGaMaZrU1NSMGVPYx6UcmxBCCCGEEFPNmBV+27a59dZbeffdd8vK29vb\nWb58+ajbtLa2sm3bNpLJZFl8U1MTzc3NxZj29vYR+2xpacEwjDFjvvKVr1zysQkhhBBCCDHlqPPY\nsmWLsm1b/exnP1OdnZ3qiSeeUH6/Xx07dkwppdQPf/hDdc899xTjw+GwamhoUGvWrFH79u1TbW1t\nKhAIqFdeeaUY09XVpXw+n1q7dq3q7OxUb7zxhrJtW/3rv/5rMWbHjh3KNE31k5/8RB04cEC99NJL\nyrIs9eGHH17wsQkhhBBCCDHVnbfCr5RSr7/+upozZ45yuVxq2bJlatu2bcV13/nOd9TcuXPL4vfu\n3avuuusu5Xa7VWNjo1q/fv2IfW7dulXdcsstyuVyqeuuu05t3LhxRMy//Mu/qEWLFinbttWSJUvU\nL3/5y4s6NiGEEEIIIaa68z6HXwghhBBCCDF5XTND0l9//XXmzp2Lx+Nh2bJlbN++fcz4vXv38tWv\nfhWv18vMmTP58Y9/fIWOVIiLO1+PHDmCrusjPsPHrwhxOfz617/m/vvvZ+bMmei6zqZNm867jVxf\nxdVyseerXF/F1fSXf/mXtLS0UFlZSV1dHffffz/79+8/73aXco29Jir8W7ZsYe3atTz77LN8/PHH\nLF++nHvvvZfjx4+PGj8wMMDKlSuZMWMGHR0dvPrqq7z88su88sorV/jIxVR0sedrwTvvvEMwGCx+\nfud3fucKHbGYyqLRKDfeeCOvvvoqHo/nvI9eluuruJou9nwtkOuruBq2bt3K448/zs6dO3nvvfcw\nTZOvf/3r9PX1nXObS77GXu0+RePhtttuU3/yJ39SVrZgwQK1bt26UeNff/11VVlZqRKJRLHshRde\nUE1NTZf1OIVQ6uLP166uLqVpmuro6LgShyfEOVVUVKhNmzaNGSPXVzFRXMj5KtdXMZFEIhFlGIZ6\n++23zxlzqdfYSX+HP5VKsWvXLlatWlVWvmrVKnbs2DHqNjt37uTOO+/E5XKVxZ86dYqjR49e1uMV\nU9ulnK8Fq1evpr6+nhUrVtDW1nY5D1OISybXVzEZyfVVTAQDAwPkcjmmTZt2zphLvcZO+gp/T08P\n2WyW+vr6svK6uroRL+4qCAaDI+ILy+faRojxcCnnq9/v52//9m/553/+Z/793/+de+65h4ceeog3\n33zzShyyEBdFrq9iMpHrq5hIvv/97/PlL3+Z1tbWc8Zc6jXWHJ9DnFwutE+fEBPB9OnTefLJJ4vL\nt9xyC2fPnuWv//qv+eM//uOreGRCjCTXVzGZyPVVTBRPPfUUO3bsYPv27WNeRy/1Gjvp7/DX1NRg\nGAahUKisPBQKMWPGjFG3aWhoGJEFFbZvaGi4PAcqBJd2vo6mpaWFzz77bLwPT4gvTK6vYrKT66u4\n0p588km2bNnCe++9x5w5c8aMvdRr7KSv8Nu2za233jriEVrt7e0sX7581G1aW1vZtm0byWSyLL6p\nqYnm5ubLerxiaruU83U0H3/8MY2NjeN9eEJ8YXJ9FZOdXF/FlfT973+/WNlfuHDheeMv+Ro7GBwf\ntwAAAatJREFUbkOLr6ItW7Yo27bVz372M9XZ2ameeOIJ5ff71bFjx5RSSv3whz9U99xzTzE+HA6r\nhoYGtWbNGrVv3z7V1tamAoGAeuWVV67WTxBTyMWerz//+c/V5s2bVWdnp/rkk0/Uyy+/rGzbVj/9\n6U+v1k8QU0gkElG7d+9Wu3fvVl6vV61fv17t3r1brq9iQrrY81Wur+Jqeuyxx1QgEFDvvfeeOn36\ndPETiUSKMeN1jb0mKvxKOY8pmjNnjnK5XGrZsmVq27ZtxXXf+c531Ny5c8vi9+7dq+666y7ldrtV\nY2OjWr9+/ZU+ZDGFXcz5umnTJrVkyRLl8/lUIBBQLS0t6s0337wahy2moP/8z/9UmqYpTdOUruvF\n+UceeUQpJddXMbFc7Pkq11dxNQ0/Twufv/iLvyjGjNc1VlNKqfFokhBCCCGEEEJMPJO+D78QQggh\nhBDi3KTCL4QQQgghxDVMKvxCCCGEEEJcw6TCL4QQQgghxDVMKvxCCCGEEEJcw6TCL4QQQgghxDVM\nKvxCCCGEEEJcw6TCL4QQQgghxDVMKvxCCCGEEEJcw/4//RuYnBMuD30AAAAASUVORK5CYII=\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import matplotlib.pyplot as plt\n", - "\n", - "thinkplot.PrePlot(num=2)\n", - "thinkplot.Pmf(time_dist1)\n", - "thinkplot.Pmf(time_dist2)\n", - "plt.legend();" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Figure 7.3: Distribution of time between goals.\n", - "\n", - "For the Bruins, the overall chance of winning the next game is 55%.\n", - "\n", - "To win the series, the Bruins can either win the next two games or split\n", - "the next two and win the third. Again, we can compute the total\n", - "probability:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/plain": [ - "0.57142746956373802" - ] - }, - "execution_count": 14, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# win the next two\n", - "p_series = p_win**2\n", - "\n", - "# split the next two, win the third\n", - "p_series += 2 * p_win * (1-p_win) * p_win\n", - "p_series" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The Bruins chance of winning the series is 57%. And in 2011, they did." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##Discussion\n", - "\n", - "As always, the analysis in this chapter is based on modeling decisions,\n", - "and modeling is almost always an iterative process. In general, you want\n", - "to start with something simple that yields an approximate answer,\n", - "identify likely sources of error, and look for opportunities for\n", - "improvement.\n", - "\n", - "In this example, I would consider these options:\n", - "\n", - "- I chose a prior based on the average goals per game for each team.\n", - " But this statistic is averaged across all opponents. Against a\n", - " particular opponent, we might expect more variability. For example,\n", - " if the team with the best offense plays the team with the worst\n", - " defense, the expected goals per game might be several standard\n", - " deviations above the mean.\n", - "\n", - "- For data I used only the first four games of the championship\n", - " series. If the same teams played each other during the regular\n", - " season, I could use the results from those games as well. One\n", - " complication is that the composition of teams changes during the\n", - " season due to trades and injuries. So it might be best to give more\n", - " weight to recent games.\n", - "\n", - "- To take advantage of all available information, we could use results\n", - " from all regular season games to estimate each team’s goal scoring\n", - " rate, possibly adjusted by estimating an additional factor for each\n", - " pairwise match-up. This approach would be more complicated, but it\n", - " is still feasible.\n", - "\n", - "For the first option, we could use the results from the regular season\n", - "to estimate the variability across all pairwise match-ups. Thanks to\n", - "Dirk Hoag at , I was able to get the\n", - "number of goals scored during regulation play (not overtime) for each\n", - "game in the regular season.\n", - "\n", - "Teams in different conferences only play each other one or two times in\n", - "the regular season, so I focused on pairs that played each other 4–6\n", - "times. For each pair, I computed the average goals per game, which is an\n", - "estimate of $\\lambda$, then plotted the distribution of these estimates.\n", - "\n", - "The mean of these estimates is 2.8, again, but the standard deviation is\n", - "0.85, substantially higher than what we got computing one estimate for\n", - "each team.\n", - "\n", - "If we run the analysis again with the higher-variance prior, the\n", - "probability that the Bruins win the series is 80%, substantially higher\n", - "than the result with the low-variance prior, 57%.\n", - "\n", - "So it turns out that the results are sensitive to the prior, which makes\n", - "sense considering how little data we have to work with. Based on the\n", - "difference between the low-variance model and the high-variable model,\n", - "it seems worthwhile to put some effort into getting the prior right.\n", - "\n", - "The code and data for this chapter are available from\n", - " and\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "If buses arrive at a bus stop every 20 minutes, and you arrive at the\n", - "bus stop at a random time, your wait time until the bus arrives is\n", - "uniformly distributed from 0 to 20 minutes.\n", - "\n", - "But in reality, there is variability in the time between buses. Suppose\n", - "you are waiting for a bus, and you know the historical distribution of\n", - "time between buses. Compute your distribution of wait times.\n", - "\n", - "Hint: Suppose that the time between buses is either 5 or 10 minutes with\n", - "equal probability. What is the probability that you arrive during one of\n", - "the 10 minute intervals?\n", - "\n", - "I solve a version of this problem in the next chapter.\n", - "\n", - "Suppose that passengers arriving at the bus stop are well-modeled by a\n", - "Poisson process with parameter $\\lambda$. If you arrive at the stop and\n", - "find 3 people waiting, what is your posterior distribution for the time\n", - "since the last bus arrived.\n", - "\n", - "I solve a version of this problem in the next chapter.\n", - "\n", - "Suppose that you are an ecologist sampling the insect population in a\n", - "new environment. You deploy 100 traps in a test area and come back the\n", - "next day to check on them. You find that 37 traps have been triggered,\n", - "trapping an insect inside. Once a trap triggers, it cannot trap another\n", - "insect until it has been reset.\n", - "\n", - "If you reset the traps and come back in two days, how many traps do you\n", - "expect to find triggered? Compute a posterior predictive distribution\n", - "for the number of traps.\n", - "\n", - "Suppose you are the manager of an apartment building with 100 light\n", - "bulbs in common areas. It is your responsibility to replace light bulbs\n", - "when they break.\n", - "\n", - "On January 1, all 100 bulbs are working. When you inspect them on\n", - "February 1, you find 3 light bulbs out. If you come back on April 1, how\n", - "many light bulbs do you expect to find broken?\n", - "\n", - "In the previous exercise, you could reasonably assume that an event is\n", - "equally likely at any time. For light bulbs, the likelihood of failure\n", - "depends on the age of the bulb. Specifically, old bulbs have an\n", - "increasing failure rate due to evaporation of the filament.\n", - "\n", - "This problem is more open-ended than some; you will have to make\n", - "modeling decisions. You might want to read about the Weibull\n", - "distribution (). Or\n", - "you might want to look around for information about light bulb survival\n", - "curves." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Prediction\n", + "\n", + "## The Boston Bruins problem\n", + "\n", + "In the 2010-11 National Hockey League (NHL) Finals, my beloved Boston\n", + "Bruins played a best-of-seven championship series against the despised\n", + "Vancouver Canucks. Boston lost the first two games 0-1 and 2-3, then won\n", + "the next two games 8-1 and 4-0. At this point in the series, what is the\n", + "probability that Boston will win the next game, and what is their\n", + "probability of winning the championship?\n", + "\n", + "As always, to answer a question like this, we need to make some\n", + "assumptions. First, it is reasonable to believe that goal scoring in\n", + "hockey is at least approximately a Poisson process, which means that it\n", + "is equally likely for a goal to be scored at any time during a game.\n", + "Second, we can assume that against a particular opponent, each team has\n", + "some long-term average goals per game, denoted $\\lambda$.\n", + "\n", + "Given these assumptions, my strategy for answering this question is\n", + "\n", + "1. Use statistics from previous games to choose a prior distribution\n", + " for $\\lambda$.\n", + "\n", + "2. Use the score from the first four games to estimate $\\lambda$ for\n", + " each team.\n", + "\n", + "3. Use the posterior distributions of $\\lambda$ to compute distribution\n", + " of goals for each team, the distribution of the goal differential,\n", + " and the probability that each team wins the next game.\n", + "\n", + "4. Compute the probability that each team wins the series.\n", + "\n", + "To choose a prior distribution, I got some statistics from\n", + ", specifically the average goals per game for each\n", + "team in the 2010-11 season. The distribution is roughly Gaussian with\n", + "mean 2.8 and standard deviation 0.3.\n", + "\n", + "The Gaussian distribution is continuous, but we’ll approximate it with a\n", + "discrete Pmf. `thinkbayes` provides `MakeGaussianPmf` to do exactly\n", + "that:\n", + "\n", + "```python\n", + "def MakeGaussianPmf(mu, sigma, num_sigmas, n=101):\n", + " pmf = Pmf()\n", + " low = mu - num_sigmas*sigma\n", + " high = mu + num_sigmas*sigma\n", + "\n", + " for x in numpy.linspace(low, high, n):\n", + " p = scipy.stats.norm.pdf(mu, sigma, x)\n", + " pmf.Set(x, p)\n", + " pmf.Normalize()\n", + " return pmf\n", + "```\n", + "\n", + "`mu` and `sigma` are the mean and standard\n", + "deviation of the Gaussian distribution. `num_sigmas` is the number of\n", + "standard deviations above and below the mean that the Pmf will span, and\n", + "`n` is the number of values in the Pmf.\n", + "\n", + "Again we use `numpy.linspace` to make an array of\n", + "`n` equally spaced values between `low` and\n", + "`high`, including both.\n", + "\n", + "`norm.pdf` evaluates the Gaussian probability density function (PDF).\n", + "\n", + "Getting back to the hockey problem, here’s the definition for a suite of\n", + "hypotheses about the value of $\\lambda$.\n", + "\n", + "```python\n", + "class Hockey(thinkbayes.Suite):\n", + "\n", + " def __init__(self):\n", + " pmf = thinkbayes.MakeGaussianPmf(mu=2.7, sigma=0.3, num_sigmas=4)\n", + " thinkbayes.Suite.__init__(self, pmf)\n", + "```\n", + "\n", + "So the prior distribution is Gaussian with mean 2.7, standard deviation\n", + "0.3, and it spans 4 sigmas above and below the mean.\n", + "\n", + "As always, we have to decide how to represent each hypothesis; in this\n", + "case I represent the hypothesis that $\\lambda=x$ with the floating-point\n", + "value `x`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Poisson processes\n", + "\n", + "In mathematical statistics, a **process** is a stochastic\n", + "model of a physical system (“stochastic” means that the model has some\n", + "kind of randomness in it). For example, a Bernoulli process is a model\n", + "of a sequence of events, called trials, in which each trial has two\n", + "possible outcomes, like success and failure. So a Bernoulli process is a\n", + "natural model for a series of coin flips, or a series of shots on goal.\n", + "\n", + "A Poisson process is the continuous version of a Bernoulli process,\n", + "where an event can occur at any point in time with equal probability.\n", + "Poisson processes can be used to model customers arriving in a store,\n", + "buses arriving at a bus stop, or goals scored in a hockey game.\n", + "\n", + "In many real systems the probability of an event changes over time.\n", + "Customers are more likely to go to a store at certain times of day,\n", + "buses are supposed to arrive at fixed intervals, and goals are more or\n", + "less likely at different times during a game.\n", + "\n", + "But all models are based on simplifications, and in this case modeling a\n", + "hockey game with a Poisson process is a reasonable choice. Heuer, Müller\n", + "and Rubner (2010) analyze scoring in a German soccer league and come to\n", + "the same conclusion; see\n", + ".\n", + "\n", + "The benefit of using this model is that we can compute the distribution\n", + "of goals per game efficiently, as well as the distribution of time\n", + "between goals. Specifically, if the average number of goals in a game is\n", + "`lam`, the distribution of goals per game is given by the\n", + "Poisson PMF:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def EvalPoissonPmf(k, lam):\n", + " return (lam)**k * math.exp(-lam) / math.factorial(k)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And the distribution of time between goals is given by the exponential\n", + "PDF:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def EvalExponentialPdf(x, lam):\n", + " return lam * math.exp(-lam * x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "I use the variable `lam` because `lambda` is a\n", + "reserved keyword in Python. Both of these functions are in\n", + "`thinkbayes.py`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The posteriors\n", + "\n", + "![Posterior distribution of the number of goals per\n", + "game.](figs/hockey1.pdf)\n", + "\n", + "[fig.hockey1]\n", + "\n", + "Now we can compute the likelihood that a team with a hypothetical value\n", + "of `lam` scores `k` goals in a game:\n", + "\n", + "```python\n", + "# class Hockey\n", + " def Likelihood(self, data, hypo):\n", + " lam = hypo\n", + " k = data\n", + " like = thinkbayes.EvalPoissonPmf(k, lam)\n", + " return like\n", + "```\n", + "\n", + "Each hypothesis is a possible value of $\\lambda$; `data` is\n", + "the observed number of goals, `k`.\n", + "\n", + "With the likelihood function in place, we can make a suite for each team\n", + "and update them with the scores from the first four games." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvYAAAEWCAYAAAANTdM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVPX+P/DXrOyrIDujKCiCIgIqblGJZGWL5RKWC7e6\nlqakX69x07xds35euy51EytNzXIpc8nScrQ0N1RQUFBwYV+Gfd9mO78/Bs84OsCowBmY9/PxuI/H\n/Rw+n8N78DTznnM+n/eHxzAMA0IIIYQQQki3xuc6AEIIIYQQQsijo8SeEEIIIYSQHoASe0IIIYQQ\nQnoASuwJIYQQQgjpASixJ4QQQgghpAegxJ4QQgghhJAegBJ7QgghhBBCegCDE/uNGzfCx8cHFhYW\nCA0NxenTp9vsn5qaioiICFhaWsLLywsrV67U+blMJsOMGTPg7+8PoVCImJgYvefZsGED/P392fPM\nnz8f9fX1hoZNCCGEEEKISTAosd+zZw9iY2OxbNkyJCcnY9SoUZg4cSLy8/P19q+trUVkZCTc3NyQ\nlJSEDRs2YM2aNVi3bh3bp7m5Gc7OzoiLi8PIkSP1nmfnzp1YunQpli9fjvT0dOzYsQOHDx9GbGzs\nQ7xUQgghhBBCei6eITvPjhw5EkOHDsWmTZvYY35+fpgyZQpWrVp1X//4+HjExcWhpKQEYrEYALBq\n1Sps2rQJeXl59/WfNGkSnJ2d8c033+gcf+edd5Camoo///yTPfavf/0L+/btw5UrVwx/lYQQQggh\nhPRw7d6xVygUSEpKQmRkpM7xCRMm4OzZs3rHJCQkYOzYsWxSDwBRUVEoLCxETk6OwcGNGTMGycnJ\nOH/+PAAgNzcXBw8exDPPPGPwOQghhBBCCDEFwvY6lJWVQaVSwcXFRee4i4sLjh8/rneMTCaDl5fX\nff0ZhoFMJoNEIjEouGnTpqG8vBzjxo0DwzBQKpWYOXMmPvnkE4PGE0IIIYQQYiraTey5dPLkSaxc\nuRKbNm3C8OHDcevWLSxYsAArVqzAhx9+qNO3urqaoygJIYQQQgh5dHZ2do80vt3E3snJCQKBAMXF\nxTrHi4uL4erqqneMq6ur3v48Hq/VMfosX74c0dHRmDNnDgAgICAAdXV1eOONN7BixQrw+VStkxBC\nCCGEEMCAOfYikQghISGQSqU6x6VSKUaPHq13THh4OE6dOgW5XM4eO3r0KNzd3Q2ehgMADQ0NEAgE\nugHz+TBgvS8hhBBCCCEmxaCpOIsWLcLMmTMRFhaG0aNHIz4+HkVFRZg7dy4AIC4uDhcvXsSxY8cA\nANHR0fj3v/+N2bNn4/3330dGRgZWr1593/SZlJQUMAyDmpoaCAQCpKSkQCwWw9/fH4CmWs66desQ\nEhKCESNG4ObNm/jggw8wadKkNu/WP+pjDNJzJSYmAgBCQ0M5joQYM7pOiCHoOiHtoWuEGKIjp5Mb\nlNhPnToVFRUVWLVqFYqKihAYGIgjR47A09MTgGaxbFZWFtvf1tYWUqkU8+bNQ1hYGBwcHLBkyZL7\n6s8HBweDx+Ox7UOHDkEikSAzMxOAZioOn8/H8uXLUVBQAGdnZ0yaNAkfffTRI79wQgghhBBCehKD\n6th3B3d/26E79qQ1dPeEGIKuE2IIuk5Ie+gaIYboyByWVp8SQgghhBDSA1BiTwghhBBCSA9AiT0h\nhBBCCCE9ACX2hBBCCCGE9ABGvfMsIYQQwjAMqqqbUVJWj+rqJtyp+MDn8WBvZ4bezlawtTHTqbJG\nCCGmiBJ7QgghRqWuTo5rGaVITS9F6vUSFBTWQqFUtTnGzEwIT3dbDPZ3RqB/b/gPcIKFuaiLIiaE\nEONAiT0hhBDOyRUqXLxUiD9OZSH5avED7zDe3KzE7awK3M6qwIHDGRAI+Agd6oYnxvbFsCBXCAQ0\n85QQ0vNRYk8IIYQzVdVN+PlIBo6dzEJdvbzVftZWYjj3soSjgwV4fM2UG7WKQXllI4pL69DUpNTp\nr1KpcT6pAOeTCmBna46oJ3zw7AQ/WFuLO/X1EEIIlyixJ4QQ0uXuJPSHj92GXK687+f9+jqy02r8\n+veCtVXrCTnDMKitkyPjZjmuXi/B1WslyMmrYn9eXdOEHw5cwy+/38SzUX54Nsq3zfMRQkh3RYk9\nIYSQLqNSqXFYegu79qXed5e9t5MVIsb0weNj+8DF2crgc/J4PNjamCFsmDvChrkDAPILa/DHqWyc\nPJ2DyupGAEBDowI/HEjDYelNvDp1CMY/1hd8Pi24JYT0HJTYE0II6RI3b5dj09YkZOVW6Rzv422P\naS8GICzYvcMSbU93W8ycNgQzXg7EmfN5+OHANRTKagEAdfVybNqaiD9PZWHunBBIvOw75HcSQgjX\nKLEnhBDSqZRKNXbvS8O+X9IBaBfFerrbYsaUwR2a0N9LIOBj3CgJxoz0xumEXOzcm4qSsnoAQMat\ncixeLsX0yQGY/Kw/3b0nhHR7lNgTQgjpNCVl9Vi3MQEZt8rZYyKhAFNfGITnnvaDSCjokjj4fB7G\njZJgZKgnfjx4DQcOZ0ClUkOtZrBzbyquXivBwrkj4Ghv0SXxEEJIZ6DEnhBCOKBSM8ipbkReTRMq\nmxSoaFSgulkz51zA40HA58FGLICrlRlcrMXwsDGHk2X3WvB58VIhPvvqAuobtNVuhgxywdyYELj2\ntuYkJrFYgBlTBmPcKAniv0lE+s0yAMDVayVY9P5RxM4dgaGDXTmJjRBCHhUl9oQQ0gXUDIP0snok\nFFThWlk9blc2QK5SP9A5nC3FCHS2RoCzNcLc7Yw20WcYBgeP3MC3u6/gztQbHo+HGVMG44WnBxjF\nlBcvD1us/GcEfjhwDT8evA6AQU1tM1Z+egp/e3Uono705TpEQgh5YJTYE0JIJ7pZUY/fb5cjoaAK\nlU2KRzpXaYMcf+ZU4M+cCvDAQ2Bva0RIHDHa0x42Zsbxdq5UqvHV9ks4djKTPebUyxKL3h6Jgb5O\nHEZ2P4GAj1deCkSgf2+sjz+PyupGMAyDzTsuo6CoFjEzhtLGVoSQbsXgd6yNGzfCx8cHFhYWCA0N\nxenTp9vsn5qaioiICFhaWsLLywsrV67U+blMJsOMGTPg7+8PoVCImJgYveepra3FggUL4OHhAXNz\nc/j5+WHv3r2Ghk0IIV1OqWbwV04F/u9YBmKPpuPI7VK9Sb2zpRgjPezxrK8zZg52x8IwCRYOl+Cd\nMAnmDvPC9EFueFziCH8na1jcMxedAYOrJbX4/GIOZv18FV9eyoOsrrmrXqJejU0KfPTpKZ2k3t/P\nGZ/+O9Lokvq7DR7UG5+ujER/H0f22JFjt/DxutP3leQkhBBjZtAtnj179iA2NhabNm3C6NGj8cUX\nX2DixIm4fv06PD097+tfW1uLyMhIREREICkpCdevX8fs2bNhbW2Nd999FwDQ3NwMZ2dnxMXF4auv\nvtL7e5VKJcaPHw8nJyfs3bsXHh4eyM/Ph5mZ2SO8ZEII6RxqhsFfOZX49mohiuvvT7LtzIQI97TH\ncHc7DOhlBXtzkcHnVqkZ3K5sQFppHRKLanClpBZqRjPNpVmlxs83SvDLzVKM83bAq4HucLPp2vfJ\nujo5Vn56CjcztYtkHxslwVt/C4VY1DULZB+Fg705Pvrn49jw5QWcu5gHALh8RYZ/f/oX3l80BlZG\nOu2JEELuxmMYhmmv08iRIzF06FBs2rSJPebn54cpU6Zg1apV9/WPj49HXFwcSkpKIBZr3gxXrVqF\nTZs2IS8v777+kyZNgrOzM7755hud41999RX+85//ID09HUJh299Bqqur2f9vZ2fX3ksiJioxMREA\nEBoaynEkxJg9zHVyWVaDb5ILkFnVoHNcyONhnMQBE3ycEOBsDT6vY+aXVzQqcCq3EtKsMmRVNd73\nOyf59cb0AFdYizt/ik51TRM+/M9fyL6rPv30yYGY8rw/eB30eruKWs1g10+p+OnQdfZYv76O+GDJ\nWNhY635ZovcT0h66RoghOjKHbXcqjkKhQFJSEiIjI3WOT5gwAWfPntU7JiEhAWPHjmWTegCIiopC\nYWEhcnJyDA7u4MGDGD16NObPnw83NzcEBATgww8/hFJJj0YJIcahqkmB1WczsezETZ2k3kYsRHSg\nG7Y9NxiLR/bF4N42HZbUA4CjhQjPD+iNz6P88e/H+iPIxYb9mZJhsD+jGG/8moajmWUw4P7NQ6us\nasIHn5zQSerfmDkMU18Y1O2SekBTFnPGlMGYEz2UPXY7qwLLVp1AVXUTh5ERQkj72r2VU1ZWBpVK\nBRcXF53jLi4uOH78uN4xMpkMXl5e9/VnGAYymQwSicSg4DIzM/HHH39gxowZOHz4MLKzs/H222+j\nvr4e//nPf1odd+cbMiGtoWuEGKKt64RhGCRVqvBzoQKNKu1xIR8Y5yRERG81LJoKcTutsAsiBSbb\nACF8NQ4VKZBTr6m201APfPJnDfYn8fGShxj24o5NtBsaldj8fRaKSzXTjng84MWJHnC2r+r2/425\nOQHjx9rj598LwTBA+o0GLFi6H6/P6AtLC92Pzu7+Wknno2uEtMXXt+OqcBlHGYVWqNVquLi44Ouv\nvwaPx0NwcDDKysqwaNGiNhN7QgjpTA1KBj/ky5FWrVuucpiDAE+7iWAn4uZOtcSKj3n9xEipVuFw\nkRKVcs2d+vQaNf5b34QXPEQIceiYt/1muQrb9+ToJPVTnvNE0CD7Djm/MRge7AixiI+9v+SDYYDi\n0mZ8+0MO5rzSB2Zi4183QAgxPe2+wzs5OUEgEKC4uFjneHFxMVxd9W/i4erqqrc/j8drdYw+bm5u\nEIvFOo9z/f390dDQgPLycvTq1UvvOJrLRlpD8x2JIdq6TjLK6/HFmUyUKPmwtNIcc7Eyw/xQbwxz\ns+3KMFsVBuBVpRrfXinAzzdKwbTUkv+5HGiyc8LcYV4wEz58GUe5QoWPPj2FyhoerKwsAQAL/j4c\nEaP7dED0xiU0FOjvm43PvrwAAKioBn470YDl/zcOV65cbulD7ydEP/rMIYa4e479o2r3nV0kEiEk\nJARSqVTnuFQqxejRo/WOCQ8Px6lTpyCXa3cbPHr0KNzd3Q2ehgMAo0ePxq1bt3SOZWRkwNLSstWk\nnhBCOgPDMDiQUYwlxzJQctdOqs/6OuOLp/yNJqm/w1zIx5vDvPDJE75wtdIu+jyaWYYlxzMeujSm\nWs1g7RcJSL1ewh57Y+awHpnU3xExug/emDmMbaell+LTL85Bre68tQuEEPIwDLpls2jRImzbtg1b\ntmxBeno6Fi5ciKKiIsydOxcAEBcXh/Hjx7P9o6OjYWlpidmzZyMtLQ379u3D6tWrsXjxYp3zpqSk\nIDk5GTU1NaioqEBKSgquX9dWInjrrbdQUVGBBQsW4MaNG/j999/xr3/9C/PmzeuI104IIQZRqNT4\n7GIuvr6cD1XLQlRrkQDvj/bBWyHesDDico6De9vgi4n+eKKPtkb77coGLPj9Oi7Lah74fNt2peDC\npQK2/eqUwZg4vn+HxGrMJo7vj9emDWHbiZcL8euxok5dmEwIIQ/KoMmWU6dORUVFBVatWoWioiIE\nBgbiyJEjbA17mUyGrKwstr+trS2kUinmzZuHsLAwODg4YMmSJYiNjdU5b3BwsM40m0OHDkEikSAz\nU7O5iaenJ44ePYpFixYhODgYrq6ueP311/H+++8/8gsnhBBD1DYrsepMJq6W1LLH/BytsHRUX7ha\nd489NcyFAiwa0Qf+Ttb46lIeFGoG9QoVVpy8hflh3pjgY9jmUYelN/HL7zfY9qSn/DB5kn9nhW10\nXnxmIGrr5DjwazoAICGpAo72YoSFcRwYIYS0MKiOfXdAdeyJIWi+IzHEnevEc+BgfHDyFgpqtWUO\nn+jjiHfCJBALHn6OOpdulNdj1elMlDVqpxNNG+SK1wa7t1meMjG5EJ+sO8PeoR4Z6on/mx8OPr/7\nlbR8FGo1g3XxCThzPg/19Q3g8YAVS8cjPOz+zRoJoc8cYogurWNPCCGmSNakxj+OZ+gk9TMHu2PR\niD7dNqkHAL9eVvhv5AD42Fuyx/Zck2Hd+RyoWpkznptfjbUbE9ikvr+PIxb+fbjJJfWAps79O28O\nx0BfzVMOhgHWx5/H7exKjiMjhBBK7Akh5D55DWrE325GeaMCACDm8xE32gfTAty65aZL93KyFOM/\nT/ohxE17Z+h4djn+cy4LCpVuCc+6Ojn+34YzaGrSbAzY28kK/3x3DMzMjLpacqcSiwR4L3Y0ejlo\nNmFUKFX4f+vP0AZWhBDOUWJPCCF3SSutw5eZzWho2eDaQijAvx7rjzFeDtwG1sEsRAKsGNsPUXfN\nrz+dV4mPz2RC3pLcq9UM1m1KgKy4DgBgZibEPxeNgb2dOScxGxNbGzPMnCqBmZnmY7S8ogGf/u8c\nlEp1OyMJIaTzUGJPCCEtrpfVYcXJW2hu2UnWRizEqsd9EeRiw21gnUTA5+GdMG8859ebPXahsBr/\n/us25Co1du9Lw+UrMvZn77wRBm9PWsN0h5OjGaY+5wVA8xTnWkYptu1K5jYoQohJo8SeEEKgWVT6\nwYlbaFRqsnprIfD/nvDDgF5WHEfWuXg8Ht4M9sRUf+3mgZeLaxC7/yp+OHSNPfbiswMxargXFyEa\ntYH9bRD9cgDbPiy9hT9PZ3MXECHEpFFiTwgxeZmVDfjg5C00tCT1VkLg7/3M0MfeguPIugaPx8Os\nIA+8GugOAFAq1ThxuwyF7tZgAAQFumLGy4O5DdKIvTTJX6cqzpfbLiE3v+N2kiSEEENRYk8IMWmF\ntU1YduImauWaSfU2YiHe9DGDq7npvT2+EuiGlwe6oLikHmo1gxpbMar62SP2LdOsgGMoHo+H+W+E\nwaNl92G5XIn/fnGOXXBMCCFdxfQ+uQghpEVVkwLLT9xCdbMmAbMWCfBRhC/cLUz3rZF3rQyWRXVs\nmy+xw4+3yjiMqHuwMBdhyTvhELfsQpxXUIOvvr3EcVSEEFNjup9ehBCT1tiy86qsvhmApqTlvx7r\nj/6Olu2M7LkuXi7Ez7/dgEtxA+yrmtHL0QLm5kLszyjGwYwSrsMzet6ednhj5jC2feJ0No6fzGpj\nBCGEdCxK7AkhJkepZrDqdCZuVTYAAPg8Ht4b3Rf+TtYcR8adyqom/O/riwA0NV6ecbbBxEHaBbVf\nX87HqVzahKk9Tz7WFxFj+rDtr3dcRqGslruACCEmhRJ7QohJYRgGm5Jycbm4hj02P9QbIzzsOYyK\nWwzD4H+bL6C2TvP0wtHBAgv/PgJLwrVfdhgw+G9CFq6WUJLanjdnDoOnu3a+/dqNCVC0LMwmhJDO\nRIk9IcSkHLxRgiO3tXPGowPdENXPqY0RPd+RY7d069W/ORy2NmYwE/Lxwdh+8LTRbEilaHnSIWv5\nAkD0MzcX4t23RkIo1HzEZmZXYs++a+2MIoSQR0eJPSHEZFworMbmywVs+4k+jogOcOMwIu7lFdRg\n+64rbHvSU34ICnBh27ZmQnz4WH/Ym4kAALVyJT786zYaFHQHui19JfZ4dYq2ROi+X9KRep3WKRBC\nOhcl9oQQk5Bd1Yj/nM0CAwYA4O9kjXfCJODxTLeMo0Kpwvp47TQRb087zJhyf716V2szLBvrA1FL\nycvcmkasOZcFNcN0abzdzbNRfhjCfklisH7TedTVyzmNiRDSs1FiTwjp8erkSnx0+ja7q6yzpRjL\nxvhALDDtt8Cffk5HVm4VAEAkFODdt0aw5Rrv5e9kjQXDJWz7QmE1tqUU6O1LNPh8Hha8ORzWVmIA\nQEVlI775PpnjqAghPZnBn2obN26Ej48PLCwsEBoaitOnT7fZPzU1FREREbC0tISXlxdWrlyp83OZ\nTIYZM2bA398fQqEQMTExbZ5v165d4PP5eO655wwNmRBCoGYYfJqQjaKWeeHmQj5WjOsHe3MRx5Fx\nKzO7Ent/1s77jn45EBKvthcQP9GnF17211bK+Sm9GH/lVnRajD2Bo4MF3ooJZdsnTmfj4qVCDiMi\nhPRkBiX2e/bsQWxsLJYtW4bk5GSMGjUKEydORH5+vt7+tbW1iIyMhJubG5KSkrBhwwasWbMG69at\nY/s0NzfD2dkZcXFxGDlyZJu/PzMzE//4xz8wbty4B3hphBAC7EorwsXCarYdO7wP+tqbbq16QDMF\n5/OvL0Kt1kylGejrhElP+Rk0dtYQdwx3t2PbGy7kIKe6sVPi7CnCwzwxNtybbW/8JhE1tbQAmRDS\n8QxK7NetW4eYmBjExMRgwIAB+Oyzz+Dm5ob4+Hi9/b/77js0NjZi+/bt8Pf3x+TJk7F06VKsXbuW\n7SORSLB+/XrMnDkTDg4Orf5upVKJ6OhofPzxx+jbt+8DvjxCiCk7X1CFnalFbPulgS4Y6936+42p\n2HvwOnLyNFNwxCIB5r8RBj7fsLUGfB4Pi0f2gbu1GQCgSanGqtOZqJfTYtq2vPHaMDjYWQAAqmua\n8NV22pWWENLx2k3sFQoFkpKSEBkZqXN8woQJOHv2rN4xCQkJGDt2LMRiMXssKioKhYWFyMnJeaAA\n//nPf8LHxwevvfbaA40jhJg2WV0z1iZks+2hLraYNcSDu4CMxO3sSuz9+TrbfnXqYLi72jzQOazF\nQrw/ph/MWtYoFNQ2Yd35bFpM2wZrazHefl07JefshTycOZ/HYUSEkJ6o3cS+rKwMKpUKLi4uOsdd\nXFwgk8n0jpHJZHr7MwzT6hh9jh49ir179+LLL780eAwhhChUaqw+m4U6hXax7D9G9YXAwLvSPZVS\nqcYXmy+CaUnA/f2c8XSk70Odq4+9BRbetZj2XEEVDmRQOce2hAS54clx2ifPm3dcRl0dVckhhHQc\nIdcBtKasrAxz5szB7t27YWPzYHeTEhMTOykq0lPQNdKz/VyoQHKpEgDA5wGT3FS4efXBq5H0tOvk\nr3OlSLteDAAQCnkYO1yMS5eSHvp8VgCGWShwukzzt/7i7A0wxTmQWJlWtaEHuU6CBqpx4pQcNXVK\n1Nc34OO1RzD5aXqS1NP1tPcS0rF8fR/uBos+7b77Ojk5QSAQoLi4WOd4cXExXF1d9Y5xdXXV25/H\n47U65l5paWmQyWR48sknIRKJIBKJ8O233+LXX3+FWCzGzZs3DToPIcS0pFWrcKolqQeAp91EJpdo\n6lNW0Yzjp7V31J8c2xtOjmaPfN5n3ITwstT8fdUM8H2uHA1KmpLTGnNzASZFubPtpJRKZObUcRgR\nIaQnafeOvUgkQkhICKRSKV566SX2uFQqxZQpU/SOCQ8Px3vvvQe5XM7Osz969Cjc3d0hkUj0jrnX\n8OHDcfXqVZ1j77//PqqqqrBx48Y2F9KGhoa2+jNi2u7cNaFrpGcqbZBjzZFrsLTSvLWNcLfDorH9\nHngTqp52najVDD745ATMzCxgZgb09bbHwrfGQ9BBdfz7DGrGgt+vo16hQjOAvxT2eH+ET4/f/Oth\nr5PQUEBWdhYJiZrKcicSGvDCpLEQi/XvIUC6r572XkI6R3V1dfudDGTQu/qiRYuwbds2bNmyBenp\n6Vi4cCGKioowd+5cAEBcXBzGjx/P9o+OjoalpSVmz56NtLQ07Nu3D6tXr8bixYt1zpuSkoLk5GTU\n1NSgoqICKSkpuH5ds6jLwsICgwYN0vmfvb09bGxs2Nr3hBByh5ph8N+EbHZefW9LMd4d0afHJ5eG\nOP5XFq5llAIAeDwe3n49rMOSekCzM23sPfPtD90s7bDz90SvvxYMSwvNXgqy4jrsOZDGcUSEkJ7A\noOx46tSpqKiowKpVq1BUVITAwEAcOXIEnp6eADSLZbOystj+tra2kEqlmDdvHsLCwuDg4IAlS5Yg\nNjZW57zBwcE6H7qHDh2CRCJBZmZmR7w2QogJ2Z9egqsltQA0JRn/MaovbMzoBkBVdRN27LnCtp+f\n6Id+fTq+5OcoLwdM8u2NQzc10322JhcgyMUGkpYSj0SXo4MFZk4Pwqatmju6B37NwJgR3ugraXuT\nMEIIaQuPYXpGfbK7H2PY2dm10ZOYMnos2jNlVjbg3aPpULa8nU0PcMNrg93bGdW6nnSdfP71Bfx5\nKhsA0NvJChs+iYJZJ33hkavUWCzNQGZVAwDAx94SayMHQNSBTweMyaNeJ3emSN15mtKvryNWr3jS\n4D0FiPHrSe8lpPN0ZA7bM99tCSEmo1mpxppz2WxS7+tohVcC3LgNykhcyyhlk3oAeGPWsE5L6gFA\nLODj/8L7QMzXfLRkVjXgu7s2CCO6+Hwe3ooJgUiomVt/O6sCvx6lwhCEkIdHiT0hpFvbfqUAuTWN\nAAAzAR9LwvtASHc8oVSq8eU27e6mI0I8EBLU+V94JHYWmBWkfVry0/VidooUuZ+Hmy2mvODPtr/f\nm4ri0noOIyKEdGeU2BNCuq1LRTU4eENbwvGNYE942JhzGJHx+PXoTeQVaB7visVCxMwY2mW/+zm/\n3hjqYgsAYKBZ1FwvV3XZ7+9uXnh6ILw9NY/f5XIlvtyWhB4yS5YQ0sUosSeEdEs1zUqsO5/Ntoe7\n2+Gpfk7cBWREyisasHu/tsrKtBcHwdnJqst+P5/Hw7sjJLARa6b9lDbIselSbpf9/u5GKORj3t9C\nAWieNCVfleHcxXxugyKEdEuU2BNCuh2GYfD5xRxUNCkAAPZmIiwcLqHSli2++T4Zzc2aTbq8PGzx\nbFTH7WpoKCdLMd4O9WLbf2RX4K/cii6Po7vw7dcLT0f2Y9vffJ+MxpbrmxBCDEWJPSGk2zmWVY6z\n+VVse8Fwb9ibiziMyHhcSinSudv75izt4syuNs7bEU/0cWTbGxPzUNYg5ySW7mD65EDY2WqmklVU\nNuLHg9c5jogQ0t1QYk8I6VbKGuT46pI2cZ3YzxkjPKj2NwDIFSp8veMy235stAQBA505jAiYO8wb\nvS01O5DXypVYdz4Hapo/rpe1lRizXhnCtn8+koG8ghoOIyKEdDeU2BNCug3NFJxcNCg1CzHdrc3w\nerAHx1EZj32H0lFcUgcAsLIUY9b0II4jAqzEAiwa2Qe8O/PHi2tw5FYZx1EZr8dGSTDQT7NWRK1m\nsHnHJVq+OqybAAAgAElEQVRISwgxGCX2hJBu48+cCiQWaTfyWDhcAnOOppkYG1lxHfb/ks62Z0wJ\nhL2dcVQIGtzbBpMH9mbbW1MKUFzfzGFExovH4+HNmcPY9SJXr5XgzPk8jqMihHQXlNgTQrqFykaF\nzhScZ32dEdjbhsOIjMvWnclQtDzJ6NfXERMe79fOiK716mB3eLaUIm1UqvD5hVy6E92KPt72eGaC\ndsHz1p0ptJCWEGIQSuwJIUaPYRhsTMpFrVxT6aW3pRizh9AUnDtSUmW4eLmQbb85Mxh8I9ukSyzg\nI3aEhJ2Sc7m4BtKsco6jMl7TXhzEPnGprGrED/uvcRwRIaQ7oMSeEGL0TudV6VTBWThcAgsRTcEB\nNDvMfvN9MtuOGNMHvv16cRhR6/ydrPG8n3Yx7+bL+VQlpxX3rpE49PsN5OZXtzGCEEIosSeEGLnq\nZiXik7SbGz3VzwlDXW05jMi4/P7HbbZyirm5EK9OHcxxRG17bYgH3KzNAAD1ChW+SKQpOa0ZN8ob\ngwZovgip1Qy+/pYW0hJC2kaJPSHEqH11KQ/VLZstOVmIERPkyXFExqOmthm792l3mH35uUFwtLfg\nMKL2mQv5WDhcwrYvFFbjRA5tXKUPj8fDm7O0C2nT0ktxOoEW0hJCWkeJPSHEaJ0vqNJJ+uaFecNK\nTFNw7tj1UyrqW6ayuPS25mSH2YcxuLcNnumvnZLz5aV8VNHiUL28Pe10/l23705BU5OSw4gIIcaM\nEntCiFGqkyvxv4vaKThP9HHEcHc7DiMyLtm5VTj6ZybbjokeCnE3WncwJ8hDZ+OqjUl0J7o1014M\nYBfSVlQ2Yv+v6e2MIISYKoMT+40bN8LHxwcWFhYIDQ3F6dOn2+yfmpqKiIgIWFpawsvLCytXrtT5\nuUwmw4wZM+Dv7w+hUIiYmJj7zrF582aMGzcOjo6OcHBwwBNPPIEzZ84YGjIhpBvbllKAipa7uPZm\nIrwZ7MVxRMaDYRhs+e4yO986KNAVocFuHEf1YCxEAiy4a0rOmbxKnM6r5DAi42VpIcKrU7RrJw78\nmoHi0noOIyKEGCuDEvs9e/YgNjYWy5YtQ3JyMkaNGoWJEyciPz9fb//a2lpERkbCzc0NSUlJ2LBh\nA9asWYN169axfZqbm+Hs7Iy4uDiMHDlS73lOnjyJ6dOn488//8SFCxcwYMAAREVF4fbt2w/xUgkh\n3UVaaR2O3NbuTvp2qBdszIQcRmRcEhILkJZeCkAzD3tOdBA7D7s7CXa1xQQfJ7Ydn5iH2maaZqJP\nxJg+6NfXEQCgUKrw7e4UjiMihBgjgxL7devWISYmBjExMRgwYAA+++wzuLm5IT4+Xm//7777Do2N\njdi+fTv8/f0xefJkLF26FGvXrmX7SCQSrF+/HjNnzoSDg4Pe8+zYsQNvv/02goKC4Ovri/j4eNjY\n2OC33357iJdKCOkOFCq1zhSckR72GO2l/z3CFMnlKmzbqU3qnh7fH96e3XeK0utDPdHLQgQAqGpW\nYNuVAo4jMk58Pg9/e3Uo2z53MR+p10s4jIgQYozaTewVCgWSkpIQGRmpc3zChAk4e/as3jEJCQkY\nO3YsxGIxeywqKgqFhYXIycl56GCbm5vR1NTU6hcBQkj3ty+9GLk1jQAAC6EAc0NoCs7dDh7JQGm5\nZhqGtZUY014M4DiiR2MlFmDuMO2/8W+3y5BaUsthRMZroK8TxoVrpy9t+S4ZajWVvySEaLX7bLus\nrAwqlQouLi46x11cXHD8+HG9Y2QyGby8vO7rzzAMZDIZJBKJ3nHtWbZsGWxsbPDcc8+12S8xMfGh\nzk9MB10jxqm0WY3NN5qhVGva491FyLl2BQ9/O+DRGNt1UlOrwPZdN6BQaJK58WPskZ5+heOoHp0Y\nQB9BM67VaP7hV0pTEOtrBpGR7Z7bmq68TgYPUOPYyUYoFAyupTdg0xYphgc7dtnvJw/H2N5LiHHx\n9e24imbdpirOhg0b8PXXX2P//v2wtrbmOhxCSAdjGAb78hVsUu9pycNop+5T5aUrHDtVwib1Ls5m\nCB3ac55evughhlnLP3dJE4MTpTTXXh87WxEeC9eWCpWeLEZDI/2tCCEa7d6xd3JygkAgQHFxsc7x\n4uJiuLq66h3j6uqqtz+Px2t1TFvWr1+PFStW4LfffkNISEi7/UNDQx/4dxDTcOeuCV0jxueP7HIU\n3s6GpRXA5/GwInIg+jtachKLMV4n2blVSL+VBSsrzd9k8fxxCAp88PdTY9bgVIIvL2nKXp6v5+E1\nv0HwtDXnOKrWcXWdDBmiwq3s39gpWbdyzREzY2g7owgXjPG9hBif6urqDjtXu3fsRSIRQkJCIJVK\ndY5LpVKMHj1a75jw8HCcOnUKcrmcPXb06FG4u7s/8DSctWvXYsWKFTh8+DDCw8MfaCwhpHuoblbi\n68vaKlvP+/XmLKk3Vtt3XwGguVsfPMS1xyX1APCsrzN8Ha0AAAo1g/8l5rIlPYmWWCzArFeGsO3D\n0pvIK6jhMCJCiLEwaCrOokWLsG3bNmzZsgXp6elYuHAhioqKMHfuXABAXFwcxo8fz/aPjo6GpaUl\nZs+ejbS0NOzbtw+rV6/G4sWLdc6bkpKC5ORk1NTUoKKiAikpKbh+/Tr78zVr1iAuLg5btmxB//79\nUVxcjOLiYtTU0BsYIT3J1uR81LSUOXS2FGNGYPeqyd7ZLqUUISVVBkBT3nLW9CCOI+ocfB4PC8K8\nwW8p3Xm1pBbSrHKOozJO4WGeGDRAMyVHrWawdWcyfQkihLQ/FQcApk6dioqKCqxatQpFRUUIDAzE\nkSNH4OnpCUCzWDYrK4vtb2trC6lUinnz5iEsLAwODg5YsmQJYmNjdc4bHBysU3v50KFDkEgkyMzU\n7Ka4ceNGKJVKTJs2TWfcrFmz8M033zzcKyaEGJUr9yRvb4V4waIb7aDa2VQqNbbfVbN8fETfbl3e\nsj0+DpZ4cUBv/JSumc75TXIBhrvbwd5cxHFkxoXH4+Fvrwbj/z6QgmEYJF+VISmlCKFD3bkOjRDC\nIYN3fJk7dy57h/5eW7duve9YQEAATpw40eY51Wp1mz+/+8sCIaTnkavU+OKumvWjvRwwwsOew4iM\nz/G/stlpFubmQkx/MZDjiDpfdKAbTudVobi+GbVyJbYk52PxyL5ch2V0+krsMT6iL6R/am6GbduZ\ngqGBrhAKu01dDEJIB6P/+gkhnNmfXoz82iYAgKVQgL8He3IckXFpbFJg10+pbPvFZwbCwd54F5N2\nFHOhAG+Haksm/5FdgcsymoKpT/RLgbBoeZpRKKvFb3/QzuyEmDJK7AkhnJDVNWP3NRnbfm2IO3pZ\nitsYYXoO/JqB6hrNFx9HBws895QfxxF1nVA3O4zz1pbz3JiYC7mq7ae8psjO1hxTXhjEtn/Yn4a6\nOnkbIwghPRkl9oQQTnx9OZ9N1HzsLfFMf+d2RpiW8ooGHDycwbZnvDwYZmYGz57sEd4M9oJVy3qL\nwrpm7L1e3M4I0/R0ZH+49Nbs71JXL8ee/WkcR0QI4Qol9oSQLnehsBoJBVVs++1QLwi6yS6jXWXn\nT6mQK1QAgD7e9nhs9MPt2N2dOViI8Npg7WLQH6/JUFTbzGFExkksEmDW9LvKXx67hfxCmrpEiCmi\nxJ4Q0qWalWp8mZTHtiP79oK/E+0mfbesnCr8eSqHbc9+JQh8E/3i84yvM/o7aPY0kKvViL+UR2Ud\n9RgR4oGAgZqnXgzDYPuulHZGEEJ6IkrsCSFdam+6DLJ6zV1XG7EQs4M8OI7IuDAMg227knFnM6qQ\nIDcMCXDhNigO8Xk8zAv1Bg+aLzZJRdU4m1/VzijTw+PxEDMjGLjzd0opwuUrsrYHEUJ6HErsCSFd\npqi2GXuvaedJzxriTvXJ75GUUoSr10oAaJK1mT10M6oH4dfLChP7O7Htry7lo7FlmhLR6iuxx5Pj\n+rDtbbuSoaIFx4SYFErsCSFdgmEYbLqUB3nL/hW+jlaI6ufUzijTolKp8e3uK2w7MsIHXh62HEZk\nPGYOdoe9meZLYFmjHN+nFnEckXGKfnkwzM01i6zzCmogPZHJcUSEkK5EiT0hpEskFFQjsagaAMAD\nD2+HeIHPM8154605djKLXfRobi7E9MkBHEdkPGzMhPhbsHba1sEbJciuauQwIuPkYG+Ol571Z9s7\n96airp7KXxJiKiixJ4R0uialGl9e0i6YfaqfE/x6WXEYkfFpaFRg9z5tmcLJzw6EvV3P34zqQTwu\ncUSgsw0AQM0w+CIxF2paSHufSRP94Nzy31ddvRw/HrzGcUSEkK5CiT0hpNP9cK0IpQ2au4a2ZkLM\nGuLezgjTs/+X9Hs2oxrAcUTGh8fjaUqjtjzpuVZWh2NZ5RxHZXzEIgFm3l3+UnoLhbJaDiMihHQV\nSuwJIZ0qv6YJP921sdCcIA/YmNhGS+0pK2/Az0dusO1Xpw6GWCzgMCLjJbGzwOSB2ipB3yQXoKZZ\nyWFExmnUcE8M9NOsYVGp1FT+khATQYk9IaTTMAyDTUl5ULZMl/B3ssb4vr04jsr4fP/jVSiUmiov\nPhIHjAs3vc2oHsT0AFc4W4oBALVyJbZfKeA4IuPD4/EQEz2UbV+8XIgrabRzLyE9HSX2hJBOczqv\nCpeLNYtB+TxaMKvP7exKnDx712ZU0aa7GZWhzIUCzB3mxbZ/u12G9LI6DiMyTv19HHV2LN66MwVq\nNa1JIKQno8SeENIpGhUqbL6cz7af6e8Mn5YdRIkGwzDYtjOZbYcGuyPQvzeHEXUfIz3tMcLdjm1/\nkZgHFSWt93l1ymCIxZqpbzl5VTh2MovjiAghncngxH7jxo3w8fGBhYUFQkNDcfr06Tb7p6amIiIi\nApaWlvDy8sLKlSt1fi6TyTBjxgz4+/tDKBQiJiZG73l++uknBAQEwNzcHIGBgThw4IChIRNCOLQr\nrQhljZoFs/ZmIrw62I3jiIxP4uUipKWXAmjZjGrakHZGkLu9OcwLYoHmYyyzqgG/3CzlOCLj08vR\nEpOfHci2d/2UioZGBYcREUI6k0GJ/Z49exAbG4tly5YhOTkZo0aNwsSJE5Gfn6+3f21tLSIjI+Hm\n5oakpCRs2LABa9aswbp169g+zc3NcHZ2RlxcHEaOHKn3POfOncP06dPx2muvISUlBdHR0ZgyZQou\nXrz4EC+VENJVcqobcSCjhG3/LdgD1mJaMHs3pVKN7bu1CxqferIfPN1pM6oH4WpthlcCtF8Yd1wt\nRHkD1Wy/1/MT/eDoYAEAqK5pwt6fr3McESGksxiU2K9btw4xMTGIiYnBgAED8Nlnn8HNzQ3x8fF6\n+3/33XdobGzE9u3b4e/vj8mTJ2Pp0qVYu3Yt20cikWD9+vWYOXMmHBwc9J5nw4YNeOKJJ/Dee+9h\nwIAB+Oc//4mIiAisX7/+IV4qIaQrMAyDjYl5ULUsmA1wtsbjEkeOozI+0hOZbAlCC3MRpr4wiOOI\nuqcXB/SGp42m3n+jUoXNyfpvOJkyMzOhztOgX36/AVkJrUkgpCdqN7FXKBRISkpCZGSkzvEJEybg\n7NmzesckJCRg7NixEIvF7LGoqCgUFhYiJydH7xh9zp07hwkTJugci4qKavX3EkK4dzKnEqmlmoRV\ns2DWGzxaMKujvkGusxnVS8/5w86WNqN6GCIBH2+HerPtv3IrkSyr4TAi4zQ23Bt+/TQVqZRKNb7d\nfYXjiAghnaHdxL6srAwqlQouLi46x11cXCCTyfSOkclkevszDNPqmAc5z4OcgxDSderlundMn/fr\njT72FhxGZJz2/ZKO2rpmAIBTL0s8G+XLcUTdW5CLjc5ToY1JeZCr1BxGZHx4PB7mzNCWv0xIzGfX\ndxBCeo4eOek1MTGR6xCIkaNrpHMcLFCgoFyzWZCtiAd/eRESE7vvF/HOuE4qq+XY/dNNKJWaqUrP\nPOmIKymXO/z3mJoQMDjW1IRmFXCzvgHrfjuPJ11EXfK7u9P7SX+JCCnXqgEAn35+HG/N6kflVbtA\nd7pGSNfz9e24mzvt3rF3cnKCQCBAcbHuxhbFxcVwdXXVO8bV1VVvfx6P1+qYBznPg5yDENI1ChvV\nOFOu3QF0krsQZgJKGO4lPVnMJvUebhYY7G/XzghiCBsRD0+5ahP5YyVKVMjprv29oh53hUio+e+y\nUNaEy6lVHEdECOlI7d6xF4lECAkJgVQqxUsvvcQel0qlmDJlit4x4eHheO+99yCXy9l59kePHoW7\nuzskEsN3VAwPD4dUKsXixYt1fu+oUaPaHBcaGmrw7yCm5c5dE7pGOpaaYbDn+A1YWGoW5AW52OBv\nEb7ddm59Z10ntzIrcCs7E1ZWmnr+i+c/joCBzh36O0xZsJpBpjQdtysbAADnlHZYMap/p/2+7vp+\nUlJpgx8PXgMAnL/ciJnREbAw75qnG6amu14jpGtVV1d32LkMqoqzaNEibNu2DVu2bEF6ejoWLlyI\noqIizJ07FwAQFxeH8ePHs/2jo6NhaWmJ2bNnIy0tDfv27cPq1at1EnQASElJQXJyMmpqalBRUYGU\nlBRcv64tw7Vw4UL88ccfWL16NTIyMvDJJ5/gxIkTePfddzvitRNCOsjxrHJca9n5U8jj4S1aMHsf\nhmGwdae2vOXwYR6U1HcwAZ+HeaHe4EFz7V0orMb5Arojfa8XnhkAh5a1L1XVTdh3KJ3jiAghHcWg\nOfZTp05FRUUFVq1ahaKiIgQGBuLIkSPw9PQEoFnkmpWl3c3O1tYWUqkU8+bNQ1hYGBwcHLBkyRLE\nxsbqnDc4OFjnw//QoUOQSCTIzMwEoLljv3v3bixbtgwrVqxAv3798MMPP9A3X0KMSG2zEltTCtj2\nCwNd4EUVXu6TkFiA6zc0ixX5fB5mTqfNqDrDgF5WeKpfLxy5XQYA2JSUhyAXG5gLBRxHZjwszEV4\ndepgfP7VBQDAz0duIPJxH/R2suI4MkLIozJ48ezcuXPZO/T32rp1633HAgICcOLEiTbPqVa3P/9x\n8uTJmDx5skExEkK63o6rhahu1sytd7YU45UAWgNzL4VShW/3aMsLTnyyP9xdbTiMqGebOcQDZ/Kr\nUNOsREmDHHvSZJgV5MF1WEblsVESHJbewu2sCiiUKuzYcwWL54VzHRYh5BEZNBWHEEL0uVlRj8O3\nytj2G8GedGdUj9+O3UZxy4ZAVpZiTH2RNqPqTLZmQsy5K5Hfl16M3OpGDiMyPnw+D3Oig9j2mfN5\nSL9Z1sYIQkh3QIk9IeShqFt2mGWgqfAS4mqLUZ72HEdlfOrq5OxCRQCY+sIg2FibcRiRaRjftxf8\nnawBAEqGQXxSHpiW3ZCJxqABzggP82Lb33yfDLWa/kaEdGeU2BNCHsrvt8two6IeACDi8/D3EC9a\nMKvHDwevoa5eDgBw6W2Np8b34zgi08Dn8TAv1Av8lmvySkktTuZUchyV8Zk5fQhELU/ZbmVW4K9z\nhu8OTwgxPpTYE0IeWFWTAtuvFLLtl/1d4WFDC2bvVSirxWHpTbY9c5o2iSKdr6+9JZ7z1VYe2pyc\nj3q5isOIjI+LsxUmPeXHtr/74SqampRtjCCEGDNK7AkhD2xbSgFq5ZoPfxcrM0zxpwWz+uzYc4Wd\n2uDv54yRobSAs6vNGOyOXhaaGu2VTQp8l1rYzgjT89JzA2HXUsmqorIRBw5ncBwRIeRhUWJPCHkg\n10rrIM0qZ9tvhXjBTEhvJfdKSy/F+SRtGdA50UE0VYkDliIBXg/2ZNu/3CxlN7AiGhbmIsyYEsi2\nD/yajvIK+hsR0h3RpzEhxGAqNYONSblsO9zDHmHudhxGZJzUagZbdyaz7XHhEvT3ceQwItM21ssB\nwS62ADSLvr9IzIWaFtLqeGJsX/Tx1ix+lytU+O6HqxxHRAh5GJTYE0IMduhmCbKqNGUDzQR8vDHM\ns50RpumvcznIzNYs1BQJBTp3Q0nX4/F4eCvUCyK+5olJRnk9jmaWtzPKtPD5PMTMGMq2T57Nwc3b\n9DcipLuhxJ4QYpDyBjm+u1rEtl8JcIOLFZVtvFdzs1LnbudzE/3gTDt6cs7DxhwvDdSuBdmWUsBu\nrEY0Av17Y0SIdh3IN98nU4lQQroZSuwJIQbZnJyPRqWmooinjTleGNCb44iM08+/3UBFpeaphp2t\nOSZPGshxROSOqYNc2S+jtXIltqUUtDPC9MyaHgSBQJMaZNwqx5nzeRxHRAh5EJTYE0LalSyrwV+5\n2hrgb4d6QySgt497VVY1Yd8v6Wx7+uQAWJiLOIyI3M1MyMff75o+djSzDNfL6jiMyPi4uljj2Qm+\nbPvbPVcgpxKhhHQb9MlMCGmTXKVGfJL2rt1j3o4IcrHhMCLjteunVDS3TO/w8rDD+Mf6chwRudcI\nD3uEe2h3SP4iMRcq2m1Vx8vP+8PWRvNko6y8AT//RuUvCekuKLEnhLTpQEYJ8mubAAAWQt3SgUQr\nJ68Kx//KYtuzX9FOaSDG5Y1hnjBr+bfJqmrEoZslHEdkXKwsxXjlJe2C758OpaOiZdE8IcS40acO\nIaRVsrpm7ErTLph9bbA7HC1oasm9GIbB1p0p7ELD4CGuCB5Cm3YZKxcrM7wS6Ma2d1wtRGmDnMOI\njM/4x/rCy0NTyra5WYmdP6ZyHBEhxBCU2BNCWvX15XzIVWoAgI+9JZ71deY4IuN06YoMV9KKAWhK\nK86aHsRxRKQ9L/j1hretBQCgSanGpiRaJHo3gYCPmBna6/iPU9m4nV3ZxghCiDGgxJ4Qotf5giok\nFFSx7bdDvSDg086p91IoVdj6vXYzqvERfeHtSZt2GTuRgI/5Yd5sO6GgCufyq9oYYXqCAl0ROtS9\npcVgK5W/JMToGZzYb9y4ET4+PrCwsEBoaChOnz7dZv/U1FRERETA0tISXl5eWLly5X19Tp48idDQ\nUFhYWKB///748ssv7+uzYcMG+Pv7s+eZP38+6uvrDQ2bEPIQmpRqfHkpn21P8HGCv5M1hxEZryPS\nWyiU1QIALMxFeGUybUbVXQQ4W+Opfk5se1NSHhoVVAHmbrNeGQJ+yxf6axmlSEikEqGEGDODEvs9\ne/YgNjYWy5YtQ3JyMkaNGoWJEyciPz9fb//a2lpERkbCzc0NSUlJ2LBhA9asWYN169axfbKzs/HM\nM89gzJgxSE5OxnvvvYd33nkH+/fvZ/vs3LkTS5cuxfLly5Geno4dO3bg8OHDiI2NfcSXTQhpyw/X\nilBc3wwAsBELMTvIo50Rpqmqugl79l9j29NeHAR7O3MOIyIPavYQD9ibadaNlDXKseNqIccRGRcP\nN1s8Haktf7l9Vwrk9OWHEKNlUGK/bt06xMTEICYmBgMGDMBnn30GNzc3xMfH6+3/3XffobGxEdu3\nb4e/vz8mT56MpUuXYu3atWyf+Ph4eHh4YP369RgwYABef/11zJo1C59++inb59y5cwgPD0d0dDS8\nvb0RERGBmTNn4vz584/4sgkhrcmpbsRP14vZ9qwh7rAzE3IYkfHa9VMqGpsUAAB3VxtMjOzPcUTk\nQdmYCfHGXZWeDt0sxY1yeip8t6nPD4K1lRgAUFJWj19+v8lxRISQ1rSb2CsUCiQlJSEyMlLn+IQJ\nE3D27Fm9YxISEjB27FiIxWL2WFRUFAoLC5GTk8P2mTBhgs64qKgoJCYmQqXS3A24czf/TiKfm5uL\ngwcP4plnnnmAl0gIMZSaYfC/i7lQtsyj9XeyRtRdUxWI1u3sSkhPaMtbxswYCpFQwGFE5GE9JnFA\nsIstAM1/A1TbXpe1tRjTJwew7R8PXkN5RQOHERFCWtNuYl9WVgaVSgUXFxed4y4uLpDJZHrHyGQy\nvf0ZhmHHtNZHqVSirKwMADBt2jSsWrUK48aNg1gsRp8+fRAUFIRPPvnE8FdICDHY0cxyXGvZiVPA\n42F+qDf4PFowey+GYbDlu8sANMnfsCFuGBbk1vYgYrR4PB7mhXpDzNd8JN6qbKDa9veY8Hg/eHlo\nvvw0NyuxbVcKxxERQvQx6ufrJ0+exMqVK7Fp0yYMHz4ct27dwoIFC7BixQp8+OGHrY5LTEzswihJ\nd0TXyP1qFQw+y2hCY8v02Sd6C1F2Kw1l3IbFqdaukyvXqpB0WbPGiM8HQofw6ZrqAUZaKfCbTLNz\ncPzZG7Auz4W9uP0vtqbybz8mzAKbb2huzh39IwMeLs3o34cW1RvCVK4R8nB8fX3b72Sgdu/YOzk5\nQSAQoLi4WOd4cXExXF31b8Di6uqqtz+Px2PHtNZHKBTCyUnz6H/58uWIjo7GnDlzEBAQgOeffx4f\nf/wx1qxZA7VabfirJIS06+dCBZvU9zLjYbyLUX/v54xcocZvf2rfu0aF9oKToxmHEZGO8pizEC7m\nmkRergb2F9CmVXfr622FoABtKddfjhZCqaLPYkKMSbuf3CKRCCEhIZBKpXjppZfY41KpFFOmTNE7\nJjw8HO+99x7kcjk7z/7o0aNwd3eHRCJh+xw4cEBn3NGjRxEaGgqBQDNPtaGhgf3/d/D5/Hbr6IaG\nhrb3soiJunPXhK4RXUlF1bhx6xYsrTTt9yN8Eexqy21QHGrrOtm9Lw1KlQhWViLY2phh0TtRsLIU\n39ePdE/L+9bhH8czAADZKqDZxQejvRz09jXF9xOf/gGY/48jaGpSoqEJKCqzxYvPDOQ6LKNlitcI\neXDV1dUddi6DquIsWrQI27Ztw5YtW5Ceno6FCxeiqKgIc+fOBQDExcVh/PjxbP/o6GhYWlpi9uzZ\nSEtLw759+7B69WosXryY7TN37lwUFBTg3XffRXp6OjZv3oxvv/0WS5YsYftMmjQJX331Ffbs2YPs\n7GxIpVJ88MEHmDRpEvh82luLkI7QpFTji0TtrpuPSxxNOqlvS2lZPfb/ks62Z0wZTEl9D3Nvbfv4\npDzUNis5jMi4ONpb6OzV8MMBWkhLiDEx6Fn71KlTUVFRgVWrVqGoqAiBgYE4cuQIPD01JcJkMhmy\nspK9g6wAACAASURBVLTVIWxtbSGVSjFv3jyEhYXBwcEBS5Ys0ak/36dPHxw+fBjvvvsuNm3aBHd3\nd3z++ed44YUX2D7Lly8Hn8/H8uXLUVBQAGdnZ0yaNAkfffRRR71+QkzenjTdmvWv31X6j+javvsK\nFErNfCUfiQOeHNeX44hIZ5gT5IELBdWoaFKgskmBLcn5iB3Rh+uwjMbE8f1x7GQW8gqq2YW0i+eF\ncx0WIQQAj+kh+0Pf/RjDzo62cyf60WNRXdlVjVjw+3WoWt4GFoZJMIHKW+q9Tq6kFeNfq0+y7Y/e\nfxyDBjh3eWyka5zLr8JHp2+z7VURvhh6z5MsU34/SUsvxfKP/2TbK5Y+hqAAlzZGmCZTvkaI4Toy\nh6X5LISYKDXD4H+JuWxSH+BsjfE+vTiOyjgplCp8/e1ltj1mpDcl9T1cuKe9ztz6zy7moklJO67e\nETDQGePCJWx787eX2adZhBDuUGJPiIn65WYprrfUrBdSzfo2/fL7TRQU1QAAzM2FmP1KEMcRka7w\n1jAv2Ig1M1aL65ux40ohxxEZl5mvDIG5uebvU1BUQzvSEmIEKLEnxATJ6pqx/UoB254yyBXedhYc\nRmS8ysobsGf/Nbb9yuRAODrQ38oUOFiI8LehHmz755ulyCiv5zAi40ILaQkxPpTYE2JiGIbB/y7m\nokmpqT/tbWuBqYP070lBgK07kyGXa6qieHva4enI/hxHRLrS+L69EOyimVuvZhhsuJADBdVuZz0d\n2R9eHpo5wc3NSp0pa4SQrkeJPSEm5lhWOS4Xa6aV8MBD7AgJxAJ6K9An+aoM5y7ms+03Zw2DgP5W\nJoXH42F+mDfMWv7dc6ob8eP14nZGmQ6BgI+/zx7Gti9cKsD5pII2RhBCOhN9QhFiQioaFdh8WZuo\nPu/njAG9rDiMyHgplWqdu4+PjZbQglkT5WpthllDtFNy9qQVIae6kcOIjMugAc6IjPBh219/ewkN\njQoOIyLEdFFiT4iJYBgG8Um5qFNoKle4WpnhtSHuHEdlvE5fKENRcS0AwNJChFnTacGsKZt015dg\nZcuUHHXPqBbdIV6bOgR2tuYAgIrKRuzce5XjiAgxTZTYE2IizuRX4Wx+FdteMNwb5kIBhxEZr8oq\nOU6cKWXb0S8Hwt7OnMOICNf4PB4WDpdA2FI5KqO8HidLaUfaO6ytxYiZMZRtH5bexs3b5RxGRIhp\nosSeEBNQ06xEfGIe236qnxOCXGzbGGHafj1WBIVScze2r7c9nnqSFswSQGJngehAN7b9u0wJWRMt\npL1jzEgvDB18ZyE+g/itSVAq6e9DSFeixJ4QE/D15XxUNWvmvPayEGFOkEc7I0xXYnIhrt+sZdtv\nzg4Bn0/1/YnGy/6u8HXUTMlRMcDuXDmUapqSA2gWGv99dgjEIs2TwOzcKhz6/QbHURFiWiixJ6SH\nSyyqxh/Z2kfi80K9Yd2y6Q7R1dikwFfbL7HtJ8f1xYD+tBsv0RLweXh3hARivubjs6CRwQ/XZBxH\nZTxcnK0wbXIA296zLw3FpVT7n5CuQok9IT1YbbMSn1/IZdvjvB0wwsOew4iM266f0lBWrtlgx9JC\ngNemDeE4ImKMJHYWmDFYOyVnd1oRblfSxkx3TIryg8RL8z4jV6jw1fYkMLTQmJAuQYk9IT3YV5fz\nUNYoBwDYmgnx92FeHEdkvG5lVuDXozfZ9tPj3WBrY8ZhRMSYTR7oAomV5iNUxTBYm5BNG1e1EAr5\neDsmBIBmCtvlKzKcOZ/X9iBCSIegxJ6QHupMXiX+yK5g2/NDvWFvLuIwIuOlVKrxxZZE9q5i///f\n3p3HRVntfwD/PLMxw77PsMkiCO6iaAIumAKZldlN20u7eVPLLa9p5s26aotWav1ySSu1suxq2mYJ\nmSKCqKCoIIjKvgybLMMAsz6/PwYHJ1YVnWHm+369eMFz5jzjGXw4853znPM9fjYYNtDByK0ipozD\nMHjChw9ey7tofl0TvsssM26jTEhQXxdMntRXf/zFN+molymM2CJCLAMF9oSYodpmFT5LbZ2Cc7+f\nMyJ9nIzYItP28++XUVCkSwUq4HMxdbIXGIYWzJLOuVlxMMWj9cPy/7LKkVNN88lveGb6YDg7iQAA\ndfXN2PH1uS7OIITcKQrsCTEzLMvi/84Uok6hy7HtKhLQFJxOlEpl2Hvgkv74qX8MgrOjwIgtIr1J\nhAsXg9zsAABalsXHp/KhpCk5AHQbu82ZNUJ/fCKlECmpxZ2cQQi5U90O7Ddv3oyAgACIRCKEhYXh\nxIkTndbPyMhAVFQUrK2t4ePjg9WrV7epk5CQgLCwMIhEIgQGBmLbtm1t6shkMixYsABeXl4QCoXo\n168f9u3b191mE2Jx/sq/jpMlrRtRLRzlS1lwOsCyLLZ+lQaVWrcbb4CfEx6KDTJyq0hvwmF0WXKE\nLXNyiuqbset8iZFbZTrChnkiaoyf/njbzrOQNdCUHELulm4F9nv37sWiRYuwcuVKpKenIyIiApMn\nT0ZxcfufvGUyGaKjo+Hh4YG0tDRs2rQJ69evx4YNG/R18vPzMWXKFIwZMwbp6elYvnw55s+fjwMH\nDujrqNVqTJo0CdeuXcO+ffuQk5ODnTt3wt/f/w5fNiHmqbJRia1prYvUJvd1w3AP2oiqI38dz0dG\nVgUAXQ7ueS+GgculG5nk1khsrfDPYd7644M5FUgrqzNii0zLi08Pg5MDTckh5F7o1jvYhg0b8OKL\nL+LFF19EcHAwPvnkE3h4eGDLli3t1v/mm2/Q1NSEXbt2oX///njsscewbNkyfPzxx/o6W7ZsgZeX\nFzZu3Ijg4GC89NJLeOGFF/Dhhx/q63z55Zeorq7GTz/9hPDwcPTp0wcREREYMWJEe/8sIRaNZVls\nPFWAxpbRZ4mNFf45jDai6khtXTN2fX9efzx1cj8E+NE6BHJ7Jvd1xSjP1gXXG04VoLZZZcQWmQ5b\nWwHmvNj6vp14shCn0uiuBiF3Q5eBvUqlQlpaGqKjow3KY2JikJyc3O45KSkpGDt2LASC1nmqsbGx\nKC0tRUFBgb5OTEyMwXmxsbFITU2FRqMLTH766SdERkbi1VdfhYeHBwYOHIh33nkHarX61l4lIRbg\nt6tVSC+vBwAwYPDaaD+IWnaAJG198c05NMh1qUDFbjZ4YtrALs4gpGMMw2DhKF84WukW09Y0q7Dp\ndAHlb28xMtQT4yN99cdbv0pDQ4PSiC0ixDx1OfG2qqoKGo0GYrHYoFwsFuPIkSPtniOVSuHj49Om\nPsuykEql8PX1hVQqbfNhQSwWQ61Wo6qqCmKxGLm5ufjrr7/wzDPP4NChQ8jPz8e8efMgl8uxbt26\nDtucmpra1csiFs7crpFKhRaf5CigalmzF+XOQ1NBNlILjNsuU5WVU4+4v1qzBo1/2B0XL6a3qWdu\n1wm5O26+TiY7aPBFni5gPXalEU6NVYhwpTUuABA6gEVishKyBjXk8kb8d91vmPGIZSzsp76EdCYo\nqOfWdpn0ZFKtVguxWIzt27cjNDQU06ZNw3//+98OpwARYonUWhbfFij1Qb1YyCBGTIFERxrkahz8\no3UaQOggRwT62xqxRcSchNhzMeamQP6XMhWkzZQlBwCsRTxMfcBTf3w+sw5ZOfVGbBEh5qfLd39X\nV1dwuVyUl5cblJeXl0MikbR7jkQiabc+wzD6czqqw+Px4OrqCgDw8PCAQCAwyCfdv39/NDY2orq6\nGi4uLu3++2FhYV29LGKhboyamNM1suNcMWo45bC2AfgcBu9GhyDAydrYzTJJLMti/acnwcIKNjaA\nk6MIK5fGwtbWML2lOV4npOd1dJ0MCdVicVw28uuaAACHG6zxUXgwBLQwG2FhwPW6U0hI1t1OPHqy\nAdMeGdPmb9BcUF9CuqOurucW23fZy/D5fIwYMQLx8fEG5fHx8YiMjGz3nPDwcCQmJkKpbJ0/FxcX\nB09PT/j6+urr/P054+LiEBYWBi5XNy84MjISV69eNahz+fJlWFtbdxjUE2JJ0srqcOBy6wfkWUO9\nKajvxImUIoM82q/8M8xsAwpiPAIuB0vD/SHg6N5ic2sbKQXmTf75bCgcHYQAdIvYt+5Mo7UIhPSQ\nbg0fvPbaa9i5cye++OILZGdnY+HChSgrK8OcOXMAAG+88QYmTZqkr//000/D2toaM2fORGZmJn78\n8Ud88MEHWLJkib7OnDlzUFJSgsWLFyM7Oxs7duzA7t27sXTpUn2duXPn4vr161iwYAFycnJw+PBh\nvP3223jllVd66vUT0mvVNqvwcUrrJPowDwc80s/NiC0ybddrmrB991n9cfSEAAwf6mHEFhFz5uco\nwos3ZaU6mFOBs2U07QTQZcmZO6t1BDv5dBGOJdGCIEJ6QrcC+xkzZmDjxo1Yu3YtQkNDkZycjN9/\n/x3e3rq8vVKpFHl5efr69vb2iI+PR2lpKUaOHIn58+dj6dKlWLRokb6On58fDh06hMTERISGhuK9\n997Dp59+ikcffVRfx9vbG3FxcTh79ixCQ0Mxb948vPTSS1izZk1PvX5CeiUty+LjlHzUKnTp9JyE\nfCy+z9dg2hppxbIsNn+Rqs+C4+5qg5lPDTVyq4i5eyjIDSNvSoH58al8SoHZYuRwT0waH6A/3r77\nLKTlDUZsESHmgWHN5P7XzfOTHBwcOqlJLJm5zHc8kF2OHem6KSUMGKyJCsQwCW1E1ZE/E3Kx+YvW\nrBSrV0zAwJCO726Yy3VC7q7uXCe1zSq88nuW/kP4ULEd1kQFgUMfwtHcrMaS/8SjrFwGAAgKcMHa\nlRPA45nPWgTqS0h39GQMaz5/PYRYiCvX5dh503zdx0LcKajvRHmlHF9+25rK8qHYfp0G9YT0JEch\nH/8O9wMDXSB/vlyGby+WGblVpkEo5GHx3PvA4eh+N1dyq/G/ny4ZuVWE9G4U2BPSizSpNFh/Mh/q\nlhttQc42eG6wZ+cnWTCtlsVnO86guVm3qZ2H2A7PTh9s5FYRSxMqscdTg1qzyH1/qQypZT2XBaM3\nCwxwxtP/GKQ/3vdzFi5drjRiiwjp3SiwJ6SXYFkWW9KKUCJrBgCIeFwsC/cHn1LodejQn1eRkVUB\nQLcz6IKXR0EgoN14yb331EAPDBO33ln78GQ+Khtp51UAeHRKiP4uGsuy2LT1NOT0uyHktlBEQEgv\ncTi3Gkfyq/XH88J84GFnZcQWmbb8wlp8/f0F/fG0KSEIDqQ0ucQ4OAyDpeF+cBHxAQAypRrvJ+VC\npaHNqzgcBgtfvg821rrUs5XVcny+62wXZxFC2kOBPSG9QE61HFvTCvXHE/1ccL8fBakdaW5W46PP\nUqBSawAAvj6OmDFtgJFbRSydo5CPZREB+oWz2dWG62UsmauLNebMGqE/TjxZiARKgUnILaPAnhAT\nV69Q472kXKi0unn1/o4izAvrY+RWmbYvvjmHkpac4QIBD0teGQ0Bn6bgEOMb6GaLF4a0ros5mFOB\npKIaI7bIdETe54P7x/rrj7ftSkOpVGbEFhHS+1BgT4gJ07IsPjyZh4qW+aY2fC5WRAZAaEbp4Hra\niZRCHDneuq/G7OdC4e1JWYOI6fhHiBj33ZTffuOpApS2rJ2xdP98bhjE7rYAdHfe1n96EgqF2sit\nIqT3oOiAEBP2XWYZ0qStu1UuGe0HTzuhEVtk2qQVDdj8ZWu++rHhfXD/OD/jNYiQdjAMg8X3+UFs\no1sj06jWYM2JXDSpNEZumfGJhHwseWW0Ppd9QVEtPt99Fmay5Q4hdx0F9oSYqNSyOnyXIdUfz+gv\nwX1ejkZskWlTqTX46LMUfWpLsbstXp45gnbjJSbJzoqHNyL9wW/J4V5Q14SPUvKhpQAWgf7O+Oez\nofrjo4n5+DMhr5MzCCE3UGBPiAmSNijw4cl8sNC9yQ8T2+O5IZSvvjN7/peBa3nXAQBcLgdL5o2G\ndUsGEkJMUZCzDV4d6as/PllSi28zaPMqAIiZEIDxka2/mx27z+FaPq1FIKQrFNgTYmKUGi3eS8qF\nTKkbeXYVCbA03I+2oO/E2fNl+On3y/rj52YMRmCAsxFbREj3TPJ3wbRgsf74+8wyHC+8bsQWmQaG\nYTBn5gj08datRVCpNVj/STIaGii/PSGdocCeEBPCsiw+PVOAqzWNAAAew2DFmAA4CmnkuSPXa5vw\nyeen9cehQyR4KLafEVtEyK2ZNdQLIyStC7w3nCrAtZY+wJJZWfHw+oIIiFr6v4oqOT75/DS0Wpqu\nREhHKLAnxIT8cEmKv/JbR+tmD/dBsIuNEVtk2tRqLT76vxTUyxQAACcHEebPHgUOh+5ukN6Dy2Hw\neoQ/vFoWxis1WqxOvIbaZpWRW2Z8nhI7vDp7pP44Nb0UB37NNmKLCDFtFNgTYiJOFNVg98VS/fED\nfV0xJdDViC0yfbv3XkBWTiUA3a37hXNGwdGBsgaR3sdWwMNbY/vCmqfbb6GyUYk1J3KhpJ1pET7S\nG488EKw/3rM/Axcyy43YIkJMFwX2hJiAnGo5PkrJ1x8PcbfD3BF9KKNLJ06kFOLXwzn642ceH4Qh\nA8WdnEGIafO2F2JZpD8Y6P7us6oasDm1kFI9Anh2xmCE9NMNdLAsiw//7yTKK+VGbhUhpqfbgf3m\nzZsREBAAkUiEsLAwnDhxotP6GRkZiIqKgrW1NXx8fLB69eo2dRISEhAWFgaRSITAwEBs27atw+f7\n7rvvwOFw8Mgjj3S3yYT0ClWNSqw5cU0/MudlJ8SKyADwaDpJhwqL6/B/O1rz1Y8a7oVpD4UYsUWE\n9IwwDwfMGuqlP47Pq8a+bBqd5vE4+Pcr4XCw192Ra5Ar8e7HJ9BE05UIMdCtwH7v3r1YtGgRVq5c\nifT0dERERGDy5MkoLi5ut75MJkN0dDQ8PDyQlpaGTZs2Yf369diwYYO+Tn5+PqZMmYIxY8YgPT0d\ny5cvx/z583HgwIE2z5ebm4vXX38d48aNu82XSYhpalZr8N/Ea6hu0r052Ql4WDWuL+yseEZumelq\naFDi/U1JULZkDfIQ22H+v0bS3Q1iNh4Lccf9fq1ZnXaeL8Ff+dVGbJFpcHYSYdnCCP3mVUUlddi4\n5RQtpiXkJt0K7Dds2IAXX3wRL774IoKDg/HJJ5/Aw8MDW7Zsabf+N998g6amJuzatQv9+/fHY489\nhmXLluHjjz/W19myZQu8vLywceNGBAcH46WXXsILL7yADz/80OC51Go1nn76abz77rvw9/e/g5dK\niGnRsiw+SsnXZ7/gMgzeiGxdQEfa0mi0+PD/TkJa3gAAEAh4WLYwAjbWAiO3jJCewzAM5o/0xWB3\nO33ZxlMFOFtW38lZliEkyBVzZ4Xpj8+cK8V3+zOM2CJCTEuXgb1KpUJaWhqio6MNymNiYpCcnNzu\nOSkpKRg7diwEgtY329jYWJSWlqKgoEBfJyYmxuC82NhYpKamQqNp3VZ7xYoVCAgIwHPPPdf9V0VI\nL/D1hVIkF9fqj+eF9cFQsX0nZ5Bd31/AhUut0xIWvjxKn+eaEHMi4HKwckwA/BxEAAANy+LdpFxc\nvU5pMCeM9cPDD7SmtN3/SxaOnsg3XoMIMSFdBvZVVVXQaDQQiw0XpYnFYkil0nbPkUql7dZnWVZ/\nTkd11Go1qqqqAABxcXHYt29fp3PvCemNDl2txA9ZrX8/j/ZzxwN9KQNOZ44k5Bkslp3x6ECEj/Q2\nYosIubtsBTy8Mz4Qbi13pJrUGqxKuAppg8LILTO+558YgtAhEv3x5i9SkZFVYcQWEWIaTHYib1VV\nFWbNmoXvv/8ednZ2XZ9wk9TU1K4rEYtmzGvkYp0GXxcocSPRRYg9B0M1UqSm0gK5juQVyvHVd3m4\nkflvQD97BHg33vX/R+pLSHfc7evkcWctNl9XoEkDNMqB+QfP4NVAK9jwLHtdyYTRVrhyVYPySt0H\nnZVr/sDLz/eFm4uVkVvWFvUlpDNBQUE99lxdjti7urqCy+WivNww6CgvL4dEImn3HIlE0m59hmH0\n53RUh8fjwdXVFZmZmZBKpZg4cSL4fD74fD52796N3377DQKBAFeuXLmlF0qIKciTa7CnsDWo97bm\n4Nk+AnBo4WeHKqsV+HZ/oT6oF7tZ4fGHvWgTKmIxJEIOZvoJ0LJmFFUKFl/mKaG08EWjQiEXz0/3\nha2NboyyqVmL3T/ko0GuNnLLCDGeLkfs+Xw+RowYgfj4ePzjH//Ql8fHx2P69OntnhMeHo7ly5dD\nqVTq59nHxcXB09MTvr6++joHDx40OC8uLg5hYWHgcrkYOXIkLl68aPD4m2++idraWmzevLnThbRh\nYWEdPkYs241RE2NcI/m1TVh/5DIEIh4EADxtrbB+UjAcW7ZLJ23VyxT44p0j4HCtYGMDONgL8cHb\nE+Huend34zXmdUJ6j3t5nYQB8Opbg/eS8sCCRRWA+GYHvDmmr8WnxvULCMHKtcegVKqhUAG/HpFh\n9YooiEygb6W+hHRHXV1djz1Xt7LivPbaa9i5cye++OILZGdnY+HChSgrK8OcOXMAAG+88QYmTZqk\nr//000/D2toaM2fORGZmJn788Ud88MEHWLJkib7OnDlzUFJSgsWLFyM7Oxs7duzA7t27sXTpUgCA\ntbU1BgwYYPDl6OgIOzs79O/fHzyeyc4iIqSNUlkz3jx6BQ0q3cJwRys+VkcFUVDfCaVKg/c3JqG8\noiUDDp+LN18bc9eDekJMVaSPE14e3rqu5HRpHT5KyYPWwjewCvR3xpJ5o/Upb3Pza/DBpmSo1Jou\nziTE/HQrsJ8xYwY2btyItWvXIjQ0FMnJyfj999/h7a3rYKRSKfLy8vT17e3tER8fj9LSUowcORLz\n58/H0qVLsWjRIn0dPz8/HDp0CImJiQgNDcV7772HTz/9FI8++mgPv0RCjKuyUYk3j15BrUKXq17E\n4+K/UYGQ2JrePFBTodWy2LjlFLKvVLWUMFg09z4EBjh3eh4h5u7hfu54YkDrNNjjhTXYdLrA4oP7\nkcM98fLMEfrjC5nl+PTzM5TjnlgchjWTvapvvo3h4EDp70j77vVt0dpmFZYdyUGxrBmALoXd6vGB\nGOR+awvCLQnLsvh811kc/uuavuyFJ4di6oPB96wNdPucdIexrhOWZfH5uWL8nNOaBWZKoBvmjvCx\n+I3afjh4Cd//2JrX/sHoQPzz2VCj/V6oLyHd0ZMxbLdG7Akht65eocbKY1f0QT2PYfDmmAAK6rvw\nv5+yDIL6h2L74ZHJ/To5gxDLwjAMZod6IyagNUXub1cr8fm5YpjJWN1tmz61Px6YGKg/PhR/FXv2\n0QZWxHJQYE/IXSBTqPHm0SvIq20CAHAYBq9H+CPMg+4mdebwX9cMRtvGhvfBzKeGWvwoJCF/x2EY\nzB/ZB+P7tE5P+zmnAtvOWnZwzzAMXnouFBGjfPRl+3/Jwv6fs4zYKkLuHQrsCelhMoUabx67gtxa\n3Q6RDBgsvs8XkT5ORm6ZaTuWlI9tO8/qj4cOkuDV2SMprSUhHeAwDJaM9sOYm/qWX65UYOvZIosO\n7jkcBgvnjELYME992bf7LuLXOEqTTcwfBfaE9KAb02+u1bQG9Yvu88X9fi5GbplpO3mmGJ9+fgaA\nLhjp6++M1xeEg8/jGrdhhJg4LofB6+H+GHtTcP/rlUp8llpo0Qtq+Twu/j0/HIMHuOvLvvzmHA7F\nU3BPzBsF9oT0kNpmFZb/lYOrLUE9ACwY1QeT/Cmo70za+TJ8vDlFP8Lo6+OIVUvHmUQOakJ6Ay6H\nwdJwf4zr0xrc/36tCh+l5ENtwVlhBHwuli+KRHBgax+84+tz+I1G7okZo8CekB5Q1ajE60dyUFCn\nm1PPgMGCkb4Gi9tIW2fPl2HdpmRoWraV9ZTYYdXr42BrKzByywjpXbgcBv8e7Y/7/Vrn3B8ruI73\nknKhvLFtswUSCfn4z9JxBsH9F9+cw6+Hc4zYKkLuHgrsCblDZTIFXj+Sg5KW7DcchsFro30R25eC\n+s6knS/D+5uS9JvIuLva4J3l4+HoIDRyywjpnbgcBovv88Pkvm76spSSWrxz/BoaVZa7WZO1qG1w\n/+W36fjxF1pQS8wPBfaE3IFrNY3495+XUS5XAAC4DIPlEf40p74LZ86V4oNNSVCrdSOJ7q42eOeN\nKLg4Wxu5ZYT0bhyGwSthPvhHiFhfll5ej2VHclDTpDJiy4zLWsTHW68bBvff/O8ivv7hgkUvNCbm\nhwJ7Qm7ThQoZlh3J0e8oK+BwsHJsX8p+04WTZ4qx7pNkg6D+vyuiIHazMXLLCDEPDMNg1lAvPD+4\nNStMbm0jlvx5WX9n0RKJhLrgflD/1gW1B37Nxvbd52iHWmI2KLAn5DacKKrBW8euoKllGoktn4vV\nUYEY5Ul56jtzJCEPH/7fSf2cerGbDVaviIK7KwX1hPQkhmHwxEAPLBzlC07LPhDlcgWWxF9GZmWD\nkVtnPCIhHyuXjDVIhfnHkavYsCUFSguerkTMBwX2hNwClmWxL0uK95JyoWoZ4XER8fHBxGDaUbYL\nP/9+GZ99cUZ/29tDbIfVKybAjYJ6Qu6amABXrBwTAAFX93YvU6rx5tEc/JVfbeSWGY9AwMXrCyIQ\neV/rJlZJp4qwev1xNMiVRmwZIXeOAntCukmtZfF/qYX46nyJvszbToh1E4Ph5ygyYstMm1bL4usf\nLmDnd+f1Zf59HLF25QS4utCcekLutvu8HPHehCA4WPEAACoti49S8vHNxVKLnV/O43GweO5oPDAx\nUF+WmV2JN9ccRfX1xk7OJMS0UWBPSDfIFGqsSriKP65V6csGudnhw0nBkNhaGbFlpk2p0mDTtlM4\n8Gu2viyknyv+uyKKst8Qcg+FuNri4+gQ9LFvHYT4LrMM7ybloslCp6BwOAxmPx+KZ2cM0ZcVldTh\n9VVHcDX3uhFbRsjto8CekC7k1zZhUVw20svr9WX3+zljTVQg7FpGwEhbDXIlVq8/jsSThfqyMFEc\nRAAAIABJREFUsGGeWLV0HGysKU89IfeaxNYKH04KxnCJvb4subgWr8VfRqmFLqplGAaPPRSCBS+P\nAoejW4tQU9eElWuPIulUkZFbR8ito8CekE4kF9VgyZ/ZkLakswSAZwZ54rX7/MDn0p9PR8qkMqxY\n/Rcysyv1ZTET+mLZwghY0YchQozGRsDF2+MC8Ui/1swwhfVNWByXjdSyOiO2zLiiIv3w1k2DDkqV\nBh99dhLf/5hJGXNIr0KRCSHt0GhZfJlejLVJuWhuScso4nGxckxfPD3IA0xLlgnS1vnMcix75wiK\nS1vvcDw7fTBenjkcXPowRIjRcTkMXh7ug8X3+UHA0f1NNqg0WJVwFbvOl0BjoYHskIFifPD2RHiI\nWxMh/HAwE+9vSoK8kRbVkt6h2++ymzdvRkBAAEQiEcLCwnDixIlO62dkZCAqKgrW1tbw8fHB6tWr\n29RJSEhAWFgYRCIRAgMDsW3bNoPHd+zYgXHjxsHZ2RlOTk64//77kZSU1N0mE3JbrjepsOLoFezP\nLteXSWx0t7DDvR2N2DLTxrIsDsVfwX/XtWaW4PO4WDx3NB57uD99GCLExEzyd8EHE/vBVdQ6Ne6H\nLClWHL2CagsNZD0ldvhg1UQMGdC6wVfquVK8vuoICost944G6T26Fdjv3bsXixYtwsqVK5Geno6I\niAhMnjwZxcXF7daXyWSIjo6Gh4cH0tLSsGnTJqxfvx4bNmzQ18nPz8eUKVMwZswYpKenY/ny5Zg/\nfz4OHDigr5OQkIAnn3wSR48exenTpxEcHIzY2Fhcu3btDl82Ie07J63H/D+ykFEp05eN8HDAhpgQ\nynzTieZmNT7Zdho7vj6nz7Lh5CDC2pUTMDa8j5FbRwjpSD8XG2yKDTGYd59RKcOrh7NwqqTWiC0z\nHltbAVb+eywefqCfvqysXIbX3z6CoyfyjdcwQrqBYbuR62r06NEYNmwYtm7dqi/r168fpk+fjrVr\n17apv2XLFrzxxhuoqKiAQKAbCVi7di22bt2KoiLdYpRly5bh4MGDuHz5sv682bNn49KlS52Oynt4\neGDlypV45ZVXDMrr6lo/STs40CZBpH2pqakAgLCwMINypUaL3RdKceBy6yg9h2Hw7CAPTB8g0W/w\nQtoqlcrwwaZkFJW0/g329XfGG4si4ezUOz8MdXSdEHIzc7pOtCyLHy5J8W1GGbQ3hQUPBrrhn8O8\nIeRZ5jS6EymF+OyLVCgUan3Z+Ehf/OuF4RAJ+V2eb07XCLl7ejKG7fIvVaVSIS0tDdHR0QblMTEx\nSE5ObveclJQUjB07Vh/UA0BsbCxKS0tRUFCgrxMTE2NwXmxsLFJTU6HRtJ96S6FQoLm5GU5OTl01\nm5BuK6pvxr//vGwQ1Dta8bEmKghPDPSgoL4TJ1IK8e+34g2C+vvH+mPtmxN6bVBPiCXiMAyeHOiB\nNVFBcBG1BqyHrlZi4eEsXK6WG7F1xjNmdB98sMpw3n1CUgH+/Z8/cS2/xogtI6R9XaanqKqqgkaj\ngVgsNigXi8U4cuRIu+dIpVL4+Pi0qc+yLKRSKXx9fSGVStt8WBCLxVCr1aiqqmrz7wHAypUrYWdn\nh0ceeaTTNt/4hExIR1JTU6FlWSRWafCHVIWW9bEAgGA7Dp6QaKEquoxUynbWLoVSg1/iynDuYuut\neh6PwcMxHggbyuDChXNGbF3Pob6EdIe5XSf/FLPYX6zCxTrdIFuOvBFzf7qO8W48xIh54HMsb7Dj\n6WnO+CVOoe/zruY24tWlP2HiWHeMG+2mT5XZEXO7RkjPCgoK6rHn6jV55zZt2oTt27fjyJEjsLW1\nNXZzSC9XqdDihyIV8uWtET2XAR7y5CPShUsLPTtRXNqIH34uRnVN6+I6Z0c+nprWB54SGqUnpLez\n4TF4zpePMzUc/FSiglILsCxwrEKNrHoNHvcWwM/GsqbmWAm4ePwhb/T1s8XPf5RCqdJCqwXiEyqQ\nfUWGxx/2hqszbVZIjK/LwN7V1RVcLhfl5eUG5eXl5ZBIJO2eI5FI2q3PMIz+nI7q8Hg8uLq6GpRv\n3LgRq1atwh9//IERI0Z0+aJoLhvpyKkzZ5BQqcbJBiGU4MLaRlce4GiN10b7wt/R2rgNNGFKlQY/\nHLiEA7/lgWV5sLHRdR/jI3wx+4XhsBZ1Pd+0t6B5saQ7zP06GQngsQYFNp4uwMUKXUIBGYCdZQwm\nB7rihSGesBX0mvHBHhEWBkx5QIZPtp1GzrVqAMD1OuCr76V48h8D8cgD/QzS+pr7NUJ6xs1z7O9U\nlx+5+Xw+RowYgfj4eIPy+Ph4REZGtntOeHg4EhMToVS2jujFxcXB09MTvr6++jp/f864uDiEhYWB\ny+Xqyz7++GOsWrUKhw4dQnh4ePdfGSF/k1XVgA05CvxepoZSqxup5zIMnh7kgQ0xIRTUd+Jafg2W\nvvUnfvw1S5/1RijkYeHL92HhnPvMKqgnhLSS2Frh3QlBmDuij34BLQsWh65WYs6hS0gouI5u5OAw\nK54SO6xdOQHPPD5YH8Sr1Bp8vfcCXl9Fc++JcXHffvvtt7uqZG9vj1WrVsHDwwPW1tZYvXo1EhMT\n8dVXX8He3h5vvPEG3n//fTz//PMAdBlztm3bhvT0dISEhODEiRNYunQpVqxYgdGjRwMAAgMD8cEH\nH6CyshJ+fn44ePAg3nvvPWzYsAEhISEAgPXr1+PNN9/Erl27EBoaCrlcDrlcDq1WCysrw1teCkXr\nzqBCobCnfj/EDNQ2q7DtbDG2phWjpqklv7qAjwBHa6waF4goX2daINuBpmYVvt57EZu/SEVdfeuW\n8wND3PDW0vEYGOJmxNbdPaWlpQAAT09PI7eEmDJLuU4YhkE/FxtE+TqjtEGBUpnu/bZZrUVScS3O\nV8jQ18kaThb0AZ/DYTAg2A0jQz2Rc60atXW6/rG2rhl/HsuDrEGBfoEuqKrSzUww92uE3JmejGG7\nle4SALZu3Yp169ahrKwMgwYNwsaNG/Uj9rNmzcLx48cN8stnZmbilVdewenTp+Hk5IS5c+di5cqV\nBs+ZmJiIxYsXIzMzE56enli+fDlmz56tf9zf3x+FhYVt2vLCCy/gyy+/NCijdJfk79RaFr9eqcC3\nF8vQqNYtAmuUN4LPAeaE98PUfu7gWuAisO46c7YUn+8+i+rrjfoygYCH558YjAcmBna5WKw3o9vn\npDss8TphWRbJxbXYmlaE680qfTmHYRAb4IKnBnrAxVrQyTOYH7Vai18O5+D7/ZlQqVuz+jnYCzF6\nuAihgxwxatRII7aQmLqejGG7HdibOgrsyQ033nh2XShFiazZ4DE/bjOmevEREzHKSK0zfcWl9fhq\nTzrOXZAalA8ZIMacWSMgEZv/4nVLDNjIrbPk66RRpcGejDL8nFMBzU1hhIDLwcNBbni8vwT2VpY1\n/75MKsO2nWdx4VLr+kG5vBFeHiIsefV+s73DSe4cBfbtoMCeAEBmZQO+Ol+CrKoGg3IvOyH+Ndwb\nKLkCwDLfiLvS0KDEDz9dwqH4K9BqW7sFO1srzHpmKMZH+FpMtiBLDthI99F1otsHZPvZIqRJ6w3K\nrXlc/KO/GFP7uUPE53ZwtvlhWRYnzxTjqz3nUX29EXK57o6njY01Rg33wnNPDIaXh30Xz0IsTU/G\nsJb1cZqYrayqBnx7sQznyg3fXGz4XDwxwAOP9HMDn8tBaomRGmjClEoNfou/gh9/yYa8UXnTIwyi\no/zx7IzBsLOlNG6EkLZ87IV4Z3wgzkrrsftCKa7W6ALZRrUGX18sxc85FZgxQIIHA90g4Jp/ikyG\nYRAxygfDh3hg/y9Z2LPvLNRq3UDJ6bMlOHOuFOMjfTHj0QGQuJv/3U9y71FgT3otlmVxsbIB+7LK\nkVZmmCqKz2HwUJA7ZgywvNvB3aVSa/DX8Xz876dLuF7TZPDYgGA3vPjMMAT40S7PhJDOMQyDER4O\nCJXYI7m4Fl9fKEVxyzTIOoUa288V43+XyvFIPzdMDnSziD5ZKOThmemDIXaWIf54Ba7k6QZNWJbF\nsRP5OJ5cgInj/DH1wWB4Suy6eDZCus/8/7qI2dFoWSQV1WB/drl+dOgGDsMgytcZzwzygIRGmdul\nVGnw1/E87P8l22BhLABIxLZ45vHBiBjlbTHTbgghPYPDMBjj44RwL0ccya/GnowyVLbcBaxVqLD7\nYin2XpIiOsAF04LFFtFHOzoIMP1hbzi6BODb/2XgfIZu7ZJWyyL+WC7ij+VhdJgXpj4YjOBAFyO3\nlpgDCuxJr9Gk0uBwbhV+ulyBCoMpIwADBlG+TnhqkAe87CjdaXsa5ErEH8vFr4evoKbWcITe0UGI\nGY8OwKTxAeDxzP92OSHk7uFyGMQEuCLK1xm/X63Cj9nlqGpJNazQaPHrlUoculqFcG9HPBbsjmAX\nG7MfSAj0d8aq18fh0uVK7NmXgUuXK1seYZGSWoyU1GKE9HPFtAdDMGKYh1lnHSN3FwX2xOQV1DUh\n7loV4vOqIVdpDB4TcDiY6O+MR4PF8LangL490vIG/H7kKuKP5aK5WW3wmL2dFaY+GIzJEwMhFFJ3\nQAjpOQIuB1OD3fFgoCsSi2pwILsCubW6u4RaVnfnNamoBv6OIsQEuGKCrzPszHyazoBgN6xeEYWM\nrEocPJRtkH0sO6cK7+WcgJeHPR6MDsTY8D6wtbGs1KHkzpn3XxDptRpVGiQUXEd8XjUuV8vbPG5v\nxcNDQW6YEugGR6HlbIrSXSq1BqfTShF/NNcg9doNjg5CTJsSgpgJAbAy8zdSQohx8bkc3O/nggm+\nzjhfLsOP2eUGWXTyapuw7WwRvkovwWhvB8QGuGKI2M5sNw5kGAaDB7hj8AB3FBbX4eChy0g8WQiN\nRrcjeklZPbbvPoude84jYpQ3Jo73x8AQN7O/q0F6Br2jE5OhZVlkVjYgPrcaJ4pqoGjp5G7mZSfE\ntGB33O/nAiuaMtKGtLwBccdycTQx32Cn2Bu8Pe0xdXIwxkb0gcCCUtARQoyPYRgMk9hjmMQeebWN\nOHi5AscLa6Bs6euVWi2OF9bgeGENxDZWmODnjLE+TvB1EJptUNvH2wEL/jUKzzw+CL/FXcHhv3LR\n1LLxl0qtQUJyARKSCyAR22LiWH9MGOsHZyeRkVtNTBnlsSdGdSOYP1FUg5PFtahuUrWpw2MYjPZ2\nREyAC0Il9nc0imOOeafr6puRklqCpFNFyMiqaKcGgxFDJYid2BfDh9Dcze4wx+uE9Dy6Tu6cXKnB\n8cLrOJxbjSvX296dBXQDOmN8HDHGxwn+jqJeFeTf6jUib1QiIakAfybkIb+wts3jDMNgQLArwkd6\nY3SYNwX5ZoI2qGoHBfa9h1rL4mKFDElFNThZXIdaRdtgHgB8HUSICXDBBD8XOPTQdBFzeSO+Ecyf\nPF2Mi1kVaO/P2NlJhEnj/TFxnD/cXG2M0Mrey1yuE3J30XXSs3JrGhGXW41jBdchU6rbreNpa4UI\nHyeM9LBHiKsteCY+UHEn18i1/BocScjD8eQCNLYz6AUwCAlyaQnyvaif78UosG8HBfamTdqgwFlp\nPc5J65EulaFRrWm3np2AhzE+jogJcEWQs3WPj8z01jdilmVRKpXh3MVypJ4t7TCYZxgGI4Z6IDoq\nAMOHSsC1gA1h7obeep2Qe4uuk7tDqdHidGkdkopqcLq0Ds3qttMyAd3utkPFdhjuYY/hEnuTTJ/Z\nE9eIUqlB8pkiHEnIQ2Z2FYD2w7YAXycMHSzGsEESBAe50HTLXoR2niUmr16hRkaFDOfKZTgnrUdZ\ng6LDuk5CPsK9HTHGxxGD3OzANfERmHulQa7ExUsVSL8oRfrFclS2s4hYh0FIPxdEjvJB+Ei6NUsI\n6d0EXA7G+DhhjI8TFGot0srqkFRci1MldWi6aVCoUa3ByZJanCzRTVnxshNiqNgOA11tMcDNFu5m\nklFGIOAiKtIPUZF+qKltxqm0Ypw8U4yMrEqDAZ7cghrkFtTgwK/ZEAh4GBjiimGDJBg6SAwfL/te\nNYWJ3D4K7Mkd07IsiuubkV0lx6WqBmRVyfW7DnbEzVqACG9HRHg7YoCbrdlmP7gVNbXNuHy1Cpev\nVCMrpwpXcq+3Oyp/Q0g/V0SM8kF4mBdcnK3vYUsJIeTesOJxEOHjhAgfJyg1WpyT1uN0aR3Syur1\nm1/dUCJrRomsGYeu6nLEu4oEGOBmgwGuthjoZgtfB1GvHzhychTigYmBeGBiIOrqm3H6bClOninG\nhcxyaLWt7xdKpRrnLkj16TRtbQQIDnRBcJALQoJcEejvTCmOzRT9r5JbomVZlDUocK2mEXk1TbhW\n24ScanmH8yFvsOJyMERsh+ESe4RK7OFtZ2XRowdKpQaFxXW4mncdl69UI/tKFcorOxqR1xEKeRgy\nQIzQIRKEDfOgYJ4QYlEEXA7u83LEfV6OYFkWxTIFzpbVI01aj4sVMn12nRuqmpQ4XqjE8cIa3fkc\nDnwdRQh0EqGvkzUCnKzh5yDqtRnWHOyFiI4KQHRUAOSNSmRmVSI9oxzpGVJIyxsM6jbIlUg7X4a0\n82UAdNM2/fs4IqivM/z6OCLA1xF9vB0o/bEZoP9B0i6WZVHVpEJJfTOK6ptRWN+MvNom5NU2djjf\n8WY8hkGAk7V+/mOIiw0EFjjfm2VZ1NQ1I7+gFvlFdcgvrEVBUS2KS2WdjsbrMOjr74Rhg3TBfL++\nLrQrLCGEQBeY+tgL4WMvxNRgdyg1WlyqbEBGpe6ucXZ1Q5v3KqVWiyvX5QbZdzgMA287IbztrdDH\nXgQfeyG87XXHQl7vmaNuYy3AqBFeGDXCCwBQXilH+kUpzmeUIyOrAg1yw7sbLMvqp+7cwDAMPCV2\n8OvjAP8+jvDxcoCnhy3Ebrb03tOLdDuw37x5Mz788EOUlZVh4MCB2LhxI8aMGdNh/YyMDLz66qs4\nffo0XFxc8K9//Qv/+c9/DOokJCRgyZIlyMzMhJeXF5YuXYqXX37ZoM7+/fvx1ltv4dq1awgMDMSa\nNWvw6KOP3uLLJO3RaFlUNipR0ahEeYMCFY1KFNc3o7hegRJZc7t55Dtib8VDfxcbDHCzRX9XWwQ6\nWffaUZDbIW9UQlohR5lUhuJSGcrKZSiVNqC0TKbPSdwVPo+LvgFOCAl0Qf9+bggOcoG9nektBiOE\nEFMj4HL0OfIB3ftbbm0jsqrkuFTZgOxqeZupO4DuLnRhfRMK65uQDMP0ku7WAnjaCSGxFUBsYwWJ\njQBiWyuIbQRwsOKZ9F1nsZsNYu/vi9j7+4JlWZSVNyA7pwqXr+ruEBeVyPD3Rbgsy6KkrB4lZfVI\nOlWkL2cYBhJ3W3h52MFDYgtPiR3cXW3g5moNd1cbCAS95wOQJehWYL93714sWrQIW7duRWRkJD77\n7DNMnjwZWVlZ8Pb2blNfJpMhOjoaUVFRSEtLQ1ZWFmbOnAlbW1ssXrwYAJCfn48pU6bgpZdewrff\nfovExETMmzcP7u7umDZtGgDg5MmTePLJJ7F69WpMmzYN+/fvx/Tp05GcnIyRI0f24K/BvLAsiwal\nBtebVahpUqGmWYWaZjWuN6lQ26xCZaMKFXIFqppU0N5GUiRHKz76Oong72SNvo4i9HW2hqeteU6t\n0WpZyBuVqKltRl29ArV1zaisbkRllRxV1Y0tPzd2O3hvxcBDbAu/Pg4IDtTNefT3cwS/F40QEUKI\nqeJyGAQ52yDI2QaP9HMHANQp1MitacQ1/VcTSmUKsB1kmaloGfhC2827YcXlwNVaACchH84iHlxE\nup9dRHw4W/Ph3PKzKbgxEu8pscP94/wB6KbmXLl2Hbn5NcgvqkVeQS1KpQ1oL+OO7oOBbsCqPfZ2\nVnBztYG7qzVcnK3h6CCEo70VHB2EcHIUwdFBCAd7K9pD5R7pVrrL0aNHY9iwYdi6dau+rF+/fpg+\nfTrWrl3bpv6WLVvwxhtvoKKiAgKBblX62rVrsXXrVhQV6T4FLlu2DAcPHsTly5f1582ePRuXLl1C\nUlISAODJJ59ETU0NDh8+rK8THR0Nd3d3fPvttwb/pjmmu9SyLBRqLRpVGjSoNJAp1JCrNJApNWhQ\nqtGg1ECmVEOu1JXVK9SoadYF7yrtnWcxtRPw4G0vhJedFbzthfB3FKGvozWcTKSzulUqtQZJSWfQ\nrNAgMLA/5E0qyOVKNDap0CBXoa6uGbX1rQF8XZ0CdbJmgwVJt0Mo5MHX2wG+Po7w93WEr48DfH0c\nIBL2zt+jJaA0hqQ76Drp/ZpUGhS2TDktqm9Gccv3sgbFbQ18/Z2qqRHWXAY+7i6ws+LCVsCDvYAL\nOwEPtgIe7Ky4sBfwYCPgwprPhYjHgTWfCyGPCysuc08HzJqb1SgsrkNeYS0Ki+tQUipDiVSG6uuN\nPfDsDBxagn1HByHsbAWwtRHAxpqv+27Dh421rszWRgBraz5sbfgQCfkW8YHgnqa7VKlUSEtLw9Kl\nSw3KY2JikJyc3O45KSkpGDt2rD6oB4DY2Fi89dZbKCgogK+vL1JSUhATE2NwXmxsLHbv3g2NRgMu\nl4uTJ09iwYIFbep89tln3X6Bd0uzWoNyuRJqLQuVRguVloVay0Kp0UL9t59VWhYqDQu1VguFRotm\n9c1fGjS3lDWpdI83qTVoVmvbLAS6G5yFfLjbCFq+rOBlawWvljmGPbUpFMuyYFlAo9WC1eq+a7Xs\nTT/rgm61Wgu1WgulqvVntVoLlUqrf1yl0kKt0UKl1kKpUEOh1KBZoYZCoYGi5VihVKO5WfddodCg\nsVEFeaMSSpUGcrmug7KxKemR13aDgM+Fm6sNPCU3blfa6UdIHB3M824GIYT0diI+F8EuNgh2Mdzc\nSaXRorRBAWmDAuVy3XTVcrkS5XIlyhoUBmk3O6PSAnVaFqraWw+OOQwDEY8DEY8LEb/1u4DLAZ/D\nQMjjgM/RHQu4TMv3v/3M0f3M5TDgcRjdd0b3ncu0LXP1soe7tz0ib9RnGKiVGkgr5CiVylBSJoO0\nogGVVXJUVjWi6npjNwe/WNTVN6OuvhkFRV3Xvhmfx4WVFRdCIQ9CKx6srHiwEnD1PwuFLT8LuLCy\n4kEg4ILP44DP54LH5YDH44DP54DP44LHaz3mcVvK+VxwW35HnL99cbkMOAwDDpcBl8MBw8Dk38+7\njNyqqqqg0WggFosNysViMY4cOdLuOVKpFD4+Pm3qsywLqVQKX19fSKVSREdHt6mjVqtRVVUFsVgM\nqVTa7r8rlUq79eLupmOXyrHirytd1mPb/NBhQefn39pp4LIsrFhAoAWsWBYCLSBgWVhpWVhpAaGG\nhVALcFpGJMpbvi7e+GfYG99Z/XetVjc1RdsSjN8I1LUsqwvUWRYaDas7vvG9pbw3sxbx4WAvhIOD\nFRzthXB1sYabi7Xuu6vuZ3sLz/JDCCHmhM/lwNdBBF+HtvuC/H266/VmFaqbdD9XN6lw/aavOxnr\n1rIs5CoN5CoN0HQHT9QDOAwDBgCHARhHPhgnR3D6Oeke1GihVml0A29qLTQ3DczpyjXQtCxkZm4K\nB3Q/t7PRYlchA6sGFGpA0bo/TttzOn+SLv+Nzuoyut8DwIDhABwwQEvAz9z8HQBafm+6H3Xl+qdh\nbjwdgx/njuh+g7pglllxbr6lcbeE+9rg6Kxhd/3fIb0Aq0B9fccbcJHeJygoCMC96UtI70XXiWVz\nBOAoAvxFPMCJB4A2ByTG12XaEldXV3C5XJSXG64eKS8vh0QiafcciUTSbn2GYfTndFSHx+PB1dW1\n0zod/buEEEIIIYRYqi4Dez6fjxEjRiA+Pt6gPD4+HpGRke2eEx4ejsTERCiVraml4uLi4OnpCV9f\nX32dvz9nXFwcwsLCwOVyO6wTHx+PiIiIbrw0QgghhBBCLAjbDXv37mWtrKzYHTt2sFlZWeyCBQtY\nOzs7tqioiGVZll2+fDk7ceJEff26ujrWw8ODfeqpp9iMjAx2//79rL29PbthwwZ9nby8PNbW1pZd\ntGgRm5WVxW7fvp21srJiDxw4oK+TnJzM8vl89v3332ezs7PZd999lxUIBOyZM2e602xCCCGEEEIs\nRrfSXQLA1q1bsW7dOpSVlWHQoEHYuHGjfsR+1qxZOH78OK5du6avn5mZiVdeeQWnT5+Gk5MT5s6d\ni5UrVxo8Z2JiIhYvXozMzEx4enpi+fLlmD17tkGdH3/8EStXrkRubi769u2Ld999F1OnTr3TzzOE\nEEIIIYSYlW4H9oQQQgghhBDT1eUce1ORmJiIqVOnwtvbGxwOB7t37+60fkFBATgcjsEXl8tFXFzc\nPWoxudfee+89jBo1Cg4ODnB3d8cjjzyCzMzMLs/LyMhAVFQUrK2t4ePjg9WrV9+D1hJjuZ3rhPoT\ny7N582YMHToUDg4OcHBwQEREBA4dOtTpOdSXWJZbvUaoHyGA7j2Iw+G02afp7263P+k16S4bGhow\nePBgvPDCC3j++ee7dQ7DMDh8+DCGDBmiL3N2dr5bTSRGdvz4cbz66qsICwsDy7L4z3/+g0mTJiEr\nKwuOjo7tniOTyRAdHY2oqCikpaUhKysLM2fOhK2tLRYvXnyPXwG5F27nOgGoP7E0Pj4+WLduHYKC\ngqDVarFz5048+uijOHv2LAYNGtSmPvUlludWrxGA+hFLl5KSgu3bt2Po0KGd1ruj/sSYE/xvl62t\nLbtr165O6+Tn57MMw7BpaWn3qFXE1DQ0NLBcLpf99ddfO6yzefNm1sHBgVUoFPqyNWvWsN7e3vei\nicQEdOc6of6EsCzLOjs7s59//nm7j1FfQli282uE+hHLVltby/bt25c9duwYGxUVxc6fP7/DunfS\nn/SaqTi367HHHoNYLMaYMWOwf/9+YzeH3EP19fXQarVwcnLqsE5KSgrGjh0LgUCgL4uNjUVpaSkK\nCgruRTOJkXXnOrmB+hPLpNVq8f3330Mul3eYbpn6EsvWnWvkBupHLNO//vUvzJgxA+P2gaXZAAAD\nsUlEQVTHj++y7p30J2Yb2Nva2uKjjz7CDz/8gN9//x0TJ07EE088gT179hi7aeQeWbhwIYYPH47w\n8PAO60ilUojFYoMysVgMlmUhlUrvdhOJCejOdUL9iWXKyMiAnZ0drKysMG/ePBw4cAADBw5sty71\nJZbpVq4R6kcs1/bt25Gbm4s1a9Z0q/6d9Ce9Zo79rXJxcTGYhzR8+HBUV1dj3bp1ePrpp43YMnIv\nvPbaa0hOTkZSUhIYhjF2c4iJ6u51Qv2JZQoJCcH58+dRV1eHffv24fnnn0dCQgIGDBhg7KYRE3Er\n1wj1I5YpJycHb775JpKSksDh3P3xdLMdsW/PqFGjcOXKFWM3g9xlixcvxt69e3H06FH9TscdkUgk\nKC8vNygrLy8HwzCQSCR3s5nEyG7lOmkP9Sfmj8fjISAgAKGhoVi7di2GDRuGDRs2tFuX+hLLdCvX\nSHuoHzF/J0+eRHV1NQYMGAA+nw8+n4+EhAR89tlnEAgEUKlUbc65k/7EogL7c+fOwcPDw9jNIHfR\nwoUL9cFaUFBQl/XDw8ORmJgIpVKpL4uLi4Onp+dtBXukd7jV66Q91J9YHq1WC4VC0e5j1JcQoPNr\npD3Uj5i/adOm4eLFizh//rz+KywsDE899RTOnz8PPp/f5pw76k/ufJ3vvdHQ0MCmp6ez586dY62t\nrdnVq1ez6enpbGFhIcuyLLt8+XJ24sSJ+vq7du1i9+zZw2ZlZbGXL19m169fz1pZWbGbNm0y1ksg\nd9m8efNYe3t79ujRo6xUKtV/NTQ06Ov8/Tqpq6tjPTw82KeeeorNyMhg9+/fz9rb27MbNmwwxksg\n98DtXCfUn1ie5cuXs4mJiWx+fj578eJFdvny5SyXy2UPHz6sf5z6Est2q9cI9SPkhr9nxenJ/qTX\nBPbHjh1jGYZhORyOwdesWbNYlmXZmTNnsgEBAfr6u3btYgcMGMDa2tqyDg4O7MiRI9k9e/YYq/nk\nHmjv+uBwOOw777yjr/P364RlWTYjI4MdP348KxKJWE9PT3b16tX3uunkHrqd64T6E8szc+ZM1s/P\njxUKhaxYLGajo6PZ+Ph4g8epL7Fst3qNUD9CbpgwYYJBYN+T/QnDsizb07cdCCGEEEIIIfeWRc2x\nJ4QQQgghxFxRYE8IIYQQQogZoMCeEEIIIYQQM0CBPSGEEEIIIWaAAntCCCGEEELMAAX2hBBCCCGE\nmAEK7AkhhBBCCDEDFNgTQgghhBBiBiiwJ4QQQgghxAz8PweM7GvHip2HAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from hockey import *\n", + "import thinkplot\n", + "\n", + "suite1 = Hockey('bruins')\n", + "suite1.UpdateSet([0, 2, 8, 4])\n", + "\n", + "suite2 = Hockey('canucks')\n", + "suite2.UpdateSet([1, 3, 1, 0])\n", + "\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(suite1)\n", + "thinkplot.Pmf(suite2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.1: Posterior distribution of the number of goals per\n", + "game.\n", + "\n", + "Figure 7.1 shows the resulting posterior distributions for\n", + "`lam`. Based on the first four games, the most likely values\n", + "for `lam` are 2.6 for the Canucks and 2.9 for the Bruins." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The distribution of goals\n", + "\n", + "To compute the probability that each team wins the next game, we need to\n", + "compute the distribution of goals for each team.\n", + "\n", + "If we knew the value of `lam` exactly, we could use the\n", + "Poisson distribution again. `thinkbayes` provides a method that computes\n", + "a truncated approximation of a Poisson distribution:\n", + "\n", + "```python\n", + "def MakePoissonPmf(lam, high):\n", + " pmf = Pmf()\n", + " for k in xrange(0, high+1):\n", + " p = EvalPoissonPmf(k, lam)\n", + " pmf.Set(k, p)\n", + " pmf.Normalize()\n", + " return pmf\n", + "```\n", + "\n", + "The range of values in the computed Pmf is from 0 to `high`.\n", + "So if the value of `lam` were exactly 3.4, we would compute:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "lam = 3.4\n", + "goal_dist = thinkbayes.MakePoissonPmf(lam, 10)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "I chose the upper bound, 10, because the probability of scoring more\n", + "than 10 goals in a game is quite low.\n", + "\n", + "That’s simple enough so far; the problem is that we don’t know the value\n", + "of `lam` exactly. Instead, we have a distribution of possible\n", + "values for `lam`.\n", + "\n", + "For each value of `lam`, the distribution of goals is\n", + "Poisson. So the overall distribution of goals is a mixture of these\n", + "Poisson distributions, weighted according to the probabilities in the\n", + "distribution of `lam`.\n", + "\n", + "Given the posterior distribution of `lam`, here’s the code\n", + "that makes the distribution of goals:\n", + "\n", + "```python\n", + "def MakeGoalPmf(suite):\n", + " metapmf = thinkbayes.Pmf()\n", + "\n", + " for lam, prob in suite.Items():\n", + " pmf = thinkbayes.MakePoissonPmf(lam, 10)\n", + " metapmf.Set(pmf, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(metapmf)\n", + " return mix\n", + "```\n", + "\n", + "For each value of `lam` we make a Poisson Pmf and add it to\n", + "the meta-Pmf. I call it a meta-Pmf because it is a Pmf that contains\n", + "Pmfs as its values.\n", + "\n", + "Then we use `MakeMixture` to compute the mixture (we saw\n", + "`MakeMixture` in Section [mixture])." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAusAAAEWCAYAAADfHdlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdYFNf+BvB3dmFh6X3pIlJsERXsFWzRNG+Kqb/0qCne\nmGYldjTG3PSoKSbG5CYxiYk3N+XGigVsoGADlN5Xeodly++PJbMSC7u4KML7eR6fOGfnzJyNI747\ne+Z7BJ1OpwMREREREXU6khs9ACIiIiIiujyGdSIiIiKiTophnYiIiIiok2JYJyIiIiLqpBjWiYiI\niIg6KYZ1IiIiIqJOimGdiIiIiKiTMjqsr1+/HoGBgZDL5YiIiMDBgwevuO++ffswffp0eHt7w9bW\nFmFhYfjiiy8u2UcikbT6JZVKce7cufa/GyIiIiKiLsSosL5161bMnTsX0dHRSEpKwsiRIzF16lTk\n5+dfdv/4+HgMGDAA27Ztw5kzZ/Dss89i5syZ+O6771rtJwgCUlJSUFxcjOLiYhQVFSE4OPja3xUR\nERERURcgGLOC6fDhwzFw4EBs3LhRbAsJCcF9992HmJgYo050//33Q6vV4ocffgCgv7MeFRWFkpIS\nuLi4tHP4RERERERdV5t31pubm5GYmIhJkya1ap88eTLi4+ONPlF1dTWcnZ1btel0OkRERMDb2xsT\nJ05EbGys0ccjIiIiIurq2gzrpaWl0Gg0UCgUrdoVCgWKi4uNOsmvv/6KPXv2YNasWWKbl5cXNm7c\niG3btuHnn39GaGgoJkyYgLi4OBPfAhERERFR12TR0SeIi4vDww8/jA8++ADh4eFie0hICEJCQsTt\nYcOGITs7G+vWrcOoUaNaHaOqqqqjh0lERERE1GEcHR3b1a/NO+tubm6QSqVQKpWt2pVKJTw9Pa/a\n9+DBg5g2bRpWrVqFmTNntjmYYcOG4fz5823uR0RERETUHbQZ1i0tLREeHo6dO3e2at+5c+cld8Av\ntn//fkybNg0rVqzAnDlzjBrMiRMn4OXlZdS+RERERERdnVHTYF5++WU8+uijGDJkCEaNGoUNGzag\nqKgIs2fPBgAsXLgQx44dw65duwAAsbGxuP322/H888/jgQceEO/KS6VSuLm5AQDee+89BAQEoF+/\nflCpVPjqq6/wyy+/4KeffrrqWNr7FQJ1TZkV9VgTl4nC2ibU19UDAGxsbQAAEkHAGD9n3NNHgV7O\nNjdymNSJJCQkAAAiIiJu8EioM+N1QsbgdUJtMcdUbqPC+owZM1BeXo6YmBgUFRWhf//++OOPP+Dr\n6wsAKC4uRlZWlrj/l19+iYaGBrz11lt46623xPYePXogMzMTAKBSqTBv3jzk5+dDLpejX79++P33\n3zFlypRrflPU9el0OvwvoxSfHM+HSqsV211kAhpbfq/V6bAvtxz7cssx2NMB9/RWIExhD0EQbsyg\niYiIiExkVJ31G+3iTyW8s04NzRp8mJCL2JxysU1uIcWtTo0Ic5TC0i8UP6YocUJZfUnfIGcb3NNH\ngVG+zpBKGNq7I94JI2PwOiFj8Dqhtpgjw3Z4NRgic8qubMCauEzk1zSKbT2d5Fg4KhBFaacBAAM9\nHTDQ0wHp5fXYllqMg3mV0LZ8Jk2vqMfa+Cx42hbi7t4KTOzpCisLoxbyJSIiIrruGNbpprEjsxQb\nEvOg0himvUwJdMOswX6wspCg6G/7B7nYYP7IQDxW24Sf05TYkVEmTpkprmvC+sRc/Pt0Ie4M8cBt\nQe6wt+JfByIiIupcmE6o02tUa7EhMRe7ssrENiupBC8M8UdUgGub/T3trPBsuD8e7OeFX8+X4Nfz\nJahRqQEAVU1qfHWqED+kFOPWQDfcFaqAh62sw94LERERkSkY1qlTy6tuxJq4TORUNYht/g5yLBjV\nEz0c5SYdy8naEo/c4o17eiuwM6sMP6UqUVKvAqD/QLD93AX893wJxvVwxj29PRHgZNrxiYiIiMyN\nYZ06rb3ZZfgwIReNasO0l6gAVzwf4QdrC2m7jyu3lOLOEA9MC3LHgdxybEtVIqtS/2FAo9NhT3Y5\n9mSXI8LLEff2UaC/ux0ryBAREdENwbBOnY5Ko8Unx/PwR0ap2CaTSPBshB8m9XQ1W3C2kAiIDHDF\n+B4uSCyuxrYUJU5eqBFfTyiqQkJRFUJdbXFPbwVG+DpBwtBORERE1xHDOnUqhTWNWBOXhczKerHN\nx94aC0f1RE+njlnYSBAERHg5IsLLEWlldfgxpRiH8qugg76CTFpZHVbHZcLH3hp391YgKsAFMikr\nyBAREVHHY1inTuNAbgXeO5qDBrVGbBvr74x/DukBuWX7p72YItTVFotH90JBTSN+TlVid1a5WEGm\noKYRHxzLwdenCnFXiAemBrnBTsa/QkRERNRxmDTohlNptNiUlI9fz5eIbZYSATMH+2FqL7cbMl/c\nx94aLwzpgYf7e+OXcxfwW3oJ6pr1HyIqGpux+WQBtp4txtQgN9wV4gE3G1aQISIiIvNjWKcbqri2\nCW/EZ+F8eZ3Y5mlrhYWjAhHk0jHTXkzhLLfEY2E+mNHXE39klOI/aRdQ2qCvINOg1uCnVCV+SbuA\nyAAX3N1bAX8TK9QQERERXQ3DOt0wh/Mr8c6RbNQ2G6a9jPR1wtyhAbCVXZ9pL8aSW0pxd28F7gh2\nR2xOOX5KvYDcan0FGbVOh51ZZdiZVYZh3o64t48n+rrb3eARExERUVfAsE7XnVqrw+bkAvycphTb\nLAQBTw3yxR3B7u2a9lJe2YDPv81CabkKocE18Payh6+XA7y97OGtsIOHuy2kZngo1FIqwaRAN0zo\n6YpjhVXYlqLEmdJa8fUjhVU4UliFvm52uLePAkO8HVlBhoiIiNqNYZ2uqwt1KqyNz0RqmWHai4eN\nDAtGBSLU1bZdx1SW1GHZ2n3IzNYf80xqCc6klrTaRyqVwEthB29Pe3h72sHnryDvaQcHeyuTPyBI\nBAHDfJwwzMcJKaW1+DFFicMFleLrZ0trseJALfwc9BVkInu4wJIVZIiIiMhEDOt03RwrrMK/Dmej\nRqUW24Z5O+KlYQGwt2rfpZhfWI1la/ehvKLhqvtpNFrkF1Yjv7D6ktfsbGX6EO9lCPI+XvbwVNhB\nZkQVmj5udnh9jB1yqxrwU6oSe7PLodbpyz7mVTfivaM5+PepItzZUkHG5jpVtiEiIqKbH8M6dTiN\nVoevThXih5RisU0iCHh8gA/u7u3R7movmdkVWP7mftTUNgEALCwETJ/qg0ED+6KouBb5RdUoLKpB\nYVEtKqquHOZr61Q4l1GGcxllf3tFgIebDby97OHzV5hv+b2LsxwSSetx+zvKMXdYAB65RV9B5vf0\nUrEMZWmDCp8n52Pr2SJMC3LHXSEecJZbtut9ExERUffBsE4dqrRehTcPZeFMiWFet5tchvkje17T\nQ5gp50oR868DqG9oBgBYWVnggbt6ILCHHSIGel+yf0NjMwqKavThvbjG8HtlLZqa1Jfsr6fDhdI6\nXCitQ9Kp4lavyGQW8PbUT6vx8bI3/NfLHm42Mjw50LdVBZmKRv0465o1+CGlGNvTlJjQ0xV391bA\nx9663f8fiIiIqGtjWKcOc6K4GusOZaHqojAc7uWIV4YHwLGd014AIPl0Mda8Gw9Vy3QaO1sZol8Z\ng+rKrCv2kVtbIqinC4J6urRq1+l0KK9o0If3v0J8sT7IXyith65lOsvfqVRqZOdWIju38pLXnByt\nW4X45/xckQMt9hRVobDlW4BmrQ7/yyjFnxllGOGrryDT3jn7RERE1HUxrJPZaXU6fHO6CN+dKYYO\n+rArEQQ80t8L9/X1vKbqKIcT8vH2+sNQq/Wrijo6WGPpvLEI8HdCQsKVw/qVCIIAVxcbuLrYYEA/\nRavXVM0aFCtrxfBe8Nd/i2pQW6e64jErqxpRWdWIs2mtH3IVJALs/RxQ6ipHrUwCS0sJZJZSHMip\nQHx+Jfq72+O+vgqEezrckIWgiIiIqPNhWCezqmhoxrrDWUhW1ohtztaWmDeyJwZ42F/TsWPjsvHB\nJ8fEu91urjZYNn8cvD2v7bhXIrOUwt/XEf6+jpe8VlPbhIKiGhQU1qBQaZheU6SsFT9I/J1Oq4Mq\npwr2OVWQ2ligzNUaJXb6lU8lEgH5hdXYc7YYXnIZJvs549ZQD/h7O0DWyWrOExER0fXDsE5mk6ys\nwbpDWeL8bAAYqHDAayMC4GR9bQ9T/rErHZ9uOS5ueynssWz+WLi73ZipI/Z2VugdbIXewW6t2rVa\n/Tz3v+7AFxTVoEip/+9fFWsEALb1atjW16LRSooyV2tUO1ihSatBU5MGNbUqnCupxcdHcuBS3ohg\niRS+Cjv4eNmLv7w87eHmYnPJQ65ERETUtTCs0zXT6nT44Wwxvj5dBG3LXW8BAh7s74kH+3ld86JA\n235Jwb9/PCVu9/BzwtJ5Y+Hk2PkezJRIBHh62MHTww6Dw7xavdbQ2IyiYv20mvzClrnxLb9qLjSg\nzNUalU5W0LYE8GZLCZQKG5RqdHC+UA2X1Auw0Bjm0MsspfDytDfUjvfWV6rx8rSHna3sur5vIiIi\n6hgM63RNKhub8fbhbCQWG+qXO1pZ4LURPTHI0+Gajq3T6fDV96ew/bdUsS2klyuiXxkDO7ubL4zK\nrS0RGOCMwADnVu06nQ4VlY0oKKpBRkEVduVWILGmAbXNGjSrtdBIBZS6yVHmag2nyia4ljVC1qyF\nqlmDnLxK5ORd+pCro4PhIdc+IW4YO9LfLCu4EhER0fXFsE7tdqakFm/GZ6G0wfCwZX93e8wbEQBX\nm2sL01qtDp9uOY4/92SIbbf09cCCuaMgv8YpNZ2NIAhwcZbDxVmOW/p6YDqAJrUWu7PL8GNKMfIq\nGtDcrEVzswYqe2sUeNrBtqoJ9kW1kDdqLnvMqupGVFU3IuVcCXbty8TBI7l45fkRsGFtdyIiopsK\nwzqZTKvT4adUJb48WShOewGAGX088cgt3pBe4zxqjUaLDz89hn3xOWJbxCBvvPrCCKNWFO0KrCwk\nmBbkjlt7uSEurxI/phQjvaLesIO3PbQhrvCytcIgaxmsqppQpKxFQVENipW1aFa3DvEnThZj4Yo9\nWPzKaHjcoHn+REREZDqGdTJJTZMabx/JxtHCKrHNXmaBV4YHYIj3pVVTTKVq1uDtjw7j6PECsW3M\nCH/MeWYoLCy63zQOiSBgjL8zRvs54eSFGvyYosTxlilHEqmA7EYVshtV6OVsg3uH98YoP2cIAErK\n6lFYVIPjJ4vw247zAIC8girMX7Ybi14aheBerjfwXREREZGxGNbJaGlldVgTl4mSesO0lz5udpg/\nsifcr3HaCwA0NqrxxrtxOHlWKbZNigzErMfCu33VE0EQEKZwQJjCAZkV9fgxRYkDeRXiNxsZFfVY\neygLipOF+EeoByYFukHhbotBAzwRGOCM9ZsSoNFoUVXdiOiYWLw4eyhGDvW7we+KiIiI2tL9blWS\nyXQ6HbanKTFvV1qroP6PUAXeiAoxS1CvrVNh+Zv7WgX16dNCMftxBvW/C3S2wbyRPfHpbf1we7A7\nZBc9OKqsa8LG43l44r+n8M3pIlQ3qRE5OgDL5o8TK8Q0qzV468ND+OnX1Cuu0EpERESdA8M6XVWt\nSo3VcZn49EQ+1C3Bzs5SitdH98LTg3xhYYYgXVnViCVrYpGWXia2PXRvf/zf/QO4kudVeNpZ4dlw\nf2y+8xY81N8L9jLDF2XVTWr8+3QhHv/lFD49nodeQa54Y+kEeCkMC0h9/f1JrN+UcMn8diIiIuo8\nGNbpitLL6/Hin6mIzzeUBgx2scV7U/pguK+TWc5RWlaP6Ji9yM41nOOpRwbh3jv7MqgbydHKAg/3\n98bmO/tj1mA/eFz0TUeTRovt5y5gxYF0uLrZYu3SCegb6i6+vnt/FlauO4DaWtXlDk1EREQ3GMM6\nXUKn0+HX8yV4dVcqiuuaxPY7gj3w5oQQeNpZmeU8RcU1WLxqLwqLawDo52XPeWYobpscbJbjdzfW\nFlLcGeKBz27vj9eG90Sgk434WrKyBqsOZkAmt8DS+WMxfnSA+NrplAtYsGI3ipW1N2DUREREdDUM\n69RKfbMGbx7KwobEXDRr9dNe5BZSLBgZiNnhfq3mR1+L3PwqLI7Zi5KyOgCAVCrBqy+MQOSYALMc\nvzuTSgSMD3DB+1N649FbvMX248XViDmYCZ0gYM4zQ/DQvf3F1wqLazBv2S6cTSu5EUMmIiKiK2BY\nJ1FmRT3m7kjF/twKsS3QyQbvT+mNMf7OV+lpmvMZZYiO2YvKqkYAgMxSikUvjcaIIb5mOwfpv6m4\nv58XHu5vCOwJRVV4Iy4Taq0O997ZF688PwKWFvra9bV1Kixbuw/74nKudEgiIiK6zhjWCTqdDn9m\nlOKVnWkoqGkU26f2csO/JoXC297abOc6nXIBS9fuQ22dfo603NoSS+aNxaABnmY7B7X2UH8vPNDX\nS9w+UliFNw9lQa3VYdQwP6xYNB4O9vqpTWq1Fu99fATf/XSGlWKIiIg6AYb1bq5RrcHbR7Lx/rEc\nqLRaAIC1hQSvDe+JF4b0MNu0FwBISCrEynUH0NioBgDY2cqwYtH4Vg88Usd45BYv3NvH8IEoPr8S\nbx3KgkarQ2iQK9Yumwg/Hwfx9e+3n8G7G49A1cxKMURERDcSw3o3llPVgLk7UrEnu1xs6+Eox7uT\n+2B8gItZz3XwcC7eeDdOLBPo7CTHqsWR6BVgvuk1dGWCIODxAd74R6hCbDuQV4G3j2RDq9NB4W6L\n1a9HYUC/i14/lItlb+xDVXXj5Q5JRERE1wHDeje1O6sML+1IRd5FQWxST1e8Pak3/BzMN+0FAHbG\nZuLt9UegbXlg1cNNHwz9fR3Neh66OkEQ8NRAH9wR7CG2xeaU490jOdDqdLC1kSH6lTGYFBkovp56\nvhQLlu9GfmH1jRgyERFRt8ew3s00qbV472gO3j6SjSaNftqLTCrB3KE9MHdYAKwtzHtJ/Pd/57Dh\n8wQA+qDu5+OAmOhIKNxtzXoeMo4gCJg12BdTe11Uaz27DB8ey4VWp4OFhQSzHw/H4w+GAdDXuVeW\n1GHhij04eUZ5haMSERFRRzE6ma1fvx6BgYGQy+WIiIjAwYMHr7jvvn37MH36dHh7e8PW1hZhYWH4\n4osvLrtfREQE5HI5goKC8PHHH7fvXZBR8qsb8fLOVOzILBXbfO2t8c6k3pgU6GbWc+l0Onz30xl8\n8U2S2BYY4IyViyLh6mJzlZ7U0QRBwHMRfphy0Z/5n5ml2JCYB51OB0EQcOfUUCx4cSRkLaui1tWr\nsGLdfuzel3Wjhk1ERNQtGRXWt27dirlz5yI6OhpJSUkYOXIkpk6divz8/MvuHx8fjwEDBmDbtm04\nc+YMnn32WcycORPfffeduE92djZuu+02jB49GklJSViwYAHmzJmDn3/+2TzvjFrZn1OOuTtSkV3V\nILZF9nDBu5N7I8BJbtZz6XQ6fPFNMr7ffkZs6xPijuULxolVR+jGkggCXhjijwkBrmLb7+kl+Ph4\nvlgFZmi4D2KiI+Hccn1otTp8tOkYtmw9KU5pIiIioo4l6IyozzZ8+HAMHDgQGzduFNtCQkJw3333\nISYmxqgT3X///dBqtfjhhx8AAPPnz8f27duRlpYm7vPMM8/g7NmziIuLa9W3qqpK/L2jI+c5m0Kl\n0eLTE/n4Pd2w2I1MIsGscD9MCXSFIAhmPZ9Wq8OGzxOwe7/hDuygAZ6YN2ckrKwszHquv0tISAAA\nREREdOh5uhKtToe3D2djb47hIeN/hCrw1EAf8dooK69HzNsHkZ1bKe4zPMIXL84a2uF/ph2B1wkZ\ng9cJGYPXCbXFHBm2zTvrzc3NSExMxKRJk1q1T548GfHx8UafqLq6Gs7Ohsofhw8fxuTJk1vtM2XK\nFCQkJECjYbk4cyiqacKru9JaBXVvOyv8a1Iobu3lZvag3qzW4O31h1sF9RFDfLFg7qibMtR1BxJB\nwEvDAjD2okWvfk5TYvPJQvEOu6uLDWKiIxEx0LC40uGEfESvjkVFJSvFEBERdaQ2w3ppaSk0Gg0U\nCkWrdoVCgeLiYqNO8uuvv2LPnj2YNWuW2FZcXHzZY6rVapSWlv79EGSiuLwK/PPPFGRU1Itto/2c\n8d6UPgh0Nv+ccZVKgzfejUP80TyxLXJMAF5+bri4QiZ1TlKJgFeG98RIXyex7ceUYnx9qkjclltb\nYsHcUbh9SojYlpFVjvnLdyEnrxJERETUMTr8dmdcXBwefvhhfPDBBwgPD7/m4/31lRNdnlqrw29F\nahwsVYttUgG4w9sSI2VlOJtcfpXe7dPYpMFXP+YgO9fwwWBEhCuGhQk4ceK42c/XFl4j7TNBpkOh\nVIWz1foqQZ8fy0B+Xg4mKSzFfQaEAvW1jvh1ZxF0OqCurh5z5v2CB6b7IaSX/Y0aervwOiFj8Doh\nY/A6oSsJDg6+5mO0eWfdzc0NUqkUSmXrsm1KpRKenldfIv7gwYOYNm0aVq1ahZkzZ7Z6zdPT87LH\ntLCwgJubeSuTdBflKi3WZ6haBXVnmYDng6wwys3C7NNeAKC+QY3Pv81uFdQjR7njtomekEjMfz7q\nOBYSAY/0kKG3g+HHwo5iNXYrm1vtNzzcFY/e1wNWMv1+TSottvyQg8OJZdd1vERERN1Bm3fWLS0t\nER4ejp07d+Kee+4R23fu3In77rvviv3279+P22+/HStXrsScOXMueX3EiBHYvn17q7YdO3YgIiIC\nUumVp03wIY7LO1JQiU2Hs1ErSGHTUsJ8uI8T5g7tAfsOmi9eXtmA5Wv3obJagK2tfmrNow+EYfq0\n0A45X1v4oI95hIdrsfJABo4X6xdC2l8DBPfyxd29DdPWIiKAUSMrEfP2QZSW6T+o7YmrgtzWA088\nNLBTf1DjdULG4HVCxuB1Qm25+AHT9jKqdOPLL7+MzZs3Y9OmTUhNTcWLL76IoqIizJ49GwCwcOFC\nTJw4Udw/NjYW06ZNw7PPPosHHngASqUSSqWy1Vz02bNno6CgAC+99BJSU1Px2WefYcuWLXjttdeu\n+U11J2qtDp8n5WPFgQzUNusfzJUKAp4Z5Ivo0YEdFtSVJXVYvGov8gr+WtlSwKzHw29YUCfzkUkl\niB7dC2EKw7SWTUn5+E/ahVb79fBzwtqlE9Grp4vY9tuO83jj3Tg0NLa+G09ERETtY1RYnzFjBt59\n913ExMRg0KBBiI+Pxx9//AFfX18A+odFs7IMFUC+/PJLNDQ04K233oK3t7f4a+jQoeI+AQEB+P33\n33HgwAEMGjQIa9aswQcffIDp06eb+S12XaX1KizYcw7bUg3TidxtZHhzQgimhyo6ZNoLAOQXVmPx\nqj1QXqgFoF9k56Vnh2FKVK8OOR9df1YWEiwZ0wv93Q2B/ZMTefj1fEmr/ZydrLFq0XgMj/AV2xKS\nCrF41V6UldeDiIiIro1RddZvNNZZv1RiURXWHcpGjcowPz3CyxGvDA+AQweWSczMrsDyN/ejprYJ\nAGBpIcWrL4zAkMHebfTsePw60vwamjVYsi8dZ0trxbYXIvwxNci91X5arQ7//vEUfv41VWxzdpJj\n4UujEHTRnffOgNcJGYPXCRmD1wm15brUWafO51xZHZbvzxCDukQQ8HiYD5aO7dWhQT3lXCmWrIkV\ng7qVlQUWvzK6UwR16hhySymWjwtCb1dbse3DhFzsyGxdXlUiEfB/MwbguacixPnqFZUNiI6JxdHE\ngus6ZiIioq6EYf0mo9Xp8FFCLjR/LVgjt8SayGDc18cTkg6a9gIAyaeLsfzN/ahv0M9FtrWRYdn8\ncRjQT9FGT7rZ2bQE9mAXQ2B//2gu9mRfWv1l4rhALHltLGxtZAAAlUqNN96Lxy9/pOEm+BKPiIio\n02FYv8n8er4E6S0LHckkErwRFYL+Hh1b3/pwQj5i3j4IVcudfEcHa6xcNB6hQa4del7qPOxkFlg5\nLgiBTvqqPzro8M6RHOzPubRu/4B+CqxZEgWF+1/hXofN3yZj4+ZEqNXa6zhqIiKimx/D+k2krF6F\nLScLxe37+3nC2966Q88ZG5eNdR8cEkPWX0vPB/g7tdGTuhp7KwvERAYjwFEOQP8tz7rD2TiYV3HJ\nvr7eDli7bCJ6BxvWTNi5NxMxbx9EXb3quo2ZiIjoZsewfhP5LCkfDWp9eUZfe+tWda87wh+70vH+\nx0fF6QteCnusjo6Et+fNtVIlmY9DS2D3dzAE9jfjs3A4v/LSfe2tsGzBOIwZ4S+2JZ8uxqKVe6As\nqbtuYyYiIrqZMazfJBKLqrA/13AH87kIP8ikHffHt+2XFHy65bi43cPPCTHRkXB3s71KL+oOnKwt\nsToyGL4t3+podDqsicvEkYJLA7vMUoq5s4dhxvR+YlteQTUWLN+NtHSueEpERNQWhvWbQJNaiw2J\neeJ2VIALwhQOHXIunU6HLVtP4t8/nhLbQnq5YuXC8XBy7NgpN3TzcJbrA7tPS2BXtwT2hKJLV2oT\nBAEP3N0PL84aBgsL/Y+cqupGLFkdi7gjeZfsT0RERAYM6zeB71OKUdRSLtHOUoqnBvq20aN9tFod\nPvnyOLb/ZqiVfUtfDyydPxZ2drIOOSfdvFxtZFgdGQxPWysAQLNWh5gDmUgqrr7s/uNG9cCy+eNg\nZ6u/lprVGvzro0P48ZezrBRDRER0BQzrnVxedSN+PFssbj8e5gMna0uzn0ej0eKDT47izz0ZYlvE\nIG8sfmUM5B1wPuoa3GxkWB0VDI+/SjVqtVh+IAMnL9Rcdv++oe54c9nEVs89fPPjaXzw6TE0tzyP\nQURERAYM652YTqfD+oRcqFvuOvZ2tcWUXm5t9DKdqlmDdR8cwr74HLFtzAh/zJszEjJLqdnPR12L\nwtYKa6JC4CZvCewaLZbtS8fpKwR2T4Ud3lgyAf37eIhtsQezsXztftTWslIMERHRxRjWO7HYnHLx\nDqVEEPB8hL/ZFz5qbFRj9b8O4uhxwyqTkyIDW80vJmqLp50V1kQFw1Wu/xamSaPFsv0ZSCmtvez+\ndnYyvP72J044AAAgAElEQVTaGESN6Sm2nU0rwfzlu1FYfPmQT0RE1B0xjXVSNU1qfHbCEKDvCvFA\noLONWc9RW6fC8jf34eRZpdg2fVooZj8eLi4ZT2Qsb3trrI4MgXPLtKkGtQZLYtORVnb5Mo2WFlI8\n/3QEHpkxQGwrUtZgwfLdOJNacl3GTERE1NkxrHdSX54sRGVTMwDATS7Dw/29zHr8yqpGLFkT26p8\n3kP39sf/3T8Agpnv3lP34etgjZjIYDhaWQAA6tUaLIk9j/Ty+svuLwgC7r69N159YQQsLfRTrmrr\nVFi2dh/2Hsy+XsMmIiLqtBjWO6HU0lr8L6NU3J452BdyM84dLy2rR3TMXmTnGupiP/XIINx7Z18G\ndbpmPRzlWB0ZAnuZPrDXNmsQHXsemRWXD+wAMHKoH1YtHg9Hh5ba7S0PPH+77TQrxRARUbfGsN7J\naLQ6fJSQBx30AWWItyNG+jqZ7fhFxTVYvGqvOC9YEAS88PQQ3DY52GznIApwkmN1ZLAY2GtUaiyO\nPY/syoYr9gnu5Yq1yybAz8dRbPvhP2fx9vrDUDWzUgwREXVPDOudzH/PX0Bmpf4OpEwqwezBfma7\n252bX4XFMXtR0jKHWCqV4NUXRiBqbM82ehKZLtDZBivHB8G25Vuh6iY1Fu89j9yqKwd2DzdbrFkS\nhUEDPMW2uCN5WLomFpVVjR0+ZiIios6GYb0TKa1X4etTReL2Q/284GlnZZZjn88oQ3TMXjHwyCyl\nWDh3FEYM6ZgFlogAINjFFivHB0PeMh+9sqkZi/aeR371lYO3jdwSi14ajSlRvcS2tPQyLFi+G7n5\nl66QSkRE1JUxrHciHx/PQ0PLwjD+DnJMD/Voo4dxTqdcwNK1+1Bbp69hLbe2xJJ5YzE4zLwPrRJd\nTqirLVaODxIDe0VjMxbtPYfCmisHdqlUgpmPDcYTDw0EoP9m6UJpHRat3IPkM8or9iMiIupqGNY7\niaOFVYjPNzzw+VyEHyyl1/7Hk5BUiJXrDqCxUQ0AsLOVYfnCcegb6n7NxyYyVh83Oywb2wtWLdd0\nWYP+DntxbdMV+wiCgDtuDcHCuaNg9Vd1mYZmrHhzP3bszbwu4yYiIrrRGNY7gUa1FhsT88TtiT1d\ncYuH/VV6GOfg4Vy88W6cuIy7s5McqxZHIqinyzUfm8hU/T3ssXRsEGQtgb2kXoVFe8/jQt3VVy0d\nMtgbMYsj4eIsB6Bf2XfjFwn48ttkaLWsFENERF0bw3onsPVMEZR1+juM9jILPBHmc83H3BmbibfX\nHxHDjIebLWKiI+Hv69hGT6KOE6awx9IxvSCT6H/0KOuasHDPOZTWXz2wBwY4Y+3SCejpb6iM9J8/\n0vDm+/Hit0ZERERdEcP6DZZT1YBtqYY5uE+G+cCpZQXI9vrv/85hw+cJQEv5R19vB8RER8LTw+6a\njktkDgM9HbB4TCAsW1bJLa5rwqK951HWRmB3dbHBquhIRAzyFtuOHi9A9Oq9KK+4coUZIiKimxnD\n+g2k0+nwUUIuNC2LvvR1s8PEQNdrOt53P53BF98kiW2BAc5YtTgSri421zxeInOJ8HLEwlGBsGgp\nS1pQ04hFe8+jsrH5qv3k1pZY8OIo3HFriNiWmV2Bect2ISun8io9iYiIbk4M6zfQrqwynCmpBQBI\nBQHPR/hD0s6a6jqdDl98k4zvt58R23qHuGH5gnFwsDdP+Ucicxrm44QFowIhbbnm82sasXBP24Fd\nIhHwxEMDMevxcHENgvKKBiyO2YOEpMIOHzcREdH1xLB+g1Q3qfF5coG4PT3UAwFO8nYdS6vVYf2m\nBPz65zmxbdAATyx9bSxsbWTXPFaijjLC1wnzRvYUP6TmVjcgOvY8qpvanoc+JaoXXn91DOQt08Ya\nG9VY804cfttxvkPHTEREdD0xrN8gm5MLxEDibiPDQ/3bV/O8Wa3B2+sPY/f+LLFteIQvFlxU7o6o\nMxvt54xXhweIgT2rUh/Ya4wI7ANv8cSaJVFwd7UFoP+GadPXJ/DpluPQaLQdOm4iIqLrgWH9Bjhb\nUos/M0vF7WfD/WDdsmCMKVQqDd54Nw7xRw1lHyPHBOCV54fDsh3HI7pRxvVwwUvDekBoWQApo6Ie\nr+9LR51K02Zff19HrF02AcEXPe/xx650rHknDg1tTKkhIiLq7BjWrzO1VocPE3LF7eE+Thjm43SV\nHpdX39CMFev248TJYrHttknBeP6pIZCaYTEloustKsAV/xzqL26fL6/Dkn3nUd/cdmB3crTGykXj\nMWKIn9h2/GQRFq7Yg5LSug4ZLxER0fXAVHedbU9TIqdKX2bOSirBrMF+bfS4VFOTGsvW7sPZtBKx\n7d47++LJRwZCImnfA6pEncHkQDe8EGEI7KlldVi6Lx0NRgR2mUyKV54fjnvu6CO25eZXYf7y3UjP\nLO+Q8RIREXU0hvXrSFnXhG9OF4nbD/f3hoet6Q+Afv39qVbh49EHwvDQvf3FyhhEN7OpQe6YfdGH\n2LOltVi2PwON6rYDu0Qi4OH7bsELTxu+YaqsakR0zF4cTsjvsDETERF1FIb16+jjxDw0tTz0FuAo\nx12hHiYfI/mMEr/tNFS7eOKhgZg+LdRsYyTqDO4I8cDMQYbAfrqkBisOZKBJbdxDo1Fje2LpvLGw\na/kwrGrW4M33D+Hn31Kha1nXgIiI6GbAsH6dHM6vxJHCKnH7hSH+sDBxykpdvQoffnpM3I4Y6I3b\npwSbbYxEncldoR54MsxX3E5W1mDVwQyojKzy0r+PB9YsiYJCXLlXh6+2nsT2Pwqh0TCwExHRzYFh\n/TpoaNZg43FDxZZbe7mhj5vdVXpc3qavk1BWXg8AsLOV4dknIzj1hbq0e/oo8NgAH3H7eHE1Yg5m\nGh3YfbwcsHbpBPQJcRfbEpIr8OX32airV5l9vERERObGsH4dfHumCCUtwcDBygKPXxQ+jHUksQCx\nB7PF7dlPhMPZydpcQyTqtGb09Wy1DkFCURXWxmdBrTXu7riDvRWWzh+LcSN7iG0Z2XVYsjoWlVWN\n5h4uERGRWTGsd7Dsygb8nHZB3H5qoC/sTVysqLKqERs+TxC3x4zwx8ihpleRIbpZPdTPC/f39RS3\nDxdUYm18ptGBXWYpxT9nDcUDd/cX27JyK7F41V4oS1jakYiIOi+G9Q6k1elrqmtbHmjr726PCQEu\nJh1Dp9Ph482JqK5pAgA4O8nxzP8NNvtYiTozQRDwf7d4457eCrEtPr8S/zqcBY2RgV0QBMyY3hf/\nmOaNv2aPFSlrsGjlHuTmV129MxER0Q3CsN6BdmaWIaW0FgBgIQh4PsLP5DnmsXE5OJJYIG6/8PQQ\n2NmZXu6R6GYnCAKeCPPB9BBDFaX9uRV450i2+IHYGBFhLnjwH36wsND/+KuobEB0zF6kni9toycR\nEdH1Z3RYX79+PQIDAyGXyxEREYGDBw9ecd+mpiY88cQTCAsLg0wmQ1RU1CX77Nu3DxKJpNUvqVSK\nc+fOte+ddDKVjc34ItkQsu/po4C/o9ykY5SW1eOzLSfE7SlRvTBogOdVehB1bYIg4OlBvrgj2BDY\n9+aU472jOSYF9n6hjnj91bGwttZPSautU2HZ2tYrAhMREXUGRoX1rVu3Yu7cuYiOjkZSUhJGjhyJ\nqVOnIj//8ouMaDQayOVyzJkzB7fffvsVjysIAlJSUlBcXIzi4mIUFRUhOLhrlCL8IrkANSo1AEBh\na4UZfb3a6NGaVqvDh58eQ0Njs/4YHnZ47IEws4+T6GYjCAJmDfbF1F6GCi+7ssrw4bFckwL7LX09\nsGLheNjbWQEAVCo1Vr9zEAcP55p9zERERO1lVFh/55138OSTT+LJJ59EaGgo3n//fXh5eWHDhg2X\n3d/Gxgbr16/H008/DR+fq1c+cXd3h4eHh/irK5QiPHWhBruyysTtZ8P9YG1h2oyj/+1Ox8mzSgD6\ncPLPmUPFu4BE3Z0gCHguwg+TA93Etj8zS7EhMc+kRY+Cerpg9euRcHWxAQBoNFq8vf4I/rc73exj\nJiIiao82E2RzczMSExMxadKkVu2TJ09GfHz8NZ1cp9MhIiIC3t7emDhxImJjY6/peJ1Bs0aLjxIM\nd+ZG+jphiLejSccoLK7Blu9Oitt3TQtFnxC3q/Qg6n4kgoA5Q/wRFeAqtv2eXoJPTuSbFNh9vByw\n5vUo+Ho7tLTo8MmXx/H99rNc7ZSIiG64Nm/VlpaWQqPRQKFQtGpXKBTYvXt3u0/s5eWFjRs3YsiQ\nIVCpVNiyZQsmTJiA/fv3Y9SoUVfsl5CQcMXXOoPdymakFeunv1hJgWESLRISKozur9Xq8MlXmaio\nbAAAKNytENKjsdO/786E/6+6lzFSHQpkzThRoQEAfHciG4X5+bjdy+Kq39T9/Tq5Z5oTvvy+CgVF\n+r97m746ipTUDEyN8oTExNWGqevgzxMyBq8TuhJzTO++YfMqQkJCEBISIm4PGzYM2dnZWLdu3VXD\nemdW1qTFrgtqcXuywhJOMtP+kd9/uAR5hfqwIJUA993hK1atIKJLSQQB9/tZQqsDkiv1gX1/iRpS\nAZjqefXAfjFbGws89VAA/r0tFxnZ+trr8cfKUN+gwd3TfCCVMrATEdH112ZYd3Nzg1QqhVKpbNWu\nVCrh6WneyiTDhg3D1q1br7pPRESEWc9pLjqdDkv3Z0Amr4IMQKCTDV6c3BtSE+7IZeVU4siJbNja\n6ufPPnRvf9xxW98OGnHX89edjc56jVDHCg/XYW18JuLzKwEAR+qAntZe+L9bvFvt19Z1MmRIBN7b\neASHjukfoD+fpcKug4149YURsDJxQTO6efHnCRmD1wm1parq2tfxaPOWraWlJcLDw7Fz585W7Tt3\n7jT7HfATJ07Ay8u0qimdRXx+JRKL9H8gAgS8MMTfpKCuatbg3Y1HoNFoAQAhvVzxj9t6d8hYiboi\nC4mAeSN6YthFz4h8d6YI354uMuk4MkspXnl+BCZFBopticlFWL5uP2rrVGYbLxERkTGMml/x8ssv\nY/Pmzdi0aRNSU1Px4osvoqioCLNnzwYALFy4EBMnTmzVJyUlBUlJSSgtLUVtbS2Sk5ORnJwsvv7e\ne+/hP//5D9LT03H27FksXLgQv/zyC+bMmWPGt3d9NDRr8MlxQxnLqUFuCHW1NekY3207jbwCfdj/\na2l0qZTTX4hMYSmVYMGoQER4GQL716cL8cNZ0+qnSyQCZj8ejnvu6CO2pZ4rxeurY1FR2Wiu4RIR\nEbXJqO90Z8yYgfLycsTExKCoqAj9+/fHH3/8AV9fXwBAcXExsrKyWvWZNm0acnMNVVEGDRoEQRCg\n0ejnlKpUKsybNw/5+fmQy+Xo168ffv/9d0yZMsVc7+26+fpUIUob9HfcnKws8dgA7zZ6tHY2rQTb\nfzcsBvXoAwPg7Wlv1jESdRcyqQSLRwdi5YEMHC+uBgBsPlkAqUTA3b0VbfQ2EAQBD993C+ztZNj8\nrf5GQ05eJRat3I2l88fB08OuQ8ZPRER0MUF3E9Qmu3i+j6OjaWUQO1pGRT3m7kgVF2N5dXgAIi8q\nJdeWhsZmvLx4B5Ql+gfaBvRVYMm8saw+0Q6cO0gXa1JrsfxAOpKVNWLbrMF+8K7W30Qw5TrZezAb\nH356TCzl6ORojSWvjUWAv5N5B02dBn+ekDF4nVBbzJFhOc/iGmh1ularJoYp7DG+h4tJx/jy25Ni\nULeRW+KFZ4YwqBOZgZWFBEvG9EJ/d8O3VB8fz0N8qfoqvS4vcnQAFrw4CpYWUgBAZVUjomP2IuVc\nqdnGS0REdDkM69fgfxmlOFeuD9qWEgHPhfubtALriZPF2LE3Q9x++tFBcHO1Mfs4iborawsplo3t\nhT5uhikrPxc043CZ6YF9yGBvLJk3FnJrSwBAfUMzlq/dh8Rk0x5gJSIiMgXDejtVNjZjc3KBuH1v\nH0/4Olgb3b+2VoUPPzsmbg+P8MW4kT3MOkYiAuSWUqwYF9Tqoe9t+c3477kLJh+rX293rFw0Ho4t\nf9dVzRqseecg9sXlmG28REREF2NYb6fPTuSjrln/sKyXnRVm9DWt5vynXx0XVyl1dLDGrMcHm3RX\nnoiMZ9MS2INdDIF94/E8fH2qEKY+thMY4IyY6Eh4uOmPpdXq8N7HR/DbjvNmHTMRERHAsN4uycpq\n7M0pF7efC/eHzIQyi3FH8nDgkKFSzrNPhIt36oioY9jJLLByXBD8bQx/V789U4T1iXnicyfG8va0\nx+rXo+Dn4yC2bfr6BL776bTJ4Z+IiOhqGNZNpNJo8VFCnrg91t8Zg70crtKjtfLKBny8OVHcjhwT\ngKHhPmYdIxFdnr2VBWYGyhBqb/jR93t6CdbGZ0HVsiCZsVyc5YhZHIWQXobqT99vP4tPt5yAVsvA\nTkRE5sGwbqJtKUoU1OgXRbGxkOKZQX5G99XpdNiwKUFcBdHN1QZPPjywQ8ZJRJdnJRXwRE8ZIi+q\n3HQwrwLL92egoWVqm7Hs7GRYNn8cBg0wTIP73+50vLPhMJrVph2LiIjochjWTVBY04jvL1oJ8dEB\n3nCRWxrdf/f+rFaVI154eghsbWRmHSMRtU0qCHh5eADuCvEQ25KU1Viw5xwqG5tNOpa1tQUWzB2F\nUcMMH9zjjuRhzTtxaGw0veoMERHRxRjWjaTT6bAhMQ8qrf6r8iBnG9wW7G50f2VJHT7/d5K4fduk\nYAzoZ/xqikRkXhJBwDODfPHoLYYVh9Mr6jFv9zko65pMOpalhRQvPTsct04IEtuSThVj+Zv7UFur\nMtuYiYio+2FYN9KB3Apx6XKJIOCFIf6QGFm9RavV4YNPjop32bw97fHIjFs6bKxEZBxBEHB/Py/M\nGdJD/PtcUNOI13alIaeqwaRjSSQCnnl0EGZM7yu2paWXYXHMHpSV15t13ERE1H0wrBuhTqXBpyfy\nxe3bgtxblYBry69/nsPZtBIA+nDw4qyhsLKyMPs4iah9bu3lhgUje8KyZfXgsoZmzN99DimltSYd\nRxAEPHB3fzz1yCCxLa+gGotX7UVhcY1Zx0xERN0Dw7oRvjpViPKWeawu1pb4v4u+Nm9Lbn4V/v3D\naXH73jv7IPii6hFE1DmM8nPG8nHBkFtIAQA1KjUW7T2PY4VVJh/rtsnBeHHWMEhawv+F0josXrUX\nmdkVZh0zERF1fQzrbThXVoff0kvE7WcG+8JWJjWqr1qtxfsfHxWrQvT0d8K9d/XpkHES0bULU9jj\njagQOLZ886XSaLHiQAb2ZJeZfKxxo3pg4UujIbPU/7yoqm7E66tjcSa15OodiYiILsKwfhVanQ4f\nJeSKC6aEezpgjJ+z0f1//CUFmTn6O2mWFlK8OHsYLC2MC/pEdGMEudhg3cRQKGytAOh/DvzrcDa2\npylNPlZ4mBeWzh8nVn1qaGzGijf349jxQrOOmYiIui6G9av47XwJ0iv0D4bJJBLMDveDYORDpemZ\n5fjhP2fF7Qfv7Q9/X8cOGScRmZePvTXWTQhBgKNcbPv0RD6+TC4weYXSPiFuWLloPJwc9asUN6s1\neOO9OOw9mG3OIRMRURfFsH4FZfUqfHnScPdrRl9PeNtbG9VXpdLgvY+Piv+o9wlxx523hnTIOImo\nY7jayLB2Qgj6udmJbd+nFOP9Y7nQmLhCaYC/E1ZHR0Hhrn8wXafTV4j67//OmXXMRETU9TCsX8Fn\nSfloaJlr7mNvjXv6GF8T/esfTqGgSF/m0crKAnNmDhEfNCOim4edzAIrxwdjmLfhW7EdmaVYE5cJ\nlUZr0rE8FXaIeT0KPfycxLYvvknCv384ZfLdeiIi6j4Y1i/jeFE19ucaqjY8H+EHmdS4/1WnUy7g\n1z8Nd8sefzAMnh52V+lBRJ2ZlYUEi0f3woQAQxWnQwWVeD02HXUqjUnHcnGSY+Wi8egd4ia2bftv\nCjZuToTWxLv1RETUPTCs/02TWov1ibnidlSAC8IUDkb1rW9oxgefHBO3Bw3wxOTIQLOPkYiuL6lE\nwEvDeuDu3oZv2E6X1GDBnnOobCnraiw7WxmWvjYW4WFeYtvOvZn410eHoGo2LfwTEVHXx7D+Nz+k\nFKOoVr/UuJ2lFE8N9DW67+dfJ6GkrE7f11aG558aYvQDqUTUuQmCgKcG+uKJMB+xLbOyHq/uSkNx\ny88MY1lZWWD+i6MwbmQPse3QsXys/tdBNJgY/omIqGtjWL9IfnUjfkwpFrcfD/OBk7WlUX2PHS/E\nngNZ4vbMxwbDxVl+lR5EdDO6t48nXhzaA5KWD+JFtU14dVcaMlsqRxnLwkKCOTOH4rZJwWLbybNK\nLH1jH6prTAv/RETUdTGst9DpdFifmIvmlnmjoa62mNLLrY1eetU1TVj/eYK4PWqYH0YP9++QcRLR\njTc50A3RowMhk+h/hFY0NmP+7nM4daHGpONIJAKefGQgHrynv9iWnlmO6Ji9KCs3LfwTEVHXxLDe\nIjanHMlK/T+0EkHACxH+4p2zq9HpdNj4RSKqqhsBAM6Ocsx8bHCHjpWIbrxhPk5YOT4Iti0rlNar\nNVgSm47D+ZUmHUcQBNx3V9+Wnxv6nzn5hdVYuHKPWFWKiIi6L4Z1ADVNanx2okDcvjPYHYHONkb1\n3R+fi8MJ+eL2c09HwN7OyuxjJKLOp7+HPd6ICoFzy3Q5lVaLmLhM7MgsNflYt04IwsvPDYO0pfJU\naVk9Fq3ci/SscrOOmYiIbi4M6wC2nCpEZZP+oS43uQyP3OJtVL+y8np89tUJcXtSZGCrCg9E1PUF\nOtvgrYmh8Gr5kK7V6fDe0ZxWz78Ya/Rwfyx6aTRkMgsAQE1tE5asicWpsxfMOmYiIrp5dPuwnlZW\nhz/SDXfBZg72hbzla+2r0el0+PCzBNTVqwAAHm62ePzBsA4bJxF1Xp52VnhrYigCnQzfyH2RXIBN\nSfnQmrjg0aABnli+YBzsbGUAgMZGNVa+tb/VN3hERNR9dOuwrtHq8OGxXOig/8d0iLcjRvo6tdFL\n7889GUg+/dedMwH/nDUUciMrxxBR1+NkbYk3okJwi4e92PZTqhLvHsmB2sQFj0KDXLFqcSScnfQV\npdRqLdZ9cAi792W10ZOIiLqabh3W/3v+AjIr9RUXZFIJZg/2M6ouelFxDTZ/e1LcvmtqCPqGunfY\nOIno5mArk2LFuKBWH/p3Z5ch5mAGGtVak47l7+uI1a9HwVOhXwFZp9Pho03H8PNvqWYdMxERdW7d\nNqyX1qvw9akicfvBfl7wNOLBUK1Wh/c/OQqVSg0A8PNxwIP39m+jFxF1FzKpBAtGBmJKoKH069HC\nKrweex41TWqTjqVwt8Xq6Cj09DeE/6+2nsSW75KhM3F6DRER3Zy6bVj/+HgeGtT6pb39HKzxj1AP\no/pt/z0NaellAPQ1kv85axhkRsxxJ6LuQyoRMGeIP+7v6ym2nS2txYI951DW8pyLsZwcrbFi0fhW\n395t/z0N6zclQKMx7W49ERHdfLplWD9WWIX4i2ohPx/hD0tp2/8rsnMr8e220+L2jOn90CvAuUPG\nSEQ3N0EQ8OgAHzwzyFdsy65qwGu7z6GgptGkY9nayLDktbGIGGSoVLV7fxbe+vAQVM0as42ZiIg6\nn24X1hvVWmxIzBO3J/Z0bfVA2JWomjV47+Oj4p2soEAX3HNH7w4bJxF1DdNDFXh1eACkLc/DKOua\n8NquNKSbuEKpTCbF/H+OxPjRAWLbkcQCrHrrAOobms05ZCIi6kS6XVj//mwRlHVNAAB7mQWeCPMx\nrt/PZ5GTp78bb2khxT9nDhUXLyEiuprIAFcsGdsLspafGVVNaizYcw7JStNWKJVKJXjh6SG449YQ\nse10ygUsWRMrrqJMRERdS7dKmzlVDdiWohS3nwzzgZMR5RZTz5e2qsDwfw8MgK+3Q4eMkYi6pggv\nR6yODIZ9y4JHDWoNlu5LR1xehUnHkUgEPP5gGB657xaxLTO7AotW7kVJaZ1Zx0xERDdetwnrOp0O\n6xPyoG6poNDXzQ4TA13b7NfYqMb7nxwVKy/07+OBaRODOnSsRNQ19XGzw9oJIXCT6xc8atbq8EZ8\nFv5ILzHpOIIg4O47+mD2ExFiudkiZQ0WrtyD3Pwqs4+biIhunG4T1ndnl+N0SQ0AQCIIeD7CHxIj\naqp/9f1JFCtrAQBya0vMeWYIJJK2+xERXU4PRznenBgCH3trAIBWp8OHCbnYeqbI5HKMkyMD8eoL\nI2Bhof9RXl7RgOiYvTifUWb2cRMR0Y3RLcJ6dZMam5IMS3X/I9QDAS0rA15N8uli/LErXdx+6pGB\ncHez7ZAxElH3obC1wpsTQhDsYvh5suVUIT45kQ+tiYF9xBBfLH5lDKys9NNrautUWPLGPiSfUbbR\nk4iIbgZGh/X169cjMDAQcrkcEREROHjw4BX3bWpqwhNPPIGwsDDIZDJERUVddr99+/YhIiICcrkc\nQUFB+Pjjj01/B0bYnFyA6pbFSNxtZHiov1ebfWrrVPjg02PidsQgb0SOCeiQ8RFR9+NkbYk1kcEY\nqDA8//LLuQt461A2mk2snx7WT4EVC8bBzlY/vaapSY1Vbx1A/NG8NnoSEVFnZ1RY37p1K+bOnYvo\n6GgkJSVh5MiRmDp1KvLz8y+7v0ajgVwux5w5c3D77bdfdp/s7GzcdtttGD16NJKSkrBgwQLMmTMH\nP//8c/vfzWWklNbiz8xScfvZcD9YW7S9iNGmr06gvKIBAGBvZ4XnnjTMDSUiMge5pRRLx/bCaD/D\neg37csux8kAGGtWm1U8P7uWKmOgouDjrvzXUaLR468PD+HNPhlnHTERE15dRYf2dd97Bk08+iSef\nfBKhoaF4//334eXlhQ0bNlx2fxsbG6xfvx5PP/00fHwuXxpxw4YN8PHxwbvvvovQ0FA8/fTTeOyx\nx/DWW2+1/938jVqrw4fHcsXt4T5OGObjdJUeeoeO5WNffI64PfuJcDg5WpttXEREf5FJJZg/siem\nBVQmeyUAACAASURBVBlWKE0srsaivefFbwSN5efjgDWvR8Hb86+1I3T4eHMifvzlrMnz4YmIqHNo\nM6w3NzcjMTERkyZNatU+efJkxMfHt/vEhw8fxuTJk1u1TZkyBQkJCdBozLMi33/SLiC7Sn933Eoq\nwazBfm32qaxqxMYvEsXtcSN7YMQQ36v0ICK6NhJBwHPhfq2m6KWV1WHe7jSU1KtMOpa7my1ioiMR\neNHqyt/8eBpffJMMrZaBnYjoZmPR1g6lpaXQaDRQKBSt2hUKBXbv3t3uExcXF1/yAUChUECtVqO0\ntPSS8/0lISHBqONXqLT4OK0JzS1TPyO9LJCbchK5V+mj0+nw9bZcFCv1VWMc7C0wuL/x56TOgX9e\nZIzOeJ2EApjspMb2Av2KpGl19Xjmx6OYGSiDh7Vp9QD+casDvv6xEpk5+trrW39Kwrlz2fjHNB9I\npZzSZ6zOeJ1Q58PrhK4kODj4mo/RZavBbC9oFoO6wlrAGPc2P5fg+KlKpJ6vEbfvuc0Xcuu257cT\nEZnLKDcLPNzDEn9ViK1q1uGjjCbk1pv20KmVTIrHZvRAv1DDA6wnTlfim59z0dxs2rGIiOjGaTPB\nurm5QSqVQqlsXQZMqVTC09Oz3Sf29PS87DEtLCzg5uZ2xX4RERFtHvtwfiWy0zNg01IVbcmEUPR1\nt7tqnwuldXj7kz9ha2sDAJg6MQgP3De4zXNR5/HXnQ1jrhHqvm6G6yQCwOCiasTEZaBRrQ/W/74g\nQfToXhjkadrqyUOHDsGGzxOwe38WACCvUIPf9tRh4UujYGsjM/fQu4yb4TqhG4/XCbWlquraF6pr\n8866paUlwsPDsXPnzlbtO3fuxKhRo9p94hEjRlxyzB07diAiIgJSafvvZjeqNdh43FCubEqgW5tB\nXavV4cNPj6GxUf8wl5fCHv83Y0C7x0BEdK0GezlgdWQI7GX6eyqNai2W7UvH/pxyk44jkQh47qkI\nTL+tt9h2Nq0Er6+ORWVVozmHTEREHcCoaTAvv/wyNm/ejE2bNiE1NRUvvvgiioqKMHv2bADAwoUL\nMXHixFZ9UlJSkJSUhNLSUtTW1iI5ORnJycni67Nnz0ZBQQFeeuklpKam4rPPPsOWLVvw2muvXdMb\n+uZ0kfhAloOVBR4Pu3w1mov9vvM8TqdcAKBfxvufs4bC2rrtaTNERB0p1NUW6yaGwr3lDrhap8Ob\nh7Lx33MXTDqOIAh49P4BePSBsP9v787Do6ru/4G/72zJzCSZ7PsOJJCALAlr2BGoWq3aulXrF1da\nFzZFQLFuRfCnFkVZ1LbaVqu0tYqtIIQ9BFAIe0ISliyEzITsk322+/tjkpsMhOyQSXi/nmceZk7O\nvXOG5z4z7znzuedKbbn5FXjxDztRVFzTk0MmIqIe1qFEeu+996KsrAwrVqyAXq/H0KFDsWXLFoSG\n2ldJMRgMyMnJcdjm1ltvRX5+8+mcI0eOhCAI0kovkZGR2Lx5MxYuXIgNGzYgODgYH3zwAe68884u\nv5jcijp8m9X8IfbYiFB4uLT9EgsKjfj7xpPS47t/PhixA326PAYiop4U5uGKd26OxfLdZ3DBWA8R\nIjYcuYDKBgseHBrUqes/3HlrLNy1Kqz7y2GIoghDUTVefH0nXlkyGeGhumv4KoiIqKsEsQ8svtuy\n3kena/0DxSaKWLIjGxkl1QCAoX7uWDV9UJsfZFarDUtf34lzOfaflSPDPfHWqzOg7MBFk8j5sHaQ\nOqKvHifGBgte23sWmaXNM+G3DvTD7xLCIOvkBdt+TLuIP649CHPjhZfctCq8uGgiBg+6+vlCN5q+\nepzQ9cXjhNrTkQzbnn6zGsz286VSUFcIAp5ODGt3xunr/2ZKQV0ul2Hek2MY1InIKXm4KLBi2iAk\ntDjBdPPZYry1Pwcma+dWdxmbEILlz0+Syv2qa0x4ZeUefLs5i2uxExE5mX4R1ivqzfjL8YvS47sH\nByBcp25zm3O55fjnt+nS4wd+ORSR4e1f3ZSIqLe4KuR4edIATI3wltr2XSjHa3vPoc7cuYvJDYvz\nxxsvToOHuwsAwGyx4m9fHcfS13cgv6D7qxcQEVHP6Bdh/dPjF1Flsq/kEqB1wX3xQW32N5mtWPPR\nj9IMUuxAH9x5a+w1HycRUXcp5TI8Ny4Sd8T4S23HioxYtusMKurNndrXgEgvvPnydIeJirPny/Dc\ny8n416YMWCxcj52IqLf1+bB+8lIVtueUSo9/lxAGV0XbL+vLf5/ChYtGAIBKpcC8J8dAJuMV/Yio\nb5AJAp4cGYqHhwVLbWfKavDCjmwU1TR0al/Bge74f6/ejF//aigUje+dVqsNX359Ci+8uh3nc8t7\ndOxERNQ5fTqsm602rDvcvKb6hFBPjA5uu3g/PbMYm7ZkS4/nPHATggLdr9kYiYiuBUEQcF98EJ5J\nDJdOML1YVY/F27OQV1nXqX0pFDL86o44vPP6TAyMbi6xyc2vwOJXtuOLf52EqZNlNkRE1DP6dFj/\nNusS8o32DyW1Qo65o8La7F9Xb8YHH/8EwF7+MmJYIGZPH3Cth0lEdM3cMtAPSydEQdn462BpnRlL\ndmTjdOMJ950RHqrDqt/PwJwHhksn24uiiK//exqLf5+M7LOl7eyBiIh6Wp8N64bqBvwjXS89fmhY\nEHzbuXT2Z/84jksl9mXPtBoVnn4ssVNrFBMROaOkMC+8NmUQ1I0Bu8pkwYu7zuCwvvMnispkAu64\nJRar35yFuFg/qf3CRSOWvbETn/3jOBoaLD02diIialufDOuiKGJ92gVpubJoTw1uH+Tf5jZpx/VI\n3n1eevzEwyPh4625puMkIrpehge4Y9X0GOgaLwRnstrw+t5z2Jnbtdnw4EB3vL5sKh7/zUi4NO5T\nFEV890MWFi7fhvTM4h4bOxERXV2fDOv7CyqkGSMBAp4ZHQ55GyeIVlU3YO2fDkmPx48OxaTx4dd8\nnERE19NAbw3evjkWAVr7coxWUcS7B3PxbVZRl/Ynkwm4deYgvPfmbNwUHyC1G4qq8fKbu/DxX4+g\nrpMr0BARUef0ubBeZ7bi4yMF0uOfDfBFrI+2zW0+/usRVFTWAwB0Hq6YOyeB5S9E1C+FuLvi7Rkx\niGhxrYlPjhbgr8cvoqsXrA7w0+KVFybj6cdGQ6NWSu0/7DiL+cu24thJQ7fHTUREretzYf2LU3qU\n1JkAAJ4uSvzfTcFt9t93MB+pPzavGPPUY4nSRUCIiPojH40Kb02PQZyvm9T2z9MGfHAoH9YuXqFU\nEATMmBKF91fORuKI5vfdktJavP72Xnz4ySFU15i6PXYiInLU58L6puxL0v3HR4bAvbGWsjVl5XX4\n+K9HpMfTJ0Vh9Mi2wz0RUX/g7qLAG1MHYUyL5Wy3ni/Bqv3npfN9usLHW4NlC5Ow4Ldj4aZtPql/\nZ0oO5i/bikNHCrs1biIictTnwrqt8Wfc4QHuDpfcvpwoilj75+aZHj8fLR59aMR1GSMRkTNwVcjw\n0sQBmBHpI7XtL6jAy7vPosbU9XXTBUHA5AkRWLPqZxg/unnJ3PKKOqx8bx9Wrz8IY1XnLs5ERESt\n63NhHQCUMgFPJYS3WXe+bdd5HD3RXEf57JOOtZZERDcChUzAwrERuHtw8wmip4qrsHRnNiq6eXKo\np84Vi58dj8XPToDOw1VqTzmQj3lLf0Dqjxe6XCdPRER2fTKs/2pIIEJbfDBcznCpGp99eVx6/PPZ\nMRg6pO2lHYmI+itBEPDYiFA8MjxEajtfUYvnt2fBUN39GfDxo0OxZtVsTJkQIbUZqxrw7toDePuD\nAyivqO/2cxAR3aj6XFgPcnPBvXGBV/27zSZizcc/SRftCA32wEP3DLtewyMiclq/GhKI+WMiIGv8\nVVJf3YDnt2chp6K22/t2d3PB/N+OxUvPTYK3V/NKNAcPF2D+sh+wOzWXs+xERF3Q58L6UwnhUMmv\nPuzvfshGZnYJAPts0rwnx0Clkl+v4RERObVZ0b5YPjEaKpn9fbS83owlO7Jx6lJVj+w/YXgQ3l85\nGzOnRUtt1TUmrPnoJ6x4dx9KSrv/xYCI6EbS58L6qCCPq/4tv6AS//j3SenxPb+Iw8Doq5+ESkR0\nIxob4onXpw6EVmmfyKgxW/HSrjP409ECVJss3d6/VqPC7x5JxCtLpsDft/k6GEdO6DFv2Q/YuvMc\nZ9mJiDqoz4X1qzFbrHh/w4+wWOxLkkVHeuFXdwzp5VERETmnYf7uWDU9Bl6u9hPvLaKIb7KK8MT3\n6dh8tlhaeas7hscHYPWbs3DbzEEA7KU39fUWfPRZGl59aw8Ml6q7/RxERP1dvwnr//r2NHLyKwAA\nSoUc854cA4Wi37w8IqIeF+2lwTs3xyK+xcWTjA0WrD2cj2d/OI3jRcZuP4faVYnHfjMSf3hpKoIC\n3KX2kxmXsODFbfjftjOwdfFCTUREN4J+kWazz5bi6/+elh4/dO8whIfq2tiCiIgAINDNBW/NiMGS\nCVHw0zRf5Ci3sg4v7jqDP6ScQ2FV91dziYv1w+oVs3DnrbHSsrsmkwV/+fwolq/YhYv67n8xICLq\nj/p8WG9osOD9j36S6h/jYv1w26xBvTwqIqK+QxAETA73xke3xuM3w4Lh0uIk/gMXK/C7zRn4y7EC\n1Jq7fiElAFCp5Hj4/uF465UZCAtpPv8o80wJFr2UjG++z4S1G1dXJSLqj/p8WP/8nyehL7KvYuDq\nqsCzT46BTHb1iyUREVHrXBQy3B8fhE9ui8f0Flc9tYgivs4swhP/S8fWcyXdrmcfGO2Nt1+fiXvv\njJPer80WK/6+8QSWvr4TeRcqurV/IqL+pE+H9ePpRfg++Yz0+NEHRyDAT9vGFkRE1B4fjQrPjYvE\nH2cOxpAW9ewVDWasOZSH+VszcbKbSz2qlHLcf/dQvP3aTERHeEnt53LK8Pzvt2PjN+kwW7o3k09E\n1B/02bBeU2vCh58ckh4nDA/CjMlRvTgiIqL+JdZHi7dnxOCF8VHwVTfXs5+vqMXSndlYse9ct6+A\nGhXhibdenYGH7hkmLQpgtdqw8Zt0vPDKDpzNKevW/omI+ro+G9b/8sUxlJbZL67hplXhqcdGSyct\nERFRzxAEAVMivPHRbfH49dAgh4vS7S+owG83p+Oz4xdR1416drlchrtvH4J335iF2IHN5Td5Fyqw\n5NUd+PyfJ2HqZr08EVFf1SfD+k9pF7ErJVd6PHdOArw8XXtvQERE/ZyrQoYHhwbj49viMS2i+WJz\nZpuIf5024Inv07HtfPfq2cNCPLBi+XQ88usRUDVesEkURfznf6fx3PJkZJ4p6fbrICLqa/pcWK+o\nrMe6vxyWHk8aH46ksWG9OCIiohuHn0aF58dH4d2bYxHj3XyOUHm9Ge//lIeF2zJxqhv17DKZgNt/\nFoP33pyN+MF+UvtFvREvvrELf/niGOrru3+VVSKivqLPhfWPPkuDscpeI+nlqcYTvxnVyyMiIrrx\nDPZ1w7szY/HcuEj4qJVS+9nyWizZmY1VqedRVNP1evbAADe8tnQq5s5JgKurorFVxP+2ZmPhS1tx\nMuNSN18BEVHf0OfC+o9pF6X7zzw+Gm5uqjZ6ExHRtSITBEyP9MHHt8Xj/vggqGTNHykpF8rx2+8z\n8PcThV2uZ5fJBMyePgDvvzkbI4YFSu1FxTV4ZdVubPg0DbV15m6/DiIiZ9bnwnqTWdMGYORNge13\nJCKia8pVIcdvhgVjw21xmBzevAyjyWbDVxl6zN2cjh05pV2uZ/fz1eLl5yfh2SfGQNviKqvbdp3D\n/GVbceS4vtuvgYjIWfXJsB7g74b/e+Cm3h4GERG1EKB1wZIJ0Xh7RiwGemmk9tI6M/74Yy4WJWfh\ndEl1l/YtCAKmTYrE+6tmY/TI4OZ9l9XiD++m4INPfkJ1tanbr4GIyNn0wbAuYN6TY6B2VbbflYiI\nrrs4PzesnjUYC8ZEwKvFe/WZsho8vz0L/29/DopruxasvT3VWLogCYueGgd3NxepfVdKLuYt+wE/\ntSiVJCLqD/pcWL/z1hgMifHt7WEQEVEbZIKAmdG++OS2eNwXF+hQz74nvwxPfp+OL04Vot5i6/S+\nBUHAxHHhWLNqtsNqYBWV9Vj1fireXXsAlcb6HnkdRES9rc+F9ft/ObS3h0BERB2kVsrx8E0hWH9r\nHCaGtahnt9rwj1N6zP0+HbtySyF2oZ5d5+GK554ejyXzk+Cpa77WRuqPFzBv6VbsO5jfpf0SETmT\nPhfWmy6UQUREfUegmwuWJUVj1fQYDGhRz15SZ8I7B3Px/PYsZJXWdGnfYxNCsGblzzBtUqTUVlXd\ngD+uO4hV76WirKKuu8MnIuo1HQ7r69atQ3R0NNRqNRITE7Fv3742+586dQpTp06FRqNBWFgY3njj\nDYe/79mzBzKZzOEml8uRnZ3dtVdCREROb5i/O96bNRjzR0fA06W5nj2ztAaLkjPx7sEclHShnt3N\nTYVnnxiD5c9Ngo9385eBQ0cLMX/pVuzcm8NZdiLqkzoU1jdu3IgFCxZg+fLlOHbsGCZMmIBbbrkF\nBQUFrfavqqrCzJkzERQUhLS0NLz//vt4++23sXr1aod+giDg9OnTMBgMMBgM0Ov1GDRoUPdfFRER\nOS2ZIGDWAF988vN4/HJwAJQyQfrbzlx7PfuXp/Ro6EI9+6jhQXh/5WzMnj5AaqupNeHDPx3CG++k\noLika7P3RES9pUNhffXq1Xj00Ufx6KOPIjY2FmvWrEFQUBDWr1/fav/PP/8cdXV1+Otf/4ohQ4bg\n7rvvxpIlS/DHP/7xir5+fn7w9/eXboIgtLJHIiLqbzRKOR4dEYr1t8RjQqin1N5gteHzU4WYuzkd\ne/PKOj0jrlErMXdOAl5bOhUBflqp/dhJA+a/uBVbd56DzcZZdiLqG9oN62azGWlpaZg5c6ZD+6xZ\ns7B///5Wtzl48CAmTZoElar54hWzZ89GYWEh8vLypDZRFJGYmIjg4GDcfPPN2L17dxdfBhER9VVB\n7i54aeIAvDktBlGeaqm9uNaEtw7k4IUd2cjuQj37sDh/rF4xGz+fHQPAPhFUX2/BR5+l4ZVVu2Eo\n6tqa70RE11O7Yb2kpARWqxUBAQEO7QEBATAYDK1uYzAYWu0viqK0TVBQEDZs2ICvv/4a33zzDWJj\nYzFjxgykpqZ29bUQEVEfNjzAHWtmD8EzieHQuSik9oySaixMzsTqH3NR2sl6dldXBR59cATefHka\nQoI8pPb0zGIseHEr/vtDNmfZicipKdrvcm3ExMQgJiZGejx27Fjk5ubi7bffRlJS0lW3O3z48PUY\nHvVhPEaoI3icOC8/AL8LFrG9yIJ9JRY0ZenvTtXih4wLmO6vwGQ/hUOte0c8eJc3du6zYO/BEogi\nUAPgw0/247vNx3DXrcHw93W9YhseJ9QRPE7oanriXMx2Z9Z9fX0hl8tRVFTk0F5UVITAwMBWtwkM\nDGy1vyAIV90GsAf2M2fOdGTcRETUj6nlAm4PVuL5WBfEeTR/VJlswA8GC97OasCJCmun6tkVChlm\nTQ3E7/5vAAL9m4N5/sVafPiXc9i9v5iz7ETkdNqdWVcqlUhISEBycjJ++ctfSu3Jycm45557Wt1m\n/PjxWLp0KUwmk1S3vm3bNgQHByMiIuKqz3X06FEEBQW1OZ7ExMT2hkw3qKaZDR4j1BYeJ33PLQCO\nGoz4+EgB8o32NdMbAHxdAmQJ7nhyVKjD2u3tSQRw6y1W/Oe/mfj3d6dhtdpXndl/uAqGYgWefnw0\nSovP2vvyOKE28P2E2lNZWdntfXRoNZhFixbhs88+w5///GdkZmZi/vz50Ov1+O1vfwsAWLZsGW6+\n+Wap/69//WtoNBrMmTMH6enp+M9//oO33noLzz33nNTn/fffx6ZNm3D27FlkZGRg2bJl+O677/Ds\ns892+0UREVH/MjLQAx/+bAh+lxAOd1XzPNOp4irM35qJ93/KQ0W9ucP7UyrkuO+ueLzz+s0YEOUt\ntZ/PK8fiV5KxI6UIFmvnl44kIuppHapZv/fee1FWVoYVK1ZAr9dj6NCh2LJlC0JDQwHYTyjNycmR\n+nt4eCA5ORlPP/00Ro8eDS8vLyxevBgLFiyQ+phMJrzwwgsoKCiAWq1GfHw8Nm/ejNmzZ/fwSyQi\nov5ALhPw80F+mBLuha/S9fjvmWJYRREiRGw7X4KU/HLcFx+IX8T4QyXv2DX/IsI8ser30/HdD9n4\n6ut0mC1W2Gwidu4rxpGTFbizQIMpSREIDnS/xq+OiKh1gtgHLunW8icEnU7XiyMhZ8afI6kjeJz0\nHxeM9fjzsQIcKnT8mTlQ64LHRoRgfKhnp67dcVFvxNo/HUbmmRLU1NQCALRae3nNoGgfTEkKx8Rx\n4fBwd+m5F0F9Gt9PqD09kWHlr7766qs9NJ5rpqGhQbrv6nrl2fpEAFBYWAgACA4O7uWRkDPjcdJ/\n6FwUmBrhjcE+Wpwtr4WxwQIAqDZbkXKhHKeKqxHtqYaXWtmh/Xm4u2DapEi4ubngZHohLBYRKpV9\n27LyOhw5YcB3P2TjbE45ZDIBgQFukHdwBp/6J76fUHt6IsP22tKNREREPSEhSIfhAR7YfLYY/zil\nR5XJHtpPXKrC/G2ZmB3tg4eGBcPTtf3QLpMJ+PmsQfD1KEfW2SroS1yRdlwvnYRqs4k4fLQQh48W\nQqNWYsKYMExJisCQGF/IOrmUJBFRRzCsExFRn6eQCbgjxh/TIrzxxSk9vj9bDJsowiaK2HKuBHvy\nyvHA0CDcPsgPyg7MhisUMsQP1uH/EhNRVd2A1B8vYE9qHrLOlkp9auvM2L7nPLbvOQ8/Hy0mTwjH\n1IkRDhdfIiLqLpbBUL/BnyOpI3ic9G8uChkSg3VICvOEoboB+mr754fZJuKowYi9+eUI0KoQ4u7S\nZj17y+PERaXAwGhv3DwlGpPHh8NN64KSsjrU1DRfTbW2zozT2SXYsv0sDh/Tw2S2wt9XC1dXzon1\nZ3w/ofb0RIZlWKd+g2+a1BE8Tm4Mnq5KTI3wxiAfLc6W1UqlMVUmK/bklyOjpAYDvDRXLY252nHi\n7uaCoUP8cdvMgRgxLBAKhQxFl2pgMlulPuUVdTjaVN9+vsxe3+7P+vb+iO8n1B7WrBMREV2FIAgY\nE6zDyAB3fH+2GF+e0qO6MVQfKzLi2a2n8bMBvnhoWDB0Lp37OBQEAYMH+WLwIF88+tAIHDluwJ7U\nPBw+VgiLxV7fLooi0o7rkXZcD7WrEuNHh2JKUgTiB/uxvp2IOoxhnYiI+jWlXIY7YwMwLdIHX5ws\nxJZzJVI9++azxdibV4YHhgbh54P8oehCiFYq5BibEIKxCSGorjYh9Sd7fXvmmRKpT129GTtTcrAz\nJQc+3hpMnhCOKRMiEB7K5YiJqG0sg6F+gz9HUkfwOLlxuSpkGB2sw4RQTxRWNcBQY/9sMdlEHDEY\nsS+/HAFuLgh2c4FerwfQ+eNEpZJjYJQ3ZkyJwpSkCLi7uaC0rA7VLerb6+rMyMwuwQ87zuHQkUI0\nmKwI8GN9e1/E9xNqT09kWF4UifoNXpyCOoLHCQH2EpWfCivxp6MFKKxucPhbQqAHxijKEegq65Hj\nRBRFnDlXhj3785ByIN8huDcRBAEjhgVgyoQIjE0IgUsny3Kod/D9hNrTExmW7wZERHTDEQQBY0M8\nMSrQA/89Y69nr7XY69nTDEbsq23ASE85av3LMDLAA+7dCM+CICBmoA9iBvpgzq+H49iJIuxOzcXh\no3qYG59TFEUcPWHA0RMGuLoqMC7RXt8+bIg/69uJbnAM60REdMNSymW4e3AApkd64/OThdh6vhQ2\nUYQoAkfKrcjcnwOZIGCwjxaJQR5ICNIh2ksNWRvLPrb5fAo5Ro8KxuhRwaiuMeHAoQLsSc1DRlax\n1Ke+3oLd+3Kxe18uvL3UmDw+HFMnRrK+negGxTIY6jf4cyR1BI8Tasv58lp8crQAB3OKAAAareaK\nPp4uSowK8sDoIA+MDOzerHuTouIapBzIw+59eSg0VLXaJzLcE1OTIjBxfDi8PdXdfk7qPr6fUHt6\nIsMyrFO/wTdN6ggeJ9QeURTxTcohnDZaUazywtmyWoho/aNSJgiI9dEiIdADCUEeGOit6fKse9Nz\nn8spx+7UXOw7eAHGqoYr+giCgJviAzAlKRzjEkJ5Ymov4vsJtYc160RERD1MEASEa2QI18iQmDgY\nFfVmHDUYkaY3Is1ghLHBIvW1iSJOl1TjdEk1Pj9V2Djr7o6EQB1GBXnAowvrtw+M9sbAaG/MeWAE\njp40YO/+PPyUVuhQ3378lAHHTxnwkcsRjEsIwdSJERgWF8D6dqJ+iGGdiIioDZ6uSkyL9MG0SB/Y\nRBFnympxWF+JNL0R2aWOs+4VDWbszC3DztwyCBAQ46NBQpAHEoN0GNTJWXeFQobRI4MxemQwampN\nOHjoInan5iI9s7m+vaHBgj3787Bnfx68PO317ZMnRCAqwrNH/w+IqPcwrBMREXVQU9lLrI8WDw4N\nRmWDBUf1RqQZ7OG9ssWsuwgRWaU1yCqtwT9O6eHhopDKZUYF6Tp11VStRoUZU6IwY0oUiktqsHd/\nPvbsz0NBoVHqU15Rh01bsrBpSxbCQ3WYkhSByePD4eN9Zd09EfUdrFmnfoO1g9QRPE6oI7pynNhE\nEWfLaqVymazSGtiu8hErQMAg76ZZdw/E+Gg7XesuiiLO5ZZjT2oe9h28gEpjfavPdFOcP6ZMjMC4\nxBCoXZWdeg5qG99PqD2sWSciInISMkFAjI8WMT5aPDA0CFUNFhw1GHFIb8QRvREVDWaprwgR2WU1\nyC6rwZfperirFBgV6IHEYA+MCvSAZwdCtSAIGBjljYFR3pjzwHAcP1WE3al5+CntIkxmq/RMJzKK\ncCKjCB99psC4hBBMnhCO4UMDIJfLrtH/BBH1JIZ1IiKia8DdRYHJEd6YHOENmyjifHkd0vSVeTf/\nCAAAE4pJREFUOKw3IvOyWfcqkwV78suwJ78MADDI277CTGKwB2I7MOsul8swangQRg0PQm2dGQcP\nFWDP/jyczCgGGmvqTSYL9h7Iw94DefDUuWLiuHBMTbLXtwvdWMGGiK4thnUiIqJrTCYIGOitwUBv\nDe6Lb5x1L2pcYUZvRHm92aH/mbIanCmrwVcZ9ln3kYHuSAjSITGo/Vl3jVqJ6ZOjMH1yFErLarH3\nQD5278vDhYvNP8dXVNbjf1uz8b+t2QgL8cCUpEhMmcD6diJnxJp16jdYO0gdweOEOuJ6Hic2UURO\nRR0OF1YizWDE6ZKr17oDwECv5hVmYn20kHdguUZRFJGbX4ndqblIOZCPisrW69uHxflh8vgIjB8T\nCo2a9e3t4fsJtYc160RERH2cTBAwwEuDAV72WfdqkwXHiqpwuLASRwxGlNY5zrqfLa/F2fJabMww\nwE0px4jGFWYSg3TwvkrAFgQBURGeiIoYgYfvuwkn0i9hT2oeDqZdhMnUtIKNiJMZl3Ay4xI++dsR\njB4VjKkTIzGC9e1EvYphnYiIyIm4qRSYGOaFiWFeEBtn3dP0RhzWG3G6pBrWFrPu1WYr9l0ox74L\n5QCAaE8NEoPs4X2Ir1urs+5yuQwjbwrEyJsCUVdvxsHDF7FnXx5OZFyCVN9utiL1xwtI/fECdB6u\nmDguDFMmRGBAlBfr24muM4Z1IiIiJyUIAqK9NIj20uCeuEDUmKw41qLWvaTO5ND/fEUtzlfU4p+n\nDdAq5Rge4I7RQfarqfpqVFfsX+2qxLSJkZg2MRKlZbVIOXgBe1LzkHehQupTaazH99vO4PttZ+Cm\nVSE0xANhIR4IC9EhLMQD4SE6eOpcGOKJrhGGdSIioj5Cq5IjKcwLSY2z7nmV9TjcuMJMRrHjrHuN\n2Yr9BRXYX2AP3lGeaukk1SG+blBcNuvu463BnbfG4s5bY5GbX4E9qXnYuz8f5ZV1Up/qGhMys0uQ\nmV3isC1DPNG1w7BORETUBwmCgEhPNSI91fjVkEDUmq04ZjDiiMFeMlNc6zjrnlNRh5yKOvz7tAEa\nhRwjAt2RGKRDQiuz7pHhnogM98Rv7rsJJzPs67cfOlKI2svq55t0JMSHh+gQyhBP1GkM60RERP2A\nRinHhDAvTGicdc831uNwoRFphkqkX6qGpcWse63FcdY9QqdurHXXIc5XC2XjCaUymYDhQwMxfGgg\nRFFESWktLlw0Ot4KK1Ffb2l1TG2F+LAQHUJD3BHeOBMfxhBP1CqGdSIion5GEARE6NSI0KnxyyEB\nqDNbcbyoSiqZuXzWPa+yDnmVdfg6swhqhRwjAtztK8wE6+DXOOsuCAL8fLXw89Vi1PAgadvWQnz+\nxUoUFBrbDPGns4txOrvYob0pxIdJJTUeCA/VQefBEE83LoZ1IiKifk6tlGNcqCfGhXpCFEVcMNZL\nK8ykF1fBbGueda+zWHHgYgUOXLTPuod7qJEY7IGEQA8M8NLATSV3CM4M8UTXFsM6ERHRDUQQBITr\n1AjXqXHXYPus+4lLVTjcuMJMUU2DQ/98Yx3yjXX4T2YRAECtkMNPo4KfVgV/jQr+WhX8NEr4a13g\np1HBR62EXCZ0KMTnXzSioIdCfHioB0KDGeKp/2FYJyIiuoGplXKMDfHE2BD7rHtBVQPS9JVI0xtx\n6lI1TDabQ/86i1UK8K2RCQJ81MoWQb4x2Dfe99eorkuIDw/VNYb35hVqGOKpL2JYJyIiIgD2Wfcw\nD1eEebjiztgA1FusOHGpGmn6SpwoqkZRTQMarLY292ETRRTXmlBca0J6Set93FUKe4jXKOHXFOIb\nZ+ojY3wx4qZAyBpDtSiKKC6tbQzvRly4WIkLF43thviMrGJkZLUe4sMuW2bSU+fa+f8souuEYZ2I\niIha5aqQY0ywDmOCdQDswbnKZMWlGpMUyC/VmHCp1oTiGvv9iobWl3dsqcpkQZXJgvMVrf9dIQjw\nbTEz769Vwd9NhYhRQRg9KQK+ahWUMgElZT0T4t3dXBzq4RniyZkwrBMREVGHCIIADxcFPFwUGOit\nabWPyWqzB/mmEN/yfmPIb3lCa2ssoghDTQMMl9XPt6RzUTTWySvhH6hB2ABPjNKo4KdWQWayoOJS\nDQoKq3DhYiXyC+whvqGh9RBfVd3QoRDfVFrDEE/XE8M6ERER9RiVXIYQd1eEuLceaG2iiMp6ixTk\nm2bpW/5bZWo9VLdU2WBBZYMFZ8quPg5/jQp+A70QNjwAI9VKuNhEWKtMqCmtRWVRNS5erOpWiBdE\nI9y0ShjKzsBNq4RWo4KbVgWtRgk3N/t9lVLe7mshagvDOhEREV03MkGAl1oJL7USsT7aVvvUma0O\nZTbN980orjWhpNYEq9j27LzJakNBVT0Kqupb/bvgoYR3gD+CkkLhLhMga7DCWm1CfXkdjEU1KC00\nwlJnQWunozaF+JqaWgDAj0errzoOpUIOrVbZHOK1jYFealO10mb/V3XZMpl0Y2JYJyIiIqeiVsql\n5SVbYxNFlNWZm2fjW4b6xtr5Wou1zecQIaK0zozSustq7D1V9luMFxQA1AAUJivEGjMaKupRXVwL\n1JqhNNsgAq2G+ZbMFisqKq2oqGz9S0NbFAoZtJrm8N4U5ptDfuNsvpvKIeRrtUq4uigY9PuJDof1\ndevW4Z133oFer0d8fDzee+89TJw48ar9T506hWeeeQY//fQTfHx88OSTT+Lll1926LNnzx4899xz\nSE9PR0hICBYvXoy5c+d2/dUQERFRvydrPAHVt/Hqqq2pMVlxqbahMcSbcammoXGG3oziGhNK68wQ\n0cbsvABYAFQBgIvcfvN2hTzKExarDdUmK2qq6yDYRDSoVBCsImCxQbTYYDNbYTPZYDFZIFhsEERA\nZhNb3ABBFNtsE0TAYrGh0liPSmPng75MJkihvvWZfZVD6U7LLwNqVwZ9Z9KhsL5x40YsWLAAGzZs\nQFJSEtauXYtbbrkFp0+fRmho6BX9q6qqMHPmTEydOhVpaWk4ffo05syZAzc3NyxcuBAAkJubi9tu\nuw2PP/44vvjiC6SkpOCpp56Cv78/7rrrrp59lURERHRD0arkiFJpEOXZ+omwFpuIklrTleU2NSYU\nNZ4Qa2ptmUrBPuOtUMgA0T4rr9G2/gsAAIg2EVabCFvjzWoVYWm6b7M5tNsuaxeszcHdMdi33yaI\nIow2K2TGOsgqaqU26YtBG/93giBAq1E2Bvp2Zva1LcK+RgmNWgmZjEG/Jwmi2E7RF4Bx48ZhxIgR\n2LBhg9QWExODe+65BytWrLii//r167Fs2TJcunQJKpX9W++KFSuwYcMGXLhwAQCwZMkSfPvtt8jK\nypK2e+KJJ5CRkYHU1FSH/VVWVkr3dTpdJ18i3SgOHz4MAEhMTOzlkZAz43FCHcHjhERRhNFklVaw\naQrzRTUN0ux8YZk9n2i0rX8h6P4YAJvV5hj2bSJs1qYvALartreX7uzB3THcNwV5x8edaxNEEUq5\nDEqFDAq5TPpio1DY2+Tyqzy+rF3R4m/KFvto6ieXt97e2r4v7ydXCNL9a/0LQk9k2HZn1s1mM9LS\n0rB48WKH9lmzZmH//v2tbnPw4EFMmjRJCuoAMHv2bPz+979HXl4eIiIicPDgQcyaNcthu9mzZ+Nv\nf/sbrFYr5HKePU1ERES9QxAE6FwU0LWxTOWPhw6hwQrE3TQM9RZb482KeqsNDdLjxpvV6thmtf/b\nYLGhwdq4naW5zWSzQRAAuUKGriQiUUTjjP1lgd5hJt/xb5bGsN+BedwOEUQAEBv/tT8WRBGADYLN\nBpiaHjf97cr+TY/b/5vjvpoet9a/5VhkMgEKmQwKmdB4X7AHf5kAhVyAXCaDUi5ALhPsX0KavizI\n7CsOKRVyqU0pF6BU2PuoFDKolHLcOrH5Kr1d1W5YLykpgdVqRUBAgEN7QEAAduzY0eo2BoMBYWFh\nV/QXRREGgwEREREwGAyYOXPmFX0sFgtKSkqueD4iIiIiZyIXBGgUaLN2vqtsonhZuLdKQd4x8Lf4\nEmBt/sLQ4PC4+UuA/UuD2Ga9flPQt9lsl5XntB32m+5L+xEAQGj8t0lfK5ER7TcRgMVqP5Hh6sv/\nX+G6hHVn0/LnBKKWBg0aBIDHCLWNxwl1BI8T6ojrcZzIAGgAaAQAysYbBADyxpvymj03OQdZex18\nfX0hl8tRVFTk0F5UVITAwMBWtwkMDGy1vyAI0jZX66NQKODr69upF0FERERE1B+1G9aVSiUSEhKQ\nnJzs0J6cnIykpKRWtxk/fjxSUlJgMpmktm3btiE4OBgRERFSn8v3uW3bNiQmJrJenYiIiIgIHVwN\n5p///CcefvhhrF27FklJSVi/fj0+/fRTZGRkIDQ0FMuWLcOhQ4ewfft2AIDRaMTgwYMxdepUvPTS\nS8jKysIjjzyC1157DQsWLABgX7px2LBhePzxxzF37lzs27cPzzzzDL766ivceeed1/ZVExERERH1\nAR2qWb/33ntRVlaGFStWQK/XY+jQodiyZYu0xrrBYEBOTo7U38PDA8nJyXj66acxevRoeHl5YfHi\nxVJQB4DIyEhs3rwZCxcuxIYNGxAcHIwPPviAQZ2IiIiIqFGHZtaJiIiIiOj6a7dm3RmsW7cO0dHR\nUKvVSExMxL59+3p7SOREVq5ciTFjxkCn08Hf3x933HEH0tPTe3tY5MRWrlwJmUyGefPm9fZQyMkY\nDAbMmTMH/v7+UKvVGDp0KFJSUnp7WOREbDYbXn75ZSmXREdH4+WXX4bN1srVTumGkZKSgl/84hcI\nDQ2FTCbD3/72tyv6vPrqqwgJCYFGo8G0adOQkZHRoX07fVjfuHEjFixYgOXLl+PYsWOYMGECbrnl\nFhQUFPT20MhJ7N27F8888wwOHDiAXbt2QaFQ4Oabb0ZFRUVvD42c0MGDB/HJJ59g+PDhvT0UcjKV\nlZVISkqCIAjYsmULMjMz8cEHH8Df37+3h0ZOZNWqVVi/fj0+/PBDZGVlYc2aNVi3bh1WrlzZ20Oj\nXlRdXY1hw4ZhzZo10GiuvIjWW2+9hdWrV2Pt2rU4fPgw/P39MXPmTNTU1LS7b6cvgxk3bhxGjBiB\nDRs2SG0xMTG45557sGLFil4cGTmrmpoa6HQ6bNq0CbfddltvD4ecSGVlJRISEvDnP/8Zr776qvTG\nSgQAL774IlJSUjiTTm26/fbb4evri08//VRqmzNnDsrKyvDdd9/14sjIWbi7u2Pt2rV4+OGHpbbg\n4GDMmzcPS5cuBQDU19fD398f7777Lp544ok29+fUM+tmsxlpaWlXXOl01qxZ2L9/fy+Nipyd0WiE\nzWaDl5dXbw+FnMyTTz6Je++9F1OmTOntoZAT2rRpE8aOHYv7778fAQEBGDlyJNauXdvbwyInM3Hi\nROzatQtZWVkAgIyMDOzcuZOTQ3RVOTk5MBgMDnnW1dUVkydP7lCedeormJaUlMBqtSIgIMChPSAg\nADt27OilUZGzmz9/PkaNGoXx48f39lDIiXzyySc4f/48vvzyy94eCjmp8+fPY926dVi4cCGWLVuG\nY8eO4ZlnnoEgCHjqqad6e3jkJJYsWYKqqirExcVBLpfDarXipZdewty5c3t7aOSkDAYDBEFoNc8W\nFha2u71Th3Wizlq0aBH279+P1NRUCILQ28MhJ5GdnY2XXnoJqampkMmc+gdF6kU2mw1jxoyRSiyH\nDx+O7OxsrF27lmGdJF999RX+/ve/46uvvkJcXByOHTuGefPmISoqCo888khvD4/6Iaf+1PL19YVc\nLkdRUZFDe1FREQIDA3tpVOSsFi5ciI0bN2LXrl3SlXKJAODAgQMoLS1FXFwclEollEol9uzZg7Vr\n10KlUsFsNvf2EMkJBAUFYciQIQ5tQ4YMQX5+fi+NiJzRCy+8gMWLF+Oee+5BfHw8HnzwQSxatIgn\nmNJVBQYGQhTFLudZpw7rSqUSCQkJSE5OdmhPTk5GUlJSL42KnNH8+fOloD5o0KDeHg45mbvuugsn\nT57E8ePHpVtiYiIeeOABHD9+HEqlsreHSE4gKSlJqkNukpWVxS//5KC2tvaKX+hkMhmXbqSrioqK\nQmBgoEOera+vR0pKSofyrNOXwSxatAgPP/wwRo8ejaSkJKxfvx56vZ61YSR5+umn8fnnn2PTpk3Q\n6XTSN1c3NzdotdpeHh05Aw8PD8TFxTm0abVaeHt7XzGTSjeuhQsXIikpCW+++Sbuu+8+HDlyBB98\n8AFWrVrV20MjJ3L77bdj1apViIyMRHx8PI4cOYLVq1djzpw5vT006kU1NTU4e/YsRFGEzWZDfn4+\njh8/Dm9vb4SFhWHBggVYuXIlYmNjMWjQIPzhD3+Au7s7HnjggfZ3LvYB69evF6OiokRXV1cxMTFR\n3LdvX28PiZyIIAiiTCa74vbaa6/19tDIiU2bNk189tlne3sY5GQ2b94sDh8+XFSr1WJsbKz44Ycf\n9vaQyMlUV1eLCxcuFCMjI0WNRiMOGDBAXL58udjQ0NDbQ6NetHv37lbzyCOPPCL1ee2118Tg4GBR\nrVaLU6dOFdPT0zu0b6dfZ52IiIiI6Ebl1DXrREREREQ3MoZ1IiIiIiInxbBOREREROSkGNaJiIiI\niJwUwzoRERERkZNiWCciIiIiclIM60RERERETophnYiIiIjISTGsExERERE5qf8PRBekL4bJQEwA\nAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "goal_dist1 = MakeGoalPmf(suite1)\n", + "goal_dist2 = MakeGoalPmf(suite2)\n", + "\n", + "thinkplot.Clf()\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(goal_dist1)\n", + "thinkplot.Pmf(goal_dist2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.2: Distribution of goals in a single game.\n", + "\n", + "Figure 7.2 shows the resulting distribution of goals for the\n", + "Bruins and Canucks. The Bruins are less likely to score 3 goals or fewer\n", + "in the next game, and more likely to score 4 or more." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The probability of winning\n", + "\n", + "To get the probability of winning, first we compute the distribution of\n", + "the goal differential:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "goal_dist1 = MakeGoalPmf(suite1)\n", + "goal_dist2 = MakeGoalPmf(suite2)\n", + "diff = goal_dist1 - goal_dist2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The subtraction operator invokes `Pmf.__sub__`, which enumerates pairs\n", + "of values and computes the difference. Subtracting two distributions is\n", + "almost the same as adding, which we saw in Section [addends].\n", + "\n", + "If the goal differential is positive, the Bruins win; if negative, the\n", + "Canucks win; if 0, it’s a tie:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.457996492072\n", + "0.370293408955\n", + "0.171710098973\n" + ] + } + ], + "source": [ + "p_win = diff.ProbGreater(0)\n", + "p_loss = diff.ProbLess(0)\n", + "p_tie = diff.Prob(0)\n", + "print(p_win)\n", + "print(p_loss)\n", + "print(p_tie)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With the distributions from the previous section, `p_win` is 46%,\n", + "`p_loss` is 37%, and `p_tie` is 17%.\n", + "\n", + "In the event of a tie at the end of “regulation play,” the teams play\n", + "overtime periods until one team scores. Since the game ends immediately\n", + "when the first goal is scored, this overtime format is known as “sudden\n", + "death.”" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Sudden death\n", + "\n", + "To compute the probability of winning in a sudden death overtime, the\n", + "important statistic is not goals per game, but time until the first\n", + "goal. The assumption that goal-scoring is a Poisson process implies that\n", + "the time between goals is exponentially distributed.\n", + "\n", + "Given `lam`, we can compute the time between goals like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "lam = 3.4\n", + "time_dist = thinkbayes.MakeExponentialPmf(lam, high=2, n=101)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`high` is the upper bound of the distribution. In this case I\n", + "chose 2, because the probability of going more than two games without\n", + "scoring is small. `n` is the number of values in the Pmf.\n", + "\n", + "If we know `lam` exactly, that’s all there is to it. But we\n", + "don’t; instead we have a posterior distribution of possible values. So\n", + "as we did with the distribution of goals, we make a meta-Pmf and compute\n", + "a mixture of Pmfs." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeGoalTimePmf(suite):\n", + " metapmf = thinkbayes.Pmf()\n", + "\n", + " for lam, prob in suite.Items():\n", + " pmf = thinkbayes.MakeExponentialPmf(lam, high=2, n=2001)\n", + " metapmf.Set(pmf, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(metapmf, name=suite.name)\n", + " return mix" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.3 below shows the resulting distributions. For time values\n", + "less than one period (one third of a game), the Bruins are more likely\n", + "to score. The time until the Canucks score is more likely to be longer.\n", + "\n", + "I set the number of values, `n`, fairly high in order to\n", + "minimize the number of ties, since it is not possible for both teams to\n", + "score simultaneously.\n", + "\n", + "Now we compute the probability that the Bruins score first:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "time_dist1 = MakeGoalTimePmf(suite1)\n", + "time_dist2 = MakeGoalTimePmf(suite2)\n", + "p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the Bruins, the probability of winning in overtime is 52%.\n", + "\n", + "Finally, the total probability of winning is the chance of winning at\n", + "the end of regulation play plus the probability of winning in overtime." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "p_tie = diff.Prob(0)\n", + "p_overtime = thinkbayes.PmfProbLess(time_dist1, time_dist2)\n", + "\n", + "p_win = diff.ProbGreater(0) + p_tie * p_overtime" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAv8AAAEWCAYAAADxWPj1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVXX+x/HXvcCFy77Koggi7uaSWK65FODSlJWa2cxk\ni8nkWFrjT52cNofMbFyyVFIbM2cmm8qaUkuycrfUwjXMXREuiiAisnN/f1BXCQzcuCDv5+Ph4+H5\n3s8593Ov3x59zrnfxWC1Wq2IiIiIiMgNz2jvBEREREREpGao+BcRERERqSdU/IuIiIiI1BMq/kVE\nRERE6gkV/yIiIiIi9YSKfxERERGRekLFv4iIiIhIPVHt4n/u3LlERERgNpuJiopiw4YNvxm/e/du\nevfujaurK6GhoUyZMqVCzNq1a4mKisJsNhMZGUlCQkK51z/44AM6d+6Mj48P7u7udOzYkSVLllx1\nbiIiIiIi9VG1iv9ly5YxduxYJk+eTFJSEt26daN///6kpKRUGp+Tk0N0dDTBwcFs376d2bNnM336\ndGbOnGmLOXLkCAMHDqRHjx4kJSUxceJExowZw/Lly20x/v7+/O1vf+Pbb79l165dPPzwwzz66KN8\n/vnnV5ybiIiIiEh9ZajODr9dunShQ4cOzJ8/39bWvHlzhgwZQnx8fIX4efPmMWnSJE6ePInJZAIg\nPj6e+fPnc/z4cQAmTJjAxx9/zL59+2znjRw5kr1797Jx48ZL5tKpUyf69etne9/LzU1EREREpL6q\n8sl/UVER27dvJzo6ulx7TEwMmzZtqvScLVu20LNnT1vhDxAbG0tqaipHjx61xcTExJQ7LzY2lm3b\ntlFSUlLpddesWcNPP/1Er169rjg3EREREZH6qsriPyMjg5KSEgIDA8u1BwYGYrFYKj3HYrFUGm+1\nWm3nXCqmuLiYjIwMW9vZs2fx8PDAZDLxu9/9jtdff91203AluYmIiIiI1FeO9k6gKh4eHuzYsYNz\n586xZs0axo0bR3h4OH369Lnsa2VnZ1+HDEVEREREaoaXl9dVnV9l8e/v74+DgwPp6enl2tPT0wkK\nCqr0nKCgoErjDQaD7ZxLxTg6OuLv729rMxgMREREANCuXTv27t3Lyy+/TJ8+fa4oNxERERGR+qrK\nYT9OTk506tSJxMTEcu2JiYl079690nO6du3K+vXrKSwstLWtXr2akJAQwsLCbDG/vubq1auJiorC\nwcHhkvmUlpZSUFBwxbmJiIiIiNRX1Rr28/TTT/PHP/6Rzp070717d+bNm0daWhpxcXEATJo0ia1b\nt/Lll18CMHz4cF566SVGjBjBs88+y759+5g2bRovvvii7ZpxcXG8+eabjBs3jlGjRrFhwwaWLFnC\ne++9Z4t5+eWXufXWW4mIiKCgoIAVK1awdOlS3njjjSpzGzVq1G9+pqv9yUTketu2bRsAUVFRds5E\npHrUZ6UuUX+VuuRaDl2vVvE/dOhQMjMziY+PJy0tjbZt27Jq1SoaNWoElE3ePXz4sC3e09OTxMRE\nRo8ebduka/z48YwdO9YWEx4ezsqVKxk3bhzz588nJCSEOXPmMGjQIFvMuXPneOKJJ0hJScFsNtOy\nZUveffddhg4dWmVuoaGhV/3liIiIiIjcSKq1zv+N4uK7Jj35l9pOT6WkrlGflbpE/VXqkmtZw1Zr\nh18REREREan7VPyLiIiIiNQTKv5FREREROqJWr/Jl4iIiEh9YLVaKSoqorS01N6pSA0yGo04OTlh\nMBhq5P3qbfF/5NgZwht72zsNEREREaxWK/n5+ZhMphotBMW+rFYrpaWl5Ofn4+LiUiP/7vV22M+/\nP9ht7xREREREACgqKsJkMuHg4KDCvx4xGAw4ODhgMpkoKiqqkfest8X/tqRUkvdn2DsNEREREUpL\nSzEa621ZVu8ZjcYaG+5Vr3vZ0vd3UY+2ORAREZFaTE/866+a/Lev18X/3n2nSNqVbu80RERERERq\nRL0u/gH+9d9dlJbq6b+IiIiI3PjqbfFvcnIA4NDRLDZvTbFzNiIiIiI3phdeeAGj0cjJkyev+3ut\nXbsWo9HIunXrrvt71VX1tvgfEB1p+/t/PtxNSYnW1BURERG51gwGQ42Oadfcid9Wb4v/e+9shavZ\nCYBUSw5fbzhi34RERERE5Kr06tWLvLw8brvtNnunUmvV2+Lf3d3EoIEtbMfvfbSHwsISO2YkIiIi\nIhfLy8u77HNMJtN1yOTGUW+Lf4CBMc3w8nQBIDMrj89W77dzRiIiIiI3ptOnTzN8+HC8vb3x9fUl\nLi6O3Nxc2+vh4eEMGDCAr776ii5dumA2m5k+fTpQtg7+Sy+9VOGa4eHhPPLII7bjysb8jxgxArPZ\nTGpqKoMGDcLDw4MGDRowfvz4Cku+v//++9xyyy14eXnh6elJ69at+fvf/36tvwq7qtfFv9nFifvv\naW07/vB/P5J9Nt+OGYmIiIjceKxWK8OGDSM7O5upU6cyZMgQ3nrrLe6//35bjMFgYP/+/QwZMoTe\nvXszZ84cunTp8pvXrWx8/6/bDAYDVquVfv36ERAQwD/+8Q969+7NjBkzeOutt2xxX375JQ888AA+\nPj688sorvPbaa/Tv35+NGzde5aevXRztnYC93dErgs++2E+qJYe8/CL++8leHvvDzfZOS0REROSG\nEhoaymeffWY7DgoK4u9//ztfffUVffv2BeDQoUP873//Y+DAgdf0vYuKirj//vt59tlnAXj88cfp\n1KkTixYtYtSoUQCsXLkST09Pvvjii2v63rVNvS/+HR2NPDSsPVNnbQDg8zUHGRDdjJAgDztnJiIi\nIlK59z7aw/sf77lu1x86qA3D7m1zza5nMBj485//XK7tySefZMqUKXz22We24r9Ro0bXvPD/xWOP\nPVbuuGfPnixdutR27OXlRW5uLp9//jn9+vW7LjnUBvV62M8vojoG07pFAAClpVaWvr/LzhmJiIiI\n3FgiIyPLHfv5+eHj48ORI0dsbREREdflvZ2cnAgMDCzX5uPjQ1ZWlu34iSeeoEWLFgwcOJBGjRox\nYsQIPv300+uSjz2p+KfsbnTEA+1tx1u2pbB33yk7ZiQiIiJS/5jN5suKLymp3kqNRmPVJW9AQAA/\n/PADK1as4L777mPz5s3cfffd3H333ZeVU21X74f9/CIywpceXRqzYcsxAN75z05eeb6vNooQERGR\nWmfYvdd2WE5N2L9/f7mn/6dPnyYrK4smTZpUea6Pjw9nzpwp11ZUVERaWto1zdHR0ZF+/frZhv1M\nmjSJV199lc2bN9O1a9dr+l72oif/F3lwyE04OpZ9JfsPnWbTdyl2zkhERESk7rNarbzxxhvl2mbP\nno3BYKjWGP+mTZuWW74TICEhodpP/qsjMzOzQluHDh0AKtx41GV68n+RwAA3BkY345NV+wBY+v5O\nOt8cgsnJwc6ZiYiIiNRtJ06cYMCAAdx5550kJSWxcOFC+vXrZ5vs+1see+wx4uLiGDx4MNHR0ezY\nsYPVq1cTEBBQIfbXa/dX12OPPUZGRga33347oaGhpKSk8OabbxISEnJD7RisJ/+/ct/vWuHuVrYz\nXPqpXD7/8oCdMxIRERGp24xGI//5z3/w8fHh2Wef5YMPPuDxxx/n/ffft8UYDIZLDrceOXIkEydO\nZP369fzlL3/h6NGjJCYm4ubmVum6/r92qete3P6HP/wBDw8PEhISGD16NG+//TZ33nknGzZswM3N\n7Uo+dq1ksF7p7VEdlJ2dbfu7q7sHTg6V3/t8+vlP/PPfSWVxZifeeLU/3l4uNZKjyC+2bdsGQFRU\nlJ0zEake9VmpS2pbf83Pz8fFRbVGffZbfeDiGtbLy+uq3qfePvn/MDn9kq/1u6MpwYFl6/yfzyvi\nPx/trqm0RERERESum3pb/L+3J42Us/mVvubk6MDDD15Y+jPx68McOpJVaayIiIiISF1Rb4v/olIr\nc7Yeo/QSo56iOoTQsV3Qz0dWFi1NuuIJJCIiIiIitUG9Lf4Bdp/KYfWh05d8/eHhHTAayyaC/PjT\nKS39KSIiIiJ1WrWL/7lz5xIREYHZbCYqKooNGzb8Zvzu3bvp3bs3rq6uhIaGMmXKlAoxa9euJSoq\nCrPZTGRkJAkJCeVeX7hwIbfddhu+vr74+PjQt29fNm7cWC7mxRdfxGg0lvsTEhJS3Y/F20kpZOYV\nVfpaoxBPBkQ3sx2/894OCgqKq31tEREREZHapFrF/7Jlyxg7diyTJ08mKSmJbt260b9/f1JSKn8S\nnpOTQ3R0NMHBwWzfvp3Zs2czffp0Zs6caYs5cuQIAwcOpEePHiQlJTFx4kTGjBnD8uXLbTFr165l\n2LBhfP3113z33Xe0aNGC2NhYDh48WO79WrZsSXp6OhaLBYvFwq5du6r8TCHuzgDkFpWQ8P3xS8YN\nHdQaT4+y2IzT5/l45b4qry0iIiIiUhtVq/ifOXMmjzzyCI888ggtWrTg9ddfJzg4mHnz5lUav3Tp\nUvLy8njnnXdo1aoV9957LxMmTGDGjBm2mHnz5tGwYUNmzZpFixYteOyxx3jooYd47bXXbDHvvvsu\nTzzxBO3bt6dZs2bMmzcPDw8PPv/883Lv5+joSEBAAA0aNKBBgwb4+flV+Zn+3Lmx7e8bjmfx7YnK\nd25zdzMxfPBNtuPlnyVzKiO3yuuLiIiIiNQ2VRb/RUVFbN++nejo6HLtMTExbNq0qdJztmzZQs+e\nPTGZTLa22NhYUlNTOXr0qC0mJiam3HmxsbFs27btkls1FxQUkJ+fj4+PT7n2Q4cO0bBhQyIiInjg\ngQc4fPhwVR+L9oGeRDe5cJMwd9txzhdV/r539GpCeGNvAAqLSliybGeV1xcRERERqW2qLP4zMjIo\nKSkhMDCwXHtgYCAWi6XScywWS6XxVqvVds6lYoqLi8nIyKj0upMnT8bDw4O77rrL1talSxcWL17M\nF198wcKFC7FYLHTr1o2srKqX5ny0QyO8nZ3KPmdeIW8nVT6MyWg08OjvO9qON357nN0/nqzy+iIi\nIiIitYmjvROortmzZ7NgwQLWrFmDu7u7rT02NrZcXJcuXWjSpAnvvPMOY8eOveT1ftnZr49HMf/K\nLJvw++HOY/ids9DMw6HScyIaO7Hrx7Id1l6ZtYY/P9IUx0vsEixyrfzSV0XqCvVZqUtqS38NCwvT\nDr/1XE5ODrt3V76xbLNmzSptvxJVVq7+/v44ODiQnl5+R9z09HSCgoIqPScoKKjSeIPBYDvnUjGO\njo74+/uXa581axbPPfccq1atolOnTr+Zr6urK23atGH//v1VfTQA2ns50NbrQrH/35QiCkoqX8+/\nf98gnE1lX9mpjAI2bb30MqEiIiIiIrVNlU/+nZyc6NSpE4mJidx333229sTERIYMGVLpOV27dmXi\nxIkUFhbaxv2vXr2akJAQwsLCbDEff/xxufNWr15NVFQUDg4XivEZM2bw4osvsnLlSrp27VrlB8rP\nzyc5OZm+ffv+ZlxUVJTt781uKiJu5V5yCospAH4wBvDnqMaVnpdb4Mvi/+wA4LukPP74QGv8/Vyr\nzEvkcv3yNOrivipSm6nPSl1S2/prfn6+vVMQO/Pw8Lhkf8zOzr5m71OtMStPP/00ixcvZtGiRSQn\nJ/PUU0+RlpZGXFwcAJMmTeKOO+6wxQ8fPhxXV1dGjBjBnj17+Oijj5g2bRrPPPOMLSYuLo4TJ04w\nbtw4kpOTWbhwIUuWLGH8+PG2mOnTpzNp0iQWLVpEZGQk6enppKenc/bsWVvM+PHjWbduHUeOHOHb\nb79l8ODBnD9/noceeqjaX4K3ixN/6hRqO1518BRJlrOVxg6IbkbjRl4AFBYW8/a/kqr9PiIiIiJS\nNxmNRp544gl7p3HVqlX8Dx06lFmzZhEfH0/Hjh3ZtGkTq1atolGjRkDZ5N2LV9jx9PQkMTGR1NRU\nOnfuzJgxYxg/fny5Mfjh4eGsXLmS9evX07FjR6ZOncqcOXMYNGiQLWbu3LkUFxdz//33ExISYvtz\n8XVSUlIYPnw4LVu2ZPDgwZjNZrZs2UJo6IVivjpua+xDt0betuPZ3x0lr5LVfxwdjTz+0M224y3b\nUvh+R9plvZeIiIiIiD0YrFZr5QPcb0AX/2Ti5eVV4fWsvCL+tKps+A/AgMgARl9i+M+cBd/x9foj\nAAQ2cGf21FhMTpVPFBa5ErXtJ2mRqqjPSl1S2/prfn6+JvzWckajkbi4OObOnXtdrv9bfaCqGvZy\naKmai/iYyw//WXngFDvSKx/+84eh7XBzLZvPkH7yHB99mlwjOYqIiIiIXCkV/79yW2Mfuja8MPxn\n1reVD//x9nLhwSFtbcfLP0vGkn6uRnIUERERqWssFgujRo0iNDQUFxcXmjRpwuOPP05ubi5ZWVmM\nHz+e9u3b4+npiYeHB3369GHDhg3lrnH06FGMRiOvvvoqCxcuJDIyEhcXF2655ZYKy7b27t270gVg\nRowYQZMmTSq0v/nmm3Ts2BFXV1cCAgKIiYlh48aNv/mZZs6cidFoZOrUqba2uXPn0q5dO9zd3fH2\n9qZjx4689dZbl/NVXVd1Zp3/mmIwGBgd1Zjdp86RU1jMyfOFJHx/nLG3hleIjenTlDXrjnDwcCZF\nxSUkvPM9z43vicFgqPnERURERGqp9PR0OnfuTGZmJqNGjaJ169akpqayfPlyTp8+zalTp/joo48Y\nOnQoERERnDlzhkWLFhEdHc3WrVtp27Ztueu999575ObmEhcXh8FgYNq0adx3330cOnTItmrkpeox\ng8FQ4bWRI0eyaNEi+vfvz8MPP4zVamXTpk2sW7eO7t27V3qd+Ph4nnvuOWbPns2YMWMAWLRoEX/+\n858ZOnQoY8aMoaioiD179rB582Yef/zxq/0arwkV/5XwMTvxRKdQpm0um8ScePg0tzb0putFE4Kh\nbOffUQ/dzIQX12C1Wtmx28LaTUfp3T3cDlmLiIhIffGv3an8e/f1W3BkeNtgHmwbcs2uN2HCBCwW\nC5s3by43z+K5554DIDg4mIMHD5Y75/HHH6dFixa8/vrrFZ6cp6SkcODAATw9PQFo3rw5gwYN4osv\nvmDAgAGXlds333xjK9pff/11W/tTTz11yXOeffZZpk2bxvz58xk5cqStfcWKFbRt25b33nvvsnKo\nSRr2cwm3hfnSq7Gv7fj1rUfJyiuqEBcZ4cuA6Ejb8T//tYOzOQU1kqOIiIhIbWe1Wvn4448ZMGDA\nJSdYOzk52f5eUFBAZmYmRUVFdO7cme3bt1eIHzx4sK3wB+jZsydWq5VDhw5ddn4ffPABBoOBF154\noVrx48aNY/r06bzzzjvlCn8om4ybkpJSa3aOroyK/9/wp06h+JvLJvWeLShm9tajVLY40vDBbW0b\nfeWcK+CfWvtfREREBIBTp05x9uxZ2rRpc8kYq9XKK6+8QtOmTTGbzfj7+9OgQQNWrFhR6QZXv17S\n3du7bHRGVlbWZed36NAhAgMD8fX1rTJ26dKlvP7668yaNYsHH3ywwusTJkzAw8ODW265hcjISP70\npz/xzTffXHZO15OG/fwGD2dHxt0axrPf7Adga2o2nx/MoH9kQLk4s4sTo0Z0Iv4f6wFYu+kovbqH\n0eGmoBrPWURERG58D7YNuabDcuzt5Zdf5m9/+xsPP/ww8fHx+Pn54eDgwMsvv1zp0/xfxvX/2sUP\naS815r+kpOJCLtXVvXt3du3axZtvvsnQoUPx9/cv93rLli3Zt28fK1eu5IsvvmDFihUkJCQwevRo\n5syZc8Xvey3pyX8VOgR5Mqh5A9vxgh9SOJFTcQvuTu2D6X7rhbvQ+f/cTkFBcY3kKCIiIlJbBQQE\n4Onpye7duy8Z88EHH9CnTx8WLVrEsGHDiI6Opm/fvuTnV6y5qsvHx4czZ85UaD969Gi546ZNm5Ke\nns7p06ervGaTJk348ssvOXXqFDExMZw9W3FJeBcXF+69914SEhI4cuQIDz74IHPnziUtrXZsCqvi\nvxoeat+Qxp5mAApKSvnHliOUlFYc/vPo7zvi7lY2TOhkRi7Llu+p0TxFREREahuDwcA999zDqlWr\n2Lp1a6UxDg4OFYZWb9q0ic2bN1/x+zZt2pTk5ORyRf2OHTsqLN85ePBgrFZrtcf8t2zZki+++ILD\nhw/Tr18/zp8/b3stMzOzXKzRaLStVFTZjYg9aNhPNZgcjPylazhPr06m2Gpl3+lclu21MLxtcLk4\nby8XHhrWnjcXlXXsT1b9RI8ujYkI97FH2iIiIiK1wtSpU0lMTKRXr162pT4tFgvLly/n448/5q67\n7uKFF17goYceomfPnvz0008sWLCANm3acO7cle2j9MgjjzBjxgxiYmJ49NFHSU9PJyEhgbZt25Z7\nYt+rVy9GjBjB3LlzOXDgAP379wdg8+bNtG/fnokTJ1a4dseOHVmxYgUxMTHcddddrFixAmdnZ2Ji\nYmjQoAE9evQgKCiI/fv388Ybb9C+fXtatWp1ZV/eNaYn/9XU1MeV3990YWzdf/ak8WNGxc7Y97Zw\n2rYqGyZktVqZu2gbJSWlNZaniIiISG0TFBTEd999xwMPPMB7773Hk08+yeLFi+nSpQv+/v5MmjSJ\nCRMm8PXXX/PUU0+xdu1ali1bRqdOnSqM3a9snf7K2lu2bMm7777L2bNneeaZZ/jss89YunQpHTt2\nrHD+22+/zYwZM0hJSWHChAn8/e9/Jysri169el3y+t26deOTTz5h06ZNDBkyhJKSEuLi4sjPz2f2\n7NmMHj2a999/n8cee4zVq1dfq6/yqhmslS1fc4O6eLa4l5fXZZ9farUy8auf2HOqrOhv4GpiTr9W\nuJvK/4CSaslh3F9XU1RcNqHkwcE3cd9dteNuT+qOX5YJu9SyaCK1jfqs1CW1rb/m5+fj4uJi7zTE\njn6rD1xtDXsxPfm/DEaDgWe6hOPuVDbD/OT5QuZsPVZhjFpIkAf333thOatlH+/hWErFZapERERE\nRGqSiv/LFOjmzJO3hNmONxzP4otDFWeH392/OZERZevFFheXMmfBVg3/ERERERG7UvF/BbqH+tC/\n6YW1/hO+P87R7LxyMQ4ORv78WGccHcu+4oOHM/l4xb4azVNERERE5GIq/q/QyI6NbMt/FpaU8uqm\nwxQUl3+y37iRF8Pu0fAfEREREakdVPxfIWdHIxO7N8FkLPsKj2TnsSgppULc3QNaaPiPiIiIiNQK\nKv6vQpiXmcdvbmQ7XnHgFJuOZ5WL0fAfEREREaktVPxfpX5N/enWyNt2PPu7o1jOFZSL0fAfERER\nEakNVPxfJYPBwJOdwwhwNQFwrqiEqRsPUfiroT2/Hv4ze/63tn0AREREROrR1kvyKzX5b6/i/xrw\ncHZkYrcmOP6869uBrPMs+KH8+P9fhv84OZbtEXD42BmWfbS3xnMVERGR2sdkMpGfn09JiR4M1jcl\nJSXk5+djMplq5P0cqw6R6mjp786jHRuR8P1xAFYeOEVrfzf6hPvZYho38uL3Q2/in/9OAmD5imRu\nbh9E6xYBlV5TRERE6gej0YiLiwuFhYUUFRXZOx2pQQaDARcXFww/P0S+3lT8X0O/axbAnlPn2PDz\npN85W48R4eNKmJfZFjMwphnbk9LYuTcdq9XK6wnfMSM+Blezk73SFhERkVrAYDDg7Oxs7zTkBqdh\nP9eQwWDgqVvCaOjhAkBBSSlTNx4ir+jCT3hGo4Exj3fG7ec5Aiczclm09Ae75CsiIiIi9YuK/2vM\n1cmBv3aPwORQ9tUeP5vPnK3Hyk3k8PN1ZdSIm23HX68/wuatFfcIEBERERG5llT8Xwfh3mZGRzW2\nHa89lsnKAxnlYnp0aUzPrhdi5r29jcwzeTWWo4iIiIjUPyr+r5M7mvgRG+FvO17ww3GSM86Vixn5\nh5vx9SmbD3Aut5A3F27VMl8iIiIict2o+L+ORt0cSoS3KwBFpVbiNxwiM+/CDH53dxNPPn6L7fiH\nnRY++2J/jecpIiIiIvWDiv/ryNnRyLM9IvAwlS2qlJlfxMsbD1F00QZg7doEcle/FrbjJct2cuBw\nZo3nKiIiIiI3vmoX/3PnziUiIgKz2UxUVBQbNmz4zfjdu3fTu3dvXF1dCQ0NZcqUKRVi1q5dS1RU\nFGazmcjISBISEsq9vnDhQm677TZ8fX3x8fGhb9++bNy48apzq0lB7s5M6NYE489rt/6Ycc62F8Av\nHhzalohwHwBKSkr5x5tbOJ+nNX5FRERE5NqqVvG/bNkyxo4dy+TJk0lKSqJbt27079+flJTKV6jJ\nyckhOjqa4OBgtm/fzuzZs5k+fTozZ860xRw5coSBAwfSo0cPkpKSmDhxImPGjGH58uW2mLVr1zJs\n2DC+/vprvvvuO1q0aEFsbCwHDx684tzsoWOQJyPaNbQdrzqYwecHL0wAdnJ04JknuuDiUvYLQfrJ\nc7y1+HuN/xcRERGRa8pgrUaF2aVLFzp06MD8+fNtbc2bN2fIkCHEx8dXiJ83bx6TJk3i5MmTtq2K\n4+PjmT9/PsePlz31njBhAh9//DH79u2znTdy5Ej27t1b6dP9XwQHBzN58mRGjx592bllZ2fb/u7l\n5VXVx76mrFYr0zcfYe2xsiE9jgYDr9zenFb+7raY9ZuPMXPeFtvxmJG30KdneI3mKbXHtm3bAIiK\nirJzJiLVoz4rdYn6q9Ql17KGrfLJf1FREdu3byc6Orpce0xMDJs2bar0nC1bttCzZ09b4Q8QGxtL\namoqR48etcXExMSUOy82NpZt27ZRUlJCZQoKCsjPz8fHx+eKc7MXg8HAk7eE2SYAF1utvLzhEKfP\nF9pienZtTN+eTWzHCe98z4m0szWeq4iIiIjcmByrCsjIyKCkpITAwMBy7YGBgaxZs6bScywWC6Gh\noRXirVYrFouFsLAwLBZLhaI9MDCQ4uJiMjIyKrwfwOTJk/Hw8OCuu+664tx+8csdf037nWcps9ML\nOF8M53Ph6U+2EtfUhJOxbE7AzW2sbP6umIzMQnJz4dmXVhL3UASOjpqbXV/Zq6+KXCn1WalL1F+l\nLmjWrNl1QKJfAAAgAElEQVQ1u1adqShnz57NggULWL58Oe7u7lWfUEv5moz8vrGJn+f/cux8KR+m\nFNnG95tMRoYNCsXRsSwg7WQ+q76y2CtdEREREbmBVPnk39/fHwcHB9LT08u1p6enExQUVOk5QUFB\nlcYbDAbbOZeKcXR0xN/fv1z7rFmzeP755/n888/p1KnTVeX2C3uO8YsCzCHpLPihbFLyj4VwxK0h\nQ1pfyNnROZgFS74HYFdyPtF9G9CjS+PKLic3KI1HlbpGfVbqEvVXqUsuHvN/tap88u/k5ESnTp1I\nTEws156YmEj37t0rPadr166sX7+ewsIL49lXr15NSEgIYWFhtphfX3P16tVERUXh4OBga5sxYwbP\nP/88K1eupGvXrledW21xd/MGxFy0A/DinSfYeDzLdtzv9qZ0iWpkO5779jZSUjX+X0RERESuXLWG\n/Tz99NMsXryYRYsWkZyczFNPPUVaWhpxcXEATJo0iTvuuMMWP3z4cFxdXRkxYgR79uzho48+Ytq0\naTzzzDO2mLi4OE6cOMG4ceNITk5m4cKFLFmyhPHjx9tipk+fzqRJk1i0aBGRkZGkp6eTnp7O2bNn\nq8xt1KhRV/3lXE8Gg4EnOoXSNsDD1vbaliPsz8y1vT76sSiCAsuGOOXnFzN9ziby8rX+v4iIiIhc\nmWoV/0OHDmXWrFnEx8fTsWNHNm3axKpVq2jUqOzJtMVi4fDhw7Z4T09PEhMTSU1NpXPnzowZM4bx\n48czduxYW0x4eDgrV65k/fr1dOzYkalTpzJnzhwGDRpki5k7dy7FxcXcf//9hISE2P5cfJ1L5fbr\nCce1kZND2Q7AIe7OABSWlDJl/UEyfl4ByM3VxPg/d8PJseyXkOMnzjL/n9u1/r+IiIiIXJFqrfN/\no7DnOv+/5fjZfP6SmMy5orIlTiN9XJl2e3Ncfi76v15/hDkLvrPFj/zjzfS/I9IuuUrN0XhUqWvU\nZ6UuUX+VuqRG1/mX6y/U04VJ3SMw/rwE0IGs8/xjyxFKf74v69MznOjeEbb4t/+VxP6Dp+2Sq4iI\niIjUXSr+a4kOQZ78qdOFoUqbUs6weMcJ2/Gjf+hIRFjZ5mYlJaVMf2MzZ3MKajxPEREREam7VPzX\nIgMiA7i7eQPb8YfJ6Xz600kATE4OjH+yG26uZbsmZ5w+z4y5WygpKbVLriIiIiJS96j4r2Ue69iI\nW0MujOVK+D6FTT8vARoY4MaTo26xvbZzTzpL399V4zmKiIiISN2k4r+WMRoM/F+3JjT3dQPAipXp\nm4/wY8Y5ADp3DGHI3a1t8Z+s2sfajUftkaqIiIiI1DEq/mshF0cHnr+t6YUlQEtLeXHdQU7k5ANw\n/z1tiOoYYoufu2gbBw5n2iVXEREREak7VPzXUt4uTrzYKxJPZ0cAcgqL+ds3BziTX4TRaGBs3K00\nCvEEoKi4hFdmbeRMdr49UxYRERGRWk7Ffy0W4uHCC7dFYnIo+2dKzy3ghXUHyS8uwdXsxMSx3W0T\ngDOz8nh1ziaKikvsmbKIiIiI1GIq/mu5Fn5uTOzWxLYHwP7MXF7eeJiiklJCgjwY96dbMfz8WvJP\nGSx89wd7pisiIiIitZiK/zrg1obe5fYA2J6Wzcxvj1JqtXJz+2B+P/Qm22uJXx/i8zUH7JGmiIiI\niNRyKv7riAGRAQxrHWw7Xnssk/nbj2O1Whk0oAU9ujS2vbZgyQ8k7bLYI00RERERqcVU/Nchv78p\nmIGRAbbjFQdOsXR3GgaDgdGPRhERXrYDsNVqZfqczRxLybZXqiIiIiJSC6n4r0MMBgNxnUK5rbGP\nre29PWl8su8kzs6O/HVcD3x9zADk5Rfx8owNWgFIRERERGxU/NcxRoOBp28Np1PwhV2A3/rhOGsO\nn8bXx8yzT/fExaVsedCTGbm8MmsjhYVaAUhEREREVPzXSU4ORv7aPYJW/u62tlnfHeXbE2doEubN\nuD91sa0A9NPB08xZ8B2lpVZ7pSsiIiIitYSK/zrKxdHI8z2b0sS7bJhPqdXK1I2H+D7tLJ07hvDw\n8Pa22I3fHmfZ8j32SlVEREREagkV/3WYh7MjU3o1I9jdGYCiUitT1h9kR3oOA2Oa0e/2SFvsfz/Z\ny9frj9gpUxERERGpDVT813E+Zifi+zQj4OedfgtLS3lx3QH2ZuTy6O870OGmIFvsm4u28v2ONHul\nKiIiIiJ2puL/BhDo5szUPs3xMzsBUFBSyvNrD3DgTB7PjO5CWKg3AKWlVqa/sZkDhzLtma6IiIiI\n2ImK/xtEsIczL/dpjrdz2Q1AXnEJf/tmP2n5xfztLz3x93MFoKCgmPgZG7Ckn7NnuiIiIiJiByr+\nbyCNPF14uW8zPJ3LlvrMLSph8jf7yTZY+dtfbsPdrWxoUPbZfF6cvk57AIiIiIjUMyr+bzBhXmbi\nezfDw1R2A5BTWMyzX++n2M2Rvz7dAydHBwDST54jfsYG8vKL7JmuiIiIiNQgFf83oAgfV17qFYnr\nz4V+dkExk77aj8nflWdGX9gD4ODhTF57YzPFxaX2TFdEREREaoiK/xtUcz83pvS+cAOQU1jMX7/e\nj08Tbx5/6GZb3A87LdoETERERKSeUPF/A2vp7058n2a4O124AXj26/2Etw9iyN2tbXHrNx/jrXe+\nx2rVDYCIiIjIjUzF/w2uuZ8b8X2a2+YA5BaV8OzX+2nXM4zYvk1tcau/Psi77++yV5oiIiIiUgNU\n/NcDkb6uvNznwipAecUlPLfuAF37NeO2rmG2uI9XJPPRpz/aK00RERERuc5U/NcTET6uTL1oH4D8\n4lJeWH+Qnne1JKpjiC1u6X93serLA/ZKU0RERESuIxX/9Ui4t5lXbm+Or8uFnYCnbDxIj3tacVPr\nBra4BUu+55uNR+yUpYiIiIhcL9Uu/ufOnUtERARms5moqCg2bNjwm/G7d++md+/euLq6EhoaypQp\nUyrErF27lqioKMxmM5GRkSQkJJR7fe/evQwZMoSmTZtiNBp56aWXKlzjxRdfxGg0lvsTEhJSIU7K\nhHq68MrtzfE3l234VVRqZfq3R7n1nlY0i/Czxc15aysbvz1urzRFRERE5DqoVvG/bNkyxo4dy+TJ\nk0lKSqJbt27079+flJSUSuNzcnKIjo4mODiY7du3M3v2bKZPn87MmTNtMUeOHGHgwIH06NGDpKQk\nJk6cyJgxY1i+fLkt5vz58zRp0oT4+HgiIiIumV/Lli1JT0/HYrFgsVjYtUsTV39LQw8XXr2jOSHu\nzgCUWq28+UMKN9/TisaNvACwWq3MmLuFzVsr/zcWERERkbqnWsX/zJkzeeSRR3jkkUdo0aIFr7/+\nOsHBwcybN6/S+KVLl5KXl8c777xDq1atuPfee5kwYQIzZsywxcybN4+GDRsya9YsWrRowWOPPcZD\nDz3Ea6+9ZouJiori1VdfZdiwYZjN5kvm5+joSEBAAA0aNKBBgwb4+fldMlbKBLo5M/2OFkR4u9ra\nlvyYRutBLQkJ9gTKbgD+8eZmvt1+wl5pioiIiMg1VGXxX1RUxPbt24mOji7XHhMTw6ZNmyo9Z8uW\nLfTs2ROTyWRri42NJTU1laNHj9piYmJiyp0XGxvLtm3bKCkpuawPcejQIRo2bEhERAQPPPAAhw8f\nvqzz6ytvFyem9m1Ga393W9snhzKI+F1zggI9ACgttfLaG5vZ+n2qvdIUERERkWvEsaqAjIwMSkpK\nCAwMLNceGBjImjVrKj3HYrEQGhpaId5qtWKxWAgLC8NisVS4oQgMDKS4uJiMjIwK73cpXbp0YfHi\nxbRs2ZKTJ08yZcoUunXrxt69e/Hx8bnkedu2bavW9euDezyt5GQWsi+nFIBP952nzc3uOK/JIjOr\nEIDnp65m+H1htIz0sGeq9ZL6qtQ16rNSl6i/Sl3QrFmza3atOr/aT2xsLIMHD6Zt27b07duXFStW\nUFpayjvvvGPv1OoMk9HAiHAT7b0dbG17zoNz71C8fMvmBZSUwr8/OsZPB3PslaaIiIiIXKUqn/z7\n+/vj4OBAenp6ufb09HSCgoIqPScoKKjSeIPBYDvnUjGOjo74+/tf1oe4mKurK23atGH//v2/GRcV\nFXXF73GjuqWzlbnbjrHqYAYAp4CGd7bFY8MxstJzAfjki0z+76mWdGofbMdM64dfnkapr0pdoT4r\ndYn6q9Ql2dnZ1+xaVT75d3JyolOnTiQmJpZrT0xMpHv37pWe07VrV9avX09hYaGtbfXq1YSEhBAW\nFmaL+fU1V69eTVRUFA4ODlyp/Px8kpOTCQ5WcXq5jAYDo6MaM7zthe/uRF4h+bc2xD2obF5AUXEJ\nr8zaqFWAREREROqgag37efrpp1m8eDGLFi0iOTmZp556irS0NOLi4gCYNGkSd9xxhy1++PDhuLq6\nMmLECPbs2cNHH33EtGnTeOaZZ2wxcXFxnDhxgnHjxpGcnMzChQtZsmQJ48ePt8UUFRWxY8cOkpKS\nyM/Px2KxsGPHDg4ePGiLGT9+POvWrePIkSN8++23DB48mPPnz/PQQw9d9ZdTHxkMBh5sG8JTncMw\nGgwAnCku4dzNQbiElI33Lykp5bU3NrN241F7pioiIiIil6nKYT8AQ4cOJTMzk/j4eNLS0mjbti2r\nVq2iUaNGQNkE34tX2PH09CQxMZHRo0fTuXNnfHx8GD9+PGPHjrXFhIeHs3LlSsaNG8f8+fMJCQlh\nzpw5DBo0yBaTmppKx44dMfxchCYkJJCQkECvXr346quvAEhJSWH48OFkZGQQEBBAly5d2LJlS4UJ\nx3J5Ypr642N2YurGQxSUlFKAlcJ2Abg5Gik6lo3VamV2wncUFJYQ0+fSezCIiIiISO1hsFqtVnsn\nUVMuHi/l5eVlx0zqjp9O5/LiuoOcKSgCoLTEivlAFtaDWRh+jnnkwQ7cGdvcfkneoDQeVeoa9Vmp\nS9RfpS65ljVsnV/tR66v5n5u/CO6BQ09XAAwOhjIi/ShoJUfv9w1vv2vJD7834/2S1JEREREqkXF\nv1QpyL1sN+BWP28GZnQwYA3z4uxNAZQYy57//+uDXbz7/k7q0Q9JIiIiInWOin+pFi9nR17u04xe\njX0BMBoNmBp6cLpdAAWmsm60/LNk3ly4jZKSUnumKiIiIiKXoOJfqs3kYGR813B+3zYEKLsB8Az2\n4HTbAHJdy+aOf7X+MNNe30RhYYk9UxURERGRSqj4l8tiMBh4oG0wE7o1wWQ0YjCAb6A7WW38yfIu\n2w142w+pvPjqOs7lFlZxNRERERGpSSr+5Yrc1tiXV25vjo+LEwYDBAS4kdfSD0ugK1bgx59OMTn+\nazKz8uydqoiIiIj8TMW/XLEWfm7MjGlJhLcrAH6+ZgyRPhwN86DYwcCxlGz+OuUrUi05ds5URERE\nREDFv1ylAFcTr97enG6NvAHw9nLBPcybwxFe5Lk4cDIjl4kvrmHvvlN2zlREREREVPzLVTM7OfDX\n7hE81K4hBgx4uJvwb+jJsSZenPEycS63kBemrWXDlmP2TlVERESkXlPxL9eEwWBgaOsgXujVFHcn\nB1xdnQgO9iA91IO0IFeKikuZMXcLH/7vR+0FICIiImInKv7lmooK9mJmTEsae5pxdnGkYbAn5xq4\ncTTMgyJHA//6YBdzF22juFh7AYiIiIjUNBX/cs2FeLgwI7oF3UN9cHIy0ijYA6uvmcNNvDjn5sia\ndYf5+z/Wk3teS4GKiIiI1CQV/3JdmJ0cmNStCSPaNcTR0UhwkAdmbxeONfbklL+ZHXvS+euUr7Ck\nn7N3qiIiIiL1hop/uW4MBgNDWgfx996R+Lg40SDADR9vM6cCzBxr7MFhSw7/98KX7NyTbu9URURE\nROoFFf9y3bUP9OSNfq24qYEHvj4uNAhw47y7E4civDhZWsqLr65jZeJ+TQQWERERuc5U/EuN8DE7\nEd+7GcNaB+PhbiIkyAOrswNHwj055evMgnd/YP4/t1NUXGLvVEVERERuWI72TkDqDwejgT+0C6FV\ngBuvbT6Co6MRy8lznGzgSq6bEys3HOZ46ln+b0w3vL1c7J2uiIiIyA1HT/6lxkUFezGnXytuCvKk\nYbAnHu4mct3KhgFtTc3m/174koNHsuydpoiIiMgNR8W/2EWAq4lX+jZnWJsgAgPc8PM1U+Jg5Hio\nB7ucYOKUr/hy7SF7pykiIiJyQ1HxL3bjaDTwx3YNeblvcyKDPAkOdMdoNJDl48JPoe7MWPoDbyzY\nSmGh5gGIiIiIXAsq/sXu2jXw4I1+rbijeQCNQjwxmRwocHbgcBNP/rsnjQkvrcFyUvsBiIiIiFwt\nFf9SK3g4O/LX7hE8060JTUO98HA3YTUYSA90Za2hhCdfWsPW71PtnaaIiIhInabiX2oNg8FAbFN/\n5vRrTZdm/gT4uWIwwHlXJ3YHu/J/i7fy7rKdlJSU2jtVERERkTpJxb/UOo08XZgR3ZKRXcJoFOyJ\no6ORUqOBtGA3Zu88wfip33AqI9feaYqIiIjUOSr+pVZycjDyx3YNeePONnRu5o+r2QmAc+5OfEEJ\nD7/yNVu2pdg5SxEREZG6RcW/1Got/NyYd2db4npG4OtjBqDEwcABPxee/HAns/+5ncIirQYkIiIi\nUh0q/qXWc3E08ninUObf246bwn1wdCzrtmc9Tbx1IpM/vvw1x1Ky7ZyliIiISO2n4l/qjHYNPFgy\nuAMjuobj5mYCyn4FSDLB4Le2sOyLn7BarXbOUkRERKT2UvEvdYrZyYG/9IhgwZD2NAvywGAoaz9r\nduDFbUd5eOZ6Tmfl2TdJERERkVqq2sX/3LlziYiIwGw2ExUVxYYNG34zfvfu3fTu3RtXV1dCQ0OZ\nMmVKhZi1a9cSFRWF2WwmMjKShISEcq/v3buXIUOG0LRpU4xGIy+99NI1yU3qvpuDvfjvH6IYcUsY\nzk4OAJQaDWwpKKTfnPUsX3fYzhmKiIiI1D7VKv6XLVvG2LFjmTx5MklJSXTr1o3+/fuTklL5ais5\nOTlER0cTHBzM9u3bmT17NtOnT2fmzJm2mCNHjjBw4EB69OhBUlISEydOZMyYMSxfvtwWc/78eZo0\naUJ8fDwRERHXJDe5cbg4OjChbzP+9ftONPl5MjBAjpORv244yENzN3L6rH4FEBEREfmFwVqNQdJd\nunShQ4cOzJ8/39bWvHlzhgwZQnx8fIX4efPmMWnSJE6ePInJVDY2Oz4+nvnz53P8+HEAJkyYwMcf\nf8y+ffts540cOZK9e/eycePGCte86aabGDJkCM8999wV55adfWFSqJeXV1UfW+qQopJSZq7Zz7tJ\nKRSVXOjSbgYDT/eMYHi3cAy/jBGqI7Zt2wZAVFSUnTMRqR71WalL1F+lLrmWNWyVT/6LiorYvn07\n0dHR5dpjYmLYtGlTpeds2bKFnj172gp/gNjYWFJTUzl69KgtJiYmptx5sbGxbNu2jZKS6i3deCW5\nyY3JycHI/8W04L9/7EyEu4utPddqZcq6gwxJ2MyR0+fsmKGIiIiI/TlWFZCRkUFJSQmBgYHl2gMD\nA1mzZk2l51gsFkJDQyvEW61WLBYLYWFhWCyWCkV7YGAgxcXFZGRkVHi/a5XbL36545cbz4tdnPlw\nZyGfphZSZCy7v915+hwD39pETIiJoa3ccDLWnV8B1FelrlGflbpE/VXqgmbNml2za2m1H7nhGAwG\nBrf35NWunjQpKYKfRwEVlcKKlEL+su4MOzKK7JukiIiIiB1U+eTf398fBwcH0tPTy7Wnp6cTFBRU\n6TlBQUGVxhsMBts5l4pxdHTE39+/WslfSW6/0Bi/+mHA7Vb+vWY/szYfIefnzcFOF8GMnXn0bOLK\n5NgWNPJ0+e2L2InGo0pdoz4rdYn6q9QlF4/5v1pVPvl3cnKiU6dOJCYmlmtPTEyke/fulZ7TtWtX\n1q9fT2Fhoa1t9erVhISEEBYWZov59TVXr15NVFQUDg4O1Ur+SnKT+sVgMPDgHc35/M89uc3NBWNp\n2c8AJSWlfHPgFPcs/o7XNx0mp6DYzpmKiIiIXH/VGvbz9NNPs3jxYhYtWkRycjJPPfUUaWlpxMXF\nATBp0iTuuOMOW/zw4cNxdXVlxIgR7Nmzh48++ohp06bxzDPP2GLi4uI4ceIE48aNIzk5mYULF7Jk\nyRLGjx9viykqKmLHjh0kJSWRn5+PxWJhx44dHDx4sMrcRo0addVfjtw4/HzMvPVkD2b3aU5Q3oUJ\n5Tm5hcxff4j7lm7js59OUlKqHYJFRETkxlXlsB+AoUOHkpmZSXx8PGlpabRt25ZVq1bRqFEjoGyC\n7+HDFzZV8vT0JDExkdGjR9O5c2d8fHwYP348Y8eOtcWEh4ezcuVKxo0bx/z58wkJCWHOnDkMGjTI\nFpOamkrHjh1tSzQmJCSQkJBAr169+Oqrr34zt19POBYBiO4WRtd2wUx/dzufpp7hvKsTpaVWjp08\nx7Of7eG/jX0Y1yOCDkGe9k5VRERE5Jqr1jr/Nwqt8y8X+2FnGi8v20Gys5Eipws/gnl5unBHqwaM\nvDmUcG/zb1zh+tJ4VKlr1GelLlF/lbqkRtf5F7lRdWwXzL+ej2ZCm2CCTufb5gNkn83n420pjPhw\nBzO2HOZkbmEVVxIRERGpG1T8S71mMjnw+/tu4t9/6cXvHEx4nykAyiYEW06e493NxxixfAeLklI0\nKVhERETqPBX/IkBIkAevTLiNV+9uS/uT+bifK9sHIC+/iEPHs3lr42FGfLKL//5ooaC41M7ZioiI\niFyZak34FakPDAYDt3UL4+b2wfznw918uPko6QFm8syOZJ8tIOdcIa+fyePTfScZ1jaY6CZ+ODno\n/llERETqDhX/Ir/i7mZi5B9vJqZPBAuXJrE55QwnG5gpNDlw6vR5snMKSMnK4797LTzQJpi+Tfxw\nNBrsnbaIiIhIlVT8i1xCWKg3L03sxZZtJ/jnf3awvzifUwFmCoFUSw7ZZ/M5kZ3H+z+W3QT0DvPF\nQTcBIiIiUoup+Bf5DQaDga6dG9GpQzD/W7WPZZ8mc9LNkQx/F3LPF5F7vogz2QWkZOexbK+F4W2C\nuS3MB6NBNwEiIiJS+6j4F6kGk5MDg+9qTZ8e4Sx9fxdfbTlKlo8Lp/3MnM0pmw+QfbaAadl5vLfX\nzNDWQfRqrF8CREREpHZR8S9yGfx8XXkq7lYGxjbj3WU7SUo+RaavC6f9XMg6k2+7ETiWnc+/dqUx\nuFUgtzfxw6SJwSIiIlILqPgXuQKRTXx5YUIvfthp4d33d3Fo/xlO+7mQ6etCxunzZJ/NJ9fbzBvn\nCvjPnjTubRlIv6b+uDg62Dt1ERERqcdU/ItcIYPBwM3tg+lwUxBrNx7l3x/u5uT+M2T6lt0EpJ/K\nJetMPud8XFjwfQrv7bEwqEUD7mwWgLtJ/+mJiIhIzVMFInKVjEYDfXqG0/3WUFYk7uejT5Pxzczi\njHfZcKD0ohKyTPn4ept5d1cqH/yYTr+m/tzdogEBriZ7py8iIiL1iIp/kWvEZHLgnoEtie4dwaef\n/8SnX/yET9YZzng7c9rPjKWwBGeTA74+ZpbvS+eTn07SM9SHe1o2oJmvm73TFxERkXpAxb/INebu\nZuKB+9pyZ2wzPl65jxWrD+CTdYZsLxOn/cykFZbg7OyAj5eZtUczWXssk7YBHtzbsgGdQ7y0TKiI\niIhcNyr+Ra4TD3dn/jC0Hb+Lbc7yFcl8/uVBvLKzOefuxGk/FywFJZicHPDxdmG3NYfdp3Jo6OHC\noBYN6BvuZ+/0RURE5Aak4l/kOvP2cuHh4R24u38LPvz0R7785jAeR3PIc3EomxhcVEJmVh7e3i5Y\nrfDmtmMs2ZlKS8ciuvppdSARERG5dlT8i9QQXx8zI/94M/fd1YpPV/3E518dxJyaS4OTeWT6OnO6\nsISsrHy8vVywelhZm1fM2lPFbCk5wJ3NAugY5KkhQSIiInJVVPyL1DBfbzMPPdCee3/XkhWrD7Ay\ncT9OJ/Pwz8jjjLcLmfnFZJ5xwOxswM3Nke9Ss/kuNZsQd2cGNgvgjiZ+WipURERErogqCBE78XB3\nZti9bbh7QHO++OoQ/1u1D4fMfHwz8znn7sRJdwcsXi7k5ZcNHUo9V8CCH1JYsjOVvuG+9GsaQKSv\nq70/hoiIiNQhKv5F7Mzs4sSgAS0YEB3JV+sO88nKfaSfysWYnk2RswP5DX044V2As6sJby9ncHVi\n1cEMVh3MINLHldim/vRq7IubSfMDRERE5Lep+BepJUxODvS7PZKYPk35dvsJ/rl0E0dTzuOdfp4G\nJ8+T7eVMpq8Lp91NeHk64+5u4kDWeQ5sO8bCH1Lo2diHfk39aennhkFzA0RERKQSKv5Fahmj0UDX\nzo1wMkRw/MR5Dh53YvPWFHzOFOB9poA8syNZPs4c83bG3cMZL08XAL48fJovD5+msaeZ2KZ+9An3\nw8tZ/4mLiIjIBaoMRGqx0Iau3HN3FOmnclmxej9frj2EIa8Y17xiSiznyfYyke7jgtHbBS8PZ9xc\nnTh2No8FP6SweMcJOod4cXu4H52CPXFyMNr744iIiIidqfgXqQMCA9x45MEODB3Umq/XH+HzNQdJ\nS8/BN6sAn6wC8l3OkeXjQpafC26eLnh6OAOwKeUMm1LO4GFypFeYD7eH+9HM11XDgkREROopFf8i\ndYi7m4nf9WvOwJhm7NyTzsovD7A9KQ1zfgnmtFxK0nM56+nMSR8XjH5mPNxNuJqdyCks5rP9p/hs\n/ykaebhwexM/+oT7EuBqsvdHEhERkRqk4l+kDjIaDXS4KYgONwVxMiOXL746yJffHCbnXAE+Z8r+\nFJzIIdvLmbQAV5y9XPD0MOHoaCQlJ593dp5gyc5U2gW607uxL10beeOh+QEiIiI3PP3fXqSOa+Dv\nxh+GtuP+e9qw6bvjfLHmIPsOnMa5sJQGp/KwnsrjvKsjZ7xdKA1yw9XDhJurCQxWdqTnsCM9hze3\nHVJwKSsAACAASURBVKNjsCe3NfahS0NvXJ20bKiIyP+3d/exUd13vsff53EePPbY2PgJEwMBSiBJ\ns8GkPASVTRbYbHbbpnvZklzdKpWSVjfNTaG5qEnVbVq5bJ+VTaskZFOpTaVES3WtZaXerYpvNiWA\noQ1eSGIwlGcMZoyNx2OP7Xn+3T/OzHjGNtg82/j7kkbnnN98z5kz5ujw+8x5EuJ2JJ1/IW4TtmWw\nasUsVq2YxZmzId59/yR/2HWavnCUgoEEBQNhUufD9BbZBMu8qDIPPp8Ll8sgoRQftIf4oD2ErevU\nVRex8o4SHqguxm3KhcJCCCHE7WLc/6u/9tprzJkzB4/HQ11dHbt27bpsfUtLC6tWrcLr9TJz5kzq\n6+tH1OzYsYO6ujo8Hg9z587ljTfeGFHT0NDAokWLcLvd3H333Wzbti3v/e9+97voup73qq6uHu/X\nEuK2dEeNny89cR9vvvK3PP/VZdx3TyWgoSsoDsWoOt7D9OYA8f0Bgid7CPZESCRSAMRSKZrO9vDD\nppP8920f8sOmEzS1BYmk3xdCCCHE5DWuX/63bt3Khg0b2LJlCytWrODVV1/lkUceobW1lZqamhH1\nfX19rF69mlWrVtHc3ExraytPPvkkPp+PjRs3AnDq1CkeffRRnnrqKd5++2127tzJM888Q3l5OY89\n9hgAe/bsYf369dTX1/PYY4/R0NDAunXraGpqYsmSJdnPW7BgATt27EApBYBhyCkLQoBzNGDFp2ay\n4lMzudDVz3++f4r/3HmSrosDWAlFaXcEuiPELJ3eIhtqClGFLnwFNrqhEUmkeP9MkPfPBLENnbqq\nIpbVFLOkyi/XCAghhBCTkKYyPebLWLp0Kffddx9btmzJts2fP59169axefPmEfWvv/46L774Ihcu\nXMC2nbuJbN68mS1bttDW1gbAN77xDbZt28aRI0ey8z399NMcOnSI3bt3A7B+/XqCwSC///3vszWr\nV6+mvLyct99+G3B++W9oaOCjjz4a88uGQqHsuN/vH7NeiFtp3759ANTV1V3X5aZSio9bL/B+02n2\nfHCWSCSR937U1gmXuNFnFqEKLLweC03PvzWooWncW17I8pnFLJ1RzDSPdV3XUUxON2qbFeJGkO1V\nTCbXsw875mk/8Xic5uZmVq9ende+Zs0ampqaRp1n7969rFy5MtvxB1i7di3t7e2cPn06W7NmzZq8\n+dauXcu+fftIJpOA88v/aDXDP/fEiRPMmDGDOXPm8Pjjj3Py5MmxvpYQU5aua3xyUQX/6+kH+OXP\nP8PG/7mU+++tyt773xVLUdoxQPG+AN697fR/2MHghX76++NkfipIKsX+jl5e3XeGL/77x/zv/3eE\n/9Ma4GxvhHH8niCEEEKIW2TM4/ZdXV0kk0kqKiry2isqKnj33XdHnScQCDBz5swR9UopAoEAtbW1\nBAKBEYGioqKCRCJBV1cXFRUVBAKBUT83EAhkp5cuXcqvfvUrFixYwIULF6ivr2f58uUcOnSIkpKS\nsb6eEFOay2WyctkdrFx2Bz2hCLv+2MaO3ac5frIbDXBHk7jP98P5fqK2zuA0D+bMImJeC6/XQtNA\noWjtCtPaFeaXH56j2ufigWo/S6r9LJrukycLCyGEEBPIpD9pd+3atXnTS5cuZfbs2bz11lts2LDh\nkvNlDvcJMdHdzG21chp84e+K6bzo4aNDIT4+HKKzK+q82Q9WMAzHOzEsnb4yL+bMQiI+F7ZLR08f\nOTjWP8CxjiDv7AeXAfN9BguKdO4qNCi05MnCU4HsX8VkIturmAzmzZt33ZY1Zue/rKwMwzDo6OjI\na+/o6KCysnLUeSorK0et1zQtO8+lakzTpKys7LI1l/pcAK/Xy6JFizh69OhYX00IcQnTS108vLKc\nhx6czoWuKC2HQ3x0KERXdwwAM57CPB+G82EMQyNa6sGoKSTid2O6DIz0NQLRJHwcSvJxKAnEqfHq\n3FWo84lCg5leLRsYhBBCCHFzjNn5tyyLxYsX09jYyN///d9n2xsbG1m3bt2o8yxbtowXXniBWCyW\nPe9/+/btVFdXU1tbm60ZftvO7du3U1dXl71bz7Jly2hsbOT555/P+9zly5dfcn0jkQiHDx/moYce\nuuz3kgt8xEQ3kS5Ge/QRUEpx5myI3X88y+4/tnG+o2+ooB840kdK62PAa+GZ5Sde6iXhMrCsodN+\nuoHdYedVaJvcW1HI/ZVF/EVlIRUFrpv+vcT1NZG2WSHGIturmExyL/i9VsZ3vvOd74xVVFRUxEsv\nvURVVRVer5f6+np27tzJL3/5S4qKinjxxRf5wQ9+wBe/+EXAuRPQG2+8wYEDB1iwYAG7du1i06ZN\nfPOb32Tp0qUAzJ07lx/+8Id0dnYya9Ystm3bxve//31efvllFixYAMCMGTN46aWXsG2b6dOn8+ab\nb/KrX/2KN998M3sv/02bNuF2u1FKceTIEZ599lmOHz/OG2+8MeJq6Gg0mh13u93X5Q8oxI3S3t4O\nMGGeW6FpGsV+N/csLOdvVs/lU4trKPa76R+I0xOKODWAHU+hdw5gnA7B2V6smMKyDQaUwjB0Mj/2\nx5Ip2noj/Kk9xL//+QI7Tgc52xchpRTTPJZcKzAJTbRtVojLke1VTCbXsw87rnP+/+Ef/oHu7m42\nb97M+fPnufvuu/nd736Xvcd/IBDIu8NOUVERjY2NfPWrX2XJkiWUlJSwadOmvHPwZ82axX/8x3+w\nceNGtmzZQnV1NT//+c/53Oc+l61ZtmwZ//qv/8q3vvUtXnrpJe68805+85vf5KX0s2fP8sQTT9DV\n1cX06dNZunQpe/fuHXHBsRDi+tE0jdm1xcyuLWb95xfR0dnPB/91jj82n+PQkS6UUmg4dw7iRBBO\nBCnQNWLT3HhqixkssklaOmbO04PP9UU41xfht0c7MTSNBWUFfLKikHvLC/lEaQG2hAEhhBDimo3r\nPv+3C7nPv5hMJush6b5wlH0HzvOn5nPs/7iDWCwxokYBMdvAXevHrPIRsnV0y+BSlwDYus5d0wu4\nt7yQe8oLmT/NK0cGJqDJus2KqUm2VzGZXM8+7KS/248QYmIp9Ln4ywdn8ZcPziIWS/LRoQ72fxSg\n+cB5LnT1A6SPCiRRR7uJH+3GrUGq2I1vdjHRYje9Otj20JO6Y6kUH3b08WH6OgPb0FlY5uPech/3\nVhQyb1oBpi4XDwshhBBjkc6/EOKGsW2Duvuqqbuvmqf+h+Lc+b5sEDh4pJNkMgWArkAPRogEnWd4\neAwNraIA70w/gz6LQUPLO0UolkxxoKOXAx298DG4DJ27ynwsnF7AwjIfC0oL8FjGqOskhBBCTGXS\n+RdC3BSaplFTXURNdRF/99fzGYzEaTnUyb4P29n/UYCuiwPZWjOpoD1MtD2MDtimhmemH9eMQvq9\nFoMa6MbQL/3R3DAA6JrG7GIPd5UVsKjMx8LpPsq89vBVEkIIIaYc6fwLIW4Jj9tiyf3VLLm/GqUU\n7YE+Pmzp4ONDF/j40AUGBuPZWiuhSJzsIXGyBwDT1PHW+rGrfPS5TSKGhpETBlJKcTw4wPHgAL89\n2gnAdK/NwjKfEwim+6j1e7LPIxBCCCGmCun8CyFuOU3TmFFVxIyqIv5m9TySyRQnTgX58KATBlr/\n3EUikcrWW4kU8eNB4seD6IBp6XiqC/FUFxLxWfRqYJj5FwR3DsTYcaabHWe6Aee6gXklXuaXFjB/\nmjOsKLDR5MFjQgghbmPS+RdCTDiGoTPvzlLm3VnKf/vMQqLRBIePdvFhSwcthzs5fjJI7o3K7HiK\n5OkQ4dPO3RBcuoZdWYCvxk+8yCaog8p5xgA41w0c7ApzsCucbfO7zHQYKOATpV7mTyug0CW7SSGE\nELcP+V9NCDHhuVwmn7y7kk/eXQnAYCTOn49d5ODhLg4d6eTo8W7iiWS23kgpku1hQu1Ox94Gkj6b\nwlo/WqmXsEsnouefKgQQiib4oD3EB+1Dt1Sr9rmYO83LnSVe5pZ4mVPipUgCgRBCiElK/gcTQkw6\nHreVFwZi8STHTnRz8HAnBw93cuTYRaLRoecLaIAZjjF40Dn/XwdsQ8Nd6cM7o4hEkU1I10ga2ohn\nDbSHo7SHo7x/Jphtm+61s0HgzhIPd5Z4KfVYcsqQEEKICU86/0KISc+2DBZ+YjoLPzGddZ+FRCLF\nydNBjhy7mH3l3k0InDsKJc710XvOeXaACWhuE19NEVa5l4jXIqQpdHPkw8c6B2J0DsTYc64n21bs\nsrizxMOcEi9zSjzMLvZS7XPJRcVCCCEmFOn8CyFuO6Y5dM3A36512rqDgxw5dpE/p8PA8ZPBvFOF\nNMCMJIgc6yZyzLko2K1BosDCN6MIq9RDzOtcTKyZ+ohA0BON0xyI0xzozbbZus5Mv5tav4dZfg+z\nit3M8nuYJkcJhBBC3CLS+RdCTAnTSjwsW1LDsiU1AMQTSU6fCXE4HQiOnwxyPv0E4QxdgR2OEzty\nkVi6zQ0kvRYF1YWYZR7iXouQDhh63rMHwHkyceaWo7kKbdMJBOkwMKvYCQfyYDIhhBA3mnT+hRBT\nkmUazJ0zjblzpsGaeQCE+2OcOBXk+Kkgx08GOX6ym47O/rz5NMAciBM91k30mNPmBmK2jlXmxV3h\nQxXaDFg6Ud35HIb9yN8XS9DS2UdLZ37YKPPY1BS5mZl+1RS5uaPITbHblCMFQgghrgvp/AshRJqv\nwObeRRXcu6gi2xYOxzh+OsjxE90cOxnkxKkgF7pGBgJXLAXtYWLpOwxZgK5rJH0WBVWFmCVuYh6L\nPh2UoWEa+ohQ0DUYo2swln1ScXa9LCMbCnLDQaXPhS6hQAghxBWQzr8QQlyGz2fzyUUVfHJYIDjV\n1sOpMz2cOhPiVFsPbWd7864hAOeWo0ZvjHjvRTLPK/YAcUsn5bPxVBSg+11E3Sb9uoZh6ljmyFAQ\njic5fLGfwxfzQ4et61T6XFQXuqjyuZiRHlYXuinzWhIMhBBCjCCdfyGEuEI+n83dd5Vz913l2bZk\nMsW5832cbgtx8kwPp9ucYBDsGcybV8N5KBnBCCoYIYmzIy7COXUo5bXwlhdgFLuJu036DUjpOpal\no+sjryk40zvImd78z4DRg4EzLsFACCGmMun8CyHEdWAYOnfU+Lmjxs/KZXdk20O9EU63hTjb3kvb\nOed1tr2X3r5o3vzZU4diUVRPlES6rQBImBpRl4Fe7MGa5kYV2EQsnaiuYVk6ljXydqTjCQYVPpuK\nApuKAhcVBTaVPhflXlueaiyEELcx2cMLIcQN5C9yc+8id951BOCEgrPtfbSdC2VDQdu5XkK9kbw6\nDbASCiuRgP4+SD+XoABw6xoxWydmm7hK3Zh+NymPScTUiesalqljXuERA4ACy6CiwEWlz6bca1Ph\nc2VDQqXPxm3KXYmEEGKyks6/EELcAv4iN/4iN4sWTM9r7wtHaTvXy7nzfZwP9DnDjjDnO8Ikk6m8\nWiOl8ESSeCJJ6I0CIQyci42T2WBgoBe6MP0uVIFF1NJJGBqWaWBZ+qgXHvfHk5zoGeBET/4tSrPr\n7jKZ7rUp89qUeS2me52Q0NGfpNjSSKaUPNxMCCEmKOn8CyHEBFLoc2WfVpwrlVJc6Oqn/Xwf7R3h\nbDAIdIS50DUAqLz6/GAQyx4x8DIUDOKWTsQ2cRW70Hw2SZdB1NTBSB81MJ1woA3ryIeiCULRBMeG\nPb9goN95GsLP2vYzzW1R5rUo89pMT78yQWG618bvNuW6AyGEuAWk8y+EEJOArmtUlvuoLPdx/7D3\nYvEkgY4w7YE+Oi70c6Grn8CFcHY8kbjMEQPicHHo9B8FJA2NmG044cDS0QpsdJ9N0mMQN3X0nGCQ\nGeYePUgplb1tKcPuUJT9PppGidtkmsdmWmbosSj1WHnDIpeEBCGEuJ6k8y+EEJOcbRnZi42HS6UU\nwZ5BOjr76bjQT0dnOD10poOhkXcjMpMKczABmbcuDl2HoHAuQI5b6XBg6sRtHcNnE9VTpGyDgYhz\nAbRp6BiGlg0IuU9ATinFxcE4FwfjXI6haZS4LUqGBYMSt0Wx26TYbeF3mZS4LVymfrV/QiGEmDKk\n8y+EELcxXdconealdJp3xKlEANFogo7OfrouDtDVPUjXxQE6L6an02251xrkXYCclxsG6O93TgPy\n+LwkTOe0orilEzV1wpZO0mWie02SLhNl6k5AMLV0UHCGhqE71wukc0Iy5yjC0TG+q8c0soGg2GVS\n7Dbxp8dLPEMhwe8y8dmGPDVZCDElSedfCCGmMJfLvORRA3COHPT0RujqGqAzEwgupse7B+jsGqAv\nnH/bUl05zzKw46lRlwmQ0iBh6umXRsLUiaUDQ9LUwWOibIOUaWQDgmFoGHp6aAwNM334wUSSwXCS\n88PWZzSWruF3OUcPilwmhbYzdMYN/K6caZdJkW3KkQUhxG1BOv9CCCEuSdc1phV7mFbsYf7c0lFr\nYvEkwZ4Iu3c30xuOUzZ9Jt3BQYI9ES4GB+kODtLdM0gkkhha7jgCAgwPCU44iKXDQtJwggMuJyho\nhoahO4FA17W8sKCn2w3dGY+ncq5LGCeXoecEBJMil5E3nTmi4LNzhpYhdz4SQkwo0vkXQghxTWzL\noGJ6AXfUeAGoq5s/at1gJO4EgWCEi8EBgj0RursHCYYi9IQihHqjhHojhPuHOuTjDQkKSOla9ihC\nJhgkTJ24oWXDQzI9rlk6hu5chzBaYMiM6/rQK0qKzoEYnQPjDwzgnI7khAGDAss5suCM5waFofEC\ny6DQNimwDWxDjjYIIa4v6fwLIYS4KTxuixlVFjOqii5bF08k6e2N0tMbJZQOBj3pYBAKRdPTTljo\n7YuilELDuYuREVPOk5LHkDmikDS0bFBIGs50fJS2pKGhNCco6JkjCemQ4IzntmvoOQGiP6kYjCfp\nHP2xCZdlGzo+y8BrGXgsJxh4LR1vus1rGXhNp63AMvDamWkDT6bNMuSOSUKILOn8CyGEmFAs08he\npDyWVErR2+cEg75wjL5wjN6+KH3hKL19Mfr6ovQOG49EEtkjCsQBkmN+jnNkgZwwMHQUIRMO4jlB\nIaUP1WRomoauM3Q0Qcs/sjBae0TXCGvpYKFpaLrG1fTj3aaeDQVDLx2P6YQEt2ngMXXcZrrN1HGZ\nOh7LwG04w9z35VQmISYv6fwLIYSYtHRdo9jvptjvHvc88UTSCQp96VAQjmZDQ29flHB/jP6BuDPs\njxHuj9M/ECcWS2Ckxh8YYOh0pPxQMBQMcqfjOXWZttFoGtmQoOWEBWc8EzIy7zvPVAjrWk47Tm1O\n25UGCltPhwMzHRzSQcKdbXPa3elxl+HUu4xh4+mhnTeuyZEKIW4g6fwLIYSYUizTyF7EfCVi8SQD\n6VAQ7o/ljQ+FhTj9A/ltg4MJBgbjqDGuWxguc7QhpWuk9KGwkAkMo42ndI2EMbJ9PIbCgBMYNG1o\neuQ4eUcihuqHlqPlBJHM+Hj79LZx6aDgMrSRbTkhwk4HCEt3hrahY+la9j0r3dYXV1g6xJMpzPQ6\nCzEVjLvz/9prr/GTn/yE8+fPs2jRIv75n/+ZBx988JL1LS0tPPvss/zpT3+itLSUL3/5y/zjP/5j\nXs2OHTt4/vnnOXjwIDNmzGDTpk185StfyatpaGjg29/+NsePH2fu3Ll873vf43Of+9w1rZsQQghx\npWzLwPYbV3SUIUMpRSSayAaBwcE4A5EEAwNxBgbiDEbiDAwm0u1Om1OXyA4HI84RCKXUlX02lw4R\nzoth0/nvJUdpvxa5YUDX8oPBlQ7z5x/fPBkD/c7D67wn96OhYRkarnRQsIyh4GDntLlyQoVl6Njp\nUGFlazQMzXnP1DXs9DUgpu7MZxnOeO60oWlYuoaZXl6mXo5+iBtlXJ3/rVu3smHDBrZs2cKKFSt4\n9dVXeeSRR2htbaWmpmZEfV9fH6tXr2bVqlU0NzfT2trKk08+ic/nY+PGjQCcOnWKRx99lKeeeoq3\n336bnTt38swzz1BeXs5jjz0GwJ49e1i/fj319fU89thjNDQ0sG7dOpqamliyZMlVrZsQQghxs2ma\nhsdt4XFbTCu5siMOuZRSRKNJBiJxIpEE0WiSwfR4JJoYGkaT2fFoNMFgzvuDkQSxWLot3Z5IjP+o\nhAKUNlpgGGpT2sj2TFvuvJnxZPZ9UDe405sJAqlUEg0Nw4w7bQwPCZnpS7flLi93GWTayA0fufMP\nBZER404Zhj501MJMB4XM+FCAcIKGoWfec4a5AcLUnYCRGWbey7TpGs50ut3MrcmdRxuaztanjwbl\nzpP7uRJgJiZNjeMnhKVLl3LfffexZcuWbNv8+fNZt24dmzdvHlH/+uuv8+KLL3LhwgVs2wZg8+bN\nbNmyhba2NgC+8Y1vsG3bNo4cOZKd7+mnn+bQoUPs3r0bgPXr1xMMBvn973+frVm9ejXl5eW8/fbb\nV7xuoVAoO+73j/5AGyEmin379gFQV1d3i9dEiPGRbXbySiRS+UEhkiAWTxKLJYnGMsNEdjoaTRKL\nJXLeGxpm6mLD2qLR5LiOWuSGi0wgyA8UjAgSw+uG2klP57eldI1EKoXSNHTTQE2CPmpueMhMa2jp\noJEz7pQNBRFyw0d+aBnPe+nFZZfJJdqyf8K8IIUTHrT8sKFngkMmUOgaRrrdyGvLCTI5R1aMdAgy\n0teHGOmwpGvpMJM+EpRdrjZUl/kMPb1umXUYmk4Pc+r0nPbsESeGPlNjqEbPqdE10NN/C0NPzzPq\nMpy/We5nDT8N7Xr2Ycf85T8ej9Pc3MymTZvy2tesWUNTU9Oo8+zdu5eVK1dmO/4Aa9eu5dvf/jan\nT5+mtraWvXv3smbNmrz51q5dy69//WuSySSGYbBnzx6ee+65ETWvvvrqVa+bEEIIIfKZpo7PtPEV\n2GMXXyWlFIlkKi8sZF/xJPFEing8STyeM0wMTcfiKWLxJImEs4xEwpkeXhvLWUZuXSKRyjvC0d/v\n3Hu1oMA7LHAMBYahoEE2QOQGj9GDxuWG468Z+fcDUOTnpys7Bex2NjywOFPkpZLc6fzxoca8v7yW\n896I94d9Vu6MmjbKcnLfzv/31YYVazn1mWDwf5+4m+tlzM5/V1cXyWSSioqKvPaKigrefffdUecJ\nBALMnDlzRL1SikAgQG1tLYFAgNWrV4+oSSQSdHV1UVFRQSAQGPVzA4HAVa+bEEIIIW4+TdOwTAPL\nNCgY+y6uN0QqpbJhYN++/yKZVCy6+x6SSScYxNMBIZlUJBJOIHHGnXCRTDgBJluXniczf/4yUiQu\n0R5PpJxlJVIkU85ykynns5z5nM9JKEUypfLCBwwPIyMDBIx8D0YPGJerGb6c4fX5bSOXczONHo7y\nKm7eykxwU/ZuP7mHT4SYiObNmwfItiomD9lmxWTyF/fdlR7L3LZVA4z0S4jb15jPDS8rK8MwDDo6\nOvLaOzo6qKysHHWeysrKUes1TcvOc6ka0zQpKyu7bE1mGVezbkIIIYQQQkxVY3b+Lcti8eLFNDY2\n5rU3NjayYsWKUedZtmwZO3fuJBaLZdu2b99OdXU1tbW12Zrhy9y+fTt1dXUYhnHJmsbGRpYvX37V\n6yaEEEIIIcSUpcZh69atyuVyqV/84heqtbVVPffcc6qwsFC1tbUppZR64YUX1MMPP5ytD4VCqqqq\nSj3++OOqpaVFNTQ0qKKiIvXyyy9na06ePKl8Pp/asGGDam1tVW+++aZyuVzq3/7t37I1TU1NyrIs\n9YMf/EAdPnxY/dM//ZOybVt98MEHY67bmTNnxvPVhBBCCCGEmDLG1flXSqnXX39dzZ49W7ndblVX\nV6d27dqVfe/JJ59Uc+bMyatvaWlRn/70p5XH41HV1dWqvr5+xDLff/99tXjxYuV2u9WcOXPUv/zL\nv4yoaWhoUHfddZdyuVxq4cKFatu2bVe0bkIIIYQQQgjHuO7zL4QQQgghhJj8xjznf7J57bXXmDNn\nDh6Ph7q6Onbt2nXZ+paWFlatWoXX62XmzJnU19ffpDUV4sq219OnT6Pret7LMAy2b99+E9dYTFU7\nd+7ks5/9LDU1Nei6zq9//esx55H9q7hVrnR7lf2ruJW+//3v88ADD+D3+ykvL+czn/kMBw8eHHO+\nq93H3lad/61bt7Jhwwa+9a1vceDAAZYvX84jjzzC2bNnR63v6+tj9erVVFVV0dzczCuvvMKPf/xj\nXn755Zu85mIqutLtFZz7ZG/fvp1AIEAgEOD8+fM89NBDN3GtxVQVDoe55557+NnPfobXO/ZN2mX/\nKm6lK91eQfav4tZ5//33efbZZ9mzZw/vvfcepmnyV3/1V/T09Fxynmvax97q846up0996lPqK1/5\nSl7bvHnz1De/+c1R61977TXl9/tVNBrNtn3ve99TNTU1N3Q9hVDqyrfXU6dOKU3TVHNz881YPSEu\nyefzqbfeeuuyNbJ/FRPFeLZX2b+KiSQcDivDMNRvf/vbS9Zcyz72tvnlPx6P09zcPOKpwWvWrKGp\nqWnUefbu3cvKlSux7aHHma9du5b29nZOnz59Q9dXTG1Xs71mfP7zn6eiooIHH3yQhoaGG7maQlw1\n2b+KyUj2r2Ii6O3tJZVKUVJScsmaa9nH3jad/66uLpLJJBUVFXntFRUVBAKBUecJBAKj1iulLjmP\nENfD1WyvPp+Pn/70p/zmN7/hd7/7HQ8//DBf+MIXeOedd27GKgtxRWT/KiYT2b+KieRrX/sa999/\nP8uWLbtkzbXsY83rspZCiBuutLSUjRs3Zqfvv/9+Ll68yI9+9COeeOKJW7hmQggxucn+VUwUX//6\n12lqamL37t1omnZDPuO2+eW/rKwMwzDo6OjIa+/o6KCysnLUeSorK0et1zTtkvMIcT1czfY6mgce\neICjR49e79UT4prJ/lVMdrJ/FTfbxo0b2bp1K++99x61tbWXrb2Wfext0/m3LIvFixfT2NiYC2Qg\nCQAAAf9JREFU197Y2MiKFStGnWfZsmXs3LmTWCyWbdu+fTvV1dVj/tGFuBZXs72OZv/+/VRVVV3v\n1RPimsn+VUx2sn8VN9PXvva1bMd/3rx5Y9Zf0z722q9Jnji2bt2qXC6X+sUvfqFaW1vVc889pwoL\nC1VbW5tSSqkXXnhBPfzww9n6UCikqqqq1OOPP65aWlpUQ0ODKioqUi+//PKt+gpiCrnS7fWtt95S\n77zzjmptbVVHjhxRP/7xj5XL5VKvvPLKrfoKYgoJh8PqwIEDav/+/crr9ar6+np14MABdebMGaWU\n7F/FxHKl26vsX8Wt9Mwzz6iioiL13nvvqUAgkH2Fw+FszfXcx95WnX+llHr99dfV7NmzldvtVnV1\ndWrXrl3Z95588kk1Z86cvPqWlhb16U9/Wnk8HlVdXa3q6+tv9iqLKexKtte33npLLVy4UPl8PuX3\n+9WSJUvUO++8cytWW0xBf/jDH5SmaUrX9bzXl770JaWU7F/FxHKl26vsX8WtNNq2quu6+u53v5ut\nuZ77WE0ppa71UIUQQgghhBBi4rttzvkXQgghhBBCXJ50/oUQQgghhJgipPMvhBBCCCHEFCGdfyGE\nEEIIIaYI6fwLIYQQQggxRUjnXwghhBBCiClCOv9CCCGEEEJMEdL5F0IIIYQQYoqQzr8QQgghhBBT\nxP8HnbvVK1gEmJIAAAAASUVORK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "thinkplot.PrePlot(num=2)\n", + "thinkplot.Pmf(time_dist1)\n", + "thinkplot.Pmf(time_dist2)\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure 7.3: Distribution of time between goals.\n", + "\n", + "For the Bruins, the overall chance of winning the next game is 55%.\n", + "\n", + "To win the series, the Bruins can either win the next two games or split\n", + "the next two and win the third. Again, we can compute the total\n", + "probability:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.57142746956373758" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# win the next two\n", + "p_series = p_win**2\n", + "\n", + "# split the next two, win the third\n", + "p_series += 2 * p_win * (1-p_win) * p_win\n", + "p_series" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The Bruins chance of winning the series is 57%. And in 2011, they did." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "As always, the analysis in this chapter is based on modeling decisions,\n", + "and modeling is almost always an iterative process. In general, you want\n", + "to start with something simple that yields an approximate answer,\n", + "identify likely sources of error, and look for opportunities for\n", + "improvement.\n", + "\n", + "In this example, I would consider these options:\n", + "\n", + "- I chose a prior based on the average goals per game for each team.\n", + " But this statistic is averaged across all opponents. Against a\n", + " particular opponent, we might expect more variability. For example,\n", + " if the team with the best offense plays the team with the worst\n", + " defense, the expected goals per game might be several standard\n", + " deviations above the mean.\n", + "\n", + "- For data I used only the first four games of the championship\n", + " series. If the same teams played each other during the regular\n", + " season, I could use the results from those games as well. One\n", + " complication is that the composition of teams changes during the\n", + " season due to trades and injuries. So it might be best to give more\n", + " weight to recent games.\n", + "\n", + "- To take advantage of all available information, we could use results\n", + " from all regular season games to estimate each team’s goal scoring\n", + " rate, possibly adjusted by estimating an additional factor for each\n", + " pairwise match-up. This approach would be more complicated, but it\n", + " is still feasible.\n", + "\n", + "For the first option, we could use the results from the regular season\n", + "to estimate the variability across all pairwise match-ups. Thanks to\n", + "Dirk Hoag at , I was able to get the\n", + "number of goals scored during regulation play (not overtime) for each\n", + "game in the regular season.\n", + "\n", + "Teams in different conferences only play each other one or two times in\n", + "the regular season, so I focused on pairs that played each other 4–6\n", + "times. For each pair, I computed the average goals per game, which is an\n", + "estimate of $\\lambda$, then plotted the distribution of these estimates.\n", + "\n", + "The mean of these estimates is 2.8, again, but the standard deviation is\n", + "0.85, substantially higher than what we got computing one estimate for\n", + "each team.\n", + "\n", + "If we run the analysis again with the higher-variance prior, the\n", + "probability that the Bruins win the series is 80%, substantially higher\n", + "than the result with the low-variance prior, 57%.\n", + "\n", + "So it turns out that the results are sensitive to the prior, which makes\n", + "sense considering how little data we have to work with. Based on the\n", + "difference between the low-variance model and the high-variable model,\n", + "it seems worthwhile to put some effort into getting the prior right.\n", + "\n", + "The code and data for this chapter are available from\n", + " and\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "If buses arrive at a bus stop every 20 minutes, and you arrive at the\n", + "bus stop at a random time, your wait time until the bus arrives is\n", + "uniformly distributed from 0 to 20 minutes.\n", + "\n", + "But in reality, there is variability in the time between buses. Suppose\n", + "you are waiting for a bus, and you know the historical distribution of\n", + "time between buses. Compute your distribution of wait times.\n", + "\n", + "Hint: Suppose that the time between buses is either 5 or 10 minutes with\n", + "equal probability. What is the probability that you arrive during one of\n", + "the 10 minute intervals?\n", + "\n", + "I solve a version of this problem in the next chapter.\n", + "\n", + "Suppose that passengers arriving at the bus stop are well-modeled by a\n", + "Poisson process with parameter $\\lambda$. If you arrive at the stop and\n", + "find 3 people waiting, what is your posterior distribution for the time\n", + "since the last bus arrived.\n", + "\n", + "I solve a version of this problem in the next chapter.\n", + "\n", + "Suppose that you are an ecologist sampling the insect population in a\n", + "new environment. You deploy 100 traps in a test area and come back the\n", + "next day to check on them. You find that 37 traps have been triggered,\n", + "trapping an insect inside. Once a trap triggers, it cannot trap another\n", + "insect until it has been reset.\n", + "\n", + "If you reset the traps and come back in two days, how many traps do you\n", + "expect to find triggered? Compute a posterior predictive distribution\n", + "for the number of traps.\n", + "\n", + "Suppose you are the manager of an apartment building with 100 light\n", + "bulbs in common areas. It is your responsibility to replace light bulbs\n", + "when they break.\n", + "\n", + "On January 1, all 100 bulbs are working. When you inspect them on\n", + "February 1, you find 3 light bulbs out. If you come back on April 1, how\n", + "many light bulbs do you expect to find broken?\n", + "\n", + "In the previous exercise, you could reasonably assume that an event is\n", + "equally likely at any time. For light bulbs, the likelihood of failure\n", + "depends on the age of the bulb. Specifically, old bulbs have an\n", + "increasing failure rate due to evaporation of the filament.\n", + "\n", + "This problem is more open-ended than some; you will have to make\n", + "modeling decisions. You might want to read about the Weibull\n", + "distribution (). Or\n", + "you might want to look around for information about light bulb survival\n", + "curves." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/08_Observer_Bias.ipynb b/notebooks/08_Observer_Bias.ipynb index b6cc953..64ae415 100644 --- a/notebooks/08_Observer_Bias.ipynb +++ b/notebooks/08_Observer_Bias.ipynb @@ -1,969 +1,1274 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n" - ], - "text/plain": [ - "" - ] - }, - "execution_count": 1, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# format the book\n", - "%matplotlib inline\n", - "import sys\n", - "from __future__ import division, print_function\n", - "%load_ext autoreload\n", - "%autoreload 2 \n", - "import sys\n", - "sys.path.insert(0,'../code')\n", - "import book_format\n", - "book_format.load_style('../code')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#Observer Bias\n", - "\n", - "## The Red Line problem\n", - "\n", - "In Massachusetts, the Red Line is a subway that connects Cambridge and\n", - "Boston. When I was working in Cambridge I took the Red Line from Kendall\n", - "Square to South Station and caught the commuter rail to Needham. During\n", - "rush hour Red Line trains run every 7–8 minutes, on average.\n", - "\n", - "When I arrived at the station, I could estimate the time until the next\n", - "train based on the number of passengers on the platform. If there were\n", - "only a few people, I inferred that I just missed a train and expected to\n", - "wait about 7 minutes. If there were more passengers, I expected the\n", - "train to arrive sooner. But if there were a large number of passengers,\n", - "I suspected that trains were not running on schedule, so I would go back\n", - "to the street level and get a taxi.\n", - "\n", - "While I was waiting for trains, I thought about how Bayesian estimation\n", - "could help predict my wait time and decide when I should give up and\n", - "take a taxi. This chapter presents the analysis I came up with.\n", - "\n", - "This chapter is based on a project by Brendan Ritter and Kai Austin, who\n", - "took a class with me at Olin College. The code in this chapter is\n", - "available from . The code I used to\n", - "collect data is in . For more\n", - "information see Section [download]." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "##The model\n", - "\n", - "![PMF of gaps between trains, based on collected data, smoothed by KDE.\n", - "`z` is the actual distribution; `zb` is the biased\n", - "distribution seen by passengers. ](figs/redline0.pdf)\n", - "\n", - "[fig.redline0]\n", - "\n", - "Before we get to the analysis, we have to make some modeling decisions.\n", - "First, I will treat passenger arrivals as a Poisson process, which means\n", - "I assume that passengers are equally likely to arrive at any time, and\n", - "that they arrive at an unknown rate, $\\lambda$, measured in passengers\n", - "per minute. Since I observe passengers during a short period of time,\n", - "and at the same time every day, I assume that $\\lambda$ is constant.\n", - "\n", - "On the other hand, the arrival process for trains is not Poisson. Trains\n", - "to Boston are supposed to leave from the end of the line (Alewife\n", - "station) every 7–8 minutes during peak times, but by the time they get\n", - "to Kendall Square, the time between trains varies between 3 and 12\n", - "minutes.\n", - "\n", - "To gather data on the time between trains, I wrote a script that\n", - "downloads real-time data from\n", - ", selects south-bound\n", - "trains arriving at Kendall square, and records their arrival times in a\n", - "database. I ran the script from 4pm to 6pm every weekday for 5 days, and\n", - "recorded about 15 arrivals per day. Then I computed the time between\n", - "consecutive arrivals; the distribution of these gaps is shown in\n", - "Figure [fig.redline0], labeled `z`.\n", - "\n", - "If you stood on the platform from 4pm to 6pm and recorded the time\n", - "between trains, this is the distribution you would see. But if you\n", - "arrive at some random time (without regard to the train schedule) you\n", - "would see a different distribution. The average time between trains, as\n", - "seen by a random passenger, is substantially higher than the true\n", - "average.\n", - "\n", - "Why? Because a passenger is more like to arrive during a large interval\n", - "than a small one. Consider a simple example: suppose that the time\n", - "between trains is either 5 minutes or 10 minutes with equal probability.\n", - "In that case the average time between trains is 7.5 minutes.\n", - "\n", - "But a passenger is more likely to arrive during a 10 minute gap than a 5\n", - "minute gap; in fact, twice as likely. If we surveyed arriving\n", - "passengers, we would find that 2/3 of them arrived during a 10 minute\n", - "gap, and only 1/3 during a 5 minute gap. So the average time between\n", - "trains, as seen by an arriving passenger, is 8.33 minutes.\n", - "\n", - "This kind of **observer bias** appears in many contexts.\n", - "Students think that classes are bigger than they are because more of\n", - "them are in the big classes. Airline passengers think that planes are\n", - "fuller than they are because more of them are on full flights.\n", - "\n", - "In each case, values from the actual distribution are oversampled in\n", - "proportion to their value. In the Red Line example, a gap that is twice\n", - "as big is twice as likely to be observed.\n", - "\n", - "So given the actual distribution of gaps, we can compute the\n", - "distribution of gaps as seen by passengers. `BiasPmf` does\n", - "this computation:" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "def BiasPmf(pmf):\n", - " new_pmf = pmf.Copy()\n", - "\n", - " for x, p in pmf.Items():\n", - " new_pmf.Mult(x, x)\n", - "\n", - " new_pmf.Normalize()\n", - " return new_pmf" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`pmf` is the actual distribution; `new_pmf` is the biased\n", - "distribution. Inside the loop, we multiply the probability of each\n", - "value, `x`, by the likelihood it will be observed, which is\n", - "proportional to `x`. Then we normalize the result.\n", - "\n", - "Figure [fig.redline0] shows the actual distribution of gaps, labeled\n", - "`z`, and the distribution of gaps seen by passengers, labeled\n", - "`zb` for “z biased”.\n", - "\n", - "Wait times\n", - "----------\n", - "\n", - "![CDF of `z`, `zb`, and the wait time seen by\n", - "passengers, `y`. ](figs/redline2.pdf)\n", - "\n", - "[fig.redline2]\n", - "\n", - "Wait time, which I call `y`, is the time between the arrival\n", - "of a passenger and the next arrival of a train. Elapsed time, which I\n", - "call `x`, is the time between the arrival of the previous\n", - "train and the arrival of a passenger. I chose these definitions so that\n", - "`zb = x + y`.\n", - "\n", - "Given the distribution of `zb`, we can compute the\n", - "distribution of `y`. I’ll start with a simple case and then\n", - "generalize. Suppose, as in the previous example, that `zb` is\n", - "either 5 minutes with probability 1/3, or 10 minutes with probability\n", - "2/3.\n", - "\n", - "If we arrive at a random time during a 5 minute gap, `y` is\n", - "uniform from 0 to 5 minutes. If we arrive during a 10 minute gap,\n", - "`y` is uniform from 0 to 10. So the overall distribution is a\n", - "mixture of uniform distributions weighted according to the probability\n", - "of each gap.\n", - "\n", - "The following function takes the distribution of `zb` and\n", - "computes the distribution of `y`:\n", - "\n", - " def PmfOfWaitTime(pmf_zb):\n", - " metapmf = thinkbayes.Pmf()\n", - " for gap, prob in pmf_zb.Items():\n", - " uniform = MakeUniformPmf(0, gap)\n", - " metapmf.Set(uniform, prob)\n", - "\n", - " pmf_y = thinkbayes.MakeMixture(metapmf)\n", - " return pmf_y\n", - "\n", - "`PmfOfWaitTime` makes a meta-Pmf that maps from each uniform\n", - "distribution to its probability. Then it uses `MakeMixture`,\n", - "which we saw in Section [mixture], to compute the mixture.\n", - "\n", - "`PmfOfWaitTime` also uses `MakeUniformPmf`,\n", - "defined here:\n", - "\n", - " def MakeUniformPmf(low, high):\n", - " pmf = thinkbayes.Pmf()\n", - " for x in MakeRange(low=low, high=high):\n", - " pmf.Set(x, 1)\n", - " pmf.Normalize()\n", - " return pmf\n", - "\n", - "`low` and `high` are the range of the uniform\n", - "distribution, (both ends included). Finally, `MakeUniformPmf`\n", - "uses `MakeRange`, defined here:\n", - "\n", - " def MakeRange(low, high, skip=10):\n", - " return range(low, high+skip, skip)\n", - "\n", - "`MakeRange` defines a set of possible values for wait time\n", - "(expressed in seconds). By default it divides the range into 10 second\n", - "intervals.\n", - "\n", - "To encapsulate the process of computing these distributions, I created a\n", - "class called `WaitTimeCalculator`:\n", - "\n", - " class WaitTimeCalculator(object):\n", - "\n", - " def __init__(self, pmf_z):\n", - " self.pmf_z = pmf_z\n", - " self.pmf_zb = BiasPmf(pmf)\n", - "\n", - " self.pmf_y = self.PmfOfWaitTime(self.pmf_zb)\n", - " self.pmf_x = self.pmf_y\n", - "\n", - "The parameter, `pmf_z`, is the unbiased distribution of `z`.\n", - "`pmf_zb` is the biased distribution of gap time, as seen by passengers.\n", - "\n", - "`pmf_y` is the distribution of wait time. `pmf_x` is the distribution of\n", - "elapsed time, which is the same as the distribution of wait time. To see\n", - "why, remember that for a particular value of `zp`, the\n", - "distribution of `y` is uniform from 0 to `zp`.\n", - "Also\n", - "\n", - " x = zp - y\n", - "\n", - "So the distribution of `x` is also uniform from 0 to\n", - "`zp`.\n", - "\n", - "Figure [fig.redline2] shows the distribution of `z`,\n", - "`zb`, and `y` based on the data I collected from\n", - "the Red Line web site.\n", - "\n", - "To present these distributions, I am switching from Pmfs to Cdfs. Most\n", - "people are more familiar with Pmfs, but I think Cdfs are easier to\n", - "interpret, once you get used to them. And if you want to plot several\n", - "distributions on the same axes, Cdfs are the way to go.\n", - "\n", - "The mean of `z` is 7.8 minutes. The mean of `zb`\n", - "is 8.8 minutes, about 13% higher. The mean of `y` is 4.4,\n", - "half the mean of `zb`.\n", - "\n", - "As an aside, the Red Line schedule reports that trains run every 9\n", - "minutes during peak times. This is close to the average of\n", - "`zb`, but higher than the average of `z`. I\n", - "exchanged email with a representative of the MBTA, who confirmed that\n", - "the reported time between trains is deliberately conservative in order\n", - "to account for variability.\n", - "\n", - "Predicting wait times {#elapsed}\n", - "---------------------\n", - "\n", - "![Prior and posterior of `x` and predicted `y`.\n", - "](figs/redline3.pdf)\n", - "\n", - "[fig.redline3]\n", - "\n", - "Let’s get back to the motivating question: suppose that when I arrive at\n", - "the platform I see 10 people waiting. How long should I expect to wait\n", - "until the next train arrives?\n", - "\n", - "As always, let’s start with the easiest version of the problem and work\n", - "our way up. Suppose we are given the actual distribution of\n", - "`z`, and we know that the passenger arrival rate, $\\lambda$,\n", - "is 2 passengers per minute.\n", - "\n", - "In that case we can:\n", - "\n", - "1. Use the distribution of `z` to compute the prior\n", - " distribution of `zp`, the time between trains as seen by\n", - " a passenger.\n", - "\n", - "2. Then we can use the number of passengers to estimate the\n", - " distribution of `x`, the elapsed time since the last\n", - " train.\n", - "\n", - "3. Finally, we use the relation `y = zp - x` to get the\n", - " distribution of `y`.\n", - "\n", - "The first step is to create a `WaitTimeCalculator` that\n", - "encapsulates the distributions of `zp`, `x`, and\n", - "`y`, prior to taking into account the number of passengers.\n", - "\n", - " wtc = WaitTimeCalculator(pmf_z)\n", - "\n", - "`pmf_z` is the given distribution of gap times.\n", - "\n", - "The next step is to make an `ElapsedTimeEstimator` (defined\n", - "below), which encapsulates the posterior distribution of `x`\n", - "and the predictive distribution of `y`.\n", - "\n", - " ete = ElapsedTimeEstimator(wtc,\n", - " lam=2.0/60,\n", - " num_passengers=15)\n", - "\n", - "The parameters are the `WaitTimeCalculator`, the passenger\n", - "arrival rate, `lam` (expressed in passengers per second), and\n", - "the observed number of passengers, let’s say 15.\n", - "\n", - "Here is the definition of `ElapsedTimeEstimator`:\n", - "\n", - " class ElapsedTimeEstimator(object):\n", - "\n", - " def __init__(self, wtc, lam, num_passengers):\n", - " self.prior_x = Elapsed(wtc.pmf_x)\n", - "\n", - " self.post_x = self.prior_x.Copy()\n", - " self.post_x.Update((lam, num_passengers))\n", - "\n", - " self.pmf_y = PredictWaitTime(wtc.pmf_zb, self.post_x)\n", - "\n", - "`prior_x` and `posterior_x` are the prior and posterior distributions of\n", - "elapsed time. `pmf_y` is the predictive distribution of wait time.\n", - "\n", - "`ElapsedTimeEstimator` uses `Elapsed` and\n", - "`PredictWaitTime`, defined below.\n", - "\n", - "`Elapsed` is a Suite that represents the hypothetical\n", - "distribution of `x`. The prior distribution of `x`\n", - "comes straight from the `WaitTimeCalculator`. Then we use the\n", - "data, which consists of the arrival rate, `lam`, and the\n", - "number of passengers on the platform, to compute the posterior\n", - "distribution.\n", - "\n", - "Here’s the definition of `Elapsed`:\n", - "\n", - " class Elapsed(thinkbayes.Suite):\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " x = hypo\n", - " lam, k = data\n", - " like = thinkbayes.EvalPoissonPmf(k, lam * x)\n", - " return like\n", - "\n", - "As always, `Likelihood` takes a hypothesis and data, and\n", - "computes the likelihood of the data under the hypothesis. In this case\n", - "`hypo` is the elapsed time since the last train and\n", - "`data` is a tuple of `lam` and the number of\n", - "passengers.\n", - "\n", - "The likelihood of the data is the probability of getting `k`\n", - "arrivals in `x` time, given arrival rate `lam`. We\n", - "compute that using the PMF of the Poisson distribution.\n", - "\n", - "Finally, here’s the definition of `PredictWaitTime`:\n", - "\n", - " def PredictWaitTime(pmf_zb, pmf_x):\n", - " pmf_y = pmf_zb - pmf_x\n", - " RemoveNegatives(pmf_y)\n", - " return pmf_y\n", - "\n", - "`pmf_zb` is the distribution of gaps between trains; `pmf_x` is the\n", - "distribution of elapsed time, based on the observed number of\n", - "passengers. Since `y = zb - x`, we can compute\n", - "\n", - " pmf_y = pmf_zb - pmf_x\n", - "\n", - "The subtraction operator invokes `Pmf.__sub__`, which enumerates all\n", - "pairs of `zb` and `x`, computes the differences,\n", - "and adds the results to `pmf_y`.\n", - "\n", - "The resulting Pmf includes some negative values, which we know are\n", - "impossible. For example, if you arrive during a gap of 5 minutes, you\n", - "can’t wait more than 5 minutes. `RemoveNegatives` removes the\n", - "impossible values from the distribution and renormalizes.\n", - "\n", - " def RemoveNegatives(pmf):\n", - " for val in pmf.Values():\n", - " if val < 0:\n", - " pmf.Remove(val)\n", - " pmf.Normalize()\n", - "\n", - "Figure [fig.redline3] shows the results. The prior distribution of\n", - "`x` is the same as the distribution of `y` in\n", - "Figure [fig.redline2]. The posterior distribution of `x`\n", - "shows that, after seeing 15 passengers on the platform, we believe that\n", - "the time since the last train is probably 5-10 minutes. The predictive\n", - "distribution of `y` indicates that we expect the next train\n", - "in less than 5 minutes, with about 80% confidence.\n", - "\n", - "Estimating the arrival rate\n", - "---------------------------\n", - "\n", - "![Prior and posterior distributions of `lam` based on five\n", - "days of passenger data. ](figs/redline1.pdf)\n", - "\n", - "[fig.redline1]\n", - "\n", - "The analysis so far has been based on the assumption that we know (1)\n", - "the distribution of gaps and (2) the passenger arrival rate. Now we are\n", - "ready to relax the second assumption.\n", - "\n", - "Suppose that you just moved to Boston, so you don’t know much about the\n", - "passenger arrival rate on the Red Line. After a few days of commuting,\n", - "you could make a guess, at least qualitatively. With a little more\n", - "effort, you could estimate $\\lambda$ quantitatively.\n", - "\n", - "Each day when you arrive at the platform, you should note the time and\n", - "the number of passengers waiting (if the platform is too big, you could\n", - "choose a sample area). Then you should record your wait time and the\n", - "number of new arrivals while you are waiting.\n", - "\n", - "After five days, you might have data like this:\n", - "\n", - " k1 y k2\n", - " -- --- --\n", - " 17 4.6 9\n", - " 22 1.0 0\n", - " 23 1.4 4\n", - " 18 5.4 12\n", - " 4 5.8 11\n", - "\n", - "where `k1` is the number of passengers waiting when you\n", - "arrive, `y` is your wait time in minutes, and `k2`\n", - "is the number of passengers who arrive while you are waiting.\n", - "\n", - "Over the course of one week, you waited 18 minutes and saw 36 passengers\n", - "arrive, so you would estimate that the arrival rate is 2 passengers per\n", - "minute. For practical purposes that estimate is good enough, but for the\n", - "sake of completeness I will compute a posterior distribution for\n", - "$\\lambda$ and show how to use that distribution in the rest of the\n", - "analysis.\n", - "\n", - "`ArrivalRate` is a `Suite` that represents\n", - "hypotheses about $\\lambda$. As always, `Likelihood` takes a\n", - "hypothesis and data, and computes the likelihood of the data under the\n", - "hypothesis.\n", - "\n", - "In this case the hypothesis is a value of $\\lambda$. The data is a pair,\n", - "`y, k`, where `y` is a wait time and\n", - "`k` is the number of passengers that arrived.\n", - "\n", - " class ArrivalRate(thinkbayes.Suite):\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " lam = hypo\n", - " y, k = data\n", - " like = thinkbayes.EvalPoissonPmf(k, lam * y)\n", - " return like\n", - "\n", - "This `Likelihood` might look familiar; it is almost identical\n", - "to `Elapsed.Likelihood` in Section [elapsed]. The difference\n", - "is that in `Elapsed.Likelihood` the hypothesis is\n", - "`x`, the elapsed time; in `ArrivalRate.Likelihood`\n", - "the hypothesis is `lam`, the arrival rate. But in both cases\n", - "the likelihood is the probability of seeing `k` arrivals in\n", - "some period of time, given `lam`.\n", - "\n", - "`ArrivalRateEstimator` encapsulates the process of estimating\n", - "$\\lambda$. The parameter, `passenger_data`, is a list of `k1, y,\n", - "k2` tuples, as in the table above.\n", - "\n", - " class ArrivalRateEstimator(object):\n", - "\n", - " def __init__(self, passenger_data):\n", - " low, high = 0, 5\n", - " n = 51\n", - " hypos = numpy.linspace(low, high, n) / 60\n", - "\n", - " self.prior_lam = ArrivalRate(hypos)\n", - "\n", - " self.post_lam = self.prior_lam.Copy()\n", - " for k1, y, k2 in passenger_data:\n", - " self.post_lam.Update((y, k2))\n", - "\n", - "`__init__` builds `hypos`, which is a sequence of\n", - "hypothetical values for `lam`, then builds the prior\n", - "distribution, `prior_lam`. The `for` loop updates the prior\n", - "with data, yielding the posterior distribution, `post_lam`.\n", - "\n", - "Figure [fig.redline1] shows the prior and posterior distributions. As\n", - "expected, the mean and median of the posterior are near the observed\n", - "rate, 2 passengers per minute. But the spread of the posterior\n", - "distribution captures our uncertainty about $\\lambda$ based on a small\n", - "sample.\n", - "\n", - "Incorporating uncertainty\n", - "-------------------------\n", - "\n", - "![Predictive distributions of `y` for possible values of\n", - "`lam`. ](figs/redline4.pdf)\n", - "\n", - "[fig.redline4]\n", - "\n", - "Whenever there is uncertainty about one of the inputs to an analysis, we\n", - "can take it into account by a process like this:\n", - "\n", - "1. Implement the analysis based on a deterministic value of the\n", - " uncertain parameter (in this case $\\lambda$).\n", - "\n", - "2. Compute the distribution of the uncertain parameter.\n", - "\n", - "3. Run the analysis for each value of the parameter, and generate a set\n", - " of predictive distributions.\n", - "\n", - "4. Compute a mixture of the predictive distributions, using the weights\n", - " from the distribution of the parameter.\n", - "\n", - "We have already done steps (1) and (2). I wrote a class called\n", - "`WaitMixtureEstimator` to handle steps (3) and (4).\n", - "\n", - " class WaitMixtureEstimator(object):\n", - "\n", - " def __init__(self, wtc, are, num_passengers=15):\n", - " self.metapmf = thinkbayes.Pmf()\n", - "\n", - " for lam, prob in sorted(are.post_lam.Items()):\n", - " ete = ElapsedTimeEstimator(wtc, lam, num_passengers)\n", - " self.metapmf.Set(ete.pmf_y, prob)\n", - "\n", - " self.mixture = thinkbayes.MakeMixture(self.metapmf)\n", - "\n", - "`wtc` is the `WaitTimeCalculator` that contains\n", - "the distribution of `zb`. `are` is the\n", - "`ArrivalTimeEstimator` that contains the distribution of\n", - "`lam`.\n", - "\n", - "The first line makes a meta-Pmf that maps from each possible\n", - "distribution of `y` to its probability. For each value of\n", - "`lam`, we use `ElapsedTimeEstimator` to compute\n", - "the corresponding distribution of `y` and store it in the\n", - "Meta-Pmf. Then we use `MakeMixture` to compute the mixture.\n", - "\n", - "Figure [fig.redline4] shows the results. The shaded lines in the\n", - "background are the distributions of `y` for each value of\n", - "`lam`, with line thickness that represents likelihood. The\n", - "dark line is the mixture of these distributions.\n", - "\n", - "In this case we could get a very similar result using a single point\n", - "estimate of `lam`. So it was not necessary, for practical\n", - "purposes, to include the uncertainty of the estimate.\n", - "\n", - "In general, it is important to include variability if the system\n", - "response is non-linear; that is, if small changes in the input can cause\n", - "big changes in the output. In this case, posterior variability in\n", - "`lam` is small and the system response is approximately\n", - "linear for small perturbations.\n", - "\n", - "Decision analysis {#decision-analysis}\n", - "-----------------\n", - "\n", - "![Probability that wait time exceeds 15 minutes as a function of the\n", - "number of passengers on the platform. ](figs/redline5.pdf)\n", - "\n", - "[fig.redline5]\n", - "\n", - "At this point we can use the number of passengers on the platform to\n", - "predict the distribution of wait times. Now let’s get to the second part\n", - "of the question: when should I stop waiting for the train and go catch a\n", - "taxi?\n", - "\n", - "Remember that in the original scenario, I am trying to get to South\n", - "Station to catch the commuter rail. Suppose I leave the office with\n", - "enough time that I can wait 15 minutes and still make my connection at\n", - "South Station.\n", - "\n", - "In that case I would like to know the probability that `y`\n", - "exceeds 15 minutes as a function of `num_passengers`. It is easy enough\n", - "to use the analysis from Section [elapsed] and run it for a range of\n", - "`num_passengers`.\n", - "\n", - "But there’s a problem. The analysis is sensitive to the frequency of\n", - "long delays, and because long delays are rare, it is hard estimate their\n", - "frequency.\n", - "\n", - "I only have data from one week, and the longest delay I observed was 15\n", - "minutes. So I can’t estimate the frequency of longer delays accurately.\n", - "\n", - "However, I can use previous observations to make at least a coarse\n", - "estimate. When I commuted by Red Line for a year, I saw three long\n", - "delays caused by a signaling problem, a power outage, and “police\n", - "activity” at another stop. So I estimate that there are about 3 major\n", - "delays per year.\n", - "\n", - "But remember that my observations are biased. I am more likely to\n", - "observe long delays because they affect a large number of passengers. So\n", - "we should treat my observations as a sample of `zb` rather\n", - "than `z`. Here’s how we can do that.\n", - "\n", - "During my year of commuting, I took the Red Line home about 220 times.\n", - "So I take the observed gap times, `gap_times`, generate a sample of 220\n", - "gaps, and compute their Pmf:\n", - "\n", - " n = 220\n", - " cdf_z = thinkbayes.MakeCdfFromList(gap_times)\n", - " sample_z = cdf_z.Sample(n)\n", - " pmf_z = thinkbayes.MakePmfFromList(sample_z)\n", - "\n", - "Next I bias `pmf_z` to get the distribution of `zb`, draw a\n", - "sample, and then add in delays of 30, 40, and 50 minutes (expressed in\n", - "seconds):\n", - "\n", - " cdf_zp = BiasPmf(pmf_z).MakeCdf()\n", - " sample_zb = cdf_zp.Sample(n) + [1800, 2400, 3000]\n", - "\n", - "`Cdf.Sample` is more efficient than `Pmf.Sample`,\n", - "so it is usually faster to convert a Pmf to a Cdf before sampling.\n", - "\n", - "Next I use the sample of `zb` to estimate a Pdf using KDE,\n", - "and then convert the Pdf to a Pmf:\n", - "\n", - " pdf_zb = thinkbayes.EstimatedPdf(sample_zb)\n", - " xs = MakeRange(low=60)\n", - " pmf_zb = pdf_zb.MakePmf(xs)\n", - "\n", - "Finally I unbias the distribution of `zb` to get the\n", - "distribution of `z`, which I use to create the\n", - "`WaitTimeCalculator`:\n", - "\n", - " pmf_z = UnbiasPmf(pmf_zb)\n", - " wtc = WaitTimeCalculator(pmf_z)\n", - "\n", - "This process is complicated, but all of the steps are operations we have\n", - "seen before. Now we are ready to compute the probability of a long wait.\n", - "\n", - " def ProbLongWait(num_passengers, minutes):\n", - " ete = ElapsedTimeEstimator(wtc, lam, num_passengers)\n", - " cdf_y = ete.pmf_y.MakeCdf()\n", - " prob = 1 - cdf_y.Prob(minutes * 60)\n", - "\n", - "Given the number of passengers on the platform,\n", - "`ProbLongWait` makes an `ElapsedTimeEstimator`,\n", - "extracts the distribution of wait time, and computes the probability\n", - "that wait time exceeds `minutes`.\n", - "\n", - "Figure [fig.redline5] shows the result. When the number of passengers is\n", - "less than 20, we infer that the system is operating normally, so the\n", - "probability of a long delay is small. If there are 30 passengers, we\n", - "estimate that it has been 15 minutes since the last train; that’s longer\n", - "than a normal delay, so we infer that something is wrong and expect\n", - "longer delays.\n", - "\n", - "If we are willing to accept a 10% chance of missing the connection at\n", - "South Station, we should stay and wait as long as there are fewer than\n", - "30 passengers, and take a taxi if there are more.\n", - "\n", - "Or, to take this analysis one step further, we could quantify the cost\n", - "of missing the connection and the cost of taking a taxi, then choose the\n", - "threshold that minimizes expected cost.\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "The analysis so far has been based on the assumption that the arrival\n", - "rate of passengers is the same every day. For a commuter train during\n", - "rush hour, that might not be a bad assumption, but there are some\n", - "obvious exceptions. For example, if there is a special event nearby, a\n", - "large number of people might arrive at the same time. In that case, the\n", - "estimate of `lam` would be too low, so the estimates of\n", - "`x` and `y` would be too high.\n", - "\n", - "If special events are as common as major delays, it would be important\n", - "to include them in the model. We could do that by extending the\n", - "distribution of `lam` to include occasional large values.\n", - "\n", - "We started with the assumption that we know distribution of\n", - "`z`. As an alternative, a passenger could estimate\n", - "`z`, but it would not be easy. As a passenger, you only\n", - "observe only your own wait time, `y`. Unless you skip the\n", - "first train and wait for the second, you don’t observe the gap between\n", - "trains, `z`.\n", - "\n", - "However, we could make some inferences about `zb`. If we note\n", - "the number of passengers waiting when we arrive, we can estimate the\n", - "elapsed time since the last train, `x`. Then we observe\n", - "`y`. If we add the posterior distribution of `x`\n", - "to the observed `y`, we get a distribution that represents\n", - "our posterior belief about the observed value of `zb`.\n", - "\n", - "We can use this distribution to update our beliefs about the\n", - "distribution of `zb`. Finally, we can compute the inverse of\n", - "`BiasPmf` to get from the distribution of `zb` to\n", - "the distribution of `z`.\n", - "\n", - "I leave this analysis as an exercise for the reader. One suggestion: you\n", - "should read Chapter [species] first. You can find the outline of a\n", - "solution in . For more information see\n", - "Section [download].\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "This exercise is from MacKay, *Information Theory, Inference, and\n", - "Learning Algorithms*:\n", - "\n", - "> Unstable particles are emitted from a source and decay at a distance\n", - "> $x$, a real number that has an exponential probability distribution\n", - "> with [parameter] $\\lambda$. Decay events can only be observed if they\n", - "> occur in a window extending from $x=1$ cm to $x=20$ cm. $N$ decays are\n", - "> observed at locations $\\{ 1.5, 2, 3, 4, 5, 12 \\}$ cm. What is the\n", - "> posterior distribution of $\\lambda$?\n", - "\n", - "You can download a solution to this exercise from\n", - "." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Observer Bias\n", + "\n", + "## The Red Line problem\n", + "\n", + "In Massachusetts, the Red Line is a subway that connects Cambridge and\n", + "Boston. When I was working in Cambridge I took the Red Line from Kendall\n", + "Square to South Station and caught the commuter rail to Needham. During\n", + "rush hour Red Line trains run every 7–8 minutes, on average.\n", + "\n", + "When I arrived at the station, I could estimate the time until the next\n", + "train based on the number of passengers on the platform. If there were\n", + "only a few people, I inferred that I just missed a train and expected to\n", + "wait about 7 minutes. If there were more passengers, I expected the\n", + "train to arrive sooner. But if there were a large number of passengers,\n", + "I suspected that trains were not running on schedule, so I would go back\n", + "to the street level and get a taxi.\n", + "\n", + "While I was waiting for trains, I thought about how Bayesian estimation\n", + "could help predict my wait time and decide when I should give up and\n", + "take a taxi. This chapter presents the analysis I came up with.\n", + "\n", + "This chapter is based on a project by Brendan Ritter and Kai Austin, who\n", + "took a class with me at Olin College. The code in this chapter is\n", + "available from . The code I used to\n", + "collect data is in . For more\n", + "information see Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The model\n", + "\n", + "![PMF of gaps between trains, based on collected data, smoothed by KDE.\n", + "`z` is the actual distribution; `zb` is the biased\n", + "distribution seen by passengers. ](figs/redline0.pdf)\n", + "\n", + "[fig.redline0]\n", + "\n", + "Before we get to the analysis, we have to make some modeling decisions.\n", + "First, I will treat passenger arrivals as a Poisson process, which means\n", + "I assume that passengers are equally likely to arrive at any time, and\n", + "that they arrive at an unknown rate, $\\lambda$, measured in passengers\n", + "per minute. Since I observe passengers during a short period of time,\n", + "and at the same time every day, I assume that $\\lambda$ is constant.\n", + "\n", + "On the other hand, the arrival process for trains is not Poisson. Trains\n", + "to Boston are supposed to leave from the end of the line (Alewife\n", + "station) every 7–8 minutes during peak times, but by the time they get\n", + "to Kendall Square, the time between trains varies between 3 and 12\n", + "minutes.\n", + "\n", + "To gather data on the time between trains, I wrote a script that\n", + "downloads real-time data from\n", + ", selects south-bound\n", + "trains arriving at Kendall square, and records their arrival times in a\n", + "database. I ran the script from 4pm to 6pm every weekday for 5 days, and\n", + "recorded about 15 arrivals per day. Then I computed the time between\n", + "consecutive arrivals; the distribution of these gaps is shown in\n", + "Figure [fig.redline0], labeled `z`.\n", + "\n", + "If you stood on the platform from 4pm to 6pm and recorded the time\n", + "between trains, this is the distribution you would see. But if you\n", + "arrive at some random time (without regard to the train schedule) you\n", + "would see a different distribution. The average time between trains, as\n", + "seen by a random passenger, is substantially higher than the true\n", + "average.\n", + "\n", + "Why? Because a passenger is more like to arrive during a large interval\n", + "than a small one. Consider a simple example: suppose that the time\n", + "between trains is either 5 minutes or 10 minutes with equal probability.\n", + "In that case the average time between trains is 7.5 minutes.\n", + "\n", + "But a passenger is more likely to arrive during a 10 minute gap than a 5\n", + "minute gap; in fact, twice as likely. If we surveyed arriving\n", + "passengers, we would find that 2/3 of them arrived during a 10 minute\n", + "gap, and only 1/3 during a 5 minute gap. So the average time between\n", + "trains, as seen by an arriving passenger, is 8.33 minutes.\n", + "\n", + "This kind of **observer bias** appears in many contexts.\n", + "Students think that classes are bigger than they are because more of\n", + "them are in the big classes. Airline passengers think that planes are\n", + "fuller than they are because more of them are on full flights.\n", + "\n", + "In each case, values from the actual distribution are oversampled in\n", + "proportion to their value. In the Red Line example, a gap that is twice\n", + "as big is twice as likely to be observed.\n", + "\n", + "So given the actual distribution of gaps, we can compute the\n", + "distribution of gaps as seen by passengers. `BiasPmf` does\n", + "this computation:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def BiasPmf(pmf):\n", + " new_pmf = pmf.Copy()\n", + "\n", + " for x, p in pmf.Items():\n", + " new_pmf.Mult(x, x)\n", + "\n", + " new_pmf.Normalize()\n", + " return new_pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pmf` is the actual distribution; `new_pmf` is the biased\n", + "distribution. Inside the loop, we multiply the probability of each\n", + "value, `x`, by the likelihood it will be observed, which is\n", + "proportional to `x`. Then we normalize the result.\n", + "\n", + "Figure [fig.redline0] shows the actual distribution of gaps, labeled\n", + "`z`, and the distribution of gaps seen by passengers, labeled\n", + "`zb` for “z biased”." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Wait times\n", + "\n", + "![CDF of `z`, `zb`, and the wait time seen by\n", + "passengers, `y`. ](figs/redline2.pdf)\n", + "\n", + "[fig.redline2]\n", + "\n", + "Wait time, which I call `y`, is the time between the arrival\n", + "of a passenger and the next arrival of a train. Elapsed time, which I\n", + "call `x`, is the time between the arrival of the previous\n", + "train and the arrival of a passenger. I chose these definitions so that\n", + "`zb = x + y`.\n", + "\n", + "Given the distribution of `zb`, we can compute the\n", + "distribution of `y`. I’ll start with a simple case and then\n", + "generalize. Suppose, as in the previous example, that `zb` is\n", + "either 5 minutes with probability 1/3, or 10 minutes with probability\n", + "2/3.\n", + "\n", + "If we arrive at a random time during a 5 minute gap, `y` is\n", + "uniform from 0 to 5 minutes. If we arrive during a 10 minute gap,\n", + "`y` is uniform from 0 to 10. So the overall distribution is a\n", + "mixture of uniform distributions weighted according to the probability\n", + "of each gap.\n", + "\n", + "The following function takes the distribution of `zb` and\n", + "computes the distribution of `y`:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def PmfOfWaitTime(pmf_zb):\n", + " metapmf = thinkbayes.Pmf()\n", + " for gap, prob in pmf_zb.Items():\n", + " uniform = MakeUniformPmf(0, gap)\n", + " metapmf.Set(uniform, prob)\n", + "\n", + " pmf_y = thinkbayes.MakeMixture(metapmf)\n", + " return pmf_y" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`PmfOfWaitTime` makes a meta-Pmf that maps from each uniform\n", + "distribution to its probability. Then it uses `MakeMixture`,\n", + "which we saw in Section [mixture], to compute the mixture.\n", + "\n", + "`PmfOfWaitTime` also uses `MakeUniformPmf`,\n", + "defined here:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeUniformPmf(low, high):\n", + " pmf = thinkbayes.Pmf()\n", + " for x in MakeRange(low=low, high=high):\n", + " pmf.Set(x, 1)\n", + " pmf.Normalize()\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`low` and `high` are the range of the uniform\n", + "distribution, (both ends included). Finally, `MakeUniformPmf`\n", + "uses `MakeRange`, defined here:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeRange(low, high, skip=10):\n", + " return range(low, high+skip, skip)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`MakeRange` defines a set of possible values for wait time\n", + "(expressed in seconds). By default it divides the range into 10 second\n", + "intervals.\n", + "\n", + "To encapsulate the process of computing these distributions, I created a\n", + "class called `WaitTimeCalculator`:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class WaitTimeCalculator(object):\n", + "\n", + " def __init__(self, pmf_z):\n", + " self.pmf_z = pmf_z\n", + " self.pmf_zb = BiasPmf(pmf)\n", + "\n", + " self.pmf_y = self.PmfOfWaitTime(self.pmf_zb)\n", + " self.pmf_x = self.pmf_y" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The parameter, `pmf_z`, is the unbiased distribution of `z`.\n", + "`pmf_zb` is the biased distribution of gap time, as seen by passengers.\n", + "\n", + "`pmf_y` is the distribution of wait time. `pmf_x` is the distribution of\n", + "elapsed time, which is the same as the distribution of wait time. To see\n", + "why, remember that for a particular value of `zp`, the\n", + "distribution of `y` is uniform from 0 to `zp`.\n", + "Also" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "x = zp - y" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So the distribution of `x` is also uniform from 0 to\n", + "`zp`.\n", + "\n", + "Figure [fig.redline2] shows the distribution of `z`,\n", + "`zb`, and `y` based on the data I collected from\n", + "the Red Line web site.\n", + "\n", + "To present these distributions, I am switching from Pmfs to Cdfs. Most\n", + "people are more familiar with Pmfs, but I think Cdfs are easier to\n", + "interpret, once you get used to them. And if you want to plot several\n", + "distributions on the same axes, Cdfs are the way to go.\n", + "\n", + "The mean of `z` is 7.8 minutes. The mean of `zb`\n", + "is 8.8 minutes, about 13% higher. The mean of `y` is 4.4,\n", + "half the mean of `zb`.\n", + "\n", + "As an aside, the Red Line schedule reports that trains run every 9\n", + "minutes during peak times. This is close to the average of\n", + "`zb`, but higher than the average of `z`. I\n", + "exchanged email with a representative of the MBTA, who confirmed that\n", + "the reported time between trains is deliberately conservative in order\n", + "to account for variability." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Predicting wait times\n", + "\n", + "![Prior and posterior of `x` and predicted `y`.\n", + "](figs/redline3.pdf)\n", + "\n", + "[fig.redline3]\n", + "\n", + "Let’s get back to the motivating question: suppose that when I arrive at\n", + "the platform I see 10 people waiting. How long should I expect to wait\n", + "until the next train arrives?\n", + "\n", + "As always, let’s start with the easiest version of the problem and work\n", + "our way up. Suppose we are given the actual distribution of\n", + "`z`, and we know that the passenger arrival rate, $\\lambda$,\n", + "is 2 passengers per minute.\n", + "\n", + "In that case we can:\n", + "\n", + "1. Use the distribution of `z` to compute the prior\n", + " distribution of `zp`, the time between trains as seen by\n", + " a passenger.\n", + "\n", + "2. Then we can use the number of passengers to estimate the\n", + " distribution of `x`, the elapsed time since the last\n", + " train.\n", + "\n", + "3. Finally, we use the relation `y = zp - x` to get the\n", + " distribution of `y`.\n", + "\n", + "The first step is to create a `WaitTimeCalculator` that\n", + "encapsulates the distributions of `zp`, `x`, and\n", + "`y`, prior to taking into account the number of passengers." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "wtc = WaitTimeCalculator(pmf_z)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pmf_z` is the given distribution of gap times.\n", + "\n", + "The next step is to make an `ElapsedTimeEstimator` (defined\n", + "below), which encapsulates the posterior distribution of `x`\n", + "and the predictive distribution of `y`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "ete = ElapsedTimeEstimator(wtc,\n", + " lam=2.0/60,\n", + " num_passengers=15)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The parameters are the `WaitTimeCalculator`, the passenger\n", + "arrival rate, `lam` (expressed in passengers per second), and\n", + "the observed number of passengers, let’s say 15.\n", + "\n", + "Here is the definition of `ElapsedTimeEstimator`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class ElapsedTimeEstimator(object):\n", + "\n", + " def __init__(self, wtc, lam, num_passengers):\n", + " self.prior_x = Elapsed(wtc.pmf_x)\n", + "\n", + " self.post_x = self.prior_x.Copy()\n", + " self.post_x.Update((lam, num_passengers))\n", + "\n", + " self.pmf_y = PredictWaitTime(wtc.pmf_zb, self.post_x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`prior_x` and `posterior_x` are the prior and posterior distributions of\n", + "elapsed time. `pmf_y` is the predictive distribution of wait time.\n", + "\n", + "`ElapsedTimeEstimator` uses `Elapsed` and\n", + "`PredictWaitTime`, defined below.\n", + "\n", + "`Elapsed` is a Suite that represents the hypothetical\n", + "distribution of `x`. The prior distribution of `x`\n", + "comes straight from the `WaitTimeCalculator`. Then we use the\n", + "data, which consists of the arrival rate, `lam`, and the\n", + "number of passengers on the platform, to compute the posterior\n", + "distribution.\n", + "\n", + "Here’s the definition of `Elapsed`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class Elapsed(thinkbayes.Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = hypo\n", + " lam, k = data\n", + " like = thinkbayes.EvalPoissonPmf(k, lam * x)\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As always, `Likelihood` takes a hypothesis and data, and\n", + "computes the likelihood of the data under the hypothesis. In this case\n", + "`hypo` is the elapsed time since the last train and\n", + "`data` is a tuple of `lam` and the number of\n", + "passengers.\n", + "\n", + "The likelihood of the data is the probability of getting `k`\n", + "arrivals in `x` time, given arrival rate `lam`. We\n", + "compute that using the PMF of the Poisson distribution.\n", + "\n", + "Finally, here’s the definition of `PredictWaitTime`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def PredictWaitTime(pmf_zb, pmf_x):\n", + " pmf_y = pmf_zb - pmf_x\n", + " RemoveNegatives(pmf_y)\n", + " return pmf_y" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`pmf_zb` is the distribution of gaps between trains; `pmf_x` is the\n", + "distribution of elapsed time, based on the observed number of\n", + "passengers. Since `y = zb - x`, we can compute" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "pmf_y = pmf_zb - pmf_x" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The subtraction operator invokes `Pmf.__sub__`, which enumerates all\n", + "pairs of `zb` and `x`, computes the differences,\n", + "and adds the results to `pmf_y`.\n", + "\n", + "The resulting Pmf includes some negative values, which we know are\n", + "impossible. For example, if you arrive during a gap of 5 minutes, you\n", + "can’t wait more than 5 minutes. `RemoveNegatives` removes the\n", + "impossible values from the distribution and renormalizes." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def RemoveNegatives(pmf):\n", + " for val in pmf.Values():\n", + " if val < 0:\n", + " pmf.Remove(val)\n", + " pmf.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure [fig.redline3] shows the results. The prior distribution of\n", + "`x` is the same as the distribution of `y` in\n", + "Figure [fig.redline2]. The posterior distribution of `x`\n", + "shows that, after seeing 15 passengers on the platform, we believe that\n", + "the time since the last train is probably 5-10 minutes. The predictive\n", + "distribution of `y` indicates that we expect the next train\n", + "in less than 5 minutes, with about 80% confidence." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Estimating the arrival rate\n", + "\n", + "![Prior and posterior distributions of `lam` based on five\n", + "days of passenger data. ](figs/redline1.pdf)\n", + "\n", + "[fig.redline1]\n", + "\n", + "The analysis so far has been based on the assumption that we know (1)\n", + "the distribution of gaps and (2) the passenger arrival rate. Now we are\n", + "ready to relax the second assumption.\n", + "\n", + "Suppose that you just moved to Boston, so you don’t know much about the\n", + "passenger arrival rate on the Red Line. After a few days of commuting,\n", + "you could make a guess, at least qualitatively. With a little more\n", + "effort, you could estimate $\\lambda$ quantitatively.\n", + "\n", + "Each day when you arrive at the platform, you should note the time and\n", + "the number of passengers waiting (if the platform is too big, you could\n", + "choose a sample area). Then you should record your wait time and the\n", + "number of new arrivals while you are waiting.\n", + "\n", + "After five days, you might have data like this:\n", + "\n", + " k1 y k2\n", + " -- --- --\n", + " 17 4.6 9\n", + " 22 1.0 0\n", + " 23 1.4 4\n", + " 18 5.4 12\n", + " 4 5.8 11\n", + "\n", + "where `k1` is the number of passengers waiting when you\n", + "arrive, `y` is your wait time in minutes, and `k2`\n", + "is the number of passengers who arrive while you are waiting.\n", + "\n", + "Over the course of one week, you waited 18 minutes and saw 36 passengers\n", + "arrive, so you would estimate that the arrival rate is 2 passengers per\n", + "minute. For practical purposes that estimate is good enough, but for the\n", + "sake of completeness I will compute a posterior distribution for\n", + "$\\lambda$ and show how to use that distribution in the rest of the\n", + "analysis.\n", + "\n", + "`ArrivalRate` is a `Suite` that represents\n", + "hypotheses about $\\lambda$. As always, `Likelihood` takes a\n", + "hypothesis and data, and computes the likelihood of the data under the\n", + "hypothesis.\n", + "\n", + "In this case the hypothesis is a value of $\\lambda$. The data is a pair,\n", + "`y, k`, where `y` is a wait time and\n", + "`k` is the number of passengers that arrived." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class ArrivalRate(thinkbayes.Suite):\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " lam = hypo\n", + " y, k = data\n", + " like = thinkbayes.EvalPoissonPmf(k, lam * y)\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This `Likelihood` might look familiar; it is almost identical\n", + "to `Elapsed.Likelihood` in Section [elapsed]. The difference\n", + "is that in `Elapsed.Likelihood` the hypothesis is\n", + "`x`, the elapsed time; in `ArrivalRate.Likelihood`\n", + "the hypothesis is `lam`, the arrival rate. But in both cases\n", + "the likelihood is the probability of seeing `k` arrivals in\n", + "some period of time, given `lam`.\n", + "\n", + "`ArrivalRateEstimator` encapsulates the process of estimating\n", + "$\\lambda$. The parameter, `passenger_data`, is a list of `k1, y,\n", + "k2` tuples, as in the table above." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "class ArrivalRateEstimator(object):\n", + "\n", + " def __init__(self, passenger_data):\n", + " low, high = 0, 5\n", + " n = 51\n", + " hypos = numpy.linspace(low, high, n) / 60\n", + "\n", + " self.prior_lam = ArrivalRate(hypos)\n", + "\n", + " self.post_lam = self.prior_lam.Copy()\n", + " for k1, y, k2 in passenger_data:\n", + " self.post_lam.Update((y, k2))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`__init__` builds `hypos`, which is a sequence of\n", + "hypothetical values for `lam`, then builds the prior\n", + "distribution, `prior_lam`. The `for` loop updates the prior\n", + "with data, yielding the posterior distribution, `post_lam`.\n", + "\n", + "Figure [fig.redline1] shows the prior and posterior distributions. As\n", + "expected, the mean and median of the posterior are near the observed\n", + "rate, 2 passengers per minute. But the spread of the posterior\n", + "distribution captures our uncertainty about $\\lambda$ based on a small\n", + "sample." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Incorporating uncertainty\n", + "\n", + "![Predictive distributions of `y` for possible values of\n", + "`lam`. ](figs/redline4.pdf)\n", + "\n", + "[fig.redline4]\n", + "\n", + "Whenever there is uncertainty about one of the inputs to an analysis, we\n", + "can take it into account by a process like this:\n", + "\n", + "1. Implement the analysis based on a deterministic value of the\n", + " uncertain parameter (in this case $\\lambda$).\n", + "\n", + "2. Compute the distribution of the uncertain parameter.\n", + "\n", + "3. Run the analysis for each value of the parameter, and generate a set\n", + " of predictive distributions.\n", + "\n", + "4. Compute a mixture of the predictive distributions, using the weights\n", + " from the distribution of the parameter.\n", + "\n", + "We have already done steps (1) and (2). I wrote a class called\n", + "`WaitMixtureEstimator` to handle steps (3) and (4)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "class WaitMixtureEstimator(object):\n", + "\n", + " def __init__(self, wtc, are, num_passengers=15):\n", + " self.metapmf = thinkbayes.Pmf()\n", + "\n", + " for lam, prob in sorted(are.post_lam.Items()):\n", + " ete = ElapsedTimeEstimator(wtc, lam, num_passengers)\n", + " self.metapmf.Set(ete.pmf_y, prob)\n", + "\n", + " self.mixture = thinkbayes.MakeMixture(self.metapmf)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`wtc` is the `WaitTimeCalculator` that contains\n", + "the distribution of `zb`. `are` is the\n", + "`ArrivalTimeEstimator` that contains the distribution of\n", + "`lam`.\n", + "\n", + "The first line makes a meta-Pmf that maps from each possible\n", + "distribution of `y` to its probability. For each value of\n", + "`lam`, we use `ElapsedTimeEstimator` to compute\n", + "the corresponding distribution of `y` and store it in the\n", + "Meta-Pmf. Then we use `MakeMixture` to compute the mixture.\n", + "\n", + "Figure [fig.redline4] shows the results. The shaded lines in the\n", + "background are the distributions of `y` for each value of\n", + "`lam`, with line thickness that represents likelihood. The\n", + "dark line is the mixture of these distributions.\n", + "\n", + "In this case we could get a very similar result using a single point\n", + "estimate of `lam`. So it was not necessary, for practical\n", + "purposes, to include the uncertainty of the estimate.\n", + "\n", + "In general, it is important to include variability if the system\n", + "response is non-linear; that is, if small changes in the input can cause\n", + "big changes in the output. In this case, posterior variability in\n", + "`lam` is small and the system response is approximately\n", + "linear for small perturbations." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Decision analysis \n", + "\n", + "![Probability that wait time exceeds 15 minutes as a function of the\n", + "number of passengers on the platform. ](figs/redline5.pdf)\n", + "\n", + "[fig.redline5]\n", + "\n", + "At this point we can use the number of passengers on the platform to\n", + "predict the distribution of wait times. Now let’s get to the second part\n", + "of the question: when should I stop waiting for the train and go catch a\n", + "taxi?\n", + "\n", + "Remember that in the original scenario, I am trying to get to South\n", + "Station to catch the commuter rail. Suppose I leave the office with\n", + "enough time that I can wait 15 minutes and still make my connection at\n", + "South Station.\n", + "\n", + "In that case I would like to know the probability that `y`\n", + "exceeds 15 minutes as a function of `num_passengers`. It is easy enough\n", + "to use the analysis from Section [elapsed] and run it for a range of\n", + "`num_passengers`.\n", + "\n", + "But there’s a problem. The analysis is sensitive to the frequency of\n", + "long delays, and because long delays are rare, it is hard estimate their\n", + "frequency.\n", + "\n", + "I only have data from one week, and the longest delay I observed was 15\n", + "minutes. So I can’t estimate the frequency of longer delays accurately.\n", + "\n", + "However, I can use previous observations to make at least a coarse\n", + "estimate. When I commuted by Red Line for a year, I saw three long\n", + "delays caused by a signaling problem, a power outage, and “police\n", + "activity” at another stop. So I estimate that there are about 3 major\n", + "delays per year.\n", + "\n", + "But remember that my observations are biased. I am more likely to\n", + "observe long delays because they affect a large number of passengers. So\n", + "we should treat my observations as a sample of `zb` rather\n", + "than `z`. Here’s how we can do that.\n", + "\n", + "During my year of commuting, I took the Red Line home about 220 times.\n", + "So I take the observed gap times, `gap_times`, generate a sample of 220\n", + "gaps, and compute their Pmf:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import thinkbayes\n", + "\n", + "n = 220\n", + "cdf_z = thinkbayes.MakeCdfFromList(gap_times)\n", + "sample_z = cdf_z.Sample(n)\n", + "pmf_z = thinkbayes.MakePmfFromList(sample_z)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next I bias `pmf_z` to get the distribution of `zb`, draw a\n", + "sample, and then add in delays of 30, 40, and 50 minutes (expressed in\n", + "seconds):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "cdf_zp = BiasPmf(pmf_z).MakeCdf()\n", + "sample_zb = cdf_zp.Sample(n) + [1800, 2400, 3000]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Cdf.Sample` is more efficient than `Pmf.Sample`,\n", + "so it is usually faster to convert a Pmf to a Cdf before sampling.\n", + "\n", + "Next I use the sample of `zb` to estimate a Pdf using KDE,\n", + "and then convert the Pdf to a Pmf:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "pdf_zb = thinkbayes.EstimatedPdf(sample_zb)\n", + "xs = MakeRange(low=60)\n", + "pmf_zb = pdf_zb.MakePmf(xs)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally I unbias the distribution of `zb` to get the\n", + "distribution of `z`, which I use to create the\n", + "`WaitTimeCalculator`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "pmf_z = UnbiasPmf(pmf_zb)\n", + "wtc = WaitTimeCalculator(pmf_z)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This process is complicated, but all of the steps are operations we have\n", + "seen before. Now we are ready to compute the probability of a long wait." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def ProbLongWait(num_passengers, minutes):\n", + " ete = ElapsedTimeEstimator(wtc, lam, num_passengers)\n", + " cdf_y = ete.pmf_y.MakeCdf()\n", + " prob = 1 - cdf_y.Prob(minutes * 60)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given the number of passengers on the platform,\n", + "`ProbLongWait` makes an `ElapsedTimeEstimator`,\n", + "extracts the distribution of wait time, and computes the probability\n", + "that wait time exceeds `minutes`.\n", + "\n", + "Figure [fig.redline5] shows the result. When the number of passengers is\n", + "less than 20, we infer that the system is operating normally, so the\n", + "probability of a long delay is small. If there are 30 passengers, we\n", + "estimate that it has been 15 minutes since the last train; that’s longer\n", + "than a normal delay, so we infer that something is wrong and expect\n", + "longer delays.\n", + "\n", + "If we are willing to accept a 10% chance of missing the connection at\n", + "South Station, we should stay and wait as long as there are fewer than\n", + "30 passengers, and take a taxi if there are more.\n", + "\n", + "Or, to take this analysis one step further, we could quantify the cost\n", + "of missing the connection and the cost of taking a taxi, then choose the\n", + "threshold that minimizes expected cost." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "The analysis so far has been based on the assumption that the arrival\n", + "rate of passengers is the same every day. For a commuter train during\n", + "rush hour, that might not be a bad assumption, but there are some\n", + "obvious exceptions. For example, if there is a special event nearby, a\n", + "large number of people might arrive at the same time. In that case, the\n", + "estimate of `lam` would be too low, so the estimates of\n", + "`x` and `y` would be too high.\n", + "\n", + "If special events are as common as major delays, it would be important\n", + "to include them in the model. We could do that by extending the\n", + "distribution of `lam` to include occasional large values.\n", + "\n", + "We started with the assumption that we know distribution of\n", + "`z`. As an alternative, a passenger could estimate\n", + "`z`, but it would not be easy. As a passenger, you only\n", + "observe only your own wait time, `y`. Unless you skip the\n", + "first train and wait for the second, you don’t observe the gap between\n", + "trains, `z`.\n", + "\n", + "However, we could make some inferences about `zb`. If we note\n", + "the number of passengers waiting when we arrive, we can estimate the\n", + "elapsed time since the last train, `x`. Then we observe\n", + "`y`. If we add the posterior distribution of `x`\n", + "to the observed `y`, we get a distribution that represents\n", + "our posterior belief about the observed value of `zb`.\n", + "\n", + "We can use this distribution to update our beliefs about the\n", + "distribution of `zb`. Finally, we can compute the inverse of\n", + "`BiasPmf` to get from the distribution of `zb` to\n", + "the distribution of `z`.\n", + "\n", + "I leave this analysis as an exercise for the reader. One suggestion: you\n", + "should read Chapter [species] first. You can find the outline of a\n", + "solution in . For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "\n", + "This exercise is from MacKay, *Information Theory, Inference, and\n", + "Learning Algorithms*:\n", + "\n", + "> Unstable particles are emitted from a source and decay at a distance\n", + "> $x$, a real number that has an exponential probability distribution\n", + "> with [parameter] $\\lambda$. Decay events can only be observed if they\n", + "> occur in a window extending from $x=1$ cm to $x=20$ cm. $N$ decays are\n", + "> observed at locations $\\{ 1.5, 2, 3, 4, 5, 12 \\}$ cm. What is the\n", + "> posterior distribution of $\\lambda$?\n", + "\n", + "You can download a solution to this exercise from\n", + "." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/09_Two_Dimensions.ipynb b/notebooks/09_Two_Dimensions.ipynb index 4f1e29e..55e6443 100644 --- a/notebooks/09_Two_Dimensions.ipynb +++ b/notebooks/09_Two_Dimensions.ipynb @@ -1,468 +1,942 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "Two Dimensions {#paintball}\n", - "==============\n", - "\n", - "Paintball\n", - "---------\n", - "\n", - "Paintball is a sport in which competing teams try to shoot each other\n", - "with guns that fire paint-filled pellets that break on impact, leaving a\n", - "colorful mark on the target. It is usually played in an arena decorated\n", - "with barriers and other objects that can be used as cover.\n", - "\n", - "Suppose you are playing paintball in an indoor arena 30 feet wide and 50\n", - "feet long. You are standing near one of the 30 foot walls, and you\n", - "suspect that one of your opponents has taken cover nearby. Along the\n", - "wall, you see several paint spatters, all the same color, that you think\n", - "your opponent fired recently.\n", - "\n", - "The spatters are at 15, 16, 18, and 21 feet, measured from the\n", - "lower-left corner of the room. Based on these data, where do you think\n", - "your opponent is hiding?\n", - "\n", - "Figure [fig.paintball] shows a diagram of the arena. Using the\n", - "lower-left corner of the room as the origin, I denote the unknown\n", - "location of the shooter with coordinates $\\alpha$ and $\\beta$, or\n", - "alpha and beta. The location of a spatter is\n", - "labeled x. The angle the opponent shoots at is $\\theta$ or\n", - "theta.\n", - "\n", - "The Paintball problem is a modified version of the Lighthouse problem, a\n", - "common example of Bayesian analysis. My notation follows the\n", - "presentation of the problem in D.S. Sivia’s, *Data Analysis: a\n", - "Bayesian Tutorial, Second Edition* (Oxford, 2006).\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "The suite\n", - "---------\n", - "\n", - "![Diagram of the layout for the paintball problem.](figs/paintball.pdf)\n", - "\n", - "[fig.paintball]\n", - "\n", - "To get started, we need a Suite that represents a set of hypotheses\n", - "about the location of the opponent. Each hypothesis is a pair of\n", - "coordinates: (alpha, beta).\n", - "\n", - "Here is the definition of the Paintball suite:\n", - "\n", - " class Paintball(thinkbayes.Suite, thinkbayes.Joint):\n", - "\n", - " def __init__(self, alphas, betas, locations):\n", - " self.locations = locations\n", - " pairs = [(alpha, beta) \n", - " for alpha in alphas \n", - " for beta in betas]\n", - " thinkbayes.Suite.__init__(self, pairs)\n", - "\n", - "Paintball inherits from Suite, which we have\n", - "seen before, and Joint, which I will explain soon.\n", - "\n", - "alphas is the list of possible values for\n", - "alpha; betas is the list of values for\n", - "beta. pairs is a list of all (alpha,\n", - "beta) pairs.\n", - "\n", - "locations is a list of possible locations along the wall;\n", - "it is stored for use in Likelihood.\n", - "\n", - "![Posterior CDFs for alpha and beta, given the\n", - "data.](figs/paintball2.pdf)\n", - "\n", - "[fig.paintball2]\n", - "\n", - "The room is 30 feet wide and 50 feet long, so here’s the code that\n", - "creates the suite:\n", - "\n", - " alphas = range(0, 31)\n", - " betas = range(1, 51)\n", - " locations = range(0, 31)\n", - "\n", - " suite = Paintball(alphas, betas, locations)\n", - "\n", - "This prior distribution assumes that all locations in the room are\n", - "equally likely. Given a map of the room, we might choose a more detailed\n", - "prior, but we’ll start simple.\n", - "\n", - "Trigonometry\n", - "------------\n", - "\n", - "Now we need a likelihood function, which means we have to figure out the\n", - "likelihood of hitting any spot along the wall, given the location of the\n", - "opponent.\n", - "\n", - "As a simple model, imagine that the opponent is like a rotating turret,\n", - "equally likely to shoot in any direction. In that case, he is most\n", - "likely to hit the wall at location alpha, and less likely\n", - "to hit the wall far away from alpha.\n", - "\n", - "With a little trigonometry, we can compute the probability of hitting\n", - "any spot along the wall. Imagine that the shooter fires a shot at angle\n", - "$\\theta$; the pellet would hit the wall at location $x$, where\n", - "$$x - \\alpha = \\beta \\tan \\theta$$ Solving this equation for $\\theta$\n", - "yields $$\\theta = tan^{-1} \\left( \\frac{x - \\alpha}{\\beta} \\right)$$ So\n", - "given a location on the wall, we can find $\\theta$.\n", - "\n", - "Taking the derivative of the first equation with respect to $\\theta$\n", - "yields $$\\frac{dx}{d\\theta} = \\frac{\\beta}{\\cos^2 \\theta}$$ This\n", - "derivative is what I’ll call the “strafing speed”, which is the speed of\n", - "the target location along the wall as $\\theta$ increases. The\n", - "probability of hitting a given point on the wall is inversely related to\n", - "strafing speed.\n", - "\n", - "If we know the coordinates of the shooter and a location along the wall,\n", - "we can compute strafing speed:\n", - "\n", - " def StrafingSpeed(alpha, beta, x):\n", - " theta = math.atan2(x - alpha, beta)\n", - " speed = beta / math.cos(theta)**2\n", - " return speed\n", - "\n", - "alpha and beta are the coordinates of the\n", - "shooter; x is the location of a spatter. The result is the\n", - "derivative of x with respect to theta.\n", - "\n", - "![PMF of location given alpha=10, for several values of\n", - "beta.](figs/paintball1.pdf)\n", - "\n", - "[fig.paintball1]\n", - "\n", - "Now we can compute a Pmf that represents the probability of hitting any\n", - "location on the wall. MakeLocationPmf takes\n", - "alpha and beta, the coordinates of the\n", - "shooter, and locations, a list of possible values of\n", - "x.\n", - "\n", - " def MakeLocationPmf(alpha, beta, locations):\n", - " pmf = thinkbayes.Pmf()\n", - " for x in locations:\n", - " prob = 1.0 / StrafingSpeed(alpha, beta, x)\n", - " pmf.Set(x, prob)\n", - " pmf.Normalize()\n", - " return pmf\n", - "\n", - "MakeLocationPmf computes the probability of hitting each\n", - "location, which is inversely related to strafing speed. The result is a\n", - "Pmf of locations and their probabilities.\n", - "\n", - "Figure [fig.paintball1] shows the Pmf of location with alpha =\n", - "10 and a range of values for beta. For all values of\n", - "beta the most likely spatter location is x = 10; as\n", - "beta increases, so does the spread of the Pmf.\n", - "\n", - "Likelihood\n", - "----------\n", - "\n", - "Now all we need is a likelihood function. We can use\n", - "MakeLocationPmf to compute the likelihood of any value of\n", - "x, given the coordinates of the opponent.\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " alpha, beta = hypo\n", - " x = data\n", - " pmf = MakeLocationPmf(alpha, beta, self.locations)\n", - " like = pmf.Prob(x)\n", - " return like\n", - "\n", - "Again, alpha and beta are the hypothetical\n", - "coordinates of the shooter, and x is the location of an\n", - "observed spatter.\n", - "\n", - "pmf contains the probability of each location, given the\n", - "coordinates of the shooter. From this Pmf, we select the probability of\n", - "the observed location.\n", - "\n", - "And we’re done. To update the suite, we can use UpdateSet,\n", - "which is inherited from Suite.\n", - "\n", - " suite.UpdateSet([15, 16, 18, 21])\n", - "\n", - "The result is a distribution that maps each (alpha, beta)\n", - "pair to a posterior probability.\n", - "\n", - "Joint distributions\n", - "-------------------\n", - "\n", - "When each value in a distribution is a tuple of variables, it is called\n", - "a **joint distribution** because it represents the\n", - "distributions of the variables together, that is “jointly”. A joint\n", - "distribution contains the distributions of the variables, as well\n", - "information about the relationships among them.\n", - "\n", - "Given a joint distribution, we can compute the distributions of each\n", - "variable independently, which are called the **marginal\n", - "distributions**.\n", - "\n", - "thinkbayes.Joint provides a method that computes marginal\n", - "distributions:\n", - "\n", - " # class Joint:\n", - "\n", - " def Marginal(self, i):\n", - " pmf = Pmf()\n", - " for vs, prob in self.Items():\n", - " pmf.Incr(vs[i], prob)\n", - " return pmf\n", - "\n", - "i is the index of the variable we want; in this example\n", - "i=0 indicates the distribution of alpha, and\n", - "i=1 indicates the distribution of beta.\n", - "\n", - "Here’s the code that extracts the marginal distributions:\n", - "\n", - " marginal_alpha = suite.Marginal(0)\n", - " marginal_beta = suite.Marginal(1)\n", - "\n", - "Figure [fig.paintball2] shows the results (converted to CDFs). The\n", - "median value for alpha is 18, near the center of mass of\n", - "the observed spatters. For beta, the most likely values are\n", - "close to the wall, but beyond 10 feet the distribution is almost\n", - "uniform, which indicates that the data do not distinguish strongly\n", - "between these possible locations.\n", - "\n", - "Given the posterior marginals, we can compute credible intervals for\n", - "each coordinate independently:\n", - "\n", - " print 'alpha CI', marginal_alpha.CredibleInterval(50)\n", - " print 'beta CI', marginal_beta.CredibleInterval(50)\n", - "\n", - "The 50% credible intervals are (14, 21) for\n", - "alpha and (5, 31) for beta. So\n", - "the data provide evidence that the shooter is in the near side of the\n", - "room. But it is not strong evidence. The 90% credible intervals cover\n", - "most of the room!\n", - "\n", - "Conditional distributions {#conditional}\n", - "-------------------------\n", - "\n", - "![Posterior distributions for alpha conditioned on several\n", - "values of beta.](figs/paintball3.pdf)\n", - "\n", - "[fig.paintball3]\n", - "\n", - "The marginal distributions contain information about the variables\n", - "independently, but they do not capture the dependence between variables,\n", - "if any.\n", - "\n", - "One way to visualize dependence is by computing **conditional\n", - "distributions**. thinkbayes.Joint provides a method\n", - "that does that:\n", - "\n", - " def Conditional(self, i, j, val):\n", - " pmf = Pmf()\n", - " for vs, prob in self.Items():\n", - " if vs[j] != val: continue\n", - " pmf.Incr(vs[i], prob)\n", - "\n", - " pmf.Normalize()\n", - " return pmf\n", - "\n", - "Again, i is the index of the variable we want;\n", - "j is the index of the conditioning variable, and\n", - "val is the conditional value.\n", - "\n", - "The result is the distribution of the $i$th variable under the condition\n", - "that the $j$th variable is val.\n", - "\n", - "For example, the following code computes the conditional distributions\n", - "of alpha for a range of values of beta:\n", - "\n", - " betas = [10, 20, 40]\n", - "\n", - " for beta in betas:\n", - " cond = suite.Conditional(0, 1, beta)\n", - "\n", - "Figure [fig.paintball3] shows the results, which we could fully describe\n", - "as “posterior conditional marginal distributions.” Whew!\n", - "\n", - "If the variables were independent, the conditional distributions would\n", - "all be the same. Since they are all different, we can tell the variables\n", - "are dependent. For example, if we know (somehow) that beta =\n", - "10, the conditional distribution of alpha is fairly\n", - "narrow. For larger values of beta, the distribution of\n", - "alpha is wider.\n", - "\n", - "Credible intervals {#credible-intervals}\n", - "------------------\n", - "\n", - "![Credible intervals for the coordinates of the\n", - "opponent.](figs/paintball5.pdf)\n", - "\n", - "[fig.paintball5]\n", - "\n", - "Another way to visualize the posterior joint distribution is to compute\n", - "credible intervals. When we looked at credible intervals in\n", - "Section [credible], I skipped over a subtle point: for a given\n", - "distribution, there are many intervals with the same level of\n", - "credibility. For example, if you want a 50% credible interval, you could\n", - "choose any set of values whose probability adds up to 50%.\n", - "\n", - "When the values are one-dimensional, it is most common to choose the\n", - "**central credible interval**; for example, the central 50%\n", - "credible interval contains all values between the 25th and 75th\n", - "percentiles.\n", - "\n", - "In multiple dimensions it is less obvious what the right credible\n", - "interval should be. The best choice might depend on context, but one\n", - "common choice is the maximum likelihood credible interval, which\n", - "contains the most likely values that add up to 50% (or some other\n", - "percentage).\n", - "\n", - "thinkbayes.Joint provides a method that computes maximum\n", - "likelihood credible intervals.\n", - "\n", - " # class Joint:\n", - "\n", - " def MaxLikeInterval(self, percentage=90):\n", - " interval = []\n", - " total = 0\n", - "\n", - " t = [(prob, val) for val, prob in self.Items()]\n", - " t.sort(reverse=True)\n", - "\n", - " for prob, val in t:\n", - " interval.append(val)\n", - " total += prob\n", - " if total >= percentage/100.0:\n", - " break\n", - "\n", - " return interval\n", - "\n", - "The first step is to make a list of the values in the suite, sorted in\n", - "descending order by probability. Next we traverse the list, adding each\n", - "value to the interval, until the total probability exceeds\n", - "percentage. The result is a list of values from the suite.\n", - "Notice that this set of values is not necessarily contiguous.\n", - "\n", - "To visualize the intervals, I wrote a function that “colors” each value\n", - "according to how many intervals it appears in:\n", - "\n", - " def MakeCrediblePlot(suite):\n", - " d = dict((pair, 0) for pair in suite.Values())\n", - "\n", - " percentages = [75, 50, 25]\n", - " for p in percentages:\n", - " interval = suite.MaxLikeInterval(p)\n", - " for pair in interval:\n", - " d[pair] += 1\n", - "\n", - " return d\n", - "\n", - "d is a dictionary that maps from each value in the suite to\n", - "the number of intervals it appears in. The loop computes intervals for\n", - "several percentages and modifies d.\n", - "\n", - "Figure [fig.paintball5] shows the result. The 25% credible interval is\n", - "the darkest region near the bottom wall. For higher percentages, the\n", - "credible interval is bigger, of course, and skewed toward the right side\n", - "of the room.\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "This chapter shows that the Bayesian framework from the previous\n", - "chapters can be extended to handle a two-dimensional parameter space.\n", - "The only difference is that each hypothesis is represented by a tuple of\n", - "parameters.\n", - "\n", - "I also presented Joint, which is a parent class that\n", - "provides methods that apply to joint distributions:\n", - "Marginal, Conditional, and\n", - "MakeLikeInterval. In object-oriented terms,\n", - "Joint is a mixin (see\n", - ").\n", - "\n", - "There is a lot of new vocabulary in this chapter, so let’s review:\n", - "\n", - "Joint distribution:\n", - ": A distribution that represents all possible values in a\n", - " multidimensional space and their probabilities. The example in this\n", - " chapter is a two-dimensional space made up of the coordinates\n", - " alpha and beta. The joint distribution\n", - " represents the probability of each (alpha,\n", - " beta) pair.\n", - "\n", - "Marginal distribution:\n", - ": The distribution of one parameter in a joint distribution, treating\n", - " the other parameters as unknown. For example,\n", - " Figure [fig.paintball2] shows the distributions of\n", - " alpha and beta independently.\n", - "\n", - "Conditional distribution:\n", - ": The distribution of one parameter in a joint distribution,\n", - " conditioned on one or more of the other parameters.\n", - " Figure [fig.paintball3] several distributions for\n", - " alpha, conditioned on different values of\n", - " beta.\n", - "\n", - "Given the joint distribution, you can compute marginal and conditional\n", - "distributions. With enough conditional distributions, you could\n", - "re-create the joint distribution, at least approximately. But given the\n", - "marginal distributions you cannot re-create the joint distribution\n", - "because you have lost information about the dependence between\n", - "variables.\n", - "\n", - "If there are $n$ possible values for each of two parameters, most\n", - "operations on the joint distribution take time proportional to $n^2$. If\n", - "there are $d$ parameters, run time is proportional to $n^d$, which\n", - "quickly becomes impractical as the number of dimensions increases.\n", - "\n", - "If you can process a million hypotheses in a reasonable amount of time,\n", - "you could handle two dimensions with 1000 values for each parameter, or\n", - "three dimensions with 100 values each, or six dimensions with 10 values\n", - "each.\n", - "\n", - "If you need more dimensions, or more values per dimension, there are\n", - "optimizations you can try. I present an example in Chapter [species].\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "In our simple model, the opponent is equally likely to shoot in any\n", - "direction. As an exercise, let’s consider improvements to this model.\n", - "\n", - "The analysis in this chapter suggests that a shooter is most likely to\n", - "hit the closest wall. But in reality, if the opponent is close to a\n", - "wall, he is unlikely to shoot at the wall because he is unlikely to see\n", - "a target between himself and the wall.\n", - "\n", - "Design an improved model that takes this behavior into account. Try to\n", - "find a model that is more realistic, but not too complicated.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Two Dimensions\n", + "\n", + "\n", + "## Paintball\n", + "\n", + "Paintball is a sport in which competing teams try to shoot each other\n", + "with guns that fire paint-filled pellets that break on impact, leaving a\n", + "colorful mark on the target. It is usually played in an arena decorated\n", + "with barriers and other objects that can be used as cover.\n", + "\n", + "Suppose you are playing paintball in an indoor arena 30 feet wide and 50\n", + "feet long. You are standing near one of the 30 foot walls, and you\n", + "suspect that one of your opponents has taken cover nearby. Along the\n", + "wall, you see several paint spatters, all the same color, that you think\n", + "your opponent fired recently.\n", + "\n", + "The spatters are at 15, 16, 18, and 21 feet, measured from the\n", + "lower-left corner of the room. Based on these data, where do you think\n", + "your opponent is hiding?\n", + "\n", + "Figure [fig.paintball] shows a diagram of the arena. Using the\n", + "lower-left corner of the room as the origin, I denote the unknown\n", + "location of the shooter with coordinates $\\alpha$ and $\\beta$, or\n", + "`alpha` and `beta`. The location of a spatter is\n", + "labeled `x`. The angle the opponent shoots at is $\\theta$ or\n", + "`theta`.\n", + "\n", + "The Paintball problem is a modified version of the Lighthouse problem, a\n", + "common example of Bayesian analysis. My notation follows the\n", + "presentation of the problem in D.S. Sivia’s, *Data Analysis: a\n", + "Bayesian Tutorial, Second Edition* (Oxford, 2006).\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The suite\n", + "\n", + "![Diagram of the layout for the paintball problem.](figs/paintball.pdf)\n", + "\n", + "[fig.paintball]\n", + "\n", + "To get started, we need a Suite that represents a set of hypotheses\n", + "about the location of the opponent. Each hypothesis is a pair of\n", + "coordinates: `(alpha, beta)`.\n", + "\n", + "Here is the definition of the Paintball suite:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import thinkbayes\n", + "\n", + "class Paintball(thinkbayes.Suite, thinkbayes.Joint):\n", + "\n", + " def __init__(self, alphas, betas, locations):\n", + " self.locations = locations\n", + " pairs = [(alpha, beta) \n", + " for alpha in alphas \n", + " for beta in betas]\n", + " thinkbayes.Suite.__init__(self, pairs)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Paintball` inherits from `Suite`, which we have\n", + "seen before, and `Joint`, which I will explain soon.\n", + "\n", + "`alphas` is the list of possible values for\n", + "`alpha`; `betas` is the list of values for\n", + "`beta`. `pairs` is a list of all `(alpha,\n", + "beta)` pairs.\n", + "\n", + "`locations` is a list of possible locations along the wall;\n", + "it is stored for use in `Likelihood`.\n", + "\n", + "![Posterior CDFs for `alpha` and `beta`, given the\n", + "data.](figs/paintball2.pdf)\n", + "\n", + "[fig.paintball2]\n", + "\n", + "The room is 30 feet wide and 50 feet long, so here’s the code that\n", + "creates the suite:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "alphas = range(0, 31)\n", + "betas = range(1, 51)\n", + "locations = range(0, 31)\n", + "\n", + "suite = Paintball(alphas, betas, locations)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This prior distribution assumes that all locations in the room are\n", + "equally likely. Given a map of the room, we might choose a more detailed\n", + "prior, but we’ll start simple." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Trigonometry\n", + "\n", + "Now we need a likelihood function, which means we have to figure out the\n", + "likelihood of hitting any spot along the wall, given the location of the\n", + "opponent.\n", + "\n", + "As a simple model, imagine that the opponent is like a rotating turret,\n", + "equally likely to shoot in any direction. In that case, he is most\n", + "likely to hit the wall at location `alpha`, and less likely\n", + "to hit the wall far away from `alpha`.\n", + "\n", + "With a little trigonometry, we can compute the probability of hitting\n", + "any spot along the wall. Imagine that the shooter fires a shot at angle\n", + "$\\theta$; the pellet would hit the wall at location $x$, where\n", + "\n", + "$$x - \\alpha = \\beta \\tan \\theta$$ Solving this equation for $\\theta$\n", + "yields $$\\theta = tan^{-1} \\left( \\frac{x - \\alpha}{\\beta} \\right)$$ \n", + "\n", + "So given a location on the wall, we can find $\\theta$.\n", + "\n", + "Taking the derivative of the first equation with respect to $\\theta$\n", + "yields\n", + "\n", + "$$\\frac{dx}{d\\theta} = \\frac{\\beta}{\\cos^2 \\theta}$$ \n", + "\n", + "This derivative is what I’ll call the “strafing speed”, which is the speed of\n", + "the target location along the wall as $\\theta$ increases. The\n", + "probability of hitting a given point on the wall is inversely related to\n", + "strafing speed.\n", + "\n", + "If we know the coordinates of the shooter and a location along the wall,\n", + "we can compute strafing speed:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def StrafingSpeed(alpha, beta, x):\n", + " theta = math.atan2(x - alpha, beta)\n", + " speed = beta / math.cos(theta)**2\n", + " return speed" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`alpha` and `beta` are the coordinates of the\n", + "shooter; `x` is the location of a spatter. The result is the\n", + "derivative of `x` with respect to `theta`.\n", + "\n", + "![PMF of location given `alpha=10`, for several values of\n", + "`beta`.](figs/paintball1.pdf)\n", + "\n", + "[fig.paintball1]\n", + "\n", + "Now we can compute a Pmf that represents the probability of hitting any\n", + "location on the wall. `MakeLocationPmf` takes\n", + "`alpha` and `beta`, the coordinates of the\n", + "shooter, and `locations`, a list of possible values of\n", + "`x`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeLocationPmf(alpha, beta, locations):\n", + " pmf = thinkbayes.Pmf()\n", + " for x in locations:\n", + " prob = 1.0 / StrafingSpeed(alpha, beta, x)\n", + " pmf.Set(x, prob)\n", + " pmf.Normalize()\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`MakeLocationPmf` computes the probability of hitting each\n", + "location, which is inversely related to strafing speed. The result is a\n", + "Pmf of locations and their probabilities.\n", + "\n", + "Figure [fig.paintball1] shows the Pmf of location with `alpha =\n", + "10` and a range of values for `beta`. For all values of\n", + "beta the most likely spatter location is `x = 10`; as\n", + "`beta` increases, so does the spread of the Pmf." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Likelihood\n", + "\n", + "Now all we need is a likelihood function. We can use\n", + "`MakeLocationPmf` to compute the likelihood of any value of\n", + "`x`, given the coordinates of the opponent." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def Likelihood(self, data, hypo):\n", + " alpha, beta = hypo\n", + " x = data\n", + " pmf = MakeLocationPmf(alpha, beta, self.locations)\n", + " like = pmf.Prob(x)\n", + " return like" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, `alpha` and `beta` are the hypothetical\n", + "coordinates of the shooter, and `x` is the location of an\n", + "observed spatter.\n", + "\n", + "`pmf` contains the probability of each location, given the\n", + "coordinates of the shooter. From this Pmf, we select the probability of\n", + "the observed location.\n", + "\n", + "And we’re done. To update the suite, we can use `UpdateSet`,\n", + "which is inherited from `Suite`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "ename": "UnimplementedMethodException", + "evalue": "", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mUnimplementedMethodException\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mUpdateSet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m15\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m16\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m18\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m21\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;32mC:\\Users\\rlabbe\\Dropbox\\ThinkBayes\\code\\thinkbayes.py\u001b[0m in \u001b[0;36mUpdateSet\u001b[1;34m(self, dataset)\u001b[0m\n\u001b[0;32m 1195\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mdata\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mdataset\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1196\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mhypo\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mValues\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1197\u001b[1;33m \u001b[0mlike\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mLikelihood\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhypo\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1198\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMult\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mhypo\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlike\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1199\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mNormalize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mC:\\Users\\rlabbe\\Dropbox\\ThinkBayes\\code\\thinkbayes.py\u001b[0m in \u001b[0;36mLikelihood\u001b[1;34m(self, data, hypo)\u001b[0m\n\u001b[0;32m 1218\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0msome\u001b[0m \u001b[0mrepresentation\u001b[0m \u001b[0mof\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1219\u001b[0m \"\"\"\n\u001b[1;32m-> 1220\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mUnimplementedMethodException\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1221\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1222\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mLogLikelihood\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhypo\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mUnimplementedMethodException\u001b[0m: " + ] + } + ], + "source": [ + "suite.UpdateSet([15, 16, 18, 21])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The result is a distribution that maps each `(alpha, beta)`\n", + "pair to a posterior probability." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Joint distributions\n", + "\n", + "When each value in a distribution is a tuple of variables, it is called\n", + "a **joint distribution** because it represents the\n", + "distributions of the variables together, that is “jointly”. A joint\n", + "distribution contains the distributions of the variables, as well\n", + "information about the relationships among them.\n", + "\n", + "Given a joint distribution, we can compute the distributions of each\n", + "variable independently, which are called the **marginal\n", + "distributions**.\n", + "\n", + "`thinkbayes.Joint` provides a method that computes marginal\n", + "distributions:\n", + "\n", + "```python\n", + "# class Joint:\n", + " def Marginal(self, i):\n", + " pmf = Pmf()\n", + " for vs, prob in self.Items():\n", + " pmf.Incr(vs[i], prob)\n", + " return pmf\n", + "```\n", + "\n", + "`i` is the index of the variable we want; in this example\n", + "`i=0` indicates the distribution of `alpha`, and\n", + "`i=1` indicates the distribution of `beta`.\n", + "\n", + "Here’s the code that extracts the marginal distributions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "marginal_alpha = suite.Marginal(0)\n", + "marginal_beta = suite.Marginal(1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure [fig.paintball2] shows the results (converted to CDFs). The\n", + "median value for `alpha` is 18, near the center of mass of\n", + "the observed spatters. For `beta`, the most likely values are\n", + "close to the wall, but beyond 10 feet the distribution is almost\n", + "uniform, which indicates that the data do not distinguish strongly\n", + "between these possible locations.\n", + "\n", + "Given the posterior marginals, we can compute credible intervals for\n", + "each coordinate independently:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "print('alpha CI', marginal_alpha.CredibleInterval(50))\n", + "print('beta CI', marginal_beta.CredibleInterval(50))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The 50% credible intervals are `(14, 21)` for\n", + "`alpha` and `(5, 31)` for `beta`. So\n", + "the data provide evidence that the shooter is in the near side of the\n", + "room. But it is not strong evidence. The 90% credible intervals cover\n", + "most of the room!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conditional distributions {#conditional}\n", + "\n", + "![Posterior distributions for `alpha` conditioned on several\n", + "values of `beta`.](figs/paintball3.pdf)\n", + "\n", + "[fig.paintball3]\n", + "\n", + "The marginal distributions contain information about the variables\n", + "independently, but they do not capture the dependence between variables,\n", + "if any.\n", + "\n", + "One way to visualize dependence is by computing **conditional\n", + "distributions**. `thinkbayes.Joint` provides a method\n", + "that does that:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def Conditional(self, i, j, val):\n", + " pmf = Pmf()\n", + " for vs, prob in self.Items():\n", + " if vs[j] != val: continue\n", + " pmf.Incr(vs[i], prob)\n", + "\n", + " pmf.Normalize()\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, `i` is the index of the variable we want;\n", + "`j` is the index of the conditioning variable, and\n", + "`val` is the conditional value.\n", + "\n", + "The result is the distribution of the $i$th variable under the condition\n", + "that the $j$th variable is `val`.\n", + "\n", + "For example, the following code computes the conditional distributions\n", + "of `alpha` for a range of values of `beta`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "betas = [10, 20, 40]\n", + "\n", + "for beta in betas:\n", + " cond = suite.Conditional(0, 1, beta)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure [fig.paintball3] shows the results, which we could fully describe\n", + "as “posterior conditional marginal distributions.” Whew!\n", + "\n", + "If the variables were independent, the conditional distributions would\n", + "all be the same. Since they are all different, we can tell the variables\n", + "are dependent. For example, if we know (somehow) that `beta = 10`, the\n", + "conditional distribution of `alpha` is fairly\n", + "narrow. For larger values of `beta`, the distribution of\n", + "`alpha` is wider." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Credible intervals {#credible-intervals}\n", + "\n", + "![Credible intervals for the coordinates of the\n", + "opponent.](figs/paintball5.pdf)\n", + "\n", + "[fig.paintball5]\n", + "\n", + "Another way to visualize the posterior joint distribution is to compute\n", + "credible intervals. When we looked at credible intervals in\n", + "Section [credible], I skipped over a subtle point: for a given\n", + "distribution, there are many intervals with the same level of\n", + "credibility. For example, if you want a 50% credible interval, you could\n", + "choose any set of values whose probability adds up to 50%.\n", + "\n", + "When the values are one-dimensional, it is most common to choose the\n", + "**central credible interval**; for example, the central 50%\n", + "credible interval contains all values between the 25th and 75th\n", + "percentiles.\n", + "\n", + "In multiple dimensions it is less obvious what the right credible\n", + "interval should be. The best choice might depend on context, but one\n", + "common choice is the maximum likelihood credible interval, which\n", + "contains the most likely values that add up to 50% (or some other\n", + "percentage).\n", + "\n", + "`thinkbayes.Joint` provides a method that computes maximum\n", + "likelihood credible intervals." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "# class Joint:\n", + " def MaxLikeInterval(self, percentage=90):\n", + " interval = []\n", + " total = 0\n", + "\n", + " t = [(prob, val) for val, prob in self.Items()]\n", + " t.sort(reverse=True)\n", + "\n", + " for prob, val in t:\n", + " interval.append(val)\n", + " total += prob\n", + " if total >= percentage/100.0:\n", + " break\n", + "\n", + " return interval\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The first step is to make a list of the values in the suite, sorted in\n", + "descending order by probability. Next we traverse the list, adding each\n", + "value to the interval, until the total probability exceeds\n", + "`percentage`. The result is a list of values from the suite.\n", + "Notice that this set of values is not necessarily contiguous.\n", + "\n", + "To visualize the intervals, I wrote a function that “colors” each value\n", + "according to how many intervals it appears in:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeCrediblePlot(suite):\n", + " d = dict((pair, 0) for pair in suite.Values())\n", + "\n", + " percentages = [75, 50, 25]\n", + " for p in percentages:\n", + " interval = suite.MaxLikeInterval(p)\n", + " for pair in interval:\n", + " d[pair] += 1\n", + "\n", + " return d" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`d` is a dictionary that maps from each value in the suite to\n", + "the number of intervals it appears in. The loop computes intervals for\n", + "several percentages and modifies `d`.\n", + "\n", + "Figure [fig.paintball5] shows the result. The 25% credible interval is\n", + "the darkest region near the bottom wall. For higher percentages, the\n", + "credible interval is bigger, of course, and skewed toward the right side\n", + "of the room." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "This chapter shows that the Bayesian framework from the previous\n", + "chapters can be extended to handle a two-dimensional parameter space.\n", + "The only difference is that each hypothesis is represented by a tuple of\n", + "parameters.\n", + "\n", + "I also presented `Joint`, which is a parent class that\n", + "provides methods that apply to joint distributions:\n", + "`Marginal`, `Conditional`, and\n", + "`MakeLikeInterval`. In object-oriented terms,\n", + "`Joint` is a mixin (see\n", + ").\n", + "\n", + "There is a lot of new vocabulary in this chapter, so let’s review:\n", + "\n", + "Joint distribution:\n", + ": A distribution that represents all possible values in a\n", + " multidimensional space and their probabilities. The example in this\n", + " chapter is a two-dimensional space made up of the coordinates\n", + " `alpha` and `beta`. The joint distribution\n", + " represents the probability of each (`alpha`,\n", + " `beta`) pair.\n", + "\n", + "Marginal distribution:\n", + ": The distribution of one parameter in a joint distribution, treating\n", + " the other parameters as unknown. For example,\n", + " Figure [fig.paintball2] shows the distributions of\n", + " `alpha` and `beta` independently.\n", + "\n", + "Conditional distribution:\n", + ": The distribution of one parameter in a joint distribution,\n", + " conditioned on one or more of the other parameters.\n", + " Figure [fig.paintball3] several distributions for\n", + " `alpha`, conditioned on different values of\n", + " `beta`.\n", + "\n", + "Given the joint distribution, you can compute marginal and conditional\n", + "distributions. With enough conditional distributions, you could\n", + "re-create the joint distribution, at least approximately. But given the\n", + "marginal distributions you cannot re-create the joint distribution\n", + "because you have lost information about the dependence between\n", + "variables.\n", + "\n", + "If there are $n$ possible values for each of two parameters, most\n", + "operations on the joint distribution take time proportional to $n^2$. If\n", + "there are $d$ parameters, run time is proportional to $n^d$, which\n", + "quickly becomes impractical as the number of dimensions increases.\n", + "\n", + "If you can process a million hypotheses in a reasonable amount of time,\n", + "you could handle two dimensions with 1000 values for each parameter, or\n", + "three dimensions with 100 values each, or six dimensions with 10 values\n", + "each.\n", + "\n", + "If you need more dimensions, or more values per dimension, there are\n", + "optimizations you can try. I present an example in Chapter [species].\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "In our simple model, the opponent is equally likely to shoot in any\n", + "direction. As an exercise, let’s consider improvements to this model.\n", + "\n", + "The analysis in this chapter suggests that a shooter is most likely to\n", + "hit the closest wall. But in reality, if the opponent is close to a\n", + "wall, he is unlikely to shoot at the wall because he is unlikely to see\n", + "a target between himself and the wall.\n", + "\n", + "Design an improved model that takes this behavior into account. Try to\n", + "find a model that is more realistic, but not too complicated." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/10_Approximate_Bayesian_Computation.ipynb b/notebooks/10_Approximate_Bayesian_Computation.ipynb index 971b4cb..693425a 100644 --- a/notebooks/10_Approximate_Bayesian_Computation.ipynb +++ b/notebooks/10_Approximate_Bayesian_Computation.ipynb @@ -1,734 +1,1194 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "Approximate Bayesian Computation\n", - "================================\n", - "\n", - "The Variability Hypothesis\n", - "--------------------------\n", - "\n", - "I have a soft spot for crank science. Recently I visited Norumbega\n", - "Tower, which is an enduring monument to the crackpot theories of Eben\n", - "Norton Horsford, inventor of double-acting baking powder and fake\n", - "history. But that’s not what this chapter is about.\n", - "\n", - "This chapter is about the Variability Hypothesis, which\n", - "\n", - "> “originated in the early nineteenth century with Johann Meckel, who\n", - "> argued that males have a greater range of ability than females,\n", - "> especially in intelligence. In other words, he believed that most\n", - "> geniuses and most mentally retarded people are men. Because he\n", - "> considered males to be the ’superior animal,’ Meckel concluded that\n", - "> females’ lack of variation was a sign of inferiority.”\n", - ">\n", - "> From .\n", - "\n", - "I particularly like that last part, because I suspect that if it turns\n", - "out that women are actually more variable, Meckel would take that as a\n", - "sign of inferiority, too. Anyway, you will not be surprised to hear that\n", - "the evidence for the Variability Hypothesis is weak.\n", - "\n", - "Nevertheless, it came up in my class recently when we looked at data\n", - "from the CDC’s Behavioral Risk Factor Surveillance System (BRFSS),\n", - "specifically the self-reported heights of adult American men and women.\n", - "The dataset includes responses from 154407 men and 254722 women. Here’s\n", - "what we found:\n", - "\n", - "- The average height for men is 178 cm; the average height for women\n", - " is 163 cm. So men are taller, on average. No surprise there.\n", - "\n", - "- For men the standard deviation is 7.7 cm; for women it is 7.3 cm. So\n", - " in absolute terms, men’s heights are more variable.\n", - "\n", - "- But to compare variability between groups, it is more meaningful to\n", - " use the coefficient of variation (CV), which is the standard\n", - " deviation divided by the mean. It is a dimensionless measure of\n", - " variability relative to scale. For men CV is 0.0433; for women it is\n", - " 0.0444.\n", - "\n", - "That’s very close, so we could conclude that this dataset provides weak\n", - "evidence against the Variability Hypothesis. But we can use Bayesian\n", - "methods to make that conclusion more precise. And answering this\n", - "question gives me a chance to demonstrate some techniques for working\n", - "with large datasets.\n", - "\n", - "I will proceed in a few steps:\n", - "\n", - "1. We’ll start with the simplest implementation, but it only works for\n", - " datasets smaller than 1000 values.\n", - "\n", - "2. By computing probabilities under a log transform, we can scale up to\n", - " the full size of the dataset, but the computation gets slow.\n", - "\n", - "3. Finally, we speed things up substantially with Approximate Bayesian\n", - " Computation, also known as ABC.\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Mean and standard deviation\n", - "---------------------------\n", - "\n", - "In Chapter [paintball] we estimated two parameters simultaneously using\n", - "a joint distribution. In this chapter we use the same method to estimate\n", - "the parameters of a Gaussian distribution: the mean, mu,\n", - "and the standard deviation, sigma.\n", - "\n", - "For this problem, I define a Suite called Height that\n", - "represents a map from each mu, sigma pair to its\n", - "probability:\n", - "\n", - " class Height(thinkbayes.Suite, thinkbayes.Joint):\n", - "\n", - " def __init__(self, mus, sigmas):\n", - " thinkbayes.Suite.__init__(self)\n", - "\n", - " pairs = [(mu, sigma) \n", - " for mu in mus\n", - " for sigma in sigmas]\n", - "\n", - " thinkbayes.Suite.__init__(self, pairs)\n", - "\n", - "mus is a sequence of possible values for mu;\n", - "sigmas is a sequence of values for sigma. The\n", - "prior distribution is uniform over all mu, sigma pairs.\n", - "\n", - "The likelihood function is easy. Given hypothetical values of\n", - "mu and sigma, we compute the likelihood of a\n", - "particular value, x. That’s what\n", - "EvalGaussianPdf does, so all we have to do is use it:\n", - "\n", - " # class Height\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " x = data\n", - " mu, sigma = hypo\n", - " like = thinkbayes.EvalGaussianPdf(x, mu, sigma)\n", - " return like\n", - "\n", - "If you have studied statistics from a mathematical perspective, you know\n", - "that when you evaluate a PDF, you get a probability density. In order to\n", - "get a probability, you have to integrate probability densities over some\n", - "range.\n", - "\n", - "But for our purposes, we don’t need a probability; we just need\n", - "something proportional to the probability we want. A probability density\n", - "does that job nicely.\n", - "\n", - "The hardest part of this problem turns out to be choosing appropriate\n", - "ranges for mus and sigmas. If the range is too\n", - "small, we omit some possibilities with non-negligible probability and\n", - "get the wrong answer. If the range is too big, we get the right answer,\n", - "but waste computational power.\n", - "\n", - "So this is an opportunity to use classical estimation to make Bayesian\n", - "techniques more efficient. Specifically, we can use classical estimators\n", - "to find a likely location for mu and sigma,\n", - "and use the standard errors of those estimates to choose a likely\n", - "spread.\n", - "\n", - "If the true parameters of the distribution are $\\mu$ and $\\sigma$, and\n", - "we take a sample of $n$ values, an estimator of $\\mu$ is the sample\n", - "mean, m.\n", - "\n", - "And an estimator of $\\sigma$ is the sample standard variance,\n", - "s.\n", - "\n", - "The standard error of the estimated $\\mu$ is $s / \\sqrt{n}$ and the\n", - "standard error of the estimated $\\sigma$ is $s / \\sqrt{2 (n-1)}$.\n", - "\n", - "Here’s the code to compute all that:\n", - "\n", - " def FindPriorRanges(xs, num_points, num_stderrs=3.0):\n", - "\n", - " # compute m and s\n", - " n = len(xs)\n", - " m = numpy.mean(xs)\n", - " s = numpy.std(xs)\n", - "\n", - " # compute ranges for m and s\n", - " stderr_m = s / math.sqrt(n)\n", - " mus = MakeRange(m, stderr_m, num_stderrs)\n", - "\n", - " stderr_s = s / math.sqrt(2 * (n-1))\n", - " sigmas = MakeRange(s, stderr_s, num_stderrs)\n", - "\n", - " return mus, sigmas\n", - "\n", - "xs is the dataset. `num_points` is the desired number of\n", - "values in the range. `num_stderrs` is the width of the range on each\n", - "side of the estimate, in number of standard errors.\n", - "\n", - "The return value is a pair of sequences, mus and\n", - "sigmas.\n", - "\n", - "Here’s MakeRange:\n", - "\n", - " def MakeRange(estimate, stderr, num_stderrs):\n", - " spread = stderr * num_stderrs\n", - " array = numpy.linspace(estimate-spread,\n", - " estimate+spread,\n", - " num_points)\n", - " return array\n", - "\n", - "numpy.linspace makes an array of equally spaced elements\n", - "between estimate-spread and estimate+spread,\n", - "including both.\n", - "\n", - "Update\n", - "------\n", - "\n", - "Finally here’s the code to make and update the suite:\n", - "\n", - " mus, sigmas = FindPriorRanges(xs, num_points)\n", - " suite = Height(mus, sigmas)\n", - " suite.UpdateSet(xs)\n", - " print suite.MaximumLikelihood() \n", - "\n", - "This process might seem bogus, because we use the data to choose the\n", - "range of the prior distribution, and then use the data again to do the\n", - "update. In general, using the same data twice is, in fact, bogus.\n", - "\n", - "But in this case it is ok. Really. We use the data to choose the range\n", - "for the prior, but only to avoid computing a lot of probabilities that\n", - "would have been very small anyway. With `num_stderrs=4`, the range is\n", - "big enough to cover all values with non-negligible likelihood. After\n", - "that, making it bigger has no effect on the results.\n", - "\n", - "In effect, the prior is uniform over all values of mu and\n", - "sigma, but for computational efficiency we ignore all the\n", - "values that don’t matter.\n", - "\n", - "The posterior distribution of CV\n", - "--------------------------------\n", - "\n", - "Once we have the posterior joint distribution of mu and\n", - "sigma, we can compute the distribution of CV for men and\n", - "women, and then the probability that one exceeds the other.\n", - "\n", - "To compute the distribution of CV, we enumerate pairs of mu\n", - "and sigma:\n", - "\n", - " def CoefVariation(suite):\n", - " pmf = thinkbayes.Pmf()\n", - " for (mu, sigma), p in suite.Items():\n", - " pmf.Incr(sigma/mu, p)\n", - " return pmf\n", - "\n", - "Then we use `thinkbayes.PmfProbGreater` to compute the probability that\n", - "men are more variable.\n", - "\n", - "The analysis itself is simple, but there are two more issues we have to\n", - "deal with:\n", - "\n", - "1. As the size of the dataset increases, we run into a series of\n", - " computational problems due to the limitations of floating-point\n", - " arithmetic.\n", - "\n", - "2. The dataset contains a number of extreme values that are almost\n", - " certainly errors. We will need to make the estimation process robust\n", - " in the presence of these outliers.\n", - "\n", - "The following sections explain these problems and their solutions.\n", - "\n", - "Underflow\n", - "---------\n", - "\n", - "If we select the first 100 values from the BRFSS dataset and run the\n", - "analysis I just described, it runs without errors and we get posterior\n", - "distributions that look reasonable.\n", - "\n", - "If we select the first 1000 values and run the program again, we get an\n", - "error in `Pmf.Normalize`:\n", - "\n", - " ValueError: total probability is zero.\n", - "\n", - "The problem is that we are using probability densities to compute\n", - "likelihoods, and densities from continuous distributions tend to be\n", - "small. And if you take 1000 small values and multiply them together, the\n", - "result is very small. In this case it is so small it can’t be\n", - "represented by a floating-point number, so it gets rounded down to zero,\n", - "which is called **underflow**. And if all probabilities in\n", - "the distribution are 0, it’s not a distribution any more.\n", - "\n", - "A possible solution is to renormalize the Pmf after each update, or\n", - "after each batch of 100. That would work, but it would be slow.\n", - "\n", - "A better alternative is to compute likelihoods under a log transform.\n", - "That way, instead of multiplying small values, we can add up log\n", - "likelihoods. Pmf provides methods Log,\n", - "LogUpdateSet and Exp to make this process\n", - "easy.\n", - "\n", - "Log computes the log of the probabilities in a Pmf:\n", - "\n", - " # class Pmf\n", - "\n", - " def Log(self):\n", - " m = self.MaxLike()\n", - " for x, p in self.d.iteritems():\n", - " if p:\n", - " self.Set(x, math.log(p/m))\n", - " else:\n", - " self.Remove(x)\n", - "\n", - "Before applying the log transform Log uses\n", - "MaxLike to find m, the highest probability in\n", - "the Pmf. It divide all probabilities by m, so the highest\n", - "probability gets normalized to 1, which yields a log of 0. The other log\n", - "probabilities are all negative. If there are any values in the Pmf with\n", - "probability 0, they are removed.\n", - "\n", - "While the Pmf is under a log transform, we can’t use\n", - "Update, UpdateSet, or Normalize.\n", - "The result would be nonsensical; if you try, Pmf raises an exception.\n", - "Instead, we have to use LogUpdate and\n", - "LogUpdateSet.\n", - "\n", - "Here’s the implementation of LogUpdateSet:\n", - "\n", - " # class Suite\n", - "\n", - " def LogUpdateSet(self, dataset):\n", - " for data in dataset:\n", - " self.LogUpdate(data)\n", - "\n", - "LogUpdateSet loops through the data and calls\n", - "LogUpdate:\n", - "\n", - " # class Suite\n", - "\n", - " def LogUpdate(self, data):\n", - " for hypo in self.Values():\n", - " like = self.LogLikelihood(data, hypo)\n", - " self.Incr(hypo, like)\n", - "\n", - "LogUpdate is just like Update except that it\n", - "calls LogLikelihood instead of Likelihood, and\n", - "Incr instead of Mult.\n", - "\n", - "Using log-likelihoods avoids the problem with underflow, but while the\n", - "Pmf is under the log transform, there’s not much we can do with it. We\n", - "have to use Exp to invert the transform:\n", - "\n", - " # class Pmf\n", - "\n", - " def Exp(self):\n", - " m = self.MaxLike()\n", - " for x, p in self.d.iteritems():\n", - " self.Set(x, math.exp(p-m))\n", - "\n", - "If the log-likelihoods are large negative numbers, the resulting\n", - "likelihoods might underflow. So Exp finds the maximum\n", - "log-likelihood, m, and shifts all the likelihoods up by\n", - "m. The resulting distribution has a maximum likelihood of\n", - "1. This process inverts the log transform with minimal loss of\n", - "precision.\n", - "\n", - "Log-likelihood\n", - "--------------\n", - "\n", - "Now all we need is LogLikelihood.\n", - "\n", - " # class Height\n", - "\n", - " def LogLikelihood(self, data, hypo):\n", - " x = data\n", - " mu, sigma = hypo\n", - " loglike = scipy.stats.norm.logpdf(x, mu, sigma)\n", - " return loglike\n", - "\n", - "norm.logpdf computes the log-likelihood of the Gaussian\n", - "PDF.\n", - "\n", - "Here’s what the whole update process looks like:\n", - "\n", - " suite.Log()\n", - " suite.LogUpdateSet(xs)\n", - " suite.Exp()\n", - " suite.Normalize()\n", - "\n", - "To review, Log puts the suite under a log transform.\n", - "LogUpdateSet calls LogUpdate, which calls\n", - "LogLikelihood. LogUpdate uses\n", - "Pmf.Incr, because adding a log-likelihood is the same as\n", - "multiplying by a likelihood.\n", - "\n", - "After the update, the log-likelihoods are large negative numbers, so\n", - "Exp shifts them up before inverting the transform, which is\n", - "how we avoid underflow.\n", - "\n", - "Once the suite is transformed back, the probabilities are “linear”\n", - "again, which means “not logarithmic”, so we can use\n", - "Normalize again.\n", - "\n", - "Using this algorithm, we can process the entire dataset without\n", - "underflow, but it is still slow. On my computer it might take an hour.\n", - "We can do better.\n", - "\n", - "A little optimization\n", - "---------------------\n", - "\n", - "This section uses math and computational optimization to speed things up\n", - "by a factor of 100. But the following section presents an algorithm that\n", - "is even faster. So if you want to get right to the good stuff, feel free\n", - "to skip this section.\n", - "\n", - "Suite.LogUpdateSet calls LogUpdate once for\n", - "each data point. We can speed it up by computing the log-likelihood of\n", - "the entire dataset at once.\n", - "\n", - "We’ll start with the Gaussian PDF:\n", - "$$\\frac{1}{\\sigma \\sqrt{2 \\pi}} \\exp \\left[ -\\frac{1}{2} \\left( \\frac{x-\\mu}{\\sigma} \\right)^2 \\right]$$\n", - "and compute the log (dropping the constant term):\n", - "$$-\\log \\sigma -\\frac{1}{2} \\left( \\frac{x-\\mu}{\\sigma} \\right)^2$$\n", - "Given a sequence of values, $x_i$, the total log-likelihood is\n", - "$$\\sum_i -\\log \\sigma - \\frac{1}{2} \\left( \\frac{x_i-\\mu}{\\sigma} \\right)^2$$\n", - "Pulling out the terms that don’t depend on $i$, we get\n", - "$$-n \\log \\sigma - \\frac{1}{2 \\sigma^2} \\sum_i (x_i - \\mu)^2$$ which we\n", - "can translate into Python:\n", - "\n", - " # class Height\n", - "\n", - " def LogUpdateSetFast(self, data):\n", - " xs = tuple(data)\n", - " n = len(xs)\n", - "\n", - " for hypo in self.Values():\n", - " mu, sigma = hypo\n", - " total = Summation(xs, mu)\n", - " loglike = -n * math.log(sigma) - total / 2 / sigma**2\n", - " self.Incr(hypo, loglike)\n", - "\n", - "By itself, this would be a small improvement, but it creates an\n", - "opportunity for a bigger one. Notice that the summation only depends on\n", - "mu, not sigma, so we only have to compute it\n", - "once for each value of mu.\n", - "\n", - "To avoid recomputing, I factor out a function that computes the\n", - "summation, and **memoize** it so it stores previously\n", - "computed results in a dictionary (see\n", - "):\n", - "\n", - " def Summation(xs, mu, cache={}):\n", - " try:\n", - " return cache[xs, mu]\n", - " except KeyError:\n", - " ds = [(x-mu)**2 for x in xs]\n", - " total = sum(ds)\n", - " cache[xs, mu] = total\n", - " return total\n", - "\n", - "cache stores previously computed sums. The try\n", - "statement returns a result from the cache if possible; otherwise it\n", - "computes the summation, then caches and returns the result.\n", - "\n", - "The only catch is that we can’t use a list as a key in the cache,\n", - "because it is not a hashable type. That’s why\n", - "LogUpdateSetFast converts the dataset to a tuple.\n", - "\n", - "This optimization speeds up the computation by about a factor of 100,\n", - "processing the entire dataset (154 407 men and 254 722 women) in less\n", - "than a minute on my not-very-fast computer.\n", - "\n", - "ABC\n", - "---\n", - "\n", - "But maybe you don’t have that kind of time. In that case, Approximate\n", - "Bayesian Computation (ABC) might be the way to go. The motivation behind\n", - "ABC is that the likelihood of any particular dataset is:\n", - "\n", - "1. Very small, especially for large datasets, which is why we had to\n", - " use the log transform,\n", - "\n", - "2. Expensive to compute, which is why we had to do so much\n", - " optimization, and\n", - "\n", - "3. Not really what we want anyway.\n", - "\n", - "We don’t really care about the likelihood of seeing the exact dataset we\n", - "saw. Especially for continuous variables, we care about the likelihood\n", - "of seeing any dataset like the one we saw.\n", - "\n", - "For example, in the Euro problem, we don’t care about the order of the\n", - "coin flips, only the total number of heads and tails. And in the\n", - "locomotive problem, we don’t care about which particular trains were\n", - "seen, only the number of trains and the maximum of the serial numbers.\n", - "\n", - "Similarly, in the BRFSS sample, we don’t really want to know the\n", - "probability of seeing one particular set of values (especially since\n", - "there are hundreds of thousands of them). It is more relevant to ask,\n", - "“If we sample 100,000 people from a population with hypothetical values\n", - "of $\\mu$ and $\\sigma$, what would be the chance of collecting a sample\n", - "with the observed mean and variance?”\n", - "\n", - "For samples from a Gaussian distribution, we can answer this question\n", - "efficiently because we can find the distribution of the sample\n", - "statistics analytically. In fact, we already did it when we computed the\n", - "range of the prior.\n", - "\n", - "If you draw $n$ values from a Gaussian distribution with parameters\n", - "$\\mu$ and $\\sigma$, and compute the sample mean, $m$, the distribution\n", - "of $m$ is Gaussian with parameters $\\mu$ and $\\sigma / \\sqrt{n}$.\n", - "\n", - "Similarly, the distribution of the sample standard deviation, $s$, is\n", - "Gaussian with parameters $\\sigma$ and $\\sigma / \\sqrt{2 (n-1)}$.\n", - "\n", - "We can use these sample distributions to compute the likelihood of the\n", - "sample statistics, $m$ and $s$, given hypothetical values for $\\mu$ and\n", - "$\\sigma$. Here’s a new version of `LogUpdateSet` that does it:\n", - "\n", - " def LogUpdateSetABC(self, data):\n", - " xs = data\n", - " n = len(xs)\n", - "\n", - " # compute sample statistics\n", - " m = numpy.mean(xs)\n", - " s = numpy.std(xs)\n", - "\n", - " for hypo in sorted(self.Values()):\n", - " mu, sigma = hypo\n", - "\n", - " # compute log likelihood of m, given hypo\n", - " stderr_m = sigma / math.sqrt(n)\n", - " loglike = EvalGaussianLogPdf(m, mu, stderr_m)\n", - "\n", - " #compute log likelihood of s, given hypo\n", - " stderr_s = sigma / math.sqrt(2 * (n-1))\n", - " loglike += EvalGaussianLogPdf(s, sigma, stderr_s)\n", - "\n", - " self.Incr(hypo, loglike)\n", - "\n", - "On my computer this function processes the entire dataset in about a\n", - "second, and the result agrees with the exact result with about 5 digits\n", - "of precision.\n", - "\n", - "Robust estimation\n", - "-----------------\n", - "\n", - "![Contour plot of the posterior joint distribution of mean and standard\n", - "deviation of height for men in the\n", - "U.S.](figs/variability_posterior_male.pdf)\n", - "\n", - "[fig.variability1]\n", - "\n", - "![Contour plot of the posterior joint distribution of mean and standard\n", - "deviation of height for women in the\n", - "U.S.](figs/variability_posterior_female.pdf)\n", - "\n", - "[fig.variability2]\n", - "\n", - "We are almost ready to look at results, but we have one more problem to\n", - "deal with. There are a number of outliers in this dataset that are\n", - "almost certainly errors. For example, there are three adults with\n", - "reported height of 61 cm, which would place them among the shortest\n", - "living adults in the world. At the other end, there are four women with\n", - "reported height 229 cm, just short of the tallest women in the world.\n", - "\n", - "It is not impossible that these values are correct, but it is unlikely,\n", - "which makes it hard to know how to deal with them. And we have to get it\n", - "right, because these extreme values have a disproportionate effect on\n", - "the estimated variability.\n", - "\n", - "Because ABC is based on summary statistics, rather than the entire\n", - "dataset, we can make it more robust by choosing summary statistics that\n", - "are robust in the presence of outliers. For example, rather than use the\n", - "sample mean and standard deviation, we could use the median and\n", - "inter-quartile range (IQR), which is the difference between the 25th and\n", - "75th percentiles.\n", - "\n", - "More generally, we could compute an inter-percentile range (IPR) that\n", - "spans any given fraction of the distribution, p:\n", - "\n", - " def MedianIPR(xs, p):\n", - " cdf = thinkbayes.MakeCdfFromList(xs)\n", - " median = cdf.Percentile(50)\n", - "\n", - " alpha = (1-p) / 2\n", - " ipr = cdf.Value(1-alpha) - cdf.Value(alpha)\n", - " return median, ipr\n", - "\n", - "xs is a sequence of values. p is the desired\n", - "range; for example, p=0.5 yields the inter-quartile range.\n", - "\n", - "MedianIPR works by computing the CDF of xs,\n", - "then extracting the median and the difference between two percentiles.\n", - "\n", - "We can convert from ipr to an estimate of\n", - "sigma using the Gaussian CDF to compute the fraction of the\n", - "distribution covered by a given number of standard deviations. For\n", - "example, it is a well-known rule of thumb that 68% of a Gaussian\n", - "distribution falls within one standard deviation of the mean, which\n", - "leaves 16% in each tail. If we compute the range between the 16th and\n", - "84th percentiles, we expect the result to be 2 \\* sigma. So\n", - "we can estimate sigma by computing the 68% IPR and dividing\n", - "by 2.\n", - "\n", - "More generally we could use any number of sigmas.\n", - "MedianS performs the more general version of this\n", - "computation:\n", - "\n", - " def MedianS(xs, num_sigmas):\n", - " half_p = thinkbayes.StandardGaussianCdf(num_sigmas) - 0.5\n", - "\n", - " median, ipr = MedianIPR(xs, half_p * 2)\n", - " s = ipr / 2 / num_sigmas\n", - "\n", - " return median, s\n", - "\n", - "Again, xs is the sequence of values; `num_sigmas` is the\n", - "number of standard deviations the results should be based on. The result\n", - "is median, which estimates $\\mu$, and s, which\n", - "estimates $\\sigma$.\n", - "\n", - "Finally, in LogUpdateSetABC we can replace the sample mean\n", - "and standard deviation with median and s. And\n", - "that pretty much does it.\n", - "\n", - "It might seem odd that we are using observed percentiles to estimate\n", - "$\\mu$ and $\\sigma$, but it is an example of the flexibility of the\n", - "Bayesian approach. In effect we are asking, “Given hypothetical values\n", - "for $\\mu$ and $\\sigma$, and a sampling process that has some chance of\n", - "introducing errors, what is the likelihood of generating a given set of\n", - "sample statistics?”\n", - "\n", - "We are free to choose any sample statistics we like, up to a point:\n", - "$\\mu$ and $\\sigma$ determine the location and spread of a distribution,\n", - "so we need to choose statistics that capture those characteristics. For\n", - "example, if we chose the 49th and 51st percentiles, we would get very\n", - "little information about spread, so it would leave the estimate of\n", - "$\\sigma$ relatively unconstrained by the data. All values of\n", - "sigma would have nearly the same likelihood of producing\n", - "the observed values, so the posterior distribution of sigma\n", - "would look a lot like the prior.\n", - "\n", - "Who is more variable?\n", - "---------------------\n", - "\n", - "![Posterior distributions of CV for men and women, based on robust\n", - "estimators.](figs/variability_cv.pdf)\n", - "\n", - "[fig.variability3]\n", - "\n", - "Finally we are ready to answer the question we started with: is the\n", - "coefficient of variation greater for men than for women?\n", - "\n", - "Using ABC based on the median and IPR with `num_sigmas=1`, I computed\n", - "posterior joint distributions for mu and\n", - "sigma. Figures [fig.variability1] and  [fig.variability2]\n", - "show the results as a contour plot with mu on the x-axis,\n", - "sigma on the y-axis, and probability on the z-axis.\n", - "\n", - "For each joint distribution, I computed the posterior distribution of\n", - "CV. Figure [fig.variability3] shows these distributions for men and\n", - "women. The mean for men is 0.0410; for women it is 0.0429. Since there\n", - "is no overlap between the distributions, we conclude with near certainty\n", - "that women are more variable in height than men.\n", - "\n", - "So is that the end of the Variability Hypothesis? Sadly, no. It turns\n", - "out that this result depends on the choice of the inter-percentile\n", - "range. With `num_sigmas=1`, we conclude that women are more variable,\n", - "but with `num_sigmas=2` we conclude with equal confidence that men are\n", - "more variable.\n", - "\n", - "The reason for the difference is that there are more men of short\n", - "stature, and their distance from the mean is greater.\n", - "\n", - "So our evaluation of the Variability Hypothesis depends on the\n", - "interpretation of “variability.” With `num_sigmas=1` we focus on people\n", - "near the mean. As we increase `num_sigmas`, we give more weight to the\n", - "extremes.\n", - "\n", - "To decide which emphasis is appropriate, we would need a more precise\n", - "statement of the hypothesis. As it is, the Variability Hypothesis may be\n", - "too vague to evaluate.\n", - "\n", - "Nevertheless, it helped me demonstrate several new ideas and, I hope you\n", - "agree, it makes an interesting example.\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "There are two ways you might think of ABC. One interpretation is that it\n", - "is, as the name suggests, an approximation that is faster to compute\n", - "than the exact value.\n", - "\n", - "But remember that Bayesian analysis is always based on modeling\n", - "decisions, which implies that there is no “exact” solution. For any\n", - "interesting physical system there are many possible models, and each\n", - "model yields different results. To interpret the results, we have to\n", - "evaluate the models.\n", - "\n", - "So another interpretation of ABC is that it represents an alternative\n", - "model of the likelihood. When we compute $\\mathrm{p}(D|H)$,\n", - "we are asking “What is the likelihood of the data under a given\n", - "hypothesis?”\n", - "\n", - "For large datasets, the likelihood of the data is very small, which is a\n", - "hint that we might not be asking the right question. What we really want\n", - "to know is the likelihood of any outcome like the data, where the\n", - "definition of “like” is yet another modeling decision.\n", - "\n", - "The underlying idea of ABC is that two datasets are alike if they yield\n", - "the same summary statistics. But in some cases, like the example in this\n", - "chapter, it is not obvious which summary statistics to choose.\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "An “effect size” is a statistic intended to measure the difference\n", - "between two groups (see ).\n", - "\n", - "For example, we could use data from the BRFSS to estimate the difference\n", - "in height between men and women. By sampling values from the posterior\n", - "distributions of $\\mu$ and $\\sigma$, we could generate the posterior\n", - "distribution of this difference.\n", - "\n", - "But it might be better to use a dimensionless measure of effect size,\n", - "rather than a difference measured in cm. One option is to use divide\n", - "through by the standard deviation (similar to what we did with the\n", - "coefficient of variation).\n", - "\n", - "If the parameters for Group 1 are $(\\mu_1, \\sigma_1)$, and the\n", - "parameters for Group 2 are $(\\mu_2, \\sigma_2)$, the dimensionless effect\n", - "size is $$\\frac{\\mu_1 - \\mu_2}{(\\sigma_1 + \\sigma_2)/2}$$ Write a\n", - "function that takes joint distributions of mu and\n", - "sigma for two groups and returns the posterior distribution\n", - "of effect size.\n", - "\n", - "Hint: if enumerating all pairs from the two distributions takes too\n", - "long, consider random sampling.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Approximate Bayesian Computation\n", + "\n", + "## The Variability Hypothesis\n", + "\n", + "I have a soft spot for crank science. Recently I visited Norumbega\n", + "Tower, which is an enduring monument to the crackpot theories of Eben\n", + "Norton Horsford, inventor of double-acting baking powder and fake\n", + "history. But that’s not what this chapter is about.\n", + "\n", + "This chapter is about the Variability Hypothesis, which\n", + "\n", + "> “originated in the early nineteenth century with Johann Meckel, who\n", + "> argued that males have a greater range of ability than females,\n", + "> especially in intelligence. In other words, he believed that most\n", + "> geniuses and most mentally retarded people are men. Because he\n", + "> considered males to be the ’superior animal,’ Meckel concluded that\n", + "> females’ lack of variation was a sign of inferiority.”\n", + ">\n", + "> From .\n", + "\n", + "I particularly like that last part, because I suspect that if it turns\n", + "out that women are actually more variable, Meckel would take that as a\n", + "sign of inferiority, too. Anyway, you will not be surprised to hear that\n", + "the evidence for the Variability Hypothesis is weak.\n", + "\n", + "Nevertheless, it came up in my class recently when we looked at data\n", + "from the CDC’s Behavioral Risk Factor Surveillance System (BRFSS),\n", + "specifically the self-reported heights of adult American men and women.\n", + "The dataset includes responses from 154407 men and 254722 women. Here’s\n", + "what we found:\n", + "\n", + "- The average height for men is 178 cm; the average height for women\n", + " is 163 cm. So men are taller, on average. No surprise there.\n", + "\n", + "- For men the standard deviation is 7.7 cm; for women it is 7.3 cm. So\n", + " in absolute terms, men’s heights are more variable.\n", + "\n", + "- But to compare variability between groups, it is more meaningful to\n", + " use the coefficient of variation (CV), which is the standard\n", + " deviation divided by the mean. It is a dimensionless measure of\n", + " variability relative to scale. For men CV is 0.0433; for women it is\n", + " 0.0444.\n", + "\n", + "That’s very close, so we could conclude that this dataset provides weak\n", + "evidence against the Variability Hypothesis. But we can use Bayesian\n", + "methods to make that conclusion more precise. And answering this\n", + "question gives me a chance to demonstrate some techniques for working\n", + "with large datasets.\n", + "\n", + "I will proceed in a few steps:\n", + "\n", + "1. We’ll start with the simplest implementation, but it only works for\n", + " datasets smaller than 1000 values.\n", + "\n", + "2. By computing probabilities under a log transform, we can scale up to\n", + " the full size of the dataset, but the computation gets slow.\n", + "\n", + "3. Finally, we speed things up substantially with Approximate Bayesian\n", + " Computation, also known as ABC.\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Mean and standard deviation\n", + "\n", + "In Chapter [paintball] we estimated two parameters simultaneously using\n", + "a joint distribution. In this chapter we use the same method to estimate\n", + "the parameters of a Gaussian distribution: the mean, `mu`,\n", + "and the standard deviation, `sigma`.\n", + "\n", + "For this problem, I define a Suite called `Height` that\n", + "represents a map from each `mu, sigma` pair to its\n", + "probability:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "import thinkbayes\n", + "\n", + "class Height(thinkbayes.Suite, thinkbayes.Joint):\n", + "\n", + " def __init__(self, mus, sigmas):\n", + " thinkbayes.Suite.__init__(self)\n", + "\n", + " pairs = [(mu, sigma) \n", + " for mu in mus\n", + " for sigma in sigmas]\n", + "\n", + " thinkbayes.Suite.__init__(self, pairs)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`mus` is a sequence of possible values for `mu`;\n", + "`sigmas` is a sequence of values for `sigma`. The\n", + "prior distribution is uniform over all `mu, sigma` pairs.\n", + "\n", + "The likelihood function is easy. Given hypothetical values of\n", + "`mu` and `sigma`, we compute the likelihood of a\n", + "particular value, `x`. That’s what\n", + "`EvalGaussianPdf` does, so all we have to do is use it:\n", + "\n", + "```python\n", + " # class Height\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " x = data\n", + " mu, sigma = hypo\n", + " like = thinkbayes.EvalGaussianPdf(x, mu, sigma)\n", + " return like\n", + "```\n", + "\n", + "If you have studied statistics from a mathematical perspective, you know\n", + "that when you evaluate a PDF, you get a probability density. In order to\n", + "get a probability, you have to integrate probability densities over some\n", + "range.\n", + "\n", + "But for our purposes, we don’t need a probability; we just need\n", + "something proportional to the probability we want. A probability density\n", + "does that job nicely.\n", + "\n", + "The hardest part of this problem turns out to be choosing appropriate\n", + "ranges for `mus` and `sigmas`. If the range is too\n", + "small, we omit some possibilities with non-negligible probability and\n", + "get the wrong answer. If the range is too big, we get the right answer,\n", + "but waste computational power.\n", + "\n", + "So this is an opportunity to use classical estimation to make Bayesian\n", + "techniques more efficient. Specifically, we can use classical estimators\n", + "to find a likely location for `mu` and `sigma`,\n", + "and use the standard errors of those estimates to choose a likely\n", + "spread.\n", + "\n", + "If the true parameters of the distribution are $\\mu$ and $\\sigma$, and\n", + "we take a sample of $n$ values, an estimator of $\\mu$ is the sample\n", + "mean, `m`.\n", + "\n", + "And an estimator of $\\sigma$ is the sample standard variance,\n", + "`s`.\n", + "\n", + "The standard error of the estimated $\\mu$ is $s / \\sqrt{n}$ and the\n", + "standard error of the estimated $\\sigma$ is $s / \\sqrt{2 (n-1)}$.\n", + "\n", + "Here’s the code to compute all that:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def FindPriorRanges(xs, num_points, num_stderrs=3.0):\n", + "\n", + " # compute m and s\n", + " n = len(xs)\n", + " m = numpy.mean(xs)\n", + " s = numpy.std(xs)\n", + "\n", + " # compute ranges for m and s\n", + " stderr_m = s / math.sqrt(n)\n", + " mus = MakeRange(m, stderr_m, num_stderrs)\n", + "\n", + " stderr_s = s / math.sqrt(2 * (n-1))\n", + " sigmas = MakeRange(s, stderr_s, num_stderrs)\n", + "\n", + " return mus, sigmas" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`xs` is the dataset. `num_points` is the desired number of\n", + "values in the range. `num_stderrs` is the width of the range on each\n", + "side of the estimate, in number of standard errors.\n", + "\n", + "The return value is a pair of sequences, `mus` and\n", + "`sigmas`.\n", + "\n", + "Here’s `MakeRange`:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeRange(estimate, stderr, num_stderrs):\n", + " spread = stderr * num_stderrs\n", + " array = numpy.linspace(estimate-spread,\n", + " estimate+spread,\n", + " num_points)\n", + " return array" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`numpy.linspace` makes an array of equally spaced elements\n", + "between `estimate-spread` and `estimate+spread`,\n", + "including both." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Update\n", + "\n", + "Finally here’s the code to make and update the suite:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'xs' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mmus\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msigmas\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mFindPriorRanges\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mxs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnum_points\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 2\u001b[0m \u001b[0msuite\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mHeight\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmus\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msigmas\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 3\u001b[0m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mUpdateSet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mxs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMaximumLikelihood\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mNameError\u001b[0m: name 'xs' is not defined" + ] + } + ], + "source": [ + "mus, sigmas = FindPriorRanges(xs, num_points)\n", + "suite = Height(mus, sigmas)\n", + "suite.UpdateSet(xs)\n", + "print(suite.MaximumLikelihood())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This process might seem bogus, because we use the data to choose the\n", + "range of the prior distribution, and then use the data again to do the\n", + "update. In general, using the same data twice is, in fact, bogus.\n", + "\n", + "But in this case it is ok. Really. We use the data to choose the range\n", + "for the prior, but only to avoid computing a lot of probabilities that\n", + "would have been very small anyway. With `num_stderrs=4`, the range is\n", + "big enough to cover all values with non-negligible likelihood. After\n", + "that, making it bigger has no effect on the results.\n", + "\n", + "In effect, the prior is uniform over all values of `mu` and\n", + "`sigma`, but for computational efficiency we ignore all the\n", + "values that don’t matter." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The posterior distribution of CV\n", + "\n", + "Once we have the posterior joint distribution of `mu` and\n", + "`sigma`, we can compute the distribution of CV for men and\n", + "women, and then the probability that one exceeds the other.\n", + "\n", + "To compute the distribution of CV, we enumerate pairs of `mu`\n", + "and `sigma`:" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def CoefVariation(suite):\n", + " pmf = thinkbayes.Pmf()\n", + " for (mu, sigma), p in suite.Items():\n", + " pmf.Incr(sigma/mu, p)\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then we use `thinkbayes.PmfProbGreater` to compute the probability that\n", + "men are more variable.\n", + "\n", + "The analysis itself is simple, but there are two more issues we have to\n", + "deal with:\n", + "\n", + "1. As the size of the dataset increases, we run into a series of\n", + " computational problems due to the limitations of floating-point\n", + " arithmetic.\n", + "\n", + "2. The dataset contains a number of extreme values that are almost\n", + " certainly errors. We will need to make the estimation process robust\n", + " in the presence of these outliers.\n", + "\n", + "The following sections explain these problems and their solutions." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Underflow\n", + "\n", + "If we select the first 100 values from the BRFSS dataset and run the\n", + "analysis I just described, it runs without errors and we get posterior\n", + "distributions that look reasonable.\n", + "\n", + "If we select the first 1000 values and run the program again, we get an\n", + "error in `Pmf.Normalize`:\n", + "\n", + " ValueError: total probability is zero.\n", + "\n", + "The problem is that we are using probability densities to compute\n", + "likelihoods, and densities from continuous distributions tend to be\n", + "small. And if you take 1000 small values and multiply them together, the\n", + "result is very small. In this case it is so small it can’t be\n", + "represented by a floating-point number, so it gets rounded down to zero,\n", + "which is called `**underflow**`. And if all probabilities in\n", + "the distribution are 0, it’s not a distribution any more.\n", + "\n", + "A possible solution is to renormalize the Pmf after each update, or\n", + "after each batch of 100. That would work, but it would be slow.\n", + "\n", + "A better alternative is to compute likelihoods under a log transform.\n", + "That way, instead of multiplying small values, we can add up log\n", + "likelihoods. `Pmf` provides methods `Log`,\n", + "`LogUpdateSet` and `Exp` to make this process\n", + "easy.\n", + "\n", + "`Log` computes the log of the probabilities in a Pmf:\n", + "\n", + "```python\n", + "# class Pmf\n", + " def Log(self):\n", + " m = self.MaxLike()\n", + " for x, p in self.d.iteritems():\n", + " if p:\n", + " self.Set(x, math.log(p/m))\n", + " else:\n", + " self.Remove(x)\n", + "```\n", + "\n", + "Before applying the log transform `Log` uses\n", + "`MaxLike` to find `m`, the highest probability in\n", + "the Pmf. It divide all probabilities by `m`, so the highest\n", + "probability gets normalized to 1, which yields a log of 0. The other log\n", + "probabilities are all negative. If there are any values in the Pmf with\n", + "probability 0, they are removed.\n", + "\n", + "While the Pmf is under a log transform, we can’t use\n", + "`Update`, `UpdateSet`, or `Normalize`.\n", + "The result would be nonsensical; if you try, Pmf raises an exception.\n", + "Instead, we have to use `LogUpdate` and\n", + "`LogUpdateSet`.\n", + "\n", + "Here’s the implementation of `LogUpdateSet`:\n", + "\n", + "```python\n", + "# class Suite\n", + " def LogUpdateSet(self, dataset):\n", + " for data in dataset:\n", + " self.LogUpdate(data)\n", + "```\n", + "\n", + "`LogUpdateSet` loops through the data and calls\n", + "`LogUpdate`:\n", + "\n", + "```python\n", + "# class Suite\n", + " def LogUpdate(self, data):\n", + " for hypo in self.Values():\n", + " like = self.LogLikelihood(data, hypo)\n", + " self.Incr(hypo, like)\n", + "```\n", + "\n", + "`LogUpdate` is just like `Update` except that it\n", + "calls `LogLikelihood` instead of `Likelihood`, and\n", + "`Incr` instead of `Mult`.\n", + "\n", + "Using log-likelihoods avoids the problem with underflow, but while the\n", + "Pmf is under the log transform, there’s not much we can do with it. We\n", + "have to use `Exp` to invert the transform:\n", + "\n", + "```python\n", + "# class Pmf\n", + " def Exp(self):\n", + " m = self.MaxLike()\n", + " for x, p in self.d.iteritems():\n", + " self.Set(x, math.exp(p-m))\n", + "```\n", + "\n", + "If the log-likelihoods are large negative numbers, the resulting\n", + "likelihoods might underflow. So `Exp` finds the maximum\n", + "log-likelihood, `m`, and shifts all the likelihoods up by\n", + "`m`. The resulting distribution has a maximum likelihood of\n", + "1. This process inverts the log transform with minimal loss of\n", + "precision." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Log-likelihood\n", + "\n", + "Now all we need is `LogLikelihood`.\n", + "\n", + "```python\n", + "# class Height\n", + "\n", + " def LogLikelihood(self, data, hypo):\n", + " x = data\n", + " mu, sigma = hypo\n", + " loglike = scipy.stats.norm.logpdf(x, mu, sigma)\n", + " return loglike\n", + "```\n", + "\n", + "`norm.logpdf` computes the log-likelihood of the Gaussian\n", + "PDF.\n", + "\n", + "Here’s what the whole update process looks like:" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'suite' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mLog\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 2\u001b[0m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mLogUpdateSet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mxs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 3\u001b[0m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mExp\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mNormalize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mNameError\u001b[0m: name 'suite' is not defined" + ] + } + ], + "source": [ + "suite.Log()\n", + "suite.LogUpdateSet(xs)\n", + "suite.Exp()\n", + "suite.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To review, `Log` puts the suite under a log transform.\n", + "`LogUpdateSet` calls `LogUpdate`, which calls\n", + "`LogLikelihood`. `LogUpdate` uses\n", + "`Pmf.Incr`, because adding a log-likelihood is the same as\n", + "multiplying by a likelihood.\n", + "\n", + "After the update, the log-likelihoods are large negative numbers, so\n", + "`Exp` shifts them up before inverting the transform, which is\n", + "how we avoid underflow.\n", + "\n", + "Once the suite is transformed back, the probabilities are “linear”\n", + "again, which means “not logarithmic”, so we can use\n", + "`Normalize` again.\n", + "\n", + "Using this algorithm, we can process the entire dataset without\n", + "underflow, but it is still slow. On my computer it might take an hour.\n", + "We can do better." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## A little optimization\n", + "\n", + "This section uses math and computational optimization to speed things up\n", + "by a factor of 100. But the following section presents an algorithm that\n", + "is even faster. So if you want to get right to the good stuff, feel free\n", + "to skip this section.\n", + "\n", + "`Suite.LogUpdateSet` calls `LogUpdate` once for\n", + "each data point. We can speed it up by computing the log-likelihood of\n", + "the entire dataset at once.\n", + "\n", + "We’ll start with the Gaussian PDF:\n", + "\n", + "$$\\frac{1}{\\sigma \\sqrt{2 \\pi}} \\exp \\left[ -\\frac{1}{2} \\left( \\frac{x-\\mu}{\\sigma} \\right)^2 \\right]$$\n", + "\n", + "and compute the log (dropping the constant term):\n", + "\n", + "$$-\\log \\sigma -\\frac{1}{2} \\left( \\frac{x-\\mu}{\\sigma} \\right)^2$$\n", + "\n", + "Given a sequence of values, $x_i$, the total log-likelihood is\n", + "\n", + "$$\\sum_i -\\log \\sigma - \\frac{1}{2} \\left( \\frac{x_i-\\mu}{\\sigma} \\right)^2$$\n", + "\n", + "Pulling out the terms that don’t depend on $i$, we get\n", + "\n", + "$$-n \\log \\sigma - \\frac{1}{2 \\sigma^2} \\sum_i (x_i - \\mu)^2$$\n", + "\n", + "which we\n", + "can translate into Python:\n", + "\n", + "```python\n", + "# class Height\n", + " def LogUpdateSetFast(self, data):\n", + " xs = tuple(data)\n", + " n = len(xs)\n", + "\n", + " for hypo in self.Values():\n", + " mu, sigma = hypo\n", + " total = Summation(xs, mu)\n", + " loglike = -n * math.log(sigma) - total / 2 / sigma**2\n", + " self.Incr(hypo, loglike)\n", + "```\n", + "\n", + "By itself, this would be a small improvement, but it creates an\n", + "opportunity for a bigger one. Notice that the summation only depends on\n", + "`mu`, not `sigma`, so we only have to compute it\n", + "once for each value of `mu`.\n", + "\n", + "To avoid recomputing, I factor out a function that computes the\n", + "summation, and `**memoize**` it so it stores previously\n", + "computed results in a dictionary (see\n", + "):\n", + "\n", + "```python\n", + "def Summation(xs, mu, cache={}):\n", + " try:\n", + " return cache[xs, mu]\n", + " except KeyError:\n", + " ds = [(x-mu)**2 for x in xs]\n", + " total = sum(ds)\n", + " cache[xs, mu] = total\n", + " return total\n", + "```\n", + "\n", + "`cache` stores previously computed sums. The `try`\n", + "statement returns a result from the cache if possible; otherwise it\n", + "computes the summation, then caches and returns the result.\n", + "\n", + "The only catch is that we can’t use a list as a key in the cache,\n", + "because it is not a hashable type. That’s why\n", + "`LogUpdateSetFast` converts the dataset to a tuple.\n", + "\n", + "This optimization speeds up the computation by about a factor of 100,\n", + "processing the entire dataset (154 407 men and 254 722 women) in less\n", + "than a minute on my not-very-fast computer." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## ABC\n", + "\n", + "But maybe you don’t have that kind of time. In that case, Approximate\n", + "Bayesian Computation (ABC) might be the way to go. The motivation behind\n", + "ABC is that the likelihood of any particular dataset is:\n", + "\n", + "1. Very small, especially for large datasets, which is why we had to\n", + " use the log transform,\n", + "\n", + "2. Expensive to compute, which is why we had to do so much\n", + " optimization, and\n", + "\n", + "3. Not really what we want anyway.\n", + "\n", + "We don’t really care about the likelihood of seeing the exact dataset we\n", + "saw. Especially for continuous variables, we care about the likelihood\n", + "of seeing any dataset like the one we saw.\n", + "\n", + "For example, in the Euro problem, we don’t care about the order of the\n", + "coin flips, only the total number of heads and tails. And in the\n", + "locomotive problem, we don’t care about which particular trains were\n", + "seen, only the number of trains and the maximum of the serial numbers.\n", + "\n", + "Similarly, in the BRFSS sample, we don’t really want to know the\n", + "probability of seeing one particular set of values (especially since\n", + "there are hundreds of thousands of them). It is more relevant to ask,\n", + "“If we sample 100,000 people from a population with hypothetical values\n", + "of $\\mu$ and $\\sigma$, what would be the chance of collecting a sample\n", + "with the observed mean and variance?”\n", + "\n", + "For samples from a Gaussian distribution, we can answer this question\n", + "efficiently because we can find the distribution of the sample\n", + "statistics analytically. In fact, we already did it when we computed the\n", + "range of the prior.\n", + "\n", + "If you draw $n$ values from a Gaussian distribution with parameters\n", + "$\\mu$ and $\\sigma$, and compute the sample mean, $m$, the distribution\n", + "of $m$ is Gaussian with parameters $\\mu$ and $\\sigma / \\sqrt{n}$.\n", + "\n", + "Similarly, the distribution of the sample standard deviation, $s$, is\n", + "Gaussian with parameters $\\sigma$ and $\\sigma / \\sqrt{2 (n-1)}$.\n", + "\n", + "We can use these sample distributions to compute the likelihood of the\n", + "sample statistics, $m$ and $s$, given hypothetical values for $\\mu$ and\n", + "$\\sigma$. Here’s a new version of `LogUpdateSet` that does it:\n", + "\n", + "```python\n", + " def LogUpdateSetABC(self, data):\n", + " xs = data\n", + " n = len(xs)\n", + "\n", + " # compute sample statistics\n", + " m = numpy.mean(xs)\n", + " s = numpy.std(xs)\n", + "\n", + " for hypo in sorted(self.Values()):\n", + " mu, sigma = hypo\n", + "\n", + " # compute log likelihood of m, given hypo\n", + " stderr_m = sigma / math.sqrt(n)\n", + " loglike = EvalGaussianLogPdf(m, mu, stderr_m)\n", + "\n", + " #compute log likelihood of s, given hypo\n", + " stderr_s = sigma / math.sqrt(2 * (n-1))\n", + " loglike += EvalGaussianLogPdf(s, sigma, stderr_s)\n", + "\n", + " self.Incr(hypo, loglike)\n", + "```\n", + "\n", + "On my computer this function processes the entire dataset in about a\n", + "second, and the result agrees with the exact result with about 5 digits\n", + "of precision." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Robust estimation\n", + "\n", + "![Contour plot of the posterior joint distribution of mean and standard\n", + "deviation of height for men in the\n", + "U.S.](figs/variability_posterior_male.pdf)\n", + "\n", + "[fig.variability1]\n", + "\n", + "![Contour plot of the posterior joint distribution of mean and standard\n", + "deviation of height for women in the\n", + "U.S.](figs/variability_posterior_female.pdf)\n", + "\n", + "[fig.variability2]\n", + "\n", + "We are almost ready to look at results, but we have one more problem to\n", + "deal with. There are a number of outliers in this dataset that are\n", + "almost certainly errors. For example, there are three adults with\n", + "reported height of 61 cm, which would place them among the shortest\n", + "living adults in the world. At the other end, there are four women with\n", + "reported height 229 cm, just short of the tallest women in the world.\n", + "\n", + "It is not impossible that these values are correct, but it is unlikely,\n", + "which makes it hard to know how to deal with them. And we have to get it\n", + "right, because these extreme values have a disproportionate effect on\n", + "the estimated variability.\n", + "\n", + "Because ABC is based on summary statistics, rather than the entire\n", + "dataset, we can make it more robust by choosing summary statistics that\n", + "are robust in the presence of outliers. For example, rather than use the\n", + "sample mean and standard deviation, we could use the median and\n", + "inter-quartile range (IQR), which is the difference between the 25th and\n", + "75th percentiles.\n", + "\n", + "More generally, we could compute an inter-percentile range (IPR) that\n", + "spans any given fraction of the distribution, `p`:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MedianIPR(xs, p):\n", + " cdf = thinkbayes.MakeCdfFromList(xs)\n", + " median = cdf.Percentile(50)\n", + "\n", + " alpha = (1-p) / 2\n", + " ipr = cdf.Value(1-alpha) - cdf.Value(alpha)\n", + " return median, ipr" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`xs` is a sequence of values. `p` is the desired\n", + "range; for example, `p=0.5` yields the inter-quartile range.\n", + "\n", + "`MedianIPR` works by computing the CDF of `xs`,\n", + "then extracting the median and the difference between two percentiles.\n", + "\n", + "We can convert from `ipr` to an estimate of\n", + "`sigma` using the Gaussian CDF to compute the fraction of the\n", + "distribution covered by a given number of standard deviations. For\n", + "example, it is a well-known rule of thumb that 68% of a Gaussian\n", + "distribution falls within one standard deviation of the mean, which\n", + "leaves 16% in each tail. If we compute the range between the 16th and\n", + "84th percentiles, we expect the result to be `2 \\* sigma`. So\n", + "we can estimate `sigma` by computing the 68% IPR and dividing\n", + "by 2.\n", + "\n", + "More generally we could use any number of `sigmas`.\n", + "`MedianS` performs the more general version of this\n", + "computation:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MedianS(xs, num_sigmas):\n", + " half_p = thinkbayes.StandardGaussianCdf(num_sigmas) - 0.5\n", + "\n", + " median, ipr = MedianIPR(xs, half_p * 2)\n", + " s = ipr / 2 / num_sigmas\n", + "\n", + " return median, s" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, `xs` is the sequence of values; `num_sigmas` is the\n", + "number of standard deviations the results should be based on. The result\n", + "is `median`, which estimates $\\mu$, and `s`, which\n", + "estimates $\\sigma$.\n", + "\n", + "Finally, in `LogUpdateSetABC` we can replace the sample mean\n", + "and standard deviation with `median` and `s`. And\n", + "that pretty much does it.\n", + "\n", + "It might seem odd that we are using observed percentiles to estimate\n", + "$\\mu$ and $\\sigma$, but it is an example of the flexibility of the\n", + "Bayesian approach. In effect we are asking, “Given hypothetical values\n", + "for $\\mu$ and $\\sigma$, and a sampling process that has some chance of\n", + "introducing errors, what is the likelihood of generating a given set of\n", + "sample statistics?”\n", + "\n", + "We are free to choose any sample statistics we like, up to a point:\n", + "$\\mu$ and $\\sigma$ determine the location and spread of a distribution,\n", + "so we need to choose statistics that capture those characteristics. For\n", + "example, if we chose the 49th and 51st percentiles, we would get very\n", + "little information about spread, so it would leave the estimate of\n", + "$\\sigma$ relatively unconstrained by the data. All values of\n", + "`sigma` would have nearly the same likelihood of producing\n", + "the observed values, so the posterior distribution of `sigma`\n", + "would look a lot like the prior." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Who is more variable?\n", + "\n", + "![Posterior distributions of CV for men and women, based on robust\n", + "estimators.](figs/variability_cv.pdf)\n", + "\n", + "[fig.variability3]\n", + "\n", + "Finally we are ready to answer the question we started with: is the\n", + "coefficient of variation greater for men than for women?\n", + "\n", + "Using ABC based on the median and IPR with `num_sigmas=1`, I computed\n", + "posterior joint distributions for `mu` and\n", + "`sigma`. Figures [fig.variability1] and  [fig.variability2]\n", + "show the results as a contour plot with `mu` on the x-axis,\n", + "`sigma` on the y-axis, and probability on the z-axis.\n", + "\n", + "For each joint distribution, I computed the posterior distribution of\n", + "CV. Figure [fig.variability3] shows these distributions for men and\n", + "women. The mean for men is 0.0410; for women it is 0.0429. Since there\n", + "is no overlap between the distributions, we conclude with near certainty\n", + "that women are more variable in height than men.\n", + "\n", + "So is that the end of the Variability Hypothesis? Sadly, no. It turns\n", + "out that this result depends on the choice of the inter-percentile\n", + "range. With `num_sigmas=1`, we conclude that women are more variable,\n", + "but with `num_sigmas=2` we conclude with equal confidence that men are\n", + "more variable.\n", + "\n", + "The reason for the difference is that there are more men of short\n", + "stature, and their distance from the mean is greater.\n", + "\n", + "So our evaluation of the Variability Hypothesis depends on the\n", + "interpretation of “variability.” With `num_sigmas=1` we focus on people\n", + "near the mean. As we increase `num_sigmas`, we give more weight to the\n", + "extremes.\n", + "\n", + "To decide which emphasis is appropriate, we would need a more precise\n", + "statement of the hypothesis. As it is, the Variability Hypothesis may be\n", + "too vague to evaluate.\n", + "\n", + "Nevertheless, it helped me demonstrate several new ideas and, I hope you\n", + "agree, it makes an interesting example." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "There are two ways you might think of ABC. One interpretation is that it\n", + "is, as the name suggests, an approximation that is faster to compute\n", + "than the exact value.\n", + "\n", + "But remember that Bayesian analysis is always based on modeling\n", + "decisions, which implies that there is no “exact” solution. For any\n", + "interesting physical system there are many possible models, and each\n", + "model yields different results. To interpret the results, we have to\n", + "evaluate the models.\n", + "\n", + "So another interpretation of ABC is that it represents an alternative\n", + "model of the likelihood. When we compute `$\\mathrm{p}(D|H)$`,\n", + "we are asking “What is the likelihood of the data under a given\n", + "hypothesis?”\n", + "\n", + "For large datasets, the likelihood of the data is very small, which is a\n", + "hint that we might not be asking the right question. What we really want\n", + "to know is the likelihood of any outcome like the data, where the\n", + "definition of “like” is yet another modeling decision.\n", + "\n", + "The underlying idea of ABC is that two datasets are alike if they yield\n", + "the same summary statistics. But in some cases, like the example in this\n", + "chapter, it is not obvious which summary statistics to choose.\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "An “effect size” is a statistic intended to measure the difference\n", + "between two groups (see ).\n", + "\n", + "For example, we could use data from the BRFSS to estimate the difference\n", + "in height between men and women. By sampling values from the posterior\n", + "distributions of $\\mu$ and $\\sigma$, we could generate the posterior\n", + "distribution of this difference.\n", + "\n", + "But it might be better to use a dimensionless measure of effect size,\n", + "rather than a difference measured in cm. One option is to use divide\n", + "through by the standard deviation (similar to what we did with the\n", + "coefficient of variation).\n", + "\n", + "If the parameters for Group 1 are $(\\mu_1, \\sigma_1)$, and the\n", + "parameters for Group 2 are $(\\mu_2, \\sigma_2)$, the dimensionless effect\n", + "size is\n", + "$$\\frac{\\mu_1 - \\mu_2}{(\\sigma_1 + \\sigma_2)/2}$$ \n", + "\n", + "Write a function that takes joint distributions of `mu` and\n", + "`sigma` for two groups and returns the posterior distribution\n", + "of effect size.\n", + "\n", + "Hint: if enumerating all pairs from the two distributions takes too\n", + "long, consider random sampling." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/11_Hypothesis_Testing.ipynb b/notebooks/11_Hypothesis_Testing.ipynb index fdeab1f..f04dc31 100644 --- a/notebooks/11_Hypothesis_Testing.ipynb +++ b/notebooks/11_Hypothesis_Testing.ipynb @@ -1,285 +1,716 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "Hypothesis Testing\n", - "==================\n", - "\n", - "Back to the Euro problem\n", - "------------------------\n", - "\n", - "In Section [euro] I presented a problem from MacKay’s *Information\n", - "Theory, Inference, and Learning Algorithms*:\n", - "\n", - "> A statistical statement appeared in \\`\\`The Guardian\" on Friday\n", - "> January 4, 2002:\n", - ">\n", - "> > When spun on edge 250 times, a Belgian one-euro coin came up heads\n", - "> > 140 times and tails 110. ‘It looks very suspicious to me,’ said\n", - "> > Barry Blight, a statistics lecturer at the London School of\n", - "> > Economics. ‘If the coin were unbiased, the chance of getting a\n", - "> > result as extreme as that would be less than 7%.’\n", - ">\n", - "> But do these data give evidence that the coin is biased rather than\n", - "> fair?\n", - "\n", - "We estimated the probability that the coin would land face up, but we\n", - "didn’t really answer MacKay’s question: Do the data give evidence that\n", - "the coin is biased?\n", - "\n", - "In Chapter [more] I proposed that data are in favor of a hypothesis if\n", - "the data are more likely under the hypothesis than under the alternative\n", - "or, equivalently, if the Bayes factor is greater than 1.\n", - "\n", - "In the Euro example, we have two hypotheses to consider: I’ll use $F$\n", - "for the hypothesis that the coin is fair and $B$ for the hypothesis that\n", - "it is biased.\n", - "\n", - "If the coin is fair, it is easy to compute the likelihood of the data,\n", - "$\\mathrm{p}(D|F)$. In fact, we already wrote the function\n", - "that does it.\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " x = hypo / 100.0\n", - " head, tails = data\n", - " like = x**heads * (1-x)**tails\n", - " return like\n", - "\n", - "To use it we can create a Euro suite and invoke\n", - "Likelihood:\n", - "\n", - " suite = Euro()\n", - " likelihood = suite.Likelihood(data, 50)\n", - "\n", - "$\\mathrm{p}(D|F)$ is $5.5 \\cdot 10^{-76}$, which doesn’t\n", - "tell us much except that the probability of seeing any particular\n", - "dataset is very small. It takes two likelihoods to make a ratio, so we\n", - "also have to compute $\\mathrm{p}(D|B)$.\n", - "\n", - "It is not obvious how to compute the likelihood of $B$, because it’s not\n", - "obvious what “biased” means.\n", - "\n", - "One possibility is to cheat and look at the data before we define the\n", - "hypothesis. In that case we would say that “biased” means that the\n", - "probability of heads is 140/250.\n", - "\n", - " actual_percent = 100.0 * 140 / 250\n", - " likelihood = suite.Likelihood(data, actual_percent)\n", - "\n", - "This version of $B$ I call `B_cheat`; the likelihood of `b_cheat` is\n", - "$34 \\cdot 10^{-76}$ and the likelihood ratio is 6.1. So we would say\n", - "that the data are evidence in favor of this version of $B$.\n", - "\n", - "But using the data to formulate the hypothesis is obviously bogus. By\n", - "that definition, any dataset would be evidence in favor of $B$, unless\n", - "the observed percentage of heads is exactly 50%.\n", - "\n", - "Making a fair comparison {#suitelike}\n", - "------------------------\n", - "\n", - "To make a legitimate comparison, we have to define $B$ without looking\n", - "at the data. So let’s try a different definition. If you inspect a\n", - "Belgian Euro coin, you might notice that the “heads” side is more\n", - "prominent than the “tails” side. You might expect the shape to have some\n", - "effect on $x$, but be unsure whether it makes heads more or less likely.\n", - "So you might say “I think the coin is biased so that $x$ is either 0.6\n", - "or 0.4, but I am not sure which.”\n", - "\n", - "We can think of this version, which I’ll call `B_two` as a hypothesis\n", - "made up of two sub-hypotheses. We can compute the likelihood for each\n", - "sub-hypothesis and then compute the average likelihood.\n", - "\n", - " like40 = suite.Likelihood(data, 40)\n", - " like60 = suite.Likelihood(data, 60)\n", - " likelihood = 0.5 * like40 + 0.5 * like60\n", - "\n", - "The likelihood ratio (or Bayes factor) for `b_two` is 1.3, which means\n", - "the data provide weak evidence in favor of `b_two`.\n", - "\n", - "More generally, suppose you suspect that the coin is biased, but you\n", - "have no clue about the value of $x$. In that case you might build a\n", - "Suite, which I call `b_uniform`, to represent sub-hypotheses from 0 to\n", - "100.\n", - "\n", - " b_uniform = Euro(xrange(0, 101))\n", - " b_uniform.Remove(50)\n", - " b_uniform.Normalize()\n", - "\n", - "I initialize `b_uniform` with values from 0 to 100. I removed the\n", - "sub-hypothesis that $x$ is 50%, because if $x$ is 50% the coin is fair,\n", - "but it has almost no effect on the result whether you remove it or not.\n", - "\n", - "To compute the likelihood of `b_uniform` we compute the likelihood of\n", - "each sub-hypothesis and accumulate a weighted average.\n", - "\n", - " def SuiteLikelihood(suite, data):\n", - " total = 0\n", - " for hypo, prob in suite.Items():\n", - " like = suite.Likelihood(data, hypo)\n", - " total += prob * like\n", - " return total\n", - "\n", - "The likelihood ratio for `b_uniform` is 0.47, which means that the data\n", - "are weak evidence against `b_uniform`, compared to $F$.\n", - "\n", - "If you think about the computation performed by `SuiteLikelihood`, you\n", - "might notice that it is similar to an update. To refresh your memory,\n", - "here’s the Update function:\n", - "\n", - " def Update(self, data):\n", - " for hypo in self.Values():\n", - " like = self.Likelihood(data, hypo)\n", - " self.Mult(hypo, like)\n", - " return self.Normalize()\n", - "\n", - "And here’s Normalize:\n", - "\n", - " def Normalize(self):\n", - " total = self.Total()\n", - " \n", - " factor = 1.0 / total\n", - " for x in self.d:\n", - " self.d[x] *= factor\n", - "\n", - " return total\n", - "\n", - "The return value from Normalize is the total of the\n", - "probabilities in the Suite, which is the average of the likelihoods for\n", - "the sub-hypotheses, weighted by the prior probabilities. And\n", - "Update passes this value along, so instead of using\n", - "SuiteLikelihood, we could compute the likelihood of\n", - "`b_uniform` like this:\n", - "\n", - " likelihood = b_uniform.Update(data)\n", - "\n", - "The triangle prior\n", - "------------------\n", - "\n", - "In Chapter [more] we also considered a triangle-shaped prior that gives\n", - "higher probability to values of $x$ near 50%. If we think of this prior\n", - "as a suite of sub-hypotheses, we can compute its likelihood like this:\n", - "\n", - " b_triangle = TrianglePrior()\n", - " likelihood = b_triangle.Update(data)\n", - "\n", - "The likelihood ratio for `b_triangle` is 0.84, compared to $F$, so again\n", - "we would say that the data are weak evidence against $B$.\n", - "\n", - "The following table shows the priors we have considered, the likelihood\n", - "of each, and the likelihood ratio (or Bayes factor) relative to $F$.\n", - "\n", - " -------------- ------------------- --------\n", - " Hypothesis Likelihood Bayes\n", - " $\\times 10^{-76}$ Factor\n", - " $F$ 5.5 –\n", - " `B_cheat` 34 6.1\n", - " `B_two` 7.4 1.3\n", - " `B_uniform` 2.6 0.47\n", - " `B_triangle` 4.6 0.84\n", - " -------------- ------------------- --------\n", - "\n", - "Depending on which definition we choose, the data might provide evidence\n", - "for or against the hypothesis that the coin is biased, but in either\n", - "case it is relatively weak evidence.\n", - "\n", - "In summary, we can use Bayesian hypothesis testing to compare the\n", - "likelihood of $F$ and $B$, but we have to do some work to specify\n", - "precisely what $B$ means. This specification depends on background\n", - "information about coins and their behavior when spun, so people could\n", - "reasonably disagree about the right definition.\n", - "\n", - "My presentation of this example follows David MacKay’s discussion, and\n", - "comes to the same conclusion. You can download the code I used in this\n", - "chapter from . For more information see\n", - "Section [download].\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "The Bayes factor for `B_uniform` is 0.47, which means that the data\n", - "provide evidence against this hypothesis, compared to $F$. In the\n", - "previous section I characterized this evidence as “weak,” but didn’t say\n", - "why.\n", - "\n", - "Part of the answer is historical. Harold Jeffreys, an early proponent of\n", - "Bayesian statistics, suggested a scale for interpreting Bayes factors:\n", - "\n", - " ---------- -------------------------\n", - " Bayes Strength\n", - " Factor \n", - " 1 – 3 Barely worth mentioning\n", - " 3 – 10 Substantial\n", - " 10 – 30 Strong\n", - " 30 – 100 Very strong\n", - " $>$ 100 Decisive\n", - " ---------- -------------------------\n", - "\n", - "In the example, the Bayes factor is 0.47 in favor of `B_uniform`, so it\n", - "is 2.1 in favor is $F$, which Jeffreys would consider “barely worth\n", - "mentioning.” Other authors have suggested variations on the wording. To\n", - "avoid arguing about adjectives, we could think about odds instead.\n", - "\n", - "If your prior odds are 1:1, and you see evidence with Bayes factor 2,\n", - "your posterior odds are 2:1. In terms of probability, the data changed\n", - "your degree of belief from 50% to 66%. For most real world problems,\n", - "that change would be small relative to modeling errors and other sources\n", - "of uncertainty.\n", - "\n", - "On the other hand, if you had seen evidence with Bayes factor 100, your\n", - "posterior odds would be 100:1 or more than 99%. Whether or not you agree\n", - "that such evidence is “decisive,” it is certainly strong.\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "Some people believe in the existence of extra-sensory perception (ESP);\n", - "for example, the ability of some people to guess the value of an unseen\n", - "playing card with probability better than chance.\n", - "\n", - "What is your prior degree of belief in this kind of ESP? Do you think it\n", - "is as likely to exist as not? Or are you more skeptical about it? Write\n", - "down your prior odds.\n", - "\n", - "Now compute the strength of the evidence it would take to convince you\n", - "that ESP is at least 50% likely to exist. What Bayes factor would be\n", - "needed to make you 90% sure that ESP exists?\n", - "\n", - "Suppose that your answer to the previous question is 1000; that is,\n", - "evidence with Bayes factor 1000 in favor of ESP would be sufficient to\n", - "change your mind.\n", - "\n", - "Now suppose that you read a paper in a respectable peer-reviewed\n", - "scientific journal that presents evidence with Bayes factor 1000 in\n", - "favor of ESP. Would that change your mind?\n", - "\n", - "If not, how do you resolve the apparent contradiction? You might find it\n", - "helpful to read about David Hume’s article, “Of Miracles,” at\n", - ".\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Hypothesis Testing\n", + "\n", + "## Back to the Euro problem\n", + "\n", + "In Section [euro] I presented a problem from MacKay’s `*Information\n", + "Theory, Inference, and Learning Algorithms*`:\n", + "\n", + "> A statistical statement appeared in \\`\\`The Guardian\" on Friday\n", + "> January 4, 2002:\n", + ">\n", + "> > When spun on edge 250 times, a Belgian one-euro coin came up heads\n", + "> > 140 times and tails 110. ‘It looks very suspicious to me,’ said\n", + "> > Barry Blight, a statistics lecturer at the London School of\n", + "> > Economics. ‘If the coin were unbiased, the chance of getting a\n", + "> > result as extreme as that would be less than 7%.’\n", + ">\n", + "> But do these data give evidence that the coin is biased rather than\n", + "> fair?\n", + "\n", + "We estimated the probability that the coin would land face up, but we\n", + "didn’t really answer MacKay’s question: Do the data give evidence that\n", + "the coin is biased?\n", + "\n", + "In Chapter [more] I proposed that data are in favor of a hypothesis if\n", + "the data are more likely under the hypothesis than under the alternative\n", + "or, equivalently, if the Bayes factor is greater than 1.\n", + "\n", + "In the Euro example, we have two hypotheses to consider: I’ll use $F$\n", + "for the hypothesis that the coin is fair and $B$ for the hypothesis that\n", + "it is biased.\n", + "\n", + "If the coin is fair, it is easy to compute the likelihood of the data,\n", + "$\\mathrm{p}(D|F)$. In fact, we already wrote the function\n", + "that does it.\n", + "\n", + "```python\n", + "def Likelihood(self, data, hypo):\n", + " x = hypo / 100.0\n", + " head, tails = data\n", + " like = x**heads * (1-x)**tails\n", + " return like\n", + "```\n", + "\n", + "To use it we can create a `Euro` suite and invoke\n", + "`Likelihood`:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import euro\n", + "from euro import Euro\n", + "\n", + "data = 'H' * 140 + 'T' * 110\n", + "\n", + "suite = Euro()\n", + "likelihood = suite.Likelihood(data, 50)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$\\mathrm{p}(D|F)$ is $5.5 \\cdot 10^{-76}$, which doesn’t\n", + "tell us much except that the probability of seeing any particular\n", + "dataset is very small. It takes two likelihoods to make a ratio, so we\n", + "also have to compute $\\mathrm{p}(D|B)$`.\n", + "\n", + "It is not obvious how to compute the likelihood of $B$, because it’s not\n", + "obvious what “biased” means.\n", + "\n", + "One possibility is to cheat and look at the data before we define the\n", + "hypothesis. In that case we would say that “biased” means that the\n", + "probability of heads is 140/250." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "actual_percent = 100.0 * 140 / 250\n", + "likelihood = suite.Likelihood(data, actual_percent)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This version of $B$ I call `B_cheat`; the likelihood of `b_cheat` is\n", + "$34 \\cdot 10^{-76}$ and the likelihood ratio is 6.1. So we would say\n", + "that the data are evidence in favor of this version of $B$.\n", + "\n", + "But using the data to formulate the hypothesis is obviously bogus. By\n", + "that definition, any dataset would be evidence in favor of $B$, unless\n", + "the observed percentage of heads is exactly 50%." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Making a fair comparison \n", + "\n", + "To make a legitimate comparison, we have to define $B$ without looking\n", + "at the data. So let’s try a different definition. If you inspect a\n", + "Belgian Euro coin, you might notice that the “heads” side is more\n", + "prominent than the “tails” side. You might expect the shape to have some\n", + "effect on $x$, but be unsure whether it makes heads more or less likely.\n", + "So you might say “I think the coin is biased so that $x$ is either 0.6\n", + "or 0.4, but I am not sure which.”\n", + "\n", + "We can think of this version, which I’ll call `B_two` as a hypothesis\n", + "made up of two sub-hypotheses. We can compute the likelihood for each\n", + "sub-hypothesis and then compute the average likelihood." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "like40 = suite.Likelihood(data, 40)\n", + "like60 = suite.Likelihood(data, 60)\n", + "likelihood = 0.5 * like40 + 0.5 * like60" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The likelihood ratio (or Bayes factor) for `b_two` is 1.3, which means\n", + "the data provide weak evidence in favor of `b_two`.\n", + "\n", + "More generally, suppose you suspect that the coin is biased, but you\n", + "have no clue about the value of $x$. In that case you might build a\n", + "Suite, which I call `b_uniform`, to represent sub-hypotheses from 0 to\n", + "100." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.9900990099009908" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "b_uniform = Euro(range(0, 101))\n", + "b_uniform.Remove(50)\n", + "b_uniform.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "I initialize `b_uniform` with values from 0 to 100. I removed the\n", + "sub-hypothesis that $x$ is 50%, because if $x$ is 50% the coin is fair,\n", + "but it has almost no effect on the result whether you remove it or not.\n", + "\n", + "To compute the likelihood of `b_uniform` we compute the likelihood of\n", + "each sub-hypothesis and accumulate a weighted average." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def SuiteLikelihood(suite, data):\n", + " total = 0\n", + " for hypo, prob in suite.Items():\n", + " like = suite.Likelihood(data, hypo)\n", + " total += prob * like\n", + " return total" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The likelihood ratio for `b_uniform` is 0.47, which means that the data\n", + "are weak evidence against `b_uniform`, compared to $F$.\n", + "\n", + "If you think about the computation performed by `SuiteLikelihood`, you\n", + "might notice that it is similar to an update. To refresh your memory,\n", + "here’s the `Update` function:\n", + "\n", + "```python\n", + "def Update(self, data):\n", + " for hypo in self.Values():\n", + " like = self.Likelihood(data, hypo)\n", + " self.Mult(hypo, like)\n", + " return self.Normalize()\n", + "```\n", + "\n", + "And here’s `Normalize`:\n", + "\n", + "```python\n", + "\n", + "def Normalize(self):\n", + " total = self.Total()\n", + "\n", + " factor = 1.0 / total\n", + " for x in self.d:\n", + " self.d[x] *= factor\n", + "\n", + " return total\n", + "```\n", + "\n", + "The return value from `Normalize` is the total of the\n", + "probabilities in the Suite, which is the average of the likelihoods for\n", + "the sub-hypotheses, weighted by the prior probabilities. And\n", + "`Update` passes this value along, so instead of using\n", + "`SuiteLikelihood`, we could compute the likelihood of\n", + "`b_uniform` like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.4999999999999996" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "likelihood = b_uniform.Update(data)\n", + "likelihood" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The triangle prior\n", + "\n", + "In Chapter [more] we also considered a triangle-shaped prior that gives\n", + "higher probability to values of $x$ near 50%. If we think of this prior\n", + "as a suite of sub-hypotheses, we can compute its likelihood like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from thinkbayes import Suite\n", + "\n", + "def TrianglePrior(suite):\n", + " for x in range(0, 51):\n", + " suite.Set(x, x)\n", + " for x in range(51, 101):\n", + " suite.Set(x, 100-x) \n", + " suite.Normalize()\n", + " return suite" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.49999999999999994" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "suite = Euro(range(0, 101))\n", + "\n", + "b_triangle = TrianglePrior(suite)\n", + "likelihood = b_triangle.Update(data)\n", + "likelihood" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The likelihood ratio for `b_triangle` is 0.84, compared to $F$, so again\n", + "we would say that the data are weak evidence against $B$.\n", + "\n", + "The following table shows the priors we have considered, the likelihood\n", + "of each, and the likelihood ratio (or Bayes factor) relative to $F$.\n", + "\n", + " -------------- ------------------- --------\n", + " Hypothesis Likelihood Bayes\n", + " $\\times 10^{-76}$ Factor\n", + " $F$ 5.5 –\n", + " `B_cheat` 34 6.1\n", + " `B_two` 7.4 1.3\n", + " `B_uniform` 2.6 0.47\n", + " `B_triangle` 4.6 0.84\n", + " -------------- ------------------- --------\n", + "\n", + "Depending on which definition we choose, the data might provide evidence\n", + "for or against the hypothesis that the coin is biased, but in either\n", + "case it is relatively weak evidence.\n", + "\n", + "In summary, we can use Bayesian hypothesis testing to compare the\n", + "likelihood of $F$ and $B$, but we have to do some work to specify\n", + "precisely what $B$ means. This specification depends on background\n", + "information about coins and their behavior when spun, so people could\n", + "reasonably disagree about the right definition.\n", + "\n", + "My presentation of this example follows David MacKay’s discussion, and\n", + "comes to the same conclusion. You can download the code I used in this\n", + "chapter from . For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "The Bayes factor for `B_uniform` is 0.47, which means that the data\n", + "provide evidence against this hypothesis, compared to $F$. In the\n", + "previous section I characterized this evidence as “weak,” but didn’t say\n", + "why.\n", + "\n", + "Part of the answer is historical. Harold Jeffreys, an early proponent of\n", + "Bayesian statistics, suggested a scale for interpreting Bayes factors:\n", + "\n", + " ---------- -------------------------\n", + " Bayes Strength\n", + " Factor \n", + " 1 – 3 Barely worth mentioning\n", + " 3 – 10 Substantial\n", + " 10 – 30 Strong\n", + " 30 – 100 Very strong\n", + " $>$ 100 Decisive\n", + " ---------- -------------------------\n", + "\n", + "In the example, the Bayes factor is 0.47 in favor of `B_uniform`, so it\n", + "is 2.1 in favor is $F$, which Jeffreys would consider “barely worth\n", + "mentioning.” Other authors have suggested variations on the wording. To\n", + "avoid arguing about adjectives, we could think about odds instead.\n", + "\n", + "If your prior odds are 1:1, and you see evidence with Bayes factor 2,\n", + "your posterior odds are 2:1. In terms of probability, the data changed\n", + "your degree of belief from 50% to 66%. For most real world problems,\n", + "that change would be small relative to modeling errors and other sources\n", + "of uncertainty.\n", + "\n", + "On the other hand, if you had seen evidence with Bayes factor 100, your\n", + "posterior odds would be 100:1 or more than 99%. Whether or not you agree\n", + "that such evidence is “decisive,” it is certainly strong." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "Some people believe in the existence of extra-sensory perception (ESP);\n", + "for example, the ability of some people to guess the value of an unseen\n", + "playing card with probability better than chance.\n", + "\n", + "What is your prior degree of belief in this kind of ESP? Do you think it\n", + "is as likely to exist as not? Or are you more skeptical about it? Write\n", + "down your prior odds.\n", + "\n", + "Now compute the strength of the evidence it would take to convince you\n", + "that ESP is at least 50% likely to exist. What Bayes factor would be\n", + "needed to make you 90% sure that ESP exists?\n", + "\n", + "Suppose that your answer to the previous question is 1000; that is,\n", + "evidence with Bayes factor 1000 in favor of ESP would be sufficient to\n", + "change your mind.\n", + "\n", + "Now suppose that you read a paper in a respectable peer-reviewed\n", + "scientific journal that presents evidence with Bayes factor 1000 in\n", + "favor of ESP. Would that change your mind?\n", + "\n", + "If not, how do you resolve the apparent contradiction? You might find it\n", + "helpful to read about David Hume’s article, “Of Miracles,” at\n", + "." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/12_Evidence.ipynb b/notebooks/12_Evidence.ipynb index 97bf6fd..2416da8 100644 --- a/notebooks/12_Evidence.ipynb +++ b/notebooks/12_Evidence.ipynb @@ -1,555 +1,996 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Evidence\n", - "========\n", - "\n", - "Interpreting SAT scores\n", - "-----------------------\n", - "\n", - "Suppose you are the Dean of Admission at a small engineering college in\n", - "Massachusetts, and you are considering two candidates, Alice and Bob,\n", - "whose qualifications are similar in many ways, with the exception that\n", - "Alice got a higher score on the Math portion of the SAT, a standardized\n", - "test intended to measure preparation for college-level work in\n", - "mathematics.\n", - "\n", - "If Alice got 780 and Bob got a 740 (out of a possible 800), you might\n", - "want to know whether that difference is evidence that Alice is better\n", - "prepared than Bob, and what the strength of that evidence is.\n", - "\n", - "Now in reality, both scores are very good, and both candidates are\n", - "probably well prepared for college math. So the real Dean of Admission\n", - "would probably suggest that we choose the candidate who best\n", - "demonstrates the other skills and attitudes we look for in students. But\n", - "as an example of Bayesian hypothesis testing, let’s stick with a\n", - "narrower question: “How strong is the evidence that Alice is better\n", - "prepared than Bob?”\n", - "\n", - "To answer that question, we need to make some modeling decisions. I’ll\n", - "start with a simplification I know is wrong; then we’ll come back and\n", - "improve the model. I pretend, temporarily, that all SAT questions are\n", - "equally difficult. Actually, the designers of the SAT choose questions\n", - "with a range of difficulty, because that improves the ability to measure\n", - "statistical differences between test-takers.\n", - "\n", - "But if we choose a model where all questions are equally difficult, we\n", - "can define a characteristic, `p_correct`, for each test-taker, which is\n", - "the probability of answering any question correctly. This simplification\n", - "makes it easy to compute the likelihood of a given score.\n", - "\n", - "The scale\n", - "---------\n", - "\n", - "In order to understand SAT scores, we have to understand the scoring and\n", - "scaling process. Each test-taker gets a raw score based on the number of\n", - "correct and incorrect questions. The raw score is converted to a scaled\n", - "score in the range 200–800.\n", - "\n", - "In 2009, there were 54 questions on the math SAT. The raw score for each\n", - "test-taker is the number of questions answered correctly minus a penalty\n", - "of $1/4$ point for each question answered incorrectly.\n", - "\n", - "The College Board, which administers the SAT, publishes the map from raw\n", - "scores to scaled scores. I have downloaded that data and wrapped it in\n", - "an Interpolator object that provides a forward lookup (from raw score to\n", - "scaled) and a reverse lookup (from scaled score to raw).\n", - "\n", - "You can download the code for this example from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "The prior\n", - "---------\n", - "\n", - "The College Board also publishes the distribution of scaled scores for\n", - "all test-takers. If we convert each scaled score to a raw score, and\n", - "divide by the number of questions, the result is an estimate of\n", - "`p_correct`. So we can use the distribution of raw scores to model the\n", - "prior distribution of `p_correct`.\n", - "\n", - "Here is the code that reads and processes the data:\n", - "\n", - " class Exam(object):\n", - "\n", - " def __init__(self):\n", - " self.scale = ReadScale()\n", - " scores = ReadRanks()\n", - " score_pmf = thinkbayes.MakePmfFromDict(dict(scores))\n", - " self.raw = self.ReverseScale(score_pmf)\n", - " self.prior = DivideValues(raw, 54)\n", - "\n", - "Exam encapsulates the information we have about the exam.\n", - "ReadScale and ReadRanks read files and return\n", - "objects that contain the data: self.scale is the\n", - "Interpolator that converts from raw to scaled scores and\n", - "back; scores is a list of (score, frequency) pairs.\n", - "\n", - "`score_pmf` is the Pmf of scaled scores. self.raw is the\n", - "Pmf of raw scores, and self.prior is the Pmf of\n", - "`p_correct`.\n", - "\n", - "![Prior distribution of p\\_correct for SAT\n", - "test-takers.](figs/sat_prior.pdf)\n", - "\n", - "[fig.satprior]\n", - "\n", - "Figure [fig.satprior] shows the prior distribution of `p_correct`. This\n", - "distribution is approximately Gaussian, but it is compressed at the\n", - "extremes. By design, the SAT has the most power to discriminate between\n", - "test-takers within two standard deviations of the mean, and less power\n", - "outside that range.\n", - "\n", - "For each test-taker, I define a Suite called Sat that\n", - "represents the distribution of `p_correct`. Here’s the definition:\n", - "\n", - " class Sat(thinkbayes.Suite):\n", - "\n", - " def __init__(self, exam, score):\n", - " thinkbayes.Suite.__init__(self)\n", - "\n", - " self.exam = exam\n", - " self.score = score\n", - "\n", - " # start with the prior distribution\n", - " for p_correct, prob in exam.prior.Items():\n", - " self.Set(p_correct, prob)\n", - "\n", - " # update based on an exam score\n", - " self.Update(score)\n", - "\n", - "`__init__` takes an Exam object and a scaled score. It makes a copy of\n", - "the prior distribution and then updates itself based on the exam score.\n", - "\n", - "As usual, we inherit Update from Suite and\n", - "provide Likelihood:\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " p_correct = hypo\n", - " score = data\n", - "\n", - " k = self.exam.Reverse(score)\n", - " n = self.exam.max_score\n", - " like = thinkbayes.EvalBinomialPmf(k, n, p_correct)\n", - " return like\n", - "\n", - "hypo is a hypothetical value of `p_correct`, and\n", - "data is a scaled score.\n", - "\n", - "To keep things simple, I interpret the raw score as the number of\n", - "correct answers, ignoring the penalty for wrong answers. With this\n", - "simplification, the likelihood is given by the binomial distribution,\n", - "which computes the probability of $k$ correct responses out of $n$\n", - "questions.\n", - "\n", - "Posterior\n", - "---------\n", - "\n", - "![Posterior distributions of p\\_correct for Alice and\n", - "Bob.](figs/sat_posteriors_p_corr.pdf)\n", - "\n", - "[fig.satposterior1]\n", - "\n", - "Figure [fig.satposterior1] shows the posterior distributions of\n", - "`p_correct` for Alice and Bob based on their exam scores. We can see\n", - "that they overlap, so it is possible that `p_correct` is actually higher\n", - "for Bob, but it seems unlikely.\n", - "\n", - "Which brings us back to the original question, “How strong is the\n", - "evidence that Alice is better prepared than Bob?” We can use the\n", - "posterior distributions of `p_correct` to answer this question.\n", - "\n", - "To formulate the question in terms of Bayesian hypothesis testing, I\n", - "define two hypotheses:\n", - "\n", - "- $A$: `p_correct` is higher for Alice than for Bob.\n", - "\n", - "- $B$: `p_correct` is higher for Bob than for Alice.\n", - "\n", - "To compute the likelihood of $A$, we can enumerate all pairs of values\n", - "from the posterior distributions and add up the total probability of the\n", - "cases where `p_correct` is higher for Alice than for Bob. And we already\n", - "have a function, `thinkbayes.PmfProbGreater`, that does that.\n", - "\n", - "So we can define a Suite that computes the posterior probabilities of\n", - "$A$ and $B$:\n", - "\n", - " class TopLevel(thinkbayes.Suite):\n", - "\n", - " def Update(self, data):\n", - " a_sat, b_sat = data\n", - "\n", - " a_like = thinkbayes.PmfProbGreater(a_sat, b_sat)\n", - " b_like = thinkbayes.PmfProbLess(a_sat, b_sat)\n", - " c_like = thinkbayes.PmfProbEqual(a_sat, b_sat)\n", - "\n", - " a_like += c_like / 2\n", - " b_like += c_like / 2\n", - "\n", - " self.Mult('A', a_like)\n", - " self.Mult('B', b_like)\n", - "\n", - " self.Normalize()\n", - "\n", - "Usually when we define a new Suite, we inherit Update and\n", - "provide Likelihood. In this case I override\n", - "Update, because it is easier to evaluate the likelihood of\n", - "both hypotheses at the same time.\n", - "\n", - "The data passed to Update are Sat objects that represent\n", - "the posterior distributions of `p_correct`.\n", - "\n", - "`a_like` is the total probability that `p_correct` is higher for Alice;\n", - "`b_like` is that probability that it is higher for Bob.\n", - "\n", - "`c_like` is the probability that they are “equal,” but this equality is\n", - "an artifact of the decision to model `p_correct` with a set of discrete\n", - "values. If we use more values, `c_like` is smaller, and in the extreme,\n", - "if `p_correct` is continuous, `c_like` is zero. So I treat `c_like` as a\n", - "kind of round-off error and split it evenly between `a_like` and\n", - "`b_like`.\n", - "\n", - "Here is the code that creates TopLevel and updates it:\n", - "\n", - " exam = Exam()\n", - " a_sat = Sat(exam, 780)\n", - " b_sat = Sat(exam, 740)\n", - "\n", - " top = TopLevel('AB')\n", - " top.Update((a_sat, b_sat))\n", - " top.Print()\n", - "\n", - "The likelihood of $A$ is 0.79 and the likelihood of $B$ is 0.21. The\n", - "likelihood ratio (or Bayes factor) is 3.8, which means that these test\n", - "scores are evidence that Alice is better than Bob at answering SAT\n", - "questions. If we believed, before seeing the test scores, that $A$ and\n", - "$B$ were equally likely, then after seeing the scores we should believe\n", - "that the probability of $A$ is 79%, which means there is still a 21%\n", - "chance that Bob is actually better prepared.\n", - "\n", - "A better model\n", - "--------------\n", - "\n", - "Remember that the analysis we have done so far is based on the\n", - "simplification that all SAT questions are equally difficult. In reality,\n", - "some are easier than others, which means that the difference between\n", - "Alice and Bob might be even smaller.\n", - "\n", - "But how big is the modeling error? If it is small, we conclude that the\n", - "first model—based on the simplification that all questions are equally\n", - "difficult—is good enough. If it’s large, we need a better model.\n", - "\n", - "In the next few sections, I develop a better model and discover (spoiler\n", - "alert!) that the modeling error is small. So if you are satisfied with\n", - "the simple mode, you can skip to the next chapter. If you want to see\n", - "how the more realistic model works, read on...\n", - "\n", - "- Assume that each test-taker has some degree of\n", - " efficacy, which measures their ability to answer SAT\n", - " questions.\n", - "\n", - "- Assume that each question has some level of difficulty.\n", - "\n", - "- Finally, assume that the chance that a test-taker answers a question\n", - " correctly is related to efficacy and\n", - " difficulty according to this function:\n", - "\n", - " def ProbCorrect(efficacy, difficulty, a=1):\n", - " return 1 / (1 + math.exp(-a * (efficacy - difficulty)))\n", - "\n", - "This function is a simplified version of the curve used in **item\n", - "response theory**, which you can read about at\n", - ".\n", - "efficacy and difficulty are considered to be\n", - "on the same scale, and the probability of getting a question right\n", - "depends only on the difference between them.\n", - "\n", - "When efficacy and difficulty are equal, the\n", - "probability of getting the question right is 50%. As\n", - "efficacy increases, this probability approaches 100%. As it\n", - "decreases (or as difficulty increases), the probability\n", - "approaches 0%.\n", - "\n", - "Given the distribution of efficacy across test-takers and\n", - "the distribution of difficulty across questions, we can\n", - "compute the expected distribution of raw scores. We’ll do that in two\n", - "steps. First, for a person with given efficacy, we’ll\n", - "compute the distribution of raw scores.\n", - "\n", - " def PmfCorrect(efficacy, difficulties):\n", - " pmf0 = thinkbayes.Pmf([0])\n", - "\n", - " ps = [ProbCorrect(efficacy, diff) for diff in difficulties]\n", - " pmfs = [BinaryPmf(p) for p in ps]\n", - " dist = sum(pmfs, pmf0)\n", - " return dist\n", - "\n", - "difficulties is a list of difficulties, one for each\n", - "question. ps is a list of probabilities, and\n", - "pmfs is a list of two-valued Pmf objects; here’s the\n", - "function that makes them:\n", - "\n", - " def BinaryPmf(p):\n", - " pmf = thinkbayes.Pmf()\n", - " pmf.Set(1, p)\n", - " pmf.Set(0, 1-p)\n", - " return pmf\n", - "\n", - "dist is the sum of these Pmfs. Remember from\n", - "Section [addends] that when we add up Pmf objects, the result is the\n", - "distribution of the sums. In order to use Python’s sum to\n", - "add up Pmfs, we have to provide pmf0 which is the identity\n", - "for Pmfs, so pmf + pmf0 is always pmf.\n", - "\n", - "If we know a person’s efficacy, we can compute their distribution of raw\n", - "scores. For a group of people with a different efficacies, the resulting\n", - "distribution of raw scores is a mixture. Here’s the code that computes\n", - "the mixture:\n", - "\n", - " # class Exam:\n", - "\n", - " def MakeRawScoreDist(self, efficacies):\n", - " pmfs = thinkbayes.Pmf()\n", - " for efficacy, prob in efficacies.Items():\n", - " scores = PmfCorrect(efficacy, self.difficulties)\n", - " pmfs.Set(scores, prob)\n", - "\n", - " mix = thinkbayes.MakeMixture(pmfs)\n", - " return mix\n", - "\n", - "MakeRawScoreDist takes efficacies, which is a\n", - "Pmf that represents the distribution of efficacy across test-takers. I\n", - "assume it is Gaussian with mean 0 and standard deviation 1.5. This\n", - "choice is mostly arbitrary. The probability of getting a question\n", - "correct depends on the difference between efficacy and difficulty, so we\n", - "can choose the units of efficacy and then calibrate the units of\n", - "difficulty accordingly.\n", - "\n", - "pmfs is a meta-Pmf that contains one Pmf for each level of\n", - "efficacy, and maps to the fraction of test-takers at that level.\n", - "MakeMixture takes the meta-pmf and computes the\n", - "distribution of the mixture (see Section [mixture]).\n", - "\n", - "Calibration\n", - "-----------\n", - "\n", - "If we were given the distribution of difficulty, we could use\n", - "`MakeRawScoreDist` to compute the distribution of raw scores. But for us\n", - "the problem is the other way around: we are given the distribution of\n", - "raw scores and we want to infer the distribution of difficulty.\n", - "\n", - "![Actual distribution of raw scores and a model to fit\n", - "it.](figs/sat_calibrate.pdf)\n", - "\n", - "[fig.satcalibrate]\n", - "\n", - "I assume that the distribution of difficulty is uniform with parameters\n", - "center and width.\n", - "MakeDifficulties makes a list of difficulties with these\n", - "parameters.\n", - "\n", - " def MakeDifficulties(center, width, n):\n", - " low, high = center-width, center+width\n", - " return numpy.linspace(low, high, n)\n", - "\n", - "By trying out a few combinations, I found that center=-0.05\n", - "and width=1.8 yield a distribution of raw scores similar to\n", - "the actual data, as shown in Figure [fig.satcalibrate].\n", - "\n", - "So, assuming that the distribution of difficulty is uniform, its range\n", - "is approximately -1.85 to 1.75, given that\n", - "efficacy is Gaussian with mean 0 and standard deviation 1.5.\n", - "\n", - "The following table shows the range of ProbCorrect for\n", - "test-takers at different levels of efficacy:\n", - "\n", - "|r|r|r|r| &\\\n", - "Efficacy & -1.85 & -0.05 & 1.75\\\n", - "3.00 & 0.99 & 0.95 & 0.78\\\n", - "1.50 & 0.97 & 0.82 & 0.44\\\n", - "0.00 & 0.86 & 0.51 & 0.15\\\n", - "-1.50 & 0.59 & 0.19 & 0.04\\\n", - "-3.00 & 0.24 & 0.05 & 0.01\\\n", - "\n", - "Someone with efficacy 3 (two standard deviations above the mean) has a\n", - "99% chance of answering the easiest questions on the exam, and a 78%\n", - "chance of answering the hardest. On the other end of the range, someone\n", - "two standard deviations below the mean has only a 24% chance of\n", - "answering the easiest questions.\n", - "\n", - "Posterior distribution of efficacy\n", - "----------------------------------\n", - "\n", - "![Posterior distributions of efficacy for Alice and\n", - "Bob.](figs/sat_posteriors_eff.pdf)\n", - "\n", - "[fig.satposterior2]\n", - "\n", - "Now that the model is calibrated, we can compute the posterior\n", - "distribution of efficacy for Alice and Bob. Here is a version of the Sat\n", - "class that uses the new model:\n", - "\n", - " class Sat2(thinkbayes.Suite):\n", - "\n", - " def __init__(self, exam, score):\n", - " self.exam = exam\n", - " self.score = score\n", - "\n", - " # start with the Gaussian prior\n", - " efficacies = thinkbayes.MakeGaussianPmf(0, 1.5, 3)\n", - " thinkbayes.Suite.__init__(self, efficacies)\n", - "\n", - " # update based on an exam score\n", - " self.Update(score)\n", - "\n", - "`Update` invokes `Likelihood`, which computes the likelihood of a given\n", - "test score for a hypothetical level of efficacy.\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " efficacy = hypo\n", - " score = data\n", - " raw = self.exam.Reverse(score)\n", - "\n", - " pmf = self.exam.PmfCorrect(efficacy)\n", - " like = pmf.Prob(raw)\n", - " return like\n", - "\n", - "pmf is the distribution of raw scores for a test-taker with\n", - "the given efficacy; like is the probability of the observed\n", - "score.\n", - "\n", - "Figure [fig.satposterior2] shows the posterior distributions of efficacy\n", - "for Alice and Bob. As expected, the location of Alice’s distribution is\n", - "farther to the right, but again there is some overlap.\n", - "\n", - "Using TopLevel again, we compare $A$, the hypothesis that\n", - "Alice’s efficacy is higher, and $B$, the hypothesis that Bob’s is\n", - "higher. The likelihood ratio is 3.4, a bit smaller than what we got from\n", - "the simple model (3.8). So this model indicates that the data are\n", - "evidence in favor of $A$, but a little weaker than the previous\n", - "estimate.\n", - "\n", - "If our prior belief is that $A$ and $B$ are equally likely, then in\n", - "light of this evidence we would give $A$ a posterior probability of 77%,\n", - "leaving a 23% chance that Bob’s efficacy is higher.\n", - "\n", - "Predictive distribution\n", - "-----------------------\n", - "\n", - "The analysis we have done so far generates estimates for Alice and Bob’s\n", - "efficacy, but since efficacy is not directly observable, it is hard to\n", - "validate the results.\n", - "\n", - "To give the model predictive power, we can use it to answer a related\n", - "question: “If Alice and Bob take the math SAT again, what is the chance\n", - "that Alice will do better again?”\n", - "\n", - "We’ll answer this question in two steps:\n", - "\n", - "- We’ll use the posterior distribution of efficacy to generate a\n", - " predictive distribution of raw score for each test-taker.\n", - "\n", - "- We’ll compare the two predictive distributions to compute the\n", - " probability that Alice gets a higher score again.\n", - "\n", - "We already have most of the code we need. To compute the predictive\n", - "distributions, we can use `MakeRawScoreDist` again:\n", - "\n", - " exam = Exam()\n", - " a_sat = Sat(exam, 780)\n", - " b_sat = Sat(exam, 740)\n", - "\n", - " a_pred = exam.MakeRawScoreDist(a_sat)\n", - " b_pred = exam.MakeRawScoreDist(b_sat)\n", - "\n", - "Then we can find the likelihood that Alice does better on the second\n", - "test, Bob does better, or they tie:\n", - "\n", - " a_like = thinkbayes.PmfProbGreater(a_pred, b_pred)\n", - " b_like = thinkbayes.PmfProbLess(a_pred, b_pred)\n", - " c_like = thinkbayes.PmfProbEqual(a_pred, b_pred)\n", - "\n", - "The probability that Alice does better on the second exam is 63%, which\n", - "means that Bob has a 37% chance of doing as well or better.\n", - "\n", - "Notice that we have more confidence about Alice’s efficacy than we do\n", - "about the outcome of the next test. The posterior odds are 3:1 that\n", - "Alice’s efficacy is higher, but only 2:1 that Alice will do better on\n", - "the next exam.\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "![Joint posterior distribution of p\\_correct for Alice and\n", - "Bob.](figs/sat_joint.pdf)\n", - "\n", - "[fig.satjoint]\n", - "\n", - "We started this chapter with the question, “How strong is the evidence\n", - "that Alice is better prepared than Bob?” On the face of it, that sounds\n", - "like we want to test two hypotheses: either Alice is more prepared or\n", - "Bob is.\n", - "\n", - "But in order to compute likelihoods for these hypotheses, we have to\n", - "solve an estimation problem. For each test-taker we have to find the\n", - "posterior distribution of either `p_correct` or `efficacy`.\n", - "\n", - "Values like this are called **nuisance parameters** because\n", - "we don’t care what they are, but we have to estimate them to answer the\n", - "question we care about.\n", - "\n", - "One way to visualize the analysis we did in this chapter is to plot the\n", - "space of these parameters. `thinkbayes.MakeJoint` takes two Pmfs,\n", - "computes their joint distribution, and returns a joint pmf of each\n", - "possible pair of values and its probability.\n", - "\n", - " def MakeJoint(pmf1, pmf2):\n", - " joint = Joint()\n", - " for v1, p1 in pmf1.Items():\n", - " for v2, p2 in pmf2.Items():\n", - " joint.Set((v1, v2), p1 * p2)\n", - " return joint\n", - "\n", - "This function assumes that the two distributions are independent.\n", - "\n", - "Figure [fig.satjoint] shows the joint posterior distribution of\n", - "`p_correct` for Alice and Bob. The diagonal line indicates the part of\n", - "the space where `p_correct` is the same for Alice and Bob. To the right\n", - "of this line, Alice is more prepared; to the left, Bob is more prepared.\n", - "\n", - "In TopLevel.Update, when we compute the likelihoods of $A$\n", - "and $B$, we add up the probability mass on each side of this line. For\n", - "the cells that fall on the line, we add up the total mass and split it\n", - "between $A$ and $B$.\n", - "\n", - "The process we used in this chapter—estimating nuisance parameters in\n", - "order to evaluate the likelihood of competing hypotheses—is a common\n", - "Bayesian approach to problems like this.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Evidence\n", + "\n", + "## Interpreting SAT scores\n", + "\n", + "Suppose you are the Dean of Admission at a small engineering college in\n", + "Massachusetts, and you are considering two candidates, Alice and Bob,\n", + "whose qualifications are similar in many ways, with the exception that\n", + "Alice got a higher score on the Math portion of the SAT, a standardized\n", + "test intended to measure preparation for college-level work in\n", + "mathematics.\n", + "\n", + "If Alice got 780 and Bob got a 740 (out of a possible 800), you might\n", + "want to know whether that difference is evidence that Alice is better\n", + "prepared than Bob, and what the strength of that evidence is.\n", + "\n", + "Now in reality, both scores are very good, and both candidates are\n", + "probably well prepared for college math. So the real Dean of Admission\n", + "would probably suggest that we choose the candidate who best\n", + "demonstrates the other skills and attitudes we look for in students. But\n", + "as an example of Bayesian hypothesis testing, let’s stick with a\n", + "narrower question: “How strong is the evidence that Alice is better\n", + "prepared than Bob?”\n", + "\n", + "To answer that question, we need to make some modeling decisions. I’ll\n", + "start with a simplification I know is wrong; then we’ll come back and\n", + "improve the model. I pretend, temporarily, that all SAT questions are\n", + "equally difficult. Actually, the designers of the SAT choose questions\n", + "with a range of difficulty, because that improves the ability to measure\n", + "statistical differences between test-takers.\n", + "\n", + "But if we choose a model where all questions are equally difficult, we\n", + "can define a characteristic, `p_correct`, for each test-taker, which is\n", + "the probability of answering any question correctly. This simplification\n", + "makes it easy to compute the likelihood of a given score." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The scale\n", + "\n", + "In order to understand SAT scores, we have to understand the scoring and\n", + "scaling process. Each test-taker gets a raw score based on the number of\n", + "correct and incorrect questions. The raw score is converted to a scaled\n", + "score in the range 200–800.\n", + "\n", + "In 2009, there were 54 questions on the math SAT. The raw score for each\n", + "test-taker is the number of questions answered correctly minus a penalty\n", + "of $1/4$ point for each question answered incorrectly.\n", + "\n", + "The College Board, which administers the SAT, publishes the map from raw\n", + "scores to scaled scores. I have downloaded that data and wrapped it in\n", + "an Interpolator object that provides a forward lookup (from raw score to\n", + "scaled) and a reverse lookup (from scaled score to raw).\n", + "\n", + "You can download the code for this example from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The prior\n", + "\n", + "The College Board also publishes the distribution of scaled scores for\n", + "all test-takers. If we convert each scaled score to a raw score, and\n", + "divide by the number of questions, the result is an estimate of\n", + "`p_correct`. So we can use the distribution of raw scores to model the\n", + "prior distribution of `p_correct`.\n", + "\n", + "Here is the code that reads and processes the data:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Exam(object):\n", + "\n", + " def __init__(self):\n", + " self.scale = ReadScale()\n", + " scores = ReadRanks()\n", + " score_pmf = thinkbayes.MakePmfFromDict(dict(scores))\n", + " self.raw = self.ReverseScale(score_pmf)\n", + " self.prior = DivideValues(raw, 54)\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`Exam` encapsulates the information we have about the exam.\n", + "`ReadScale` and `ReadRanks` read files and return\n", + "objects that contain the data: `self.scale` is the\n", + "`Interpolator` that converts from raw to scaled scores and\n", + "back; `scores` is a list of (score, frequency) pairs.\n", + "\n", + "`score_pmf` is the Pmf of scaled scores. `self.raw` is the\n", + "Pmf of raw scores, and `self.prior` is the Pmf of\n", + "`p_correct`.\n", + "\n", + "![Prior distribution of `p\\_correct` for SAT\n", + "test-takers.](figs/sat_prior.pdf)\n", + "\n", + "[fig.satprior]\n", + "\n", + "Figure [fig.satprior] shows the prior distribution of `p_correct`. This\n", + "distribution is approximately Gaussian, but it is compressed at the\n", + "extremes. By design, the SAT has the most power to discriminate between\n", + "test-takers within two standard deviations of the mean, and less power\n", + "outside that range.\n", + "\n", + "For each test-taker, I define a Suite called `Sat` that\n", + "represents the distribution of `p_correct`. Here’s the definition:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "```python\n", + "class Sat(thinkbayes.Suite):\n", + "\n", + " def __init__(self, exam, score):\n", + " thinkbayes.Suite.__init__(self)\n", + "\n", + " self.exam = exam\n", + " self.score = score\n", + "\n", + " # start with the prior distribution\n", + " for p_correct, prob in exam.prior.Items():\n", + " self.Set(p_correct, prob)\n", + "\n", + " # update based on an exam score\n", + " self.Update(score)\n", + " ```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`__init__` takes an Exam object and a scaled score. It makes a copy of\n", + "the prior distribution and then updates itself based on the exam score.\n", + "\n", + "As usual, we inherit `Update` from `Suite` and\n", + "provide `Likelihood`:\n", + "```python\n", + " def Likelihood(self, data, hypo):\n", + " p_correct = hypo\n", + " score = data\n", + "\n", + " k = self.exam.Reverse(score)\n", + " n = self.exam.max_score\n", + " like = thinkbayes.EvalBinomialPmf(k, n, p_correct)\n", + " return like\n", + "```\n", + "\n", + "`hypo` is a hypothetical value of `p_correct`, and\n", + "`data` is a scaled score.\n", + "\n", + "To keep things simple, I interpret the raw score as the number of\n", + "correct answers, ignoring the penalty for wrong answers. With this\n", + "simplification, the likelihood is given by the binomial distribution,\n", + "which computes the probability of $k$ correct responses out of $n$\n", + "questions." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Posterior\n", + "\n", + "![Posterior distributions of `p\\_correct` for Alice and\n", + "Bob.](figs/sat_posteriors_p_corr.pdf)\n", + "\n", + "[fig.satposterior1]\n", + "\n", + "Figure [fig.satposterior1] shows the posterior distributions of\n", + "`p_correct` for Alice and Bob based on their exam scores. We can see\n", + "that they overlap, so it is possible that `p_correct` is actually higher\n", + "for Bob, but it seems unlikely.\n", + "\n", + "Which brings us back to the original question, “How strong is the\n", + "evidence that Alice is better prepared than Bob?” We can use the\n", + "posterior distributions of `p_correct` to answer this question.\n", + "\n", + "To formulate the question in terms of Bayesian hypothesis testing, I\n", + "define two hypotheses:\n", + "\n", + "- $A$: `p_correct` is higher for Alice than for Bob.\n", + "\n", + "- $B$: `p_correct` is higher for Bob than for Alice.\n", + "\n", + "To compute the likelihood of $A$, we can enumerate all pairs of values\n", + "from the posterior distributions and add up the total probability of the\n", + "cases where `p_correct` is higher for Alice than for Bob. And we already\n", + "have a function, `thinkbayes.PmfProbGreater`, that does that.\n", + "\n", + "So we can define a Suite that computes the posterior probabilities of\n", + "$A$ and $B$:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import thinkbayes\n", + "\n", + "class TopLevel(thinkbayes.Suite):\n", + "\n", + " def Update(self, data):\n", + " a_sat, b_sat = data\n", + "\n", + " a_like = thinkbayes.PmfProbGreater(a_sat, b_sat)\n", + " b_like = thinkbayes.PmfProbLess(a_sat, b_sat)\n", + " c_like = thinkbayes.PmfProbEqual(a_sat, b_sat)\n", + "\n", + " a_like += c_like / 2\n", + " b_like += c_like / 2\n", + "\n", + " self.Mult('A', a_like)\n", + " self.Mult('B', b_like)\n", + "\n", + " self.Normalize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Usually when we define a new Suite, we inherit `Update` and\n", + "provide `Likelihood`. In this case I override\n", + "`Update`, because it is easier to evaluate the likelihood of\n", + "both hypotheses at the same time.\n", + "\n", + "The data passed to `Update` are Sat objects that represent\n", + "the posterior distributions of `p_correct`.\n", + "\n", + "`a_like` is the total probability that `p_correct` is higher for Alice;\n", + "`b_like` is that probability that it is higher for Bob.\n", + "\n", + "`c_like` is the probability that they are “equal,” but this equality is\n", + "an artifact of the decision to model `p_correct` with a set of discrete\n", + "values. If we use more values, `c_like` is smaller, and in the extreme,\n", + "if `p_correct` is continuous, `c_like` is zero. So I treat `c_like` as a\n", + "kind of round-off error and split it evenly between `a_like` and\n", + "`b_like`.\n", + "\n", + "Here is the code that creates `TopLevel` and updates it:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A 0.793123245041\n", + "B 0.206876754959\n" + ] + } + ], + "source": [ + "from sat import Exam, Sat\n", + "\n", + "exam = Exam()\n", + "a_sat = Sat(exam, 780)\n", + "b_sat = Sat(exam, 740)\n", + "\n", + "top = TopLevel('AB')\n", + "top.Update((a_sat, b_sat))\n", + "top.Print()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The likelihood of $A$ is 0.79 and the likelihood of $B$ is 0.21. The\n", + "likelihood ratio (or Bayes factor) is 3.8, which means that these test\n", + "scores are evidence that Alice is better than Bob at answering SAT\n", + "questions. If we believed, before seeing the test scores, that $A$ and\n", + "$B$ were equally likely, then after seeing the scores we should believe\n", + "that the probability of $A$ is 79%, which means there is still a 21%\n", + "chance that Bob is actually better prepared." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## A better model\n", + "\n", + "Remember that the analysis we have done so far is based on the\n", + "simplification that all SAT questions are equally difficult. In reality,\n", + "some are easier than others, which means that the difference between\n", + "Alice and Bob might be even smaller.\n", + "\n", + "But how big is the modeling error? If it is small, we conclude that the\n", + "first model—based on the simplification that all questions are equally\n", + "difficult—is good enough. If it’s large, we need a better model.\n", + "\n", + "In the next few sections, I develop a better model and discover (spoiler\n", + "alert!) that the modeling error is small. So if you are satisfied with\n", + "the simple mode, you can skip to the next chapter. If you want to see\n", + "how the more realistic model works, read on...\n", + "\n", + "- Assume that each test-taker has some degree of\n", + " `efficacy`, which measures their ability to answer SAT\n", + " questions.\n", + "\n", + "- Assume that each question has some level of `difficulty`.\n", + "\n", + "- Finally, assume that the chance that a test-taker answers a question\n", + " correctly is related to `efficacy` and\n", + " `difficulty` according to this function:\n", + "\n", + " def ProbCorrect(efficacy, difficulty, a=1):\n", + " return 1 / (1 + math.exp(-a * (efficacy - difficulty)))\n", + "\n", + "This function is a simplified version of the curve used in `**item\n", + "response theory**`, which you can read about at\n", + ".\n", + "`efficacy` and `difficulty` are considered to be\n", + "on the same scale, and the probability of getting a question right\n", + "depends only on the difference between them.\n", + "\n", + "When `efficacy` and `difficulty` are equal, the\n", + "probability of getting the question right is 50%. As\n", + "`efficacy` increases, this probability approaches 100%. As it\n", + "decreases (or as `difficulty` increases), the probability\n", + "approaches 0%.\n", + "\n", + "Given the distribution of `efficacy` across test-takers and\n", + "the distribution of `difficulty` across questions, we can\n", + "compute the expected distribution of raw scores. We’ll do that in two\n", + "steps. First, for a person with given `efficacy`, we’ll\n", + "compute the distribution of raw scores." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def PmfCorrect(efficacy, difficulties):\n", + " pmf0 = thinkbayes.Pmf([0])\n", + "\n", + " ps = [ProbCorrect(efficacy, diff) for diff in difficulties]\n", + " pmfs = [BinaryPmf(p) for p in ps]\n", + " dist = sum(pmfs, pmf0)\n", + " return dist" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`difficulties` is a list of difficulties, one for each\n", + "question. `ps` is a list of probabilities, and\n", + "`pmfs` is a list of two-valued Pmf objects; here’s the\n", + "function that makes them:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def BinaryPmf(p):\n", + " pmf = thinkbayes.Pmf()\n", + " pmf.Set(1, p)\n", + " pmf.Set(0, 1-p)\n", + " return pmf" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`dist` is the sum of these Pmfs. Remember from\n", + "Section [addends] that when we add up Pmf objects, the result is the\n", + "distribution of the sums. In order to use Python’s `sum` to\n", + "add up Pmfs, we have to provide `pmf0` which is the identity\n", + "for Pmfs, so `pmf + pmf0` is always `pmf`.\n", + "\n", + "If we know a person’s efficacy, we can compute their distribution of raw\n", + "scores. For a group of people with a different efficacies, the resulting\n", + "distribution of raw scores is a mixture. Here’s the code that computes\n", + "the mixture:\n", + "\n", + "```python\n", + "# class Exam:\n", + " def MakeRawScoreDist(self, efficacies):\n", + " pmfs = thinkbayes.Pmf()\n", + " for efficacy, prob in efficacies.Items():\n", + " scores = PmfCorrect(efficacy, self.difficulties)\n", + " pmfs.Set(scores, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(pmfs)\n", + " return mix\n", + "```\n", + "`MakeRawScoreDist` takes `efficacies`, which is a\n", + "Pmf that represents the distribution of efficacy across test-takers. I\n", + "assume it is Gaussian with mean 0 and standard deviation 1.5. This\n", + "choice is mostly arbitrary. The probability of getting a question\n", + "correct depends on the difference between efficacy and difficulty, so we\n", + "can choose the units of efficacy and then calibrate the units of\n", + "difficulty accordingly.\n", + "\n", + "`pmfs` is a meta-Pmf that contains one Pmf for each level of\n", + "efficacy, and maps to the fraction of test-takers at that level.\n", + "`MakeMixture` takes the meta-pmf and computes the\n", + "distribution of the mixture (see Section [mixture])." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Calibration\n", + "\n", + "If we were given the distribution of difficulty, we could use\n", + "`MakeRawScoreDist` to compute the distribution of raw scores. But for us\n", + "the problem is the other way around: we are given the distribution of\n", + "raw scores and we want to infer the distribution of difficulty.\n", + "\n", + "![Actual distribution of raw scores and a model to fit\n", + "it.](figs/sat_calibrate.pdf)\n", + "\n", + "[fig.satcalibrate]\n", + "\n", + "I assume that the distribution of difficulty is uniform with parameters\n", + "`center` and `width`.\n", + "`MakeDifficulties` makes a list of difficulties with these\n", + "parameters." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeDifficulties(center, width, n):\n", + " low, high = center-width, center+width\n", + " return numpy.linspace(low, high, n)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By trying out a few combinations, I found that `center=-0.05`\n", + "and `width=1.8` yield a distribution of raw scores similar to\n", + "the actual data, as shown in Figure [fig.satcalibrate].\n", + "\n", + "So, assuming that the distribution of difficulty is uniform, its range\n", + "is approximately `-1.85` to `1.75`, given that\n", + "efficacy is Gaussian with mean 0 and standard deviation 1.5.\n", + "\n", + "The following table shows the range of `ProbCorrect` for\n", + "test-takers at different levels of efficacy:\n", + "\n", + " Difficulty\n", + " Efficacy -1.85 -0.05 1.75\n", + " 3.00 0.99 0.95 0.78\n", + " 1.50 0.97 0.82 0.44\n", + " 0.00 0.86 0.51 0.15\n", + " -1.50 0.59 0.19 0.04\n", + " -3.00 0.24 0.05 0.01\n", + "\n", + "Someone with efficacy 3 (two standard deviations above the mean) has a\n", + "99% chance of answering the easiest questions on the exam, and a 78%\n", + "chance of answering the hardest. On the other end of the range, someone\n", + "two standard deviations below the mean has only a 24% chance of\n", + "answering the easiest questions." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Posterior distribution of efficacy\n", + "\n", + "![Posterior distributions of efficacy for Alice and\n", + "Bob.](figs/sat_posteriors_eff.pdf)\n", + "\n", + "[fig.satposterior2]\n", + "\n", + "Now that the model is calibrated, we can compute the posterior\n", + "distribution of efficacy for Alice and Bob. Here is a version of the Sat\n", + "class that uses the new model:\n", + "\n", + "```python\n", + "class Sat2(thinkbayes.Suite):\n", + "\n", + " def __init__(self, exam, score):\n", + " self.exam = exam\n", + " self.score = score\n", + "\n", + " # start with the Gaussian prior\n", + " efficacies = thinkbayes.MakeGaussianPmf(0, 1.5, 3)\n", + " thinkbayes.Suite.__init__(self, efficacies)\n", + "\n", + " # update based on an exam score\n", + " self.Update(score)\n", + "```\n", + "\n", + "`Update` invokes `Likelihood`, which computes the likelihood of a given\n", + "test score for a hypothetical level of efficacy.\n", + "\n", + "```python\n", + " def Likelihood(self, data, hypo):\n", + " efficacy = hypo\n", + " score = data\n", + " raw = self.exam.Reverse(score)\n", + "\n", + " pmf = self.exam.PmfCorrect(efficacy)\n", + " like = pmf.Prob(raw)\n", + " return like\n", + "```\n", + "\n", + "`pmf` is the distribution of raw scores for a test-taker with\n", + "the given efficacy; `like` is the probability of the observed\n", + "score.\n", + "\n", + "Figure [fig.satposterior2] shows the posterior distributions of efficacy\n", + "for Alice and Bob. As expected, the location of Alice’s distribution is\n", + "farther to the right, but again there is some overlap.\n", + "\n", + "Using `TopLevel` again, we compare $A$, the hypothesis that\n", + "Alice’s efficacy is higher, and $B$, the hypothesis that Bob’s is\n", + "higher. The likelihood ratio is 3.4, a bit smaller than what we got from\n", + "the simple model (3.8). So this model indicates that the data are\n", + "evidence in favor of $A$, but a little weaker than the previous\n", + "estimate.\n", + "\n", + "If our prior belief is that $A$ and $B$ are equally likely, then in\n", + "light of this evidence we would give $A$ a posterior probability of 77%,\n", + "leaving a 23% chance that Bob’s efficacy is higher." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Predictive distribution\n", + "\n", + "The analysis we have done so far generates estimates for Alice and Bob’s\n", + "efficacy, but since efficacy is not directly observable, it is hard to\n", + "validate the results.\n", + "\n", + "To give the model predictive power, we can use it to answer a related\n", + "question: “If Alice and Bob take the math SAT again, what is the chance\n", + "that Alice will do better again?”\n", + "\n", + "We’ll answer this question in two steps:\n", + "\n", + "- We’ll use the posterior distribution of efficacy to generate a\n", + " predictive distribution of raw score for each test-taker.\n", + "\n", + "- We’ll compare the two predictive distributions to compute the\n", + " probability that Alice gets a higher score again.\n", + "\n", + "We already have most of the code we need. To compute the predictive\n", + "distributions, we can use `MakeRawScoreDist` again:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "exam = Exam()\n", + "a_sat = Sat(exam, 780)\n", + "b_sat = Sat(exam, 740)\n", + "\n", + "a_pred = exam.MakeRawScoreDist(a_sat)\n", + "b_pred = exam.MakeRawScoreDist(b_sat)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then we can find the likelihood that Alice does better on the second\n", + "test, Bob does better, or they tie:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "a_like = thinkbayes.PmfProbGreater(a_pred, b_pred)\n", + "b_like = thinkbayes.PmfProbLess(a_pred, b_pred)\n", + "c_like = thinkbayes.PmfProbEqual(a_pred, b_pred)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The probability that Alice does better on the second exam is 63%, which\n", + "means that Bob has a 37% chance of doing as well or better.\n", + "\n", + "Notice that we have more confidence about Alice’s efficacy than we do\n", + "about the outcome of the next test. The posterior odds are 3:1 that\n", + "Alice’s efficacy is higher, but only 2:1 that Alice will do better on\n", + "the next exam." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "![Joint posterior distribution of `p\\_correct` for Alice and\n", + "Bob.](figs/sat_joint.pdf)\n", + "\n", + "[fig.satjoint]\n", + "\n", + "We started this chapter with the question, “How strong is the evidence\n", + "that Alice is better prepared than Bob?” On the face of it, that sounds\n", + "like we want to test two hypotheses: either Alice is more prepared or\n", + "Bob is.\n", + "\n", + "But in order to compute likelihoods for these hypotheses, we have to\n", + "solve an estimation problem. For each test-taker we have to find the\n", + "posterior distribution of either `p_correct` or `efficacy`.\n", + "\n", + "Values like this are called `**nuisance parameters**` because\n", + "we don’t care what they are, but we have to estimate them to answer the\n", + "question we care about.\n", + "\n", + "One way to visualize the analysis we did in this chapter is to plot the\n", + "space of these parameters. `thinkbayes.MakeJoint` takes two Pmfs,\n", + "computes their joint distribution, and returns a joint pmf of each\n", + "possible pair of values and its probability." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeJoint(pmf1, pmf2):\n", + " joint = Joint()\n", + " for v1, p1 in pmf1.Items():\n", + " for v2, p2 in pmf2.Items():\n", + " joint.Set((v1, v2), p1 * p2)\n", + " return joint" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This function assumes that the two distributions are independent.\n", + "\n", + "Figure [fig.satjoint] shows the joint posterior distribution of\n", + "`p_correct` for Alice and Bob. The diagonal line indicates the part of\n", + "the space where `p_correct` is the same for Alice and Bob. To the right\n", + "of this line, Alice is more prepared; to the left, Bob is more prepared.\n", + "\n", + "In `TopLevel.Update`, when we compute the likelihoods of $A$\n", + "and $B$, we add up the probability mass on each side of this line. For\n", + "the cells that fall on the line, we add up the total mass and split it\n", + "between $A$ and $B$.\n", + "\n", + "The process we used in this chapter—estimating nuisance parameters in\n", + "order to evaluate the likelihood of competing hypotheses—is a common\n", + "Bayesian approach to problems like this." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/13_Simulation.ipynb b/notebooks/13_Simulation.ipynb index 7f220ce..829a948 100644 --- a/notebooks/13_Simulation.ipynb +++ b/notebooks/13_Simulation.ipynb @@ -1,570 +1,1021 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "\n", - "Simulation\n", - "==========\n", - "\n", - "In this chapter I describe my solution to a problem posed by a patient\n", - "with a kidney tumor. I think the problem is important and relevant to\n", - "patients with these tumors and doctors treating them.\n", - "\n", - "And I think the solution is interesting because, although it is a\n", - "Bayesian approach to the problem, the use of Bayes’s theorem is\n", - "implicit. I present the solution and my code; at the end of the chapter\n", - "I will explain the Bayesian part.\n", - "\n", - "If you want more technical detail than I present here, you can read my\n", - "paper on this work at .\n", - "\n", - "The Kidney Tumor problem\n", - "------------------------\n", - "\n", - "I am a frequent reader and occasional contributor to the online\n", - "statistics forum at . In November 2011,\n", - "I read the following message:\n", - "\n", - "> “I have Stage IV Kidney Cancer and am trying to determine if the\n", - "> cancer formed before I retired from the military. ... Given the dates\n", - "> of retirement and detection is it possible to determine when there was\n", - "> a 50/50 chance that I developed the disease? Is it possible to\n", - "> determine the probability on the retirement date? My tumor was 15.5 cm\n", - "> x 15 cm at detection. Grade II.”\n", - "\n", - "I contacted the author of the message and got more information; I\n", - "learned that veterans get different benefits if it is “more likely than\n", - "not” that a tumor formed while they were in military service (among\n", - "other considerations).\n", - "\n", - "Because renal tumors grow slowly, and often do not cause symptoms, they\n", - "are sometimes left untreated. As a result, doctors can observe the rate\n", - "of growth for untreated tumors by comparing scans from the same patient\n", - "at different times. Several papers have reported these growth rates.\n", - "\n", - "I collected data from a paper by Zhang et al[^3]. I contacted the\n", - "authors to see if I could get raw data, but they refused on grounds of\n", - "medical privacy. Nevertheless, I was able to extract the data I needed\n", - "by printing one of their graphs and measuring it with a ruler.\n", - "\n", - "![CDF of RDT in doublings per year.](figs/kidney2.pdf)\n", - "\n", - "[fig.kidney2]\n", - "\n", - "They report growth rates in reciprocal doubling time (RDT), which is in\n", - "units of doublings per year. So a tumor with $RDT=1$ doubles in volume\n", - "each year; with $RDT=2$ it quadruples in the same time, and with\n", - "$RDT=-1$, it halves. Figure [fig.kidney2] shows the distribution of RDT\n", - "for 53 patients.\n", - "\n", - "The squares are the data points from the paper; the line is a model I\n", - "fit to the data. The positive tail fits an exponential distribution\n", - "well, so I used a mixture of two exponentials.\n", - "\n", - "A simple model\n", - "--------------\n", - "\n", - "It is usually a good idea to start with a simple model before trying\n", - "something more challenging. Sometimes the simple model is sufficient for\n", - "the problem at hand, and if not, you can use it to validate the more\n", - "complex model.\n", - "\n", - "For my simple model, I assume that tumors grow with a constant doubling\n", - "time, and that they are three-dimensional in the sense that if the\n", - "maximum linear measurement doubles, the volume is multiplied by eight.\n", - "\n", - "I learned from my correspondent that the time between his discharge from\n", - "the military and his diagnosis was 3291 days (about 9 years). So my\n", - "first calculation was, “If this tumor grew at the median rate, how big\n", - "would it have been at the date of discharge?”\n", - "\n", - "The median volume doubling time reported by Zhang et al is 811 days.\n", - "Assuming 3-dimensional geometry, the doubling time for a linear measure\n", - "is three times longer.\n", - "\n", - " # time between discharge and diagnosis, in days \n", - " interval = 3291.0\n", - "\n", - " # doubling time in linear measure is doubling time in volume * 3\n", - " dt = 811.0 * 3\n", - "\n", - " # number of doublings since discharge\n", - " doublings = interval / dt\n", - "\n", - " # how big was the tumor at time of discharge (diameter in cm)\n", - " d1 = 15.5\n", - " d0 = d1 / 2.0 ** doublings\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "The result, d0, is about 6 cm. So if this tumor formed\n", - "after the date of discharge, it must have grown substantially faster\n", - "than the median rate. Therefore I concluded that it is “more likely than\n", - "not” that this tumor formed before the date of discharge.\n", - "\n", - "In addition, I computed the growth rate that would be implied if this\n", - "tumor had formed after the date of discharge. If we assume an initial\n", - "size of 0.1 cm, we can compute the number of doublings to get to a final\n", - "size of 15.5 cm:\n", - "\n", - " # assume an initial linear measure of 0.1 cm\n", - " d0 = 0.1\n", - " d1 = 15.5\n", - "\n", - " # how many doublings would it take to get from d0 to d1\n", - " doublings = log2(d1 / d0)\n", - "\n", - " # what linear doubling time does that imply?\n", - " dt = interval / doublings\n", - "\n", - " # compute the volumetric doubling time and RDT\n", - " vdt = dt / 3\n", - " rdt = 365 / vdt\n", - "\n", - "dt is linear doubling time, so vdt is\n", - "volumetric doubling time, and rdt is reciprocal doubling\n", - "time.\n", - "\n", - "The number of doublings, in linear measure, is 7.3, which implies an RDT\n", - "of 2.4. In the data from Zhang et al, only 20% of tumors grew this fast\n", - "during a period of observation. So again, I concluded that is “more\n", - "likely than not” that the tumor formed prior to the date of discharge.\n", - "\n", - "These calculations are sufficient to answer the question as posed, and\n", - "on behalf of my correspondent, I wrote a letter explaining my\n", - "conclusions to the Veterans’ Benefit Administration.\n", - "\n", - "Later I told a friend, who is an oncologist, about my results. He was\n", - "surprised by the growth rates observed by Zhang et al, and by what they\n", - "imply about the ages of these tumors. He suggested that the results\n", - "might be interesting to researchers and doctors.\n", - "\n", - "But in order to make them useful, I wanted a more general model of the\n", - "relationship between age and size.\n", - "\n", - "A more general model\n", - "--------------------\n", - "\n", - "Given the size of a tumor at time of diagnosis, it would be most useful\n", - "to know the probability that the tumor formed before any given date; in\n", - "other words, the distribution of ages.\n", - "\n", - "To find it, I run simulations of tumor growth to get the distribution of\n", - "size conditioned on age. Then we can use a Bayesian approach to get the\n", - "distribution of age conditioned on size.\n", - "\n", - "The simulation starts with a small tumor and runs these steps:\n", - "\n", - "1. Choose a growth rate from the distribution of RDT.\n", - "\n", - "2. Compute the size of the tumor at the end of an interval.\n", - "\n", - "3. Record the size of the tumor at each interval.\n", - "\n", - "4. Repeat until the tumor exceeds the maximum relevant size.\n", - "\n", - "For the initial size I chose 0.3 cm, because carcinomas smaller than\n", - "that are less likely to be invasive and less likely to have the blood\n", - "supply needed for rapid growth (see\n", - ").\n", - "\n", - "I chose an interval of 245 days (about 8 months) because that is the\n", - "median time between measurements in the data source.\n", - "\n", - "For the maximum size I chose 20 cm. In the data source, the range of\n", - "observed sizes is 1.0 to 12.0 cm, so we are extrapolating beyond the\n", - "observed range at each end, but not by far, and not in a way likely to\n", - "have a strong effect on the results.\n", - "\n", - "![Simulations of tumor growth, size vs. time.](figs/kidney4.pdf)\n", - "\n", - "[fig.kidney4]\n", - "\n", - "The simulation is based on one big simplification: the growth rate is\n", - "chosen independently during each interval, so it does not depend on age,\n", - "size, or growth rate during previous intervals.\n", - "\n", - "In Section [serial] I review these assumptions and consider more\n", - "detailed models. But first let’s look at some examples.\n", - "\n", - "Figure [fig.kidney4] shows the size of simulated tumors as a function of\n", - "age. The dashed line at 10 cm shows the range of ages for tumors at that\n", - "size: the fastest-growing tumor gets there in 8 years; the slowest takes\n", - "more than 35.\n", - "\n", - "I am presenting results in terms of linear measurements, but the\n", - "calculations are in terms of volume. To convert from one to the other,\n", - "again, I use the volume of a sphere with the given diameter.\n", - "\n", - "Implementation\n", - "--------------\n", - "\n", - "Here is the kernel of the simulation:\n", - "\n", - " def MakeSequence(rdt_seq, v0=0.01, interval=0.67, vmax=Volume(20.0)):\n", - " seq = v0,\n", - " age = 0\n", - "\n", - " for rdt in rdt_seq:\n", - " age += interval\n", - " final, seq = ExtendSequence(age, seq, rdt, interval)\n", - " if final > vmax:\n", - " break\n", - "\n", - " return seq\n", - "\n", - "`rdt_seq` is an iterator that yields random values from the CDF of\n", - "growth rate. v0 is the initial volume in mL.\n", - "interval is the time step in years. vmax is\n", - "the final volume corresponding to a linear measurement of 20 cm.\n", - "\n", - "Volume converts from linear measurement in cm to volume in\n", - "mL, based on the simplification that the tumor is a sphere:\n", - "\n", - " def Volume(diameter, factor=4*math.pi/3):\n", - " return factor * (diameter/2.0)**3\n", - "\n", - "ExtendSequence computes the volume of the tumor at the end\n", - "of the interval.\n", - "\n", - " def ExtendSequence(age, seq, rdt, interval):\n", - " initial = seq[-1]\n", - " doublings = rdt * interval\n", - " final = initial * 2**doublings\n", - " new_seq = seq + (final,)\n", - " cache.Add(age, new_seq, rdt)\n", - " \n", - " return final, new_seq\n", - "\n", - "age is the age of the tumor at the end of the interval.\n", - "seq is a tuple that contains the volumes so far.\n", - "rdt is the growth rate during the interval, in doublings\n", - "per year. interval is the size of the time step in years.\n", - "\n", - "The return values are final, the volume of the tumor at the\n", - "end of the interval, and `new_seq`, a new tuple containing the volumes\n", - "in seq plus the new volume final.\n", - "\n", - "Cache.Add records the age and size of each tumor at the end\n", - "of each interval, as explained in the next section.\n", - "\n", - "Caching the joint distribution\n", - "------------------------------\n", - "\n", - "![Joint distribution of age and tumor size.](figs/kidney8.pdf)\n", - "\n", - "[fig.kidney8]\n", - "\n", - "Here’s how the cache works.\n", - "\n", - " class Cache(object):\n", - "\n", - " def __init__(self):\n", - " self.joint = thinkbayes.Joint()\n", - "\n", - "joint is a joint Pmf that records the frequency of each\n", - "age-size pair, so it approximates the joint distribution of age and\n", - "size.\n", - "\n", - "At the end of each simulated interval, ExtendSequence calls\n", - "Add:\n", - "\n", - " # class Cache\n", - "\n", - " def Add(self, age, seq):\n", - " final = seq[-1]\n", - " cm = Diameter(final)\n", - " bucket = round(CmToBucket(cm))\n", - " self.joint.Incr((age, bucket))\n", - "\n", - "Again, age is the age of the tumor, and seq is\n", - "the sequence of volumes so far.\n", - "\n", - "![Distributions of age, conditioned on size.](figs/kidney6.pdf)\n", - "\n", - "[fig.kidney6]\n", - "\n", - "Before adding the new data to the joint distribution, we use\n", - "Diameter to convert from volume to diameter in centimeters:\n", - "\n", - " def Diameter(volume, factor=3/math.pi/4, exp=1/3.0):\n", - " return 2 * (factor * volume) ** exp\n", - "\n", - "And CmToBucket to convert from centimeters to a discrete\n", - "bucket number:\n", - "\n", - " def CmToBucket(x, factor=10):\n", - " return factor * math.log(x)\n", - "\n", - "The buckets are equally spaced on a log scale. Using\n", - "factor=10 yields a reasonable number of buckets; for\n", - "example, 1 cm maps to bucket 0 and 10 cm maps to bucket 23.\n", - "\n", - "After running the simulations, we can plot the joint distribution as a\n", - "pseudocolor plot, where each cell represents the number of tumors\n", - "observed at a given size-age pair. Figure [fig.kidney8] shows the joint\n", - "distribution after 1000 simulations.\n", - "\n", - "Conditional distributions {#conditional-distributions}\n", - "-------------------------\n", - "\n", - "![Percentiles of tumor age as a function of size.](figs/kidney7.pdf)\n", - "\n", - "[fig.kidney7]\n", - "\n", - "By taking a vertical slice from the joint distribution, we can get the\n", - "distribution of sizes for any given age. By taking a horizontal slice,\n", - "we can get the distribution of ages conditioned on size.\n", - "\n", - "Here’s the code that reads the joint distribution and builds the\n", - "conditional distribution for a given size.\n", - "\n", - " # class Cache\n", - "\n", - " def ConditionalCdf(self, bucket):\n", - " pmf = self.joint.Conditional(0, 1, bucket)\n", - " cdf = pmf.MakeCdf()\n", - " return cdf\n", - "\n", - "`bucket` is the integer bucket number corresponding to tumor size.\n", - "Joint.Conditional computes the PMF of age conditioned on\n", - "bucket. The result is the CDF of age conditioned on\n", - "bucket.\n", - "\n", - "Figure [fig.kidney6] shows several of these CDFs, for a range of sizes.\n", - "To summarize these distributions, we can compute percentiles as a\n", - "function of size.\n", - "\n", - " percentiles = [95, 75, 50, 25, 5]\n", - "\n", - " for bucket in cache.GetBuckets():\n", - " cdf = ConditionalCdf(bucket) \n", - " ps = [cdf.Percentile(p) for p in percentiles]\n", - "\n", - "Figure [fig.kidney7] shows these percentiles for each size bucket. The\n", - "data points are computed from the estimated joint distribution. In the\n", - "model, size and time are discrete, which contributes numerical errors,\n", - "so I also show a least squares fit for each sequence of percentiles.\n", - "\n", - "Serial Correlation {#serial}\n", - "------------------\n", - "\n", - "The results so far are based on a number of modeling decisions; let’s\n", - "review them and consider which ones are the most likely sources of\n", - "error:\n", - "\n", - "- To convert from linear measure to volume, we assume that tumors are\n", - " approximately spherical. This assumption is probably fine for tumors\n", - " up to a few centimeters, but not for very large tumors.\n", - "\n", - "- The distribution of growth rates in the simulations are based on a\n", - " continuous model we chose to fit the data reported by Zhang et al,\n", - " which is based on 53 patients. The fit is only approximate and, more\n", - " importantly, a larger sample would yield a different distribution.\n", - "\n", - "- The growth model does not take into account tumor subtype or grade;\n", - " this assumption is consistent with the conclusion of Zhang et al:\n", - " “Growth rates in renal tumors of different sizes, subtypes and\n", - " grades represent a wide range and overlap substantially.” But with a\n", - " larger sample, a difference might become apparent.\n", - "\n", - "- The distribution of growth rate does not depend on the size of the\n", - " tumor. This assumption would not be realistic for very small and\n", - " very large tumors, whose growth is limited by blood supply.\n", - "\n", - " But tumors observed by Zhang et al ranged from 1 to 12 cm, and they\n", - " found no statistically significant relationship between size and\n", - " growth rate. So if there is a relationship, it is likely to be weak,\n", - " at least in this size range.\n", - "\n", - "- In the simulations, growth rate during each interval is independent\n", - " of previous growth rates. In reality it is plausible that tumors\n", - " that have grown quickly in the past are more likely to grow quickly.\n", - " In other words, there is probably a serial correlation in growth\n", - " rate.\n", - "\n", - "Of these, the first and last seem the most problematic. I’ll investigate\n", - "serial correlation first, then come back to spherical geometry.\n", - "\n", - "To simulate correlated growth, I wrote a generator[^4] that yields a\n", - "correlated series from a given Cdf. Here’s how the algorithm works:\n", - "\n", - "1. Generate correlated values from a Gaussian distribution. This is\n", - " easy to do because we can compute the distribution of the next value\n", - " conditioned on the previous value.\n", - "\n", - "2. Transform each value to its cumulative probability using the\n", - " Gaussian CDF.\n", - "\n", - "3. Transform each cumulative probability to the corresponding value\n", - " using the given Cdf.\n", - "\n", - "Here’s what that looks like in code:\n", - "\n", - " def CorrelatedGenerator(cdf, rho):\n", - " x = random.gauss(0, 1)\n", - " yield Transform(x)\n", - "\n", - " sigma = math.sqrt(1 - rho**2); \n", - " while True:\n", - " x = random.gauss(x * rho, sigma)\n", - " yield Transform(x)\n", - "\n", - "cdf is the desired Cdf; rho is the desired\n", - "correlation. The values of x are Gaussian;\n", - "Transform converts them to the desired distribution.\n", - "\n", - "The first value of x is Gaussian with mean 0 and standard\n", - "deviation 1. For subsequent values, the mean and standard deviation\n", - "depend on the previous value. Given the previous x, the\n", - "mean of the next value is x \\* rho, and the variance is\n", - "1 - rho\\*\\*2.\n", - "\n", - "Transform maps from each Gaussian value, x, to\n", - "a value from the given Cdf, y.\n", - "\n", - " def Transform(x):\n", - " p = thinkbayes.GaussianCdf(x)\n", - " y = cdf.Value(p)\n", - " return y\n", - "\n", - "GaussianCdf computes the CDF of the standard Gaussian\n", - "distribution at x, returning a cumulative probability.\n", - "Cdf.Value maps from a cumulative probability to the\n", - "corresponding value in cdf.\n", - "\n", - "Depending on the shape of cdf, information can be lost in\n", - "transformation, so the actual correlation might be lower than\n", - "rho. For example, when I generate 10000 values from the\n", - "distribution of growth rates with rho=0.4, the actual\n", - "correlation is 0.37. But since we are guessing at the right correlation\n", - "anyway, that’s close enough.\n", - "\n", - "Remember that MakeSequence takes an iterator as an\n", - "argument. That interface allows it to work with different generators:\n", - "\n", - " iterator = UncorrelatedGenerator(cdf)\n", - " seq1 = MakeSequence(iterator)\n", - "\n", - " iterator = CorrelatedGenerator(cdf, rho)\n", - " seq2 = MakeSequence(iterator)\n", - "\n", - "In this example, seq1 and seq2 are drawn from\n", - "the same distribution, but the values in seq1 are\n", - "uncorrelated and the values in seq2 are correlated with a\n", - "coefficient of approximately rho.\n", - "\n", - "Now we can see what effect serial correlation has on the results; the\n", - "following table shows percentiles of age for a 6 cm tumor, using the\n", - "uncorrelated generator and a correlated generator with target\n", - "$\\rho = 0.4$.\n", - "\n", - "Correlation makes the fastest growing tumors faster and the slowest\n", - "slower, so the range of ages is wider. The difference is modest for low\n", - "percentiles, but for the 95th percentile it is more than 6 years. To\n", - "compute these percentiles precisely, we would need a better estimate of\n", - "the actual serial correlation.\n", - "\n", - "However, this model is sufficient to answer the question we started\n", - "with: given a tumor with a linear dimension of 15.5 cm, what is the\n", - "probability that it formed more than 8 years ago?\n", - "\n", - "Here’s the code:\n", - "\n", - " # class Cache\n", - "\n", - " def ProbOlder(self, cm, age):\n", - " bucket = CmToBucket(cm)\n", - " cdf = self.ConditionalCdf(bucket)\n", - " p = cdf.Prob(age)\n", - " return 1-p\n", - "\n", - "cm is the size of the tumor; age is the age\n", - "threshold in years. ProbOlder converts size to a bucket\n", - "number, gets the Cdf of age conditioned on bucket, and computes the\n", - "probability that age exceeds the given value.\n", - "\n", - "With no serial correlation, the probability that a 15.5 cm tumor is\n", - "older than 8 years is 0.999, or almost certain. With correlation 0.4,\n", - "faster-growing tumors are more likely, but the probability is still\n", - "0.995. Even with correlation 0.8, the probability is 0.978.\n", - "\n", - "Another likely source of error is the assumption that tumors are\n", - "approximately spherical. For a tumor with linear dimensions 15.5 x 15\n", - "cm, this assumption is probably not valid. If, as seems likely, a tumor\n", - "this size is relatively flat, it might have the same volume as a 6 cm\n", - "sphere. With this smaller volume and correlation 0.8, the probability of\n", - "age greater than 8 is still 95%.\n", - "\n", - "So even taking into account modeling errors, it is unlikely that such a\n", - "large tumor could have formed less than 8 years prior to the date of\n", - "diagnosis.\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "Well, we got through a whole chapter without using Bayes’s theorem or\n", - "the Suite class that encapsulates Bayesian updates. What\n", - "happened?\n", - "\n", - "One way to think about Bayes’s theorem is as an algorithm for inverting\n", - "conditional probabilities. Given $\\mathrm{p}(B|A)$, we can\n", - "compute $\\mathrm{p}(A|B)$, provided we know\n", - "$\\mathrm{p}(A)$ and $\\mathrm{p}(B)$. Of course\n", - "this algorithm is only useful if, for some reason, it is easier to\n", - "compute $\\mathrm{p}(B|A)$ than\n", - "$\\mathrm{p}(A|B)$.\n", - "\n", - "In this example, it is. By running simulations, we can estimate the\n", - "distribution of size conditioned on age, or\n", - "$\\mathrm{p}(size|age)$. But it is harder to get the\n", - "distribution of age conditioned on size, or\n", - "$\\mathrm{p}(age|size)$. So this seems like a perfect\n", - "opportunity to use Bayes’s theorem.\n", - "\n", - "The reason I didn’t is computational efficiency. To estimate\n", - "$\\mathrm{p}(size|age)$ for any given size, you have to run\n", - "a lot of simulations. Along the way, you end up computing\n", - "$\\mathrm{p}(size|age)$ for a lot of sizes. In fact, you end\n", - "up computing the entire joint distribution of size and age,\n", - "$\\mathrm{p}(size, age)$.\n", - "\n", - "And once you have the joint distribution, you don’t really need Bayes’s\n", - "theorem, you can extract $\\mathrm{p}(age|size)$ by taking\n", - "slices from the joint distribution, as demonstrated in\n", - "ConditionalCdf.\n", - "\n", - "So we side-stepped Bayes, but he was with us in spirit.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Simulation\n", + "\n", + "In this chapter I describe my solution to a problem posed by a patient\n", + "with a kidney tumor. I think the problem is important and relevant to\n", + "patients with these tumors and doctors treating them.\n", + "\n", + "And I think the solution is interesting because, although it is a\n", + "Bayesian approach to the problem, the use of Bayes’s theorem is\n", + "implicit. I present the solution and my code; at the end of the chapter\n", + "I will explain the Bayesian part.\n", + "\n", + "If you want more technical detail than I present here, you can read my\n", + "paper on this work at ." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The Kidney Tumor problem\n", + "\n", + "I am a frequent reader and occasional contributor to the online\n", + "statistics forum at . In November 2011,\n", + "I read the following message:\n", + "\n", + "> “I have Stage IV Kidney Cancer and am trying to determine if the\n", + "> cancer formed before I retired from the military. ... Given the dates\n", + "> of retirement and detection is it possible to determine when there was\n", + "> a 50/50 chance that I developed the disease? Is it possible to\n", + "> determine the probability on the retirement date? My tumor was 15.5 cm\n", + "> x 15 cm at detection. Grade II.”\n", + "\n", + "I contacted the author of the message and got more information; I\n", + "learned that veterans get different benefits if it is “more likely than\n", + "not” that a tumor formed while they were in military service (among\n", + "other considerations).\n", + "\n", + "Because renal tumors grow slowly, and often do not cause symptoms, they\n", + "are sometimes left untreated. As a result, doctors can observe the rate\n", + "of growth for untreated tumors by comparing scans from the same patient\n", + "at different times. Several papers have reported these growth rates.\n", + "\n", + "I collected data from a paper by Zhang et al[^3]. I contacted the\n", + "authors to see if I could get raw data, but they refused on grounds of\n", + "medical privacy. Nevertheless, I was able to extract the data I needed\n", + "by printing one of their graphs and measuring it with a ruler.\n", + "\n", + "![CDF of RDT in doublings per year.](figs/kidney2.pdf)\n", + "\n", + "[fig.kidney2]\n", + "\n", + "They report growth rates in reciprocal doubling time (RDT), which is in\n", + "units of doublings per year. So a tumor with $RDT=1$ doubles in volume\n", + "each year; with $RDT=2$ it quadruples in the same time, and with\n", + "$RDT=-1$, it halves. Figure [fig.kidney2] shows the distribution of RDT\n", + "for 53 patients.\n", + "\n", + "The squares are the data points from the paper; the line is a model I\n", + "fit to the data. The positive tail fits an exponential distribution\n", + "well, so I used a mixture of two exponentials." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## A simple model\n", + "\n", + "It is usually a good idea to start with a simple model before trying\n", + "something more challenging. Sometimes the simple model is sufficient for\n", + "the problem at hand, and if not, you can use it to validate the more\n", + "complex model.\n", + "\n", + "For my simple model, I assume that tumors grow with a constant doubling\n", + "time, and that they are three-dimensional in the sense that if the\n", + "maximum linear measurement doubles, the volume is multiplied by eight.\n", + "\n", + "I learned from my correspondent that the time between his discharge from\n", + "the military and his diagnosis was 3291 days (about 9 years). So my\n", + "first calculation was, “If this tumor grew at the median rate, how big\n", + "would it have been at the date of discharge?”\n", + "\n", + "The median volume doubling time reported by Zhang et al is 811 days.\n", + "Assuming 3-dimensional geometry, the doubling time for a linear measure\n", + "is three times longer." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "# time between discharge and diagnosis, in days \n", + "interval = 3291.0\n", + "\n", + "# doubling time in linear measure is doubling time in volume * 3\n", + "dt = 811.0 * 3\n", + "\n", + "# number of doublings since discharge\n", + "doublings = interval / dt\n", + "\n", + "# how big was the tumor at time of discharge (diameter in cm)\n", + "d1 = 15.5\n", + "d0 = d1 / 2.0 ** doublings" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "The result, `d0`, is about 6 cm. So if this tumor formed\n", + "after the date of discharge, it must have grown substantially faster\n", + "than the median rate. Therefore I concluded that it is “more likely than\n", + "not” that this tumor formed before the date of discharge.\n", + "\n", + "In addition, I computed the growth rate that would be implied if this\n", + "tumor had formed after the date of discharge. If we assume an initial\n", + "size of 0.1 cm, we can compute the number of doublings to get to a final\n", + "size of 15.5 cm:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "150.76707583570433 2.4209529698496777\n" + ] + } + ], + "source": [ + "from math import log2\n", + "\n", + "# assume an initial linear measure of 0.1 cm\n", + "d0 = 0.1\n", + "d1 = 15.5\n", + "\n", + "# how many doublings would it take to get from d0 to d1\n", + "doublings = log2(d1 / d0)\n", + "\n", + "# what linear doubling time does that imply?\n", + "dt = interval / doublings\n", + "\n", + "# compute the volumetric doubling time and RDT\n", + "vdt = dt / 3\n", + "rdt = 365 / vdt\n", + "print(vdt, rdt)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`dt` is linear doubling time, so `vdt` is\n", + "volumetric doubling time, and `rdt` is reciprocal doubling\n", + "time.\n", + "\n", + "The number of doublings, in linear measure, is 7.3, which implies an RDT\n", + "of 2.4. In the data from Zhang et al, only 20% of tumors grew this fast\n", + "during a period of observation. So again, I concluded that is “more\n", + "likely than not” that the tumor formed prior to the date of discharge.\n", + "\n", + "These calculations are sufficient to answer the question as posed, and\n", + "on behalf of my correspondent, I wrote a letter explaining my\n", + "conclusions to the Veterans’ Benefit Administration.\n", + "\n", + "Later I told a friend, who is an oncologist, about my results. He was\n", + "surprised by the growth rates observed by Zhang et al, and by what they\n", + "imply about the ages of these tumors. He suggested that the results\n", + "might be interesting to researchers and doctors.\n", + "\n", + "But in order to make them useful, I wanted a more general model of the\n", + "relationship between age and size." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## A more general model\n", + "\n", + "Given the size of a tumor at time of diagnosis, it would be most useful\n", + "to know the probability that the tumor formed before any given date; in\n", + "other words, the distribution of ages.\n", + "\n", + "To find it, I run simulations of tumor growth to get the distribution of\n", + "size conditioned on age. Then we can use a Bayesian approach to get the\n", + "distribution of age conditioned on size.\n", + "\n", + "The simulation starts with a small tumor and runs these steps:\n", + "\n", + "1. Choose a growth rate from the distribution of RDT.\n", + "\n", + "2. Compute the size of the tumor at the end of an interval.\n", + "\n", + "3. Record the size of the tumor at each interval.\n", + "\n", + "4. Repeat until the tumor exceeds the maximum relevant size.\n", + "\n", + "For the initial size I chose 0.3 cm, because carcinomas smaller than\n", + "that are less likely to be invasive and less likely to have the blood\n", + "supply needed for rapid growth (see\n", + ").\n", + "\n", + "I chose an interval of 245 days (about 8 months) because that is the\n", + "median time between measurements in the data source.\n", + "\n", + "For the maximum size I chose 20 cm. In the data source, the range of\n", + "observed sizes is 1.0 to 12.0 cm, so we are extrapolating beyond the\n", + "observed range at each end, but not by far, and not in a way likely to\n", + "have a strong effect on the results.\n", + "\n", + "![Simulations of tumor growth, size vs. time.](figs/kidney4.pdf)\n", + "\n", + "[fig.kidney4]\n", + "\n", + "The simulation is based on one big simplification: the growth rate is\n", + "chosen independently during each interval, so it does not depend on age,\n", + "size, or growth rate during previous intervals.\n", + "\n", + "In Section [serial] I review these assumptions and consider more\n", + "detailed models. But first let’s look at some examples.\n", + "\n", + "Figure [fig.kidney4] shows the size of simulated tumors as a function of\n", + "age. The dashed line at 10 cm shows the range of ages for tumors at that\n", + "size: the fastest-growing tumor gets there in 8 years; the slowest takes\n", + "more than 35.\n", + "\n", + "I am presenting results in terms of linear measurements, but the\n", + "calculations are in terms of volume. To convert from one to the other,\n", + "again, I use the volume of a sphere with the given diameter." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implementation\n", + "\n", + "Here is the kernel of the simulation:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from kidney import Volume\n", + "\n", + "def MakeSequence(rdt_seq, v0=0.01, interval=0.67, vmax=Volume(20.0)):\n", + " seq = v0,\n", + " age = 0\n", + "\n", + " for rdt in rdt_seq:\n", + " age += interval\n", + " final, seq = ExtendSequence(age, seq, rdt, interval)\n", + " if final > vmax:\n", + " break\n", + "\n", + " return seq" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`rdt_seq` is an iterator that yields random values from the CDF of\n", + "growth rate. `v0` is the initial volume in mL.\n", + "`interval` is the time step in years. `vmax` is\n", + "the final volume corresponding to a linear measurement of 20 cm.\n", + "\n", + "`Volume` converts from linear measurement in cm to volume in\n", + "mL, based on the simplification that the tumor is a sphere:\n", + "\n", + " def Volume(diameter, factor=4*math.pi/3):\n", + " return factor * (diameter/2.0)**3\n", + "\n", + "`ExtendSequence` computes the volume of the tumor at the end\n", + "of the interval." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def ExtendSequence(age, seq, rdt, interval):\n", + " initial = seq[-1]\n", + " doublings = rdt * interval\n", + " final = initial * 2**doublings\n", + " new_seq = seq + (final,)\n", + " cache.Add(age, new_seq, rdt)\n", + "\n", + " return final, new_seq" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`age` is the age of the tumor at the end of the interval.\n", + "`seq` is a tuple that contains the volumes so far.\n", + "`rdt` is the growth rate during the interval, in doublings\n", + "per year. `interval` is the size of the time step in years.\n", + "\n", + "The return values are `final`, the volume of the tumor at the\n", + "end of the interval, and `new_seq`, a new tuple containing the volumes\n", + "in `seq` plus the new volume `final`.\n", + "\n", + "`Cache.Add` records the age and size of each tumor at the end\n", + "of each interval, as explained in the next section." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Caching the joint distribution\n", + "\n", + "![Joint distribution of age and tumor size.](figs/kidney8.pdf)\n", + "\n", + "[fig.kidney8]\n", + "\n", + "Here’s how the cache works.\n", + "\n", + "```python\n", + "class Cache(object):\n", + "\n", + " def __init__(self):\n", + " self.joint = thinkbayes.Joint()\n", + "```\n", + "\n", + "`joint` is a joint Pmf that records the frequency of each\n", + "age-size pair, so it approximates the joint distribution of age and\n", + "size.\n", + "\n", + "At the end of each simulated interval, `ExtendSequence` calls\n", + "`Add`:\n", + "\n", + "```python\n", + " # class Cache\n", + "\n", + " def Add(self, age, seq):\n", + " final = seq[-1]\n", + " cm = Diameter(final)\n", + " bucket = round(CmToBucket(cm))\n", + " self.joint.Incr((age, bucket))\n", + "```\n", + "\n", + "Again, `age` is the age of the tumor, and `seq` is\n", + "the sequence of volumes so far.\n", + "\n", + "![Distributions of age, conditioned on size.](figs/kidney6.pdf)\n", + "\n", + "[fig.kidney6]\n", + "\n", + "Before adding the new data to the joint distribution, we use\n", + "`Diameter` to convert from volume to diameter in centimeters:\n", + "\n", + " def Diameter(volume, factor=3/math.pi/4, exp=1/3.0):\n", + " return 2 * (factor * volume) ** exp\n", + "\n", + "And `CmToBucket` to convert from centimeters to a discrete\n", + "bucket number:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "import math\n", + "\n", + "def CmToBucket(x, factor=10):\n", + " return factor * math.log(x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The buckets are equally spaced on a log scale. Using\n", + "`factor=10` yields a reasonable number of buckets; for\n", + "example, 1 cm maps to bucket 0 and 10 cm maps to bucket 23.\n", + "\n", + "After running the simulations, we can plot the joint distribution as a\n", + "pseudocolor plot, where each cell represents the number of tumors\n", + "observed at a given size-age pair. Figure [fig.kidney8] shows the joint\n", + "distribution after 1000 simulations." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conditional distributions\n", + "\n", + "![Percentiles of tumor age as a function of size.](figs/kidney7.pdf)\n", + "\n", + "[fig.kidney7]\n", + "\n", + "By taking a vertical slice from the joint distribution, we can get the\n", + "distribution of sizes for any given age. By taking a horizontal slice,\n", + "we can get the distribution of ages conditioned on size.\n", + "\n", + "Here’s the code that reads the joint distribution and builds the\n", + "conditional distribution for a given size.\n", + "\n", + "```python\n", + "# class Cache\n", + "\n", + " def ConditionalCdf(self, bucket):\n", + " pmf = self.joint.Conditional(0, 1, bucket)\n", + " cdf = pmf.MakeCdf()\n", + " return cdf\n", + "```\n", + "\n", + "`bucket` is the integer bucket number corresponding to tumor size.\n", + "`Joint.Conditional` computes the PMF of age conditioned on\n", + "`bucket`. The result is the CDF of age conditioned on\n", + "`bucket`.\n", + "\n", + "Figure [fig.kidney6] shows several of these CDFs, for a range of sizes.\n", + "To summarize these distributions, we can compute percentiles as a\n", + "function of size." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from kidney import Cache\n", + "\n", + "percentiles = [95, 75, 50, 25, 5]\n", + "cache = Cache()\n", + "\n", + "for bucket in cache.GetBuckets():\n", + " cdf = ConditionalCdf(bucket) \n", + " ps = [cdf.Percentile(p) for p in percentiles]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure [fig.kidney7] shows these percentiles for each size bucket. The\n", + "data points are computed from the estimated joint distribution. In the\n", + "model, size and time are discrete, which contributes numerical errors,\n", + "so I also show a least squares fit for each sequence of percentiles." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Serial Correlation\n", + "\n", + "The results so far are based on a number of modeling decisions; let’s\n", + "review them and consider which ones are the most likely sources of\n", + "error:\n", + "\n", + "- To convert from linear measure to volume, we assume that tumors are\n", + " approximately spherical. This assumption is probably fine for tumors\n", + " up to a few centimeters, but not for very large tumors.\n", + "\n", + "- The distribution of growth rates in the simulations are based on a\n", + " continuous model we chose to fit the data reported by Zhang et al,\n", + " which is based on 53 patients. The fit is only approximate and, more\n", + " importantly, a larger sample would yield a different distribution.\n", + "\n", + "- The growth model does not take into account tumor subtype or grade;\n", + " this assumption is consistent with the conclusion of Zhang et al:\n", + " “Growth rates in renal tumors of different sizes, subtypes and\n", + " grades represent a wide range and overlap substantially.” But with a\n", + " larger sample, a difference might become apparent.\n", + "\n", + "- The distribution of growth rate does not depend on the size of the\n", + " tumor. This assumption would not be realistic for very small and\n", + " very large tumors, whose growth is limited by blood supply.\n", + "\n", + " But tumors observed by Zhang et al ranged from 1 to 12 cm, and they\n", + " found no statistically significant relationship between size and\n", + " growth rate. So if there is a relationship, it is likely to be weak,\n", + " at least in this size range.\n", + "\n", + "- In the simulations, growth rate during each interval is independent\n", + " of previous growth rates. In reality it is plausible that tumors\n", + " that have grown quickly in the past are more likely to grow quickly.\n", + " In other words, there is probably a serial correlation in growth\n", + " rate.\n", + "\n", + "Of these, the first and last seem the most problematic. I’ll investigate\n", + "serial correlation first, then come back to spherical geometry.\n", + "\n", + "To simulate correlated growth, I wrote a generator[^4] that yields a\n", + "correlated series from a given Cdf. Here’s how the algorithm works:\n", + "\n", + "1. Generate correlated values from a Gaussian distribution. This is\n", + " easy to do because we can compute the distribution of the next value\n", + " conditioned on the previous value.\n", + "\n", + "2. Transform each value to its cumulative probability using the\n", + " Gaussian CDF.\n", + "\n", + "3. Transform each cumulative probability to the corresponding value\n", + " using the given Cdf.\n", + "\n", + "Here’s what that looks like in code:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def CorrelatedGenerator(cdf, rho):\n", + " x = random.gauss(0, 1)\n", + " yield Transform(x)\n", + "\n", + " sigma = math.sqrt(1 - rho**2); \n", + " while True:\n", + " x = random.gauss(x * rho, sigma)\n", + " yield Transform(x)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`cdf` is the desired Cdf; `rho` is the desired\n", + "correlation. The values of `x` are Gaussian;\n", + "`Transform` converts them to the desired distribution.\n", + "\n", + "The first value of `x` is Gaussian with mean 0 and standard\n", + "deviation 1. For subsequent values, the mean and standard deviation\n", + "depend on the previous value. Given the previous `x`, the\n", + "mean of the next value is `x \\* rho`, and the variance is\n", + "`1 - rho\\*\\*2`.\n", + "\n", + "`Transform` maps from each Gaussian value, `x`, to\n", + "a value from the given Cdf, `y`." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def Transform(x):\n", + " p = thinkbayes.GaussianCdf(x)\n", + " y = cdf.Value(p)\n", + " return y" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`GaussianCdf` computes the CDF of the standard Gaussian\n", + "distribution at `x`, returning a cumulative probability.\n", + "`Cdf.Value` maps from a cumulative probability to the\n", + "corresponding value in `cdf`.\n", + "\n", + "Depending on the shape of `cdf`, information can be lost in\n", + "transformation, so the actual correlation might be lower than\n", + "`rho`. For example, when I generate 10000 values from the\n", + "distribution of growth rates with `rho=0.4`, the actual\n", + "correlation is 0.37. But since we are guessing at the right correlation\n", + "anyway, that’s close enough.\n", + "\n", + "Remember that `MakeSequence` takes an iterator as an\n", + "argument. That interface allows it to work with different generators:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'cdf' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[1;32mfrom\u001b[0m \u001b[0mkidney\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mUncorrelatedGenerator\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mCorrelatedGenerator\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[0miterator\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mUncorrelatedGenerator\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcdf\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 4\u001b[0m \u001b[0mseq1\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mMakeSequence\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mNameError\u001b[0m: name 'cdf' is not defined" + ] + } + ], + "source": [ + "from kidney import UncorrelatedGenerator, CorrelatedGenerator\n", + "\n", + "iterator = UncorrelatedGenerator(cdf)\n", + "seq1 = MakeSequence(iterator)\n", + "\n", + "iterator = CorrelatedGenerator(cdf, rho)\n", + "seq2 = MakeSequence(iterator)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, `seq1` and `seq2` are drawn from\n", + "the same distribution, but the values in `seq1` are\n", + "uncorrelated and the values in `seq2` are correlated with a\n", + "coefficient of approximately `rho`.\n", + "\n", + "Now we can see what effect serial correlation has on the results; the\n", + "following table shows percentiles of age for a 6 cm tumor, using the\n", + "uncorrelated generator and a correlated generator with target\n", + "$\\rho = 0.4$.\n", + "\n", + "Correlation makes the fastest growing tumors faster and the slowest\n", + "slower, so the range of ages is wider. The difference is modest for low\n", + "percentiles, but for the 95th percentile it is more than 6 years. To\n", + "compute these percentiles precisely, we would need a better estimate of\n", + "the actual serial correlation.\n", + "\n", + "However, this model is sufficient to answer the question we started\n", + "with: given a tumor with a linear dimension of 15.5 cm, what is the\n", + "probability that it formed more than 8 years ago?\n", + "\n", + "Here’s the code:\n", + "\n", + "```python\n", + "# class Cache\n", + "\n", + " def ProbOlder(self, cm, age):\n", + " bucket = CmToBucket(cm)\n", + " cdf = self.ConditionalCdf(bucket)\n", + " p = cdf.Prob(age)\n", + " return 1-p\n", + "```\n", + "\n", + "`cm` is the size of the tumor; `age` is the age\n", + "threshold in years. `ProbOlder` converts size to a bucket\n", + "number, gets the Cdf of age conditioned on bucket, and computes the\n", + "probability that age exceeds the given value.\n", + "\n", + "With no serial correlation, the probability that a 15.5 cm tumor is\n", + "older than 8 years is 0.999, or almost certain. With correlation 0.4,\n", + "faster-growing tumors are more likely, but the probability is still\n", + "0.995. Even with correlation 0.8, the probability is 0.978.\n", + "\n", + "Another likely source of error is the assumption that tumors are\n", + "approximately spherical. For a tumor with linear dimensions 15.5 x 15\n", + "cm, this assumption is probably not valid. If, as seems likely, a tumor\n", + "this size is relatively flat, it might have the same volume as a 6 cm\n", + "sphere. With this smaller volume and correlation 0.8, the probability of\n", + "age greater than 8 is still 95%.\n", + "\n", + "So even taking into account modeling errors, it is unlikely that such a\n", + "large tumor could have formed less than 8 years prior to the date of\n", + "diagnosis." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "## Discussion\n", + "\n", + "Well, we got through a whole chapter without using Bayes’s theorem or\n", + "the `Suite` class that encapsulates Bayesian updates. What\n", + "happened?\n", + "\n", + "One way to think about Bayes’s theorem is as an algorithm for inverting\n", + "conditional probabilities. Given $\\mathrm{p}(B|A)$, we can\n", + "compute $\\mathrm{p}(A|B)$, provided we know\n", + "$\\mathrm{p}(A)$ and $\\mathrm{p}(B)$. Of course\n", + "this algorithm is only useful if, for some reason, it is easier to\n", + "compute $\\mathrm{p}(B|A)$ than\n", + "$\\mathrm{p}(A|B)$.\n", + "\n", + "In this example, it is. By running simulations, we can estimate the\n", + "distribution of size conditioned on age, or\n", + "$\\mathrm{p}(size|age)$. But it is harder to get the\n", + "distribution of age conditioned on size, or\n", + "$\\mathrm{p}(age|size)$. So this seems like a perfect\n", + "opportunity to use Bayes’s theorem.\n", + "\n", + "The reason I didn’t is computational efficiency. To estimate\n", + "$\\mathrm{p}(size|age)$ for any given size, you have to run\n", + "a lot of simulations. Along the way, you end up computing\n", + "$\\mathrm{p}(size|age)$ for a lot of sizes. In fact, you end\n", + "up computing the entire joint distribution of size and age,\n", + "$\\mathrm{p}(size, age)$.\n", + "\n", + "And once you have the joint distribution, you don’t really need Bayes’s\n", + "theorem, you can extract $\\mathrm{p}(age|size)$ by taking\n", + "slices from the joint distribution, as demonstrated in\n", + "`ConditionalCdf`.\n", + "\n", + "So we side-stepped Bayes, but he was with us in spirit." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/14_A_Hierarchical_Model.ipynb b/notebooks/14_A_Hierarchical_Model.ipynb index 6aa17c9..25b9f53 100644 --- a/notebooks/14_A_Hierarchical_Model.ipynb +++ b/notebooks/14_A_Hierarchical_Model.ipynb @@ -1,323 +1,649 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "A Hierarchical Model {#hierarchical}\n", - "====================\n", - "\n", - "The Geiger counter problem\n", - "--------------------------\n", - "\n", - "I got the idea for the following problem from Tom Campbell-Ricketts,\n", - "author of the Maximum Entropy blog at\n", - ". And he got the idea from\n", - "E. T. Jaynes, author of the classic *Probability Theory: The Logic\n", - "of Science*:\n", - "\n", - "> Suppose that a radioactive source emits particles toward a Geiger\n", - "> counter at an average rate of $r$ particles per second, but the\n", - "> counter only registers a fraction, $f$, of the particles that hit it.\n", - "> If $f$ is 10% and the counter registers 15 particles in a one second\n", - "> interval, what is the posterior distribution of $n$, the actual number\n", - "> of particles that hit the counter, and $r$, the average rate particles\n", - "> are emitted?\n", - "\n", - "To get started on a problem like this, think about the chain of\n", - "causation that starts with the parameters of the system and ends with\n", - "the observed data:\n", - "\n", - "1. The source emits particles at an average rate, $r$.\n", - "\n", - "2. During any given second, the source emits $n$ particles toward the\n", - " counter.\n", - "\n", - "3. Out of those $n$ particles, some number, $k$, get counted.\n", - "\n", - "The probability that an atom decays is the same at any point in time, so\n", - "radioactive decay is well modeled by a Poisson process. Given $r$, the\n", - "distribution of $n$ is Poisson distribution with parameter $r$.\n", - "\n", - "And if we assume that the probability of detection for each particle is\n", - "independent of the others, the distribution of $k$ is the binomial\n", - "distribution with parameters $n$ and $f$.\n", - "\n", - "Given the parameters of the system, we can find the distribution of the\n", - "data. So we can solve what is called the **forward\n", - "problem**.\n", - "\n", - "Now we want to go the other way: given the data, we want the\n", - "distribution of the parameters. This is called the **inverse\n", - "problem**. And if you can solve the forward problem, you can use\n", - "Bayesian methods to solve the inverse problem.\n", - "\n", - "Start simple\n", - "------------\n", - "\n", - "![Posterior distribution of $n$ for three values of\n", - "$r$.](figs/jaynes1.pdf)\n", - "\n", - "[fig.jaynes1]\n", - "\n", - "Let’s start with a simple version of the problem where we know the value\n", - "of $r$. We are given the value of $f$, so all we have to do is estimate\n", - "$n$.\n", - "\n", - "I define a Suite called Detector that models the behavior\n", - "of the detector and estimates $n$.\n", - "\n", - " class Detector(thinkbayes.Suite):\n", - "\n", - " def __init__(self, r, f, high=500, step=1):\n", - " pmf = thinkbayes.MakePoissonPmf(r, high, step=step)\n", - " thinkbayes.Suite.__init__(self, pmf, name=r)\n", - " self.r = r\n", - " self.f = f\n", - "\n", - "If the average emission rate is $r$ particles per second, the\n", - "distribution of $n$ is Poisson with parameter $r$. high and\n", - "step determine the upper bound for $n$ and the step size\n", - "between hypothetical values.\n", - "\n", - "Now we need a likelihood function:\n", - "\n", - " # class Detector\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " k = data\n", - " n = hypo\n", - " p = self.f\n", - "\n", - " return thinkbayes.EvalBinomialPmf(k, n, p)\n", - "\n", - "data is the number of particles detected, and\n", - "hypo is the hypothetical number of particles emitted, $n$.\n", - "\n", - "If there are actually $n$ particles, and the probability of detecting\n", - "any one of them is $f$, the probability of detecting $k$ particles is\n", - "given by the binomial distribution.\n", - "\n", - "That’s it for the Detector. We can try it out for a range of values of\n", - "$r$:\n", - "\n", - " f = 0.1\n", - " k = 15\n", - "\n", - " for r in [100, 250, 400]:\n", - " suite = Detector(r, f, step=1)\n", - " suite.Update(k)\n", - " print suite.MaximumLikelihood()\n", - "\n", - "Figure [fig.jaynes1] shows the posterior distribution of $n$ for several\n", - "given values of $r$.\n", - "\n", - "Make it hierarchical\n", - "--------------------\n", - "\n", - "In the previous section, we assume $r$ is known. Now let’s relax that\n", - "assumption. I define another Suite, called Emitter, that\n", - "models the behavior of the emitter and estimates $r$:\n", - "\n", - " class Emitter(thinkbayes.Suite):\n", - "\n", - " def __init__(self, rs, f=0.1):\n", - " detectors = [Detector(r, f) for r in rs]\n", - " thinkbayes.Suite.__init__(self, detectors)\n", - "\n", - "rs is a sequence of hypothetical value for $r$.\n", - "detectors is a sequence of Detector objects, one for each\n", - "value of $r$. The values in the Suite are Detectors, so Emitter is a\n", - "**meta-Suite**; that is, a Suite that contains other Suites\n", - "as values.\n", - "\n", - "To update the Emitter, we have to compute the likelihood of the data\n", - "under each hypothetical value of $r$. But each value of $r$ is\n", - "represented by a Detector that contains a range of values for $n$.\n", - "\n", - "To compute the likelihood of the data for a given Detector, we loop\n", - "through the values of $n$ and add up the total probability of $k$.\n", - "That’s what SuiteLikelihood does:\n", - "\n", - " # class Detector\n", - "\n", - " def SuiteLikelihood(self, data):\n", - " total = 0\n", - " for hypo, prob in self.Items():\n", - " like = self.Likelihood(data, hypo)\n", - " total += prob * like\n", - " return total\n", - "\n", - "Now we can write the Likelihood function for the Emitter:\n", - "\n", - " # class Detector\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " detector = hypo\n", - " like = detector.SuiteLikelihood(data)\n", - " return like\n", - "\n", - "Each hypo is a Detector, so we can invoke\n", - "SuiteLikelihood to get the likelihood of the data under the\n", - "hypothesis.\n", - "\n", - "After we update the Emitter, we have to update each of the Detectors,\n", - "too.\n", - "\n", - " # class Detector\n", - "\n", - " def Update(self, data):\n", - " thinkbayes.Suite.Update(self, data)\n", - " \n", - " for detector in self.Values():\n", - " detector.Update()\n", - "\n", - "A model like this, with multiple levels of Suites, is called\n", - "**hierarchical**.\n", - "\n", - "A little optimization\n", - "---------------------\n", - "\n", - "You might recognize SuiteLikelihood; we saw it in\n", - "Section [suitelike]. At the time, I pointed out that we didn’t really\n", - "need it, because the total probability computed by\n", - "SuiteLikelihood is exactly the normalizing constant\n", - "computed and returned by Update.\n", - "\n", - "So instead of updating the Emitter and then updating the Detectors, we\n", - "can do both steps at the same time, using the result from\n", - "Detector.Update as the likelihood of Emitter.\n", - "\n", - "Here’s the streamlined version of Emitter.Likelihood:\n", - "\n", - " # class Emitter\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " return hypo.Update(data)\n", - "\n", - "And with this version of Likelihood we can use the default\n", - "version of Update. So this version has fewer lines of code,\n", - "and it runs faster because it does not compute the normalizing constant\n", - "twice.\n", - "\n", - "Extracting the posteriors\n", - "-------------------------\n", - "\n", - "![Posterior distributions of $n$ and $r$.](figs/jaynes2.pdf)\n", - "\n", - "[fig.jaynes2]\n", - "\n", - "After we update the Emitter, we can get the posterior distribution of\n", - "$r$ by looping through the Detectors and their probabilities:\n", - "\n", - " # class Emitter\n", - "\n", - " def DistOfR(self):\n", - " items = [(detector.r, prob) for detector, prob in self.Items()]\n", - " return thinkbayes.MakePmfFromItems(items)\n", - "\n", - "items is a list of values of $r$ and their probabilities.\n", - "The result is the Pmf of $r$.\n", - "\n", - "To get the posterior distribution of $n$, we have to compute the mixture\n", - "of the Detectors. We can use thinkbayes.MakeMixture, which\n", - "takes a meta-Pmf that maps from each distribution to its probability.\n", - "And that’s exactly what the Emitter is:\n", - "\n", - " # class Emitter\n", - "\n", - " def DistOfN(self):\n", - " return thinkbayes.MakeMixture(self)\n", - "\n", - "Figure [fig.jaynes2] shows the results. Not surprisingly, the most\n", - "likely value for $n$ is 150. Given $f$ and $n$, the expected count is\n", - "$k = f n$, so given $f$ and $k$, the expected value of $n$ is $k / f$,\n", - "which is 150.\n", - "\n", - "And if 150 particles are emitted in one second, the most likely value of\n", - "$r$ is 150 particles per second. So the posterior distribution of $r$ is\n", - "also centered on 150.\n", - "\n", - "The posterior distributions of $r$ and $n$ are similar; the only\n", - "difference is that we are slightly less certain about $n$. In general,\n", - "we can be more certain about the long-range emission rate, $r$, than\n", - "about the number of particles emitted in any particular second, $n$.\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "The Geiger counter problem demonstrates the connection between causation\n", - "and hierarchical modeling. In the example, the emission rate $r$ has a\n", - "causal effect on the number of particles, $n$, which has a causal effect\n", - "on the particle count, $k$.\n", - "\n", - "The hierarchical model reflects the structure of the system, with causes\n", - "at the top and effects at the bottom.\n", - "\n", - "1. At the top level, we start with a range of hypothetical values for\n", - " $r$.\n", - "\n", - "2. For each value of $r$, we have a range of values for $n$, and the\n", - " prior distribution of $n$ depends on $r$.\n", - "\n", - "3. When we update the model, we go bottom-up. We compute a posterior\n", - " distribution of $n$ for each value of $r$, then compute the\n", - " posterior distribution of $r$.\n", - "\n", - "So causal information flows down the hierarchy, and inference flows up.\n", - "\n", - "Exercises\n", - "---------\n", - "\n", - "This exercise is also inspired by an example in Jaynes,\n", - "*Probability Theory*.\n", - "\n", - "Suppose you buy a mosquito trap that is supposed to reduce the\n", - "population of mosquitoes near your house. Each week, you empty the trap\n", - "and count the number of mosquitoes captured. After the first week, you\n", - "count 30 mosquitoes. After the second week, you count 20 mosquitoes.\n", - "Estimate the percentage change in the number of mosquitoes in your yard.\n", - "\n", - "To answer this question, you have to make some modeling decisions. Here\n", - "are some suggestions:\n", - "\n", - "- Suppose that each week a large number of mosquitoes, $N$, is bred in\n", - " a wetland near your home.\n", - "\n", - "- During the week, some fraction of them, $f_1$, wander into your\n", - " yard, and of those some fraction, $f_2$, are caught in the trap.\n", - "\n", - "- Your solution should take into account your prior belief about how\n", - " much $N$ is likely to change from one week to the next. You can do\n", - " that by adding a level to the hierarchy to model the percent change\n", - " in $N$.\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# A Hierarchical Model\n", + "\n", + "## The Geiger counter problem\n", + "\n", + "I got the idea for the following problem from Tom Campbell-Ricketts,\n", + "author of the Maximum Entropy blog at\n", + ". And he got the idea from\n", + "E. T. Jaynes, author of the classic `Probability Theory: The Logic\n", + "of Science`:\n", + "\n", + "> Suppose that a radioactive source emits particles toward a Geiger\n", + "> counter at an average rate of $r$ particles per second, but the\n", + "> counter only registers a fraction, $f$, of the particles that hit it.\n", + "> If $f$ is 10% and the counter registers 15 particles in a one second\n", + "> interval, what is the posterior distribution of $n$, the actual number\n", + "> of particles that hit the counter, and $r$, the average rate particles\n", + "> are emitted?\n", + "\n", + "To get started on a problem like this, think about the chain of\n", + "causation that starts with the parameters of the system and ends with\n", + "the observed data:\n", + "\n", + "1. The source emits particles at an average rate, $r$.\n", + "\n", + "2. During any given second, the source emits $n$ particles toward the\n", + " counter.\n", + "\n", + "3. Out of those $n$ particles, some number, $k$, get counted.\n", + "\n", + "The probability that an atom decays is the same at any point in time, so\n", + "radioactive decay is well modeled by a Poisson process. Given $r$, the\n", + "distribution of $n$ is Poisson distribution with parameter $r$.\n", + "\n", + "And if we assume that the probability of detection for each particle is\n", + "independent of the others, the distribution of $k$ is the binomial\n", + "distribution with parameters $n$ and $f$.\n", + "\n", + "Given the parameters of the system, we can find the distribution of the\n", + "data. So we can solve what is called the **forward problem**.\n", + "\n", + "Now we want to go the other way: given the data, we want the\n", + "distribution of the parameters. This is called the **inverse problem**. \n", + "And if you can solve the forward problem, you can use\n", + "Bayesian methods to solve the inverse problem." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Start simple\n", + "\n", + "![Posterior distribution of $n$ for three values of\n", + "$r$.](figs/jaynes1.pdf)\n", + "\n", + "[fig.jaynes1]\n", + "\n", + "Let’s start with a simple version of the problem where we know the value\n", + "of $r$. We are given the value of $f$, so all we have to do is estimate\n", + "$n$.\n", + "\n", + "I define a Suite called `Detector` that models the behavior\n", + "of the detector and estimates $n$.\n", + "\n", + "```python\n", + "class Detector(thinkbayes.Suite):\n", + "\n", + " def __init__(self, r, f, high=500, step=1):\n", + " pmf = thinkbayes.MakePoissonPmf(r, high, step=step)\n", + " thinkbayes.Suite.__init__(self, pmf, name=r)\n", + " self.r = r\n", + " self.f = f\n", + "```\n", + "\n", + "If the average emission rate is $r$ particles per second, the\n", + "distribution of $n$ is Poisson with parameter $r$. `high` and\n", + "`step` determine the upper bound for $n$ and the step size\n", + "between hypothetical values.\n", + "\n", + "Now we need a likelihood function:\n", + "\n", + "```python\n", + "# class Detector\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " k = data\n", + " n = hypo\n", + " p = self.f\n", + "\n", + " return thinkbayes.EvalBinomialPmf(k, n, p)\n", + "```\n", + "`data` is the number of particles detected, and\n", + "`hypo` is the hypothetical number of particles emitted, $n$.\n", + "\n", + "If there are actually $n$ particles, and the probability of detecting\n", + "any one of them is $f$, the probability of detecting $k$ particles is\n", + "given by the binomial distribution.\n", + "\n", + "That’s it for the Detector. We can try it out for a range of values of\n", + "$r$:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "ename": "OverflowError", + "evalue": "int too large to convert to float", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mOverflowError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 7\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mr\u001b[0m \u001b[1;32min\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;36m100\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m250\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m400\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m \u001b[0msuite\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mDetector\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mr\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstep\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 9\u001b[0m \u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mUpdate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMaximumLikelihood\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mC:\\Users\\rlabbe\\Dropbox\\ThinkBayes\\code\\jaynes.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, r, f, high, step)\u001b[0m\n\u001b[0;32m 132\u001b[0m \u001b[0mstep\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstep\u001b[0m \u001b[0msize\u001b[0m \u001b[0mbetween\u001b[0m \u001b[0mhypothetical\u001b[0m \u001b[0mvalues\u001b[0m \u001b[0mof\u001b[0m \u001b[0mn\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 133\u001b[0m \"\"\"\n\u001b[1;32m--> 134\u001b[1;33m \u001b[0mpmf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mthinkbayes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMakePoissonPmf\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mr\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhigh\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstep\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mstep\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 135\u001b[0m \u001b[0mthinkbayes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mSuite\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__init__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpmf\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mname\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 136\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mr\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mC:\\Users\\rlabbe\\Dropbox\\ThinkBayes\\code\\thinkbayes.py\u001b[0m in \u001b[0;36mMakePoissonPmf\u001b[1;34m(lam, high, step)\u001b[0m\n\u001b[0;32m 1563\u001b[0m \u001b[0mpmf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mPmf\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1564\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mk\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhigh\u001b[0m \u001b[1;33m+\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstep\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1565\u001b[1;33m \u001b[0mp\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mEvalPoissonPmf\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlam\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1566\u001b[0m \u001b[0mpmf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mSet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mp\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1567\u001b[0m \u001b[0mpmf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mNormalize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mC:\\Users\\rlabbe\\Dropbox\\ThinkBayes\\code\\thinkbayes.py\u001b[0m in \u001b[0;36mEvalPoissonPmf\u001b[1;34m(k, lam)\u001b[0m\n\u001b[0;32m 1550\u001b[0m \u001b[1;31m# return scipy.stats.poisson.pmf(k, lam)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1552\u001b[1;33m \u001b[1;32mreturn\u001b[0m \u001b[0mlam\u001b[0m \u001b[1;33m**\u001b[0m \u001b[0mk\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0mmath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mexp\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m-\u001b[0m\u001b[0mlam\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m/\u001b[0m \u001b[0mmath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfactorial\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mk\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1553\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1554\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mOverflowError\u001b[0m: int too large to convert to float" + ] + } + ], + "source": [ + "from jaynes import Detector\n", + "\n", + "f = 0.1\n", + "k = 15\n", + "\n", + "\n", + "for r in [100, 250, 400]:\n", + " suite = Detector(r, f, step=1)\n", + " suite.Update(k)\n", + " print(suite.MaximumLikelihood())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure [fig.jaynes1] shows the posterior distribution of $n$ for several\n", + "given values of $r$." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Make it hierarchical\n", + "\n", + "In the previous section, we assume $r$ is known. Now let’s relax that\n", + "assumption. I define another Suite, called `Emitter`, that\n", + "models the behavior of the emitter and estimates $r$:\n", + "\n", + "```python\n", + "class Emitter(thinkbayes.Suite):\n", + "\n", + " def __init__(self, rs, f=0.1):\n", + " detectors = [Detector(r, f) for r in rs]\n", + " thinkbayes.Suite.__init__(self, detectors)\n", + "```\n", + "\n", + "`rs` is a sequence of hypothetical value for $r$.\n", + "`detectors` is a sequence of Detector objects, one for each\n", + "value of $r$. The values in the Suite are Detectors, so Emitter is a\n", + "`**meta-Suite**`; that is, a Suite that contains other Suites\n", + "as values.\n", + "\n", + "To update the Emitter, we have to compute the likelihood of the data\n", + "under each hypothetical value of $r$. But each value of $r$ is\n", + "represented by a Detector that contains a range of values for $n$.\n", + "\n", + "To compute the likelihood of the data for a given Detector, we loop\n", + "through the values of $n$ and add up the total probability of $k$.\n", + "That’s what `SuiteLikelihood` does:\n", + "\n", + "```python\n", + "# class Detector\n", + "\n", + " def SuiteLikelihood(self, data):\n", + " total = 0\n", + " for hypo, prob in self.Items():\n", + " like = self.Likelihood(data, hypo)\n", + " total += prob * like\n", + " return total\n", + "```\n", + "\n", + "Now we can write the Likelihood function for the Emitter:\n", + "\n", + "```python\n", + "# class Detector\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " detector = hypo\n", + " like = detector.SuiteLikelihood(data)\n", + " return like\n", + "```\n", + "\n", + "Each `hypo` is a Detector, so we can invoke\n", + "`SuiteLikelihood` to get the likelihood of the data under the\n", + "hypothesis.\n", + "\n", + "After we update the Emitter, we have to update each of the Detectors,\n", + "too.\n", + "\n", + "```python\n", + "# class Detector\n", + "\n", + " def Update(self, data):\n", + " thinkbayes.Suite.Update(self, data)\n", + "\n", + " for detector in self.Values():\n", + " detector.Update()\n", + "```\n", + "\n", + "A model like this, with multiple levels of Suites, is called\n", + "**hierarchical**." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## A little optimization\n", + "\n", + "You might recognize `SuiteLikelihood`; we saw it in\n", + "Section [suitelike]. At the time, I pointed out that we didn’t really\n", + "need it, because the total probability computed by\n", + "`SuiteLikelihood` is exactly the normalizing constant\n", + "computed and returned by `Update`.\n", + "\n", + "So instead of updating the Emitter and then updating the Detectors, we\n", + "can do both steps at the same time, using the result from\n", + "`Detector.Update` as the likelihood of Emitter.\n", + "\n", + "Here’s the streamlined version of `Emitter.Likelihood`:\n", + "\n", + "```python\n", + "# class Emitter\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " return hypo.Update(data)\n", + "```\n", + "\n", + "And with this version of `Likelihood` we can use the default\n", + "version of `Update`. So this version has fewer lines of code,\n", + "and it runs faster because it does not compute the normalizing constant\n", + "twice." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Extracting the posteriors\n", + "\n", + "![Posterior distributions of $n$ and $r$.](figs/jaynes2.pdf)\n", + "\n", + "[fig.jaynes2]\n", + "\n", + "After we update the Emitter, we can get the posterior distribution of\n", + "$r$ by looping through the Detectors and their probabilities:\n", + "\n", + "```python\n", + "# class Emitter\n", + "\n", + " def DistOfR(self):\n", + " items = [(detector.r, prob) for detector, prob in self.Items()]\n", + " return thinkbayes.MakePmfFromItems(items)\n", + "```\n", + "\n", + "`items` is a list of values of $r$ and their probabilities.\n", + "The result is the Pmf of $r$.\n", + "\n", + "To get the posterior distribution of $n$, we have to compute the mixture\n", + "of the Detectors. We can use `thinkbayes.MakeMixture`, which\n", + "takes a meta-Pmf that maps from each distribution to its probability.\n", + "And that’s exactly what the Emitter is:\n", + "\n", + "```python\n", + "# class Emitter\n", + "\n", + " def DistOfN(self):\n", + " return thinkbayes.MakeMixture(self)\n", + "```\n", + "\n", + "Figure [fig.jaynes2] shows the results. Not surprisingly, the most\n", + "likely value for $n$ is 150. Given $f$ and $n$, the expected count is\n", + "$k = f n$, so given $f$ and $k$, the expected value of $n$ is $k / f$,\n", + "which is 150.\n", + "\n", + "And if 150 particles are emitted in one second, the most likely value of\n", + "$r$ is 150 particles per second. So the posterior distribution of $r$ is\n", + "also centered on 150.\n", + "\n", + "The posterior distributions of $r$ and $n$ are similar; the only\n", + "difference is that we are slightly less certain about $n$. In general,\n", + "we can be more certain about the long-range emission rate, $r$, than\n", + "about the number of particles emitted in any particular second, $n$.\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "The Geiger counter problem demonstrates the connection between causation\n", + "and hierarchical modeling. In the example, the emission rate $r$ has a\n", + "causal effect on the number of particles, $n$, which has a causal effect\n", + "on the particle count, $k$.\n", + "\n", + "The hierarchical model reflects the structure of the system, with causes\n", + "at the top and effects at the bottom.\n", + "\n", + "1. At the top level, we start with a range of hypothetical values for\n", + " $r$.\n", + "\n", + "2. For each value of $r$, we have a range of values for $n$, and the\n", + " prior distribution of $n$ depends on $r$.\n", + "\n", + "3. When we update the model, we go bottom-up. We compute a posterior\n", + " distribution of $n$ for each value of $r$, then compute the\n", + " posterior distribution of $r$.\n", + "\n", + "So causal information flows down the hierarchy, and inference flows up." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "This exercise is also inspired by an example in Jaynes,\n", + "*Probability Theory*.\n", + "\n", + "Suppose you buy a mosquito trap that is supposed to reduce the\n", + "population of mosquitoes near your house. Each week, you empty the trap\n", + "and count the number of mosquitoes captured. After the first week, you\n", + "count 30 mosquitoes. After the second week, you count 20 mosquitoes.\n", + "Estimate the percentage change in the number of mosquitoes in your yard.\n", + "\n", + "To answer this question, you have to make some modeling decisions. Here\n", + "are some suggestions:\n", + "\n", + "- Suppose that each week a large number of mosquitoes, $N$, is bred in\n", + " a wetland near your home.\n", + "\n", + "- During the week, some fraction of them, $f_1$, wander into your\n", + " yard, and of those some fraction, $f_2$, are caught in the trap.\n", + "\n", + "- Your solution should take into account your prior belief about how\n", + " much $N$ is likely to change from one week to the next. You can do\n", + " that by adding a level to the hierarchy to model the percent change\n", + " in $N$." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/15_Dealing_with_Dimensions.ipynb b/notebooks/15_Dealing_with_Dimensions.ipynb index f81908d..bb04a8c 100644 --- a/notebooks/15_Dealing_with_Dimensions.ipynb +++ b/notebooks/15_Dealing_with_Dimensions.ipynb @@ -1,1120 +1,1644 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "Dealing with Dimensions {#species}\n", - "=======================\n", - "\n", - "Belly button bacteria\n", - "---------------------\n", - "\n", - "Belly Button Biodiversity 2.0 (BBB2) is a nation-wide citizen science\n", - "project with the goal of identifying bacterial species that can be found\n", - "in human navels (). The project might\n", - "seem whimsical, but it is part of an increasing interest in the human\n", - "microbiome, the set of microorganisms that live on human skin and parts\n", - "of the body.\n", - "\n", - "In their pilot study, BBB2 researchers collected swabs from the navels\n", - "of 60 volunteers, used multiplex pyrosequencing to extract and sequence\n", - "fragments of 16S rDNA, then identified the species or genus the\n", - "fragments came from. Each identified fragment is called a “read.”\n", - "\n", - "We can use these data to answer several related questions:\n", - "\n", - "- Based on the number of species observed, can we estimate the total\n", - " number of species in the environment?\n", - "\n", - "- Can we estimate the prevalence of each species; that is, the\n", - " fraction of the total population belonging to each species?\n", - "\n", - "- If we are planning to collect additional samples, can we predict how\n", - " many new species we are likely to discover?\n", - "\n", - "- How many additional reads are needed to increase the fraction of\n", - " observed species to a given threshold?\n", - "\n", - "These questions make up what is called the **Unseen Species\n", - "problem**.\n", - "\n", - "Lions and tigers and bears\n", - "--------------------------\n", - "\n", - "I’ll start with a simplified version of the problem where we know that\n", - "there are exactly three species. Let’s call them lions, tigers and\n", - "bears. Suppose we visit a wild animal preserve and see 3 lions, 2 tigers\n", - "and one bear.\n", - "\n", - "If we have an equal chance of observing any animal in the preserve, the\n", - "number of each species we see is governed by the multinomial\n", - "distribution. If the prevalence of lions and tigers and bears is\n", - "`p_lion` and `p_tiger` and `p_bear`, the likelihood of seeing 3 lions, 2\n", - "tigers and one bear is\n", - "\n", - " p_lion**3 * p_tiger**2 * p_bear**1\n", - "\n", - "An approach that is tempting, but not correct, is to use beta\n", - "distributions, as in Section [beta], to describe the prevalence of each\n", - "species separately. For example, we saw 3 lions and 3 non-lions; if we\n", - "think of that as 3 “heads” and 3 “tails,” then the posterior\n", - "distribution of `p_lion` is:\n", - "\n", - " beta = thinkbayes.Beta()\n", - " beta.Update((3, 3))\n", - " print beta.MaximumLikelihood()\n", - "\n", - "The maximum likelihood estimate for `p_lion` is the observed rate, 50%.\n", - "Similarly the MLEs for `p_tiger` and `p_bear` are 33% and 17%.\n", - "\n", - "But there are two problems:\n", - "\n", - "1. We have implicitly used a prior for each species that is uniform\n", - " from 0 to 1, but since we know that there are three species, that\n", - " prior is not correct. The right prior should have a mean of 1/3, and\n", - " there should be zero likelihood that any species has a prevalence of\n", - " 100%.\n", - "\n", - "2. The distributions for each species are not independent, because the\n", - " prevalences have to add up to 1. To capture this dependence, we need\n", - " a joint distribution for the three prevalences.\n", - "\n", - "We can use a Dirichlet distribution to solve both of these problems (see\n", - "). In the same way\n", - "we used the beta distribution to describe the distribution of bias for a\n", - "coin, we can use a Dirichlet distribution to describe the joint\n", - "distribution of `p_lion`, `p_tiger` and `p_bear`.\n", - "\n", - "The Dirichlet distribution is the multi-dimensional generalization of\n", - "the beta distribution. Instead of two possible outcomes, like heads and\n", - "tails, the Dirichlet distribution handles any number of outcomes: in\n", - "this example, three species.\n", - "\n", - "If there are n outcomes, the Dirichlet distribution is\n", - "described by n parameters, written $\\alpha_1$ through\n", - "$\\alpha_n$.\n", - "\n", - "Here’s the definition, from thinkbayes.py, of a class that\n", - "represents a Dirichlet distribution:\n", - "\n", - " class Dirichlet(object):\n", - "\n", - " def __init__(self, n):\n", - " self.n = n\n", - " self.params = numpy.ones(n, dtype=numpy.int)\n", - "\n", - "n is the number of dimensions; initially the parameters are\n", - "all 1. I use a numpy array to store the parameters so I can\n", - "take advantage of array operations.\n", - "\n", - "Given a Dirichlet distribution, the marginal distribution for each\n", - "prevalence is a beta distribution, which we can compute like this:\n", - "\n", - " def MarginalBeta(self, i):\n", - " alpha0 = self.params.sum()\n", - " alpha = self.params[i]\n", - " return Beta(alpha, alpha0-alpha)\n", - "\n", - "i is the index of the marginal distribution we want.\n", - "alpha0 is the sum of the parameters; alpha is\n", - "the parameter for the given species.\n", - "\n", - "In the example, the prior marginal distribution for each species is\n", - "Beta(1, 2). We can compute the prior means like this:\n", - "\n", - " dirichlet = thinkbayes.Dirichlet(3)\n", - " for i in range(3):\n", - " beta = dirichlet.MarginalBeta(i)\n", - " print beta.Mean()\n", - "\n", - "As expected, the prior mean prevalence for each species is 1/3.\n", - "\n", - "To update the Dirichlet distribution, we add the observations to the\n", - "parameters like this:\n", - "\n", - " def Update(self, data):\n", - " m = len(data)\n", - " self.params[:m] += data\n", - "\n", - "Here data is a sequence of counts in the same order as\n", - "params, so in this example, it should be the number of\n", - "lions, tigers and bears.\n", - "\n", - "data can be shorter than params; in that case\n", - "there are some species that have not been observed.\n", - "\n", - "Here’s code that updates dirichlet with the observed data\n", - "and computes the posterior marginal distributions.\n", - "\n", - " data = [3, 2, 1]\n", - " dirichlet.Update(data)\n", - "\n", - " for i in range(3):\n", - " beta = dirichlet.MarginalBeta(i)\n", - " pmf = beta.MakePmf()\n", - " print i, pmf.Mean()\n", - "\n", - "![Distribution of prevalences for three species.](figs/species1.pdf)\n", - "\n", - "[fig.species1]\n", - "\n", - "Figure [fig.species1] shows the results. The posterior mean prevalences\n", - "are 44%, 33%, and 22%.\n", - "\n", - "The hierarchical version\n", - "------------------------\n", - "\n", - "We have solved a simplified version of the problem: if we know how many\n", - "species there are, we can estimate the prevalence of each.\n", - "\n", - "Now let’s get back to the original problem, estimating the total number\n", - "of species. To solve this problem I’ll define a meta-Suite, which is a\n", - "Suite that contains other Suites as hypotheses. In this case, the\n", - "top-level Suite contains hypotheses about the number of species; the\n", - "bottom level contains hypotheses about prevalences.\n", - "\n", - "Here’s the class definition:\n", - "\n", - " class Species(thinkbayes.Suite):\n", - "\n", - " def __init__(self, ns):\n", - " hypos = [thinkbayes.Dirichlet(n) for n in ns]\n", - " thinkbayes.Suite.__init__(self, hypos)\n", - "\n", - "`__init__` takes a list of possible values for n and makes\n", - "a list of Dirichlet objects.\n", - "\n", - "Here’s the code that creates the top-level suite:\n", - "\n", - " ns = range(3, 30)\n", - " suite = Species(ns)\n", - "\n", - "ns is the list of possible values for n. We\n", - "have seen 3 species, so there have to be at least that many. I chose an\n", - "upper bound that seems reasonable, but we will check later that the\n", - "probability of exceeding this bound is low. And at least initially we\n", - "assume that any value in this range is equally likely.\n", - "\n", - "To update a hierarchical model, you have to update all levels. Usually\n", - "you have to update the bottom level first and work up, but in this case\n", - "we can update the top level first:\n", - "\n", - " #class Species\n", - "\n", - " def Update(self, data):\n", - " thinkbayes.Suite.Update(self, data)\n", - " for hypo in self.Values():\n", - " hypo.Update(data)\n", - "\n", - "Species.Update invokes Update in the parent\n", - "class, then loops through the sub-hypotheses and updates them.\n", - "\n", - "Now all we need is a likelihood function:\n", - "\n", - " # class Species\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " dirichlet = hypo\n", - " like = 0\n", - " for i in range(1000):\n", - " like += dirichlet.Likelihood(data)\n", - "\n", - " return like\n", - "\n", - "data is a sequence of observed counts; hypo is\n", - "a Dirichlet object. Species.Likelihood calls\n", - "Dirichlet.Likelihood 1000 times and returns the total.\n", - "\n", - "Why call it 1000 times? Because Dirichlet.Likelihood\n", - "doesn’t actually compute the likelihood of the data under the whole\n", - "Dirichlet distribution. Instead, it draws one sample from the\n", - "hypothetical distribution and computes the likelihood of the data under\n", - "the sampled set of prevalences.\n", - "\n", - "Here’s what it looks like:\n", - "\n", - " # class Dirichlet\n", - "\n", - " def Likelihood(self, data):\n", - " m = len(data)\n", - " if self.n < m:\n", - " return 0\n", - "\n", - " x = data\n", - " p = self.Random()\n", - " q = p[:m]**x\n", - " return q.prod()\n", - "\n", - "The length of data is the number of species observed. If we\n", - "see more species than we thought existed, the likelihood is 0.\n", - "\n", - "Otherwise we select a random set of prevalences, p, and\n", - "compute the multinomial PMF, which is\n", - "$$c_x p_1^{x_1} \\cdots p_n^{x_n}$$ $p_i$ is the prevalence of the $i$th\n", - "species, and $x_i$ is the observed number. The first term, $c_x$, is the\n", - "multinomial coefficient; I leave it out of the computation because it is\n", - "a multiplicative factor that depends only on the data, not the\n", - "hypothesis, so it gets normalized away (see\n", - ").\n", - "\n", - "m is the number of observed species. We only need the first\n", - "m elements of p; for the others, $x_i$ is 0,\n", - "so $p_i^{x_i}$ is 1, and we can leave them out of the product.\n", - "\n", - "Random sampling {#randomdir}\n", - "---------------\n", - "\n", - "There are two ways to generate a random sample from a Dirichlet\n", - "distribution. One is to use the marginal beta distributions, but in that\n", - "case you have to select one at a time and scale the rest so they add up\n", - "to 1 (see\n", - ").\n", - "\n", - "A less obvious, but faster, way is to select values from n\n", - "gamma distributions, then normalize by dividing through by the total.\n", - "Here’s the code:\n", - "\n", - " # class Dirichlet\n", - "\n", - " def Random(self):\n", - " p = numpy.random.gamma(self.params)\n", - " return p / p.sum()\n", - "\n", - "Now we’re ready to look at some results. Here is the code that extracts\n", - "the posterior distribution of n:\n", - "\n", - " def DistOfN(self):\n", - " pmf = thinkbayes.Pmf()\n", - " for hypo, prob in self.Items():\n", - " pmf.Set(hypo.n, prob)\n", - " return pmf\n", - "\n", - "DistOfN iterates through the top-level hypotheses and\n", - "accumulates the probability of each n.\n", - "\n", - "![Posterior distribution of n.](figs/species2.pdf)\n", - "\n", - "[fig.species2]\n", - "\n", - "Figure [fig.species2] shows the result. The most likely value is 4.\n", - "Values from 3 to 7 are reasonably likely; after that the probabilities\n", - "drop off quickly. The probability that there are 29 species is low\n", - "enough to be negligible; if we chose a higher bound, we would get nearly\n", - "the same result.\n", - "\n", - "Remember that this result is based on a uniform prior for\n", - "n. If we have background information about the number of\n", - "species in the environment, we might choose a different prior.\n", - "\n", - "Optimization\n", - "------------\n", - "\n", - "I have to admit that I am proud of this example. The Unseen Species\n", - "problem is not easy, and I think this solution is simple and clear, and\n", - "takes surprisingly few lines of code (about 50 so far).\n", - "\n", - "The only problem is that it is slow. It’s good enough for the example\n", - "with only 3 observed species, but not good enough for the belly button\n", - "data, with more than 100 species in some samples.\n", - "\n", - "The next few sections present a series of optimizations we need to make\n", - "this solution scale. Before we get into the details, here’s a road map.\n", - "\n", - "- The first step is to recognize that if we update the Dirichlet\n", - " distributions with the same data, the first m\n", - " parameters are the same for all of them. The only difference is the\n", - " number of hypothetical unseen species. So we don’t really need\n", - " n Dirichlet objects; we can store the parameters in the\n", - " top level of the hierarchy. Species2 implements this\n", - " optimization.\n", - "\n", - "- Species2 also uses the same set of random values for\n", - " all of the hypotheses. This saves time generating random values, but\n", - " it has a second benefit that turns out to be more important: by\n", - " giving all hypotheses the same selection from the sample space, we\n", - " make the comparison between the hypotheses more fair, so it takes\n", - " fewer iterations to converge.\n", - "\n", - "- Even with these changes there is a major performance problem. As the\n", - " number of observed species increases, the array of random\n", - " prevalences gets bigger, and the chance of choosing one that is\n", - " approximately right becomes small. So the vast majority of\n", - " iterations yield small likelihoods that don’t contribute much to the\n", - " total, and don’t discriminate between hypotheses.\n", - "\n", - " The solution is to do the updates one species at a time.\n", - " Species4 is a simple implementation of this strategy\n", - " using Dirichlet objects to represent the sub-hypotheses.\n", - "\n", - "- Finally, Species5 combines the sub-hypotheses into the\n", - " top level and uses numpy array operations to speed\n", - " things up.\n", - "\n", - "If you are not interested in the details, feel free to skip to\n", - "Section [belly] where we look at results from the belly button data.\n", - "\n", - "Collapsing the hierarchy {#collapsing}\n", - "------------------------\n", - "\n", - "All of the bottom-level Dirichlet distributions are updated with the\n", - "same data, so the first m parameters are the same for all\n", - "of them. We can eliminate them and merge the parameters into the\n", - "top-level suite. Species2 implements this optimization:\n", - "\n", - " class Species2(object):\n", - " \n", - " def __init__(self, ns):\n", - " self.ns = ns\n", - " self.probs = numpy.ones(len(ns), dtype=numpy.double)\n", - " self.params = numpy.ones(self.high, dtype=numpy.int)\n", - "\n", - "ns is the list of hypothetical values for n;\n", - "probs is the list of corresponding probabilities. And\n", - "params is the sequence of Dirichlet parameters, initially\n", - "all 1.\n", - "\n", - "Species2.Update updates both levels of the hierarchy: first\n", - "the probability for each value of n, then the Dirichlet\n", - "parameters:\n", - "\n", - " # class Species2\n", - "\n", - " def Update(self, data):\n", - " like = numpy.zeros(len(self.ns), dtype=numpy.double)\n", - " for i in range(1000):\n", - " like += self.SampleLikelihood(data)\n", - "\n", - " self.probs *= like\n", - " self.probs /= self.probs.sum()\n", - "\n", - " m = len(data)\n", - " self.params[:m] += data\n", - "\n", - "SampleLikelihood returns an array of likelihoods, one for\n", - "each value of n. like accumulates the total\n", - "likelihood for 1000 samples. self.probs is multiplied by\n", - "the total likelihood, then normalized. The last two lines, which update\n", - "the parameters, are the same as in Dirichlet.Update.\n", - "\n", - "Now let’s look at SampleLikelihood. There are two\n", - "opportunities for optimization here:\n", - "\n", - "- When the hypothetical number of species, n, exceeds the\n", - " observed number, m, we only need the first\n", - " m terms of the multinomial PMF; the rest are 1.\n", - "\n", - "- If the number of species is large, the likelihood of the data might\n", - " be too small for floating-point (see  [underflow]). So it is safer\n", - " to compute log-likelihoods.\n", - "\n", - "Again, the multinomial PMF is $$c_x p_1^{x_1} \\cdots p_n^{x_n}$$ So the\n", - "log-likelihood is $$\\log c_x + x_1 \\log p_1 + \\cdots + x_n \\log p_n$$\n", - "which is fast and easy to compute. Again, $c_x$ it is the same for all\n", - "hypotheses, so we can drop it. Here’s the code:\n", - "\n", - " # class Species2\n", - "\n", - " def SampleLikelihood(self, data):\n", - " gammas = numpy.random.gamma(self.params)\n", - "\n", - " m = len(data)\n", - " row = gammas[:m]\n", - " col = numpy.cumsum(gammas)\n", - "\n", - " log_likes = []\n", - " for n in self.ns:\n", - " ps = row / col[n-1]\n", - " terms = data * numpy.log(ps)\n", - " log_like = terms.sum()\n", - " log_likes.append(log_like)\n", - "\n", - " log_likes -= numpy.max(log_likes)\n", - " likes = numpy.exp(log_likes)\n", - "\n", - " coefs = [thinkbayes.BinomialCoef(n, m) for n in self.ns]\n", - " likes *= coefs\n", - "\n", - " return likes\n", - "\n", - "gammas is an array of values from a gamma distribution; its\n", - "length is the largest hypothetical value of n.\n", - "row is just the first m elements of\n", - "gammas; since these are the only elements that depend on\n", - "the data, they are the only ones we need.\n", - "\n", - "For each value of n we need to divide row by\n", - "the total of the first n values from gamma.\n", - "cumsum computes these cumulative sums and stores them in\n", - "col.\n", - "\n", - "The loop iterates through the values of n and accumulates a\n", - "list of log-likelihoods.\n", - "\n", - "Inside the loop, ps contains the row of probabilities,\n", - "normalized with the appropriate cumulative sum. terms\n", - "contains the terms of the summation, $x_i \\log p_i$, and `log_like`\n", - "contains their sum.\n", - "\n", - "After the loop, we want to convert the log-likelihoods to linear\n", - "likelihoods, but first it’s a good idea to shift them so the largest\n", - "log-likelihood is 0; that way the linear likelihoods are not too small\n", - "(see  [underflow]).\n", - "\n", - "Finally, before we return the likelihood, we have to apply a correction\n", - "factor, which is the number of ways we could have observed these\n", - "m species, if the total number of species is\n", - "n. BinomialCoefficient computes “n choose m”,\n", - "which is written $\\binom{n}{m}$.\n", - "\n", - "As often happens, the optimized version is less readable and more\n", - "error-prone than the original. But that’s one reason I think it is a\n", - "good idea to start with the simple version; we can use it for regression\n", - "testing. I plotted results from both versions and confirmed that they\n", - "are approximately equal, and that they converge as the number of\n", - "iterations increases.\n", - "\n", - "One more problem\n", - "----------------\n", - "\n", - "There’s more we could do to optimize this code, but there’s another\n", - "problem we need to fix first. As the number of observed species\n", - "increases, this version gets noisier and takes more iterations to\n", - "converge on a good answer.\n", - "\n", - "The problem is that if the prevalences we choose from the Dirichlet\n", - "distribution, the ps, are not at least approximately right,\n", - "the likelihood of the observed data is close to zero and almost equally\n", - "bad for all values of n. So most iterations don’t provide\n", - "any useful contribution to the total likelihood. And as the number of\n", - "observed species, m, gets large, the probability of\n", - "choosing ps with non-negligible likelihood gets small.\n", - "Really small.\n", - "\n", - "Fortunately, there is a solution. Remember that if you observe a set of\n", - "data, you can update the prior distribution with the entire dataset, or\n", - "you can break it up into a series of updates with subsets of the data,\n", - "and the result is the same either way.\n", - "\n", - "For this example, the key is to perform the updates one species at a\n", - "time. That way when we generate a random set of ps, only\n", - "one of them affects the computed likelihood, so the chance of choosing a\n", - "good one is much better.\n", - "\n", - "Here’s a new version that updates one species at a time:\n", - "\n", - " class Species4(Species):\n", - "\n", - " def Update(self, data):\n", - " m = len(data)\n", - "\n", - " for i in range(m):\n", - " one = numpy.zeros(i+1)\n", - " one[i] = data[i] \n", - " Species.Update(self, one)\n", - "\n", - "This version inherits `__init__` from Species, so it\n", - "represents the hypotheses as a list of Dirichlet objects (unlike\n", - "Species2).\n", - "\n", - "Update loops through the observed species and makes an\n", - "array, one, with all zeros and one species count. Then it\n", - "calls Update in the parent class, which computes the\n", - "likelihoods and updates the sub-hypotheses.\n", - "\n", - "So in the running example, we do three updates. The first is something\n", - "like “I have seen three lions.” The second is “I have seen two tigers\n", - "and no additional lions.” And the third is “I have seen one bear and no\n", - "more lions and tigers.”\n", - "\n", - "Here’s the new version of Likelihood:\n", - "\n", - " # class Species4\n", - "\n", - " def Likelihood(self, data, hypo):\n", - " dirichlet = hypo\n", - " like = 0\n", - " for i in range(self.iterations):\n", - " like += dirichlet.Likelihood(data)\n", - "\n", - " # correct for the number of unseen species the new one\n", - " # could have been\n", - " m = len(data)\n", - " num_unseen = dirichlet.n - m + 1\n", - " like *= num_unseen\n", - "\n", - " return like\n", - "\n", - "This is almost the same as Species.Likelihood. The\n", - "difference is the factor, `num_unseen`. This correction is necessary\n", - "because each time we see a species for the first time, we have to\n", - "consider that there were some number of other unseen species that we\n", - "might have seen. For larger values of n there are more\n", - "unseen species that we could have seen, which increases the likelihood\n", - "of the data.\n", - "\n", - "This is a subtle point and I have to admit that I did not get it right\n", - "the first time. But again I was able to validate this version by\n", - "comparing it to the previous versions.\n", - "\n", - "We’re not done yet\n", - "------------------\n", - "\n", - "Performing the updates one species at a time solves one problem, but it\n", - "creates another. Each update takes time proportional to $k m$, where $k$\n", - "is the number of hypotheses and $m$ is the number of observed species.\n", - "So if we do $m$ updates, the total run time is proportional to $k m^2$.\n", - "\n", - "But we can speed things up using the same trick we used in\n", - "Section [collapsing]: we’ll get rid of the Dirichlet objects and\n", - "collapse the two levels of the hierarchy into a single object. So here’s\n", - "yet another version of Species:\n", - "\n", - " class Species5(Species2):\n", - " \n", - " def Update(self, data):\n", - " m = len(data)\n", - " for i in range(m):\n", - " self.UpdateOne(i+1, data[i])\n", - " self.params[i] += data[i]\n", - "\n", - "This version inherits `__init__` from Species2, so it uses\n", - "ns and probs to represent the distribution of\n", - "n, and params to represent the parameters of\n", - "the Dirichlet distribution.\n", - "\n", - "Update is similar to what we saw in the previous section.\n", - "It loops through the observed species and calls UpdateOne:\n", - "\n", - " # class Species5\n", - "\n", - " def UpdateOne(self, i, count):\n", - " likes = numpy.zeros(len(self.ns), dtype=numpy.double)\n", - " for i in range(self.iterations):\n", - " likes += self.SampleLikelihood(i, count)\n", - "\n", - " unseen_species = [n-i+1 for n in self.ns]\n", - " likes *= unseen_species\n", - "\n", - " self.probs *= likes\n", - " self.probs /= self.probs.sum()\n", - "\n", - "This function is similar to Species2.Update, with two\n", - "changes:\n", - "\n", - "- The interface is different. Instead of the whole dataset, we get\n", - " i, the index of the observed species, and\n", - " count, how many of that species we’ve seen.\n", - "\n", - "- We have to apply a correction factor for the number of unseen\n", - " species, as in Species4.Likelihood. The difference here\n", - " is that we update all of the likelihoods at once with array\n", - " multiplication.\n", - "\n", - "Finally, here’s SampleLikelihood:\n", - "\n", - " # class Species5\n", - "\n", - " def SampleLikelihood(self, i, count):\n", - " gammas = numpy.random.gamma(self.params)\n", - "\n", - " sums = numpy.cumsum(gammas)[self.ns[0]-1:]\n", - "\n", - " ps = gammas[i-1] / sums\n", - " log_likes = numpy.log(ps) * count\n", - "\n", - " log_likes -= numpy.max(log_likes)\n", - " likes = numpy.exp(log_likes)\n", - "\n", - " return likes\n", - "\n", - "This is similar to Species2.SampleLikelihood; the\n", - "difference is that each update only includes a single species, so we\n", - "don’t need a loop.\n", - "\n", - "The runtime of this function is proportional to the number of\n", - "hypotheses, $k$. It runs $m$ times, so the run time of the update is\n", - "proportional to $k m$. And the number of iterations we need to get an\n", - "accurate result is usually small.\n", - "\n", - "The belly button data {#belly}\n", - "---------------------\n", - "\n", - "That’s enough about lions and tigers and bears. Let’s get back to belly\n", - "buttons. To get a sense of what the data look like, consider subject\n", - "B1242, whose sample of 400 reads yielded 61 species with the following\n", - "counts:\n", - "\n", - " 92, 53, 47, 38, 15, 14, 12, 10, 8, 7, 7, 5, 5, \n", - " 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2,\n", - " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n", - " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1\n", - "\n", - "There are a few dominant species that make up a large fraction of the\n", - "whole, but many species that yielded only a single read. The number of\n", - "these “singletons” suggests that there are likely to be at least a few\n", - "unseen species.\n", - "\n", - "In the example with lions and tigers, we assume that each animal in the\n", - "preserve is equally likely to be observed. Similarly, for the belly\n", - "button data, we assume that each bacterium is equally likely to yield a\n", - "read.\n", - "\n", - "In reality, each step in the data-collection process might introduce\n", - "biases. Some species might be more likely to be picked up by a swab, or\n", - "to yield identifiable amplicons. So when we talk about the prevalence of\n", - "each species, we should remember this source of error.\n", - "\n", - "I should also acknowledge that I am using the term “species” loosely.\n", - "First, bacterial species are not well defined. Second, some reads\n", - "identify a particular species, others only identify a genus. To be more\n", - "precise, I should say “operational taxonomic unit”, or OTU.\n", - "\n", - "Now let’s process some of the belly button data. I define a class called\n", - "Subject to represent information about each subject in the\n", - "study:\n", - "\n", - " class Subject(object):\n", - "\n", - " def __init__(self, code):\n", - " self.code = code\n", - " self.species = []\n", - "\n", - "Each subject has a string code, like “B1242”, and a list of (count,\n", - "species name) pairs, sorted in increasing order by count.\n", - "Subject provides several methods to make it easy to access\n", - "these counts and species names. You can see the details in\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "![Distribution of n for subject\n", - "B1242.](figs/species-ndist-B1242.pdf)\n", - "\n", - "[species-ndist]\n", - "\n", - "Subject provides a method named Process that\n", - "creates and updates a Species5 suite, which represents the\n", - "distributions of n and the prevalences.\n", - "\n", - "And Suite2 provides DistOfN, which returns the\n", - "posterior distribution of n.\n", - "\n", - " # class Suite2\n", - "\n", - " def DistN(self):\n", - " items = zip(self.ns, self.probs)\n", - " pmf = thinkbayes.MakePmfFromItems(items)\n", - " return pmf\n", - "\n", - "Figure [species-ndist] shows the distribution of n for\n", - "subject B1242. The probability that there are exactly 61 species, and no\n", - "unseen species, is nearly zero. The most likely value is 72, with 90%\n", - "credible interval 66 to 79. At the high end, it is unlikely that there\n", - "are as many as 87 species.\n", - "\n", - "Next we compute the posterior distribution of prevalence for each\n", - "species. Species2 provides DistOfPrevalence:\n", - "\n", - " # class Species2\n", - "\n", - " def DistOfPrevalence(self, index):\n", - " metapmf = thinkbayes.Pmf()\n", - "\n", - " for n, prob in zip(self.ns, self.probs):\n", - " beta = self.MarginalBeta(n, index)\n", - " pmf = beta.MakePmf()\n", - " metapmf.Set(pmf, prob)\n", - "\n", - " mix = thinkbayes.MakeMixture(metapmf)\n", - " return metapmf, mix\n", - "\n", - "index indicates which species we want. For each\n", - "n, we have a different posterior distribution of\n", - "prevalence.\n", - "\n", - "![Distribution of prevalences for subject\n", - "B1242.](figs/species-prev-B1242.pdf)\n", - "\n", - "[species-prev]\n", - "\n", - "The loop iterates through the possible values of n and\n", - "their probabilities. For each value of n it gets a Beta\n", - "object representing the marginal distribution for the indicated species.\n", - "Remember that Beta objects contain the parameters alpha and\n", - "beta; they don’t have values and probabilities like a Pmf,\n", - "but they provide MakePmf, which generates a discrete\n", - "approximation to the continuous beta distribution.\n", - "\n", - "metapmf is a meta-Pmf that contains the distributions of\n", - "prevalence, conditioned on n. MakeMixture\n", - "combines the meta-Pmf into mix, which combines the\n", - "conditional distributions into a single distribution of prevalence.\n", - "\n", - "Figure [species-prev] shows results for the five species with the most\n", - "reads. The most prevalent species accounts for 23% of the 400 reads, but\n", - "since there are almost certainly unseen species, the most likely\n", - "estimate for its prevalence is 20%, with 90% credible interval between\n", - "17% and 23%.\n", - "\n", - "Predictive distributions\n", - "------------------------\n", - "\n", - "![Simulated rarefaction curves for subject\n", - "B1242.](figs/species-rare-B1242.pdf)\n", - "\n", - "[species-rare]\n", - "\n", - "I introduced the hidden species problem in the form of four related\n", - "questions. We have answered the first two by computing the posterior\n", - "distribution for n and the prevalence of each species.\n", - "\n", - "The other two questions are:\n", - "\n", - "- If we are planning to collect additional reads, can we predict how\n", - " many new species we are likely to discover?\n", - "\n", - "- How many additional reads are needed to increase the fraction of\n", - " observed species to a given threshold?\n", - "\n", - "To answer predictive questions like this we can use the posterior\n", - "distributions to simulate possible future events and compute predictive\n", - "distributions for the number of species, and fraction of the total, we\n", - "are likely to see.\n", - "\n", - "The kernel of these simulations looks like this:\n", - "\n", - "1. Choose n from its posterior distribution.\n", - "\n", - "2. Choose a prevalence for each species, including possible unseen\n", - " species, using the Dirichlet distribution.\n", - "\n", - "3. Generate a random sequence of future observations.\n", - "\n", - "4. Compute the number of new species, `num_new`, as a function of the\n", - " number of additional reads, k.\n", - "\n", - "5. Repeat the previous steps and accumulate the joint distribution of\n", - " `num_new` and k.\n", - "\n", - "And here’s the code. RunSimulation runs a single\n", - "simulation:\n", - "\n", - " # class Subject\n", - "\n", - " def RunSimulation(self, num_reads):\n", - " m, seen = self.GetSeenSpecies()\n", - " n, observations = self.GenerateObservations(num_reads)\n", - "\n", - " curve = []\n", - " for k, obs in enumerate(observations):\n", - " seen.add(obs)\n", - "\n", - " num_new = len(seen) - m\n", - " curve.append((k+1, num_new))\n", - "\n", - " return curve\n", - "\n", - "`num_reads` is the number of additional reads to simulate.\n", - "m is the number of seen species, and seen is a\n", - "set of strings with a unique name for each species. n is a\n", - "random value from the posterior distribution, and\n", - "observations is a random sequence of species names.\n", - "\n", - "Each time through the loop, we add the new observation to\n", - "seen and record the number of reads and the number of new\n", - "species so far.\n", - "\n", - "The result of RunSimulation is a **rarefaction\n", - "curve**, represented as a list of pairs with the number of reads\n", - "and the number of new species.\n", - "\n", - "Before we see the results, let’s look at GetSeenSpecies and\n", - "GenerateObservations.\n", - "\n", - " #class Subject\n", - "\n", - " def GetSeenSpecies(self):\n", - " names = self.GetNames()\n", - " m = len(names)\n", - " seen = set(SpeciesGenerator(names, m))\n", - " return m, seen\n", - "\n", - "GetNames returns the list of species names that appear in\n", - "the data files, but for many subjects these names are not unique. So I\n", - "use SpeciesGenerator to extend each name with a serial\n", - "number:\n", - "\n", - " def SpeciesGenerator(names, num):\n", - " i = 0\n", - " for name in names:\n", - " yield '%s-%d' % (name, i)\n", - " i += 1\n", - "\n", - " while i < num:\n", - " yield 'unseen-%d' % i\n", - " i += 1\n", - "\n", - "Given a name like Corynebacterium,\n", - "SpeciesGenerator yields Corynebacterium-1.\n", - "When the list of names is exhausted, it yields names like\n", - "unseen-62.\n", - "\n", - "Here is GenerateObservations:\n", - "\n", - " # class Subject\n", - "\n", - " def GenerateObservations(self, num_reads):\n", - " n, prevalences = self.suite.SamplePosterior()\n", - "\n", - " names = self.GetNames()\n", - " name_iter = SpeciesGenerator(names, n)\n", - "\n", - " d = dict(zip(name_iter, prevalences))\n", - " cdf = thinkbayes.MakeCdfFromDict(d)\n", - " observations = cdf.Sample(num_reads)\n", - "\n", - " return n, observations\n", - "\n", - "Again, `num_reads` is the number of additional reads to generate.\n", - "n and prevalences are samples from the\n", - "posterior distribution.\n", - "\n", - "cdf is a Cdf object that maps species names, including the\n", - "unseen, to cumulative probabilities. Using a Cdf makes it efficient to\n", - "generate a random sequence of species names.\n", - "\n", - "Finally, here is Species2.SamplePosterior:\n", - "\n", - " def SamplePosterior(self):\n", - " pmf = self.DistOfN()\n", - " n = pmf.Random()\n", - " prevalences = self.SamplePrevalences(n)\n", - " return n, prevalences\n", - "\n", - "And SamplePrevalences, which generates a sample of\n", - "prevalences conditioned on n:\n", - "\n", - " # class Species2\n", - "\n", - " def SamplePrevalences(self, n):\n", - " params = self.params[:n]\n", - " gammas = numpy.random.gamma(params)\n", - " gammas /= gammas.sum()\n", - " return gammas\n", - "\n", - "We saw this algorithm for generating random values from a Dirichlet\n", - "distribution in Section [randomdir].\n", - "\n", - "Figure [species-rare] shows 100 simulated rarefaction curves for subject\n", - "B1242. The curves are “jittered;” that is, I shifted each curve by a\n", - "random offset so they would not all overlap. By inspection we can\n", - "estimate that after 400 more reads we are likely to find 2–6 new\n", - "species.\n", - "\n", - "Joint posterior\n", - "---------------\n", - "\n", - "![Distributions of the number of new species conditioned on the number\n", - "of additional reads.](figs/species-cond-B1242.pdf)\n", - "\n", - "[species-cond]\n", - "\n", - "We can use these simulations to estimate the joint distribution of\n", - "`num_new` and k, and from that we can get the distribution\n", - "of `num_new` conditioned on any value of k.\n", - "\n", - " def MakeJointPredictive(curves):\n", - " joint = thinkbayes.Joint()\n", - " for curve in curves:\n", - " for k, num_new in curve:\n", - " joint.Incr((k, num_new))\n", - " joint.Normalize()\n", - " return joint\n", - "\n", - "MakeJointPredictive makes a Joint object, which is a Pmf\n", - "whose values are tuples.\n", - "\n", - "curves is a list of rarefaction curves created by\n", - "RunSimulation. Each curve contains a list of pairs of\n", - "k and `num_new`.\n", - "\n", - "The resulting joint distribution is a map from each pair to its\n", - "probability of occurring. Given the joint distribution, we can use\n", - "Joint.Conditional get the distribution of `num_new`\n", - "conditioned on k (see Section [conditional]).\n", - "\n", - "Subject.MakeConditionals takes a list of ks\n", - "and computes the conditional distribution of `num_new` for each\n", - "k. The result is a list of Cdf objects.\n", - "\n", - " def MakeConditionals(curves, ks):\n", - " joint = MakeJointPredictive(curves)\n", - "\n", - " cdfs = []\n", - " for k in ks:\n", - " pmf = joint.Conditional(1, 0, k)\n", - " pmf.name = 'k=%d' % k\n", - " cdf = pmf.MakeCdf()\n", - " cdfs.append(cdf)\n", - "\n", - " return cdfs\n", - "\n", - "Figure [species-cond] shows the results. After 100 reads, the median\n", - "predicted number of new species is 2; the 90% credible interval is 0 to\n", - "5. After 800 reads, we expect to see 3 to 12 new species.\n", - "\n", - "Coverage\n", - "--------\n", - "\n", - "![Complementary CDF of coverage for a range of additional\n", - "reads.](figs/species-frac-B1242.pdf)\n", - "\n", - "[species-frac]\n", - "\n", - "The last question we want to answer is, “How many additional reads are\n", - "needed to increase the fraction of observed species to a given\n", - "threshold?”\n", - "\n", - "To answer this question, we need a version of RunSimulation\n", - "that computes the fraction of observed species rather than the number of\n", - "new species.\n", - "\n", - " # class Subject\n", - "\n", - " def RunSimulation(self, num_reads):\n", - " m, seen = self.GetSeenSpecies()\n", - " n, observations = self.GenerateObservations(num_reads)\n", - "\n", - " curve = []\n", - " for k, obs in enumerate(observations):\n", - " seen.add(obs)\n", - "\n", - " frac_seen = len(seen) / float(n)\n", - " curve.append((k+1, frac_seen))\n", - "\n", - " return curve\n", - "\n", - "Next we loop through each curve and make a dictionary, d,\n", - "that maps from the number of additional reads, k, to a list\n", - "of fracs; that is, a list of values for the coverage\n", - "achieved after k reads.\n", - "\n", - " def MakeFracCdfs(self, curves):\n", - " d = {}\n", - " for curve in curves:\n", - " for k, frac in curve:\n", - " d.setdefault(k, []).append(frac)\n", - "\n", - " cdfs = {}\n", - " for k, fracs in d.iteritems():\n", - " cdf = thinkbayes.MakeCdfFromList(fracs)\n", - " cdfs[k] = cdf\n", - "\n", - " return cdfs\n", - "\n", - "Then for each value of k we make a Cdf of\n", - "fracs; this Cdf represents the distribution of coverage\n", - "after k reads.\n", - "\n", - "Remember that the CDF tells you the probability of falling below a given\n", - "threshold, so the *complementary* CDF tells you the\n", - "probability of exceeding it. Figure [species-frac] shows complementary\n", - "CDFs for a range of values of k.\n", - "\n", - "To read this figure, select the level of coverage you want to achieve\n", - "along the $x$-axis. As an example, choose 90%.\n", - "\n", - "Now you can read up the chart to find the probability of achieving 90%\n", - "coverage after k reads. For example, with 200 reads, you\n", - "have about a 40% chance of getting 90% coverage. With 1000 reads, you\n", - "have a 90% chance of getting 90% coverage.\n", - "\n", - "With that, we have answered the four questions that make up the unseen\n", - "species problem. To validate the algorithms in this chapter with real\n", - "data, I had to deal with a few more details. But this chapter is already\n", - "too long, so I won’t discuss them here.\n", - "\n", - "You can read about the problems, and how I addressed them, at\n", - ".\n", - "\n", - "You can download the code in this chapter from\n", - ". For more information see\n", - "Section [download].\n", - "\n", - "Discussion\n", - "----------\n", - "\n", - "The Unseen Species problem is an area of active research, and I believe\n", - "the algorithm in this chapter is a novel contribution. So in fewer than\n", - "200 pages we have made it from the basics of probability to the research\n", - "frontier. I’m very happy about that.\n", - "\n", - "My goal for this book is to present three related ideas:\n", - "\n", - "- **Bayesian thinking**: The foundation of Bayesian\n", - " analysis is the idea of using probability distributions to represent\n", - " uncertain beliefs, using data to update those distributions, and\n", - " using the results to make predictions and inform decisions.\n", - "\n", - "- **A computational approach**: The premise of this book\n", - " is that it is easier to understand Bayesian analysis using\n", - " computation rather than math, and easier to implement Bayesian\n", - " methods with reusable building blocks that can be rearranged to\n", - " solve real-world problems quickly.\n", - "\n", - "- **Iterative modeling**: Most real-world problems\n", - " involve modeling decisions and trade-offs between realism and\n", - " complexity. It is often impossible to know ahead of time what\n", - " factors should be included in the model and which can be abstracted\n", - " away. The best approach is to iterate, starting with simple models\n", - " and adding complexity gradually, using each model to validate the\n", - " others.\n", - "\n", - "These ideas are versatile and powerful; they are applicable to problems\n", - "in every area of science and engineering, from simple examples to topics\n", - "of current research.\n", - "\n", - "If you made it this far, you should be prepared to apply these tools to\n", - "new problems relevant to your work. I hope you find them useful; let me\n", - "know how it goes!\n", - "\n", - "[^1]: Based on an example from\n", - " that is no longer\n", - " there.\n", - "\n", - "[^2]: Ruggles and Brodie, “An Empirical Approach to Economic\n", - " Intelligence in World War II,” *Journal of the American\n", - " Statistical Association*, Vol. 42, No. 237 (March 1947).\n", - "\n", - "[^3]: Zhang et al, Distribution of Renal Tumor Growth Rates Determined\n", - " by Using Serial Volumetric CT Measurements, January 2009\n", - " *Radiology*, 250, 137-144.\n", - "\n", - "[^4]: If you are not familiar with Python generators, see\n", - " .\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n" + ], + "text/plain": [ + "" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# format the book\n", + "%matplotlib inline\n", + "import sys\n", + "from __future__ import division, print_function\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", + "import book_format\n", + "book_format.load_style('../code')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Dealing with Dimensions\n", + "\n", + "## Belly button bacteria\n", + "\n", + "Belly Button Biodiversity 2.0 (BBB2) is a nation-wide citizen science\n", + "project with the goal of identifying bacterial species that can be found\n", + "in human navels (). The project might\n", + "seem whimsical, but it is part of an increasing interest in the human\n", + "microbiome, the set of microorganisms that live on human skin and parts\n", + "of the body.\n", + "\n", + "In their pilot study, BBB2 researchers collected swabs from the navels\n", + "of 60 volunteers, used multiplex pyrosequencing to extract and sequence\n", + "fragments of 16S rDNA, then identified the species or genus the\n", + "fragments came from. Each identified fragment is called a “read.”\n", + "\n", + "We can use these data to answer several related questions:\n", + "\n", + "- Based on the number of species observed, can we estimate the total\n", + " number of species in the environment?\n", + "\n", + "- Can we estimate the prevalence of each species; that is, the\n", + " fraction of the total population belonging to each species?\n", + "\n", + "- If we are planning to collect additional samples, can we predict how\n", + " many new species we are likely to discover?\n", + "\n", + "- How many additional reads are needed to increase the fraction of\n", + " observed species to a given threshold?\n", + "\n", + "These questions make up what is called the **Unseen Species\n", + "problem**." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Lions and tigers and bears\n", + "\n", + "I’ll start with a simplified version of the problem where we know that\n", + "there are exactly three species. Let’s call them lions, tigers and\n", + "bears. Suppose we visit a wild animal preserve and see 3 lions, 2 tigers\n", + "and one bear.\n", + "\n", + "If we have an equal chance of observing any animal in the preserve, the\n", + "number of each species we see is governed by the multinomial\n", + "distribution. If the prevalence of lions and tigers and bears is\n", + "`p_lion` and `p_tiger` and `p_bear`, the likelihood of seeing 3 lions, 2\n", + "tigers and one bear is\n", + "\n", + "```python\n", + " p_lion**3 * p_tiger**2 * p_bear**1\n", + "```\n", + "\n", + "An approach that is tempting, but not correct, is to use beta\n", + "distributions, as in Section [beta], to describe the prevalence of each\n", + "species separately. For example, we saw 3 lions and 3 non-lions; if we\n", + "think of that as 3 “heads” and 3 “tails,” then the posterior\n", + "distribution of `p_lion` is:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "ename": "AttributeError", + "evalue": "'Beta' object has no attribute 'MaximumLikelihood'", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 3\u001b[0m \u001b[0mbeta\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mthinkbayes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mBeta\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[0mbeta\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mUpdate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m3\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbeta\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mMaximumLikelihood\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;31mAttributeError\u001b[0m: 'Beta' object has no attribute 'MaximumLikelihood'" + ] + } + ], + "source": [ + "import thinkbayes\n", + "\n", + "beta = thinkbayes.Beta()\n", + "beta.Update((3, 3))\n", + "print(beta.MaximumLikelihood())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The maximum likelihood estimate for `p_lion` is the observed rate, 50%.\n", + "Similarly the MLEs for `p_tiger` and `p_bear` are 33% and 17%.\n", + "\n", + "But there are two problems:\n", + "\n", + "1. We have implicitly used a prior for each species that is uniform\n", + " from 0 to 1, but since we know that there are three species, that\n", + " prior is not correct. The right prior should have a mean of 1/3, and\n", + " there should be zero likelihood that any species has a prevalence of\n", + " 100%.\n", + "\n", + "2. The distributions for each species are not independent, because the\n", + " prevalences have to add up to 1. To capture this dependence, we need\n", + " a joint distribution for the three prevalences.\n", + "\n", + "We can use a Dirichlet distribution to solve both of these problems (see\n", + "). In the same way\n", + "we used the beta distribution to describe the distribution of bias for a\n", + "coin, we can use a Dirichlet distribution to describe the joint\n", + "distribution of `p_lion`, `p_tiger` and `p_bear`.\n", + "\n", + "The Dirichlet distribution is the multi-dimensional generalization of\n", + "the beta distribution. Instead of two possible outcomes, like heads and\n", + "tails, the Dirichlet distribution handles any number of outcomes: in\n", + "this example, three species.\n", + "\n", + "If there are `n` outcomes, the Dirichlet distribution is\n", + "described by `n` parameters, written $\\alpha_1$ through\n", + "$\\alpha_n$.\n", + "\n", + "Here’s the definition, from `thinkbayes.py`, of a class that\n", + "represents a Dirichlet distribution:\n", + "\n", + "```python\n", + " class Dirichlet(object):\n", + "\n", + " def __init__(self, n):\n", + " self.n = n\n", + " self.params = numpy.ones(n, dtype=numpy.int)\n", + "```\n", + "\n", + "`n` is the number of dimensions; initially the parameters are\n", + "all 1. I use a `numpy` array to store the parameters so I can\n", + "take advantage of array operations.\n", + "\n", + "Given a Dirichlet distribution, the marginal distribution for each\n", + "prevalence is a beta distribution, which we can compute like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MarginalBeta(self, i):\n", + " alpha0 = self.params.sum()\n", + " alpha = self.params[i]\n", + " return Beta(alpha, alpha0-alpha)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`i` is the index of the marginal distribution we want.\n", + "`alpha0` is the sum of the parameters; `alpha` is\n", + "the parameter for the given species.\n", + "\n", + "In the example, the prior marginal distribution for each species is\n", + "`Beta(1, 2)`. We can compute the prior means like this:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.333333333333\n", + "0.333333333333\n", + "0.333333333333\n" + ] + } + ], + "source": [ + "dirichlet = thinkbayes.Dirichlet(3)\n", + "for i in range(3):\n", + " beta = dirichlet.MarginalBeta(i)\n", + " print(beta.Mean())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As expected, the prior mean prevalence for each species is 1/3.\n", + "\n", + "To update the Dirichlet distribution, we add the observations to the\n", + "parameters like this:\n", + "\n", + "```python\n", + " def Update(self, data):\n", + " m = len(data)\n", + " self.params[:m] += data\n", + "```\n", + "\n", + "`data` is a sequence of counts in the same order as\n", + "`params`, so in this example, it should be the number of\n", + "lions, tigers and bears.\n", + "\n", + "`data` can be shorter than `params`; in that case\n", + "there are some species that have not been observed.\n", + "\n", + "Here’s code that updates `dirichlet` with the observed data\n", + "and computes the posterior marginal distributions." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0 0.444444434084\n", + "1 0.333333370657\n", + "2 0.222325930766\n" + ] + } + ], + "source": [ + "data = [3, 2, 1]\n", + "dirichlet.Update(data)\n", + "\n", + "for i in range(3):\n", + " beta = dirichlet.MarginalBeta(i)\n", + " pmf = beta.MakePmf()\n", + " print(i, pmf.Mean())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Distribution of prevalences for three species.](figs/species1.pdf)\n", + "\n", + "[fig.species1]\n", + "\n", + "Figure [fig.species1] shows the results. The posterior mean prevalences\n", + "are 44%, 33%, and 22%." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The hierarchical version\n", + "\n", + "We have solved a simplified version of the problem: if we know how many\n", + "species there are, we can estimate the prevalence of each.\n", + "\n", + "Now let’s get back to the original problem, estimating the total number\n", + "of species. To solve this problem I’ll define a meta-Suite, which is a\n", + "Suite that contains other Suites as hypotheses. In this case, the\n", + "top-level Suite contains hypotheses about the number of species; the\n", + "bottom level contains hypotheses about prevalences.\n", + "\n", + "Here’s the class definition:\n", + "\n", + " class Species(thinkbayes.Suite):\n", + "\n", + " def __init__(self, ns):\n", + " hypos = [thinkbayes.Dirichlet(n) for n in ns]\n", + " thinkbayes.Suite.__init__(self, hypos)\n", + "\n", + "`__init__` takes a list of possible values for `n` and makes\n", + "a list of Dirichlet objects.\n", + "\n", + "Here’s the code that creates the top-level suite:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from species import Species\n", + "\n", + "ns = range(3, 30)\n", + "suite = Species(ns)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`ns` is the list of possible values for `n`. We\n", + "have seen 3 species, so there have to be at least that many. I chose an\n", + "upper bound that seems reasonable, but we will check later that the\n", + "probability of exceeding this bound is low. And at least initially we\n", + "assume that any value in this range is equally likely.\n", + "\n", + "To update a hierarchical model, you have to update all levels. Usually\n", + "you have to update the bottom level first and work up, but in this case\n", + "we can update the top level first:\n", + "\n", + "```python\n", + "#class Species\n", + "\n", + " def Update(self, data):\n", + " thinkbayes.Suite.Update(self, data)\n", + " for hypo in self.Values():\n", + " hypo.Update(data)\n", + "```\n", + "\n", + "`Species.Update` invokes `Update` in the parent\n", + "class, then loops through the sub-hypotheses and updates them.\n", + "\n", + "Now all we need is a likelihood function:\n", + "\n", + "```python\n", + "# class Species\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " dirichlet = hypo\n", + " like = 0\n", + " for i in range(1000):\n", + " like += dirichlet.Likelihood(data)\n", + "\n", + " return like\n", + "```\n", + "\n", + "`data` is a sequence of observed counts; `hypo` is\n", + "a Dirichlet object. `Species.Likelihood` calls\n", + "`Dirichlet.Likelihood` 1000 times and returns the total.\n", + "\n", + "Why call it 1000 times? Because `Dirichlet.Likelihood`\n", + "doesn’t actually compute the likelihood of the data under the whole\n", + "Dirichlet distribution. Instead, it draws one sample from the\n", + "hypothetical distribution and computes the likelihood of the data under\n", + "the sampled set of prevalences.\n", + "\n", + "Here’s what it looks like:\n", + "\n", + "```python\n", + "# class Dirichlet\n", + "\n", + " def Likelihood(self, data):\n", + " m = len(data)\n", + " if self.n < m:\n", + " return 0\n", + "\n", + " x = data\n", + " p = self.Random()\n", + " q = p[:m]**x\n", + " return q.prod()\n", + "```\n", + "\n", + "The length of `data` is the number of species observed. If we\n", + "see more species than we thought existed, the likelihood is 0.\n", + "\n", + "Otherwise we select a random set of prevalences, `p`, and\n", + "compute the multinomial PMF, which is\n", + "\n", + "$$c_x p_1^{x_1} \\cdots p_n^{x_n}$$ \n", + "\n", + "$p_i$ is the prevalence of the $i$th\n", + "species, and $x_i$ is the observed number. The first term, $c_x$, is the\n", + "multinomial coefficient; I leave it out of the computation because it is\n", + "a multiplicative factor that depends only on the data, not the\n", + "hypothesis, so it gets normalized away (see\n", + ").\n", + "\n", + "`m` is the number of observed species. We only need the first\n", + "`m` elements of `p`; for the others, $x_i$ is 0,\n", + "so $p_i^{x_i}$ is 1, and we can leave them out of the product." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Random sampling\n", + "\n", + "There are two ways to generate a random sample from a Dirichlet\n", + "distribution. One is to use the marginal beta distributions, but in that\n", + "case you have to select one at a time and scale the rest so they add up\n", + "to 1 (see\n", + ").\n", + "\n", + "A less obvious, but faster, way is to select values from `n`\n", + "gamma distributions, then normalize by dividing through by the total.\n", + "Here’s the code:\n", + "\n", + "```python\n", + "# class Dirichlet\n", + "\n", + " def Random(self):\n", + " p = numpy.random.gamma(self.params)\n", + " return p / p.sum()\n", + "```\n", + "\n", + "Now we’re ready to look at some results. Here is the code that extracts\n", + "the posterior distribution of `n`:\n", + "\n", + " ```python\n", + " \n", + " def DistOfN(self):\n", + " pmf = thinkbayes.Pmf()\n", + " for hypo, prob in self.Items():\n", + " pmf.Set(hypo.n, prob)\n", + " return pmf\n", + "```\n", + "\n", + "`DistOfN` iterates through the top-level hypotheses and\n", + "accumulates the probability of each `n`.\n", + "\n", + "![Posterior distribution of `n`.](figs/species2.pdf)\n", + "\n", + "[fig.species2]\n", + "\n", + "Figure [fig.species2] shows the result. The most likely value is 4.\n", + "Values from 3 to 7 are reasonably likely; after that the probabilities\n", + "drop off quickly. The probability that there are 29 species is low\n", + "enough to be negligible; if we chose a higher bound, we would get nearly\n", + "the same result.\n", + "\n", + "Remember that this result is based on a uniform prior for\n", + "`n`. If we have background information about the number of\n", + "species in the environment, we might choose a different prior." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Optimization\n", + "\n", + "I have to admit that I am proud of this example. The Unseen Species\n", + "problem is not easy, and I think this solution is simple and clear, and\n", + "takes surprisingly few lines of code (about 50 so far).\n", + "\n", + "The only problem is that it is slow. It’s good enough for the example\n", + "with only 3 observed species, but not good enough for the belly button\n", + "data, with more than 100 species in some samples.\n", + "\n", + "The next few sections present a series of optimizations we need to make\n", + "this solution scale. Before we get into the details, here’s a road map.\n", + "\n", + "- The first step is to recognize that if we update the Dirichlet\n", + " distributions with the same data, the first `m`\n", + " parameters are the same for all of them. The only difference is the\n", + " number of hypothetical unseen species. So we don’t really need\n", + " `n` Dirichlet objects; we can store the parameters in the\n", + " top level of the hierarchy. `Species2` implements this\n", + " optimization.\n", + "\n", + "- `Species2` also uses the same set of random values for\n", + " all of the hypotheses. This saves time generating random values, but\n", + " it has a second benefit that turns out to be more important: by\n", + " giving all hypotheses the same selection from the sample space, we\n", + " make the comparison between the hypotheses more fair, so it takes\n", + " fewer iterations to converge.\n", + "\n", + "- Even with these changes there is a major performance problem. As the\n", + " number of observed species increases, the array of random\n", + " prevalences gets bigger, and the chance of choosing one that is\n", + " approximately right becomes small. So the vast majority of\n", + " iterations yield small likelihoods that don’t contribute much to the\n", + " total, and don’t discriminate between hypotheses.\n", + "\n", + " The solution is to do the updates one species at a time.\n", + " `Species4` is a simple implementation of this strategy\n", + " using Dirichlet objects to represent the sub-hypotheses.\n", + "\n", + "- Finally, `Species5` combines the sub-hypotheses into the\n", + " top level and uses `numpy` array operations to speed\n", + " things up.\n", + "\n", + "If you are not interested in the details, feel free to skip to\n", + "Section [belly] where we look at results from the belly button data." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Collapsing the hierarchy\n", + "\n", + "All of the bottom-level Dirichlet distributions are updated with the\n", + "same data, so the first `m` parameters are the same for all\n", + "of them. We can eliminate them and merge the parameters into the\n", + "top-level suite. `Species2` implements this optimization:\n", + "\n", + "```python\n", + "class Species2(object):\n", + "\n", + " def __init__(self, ns):\n", + " self.ns = ns\n", + " self.probs = numpy.ones(len(ns), dtype=numpy.double)\n", + " self.params = numpy.ones(self.high, dtype=numpy.int)\n", + "```\n", + "\n", + "`ns` is the list of hypothetical values for `n`;\n", + "`probs` is the list of corresponding probabilities. And\n", + "`params` is the sequence of Dirichlet parameters, initially\n", + "all 1.\n", + "\n", + "`Species2.Update` updates both levels of the hierarchy: first\n", + "the probability for each value of `n`, then the Dirichlet\n", + "parameters:\n", + "\n", + "```python\n", + "# class Species2\n", + "\n", + " def Update(self, data):\n", + " like = numpy.zeros(len(self.ns), dtype=numpy.double)\n", + " for i in range(1000):\n", + " like += self.SampleLikelihood(data)\n", + "\n", + " self.probs *= like\n", + " self.probs /= self.probs.sum()\n", + "\n", + " m = len(data)\n", + " self.params[:m] += data\n", + "```\n", + "\n", + "`SampleLikelihood` returns an array of likelihoods, one for\n", + "each value of `n`. `like` accumulates the total\n", + "likelihood for 1000 samples. `self.probs` is multiplied by\n", + "the total likelihood, then normalized. The last two lines, which update\n", + "the parameters, are the same as in `Dirichlet.Update`.\n", + "\n", + "Now let’s look at `SampleLikelihood`. There are two\n", + "opportunities for optimization here:\n", + "\n", + "- When the hypothetical number of species, `n`, exceeds the\n", + " observed number, `m`, we only need the first\n", + " `m` terms of the multinomial PMF; the rest are 1.\n", + "\n", + "- If the number of species is large, the likelihood of the data might\n", + " be too small for floating-point (see  [underflow]). So it is safer\n", + " to compute log-likelihoods.\n", + "\n", + "Again, the multinomial PMF is \n", + "\n", + "$$c_x p_1^{x_1} \\cdots p_n^{x_n}$$ \n", + "\n", + "So the log-likelihood is\n", + "\n", + "$$\\log c_x + x_1 \\log p_1 + \\cdots + x_n \\log p_n$$\n", + "\n", + "which is fast and easy to compute. Again, $c_x$ it is the same for all\n", + "hypotheses, so we can drop it. Here’s the code:\n", + "\n", + "```python\n", + "# class Species2\n", + "\n", + " def SampleLikelihood(self, data):\n", + " gammas = numpy.random.gamma(self.params)\n", + "\n", + " m = len(data)\n", + " row = gammas[:m]\n", + " col = numpy.cumsum(gammas)\n", + "\n", + " log_likes = []\n", + " for n in self.ns:\n", + " ps = row / col[n-1]\n", + " terms = data * numpy.log(ps)\n", + " log_like = terms.sum()\n", + " log_likes.append(log_like)\n", + "\n", + " log_likes -= numpy.max(log_likes)\n", + " likes = numpy.exp(log_likes)\n", + "\n", + " coefs = [thinkbayes.BinomialCoef(n, m) for n in self.ns]\n", + " likes *= coefs\n", + "\n", + " return likes\n", + "```\n", + "\n", + "`gammas` is an array of values from a gamma distribution; its\n", + "length is the largest hypothetical value of `n`.\n", + "`row` is just the first `m` elements of\n", + "`gammas`; since these are the only elements that depend on\n", + "the data, they are the only ones we need.\n", + "\n", + "For each value of `n` we need to divide `row` by\n", + "the total of the first `n` values from `gamma`.\n", + "`cumsum` computes these cumulative sums and stores them in\n", + "`col`.\n", + "\n", + "The loop iterates through the values of `n` and accumulates a\n", + "list of log-likelihoods.\n", + "\n", + "Inside the loop, `ps` contains the row of probabilities,\n", + "normalized with the appropriate cumulative sum. `terms`\n", + "contains the terms of the summation, $x_i \\log p_i$, and `log_like`\n", + "contains their sum.\n", + "\n", + "After the loop, we want to convert the log-likelihoods to linear\n", + "likelihoods, but first it’s a good idea to shift them so the largest\n", + "log-likelihood is 0; that way the linear likelihoods are not too small\n", + "(see  [underflow]).\n", + "\n", + "Finally, before we return the likelihood, we have to apply a correction\n", + "factor, which is the number of ways we could have observed these\n", + "`m` species, if the total number of species is\n", + "`n`. `BinomialCoefficient` computes “n choose m”,\n", + "which is written $\\binom{n}{m}$.\n", + "\n", + "As often happens, the optimized version is less readable and more\n", + "error-prone than the original. But that’s one reason I think it is a\n", + "good idea to start with the simple version; we can use it for regression\n", + "testing. I plotted results from both versions and confirmed that they\n", + "are approximately equal, and that they converge as the number of\n", + "iterations increases." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## One more problem\n", + "\n", + "There’s more we could do to optimize this code, but there’s another\n", + "problem we need to fix first. As the number of observed species\n", + "increases, this version gets noisier and takes more iterations to\n", + "converge on a good answer.\n", + "\n", + "The problem is that if the prevalences we choose from the Dirichlet\n", + "distribution, the `ps`, are not at least approximately right,\n", + "the likelihood of the observed data is close to zero and almost equally\n", + "bad for all values of `n`. So most iterations don’t provide\n", + "any useful contribution to the total likelihood. And as the number of\n", + "observed species, `m`, gets large, the probability of\n", + "choosing `ps` with non-negligible likelihood gets small.\n", + "Really small.\n", + "\n", + "Fortunately, there is a solution. Remember that if you observe a set of\n", + "data, you can update the prior distribution with the entire dataset, or\n", + "you can break it up into a series of updates with subsets of the data,\n", + "and the result is the same either way.\n", + "\n", + "For this example, the key is to perform the updates one species at a\n", + "time. That way when we generate a random set of `ps`, only\n", + "one of them affects the computed likelihood, so the chance of choosing a\n", + "good one is much better.\n", + "\n", + "Here’s a new version that updates one species at a time:\n", + "\n", + "```python\n", + "class Species4(Species):\n", + "\n", + " def Update(self, data):\n", + " m = len(data)\n", + "\n", + " for i in range(m):\n", + " one = numpy.zeros(i+1)\n", + " one[i] = data[i] \n", + " Species.Update(self, one)\n", + "```\n", + "\n", + "This version inherits `__init__` from `Species`, so it\n", + "represents the hypotheses as a list of Dirichlet objects (unlike\n", + "`Species2`).\n", + "\n", + "`Update` loops through the observed species and makes an\n", + "array, `one`, with all zeros and one species count. Then it\n", + "calls `Update` in the parent class, which computes the\n", + "likelihoods and updates the sub-hypotheses.\n", + "\n", + "So in the running example, we do three updates. The first is something\n", + "like “I have seen three lions.” The second is “I have seen two tigers\n", + "and no additional lions.” And the third is “I have seen one bear and no\n", + "more lions and tigers.”\n", + "\n", + "Here’s the new version of `Likelihood`:\n", + "\n", + "```python\n", + "\n", + "# class Species4\n", + "\n", + " def Likelihood(self, data, hypo):\n", + " dirichlet = hypo\n", + " like = 0\n", + " for i in range(self.iterations):\n", + " like += dirichlet.Likelihood(data)\n", + "\n", + " # correct for the number of unseen species the new one\n", + " # could have been\n", + " m = len(data)\n", + " num_unseen = dirichlet.n - m + 1\n", + " like *= num_unseen\n", + "\n", + " return like\n", + "```\n", + "\n", + "This is almost the same as `Species.Likelihood`. The\n", + "difference is the factor, `num_unseen`. This correction is necessary\n", + "because each time we see a species for the first time, we have to\n", + "consider that there were some number of other unseen species that we\n", + "might have seen. For larger values of `n` there are more\n", + "unseen species that we could have seen, which increases the likelihood\n", + "of the data.\n", + "\n", + "This is a subtle point and I have to admit that I did not get it right\n", + "the first time. But again I was able to validate this version by\n", + "comparing it to the previous versions." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## We’re not done yet\n", + "\n", + "Performing the updates one species at a time solves one problem, but it\n", + "creates another. Each update takes time proportional to $k m$, where $k$\n", + "is the number of hypotheses and $m$ is the number of observed species.\n", + "So if we do $m$ updates, the total run time is proportional to $k m^2$.\n", + "\n", + "But we can speed things up using the same trick we used in\n", + "Section [collapsing]: we’ll get rid of the Dirichlet objects and\n", + "collapse the two levels of the hierarchy into a single object. So here’s\n", + "yet another version of `Species`:\n", + "\n", + "```python\n", + "class Species5(Species2):\n", + "\n", + " def Update(self, data):\n", + " m = len(data)\n", + " for i in range(m):\n", + " self.UpdateOne(i+1, data[i])\n", + " self.params[i] += data[i]\n", + "```\n", + "\n", + "This version inherits `__init__` from `Species2`, so it uses\n", + "`ns` and `probs` to represent the distribution of\n", + "`n`, and `params` to represent the parameters of\n", + "the Dirichlet distribution.\n", + "\n", + "`Update` is similar to what we saw in the previous section.\n", + "It loops through the observed species and calls `UpdateOne`:\n", + "\n", + "```python\n", + "# class Species5\n", + "\n", + " def UpdateOne(self, i, count):\n", + " likes = numpy.zeros(len(self.ns), dtype=numpy.double)\n", + " for i in range(self.iterations):\n", + " likes += self.SampleLikelihood(i, count)\n", + "\n", + " unseen_species = [n-i+1 for n in self.ns]\n", + " likes *= unseen_species\n", + "\n", + " self.probs *= likes\n", + " self.probs /= self.probs.sum()\n", + "```\n", + "\n", + "This function is similar to `Species2.Update`, with two\n", + "changes:\n", + "\n", + "- The interface is different. Instead of the whole dataset, we get\n", + " `i`, the index of the observed species, and\n", + " `count`, how many of that species we’ve seen.\n", + "\n", + "- We have to apply a correction factor for the number of unseen\n", + " species, as in `Species4.Likelihood`. The difference here\n", + " is that we update all of the likelihoods at once with array\n", + " multiplication.\n", + "\n", + "Finally, here’s `SampleLikelihood`:\n", + "\n", + "```python\n", + "# class Species5\n", + "\n", + " def SampleLikelihood(self, i, count):\n", + " gammas = numpy.random.gamma(self.params)\n", + "\n", + " sums = numpy.cumsum(gammas)[self.ns[0]-1:]\n", + "\n", + " ps = gammas[i-1] / sums\n", + " log_likes = numpy.log(ps) * count\n", + "\n", + " log_likes -= numpy.max(log_likes)\n", + " likes = numpy.exp(log_likes)\n", + "\n", + " return likes\n", + "```\n", + "\n", + "This is similar to `Species2.SampleLikelihood`; the\n", + "difference is that each update only includes a single species, so we\n", + "don’t need a loop.\n", + "\n", + "The runtime of this function is proportional to the number of\n", + "hypotheses, $k$. It runs $m$ times, so the run time of the update is\n", + "proportional to $k m$. And the number of iterations we need to get an\n", + "accurate result is usually small." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The belly button data\n", + "\n", + "That’s enough about lions and tigers and bears. Let’s get back to belly\n", + "buttons. To get a sense of what the data look like, consider subject\n", + "B1242, whose sample of 400 reads yielded 61 species with the following\n", + "counts:\n", + "\n", + " 92, 53, 47, 38, 15, 14, 12, 10, 8, 7, 7, 5, 5, \n", + " 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2,\n", + " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n", + " 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1\n", + "\n", + "There are a few dominant species that make up a large fraction of the\n", + "whole, but many species that yielded only a single read. The number of\n", + "these “singletons” suggests that there are likely to be at least a few\n", + "unseen species.\n", + "\n", + "In the example with lions and tigers, we assume that each animal in the\n", + "preserve is equally likely to be observed. Similarly, for the belly\n", + "button data, we assume that each bacterium is equally likely to yield a\n", + "read.\n", + "\n", + "In reality, each step in the data-collection process might introduce\n", + "biases. Some species might be more likely to be picked up by a swab, or\n", + "to yield identifiable amplicons. So when we talk about the prevalence of\n", + "each species, we should remember this source of error.\n", + "\n", + "I should also acknowledge that I am using the term “species” loosely.\n", + "First, bacterial species are not well defined. Second, some reads\n", + "identify a particular species, others only identify a genus. To be more\n", + "precise, I should say “operational taxonomic unit”, or OTU.\n", + "\n", + "Now let’s process some of the belly button data. I define a class called\n", + "`Subject` to represent information about each subject in the\n", + "study:\n", + "\n", + "```python\n", + "class Subject(object):\n", + "\n", + " def __init__(self, code):\n", + " self.code = code\n", + " self.species = []\n", + "```\n", + "\n", + "Each subject has a string code, like “B1242”, and a list of (count,\n", + "species name) pairs, sorted in increasing order by count.\n", + "`Subject` provides several methods to make it easy to access\n", + "these counts and species names. You can see the details in\n", + ". For more information see\n", + "Section [download].\n", + "\n", + "![Distribution of `n` for subject\n", + "B1242.](figs/species-ndist-B1242.pdf)\n", + "\n", + "[species-ndist]\n", + "\n", + "`Subject` provides a method named `Process` that\n", + "creates and updates a `Species5` suite, which represents the\n", + "distributions of `n` and the prevalences.\n", + "\n", + "And `Suite2` provides `DistOfN`, which returns the\n", + "posterior distribution of `n`.\n", + "\n", + "```python\n", + "# class Suite2\n", + "\n", + " def DistN(self):\n", + " items = zip(self.ns, self.probs)\n", + " pmf = thinkbayes.MakePmfFromItems(items)\n", + " return pmf\n", + "```\n", + "\n", + "Figure [species-ndist] shows the distribution of `n` for\n", + "subject B1242. The probability that there are exactly 61 species, and no\n", + "unseen species, is nearly zero. The most likely value is 72, with 90%\n", + "credible interval 66 to 79. At the high end, it is unlikely that there\n", + "are as many as 87 species.\n", + "\n", + "Next we compute the posterior distribution of prevalence for each\n", + "species. `Species2` provides `DistOfPrevalence`:\n", + "\n", + " # class Species2\n", + "\n", + " def DistOfPrevalence(self, index):\n", + " metapmf = thinkbayes.Pmf()\n", + "\n", + " for n, prob in zip(self.ns, self.probs):\n", + " beta = self.MarginalBeta(n, index)\n", + " pmf = beta.MakePmf()\n", + " metapmf.Set(pmf, prob)\n", + "\n", + " mix = thinkbayes.MakeMixture(metapmf)\n", + " return metapmf, mix\n", + "\n", + "`index` indicates which species we want. For each\n", + "`n`, we have a different posterior distribution of\n", + "prevalence.\n", + "\n", + "![Distribution of prevalences for subject\n", + "B1242.](figs/species-prev-B1242.pdf)\n", + "\n", + "[species-prev]\n", + "\n", + "The loop iterates through the possible values of `n` and\n", + "their probabilities. For each value of `n` it gets a Beta\n", + "object representing the marginal distribution for the indicated species.\n", + "Remember that Beta objects contain the parameters `alpha` and\n", + "`beta`; they don’t have values and probabilities like a Pmf,\n", + "but they provide `MakePmf`, which generates a discrete\n", + "approximation to the continuous beta distribution.\n", + "\n", + "`metapmf` is a meta-Pmf that contains the distributions of\n", + "prevalence, conditioned on `n`. `MakeMixture`\n", + "combines the meta-Pmf into `mix`, which combines the\n", + "conditional distributions into a single distribution of prevalence.\n", + "\n", + "Figure [species-prev] shows results for the five species with the most\n", + "reads. The most prevalent species accounts for 23% of the 400 reads, but\n", + "since there are almost certainly unseen species, the most likely\n", + "estimate for its prevalence is 20%, with 90% credible interval between\n", + "17% and 23%." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Predictive distributions\n", + "\n", + "![Simulated rarefaction curves for subject\n", + "B1242.](figs/species-rare-B1242.pdf)\n", + "\n", + "[species-rare]\n", + "\n", + "I introduced the hidden species problem in the form of four related\n", + "questions. We have answered the first two by computing the posterior\n", + "distribution for `n` and the prevalence of each species.\n", + "\n", + "The other two questions are:\n", + "\n", + "- If we are planning to collect additional reads, can we predict how\n", + " many new species we are likely to discover?\n", + "\n", + "- How many additional reads are needed to increase the fraction of\n", + " observed species to a given threshold?\n", + "\n", + "To answer predictive questions like this we can use the posterior\n", + "distributions to simulate possible future events and compute predictive\n", + "distributions for the number of species, and fraction of the total, we\n", + "are likely to see.\n", + "\n", + "The kernel of these simulations looks like this:\n", + "\n", + "1. Choose `n` from its posterior distribution.\n", + "\n", + "2. Choose a prevalence for each species, including possible unseen\n", + " species, using the Dirichlet distribution.\n", + "\n", + "3. Generate a random sequence of future observations.\n", + "\n", + "4. Compute the number of new species, `num_new`, as a function of the\n", + " number of additional reads, `k`.\n", + "\n", + "5. Repeat the previous steps and accumulate the joint distribution of\n", + " `num_new` and `k`.\n", + "\n", + "And here’s the code. `RunSimulation` runs a single\n", + "simulation:\n", + "\n", + "```python\n", + "# class Subject\n", + "\n", + " def RunSimulation(self, num_reads):\n", + " m, seen = self.GetSeenSpecies()\n", + " n, observations = self.GenerateObservations(num_reads)\n", + "\n", + " curve = []\n", + " for k, obs in enumerate(observations):\n", + " seen.add(obs)\n", + "\n", + " num_new = len(seen) - m\n", + " curve.append((k+1, num_new))\n", + "\n", + " return curve\n", + "```\n", + "\n", + "`num_reads` is the number of additional reads to simulate.\n", + "`m` is the number of seen species, and `seen` is a\n", + "set of strings with a unique name for each species. `n` is a\n", + "random value from the posterior distribution, and\n", + "`observations` is a random sequence of species names.\n", + "\n", + "Each time through the loop, we add the new observation to\n", + "`seen` and record the number of reads and the number of new\n", + "species so far.\n", + "\n", + "The result of `RunSimulation` is a **rarefaction\n", + "curve**, represented as a list of pairs with the number of reads\n", + "and the number of new species.\n", + "\n", + "Before we see the results, let’s look at `GetSeenSpecies` and\n", + "`GenerateObservations`.\n", + "\n", + "```python\n", + "#class Subject\n", + "\n", + " def GetSeenSpecies(self):\n", + " names = self.GetNames()\n", + " m = len(names)\n", + " seen = set(SpeciesGenerator(names, m))\n", + " return m, seen\n", + "```\n", + "\n", + "`GetNames` returns the list of species names that appear in\n", + "the data files, but for many subjects these names are not unique. So I\n", + "use `SpeciesGenerator` to extend each name with a serial\n", + "number:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def SpeciesGenerator(names, num):\n", + " i = 0\n", + " for name in names:\n", + " yield '%s-%d' % (name, i)\n", + " i += 1\n", + "\n", + " while i < num:\n", + " yield 'unseen-%d' % i\n", + " i += 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given a name like `Corynebacterium`,\n", + "`SpeciesGenerator` yields `Corynebacterium-1`.\n", + "When the list of names is exhausted, it yields names like\n", + "`unseen-62`.\n", + "\n", + "Here is `GenerateObservations`:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Given a name like `Corynebacterium`,\n", + "`SpeciesGenerator` yields `Corynebacterium-1`.\n", + "When the list of names is exhausted, it yields names like\n", + "`unseen-62`.\n", + "\n", + "Here is `GenerateObservations`:\n", + "\n", + "```python\n", + "# class Subject\n", + "\n", + " def GenerateObservations(self, num_reads):\n", + " n, prevalences = self.suite.SamplePosterior()\n", + "\n", + " names = self.GetNames()\n", + " name_iter = SpeciesGenerator(names, n)\n", + "\n", + " d = dict(zip(name_iter, prevalences))\n", + " cdf = thinkbayes.MakeCdfFromDict(d)\n", + " observations = cdf.Sample(num_reads)\n", + "\n", + " return n, observations\n", + "```\n", + "\n", + "Again, `num_reads` is the number of additional reads to generate.\n", + "`n` and `prevalences` are samples from the\n", + "posterior distribution.\n", + "\n", + "`cdf` is a Cdf object that maps species names, including the\n", + "unseen, to cumulative probabilities. Using a Cdf makes it efficient to\n", + "generate a random sequence of species names.\n", + "\n", + "Finally, here is `Species2.SamplePosterior`:\n", + "\n", + "```python\n", + " def SamplePosterior(self):\n", + " pmf = self.DistOfN()\n", + " n = pmf.Random()\n", + " prevalences = self.SamplePrevalences(n)\n", + " return n, prevalences\n", + "```\n", + "\n", + "And `SamplePrevalences`, which generates a sample of\n", + "prevalences conditioned on `n`:\n", + "\n", + "```python\n", + "# class Species2\n", + "\n", + " def SamplePrevalences(self, n):\n", + " params = self.params[:n]\n", + " gammas = numpy.random.gamma(params)\n", + " gammas /= gammas.sum()\n", + " return gammas\n", + "```\n", + "\n", + "We saw this algorithm for generating random values from a Dirichlet\n", + "distribution in Section [randomdir].\n", + "\n", + "Figure [species-rare] shows 100 simulated rarefaction curves for subject\n", + "B1242. The curves are “jittered;” that is, I shifted each curve by a\n", + "random offset so they would not all overlap. By inspection we can\n", + "estimate that after 400 more reads we are likely to find 2–6 new\n", + "species." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Joint posterior\n", + "\n", + "![Distributions of the number of new species conditioned on the number\n", + "of additional reads.](figs/species-cond-B1242.pdf)\n", + "\n", + "[species-cond]\n", + "\n", + "We can use these simulations to estimate the joint distribution of\n", + "`num_new` and `k`, and from that we can get the distribution\n", + "of `num_new` conditioned on any value of `k`." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeJointPredictive(curves):\n", + " joint = thinkbayes.Joint()\n", + " for curve in curves:\n", + " for k, num_new in curve:\n", + " joint.Incr((k, num_new))\n", + " joint.Normalize()\n", + " return joint" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`MakeJointPredictive` makes a Joint object, which is a Pmf\n", + "whose values are tuples.\n", + "\n", + "`curves` is a list of rarefaction curves created by\n", + "`RunSimulation`. Each curve contains a list of pairs of\n", + "`k` and `num_new`.\n", + "\n", + "The resulting joint distribution is a map from each pair to its\n", + "probability of occurring. Given the joint distribution, we can use\n", + "`Joint.Conditional` get the distribution of `num_new`\n", + "conditioned on `k` (see Section [conditional]).\n", + "\n", + "`Subject.MakeConditionals` takes a list of `ks`\n", + "and computes the conditional distribution of `num_new` for each\n", + "`k`. The result is a list of Cdf objects." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def MakeConditionals(curves, ks):\n", + " joint = MakeJointPredictive(curves)\n", + "\n", + " cdfs = []\n", + " for k in ks:\n", + " pmf = joint.Conditional(1, 0, k)\n", + " pmf.name = 'k=%d' % k\n", + " cdf = pmf.MakeCdf()\n", + " cdfs.append(cdf)\n", + "\n", + " return cdfs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Figure [species-cond] shows the results. After 100 reads, the median\n", + "predicted number of new species is 2; the 90% credible interval is 0 to\n", + "5. After 800 reads, we expect to see 3 to 12 new species." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Coverage\n", + "\n", + "![Complementary CDF of coverage for a range of additional\n", + "reads.](figs/species-frac-B1242.pdf)\n", + "\n", + "[species-frac]\n", + "\n", + "The last question we want to answer is, “How many additional reads are\n", + "needed to increase the fraction of observed species to a given\n", + "threshold?”\n", + "\n", + "To answer this question, we need a version of `RunSimulation`\n", + "that computes the fraction of observed species rather than the number of\n", + "new species.\n", + "\n", + "```python\n", + "# class Subject\n", + "\n", + " def RunSimulation(self, num_reads):\n", + " m, seen = self.GetSeenSpecies()\n", + " n, observations = self.GenerateObservations(num_reads)\n", + "\n", + " curve = []\n", + " for k, obs in enumerate(observations):\n", + " seen.add(obs)\n", + "\n", + " frac_seen = len(seen) / float(n)\n", + " curve.append((k+1, frac_seen))\n", + "\n", + " return curve\n", + "```\n", + "\n", + "Next we loop through each curve and make a dictionary, `d`,\n", + "that maps from the number of additional reads, `k`, to a list\n", + "of `fracs`; that is, a list of values for the coverage\n", + "achieved after `k` reads.\n", + "\n", + "```python\n", + "\n", + " def MakeFracCdfs(self, curves):\n", + " d = {}\n", + " for curve in curves:\n", + " for k, frac in curve:\n", + " d.setdefault(k, []).append(frac)\n", + "\n", + " cdfs = {}\n", + " for k, fracs in d.iteritems():\n", + " cdf = thinkbayes.MakeCdfFromList(fracs)\n", + " cdfs[k] = cdf\n", + "\n", + " return cdfs\n", + "```\n", + "\n", + "Then for each value of `k` we make a Cdf of\n", + "`fracs`; this Cdf represents the distribution of coverage\n", + "after `k` reads.\n", + "\n", + "Remember that the CDF tells you the probability of falling below a given\n", + "threshold, so the *complementary* CDF tells you the\n", + "probability of exceeding it. Figure [species-frac] shows complementary\n", + "CDFs for a range of values of `k`.\n", + "\n", + "To read this figure, select the level of coverage you want to achieve\n", + "along the $x$-axis. As an example, choose 90%.\n", + "\n", + "Now you can read up the chart to find the probability of achieving 90%\n", + "coverage after `k` reads. For example, with 200 reads, you\n", + "have about a 40% chance of getting 90% coverage. With 1000 reads, you\n", + "have a 90% chance of getting 90% coverage.\n", + "\n", + "With that, we have answered the four questions that make up the unseen\n", + "species problem. To validate the algorithms in this chapter with real\n", + "data, I had to deal with a few more details. But this chapter is already\n", + "too long, so I won’t discuss them here.\n", + "\n", + "You can read about the problems, and how I addressed them, at\n", + ".\n", + "\n", + "You can download the code in this chapter from\n", + ". For more information see\n", + "Section [download]." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Discussion\n", + "\n", + "The Unseen Species problem is an area of active research, and I believe\n", + "the algorithm in this chapter is a novel contribution. So in fewer than\n", + "200 pages we have made it from the basics of probability to the research\n", + "frontier. I’m very happy about that.\n", + "\n", + "My goal for this book is to present three related ideas:\n", + "\n", + "- **Bayesian thinking**: The foundation of Bayesian\n", + " analysis is the idea of using probability distributions to represent\n", + " uncertain beliefs, using data to update those distributions, and\n", + " using the results to make predictions and inform decisions.\n", + "\n", + "- **A computational approach**: The premise of this book\n", + " is that it is easier to understand Bayesian analysis using\n", + " computation rather than math, and easier to implement Bayesian\n", + " methods with reusable building blocks that can be rearranged to\n", + " solve real-world problems quickly.\n", + "\n", + "- **Iterative modeling**: Most real-world problems\n", + " involve modeling decisions and trade-offs between realism and\n", + " complexity. It is often impossible to know ahead of time what\n", + " factors should be included in the model and which can be abstracted\n", + " away. The best approach is to iterate, starting with simple models\n", + " and adding complexity gradually, using each model to validate the\n", + " others.\n", + "\n", + "These ideas are versatile and powerful; they are applicable to problems\n", + "in every area of science and engineering, from simple examples to topics\n", + "of current research.\n", + "\n", + "If you made it this far, you should be prepared to apply these tools to\n", + "new problems relevant to your work. I hope you find them useful; let me\n", + "know how it goes!\n", + "\n", + "[^1]: Based on an example from\n", + " that is no longer\n", + " there.\n", + "\n", + "[^2]: Ruggles and Brodie, “An Empirical Approach to Economic\n", + " Intelligence in World War II,” *Journal of the American\n", + " Statistical Association*, Vol. 42, No. 237 (March 1947).\n", + "\n", + "[^3]: Zhang et al, Distribution of Renal Tumor Growth Rates Determined\n", + " by Using Serial Volumetric CT Measurements, January 2009\n", + " *Radiology*, 250, 137-144.\n", + "\n", + "[^4]: If you are not familiar with Python generators, see\n", + " ." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/Preface.ipynb b/notebooks/Preface.ipynb index 6075f89..2bb550c 100644 --- a/notebooks/Preface.ipynb +++ b/notebooks/Preface.ipynb @@ -1,545 +1,280 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#Preface\n", - "\n", - "## My theory, which is mine\n", - "\n", - "The premise of this book, and the other books in the *Think X* series,\n", - "is that if you know how to program, you can use that skill to learn\n", - "other topics.\n", - "\n", - "Most books on Bayesian statistics use mathematical notation and present\n", - "ideas in terms of mathematical concepts like calculus. This book uses\n", - "Python code instead of math, and discrete approximations instead of\n", - "continuous mathematics. As a result, what would be an integral in a math\n", - "book becomes a summation, and most operations on probability\n", - "distributions are simple loops.\n", - "\n", - "I think this presentation is easier to understand, at least for people\n", - "with programming skills. It is also more general, because when we make\n", - "modeling decisions, we can choose the most appropriate model without\n", - "worrying too much about whether the model lends itself to conventional\n", - "analysis.\n", - "\n", - "Also, it provides a smooth development path from simple examples to\n", - "real-world problems. Chapter [estimation] is a good example. It starts\n", - "with a simple example involving dice, one of the staples of basic\n", - "probability. From there it proceeds in small steps to the locomotive\n", - "problem, which I borrowed from Mosteller’s *Fifty Challenging Problems\n", - "in Probability with Solutions*, and from there to the German tank\n", - "problem, a famously successful application of Bayesian methods during\n", - "World War II.\n", - "\n", - "##Modeling and approximation\n", - "\n", - "Most chapters in this book are motivated by a real-world problem, so\n", - "they involve some degree of modeling. Before we can apply Bayesian\n", - "methods (or any other analysis), we have to make decisions about which\n", - "parts of the real-world system to include in the model and which details\n", - "we can abstract away.\n", - "\n", - "For example, in Chapter [prediction], the motivating problem is to\n", - "predict the winner of a hockey game. I model goal-scoring as a Poisson\n", - "process, which implies that a goal is equally likely at any point in the\n", - "game. That is not exactly true, but it is probably a good enough model\n", - "for most purposes.\n", - "\n", - "In Chapter [evidence] the motivating problem is interpreting SAT scores\n", - "(the SAT is a standardized test used for college admissions in the\n", - "United States). I start with a simple model that assumes that all SAT\n", - "questions are equally difficult, but in fact the designers of the SAT\n", - "deliberately include some questions that are relatively easy and some\n", - "that are relatively hard. I present a second model that accounts for\n", - "this aspect of the design, and show that it doesn’t have a big effect on\n", - "the results after all.\n", - "\n", - "I think it is important to include modeling as an explicit part of\n", - "problem solving because it reminds us to think about modeling errors\n", - "(that is, errors due to simplifications and assumptions of the model).\n", - "\n", - "Many of the methods in this book are based on discrete distributions,\n", - "which makes some people worry about numerical errors. But for real-world\n", - "problems, numerical errors are almost always smaller than modeling\n", - "errors.\n", - "\n", - "Furthermore, the discrete approach often allows better modeling\n", - "decisions, and I would rather have an approximate solution to a good\n", - "model than an exact solution to a bad model.\n", - "\n", - "On the other hand, continuous methods sometimes yield performance\n", - "advantages—for example by replacing a linear- or quadratic-time\n", - "computation with a constant-time solution.\n", - "\n", - "So I recommend a general process with these steps:\n", - "\n", - "1. While you are exploring a problem, start with simple models and\n", - " implement them in code that is clear, readable, and demonstrably\n", - " correct. Focus your attention on good modeling decisions, not\n", - " optimization.\n", - "\n", - "2. Once you have a simple model working, identify the biggest sources of\n", - " error. You might need to increase the number of values in a discrete\n", - " approximation, or increase the number of iterations in a Monte Carlo\n", - " simulation, or add details to the model.\n", - "\n", - "3. If the performance of your solution is good enough for your\n", - " application, you might not have to do any optimization. But if you\n", - " do, there are two approaches to consider. You can review your code\n", - " and look for optimizations; for example, if you cache previously\n", - " computed results you might be able to avoid redundant computation. Or\n", - " you can look for analytic methods that yield computational shortcuts.\n", - "\n", - "One benefit of this process is that Steps 1 and 2 tend to be fast, so\n", - "you can explore several alternative models before investing heavily in\n", - "any of them.\n", - "\n", - "Another benefit is that if you get to Step 3, you will be starting with\n", - "a reference implementation that is likely to be correct, which you can\n", - "use for regression testing (that is, checking that the optimized code\n", - "yields the same results, at least approximately).\n", - "\n", - "Working with the code\n", - "---------------------\n", - "\n", - "Many of the examples in this book use classes and functions defined in\n", - "thinkbayes.py. You can download this module from\n", - "http://thinkbayes.com/thinkbayes.py.\n", - "\n", - "Most chapters contain references to code you can download from\n", - "http://thinkbayes.com. Some of those files have dependencies you will\n", - "also have to download. I suggest you keep all of these files in the same\n", - "directory so they can import each other without changing the Python\n", - "search path.\n", - "\n", - "You can download these files one at a time as you need them, or you can\n", - "download them all at once from\n", - "http://thinkbayes.com/thinkbayes_code.zip. This file also contains the\n", - "data files used by some of the programs. When you unzip it, it creates a\n", - "directory named ``thinkbayes_code`` that contains all the code used in\n", - "this book.\n", - "\n", - "Or, if you are a Git user, you can get all of the files at once by\n", - "forking and cloning this repository:\n", - "https://github.com/AllenDowney/ThinkBayes.\n", - "\n", - "One of the modules I use is thinkplot.py, which provides wrappers for\n", - "some of the functions in pyplot. To use it, you need to install\n", - "matplotlib. If you don’t already have it, check your package manager to\n", - "see if it is available. Otherwise you can get download instructions from\n", - "http://matplotlib.org.\n", - "\n", - "Finally, some programs in this book use NumPy and SciPy, which are\n", - "available from http://numpy.org and http://scipy.org.\n", - "\n", - "Code style\n", - "----------\n", - "\n", - "Experienced Python programmers will notice that the code in this book\n", - "does not comply with PEP 8, which is the most common style guide for\n", - "Python (http://www.python.org/dev/peps/pep-0008/).\n", - "\n", - "Specifically, PEP 8 calls for lowercase function names with underscores\n", - "between words, ``like_this``. In this book and the accompanying code,\n", - "function and method names begin with a capital letter and use camel\n", - "case, ``LikeThis``.\n", - "\n", - "I broke this rule because I developed some of the code while I was a\n", - "Visiting Scientist at Google, so I followed the Google style guide,\n", - "which deviates from PEP 8 in a few places. Once I got used to Google\n", - "style, I found that I liked it. And at this point, it would be too much\n", - "trouble to change.\n", - "\n", - "Also on the topic of style, I write “Bayes’s theorem” with an *s* after\n", - "the apostrophe, which is preferred in some style guides and deprecated\n", - "in others. I don’t have a strong preference. I had to choose one, and\n", - "this is the one I chose.\n", - "\n", - "And finally one typographical note: throughout the book, I use PMF and\n", - "CDF for the mathematical concept of a probability mass function or\n", - "cumulative distribution function, and Pmf and Cdf to refer to the Python\n", - "objects I use to represent them.\n", - "\n", - "Prerequisites\n", - "-------------\n", - "\n", - "There are several excellent modules for doing Bayesian statistics in\n", - "Python, including pymc and OpenBUGS. I chose not to use them for this\n", - "book because you need a fair amount of background knowledge to get\n", - "started with these modules, and I want to keep the prerequisites\n", - "minimal. If you know Python and a little bit about probability, you are\n", - "ready to start this book.\n", - "\n", - "Chapter [intro] is about probability and Bayes’s theorem; it has no\n", - "code. Chapter [compstat] introduces Pmf, a thinly disguised Python\n", - "dictionary I use to represent a probability mass function (PMF). Then\n", - "Chapter [estimation] introduces Suite, a kind of Pmf that provides a\n", - "framework for doing Bayesian updates. And that’s just about all there is\n", - "to it.\n", - "\n", - "Well, almost. In some of the later chapters, I use analytic\n", - "distributions including the Gaussian (normal) distribution, the\n", - "exponential and Poisson distributions, and the beta distribution. In\n", - "Chapter [species] I break out the less-common Dirichlet distribution,\n", - "but I explain it as I go along. If you are not familiar with these\n", - "distributions, you can read about them on Wikipedia. You could also read\n", - "the companion to this book, *Think Stats*, or an introductory statistics\n", - "book (although I’m afraid most of them take a mathematical approach that\n", - "is not particularly helpful for practical purposes).\n", - "\n", - "Contributor List\n", - "----------------\n", - "\n", - "If you have a suggestion or correction, please send email to\n", - "*downey@allendowney.com*. If I make a change based on your feedback, I\n", - "will add you to the contributor list (unless you ask to be omitted).\n", - "\n", - "If you include at least part of the sentence the error appears in, that\n", - "makes it easy for me to search. Page and section numbers are fine, too,\n", - "but not as easy to work with. Thanks!\n", - "\n", - "- First, I have to acknowledge David MacKay’s excellent book,\n", - " *Information Theory, Inference, and Learning Algorithms*, which is\n", - " where I first came to understand Bayesian methods. With his\n", - " permission, I use several problems from his book as examples.\n", - "\n", - "- This book also benefited from my interactions with Sanjoy Mahajan,\n", - " especially in fall 2012, when I audited his class on Bayesian\n", - " Inference at Olin College.\n", - "\n", - "- I wrote parts of this book during project nights with the Boston\n", - " Python User Group, so I would like to thank them for their company\n", - " and pizza.\n", - "\n", - "- Jonathan Edwards sent in the first typo.\n", - "\n", - "- George Purkins found a markup error.\n", - "\n", - "- Olivier Yiptong sent several helpful suggestions.\n", - "\n", - "- Yuriy Pasichnyk found several errors.\n", - "\n", - "- Kristopher Overholt sent a long list of corrections and suggestions.\n", - "\n", - "- Robert Marcus found a misplaced *i*.\n", - "\n", - "- Max Hailperin suggested a clarification in Chapter [intro].\n", - "\n", - "- Markus Dobler pointed out that drawing cookies from a bowl with\n", - " replacement is an unrealistic scenario.\n", - "\n", - "- Tom Pollard and Paul A. Giannaros spotted a version problem with some\n", - " of the numbers in the train example.\n", - "\n", - "- Ram Limbu found a typo and suggested a clarification.\n", - "\n", - "- In spring 2013, students in my class, Computational Bayesian\n", - " Statistics, made many helpful corrections and suggestions: Kai\n", - " Austin, Claire Barnes, Kari Bender, Rachel Boy, Kat Mendoza, Arjun\n", - " Iyer, Ben Kroop, Nathan Lintz, Kyle McConnaughay, Alec Radford,\n", - " Brendan Ritter, and Evan Simpson.\n", - "\n", - "- Greg Marra and Matt Aasted helped me clarify the discussion of *The\n", - " Price is Right* problem.\n", - "\n", - "- Marcus Ogren pointed out that the original statement of the\n", - " locomotive problem was ambiguous.\n", - "\n", - "- Jasmine Kwityn and Dan Fauxsmith at O’Reilly Media proofread the book\n", - " and found many opportunities for improvement.\n", - "\n", - "- James Lawry spotted a math error.\n", - "\n", - "- Ben Kahle found a reference to the wrong figure.\n", - "\n", - "- Jeffrey Law found an inconsistency between the text and the code.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "# Preface\n", - "\n", - "##My theory, which is mine\n", - "\n", - "\n", - "The premise of this book, and the other books in the *Think\n", - "X* series, is that if you know how to program, you can use that\n", - "skill to learn other topics.\n", - "\n", - "Most books on Bayesian statistics use mathematical notation and present\n", - "ideas in terms of mathematical concepts like calculus. This book uses\n", - "Python code instead of math, and discrete approximations instead of\n", - "continuous mathematics. As a result, what would be an integral in a math\n", - "book becomes a summation, and most operations on probability\n", - "distributions are simple loops.\n", - "\n", - "I think this presentation is easier to understand, at least for people\n", - "with programming skills. It is also more general, because when we make\n", - "modeling decisions, we can choose the most appropriate model without\n", - "worrying too much about whether the model lends itself to conventional\n", - "analysis.\n", - "\n", - "Also, it provides a smooth development path from simple examples to\n", - "real-world problems. Chapter [estimation] is a good example. It starts\n", - "with a simple example involving dice, one of the staples of basic\n", - "probability. From there it proceeds in small steps to the locomotive\n", - "problem, which I borrowed from Mosteller’s *Fifty Challenging\n", - "Problems in Probability with Solutions*, and from there to the\n", - "German tank problem, a famously successful application of Bayesian\n", - "methods during World War II.\n", - "\n", - "##Modeling and approximation\n", - "\n", - "Most chapters in this book are motivated by a real-world problem, so\n", - "they involve some degree of modeling. Before we can apply Bayesian\n", - "methods (or any other analysis), we have to make decisions about which\n", - "parts of the real-world system to include in the model and which details\n", - "we can abstract away.\n", - "\n", - "For example, in Chapter [prediction], the motivating problem is to\n", - "predict the winner of a hockey game. I model goal-scoring as a Poisson\n", - "process, which implies that a goal is equally likely at any point in the\n", - "game. That is not exactly true, but it is probably a good enough model\n", - "for most purposes.\n", - "\n", - "In Chapter [evidence] the motivating problem is interpreting SAT scores\n", - "(the SAT is a standardized test used for college admissions in the\n", - "United States). I start with a simple model that assumes that all SAT\n", - "questions are equally difficult, but in fact the designers of the SAT\n", - "deliberately include some questions that are relatively easy and some\n", - "that are relatively hard. I present a second model that accounts for\n", - "this aspect of the design, and show that it doesn’t have a big effect on\n", - "the results after all.\n", - "\n", - "I think it is important to include modeling as an explicit part of\n", - "problem solving because it reminds us to think about modeling errors\n", - "(that is, errors due to simplifications and assumptions of the model).\n", - "\n", - "Many of the methods in this book are based on discrete distributions,\n", - "which makes some people worry about numerical errors. But for real-world\n", - "problems, numerical errors are almost always smaller than modeling\n", - "errors.\n", - "\n", - "Furthermore, the discrete approach often allows better modeling\n", - "decisions, and I would rather have an approximate solution to a good\n", - "model than an exact solution to a bad model.\n", - "\n", - "On the other hand, continuous methods sometimes yield performance\n", - "advantages—for example by replacing a linear- or quadratic-time\n", - "computation with a constant-time solution.\n", - "\n", - "So I recommend a general process with these steps:\n", - "\n", - "1. While you are exploring a problem, start with simple models and\n", - " implement them in code that is clear, readable, and demonstrably\n", - " correct. Focus your attention on good modeling decisions, not\n", - " optimization.\n", - "\n", - "2. Once you have a simple model working, identify the biggest sources\n", - " of error. You might need to increase the number of values in a\n", - " discrete approximation, or increase the number of iterations in a\n", - " Monte Carlo simulation, or add details to the model.\n", - "\n", - "3. If the performance of your solution is good enough for your\n", - " application, you might not have to do any optimization. But if you\n", - " do, there are two approaches to consider. You can review your code\n", - " and look for optimizations; for example, if you cache previously\n", - " computed results you might be able to avoid redundant computation.\n", - " Or you can look for analytic methods that yield computational\n", - " shortcuts.\n", - "\n", - "One benefit of this process is that Steps 1 and 2 tend to be fast, so\n", - "you can explore several alternative models before investing heavily in\n", - "any of them.\n", - "\n", - "Another benefit is that if you get to Step 3, you will be starting with\n", - "a reference implementation that is likely to be correct, which you can\n", - "use for regression testing (that is, checking that the optimized code\n", - "yields the same results, at least approximately).\n", - "\n", - "## Working with the code {#download}\n", - "\n", - "Many of the examples in this book use classes and functions defined in\n", - "`thinkbayes.py`. You can download this module from\n", - ".\n", - "\n", - "Most chapters contain references to code you can download from\n", - ". Some of those files have dependencies you will\n", - "also have to download. I suggest you keep all of these files in the same\n", - "directory so they can import each other without changing the Python\n", - "search path.\n", - "\n", - "You can download these files one at a time as you need them, or you can\n", - "download them all at once from\n", - ". This file also contains the\n", - "data files used by some of the programs. When you unzip it, it creates a\n", - "directory named `thinkbayes_code` that contains all the code used in\n", - "this book.\n", - "\n", - "Or, if you are a Git user, you can get all of the files at once by\n", - "forking and cloning this repository:\n", - ".\n", - "\n", - "One of the modules I use is `thinkplot.py`, which provides\n", - "wrappers for some of the functions in `pyplot`. To use it,\n", - "you need to install `matplotlib`. If you don’t already have\n", - "it, check your package manager to see if it is available. Otherwise you\n", - "can get download instructions from .\n", - "\n", - "Finally, some programs in this book use NumPy and SciPy, which are\n", - "available from and .\n", - "\n", - "Code style\n", - "----------\n", - "\n", - "Experienced Python programmers will notice that the code in this book\n", - "does not comply with PEP 8, which is the most common style guide for\n", - "Python ().\n", - "\n", - "Specifically, PEP 8 calls for lowercase function names with underscores\n", - "between words, `like_this`. In this book and the accompanying code,\n", - "function and method names begin with a capital letter and use camel\n", - "case, `LikeThis`.\n", - "\n", - "I broke this rule because I developed some of the code while I was a\n", - "Visiting Scientist at Google, so I followed the Google style guide,\n", - "which deviates from PEP 8 in a few places. Once I got used to Google\n", - "style, I found that I liked it. And at this point, it would be too much\n", - "trouble to change.\n", - "\n", - "Also on the topic of style, I write “Bayes’s theorem” with an\n", - "*s* after the apostrophe, which is preferred in some style\n", - "guides and deprecated in others. I don’t have a strong preference. I had\n", - "to choose one, and this is the one I chose.\n", - "\n", - "And finally one typographical note: throughout the book, I use PMF and\n", - "CDF for the mathematical concept of a probability mass function or\n", - "cumulative distribution function, and Pmf and Cdf to refer to the Python\n", - "objects I use to represent them.\n", - "\n", - "##Prerequisites\n", - "\n", - "There are several excellent modules for doing Bayesian statistics in\n", - "Python, including `pymc` and OpenBUGS. I chose not to use\n", - "them for this book because you need a fair amount of background\n", - "knowledge to get started with these modules, and I want to keep the\n", - "prerequisites minimal. If you know Python and a little bit about\n", - "probability, you are ready to start this book.\n", - "\n", - "Chapter [intro] is about probability and Bayes’s theorem; it has no\n", - "code. Chapter [compstat] introduces `Pmf`, a thinly disguised\n", - "Python dictionary I use to represent a probability mass function (PMF).\n", - "Then Chapter [estimation] introduces `Suite`, a kind of Pmf\n", - "that provides a framework for doing Bayesian updates. And that’s just\n", - "about all there is to it.\n", - "\n", - "Well, almost. In some of the later chapters, I use analytic\n", - "distributions including the Gaussian (normal) distribution, the\n", - "exponential and Poisson distributions, and the beta distribution. In\n", - "Chapter [species] I break out the less-common Dirichlet distribution,\n", - "but I explain it as I go along. If you are not familiar with these\n", - "distributions, you can read about them on Wikipedia. You could also read\n", - "the companion to this book, *Think Stats*, or an\n", - "introductory statistics book (although I’m afraid most of them take a\n", - "mathematical approach that is not particularly helpful for practical\n", - "purposes).\n", - "\n", - "Contributor List {#contributor-list .unnumbered}\n", - "----------------\n", - "\n", - "If you have a suggestion or correction, please send email to\n", - "*downey@allendowney.com*. If I make a change based on your\n", - "feedback, I will add you to the contributor list (unless you ask to be\n", - "omitted).\n", - "\n", - "If you include at least part of the sentence the error appears in, that\n", - "makes it easy for me to search. Page and section numbers are fine, too,\n", - "but not as easy to work with. Thanks!\n", - "\n", - "- First, I have to acknowledge David MacKay’s excellent book,\n", - " *Information Theory, Inference, and Learning\n", - " Algorithms*, which is where I first came to understand\n", - " Bayesian methods. With his permission, I use several problems from\n", - " his book as examples.\n", - "\n", - "- This book also benefited from my interactions with Sanjoy Mahajan,\n", - " especially in fall 2012, when I audited his class on Bayesian\n", - " Inference at Olin College.\n", - "\n", - "- I wrote parts of this book during project nights with the Boston\n", - " Python User Group, so I would like to thank them for their company\n", - " and pizza.\n", - "\n", - "- Jonathan Edwards sent in the first typo.\n", - "\n", - "- George Purkins found a markup error.\n", - "\n", - "- Olivier Yiptong sent several helpful suggestions.\n", - "\n", - "- Yuriy Pasichnyk found several errors.\n", - "\n", - "- Kristopher Overholt sent a long list of corrections and suggestions.\n", - "\n", - "- Robert Marcus found a misplaced *i*.\n", - "\n", - "- Max Hailperin suggested a clarification in Chapter [intro].\n", - "\n", - "- Markus Dobler pointed out that drawing cookies from a bowl with\n", - " replacement is an unrealistic scenario.\n", - "\n", - "- Tom Pollard and Paul A. Giannaros spotted a version problem with\n", - " some of the numbers in the train example.\n", - "\n", - "- Ram Limbu found a typo and suggested a clarification.\n", - "\n", - "- In spring 2013, students in my class, Computational Bayesian\n", - " Statistics, made many helpful corrections and suggestions: Kai\n", - " Austin, Claire Barnes, Kari Bender, Rachel Boy, Kat Mendoza, Arjun\n", - " Iyer, Ben Kroop, Nathan Lintz, Kyle McConnaughay, Alec Radford,\n", - " Brendan Ritter, and Evan Simpson.\n", - "\n", - "- Greg Marra and Matt Aasted helped me clarify the discussion of\n", - " *The Price is Right* problem.\n", - "\n", - "- Marcus Ogren pointed out that the original statement of the\n", - " locomotive problem was ambiguous.\n", - "\n", - "- Jasmine Kwityn and Dan Fauxsmith at O’Reilly Media proofread the\n", - " book and found many opportunities for improvement.\n", - "\n", - "- James Lawry spotted a math error.\n", - "\n", - "- Ben Kahle found a reference to the wrong figure.\n", - "\n", - "- Jeffrey Law found an inconsistency between the text and the code." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.4.1" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Preface\n", + "\n", + "## My theory, which is mine\n", + "\n", + "The premise of this book, and the other books in the *Think X* series,\n", + "is that if you know how to program, you can use that skill to learn\n", + "other topics.\n", + "\n", + "Most books on Bayesian statistics use mathematical notation and present\n", + "ideas in terms of mathematical concepts like calculus. This book uses\n", + "Python code instead of math, and discrete approximations instead of\n", + "continuous mathematics. As a result, what would be an integral in a math\n", + "book becomes a summation, and most operations on probability\n", + "distributions are simple loops.\n", + "\n", + "I think this presentation is easier to understand, at least for people\n", + "with programming skills. It is also more general, because when we make\n", + "modeling decisions, we can choose the most appropriate model without\n", + "worrying too much about whether the model lends itself to conventional\n", + "analysis.\n", + "\n", + "Also, it provides a smooth development path from simple examples to\n", + "real-world problems. Chapter [estimation] is a good example. It starts\n", + "with a simple example involving dice, one of the staples of basic\n", + "probability. From there it proceeds in small steps to the locomotive\n", + "problem, which I borrowed from Mosteller’s *Fifty Challenging Problems\n", + "in Probability with Solutions*, and from there to the German tank\n", + "problem, a famously successful application of Bayesian methods during\n", + "World War II.\n", + "\n", + "## Modeling and approximation\n", + "\n", + "Most chapters in this book are motivated by a real-world problem, so\n", + "they involve some degree of modeling. Before we can apply Bayesian\n", + "methods (or any other analysis), we have to make decisions about which\n", + "parts of the real-world system to include in the model and which details\n", + "we can abstract away.\n", + "\n", + "For example, in Chapter [prediction], the motivating problem is to\n", + "predict the winner of a hockey game. I model goal-scoring as a Poisson\n", + "process, which implies that a goal is equally likely at any point in the\n", + "game. That is not exactly true, but it is probably a good enough model\n", + "for most purposes.\n", + "\n", + "In Chapter [evidence] the motivating problem is interpreting SAT scores\n", + "(the SAT is a standardized test used for college admissions in the\n", + "United States). I start with a simple model that assumes that all SAT\n", + "questions are equally difficult, but in fact the designers of the SAT\n", + "deliberately include some questions that are relatively easy and some\n", + "that are relatively hard. I present a second model that accounts for\n", + "this aspect of the design, and show that it doesn’t have a big effect on\n", + "the results after all.\n", + "\n", + "I think it is important to include modeling as an explicit part of\n", + "problem solving because it reminds us to think about modeling errors\n", + "(that is, errors due to simplifications and assumptions of the model).\n", + "\n", + "Many of the methods in this book are based on discrete distributions,\n", + "which makes some people worry about numerical errors. But for real-world\n", + "problems, numerical errors are almost always smaller than modeling\n", + "errors.\n", + "\n", + "Furthermore, the discrete approach often allows better modeling\n", + "decisions, and I would rather have an approximate solution to a good\n", + "model than an exact solution to a bad model.\n", + "\n", + "On the other hand, continuous methods sometimes yield performance\n", + "advantages—for example by replacing a linear- or quadratic-time\n", + "computation with a constant-time solution.\n", + "\n", + "So I recommend a general process with these steps:\n", + "\n", + "1. While you are exploring a problem, start with simple models and\n", + " implement them in code that is clear, readable, and demonstrably\n", + " correct. Focus your attention on good modeling decisions, not\n", + " optimization.\n", + "\n", + "2. Once you have a simple model working, identify the biggest sources of\n", + " error. You might need to increase the number of values in a discrete\n", + " approximation, or increase the number of iterations in a Monte Carlo\n", + " simulation, or add details to the model.\n", + "\n", + "3. If the performance of your solution is good enough for your\n", + " application, you might not have to do any optimization. But if you\n", + " do, there are two approaches to consider. You can review your code\n", + " and look for optimizations; for example, if you cache previously\n", + " computed results you might be able to avoid redundant computation. Or\n", + " you can look for analytic methods that yield computational shortcuts.\n", + "\n", + "One benefit of this process is that Steps 1 and 2 tend to be fast, so\n", + "you can explore several alternative models before investing heavily in\n", + "any of them.\n", + "\n", + "Another benefit is that if you get to Step 3, you will be starting with\n", + "a reference implementation that is likely to be correct, which you can\n", + "use for regression testing (that is, checking that the optimized code\n", + "yields the same results, at least approximately).\n", + "\n", + "Working with the code\n", + "---------------------\n", + "\n", + "Many of the examples in this book use classes and functions defined in\n", + "thinkbayes.py. You can download this module from\n", + "http://thinkbayes.com/thinkbayes.py.\n", + "\n", + "Most chapters contain references to code you can download from\n", + "http://thinkbayes.com. Some of those files have dependencies you will\n", + "also have to download. I suggest you keep all of these files in the same\n", + "directory so they can import each other without changing the Python\n", + "search path.\n", + "\n", + "You can download these files one at a time as you need them, or you can\n", + "download them all at once from\n", + "http://thinkbayes.com/thinkbayes_code.zip. This file also contains the\n", + "data files used by some of the programs. When you unzip it, it creates a\n", + "directory named ``thinkbayes_code`` that contains all the code used in\n", + "this book.\n", + "\n", + "Or, if you are a Git user, you can get all of the files at once by\n", + "forking and cloning this repository:\n", + "https://github.com/AllenDowney/ThinkBayes.\n", + "\n", + "One of the modules I use is thinkplot.py, which provides wrappers for\n", + "some of the functions in pyplot. To use it, you need to install\n", + "matplotlib. If you don’t already have it, check your package manager to\n", + "see if it is available. Otherwise you can get download instructions from\n", + "http://matplotlib.org.\n", + "\n", + "Finally, some programs in this book use NumPy and SciPy, which are\n", + "available from http://numpy.org and http://scipy.org.\n", + "\n", + "## Code style\n", + "\n", + "Experienced Python programmers will notice that the code in this book\n", + "does not comply with PEP 8, which is the most common style guide for\n", + "Python (http://www.python.org/dev/peps/pep-0008/).\n", + "\n", + "Specifically, PEP 8 calls for lowercase function names with underscores\n", + "between words, ``like_this``. In this book and the accompanying code,\n", + "function and method names begin with a capital letter and use camel\n", + "case, ``LikeThis``.\n", + "\n", + "I broke this rule because I developed some of the code while I was a\n", + "Visiting Scientist at Google, so I followed the Google style guide,\n", + "which deviates from PEP 8 in a few places. Once I got used to Google\n", + "style, I found that I liked it. And at this point, it would be too much\n", + "trouble to change.\n", + "\n", + "Also on the topic of style, I write “Bayes’s theorem” with an *s* after\n", + "the apostrophe, which is preferred in some style guides and deprecated\n", + "in others. I don’t have a strong preference. I had to choose one, and\n", + "this is the one I chose.\n", + "\n", + "And finally one typographical note: throughout the book, I use PMF and\n", + "CDF for the mathematical concept of a probability mass function or\n", + "cumulative distribution function, and Pmf and Cdf to refer to the Python\n", + "objects I use to represent them.\n", + "\n", + "## Prerequisites\n", + "\n", + "There are several excellent modules for doing Bayesian statistics in\n", + "Python, including pymc and OpenBUGS. I chose not to use them for this\n", + "book because you need a fair amount of background knowledge to get\n", + "started with these modules, and I want to keep the prerequisites\n", + "minimal. If you know Python and a little bit about probability, you are\n", + "ready to start this book.\n", + "\n", + "Chapter [intro] is about probability and Bayes’s theorem; it has no\n", + "code. Chapter [compstat] introduces Pmf, a thinly disguised Python\n", + "dictionary I use to represent a probability mass function (PMF). Then\n", + "Chapter [estimation] introduces Suite, a kind of Pmf that provides a\n", + "framework for doing Bayesian updates. And that’s just about all there is\n", + "to it.\n", + "\n", + "Well, almost. In some of the later chapters, I use analytic\n", + "distributions including the Gaussian (normal) distribution, the\n", + "exponential and Poisson distributions, and the beta distribution. In\n", + "Chapter [species] I break out the less-common Dirichlet distribution,\n", + "but I explain it as I go along. If you are not familiar with these\n", + "distributions, you can read about them on Wikipedia. You could also read\n", + "the companion to this book, *Think Stats*, or an introductory statistics\n", + "book (although I’m afraid most of them take a mathematical approach that\n", + "is not particularly helpful for practical purposes).\n", + "\n", + "## Contributor List\n", + "\n", + "If you have a suggestion or correction, please send email to\n", + "*downey@allendowney.com*. If I make a change based on your feedback, I\n", + "will add you to the contributor list (unless you ask to be omitted).\n", + "\n", + "If you include at least part of the sentence the error appears in, that\n", + "makes it easy for me to search. Page and section numbers are fine, too,\n", + "but not as easy to work with. Thanks!\n", + "\n", + "- First, I have to acknowledge David MacKay’s excellent book,\n", + " *Information Theory, Inference, and Learning Algorithms*, which is\n", + " where I first came to understand Bayesian methods. With his\n", + " permission, I use several problems from his book as examples.\n", + "\n", + "- This book also benefited from my interactions with Sanjoy Mahajan,\n", + " especially in fall 2012, when I audited his class on Bayesian\n", + " Inference at Olin College.\n", + "\n", + "- I wrote parts of this book during project nights with the Boston\n", + " Python User Group, so I would like to thank them for their company\n", + " and pizza.\n", + "\n", + "- Jonathan Edwards sent in the first typo.\n", + "\n", + "- George Purkins found a markup error.\n", + "\n", + "- Olivier Yiptong sent several helpful suggestions.\n", + "\n", + "- Yuriy Pasichnyk found several errors.\n", + "\n", + "- Kristopher Overholt sent a long list of corrections and suggestions.\n", + "\n", + "- Robert Marcus found a misplaced *i*.\n", + "\n", + "- Max Hailperin suggested a clarification in Chapter [intro].\n", + "\n", + "- Markus Dobler pointed out that drawing cookies from a bowl with\n", + " replacement is an unrealistic scenario.\n", + "\n", + "- Tom Pollard and Paul A. Giannaros spotted a version problem with some\n", + " of the numbers in the train example.\n", + "\n", + "- Ram Limbu found a typo and suggested a clarification.\n", + "\n", + "- In spring 2013, students in my class, Computational Bayesian\n", + " Statistics, made many helpful corrections and suggestions: Kai\n", + " Austin, Claire Barnes, Kari Bender, Rachel Boy, Kat Mendoza, Arjun\n", + " Iyer, Ben Kroop, Nathan Lintz, Kyle McConnaughay, Alec Radford,\n", + " Brendan Ritter, and Evan Simpson.\n", + "\n", + "- Greg Marra and Matt Aasted helped me clarify the discussion of *The\n", + " Price is Right* problem.\n", + "\n", + "- Marcus Ogren pointed out that the original statement of the\n", + " locomotive problem was ambiguous.\n", + "\n", + "- Jasmine Kwityn and Dan Fauxsmith at O’Reilly Media proofread the book\n", + " and found many opportunities for improvement.\n", + "\n", + "- James Lawry spotted a math error.\n", + "\n", + "- Ben Kahle found a reference to the wrong figure.\n", + "\n", + "- Jeffrey Law found an inconsistency between the text and the code.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/notebooks/hockey_data.csv b/notebooks/hockey_data.csv new file mode 100644 index 0000000..897a1fd --- /dev/null +++ b/notebooks/hockey_data.csv @@ -0,0 +1,9841 @@ +Season,Game,Team,Pd1,Pd2,Pd3,OT,SO Win,Total +2009,20001,NYR,0,1,1,0,0,2 +2009,20001,TBL,0,0,1,0,0,1 +2009,20002,OTT,1,2,0,0,0,3 +2009,20002,PIT,1,1,1,1,0,4 +2009,20003,NYR,0,2,0,0,0,2 +2009,20003,TBL,1,0,0,0,0,1 +2009,20004,OTT,0,1,2,0,0,3 +2009,20004,PIT,0,0,1,0,0,1 +2009,20005,DET,0,1,1,0,0,2 +2009,20005,TOR,1,1,1,0,0,3 +2009,20006,BOS,2,1,2,0,0,5 +2009,20006,COL,2,1,1,0,0,4 +2009,20007,CGY,0,0,0,0,0,0 +2009,20007,VAN,1,2,3,0,0,6 +2009,20008,ANA,0,0,1,0,0,1 +2009,20008,SJS,0,2,2,0,0,4 +2009,20009,NJD,1,1,0,0,0,2 +2009,20009,NYI,1,0,0,0,0,1 +2009,20010,CHI,1,0,1,0,0,2 +2009,20010,NYR,1,2,1,0,0,4 +2009,20011,CAR,2,2,2,0,0,6 +2009,20011,FLA,2,1,1,0,0,4 +2009,20012,BUF,1,0,0,0,1,2 +2009,20012,MTL,1,0,0,0,0,1 +2009,20013,ATL,3,1,3,0,0,7 +2009,20013,WSH,1,3,0,0,0,4 +2009,20014,NSH,1,1,0,0,0,2 +2009,20014,STL,2,3,0,0,0,5 +2009,20015,CBJ,1,0,3,1,0,5 +2009,20015,DAL,0,2,2,0,0,4 +2009,20016,MTL,2,4,0,0,0,6 +2009,20016,TOR,0,1,0,0,0,1 +2009,20017,DET,1,0,2,0,0,3 +2009,20017,OTT,1,1,0,0,0,2 +2009,20018,NYI,4,0,1,0,0,5 +2009,20018,STL,1,0,1,0,0,2 +2009,20019,NYR,4,0,0,0,0,4 +2009,20019,PHI,0,2,1,0,0,3 +2009,20020,CHI,2,0,0,0,0,2 +2009,20020,WSH,1,1,2,0,0,4 +2009,20021,ATL,1,1,0,0,0,2 +2009,20021,FLA,1,1,0,1,0,3 +2009,20022,NJD,0,0,1,1,0,2 +2009,20022,PIT,1,0,0,0,0,1 +2009,20023,CAR,0,1,2,1,0,4 +2009,20023,TBL,2,1,0,0,0,3 +2009,20024,DAL,1,0,0,0,0,1 +2009,20024,NSH,2,0,1,0,0,3 +2009,20025,BOS,1,0,2,0,0,3 +2009,20025,MIN,1,3,0,0,0,4 +2009,20026,CGY,3,1,0,0,0,4 +2009,20026,VAN,1,2,1,1,0,5 +2009,20027,CBJ,0,1,0,0,0,1 +2009,20027,PHX,2,1,0,0,0,3 +2009,20028,LAK,1,0,0,0,0,1 +2009,20028,SJS,0,2,1,0,0,3 +2009,20029,COL,0,1,1,0,0,2 +2009,20029,EDM,1,0,2,0,0,3 +2009,20030,ANA,0,1,1,0,0,2 +2009,20030,PHX,1,2,1,0,0,4 +2009,20031,LAK,0,0,0,0,0,0 +2009,20031,SJS,0,1,0,0,0,1 +2009,20032,STL,0,3,1,0,1,5 +2009,20032,TOR,3,1,0,0,0,4 +2009,20033,BUF,1,4,2,0,0,7 +2009,20033,NYI,0,0,1,0,0,1 +2009,20034,NJD,0,1,0,0,0,1 +2009,20034,NYR,1,1,2,0,0,4 +2009,20035,MTL,0,1,4,0,0,5 +2009,20035,PHI,0,2,1,0,0,3 +2009,20036,VAN,1,0,0,0,0,1 +2009,20036,WSH,2,3,0,0,0,5 +2009,20037,CAR,0,1,0,0,0,1 +2009,20037,DET,1,1,1,0,0,3 +2009,20038,CHI,1,1,0,0,0,2 +2009,20038,NSH,2,0,0,0,1,3 +2009,20039,PHI,0,2,0,0,0,2 +2009,20039,PIT,0,2,0,1,0,3 +2009,20040,ATL,0,1,1,0,0,2 +2009,20040,MIN,1,0,3,0,0,4 +2009,20041,CGY,3,0,2,0,0,5 +2009,20041,COL,2,1,1,0,0,4 +2009,20042,ANA,2,0,1,0,0,3 +2009,20042,LAK,1,2,3,0,0,6 +2009,20043,CBJ,1,1,0,0,0,2 +2009,20043,SJS,0,3,2,0,0,5 +2009,20044,BUF,0,2,1,0,0,3 +2009,20044,NYR,1,0,0,0,0,1 +2009,20045,BOS,0,1,2,0,0,3 +2009,20045,MTL,3,0,0,0,1,4 +2009,20046,CHI,0,2,2,0,0,4 +2009,20046,PHX,0,1,0,0,0,1 +2009,20047,DAL,1,3,2,0,0,6 +2009,20047,NSH,0,3,1,0,0,4 +2009,20048,ANA,1,1,0,0,0,2 +2009,20048,EDM,1,1,1,0,0,3 +2009,20049,ATL,0,0,0,0,0,0 +2009,20049,NJD,0,1,0,0,0,1 +2009,20050,PIT,2,1,0,0,0,3 +2009,20050,WSH,0,1,3,0,0,4 +2009,20051,NYI,1,1,1,1,0,4 +2009,20051,TBL,0,0,3,0,0,3 +2009,20052,FLA,0,2,0,0,0,2 +2009,20052,MIN,1,3,2,0,0,6 +2009,20053,DET,1,2,0,0,0,3 +2009,20053,VAN,2,0,1,1,0,4 +2009,20054,DAL,1,0,0,0,0,1 +2009,20054,STL,2,3,1,0,0,6 +2009,20055,COL,2,2,1,0,0,5 +2009,20055,PHI,0,2,0,0,0,2 +2009,20056,OTT,3,1,2,0,0,6 +2009,20056,PHX,0,0,3,0,0,3 +2009,20057,NYR,0,0,0,0,1,1 +2009,20057,TOR,0,0,0,0,0,0 +2009,20058,CBJ,3,1,1,0,0,5 +2009,20058,NSH,0,2,1,0,0,3 +2009,20059,BUF,3,1,1,0,0,5 +2009,20059,VAN,0,1,1,0,0,2 +2009,20060,CGY,1,1,1,0,0,3 +2009,20060,EDM,1,3,0,0,0,4 +2009,20061,ANA,1,1,2,0,0,4 +2009,20061,SJS,0,0,0,0,0,0 +2009,20062,CAR,0,3,0,0,0,3 +2009,20062,LAK,1,1,1,1,0,4 +2009,20063,COL,2,3,0,0,0,5 +2009,20063,DAL,1,1,2,0,0,4 +2009,20064,MTL,2,2,0,0,0,4 +2009,20064,PHX,0,0,1,0,0,1 +2009,20065,BOS,1,2,1,0,0,4 +2009,20065,OTT,1,1,0,0,0,2 +2009,20066,PIT,1,1,2,0,0,4 +2009,20066,TOR,1,0,0,0,0,1 +2009,20067,NJD,1,0,2,0,1,4 +2009,20067,WSH,1,1,1,0,0,3 +2009,20068,ATL,2,0,0,0,1,3 +2009,20068,BUF,0,1,1,0,0,2 +2009,20069,FLA,1,1,0,0,0,2 +2009,20069,NYI,0,0,0,0,0,0 +2009,20070,DET,2,1,1,1,0,5 +2009,20070,NYR,1,1,2,0,0,4 +2009,20071,MIN,0,0,0,0,1,1 +2009,20071,TBL,0,0,0,0,0,0 +2009,20072,CBJ,0,2,1,0,0,3 +2009,20072,NSH,3,1,2,0,0,6 +2009,20073,CHI,1,1,1,0,0,3 +2009,20073,STL,0,1,2,0,1,4 +2009,20074,CGY,2,0,0,0,0,2 +2009,20074,EDM,0,3,0,0,0,3 +2009,20075,PHI,1,2,1,0,0,4 +2009,20075,SJS,0,2,2,1,0,5 +2009,20076,CHI,2,0,2,0,0,4 +2009,20076,VAN,1,0,1,0,0,2 +2009,20077,ANA,0,1,0,0,0,1 +2009,20077,CAR,1,1,1,0,0,3 +2009,20078,BOS,0,1,0,0,0,1 +2009,20078,PIT,1,0,0,0,1,2 +2009,20079,DAL,0,1,1,0,0,2 +2009,20079,NYR,1,0,0,0,0,1 +2009,20080,MTL,0,1,2,0,0,3 +2009,20080,FLA,0,0,1,0,0,1 +2009,20081,COL,1,3,0,0,0,4 +2009,20081,LAK,1,1,1,0,0,3 +2009,20082,BOS,1,1,0,0,0,2 +2009,20082,BUF,0,1,1,0,1,3 +2009,20083,ANA,2,0,0,0,1,3 +2009,20083,TOR,0,1,1,0,0,2 +2009,20084,CBJ,2,1,1,0,0,4 +2009,20084,VAN,1,0,1,0,0,2 +2009,20085,ATL,0,0,2,0,0,2 +2009,20085,TBL,1,1,0,1,0,3 +2009,20086,CGY,0,2,0,0,0,2 +2009,20086,WSH,1,0,0,0,0,1 +2009,20087,FLA,2,1,0,0,0,3 +2009,20087,OTT,0,1,0,0,0,1 +2009,20088,DAL,0,0,0,0,0,0 +2009,20088,NJD,1,2,2,0,0,5 +2009,20089,PHI,3,1,2,0,0,6 +2009,20089,SJS,4,1,1,0,1,7 +2009,20090,DET,1,2,1,0,0,4 +2009,20090,STL,0,1,2,0,0,3 +2009,20091,CHI,2,1,0,0,0,3 +2009,20091,EDM,0,0,0,0,0,0 +2009,20092,BOS,2,0,0,0,0,2 +2009,20092,TOR,0,2,2,0,0,4 +2009,20093,DAL,3,2,0,0,0,5 +2009,20093,NYI,0,1,2,0,0,3 +2009,20094,CAR,1,0,0,0,0,1 +2009,20094,PIT,0,0,4,0,0,4 +2009,20095,CGY,0,1,4,0,0,5 +2009,20095,NSH,0,3,0,0,0,3 +2009,20096,BUF,1,0,2,1,0,4 +2009,20096,MIN,0,2,1,0,0,3 +2009,20097,COL,0,3,1,0,0,4 +2009,20097,EDM,0,1,0,0,0,1 +2009,20098,PHX,0,0,2,0,0,2 +2009,20098,WSH,0,1,0,0,0,1 +2009,20099,ANA,1,3,0,0,0,4 +2009,20099,OTT,0,0,3,0,0,3 +2009,20100,NJD,3,0,0,0,0,3 +2009,20100,PHI,2,2,2,0,0,6 +2009,20101,CBJ,1,0,0,0,0,1 +2009,20101,NYR,0,2,1,0,0,3 +2009,20102,FLA,3,1,0,0,0,4 +2009,20102,SJS,1,2,0,0,0,3 +2009,20103,ATL,1,0,2,0,0,3 +2009,20103,DET,2,2,1,0,0,5 +2009,20104,LAK,2,0,2,0,0,4 +2009,20104,STL,0,0,0,0,0,0 +2009,20105,ATL,2,1,1,0,0,4 +2009,20105,BOS,0,3,2,0,0,5 +2009,20106,OTT,0,1,1,0,0,2 +2009,20106,TOR,1,1,1,0,0,3 +2009,20107,ANA,2,3,1,0,0,6 +2009,20107,MTL,2,1,1,0,0,4 +2009,20108,CAR,2,1,1,0,0,4 +2009,20108,NYI,0,1,2,0,0,3 +2009,20109,NYR,0,0,2,0,1,3 +2009,20109,PIT,1,1,0,0,0,2 +2009,20110,NJD,0,2,0,0,0,2 +2009,20110,PHI,1,1,0,1,0,3 +2009,20111,SJS,0,2,1,0,0,3 +2009,20111,TBL,0,0,0,0,0,0 +2009,20112,LAK,0,2,2,0,0,4 +2009,20112,NSH,3,0,2,0,0,5 +2009,20113,CBJ,0,0,1,0,0,1 +2009,20113,MIN,1,0,1,0,0,2 +2009,20114,DAL,2,1,2,0,0,5 +2009,20114,WSH,2,2,1,1,0,6 +2009,20115,CHI,1,3,1,0,0,5 +2009,20115,DET,1,1,3,0,1,6 +2009,20116,FLA,0,0,0,0,0,0 +2009,20116,STL,1,2,1,0,0,4 +2009,20117,BUF,0,1,0,0,0,1 +2009,20117,COL,0,0,1,0,1,2 +2009,20118,EDM,0,3,0,0,0,3 +2009,20118,VAN,1,2,3,0,0,6 +2009,20119,CGY,2,1,1,0,0,4 +2009,20119,PHX,1,0,0,0,0,1 +2009,20120,BUF,0,0,2,0,0,2 +2009,20120,OTT,1,3,1,0,0,5 +2009,20121,NYI,1,0,1,0,0,2 +2009,20121,NYR,1,1,2,0,0,4 +2009,20122,ANA,1,0,2,0,0,3 +2009,20122,CBJ,1,1,0,0,0,2 +2009,20123,CHI,1,0,1,0,0,2 +2009,20123,MIN,1,2,0,0,0,3 +2009,20124,BOS,0,0,0,1,0,1 +2009,20124,EDM,0,0,0,0,0,0 +2009,20125,DET,1,1,1,0,1,4 +2009,20125,LAK,1,1,1,0,0,3 +2009,20126,NSH,1,1,1,0,0,3 +2009,20126,WSH,2,0,1,0,1,4 +2009,20127,ATL,0,0,0,0,0,0 +2009,20127,PHI,1,2,4,0,0,7 +2009,20128,TBL,2,1,0,0,0,3 +2009,20128,TOR,1,1,0,0,0,2 +2009,20129,MTL,2,0,0,0,1,3 +2009,20129,CAR,1,0,1,0,0,2 +2009,20130,CGY,0,0,3,0,0,3 +2009,20130,COL,0,0,0,0,0,0 +2009,20131,BOS,1,0,0,0,0,1 +2009,20131,VAN,0,0,0,0,0,0 +2009,20132,PIT,0,0,1,0,0,1 +2009,20132,SJS,1,0,1,0,0,2 +2009,20133,NJD,2,2,1,0,0,5 +2009,20133,TOR,0,3,2,0,1,6 +2009,20134,DAL,3,1,0,0,0,4 +2009,20134,MIN,1,0,1,0,0,2 +2009,20135,ANA,1,1,2,1,0,5 +2009,20135,DET,1,2,1,0,0,4 +2009,20136,BUF,1,1,0,0,0,2 +2009,20136,TBL,3,2,0,0,0,5 +2009,20137,ATL,1,0,1,0,0,2 +2009,20137,NYR,1,0,2,0,0,3 +2009,20138,NYI,1,0,1,0,0,2 +2009,20138,PHI,2,0,0,1,0,3 +2009,20139,FLA,1,0,0,0,0,1 +2009,20139,OTT,0,0,2,0,0,2 +2009,20140,EDM,0,1,0,0,0,1 +2009,20140,NSH,1,1,1,0,0,3 +2009,20141,MTL,1,1,0,0,0,2 +2009,20141,MIN,1,0,0,0,0,1 +2009,20142,CAR,0,0,1,0,0,1 +2009,20142,STL,0,0,0,0,0,0 +2009,20143,COL,0,1,1,0,0,2 +2009,20143,CBJ,1,3,0,0,0,4 +2009,20144,BOS,1,0,1,0,0,2 +2009,20144,CGY,0,2,1,0,0,3 +2009,20145,PHX,0,2,2,0,0,4 +2009,20145,PIT,0,1,0,0,0,1 +2009,20146,LAK,0,0,0,0,0,0 +2009,20146,VAN,2,1,1,0,0,4 +2009,20147,DET,0,1,1,0,0,2 +2009,20147,SJS,0,3,1,0,0,4 +2009,20148,CHI,2,1,2,0,0,5 +2009,20148,DAL,1,1,0,0,0,2 +2009,20149,ANA,2,3,1,0,0,6 +2009,20149,VAN,1,4,1,0,1,7 +2009,20150,CAR,1,0,0,0,0,1 +2009,20150,EDM,0,1,2,0,0,3 +2009,20151,BOS,1,1,3,0,0,5 +2009,20151,DAL,1,0,0,0,0,1 +2009,20152,BUF,3,1,1,0,0,5 +2009,20152,WSH,0,0,0,0,0,0 +2009,20153,NYR,1,1,0,0,0,2 +2009,20153,TOR,0,0,5,0,0,5 +2009,20154,ATL,0,0,1,0,0,1 +2009,20154,NJD,1,5,0,0,0,6 +2009,20155,MTL,1,0,4,0,0,5 +2009,20155,NYI,2,2,0,0,0,4 +2009,20156,CHI,1,1,1,0,1,4 +2009,20156,CBJ,1,0,2,0,0,3 +2009,20157,OTT,0,2,0,0,0,2 +2009,20157,TBL,0,1,1,0,1,3 +2009,20158,FLA,1,1,0,0,0,2 +2009,20158,NSH,1,0,1,0,1,3 +2009,20159,PIT,1,3,2,0,0,6 +2009,20159,STL,2,0,1,0,0,3 +2009,20160,MIN,0,2,1,0,0,3 +2009,20160,PHX,1,1,0,0,0,2 +2009,20161,CGY,1,1,1,0,0,3 +2009,20161,LAK,0,1,1,0,0,2 +2009,20162,EDM,2,3,0,0,0,5 +2009,20162,PHI,1,2,1,0,0,4 +2009,20163,CAR,0,4,2,0,0,6 +2009,20163,TOR,1,2,1,0,0,4 +2009,20164,ATL,1,3,1,0,0,5 +2009,20164,FLA,0,2,1,0,0,3 +2009,20165,COL,1,0,2,0,0,3 +2009,20165,SJS,2,1,2,0,0,5 +2009,20166,ANA,1,2,0,0,0,3 +2009,20166,CGY,0,1,1,0,0,2 +2009,20167,DET,1,2,0,0,0,3 +2009,20167,VAN,0,1,1,0,0,2 +2009,20168,BUF,1,0,1,0,0,2 +2009,20168,NJD,0,0,0,0,0,0 +2009,20169,CBJ,0,0,3,0,0,3 +2009,20169,NYI,0,3,0,1,0,4 +2009,20170,CHI,0,2,4,0,0,6 +2009,20170,COL,1,0,1,0,0,2 +2009,20171,NYI,0,0,2,0,0,2 +2009,20171,NYR,0,0,1,0,0,1 +2009,20172,CAR,4,0,0,1,0,5 +2009,20172,TOR,1,2,1,0,0,4 +2009,20173,OTT,0,1,0,1,0,2 +2009,20173,WSH,0,1,0,0,0,1 +2009,20174,CGY,2,0,0,0,0,2 +2009,20174,PHX,2,2,0,0,0,4 +2009,20175,NSH,0,0,0,0,0,0 +2009,20175,VAN,0,1,3,0,0,4 +2009,20176,ANA,0,0,0,1,0,1 +2009,20176,LAK,0,0,0,0,0,0 +2009,20177,MIN,1,0,0,0,0,1 +2009,20177,SJS,1,0,2,0,0,3 +2009,20178,NJD,0,2,1,0,1,4 +2009,20178,TBL,1,0,2,0,0,3 +2009,20179,CBJ,1,1,3,0,0,5 +2009,20179,EDM,0,2,2,0,0,4 +2009,20180,ANA,1,3,1,0,0,5 +2009,20180,STL,1,1,0,0,0,2 +2009,20181,BOS,2,1,2,0,0,5 +2009,20181,TOR,0,0,2,0,0,2 +2009,20182,NYR,1,1,3,0,0,5 +2009,20182,TBL,0,1,1,0,0,2 +2009,20183,CAR,1,1,0,0,0,2 +2009,20183,WSH,0,1,2,0,0,3 +2009,20184,ATL,1,0,3,0,0,4 +2009,20184,NYI,1,2,0,0,0,3 +2009,20185,OTT,2,1,1,0,0,4 +2009,20185,PHI,0,1,0,0,0,1 +2009,20186,EDM,0,1,3,0,0,4 +2009,20186,PIT,2,3,0,0,0,5 +2009,20187,COL,0,1,0,0,0,1 +2009,20187,MIN,2,1,0,0,0,3 +2009,20188,CGY,4,2,1,0,0,7 +2009,20188,NSH,0,3,3,0,0,6 +2009,20189,PHX,0,0,0,0,0,0 +2009,20189,VAN,1,0,0,0,0,1 +2009,20190,FLA,0,0,2,0,0,2 +2009,20190,LAK,2,0,1,0,0,3 +2009,20191,SJS,1,1,2,0,1,5 +2009,20191,STL,2,0,2,0,0,4 +2009,20192,CAR,0,0,2,0,0,2 +2009,20192,OTT,0,1,0,0,0,1 +2009,20193,MTL,1,0,2,0,0,3 +2009,20193,CBJ,1,0,2,0,1,4 +2009,20194,ATL,1,2,1,1,0,5 +2009,20194,BUF,2,2,0,0,0,4 +2009,20195,ANA,1,0,1,0,0,2 +2009,20195,DAL,2,2,1,0,0,5 +2009,20196,BOS,1,2,0,0,0,3 +2009,20196,BUF,1,0,0,0,0,1 +2009,20197,MTL,0,3,0,0,0,3 +2009,20197,TOR,1,3,2,0,0,6 +2009,20198,NYI,1,2,0,0,0,3 +2009,20198,PIT,0,2,1,0,1,4 +2009,20199,PHI,1,0,0,0,0,1 +2009,20199,TBL,0,2,0,0,0,2 +2009,20200,NYR,0,0,1,0,0,1 +2009,20200,WSH,2,0,1,0,0,3 +2009,20201,DET,1,1,1,0,0,3 +2009,20201,NJD,0,1,0,0,0,1 +2009,20202,CGY,0,1,0,0,0,1 +2009,20202,CBJ,1,1,1,0,0,3 +2009,20203,COL,0,1,0,0,0,1 +2009,20203,NSH,0,0,0,0,0,0 +2009,20204,FLA,1,0,0,0,0,1 +2009,20204,PHX,0,2,2,0,0,4 +2009,20205,MIN,0,0,0,0,0,0 +2009,20205,VAN,1,0,1,0,0,2 +2009,20206,DAL,0,0,1,0,0,1 +2009,20206,SJS,1,0,1,0,0,2 +2009,20207,LAK,1,2,2,0,0,5 +2009,20207,STL,0,0,3,0,0,3 +2009,20208,ATL,0,4,1,0,0,5 +2009,20208,CAR,0,2,0,0,0,2 +2009,20209,EDM,0,2,0,0,0,2 +2009,20209,NJD,0,0,1,0,0,1 +2009,20210,CGY,0,1,0,0,0,1 +2009,20210,CHI,2,1,3,0,0,6 +2009,20211,PHX,2,1,1,0,0,4 +2009,20211,SJS,2,0,0,0,0,2 +2009,20212,ANA,1,0,0,0,0,1 +2009,20212,FLA,1,2,0,0,0,3 +2009,20213,EDM,1,1,0,0,1,3 +2009,20213,NYR,0,1,1,0,0,2 +2009,20214,TBL,0,1,1,0,0,2 +2009,20214,WSH,3,0,1,0,0,4 +2009,20215,NYI,0,0,1,0,0,1 +2009,20215,PHI,0,1,2,0,0,3 +2009,20216,DET,2,2,2,0,0,6 +2009,20216,PIT,1,1,4,1,0,7 +2009,20217,MTL,1,2,1,0,0,4 +2009,20217,OTT,0,0,0,0,0,0 +2009,20218,CGY,2,1,1,0,0,4 +2009,20218,TOR,1,0,2,0,0,3 +2009,20219,DAL,1,1,0,0,0,2 +2009,20219,LAK,2,0,0,0,1,3 +2009,20220,NSH,0,1,2,1,0,4 +2009,20220,SJS,1,1,1,0,0,3 +2009,20221,BUF,3,0,1,0,0,4 +2009,20221,STL,1,1,1,0,0,3 +2009,20222,NJD,1,0,1,0,0,2 +2009,20222,NYR,0,4,1,0,0,5 +2009,20223,CAR,1,0,0,0,0,1 +2009,20223,WSH,3,1,1,0,0,5 +2009,20224,CBJ,0,0,2,0,0,2 +2009,20224,PHX,1,1,3,0,0,5 +2009,20225,FLA,1,2,1,0,0,4 +2009,20225,TBL,0,0,0,0,0,0 +2009,20226,BOS,0,1,0,0,1,2 +2009,20226,CHI,0,0,1,0,0,1 +2009,20227,COL,0,0,1,0,1,2 +2009,20227,VAN,0,1,0,0,0,1 +2009,20228,BOS,3,1,2,0,0,6 +2009,20228,MTL,0,1,0,0,0,1 +2009,20229,NYI,1,1,1,0,0,3 +2009,20229,OTT,1,0,0,0,0,1 +2009,20230,PHI,0,4,0,0,0,4 +2009,20230,PIT,1,2,1,0,1,5 +2009,20231,DET,0,1,3,0,0,4 +2009,20231,TBL,2,0,1,0,0,3 +2009,20232,MIN,0,2,2,0,0,4 +2009,20232,PHX,0,0,0,0,0,0 +2009,20233,DAL,1,1,0,0,0,2 +2009,20233,LAK,2,1,0,0,0,3 +2009,20234,EDM,1,1,0,0,0,2 +2009,20234,TOR,0,3,2,0,0,5 +2009,20235,CGY,0,0,1,0,0,1 +2009,20235,SJS,4,1,1,0,0,6 +2009,20236,NJD,0,0,1,0,0,1 +2009,20236,WSH,1,1,1,0,0,3 +2009,20237,BUF,0,1,0,0,0,1 +2009,20237,CBJ,3,1,2,0,0,6 +2009,20238,ATL,1,1,1,0,0,3 +2009,20238,CAR,0,1,1,0,0,2 +2009,20239,DET,1,1,1,0,0,3 +2009,20239,FLA,0,2,0,0,0,2 +2009,20240,CHI,1,1,1,0,0,3 +2009,20240,STL,3,0,0,1,0,4 +2009,20241,ANA,2,0,1,0,0,3 +2009,20241,NSH,2,0,1,1,0,4 +2009,20242,MTL,0,0,1,0,0,1 +2009,20242,PHI,0,2,0,0,0,2 +2009,20243,NJD,1,2,2,0,1,6 +2009,20243,WSH,2,1,2,0,0,5 +2009,20244,NYI,0,2,1,0,0,3 +2009,20244,OTT,0,0,2,0,0,2 +2009,20245,BOS,0,2,0,0,0,2 +2009,20245,NYR,0,0,2,0,1,3 +2009,20246,TOR,0,0,2,0,0,2 +2009,20246,VAN,2,2,0,0,0,4 +2009,20247,BUF,1,1,0,0,0,2 +2009,20247,PIT,0,1,4,0,0,5 +2009,20248,CBJ,1,0,1,0,0,2 +2009,20248,MIN,1,0,1,0,1,3 +2009,20249,DAL,1,2,0,0,0,3 +2009,20249,PHX,0,2,0,0,0,2 +2009,20250,COL,1,0,1,0,1,3 +2009,20250,EDM,1,1,0,0,0,2 +2009,20251,LAK,0,1,0,0,0,1 +2009,20251,NSH,0,1,2,0,0,3 +2009,20252,CAR,0,2,0,0,1,3 +2009,20252,TBL,1,1,0,0,0,2 +2009,20253,MTL,0,1,1,0,1,3 +2009,20253,STL,1,0,1,0,0,2 +2009,20254,ATL,0,1,2,0,0,3 +2009,20254,PHI,2,1,1,0,0,4 +2009,20255,CHI,2,3,0,0,0,5 +2009,20255,SJS,2,2,2,0,0,6 +2009,20256,ANA,0,1,1,0,0,2 +2009,20256,LAK,0,0,0,0,0,0 +2009,20257,NYI,0,1,0,0,1,2 +2009,20257,VAN,1,0,0,0,0,1 +2009,20258,NYR,0,0,1,0,1,2 +2009,20258,OTT,0,1,0,0,0,1 +2009,20259,CAR,0,0,0,0,0,0 +2009,20259,TBL,0,0,0,0,0,0 +2009,20260,DET,1,2,1,0,0,4 +2009,20260,EDM,0,0,0,0,0,0 +2009,20261,NSH,0,0,1,0,0,1 +2009,20261,SJS,2,2,0,0,0,4 +2009,20262,MIN,1,0,0,0,1,2 +2009,20262,PIT,1,0,0,0,0,1 +2009,20263,MTL,0,1,0,0,0,1 +2009,20263,CAR,0,0,2,0,0,2 +2009,20264,CBJ,0,1,1,0,0,2 +2009,20264,EDM,2,1,4,0,0,7 +2009,20265,FLA,2,1,0,0,1,4 +2009,20265,TBL,0,1,2,0,0,3 +2009,20266,CHI,2,0,0,0,1,3 +2009,20266,PHX,0,1,1,0,0,2 +2009,20267,CGY,0,1,3,0,0,4 +2009,20267,COL,1,0,0,0,0,1 +2009,20268,BOS,3,3,1,0,0,7 +2009,20268,BUF,4,0,0,0,0,4 +2009,20269,NYR,0,2,1,0,0,3 +2009,20269,VAN,2,3,1,0,0,6 +2009,20270,ANA,2,0,2,0,0,4 +2009,20270,WSH,3,2,1,0,0,6 +2009,20271,FLA,0,0,1,0,0,1 +2009,20271,NJD,0,1,2,0,0,3 +2009,20272,ATL,1,1,0,0,0,2 +2009,20272,PIT,1,1,1,0,0,3 +2009,20273,MTL,1,0,1,0,1,3 +2009,20273,OTT,1,0,1,0,0,2 +2009,20274,MIN,0,2,0,0,0,2 +2009,20274,VAN,0,2,1,0,0,3 +2009,20275,CHI,1,3,2,0,0,6 +2009,20275,DAL,1,2,0,0,0,3 +2009,20276,CGY,0,1,0,0,0,1 +2009,20276,COL,0,0,0,0,0,0 +2009,20277,DET,3,1,0,0,0,4 +2009,20277,EDM,0,1,2,0,0,3 +2009,20278,LAK,1,1,3,0,0,5 +2009,20278,WSH,0,1,1,0,0,2 +2009,20279,BOS,2,2,0,0,0,4 +2009,20279,FLA,1,0,1,0,0,2 +2009,20280,NJD,2,2,1,0,0,5 +2009,20280,NYI,0,2,0,0,0,2 +2009,20281,CAR,1,2,2,0,0,5 +2009,20281,PHX,2,0,0,0,0,2 +2009,20282,BUF,0,0,0,0,0,0 +2009,20282,PHI,1,0,2,0,0,3 +2009,20283,NSH,1,0,0,0,0,1 +2009,20283,TBL,3,1,0,0,0,4 +2009,20284,ANA,0,0,2,0,0,2 +2009,20284,STL,1,0,1,1,0,3 +2009,20285,PIT,0,0,1,0,0,1 +2009,20285,VAN,1,1,1,0,0,3 +2009,20286,NYR,0,0,1,0,0,1 +2009,20286,OTT,1,2,1,0,0,4 +2009,20287,BUF,1,0,1,0,0,2 +2009,20287,NYI,1,1,2,0,0,4 +2009,20288,CHI,0,2,2,1,0,5 +2009,20288,TOR,1,3,0,0,0,4 +2009,20289,BOS,0,1,1,0,1,3 +2009,20289,MTL,0,1,1,0,0,2 +2009,20290,PHI,1,0,2,1,0,4 +2009,20290,PHX,1,1,1,0,0,3 +2009,20291,ATL,0,0,0,0,0,0 +2009,20291,CBJ,1,0,1,0,0,2 +2009,20292,MIN,0,1,0,0,0,1 +2009,20292,STL,0,2,0,0,0,2 +2009,20293,ANA,1,0,0,0,1,2 +2009,20293,DAL,0,1,0,0,0,1 +2009,20294,CGY,0,1,1,0,0,2 +2009,20294,DET,1,1,3,0,0,5 +2009,20295,COL,1,1,1,0,1,4 +2009,20295,LAK,0,1,2,0,0,3 +2009,20296,SJS,3,2,2,0,0,7 +2009,20296,WSH,0,1,1,0,0,2 +2009,20297,CAR,0,0,2,0,0,2 +2009,20297,NSH,1,3,1,0,0,5 +2009,20298,NJD,2,3,2,0,0,7 +2009,20298,TBL,0,3,0,0,0,3 +2009,20299,NYR,1,1,2,0,0,4 +2009,20299,PHX,1,0,0,0,0,1 +2009,20300,DAL,1,2,0,0,0,3 +2009,20300,PHI,2,1,1,0,0,4 +2009,20301,MTL,2,1,0,0,0,3 +2009,20301,NYI,1,1,1,0,1,4 +2009,20302,CAR,0,2,0,0,0,2 +2009,20302,FLA,2,1,0,0,0,3 +2009,20303,MIN,1,1,2,0,0,4 +2009,20303,WSH,0,0,3,0,0,3 +2009,20304,DET,1,0,1,0,0,2 +2009,20304,VAN,0,1,1,1,0,3 +2009,20305,ANA,2,1,1,0,0,4 +2009,20305,COL,0,0,1,0,0,1 +2009,20306,NSH,0,0,0,0,0,0 +2009,20306,STL,0,0,0,0,1,1 +2009,20307,ATL,1,3,2,0,0,6 +2009,20307,TOR,0,1,2,0,0,3 +2009,20308,CGY,2,2,2,0,0,6 +2009,20308,LAK,0,0,2,0,0,2 +2009,20309,BOS,0,1,1,0,0,2 +2009,20309,BUF,2,1,0,0,0,3 +2009,20310,NYI,2,1,0,0,0,3 +2009,20310,PIT,0,1,4,0,0,5 +2009,20311,ATL,1,1,1,0,0,3 +2009,20311,WSH,3,1,1,0,0,5 +2009,20312,CAR,0,1,0,0,0,1 +2009,20312,PHI,1,0,2,0,0,3 +2009,20313,CBJ,1,0,1,0,0,2 +2009,20313,PHX,0,2,1,0,0,3 +2009,20314,NYR,0,2,0,0,1,3 +2009,20314,TBL,0,1,1,0,0,2 +2009,20315,FLA,0,0,2,0,0,2 +2009,20315,NJD,2,0,0,1,0,3 +2009,20316,MTL,0,3,0,0,0,3 +2009,20316,DET,0,0,1,0,0,1 +2009,20317,DAL,1,1,2,0,0,4 +2009,20317,MIN,2,1,0,0,0,3 +2009,20318,COL,1,1,1,0,0,3 +2009,20318,STL,0,1,0,0,0,1 +2009,20319,EDM,1,0,0,0,0,1 +2009,20319,LAK,1,1,0,0,0,2 +2009,20320,CHI,0,1,1,0,0,2 +2009,20320,SJS,1,0,1,1,0,3 +2009,20321,OTT,1,0,0,0,1,2 +2009,20321,TOR,1,0,0,0,0,1 +2009,20322,CGY,1,1,2,0,0,4 +2009,20322,VAN,2,0,1,0,0,3 +2009,20323,BOS,0,2,5,0,0,7 +2009,20323,NYI,1,0,1,0,0,2 +2009,20324,CAR,0,2,0,1,0,3 +2009,20324,PHI,1,1,0,0,0,2 +2009,20325,MIN,1,0,3,0,0,4 +2009,20325,TBL,1,1,0,0,0,2 +2009,20326,COL,0,0,1,0,0,1 +2009,20326,PHX,1,1,0,0,0,2 +2009,20327,ANA,0,1,0,0,0,1 +2009,20327,CHI,0,0,0,0,0,0 +2009,20328,MTL,0,0,0,0,0,0 +2009,20328,WSH,1,0,2,0,0,3 +2009,20329,BUF,1,1,2,0,0,4 +2009,20329,PIT,1,2,0,0,0,3 +2009,20330,ATL,1,1,1,0,0,3 +2009,20330,NSH,1,1,1,1,0,4 +2009,20331,FLA,1,1,1,0,0,3 +2009,20331,NYR,2,0,1,0,1,4 +2009,20332,CBJ,1,1,1,0,0,3 +2009,20332,DET,1,2,2,0,0,5 +2009,20333,DAL,1,0,1,0,0,2 +2009,20333,SJS,3,2,1,0,0,6 +2009,20334,EDM,0,2,2,0,0,4 +2009,20334,STL,2,0,0,0,0,2 +2009,20335,CHI,2,0,0,0,0,2 +2009,20335,LAK,1,1,3,0,0,5 +2009,20336,BOS,2,1,1,0,0,4 +2009,20336,DET,0,1,0,0,0,1 +2009,20337,PHI,1,1,0,0,0,2 +2009,20337,TOR,1,2,1,0,0,4 +2009,20338,BUF,1,1,0,0,0,2 +2009,20338,MTL,0,3,0,0,0,3 +2009,20339,NYI,1,1,2,0,0,4 +2009,20339,OTT,1,0,1,0,0,2 +2009,20340,CBJ,1,1,1,0,0,3 +2009,20340,WSH,0,0,0,0,0,0 +2009,20341,NJD,0,0,1,0,0,1 +2009,20341,PIT,1,2,1,0,0,4 +2009,20342,MIN,2,2,2,0,0,6 +2009,20342,NSH,0,1,1,0,0,2 +2009,20343,COL,3,0,1,0,0,4 +2009,20343,TBL,0,1,2,0,0,3 +2009,20344,PHX,2,0,0,0,0,2 +2009,20344,SJS,2,0,1,0,0,3 +2009,20345,CGY,0,1,2,0,0,3 +2009,20345,VAN,0,0,1,0,0,1 +2009,20346,FLA,0,3,1,0,0,4 +2009,20346,NYR,0,0,0,0,0,0 +2009,20347,ATL,2,0,0,0,0,2 +2009,20347,STL,1,1,2,0,0,4 +2009,20348,DAL,2,0,2,0,0,4 +2009,20348,EDM,0,1,2,0,0,3 +2009,20349,ANA,1,0,3,0,0,4 +2009,20349,CAR,1,0,0,0,0,1 +2009,20350,BUF,0,0,0,0,0,0 +2009,20350,NSH,0,0,2,0,0,2 +2009,20351,CBJ,0,1,2,0,0,3 +2009,20351,VAN,0,1,1,0,0,2 +2009,20352,ANA,1,0,0,0,0,1 +2009,20352,DET,0,1,1,0,0,2 +2009,20353,COL,2,4,0,0,0,6 +2009,20353,MIN,2,2,1,0,0,5 +2009,20354,LAK,1,0,0,0,0,1 +2009,20354,TOR,0,0,3,0,0,3 +2009,20355,PHI,1,0,2,1,0,4 +2009,20355,TBL,1,0,2,0,0,3 +2009,20356,FLA,0,3,2,0,0,5 +2009,20356,WSH,1,0,2,0,0,3 +2009,20357,ATL,0,0,4,0,0,4 +2009,20357,MTL,1,2,2,0,0,5 +2009,20358,CGY,0,1,0,0,0,1 +2009,20358,DAL,1,1,1,0,0,3 +2009,20359,LAK,1,1,0,0,0,2 +2009,20359,PHX,1,0,3,0,0,4 +2009,20360,SJS,4,0,1,0,0,5 +2009,20360,TOR,0,1,1,0,0,2 +2009,20361,ATL,1,0,0,0,0,1 +2009,20361,OTT,3,1,1,0,0,5 +2009,20362,NYR,0,1,1,0,1,3 +2009,20362,PIT,1,1,0,0,0,2 +2009,20363,MIN,0,1,3,0,0,4 +2009,20363,STL,0,0,0,0,0,0 +2009,20364,ANA,0,1,1,0,0,2 +2009,20364,CHI,1,0,3,0,0,4 +2009,20365,DAL,0,0,2,0,0,2 +2009,20365,EDM,2,1,2,0,0,5 +2009,20366,MTL,3,1,2,0,0,6 +2009,20366,NYR,0,2,0,0,0,2 +2009,20367,NJD,0,1,1,1,0,3 +2009,20367,PHI,0,1,1,0,0,2 +2009,20368,NYI,1,1,0,0,0,2 +2009,20368,WSH,2,0,3,0,0,5 +2009,20369,CAR,0,2,0,0,0,2 +2009,20369,PIT,2,3,0,0,0,5 +2009,20370,BOS,0,1,2,0,0,3 +2009,20370,TBL,1,0,0,0,0,1 +2009,20371,BUF,0,0,1,0,0,1 +2009,20371,FLA,0,2,0,0,0,2 +2009,20372,DET,2,1,3,0,0,6 +2009,20372,VAN,1,2,2,0,0,5 +2009,20373,COL,2,0,0,0,0,2 +2009,20373,NSH,3,0,0,0,0,3 +2009,20374,PHX,2,1,3,0,0,6 +2009,20374,TOR,2,1,0,0,0,3 +2009,20375,CBJ,1,0,1,0,0,2 +2009,20375,SJS,0,1,2,0,0,3 +2009,20376,MIN,0,1,0,0,0,1 +2009,20376,VAN,1,1,0,0,0,2 +2009,20377,CGY,0,1,2,1,0,4 +2009,20377,STL,2,1,0,0,0,3 +2009,20378,COL,0,0,1,0,0,1 +2009,20378,DAL,0,1,0,0,1,2 +2009,20379,EDM,3,0,1,0,1,5 +2009,20379,LAK,0,4,0,0,0,4 +2009,20380,TOR,0,0,1,0,0,1 +2009,20380,WSH,0,1,1,0,0,2 +2009,20381,MTL,1,0,0,0,0,1 +2009,20381,NJD,0,1,0,1,0,2 +2009,20382,OTT,0,3,0,0,0,3 +2009,20382,PIT,0,1,1,0,0,2 +2009,20383,ATL,1,2,2,0,0,5 +2009,20383,NYI,1,0,0,0,0,1 +2009,20384,CAR,0,0,1,0,0,1 +2009,20384,PHI,1,0,0,1,0,2 +2009,20385,BOS,1,1,2,0,0,4 +2009,20385,FLA,0,0,0,0,0,0 +2009,20386,CHI,3,1,0,0,0,4 +2009,20386,DET,2,0,2,0,1,5 +2009,20387,BUF,1,2,1,0,0,4 +2009,20387,TBL,1,2,0,0,0,3 +2009,20388,MIN,0,0,0,0,0,0 +2009,20388,NSH,0,0,1,0,0,1 +2009,20389,PHX,1,2,0,0,0,3 +2009,20389,STL,1,3,0,0,0,4 +2009,20390,EDM,1,1,0,1,0,3 +2009,20390,SJS,1,1,0,0,0,2 +2009,20391,CBJ,0,0,0,0,0,0 +2009,20391,LAK,0,2,1,0,0,3 +2009,20392,CAR,1,0,2,0,0,3 +2009,20392,WSH,0,0,1,0,0,1 +2009,20393,CGY,0,1,2,0,0,3 +2009,20393,NYR,0,0,0,0,0,0 +2009,20394,CHI,4,3,0,0,0,7 +2009,20394,PHX,0,1,0,0,0,1 +2009,20395,COL,0,2,2,0,1,5 +2009,20395,VAN,1,2,1,0,0,4 +2009,20396,ANA,0,3,2,0,0,5 +2009,20396,CBJ,0,1,2,0,0,3 +2009,20397,BOS,3,1,1,0,0,5 +2009,20397,TBL,0,1,2,0,0,3 +2009,20398,NYI,1,1,0,0,0,2 +2009,20398,TOR,1,2,1,0,0,4 +2009,20399,FLA,1,2,0,1,0,4 +2009,20399,OTT,1,2,0,0,0,3 +2009,20400,BUF,1,2,1,0,0,4 +2009,20400,PIT,2,1,0,0,0,3 +2009,20401,NSH,1,2,0,0,0,3 +2009,20401,STL,1,2,3,0,0,6 +2009,20402,NYI,2,0,1,0,0,3 +2009,20402,PHI,1,2,1,0,0,4 +2009,20403,CGY,0,1,0,0,0,1 +2009,20403,MTL,1,2,1,0,0,4 +2009,20404,NSH,1,0,0,0,0,1 +2009,20404,VAN,1,0,2,0,0,3 +2009,20405,BUF,1,2,1,0,0,4 +2009,20405,TBL,2,0,0,0,0,2 +2009,20406,NJD,0,1,3,0,0,4 +2009,20406,PIT,0,0,1,0,0,1 +2009,20407,BOS,0,1,0,0,0,1 +2009,20407,WSH,1,1,1,0,0,3 +2009,20408,ATL,0,1,1,0,0,2 +2009,20408,NYR,0,1,1,1,0,3 +2009,20409,CGY,0,0,0,0,0,0 +2009,20409,DET,0,0,0,0,0,0 +2009,20410,CHI,1,0,1,0,0,2 +2009,20410,OTT,0,0,0,0,0,0 +2009,20411,DAL,2,1,0,0,0,3 +2009,20411,PHX,2,1,2,0,0,5 +2009,20412,COL,3,3,0,0,0,6 +2009,20412,LAK,0,1,0,0,0,1 +2009,20413,ANA,1,1,2,0,0,4 +2009,20413,STL,1,0,1,0,0,2 +2009,20414,CAR,1,4,0,0,0,5 +2009,20414,PHI,0,1,4,0,1,6 +2009,20415,CBJ,0,0,1,0,1,2 +2009,20415,NSH,0,0,1,0,0,1 +2009,20416,MTL,1,0,0,0,0,1 +2009,20416,TBL,2,1,0,0,0,3 +2009,20417,NYI,1,0,1,0,0,2 +2009,20417,PIT,4,4,1,0,0,9 +2009,20418,EDM,0,0,0,0,0,0 +2009,20418,FLA,0,1,1,0,0,2 +2009,20419,MIN,0,0,1,0,0,1 +2009,20419,PHX,0,2,1,0,0,3 +2009,20420,LAK,1,3,2,0,0,6 +2009,20420,STL,0,1,1,0,0,2 +2009,20421,ANA,0,0,0,0,0,0 +2009,20421,SJS,1,1,0,0,0,2 +2009,20422,NJD,2,3,3,0,0,8 +2009,20422,NYR,1,2,2,0,0,5 +2009,20423,OTT,0,0,1,0,0,1 +2009,20423,WSH,1,2,2,0,0,5 +2009,20424,BUF,1,0,0,0,0,1 +2009,20424,TOR,0,1,1,0,0,2 +2009,20425,ATL,1,0,2,0,0,3 +2009,20425,BOS,4,0,3,0,0,7 +2009,20426,DAL,0,2,1,0,0,3 +2009,20426,DET,1,0,0,0,0,1 +2009,20427,CHI,1,1,2,0,0,4 +2009,20427,COL,2,0,1,0,0,3 +2009,20428,CGY,1,1,0,0,0,2 +2009,20428,FLA,1,1,0,0,1,3 +2009,20429,PHI,2,3,1,0,0,6 +2009,20429,PIT,0,1,2,0,0,3 +2009,20430,LAK,1,0,2,0,0,3 +2009,20430,MIN,0,0,1,0,0,1 +2009,20431,ATL,1,0,1,0,0,2 +2009,20431,BOS,2,1,1,0,0,4 +2009,20432,MTL,0,1,0,0,0,1 +2009,20432,WSH,1,0,1,0,0,2 +2009,20433,OTT,0,1,1,0,0,2 +2009,20433,TBL,0,0,0,0,0,0 +2009,20434,BUF,1,1,2,0,0,4 +2009,20434,NJD,0,2,0,0,0,2 +2009,20435,CAR,1,1,0,0,0,2 +2009,20435,NYR,2,0,0,0,1,3 +2009,20436,CBJ,0,2,1,0,0,3 +2009,20436,NYI,0,1,0,0,0,1 +2009,20437,DAL,0,0,0,0,0,0 +2009,20437,NSH,1,0,2,0,0,3 +2009,20438,DET,1,2,1,0,1,5 +2009,20438,PHX,1,1,2,0,0,4 +2009,20439,EDM,1,1,1,0,0,3 +2009,20439,VAN,0,0,0,0,0,0 +2009,20440,SJS,1,1,3,0,0,5 +2009,20440,STL,1,2,1,0,0,4 +2009,20441,CHI,0,1,2,0,0,3 +2009,20441,CBJ,1,0,0,0,0,1 +2009,20442,ANA,2,2,0,0,0,4 +2009,20442,MIN,1,1,0,0,0,2 +2009,20443,FLA,2,0,1,0,0,3 +2009,20443,VAN,1,4,0,0,0,5 +2009,20444,COL,2,0,1,0,0,3 +2009,20444,DET,0,1,1,0,0,2 +2009,20445,LAK,2,0,0,0,0,2 +2009,20445,SJS,1,1,0,0,1,3 +2009,20446,NYI,1,1,2,0,0,4 +2009,20446,WSH,2,2,0,1,0,5 +2009,20447,COL,1,1,0,0,0,2 +2009,20447,PHI,2,2,1,0,0,5 +2009,20448,MTL,0,2,0,0,0,2 +2009,20448,CAR,2,1,0,0,0,3 +2009,20449,NJD,1,1,0,0,0,2 +2009,20449,TOR,0,2,0,0,1,3 +2009,20450,ATL,1,0,3,0,0,4 +2009,20450,OTT,0,0,1,0,0,1 +2009,20451,CGY,2,3,1,0,0,6 +2009,20451,STL,0,2,1,0,0,3 +2009,20452,DAL,1,0,0,1,0,2 +2009,20452,PHX,0,0,1,0,0,1 +2009,20453,CHI,2,5,2,0,0,9 +2009,20453,EDM,0,1,1,0,0,2 +2009,20454,ANA,0,1,0,0,0,1 +2009,20454,NYR,0,1,2,0,0,3 +2009,20455,BUF,1,1,1,0,0,3 +2009,20455,NJD,2,1,2,0,0,5 +2009,20456,CBJ,0,0,1,1,0,2 +2009,20456,SJS,0,0,1,0,0,1 +2009,20457,CGY,0,2,0,1,0,3 +2009,20457,MIN,0,1,1,0,0,2 +2009,20458,EDM,0,0,2,0,0,2 +2009,20458,VAN,0,3,1,0,0,4 +2009,20459,LAK,1,0,1,0,0,2 +2009,20459,NYR,1,1,0,1,0,3 +2009,20460,BOS,2,4,2,0,0,8 +2009,20460,TOR,1,4,0,0,0,5 +2009,20461,STL,0,1,1,0,0,2 +2009,20461,WSH,1,2,1,0,0,4 +2009,20462,CAR,1,0,0,1,0,2 +2009,20462,FLA,0,0,1,0,0,1 +2009,20463,ATL,1,1,1,0,0,3 +2009,20463,PIT,2,3,1,0,0,6 +2009,20464,MTL,3,1,1,0,0,5 +2009,20464,PHI,1,0,1,0,0,2 +2009,20465,COL,1,0,0,0,1,2 +2009,20465,TBL,0,1,0,0,0,1 +2009,20466,DET,2,1,3,0,0,6 +2009,20466,SJS,0,0,0,0,0,0 +2009,20467,CBJ,2,2,1,0,0,5 +2009,20467,DAL,2,3,0,0,1,6 +2009,20468,NSH,0,1,0,0,0,1 +2009,20468,PHX,1,0,1,0,0,2 +2009,20469,NJD,2,1,2,0,0,5 +2009,20469,OTT,0,1,0,0,0,1 +2009,20470,BUF,1,2,2,0,0,5 +2009,20470,LAK,0,0,0,0,0,0 +2009,20471,MIN,0,1,3,0,0,4 +2009,20471,NYI,0,1,0,0,0,1 +2009,20472,CGY,1,1,0,0,0,2 +2009,20472,CHI,0,2,0,1,0,3 +2009,20473,ANA,1,1,0,0,1,3 +2009,20473,EDM,0,0,2,0,0,2 +2009,20474,BOS,0,1,3,0,0,4 +2009,20474,CAR,1,0,1,0,0,2 +2009,20475,PHI,1,2,4,0,0,7 +2009,20475,WSH,0,0,1,0,0,1 +2009,20476,BUF,1,1,1,0,0,3 +2009,20476,MTL,0,1,2,1,0,4 +2009,20477,DAL,1,2,1,0,0,4 +2009,20477,OTT,1,2,1,1,0,5 +2009,20478,PIT,1,0,2,0,0,3 +2009,20478,TOR,3,2,2,0,0,7 +2009,20479,ATL,3,0,1,0,0,4 +2009,20479,TBL,2,0,1,0,0,3 +2009,20480,DET,2,1,3,0,0,6 +2009,20480,LAK,2,1,1,0,0,4 +2009,20481,NSH,1,0,0,0,0,1 +2009,20481,NYI,0,0,0,0,0,0 +2009,20482,MIN,1,0,1,0,0,2 +2009,20482,STL,2,0,2,0,0,4 +2009,20483,CBJ,0,0,0,0,0,0 +2009,20483,PHX,0,1,1,0,0,2 +2009,20484,CHI,0,2,1,0,0,3 +2009,20484,VAN,1,0,0,0,0,1 +2009,20485,NYR,1,1,0,0,0,2 +2009,20485,SJS,2,1,0,0,0,3 +2009,20486,COL,0,0,0,0,0,0 +2009,20486,FLA,0,2,1,0,0,3 +2009,20487,BOS,3,2,1,0,0,6 +2009,20487,STL,2,1,0,0,0,3 +2009,20488,MTL,0,1,1,0,0,2 +2009,20488,CAR,1,1,0,1,0,3 +2009,20489,NJD,0,2,0,0,1,3 +2009,20489,PHI,0,1,1,0,0,2 +2009,20490,BUF,1,2,0,0,0,3 +2009,20490,PIT,0,2,1,1,0,4 +2009,20491,ATL,0,1,1,0,0,2 +2009,20491,TOR,2,1,3,0,0,6 +2009,20492,EDM,2,0,2,0,0,4 +2009,20492,PHX,0,2,0,0,0,2 +2009,20493,ANA,1,0,2,0,0,3 +2009,20493,VAN,2,1,1,0,0,4 +2009,20494,BOS,0,0,2,0,0,2 +2009,20494,NJD,0,0,0,0,0,0 +2009,20495,ATL,1,1,2,0,0,4 +2009,20495,NYI,0,1,1,0,0,2 +2009,20496,NYR,3,1,0,0,0,4 +2009,20496,WSH,0,1,3,1,0,5 +2009,20497,OTT,1,1,2,0,0,4 +2009,20497,PHI,2,2,2,0,0,6 +2009,20498,CBJ,0,0,0,0,0,0 +2009,20498,LAK,2,0,1,0,0,3 +2009,20499,DAL,4,3,1,0,0,8 +2009,20499,TOR,0,1,1,0,0,2 +2009,20500,PIT,0,0,0,0,0,0 +2009,20500,TBL,0,1,1,0,0,2 +2009,20501,FLA,0,1,2,0,0,3 +2009,20501,NSH,0,0,0,0,0,0 +2009,20502,DET,2,2,0,0,0,4 +2009,20502,STL,0,0,1,0,0,1 +2009,20503,CAR,1,1,0,0,0,2 +2009,20503,MIN,0,3,0,0,0,3 +2009,20504,COL,1,1,2,1,0,5 +2009,20504,PHX,1,0,3,0,0,4 +2009,20505,ANA,0,1,2,0,0,3 +2009,20505,CGY,0,1,3,0,0,4 +2009,20506,SJS,3,2,0,0,0,5 +2009,20506,VAN,0,0,0,0,0,0 +2009,20507,NJD,0,0,0,0,0,0 +2009,20507,PIT,0,1,0,0,0,1 +2009,20508,NYI,1,1,2,0,0,4 +2009,20508,TOR,1,0,0,0,0,1 +2009,20509,BUF,0,0,2,0,0,2 +2009,20509,WSH,0,1,2,0,0,3 +2009,20510,ATL,1,2,1,0,0,4 +2009,20510,CAR,1,1,3,0,0,5 +2009,20511,FLA,1,0,2,0,0,3 +2009,20511,TBL,1,2,0,0,1,4 +2009,20512,DET,1,0,1,0,0,2 +2009,20512,NSH,1,2,0,0,0,3 +2009,20513,CHI,2,0,3,0,0,5 +2009,20513,PHI,1,0,0,0,0,1 +2009,20514,EDM,0,3,0,0,0,3 +2009,20514,VAN,0,1,1,0,0,2 +2009,20515,LAK,0,0,1,0,0,1 +2009,20515,PHX,1,1,0,0,0,2 +2009,20516,BUF,1,1,1,0,1,4 +2009,20516,NYI,0,1,2,0,0,3 +2009,20517,MTL,2,0,1,0,0,3 +2009,20517,PIT,1,1,0,0,0,2 +2009,20518,BOS,0,2,2,0,0,4 +2009,20518,CAR,1,1,0,0,0,2 +2009,20519,CBJ,2,1,0,0,0,3 +2009,20519,PHI,0,0,0,0,0,0 +2009,20520,NJD,2,1,1,0,0,4 +2009,20520,NYR,1,0,1,0,0,2 +2009,20521,FLA,0,3,1,0,0,4 +2009,20521,TBL,2,2,2,0,0,6 +2009,20522,ANA,1,0,2,0,0,3 +2009,20522,DAL,0,1,2,1,0,4 +2009,20523,SJS,0,2,0,0,0,2 +2009,20523,STL,1,0,1,0,1,3 +2009,20524,COL,3,0,0,0,1,4 +2009,20524,DET,0,3,0,0,0,3 +2009,20525,LAK,3,1,0,0,0,4 +2009,20525,PHX,0,0,0,0,0,0 +2009,20526,CGY,0,3,3,0,0,6 +2009,20526,OTT,0,2,1,0,0,3 +2009,20527,ATL,1,0,0,0,0,1 +2009,20527,BOS,0,1,1,0,0,2 +2009,20528,ANA,1,1,2,0,0,4 +2009,20528,STL,1,1,1,0,0,3 +2009,20529,CHI,2,1,1,0,0,4 +2009,20529,MIN,0,0,1,0,0,1 +2009,20530,TOR,1,0,0,0,0,1 +2009,20530,WSH,0,2,2,0,0,4 +2009,20531,EDM,2,3,0,0,0,5 +2009,20531,NSH,2,0,0,0,0,2 +2009,20532,OTT,0,0,0,0,0,0 +2009,20532,VAN,1,2,0,0,0,3 +2009,20533,NYI,1,1,2,0,0,4 +2009,20533,NYR,0,1,4,0,0,5 +2009,20534,MTL,1,2,2,0,0,5 +2009,20534,FLA,1,1,0,0,0,2 +2009,20535,DAL,0,0,1,0,0,1 +2009,20535,SJS,1,0,2,0,0,3 +2009,20536,COL,2,2,1,0,0,5 +2009,20536,NSH,0,1,0,0,0,1 +2009,20537,CGY,1,0,1,0,0,2 +2009,20537,MIN,0,1,0,0,0,1 +2009,20538,CBJ,0,1,1,0,0,2 +2009,20538,LAK,0,0,0,0,0,0 +2009,20539,BUF,0,0,2,0,0,2 +2009,20539,WSH,2,1,1,0,0,4 +2009,20540,ATL,2,0,1,0,0,3 +2009,20540,TOR,1,2,0,1,0,4 +2009,20541,BOS,0,3,2,0,0,5 +2009,20541,PIT,1,1,0,0,0,2 +2009,20542,MTL,0,0,1,0,1,2 +2009,20542,TBL,1,0,0,0,0,1 +2009,20543,CHI,0,0,0,0,0,0 +2009,20543,DET,1,1,2,0,0,4 +2009,20544,NJD,1,1,2,0,0,4 +2009,20544,STL,2,0,1,0,0,3 +2009,20545,EDM,1,1,0,0,0,2 +2009,20545,OTT,1,1,1,0,0,3 +2009,20546,PHI,2,1,0,0,0,3 +2009,20546,VAN,1,1,0,0,0,2 +2009,20547,FLA,1,0,1,0,0,2 +2009,20547,NYI,0,3,1,0,0,4 +2009,20548,MIN,1,0,1,1,0,3 +2009,20548,SJS,0,1,1,0,0,2 +2009,20549,ATL,1,0,0,0,0,1 +2009,20549,CAR,1,1,1,0,0,3 +2009,20550,COL,1,0,0,0,0,1 +2009,20550,PHX,0,2,1,0,0,3 +2009,20551,DAL,1,2,1,0,0,4 +2009,20551,NJD,2,0,0,0,0,2 +2009,20552,ANA,0,0,0,0,0,0 +2009,20552,CBJ,0,0,2,0,0,2 +2009,20553,CGY,2,1,3,0,0,6 +2009,20553,EDM,2,1,1,0,0,4 +2009,20554,CHI,3,0,1,0,0,4 +2009,20554,DET,1,3,2,0,0,6 +2009,20555,BOS,2,1,1,0,0,4 +2009,20555,PIT,1,0,1,0,0,2 +2009,20556,BUF,0,3,1,0,0,4 +2009,20556,TOR,0,0,1,0,0,1 +2009,20557,TBL,0,3,1,0,0,4 +2009,20557,WSH,3,1,3,0,0,7 +2009,20558,NSH,0,1,0,0,0,1 +2009,20558,VAN,1,0,1,0,0,2 +2009,20559,MTL,1,0,0,0,0,1 +2009,20559,NJD,2,1,1,0,0,4 +2009,20560,CAR,1,0,1,0,0,2 +2009,20560,STL,0,0,1,0,0,1 +2009,20561,ATL,1,1,1,0,1,4 +2009,20561,VAN,1,1,1,0,0,3 +2009,20562,COL,1,0,0,0,0,1 +2009,20562,CBJ,0,3,3,0,0,6 +2009,20563,NYI,1,0,3,0,0,4 +2009,20563,PHX,2,1,2,0,0,5 +2009,20564,ANA,1,3,0,0,0,4 +2009,20564,PHI,1,3,0,0,1,5 +2009,20565,BOS,1,1,0,0,0,2 +2009,20565,BUF,2,2,0,0,0,4 +2009,20566,FLA,2,3,1,0,0,6 +2009,20566,PIT,0,1,0,0,0,1 +2009,20567,CGY,1,0,2,0,0,3 +2009,20567,NSH,1,1,0,0,0,2 +2009,20568,OTT,1,0,0,0,0,1 +2009,20568,TOR,0,1,2,0,0,3 +2009,20569,NYR,0,1,0,0,0,1 +2009,20569,WSH,0,2,0,0,0,2 +2009,20570,CAR,2,1,0,0,0,3 +2009,20570,TBL,2,0,0,0,0,2 +2009,20571,DET,1,0,1,0,1,3 +2009,20571,MIN,0,0,2,0,0,2 +2009,20572,CBJ,2,0,0,0,0,2 +2009,20572,STL,2,1,2,0,0,5 +2009,20573,DAL,0,1,0,0,0,1 +2009,20573,EDM,2,1,1,0,0,4 +2009,20574,LAK,0,1,0,0,1,2 +2009,20574,PHI,0,1,0,0,0,1 +2009,20575,NYI,0,1,2,0,0,3 +2009,20575,SJS,2,1,2,0,0,5 +2009,20576,MTL,0,4,1,0,1,6 +2009,20576,FLA,2,0,3,0,0,5 +2009,20577,ATL,0,0,1,0,0,1 +2009,20577,TBL,1,1,2,0,0,4 +2009,20578,NJD,1,1,1,1,0,4 +2009,20578,OTT,1,1,1,0,0,3 +2009,20579,CGY,0,1,1,0,0,2 +2009,20579,CHI,1,3,1,0,0,5 +2009,20580,COL,0,0,0,0,0,0 +2009,20580,MIN,2,0,0,0,0,2 +2009,20581,ANA,0,1,1,0,0,2 +2009,20581,PHX,0,0,0,0,0,0 +2009,20582,DAL,1,1,0,0,1,3 +2009,20582,VAN,2,0,0,0,0,2 +2009,20583,NYR,1,1,2,0,0,4 +2009,20583,PIT,0,0,0,0,0,0 +2009,20584,EDM,0,2,1,0,0,3 +2009,20584,NYI,2,0,0,0,0,2 +2009,20585,BOS,0,0,0,0,0,0 +2009,20585,MIN,0,1,0,0,0,1 +2009,20586,BUF,2,1,1,0,0,4 +2009,20586,OTT,0,2,0,0,0,2 +2009,20587,PHI,0,0,1,0,0,1 +2009,20587,WSH,1,0,0,0,1,2 +2009,20588,CAR,1,1,1,0,0,3 +2009,20588,NJD,0,0,2,0,0,2 +2009,20589,FLA,2,1,1,0,0,4 +2009,20589,TOR,0,0,2,0,0,2 +2009,20590,ATL,0,0,1,0,0,1 +2009,20590,PIT,1,2,0,0,0,3 +2009,20591,CBJ,0,0,0,0,0,0 +2009,20591,DET,0,0,3,0,0,3 +2009,20592,COL,1,1,0,0,0,2 +2009,20592,NSH,1,0,0,0,0,1 +2009,20593,CGY,3,1,1,0,0,5 +2009,20593,SJS,0,2,0,0,0,2 +2009,20594,CHI,4,1,1,0,0,6 +2009,20594,PHX,0,0,0,0,0,0 +2009,20595,ANA,1,0,2,0,0,3 +2009,20595,LAK,0,1,0,0,0,1 +2009,20596,MTL,2,0,4,0,0,6 +2009,20596,NYR,1,1,1,0,0,3 +2009,20597,EDM,0,1,1,0,0,2 +2009,20597,VAN,0,3,1,0,0,4 +2009,20598,BOS,2,1,3,0,0,6 +2009,20598,OTT,1,2,1,0,0,4 +2009,20599,ATL,0,3,1,0,0,4 +2009,20599,NJD,0,0,0,0,0,0 +2009,20600,MIN,1,0,0,0,0,1 +2009,20600,PHI,0,1,2,0,0,3 +2009,20601,MTL,3,2,1,0,0,6 +2009,20601,TOR,1,0,1,0,0,2 +2009,20602,CAR,1,0,1,0,0,2 +2009,20602,FLA,1,0,3,0,0,4 +2009,20603,DAL,0,0,1,0,0,1 +2009,20603,DET,2,2,2,0,0,6 +2009,20604,NSH,0,3,2,0,0,5 +2009,20604,PIT,1,2,0,0,0,3 +2009,20605,CHI,0,1,0,0,0,1 +2009,20605,COL,1,0,1,0,0,2 +2009,20606,CGY,1,2,2,0,0,5 +2009,20606,NYI,0,2,0,0,0,2 +2009,20607,PHX,1,1,1,0,0,3 +2009,20607,TBL,1,1,0,0,0,2 +2009,20608,ANA,0,1,2,0,0,3 +2009,20608,LAK,1,2,1,0,0,4 +2009,20609,CBJ,1,1,1,0,0,3 +2009,20609,WSH,0,0,0,0,0,0 +2009,20610,BUF,0,1,0,0,1,2 +2009,20610,NYR,1,0,0,0,0,1 +2009,20611,EDM,1,0,0,0,0,1 +2009,20611,SJS,3,1,0,0,0,4 +2009,20612,STL,1,2,3,0,0,6 +2009,20612,VAN,2,0,2,0,0,4 +2009,20613,ANA,0,3,0,0,0,3 +2009,20613,TBL,1,2,1,0,0,4 +2009,20614,BOS,2,3,0,0,0,5 +2009,20614,CAR,0,0,1,0,0,1 +2009,20615,COL,1,3,1,0,0,5 +2009,20615,PIT,1,1,1,0,0,3 +2009,20616,MTL,1,0,4,0,0,5 +2009,20616,WSH,1,1,2,0,0,4 +2009,20617,NYR,1,0,1,0,0,2 +2009,20617,OTT,0,0,0,0,0,0 +2009,20618,PHI,1,0,3,0,0,4 +2009,20618,TOR,0,0,1,0,0,1 +2009,20619,ATL,1,2,1,0,0,4 +2009,20619,FLA,3,0,5,0,0,8 +2009,20620,BUF,1,0,0,0,0,1 +2009,20620,DET,0,1,2,0,0,3 +2009,20621,CBJ,1,0,3,0,0,4 +2009,20621,MIN,1,0,1,0,0,2 +2009,20622,CHI,0,0,1,0,0,1 +2009,20622,NSH,1,1,2,0,0,4 +2009,20623,DAL,0,0,0,0,0,0 +2009,20623,PHX,0,0,0,0,1,1 +2009,20624,SJS,1,1,2,0,0,4 +2009,20624,VAN,1,1,0,0,0,2 +2009,20625,LAK,0,0,1,0,0,1 +2009,20625,NJD,1,2,2,0,0,5 +2009,20626,CHI,1,1,1,0,0,3 +2009,20626,NSH,1,0,0,0,0,1 +2009,20627,EDM,0,1,1,0,0,2 +2009,20627,STL,0,1,0,0,0,1 +2009,20628,ANA,2,0,2,0,0,4 +2009,20628,NJD,1,1,1,0,0,3 +2009,20629,DAL,1,2,1,1,0,5 +2009,20629,DET,3,0,1,0,0,4 +2009,20630,LAK,0,1,0,0,0,1 +2009,20630,TBL,2,1,0,0,0,3 +2009,20631,BOS,0,2,1,0,0,3 +2009,20631,MTL,0,1,0,0,0,1 +2009,20632,NYI,1,0,0,0,0,1 +2009,20632,NYR,0,2,0,0,0,2 +2009,20633,PHI,1,0,1,0,0,2 +2009,20633,PIT,1,3,0,0,0,4 +2009,20634,EDM,2,1,2,0,0,5 +2009,20634,WSH,0,1,1,0,0,2 +2009,20635,COL,1,0,2,0,0,3 +2009,20635,CBJ,0,1,3,0,0,4 +2009,20636,NSH,0,0,2,0,0,2 +2009,20636,TOR,0,0,0,0,0,0 +2009,20637,CAR,0,1,0,0,0,1 +2009,20637,OTT,2,1,2,0,0,5 +2009,20638,MIN,2,1,3,0,0,6 +2009,20638,PHX,0,1,2,0,0,3 +2009,20639,CGY,1,0,2,0,0,3 +2009,20639,STL,1,0,0,0,0,1 +2009,20640,NJD,3,0,2,0,0,5 +2009,20640,VAN,0,1,2,0,0,3 +2009,20641,SJS,2,3,2,0,0,7 +2009,20641,TBL,0,1,0,0,0,1 +2009,20642,PIT,1,1,1,0,0,3 +2009,20642,WSH,0,2,4,0,0,6 +2009,20643,ATL,2,0,0,0,0,2 +2009,20643,OTT,2,1,0,0,0,3 +2009,20644,BUF,1,0,0,0,0,1 +2009,20644,CHI,2,1,1,0,0,4 +2009,20645,ANA,2,1,0,0,0,3 +2009,20645,DET,1,1,2,0,0,4 +2009,20646,BOS,0,1,1,0,0,2 +2009,20646,NYI,0,0,1,0,0,1 +2009,20647,CAR,0,1,3,0,0,4 +2009,20647,TOR,1,3,2,0,0,6 +2009,20648,MTL,1,1,1,0,0,3 +2009,20648,NSH,0,1,1,0,0,2 +2009,20649,PHI,0,1,0,0,0,1 +2009,20649,TBL,0,3,1,0,0,4 +2009,20650,EDM,1,0,0,0,0,1 +2009,20650,MIN,2,0,3,0,0,5 +2009,20651,COL,0,0,2,0,0,2 +2009,20651,STL,1,1,3,0,0,5 +2009,20652,BUF,0,1,3,0,1,5 +2009,20652,DAL,2,1,1,0,0,4 +2009,20653,PHX,2,2,0,0,0,4 +2009,20653,VAN,0,0,1,0,0,1 +2009,20654,DET,2,2,0,0,0,4 +2009,20654,LAK,0,0,0,0,0,0 +2009,20655,CGY,0,2,1,0,0,3 +2009,20655,SJS,1,1,0,0,0,2 +2009,20656,CBJ,0,1,0,0,0,1 +2009,20656,NJD,0,0,2,0,0,2 +2009,20657,ANA,0,1,0,0,0,1 +2009,20657,PIT,1,1,1,0,0,3 +2009,20658,ATL,0,2,1,1,0,4 +2009,20658,TOR,3,0,0,0,0,3 +2009,20659,FLA,0,2,0,0,0,2 +2009,20659,PHI,2,0,0,0,1,3 +2009,20660,CHI,1,0,1,0,0,2 +2009,20660,NYR,0,1,1,1,0,3 +2009,20661,COL,1,1,0,0,0,2 +2009,20661,EDM,2,0,1,0,0,3 +2009,20662,DAL,0,0,2,0,1,3 +2009,20662,LAK,0,0,2,0,0,2 +2009,20663,BUF,2,1,0,0,0,3 +2009,20663,CAR,0,0,1,0,0,1 +2009,20664,MTL,2,1,1,0,1,5 +2009,20664,OTT,1,0,3,0,0,4 +2009,20665,NJD,1,2,0,0,0,3 +2009,20665,NYI,0,0,1,0,0,1 +2009,20666,BOS,0,1,0,0,0,1 +2009,20666,WSH,0,1,1,0,0,2 +2009,20667,FLA,1,3,0,0,0,4 +2009,20667,TBL,3,0,0,0,0,3 +2009,20668,ATL,4,1,2,0,0,7 +2009,20668,NSH,0,1,1,0,0,2 +2009,20669,CHI,1,0,0,1,0,2 +2009,20669,STL,0,0,1,0,0,1 +2009,20670,ANA,0,1,2,0,0,3 +2009,20670,MIN,0,0,0,0,0,0 +2009,20671,CGY,0,1,2,0,0,3 +2009,20671,PHX,1,2,1,0,0,4 +2009,20672,DET,1,3,1,0,0,5 +2009,20672,SJS,2,2,2,0,0,6 +2009,20673,NYR,0,0,0,0,0,0 +2009,20673,PIT,2,1,0,0,0,3 +2009,20674,CGY,0,1,1,0,0,2 +2009,20674,COL,2,2,2,0,0,6 +2009,20675,EDM,3,1,2,0,0,6 +2009,20675,PHX,1,2,0,0,0,3 +2009,20676,CBJ,1,2,2,0,1,6 +2009,20676,VAN,1,2,2,0,0,5 +2009,20677,BOS,1,0,3,0,0,4 +2009,20677,STL,1,1,2,0,1,5 +2009,20678,NYI,0,0,1,0,0,1 +2009,20678,WSH,1,0,0,1,0,2 +2009,20679,CAR,0,2,0,0,0,2 +2009,20679,TOR,0,0,0,0,0,0 +2009,20680,DAL,1,1,0,0,0,2 +2009,20680,TBL,1,0,3,0,0,4 +2009,20681,BUF,0,1,1,0,1,3 +2009,20681,FLA,1,0,1,0,0,2 +2009,20682,CHI,0,1,0,0,0,1 +2009,20682,MIN,2,1,1,0,0,4 +2009,20683,NSH,0,0,1,0,0,1 +2009,20683,NJD,0,2,1,0,0,3 +2009,20684,ANA,1,1,0,0,0,2 +2009,20684,NYR,1,1,2,0,0,4 +2009,20685,ATL,2,1,1,0,0,4 +2009,20685,MTL,0,2,0,0,0,2 +2009,20686,OTT,2,0,1,0,0,3 +2009,20686,WSH,0,2,0,0,0,2 +2009,20687,CAR,1,1,0,0,0,2 +2009,20687,PIT,1,0,0,0,0,1 +2009,20688,LAK,1,3,1,0,0,5 +2009,20688,MIN,1,1,0,0,0,2 +2009,20689,CBJ,0,3,0,0,0,3 +2009,20689,EDM,1,1,2,0,0,4 +2009,20690,DET,1,2,0,0,0,3 +2009,20690,PHX,1,2,3,0,0,6 +2009,20691,SJS,0,0,1,1,0,2 +2009,20691,VAN,1,0,0,0,0,1 +2009,20692,MTL,0,1,1,0,0,2 +2009,20692,NJD,1,2,2,0,0,5 +2009,20693,ANA,0,1,0,0,0,1 +2009,20693,NYI,2,0,0,0,0,2 +2009,20694,ATL,0,0,3,0,0,3 +2009,20694,PHI,2,1,2,0,0,5 +2009,20695,BOS,1,0,2,0,1,4 +2009,20695,TOR,1,2,0,0,0,3 +2009,20696,BUF,0,3,0,0,0,3 +2009,20696,TBL,2,3,0,0,0,5 +2009,20697,DAL,0,2,2,0,0,4 +2009,20697,FLA,1,0,0,0,0,1 +2009,20698,CHI,0,0,0,0,0,0 +2009,20698,STL,1,0,1,0,0,2 +2009,20699,COL,1,2,2,0,0,5 +2009,20699,LAK,1,5,0,0,0,6 +2009,20700,CGY,1,2,1,0,1,5 +2009,20700,CBJ,2,1,1,0,0,4 +2009,20701,BOS,1,1,0,1,0,3 +2009,20701,WSH,2,0,0,0,0,2 +2009,20702,CAR,1,0,1,0,0,2 +2009,20702,NYR,2,0,1,0,0,3 +2009,20703,CBJ,0,2,0,1,0,3 +2009,20703,DET,1,1,0,0,0,2 +2009,20704,NJD,1,1,2,0,0,4 +2009,20704,OTT,0,1,0,0,0,1 +2009,20705,MTL,2,0,1,0,0,3 +2009,20705,TBL,1,3,1,0,0,5 +2009,20706,FLA,1,2,0,0,0,3 +2009,20706,PHI,1,0,1,0,0,2 +2009,20707,MIN,1,4,1,0,0,6 +2009,20707,TOR,0,1,0,0,0,1 +2009,20708,ATL,0,0,0,0,0,0 +2009,20708,DAL,1,1,0,0,0,2 +2009,20709,COL,0,0,0,0,0,0 +2009,20709,SJS,1,1,1,0,0,3 +2009,20710,BUF,3,4,3,0,0,10 +2009,20710,EDM,0,1,1,0,0,2 +2009,20711,ANA,1,4,2,0,0,7 +2009,20711,PHX,1,2,0,0,0,3 +2009,20712,NYR,1,0,1,0,0,2 +2009,20712,PIT,1,0,5,0,0,6 +2009,20713,BUF,0,2,0,0,0,2 +2009,20713,CGY,1,1,3,0,0,5 +2009,20714,ANA,1,1,0,0,0,2 +2009,20714,CHI,2,1,0,0,0,3 +2009,20715,NSH,1,2,2,0,0,5 +2009,20715,VAN,0,3,0,0,0,3 +2009,20716,BOS,0,0,3,0,0,3 +2009,20716,NJD,1,1,1,1,0,4 +2009,20717,CAR,0,2,1,0,0,3 +2009,20717,TBL,1,0,1,0,0,2 +2009,20718,ATL,0,0,4,0,0,4 +2009,20718,NYI,4,0,1,0,0,5 +2009,20719,MTL,0,1,0,0,0,1 +2009,20719,FLA,2,1,2,0,0,5 +2009,20720,DAL,3,1,0,0,0,4 +2009,20720,DET,0,2,0,0,0,2 +2009,20721,OTT,1,0,2,0,0,3 +2009,20721,STL,1,0,0,0,0,1 +2009,20722,COL,1,2,1,0,0,4 +2009,20722,TOR,1,3,3,0,0,7 +2009,20723,CHI,0,0,2,0,0,2 +2009,20723,LAK,2,1,2,0,0,5 +2009,20724,PHX,0,0,0,0,0,0 +2009,20724,SJS,0,1,1,0,0,2 +2009,20725,NJD,1,0,2,1,0,4 +2009,20725,PIT,2,0,1,0,0,3 +2009,20726,CBJ,0,1,0,0,0,1 +2009,20726,OTT,0,0,0,0,0,0 +2009,20727,PHI,1,2,3,0,0,6 +2009,20727,TBL,1,0,0,0,0,1 +2009,20728,CGY,0,2,1,0,0,3 +2009,20728,NSH,0,0,1,0,0,1 +2009,20729,EDM,2,1,0,0,0,3 +2009,20729,MIN,0,0,1,0,0,1 +2009,20730,DET,1,1,0,0,0,2 +2009,20730,WSH,1,1,2,0,0,4 +2009,20731,BOS,0,1,0,0,0,1 +2009,20731,NYR,0,0,0,0,0,0 +2009,20732,MTL,1,1,2,0,0,4 +2009,20732,LAK,1,2,0,0,0,3 +2009,20733,ANA,2,1,1,0,0,4 +2009,20733,COL,1,2,0,0,0,3 +2009,20734,PIT,1,0,3,0,0,4 +2009,20734,TOR,1,2,2,0,0,5 +2009,20735,FLA,0,0,1,0,0,1 +2009,20735,NYI,1,0,2,0,0,3 +2009,20736,CBJ,1,1,1,0,0,3 +2009,20736,DAL,3,2,2,0,0,7 +2009,20737,ATL,0,0,0,0,0,0 +2009,20737,CAR,2,0,0,0,0,2 +2009,20738,PHI,0,0,0,0,0,0 +2009,20738,STL,1,2,1,0,0,4 +2009,20739,BUF,0,2,0,0,0,2 +2009,20739,PHX,0,0,0,0,0,0 +2009,20740,MIN,2,0,1,1,0,4 +2009,20740,VAN,0,1,2,0,0,3 +2009,20741,CHI,1,2,1,0,0,4 +2009,20741,SJS,1,0,1,0,0,2 +2009,20742,OTT,1,1,2,0,0,4 +2009,20742,WSH,2,4,1,0,0,7 +2009,20743,BOS,1,1,1,0,0,3 +2009,20743,MTL,1,0,0,0,0,1 +2009,20744,EDM,0,0,1,0,0,1 +2009,20744,NSH,0,1,1,0,0,2 +2009,20745,DET,0,3,0,0,1,4 +2009,20745,STL,0,2,1,0,0,3 +2009,20746,CGY,1,2,0,0,0,3 +2009,20746,COL,1,3,0,0,0,4 +2009,20747,ANA,0,1,2,0,0,3 +2009,20747,BUF,1,0,1,0,0,2 +2009,20748,NJD,0,1,1,0,0,2 +2009,20748,WSH,2,1,2,0,0,5 +2009,20749,NYI,0,3,0,0,0,3 +2009,20749,TBL,0,0,1,0,0,1 +2009,20750,ATL,0,0,1,0,1,2 +2009,20750,NYR,0,0,1,0,0,1 +2009,20751,CBJ,2,0,0,0,0,2 +2009,20751,STL,2,1,1,0,0,4 +2009,20752,FLA,1,0,2,1,0,4 +2009,20752,TOR,2,1,0,0,0,3 +2009,20753,MTL,1,2,1,0,0,4 +2009,20753,PIT,1,0,1,0,0,2 +2009,20754,LAK,0,0,1,0,0,1 +2009,20754,OTT,0,0,0,0,0,0 +2009,20755,NSH,1,1,0,0,0,2 +2009,20755,PHX,0,0,1,0,0,1 +2009,20756,CGY,1,0,0,0,0,1 +2009,20756,DAL,1,1,1,0,0,3 +2009,20757,CHI,0,2,1,0,0,3 +2009,20757,EDM,1,0,0,0,0,1 +2009,20758,CAR,0,2,1,0,0,3 +2009,20758,VAN,2,1,1,0,0,4 +2009,20759,BOS,0,1,2,0,0,3 +2009,20759,PHI,0,1,0,0,0,1 +2009,20760,BUF,2,1,2,0,0,5 +2009,20760,TOR,0,0,0,0,0,0 +2009,20761,PIT,0,0,3,1,0,4 +2009,20761,TBL,1,2,0,0,0,3 +2009,20762,DET,1,2,2,0,0,5 +2009,20762,PHX,1,1,2,0,0,4 +2009,20763,ANA,0,0,0,0,0,0 +2009,20763,MIN,1,1,1,0,0,3 +2009,20764,LAK,1,3,1,0,0,5 +2009,20764,WSH,1,1,2,0,0,4 +2009,20765,BOS,1,1,1,0,1,4 +2009,20765,OTT,0,1,2,0,0,3 +2009,20766,FLA,2,1,0,0,0,3 +2009,20766,NYI,0,2,0,0,0,2 +2009,20767,ANA,1,1,0,0,0,2 +2009,20767,NSH,1,2,1,0,0,4 +2009,20768,EDM,0,1,0,0,1,2 +2009,20768,STL,0,0,1,0,0,1 +2009,20769,COL,0,1,2,0,0,3 +2009,20769,DAL,1,0,1,0,0,2 +2009,20770,CGY,2,0,0,0,0,2 +2009,20770,CHI,2,1,2,0,0,5 +2009,20771,CAR,0,2,1,0,1,4 +2009,20771,SJS,1,2,0,0,0,3 +2009,20772,BUF,0,2,1,0,0,3 +2009,20772,MTL,0,1,1,0,0,2 +2009,20773,CBJ,0,0,1,0,0,1 +2009,20773,PIT,0,3,1,0,0,4 +2009,20774,ATL,0,1,0,0,0,1 +2009,20774,NJD,3,2,0,0,0,5 +2009,20775,MIN,0,0,0,0,0,0 +2009,20775,NSH,0,1,1,0,0,2 +2009,20776,DAL,3,1,6,0,0,10 +2009,20776,NYR,1,1,0,0,0,2 +2009,20777,BOS,3,0,0,0,0,3 +2009,20777,PHI,1,2,0,1,0,4 +2009,20778,DET,5,1,2,0,0,8 +2009,20778,EDM,0,0,3,0,0,3 +2009,20779,ANA,1,1,0,0,0,2 +2009,20779,CGY,0,0,1,0,0,1 +2009,20780,MTL,0,1,1,0,0,2 +2009,20780,TOR,1,1,3,0,0,5 +2009,20781,BUF,0,0,2,0,0,2 +2009,20781,OTT,0,2,0,0,1,3 +2009,20782,LAK,0,3,0,0,0,3 +2009,20782,NJD,0,0,1,0,0,1 +2009,20783,FLA,1,0,0,0,0,1 +2009,20783,WSH,1,0,2,0,0,3 +2009,20784,CBJ,2,0,0,1,0,3 +2009,20784,SJS,0,1,1,0,0,2 +2009,20785,NYI,0,0,0,0,0,0 +2009,20785,TBL,0,0,1,0,0,1 +2009,20786,COL,1,0,0,0,0,1 +2009,20786,STL,0,3,1,0,0,4 +2009,20787,CAR,1,3,3,0,0,7 +2009,20787,PHX,1,1,0,0,0,2 +2009,20788,CHI,0,1,2,0,0,3 +2009,20788,VAN,3,3,1,0,0,7 +2009,20789,DET,0,1,2,0,0,3 +2009,20789,PIT,0,0,0,0,0,0 +2009,20790,ATL,0,0,2,0,0,2 +2009,20790,PHI,1,1,1,0,0,3 +2009,20791,EDM,1,0,1,0,0,2 +2009,20791,MIN,1,0,1,0,1,3 +2009,20792,DAL,2,0,2,0,0,4 +2009,20792,NSH,0,1,0,0,0,1 +2009,20793,NJD,0,2,1,0,0,3 +2009,20793,NYR,0,0,0,0,0,0 +2009,20794,CGY,1,3,2,0,0,6 +2009,20794,MTL,2,0,0,0,0,2 +2009,20795,BOS,2,0,0,0,0,2 +2009,20795,SJS,1,0,4,0,0,5 +2009,20796,LAK,1,1,1,0,1,4 +2009,20796,NYI,1,2,0,0,0,3 +2009,20797,COL,0,0,0,0,0,0 +2009,20797,CBJ,0,0,3,0,0,3 +2009,20798,ATL,1,1,1,0,0,3 +2009,20798,TBL,1,0,0,0,0,1 +2009,20799,FLA,0,1,3,1,0,5 +2009,20799,TOR,1,2,1,0,0,4 +2009,20800,DET,2,1,2,0,0,5 +2009,20800,NSH,0,2,1,0,0,3 +2009,20801,STL,2,1,1,0,0,4 +2009,20801,VAN,0,2,4,0,0,6 +2009,20802,NJD,1,1,2,0,0,4 +2009,20802,NYI,1,1,0,0,0,2 +2009,20803,NYR,1,3,0,0,1,5 +2009,20803,WSH,2,2,0,0,0,4 +2009,20804,ATL,0,1,0,0,0,1 +2009,20804,CHI,0,2,1,0,0,3 +2009,20805,BUF,0,0,1,0,0,1 +2009,20805,OTT,1,2,0,0,0,3 +2009,20806,PIT,0,1,0,0,1,2 +2009,20806,SJS,0,0,1,0,0,1 +2009,20807,COL,1,0,1,0,0,2 +2009,20807,MIN,1,1,1,0,0,3 +2009,20808,DAL,0,0,0,0,0,0 +2009,20808,PHX,0,1,0,0,0,1 +2009,20809,MTL,1,0,1,0,0,2 +2009,20809,EDM,4,2,1,0,0,7 +2009,20810,ANA,0,1,1,1,0,3 +2009,20810,CGY,1,1,0,0,0,2 +2009,20811,OTT,1,2,2,0,0,5 +2009,20811,PHI,0,0,2,0,0,2 +2009,20812,CAR,0,0,0,0,0,0 +2009,20812,FLA,2,3,0,0,0,5 +2009,20813,TBL,4,0,2,0,0,6 +2009,20813,TOR,1,2,1,0,0,4 +2009,20814,DET,1,3,0,0,0,4 +2009,20814,MIN,1,0,1,0,0,2 +2009,20815,NSH,1,0,2,0,1,4 +2009,20815,STL,2,1,0,0,0,3 +2009,20816,PHX,2,1,0,0,0,3 +2009,20816,VAN,0,1,3,0,0,4 +2009,20817,CGY,1,0,1,0,0,2 +2009,20817,LAK,0,0,0,0,0,0 +2009,20818,BOS,0,0,0,0,0,0 +2009,20818,NJD,0,1,0,0,0,1 +2009,20819,CBJ,1,1,1,0,0,3 +2009,20819,DET,0,1,1,0,0,2 +2009,20820,BUF,3,1,1,0,1,6 +2009,20820,SJS,1,2,2,0,0,5 +2009,20821,FLA,0,1,0,0,1,2 +2009,20821,NYR,1,0,0,0,0,1 +2009,20822,CHI,0,0,0,0,0,0 +2009,20822,STL,0,0,1,0,0,1 +2009,20823,DAL,0,2,0,0,0,2 +2009,20823,VAN,1,0,0,0,0,1 +2009,20824,MTL,2,0,2,0,0,4 +2009,20824,COL,0,1,1,0,0,2 +2009,20825,NYI,0,0,1,0,0,1 +2009,20825,PHI,2,0,3,0,0,5 +2009,20826,EDM,2,0,0,0,1,3 +2009,20826,LAK,0,1,1,0,0,2 +2009,20827,PIT,2,0,0,0,0,2 +2009,20827,TOR,0,1,5,0,0,6 +2009,20828,CAR,0,0,1,0,0,1 +2009,20828,CBJ,1,1,3,0,0,5 +2009,20829,TBL,0,1,0,0,0,1 +2009,20829,WSH,3,1,1,0,0,5 +2009,20830,BOS,0,1,1,0,0,2 +2009,20830,NSH,1,1,0,0,1,3 +2009,20831,MIN,3,0,0,0,0,3 +2009,20831,OTT,0,3,2,0,0,5 +2009,20832,CHI,1,2,3,0,0,6 +2009,20832,DAL,0,0,2,0,0,2 +2009,20833,CGY,1,4,2,0,0,7 +2009,20833,PHX,1,1,3,0,0,5 +2009,20834,NJD,3,0,3,0,0,6 +2009,20834,SJS,2,2,1,0,0,5 +2009,20835,NYR,0,1,1,0,0,2 +2009,20835,PHI,1,4,0,0,0,5 +2009,20836,FLA,0,1,1,0,0,2 +2009,20836,WSH,1,1,2,0,0,4 +2009,20837,COL,1,1,3,0,1,6 +2009,20837,DET,0,3,2,0,0,5 +2009,20838,BUF,0,0,0,0,0,0 +2009,20838,CAR,2,0,1,0,0,3 +2009,20839,ANA,1,1,2,0,0,4 +2009,20839,ATL,2,4,2,0,0,8 +2009,20840,MTL,1,0,1,0,0,2 +2009,20840,VAN,2,1,1,0,0,4 +2009,20841,NYI,1,1,0,0,1,3 +2009,20841,PIT,1,1,0,0,0,2 +2009,20842,CBJ,0,2,0,0,0,2 +2009,20842,DAL,1,1,0,0,1,3 +2009,20843,NYR,0,1,0,0,0,1 +2009,20843,STL,0,1,1,0,0,2 +2009,20844,NSH,0,0,0,0,0,0 +2009,20844,OTT,2,0,0,0,0,2 +2009,20845,EDM,0,1,2,0,0,3 +2009,20845,PHX,0,1,0,0,0,1 +2009,20846,ATL,4,1,1,0,1,7 +2009,20846,LAK,2,1,3,0,0,6 +2009,20847,BOS,1,1,3,0,0,5 +2009,20847,CAR,1,0,0,0,0,1 +2009,20848,BUF,3,0,1,0,0,4 +2009,20848,TOR,0,1,0,0,0,1 +2009,20849,CHI,1,1,3,0,0,5 +2009,20849,TBL,1,2,0,0,0,3 +2009,20850,FLA,1,2,1,0,0,4 +2009,20850,NJD,0,0,0,0,0,0 +2009,20851,COL,1,1,0,1,0,3 +2009,20851,OTT,1,1,0,0,0,2 +2009,20852,CGY,1,1,1,0,0,3 +2009,20852,VAN,1,1,1,0,1,4 +2009,20853,EDM,2,0,0,0,0,2 +2009,20853,SJS,1,3,0,0,0,4 +2009,20854,NYI,1,0,0,0,0,1 +2009,20854,NYR,1,1,1,0,0,3 +2009,20855,CBJ,2,2,0,0,0,4 +2009,20855,STL,0,1,2,0,0,3 +2009,20856,MTL,2,0,1,0,0,3 +2009,20856,WSH,2,0,1,0,1,4 +2009,20857,DET,2,2,2,0,0,6 +2009,20857,NSH,1,1,0,0,0,2 +2009,20858,ANA,0,1,2,0,0,3 +2009,20858,LAK,1,1,2,0,0,4 +2009,20859,CAR,3,2,1,0,0,6 +2009,20859,NYI,1,0,1,0,0,2 +2009,20860,BUF,0,3,0,0,0,3 +2009,20860,PHI,1,2,3,0,0,6 +2009,20861,MTL,0,2,2,0,0,4 +2009,20861,PIT,1,1,3,0,0,5 +2009,20862,CBJ,1,1,1,0,1,4 +2009,20862,TOR,1,2,0,0,0,3 +2009,20863,OTT,0,2,0,0,0,2 +2009,20863,VAN,3,1,1,0,0,5 +2009,20864,NJD,0,1,1,0,1,3 +2009,20864,TBL,2,0,0,0,0,2 +2009,20865,CHI,1,0,3,0,0,4 +2009,20865,FLA,0,0,0,0,0,0 +2009,20866,NSH,1,0,0,0,0,1 +2009,20866,STL,0,1,0,1,0,2 +2009,20867,CGY,1,1,0,1,0,3 +2009,20867,MIN,0,2,0,0,0,2 +2009,20868,DAL,2,1,1,0,0,4 +2009,20868,EDM,1,1,0,0,0,2 +2009,20869,ATL,1,1,1,0,0,3 +2009,20869,PHX,1,2,0,0,1,4 +2009,20870,LAK,1,1,0,0,0,2 +2009,20870,SJS,2,2,0,0,0,4 +2009,20871,COL,1,1,2,0,0,4 +2009,20871,WSH,1,0,0,0,0,1 +2009,20872,CAR,1,1,2,0,0,4 +2009,20872,TBL,0,1,0,0,0,1 +2009,20873,ANA,1,0,1,0,0,2 +2009,20873,DET,2,1,2,0,0,5 +2009,20874,PHI,1,0,3,0,0,4 +2009,20874,PIT,0,3,2,0,0,5 +2009,20875,MTL,2,3,0,0,0,5 +2009,20875,OTT,0,2,1,0,0,3 +2009,20876,CHI,1,2,0,0,0,3 +2009,20876,DAL,0,0,1,0,0,1 +2009,20877,LAK,1,1,1,0,0,3 +2009,20877,PHX,1,3,2,0,0,6 +2009,20878,ATL,0,0,1,0,0,1 +2009,20878,SJS,0,0,3,0,0,3 +2009,20879,BUF,2,1,1,0,0,4 +2009,20879,NYR,0,0,2,0,0,2 +2009,20880,TOR,1,1,0,0,0,2 +2009,20880,VAN,0,1,1,0,1,3 +2009,20881,NJD,0,0,0,0,0,0 +2009,20881,NYI,1,1,2,0,0,4 +2009,20882,BOS,0,0,0,0,0,0 +2009,20882,FLA,0,0,2,0,0,2 +2009,20883,ANA,2,2,1,0,0,5 +2009,20883,CBJ,0,1,1,0,0,2 +2009,20884,DET,0,1,1,0,0,2 +2009,20884,MIN,2,2,1,0,0,5 +2009,20885,NSH,0,0,0,1,0,1 +2009,20885,STL,0,0,0,0,0,0 +2009,20886,CGY,0,1,1,0,1,3 +2009,20886,EDM,1,0,1,0,0,2 +2009,20887,PIT,1,1,0,0,0,2 +2009,20887,WSH,2,3,0,0,0,5 +2009,20888,CAR,1,2,2,0,0,5 +2009,20888,COL,1,1,0,0,0,2 +2009,20889,BOS,2,1,0,0,0,3 +2009,20889,TBL,2,1,1,0,0,4 +2009,20890,NYR,0,0,2,0,0,2 +2009,20890,TOR,0,0,2,1,0,3 +2009,20891,CHI,0,1,0,0,0,1 +2009,20891,MIN,0,1,1,0,0,2 +2009,20892,DAL,0,0,0,0,0,0 +2009,20892,SJS,0,0,1,0,0,1 +2009,20893,BOS,2,1,3,0,0,6 +2009,20893,FLA,1,0,0,0,0,1 +2009,20894,ANA,1,2,0,0,0,3 +2009,20894,BUF,1,0,1,0,0,2 +2009,20895,PHI,0,1,3,0,0,4 +2009,20895,WSH,1,1,0,0,0,2 +2009,20896,ATL,3,1,0,0,0,4 +2009,20896,COL,1,1,1,0,0,3 +2009,20897,MTL,1,1,1,0,0,3 +2009,20897,VAN,0,0,0,0,0,0 +2009,20898,CAR,1,1,0,0,0,2 +2009,20898,OTT,3,1,0,0,0,4 +2009,20899,CHI,0,3,0,0,0,3 +2009,20899,NSH,0,1,4,0,0,5 +2009,20900,LAK,0,1,0,0,1,2 +2009,20900,MIN,0,1,0,0,0,1 +2009,20901,PHX,1,0,0,0,0,1 +2009,20901,STL,0,2,0,0,0,2 +2009,20902,CGY,1,2,1,0,0,4 +2009,20902,CBJ,1,0,0,0,0,1 +2009,20903,EDM,2,1,2,0,0,5 +2009,20903,TBL,0,2,1,0,0,3 +2009,20904,LAK,0,0,0,0,0,0 +2009,20904,PHI,0,1,1,0,0,2 +2009,20905,NYR,0,1,0,0,0,1 +2009,20905,TOR,0,0,1,0,1,2 +2009,20906,NYI,0,0,0,0,0,0 +2009,20906,PIT,0,0,1,0,0,1 +2009,20907,DET,2,1,1,0,0,4 +2009,20907,SJS,0,1,0,0,0,1 +2009,20908,ANA,0,0,0,0,0,0 +2009,20908,BOS,1,4,1,0,0,6 +2009,20909,COL,0,0,0,0,0,0 +2009,20909,NJD,1,1,2,0,0,4 +2009,20910,NYI,0,1,3,0,0,4 +2009,20910,TOR,1,1,2,0,1,5 +2009,20911,FLA,0,0,2,0,0,2 +2009,20911,NYR,1,0,0,0,0,1 +2009,20912,ATL,1,1,1,0,0,3 +2009,20912,WSH,2,1,1,0,0,4 +2009,20913,BUF,0,1,0,0,0,1 +2009,20913,CAR,0,0,1,0,1,2 +2009,20914,OTT,1,0,0,0,0,1 +2009,20914,SJS,0,2,0,0,0,2 +2009,20915,NSH,1,1,2,0,0,4 +2009,20915,PHX,0,1,0,0,0,1 +2009,20916,DAL,1,0,0,0,0,1 +2009,20916,STL,1,0,2,0,0,3 +2009,20917,CBJ,0,0,1,0,0,1 +2009,20917,EDM,0,0,0,0,0,0 +2009,20918,MTL,3,0,0,1,0,4 +2009,20918,PHI,2,1,0,0,0,3 +2009,20919,DET,0,1,1,0,0,2 +2009,20919,LAK,0,0,1,0,0,1 +2009,20920,CHI,1,1,2,0,0,4 +2009,20920,PIT,2,2,0,1,0,5 +2009,20921,CGY,0,3,1,0,0,4 +2009,20921,MIN,0,0,1,0,0,1 +2009,20922,TBL,0,1,0,0,0,1 +2009,20922,VAN,1,0,1,0,0,2 +2009,20923,BOS,1,1,1,0,0,3 +2009,20923,WSH,1,1,1,1,0,4 +2009,20924,FLA,0,1,1,0,0,2 +2009,20924,NJD,2,4,1,0,0,7 +2009,20925,ANA,2,0,2,0,0,4 +2009,20925,DAL,0,2,1,0,0,3 +2009,20926,MTL,3,0,0,0,0,3 +2009,20926,SJS,0,2,0,0,0,2 +2009,20927,OTT,0,2,1,0,0,3 +2009,20927,TOR,2,0,1,1,0,4 +2009,20928,BUF,0,0,0,0,0,0 +2009,20928,NYI,0,1,1,0,0,2 +2009,20929,COL,0,1,0,0,0,1 +2009,20929,NYR,4,1,1,0,0,6 +2009,20930,ATL,0,1,4,0,0,5 +2009,20930,CAR,2,0,1,0,0,3 +2009,20931,DET,0,0,0,0,0,0 +2009,20931,NSH,5,1,2,0,0,8 +2009,20932,PHX,0,0,1,0,0,1 +2009,20932,STL,0,1,2,0,0,3 +2009,20933,EDM,1,2,0,0,0,3 +2009,20933,MIN,2,0,0,0,0,2 +2009,20934,NJD,1,1,1,0,0,3 +2009,20934,PHI,0,0,0,0,0,0 +2009,20935,FLA,4,1,1,0,0,6 +2009,20935,WSH,1,0,1,0,0,2 +2009,20936,CHI,3,0,1,0,0,4 +2009,20936,LAK,2,0,0,0,0,2 +2009,20937,DAL,0,0,1,0,0,1 +2009,20937,PIT,2,2,0,0,0,4 +2009,20938,CGY,1,3,2,0,0,6 +2009,20938,TBL,1,4,3,0,0,8 +2009,20939,CBJ,0,0,1,0,0,1 +2009,20939,VAN,1,1,1,0,0,3 +2009,20940,COL,0,1,1,0,0,2 +2009,20940,NYI,1,2,1,0,0,4 +2009,20941,BOS,1,0,1,0,0,2 +2009,20941,PHI,0,1,3,0,0,4 +2009,20942,CAR,1,4,0,0,0,5 +2009,20942,WSH,0,1,1,0,0,2 +2009,20943,ATL,2,0,1,0,0,3 +2009,20943,FLA,1,3,0,0,0,4 +2009,20944,CBJ,2,2,1,0,0,5 +2009,20944,LAK,1,3,0,0,0,4 +2009,20945,NJD,1,0,1,1,0,3 +2009,20945,TOR,0,0,2,0,0,2 +2009,20946,CGY,3,1,2,0,0,6 +2009,20946,OTT,2,0,1,0,0,3 +2009,20947,PIT,2,1,0,0,0,3 +2009,20947,TBL,1,0,0,0,0,1 +2009,20948,EDM,2,0,3,0,0,5 +2009,20948,NSH,2,1,2,1,0,6 +2009,20949,ANA,0,2,0,0,0,2 +2009,20949,CHI,2,0,0,1,0,3 +2009,20950,DET,1,1,3,0,0,5 +2009,20950,STL,0,0,0,0,0,0 +2009,20951,MIN,2,0,0,0,0,2 +2009,20951,VAN,1,2,1,0,0,4 +2009,20952,DAL,1,1,2,0,0,4 +2009,20952,SJS,1,0,0,0,0,1 +2009,20953,BUF,1,2,2,0,0,5 +2009,20953,MTL,0,0,1,0,0,1 +2009,20954,COL,1,0,1,0,0,2 +2009,20954,DET,1,2,0,0,0,3 +2009,20955,BOS,1,0,0,0,0,1 +2009,20955,PHX,1,1,0,0,0,2 +2009,20956,NYI,0,1,1,0,0,2 +2009,20956,NYR,1,1,2,0,0,4 +2009,20957,CGY,4,0,1,0,0,5 +2009,20957,PHI,0,0,1,0,0,1 +2009,20958,TOR,0,0,2,0,0,2 +2009,20958,WSH,0,0,1,0,0,1 +2009,20959,EDM,0,1,1,0,0,2 +2009,20959,OTT,1,2,1,0,0,4 +2009,20960,FLA,0,1,0,0,0,1 +2009,20960,PIT,0,1,3,0,0,4 +2009,20961,CBJ,0,2,0,0,0,2 +2009,20961,NSH,0,3,1,0,0,4 +2009,20962,DAL,0,4,0,0,0,4 +2009,20962,LAK,0,2,2,1,0,5 +2009,20963,MIN,0,2,1,1,0,4 +2009,20963,SJS,2,1,0,0,0,3 +2009,20964,CGY,0,0,1,0,0,1 +2009,20964,CAR,3,2,1,0,0,6 +2009,20965,BUF,2,1,2,0,0,5 +2009,20965,PHX,0,0,1,0,0,1 +2009,20966,ATL,1,0,1,0,0,2 +2009,20966,MTL,0,0,0,0,0,0 +2009,20967,STL,0,2,1,1,0,4 +2009,20967,TBL,0,0,3,0,0,3 +2009,20968,ANA,0,0,2,0,0,2 +2009,20968,DAL,1,1,1,0,0,3 +2009,20969,BOS,1,2,2,0,0,5 +2009,20969,CHI,0,1,2,0,0,3 +2009,20970,NJD,0,2,1,0,0,3 +2009,20970,NYI,3,3,1,0,0,7 +2009,20971,LAK,2,2,0,0,0,4 +2009,20971,MIN,2,0,1,0,0,3 +2009,20972,EDM,0,3,1,0,0,4 +2009,20972,TOR,0,1,0,0,0,1 +2009,20973,BUF,1,1,1,0,0,3 +2009,20973,OTT,1,2,3,0,0,6 +2009,20974,NSH,0,0,1,0,0,1 +2009,20974,PHI,0,2,2,0,0,4 +2009,20975,FLA,2,2,1,0,0,5 +2009,20975,STL,0,2,1,0,0,3 +2009,20976,CBJ,2,2,4,0,0,8 +2009,20976,DET,0,1,1,0,0,2 +2009,20977,CAR,3,6,0,0,0,9 +2009,20977,TBL,1,0,2,0,0,3 +2009,20978,SJS,0,1,0,0,0,1 +2009,20978,VAN,2,0,1,0,0,3 +2009,20979,NYI,0,3,0,0,0,3 +2009,20979,PHX,1,0,1,0,0,2 +2009,20980,BOS,0,1,2,0,0,3 +2009,20980,NYR,0,3,1,0,0,4 +2009,20981,PIT,1,2,0,0,1,4 +2009,20981,WSH,1,0,2,0,0,3 +2009,20982,ATL,2,1,2,0,0,5 +2009,20982,CGY,1,0,1,0,0,2 +2009,20983,CHI,0,1,0,0,0,1 +2009,20983,COL,2,2,1,0,0,5 +2009,20984,MTL,0,2,1,0,0,3 +2009,20984,DAL,1,0,0,0,0,1 +2009,20985,ANA,0,1,1,0,0,2 +2009,20985,MIN,1,1,1,0,0,3 +2009,20986,CAR,0,1,2,0,0,3 +2009,20986,NYR,0,0,0,0,0,0 +2009,20987,OTT,0,2,0,0,0,2 +2009,20987,TOR,1,0,0,0,0,1 +2009,20988,LAK,2,1,0,0,0,3 +2009,20988,VAN,0,1,1,0,0,2 +2009,20989,CGY,1,0,1,0,0,2 +2009,20989,NJD,0,2,1,0,0,3 +2009,20990,BUF,1,0,1,0,0,2 +2009,20990,PHI,1,0,4,0,0,5 +2009,20991,BOS,0,0,0,0,0,0 +2009,20991,CBJ,0,0,2,0,0,2 +2009,20992,NYI,2,0,0,0,0,2 +2009,20992,TOR,1,0,1,1,0,3 +2009,20993,MTL,1,1,1,1,0,4 +2009,20993,EDM,0,3,0,0,0,3 +2009,20994,FLA,0,3,0,0,0,3 +2009,20994,PIT,1,0,2,0,1,4 +2009,20995,DET,1,0,1,1,0,3 +2009,20995,PHX,1,0,1,0,0,2 +2009,20996,NSH,1,0,0,0,0,1 +2009,20996,WSH,0,1,0,1,0,2 +2009,20997,MIN,0,1,3,0,0,4 +2009,20997,SJS,2,1,1,1,0,5 +2009,20998,DAL,0,2,0,0,0,2 +2009,20998,STL,3,2,0,0,0,5 +2009,20999,ATL,0,2,1,0,0,3 +2009,20999,COL,0,0,0,0,0,0 +2009,21000,OTT,1,1,0,1,0,3 +2009,21000,TBL,1,1,0,0,0,2 +2009,21001,CAR,1,0,1,0,0,2 +2009,21001,CHI,0,1,1,0,1,3 +2009,21002,ANA,1,2,0,1,0,4 +2009,21002,VAN,0,2,1,0,0,3 +2009,21003,BOS,3,0,2,0,0,5 +2009,21003,OTT,1,0,2,0,0,3 +2009,21004,BUF,2,0,1,0,0,3 +2009,21004,FLA,1,0,0,0,0,1 +2009,21005,NJD,2,2,1,0,0,5 +2009,21005,PHX,1,0,1,0,0,2 +2009,21006,PHI,0,1,0,0,0,1 +2009,21006,WSH,1,1,0,0,0,2 +2009,21007,CBJ,1,1,1,0,1,4 +2009,21007,PIT,0,0,3,0,0,3 +2009,21008,TBL,0,2,2,0,0,4 +2009,21008,TOR,1,0,0,0,0,1 +2009,21009,MTL,1,0,1,0,0,2 +2009,21009,NYI,1,0,1,1,0,3 +2009,21010,CGY,1,1,3,0,1,6 +2009,21010,DET,2,2,1,0,0,5 +2009,21011,NSH,2,0,0,0,0,2 +2009,21011,NYR,1,2,1,0,0,4 +2009,21012,SJS,1,0,0,0,0,1 +2009,21012,STL,1,1,1,0,0,3 +2009,21013,CAR,1,0,1,0,0,2 +2009,21013,DAL,0,3,0,0,0,3 +2009,21014,COL,0,1,0,0,1,2 +2009,21014,MIN,1,0,0,0,0,1 +2009,21015,ATL,2,1,0,1,0,4 +2009,21015,EDM,1,2,0,0,0,3 +2009,21016,CHI,0,2,1,0,0,3 +2009,21016,CBJ,2,1,2,0,0,5 +2009,21017,LAK,0,0,2,0,0,2 +2009,21017,VAN,2,0,2,0,0,4 +2009,21018,BOS,2,0,0,0,0,2 +2009,21018,NYI,0,1,0,0,0,1 +2009,21019,NYR,1,0,1,0,0,2 +2009,21019,PHI,1,1,2,0,0,4 +2009,21020,DET,0,4,1,0,0,5 +2009,21020,STL,1,1,0,0,0,2 +2009,21021,OTT,1,0,2,0,1,4 +2009,21021,PIT,1,2,0,0,0,3 +2009,21022,ATL,1,0,2,0,1,4 +2009,21022,BUF,0,3,0,0,0,3 +2009,21023,CGY,3,1,2,0,0,6 +2009,21023,TOR,3,2,3,0,0,8 +2009,21024,MTL,1,0,0,0,0,1 +2009,21024,NJD,2,0,1,0,0,3 +2009,21025,CAR,1,2,1,0,0,4 +2009,21025,WSH,2,2,0,0,1,5 +2009,21026,FLA,1,1,1,0,0,3 +2009,21026,TBL,0,1,2,0,1,4 +2009,21027,DAL,1,1,0,1,0,3 +2009,21027,MIN,0,2,0,0,0,2 +2009,21028,COL,1,0,1,1,0,3 +2009,21028,EDM,1,0,1,0,0,2 +2009,21029,NSH,0,1,1,0,0,2 +2009,21029,PHX,0,0,0,0,0,0 +2009,21030,LAK,0,1,0,0,0,1 +2009,21030,SJS,1,0,0,0,1,2 +2009,21031,NYR,1,1,2,0,0,4 +2009,21031,PHI,0,1,0,0,0,1 +2009,21032,BOS,2,1,1,0,0,4 +2009,21032,PIT,2,0,4,0,0,6 +2009,21033,CHI,0,1,1,0,0,2 +2009,21033,NYI,0,3,1,0,0,4 +2009,21034,CBJ,0,0,0,0,0,0 +2009,21034,DET,2,1,1,0,0,4 +2009,21035,MIN,0,1,2,0,0,3 +2009,21035,STL,1,3,1,0,0,5 +2009,21036,ANA,0,0,0,0,0,0 +2009,21036,SJS,0,1,0,0,0,1 +2009,21037,COL,0,1,1,0,0,2 +2009,21037,VAN,3,0,1,0,0,4 +2009,21038,ATL,2,2,1,0,0,5 +2009,21038,WSH,0,0,1,0,0,1 +2009,21039,LAK,1,1,1,0,0,3 +2009,21039,NSH,1,2,1,0,0,4 +2009,21040,CHI,0,1,1,0,0,2 +2009,21040,NJD,2,1,0,0,0,3 +2009,21041,MTL,0,1,2,0,0,3 +2009,21041,NYR,0,1,2,0,1,4 +2009,21042,BUF,2,0,0,0,0,2 +2009,21042,OTT,0,2,2,0,0,4 +2009,21043,ATL,1,1,0,0,0,2 +2009,21043,PIT,2,3,1,0,0,6 +2009,21044,TBL,1,2,0,0,0,3 +2009,21044,TOR,0,1,2,0,1,4 +2009,21045,FLA,0,0,0,0,0,0 +2009,21045,WSH,0,1,2,0,0,3 +2009,21046,DET,0,1,2,0,0,3 +2009,21046,PHI,1,1,0,0,0,2 +2009,21047,COL,0,2,0,0,0,2 +2009,21047,MIN,0,1,1,0,1,3 +2009,21048,EDM,1,0,0,0,1,2 +2009,21048,STL,0,1,0,0,0,1 +2009,21049,DAL,1,0,1,0,0,2 +2009,21049,VAN,1,2,1,0,0,4 +2009,21050,PHX,3,0,1,0,0,4 +2009,21050,SJS,0,1,2,0,0,3 +2009,21051,CAR,1,2,1,0,0,4 +2009,21051,NJD,0,1,1,0,0,2 +2009,21052,CHI,2,1,0,0,0,3 +2009,21052,CBJ,2,1,0,1,0,4 +2009,21053,CGY,0,1,1,0,0,2 +2009,21053,DAL,0,1,0,0,0,1 +2009,21054,ANA,1,1,1,1,0,4 +2009,21054,NSH,1,1,1,0,0,3 +2009,21055,BOS,1,1,0,0,0,2 +2009,21055,LAK,0,0,2,1,0,3 +2009,21056,MTL,1,3,0,0,0,4 +2009,21056,OTT,3,2,0,0,0,5 +2009,21057,TBL,1,1,0,0,0,2 +2009,21057,WSH,2,0,3,0,0,5 +2009,21058,FLA,1,2,0,0,0,3 +2009,21058,TOR,1,0,0,0,0,1 +2009,21059,COL,1,0,0,0,0,1 +2009,21059,EDM,2,2,4,0,0,8 +2009,21060,STL,0,0,0,0,0,0 +2009,21060,VAN,0,1,2,0,0,3 +2009,21061,ANA,1,0,1,0,1,3 +2009,21061,PHX,1,0,1,0,0,2 +2009,21062,NSH,2,0,0,0,0,2 +2009,21062,SJS,2,0,0,0,1,3 +2009,21063,MIN,0,0,0,0,0,0 +2009,21063,NJD,0,3,1,0,0,4 +2009,21064,CAR,2,1,2,0,0,5 +2009,21064,NYI,0,4,0,0,0,4 +2009,21065,BUF,0,3,1,0,0,4 +2009,21065,PHI,1,3,2,0,0,6 +2009,21066,LAK,0,0,1,0,0,1 +2009,21066,PIT,2,2,0,0,0,4 +2009,21067,ATL,0,0,3,0,0,3 +2009,21067,DET,1,2,3,0,0,6 +2009,21068,CHI,2,1,1,0,0,4 +2009,21068,EDM,2,2,0,0,1,5 +2009,21069,CGY,0,2,0,0,0,2 +2009,21069,STL,1,0,2,0,0,3 +2009,21070,DAL,1,0,1,0,0,2 +2009,21070,SJS,2,2,1,0,0,5 +2009,21071,MTL,0,2,0,0,0,2 +2009,21071,TOR,2,2,1,0,0,5 +2009,21072,NYI,0,0,2,0,0,2 +2009,21072,OTT,0,4,1,0,0,5 +2009,21073,BUF,0,1,2,0,0,3 +2009,21073,NYR,1,3,1,0,0,5 +2009,21074,CAR,0,1,3,0,0,4 +2009,21074,WSH,0,1,0,0,0,1 +2009,21075,CBJ,0,0,3,0,0,3 +2009,21075,FLA,1,0,0,0,0,1 +2009,21076,ATL,2,1,0,0,1,4 +2009,21076,TBL,0,3,0,0,0,3 +2009,21077,PHX,0,0,0,0,0,0 +2009,21077,VAN,0,0,0,0,0,0 +2009,21078,BOS,1,3,0,0,0,4 +2009,21078,NJD,0,1,0,0,0,1 +2009,21079,PHI,1,1,1,0,0,3 +2009,21079,PIT,0,0,1,0,0,1 +2009,21080,CHI,1,1,2,0,0,4 +2009,21080,LAK,0,0,1,0,0,1 +2009,21081,EDM,0,0,0,0,0,0 +2009,21081,MIN,0,1,2,0,0,3 +2009,21082,NYR,1,0,0,0,0,1 +2009,21082,OTT,0,1,1,0,0,2 +2009,21083,ANA,2,1,3,0,0,6 +2009,21083,PHX,0,2,0,0,0,2 +2009,21084,COL,0,1,0,0,0,1 +2009,21084,SJS,0,2,1,0,0,3 +2009,21085,NJD,1,1,0,0,0,2 +2009,21085,PHI,1,1,2,0,0,4 +2009,21086,CAR,1,1,0,1,0,3 +2009,21086,FLA,1,0,1,0,0,2 +2009,21087,CGY,1,2,2,0,0,5 +2009,21087,DET,0,1,2,0,0,3 +2009,21088,TOR,0,1,1,0,1,3 +2009,21088,WSH,0,1,1,0,0,2 +2009,21089,MIN,0,1,0,0,0,1 +2009,21089,NYR,1,1,0,0,0,2 +2009,21090,ATL,1,1,1,0,0,3 +2009,21090,MTL,1,4,1,0,0,6 +2009,21091,CBJ,0,0,1,0,0,1 +2009,21091,TBL,0,0,1,1,0,2 +2009,21092,ANA,0,1,0,0,1,2 +2009,21092,NSH,0,0,1,0,0,1 +2009,21093,LAK,0,0,0,0,0,0 +2009,21093,STL,0,0,2,0,0,2 +2009,21094,DAL,1,0,1,0,0,2 +2009,21094,VAN,2,1,2,0,0,5 +2009,21095,DET,0,0,3,0,0,3 +2009,21095,EDM,1,0,1,0,0,2 +2009,21096,BUF,1,0,4,0,0,5 +2009,21096,FLA,1,1,1,0,0,3 +2009,21097,MIN,2,1,3,0,0,6 +2009,21097,NYI,1,0,1,0,0,2 +2009,21098,CAR,0,0,2,0,0,2 +2009,21098,OTT,0,0,1,0,0,1 +2009,21099,CGY,0,0,0,0,0,0 +2009,21099,PIT,1,0,1,0,0,2 +2009,21100,CHI,3,1,1,0,1,6 +2009,21100,SJS,1,2,2,0,0,5 +2009,21101,ANA,3,4,0,0,0,7 +2009,21101,COL,0,1,1,0,0,2 +2009,21102,FLA,1,1,2,0,0,4 +2009,21102,PHI,0,2,0,0,0,2 +2009,21103,ATL,0,2,2,0,1,5 +2009,21103,NYR,1,3,0,0,0,4 +2009,21104,CGY,0,0,0,0,0,0 +2009,21104,CBJ,2,2,1,0,0,5 +2009,21105,MTL,1,0,1,1,0,3 +2009,21105,TBL,0,0,2,0,0,2 +2009,21106,NSH,0,3,0,0,0,3 +2009,21106,SJS,1,1,0,0,0,2 +2009,21107,STL,0,2,2,0,0,4 +2009,21107,VAN,1,1,0,0,0,2 +2009,21108,DAL,0,0,0,0,0,0 +2009,21108,LAK,0,0,0,0,1,1 +2009,21109,EDM,0,0,2,0,0,2 +2009,21109,PHX,1,2,0,0,0,3 +2009,21110,TBL,1,1,1,0,0,3 +2009,21110,WSH,3,0,2,0,0,5 +2009,21111,BUF,2,2,1,0,0,5 +2009,21111,TOR,0,1,2,0,0,3 +2009,21112,DET,0,0,0,0,0,0 +2009,21112,NYI,0,2,0,0,0,2 +2009,21113,CHI,1,1,0,1,0,3 +2009,21113,NJD,2,0,0,0,0,2 +2009,21114,COL,1,0,0,0,0,1 +2009,21114,VAN,0,1,3,0,0,4 +2009,21115,ANA,1,1,1,0,0,3 +2009,21115,EDM,2,2,1,0,0,5 +2009,21116,NYR,2,1,0,0,0,3 +2009,21116,PIT,2,1,1,0,0,4 +2009,21117,BOS,3,4,0,0,0,7 +2009,21117,TOR,3,1,1,0,0,5 +2009,21118,BUF,1,1,1,0,1,4 +2009,21118,MTL,0,3,0,0,0,3 +2009,21119,CAR,0,0,2,0,0,2 +2009,21119,NJD,1,0,0,0,0,1 +2009,21120,NYI,1,1,1,0,0,3 +2009,21120,PHI,0,0,3,0,1,4 +2009,21121,ATL,2,3,1,0,0,6 +2009,21121,OTT,2,0,1,0,0,3 +2009,21122,LAK,1,1,1,0,0,3 +2009,21122,NSH,1,0,2,1,0,4 +2009,21123,DAL,1,1,1,0,0,3 +2009,21123,FLA,0,3,3,0,0,6 +2009,21124,CBJ,1,0,2,0,0,3 +2009,21124,STL,2,1,0,0,1,4 +2009,21125,CGY,2,1,0,0,0,3 +2009,21125,MIN,1,1,0,0,0,2 +2009,21126,PHX,0,2,0,0,0,2 +2009,21126,SJS,0,2,1,0,0,3 +2009,21127,DET,1,1,1,0,0,3 +2009,21127,NSH,1,0,3,0,0,4 +2009,21128,CBJ,0,1,1,0,0,2 +2009,21128,STL,1,1,3,0,0,5 +2009,21129,OTT,2,0,1,0,0,3 +2009,21129,TBL,0,0,0,0,0,0 +2009,21130,BOS,3,0,1,0,0,4 +2009,21130,PHI,1,1,1,0,0,3 +2009,21131,CHI,0,0,0,0,0,0 +2009,21131,VAN,1,1,2,0,0,4 +2009,21132,EDM,0,0,2,0,0,2 +2009,21132,MIN,2,0,1,0,0,3 +2009,21133,ANA,1,1,2,0,0,4 +2009,21133,COL,1,0,0,0,0,1 +2009,21134,NJD,0,0,0,0,0,0 +2009,21134,NYR,0,3,0,0,0,3 +2009,21135,CGY,1,0,0,0,0,1 +2009,21135,SJS,1,1,0,0,0,2 +2009,21136,DAL,1,2,2,0,0,5 +2009,21136,PHX,2,2,1,1,0,6 +2009,21137,BOS,1,2,0,0,0,3 +2009,21137,TBL,0,0,1,0,0,1 +2009,21138,CBJ,1,0,1,0,0,2 +2009,21138,NSH,1,0,0,0,0,1 +2009,21139,MTL,1,2,1,0,0,4 +2009,21139,CHI,0,0,1,0,0,1 +2009,21140,FLA,0,3,2,0,0,5 +2009,21140,OTT,0,1,1,0,0,2 +2009,21141,MIN,0,0,1,0,0,1 +2009,21141,VAN,0,1,0,1,0,2 +2009,21142,ANA,1,3,1,0,0,5 +2009,21142,EDM,1,0,2,0,0,3 +2009,21143,DAL,1,2,0,0,0,3 +2009,21143,LAK,1,0,1,0,0,2 +2009,21144,NYI,1,1,1,0,0,3 +2009,21144,WSH,0,2,3,0,0,5 +2009,21145,ATL,1,1,0,1,0,3 +2009,21145,BUF,1,1,0,0,0,2 +2009,21146,PHI,1,0,1,0,0,2 +2009,21146,TOR,1,2,0,0,0,3 +2009,21147,NJD,1,0,0,0,0,1 +2009,21147,PIT,3,2,1,0,0,6 +2009,21148,CHI,2,0,1,0,0,3 +2009,21148,STL,0,0,1,0,0,1 +2009,21149,COL,0,0,0,0,0,0 +2009,21149,PHX,1,0,2,0,0,3 +2009,21150,BOS,0,1,1,0,0,2 +2009,21150,OTT,1,0,0,0,0,1 +2009,21151,MTL,2,2,1,0,0,5 +2009,21151,NYI,0,1,0,0,0,1 +2009,21152,CAR,1,1,2,0,0,4 +2009,21152,NYR,1,1,0,0,0,2 +2009,21153,DET,0,1,3,0,0,4 +2009,21153,STL,0,2,3,0,0,5 +2009,21154,CGY,1,1,0,0,0,2 +2009,21154,DAL,1,0,0,0,0,1 +2009,21155,EDM,0,1,0,0,0,1 +2009,21155,SJS,1,1,0,0,0,2 +2009,21156,ANA,2,3,0,0,1,6 +2009,21156,VAN,2,3,0,0,0,5 +2009,21157,LAK,1,0,0,0,0,1 +2009,21157,PHX,0,0,2,0,0,2 +2009,21158,NJD,0,3,1,1,0,5 +2009,21158,TBL,1,1,2,0,0,4 +2009,21159,PHI,5,1,2,0,0,8 +2009,21159,TOR,0,3,2,0,0,5 +2009,21160,BUF,2,1,1,1,0,5 +2009,21160,WSH,1,3,0,0,0,4 +2009,21161,ATL,0,1,2,0,0,3 +2009,21161,FLA,0,1,0,0,0,1 +2009,21162,CGY,0,0,0,0,0,0 +2009,21162,MIN,3,0,1,0,0,4 +2009,21163,CHI,1,1,1,0,0,3 +2009,21163,NSH,1,0,0,0,0,1 +2009,21164,BOS,1,0,0,0,0,1 +2009,21164,NYR,0,0,0,0,0,0 +2009,21165,BUF,0,0,2,0,0,2 +2009,21165,NJD,2,1,0,0,0,3 +2009,21166,MTL,2,2,2,0,0,6 +2009,21166,TOR,0,0,2,0,0,2 +2009,21167,OTT,1,1,1,0,1,4 +2009,21167,PHI,0,2,1,0,0,3 +2009,21168,NYI,1,1,1,0,0,3 +2009,21168,TBL,0,1,0,0,0,1 +2009,21169,CAR,1,1,0,1,0,3 +2009,21169,PIT,1,0,1,0,0,2 +2009,21170,CBJ,2,1,1,0,0,4 +2009,21170,NSH,1,2,1,0,1,5 +2009,21171,DAL,1,1,2,1,0,5 +2009,21171,STL,0,3,1,0,0,4 +2009,21172,EDM,1,1,3,0,0,5 +2009,21172,VAN,0,0,3,0,0,3 +2009,21173,ANA,2,2,1,0,0,5 +2009,21173,SJS,1,1,0,0,0,2 +2009,21174,LAK,2,1,3,0,0,6 +2009,21174,PHX,0,1,0,0,0,1 +2009,21175,ATL,1,2,1,0,0,4 +2009,21175,WSH,1,1,4,0,0,6 +2009,21176,FLA,2,1,1,0,0,4 +2009,21176,PIT,1,0,1,0,0,2 +2009,21177,DET,2,0,1,0,0,3 +2009,21177,MIN,1,1,0,0,0,2 +2009,21178,CHI,0,0,0,1,0,1 +2009,21178,CBJ,0,0,0,0,0,0 +2009,21179,ANA,1,1,0,0,0,2 +2009,21179,SJS,0,2,1,0,0,3 +2009,21180,COL,1,1,2,0,0,4 +2009,21180,VAN,0,0,1,0,0,1 +2009,21181,BUF,0,0,1,0,0,1 +2009,21181,DET,0,1,3,0,0,4 +2009,21182,MTL,0,2,0,0,0,2 +2009,21182,OTT,0,1,2,0,0,3 +2009,21183,CGY,0,2,2,0,0,4 +2009,21183,LAK,0,1,0,0,0,1 +2009,21184,NJD,0,1,0,0,0,1 +2009,21184,TOR,3,0,1,0,0,4 +2009,21185,MTL,1,0,0,0,0,1 +2009,21185,NYR,1,2,0,0,0,3 +2009,21186,FLA,0,1,0,0,0,1 +2009,21186,PHI,0,1,1,0,0,2 +2009,21187,CAR,3,3,3,0,0,9 +2009,21187,NYI,0,0,0,0,0,0 +2009,21188,ATL,1,1,0,0,0,2 +2009,21188,WSH,3,0,1,0,0,4 +2009,21189,BOS,0,2,0,0,0,2 +2009,21189,OTT,2,1,0,0,0,3 +2009,21190,PIT,0,3,3,0,0,6 +2009,21190,TBL,0,0,4,0,0,4 +2009,21191,CHI,0,3,1,0,0,4 +2009,21191,NSH,1,0,1,0,0,2 +2009,21192,DAL,0,0,1,0,0,1 +2009,21192,MIN,1,2,0,0,0,3 +2009,21193,EDM,0,1,0,0,0,1 +2009,21193,LAK,2,0,0,0,0,2 +2009,21194,CGY,0,1,0,0,0,1 +2009,21194,VAN,1,1,2,0,0,4 +2009,21195,PHX,1,0,0,0,0,1 +2009,21195,STL,1,3,1,0,0,5 +2009,21196,COL,0,0,0,0,0,0 +2009,21196,SJS,0,0,0,0,1,1 +2009,21197,BUF,0,1,2,0,0,3 +2009,21197,TOR,1,0,0,0,0,1 +2009,21198,CHI,2,1,0,0,0,3 +2009,21198,CBJ,0,2,1,0,1,4 +2009,21199,BOS,1,2,1,1,0,5 +2009,21199,MTL,0,4,0,0,0,4 +2009,21200,NYR,2,0,0,0,0,2 +2009,21200,PHI,0,1,0,0,0,1 +2009,21201,BUF,1,3,1,0,0,5 +2009,21201,CAR,0,0,1,0,0,1 +2009,21202,ATL,1,0,1,0,0,2 +2009,21202,FLA,0,1,2,0,0,3 +2009,21203,NJD,0,2,0,0,1,3 +2009,21203,OTT,1,0,1,0,0,2 +2009,21204,NYI,1,0,0,0,0,1 +2009,21204,PIT,1,2,3,0,0,6 +2009,21205,TBL,1,0,1,0,0,2 +2009,21205,WSH,1,2,1,0,0,4 +2009,21206,DET,1,1,1,0,0,3 +2009,21206,NSH,0,1,2,0,1,4 +2009,21207,COL,0,1,1,0,0,2 +2009,21207,DAL,0,0,2,0,1,3 +2009,21208,LAK,0,0,0,0,0,0 +2009,21208,VAN,0,1,0,0,0,1 +2009,21209,PHX,2,1,1,0,0,4 +2009,21209,SJS,0,0,1,0,0,1 +2009,21210,MIN,2,3,3,0,0,8 +2009,21210,NSH,1,1,2,0,0,4 +2009,21211,CBJ,0,1,0,0,0,1 +2009,21211,STL,2,1,0,0,0,3 +2009,21212,CGY,0,0,1,0,0,1 +2009,21212,EDM,0,2,3,0,0,5 +2009,21213,ANA,1,0,2,0,1,4 +2009,21213,DAL,0,1,2,0,0,3 +2009,21214,CAR,1,0,1,0,0,2 +2009,21214,NJD,2,0,1,0,0,3 +2009,21215,NYI,1,1,0,0,0,2 +2009,21215,PHI,0,3,0,0,0,3 +2009,21216,CHI,1,0,3,0,0,4 +2009,21216,DET,1,1,0,0,0,2 +2009,21217,COL,0,0,0,0,0,0 +2009,21217,VAN,0,0,0,1,0,1 +2009,21218,BOS,0,0,1,0,0,1 +2009,21218,BUF,4,1,1,0,0,6 +2009,21219,LAK,2,1,1,0,0,4 +2009,21219,SJS,1,1,1,0,0,3 +2009,21220,OTT,0,2,0,0,0,2 +2009,21220,TOR,1,1,3,0,0,5 +2009,21221,MTL,1,0,0,0,0,1 +2009,21221,PIT,1,0,2,0,0,3 +2009,21222,ATL,2,3,1,0,0,6 +2009,21222,TBL,1,0,1,0,0,2 +2009,21223,FLA,1,2,4,0,0,7 +2009,21223,WSH,1,1,2,0,0,4 +2009,21224,CBJ,2,0,1,0,0,3 +2009,21224,MIN,0,3,3,0,0,6 +2009,21225,ANA,0,2,2,0,0,4 +2009,21225,PHX,1,3,0,0,1,5 +2009,21226,CGY,0,2,2,0,0,4 +2009,21226,EDM,0,1,0,0,0,1 +2009,21227,COL,0,0,0,0,0,0 +2009,21227,STL,1,0,0,0,0,1 +2009,21228,BOS,2,3,1,0,0,6 +2009,21228,NYI,0,1,1,0,0,2 +2009,21229,NYR,2,0,2,0,0,4 +2009,21229,PHI,2,1,0,0,0,3 +2009,21230,CHI,2,0,1,0,0,3 +2009,21230,DET,0,0,0,0,0,0 +2010,20001,BOS,0,0,1,0,0,1 +2010,20001,WSH,1,1,2,0,0,4 +2010,20002,MTL,1,1,1,1,0,4 +2010,20002,TOR,1,1,1,0,0,3 +2010,20003,COL,2,3,0,0,0,5 +2010,20003,SJS,1,1,0,0,0,2 +2010,20004,CGY,3,1,1,0,0,5 +2010,20004,VAN,0,2,1,0,0,3 +2010,20005,CAR,0,0,0,0,0,0 +2010,20005,PHI,0,2,0,0,0,2 +2010,20006,NYR,1,0,1,0,0,2 +2010,20006,PIT,1,2,0,0,0,3 +2010,20007,CHI,0,1,2,0,0,3 +2010,20007,FLA,0,1,2,0,1,4 +2010,20008,DET,2,1,0,0,0,3 +2010,20008,STL,1,3,0,0,0,4 +2010,20009,COL,1,1,1,0,0,3 +2010,20009,VAN,0,0,0,0,0,0 +2010,20010,BOS,3,3,1,0,0,7 +2010,20010,CAR,0,1,1,0,0,2 +2010,20011,BUF,1,0,0,0,0,1 +2010,20011,MTL,0,1,0,1,0,2 +2010,20012,NJD,0,0,2,0,0,2 +2010,20012,PHI,1,2,2,0,0,5 +2010,20013,NYI,1,1,1,0,0,3 +2010,20013,PIT,1,0,2,0,1,4 +2010,20014,NYR,0,3,2,0,0,5 +2010,20014,OTT,0,1,1,0,0,2 +2010,20015,TOR,0,1,3,0,0,4 +2010,20015,WSH,3,3,0,0,0,6 +2010,20016,ATL,2,2,2,0,0,6 +2010,20016,TBL,0,2,1,0,0,3 +2010,20017,CHI,1,2,1,0,0,4 +2010,20017,FLA,0,0,0,0,0,0 +2010,20018,CBJ,0,1,1,0,0,2 +2010,20018,MIN,0,0,1,0,0,1 +2010,20019,DAL,0,1,1,0,0,2 +2010,20019,NSH,1,1,0,0,1,3 +2010,20020,DET,2,1,0,0,0,3 +2010,20020,STL,1,3,1,0,0,5 +2010,20021,CGY,2,1,1,0,0,4 +2010,20021,EDM,1,1,1,0,0,3 +2010,20022,ANA,0,0,1,0,0,1 +2010,20022,SJS,3,1,0,0,0,4 +2010,20023,LAK,0,2,1,0,0,3 +2010,20023,PHX,2,2,2,0,0,6 +2010,20024,NJD,2,0,0,0,0,2 +2010,20024,NYR,2,1,0,0,0,3 +2010,20025,CBJ,1,3,1,0,0,5 +2010,20025,VAN,1,1,1,0,0,3 +2010,20026,OTT,0,2,0,0,0,2 +2010,20026,TOR,0,0,1,0,0,1 +2010,20027,PHI,0,4,1,1,0,6 +2010,20027,WSH,0,3,2,0,0,5 +2010,20028,CAR,1,0,0,0,1,2 +2010,20028,TBL,0,1,0,0,0,1 +2010,20029,ANA,0,3,0,0,0,3 +2010,20029,MIN,0,0,3,1,0,4 +2010,20030,CGY,1,2,1,0,0,4 +2010,20030,MTL,2,1,0,0,0,3 +2010,20031,DAL,1,2,1,0,0,4 +2010,20031,EDM,1,2,1,0,1,5 +2010,20032,LAK,1,3,2,0,0,6 +2010,20032,SJS,0,2,2,0,0,4 +2010,20033,PHX,1,1,1,0,0,3 +2010,20033,PIT,0,0,0,0,0,0 +2010,20034,MTL,0,1,0,0,0,1 +2010,20034,VAN,3,2,2,0,0,7 +2010,20035,ANA,0,3,3,0,0,6 +2010,20035,BOS,1,0,0,0,0,1 +2010,20036,BUF,0,0,2,0,0,2 +2010,20036,PHX,0,0,1,0,0,1 +2010,20037,NYI,0,1,1,0,0,2 +2010,20037,OTT,1,1,0,1,0,3 +2010,20038,PHI,1,2,1,0,0,4 +2010,20038,PIT,2,2,1,0,0,5 +2010,20039,NYR,0,2,2,0,0,4 +2010,20039,WSH,1,0,2,0,0,3 +2010,20040,NJD,1,1,1,0,1,4 +2010,20040,TBL,0,1,2,0,0,3 +2010,20041,CHI,0,1,1,0,0,2 +2010,20041,DET,0,3,0,0,0,3 +2010,20042,ATL,2,1,1,0,0,4 +2010,20042,STL,1,0,1,0,0,2 +2010,20043,COL,0,1,1,0,0,2 +2010,20043,NSH,0,1,2,0,0,3 +2010,20044,CGY,0,1,2,0,1,4 +2010,20044,EDM,2,0,1,0,0,3 +2010,20045,LAK,3,1,2,0,0,6 +2010,20045,MIN,0,2,1,0,0,3 +2010,20046,CBJ,1,1,1,0,0,3 +2010,20046,SJS,0,4,2,0,0,6 +2010,20047,CAR,3,1,3,0,0,7 +2010,20047,FLA,0,1,1,0,0,2 +2010,20048,CGY,0,1,1,0,0,2 +2010,20048,DAL,1,2,2,0,0,5 +2010,20049,BOS,0,0,3,0,1,4 +2010,20049,NYI,0,3,0,0,0,3 +2010,20050,PIT,2,2,1,0,0,5 +2010,20050,TOR,0,1,1,0,0,2 +2010,20051,ATL,2,0,0,0,0,2 +2010,20051,OTT,2,1,1,0,0,4 +2010,20052,ANA,0,0,2,0,1,3 +2010,20052,PHI,1,1,0,0,0,2 +2010,20053,FLA,0,1,1,0,0,2 +2010,20053,NJD,0,2,1,0,0,3 +2010,20054,DET,1,1,1,0,0,3 +2010,20054,WSH,1,0,1,0,0,2 +2010,20055,CAR,0,2,0,0,0,2 +2010,20055,TBL,0,2,3,0,0,5 +2010,20056,LAK,1,1,0,0,0,2 +2010,20056,STL,0,0,1,0,0,1 +2010,20057,BUF,0,0,1,0,0,1 +2010,20057,NSH,0,0,0,0,0,0 +2010,20058,CHI,1,2,0,0,1,4 +2010,20058,COL,0,3,0,0,0,3 +2010,20059,CBJ,1,0,1,0,0,2 +2010,20059,PHX,0,0,0,0,0,0 +2010,20060,MTL,0,1,1,0,0,2 +2010,20060,EDM,1,2,0,0,0,3 +2010,20061,MIN,1,1,0,0,0,2 +2010,20061,SJS,0,3,1,0,0,4 +2010,20062,ANA,0,0,0,0,0,0 +2010,20062,NYR,0,0,3,0,0,3 +2010,20063,DAL,1,0,2,0,0,3 +2010,20063,VAN,1,1,1,0,1,4 +2010,20064,BOS,0,2,1,0,0,3 +2010,20064,COL,2,2,0,0,0,4 +2010,20065,LAK,0,1,1,0,0,2 +2010,20065,NYI,0,0,1,0,0,1 +2010,20066,NYR,2,1,4,0,0,7 +2010,20066,TOR,1,1,0,0,0,2 +2010,20067,NJD,0,1,1,0,1,3 +2010,20067,WSH,2,0,0,0,0,2 +2010,20068,OTT,1,0,0,0,0,1 +2010,20068,PIT,1,2,1,0,0,4 +2010,20069,FLA,1,1,0,0,0,2 +2010,20069,TBL,1,0,2,0,0,3 +2010,20070,EDM,4,2,0,0,0,6 +2010,20070,NSH,0,1,0,0,0,1 +2010,20071,CGY,5,0,0,0,0,5 +2010,20071,CHI,1,3,1,1,0,6 +2010,20072,PHX,0,0,0,0,1,1 +2010,20072,SJS,0,0,0,0,0,0 +2010,20073,BUF,1,4,1,0,0,6 +2010,20073,DET,1,0,1,0,0,2 +2010,20074,CGY,1,0,0,0,0,1 +2010,20074,CBJ,0,2,0,0,0,2 +2010,20075,COL,2,2,0,0,0,4 +2010,20075,TOR,0,1,0,0,0,1 +2010,20076,LAK,1,1,0,0,0,2 +2010,20076,NYR,2,1,1,0,0,4 +2010,20077,CHI,2,1,1,0,0,4 +2010,20077,EDM,1,0,2,0,0,3 +2010,20078,DAL,4,1,1,0,0,6 +2010,20078,NSH,0,0,0,0,0,0 +2010,20079,ANA,2,1,0,0,0,3 +2010,20079,MIN,0,1,1,0,0,2 +2010,20080,SJS,1,0,0,0,0,1 +2010,20080,WSH,1,2,1,0,0,4 +2010,20081,CAR,0,0,0,0,0,0 +2010,20081,PIT,0,0,0,0,0,0 +2010,20082,MTL,1,0,1,0,0,2 +2010,20082,COL,0,2,1,0,0,3 +2010,20083,OTT,2,3,2,0,0,7 +2010,20083,TBL,0,1,0,0,0,1 +2010,20084,DET,0,2,3,0,0,5 +2010,20084,LAK,1,0,1,0,0,2 +2010,20085,CHI,0,2,1,0,0,3 +2010,20085,NSH,0,1,0,0,0,1 +2010,20086,PHX,0,1,1,1,0,3 +2010,20086,STL,0,2,0,0,0,2 +2010,20087,ATL,0,1,3,0,0,4 +2010,20087,NJD,0,1,1,0,0,2 +2010,20088,BUF,2,2,2,0,0,6 +2010,20088,NYI,0,2,1,0,0,3 +2010,20089,FLA,0,2,2,0,0,4 +2010,20089,PHI,1,1,0,0,0,2 +2010,20090,BOS,1,2,0,0,0,3 +2010,20090,DAL,0,0,0,0,0,0 +2010,20091,CGY,2,3,0,0,0,5 +2010,20091,VAN,0,0,3,0,0,3 +2010,20092,EDM,0,3,2,0,0,5 +2010,20092,MIN,0,1,1,0,0,2 +2010,20093,ATL,2,1,1,0,0,4 +2010,20093,BUF,0,1,1,0,0,2 +2010,20094,NYR,1,2,1,0,0,4 +2010,20094,TOR,0,1,0,0,0,1 +2010,20095,MTL,1,0,0,0,0,1 +2010,20095,OTT,1,1,1,0,0,3 +2010,20096,CAR,0,0,0,0,0,0 +2010,20096,NJD,0,0,2,0,0,2 +2010,20097,NYI,1,0,0,0,0,1 +2010,20097,SJS,0,2,2,0,0,4 +2010,20098,NSH,0,2,0,0,0,2 +2010,20098,WSH,2,0,0,0,1,3 +2010,20099,COL,0,0,3,0,1,4 +2010,20099,DET,1,1,1,0,0,3 +2010,20100,CBJ,1,1,2,0,0,4 +2010,20100,LAK,1,0,0,0,0,1 +2010,20101,PIT,2,1,1,0,0,4 +2010,20101,TBL,0,1,0,0,0,1 +2010,20102,CHI,0,2,1,0,0,3 +2010,20102,DAL,0,3,1,0,0,4 +2010,20103,BOS,0,1,0,0,0,1 +2010,20103,PHX,1,3,0,0,0,4 +2010,20104,MIN,1,0,0,0,0,1 +2010,20104,VAN,0,1,1,0,0,2 +2010,20105,ANA,0,0,0,0,0,0 +2010,20105,STL,1,2,2,0,0,5 +2010,20106,NYR,2,0,1,0,0,3 +2010,20106,SJS,2,3,2,0,0,7 +2010,20107,DAL,1,0,0,0,0,1 +2010,20107,LAK,0,3,1,0,0,4 +2010,20108,EDM,0,1,1,0,0,2 +2010,20108,VAN,1,0,0,0,0,1 +2010,20109,PIT,2,2,1,0,0,5 +2010,20109,STL,0,1,0,0,0,1 +2010,20110,ATL,0,0,1,0,0,1 +2010,20110,MTL,1,0,0,0,1,2 +2010,20111,CGY,3,1,2,0,0,6 +2010,20111,CBJ,1,2,0,0,0,3 +2010,20112,BOS,0,2,1,0,0,3 +2010,20112,NSH,1,1,0,0,0,2 +2010,20113,CAR,0,1,2,0,0,3 +2010,20113,NYI,1,2,0,0,1,4 +2010,20114,BUF,5,0,0,0,0,5 +2010,20114,FLA,1,0,1,0,0,2 +2010,20115,COL,0,2,0,0,0,2 +2010,20115,MIN,1,0,1,0,1,3 +2010,20116,CHI,1,1,0,0,0,2 +2010,20116,VAN,0,1,2,0,0,3 +2010,20117,ANA,0,1,1,0,0,2 +2010,20117,DAL,1,2,1,0,0,4 +2010,20118,MTL,2,1,2,0,0,5 +2010,20118,NYI,0,0,1,0,0,1 +2010,20119,NJD,2,0,2,0,0,4 +2010,20119,NYR,0,2,0,0,0,2 +2010,20120,BOS,1,2,0,0,0,3 +2010,20120,PHI,1,2,0,0,1,4 +2010,20121,ATL,2,0,2,0,0,4 +2010,20121,WSH,2,3,0,0,0,5 +2010,20122,NSH,3,0,2,1,0,6 +2010,20122,OTT,0,0,5,0,0,5 +2010,20123,SJS,0,1,1,0,0,2 +2010,20123,TBL,2,1,2,0,0,5 +2010,20124,CBJ,2,2,0,0,0,4 +2010,20124,EDM,1,2,3,0,0,6 +2010,20125,DET,1,1,0,0,0,2 +2010,20125,PHX,1,0,1,1,0,3 +2010,20126,DAL,1,0,3,0,0,4 +2010,20126,LAK,2,2,0,1,0,5 +2010,20127,FLA,2,0,0,0,0,2 +2010,20127,PIT,0,0,2,0,1,3 +2010,20128,MIN,0,1,0,0,0,1 +2010,20128,STL,1,1,1,0,0,3 +2010,20129,CAR,2,1,1,0,0,4 +2010,20129,COL,2,2,1,0,0,5 +2010,20130,MTL,1,3,0,1,0,5 +2010,20130,NYR,3,1,0,0,0,4 +2010,20131,BOS,0,1,2,0,1,4 +2010,20131,OTT,0,1,2,0,0,3 +2010,20132,NYI,1,0,1,0,0,2 +2010,20132,WSH,0,0,2,1,0,3 +2010,20133,FLA,1,0,0,0,0,1 +2010,20133,PHI,1,2,2,0,0,5 +2010,20134,ATL,0,2,1,0,0,3 +2010,20134,SJS,2,2,0,0,0,4 +2010,20135,TOR,0,1,0,0,0,1 +2010,20135,VAN,2,0,1,0,0,3 +2010,20136,NJD,1,1,2,0,0,4 +2010,20136,PIT,0,0,1,0,0,1 +2010,20137,BUF,0,1,1,0,1,3 +2010,20137,TBL,1,0,1,0,0,2 +2010,20138,DAL,0,2,2,0,0,4 +2010,20138,STL,0,0,1,0,0,1 +2010,20139,CAR,1,1,0,0,0,2 +2010,20139,MIN,1,1,0,1,0,3 +2010,20140,CHI,0,2,0,0,0,2 +2010,20140,NSH,0,0,0,0,0,0 +2010,20141,COL,0,1,2,0,0,3 +2010,20141,DET,0,1,0,0,0,1 +2010,20142,LAK,0,2,3,0,0,5 +2010,20142,PHX,1,1,1,0,0,3 +2010,20143,CGY,1,3,1,0,0,5 +2010,20143,EDM,1,1,0,0,0,2 +2010,20144,ANA,1,1,2,0,0,4 +2010,20144,CBJ,0,4,2,0,0,6 +2010,20145,PHI,1,0,0,0,0,1 +2010,20145,SJS,1,2,1,0,0,4 +2010,20146,CBJ,0,1,1,0,0,2 +2010,20146,LAK,2,2,2,0,0,6 +2010,20147,EDM,0,0,0,0,0,0 +2010,20147,VAN,1,1,0,0,0,2 +2010,20148,NYR,2,2,1,0,0,5 +2010,20148,PHX,0,2,0,0,0,2 +2010,20149,MTL,1,0,1,1,0,3 +2010,20149,NYI,0,1,1,0,0,2 +2010,20150,CHI,2,0,1,0,0,3 +2010,20150,MIN,0,0,1,0,0,1 +2010,20151,ANA,1,1,1,0,0,3 +2010,20151,TOR,2,1,3,0,0,6 +2010,20152,PHI,0,2,0,0,0,2 +2010,20152,WSH,0,2,2,0,0,4 +2010,20153,COL,1,0,2,0,0,3 +2010,20153,EDM,0,0,0,0,0,0 +2010,20154,DET,1,0,4,0,0,5 +2010,20154,VAN,2,0,2,0,0,4 +2010,20155,BUF,2,0,2,0,0,4 +2010,20155,NJD,0,1,0,0,0,1 +2010,20156,NYI,1,1,1,0,0,3 +2010,20156,NYR,1,0,0,0,0,1 +2010,20157,CAR,1,0,1,0,0,2 +2010,20157,STL,3,1,1,0,0,5 +2010,20158,CBJ,1,0,0,0,0,1 +2010,20158,PHX,0,2,2,0,0,4 +2010,20159,MTL,0,0,1,0,0,1 +2010,20159,PIT,1,3,2,0,0,6 +2010,20160,FLA,1,1,1,0,0,3 +2010,20160,OTT,1,3,0,0,0,4 +2010,20161,MIN,0,3,0,0,0,3 +2010,20161,NSH,2,1,1,0,0,4 +2010,20162,DAL,0,1,2,1,0,4 +2010,20162,TOR,0,1,2,0,0,3 +2010,20163,CGY,2,0,0,0,0,2 +2010,20163,COL,2,0,1,0,0,3 +2010,20164,LAK,0,1,0,0,0,1 +2010,20164,SJS,0,1,0,0,1,2 +2010,20165,BOS,0,1,0,0,0,1 +2010,20165,NJD,1,0,1,0,0,2 +2010,20166,ATL,0,0,3,0,0,3 +2010,20166,WSH,2,1,1,0,0,4 +2010,20167,OTT,0,0,2,0,0,2 +2010,20167,TBL,1,2,2,0,0,5 +2010,20168,PHX,0,1,1,0,0,2 +2010,20168,STL,0,0,0,0,0,0 +2010,20169,CHI,0,0,0,0,0,0 +2010,20169,NSH,0,1,1,0,0,2 +2010,20170,DET,0,2,3,0,0,5 +2010,20170,EDM,3,2,0,0,1,6 +2010,20171,LAK,1,0,0,0,0,1 +2010,20171,VAN,1,0,0,0,1,2 +2010,20172,NYI,1,2,0,1,0,4 +2010,20172,WSH,1,2,0,0,0,3 +2010,20173,CBJ,1,2,0,0,0,3 +2010,20173,PIT,0,1,2,0,1,4 +2010,20174,BUF,1,1,0,1,0,3 +2010,20174,TOR,1,0,1,0,0,2 +2010,20175,MIN,1,2,0,0,0,3 +2010,20175,NYR,0,1,1,0,0,2 +2010,20176,MTL,0,2,0,0,0,2 +2010,20176,CHI,1,1,1,0,0,3 +2010,20177,DAL,2,2,1,0,0,5 +2010,20177,FLA,3,1,1,0,1,6 +2010,20178,ANA,1,3,3,0,0,7 +2010,20178,VAN,2,0,0,0,0,2 +2010,20179,COL,0,0,1,0,0,1 +2010,20179,SJS,0,1,2,0,0,3 +2010,20180,BOS,0,0,2,0,0,2 +2010,20180,EDM,0,0,0,0,0,0 +2010,20181,CAR,0,1,0,0,0,1 +2010,20181,PHI,3,1,2,0,0,6 +2010,20182,ATL,1,1,1,0,0,3 +2010,20182,OTT,0,0,1,0,0,1 +2010,20183,NJD,0,1,0,0,1,2 +2010,20183,TBL,0,0,1,0,0,1 +2010,20184,MTL,0,3,1,0,1,5 +2010,20184,TOR,0,2,2,0,0,4 +2010,20185,BUF,0,0,0,0,0,0 +2010,20185,NYI,1,3,1,0,0,5 +2010,20186,MIN,2,0,0,0,0,2 +2010,20186,PIT,1,0,0,0,0,1 +2010,20187,FLA,2,0,2,0,0,4 +2010,20187,STL,0,0,0,0,0,0 +2010,20188,DAL,1,1,0,0,0,2 +2010,20188,NSH,1,0,3,0,0,4 +2010,20189,ANA,0,1,1,0,0,2 +2010,20189,PHX,1,0,1,0,1,3 +2010,20190,CGY,0,1,0,0,0,1 +2010,20190,DET,0,2,1,0,0,3 +2010,20191,BOS,0,0,0,0,0,0 +2010,20191,NYR,0,1,0,0,0,1 +2010,20192,CAR,0,1,0,0,0,1 +2010,20192,SJS,0,3,2,0,0,5 +2010,20193,CBJ,0,2,2,1,0,5 +2010,20193,WSH,1,0,3,0,0,4 +2010,20194,COL,0,0,0,0,0,0 +2010,20194,VAN,1,1,1,0,0,3 +2010,20195,EDM,1,0,0,0,0,1 +2010,20195,NYI,1,2,0,0,0,3 +2010,20196,PHI,2,3,1,0,0,6 +2010,20196,TBL,0,1,1,0,0,2 +2010,20197,LAK,1,1,3,0,0,5 +2010,20197,PHX,1,2,0,0,0,3 +2010,20198,BOS,0,0,0,0,0,0 +2010,20198,DET,2,0,0,0,0,2 +2010,20199,TBL,0,1,0,1,0,2 +2010,20199,TOR,0,0,1,0,0,1 +2010,20200,ATL,2,1,2,0,0,5 +2010,20200,MTL,0,2,2,0,0,4 +2010,20201,NYR,0,0,1,0,0,1 +2010,20201,VAN,1,0,3,0,0,4 +2010,20202,ANA,1,1,1,0,0,3 +2010,20202,PIT,0,2,2,0,0,4 +2010,20203,BUF,1,1,1,0,0,3 +2010,20203,NYI,0,0,0,0,0,0 +2010,20204,NJD,0,1,2,0,0,3 +2010,20204,WSH,1,0,1,0,0,2 +2010,20205,CBJ,1,1,0,0,0,2 +2010,20205,SJS,1,1,0,0,1,3 +2010,20206,CAR,0,0,0,0,0,0 +2010,20206,FLA,0,0,3,0,0,3 +2010,20207,CGY,0,1,1,1,0,3 +2010,20207,DAL,0,0,2,0,0,2 +2010,20208,COL,1,0,3,0,0,4 +2010,20208,PHX,0,0,1,0,0,1 +2010,20209,BOS,0,0,1,0,0,1 +2010,20209,MTL,1,0,0,0,1,2 +2010,20210,ATL,0,2,1,0,0,3 +2010,20210,CBJ,2,2,0,0,0,4 +2010,20211,OTT,1,1,0,1,0,3 +2010,20211,TBL,1,1,0,0,0,2 +2010,20212,DET,0,0,1,0,1,2 +2010,20212,SJS,0,1,0,0,0,1 +2010,20213,CGY,1,0,0,1,0,2 +2010,20213,STL,0,0,1,0,0,1 +2010,20214,MIN,0,2,0,0,0,2 +2010,20214,VAN,1,3,1,0,0,5 +2010,20215,CHI,0,1,0,0,0,1 +2010,20215,PHX,2,1,0,0,0,3 +2010,20216,EDM,0,1,1,0,0,2 +2010,20216,NYR,1,2,1,0,0,4 +2010,20217,ANA,1,0,3,0,0,4 +2010,20217,NSH,0,0,0,0,0,0 +2010,20218,LAK,1,0,4,0,0,5 +2010,20218,PIT,1,1,0,0,0,2 +2010,20219,NJD,0,2,0,0,0,2 +2010,20219,NYI,1,0,0,0,0,1 +2010,20220,CAR,2,0,0,0,0,2 +2010,20220,TOR,0,2,1,0,0,3 +2010,20221,BUF,0,1,1,0,0,2 +2010,20221,PHI,1,2,2,0,0,5 +2010,20222,FLA,0,0,1,0,0,1 +2010,20222,WSH,1,1,2,0,0,4 +2010,20223,DAL,0,1,1,0,0,2 +2010,20223,VAN,0,0,1,0,0,1 +2010,20224,CHI,2,1,0,0,0,3 +2010,20224,COL,1,2,0,0,1,4 +2010,20225,LAK,0,0,1,0,0,1 +2010,20225,NSH,0,1,2,0,0,3 +2010,20226,BOS,2,2,0,0,0,4 +2010,20226,BUF,0,1,1,0,0,2 +2010,20227,DET,0,0,1,0,0,1 +2010,20227,TOR,2,1,2,0,0,5 +2010,20228,MTL,0,1,0,0,0,1 +2010,20228,TBL,2,1,0,0,0,3 +2010,20229,NJD,0,1,2,0,0,3 +2010,20229,OTT,1,1,0,0,0,2 +2010,20230,ATL,1,1,1,0,0,3 +2010,20230,NYI,3,2,1,0,0,6 +2010,20231,PHI,0,1,0,0,1,2 +2010,20231,STL,0,0,1,0,0,1 +2010,20232,FLA,0,3,1,0,0,4 +2010,20232,WSH,1,1,5,0,0,7 +2010,20233,CAR,1,0,1,0,0,2 +2010,20233,CBJ,0,0,3,0,0,3 +2010,20234,DAL,0,2,0,0,0,2 +2010,20234,MIN,2,1,0,0,0,3 +2010,20235,CGY,1,1,1,0,0,3 +2010,20235,NYR,1,0,0,0,0,1 +2010,20236,ANA,1,2,1,0,0,4 +2010,20236,PHX,0,0,3,0,0,3 +2010,20237,PIT,0,0,0,0,0,0 +2010,20237,SJS,2,2,1,0,0,5 +2010,20238,ATL,0,0,2,0,1,3 +2010,20238,STL,0,1,1,0,0,2 +2010,20239,COL,1,1,1,0,0,3 +2010,20239,EDM,3,0,2,0,0,5 +2010,20240,CHI,0,1,3,0,0,4 +2010,20240,LAK,0,1,0,0,0,1 +2010,20241,BOS,0,1,2,0,0,3 +2010,20241,PIT,0,0,0,0,0,0 +2010,20242,MIN,1,3,1,0,0,5 +2010,20242,TOR,1,0,1,0,0,2 +2010,20243,CGY,1,0,0,0,0,1 +2010,20243,MTL,0,0,0,0,0,0 +2010,20244,EDM,2,0,1,0,0,3 +2010,20244,OTT,1,1,1,0,1,4 +2010,20245,STL,4,1,1,0,0,6 +2010,20245,VAN,0,0,1,0,0,1 +2010,20246,NSH,1,1,1,0,0,3 +2010,20246,SJS,1,1,2,0,0,4 +2010,20247,BUF,1,1,1,0,0,3 +2010,20247,EDM,0,1,0,0,0,1 +2010,20248,ANA,0,1,0,0,0,1 +2010,20248,NJD,1,2,0,0,0,3 +2010,20249,NYI,3,0,1,0,0,4 +2010,20249,WSH,2,2,0,0,1,5 +2010,20250,CAR,0,1,1,0,0,2 +2010,20250,LAK,1,1,3,0,0,5 +2010,20251,CBJ,0,1,0,0,0,1 +2010,20251,DET,4,2,3,0,0,9 +2010,20252,CHI,1,0,1,0,1,3 +2010,20252,COL,1,1,0,0,0,2 +2010,20253,BOS,0,0,0,0,0,0 +2010,20253,FLA,0,0,0,0,1,1 +2010,20254,ATL,1,3,1,0,0,5 +2010,20254,NYR,1,1,1,0,0,3 +2010,20255,OTT,0,1,0,0,0,1 +2010,20255,PHI,0,2,3,0,0,5 +2010,20256,NJD,0,2,2,0,0,4 +2010,20256,PIT,1,0,0,0,0,1 +2010,20257,MIN,1,2,0,0,0,3 +2010,20257,TBL,0,1,2,0,1,4 +2010,20258,DET,1,0,2,0,0,3 +2010,20258,VAN,0,1,0,0,0,1 +2010,20259,NSH,0,1,2,0,0,3 +2010,20259,STL,1,0,0,0,0,1 +2010,20260,MTL,0,1,3,0,0,4 +2010,20260,PHX,0,1,1,0,0,2 +2010,20261,DAL,0,0,2,0,1,3 +2010,20261,SJS,0,2,0,0,0,2 +2010,20262,MIN,0,1,0,0,0,1 +2010,20262,WSH,0,1,2,0,0,3 +2010,20263,CAR,0,2,1,0,0,3 +2010,20263,NYI,3,0,0,1,0,4 +2010,20264,ANA,1,1,0,0,0,2 +2010,20264,CBJ,1,1,0,0,1,3 +2010,20265,BUF,1,0,0,0,1,2 +2010,20265,CGY,1,0,0,0,0,1 +2010,20266,ATL,0,4,3,0,0,7 +2010,20266,LAK,0,0,0,0,0,0 +2010,20267,CHI,2,1,0,0,0,3 +2010,20267,TOR,0,1,1,0,0,2 +2010,20268,CGY,3,0,2,0,0,5 +2010,20268,TOR,1,1,0,0,0,2 +2010,20269,NYR,0,1,0,0,1,2 +2010,20269,OTT,0,1,0,0,0,1 +2010,20270,NJD,1,3,1,0,0,5 +2010,20270,WSH,2,0,0,0,0,2 +2010,20271,BUF,1,1,1,0,0,3 +2010,20271,PHI,0,0,2,0,0,2 +2010,20272,FLA,0,3,1,0,1,5 +2010,20272,NYI,2,0,2,0,0,4 +2010,20273,ANA,0,1,3,0,0,4 +2010,20273,DET,2,0,5,0,0,7 +2010,20274,BOS,1,2,2,0,0,5 +2010,20274,PIT,2,1,2,1,0,6 +2010,20275,LAK,0,0,1,0,1,2 +2010,20275,TBL,0,0,1,0,0,1 +2010,20276,SJS,0,2,1,0,0,3 +2010,20276,STL,1,0,0,0,0,1 +2010,20277,MTL,0,0,0,0,0,0 +2010,20277,NSH,1,0,1,0,0,2 +2010,20278,DAL,0,1,1,0,0,2 +2010,20278,PHX,0,2,1,0,0,3 +2010,20279,COL,0,0,2,0,0,2 +2010,20279,VAN,3,1,4,0,0,8 +2010,20280,CAR,2,2,0,0,1,5 +2010,20280,MIN,0,3,1,0,0,4 +2010,20281,ATL,1,1,1,0,0,3 +2010,20281,EDM,0,2,0,0,0,2 +2010,20282,CHI,1,2,0,1,0,4 +2010,20282,SJS,1,2,0,0,0,3 +2010,20283,BOS,0,1,0,0,0,1 +2010,20283,NYI,1,1,2,0,0,4 +2010,20284,NJD,0,1,1,0,0,2 +2010,20284,PHI,1,1,1,0,0,3 +2010,20285,CBJ,0,1,1,0,1,3 +2010,20285,EDM,0,1,1,0,0,2 +2010,20286,ANA,1,0,1,0,0,2 +2010,20286,PIT,2,1,2,0,0,5 +2010,20287,FLA,0,2,1,0,0,3 +2010,20287,LAK,0,1,2,0,1,4 +2010,20288,PHX,1,0,0,0,0,1 +2010,20288,TBL,2,1,1,0,0,4 +2010,20289,NYR,1,0,1,0,0,2 +2010,20289,WSH,1,1,2,0,0,4 +2010,20290,MTL,1,0,1,0,1,3 +2010,20290,CAR,0,2,0,0,0,2 +2010,20291,OTT,1,2,0,0,0,3 +2010,20291,TOR,1,1,0,0,0,2 +2010,20292,NSH,2,2,0,0,0,4 +2010,20292,SJS,1,1,1,0,0,3 +2010,20293,CGY,0,2,0,0,0,2 +2010,20293,COL,1,2,0,0,0,3 +2010,20294,BUF,1,0,1,0,0,2 +2010,20294,FLA,1,1,4,0,0,6 +2010,20295,DAL,1,1,1,0,0,3 +2010,20295,DET,0,1,0,0,0,1 +2010,20296,MIN,0,0,2,0,0,2 +2010,20296,PHX,0,1,2,0,0,3 +2010,20297,COL,2,2,0,0,0,4 +2010,20297,EDM,1,2,3,0,0,6 +2010,20298,LAK,1,0,1,0,0,2 +2010,20298,PHI,0,2,1,0,0,3 +2010,20299,CAR,0,2,3,0,1,6 +2010,20299,TOR,3,0,2,0,0,5 +2010,20300,ATL,0,2,1,0,0,3 +2010,20300,BOS,2,1,0,0,1,4 +2010,20301,OTT,1,1,4,0,0,6 +2010,20301,PIT,1,0,1,0,0,2 +2010,20302,PHX,1,1,0,0,0,2 +2010,20302,STL,1,0,1,1,0,3 +2010,20303,NSH,1,0,1,0,1,3 +2010,20303,NJD,1,0,1,0,0,2 +2010,20304,CBJ,2,2,0,0,0,4 +2010,20304,DAL,0,1,0,0,0,1 +2010,20305,CGY,0,1,0,0,0,1 +2010,20305,CHI,1,5,1,0,0,7 +2010,20306,ANA,2,1,0,1,0,4 +2010,20306,TBL,0,2,1,0,0,3 +2010,20307,MTL,0,2,1,0,0,3 +2010,20307,WSH,1,0,1,0,0,2 +2010,20308,BOS,0,1,0,1,0,2 +2010,20308,BUF,1,0,0,0,0,1 +2010,20309,DET,0,1,0,0,0,1 +2010,20309,FLA,0,0,1,1,0,2 +2010,20310,MIN,1,0,2,0,0,3 +2010,20310,NYI,0,2,0,0,0,2 +2010,20311,COL,2,0,0,0,0,2 +2010,20311,VAN,0,1,4,0,0,5 +2010,20312,PHI,1,1,1,0,0,3 +2010,20312,SJS,1,2,3,0,0,6 +2010,20313,CGY,1,1,3,0,0,5 +2010,20313,LAK,0,2,0,0,0,2 +2010,20314,TOR,0,1,0,0,1,2 +2010,20314,WSH,1,0,0,0,0,1 +2010,20315,MTL,0,0,2,0,0,2 +2010,20315,DET,2,0,0,0,1,3 +2010,20316,BUF,1,0,2,0,0,3 +2010,20316,OTT,0,2,3,0,0,5 +2010,20317,FLA,0,2,1,0,0,3 +2010,20317,NYR,1,0,1,0,0,2 +2010,20318,CAR,0,2,1,0,0,3 +2010,20318,TBL,1,0,0,0,0,1 +2010,20319,ATL,0,0,2,0,0,2 +2010,20319,PIT,1,2,0,0,0,3 +2010,20320,NYI,1,0,0,0,0,1 +2010,20320,STL,1,1,2,0,0,4 +2010,20321,CBJ,2,1,0,0,0,3 +2010,20321,NSH,1,2,0,0,1,4 +2010,20322,DAL,3,0,2,0,0,5 +2010,20322,NJD,1,1,1,0,0,3 +2010,20323,PHI,0,1,0,0,0,1 +2010,20323,PHX,0,1,2,0,0,3 +2010,20324,CHI,3,1,1,0,0,5 +2010,20324,EDM,1,0,1,0,0,2 +2010,20325,ANA,0,2,0,0,0,2 +2010,20325,SJS,0,2,1,0,0,3 +2010,20326,ATL,0,0,3,0,0,3 +2010,20326,TBL,0,2,1,1,0,4 +2010,20327,CHI,0,0,1,0,0,1 +2010,20327,VAN,0,0,0,0,0,0 +2010,20328,DET,0,1,0,0,0,1 +2010,20328,NSH,1,1,1,0,0,3 +2010,20329,NYI,0,3,0,1,0,4 +2010,20329,TOR,0,1,2,0,0,3 +2010,20330,CBJ,2,1,1,0,0,4 +2010,20330,NYR,3,4,0,0,0,7 +2010,20331,OTT,1,0,2,1,0,4 +2010,20331,WSH,2,1,0,0,0,3 +2010,20332,FLA,1,1,0,0,0,2 +2010,20332,PIT,0,0,2,1,0,3 +2010,20333,BOS,1,2,1,0,0,4 +2010,20333,STL,1,1,0,0,0,2 +2010,20334,CAR,0,0,0,0,0,0 +2010,20334,DAL,1,0,1,0,0,2 +2010,20335,COL,2,3,0,0,0,5 +2010,20335,PHI,1,1,2,0,0,4 +2010,20336,EDM,3,1,0,0,0,4 +2010,20336,PHX,0,0,0,0,0,0 +2010,20337,ANA,2,0,0,0,1,3 +2010,20337,CGY,1,0,1,0,0,2 +2010,20338,MTL,1,1,3,0,0,5 +2010,20338,CBJ,2,1,0,0,0,3 +2010,20339,NJD,1,1,1,0,0,3 +2010,20339,OTT,1,0,0,0,0,1 +2010,20340,NYI,1,0,0,0,0,1 +2010,20340,PHI,0,1,1,0,0,2 +2010,20341,BUF,0,0,0,0,0,0 +2010,20341,WSH,1,0,1,0,0,2 +2010,20342,TBL,2,1,0,0,0,3 +2010,20342,TOR,2,2,0,0,0,4 +2010,20343,MTL,0,0,1,0,0,1 +2010,20343,PIT,1,2,0,0,0,3 +2010,20344,FLA,0,0,1,0,0,1 +2010,20344,NYR,0,1,0,0,1,2 +2010,20345,ATL,1,1,0,0,0,2 +2010,20345,DET,0,0,0,0,0,0 +2010,20346,BOS,1,0,0,0,1,2 +2010,20346,MIN,0,1,0,0,0,1 +2010,20347,DAL,2,0,1,0,0,3 +2010,20347,STL,2,0,1,0,1,4 +2010,20348,COL,0,3,0,0,0,3 +2010,20348,NSH,2,1,0,1,0,4 +2010,20349,CGY,0,1,1,0,0,2 +2010,20349,PHX,0,0,1,0,0,1 +2010,20350,EDM,1,0,0,0,0,1 +2010,20350,LAK,0,1,2,0,0,3 +2010,20351,ANA,0,1,2,0,0,3 +2010,20351,CAR,1,0,1,0,0,2 +2010,20352,CHI,1,3,3,0,0,7 +2010,20352,SJS,0,0,2,0,0,2 +2010,20353,CBJ,1,0,0,0,0,1 +2010,20353,OTT,1,0,1,0,0,2 +2010,20354,LAK,0,1,0,0,0,1 +2010,20354,VAN,1,0,3,0,0,4 +2010,20355,BOS,0,0,1,0,0,1 +2010,20355,NJD,0,1,0,0,1,2 +2010,20356,BUF,1,1,2,0,0,4 +2010,20356,PHI,2,0,0,0,0,2 +2010,20357,NYI,1,0,2,0,0,3 +2010,20357,PIT,1,1,0,0,0,2 +2010,20358,COL,1,2,0,0,0,3 +2010,20358,MIN,1,2,2,0,0,5 +2010,20359,ANA,1,2,0,0,0,3 +2010,20359,CHI,0,0,0,0,0,0 +2010,20360,ATL,0,1,5,0,0,6 +2010,20360,CAR,1,3,0,0,0,4 +2010,20361,NYR,0,0,1,0,0,1 +2010,20361,TBL,2,2,1,0,0,5 +2010,20362,FLA,2,1,1,0,0,4 +2010,20362,TOR,1,2,3,0,0,6 +2010,20363,CGY,2,0,1,0,0,3 +2010,20363,DET,0,0,0,0,0,0 +2010,20364,NSH,0,0,1,0,0,1 +2010,20364,STL,0,1,2,0,0,3 +2010,20365,EDM,0,1,3,0,0,4 +2010,20365,SJS,1,0,3,0,1,5 +2010,20366,DAL,0,0,2,0,0,2 +2010,20366,PHX,1,3,1,0,0,5 +2010,20367,NJD,0,2,4,0,0,6 +2010,20367,NYI,1,0,0,0,0,1 +2010,20368,BOS,0,2,1,0,1,4 +2010,20368,OTT,2,0,1,0,0,3 +2010,20369,BUF,0,0,5,0,0,5 +2010,20369,CAR,1,0,0,0,0,1 +2010,20370,MTL,0,2,1,0,0,3 +2010,20370,WSH,2,0,1,0,1,4 +2010,20371,ATL,0,1,0,0,0,1 +2010,20371,PHI,0,0,0,0,0,0 +2010,20372,CGY,0,1,2,0,1,4 +2010,20372,CBJ,0,3,0,0,0,3 +2010,20373,NYR,1,1,1,0,0,3 +2010,20373,PIT,3,1,4,0,0,8 +2010,20374,DET,0,1,2,0,1,4 +2010,20374,STL,2,0,1,0,0,3 +2010,20375,FLA,0,1,0,0,0,1 +2010,20375,NSH,1,0,3,0,0,4 +2010,20376,DAL,0,2,1,1,0,4 +2010,20376,TBL,0,1,2,0,0,3 +2010,20377,COL,2,0,0,0,0,2 +2010,20377,MIN,0,1,1,0,1,3 +2010,20378,EDM,1,2,0,0,0,3 +2010,20378,VAN,5,1,1,0,0,7 +2010,20379,CHI,0,0,1,0,0,1 +2010,20379,LAK,0,1,0,0,1,2 +2010,20380,ANA,0,1,1,0,0,2 +2010,20380,PHX,0,0,2,1,0,3 +2010,20381,SJS,1,1,2,0,0,4 +2010,20381,VAN,1,0,1,0,0,2 +2010,20382,BUF,0,1,2,0,0,3 +2010,20382,TOR,0,0,0,0,0,0 +2010,20383,NYR,0,2,0,0,0,2 +2010,20383,PIT,0,2,3,0,0,5 +2010,20384,CAR,1,0,1,0,0,2 +2010,20384,WSH,2,1,0,0,0,3 +2010,20385,ATL,2,1,1,0,0,4 +2010,20385,FLA,2,1,0,0,0,3 +2010,20386,CBJ,2,0,3,0,0,5 +2010,20386,STL,1,1,0,0,0,2 +2010,20387,COL,1,1,1,0,0,3 +2010,20387,TBL,0,0,0,0,0,0 +2010,20388,DAL,1,0,0,0,0,1 +2010,20388,DET,1,2,1,0,0,4 +2010,20389,CGY,0,2,3,0,0,5 +2010,20389,NSH,0,0,0,0,0,0 +2010,20390,MTL,0,0,0,0,0,0 +2010,20390,TOR,2,1,0,0,0,3 +2010,20391,CHI,1,1,1,0,1,4 +2010,20391,CBJ,0,2,1,0,0,3 +2010,20392,ANA,1,1,1,0,0,3 +2010,20392,LAK,2,1,1,0,0,4 +2010,20393,OTT,1,1,0,0,0,2 +2010,20393,SJS,2,2,1,0,0,5 +2010,20394,BOS,2,2,0,0,0,4 +2010,20394,TBL,0,0,1,0,0,1 +2010,20395,NJD,2,0,0,0,0,2 +2010,20395,VAN,3,0,2,0,0,5 +2010,20396,COL,0,2,3,0,0,5 +2010,20396,FLA,2,2,1,0,1,6 +2010,20397,MIN,3,0,1,1,0,5 +2010,20397,NSH,2,1,1,0,0,4 +2010,20398,BUF,3,1,2,0,0,6 +2010,20398,MTL,0,1,1,0,0,2 +2010,20399,PHI,0,0,0,0,0,0 +2010,20399,VAN,0,2,1,0,0,3 +2010,20400,FLA,0,0,2,0,0,2 +2010,20400,WSH,2,2,2,0,0,6 +2010,20401,ATL,1,0,0,0,0,1 +2010,20401,NYI,0,2,2,0,0,4 +2010,20402,CBJ,0,1,2,0,0,3 +2010,20402,TOR,2,3,1,0,0,6 +2010,20403,COL,1,0,0,0,0,1 +2010,20403,PIT,1,0,3,0,0,4 +2010,20404,DET,1,0,0,0,0,1 +2010,20404,EDM,2,2,0,0,0,4 +2010,20405,ANA,1,0,0,0,0,1 +2010,20405,DAL,0,0,3,0,0,3 +2010,20406,CGY,1,0,0,0,0,1 +2010,20406,PHX,0,1,1,0,0,2 +2010,20407,LAK,2,2,2,0,0,6 +2010,20407,OTT,1,2,0,0,0,3 +2010,20408,SJS,1,1,0,0,0,2 +2010,20408,STL,1,0,1,0,1,3 +2010,20409,NJD,0,0,3,0,0,3 +2010,20409,TBL,0,1,1,0,0,2 +2010,20410,BOS,0,0,1,0,0,1 +2010,20410,MTL,1,4,0,0,0,5 +2010,20411,ANA,1,1,2,0,0,4 +2010,20411,MIN,0,1,3,0,1,5 +2010,20412,CHI,1,0,0,0,0,1 +2010,20412,NSH,1,1,2,0,0,4 +2010,20413,CAR,1,3,1,0,0,5 +2010,20413,VAN,1,0,2,0,0,3 +2010,20414,DAL,2,0,0,0,0,2 +2010,20414,EDM,0,1,1,0,1,3 +2010,20415,LAK,1,1,2,0,0,4 +2010,20415,STL,0,2,2,0,1,5 +2010,20416,BOS,2,2,3,0,0,7 +2010,20416,TOR,0,0,2,0,0,2 +2010,20417,BUF,0,0,1,0,0,1 +2010,20417,NYR,1,0,1,0,0,2 +2010,20418,DET,0,2,1,0,0,3 +2010,20418,NJD,2,1,0,0,1,4 +2010,20419,PHI,1,0,1,0,0,2 +2010,20419,WSH,3,4,1,0,0,8 +2010,20420,ATL,0,0,1,0,1,2 +2010,20420,FLA,0,0,1,0,0,1 +2010,20421,COL,0,2,1,0,0,3 +2010,20421,CBJ,0,0,2,0,0,2 +2010,20422,CHI,1,0,0,1,0,2 +2010,20422,PIT,0,0,1,0,0,1 +2010,20423,NYI,0,0,0,0,0,0 +2010,20423,TBL,2,0,2,0,0,4 +2010,20424,MIN,2,2,1,0,0,5 +2010,20424,NSH,1,0,2,0,0,3 +2010,20425,OTT,0,0,2,0,0,2 +2010,20425,PHX,2,1,0,0,0,3 +2010,20426,CGY,0,1,1,0,0,2 +2010,20426,SJS,0,1,0,0,0,1 +2010,20427,DET,0,1,2,0,0,3 +2010,20427,NYR,1,0,0,0,0,1 +2010,20428,ANA,2,0,1,0,0,3 +2010,20428,OTT,2,0,1,0,1,4 +2010,20429,BUF,0,0,0,0,0,0 +2010,20429,NJD,2,1,0,0,0,3 +2010,20430,ATL,0,0,2,0,0,2 +2010,20430,TOR,0,5,0,0,0,5 +2010,20431,MTL,0,2,1,0,0,3 +2010,20431,PHI,1,0,0,0,0,1 +2010,20432,CAR,2,1,0,0,0,3 +2010,20432,PIT,0,2,0,0,0,2 +2010,20433,TBL,0,0,0,0,0,0 +2010,20433,WSH,0,3,0,0,0,3 +2010,20434,EDM,0,1,1,0,1,3 +2010,20434,FLA,0,1,1,0,0,2 +2010,20435,COL,1,2,1,0,0,4 +2010,20435,STL,0,0,0,0,0,0 +2010,20436,MIN,0,0,0,0,0,0 +2010,20436,PHX,0,2,0,0,0,2 +2010,20437,CGY,1,0,0,0,0,1 +2010,20437,LAK,1,1,0,0,0,2 +2010,20438,NYI,0,2,0,0,0,2 +2010,20438,PHI,2,3,1,0,0,6 +2010,20439,MTL,1,1,2,0,0,4 +2010,20439,OTT,0,1,0,0,0,1 +2010,20440,NSH,1,1,2,0,0,4 +2010,20440,VAN,0,1,1,0,0,2 +2010,20441,ANA,0,1,2,1,0,4 +2010,20441,DAL,2,1,0,0,0,3 +2010,20442,BUF,1,2,0,0,0,3 +2010,20442,WSH,0,0,0,0,0,0 +2010,20443,NYI,0,0,2,0,0,2 +2010,20443,TOR,2,0,1,0,0,3 +2010,20444,CAR,0,2,0,0,0,2 +2010,20444,NJD,2,1,1,0,0,4 +2010,20445,CBJ,1,0,2,0,0,3 +2010,20445,FLA,0,0,0,0,0,0 +2010,20446,EDM,1,2,0,0,0,3 +2010,20446,TBL,1,1,0,0,0,2 +2010,20447,DET,0,0,0,0,0,0 +2010,20447,STL,1,0,0,0,0,1 +2010,20448,CHI,0,0,1,1,0,2 +2010,20448,NYR,1,0,0,0,0,1 +2010,20449,COL,0,0,0,0,0,0 +2010,20449,MIN,1,0,0,0,0,1 +2010,20450,ATL,0,0,1,0,0,1 +2010,20450,CGY,0,2,1,0,0,3 +2010,20451,LAK,1,3,0,1,0,5 +2010,20451,SJS,2,1,1,0,0,4 +2010,20452,BOS,1,2,2,0,0,5 +2010,20452,TOR,0,0,2,0,0,2 +2010,20453,OTT,0,0,2,0,0,2 +2010,20453,PHI,0,0,0,0,0,0 +2010,20454,MTL,0,2,0,0,0,2 +2010,20454,PIT,1,1,1,0,0,3 +2010,20455,CBJ,0,2,1,0,0,3 +2010,20455,NSH,1,2,0,0,1,4 +2010,20456,ATL,1,1,0,0,0,2 +2010,20456,VAN,2,2,0,0,0,4 +2010,20457,LAK,2,0,0,0,1,3 +2010,20457,PHX,1,0,1,0,0,2 +2010,20458,FLA,1,2,1,0,0,4 +2010,20458,NJD,0,1,1,0,0,2 +2010,20459,CAR,2,0,1,0,0,3 +2010,20459,WSH,0,2,1,1,0,4 +2010,20460,BUF,0,2,0,0,0,2 +2010,20460,CHI,0,0,1,0,0,1 +2010,20461,ANA,0,2,0,0,0,2 +2010,20461,DET,0,0,2,1,0,3 +2010,20462,EDM,0,1,4,0,0,5 +2010,20462,STL,1,2,0,0,0,3 +2010,20463,COL,0,0,1,0,1,2 +2010,20463,TBL,1,0,0,0,0,1 +2010,20464,CGY,0,0,1,0,0,1 +2010,20464,MIN,0,0,1,1,0,2 +2010,20465,DAL,1,0,1,0,1,3 +2010,20465,SJS,2,0,0,0,0,2 +2010,20466,TOR,0,3,3,0,0,6 +2010,20466,WSH,2,1,0,0,0,3 +2010,20467,CAR,0,2,0,0,0,2 +2010,20467,OTT,1,1,2,0,0,4 +2010,20468,NJD,3,1,0,0,0,4 +2010,20468,PHI,0,1,0,0,0,1 +2010,20469,BOS,1,1,0,0,0,2 +2010,20469,NYI,1,1,0,1,0,3 +2010,20470,BUF,2,1,0,0,0,3 +2010,20470,NYR,2,0,0,0,0,2 +2010,20471,ATL,0,3,0,1,0,4 +2010,20471,MTL,1,2,0,0,0,3 +2010,20472,ANA,1,1,1,0,0,3 +2010,20472,CBJ,0,1,0,0,0,1 +2010,20473,FLA,1,0,1,0,0,2 +2010,20473,PIT,1,1,0,1,0,3 +2010,20474,DET,0,2,0,1,0,3 +2010,20474,NSH,0,1,1,0,0,2 +2010,20475,PHX,0,2,0,0,0,2 +2010,20475,SJS,0,0,1,0,0,1 +2010,20476,MIN,2,0,1,0,0,3 +2010,20476,VAN,3,0,1,0,0,4 +2010,20477,DAL,1,1,0,0,0,2 +2010,20477,LAK,0,2,0,0,1,3 +2010,20478,CHI,1,2,1,0,0,4 +2010,20478,TBL,0,0,0,0,0,0 +2010,20479,CGY,0,1,1,0,0,2 +2010,20479,COL,1,2,0,0,0,3 +2010,20480,BOS,0,1,0,0,0,1 +2010,20480,PHI,0,0,3,0,0,3 +2010,20481,OTT,0,1,1,0,0,2 +2010,20481,TOR,0,2,1,0,0,3 +2010,20482,FLA,1,2,4,0,0,7 +2010,20482,NYI,0,1,0,0,0,1 +2010,20483,ATL,1,0,1,0,1,3 +2010,20483,NYR,0,1,1,0,0,2 +2010,20484,CBJ,1,1,1,0,0,3 +2010,20484,NSH,1,3,1,0,0,5 +2010,20485,BUF,0,2,2,0,0,4 +2010,20485,MTL,1,0,2,0,0,3 +2010,20486,DET,2,1,0,0,0,3 +2010,20486,PHX,1,1,0,0,0,2 +2010,20487,LAK,1,0,0,0,0,1 +2010,20487,VAN,2,0,1,0,0,3 +2010,20488,PHI,1,0,0,0,0,1 +2010,20488,PIT,3,1,2,0,0,6 +2010,20489,CGY,1,2,0,0,0,3 +2010,20489,STL,2,1,1,0,0,4 +2010,20490,NSH,1,5,1,0,0,7 +2010,20490,TBL,2,0,2,0,0,4 +2010,20491,CBJ,0,1,0,0,0,1 +2010,20491,MIN,0,1,1,0,0,2 +2010,20492,COL,0,0,1,0,0,1 +2010,20492,WSH,2,3,1,0,0,6 +2010,20493,EDM,0,1,1,0,0,2 +2010,20493,LAK,1,1,1,0,0,3 +2010,20494,MTL,1,0,0,0,0,1 +2010,20494,NJD,0,1,1,0,0,2 +2010,20495,NYI,1,0,1,0,0,2 +2010,20495,NYR,0,0,1,0,0,1 +2010,20496,CAR,3,0,2,0,0,5 +2010,20496,DAL,1,1,1,0,0,3 +2010,20497,PHX,1,4,1,0,0,6 +2010,20497,TOR,0,2,1,0,0,3 +2010,20498,BUF,0,0,0,0,0,0 +2010,20498,OTT,1,0,1,0,0,2 +2010,20499,ATL,0,1,2,0,0,3 +2010,20499,FLA,2,1,1,0,0,4 +2010,20500,CHI,2,1,0,0,0,3 +2010,20500,STL,0,0,0,0,0,0 +2010,20501,ANA,1,1,1,0,0,3 +2010,20501,VAN,1,0,1,0,0,2 +2010,20502,MTL,1,0,0,0,0,1 +2010,20502,MIN,2,0,1,0,0,3 +2010,20503,NYI,0,1,1,0,0,2 +2010,20503,NYR,2,1,2,0,0,5 +2010,20504,PHI,1,1,0,0,0,2 +2010,20504,PIT,1,1,0,0,1,3 +2010,20505,ATL,1,2,2,1,0,6 +2010,20505,DAL,2,1,2,0,0,5 +2010,20506,CBJ,0,0,1,0,0,1 +2010,20506,PHX,1,0,0,0,1,2 +2010,20507,DET,1,0,2,0,0,3 +2010,20507,TBL,0,0,0,0,0,0 +2010,20508,CGY,1,0,1,0,0,2 +2010,20508,LAK,0,0,1,0,0,1 +2010,20509,EDM,2,1,0,0,0,3 +2010,20509,NSH,3,2,1,0,0,6 +2010,20510,ANA,0,0,1,0,0,1 +2010,20510,SJS,0,2,2,0,0,4 +2010,20511,NJD,1,2,1,0,0,4 +2010,20511,OTT,0,2,0,0,0,2 +2010,20512,BUF,0,4,1,0,0,5 +2010,20512,TOR,1,0,1,0,0,2 +2010,20513,CAR,0,1,2,0,0,3 +2010,20513,FLA,0,3,3,0,0,6 +2010,20514,STL,1,0,2,0,0,3 +2010,20514,TBL,1,2,3,0,0,6 +2010,20515,BOS,1,1,2,0,0,4 +2010,20515,CHI,0,3,1,0,1,5 +2010,20516,VAN,1,1,1,0,0,3 +2010,20516,WSH,2,0,0,0,0,2 +2010,20517,NYR,1,1,0,0,0,2 +2010,20517,PHI,1,0,0,0,0,1 +2010,20518,DAL,0,3,1,0,0,4 +2010,20518,DET,1,2,0,0,0,3 +2010,20519,ANA,1,2,1,0,0,4 +2010,20519,PHX,0,1,1,0,0,2 +2010,20520,BUF,1,0,0,0,0,1 +2010,20520,PIT,1,0,0,0,1,2 +2010,20521,BOS,0,0,0,0,0,0 +2010,20521,TOR,0,1,1,0,0,2 +2010,20522,MIN,0,0,1,0,0,1 +2010,20522,OTT,1,2,1,0,0,4 +2010,20523,MTL,1,1,1,0,0,3 +2010,20523,NYI,0,0,0,0,0,0 +2010,20524,CAR,2,1,0,0,0,3 +2010,20524,FLA,0,0,2,0,0,2 +2010,20525,ATL,3,0,1,0,0,4 +2010,20525,NJD,1,3,1,0,0,5 +2010,20526,CGY,1,2,0,0,0,3 +2010,20526,NSH,2,1,2,0,0,5 +2010,20527,COL,4,0,1,0,0,5 +2010,20527,CBJ,0,2,0,0,0,2 +2010,20528,EDM,0,2,0,0,0,2 +2010,20528,WSH,0,0,4,0,0,4 +2010,20529,CHI,1,2,0,0,0,3 +2010,20529,DET,0,0,0,0,0,0 +2010,20530,STL,1,2,0,0,0,3 +2010,20530,VAN,1,0,0,0,0,1 +2010,20531,BUF,1,0,1,1,0,3 +2010,20531,TOR,0,2,0,0,0,2 +2010,20532,NYI,1,1,0,0,0,2 +2010,20532,TBL,0,2,2,0,0,4 +2010,20533,FLA,1,1,2,0,0,4 +2010,20533,PHI,0,0,1,0,0,1 +2010,20534,CAR,0,1,0,0,0,1 +2010,20534,NYR,0,1,2,0,0,3 +2010,20535,ATL,1,0,2,0,0,3 +2010,20535,MTL,0,2,1,1,0,4 +2010,20536,BOS,0,0,2,0,0,2 +2010,20536,OTT,0,0,0,0,0,0 +2010,20537,NJD,1,3,0,0,0,4 +2010,20537,PIT,0,0,0,0,0,0 +2010,20538,COL,1,3,0,0,0,4 +2010,20538,MIN,0,1,2,0,0,3 +2010,20539,DAL,0,0,2,0,0,2 +2010,20539,SJS,0,2,2,0,0,4 +2010,20540,EDM,1,1,0,0,0,2 +2010,20540,STL,1,4,2,0,0,7 +2010,20541,CBJ,0,1,1,0,0,2 +2010,20541,PHX,2,2,1,0,0,5 +2010,20542,CHI,0,0,2,0,0,2 +2010,20542,SJS,1,1,1,0,0,3 +2010,20543,ANA,0,0,4,0,0,4 +2010,20543,COL,1,0,1,0,0,2 +2010,20544,NSH,0,0,1,0,0,1 +2010,20544,VAN,0,2,2,0,0,4 +2010,20545,ATL,2,0,2,0,0,4 +2010,20545,BOS,3,1,2,0,0,6 +2010,20546,NYI,0,2,1,0,0,3 +2010,20546,TOR,0,0,1,0,0,1 +2010,20547,FLA,1,0,0,0,0,1 +2010,20547,NYR,1,1,2,0,0,4 +2010,20548,OTT,1,0,1,0,0,2 +2010,20548,PIT,1,5,2,0,0,8 +2010,20549,BUF,0,2,0,0,0,2 +2010,20549,WSH,2,1,2,0,0,5 +2010,20550,MTL,4,1,0,0,0,5 +2010,20550,CAR,1,0,0,0,0,1 +2010,20551,EDM,1,0,0,0,0,1 +2010,20551,MIN,1,1,1,0,0,3 +2010,20552,PHI,2,0,3,0,0,5 +2010,20552,TBL,0,2,0,0,0,2 +2010,20553,CHI,0,1,2,0,0,3 +2010,20553,DET,0,0,0,0,0,0 +2010,20554,CBJ,1,0,0,0,0,1 +2010,20554,DAL,1,1,1,0,0,3 +2010,20555,ANA,0,0,0,0,0,0 +2010,20555,PHX,2,1,1,0,0,4 +2010,20556,CGY,0,0,1,0,0,1 +2010,20556,STL,0,1,0,0,1,2 +2010,20557,BUF,0,2,0,0,0,2 +2010,20557,OTT,0,2,0,0,1,3 +2010,20558,MTL,2,0,0,1,0,3 +2010,20558,TOR,0,1,1,0,0,2 +2010,20559,NYI,1,1,0,1,0,3 +2010,20559,NYR,0,0,2,0,0,2 +2010,20560,NJD,1,0,0,0,0,1 +2010,20560,WSH,2,1,1,0,0,4 +2010,20561,CAR,0,0,3,0,0,3 +2010,20561,PHI,2,0,1,0,1,4 +2010,20562,CBJ,1,0,0,0,0,1 +2010,20562,DET,0,1,1,0,0,2 +2010,20563,ATL,1,1,1,0,0,3 +2010,20563,TBL,1,1,2,0,0,4 +2010,20564,CHI,1,3,0,0,0,4 +2010,20564,NSH,0,1,0,0,0,1 +2010,20565,MIN,1,0,3,0,0,4 +2010,20565,STL,0,1,2,0,0,3 +2010,20566,LAK,1,0,1,0,0,2 +2010,20566,PHX,2,0,1,0,0,3 +2010,20567,COL,1,1,2,0,0,4 +2010,20567,DAL,0,0,1,0,0,1 +2010,20568,EDM,0,1,0,0,0,1 +2010,20568,VAN,0,2,2,0,0,4 +2010,20569,ANA,0,1,1,0,0,2 +2010,20569,SJS,2,3,0,0,0,5 +2010,20570,NYI,0,1,0,0,0,1 +2010,20570,PHI,1,1,0,0,0,2 +2010,20571,BOS,0,1,1,0,0,2 +2010,20571,FLA,1,0,0,0,0,1 +2010,20572,BUF,1,1,3,0,0,5 +2010,20572,STL,1,2,0,0,0,3 +2010,20573,PIT,1,1,1,0,0,3 +2010,20573,TOR,2,1,1,0,0,4 +2010,20574,CHI,1,0,4,0,0,5 +2010,20574,NSH,1,1,2,0,0,4 +2010,20575,CGY,1,0,0,0,0,1 +2010,20575,VAN,4,1,0,0,0,5 +2010,20576,ATL,1,0,1,0,0,2 +2010,20576,NJD,2,0,1,0,0,3 +2010,20577,CAR,3,1,2,0,0,6 +2010,20577,WSH,0,2,1,0,0,3 +2010,20578,CBJ,0,0,0,1,0,1 +2010,20578,DET,0,0,0,0,0,0 +2010,20579,MTL,2,0,0,0,0,2 +2010,20579,OTT,1,1,2,0,0,4 +2010,20580,BOS,0,0,1,0,0,1 +2010,20580,TBL,1,1,0,0,0,2 +2010,20581,CGY,1,1,2,0,0,4 +2010,20581,EDM,1,0,0,0,0,1 +2010,20582,LAK,0,1,2,0,0,3 +2010,20582,MIN,0,3,1,0,0,4 +2010,20583,PHX,0,2,0,0,0,2 +2010,20583,SJS,1,1,0,0,1,3 +2010,20584,BUF,0,2,2,0,0,4 +2010,20584,PIT,2,1,0,0,0,3 +2010,20585,CBJ,0,0,1,0,0,1 +2010,20585,NYI,1,0,0,0,1,2 +2010,20586,NSH,2,1,1,0,0,4 +2010,20586,STL,1,0,2,0,0,3 +2010,20587,CHI,3,1,0,0,0,4 +2010,20587,DAL,2,2,1,0,0,5 +2010,20588,PHX,0,2,0,0,1,3 +2010,20588,VAN,0,1,1,0,0,2 +2010,20589,ANA,2,1,1,0,0,4 +2010,20589,MIN,1,0,1,0,0,2 +2010,20590,ATL,0,0,0,0,0,0 +2010,20590,BOS,3,1,0,0,0,4 +2010,20591,NJD,1,0,1,0,0,2 +2010,20591,PIT,0,0,0,0,0,0 +2010,20592,NYR,0,0,0,0,0,0 +2010,20592,PHI,3,2,1,0,0,6 +2010,20593,MTL,0,1,0,1,0,2 +2010,20593,TBL,0,0,1,0,0,1 +2010,20594,COL,2,1,1,0,0,4 +2010,20594,OTT,1,0,2,0,0,3 +2010,20595,CGY,1,1,0,0,0,2 +2010,20595,LAK,0,1,0,0,0,1 +2010,20596,EDM,1,1,1,0,0,3 +2010,20596,TOR,0,0,1,0,0,1 +2010,20597,SJS,2,1,2,0,0,5 +2010,20597,WSH,0,0,2,0,0,2 +2010,20598,MTL,1,2,2,0,0,5 +2010,20598,FLA,2,1,1,0,0,4 +2010,20599,NYI,0,1,1,0,0,2 +2010,20599,OTT,0,2,0,0,1,3 +2010,20600,COL,0,1,1,0,0,2 +2010,20600,DET,0,2,2,0,0,4 +2010,20601,CBJ,1,0,0,0,0,1 +2010,20601,NSH,0,0,1,1,0,2 +2010,20602,PHX,0,2,0,0,0,2 +2010,20602,SJS,0,1,1,0,1,3 +2010,20603,CAR,1,0,0,0,0,1 +2010,20603,NYR,1,0,1,0,0,2 +2010,20604,STL,1,2,0,0,0,3 +2010,20604,VAN,0,1,2,1,0,4 +2010,20605,LAK,3,1,1,0,0,5 +2010,20605,MIN,0,1,1,0,0,2 +2010,20606,ANA,1,1,1,0,0,3 +2010,20606,DAL,1,3,1,0,0,5 +2010,20607,CHI,2,1,2,0,0,5 +2010,20607,NJD,0,0,1,0,0,1 +2010,20608,CGY,0,1,1,0,0,2 +2010,20608,EDM,0,0,1,0,0,1 +2010,20609,BOS,0,0,1,1,0,2 +2010,20609,PHI,0,1,0,0,0,1 +2010,20610,ATL,0,3,0,0,0,3 +2010,20610,BUF,0,1,2,1,0,4 +2010,20611,CAR,0,1,0,1,0,2 +2010,20611,NYR,0,0,1,0,0,1 +2010,20612,DAL,0,0,1,0,0,1 +2010,20612,VAN,0,1,2,0,0,3 +2010,20613,PIT,1,0,0,0,0,1 +2010,20613,TBL,1,1,1,0,0,3 +2010,20614,LAK,1,0,1,0,0,2 +2010,20614,WSH,0,1,0,0,0,1 +2010,20615,ATL,0,4,1,0,0,5 +2010,20615,NYI,3,2,0,0,1,6 +2010,20616,COL,0,1,2,0,0,3 +2010,20616,CBJ,1,1,0,0,0,2 +2010,20617,CGY,0,2,1,0,0,3 +2010,20617,TOR,1,0,0,0,0,1 +2010,20618,CHI,2,2,2,0,0,6 +2010,20618,STL,2,0,1,0,0,3 +2010,20619,ANA,1,0,0,0,0,1 +2010,20619,NSH,0,1,2,0,0,3 +2010,20620,MIN,0,2,1,0,0,3 +2010,20620,NJD,0,3,2,0,0,5 +2010,20621,DET,1,1,2,0,0,4 +2010,20621,PHX,0,1,0,0,0,1 +2010,20622,EDM,0,1,0,0,0,1 +2010,20622,SJS,2,2,0,0,0,4 +2010,20623,BUF,0,1,0,0,0,1 +2010,20623,MTL,0,0,0,0,0,0 +2010,20624,OTT,4,0,3,0,0,7 +2010,20624,PHI,1,3,0,0,0,4 +2010,20625,FLA,1,2,3,0,0,6 +2010,20625,PIT,2,0,0,0,0,2 +2010,20626,ANA,0,0,2,0,0,2 +2010,20626,CHI,0,2,3,0,0,5 +2010,20627,BOS,0,0,2,0,0,2 +2010,20627,NYR,1,1,1,0,0,3 +2010,20628,LAK,3,2,1,0,0,6 +2010,20628,SJS,0,0,2,0,0,2 +2010,20629,FLA,2,0,0,0,0,2 +2010,20629,TOR,1,2,0,0,0,3 +2010,20630,DAL,0,0,0,0,0,0 +2010,20630,NJD,1,3,0,0,0,4 +2010,20631,MTL,0,1,1,0,0,2 +2010,20631,WSH,0,3,1,0,0,4 +2010,20632,BOS,4,0,0,0,0,4 +2010,20632,OTT,0,1,0,0,0,1 +2010,20633,ATL,0,2,0,0,0,2 +2010,20633,PIT,0,3,2,0,0,5 +2010,20634,CGY,0,2,1,0,0,3 +2010,20634,NSH,0,0,1,0,0,1 +2010,20635,CHI,1,1,2,0,0,4 +2010,20635,MIN,1,0,0,0,0,1 +2010,20636,EDM,0,2,2,0,0,4 +2010,20636,PHX,1,3,0,1,0,5 +2010,20637,CBJ,2,1,0,0,0,3 +2010,20637,VAN,3,3,1,0,0,7 +2010,20638,ANA,2,0,2,0,0,4 +2010,20638,DET,0,0,1,0,0,1 +2010,20639,BUF,3,2,0,0,0,5 +2010,20639,TBL,2,1,0,0,0,3 +2010,20640,DAL,1,1,0,0,0,2 +2010,20640,NYR,2,1,2,0,0,5 +2010,20641,PHI,2,1,3,0,0,6 +2010,20641,TOR,0,0,2,0,0,2 +2010,20642,CGY,1,0,0,0,0,1 +2010,20642,MIN,1,2,1,0,0,4 +2010,20643,COL,1,0,1,0,0,2 +2010,20643,NYI,1,1,1,0,0,3 +2010,20644,SJS,0,0,1,1,0,2 +2010,20644,STL,0,1,0,0,0,1 +2010,20645,BOS,2,0,0,0,0,2 +2010,20645,CHI,2,2,1,0,0,5 +2010,20646,OTT,0,1,1,0,0,2 +2010,20646,WSH,2,0,3,0,0,5 +2010,20647,ATL,0,0,1,0,1,2 +2010,20647,NYR,0,1,0,0,0,1 +2010,20648,MTL,1,0,1,0,0,2 +2010,20648,FLA,0,0,0,0,0,0 +2010,20649,PHI,3,2,2,0,0,7 +2010,20649,PIT,2,1,1,0,0,4 +2010,20650,CAR,1,0,1,0,0,2 +2010,20650,NSH,2,1,1,0,0,4 +2010,20651,CBJ,1,1,2,0,0,4 +2010,20651,EDM,1,0,1,0,0,2 +2010,20652,PHX,0,0,0,0,0,0 +2010,20652,VAN,1,1,2,0,0,4 +2010,20653,ANA,0,1,3,0,0,4 +2010,20653,STL,1,0,1,0,0,2 +2010,20654,DET,1,0,1,0,0,2 +2010,20654,LAK,0,0,1,0,0,1 +2010,20655,NJD,0,0,2,0,0,2 +2010,20655,TBL,1,2,1,0,0,4 +2010,20656,CAR,1,1,0,0,0,2 +2010,20656,COL,0,0,1,0,0,1 +2010,20657,BUF,2,1,0,0,0,3 +2010,20657,TOR,0,0,2,0,0,2 +2010,20658,DAL,1,3,0,0,0,4 +2010,20658,NYI,2,1,0,0,0,3 +2010,20659,CGY,1,0,1,0,0,2 +2010,20659,CBJ,1,1,1,0,0,3 +2010,20660,BOS,0,0,1,0,0,1 +2010,20660,NYR,1,1,1,0,0,3 +2010,20661,BUF,0,1,2,0,0,3 +2010,20661,COL,2,1,0,0,1,4 +2010,20662,PIT,1,2,1,0,0,4 +2010,20662,TOR,1,0,0,0,0,1 +2010,20663,MTL,0,1,0,0,0,1 +2010,20663,NJD,1,0,0,1,0,2 +2010,20664,FLA,1,2,0,0,0,3 +2010,20664,OTT,0,0,0,0,0,0 +2010,20665,PHI,1,0,3,0,0,4 +2010,20665,TBL,0,0,1,0,0,1 +2010,20666,ATL,0,1,0,0,0,1 +2010,20666,WSH,3,1,4,0,0,8 +2010,20667,ANA,1,2,0,0,0,3 +2010,20667,NSH,0,0,2,0,0,2 +2010,20668,CHI,3,2,0,0,0,5 +2010,20668,MIN,1,0,4,0,1,6 +2010,20669,NYI,2,2,0,0,1,5 +2010,20669,PHX,1,0,3,0,0,4 +2010,20670,CGY,1,1,0,0,1,3 +2010,20670,VAN,1,1,0,0,0,2 +2010,20671,LAK,0,1,2,0,0,3 +2010,20671,STL,1,1,2,0,0,4 +2010,20672,DET,0,3,1,0,0,4 +2010,20672,SJS,1,0,0,0,0,1 +2010,20673,CAR,0,2,2,0,0,4 +2010,20673,OTT,0,1,0,0,0,1 +2010,20674,CBJ,1,0,1,0,0,2 +2010,20674,DAL,0,0,0,0,0,0 +2010,20675,ANA,2,0,1,0,0,3 +2010,20675,CHI,0,0,1,0,0,1 +2010,20676,MIN,1,2,1,0,0,4 +2010,20676,PIT,1,1,1,0,0,3 +2010,20677,CGY,1,1,0,0,0,2 +2010,20677,COL,0,2,0,0,1,3 +2010,20678,NSH,2,0,1,0,0,3 +2010,20678,VAN,1,1,0,0,0,2 +2010,20679,LAK,0,1,0,0,0,1 +2010,20679,SJS,1,1,0,0,0,2 +2010,20680,CAR,1,1,2,0,0,4 +2010,20680,TOR,1,1,0,0,0,2 +2010,20681,DET,0,0,0,0,0,0 +2010,20681,NYI,2,2,2,0,0,6 +2010,20682,NJD,0,0,0,0,1,1 +2010,20682,NYR,0,0,0,0,0,0 +2010,20683,DAL,0,1,2,0,0,3 +2010,20683,PHI,1,4,1,0,0,6 +2010,20684,ATL,3,2,1,0,0,6 +2010,20684,OTT,1,0,0,0,0,1 +2010,20685,TBL,4,1,2,0,0,7 +2010,20685,WSH,1,3,0,0,0,4 +2010,20686,CBJ,0,1,0,0,0,1 +2010,20686,STL,3,0,1,0,0,4 +2010,20687,EDM,0,0,3,0,0,3 +2010,20687,NSH,3,1,1,0,0,5 +2010,20688,PHX,0,0,1,0,0,1 +2010,20688,SJS,2,1,0,0,0,3 +2010,20689,MIN,2,0,3,0,0,5 +2010,20689,VAN,1,1,0,0,0,2 +2010,20690,FLA,2,2,0,0,0,4 +2010,20690,WSH,0,2,2,0,1,5 +2010,20691,CGY,0,1,0,0,0,1 +2010,20691,PIT,1,1,1,0,0,3 +2010,20692,ANA,1,1,2,0,0,4 +2010,20692,BOS,0,3,0,0,0,3 +2010,20693,PHI,0,0,0,0,0,0 +2010,20693,TOR,0,2,2,0,0,4 +2010,20694,NYR,0,0,0,0,0,0 +2010,20694,OTT,0,0,2,0,0,2 +2010,20695,ATL,0,1,0,0,0,1 +2010,20695,BUF,0,1,0,1,0,2 +2010,20696,MTL,1,2,2,0,0,5 +2010,20696,DAL,2,1,0,0,0,3 +2010,20697,FLA,1,0,2,0,0,3 +2010,20697,TBL,2,0,0,0,0,2 +2010,20698,CAR,0,0,1,0,0,1 +2010,20698,DET,1,1,1,0,0,3 +2010,20699,MIN,0,0,0,0,0,0 +2010,20699,STL,0,0,1,0,0,1 +2010,20700,CHI,1,1,1,0,0,3 +2010,20700,CBJ,0,0,0,0,0,0 +2010,20701,NJD,0,2,1,0,0,3 +2010,20701,PHX,2,0,2,0,0,4 +2010,20702,EDM,0,2,0,0,0,2 +2010,20702,PIT,0,0,3,0,0,3 +2010,20703,ANA,0,0,0,0,0,0 +2010,20703,LAK,1,3,0,0,0,4 +2010,20704,BOS,0,1,0,0,1,2 +2010,20704,SJS,0,1,0,0,0,1 +2010,20705,TOR,0,1,0,0,0,1 +2010,20705,WSH,2,2,2,0,0,6 +2010,20706,CGY,0,0,0,0,0,0 +2010,20706,NSH,0,0,1,0,0,1 +2010,20707,CHI,3,0,3,0,0,6 +2010,20707,CBJ,2,2,1,0,0,5 +2010,20708,DAL,0,1,1,0,1,3 +2010,20708,DET,2,0,0,0,0,2 +2010,20709,COL,2,0,1,0,0,3 +2010,20709,NJD,0,0,1,0,0,1 +2010,20710,BOS,1,2,0,0,0,3 +2010,20710,LAK,1,0,2,0,1,4 +2010,20711,EDM,1,0,1,0,0,2 +2010,20711,SJS,2,2,0,0,0,4 +2010,20712,MTL,1,1,0,0,0,2 +2010,20712,OTT,1,2,1,0,0,4 +2010,20713,BUF,0,1,1,0,0,2 +2010,20713,NYI,1,1,0,0,1,3 +2010,20714,FLA,3,2,0,0,0,5 +2010,20714,TBL,2,0,0,0,0,2 +2010,20715,ATL,1,2,2,0,0,5 +2010,20715,CAR,1,0,2,0,0,3 +2010,20716,NYR,0,1,0,0,0,1 +2010,20716,STL,1,0,3,0,0,4 +2010,20717,MIN,0,1,3,0,0,4 +2010,20717,PHX,2,0,4,0,0,6 +2010,20718,PIT,1,1,0,0,0,2 +2010,20718,VAN,2,4,0,0,0,6 +2010,20719,PHI,2,0,1,0,0,3 +2010,20719,WSH,2,1,2,0,0,5 +2010,20720,CHI,2,1,0,0,1,4 +2010,20720,DET,1,1,1,0,0,3 +2010,20721,MTL,2,0,0,0,0,2 +2010,20721,NYR,0,3,3,0,0,6 +2010,20722,ANA,3,0,2,0,0,5 +2010,20722,CGY,2,2,0,0,0,4 +2010,20723,BOS,0,0,1,0,0,1 +2010,20723,OTT,2,2,1,0,0,5 +2010,20724,NJD,0,0,0,0,0,0 +2010,20724,NYI,2,2,0,0,0,4 +2010,20725,CAR,0,1,1,0,0,2 +2010,20725,TBL,1,1,1,0,0,3 +2010,20726,CBJ,2,1,1,0,0,4 +2010,20726,STL,1,1,0,0,0,2 +2010,20727,BUF,1,4,2,0,0,7 +2010,20727,PHX,0,1,1,0,0,2 +2010,20728,ATL,0,0,0,0,0,0 +2010,20728,FLA,0,1,0,0,0,1 +2010,20729,NSH,0,1,2,0,0,3 +2010,20729,TOR,3,0,1,0,0,4 +2010,20730,DAL,1,3,0,0,0,4 +2010,20730,MIN,0,1,2,0,0,3 +2010,20731,COL,2,2,2,0,0,6 +2010,20731,EDM,0,0,0,0,0,0 +2010,20732,CGY,0,0,1,0,0,1 +2010,20732,SJS,3,3,3,0,0,9 +2010,20733,NYR,3,2,3,0,0,8 +2010,20733,TBL,1,1,0,0,0,2 +2010,20734,CBJ,0,2,1,0,0,3 +2010,20734,PHI,1,3,1,0,0,5 +2010,20735,DET,0,1,1,0,0,2 +2010,20735,WSH,0,1,2,0,0,3 +2010,20736,ATL,0,2,2,0,0,4 +2010,20736,TOR,2,0,1,0,0,3 +2010,20737,CHI,0,1,0,0,0,1 +2010,20737,OTT,2,1,1,0,0,4 +2010,20738,NYI,0,2,2,0,0,4 +2010,20738,PIT,1,3,2,0,0,6 +2010,20739,ANA,4,0,1,0,0,5 +2010,20739,BUF,0,3,1,0,0,4 +2010,20740,LAK,0,1,0,0,0,1 +2010,20740,SJS,3,1,1,0,0,5 +2010,20741,FLA,0,0,0,0,0,0 +2010,20741,NJD,0,1,1,0,0,2 +2010,20742,MTL,1,0,2,0,0,3 +2010,20742,STL,3,0,0,1,0,4 +2010,20743,EDM,1,1,0,0,0,2 +2010,20743,VAN,1,0,1,1,0,3 +2010,20744,BOS,1,1,0,0,0,2 +2010,20744,CBJ,1,0,2,0,0,3 +2010,20745,OTT,1,1,1,0,0,3 +2010,20745,STL,0,1,1,0,0,2 +2010,20746,FLA,0,1,0,0,0,1 +2010,20746,NYI,0,1,0,0,1,2 +2010,20747,NYR,0,0,0,0,0,0 +2010,20747,PHI,1,1,0,0,0,2 +2010,20748,ATL,0,1,1,0,0,2 +2010,20748,CAR,2,2,1,0,0,5 +2010,20749,TBL,0,1,1,1,0,3 +2010,20749,TOR,0,1,1,0,0,2 +2010,20750,PIT,1,2,0,0,0,3 +2010,20750,WSH,1,2,3,0,0,6 +2010,20751,DET,1,0,2,0,1,4 +2010,20751,MIN,0,1,2,0,0,3 +2010,20752,NSH,0,2,0,0,0,2 +2010,20752,PHX,0,1,3,0,0,4 +2010,20753,CGY,0,1,0,0,0,1 +2010,20753,CHI,0,1,2,0,0,3 +2010,20754,DAL,1,1,1,0,0,3 +2010,20754,VAN,1,1,2,0,0,4 +2010,20755,BUF,0,2,1,0,0,3 +2010,20755,LAK,1,1,1,0,1,4 +2010,20756,ANA,0,0,1,0,0,1 +2010,20756,SJS,0,2,1,0,0,3 +2010,20757,MTL,1,1,1,0,0,3 +2010,20757,NJD,1,0,0,0,0,1 +2010,20758,COL,1,1,0,0,0,2 +2010,20758,NSH,0,1,0,0,0,1 +2010,20759,DAL,2,1,1,0,0,4 +2010,20759,EDM,1,1,1,0,0,3 +2010,20760,BOS,0,1,0,0,0,1 +2010,20760,OTT,1,1,0,0,0,2 +2010,20761,CAR,1,0,1,0,0,2 +2010,20761,PHI,1,2,1,0,0,4 +2010,20762,MTL,1,4,1,0,0,6 +2010,20762,NYR,0,0,0,0,0,0 +2010,20763,NJD,1,1,2,0,0,4 +2010,20763,NYI,0,2,0,0,0,2 +2010,20764,PHX,1,0,1,0,0,2 +2010,20764,WSH,1,2,1,0,0,4 +2010,20765,FLA,0,1,1,0,0,2 +2010,20765,TOR,0,0,0,0,0,0 +2010,20766,DET,2,0,0,0,0,2 +2010,20766,LAK,0,1,2,0,0,3 +2010,20767,ATL,1,0,0,0,0,1 +2010,20767,TBL,1,0,0,0,1,2 +2010,20768,ANA,0,0,3,0,1,4 +2010,20768,STL,0,2,1,0,0,3 +2010,20769,CBJ,0,0,2,0,0,2 +2010,20769,MIN,1,2,1,0,0,4 +2010,20770,CHI,0,0,1,0,0,1 +2010,20770,VAN,3,0,2,0,0,5 +2010,20771,BUF,1,0,1,0,0,2 +2010,20771,SJS,0,2,3,0,0,5 +2010,20772,PHI,1,0,0,0,0,1 +2010,20772,PIT,1,0,1,0,0,2 +2010,20773,BOS,0,0,1,0,0,1 +2010,20773,CAR,1,3,1,0,0,5 +2010,20774,COL,2,1,1,0,0,4 +2010,20774,DAL,0,0,0,0,0,0 +2010,20775,NYR,0,1,1,0,0,2 +2010,20775,PIT,1,0,3,0,0,4 +2010,20776,CGY,0,0,0,0,0,0 +2010,20776,STL,0,1,1,0,0,2 +2010,20777,BUF,0,1,1,0,0,2 +2010,20777,VAN,0,2,1,0,0,3 +2010,20778,LAK,1,3,1,0,0,5 +2010,20778,TOR,1,1,1,0,0,3 +2010,20779,NYI,1,1,0,0,0,2 +2010,20779,WSH,4,2,1,0,0,7 +2010,20780,ANA,0,1,0,0,0,1 +2010,20780,ATL,0,1,1,0,0,2 +2010,20781,CBJ,0,1,2,0,0,3 +2010,20781,NSH,0,0,2,0,0,2 +2010,20782,NJD,0,0,0,0,0,0 +2010,20782,OTT,2,1,0,0,0,3 +2010,20783,MTL,0,1,0,0,0,1 +2010,20783,FLA,0,0,2,0,0,2 +2010,20784,DET,1,1,2,0,0,4 +2010,20784,PHX,0,2,2,1,0,5 +2010,20785,CHI,1,3,0,0,0,4 +2010,20785,EDM,0,1,1,0,0,2 +2010,20786,BUF,1,0,0,0,1,2 +2010,20786,NJD,0,1,0,0,0,1 +2010,20787,CAR,2,1,2,0,0,5 +2010,20787,NYR,0,1,0,0,0,1 +2010,20788,ANA,0,1,0,0,0,1 +2010,20788,WSH,1,0,4,0,0,5 +2010,20789,MTL,0,0,0,0,0,0 +2010,20789,TBL,1,2,0,0,0,3 +2010,20790,DET,2,0,0,0,0,2 +2010,20790,MIN,3,0,2,0,0,5 +2010,20791,CGY,2,1,0,0,0,3 +2010,20791,DAL,1,1,1,0,1,4 +2010,20792,STL,1,0,1,0,0,2 +2010,20792,VAN,1,0,2,0,0,3 +2010,20793,ATL,1,0,3,0,0,4 +2010,20793,PHI,0,3,0,0,0,3 +2010,20794,OTT,2,0,2,0,0,4 +2010,20794,PIT,1,0,0,0,0,1 +2010,20795,CAR,2,0,2,0,0,4 +2010,20795,NYI,0,0,1,0,0,1 +2010,20796,CBJ,0,0,1,0,0,1 +2010,20796,LAK,1,1,2,0,0,4 +2010,20797,COL,0,0,0,0,0,0 +2010,20797,MIN,0,0,1,0,0,1 +2010,20798,CGY,0,1,1,0,0,2 +2010,20798,PHX,0,2,0,0,1,3 +2010,20799,EDM,0,1,0,0,0,1 +2010,20799,STL,0,1,1,0,0,2 +2010,20800,CHI,3,0,0,1,0,4 +2010,20800,SJS,1,1,1,0,0,3 +2010,20801,NJD,2,2,0,1,0,5 +2010,20801,TOR,1,1,2,0,0,4 +2010,20802,FLA,0,1,0,0,0,1 +2010,20802,WSH,1,1,2,0,0,4 +2010,20803,BOS,0,1,0,0,0,1 +2010,20803,BUF,2,0,0,0,0,2 +2010,20804,ANA,0,1,0,0,1,2 +2010,20804,TBL,1,0,0,0,0,1 +2010,20805,DET,1,2,1,0,0,4 +2010,20805,NSH,1,0,1,0,0,2 +2010,20806,COL,1,1,0,0,0,2 +2010,20806,DAL,2,0,1,0,0,3 +2010,20807,MTL,0,1,1,0,0,2 +2010,20807,OTT,1,1,0,1,0,3 +2010,20808,BOS,0,1,1,0,0,2 +2010,20808,LAK,1,0,1,0,1,3 +2010,20809,TOR,3,0,0,0,0,3 +2010,20809,VAN,0,1,4,0,0,5 +2010,20810,NYI,0,1,0,0,0,1 +2010,20810,PHI,2,0,0,0,0,2 +2010,20811,CAR,1,1,2,0,0,4 +2010,20811,CHI,0,1,1,0,0,2 +2010,20812,CBJ,1,1,0,1,0,3 +2010,20812,STL,1,0,1,0,0,2 +2010,20813,ATL,1,2,0,0,0,3 +2010,20813,NSH,2,1,1,0,0,4 +2010,20814,NYR,0,0,2,0,0,2 +2010,20814,PHX,3,0,0,0,0,3 +2010,20815,CGY,2,3,1,0,0,6 +2010,20815,EDM,1,0,0,0,0,1 +2010,20816,MIN,2,0,0,0,0,2 +2010,20816,SJS,1,2,2,0,0,5 +2010,20817,DET,0,0,1,0,0,1 +2010,20817,PIT,0,1,0,0,1,2 +2010,20818,TBL,0,0,2,0,0,2 +2010,20818,WSH,0,2,1,0,0,3 +2010,20819,LAK,0,1,2,0,0,3 +2010,20819,NJD,1,1,0,0,0,2 +2010,20820,FLA,1,1,0,0,0,2 +2010,20820,NYI,0,0,0,0,0,0 +2010,20821,DAL,0,1,1,0,0,2 +2010,20821,PHX,1,1,2,0,0,4 +2010,20822,COL,0,0,1,0,0,1 +2010,20822,NYR,1,1,1,0,0,3 +2010,20823,BUF,2,1,1,0,0,4 +2010,20823,PIT,1,4,0,0,0,5 +2010,20824,ANA,0,2,1,0,0,3 +2010,20824,FLA,0,0,0,0,0,0 +2010,20825,CGY,0,0,0,0,0,0 +2010,20825,PHI,0,1,2,0,0,3 +2010,20826,CAR,1,1,0,0,0,2 +2010,20826,EDM,2,2,0,0,0,4 +2010,20827,BOS,0,0,0,0,0,0 +2010,20827,WSH,0,0,0,0,0,0 +2010,20828,NJD,0,0,0,0,0,0 +2010,20828,TOR,1,2,0,0,0,3 +2010,20829,ATL,0,1,0,0,0,1 +2010,20829,TBL,2,0,0,0,0,2 +2010,20830,MTL,2,0,1,0,0,3 +2010,20830,VAN,1,0,1,0,0,2 +2010,20831,NSH,0,0,0,0,0,0 +2010,20831,PHX,0,0,0,0,1,1 +2010,20832,DAL,1,2,1,0,0,4 +2010,20832,MIN,0,1,1,0,0,2 +2010,20833,COL,1,1,3,0,0,5 +2010,20833,CBJ,1,0,0,0,0,1 +2010,20834,LAK,0,1,1,0,0,2 +2010,20834,NYR,0,0,1,0,0,1 +2010,20835,DET,2,0,2,0,0,4 +2010,20835,SJS,2,0,0,0,0,2 +2010,20836,BUF,0,0,0,0,0,0 +2010,20836,OTT,0,0,0,0,0,0 +2010,20837,CHI,1,0,1,0,0,2 +2010,20837,STL,0,3,0,0,0,3 +2010,20838,CGY,1,2,1,0,0,4 +2010,20838,CAR,0,1,0,0,0,1 +2010,20839,EDM,0,0,1,0,0,1 +2010,20839,PHI,0,0,0,0,0,0 +2010,20840,ANA,2,1,0,0,0,3 +2010,20840,DET,0,1,0,0,0,1 +2010,20841,BOS,1,1,0,0,0,2 +2010,20841,MTL,0,2,0,0,1,3 +2010,20842,NYR,2,3,0,0,0,5 +2010,20842,WSH,1,3,2,0,0,6 +2010,20843,CBJ,1,0,1,0,0,2 +2010,20843,DAL,0,0,1,0,0,1 +2010,20844,OTT,2,0,1,0,0,3 +2010,20844,VAN,0,0,1,0,0,1 +2010,20845,NYI,1,1,0,0,0,2 +2010,20845,TBL,1,2,2,0,0,5 +2010,20846,SJS,2,1,1,0,0,4 +2010,20846,STL,1,0,1,0,0,2 +2010,20847,COL,1,1,1,0,0,3 +2010,20847,NSH,0,4,1,0,0,5 +2010,20848,EDM,0,1,1,0,0,2 +2010,20848,MIN,0,1,3,0,0,4 +2010,20849,ANA,1,0,3,0,0,4 +2010,20849,LAK,1,2,3,0,0,6 +2010,20850,NJD,1,0,3,0,0,4 +2010,20850,TOR,0,3,0,0,0,3 +2010,20851,ATL,1,0,1,0,0,2 +2010,20851,WSH,1,1,3,0,0,5 +2010,20852,BUF,1,0,2,0,0,3 +2010,20852,CAR,1,0,3,0,0,4 +2010,20853,CGY,1,1,0,0,0,2 +2010,20853,FLA,0,1,0,0,0,1 +2010,20854,CHI,1,0,0,0,0,1 +2010,20854,PHX,0,1,0,0,1,2 +2010,20855,BOS,2,0,0,0,0,2 +2010,20855,VAN,0,1,1,0,1,3 +2010,20856,MTL,1,2,2,0,0,5 +2010,20856,PIT,1,1,1,0,0,3 +2010,20857,DET,0,0,0,0,0,0 +2010,20857,LAK,0,0,0,0,0,0 +2010,20858,DAL,2,2,0,0,0,4 +2010,20858,PHX,0,0,0,0,0,0 +2010,20859,OTT,0,0,0,0,0,0 +2010,20859,TOR,3,1,1,0,0,5 +2010,20860,CAR,1,1,1,0,0,3 +2010,20860,NYI,0,1,0,0,0,1 +2010,20861,NJD,0,0,1,0,0,1 +2010,20861,NYR,0,3,0,0,0,3 +2010,20862,ATL,0,2,2,0,0,4 +2010,20862,FLA,2,0,0,0,0,2 +2010,20863,CGY,0,0,0,0,0,0 +2010,20863,TBL,0,0,0,0,0,0 +2010,20864,BUF,0,0,0,0,0,0 +2010,20864,CBJ,0,2,2,0,0,4 +2010,20865,CHI,2,0,0,0,0,2 +2010,20865,STL,0,0,1,0,0,1 +2010,20866,NSH,1,2,0,0,0,3 +2010,20866,SJS,1,2,1,0,0,4 +2010,20867,MIN,1,1,0,0,0,2 +2010,20867,PHI,1,0,0,0,0,1 +2010,20868,COL,0,1,2,0,0,3 +2010,20868,EDM,0,0,0,0,0,0 +2010,20869,BOS,2,0,1,0,0,3 +2010,20869,MTL,0,0,0,0,0,0 +2010,20870,PIT,2,2,0,0,0,4 +2010,20870,WSH,0,2,2,1,0,5 +2010,20871,SJS,0,2,1,0,0,3 +2010,20871,TOR,1,1,0,0,0,2 +2010,20872,NJD,1,1,0,0,0,2 +2010,20872,PHI,0,2,1,0,0,3 +2010,20873,COL,2,3,0,0,0,5 +2010,20873,STL,2,0,0,0,0,2 +2010,20874,EDM,0,0,0,0,0,0 +2010,20874,PHX,0,0,0,0,0,0 +2010,20875,ANA,1,3,0,0,0,4 +2010,20875,LAK,1,1,0,0,0,2 +2010,20876,BOS,2,0,0,0,1,3 +2010,20876,BUF,0,2,0,0,0,2 +2010,20877,NSH,0,2,1,0,0,3 +2010,20877,NYI,0,2,1,0,1,4 +2010,20878,CAR,0,2,2,0,0,4 +2010,20878,FLA,0,1,0,0,0,1 +2010,20879,CGY,1,1,0,0,0,2 +2010,20879,OTT,1,2,0,0,0,3 +2010,20880,TBL,1,0,2,0,0,3 +2010,20880,VAN,0,1,0,0,0,1 +2010,20881,DET,1,0,2,0,0,3 +2010,20881,STL,0,2,1,0,1,4 +2010,20882,CHI,2,0,1,0,1,4 +2010,20882,DAL,2,0,1,0,0,3 +2010,20883,NJD,2,0,0,0,0,2 +2010,20883,PHI,1,1,0,1,0,3 +2010,20884,NSH,1,1,0,0,0,2 +2010,20884,NYR,0,1,0,0,0,1 +2010,20885,CBJ,1,1,1,0,0,3 +2010,20885,SJS,0,0,0,0,0,0 +2010,20886,MTL,1,4,0,1,0,6 +2010,20886,WSH,1,1,3,0,0,5 +2010,20887,NYI,0,0,1,0,0,1 +2010,20887,PIT,0,2,1,0,0,3 +2010,20888,MIN,1,0,1,0,0,2 +2010,20888,PHX,1,1,1,0,0,3 +2010,20889,ATL,1,1,1,0,0,3 +2010,20889,COL,1,0,2,1,0,4 +2010,20890,ANA,1,1,1,0,0,3 +2010,20890,EDM,0,2,0,0,0,2 +2010,20891,OTT,3,1,2,0,0,6 +2010,20891,WSH,1,3,1,0,0,5 +2010,20892,BUF,2,0,1,0,0,3 +2010,20892,CAR,2,1,0,1,0,4 +2010,20893,BOS,4,1,0,0,0,5 +2010,20893,TBL,0,2,2,0,0,4 +2010,20894,FLA,0,0,0,0,0,0 +2010,20894,VAN,0,2,1,0,0,3 +2010,20895,DET,1,0,1,0,0,2 +2010,20895,SJS,2,0,0,0,1,3 +2010,20896,CGY,1,0,0,0,0,1 +2010,20896,DAL,0,1,2,0,0,3 +2010,20897,EDM,0,1,1,0,1,3 +2010,20897,LAK,0,0,2,0,0,2 +2010,20898,NSH,1,1,0,0,0,2 +2010,20898,NJD,1,1,3,0,0,5 +2010,20899,MTL,0,0,2,0,0,2 +2010,20899,PHI,1,2,0,0,0,3 +2010,20900,CBJ,2,1,0,0,0,3 +2010,20900,VAN,1,1,2,0,0,4 +2010,20901,NYR,0,2,0,1,0,3 +2010,20901,PIT,1,0,1,0,0,2 +2010,20902,STL,1,2,1,0,0,4 +2010,20902,TOR,0,0,0,0,0,0 +2010,20903,ATL,2,0,1,0,0,3 +2010,20903,MIN,2,0,0,0,0,2 +2010,20904,COL,1,0,1,0,0,2 +2010,20904,PHX,0,1,0,0,0,1 +2010,20905,NYI,1,3,1,0,0,5 +2010,20905,TBL,2,1,1,0,0,4 +2010,20906,BUF,0,3,0,0,0,3 +2010,20906,SJS,0,0,1,0,0,1 +2010,20907,MTL,0,1,1,0,0,2 +2010,20907,PHI,3,2,1,0,0,6 +2010,20908,CAR,1,2,2,0,0,5 +2010,20908,NJD,0,0,2,0,0,2 +2010,20909,BOS,1,0,1,0,1,3 +2010,20909,FLA,2,0,0,0,0,2 +2010,20910,DET,2,1,1,0,0,4 +2010,20910,OTT,0,1,0,0,0,1 +2010,20911,STL,1,2,0,0,1,4 +2010,20911,WSH,1,2,0,0,0,3 +2010,20912,DAL,0,2,1,0,0,3 +2010,20912,PHX,0,0,0,0,0,0 +2010,20913,ATL,1,3,0,0,0,4 +2010,20913,CHI,3,0,1,0,1,5 +2010,20914,ANA,1,0,0,0,0,1 +2010,20914,CGY,1,2,0,0,0,3 +2010,20915,COL,0,0,0,0,0,0 +2010,20915,LAK,0,1,2,0,0,3 +2010,20916,NYR,0,4,1,0,0,5 +2010,20916,TBL,2,0,0,0,0,2 +2010,20917,NSH,0,2,1,0,1,4 +2010,20917,PIT,1,1,1,0,0,3 +2010,20918,MIN,1,2,3,0,0,6 +2010,20918,VAN,0,0,2,0,0,2 +2010,20919,NYI,1,2,0,0,0,3 +2010,20919,OTT,1,0,3,0,0,4 +2010,20920,CHI,1,3,0,0,1,5 +2010,20920,CBJ,2,1,1,0,0,4 +2010,20921,ANA,2,1,4,0,0,7 +2010,20921,EDM,1,1,1,0,0,3 +2010,20922,COL,0,2,0,0,0,2 +2010,20922,DET,2,0,1,0,0,3 +2010,20923,BOS,1,0,0,0,0,1 +2010,20923,MTL,0,0,4,0,0,4 +2010,20924,CAR,1,3,1,0,0,5 +2010,20924,TOR,0,0,1,0,0,1 +2010,20925,CHI,2,0,1,0,0,3 +2010,20925,NYI,1,4,0,0,0,5 +2010,20926,ATL,2,2,0,0,0,4 +2010,20926,FLA,2,0,0,0,0,2 +2010,20927,CBJ,2,1,0,0,0,3 +2010,20927,VAN,0,2,1,1,0,4 +2010,20928,NYR,0,4,0,0,0,4 +2010,20928,OTT,1,0,0,0,0,1 +2010,20929,BUF,0,1,1,0,0,2 +2010,20929,PIT,1,1,1,0,0,3 +2010,20930,PHI,1,1,5,0,0,7 +2010,20930,TBL,1,1,0,0,0,2 +2010,20931,EDM,1,1,1,0,0,3 +2010,20931,NSH,2,1,1,0,0,4 +2010,20932,DAL,0,0,1,0,0,1 +2010,20932,LAK,1,3,1,0,0,5 +2010,20933,PHX,1,0,1,0,0,2 +2010,20933,STL,2,3,0,0,0,5 +2010,20934,NJD,0,3,1,0,0,4 +2010,20934,SJS,0,0,3,0,0,3 +2010,20935,BUF,0,1,0,0,0,1 +2010,20935,WSH,0,1,2,0,0,3 +2010,20936,FLA,2,4,1,0,0,7 +2010,20936,PHI,0,3,1,0,0,4 +2010,20937,DET,1,1,1,0,0,3 +2010,20937,VAN,2,3,1,0,0,6 +2010,20938,CHI,0,2,3,0,0,5 +2010,20938,EDM,0,2,0,0,0,2 +2010,20939,CGY,0,0,0,0,0,0 +2010,20939,MIN,1,2,1,0,0,4 +2010,20940,ANA,1,1,1,0,0,3 +2010,20940,COL,0,2,2,0,0,4 +2010,20941,BOS,1,1,0,0,1,3 +2010,20941,TOR,1,0,1,0,0,2 +2010,20942,NYR,2,2,0,0,0,4 +2010,20942,PIT,2,1,1,1,0,5 +2010,20943,TBL,1,1,2,0,0,4 +2010,20943,WSH,1,2,2,0,0,5 +2010,20944,CAR,2,1,1,0,0,4 +2010,20944,OTT,0,0,1,0,0,1 +2010,20945,ATL,4,2,0,0,0,6 +2010,20945,NYI,1,0,2,0,0,3 +2010,20946,LAK,1,0,1,0,0,2 +2010,20946,NSH,1,1,2,0,0,4 +2010,20947,DAL,1,0,0,0,0,1 +2010,20947,STL,3,1,2,0,0,6 +2010,20948,COL,0,1,0,0,0,1 +2010,20948,PHX,1,0,2,0,0,3 +2010,20949,MTL,1,1,0,0,0,2 +2010,20949,SJS,0,1,2,0,0,3 +2010,20950,BUF,0,1,1,1,0,3 +2010,20950,PHI,0,1,1,0,0,2 +2010,20951,DET,2,2,1,0,0,5 +2010,20951,NSH,0,1,1,0,0,2 +2010,20952,CHI,5,1,0,0,0,6 +2010,20952,VAN,1,1,1,0,0,3 +2010,20953,CGY,0,3,2,0,0,5 +2010,20953,NJD,1,0,2,0,0,3 +2010,20954,EDM,1,0,0,0,1,2 +2010,20954,MIN,0,1,0,0,0,1 +2010,20955,DAL,1,2,0,0,0,3 +2010,20955,PIT,1,3,2,0,0,6 +2010,20956,BOS,1,2,0,0,0,3 +2010,20956,NYI,0,1,1,0,0,2 +2010,20957,OTT,1,0,0,0,0,1 +2010,20957,TOR,0,1,0,0,1,2 +2010,20958,NYR,0,0,0,0,0,0 +2010,20958,WSH,1,1,0,0,0,2 +2010,20959,CAR,0,0,1,0,0,1 +2010,20959,FLA,2,0,2,0,0,4 +2010,20960,ATL,1,1,0,0,0,2 +2010,20960,TBL,3,1,2,0,0,6 +2010,20961,ANA,0,0,0,0,0,0 +2010,20961,PHX,1,2,1,0,0,4 +2010,20962,COL,2,4,1,0,0,7 +2010,20962,STL,1,1,1,0,0,3 +2010,20963,MTL,1,1,2,0,0,4 +2010,20963,LAK,1,0,1,0,0,2 +2010,20964,CBJ,0,1,0,0,0,1 +2010,20964,SJS,0,0,2,0,0,2 +2010,20965,BOS,0,1,0,0,0,1 +2010,20965,PIT,0,1,1,0,0,2 +2010,20966,NSH,1,1,0,0,0,2 +2010,20966,VAN,1,0,3,0,0,4 +2010,20967,CGY,2,1,2,0,0,5 +2010,20967,MIN,0,2,0,0,0,2 +2010,20968,ATL,0,0,0,0,0,0 +2010,20968,CAR,1,1,2,0,0,4 +2010,20969,BUF,0,0,1,1,0,2 +2010,20969,NYR,0,0,1,0,0,1 +2010,20970,PHI,1,2,0,0,0,3 +2010,20970,TOR,0,1,0,0,0,1 +2010,20971,CHI,2,0,2,0,0,4 +2010,20971,DET,0,5,0,0,0,5 +2010,20972,EDM,0,1,1,0,0,2 +2010,20972,NJD,0,0,0,0,0,0 +2010,20973,ANA,3,0,0,0,0,3 +2010,20973,MTL,0,1,2,0,1,4 +2010,20974,DAL,0,0,3,0,1,4 +2010,20974,WSH,1,1,1,0,0,3 +2010,20975,CBJ,0,0,0,0,0,0 +2010,20975,LAK,4,2,0,0,0,6 +2010,20976,BOS,1,1,1,0,0,3 +2010,20976,TOR,0,2,1,1,0,4 +2010,20977,NYI,1,1,0,0,0,2 +2010,20977,PHI,0,1,2,0,0,3 +2010,20978,ATL,0,1,0,0,0,1 +2010,20978,NSH,2,0,0,0,0,2 +2010,20979,MTL,2,2,1,0,0,5 +2010,20979,TBL,0,2,1,0,0,3 +2010,20980,CGY,0,1,3,0,0,4 +2010,20980,DET,0,2,0,0,0,2 +2010,20981,FLA,0,0,2,0,1,3 +2010,20981,MIN,2,0,0,0,0,2 +2010,20982,COL,3,1,0,0,0,4 +2010,20982,VAN,0,3,3,0,0,6 +2010,20983,EDM,0,1,0,0,0,1 +2010,20983,OTT,0,1,3,0,0,4 +2010,20984,ANA,0,0,2,0,0,2 +2010,20984,CBJ,1,2,2,0,0,5 +2010,20985,BUF,3,1,1,0,0,5 +2010,20985,DAL,3,0,0,0,0,3 +2010,20986,NJD,2,3,1,0,0,6 +2010,20986,NYR,1,2,0,0,0,3 +2010,20987,CAR,1,2,0,0,0,3 +2010,20987,WSH,2,1,0,1,0,4 +2010,20988,CHI,1,1,0,1,0,3 +2010,20988,LAK,0,2,0,0,0,2 +2010,20989,PHX,1,1,1,0,1,4 +2010,20989,VAN,1,1,1,0,0,3 +2010,20990,TBL,1,1,1,0,0,3 +2010,20990,TOR,0,2,1,1,0,4 +2010,20991,NYI,1,0,0,0,0,1 +2010,20991,STL,0,1,0,0,1,2 +2010,20992,BOS,1,3,1,0,0,5 +2010,20992,PHI,0,1,0,0,0,1 +2010,20993,CAR,2,1,0,1,0,4 +2010,20993,PIT,2,1,0,0,0,3 +2010,20994,ATL,0,0,1,0,0,1 +2010,20994,CBJ,1,1,0,0,0,2 +2010,20995,MTL,2,1,1,0,1,5 +2010,20995,EDM,2,0,2,0,0,4 +2010,20996,DET,3,1,1,0,0,5 +2010,20996,MIN,1,0,0,0,0,1 +2010,20997,COL,0,2,1,0,0,3 +2010,20997,FLA,0,0,0,0,0,0 +2010,20998,CGY,1,0,1,0,0,2 +2010,20998,OTT,0,0,0,0,0,0 +2010,20999,NSH,1,3,1,0,0,5 +2010,20999,SJS,1,1,6,0,0,8 +2010,21000,NJD,1,0,2,0,0,3 +2010,21000,PIT,1,0,0,0,0,1 +2010,21001,TBL,1,2,0,0,0,3 +2010,21001,WSH,1,0,1,0,0,2 +2010,21002,BUF,0,0,2,0,0,2 +2010,21002,MIN,2,1,0,0,0,3 +2010,21003,ATL,0,1,1,0,0,2 +2010,21003,NYR,3,1,1,0,0,5 +2010,21004,DAL,0,0,1,0,0,1 +2010,21004,LAK,1,0,0,0,1,2 +2010,21005,ANA,0,0,0,0,0,0 +2010,21005,NSH,1,0,0,0,0,1 +2010,21006,CHI,0,0,2,0,0,2 +2010,21006,PHI,0,0,3,0,0,3 +2010,21007,EDM,0,2,2,0,0,4 +2010,21007,TOR,2,3,1,0,0,6 +2010,21008,BOS,0,1,1,0,0,2 +2010,21008,MTL,2,0,1,0,0,3 +2010,21009,NJD,1,0,1,0,0,2 +2010,21009,NYI,1,2,1,0,0,4 +2010,21010,CAR,0,0,0,0,0,0 +2010,21010,PHX,2,1,1,0,0,4 +2010,21011,BUF,1,1,0,0,0,2 +2010,21011,DET,2,0,0,1,0,3 +2010,21012,CBJ,0,0,1,0,0,1 +2010,21012,STL,0,3,2,0,0,5 +2010,21013,OTT,0,1,0,0,0,1 +2010,21013,VAN,1,3,1,0,0,5 +2010,21014,FLA,0,1,1,1,0,3 +2010,21014,SJS,2,0,0,0,0,2 +2010,21015,NYR,0,2,1,0,0,3 +2010,21015,PHI,1,0,0,0,0,1 +2010,21016,COL,1,3,1,0,0,5 +2010,21016,DAL,1,2,0,0,0,3 +2010,21017,LAK,1,0,1,0,0,2 +2010,21017,NSH,1,0,2,0,0,3 +2010,21018,NYI,2,1,1,0,0,4 +2010,21018,TOR,1,0,0,0,0,1 +2010,21019,ATL,0,0,2,0,0,2 +2010,21019,PHX,0,1,1,0,1,3 +2010,21020,PIT,0,0,2,0,0,2 +2010,21020,TBL,0,1,0,0,0,1 +2010,21021,MIN,3,1,0,0,0,4 +2010,21021,STL,0,1,1,0,0,2 +2010,21022,CHI,2,1,0,0,0,3 +2010,21022,WSH,0,0,3,1,0,4 +2010,21023,ANA,3,0,1,0,0,4 +2010,21023,SJS,0,0,2,0,0,2 +2010,21024,CGY,0,0,1,0,0,1 +2010,21024,VAN,3,0,0,0,0,3 +2010,21025,BOS,0,1,1,0,0,2 +2010,21025,NJD,3,0,0,0,0,3 +2010,21026,CBJ,1,1,3,0,0,5 +2010,21026,EDM,0,1,2,0,0,3 +2010,21027,CGY,1,0,0,0,0,1 +2010,21027,DET,0,1,1,0,0,2 +2010,21028,MTL,1,0,2,0,0,3 +2010,21028,NYR,0,1,0,0,0,1 +2010,21029,BOS,1,1,3,0,0,5 +2010,21029,CAR,0,1,1,0,0,2 +2010,21030,ATL,3,0,1,0,0,4 +2010,21030,BUF,1,1,1,0,0,3 +2010,21031,OTT,0,1,0,0,0,1 +2010,21031,TOR,1,2,1,0,0,4 +2010,21032,PHX,2,0,0,0,0,2 +2010,21032,TBL,0,1,0,0,0,1 +2010,21033,FLA,1,1,1,0,0,3 +2010,21033,WSH,2,4,1,0,0,7 +2010,21034,COL,2,2,1,0,0,5 +2010,21034,STL,1,2,0,0,0,3 +2010,21035,NSH,3,0,0,0,1,4 +2010,21035,PHI,1,0,2,0,0,3 +2010,21036,EDM,0,1,1,0,0,2 +2010,21036,MIN,1,1,2,0,0,4 +2010,21037,DAL,1,3,4,0,0,8 +2010,21037,SJS,0,1,1,0,0,2 +2010,21038,NYI,2,2,1,0,0,5 +2010,21038,VAN,0,2,0,0,0,2 +2010,21039,NJD,2,2,1,0,0,5 +2010,21039,PIT,1,0,1,0,0,2 +2010,21040,CGY,2,1,0,0,0,3 +2010,21040,COL,0,1,1,0,0,2 +2010,21041,ANA,1,1,2,0,0,4 +2010,21041,CHI,1,0,1,0,0,2 +2010,21042,BOS,0,0,0,0,0,0 +2010,21042,PIT,1,1,1,0,0,3 +2010,21043,NYR,2,0,1,0,0,3 +2010,21043,STL,1,2,1,0,0,4 +2010,21044,CAR,0,1,2,1,0,4 +2010,21044,WSH,0,2,1,0,0,3 +2010,21045,ATL,1,2,3,0,0,6 +2010,21045,OTT,0,3,0,0,0,3 +2010,21046,NJD,0,1,0,0,0,1 +2010,21046,TOR,1,0,0,0,1,2 +2010,21047,BUF,3,1,2,0,0,6 +2010,21047,TBL,0,0,2,0,0,2 +2010,21048,FLA,1,2,0,0,0,3 +2010,21048,PHX,0,0,3,0,1,4 +2010,21049,MIN,0,0,0,0,0,0 +2010,21049,NSH,3,1,1,0,0,5 +2010,21050,DAL,2,0,0,0,0,2 +2010,21050,PHI,1,1,1,0,0,3 +2010,21051,SJS,0,1,1,0,0,2 +2010,21051,VAN,2,1,0,0,0,3 +2010,21052,CHI,0,1,2,0,0,3 +2010,21052,LAK,0,0,0,0,0,0 +2010,21053,CBJ,0,1,3,0,0,4 +2010,21053,MIN,1,1,0,0,0,2 +2010,21054,CGY,2,2,0,0,0,4 +2010,21054,SJS,0,1,2,0,0,3 +2010,21055,DET,0,1,1,0,0,2 +2010,21055,EDM,2,0,0,0,1,3 +2010,21056,ANA,2,0,2,1,0,5 +2010,21056,NYI,2,2,0,0,0,4 +2010,21057,CAR,1,0,1,1,0,3 +2010,21057,PIT,0,1,1,0,0,2 +2010,21058,DAL,2,2,1,0,0,5 +2010,21058,OTT,2,1,1,0,0,4 +2010,21059,MTL,1,0,1,0,0,2 +2010,21059,TOR,1,1,0,0,1,3 +2010,21060,NJD,0,0,0,0,0,0 +2010,21060,STL,1,0,0,0,0,1 +2010,21061,ATL,2,1,2,0,0,5 +2010,21061,PHI,1,1,0,0,0,2 +2010,21062,BUF,1,1,1,0,0,3 +2010,21062,FLA,0,1,0,0,0,1 +2010,21063,TBL,1,0,0,0,0,1 +2010,21063,WSH,2,0,1,0,0,3 +2010,21064,CBJ,0,0,0,0,0,0 +2010,21064,NSH,0,0,0,1,0,1 +2010,21065,CHI,1,1,2,0,0,4 +2010,21065,PHX,0,1,3,0,1,5 +2010,21066,DET,0,3,0,1,0,4 +2010,21066,VAN,2,0,1,0,0,3 +2010,21067,LAK,1,0,0,0,0,1 +2010,21067,NYI,0,0,0,0,0,0 +2010,21068,BOS,0,1,1,0,0,2 +2010,21068,NYR,0,0,1,0,0,1 +2010,21069,CGY,1,1,1,0,0,3 +2010,21069,MIN,2,2,0,0,0,4 +2010,21070,BUF,1,4,0,0,0,5 +2010,21070,CAR,0,1,2,0,0,3 +2010,21071,FLA,3,0,2,0,0,5 +2010,21071,TBL,1,0,1,0,0,2 +2010,21072,NSH,0,1,2,0,0,3 +2010,21072,STL,1,0,1,0,0,2 +2010,21073,ATL,2,0,1,0,0,3 +2010,21073,PHI,0,0,1,0,0,1 +2010,21074,DAL,0,1,1,0,0,2 +2010,21074,PHX,1,1,0,0,1,3 +2010,21075,EDM,2,1,2,0,0,5 +2010,21075,SJS,0,0,1,0,0,1 +2010,21076,ANA,2,2,1,0,0,5 +2010,21076,COL,0,0,2,0,0,2 +2010,21077,DET,1,1,1,0,0,3 +2010,21077,PIT,0,1,0,0,0,1 +2010,21078,MTL,0,0,0,0,0,0 +2010,21078,OTT,1,0,1,0,0,2 +2010,21079,COL,1,0,2,0,0,3 +2010,21079,LAK,1,1,1,1,0,4 +2010,21080,FLA,1,0,3,0,0,4 +2010,21080,TOR,0,0,1,0,0,1 +2010,21081,CBJ,0,2,1,0,0,3 +2010,21081,NJD,1,4,1,0,0,6 +2010,21082,ATL,0,0,0,0,0,0 +2010,21082,BOS,1,2,1,0,0,4 +2010,21083,OTT,1,0,1,0,0,2 +2010,21083,PHI,0,0,0,0,0,0 +2010,21084,CAR,0,2,0,0,0,2 +2010,21084,TBL,0,1,1,1,0,3 +2010,21085,DAL,0,1,2,0,0,3 +2010,21085,NSH,0,1,0,0,0,1 +2010,21086,MIN,0,1,0,0,0,1 +2010,21086,SJS,1,1,2,0,0,4 +2010,21087,CHI,0,2,0,0,0,2 +2010,21087,PHX,0,0,0,0,0,0 +2010,21088,ANA,0,1,0,0,0,1 +2010,21088,CGY,1,1,1,0,0,3 +2010,21089,EDM,2,1,0,0,0,3 +2010,21089,VAN,1,1,0,0,0,2 +2010,21090,NYI,0,0,0,0,0,0 +2010,21090,NYR,3,0,2,0,0,5 +2010,21091,PIT,0,2,1,0,0,3 +2010,21091,WSH,0,1,2,0,1,4 +2010,21092,BUF,0,0,2,0,1,3 +2010,21092,MTL,1,1,0,0,0,2 +2010,21093,DET,0,2,2,0,0,4 +2010,21093,STL,1,0,1,0,0,2 +2010,21094,COL,3,0,0,0,1,4 +2010,21094,LAK,1,0,2,0,0,3 +2010,21095,ANA,0,0,1,0,0,1 +2010,21095,VAN,2,1,1,0,0,4 +2010,21096,BOS,1,1,1,0,0,3 +2010,21096,TBL,2,2,1,0,0,5 +2010,21097,NJD,1,0,2,0,0,3 +2010,21097,NYR,0,1,2,0,1,4 +2010,21098,CGY,1,1,0,0,0,2 +2010,21098,NYI,1,0,2,0,0,3 +2010,21099,MIN,0,1,2,1,0,4 +2010,21099,PHI,2,1,0,0,0,3 +2010,21100,CAR,2,0,0,0,1,3 +2010,21100,WSH,0,1,1,0,0,2 +2010,21101,ATL,1,0,0,0,0,1 +2010,21101,TOR,1,0,0,1,0,2 +2010,21102,CHI,0,1,2,0,0,3 +2010,21102,CBJ,2,4,2,0,0,8 +2010,21103,MTL,2,0,2,0,0,4 +2010,21103,FLA,0,0,1,0,0,1 +2010,21104,LAK,0,0,1,0,0,1 +2010,21104,STL,1,1,1,0,0,3 +2010,21105,NSH,1,2,0,0,1,4 +2010,21105,PHX,1,0,2,0,0,3 +2010,21106,DAL,0,0,0,0,0,0 +2010,21106,SJS,0,2,1,0,0,3 +2010,21107,BUF,0,1,1,0,0,2 +2010,21107,OTT,1,2,1,0,0,4 +2010,21108,DET,0,4,2,0,0,6 +2010,21108,MIN,0,1,1,0,0,2 +2010,21109,ANA,1,0,2,0,0,3 +2010,21109,EDM,0,1,1,0,0,2 +2010,21110,BOS,1,2,2,0,0,5 +2010,21110,CGY,0,0,0,0,0,0 +2010,21111,PHI,1,0,0,0,0,1 +2010,21111,PIT,1,1,2,0,0,4 +2010,21112,BUF,3,3,1,0,0,7 +2010,21112,TBL,0,1,0,0,0,1 +2010,21113,NYR,2,0,0,0,0,2 +2010,21113,TOR,0,1,1,1,0,3 +2010,21114,MTL,0,2,0,0,0,2 +2010,21114,NJD,1,2,1,0,0,4 +2010,21115,FLA,0,1,1,0,0,2 +2010,21115,OTT,0,2,1,0,0,3 +2010,21116,ATL,0,3,1,0,0,4 +2010,21116,CAR,0,0,0,0,0,0 +2010,21117,CBJ,0,1,2,0,0,3 +2010,21117,NYI,2,0,1,1,0,4 +2010,21118,DET,0,0,0,0,1,1 +2010,21118,NSH,0,0,0,0,0,0 +2010,21119,COL,1,1,0,0,0,2 +2010,21119,PHX,1,2,3,0,0,6 +2010,21120,SJS,0,3,1,0,0,4 +2010,21120,VAN,1,0,1,0,0,2 +2010,21121,DAL,0,3,1,0,0,4 +2010,21121,LAK,1,0,0,0,0,1 +2010,21122,CGY,4,1,0,0,0,5 +2010,21122,WSH,0,2,1,0,0,3 +2010,21123,PIT,1,2,1,0,1,5 +2010,21123,TOR,1,2,1,0,0,4 +2010,21124,EDM,1,0,0,0,0,1 +2010,21124,STL,0,1,1,0,0,2 +2010,21125,NJD,0,0,1,0,0,1 +2010,21125,PHI,2,2,1,0,0,5 +2010,21126,CHI,1,0,1,0,0,2 +2010,21126,CBJ,1,2,1,0,0,4 +2010,21127,COL,0,2,1,0,0,3 +2010,21127,SJS,1,1,2,0,0,4 +2010,21128,BOS,1,0,1,0,0,2 +2010,21128,BUF,2,1,0,0,0,3 +2010,21129,ATL,1,0,0,0,0,1 +2010,21129,CAR,0,2,2,0,0,4 +2010,21130,FLA,0,0,2,0,0,2 +2010,21130,NSH,0,1,1,1,0,3 +2010,21131,LAK,0,1,1,0,0,2 +2010,21131,MIN,0,2,1,0,0,3 +2010,21132,ANA,0,3,0,0,0,3 +2010,21132,DAL,0,1,0,0,0,1 +2010,21133,ATL,0,2,1,0,0,3 +2010,21133,TOR,0,1,1,0,0,2 +2010,21134,BOS,0,0,0,1,0,1 +2010,21134,NJD,0,0,0,0,0,0 +2010,21135,NYI,2,0,1,0,0,3 +2010,21135,NYR,0,3,1,0,0,4 +2010,21136,OTT,3,0,1,1,0,5 +2010,21136,WSH,1,2,1,0,0,4 +2010,21137,CBJ,1,0,2,0,0,3 +2010,21137,TBL,1,1,0,0,0,2 +2010,21138,DET,3,1,1,0,0,5 +2010,21138,EDM,0,2,2,0,0,4 +2010,21139,CHI,2,0,0,0,0,2 +2010,21139,STL,1,1,2,0,0,4 +2010,21140,LAK,0,0,2,0,0,2 +2010,21140,NSH,0,0,0,0,0,0 +2010,21141,PHX,1,0,0,0,0,1 +2010,21141,VAN,3,1,0,0,0,4 +2010,21142,BUF,2,2,2,0,0,6 +2010,21142,FLA,1,0,1,0,0,2 +2010,21143,MTL,1,0,0,0,0,1 +2010,21143,CAR,0,1,1,0,0,2 +2010,21144,PIT,0,0,0,0,0,0 +2010,21144,TBL,1,1,0,0,0,2 +2010,21145,CHI,1,1,2,0,0,4 +2010,21145,MIN,0,0,0,0,0,0 +2010,21146,DAL,2,3,0,0,0,5 +2010,21146,SJS,1,0,0,0,0,1 +2010,21147,ANA,2,1,2,0,0,5 +2010,21147,COL,0,1,1,0,0,2 +2010,21148,CGY,0,1,1,0,0,2 +2010,21148,PHX,0,0,1,0,0,1 +2010,21149,BOS,0,0,0,0,0,0 +2010,21149,FLA,1,0,0,0,0,1 +2010,21150,BUF,1,1,0,0,0,2 +2010,21150,TOR,1,1,2,0,0,4 +2010,21151,NYI,3,1,2,0,0,6 +2010,21151,PHI,0,1,3,0,0,4 +2010,21152,ATL,0,1,0,0,0,1 +2010,21152,WSH,1,0,1,0,0,2 +2010,21153,CAR,0,2,1,0,0,3 +2010,21153,OTT,1,0,2,0,1,4 +2010,21154,CBJ,1,0,1,0,0,2 +2010,21154,DET,1,1,1,0,0,3 +2010,21155,NSH,0,2,1,0,0,3 +2010,21155,STL,1,1,0,0,0,2 +2010,21156,LAK,3,2,3,0,0,8 +2010,21156,VAN,0,2,1,0,0,3 +2010,21157,CHI,0,0,1,0,1,2 +2010,21157,NJD,1,0,0,0,0,1 +2010,21158,MTL,1,0,0,0,0,1 +2010,21158,PHI,0,0,0,0,0,0 +2010,21159,NYR,4,0,1,0,0,5 +2010,21159,TBL,0,0,0,0,0,0 +2010,21160,MIN,0,2,0,0,0,2 +2010,21160,SJS,2,0,1,0,0,3 +2010,21161,DAL,3,0,3,0,0,6 +2010,21161,EDM,0,3,0,0,0,3 +2010,21162,CGY,0,1,1,0,0,2 +2010,21162,COL,0,0,1,0,0,1 +2010,21163,ANA,0,2,2,0,0,4 +2010,21163,VAN,0,1,3,0,1,5 +2010,21164,ATL,1,1,1,0,0,3 +2010,21164,PIT,0,1,2,1,0,4 +2010,21165,NYI,0,3,1,0,0,4 +2010,21165,OTT,0,0,1,0,0,1 +2010,21166,DET,1,1,1,0,0,3 +2010,21166,NSH,0,3,0,1,0,4 +2010,21167,BOS,0,1,0,1,0,2 +2010,21167,TOR,1,0,0,0,0,1 +2010,21168,BUF,0,0,0,0,0,0 +2010,21168,MTL,1,1,1,0,0,3 +2010,21169,CAR,0,0,0,0,0,0 +2010,21169,NJD,2,0,2,0,0,4 +2010,21170,FLA,0,1,0,0,0,1 +2010,21170,NYR,0,0,4,0,0,4 +2010,21171,CBJ,1,0,1,0,0,2 +2010,21171,WSH,3,0,0,0,0,3 +2010,21172,DAL,0,1,0,0,0,1 +2010,21172,STL,0,1,1,0,0,2 +2010,21173,EDM,1,0,1,0,0,2 +2010,21173,PHX,1,1,0,0,1,3 +2010,21174,ANA,0,0,1,0,1,2 +2010,21174,LAK,1,0,0,0,0,1 +2010,21175,DET,2,0,1,0,0,3 +2010,21175,PHI,2,2,0,0,0,4 +2010,21176,CGY,0,1,0,0,0,1 +2010,21176,CHI,2,1,1,0,0,4 +2010,21177,COL,1,2,1,1,0,5 +2010,21177,SJS,2,0,2,0,0,4 +2010,21178,MIN,0,0,3,0,0,3 +2010,21178,VAN,1,1,1,1,0,4 +2010,21179,BOS,1,1,0,0,0,2 +2010,21179,WSH,1,1,0,1,0,3 +2010,21180,CBJ,1,0,0,0,0,1 +2010,21180,STL,0,1,0,1,0,2 +2010,21181,EDM,2,0,2,0,0,4 +2010,21181,MIN,0,1,0,0,0,1 +2010,21182,BUF,2,2,1,0,0,5 +2010,21182,NYR,2,0,0,0,0,2 +2010,21183,PHI,1,0,1,0,0,2 +2010,21183,TOR,0,0,0,0,0,0 +2010,21184,MTL,0,2,1,0,0,3 +2010,21184,NYI,1,0,2,0,1,4 +2010,21185,ATL,0,0,0,0,0,0 +2010,21185,NJD,1,1,1,0,0,3 +2010,21186,PIT,0,2,1,0,0,3 +2010,21186,WSH,1,3,2,0,0,6 +2010,21187,CAR,5,2,1,0,0,8 +2010,21187,TBL,1,2,2,0,0,5 +2010,21188,FLA,0,1,1,0,0,2 +2010,21188,OTT,1,2,2,0,0,5 +2010,21189,CHI,2,1,2,0,0,5 +2010,21189,DAL,0,1,1,0,0,2 +2010,21190,CGY,0,0,1,0,0,1 +2010,21190,SJS,1,1,0,0,0,2 +2010,21191,COL,2,1,0,0,1,4 +2010,21191,VAN,1,1,1,0,0,3 +2010,21192,ANA,2,2,0,0,0,4 +2010,21192,LAK,0,2,2,0,1,5 +2010,21193,NYR,3,0,2,0,0,5 +2010,21193,TOR,0,0,1,0,0,1 +2010,21194,CBJ,2,0,1,0,0,3 +2010,21194,DET,1,0,3,0,0,4 +2010,21195,CHI,4,2,0,0,0,6 +2010,21195,STL,1,1,3,0,0,5 +2010,21196,COL,3,0,1,0,0,4 +2010,21196,EDM,2,1,1,1,0,5 +2010,21197,NSH,1,0,1,0,0,2 +2010,21197,PHX,1,1,3,0,0,5 +2010,21198,BOS,0,1,2,0,0,3 +2010,21198,BUF,1,0,0,0,0,1 +2010,21199,MTL,0,1,1,0,0,2 +2010,21199,CAR,0,2,3,0,0,5 +2010,21200,NYI,2,1,0,0,0,3 +2010,21200,PIT,4,2,1,0,0,7 +2010,21201,OTT,0,2,1,0,0,3 +2010,21201,TBL,2,1,0,0,1,4 +2010,21202,FLA,0,1,2,0,0,3 +2010,21202,NJD,0,1,1,0,0,2 +2010,21203,ANA,0,1,1,0,0,2 +2010,21203,DAL,0,1,1,0,1,3 +2010,21204,CGY,0,1,0,0,0,1 +2010,21204,MIN,1,0,0,0,1,2 +2010,21205,LAK,0,1,1,0,0,2 +2010,21205,PHX,0,0,2,0,1,3 +2010,21206,SJS,1,3,0,0,0,4 +2010,21206,VAN,0,0,2,0,0,2 +2010,21207,NYR,2,2,0,0,0,4 +2010,21207,PHI,1,2,0,0,0,3 +2010,21208,ATL,0,2,0,0,0,2 +2010,21208,WSH,1,1,3,0,0,5 +2010,21209,CBJ,0,0,0,0,0,0 +2010,21209,DET,0,0,0,0,1,1 +2010,21210,ANA,1,2,0,0,0,3 +2010,21210,STL,1,1,4,0,0,6 +2010,21211,CHI,2,1,2,0,0,5 +2010,21211,COL,0,1,1,0,0,2 +2010,21212,BOS,0,3,1,0,0,4 +2010,21212,CAR,0,2,0,0,0,2 +2010,21213,EDM,1,1,1,0,1,4 +2010,21213,LAK,0,0,3,0,0,3 +2010,21214,MTL,2,1,0,0,0,3 +2010,21214,TOR,1,1,1,1,0,4 +2010,21215,BUF,2,2,1,0,0,5 +2010,21215,OTT,0,1,1,0,0,2 +2010,21216,NJD,3,2,2,0,0,7 +2010,21216,NYI,0,0,1,0,0,1 +2010,21217,ATL,0,1,0,0,0,1 +2010,21217,PIT,0,0,0,0,0,0 +2010,21218,FLA,1,1,1,0,0,3 +2010,21218,TBL,1,2,0,0,1,4 +2010,21219,NSH,0,0,1,0,1,2 +2010,21219,STL,0,0,1,0,0,1 +2010,21220,DAL,0,2,1,0,1,4 +2010,21220,MIN,2,0,1,0,0,3 +2010,21221,CGY,1,1,1,0,0,3 +2010,21221,VAN,3,1,3,0,0,7 +2010,21222,PHX,0,1,1,0,0,2 +2010,21222,SJS,1,1,0,0,1,3 +2010,21223,NYR,1,0,0,0,0,1 +2010,21223,PHI,0,0,1,0,1,2 +2010,21224,BOS,2,0,1,0,1,4 +2010,21224,WSH,2,0,1,0,0,3 +2010,21225,CHI,1,0,1,0,0,2 +2010,21225,DET,0,1,1,1,0,3 +2010,21226,COL,1,0,0,0,0,1 +2010,21226,LAK,0,0,1,1,0,2 +2010,21227,BUF,0,1,0,0,0,1 +2010,21227,NJD,0,1,1,0,0,2 +2010,21228,NYI,1,1,3,0,0,5 +2010,21228,PIT,3,2,0,1,0,6 +2010,21229,FLA,1,0,0,0,0,1 +2010,21229,TBL,2,0,1,0,0,3 +2010,21230,ANA,3,1,3,0,0,7 +2010,21230,EDM,1,0,1,0,0,2 +2011,20001,MTL,1,0,1,0,0,2 +2011,20001,TOR,2,0,1,0,0,3 +2011,20002,PHI,0,2,1,0,0,3 +2011,20002,PIT,0,0,2,0,0,2 +2011,20003,CAR,1,3,0,0,0,4 +2011,20003,MIN,1,1,1,0,0,3 +2011,20004,CHI,1,1,1,0,0,3 +2011,20004,COL,1,2,0,1,0,4 +2011,20005,CGY,0,0,0,0,0,0 +2011,20005,EDM,1,0,3,0,0,4 +2011,20006,CBJ,2,0,0,0,0,2 +2011,20006,SJS,2,0,1,0,0,3 +2011,20007,DAL,1,2,0,1,0,4 +2011,20007,NJD,2,1,0,0,0,3 +2011,20008,CAR,0,1,0,0,1,2 +2011,20008,MIN,1,0,0,0,0,1 +2011,20009,ANA,0,0,0,0,0,0 +2011,20009,DET,2,2,0,0,0,4 +2011,20010,BUF,1,0,1,0,0,2 +2011,20010,OTT,0,1,0,0,0,1 +2011,20011,ATL,1,2,1,0,0,4 +2011,20011,WSH,1,0,1,0,0,2 +2011,20012,BOS,0,0,2,0,0,2 +2011,20012,PHX,1,3,1,0,0,5 +2011,20013,BUF,0,1,2,0,0,3 +2011,20013,NYR,2,2,2,0,0,6 +2011,20014,OTT,0,0,1,0,0,1 +2011,20014,TOR,2,1,2,0,0,5 +2011,20015,DAL,2,1,1,0,1,5 +2011,20015,NYI,0,2,2,0,0,4 +2011,20016,MTL,1,0,2,0,0,3 +2011,20016,PIT,0,1,1,0,0,2 +2011,20017,NJD,2,0,0,0,0,2 +2011,20017,WSH,1,4,2,0,0,7 +2011,20018,ATL,0,2,1,0,0,3 +2011,20018,TBL,2,2,1,0,0,5 +2011,20019,PHI,0,0,1,0,0,1 +2011,20019,STL,1,0,0,1,0,2 +2011,20020,ANA,0,1,0,0,0,1 +2011,20020,NSH,0,3,1,0,0,4 +2011,20021,CHI,1,1,0,0,0,2 +2011,20021,DET,1,1,1,0,0,3 +2011,20022,CBJ,1,0,1,1,0,3 +2011,20022,SJS,1,0,1,0,0,2 +2011,20023,LAK,0,0,1,0,1,2 +2011,20023,VAN,0,1,0,0,0,1 +2011,20024,BOS,0,2,1,0,0,3 +2011,20024,PHX,0,0,0,0,0,0 +2011,20025,CGY,0,1,2,0,0,3 +2011,20025,LAK,0,0,1,0,0,1 +2011,20026,EDM,0,3,0,0,0,3 +2011,20026,FLA,0,2,0,0,0,2 +2011,20027,NJD,0,0,1,0,0,1 +2011,20027,PIT,1,1,1,0,0,3 +2011,20028,NYI,2,1,3,0,0,6 +2011,20028,NYR,1,2,1,0,0,4 +2011,20029,ANA,0,1,0,0,0,1 +2011,20029,STL,2,2,1,0,0,5 +2011,20030,BUF,2,0,1,0,0,3 +2011,20030,CHI,1,2,1,0,0,4 +2011,20031,COL,0,1,1,0,0,2 +2011,20031,PHI,2,0,2,0,0,4 +2011,20032,OTT,0,1,1,0,0,2 +2011,20032,WSH,1,1,0,1,0,3 +2011,20033,FLA,0,1,0,0,0,1 +2011,20033,VAN,1,0,1,0,0,2 +2011,20034,COL,1,1,2,0,1,5 +2011,20034,DET,1,2,1,0,0,4 +2011,20035,ATL,0,1,0,0,0,1 +2011,20035,LAK,0,0,3,0,0,3 +2011,20036,BUF,0,0,0,0,0,0 +2011,20036,NJD,0,0,0,1,0,1 +2011,20037,NYI,1,0,0,0,0,1 +2011,20037,WSH,0,1,1,0,0,2 +2011,20038,MTL,2,0,1,0,0,3 +2011,20038,TBL,0,1,2,1,0,4 +2011,20039,PIT,2,1,0,0,0,3 +2011,20039,TOR,1,3,0,0,0,4 +2011,20040,CHI,2,0,0,0,0,2 +2011,20040,NSH,1,0,2,0,0,3 +2011,20041,ANA,1,1,2,0,0,4 +2011,20041,VAN,2,0,1,0,0,3 +2011,20042,PHI,1,0,1,0,0,2 +2011,20042,TBL,1,1,1,0,0,3 +2011,20043,CAR,0,0,2,0,0,2 +2011,20043,OTT,1,1,1,0,0,3 +2011,20044,NSH,3,1,0,0,0,4 +2011,20044,STL,1,1,1,0,0,3 +2011,20045,EDM,0,2,0,0,0,2 +2011,20045,MIN,1,2,1,0,0,4 +2011,20046,DAL,1,3,0,0,0,4 +2011,20046,DET,0,0,1,0,0,1 +2011,20047,CGY,0,0,0,0,0,0 +2011,20047,FLA,2,0,1,0,0,3 +2011,20048,COL,1,1,1,0,0,3 +2011,20048,NJD,0,1,1,0,0,2 +2011,20049,NYR,1,0,2,0,0,3 +2011,20049,TOR,0,3,0,1,0,4 +2011,20050,NYI,0,2,0,0,0,2 +2011,20050,PIT,0,2,0,1,0,3 +2011,20051,CHI,2,2,1,0,0,5 +2011,20051,CBJ,0,1,1,0,0,2 +2011,20052,BUF,0,0,1,0,0,1 +2011,20052,MTL,0,2,0,0,0,2 +2011,20053,ANA,1,2,1,0,0,4 +2011,20053,ATL,1,1,2,0,1,5 +2011,20054,LAK,1,1,2,0,0,4 +2011,20054,VAN,0,0,1,0,0,1 +2011,20055,MTL,1,2,1,0,0,4 +2011,20055,OTT,1,2,0,0,0,3 +2011,20056,BOS,0,4,0,0,0,4 +2011,20056,NJD,0,1,0,0,0,1 +2011,20057,COL,0,0,2,0,0,2 +2011,20057,NYI,0,3,2,0,0,5 +2011,20058,PHI,1,0,0,0,0,1 +2011,20058,PIT,1,1,3,0,0,5 +2011,20059,FLA,4,0,2,0,0,6 +2011,20059,TBL,0,0,0,0,0,0 +2011,20060,NSH,1,1,0,0,0,2 +2011,20060,WSH,0,0,2,1,0,3 +2011,20061,CBJ,1,1,1,0,0,3 +2011,20061,MIN,0,2,0,0,0,2 +2011,20062,DAL,1,1,0,0,1,3 +2011,20062,STL,2,0,0,0,0,2 +2011,20063,BUF,2,1,0,0,0,3 +2011,20063,CHI,1,1,2,0,0,4 +2011,20064,DET,0,1,0,1,0,2 +2011,20064,PHX,1,0,0,0,0,1 +2011,20065,CGY,1,1,3,0,0,5 +2011,20065,EDM,1,2,0,0,0,3 +2011,20066,ATL,0,2,2,0,0,4 +2011,20066,SJS,2,0,0,0,0,2 +2011,20067,ANA,0,2,1,0,0,3 +2011,20067,PHX,0,0,2,0,0,2 +2011,20068,CAR,0,1,0,0,0,1 +2011,20068,VAN,2,1,2,0,0,5 +2011,20069,NYI,0,1,0,1,0,2 +2011,20069,TOR,0,0,1,0,0,1 +2011,20070,COL,1,0,2,0,0,3 +2011,20070,NYR,1,0,0,0,0,1 +2011,20071,OTT,0,1,1,0,0,2 +2011,20071,PIT,3,2,0,0,0,5 +2011,20072,DAL,1,1,2,0,0,4 +2011,20072,TBL,2,2,1,0,0,5 +2011,20073,CHI,0,0,2,1,0,3 +2011,20073,STL,0,1,1,0,0,2 +2011,20074,BOS,2,0,1,0,0,3 +2011,20074,WSH,0,1,0,0,0,1 +2011,20075,CGY,0,0,0,1,0,1 +2011,20075,NSH,0,0,0,0,0,0 +2011,20076,MIN,3,3,0,0,0,6 +2011,20076,VAN,1,0,1,0,0,2 +2011,20077,CAR,2,1,2,0,0,5 +2011,20077,SJS,0,2,0,0,0,2 +2011,20078,ATL,0,0,1,0,0,1 +2011,20078,BUF,1,2,1,0,0,4 +2011,20079,ANA,0,0,1,0,0,1 +2011,20079,CBJ,0,2,1,0,0,3 +2011,20080,CHI,1,0,0,0,1,2 +2011,20080,VAN,0,0,1,0,0,1 +2011,20081,CAR,2,1,0,0,0,3 +2011,20081,LAK,2,1,1,0,0,4 +2011,20082,BOS,1,2,1,0,0,4 +2011,20082,WSH,0,0,1,0,0,1 +2011,20083,NYR,2,0,0,0,0,2 +2011,20083,TOR,0,0,1,0,0,1 +2011,20084,ANA,2,0,1,0,0,3 +2011,20084,PHI,1,1,0,0,0,2 +2011,20085,CGY,2,0,0,0,0,2 +2011,20085,DET,1,1,2,0,0,4 +2011,20086,MTL,0,0,0,0,0,0 +2011,20086,NJD,1,1,1,0,0,3 +2011,20087,NYI,1,1,0,1,0,3 +2011,20087,TBL,1,1,0,0,0,2 +2011,20088,DAL,2,2,0,0,0,4 +2011,20088,FLA,0,0,1,0,0,1 +2011,20089,NSH,1,1,1,0,0,3 +2011,20089,PIT,0,2,1,1,0,4 +2011,20090,COL,1,0,1,0,0,2 +2011,20090,SJS,1,2,1,0,0,4 +2011,20091,EDM,1,0,1,0,0,2 +2011,20091,MIN,3,0,1,0,0,4 +2011,20092,LAK,0,1,1,0,0,2 +2011,20092,PHX,2,1,1,0,0,4 +2011,20093,CGY,3,1,2,0,0,6 +2011,20093,CBJ,1,0,1,0,0,2 +2011,20094,BUF,1,1,0,0,0,2 +2011,20094,OTT,1,2,1,0,0,4 +2011,20095,ATL,0,1,1,0,0,2 +2011,20095,TBL,3,1,1,0,0,5 +2011,20096,CHI,1,1,0,0,0,2 +2011,20096,STL,1,1,2,0,0,4 +2011,20097,MIN,0,0,1,0,0,1 +2011,20097,VAN,1,1,3,0,0,5 +2011,20098,BOS,1,1,0,0,0,2 +2011,20098,NYR,2,1,0,0,0,3 +2011,20099,MTL,1,2,0,0,0,3 +2011,20099,OTT,0,0,0,0,0,0 +2011,20100,BUF,2,3,1,0,0,6 +2011,20100,NJD,0,0,1,0,0,1 +2011,20101,PHI,2,1,2,0,0,5 +2011,20101,TOR,0,2,0,0,0,2 +2011,20102,ATL,1,1,1,0,0,3 +2011,20102,WSH,0,1,2,1,0,4 +2011,20103,FLA,2,1,1,0,0,4 +2011,20103,NYI,1,2,0,0,0,3 +2011,20104,ANA,1,2,1,0,0,4 +2011,20104,DET,1,2,2,0,0,5 +2011,20105,PIT,0,0,0,0,0,0 +2011,20105,STL,0,0,0,1,0,1 +2011,20106,DAL,0,0,0,0,0,0 +2011,20106,NSH,1,0,0,0,0,1 +2011,20107,CHI,2,0,0,0,0,2 +2011,20107,CBJ,0,1,2,0,0,3 +2011,20108,COL,2,1,1,0,0,4 +2011,20108,LAK,2,2,2,0,0,6 +2011,20109,CAR,0,2,1,1,0,4 +2011,20109,PHX,0,0,3,0,0,3 +2011,20110,EDM,1,0,0,0,0,1 +2011,20110,SJS,1,2,3,0,0,6 +2011,20111,NSH,2,1,1,0,0,4 +2011,20111,TBL,1,0,2,0,0,3 +2011,20112,NJD,0,0,1,0,0,1 +2011,20112,NYR,0,2,1,0,0,3 +2011,20113,CGY,4,0,0,0,0,4 +2011,20113,SJS,0,0,0,0,0,0 +2011,20114,CBJ,1,1,0,0,0,2 +2011,20114,PHI,0,0,1,0,0,1 +2011,20115,MTL,0,2,0,1,0,3 +2011,20115,PHX,0,1,1,0,0,2 +2011,20116,LAK,1,1,0,0,1,3 +2011,20116,MIN,2,0,0,0,0,2 +2011,20117,FLA,0,1,0,0,0,1 +2011,20117,TOR,1,0,2,0,0,3 +2011,20118,BUF,1,0,2,0,0,3 +2011,20118,PHI,1,3,2,0,0,6 +2011,20119,OTT,3,0,2,0,0,5 +2011,20119,PHX,0,1,1,0,0,2 +2011,20120,CGY,2,2,0,0,1,5 +2011,20120,EDM,1,1,2,0,0,4 +2011,20121,ANA,0,3,2,0,0,5 +2011,20121,DAL,1,0,1,0,0,2 +2011,20122,COL,1,1,1,0,0,3 +2011,20122,VAN,1,1,1,1,0,4 +2011,20123,ATL,1,2,3,0,0,6 +2011,20123,NYR,1,1,2,0,0,4 +2011,20124,CAR,0,0,0,0,0,0 +2011,20124,WSH,0,0,0,0,0,0 +2011,20125,MTL,2,1,2,0,0,5 +2011,20125,NYI,1,0,2,0,0,3 +2011,20126,PIT,3,0,0,0,0,3 +2011,20126,TBL,2,1,2,0,0,5 +2011,20127,CHI,1,1,1,0,0,3 +2011,20127,LAK,1,0,0,0,0,1 +2011,20128,NJD,0,1,1,0,0,2 +2011,20128,SJS,3,1,1,0,0,5 +2011,20129,BOS,1,1,0,0,0,2 +2011,20129,TOR,0,0,0,0,0,0 +2011,20130,DET,0,1,1,0,0,2 +2011,20130,PHX,3,0,1,0,0,4 +2011,20131,CBJ,2,0,0,0,1,3 +2011,20131,EDM,1,0,1,0,0,2 +2011,20132,FLA,1,0,2,0,0,3 +2011,20132,OTT,0,3,2,0,0,5 +2011,20133,NSH,0,0,0,0,0,0 +2011,20133,STL,0,1,2,0,0,3 +2011,20134,MIN,1,1,0,0,0,2 +2011,20134,WSH,0,0,1,0,0,1 +2011,20135,DAL,1,0,1,0,0,2 +2011,20135,LAK,1,2,2,0,0,5 +2011,20136,CGY,2,2,1,0,0,5 +2011,20136,COL,1,2,3,0,0,6 +2011,20137,MTL,1,0,2,0,0,3 +2011,20137,NYI,0,1,0,0,0,1 +2011,20138,CAR,1,2,1,0,0,4 +2011,20138,NYR,0,3,0,0,0,3 +2011,20139,PHI,0,2,1,0,0,3 +2011,20139,PIT,1,0,1,0,0,2 +2011,20140,ATL,0,2,1,1,0,4 +2011,20140,BUF,2,0,1,0,0,3 +2011,20141,CHI,2,1,1,0,0,4 +2011,20141,EDM,4,2,1,0,0,7 +2011,20142,ANA,0,1,0,0,0,1 +2011,20142,NJD,0,1,1,0,0,2 +2011,20143,NYR,1,1,0,0,0,2 +2011,20143,TOR,0,0,0,0,0,0 +2011,20144,FLA,1,1,1,0,0,3 +2011,20144,MTL,0,1,0,0,0,1 +2011,20145,BOS,1,2,1,0,0,4 +2011,20145,OTT,0,0,0,0,0,0 +2011,20146,NYI,0,1,0,0,0,1 +2011,20146,PHI,2,4,0,0,0,6 +2011,20147,CAR,0,0,0,0,0,0 +2011,20147,PIT,1,1,1,0,0,3 +2011,20148,DET,2,1,2,0,0,5 +2011,20148,NSH,0,1,1,0,0,2 +2011,20149,ATL,3,0,0,0,0,3 +2011,20149,STL,1,0,2,0,1,4 +2011,20150,CHI,0,1,2,0,0,3 +2011,20150,MIN,0,0,1,0,0,1 +2011,20151,BUF,0,0,0,0,0,0 +2011,20151,DAL,1,1,2,0,0,4 +2011,20152,COL,1,3,1,0,0,5 +2011,20152,CBJ,0,0,1,0,0,1 +2011,20153,PHX,0,0,0,0,0,0 +2011,20153,TBL,0,1,2,0,0,3 +2011,20154,CGY,2,0,0,0,0,2 +2011,20154,WSH,1,6,0,0,0,7 +2011,20155,LAK,1,2,0,0,0,3 +2011,20155,NJD,0,1,0,0,0,1 +2011,20156,ANA,0,0,2,0,0,2 +2011,20156,SJS,2,3,0,0,0,5 +2011,20157,CHI,1,0,1,0,0,2 +2011,20157,NYR,1,0,2,0,0,3 +2011,20158,CAR,0,1,1,0,0,2 +2011,20158,PHI,1,1,1,0,0,3 +2011,20159,NJD,0,0,0,0,0,0 +2011,20159,VAN,1,1,1,0,0,3 +2011,20160,OTT,1,2,0,0,0,3 +2011,20160,TOR,0,0,2,0,0,2 +2011,20161,CBJ,2,0,1,0,0,3 +2011,20161,MTL,0,0,0,0,0,0 +2011,20162,MIN,0,1,0,0,0,1 +2011,20162,SJS,0,0,0,0,0,0 +2011,20163,EDM,0,1,2,0,0,3 +2011,20163,VAN,3,0,1,0,0,4 +2011,20164,BOS,3,1,1,0,0,5 +2011,20164,BUF,0,1,1,0,0,2 +2011,20165,TOR,1,0,3,0,0,4 +2011,20165,WSH,0,3,1,0,1,5 +2011,20166,CAR,3,2,2,0,0,7 +2011,20166,NYI,0,0,2,0,0,2 +2011,20167,ATL,1,1,2,0,0,4 +2011,20167,FLA,1,1,1,0,0,3 +2011,20168,CHI,0,1,2,0,0,3 +2011,20168,NJD,1,1,3,0,0,5 +2011,20169,DAL,2,2,1,0,0,5 +2011,20169,PIT,0,1,1,0,0,2 +2011,20170,CGY,1,0,0,0,0,1 +2011,20170,DET,0,2,0,0,0,2 +2011,20171,NSH,2,1,0,0,0,3 +2011,20171,PHX,1,1,2,0,0,4 +2011,20172,ANA,1,0,1,1,0,3 +2011,20172,TBL,0,1,1,0,0,2 +2011,20173,NYR,0,1,0,0,0,1 +2011,20173,PHI,0,3,1,0,0,4 +2011,20174,ATL,0,0,0,0,0,0 +2011,20174,CBJ,0,1,2,0,0,3 +2011,20175,NYI,0,0,1,0,0,1 +2011,20175,OTT,1,2,1,0,0,4 +2011,20176,SJS,0,0,0,0,0,0 +2011,20176,STL,1,1,0,0,0,2 +2011,20177,COL,0,0,1,0,0,1 +2011,20177,VAN,1,1,1,0,0,3 +2011,20178,LAK,0,0,1,0,0,1 +2011,20178,TBL,0,0,0,0,0,0 +2011,20179,NJD,0,0,0,0,0,0 +2011,20179,NYR,1,1,1,0,0,3 +2011,20180,BOS,0,0,3,0,0,3 +2011,20180,WSH,0,3,2,0,0,5 +2011,20181,BUF,1,1,0,0,0,2 +2011,20181,MTL,1,2,0,0,0,3 +2011,20182,CAR,3,0,1,0,0,4 +2011,20182,FLA,3,1,3,0,0,7 +2011,20183,CGY,1,0,0,0,0,1 +2011,20183,MIN,1,0,1,0,0,2 +2011,20184,DAL,1,4,1,0,0,6 +2011,20184,PHX,0,1,2,0,0,3 +2011,20185,DET,1,1,1,0,0,3 +2011,20185,EDM,0,1,0,0,0,1 +2011,20186,ANA,0,3,0,0,0,3 +2011,20186,PIT,0,1,1,0,0,2 +2011,20187,LAK,2,2,0,0,0,4 +2011,20187,NSH,0,0,1,0,0,1 +2011,20188,BOS,0,0,1,0,0,1 +2011,20188,STL,1,0,0,0,1,2 +2011,20189,BUF,0,1,1,0,1,3 +2011,20189,TOR,1,1,0,0,0,2 +2011,20190,MTL,0,1,1,0,0,2 +2011,20190,OTT,1,1,1,0,0,3 +2011,20191,NYI,0,1,0,0,0,1 +2011,20191,PHI,1,0,1,0,0,2 +2011,20192,CAR,0,1,2,0,0,3 +2011,20192,FLA,0,0,2,0,0,2 +2011,20193,ATL,1,3,0,0,0,4 +2011,20193,CHI,2,2,0,0,1,5 +2011,20194,CBJ,1,1,0,0,0,2 +2011,20194,MIN,1,1,1,0,0,3 +2011,20195,COL,1,2,2,0,0,5 +2011,20195,DAL,0,0,0,0,0,0 +2011,20196,PHX,2,1,0,0,0,3 +2011,20196,PIT,0,3,0,0,1,4 +2011,20197,DET,0,3,1,0,0,4 +2011,20197,VAN,1,2,3,0,0,6 +2011,20198,SJS,2,2,1,0,0,5 +2011,20198,TBL,1,0,1,0,0,2 +2011,20199,PHI,1,1,0,0,0,2 +2011,20199,WSH,1,1,0,1,0,3 +2011,20200,NYR,0,0,0,0,0,0 +2011,20200,STL,0,1,1,0,0,2 +2011,20201,CHI,1,0,0,0,0,1 +2011,20201,EDM,0,0,2,0,0,2 +2011,20202,ANA,2,0,3,0,0,5 +2011,20202,NSH,2,0,2,0,0,4 +2011,20203,DET,1,0,1,1,0,3 +2011,20203,PHX,1,1,0,0,0,2 +2011,20204,NYR,2,1,0,0,0,3 +2011,20204,WSH,1,2,2,0,0,5 +2011,20205,CAR,4,1,2,0,0,7 +2011,20205,EDM,0,0,1,0,0,1 +2011,20206,MTL,1,0,1,0,0,2 +2011,20206,VAN,0,0,0,0,0,0 +2011,20207,ATL,1,0,1,0,0,2 +2011,20207,OTT,2,2,1,0,0,5 +2011,20208,TBL,3,0,1,0,0,4 +2011,20208,TOR,0,0,0,0,0,0 +2011,20209,CGY,0,1,3,0,0,4 +2011,20209,COL,1,0,1,0,0,2 +2011,20210,ANA,1,1,0,1,0,3 +2011,20210,SJS,1,0,1,0,0,2 +2011,20211,BUF,0,3,1,0,1,5 +2011,20211,NJD,1,3,0,0,0,4 +2011,20212,BOS,2,0,5,0,0,7 +2011,20212,PIT,2,2,0,0,0,4 +2011,20213,CBJ,3,3,2,0,0,8 +2011,20213,STL,0,1,0,0,0,1 +2011,20214,FLA,1,1,2,0,0,4 +2011,20214,TOR,0,0,1,0,0,1 +2011,20215,CHI,1,0,0,0,0,1 +2011,20215,PHX,0,2,0,0,0,2 +2011,20216,ANA,0,0,1,0,0,1 +2011,20216,NYI,0,0,0,0,0,0 +2011,20217,BOS,1,0,0,0,0,1 +2011,20217,MTL,1,0,2,0,0,3 +2011,20218,BUF,0,2,0,0,0,2 +2011,20218,NYR,1,1,0,1,0,3 +2011,20219,TBL,1,0,2,0,0,3 +2011,20219,WSH,0,2,4,0,0,6 +2011,20220,CAR,0,0,1,0,0,1 +2011,20220,PHI,2,2,4,0,0,8 +2011,20221,ATL,2,1,2,0,0,5 +2011,20221,MIN,0,0,1,0,0,1 +2011,20222,DET,3,2,1,0,0,6 +2011,20222,EDM,0,0,2,0,0,2 +2011,20223,OTT,1,0,1,0,0,2 +2011,20223,VAN,1,1,4,0,0,6 +2011,20224,NSH,0,1,1,0,1,3 +2011,20224,STL,1,0,1,0,0,2 +2011,20225,DAL,1,0,0,0,0,1 +2011,20225,LAK,1,0,2,0,0,3 +2011,20226,NYI,0,1,0,0,0,1 +2011,20226,SJS,0,1,0,0,1,2 +2011,20227,EDM,2,1,0,0,0,3 +2011,20227,NJD,1,1,1,1,0,4 +2011,20228,PIT,0,2,3,0,0,5 +2011,20228,TBL,0,0,1,0,0,1 +2011,20229,COL,1,3,1,0,0,5 +2011,20229,CBJ,1,0,0,0,0,1 +2011,20230,FLA,2,0,0,0,0,2 +2011,20230,MIN,1,0,0,0,0,1 +2011,20231,CGY,1,1,2,0,0,4 +2011,20231,PHX,0,3,2,0,0,5 +2011,20232,ANA,1,0,3,0,0,4 +2011,20232,DAL,1,0,1,0,0,2 +2011,20233,BOS,0,0,0,0,0,0 +2011,20233,OTT,1,0,1,0,0,2 +2011,20234,BUF,0,2,0,1,0,3 +2011,20234,WSH,1,0,1,0,0,2 +2011,20235,TOR,2,1,0,0,0,3 +2011,20235,VAN,2,1,2,0,0,5 +2011,20236,CAR,0,2,0,0,0,2 +2011,20236,MTL,2,3,2,0,0,7 +2011,20237,FLA,0,1,1,0,0,2 +2011,20237,PHI,2,3,0,0,0,5 +2011,20238,ATL,1,0,1,0,0,2 +2011,20238,PIT,0,1,3,0,0,4 +2011,20239,COL,0,0,1,0,0,1 +2011,20239,DET,1,2,0,0,0,3 +2011,20240,CHI,1,2,0,0,0,3 +2011,20240,NSH,0,2,1,0,1,4 +2011,20241,PHX,1,3,1,0,0,5 +2011,20241,STL,1,1,1,0,0,3 +2011,20242,CGY,0,1,2,0,0,3 +2011,20242,SJS,2,1,1,0,0,4 +2011,20243,LAK,2,1,2,0,0,5 +2011,20243,NYI,1,0,0,0,0,1 +2011,20244,EDM,0,2,0,0,0,2 +2011,20244,NYR,1,4,3,0,0,8 +2011,20245,ATL,1,3,0,0,0,4 +2011,20245,WSH,3,1,2,0,0,6 +2011,20246,MIN,3,0,1,0,0,4 +2011,20246,TBL,1,0,0,0,0,1 +2011,20247,ANA,1,0,1,0,0,2 +2011,20247,CHI,1,1,0,1,0,3 +2011,20248,BOS,1,1,1,0,0,3 +2011,20248,NJD,0,0,0,0,0,0 +2011,20249,BUF,2,1,0,1,0,4 +2011,20249,VAN,0,1,2,0,0,3 +2011,20250,OTT,1,0,0,0,0,1 +2011,20250,PHI,2,1,2,0,0,5 +2011,20251,NYR,0,1,1,1,0,3 +2011,20251,PIT,0,0,2,0,0,2 +2011,20252,COL,3,2,1,0,0,6 +2011,20252,STL,1,1,1,0,0,3 +2011,20253,LAK,1,1,1,0,0,3 +2011,20253,SJS,2,1,3,0,0,6 +2011,20254,NSH,3,1,0,0,0,4 +2011,20254,TOR,1,4,0,0,0,5 +2011,20255,MTL,1,1,1,0,0,3 +2011,20255,PHI,0,0,0,0,0,0 +2011,20256,ANA,0,1,0,0,0,1 +2011,20256,DAL,1,0,1,0,0,2 +2011,20257,NYI,0,1,1,0,0,2 +2011,20257,TBL,1,1,2,0,0,4 +2011,20258,BOS,0,2,1,0,0,3 +2011,20258,NYR,0,1,1,0,0,2 +2011,20259,PIT,1,2,0,0,0,3 +2011,20259,VAN,0,1,0,0,0,1 +2011,20260,BUF,0,2,0,0,0,2 +2011,20260,WSH,2,1,1,0,0,4 +2011,20261,CAR,4,0,3,0,0,7 +2011,20261,OTT,0,0,1,0,0,1 +2011,20262,ATL,1,0,0,0,0,1 +2011,20262,FLA,0,2,0,0,0,2 +2011,20263,DET,1,2,4,0,0,7 +2011,20263,STL,1,2,0,0,0,3 +2011,20264,ANA,0,1,0,0,0,1 +2011,20264,MIN,0,1,0,1,0,2 +2011,20265,COL,1,0,2,1,0,4 +2011,20265,SJS,1,2,0,0,0,3 +2011,20266,CGY,0,0,1,0,0,1 +2011,20266,PHX,0,2,1,0,0,3 +2011,20267,CHI,2,2,1,0,0,5 +2011,20267,EDM,0,0,0,0,0,0 +2011,20268,CBJ,1,1,3,0,0,5 +2011,20268,LAK,1,1,1,0,0,3 +2011,20269,BOS,1,0,3,0,0,4 +2011,20269,FLA,0,0,0,0,0,0 +2011,20270,NJD,0,1,0,0,0,1 +2011,20270,TOR,0,2,1,0,0,3 +2011,20271,PHI,5,2,0,0,0,7 +2011,20271,TBL,4,3,1,0,0,8 +2011,20272,MTL,0,0,0,0,0,0 +2011,20272,NSH,0,1,2,0,0,3 +2011,20273,DAL,2,0,2,1,0,5 +2011,20273,SJS,1,2,1,0,0,4 +2011,20274,CAR,2,1,1,0,0,4 +2011,20274,PIT,2,1,1,0,1,5 +2011,20275,DET,0,1,2,0,0,3 +2011,20275,MIN,1,1,1,1,0,4 +2011,20276,BUF,0,2,2,0,0,4 +2011,20276,LAK,1,1,0,0,0,2 +2011,20277,ATL,3,1,1,0,0,5 +2011,20277,WSH,0,0,0,0,0,0 +2011,20278,OTT,1,0,1,0,0,2 +2011,20278,STL,0,3,2,0,0,5 +2011,20279,COL,0,4,1,0,0,5 +2011,20279,NYR,0,0,1,0,0,1 +2011,20280,CGY,2,2,3,0,0,7 +2011,20280,CHI,1,0,1,0,0,2 +2011,20281,EDM,1,2,0,0,0,3 +2011,20281,PHX,0,2,1,0,1,4 +2011,20282,ANA,0,2,1,0,0,3 +2011,20282,CBJ,1,3,0,0,0,4 +2011,20283,BOS,0,2,1,0,0,3 +2011,20283,LAK,2,1,0,0,1,4 +2011,20284,BUF,1,0,0,0,0,1 +2011,20284,TBL,1,1,0,0,0,2 +2011,20285,MTL,0,1,1,0,0,2 +2011,20285,TOR,0,0,0,0,0,0 +2011,20286,FLA,0,4,0,0,0,4 +2011,20286,NYI,1,0,0,0,0,1 +2011,20287,PHI,0,2,2,0,1,5 +2011,20287,WSH,1,0,3,0,0,4 +2011,20288,CAR,1,0,0,0,0,1 +2011,20288,NSH,0,1,0,0,1,2 +2011,20289,NJD,1,1,0,0,0,2 +2011,20289,STL,1,1,1,0,0,3 +2011,20290,MIN,0,0,2,0,0,2 +2011,20290,NYR,0,3,2,0,0,5 +2011,20291,COL,1,2,0,0,1,4 +2011,20291,DAL,0,2,1,0,0,3 +2011,20292,CHI,0,4,3,0,0,7 +2011,20292,VAN,0,0,1,0,0,1 +2011,20293,CBJ,1,1,1,0,0,3 +2011,20293,SJS,0,0,0,0,0,0 +2011,20294,ATL,0,0,1,1,0,2 +2011,20294,NYI,0,1,0,0,0,1 +2011,20295,CGY,1,2,1,0,0,4 +2011,20295,DET,1,0,3,1,0,5 +2011,20296,ANA,0,2,0,0,0,2 +2011,20296,EDM,0,2,2,0,0,4 +2011,20297,PHX,0,2,1,0,0,3 +2011,20297,VAN,0,2,0,0,0,2 +2011,20298,DAL,0,1,0,0,0,1 +2011,20298,TOR,1,2,1,0,0,4 +2011,20299,NJD,3,2,0,0,0,5 +2011,20299,WSH,0,0,0,0,0,0 +2011,20300,CGY,0,1,0,0,0,1 +2011,20300,NYR,0,2,0,0,0,2 +2011,20301,MTL,2,0,0,0,0,2 +2011,20301,PHI,0,1,2,0,0,3 +2011,20302,CBJ,1,1,0,0,0,2 +2011,20302,NSH,0,0,0,0,0,0 +2011,20303,LAK,1,1,0,0,0,2 +2011,20303,OTT,1,1,1,0,0,3 +2011,20304,BOS,0,0,1,0,0,1 +2011,20304,TBL,0,3,0,0,0,3 +2011,20305,FLA,1,1,0,0,0,2 +2011,20305,PIT,2,0,1,0,0,3 +2011,20306,EDM,0,0,0,0,0,0 +2011,20306,PHX,1,2,2,0,0,5 +2011,20307,BUF,0,0,0,0,0,0 +2011,20307,PIT,1,0,0,0,0,1 +2011,20308,CBJ,2,1,0,1,0,4 +2011,20308,NYI,1,2,0,0,0,3 +2011,20309,CAR,0,0,2,0,0,2 +2011,20309,WSH,1,1,1,0,0,3 +2011,20310,ATL,2,2,1,0,0,5 +2011,20310,DET,1,0,0,0,0,1 +2011,20311,LAK,0,1,0,0,0,1 +2011,20311,MTL,2,2,0,0,0,4 +2011,20312,DAL,0,1,1,0,0,2 +2011,20312,OTT,0,1,0,0,0,1 +2011,20313,CGY,0,0,1,0,0,1 +2011,20313,NJD,1,0,0,0,1,2 +2011,20314,NYR,0,0,3,0,0,3 +2011,20314,TBL,2,3,0,0,0,5 +2011,20315,BOS,0,0,3,0,0,3 +2011,20315,FLA,0,1,0,0,0,1 +2011,20316,NSH,0,1,0,0,0,1 +2011,20316,STL,1,0,0,0,1,2 +2011,20317,MIN,0,1,0,0,0,1 +2011,20317,PHI,2,1,3,0,0,6 +2011,20318,COL,0,1,1,0,0,2 +2011,20318,VAN,1,1,2,0,0,4 +2011,20319,CHI,0,1,1,0,0,2 +2011,20319,SJS,1,2,2,0,0,5 +2011,20320,COL,1,1,0,0,0,2 +2011,20320,EDM,2,0,1,0,0,3 +2011,20321,BOS,0,0,0,0,0,0 +2011,20321,CAR,1,1,1,0,0,3 +2011,20322,NJD,0,0,0,0,0,0 +2011,20322,NYI,1,1,0,0,0,2 +2011,20323,CGY,0,1,1,0,1,3 +2011,20323,PHI,0,2,0,0,0,2 +2011,20324,OTT,1,0,0,0,0,1 +2011,20324,PIT,0,1,1,0,0,2 +2011,20325,MIN,3,1,1,0,0,5 +2011,20325,NSH,0,0,2,0,0,2 +2011,20326,ANA,1,0,0,0,0,1 +2011,20326,CHI,0,1,3,0,0,4 +2011,20327,TBL,0,0,0,0,0,0 +2011,20327,WSH,1,4,1,0,0,6 +2011,20328,CBJ,0,0,1,0,0,1 +2011,20328,DET,0,1,1,0,0,2 +2011,20329,BUF,2,0,1,0,0,3 +2011,20329,TOR,0,0,1,0,0,1 +2011,20330,ATL,1,2,0,0,0,3 +2011,20330,MTL,0,0,0,0,0,0 +2011,20331,FLA,0,0,0,0,0,0 +2011,20331,NYR,1,0,2,0,0,3 +2011,20332,DAL,0,1,2,0,0,3 +2011,20332,STL,1,1,0,0,0,2 +2011,20333,SJS,1,0,0,0,0,1 +2011,20333,VAN,1,2,3,0,0,6 +2011,20334,NJD,1,0,0,0,1,2 +2011,20334,PHI,0,0,1,0,0,1 +2011,20335,CGY,0,0,1,0,0,1 +2011,20335,PIT,0,2,2,0,0,4 +2011,20336,BUF,0,0,1,0,0,1 +2011,20336,MTL,1,2,0,0,0,3 +2011,20337,OTT,1,2,0,0,0,3 +2011,20337,TOR,0,0,0,0,0,0 +2011,20338,FLA,0,1,2,0,1,4 +2011,20338,TBL,1,1,1,0,0,3 +2011,20339,DAL,0,0,2,0,0,2 +2011,20339,STL,0,1,0,0,0,1 +2011,20340,NSH,1,0,0,0,0,1 +2011,20340,NYR,0,0,1,0,1,2 +2011,20341,ANA,1,2,3,0,0,6 +2011,20341,PHX,1,2,1,0,0,4 +2011,20342,COL,1,4,2,0,0,7 +2011,20342,MIN,2,1,1,0,0,4 +2011,20343,EDM,1,0,2,0,0,3 +2011,20343,SJS,1,2,1,0,0,4 +2011,20344,CHI,0,1,1,0,0,2 +2011,20344,LAK,0,0,1,0,0,1 +2011,20345,CAR,0,1,1,0,0,2 +2011,20345,WSH,2,0,0,0,1,3 +2011,20346,ATL,3,1,0,0,0,4 +2011,20346,BOS,0,1,0,0,0,1 +2011,20347,CBJ,1,0,1,0,0,2 +2011,20347,DET,0,2,2,0,0,4 +2011,20348,NYR,0,1,0,0,0,1 +2011,20348,PIT,1,2,0,0,0,3 +2011,20349,CAR,0,1,0,0,0,1 +2011,20349,DAL,1,2,1,0,0,4 +2011,20350,EDM,0,1,3,0,0,4 +2011,20350,OTT,1,0,0,0,0,1 +2011,20351,CGY,0,2,1,0,0,3 +2011,20351,MIN,0,0,0,0,0,0 +2011,20352,ANA,0,1,1,0,0,2 +2011,20352,LAK,0,0,0,0,0,0 +2011,20353,TBL,0,1,2,1,0,4 +2011,20353,TOR,1,1,1,0,0,3 +2011,20354,NSH,1,1,1,0,0,3 +2011,20354,PHX,0,0,0,0,0,0 +2011,20355,CHI,2,3,2,0,0,7 +2011,20355,STL,1,1,3,0,0,5 +2011,20356,ATL,1,0,1,1,0,3 +2011,20356,COL,0,0,2,0,0,2 +2011,20357,DET,2,2,1,0,0,5 +2011,20357,SJS,1,1,1,0,0,3 +2011,20358,EDM,1,0,2,1,0,4 +2011,20358,MTL,1,2,0,0,0,3 +2011,20359,BOS,2,0,1,0,0,3 +2011,20359,PHI,0,0,0,0,0,0 +2011,20360,CBJ,0,1,2,0,0,3 +2011,20360,NSH,1,1,1,0,1,4 +2011,20361,STL,0,1,0,0,0,1 +2011,20361,WSH,1,1,2,0,0,4 +2011,20362,MIN,0,0,2,0,0,2 +2011,20362,PHX,1,1,2,0,0,4 +2011,20363,CGY,1,1,0,0,0,2 +2011,20363,VAN,2,1,4,0,0,7 +2011,20364,ANA,1,2,2,0,0,5 +2011,20364,FLA,0,1,2,0,0,3 +2011,20365,BOS,2,2,4,0,0,8 +2011,20365,TBL,0,1,0,0,0,1 +2011,20366,EDM,1,2,2,0,0,5 +2011,20366,TOR,0,0,0,0,0,0 +2011,20367,MTL,2,3,0,0,0,5 +2011,20367,NJD,0,0,1,0,0,1 +2011,20368,NYI,1,2,2,0,0,5 +2011,20368,NYR,2,2,2,0,0,6 +2011,20369,ATL,1,1,0,0,0,2 +2011,20369,PIT,1,2,0,0,0,3 +2011,20370,OTT,0,0,0,0,0,0 +2011,20370,SJS,1,1,2,0,0,4 +2011,20371,DAL,0,1,1,0,0,2 +2011,20371,WSH,0,0,1,0,0,1 +2011,20372,FLA,1,1,0,0,0,2 +2011,20372,LAK,1,0,2,0,0,3 +2011,20373,NYI,0,0,0,0,0,0 +2011,20373,NYR,0,1,1,0,0,2 +2011,20374,CAR,0,0,1,1,0,2 +2011,20374,COL,0,0,1,0,0,1 +2011,20375,BUF,2,1,2,0,0,5 +2011,20375,CBJ,0,0,0,0,0,0 +2011,20376,CGY,0,1,1,0,1,3 +2011,20376,MIN,0,1,1,0,0,2 +2011,20377,CHI,0,0,0,0,0,0 +2011,20377,VAN,0,1,2,0,0,3 +2011,20378,ANA,0,0,0,0,0,0 +2011,20378,DET,1,2,1,0,0,4 +2011,20379,NJD,2,0,1,0,0,3 +2011,20379,PHI,1,1,3,0,0,5 +2011,20380,MTL,1,1,1,0,0,3 +2011,20380,SJS,1,0,0,0,0,1 +2011,20381,BOS,1,0,1,0,0,2 +2011,20381,TOR,1,0,1,0,1,3 +2011,20382,BUF,0,0,0,0,1,1 +2011,20382,OTT,0,0,0,0,0,0 +2011,20383,ATL,0,2,1,0,0,3 +2011,20383,WSH,0,1,0,0,0,1 +2011,20384,CBJ,0,1,1,0,0,2 +2011,20384,PIT,4,3,0,0,0,7 +2011,20385,COL,1,1,3,0,0,5 +2011,20385,TBL,1,3,2,0,0,6 +2011,20386,CAR,1,1,0,0,0,2 +2011,20386,NSH,3,0,2,0,0,5 +2011,20387,DAL,1,1,1,1,0,4 +2011,20387,MIN,0,1,2,0,0,3 +2011,20388,FLA,0,1,0,0,1,2 +2011,20388,PHX,0,1,0,0,0,1 +2011,20389,EDM,0,0,1,1,0,2 +2011,20389,STL,0,0,1,0,0,1 +2011,20390,DET,1,1,0,0,0,2 +2011,20390,LAK,0,2,0,1,0,3 +2011,20391,NYI,0,0,2,0,0,2 +2011,20391,PHI,1,0,2,0,0,3 +2011,20392,NYR,0,1,0,0,0,1 +2011,20392,OTT,0,1,2,0,0,3 +2011,20393,CGY,2,0,0,0,0,2 +2011,20393,CHI,3,0,1,0,0,4 +2011,20394,ANA,0,0,0,0,0,0 +2011,20394,PHX,1,1,1,0,0,3 +2011,20395,STL,1,1,1,0,0,3 +2011,20395,VAN,1,1,0,0,0,2 +2011,20396,NJD,1,0,0,0,0,1 +2011,20396,PIT,1,1,0,0,0,2 +2011,20397,TOR,1,0,3,0,1,5 +2011,20397,WSH,2,2,0,0,0,4 +2011,20398,ATL,0,1,1,1,0,3 +2011,20398,NSH,2,0,0,0,0,2 +2011,20399,DET,2,0,0,0,0,2 +2011,20399,SJS,1,3,1,0,0,5 +2011,20400,CBJ,1,0,1,0,1,3 +2011,20400,DAL,0,1,1,0,0,2 +2011,20401,BOS,1,0,1,1,0,3 +2011,20401,BUF,0,1,1,0,0,2 +2011,20402,MTL,1,0,3,0,0,4 +2011,20402,OTT,1,0,0,0,0,1 +2011,20403,COL,0,2,1,0,0,3 +2011,20403,FLA,1,1,1,1,0,4 +2011,20404,CGY,2,1,1,0,0,4 +2011,20404,TBL,0,2,0,0,0,2 +2011,20405,ANA,0,1,1,0,1,3 +2011,20405,EDM,0,0,2,0,0,2 +2011,20406,PHI,0,3,1,0,0,4 +2011,20406,SJS,1,0,3,0,1,5 +2011,20407,PIT,1,3,1,0,0,5 +2011,20407,TOR,0,0,2,0,0,2 +2011,20408,DET,0,1,1,0,0,2 +2011,20408,NSH,1,1,1,0,0,3 +2011,20409,CHI,2,2,1,0,0,5 +2011,20409,DAL,0,3,0,0,0,3 +2011,20410,ANA,1,2,1,0,0,4 +2011,20410,VAN,2,0,2,0,1,5 +2011,20411,BOS,1,1,3,0,0,5 +2011,20411,NYI,0,1,1,0,0,2 +2011,20412,BUF,1,2,3,0,0,6 +2011,20412,SJS,0,2,1,0,0,3 +2011,20413,PHI,2,2,0,0,0,4 +2011,20413,TOR,0,1,0,0,0,1 +2011,20414,FLA,0,1,2,0,0,3 +2011,20414,WSH,0,0,0,0,0,0 +2011,20415,NYR,1,1,3,0,0,5 +2011,20415,OTT,1,2,0,0,0,3 +2011,20416,CBJ,0,0,1,0,0,1 +2011,20416,STL,2,1,1,0,0,4 +2011,20417,MIN,2,0,1,0,0,3 +2011,20417,PHX,1,0,1,0,0,2 +2011,20418,CGY,0,0,1,0,0,1 +2011,20418,LAK,1,0,1,0,0,2 +2011,20419,DET,1,2,1,0,0,4 +2011,20419,MTL,1,0,1,0,0,2 +2011,20420,NJD,0,2,0,0,0,2 +2011,20420,OTT,1,2,0,0,0,3 +2011,20421,ATL,1,1,0,0,0,2 +2011,20421,COL,0,2,2,0,0,4 +2011,20422,CAR,0,1,0,0,0,1 +2011,20422,DAL,1,0,0,0,1,2 +2011,20423,EDM,1,2,0,0,1,4 +2011,20423,TBL,2,0,1,0,0,3 +2011,20424,ANA,1,1,0,0,1,3 +2011,20424,CGY,0,0,2,0,0,2 +2011,20425,BOS,0,0,1,0,0,1 +2011,20425,PHI,0,1,0,1,0,2 +2011,20426,BUF,1,0,1,0,0,2 +2011,20426,PIT,2,0,3,0,0,5 +2011,20427,MTL,0,1,0,0,0,1 +2011,20427,TOR,2,0,1,0,0,3 +2011,20428,DET,2,1,1,0,0,4 +2011,20428,NJD,0,0,1,0,0,1 +2011,20429,ATL,0,0,0,0,0,0 +2011,20429,NYI,0,0,0,0,0,0 +2011,20430,COL,2,1,0,0,0,3 +2011,20430,WSH,1,0,1,0,0,2 +2011,20431,CBJ,0,1,2,0,0,3 +2011,20431,NYR,0,1,0,0,0,1 +2011,20432,CAR,0,0,1,0,1,2 +2011,20432,STL,0,0,1,0,0,1 +2011,20433,FLA,0,0,0,0,0,0 +2011,20433,NSH,2,0,1,0,0,3 +2011,20434,DAL,0,1,1,0,0,2 +2011,20434,PHX,3,0,2,0,0,5 +2011,20435,TBL,1,3,0,1,0,5 +2011,20435,VAN,1,1,2,0,0,4 +2011,20436,LAK,0,1,1,0,0,2 +2011,20436,MIN,1,1,0,1,0,3 +2011,20437,CHI,1,0,0,0,0,1 +2011,20437,SJS,1,0,0,1,0,2 +2011,20438,NYR,1,4,2,0,0,7 +2011,20438,WSH,0,0,0,0,0,0 +2011,20439,EDM,0,0,1,0,0,1 +2011,20439,VAN,1,1,0,0,0,2 +2011,20440,ANA,2,3,1,0,0,6 +2011,20440,MIN,1,1,0,0,0,2 +2011,20441,DET,0,0,0,0,0,0 +2011,20441,LAK,1,1,3,0,0,5 +2011,20442,ATL,2,1,0,1,0,4 +2011,20442,OTT,0,1,2,0,0,3 +2011,20443,NSH,1,2,2,0,0,5 +2011,20443,NYI,0,0,0,0,0,0 +2011,20444,CHI,3,1,1,0,0,5 +2011,20444,COL,3,1,3,0,0,7 +2011,20445,CGY,2,0,0,1,0,3 +2011,20445,CBJ,0,1,1,0,0,2 +2011,20446,DAL,1,1,0,0,1,3 +2011,20446,SJS,0,2,0,0,0,2 +2011,20447,PHI,1,1,1,0,0,3 +2011,20447,PIT,0,1,1,0,0,2 +2011,20448,EDM,1,0,0,0,0,1 +2011,20448,TOR,1,2,1,0,0,4 +2011,20449,BOS,1,0,1,0,0,2 +2011,20449,BUF,1,0,2,0,0,3 +2011,20450,NJD,2,0,1,0,0,3 +2011,20450,PHX,0,0,0,0,0,0 +2011,20451,NYR,0,0,4,0,0,4 +2011,20451,PIT,1,0,0,0,0,1 +2011,20452,ANA,0,1,0,1,0,2 +2011,20452,WSH,1,0,0,0,0,1 +2011,20453,DET,0,3,2,0,0,5 +2011,20453,STL,0,1,1,0,0,2 +2011,20454,MTL,0,1,2,0,0,3 +2011,20454,PHI,1,1,3,0,0,5 +2011,20455,ATL,1,0,0,0,0,1 +2011,20455,TBL,0,1,0,0,1,2 +2011,20456,CAR,0,1,3,0,0,4 +2011,20456,FLA,3,0,0,0,0,3 +2011,20457,NSH,0,1,2,0,0,3 +2011,20457,SJS,1,1,0,0,0,2 +2011,20458,CHI,0,1,2,0,0,3 +2011,20458,COL,1,0,3,0,0,4 +2011,20459,CBJ,0,1,1,0,0,2 +2011,20459,VAN,1,1,0,1,0,3 +2011,20460,ANA,0,1,1,0,0,2 +2011,20460,NYI,3,0,0,0,0,3 +2011,20461,NYR,1,1,1,0,1,4 +2011,20461,PHX,2,1,0,0,0,3 +2011,20462,ATL,0,2,0,0,0,2 +2011,20462,CAR,0,0,2,0,1,3 +2011,20463,BOS,1,1,1,0,0,3 +2011,20463,MTL,3,1,0,0,0,4 +2011,20464,LAK,1,3,0,0,0,4 +2011,20464,STL,1,3,2,0,0,6 +2011,20465,MIN,1,0,0,0,0,1 +2011,20465,OTT,0,2,1,0,0,3 +2011,20466,DAL,1,1,1,0,0,3 +2011,20466,SJS,1,2,0,1,0,4 +2011,20467,CGY,1,3,1,0,0,5 +2011,20467,TOR,1,0,1,0,0,2 +2011,20468,CBJ,2,0,1,0,0,3 +2011,20468,EDM,3,3,0,0,0,6 +2011,20469,NSH,1,1,1,0,0,3 +2011,20469,NJD,0,0,1,0,0,1 +2011,20470,BUF,0,1,1,0,0,2 +2011,20470,FLA,4,1,1,0,0,6 +2011,20471,CHI,2,1,1,0,0,4 +2011,20471,DET,1,0,0,0,0,1 +2011,20472,COL,1,2,2,1,0,6 +2011,20472,OTT,2,1,2,0,0,5 +2011,20473,NYR,0,0,1,0,0,1 +2011,20473,PHI,0,1,3,0,0,4 +2011,20474,BOS,3,0,0,0,0,3 +2011,20474,WSH,0,1,1,0,0,2 +2011,20475,NYI,0,2,1,0,0,3 +2011,20475,PHX,1,0,2,0,1,4 +2011,20476,ANA,1,0,1,0,0,2 +2011,20476,CAR,2,1,1,0,0,4 +2011,20477,ATL,3,1,3,0,0,7 +2011,20477,NJD,0,1,0,0,0,1 +2011,20478,CBJ,1,0,0,0,0,1 +2011,20478,DAL,1,0,1,0,0,2 +2011,20479,BUF,0,1,0,0,0,1 +2011,20479,TBL,0,2,1,0,0,3 +2011,20480,SJS,2,1,1,0,0,4 +2011,20480,STL,0,1,0,0,0,1 +2011,20481,LAK,4,2,0,0,0,6 +2011,20481,NSH,1,0,0,0,0,1 +2011,20482,CGY,0,0,1,0,0,1 +2011,20482,MIN,0,2,1,0,0,3 +2011,20483,TOR,0,0,1,0,0,1 +2011,20483,VAN,1,1,2,0,0,4 +2011,20484,OTT,2,0,0,0,0,2 +2011,20484,WSH,0,3,0,0,0,3 +2011,20485,DAL,2,0,1,1,0,4 +2011,20485,DET,2,1,0,0,0,3 +2011,20486,CHI,1,2,0,0,0,3 +2011,20486,LAK,0,1,1,0,0,2 +2011,20487,COL,1,2,0,0,0,3 +2011,20487,MTL,1,1,0,0,0,2 +2011,20488,ANA,1,2,0,0,0,3 +2011,20488,BOS,0,0,0,0,0,0 +2011,20489,ATL,2,1,3,0,0,6 +2011,20489,TOR,0,1,2,0,0,3 +2011,20490,FLA,2,2,1,0,0,5 +2011,20490,PHI,0,0,0,0,0,0 +2011,20491,PHX,0,0,1,0,0,1 +2011,20491,PIT,2,3,1,0,0,6 +2011,20492,CAR,1,0,0,0,0,1 +2011,20492,TBL,2,3,0,0,0,5 +2011,20493,STL,0,1,0,0,0,1 +2011,20493,VAN,0,2,1,0,0,3 +2011,20494,CGY,0,1,0,0,0,1 +2011,20494,MIN,1,0,3,0,0,4 +2011,20495,ANA,0,0,2,0,0,2 +2011,20495,BUF,3,1,1,0,0,5 +2011,20496,NJD,0,1,0,0,0,1 +2011,20496,WSH,1,3,1,0,0,5 +2011,20497,ATL,1,0,1,0,0,2 +2011,20497,STL,1,1,2,0,0,4 +2011,20498,CGY,0,1,0,0,0,1 +2011,20498,CBJ,1,1,1,0,0,3 +2011,20499,DAL,2,2,1,0,0,5 +2011,20499,MTL,0,1,1,0,0,2 +2011,20500,COL,0,0,0,0,0,0 +2011,20500,LAK,1,1,3,0,0,5 +2011,20501,EDM,0,0,1,0,0,1 +2011,20501,SJS,0,2,0,0,0,2 +2011,20502,NYI,0,1,0,1,0,2 +2011,20502,TBL,1,0,0,0,0,1 +2011,20503,FLA,1,0,1,0,0,2 +2011,20503,PIT,3,1,1,0,0,5 +2011,20504,DET,1,1,2,1,0,5 +2011,20504,VAN,0,3,1,0,0,4 +2011,20505,CHI,1,1,2,0,0,4 +2011,20505,NSH,0,0,1,0,0,1 +2011,20506,ATL,0,1,0,0,0,1 +2011,20506,BOS,1,2,1,0,0,4 +2011,20507,BUF,0,3,0,0,0,3 +2011,20507,FLA,3,0,1,0,0,4 +2011,20508,NJD,0,1,0,0,0,1 +2011,20508,NYI,3,1,1,0,0,5 +2011,20509,NYR,2,0,1,0,0,3 +2011,20509,TBL,2,1,0,0,1,4 +2011,20510,PIT,1,0,1,0,1,3 +2011,20510,WSH,0,1,1,0,0,2 +2011,20511,CAR,0,2,0,0,0,2 +2011,20511,MTL,0,3,0,0,0,3 +2011,20512,CBJ,0,1,2,0,0,3 +2011,20512,VAN,3,3,1,0,0,7 +2011,20513,DET,1,2,0,0,0,3 +2011,20513,STL,3,1,0,0,0,4 +2011,20514,NSH,0,1,0,0,0,1 +2011,20514,OTT,1,1,0,0,0,2 +2011,20515,CGY,1,0,1,0,1,3 +2011,20515,DAL,1,0,1,0,0,2 +2011,20516,COL,1,0,0,0,0,1 +2011,20516,MIN,2,1,0,0,0,3 +2011,20517,EDM,0,1,1,0,0,2 +2011,20517,LAK,0,1,1,0,1,3 +2011,20518,PHX,0,1,0,0,0,1 +2011,20518,SJS,0,2,2,0,0,4 +2011,20519,OTT,2,1,0,0,0,3 +2011,20519,PIT,0,0,1,0,0,1 +2011,20520,NJD,0,0,1,0,0,1 +2011,20520,TOR,1,2,1,0,0,4 +2011,20521,MTL,0,0,1,0,0,1 +2011,20521,NYI,1,1,2,0,0,4 +2011,20522,CAR,0,1,1,0,0,2 +2011,20522,WSH,1,2,0,0,0,3 +2011,20523,ATL,0,1,1,0,0,2 +2011,20523,TBL,1,1,0,1,0,3 +2011,20524,CHI,1,0,3,0,0,4 +2011,20524,CBJ,0,0,1,0,0,1 +2011,20525,NSH,0,0,0,0,0,0 +2011,20525,STL,0,1,1,0,0,2 +2011,20526,DET,2,2,0,0,0,4 +2011,20526,MIN,0,0,1,0,0,1 +2011,20527,DAL,0,0,0,0,0,0 +2011,20527,PHX,0,1,0,0,0,1 +2011,20528,EDM,0,2,0,0,0,2 +2011,20528,VAN,0,1,2,0,0,3 +2011,20529,ANA,0,1,0,0,0,1 +2011,20529,LAK,0,4,0,0,0,4 +2011,20530,NYI,2,0,0,0,0,2 +2011,20530,NYR,2,2,3,0,0,7 +2011,20531,CBJ,0,1,2,0,1,4 +2011,20531,MIN,1,1,1,0,0,3 +2011,20532,BOS,0,1,1,0,1,3 +2011,20532,FLA,0,2,0,0,0,2 +2011,20533,COL,2,0,1,0,0,3 +2011,20533,DET,2,0,1,1,0,4 +2011,20534,BUF,1,0,1,0,0,2 +2011,20534,CGY,0,2,3,0,0,5 +2011,20535,LAK,0,0,4,0,0,4 +2011,20535,SJS,0,0,0,0,0,0 +2011,20536,CAR,3,0,1,0,0,4 +2011,20536,TOR,2,1,0,0,0,3 +2011,20537,ATL,2,0,1,0,0,3 +2011,20537,PIT,2,1,3,0,0,6 +2011,20538,MTL,0,0,0,0,0,0 +2011,20538,WSH,2,0,1,0,0,3 +2011,20539,BOS,1,1,2,0,0,4 +2011,20539,TBL,1,1,1,0,0,3 +2011,20540,CHI,1,0,0,0,0,1 +2011,20540,STL,0,1,2,0,0,3 +2011,20541,DAL,0,1,3,0,0,4 +2011,20541,NSH,0,1,1,0,0,2 +2011,20542,BUF,1,1,2,0,0,4 +2011,20542,EDM,1,0,1,0,0,2 +2011,20543,ANA,3,0,0,0,0,3 +2011,20543,PHX,0,1,0,0,0,1 +2011,20544,PHI,0,1,1,0,0,2 +2011,20544,VAN,1,3,2,0,0,6 +2011,20545,NJD,1,0,0,0,0,1 +2011,20545,NYR,1,1,1,0,0,3 +2011,20546,NYI,0,1,0,0,1,2 +2011,20546,PIT,0,1,0,0,0,1 +2011,20547,CAR,2,1,1,0,0,4 +2011,20547,OTT,0,0,0,0,0,0 +2011,20548,MIN,1,1,3,0,0,5 +2011,20548,SJS,1,2,0,0,0,3 +2011,20549,DAL,0,3,0,0,0,3 +2011,20549,DET,1,2,4,0,0,7 +2011,20550,LAK,3,0,0,0,0,3 +2011,20550,PHX,4,2,0,0,0,6 +2011,20551,CBJ,1,1,1,0,0,3 +2011,20551,TOR,2,0,0,0,0,2 +2011,20552,ATL,1,1,0,0,1,3 +2011,20552,BOS,1,1,0,0,0,2 +2011,20553,MTL,1,0,0,0,0,1 +2011,20553,TBL,0,2,2,0,0,4 +2011,20554,CHI,1,2,0,0,0,3 +2011,20554,SJS,1,2,2,0,0,5 +2011,20555,COL,3,0,0,0,1,4 +2011,20555,EDM,0,1,2,0,0,3 +2011,20556,LAK,2,2,0,0,0,4 +2011,20556,PHI,2,4,1,0,0,7 +2011,20557,ATL,0,0,1,0,0,1 +2011,20557,NJD,1,1,1,0,0,3 +2011,20558,FLA,0,2,0,0,0,2 +2011,20558,MTL,0,2,0,1,0,3 +2011,20559,MIN,0,0,1,0,0,1 +2011,20559,NSH,1,1,2,0,0,4 +2011,20560,DET,1,1,1,0,0,3 +2011,20560,NYI,2,1,0,1,0,4 +2011,20561,CBJ,1,2,0,1,0,4 +2011,20561,OTT,0,1,2,0,0,3 +2011,20562,PHX,0,0,3,0,0,3 +2011,20562,STL,1,2,1,0,0,4 +2011,20563,ANA,1,2,2,0,0,5 +2011,20563,PHI,0,1,1,0,0,2 +2011,20564,DAL,0,0,1,0,0,1 +2011,20564,VAN,2,2,0,0,0,4 +2011,20565,CGY,0,3,0,0,0,3 +2011,20565,COL,0,0,2,0,0,2 +2011,20566,PIT,0,1,0,0,0,1 +2011,20566,WSH,0,2,1,0,0,3 +2011,20567,BOS,4,0,2,0,0,6 +2011,20567,BUF,3,2,1,0,1,7 +2011,20568,OTT,0,1,0,0,0,1 +2011,20568,TOR,3,2,0,0,0,5 +2011,20569,CAR,4,1,1,0,0,6 +2011,20569,NJD,1,1,1,0,0,3 +2011,20570,NYR,0,0,1,0,0,1 +2011,20570,TBL,1,0,0,1,0,2 +2011,20571,LAK,0,0,0,0,0,0 +2011,20571,SJS,0,1,0,0,0,1 +2011,20572,CGY,1,1,0,0,0,2 +2011,20572,EDM,0,1,0,0,0,1 +2011,20573,ATL,1,2,0,1,0,4 +2011,20573,MTL,1,1,1,0,0,3 +2011,20574,FLA,0,1,2,0,0,3 +2011,20574,NYR,0,0,0,0,0,0 +2011,20575,DET,0,0,2,0,0,2 +2011,20575,PHI,0,3,0,0,0,3 +2011,20576,DAL,1,0,3,0,0,4 +2011,20576,STL,2,0,0,0,0,2 +2011,20577,CBJ,1,0,0,0,0,1 +2011,20577,NSH,0,3,1,0,0,4 +2011,20578,MIN,1,2,2,1,0,6 +2011,20578,PHX,1,0,4,0,0,5 +2011,20579,COL,0,0,1,0,0,1 +2011,20579,VAN,1,0,1,0,0,2 +2011,20580,ANA,1,1,0,0,0,2 +2011,20580,CHI,1,0,0,0,0,1 +2011,20581,BOS,0,2,0,0,0,2 +2011,20581,TOR,1,0,0,0,0,1 +2011,20582,CAR,1,0,2,0,0,3 +2011,20582,FLA,1,1,1,1,0,4 +2011,20583,CGY,1,1,0,0,0,2 +2011,20583,NYI,3,1,1,0,0,5 +2011,20584,CHI,1,1,2,0,0,4 +2011,20584,LAK,1,1,1,0,0,3 +2011,20585,SJS,0,3,0,0,0,3 +2011,20585,VAN,1,2,1,0,0,4 +2011,20586,MIN,1,0,1,0,0,2 +2011,20586,NJD,0,1,0,0,0,1 +2011,20587,TBL,0,0,0,1,0,1 +2011,20587,WSH,0,0,0,0,0,0 +2011,20588,BUF,1,0,2,0,0,3 +2011,20588,COL,2,0,1,1,0,4 +2011,20589,DET,1,2,2,0,0,5 +2011,20589,EDM,1,0,2,0,0,3 +2011,20590,CBJ,0,1,1,0,0,2 +2011,20590,PHX,0,2,2,0,0,4 +2011,20591,CAR,0,0,1,0,0,1 +2011,20591,NYR,0,0,1,1,0,2 +2011,20592,PIT,5,2,1,0,0,8 +2011,20592,TBL,0,0,1,0,0,1 +2011,20593,ATL,2,1,0,0,0,3 +2011,20593,FLA,1,0,1,0,0,2 +2011,20594,CHI,0,2,0,0,0,2 +2011,20594,DAL,2,0,2,0,0,4 +2011,20595,CGY,0,0,1,0,0,1 +2011,20595,VAN,2,1,0,0,0,3 +2011,20596,ANA,0,0,1,0,0,1 +2011,20596,NSH,1,1,2,0,0,4 +2011,20597,BOS,0,1,0,0,0,1 +2011,20597,MIN,0,1,2,0,0,3 +2011,20598,STL,2,0,3,0,0,5 +2011,20598,TOR,1,3,1,0,1,6 +2011,20599,NJD,1,1,0,0,0,2 +2011,20599,PHI,0,2,2,0,0,4 +2011,20600,MTL,0,1,0,0,1,2 +2011,20600,PIT,1,0,0,0,0,1 +2011,20601,COL,0,0,0,0,0,0 +2011,20601,PHX,1,0,1,0,0,2 +2011,20602,EDM,2,0,0,0,0,2 +2011,20602,NYI,0,1,0,0,0,1 +2011,20603,LAK,0,2,0,0,0,2 +2011,20603,NSH,0,3,2,0,0,5 +2011,20604,BUF,2,1,0,0,0,3 +2011,20604,SJS,0,0,0,0,0,0 +2011,20605,ATL,1,0,2,0,0,3 +2011,20605,TOR,2,6,1,0,0,9 +2011,20606,CAR,0,2,3,0,0,5 +2011,20606,FLA,2,0,1,0,0,3 +2011,20607,CHI,0,1,1,0,1,3 +2011,20607,OTT,0,1,1,0,0,2 +2011,20608,DAL,1,1,0,0,0,2 +2011,20608,NYR,0,1,1,0,1,3 +2011,20609,CGY,2,2,0,0,0,4 +2011,20609,DET,1,2,1,0,1,5 +2011,20610,EDM,0,1,0,0,0,1 +2011,20610,VAN,0,3,3,0,0,6 +2011,20611,ANA,1,2,3,0,0,6 +2011,20611,CBJ,0,0,0,0,0,0 +2011,20612,NJD,0,1,0,0,0,1 +2011,20612,PHI,2,0,0,0,0,2 +2011,20613,COL,2,0,1,0,0,3 +2011,20613,NYI,2,1,0,1,0,4 +2011,20614,BOS,0,2,0,0,0,2 +2011,20614,MTL,0,0,2,1,0,3 +2011,20615,OTT,0,1,0,0,0,1 +2011,20615,TBL,0,0,2,0,0,2 +2011,20616,MIN,2,0,2,0,0,4 +2011,20616,PIT,0,0,0,0,0,0 +2011,20617,FLA,1,0,1,0,0,2 +2011,20617,WSH,0,1,2,0,0,3 +2011,20618,NYR,0,2,0,0,0,2 +2011,20618,STL,0,1,0,0,0,1 +2011,20619,BUF,0,1,0,1,0,2 +2011,20619,PHX,0,1,0,0,0,1 +2011,20620,NSH,1,0,1,0,0,2 +2011,20620,SJS,0,1,0,0,0,1 +2011,20621,DET,0,0,1,0,1,2 +2011,20621,VAN,0,1,0,0,0,1 +2011,20622,CBJ,0,1,3,0,0,4 +2011,20622,LAK,3,1,2,0,0,6 +2011,20623,ATL,1,1,1,0,0,3 +2011,20623,CAR,1,2,0,1,0,4 +2011,20624,NJD,1,0,5,0,0,6 +2011,20624,TBL,2,0,1,0,0,3 +2011,20625,DAL,2,0,2,0,0,4 +2011,20625,MIN,0,0,0,0,0,0 +2011,20626,CHI,1,4,0,0,0,5 +2011,20626,NYI,0,0,0,0,0,0 +2011,20627,ANA,0,1,0,0,0,1 +2011,20627,SJS,0,0,0,0,0,0 +2011,20628,BOS,0,0,4,0,0,4 +2011,20628,PIT,0,2,0,0,0,2 +2011,20629,PHX,0,2,2,0,0,4 +2011,20629,STL,1,0,2,0,0,3 +2011,20630,COL,4,1,0,0,0,5 +2011,20630,DET,2,1,1,0,0,4 +2011,20631,LAK,1,1,0,0,0,2 +2011,20631,TOR,0,2,1,0,0,3 +2011,20632,BOS,2,3,1,0,0,6 +2011,20632,OTT,0,0,0,0,0,0 +2011,20633,BUF,2,0,0,0,0,2 +2011,20633,PHI,1,1,3,0,0,5 +2011,20634,NYI,0,1,2,0,0,3 +2011,20634,VAN,1,1,1,0,1,4 +2011,20635,MTL,0,1,1,0,0,2 +2011,20635,NYR,1,0,0,0,0,1 +2011,20636,CGY,2,1,2,0,0,5 +2011,20636,CAR,2,3,0,0,1,6 +2011,20637,CBJ,2,1,0,0,0,3 +2011,20637,PHX,0,2,2,0,0,4 +2011,20638,FLA,2,1,0,1,0,4 +2011,20638,WSH,0,2,1,0,0,3 +2011,20639,MIN,1,0,0,0,0,1 +2011,20639,NSH,1,1,3,0,0,5 +2011,20640,DAL,1,1,1,0,0,3 +2011,20640,EDM,0,0,2,0,0,2 +2011,20641,SJS,1,0,1,0,0,2 +2011,20641,TOR,0,0,4,0,0,4 +2011,20642,MTL,1,1,0,0,0,2 +2011,20642,PIT,1,2,2,0,0,5 +2011,20643,TBL,2,1,0,0,0,3 +2011,20643,WSH,0,0,0,0,0,0 +2011,20644,CHI,2,1,1,0,0,4 +2011,20644,COL,0,0,0,0,0,0 +2011,20645,ANA,3,0,4,0,0,7 +2011,20645,STL,0,2,2,0,0,4 +2011,20646,BOS,1,1,5,0,0,7 +2011,20646,PHI,1,2,2,0,0,5 +2011,20647,BUF,1,1,1,0,0,3 +2011,20647,CAR,0,1,1,0,0,2 +2011,20648,NYI,1,1,2,0,0,4 +2011,20648,OTT,2,4,0,0,0,6 +2011,20649,NYR,0,1,0,0,0,1 +2011,20649,VAN,0,0,0,0,0,0 +2011,20650,FLA,1,0,2,0,0,3 +2011,20650,NSH,1,0,1,0,0,2 +2011,20651,PHX,1,0,4,0,0,5 +2011,20651,TOR,0,0,1,0,0,1 +2011,20652,LAK,0,1,0,0,0,1 +2011,20652,STL,1,2,0,0,0,3 +2011,20653,EDM,1,2,2,0,0,5 +2011,20653,SJS,0,0,2,0,0,2 +2011,20654,VAN,1,2,1,0,0,4 +2011,20654,WSH,1,0,1,0,0,2 +2011,20655,CBJ,2,0,0,0,1,3 +2011,20655,DET,0,2,0,0,0,2 +2011,20656,CGY,0,3,0,0,0,3 +2011,20656,OTT,0,0,2,0,0,2 +2011,20657,ATL,0,2,0,0,0,2 +2011,20657,PHI,0,2,3,0,0,5 +2011,20658,NJD,4,0,1,0,0,5 +2011,20658,TBL,1,1,0,0,0,2 +2011,20659,COL,2,1,1,0,0,4 +2011,20659,MIN,0,0,1,0,0,1 +2011,20660,BOS,0,2,0,0,0,2 +2011,20660,PIT,1,1,1,0,0,3 +2011,20661,CGY,0,0,1,0,1,2 +2011,20661,TOR,0,0,1,0,0,1 +2011,20662,MTL,3,0,0,0,0,3 +2011,20662,NYR,1,0,1,0,0,2 +2011,20663,BUF,0,1,2,0,0,3 +2011,20663,NYI,0,4,1,0,0,5 +2011,20664,CAR,2,3,1,0,0,6 +2011,20664,TBL,2,1,1,0,0,4 +2011,20665,FLA,1,1,0,1,0,3 +2011,20665,NJD,1,0,1,0,0,2 +2011,20666,CBJ,2,1,2,0,0,5 +2011,20666,DET,2,2,1,1,0,6 +2011,20667,CHI,1,1,0,0,0,2 +2011,20667,NSH,0,0,2,0,1,3 +2011,20668,ATL,0,1,0,0,0,1 +2011,20668,DAL,1,2,3,0,0,6 +2011,20669,ANA,1,0,1,0,0,2 +2011,20669,PHX,1,3,2,0,0,6 +2011,20670,EDM,2,0,0,0,0,2 +2011,20670,LAK,2,1,2,0,0,5 +2011,20671,SJS,1,1,2,0,0,4 +2011,20671,STL,1,1,0,0,0,2 +2011,20672,OTT,1,0,0,0,0,1 +2011,20672,WSH,0,0,3,0,0,3 +2011,20673,MIN,0,2,2,0,0,4 +2011,20673,VAN,0,0,0,0,0,0 +2011,20674,NYR,0,0,2,0,0,2 +2011,20674,PHI,2,1,0,0,0,3 +2011,20675,CHI,1,1,4,0,0,6 +2011,20675,NSH,2,0,1,0,0,3 +2011,20676,ANA,2,1,0,0,0,3 +2011,20676,EDM,0,2,0,0,0,2 +2011,20677,BOS,3,2,2,0,0,7 +2011,20677,CAR,0,0,0,0,0,0 +2011,20678,NJD,2,3,0,0,0,5 +2011,20678,NYI,0,2,0,0,0,2 +2011,20679,PHX,1,0,1,0,0,2 +2011,20679,SJS,1,2,1,0,0,4 +2011,20680,CGY,0,2,2,0,0,4 +2011,20680,MTL,2,2,0,1,0,5 +2011,20681,ATL,0,0,2,0,1,3 +2011,20681,FLA,0,0,2,0,0,2 +2011,20682,DAL,0,2,0,0,0,2 +2011,20682,LAK,1,0,0,0,0,1 +2011,20683,BUF,0,1,0,1,0,2 +2011,20683,MTL,1,0,0,0,0,1 +2011,20684,PHI,1,1,0,1,0,3 +2011,20684,WSH,0,0,2,0,0,2 +2011,20685,DET,0,1,0,0,0,1 +2011,20685,PIT,2,1,1,0,0,4 +2011,20686,BOS,1,0,2,0,0,3 +2011,20686,CAR,1,0,1,0,0,2 +2011,20687,ANA,1,0,0,0,1,2 +2011,20687,OTT,0,1,0,0,0,1 +2011,20688,CBJ,1,1,0,0,0,2 +2011,20688,TBL,1,1,0,0,1,3 +2011,20689,LAK,0,1,0,0,0,1 +2011,20689,STL,1,0,1,0,0,2 +2011,20690,COL,1,2,0,1,0,4 +2011,20690,VAN,1,2,0,0,0,3 +2011,20691,EDM,0,0,1,0,0,1 +2011,20691,MIN,2,0,2,0,0,4 +2011,20692,NSH,2,2,1,0,0,5 +2011,20692,PHX,2,0,0,0,0,2 +2011,20693,NYR,4,2,1,0,0,7 +2011,20693,TOR,0,0,0,0,0,0 +2011,20694,CBJ,1,0,1,1,0,3 +2011,20694,FLA,2,0,0,0,0,2 +2011,20695,CGY,0,0,0,0,0,0 +2011,20695,MIN,2,1,3,0,0,6 +2011,20696,BOS,1,1,0,0,0,2 +2011,20696,BUF,0,2,2,0,0,4 +2011,20697,ANA,1,1,0,0,0,2 +2011,20697,TOR,1,2,2,0,0,5 +2011,20698,NJD,2,0,0,0,0,2 +2011,20698,PIT,0,0,0,0,0,0 +2011,20699,NYI,0,1,0,0,0,1 +2011,20699,WSH,1,1,0,0,0,2 +2011,20700,OTT,0,2,0,0,0,2 +2011,20700,PHI,2,1,3,0,0,6 +2011,20701,CAR,2,1,1,0,0,4 +2011,20701,NYR,0,0,1,0,0,1 +2011,20702,ATL,1,1,0,0,0,2 +2011,20702,TBL,1,0,1,0,1,3 +2011,20703,DET,1,2,0,1,0,4 +2011,20703,STL,0,1,2,0,0,3 +2011,20704,COL,0,0,1,0,0,1 +2011,20704,NSH,2,0,3,0,0,5 +2011,20705,DAL,1,2,1,0,0,4 +2011,20705,EDM,1,1,0,0,0,2 +2011,20706,SJS,0,1,0,0,1,2 +2011,20706,VAN,1,0,0,0,0,1 +2011,20707,LAK,0,0,0,0,0,0 +2011,20707,PHX,0,2,0,0,0,2 +2011,20708,BUF,0,0,2,0,0,2 +2011,20708,NYI,2,1,2,0,0,5 +2011,20709,MTL,2,3,2,0,0,7 +2011,20709,OTT,1,0,0,0,0,1 +2011,20710,FLA,1,0,0,0,0,1 +2011,20710,TBL,1,0,0,0,1,2 +2011,20711,CGY,2,3,2,0,0,7 +2011,20711,DAL,2,1,1,0,0,4 +2011,20712,NJD,2,0,1,0,0,3 +2011,20712,PHI,0,1,0,0,0,1 +2011,20713,CHI,1,2,1,0,0,4 +2011,20713,DET,1,0,0,0,0,1 +2011,20714,BOS,2,2,2,0,0,6 +2011,20714,COL,1,0,1,0,0,2 +2011,20715,TOR,0,0,1,0,0,1 +2011,20715,WSH,1,0,3,0,0,4 +2011,20716,ANA,1,2,0,0,1,4 +2011,20716,MTL,0,1,2,0,0,3 +2011,20717,CAR,0,0,2,0,0,2 +2011,20717,PIT,0,1,2,0,0,3 +2011,20718,ATL,1,1,0,0,0,2 +2011,20718,NYR,0,1,1,0,1,3 +2011,20719,CBJ,1,1,3,0,0,5 +2011,20719,STL,2,0,0,0,0,2 +2011,20720,LAK,3,0,1,0,0,4 +2011,20720,PHX,1,2,0,0,0,3 +2011,20721,CGY,2,0,1,0,1,4 +2011,20721,VAN,1,1,1,0,0,3 +2011,20722,MIN,1,1,1,0,0,3 +2011,20722,SJS,1,2,1,0,0,4 +2011,20723,FLA,0,1,1,0,0,2 +2011,20723,NJD,2,2,1,0,0,5 +2011,20724,BUF,1,1,3,0,0,5 +2011,20724,NYI,0,2,1,0,0,3 +2011,20725,CHI,0,0,1,0,0,1 +2011,20725,PHI,0,1,3,0,0,4 +2011,20726,ATL,0,0,1,0,0,1 +2011,20726,TBL,0,5,2,0,0,7 +2011,20727,EDM,0,0,2,0,0,2 +2011,20727,NSH,1,0,1,0,1,3 +2011,20728,NYR,0,0,1,0,1,2 +2011,20728,WSH,0,1,0,0,0,1 +2011,20729,CAR,1,1,4,0,0,6 +2011,20729,TOR,1,0,3,0,0,4 +2011,20730,CGY,1,1,1,0,0,3 +2011,20730,NSH,0,1,0,0,0,1 +2011,20731,COL,0,2,2,0,0,4 +2011,20731,STL,0,2,1,0,0,3 +2011,20732,DAL,1,0,0,0,0,1 +2011,20732,VAN,3,2,2,0,0,7 +2011,20733,BOS,0,0,0,0,0,0 +2011,20733,LAK,1,0,1,0,0,2 +2011,20734,FLA,1,2,1,0,0,4 +2011,20734,NYR,0,1,2,0,0,3 +2011,20735,MTL,0,1,1,0,0,2 +2011,20735,PHI,2,2,1,0,0,5 +2011,20736,NYI,0,0,0,0,0,0 +2011,20736,PIT,0,0,1,0,0,1 +2011,20737,ANA,2,0,1,0,0,3 +2011,20737,CBJ,1,0,1,0,0,2 +2011,20738,BUF,2,0,0,1,0,3 +2011,20738,OTT,1,0,1,0,0,2 +2011,20739,TBL,2,0,0,0,0,2 +2011,20739,TOR,0,0,0,0,0,0 +2011,20740,CHI,2,0,0,0,0,2 +2011,20740,MIN,1,2,1,0,0,4 +2011,20741,EDM,0,1,3,0,0,4 +2011,20741,PHX,1,1,1,0,0,3 +2011,20742,BOS,1,1,0,0,0,2 +2011,20742,FLA,0,0,1,0,0,1 +2011,20743,CAR,1,2,1,0,0,4 +2011,20743,NYI,0,1,1,0,0,2 +2011,20744,ATL,0,1,0,0,0,1 +2011,20744,WSH,0,0,0,0,0,0 +2011,20745,DET,1,0,2,0,0,3 +2011,20745,NJD,1,0,0,0,0,1 +2011,20746,DAL,1,2,0,0,0,3 +2011,20746,EDM,0,1,0,0,0,1 +2011,20747,COL,1,0,1,0,0,2 +2011,20747,PHX,2,1,2,0,0,5 +2011,20748,CGY,1,2,1,0,0,4 +2011,20748,STL,0,0,1,0,0,1 +2011,20749,NSH,0,0,1,0,0,1 +2011,20749,VAN,0,0,2,0,0,2 +2011,20750,LAK,0,1,1,0,1,3 +2011,20750,SJS,0,2,0,0,0,2 +2011,20751,FLA,1,1,1,0,0,3 +2011,20751,TOR,2,1,0,0,1,4 +2011,20752,NJD,0,1,1,0,0,2 +2011,20752,OTT,0,1,0,0,0,1 +2011,20753,NYR,1,2,0,0,0,3 +2011,20753,PIT,0,3,0,0,1,4 +2011,20754,MTL,0,2,0,0,1,3 +2011,20754,WSH,2,0,0,0,0,2 +2011,20755,BOS,0,1,2,0,0,3 +2011,20755,CAR,0,1,1,0,0,2 +2011,20756,ATL,1,0,0,0,0,1 +2011,20756,NYI,0,3,1,0,0,4 +2011,20757,CHI,2,3,2,0,0,7 +2011,20757,CBJ,1,2,1,0,0,4 +2011,20758,PHI,0,0,0,0,0,0 +2011,20758,TBL,3,0,1,0,0,4 +2011,20759,COL,2,0,2,0,0,4 +2011,20759,STL,1,1,1,0,0,3 +2011,20760,CGY,0,0,2,0,1,3 +2011,20760,NSH,1,0,1,0,0,2 +2011,20761,LAK,0,0,0,0,0,0 +2011,20761,MIN,0,0,0,0,1,1 +2011,20762,DAL,0,1,0,0,0,1 +2011,20762,VAN,0,3,1,0,0,4 +2011,20763,PHX,2,1,0,0,0,3 +2011,20763,SJS,0,1,4,0,0,5 +2011,20764,DET,2,2,3,0,0,7 +2011,20764,OTT,3,1,1,0,0,5 +2011,20765,NYI,0,0,0,0,0,0 +2011,20765,PIT,2,0,1,0,0,3 +2011,20766,FLA,1,0,1,0,0,2 +2011,20766,MTL,0,1,2,0,0,3 +2011,20767,PHX,0,0,0,0,0,0 +2011,20767,VAN,1,2,3,0,0,6 +2011,20768,EDM,0,1,0,0,0,1 +2011,20768,LAK,0,2,1,0,0,3 +2011,20769,ANA,0,2,1,0,0,3 +2011,20769,SJS,3,1,0,0,0,4 +2011,20770,BOS,4,0,2,0,0,6 +2011,20770,DAL,0,1,2,0,0,3 +2011,20771,CAR,0,0,0,0,0,0 +2011,20771,TOR,0,2,1,0,0,3 +2011,20772,NJD,1,2,0,0,0,3 +2011,20772,NYR,0,1,1,0,0,2 +2011,20773,NSH,1,0,1,0,0,2 +2011,20773,PHI,1,0,2,0,0,3 +2011,20774,ATL,0,2,0,0,0,2 +2011,20774,CGY,1,1,2,0,0,4 +2011,20775,COL,0,1,2,0,0,3 +2011,20775,MIN,2,1,1,0,0,4 +2011,20776,FLA,2,0,1,1,0,4 +2011,20776,NJD,2,1,0,0,0,3 +2011,20777,BUF,2,0,0,0,0,2 +2011,20777,PIT,0,3,0,0,0,3 +2011,20778,CBJ,1,1,1,0,0,3 +2011,20778,DET,0,0,0,0,0,0 +2011,20779,TBL,1,1,0,0,0,2 +2011,20779,WSH,1,2,2,0,0,5 +2011,20780,EDM,1,2,0,0,0,3 +2011,20780,STL,2,2,1,0,0,5 +2011,20781,CHI,0,2,1,0,0,3 +2011,20781,VAN,1,1,2,0,0,4 +2011,20782,BOS,0,0,0,0,0,0 +2011,20782,SJS,1,0,1,0,0,2 +2011,20783,MTL,0,1,1,0,0,2 +2011,20783,NYR,0,0,0,0,0,0 +2011,20784,ANA,1,0,2,0,0,3 +2011,20784,COL,0,0,0,0,0,0 +2011,20785,BUF,2,3,1,0,0,6 +2011,20785,TOR,0,1,1,0,0,2 +2011,20786,NYI,3,0,2,0,0,5 +2011,20786,OTT,3,0,0,0,0,3 +2011,20787,DAL,0,1,0,0,0,1 +2011,20787,PHI,1,1,1,0,0,3 +2011,20788,ATL,1,0,2,0,0,3 +2011,20788,CAR,0,1,2,1,0,4 +2011,20789,CBJ,1,1,2,0,0,4 +2011,20789,EDM,2,0,1,0,0,3 +2011,20790,DET,0,0,0,0,0,0 +2011,20790,NSH,1,1,1,0,0,3 +2011,20791,MIN,0,0,0,0,0,0 +2011,20791,PHX,0,0,1,0,0,1 +2011,20792,CGY,1,1,1,0,0,3 +2011,20792,LAK,2,1,0,0,1,4 +2011,20793,PIT,0,0,0,0,0,0 +2011,20793,WSH,1,1,1,0,0,3 +2011,20794,MTL,0,1,0,0,0,1 +2011,20794,NJD,2,1,1,0,0,4 +2011,20795,STL,0,3,0,0,0,3 +2011,20795,TBL,1,2,0,1,0,4 +2011,20796,ATL,2,1,1,0,0,4 +2011,20796,TOR,0,3,2,0,0,5 +2011,20797,DET,0,1,2,0,0,3 +2011,20797,NYR,0,0,2,0,0,2 +2011,20798,EDM,1,1,2,0,0,4 +2011,20798,NSH,0,0,0,0,0,0 +2011,20799,CGY,1,0,2,0,0,3 +2011,20799,CHI,0,1,0,0,0,1 +2011,20800,COL,0,0,0,0,0,0 +2011,20800,PHX,1,1,1,0,0,3 +2011,20801,OTT,0,2,0,0,0,2 +2011,20801,VAN,2,1,1,0,0,4 +2011,20802,CAR,0,1,1,0,0,2 +2011,20802,NJD,0,1,1,1,0,3 +2011,20803,NYI,1,1,1,0,0,3 +2011,20803,TOR,2,0,3,0,0,5 +2011,20804,CBJ,1,2,1,0,0,4 +2011,20804,PIT,0,0,1,0,0,1 +2011,20805,SJS,0,0,2,0,0,2 +2011,20805,WSH,0,0,0,0,0,0 +2011,20806,BUF,1,1,5,0,0,7 +2011,20806,TBL,1,2,1,0,0,4 +2011,20807,FLA,1,0,0,0,0,1 +2011,20807,STL,0,1,1,0,0,2 +2011,20808,BOS,2,4,2,0,0,8 +2011,20808,MTL,0,4,2,0,0,6 +2011,20809,DET,0,1,0,0,0,1 +2011,20809,NSH,2,2,0,0,0,4 +2011,20810,CBJ,2,0,0,0,0,2 +2011,20810,SJS,0,2,1,0,0,3 +2011,20811,COL,1,0,1,0,0,2 +2011,20811,MIN,2,1,0,0,0,3 +2011,20812,DAL,1,1,0,0,0,2 +2011,20812,PHX,1,0,1,1,0,3 +2011,20813,CGY,1,2,2,0,0,5 +2011,20813,OTT,2,0,0,0,0,2 +2011,20814,CHI,1,1,2,0,0,4 +2011,20814,EDM,0,0,1,0,0,1 +2011,20815,ANA,1,2,1,0,0,4 +2011,20815,VAN,0,1,2,0,0,3 +2011,20816,NJD,0,0,1,1,0,2 +2011,20816,TOR,0,1,0,0,0,1 +2011,20817,CAR,0,0,1,0,0,1 +2011,20817,PHI,0,0,2,0,0,2 +2011,20818,LAK,1,0,0,0,0,1 +2011,20818,PIT,1,0,0,1,0,2 +2011,20819,MTL,1,2,0,0,0,3 +2011,20819,NYI,1,1,1,0,1,4 +2011,20820,BUF,0,1,1,1,0,3 +2011,20820,FLA,0,1,1,0,0,2 +2011,20821,BOS,1,0,0,0,0,1 +2011,20821,DET,2,3,1,0,0,6 +2011,20822,NJD,0,0,2,0,0,2 +2011,20822,SJS,0,0,1,0,0,1 +2011,20823,NYI,4,4,1,0,0,9 +2011,20823,PIT,0,2,1,0,0,3 +2011,20824,COL,1,0,0,0,0,1 +2011,20824,CBJ,0,2,1,0,0,3 +2011,20825,ATL,1,0,2,0,0,3 +2011,20825,NYR,0,1,1,0,0,2 +2011,20826,MIN,1,1,2,0,1,5 +2011,20826,STL,2,0,2,0,0,4 +2011,20827,CHI,3,0,0,0,0,3 +2011,20827,DAL,0,2,1,0,1,4 +2011,20828,ANA,1,2,1,1,0,5 +2011,20828,CGY,0,2,2,0,0,4 +2011,20829,LAK,0,1,3,0,0,4 +2011,20829,WSH,1,0,0,0,0,1 +2011,20830,EDM,1,1,1,0,0,3 +2011,20830,OTT,2,0,3,0,0,5 +2011,20831,MTL,0,2,1,0,0,3 +2011,20831,TOR,0,0,0,0,0,0 +2011,20832,CAR,0,1,2,0,0,3 +2011,20832,TBL,0,2,1,1,0,4 +2011,20833,COL,1,1,1,0,0,3 +2011,20833,NSH,1,1,3,0,0,5 +2011,20834,CHI,0,1,1,0,0,2 +2011,20834,PHX,1,1,0,0,1,3 +2011,20835,MIN,1,2,0,0,0,3 +2011,20835,STL,0,1,0,0,0,1 +2011,20836,CGY,0,2,0,0,0,2 +2011,20836,VAN,1,2,1,0,0,4 +2011,20837,BUF,3,1,2,0,0,6 +2011,20837,NYI,2,1,3,1,0,7 +2011,20838,NYR,2,3,0,0,0,5 +2011,20838,PIT,2,0,1,0,0,3 +2011,20839,CBJ,1,1,0,0,0,2 +2011,20839,DAL,1,0,0,0,0,1 +2011,20840,ATL,0,1,1,0,0,2 +2011,20840,CAR,1,1,1,0,0,3 +2011,20841,FLA,0,1,2,0,0,3 +2011,20841,SJS,1,0,1,0,0,2 +2011,20842,BOS,2,0,0,0,0,2 +2011,20842,DET,1,2,1,0,0,4 +2011,20843,LAK,0,1,0,0,0,1 +2011,20843,PHI,0,0,0,0,0,0 +2011,20844,ANA,1,2,1,0,0,4 +2011,20844,EDM,0,0,0,0,0,0 +2011,20845,STL,2,1,0,0,0,3 +2011,20845,VAN,1,0,1,0,0,2 +2011,20846,CGY,5,0,4,0,0,9 +2011,20846,COL,0,1,0,0,0,1 +2011,20847,PHX,0,2,1,0,0,3 +2011,20847,WSH,0,1,1,0,0,2 +2011,20848,BOS,1,1,1,0,0,3 +2011,20848,TOR,1,1,2,0,0,4 +2011,20849,BUF,0,1,1,0,1,3 +2011,20849,MTL,1,1,0,0,0,2 +2011,20850,NYI,0,1,2,0,1,4 +2011,20850,OTT,1,1,1,0,0,3 +2011,20851,PHI,1,2,0,0,1,4 +2011,20851,TBL,0,3,0,0,0,3 +2011,20852,NSH,1,0,0,0,0,1 +2011,20852,SJS,0,1,0,1,0,2 +2011,20853,MIN,1,0,0,0,0,1 +2011,20853,VAN,1,2,1,0,0,4 +2011,20854,DAL,0,1,0,0,0,1 +2011,20854,EDM,1,2,1,0,0,4 +2011,20855,BUF,0,0,1,0,0,1 +2011,20855,TOR,0,1,1,0,0,2 +2011,20856,CAR,0,0,2,0,0,2 +2011,20856,NJD,0,1,2,0,0,3 +2011,20857,CBJ,1,1,1,0,0,3 +2011,20857,LAK,1,1,1,0,1,4 +2011,20858,FLA,0,0,2,0,0,2 +2011,20858,PHI,2,1,1,0,0,4 +2011,20859,CHI,1,0,2,0,0,3 +2011,20859,MIN,0,0,1,0,0,1 +2011,20860,COL,0,2,0,0,0,2 +2011,20860,PIT,1,1,0,1,0,3 +2011,20861,CGY,1,1,2,0,0,4 +2011,20861,DAL,0,2,0,0,0,2 +2011,20862,ANA,3,2,1,0,0,6 +2011,20862,WSH,2,2,3,0,0,7 +2011,20863,BOS,3,3,0,0,0,6 +2011,20863,NYI,0,2,1,0,0,3 +2011,20864,LAK,1,0,2,0,0,3 +2011,20864,NYR,0,1,2,0,1,4 +2011,20865,DET,2,2,2,0,0,6 +2011,20865,TBL,0,2,0,0,0,2 +2011,20866,NSH,0,3,0,0,0,3 +2011,20866,VAN,0,1,0,0,0,1 +2011,20867,EDM,1,0,3,0,0,4 +2011,20867,MTL,0,1,0,0,0,1 +2011,20868,ATL,1,2,0,0,0,3 +2011,20868,PHX,2,2,0,0,0,4 +2011,20869,SJS,1,0,2,0,0,3 +2011,20869,WSH,1,0,1,0,0,2 +2011,20870,NJD,0,1,0,0,0,1 +2011,20870,NYR,0,0,0,0,0,0 +2011,20871,CAR,2,0,1,0,0,3 +2011,20871,PHI,0,1,1,0,0,2 +2011,20872,BUF,0,0,0,0,0,0 +2011,20872,STL,0,2,1,0,0,3 +2011,20873,BOS,0,1,3,0,0,4 +2011,20873,OTT,0,1,1,0,0,2 +2011,20874,DET,2,1,1,0,0,4 +2011,20874,FLA,1,1,1,0,0,3 +2011,20875,ANA,1,0,0,0,0,1 +2011,20875,MIN,2,2,1,0,0,5 +2011,20876,CHI,2,1,0,0,0,3 +2011,20876,CBJ,1,3,0,0,0,4 +2011,20877,ATL,1,1,1,0,0,3 +2011,20877,EDM,0,1,4,0,0,5 +2011,20878,OTT,0,0,0,0,1,1 +2011,20878,TOR,0,0,0,0,0,0 +2011,20879,LAK,0,0,0,0,0,0 +2011,20879,NYI,1,1,1,0,0,3 +2011,20880,CAR,0,0,1,0,0,1 +2011,20880,NJD,3,0,1,0,0,4 +2011,20881,FLA,0,1,1,0,1,3 +2011,20881,TBL,1,0,1,0,0,2 +2011,20882,ANA,3,0,0,0,0,3 +2011,20882,STL,4,4,1,0,0,9 +2011,20883,NSH,0,1,1,0,0,2 +2011,20883,PHX,0,2,1,0,0,3 +2011,20884,DAL,1,0,1,0,0,2 +2011,20884,VAN,1,1,3,0,0,5 +2011,20885,COL,0,0,0,0,0,0 +2011,20885,SJS,3,0,1,0,0,4 +2011,20886,BUF,0,1,0,0,0,1 +2011,20886,WSH,0,1,1,0,0,2 +2011,20887,NYR,1,0,1,0,0,2 +2011,20887,PHI,1,2,1,0,0,4 +2011,20888,CHI,1,0,1,0,1,3 +2011,20888,PIT,0,1,1,0,0,2 +2011,20889,DET,0,0,1,0,1,2 +2011,20889,MIN,0,0,1,0,0,1 +2011,20890,CGY,1,2,1,0,0,4 +2011,20890,MTL,0,0,0,0,0,0 +2011,20891,FLA,0,0,1,0,0,1 +2011,20891,NYI,2,2,1,0,0,5 +2011,20892,CHI,0,4,1,0,0,5 +2011,20892,STL,2,0,1,0,0,3 +2011,20893,PIT,0,0,0,0,0,0 +2011,20893,WSH,0,1,0,0,0,1 +2011,20894,NYI,0,1,0,0,0,1 +2011,20894,TOR,1,0,1,0,0,2 +2011,20895,PHI,1,0,1,0,0,2 +2011,20895,PHX,0,0,2,1,0,3 +2011,20896,CAR,1,0,2,0,0,3 +2011,20896,NYR,1,1,1,0,1,4 +2011,20897,DET,0,2,1,0,0,3 +2011,20897,SJS,1,2,1,0,0,4 +2011,20898,CBJ,0,0,4,0,0,4 +2011,20898,NSH,0,0,0,0,0,0 +2011,20899,EDM,0,1,0,0,0,1 +2011,20899,MIN,0,1,3,0,0,4 +2011,20900,DAL,0,0,0,0,0,0 +2011,20900,NJD,0,0,1,0,0,1 +2011,20901,MTL,2,1,0,0,0,3 +2011,20901,VAN,0,1,1,0,0,2 +2011,20902,ATL,0,1,0,0,0,1 +2011,20902,BUF,1,2,1,0,0,4 +2011,20903,FLA,0,1,0,0,0,1 +2011,20903,OTT,1,2,2,0,0,5 +2011,20904,PIT,1,0,1,0,0,2 +2011,20904,SJS,0,1,1,1,0,3 +2011,20905,PHX,0,1,2,0,0,3 +2011,20905,TBL,5,2,1,0,0,8 +2011,20906,BOS,1,0,2,0,0,3 +2011,20906,CGY,0,0,1,0,0,1 +2011,20907,COL,0,1,0,0,0,1 +2011,20907,EDM,1,2,2,0,0,5 +2011,20908,ANA,1,1,0,0,0,2 +2011,20908,LAK,2,0,1,0,0,3 +2011,20909,NYI,1,0,2,0,0,3 +2011,20909,PHI,0,2,1,1,0,4 +2011,20910,DAL,2,1,1,0,0,4 +2011,20910,DET,0,0,1,0,0,1 +2011,20911,MTL,1,2,1,0,0,4 +2011,20911,TOR,3,1,1,0,0,5 +2011,20912,CHI,1,0,2,0,0,3 +2011,20912,NSH,0,0,0,0,0,0 +2011,20913,STL,0,2,0,0,0,2 +2011,20913,VAN,2,0,1,0,0,3 +2011,20914,LAK,1,1,2,0,0,4 +2011,20914,MIN,0,2,0,0,0,2 +2011,20915,NYR,2,2,2,0,0,6 +2011,20915,WSH,0,0,0,0,0,0 +2011,20916,CAR,2,1,1,0,0,4 +2011,20916,PIT,0,1,0,0,0,1 +2011,20917,CBJ,2,2,1,0,0,5 +2011,20917,PHX,2,0,1,0,0,3 +2011,20918,BUF,1,1,2,0,0,4 +2011,20918,OTT,0,1,1,0,0,2 +2011,20919,ATL,0,1,0,0,0,1 +2011,20919,FLA,1,0,0,0,1,2 +2011,20920,NJD,0,0,1,0,0,1 +2011,20920,TBL,0,2,0,0,0,2 +2011,20921,CGY,1,2,0,0,0,3 +2011,20921,SJS,1,1,1,0,1,4 +2011,20922,EDM,0,0,0,0,0,0 +2011,20922,STL,2,1,2,0,0,5 +2011,20923,ANA,0,1,1,0,0,2 +2011,20923,MIN,1,0,1,1,0,3 +2011,20924,DAL,1,0,2,0,0,3 +2011,20924,NSH,0,2,0,0,0,2 +2011,20925,COL,0,0,3,0,0,3 +2011,20925,LAK,1,2,1,0,0,4 +2011,20926,BUF,2,0,0,0,0,2 +2011,20926,DET,0,1,1,0,1,3 +2011,20927,PIT,1,2,2,0,1,6 +2011,20927,TOR,1,1,3,0,0,5 +2011,20928,CAR,1,2,0,0,0,3 +2011,20928,MTL,2,1,1,0,0,4 +2011,20929,OTT,0,2,2,0,0,4 +2011,20929,PHI,1,0,0,0,0,1 +2011,20930,NYI,1,1,0,0,0,2 +2011,20930,WSH,0,1,2,0,0,3 +2011,20931,BOS,0,1,2,0,0,3 +2011,20931,VAN,1,0,0,0,0,1 +2011,20932,NYR,0,1,0,0,0,1 +2011,20932,TBL,1,0,1,0,0,2 +2011,20933,CBJ,0,1,1,0,0,2 +2011,20933,NSH,0,0,3,0,0,3 +2011,20934,ATL,0,0,2,1,0,3 +2011,20934,TOR,2,0,0,0,0,2 +2011,20935,FLA,0,0,1,0,0,1 +2011,20935,NJD,0,2,0,0,0,2 +2011,20936,CHI,2,1,0,0,1,4 +2011,20936,PHX,0,3,0,0,0,3 +2011,20937,CGY,0,0,1,0,0,1 +2011,20937,STL,0,0,0,0,0,0 +2011,20938,BOS,2,1,0,0,0,3 +2011,20938,EDM,1,0,1,0,0,2 +2011,20939,ANA,0,2,1,0,0,3 +2011,20939,COL,1,0,1,0,0,2 +2011,20940,CHI,0,3,1,0,0,4 +2011,20940,MIN,0,0,2,0,0,2 +2011,20941,DET,3,1,3,0,0,7 +2011,20941,LAK,1,0,3,0,0,4 +2011,20942,BUF,1,1,1,0,0,3 +2011,20942,NYR,0,1,1,0,0,2 +2011,20943,NYI,0,1,0,0,0,1 +2011,20943,WSH,0,0,1,1,0,2 +2011,20944,CAR,2,0,0,0,0,2 +2011,20944,FLA,0,1,0,0,0,1 +2011,20945,ATL,0,0,1,0,0,1 +2011,20945,MTL,1,1,1,0,0,3 +2011,20946,BOS,0,0,1,0,0,1 +2011,20946,OTT,0,0,0,0,0,0 +2011,20947,CGY,1,0,5,0,0,6 +2011,20947,STL,0,0,0,0,0,0 +2011,20948,EDM,1,0,0,0,1,2 +2011,20948,NSH,1,0,0,0,0,1 +2011,20949,DAL,1,0,2,0,0,3 +2011,20949,PHX,0,0,2,0,0,2 +2011,20950,CBJ,0,1,0,0,0,1 +2011,20950,VAN,1,0,0,0,1,2 +2011,20951,COL,0,0,1,0,0,1 +2011,20951,SJS,0,0,1,0,1,2 +2011,20952,PIT,1,1,0,0,0,2 +2011,20952,TOR,1,0,1,1,0,3 +2011,20953,NJD,0,1,1,0,0,2 +2011,20953,TBL,0,1,0,0,0,1 +2011,20954,MIN,0,0,1,0,0,1 +2011,20954,NYI,2,2,0,0,0,4 +2011,20955,CGY,1,0,3,0,0,4 +2011,20955,CHI,2,2,2,0,0,6 +2011,20956,ANA,0,0,1,1,0,2 +2011,20956,DET,0,1,0,0,0,1 +2011,20957,BOS,0,1,1,0,0,2 +2011,20957,TBL,0,1,0,0,0,1 +2011,20958,MIN,0,2,1,0,0,3 +2011,20958,NYR,1,0,0,0,0,1 +2011,20959,PHI,1,1,0,0,0,2 +2011,20959,TOR,1,1,1,0,0,3 +2011,20960,STL,1,1,0,0,0,2 +2011,20960,WSH,0,2,1,0,0,3 +2011,20961,BUF,0,2,0,0,0,2 +2011,20961,CAR,0,2,0,1,0,3 +2011,20962,ATL,0,0,1,0,0,1 +2011,20962,OTT,1,0,2,0,0,3 +2011,20963,FLA,0,0,0,0,0,0 +2011,20963,MTL,2,1,1,0,0,4 +2011,20964,CBJ,0,1,1,0,0,2 +2011,20964,EDM,2,1,1,0,0,4 +2011,20965,NSH,0,0,3,0,0,3 +2011,20965,VAN,0,0,0,0,0,0 +2011,20966,LAK,0,0,1,0,0,1 +2011,20966,PHX,0,0,0,0,0,0 +2011,20967,DET,1,0,0,0,0,1 +2011,20967,SJS,1,1,1,0,0,3 +2011,20968,NJD,0,1,0,1,0,2 +2011,20968,PIT,0,1,0,0,0,1 +2011,20969,NYR,1,2,1,0,0,4 +2011,20969,OTT,0,0,1,0,0,1 +2011,20970,CAR,1,0,1,0,0,2 +2011,20970,CHI,2,0,3,0,0,5 +2011,20971,CGY,2,2,0,0,0,4 +2011,20971,CBJ,1,1,1,0,0,3 +2011,20972,ANA,1,0,2,1,0,4 +2011,20972,DAL,0,3,0,0,0,3 +2011,20973,NYI,2,1,2,0,0,5 +2011,20973,STL,0,0,2,0,0,2 +2011,20974,BUF,0,3,2,0,0,5 +2011,20974,PHI,2,1,0,0,0,3 +2011,20975,LAK,1,0,0,0,0,1 +2011,20975,VAN,1,0,2,0,0,3 +2011,20976,BOS,0,1,1,0,0,2 +2011,20976,PIT,0,2,0,1,0,3 +2011,20977,CHI,3,2,0,0,0,5 +2011,20977,TOR,0,1,2,0,0,3 +2011,20978,ATL,0,1,2,1,0,4 +2011,20978,FLA,1,0,2,0,0,3 +2011,20979,MTL,2,1,1,0,0,4 +2011,20979,TBL,0,1,1,0,0,2 +2011,20980,DET,2,2,0,0,0,4 +2011,20980,PHX,1,0,3,0,1,5 +2011,20981,COL,0,1,0,0,0,1 +2011,20981,EDM,2,2,1,0,0,5 +2011,20982,DAL,1,0,2,0,0,3 +2011,20982,SJS,1,1,0,0,0,2 +2011,20983,NJD,0,1,1,0,1,3 +2011,20983,NYI,0,1,1,0,0,2 +2011,20984,NYR,2,2,3,0,0,7 +2011,20984,PHI,0,0,0,0,0,0 +2011,20985,FLA,1,0,1,0,0,2 +2011,20985,WSH,1,1,0,1,0,3 +2011,20986,BUF,1,1,0,1,0,3 +2011,20986,MIN,0,2,0,0,0,2 +2011,20987,CGY,1,1,1,0,0,3 +2011,20987,NSH,2,0,0,0,0,2 +2011,20988,ANA,0,0,0,0,0,0 +2011,20988,VAN,1,1,1,0,0,3 +2011,20989,TBL,1,0,0,0,0,1 +2011,20989,WSH,0,0,1,0,1,2 +2011,20990,CBJ,3,1,0,0,0,4 +2011,20990,STL,3,1,0,0,1,5 +2011,20991,DAL,0,1,2,1,0,4 +2011,20991,LAK,1,1,1,0,0,3 +2011,20992,NJD,0,0,1,0,0,1 +2011,20992,OTT,0,1,1,0,0,2 +2011,20993,NYI,0,2,1,1,0,4 +2011,20993,TOR,0,2,1,0,0,3 +2011,20994,EDM,0,1,0,0,0,1 +2011,20994,PHI,2,1,1,0,0,4 +2011,20995,BUF,1,0,0,0,0,1 +2011,20995,PIT,0,2,1,0,0,3 +2011,20996,BOS,0,0,1,0,0,1 +2011,20996,MTL,2,2,0,0,0,4 +2011,20997,CHI,0,2,0,0,0,2 +2011,20997,FLA,3,0,0,0,0,3 +2011,20998,COL,0,2,0,0,0,2 +2011,20998,MIN,1,1,3,0,0,5 +2011,20999,PHX,2,1,0,0,0,3 +2011,20999,VAN,1,2,0,1,0,4 +2011,21000,NSH,1,1,0,0,0,2 +2011,21000,SJS,1,1,0,1,0,3 +2011,21001,EDM,0,0,0,0,0,0 +2011,21001,WSH,0,2,3,0,0,5 +2011,21002,ATL,2,0,0,1,0,3 +2011,21002,CAR,0,1,1,0,0,2 +2011,21003,DET,1,0,0,0,0,1 +2011,21003,LAK,0,2,0,0,0,2 +2011,21004,CBJ,0,3,0,0,0,3 +2011,21004,STL,0,1,2,1,0,4 +2011,21005,CHI,1,1,1,0,0,3 +2011,21005,TBL,2,1,0,0,1,4 +2011,21006,CGY,3,0,0,0,1,4 +2011,21006,DAL,1,1,1,0,0,3 +2011,21007,ANA,3,0,2,0,0,5 +2011,21007,NYR,1,0,1,0,0,2 +2011,21008,BOS,1,1,1,0,0,3 +2011,21008,BUF,0,2,1,1,0,4 +2011,21009,PHI,1,2,0,0,0,3 +2011,21009,TOR,0,1,1,0,0,2 +2011,21010,FLA,0,0,1,0,0,1 +2011,21010,OTT,0,0,2,0,0,2 +2011,21011,MTL,1,0,0,0,0,1 +2011,21011,STL,1,1,2,0,0,4 +2011,21012,MIN,0,0,0,0,0,0 +2011,21012,NSH,3,0,1,0,0,4 +2011,21013,CGY,0,0,0,0,0,0 +2011,21013,PHX,0,1,2,0,0,3 +2011,21014,SJS,0,2,2,0,0,4 +2011,21014,VAN,2,1,1,0,1,5 +2011,21015,BOS,1,1,0,0,0,2 +2011,21015,NYI,0,1,3,0,0,4 +2011,21016,CAR,0,1,0,0,0,1 +2011,21016,WSH,0,0,2,0,0,2 +2011,21017,DET,0,0,1,1,0,2 +2011,21017,EDM,1,0,0,0,0,1 +2011,21018,CBJ,1,0,1,0,0,2 +2011,21018,LAK,2,1,1,0,0,4 +2011,21019,ATL,0,1,1,0,0,2 +2011,21019,NJD,0,1,1,1,0,3 +2011,21020,OTT,1,1,0,0,0,2 +2011,21020,TBL,0,0,1,0,0,1 +2011,21021,DAL,1,3,0,0,0,4 +2011,21021,MIN,0,0,0,0,0,0 +2011,21022,ANA,0,4,2,0,0,6 +2011,21022,COL,1,1,0,0,0,2 +2011,21023,MTL,1,2,0,0,0,3 +2011,21023,PIT,0,0,0,0,0,0 +2011,21024,BUF,1,2,0,0,0,3 +2011,21024,TOR,1,1,2,0,0,4 +2011,21025,NJD,1,0,1,1,0,3 +2011,21025,NYI,0,2,0,0,0,2 +2011,21026,ATL,0,0,4,1,0,5 +2011,21026,PHI,1,2,1,0,0,4 +2011,21027,CAR,0,0,2,0,0,2 +2011,21027,CBJ,1,1,1,0,0,3 +2011,21028,FLA,0,1,2,1,0,4 +2011,21028,TBL,0,1,2,0,0,3 +2011,21029,DET,2,1,2,0,0,5 +2011,21029,STL,1,0,2,0,0,3 +2011,21030,COL,0,2,0,0,0,2 +2011,21030,NSH,2,1,1,0,0,4 +2011,21031,CGY,2,1,0,0,0,3 +2011,21031,VAN,1,3,0,0,0,4 +2011,21032,NYR,1,1,0,0,1,3 +2011,21032,SJS,1,1,0,0,0,2 +2011,21033,EDM,0,0,1,0,0,1 +2011,21033,PIT,1,2,2,0,0,5 +2011,21034,CHI,1,1,1,0,0,3 +2011,21034,WSH,2,0,1,1,0,4 +2011,21035,DAL,0,1,1,0,0,2 +2011,21035,LAK,1,1,1,0,0,3 +2011,21036,BUF,2,3,1,0,0,6 +2011,21036,OTT,2,1,1,0,0,4 +2011,21037,ANA,1,1,0,0,0,2 +2011,21037,PHX,2,2,1,0,0,5 +2011,21038,TBL,1,3,2,0,0,6 +2011,21038,TOR,0,2,0,0,0,2 +2011,21039,CHI,1,5,0,0,0,6 +2011,21039,SJS,2,1,0,0,0,3 +2011,21040,MIN,0,1,1,0,0,2 +2011,21040,VAN,3,0,1,0,0,4 +2011,21041,BUF,0,0,0,0,0,0 +2011,21041,CAR,1,0,0,0,0,1 +2011,21042,ATL,2,0,0,0,0,2 +2011,21042,NJD,1,1,2,0,0,4 +2011,21043,NYI,0,2,1,0,0,3 +2011,21043,NYR,2,3,1,0,0,6 +2011,21044,BOS,0,1,1,0,1,3 +2011,21044,CBJ,1,0,1,0,0,2 +2011,21045,MTL,1,1,0,0,0,2 +2011,21045,WSH,2,0,2,0,0,4 +2011,21046,OTT,0,1,0,0,0,1 +2011,21046,PIT,1,3,1,0,0,5 +2011,21047,FLA,2,0,0,0,0,2 +2011,21047,PHI,1,2,0,0,0,3 +2011,21048,LAK,1,2,1,0,0,4 +2011,21048,NSH,1,0,1,0,0,2 +2011,21049,DAL,0,2,1,0,0,3 +2011,21049,SJS,0,2,4,0,0,6 +2011,21050,CGY,1,0,2,0,0,3 +2011,21050,PHX,1,1,2,0,0,4 +2011,21051,CAR,1,0,0,0,0,1 +2011,21051,TOR,1,2,0,0,0,3 +2011,21052,DET,2,0,1,0,0,3 +2011,21052,WSH,1,1,0,0,0,2 +2011,21053,COL,2,0,0,0,0,2 +2011,21053,VAN,0,1,3,0,0,4 +2011,21054,ANA,0,1,1,0,0,2 +2011,21054,STL,0,1,0,0,0,1 +2011,21055,ATL,1,2,0,0,1,4 +2011,21055,PHI,1,1,1,0,0,3 +2011,21056,CBJ,0,0,0,0,0,0 +2011,21056,DET,2,0,0,0,0,2 +2011,21057,MTL,0,1,1,0,1,3 +2011,21057,TBL,0,1,1,0,0,2 +2011,21058,NJD,0,0,1,0,0,1 +2011,21058,OTT,1,0,2,0,0,3 +2011,21059,FLA,0,3,1,0,0,4 +2011,21059,TOR,0,0,0,0,0,0 +2011,21060,BOS,1,1,1,0,0,3 +2011,21060,NSH,1,1,1,1,0,4 +2011,21061,CHI,0,0,0,0,0,0 +2011,21061,DAL,1,2,2,0,0,5 +2011,21062,CGY,3,0,2,0,0,5 +2011,21062,COL,1,0,1,0,0,2 +2011,21063,EDM,0,0,1,0,0,1 +2011,21063,PHX,1,1,1,0,0,3 +2011,21064,LAK,0,0,0,0,0,0 +2011,21064,STL,1,1,2,0,0,4 +2011,21065,MIN,1,1,0,0,0,2 +2011,21065,SJS,1,1,1,0,0,3 +2011,21066,NJD,0,0,0,0,0,0 +2011,21066,WSH,1,1,1,0,0,3 +2011,21067,MTL,1,1,1,0,0,3 +2011,21067,NYR,5,0,1,0,0,6 +2011,21068,CAR,1,0,1,1,0,3 +2011,21068,NYI,1,1,0,0,0,2 +2011,21069,PHX,0,0,3,0,0,3 +2011,21069,VAN,0,0,1,0,0,1 +2011,21070,CBJ,1,1,2,1,0,5 +2011,21070,MIN,1,1,2,0,0,4 +2011,21071,ATL,1,1,0,0,0,2 +2011,21071,BUF,3,2,3,0,0,8 +2011,21072,BOS,1,0,1,0,0,2 +2011,21072,TOR,2,3,0,0,0,5 +2011,21073,OTT,0,0,2,1,0,3 +2011,21073,TBL,1,1,0,0,0,2 +2011,21074,FLA,0,2,1,0,0,3 +2011,21074,NYI,0,1,2,0,1,4 +2011,21075,DET,1,0,0,0,0,1 +2011,21075,NSH,0,1,2,0,0,3 +2011,21076,DAL,0,1,1,0,0,2 +2011,21076,PHI,1,1,0,0,1,3 +2011,21077,COL,1,1,0,0,1,3 +2011,21077,EDM,0,1,1,0,0,2 +2011,21078,ANA,1,0,0,1,0,2 +2011,21078,LAK,0,0,1,0,0,1 +2011,21079,SJS,3,2,0,0,0,5 +2011,21079,STL,0,1,2,0,0,3 +2011,21080,NYR,1,0,4,0,0,5 +2011,21080,PIT,1,0,1,0,0,2 +2011,21081,BUF,0,3,0,0,0,3 +2011,21081,NSH,1,0,2,1,0,4 +2011,21082,CBJ,0,0,0,0,0,0 +2011,21082,NJD,1,1,1,0,0,3 +2011,21083,MIN,0,0,1,0,0,1 +2011,21083,MTL,2,4,2,0,0,8 +2011,21084,CHI,1,0,1,0,0,2 +2011,21084,PHX,0,1,0,0,0,1 +2011,21085,ANA,3,0,1,1,0,5 +2011,21085,CGY,1,2,1,0,0,4 +2011,21086,DET,0,1,3,0,0,4 +2011,21086,PIT,2,2,0,0,1,5 +2011,21087,CGY,0,0,1,0,0,1 +2011,21087,LAK,0,0,1,0,1,2 +2011,21088,BOS,1,1,2,0,0,4 +2011,21088,NJD,1,0,0,0,0,1 +2011,21089,FLA,0,0,0,0,0,0 +2011,21089,NYR,0,0,1,0,0,1 +2011,21090,PHI,0,2,2,0,0,4 +2011,21090,WSH,2,1,1,0,1,5 +2011,21091,CAR,2,1,1,0,0,4 +2011,21091,OTT,2,0,1,0,0,3 +2011,21092,BUF,0,1,1,0,0,2 +2011,21092,MTL,0,0,0,0,0,0 +2011,21093,NYI,1,2,2,0,0,5 +2011,21093,TBL,1,0,1,0,0,2 +2011,21094,EDM,0,1,0,0,0,1 +2011,21094,NSH,2,1,0,0,0,3 +2011,21095,MIN,0,0,0,0,0,0 +2011,21095,TOR,1,1,1,0,0,3 +2011,21096,COL,1,0,3,0,1,5 +2011,21096,CBJ,1,1,2,0,0,4 +2011,21097,PHX,0,2,0,0,0,2 +2011,21097,STL,1,0,0,0,0,1 +2011,21098,DET,0,0,1,0,0,1 +2011,21098,VAN,0,1,1,0,0,2 +2011,21099,CHI,0,1,3,0,0,4 +2011,21099,FLA,0,0,0,0,0,0 +2011,21100,ANA,0,1,2,1,0,4 +2011,21100,DAL,0,1,2,0,0,3 +2011,21101,CGY,2,1,0,0,0,3 +2011,21101,SJS,4,1,1,0,0,6 +2011,21102,BOS,3,0,4,0,0,7 +2011,21102,MTL,0,0,0,0,0,0 +2011,21103,ATL,1,1,0,0,0,2 +2011,21103,NYI,0,1,0,0,0,1 +2011,21104,NYR,0,0,1,0,0,1 +2011,21104,OTT,0,1,0,0,1,2 +2011,21105,PHI,1,0,0,0,0,1 +2011,21105,PIT,0,1,0,0,1,2 +2011,21106,EDM,0,0,0,0,0,0 +2011,21106,STL,0,1,3,0,0,4 +2011,21107,ANA,1,0,3,0,0,4 +2011,21107,NSH,3,1,1,0,0,5 +2011,21108,COL,1,1,1,0,0,3 +2011,21108,TOR,2,1,1,0,0,4 +2011,21109,CBJ,0,0,0,0,0,0 +2011,21109,PHX,0,2,1,0,0,3 +2011,21110,LAK,0,2,1,0,1,4 +2011,21110,SJS,0,1,2,0,0,3 +2011,21111,NJD,0,0,0,0,0,0 +2011,21111,PIT,0,0,0,0,1,1 +2011,21112,BUF,1,1,2,0,0,4 +2011,21112,FLA,1,0,1,0,0,2 +2011,21113,OTT,0,1,1,0,0,2 +2011,21113,WSH,0,0,0,0,0,0 +2011,21114,ATL,0,0,1,0,0,1 +2011,21114,VAN,0,2,1,0,0,3 +2011,21115,CAR,1,1,2,0,0,4 +2011,21115,TBL,1,0,2,0,0,3 +2011,21116,BOS,0,0,0,0,0,0 +2011,21116,NYR,1,0,0,0,0,1 +2011,21117,COL,0,1,0,0,0,1 +2011,21117,LAK,2,0,2,0,0,4 +2011,21118,BUF,2,0,0,0,0,2 +2011,21118,NJD,0,0,0,0,0,0 +2011,21119,MTL,0,0,0,0,0,0 +2011,21119,WSH,1,0,1,0,0,2 +2011,21120,NYI,1,0,0,0,0,1 +2011,21120,PHI,3,0,1,0,0,4 +2011,21121,CAR,2,0,0,0,0,2 +2011,21121,TBL,2,1,1,0,0,4 +2011,21122,DET,1,1,2,0,0,4 +2011,21122,TOR,1,0,1,0,0,2 +2011,21123,DAL,1,1,0,0,0,2 +2011,21123,NSH,1,2,1,0,0,4 +2011,21124,MIN,2,0,1,0,0,3 +2011,21124,STL,2,2,2,0,0,6 +2011,21125,ANA,0,0,2,0,0,2 +2011,21125,CHI,1,0,0,0,0,1 +2011,21126,PHX,1,0,0,0,0,1 +2011,21126,SJS,1,2,1,0,0,4 +2011,21127,CGY,0,1,3,0,1,5 +2011,21127,EDM,1,2,1,0,0,4 +2011,21128,FLA,0,1,0,0,0,1 +2011,21128,PIT,0,1,0,0,1,2 +2011,21129,ATL,1,2,1,0,1,5 +2011,21129,OTT,1,1,2,0,0,4 +2011,21130,CBJ,0,0,1,0,0,1 +2011,21130,VAN,1,0,3,0,0,4 +2011,21131,BOS,0,1,1,0,0,2 +2011,21131,PHI,1,0,0,0,0,1 +2011,21132,CHI,2,0,0,1,0,3 +2011,21132,DET,1,0,1,0,0,2 +2011,21133,ANA,0,3,2,0,0,5 +2011,21133,COL,2,1,1,0,0,4 +2011,21134,BOS,0,2,1,0,0,3 +2011,21134,CHI,0,0,0,0,0,0 +2011,21135,BUF,1,2,0,0,0,3 +2011,21135,TOR,2,2,0,0,0,4 +2011,21136,PHI,1,2,2,0,0,5 +2011,21136,PIT,1,1,0,0,0,2 +2011,21137,CAR,0,1,1,0,1,3 +2011,21137,WSH,0,2,0,0,0,2 +2011,21138,CBJ,1,1,0,0,1,3 +2011,21138,FLA,0,1,1,0,0,2 +2011,21139,ATL,0,0,1,0,0,1 +2011,21139,MTL,2,0,1,0,0,3 +2011,21140,OTT,0,0,2,0,0,2 +2011,21140,TBL,0,2,3,0,0,5 +2011,21141,MIN,1,0,1,0,1,3 +2011,21141,STL,1,0,1,0,0,2 +2011,21142,NSH,0,1,0,0,0,1 +2011,21142,VAN,0,0,3,0,0,3 +2011,21143,EDM,0,0,0,0,0,0 +2011,21143,LAK,0,2,0,0,0,2 +2011,21144,DAL,0,1,0,0,0,1 +2011,21144,PHX,0,1,0,0,1,2 +2011,21145,BUF,0,1,0,0,0,1 +2011,21145,NYR,0,0,0,0,0,0 +2011,21146,NJD,1,1,1,0,0,3 +2011,21146,NYI,0,0,2,0,0,2 +2011,21147,CAR,2,2,2,0,0,6 +2011,21147,MTL,0,1,1,0,0,2 +2011,21148,DET,2,1,0,0,0,3 +2011,21148,STL,3,5,2,0,0,10 +2011,21149,ANA,2,1,1,0,0,4 +2011,21149,CGY,1,0,1,0,0,2 +2011,21150,BOS,0,3,0,0,0,3 +2011,21150,TOR,1,1,1,0,1,4 +2011,21151,NYI,0,4,2,0,0,6 +2011,21151,NYR,1,0,1,0,0,2 +2011,21152,ATL,0,0,1,0,0,1 +2011,21152,PHI,0,0,0,0,0,0 +2011,21153,CBJ,0,2,1,0,0,3 +2011,21153,WSH,1,2,0,1,0,4 +2011,21154,PIT,0,0,1,0,0,1 +2011,21154,TBL,2,0,0,0,0,2 +2011,21155,FLA,0,1,0,0,0,1 +2011,21155,OTT,1,3,0,0,0,4 +2011,21156,EDM,0,1,1,0,0,2 +2011,21156,MIN,2,2,0,0,0,4 +2011,21157,COL,0,0,2,0,0,2 +2011,21157,NSH,1,1,1,0,0,3 +2011,21158,LAK,1,0,0,0,0,1 +2011,21158,VAN,0,2,1,0,0,3 +2011,21159,DAL,0,0,0,0,0,0 +2011,21159,SJS,1,2,3,0,0,6 +2011,21160,NJD,0,2,2,0,0,4 +2011,21160,PHI,1,0,1,0,0,2 +2011,21161,CHI,2,0,1,0,1,4 +2011,21161,CBJ,1,1,1,0,0,3 +2011,21162,CGY,0,1,2,0,0,3 +2011,21162,STL,1,1,0,0,0,2 +2011,21163,COL,1,1,1,0,1,4 +2011,21163,PHX,0,1,2,0,0,3 +2011,21164,ATL,1,1,0,0,0,2 +2011,21164,BOS,1,1,1,0,0,3 +2011,21165,MIN,1,0,0,0,0,1 +2011,21165,TBL,0,2,1,0,0,3 +2011,21166,DET,0,2,1,1,0,4 +2011,21166,NSH,1,2,0,0,0,3 +2011,21167,DAL,0,1,0,0,0,1 +2011,21167,LAK,0,1,2,0,0,3 +2011,21168,OTT,0,1,1,0,0,2 +2011,21168,TOR,1,1,2,0,0,4 +2011,21169,MTL,1,0,2,0,0,3 +2011,21169,NJD,0,0,1,0,0,1 +2011,21170,CAR,1,0,3,0,0,4 +2011,21170,NYI,1,1,0,0,0,2 +2011,21171,BUF,2,0,2,0,0,4 +2011,21171,WSH,2,0,2,1,0,5 +2011,21172,FLA,2,0,0,0,0,2 +2011,21172,PIT,1,1,2,0,0,4 +2011,21173,EDM,1,2,1,0,0,4 +2011,21173,VAN,0,1,0,0,0,1 +2011,21174,ANA,2,0,0,0,0,2 +2011,21174,SJS,0,2,2,0,0,4 +2011,21175,NYR,1,1,0,0,1,3 +2011,21175,PHI,1,0,1,0,0,2 +2011,21176,BUF,1,0,0,1,0,2 +2011,21176,CAR,0,1,0,0,0,1 +2011,21177,DET,2,1,1,0,0,4 +2011,21177,MIN,0,1,1,0,0,2 +2011,21178,CBJ,1,0,0,0,0,1 +2011,21178,STL,2,2,2,0,0,6 +2011,21179,CHI,0,0,0,0,0,0 +2011,21179,TBL,1,0,1,0,0,2 +2011,21180,CGY,0,1,1,0,0,2 +2011,21180,COL,0,0,1,0,0,1 +2011,21181,ANA,2,1,0,0,0,3 +2011,21181,DAL,2,2,0,0,0,4 +2011,21182,BOS,2,1,0,0,0,3 +2011,21182,NYR,0,2,3,0,0,5 +2011,21183,LAK,0,1,0,0,0,1 +2011,21183,SJS,2,4,0,0,0,6 +2011,21184,BUF,1,2,1,0,0,4 +2011,21184,TBL,1,0,1,0,0,2 +2011,21185,TOR,1,1,0,0,0,2 +2011,21185,WSH,1,1,0,0,1,3 +2011,21186,NJD,0,1,1,0,0,2 +2011,21186,PIT,1,1,2,0,0,4 +2011,21187,CHI,0,1,0,0,0,1 +2011,21187,MTL,0,1,0,1,0,2 +2011,21188,OTT,1,2,2,0,0,5 +2011,21188,PHI,1,1,0,0,0,2 +2011,21189,COL,1,0,0,0,0,1 +2011,21189,STL,1,1,1,0,0,3 +2011,21190,ATL,0,0,3,0,0,3 +2011,21190,NSH,3,1,2,0,0,6 +2011,21191,CBJ,0,0,0,0,0,0 +2011,21191,DAL,1,0,2,0,0,3 +2011,21192,EDM,1,0,1,0,0,2 +2011,21192,VAN,0,0,0,0,0,0 +2011,21193,BOS,1,2,0,0,0,3 +2011,21193,NYI,0,2,0,0,0,2 +2011,21194,NJD,1,2,1,0,0,4 +2011,21194,TOR,0,0,2,0,0,2 +2011,21195,FLA,0,0,2,0,0,2 +2011,21195,WSH,2,1,2,0,0,5 +2011,21196,CAR,0,2,1,0,0,3 +2011,21196,DET,0,0,0,0,0,0 +2011,21197,CHI,0,3,0,1,0,4 +2011,21197,STL,1,1,1,0,0,3 +2011,21198,CGY,0,3,3,0,0,6 +2011,21198,EDM,0,0,1,0,0,1 +2011,21199,ANA,2,4,0,0,0,6 +2011,21199,SJS,0,1,1,0,0,2 +2011,21200,LAK,1,1,0,0,1,3 +2011,21200,PHX,2,0,0,0,0,2 +2011,21201,ATL,0,2,1,0,0,3 +2011,21201,NYR,0,0,0,0,0,0 +2011,21202,MTL,0,1,1,0,0,2 +2011,21202,OTT,0,0,2,1,0,3 +2011,21203,COL,0,1,1,0,0,2 +2011,21203,DAL,2,0,2,0,0,4 +2011,21204,MIN,0,0,0,0,0,0 +2011,21204,VAN,2,2,1,0,0,5 +2011,21205,NYI,0,1,2,0,0,3 +2011,21205,PIT,1,1,1,0,1,4 +2011,21206,CHI,3,1,0,0,0,4 +2011,21206,DET,0,1,1,0,0,2 +2011,21207,BUF,0,2,1,1,0,4 +2011,21207,PHI,1,2,0,0,0,3 +2011,21208,ATL,0,0,1,0,0,1 +2011,21208,CAR,2,1,3,0,0,6 +2011,21209,FLA,1,0,1,0,0,2 +2011,21209,TBL,2,2,0,0,0,4 +2011,21210,CBJ,0,1,0,0,0,1 +2011,21210,NSH,1,2,1,0,0,4 +2011,21211,COL,0,1,1,0,0,2 +2011,21211,DAL,1,1,1,0,0,3 +2011,21212,EDM,0,1,0,0,0,1 +2011,21212,MIN,1,1,1,0,0,3 +2011,21213,PHX,1,2,1,0,0,4 +2011,21213,SJS,0,1,2,0,0,3 +2011,21214,ANA,0,1,1,0,0,2 +2011,21214,LAK,1,0,0,0,0,1 +2011,21215,NJD,2,0,0,0,0,2 +2011,21215,NYR,1,3,1,0,0,5 +2011,21216,BOS,1,1,1,0,0,3 +2011,21216,OTT,1,0,0,0,0,1 +2011,21217,MTL,2,1,1,0,0,4 +2011,21217,TOR,1,0,0,0,0,1 +2011,21218,NYI,3,1,0,0,0,4 +2011,21218,PHI,3,2,2,0,0,7 +2011,21219,CAR,0,1,1,0,0,2 +2011,21219,TBL,3,1,2,0,0,6 +2011,21220,FLA,0,0,1,0,0,1 +2011,21220,WSH,0,0,0,0,0,0 +2011,21221,BUF,0,3,2,0,0,5 +2011,21221,CBJ,0,2,2,0,0,4 +2011,21222,NSH,0,0,0,0,0,0 +2011,21222,STL,0,1,1,0,0,2 +2011,21223,CGY,0,2,0,0,0,2 +2011,21223,VAN,0,0,2,1,0,3 +2011,21224,ANA,2,1,0,0,0,3 +2011,21224,LAK,0,1,0,0,0,1 +2011,21225,PHX,0,0,1,0,0,1 +2011,21225,SJS,0,2,1,0,0,3 +2011,21226,BOS,1,0,1,0,0,2 +2011,21226,NJD,1,0,2,0,0,3 +2011,21227,ATL,1,0,1,0,0,2 +2011,21227,PIT,0,4,1,0,0,5 +2011,21228,COL,2,0,1,1,0,4 +2011,21228,EDM,1,2,0,0,0,3 +2011,21229,CHI,0,1,2,0,0,3 +2011,21229,DET,0,2,2,0,0,4 +2011,21230,DAL,2,1,0,0,0,3 +2011,21230,MIN,1,2,2,0,0,5 +2012,20001,BOS,1,0,0,0,0,1 +2012,20001,PHI,2,0,0,0,0,2 +2012,20002,MTL,0,0,0,0,0,0 +2012,20002,TOR,0,1,1,0,0,2 +2012,20003,PIT,2,1,0,0,1,4 +2012,20003,VAN,1,1,1,0,0,3 +2012,20004,CAR,1,0,0,0,0,1 +2012,20004,TBL,0,2,3,0,0,5 +2012,20005,DET,1,3,1,0,0,5 +2012,20005,OTT,0,0,3,0,0,3 +2012,20006,CBJ,0,1,1,0,0,2 +2012,20006,NSH,0,2,1,0,0,3 +2012,20007,ANA,0,1,0,0,0,1 +2012,20007,BUF,2,2,0,0,0,4 +2012,20008,CHI,0,0,1,0,0,1 +2012,20008,DAL,0,2,0,0,0,2 +2012,20009,LAK,1,0,1,1,0,3 +2012,20009,NYR,1,0,1,0,0,2 +2012,20010,BOS,0,2,2,0,0,4 +2012,20010,TBL,0,1,0,0,0,1 +2012,20011,OTT,0,0,5,0,0,5 +2012,20011,TOR,2,2,2,0,0,6 +2012,20012,NJD,0,0,0,0,0,0 +2012,20012,PHI,1,0,2,0,0,3 +2012,20013,CAR,0,1,2,0,0,3 +2012,20013,WSH,0,2,1,1,0,4 +2012,20014,FLA,2,0,0,0,0,2 +2012,20014,NYI,0,0,0,0,0,0 +2012,20015,NSH,1,0,3,0,0,4 +2012,20015,STL,1,0,1,0,0,2 +2012,20016,CBJ,0,1,1,0,0,2 +2012,20016,MIN,2,2,0,0,0,4 +2012,20017,CHI,1,2,2,0,0,5 +2012,20017,DAL,0,1,1,0,0,2 +2012,20018,COL,0,0,0,0,0,0 +2012,20018,DET,0,1,2,0,0,3 +2012,20019,CGY,1,0,2,0,0,3 +2012,20019,PIT,0,4,1,0,0,5 +2012,20020,ANA,1,0,0,0,1,2 +2012,20020,NYR,0,0,1,0,0,1 +2012,20021,PHX,0,1,2,0,0,3 +2012,20021,SJS,2,4,0,0,0,6 +2012,20022,BUF,0,3,1,0,0,4 +2012,20022,LAK,0,1,1,0,0,2 +2012,20023,MTL,1,1,3,0,0,5 +2012,20023,WPG,0,0,1,0,0,1 +2012,20024,EDM,0,0,1,0,1,2 +2012,20024,PIT,1,0,0,0,0,1 +2012,20025,BOS,0,0,0,0,0,0 +2012,20025,COL,0,0,1,0,0,1 +2012,20026,CAR,0,2,0,0,0,2 +2012,20026,NJD,1,1,2,0,0,4 +2012,20027,MIN,0,0,1,0,0,1 +2012,20027,NYI,2,0,0,0,0,2 +2012,20028,CGY,1,1,0,0,0,2 +2012,20028,STL,1,2,2,0,0,5 +2012,20029,TBL,2,1,2,0,0,5 +2012,20029,WSH,2,1,2,0,1,6 +2012,20030,CBJ,1,1,0,0,0,2 +2012,20030,VAN,1,0,2,0,0,3 +2012,20031,DAL,0,0,1,0,1,2 +2012,20031,PHX,0,0,1,0,0,1 +2012,20032,FLA,0,1,1,0,0,2 +2012,20032,PIT,1,2,1,0,0,4 +2012,20033,MIN,2,0,1,0,0,3 +2012,20033,OTT,0,1,2,0,1,4 +2012,20034,COL,1,0,1,0,1,3 +2012,20034,CBJ,0,1,1,0,0,2 +2012,20035,PHI,3,1,1,0,0,5 +2012,20035,VAN,1,2,1,0,0,4 +2012,20036,BOS,0,0,2,0,0,2 +2012,20036,CAR,0,1,2,0,0,3 +2012,20037,LAK,1,0,0,0,0,1 +2012,20037,NJD,1,0,0,0,1,2 +2012,20038,NYI,4,1,0,0,0,5 +2012,20038,TBL,1,0,0,0,0,1 +2012,20039,PIT,1,0,1,0,0,2 +2012,20039,WSH,0,1,1,1,0,3 +2012,20040,CGY,3,1,0,0,0,4 +2012,20040,MTL,1,0,0,0,0,1 +2012,20041,COL,2,1,4,0,0,7 +2012,20041,OTT,1,0,0,0,0,1 +2012,20042,DET,0,2,0,0,0,2 +2012,20042,VAN,0,0,0,0,0,0 +2012,20043,NSH,0,1,1,0,0,2 +2012,20043,PHX,3,1,1,0,0,5 +2012,20044,EDM,1,0,0,0,0,1 +2012,20044,MIN,0,1,0,0,1,2 +2012,20045,CHI,2,2,0,0,0,4 +2012,20045,WPG,2,1,0,0,0,3 +2012,20046,DAL,1,2,0,0,0,3 +2012,20046,STL,0,1,1,0,0,2 +2012,20047,BUF,2,0,1,0,0,3 +2012,20047,CAR,2,1,1,0,0,4 +2012,20048,ANA,1,0,0,0,0,1 +2012,20048,SJS,0,0,0,0,0,0 +2012,20049,CGY,2,0,0,0,0,2 +2012,20049,TOR,0,2,1,0,0,3 +2012,20050,COL,1,3,1,0,1,6 +2012,20050,MTL,1,2,2,0,0,5 +2012,20051,NYI,1,1,2,0,0,4 +2012,20051,NYR,0,2,0,0,0,2 +2012,20052,LAK,1,0,1,1,0,3 +2012,20052,PHI,1,0,1,0,0,2 +2012,20053,BUF,2,0,1,0,0,3 +2012,20053,PIT,0,1,1,0,0,2 +2012,20054,OTT,1,0,0,0,0,1 +2012,20054,WSH,2,0,0,0,0,2 +2012,20055,PHX,2,1,1,0,0,4 +2012,20055,WPG,0,1,0,0,0,1 +2012,20056,FLA,1,0,1,0,1,3 +2012,20056,TBL,0,0,2,0,0,2 +2012,20057,NSH,2,0,0,0,0,2 +2012,20057,NJD,0,0,2,0,1,3 +2012,20058,DET,0,1,1,1,0,3 +2012,20058,MIN,1,1,0,0,0,2 +2012,20059,CBJ,1,1,0,0,0,2 +2012,20059,DAL,1,1,2,0,0,4 +2012,20060,BOS,0,1,1,0,1,3 +2012,20060,CHI,1,1,0,0,0,2 +2012,20061,EDM,2,1,0,0,0,3 +2012,20061,VAN,1,2,1,0,0,4 +2012,20062,SJS,1,1,0,0,0,2 +2012,20062,STL,1,0,3,0,0,4 +2012,20063,ANA,1,0,3,0,0,4 +2012,20063,STL,0,1,1,0,0,2 +2012,20064,COL,0,2,0,1,0,3 +2012,20064,TOR,0,1,1,0,0,2 +2012,20065,FLA,2,2,3,0,0,7 +2012,20065,TBL,2,1,1,0,0,4 +2012,20066,PIT,0,1,0,0,0,1 +2012,20066,WPG,2,0,0,0,0,2 +2012,20067,EDM,0,0,3,0,0,3 +2012,20067,NSH,0,1,0,0,0,1 +2012,20068,ANA,2,0,1,0,0,3 +2012,20068,SJS,0,1,1,0,0,2 +2012,20069,BOS,0,0,1,0,0,1 +2012,20069,CAR,1,1,2,0,0,4 +2012,20070,FLA,0,0,0,0,0,0 +2012,20070,WSH,1,0,2,0,0,3 +2012,20071,CBJ,0,0,2,0,0,2 +2012,20071,DAL,0,1,2,0,0,3 +2012,20072,BUF,0,2,1,0,0,3 +2012,20072,MTL,0,1,0,0,0,1 +2012,20073,OTT,0,1,1,0,0,2 +2012,20073,PHI,4,0,3,0,0,7 +2012,20074,MIN,0,1,1,0,0,2 +2012,20074,PIT,0,2,2,0,0,4 +2012,20075,CGY,0,0,2,0,0,2 +2012,20075,EDM,1,0,0,0,0,1 +2012,20076,NYR,0,0,4,0,0,4 +2012,20076,VAN,0,0,0,0,0,0 +2012,20077,CHI,1,2,2,0,0,5 +2012,20077,PHX,0,2,0,0,0,2 +2012,20078,LAK,1,2,2,0,0,5 +2012,20078,STL,0,0,0,0,0,0 +2012,20079,TOR,1,0,2,0,1,4 +2012,20079,WPG,1,2,0,0,0,3 +2012,20080,BOS,3,0,3,0,0,6 +2012,20080,TOR,1,0,1,0,0,2 +2012,20081,PHI,1,0,1,0,0,2 +2012,20081,WSH,2,0,3,0,0,5 +2012,20082,MTL,0,0,1,0,0,1 +2012,20082,PIT,1,1,1,0,0,3 +2012,20083,OTT,0,2,2,0,0,4 +2012,20083,WPG,1,0,0,0,0,1 +2012,20084,NYI,0,0,1,0,0,1 +2012,20084,TBL,1,1,2,0,0,4 +2012,20085,BUF,0,2,1,0,0,3 +2012,20085,FLA,0,0,0,0,0,0 +2012,20086,CHI,1,0,2,0,0,3 +2012,20086,COL,0,1,0,0,0,1 +2012,20087,CGY,2,0,0,0,0,2 +2012,20087,NYR,2,0,0,1,0,3 +2012,20088,EDM,1,0,0,0,0,1 +2012,20088,MIN,0,0,1,0,1,2 +2012,20089,NSH,1,0,0,0,0,1 +2012,20089,VAN,4,0,1,0,0,5 +2012,20090,LAK,0,2,0,0,0,2 +2012,20090,PHX,0,0,0,0,0,0 +2012,20091,NJD,1,1,1,0,0,3 +2012,20091,SJS,0,2,1,0,1,4 +2012,20092,CBJ,1,0,1,0,0,2 +2012,20092,DET,2,1,2,0,0,5 +2012,20093,CAR,1,1,0,0,0,2 +2012,20093,STL,0,2,0,1,0,3 +2012,20094,ANA,0,1,0,0,0,1 +2012,20094,DAL,2,0,1,0,0,3 +2012,20095,CGY,0,0,0,0,0,0 +2012,20095,NSH,0,1,1,0,0,2 +2012,20096,MIN,1,1,0,0,0,2 +2012,20096,VAN,0,1,1,1,0,3 +2012,20097,BOS,0,0,2,0,0,2 +2012,20097,SJS,1,1,2,0,0,4 +2012,20098,MTL,1,1,2,0,0,4 +2012,20098,TOR,1,2,1,1,0,5 +2012,20099,CBJ,1,1,1,0,0,3 +2012,20099,OTT,1,1,2,0,0,4 +2012,20100,PHI,0,1,1,0,0,2 +2012,20100,STL,2,1,1,0,0,4 +2012,20101,NJD,0,0,1,0,0,1 +2012,20101,PIT,1,0,3,0,0,4 +2012,20102,DET,0,1,0,0,0,1 +2012,20102,WSH,2,2,3,0,0,7 +2012,20103,CAR,2,0,1,0,0,3 +2012,20103,WPG,2,3,0,0,0,5 +2012,20104,BUF,0,0,0,0,0,0 +2012,20104,TBL,0,1,2,0,0,3 +2012,20105,FLA,1,1,2,0,0,4 +2012,20105,NYI,1,1,0,0,0,2 +2012,20106,CHI,0,2,2,0,0,4 +2012,20106,COL,1,2,1,0,1,5 +2012,20107,EDM,0,1,1,0,0,2 +2012,20107,NYR,0,0,0,0,0,0 +2012,20108,DAL,0,0,0,0,0,0 +2012,20108,LAK,0,0,1,0,0,1 +2012,20109,ANA,2,1,1,0,0,4 +2012,20109,PHX,2,3,0,0,0,5 +2012,20110,PHI,1,1,2,0,0,4 +2012,20110,TOR,1,0,1,0,0,2 +2012,20111,FLA,1,0,1,0,0,2 +2012,20111,MTL,1,0,0,0,0,1 +2012,20112,NYR,0,1,1,0,0,2 +2012,20112,WPG,0,1,0,0,0,1 +2012,20113,BUF,2,0,1,0,0,3 +2012,20113,TBL,2,1,1,0,0,4 +2012,20114,NYI,0,0,0,0,0,0 +2012,20114,PIT,1,1,1,0,0,3 +2012,20115,CAR,0,0,2,0,0,2 +2012,20115,OTT,0,2,0,0,1,3 +2012,20116,CBJ,2,1,1,0,0,4 +2012,20116,DET,1,0,0,0,0,1 +2012,20117,NSH,0,0,1,0,0,1 +2012,20117,SJS,1,0,2,0,0,3 +2012,20118,ANA,0,2,0,0,0,2 +2012,20118,CHI,0,1,1,0,1,3 +2012,20119,EDM,0,3,0,0,0,3 +2012,20119,VAN,0,1,1,0,0,2 +2012,20120,DAL,1,0,1,0,1,3 +2012,20120,PHX,1,0,1,0,0,2 +2012,20121,LAK,0,0,0,0,0,0 +2012,20121,NJD,0,3,0,0,0,3 +2012,20122,MTL,1,2,2,0,0,5 +2012,20122,PHI,1,0,0,0,0,1 +2012,20123,CGY,1,2,1,0,0,4 +2012,20123,COL,1,0,1,0,0,2 +2012,20124,STL,0,1,2,0,0,3 +2012,20124,VAN,0,0,0,0,0,0 +2012,20125,BOS,1,0,0,0,0,1 +2012,20125,MTL,0,1,1,0,0,2 +2012,20126,BUF,1,1,2,0,0,4 +2012,20126,CBJ,0,1,1,0,0,2 +2012,20127,NYR,1,0,1,0,0,2 +2012,20127,TOR,0,1,3,0,0,4 +2012,20128,PHI,1,3,4,0,0,8 +2012,20128,WPG,3,3,3,0,0,9 +2012,20129,NYI,1,0,1,0,0,2 +2012,20129,PIT,0,0,2,0,1,3 +2012,20130,FLA,1,1,1,0,0,3 +2012,20130,OTT,0,2,2,0,0,4 +2012,20131,NSH,2,3,0,0,0,5 +2012,20131,TBL,2,0,1,0,0,3 +2012,20132,ANA,1,1,1,0,0,3 +2012,20132,MIN,0,1,1,0,0,2 +2012,20133,DAL,1,0,2,0,0,3 +2012,20133,LAK,0,3,2,0,0,5 +2012,20134,EDM,1,1,0,0,0,2 +2012,20134,WSH,1,0,0,0,0,1 +2012,20135,NJD,1,1,1,0,0,3 +2012,20135,PHX,1,2,2,0,0,5 +2012,20136,CAR,1,0,2,0,0,3 +2012,20136,CHI,0,0,0,0,0,0 +2012,20137,DET,1,1,0,0,0,2 +2012,20137,SJS,0,3,1,0,0,4 +2012,20138,COL,0,0,1,0,0,1 +2012,20138,EDM,0,2,1,0,0,3 +2012,20139,CGY,0,1,2,0,0,3 +2012,20139,STL,0,0,1,0,0,1 +2012,20140,NYR,0,2,2,0,0,4 +2012,20140,OTT,1,0,3,0,1,5 +2012,20141,BUF,0,1,1,0,0,2 +2012,20141,FLA,0,1,2,0,0,3 +2012,20142,PIT,0,2,1,0,0,3 +2012,20142,TOR,1,1,2,0,0,4 +2012,20143,BOS,0,1,1,0,0,2 +2012,20143,MTL,2,1,1,0,0,4 +2012,20144,NYI,0,2,0,0,0,2 +2012,20144,SJS,1,1,0,1,0,3 +2012,20145,CAR,0,1,0,0,0,1 +2012,20145,PHI,1,0,4,0,0,5 +2012,20146,TBL,0,1,0,0,0,1 +2012,20146,WPG,0,0,0,0,0,0 +2012,20147,ANA,0,0,0,0,0,0 +2012,20147,NSH,2,0,1,0,0,3 +2012,20148,DET,0,0,0,0,0,0 +2012,20148,MIN,1,0,0,0,0,1 +2012,20149,DAL,1,0,2,0,0,3 +2012,20149,NJD,0,1,0,0,0,1 +2012,20150,CHI,2,0,3,0,0,5 +2012,20150,CBJ,1,1,0,0,0,2 +2012,20151,LAK,0,1,1,0,0,2 +2012,20151,PHX,0,2,0,1,0,3 +2012,20152,VAN,3,1,3,0,0,7 +2012,20152,WSH,1,3,0,0,0,4 +2012,20153,ANA,1,0,0,0,0,1 +2012,20153,CBJ,1,2,0,0,0,3 +2012,20154,OTT,1,1,1,0,0,3 +2012,20154,TOR,1,0,1,0,0,2 +2012,20155,COL,1,1,1,0,0,3 +2012,20155,LAK,1,1,0,0,0,2 +2012,20156,EDM,1,3,0,0,0,4 +2012,20156,STL,0,1,1,0,0,2 +2012,20157,NYR,2,3,0,0,0,5 +2012,20157,SJS,1,1,0,0,0,2 +2012,20158,FLA,1,0,2,0,0,3 +2012,20158,WPG,2,0,1,0,1,4 +2012,20159,CHI,1,1,2,1,0,5 +2012,20159,NSH,1,2,1,0,0,4 +2012,20160,BOS,1,2,2,0,0,5 +2012,20160,OTT,2,0,1,0,0,3 +2012,20161,ANA,2,1,1,0,0,4 +2012,20161,WSH,0,2,2,1,0,5 +2012,20162,CAR,1,2,1,0,0,4 +2012,20162,TBL,1,0,1,0,0,2 +2012,20163,DET,1,0,0,0,0,1 +2012,20163,MIN,0,0,1,1,0,2 +2012,20164,CGY,0,0,1,0,0,1 +2012,20164,VAN,3,2,0,0,0,5 +2012,20165,BUF,0,1,1,0,0,2 +2012,20165,PHI,3,0,0,0,0,3 +2012,20166,NJD,1,2,0,0,0,3 +2012,20166,TOR,2,3,0,0,0,5 +2012,20167,COL,0,1,0,0,0,1 +2012,20167,PHX,1,1,2,0,0,4 +2012,20168,NYI,0,0,0,0,0,0 +2012,20168,WPG,1,0,2,0,0,3 +2012,20169,ANA,0,1,0,0,0,1 +2012,20169,NYR,1,0,0,0,1,2 +2012,20170,NJD,0,2,1,0,1,4 +2012,20170,PHI,0,2,1,0,0,3 +2012,20171,CBJ,0,1,0,0,0,1 +2012,20171,TOR,2,2,0,0,0,4 +2012,20172,CHI,1,1,0,0,1,3 +2012,20172,FLA,0,1,1,0,0,2 +2012,20173,CGY,1,1,2,0,0,4 +2012,20173,DET,0,1,0,0,0,1 +2012,20174,MIN,0,3,2,0,0,5 +2012,20174,VAN,1,0,0,0,0,1 +2012,20175,NSH,0,2,1,0,0,3 +2012,20175,PHX,0,0,0,0,0,0 +2012,20176,EDM,0,2,1,0,0,3 +2012,20176,LAK,0,0,0,0,0,0 +2012,20177,PIT,2,1,0,0,0,3 +2012,20177,SJS,0,1,2,0,1,4 +2012,20178,CAR,1,0,0,0,0,1 +2012,20178,WSH,0,2,3,0,0,5 +2012,20179,BUF,1,0,1,0,0,2 +2012,20179,CGY,0,0,1,0,0,1 +2012,20180,MTL,0,2,0,0,0,2 +2012,20180,OTT,0,0,1,0,0,1 +2012,20181,CHI,0,4,0,0,0,4 +2012,20181,TBL,0,4,0,1,0,5 +2012,20182,STL,1,1,1,0,0,3 +2012,20182,VAN,1,0,1,0,0,2 +2012,20183,COL,2,2,2,0,0,6 +2012,20183,DAL,0,4,2,1,0,7 +2012,20184,BOS,1,4,2,0,0,7 +2012,20184,TOR,0,0,0,0,0,0 +2012,20185,BUF,0,2,0,0,1,3 +2012,20185,OTT,1,1,0,0,0,2 +2012,20186,NJD,0,2,0,1,0,3 +2012,20186,WPG,0,0,2,0,0,2 +2012,20187,NYI,0,2,3,0,0,5 +2012,20187,WSH,2,0,1,0,0,3 +2012,20188,MTL,0,2,1,0,0,3 +2012,20188,NYR,3,1,1,0,0,5 +2012,20189,CBJ,0,0,2,0,0,2 +2012,20189,PHI,5,3,1,0,0,9 +2012,20190,ANA,0,0,0,0,0,0 +2012,20190,DET,1,2,2,0,0,5 +2012,20191,MIN,0,2,0,0,0,2 +2012,20191,STL,0,0,1,0,0,1 +2012,20192,EDM,0,0,2,0,0,2 +2012,20192,PHX,0,2,2,0,0,4 +2012,20193,LAK,0,1,1,0,0,2 +2012,20193,PIT,1,0,1,0,1,3 +2012,20194,NSH,0,1,2,1,0,4 +2012,20194,SJS,0,2,1,0,0,3 +2012,20195,CAR,1,0,1,0,0,2 +2012,20195,DAL,3,2,0,0,0,5 +2012,20196,FLA,3,0,0,0,0,3 +2012,20196,TBL,0,2,1,0,1,4 +2012,20197,NYR,0,1,2,0,0,3 +2012,20197,WPG,0,0,0,0,0,0 +2012,20198,CHI,1,1,0,0,0,2 +2012,20198,VAN,1,4,1,0,0,6 +2012,20199,CGY,1,1,0,0,0,2 +2012,20199,COL,1,0,0,0,0,1 +2012,20200,BOS,3,0,3,0,0,6 +2012,20200,NYI,2,0,0,0,0,2 +2012,20201,LAK,0,1,1,0,0,2 +2012,20201,SJS,0,3,1,0,0,4 +2012,20202,BUF,3,1,1,1,0,6 +2012,20202,WPG,3,1,1,0,0,5 +2012,20203,FLA,1,2,2,0,0,5 +2012,20203,TOR,0,0,1,0,0,1 +2012,20204,CAR,1,0,1,0,0,2 +2012,20204,NJD,0,1,2,0,0,3 +2012,20205,DAL,2,0,3,0,0,5 +2012,20205,WSH,1,1,0,0,0,2 +2012,20206,EDM,0,1,2,0,0,3 +2012,20206,MTL,0,0,1,0,0,1 +2012,20207,COL,0,1,1,0,0,2 +2012,20207,DET,1,2,2,0,0,5 +2012,20208,CHI,0,0,0,0,0,0 +2012,20208,STL,1,1,1,0,0,3 +2012,20209,CGY,0,0,0,0,0,0 +2012,20209,MIN,1,1,1,0,0,3 +2012,20210,LAK,2,2,0,0,0,4 +2012,20210,NSH,2,0,1,0,0,3 +2012,20211,NYR,0,2,1,0,0,3 +2012,20211,OTT,0,1,1,0,0,2 +2012,20212,PHI,0,1,0,0,0,1 +2012,20212,TBL,0,0,1,1,0,2 +2012,20213,ANA,1,1,0,0,0,2 +2012,20213,NSH,2,1,1,0,0,4 +2012,20214,BOS,2,2,2,0,0,6 +2012,20214,EDM,2,0,1,0,0,3 +2012,20215,CHI,2,3,1,0,0,6 +2012,20215,CBJ,1,0,2,0,0,3 +2012,20216,STL,0,1,1,0,0,2 +2012,20216,TOR,2,0,0,0,1,3 +2012,20217,FLA,2,2,1,0,0,5 +2012,20217,WPG,0,0,2,0,0,2 +2012,20218,COL,0,1,2,1,0,4 +2012,20218,NYI,1,2,0,0,0,3 +2012,20219,MTL,0,2,0,1,0,3 +2012,20219,PHX,1,0,1,0,0,2 +2012,20220,LAK,0,1,1,0,0,2 +2012,20220,VAN,3,0,0,0,0,3 +2012,20221,MIN,0,0,1,0,0,1 +2012,20221,SJS,0,3,0,0,0,3 +2012,20222,NJD,1,0,0,0,0,1 +2012,20222,WSH,0,1,2,0,0,3 +2012,20223,CAR,0,1,0,0,0,1 +2012,20223,NYR,1,0,4,0,0,5 +2012,20224,DAL,1,0,0,0,0,1 +2012,20224,PIT,0,1,2,0,0,3 +2012,20225,BUF,3,0,2,0,0,5 +2012,20225,OTT,1,0,0,0,0,1 +2012,20226,DET,1,1,1,0,0,3 +2012,20226,EDM,0,0,0,0,0,0 +2012,20227,CGY,0,0,1,0,0,1 +2012,20227,CHI,1,1,2,0,0,4 +2012,20228,ANA,0,4,0,0,0,4 +2012,20228,VAN,0,0,3,0,0,3 +2012,20229,BOS,0,3,3,0,0,6 +2012,20229,BUF,1,0,1,0,0,2 +2012,20230,OTT,0,2,3,0,0,5 +2012,20230,TOR,1,0,1,0,0,2 +2012,20231,NJD,0,2,0,0,1,3 +2012,20231,WSH,2,0,0,0,0,2 +2012,20232,CAR,1,2,2,0,0,5 +2012,20232,PIT,0,0,3,0,0,3 +2012,20233,DAL,1,1,0,0,0,2 +2012,20233,DET,1,1,3,0,0,5 +2012,20234,CBJ,1,0,1,0,0,2 +2012,20234,WPG,0,0,1,0,0,1 +2012,20235,MTL,1,0,0,1,0,2 +2012,20235,NSH,0,1,0,0,0,1 +2012,20236,STL,1,1,1,0,0,3 +2012,20236,TBL,0,0,0,0,0,0 +2012,20237,CGY,3,1,0,0,0,4 +2012,20237,COL,0,1,2,0,0,3 +2012,20238,LAK,1,4,0,0,0,5 +2012,20238,MIN,0,0,2,0,0,2 +2012,20239,PHX,2,0,1,0,0,3 +2012,20239,SJS,0,0,0,0,0,0 +2012,20240,FLA,0,1,1,0,0,2 +2012,20240,PHI,1,1,1,0,0,3 +2012,20241,CHI,3,2,1,0,0,6 +2012,20241,EDM,1,2,0,0,0,3 +2012,20242,ANA,0,1,1,0,0,2 +2012,20242,MIN,3,0,0,0,0,3 +2012,20243,NYI,0,1,0,0,0,1 +2012,20243,VAN,1,1,2,0,0,4 +2012,20244,CAR,1,1,1,0,0,3 +2012,20244,PHI,2,2,1,0,0,5 +2012,20245,BUF,0,0,2,0,1,3 +2012,20245,MTL,0,2,0,0,0,2 +2012,20246,TBL,0,1,1,0,0,2 +2012,20246,WPG,2,2,1,0,0,5 +2012,20247,BOS,0,1,3,0,0,4 +2012,20247,NJD,0,1,2,0,0,3 +2012,20248,PHX,2,0,0,0,1,3 +2012,20248,TOR,0,0,2,0,0,2 +2012,20249,NYI,1,0,1,0,0,2 +2012,20249,NYR,1,1,2,0,0,4 +2012,20250,COL,3,0,0,0,0,3 +2012,20250,PIT,1,1,4,0,0,6 +2012,20251,CBJ,2,0,0,0,0,2 +2012,20251,MIN,0,2,2,0,0,4 +2012,20252,DET,1,0,0,0,0,1 +2012,20252,STL,1,1,0,0,0,2 +2012,20253,NSH,0,0,3,0,0,3 +2012,20253,WSH,0,0,1,0,0,1 +2012,20254,DAL,0,0,0,0,0,0 +2012,20254,FLA,2,2,2,0,0,6 +2012,20255,CGY,0,1,0,0,0,1 +2012,20255,OTT,0,1,2,0,0,3 +2012,20256,CAR,0,0,0,0,0,0 +2012,20256,MTL,2,2,0,0,0,4 +2012,20257,BUF,1,0,2,0,0,3 +2012,20257,NJD,2,1,2,0,0,5 +2012,20258,CHI,0,2,3,0,0,5 +2012,20258,VAN,0,1,0,0,0,1 +2012,20259,CAR,0,0,0,0,0,0 +2012,20259,TBL,0,0,0,0,0,0 +2012,20260,BOS,0,1,0,0,1,2 +2012,20260,CBJ,0,1,0,0,0,1 +2012,20261,MTL,0,2,1,0,0,3 +2012,20261,NYI,0,4,0,0,0,4 +2012,20262,PHI,0,1,1,0,0,2 +2012,20262,PHX,0,0,1,0,0,1 +2012,20263,PIT,0,0,1,0,0,1 +2012,20263,TBL,1,1,2,0,0,4 +2012,20264,FLA,1,0,0,0,0,1 +2012,20264,STL,1,1,2,0,0,4 +2012,20265,NSH,1,1,2,0,0,4 +2012,20265,TOR,0,1,0,0,0,1 +2012,20266,COL,0,0,0,0,0,0 +2012,20266,MIN,0,0,1,0,0,1 +2012,20267,WSH,1,0,0,0,0,1 +2012,20267,WPG,1,3,0,0,0,4 +2012,20268,EDM,0,1,1,0,0,2 +2012,20268,OTT,3,1,1,0,0,5 +2012,20269,ANA,0,1,2,0,0,3 +2012,20269,LAK,0,2,3,0,0,5 +2012,20270,DET,1,0,1,0,0,2 +2012,20270,SJS,1,3,1,0,0,5 +2012,20271,BUF,1,0,0,0,0,1 +2012,20271,CAR,0,0,0,0,0,0 +2012,20272,COL,1,0,2,0,0,3 +2012,20272,DAL,0,0,0,0,0,0 +2012,20273,CGY,1,3,1,0,0,5 +2012,20273,CHI,1,1,0,0,0,2 +2012,20274,PHI,1,1,2,0,0,4 +2012,20274,WPG,3,2,1,0,0,6 +2012,20275,DET,1,2,1,0,0,4 +2012,20275,LAK,1,0,0,0,0,1 +2012,20276,BUF,1,1,0,0,0,2 +2012,20276,PHX,1,2,1,0,0,4 +2012,20277,TOR,2,4,1,0,0,7 +2012,20277,WSH,1,0,0,0,0,1 +2012,20278,MTL,1,1,2,0,0,4 +2012,20278,NYR,0,0,0,0,0,0 +2012,20279,BOS,3,0,3,0,0,6 +2012,20279,NYI,0,0,0,0,0,0 +2012,20280,NJD,0,3,1,0,0,4 +2012,20280,TBL,0,0,2,0,0,2 +2012,20281,FLA,1,1,1,0,0,3 +2012,20281,PIT,1,1,0,0,0,2 +2012,20282,CBJ,0,2,1,1,0,4 +2012,20282,NSH,1,1,1,0,0,3 +2012,20283,MIN,0,1,1,0,1,3 +2012,20283,STL,1,0,1,0,0,2 +2012,20284,DAL,0,0,1,0,0,1 +2012,20284,SJS,1,2,1,0,0,4 +2012,20285,CHI,1,1,0,0,0,2 +2012,20285,EDM,5,3,1,0,0,9 +2012,20286,CAR,1,1,1,0,0,3 +2012,20286,TOR,0,0,2,0,0,2 +2012,20287,COL,1,0,0,0,0,1 +2012,20287,SJS,1,2,1,0,0,4 +2012,20288,ANA,0,2,0,0,0,2 +2012,20288,DET,1,2,1,0,0,4 +2012,20289,OTT,0,0,1,0,0,1 +2012,20289,VAN,0,1,0,1,0,2 +2012,20290,CAR,2,2,0,0,0,4 +2012,20290,PHI,0,2,0,0,0,2 +2012,20291,NYI,0,0,0,0,0,0 +2012,20291,PIT,2,2,1,0,0,5 +2012,20292,PHX,1,1,1,0,0,3 +2012,20292,WSH,0,2,2,0,0,4 +2012,20293,CGY,0,1,0,0,0,1 +2012,20293,CBJ,2,1,1,0,0,4 +2012,20294,BOS,1,0,0,0,0,1 +2012,20294,MTL,0,0,0,0,0,0 +2012,20295,FLA,0,2,2,0,0,4 +2012,20295,NJD,3,0,0,0,0,3 +2012,20296,DAL,1,2,1,0,0,4 +2012,20296,EDM,0,1,0,0,0,1 +2012,20297,TBL,1,0,0,0,0,1 +2012,20297,TOR,3,1,3,0,0,7 +2012,20298,LAK,0,1,2,0,0,3 +2012,20298,STL,1,0,1,0,0,2 +2012,20299,EDM,3,0,3,0,0,6 +2012,20299,NSH,0,0,2,0,0,2 +2012,20300,BOS,0,2,1,0,1,4 +2012,20300,BUF,2,1,0,0,0,3 +2012,20301,CBJ,0,0,1,0,0,1 +2012,20301,NJD,0,1,0,0,1,2 +2012,20302,NYI,3,0,0,0,0,3 +2012,20302,PHI,1,1,1,1,0,4 +2012,20303,PIT,0,0,2,0,0,2 +2012,20303,STL,1,0,1,1,0,3 +2012,20304,WSH,2,1,0,1,0,4 +2012,20304,WPG,2,0,1,0,0,3 +2012,20305,CAR,2,0,1,0,0,3 +2012,20305,MTL,0,1,2,0,1,4 +2012,20306,CGY,0,2,1,0,0,3 +2012,20306,DET,1,1,3,0,0,5 +2012,20307,FLA,1,1,0,0,0,2 +2012,20307,NYR,0,1,0,0,0,1 +2012,20308,MIN,0,0,3,0,0,3 +2012,20308,NSH,1,0,1,0,0,2 +2012,20309,DAL,0,0,2,1,0,3 +2012,20309,LAK,0,2,0,0,0,2 +2012,20310,COL,0,0,0,0,0,0 +2012,20310,VAN,1,1,1,0,0,3 +2012,20311,ANA,1,0,1,0,0,2 +2012,20311,PHX,2,1,1,0,0,4 +2012,20312,CHI,0,0,0,0,0,0 +2012,20312,SJS,0,1,0,0,0,1 +2012,20313,BOS,0,1,1,0,0,2 +2012,20313,DET,1,1,0,0,1,3 +2012,20314,NJD,0,1,0,0,0,1 +2012,20314,NYI,0,0,0,0,0,0 +2012,20315,MTL,1,0,0,0,0,1 +2012,20315,PHI,0,3,0,0,0,3 +2012,20316,EDM,2,1,2,0,0,5 +2012,20316,MIN,1,1,0,0,0,2 +2012,20317,NYR,0,4,2,0,0,6 +2012,20317,WSH,0,2,1,0,0,3 +2012,20318,ANA,3,1,1,0,0,5 +2012,20318,CHI,2,0,4,0,0,6 +2012,20319,OTT,1,1,1,0,0,3 +2012,20319,PIT,4,1,1,0,0,6 +2012,20320,CAR,0,1,0,0,0,1 +2012,20320,WPG,1,1,1,0,0,3 +2012,20321,BUF,0,1,0,0,0,1 +2012,20321,CBJ,2,3,0,0,0,5 +2012,20322,FLA,0,1,0,0,0,1 +2012,20322,TBL,0,0,1,1,0,2 +2012,20323,CGY,0,0,0,0,0,0 +2012,20323,STL,1,0,1,0,0,2 +2012,20324,DAL,1,1,1,0,0,3 +2012,20324,TOR,2,0,1,0,1,4 +2012,20325,PHX,0,0,0,0,0,0 +2012,20325,VAN,0,3,2,0,0,5 +2012,20326,NJD,0,1,1,0,0,2 +2012,20326,NYI,1,1,1,0,0,3 +2012,20327,NYR,0,1,1,0,0,2 +2012,20327,PHI,0,0,0,0,0,0 +2012,20328,COL,1,0,4,0,0,5 +2012,20328,EDM,1,1,0,0,0,2 +2012,20329,BOS,1,2,1,0,0,4 +2012,20329,WPG,2,0,0,0,0,2 +2012,20330,BUF,2,1,2,0,0,5 +2012,20330,WSH,0,1,0,0,0,1 +2012,20331,MTL,2,1,0,0,0,3 +2012,20331,PIT,1,1,1,1,0,4 +2012,20332,FLA,0,0,1,0,0,1 +2012,20332,TBL,1,3,1,0,0,5 +2012,20333,DET,1,2,1,0,0,4 +2012,20333,NSH,0,0,1,0,0,1 +2012,20334,DAL,0,0,0,0,0,0 +2012,20334,PHX,1,1,1,0,0,3 +2012,20335,SJS,1,0,1,0,0,2 +2012,20335,VAN,1,2,0,0,0,3 +2012,20336,CHI,1,1,0,0,0,2 +2012,20336,LAK,0,1,0,0,0,1 +2012,20337,CAR,1,0,2,0,0,3 +2012,20337,OTT,2,1,1,0,0,4 +2012,20338,CBJ,0,1,0,0,0,1 +2012,20338,STL,0,1,1,0,0,2 +2012,20339,CGY,3,0,2,0,0,5 +2012,20339,MIN,2,0,0,0,0,2 +2012,20340,ANA,1,0,1,0,0,2 +2012,20340,TOR,2,1,2,0,0,5 +2012,20341,MIN,0,2,1,0,0,3 +2012,20341,TBL,0,1,0,0,0,1 +2012,20342,COL,0,0,1,0,0,1 +2012,20342,DAL,1,0,2,0,0,3 +2012,20343,EDM,1,0,0,0,0,1 +2012,20343,NSH,0,1,1,0,0,2 +2012,20344,LAK,1,1,0,0,0,2 +2012,20344,SJS,0,0,0,0,0,0 +2012,20345,BUF,0,0,1,0,0,1 +2012,20345,NYI,1,0,1,0,0,2 +2012,20346,NYR,0,4,0,0,0,4 +2012,20346,PIT,1,1,1,0,0,3 +2012,20347,STL,1,1,0,0,0,2 +2012,20347,WSH,1,0,0,0,0,1 +2012,20348,CAR,0,1,0,0,0,1 +2012,20348,FLA,0,1,2,0,0,3 +2012,20349,OTT,1,2,3,0,0,6 +2012,20349,WPG,0,2,2,0,0,4 +2012,20350,CHI,0,0,1,0,0,1 +2012,20350,PHX,1,1,2,0,0,4 +2012,20351,CGY,1,0,0,0,0,1 +2012,20351,NSH,0,0,0,0,0,0 +2012,20352,CBJ,0,0,1,0,0,1 +2012,20352,VAN,2,0,2,0,0,4 +2012,20353,BOS,1,2,3,0,0,6 +2012,20353,TOR,1,1,1,0,0,3 +2012,20354,DET,2,0,2,0,0,4 +2012,20354,TBL,0,2,0,0,0,2 +2012,20355,COL,3,0,3,0,0,6 +2012,20355,NJD,0,1,0,0,0,1 +2012,20356,EDM,1,1,0,0,0,2 +2012,20356,MIN,0,2,0,0,1,3 +2012,20357,ANA,1,1,2,0,0,4 +2012,20357,MTL,1,0,0,0,0,1 +2012,20358,PIT,1,0,1,0,0,2 +2012,20358,WSH,0,1,0,0,0,1 +2012,20359,CAR,2,1,0,0,0,3 +2012,20359,NYR,1,3,1,0,0,5 +2012,20360,PHX,0,0,0,0,0,0 +2012,20360,WPG,1,0,0,0,0,1 +2012,20361,DAL,1,0,2,0,0,3 +2012,20361,OTT,1,0,1,0,0,2 +2012,20362,CGY,1,1,1,0,0,3 +2012,20362,CBJ,1,0,2,0,1,4 +2012,20363,NSH,3,2,1,0,0,6 +2012,20363,VAN,1,4,0,0,0,5 +2012,20364,FLA,0,1,0,0,0,1 +2012,20364,LAK,1,1,0,0,0,2 +2012,20365,MTL,1,1,1,0,0,3 +2012,20365,SJS,1,1,1,0,1,4 +2012,20366,BUF,0,0,1,0,0,1 +2012,20366,DET,3,0,1,0,0,4 +2012,20367,MIN,3,0,1,0,0,4 +2012,20367,NJD,2,0,0,0,0,2 +2012,20368,CHI,1,2,1,0,1,5 +2012,20368,NYI,2,0,2,0,0,4 +2012,20369,COL,1,0,1,0,1,3 +2012,20369,STL,1,1,0,0,0,2 +2012,20370,CBJ,1,1,1,0,0,3 +2012,20370,EDM,1,0,5,0,0,6 +2012,20371,ANA,2,1,0,0,0,3 +2012,20371,PHI,0,1,2,1,0,4 +2012,20372,LAK,0,1,0,0,0,1 +2012,20372,MTL,1,1,0,0,0,2 +2012,20373,BOS,0,2,2,0,0,4 +2012,20373,TOR,0,1,0,0,0,1 +2012,20374,OTT,0,1,1,0,0,2 +2012,20374,WSH,1,0,1,1,0,3 +2012,20375,CAR,0,1,1,0,0,2 +2012,20375,PIT,0,1,2,0,0,3 +2012,20376,NJD,1,1,0,0,0,2 +2012,20376,WPG,1,1,2,0,0,4 +2012,20377,NYR,0,1,3,0,0,4 +2012,20377,TBL,1,1,0,0,0,2 +2012,20378,CHI,0,3,2,0,0,5 +2012,20378,STL,1,1,0,0,0,2 +2012,20379,BUF,2,0,1,0,0,3 +2012,20379,NSH,0,0,2,0,0,2 +2012,20380,PHI,4,0,0,0,0,4 +2012,20380,PHX,0,1,1,0,0,2 +2012,20381,DAL,0,4,0,0,0,4 +2012,20381,NYI,2,3,0,0,0,5 +2012,20382,CGY,2,0,3,0,0,5 +2012,20382,EDM,2,0,1,0,0,3 +2012,20383,FLA,1,3,1,0,0,5 +2012,20383,SJS,2,0,1,0,0,3 +2012,20384,COL,0,2,2,0,0,4 +2012,20384,DET,1,0,1,0,0,2 +2012,20385,ANA,2,0,1,0,0,3 +2012,20385,MIN,1,2,2,0,0,5 +2012,20386,CGY,1,0,0,0,0,1 +2012,20386,VAN,0,2,3,0,0,5 +2012,20387,NYR,0,2,0,0,0,2 +2012,20387,TOR,1,2,1,0,0,4 +2012,20388,BOS,0,2,1,0,0,3 +2012,20388,PIT,0,0,1,0,0,1 +2012,20389,OTT,0,1,3,0,0,4 +2012,20389,TBL,0,1,1,0,0,2 +2012,20390,FLA,3,2,0,0,0,5 +2012,20390,WSH,1,1,2,0,0,4 +2012,20391,CHI,0,3,0,0,0,3 +2012,20391,PHX,2,1,0,0,1,4 +2012,20392,NJD,2,0,0,1,0,3 +2012,20392,TOR,0,1,1,0,0,2 +2012,20393,NYI,3,0,2,0,0,5 +2012,20393,TBL,1,0,0,0,0,1 +2012,20394,CBJ,1,1,0,0,1,3 +2012,20394,MTL,1,0,1,0,0,2 +2012,20395,DET,0,1,1,0,0,2 +2012,20395,STL,0,1,2,0,0,3 +2012,20396,NSH,1,0,1,0,0,2 +2012,20396,PHX,1,0,2,0,0,3 +2012,20397,BOS,0,0,1,0,0,1 +2012,20397,WPG,1,0,1,0,0,2 +2012,20398,CGY,2,2,3,0,0,7 +2012,20398,CAR,0,2,4,0,0,6 +2012,20399,COL,0,0,0,0,0,0 +2012,20399,VAN,1,1,4,0,0,6 +2012,20400,ANA,2,0,1,0,0,3 +2012,20400,LAK,0,1,1,0,0,2 +2012,20401,MIN,2,0,0,0,0,2 +2012,20401,SJS,1,0,0,0,0,1 +2012,20402,OTT,0,2,1,0,0,3 +2012,20402,WSH,0,1,4,0,0,5 +2012,20403,BUF,3,0,1,0,0,4 +2012,20403,PHI,1,3,0,1,0,5 +2012,20404,CAR,1,3,1,0,0,5 +2012,20404,EDM,1,1,1,0,0,3 +2012,20405,BOS,0,0,0,0,0,0 +2012,20405,FLA,0,0,2,0,0,2 +2012,20406,NJD,0,1,3,0,1,5 +2012,20406,OTT,2,1,1,0,0,4 +2012,20407,CHI,1,1,0,1,0,3 +2012,20407,NYI,0,1,1,0,0,2 +2012,20408,NYR,1,1,0,0,0,2 +2012,20408,TBL,0,1,1,0,1,3 +2012,20409,PHI,1,2,0,0,0,3 +2012,20409,PIT,0,1,1,0,0,2 +2012,20410,CBJ,1,1,1,0,0,3 +2012,20410,NSH,0,1,2,1,0,4 +2012,20411,MTL,2,1,0,0,0,3 +2012,20411,VAN,0,1,2,0,1,4 +2012,20412,DET,5,0,0,0,0,5 +2012,20412,PHX,0,1,1,0,0,2 +2012,20413,ANA,1,0,1,0,0,2 +2012,20413,STL,1,2,1,0,0,4 +2012,20414,CGY,0,0,3,0,0,3 +2012,20414,COL,0,1,1,0,0,2 +2012,20415,LAK,0,0,2,0,0,2 +2012,20415,MIN,1,2,1,0,0,4 +2012,20416,DAL,1,1,0,0,0,2 +2012,20416,SJS,2,3,0,0,0,5 +2012,20417,TOR,0,1,1,0,0,2 +2012,20417,WSH,1,2,1,0,0,4 +2012,20418,BUF,1,0,0,1,0,2 +2012,20418,FLA,0,1,0,0,0,1 +2012,20419,CAR,0,0,2,0,0,2 +2012,20419,WPG,2,2,0,0,0,4 +2012,20420,COL,0,1,0,0,0,1 +2012,20420,EDM,2,2,0,0,0,4 +2012,20421,MTL,0,1,1,0,0,2 +2012,20421,NJD,0,0,1,0,0,1 +2012,20422,BUF,0,1,0,0,0,1 +2012,20422,NYR,0,2,2,0,0,4 +2012,20423,OTT,0,1,0,0,0,1 +2012,20423,VAN,2,1,1,0,0,4 +2012,20424,NYI,2,1,0,0,0,3 +2012,20424,PIT,2,2,2,0,0,6 +2012,20425,PHI,0,3,2,0,0,5 +2012,20425,TBL,1,0,1,0,0,2 +2012,20426,DET,2,4,1,0,0,7 +2012,20426,WPG,1,0,0,0,0,1 +2012,20427,BOS,2,1,2,0,0,5 +2012,20427,CBJ,2,1,0,0,0,3 +2012,20428,SJS,0,0,0,0,0,0 +2012,20428,STL,1,0,0,0,0,1 +2012,20429,ANA,0,1,1,0,0,2 +2012,20429,NSH,0,1,2,0,0,3 +2012,20430,MIN,1,2,1,0,0,4 +2012,20430,PHX,0,0,1,0,0,1 +2012,20431,CGY,0,2,1,0,0,3 +2012,20431,EDM,0,0,0,0,0,0 +2012,20432,DAL,1,1,0,0,0,2 +2012,20432,LAK,0,1,0,0,0,1 +2012,20433,FLA,1,0,0,0,0,1 +2012,20433,NYR,2,4,0,0,0,6 +2012,20434,CHI,0,1,1,1,0,3 +2012,20434,SJS,1,1,0,0,0,2 +2012,20435,NJD,2,3,0,0,0,5 +2012,20435,TBL,2,0,2,0,0,4 +2012,20436,BOS,1,1,1,0,0,3 +2012,20436,LAK,0,0,0,0,0,0 +2012,20437,BUF,2,0,0,0,0,2 +2012,20437,OTT,1,0,1,1,0,3 +2012,20438,CAR,0,0,1,0,0,1 +2012,20438,TOR,0,0,1,1,0,2 +2012,20439,DAL,0,0,1,0,0,1 +2012,20439,NYR,0,0,0,0,0,0 +2012,20440,DET,0,1,3,0,0,4 +2012,20440,PIT,1,0,0,0,0,1 +2012,20441,PHI,1,3,1,0,0,5 +2012,20441,WSH,0,0,1,0,0,1 +2012,20442,CBJ,1,0,0,0,1,2 +2012,20442,VAN,0,0,1,0,0,1 +2012,20443,MTL,2,0,3,0,0,5 +2012,20443,NYI,1,0,2,0,0,3 +2012,20444,FLA,1,1,0,0,0,2 +2012,20444,NJD,0,1,1,0,1,3 +2012,20445,CGY,0,1,0,0,0,1 +2012,20445,NSH,1,0,1,0,0,2 +2012,20446,MIN,0,1,0,0,0,1 +2012,20446,WPG,0,1,1,0,0,2 +2012,20447,COL,1,0,2,0,1,4 +2012,20447,SJS,0,2,1,0,0,3 +2012,20448,BOS,1,1,3,0,0,5 +2012,20448,OTT,0,1,1,0,0,2 +2012,20449,CHI,0,2,1,0,1,4 +2012,20449,MIN,0,2,1,0,0,3 +2012,20450,ANA,1,2,1,0,0,4 +2012,20450,PHX,0,1,0,0,0,1 +2012,20451,DAL,1,0,2,0,0,3 +2012,20451,NYI,1,1,0,0,0,2 +2012,20452,CAR,0,2,2,0,0,4 +2012,20452,VAN,1,1,1,0,0,3 +2012,20453,CBJ,0,1,0,0,0,1 +2012,20453,LAK,0,0,2,0,0,2 +2012,20454,MTL,0,3,0,0,0,3 +2012,20454,PHI,1,2,1,0,0,4 +2012,20455,CGY,1,1,2,0,0,4 +2012,20455,TBL,1,3,0,1,0,5 +2012,20456,NYR,0,1,0,0,0,1 +2012,20456,STL,2,0,2,0,0,4 +2012,20457,DET,1,2,0,0,0,3 +2012,20457,NSH,1,1,2,0,0,4 +2012,20458,WSH,0,0,1,0,0,1 +2012,20458,WPG,0,0,0,0,0,0 +2012,20459,EDM,0,1,1,0,0,2 +2012,20459,PHX,1,1,2,0,0,4 +2012,20460,COL,2,2,0,0,0,4 +2012,20460,SJS,1,1,3,0,0,5 +2012,20461,DAL,1,1,1,0,0,3 +2012,20461,NJD,1,3,2,0,0,6 +2012,20462,BUF,0,4,1,0,0,5 +2012,20462,TOR,0,2,2,0,0,4 +2012,20463,OTT,0,5,1,0,0,6 +2012,20463,PIT,0,3,1,0,0,4 +2012,20464,CGY,1,0,1,0,0,2 +2012,20464,FLA,1,0,1,0,1,3 +2012,20465,ANA,0,0,1,0,0,1 +2012,20465,CHI,1,2,1,0,0,4 +2012,20466,BOS,4,1,1,0,0,6 +2012,20466,PHI,0,0,0,0,0,0 +2012,20467,TOR,1,1,1,0,0,3 +2012,20467,VAN,1,2,2,0,0,5 +2012,20468,MTL,0,3,0,0,0,3 +2012,20468,NJD,1,2,2,0,0,5 +2012,20469,BUF,1,0,2,0,0,3 +2012,20469,PIT,4,2,2,0,0,8 +2012,20470,ANA,2,0,1,0,0,3 +2012,20470,WPG,2,2,1,0,0,5 +2012,20471,DET,4,2,2,0,0,8 +2012,20471,LAK,1,1,0,0,0,2 +2012,20472,CBJ,0,2,0,0,0,2 +2012,20472,TBL,2,1,0,0,0,3 +2012,20473,NSH,0,0,1,0,1,2 +2012,20473,STL,0,0,1,0,0,1 +2012,20474,MIN,0,0,1,0,0,1 +2012,20474,NYI,0,1,0,0,1,2 +2012,20475,NYR,1,0,2,0,0,3 +2012,20475,PHX,1,1,0,0,0,2 +2012,20476,COL,1,1,0,0,0,2 +2012,20476,WSH,0,1,0,0,0,1 +2012,20477,EDM,1,1,0,0,0,2 +2012,20477,SJS,1,1,1,0,0,3 +2012,20478,CAR,1,1,0,0,0,2 +2012,20478,FLA,1,0,1,1,0,3 +2012,20479,CGY,0,0,2,0,0,2 +2012,20479,CHI,1,2,1,0,0,4 +2012,20480,CBJ,1,2,1,0,0,4 +2012,20480,STL,2,0,4,0,0,6 +2012,20481,BOS,1,1,1,0,0,3 +2012,20481,MTL,1,0,1,0,0,2 +2012,20482,LAK,1,1,0,0,1,3 +2012,20482,TOR,1,0,1,0,0,2 +2012,20483,ANA,0,3,0,0,0,3 +2012,20483,DAL,1,2,2,0,0,5 +2012,20484,COL,0,2,0,0,1,3 +2012,20484,PHI,1,0,1,0,0,2 +2012,20485,DET,1,1,1,0,0,3 +2012,20485,EDM,0,2,0,0,0,2 +2012,20486,MIN,0,0,0,0,0,0 +2012,20486,VAN,1,2,1,0,0,4 +2012,20487,NJD,1,0,0,0,0,1 +2012,20487,NYR,0,1,3,0,0,4 +2012,20488,CHI,0,0,2,0,0,2 +2012,20488,PIT,1,2,0,0,0,3 +2012,20489,NSH,0,0,1,0,0,1 +2012,20489,WSH,2,0,2,0,0,4 +2012,20490,BUF,0,1,0,0,0,1 +2012,20490,OTT,1,0,3,0,0,4 +2012,20491,FLA,0,1,0,0,0,1 +2012,20491,PHX,1,1,0,0,0,2 +2012,20492,NYI,1,1,0,0,1,3 +2012,20492,WPG,1,1,0,0,0,2 +2012,20493,CGY,1,1,0,0,0,2 +2012,20493,MIN,1,0,0,0,0,1 +2012,20494,CAR,1,2,0,0,0,3 +2012,20494,PHX,1,1,2,0,0,4 +2012,20495,CHI,0,2,3,0,0,5 +2012,20495,MTL,0,1,0,0,0,1 +2012,20496,DAL,1,0,0,0,0,1 +2012,20496,PHI,2,1,1,0,0,4 +2012,20497,COL,2,1,0,0,0,3 +2012,20497,STL,0,2,0,0,0,2 +2012,20498,DET,1,1,0,0,0,2 +2012,20498,VAN,3,0,1,0,0,4 +2012,20499,SJS,5,0,2,0,0,7 +2012,20499,TBL,1,0,1,0,0,2 +2012,20500,BUF,0,1,1,0,0,2 +2012,20500,TOR,0,2,1,0,0,3 +2012,20501,NYI,0,1,1,0,0,2 +2012,20501,NYR,1,2,1,0,0,4 +2012,20502,FLA,0,1,2,0,0,3 +2012,20502,OTT,1,0,2,1,0,4 +2012,20503,CBJ,4,1,0,0,0,5 +2012,20503,NSH,1,3,2,0,0,6 +2012,20504,MTL,0,0,0,0,0,0 +2012,20504,WPG,2,1,1,0,0,4 +2012,20505,CGY,1,2,0,0,0,3 +2012,20505,DET,0,0,2,0,0,2 +2012,20506,EDM,2,2,0,0,0,4 +2012,20506,MIN,0,0,1,0,0,1 +2012,20507,ANA,0,1,1,0,0,2 +2012,20507,LAK,0,1,1,0,1,3 +2012,20508,BOS,4,2,2,0,0,8 +2012,20508,FLA,0,0,0,0,0,0 +2012,20509,NJD,0,3,0,0,1,4 +2012,20509,WSH,0,0,3,0,0,3 +2012,20510,NYI,1,1,1,0,0,3 +2012,20510,TOR,3,1,1,0,0,5 +2012,20511,NYR,0,2,2,0,0,4 +2012,20511,PHI,0,0,2,0,0,2 +2012,20512,CAR,1,0,0,1,0,2 +2012,20512,OTT,1,0,0,0,0,1 +2012,20513,PIT,1,0,3,0,0,4 +2012,20513,WPG,0,1,0,0,0,1 +2012,20514,DAL,3,1,2,0,0,6 +2012,20514,NSH,0,2,1,0,0,3 +2012,20515,COL,0,1,0,1,0,2 +2012,20515,TBL,1,0,0,0,0,1 +2012,20516,PHX,0,1,1,0,0,2 +2012,20516,STL,0,2,1,0,0,3 +2012,20517,CGY,0,0,3,0,0,3 +2012,20517,VAN,0,0,1,0,0,1 +2012,20518,LAK,0,1,0,0,0,1 +2012,20518,SJS,0,1,0,0,1,2 +2012,20519,COL,0,2,2,0,0,4 +2012,20519,MIN,1,1,0,0,0,2 +2012,20520,BUF,4,0,0,0,0,4 +2012,20520,WSH,0,1,1,0,0,2 +2012,20521,NYI,0,0,0,0,0,0 +2012,20521,NYR,0,1,2,0,0,3 +2012,20522,CAR,1,2,1,0,0,4 +2012,20522,NJD,0,0,2,0,0,2 +2012,20523,DAL,0,1,2,0,0,3 +2012,20523,STL,1,2,2,0,0,5 +2012,20524,DET,1,2,1,0,0,4 +2012,20524,NSH,0,0,1,0,0,1 +2012,20525,CHI,0,3,1,0,0,4 +2012,20525,CBJ,0,0,1,0,0,1 +2012,20526,EDM,0,2,1,0,0,3 +2012,20526,VAN,3,1,1,0,0,5 +2012,20527,LAK,2,0,2,0,0,4 +2012,20527,PHX,1,1,1,0,0,3 +2012,20528,ANA,1,2,0,0,0,3 +2012,20528,SJS,1,1,0,0,0,2 +2012,20529,CAR,1,0,1,0,0,2 +2012,20529,PIT,0,1,3,0,0,4 +2012,20530,CGY,0,0,1,0,1,2 +2012,20530,CBJ,0,1,0,0,0,1 +2012,20531,MTL,3,2,1,0,0,6 +2012,20531,OTT,1,1,0,0,0,2 +2012,20532,PHI,1,0,0,0,0,1 +2012,20532,TBL,1,3,1,0,0,5 +2012,20533,FLA,1,3,1,0,0,5 +2012,20533,TOR,0,2,1,0,0,3 +2012,20534,DET,0,1,2,0,0,3 +2012,20534,STL,1,1,0,0,0,2 +2012,20535,COL,0,0,1,0,0,1 +2012,20535,WPG,1,2,1,0,0,4 +2012,20536,BUF,0,1,0,0,0,1 +2012,20536,NJD,1,2,0,0,0,3 +2012,20537,NYR,1,0,0,0,0,1 +2012,20537,WSH,1,2,1,0,0,4 +2012,20538,MIN,0,0,1,0,0,1 +2012,20538,NSH,0,0,1,0,1,2 +2012,20539,CHI,0,0,0,0,0,0 +2012,20539,LAK,0,1,1,0,0,2 +2012,20540,BOS,1,0,0,1,0,2 +2012,20540,PHX,1,0,0,0,0,1 +2012,20541,SJS,0,2,0,0,0,2 +2012,20541,VAN,1,1,0,1,0,3 +2012,20542,CGY,1,0,0,0,0,1 +2012,20542,NYI,0,1,2,0,0,3 +2012,20543,PHI,1,2,1,0,0,4 +2012,20543,PIT,1,0,1,0,0,2 +2012,20544,CAR,1,0,2,1,0,4 +2012,20544,TOR,0,1,2,0,0,3 +2012,20545,MTL,1,2,0,0,0,3 +2012,20545,TBL,1,1,2,0,0,4 +2012,20546,EDM,1,1,1,0,0,3 +2012,20546,MIN,1,3,0,0,0,4 +2012,20547,LAK,0,0,0,0,0,0 +2012,20547,WPG,0,0,0,1,0,1 +2012,20548,CBJ,1,1,2,0,0,4 +2012,20548,DAL,1,0,0,0,0,1 +2012,20549,COL,2,0,1,0,0,3 +2012,20549,PHX,1,1,0,0,0,2 +2012,20550,ANA,0,0,2,0,0,2 +2012,20550,VAN,2,2,1,0,0,5 +2012,20551,BUF,0,1,0,0,0,1 +2012,20551,WSH,1,1,1,0,0,3 +2012,20552,CGY,3,0,0,0,0,3 +2012,20552,OTT,0,2,1,1,0,4 +2012,20553,FLA,0,1,0,0,0,1 +2012,20553,NYR,3,1,0,0,0,4 +2012,20554,NSH,1,0,0,0,1,2 +2012,20554,STL,1,0,0,0,0,1 +2012,20555,CHI,1,1,1,0,0,3 +2012,20555,DET,1,1,0,0,0,2 +2012,20556,NJD,2,0,1,0,0,3 +2012,20556,PIT,0,0,1,0,0,1 +2012,20557,EDM,1,0,0,0,0,1 +2012,20557,NYI,2,2,0,0,0,4 +2012,20558,CAR,1,1,0,0,0,2 +2012,20558,TBL,1,1,3,0,0,5 +2012,20559,FLA,0,1,2,0,0,3 +2012,20559,MTL,1,0,1,0,0,2 +2012,20560,MIN,0,1,1,0,0,2 +2012,20560,PHX,0,1,3,0,0,4 +2012,20561,BUF,1,1,0,0,0,2 +2012,20561,OTT,0,2,0,0,1,3 +2012,20562,TOR,1,1,0,0,0,2 +2012,20562,WPG,0,3,0,0,0,3 +2012,20563,DET,0,3,0,0,0,3 +2012,20563,STL,0,0,0,0,0,0 +2012,20564,CBJ,0,2,0,0,0,2 +2012,20564,WSH,0,0,4,0,0,4 +2012,20565,BOS,1,0,1,0,0,2 +2012,20565,DAL,3,1,0,0,0,4 +2012,20566,ANA,0,1,1,0,0,2 +2012,20566,COL,1,2,1,0,0,4 +2012,20567,LAK,1,2,1,0,0,4 +2012,20567,VAN,1,0,0,0,0,1 +2012,20568,CGY,1,0,2,0,0,3 +2012,20568,NSH,2,2,1,0,0,5 +2012,20569,NYR,0,1,2,0,0,3 +2012,20569,PHI,0,2,0,0,0,2 +2012,20570,NJD,0,2,0,0,0,2 +2012,20570,OTT,0,0,2,1,0,3 +2012,20571,CHI,1,1,1,0,0,3 +2012,20571,EDM,0,3,1,0,0,4 +2012,20572,SJS,0,2,0,0,1,3 +2012,20572,VAN,1,0,1,0,0,2 +2012,20573,COL,0,0,1,0,1,2 +2012,20573,LAK,0,1,0,0,0,1 +2012,20574,BUF,1,1,2,0,0,4 +2012,20574,EDM,1,1,1,0,0,3 +2012,20575,TBL,1,2,0,0,0,3 +2012,20575,TOR,2,3,2,0,0,7 +2012,20576,CGY,0,1,0,0,0,1 +2012,20576,WSH,1,2,0,0,0,3 +2012,20577,CAR,1,1,1,0,0,3 +2012,20577,NYI,1,0,2,0,1,4 +2012,20578,PHX,1,0,0,0,0,1 +2012,20578,STL,2,1,1,0,0,4 +2012,20579,DAL,1,2,1,0,0,4 +2012,20579,DET,3,2,0,0,0,5 +2012,20580,MTL,2,1,4,0,0,7 +2012,20580,WPG,1,1,1,0,0,3 +2012,20581,BOS,2,1,3,0,0,6 +2012,20581,NJD,1,0,0,0,0,1 +2012,20582,MIN,0,0,0,0,0,0 +2012,20582,VAN,2,0,1,0,0,3 +2012,20583,ANA,1,0,0,0,0,1 +2012,20583,SJS,0,2,1,0,0,3 +2012,20584,BOS,3,4,2,0,0,9 +2012,20584,CGY,0,0,0,0,0,0 +2012,20585,TOR,1,1,2,0,0,4 +2012,20585,WPG,0,0,0,0,0,0 +2012,20586,FLA,1,0,1,0,0,2 +2012,20586,NYR,1,0,1,1,0,3 +2012,20587,CHI,1,1,2,0,0,4 +2012,20587,PHI,1,3,1,0,0,5 +2012,20588,OTT,1,1,2,0,0,4 +2012,20588,TBL,0,0,1,0,0,1 +2012,20589,EDM,0,3,0,0,0,3 +2012,20589,STL,1,0,3,0,0,4 +2012,20590,DAL,2,1,1,0,0,4 +2012,20590,NSH,1,0,0,0,0,1 +2012,20591,LAK,0,0,0,1,0,1 +2012,20591,PHX,0,0,0,0,0,0 +2012,20592,CBJ,1,0,0,0,0,1 +2012,20592,SJS,1,1,0,0,0,2 +2012,20593,FLA,1,1,0,0,0,2 +2012,20593,NJD,1,1,3,0,0,5 +2012,20594,NYR,1,1,1,0,0,3 +2012,20594,PIT,1,0,0,0,0,1 +2012,20595,BUF,0,1,1,0,0,2 +2012,20595,CAR,0,1,3,0,0,4 +2012,20596,CHI,0,0,0,0,0,0 +2012,20596,COL,1,2,1,0,0,4 +2012,20597,ANA,0,1,3,0,0,4 +2012,20597,NYI,0,1,1,0,0,2 +2012,20598,BOS,1,1,1,0,0,3 +2012,20598,VAN,1,2,1,0,0,4 +2012,20599,OTT,0,1,1,0,0,2 +2012,20599,PHI,1,0,1,1,0,3 +2012,20600,DAL,0,1,3,0,0,4 +2012,20600,EDM,1,0,0,0,0,1 +2012,20601,CBJ,0,1,0,0,0,1 +2012,20601,LAK,0,0,0,0,0,0 +2012,20602,BUF,0,1,0,0,0,1 +2012,20602,WPG,0,1,0,1,0,2 +2012,20603,DET,0,2,1,0,0,3 +2012,20603,TOR,3,0,1,0,0,4 +2012,20604,MTL,0,2,1,0,0,3 +2012,20604,TBL,1,0,0,0,0,1 +2012,20605,NJD,0,2,1,0,0,3 +2012,20605,PIT,1,0,0,0,0,1 +2012,20606,COL,0,0,0,0,0,0 +2012,20606,STL,2,2,0,0,0,4 +2012,20607,CAR,0,0,2,0,0,2 +2012,20607,NSH,2,1,2,0,0,5 +2012,20608,NYI,1,0,0,0,0,1 +2012,20608,PHX,1,3,1,0,0,5 +2012,20609,CGY,0,0,3,0,0,3 +2012,20609,MIN,0,0,1,0,0,1 +2012,20610,SJS,0,2,3,0,0,5 +2012,20610,WSH,0,1,1,0,0,2 +2012,20611,OTT,2,0,4,0,0,6 +2012,20611,PHI,3,1,0,0,0,4 +2012,20612,CHI,2,0,0,0,0,2 +2012,20612,DET,1,0,1,1,0,3 +2012,20613,ANA,4,1,2,0,0,7 +2012,20613,CBJ,1,1,2,0,0,4 +2012,20614,FLA,0,2,0,0,0,2 +2012,20614,VAN,1,0,0,0,0,1 +2012,20615,LAK,2,2,1,0,0,5 +2012,20615,WSH,1,0,1,0,0,2 +2012,20616,BOS,1,1,3,0,0,5 +2012,20616,WPG,1,2,0,0,0,3 +2012,20617,BUF,0,0,0,0,0,0 +2012,20617,TOR,2,0,0,0,0,2 +2012,20618,DET,0,0,1,0,0,1 +2012,20618,NYI,1,3,1,0,0,5 +2012,20619,NYR,0,0,1,0,1,2 +2012,20619,PHX,0,0,1,0,0,1 +2012,20620,OTT,2,2,1,0,0,5 +2012,20620,PIT,0,1,0,0,0,1 +2012,20621,CAR,0,1,0,0,0,1 +2012,20621,PHI,0,1,1,0,0,2 +2012,20622,MTL,0,0,0,0,0,0 +2012,20622,STL,0,2,1,0,0,3 +2012,20623,TBL,0,2,2,0,0,4 +2012,20623,VAN,2,2,0,0,1,5 +2012,20624,MIN,1,1,2,0,1,5 +2012,20624,SJS,1,0,3,0,0,4 +2012,20625,CHI,0,2,3,0,0,5 +2012,20625,CBJ,0,1,1,0,0,2 +2012,20626,COL,0,1,0,0,0,1 +2012,20626,NSH,3,0,1,0,0,4 +2012,20627,CGY,4,1,1,0,0,6 +2012,20627,NJD,0,2,1,0,0,3 +2012,20628,ANA,1,1,3,0,0,5 +2012,20628,DAL,0,0,2,0,0,2 +2012,20629,PIT,0,0,0,0,0,0 +2012,20629,WSH,1,0,0,0,0,1 +2012,20630,EDM,0,0,1,0,0,1 +2012,20630,NJD,0,1,0,1,0,2 +2012,20631,BOS,1,0,1,0,0,2 +2012,20631,MTL,0,0,1,0,0,1 +2012,20632,NYI,0,0,2,0,0,2 +2012,20632,PHI,1,1,1,0,0,3 +2012,20633,NYR,0,0,0,0,0,0 +2012,20633,OTT,0,1,2,0,0,3 +2012,20634,CAR,4,0,1,0,0,5 +2012,20634,TBL,1,1,0,0,0,2 +2012,20635,DET,1,0,1,0,1,3 +2012,20635,PHX,1,0,1,0,0,2 +2012,20636,STL,1,1,0,0,0,2 +2012,20636,VAN,1,1,0,1,0,3 +2012,20637,COL,2,0,0,0,0,2 +2012,20637,NSH,0,1,1,1,0,3 +2012,20638,SJS,0,1,1,0,0,2 +2012,20638,WPG,0,0,0,0,0,0 +2012,20639,CHI,1,3,1,0,0,5 +2012,20639,MIN,1,0,1,0,0,2 +2012,20640,ANA,0,0,0,0,0,0 +2012,20640,CGY,0,0,0,1,0,1 +2012,20641,DAL,2,1,1,0,1,5 +2012,20641,LAK,0,2,2,0,0,4 +2012,20642,TBL,0,1,2,0,0,3 +2012,20642,WSH,2,1,1,0,0,4 +2012,20643,CBJ,2,2,0,0,0,4 +2012,20643,PHX,1,1,1,0,0,3 +2012,20644,BUF,2,1,0,0,0,3 +2012,20644,TOR,2,0,0,0,0,2 +2012,20645,FLA,0,0,1,0,0,1 +2012,20645,PIT,0,2,2,0,0,4 +2012,20646,ANA,1,3,1,0,0,5 +2012,20646,EDM,0,0,0,0,0,0 +2012,20647,CHI,0,1,1,0,0,2 +2012,20647,DET,2,0,0,1,0,3 +2012,20648,COL,1,0,1,0,0,2 +2012,20648,DAL,0,0,1,0,0,1 +2012,20649,NYR,0,2,1,0,0,3 +2012,20649,TOR,0,0,0,0,0,0 +2012,20650,MTL,0,0,2,0,0,2 +2012,20650,OTT,0,0,2,0,1,3 +2012,20651,BUF,0,2,0,0,0,2 +2012,20651,NYI,1,1,2,0,0,4 +2012,20652,BOS,0,1,1,0,0,2 +2012,20652,CAR,0,1,3,0,0,4 +2012,20653,NJD,0,0,2,0,0,2 +2012,20653,WPG,1,0,0,0,0,1 +2012,20654,CBJ,0,0,1,0,0,1 +2012,20654,SJS,0,1,1,0,0,2 +2012,20655,MIN,1,1,0,0,0,2 +2012,20655,STL,0,2,0,0,1,3 +2012,20656,NSH,1,1,2,0,0,4 +2012,20656,PHI,0,1,1,0,0,2 +2012,20657,CGY,0,1,0,0,0,1 +2012,20657,LAK,0,3,1,0,0,4 +2012,20658,PIT,2,0,4,0,0,6 +2012,20658,TBL,0,0,3,0,0,3 +2012,20659,CAR,0,1,0,0,0,1 +2012,20659,WSH,0,1,1,0,0,2 +2012,20660,MTL,1,3,0,0,0,4 +2012,20660,NYR,1,0,0,0,0,1 +2012,20661,CHI,2,1,1,0,0,4 +2012,20661,SJS,0,2,1,0,0,3 +2012,20662,EDM,1,0,0,1,0,2 +2012,20662,LAK,0,0,1,0,0,1 +2012,20663,ANA,1,2,1,0,0,4 +2012,20663,VAN,1,0,1,0,0,2 +2012,20664,NSH,3,0,0,0,0,3 +2012,20664,NYI,0,0,1,0,0,1 +2012,20665,COL,1,0,0,0,0,1 +2012,20665,PHX,2,2,2,0,0,6 +2012,20666,OTT,0,0,0,0,0,0 +2012,20666,WPG,1,1,0,0,0,2 +2012,20667,BOS,1,1,0,0,1,3 +2012,20667,FLA,0,2,0,0,0,2 +2012,20668,BUF,0,0,0,0,0,0 +2012,20668,DET,3,2,0,0,0,5 +2012,20669,DAL,0,0,0,0,0,0 +2012,20669,STL,0,0,1,0,0,1 +2012,20670,OTT,1,1,1,0,0,3 +2012,20670,TOR,2,0,0,0,0,2 +2012,20671,NJD,1,3,1,0,0,5 +2012,20671,WPG,0,0,1,0,0,1 +2012,20672,NSH,0,0,0,0,0,0 +2012,20672,NYR,1,1,1,0,0,3 +2012,20673,MIN,0,1,0,0,0,1 +2012,20673,PHI,1,3,1,0,0,5 +2012,20674,CAR,1,0,0,0,0,1 +2012,20674,PIT,1,0,0,0,1,2 +2012,20675,NYI,1,1,1,0,0,3 +2012,20675,WSH,0,0,0,0,0,0 +2012,20676,CBJ,0,2,2,0,0,4 +2012,20676,EDM,2,0,0,0,0,2 +2012,20677,BOS,0,2,1,0,0,3 +2012,20677,TBL,1,1,3,0,0,5 +2012,20678,DAL,2,0,0,0,0,2 +2012,20678,DET,1,1,0,0,1,3 +2012,20679,LAK,1,1,0,0,1,3 +2012,20679,VAN,0,1,1,0,0,2 +2012,20680,CGY,0,1,0,0,0,1 +2012,20680,SJS,0,0,1,0,1,2 +2012,20681,MTL,0,0,0,0,0,0 +2012,20681,WSH,2,1,0,0,0,3 +2012,20682,BUF,1,1,0,0,0,2 +2012,20682,CHI,2,2,2,0,0,6 +2012,20683,COL,0,1,2,1,0,4 +2012,20683,FLA,1,1,1,0,0,3 +2012,20684,ANA,2,2,2,0,0,6 +2012,20684,PHX,1,1,0,0,0,2 +2012,20685,MIN,0,0,1,0,0,1 +2012,20685,TOR,2,1,1,0,0,4 +2012,20686,BOS,0,0,4,0,0,4 +2012,20686,NJD,1,0,0,0,0,1 +2012,20687,NYR,0,1,0,0,0,1 +2012,20687,PIT,1,0,3,0,0,4 +2012,20688,NYI,0,2,2,0,0,4 +2012,20688,PHI,0,0,1,0,0,1 +2012,20689,CBJ,0,0,0,0,0,0 +2012,20689,NSH,0,2,1,0,0,3 +2012,20690,EDM,0,0,0,0,0,0 +2012,20690,STL,0,0,1,0,0,1 +2012,20691,BUF,0,1,0,0,0,1 +2012,20691,WPG,1,1,2,0,0,4 +2012,20692,DET,0,2,0,0,1,3 +2012,20692,PHX,1,1,0,0,0,2 +2012,20693,CGY,0,1,0,0,1,2 +2012,20693,LAK,1,0,0,0,0,1 +2012,20694,OTT,1,2,1,0,0,4 +2012,20694,SJS,1,0,0,0,0,1 +2012,20695,MTL,2,2,0,0,0,4 +2012,20695,PIT,1,1,2,0,1,5 +2012,20696,CAR,1,1,1,0,0,3 +2012,20696,WSH,0,0,0,0,0,0 +2012,20697,CHI,2,1,0,0,0,3 +2012,20697,FLA,0,0,1,0,0,1 +2012,20698,DAL,0,0,1,0,0,1 +2012,20698,TBL,1,0,1,0,0,2 +2012,20699,BOS,0,2,0,0,0,2 +2012,20699,NYR,0,2,0,1,0,3 +2012,20700,NJD,0,1,0,0,0,1 +2012,20700,PHI,0,2,2,0,0,4 +2012,20701,SJS,1,1,1,0,0,3 +2012,20701,VAN,1,1,2,0,0,4 +2012,20702,MTL,1,0,2,0,0,3 +2012,20702,TOR,1,0,0,0,0,1 +2012,20703,CAR,1,0,0,0,0,1 +2012,20703,NYI,0,1,0,1,0,2 +2012,20704,CBJ,1,1,0,0,0,2 +2012,20704,DET,1,1,0,0,1,3 +2012,20705,ANA,0,2,0,0,0,2 +2012,20705,OTT,0,0,1,0,0,1 +2012,20706,FLA,2,0,1,0,1,4 +2012,20706,WPG,0,1,2,0,0,3 +2012,20707,BUF,1,0,1,0,0,2 +2012,20707,STL,0,2,2,0,0,4 +2012,20708,CHI,1,0,1,0,0,2 +2012,20708,NSH,1,3,1,0,0,5 +2012,20709,PHX,0,1,2,0,0,3 +2012,20709,TBL,2,2,0,0,0,4 +2012,20710,DAL,1,1,0,0,0,2 +2012,20710,MIN,0,4,1,0,0,5 +2012,20711,CGY,3,1,2,0,0,6 +2012,20711,EDM,0,0,2,0,0,2 +2012,20712,COL,0,2,1,0,0,3 +2012,20712,LAK,0,1,0,0,0,1 +2012,20713,BOS,3,0,2,0,1,6 +2012,20713,PHI,1,3,1,0,0,5 +2012,20714,PIT,2,0,1,1,0,4 +2012,20714,WSH,0,2,1,0,0,3 +2012,20715,ANA,1,0,2,0,0,3 +2012,20715,COL,0,0,2,0,0,2 +2012,20716,NYI,0,0,0,0,0,0 +2012,20716,TOR,1,0,2,0,0,3 +2012,20717,CAR,2,0,0,0,0,2 +2012,20717,WPG,0,1,0,0,0,1 +2012,20718,DET,0,2,1,0,0,3 +2012,20718,STL,1,0,0,0,0,1 +2012,20719,CBJ,0,1,0,0,0,1 +2012,20719,NSH,1,1,2,0,0,4 +2012,20720,EDM,0,1,0,0,1,2 +2012,20720,SJS,0,0,1,0,0,1 +2012,20721,LAK,1,3,0,0,0,4 +2012,20721,OTT,0,0,1,0,0,1 +2012,20722,BUF,1,0,0,0,1,2 +2012,20722,NJD,0,1,0,0,0,1 +2012,20723,NYI,1,1,1,0,0,3 +2012,20723,TOR,0,1,2,1,0,4 +2012,20724,NYR,1,1,1,0,0,3 +2012,20724,WPG,0,0,0,0,0,0 +2012,20725,BOS,1,2,0,0,0,3 +2012,20725,WSH,0,3,2,0,0,5 +2012,20726,CBJ,1,0,1,0,0,2 +2012,20726,TBL,1,2,1,0,0,4 +2012,20727,FLA,1,1,0,0,0,2 +2012,20727,PHI,2,0,0,0,1,3 +2012,20728,PIT,0,2,0,0,1,3 +2012,20728,STL,0,1,1,0,0,2 +2012,20729,CHI,0,0,1,0,0,1 +2012,20729,NSH,2,0,1,0,0,3 +2012,20730,ANA,0,0,0,0,0,0 +2012,20730,DAL,1,0,0,0,0,1 +2012,20731,COL,1,1,0,0,0,2 +2012,20731,MIN,2,0,1,0,0,3 +2012,20732,CGY,0,0,0,0,0,0 +2012,20732,SJS,0,0,1,0,0,1 +2012,20733,OTT,0,1,1,0,0,2 +2012,20733,PHX,1,1,1,0,0,3 +2012,20734,EDM,0,1,1,0,0,2 +2012,20734,VAN,1,0,1,0,1,3 +2012,20735,DET,0,0,2,0,0,2 +2012,20735,MTL,4,2,1,0,0,7 +2012,20736,BOS,1,1,2,0,0,4 +2012,20736,OTT,1,2,0,0,0,3 +2012,20737,NJD,0,1,2,0,1,4 +2012,20737,NYR,1,0,2,0,0,3 +2012,20738,PHI,1,0,0,0,0,1 +2012,20738,WPG,0,1,0,0,1,2 +2012,20739,PIT,0,1,3,0,1,5 +2012,20739,TOR,0,2,2,0,0,4 +2012,20740,CAR,0,0,2,0,0,2 +2012,20740,NYI,1,1,3,0,0,5 +2012,20741,BUF,0,1,2,0,0,3 +2012,20741,MTL,1,0,0,0,0,1 +2012,20742,TBL,1,2,0,1,0,4 +2012,20742,WSH,1,1,1,0,0,3 +2012,20743,MIN,2,1,1,0,0,4 +2012,20743,NSH,0,1,4,0,0,5 +2012,20744,CGY,0,1,0,0,0,1 +2012,20744,DET,1,0,2,0,0,3 +2012,20745,ANA,0,3,1,0,0,4 +2012,20745,PHX,1,0,0,0,0,1 +2012,20746,COL,0,1,1,0,0,2 +2012,20746,EDM,2,0,1,0,0,3 +2012,20747,CHI,1,0,1,0,0,2 +2012,20747,VAN,1,0,1,1,0,3 +2012,20748,CBJ,0,0,0,0,0,0 +2012,20748,SJS,2,2,2,0,0,6 +2012,20749,BUF,0,0,0,0,0,0 +2012,20749,NYR,0,0,0,0,1,1 +2012,20750,PIT,0,0,0,0,0,0 +2012,20750,TOR,0,0,1,0,0,1 +2012,20751,FLA,0,1,3,0,0,4 +2012,20751,WSH,0,1,1,0,0,2 +2012,20752,ANA,1,1,0,0,0,2 +2012,20752,DAL,3,1,2,0,0,6 +2012,20753,CBJ,0,1,1,0,0,2 +2012,20753,LAK,1,1,1,0,0,3 +2012,20754,BOS,0,0,0,0,0,0 +2012,20754,CAR,1,1,1,0,0,3 +2012,20755,MTL,2,1,0,0,0,3 +2012,20755,NJD,1,1,3,0,0,5 +2012,20756,NSH,0,0,1,0,0,1 +2012,20756,PHI,1,1,2,0,0,4 +2012,20757,TBL,0,0,1,0,0,1 +2012,20757,WPG,0,1,0,1,0,2 +2012,20758,COL,0,0,0,0,0,0 +2012,20758,MIN,1,0,0,0,0,1 +2012,20759,CHI,1,2,1,0,0,4 +2012,20759,EDM,0,3,5,0,0,8 +2012,20760,DET,1,1,1,0,1,4 +2012,20760,VAN,1,0,2,0,0,3 +2012,20761,DAL,0,1,1,0,0,2 +2012,20761,SJS,2,2,1,0,0,5 +2012,20762,NYI,0,0,1,1,0,2 +2012,20762,OTT,0,1,0,0,0,1 +2012,20763,FLA,1,1,0,0,0,2 +2012,20763,WPG,0,1,0,0,0,1 +2012,20764,LAK,0,0,0,0,0,0 +2012,20764,STL,0,1,0,0,0,1 +2012,20765,CGY,1,1,1,0,0,3 +2012,20765,CHI,1,0,0,0,0,1 +2012,20766,ANA,1,1,0,0,0,2 +2012,20766,CBJ,0,1,1,1,0,3 +2012,20767,BOS,0,0,1,0,0,1 +2012,20767,PIT,1,0,1,0,0,2 +2012,20768,NJD,3,3,0,0,0,6 +2012,20768,PHI,0,0,4,0,0,4 +2012,20769,MTL,0,0,0,0,0,0 +2012,20769,WSH,1,0,2,0,0,3 +2012,20770,COL,1,1,0,0,0,2 +2012,20770,VAN,1,0,1,0,1,3 +2012,20771,OTT,0,0,0,0,0,0 +2012,20771,TOR,2,1,2,0,0,5 +2012,20772,BUF,1,1,1,0,1,4 +2012,20772,NYI,3,0,0,0,0,3 +2012,20773,CAR,0,1,1,0,0,2 +2012,20773,LAK,1,0,0,0,0,1 +2012,20774,FLA,1,2,0,0,0,3 +2012,20774,TBL,1,4,1,0,0,6 +2012,20775,NSH,2,0,1,0,0,3 +2012,20775,STL,0,0,1,0,0,1 +2012,20776,DAL,0,1,0,0,1,2 +2012,20776,MIN,0,0,1,0,0,1 +2012,20777,PHX,0,3,2,0,0,5 +2012,20777,SJS,0,2,1,0,0,3 +2012,20778,DET,1,0,3,0,0,4 +2012,20778,EDM,3,0,1,0,1,5 +2012,20779,BOS,2,0,2,0,0,4 +2012,20779,WSH,0,0,1,0,0,1 +2012,20780,NJD,2,2,1,0,0,5 +2012,20780,PIT,0,1,1,0,0,2 +2012,20781,NYR,1,1,3,0,0,5 +2012,20781,PHI,0,1,1,0,0,2 +2012,20782,MTL,1,2,0,0,0,3 +2012,20782,WPG,0,0,0,0,0,0 +2012,20783,EDM,2,1,0,0,0,3 +2012,20783,TOR,2,3,1,0,0,6 +2012,20784,DET,0,1,0,0,0,1 +2012,20784,PHX,1,1,1,0,0,3 +2012,20785,ANA,2,0,0,0,1,3 +2012,20785,CGY,0,1,1,0,0,2 +2012,20786,NJD,1,0,0,0,0,1 +2012,20786,NYR,0,0,0,0,0,0 +2012,20787,NYI,0,0,0,0,1,1 +2012,20787,PHI,0,0,0,0,0,0 +2012,20788,FLA,0,0,0,0,0,0 +2012,20788,WSH,2,2,0,0,0,4 +2012,20789,CBJ,1,1,1,0,0,3 +2012,20789,MIN,1,0,0,0,0,1 +2012,20790,MTL,0,1,1,0,1,3 +2012,20790,PIT,0,0,2,0,0,2 +2012,20791,OTT,0,1,0,0,0,1 +2012,20791,STL,2,1,0,0,0,3 +2012,20792,LAK,1,2,0,0,0,3 +2012,20792,TBL,0,1,0,0,0,1 +2012,20793,NSH,1,2,0,0,0,3 +2012,20793,VAN,3,0,0,0,1,4 +2012,20794,TOR,1,0,0,0,0,1 +2012,20794,WPG,1,1,0,0,0,2 +2012,20795,DAL,0,1,0,0,0,1 +2012,20795,PHX,1,1,2,0,0,4 +2012,20796,CHI,0,2,0,0,0,2 +2012,20796,COL,0,2,3,0,0,5 +2012,20797,BOS,0,0,0,0,0,0 +2012,20797,BUF,2,2,2,0,0,6 +2012,20798,DET,1,1,2,0,0,4 +2012,20798,EDM,0,1,1,0,0,2 +2012,20799,ANA,0,1,1,1,0,3 +2012,20799,CAR,1,0,1,0,0,2 +2012,20800,CGY,1,2,1,0,0,4 +2012,20800,SJS,0,2,1,0,0,3 +2012,20801,NJD,1,2,0,0,0,3 +2012,20801,STL,1,1,1,0,1,4 +2012,20802,MTL,1,1,2,0,0,4 +2012,20802,NYI,0,0,2,0,0,2 +2012,20803,NYR,0,2,1,1,0,4 +2012,20803,TBL,1,2,0,0,0,3 +2012,20804,PHI,1,3,0,0,0,4 +2012,20804,TOR,0,3,0,0,0,3 +2012,20805,WSH,0,0,2,0,0,2 +2012,20805,WPG,0,0,2,0,1,3 +2012,20806,CBJ,0,2,0,0,0,2 +2012,20806,DAL,2,1,1,0,0,4 +2012,20807,NSH,1,1,1,0,0,3 +2012,20807,OTT,1,2,1,0,0,4 +2012,20808,FLA,0,2,1,0,0,3 +2012,20808,LAK,0,0,1,0,0,1 +2012,20809,MIN,1,0,1,0,0,2 +2012,20809,VAN,2,1,2,0,0,5 +2012,20810,CGY,0,0,1,0,0,1 +2012,20810,PHX,1,0,0,1,0,2 +2012,20811,PIT,2,3,3,0,0,8 +2012,20811,WPG,2,1,2,0,0,5 +2012,20812,BUF,0,0,2,0,1,3 +2012,20812,DAL,1,0,1,0,0,2 +2012,20813,ANA,1,0,0,0,0,1 +2012,20813,DET,1,0,0,0,1,2 +2012,20814,CAR,2,1,0,0,0,3 +2012,20814,COL,1,2,0,1,0,4 +2012,20815,CHI,0,2,1,0,0,3 +2012,20815,SJS,2,1,2,0,0,5 +2012,20816,BOS,1,0,2,0,1,4 +2012,20816,NSH,0,1,2,0,0,3 +2012,20817,FLA,1,1,1,0,0,3 +2012,20817,NJD,1,0,0,0,0,1 +2012,20818,LAK,0,1,0,0,0,1 +2012,20818,NYI,1,0,0,1,0,2 +2012,20819,NYR,2,1,2,0,0,5 +2012,20819,PHI,1,1,0,0,0,2 +2012,20820,EDM,0,2,1,1,0,4 +2012,20820,OTT,1,1,1,0,0,3 +2012,20821,BUF,0,0,1,0,0,1 +2012,20821,TBL,0,2,0,0,0,2 +2012,20822,MTL,0,4,1,0,0,5 +2012,20822,TOR,0,0,0,0,0,0 +2012,20823,COL,2,0,0,0,0,2 +2012,20823,STL,2,0,0,1,0,3 +2012,20824,CBJ,1,1,1,0,0,3 +2012,20824,MIN,1,0,0,0,0,1 +2012,20825,CHI,0,0,0,0,0,0 +2012,20825,PHX,2,1,0,0,0,3 +2012,20826,CGY,1,1,0,0,1,3 +2012,20826,VAN,0,1,1,0,0,2 +2012,20827,FLA,1,3,0,0,0,4 +2012,20827,NYI,1,0,0,0,0,1 +2012,20828,NYR,1,1,1,0,0,3 +2012,20828,WSH,0,1,1,0,0,2 +2012,20829,DET,1,2,1,0,0,4 +2012,20829,PHI,1,2,0,0,0,3 +2012,20830,ANA,2,3,0,0,0,5 +2012,20830,CBJ,1,1,1,0,0,3 +2012,20831,DAL,0,1,1,0,0,2 +2012,20831,LAK,1,1,2,0,0,4 +2012,20832,PIT,1,3,0,0,0,4 +2012,20832,TBL,2,0,0,0,0,2 +2012,20833,SJS,0,0,0,0,0,0 +2012,20833,STL,1,1,1,0,0,3 +2012,20834,SJS,1,2,2,0,0,5 +2012,20834,WSH,0,1,2,0,0,3 +2012,20835,CAR,1,1,3,0,0,5 +2012,20835,MTL,0,3,0,0,0,3 +2012,20836,PHX,0,0,1,0,0,1 +2012,20836,VAN,0,1,0,0,1,2 +2012,20837,BOS,0,0,0,0,0,0 +2012,20837,NYR,2,0,1,0,0,3 +2012,20838,BUF,0,1,0,0,0,1 +2012,20838,NJD,1,0,3,0,0,4 +2012,20839,CBJ,1,0,1,0,0,2 +2012,20839,STL,1,0,0,0,0,1 +2012,20840,OTT,1,1,2,0,0,4 +2012,20840,TBL,0,0,0,0,0,0 +2012,20841,DAL,0,0,1,0,0,1 +2012,20841,DET,2,0,1,0,0,3 +2012,20842,CHI,1,0,1,0,0,2 +2012,20842,NSH,1,1,1,0,0,3 +2012,20843,ANA,0,0,2,0,0,2 +2012,20843,MIN,1,0,0,0,0,1 +2012,20844,NYI,0,1,2,0,0,3 +2012,20844,WPG,1,0,0,0,0,1 +2012,20845,CGY,2,1,2,0,0,5 +2012,20845,TOR,1,0,0,0,0,1 +2012,20846,ANA,0,1,1,0,0,2 +2012,20846,PIT,1,0,0,0,0,1 +2012,20847,BOS,1,2,0,0,1,4 +2012,20847,MTL,0,1,2,0,0,3 +2012,20848,FLA,1,0,1,0,0,2 +2012,20848,OTT,1,3,2,0,0,6 +2012,20849,EDM,1,1,1,0,0,3 +2012,20849,TOR,2,0,1,1,0,4 +2012,20850,COL,0,1,0,0,0,1 +2012,20850,VAN,1,0,2,0,0,3 +2012,20851,CHI,4,0,0,0,0,4 +2012,20851,NYR,0,1,1,0,0,2 +2012,20852,BUF,2,0,0,0,0,2 +2012,20852,PHI,0,4,3,0,0,7 +2012,20853,SJS,1,3,1,0,0,5 +2012,20853,TBL,2,1,2,1,0,6 +2012,20854,NYI,1,0,0,0,0,1 +2012,20854,STL,2,3,0,0,0,5 +2012,20855,MIN,1,1,1,0,0,3 +2012,20855,WPG,1,1,1,0,1,4 +2012,20856,CGY,1,0,1,0,0,2 +2012,20856,DAL,1,1,0,1,0,3 +2012,20857,LAK,0,0,0,0,0,0 +2012,20857,PHX,0,1,0,0,0,1 +2012,20858,ANA,0,1,1,0,0,2 +2012,20858,NJD,0,2,0,0,1,3 +2012,20859,CAR,2,0,1,0,0,3 +2012,20859,SJS,0,0,2,0,0,2 +2012,20860,BUF,2,0,1,0,0,3 +2012,20860,MTL,2,1,0,0,1,4 +2012,20861,FLA,1,0,0,0,0,1 +2012,20861,WSH,0,0,2,0,0,2 +2012,20862,DET,1,0,1,0,0,2 +2012,20862,NSH,0,1,0,0,0,1 +2012,20863,BOS,0,1,1,0,0,2 +2012,20863,WPG,0,2,2,0,0,4 +2012,20864,COL,1,2,0,0,0,3 +2012,20864,EDM,0,1,0,0,0,1 +2012,20865,PHI,2,1,1,0,0,4 +2012,20865,PIT,1,2,3,0,0,6 +2012,20866,CHI,2,2,2,0,0,6 +2012,20866,CBJ,1,0,0,0,0,1 +2012,20867,MIN,0,0,0,0,0,0 +2012,20867,STL,1,2,1,0,0,4 +2012,20868,CAR,1,2,0,0,0,3 +2012,20868,NYI,1,2,1,0,0,4 +2012,20869,TBL,1,1,0,0,0,2 +2012,20869,WSH,0,1,0,0,0,1 +2012,20870,TOR,0,1,1,0,0,2 +2012,20870,VAN,2,3,1,0,0,6 +2012,20871,DAL,0,1,0,0,0,1 +2012,20871,PHX,1,0,0,1,0,2 +2012,20872,CGY,0,0,1,0,0,1 +2012,20872,LAK,0,0,0,0,0,0 +2012,20873,BUF,2,1,3,0,0,6 +2012,20873,PIT,0,1,1,0,0,2 +2012,20874,DET,1,1,1,0,0,3 +2012,20874,SJS,0,1,1,0,0,2 +2012,20875,CHI,0,0,3,0,0,3 +2012,20875,STL,1,0,0,0,0,1 +2012,20876,BOS,0,0,0,0,0,0 +2012,20876,MIN,0,2,0,0,0,2 +2012,20877,MTL,0,0,1,0,0,1 +2012,20877,NJD,1,1,1,0,0,3 +2012,20878,ANA,1,0,1,0,0,2 +2012,20878,FLA,0,0,0,0,0,0 +2012,20879,CBJ,0,1,1,0,0,2 +2012,20879,NYR,1,1,0,1,0,3 +2012,20880,DAL,1,0,1,0,0,2 +2012,20880,NSH,2,1,0,0,0,3 +2012,20881,EDM,0,2,0,0,0,2 +2012,20881,VAN,2,2,1,0,0,5 +2012,20882,COL,0,1,0,0,0,1 +2012,20882,WPG,0,3,2,0,0,5 +2012,20883,NYI,0,0,0,0,0,0 +2012,20883,OTT,3,1,2,0,0,6 +2012,20884,CAR,3,1,1,0,0,5 +2012,20884,WSH,0,0,0,0,0,0 +2012,20885,BUF,1,1,0,0,0,2 +2012,20885,NYI,0,0,1,0,0,1 +2012,20886,NJD,2,1,0,1,0,4 +2012,20886,TOR,1,1,1,0,0,3 +2012,20887,NYR,0,0,0,0,0,0 +2012,20887,PIT,0,2,0,0,0,2 +2012,20888,CBJ,4,1,1,0,0,6 +2012,20888,SJS,0,2,1,0,0,3 +2012,20889,DAL,1,0,2,0,0,3 +2012,20889,MTL,0,0,0,0,0,0 +2012,20890,ANA,0,2,0,0,0,2 +2012,20890,TBL,2,1,0,0,0,3 +2012,20891,NSH,0,2,1,0,0,3 +2012,20891,VAN,0,1,0,0,0,1 +2012,20892,PHI,1,2,1,1,0,5 +2012,20892,WPG,1,2,1,0,0,4 +2012,20893,CHI,0,1,1,0,0,2 +2012,20893,DET,1,0,0,0,0,1 +2012,20894,CGY,1,0,0,0,0,1 +2012,20894,EDM,1,4,1,0,0,6 +2012,20895,LAK,3,1,0,0,0,4 +2012,20895,PHX,0,3,1,0,1,5 +2012,20896,OTT,2,2,1,0,0,5 +2012,20896,WSH,0,0,2,0,0,2 +2012,20897,BOS,3,0,1,0,0,4 +2012,20897,STL,2,0,0,0,0,2 +2012,20898,COL,3,1,0,0,0,4 +2012,20898,LAK,0,0,1,0,0,1 +2012,20899,SJS,0,2,0,0,0,2 +2012,20899,TOR,0,0,1,0,0,1 +2012,20900,ANA,1,0,1,0,1,3 +2012,20900,CAR,1,1,0,0,0,2 +2012,20901,FLA,0,1,1,0,0,2 +2012,20901,MIN,0,1,1,0,1,3 +2012,20902,DET,1,0,2,0,0,3 +2012,20902,VAN,0,1,2,0,1,4 +2012,20903,NSH,0,2,0,0,0,2 +2012,20903,STL,1,1,0,0,1,3 +2012,20904,TBL,0,0,3,0,0,3 +2012,20904,WPG,2,2,0,0,0,4 +2012,20905,CHI,0,1,0,0,0,1 +2012,20905,DAL,0,0,3,0,0,3 +2012,20906,CGY,2,0,1,0,0,3 +2012,20906,PHX,0,2,1,0,1,4 +2012,20907,EDM,0,2,0,0,0,2 +2012,20907,PHI,0,0,0,0,0,0 +2012,20908,NJD,0,1,0,0,0,1 +2012,20908,VAN,1,1,0,0,0,2 +2012,20909,NYI,2,1,0,0,1,4 +2012,20909,NYR,1,2,0,0,0,3 +2012,20910,MTL,0,0,1,0,0,1 +2012,20910,WSH,1,2,1,0,0,4 +2012,20911,COL,1,1,3,0,0,5 +2012,20911,CBJ,0,0,0,0,0,0 +2012,20912,BOS,0,0,1,0,0,1 +2012,20912,BUF,0,1,0,0,1,2 +2012,20913,DAL,2,1,1,0,0,4 +2012,20913,MIN,0,0,1,0,0,1 +2012,20914,PIT,3,2,3,0,0,8 +2012,20914,TBL,0,1,0,0,0,1 +2012,20915,STL,2,0,0,0,1,3 +2012,20915,WPG,0,2,0,0,0,2 +2012,20916,EDM,0,0,1,0,0,1 +2012,20916,PHX,2,0,1,0,0,3 +2012,20917,CHI,0,0,0,0,0,0 +2012,20917,LAK,2,2,0,0,0,4 +2012,20918,TOR,0,0,2,0,0,2 +2012,20918,WSH,2,2,0,0,0,4 +2012,20919,BOS,2,1,2,0,0,5 +2012,20919,OTT,1,0,2,0,0,3 +2012,20920,BUF,1,0,1,0,0,2 +2012,20920,NYR,0,1,1,1,0,3 +2012,20921,CAR,1,1,0,0,0,2 +2012,20921,FLA,0,0,2,0,1,3 +2012,20922,COL,2,1,1,0,0,4 +2012,20922,DET,0,0,3,0,0,3 +2012,20923,NSH,3,1,2,0,0,6 +2012,20923,SJS,1,1,0,0,0,2 +2012,20924,CGY,2,1,1,0,0,4 +2012,20924,PHI,0,2,2,0,1,5 +2012,20925,NJD,1,0,2,0,0,3 +2012,20925,TBL,0,2,2,0,0,4 +2012,20926,CBJ,0,1,1,0,0,2 +2012,20926,PIT,0,1,3,0,0,4 +2012,20927,DAL,0,1,1,1,0,3 +2012,20927,VAN,0,2,0,0,0,2 +2012,20928,NYI,1,0,1,0,0,2 +2012,20928,OTT,0,2,3,0,0,5 +2012,20929,FLA,1,2,1,0,0,4 +2012,20929,MTL,2,0,0,0,0,2 +2012,20930,MIN,1,1,2,0,0,4 +2012,20930,SJS,2,0,1,0,0,3 +2012,20931,ANA,1,1,1,0,0,3 +2012,20931,CHI,1,0,0,0,0,1 +2012,20932,NJD,0,0,0,0,0,0 +2012,20932,NYR,1,0,1,0,0,2 +2012,20933,LAK,0,0,1,0,0,1 +2012,20933,NSH,1,1,0,0,0,2 +2012,20934,EDM,0,1,4,0,0,5 +2012,20934,WPG,0,2,1,0,0,3 +2012,20935,ANA,0,1,0,0,0,1 +2012,20935,COL,1,0,3,0,0,4 +2012,20936,CGY,1,0,0,0,0,1 +2012,20936,STL,2,1,0,0,0,3 +2012,20937,BOS,0,0,0,0,0,0 +2012,20937,OTT,1,0,0,0,0,1 +2012,20938,FLA,2,1,2,0,0,5 +2012,20938,TOR,0,1,2,0,0,3 +2012,20939,NYI,1,0,1,0,0,2 +2012,20939,WSH,0,0,2,1,0,3 +2012,20940,CAR,0,3,1,0,0,4 +2012,20940,NSH,1,1,1,0,0,3 +2012,20941,CBJ,1,0,1,0,0,2 +2012,20941,DET,1,0,4,0,0,5 +2012,20942,MTL,1,0,0,0,0,1 +2012,20942,TBL,1,1,0,0,0,2 +2012,20943,LAK,3,1,0,0,0,4 +2012,20943,MIN,0,0,0,0,0,0 +2012,20944,PHX,0,0,1,0,0,1 +2012,20944,VAN,0,1,0,0,1,2 +2012,20945,PHI,0,0,0,0,0,0 +2012,20945,SJS,1,0,0,0,0,1 +2012,20946,DAL,1,1,1,0,0,3 +2012,20946,PIT,1,0,2,0,1,4 +2012,20947,CHI,2,2,1,0,0,5 +2012,20947,TOR,3,0,1,0,0,4 +2012,20948,EDM,0,2,0,0,0,2 +2012,20948,STL,1,2,2,0,0,5 +2012,20949,ANA,0,0,0,0,0,0 +2012,20949,BUF,1,1,0,0,0,2 +2012,20950,BOS,2,0,1,1,0,4 +2012,20950,NJD,0,3,0,0,0,3 +2012,20951,MIN,1,0,3,0,0,4 +2012,20951,MTL,2,1,1,0,1,5 +2012,20952,NYI,1,1,1,0,0,3 +2012,20952,PHI,2,2,2,0,0,6 +2012,20953,CAR,0,1,1,0,0,2 +2012,20953,NYR,0,2,1,0,0,3 +2012,20954,FLA,0,0,0,0,0,0 +2012,20954,WPG,2,0,5,0,0,7 +2012,20955,COL,0,0,0,0,0,0 +2012,20955,CBJ,1,1,0,0,0,2 +2012,20956,CGY,3,0,1,0,0,4 +2012,20956,PHX,0,2,0,0,0,2 +2012,20957,STL,0,0,0,0,0,0 +2012,20957,VAN,0,0,2,0,0,2 +2012,20958,BUF,1,0,0,0,0,1 +2012,20958,SJS,0,0,0,0,0,0 +2012,20959,NJD,3,2,0,0,0,5 +2012,20959,WSH,0,0,0,0,0,0 +2012,20960,CHI,0,2,0,0,0,2 +2012,20960,OTT,1,0,0,0,0,1 +2012,20961,NYR,1,1,1,0,0,3 +2012,20961,TBL,0,1,2,1,0,4 +2012,20962,DET,2,3,1,0,0,6 +2012,20962,MIN,0,0,0,0,0,0 +2012,20963,DAL,0,2,1,0,0,3 +2012,20963,EDM,0,0,1,0,0,1 +2012,20964,ANA,1,1,1,0,0,3 +2012,20964,CGY,1,0,1,0,0,2 +2012,20965,BOS,1,0,1,0,0,2 +2012,20965,NYI,1,1,1,0,0,3 +2012,20966,MTL,1,0,0,0,0,1 +2012,20966,TOR,0,1,2,0,0,3 +2012,20967,CAR,0,3,0,0,0,3 +2012,20967,TBL,1,1,1,1,0,4 +2012,20968,FLA,0,1,0,0,0,1 +2012,20968,NSH,1,1,1,0,0,3 +2012,20969,CBJ,1,1,3,0,0,5 +2012,20969,PHX,0,2,0,0,0,2 +2012,20970,COL,0,0,1,0,0,1 +2012,20970,PIT,2,2,1,0,0,5 +2012,20971,BUF,3,0,2,0,0,5 +2012,20971,VAN,0,1,2,0,0,3 +2012,20972,ANA,1,1,0,0,0,2 +2012,20972,LAK,1,2,1,0,0,4 +2012,20973,SJS,1,0,0,0,0,1 +2012,20973,STL,2,1,0,0,0,3 +2012,20974,NJD,0,0,0,0,0,0 +2012,20974,NYI,0,1,0,0,0,1 +2012,20975,BOS,1,1,1,0,0,3 +2012,20975,NYR,2,0,2,0,0,4 +2012,20976,CHI,1,1,0,0,0,2 +2012,20976,DET,1,0,0,0,0,1 +2012,20977,FLA,0,3,1,0,0,4 +2012,20977,OTT,2,0,0,0,0,2 +2012,20978,CGY,0,1,1,0,0,2 +2012,20978,DAL,0,2,0,0,1,3 +2012,20979,PHI,0,1,0,0,0,1 +2012,20979,WSH,0,0,0,0,0,0 +2012,20980,COL,2,0,0,0,0,2 +2012,20980,MIN,0,0,0,0,0,0 +2012,20981,PHX,0,0,1,0,0,1 +2012,20981,PIT,2,0,0,0,0,2 +2012,20982,BUF,0,1,0,0,0,1 +2012,20982,WPG,1,0,2,0,0,3 +2012,20983,ANA,1,0,3,0,0,4 +2012,20983,EDM,0,1,1,0,0,2 +2012,20984,BOS,1,4,0,0,0,5 +2012,20984,TOR,1,2,1,0,0,4 +2012,20985,NJD,1,0,3,0,0,4 +2012,20985,NYR,0,1,0,0,0,1 +2012,20986,DET,1,0,1,0,0,2 +2012,20986,PHI,2,1,0,0,0,3 +2012,20987,CAR,0,2,1,1,0,4 +2012,20987,WSH,1,1,1,0,0,3 +2012,20988,CBJ,2,1,0,0,0,3 +2012,20988,PHX,1,1,0,0,0,2 +2012,20989,OTT,3,1,3,0,0,7 +2012,20989,TBL,0,2,1,0,0,3 +2012,20990,CHI,0,0,1,0,0,1 +2012,20990,STL,2,1,2,0,0,5 +2012,20991,LAK,2,1,2,0,0,5 +2012,20991,NSH,1,2,1,0,0,4 +2012,20992,COL,1,4,2,0,0,7 +2012,20992,MIN,0,0,1,0,0,1 +2012,20993,CGY,2,2,1,0,0,5 +2012,20993,MTL,1,1,2,0,0,4 +2012,20994,DAL,0,2,3,0,0,5 +2012,20994,VAN,0,1,1,0,0,2 +2012,20995,EDM,1,1,0,0,1,3 +2012,20995,SJS,1,1,0,0,0,2 +2012,20996,BUF,0,2,0,1,0,3 +2012,20996,CAR,1,1,0,0,0,2 +2012,20997,PIT,0,1,2,0,0,3 +2012,20997,TOR,0,2,0,0,0,2 +2012,20998,BOS,0,1,2,0,0,3 +2012,20998,BUF,1,0,0,0,0,1 +2012,20999,NJD,1,1,3,0,0,5 +2012,20999,NYI,0,1,0,0,0,1 +2012,21000,FLA,0,0,0,0,0,0 +2012,21000,PHI,1,2,2,0,0,5 +2012,21001,TBL,0,2,0,0,0,2 +2012,21001,WSH,1,0,1,1,0,3 +2012,21002,CBJ,2,1,0,0,0,3 +2012,21002,LAK,1,0,0,0,0,1 +2012,21003,NYR,1,0,0,0,0,1 +2012,21003,OTT,2,0,2,0,0,4 +2012,21004,ANA,0,1,0,0,0,1 +2012,21004,STL,0,1,2,0,0,3 +2012,21005,COL,1,0,1,0,0,2 +2012,21005,NSH,2,1,1,0,0,4 +2012,21006,DAL,0,1,2,0,1,4 +2012,21006,SJS,0,1,2,0,0,3 +2012,21007,MIN,2,0,0,0,1,3 +2012,21007,PHX,0,0,2,0,0,2 +2012,21008,EDM,1,1,1,0,0,3 +2012,21008,MTL,1,1,3,0,0,5 +2012,21009,VAN,0,1,2,0,0,3 +2012,21009,WPG,0,1,1,0,0,2 +2012,21010,FLA,0,1,0,0,0,1 +2012,21010,PIT,0,0,1,0,1,2 +2012,21011,DET,1,1,2,0,0,4 +2012,21011,LAK,2,0,1,0,0,3 +2012,21012,CHI,0,1,3,0,0,4 +2012,21012,NYR,1,0,2,0,0,3 +2012,21013,CGY,3,1,1,0,0,5 +2012,21013,WPG,2,0,1,0,0,3 +2012,21014,BOS,1,1,1,0,0,3 +2012,21014,WSH,2,2,0,0,0,4 +2012,21015,COL,0,2,0,0,1,3 +2012,21015,EDM,1,0,1,0,0,2 +2012,21016,PHI,0,0,0,0,1,1 +2012,21016,TOR,0,0,0,0,0,0 +2012,21017,BUF,0,2,1,0,1,4 +2012,21017,OTT,1,1,1,0,0,3 +2012,21018,NJD,0,0,2,0,0,2 +2012,21018,NYI,0,0,1,0,0,1 +2012,21019,CAR,2,0,2,0,0,4 +2012,21019,TBL,2,0,0,0,0,2 +2012,21020,CBJ,0,1,0,0,0,1 +2012,21020,STL,1,2,1,0,0,4 +2012,21021,DET,1,0,1,0,0,2 +2012,21021,NSH,2,0,1,0,0,3 +2012,21022,ANA,0,0,0,0,0,0 +2012,21022,DAL,0,1,1,0,0,2 +2012,21023,PHX,0,2,1,0,0,3 +2012,21023,SJS,0,0,0,0,0,0 +2012,21024,MTL,0,1,3,0,0,4 +2012,21024,VAN,0,1,0,0,0,1 +2012,21025,TOR,0,0,0,0,0,0 +2012,21025,WSH,1,0,1,0,0,2 +2012,21026,CAR,0,0,0,0,0,0 +2012,21026,FLA,0,1,1,0,0,2 +2012,21027,CBJ,0,1,0,0,0,1 +2012,21027,STL,0,1,1,0,0,2 +2012,21028,CGY,1,0,3,0,0,4 +2012,21028,MIN,0,0,3,0,0,3 +2012,21029,NJD,1,0,3,0,0,4 +2012,21029,PHI,0,0,1,0,0,1 +2012,21030,NYI,1,2,0,0,0,3 +2012,21030,NYR,1,1,1,1,0,4 +2012,21031,CHI,0,1,1,0,0,2 +2012,21031,LAK,1,1,0,0,1,3 +2012,21032,BOS,0,2,0,0,0,2 +2012,21032,PIT,3,1,1,0,0,5 +2012,21033,BUF,0,1,1,1,0,3 +2012,21033,MTL,1,0,1,0,0,2 +2012,21034,ANA,0,1,1,0,0,2 +2012,21034,COL,1,0,1,1,0,3 +2012,21035,EDM,1,1,0,0,0,2 +2012,21035,SJS,1,1,1,0,0,3 +2012,21036,NSH,1,2,1,0,1,5 +2012,21036,PHX,0,2,2,0,0,4 +2012,21037,NYI,2,2,0,0,0,4 +2012,21037,WSH,0,2,2,0,1,5 +2012,21038,CAR,0,2,0,0,0,2 +2012,21038,NYR,1,2,1,0,0,4 +2012,21039,NJD,0,0,0,0,0,0 +2012,21039,PHI,0,1,2,0,0,3 +2012,21040,BOS,0,0,1,0,0,1 +2012,21040,TBL,4,1,1,0,0,6 +2012,21041,FLA,1,1,3,0,0,5 +2012,21041,TOR,0,1,1,0,0,2 +2012,21042,DAL,0,0,1,0,0,1 +2012,21042,MIN,0,0,0,0,0,0 +2012,21043,CHI,1,1,1,0,1,4 +2012,21043,STL,3,0,0,0,0,3 +2012,21044,CGY,0,2,0,1,0,3 +2012,21044,SJS,1,0,1,0,0,2 +2012,21045,DET,0,1,1,0,0,2 +2012,21045,LAK,2,1,2,0,0,5 +2012,21046,BUF,2,1,1,0,0,4 +2012,21046,COL,2,1,1,0,1,5 +2012,21047,MTL,1,0,1,0,1,3 +2012,21047,OTT,1,0,1,0,0,2 +2012,21048,DAL,0,1,1,0,0,2 +2012,21048,WPG,1,4,0,0,0,5 +2012,21049,CBJ,0,0,0,0,0,0 +2012,21049,EDM,0,2,1,0,0,3 +2012,21050,PHX,2,2,1,0,0,5 +2012,21050,VAN,2,1,1,0,0,4 +2012,21051,ANA,0,3,1,0,0,4 +2012,21051,DET,0,0,0,0,0,0 +2012,21052,COL,0,0,0,0,0,0 +2012,21052,NJD,0,0,0,0,1,1 +2012,21053,NYI,0,0,2,0,0,2 +2012,21053,PHI,2,1,0,0,0,3 +2012,21054,NYR,1,1,0,0,0,2 +2012,21054,PIT,1,2,2,0,0,5 +2012,21055,CAR,0,0,2,0,0,2 +2012,21055,STL,0,0,0,0,0,0 +2012,21056,TBL,0,0,1,0,0,1 +2012,21056,TOR,1,2,0,0,0,3 +2012,21057,BOS,0,1,1,0,0,2 +2012,21057,FLA,1,3,2,0,0,6 +2012,21058,CGY,1,2,1,0,0,4 +2012,21058,PHX,0,1,0,0,0,1 +2012,21059,NSH,1,0,0,0,0,1 +2012,21059,SJS,0,0,1,0,1,2 +2012,21060,MTL,1,0,0,0,0,1 +2012,21060,OTT,0,0,1,1,0,2 +2012,21061,WSH,1,0,1,0,0,2 +2012,21061,WPG,1,1,1,0,0,3 +2012,21062,CHI,2,0,2,0,0,4 +2012,21062,DAL,0,0,1,0,0,1 +2012,21063,CGY,0,1,0,0,0,1 +2012,21063,EDM,2,0,1,0,0,3 +2012,21064,ANA,0,1,1,0,0,2 +2012,21064,LAK,3,0,1,0,0,4 +2012,21065,BOS,2,0,0,0,1,3 +2012,21065,PHI,0,1,1,0,0,2 +2012,21066,NJD,1,0,1,0,0,2 +2012,21066,PIT,1,3,1,0,0,5 +2012,21067,CAR,1,1,3,0,0,5 +2012,21067,MIN,1,2,0,0,0,3 +2012,21068,MTL,1,0,1,0,0,2 +2012,21068,NYI,0,2,0,0,1,3 +2012,21069,OTT,0,0,1,0,0,1 +2012,21069,TOR,0,1,2,0,0,3 +2012,21070,COL,1,1,1,0,0,3 +2012,21070,NYR,1,0,0,0,0,1 +2012,21071,STL,2,1,0,0,0,3 +2012,21071,TBL,0,0,1,0,0,1 +2012,21072,BUF,1,1,0,0,0,2 +2012,21072,FLA,2,0,0,0,1,3 +2012,21073,CBJ,1,1,1,0,0,3 +2012,21073,VAN,2,1,1,0,0,4 +2012,21074,LAK,1,0,3,0,0,4 +2012,21074,NSH,0,1,1,0,0,2 +2012,21075,DET,1,0,1,0,0,2 +2012,21075,SJS,2,0,0,1,0,3 +2012,21076,PHI,0,0,2,1,0,3 +2012,21076,PIT,1,1,0,0,0,2 +2012,21077,CHI,2,2,1,0,0,5 +2012,21077,WSH,0,2,0,0,0,2 +2012,21078,CGY,0,0,1,0,0,1 +2012,21078,CBJ,1,0,0,0,1,2 +2012,21079,EDM,0,2,0,0,0,2 +2012,21079,PHX,0,0,2,0,1,3 +2012,21080,ANA,1,0,0,0,0,1 +2012,21080,NSH,0,0,3,0,0,3 +2012,21081,CAR,1,1,2,0,0,4 +2012,21081,WPG,1,2,0,0,0,3 +2012,21082,BOS,4,3,1,0,0,8 +2012,21082,TOR,0,0,0,0,0,0 +2012,21083,NJD,0,2,0,0,0,2 +2012,21083,NYR,1,2,1,0,0,4 +2012,21084,BUF,2,4,1,0,0,7 +2012,21084,TBL,0,2,1,0,0,3 +2012,21085,DET,0,1,2,0,0,3 +2012,21085,WSH,3,1,1,0,0,5 +2012,21086,MIN,1,0,1,0,0,2 +2012,21086,VAN,0,0,0,0,0,0 +2012,21087,ANA,2,3,0,0,0,5 +2012,21087,SJS,2,1,0,0,0,3 +2012,21088,NYI,1,1,3,0,0,5 +2012,21088,TOR,1,1,0,0,0,2 +2012,21089,FLA,1,1,0,0,0,2 +2012,21089,PHI,0,1,0,0,0,1 +2012,21090,PIT,2,4,2,0,0,8 +2012,21090,WPG,2,2,0,0,0,4 +2012,21091,CHI,3,0,2,0,0,5 +2012,21091,CBJ,0,0,1,0,0,1 +2012,21092,NJD,0,1,0,0,0,1 +2012,21092,OTT,0,0,0,0,0,0 +2012,21093,EDM,2,3,1,0,0,6 +2012,21093,NSH,1,1,1,0,0,3 +2012,21094,DAL,1,2,0,0,1,4 +2012,21094,PHX,1,0,2,0,0,3 +2012,21095,CGY,1,0,0,0,0,1 +2012,21095,COL,1,0,0,1,0,2 +2012,21096,LAK,1,1,3,0,0,5 +2012,21096,SJS,1,0,1,0,0,2 +2012,21097,BUF,1,1,1,0,0,3 +2012,21097,MTL,0,0,0,0,0,0 +2012,21098,CAR,1,1,1,0,0,3 +2012,21098,FLA,0,1,0,0,0,1 +2012,21099,DET,1,0,0,0,0,1 +2012,21099,NYR,1,0,0,1,0,2 +2012,21100,CHI,0,1,0,1,0,2 +2012,21100,VAN,1,0,0,0,0,1 +2012,21101,ANA,1,1,2,0,0,4 +2012,21101,STL,2,0,1,0,0,3 +2012,21102,PHI,0,1,0,0,1,2 +2012,21102,WSH,1,0,0,0,0,1 +2012,21103,NSH,0,1,0,0,0,1 +2012,21103,PIT,2,2,1,0,0,5 +2012,21104,EDM,0,1,1,0,0,2 +2012,21104,TBL,0,2,0,0,1,3 +2012,21105,CGY,1,1,0,0,0,2 +2012,21105,MIN,0,1,1,0,1,3 +2012,21106,DAL,0,0,1,0,0,1 +2012,21106,VAN,1,0,1,0,0,2 +2012,21107,COL,0,0,2,0,0,2 +2012,21107,PHX,2,1,0,0,0,3 +2012,21108,LAK,0,0,0,0,1,1 +2012,21108,STL,0,0,0,0,0,0 +2012,21109,BOS,0,0,1,0,0,1 +2012,21109,SJS,1,0,1,0,0,2 +2012,21110,NJD,0,1,2,0,0,3 +2012,21110,TOR,0,2,1,0,1,4 +2012,21111,BUF,0,2,2,0,0,4 +2012,21111,NYR,0,1,0,0,0,1 +2012,21112,WSH,1,2,0,0,0,3 +2012,21112,WPG,0,2,1,1,0,4 +2012,21113,CAR,0,1,0,0,0,1 +2012,21113,CBJ,1,2,2,0,0,5 +2012,21114,MTL,4,0,1,0,0,5 +2012,21114,OTT,1,0,0,0,0,1 +2012,21115,EDM,0,1,0,0,1,2 +2012,21115,FLA,0,1,0,0,0,1 +2012,21116,CGY,1,0,0,0,0,1 +2012,21116,DAL,2,0,2,0,0,4 +2012,21117,BOS,0,1,3,0,0,4 +2012,21117,LAK,0,1,1,0,0,2 +2012,21118,BUF,0,1,2,0,0,3 +2012,21118,MIN,1,0,0,0,0,1 +2012,21119,NYR,1,1,1,0,1,4 +2012,21119,TOR,1,1,1,0,0,3 +2012,21120,OTT,1,4,3,0,0,8 +2012,21120,PIT,2,1,1,0,0,4 +2012,21121,MTL,0,1,0,0,0,1 +2012,21121,PHI,1,2,1,0,0,4 +2012,21122,NYI,0,2,1,0,0,3 +2012,21122,TBL,1,1,2,0,0,4 +2012,21123,CAR,1,3,0,0,0,4 +2012,21123,DET,0,3,2,0,0,5 +2012,21124,NSH,0,2,1,0,0,3 +2012,21124,WPG,0,0,1,0,0,1 +2012,21125,COL,1,1,0,0,0,2 +2012,21125,VAN,0,2,0,1,0,3 +2012,21126,PHX,2,1,0,0,0,3 +2012,21126,SJS,2,1,0,0,1,4 +2012,21127,CBJ,1,1,1,0,0,3 +2012,21127,EDM,0,5,1,0,0,6 +2012,21128,MIN,0,0,0,0,0,0 +2012,21128,WSH,0,2,1,0,0,3 +2012,21129,FLA,1,0,1,0,0,2 +2012,21129,NYI,0,1,1,0,1,3 +2012,21130,NJD,1,0,1,0,0,2 +2012,21130,PIT,2,1,2,0,0,5 +2012,21131,CHI,0,0,1,0,0,1 +2012,21131,NSH,2,1,3,0,0,6 +2012,21132,ANA,0,1,1,0,0,2 +2012,21132,BOS,0,2,1,0,0,3 +2012,21133,PHX,0,0,0,0,0,0 +2012,21133,STL,0,1,3,0,0,4 +2012,21134,PHI,0,2,1,0,0,3 +2012,21134,TBL,2,0,3,0,0,5 +2012,21135,CBJ,0,0,2,0,0,2 +2012,21135,DET,4,2,1,0,0,7 +2012,21136,OTT,1,2,3,0,0,6 +2012,21136,WPG,0,3,1,0,0,4 +2012,21137,CGY,1,4,0,0,0,5 +2012,21137,DAL,1,2,1,0,0,4 +2012,21138,LAK,0,0,0,0,0,0 +2012,21138,VAN,1,0,0,0,0,1 +2012,21139,COL,0,1,0,0,0,1 +2012,21139,SJS,0,3,2,0,0,5 +2012,21140,BOS,1,1,3,0,0,5 +2012,21140,TBL,1,0,1,0,0,2 +2012,21141,CAR,2,1,0,0,0,3 +2012,21141,TOR,0,0,0,0,0,0 +2012,21142,CHI,0,0,1,0,0,1 +2012,21142,NJD,1,0,0,0,1,2 +2012,21143,NYI,2,3,0,0,0,5 +2012,21143,PIT,0,2,1,0,0,3 +2012,21144,BUF,2,2,1,0,0,5 +2012,21144,WSH,0,1,0,0,0,1 +2012,21145,FLA,1,0,1,0,1,3 +2012,21145,MTL,1,1,0,0,0,2 +2012,21146,NSH,0,0,0,0,0,0 +2012,21146,STL,0,1,2,0,0,3 +2012,21147,MIN,0,2,0,0,0,2 +2012,21147,NYR,1,1,1,0,0,3 +2012,21148,NYR,0,2,2,0,0,4 +2012,21148,WPG,0,2,0,0,0,2 +2012,21149,CBJ,2,2,0,0,0,4 +2012,21149,DET,1,0,1,0,0,2 +2012,21150,CGY,0,0,0,0,0,0 +2012,21150,LAK,1,1,1,0,0,3 +2012,21151,DAL,1,1,1,0,0,3 +2012,21151,EDM,0,0,1,0,0,1 +2012,21152,COL,0,0,0,0,0,0 +2012,21152,VAN,1,0,0,0,0,1 +2012,21153,ANA,0,1,2,0,0,3 +2012,21153,SJS,0,1,0,0,0,1 +2012,21154,BOS,0,0,2,0,0,2 +2012,21154,WSH,0,0,2,0,1,3 +2012,21155,PHI,2,2,3,0,0,7 +2012,21155,TOR,1,0,0,0,0,1 +2012,21156,NJD,0,4,2,0,0,6 +2012,21156,TBL,1,2,1,0,0,4 +2012,21157,NYI,1,2,2,0,0,5 +2012,21157,PIT,0,1,2,0,0,3 +2012,21158,FLA,0,1,1,0,0,2 +2012,21158,MIN,0,0,2,1,0,3 +2012,21159,CHI,1,0,2,0,1,4 +2012,21159,STL,1,0,2,0,0,3 +2012,21160,PHX,0,1,1,0,0,2 +2012,21160,SJS,0,0,0,0,0,0 +2012,21161,MTL,0,0,1,0,0,1 +2012,21161,NYR,1,1,2,0,0,4 +2012,21162,CAR,2,1,0,0,0,3 +2012,21162,WPG,1,1,1,1,0,4 +2012,21163,CBJ,1,1,2,0,0,4 +2012,21163,FLA,0,0,1,0,0,1 +2012,21164,BUF,2,1,0,0,0,3 +2012,21164,PIT,2,2,1,0,0,5 +2012,21165,DET,0,0,1,0,0,1 +2012,21165,NSH,0,2,2,0,0,4 +2012,21166,CGY,0,0,1,0,0,1 +2012,21166,COL,2,1,1,0,0,4 +2012,21167,EDM,1,0,0,0,0,1 +2012,21167,LAK,2,1,1,0,0,4 +2012,21168,DAL,0,1,1,0,0,2 +2012,21168,VAN,1,2,2,0,0,5 +2012,21169,BOS,1,1,4,0,0,6 +2012,21169,NYI,1,1,1,0,0,3 +2012,21170,OTT,3,0,0,0,1,4 +2012,21170,PHI,0,3,0,0,0,3 +2012,21171,BUF,0,1,2,0,0,3 +2012,21171,TOR,1,2,1,0,0,4 +2012,21172,MTL,1,1,0,0,0,2 +2012,21172,WSH,2,0,0,0,1,3 +2012,21173,CAR,0,0,0,0,0,0 +2012,21173,NJD,1,3,1,0,0,5 +2012,21174,TBL,1,0,1,1,0,3 +2012,21174,WPG,0,2,0,0,0,2 +2012,21175,CBJ,2,1,2,0,0,5 +2012,21175,STL,1,1,0,0,0,2 +2012,21176,CHI,1,3,1,0,0,5 +2012,21176,NSH,0,2,2,0,0,4 +2012,21177,LAK,2,1,0,0,0,3 +2012,21177,MIN,2,0,1,0,1,4 +2012,21178,ANA,0,0,0,0,0,0 +2012,21178,PHX,1,3,0,0,0,4 +2012,21179,CGY,0,1,1,0,0,2 +2012,21179,VAN,1,0,1,1,0,3 +2012,21180,DAL,0,0,0,0,0,0 +2012,21180,SJS,2,1,0,0,0,3 +2012,21181,NYI,1,0,0,0,0,1 +2012,21181,OTT,1,3,1,0,0,5 +2012,21182,BOS,0,2,0,0,0,2 +2012,21182,NYR,1,0,0,0,0,1 +2012,21183,DET,0,0,1,0,1,2 +2012,21183,FLA,1,0,0,0,0,1 +2012,21184,CHI,1,2,1,0,0,4 +2012,21184,MIN,2,2,0,0,1,5 +2012,21185,PHI,1,1,4,0,0,6 +2012,21185,PIT,2,0,2,0,0,4 +2012,21186,ANA,1,0,0,0,0,1 +2012,21186,EDM,2,0,0,0,0,2 +2012,21187,TBL,0,2,2,0,0,4 +2012,21187,WSH,0,1,1,0,0,2 +2012,21188,EDM,0,0,0,0,0,0 +2012,21188,LAK,0,0,2,0,0,2 +2012,21189,BOS,1,1,1,0,0,3 +2012,21189,PIT,2,2,1,0,0,5 +2012,21190,BUF,0,2,3,1,0,6 +2012,21190,TOR,3,0,2,0,0,5 +2012,21191,NJD,1,1,1,0,0,3 +2012,21191,NYI,1,0,0,0,0,1 +2012,21192,NYR,4,1,0,0,0,5 +2012,21192,PHI,0,2,1,0,0,3 +2012,21193,CAR,0,2,0,0,0,2 +2012,21193,OTT,0,0,1,0,0,1 +2012,21194,FLA,2,1,1,0,0,4 +2012,21194,WPG,0,2,2,1,0,5 +2012,21195,MIN,0,1,0,0,0,1 +2012,21195,NSH,0,1,0,0,1,2 +2012,21196,DAL,1,1,0,0,0,2 +2012,21196,SJS,1,2,2,0,0,5 +2012,21197,ANA,2,2,0,0,0,4 +2012,21197,VAN,2,2,0,0,1,5 +2012,21198,CBJ,0,0,0,0,0,0 +2012,21198,PHX,0,2,0,0,0,2 +2012,21199,MTL,1,2,2,0,0,5 +2012,21199,TBL,1,0,1,0,0,2 +2012,21200,DET,0,0,2,0,1,3 +2012,21200,STL,0,0,2,0,0,2 +2012,21201,TBL,1,0,1,0,0,2 +2012,21201,TOR,1,0,1,1,0,3 +2012,21202,NYI,2,1,2,0,0,5 +2012,21202,WPG,2,0,2,0,0,4 +2012,21203,BUF,0,1,0,0,0,1 +2012,21203,PHI,0,0,2,0,0,2 +2012,21204,NYR,1,0,1,0,0,2 +2012,21204,PIT,2,1,2,0,0,5 +2012,21205,FLA,0,1,1,0,0,2 +2012,21205,WSH,1,2,1,0,0,4 +2012,21206,CAR,0,1,0,0,1,2 +2012,21206,MTL,0,1,0,0,0,1 +2012,21207,BOS,1,1,1,0,0,3 +2012,21207,OTT,0,1,0,0,0,1 +2012,21208,DET,0,1,0,0,0,1 +2012,21208,NJD,1,0,1,0,0,2 +2012,21209,DAL,0,0,0,0,0,0 +2012,21209,NSH,0,0,2,0,0,2 +2012,21210,CHI,1,0,0,0,0,1 +2012,21210,MIN,0,0,1,0,1,2 +2012,21211,COL,1,1,0,0,0,2 +2012,21211,CBJ,2,2,1,0,0,5 +2012,21212,CGY,0,0,3,0,0,3 +2012,21212,VAN,1,0,1,0,0,2 +2012,21213,ANA,0,2,0,1,0,3 +2012,21213,EDM,0,1,1,0,0,2 +2012,21214,LAK,1,2,2,0,0,5 +2012,21214,SJS,1,1,3,0,1,6 +2012,21215,PHX,1,2,1,0,0,4 +2012,21215,STL,1,0,0,0,0,1 +2012,21216,BOS,0,1,2,0,1,4 +2012,21216,BUF,0,1,2,0,0,3 +2012,21217,NJD,1,1,2,0,0,4 +2012,21217,OTT,1,1,0,0,0,2 +2012,21218,NYR,0,1,0,0,0,1 +2012,21218,WSH,3,1,0,0,0,4 +2012,21219,PHI,1,1,0,0,0,2 +2012,21219,PIT,1,3,0,0,0,4 +2012,21220,CHI,1,1,0,0,1,3 +2012,21220,DET,0,0,2,0,0,2 +2012,21221,ANA,0,1,1,0,0,2 +2012,21221,CGY,1,2,2,0,0,5 +2012,21222,MTL,1,1,2,0,0,4 +2012,21222,TOR,0,0,1,0,0,1 +2012,21223,TBL,1,1,1,1,0,4 +2012,21223,WPG,0,1,2,0,0,3 +2012,21224,CBJ,1,4,2,0,0,7 +2012,21224,NYI,0,2,1,0,0,3 +2012,21225,CAR,0,0,1,0,0,1 +2012,21225,FLA,2,1,1,0,0,4 +2012,21226,MIN,0,1,0,0,0,1 +2012,21226,PHX,2,1,1,0,0,4 +2012,21227,DAL,0,0,2,0,0,2 +2012,21227,STL,0,1,2,0,0,3 +2012,21228,COL,1,0,0,0,0,1 +2012,21228,NSH,3,1,2,0,0,6 +2012,21229,EDM,0,0,0,0,0,0 +2012,21229,VAN,0,1,2,0,0,3 +2012,21230,LAK,1,1,0,0,0,2 +2012,21230,SJS,0,0,2,1,0,3 diff --git a/notebooks/journal.pone.0047712.s001.csv b/notebooks/journal.pone.0047712.s001.csv new file mode 100644 index 0000000..e64ab45 --- /dev/null +++ b/notebooks/journal.pone.0047712.s001.csv @@ -0,0 +1,3968 @@ +sample,best available identification,reads/400,,,, +B1234,Actinobacteridae,1,,,sample:,belly button sample identifier +B1234,Actinobacteridae,2,,,reads/400:,the number of reads of a particular phylotype in a given sample after a rarefaction to 400 reads per sample +B1234,Actinobacteridae,5,,,, +B1234,Actinomyces,7,,,, +B1234,Anaerococcus,1,,,, +B1234,Anaerococcus,1,,,, +B1234,Anaerococcus,1,,,, +B1234,Anaerococcus,1,,,, +B1234,Anaerococcus,1,,,, +B1234,Anaerococcus,11,,,, +B1234,Anaerococcus,2,,,, +B1234,Anaerococcus,4,,,, +B1234,Anaerococcus,56,,,, +B1234,Anaerococcus,8,,,, +B1234,Bacteroidetes,43,,,, +B1234,Caenibacterium,1,,,, +B1234,Campylobacter,2,,,, +B1234,Campylobacter,7,,,, +B1234,Campylobacterales,1,,,, +B1234,Clostridiales,1,,,, +B1234,Clostridiales,1,,,, +B1234,Clostridiales,2,,,, +B1234,Clostridiales,9,,,, +B1234,Corynebacterineae,1,,,, +B1234,Corynebacterineae,1,,,, +B1234,Corynebacterium,1,,,, +B1234,Corynebacterium,2,,,, +B1234,Dialister,2,,,, +B1234,Finegoldia,1,,,, +B1234,Finegoldia,37,,,, +B1234,Firmicutes,1,,,, +B1234,Mobiluncus,29,,,, +B1234,Peptoniphilus,1,,,, +B1234,Peptoniphilus,16,,,, +B1234,Peptoniphilus,35,,,, +B1234,Peptoniphilus,9,,,, +B1234,Porphyromonas,1,,,, +B1234,Porphyromonas,23,,,, +B1234,Prevotella,9,,,, +B1234,Sphingobacterium,1,,,, +B1234,unidentified,1,,,, +B1234,unidentified,1,,,, +B1234,unidentified,1,,,, +B1234,unidentified,1,,,, +B1234,unidentified,40,,,, +B1234,unidentified,6,,,, +B1234,unidentified,9,,,, +B1234,Varibaculum,3,,,, +B1235,Aerococcaceae,1,,,, +B1235,Anaerococcus,1,,,, +B1235,Anaerococcus,1,,,, +B1235,Anaerococcus,1,,,, +B1235,Anaerococcus,1,,,, +B1235,Anaerococcus,1,,,, +B1235,Anaerococcus,3,,,, +B1235,Anaerococcus,39,,,, +B1235,Anaerococcus,6,,,, +B1235,Anaerococcus,6,,,, +B1235,Bacteroidetes,1,,,, +B1235,Bacteroidetes,1,,,, +B1235,Bacteroidetes,1,,,, +B1235,Bacteroidetes,2,,,, +B1235,Bacteroidetes,21,,,, +B1235,Bacteroidetes,3,,,, +B1235,Campylobacter,4,,,, +B1235,Clostridiales,1,,,, +B1235,Clostridiales,1,,,, +B1235,Clostridiales,1,,,, +B1235,Clostridiales,1,,,, +B1235,Clostridiales,1,,,, +B1235,Clostridiales,1,,,, +B1235,Clostridiales,29,,,, +B1235,Clostridiales,3,,,, +B1235,Clostridiales,9,,,, +B1235,Corynebacterineae,1,,,, +B1235,Corynebacterineae,14,,,, +B1235,Corynebacterineae,4,,,, +B1235,Corynebacterium,2,,,, +B1235,Corynebacterium,2,,,, +B1235,Corynebacterium,36,,,, +B1235,Cupriavidus,1,,,, +B1235,Dermabacter,1,,,, +B1235,Dialister,1,,,, +B1235,Dialister,4,,,, +B1235,Fastidiosipila,1,,,, +B1235,Finegoldia,1,,,, +B1235,Finegoldia,9,,,, +B1235,Lactobacillales,1,,,, +B1235,Lactobacillales,18,,,, +B1235,Mobiluncus,3,,,, +B1235,Pasteurellaceae,1,,,, +B1235,Peptococcus,1,,,, +B1235,Peptoniphilus,1,,,, +B1235,Peptoniphilus,1,,,, +B1235,Peptoniphilus,1,,,, +B1235,Peptoniphilus,1,,,, +B1235,Peptoniphilus,1,,,, +B1235,Peptoniphilus,10,,,, +B1235,Peptoniphilus,40,,,, +B1235,Porphyromonadaceae,1,,,, +B1235,Porphyromonas,1,,,, +B1235,Porphyromonas,11,,,, +B1235,Prevotella,2,,,, +B1235,Prevotella,7,,,, +B1235,Prevotella,9,,,, +B1235,Prevotellaceae,1,,,, +B1235,Prevotellaceae,1,,,, +B1235,Staphylococcus,21,,,, +B1235,unidentified,1,,,, +B1235,unidentified,1,,,, +B1235,unidentified,1,,,, +B1235,unidentified,2,,,, +B1235,unidentified,2,,,, +B1235,unidentified,3,,,, +B1235,unidentified,37,,,, +B1235,Varibaculum,1,,,, +B1235,Veillonellaceae,2,,,, +B1236,Acinetobacter,1,,,, +B1236,Actinobacteridae,1,,,, +B1236,Actinobacteridae,38,,,, +B1236,Actinobacteridae,8,,,, +B1236,Actinomyces,1,,,, +B1236,Actinomyces,13,,,, +B1236,Actinomycetales,1,,,, +B1236,Aerococcaceae,2,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,1,,,, +B1236,Anaerococcus,11,,,, +B1236,Anaerococcus,2,,,, +B1236,Anaerococcus,28,,,, +B1236,Anaerococcus,42,,,, +B1236,Anaerococcus,55,,,, +B1236,Bacteroidetes,1,,,, +B1236,Bacteroidetes,1,,,, +B1236,Bacteroidetes,15,,,, +B1236,Chloroplast,1,,,, +B1236,Corynebacterineae,1,,,, +B1236,Corynebacterineae,1,,,, +B1236,Corynebacterineae,1,,,, +B1236,Corynebacterineae,1,,,, +B1236,Corynebacterineae,2,,,, +B1236,Corynebacterineae,2,,,, +B1236,Corynebacterineae,2,,,, +B1236,Corynebacterineae,3,,,, +B1236,Corynebacterineae,84,,,, +B1236,Corynebacterium,1,,,, +B1236,Corynebacterium,14,,,, +B1236,Corynebacterium,24,,,, +B1236,Dermabacter,9,,,, +B1236,Finegoldia,13,,,, +B1236,Mobiluncus,2,,,, +B1236,Peptoniphilus,5,,,, +B1236,Prevotella,2,,,, +B1236,Propionibacteriaceae,1,,,, +B1236,Turicella,2,,,, +B1237,Actinobacteridae,1,,,, +B1237,Actinobacteridae,1,,,, +B1237,Actinobacteridae,1,,,, +B1237,Aerococcaceae,2,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,1,,,, +B1237,Anaerococcus,17,,,, +B1237,Anaerococcus,2,,,, +B1237,Anaerococcus,5,,,, +B1237,Anaerococcus,50,,,, +B1237,Anaerococcus,61,,,, +B1237,Bacillales,1,,,, +B1237,Bacillales,1,,,, +B1237,Bacilli,1,,,, +B1237,Bacilli,1,,,, +B1237,Bacilli,4,,,, +B1237,Bacillus,1,,,, +B1237,Bacteroidetes,2,,,, +B1237,Bradyrhizobiaceae,1,,,, +B1237,Chloroplast,1,,,, +B1237,Chloroplast,2,,,, +B1237,Clostridiales,1,,,, +B1237,Clostridiales,1,,,, +B1237,Corynebacterineae,1,,,, +B1237,Corynebacterineae,1,,,, +B1237,Corynebacterineae,2,,,, +B1237,Corynebacterineae,5,,,, +B1237,Corynebacterium,6,,,, +B1237,Dermabacter,1,,,, +B1237,Enterobacteriaceae,1,,,, +B1237,Fastidiosipila,1,,,, +B1237,Finegoldia,1,,,, +B1237,Finegoldia,1,,,, +B1237,Finegoldia,27,,,, +B1237,Firmicutes,1,,,, +B1237,Firmicutes,1,,,, +B1237,Flavobacteriaceae,1,,,, +B1237,Ignavigranum,1,,,, +B1237,Lactobacillales,1,,,, +B1237,Mobiluncus,1,,,, +B1237,Peptococcus,1,,,, +B1237,Peptoniphilus,1,,,, +B1237,Peptoniphilus,1,,,, +B1237,Peptoniphilus,1,,,, +B1237,Peptoniphilus,1,,,, +B1237,Peptoniphilus,2,,,, +B1237,Peptoniphilus,21,,,, +B1237,Porphyromonadaceae,1,,,, +B1237,Staphylococcaceae,1,,,, +B1237,Staphylococcus,1,,,, +B1237,Staphylococcus,1,,,, +B1237,Staphylococcus,146,,,, +B1237,unidentified,1,,,, +B1237,unidentified,1,,,, +B1237,unidentified,1,,,, +B1237,Varibaculum,1,,,, +B1242,Actinobacteridae,1,,,, +B1242,Actinobacteridae,1,,,, +B1242,Actinobacteridae,1,,,, +B1242,Actinobacteridae,1,,,, +B1242,Actinobacteridae,2,,,, +B1242,Aerococcaceae,4,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,1,,,, +B1242,Anaerococcus,14,,,, +B1242,Anaerococcus,3,,,, +B1242,Anaerococcus,3,,,, +B1242,Anaerococcus,4,,,, +B1242,Anaerococcus,4,,,, +B1242,Anaerococcus,53,,,, +B1242,Bacteroidetes,7,,,, +B1242,Branhamella,1,,,, +B1242,Campylobacter,5,,,, +B1242,Chloroplast,1,,,, +B1242,Clostridia,1,,,, +B1242,Clostridiales,1,,,, +B1242,Clostridiales,10,,,, +B1242,Clostridiales,4,,,, +B1242,Corynebacterineae,1,,,, +B1242,Corynebacterineae,1,,,, +B1242,Corynebacterineae,4,,,, +B1242,Corynebacterineae,4,,,, +B1242,Corynebacterium,1,,,, +B1242,Corynebacterium,1,,,, +B1242,Corynebacterium,5,,,, +B1242,Corynebacterium,7,,,, +B1242,Corynebacterium,92,,,, +B1242,Dermabacter,3,,,, +B1242,Dialister,4,,,, +B1242,Finegoldia,1,,,, +B1242,Finegoldia,3,,,, +B1242,Finegoldia,47,,,, +B1242,Lactobacillales,8,,,, +B1242,Mobiluncus,1,,,, +B1242,Peptococcus,1,,,, +B1242,Peptoniphilus,1,,,, +B1242,Peptoniphilus,1,,,, +B1242,Peptoniphilus,1,,,, +B1242,Peptoniphilus,15,,,, +B1242,Peptoniphilus,2,,,, +B1242,Porphyromonas,3,,,, +B1242,Prevotella,1,,,, +B1242,Prevotella,3,,,, +B1242,Staphylococcus,1,,,, +B1242,Staphylococcus,1,,,, +B1242,Staphylococcus,38,,,, +B1242,unidentified,1,,,, +B1242,unidentified,12,,,, +B1242,unidentified,2,,,, +B1242,unidentified,2,,,, +B1242,Veillonellaceae,3,,,, +B1243,Acinetobacter,1,,,, +B1243,Acinetobacter,3,,,, +B1243,Actinobacteridae,1,,,, +B1243,Actinobacteridae,18,,,, +B1243,Aerococcaceae,3,,,, +B1243,Aerococcus,1,,,, +B1243,Anaerococcus,1,,,, +B1243,Anaerococcus,1,,,, +B1243,Anaerococcus,1,,,, +B1243,Anaerococcus,2,,,, +B1243,Anaerococcus,2,,,, +B1243,Anaerococcus,2,,,, +B1243,Anaerococcus,26,,,, +B1243,Anaerococcus,26,,,, +B1243,Anaerococcus,4,,,, +B1243,Anaerococcus,4,,,, +B1243,Aquabacterium,3,,,, +B1243,Arcanobacterium,1,,,, +B1243,Bacilli,1,,,, +B1243,Bacilli,1,,,, +B1243,Bacilli,2,,,, +B1243,Bacteroidetes,1,,,, +B1243,Bacteroidetes,1,,,, +B1243,Bacteroidetes,1,,,, +B1243,Bacteroidetes,6,,,, +B1243,Bradyrhizobiaceae,2,,,, +B1243,Branhamella,11,,,, +B1243,Brevibacterium,3,,,, +B1243,Burkholderiales,1,,,, +B1243,Campylobacter,2,,,, +B1243,Chloroplast,14,,,, +B1243,Cloacibacterium,7,,,, +B1243,Clostridiales,1,,,, +B1243,Clostridiales,12,,,, +B1243,Clostridiales,3,,,, +B1243,Corynebacterineae,2,,,, +B1243,Corynebacterineae,34,,,, +B1243,Corynebacterium,1,,,, +B1243,Corynebacterium,1,,,, +B1243,Corynebacterium,1,,,, +B1243,Corynebacterium,3,,,, +B1243,Corynebacterium,38,,,, +B1243,Dermabacter,9,,,, +B1243,Faecalibacterium,5,,,, +B1243,Finegoldia,1,,,, +B1243,Finegoldia,5,,,, +B1243,Lactobacillales,2,,,, +B1243,Micrococcus,2,,,, +B1243,Moraxellaceae,1,,,, +B1243,Mycobacterium,1,,,, +B1243,Ornithinimicrobium,1,,,, +B1243,Peptoniphilus,11,,,, +B1243,Peptoniphilus,3,,,, +B1243,Porphyromonadaceae,1,,,, +B1243,Porphyromonas,1,,,, +B1243,Porphyromonas,3,,,, +B1243,Prevotella,1,,,, +B1243,Pseudomonas,1,,,, +B1243,Pseudomonas,2,,,, +B1243,Ralstonia,2,,,, +B1243,Rhodococcus,1,,,, +B1243,Sphingomonas,2,,,, +B1243,Staphylococcaceae,1,,,, +B1243,Staphylococcus,81,,,, +B1243,unidentified,1,,,, +B1243,unidentified,1,,,, +B1243,unidentified,1,,,, +B1243,unidentified,1,,,, +B1243,unidentified,3,,,, +B1243,unidentified,5,,,, +B1243,Veillonella,2,,,, +B1246,Actinobacteridae,1,,,, +B1246,Actinobacteridae,17,,,, +B1246,Actinobacteridae,3,,,, +B1246,Actinomyces,1,,,, +B1246,Actinomyces,4,,,, +B1246,Actinomycetales,1,,,, +B1246,Actinomycetales,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,1,,,, +B1246,Anaerococcus,10,,,, +B1246,Anaerococcus,14,,,, +B1246,Anaerococcus,16,,,, +B1246,Anaerococcus,2,,,, +B1246,Anaerococcus,2,,,, +B1246,Anaerococcus,2,,,, +B1246,Anaerococcus,2,,,, +B1246,Anaerococcus,3,,,, +B1246,Anaerococcus,3,,,, +B1246,Anaerococcus,3,,,, +B1246,Anaerococcus,36,,,, +B1246,Anaerococcus,6,,,, +B1246,Anaerococcus,9,,,, +B1246,Bacteroidetes,1,,,, +B1246,Bacteroidetes,2,,,, +B1246,Bacteroidetes,35,,,, +B1246,Campylobacter,5,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,1,,,, +B1246,Clostridiales,13,,,, +B1246,Clostridiales,3,,,, +B1246,Clostridiales,4,,,, +B1246,Clostridiales,8,,,, +B1246,Corynebacterineae,1,,,, +B1246,Corynebacterium,5,,,, +B1246,Fastidiosipila,2,,,, +B1246,Finegoldia,3,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,1,,,, +B1246,Firmicutes,14,,,, +B1246,Fusobacterium,1,,,, +B1246,Peptococcus,2,,,, +B1246,Peptoniphilus,1,,,, +B1246,Peptoniphilus,1,,,, +B1246,Peptoniphilus,1,,,, +B1246,Peptoniphilus,1,,,, +B1246,Peptoniphilus,18,,,, +B1246,Peptoniphilus,35,,,, +B1246,Porphyromonas,1,,,, +B1246,Porphyromonas,3,,,, +B1246,Prevotella,1,,,, +B1246,Pseudonocardia,1,,,, +B1246,Staphylococcaceae,1,,,, +B1246,Staphylococcus,1,,,, +B1246,Staphylococcus,8,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,1,,,, +B1246,unidentified,13,,,, +B1246,unidentified,22,,,, +B1246,unidentified,9,,,, +B1246,Varibaculum,1,,,, +B1246,Varibaculum,7,,,, +B1246,Veillonellaceae,3,,,, +B1253,Acinetobacter,4,,,, +B1253,Actinobacteridae,1,,,, +B1253,Actinobacteridae,1,,,, +B1253,Actinobacteridae,3,,,, +B1253,Actinobacteridae,6,,,, +B1253,Aerococcaceae,6,,,, +B1253,Anaerococcus,1,,,, +B1253,Anaerococcus,1,,,, +B1253,Anaerococcus,1,,,, +B1253,Anaerococcus,1,,,, +B1253,Anaerococcus,1,,,, +B1253,Anaerococcus,1,,,, +B1253,Anaerococcus,11,,,, +B1253,Anaerococcus,2,,,, +B1253,Anaerococcus,2,,,, +B1253,Anaerococcus,21,,,, +B1253,Anaerococcus,3,,,, +B1253,Anaerococcus,4,,,, +B1253,Anaerococcus,6,,,, +B1253,Bacteroidetes,1,,,, +B1253,Bacteroidetes,1,,,, +B1253,Bacteroidetes,2,,,, +B1253,Bacteroidetes,26,,,, +B1253,Bacteroidetes,4,,,, +B1253,Brachybacterium,2,,,, +B1253,Bradyrhizobiaceae,3,,,, +B1253,Branhamella,1,,,, +B1253,Brevibacterium,1,,,, +B1253,Brevibacterium,6,,,, +B1253,Brevundimonas,1,,,, +B1253,Burkholderiales,1,,,, +B1253,Caulobacter,1,,,, +B1253,Chloroplast,1,,,, +B1253,Clostridiales,1,,,, +B1253,Clostridiales,1,,,, +B1253,Clostridiales,2,,,, +B1253,Clostridiales,5,,,, +B1253,Clostridiales,6,,,, +B1253,Corynebacterineae,1,,,, +B1253,Corynebacterineae,2,,,, +B1253,Corynebacterineae,3,,,, +B1253,Corynebacterineae,40,,,, +B1253,Corynebacterium,1,,,, +B1253,Corynebacterium,1,,,, +B1253,Corynebacterium,12,,,, +B1253,Corynebacterium,21,,,, +B1253,Dermabacter,3,,,, +B1253,Dialister,3,,,, +B1253,Finegoldia,5,,,, +B1253,Firmicutes,1,,,, +B1253,Granulicatella,2,,,, +B1253,Lactobacillales,4,,,, +B1253,Mesorhizobium,4,,,, +B1253,Methylobacterium,1,,,, +B1253,Mobiluncus,1,,,, +B1253,Mobiluncus,3,,,, +B1253,Novosphingobium,4,,,, +B1253,Peptococcus,1,,,, +B1253,Peptoniphilus,1,,,, +B1253,Peptoniphilus,1,,,, +B1253,Peptoniphilus,13,,,, +B1253,Peptoniphilus,2,,,, +B1253,Peptoniphilus,4,,,, +B1253,Polyangiaceae,1,,,, +B1253,Porphyromonadaceae,1,,,, +B1253,Porphyromonadaceae,7,,,, +B1253,Porphyromonas,2,,,, +B1253,Porphyromonas,3,,,, +B1253,Prevotella,1,,,, +B1253,Prevotella,2,,,, +B1253,Prevotellaceae,2,,,, +B1253,Staphylococcaceae,2,,,, +B1253,Staphylococcus,30,,,, +B1253,Streptococcus,1,,,, +B1253,Terrimonas,1,,,, +B1253,unidentified,1,,,, +B1253,unidentified,1,,,, +B1253,unidentified,1,,,, +B1253,unidentified,14,,,, +B1253,unidentified,2,,,, +B1253,unidentified,2,,,, +B1253,unidentified,24,,,, +B1253,unidentified,4,,,, +B1253,unidentified,6,,,, +B1253,unidentified,7,,,, +B1253,unidentified,9,,,, +B1254,Acidobacteriaceae,1,,,, +B1254,Acidobacteriaceae,1,,,, +B1254,Acinetobacter,1,,,, +B1254,Actinobacteridae,1,,,, +B1254,Actinobacteridae,1,,,, +B1254,Actinobacteridae,2,,,, +B1254,Actinomycetales,1,,,, +B1254,Aerococcaceae,1,,,, +B1254,Aerococcaceae,1,,,, +B1254,Anaerococcus,1,,,, +B1254,Anaerococcus,1,,,, +B1254,Anaerococcus,1,,,, +B1254,Anaerococcus,1,,,, +B1254,Anaerococcus,2,,,, +B1254,Anaerococcus,3,,,, +B1254,Anaerococcus,4,,,, +B1254,Anaerococcus,6,,,, +B1254,Aquabacterium,3,,,, +B1254,Bacteroidetes,2,,,, +B1254,Bacteroidetes,4,,,, +B1254,Bacteroidetes,7,,,, +B1254,Betaproteobacteria,2,,,, +B1254,Bradyrhizobiaceae,4,,,, +B1254,Branhamella,16,,,, +B1254,Brucellaceae,1,,,, +B1254,Cloacibacterium,1,,,, +B1254,Clostridiales,1,,,, +B1254,Corynebacterineae,1,,,, +B1254,Corynebacterineae,1,,,, +B1254,Corynebacterineae,53,,,, +B1254,Corynebacterium,1,,,, +B1254,Corynebacterium,155,,,, +B1254,Corynebacterium,2,,,, +B1254,Corynebacterium,26,,,, +B1254,Corynebacterium,6,,,, +B1254,Finegoldia,1,,,, +B1254,Flavobacteriaceae,10,,,, +B1254,Methylophilaceae,2,,,, +B1254,Microbacteriaceae,1,,,, +B1254,Microbacteriaceae,1,,,, +B1254,Microbacteriaceae,4,,,, +B1254,Microbacteriaceae,4,,,, +B1254,Microbacteriaceae,7,,,, +B1254,Micrococcus,1,,,, +B1254,Myxococcales,1,,,, +B1254,Novosphingobium,1,,,, +B1254,Phenylobacterium,5,,,, +B1254,Proteobacteria,1,,,, +B1254,Proteobacteria,1,,,, +B1254,Proteobacteria,1,,,, +B1254,Ralstonia,1,,,, +B1254,Rhodospirillales,1,,,, +B1254,Sphingobacteriales,2,,,, +B1254,Staphylococcus,1,,,, +B1254,Staphylococcus,22,,,, +B1254,Terrimonas,2,,,, +B1254,unidentified,3,,,, +B1254,Zimmermannella,12,,,, +B1258,Achromobacter,13,,,, +B1258,Acidobacteriaceae,1,,,, +B1258,Actinobacteridae,1,,,, +B1258,Actinobacteridae,2,,,, +B1258,Actinobacteridae,5,,,, +B1258,Actinomycetales,1,,,, +B1258,Aerococcaceae,16,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,1,,,, +B1258,Anaerococcus,10,,,, +B1258,Anaerococcus,16,,,, +B1258,Anaerococcus,18,,,, +B1258,Anaerococcus,3,,,, +B1258,Anaerococcus,3,,,, +B1258,Anaerococcus,8,,,, +B1258,Bacteroidetes,10,,,, +B1258,Bacteroidetes,6,,,, +B1258,Bradyrhizobiaceae,4,,,, +B1258,Campylobacter,1,,,, +B1258,Chryseobacterium,1,,,, +B1258,Clostridiales,1,,,, +B1258,Clostridiales,1,,,, +B1258,Clostridiales,18,,,, +B1258,Clostridiales,2,,,, +B1258,Clostridiales,3,,,, +B1258,Clostridiales,3,,,, +B1258,Clostridiales,9,,,, +B1258,Coriobacteriaceae,4,,,, +B1258,Corynebacterineae,1,,,, +B1258,Corynebacterineae,14,,,, +B1258,Corynebacterineae,15,,,, +B1258,Corynebacterium,19,,,, +B1258,Corynebacterium,2,,,, +B1258,Corynebacterium,2,,,, +B1258,Crenotrichaceae,2,,,, +B1258,Dermabacter,9,,,, +B1258,Dialister,1,,,, +B1258,Finegoldia,1,,,, +B1258,Firmicutes,1,,,, +B1258,Lactobacillales,9,,,, +B1258,Leuconostoc,1,,,, +B1258,Micrococcaceae,3,,,, +B1258,Mobiluncus,2,,,, +B1258,Novosphingobium,7,,,, +B1258,Peptoniphilus,1,,,, +B1258,Peptoniphilus,1,,,, +B1258,Peptoniphilus,23,,,, +B1258,Peptoniphilus,8,,,, +B1258,Phenylobacterium,5,,,, +B1258,Porphyromonadaceae,1,,,, +B1258,Porphyromonas,22,,,, +B1258,Prevotella,1,,,, +B1258,Prevotella,2,,,, +B1258,Prevotellaceae,1,,,, +B1258,Proteobacteria,1,,,, +B1258,Pseudomonas,1,,,, +B1258,Rhizobiales,1,,,, +B1258,Rhodobacteraceae,1,,,, +B1258,Rhodospirillales,3,,,, +B1258,Sphingobacteriales,3,,,, +B1258,Sphingobacterium,2,,,, +B1258,Staphylococcus,1,,,, +B1258,Staphylococcus,14,,,, +B1258,Streptococcus,4,,,, +B1258,Terrimonas,3,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,1,,,, +B1258,unidentified,16,,,, +B1258,unidentified,2,,,, +B1258,unidentified,4,,,, +B1258,unidentified,4,,,, +B1258,unidentified,5,,,, +B1258,Varibaculum,3,,,, +B1258,Veillonellaceae,1,,,, +B1259,Acinetobacter,1,,,, +B1259,Acinetobacter,1,,,, +B1259,Acinetobacter,127,,,, +B1259,Acinetobacter,4,,,, +B1259,Actinobacteridae,11,,,, +B1259,Actinomycetales,1,,,, +B1259,Anaerococcus,1,,,, +B1259,Anaerococcus,4,,,, +B1259,Anaerococcus,6,,,, +B1259,Arsenicicoccus,8,,,, +B1259,Bacillales,1,,,, +B1259,Bacteroidetes,1,,,, +B1259,Bacteroidetes,1,,,, +B1259,Bacteroidetes,1,,,, +B1259,Bacteroidetes,10,,,, +B1259,Bacteroidetes,2,,,, +B1259,Blastococcus,2,,,, +B1259,Brachybacterium,2,,,, +B1259,Bradyrhizobiaceae,1,,,, +B1259,Branhamella,26,,,, +B1259,Burkholderiales,1,,,, +B1259,Burkholderiales,4,,,, +B1259,Chloroplast,5,,,, +B1259,Clostridiales,5,,,, +B1259,Corynebacterineae,1,,,, +B1259,Corynebacterineae,2,,,, +B1259,Corynebacterium,1,,,, +B1259,Corynebacterium,19,,,, +B1259,Corynebacterium,3,,,, +B1259,Erythrobacter,2,,,, +B1259,Fusobacterium,5,,,, +B1259,Gammaproteobacteria,2,,,, +B1259,Gordoniaceae,2,,,, +B1259,Halomonadaceae,1,,,, +B1259,Methylobacterium,1,,,, +B1259,Methylobacterium,1,,,, +B1259,Methylobacterium,1,,,, +B1259,Micrococcus,6,,,, +B1259,Moraxellaceae,3,,,, +B1259,Peptoniphilus,2,,,, +B1259,Peptostreptococcaceae,1,,,, +B1259,Porphyromonadaceae,1,,,, +B1259,Porphyromonadaceae,2,,,, +B1259,Prevotella,1,,,, +B1259,Prevotellaceae,1,,,, +B1259,Propionibacterium,1,,,, +B1259,Pseudomonadaceae,1,,,, +B1259,Pseudomonas,10,,,, +B1259,Pseudonocardiaceae,1,,,, +B1259,Pseudoxanthomonas,2,,,, +B1259,Ralstonia,1,,,, +B1259,Rhodobacteraceae,1,,,, +B1259,Rhodobacteraceae,2,,,, +B1259,Rhodobacteraceae,3,,,, +B1259,Rhodocyclaceae,2,,,, +B1259,Skermanella,1,,,, +B1259,Skermanella,3,,,, +B1259,Sphingomonadaceae,1,,,, +B1259,Sphingomonadaceae,2,,,, +B1259,Sphingomonas,7,,,, +B1259,Sphingopyxis,1,,,, +B1259,Spirosoma,4,,,, +B1259,Staphylococcus,2,,,, +B1259,Staphylococcus,28,,,, +B1259,Stenotrophomonas,1,,,, +B1259,Stenotrophomonas,1,,,, +B1259,Stenotrophomonas,3,,,, +B1259,Streptococcus,1,,,, +B1259,Streptococcus,1,,,, +B1259,Streptococcus,5,,,, +B1259,unidentified,1,,,, +B1259,unidentified,1,,,, +B1259,unidentified,2,,,, +B1259,unidentified,2,,,, +B1259,unidentified,2,,,, +B1259,unidentified,6,,,, +B1259,unidentified,9,,,, +B1259,Weissella,2,,,, +B1259,Xanthomonadaceae,1,,,, +B1259,Xanthomonadaceae,5,,,, +B1260,Acidobacteriaceae,1,,,, +B1260,Acidobacteriaceae,5,,,, +B1260,Acinetobacter,7,,,, +B1260,Actinobacteridae,10,,,, +B1260,Actinomycetaceae,1,,,, +B1260,Aerococcaceae,1,,,, +B1260,Agromonas,1,,,, +B1260,Alphaproteobacteria,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,1,,,, +B1260,Anaerococcus,12,,,, +B1260,Anaerococcus,2,,,, +B1260,Anaerococcus,36,,,, +B1260,Anaerococcus,38,,,, +B1260,Anaerococcus,4,,,, +B1260,Aquabacterium,1,,,, +B1260,Aquabacterium,4,,,, +B1260,Bacillaceae,1,,,, +B1260,Bacillales,1,,,, +B1260,Bacillales,2,,,, +B1260,Bacilli,1,,,, +B1260,Bacilli,1,,,, +B1260,Bacteroides,1,,,, +B1260,Bacteroidetes,10,,,, +B1260,Bacteroidetes,2,,,, +B1260,Bacteroidetes,2,,,, +B1260,Bacteroidetes,7,,,, +B1260,Bradyrhizobiaceae,8,,,, +B1260,Bradyrhizobium,1,,,, +B1260,Burkholderiales,1,,,, +B1260,Campylobacter,1,,,, +B1260,Campylobacter,2,,,, +B1260,Chloroplast,2,,,, +B1260,Clostridia,2,,,, +B1260,Clostridiales,1,,,, +B1260,Clostridiales,1,,,, +B1260,Clostridiales,2,,,, +B1260,Clostridiales,2,,,, +B1260,Clostridiales,7,,,, +B1260,Comamonadaceae,2,,,, +B1260,Corynebacterineae,1,,,, +B1260,Corynebacterineae,4,,,, +B1260,Corynebacterium,29,,,, +B1260,Dermabacter,2,,,, +B1260,Enterobacteriaceae,1,,,, +B1260,Finegoldia,1,,,, +B1260,Finegoldia,3,,,, +B1260,Firmicutes,1,,,, +B1260,Gammaproteobacteria,1,,,, +B1260,Gammaproteobacteria,2,,,, +B1260,Gemella,1,,,, +B1260,Hyphomicrobium,1,,,, +B1260,Lactobacillales,1,,,, +B1260,Lactobacillales,2,,,, +B1260,Methylophilaceae,3,,,, +B1260,Micrococcineae,1,,,, +B1260,Mobiluncus,1,,,, +B1260,Myxococcales,1,,,, +B1260,Novosphingobium,4,,,, +B1260,Peptococcus,5,,,, +B1260,Peptoniphilus,1,,,, +B1260,Peptoniphilus,1,,,, +B1260,Peptoniphilus,1,,,, +B1260,Peptoniphilus,6,,,, +B1260,Phenylobacterium,1,,,, +B1260,Phenylobacterium,3,,,, +B1260,Polyangiaceae,2,,,, +B1260,Porphyromonas,1,,,, +B1260,Prevotella,1,,,, +B1260,Prevotella,1,,,, +B1260,Proteobacteria,1,,,, +B1260,Pseudomonadaceae,1,,,, +B1260,Rhodobacteraceae,1,,,, +B1260,Rhodobacteraceae,1,,,, +B1260,Rhodospirillales,2,,,, +B1260,Rubrobacteraceae,1,,,, +B1260,Staphylococcus,1,,,, +B1260,Staphylococcus,1,,,, +B1260,Staphylococcus,82,,,, +B1260,Streptococcus,1,,,, +B1260,Streptococcus,1,,,, +B1260,Streptococcus,1,,,, +B1260,Streptococcus,5,,,, +B1260,Terrimonas,1,,,, +B1260,unidentified,1,,,, +B1260,unidentified,1,,,, +B1260,unidentified,11,,,, +B1260,unidentified,2,,,, +B1260,unidentified,3,,,, +B1260,unidentified,4,,,, +B1260,unidentified,4,,,, +B1264,Acidobacteriaceae,1,,,, +B1264,Acinetobacter,1,,,, +B1264,Actinobacteridae,1,,,, +B1264,Actinobacteridae,1,,,, +B1264,Actinobacteridae,5,,,, +B1264,Anaerococcus,1,,,, +B1264,Anaerococcus,1,,,, +B1264,Anaerococcus,1,,,, +B1264,Anaerococcus,12,,,, +B1264,Anaerococcus,3,,,, +B1264,Anaerococcus,5,,,, +B1264,Anaerococcus,6,,,, +B1264,Aquabacterium,2,,,, +B1264,Bacilli,1,,,, +B1264,Bacteroidetes,1,,,, +B1264,Bacteroidetes,1,,,, +B1264,Bacteroidetes,10,,,, +B1264,Bacteroidetes,3,,,, +B1264,Bacteroidetes,5,,,, +B1264,Betaproteobacteria,1,,,, +B1264,Betaproteobacteria,1,,,, +B1264,Betaproteobacteria,1,,,, +B1264,Bradyrhizobiaceae,1,,,, +B1264,Bradyrhizobiaceae,4,,,, +B1264,Bradyrhizobium,1,,,, +B1264,Brevibacterium,2,,,, +B1264,Burkholderiales,3,,,, +B1264,Campylobacter,1,,,, +B1264,Clostridia,2,,,, +B1264,Clostridiales,1,,,, +B1264,Clostridiales,22,,,, +B1264,Clostridiales,3,,,, +B1264,Clostridiales,4,,,, +B1264,Clostridiales,9,,,, +B1264,Corynebacterineae,1,,,, +B1264,Corynebacterineae,19,,,, +B1264,Corynebacterium,1,,,, +B1264,Corynebacterium,1,,,, +B1264,Corynebacterium,4,,,, +B1264,Crenotrichaceae,2,,,, +B1264,Dialister,4,,,, +B1264,Finegoldia,1,,,, +B1264,Finegoldia,13,,,, +B1264,Lactobacillales,4,,,, +B1264,Mesorhizobium,2,,,, +B1264,Methylophilaceae,5,,,, +B1264,Microbacteriaceae,1,,,, +B1264,Microbacteriaceae,1,,,, +B1264,Micromonosporaceae,1,,,, +B1264,Mobiluncus,9,,,, +B1264,Novosphingobium,19,,,, +B1264,Peptococcus,21,,,, +B1264,Peptoniphilus,1,,,, +B1264,Peptoniphilus,1,,,, +B1264,Peptoniphilus,2,,,, +B1264,Peptoniphilus,31,,,, +B1264,Peptoniphilus,4,,,, +B1264,Phenylobacterium,9,,,, +B1264,Porphyromonadaceae,2,,,, +B1264,Porphyromonas,2,,,, +B1264,Prevotella,4,,,, +B1264,Proteobacteria,1,,,, +B1264,Proteobacteria,3,,,, +B1264,Pseudomonas,1,,,, +B1264,Ralstonia,6,,,, +B1264,Rhizobiales,1,,,, +B1264,Rhodospirillales,1,,,, +B1264,Sphingobacteriales,6,,,, +B1264,Sphingomonas,1,,,, +B1264,Staphylococcus,7,,,, +B1264,Streptococcus,3,,,, +B1264,Streptococcus,3,,,, +B1264,Terrimonas,6,,,, +B1264,unidentified,1,,,, +B1264,unidentified,1,,,, +B1264,unidentified,1,,,, +B1264,unidentified,16,,,, +B1264,unidentified,2,,,, +B1264,unidentified,2,,,, +B1264,unidentified,21,,,, +B1264,unidentified,24,,,, +B1264,unidentified,4,,,, +B1264,unidentified,7,,,, +B1265,Acidobacteriaceae,1,,,, +B1265,Actinobacteridae,3,,,, +B1265,Actinobacteridae,7,,,, +B1265,Actinomycetaceae,3,,,, +B1265,Actinomycetales,1,,,, +B1265,Anaerococcus,1,,,, +B1265,Anaerococcus,1,,,, +B1265,Anaerococcus,3,,,, +B1265,Anaerococcus,8,,,, +B1265,Anaerococcus,8,,,, +B1265,Bacillales,1,,,, +B1265,Bacillales,1,,,, +B1265,Bacilli,2,,,, +B1265,Bacteroidetes,1,,,, +B1265,Bacteroidetes,1,,,, +B1265,Bacteroidetes,14,,,, +B1265,Bacteroidetes,5,,,, +B1265,Bradyrhizobiaceae,1,,,, +B1265,Bradyrhizobiaceae,1,,,, +B1265,Bradyrhizobiaceae,5,,,, +B1265,Branhamella,1,,,, +B1265,Brevibacterium,1,,,, +B1265,Cellvibrio,1,,,, +B1265,Chloroplast,1,,,, +B1265,Clostridia,4,,,, +B1265,Clostridiales,1,,,, +B1265,Clostridiales,15,,,, +B1265,Clostridiales,4,,,, +B1265,Clostridiales,5,,,, +B1265,Corynebacterineae,22,,,, +B1265,Corynebacterium,1,,,, +B1265,Corynebacterium,7,,,, +B1265,Dermabacter,1,,,, +B1265,Dialister,2,,,, +B1265,Finegoldia,1,,,, +B1265,Finegoldia,1,,,, +B1265,Finegoldia,1,,,, +B1265,Finegoldia,12,,,, +B1265,Lactobacillales,9,,,, +B1265,Limnobacter,1,,,, +B1265,Methanobrevibacter,3,,,, +B1265,Methylophilaceae,1,,,, +B1265,Microbulbifer,1,,,, +B1265,Mobiluncus,18,,,, +B1265,Novosphingobium,3,,,, +B1265,Paenibacillaceae,1,,,, +B1265,Pedomicrobium,1,,,, +B1265,Peptococcus,1,,,, +B1265,Peptococcus,13,,,, +B1265,Peptoniphilus,1,,,, +B1265,Peptoniphilus,1,,,, +B1265,Peptoniphilus,1,,,, +B1265,Peptoniphilus,1,,,, +B1265,Peptoniphilus,1,,,, +B1265,Peptoniphilus,1,,,, +B1265,Peptoniphilus,4,,,, +B1265,Peptoniphilus,43,,,, +B1265,Peptoniphilus,5,,,, +B1265,Phenylobacterium,2,,,, +B1265,Polyangiaceae,2,,,, +B1265,Porphyromonadaceae,7,,,, +B1265,Porphyromonas,2,,,, +B1265,Porphyromonas,3,,,, +B1265,Prevotella,3,,,, +B1265,Pseudomonas,1,,,, +B1265,Sphingomonas,1,,,, +B1265,Staphylococcus,7,,,, +B1265,Streptococcus,1,,,, +B1265,Terrimonas,2,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,1,,,, +B1265,unidentified,12,,,, +B1265,unidentified,20,,,, +B1265,unidentified,27,,,, +B1265,unidentified,3,,,, +B1265,unidentified,37,,,, +B1265,Varibaculum,1,,,, +B1265,Veillonellaceae,1,,,, +B1273,Actinobacteridae,2,,,, +B1273,Actinomyces,1,,,, +B1273,Actinomycetales,1,,,, +B1273,Actinomycetales,1,,,, +B1273,Actinomycetales,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,1,,,, +B1273,Anaerococcus,11,,,, +B1273,Anaerococcus,22,,,, +B1273,Anaerococcus,24,,,, +B1273,Anaerococcus,4,,,, +B1273,Bacilli,1,,,, +B1273,Bacilli,1,,,, +B1273,Bacteroidetes,1,,,, +B1273,Bacteroidetes,30,,,, +B1273,Brevibacterium,2,,,, +B1273,Carnobacteriaceae,1,,,, +B1273,Carnobacteriaceae,1,,,, +B1273,Clostridiales,5,,,, +B1273,Corynebacterineae,2,,,, +B1273,Corynebacterineae,59,,,, +B1273,Corynebacterium,21,,,, +B1273,Corynebacterium,5,,,, +B1273,Dermabacter,1,,,, +B1273,Dermabacter,2,,,, +B1273,Finegoldia,1,,,, +B1273,Finegoldia,1,,,, +B1273,Finegoldia,1,,,, +B1273,Finegoldia,1,,,, +B1273,Finegoldia,57,,,, +B1273,Firmicutes,1,,,, +B1273,Lactobacillales,1,,,, +B1273,Lactobacillales,1,,,, +B1273,Lactobacillales,1,,,, +B1273,Lactobacillales,23,,,, +B1273,Microbacteriaceae,2,,,, +B1273,Micrococcineae,1,,,, +B1273,Micrococcineae,1,,,, +B1273,Peptococcus,3,,,, +B1273,Peptoniphilus,1,,,, +B1273,Peptoniphilus,1,,,, +B1273,Peptoniphilus,1,,,, +B1273,Peptoniphilus,1,,,, +B1273,Peptoniphilus,1,,,, +B1273,Peptoniphilus,2,,,, +B1273,Peptoniphilus,61,,,, +B1273,Prevotella,7,,,, +B1273,Staphylococcus,10,,,, +B1273,unidentified,1,,,, +B1273,unidentified,1,,,, +B1273,Varibaculum,1,,,, +B1273,Varibaculum,7,,,, +B1275,Actinobacteridae,1,,,, +B1275,Actinobacteridae,4,,,, +B1275,Actinobacteridae,6,,,, +B1275,Actinomycetaceae,1,,,, +B1275,Actinomycetales,1,,,, +B1275,Anaerococcus,1,,,, +B1275,Anaerococcus,1,,,, +B1275,Anaerococcus,1,,,, +B1275,Anaerococcus,1,,,, +B1275,Anaerococcus,1,,,, +B1275,Anaerococcus,1,,,, +B1275,Anaerococcus,14,,,, +B1275,Anaerococcus,4,,,, +B1275,Anaerococcus,5,,,, +B1275,Bacillales,1,,,, +B1275,Bacteroidetes,1,,,, +B1275,Bacteroidetes,1,,,, +B1275,Bacteroidetes,1,,,, +B1275,Bacteroidetes,70,,,, +B1275,Branhamella,1,,,, +B1275,Campylobacter,1,,,, +B1275,Campylobacter,4,,,, +B1275,Clostridia,1,,,, +B1275,Clostridia,1,,,, +B1275,Clostridiales,1,,,, +B1275,Clostridiales,1,,,, +B1275,Clostridiales,1,,,, +B1275,Clostridiales,1,,,, +B1275,Clostridiales,10,,,, +B1275,Clostridiales,2,,,, +B1275,Clostridiales,4,,,, +B1275,Clostridiales,7,,,, +B1275,Corynebacterium,2,,,, +B1275,Dialister,2,,,, +B1275,Finegoldia,10,,,, +B1275,Firmicutes,1,,,, +B1275,Firmicutes,1,,,, +B1275,Firmicutes,1,,,, +B1275,Firmicutes,1,,,, +B1275,Firmicutes,1,,,, +B1275,Firmicutes,3,,,, +B1275,Mobiluncus,8,,,, +B1275,Peptococcus,1,,,, +B1275,Peptoniphilus,3,,,, +B1275,Peptoniphilus,9,,,, +B1275,Porphyromonadaceae,1,,,, +B1275,Porphyromonadaceae,1,,,, +B1275,Porphyromonadaceae,52,,,, +B1275,Porphyromonas,1,,,, +B1275,Porphyromonas,1,,,, +B1275,Porphyromonas,1,,,, +B1275,Porphyromonas,11,,,, +B1275,Porphyromonas,2,,,, +B1275,Prevotella,8,,,, +B1275,Staphylococcus,5,,,, +B1275,Succiniclasticum,5,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,1,,,, +B1275,unidentified,10,,,, +B1275,unidentified,13,,,, +B1275,unidentified,19,,,, +B1275,unidentified,2,,,, +B1275,unidentified,2,,,, +B1275,unidentified,31,,,, +B1275,unidentified,4,,,, +B1275,unidentified,5,,,, +B1275,unidentified,6,,,, +B1275,Varibaculum,1,,,, +B1275,Varibaculum,7,,,, +B1275,Veillonellaceae,1,,,, +B1275,Veillonellaceae,1,,,, +B1278,Acidobacteriaceae,1,,,, +B1278,Actinobacteridae,1,,,, +B1278,Actinobacteridae,2,,,, +B1278,Actinobacteridae,2,,,, +B1278,Actinomycetales,1,,,, +B1278,Actinomycetales,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,1,,,, +B1278,Anaerococcus,2,,,, +B1278,Anaerococcus,31,,,, +B1278,Anaerococcus,65,,,, +B1278,Bacillales,1,,,, +B1278,Bacilli,2,,,, +B1278,Bacilli,2,,,, +B1278,Bacteroidetes,3,,,, +B1278,Bradyrhizobiaceae,1,,,, +B1278,Burkholderiales,1,,,, +B1278,Corynebacterineae,1,,,, +B1278,Corynebacterineae,1,,,, +B1278,Corynebacterineae,1,,,, +B1278,Corynebacterineae,1,,,, +B1278,Corynebacterineae,127,,,, +B1278,Corynebacterineae,2,,,, +B1278,Corynebacterium,1,,,, +B1278,Corynebacterium,1,,,, +B1278,Corynebacterium,1,,,, +B1278,Corynebacterium,2,,,, +B1278,Corynebacterium,3,,,, +B1278,Finegoldia,4,,,, +B1278,Lactobacillales,1,,,, +B1278,Novosphingobium,1,,,, +B1278,Pelomonas,1,,,, +B1278,Peptoniphilus,5,,,, +B1278,Phenylobacterium,1,,,, +B1278,Rhodospirillales,1,,,, +B1278,Staphylococcaceae,1,,,, +B1278,Staphylococcus,1,,,, +B1278,Staphylococcus,115,,,, +B1278,Streptococcus,1,,,, +B1278,unidentified,1,,,, +B1278,unidentified,2,,,, +B1280,Actinobacteridae,1,,,, +B1280,Actinobacteridae,1,,,, +B1280,Actinobacteridae,38,,,, +B1280,Actinomyces,1,,,, +B1280,Aerococcaceae,1,,,, +B1280,Aerococcaceae,1,,,, +B1280,Aerococcaceae,1,,,, +B1280,Aerococcaceae,27,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,1,,,, +B1280,Anaerococcus,3,,,, +B1280,Anaerococcus,3,,,, +B1280,Anaerococcus,32,,,, +B1280,Anaerococcus,4,,,, +B1280,Anaerococcus,8,,,, +B1280,Anaerococcus,9,,,, +B1280,Bacillales,1,,,, +B1280,Bacteroidetes,1,,,, +B1280,Bacteroidetes,3,,,, +B1280,Bradyrhizobiaceae,2,,,, +B1280,Clostridiales,1,,,, +B1280,Clostridiales,5,,,, +B1280,Corynebacterineae,38,,,, +B1280,Corynebacterium,1,,,, +B1280,Corynebacterium,128,,,, +B1280,Dermabacter,1,,,, +B1280,Dermabacter,13,,,, +B1280,Dermabacter,2,,,, +B1280,Faecalibacterium,1,,,, +B1280,Gammaproteobacteria,1,,,, +B1280,Methylophilaceae,2,,,, +B1280,Micrococcineae,1,,,, +B1280,Nevskia,1,,,, +B1280,Novosphingobium,4,,,, +B1280,Phenylobacterium,1,,,, +B1280,Rhodospirillales,2,,,, +B1280,Staphylococcaceae,1,,,, +B1280,Staphylococcus,49,,,, +B1280,Terrimonas,1,,,, +B1280,unidentified,2,,,, +B1280,Varibaculum,1,,,, +B1282,Actinobacteridae,1,,,, +B1282,Actinobacteridae,1,,,, +B1282,Actinomycetaceae,1,,,, +B1282,Actinomycetales,1,,,, +B1282,Actinomycetales,2,,,, +B1282,Aerococcaceae,1,,,, +B1282,Aerococcaceae,3,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,1,,,, +B1282,Anaerococcus,23,,,, +B1282,Anaerococcus,3,,,, +B1282,Anaerococcus,31,,,, +B1282,Anaerococcus,4,,,, +B1282,Anaerococcus,7,,,, +B1282,Anaerococcus,8,,,, +B1282,Bacillales,1,,,, +B1282,Bacillales,2,,,, +B1282,Bacillales,4,,,, +B1282,Bacillales,93,,,, +B1282,Bacilli,1,,,, +B1282,Bradyrhizobiaceae,1,,,, +B1282,Burkholderiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,1,,,, +B1282,Clostridiales,2,,,, +B1282,Clostridiales,3,,,, +B1282,Clostridiales,3,,,, +B1282,Corynebacterineae,1,,,, +B1282,Corynebacterineae,1,,,, +B1282,Corynebacterineae,1,,,, +B1282,Corynebacterineae,1,,,, +B1282,Corynebacterineae,1,,,, +B1282,Corynebacterineae,2,,,, +B1282,Corynebacterineae,33,,,, +B1282,Corynebacterium,18,,,, +B1282,Corynebacterium,24,,,, +B1282,Corynebacterium,6,,,, +B1282,Dermabacter,2,,,, +B1282,Firmicutes,2,,,, +B1282,Nocardioidaceae,1,,,, +B1282,Novosphingobium,1,,,, +B1282,Peptoniphilus,2,,,, +B1282,Peptostreptococcaceae Incertae Sedis,1,,,, +B1282,Porphyromonas,1,,,, +B1282,Rhodobacteraceae,1,,,, +B1282,Rhodococcus,6,,,, +B1282,Rhodococcus,69,,,, +B1282,Staphylococcus,1,,,, +B1282,Staphylococcus,4,,,, +B1282,unidentified,1,,,, +B1282,unidentified,1,,,, +B1282,unidentified,1,,,, +B1282,unidentified,3,,,, +B1283,Actinobacteridae,1,,,, +B1283,Actinobacteridae,12,,,, +B1283,Actinomycetaceae,3,,,, +B1283,Actinomycetales,1,,,, +B1283,Anaerococcus,7,,,, +B1283,Bacteroidetes,1,,,, +B1283,Bacteroidetes,1,,,, +B1283,Bacteroidetes,1,,,, +B1283,Bacteroidetes,1,,,, +B1283,Bacteroidetes,1,,,, +B1283,Bacteroidetes,68,,,, +B1283,Campylobacter,3,,,, +B1283,Clostridiales,1,,,, +B1283,Clostridiales,1,,,, +B1283,Clostridiales,10,,,, +B1283,Clostridiales,2,,,, +B1283,Clostridiales,20,,,, +B1283,Clostridiales,3,,,, +B1283,Coriobacteriaceae,1,,,, +B1283,Corynebacterineae,6,,,, +B1283,Corynebacterium,1,,,, +B1283,Corynebacterium,2,,,, +B1283,Dermabacter,1,,,, +B1283,Dialister,2,,,, +B1283,Dialister,3,,,, +B1283,Finegoldia,2,,,, +B1283,Firmicutes,2,,,, +B1283,Firmicutes,2,,,, +B1283,Methylobacterium,1,,,, +B1283,Mobiluncus,1,,,, +B1283,Mobiluncus,15,,,, +B1283,Novosphingobium,1,,,, +B1283,Peptococcus,1,,,, +B1283,Peptoniphilus,1,,,, +B1283,Peptoniphilus,16,,,, +B1283,Porphyromonadaceae,8,,,, +B1283,Porphyromonas,1,,,, +B1283,Porphyromonas,1,,,, +B1283,Porphyromonas,29,,,, +B1283,Porphyromonas,5,,,, +B1283,Prevotella,2,,,, +B1283,Prevotella,4,,,, +B1283,Prevotellaceae,5,,,, +B1283,Rhodospirillales,1,,,, +B1283,Streptococcus,2,,,, +B1283,Terrimonas,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,1,,,, +B1283,unidentified,12,,,, +B1283,unidentified,19,,,, +B1283,unidentified,2,,,, +B1283,unidentified,2,,,, +B1283,unidentified,3,,,, +B1283,unidentified,34,,,, +B1283,unidentified,54,,,, +B1283,unidentified,7,,,, +B1283,Veillonellaceae,1,,,, +B1284,Actinobacteridae,1,,,, +B1284,Actinobacteridae,1,,,, +B1284,Actinobacteridae,15,,,, +B1284,Actinomyces,1,,,, +B1284,Actinomycetales,1,,,, +B1284,Actinomycetales,1,,,, +B1284,Aerococcaceae,1,,,, +B1284,Anaerococcus,1,,,, +B1284,Anaerococcus,1,,,, +B1284,Anaerococcus,1,,,, +B1284,Anaerococcus,2,,,, +B1284,Bacteroidetes,1,,,, +B1284,Bacteroidetes,1,,,, +B1284,Bacteroidetes,1,,,, +B1284,Bacteroidetes,2,,,, +B1284,Bacteroidetes,3,,,, +B1284,Bacteroidetes,43,,,, +B1284,Clostridiales,1,,,, +B1284,Clostridiales,1,,,, +B1284,Clostridiales,1,,,, +B1284,Clostridiales,1,,,, +B1284,Clostridiales,1,,,, +B1284,Clostridiales,14,,,, +B1284,Clostridiales,2,,,, +B1284,Corynebacterineae,6,,,, +B1284,Dialister,1,,,, +B1284,Dialister,11,,,, +B1284,Firmicutes,1,,,, +B1284,Firmicutes,1,,,, +B1284,Firmicutes,2,,,, +B1284,Hallella,1,,,, +B1284,Mobiluncus,1,,,, +B1284,Mobiluncus,24,,,, +B1284,Peptoniphilus,1,,,, +B1284,Peptoniphilus,2,,,, +B1284,Peptoniphilus,9,,,, +B1284,Porphyromonadaceae,1,,,, +B1284,Porphyromonadaceae,1,,,, +B1284,Porphyromonadaceae,1,,,, +B1284,Porphyromonadaceae,10,,,, +B1284,Porphyromonas,1,,,, +B1284,Porphyromonas,1,,,, +B1284,Porphyromonas,1,,,, +B1284,Porphyromonas,1,,,, +B1284,Porphyromonas,2,,,, +B1284,Porphyromonas,20,,,, +B1284,Porphyromonas,3,,,, +B1284,Prevotella,7,,,, +B1284,Prevotellaceae,1,,,, +B1284,Prevotellaceae,9,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,1,,,, +B1284,unidentified,17,,,, +B1284,unidentified,2,,,, +B1284,unidentified,2,,,, +B1284,unidentified,2,,,, +B1284,unidentified,28,,,, +B1284,unidentified,3,,,, +B1284,unidentified,3,,,, +B1284,unidentified,4,,,, +B1284,unidentified,4,,,, +B1284,unidentified,7,,,, +B1284,unidentified,81,,,, +B1284,Varibaculum,1,,,, +B1284,Varibaculum,1,,,, +B1285,Actinobacteridae,1,,,, +B1285,Actinobacteridae,1,,,, +B1285,Actinobacteridae,2,,,, +B1285,Actinobacteridae,3,,,, +B1285,Actinomycetales,1,,,, +B1285,Actinomycetales,1,,,, +B1285,Alcaligenaceae,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,1,,,, +B1285,Anaerococcus,19,,,, +B1285,Anaerococcus,2,,,, +B1285,Anaerococcus,2,,,, +B1285,Anaerococcus,2,,,, +B1285,Anaerococcus,63,,,, +B1285,Anaerococcus,8,,,, +B1285,Anaerococcus,82,,,, +B1285,Branhamella,1,,,, +B1285,Branhamella,2,,,, +B1285,Branhamella,32,,,, +B1285,Chloroplast,1,,,, +B1285,Chloroplast,1,,,, +B1285,Clostridia,1,,,, +B1285,Clostridia,1,,,, +B1285,Clostridiales,1,,,, +B1285,Clostridiales,28,,,, +B1285,Corynebacterineae,1,,,, +B1285,Corynebacterineae,3,,,, +B1285,Corynebacterineae,75,,,, +B1285,Corynebacterium,1,,,, +B1285,Corynebacterium,20,,,, +B1285,Finegoldia,1,,,, +B1285,Finegoldia,8,,,, +B1285,Lactobacillaceae,1,,,, +B1285,Moraxellaceae,1,,,, +B1285,Peptoniphilus,1,,,, +B1285,Peptoniphilus,1,,,, +B1285,Peptoniphilus,1,,,, +B1285,Peptoniphilus,7,,,, +B1285,Porphyromonadaceae,1,,,, +B1285,Prevotella,1,,,, +B1285,Staphylococcus,5,,,, +B1285,Streptococcus,1,,,, +B1285,unidentified,1,,,, +B1285,unidentified,1,,,, +B1285,unidentified,1,,,, +B1286,Actinobacteridae,3,,,, +B1286,Actinomycetaceae,2,,,, +B1286,Actinomycetales,1,,,, +B1286,Actinomycetales,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,1,,,, +B1286,Anaerococcus,11,,,, +B1286,Anaerococcus,19,,,, +B1286,Anaerococcus,2,,,, +B1286,Campylobacter,1,,,, +B1286,Campylobacter,1,,,, +B1286,Campylobacter,34,,,, +B1286,Campylobacterales,1,,,, +B1286,Campylobacterales,2,,,, +B1286,Clostridia,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,1,,,, +B1286,Clostridiales,2,,,, +B1286,Clostridiales,68,,,, +B1286,Corynebacterineae,18,,,, +B1286,Corynebacterineae,6,,,, +B1286,Corynebacterium,1,,,, +B1286,Corynebacterium,1,,,, +B1286,Dermabacter,3,,,, +B1286,Dialister,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,1,,,, +B1286,Finegoldia,28,,,, +B1286,Mobiluncus,2,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,1,,,, +B1286,Peptoniphilus,2,,,, +B1286,Peptoniphilus,3,,,, +B1286,Peptoniphilus,4,,,, +B1286,Peptoniphilus,96,,,, +B1286,Prevotella,1,,,, +B1286,Prevotella,43,,,, +B1286,Prevotellaceae,1,,,, +B1286,Ralstonia,1,,,, +B1286,unidentified,1,,,, +B1286,unidentified,1,,,, +B1286,unidentified,1,,,, +B1286,Varibaculum,1,,,, +B1286,Veillonellaceae,6,,,, +B1288,Actinobacteridae,1,,,, +B1288,Actinobacteridae,3,,,, +B1288,Actinobacteridae,3,,,, +B1288,Actinomycetaceae,1,,,, +B1288,Actinomycetales,1,,,, +B1288,Actinomycetales,1,,,, +B1288,Actinomycetales,1,,,, +B1288,Actinomycetales,1,,,, +B1288,Actinomycetales,1,,,, +B1288,Actinomycetales,1,,,, +B1288,Aerococcaceae,1,,,, +B1288,Aerococcaceae,1,,,, +B1288,Aerococcaceae,11,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,1,,,, +B1288,Anaerococcus,11,,,, +B1288,Anaerococcus,12,,,, +B1288,Anaerococcus,2,,,, +B1288,Anaerococcus,2,,,, +B1288,Anaerococcus,2,,,, +B1288,Anaerococcus,2,,,, +B1288,Anaerococcus,2,,,, +B1288,Anaerococcus,2,,,, +B1288,Anaerococcus,3,,,, +B1288,Anaerococcus,3,,,, +B1288,Anaerococcus,4,,,, +B1288,Anaerococcus,59,,,, +B1288,Arcanobacterium,1,,,, +B1288,Bacilli,1,,,, +B1288,Bacteroidetes,3,,,, +B1288,Brevibacterium,2,,,, +B1288,Campylobacter,1,,,, +B1288,Clostridia,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,1,,,, +B1288,Clostridiales,13,,,, +B1288,Clostridiales,14,,,, +B1288,Clostridiales,2,,,, +B1288,Clostridiales,2,,,, +B1288,Clostridiales,3,,,, +B1288,Clostridiales,44,,,, +B1288,Clostridiales,6,,,, +B1288,Corynebacterineae,1,,,, +B1288,Corynebacterineae,2,,,, +B1288,Corynebacterineae,2,,,, +B1288,Corynebacterineae,22,,,, +B1288,Corynebacterium,20,,,, +B1288,Dialister,1,,,, +B1288,Finegoldia,1,,,, +B1288,Firmicutes,1,,,, +B1288,Firmicutes,5,,,, +B1288,Gallicola,1,,,, +B1288,Mobiluncus,1,,,, +B1288,Peptococcus,1,,,, +B1288,Peptococcus,2,,,, +B1288,Peptoniphilus,1,,,, +B1288,Peptoniphilus,1,,,, +B1288,Peptoniphilus,1,,,, +B1288,Peptoniphilus,1,,,, +B1288,Peptoniphilus,2,,,, +B1288,Peptoniphilus,27,,,, +B1288,Peptoniphilus,7,,,, +B1288,Staphylococcus,2,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,1,,,, +B1288,unidentified,11,,,, +B1288,unidentified,2,,,, +B1288,unidentified,4,,,, +B1288,unidentified,8,,,, +B1288,Varibaculum,1,,,, +B1288,Varibaculum,1,,,, +B1288,Varibaculum,5,,,, +B1288,Veillonellaceae,2,,,, +B1289,Actinobacteridae,1,,,, +B1289,Actinobacteridae,1,,,, +B1289,Actinobacteridae,12,,,, +B1289,Actinobaculum,1,,,, +B1289,Actinomyces,2,,,, +B1289,Actinomycetaceae,1,,,, +B1289,Actinomycetales,1,,,, +B1289,Actinomycetales,1,,,, +B1289,Actinomycetales,1,,,, +B1289,Actinomycetales,1,,,, +B1289,Actinomycetales,1,,,, +B1289,Anaerococcus,1,,,, +B1289,Bacteroidales,1,,,, +B1289,Bacteroidetes,1,,,, +B1289,Bacteroidetes,1,,,, +B1289,Bacteroidetes,14,,,, +B1289,Bacteroidetes,2,,,, +B1289,Campylobacter,3,,,, +B1289,Clostridia,1,,,, +B1289,Clostridia,1,,,, +B1289,Clostridia,15,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,1,,,, +B1289,Clostridiales,2,,,, +B1289,Clostridiales,29,,,, +B1289,Clostridiales,4,,,, +B1289,Clostridiales,4,,,, +B1289,Dialister,1,,,, +B1289,Dialister,4,,,, +B1289,Firmicutes,1,,,, +B1289,Firmicutes,1,,,, +B1289,Firmicutes,1,,,, +B1289,Methanobrevibacter,11,,,, +B1289,Mobiluncus,9,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,1,,,, +B1289,Peptococcus,2,,,, +B1289,Peptococcus,3,,,, +B1289,Peptococcus,32,,,, +B1289,Peptoniphilus,1,,,, +B1289,Peptoniphilus,1,,,, +B1289,Peptoniphilus,1,,,, +B1289,Peptoniphilus,2,,,, +B1289,Peptoniphilus,2,,,, +B1289,Peptoniphilus,20,,,, +B1289,Peptoniphilus,4,,,, +B1289,Porphyromonadaceae,18,,,, +B1289,Porphyromonas,1,,,, +B1289,Porphyromonas,3,,,, +B1289,Prevotella,4,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,1,,,, +B1289,unidentified,18,,,, +B1289,unidentified,2,,,, +B1289,unidentified,2,,,, +B1289,unidentified,2,,,, +B1289,unidentified,2,,,, +B1289,unidentified,2,,,, +B1289,unidentified,2,,,, +B1289,unidentified,26,,,, +B1289,unidentified,3,,,, +B1289,unidentified,30,,,, +B1289,unidentified,41,,,, +B1291,Actinobacteridae,1,,,, +B1291,Actinobacteridae,1,,,, +B1291,Actinobacteridae,1,,,, +B1291,Actinobacteridae,13,,,, +B1291,Actinomyces,1,,,, +B1291,Actinomycetales,1,,,, +B1291,Actinomycetales,1,,,, +B1291,Aerococcaceae,1,,,, +B1291,Anaerococcus,1,,,, +B1291,Anaerococcus,1,,,, +B1291,Anaerococcus,1,,,, +B1291,Anaerococcus,1,,,, +B1291,Anaerococcus,1,,,, +B1291,Anaerococcus,1,,,, +B1291,Anaerococcus,10,,,, +B1291,Anaerococcus,2,,,, +B1291,Anaerococcus,2,,,, +B1291,Anaerococcus,2,,,, +B1291,Arcanobacterium,1,,,, +B1291,Bacteroidetes,1,,,, +B1291,Bacteroidetes,1,,,, +B1291,Bacteroidetes,1,,,, +B1291,Bacteroidetes,2,,,, +B1291,Bacteroidetes,59,,,, +B1291,Campylobacter,1,,,, +B1291,Campylobacter,3,,,, +B1291,Clostridiales,1,,,, +B1291,Clostridiales,12,,,, +B1291,Clostridiales,2,,,, +B1291,Clostridiales,3,,,, +B1291,Clostridiales,3,,,, +B1291,Clostridiales,4,,,, +B1291,Corynebacterineae,2,,,, +B1291,Corynebacterineae,4,,,, +B1291,Corynebacterium,1,,,, +B1291,Corynebacterium,3,,,, +B1291,Dermabacter,1,,,, +B1291,Dialister,4,,,, +B1291,Finegoldia,2,,,, +B1291,Firmicutes,1,,,, +B1291,Firmicutes,1,,,, +B1291,Firmicutes,3,,,, +B1291,Fusobacterium,1,,,, +B1291,Hallella,1,,,, +B1291,Lachnospiraceae,2,,,, +B1291,Lactobacillales,2,,,, +B1291,Millisia,1,,,, +B1291,Mobiluncus,1,,,, +B1291,Mobiluncus,29,,,, +B1291,Peptoniphilus,2,,,, +B1291,Peptoniphilus,2,,,, +B1291,Peptoniphilus,5,,,, +B1291,Porphyromonadaceae,8,,,, +B1291,Porphyromonas,1,,,, +B1291,Porphyromonas,19,,,, +B1291,Porphyromonas,3,,,, +B1291,Porphyromonas,8,,,, +B1291,Prevotella,1,,,, +B1291,Prevotella,1,,,, +B1291,Prevotella,1,,,, +B1291,Prevotella,1,,,, +B1291,Prevotella,3,,,, +B1291,Prevotella,3,,,, +B1291,Prevotellaceae,1,,,, +B1291,Prevotellaceae,4,,,, +B1291,Staphylococcus,3,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,1,,,, +B1291,unidentified,15,,,, +B1291,unidentified,2,,,, +B1291,unidentified,2,,,, +B1291,unidentified,29,,,, +B1291,unidentified,3,,,, +B1291,unidentified,3,,,, +B1291,unidentified,3,,,, +B1291,unidentified,3,,,, +B1291,unidentified,4,,,, +B1291,unidentified,4,,,, +B1291,unidentified,54,,,, +B1292,Actinobacteria,1,,,, +B1292,Actinobacteridae,1,,,, +B1292,Actinobacteridae,11,,,, +B1292,Actinobacteridae,2,,,, +B1292,Actinomycetaceae,1,,,, +B1292,Actinomycetales,1,,,, +B1292,Aerococcaceae,2,,,, +B1292,Anaerococcus,1,,,, +B1292,Anaerococcus,1,,,, +B1292,Anaerococcus,16,,,, +B1292,Anaerococcus,2,,,, +B1292,Anaerococcus,2,,,, +B1292,Anaerococcus,3,,,, +B1292,Anaerococcus,3,,,, +B1292,Anaerococcus,5,,,, +B1292,Bacilli,1,,,, +B1292,Bacteroidetes,1,,,, +B1292,Bacteroidetes,3,,,, +B1292,Bacteroidetes,48,,,, +B1292,Burkholderiales,1,,,, +B1292,Caenibacterium,1,,,, +B1292,Campylobacter,2,,,, +B1292,Chloroplast,1,,,, +B1292,Clostridiales,1,,,, +B1292,Clostridiales,1,,,, +B1292,Clostridiales,1,,,, +B1292,Clostridiales,13,,,, +B1292,Clostridiales,14,,,, +B1292,Clostridiales,2,,,, +B1292,Clostridiales,2,,,, +B1292,Clostridiales,4,,,, +B1292,Corynebacterineae,14,,,, +B1292,Corynebacterineae,7,,,, +B1292,Corynebacterium,1,,,, +B1292,Corynebacterium,12,,,, +B1292,Dermabacter,1,,,, +B1292,Dialister,3,,,, +B1292,Finegoldia,6,,,, +B1292,Firmicutes,1,,,, +B1292,Firmicutes,1,,,, +B1292,Lactobacillales,1,,,, +B1292,Lactobacillales,1,,,, +B1292,Lactobacillales,2,,,, +B1292,Mobiluncus,7,,,, +B1292,Peptococcus,1,,,, +B1292,Peptoniphilus,1,,,, +B1292,Peptoniphilus,1,,,, +B1292,Peptoniphilus,18,,,, +B1292,Porphyromonadaceae,13,,,, +B1292,Porphyromonas,1,,,, +B1292,Porphyromonas,1,,,, +B1292,Porphyromonas,1,,,, +B1292,Porphyromonas,1,,,, +B1292,Porphyromonas,1,,,, +B1292,Porphyromonas,4,,,, +B1292,Porphyromonas,8,,,, +B1292,Prevotella,1,,,, +B1292,Prevotella,2,,,, +B1292,Prevotella,4,,,, +B1292,Prevotellaceae,3,,,, +B1292,Ruminococcaceae,2,,,, +B1292,Sphingobacterium,1,,,, +B1292,Staphylococcus,9,,,, +B1292,unidentified,1,,,, +B1292,unidentified,1,,,, +B1292,unidentified,1,,,, +B1292,unidentified,1,,,, +B1292,unidentified,1,,,, +B1292,unidentified,1,,,, +B1292,unidentified,1,,,, +B1292,unidentified,14,,,, +B1292,unidentified,2,,,, +B1292,unidentified,2,,,, +B1292,unidentified,2,,,, +B1292,unidentified,3,,,, +B1292,unidentified,3,,,, +B1292,unidentified,30,,,, +B1292,unidentified,45,,,, +B1292,unidentified,7,,,, +B1292,Varibaculum,2,,,, +B1292,Veillonellaceae,4,,,, +B1292,Xanthomonadaceae,1,,,, +B1293,Abiotrophia,1,,,, +B1293,Acidobacteriaceae,1,,,, +B1293,Acinetobacter,1,,,, +B1293,Actinobacteridae,1,,,, +B1293,Actinobacteridae,1,,,, +B1293,Actinobacteridae,9,,,, +B1293,Aerococcaceae,1,,,, +B1293,Alphaproteobacteria,1,,,, +B1293,Anaerococcus,1,,,, +B1293,Anaerococcus,1,,,, +B1293,Anaerococcus,1,,,, +B1293,Anaerococcus,1,,,, +B1293,Anaerococcus,1,,,, +B1293,Anaerococcus,14,,,, +B1293,Anaerococcus,2,,,, +B1293,Anaerococcus,2,,,, +B1293,Anaerococcus,8,,,, +B1293,Bacteroidetes,1,,,, +B1293,Bacteroidetes,1,,,, +B1293,Bacteroidetes,41,,,, +B1293,Campylobacter,3,,,, +B1293,Clostridiales,1,,,, +B1293,Clostridiales,1,,,, +B1293,Clostridiales,1,,,, +B1293,Clostridiales,12,,,, +B1293,Clostridiales,13,,,, +B1293,Clostridiales,2,,,, +B1293,Clostridiales,2,,,, +B1293,Clostridiales,2,,,, +B1293,Clostridiales,2,,,, +B1293,Corynebacterineae,12,,,, +B1293,Corynebacterineae,4,,,, +B1293,Corynebacterium,1,,,, +B1293,Corynebacterium,9,,,, +B1293,Dermabacter,1,,,, +B1293,Dialister,7,,,, +B1293,Finegoldia,2,,,, +B1293,Firmicutes,2,,,, +B1293,Lachnospiraceae,1,,,, +B1293,Mobiluncus,17,,,, +B1293,Peptoniphilus,1,,,, +B1293,Peptoniphilus,1,,,, +B1293,Peptoniphilus,1,,,, +B1293,Peptoniphilus,2,,,, +B1293,Peptoniphilus,20,,,, +B1293,Porphyromonadaceae,10,,,, +B1293,Porphyromonas,1,,,, +B1293,Porphyromonas,11,,,, +B1293,Porphyromonas,19,,,, +B1293,Prevotella,1,,,, +B1293,Prevotella,1,,,, +B1293,Prevotella,3,,,, +B1293,Prevotellaceae,1,,,, +B1293,Prevotellaceae,5,,,, +B1293,Staphylococcus,3,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,1,,,, +B1293,unidentified,10,,,, +B1293,unidentified,14,,,, +B1293,unidentified,2,,,, +B1293,unidentified,2,,,, +B1293,unidentified,2,,,, +B1293,unidentified,4,,,, +B1293,unidentified,40,,,, +B1293,unidentified,46,,,, +B1293,Varibaculum,2,,,, +B1293,Veillonellaceae,2,,,, +B1294,Actinobacteridae,1,,,, +B1294,Actinobacteridae,7,,,, +B1294,Actinomycetaceae,2,,,, +B1294,Actinomycetales,1,,,, +B1294,Actinomycetales,1,,,, +B1294,Aerococcaceae,1,,,, +B1294,Aerococcaceae,1,,,, +B1294,Aerococcaceae,19,,,, +B1294,Aerococcaceae,2,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,1,,,, +B1294,Anaerococcus,10,,,, +B1294,Anaerococcus,10,,,, +B1294,Anaerococcus,14,,,, +B1294,Anaerococcus,2,,,, +B1294,Anaerococcus,2,,,, +B1294,Anaerococcus,2,,,, +B1294,Anaerococcus,2,,,, +B1294,Anaerococcus,3,,,, +B1294,Anaerococcus,40,,,, +B1294,Bacilli,1,,,, +B1294,Bacilli,1,,,, +B1294,Bacilli,1,,,, +B1294,Bacteroidetes,1,,,, +B1294,Bacteroidetes,20,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,1,,,, +B1294,Clostridiales,2,,,, +B1294,Clostridiales,29,,,, +B1294,Clostridiales,3,,,, +B1294,Clostridiales,9,,,, +B1294,Corynebacterineae,1,,,, +B1294,Corynebacterineae,1,,,, +B1294,Corynebacterineae,1,,,, +B1294,Corynebacterineae,14,,,, +B1294,Corynebacterineae,2,,,, +B1294,Corynebacterineae,57,,,, +B1294,Corynebacterium,1,,,, +B1294,Corynebacterium,2,,,, +B1294,Corynebacterium,26,,,, +B1294,Dermabacter,1,,,, +B1294,Fastidiosipila,2,,,, +B1294,Finegoldia,8,,,, +B1294,Firmicutes,1,,,, +B1294,Firmicutes,1,,,, +B1294,Ignavigranum,6,,,, +B1294,Microbacteriaceae,1,,,, +B1294,Microbacteriaceae,22,,,, +B1294,Peptococcus,1,,,, +B1294,Peptococcus,1,,,, +B1294,Peptoniphilus,1,,,, +B1294,Peptoniphilus,1,,,, +B1294,Peptoniphilus,1,,,, +B1294,Peptoniphilus,3,,,, +B1294,Peptoniphilus,4,,,, +B1294,Peptoniphilus,4,,,, +B1294,Peptoniphilus,9,,,, +B1294,Porphyromonas,7,,,, +B1294,Staphylococcus,9,,,, +B1294,unidentified,1,,,, +B1294,unidentified,1,,,, +B1294,unidentified,2,,,, +B1294,unidentified,2,,,, +B1295,Abiotrophia,3,,,, +B1295,Acetobacteraceae,1,,,, +B1295,Actinobacteridae,1,,,, +B1295,Actinobacteridae,7,,,, +B1295,Actinomyces,1,,,, +B1295,Aerococcaceae,2,,,, +B1295,Anaerococcus,1,,,, +B1295,Anaerococcus,1,,,, +B1295,Anaerococcus,10,,,, +B1295,Anaerococcus,7,,,, +B1295,Bacillales,1,,,, +B1295,Bacillales,1,,,, +B1295,Bacillales,1,,,, +B1295,Bacillales,1,,,, +B1295,Bacilli,1,,,, +B1295,Bacilli,6,,,, +B1295,Bacteroidetes,1,,,, +B1295,Bacteroidetes,1,,,, +B1295,Bacteroidetes,1,,,, +B1295,Bacteroidetes,1,,,, +B1295,Bacteroidetes,10,,,, +B1295,Branhamella,2,,,, +B1295,Burkholderiales,1,,,, +B1295,Campylobacter,2,,,, +B1295,Clostridiales,1,,,, +B1295,Clostridiales,1,,,, +B1295,Clostridiales,1,,,, +B1295,Clostridiales,2,,,, +B1295,Clostridiales,51,,,, +B1295,Corynebacterineae,2,,,, +B1295,Corynebacterineae,9,,,, +B1295,Corynebacterium,6,,,, +B1295,Finegoldia,12,,,, +B1295,Firmicutes,1,,,, +B1295,Gemella,1,,,, +B1295,GpI,1,,,, +B1295,Halobacteriaceae,1,,,, +B1295,Ignavigranum,1,,,, +B1295,Lactobacillales,1,,,, +B1295,Methanobrevibacter,1,,,, +B1295,Microbacteriaceae,2,,,, +B1295,Micrococcaceae,1,,,, +B1295,Mobiluncus,1,,,, +B1295,Neochlamydia,1,,,, +B1295,Nocardioides,1,,,, +B1295,Paenibacillaceae,1,,,, +B1295,Pasteurellaceae,1,,,, +B1295,Peptococcus,2,,,, +B1295,Peptoniphilus,11,,,, +B1295,Peptoniphilus,2,,,, +B1295,Peptoniphilus,5,,,, +B1295,Phenylobacterium,2,,,, +B1295,Porphyromonadaceae,3,,,, +B1295,Porphyromonas,4,,,, +B1295,Porphyromonas,5,,,, +B1295,Prevotella,1,,,, +B1295,Prevotella,3,,,, +B1295,Ralstonia,2,,,, +B1295,Staphylococcus,1,,,, +B1295,Staphylococcus,146,,,, +B1295,Streptococcus,1,,,, +B1295,Streptococcus,12,,,, +B1295,Streptococcus,2,,,, +B1295,unidentified,1,,,, +B1295,unidentified,1,,,, +B1295,unidentified,1,,,, +B1295,unidentified,2,,,, +B1295,unidentified,24,,,, +B1295,unidentified,3,,,, +B1295,Veillonellaceae,2,,,, +B1296,Actinomyces,20,,,, +B1296,Anaerococcus,1,,,, +B1296,Anaerococcus,1,,,, +B1296,Anaerococcus,1,,,, +B1296,Anaerococcus,13,,,, +B1296,Bacillales,1,,,, +B1296,Bacilli,1,,,, +B1296,Bacilli,1,,,, +B1296,Bacilli,1,,,, +B1296,Bacilli,1,,,, +B1296,Bacilli,1,,,, +B1296,Bacilli,2,,,, +B1296,Bacilli,2,,,, +B1296,Bacilli,3,,,, +B1296,Bacilli,4,,,, +B1296,Bacilli,5,,,, +B1296,Bradyrhizobiaceae,1,,,, +B1296,Campylobacter,1,,,, +B1296,Corynebacterium,9,,,, +B1296,Dialister,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,1,,,, +B1296,Finegoldia,100,,,, +B1296,Finegoldia,3,,,, +B1296,Finegoldia,3,,,, +B1296,Finegoldia,3,,,, +B1296,Fusobacterium,1,,,, +B1296,Lactobacillales,1,,,, +B1296,Lactobacillales,27,,,, +B1296,Micrococcineae,1,,,, +B1296,Peptoniphilus,1,,,, +B1296,Peptoniphilus,1,,,, +B1296,Peptoniphilus,16,,,, +B1296,Staphylococcaceae,2,,,, +B1296,Staphylococcus,1,,,, +B1296,Staphylococcus,1,,,, +B1296,Staphylococcus,107,,,, +B1296,Streptococcus,37,,,, +B1296,unidentified,1,,,, +B1296,unidentified,1,,,, +B1296,Varibaculum,1,,,, +B1296,Veillonella,1,,,, +B1296,Veillonella,1,,,, +B1296,Veillonella,1,,,, +B1296,Veillonella,1,,,, +B1296,Veillonella,1,,,, +B1296,Veillonella,3,,,, +B1296,Veillonella,5,,,, +B1296,Veillonellaceae,1,,,, +B1298,Actinobacteridae,1,,,, +B1298,Actinobacteridae,2,,,, +B1298,Actinobacteridae,2,,,, +B1298,Actinobaculum,1,,,, +B1298,Actinomyces,1,,,, +B1298,Actinomycetaceae,5,,,, +B1298,Aerococcaceae,1,,,, +B1298,Aerococcaceae,1,,,, +B1298,Anaerococcus,1,,,, +B1298,Anaerococcus,1,,,, +B1298,Anaerococcus,1,,,, +B1298,Anaerococcus,1,,,, +B1298,Anaerococcus,1,,,, +B1298,Anaerococcus,1,,,, +B1298,Anaerococcus,2,,,, +B1298,Anaerococcus,5,,,, +B1298,Anaerococcus,7,,,, +B1298,Bacilli,1,,,, +B1298,Bacteroidetes,1,,,, +B1298,Bacteroidetes,1,,,, +B1298,Bacteroidetes,1,,,, +B1298,Bacteroidetes,15,,,, +B1298,Branhamella,2,,,, +B1298,Campylobacter,2,,,, +B1298,Clostridia,13,,,, +B1298,Clostridiales,1,,,, +B1298,Clostridiales,1,,,, +B1298,Clostridiales,1,,,, +B1298,Clostridiales,1,,,, +B1298,Clostridiales,1,,,, +B1298,Clostridiales,2,,,, +B1298,Clostridiales,2,,,, +B1298,Clostridiales,2,,,, +B1298,Clostridiales,2,,,, +B1298,Clostridiales,35,,,, +B1298,Corynebacterineae,1,,,, +B1298,Corynebacterineae,2,,,, +B1298,Corynebacterineae,20,,,, +B1298,Corynebacterium,2,,,, +B1298,Dermabacter,1,,,, +B1298,Dialister,2,,,, +B1298,Finegoldia,4,,,, +B1298,Firmicutes,1,,,, +B1298,Firmicutes,1,,,, +B1298,Firmicutes,1,,,, +B1298,Methanobrevibacter,9,,,, +B1298,Mobiluncus,8,,,, +B1298,Peptococcus,1,,,, +B1298,Peptococcus,33,,,, +B1298,Peptoniphilus,1,,,, +B1298,Peptoniphilus,17,,,, +B1298,Peptoniphilus,2,,,, +B1298,Peptoniphilus,4,,,, +B1298,Peptoniphilus,6,,,, +B1298,Porphyromonadaceae,12,,,, +B1298,Porphyromonas,1,,,, +B1298,Porphyromonas,1,,,, +B1298,Porphyromonas,5,,,, +B1298,Prevotella,1,,,, +B1298,Prevotella,4,,,, +B1298,Staphylococcus,1,,,, +B1298,Staphylococcus,26,,,, +B1298,unidentified,1,,,, +B1298,unidentified,1,,,, +B1298,unidentified,1,,,, +B1298,unidentified,1,,,, +B1298,unidentified,1,,,, +B1298,unidentified,1,,,, +B1298,unidentified,20,,,, +B1298,unidentified,21,,,, +B1298,unidentified,23,,,, +B1298,unidentified,45,,,, +B1308,Acinetobacter,1,,,, +B1308,Actinobacteridae,1,,,, +B1308,Actinobacteridae,11,,,, +B1308,Actinobacteridae,74,,,, +B1308,Actinomyces,2,,,, +B1308,Actinomyces,7,,,, +B1308,Aerococcaceae,3,,,, +B1308,Aeromonas,1,,,, +B1308,Anaerococcus,1,,,, +B1308,Anaerococcus,1,,,, +B1308,Anaerococcus,1,,,, +B1308,Anaerococcus,2,,,, +B1308,Anaerococcus,3,,,, +B1308,Anaerococcus,7,,,, +B1308,Bacillaceae,1,,,, +B1308,Bacillales,1,,,, +B1308,Bacillales,1,,,, +B1308,Bacillales,2,,,, +B1308,Bacilli,1,,,, +B1308,Bacteroidetes,1,,,, +B1308,Bacteroidetes,33,,,, +B1308,Burkholderiales,1,,,, +B1308,Campylobacter,1,,,, +B1308,Campylobacter,12,,,, +B1308,Clostridiales,11,,,, +B1308,Corynebacterineae,1,,,, +B1308,Corynebacterium,36,,,, +B1308,Dialister,1,,,, +B1308,Fastidiosipila,1,,,, +B1308,Finegoldia,1,,,, +B1308,Finegoldia,1,,,, +B1308,Finegoldia,1,,,, +B1308,Finegoldia,13,,,, +B1308,Mobiluncus,1,,,, +B1308,Neisseria,1,,,, +B1308,Novosphingobium,1,,,, +B1308,Paenibacillaceae,1,,,, +B1308,Pasteurellaceae,1,,,, +B1308,Peptoniphilus,10,,,, +B1308,Peptoniphilus,2,,,, +B1308,Peptoniphilus,3,,,, +B1308,Peptostreptococcus,4,,,, +B1308,Porphyromonas,5,,,, +B1308,Prevotella,4,,,, +B1308,Prevotella,43,,,, +B1308,Rhodococcus,1,,,, +B1308,Sphingomonas,2,,,, +B1308,Staphylococcus,1,,,, +B1308,Staphylococcus,1,,,, +B1308,Staphylococcus,60,,,, +B1308,unidentified,1,,,, +B1308,unidentified,1,,,, +B1308,unidentified,1,,,, +B1308,unidentified,1,,,, +B1308,unidentified,1,,,, +B1308,unidentified,6,,,, +B1308,unidentified,7,,,, +B1308,Veillonellaceae,6,,,, +B1310,Actinobacteridae,1,,,, +B1310,Actinobacteridae,4,,,, +B1310,Actinomyces,2,,,, +B1310,Actinomycetaceae,3,,,, +B1310,Aerococcaceae,11,,,, +B1310,Anaerococcus,1,,,, +B1310,Anaerococcus,1,,,, +B1310,Anaerococcus,1,,,, +B1310,Anaerococcus,1,,,, +B1310,Anaerococcus,1,,,, +B1310,Anaerococcus,1,,,, +B1310,Anaerococcus,10,,,, +B1310,Anaerococcus,2,,,, +B1310,Anaerococcus,21,,,, +B1310,Anaerococcus,3,,,, +B1310,Anaerococcus,3,,,, +B1310,Bacteroidetes,1,,,, +B1310,Bacteroidetes,1,,,, +B1310,Bacteroidetes,1,,,, +B1310,Bacteroidetes,1,,,, +B1310,Bacteroidetes,16,,,, +B1310,Bacteroidetes,3,,,, +B1310,Branhamella,2,,,, +B1310,Brevundimonas,1,,,, +B1310,Campylobacter,8,,,, +B1310,Clostridia,4,,,, +B1310,Clostridiales,1,,,, +B1310,Clostridiales,2,,,, +B1310,Clostridiales,2,,,, +B1310,Clostridiales,3,,,, +B1310,Clostridiales,4,,,, +B1310,Clostridiales,5,,,, +B1310,Clostridiales,5,,,, +B1310,Clostridiales,50,,,, +B1310,Comamonadaceae,1,,,, +B1310,Corynebacterineae,2,,,, +B1310,Corynebacterineae,24,,,, +B1310,Corynebacterium,19,,,, +B1310,Dermabacter,1,,,, +B1310,Dialister,5,,,, +B1310,Finegoldia,11,,,, +B1310,Firmicutes,1,,,, +B1310,Helcococcus,1,,,, +B1310,Ignavigranum,3,,,, +B1310,Lachnospiraceae,1,,,, +B1310,Methanobrevibacter,2,,,, +B1310,Microbacteriaceae,1,,,, +B1310,Microbacteriaceae,5,,,, +B1310,Mobiluncus,1,,,, +B1310,Mobiluncus,8,,,, +B1310,Peptococcus,1,,,, +B1310,Peptococcus,11,,,, +B1310,Peptoniphilus,1,,,, +B1310,Peptoniphilus,1,,,, +B1310,Peptoniphilus,1,,,, +B1310,Peptoniphilus,18,,,, +B1310,Peptoniphilus,4,,,, +B1310,Peptoniphilus,7,,,, +B1310,Porphyromonadaceae,4,,,, +B1310,Porphyromonas,1,,,, +B1310,Porphyromonas,1,,,, +B1310,Porphyromonas,5,,,, +B1310,Prevotella,3,,,, +B1310,Rothia,1,,,, +B1310,Rubrobacteraceae,1,,,, +B1310,Staphylococcus,13,,,, +B1310,Streptococcus,1,,,, +B1310,Streptococcus,2,,,, +B1310,unidentified,1,,,, +B1310,unidentified,1,,,, +B1310,unidentified,1,,,, +B1310,unidentified,1,,,, +B1310,unidentified,1,,,, +B1310,unidentified,1,,,, +B1310,unidentified,1,,,, +B1310,unidentified,33,,,, +B1310,unidentified,4,,,, +B1310,unidentified,6,,,, +B1310,unidentified,9,,,, +B1310,Veillonellaceae,3,,,, +B1374,Actinobacteridae,1,,,, +B1374,Actinobacteridae,1,,,, +B1374,Actinobacteridae,2,,,, +B1374,Actinomyces,4,,,, +B1374,Anaerococcus,1,,,, +B1374,Anaerococcus,1,,,, +B1374,Anaerococcus,1,,,, +B1374,Anaerococcus,1,,,, +B1374,Anaerococcus,4,,,, +B1374,Anaerococcus,57,,,, +B1374,Anaerococcus,8,,,, +B1374,Aquabacterium,1,,,, +B1374,Bacillales,1,,,, +B1374,Bacillales,1,,,, +B1374,Bacilli,1,,,, +B1374,Bacilli,1,,,, +B1374,Bacilli,2,,,, +B1374,Burkholderiales,1,,,, +B1374,Chloroplast,1,,,, +B1374,Clostridiales,1,,,, +B1374,Clostridiales,1,,,, +B1374,Clostridiales,1,,,, +B1374,Corynebacterineae,1,,,, +B1374,Corynebacterineae,1,,,, +B1374,Corynebacterineae,1,,,, +B1374,Corynebacterineae,1,,,, +B1374,Corynebacterineae,85,,,, +B1374,Corynebacterium,1,,,, +B1374,Corynebacterium,36,,,, +B1374,Corynebacterium,5,,,, +B1374,Dermabacter,3,,,, +B1374,Finegoldia,23,,,, +B1374,Paenibacillaceae,11,,,, +B1374,Peptococcus,2,,,, +B1374,Peptoniphilus,1,,,, +B1374,Peptoniphilus,16,,,, +B1374,Peptoniphilus,5,,,, +B1374,Peptoniphilus,7,,,, +B1374,Pseudomonas,1,,,, +B1374,Ralstonia,2,,,, +B1374,Sphingomonas,1,,,, +B1374,Staphylococcus,1,,,, +B1374,Staphylococcus,1,,,, +B1374,Staphylococcus,96,,,, +B1374,unidentified,1,,,, +B1374,unidentified,1,,,, +B1374,Varibaculum,3,,,, +B1374,Zoogloea,1,,,, +B940,Actinobacteridae,4,,,, +B940,Actinomyces,1,,,, +B940,Actinomycetaceae,1,,,, +B940,Actinomycetales,1,,,, +B940,Actinomycetales,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,1,,,, +B940,Anaerococcus,2,,,, +B940,Anaerococcus,4,,,, +B940,Anaerococcus,6,,,, +B940,Bacteroidetes,47,,,, +B940,Campylobacter,1,,,, +B940,Campylobacter,1,,,, +B940,Campylobacter,1,,,, +B940,Campylobacter,16,,,, +B940,Clostridia,6,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,1,,,, +B940,Clostridiales,2,,,, +B940,Clostridiales,20,,,, +B940,Clostridiales,3,,,, +B940,Clostridiales,5,,,, +B940,Clostridiales,5,,,, +B940,Clostridiales,5,,,, +B940,Clostridiales,5,,,, +B940,Clostridiales,5,,,, +B940,Clostridiales,5,,,, +B940,Clostridiales,8,,,, +B940,Clostridiales,9,,,, +B940,Corynebacterineae,1,,,, +B940,Corynebacterium,2,,,, +B940,Fastidiosipila,1,,,, +B940,Finegoldia,2,,,, +B940,Firmicutes,1,,,, +B940,Firmicutes,1,,,, +B940,Firmicutes,2,,,, +B940,Gallicola,1,,,, +B940,Gallicola,3,,,, +B940,Mobiluncus,1,,,, +B940,Peptococcus,1,,,, +B940,Peptococcus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,1,,,, +B940,Peptoniphilus,38,,,, +B940,Porphyromonadaceae,2,,,, +B940,Porphyromonas,1,,,, +B940,Porphyromonas,17,,,, +B940,Porphyromonas,2,,,, +B940,Prevotellaceae,1,,,, +B940,Staphylococcus,4,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,1,,,, +B940,unidentified,13,,,, +B940,unidentified,16,,,, +B940,unidentified,17,,,, +B940,unidentified,2,,,, +B940,unidentified,3,,,, +B940,unidentified,37,,,, +B940,unidentified,5,,,, +B940,unidentified,7,,,, +B940,Varibaculum,1,,,, +B940,Varibaculum,12,,,, +B940,Veillonellaceae,3,,,, +B941,Actinobacteridae,1,,,, +B941,Actinobacteridae,1,,,, +B941,Aerococcaceae,2,,,, +B941,Anaerococcus,1,,,, +B941,Anaerococcus,15,,,, +B941,Anaerococcus,2,,,, +B941,Anaerococcus,3,,,, +B941,Anaerococcus,36,,,, +B941,Anaerococcus,5,,,, +B941,Anaerococcus,60,,,, +B941,Bacteroidetes,1,,,, +B941,Bacteroidetes,1,,,, +B941,Bacteroidetes,1,,,, +B941,Bacteroidetes,9,,,, +B941,Betaproteobacteria,1,,,, +B941,Burkholderiales,1,,,, +B941,Campylobacter,1,,,, +B941,Campylobacter,1,,,, +B941,Campylobacter,39,,,, +B941,Clostridiales,1,,,, +B941,Clostridiales,1,,,, +B941,Clostridiales,2,,,, +B941,Corynebacterineae,1,,,, +B941,Corynebacterineae,7,,,, +B941,Corynebacterium,2,,,, +B941,Corynebacterium,64,,,, +B941,Dermabacter,4,,,, +B941,Finegoldia,1,,,, +B941,Finegoldia,1,,,, +B941,Finegoldia,71,,,, +B941,Fusobacterium,1,,,, +B941,Lactobacillales,1,,,, +B941,Moryella,1,,,, +B941,Peptoniphilus,1,,,, +B941,Peptoniphilus,10,,,, +B941,Porphyromonadaceae,1,,,, +B941,Prevotella,14,,,, +B941,Prevotellaceae,2,,,, +B941,Staphylococcus,15,,,, +B941,Streptococcus,1,,,, +B941,unidentified,1,,,, +B941,unidentified,1,,,, +B941,unidentified,1,,,, +B941,unidentified,1,,,, +B941,unidentified,2,,,, +B941,unidentified,3,,,, +B941,Varibaculum,3,,,, +B941,Veillonellaceae,5,,,, +B944,Acinetobacter,4,,,, +B944,Actinobacillus,1,,,, +B944,Actinobacteridae,1,,,, +B944,Actinobacteridae,1,,,, +B944,Actinobacteridae,1,,,, +B944,Actinobacteridae,1,,,, +B944,Aerococcaceae,1,,,, +B944,Anaerococcus,3,,,, +B944,Anaerococcus,3,,,, +B944,Bradyrhizobiaceae,1,,,, +B944,Brevundimonas,3,,,, +B944,Burkholderiales,3,,,, +B944,Burkholderiales,7,,,, +B944,Campylobacter,2,,,, +B944,Clostridiales,1,,,, +B944,Corynebacterineae,1,,,, +B944,Corynebacterineae,2,,,, +B944,Corynebacterineae,2,,,, +B944,Corynebacterineae,238,,,, +B944,Corynebacterium,1,,,, +B944,Corynebacterium,1,,,, +B944,Corynebacterium,1,,,, +B944,Corynebacterium,1,,,, +B944,Corynebacterium,15,,,, +B944,Corynebacterium,4,,,, +B944,Corynebacterium,7,,,, +B944,Dermacoccus,2,,,, +B944,Enhydrobacter,29,,,, +B944,Faecalibacterium,1,,,, +B944,Faecalibacterium,1,,,, +B944,Finegoldia,2,,,, +B944,Firmicutes,1,,,, +B944,Gemella,3,,,, +B944,Georgenia,1,,,, +B944,Kocuria,15,,,, +B944,Lachnospiraceae,1,,,, +B944,Marinomonas,4,,,, +B944,Micrococcaceae,1,,,, +B944,Nocardioidaceae,2,,,, +B944,Pantoea,1,,,, +B944,Pasteurellaceae,1,,,, +B944,Pasteurellaceae,1,,,, +B944,Pasteurellaceae,6,,,, +B944,Ruminococcaceae,1,,,, +B944,Ruminococcaceae,2,,,, +B944,Sphingomonas,1,,,, +B944,Staphylococcus,8,,,, +B944,Streptococcus,2,,,, +B944,Streptococcus,3,,,, +B944,unidentified,1,,,, +B944,unidentified,1,,,, +B944,unidentified,3,,,, +B945,Acinetobacter,1,,,, +B945,Actinobacteridae,2,,,, +B945,Actinobacteridae,3,,,, +B945,Actinobacteridae,3,,,, +B945,Actinomycetales,1,,,, +B945,Anaerococcus,1,,,, +B945,Anaerococcus,1,,,, +B945,Anaerococcus,1,,,, +B945,Anaerococcus,1,,,, +B945,Anaerococcus,1,,,, +B945,Anaerococcus,1,,,, +B945,Anaerococcus,10,,,, +B945,Anaerococcus,2,,,, +B945,Anaerococcus,2,,,, +B945,Anaerococcus,3,,,, +B945,Anaerococcus,30,,,, +B945,Anaerococcus,4,,,, +B945,Bacillales,1,,,, +B945,Bacillales,1,,,, +B945,Bacteroidetes,1,,,, +B945,Bacteroidetes,12,,,, +B945,Betaproteobacteria,1,,,, +B945,Brevundimonas,5,,,, +B945,Campylobacter,1,,,, +B945,Campylobacter,2,,,, +B945,Clostridiales,1,,,, +B945,Clostridiales,1,,,, +B945,Clostridiales,1,,,, +B945,Clostridiales,1,,,, +B945,Clostridiales,5,,,, +B945,Coriobacteriaceae,1,,,, +B945,Corynebacterineae,1,,,, +B945,Corynebacterineae,1,,,, +B945,Corynebacterineae,1,,,, +B945,Corynebacterineae,2,,,, +B945,Corynebacterineae,3,,,, +B945,Corynebacterineae,4,,,, +B945,Corynebacterineae,43,,,, +B945,Corynebacterium,12,,,, +B945,Corynebacterium,2,,,, +B945,Corynebacterium,87,,,, +B945,Dialister,1,,,, +B945,Enhydrobacter,1,,,, +B945,Finegoldia,1,,,, +B945,Finegoldia,7,,,, +B945,Janthinobacterium,2,,,, +B945,Oxalobacteraceae,1,,,, +B945,Peptoniphilus,1,,,, +B945,Peptoniphilus,1,,,, +B945,Peptoniphilus,1,,,, +B945,Peptoniphilus,17,,,, +B945,Peptoniphilus,5,,,, +B945,Porphyromonas,1,,,, +B945,Porphyromonas,6,,,, +B945,Prevotella,7,,,, +B945,Prevotellaceae,3,,,, +B945,Pseudoxanthomonas,1,,,, +B945,Sphingopyxis,1,,,, +B945,Staphylococcus,60,,,, +B945,Streptococcus,1,,,, +B945,Thermoactinomyces,1,,,, +B945,unidentified,1,,,, +B945,unidentified,1,,,, +B945,unidentified,4,,,, +B945,unidentified,5,,,, +B945,unidentified,6,,,, +B945,Veillonellaceae,7,,,, +B946,Actinobacteridae,20,,,, +B946,Actinobacteridae,6,,,, +B946,Actinomyces,1,,,, +B946,Actinomycetales,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,1,,,, +B946,Anaerococcus,2,,,, +B946,Anaerococcus,2,,,, +B946,Anaerococcus,2,,,, +B946,Anaerococcus,4,,,, +B946,Anaerococcus,6,,,, +B946,Anaerococcus,7,,,, +B946,Bacteroidetes,1,,,, +B946,Bacteroidetes,1,,,, +B946,Bacteroidetes,1,,,, +B946,Bacteroidetes,1,,,, +B946,Bacteroidetes,1,,,, +B946,Bacteroidetes,2,,,, +B946,Bacteroidetes,42,,,, +B946,Campylobacter,3,,,, +B946,Campylobacter,5,,,, +B946,Clostridia,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,1,,,, +B946,Clostridiales,2,,,, +B946,Clostridiales,3,,,, +B946,Clostridiales,3,,,, +B946,Clostridiales,31,,,, +B946,Corynebacterineae,1,,,, +B946,Corynebacterineae,1,,,, +B946,Corynebacterium,1,,,, +B946,Dialister,2,,,, +B946,Finegoldia,1,,,, +B946,Finegoldia,1,,,, +B946,Finegoldia,1,,,, +B946,Finegoldia,18,,,, +B946,Firmicutes,1,,,, +B946,Gallicola,1,,,, +B946,Mobiluncus,3,,,, +B946,Peptococcus,7,,,, +B946,Peptoniphilus,1,,,, +B946,Peptoniphilus,1,,,, +B946,Peptoniphilus,1,,,, +B946,Peptoniphilus,1,,,, +B946,Peptoniphilus,1,,,, +B946,Peptoniphilus,14,,,, +B946,Peptoniphilus,42,,,, +B946,Peptoniphilus,8,,,, +B946,Porphyromonadaceae,1,,,, +B946,Porphyromonas,1,,,, +B946,Porphyromonas,1,,,, +B946,Porphyromonas,1,,,, +B946,Porphyromonas,12,,,, +B946,Porphyromonas,5,,,, +B946,Prevotella,15,,,, +B946,Prevotellaceae,1,,,, +B946,Prevotellaceae,13,,,, +B946,Staphylococcus,1,,,, +B946,Turicella,1,,,, +B946,unidentified,1,,,, +B946,unidentified,1,,,, +B946,unidentified,1,,,, +B946,unidentified,1,,,, +B946,unidentified,1,,,, +B946,unidentified,1,,,, +B946,unidentified,1,,,, +B946,unidentified,19,,,, +B946,unidentified,25,,,, +B946,unidentified,3,,,, +B946,unidentified,4,,,, +B946,Veillonellaceae,17,,,, +B947,Actinobacteridae,1,,,, +B947,Actinobacteridae,2,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,1,,,, +B947,Anaerococcus,10,,,, +B947,Anaerococcus,101,,,, +B947,Anaerococcus,2,,,, +B947,Anaerococcus,2,,,, +B947,Anaerococcus,28,,,, +B947,Bacilli,1,,,, +B947,Bacteroidetes,1,,,, +B947,Bacteroidetes,5,,,, +B947,Campylobacter,10,,,, +B947,Campylobacter,2,,,, +B947,Chloroplast,1,,,, +B947,Clostridiales,1,,,, +B947,Clostridiales,1,,,, +B947,Clostridiales,1,,,, +B947,Clostridiales,1,,,, +B947,Clostridiales,3,,,, +B947,Corynebacterineae,1,,,, +B947,Corynebacterineae,1,,,, +B947,Corynebacterineae,10,,,, +B947,Corynebacterineae,4,,,, +B947,Corynebacterium,36,,,, +B947,Finegoldia,1,,,, +B947,Finegoldia,1,,,, +B947,Finegoldia,50,,,, +B947,Fusobacterium,1,,,, +B947,Peptococcus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,1,,,, +B947,Peptoniphilus,2,,,, +B947,Peptoniphilus,56,,,, +B947,Porphyromonas,1,,,, +B947,Porphyromonas,3,,,, +B947,Prevotella,2,,,, +B947,Staphylococcus,19,,,, +B947,unidentified,2,,,, +B947,unidentified,3,,,, +B947,unidentified,4,,,, +B947,Veillonellaceae,5,,,, +B948,Actinobacteridae,1,,,, +B948,Actinobacteridae,1,,,, +B948,Actinobacteridae,1,,,, +B948,Actinobacteridae,1,,,, +B948,Actinobacteridae,3,,,, +B948,Actinomycetaceae,4,,,, +B948,Actinomycetales,1,,,, +B948,Actinomycetales,1,,,, +B948,Actinomycetales,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,1,,,, +B948,Anaerococcus,19,,,, +B948,Anaerococcus,2,,,, +B948,Anaerococcus,4,,,, +B948,Anaerococcus,4,,,, +B948,Anaeroglobus,1,,,, +B948,Bacteroidetes,6,,,, +B948,Brevibacterium,9,,,, +B948,Campylobacter,1,,,, +B948,Campylobacter,3,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,1,,,, +B948,Clostridiales,4,,,, +B948,Clostridiales,7,,,, +B948,Clostridiales,8,,,, +B948,Corynebacterineae,1,,,, +B948,Corynebacterineae,1,,,, +B948,Corynebacterineae,1,,,, +B948,Corynebacterineae,1,,,, +B948,Corynebacterineae,72,,,, +B948,Corynebacterineae,81,,,, +B948,Corynebacterium,1,,,, +B948,Corynebacterium,17,,,, +B948,Corynebacterium,4,,,, +B948,Corynebacterium,4,,,, +B948,Dermabacter,1,,,, +B948,Dialister,6,,,, +B948,Enhydrobacter,1,,,, +B948,Finegoldia,1,,,, +B948,Finegoldia,1,,,, +B948,Finegoldia,21,,,, +B948,Firmicutes,2,,,, +B948,Fusobacterium,10,,,, +B948,Microbacteriaceae,2,,,, +B948,Mobiluncus,1,,,, +B948,Parvimonas,3,,,, +B948,Peptococcus,4,,,, +B948,Peptoniphilus,1,,,, +B948,Peptoniphilus,1,,,, +B948,Peptoniphilus,9,,,, +B948,Peptoniphilus,9,,,, +B948,Porphyromonas,1,,,, +B948,Porphyromonas,6,,,, +B948,Prevotella,2,,,, +B948,Staphylococcus,1,,,, +B948,unidentified,1,,,, +B948,unidentified,1,,,, +B948,unidentified,1,,,, +B948,unidentified,1,,,, +B948,unidentified,1,,,, +B948,unidentified,1,,,, +B948,unidentified,1,,,, +B948,unidentified,14,,,, +B948,unidentified,2,,,, +B948,unidentified,5,,,, +B948,unidentified,6,,,, +B950,Acinetobacter,1,,,, +B950,Acinetobacter,4,,,, +B950,Actinobacillus,1,,,, +B950,Actinobacteridae,3,,,, +B950,Actinomycetales,1,,,, +B950,Actinomycetales,1,,,, +B950,Actinomycetales,1,,,, +B950,Actinomycetales,1,,,, +B950,Anaerococcus,2,,,, +B950,Aquabacterium,4,,,, +B950,Bacillaceae,1,,,, +B950,Bacillales,1,,,, +B950,Bacilli,1,,,, +B950,Bacilli,1,,,, +B950,Bacilli,5,,,, +B950,Bacteroidales,1,,,, +B950,Bacteroidetes,2,,,, +B950,Bosea,1,,,, +B950,Bradyrhizobiaceae,3,,,, +B950,Burkholderiales,1,,,, +B950,Burkholderiales,1,,,, +B950,Burkholderiales,1,,,, +B950,Burkholderiales,2,,,, +B950,Caulobacter,2,,,, +B950,Chloroplast,2,,,, +B950,Comamonadaceae,1,,,, +B950,Corynebacterineae,1,,,, +B950,Corynebacterineae,2,,,, +B950,Corynebacterineae,58,,,, +B950,Corynebacterium,9,,,, +B950,Firmicutes,1,,,, +B950,Firmicutes,1,,,, +B950,Firmicutes,1,,,, +B950,Firmicutes,1,,,, +B950,Fusobacterium,3,,,, +B950,Gemella,1,,,, +B950,Intrasporangiaceae,1,,,, +B950,Leptotrichia,1,,,, +B950,Leptotrichia,2,,,, +B950,Neisseria,2,,,, +B950,Neisseriaceae,1,,,, +B950,Pasteurellaceae,2,,,, +B950,Pasteurellaceae,2,,,, +B950,Peptoniphilus,3,,,, +B950,Prevotella,1,,,, +B950,Propionibacterium,1,,,, +B950,Pseudomonas,1,,,, +B950,Pseudomonas,3,,,, +B950,Rhodocyclaceae,3,,,, +B950,Sphingomonas,1,,,, +B950,Staphylococcaceae,1,,,, +B950,Staphylococcus,1,,,, +B950,Staphylococcus,1,,,, +B950,Staphylococcus,1,,,, +B950,Staphylococcus,1,,,, +B950,Staphylococcus,1,,,, +B950,Staphylococcus,235,,,, +B950,Streptococcus,1,,,, +B950,Streptococcus,3,,,, +B950,Streptococcus,3,,,, +B950,unidentified,1,,,, +B950,Xanthomonadaceae,2,,,, +B950,Zoogloea,2,,,, +B952,Actinobacteridae,1,,,, +B952,Actinobacteridae,1,,,, +B952,Actinobacteridae,5,,,, +B952,Actinobacteridae,6,,,, +B952,Actinobacteridae,7,,,, +B952,Actinomycetales,4,,,, +B952,Aerococcaceae,4,,,, +B952,Anaerococcus,1,,,, +B952,Anaerococcus,1,,,, +B952,Anaerococcus,13,,,, +B952,Anaerococcus,13,,,, +B952,Anaerococcus,2,,,, +B952,Anaerococcus,29,,,, +B952,Anaerococcus,3,,,, +B952,Bacteroidetes,1,,,, +B952,Brevibacterium,19,,,, +B952,Burkholderiales,2,,,, +B952,Clostridiales,1,,,, +B952,Clostridiales,1,,,, +B952,Clostridiales,1,,,, +B952,Clostridiales,1,,,, +B952,Clostridiales,2,,,, +B952,Clostridiales,2,,,, +B952,Clostridiales,2,,,, +B952,Clostridiales,3,,,, +B952,Clostridiales,6,,,, +B952,Corynebacterineae,1,,,, +B952,Corynebacterineae,1,,,, +B952,Corynebacterineae,2,,,, +B952,Corynebacterineae,2,,,, +B952,Corynebacterineae,23,,,, +B952,Corynebacterineae,7,,,, +B952,Corynebacterineae,93,,,, +B952,Corynebacterium,18,,,, +B952,Corynebacterium,4,,,, +B952,Corynebacterium,43,,,, +B952,Dermabacter,2,,,, +B952,Enhydrobacter,1,,,, +B952,Finegoldia,1,,,, +B952,Finegoldia,1,,,, +B952,Finegoldia,1,,,, +B952,Finegoldia,17,,,, +B952,Microbacteriaceae,1,,,, +B952,Microbacteriaceae,1,,,, +B952,Microbacteriaceae,1,,,, +B952,Microbacteriaceae,18,,,, +B952,Nocardioides,1,,,, +B952,Peptoniphilus,1,,,, +B952,Peptoniphilus,1,,,, +B952,Prevotella,1,,,, +B952,Staphylococcus,1,,,, +B952,Streptococcus,1,,,, +B952,unidentified,1,,,, +B952,unidentified,1,,,, +B952,unidentified,1,,,, +B952,unidentified,4,,,, +B952,Zimmermannella,18,,,, +B953,Actinobacteridae,1,,,, +B953,Actinobacteridae,1,,,, +B953,Actinobacteridae,1,,,, +B953,Actinobacteridae,3,,,, +B953,Actinobacteridae,3,,,, +B953,Anaerococcus,1,,,, +B953,Anaerococcus,1,,,, +B953,Anaerococcus,3,,,, +B953,Anaerococcus,5,,,, +B953,Bacillales,1,,,, +B953,Bacillales,3,,,, +B953,Bacilli,2,,,, +B953,Corynebacterineae,1,,,, +B953,Corynebacterineae,1,,,, +B953,Corynebacterineae,1,,,, +B953,Corynebacterineae,1,,,, +B953,Corynebacterineae,167,,,, +B953,Corynebacterineae,2,,,, +B953,Corynebacterineae,2,,,, +B953,Corynebacterium,1,,,, +B953,Corynebacterium,1,,,, +B953,Corynebacterium,1,,,, +B953,Corynebacterium,13,,,, +B953,Corynebacterium,33,,,, +B953,Dermabacter,1,,,, +B953,Dermabacter,10,,,, +B953,Finegoldia,1,,,, +B953,Oxalobacteraceae,1,,,, +B953,Staphylococcaceae,1,,,, +B953,Staphylococcaceae,3,,,, +B953,Staphylococcus,1,,,, +B953,Staphylococcus,133,,,, +B954,Actinobacteridae,1,,,, +B954,Actinobacteridae,2,,,, +B954,Actinomycetales,3,,,, +B954,Bacillales,1,,,, +B954,Bacilli,1,,,, +B954,Bacilli,1,,,, +B954,Bacilli,4,,,, +B954,Brevundimonas,1,,,, +B954,Brevundimonas,1,,,, +B954,Brevundimonas,6,,,, +B954,Burkholderiales,1,,,, +B954,Corynebacterineae,1,,,, +B954,Corynebacterineae,1,,,, +B954,Corynebacterineae,1,,,, +B954,Corynebacterineae,1,,,, +B954,Corynebacterineae,114,,,, +B954,Corynebacterineae,2,,,, +B954,Corynebacterium,1,,,, +B954,Corynebacterium,1,,,, +B954,Corynebacterium,1,,,, +B954,Corynebacterium,1,,,, +B954,Corynebacterium,136,,,, +B954,Corynebacterium,4,,,, +B954,Oxalobacteraceae,1,,,, +B954,Rhodocyclaceae,1,,,, +B954,Sphingomonas,1,,,, +B954,Sphingopyxis,1,,,, +B954,Staphylococcus,109,,,, +B954,unidentified,1,,,, +B955,Acinetobacter,1,,,, +B955,Acinetobacter,2,,,, +B955,Actinobacteridae,1,,,, +B955,Actinobacteridae,1,,,, +B955,Actinobacteridae,4,,,, +B955,Actinobacteridae,9,,,, +B955,Actinomyces,1,,,, +B955,Actinomycetaceae,1,,,, +B955,Anaerococcus,1,,,, +B955,Anaerococcus,1,,,, +B955,Anaerococcus,1,,,, +B955,Anaerococcus,1,,,, +B955,Anaerococcus,1,,,, +B955,Anaerococcus,1,,,, +B955,Anaerococcus,11,,,, +B955,Anaerococcus,13,,,, +B955,Anaerococcus,54,,,, +B955,Bacilli,1,,,, +B955,Bacteroidetes,1,,,, +B955,Bacteroidetes,1,,,, +B955,Bacteroidetes,8,,,, +B955,Brevundimonas,4,,,, +B955,Burkholderiales,9,,,, +B955,Campylobacter,2,,,, +B955,Campylobacter,2,,,, +B955,Chloroplast,2,,,, +B955,Clostridiales,1,,,, +B955,Clostridiales,1,,,, +B955,Clostridiales,3,,,, +B955,Corynebacterineae,1,,,, +B955,Corynebacterineae,1,,,, +B955,Corynebacterineae,1,,,, +B955,Corynebacterineae,2,,,, +B955,Corynebacterineae,2,,,, +B955,Corynebacterineae,76,,,, +B955,Corynebacterium,1,,,, +B955,Corynebacterium,1,,,, +B955,Corynebacterium,22,,,, +B955,Corynebacterium,24,,,, +B955,Dermabacter,3,,,, +B955,Dialister,2,,,, +B955,Finegoldia,1,,,, +B955,Finegoldia,1,,,, +B955,Finegoldia,1,,,, +B955,Finegoldia,1,,,, +B955,Finegoldia,32,,,, +B955,Oxalobacteraceae,1,,,, +B955,Peptococcus,4,,,, +B955,Peptoniphilus,1,,,, +B955,Peptoniphilus,1,,,, +B955,Peptoniphilus,1,,,, +B955,Peptoniphilus,2,,,, +B955,Peptoniphilus,33,,,, +B955,Peptoniphilus,4,,,, +B955,Porphyromonas,1,,,, +B955,Prevotella,3,,,, +B955,Prevotellaceae,6,,,, +B955,Sphingopyxis,1,,,, +B955,Staphylococcus,14,,,, +B955,unidentified,1,,,, +B955,unidentified,1,,,, +B955,unidentified,2,,,, +B955,unidentified,4,,,, +B955,unidentified,6,,,, +B955,Veillonellaceae,3,,,, +B956,Acinetobacter,2,,,, +B956,Actinobacteridae,1,,,, +B956,Actinobacteridae,3,,,, +B956,Actinomyces,1,,,, +B956,Actinomycetaceae,2,,,, +B956,Anaerococcus,1,,,, +B956,Anaerococcus,1,,,, +B956,Anaerococcus,1,,,, +B956,Anaerococcus,16,,,, +B956,Anaerococcus,2,,,, +B956,Anaerococcus,2,,,, +B956,Anaerococcus,26,,,, +B956,Anaeroglobus,1,,,, +B956,Bacilli,1,,,, +B956,Bacilli,1,,,, +B956,Bacteroidetes,1,,,, +B956,Bacteroidetes,20,,,, +B956,Branhamella,8,,,, +B956,Campylobacter,1,,,, +B956,Campylobacter,4,,,, +B956,Clostridia,5,,,, +B956,Clostridiales,1,,,, +B956,Clostridiales,1,,,, +B956,Clostridiales,1,,,, +B956,Clostridiales,1,,,, +B956,Clostridiales,1,,,, +B956,Clostridiales,2,,,, +B956,Clostridiales,3,,,, +B956,Clostridiales,3,,,, +B956,Clostridiales,4,,,, +B956,Clostridiales,5,,,, +B956,Clostridiales,5,,,, +B956,Clostridiales,7,,,, +B956,Clostridiales,8,,,, +B956,Corynebacterineae,1,,,, +B956,Corynebacterineae,1,,,, +B956,Corynebacterineae,14,,,, +B956,Corynebacterineae,18,,,, +B956,Corynebacterium,20,,,, +B956,Corynebacterium,3,,,, +B956,Finegoldia,2,,,, +B956,Firmicutes,4,,,, +B956,Fusobacterium,2,,,, +B956,Gammaproteobacteria,1,,,, +B956,Micrococcus,1,,,, +B956,Parvimonas,1,,,, +B956,Peptococcus,1,,,, +B956,Peptococcus,4,,,, +B956,Peptoniphilus,1,,,, +B956,Peptoniphilus,13,,,, +B956,Peptoniphilus,3,,,, +B956,Peptoniphilus,4,,,, +B956,Porphyromonadaceae,1,,,, +B956,Porphyromonas,1,,,, +B956,Porphyromonas,1,,,, +B956,Porphyromonas,7,,,, +B956,Staphylococcaceae,2,,,, +B956,Staphylococcus,1,,,, +B956,Staphylococcus,92,,,, +B956,unidentified,1,,,, +B956,unidentified,1,,,, +B956,unidentified,1,,,, +B956,unidentified,1,,,, +B956,unidentified,12,,,, +B956,unidentified,17,,,, +B956,unidentified,2,,,, +B956,unidentified,4,,,, +B956,unidentified,5,,,, +B956,unidentified,9,,,, +B956,Varibaculum,5,,,, +B956,Zimmermannella,1,,,, +B957,Acinetobacter,1,,,, +B957,Actinobacteridae,1,,,, +B957,Actinobacteridae,1,,,, +B957,Actinomycetales,1,,,, +B957,Actinomycetales,1,,,, +B957,Actinomycetales,1,,,, +B957,Anaerococcus,1,,,, +B957,Anaerococcus,1,,,, +B957,Anaerococcus,1,,,, +B957,Anaerococcus,1,,,, +B957,Anaerococcus,12,,,, +B957,Anaerococcus,14,,,, +B957,Anaerococcus,49,,,, +B957,Bacillaceae,1,,,, +B957,Bacillales,1,,,, +B957,Bacillales,1,,,, +B957,Bacillales,1,,,, +B957,Bacilli,2,,,, +B957,Betaproteobacteria,1,,,, +B957,Corynebacterineae,1,,,, +B957,Corynebacterineae,1,,,, +B957,Corynebacterineae,3,,,, +B957,Corynebacterineae,50,,,, +B957,Corynebacterineae,75,,,, +B957,Corynebacterium,3,,,, +B957,Finegoldia,1,,,, +B957,Firmicutes,2,,,, +B957,Millisia,1,,,, +B957,Prevotella,1,,,, +B957,Pseudomonadaceae,1,,,, +B957,Staphylococcaceae,1,,,, +B957,Staphylococcaceae,1,,,, +B957,Staphylococcus,1,,,, +B957,Staphylococcus,164,,,, +B957,Stenotrophomonas,1,,,, +B957,unidentified,1,,,, +B958,Acinetobacter,2,,,, +B958,Actinomycetales,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,1,,,, +B958,Anaerococcus,125,,,, +B958,Anaerococcus,14,,,, +B958,Anaerococcus,2,,,, +B958,Anaerococcus,3,,,, +B958,Bacillales,1,,,, +B958,Bacillales,1,,,, +B958,Bacilli,1,,,, +B958,Bacilli,1,,,, +B958,Branhamella,2,,,, +B958,Corynebacterineae,1,,,, +B958,Corynebacterineae,178,,,, +B958,Corynebacterium,1,,,, +B958,Corynebacterium,1,,,, +B958,Corynebacterium,14,,,, +B958,Firmicutes,1,,,, +B958,Gammaproteobacteria,1,,,, +B958,Staphylococcaceae,1,,,, +B958,Staphylococcus,39,,,, +B958,unidentified,2,,,, +B961,Acinetobacter,3,,,, +B961,Actinobacteridae,13,,,, +B961,Actinobacteridae,5,,,, +B961,Aerococcaceae,1,,,, +B961,Anaerococcus,1,,,, +B961,Anaerococcus,1,,,, +B961,Anaerococcus,1,,,, +B961,Anaerococcus,7,,,, +B961,Bacillales,1,,,, +B961,Bacillales,1,,,, +B961,Bacillales,2,,,, +B961,Bacilli,3,,,, +B961,Bacilli,7,,,, +B961,Bacteroidetes,5,,,, +B961,Bifidobacteriaceae,2,,,, +B961,Burkholderiales,1,,,, +B961,Corynebacterineae,2,,,, +B961,Corynebacterium,4,,,, +B961,Finegoldia,1,,,, +B961,Finegoldia,1,,,, +B961,Finegoldia,21,,,, +B961,Firmicutes,2,,,, +B961,Peptoniphilus,1,,,, +B961,Peptoniphilus,1,,,, +B961,Peptoniphilus,1,,,, +B961,Peptoniphilus,1,,,, +B961,Peptoniphilus,11,,,, +B961,Peptoniphilus,2,,,, +B961,Prevotella,5,,,, +B961,Pseudomonas,1,,,, +B961,Staphylococcaceae,1,,,, +B961,Staphylococcus,1,,,, +B961,Staphylococcus,1,,,, +B961,Staphylococcus,287,,,, +B961,unidentified,1,,,, +B961,Varibaculum,1,,,, +B962,Acinetobacter,1,,,, +B962,Acinetobacter,10,,,, +B962,Betaproteobacteria,1,,,, +B962,Bosea,5,,,, +B962,Bradyrhizobiaceae,1,,,, +B962,Brevibacterium,4,,,, +B962,Brevundimonas,1,,,, +B962,Brevundimonas,1,,,, +B962,Brevundimonas,2,,,, +B962,Brevundimonas,54,,,, +B962,Brevundimonas,8,,,, +B962,Burkholderiales,1,,,, +B962,Burkholderiales,1,,,, +B962,Burkholderiales,1,,,, +B962,Burkholderiales,1,,,, +B962,Burkholderiales,125,,,, +B962,Burkholderiales,3,,,, +B962,Comamonadaceae,1,,,, +B962,Corynebacterium,1,,,, +B962,Enhydrobacter,1,,,, +B962,Incertae sedis 5,3,,,, +B962,Lactobacillales,1,,,, +B962,Lactococcus,12,,,, +B962,Methylobacterium,1,,,, +B962,Micrococcineae,1,,,, +B962,Oxalobacteraceae,10,,,, +B962,Pantoea,1,,,, +B962,Prevotella,2,,,, +B962,Pseudomonadaceae,2,,,, +B962,Pseudomonas,5,,,, +B962,Ralstonia,7,,,, +B962,Rhodobacteraceae,1,,,, +B962,Sphingomonadaceae,1,,,, +B962,Sphingomonas,1,,,, +B962,Sphingomonas,1,,,, +B962,Sphingomonas,35,,,, +B962,Sphingopyxis,72,,,, +B962,Staphylococcus,12,,,, +B962,Streptococcus,6,,,, +B962,unidentified,1,,,, +B962,Xanthomonadaceae,2,,,, +B963,Acinetobacter,6,,,, +B963,Actinobacteridae,1,,,, +B963,Actinobacteridae,1,,,, +B963,Actinobacteridae,3,,,, +B963,Actinobaculum,1,,,, +B963,Actinomyces,2,,,, +B963,Actinomycetaceae,2,,,, +B963,Actinomycetales,1,,,, +B963,Actinomycetales,1,,,, +B963,Aerococcaceae,26,,,, +B963,Anaerococcus,1,,,, +B963,Anaerococcus,1,,,, +B963,Anaerococcus,1,,,, +B963,Anaerococcus,27,,,, +B963,Anaerococcus,36,,,, +B963,Bacteroidetes,1,,,, +B963,Bacteroidetes,12,,,, +B963,Bilophila,1,,,, +B963,Bradyrhizobiaceae,1,,,, +B963,Brevundimonas,1,,,, +B963,Brevundimonas,7,,,, +B963,Burkholderiales,13,,,, +B963,Campylobacter,3,,,, +B963,Clostridiales,1,,,, +B963,Clostridiales,1,,,, +B963,Clostridiales,12,,,, +B963,Clostridiales,2,,,, +B963,Clostridiales,3,,,, +B963,Clostridiales,3,,,, +B963,Clostridiales,3,,,, +B963,Clostridiales,6,,,, +B963,Corynebacterineae,17,,,, +B963,Corynebacterineae,44,,,, +B963,Corynebacterium,1,,,, +B963,Corynebacterium,1,,,, +B963,Corynebacterium,18,,,, +B963,Corynebacterium,3,,,, +B963,Desulfovibrio,2,,,, +B963,Desulfovibrio,6,,,, +B963,Finegoldia,5,,,, +B963,Firmicutes,4,,,, +B963,Intrasporangiaceae,2,,,, +B963,Lactobacillales,1,,,, +B963,Mobiluncus,3,,,, +B963,Oxalobacteraceae,3,,,, +B963,Peptococcus,4,,,, +B963,Peptoniphilus,1,,,, +B963,Peptoniphilus,2,,,, +B963,Peptoniphilus,20,,,, +B963,Peptoniphilus,3,,,, +B963,Porphyromonadaceae,2,,,, +B963,Porphyromonas,1,,,, +B963,Porphyromonas,9,,,, +B963,Prevotella,1,,,, +B963,Prevotellaceae,1,,,, +B963,Sphingomonas,3,,,, +B963,Sphingopyxis,11,,,, +B963,Staphylococcus,9,,,, +B963,Streptococcus,1,,,, +B963,unidentified,1,,,, +B963,unidentified,1,,,, +B963,unidentified,1,,,, +B963,unidentified,1,,,, +B963,unidentified,1,,,, +B963,unidentified,1,,,, +B963,unidentified,1,,,, +B963,unidentified,11,,,, +B963,unidentified,2,,,, +B963,unidentified,9,,,, +B963,unidentified,9,,,, +B963,Veillonellaceae,4,,,, +B966,Abiotrophia,6,,,, +B966,Acidobacteriaceae,2,,,, +B966,Acinetobacter,1,,,, +B966,Acinetobacter,1,,,, +B966,Acinetobacter,3,,,, +B966,Actinobacteridae,1,,,, +B966,Actinobacteridae,2,,,, +B966,Actinobacteridae,3,,,, +B966,Actinobacteridae,6,,,, +B966,Aerococcaceae,5,,,, +B966,Anaerococcus,1,,,, +B966,Anaerococcus,1,,,, +B966,Anaerococcus,1,,,, +B966,Anaerococcus,2,,,, +B966,Anaerococcus,2,,,, +B966,Anaerococcus,4,,,, +B966,Bacteroidetes,1,,,, +B966,Brachybacterium,6,,,, +B966,Branhamella,8,,,, +B966,Brevibacterium,2,,,, +B966,Burkholderiales,1,,,, +B966,Chloroplast,1,,,, +B966,Chloroplast,1,,,, +B966,Chloroplast,12,,,, +B966,Clostridiales,1,,,, +B966,Clostridiales,1,,,, +B966,Comamonadaceae,1,,,, +B966,Corynebacterineae,4,,,, +B966,Corynebacterineae,77,,,, +B966,Corynebacterium,108,,,, +B966,Corynebacterium,11,,,, +B966,Corynebacterium,2,,,, +B966,Corynebacterium,6,,,, +B966,Corynebacterium,7,,,, +B966,Dolosigranulum,7,,,, +B966,Enhydrobacter,2,,,, +B966,Finegoldia,2,,,, +B966,Fusobacterium,2,,,, +B966,Gammaproteobacteria,1,,,, +B966,Gammaproteobacteria,3,,,, +B966,Gemella,1,,,, +B966,Halobacteriaceae,1,,,, +B966,Halobacteriaceae,3,,,, +B966,Lachnospiraceae,2,,,, +B966,Leptotrichia,1,,,, +B966,Leptotrichia,1,,,, +B966,Leptotrichia,1,,,, +B966,Marmoricola,1,,,, +B966,Micrococcaceae,1,,,, +B966,Micrococcineae,1,,,, +B966,Micrococcus,5,,,, +B966,Neisseria,2,,,, +B966,Nitrosomonas,1,,,, +B966,Peptoniphilus,1,,,, +B966,Peptoniphilus,2,,,, +B966,Phenylobacterium,1,,,, +B966,Porphyromonadaceae,1,,,, +B966,Porphyromonadaceae,3,,,, +B966,Propionibacterium,1,,,, +B966,Pseudomonas,1,,,, +B966,Pseudomonas,3,,,, +B966,Sphingomonadaceae,1,,,, +B966,Staphylococcaceae,1,,,, +B966,Staphylococcaceae,2,,,, +B966,Staphylococcus,19,,,, +B966,Streptococcus,1,,,, +B966,Streptococcus,1,,,, +B966,Streptococcus,15,,,, +B966,Thermomonosporaceae,1,,,, +B966,Treponema,2,,,, +B966,unidentified,1,,,, +B966,unidentified,1,,,, +B966,unidentified,1,,,, +B966,unidentified,1,,,, +B966,unidentified,1,,,, +B966,unidentified,1,,,, +B966,unidentified,3,,,, +B966,Veillonella,2,,,, +B966,Xanthomonadaceae,2,,,, +B966,Zoogloea,2,,,, +B967,Actinobacteridae,1,,,, +B967,Actinomyces,1,,,, +B967,Actinomyces,1,,,, +B967,Actinomycetales,1,,,, +B967,Aerococcaceae,3,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,1,,,, +B967,Anaerococcus,178,,,, +B967,Anaerococcus,2,,,, +B967,Anaerococcus,29,,,, +B967,Bacilli,1,,,, +B967,Bacilli,1,,,, +B967,Campylobacter,1,,,, +B967,Campylobacter,35,,,, +B967,Campylobacterales,1,,,, +B967,Campylobacterales,1,,,, +B967,Campylobacterales,1,,,, +B967,Clostridiales,1,,,, +B967,Clostridiales,1,,,, +B967,Clostridiales,1,,,, +B967,Corynebacterineae,3,,,, +B967,Corynebacterium,1,,,, +B967,Corynebacterium,11,,,, +B967,Corynebacterium,17,,,, +B967,Dermabacter,1,,,, +B967,Dermabacter,17,,,, +B967,Dermabacter,2,,,, +B967,Dermabacter,2,,,, +B967,Finegoldia,1,,,, +B967,Finegoldia,1,,,, +B967,Finegoldia,1,,,, +B967,Finegoldia,1,,,, +B967,Finegoldia,1,,,, +B967,Finegoldia,47,,,, +B967,Micrococcineae,1,,,, +B967,Micrococcineae,2,,,, +B967,Staphylococcus,2,,,, +B967,unidentified,1,,,, +B967,unidentified,3,,,, +B967,Veillonellaceae,16,,,, +B968,Actinobacteridae,1,,,, +B968,Actinobacteridae,1,,,, +B968,Actinomycetales,1,,,, +B968,Actinomycetales,1,,,, +B968,Aerococcaceae,1,,,, +B968,Aerococcaceae,6,,,, +B968,Anaerococcus,1,,,, +B968,Anaerococcus,1,,,, +B968,Anaerococcus,1,,,, +B968,Anaerococcus,1,,,, +B968,Anaerococcus,1,,,, +B968,Anaerococcus,1,,,, +B968,Anaerococcus,2,,,, +B968,Anaerococcus,3,,,, +B968,Anaerococcus,4,,,, +B968,Anaerococcus,4,,,, +B968,Anaerococcus,4,,,, +B968,Anaerococcus,5,,,, +B968,Anaerococcus,5,,,, +B968,Anaerococcus,57,,,, +B968,Anaerococcus,9,,,, +B968,Bacillales,1,,,, +B968,Bacilli,1,,,, +B968,Bacteroidetes,1,,,, +B968,Bacteroidetes,1,,,, +B968,Brachybacterium,1,,,, +B968,Bradyrhizobiaceae,1,,,, +B968,Clostridia,1,,,, +B968,Clostridiales,1,,,, +B968,Clostridiales,1,,,, +B968,Clostridiales,4,,,, +B968,Corynebacterineae,1,,,, +B968,Corynebacterineae,1,,,, +B968,Corynebacterineae,1,,,, +B968,Corynebacterineae,133,,,, +B968,Corynebacterium,1,,,, +B968,Corynebacterium,4,,,, +B968,Corynebacterium,97,,,, +B968,Dermabacter,2,,,, +B968,Finegoldia,11,,,, +B968,Firmicutes,1,,,, +B968,Peptoniphilus,1,,,, +B968,Peptoniphilus,2,,,, +B968,Peptoniphilus,3,,,, +B968,Prevotella,1,,,, +B968,Pseudomonas,1,,,, +B968,Ralstonia,1,,,, +B968,Sphingomonadaceae,1,,,, +B968,Staphylococcus,9,,,, +B968,unidentified,2,,,, +B968,Veillonellaceae,3,,,, +B968,Zimmermannella,1,,,, +B971,Acinetobacter,5,,,, +B971,Actinobacteridae,1,,,, +B971,Actinobacteridae,3,,,, +B971,Actinobacteridae,3,,,, +B971,Actinomycetales,1,,,, +B971,Actinomycetales,1,,,, +B971,Actinomycetales,1,,,, +B971,Aerococcaceae,3,,,, +B971,Anaerococcus,1,,,, +B971,Anaerococcus,1,,,, +B971,Anaerococcus,1,,,, +B971,Anaerococcus,1,,,, +B971,Anaerococcus,1,,,, +B971,Anaerococcus,15,,,, +B971,Anaerococcus,20,,,, +B971,Anaerococcus,27,,,, +B971,Anaerococcus,3,,,, +B971,Anaerococcus,3,,,, +B971,Anaerococcus,3,,,, +B971,Anaerococcus,35,,,, +B971,Aquabacterium,1,,,, +B971,Bacteroidetes,3,,,, +B971,Brevibacterium,1,,,, +B971,Brevibacterium,1,,,, +B971,Burkholderiales,1,,,, +B971,Clostridiales,1,,,, +B971,Clostridiales,1,,,, +B971,Corynebacterineae,1,,,, +B971,Corynebacterineae,1,,,, +B971,Corynebacterineae,1,,,, +B971,Corynebacterineae,1,,,, +B971,Corynebacterineae,1,,,, +B971,Corynebacterineae,158,,,, +B971,Corynebacterineae,4,,,, +B971,Corynebacterium,1,,,, +B971,Corynebacterium,1,,,, +B971,Corynebacterium,3,,,, +B971,Corynebacterium,53,,,, +B971,Corynebacterium,9,,,, +B971,Dialister,2,,,, +B971,Gammaproteobacteria,1,,,, +B971,Lactobacillales,4,,,, +B971,Neisseria,1,,,, +B971,Peptoniphilus,1,,,, +B971,Peptoniphilus,8,,,, +B971,Rhizobiales,1,,,, +B971,Rhodobacteraceae,1,,,, +B971,Staphylococcus,5,,,, +B971,unidentified,4,,,, +B972,Acinetobacter,4,,,, +B972,Actinobacteridae,1,,,, +B972,Actinobacteridae,1,,,, +B972,Actinobacteridae,6,,,, +B972,Actinobacteridae,7,,,, +B972,Actinomyces,1,,,, +B972,Actinomyces,3,,,, +B972,Actinomycetales,1,,,, +B972,Aerococcaceae,10,,,, +B972,Anaerococcus,1,,,, +B972,Anaerococcus,1,,,, +B972,Anaerococcus,1,,,, +B972,Anaerococcus,1,,,, +B972,Anaerococcus,24,,,, +B972,Anaerococcus,30,,,, +B972,Anaerococcus,4,,,, +B972,Anaerococcus,5,,,, +B972,Anaerococcus,5,,,, +B972,Anaerococcus,7,,,, +B972,Anaerococcus,7,,,, +B972,Bacillales,1,,,, +B972,Bacillales,1,,,, +B972,Bacilli,1,,,, +B972,Bacteroidetes,1,,,, +B972,Bacteroidetes,2,,,, +B972,Bacteroidetes,8,,,, +B972,Branhamella,2,,,, +B972,Brevibacterium,1,,,, +B972,Burkholderiales,2,,,, +B972,Campylobacter,6,,,, +B972,Clostridia,3,,,, +B972,Clostridiales,1,,,, +B972,Clostridiales,1,,,, +B972,Clostridiales,1,,,, +B972,Clostridiales,2,,,, +B972,Clostridiales,3,,,, +B972,Clostridiales,5,,,, +B972,Clostridiales,7,,,, +B972,Corynebacterineae,1,,,, +B972,Corynebacterineae,2,,,, +B972,Corynebacterineae,24,,,, +B972,Corynebacterineae,33,,,, +B972,Corynebacterium,1,,,, +B972,Corynebacterium,13,,,, +B972,Corynebacterium,3,,,, +B972,Corynebacterium,3,,,, +B972,Corynebacterium,5,,,, +B972,Dermabacter,2,,,, +B972,Finegoldia,1,,,, +B972,Finegoldia,13,,,, +B972,Firmicutes,1,,,, +B972,Firmicutes,3,,,, +B972,Gordoniaceae,2,,,, +B972,Lactobacillales,4,,,, +B972,Mobiluncus,5,,,, +B972,Peptococcus,2,,,, +B972,Peptoniphilus,1,,,, +B972,Peptoniphilus,35,,,, +B972,Porphyromonas,1,,,, +B972,Porphyromonas,4,,,, +B972,Prevotella,1,,,, +B972,Prevotella,3,,,, +B972,Pseudomonadaceae,1,,,, +B972,Pseudomonadaceae,1,,,, +B972,Staphylococcus,1,,,, +B972,Staphylococcus,1,,,, +B972,Staphylococcus,34,,,, +B972,unidentified,1,,,, +B972,unidentified,1,,,, +B972,unidentified,10,,,, +B972,unidentified,2,,,, +B972,unidentified,3,,,, +B972,unidentified,5,,,, +B972,unidentified,6,,,, +B972,Varibaculum,3,,,, +B974,Actinobacteridae,1,,,, +B974,Actinobacteridae,1,,,, +B974,Actinobacteridae,16,,,, +B974,Actinomycetaceae,1,,,, +B974,Actinomycetales,1,,,, +B974,Actinomycetales,1,,,, +B974,Actinomycetales,1,,,, +B974,Aerococcaceae,1,,,, +B974,Aerococcaceae,88,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,1,,,, +B974,Anaerococcus,16,,,, +B974,Anaerococcus,2,,,, +B974,Anaerococcus,2,,,, +B974,Anaerococcus,2,,,, +B974,Anaerococcus,28,,,, +B974,Anaerococcus,3,,,, +B974,Anaerococcus,39,,,, +B974,Anaerococcus,4,,,, +B974,Anaerococcus,54,,,, +B974,Anaerococcus,6,,,, +B974,Brevibacterium,1,,,, +B974,Clostridia,1,,,, +B974,Clostridiales,1,,,, +B974,Clostridiales,1,,,, +B974,Clostridiales,10,,,, +B974,Clostridiales,13,,,, +B974,Corynebacterineae,23,,,, +B974,Corynebacterineae,28,,,, +B974,Corynebacterium,2,,,, +B974,Corynebacterium,9,,,, +B974,Finegoldia,2,,,, +B974,Helcococcus,1,,,, +B974,Ignavigranum,2,,,, +B974,Peptoniphilus,1,,,, +B974,Peptoniphilus,10,,,, +B974,Peptoniphilus,6,,,, +B974,Staphylococcus,1,,,, +B974,unidentified,1,,,, +B974,unidentified,4,,,, +B974,Veillonellaceae,1,,,, +B975,Actinobacteridae,1,,,, +B975,Actinobacteridae,1,,,, +B975,Actinobacteridae,1,,,, +B975,Actinobacteridae,9,,,, +B975,Actinomyces,1,,,, +B975,Anaerococcus,1,,,, +B975,Anaerococcus,1,,,, +B975,Anaerococcus,1,,,, +B975,Anaerococcus,1,,,, +B975,Anaerococcus,1,,,, +B975,Anaerococcus,17,,,, +B975,Anaerococcus,2,,,, +B975,Anaerococcus,2,,,, +B975,Anaerococcus,2,,,, +B975,Anaerococcus,2,,,, +B975,Anaerococcus,21,,,, +B975,Anaerococcus,3,,,, +B975,Anaerococcus,6,,,, +B975,Bacillales,1,,,, +B975,Bacillales,1,,,, +B975,Bacilli,1,,,, +B975,Bacilli,1,,,, +B975,Bacilli,1,,,, +B975,Bacilli,1,,,, +B975,Bacteroidetes,2,,,, +B975,Bacteroidetes,8,,,, +B975,Bradyrhizobiaceae,1,,,, +B975,Campylobacter,1,,,, +B975,Chloroplast,1,,,, +B975,Clostridiales,1,,,, +B975,Clostridiales,1,,,, +B975,Clostridiales,1,,,, +B975,Clostridiales,2,,,, +B975,Clostridiales,5,,,, +B975,Coriobacteriaceae,1,,,, +B975,Corynebacterineae,2,,,, +B975,Corynebacterineae,36,,,, +B975,Corynebacterium,1,,,, +B975,Corynebacterium,36,,,, +B975,Dermabacter,1,,,, +B975,Dermabacter,12,,,, +B975,Dialister,2,,,, +B975,Fastidiosipila,1,,,, +B975,Finegoldia,5,,,, +B975,Firmicutes,1,,,, +B975,Firmicutes,1,,,, +B975,Lactobacillales,1,,,, +B975,Lactobacillales,1,,,, +B975,Peptoniphilus,13,,,, +B975,Peptoniphilus,2,,,, +B975,Porphyromonas,1,,,, +B975,Prevotella,1,,,, +B975,Prevotella,3,,,, +B975,Staphylococcaceae,1,,,, +B975,Staphylococcaceae,1,,,, +B975,Staphylococcus,162,,,, +B975,Streptococcus,1,,,, +B975,unidentified,1,,,, +B975,unidentified,1,,,, +B975,unidentified,1,,,, +B975,unidentified,1,,,, +B975,unidentified,7,,,, +B975,Varibaculum,1,,,, diff --git a/notebooks/sat_ranks.csv b/notebooks/sat_ranks.csv new file mode 100644 index 0000000..7aee750 --- /dev/null +++ b/notebooks/sat_ranks.csv @@ -0,0 +1,67 @@ +"http://professionals.collegeboard.com/profdownload/sat-mathematics-percentile-ranks-2010.pdf",,,,,, +,"Total",,"Male",,"Female", +"Score","Number","Percentile","Number","Percentile","Number","Percentile" +800,11959,99,8072,99,3887,"99+" +790,3588,99,2327,99,1261,99 +780,3770,99,2377,98,1393,99 +770,9663,98,6340,97,3323,99 +760,7016,98,4413,97,2603,98 +750,10313,97,6721,96,3592,98 +740,7542,97,4612,95,2930,98 +730,9721,96,6004,94,3717,97 +720,7674,95,4664,94,3010,97 +710,15694,94,9508,92,6186,96 +700,17394,93,10568,91,6826,95 +690,21305,92,12858,89,8447,94 +680,22953,90,13552,87,9401,93 +670,24377,89,14205,85,10172,92 +660,27356,87,15564,83,11792,91 +650,26303,85,14685,81,11618,89 +640,26844,84,14906,79,11938,88 +630,28425,82,15675,77,12750,86 +620,31515,80,17018,74,14497,84 +610,44082,77,23042,71,21040,82 +600,33358,75,17325,69,16033,80 +590,38788,72,20170,66,18618,78 +580,40879,70,20699,63,20180,75 +570,51731,66,25790,60,25941,72 +560,38232,64,18782,57,19450,70 +550,38473,61,18659,54,19814,67 +540,54552,58,26165,51,28387,64 +530,55065,54,25373,47,29692,60 +520,43599,51,20022,44,23577,57 +510,45452,48,20673,42,24779,54 +500,61084,45,26823,38,34261,50 +490,57090,41,25113,34,31977,46 +480,40871,38,17594,32,23277,44 +470,47521,35,20181,29,27340,40 +460,58137,31,23544,26,34593,36 +450,50172,28,20777,23,29395,33 +440,39180,26,15576,21,23604,30 +430,46320,23,18412,18,27908,26 +420,47752,20,18154,16,29598,23 +410,41915,17,16052,14,25863,20 +400,30052,15,11210,12,18842,17 +390,32466,13,12140,10,20326,15 +380,27135,11,9984,9,17151,13 +370,25852,9,9484,8,16368,11 +360,23847,8,8809,6,15038,9 +350,20800,6,7755,5,13045,7 +340,17639,5,6554,4,11085,6 +330,14623,4,5371,4,9252,5 +320,11400,4,4295,3,7105,4 +310,11776,3,4411,2,7365,3 +300,8430,2,3300,2,5130,3 +290,7358,2,2898,2,4460,2 +280,7358,1,2872,1,4486,2 +270,3194,1,1301,1,1893,1 +260,3495,1,1375,1,2120,1 +250,2163,1,821,1,1342,1 +240,3278,1,1303,1,1975,1 +230,1454,1,555,"1-",899,1 +220,2519,"1-",991,"1-",1528,"1-" +210,1221,"1-",501,"1-",720,"1-" +200,4265,"–",1868,"–",2397,"–" +,1547990,,720793,,827197, +,,,65606,,38728, +,,,0.0910191969,,0.0468183516, diff --git a/notebooks/sat_scale.csv b/notebooks/sat_scale.csv new file mode 100644 index 0000000..0ff29a5 --- /dev/null +++ b/notebooks/sat_scale.csv @@ -0,0 +1,18 @@ +"http://professionals.collegeboard.com/profdownload/sat-raw-score-to-scaled-score-ranges-2010.pdf",,,,, +"Raw score","Critical Reading","Raw score","Mathematics","Raw score","Writing Skills" +67,800,,,, +65,"790-800",,,, +60,"710-730",,,, +55,"660-670",54,800,, +50,"620-630",50,"710-730",49,800 +45,"580-590",45,"650-670",45,"690-730" +40,"550-560",40,"610-630",40,"620-660" +35,"520-530",35,"570-580",35,"570-660" +30,"490-510",30,"530-540",30,"520-550" +25,"460-480",25,"490-500",25,"480-510" +20,"420-450",20,"450-460",20,"440-460" +15,"390-420",15,"410-420",15,"400-420" +10,"350-380",10,"360-380",10,"350-370" +5,"300-330",5,"300-330",5,"300-330" +0,"220-270",0,"220-260",0,"210-260" +-5,200,-5,200,-5,200 diff --git a/notebooks/showcases.2011.csv b/notebooks/showcases.2011.csv new file mode 100644 index 0000000..e64fb13 --- /dev/null +++ b/notebooks/showcases.2011.csv @@ -0,0 +1,11 @@ +,"Sep. 19","Sep. 20","Sep. 21","Sep. 22","Sep. 23","Sep. 26","Sep. 27","Sep. 28","Sep. 29","Sep. 30","Oct. 3","Oct. 4","Oct. 5","Oct. 7","Oct. 10","Oct. 11","Oct. 12","Oct. 13","Oct. 14","Oct. 17","Oct. 18","Oct. 19","Oct. 20","Oct. 21","Oct. 24","Oct. 25","Oct. 26","Oct. 27","Oct. 28","Oct. 31","Nov. 1","Nov. 2","Nov. 3","Nov. 4","Nov. 7","Nov. 8","Nov. 9","Nov. 10","Nov. 11","Nov. 14","Nov. 15","Nov. 16","Nov. 17","Nov. 18","Nov. 21","Nov. 22","Nov. 23","Nov. 28","Nov. 29","Nov. 30","Dec. 1","Dec. 2","Dec. 5","Dec. 6","Dec. 7","Dec. 8","Dec. 9","Dec. 12","Dec. 13","Dec. 14","Dec. 15","Dec. 16","Dec. 19","Dec. 20","Dec. 21","Dec. 22","Dec. 23","Dec. 26","Dec. 30","Jan. 2","Jan. 3","Jan. 4","Jan. 5","Jan. 6","Jan. 9","Jan. 10","Jan. 11","Jan. 12","Jan. 13","Jan. 16","Jan. 18","Jan. 19","Jan. 20","Jan. 23","Jan. 24","Jan. 25","Jan. 26","Jan. 27","Jan. 30","Jan. 31","Feb. 1","Feb. 2","Feb. 3","Feb. 6","Feb. 7","Feb. 8","Feb. 9","Feb. 10","Feb. 13","Feb. 14","Feb. 15","Feb. 16","Feb. 17","Feb. 20","Feb. 21","Feb. 22","Feb. 23","Feb. 24","Feb. 27","Feb. 28","Feb. 29","Mar. 1","Mar. 2","Mar. 5","Mar. 6","Mar. 7","Mar. 8","Mar. 9","Mar. 12","Mar. 13","Mar. 14","Mar. 19","Mar. 20","Mar. 21","Mar. 22","Mar. 23","Mar. 26","Mar. 27","Mar. 28","Mar. 29","Mar. 30","Apr. 2","Apr. 3","Apr. 4","Apr. 5","Apr. 6","Apr. 9","Apr. 10","Apr. 11","Apr. 12","Apr. 13","Apr. 16","Apr. 17","Apr. 18","Apr. 19","Apr. 20","Apr. 23","Apr. 24","Apr. 25","Apr. 26","Apr. 27","Apr. 30","May 1","May 2","May 3","May 4","May 7","May 8","May 9","May 10","May 11","May 14","May 15","May 16","May 17","May 18","May 21","May 22","May 23","May 24","May 25","May 28","May 29","May 30","May 31","Jun. 1","Jun. 4","Jun. 5","Jun. 6","Jun. 7","Jun. 8","Jun. 11","Jun. 12","Jun. 13","Jun. 14","Jun. 15","Jun. 18","Jun. 19","Jul. 4","Aug. 17","Sep. 4" +,"5631K","5632K","5633K","5634K","5635K","5641K","5642K","5643K","5644K","5645K","5681K","5682K","5683K","5754K","5651K","5652K","5653K","5654K","5655K","5671K","5672K","5673K","5674K","5675K","5781K","5783K","5782K","5784K","5785K","5691K","5692K","5693K","5694K","5695K","5701K","5702K","5703K","5704K","5705K","5711K","5712K","5713K","5714K","5715K","5721K","5722K","5723K","5731K","5732K","5733K","5734K","5735K","5741K","5742K","5743K","5744K","5745K","5751K","5752K","5753K","5684K","5755K","5761K","5762K","5763K","5764K","5765K","5802K","5775K","5662K","5665K","5663K","5664K","5661K","5791K","5792K","5793K","5795K","5794K","5821K","5803K","5804K","5805K","5811K","5812K","5813K","5814K","5815K","5801K","5822K","5823K","5824K","5825K","5831K","5832K","5833K","5834K","5835K","5841K","5842K","5843K","5844K","5845K","5852K","5853K","5851K","5854K","5855K","5861K","5862K","5863K","5864K","5865K","5871K","5872K","5873K","5874K","5875K","5881K","5882K","5883K","5891K","5892K","5893K","5894K","5895K","5901K","5902K","5903K","5904K","5905K","5911K","5912K","5913K","5915K","5914K","5941K","5942K","5943K","5944K","5945K","5931K","5932K","5933K","5934K","5935K","5921K","5922K","5923K","5924K","5925K","5951K","5952K","5953K","5954K","5955K","5961K","5962K","5963K","5964K","5965K","5983K","5972K","5995K","5974K","5975K","5971K","5982K","5981K","5984K","5985K","5991K","5992K","5993K","5994K","5973K","6001K","6002K","6003K","6004K","6005K","6012K","6011K","6013K","6014K","6015K","6024K","5685K","6021K","6022K","6023K" +,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +"Showcase 1",50969,21901,32815,44432,24273,30554,20963,28941,25851,28800,37703,34774,25268,44044,32516,30827,40374,27698,23590,38625,24026,37449,34268,22281,22890,24226,25448,32525,36799,27380,25903,31769,22853,23202,20721,28451,26154,27019,51959,35811,24885,24648,30707,23744,45078,20773,30892,29971,37423,33593,24866,24628,24827,29248,24818,28762,31838,22268,38892,22599,31705,23299,21394,22589,30573,33009,19612,28550,49937,36915,33400,34516,31516,51108,23981,26383,39801,39201,19563,20418,28465,22356,24150,20425,23926,20860,35526,20912,27426,26393,25329,22434,31933,23986,24729,21747,27273,21681,29324,35874,30397,27310,28414,32535,22886,29842,35954,27883,31268,21563,39642,21862,33499,24313,26809,29674,21711,25618,35504,23327,34581,25918,20418,26511,28288,30348,34428,26732,29631,30253,30256,21399,30988,26536,39924,26254,29137,32666,30706,34805,33343,29671,20408,30406,20990,32391,20828,36439,31785,27663,30291,33480,28714,36803,24281,23841,33844,31809,28309,27332,28957,54579,28649,26854,20962,38787,26738,28958,28604,25269,30624,25366,31328,36813,31434,30321,36414,34736,21707,38297,28996,27690,29426,21711,29307,25263,26993,29390,34920,30323,46638 +"Showcase 2",45429,34061,53186,31428,22320,24337,41373,45437,41125,36319,38752,28450,39745,40894,30608,30463,29405,31290,38597,33917,43036,36623,31681,29080,34137,24925,36174,23705,24897,37776,19792,20182,36070,32357,24054,21527,30623,34089,58458,69485,29349,23955,28244,19756,28336,34646,26712,24349,24366,23276,34144,28136,33508,38836,27654,23097,23255,33801,35658,42174,34238,29515,26949,22797,25211,29737,23084,24891,71628,29460,24974,32693,26937,30939,32252,29551,22193,29344,33551,29524,23504,36954,18349,19508,20068,27319,26055,25875,29831,30918,36442,28854,36664,24753,25264,30843,32968,28487,27806,22513,25585,23677,29286,29299,31501,21902,20320,25830,32687,30493,20043,28598,34432,27386,19693,25812,24631,30858,26993,27570,23566,20908,25541,32310,37393,21827,26934,28480,39620,39263,34607,26375,43998,25357,33251,42675,20267,32390,20513,32782,29488,27498,34453,36562,19827,49869,30602,28979,36419,29993,36548,24984,22407,37746,30764,21255,37125,23904,37776,29317,49419,27712,31352,34808,33302,30317,26784,25612,22438,23548,29445,26525,30571,25755,38715,29281,38505,33612,25820,20344,25938,41765,34304,26277,26617,32646,33337,26314,31278,31285,42319 +,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +"Bid 1",42000,14000,32000,27000,18750,27222,25000,35000,22500,21300,21567,35500,26000,38500,24500,27507,30000,36500,25000,36000,24000,37000,22575,20000,18200,25000,23000,36750,35000,38000,23045,22000,20000,31100,12500,29000,25400,22000,32750,32000,22486,29600,29000,23225,37000,17000,32450,28500,29100,24021,25000,24350,18500,25000,28750,25500,26000,21569,41000,34500,32614,22000,15000,14532,24000,28000,17500,24533,32000,33000,26132,27000,23000,50972,15500,27000,35000,27852,28500,22000,24500,19000,27000,25000,20000,18500,23000,20000,19000,34000,24000,22575,21995,27000,22500,26400,27000,20000,31250,38000,25000,26000,26000,23000,22100,29300,22500,22000,27000,18500,24240,16500,23000,19500,23000,24075,17500,5500,24000,18358,27000,16522,24622,21500,22475,25000,28000,18000,27000,32000,26550,20950,43200,22500,22135,21000,27237,24500,23000,33000,22000,22000,18200,18550,23990,27200,22000,15500,32500,18500,27000,27500,26576,27000,22000,28900,22253,27000,22000,26000,36000,40000,30450,25500,22000,23000,33000,24500,24000,35000,26121,21000,25700,30000,27000,31000,23723,28000,17500,27000,23500,23000,27000,19000,25300,25500,41000,23052,27800,27000,29900 +"Bid 2",34000,59900,45000,38000,23000,18525,32000,45000,32000,27500,23800,18975,35555,35000,29765,34000,23000,25000,24000,42333,27000,24500,27500,29500,25000,28000,22000,27500,22000,37000,22500,19900,32100,27590,20000,26500,20000,28000,58000,48000,30100,21000,22750,26500,27000,27500,26000,24250,26250,31000,28500,25000,20500,34000,24250,33250,15000,29250,25700,30000,34000,28950,26420,22000,27500,23456,21000,30500,28000,25200,19401,25000,28000,32310,27500,30000,23000,28000,30000,22750,33000,24500,20400,27000,25000,25000,24859,30100,23500,24150,33500,36500,32000,24000,27890,22000,27717,31000,26400,21000,27100,24000,27500,23695,23200,21500,21000,26535,19500,26500,26000,31860,25000,25496,24500,31000,23000,17000,34800,29000,21000,24000,23000,25000,27500,22525,23500,21000,38000,30000,32000,22000,35000,23000,23374,28999,26000,27500,21100,30777,18000,24000,36100,24000,18000,33057,28000,25500,28000,23000,23100,18200,24500,35000,22000,18605,31229,20000,34000,28300,38675,18522,33500,22000,26262,24500,23000,25000,22013,18500,28000,24050,31000,31000,32000,30972,28543,38000,26000,22300,29972,36000,30000,24750,28000,26800,26888,16000,30022,21000,33000 +,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +"Difference 1",8969,7901,815,17432,5523,3332,-4037,-6059,3351,7500,16136,-726,-732,5544,8016,3320,10374,-8802,-1410,2625,26,449,11693,2281,4690,-774,2448,-4225,1799,-10620,2858,9769,2853,-7898,8221,-549,754,5019,19209,3811,2399,-4952,1707,519,8078,3773,-1558,1471,8323,9572,-134,278,6327,4248,-3932,3262,5838,699,-2108,-11901,-909,1299,6394,8057,6573,5009,2112,4017,17937,3915,7268,7516,8516,136,8481,-617,4801,11349,-8937,-1582,3965,3356,-2850,-4575,3926,2360,12526,912,8426,-7607,1329,-141,9938,-3014,2229,-4653,273,1681,-1926,-2126,5397,1310,2414,9535,786,542,13454,5883,4268,3063,15402,5362,10499,4813,3809,5599,4211,20118,11504,4969,7581,9396,-4204,5011,5813,5348,6428,8732,2631,-1747,3706,449,-12212,4036,17789,5254,1900,8166,7706,1805,11343,7671,2208,11856,-3000,5191,-1172,20939,-715,9163,3291,5980,2138,9803,2281,-5059,11591,4809,6309,1332,-7043,14579,-1801,1354,-1038,15787,-6262,4458,4604,-9731,4503,4366,5628,6813,4434,-679,12691,6736,4207,11297,5496,4690,2426,2711,4007,-237,-14007,6338,7120,3323,16738 +"Difference 2",11429,-25839,8186,-6572,-680,5812,9373,437,9125,8819,14952,9475,4190,5894,843,-3537,6405,6290,14597,-8416,16036,12123,4181,-420,9137,-3075,14174,-3795,2897,776,-2708,282,3970,4767,4054,-4973,10623,6089,458,21485,-751,2955,5494,-6744,1336,7146,712,99,-1884,-7724,5644,3136,13008,4836,3404,-10153,8255,4551,9958,12174,238,565,529,797,-2289,6281,2084,-5609,43628,4260,5573,7693,-1063,-1371,4752,-449,-807,1344,3551,6774,-9496,12454,-2051,-7492,-4932,2319,1196,-4225,6331,6768,2942,-7646,4664,753,-2626,8843,5251,-2513,1406,1513,-1515,-323,1786,5604,8301,402,-680,-705,13187,3993,-5957,-3262,9432,1890,-4807,-5188,1631,13858,-7807,-1430,2566,-3092,2541,7310,9893,-698,3434,7480,1620,9263,2607,4375,8998,2357,9877,13676,-5733,4890,-587,2005,11488,3498,-1647,12562,1827,16812,2602,3479,8419,6993,13448,6784,-2093,2746,8764,2650,5896,3904,3776,1017,10744,9190,-2148,12808,7040,5817,3784,612,425,5048,1445,2475,-429,-5245,6715,-1691,9962,-4388,-180,-1956,-4034,5765,4304,1527,-1383,5846,6449,10314,1256,10285,9319 diff --git a/notebooks/showcases.2012.csv b/notebooks/showcases.2012.csv new file mode 100644 index 0000000..cef29ce --- /dev/null +++ b/notebooks/showcases.2012.csv @@ -0,0 +1,10 @@ +,"Sep. 24","Sep. 25","Sep. 26","Sep. 27","Sep. 28","Oct. 1","Oct. 2","Oct. 3","Oct. 4","Oct. 5","Oct. 8","Oct. 9","Oct. 10","Oct. 11","Oct. 12","Oct. 15","Oct. 16","Oct. 17","Oct. 18","Oct. 19","Oct. 22","Oct. 23","Oct. 24","Oct. 25","Oct. 26","Oct. 29","Oct. 30","Oct. 31","Nov. 1","Nov. 2","Nov. 5","Nov. 6","Nov. 7","Nov. 8","Nov. 9","Nov. 12","Nov. 13","Nov. 14","Nov. 15","Nov. 16","Nov. 19","Nov. 20","Nov. 21","Dec. 3","Dec. 4","Dec. 5","Dec. 6","Dec. 7","Dec. 10","Dec. 11","Dec. 12","Dec. 13","Dec. 14","Dec. 17","Dec. 18","Dec. 19","Dec. 20","Dec. 21","Dec. 24","Dec. 26","Dec. 27","Dec. 28","Dec. 31","Jan. 1","Jan. 2","Jan. 3","Jan. 4","Jan. 7","Jan. 8","Jan. 9","Jan. 10","Jan. 11","Jan. 14","Jan. 15","Jan. 16","Jan. 17","Jan. 18","Jan. 22","Jan. 23","Jan. 24","Jan. 25","Jan. 28","Jan. 29","Jan. 30","Jan. 31","Feb. 1","Feb. 4","Feb. 5","Feb. 6","Feb. 7","Feb. 8","Feb. 11","Feb. 12","Feb. 13","Feb. 14","Feb. 15","Feb. 18","Feb. 19","Feb. 20","Feb. 21","Feb. 22","Feb. 25","Feb. 26","Feb. 27","Feb. 28","Mar. 1","Mar. 4","Mar. 5","Mar. 6","Mar. 7","Mar. 8","Mar. 11","Mar. 12","Mar. 13","Mar. 14","Mar. 15","Mar. 18","Mar. 19","Mar. 20","Mar. 25","Mar. 26","Mar. 27" +,"6031K","6032K","6033K","6034K","6035K","6081K","6082K","6083K","6084K","6041K","6085K","6042K","6043K","6044K","6045K","6171K","6172K","6173K","6174K","6715K","6051K","6052K","6053K","6054K","6055K","6091K","6092K","6093K","6094K","6095K","6101K","6102K","6103K","6104K","6105K","6071K","6072K","6073K","6074K","6075K","6113K","6112K","6111K","6121K","6122K","6123K","6124K","6125K","6131K","6132K","6133K","6134K","6135K","6141K","6142K","6143K","6144K","6145K","6151K","6153K","6221K","6155K","6161K","6162K","6163K","6164K","6165K","6201K","6202K","6203K","6204K","6205K","6181K","6182K","6183K","6184K","6185K","6231K","6233K","6234K","6235K","6211K","6212K","6213K","6214K","6215K","6061K","6062K","6063K","6064K","6152K","6154K","6222K","6223K","6224K","6225K","6192K","6195K","6193K","6191K","6194K","6241K","6242K","6243K","6244K","6245K","6251K","6252K","6253K","6254K","6255K","6261K","6262K","6263K","6264K","6265K","6331K","6271K","6273K","6281K","6283K","6282K" +"Showcase 1",40811,31259,41943,24946,26257,49928,30744,37671,32255,21983,40311,28573,28510,27341,29393,24379,24289,32784,41995,39939,46829,24413,36607,43902,36820,27575,24459,31749,22466,25809,35252,58342,21981,40294,45433,27480,28868,30864,36577,38468,34353,26262,23515,41374,22476,27762,28744,46872,32416,30296,27813,34608,25540,33795,28981,41280,28400,26462,36419,22716,35855,23261,48442,25643,25975,52557,21258,34640,33798,33513,38934,24988,27614,31816,39315,30396,37649,22542,22576,27288,40830,29943,28583,36407,27791,22703,29633,20869,29412,23437,27266,32922,27268,27382,27736,41781,38875,46339,24422,38902,40408,30764,27440,29271,37446,34763,22992,37517,27677,23988,29323,35379,34330,27793,22412,29528,22774,25375,24949,23662,23704,20898 +"Showcase 2",62485,32972,24755,34665,48635,42621,22253,24251,45871,31820,31399,33797,37252,30320,19290,32262,20545,34727,22554,21969,23624,38948,29310,43453,27734,30919,23309,33824,35704,40812,21261,33968,25903,35784,53263,27572,38588,20660,35617,26715,25832,28869,37668,24374,30649,28659,27518,45829,26685,46648,25761,29552,25336,28428,44564,22311,22564,38649,25419,21470,25510,43850,66996,37319,29509,32888,35368,21263,29856,25770,45916,20403,48281,33822,21361,29336,40328,28725,23928,32706,32349,39844,36804,25614,30964,28525,28269,30290,25815,33598,23318,23880,35341,22580,28976,28926,34637,34255,38823,42309,46833,42965,39296,28959,23576,35926,26521,26745,24335,27439,21340,43923,24585,27441,24285,23667,22812,31986,30696,22329,34325,23876 +,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +"Bid 1",22000,10000,39500,21513,23420,36500,26500,42000,25000,21650,22000,20582,14575,21200,30000,27000,23491,32500,28450,24000,27000,19425,32813,27500,28500,28000,23313,42500,21400,23000,29500,26968,21000,32364,32000,23000,29800,18200,27500,28500,30000,30500,17823,31500,23000,19000,18500,16000,30000,25000,42000,30000,18500,27500,18965,37500,20600,34000,26500,22000,42000,21000,36000,25620,9950,30200,9000,27000,20000,33225,34500,32125,35000,32000,24500,24950,35000,19152,18750,23000,30000,37000,30000,32000,30000,22782,23000,22000,27000,28000,28000,21650,25000,26000,23500,28900,31500,41000,24000,35000,26000,36000,30736,27300,38000,18000,25700,27436,22000,26129,40000,21000,27000,27300,25000,27000,27000,36000,20500,26000,23800,28000 +"Bid 2",40500,21000,22500,32000,35000,34000,18000,26000,29000,33000,29000,27000,30500,28000,25000,35000,31500,29000,17500,27000,24175,27528,22000,33001,26000,22548,17000,15000,25044,27000,17500,31500,18000,27500,37000,21433,29000,25000,30578,20500,23250,27830,28000,19652,27500,28500,28500,38000,30000,33000,38500,28000,22307,19500,29600,20000,23000,35000,28000,25150,31000,32000,45000,22000,22000,36000,23000,20000,23000,28500,37450,29200,21850,25800,18000,31925,37000,29100,20000,24000,35500,23750,27500,22632,26400,32324,29000,26500,26000,32300,28666,19500,25000,35000,27000,29999,29300,27500,27345,23500,48750,43000,36000,30000,18888,25999,26500,32500,21000,25000,26000,31000,21500,18500,28500,25500,22500,32000,31000,20000,34029,25000 +,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +"Difference 1",18811,21259,2443,3433,2837,13428,4244,-4329,7255,333,18311,7991,13935,6141,-607,-2621,798,284,13545,15939,19829,4988,3794,16402,8320,-425,1146,-10751,1066,2809,5752,31374,981,7930,13433,4480,-932,12664,9077,9968,4353,-4238,5692,9874,-524,8762,10244,30872,2416,5296,-14187,4608,7040,6295,10016,3780,7800,-7538,9919,716,-6145,2261,12442,23,16025,22357,12258,7640,13798,288,4434,-7137,-7386,-184,14815,5446,2649,3390,3826,4288,10830,-7057,-1417,4407,-2209,-79,6633,-1131,2412,-4563,-734,11272,2268,1382,4236,12881,7375,5339,422,3902,14408,-5236,-3296,1971,-554,16763,-2708,10081,5677,-2141,-10677,14379,7330,493,-2588,2528,-4226,-10625,4449,-2338,-96,-7102 +"Difference 2",21985,11972,2255,2665,13635,8621,4253,-1749,16871,-1180,2399,6797,6752,2320,-5710,-2738,-10955,5727,5054,-5031,-551,11420,7310,10452,1734,8371,6309,18824,10660,13812,3761,2468,7903,8284,16263,6139,9588,-4340,5039,6215,2582,1039,9668,4722,3149,159,-982,7829,-3315,13648,-12739,1552,3029,8928,14964,2311,-436,3649,-2581,-3680,-5490,11850,21996,15319,7509,-3112,12368,1263,6856,-2730,8466,-8797,26431,8022,3361,-2589,3328,-375,3928,8706,-3151,16094,9304,2982,4564,-3799,-731,3790,-185,1298,-5348,4380,10341,-12420,1976,-1073,5337,6755,11478,18809,-1917,-35,3296,-1041,4688,9927,21,-5755,3335,2439,-4660,12923,3085,8941,-4215,-1833,312,-14,-304,2329,296,-1124 From 18877cfd3a7aba5eca977edcb1d38b1e18c33ecc Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Sun, 14 Feb 2016 12:55:08 -0800 Subject: [PATCH 05/10] Added explanation of Jupyter notebook conversion. --- README.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/README.md b/README.md index 4860c0c..e9dd441 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,49 @@ ThinkBayes ========== +This material was adapted from Alan Downey's Think Bayes +Github repository by Roger Labbe. + +Mostly I took his code and tex file and converted them into +a series of Jupyter notebooks. This was sometimes problamatic. +Alan uses a lot of Python classes, and his exposition splits +the code across multiple paragraphs. This does not cleanly +work in notebooks. Furthermore, most of his code is in .py +files. + +I made this work as best I could, but sometimes there were +authorial decisions that I do not feel comfortable making. +The code just doesn't run in those spots. + +He generates many graphs, but does not supply the source code. +Maybe the code is in the /code subdirectory? In a few places +I took the liberty of supplying the Python, but in many places +I did not. Maybe somebody else will feel like doing that; +I don't really have time right now. Pull requests accepted! + +There are several places in the text where it was not clear +to me where he got his datasets. Rather than making something +up, I just let the code cells fail to execute. I hope Alan +will help me rectify these situations. If not, and there is +enough interest, we can make some decisions and get the cells +working. + + +I put his code in a /code subdirectory. I did not fix the references +in the book, which specify going to his home page to get the code. + +I did alter most of the code to work with Python 3. Mainly by using + + from __future__ import print_function + +but in a few places I had to fix things like removing `xrange` or +`iterkeys()`. I only did just enough work to get it working. + + +His original README is: + +https://github.com/AllenDowney/ThinkBayes + Code repository for Think Bayes: Bayesian Statistics Made Simple by Allen B. Downey From 3348c69d119664f276d9682fde3e190b8e05c4dd Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Wed, 24 Feb 2016 18:00:44 -0800 Subject: [PATCH 06/10] Added binder badge --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index e9dd441..3bb0521 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,10 @@ ThinkBayes ========== +Launch now and run interactively by clicking this badge: +[![Binder](http://mybinder.org/badge.svg)](http://mybinder.org/repo/rlabbe/ThinkBayes) + + This material was adapted from Alan Downey's Think Bayes Github repository by Roger Labbe. From b27ab3f7eaaf20024ea4a2ea91f21a1c01d285b0 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Wed, 24 Feb 2016 18:04:35 -0800 Subject: [PATCH 07/10] Removed irrelevant import of filterpy This was copied from my KF book. Filterppy is not used by this project. --- code/book_format.py | 34 ---------------------------------- 1 file changed, 34 deletions(-) diff --git a/code/book_format.py b/code/book_format.py index 57e5d89..4507f0e 100644 --- a/code/book_format.py +++ b/code/book_format.py @@ -1,16 +1,5 @@ # -*- coding: utf-8 -*- -"""Copyright 2015 Roger R Labbe Jr. - - -Code supporting the book - -Kalman and Bayesian Filters in Python -https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python - - -This is licensed under an MIT license. See the LICENSE.txt file -for more information. """ from __future__ import (absolute_import, division, print_function, @@ -33,29 +22,6 @@ sys.path.insert(0, './code') # allow us to import book_format -def test_filterpy_version(): - import filterpy - min_version = [0,0,25] - v = filterpy.__version__ - tokens = v.split('.') - for i,v in enumerate(tokens): - if int(v) > min_version[i]: - return - - i = len(tokens) - 1 - if min_version[i] > int(tokens[i]): - raise Exception("Minimum FilterPy version supported is {}.{}.{}.\n" - "Please install a more recent version.\n" - " ex: pip install filterpy --upgrade".format( - *min_version)) - v = int(tokens[0]*1000) - - -# ensure that we have the correct filterpy loaded. This is -# called when this module is imported at the top of each book -# chapter so the reader can see that they need to update FilterPy. -test_filterpy_version() - def equal_axis(): pylab.rcParams['figure.figsize'] = 10,10 From a1c4dcd6e5ddbeaf51bc6a489ac21bab380926f9 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Wed, 24 Feb 2016 18:08:54 -0800 Subject: [PATCH 08/10] Fixed formatting issue with notebooks. --- code/book_format.py | 1 + notebooks/01_Bayes_Theorem.ipynb | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/code/book_format.py b/code/book_format.py index 4507f0e..a7631f3 100644 --- a/code/book_format.py +++ b/code/book_format.py @@ -1,3 +1,4 @@ +""" # -*- coding: utf-8 -*- """ diff --git a/notebooks/01_Bayes_Theorem.ipynb b/notebooks/01_Bayes_Theorem.ipynb index a140099..59c37bf 100644 --- a/notebooks/01_Bayes_Theorem.ipynb +++ b/notebooks/01_Bayes_Theorem.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 3, + "execution_count": 1, "metadata": { "collapsed": false }, @@ -241,13 +241,15 @@ "" ] }, - "execution_count": 3, + "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# format the book\n", + "import sys\n", + "sys.path.insert(0,'../code')\n", "import book_format\n", "book_format.load_style('../code')" ] From 517212e7701a17aa535af42a25a1fd4f91cb96f0 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Thu, 25 Feb 2016 07:33:34 -0800 Subject: [PATCH 09/10] Corrected my misspelling of Allen's name! --- README.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 3bb0521..1516a34 100644 --- a/README.md +++ b/README.md @@ -5,19 +5,24 @@ Launch now and run interactively by clicking this badge: [![Binder](http://mybinder.org/badge.svg)](http://mybinder.org/repo/rlabbe/ThinkBayes) -This material was adapted from Alan Downey's Think Bayes +This material was adapted from Allen Downey's Think Bayes Github repository by Roger Labbe. Mostly I took his code and tex file and converted them into a series of Jupyter notebooks. This was sometimes problamatic. -Alan uses a lot of Python classes, and his exposition splits -the code across multiple paragraphs. This does not cleanly -work in notebooks. Furthermore, most of his code is in .py -files. +Allen uses a lot of Python classes, and his exposition splits +the code across multiple paragraphs. It's a great pedagogical +technique, but does not cleanly work in notebooks, where you have +to specify the entire class in one cell. Furthermore, most of +his code is in .py files. I made this work as best I could, but sometimes there were authorial decisions that I do not feel comfortable making. -The code just doesn't run in those spots. +The code just doesn't run in those spots. Sometimes I left +the code in markdown cells rather than putting it in code +cells, and then imported the class from the .py file. I kind +of figured this out as I proceeded, so perhaps there are +stylistic differences in the back vs the front of the book. He generates many graphs, but does not supply the source code. Maybe the code is in the /code subdirectory? In a few places @@ -27,14 +32,14 @@ I don't really have time right now. Pull requests accepted! There are several places in the text where it was not clear to me where he got his datasets. Rather than making something -up, I just let the code cells fail to execute. I hope Alan +up, I just let the code cells fail to execute. I hope Allen will help me rectify these situations. If not, and there is enough interest, we can make some decisions and get the cells working. - I put his code in a /code subdirectory. I did not fix the references in the book, which specify going to his home page to get the code. +Again, I wanted to make as few editorial decisions as possible. I did alter most of the code to work with Python 3. Mainly by using From a3d8ab1477486807ea0751674bf607bf0ddced71 Mon Sep 17 00:00:00 2001 From: Roger Labbe Date: Thu, 25 Feb 2016 07:34:31 -0800 Subject: [PATCH 10/10] Expanded binder instructions --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 1516a34..819ba6c 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ ThinkBayes ========== -Launch now and run interactively by clicking this badge: +Launch now and run interactively by clicking this badge, +then navigate to the \notebooks directory and open the +notebook corresponding to the chapter you want to read: [![Binder](http://mybinder.org/badge.svg)](http://mybinder.org/repo/rlabbe/ThinkBayes)