CS 151: Artificial Intelligence
Homework 4
Due date: 3/7/13 by 11:59 pm

Your answers to the following questions should be typed and submitted as a pdf (e.g. you can type your answers using Word and print to pdf, or use LaTeX, or write your answers by hand and then scan your paper).

1. [10 points] AIMA 13.8


2. [10 points] AIMA 13.15


3. [10 points] AIMA 14.4a


4. Suppose that the joint probability distribution of four variables {X, Y, Z, W} can be factorized as: p(x,y,z,w) = p(x)p(y|x)p(z|x)p(w|y,z). Draw the Bayesian network that corresponds to this factoring of the joint distribution. Use d-separation to determine whether or not each of the following independencies holds:
   1. [5 points] X is independent of W given Y
   2. [5 points] X is independent of W given Z
   3. [5 points] X is independent of W given {Y, Z}
   4. [5 points] Y is independent of Z given {X, W}
   5. [5 points] Assume X, Y, Z, and W are binary random variables. How many probabilities must be specified for the conditional distributions of the Bayesian network?
   
   


5. Consider the WetGrass Bayesian network shown in Figure 14.12a in AIMA. Assume we observe WetGrass=false.
   1. [10 points] Compute the exact posterior probabilities p(Cloudy|WetGrass=false), p(Sprinkler|WetGrass=false), and p(Rain|WetGrass=false).
   2. [5 points] Show how to generate a sample using rejection sampling.
   3. [15 points] Generate 10 samples using rejection sampling. How many samples do you get where WetGrass=false and how many do you have to reject? Estimate posterior probabilities for Cloudy, Sprinkler, and Rain based on those 10 samples.
   4. [5 points] Show how to generate a sample using likelihood weighting
   5. [15 points] Generate 10 samples using likelihood weighting. Show the weights for each sample. Estimate posterior probabilities for Cloudy, Sprinkler, and Rain based on those 10 samples.
   
   Note: To generate a random double between [0,1] in Python use the commands: "import random" and "random.uniform(0,1)"

1. A discrete probability distribution over some set of random variables (X1,...,XN) assigns a real-valued number to each possible assignment of the random variables. These real-valued numbers are called "probabilities". What are the constraints to have a valid probability distribution?
2. How many probabilities must you specify for the joint distribution of n binary random variables? For the joint distribution of n random variables with domain size d?
3. Mathematically define: marginalization, conditional distribution, product rule, chain rule, Bayes' rule.
4. Derive Bayes' rule using the definition of the conditional distribution and the product rule
5. Explain (in a mathematically precise way) what is meant by the statement: "The conditional distribution and the joint distribution are proportional"
6. AIMA 13.12
7. AIMA 13.14
8. AIMA 13.17 (Note there is a typo: B should be Y)
9. AIMA 13.22 - Naive Bayes model
10. Define "Bayesian network".
11. Why are Bayesian networks useful?
12. Type of question I might ask on a midterm: I might describe a story and then ask you to (1) construct a Bayesian network that captures the causual structure of the story (2) compute the number of probabilities that must be specified for the BN (3) compute the number of probabilities that must be specified for the joint (without the BN) (4) list the independencies using d-separation (3) compute some query (or describe given a piece of evidence which exact and which approximate inference algorithm you might use to compute the query)
13. AIMA 14.5

You only need to submit the pdf of your answers to the written questions. Rename your pdf to "hw4_LastName_FirstName.pdf" and upload it using this URL.