$30
ECSE 310 Thermodynamics of Computing
Homework 4
Submission instructions: Please submit your assignment online via MyCourses assignment tool prior to
the deadline. If your submission is handwritten please provide a high quality image (preferably scanned
using uPrint or similar). Poor quality digital photos may be rejected. Include any workings and calculations
that you performed in completing the assignment.
Late policy: 1% deduction per hour
Academic integrity reminder: In submitting this assignment on MyCourses you are attesting that it is the
result of your own work.
Each question has the same weight
1. a) Using your own words, define the following:
a) Shannon’s measure of information
b) Joint entropy
c) Mutual information
d) Marginal information
e) Conditional entropy
f) Algorithmic entropy
g) Johnson noise
h) Brownian ratchet
i) Differential entropy
j) Probability density function
k) Uncertainty principle
l) Shannon’s coding theorem
m) Microstate and macrostate
2. Consider the probability that parrots are grey is 0.3, and the probability that they are male or
female is equal. However, the male parrots are 1.5 times more likely than the female parrots
to be grey.
a) You learn that non-grey parrots are male, how much information do you gain from
this knowledge?
b) Rank the following uncertainties in descending order:
i. uncertainty about colour
ii. uncertainty about gender
iii. uncertainty about colour, provided that a parrot is male
iv. uncertainty about gender, provided that a parrot is non-grey
3. The average temperatures in Paris, France in the year 2017 were taken from
www.wunderground.com and are given in the attached excel sheet.
a) What is the uncertainty of the temperature?
b) What is the joint entropy of temperatures and months?
c) How much information is shared between temperatures and months?
d) What is the uncertainty given that month is May?