In this lab, you'll use point data, describing the movement of a talking person, to animate a cartoon face.
In order to make sure everything works, you might want to go to the command line, and run
pip install -r requirements.txt
This will install the modules that are used on the autograder, including numpy, h5py, and the gradescope utilities.
The data is made available to you in an HDF5 file, data.hdf5. For more information about the hdf5 data storage format, see HDF5 for Python.
import h5py
data = {}
with h5py.File('data.hdf5') as f:
for group in f.keys():
data[group] = {}
for dataset in f[group].keys():
data[group][dataset] = f[group][dataset][:]
print('data[%s][%s] has shape %s'%
(group,dataset,data[group][dataset].shape))
data[mr][corner] has shape (4, 2) data[mr][image] has shape (224, 224) data[mr][moving] has shape (8, 2) data[mr][static] has shape (11, 2) data[mr][triangles] has shape (40, 3) data[xr][fps] has shape (1,) data[xr][fs] has shape (1,) data[xr][moving] has shape (438, 8, 2) data[xr][static] has shape (11, 2) data[xr][wav] has shape (66150,)
As you can see, data.hdf5 includes data from two different recording sources:
mr data is from a magnetic resonance image, posted on Wikipedia with CC-SA 3.0 license by Pogrebnoj-Alexandroff.
xr data is from an X-ray microbeam recording created by the University of Wisconsin X-ray Microbeam. At the time of this writing, the data are available from UC Berkeley. The file data.hdf5 contains three seconds extracted from one of the recordings, recording tp010 from participant JW14.We can listen to the waveform using IPython.display:
import IPython.display
IPython.display.Audio(data['xr']['wav'],rate=data['xr']['fs'][0])
The xr data includes two sets of points: static points, and moving points. There are 11 static points: 9 on the palate, and 2 on the pharynx. Each point is a set of (X,Y) coordinates. Notice that the speaker is facing to the right.
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1,1,figsize=(5,5))
ax.plot(data['xr']['static'][:,0],data['xr']['static'][:,1],'b.')
ax.set_title('Palate and Pharynx of XRMB participant JW14')
Text(0.5, 1.0, 'Palate and Pharynx of XRMB participant JW14')
The moving points are a set of 1695 frames, each of which contains 8 points:
This file has a constant frame rate of data['xr']['fps'] (not all XRMB files have constant frame rate, but this one does.
print('XRMB frame rate, in frames per second, is',data['xr']['fps'][0])
print('Downsampling by 10 would give',10000/data['xr']['fps'][0],'ms per frame')
XRMB frame rate, in frames per second, is 145.64520827264784 Downsampling by 10 would give 68.66 ms per frame
The provided frame rate, 146fps, is too fast for the human eye to follow, so displaying it is a waste of processor power. Instead, let's downsample by a factor of 10, to 14.6 fps (69ms per frame).
We can create an animation object using matplotlib.FuncAnimation, then show it inline using IPython.display.HTML.
This will show four pellets on the tongue, two on the jaw, and two on the lips, moving up and down.
from matplotlib import animation
# First set up the figure, the axis, and the plot element we want to animate
fig = plt.figure(figsize=(5,5))
ax = plt.axes(xlim=(-80000, 30000), ylim=(-20000, 20000))
line, = ax.plot([], [], 'C1.', lw=1)
# Initialization function: plot the background of each frame
def init():
ax.plot(data['xr']['static'][:,0],data['xr']['static'][:,1],'b.')
line.set_data([],[])
return line,
# Animation function which updates figure data, showing (x,y) points
def animate(i):
line.set_data(data['xr']['moving'][10*i,:,0],data['xr']['moving'][10*i,:,1])
return line,
# Call the animator. blit=True means only re-draw the parts that have changed.
nframes = int(data['xr']['moving'].shape[0]/10)
anim = animation.FuncAnimation(fig, animate, init_func=init,
frames=nframes, interval=69, blit=True)
plt.close(anim._fig)
import IPython
IPython.display.HTML(anim.to_html5_video())
The magnetic resonance data contains:
data['mr']['image'] - a grayscale imagedata['mr']['static'] - 11 points along the palate and pharynx, listed in the same order as the points in data['xr']['static'].data['mr']['moving'] - 8 points on the lips, tongue, and jaw, listed in the same order as the points in data['xr']['moving']. Note that these are caled moving because the XRMB points move - the MRI points do not movedata['mr']['corner'] - 4 points on the corners of the imagedata['mr']['triangles'] - a list of triangles, connecting the points, and tiling the entire image.First, let's see all the points.
fig, ax = plt.subplots(1,1,figsize=(10,10))
ax.imshow(data['mr']['image'],cmap='gray')
ax.plot(data['mr']['corner'][:,0],data['mr']['corner'][:,1],'C2o')
ax.plot(data['mr']['moving'][:,0],data['mr']['moving'][:,1],'C1o')
ax.plot(data['mr']['static'][:,0],data['mr']['static'][:,1],'C0o')
[<matplotlib.lines.Line2D at 0x7fdf25a775e0>]
Now let's plot the triangles. Each triangle is a triplet of integers. These integers are indices into the combined points array. The points array is a vstack of the three different types of points in the MR image, in alphabetical order, i.e., corner first, then moving, then static.
import numpy as np
points = np.vstack((data['mr']['corner'],data['mr']['moving'],data['mr']['static']))
fig, ax = plt.subplots(1,1,figsize=(10,10))
ax.imshow(data['mr']['image'],cmap='gray')
for i in range(data['mr']['triangles'].shape[0]):
tri = data['mr']['triangles'][i,:]
x = points[(tri[0],tri[1],tri[2],tri[0]),0]
y = points[(tri[0],tri[1],tri[2],tri[0]),1]
ax.plot(x,y,'C1')
The goal of this MP is to use the XRMB data to animate the MR image, using the triangles shown above. Notice that all points in the triangles exist in both MR and XRMB except the four corners -- it's not obvious what points in XRMB correspond to the four corners.
In order to define the correspondences for all 23 points (4 corners, 8 moving points, and 11 static points), let's scale, reflect, shift, and rotate the XRMB points in order to put them into the space of the MR image. In other words, we want to find an affine transformation that converts the head of the person recorded by X-ray microbeam so it resembles, as closely as possible, the head of the person recorded by MRI. We will define "as closely as possible" to be the affine transform that minimizes the mean-squared error between the two heads. Notice that this won't be perfect -- just rotating, scaling, and shearing one head is not sufficient to make it exactly match the other head. But at least it will take care of the unimportant differences like difference in rotation.
MMSE Affine Transformation is very similar to Procrustes Analysis, except that Procrustes analysis is more limited. Procrustes analysis requires that you scale both $x$ and $y$ directions by the same amount, and it doesn't permit any shear. MMSE Affine Transformation allows shear, and it allows different scalings in the $x$ and $y$ directions.
Suppose we have a set of source points $s_k=[s_{k,1},s_{k,2},1]$ for $1\le k\le N$, and we want to transform them to predictions $p_k=[p_{k,1},p_{k,2},1]$ in order to minimize $\sum_k\Vert p_k-t_k\Vert^2$, where $t_{k}=[t_{k,1},t_{k,2},1]$ is the target value of the point. The goal, then, is to find the six parameters $a_{1,1},\ldots,a_{3,2}$ in order to minimize:
$${\mathcal J}= \sum_k\Vert t_k^T-A^Ts_k^T\Vert^2= \sum_k\left\Vert \left[\begin{array}{c}t_{k,1}\\t_{k,2}\\1\end{array}\right]- \left[\begin{array}{ccc}a_{1,1}&a_{2,1}&a_{3,1}\\ a_{1,2}&a_{2,2}&a_{3,2}\\0&0&1\end{array}\right] \left[\begin{array}{c}s_{k,1}\\s_{k,2}\\1\end{array}\right]\right\Vert^2$$The file submitted.py contains a function called find_affine, which currently throws a RuntimeError. You should delete the RuntimeError, then complete find_affine so that it does the task desribed by its docstring:
import importlib, submitted
importlib.reload(submitted)
help(submitted.find_affine)
Help on function find_affine in module submitted:
find_affine(sourcepoints, targetpoints)
Find the MMSE affine transform from sourcepoints to
targetpoints. Assume that sourcepoints and
targetpoints are sorted, so that they are in
one-to-one correspondence.
@param:
sourcepoints (Nx2): a set of source points
targetpoints (Nx2): a set of target points
@return:
predictions (Nx2): the set of predicted target points,
augmentedsource = np.hstack((sourcepoints, np.ones((N,1))))
predictions = augmentedsource @ affine
affine (3x2): linear transform matrix,
chosen to minimize np.sum(np.square(targetpoints-predictions))
You should compute the affine transform matrix, affine, based on sourcepoints composed of the XRMB static points, and target points composed of the MRI static points:
importlib.reload(submitted)
sourcepts = data['xr']['static']
targetpts = data['mr']['static']
predstatic,affine = submitted.find_affine(sourcepts, targetpts)
print('Affine transform matrix is')
print(affine)
Affine transform matrix is [[-1.32212835e-03 -1.32467450e-04] [-2.00948347e-04 -2.58811101e-03] [ 7.26985460e+01 1.33733700e+02]]
fig, ax = plt.subplots(1,1,figsize=(5,5))
ax.imshow(data['mr']['image'],cmap='gray')
ax.plot(predstatic[:,0],predstatic[:,1],'c.')
[<matplotlib.lines.Line2D at 0x7fdf258ed750>]
Notice -- an MMSE affine transform is not sufficient to perfectly match one person's head to the head of a different person! The person who was imaged by XRMB has a much deeper palate than the person imaged using MRI; there's no way to make their two palates line up, perfectly, by just rotating, scaling, shifting and skewing. The extra little bit of alignment will have to be done using piece-wise affine transformations in parts two and three of this MP.
Now that we have the points in the original MR image coordinates, we can create a video showing how the points are changing, as compared to the original positions on the MR image.
from matplotlib import animation
fig, ax = plt.subplots(1,1,figsize=(5,5))
ax.imshow(data['mr']['image'],cmap='gray')
ax.plot(predstatic[:,0],predstatic[:,1],'c.')
line, = ax.plot([], [], 'C1o')
# Initialization function: plot the background of each frame
def init():
line.set_data([],[])
return line,
# Animation function which updates figure data, showing (x,y) points
def animate(i):
augmented = np.hstack((data['xr']['moving'][10*i,:,:],np.ones((8,1))))
pts = augmented @ affine
line.set_data(pts[:,0],pts[:,1])
return line,
# Call the animator. blit=True means only re-draw the parts that have changed.
nframes = int(data['xr']['moving'].shape[0]/10)
anim = animation.FuncAnimation(fig, animate, init_func=init,
frames=nframes, interval=69, blit=True)
plt.close(anim._fig)
import IPython
IPython.display.HTML(anim.to_html5_video())
That view is a little deceptive. What we really want to do is find the triangles, in each synthesized view, and use those triangles to fill in the pixel values. Let's plot the triangles for each frame of the movie.
from matplotlib import animation
fig, ax = plt.subplots(1,1,figsize=(5,5))
ax.set_xlim(0,224)
ax.set_ylim(0,224)
ax.invert_yaxis()
frames = []
nframes = int(data['xr']['moving'].shape[0]/10)
for i in range(nframes):
artists = []
augmented = np.hstack((data['xr']['moving'][10*i,:,:],np.ones((8,1))))
moving = augmented @ affine
points = np.vstack((data['mr']['corner'],moving,predstatic))
for j in range(data['mr']['triangles'].shape[0]):
tri = data['mr']['triangles'][j,:]
x = points[(tri[0],tri[1],tri[2],tri[0]),0]
y = points[(tri[0],tri[1],tri[2],tri[0]),1]
line, = ax.plot(x,y,'C1')
artists.append(line)
frames.append(artists)
# Call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.ArtistAnimation(fig, frames, interval=69, blit=True)
plt.close(anim._fig)
import IPython
IPython.display.HTML(anim.to_html5_video())
The procedure for creating a movie will be as follows.
importlib.reload(submitted)
help(submitted.cartesian2barycentric)
Help on function cartesian2barycentric in module submitted:
cartesian2barycentric(cartesian, triangle)
Convert an array full of pixels from
cartesian coordinates to barycentric
coordinates.
@param:
cartesian (npix,2): x,y coordinates of npix pixels
triangle (3,2): x,y coordinates of each corner of the triangle
@return:
barycentric (npix,3): barycentric coordinates of the pixels.
Each row should be a 3-vector such that
barycentric @ triangle = cartesian
and
np.sum(barycentric[i,:])=1 for all i.
If the corners of the triangle are all in a straight line,
then solving this problem requires inverting a singular matrix.
If np.linalg.matrix_rank tells you that the matrix is singular,
return the value "None" in place of barycentric.
We can debug with some known cases.
The pixel $r_1=[1,1]$ is inside the triangle $T_1=[[0,0],[0,2],[2,0]]$, with barycentric coordinates $b_1=[0,\frac{1}{2},\frac{1}{2}]$.
The pixel $r_2=[0.5,0.5]$ is also inside that triangle, so its barycentric coordinates should be non-negative numbers that sum to one.
The pixel $r_2=[1.5,1.5]$ is not inside that triangle, so its barycentric coordinates should include some negative numbers.
The triangle $T_2=[[0,0],[1,1],[2,2]]$ has zero area (its points are all in a line), so the required matrix inversion is singular. In this case, the method should return None in place of the desired barycentric coordinates.
importlib.reload(submitted)
tri = np.array([[0,0],[0,2],[2,0]])
print('The triangle is:\n',tri)
c = np.array([[1,1],[0.5,0.5],[1.5,1.5]])
b = submitted.cartesian2barycentric(c,tri)
for i in range(3):
print('Cartesian %s has barycentric %s'%(str(c[i,:]),str(b[i,:])))
print('\n')
tri = np.array([[0,0],[1,1],[2,2]])
b = submitted.cartesian2barycentric(c,tri)
print('With this triangle:\n',tri)
print('Cartesian:\n %s\n has barycentric %s'%(str(c),str(b)))
The triangle is: [[0 0] [0 2] [2 0]] Cartesian [1. 1.] has barycentric [0. 0.5 0.5] Cartesian [0.5 0.5] has barycentric [0.5 0.25 0.25] Cartesian [1.5 1.5] has barycentric [-0.5 0.75 0.75] With this triangle: [[0 0] [1 1] [2 2]] Cartesian: [[1. 1. ] [0.5 0.5] [1.5 1.5]] has barycentric None
Of course, we also need to be able to convert from barycentric coordinates back to cartesian coordinates:
importlib.reload(submitted)
help(submitted.barycentric2cartesian)
Help on function barycentric2cartesian in module submitted:
barycentric2cartesian(barycentric, triangle)
Convert a pixel from barycentric coordinates
to cartesian coordinates.
@param:
barycentric (npix, 3): barycentric coordinates.
Each row is a 3-vector, all
barycentric coordinates are non-negative, and
np.sum(barycentric[i,:])=1 for all i.
triangle (3,2): x,y coordinates of each corner of the triangle
@return:
cartesian (npix, 2): cartesian coordinates
importlib.reload(submitted)
tri = np.array([[0,0],[0,2],[2,0]])
print('The triangle is:\n',tri)
bary = np.array([[0,0.5,0.5], [0.5,0.25,0.25]])
c = submitted.barycentric2cartesian(bary,tri)
print('Barycentric:\n %s\nhas cartesian:\n%s'%(str(bary),str(c)))
The triangle is: [[0 0] [0 2] [2 0]] Barycentric: [[0. 0.5 0.5 ] [0.5 0.25 0.25]] has cartesian: [[1. 1. ] [0.5 0.5]]
Next, let's use cartesian2barycentric and barycentric2cartesian to match pixels between the movie frame and the static MR image. The basic strategy is:
In this function, we will assume that triangles is provided in the same format as data['mr']['triangles'], i.e., as a set of integers into the lists of points given by vstacking the corner, moving, and static points of each frame.
importlib.reload(submitted)
help(submitted.moving2static)
Help on function moving2static in module submitted:
moving2static(moving, moviepts, staticpts, triangles)
Convert from a location in the movie frame
to the corresponding location in the static image.
It may sometimes happen that, due to rounding errors,
one or more rows of moving can't be assigned to any
triangle. If that happens, set those rows to zero.
@param:
moving (npix,2): x,y pixel locations in the movie frame
moviepts (23,2): array of x,y points in the movie frame
staticpts (23,2): array of x,y points in the static image
triangles (ntri,3): array of indices of triangle corners
@return:
static (npix,2): x,y location of pixel in static image
We can test this by choosing the center of any two particular triangles. They should have (x,y) coordinates that move as the triangle moves, but the result of applying moving2static should always be the same static coordinates.
importlib.reload(submitted)
staticpts=np.vstack((data['mr']['corner'],data['mr']['moving'],data['mr']['static']))
nframes = data['xr']['moving'].shape[0]
moving = np.zeros((nframes,2,2))
static = np.zeros((nframes,2,2))
for i in range(nframes):
augmented = np.hstack((data['xr']['moving'][i,:,:],np.ones((8,1))))
predmoving = augmented @ affine
moviepts = np.vstack((data['mr']['corner'],predmoving,predstatic))
moving[i,0,:] = (moviepts[11,:]+moviepts[8,:]+moviepts[7,:])/3
moving[i,1,:] = (moviepts[11,:]+moviepts[8,:]+moviepts[9,:])/3
static[i,:,:] = submitted.moving2static(moving[i,:,:],moviepts,staticpts,data['mr']['triangles'])
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2,1,figsize=(14,8))
ax[0].plot(np.arange(nframes),moving[:,0],np.arange(nframes),static[:,0])
ax[0].set_title('X coordinates, moving and static',fontsize=18)
ax[0].set_ylim((0,224))
ax[1].plot(np.arange(nframes),moving[:,1],np.arange(nframes),static[:,1])
ax[1].set_title('Y coordinates, moving and static',fontsize=18)
ax[1].set_ylim((0,224))
fig.tight_layout()
The MR image has pixel values defined for integer values of $(x,y)$. What is its value for a non-integer $(x,y)$?
Piece-wise constant interpolation gives results that are sometimes noticeably blocky. Sinc interpolation is expensive, and doesn't really work very well for images, because an image has a finite spatial extent. Bilinear interpolation tends to give the best results.
importlib.reload(submitted)
help(submitted.bilinear2d)
Help on function bilinear2d in module submitted:
bilinear2d(image, coordinates)
Use bilinear interpolation to find the value of an image
at non-integer coordinates. Notice that the image
is in (y,x) order while the coordinates are in (x,y)
order. Notice also that there is no guarantee that
the input coordinates are inside the original image;
you need to deal with the case that coordinates might
go outside the image.
@param:
image (ny,nx): grayscale image
coordinates (npix,2): x,y of desired pixels
@return:
intensities (npix,): intensities of those pixels,
computed by bilinear interpolation.
Again, let's test this with some known cases. For example, if the image is $[[0.01,1],[100,10000]]$, and the coordinate is $(0.6,0.7)$, then the returned value should be $0.0012+0.18+28+4200$.
If input pixels are beyond one edge of the input image, bilinear2d should respond by interpolating the nearest pixels. For example, the input coordinate $(-0.6,0.7)$ should find the value $0.003+70$.
If input pixels are beyond one corner of the input image, bilinear2d should respond by returning the nearest corner value. For example, the input coordinate $(3,3)$ should return the value $10000$.
importlib.reload(submitted)
img = np.array([[0.01,1],[100,10000]])
r = np.array([[0.6,0.7],[-0.6,0.7],[3,3]])
v = submitted.bilinear2d(img, r)
for i in range(3):
print('img[%s] = %8.8g'%(str(r[i,:]), v[i]))
img[[0.6 0.7]] = 4228.1812 img[[-0.6 0.7]] = 70.003 img[[3. 3.]] = 10000
The last part of this MP will generate the movie. Given a stack of points of dimension $(\text{nframes},23,2)$ and a static image of size $(\text{ny},\text{nx})$, you should create an output array of size $(\text{nframes},\text{ny},\text{nx})$, and then fill its pixel values as possible:
movie[t,:,:].importlib.reload(submitted)
help(submitted.animate)
Help on function animate in module submitted:
animate(image, moviepts, staticpts, triangles)
Generate a movie by animating a static image.
@param:
image (ny,nx): grayscale image
moviepts (nframes,23,2): points in each frame
staticpts (23,2): points in the static image
triangles (ntri,3): triplets of indices
@return:
movie (nframes,ny,nx): frames of the movie
staticpts = np.vstack((data['mr']['corner'],data['mr']['moving'],data['mr']['static']))
nframes = int(data['xr']['moving'].shape[0]/10)
moviepts = np.zeros((nframes,23,2))
for t in range(nframes):
augmented = np.hstack((data['xr']['moving'][10*t,:,:],np.ones((8,1))))
moving = augmented @ affine
moviepts[t,:,:] = np.vstack((data['mr']['corner'],moving,predstatic))
image = data['mr']['image']
triangles = data['mr']['triangles']
Doing the actual animation can take a lot of time. You might want to have your code print out frame numbers, as shown below.
importlib.reload(submitted)
movie = submitted.animate(image,moviepts,staticpts,triangles)
Synthesizing frame 0 out of 43 Synthesizing frame 1 out of 43 Synthesizing frame 2 out of 43 Synthesizing frame 3 out of 43 Synthesizing frame 4 out of 43 Synthesizing frame 5 out of 43 Synthesizing frame 6 out of 43 Synthesizing frame 7 out of 43 Synthesizing frame 8 out of 43 Synthesizing frame 9 out of 43 Synthesizing frame 10 out of 43 Synthesizing frame 11 out of 43 Synthesizing frame 12 out of 43 Synthesizing frame 13 out of 43 Synthesizing frame 14 out of 43 Synthesizing frame 15 out of 43 Synthesizing frame 16 out of 43 Synthesizing frame 17 out of 43 Synthesizing frame 18 out of 43 Synthesizing frame 19 out of 43 Synthesizing frame 20 out of 43 Synthesizing frame 21 out of 43 Synthesizing frame 22 out of 43 Synthesizing frame 23 out of 43 Synthesizing frame 24 out of 43 Synthesizing frame 25 out of 43 Synthesizing frame 26 out of 43 Synthesizing frame 27 out of 43 Synthesizing frame 28 out of 43 Synthesizing frame 29 out of 43 Synthesizing frame 30 out of 43 Synthesizing frame 31 out of 43 Synthesizing frame 32 out of 43 Synthesizing frame 33 out of 43 Synthesizing frame 34 out of 43 Synthesizing frame 35 out of 43 Synthesizing frame 36 out of 43 Synthesizing frame 37 out of 43 Synthesizing frame 38 out of 43 Synthesizing frame 39 out of 43 Synthesizing frame 40 out of 43 Synthesizing frame 41 out of 43 Synthesizing frame 42 out of 43
nframes, ny, nx = movie.shape
fig, ax = plt.subplots(1,1,figsize=(5,5))
ax.set_xlim(0,nx)
ax.set_ylim(0,ny)
ax.invert_yaxis()
frames = []
for t in range(nframes):
artist = ax.imshow(movie[t,:,:], cmap='gray')
frames.append([ artist ])
# Call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.ArtistAnimation(fig, frames, interval=69, blit=True)
plt.close(anim._fig)
import IPython
IPython.display.HTML(anim.to_html5_video())
Congratulations!
If you have openCV or ffmpeg installed, you can save the video to a file, add the audio to it, and then load it back in to play in the jupyter notebook. Alternatively, you can save the video as GIF, then use tools of your choice to convert it to mp4 and add the audio.
If you reached this point in the notebook, then probably your code is working well, but before you run the autograder on the server, you should first run it on your own machine.
You can do that by running the following command line. If everything works perfectly, it should give a result like this:
!python grade.py
............ ---------------------------------------------------------------------- Ran 12 tests in 4.973s OK
But suppose that something didn't work well. For example, suppose you run python grade.py, and you get error messages.
You can debug by loading the reference solutions, and comparing your answer to the answer in the reference solutions.
The solutions the autograder was expecting are in the file solutions.hdf5. You are strongly encouraged to browse that file, in case you need to debug any problems with the autograder. First, make sure that you have the module h5py installed on your computer:
!pip install h5py
Requirement already satisfied: h5py in /Users/jhasegaw/anaconda3/lib/python3.10/site-packages (3.9.0) Requirement already satisfied: numpy>=1.17.3 in /Users/jhasegaw/anaconda3/lib/python3.10/site-packages (from h5py) (1.24.3)
import h5py
with h5py.File('solutions.hdf5','r') as f:
print(list(f.keys()))
['affine', 'bary', 'cartesian', 'coordinates', 'image', 'intensities', 'movie', 'moviepts', 'moving', 'predstatic', 'static', 'staticpts', 'triangle']
As you can see, this file contains a lot of objects. These are named more or less as in the Jupyter Notebook above, so you can look at each object in detail to figure out why your code is different from the solution.
By the way, grade.py is running python's unittest facility, which is useful to understand if you ever intend to develop large-scale software projects. It runs all of the tests in the subdirectory tests, and raises an error only if one of the enclosed assert commands is found to be false.
When gradescope grades your code, it will run grade.py. It will test your code using the solutions in solutions.hdf5, and using the test code tests/test_visible.py. It will also test your code using some hidden tests. The hidden tests are actually exactly the same as those in tests/test_visible.py, but with different input parameters.
You can earn up to 10% extra credit on this MP by finishing the file called extra.py, and submitting it to MP1 Extra on Gradescope.
This function uses bilinear interpolation to do a straightforward upsampling of the image, from $n_{y1}\times n_{x1}$ pixels to $n_{y2}\times n_{x2}$ pixels.
import extra, importlib
importlib.reload(extra)
help(extra.bilinear_upsample)
Help on function bilinear_upsample in module extra:
bilinear_upsample(X, newshape)
Upsample an image using bilinear interpolation.
You can check your result using PIL.Image.BILINEAR
or scipy.misc.imresize, but those won't be
available on the autograder (see requirements.txt).
@param:
X (ny1,nx1) - input image
newshape (2,) - new shape, in y,x order (ny2,nx2)
@return:
Y (ny2,nx2) - output image
Note that you should find ny2 rows linearly spaced
in the range [0,ny1) in the input image, i.e.,
including the coordinate 0, but not including the coordinate
ny1. Likewise for nx.
When you think you have it working, you can test it by running:
python grade.py
in your terminal. Yes, indeed, this is the same code you ran for the regular MP! The only difference is that, when you unzipped mp1_extra.zip, it gave you the test file tests/text_extra.py. So now, when you run grade.py, it will grade both your regular MP, and the extra credit part:
!python grade.py
............ ---------------------------------------------------------------------- Ran 12 tests in 4.808s OK
Congratulations! That's the end of MP1. Good luck!