Lab 7: Open-ended

Introduction

This is an open-ended lab working with neural networks. The first thing to do is decide on your topic, and then email me and get my approval. Then, work on the project and create a submittal in the form of a notebook containing a detailed description of the problem, investigation/research you've done, your implementation, and a summary of your results. It's not a requirement that you have positive results; even negative results can be useful.

As with all labs, you'll submit to me and we'll go through a back-and-forth review process until it is finally accepted.

I recommend using fastai for this lab, but I'm open to other libraries.

Expected time spent on this lab: 20-30 hours.

Topics

Implement Object detection. You are given an image with multiple objects in it. You predict a tight bounding box and category around each object. Potential papers of interest: https://arxiv.org/abs/1904.11490, https://github.com/facebookresearch/detr.
Implement finding facial features, and use that to add mustache, glasses, etc. to: a picture or a movie or a live incoming video feed.
Speech commands: create a simple Alexa clone: Given audio, it'll alert when somebody says "Alexa", and will then listen for one of four commands (what time is it? What is the weather?, ...) and answer. You can generate a synthetic dataset by taking any audio stream and splicing in yourself or others saying "Alexa" or one of the four audio command.
Given a low-quality (skype or mobile phone call) voice call, and high-quality examples of speech from the same speaker, process the low-quality voice call into high quality.
Could generate samples by taking high-quality speech and then programatically downgrading it to low-quality.
Or, could use a GAN with Generator generating high-quality speech from low-quality speech, and an adversary that can learn to spot fake high-quality speech). To make sure the Generator is generating a high-quality version of the original low-quality words, you can use Google Cloud's Speech-to-text API to ensure that both versions represent the same words.
Colorize black-and-white photos. For dataset, use exising color images, and programatically make them grayscale.
Participate in one of the following past Kaggle competitions:
Participate in one of the following current Kaggle competitions:
Come up with a topic of your own.

For any of the topics, see me for brainstorming and further details or pointers.

Challenges:

Where will you get (labeled?) data?
How will you create architecture?
How long will training take?

Challenge 1 Participate in a current Kaggle competition and win prize money:)

This completes the lab. Submit instructions

Make sure that the output of all cells is up-to-date.
Rename your notebook:
1. Click on notebook name at the top of the window.
2. Rename to "CS152Sp21Lab7 FirstName1/FirstName2" (using the correct lab number, along with your two first names). I need this naming so I can easily navigate through the large number of shared docs I will have by the end of the semester.
Choose File/Save
Share your notebook with me:
1. Click on the Share button at the top-right of your notebook.
2. Enter rhodes@g.hmc.edu as the email address.
3. Click the pencil icon and select Can comment.
4. Click on Done.
Enter the URL of your colab notebook in this submittal form. Do not copy the URL from the address bar (which may contain an authuser parameter and which I will not be able to open). Instead, click Share and Copy link to obtain the correct link. Enter your names in alphabetical order.
At this point, you and I will go back and forth until the lab is approved.
1. I will provide inline comments as I evaluate the submission (Google should notify you of these comments via email).
2. You will then need to address those comments. Please do not resolve or delete the comments. I will use them as a record of our conversation. You can respond to them ("Fixed" perhaps).
3. Once you have addressed all the comments in this round, fill out the submittal form again.
4. Once I am completely satisifed with your lab, I will add a LGTM (Looks Good to Me) comment
5. At that point, setup an office hour appointment with me. Ill meet with you and your partner and we'll have a short discussiona about the lab. Both of you should be able to answer questions about any part of the lab.