Question 1

What is the difference between classical AI and machine learning (ML)?

1. Classical AI is a broad field of which ML is one part.
2. Classical AI is an old approach that has been replaced by ML.
3. Classical AI uses human-designed algorithms to solve problems, ML uses machine-selected algorithms to solve problems.
4. ML is an idea or objective, classical AI is the closest we’ve come to realizing that ideal thus far.

Question 2

When an AI algorithm is described as supervised or unsupervised, that is a comment about

1. Whether a human is overseeing the AI’s work or the AI is working independently.
2. Whether the algorithm is trained before it is used or the training happens incrementally as new training data becomes available.
3. Whether the algorithm requires access to the Internet to operate or not.
4. Whether the output of the trained function is made up of real numbers or selected discrete options.
5. Whether the training data includes desired outputs for each input or not.

Question 3

When an AI algorithm is described as offline or online, that is a comment about

1. Whether a human is overseeing the AI’s work or the AI is working independently.
2. Whether the algorithm is trained before it is used or the training happens incrementally as new training data becomes available.
3. Whether the algorithm requires access to the Internet to operate or not.
4. Whether the output of the trained function is made up of real numbers or selected discrete options.
5. Whether the training data includes desired outputs for each input or not.

Question 4

When an AI algorithm is described as continuous or categorical, that is a comment about

1. Whether a human is overseeing the AI’s work or the AI is working independently.
2. Whether the algorithm is trained before it is used or the training happens incrementally as new training data becomes available.
3. Whether the algorithm requires access to the Internet to operate or not.
4. Whether the output of the trained function is made up of real numbers or selected discrete options.
5. Whether the training data includes desired outputs for each input or not.

Question 5

Which of the following are valid strategies for reducing overfitting?

1. Switch to a less flexible function family that can capture fewer of the details of the training data
2. Switch to a more flexible function family that can capture more of the shape of the training data
3. Try again with a lot less training data
4. Try again with a lot more training data
5. Try again with a model with fewer parameters
6. Try again with a model with more parameters
7. Use a fine-tuning pass to remove the over-fitting
8. Use a supervised approach to notice and remove the over-fitting

Question 6

The P in ChatGPT, one of the first highly-successful chatbots, stands for pretrained. This is suggests that ChatGPT

1. Is a revision of an older trained model.
2. Is provided in an untrained state, allowing users to train it by what they type into its prompt.
3. Was trained by OpenAI so that its users don’t have to do the training themselves.
4. Was trained twice: once to learn English, then a second time to learn what part of English answers questions.

Question 7

Both clustering and classification pick a label for each input, effectively breaking the input into groups that share a label. The difference between the two is

1. Classification gives a class of related answers, while clustering gives a single answer.
2. Clustering gives a cluster of related answers, while classification gives a single answer.
3. Clustering is given the labels in the training data, while classification is trained without knowing what the labels it should produce are.
4. Clustering is trained without knowing what the labels it should produce are, while classification is given the labels in the training data.

Question 8

The term generative AI refers to AI systems that

1. Can act in creative ways, generating new ideas.
2. Can generate the missing value in a pattern.
3. Generate other AI systems.
4. Have been generated as code, not just as abstract theory.
5. Move data through several steps called generations before finding the final answer.

Question 9

Which best describes how LLMs like ChatGPT, Gemini, Claude, and CoPilot respond to something you type?

1. 1. The text you type is parsed into its semantic meaning.
   2. The LLM compares the meaning to what it knows to find an answer.
   3. The answer is converted into text to send back to you.
2. 1. The text you type is parsed into its semantic meaning.
   2. The LLM compares the meaning to what it knows to find one thought that expands on that meaning.
   3. That thought is added to the meaning and converted into text to send back to you.
   4. The LLM compares the expanded meaning to what it knows to find one thought that expands on that meaning.
   5. That thought is added to the meaning and converted into text to send back to you.
   6. Steps 5 and 6 repeat until there’s no additional thoughts to add.
3. 1. The text you type is placed in a pattern Question: «your text». Answer:
   2. A fill-in-the-blank AI system tries to find one word that is most likely to follow that pattern; that word is shown to you and added to the pattern.
   3. A fill-in-the-blank AI system tries to find one word that is most likely to follow that now larger pattern; that word is shown to you and added to the pattern.
   4. Step 3 repeats until the word produced is a special end of response word.
4. 1. The text you type is placed in a pattern Question: «your text». Answer: «blank».
   2. a fill-in-the-blank AI system tries to find the text that best fills that blank based on the patterns of all text online.

Question 10

Which best describes the human-designed component of LLMs like ChatGPT, Gemini, Claude, and CoPilot?

1. Humans programmed how to break text into words and decide if words have similar meaning; training found patterns in words on the Internet without bothering with meaning.
2. Humans programmed how to break text into words; training figured out what words mean and built an understanding of the world by reading the Internet.
3. Humans programmed how to understand text’s meaning; training built an understanding of the world by reading the Internet.
4. Humans programmed most of their behavior; training only provides a little extra detail.
5. Humans programmed very little; training found what words are and patterns or meaning of words by reading the Internet.

Question 11

AIs hallucinate because

1. They are not able to capture the full patterns in the data due to too few parameters or too little training.
2. They are trained on human-generated data and humans say and do false things.
3. They don’t understand what they’re being asked to do and can’t tell when they can’t do something.

Question 12

Asking an AI about how to interact with an AI

1. Causes it to share relevant parts of its own documentation.
2. Is handled like other questions, summarizing what people say about AI interaction online.
3. Is pointless because AIs can’t tell you anything useful about this.
4. Reveals insights only an AI could have about an AI.

Question 13

Which of the following most impacts the energy used by an LLM? Assume the LLM has already been trained and is now just being used to respond to user prompts.

1. The length of the LLM’s reply (short answers require less work than longer ones)
2. The length of your prompt (short prompts require less work than longer ones)
3. The novelty of your prompt (common questions require less work than unusual ones)
4. The simplicity of the LLM’s reply (plain text requires less work than fancier formatting)

Question 14

Which of the following most impacts the energy used by an LLM? Assume the LLM has already been trained and is now just being used to respond to user prompts.

1. How many fine-tuning passes the LLM went through after being trained
2. How many layers deep the ANN inside the LLM is
3. The amount of training data used to create the LLM
4. The number of parameters given values during the LLMs training

Question 15

We discussed two reasons that an LLM might give different responses if you keep giving it the same prompt. They are:

1. The LLM considers not just your most recent prompt but also the prompts and replies that preceded it.
2. The artificial neural network inside the LLM includes artificial hormones and neurotransmitters and gets tired or annoyed at repeated prompts the same way that humans do.
3. The decoder might be randomized, picking any one of several likely words when converting from vector to text.
4. The encoder might be randomized, picking any one of several likely vectors when converting from text to vector.

Question 16

If I ask an LLM if its response to my last question is correct, it might identify errors in what it said before. It does this because

1. The LLM interface detects questions like this and switches to a more expensive and powerful model when they occur.
2. The LLM is lazy and doesn’t check its work unless I specifically ask it to.
3. The LLM is random in its ability to get things right; asking several times improves the chance that it randomly gets the right answer.
4. The LLM is trying to please me; I wouldn’t have asked if I thought it was correct.
5. The LLM matches the behavior of Internet communities, not single individuals, and members of communities often find problems other members’ work.

Question 17

Companies that provide LLMs typically offer several models: simple models, slower models that apply more reasoning, and agentic models that can do than just create responses.

The reasoning models typically differ from the simple models by

1. Converting the outputs of the simple models into actions to take, using other programs to take those actions, then showing you the results
2. Having more parameters
3. Having more training data
4. Repeatedly asking the simple models what needs to be done next and doing that, not engaging you again until the answer is either nothing or I don’t know
5. Sending a series of prompts to the simple models, chosen to ask better questions that you did

Question 18

Companies that provide LLMs typically offer several models: simple models, slower models that apply more reasoning, and agentic models that can do than just create responses.

The agentic models typically differ from the simple models by

1. Converting the outputs of the simple models into actions to take, using other programs to take those actions, then showing you the results
2. Having more parameters
3. Having more training data
4. Repeatedly asking the simple models what needs to be done next and doing that, not engaging you again until the answer is either nothing or I don’t know
5. Sending a series of prompts to the simple models, chosen to ask better questions that you did

Question 19

A biased algorithm

1. Accidentally treats some groups of people better than others
2. Gives answers that are incorrect in predictable ways
3. Gives incorrect answers often
4. Intentionally treats some groups of people better than others

Question 20

Which of the following are biases that are hard to avoid in machine learning?

1. AIs are less accurate when faced with situations that are uncommon in their training data.
2. AIs learn from the biases and bad habits of humans present in their training data.
3. AIs pick the most common patterns in their training data every time, effectively hiding minority views and uncommon patterns.

Question 21

Which of the following is not a pick 2 of 3 trade-off in software?

1. easy-to-use, secure, feature-rich
2. good design, solid implementation, complete testing
3. high quality, low cost, built quickly

Question 22

Which of the following are non-functional requirements for a texting/chat/IM app?

1. Allow adding new parties to existing group chats.
2. Auto-scroll to the most recent messages.
3. Be ready by June 25^th.
4. Follow a Spiral development model.
5. Run on both Android and iOS.

Question 23

In the time when laptops had good battery life but tablets were not yet common, student note taking on paper changed from having circles and colors and diagrams to being paragraphs and lists of text. This is an example of

1. Beta testing
2. Cultural change diving technical design
3. Implicit bias
4. Requirements spill-over
5. Unintended consequences of design

Question 24

Suppose I report a bug with software I use to the developers and they reply with that’s an upstream problem. They mean

1. The bug has been identified as something that isn’t important; fixing it will be deferred until some distant future date.
2. The bug has been identified as something that will be fixed soon.
3. The bug is based on an old version of the program; if I update it will go away.
4. The bug is caused by how that program is used by my computer; I should check the app that uses the software
5. The bug is in one of the program’s libraries; I should report it to that library.

Question 25

If code is brittle, that means that

1. It crashes easily.
2. It is easy for cyber criminals to hack into.
3. It is hard to maintain.
4. It meets only a small set of requirements.

Question 26

We discussed three common things to do during maintenance. Which of the following is not one of those?

1. Add new features.
2. Fix bugs discovered after the product is released.
3. Replace things that wear out.
4. Update code to react to upstream changes.

Question 27

Technical debt typically accumulates because

1. Accountants try to create technical debts (even if practically solvent) to claim as tax writeoffs.
2. Adding features quickly is more appealing than taking more time and money, even if adding them hurts other things.
3. Software is expensive to develop, so people take out loans to pay for it.
4. There’s so much technical advancement, developer’s knowledge gets out of date easily.

Question 28

Suppose your development team tells you they’ll be spending the next two weeks refactoring. At the end of that time you should expect

1. Bugs to have been removed
2. Features to have been added
3. Nothing to have changed
4. Requirements to have been created
5. Tests to have been written

Question 29

Suppose I am using software that is in beta. That means

1. It’s a secondary product; the premier product is the alpha.
2. It’s not fully tested; I should be prepared for it to crash.
3. It’s the second version of the software; the first version was alpha.

Question 30

What is the difference between Classical AI and Machine Learning?

Question 31

For the following types of machine learning, label each as either supervised or unsupervised, and as either categorical, continuous, or both. Discuss your answers and the meaning of those terms with your partner.

• Classification:
• Clustering:
• Generative:
• Regression:

Question 32

What is training data?

Question 33

What does it mean for a model to have many parameters?

Question 34

What is overfitting?

Question 35

How is overfitting related to training data and parameters?

Question 36

Write a description of the data that is passed between steps in LLM processing next to each arrow

• User
• ↓ ____
• Lexical Analysis
• ↓ ____
• Encoder
• ↓ ____
• Transformer
• ↓ ____
• Neural Network
• ↓ ____
• Decoder
• ↓ ____
• User

To create long responses, LLMs repeat some steps; add a ⭮-shaped arrow on the left showing that repetition.

Question 37

How does an LLM engage in reasoning?

Question 38

Is LLM reasoning an example of online learning? Explain your answer

Question 39

What is an agentic AI?

Question 40

Do agentic AIs engage in online learning? Explain your answer

Question 41

What are the five stages of software development in the Waterfall Model we learned in class?

1. _____
2. _____
3. _____
4. _____
5. _____

Question 42

In scrum and other agile methods, the backlog is a list of things each called ______

Question 43

The scrum backlog matches which of the five waterfall stages?