Practice quiz: Advice for applying machine learning
第 1 个问题：In the context of machine learning, what is a diagnostic?
【正确】A test that you run to gain insight into what is/isn’t working with a learning algorithm.
An application of machine learning to medical applications, with the goal of diagnosing patients’ conditions.
A process by which we quickly try as many different ways to improve an algorithm as possible, so as to see what works.
This refers to the process of measuring how well a learning algorithm does on a test set (data that the algorithm was not trained on).
【解释】Yes! A diagnostic is a test that you run to gain insight into what is/isn’t working with a learning algorithm, to gain guidance into improving its performance.
第 2 个问题：True/False? It is always true that the better an algorithm does on the training set, the better it will do on generalizing to new data.
【正确】False
True
【解释】Actually, if a model overfits the training set, it may not generalize well to new data.
第 3 个问题：For a classification task; suppose you train three different models using three different neural network architectures. Which data do you use to evaluate the three models in order to choose the best one?
The training set
All the data -- training, cross validation and test sets put together.
The test set
【正确】The cross validation set
【解释】Correct. Use the cross validation set to calculate the cross validation error on all three models in order to compare which of the three models is best.
Practice quiz: Bias and variance
第 1 个问题：If the model's cross validation error Jcv is much higher than the training error J_{train}J
train, this is an indication that the model has…
Low bias
【正确】high variance
high bias
Low variance
【解释】When Jcv >> Jtrain (whether Jtrain is also high or not, this is a sign that the model is overfitting to the training data and performing much worse on new examples.
第 2 个问题：Which of these is the best way to determine whether your model has high bias (has underfit the training data)?
See if the cross validation error is high compared to the baseline level of performance
【正确】Compare the training error to the baseline level of performance
See if the training error is high (above 15% or so)
Compare the training error to the cross validation error.
【解释】Correct. If comparing your model's training error to a baseline level of performance (such as human level performance, or performance of other well-established models), if your model's training error is much higher, then this is a sign that the model has high bias (has underfit).
第 3 个问题：You find that your algorithm has high bias. Which of these seem like good options for improving the algorithm’s performance? Hint: two of these are correct.
【正确】Collect additional features or add polynomial features
【解释】Correct. More features could potentially help the model better fit the training examples.
Collect more training examples
【正确】Decrease the regularization parameter λ (lambda)
【解释】Correct. Decreasing regularization can help the model better fit the training data.
Remove examples from the training set
第 4 个问题：You find that your algorithm has a training error of 2%, and a cross validation error of 20% (much higher than the training error). Based on the conclusion you would draw about whether the algorithm has a high bias or high variance problem, which of these seem like good options for improving the algorithm’s performance? Hint: two of these are correct.
Decrease the regularization parameter λ
Reduce the training set size
【正确】Increase the regularization parameter λ
【解释】Yes, the model appears to have high variance (overfit), and increasing regularization would help reduce high variance.
【正确】Collect more training data
【解释】Yes, the model appears to have high variance (overfit), and collecting more training examples would help reduce high variance.
Practice quiz: Machine learning development process
第 1 个问题：Which of these is a way to do error analysis?
Collecting additional training data in order to help the algorithm do better.
【正确】Manually examine a sample of the training examples that the model misclassified in order to identify common traits and trends.
Calculating the test error Jtest
Calculating the training error Jtrain
【解释】Correct. By identifying similar types of errors, you can collect more data that are similar to these misclassified examples in order to train the model to improve on these types of examples.
第 2 个问题：We sometimes take an existing training example and modify it (for example, by rotating an image slightly) to create a new example with the same label. What is this process called?
Machine learning diagnostic
Error analysis
Bias/variance analysis
【正确】Data augmentation
【解释】Yes! Modifying existing data (such as images, or audio) is called data augmentation.
第 3 个问题：What are two possible ways to perform transfer learning? Hint: two of the four choices are correct.
Download a pre-trained model and use it for prediction without modifying or re-training it.
【正确】You can choose to train just the output layers' parameters and leave the other parameters of the model fixed.
【解释】Correct. The earlier layers of the model may be reusable as is, because they are identifying low level features that are relevant to your task.
【正确】You can choose to train all parameters of the model, including the output layers, as well as the earlier layers.
【解释】Correct. It may help to train all the layers of the model on your own training set. This may take more time compared to if you just trained the parameters of the output layers.
Given a dataset, pre-train and then further fine tune a neural network on the same dataset.