Summarising the diagram above, with regularisation:

  1. Large λ -> High Bias (Underfit)
  2. Intermediate λ -> "Just right"
  3. Small λ -> High Variance (Overfit)

How do we choose our parameter λ to get it 'just right' ?