這三個(gè)名詞在機(jī)器學(xué)習(xí)領(lǐng)域的文章中極其常見，但很多人對(duì)他們的概念并不是特別清楚，尤其是后兩個(gè)經(jīng)常被人混用。Ripley, B.D（1996）在他的經(jīng)典專著Pattern Recognition and Neural Networks中給出了這三個(gè)詞的定義。
Training set: A set of examples used for learning, which is to fit the parameters [i.e., weights] of the classifier.
Validation set: A set of examples used to tune the parameters [i.e., architecture, not weights] of a classifier, for example to choose the number of hidden units in a neural network.
Test set: A set of examples used only to assess the performance [generalization] of a fully specified classifier.
顯然，training set是用來訓(xùn)練模型或確定模型參數(shù)的，如ANN中權(quán)值等； validation set是用來做模型選擇（model selection），即做模型的最終優(yōu)化及確定的，如ANN的結(jié)構(gòu)；而 test set則純粹是為了測試已經(jīng)訓(xùn)練好的模型的推廣能力。當(dāng)然，test set這并不能保證模型的正確性，他只是說相似的數(shù)據(jù)用此模型會(huì)得出相似的結(jié)果。但實(shí)際應(yīng)用中，一般只將數(shù)據(jù)集分成兩類，即training set 和test set，大多數(shù)文章并不涉及validation set。
Ripley還談到了Why separate test and validation sets?
1. The error rate estimate of the final model on validation data will be biased (smaller than the true error rate) since the validation set is used to select the final model.
2. After assessing the final model with the test set, YOU MUST NOT tune the model any further.