這三個名詞在機器學習領域的文章中極其常見，但很多人對他們的概念并不是特別清楚，尤其是后兩個經常被人混用。Ripley, B.D（1996）在他的經典專著Pattern Recognition and Neural Networks中給出了這三個詞的定義。
Training set: A set of examples used for learning, which is to fit the parameters [i.e., weights] of the classifier.
Validation set: A set of examples used to tune the parameters [i.e., architecture, not weights] of a classifier, for example to choose the number of hidden units in a neural network.
Test set: A set of examples used only to assess the performance [generalization] of a fully specified classifier.
顯然，training set是用來訓練模型或確定模型參數的，如ANN中權值等； validation set是用來做模型選擇（model selection），即做模型的最終優化及確定的，如ANN的結構；而 test set則純粹是為了測試已經訓練好的模型的推廣能力。當然，test set這并不能保證模型的正確性，他只是說相似的數據用此模型會得出相似的結果。但實際應用中，一般只將數據集分成兩類，即training set 和test set，大多數文章并不涉及validation set。
Ripley還談到了Why separate test and validation sets?
1. The error rate estimate of the final model on validation data will be biased (smaller than the true error rate) since the validation set is used to select the final model.
2. After assessing the final model with the test set, YOU MUST NOT tune the model any further.