在電腦文件夾E:\other\matlab 2007a\work\SVM\libsvm-mat-3.0-1 ，這個(gè)是已經(jīng)編譯好的，到64位機(jī)上要重新編譯(不要利用別人傳的，因?yàn)榭赡芨倪^SVM程序，例如Libing wang他改過其中程序，最原始版本：E:\other\matlab 2007a\work\SVM\libsvm-mat-3.0-1.zip,從http://www.csie.ntu.edu.tw/~cjlin/libsvm/matlab/libsvm-mat-3.0-1.zip下載)。svmtrain在matlab自帶的工具箱中也有這個(gè)函數(shù)， libing 講libsvm-mat-3.0-1放到C:\Program Files\MATLAB\R2010a\toolbox\目錄，再adddpath和savepath即可。如果產(chǎn)生以下問題：每次都要 adddpath和savepath ，在matlab重新啟動(dòng)后要重新
adddpath和savepath。解決方案：可以在要運(yùn)行的程序前面添加如下語句即可：
addpath('C:\Program Files\MATLAB\R2010a\toolbox\libsvm-mat-3.0-1');
README文件寫得很好，其中的Examples完全理解(包括Precomputed Kernels.Constructing a linear kernel matrix and then using the precomputed kernel gives exactly the same testing error as using the LIBSVM built-in linear kernel.核就是相似度，自己想定義什么相似度都可以)
(1) model = svmtrain(training_label_vector, training_instance_matrix [, 'libsvm_options']);
libsvm_options的設(shè)置：
Examples of options: -s 0 -c 10 -t 1 -g 1 -r 1 -d 3
Classify a binary data with polynomial kernel (u'v+1)^3 and C = 10
options:
-s svm_type : set type of SVM (default 0)
0 -- C-SVC
1 -- nu-SVC
2 -- one-class SVM
3 -- epsilon-SVR
4 -- nu-SVR
C-SVC全稱是什么？
C-SVC(C-support vector classification),nu-SVC(nu-support vector classification),one-class SVM(distribution estimation),epsilon-SVR(epsilon-support vector regression),nu-SVR(nu-support vector regression)
-t kernel_type : set type of kernel function (default 2)
0 -- linear: u'*v
1 -- polynomial: (gamma*u'*v + coef0)^degree
2 -- radial basis function: exp(-gamma*|u-v|^2)
3 -- sigmoid: tanh(gamma*u'*v + coef0)
-d degree : set degree in kernel function (default 3)
-g gamma : set gamma in kernel function (default 1/num_features)
-r coef0 : set coef0 in kernel function (default 0)
-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
-m cachesize : set cache memory size in MB (default 100)
-e epsilon : set tolerance of termination criterion (default 0.001)
-h shrinking: whether to use the shrinking heuristics, 0 or 1 (default 1)
-b probability_estimates: whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
-wi weight: set the parameter C of class i to weight*C, for C-SVC (default 1)
The k in the -g option means the number of attributes in the input data.
(2)如何采用線性核？
matlab> % Linear Kernel
matlab> model_linear = svmtrain(train_label, train_data, '-t 0');
嚴(yán)格講，線性核也要像高斯核一樣調(diào)整c這個(gè)參數(shù)，Libing wang講一般C=1效果比較好，可能調(diào)整效果差異不大，當(dāng)然要看具體的數(shù)據(jù)集。c大，從SVM目標(biāo)函數(shù)可以看出，c越大，相當(dāng)于懲罰松弛變量，希望松弛變量接近0，即都趨向于對(duì)訓(xùn)練集全分對(duì)的情況，這樣對(duì)訓(xùn)練集測(cè)試時(shí)準(zhǔn)確率很高，但推廣能力未必好，即在測(cè)試集上未必好。c小點(diǎn)，相當(dāng)于邊界的有些點(diǎn)容許分錯(cuò)，將他們當(dāng)成噪聲點(diǎn)，這樣外推能力比較好。
(3)如何采用高斯核？
matlab> load heart_scale.mat
matlab> model = svmtrain(heart_scale_label, heart_scale_inst, '-c 1 -g 0.07');
高斯的SVM比線性SVM效果要差，為什么？
20150420 libing討論，可能的解釋：樣本少，不適合高斯核。范圍有限，也許更廣泛的參數(shù)范圍會(huì)有更好的效果
(4)如何實(shí)現(xiàn)交叉驗(yàn)證？
README文件有如下一句話：If the '-v' option is specified, cross validation is
conducted and the returned model is just a scalar: cross-validation
accuracy for classification and mean-squared error for regression.
(5) 如何調(diào)整高斯核的兩個(gè)參數(shù)？
思路1：在訓(xùn)練集上調(diào)整兩個(gè)參數(shù)使在訓(xùn)練集上測(cè)試錯(cuò)誤率最低，就選這樣的參數(shù)來測(cè)試測(cè)試集
思路1的問題：Libing Wang講這樣很容易過學(xué)習(xí)，因?yàn)樵谟?xùn)練集上很容易達(dá)到100%準(zhǔn)確率，但在測(cè)試集上未必好，即過學(xué)習(xí)。用思路2有交叉驗(yàn)證，推廣性能比較好(交叉驗(yàn)證將訓(xùn)練集隨機(jī)打亂，推廣性能很好)
思路2：% E:\other\matlab 2007a\work\DCT\DCT_original\network.m
思路2的問題：針對(duì)不同的數(shù)據(jù)集，這兩個(gè)參數(shù)分別在什么范圍內(nèi)調(diào)整,有沒有什么經(jīng)驗(yàn)？
方式1：就是network.m中g(shù)amma的取值
方式2：http://www.shnenglu.com/guijie/archive/2010/12/02/135243.html.
其他答案：除了在訓(xùn)練集上做交叉驗(yàn)證，還有另外一種思路：類似A Regularized Approach to Feature Selection for Face Detection (ACCV 2007)的4.2節(jié):訓(xùn)練集、驗(yàn)證集和測(cè)試集，Libing講該文4.2節(jié)調(diào)參數(shù)除了分成訓(xùn)練集、驗(yàn)證集和測(cè)試集，沒有其他什么的。Libing講在訓(xùn)練集上交叉驗(yàn)證也相當(dāng)于訓(xùn)練集挑了一部分做驗(yàn)證，原理一樣。
(6)如何采用預(yù)定義核？
To use precomputed kernel, you must include sample serial number asthe first column of the training and testing data (assume your kernel matrix is K, # of instances is n):
matlab> K1 = [(1:n)', K]; % include sample serial number as first column
matlab> model = svmtrain(label_vector, K1, '-t 4');
matlab> [predict_label, accuracy, dec_values] = svmpredict(label_vector, K1, model); % test the training data
We give the following detailed example by splitting heart_scale into 150 training and 120 testing data. Constructing a linear kernel matrix and then using the precomputed kernel gives exactly the same testing error as using the LIBSVM built-in linear kernel.
matlab> load heart_scale.mat
matlab>
matlab> % Split Data
matlab> train_data = heart_scale_inst(1:150,:);
matlab> train_label = heart_scale_label(1:150,:);
matlab> test_data = heart_scale_inst(151:270,:);
matlab> test_label = heart_scale_label(151:270,:);
matlab>
matlab> % Linear Kernel
matlab> model_linear = svmtrain(train_label, train_data, '-t 0');
matlab> [predict_label_L, accuracy_L, dec_values_L] = svmpredict(test_label, test_data, model_linear);
matlab>
matlab> % Precomputed Kernel
matlab> model_precomputed = svmtrain(train_label, [(1:150)', train_data*train_data'], '-t 4');
matlab> [predict_label_P, accuracy_P, dec_values_P] = svmpredict(test_label, [(1:120)', test_data*train_data'], model_precomputed);
matlab>
matlab> accuracy_L % Display the accuracy using linear kernel
matlab> accuracy_P % Display the accuracy using precomputed kernel
(7)如何實(shí)現(xiàn)概率估計(jì)？
For probability estimates, you need '-b 1' for training and testing:
matlab> load heart_scale.mat
matlab> model = svmtrain(heart_scale_label, heart_scale_inst, '-c 1 -g 0.07 -b 1');
matlab> load heart_scale.mat
matlab> [predict_label, accuracy, prob_estimates] = svmpredict(heart_scale_label, heart_scale_inst, model, '-b 1');
非概率估計(jì)
matlab> load heart_scale.mat
matlab> model = svmtrain(heart_scale_label, heart_scale_inst, '-c 1 -g 0.07');
matlab> [predict_label, accuracy, dec_values] = svmpredict(heart_scale_label, heart_scale_inst, model); % test the training data
(8) svmpredict的用法(摘自libsvm-mat-2.9-1的README)
[predicted_label, accuracy, decision_values/prob_estimates] = svmpredict(testing_label_vector, testing_instance_matrix, model [, 'libsvm_options']);
輸入：testing_label_vector， If labels of test data are unknown, simply use any random values. (type must be double)。模型一旦確定，預(yù)測(cè)的標(biāo)記就確定了，如果不利用第二個(gè)輸出accuracy，則testing_label_vector隨便設(shè)置，當(dāng)然如果要利用accuracy，就要將testing_label_vector設(shè)置成測(cè)試標(biāo)記了。(Action recognition\ASLAN database中的代碼CLSlibsvmC，第九行用到svmpredict，testing_label_vector設(shè)置成ones(size(Samples,2),1)，是無所謂的)。
svmpredict輸出的含義：
predictd_label, is a vector of predicted labels(故CLSlibsvmC的12到14行沒用);
胥志偉I-Rising(285308540) 2015/5/31 21:56:30
各位老師，同學(xué)。請(qǐng)問有人研究過svm 中predict的decision value嗎？麻煩幫忙解釋下怎么計(jì)算的。謝謝。
蘇松志-T-廈大(14291414) 2015/5/31 22:04:40
指的是libsvm？優(yōu)化完之后得到每個(gè)支持向量的alph_i,然后，計(jì)算輸入的x和各個(gè)支持向量的核函數(shù)距離ki，sum(ki*alph_i*yi) for all SUPPORT VECTORS,+b,就是按照教科書上的公式
說明：以上understand completely,對(duì)應(yīng)楊光正教材P28公式(2.54)
胥志偉I-Rising(285308540) 2015/5/31 22:08:30
多類的問題也是這樣嗎？
蘇松志-T-廈大(14291414) 2015/5/31 22:13:41
是的，多類采用的是1vs1，所有里面的sv_coef是一個(gè)mxn的矩陣，m：支持向量的數(shù)目，n： 類別數(shù)目-1，然后有一個(gè)rho向量，套用公式，計(jì)算
摘自libsvm-mat-3.0-1的README
The function 'svmpredict' has three outputs. The first one, predictd_label, is a vector of predicted labels. The second output, accuracy, is a vector including accuracy (for classification), mean squared error, and squared correlation coefficient (for regression). The third is a matrix containing decision values or probability estimates (if '-b 1' is specified).
If k is the number of classes, for decision values, each row includes results of predicting
k(k-1)/2 binary-class SVMs.
For probabilities, each row contains k values indicating the probability that the testing instance is in each class. Note that the order of classes here is the same as 'Label' field in the model structure.
(9)LibSVM是如何采用one-versus-rest和one-verse-one實(shí)現(xiàn)多類分類的？one-versus-rest和one-verse-one的定義見模式識(shí)別筆記第四頁反面(同時(shí)見孫即祥教材P47)。找libing wang和junge zhang,他們都講沒對(duì)這個(gè)深究過。根據(jù)“If k is the number of classes, for decision values, each row includes results of predicting
k(k-1)/2 binary-class SVMs.
For probabilities, each row contains k values indicating the probability that the testing instance is in each class. ”，我覺得應(yīng)該是
probabilities實(shí)現(xiàn)的是one-versus-rest，即采用-b 1這個(gè)選項(xiàng)，他倆都覺得我理解應(yīng)該是正確的。junge講參加pascal競賽和imagenet，他們都是訓(xùn)練k個(gè)SVM(即one-versus-rest,沒用one-versus-one,后者太慢，而且估計(jì)效果差不多)，沒有直接采用SVM做多類問題。
20130910 LibSVM作者回信：
Libsvm implements only 1vs1.For 1vsrest, you can check the followinglibsvm faqQ: LIBSVM supports 1-vs-1 multi-class classification. If instead I wouldlike to use 1-vs-rest, how to implement it using MATLAB interface?
網(wǎng)址：http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f808
Q: LIBSVM supports 1-vs-1 multi-class classification. If instead I would like to use 1-vs-rest, how to implement it using MATLAB interface?
Please use code in the following directory. The following example shows how to train and test the problem dna (training and testing).
Load, train and predict data:
[trainY trainX] = libsvmread('./dna.scale');
[testY testX] = libsvmread('./dna.scale.t');
model = ovrtrain(trainY, trainX, '-c 8 -g 4');
[pred ac decv] = ovrpredict(testY, testX, model);
fprintf('Accuracy = %g%%\n', ac * 100); Conduct CV on a grid of parametersbestcv = 0;
for log2c = -1:2:3,
for log2g = -4:2:1,
cmd = ['-q -c ', num2str(2^log2c), ' -g ', num2str(2^log2g)];
cv = get_cv_ac(trainY, trainX, cmd, 3);
if (cv >= bestcv),
bestcv = cv; bestc = 2^log2c; bestg = 2^log2g;
end
fprintf('%g %g %g (best c=%g, g=%g, rate=%g)\n', log2c, log2g, cv, bestc, bestg, bestcv);
end
end
(9)如何實(shí)現(xiàn)驗(yàn)證模式下的準(zhǔn)確率？見我寫的程序RVM\code\Yale\SVM\TestYale_SVM_2classes
--------------------------------------------------------------------------------------------------------------------------------------------------------http://blog.sina.com.cn/s/blog_64b046c701018c8n.html
MATLAB自帶的svm實(shí)現(xiàn)函數(shù)與libsvm差別小議
1 MATLAB自帶的svm實(shí)現(xiàn)函數(shù)僅有的模型是C-SVC(C-support vector classification)； 而libsvm工具箱有C-SVC(C-support vector classification),nu-SVC(nu-support vector classification),one-class SVM(distribution estimation),epsilon-SVR(epsilon-support vector regression),nu-SVR(nu-support vector regression)等多種模型可供使用。 2 MATLAB自帶的svm實(shí)現(xiàn)函數(shù)僅支持分類問題，不支持回歸問題；而libsvm不僅支持分類問題，亦支持回歸問題。 3 MATLAB自帶的svm實(shí)現(xiàn)函數(shù)僅支持二分類問題，多分類問題需按照多分類的相應(yīng)算法編程實(shí)現(xiàn)；而libsvm采用1v1算法支持多分類。 4 MATLAB自帶的svm實(shí)現(xiàn)函數(shù)采用RBF核函數(shù)時(shí)無法調(diào)節(jié)核函數(shù)的參數(shù)gamma，貌似僅能用默認(rèn)的；而libsvm可以進(jìn)行該參數(shù)的調(diào)節(jié)。 5 libsvm中的二次規(guī)劃問題的解決算法是SMO；而MATLAB自帶的svm實(shí)現(xiàn)函數(shù)中二次規(guī)劃問題的解法有三種可以選擇：經(jīng)典二次方法；SMO；最小二乘。（這個(gè)是我目前發(fā)現(xiàn)的MATLAB自帶的svm實(shí)現(xiàn)函數(shù)唯一的優(yōu)點(diǎn)~） SVM 理論部分
SVM下面推導(dǎo)核化形式(Eric Xing教材)+M. Belkin, P. Niyogi, and V. Sindhwani, “Manifold Regularization: AGeometric Framework for Learning from Labeled and Unlabeled Examples,” J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006的4.3和4.4節(jié).+Ensemble Manifold Regularization (TPAMI 2012)
電腦里的"ZhuMLSS14.pdf
"是很好的入門材料