Evaluate the convergence of our algorithms

We have four codes that should be computing identical quantities in the limit as some computational resource parameter goes to infinity. In this experiment, we test to make sure all the algorithms converge as we increase the computational resourses supplied. Convergence is measured as the change in the solution when adding a bit more computational power.

Contents

Experimental setup

This experiment should be run from the rapr/experiments/correctness directory. To reduce the computational load, we'll load the cs-stanford graph and just take the largest strong component.

cwd = pwd;
dirtail = 'experiments/correctness';
if strcmp(cwd(end-length(dirtail)+1:end),dirtail) == 0
    warning('%s should be executed from rapr/%s\n', mfilename, dirtail);
end
load ('../../data/wb-cs.stanford.mat');

Vary the graph (the commented line was used in initial testing...)

% just use the largest strong component for this experiment;
P = Pcc;

%load ('../../data/example-small.mat');

Add the necessary path

addpath('../../matlab');

Show convergence of each algorithm

We'll use a series of distributions and show how each algorithm converges. Then, for each algorithm and each distribution, we will look at how much each solution changes from one step to the next. Again, we use the same distributions we've been using throughout the paper.

Nmc=1e5; Npd=Nmc; gqNs=0:10:100; pceNs=gqNs;
mcalg='direct';pcealg='direct';gqalg='direct';
du1 = alphadist('unif',0.6,0.9);           % uniform [0.6,0.9]
db1 = alphadist('beta',2,16,0,1);          % skewed right
db2 = alphadist('beta',1,1,0.1,0.9);       % equicentric
db3 = alphadist('beta',-0.5,-0.5,0.2,0.7); % bi-modal
ds = {du1,db1,db2,db3}; nds = length(ds);
results = []; [results(1:nds).d] = deal(ds{:}); % intialize results

Compute a monte-carlo approximation and store the convergence results

tic;
for di=1:nds
    [ex stdx alphas conv]=mcrapr(P,Nmc,results(di).d,mcalg);
    results(di).ex.mc=ex; results(di).stdx.mc=stdx; results(di).conv.mc=conv;
end
fprintf('... %7i monte carlo samples took %f seconds\n', Nmc, toc);
save 'converge-results.mat' results P % save intermediate results
...  100000 monte carlo samples took 6590.400596 seconds

Compute a path damping approximation and store the convergence results

tic;
for di=1:nds
    [ex stdx pdcoeffs conv] = pdrapr(P,results(di).d,0,Npd);
    results(di).ex.pd=ex; results(di).stdx.pd=stdx; results(di).conv.pd=conv;
end
fprintf('... %7i path damping iterations took %f seconds\n', Npd, toc);
save 'converge-results.mat' results P % save intermediate results
Warning: not computing stdx
Warning: not computing stdx
Warning: not computing stdx
Warning: not computing stdx
...  100000 path damping iterations took 918.683301 seconds

Compute a seris of GQ approximations and explicitly compute the changes that result.

tic
for di=1:nds
    conv=zeros(length(gqNs),3); d=results(di).d;
    for Ni=1:length(gqNs)
        [ex stdx] = gqrapr(P,gqNs(Ni)+1,d,gqalg);
        [ex1 stdx1] = gqrapr(P,gqNs(Ni)+2,d,gqalg);
        conv(Ni,:) = [gqNs(Ni),norm(ex-ex1,1),norm(stdx-stdx1,1)];
    end
    results(di).ex.gq=ex; results(di).stdx.gq=stdx; results(di).conv.gq=conv;
end
fprintf('... %i GQPR solves took %f seconds\n',sum(gqNs)+sum(gqNs+1),toc);
save 'converge-results.mat' results P % save intermediate results
... 1111 GQPR solves took 74.039305 seconds

Compute a seris of PCE approximations and explicitly compute the changes that result. (Note that this will be identical to the GQ results.)

tic
for di=1:nds
    conv=zeros(length(pceNs),3); d=results(di).d;
    for Ni=1:length(pceNs)
        [ex stdx] = pcerapr(P,pceNs(Ni),d,pcealg);
        [ex1 stdx1] = pcerapr(P,pceNs(Ni)+1,d,pcealg);
        conv(Ni,:) = [pceNs(Ni),norm(ex-ex1,1),norm(stdx-stdx1,1)];
    end
    results(di).ex.pce=ex; results(di).stdx.pce=stdx; results(di).conv.pce=conv;
end
fprintf('... %i PCEPR solves took %f seconds\n',sum(pceNs)+sum(pceNs+1),toc);
Warning: stdx not computed when N=0
Warning: stdx not computed when N=0
Warning: stdx not computed when N=0
Warning: stdx not computed when N=0
... 1111 PCEPR solves took 2529.075452 seconds

Save the results

save 'converge-results.mat' results P

Plot the convergence results by algorithm.

Each distribution is drawn with its native colors from previous plots with points indicated by '.' for an expectation and '+' for a standard deviation. We do not es

algs={'mc','pd','gq','pce'}; nas=length(algs); ls={'LineWidth',1};
s={'-.','-','--',':'}; c={[1 0.5 0.5],[0.5 0.5 1],[0 0.75 0.5],[1 0 0.25]};
for ai=1:nas
    figure(ai); set(gcf,'Color','w'); clf;
    for di=1:nds
        cd=results(di).conv.(algs{ai}); x=cd(:,1); y1=cd(:,2);
        if log10(max(x))>2.5, pfun='loglog'; else pfun='semilogy'; end
        if size(cd,2)>2, h=feval(pfun,x,y1,['.',s{di}],x,cd(:,3),['+',s{di}]);
        else h=feval(pfun,x,y1,['.',s{di}]); end
        hold on; set(h,'Color',c{di},ls{:}); pbaspect([2.5 1 1]); ylim([1e-16 1]);
        box off; print(gcf,sprintf('converge-%s.eps',algs{ai}),'-depsc2');
    end
end