evaluation
Class EvaluationFromFile

java.lang.Object
  evaluation.EvaluationFromFile

public class EvaluationFromFile
extends java.lang.Object

Evaluation class.

Author:

Eleftherios Spyromitros-Xioufis

Nested Class Summary

static class

EvaluationFromFile.SearchMethod

Field Summary

private double[]	averagePrecision Average precision per query.
double	avgLookUpTimeCol
double	avgLookUpTimeDist
double	avgSearchTimeCol
double	avgSearchTimeDist
private static AbstractSearchStructure	collectionIndex A VladArray containing the vlad vectors of all the collection images.
private static VladArray	collectionVladArray A VladArray containing the vlad vectors of all the collection images.
private static AbstractSearchStructure	distractorsIndex A VladArray containing the vlad vectors of all the distractor images.
static int	HolidaysQueries
static int	HolidaysSize
private java.util.HashMap<java.lang.String,java.util.HashSet<java.lang.String>>	junkSet Contains the names of junk images.
static int	OxfordQueries The number of query images in each dataset.
static int	OxfordSize The sizes of the evaluation datasets.
static int	ParisQueries
static int	ParisSize
private java.util.HashMap<java.lang.String,java.util.HashSet<java.lang.String>>	posSet Contains the names of positive images.
private java.util.ArrayList<java.lang.String>	queryNames Contains the names of query images.
private double[]	recallAtk Recall @ k per query.
static int	recallLevel
static int	w Number of lists to visit in the IVFADC method.

Constructor Summary

EvaluationFromFile()

Method Summary

private double	computeAveragePrecision(Result[] nearestNeighbors, java.lang.String queryName) Computes the AP for a given queryId and a sorted list of nearest neighbors.
void	computeMeasures(int numResults) Computes the measures.
private double	computeRecallAtk(Result[] nearestNeighbors, java.lang.String queryName, int k, boolean ignoreQuery)
double[]	getAveragePrecision()
double	getAvgLookUpTimeCol()
double	getAvgLookUpTimeDist()
double	getAvgSearchTimeCol()
double	getAvgSearchTimeDist()
double[]	getRecallAtk()
void	loadGroundTruthHolidays(java.lang.String filename)
void	loadGroundTruthOxford(java.lang.String gtFolder)
void	loadGroundTruthParis(java.lang.String gtFolder)
void	loadGroundTruthUKB(java.lang.String gtFolder)
static void	main(java.lang.String[] args)
void	outputAverageMeasures() Used for average measure reporting.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OxfordSize

public static final int OxfordSize

The sizes of the evaluation datasets.

See Also:

Constant Field Values

ParisSize

public static final int ParisSize

See Also:

Constant Field Values

HolidaysSize

public static final int HolidaysSize

See Also:

Constant Field Values

OxfordQueries

public static final int OxfordQueries

The number of query images in each dataset.

See Also:

Constant Field Values

ParisQueries

public static final int ParisQueries

See Also:

Constant Field Values

HolidaysQueries

public static final int HolidaysQueries

See Also:

Constant Field Values

averagePrecision

private double[] averagePrecision

Average precision per query.

avgLookUpTimeCol

public double avgLookUpTimeCol

avgSearchTimeCol

public double avgSearchTimeCol

avgLookUpTimeDist

public double avgLookUpTimeDist

avgSearchTimeDist

public double avgSearchTimeDist

recallAtk

private double[] recallAtk

Recall @ k per query.

recallLevel

public static final int recallLevel

See Also:

Constant Field Values

w

public static final int w

Number of lists to visit in the IVFADC method.

See Also:

Constant Field Values

collectionVladArray

private static VladArray collectionVladArray

A VladArray containing the vlad vectors of all the collection images.

collectionIndex

private static AbstractSearchStructure collectionIndex

A VladArray containing the vlad vectors of all the collection images.

distractorsIndex

private static AbstractSearchStructure distractorsIndex

A VladArray containing the vlad vectors of all the distractor images.

junkSet

private java.util.HashMap<java.lang.String,java.util.HashSet<java.lang.String>> junkSet

Contains the names of junk images.

queryNames

private java.util.ArrayList<java.lang.String> queryNames

Contains the names of query images.

posSet

private java.util.HashMap<java.lang.String,java.util.HashSet<java.lang.String>> posSet

Contains the names of positive images.

Constructor Detail

EvaluationFromFile

public EvaluationFromFile()

Method Detail

getAveragePrecision

public double[] getAveragePrecision()

getRecallAtk

public double[] getRecallAtk()

getAvgLookUpTimeCol

public double getAvgLookUpTimeCol()

getAvgSearchTimeCol

public double getAvgSearchTimeCol()

getAvgLookUpTimeDist

public double getAvgLookUpTimeDist()

getAvgSearchTimeDist

public double getAvgSearchTimeDist()

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Parameters:

args - [0] Full path to the evaluation setup file.

args - [1] Full path to the evaluation output file (where the evaluation results will be written).

Throws:

java.lang.Exception

computeAveragePrecision

private double computeAveragePrecision(Result[] nearestNeighbors,
                                       java.lang.String queryName)

Computes the AP for a given queryId and a sorted list of nearest neighbors. The queryId is used to obtain the set of positive and junk images. Positive images are treated as correct hits and junk images are ignored in AP calculation (AP is not affected if a junk image is returned). The query itself is also ignored.

Parameters:

nearestNeighbors -

queryName -

Returns:

computeMeasures

public void computeMeasures(int numResults)
                     throws java.lang.Exception

Computes the measures.

Parameters:

numResults -

threshold -

Throws:

java.lang.Exception

computeRecallAtk

private double computeRecallAtk(Result[] nearestNeighbors,
                                java.lang.String queryName,
                                int k,
                                boolean ignoreQuery)

outputAverageMeasures

public void outputAverageMeasures()

Used for average measure reporting. ComputeMeasures should have been called first.

loadGroundTruthOxford

public void loadGroundTruthOxford(java.lang.String gtFolder)
                           throws java.io.IOException

Throws:

java.io.IOException

loadGroundTruthParis

public void loadGroundTruthParis(java.lang.String gtFolder)
                          throws java.io.IOException

Throws:

java.io.IOException

loadGroundTruthHolidays

public void loadGroundTruthHolidays(java.lang.String filename)
                             throws java.io.IOException

Throws:

java.io.IOException

loadGroundTruthUKB

public void loadGroundTruthUKB(java.lang.String gtFolder)
                        throws java.io.IOException

Throws:

java.io.IOException