Phd-Evaluation

Towards Efficient 3D Reconstruction from UAV Imagery: Evaluation of OpenCV Feature Detection, Description and Matching Combinations

Author: Abbas Elmas
PhD Thesis - Computer Vision and Photogrammetry
Supported by: TÜBİTAK BİDEB 2211-A National PhD Scholarship Program

This repository contains experimental framework, datasets, evaluation outputs, and interactive visualizations for comprehensive evaluation of OpenCV feature detection, description, and matching algorithm combinations for UAV-based 3D reconstruction.

Research Summary

• 784 algorithm combinations evaluated across 5 diverse datasets
• Composite Unsupervised Efficiency Score (CUES) - Entropy + CRITIC + PCA + Variance
• 7-Phase comprehensive analysis: Dataset-specific, Algorithm-specific, Component-based, Cross-dataset, Unified, Mobile-optimized, Sensitivity
• Cross-dataset stability: 0.805 average correlation
• Cross-platform consistency: ≥0.994 correlation for UAV/AirSim/Oxford/Synthetic; Drone dataset shows platform-specific ranking shifts (r = 0.219 Desktop vs Mobile)
• Best overall performer: ORB-ORB-HAM-BF (0.8538 synthetic)
• Best unified performer: ORB-BEBLID-HAM-BF (0.7517 mean, cross-dataset champion)
• Best UAV: ORB-BEBLID-HAM-BF (0.7249)
• Most stable high performer: AKAZE-SIFT-L2-BF (CoV: 0.0801)
• Sensitivity: α parameter stability avg ρ=0.9845, weight aggregation stability avg ρ=0.9993

Interactive Dashboards

Access comprehensive analysis dashboards:

Dataset	Main	Efficiency	Timing	Correlation	Heatmap	Violin
Synthetic	Analysis	Efficiency	Timing	Correlation	Heatmap	Violin
Oxford	Analysis	Efficiency	Timing	Correlation	Heatmap	Violin
AirSim	Analysis	Efficiency	Timing	Correlation	Heatmap	Violin
UAV	Analysis	Efficiency	Timing	Correlation	Heatmap	Violin
Drone	Analysis	Efficiency	Timing	Correlation	Heatmap	Violin

Detailed dashboards with variants and mobile analyses

Dataset	Main Page	Efficiency	Variants	Timing	Correlation	Heatmap	Violin
Synthetic	synthetic	Efficiency	4, All4	Timing	Correlation	Heatmap	Violin
Synthetic	Mobile 1	Efficiency m1	4 m1, All4 m1	Timing m1	Correlation m1	Heatmap m1	Violin m1
Synthetic	Mobile 2	Efficiency m2	4 m2, All4 m2	Timing m2	Correlation m2	Heatmap m2	Violin m2
Oxford	oxford	Efficiency	9, All9, All	Timing	Correlation	Heatmap	Violin
Oxford	Mobile 1	Efficiency m1	9 m1, All9 m1, All m1	Timing m1	Correlation m1	Heatmap m1	Violin m1
Oxford	Mobile 2	Efficiency m2	9 m2, All9 m2, All m2	Timing m2	Correlation m2	Heatmap m2	Violin m2
Drone	drone	Efficiency	All, AllXY	Timing	Correlation	Heatmap	Violin
UAV	uav	Efficiency	All	Timing	Correlation	Heatmap	Violin
AirSim	airsim	Efficiency	All	Timing	Correlation	Heatmap	Violin

Feature Matching Visualizations

Interactive JPG and HTML visualizations for top-performing algorithm combinations across 784 evaluated configurations. Each visualization includes match quality metrics, timing statistics, and toggleable information overlays.

Top 5 Algorithm Combinations per Dataset

Dataset	Rank	Algorithm Combination	CUES	Static	Interactive
Synthetic	1	ORB-ORB-HAM-BF	0.8538	JPG	HTML
	2	ORB-ORB-HAM-FLANN	0.8335	JPG	HTML
	3	SIFT-BEBLID-HAM-BF	0.8229	JPG	HTML
	4	SIFT-BOOST-HAM-BF	0.8190	JPG	HTML
	5	ORB-TEBLID-HAM-BF	0.8188	JPG	HTML
Oxford	1	ORB-BEBLID-HAM-BF	0.6264	JPG	HTML
	2	ORB-TEBLID-HAM-BF	0.6202	JPG	HTML
	3	ORB-ORB-HAM-BF	0.6038	JPG	HTML
	4	ORB-TEBLID-HAM-FLANN	0.5902	JPG	HTML
	5	ORB-BEBLID-HAM-FLANN	0.5868	JPG	HTML
AirSim	1	ORB-BEBLID-HAM-BF	0.8064	JPG	HTML
	2	ORB-TEBLID-HAM-BF	0.7968	JPG	HTML
	3	MSD-BEBLID-HAM-BF	0.7547	JPG	HTML
	4	KAZE-BEBLID-HAM-BF	0.7528	JPG	HTML
	5	ORB-BEBLID-HAM-FLANN	0.7458	JPG	HTML
UAV	1	ORB-BEBLID-HAM-BF	0.7249	JPG	HTML
	2	AGAST-BEBLID-HAM-BF	0.7212	JPG	HTML
	3	ORB-TEBLID-HAM-BF	0.7123	JPG	HTML
	4	AGAST-TEBLID-HAM-BF	0.7118	JPG	HTML
	5	AKAZE-DAISY-L2-FLANN	0.6949	JPG	HTML
Drone	1	ORB-ORB-HAM-BF	0.8028	JPG	HTML
	2	ORB-BEBLID-HAM-BF	0.8008	JPG	HTML
	3	ORB-TEBLID-HAM-BF	0.7786	JPG	HTML
	4	ORB-ORB-HAM-FLANN	0.7611	JPG	HTML
	5	ORB-DAISY-L2-FLANN	0.7600	JPG	HTML

Drone Dataset Visualizations

Algorithm Combination	CUES	Static	Interactive
ORB-ORB-HAM-BF	0.8028	JPG	HTML
ORB-BEBLID-HAM-BF	0.8008	JPG	HTML
ORB-TEBLID-HAM-BF	0.7786	JPG	HTML
ORB-ORB-HAM-FLANN	0.7611	JPG	HTML
ORB-DAISY-L2-FLANN	0.7600	JPG	HTML
ORB-BEBLID-HAM-FLANN	0.7511	JPG	HTML
ORB-DAISY-L2-BF	0.7494	JPG	HTML
GFTT-DAISY-L2-FLANN	0.7473	JPG	HTML
ORB-TEBLID-HAM-FLANN	0.7403	JPG	HTML
GFTT-DAISY-L2-BF	0.7363	JPG	HTML

UAV Dataset Visualizations

Algorithm Combination	CUES	Static	Interactive
ORB-BEBLID-HAM-BF	0.7249	JPG	HTML
AGAST-BEBLID-HAM-BF	0.7212	JPG	HTML
ORB-TEBLID-HAM-BF	0.7123	JPG	HTML
AGAST-TEBLID-HAM-BF	0.7118	JPG	HTML
AKAZE-DAISY-L2-FLANN	0.6949	JPG	HTML
AGAST-VGG-L2-BF	0.6917	JPG	HTML
AGAST-SIFT-L2-BF	0.6910	JPG	HTML
MSD-BEBLID-HAM-BF	0.6905	JPG	HTML
FAST-BEBLID-HAM-BF	0.6885	JPG	HTML
GFTT-DAISY-L2-BF	0.6880	JPG	HTML

AirSim Dataset Visualizations

Algorithm Combination	CUES	Static	Interactive
ORB-BEBLID-HAM-BF	0.8064	JPG	HTML
ORB-TEBLID-HAM-BF	0.7968	JPG	HTML
MSD-BEBLID-HAM-BF	0.7547	JPG	HTML
KAZE-BEBLID-HAM-BF	0.7528	JPG	HTML
ORB-BEBLID-HAM-FLANN	0.7458	JPG	HTML
MSD-TEBLID-HAM-BF	0.7451	JPG	HTML
ORB-TEBLID-HAM-FLANN	0.7450	JPG	HTML
KAZE-TEBLID-HAM-BF	0.7399	JPG	HTML
KAZE-SIFT-L2-FLANN	0.7375	JPG	HTML
KAZE-SIFT-L2-BF	0.7343	JPG	HTML

Synthetic Dataset Visualizations (Rotation)

Algorithm Combination	CUES	Static	Interactive
ORB-ORB-HAM-BF	0.8538	JPG	HTML
ORB-ORB-HAM-FLANN	0.8335	JPG	HTML
SIFT-BEBLID-HAM-BF	0.8229	JPG	HTML
SIFT-BOOST-HAM-BF	0.8190	JPG	HTML
ORB-TEBLID-HAM-BF	0.8188	JPG	HTML
SIFT-TEBLID-HAM-BF	0.8186	JPG	HTML
ORB-BOOST-HAM-BF	0.8157	JPG	HTML
STAR-BRISK-HAM-BF	0.8096	JPG	HTML
ORB-BEBLID-HAM-BF	0.8002	JPG	HTML
STAR-BRISK-HAM-FLANN	0.7973	JPG	HTML

Oxford Affine Dataset Visualizations

Representative visualizations for top Oxford performers across 8 scenarios (bark, bikes, boat, graf, leuven, trees, ubc, wall).

Bark (Viewpoint Change)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML
AKAZE-AKAZE-HAM-BF	JPG	HTML

Bikes (Blur)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AKAZE-AKAZE-HAM-BF	JPG	HTML

Boat (Zoom + Rotation)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML

Graf (Viewpoint Change)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML

Leuven (Illumination)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML

Trees (Blur)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML

UBC (JPEG Compression)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML

Wall (Viewpoint Change)

Algorithm Combination	Static	Interactive
ORB-BEBLID-HAM-BF	JPG	HTML
ORB-TEBLID-HAM-BF	JPG	HTML
ORB-ORB-HAM-BF	JPG	HTML
AKAZE-SIFT-L2-BF	JPG	HTML
AGAST-BEBLID-HAM-BF	JPG	HTML

Cross-Dataset Top Performers

Algorithm Combination	Drone	UAV	AirSim	Oxford	Synthetic	Average
ORB-BEBLID-HAM-BF	0.801	0.725	0.806	0.626	0.800	0.752
ORB-TEBLID-HAM-BF	0.779	0.712	0.797	0.620	0.819	0.745
ORB-ORB-HAM-BF	0.803	0.673	0.637	0.604	0.854	0.714
ORB-BEBLID-HAM-FLANN	0.751	0.670	0.746	0.587	0.776	0.706
AGAST-BEBLID-HAM-BF	0.713	0.721	0.708	0.576	0.719	0.687
MSD-BEBLID-HAM-BF	0.660	0.691	0.755	0.571	0.742	0.684

Unified Cross-Dataset Algorithm Ranking (Top 15)

Rank	Algorithm	Mean	StdDev	Min	Max
1	ORB-BEBLID-HAM-BF	0.7517	0.0695	0.6264	0.8064
2	ORB-TEBLID-HAM-BF	0.7453	0.0720	0.6202	0.8188
3	ORB-ORB-HAM-BF	0.7141	0.0971	0.6038	0.8538
4	ORB-BEBLID-HAM-FLANN	0.7059	0.0692	0.5868	0.7756
5	ORB-TEBLID-HAM-FLANN	0.6990	0.0626	0.5902	0.7529
6	AGAST-BEBLID-HAM-BF	0.6874	0.0557	0.5763	0.7212
7	MSD-BEBLID-HAM-BF	0.6837	0.0660	0.5711	0.7547
8	AGAST-TEBLID-HAM-BF	0.6801	0.0557	0.5697	0.7179
9	MSD-TEBLID-HAM-BF	0.6777	0.0658	0.5649	0.7451
10	ORB-ORB-HAM-FLANN	0.6768	0.1026	0.5702	0.8335
11	AKAZE-BEBLID-HAM-BF	0.6694	0.0593	0.5734	0.7603
12	ORB-BEBLID-L2-BF	0.6658	0.0790	0.5581	0.7895
13	AKAZE-TEBLID-HAM-BF	0.6642	0.0603	0.5694	0.7599
14	ORB-BOOST-HAM-BF	0.6637	0.0998	0.5288	0.8157
15	AKAZE-SIFT-L2-BF	0.6633	0.0531	0.5792	0.7457

Feature Detector Performance Rankings

Rank	Detector	Mean CUES	Max CUES	Evaluations
1	AKAZE	0.5455	0.7621	175
2	ORB	0.5405	0.8538	155
3	STAR	0.5304	0.8096	153
4	AGAST	0.4974	0.7222	153
5	BRISK	0.4796	0.7344	153
6	KAZE	0.4722	0.7528	176
7	FAST	0.4631	0.7199	152
8	SIFT	0.4614	0.8229	138
9	MSD	0.4449	0.7547	141
10	GFTT	0.4318	0.7473	151

Feature Descriptor Performance Rankings

Rank	Descriptor	Mean CUES	Max CUES	Evaluations
1	SIFT	0.6147	0.7937	126
2	DAISY	0.5972	0.7600	137
3	VGG	0.5951	0.7915	136
4	BEBLID	0.5881	0.8229	204
5	AKAZE	0.5767	0.7568	30
6	TEBLID	0.5658	0.8188	203
7	BOOST	0.5464	0.8190	204
8	KAZE	0.5190	0.6762	13
9	BRISK	0.4009	0.8096	186
10	ORB	0.3670	0.8538	171

Optimal Detector-Descriptor Combinations (Top 10)

Rank	Combination	Mean CUES	Max CUES
1	ORB-BEBLID	0.7078	0.8064
2	AKAZE-SIFT	0.6625	0.7560
3	STAR-SIFT	0.6513	0.7348
4	AKAZE-VGG	0.6511	0.7575
5	AGAST-SIFT	0.6510	0.6984
6	AGAST-VGG	0.6457	0.7222
7	AKAZE-AKAZE	0.6401	0.7568
8	AKAZE-DAISY	0.6396	0.7059
9	STAR-VGG	0.6390	0.7414
10	ORB-DAISY	0.6343	0.7600

Sensitivity Analysis

The CUES parameter choices are validated through systematic sensitivity analysis across all 5 datasets:

Normalization Parameter α (Eq. 1-2):

• Default α = 0.5 (square root transformation)
• Stable across adjacent values: avg ρ = 0.9845
• Lowest correlation: synthetic ρ = 0.9518 (α=0.25 vs 0.5)
• Highest correlation: airsim ρ = 0.9972 (α=0.25 vs 0.5)

Weight Aggregation Method (Eq. 11):

• Default: arithmetic mean
• Highly stable across arithmetic, geometric, harmonic, and trimmed mean: avg ρ = 0.9993
• All weighting methods (Entropy, PCA, CRITIC, Variance) produce consistent rankings

Most Consistent High Performers (Lowest Coefficient of Variation):

Rank	Algorithm	CoV	Mean	StdDev
1	AKAZE-SIFT-L2-BF	0.0801	0.6633	0.0531
2	AGAST-SIFT-L2-BF	0.0802	0.6579	0.0528
3	AGAST-BEBLID-HAM-BF	0.0811	0.6874	0.0557
4	BRISK-TEBLID-HAM-BF	0.0815	0.6470	0.0527
5	AGAST-TEBLID-HAM-BF	0.0819	0.6801	0.0557