keypoints at the border

[hit2sjtu]

When I increase the number of detections, I find many keypoints start to emerge at the border of the images. Look at the keypoints on the top of below image. Why is it? Image padding?

[Parskatt]

That commonly happens with keypoint detectors (there will basically always be local max near borders).
You can disable borders by setting
detector.remove_borders = True
For all settings see here:

dad/dad/detectors/dedode_detector.py

Lines 14 to 28 in 90103c8

	class DeDoDeDetector(Detector):
	def __init__(
	self,
	*args,
	encoder: nn.Module,
	decoder: nn.Module,
	resize: int,
	nms_size: int,
	subpixel: bool,
	subpixel_temp: float,
	keep_aspect_ratio: bool,
	remove_borders: bool,
	increase_coverage: bool,
	coverage_pow: float,
	coverage_size: int,

[Parskatt]

I might update to remove borders by default, or make these settings a bit more obvious.
However, paper results were gotten by it disabled, so need to rerun some tests to see effect.

[Parskatt]

@hit2sjtu could you tell me the exact name of this image? I assume it's from mega1500

[hit2sjtu]

@hit2sjtu could you tell me the exact name of this image? I assume it's from mega1500

Yes. It is from megadepth1500. When I set the number of keypoints to 2048, I can see it happens quite often.
And I have tried remove border options. It indeed can remove most keypoints near the border. But not all. For example,the below image

[Parskatt]

Ah interesting, didn't expect that. I'll check that pair out.

[hit2sjtu]

For the initial pairs I posted, turing on the remove-border option works fine.

[hit2sjtu]

I might update to remove borders by default, or make these settings a bit more obvious. However, paper results were gotten by it disabled, so need to rerun some tests to see effect.

For your reference, my limited tests show no much difference with remove border or not on megadepth1500. I am just annoyed by the regular pattern during visualization :(

[Parskatt]

@hit2sjtu I'm not sure I am able to reproduce your results. Running our provided qualitative script on the dark image, with 8192 keypoints and removing border gives me this result:

[Parskatt]

How are you loading the images exactly, are you loading them differently than the load_image method that we use?

[Parskatt]

But I do see some of the artifacts far away from the border:

[Parskatt]

I would conjecture this might have something to do with us using 32 pixel shifts during training due to the megadepth ground truth being bad.

[Parskatt]

For the other image.
Also see traces of the 32 pixel stuff.

Are you specifically interested in large numbers of keypoints? I could retrain models specifically for this purpose.

[Parskatt]

I ran tests and got in general a slight boost in performance, will change default to remove borders.