Leaderboard jumps of a point or two on a small test set are often within variance. Reporting mean and interval, or paired tests on the same items, is how you tell a real improvement from a lucky seed. Significance is not the same as practical importance: a tiny, real gain may not be worth the cost.