The song played during the cricket match is "Victory Test Match" by Lord Beginner, celebrating the 1950 victory of West Indies at Lord's.
When Father Brown and Professor Milton are sitting on the bench talking about death and atheism, Father Brown says: "My heart is poured out like water. My bones are scattered. And my heart, like wax, is melted." He is quoting a verse from the Bible, Psalms 22:14.
The bench on which Father Brown is sitting in the last scene has a plaque that reads "Cor Cordium". In Latin, these words mean "Heart of Hearts". "Cor Cordium" is the phrase inscribed on the tomb of the poet Percy Bysshe Shelley (1792-1822), and also the title of a famous poem by Algernon Swinburne (1869). Father Brown most likely chose this inscription in reference to the "dodgy ticker" of Professor Jane Milton.
More than once someone is described as "a gentleman and a player", which could be considered paradoxical. Until 1962 there was a series of matches between "Players", working-class men who were paid to play cricket, and "Gentlemen", elite amateurs.