Olaf calls Nero, "Caligula." Caligula and Nero were both notoriously terrible Roman emperors.
The Incomplete History of Secret Organizations contains a very complicated riddle about how V.F.D. Agents take their tea.
The mascot of Prufrock Preparatory School is a dead horse. At the pep rally the students chant, "You can't beat a dead horse." This is a reference to the common idiom "beating a dead horse," which means to continue a particular endeavour that is a waste of time, as the outcome is already decided - much like the Baudelaires' fight against Count Olaf.
At 39 minutes, Olaf calls Poe "Hat Stevens". Cat Stevens is a folk singer who had five top-ten albums between 1967 and 1974.
The song the Snicket Brothers whistle in the car is That's Not How the Story Goes from Season 1 and the unofficial theme song for the show amongst fans.