On Mr. Webber's coat there are buttons. The Amish don't use buttons. They are not considered plain.
Mr. Webber refutes any mode of transportation that's not in the Bible, but says he has no money for train fare.
Webber states that he came to New York to see about a "hay critter" (presumably a cow). There is no place in Manhattan to purchase livestock, and he logically could obtain one from a source more local to his farm.