Re: .ready and .done files considered harmful

+	/*
+	 * Perform a full directory scan to identify the next log segment. There
+	 * may be one of the following scenarios which may require us to perform a
+	 * full directory scan.
+	 *
+	 * 1. This is the first cycle since archiver has started and there is no
+	 * idea about the next anticipated log segment.
+	 *
+	 * 2. There is a timeline switch, i.e. the timeline ID tracked at archiver
+	 * does not match with current timeline ID. Archive history file as part of
+	 * this timeline switch.
+	 *
+	 * 3. The next anticipated log segment is not available.

One benefit of the current implementation of pgarch_readyXlog() is
that .ready files created out of order will be prioritized before
segments with greater LSNs.  IIUC, with this patch, as long as there
is a "next anticipated" segment available, the archiver won't go back
and archive segments it missed.  I don't think the archive status
files are regularly created out of order, but XLogArchiveCheckDone()
has handling for that case, and the work to avoid creating .ready
files too early [0] seems to make it more likely.  Perhaps we should
also force a directory scan when we detect that we are creating a
.ready file for a segment that is older than the "next anticipated"
segment.

Nathan

[0] https://postgr.es/m/DA71434B-7340-4984-9B91-F085BC47A778%40amazon.com