Re: In-placre persistance change of a relation

Hello.

It's been a while. Based on our previous face-to-face discussions, I
have been restructuring the patch set. During this process, I found
several missing parts and issues, which led to almost everything being
rewritten. However, I believe the updates are now better organized and
more understandable.

The current patch set broadly consists of the following elements:

- Core feature: Switching buffer persistence (0007) remains mostly the
  same as before, but the creation and deletion of INIT fork files
  have undergone significant modifications. Part of this functionality
  has been moved to commit records.

- UNDO log(0002): This handles file deletion during transaction aborts,
  which was previously managed, in part, by the commit XLOG record at
  the end of a transaction.

- Prevent orphan files after a crash (0005): This is another use-case
  of the UNDO log system.

- Extension of smgr (0012), pendingDeletes (0014), and commit XLOG
  records (0013): These have been extended to handle file deletion at
  the fork level instead of the relfilenumber level. While this
  extension applies to both commit and abort operations, only the file
  deletion process for aborts has been moved to the UNDO log. As a
  result, file deletions during commits continue to be managed by
  commit records.

Here are some issues. Depending on how these points are addressed,
this patch set might be dropped. (Or, this patch might already be too
large for its intended effect.)

- Consecutive changes to the persistence of the same table within a
  single transaction are prohibited (0007). Allowing this would
  complicate pendingDeletes and a similar mechanism added to
  bufmgr. Also, due to the append-only nature of the UNDO log, the
  entire process, including subtransaction handling, could not be made
  consistent easily.

- PREPARE is prohibited for transactions that have altered table
  persistence(0009). This is because I haven't found a simple way to
  ensure consistent switching of buffer persistence if the server
  crashes after PREPARE and then commits the transaction after
  recovery.

- Data updates within a single transaction after changing the table's
  persistence are also prohibited(0008). This restriction is necessary
  because if an index update triggers page splits after changing the
  persistence to UNLOGGED, WAL might become inapplicable.

The last point, in particular, has a significant impact on usability,
but it seems to be fundamentally unavoidable. Since heap updates
appear to be fine, one possible approach could be to give up on
in-place persistence changes for indexes.

Regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center