Re: Importing Large Amounts of Data

Gavin Sherry <swm@linuxworld.com.au>

From: Gavin Sherry <swm@linuxworld.com.au>
To: Curt Sampson <cjs@cynic.net>
Cc: pgsql-hackers@postgresql.org
Date: 2002-04-16T05:17:25Z
Lists: pgsql-hackers
On Tue, 16 Apr 2002, Curt Sampson wrote:

> > Given the very low parsing and 'planning' overhead, the real cost would be
> > WAL (the bootstrapper could fail and render the database unusable) and the
> > subsequent updating of on-disk relations.
> 
> MS SQL Server, when doing a BULK INSERT or BCP, can do it as a fully or
> "minimally" logged operation. When minimally logged, there's no ability
> to roll-forward or recover inserted data, just the ability to go back
> to the state at the beginning of the operation. This technique can work
> even though an on-line database. A bit more information is available at

The other reason I say that this bootstrap tool would still use WAL is
that bypassing WAL would require writing a fairly significant amount of
code (unless the pre-WAL heap_insert() code could be used, with relevant
modification).

On the other hand, I would imagine it to be very difficult to implement
an 'interactive' roll back facility with the kind of tool I am
describing.

Gavin