Horton

Re: VLDB Features

Michał Zaborowski <michal.zaborowski@gmail.com>

View thread
From: Michał Zaborowski <michal.zaborowski@gmail.com>
To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "Neil Conway" <neilc@samurai.com>, "Hannu Krosing" <hannu@skype.net>, "Josh Berkus" <josh@agliodbs.com>, pgsql-hackers@postgresql.org, "Simon Riggs" <simon@2ndquadrant.com>
Date: 2007-12-18T15:01:52Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. doc: Fix COPY ON_ERROR option syntax synopsis.

  2. Disallow specifying ON_ERROR option without value.

  3. Rename COPY option from SAVE_ERROR_TO to ON_ERROR

  4. Fix spelling in notice

  5. Add new COPY option SAVE_ERROR_TO

2007/12/16, Tom Lane <tgl@sss.pgh.pa.us>:
> Hannu Krosing <hannu@skype.net> writes:
> > But can't we _define_ such a subset, where we can do a transactionless
> > load ?
>
> Sure ... but you'll find that it's not large enough to be useful.
> Once you remove all the interesting consistency checks such as
> unique indexes and foreign keys, the COPY will tend to go through
> just fine, and then you're still stuck trying to weed out bad data
> without very good tools for it.  The only errors we could really
> separate out without subtransaction fencing are extremely trivial
> ones like too many or too few fields on a line ... which can be
> caught with a sed script.
>
I have dump file. I would like to load it ASAP.
Constraints will be applied at the end, so any problem can be detected.
I would like it to be as direct as possible and as bulk as possibe - just
allocate pages and fill them with the data. Maybe it should be different
mode - single user or so. Right now I can save some IO - like turn off
fsync, but that is all :(

I got something like that:
http://www.tbray.org/ongoing/When/200x/2007/10/30/WF-Results
I have no idea how to load single file in many threads, but... the point is
that it can be much faster that single-thread load - surprisingly - at
least for me.

-- 
Regards,
  Michał Zaborowski (TeXXaS)