Re: Best COPY Performance
Michael Artz <mlartz@gmail.com>
From: "Michael Artz" <mlartz@gmail.com>
To: "Luke Lonergan" <llonergan@greenplum.com>
Cc: "Worky Workerson" <worky.workerson@gmail.com>, "Merlin Moncure" <mmoncure@gmail.com>, pgsql-performance@postgresql.org
Date: 2006-10-28T12:03:10Z
Lists: pgsql-performance
> > Are you saying that I should be able to issue multiple COPY commands > > because my I/O wait is low? I was under the impression that I am I/O > > bound, so multiple simeoultaneous loads would have a detrimental > > effect ... > > The reason I asked how many CPUs was to make sense of the 12% usr CPU time > in the above. That means you are CPU bound and are fully using one CPU. So > you aren't being limited by the I/O in this case, it's the CPU. > > I agree with Merlin that you can speed things up by breaking the file up. > Alternately you can use the OSS Bizgres java loader, which lets you specify > the number of I/O threads with the "-n" option on a single file. Thanks, I'll try that on Monday. > The other thing to wonder about though is why you are so CPU bound at 5 > MB/s. What version of Postgres is this? I was wondering about that as well, and the only thing that I can think of is that its the PK btree index creation on the IP4. PG 8.1.3 x86_64. I installed it via a RH rpm for their "Web Services Beta", or something like that. I know I'm a bit behind the times, but getting stuff in (and out) of my isolated lab is a bit of a pain. I'll compile up a 8.2 beta as well and see how that works out.