Improve performance of binary COPY FROM through better buffering.
Tom Lane <tgl@sss.pgh.pa.us>
Improve performance of binary COPY FROM through better buffering. At least on Linux and macOS, fread() turns out to have far higher per-call overhead than one could wish. Reading 64KB of data at a time and then parceling it out with our own memcpy logic makes binary COPY from a file significantly faster --- around 30% in simple testing for cases with narrow text columns (on Linux ... even more on macOS). In binary COPY from frontend, there's no per-call fread(), and this patch introduces an extra layer of memcpy'ing, but it still manages to eke out a small win. Apparently, the control-logic overhead in CopyGetData() is enough to be worth avoiding for small fetches. Bharath Rupireddy and Amit Langote, reviewed by Vignesh C, cosmetic tweaks by me Discussion: https://postgr.es/m/CALj2ACU5Bz06HWLwqSzNMN=Gupoj6Rcn_QVC+k070V4em9wu=A@mail.gmail.com
Files
| Path | Change | +/− |
|---|---|---|
| src/backend/commands/copy.c | modified | +83 −35 |
Discussion
- [PATCH] Performance Improvement For Copy From Binary Files 27 messages · 2020-06-29 → 2020-07-27