Re: logical decoding : exceeded maxAllocatedDescs for .spill files

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>

To: Amit Khandekar <amitdkhan.pg@gmail.com>

Cc: Alvaro Herrera from 2ndQuadrant <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Robert Haas <robertmhaas@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Tomas Vondra <tomas.vondra@2ndquadrant.com>

Date: 2020-01-03T02:59:00Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

When a TAP file has non-zero exit status, retain temporary directories.
- 048c7ccd7d6d 9.6.17 landed
- d8efc5900f7c 10.12 landed
- 887657d183fc 11.7 landed
- 78a26c3edd85 12.2 landed
- bf989aaf3561 13.0 landed
Fix running out of file descriptors for spill files.
- 1ad47e8757bb 9.4.26 landed
- a6f4f407ada0 9.5.21 landed
- 27b5f48c79f7 10.12 landed
- 3e3a79735235 11.7 landed
- f8a6d8e71b17 12.2 landed
- d20703805383 13.0 landed
- ba5b4e506489 9.6.17 landed
Track statistics for spilling of changes from ReorderBuffer.
- 9290ad198b15 13.0 cited
Handle ReadFile() EOF correctly on Windows.
- 2189f49c420f 12.2 landed
- 6969deeb8d39 13.0 landed
Add logical_decoding_work_mem to limit ReorderBuffer memory usage.
- cec2edfa7859 13.0 cited
Generational memory allocator
- a4ccc1cef5a0 11.0 cited
Support retaining data dirs on successful TAP tests
- 90627cf98a8e 11.0 cited

On Thu, Jan 2, 2020 at 5:44 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

> On Tue, Dec 24, 2019 at 2:31 PM Amit Khandekar <amitdkhan.pg@gmail.com>
> wrote:
> >
> >
> > Ok. I tested pg94_95_use_vfd_for_logrep.patch for 9.6 branch, and it
> > works there. So please use this patch for all the three branches.
> >
>
> Pushed!
>

I see one failure in REL_10_STABLE [1] which seems to be due to this commit:

Test Summary Report
-------------------
t/006_logical_decoding.pl          (Wstat: 7424 Tests: 10 Failed: 0)
  Non-zero exit status: 29
  Parse errors: Bad plan.  You planned 11 tests but ran 10.
Files=14, Tests=122, 1968 wallclock secs ( 0.10 usr  0.03 sys + 19.00 cusr
21.98 csys = 41.11 CPU)
Result: FAIL
Makefile:19: recipe for target 'check' failed
make: *** [check] Error 1

See below snippet from 006_logical_decoding_master.log
..
..
2020-01-03 01:30:48.254 UTC [12189836:9] t/006_logical_decoding.pl
STATEMENT:  SELECT data from pg_logical_slot_get_changes('test_slot', NULL,
NULL)
   WHERE data LIKE '%INSERT%' ORDER BY lsn LIMIT 1;
2020-01-03 01:30:51.990 UTC [6882174:3] LOG:  server process (PID 12189836)
was terminated by signal 11
2020-01-03 01:30:51.990 UTC [6882174:4] DETAIL:  Failed process was
running: SELECT data from pg_logical_slot_get_changes('test_slot', NULL,
NULL)
   WHERE data LIKE '%INSERT%' ORDER BY lsn LIMIT 1;
2020-01-03 01:30:51.990 UTC [6882174:5] LOG:  terminating any other active
server processes

The strange thing is that the same test passes on master on the same
machine [2] and for 10 as well, it passes on other machines, so not sure
what could cause this.  Any clue?

[1] -
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2020-01-02%2023%3A36%3A31
[2] -
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2020-01-02%2010%3A37%3A33

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com