Thread

Debugging initdb breakage

Dimitri Fontaine <dimitri@2ndquadrant.fr> — 2010-10-10T20:38:01Z

Hi,

So I'm beginning to work on the extension support for dump and restore,
and that begins with a new pg_extension catalog. I managed to break
initdb already, of course, but I'm fighting my way out — no luck with
gdb, it won't catch the Assert failure and show me a backtrace. I tried
"set follow-fork-mode child" in gdb, in different systems, to no avail.

Please find attached the detailed initdb.log and the very few items I
needed to obtain:

TRAP: FailedAssertion("!(reln->md_fd[forkNum] == ((void *)0))", File: "md.c", Line: 254)
child process was terminated by signal 6: Abort trap

How to have gdb help me? What's my error, that I guess is obvious? Where
do I read more now in order not to need too much assistance after that?

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

Re: Debugging initdb breakage

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2010-10-10T21:24:17Z

On 10.10.2010 23:38, Dimitri Fontaine wrote:
> So I'm beginning to work on the extension support for dump and restore,
> and that begins with a new pg_extension catalog. I managed to break
> initdb already, of course, but I'm fighting my way out — no luck with
> gdb, it won't catch the Assert failure and show me a backtrace. I tried
> "set follow-fork-mode child" in gdb, in different systems, to no avail.
>
> Please find attached the detailed initdb.log and the very few items I
> needed to obtain:
>
> TRAP: FailedAssertion("!(reln->md_fd[forkNum] == ((void *)0))", File: "md.c", Line: 254)
> child process was terminated by signal 6: Abort trap
>
> How to have gdb help me? What's my error, that I guess is obvious? Where
> do I read more now in order not to need too much assistance after that?

At least on my system, assertion failure creates a core dump that you 
can load in gdb. Make sure you use "ulimit -c unlimited" or similar.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

Re: Debugging initdb breakage

Tom Lane <tgl@sss.pgh.pa.us> — 2010-10-10T21:33:51Z

Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes:
> How to have gdb help me? What's my error, that I guess is obvious?

Might have something to do with using the same OID for the catalog
and its index ...

			regards, tom lane

Re: Debugging initdb breakage

Dimitri Fontaine <dimitri@2ndquadrant.fr> — 2010-10-11T07:20:15Z

Tom Lane <tgl@sss.pgh.pa.us> writes:
> Might have something to do with using the same OID for the catalog
> and its index ...

Ahah, that's how obvious it was, thank you. I feel ashamed, but at the
same time, thanks to your answer, it now feels like the week-end was a
good preparatory step for opening this week.

Back to producing cores (ulimit -c unlimited),
-- 
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

Re: Debugging initdb breakage

Alvaro Herrera <alvherre@commandprompt.com> — 2010-10-12T14:51:58Z

Excerpts from Dimitri Fontaine's message of dom oct 10 17:38:01 -0300 2010:
> Hi,
> 
> So I'm beginning to work on the extension support for dump and restore,
> and that begins with a new pg_extension catalog. I managed to break
> initdb already, of course, but I'm fighting my way out — no luck with
> gdb, it won't catch the Assert failure and show me a backtrace. I tried
> "set follow-fork-mode child" in gdb, in different systems, to no avail.

As a note, I've had luck finding bootstrap-time bugs by manually feeding
the bootstrapping commands into bootstrap mode, with a leftover dir from
"initdb --noclean".  This has helped a few times that there has been no
PANIC but just a FATAL or ERROR, for example.  It's easy to attach GDB
to such a backend.

-- 
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Re: Debugging initdb breakage

Dimitri Fontaine <dimitri@2ndquadrant.fr> — 2010-10-12T20:05:57Z

Alvaro Herrera <alvherre@commandprompt.com> writes:
> As a note, I've had luck finding bootstrap-time bugs by manually feeding
> the bootstrapping commands into bootstrap mode, with a leftover dir from
> "initdb --noclean".  This has helped a few times that there has been no
> PANIC but just a FATAL or ERROR, for example.  It's easy to attach GDB
> to such a backend.

Nice tip, thanks!
-- 
Dimitri Fontaine
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

Re: Debugging initdb breakage

David Fetter <david@fetter.org> — 2010-10-12T20:35:44Z

On Tue, Oct 12, 2010 at 10:05:57PM +0200, Dimitri Fontaine wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > As a note, I've had luck finding bootstrap-time bugs by manually
> > feeding the bootstrapping commands into bootstrap mode, with a
> > leftover dir from "initdb --noclean".  This has helped a few times
> > that there has been no PANIC but just a FATAL or ERROR, for
> > example.  It's easy to attach GDB to such a backend.
> 
> Nice tip, thanks!

Where should we preserve this, other than the mailing list archives?

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Debugging initdb breakage

Greg Smith <greg@2ndquadrant.com> — 2010-10-18T02:15:36Z

David Fetter wrote:
> Where should we preserve this, other than the mailing list archives?
>   

http://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD 
has most of the other trivia in this area, so I just added Alvaro's 
technique to the bottom of it with a quick intro to add some context.

-- 
Greg Smith, 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services and Support  www.2ndQuadrant.us