Thread
Commits
-
Fix portability problem in Catalog.pm.
- 15bb93e28e49 10.0 landed
-
Throw an error if a DATA() line contains wrong # of attributes.
- 7666e73a2e9e 10.0 landed
-
[PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
David Christensen <david@endpoint.com> — 2017-02-15T15:40:18Z
Throws a build error if we encounter a different number of fields in a DATA() line than we expect for the catalog in question. Previously, it was possible to silently ignore any mismatches at build time which could result in symbol undefined errors at link time. Now we stop and identify the infringing line as soon as we encounter it, which greatly speeds up the debugging process. --- src/backend/catalog/Catalog.pm | 26 +++++++++++++++++++++++++- src/backend/utils/Gen_fmgrtab.pl | 18 ++++++++++++++++-- 2 files changed, 41 insertions(+), 3 deletions(-) diff --git a/src/backend/catalog/Catalog.pm b/src/backend/catalog/Catalog.pm index e1f3c3a..86f5b59 100644 --- a/src/backend/catalog/Catalog.pm +++ b/src/backend/catalog/Catalog.pm @@ -46,6 +46,9 @@ sub Catalogs open(INPUT_FILE, '<', $input_file) || die "$input_file: $!"; + my ($filename) = ($input_file =~ m/(\w+)\.h$/); + my $natts_pat = "Natts_$filename"; + # Scan the input file. while (<INPUT_FILE>) { @@ -70,8 +73,15 @@ sub Catalogs s/\s+/ /g; # Push the data into the appropriate data structure. - if (/^DATA\(insert(\s+OID\s+=\s+(\d+))?\s+\(\s*(.*)\s*\)\s*\)$/) + if (/$natts_pat\s+(\d+)/) + { + $catalog{natts} = $1; + } + elsif (/^DATA\(insert(\s+OID\s+=\s+(\d+))?\s+\(\s*(.*)\s*\)\s*\)$/) { + check_natts($filename, $catalog{natts},$3) or + die sprintf "Wrong number of Natts in DATA() line %s:%d\n", $input_file, INPUT_FILE->input_line_number; + push @{ $catalog{data} }, { oid => $2, bki_values => $3 }; } elsif (/^DESCR\(\"(.*)\"\)$/) @@ -216,4 +226,18 @@ sub RenameTempFile rename($temp_name, $final_name) || die "rename: $temp_name: $!"; } +# verify the number of fields in the passed-in bki structure +sub check_natts +{ + my ($catname, $natts, $bki_val) = @_; + unless ($natts) + { + die "Could not find definition for Natts_${catname} before start of DATA()\n"; + } + + # we're working with a copy and need to count the fields only, so collapse + $bki_val =~ s/"[^"]*?"/xxx/g; + + return (split /\s+/ => $bki_val) == $natts; +} 1; diff --git a/src/backend/utils/Gen_fmgrtab.pl b/src/backend/utils/Gen_fmgrtab.pl index cdd603a..49a5d80 100644 --- a/src/backend/utils/Gen_fmgrtab.pl +++ b/src/backend/utils/Gen_fmgrtab.pl @@ -56,9 +56,11 @@ foreach my $column (@{ $catalogs->{pg_proc}->{columns} }) } my $data = $catalogs->{pg_proc}->{data}; +my $natts = $catalogs->{pg_proc}->{natts}; +my $elem = 0; + foreach my $row (@$data) { - # To construct fmgroids.h and fmgrtab.c, we need to inspect some # of the individual data fields. Just splitting on whitespace # won't work, because some quoted fields might contain internal @@ -67,7 +69,19 @@ foreach my $row (@$data) # fields that might need quoting, so this simple hack is # sufficient. $row->{bki_values} =~ s/"[^"]*"/"xxx"/g; - @{$row}{@attnames} = split /\s+/, $row->{bki_values}; + my @bki_values = split /\s+/, $row->{bki_values}; + + # verify we've got the expected number of data fields + if (@bki_values != $natts) + { + die sprintf <<EOF, $elem, $natts, scalar @bki_values; +Wrong number of attributes in pg_proc DATA() entry %d (expected %d but got %d) +EOF + } + $elem++; + + @{$row}{@attnames} = @bki_values; + # Select out just the rows for internal-language procedures. # Note assumption here that INTERNALlanguageId is 12. -- 2.8.4 (Apple Git-73) -
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> — 2017-02-18T00:16:08Z
On 2/15/17 10:40, David Christensen wrote: > Throws a build error if we encounter a different number of fields in a > DATA() line than we expect for the catalog in question. > > Previously, it was possible to silently ignore any mismatches at build > time which could result in symbol undefined errors at link time. Now > we stop and identify the infringing line as soon as we encounter it, > which greatly speeds up the debugging process. I think this would be useful. I haven't reviewed the code, but the general idea looks reasonable. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2017-03-06T15:14:28Z
Hi David, Here's a review of your patch. David Christensen <david@endpoint.com> writes: > Throws a build error if we encounter a different number of fields in a > DATA() line than we expect for the catalog in question. The patch is a good idea, and as-is implements the suggested feature. Tested by removing an attribute from a line in catalog/pg_proc.h and observing the expected failure. With the attribute in place, it builds and passes make check-world. Detailed code review: […] > diff --git a/src/backend/catalog/Catalog.pm b/src/backend/catalog/Catalog.pm […] > + check_natts($filename, $catalog{natts},$3) or > + die sprintf "Wrong number of Natts in DATA() line %s:%d\n", $input_file,INPUT_FILE->input_line_number; Including the expected/actual number of attributes in the error message would be nice. In fact, the 'die' could be moved into check_natts(), where the actual number is already available, and would clutter the main loop less. > + unless ($natts) > + { > + die "Could not find definition for Natts_${catname} before start of DATA()\n"; > + } More idiomatically written as: die .... unless $natts; > diff --git a/src/backend/utils/Gen_fmgrtab.pl b/src/backend/utils/Gen_fmgrtab.pl The changes to this file are redundant, since it calls Catalog::Catalogs(), which already checks that the number of attributes matches. Attached is a suggested revised patch. - ilmari -- - Twitter seems more influential [than blogs] in the 'gets reported in the mainstream press' sense at least. - Matt McLeod - That'd be because the content of a tweet is easier to condense down to a mainstream media article. - Calle Dybedahl -
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
David Christensen <david@endpoint.com> — 2017-03-06T16:25:47Z
> Hi David, > > Here's a review of your patch. Hi Ilmari, thanks for your time and review. I’m fine with the revised version. Best, David -- David Christensen End Point Corporation david@endpoint.com 785-727-1171
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2017-03-06T16:37:46Z
David Christensen <david@endpoint.com> writes: >> Hi David, >> >> Here's a review of your patch. > > Hi Ilmari, thanks for your time and review. I’m fine with the revised version. Okay, I've marked the patch as Ready For Committer. Thanks, Ilmari -- "The surreality of the universe tends towards a maximum" -- Skud's Law "Never formulate a law or axiom that you're not prepared to live with the consequences of." -- Skud's Meta-Law
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Robert Haas <robertmhaas@gmail.com> — 2017-03-09T23:20:49Z
On Mon, Mar 6, 2017 at 11:37 AM, Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> wrote: > David Christensen <david@endpoint.com> writes: >>> Hi David, >>> >>> Here's a review of your patch. >> >> Hi Ilmari, thanks for your time and review. I’m fine with the revised version. > > Okay, I've marked the patch as Ready For Committer. Committed. Hopefully this doesn't contain any Perl bits that are sufficiently new as to cause problems for our older BF members ... I guess we'll see. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Jeff Janes <jeff.janes@gmail.com> — 2017-03-10T00:14:34Z
On Thu, Mar 9, 2017 at 3:20 PM, Robert Haas <robertmhaas@gmail.com> wrote: > On Mon, Mar 6, 2017 at 11:37 AM, Dagfinn Ilmari Mannsåker > <ilmari@ilmari.org> wrote: > > David Christensen <david@endpoint.com> writes: > >>> Hi David, > >>> > >>> Here's a review of your patch. > >> > >> Hi Ilmari, thanks for your time and review. I’m fine with the revised > version. > > > > Okay, I've marked the patch as Ready For Committer. > > Committed. Hopefully this doesn't contain any Perl bits that are > sufficiently new as to cause problems for our older BF members ... I > guess we'll see. > Bad luck there. I'm getting this error on CentOS6.8, perl v5.10.1 Can't locate object method "input_line_number" via package "IO::Handle" at ../../../src/backend/catalog/Catalog.pm line 82, <INPUT_FILE> line 148. make[3]: *** [fmgrtab.c] Error 25 make[2]: *** [utils/fmgroids.h] Error 2 make[2]: *** Waiting for unfinished jobs.... Can't locate object method "input_line_number" via package "IO::Handle" at ../../../src/backend/catalog/Catalog.pm line 82, <INPUT_FILE> line 148. make[3]: *** [postgres.bki] Error 25 make[2]: *** [submake-schemapg] Error 2 make[1]: *** [all-backend-recurse] Error 2 make: *** [all-src-recurse] Error 2 I think we can just save $. and use that, as in the attached. I as sabotaged a random line in src/include/catalog/pg_amop.h and it seems to report the error correctly. Cheers, Jeff
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Amit Langote <langote_amit_f8@lab.ntt.co.jp> — 2017-03-10T01:04:30Z
On 2017/03/10 9:14, Jeff Janes wrote: > On Thu, Mar 9, 2017 at 3:20 PM, Robert Haas <robertmhaas@gmail.com > <mailto:robertmhaas@gmail.com>> wrote: > > On Mon, Mar 6, 2017 at 11:37 AM, Dagfinn Ilmari Mannsåker > <ilmari@ilmari.org <mailto:ilmari@ilmari.org>> wrote: > > David Christensen <david@endpoint.com <mailto:david@endpoint.com>> writes: > >>> Hi David, > >>> > >>> Here's a review of your patch. > >> > >> Hi Ilmari, thanks for your time and review. I’m fine with the revised version. > > > > Okay, I've marked the patch as Ready For Committer. > > Committed. Hopefully this doesn't contain any Perl bits that are > sufficiently new as to cause problems for our older BF members ... I > guess we'll see. > > > > Bad luck there. I'm getting this error on CentOS6.8, perl v5.10.1 > > Can't locate object method "input_line_number" via package "IO::Handle" at > ../../../src/backend/catalog/Catalog.pm line 82, <INPUT_FILE> line 148. > make[3]: *** [fmgrtab.c] Error 25 > make[2]: *** [utils/fmgroids.h] Error 2 > make[2]: *** Waiting for unfinished jobs.... > Can't locate object method "input_line_number" via package "IO::Handle" at > ../../../src/backend/catalog/Catalog.pm line 82, <INPUT_FILE> line 148. > make[3]: *** [postgres.bki] Error 25 > make[2]: *** [submake-schemapg] Error 2 > make[1]: *** [all-backend-recurse] Error 2 > make: *** [all-src-recurse] Error 2 Same here. Some buildfarm animals failed too. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dromedary&dt=2017-03-10%2000%3A55%3A42 > I think we can just save $. and use that, as in the attached. The patch works for me. Thanks, Amit
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Tom Lane <tgl@sss.pgh.pa.us> — 2017-03-10T01:47:45Z
Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> writes: > On 2017/03/10 9:14, Jeff Janes wrote: >> I think we can just save $. and use that, as in the attached. > The patch works for me. Me too. Pushed; we'll soon see if that makes the oldest buildfarm critters happy. regards, tom lane
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Robert Haas <robertmhaas@gmail.com> — 2017-03-10T01:57:05Z
On Thu, Mar 9, 2017 at 8:47 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> writes: >> On 2017/03/10 9:14, Jeff Janes wrote: >>> I think we can just save $. and use that, as in the attached. > >> The patch works for me. > > Me too. Pushed; we'll soon see if that makes the oldest buildfarm > critters happy. $. has been around since the stone age (a.k.a. when I was a teenager) so we should be fine there. I hope. That whole input_line_number thing was making me a bit nervous, but I couldn't find any reference to when that had been added in a quick Google search, which gave me hope that it was old enough that we'd be OK. Sorry for the hassle. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line
Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2017-03-10T10:38:49Z
Jeff Janes <jeff.janes@gmail.com> writes: > On Thu, Mar 9, 2017 at 3:20 PM, Robert Haas <robertmhaas@gmail.com> wrote: > >> Committed. Hopefully this doesn't contain any Perl bits that are >> sufficiently new as to cause problems for our older BF members ... I >> guess we'll see. > > Bad luck there. I'm getting this error on CentOS6.8, perl v5.10.1 > > Can't locate object method "input_line_number" via package "IO::Handle" at [...] > I think we can just save $. and use that, as in the attached. Another option would have been to add an explicit 'use IO::Handle;', which makes the OO interface available on filehandles on all versions back to 5.8 (5.14 and newer do this automatically). - ilmari -- "I use RMS as a guide in the same way that a boat captain would use a lighthouse. It's good to know where it is, but you generally don't want to find yourself in the same spot." - Tollef Fog Heen