Horton

Thread

  1. Re: Var->sublevels_up

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-01-20T06:55:43Z

    > 
> I can't figure out how to properly do the the Var->sublevels_up search in the
> existing code, partly because I can't figure out what it is doing.
> 
> It will take me a few days more to do it.  I just can't look at the code
> anymore.

OK, I think I have a handle on it now.  Should be two days.

-- 
Bruce Momjian
maillist@candle.pha.pa.us

  2. Re: Var->sublevels_up

    Vadim B. Mikheev <vadim@sable.krasnoyarsk.su> — 1998-01-20T07:11:43Z

    Bruce Momjian wrote:
> 
> >
> > I can't figure out how to properly do the the Var->sublevels_up search in the
> > existing code, partly because I can't figure out what it is doing.
                                                     ^^^^^^^^^^^^^^^^
It says what range table should be used to find Var' relation.

> >
> > It will take me a few days more to do it.  I just can't look at the code
> > anymore.
> 
> OK, I think I have a handle on it now.  Should be two days.

Ok.

Vadim

  3. Re: [HACKERS] Re: Var->sublevels_up

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-01-20T14:42:12Z

    > 
> Bruce Momjian wrote:
> > 
> > >
> > > I can't figure out how to properly do the the Var->sublevels_up search in the
> > > existing code, partly because I can't figure out what it is doing.
>                                                      ^^^^^^^^^^^^^^^^
> It says what range table should be used to find Var' relation.

My problem was the ParseFunc really does much more than handle
functions.  It handles:

	colname
	rel.colname
	rel.colname.func
	sum(colname)
	sum(rel.colname)
	
and probably much more than that.

> 
> > >
> > > It will take me a few days more to do it.  I just can't look at the code
> > > anymore.
> > 
> > OK, I think I have a handle on it now.  Should be two days.
> 
> Ok.
> 
> Vadim
> 
> 


-- 
Bruce Momjian
maillist@candle.pha.pa.us

  4. Re: atttypmod is wrong in system catalogs

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-02-05T19:00:53Z

    > 
> Can someone figure out why atttypmod for system relations is not always
> zero?  Sometimes it is -1 and other times it is 1, and I can't figure
> out why.

I found the problem.  You will have to initdb to see the fix.

-- 
Bruce Momjian
maillist@candle.pha.pa.us

  5. Re: [HACKERS] PostGreSQL v6.2.1 for Linux Alpha

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-02-12T14:39:50Z

    > 
> > > CREATED relation pg_description with OID 17847
> > > Commit End
> > Amopen: relation pg_description. attrsize 63
> > create attribute 0 name objoid len 4 num 1 type 26
> > create attribute 1 name description len -1 num 2 type 25
> > > Amclose: relation (null).
> > > initdb: could not create template database
> > initdb: cleaning up by wiping out /usr/local/pgsql/data/base/template1
> > 
> > Installing the "#define long int" gives about 40 pages of errors.

I saw this in hsearch.h:
	
	typedef struct element
	{
	    unsigned long next;         /* secret from user      */
	    long        key;
	} ELEMENT;
	    
	typedef unsigned long BUCKET_INDEX;

I wonder is this is the problem.  Should then be ints.  It would be
great if you could read the hash value going in for mkoidname function,
and then see if the same key is being used for the lookup.  Perhaps some
elog(NOTICE) lines near the hash function would tell you this.  


-- 
Bruce Momjian
maillist@candle.pha.pa.us

  6. Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables?

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-13T14:49:01Z

    > Yes, the threading topic has come up before, and I have never considered
> it a big win.  We want to remove the exec() from the startup, so we just
> do a fork.  Will save 0.001 seconds of startup.
> 
> That is a very easy win for us.  I hadn't considered the synchonization
> problems with palloc/pfree, and that could be a real problem.

I was wrong here.  Removing exec() will save 0.01 seconds, not 0.001
seconds.  Typical backend startup and a single query is 0.08 seconds. 
Removal of exec() will take this down to 0.07 seconds.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  7. Re: indexing words slow

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-14T06:27:33Z

    > OK, I have the artist_fti table with 4.5 million rows, with an index
> artist_fti_idx on artist_fti.string.  I don't have a 'product' table
> because I don't have the disk space, but that is not really a problem
> for testing.  
> 
> I used the product table to populate the artits_fti, then deleted the
> table so I could create the artist_fti_idx index.  Single table, no
> join.
> 
> I am running on BSD/OS 3.1 PP200 with SCSI Ultra Barracuda disks.
> 
> I am seeing the same thing you are.  'lling' and 'tones' take 11-22
> seconds on the first few runs, then return almost immediately.  If I do
> another query and come back to the first one, I have the same slowness,
> with the disk light being on almost constantly.  Certain queries seem to
> lend themselves to speeding up, while 'rol' never seems to get really
> fast.
> 
> I have to conclude that because of the way this table is created by
> slicing words, its layout is almost random.  The table is 272MB, and
> with 8k/page, that is 34,000 pages.  If we are going for 2,500 rows,
> odds are that each row is in a separate page.  So, to do the query, we
> are retrieving 2,500 8k pages, or 20MB of random data on the disk.  How
> long does it take to issue 2,500 disk requests that are scattered all
> over the disk.  Probably 11-22 seconds.
> 
> My OS only lets me have 400 8k buffers, or 3.2MB of buffer.  As we are
> transferring 2,500 8k pages or 20MB of data, it is no surprise that the
> buffer cache doesn't help much.  Sometime, the data is grouped together
> on the disk, and that is why some are fast, but often they are not, and
> certainly in a situation where you are looking for two words to appear
> on the same row, they certainly are not on adjacent pages.
> 
> Just started running CLUSTER, and it appears to be taking forever.  Does
> not seem to be using the disk, but a lot of CPU.  It appears to be
> caught in an endless loop.

OK, I have an answer for you.

First, the CLUSTER problem I was having was some fluke, probably
something else that got stuck somewhere.  Not a bug.

Second, CLUSTER takes forever because it is moving all over the disk
retrieving each row in order.

Third, I should have been able to solve this for you sooner.  The issues
of slowness you are seeing are the exact issues I dealt with five years
ago when I designed this fragmentation system.  I was so excited that
you could develop triggers to slice the words, I had forgotten the other
issues.

OK, it is.  As part of this fragmentation job, EVERY NIGHT, I re-slice
the user strings and dump them into a flat file.  I then load them into
Ingres as a heap table, then modify the table to ISAM.

You may say, why every night.  Well, the problem is I load the data at
100% fill-factor, and ISAM rapidly becomes fragmented with
insertions/deletions.

Btree does not become fragmented, but the problem there is that btree
use was very slow.  I believe this is because ISAM is basically a SORTED
HEAP with an index, so everything is close and packed tight.  Btree
doesn't seem to do that as well.  It is more spread out.

Users complain about the nightly re-index, but I tell them, 'Hey, you
are searching for fragments of words in 200k entries in 6 seconds.  I can
design it without the night job, but your searches all day will take
much longer."  That stops the conversation.

So, to your solution.  CLUSTER is too slow.  The disk is going crazy
moving single rows into the temp table.  I recommend doing a COPY of
artist_fti to a flat file, doing a Unix 'sort' on the flat file, then
re-loading the data into the artist_fti, and then putting the index on
the table and vacuum.

I have done this, and now all searches are instantaneous THE FIRST TIME
and every time.

With this change, I am anxious to hear how fast you can now do your
multi-word searches.  Daily changes will not really impact performance
because they are a small part of the total search, but this process of
COPY/sort/reload/reindex will need to be done on a regular basis to
maintain performance.


-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  8. Re: [HACKERS] Re: indexing words slow

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-03-14T06:40:12Z

    > Second, CLUSTER takes forever because it is moving all over the disk
> retrieving each row in order.
> 

> So, to your solution.  CLUSTER is too slow.  The disk is going crazy
> moving single rows into the temp table.  I recommend doing a COPY of
> artist_fti to a flat file, doing a Unix 'sort' on the flat file, then
> re-loading the data into the artist_fti, and then putting the index on
> the table and vacuum.
> 
> I have done this, and now all searches are instantaneous THE FIRST TIME
> and every time.
> 
> With this change, I am anxious to hear how fast you can now do your
> multi-word searches.  Daily changes will not really impact performance
> because they are a small part of the total search, but this process of
> COPY/sort/reload/reindex will need to be done on a regular basis to
> maintain performance.

One more piece of good news.  The reason CLUSTER was so slow is because
you loaded massive unordered amounts of data into the system.  Once you
do the COPY out/reload, subsequent clusters will run very quickly,
because 99% of the data is already ordered.  Only the new/changed data
is unordered, so you should be able to rapidly run CLUSTER from then on
to keep good performance.

I think a user module allowing this word fragment searching will be a
big hit with users.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  9. Re: [DOCS] FAQ organization

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-19T02:50:17Z

    Here is a general comments on maintaining the FAQ, and pointing people
to answers.

First, the FAQ is a pretty weird way to give people information.  It is
an almost random list of questions and answers.  If the list is long,
people can get very impatient, and start skimming over the list.  That
is OK, as long as they can find the information when they need it. 
Unfortunately, because it is in random order, it makes things very hard
to find.

OK, my strategy.  Let me give you two examples.  People get deadlocks
quite often, usually novice SQL users.  We need to explain to them what
they have done so they can re-design their queries to avoid it.  I could
have put it in the FAQ, but it really belongs in the LOCK manual page. 
So I put it in the lock manual page, and added a mention in the
'deadlock' error message to look at the 'lock' manual page for a
description of the problem.  This is perfect, because as soon as they
get the error, they are pointed to the proper place that has the exact
answer they need.  Same with the new postmaster -i option.  By putting a
mention in the libpq connect failure message to look for postmaster -i,
people are pointed right to the problem.

This has cut down on the problem reports tremendously.  I think
commercial software doesn't do very good in this area, probably because
the support people are not the same as the developers.  Because we are
the same people, we think of these things.

Second, if an FAQ issue is described in the manual pages or docs, I
point them to those, rather than re-iterating a long description of the
problem.  I have tried to move as much as possible OUT of the FAQ, and
into the well-structured manual pages or error message.  This leaves
real FAQ items on the FAQ, things that really don't fit in a manual page
or are too general to fit anywhere else.

I must say that people usually have been reading the FAQ, because we
rarely get a question that is answered on the FAQ.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  10. Re: [DOCS] FAQ organization

    Patrice Hédé <patrice@idf.net> — 1998-04-19T16:23:41Z

    On Sat, 18 Apr 1998, Bruce Momjian wrote:

> Here is a general comments on maintaining the FAQ, and pointing people
> to answers.
> 
> First, the FAQ is a pretty weird way to give people information.  It is
> an almost random list of questions and answers.  If the list is long,
> people can get very impatient, and start skimming over the list.  That
> is OK, as long as they can find the information when they need it. 
> Unfortunately, because it is in random order, it makes things very hard
> to find.

Well, this is why we need to structure it, so that it is _not_ a long
unsorted list of questions, and trying to group questions that look alike.

> OK, my strategy.  Let me give you two examples.  People get deadlocks
> quite often, usually novice SQL users.  We need to explain to them what
> they have done so they can re-design their queries to avoid it.  I could
> have put it in the FAQ, but it really belongs in the LOCK manual page. 
> So I put it in the lock manual page, and added a mention in the
> 'deadlock' error message to look at the 'lock' manual page for a
> description of the problem.  This is perfect, because as soon as they
> get the error, they are pointed to the proper place that has the exact
> answer they need.  Same with the new postmaster -i option.  By putting a
> mention in the libpq connect failure message to look for postmaster -i,
> people are pointed right to the problem.

Well, I understand what you want. Sure, the right solution for the FAQ
would be to give pointers to the manual pages. But sometimes it is not
sufficient. Sometimes, people have problems and have no idea where they
come from, they don't know where to look at, and what they should look at.
Sometimes there are more than one suitable way of solving a problem, since
similar problems can sometimes be best resolved by completely different
solutions, which are spread over the docs. There, the FAQ can be : "you've
got this kind of problems, you can solve it by doing that, or that, or
that. Look here, there, and there for details". 

Sometimes, it's better not to wait for people to get the error messages.
As a software user, I often read quickly through the FAQs to look at
common errors people do, so that, even before I start using the software,
I have a vague idea of what I should not do. So, people dealing with
concurrent access/update have a chance to be said : "Beware! You should
look at the LOCK man page / `Concurrent access under Postgres' user manual
section if you feel you might have deadlock problems". Or to people using
JDBC, "don't forget to use the -i flag !", instead of waiting for them to
panic because it doesn't work the way it should.

> This has cut down on the problem reports tremendously.  I think
> commercial software doesn't do very good in this area, probably because
> the support people are not the same as the developers.  Because we are
> the same people, we think of these things.

Well, actually, I've always found commercial software support docs - when
there are any - to be much worse that support files in free software :)
Especially the "troubleshooting" section, where they often consider 1. you
are dumb, and forgot to power on the computer 2. the software never fails,
so they don't describe at all the error code -3257.

> Second, if an FAQ issue is described in the manual pages or docs, I
> point them to those, rather than re-iterating a long description of the
> problem.  I have tried to move as much as possible OUT of the FAQ, and
> into the well-structured manual pages or error message.  This leaves
> real FAQ items on the FAQ, things that really don't fit in a manual page
> or are too general to fit anywhere else.

Well, the FAQ shouldn't re-say things in detail when they can be found
somewhere else. But sometimes, you have to explain the same things
differently because the problem doesn't direct people necessarily to the
right page, or maybe they should find it by themselves, but they don't,
because they are not familiar with the structure of the software or of the
support documentations.

Taking your example of the -i option : often people which have problems
concerning this don't even realize their problem comes from not allowing
the postmaster to listen to TCP/IP ports... they think JDBC, or perl
interface, or postgres itself, are not working, and the symptoms can be
quite different. You have to tell them the cause of their problem, and
where to look, and if you can provide a quick fix without obliging them to
dig in the docs, it can be only better.

Another example : constraints, foreign keys (I know the man pages will be
updated :) ). People ask questions like : "I did this which should work,
and it doesn't work in Postgres!". If they read the CREATE TABLE man page,
they would have the answer. But somehow there are people who, even after
reading the man page, won't see what's wrong. So the FAQ may be the
solution to their problem, saying quickly : "You cannot specify
constraints this way, do them that way (quick example). Read the CREATE
TABLE, CREATE INDEX man pages".

Hmm... actually, for this specific example, we could do a FAQ like this :

Qxx.xx : I tried to enter this SQL command as I used to in (MySQL,
Informix, ORACLE), and it doesn't work. Why ?

A : Every DBMS has some particularities in its SQL syntax. Please refer to
the Postgres man pages to see what syntax the Postgres parser follows.

etc...

and maybe some other questions, answering special cases where it is not so
evident... (we don't do subselects in target ?).

> I must say that people usually have been reading the FAQ, because we
> rarely get a question that is answered on the FAQ.

So by collecting questions on the mailing list, we can see what answers
the FAQ does not provide yet :)

Anyway, I think it's important for the FAQ to redirect people to the right
sections of the different manuals, but if you can have quick fixes, maybe
give them in the answer itself ("did you use the -i flag on the command
line ?").

It is better to say it twice, if it can help people solve their problem
faster, though answers _must_ stay short (or else they deserve to have a
special section in the main docs and a pointer to it :) ).

Do I seem reasonable ??

Patrice

--
Patrice HÉDÉ --------------------------------- patrice@idf.net -----

  11. Re: [DOCS] FAQ organization

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-20T17:50:23Z

    > So by collecting questions on the mailing list, we can see what answers
> the FAQ does not provide yet :)
> 
> Anyway, I think it's important for the FAQ to redirect people to the right
> sections of the different manuals, but if you can have quick fixes, maybe
> give them in the answer itself ("did you use the -i flag on the command
> line ?").
> 
> It is better to say it twice, if it can help people solve their problem
> faster, though answers _must_ stay short (or else they deserve to have a
> special section in the main docs and a pointer to it :) ).
> 
> Do I seem reasonable ??

I agree 100% with everything you said.  I just wanted to mention that if
you find frequent questions that should addressed in the manual
pages/docs, or error messages, please let us know so we can add them
there too.  I am sure there are many that I have missed.

I have taken a minimalist approach to the FAQ, partially because I have
not concentrated on it.  I am looking forward to a better, more
comprehensive FAQ.  I have been particularly lacking in adding FAQ items
for novice questions.


-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  12. Re: [DOCS] FAQ organization

    Patrice Hédé <patrice@idf.net> — 1998-04-21T10:33:33Z

    On Mon, 20 Apr 1998, Bruce Momjian wrote:

> I agree 100% with everything you said.  I just wanted to mention that if
> you find frequent questions that should addressed in the manual
> pages/docs, or error messages, please let us know so we can add them
> there too.  I am sure there are many that I have missed.

Ok. I'll do it in two steps : first, SGMLize the FAQ and adding entries,
then look at it and say "this should go here, that should move there,
etc..." in coordination with Tom and all the docs-enthousiasts :)

> I have taken a minimalist approach to the FAQ, partially because I have
> not concentrated on it.  I am looking forward to a better, more
> comprehensive FAQ.  I have been particularly lacking in adding FAQ items
> for novice questions.

Well, you have so much to do besides the FAQ. As I don't do any coding,
I'll be able to focus on docs, and especially the FAQ. I still wonder how
you all are able to find the time to code, debug, document, and still
keeping answering people on the mailing list and all ! :) 

Patrice

--
Patrice HÉDÉ --------------------------------- patrice@idf.net -----

  13. Re: [DOCS] FAQ organization

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-21T16:43:40Z

    > > I have taken a minimalist approach to the FAQ, partially because I have
> > not concentrated on it.  I am looking forward to a better, more
> > comprehensive FAQ.  I have been particularly lacking in adding FAQ items
> > for novice questions.
> 
> Well, you have so much to do besides the FAQ. As I don't do any coding,
> I'll be able to focus on docs, and especially the FAQ. I still wonder how
> you all are able to find the time to code, debug, document, and still
> keeping answering people on the mailing list and all ! :) 

I only spend about 2-3 days a month coding.  The rest is just little
tweaking.


-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  14. Re: [HACKERS] removing the exec() from doexec()

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-30T01:53:36Z

    > This exec() takes 15% of our startup time.  I have wanted it removed for
> many releases now.  The only problem is to rip out the code that
> re-attached to shared memory and stuff like that, because you will no
> longer loose the shared memory in the exec().  The IPC code is
> complicated, so good luck.  I or others can help if you get stuck.
> 

Another item is to no longer use SYSV shared memory but use
mmap(MAP_ANON) because this allows a much larger amount of shared memory
to be used.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  15. Re: [HACKERS] removing the exec() from doexec()

    Brett McCormick <brett@work.chicken.org> — 1998-04-30T02:01:01Z

    On Wed, 29 April 1998, at 21:53:36, Bruce Momjian wrote:

> > This exec() takes 15% of our startup time.  I have wanted it removed for
> > many releases now.  The only problem is to rip out the code that
> > re-attached to shared memory and stuff like that, because you will no
> > longer loose the shared memory in the exec().  The IPC code is
> > complicated, so good luck.  I or others can help if you get stuck.
> > 
> 
> Another item is to no longer use SYSV shared memory but use
> mmap(MAP_ANON) because this allows a much larger amount of shared memory
> to be used.

What are the portability issues?  I haven't written much portable
code, and certainly not with IPC.

  16. Re: [HACKERS] removing the exec() from doexec()

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-04-30T02:05:41Z

    > 
> On Wed, 29 April 1998, at 21:53:36, Bruce Momjian wrote:
> 
> > > This exec() takes 15% of our startup time.  I have wanted it removed for
> > > many releases now.  The only problem is to rip out the code that
> > > re-attached to shared memory and stuff like that, because you will no
> > > longer loose the shared memory in the exec().  The IPC code is
> > > complicated, so good luck.  I or others can help if you get stuck.
> > > 
> > 
> > Another item is to no longer use SYSV shared memory but use
> > mmap(MAP_ANON) because this allows a much larger amount of shared memory
> > to be used.
> 
> What are the portability issues?  I haven't written much portable
> code, and certainly not with IPC.

Not sure.  mmap() is pretty portable.  We will shake out any portability
issues as we go, or you can ask the list if everyone has such-and-such a
function.


-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  17. Re: [HACKERS] Bug in postgresql-6.3.2

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-05-22T04:33:50Z

    > 
> Quick note... Just to say that I found a bug in postgres 6.3.2 that I just this
> minute downloaded from the ftp site... It doesn't compile under AIX 4.2.1 with
> the latest C for AIX ver 3.1.4
>  
> It's only aminor problem, some of the variables in pqcomm.c are declared as
> int, and being passed to functions that expect a long * variable (Actually the
> function paramaters are declared as size_t).
>  
> The fix is to change the addrlen variable used on line 673 to a size_t instead
> of an int, and also for the len variable used on line 787.
>  
> Sorry... No diffs... No time, and I dont' subscribe to the list... I just like
> postgres (Maybe I'll subscribe one day... Too busy at the moment).

The line you are complaining about is:

    if ((port->sock = accept(server_fd,
                             (struct sockaddr *) & port->raddr,
                             &addrlen)) < 0)

while BSDI has accept defined as:

	int accept(int s, struct sockaddr *addr, int *addrlen);

So AIX has the last parameter defined as size_t, huh?  I looked at the
accept manual page, and addrlen is the length of the addr field.  Hard
to imagine that is ever going to be larger than an int.  Does any other
OS have that third parameter as anything but an int*?

We may need to add some aix-specific check on a configure check for
this.


-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  18. RE: [HACKERS] Bug in postgresql-6.3.2

    Stupor Genius <stuporg@erols.com> — 1998-05-22T05:27:41Z

    > The line you are complaining about is:
> 
>     if ((port->sock = accept(server_fd,
>                              (struct sockaddr *) & port->raddr,
>                              &addrlen)) < 0)
> 
> while BSDI has accept defined as:
> 
> 	int accept(int s, struct sockaddr *addr, int *addrlen);
> 
> So AIX has the last parameter defined as size_t, huh?  I looked at the
> accept manual page, and addrlen is the length of the addr field.  Hard
> to imagine that is ever going to be larger than an int.  Does any other
> OS have that third parameter as anything but an int*?
> 
> We may need to add some aix-specific check on a configure check for
> this.

>From aix 4.1 to 4.2, it changed from an int* to an unsigned long*, which
is probably what size_t is defined as.

Wasn't just accept() though. There were other socket functions, but I
don't recall the names offhand.  Not around aix anymore either... :)

Check thru the questions digests. I helped a couple of people compile
past this glitch, latest being Jim Kraii I believe.

darrenk

  19. Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-06-13T05:31:19Z

    > 
> > Please enter a FULL description of your problem:
> > ------------------------------------------------
> > Dropping table after aborting a transanction makes PosgresSQL unsable.
> > 
> > 
> > Please describe a way to repeat the problem.   Please try to provide a
> > concise reproducible example, if at all possible: 
> > ----------------------------------------------------------------------
> > [srashd]t-ishii{67} psql -e test < b
> > QUERY: drop table test;
> > WARN:Relation test Does Not Exist!
> > QUERY: create table test (i int4);
> > QUERY: create index iindex on test using btree(i);
> > QUERY: begin;
> > QUERY: insert into test values (100);
> > QUERY: select * from test;
> >   i
> > ---
> > 100
> > (1 row)
> > 
> > QUERY: rollback;
> > QUERY: drop table test;
> > NOTICE:AbortTransaction and not in in-progress state 
> > NOTICE:AbortTransaction and not in in-progress state 
> > 
> > Note that if I do not make an index, it would be ok.
> 
> Can someone comment on the cause of the above problem?  Is it a bug to
> add to the TODO list?  I have verified it still exists in the current
> sources.
> 

The message I see in the logs for this is:

ERROR:  cannot write block 1 of iindex [test] blind
NOTICE:  AbortTransaction and not in in-progress state
NOTICE:  EndTransactionBlock and not inprogress/abort state

Vadim, sounds familiar.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  20. Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    Vadim Mikheev <vadim@krs.ru> — 1998-06-13T06:18:05Z

    Bruce Momjian wrote:
> 
> > > QUERY: drop table test;
> > > WARN:Relation test Does Not Exist!
> > > QUERY: create table test (i int4);
> > > QUERY: create index iindex on test using btree(i);
> > > QUERY: begin;
> > > QUERY: insert into test values (100);

There will be dirty heap & index buffers in the pool after insertion ...

> > > QUERY: select * from test;
> > >   i
> > > ---
> > > 100
> > > (1 row)
> > >
> > > QUERY: rollback;

They are still marked as dirty...

> > > QUERY: drop table test;

heap_destroy_with_catalog () calls ReleaseRelationBuffers:

 *      this function unmarks all the dirty pages of a relation
 *      in the buffer pool so that at the end of transaction
 *      these pages will not be flushed.

before unlinking relation, BUT index_destroy() unlinks index
and DOESN'T call this func ...

> > > NOTICE:AbortTransaction and not in in-progress state
> > > NOTICE:AbortTransaction and not in in-progress state

COMMIT (of drop table) tries to flush all dirty buffers
from pool but there is no index file any more ...

> ERROR:  cannot write block 1 of iindex [test] blind

smgrblindwrt () fails.

> NOTICE:  AbortTransaction and not in in-progress state
> NOTICE:  EndTransactionBlock and not inprogress/abort state

...transaction state is IN_COMMIT...

Seems that ReleaseRelationBuffers() should be called by
index_destroy() ... Note that heap_destroy() also calls

    /* ok - flush the relation from the relcache */
    RelationForgetRelation(rid);

at the end...

Vadim

  21. Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-06-13T20:18:36Z

    > 
> Bruce Momjian wrote:
> > 
> > > > QUERY: drop table test;
> > > > WARN:Relation test Does Not Exist!
> > > > QUERY: create table test (i int4);
> > > > QUERY: create index iindex on test using btree(i);
> > > > QUERY: begin;
> > > > QUERY: insert into test values (100);
> 
> There will be dirty heap & index buffers in the pool after insertion ...
> 
> > > > QUERY: select * from test;
> > > >   i
> > > > ---
> > > > 100
> > > > (1 row)
> > > >
> > > > QUERY: rollback;
> 
> They are still marked as dirty...
> 
> > > > QUERY: drop table test;
> 
> heap_destroy_with_catalog () calls ReleaseRelationBuffers:
> 
>  *      this function unmarks all the dirty pages of a relation
>  *      in the buffer pool so that at the end of transaction
>  *      these pages will not be flushed.
> 
> before unlinking relation, BUT index_destroy() unlinks index
> and DOESN'T call this func ...
> 
> > > > NOTICE:AbortTransaction and not in in-progress state
> > > > NOTICE:AbortTransaction and not in in-progress state
> 
> COMMIT (of drop table) tries to flush all dirty buffers
> from pool but there is no index file any more ...
> 
> > ERROR:  cannot write block 1 of iindex [test] blind
> 
> smgrblindwrt () fails.
> 
> > NOTICE:  AbortTransaction and not in in-progress state
> > NOTICE:  EndTransactionBlock and not inprogress/abort state
> 
> ...transaction state is IN_COMMIT...
> 
> Seems that ReleaseRelationBuffers() should be called by
> index_destroy() ... Note that heap_destroy() also calls
> 
>     /* ok - flush the relation from the relcache */
>     RelationForgetRelation(rid);
> 
> at the end...
> 
> Vadim
> 

OK, the following patch fixes the problem.  Vadim, I added both function
calls to index_destroy and made heap_destroy consistent with
heap_destroy_with_catalog.

---------------------------------------------------------------------------

Index: src/backend/catalog/heap.c
===================================================================
RCS file: /usr/local/cvsroot/pgsql/src/backend/catalog/heap.c,v
retrieving revision 1.48
diff -c -r1.48 heap.c
*** heap.c	1998/04/27 04:04:47	1.48
--- heap.c	1998/06/13 20:13:18
***************
*** 12,18 ****
   * INTERFACE ROUTINES
   *		heap_create()			- Create an uncataloged heap relation
   *		heap_create_with_catalog() - Create a cataloged relation
!  *		heap_destroy_with_catalog()			- Removes named relation from catalogs
   *
   * NOTES
   *	  this code taken from access/heap/create.c, which contains
--- 12,18 ----
   * INTERFACE ROUTINES
   *		heap_create()			- Create an uncataloged heap relation
   *		heap_create_with_catalog() - Create a cataloged relation
!  *		heap_destroy_with_catalog()	- Removes named relation from catalogs
   *
   * NOTES
   *	  this code taken from access/heap/create.c, which contains
***************
*** 1290,1307 ****
  	 * ----------------
  	 */
  	if (rdesc->rd_rel->relhasindex)
- 	{
  		RelationRemoveIndexes(rdesc);
- 	}
  
  	/* ----------------
  	 *	remove rules if necessary
  	 * ----------------
  	 */
  	if (rdesc->rd_rules != NULL)
- 	{
  		RelationRemoveRules(rid);
- 	}
  
  	/* triggers */
  	if (rdesc->rd_rel->reltriggers > 0)
--- 1290,1303 ----
***************
*** 1347,1355 ****
  	 * ----------------
  	 */
  	if (!(rdesc->rd_istemp) || !(rdesc->rd_tmpunlinked))
- 	{
  		smgrunlink(DEFAULT_SMGR, rdesc);
! 	}
  	rdesc->rd_tmpunlinked = TRUE;
  
  	RelationUnsetLockForWrite(rdesc);
--- 1343,1350 ----
  	 * ----------------
  	 */
  	if (!(rdesc->rd_istemp) || !(rdesc->rd_tmpunlinked))
  		smgrunlink(DEFAULT_SMGR, rdesc);
! 
  	rdesc->rd_tmpunlinked = TRUE;
  
  	RelationUnsetLockForWrite(rdesc);
***************
*** 1375,1380 ****
--- 1370,1376 ----
  	rdesc->rd_tmpunlinked = TRUE;
  	heap_close(rdesc);
  	RemoveFromTempRelList(rdesc);
+ 	RelationForgetRelation(rdesc->rd_id);
  }
  
  
Index: src/backend/catalog/index.c
===================================================================
RCS file: /usr/local/cvsroot/pgsql/src/backend/catalog/index.c,v
retrieving revision 1.41
diff -c -r1.41 index.c
*** index.c	1998/05/09 23:42:59	1.41
--- index.c	1998/06/13 20:13:27
***************
*** 1270,1276 ****
  	while (tuple = heap_getnext(scan, 0, (Buffer *) NULL),
  		   HeapTupleIsValid(tuple))
  	{
- 
  		heap_delete(catalogRelation, &tuple->t_ctid);
  	}
  	heap_endscan(scan);
--- 1270,1275 ----
***************
*** 1296,1307 ****
  	heap_close(catalogRelation);
  
  	/*
! 	 * physically remove the file
  	 */
  	if (FileNameUnlink(relpath(indexRelation->rd_rel->relname.data)) < 0)
  		elog(ERROR, "amdestroyr: unlink: %m");
  
  	index_close(indexRelation);
  }
  
  /* ----------------------------------------------------------------
--- 1295,1309 ----
  	heap_close(catalogRelation);
  
  	/*
! 	 * flush cache and physically remove the file
  	 */
+ 	ReleaseRelationBuffers(indexRelation);
+ 
  	if (FileNameUnlink(relpath(indexRelation->rd_rel->relname.data)) < 0)
  		elog(ERROR, "amdestroyr: unlink: %m");
  
  	index_close(indexRelation);
+ 	RelationForgetRelation(indexRelation->rd_id);
  }
  
  /* ----------------------------------------------------------------

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  22. Re: [HACKERS] Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    David Gould <dg@illustra.com> — 1998-06-14T01:32:16Z

    Bruce Momjian wrote:
> > Bruce Momjian wrote:
> > > 
> > > > > QUERY: drop table test;
> > > > > WARN:Relation test Does Not Exist!
> > > > > QUERY: create table test (i int4);
> > > > > QUERY: create index iindex on test using btree(i);
> > > > > QUERY: begin;
> > > > > QUERY: insert into test values (100);
> > 
> > There will be dirty heap & index buffers in the pool after insertion ...
> > 
> > > > > QUERY: select * from test;
> > > > >   i
> > > > > ---
> > > > > 100
> > > > > (1 row)
> > > > >
> > > > > QUERY: rollback;
> > 
> > They are still marked as dirty...
> > 
> > > > > QUERY: drop table test;
> > 
> > heap_destroy_with_catalog () calls ReleaseRelationBuffers:
> > 
> >  *      this function unmarks all the dirty pages of a relation
> >  *      in the buffer pool so that at the end of transaction
> >  *      these pages will not be flushed.
> > 
> > before unlinking relation, BUT index_destroy() unlinks index
> > and DOESN'T call this func ...
> > 
> > > > > NOTICE:AbortTransaction and not in in-progress state
> > > > > NOTICE:AbortTransaction and not in in-progress state
> > 
> > COMMIT (of drop table) tries to flush all dirty buffers
> > from pool but there is no index file any more ...
> > 
> > > ERROR:  cannot write block 1 of iindex [test] blind
> > 
> > smgrblindwrt () fails.
> > 
> > > NOTICE:  AbortTransaction and not in in-progress state
> > > NOTICE:  EndTransactionBlock and not inprogress/abort state
> > 
> > ...transaction state is IN_COMMIT...
> > 
> > Seems that ReleaseRelationBuffers() should be called by
> > index_destroy() ... Note that heap_destroy() also calls
> > 
> >     /* ok - flush the relation from the relcache */
> >     RelationForgetRelation(rid);
> > 
> > at the end...
> > 
> > Vadim
> > 
> 
> OK, the following patch fixes the problem.  Vadim, I added both function
> calls to index_destroy and made heap_destroy consistent with
> heap_destroy_with_catalog.
> 
> ---------------------------------------------------------------------------
> 
> Index: src/backend/catalog/heap.c
> ===================================================================
> RCS file: /usr/local/cvsroot/pgsql/src/backend/catalog/heap.c,v
> retrieving revision 1.48
> diff -c -r1.48 heap.c
> *** heap.c	1998/04/27 04:04:47	1.48
> --- heap.c	1998/06/13 20:13:18
> ***************
> *** 12,18 ****
>    * INTERFACE ROUTINES
>    *		heap_create()			- Create an uncataloged heap relation
>    *		heap_create_with_catalog() - Create a cataloged relation
> !  *		heap_destroy_with_catalog()			- Removes named relation from catalogs
>    *
>    * NOTES
>    *	  this code taken from access/heap/create.c, which contains
> --- 12,18 ----
>    * INTERFACE ROUTINES
>    *		heap_create()			- Create an uncataloged heap relation
>    *		heap_create_with_catalog() - Create a cataloged relation
> !  *		heap_destroy_with_catalog()	- Removes named relation from catalogs
>    *
>    * NOTES
>    *	  this code taken from access/heap/create.c, which contains
> ***************
> *** 1290,1307 ****
>   	 * ----------------
>   	 */
>   	if (rdesc->rd_rel->relhasindex)
> - 	{
>   		RelationRemoveIndexes(rdesc);
> - 	}
>   
>   	/* ----------------
>   	 *	remove rules if necessary
>   	 * ----------------
>   	 */
>   	if (rdesc->rd_rules != NULL)
> - 	{
>   		RelationRemoveRules(rid);
> - 	}
>   
>   	/* triggers */
>   	if (rdesc->rd_rel->reltriggers > 0)
> --- 1290,1303 ----
> ***************
> *** 1347,1355 ****
>   	 * ----------------
>   	 */
>   	if (!(rdesc->rd_istemp) || !(rdesc->rd_tmpunlinked))
> - 	{
>   		smgrunlink(DEFAULT_SMGR, rdesc);
> ! 	}
>   	rdesc->rd_tmpunlinked = TRUE;
>   
>   	RelationUnsetLockForWrite(rdesc);
> --- 1343,1350 ----
>   	 * ----------------
>   	 */
>   	if (!(rdesc->rd_istemp) || !(rdesc->rd_tmpunlinked))
>   		smgrunlink(DEFAULT_SMGR, rdesc);
> ! 
>   	rdesc->rd_tmpunlinked = TRUE;
>   
>   	RelationUnsetLockForWrite(rdesc);
> ***************
> *** 1375,1380 ****
> --- 1370,1376 ----
>   	rdesc->rd_tmpunlinked = TRUE;
>   	heap_close(rdesc);
>   	RemoveFromTempRelList(rdesc);
> + 	RelationForgetRelation(rdesc->rd_id);
>   }
>   
>   
> Index: src/backend/catalog/index.c
> ===================================================================
> RCS file: /usr/local/cvsroot/pgsql/src/backend/catalog/index.c,v
> retrieving revision 1.41
> diff -c -r1.41 index.c
> *** index.c	1998/05/09 23:42:59	1.41
> --- index.c	1998/06/13 20:13:27
> ***************
> *** 1270,1276 ****
>   	while (tuple = heap_getnext(scan, 0, (Buffer *) NULL),
>   		   HeapTupleIsValid(tuple))
>   	{
> - 
>   		heap_delete(catalogRelation, &tuple->t_ctid);
>   	}
>   	heap_endscan(scan);
> --- 1270,1275 ----
> ***************
> *** 1296,1307 ****
>   	heap_close(catalogRelation);
>   
>   	/*
> ! 	 * physically remove the file
>   	 */
>   	if (FileNameUnlink(relpath(indexRelation->rd_rel->relname.data)) < 0)
>   		elog(ERROR, "amdestroyr: unlink: %m");
>   
>   	index_close(indexRelation);
>   }
>   
>   /* ----------------------------------------------------------------
> --- 1295,1309 ----
>   	heap_close(catalogRelation);
>   
>   	/*
> ! 	 * flush cache and physically remove the file
>   	 */
> + 	ReleaseRelationBuffers(indexRelation);
> + 
>   	if (FileNameUnlink(relpath(indexRelation->rd_rel->relname.data)) < 0)
>   		elog(ERROR, "amdestroyr: unlink: %m");
>   
>   	index_close(indexRelation);
> + 	RelationForgetRelation(indexRelation->rd_id);
>   }
>   
>   /* ----------------------------------------------------------------


Two comments:

 - I notice you getting rid of { } pairs eg:

   if (condition)
   {
       dosomething();
   }

   becomes

   if (condition)
       dosomething();

   Is this policy? I prefer to have the braces almost always as I find
   it easier to read, and less error prone if someone adds a statement or an
   else clause, so in most of my patches, I would tend to put braces in.
   If you are busy taking them out simaltaniously, this could get silly.

   Btw, I have been badly bit by:

   if (condition);
       dosomething();

   which turned out to be very hard to see indeed.


 - I think the bit at line 1295-1309 might want to do all the work before
   the elog. Otherwise the elog leave the buffer cache polluted with buffers
   belonging to a mostly deleted index. Eg:

   +     ReleaseRelationBuffers(indexRelation);
   +
         fname = relpath(indexRelation->rd_rel->relname.data);
	 status = FileNameUnlink(fname);

         index_close(indexRelation);
   +     RelationForgetRelation(indexRelation->rd_id);

         if (status < 0)
                 elog(ERROR, "amdestroyr: unlink: %m");

-dg
 
David Gould            dg@illustra.com           510.628.3783 or 510.305.9468 
Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
"Don't worry about people stealing your ideas.  If your ideas are any
 good, you'll have to ram them down people's throats." -- Howard Aiken

  23. Re: [HACKERS] Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    Bruce Momjian <maillist@candle.pha.pa.us> — 1998-06-14T02:35:28Z

    > Two comments:
> 
>  - I notice you getting rid of { } pairs eg:
> 
>    if (condition)
>    {
>        dosomething();
>    }
> 
>    becomes
> 
>    if (condition)
>        dosomething();
> 
>    Is this policy? I prefer to have the braces almost always as I find
>    it easier to read, and less error prone if someone adds a statement or an
>    else clause, so in most of my patches, I would tend to put braces in.
>    If you are busy taking them out simaltaniously, this could get silly.

I think several developers agreed that they were just wasting screen
space.  The code is large enough without brace noise.  I have considered
writing a script to remove the single-statement braces, but have not
done it yet.

If people would like to re-vote on this issue, I would be glad to hear
about it.

> 
>    Btw, I have been badly bit by:
> 
>    if (condition);
>        dosomething();
> 
>    which turned out to be very hard to see indeed.

Sure, but braces don't help you either.  This is just as legal:

	if (condition);
	{
		dosomething();
	}


> 
> 
>  - I think the bit at line 1295-1309 might want to do all the work before
>    the elog. Otherwise the elog leave the buffer cache polluted with buffers
>    belonging to a mostly deleted index. Eg:
> 
>    +     ReleaseRelationBuffers(indexRelation);
>    +
>          fname = relpath(indexRelation->rd_rel->relname.data);
> 	 status = FileNameUnlink(fname);
> 
>          index_close(indexRelation);
>    +     RelationForgetRelation(indexRelation->rd_id);
> 
>          if (status < 0)
>                  elog(ERROR, "amdestroyr: unlink: %m");

Yes, that is true, but I kept the order as used in the rest of the code,
figuring the original coder knew better than I do.  IMHO, if we get that
"amdestroyr" error, we have bigger problems than an invalid relation
cache

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

  24. Re: [HACKERS] Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    David Gould <dg@illustra.com> — 1998-06-14T02:59:12Z

    > I think several developers agreed that they were just wasting screen
> space.  The code is large enough without brace noise.  I have considered
> writing a script to remove the single-statement braces, but have not
> done it yet.
> 
> If people would like to re-vote on this issue, I would be glad to hear
> about it.

Ok, I won't add them. I'm not taking them out if I see them though ;-). 

> Sure, but braces don't help you either.  This is just as legal:
> 
> 	if (condition);
> 	{
> 		dosomething();
> 	}

True enough, but I think less likely to happen. 

> >  - I think the bit at line 1295-1309 might want to do all the work before
> >    the elog. Otherwise the elog leave the buffer cache polluted with buffers
> >    belonging to a mostly deleted index. Eg:
> > 
> >    +     ReleaseRelationBuffers(indexRelation);
> >    +
> >          fname = relpath(indexRelation->rd_rel->relname.data);
> > 	 status = FileNameUnlink(fname);
> > 
> >          index_close(indexRelation);
> >    +     RelationForgetRelation(indexRelation->rd_id);
> > 
> >          if (status < 0)
> >                  elog(ERROR, "amdestroyr: unlink: %m");
> 
> Yes, that is true, but I kept the order as used in the rest of the code,
> figuring the original coder knew better than I do.  IMHO, if we get that
> "amdestroyr" error, we have bigger problems than an invalid relation
> cache

Well, the code in heap.c calls smgrunlink() without checking the return code.
smgrunlink() calls mdunlink() which contains:

        if (FileNameUnlink(fname) < 0)
                return (SM_FAIL);

So heap_destroy does not even through an elog() at all if the FileNameUnlink()
fails. I think this is actually the right policy since if FileNameUnlink()
fails the only consequence is that a file is left on the disk (or maybe
the unlink failed because it was already gone). The system state (buffers etc)
and catalogs are consitant with the heap having been destroyed. So not a
problem from the database's perspective. 

I suggest you change your patch to simply ignore the return code from
FileNameUnlink(). As in:

     ReleaseRelationBuffers(indexRelation);

     (void) FileNameUnlink(relpath(indexRelation->rd_rel->relname.data));

     index_close(indexRelation);
     RelationForgetRelation(indexRelation->rd_id); 

In this way it will have the same behaviour as heap_destroy...

-dg

David Gould           dg@illustra.com            510.628.3783 or 510.305.9468
Informix Software                      300 Lakeside Drive   Oakland, CA 94612
 - A child of five could understand this!  Fetch me a child of five.

  25. Re: [HACKERS] Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    Brett McCormick <brett@work.chicken.org> — 1998-06-14T03:07:35Z

    I've been trying to use them in my patches as well, maily for
consistency as its something I'd never do myself.  I'll gladly
stop though.

On Sat, 13 June 1998, at 19:59:12, David Gould wrote:

> Ok, I won't add them. I'm not taking them out if I see them though ;-). 
> 
> > Sure, but braces don't help you either.  This is just as legal:
> > 
> > 	if (condition);
> > 	{
> > 		dosomething();
> > 	}
> 
> True enough, but I think less likely to happen.

  26. Re: [HACKERS] Re: [BUGS] NOTICE:AbortTransaction and not in in-progress state

    Vadim Mikheev <vadim@krs.ru> — 1998-06-14T06:35:31Z

    Bruce Momjian wrote:
> 
> Index: src/backend/catalog/heap.c
> ===================================================================
> ***************
> *** 1375,1380 ****
> --- 1370,1376 ----
>         rdesc->rd_tmpunlinked = TRUE;
>         heap_close(rdesc);
>         RemoveFromTempRelList(rdesc);
> +       RelationForgetRelation(rdesc->rd_id);

We need not in RelationForgetRelation() in heap_destroy().
Local relations are handled in other way...

Vadim

  27. Re: SGI Port of Postgresql 6.4 snapshot of 09/28/98

    Robert Bruccoleri <bruc@pluto.njcc.com> — 1998-09-29T15:01:35Z

    > Gentlemen:
	The test of the Irix port of PostgreSQL running with default
optimization passes all the regression tests the same as before
except that the random number test is now different. The new output is
included below. It appears to be OK, but I would like confirmation.
Thanks.

=============================random.out=================================

QUERY: SELECT count(*) FROM onek;
count
-----
 1000
(1 row)

QUERY: SELECT count(*) AS random INTO RANDOM_TBL
  FROM onek WHERE oidrand(onek.oid, 10);
QUERY: INSERT INTO RANDOM_TBL (random)
  SELECT count(*)
  FROM onek WHERE oidrand(onek.oid, 10);
QUERY: SELECT random, count(random) FROM RANDOM_TBL
  GROUP BY random HAVING count(random) > 1;
random|count
------+-----
(0 rows)

QUERY: SELECT random FROM RANDOM_TBL
  WHERE random NOT BETWEEN 80 AND 120;
random
------
    74
(1 row)

+------------------------------------------+------------------------------+
| Robert E. Bruccoleri, Ph.D.              | Associate Research Professor |
| phone: 732 235 5796                      | Center for Advanced          |
| Fax:   732 235 4850                      |   Biotechnology and Medicine |
| email: bruc@acm.org                      | Rutgers University           |
| URL:   http://www.cabm.rutgers.edu/~bruc | 679 Hoes Lane                |
|                                          | Piscataway, NJ 08854-5638    |
+------------------------------------------+------------------------------+

  28. Re: [HACKERS] Permissions not working?

    D'Arcy Cain <darcy@druid.net> — 1998-10-13T17:13:02Z

    Thus spake darcy
> >     Here I assume user darcy has usesuper set in pg_shadow. Check
> >     and correct me if I'm wrong. The superuser flag is set if you
> >     allow darcy to create users on createuser time.
> 
> Correct again.  I half suspected something like this.  Perhaps the
> prompt in createuser should be changed to reflect that the user is
> being granted full superuser privileges rather than just being able
> to create more users.

Should I send in the (trivial) diffs to effect this change?  I figure
to ask the following.

    "Does user "x" have superuser privileges? (y/n)"

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 424 2871     (DoD#0082)    (eNTP)   |  what's for dinner.

  29. Re: [HACKERS] regress.sh

    Jan Wieck <wieck@debis.com> — 1999-06-14T20:14:03Z

    > > I strongly suggest patching this before 6.5 ...
>
>     No comment other that "sorry".

    Mark: Should I commit the fix on regress.sh?


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#========================================= wieck@debis.com (Jan Wieck) #

  30. Re: Link to bug webpage

    D'Arcy Cain <darcy@druid.net> — 2001-08-21T11:24:57Z

    Thus spake Bruce Momjian
> If anyone was concerned about our bug database being visible and giving
> the impression we don't fix any bugs, see this URL:
> 
> 	http://www.isthisthingon.org/nisca/postgres.html

Jeez, Louise.  Talk about a blaming the tools because you don't know
anything about database design.  I mean, his biggest complaint is that
PostgreSQL makes it hard (not impossible as he implies) to change the
schema.  Perhaps that is because it was written by GOOD database designers
who don't have to change their schema every other week and so that issue
hasn't been a squeaky wheel.

I can't believe that anyone important is listening to this guy.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.

  31. Re: Proposal: 7.2b2 today

    D'Arcy Cain <darcy@druid.net> — 2001-11-06T09:12:56Z

    Thus spake Bruce Momjian
> > Unless someone has something they are sitting on, I'd like to wrap up a
> > 7.2b2 this afternoon, and do a proper release announcement for it like
> > didn't happen for 7.2b1 ...
> 
> I have been working with Tom on some pgindent issues and have made
> slight improvements to the script.  Because we are early in beta and no
> one has outstanding patches, I would like to run it again and commit the
> changes.  It should improve variables defined as structs and alignment
> of include/catalog/*.h files.

I have a change I would like to discuss.  It doesn't change the code
base, only the build system.  The patches I would like to commit follow
this text.  The only thing it does is create a config option to bytecode
compile the Python modules.  It also cleans up the install in the Makefile
a little bit.

Index: configure.in
===================================================================
RCS file: /cvsroot/pgsql/configure.in,v
retrieving revision 1.150
diff -u -r1.150 configure.in
--- configure.in	2001/10/25 13:02:01	1.150
+++ configure.in	2001/11/06 09:09:50
@@ -398,6 +398,18 @@
 AC_MSG_RESULT([$with_python])
 AC_SUBST(with_python)
 
+# If python is enabled (above), then optionally byte-compile the modules.
+AC_MSG_CHECKING([whether to byte-compile Python modules])
+if test "$with_python" = yes; then
+  PGAC_ARG_BOOL(with, python_compile, no,
+    [  --with-python-compile   byte-compile modules if Python is enabled])
+else
+  with_python_compile=no
+fi
+AC_MSG_RESULT([$with_python_compile])
+AC_SUBST([with_python_compile])
+
+
 #
 # Optionally build the Java/JDBC tools
 #
Index: src/Makefile.global.in
===================================================================
RCS file: /cvsroot/pgsql/src/Makefile.global.in,v
retrieving revision 1.140
diff -u -r1.140 Makefile.global.in
--- src/Makefile.global.in	2001/10/13 15:24:23	1.140
+++ src/Makefile.global.in	2001/11/06 09:09:54
@@ -123,6 +123,7 @@
 with_java	= @with_java@
 with_perl	= @with_perl@
 with_python	= @with_python@
+with_python_compile	= @with_python_compile@
 with_tcl	= @with_tcl@
 with_tk		= @with_tk@
 enable_odbc	= @enable_odbc@
Index: src/interfaces/python/GNUmakefile
===================================================================
RCS file: /cvsroot/pgsql/src/interfaces/python/GNUmakefile,v
retrieving revision 1.11
diff -u -r1.11 GNUmakefile
--- src/interfaces/python/GNUmakefile	2001/08/24 14:07:50	1.11
+++ src/interfaces/python/GNUmakefile	2001/11/06 09:10:00
@@ -19,10 +19,23 @@
 
 override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS) $(python_includespec)
 
-all: all-lib
+PY_SCRIPTS = pg.py pgdb.py
+ifeq ($(with_python_compile), yes)
+PY_COMPILED_SCRIPTS = $(PY_SCRIPTS:%.py=%.pyc) $(PY_SCRIPTS:%.py=%.pyo)
+else
+PY_COMPILED_SCRIPTS =
+endif
 
+all: all-lib $(PY_COMPILED_SCRIPTS)
+
 all-lib: libpq-all
 
+%.pyc: %.py
+	python -c "import py_compile; py_compile.compile(\"$<\")"
+
+%.pyo: %.py
+	python -O -c "import py_compile; py_compile.compile(\"$<\")"
+
 .PHONY: libpq-all
 libpq-all:
 	$(MAKE) -C $(libpq_builddir) all
@@ -37,12 +50,11 @@
 	@if test -w $(DESTDIR)$(python_moduleexecdir) && test -w $(DESTDIR)$(python_moduledir); then \
 	  echo "$(INSTALL_SHLIB) $(shlib) $(DESTDIR)$(python_moduleexecdir)/_pgmodule$(DLSUFFIX)"; \
 	  $(INSTALL_SHLIB) $(shlib) $(DESTDIR)$(python_moduleexecdir)/_pgmodule$(DLSUFFIX); \
-	\
-	  echo "$(INSTALL_DATA) $(srcdir)/pg.py $(DESTDIR)$(python_moduledir)/pg.py"; \
-	  $(INSTALL_DATA) $(srcdir)/pg.py $(DESTDIR)$(python_moduledir)/pg.py; \
 	\
-	  echo "$(INSTALL_DATA) $(srcdir)/pgdb.py $(DESTDIR)$(python_moduledir)/pgdb.py"; \
-	  $(INSTALL_DATA) $(srcdir)/pgdb.py $(DESTDIR)$(python_moduledir)/pgdb.py; \
+	  for i in $(PY_SCRIPTS) $(PY_COMPILED_SCRIPTS); do \
+		echo $(INSTALL_DATA) $$i $(python_moduledir); \
+		$(INSTALL_DATA) $$i $(python_moduledir); \
+	  done \
 	else \
 	  $(install-warning-msg); \
 	fi

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.

  32. Re: Proposal: 7.2b2 today

    Marc G. Fournier <scrappy@hub.org> — 2001-11-06T13:07:14Z

    Okay, I heard a 'yelp' from Tom concerning hte pgindent stuff, so I
haven't done beta2 up ... can someoen comment on this, as to whether we
can get it in, before I throw together beta2?

Tom ... are/were you okay with Bruce's last pgindent run?



On Tue, 6 Nov 2001, D'Arcy J.M. Cain wrote:

> Thus spake Bruce Momjian
> > > Unless someone has something they are sitting on, I'd like to wrap up a
> > > 7.2b2 this afternoon, and do a proper release announcement for it like
> > > didn't happen for 7.2b1 ...
> >
> > I have been working with Tom on some pgindent issues and have made
> > slight improvements to the script.  Because we are early in beta and no
> > one has outstanding patches, I would like to run it again and commit the
> > changes.  It should improve variables defined as structs and alignment
> > of include/catalog/*.h files.
>
> I have a change I would like to discuss.  It doesn't change the code
> base, only the build system.  The patches I would like to commit follow
> this text.  The only thing it does is create a config option to bytecode
> compile the Python modules.  It also cleans up the install in the Makefile
> a little bit.
>
> Index: configure.in
> ===================================================================
> RCS file: /cvsroot/pgsql/configure.in,v
> retrieving revision 1.150
> diff -u -r1.150 configure.in
> --- configure.in	2001/10/25 13:02:01	1.150
> +++ configure.in	2001/11/06 09:09:50
> @@ -398,6 +398,18 @@
>  AC_MSG_RESULT([$with_python])
>  AC_SUBST(with_python)
>
> +# If python is enabled (above), then optionally byte-compile the modules.
> +AC_MSG_CHECKING([whether to byte-compile Python modules])
> +if test "$with_python" = yes; then
> +  PGAC_ARG_BOOL(with, python_compile, no,
> +    [  --with-python-compile   byte-compile modules if Python is enabled])
> +else
> +  with_python_compile=no
> +fi
> +AC_MSG_RESULT([$with_python_compile])
> +AC_SUBST([with_python_compile])
> +
> +
>  #
>  # Optionally build the Java/JDBC tools
>  #
> Index: src/Makefile.global.in
> ===================================================================
> RCS file: /cvsroot/pgsql/src/Makefile.global.in,v
> retrieving revision 1.140
> diff -u -r1.140 Makefile.global.in
> --- src/Makefile.global.in	2001/10/13 15:24:23	1.140
> +++ src/Makefile.global.in	2001/11/06 09:09:54
> @@ -123,6 +123,7 @@
>  with_java	= @with_java@
>  with_perl	= @with_perl@
>  with_python	= @with_python@
> +with_python_compile	= @with_python_compile@
>  with_tcl	= @with_tcl@
>  with_tk		= @with_tk@
>  enable_odbc	= @enable_odbc@
> Index: src/interfaces/python/GNUmakefile
> ===================================================================
> RCS file: /cvsroot/pgsql/src/interfaces/python/GNUmakefile,v
> retrieving revision 1.11
> diff -u -r1.11 GNUmakefile
> --- src/interfaces/python/GNUmakefile	2001/08/24 14:07:50	1.11
> +++ src/interfaces/python/GNUmakefile	2001/11/06 09:10:00
> @@ -19,10 +19,23 @@
>
>  override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS) $(python_includespec)
>
> -all: all-lib
> +PY_SCRIPTS = pg.py pgdb.py
> +ifeq ($(with_python_compile), yes)
> +PY_COMPILED_SCRIPTS = $(PY_SCRIPTS:%.py=%.pyc) $(PY_SCRIPTS:%.py=%.pyo)
> +else
> +PY_COMPILED_SCRIPTS =
> +endif
>
> +all: all-lib $(PY_COMPILED_SCRIPTS)
> +
>  all-lib: libpq-all
>
> +%.pyc: %.py
> +	python -c "import py_compile; py_compile.compile(\"$<\")"
> +
> +%.pyo: %.py
> +	python -O -c "import py_compile; py_compile.compile(\"$<\")"
> +
>  .PHONY: libpq-all
>  libpq-all:
>  	$(MAKE) -C $(libpq_builddir) all
> @@ -37,12 +50,11 @@
>  	@if test -w $(DESTDIR)$(python_moduleexecdir) && test -w $(DESTDIR)$(python_moduledir); then \
>  	  echo "$(INSTALL_SHLIB) $(shlib) $(DESTDIR)$(python_moduleexecdir)/_pgmodule$(DLSUFFIX)"; \
>  	  $(INSTALL_SHLIB) $(shlib) $(DESTDIR)$(python_moduleexecdir)/_pgmodule$(DLSUFFIX); \
> -	\
> -	  echo "$(INSTALL_DATA) $(srcdir)/pg.py $(DESTDIR)$(python_moduledir)/pg.py"; \
> -	  $(INSTALL_DATA) $(srcdir)/pg.py $(DESTDIR)$(python_moduledir)/pg.py; \
>  	\
> -	  echo "$(INSTALL_DATA) $(srcdir)/pgdb.py $(DESTDIR)$(python_moduledir)/pgdb.py"; \
> -	  $(INSTALL_DATA) $(srcdir)/pgdb.py $(DESTDIR)$(python_moduledir)/pgdb.py; \
> +	  for i in $(PY_SCRIPTS) $(PY_COMPILED_SCRIPTS); do \
> +		echo $(INSTALL_DATA) $$i $(python_moduledir); \
> +		$(INSTALL_DATA) $$i $(python_moduledir); \
> +	  done \
>  	else \
>  	  $(install-warning-msg); \
>  	fi
>
> --
> D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
> http://www.druid.net/darcy/                |  and a sheep voting on
> +1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.
>

  33. Re: Proposal: 7.2b2 today

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-11-06T16:16:55Z

    > 
> Okay, I heard a 'yelp' from Tom concerning hte pgindent stuff, so I
> haven't done beta2 up ... can someoen comment on this, as to whether we
> can get it in, before I throw together beta2?
> 
> Tom ... are/were you okay with Bruce's last pgindent run?

I threw it up on a web site so people could review it.  With no
objections, I think we are fine for beta2.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  34. Re: Proposal: 7.2b2 today

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-11-06T17:07:07Z

    >> Okay, I heard a 'yelp' from Tom concerning hte pgindent stuff, so I
>> haven't done beta2 up ... can someoen comment on this, as to whether we
>> can get it in, before I throw together beta2?

I looked over the diffs, they seem okay.

Since Thomas just committed a horology regress test fix, the regression
tests are broken on platforms that use variant horology test files.
Give me an hour to do something about that, and then we can roll beta2.

			regards, tom lane

  35. Re: Proposal: 7.2b2 today

    Marc G. Fournier <scrappy@hub.org> — 2001-11-06T17:07:51Z

    sounds cool to me ...


On Tue, 6 Nov 2001, Tom Lane wrote:

> >> Okay, I heard a 'yelp' from Tom concerning hte pgindent stuff, so I
> >> haven't done beta2 up ... can someoen comment on this, as to whether we
> >> can get it in, before I throw together beta2?
>
> I looked over the diffs, they seem okay.
>
> Since Thomas just committed a horology regress test fix, the regression
> tests are broken on platforms that use variant horology test files.
> Give me an hour to do something about that, and then we can roll beta2.
>
> 			regards, tom lane
>

  36. Re: Proposal: 7.2b2 today

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-11-06T17:16:04Z

    > 
> sounds cool to me ...
> 

I am sorry about my pgindent run.  If I had realized it would hold up
beta for a day, I wouldn't have done it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  37. Re: Proposal: 7.2b2 today

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-11-06T17:29:45Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I am sorry about my pgindent run.  If I had realized it would hold up
> beta for a day, I wouldn't have done it.

Well, we needed the regression fix anyway.  Not a problem.

			regards, tom lane

  38. Re: Proposal: 7.2b2 today

    Marc G. Fournier <scrappy@hub.org> — 2001-11-06T17:48:14Z

    what about D'Arcy's python patch?

On Tue, 6 Nov 2001, Tom Lane wrote:

> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I am sorry about my pgindent run.  If I had realized it would hold up
> > beta for a day, I wouldn't have done it.
>
> Well, we needed the regression fix anyway.  Not a problem.
>
> 			regards, tom lane
>

  39. Re: Proposal: 7.2b2 today

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-11-06T17:49:19Z

    > 
> what about D'Arcy's python patch?

I think it has to wait for review or 7.3.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  40. Re: Proposal: 7.2b2 today

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-11-06T18:05:20Z

    > Since Thomas just committed a horology regress test fix, the regression
> tests are broken on platforms that use variant horology test files.
> Give me an hour to do something about that, and then we can roll beta2.

Done --- we're good to go, I think.

			regards, tom lane

  41. Re: Proposal: 7.2b2 today

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-11-06T18:23:14Z

    > 
> what about D'Arcy's python patch?

I will work up the Open Items list today and we can see what needs to be
put into beta3, or rc1, or whatever.  :-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  42. Re: Proposal: 7.2b2 today

    Marc G. Fournier <scrappy@hub.org> — 2001-11-06T18:43:07Z

    the patch adds a --with-python switch to confiugure, and appropriate lines
to the make files to compile it ... I would *like* to see it in beta2
unless someone can see a glaring error in it that would cause us to have
to delay beta2 ...

its less then 50 lines ... cna you please take a quick peak at it adn
apply it if you don't see anything jump out at you?

On Tue, 6 Nov 2001, Bruce Momjian wrote:

> >
> > what about D'Arcy's python patch?
>
> I will work up the Open Items list today and we can see what needs to be
> put into beta3, or rc1, or whatever.  :-)
>
> --
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 853-3000
>   +  If your life is a hard drive,     |  830 Blythe Avenue
>   +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
>

  43. Re: Proposal: 7.2b2 today

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-11-06T19:22:48Z

    > 
> the patch adds a --with-python switch to confiugure, and appropriate lines
> to the make files to compile it ... I would *like* to see it in beta2
> unless someone can see a glaring error in it that would cause us to have
> to delay beta2 ...
> 
> its less then 50 lines ... cna you please take a quick peak at it adn
> apply it if you don't see anything jump out at you?

Unfortunately I don't understand configure.in well enough to have any
comment on the code.  I recommend we make beta2 and give others time to
review it.  If it is OK, we can add it later.  We do have some other
open items like the Libpq signal handling and AIX compile so we are not
done applying things yet anyway.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  44. Re: Proposal: 7.2b2 today

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-11-06T19:29:22Z

    "Marc G. Fournier" <scrappy@hub.org> writes:
> what about D'Arcy's python patch?

Since it's a configure/build thing, I'd want to see Peter E's reaction
to it before making a decision.  I'd not suggest holding up beta2 for
it, anyway.

			regards, tom lane

  45. Re: Proposal: 7.2b2 today

    Peter Eisentraut <peter_e@gmx.net> — 2001-11-07T01:07:49Z

    D'Arcy J.M. Cain writes:

> I have a change I would like to discuss.  It doesn't change the code
> base, only the build system.  The patches I would like to commit follow
> this text.  The only thing it does is create a config option to bytecode
> compile the Python modules.

We've seen such a patch before, but I'm still not convinced it works.
According to my knowledge, the pre-compiled bytecode files need to be
created after the source files have been installed in their final
location, because the file name and timestamp is encoded in the compiled
file (it's sort of used as a cache file).  While this can be accomplished
with a different patch, it wouldn't really work when DESTDIR is used
because you'd create a "dead" cache file.  In a sense, this operation is
like running ldconfig -- it's outside the scope of the build system.
Package managers typically put it in the "post install" section.

> +# If python is enabled (above), then optionally byte-compile the modules.
> +AC_MSG_CHECKING([whether to byte-compile Python modules])
> +if test "$with_python" = yes; then
> +  PGAC_ARG_BOOL(with, python_compile, no,
> +    [  --with-python-compile   byte-compile modules if Python is enabled])

--enable

> +else
> +  with_python_compile=no
> +fi
> +AC_MSG_RESULT([$with_python_compile])
> +AC_SUBST([with_python_compile])

> +%.pyc: %.py
> +	python -c "import py_compile; py_compile.compile(\"$<\")"
> +
> +%.pyo: %.py
> +	python -O -c "import py_compile; py_compile.compile(\"$<\")"
> +

$(PYTHON)

-- 
Peter Eisentraut   peter_e@gmx.net

  46. Re: Proposal: 7.2b2 today

    D'Arcy Cain <darcy@druid.net> — 2001-11-08T01:50:28Z

    Thus spake Marc G. Fournier
> the patch adds a --with-python switch to confiugure, and appropriate lines
> to the make files to compile it ... I would *like* to see it in beta2
> unless someone can see a glaring error in it that would cause us to have
> to delay beta2 ...
> 
> its less then 50 lines ... cna you please take a quick peak at it adn
> apply it if you don't see anything jump out at you?

I have one minor change to it.  Where I call "python" to bytecode compile
it should be "$(PYTHON)" instead.  If you want I can just commit the
changes directly.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.

  47. Re: Proposal: 7.2b2 today

    D'Arcy Cain <darcy@druid.net> — 2002-03-20T18:25:43Z

    * Peter Eisentraut <peter_e@gmx.net> [011106 20:01]:
> D'Arcy J.M. Cain writes:
> > I have a change I would like to discuss.  It doesn't change the code
> > base, only the build system.  The patches I would like to commit follow
> > this text.  The only thing it does is create a config option to bytecode
> > compile the Python modules.
> 
> We've seen such a patch before, but I'm still not convinced it works.
> According to my knowledge, the pre-compiled bytecode files need to be
> created after the source files have been installed in their final
> location, because the file name and timestamp is encoded in the compiled
> file (it's sort of used as a cache file).  While this can be accomplished
> with a different patch, it wouldn't really work when DESTDIR is used
> because you'd create a "dead" cache file.  In a sense, this operation is
> like running ldconfig -- it's outside the scope of the build system.
> Package managers typically put it in the "post install" section.

Do you have a reference for this?  I tried looking for one but the only
thing I could find was http://www.python.org/doc/1.6/dist/built-dist.html
which suggests to me that they can be compiled before shipping which of
course certainly involves moving them.  In any case NetBSD does this
patch before building and everything works there.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.

  48. Re: Proposal: 7.2b2 today

    Peter Eisentraut <peter_e@gmx.net> — 2002-03-20T19:47:59Z

    D'Arcy J.M. Cain writes:

> Do you have a reference for this?  I tried looking for one but the only
> thing I could find was http://www.python.org/doc/1.6/dist/built-dist.html
> which suggests to me that they can be compiled before shipping which of
> course certainly involves moving them.  In any case NetBSD does this
> patch before building and everything works there.

My reference is Automake.  They go out of their way to compile the Python
files at the right time.  We could use this as a model.

It's easy to determine that the time stamp appears to be encoded into the
compiled output file:

$ cat test.py
print "test"
$ python -c 'import py_compile; py_compile.compile("test.py", "test.pyc")'
$ md5sum test.pyc
a0e690271636fcbf067db628f9c7d0c3 test.pyc
$ python -c 'import py_compile; py_compile.compile("test.py", "test.pyc")'
$ md5sum test.pyc
a0e690271636fcbf067db628f9c7d0c3 test.pyc
$ touch test.py
$ python -c 'import py_compile; py_compile.compile("test.py", "test.pyc")'
$ md5sum test.pyc
1d78ae79994b102c89a14a2dd2addc55  test.pyc

What you need to do is to create the compiled files after you have
installed the original.  Binary packaging probably preserves the time
stamps of the files, so that shouldn't be a problem.  I withdraw that part
of the objection.

Also, I think if we add this feature, let's just make it the default and
not add another configure option for it.

-- 
Peter Eisentraut   peter_e@gmx.net

  49. Re: initcap incompatibility issue

    nolan@celery.tssi.com — 2003-07-09T18:44:44Z

    > As far as I can tell, not capitalizing the first letter after a dash
> is the only inconsistency with Oracle's implementation of this function.

Wrong again.  Oracle also capitalizes the first letter after a comma, 
semicolon, colon, period, and both a single and double quote.  (And that's 
all I've tested so far.)

So, I guess I need to write a program to test all possible combinations
to see how incompatible the function is.

Making this change will be a larger patch than I had initially anticipated.

That also brings into question whether this is really a bugfix or a
specification change, a question which is relevant since we're in the 
feature freeze for 7.4.
--
Mike Nolan

  50. Re: initcap incompatibility issue

    scott.marlowe <scott.marlowe@ihs.com> — 2003-07-09T19:42:50Z

    On Wed, 9 Jul 2003 nolan@celery.tssi.com wrote:

> > As far as I can tell, not capitalizing the first letter after a dash
> > is the only inconsistency with Oracle's implementation of this function.
> 
> Wrong again.  Oracle also capitalizes the first letter after a comma, 
> semicolon, colon, period, and both a single and double quote.  (And that's 
> all I've tested so far.)
> 
> So, I guess I need to write a program to test all possible combinations
> to see how incompatible the function is.
> 
> Making this change will be a larger patch than I had initially anticipated.
> 
> That also brings into question whether this is really a bugfix or a
> specification change, a question which is relevant since we're in the 
> feature freeze for 7.4.

It sounds like Oracle is simply regexing for anything that ISN'T a letter 
to initcap right after it.  If that's the case, you could just regex too.

  51. Re: initcap incompatibility issue

    Tom Lane <tgl@sss.pgh.pa.us> — 2003-07-10T02:31:19Z

    "scott.marlowe" <scott.marlowe@ihs.com> writes:
> On Wed, 9 Jul 2003 nolan@celery.tssi.com wrote:
>> Wrong again.  Oracle also capitalizes the first letter after a comma, 
>> semicolon, colon, period, and both a single and double quote.  (And that's 
>> all I've tested so far.)

> It sounds like Oracle is simply regexing for anything that ISN'T a letter 
> to initcap right after it.  If that's the case, you could just regex too.

Or more likely, use the appropriate ctype.h function (isalpha, probably).

>> That also brings into question whether this is really a bugfix or a
>> specification change, a question which is relevant since we're in the 
>> feature freeze for 7.4.

AFAIK, our specification for this function is "be like Oracle", so it's
a bug fix and fair game for 7.4.  Of course, the sooner you get it in
the more likely we'll see it that way ;-).  Later in beta, only critical
bugfixes will be accepted, and this one surely ain't very critical.

			regards, tom lane

  52. Re: initcap incompatibility issue

    nolan@celery.tssi.com — 2003-07-11T06:34:48Z

    > > It sounds like Oracle is simply regexing for anything that ISN'T a letter 
> > to initcap right after it.  If that's the case, you could just regex too.
> 
> Or more likely, use the appropriate ctype.h function (isalpha, probably).

Having tested it, Oracle capitalizes after all non-alphanumeric characters,
so !isalnum() is the appropriate function.  (That makes it a one-line 
patch on 7.3.3, which I've already tested.)

> AFAIK, our specification for this function is "be like Oracle", so it's
> a bug fix and fair game for 7.4.  Of course, the sooner you get it in
> the more likely we'll see it that way ;-).  Later in beta, only critical
> bugfixes will be accepted, and this one surely ain't very critical.

Now if I can just get CVS working on Redhat 8 and remember how to build
a patch, even a one-liner. :-)
--
Mike Nolan