Thread

gitmaster server problem?

Bruce Momjian <bruce@momjian.us> — 2024-08-17T16:41:19Z

About 20 minutes ago I starting getting gitmaster pull errors:

	$ git pull ssh://git@gitmaster.postgresql.org/postgresql.git
	ssh: connect to host gitmaster.postgresql.org port 22: Connection timed
	out
	fatal: Could not read from remote repository.
	
	Please make sure you have the correct access rights
	and the repository exists.

Public git seems fine:

	$ git pull https://git.postgresql.org/git/postgresql.git
	From https://git.postgresql.org/git/postgresql
	 * branch                    HEAD       -> FETCH_HEAD
	Already up to date.

Can anyone confirm or report on the cause?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: gitmaster server problem?

Tom Lane <tgl@sss.pgh.pa.us> — 2024-08-17T16:49:20Z

Bruce Momjian <bruce@momjian.us> writes:
> About 20 minutes ago I starting getting gitmaster pull errors:
> ...
> Can anyone confirm or report on the cause?

Yeah, for me it's hanging up at the connection establishment
stage ("ssh -fMN gitmaster.postgresql.org").  Traceroute gets
as far as some rackspace.net machines, but gemulon itself
isn't answering.

			regards, tom lane

Re: gitmaster server problem?

Tom Lane <tgl@sss.pgh.pa.us> — 2024-08-17T17:22:18Z

I wrote:
> Bruce Momjian <bruce@momjian.us> writes:
>> About 20 minutes ago I starting getting gitmaster pull errors:
>> Can anyone confirm or report on the cause?

> Yeah, for me it's hanging up at the connection establishment
> stage ("ssh -fMN gitmaster.postgresql.org").  Traceroute gets
> as far as some rackspace.net machines, but gemulon itself
> isn't answering.

ssh and git pull are working again now for me.

Oddly, "ping gemulon.postgresql.org" is still showing 98%+
packet drop rate.  Not sure if that's normal.

			regards, tom lane

Re: gitmaster server problem?

Matthias van de Meent <boekewurm+postgres@gmail.com> — 2024-08-17T17:59:03Z

On Sat, 17 Aug 2024 at 18:41, Bruce Momjian <bruce@momjian.us> wrote:
>
> About 20 minutes ago I starting getting gitmaster pull errors:
>
>         $ git pull ssh://git@gitmaster.postgresql.org/postgresql.git
>         ssh: connect to host gitmaster.postgresql.org port 22: Connection timed
>         out
>         fatal: Could not read from remote repository.
>
>         Please make sure you have the correct access rights
>         and the repository exists.
[...]
> Can anyone confirm or report on the cause?

Additional infra that doesn't seem to work right now: mailing list
archives [0] seem to fail at a 503 produced by Varnish Cache server:
Error 503 Backend fetch failed. Maybe more of infra is down, or
otherwise under maintenance?

Kind regards,

Matthias van de Meent

[0] https://www.postgresql.org/list/pgsql-hackers/2024-08/

Re: gitmaster server problem?

Tom Lane <tgl@sss.pgh.pa.us> — 2024-08-17T18:15:49Z

Matthias van de Meent <boekewurm+postgres@gmail.com> writes:
> Additional infra that doesn't seem to work right now: mailing list
> archives [0] seem to fail at a 503 produced by Varnish Cache server:
> Error 503 Backend fetch failed. Maybe more of infra is down, or
> otherwise under maintenance?

Yeah, I'm seeing that too; not at the top level of the site but
when you try to drill down to individual messages.

commitfest.postgresql.org isn't responding either,
nor wiki.postgresql.org.

			regards, tom lane

Re: gitmaster server problem?

Bruce Momjian <bruce@momjian.us> — 2024-08-17T18:18:11Z

On Sat, Aug 17, 2024 at 02:15:49PM -0400, Tom Lane wrote:
> Matthias van de Meent <boekewurm+postgres@gmail.com> writes:
> > Additional infra that doesn't seem to work right now: mailing list
> > archives [0] seem to fail at a 503 produced by Varnish Cache server:
> > Error 503 Backend fetch failed. Maybe more of infra is down, or
> > otherwise under maintenance?
> 
> Yeah, I'm seeing that too; not at the top level of the site but
> when you try to drill down to individual messages.
> 
> commitfest.postgresql.org isn't responding either,
> nor wiki.postgresql.org.

Okay, thanks for the confirmation.  I emailed pginfra at the same time I
posted the first email, so they are aware.  I have not received a reply
from them yet.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: gitmaster server problem?

Adrian Klaver <adrian.klaver@aklaver.com> — 2024-08-17T18:22:29Z

On 8/17/24 10:22, Tom Lane wrote:
> I wrote:
>> Bruce Momjian <bruce@momjian.us> writes:
>>> About 20 minutes ago I starting getting gitmaster pull errors:
>>> Can anyone confirm or report on the cause?
> 
>> Yeah, for me it's hanging up at the connection establishment
>> stage ("ssh -fMN gitmaster.postgresql.org").  Traceroute gets
>> as far as some rackspace.net machines, but gemulon itself
>> isn't answering.
> 
> ssh and git pull are working again now for me.
> 
> Oddly, "ping gemulon.postgresql.org" is still showing 98%+
> packet drop rate.  Not sure if that's normal.

I don't even get that, it is still dying at:

9  cored-aggr181a-1.dfw3.rackspace.net (148.62.65.5)

> 
> 			regards, tom lane
> 
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: gitmaster server problem?

Bruce Momjian <bruce@momjian.us> — 2024-08-17T18:24:54Z

On Sat, Aug 17, 2024 at 01:22:18PM -0400, Tom Lane wrote:
> I wrote:
> > Bruce Momjian <bruce@momjian.us> writes:
> >> About 20 minutes ago I starting getting gitmaster pull errors:
> >> Can anyone confirm or report on the cause?
> 
> > Yeah, for me it's hanging up at the connection establishment
> > stage ("ssh -fMN gitmaster.postgresql.org").  Traceroute gets
> > as far as some rackspace.net machines, but gemulon itself
> > isn't answering.
> 
> ssh and git pull are working again now for me.

ssh and git pull are still failing for me to gitmaster.postgresql.org.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: gitmaster server problem?

Bruce Momjian <bruce@momjian.us> — 2024-08-17T18:53:44Z

On Sat, Aug 17, 2024 at 02:24:54PM -0400, Bruce Momjian wrote:
> On Sat, Aug 17, 2024 at 01:22:18PM -0400, Tom Lane wrote:
> > I wrote:
> > > Bruce Momjian <bruce@momjian.us> writes:
> > >> About 20 minutes ago I starting getting gitmaster pull errors:
> > >> Can anyone confirm or report on the cause?
> > 
> > > Yeah, for me it's hanging up at the connection establishment
> > > stage ("ssh -fMN gitmaster.postgresql.org").  Traceroute gets
> > > as far as some rackspace.net machines, but gemulon itself
> > > isn't answering.
> > 
> > ssh and git pull are working again now for me.
> 
> ssh and git pull are still failing for me to gitmaster.postgresql.org.

Joe Conway, CC'ed, is aware of the issue and is now working on it.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: gitmaster server problem?

Bruce Momjian <bruce@momjian.us> — 2024-08-17T19:04:31Z

On Sat, Aug 17, 2024 at 02:53:44PM -0400, Bruce Momjian wrote:
> On Sat, Aug 17, 2024 at 02:24:54PM -0400, Bruce Momjian wrote:
> > On Sat, Aug 17, 2024 at 01:22:18PM -0400, Tom Lane wrote:
> > > I wrote:
> > > > Bruce Momjian <bruce@momjian.us> writes:
> > > >> About 20 minutes ago I starting getting gitmaster pull errors:
> > > >> Can anyone confirm or report on the cause?
> > > 
> > > > Yeah, for me it's hanging up at the connection establishment
> > > > stage ("ssh -fMN gitmaster.postgresql.org").  Traceroute gets
> > > > as far as some rackspace.net machines, but gemulon itself
> > > > isn't answering.
> > > 
> > > ssh and git pull are working again now for me.
> > 
> > ssh and git pull are still failing for me to gitmaster.postgresql.org.
> 
> Joe Conway, CC'ed, is aware of the issue and is now working on it.

Joe reports that everything should be working normally now.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: gitmaster server problem?

Tom Lane <tgl@sss.pgh.pa.us> — 2024-08-17T19:27:37Z

Bruce Momjian <bruce@momjian.us> writes:
> Joe reports that everything should be working normally now.

ACK - things look good from here.  (I did have to restart my
ssh tunnel to gitmaster again.)

Thanks Joe!

			regards, tom lane

Re: gitmaster server problem?

Joe Conway <mail@joeconway.com> — 2024-08-17T20:05:30Z

On 8/17/24 13:59, Matthias van de Meent wrote:
> On Sat, 17 Aug 2024 at 18:41, Bruce Momjian <bruce@momjian.us> wrote:
>>
>> About 20 minutes ago I starting getting gitmaster pull errors:
>>
>>         $ git pull ssh://git@gitmaster.postgresql.org/postgresql.git
>>         ssh: connect to host gitmaster.postgresql.org port 22: Connection timed
>>         out
>>         fatal: Could not read from remote repository.
>>
>>         Please make sure you have the correct access rights
>>         and the repository exists.
> [...]
>> Can anyone confirm or report on the cause?
> 
> Additional infra that doesn't seem to work right now: mailing list
> archives [0] seem to fail at a 503 produced by Varnish Cache server:
> Error 503 Backend fetch failed. Maybe more of infra is down, or
> otherwise under maintenance?
> 
> Kind regards,
> 
> Matthias van de Meent
> 
> [0] https://www.postgresql.org/list/pgsql-hackers/2024-08/


Just to close this out -- problem found and fixed about an hour ago. 
Sorry for the interruption.

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Re: gitmaster server problem?

Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-08-19T09:04:19Z

On Sat, 17 Aug 2024 at 22:05, Joe Conway <mail@joeconway.com> wrote:
> Just to close this out -- problem found and fixed about an hour ago.
> Sorry for the interruption.

Whatever the problem was it seems to have returned

Re: gitmaster server problem?

Joe Conway <mail@joeconway.com> — 2024-08-19T11:39:27Z

On 8/19/24 05:04, Jelte Fennema-Nio wrote:
> On Sat, 17 Aug 2024 at 22:05, Joe Conway <mail@joeconway.com> wrote:
>> Just to close this out -- problem found and fixed about an hour ago.
>> Sorry for the interruption.
> 
> Whatever the problem was it seems to have returned


What specifically? I am not seeing anything at the moment.


-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Re: gitmaster server problem?

Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-08-19T11:57:07Z

On Mon, 19 Aug 2024 at 13:39, Joe Conway <mail@joeconway.com> wrote:
> What specifically? I am not seeing anything at the moment.

It seems to be working again fine. But before
https://git.postgresql.org was throwing 502 and 503 errors. Depending
on the request either by Varnish or nginx.

Re: gitmaster server problem?

Robert Haas <robertmhaas@gmail.com> — 2024-08-19T12:04:11Z

On Mon, Aug 19, 2024 at 7:57 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> On Mon, 19 Aug 2024 at 13:39, Joe Conway <mail@joeconway.com> wrote:
> > What specifically? I am not seeing anything at the moment.
>
> It seems to be working again fine. But before
> https://git.postgresql.org was throwing 502 and 503 errors. Depending
> on the request either by Varnish or nginx.

I'm still unable to access any https://git.postgresql.org/ URL.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Re: gitmaster server problem?

Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-08-19T12:05:36Z

On Mon, 19 Aug 2024 at 14:04, Robert Haas <robertmhaas@gmail.com> wrote:
> I'm still unable to access any https://git.postgresql.org/ URL.

Yeah it's broken for me again too.

Re: gitmaster server problem?

Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-08-19T12:06:22Z

On Mon, 19 Aug 2024 at 14:05, Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
>
> On Mon, 19 Aug 2024 at 14:04, Robert Haas <robertmhaas@gmail.com> wrote:
> > I'm still unable to access any https://git.postgresql.org/ URL.
>
> Yeah it's broken for me again too.

In case the actual error is helpful (although I doubt it):

Error 503 Backend fetch failed

Backend fetch failed

Guru Meditation:

XID: 1030914118

________________________________

Varnish cache server

Re: gitmaster server problem?

Joe Conway <mail@joeconway.com> — 2024-08-19T12:11:36Z

On 8/19/24 08:06, Jelte Fennema-Nio wrote:
> On Mon, 19 Aug 2024 at 14:05, Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
>>
>> On Mon, 19 Aug 2024 at 14:04, Robert Haas <robertmhaas@gmail.com> wrote:
>> > I'm still unable to access any https://git.postgresql.org/ URL.
>>
>> Yeah it's broken for me again too.
> 
> In case the actual error is helpful (although I doubt it):
> 
> Error 503 Backend fetch failed
> 
> Backend fetch failed
> 
> Guru Meditation:
> 
> XID: 1030914118


I am not seeing any errors here. nagios monitoring is showing an ipv6 
error though. Not the same issue as yesterday.

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Re: gitmaster server problem?

jian he <jian.universality@gmail.com> — 2024-08-19T14:03:28Z

On Mon, Aug 19, 2024 at 9:56 PM Bruce Momjian <bruce@momjian.us> wrote:
>

as of now.
cgit working
https://git.postgresql.org/cgit/postgresql.git/


git not working
( 403 Forbidden nginx)
https://git.postgresql.org/git/postgresql.git/

Re: gitmaster server problem?

Dave Page <dpage@pgadmin.org> — 2024-08-19T14:07:19Z

On Mon, 19 Aug 2024 at 14:56, Bruce Momjian <bruce@momjian.us> wrote:

> On Sat, Aug 17, 2024 at 02:15:49PM -0400, Tom Lane wrote:
> > Matthias van de Meent <boekewurm+postgres@gmail.com> writes:
> > > Additional infra that doesn't seem to work right now: mailing list
> > > archives [0] seem to fail at a 503 produced by Varnish Cache server:
> > > Error 503 Backend fetch failed. Maybe more of infra is down, or
> > > otherwise under maintenance?
> >
> > Yeah, I'm seeing that too; not at the top level of the site but
> > when you try to drill down to individual messages.
> >
> > commitfest.postgresql.org isn't responding either,
> > nor wiki.postgresql.org.
>
> Okay, thanks for the confirmation.  I emailed pginfra at the same time I
> posted the first email, so they are aware.  I have not received a reply
> from them yet.
>

Joe replied.

I've just checked again myself and I don't see any problems with the sites
mentioned from a user perspective, but there are some non-obvious issues
being reported by Nagios on git.postgresql.org (which is not the same as
gitmaster). We'll dig in some more.

-- 
Dave Page
pgAdmin: https://www.pgadmin.org
PostgreSQL: https://www.postgresql.org
EDB: https://www.enterprisedb.com

PGDay UK 2024, 11th September, London: https://2024.pgday.uk/

Re: gitmaster server problem?

Magnus Hagander <magnus@hagander.net> — 2024-08-19T14:19:05Z

On Mon, Aug 19, 2024 at 3:56 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Matthias van de Meent <boekewurm+postgres@gmail.com> writes:
> > Additional infra that doesn't seem to work right now: mailing list
> > archives [0] seem to fail at a 503 produced by Varnish Cache server:
> > Error 503 Backend fetch failed. Maybe more of infra is down, or
> > otherwise under maintenance?
>
> Yeah, I'm seeing that too; not at the top level of the site but
> when you try to drill down to individual messages.
>
> commitfest.postgresql.org isn't responding either,
> nor wiki.postgresql.org.
>

Are you still seeing these issues? It works fine both from here and from
our monitoring, but there could be something source-network-specific
maybe....

The git.postgresql.org one specifically looks a lot like a DOS. It may not
be an intentional DOS, but the git server side is very inefficient and can
rapidly turn valid attempts into DOSes if they parallellize too much...

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

Re: gitmaster server problem?

Bruce Momjian <bruce@momjian.us> — 2024-08-19T14:24:32Z

On Mon, Aug 19, 2024 at 08:11:36AM -0400, Joe Conway wrote:
> On 8/19/24 08:06, Jelte Fennema-Nio wrote:
> > On Mon, 19 Aug 2024 at 14:05, Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> > > 
> > > On Mon, 19 Aug 2024 at 14:04, Robert Haas <robertmhaas@gmail.com> wrote:
> > > > I'm still unable to access any https://git.postgresql.org/ URL.
> > > 
> > > Yeah it's broken for me again too.
> > 
> > In case the actual error is helpful (although I doubt it):
> > 
> > Error 503 Backend fetch failed
> > 
> > Backend fetch failed
> > 
> > Guru Meditation:
> > 
> > XID: 1030914118
> 
> I am not seeing any errors here. nagios monitoring is showing an ipv6 error
> though. Not the same issue as yesterday.

gitmaster.postgresql.org is working fine for me right now.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: gitmaster server problem?

Tom Lane <tgl@sss.pgh.pa.us> — 2024-08-19T14:27:42Z

Magnus Hagander <magnus@hagander.net> writes:
> On Mon, Aug 19, 2024 at 3:56 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Yeah, I'm seeing that too; not at the top level of the site but
>> when you try to drill down to individual messages.
>> commitfest.postgresql.org isn't responding either,
>> nor wiki.postgresql.org.

> Are you still seeing these issues? It works fine both from here and from
> our monitoring, but there could be something source-network-specific
> maybe....

Right at the moment,

https://git.postgresql.org/gitweb/?p=postgresql.git

is failing for me with "Backend fetch failed".  However, the mail
archives, commitfest.p.o, and wiki.p.o seem to be responding normally,
as is gitmaster.

			regards, tom lane

Re: gitmaster server problem?

Dave Page <dpage@pgadmin.org> — 2024-08-19T14:30:33Z

On Mon, 19 Aug 2024 at 15:27, jian he <jian.universality@gmail.com> wrote:

> On Mon, Aug 19, 2024 at 9:56 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
>
> as of now.
> cgit working
> https://git.postgresql.org/cgit/postgresql.git/


Yes.


>
>
>
> git not working
> ( 403 Forbidden nginx)
> https://git.postgresql.org/git/postgresql.git/


From a browser, that's what I'd expect. You should be able to clone from it
though:

git clone https://git.postgresql.org/git/postgresql.git
Cloning into 'postgresql'...
remote: Enumerating objects: 42814, done.
remote: Counting objects: 100% (42814/42814), done.
remote: Compressing objects: 100% (15585/15585), done.
remote: Total 1014138 (delta 34190), reused 33980 (delta 26978),
pack-reused 971324
Receiving objects: 100% (1014138/1014138), 343.01 MiB | 15.49 MiB/s, done.
Resolving deltas: 100% (873517/873517), done.

-- 
Dave Page
pgAdmin: https://www.pgadmin.org
PostgreSQL: https://www.postgresql.org
EDB: https://www.enterprisedb.com

PGDay UK 2024, 11th September, London: https://2024.pgday.uk/

Re: gitmaster server problem?

Dave Page <dpage@pgadmin.org> — 2024-08-19T14:32:15Z

On Mon, 19 Aug 2024 at 15:29, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Magnus Hagander <magnus@hagander.net> writes:
> > On Mon, Aug 19, 2024 at 3:56 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Yeah, I'm seeing that too; not at the top level of the site but
> >> when you try to drill down to individual messages.
> >> commitfest.postgresql.org isn't responding either,
> >> nor wiki.postgresql.org.
>
> > Are you still seeing these issues? It works fine both from here and from
> > our monitoring, but there could be something source-network-specific
> > maybe....
>
> Right at the moment,
>
> https://git.postgresql.org/gitweb/?p=postgresql.git
>
> is failing for me with "Backend fetch failed".


That's working fine for me now, however Magnus did just give the box some
more CPUs.

We're also considering rate limiting by IP on that server, as it seems to
be getting hammered with requests from China.

-- 
Dave Page
pgAdmin: https://www.pgadmin.org
PostgreSQL: https://www.postgresql.org
EDB: https://www.enterprisedb.com

PGDay UK 2024, 11th September, London: https://2024.pgday.uk/

Re: gitmaster server problem?

Magnus Hagander <magnus@hagander.net> — 2024-08-19T14:39:10Z

On Mon, Aug 19, 2024 at 4:28 PM Dave Page <dpage@pgadmin.org> wrote:

>
>
> On Mon, 19 Aug 2024 at 14:56, Bruce Momjian <bruce@momjian.us> wrote:
>
>> On Sat, Aug 17, 2024 at 02:15:49PM -0400, Tom Lane wrote:
>> > Matthias van de Meent <boekewurm+postgres@gmail.com> writes:
>> > > Additional infra that doesn't seem to work right now: mailing list
>> > > archives [0] seem to fail at a 503 produced by Varnish Cache server:
>> > > Error 503 Backend fetch failed. Maybe more of infra is down, or
>> > > otherwise under maintenance?
>> >
>> > Yeah, I'm seeing that too; not at the top level of the site but
>> > when you try to drill down to individual messages.
>> >
>> > commitfest.postgresql.org isn't responding either,
>> > nor wiki.postgresql.org.
>>
>> Okay, thanks for the confirmation.  I emailed pginfra at the same time I
>> posted the first email, so they are aware.  I have not received a reply
>> from them yet.
>>
>
> Joe replied.
>
> In the context of replying it's also worth noting that CCing the same
email to both -hackers and -www ensures it gets caught in moderation, and
will take longer to reach someone.

/Magnus

Re: gitmaster server problem?

Adrian Klaver <adrian.klaver@aklaver.com> — 2024-08-19T14:39:27Z

On 8/19/24 07:27, Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:

> Right at the moment,
> 
> https://git.postgresql.org/gitweb/?p=postgresql.git
> 
> is failing for me with "Backend fetch failed".  However, the mail

I just tried it and it worked.

> archives, commitfest.p.o, and wiki.p.o seem to be responding normally,
> as is gitmaster.
> 
> 			regards, tom lane
> 
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: gitmaster server problem?

Tom Lane <tgl@sss.pgh.pa.us> — 2024-08-19T15:51:55Z

Magnus Hagander <magnus@hagander.net> writes:
> In the context of replying it's also worth noting that CCing the same
> email to both -hackers and -www ensures it gets caught in moderation, and
> will take longer to reach someone.

Yeah, I think the double CC on this thread is my fault --- I
momentarily forgot about that rule.

			regards, tom lane