Horton

Thread

  1. Schedule for migration to pglister

    Stephen Frost <sfrost@snowman.net> — 2017-11-06T15:36:38Z

    Greetings,

The PostgreSQL Infrastructure team is working to migrate the project's
mailing lists from the existing system (an ancient and unmaintained
piece of software called "majordomo2") to a newly developed mailing list
system (known as "PGLister"), which better addresses the needs of the
PostgreSQL community and is updated to work with recent improvements in
email technology and spam filtering. These changes will impact certain
aspects of the system but we are hopeful that these changes will have a
minimal impact on users, although everyone will notice the differences.

The changes which we expect to be most significant to users can be found
on the wiki here: https://wiki.postgresql.org/wiki/PGLister_Announce

Our planned migration schedule is as follows:

Nov 6 -
  pgsql-www

Nov 13 -
  pgsql-hackers
  pgsql-bugs
  pgsql-committers

Nov 20 -
  pgsql-admin
  pgsql-general
  pgsql-sql
  pgsql-jobs
  pgsql-novice

Nov 27 -
  pgsql-announce

After -
  the rest

We will be starting the migration of pgsql-www shortly.

Each list will receive an email with a link to the wiki about the
migration after the list has been migrated.

Thanks!

Stephen

  2. Re: Schedule for migration to pglister

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-11-06T15:40:16Z

    Stephen Frost <sfrost@snowman.net> writes:
> Each list will receive an email with a link to the wiki about the
> migration after the list has been migrated.

I suggest doing that the other way 'round.  Otherwise, the email
about the change will inevitably go into a lot of peoples' bit
buckets if they haven't adjusted their mail filters yet.

			regards, tom lane

  3. Re: [HACKERS] Schedule for migration to pglister

    Magnus Hagander <magnus@hagander.net> — 2017-11-06T15:42:49Z

    On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Stephen Frost <sfrost@snowman.net> writes:
> > Each list will receive an email with a link to the wiki about the
> > migration after the list has been migrated.
>
> I suggest doing that the other way 'round.  Otherwise, the email
> about the change will inevitably go into a lot of peoples' bit
> buckets if they haven't adjusted their mail filters yet.
>

The argument for doing it after the migration is that the complaints that
we have received so far have all been from people where email ends up in
the *inbox* after the migration, not the bitbucket. That's the default
action in most peoples MUAs when their rules no longer match...

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  4. Re: Schedule for migration to pglister

    Stephen Frost <sfrost@snowman.net> — 2017-11-06T15:43:15Z

    Tom,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > Each list will receive an email with a link to the wiki about the
> > migration after the list has been migrated.
> 
> I suggest doing that the other way 'round.  Otherwise, the email
> about the change will inevitably go into a lot of peoples' bit
> buckets if they haven't adjusted their mail filters yet.

My thought had been to do before-and-after, but I got complaints from
others that we'd then be spamming a lot of people with email.

We definitely need one after the migration because the new mail *won't*
end up caught in people's filters, and for those who intentionally
filter the list traffic into the garbage because they couldn't figure
out how to unsubscribe, this is going to be most annoying (this is what
we saw with the pgadmin lists and it was quite painful).

Thanks!

Stephen

  5. Re: [HACKERS] Schedule for migration to pglister

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-11-06T15:46:24Z

    Magnus Hagander <magnus@hagander.net> writes:
> On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I suggest doing that the other way 'round.  Otherwise, the email
>> about the change will inevitably go into a lot of peoples' bit
>> buckets if they haven't adjusted their mail filters yet.

> The argument for doing it after the migration is that the complaints that
> we have received so far have all been from people where email ends up in
> the *inbox* after the migration, not the bitbucket. That's the default
> action in most peoples MUAs when their rules no longer match...

Hm, around here it's no match -> spam bucket.  But in any case, why
would you not want to send it before so that it would end up where
they're accustomed to seeing the list's traffic?

			regards, tom lane

  6. Re: [HACKERS] Schedule for migration to pglister

    Magnus Hagander <magnus@hagander.net> — 2017-11-06T15:51:45Z

    On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Magnus Hagander <magnus@hagander.net> writes:
> > On Mon, Nov 6, 2017 at 4:40 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> I suggest doing that the other way 'round.  Otherwise, the email
> >> about the change will inevitably go into a lot of peoples' bit
> >> buckets if they haven't adjusted their mail filters yet.
>
> > The argument for doing it after the migration is that the complaints that
> > we have received so far have all been from people where email ends up in
> > the *inbox* after the migration, not the bitbucket. That's the default
> > action in most peoples MUAs when their rules no longer match...
>
> Hm, around here it's no match -> spam bucket.  But in any case, why
>

I think you're quite uncommon in that setup. For obvious reasons, but I've
never heard of anybody other than you doing that :)



> would you not want to send it before so that it would end up where
> they're accustomed to seeing the list's traffic?
>

The experience from the pgadmin lists is that a lot of people have the
lists filtered into folders that they don't check often (or at all). So
they don't notice the migraiton message. But they start noticing once all
the list mail shows up in their inbox instead.

It might well be that we end up getting the other half of people when we do
it this order, but we definitely at a *lot* of people in that first bucket.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  7. Re: [HACKERS] [pgsql-www] Schedule for migration to pglister

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-11-06T16:00:20Z

    Magnus Hagander <magnus@hagander.net> writes:
> On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Hm, around here it's no match -> spam bucket.  But in any case, why

> I think you're quite uncommon in that setup.

Interesting, because "it's not addressed to me (or any list I'm on)"
is the best single spam filtering rule I know, and has been for a
decade or two.

But we veer far off topic here.  Do as you will.

			regards, tom lane

  8. Re: [pgsql-www] Schedule for migration to pglister

    Magnus Hagander <magnus@hagander.net> — 2017-11-06T16:07:19Z

    On Mon, Nov 6, 2017 at 5:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Magnus Hagander <magnus@hagander.net> writes:
> > On Mon, Nov 6, 2017 at 4:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Hm, around here it's no match -> spam bucket.  But in any case, why
>
> > I think you're quite uncommon in that setup.
>
> Interesting, because "it's not addressed to me (or any list I'm on)"
> is the best single spam filtering rule I know, and has been for a
> decade or two.
>

Oh, I think I misunderstood you. I thought you meant "any list tagged email
that's for a list I don't know what it is". As in "list-id exists but is
unknown".

The way you explain there makes a lot more sense. I think not many people
do that either, mainly since gmail/yahoo/whatnot doesn't make it very easy
to do that. But it does make a lot more sense that way.


-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  9. Re: Schedule for migration to pglister

    Noah Misch <noah@leadboat.com> — 2017-11-15T09:01:31Z

    On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> Our planned migration schedule is as follows:
> 
> Nov 6 -
>   pgsql-www
> 
> Nov 13 -
>   pgsql-hackers

When each list migrated, its mbox archives stopped receiving new messages:

https://www.postgresql.org/list/pgsql-www/mbox/pgsql-www.201711
https://www.postgresql.org/list/pgsql-hackers/mbox/pgsql-hackers.201711

  10. Re: Schedule for migration to pglister

    Stephen Frost <sfrost@snowman.net> — 2017-11-15T14:18:31Z

    Noah,

* Noah Misch (noah@leadboat.com) wrote:
> On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> > Our planned migration schedule is as follows:
> > 
> > Nov 6 -
> >   pgsql-www
> > 
> > Nov 13 -
> >   pgsql-hackers
> 
> When each list migrated, its mbox archives stopped receiving new messages:

This was actually anticipated, though we had been thinking that the
migration would be faster and so it wouldn't end up being such a long
time that the mbox's wouldn't get updated.

The plan is to replace those mbox's (which are created by mj2 and then
rsync'd over currently) with ones generated from the archives database,
but that's something which will basically happen to the entire site and
all the lists at once.

Further, there will actually be some differences between the
archive-generated mbox's vs. those that came from mj2; in particular,
the 'date' used by mj2 is the 'received' date (from what I can tell)
while the 'date' in the archives database is the 'Date:' header from
when the message is sent by the user.

That said, this will make the mbox's match the actual webpages, which
they don't currently because the webpages are built from the archive
database while the mbox's are from mj2.  In addition, while trying to
cross-compare the two, I think I'm seeing some cases where mj2 is just
outright dropping messages also (for example, I don't see

https://www.postgresql.org/message-id/20170930224424.ud5ilchmclbl5y5n%40alap3.anarazel.de

anywhere in the mj2 mboxes..).

I'm chatting w/ Magnus about this now and I'm not sure exactly when
we'll end up making the change, but my feeling is that we should either
do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
to me like we should delay it any longer than that.

Thanks!

Stephen

  11. Re: Schedule for migration to pglister

    Noah Misch <noah@leadboat.com> — 2017-11-15T16:05:50Z

    On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> * Noah Misch (noah@leadboat.com) wrote:
> > When each list migrated, its mbox archives stopped receiving new messages:
> 
> This was actually anticipated, though we had been thinking that the
> migration would be faster and so it wouldn't end up being such a long
> time that the mbox's wouldn't get updated.
> 
> The plan is to replace those mbox's (which are created by mj2 and then
> rsync'd over currently) with ones generated from the archives database,
> but that's something which will basically happen to the entire site and
> all the lists at once.

> I'm chatting w/ Magnus about this now and I'm not sure exactly when
> we'll end up making the change, but my feeling is that we should either
> do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
> to me like we should delay it any longer than that.

Gotcha.  I, for one, can wait awhile.

  12. Re: Schedule for migration to pglister

    Magnus Hagander <magnus@hagander.net> — 2017-11-16T15:20:48Z

    On Wed, Nov 15, 2017 at 5:05 PM, Noah Misch <noah@leadboat.com> wrote:

> On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> > * Noah Misch (noah@leadboat.com) wrote:
> > > When each list migrated, its mbox archives stopped receiving new
> messages:
> >
> > This was actually anticipated, though we had been thinking that the
> > migration would be faster and so it wouldn't end up being such a long
> > time that the mbox's wouldn't get updated.
> >
> > The plan is to replace those mbox's (which are created by mj2 and then
> > rsync'd over currently) with ones generated from the archives database,
> > but that's something which will basically happen to the entire site and
> > all the lists at once.
>
> > I'm chatting w/ Magnus about this now and I'm not sure exactly when
> > we'll end up making the change, but my feeling is that we should either
> > do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
> > to me like we should delay it any longer than that.
>
> Gotcha.  I, for one, can wait awhile.
>

mbox access should now be restored, and as Stephen says, now handled by the
Date in the message. Let me know if there are any issues with the new ones.
(They're now generated by the same code as the per-thread ones).

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

  13. Re: Schedule for migration to pglister

    Stephen Frost <sfrost@snowman.net> — 2017-11-16T15:34:06Z

    Magnus, all,

* Magnus Hagander (magnus@hagander.net) wrote:
> mbox access should now be restored, and as Stephen says, now handled by the
> Date in the message. Let me know if there are any issues with the new ones.
> (They're now generated by the same code as the per-thread ones).

And just fyi, that also means that complete threads end up in each mbox,
so you'll see cases where a message was posted to, eg, -committers
initially but the reply cc'd hackers, bringing the thread over there,
and now the initial email to -committers and all of the subsequent
emails on the thread will be included in the mbox for both -hackers and
-committers.

Hopefully everyone finds that to actually provide a better experience,
but it's a difference to be aware of.

Thanks!

Stephen

  14. Re: Schedule for migration to pglister

    Justin Clift <justin@postgresql.org> — 2017-11-16T15:51:41Z

    On 2017-11-16 15:34, Stephen Frost wrote:
> Magnus, all,
> 
> * Magnus Hagander (magnus@hagander.net) wrote:
>> mbox access should now be restored, and as Stephen says, now handled 
>> by the
>> Date in the message. Let me know if there are any issues with the new 
>> ones.
>> (They're now generated by the same code as the per-thread ones).
> 
> And just fyi, that also means that complete threads end up in each 
> mbox,
> so you'll see cases where a message was posted to, eg, -committers
> initially but the reply cc'd hackers, bringing the thread over there,
> and now the initial email to -committers and all of the subsequent
> emails on the thread will be included in the mbox for both -hackers and
> -committers.
> 
> Hopefully everyone finds that to actually provide a better experience,
> but it's a difference to be aware of.

In theory :), that sounds better for people reading the web archives.

Only one real way to find out "for sure" though, so fingers crossed. :)

+ Justin

  15. Re: Schedule for migration to pglister

    Magnus Hagander <magnus@hagander.net> — 2017-11-16T21:37:21Z

    On Thu, Nov 16, 2017 at 4:51 PM, Justin Clift <justin@postgresql.org> wrote:

> On 2017-11-16 15:34, Stephen Frost wrote:
>
>> Magnus, all,
>>
>> * Magnus Hagander (magnus@hagander.net) wrote:
>>
>>> mbox access should now be restored, and as Stephen says, now handled by
>>> the
>>> Date in the message. Let me know if there are any issues with the new
>>> ones.
>>> (They're now generated by the same code as the per-thread ones).
>>>
>>
>> And just fyi, that also means that complete threads end up in each mbox,
>> so you'll see cases where a message was posted to, eg, -committers
>> initially but the reply cc'd hackers, bringing the thread over there,
>> and now the initial email to -committers and all of the subsequent
>> emails on the thread will be included in the mbox for both -hackers and
>> -committers.
>>
>> Hopefully everyone finds that to actually provide a better experience,
>> but it's a difference to be aware of.
>>
>
> In theory :), that sounds better for people reading the web archives.
>

Um. Nothing has changed in the web interface, only the mbox downloads.

-- 
 Magnus Hagander
 Me: https://www.hagander.net/ <http://www.hagander.net/>
 Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>