Horton

Re: [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@2ndquadrant.com>
Cc: Steve Singer <steve@ssinger.info>, pgsql-hackers@postgresql.org
Date: 2012-07-03T15:02:15Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Don't waste the last segment of each 4GB logical log file.

  2. Stamp HEAD as 9.3devel.

  3. Wake WALSender to reduce data loss at failover for async commit.

  4. Make the visibility map crash-safe.

On Thu, Jun 28, 2012 at 12:04 PM, Andres Freund <andres@2ndquadrant.com> wrote:
> On Thursday, June 28, 2012 06:01:10 PM Robert Haas wrote:
>> On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund <andres@2ndquadrant.com>
> wrote:
>> > It even can be significantly higher than max_connections because
>> > subtransactions are only recognizable as part of their parent transaction
>> > uppon commit.
>>
>> I've been wondering whether sub-XID assignment was going to end up on
>> the list of things that need to be WAL-logged to enable logical
>> replication.  It would be nicer to avoid that if we can, but I have a
>> feeling that we may not be able to.
> I don't think it needs to. We only need that information during commit and we
> have it there. If a subtxn aborts a separate abort is logged, so thats no
> problem. The 'merging' of the transactions would be slightly easier if we had
> the knowledge from the get go but that would add complications again in the
> case of rollbacks.
>
> What do you think we need it?

Well, I don't know for sure that we do.  But, ultimately, I think
we're going to find that applying the whole transaction at
transaction-end is something that we need to optimize - so that we can
apply the transaction as it happens and commit it at the end.  For
small transactions this is no big deal, but for big transactions it
could lead to hours and hours of replication delay.  I'm not sure
whether it's practical to fix that in version 1 - certainly there are
a lot of issues there - but ultimately I think it's something that our
users are going to demand.  And it seems like that might require
knowing the transaction tree at some point before the commit record
shows up.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company