Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add new replication mode synchronous_commit = 'write'.

  1. pgsql: Add new replication mode synchronous_commit = 'write'.

    Simon Riggs <simon@2ndquadrant.com> — 2012-01-24T20:22:57Z

    Add new replication mode synchronous_commit = 'write'.
Replication occurs only to memory on standby, not to disk,
so provides additional performance if user wishes to
reduce durability level slightly. Adds concept of multiple
independent sync rep queues.

Fujii Masao and Simon Riggs

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/443b4821f1649bc617c5ce1f6f3ffc65842a8930

Modified Files
--------------
doc/src/sgml/config.sgml                    |   18 +++-
doc/src/sgml/high-availability.sgml         |   16 +++-
src/backend/replication/syncrep.c           |  112 ++++++++++++++++++---------
src/backend/replication/walsender.c         |    3 +-
src/backend/utils/misc/guc.c                |    5 +-
src/include/access/xact.h                   |    1 +
src/include/replication/syncrep.h           |   13 +++-
src/include/replication/walsender_private.h |    8 +-
8 files changed, 124 insertions(+), 52 deletions(-)

  2. Re: pgsql: Add new replication mode synchronous_commit = 'write'.

    Jaime Casanova <jaime@2ndquadrant.com> — 2012-01-24T20:35:22Z

    On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> Add new replication mode synchronous_commit = 'write'.
> Replication occurs only to memory on standby, not to disk,
> so provides additional performance if user wishes to
> reduce durability level slightly. Adds concept of multiple
> independent sync rep queues.
>
> Fujii Masao and Simon Riggs
>

i guess, you should add the new value in postgresql.conf too.
just a question, why not add a flush value and make it behave like on?
actually sync rep is on in both cases and having the different names
makes more easy to unserstand what is happening.
i only want to keep on for compatibility but i wouldn't mind if we
remove it in favor of more descriptive names.

or there's any difference between on and flush that i'm failing to see?

-- 
Jaime Casanova         www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitación

  3. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Fujii Masao <masao.fujii@gmail.com> — 2012-01-25T06:23:40Z

    On Wed, Jan 25, 2012 at 5:35 AM, Jaime Casanova <jaime@2ndquadrant.com> wrote:
> On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>> Add new replication mode synchronous_commit = 'write'.
>> Replication occurs only to memory on standby, not to disk,
>> so provides additional performance if user wishes to
>> reduce durability level slightly. Adds concept of multiple
>> independent sync rep queues.
>>
>> Fujii Masao and Simon Riggs
>>
>
> i guess, you should add the new value in postgresql.conf too.

Yes, I forgot to do that. Patch attached.

> just a question, why not add a flush value and make it behave like on?
> actually sync rep is on in both cases and having the different names
> makes more easy to unserstand what is happening.
> i only want to keep on for compatibility but i wouldn't mind if we
> remove it in favor of more descriptive names.

Also we should add "async" as an alias for "off"?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

  4. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Robert Haas <robertmhaas@gmail.com> — 2012-01-25T13:57:49Z

    On Wed, Jan 25, 2012 at 1:23 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Wed, Jan 25, 2012 at 5:35 AM, Jaime Casanova <jaime@2ndquadrant.com> wrote:
>> On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>>> Add new replication mode synchronous_commit = 'write'.
>>> Replication occurs only to memory on standby, not to disk,
>>> so provides additional performance if user wishes to
>>> reduce durability level slightly. Adds concept of multiple
>>> independent sync rep queues.
>>>
>>> Fujii Masao and Simon Riggs
>>>
>>
>> i guess, you should add the new value in postgresql.conf too.
>
> Yes, I forgot to do that. Patch attached.

I think that this would be a lot more clear if we described this as
synchronous_commit = remote_write rather than just synchronous_commit
= write.  Actually, the internal constant is named that way already,
but it's not exposed as such to the user.

There's a logical hierarchy here:

fully async commit ("off") < local flush only ("local") < local flush
+ remote write (currently "write") < local flush + remote flush
(currently "on") < local flush + remote apply

All of the levels except for "off" involve waiting for local flush;
the higher levels also involve waiting for something on the remote
side.  But the name of the variable is just synchronous_commit, so I
thik that if the word "remote" doesn't appear anywhere in the
user-visible parameter name, it's not as clear as it could be.  In
addition to renaming "write" to "remote_write", I think we might also
want to add "remote_flush" as an alias for "on".

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  5. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Simon Riggs <simon@2ndquadrant.com> — 2012-01-25T14:12:54Z

    On Wed, Jan 25, 2012 at 1:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:

> I think that this would be a lot more clear if we described this as
> synchronous_commit = remote_write rather than just synchronous_commit
> = write.  Actually, the internal constant is named that way already,
> but it's not exposed as such to the user.

That's something to discuss at the end of the CF when people are less
busy and we get more input.

It's an easy change whatever we decide to do.

-- 
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

  6. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Fujii Masao <masao.fujii@gmail.com> — 2012-01-26T06:21:47Z

    On Wed, Jan 25, 2012 at 11:12 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> On Wed, Jan 25, 2012 at 1:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>
>> I think that this would be a lot more clear if we described this as
>> synchronous_commit = remote_write rather than just synchronous_commit
>> = write.  Actually, the internal constant is named that way already,
>> but it's not exposed as such to the user.
>
> That's something to discuss at the end of the CF when people are less
> busy and we get more input.
>
> It's an easy change whatever we decide to do.

Added this to 9.2 Open Items.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

  7. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Robert Haas <robertmhaas@gmail.com> — 2012-04-13T17:15:18Z

    On Thu, Jan 26, 2012 at 1:21 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Wed, Jan 25, 2012 at 11:12 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>> On Wed, Jan 25, 2012 at 1:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>>
>>> I think that this would be a lot more clear if we described this as
>>> synchronous_commit = remote_write rather than just synchronous_commit
>>> = write.  Actually, the internal constant is named that way already,
>>> but it's not exposed as such to the user.
>>
>> That's something to discuss at the end of the CF when people are less
>> busy and we get more input.
>>
>> It's an easy change whatever we decide to do.
>
> Added this to 9.2 Open Items.

OK, so I think it's time to decide what we want to do here.  In my
view, remote_write seems a lot more clear than write (since someone
might otherwise think we were talking about a local write) and it has
the additional advantage of matching the internal naming convention -
surely, if it's write to call it SYNCHRONOUS_COMMIT_REMOTE_WRITE
inside the system, then there's not really much reason to drop the
word "remote" on the user-visible side of things.  However, I just
work here.  Does anyone want to make a counter-argument?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  8. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Kevin Grittner <kevin.grittner@wicourts.gov> — 2012-04-13T18:15:10Z

    Robert Haas <robertmhaas@gmail.com> wrote:
 
> In my view, remote_write seems a lot more clear than write
 
+1
 
I sure didn't understand it to mean remote_write when I read the
subject line.
 
-Kevin

  9. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Thom Brown <thom@linux.com> — 2012-04-13T19:16:28Z

    On 13 April 2012 19:15, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:
> Robert Haas <robertmhaas@gmail.com> wrote:
>
>> In my view, remote_write seems a lot more clear than write
>
> +1
>
> I sure didn't understand it to mean remote_write when I read the
> subject line.

Whatever this option value is named, it needs to be referenced in the
postgresql.conf comment for this option, as it isn't currently.

I have a question though.  What happens when this is set to "write"
(or "remote_write" as proposed) but it's being used on a standalone
primary?  At the moment it's not documented what level of guarantee
this would provide.

-- 
Thom

  10. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Guillaume Lelarge <guillaume@lelarge.info> — 2012-04-13T19:57:58Z

    On 04/13/2012 08:15 PM, Kevin Grittner wrote:
> Robert Haas<robertmhaas@gmail.com>  wrote:
>
>> In my view, remote_write seems a lot more clear than write
>
> +1
>
> I sure didn't understand it to mean remote_write when I read the
> subject line.
>

Neither did I. So definitely +1.


-- 
Guillaume
  http://www.postgresql.fr
  http://dalibo.com

  11. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Fujii Masao <masao.fujii@gmail.com> — 2012-04-14T14:28:39Z

    On Sat, Apr 14, 2012 at 4:57 AM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:
> On 04/13/2012 08:15 PM, Kevin Grittner wrote:
>>
>> Robert Haas<robertmhaas@gmail.com>  wrote:
>>
>>> In my view, remote_write seems a lot more clear than write
>>
>>
>> +1
>>
>> I sure didn't understand it to mean remote_write when I read the
>> subject line.
>>
>
> Neither did I. So definitely +1.

+1

Patch attached.

Regards,

-- 
Fujii Masao

  12. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Robert Haas <robertmhaas@gmail.com> — 2012-04-14T14:58:12Z

    On Sat, Apr 14, 2012 at 10:28 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Sat, Apr 14, 2012 at 4:57 AM, Guillaume Lelarge
> <guillaume@lelarge.info> wrote:
>> On 04/13/2012 08:15 PM, Kevin Grittner wrote:
>>> Robert Haas<robertmhaas@gmail.com>  wrote:
>>>> In my view, remote_write seems a lot more clear than write
>>>
>>> +1
>>>
>>> I sure didn't understand it to mean remote_write when I read the
>>> subject line.
>>
>> Neither did I. So definitely +1.
>
> +1
>
> Patch attached.

Thanks.  Committed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  13. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Fujii Masao <masao.fujii@gmail.com> — 2012-04-14T14:58:50Z

    On Sat, Apr 14, 2012 at 4:16 AM, Thom Brown <thom@linux.com> wrote:
> On 13 April 2012 19:15, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:
>> Robert Haas <robertmhaas@gmail.com> wrote:
>>
>>> In my view, remote_write seems a lot more clear than write
>>
>> +1
>>
>> I sure didn't understand it to mean remote_write when I read the
>> subject line.
>
> Whatever this option value is named, it needs to be referenced in the
> postgresql.conf comment for this option, as it isn't currently.

Yes. The patch I've posted does this.

> I have a question though.  What happens when this is set to "write"
> (or "remote_write" as proposed) but it's being used on a standalone
> primary?  At the moment it's not documented what level of guarantee
> this would provide.

http://www.postgresql.org/docs/devel/static/warm-standby.html#SYNCHRONOUS-REPLICATION-HA
-----------------
Commits made when synchronous_commit is set to on or write will
wait until the synchronous standby responds. The response may
never occur if the last, or only, standby should crash.
-----------------

Is this description not enough? If not enough, how should we change
the document?

Regards,

-- 
Fujii Masao

  14. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Thom Brown <thom@linux.com> — 2012-04-14T15:13:39Z

    On 14 April 2012 15:58, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Sat, Apr 14, 2012 at 4:16 AM, Thom Brown <thom@linux.com> wrote:
>> I have a question though.  What happens when this is set to "write"
>> (or "remote_write" as proposed) but it's being used on a standalone
>> primary?  At the moment it's not documented what level of guarantee
>> this would provide.
>
> http://www.postgresql.org/docs/devel/static/warm-standby.html#SYNCHRONOUS-REPLICATION-HA
> -----------------
> Commits made when synchronous_commit is set to on or write will
> wait until the synchronous standby responds. The response may
> never occur if the last, or only, standby should crash.
> -----------------
>
> Is this description not enough? If not enough, how should we change
> the document?

No, that's not what I was referring to.  If you don't have a standby
(i.e. a single, isolated database cluster with no replication), and
its synchronous_commit is set to 'remote_write', what effect does that
have?

-- 
Thom

  15. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Fujii Masao <masao.fujii@gmail.com> — 2012-04-16T16:21:52Z

    On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown <thom@linux.com> wrote:
> On 14 April 2012 15:58, Fujii Masao <masao.fujii@gmail.com> wrote:
>> On Sat, Apr 14, 2012 at 4:16 AM, Thom Brown <thom@linux.com> wrote:
>>> I have a question though.  What happens when this is set to "write"
>>> (or "remote_write" as proposed) but it's being used on a standalone
>>> primary?  At the moment it's not documented what level of guarantee
>>> this would provide.
>>
>> http://www.postgresql.org/docs/devel/static/warm-standby.html#SYNCHRONOUS-REPLICATION-HA
>> -----------------
>> Commits made when synchronous_commit is set to on or write will
>> wait until the synchronous standby responds. The response may
>> never occur if the last, or only, standby should crash.
>> -----------------
>>
>> Is this description not enough? If not enough, how should we change
>> the document?
>
> No, that's not what I was referring to.  If you don't have a standby
> (i.e. a single, isolated database cluster with no replication), and
> its synchronous_commit is set to 'remote_write', what effect does that
> have?

It's the same effect as 'on' and 'local' do, i.e., transaction commit waits
for only local WAL flush. This behavior is not documented explicitly...
How should we change the document? What about adding the following
into the explanation of synchronous_commit parameter (maybe the end
of second paragraph of that)?

-----------------
If synchronous_standby_names is not set, on, remote_write and local
provide the same synchronization level; transaction commit only waits for
local flush.
-----------------

Regards,

-- 
Fujii Masao

  16. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Thom Brown <thom@linux.com> — 2012-04-17T12:52:00Z

    On 16 April 2012 17:21, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown <thom@linux.com> wrote:
>> No, that's not what I was referring to.  If you don't have a standby
>> (i.e. a single, isolated database cluster with no replication), and
>> its synchronous_commit is set to 'remote_write', what effect does that
>> have?
>
> It's the same effect as 'on' and 'local' do, i.e., transaction commit waits
> for only local WAL flush. This behavior is not documented explicitly...
> How should we change the document? What about adding the following
> into the explanation of synchronous_commit parameter (maybe the end
> of second paragraph of that)?
>
> -----------------
> If synchronous_standby_names is not set, on, remote_write and local
> provide the same synchronization level; transaction commit only waits for
> local flush.
> -----------------

Yes, that sounds fine.

-- 
Thom

  17. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Fujii Masao <masao.fujii@gmail.com> — 2012-04-17T19:29:12Z

    On Tue, Apr 17, 2012 at 9:52 PM, Thom Brown <thom@linux.com> wrote:
> On 16 April 2012 17:21, Fujii Masao <masao.fujii@gmail.com> wrote:
>> On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown <thom@linux.com> wrote:
>>> No, that's not what I was referring to.  If you don't have a standby
>>> (i.e. a single, isolated database cluster with no replication), and
>>> its synchronous_commit is set to 'remote_write', what effect does that
>>> have?
>>
>> It's the same effect as 'on' and 'local' do, i.e., transaction commit waits
>> for only local WAL flush. This behavior is not documented explicitly...
>> How should we change the document? What about adding the following
>> into the explanation of synchronous_commit parameter (maybe the end
>> of second paragraph of that)?
>>
>> -----------------
>> If synchronous_standby_names is not set, on, remote_write and local
>> provide the same synchronization level; transaction commit only waits for
>> local flush.
>> -----------------
>
> Yes, that sounds fine.

Okay, patch attached.

Regards,

-- 
Fujii Masao

  18. Re: [COMMITTERS] pgsql: Add new replication mode synchronous_commit = 'write'.

    Robert Haas <robertmhaas@gmail.com> — 2012-04-18T14:11:59Z

    On Tue, Apr 17, 2012 at 3:29 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
> Okay, patch attached.

Committed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company