Horton

Thread

Commits

  1. Allow "COPY table TO" command to copy rows from materialized views.

  1. in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2024-12-19T06:28:21Z

    hi.
generally  COPY TO``COPY table`` is faster than ``COPY (select * from table)``.

in BeginCopyTo. we have
"""
        else if (rel->rd_rel->relkind == RELKIND_MATVIEW)
            ereport(ERROR,
                    (errcode(ERRCODE_WRONG_OBJECT_TYPE),
                     errmsg("cannot copy from materialized view \"%s\"",
                            RelationGetRelationName(rel)),
                     errhint("Try the COPY (SELECT ...) TO variant.")));
"""
Since materialized views have physical storage,

we can make materialized views also using COPY table_name, instead of
COPY(query).

Some simple tests show around %3.7 or 4.3% speed up.

  2. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Michael Paquier <michael@paquier.xyz> — 2024-12-20T00:01:48Z

    On Thu, Dec 19, 2024 at 02:28:21PM +0800, jian he wrote:
> Since materialized views have physical storage,
> 
> we can make materialized views also using COPY table_name, instead of
> COPY(query).
> 
> Some simple tests show around %3.7 or 4.3% speed up.

This restriction comes from 3bf3ab8c5636 as such relations may not be
scannable when they have no data, no?  Perhaps this restriction could
be lifted, but I'd suggest to dig more into the lists, there should be
arguments and ideas explaining what could be done in this case
(spoiler: I did not look at that).
--
Michael

  3. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2024-12-20T02:53:06Z

    On Fri, Dec 20, 2024 at 8:02 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Thu, Dec 19, 2024 at 02:28:21PM +0800, jian he wrote:
> > Since materialized views have physical storage,
> >
> > we can make materialized views also using COPY table_name, instead of
> > COPY(query).
> >
> > Some simple tests show around %3.7 or 4.3% speed up.
>
> This restriction comes from 3bf3ab8c5636 as such relations may not be
> scannable when they have no data, no?  Perhaps this restriction could
> be lifted, but I'd suggest to dig more into the lists, there should be
> arguments and ideas explaining what could be done in this case
> (spoiler: I did not look at that).
> --


Thanks for the suggestion.
it was mentioned in link [1] and [2].

[1] https://www.postgresql.org/message-id/flat/8967.1353167301%40sss.pgh.pa.us#f5e947cfa9357dba780d238f1c5f6932
[2] https://www.postgresql.org/message-id/20121116162558.90150%40gmx.com


Basically we want to have the two directions of COPY.
so
copy the_materialized_view to stdout;
copy the_materialized_view  from stdin;

both will work fine.
obviously "copy the_materialized_view  from stdin; " will not work.

  4. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2025-01-06T09:34:43Z

    hi.

about this issue,
last email in 2012 (https://postgr.es/m/8967.1353167301@sss.pgh.pa.us)
"""
Even if it happens to be trivial in the current patch, it's an added
functional requirement that we might later regret having cavalierly
signed up for. And, as noted upthread, relations that support only
one direction of COPY don't exist at the moment; that would be adding
an asymmetry that we might later regret, too.

regards, tom lane
"""

but now we have numerous COPY options that work solely in a single
direction of COPY.
I think now we can make some kind of relation (pg_class.relkind) that
only works in one direction of COPY.

  5. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2025-01-28T02:48:25Z

    On Mon, Jan 6, 2025 at 5:34 PM jian he <jian.universality@gmail.com> wrote:
>
> hi.
>
> about this issue,
> last email in 2012 (https://postgr.es/m/8967.1353167301@sss.pgh.pa.us)
> """
> Even if it happens to be trivial in the current patch, it's an added
> functional requirement that we might later regret having cavalierly
> signed up for. And, as noted upthread, relations that support only
> one direction of COPY don't exist at the moment; that would be adding
> an asymmetry that we might later regret, too.
>
> regards, tom lane
> """
>
> but now we have numerous COPY options that work solely in a single
> direction of COPY.
> I think now we can make some kind of relation (pg_class.relkind) that
> only works in one direction of COPY.

hi.
patch attached.
also cc to Tom,
since at that time, you are against the idea of ``COPY matview TO``.

  6. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    newtglobal postgresql_contributors <postgresql_contributors@newtglobalcorp.com> — 2025-03-13T06:50:31Z

    The following review has been posted through the commitfest application:
make installcheck-world:  not tested
Implements feature:       not tested
Spec compliant:           not tested
Documentation:            not tested

Hi,
Tested the latest patch that allows direct `COPY` operations on Materialized Views, removing the need for `COPY (SELECT ...)`. This enhancement reduces query overhead, improving performance by **4–5%**.  
Example:  
Previous approach: 
COPY (SELECT * FROM staff_summary) TO STDOUT WITH CSV HEADER;
Optimized approach:  
COPY staff_summary TO STDOUT WITH CSV HEADER;
Performance tests were conducted using a Materialized View containing around 80,000 records, confirming that the new approach is faster and more efficient for exporting data.

Regards,
Newt Global PostgreSQL Contributors

  7. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Kirill Reshke <reshkekirill@gmail.com> — 2025-03-26T19:04:34Z

    On Tue, 28 Jan 2025 at 07:49, jian he <jian.universality@gmail.com> wrote:
>
> On Mon, Jan 6, 2025 at 5:34 PM jian he <jian.universality@gmail.com> wrote:
> >
> > hi.
> >
> > about this issue,
> > last email in 2012 (https://postgr.es/m/8967.1353167301@sss.pgh.pa.us)
> > """
> > Even if it happens to be trivial in the current patch, it's an added
> > functional requirement that we might later regret having cavalierly
> > signed up for. And, as noted upthread, relations that support only
> > one direction of COPY don't exist at the moment; that would be adding
> > an asymmetry that we might later regret, too.
> >
> > regards, tom lane
> > """
> >
> > but now we have numerous COPY options that work solely in a single
> > direction of COPY.
> > I think now we can make some kind of relation (pg_class.relkind) that
> > only works in one direction of COPY.
>
> hi.
> patch attached.
> also cc to Tom,
> since at that time, you are against the idea of ``COPY matview TO``.

Hi! With this patch it is possible to COPY matview TO, but not regular
view, which is surprising. Let's fix that?


-- 
Best regards,
Kirill Reshke

  8. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2025-03-29T06:46:37Z

    On Thu, Mar 27, 2025 at 3:04 AM Kirill Reshke <reshkekirill@gmail.com> wrote:
> >
> > hi.
> > patch attached.
> > also cc to Tom,
> > since at that time, you are against the idea of ``COPY matview TO``.
>
> Hi! With this patch it is possible to COPY matview TO, but not regular
> view, which is surprising. Let's fix that?

create view v1 as select 1;
copy v1 to stdout;

if you specifying table name, not query, then
{
        cstate = BeginCopyTo(pstate, rel, query, relid,
                             stmt->filename, stmt->is_program,
                             NULL, stmt->attlist, stmt->options);
        *processed = DoCopyTo(cstate);    /* copy from database to file *
}
will use {table_beginscan, table_scan_getnextslot, table_endscan}
to output the data.
but views don't have storage, table_beginscan mechanism won't work.

so i don't think this is possible for view.

  9. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Kirill Reshke <reshkekirill@gmail.com> — 2025-03-29T12:38:15Z

    On Sat, 29 Mar 2025 at 09:47, jian he <jian.universality@gmail.com> wrote:
>
> will use {table_beginscan, table_scan_getnextslot, table_endscan}
> to output the data.
> but views don't have storage, table_beginscan mechanism won't work.
>
> so i don't think this is possible for view.

Well... So you are saying that let us have inconsistent features
because of how things are implemented in core... I don't sure I'm
buying that, but whatever, let's hear some other voices from the
community. My argument is that while we are working on it, perhaps we
should revise certain implementation specifics along the way. However,
this is merely my opinion on the matter.

-- 
Best regards,
Kirill Reshke

  10. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    David G. Johnston <david.g.johnston@gmail.com> — 2025-03-29T14:40:55Z

    On Saturday, March 29, 2025, Kirill Reshke <reshkekirill@gmail.com> wrote:

> On Sat, 29 Mar 2025 at 09:47, jian he <jian.universality@gmail.com> wrote:
> >
> > will use {table_beginscan, table_scan_getnextslot, table_endscan}
> > to output the data.
> > but views don't have storage, table_beginscan mechanism won't work.
> >
> > so i don't think this is possible for view.
>
> Well... So you are saying that let us have inconsistent features
> because of how things are implemented in core... I don't sure I'm
> buying that, but whatever, let's hear some other voices from the
> community. My argument is that while we are working on it, perhaps we
> should revise certain implementation specifics along the way. However,
> this is merely my opinion on the matter.
>

 At present copy {table} to only exists to support pg_dump.  It is not
marketed as a general purpose export facility.  While copy {relation} from
is a general purpose importer.  Do we want to turn copy to into a general
purpose exporter and start accepting patches to export foreign tables, deal
with partitioned tables, views of both kinds?  Given we already accept
things in copy from that copy to cannot produce the symmetry argument seems
flawed.

I’m on board with making copy {relation} to a general purpose export
facility and allowing for incremental implementations as people wish to
spend time developing them.  Consistency should not prevent progress here.

On the topic of copy {matview} from, why not permit it?  In particular,
with dump/restore we could dump the materialized view and restore it, which
seems like a win in terms of time spent restoring.  That wouldn’t be this
patch.

David J.

  11. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Andrew Dunstan <andrew@dunslane.net> — 2025-03-29T16:06:53Z

    On 2025-03-29 Sa 10:40 AM, David G. Johnston wrote:
> On Saturday, March 29, 2025, Kirill Reshke <reshkekirill@gmail.com> wrote:
>
>     On Sat, 29 Mar 2025 at 09:47, jian he
>     <jian.universality@gmail.com> wrote:
>     >
>     > will use {table_beginscan, table_scan_getnextslot, table_endscan}
>     > to output the data.
>     > but views don't have storage, table_beginscan mechanism won't work.
>     >
>     > so i don't think this is possible for view.
>
>     Well... So you are saying that let us have inconsistent features
>     because of how things are implemented in core... I don't sure I'm
>     buying that, but whatever, let's hear some other voices from the
>     community. My argument is that while we are working on it, perhaps we
>     should revise certain implementation specifics along the way. However,
>     this is merely my opinion on the matter.
>
>
>  At present copy {table} to only exists to support pg_dump. It is not 
> marketed as a general purpose export facility.


*ahem*


What is your evidence for that proposition? If this were true we would 
not support CSV mode, which pg_dump does not use. It might have 
limitations, but its use goes far beyond just pg_dump, both in theory 
and practice.


cheers


andew


--
Andrew Dunstan
EDB:https://www.enterprisedb.com

  12. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    David G. Johnston <david.g.johnston@gmail.com> — 2025-03-29T16:17:21Z

    On Sat, Mar 29, 2025 at 9:06 AM Andrew Dunstan <andrew@dunslane.net> wrote:

>
> On 2025-03-29 Sa 10:40 AM, David G. Johnston wrote:
>
> On Saturday, March 29, 2025, Kirill Reshke <reshkekirill@gmail.com> wrote:
>
>> On Sat, 29 Mar 2025 at 09:47, jian he <jian.universality@gmail.com>
>> wrote:
>> >
>> > will use {table_beginscan, table_scan_getnextslot, table_endscan}
>> > to output the data.
>> > but views don't have storage, table_beginscan mechanism won't work.
>> >
>> > so i don't think this is possible for view.
>>
>> Well... So you are saying that let us have inconsistent features
>> because of how things are implemented in core... I don't sure I'm
>> buying that, but whatever, let's hear some other voices from the
>> community. My argument is that while we are working on it, perhaps we
>> should revise certain implementation specifics along the way. However,
>> this is merely my opinion on the matter.
>>
>
>  At present copy {table} to only exists to support pg_dump.  It is not
> marketed as a general purpose export facility.
>
>
> *ahem*
>
>
> What is your evidence for that proposition? If this were true we would not
> support CSV mode, which pg_dump does not use. It might have limitations,
> but its use goes far beyond just pg_dump, both in theory and practice.
>
>
>
>
"copy {subquery} to" is a general-purpose exporter that makes use of those
additional features.  Sure, they also work for the narrowed case of "copy
{relation/table} to" but I make my claim on the very fact that {relation}
cannot be stuff like foreign tables or partitioned tables, which pg_dump
has no need to target.

David J.

  13. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Andrew Dunstan <andrew@dunslane.net> — 2025-03-29T18:56:03Z

    On 2025-03-29 Sa 12:17 PM, David G. Johnston wrote:
> On Sat, Mar 29, 2025 at 9:06 AM Andrew Dunstan <andrew@dunslane.net> 
> wrote:
>
>
>     On 2025-03-29 Sa 10:40 AM, David G. Johnston wrote:
>>     On Saturday, March 29, 2025, Kirill Reshke
>>     <reshkekirill@gmail.com> wrote:
>>
>>         On Sat, 29 Mar 2025 at 09:47, jian he
>>         <jian.universality@gmail.com> wrote:
>>         >
>>         > will use {table_beginscan, table_scan_getnextslot,
>>         table_endscan}
>>         > to output the data.
>>         > but views don't have storage, table_beginscan mechanism
>>         won't work.
>>         >
>>         > so i don't think this is possible for view.
>>
>>         Well... So you are saying that let us have inconsistent features
>>         because of how things are implemented in core... I don't sure I'm
>>         buying that, but whatever, let's hear some other voices from the
>>         community. My argument is that while we are working on it,
>>         perhaps we
>>         should revise certain implementation specifics along the way.
>>         However,
>>         this is merely my opinion on the matter.
>>
>>
>>      At present copy {table} to only exists to support pg_dump.  It
>>     is not marketed as a general purpose export facility.
>
>
>     *ahem*
>
>
>     What is your evidence for that proposition? If this were true we
>     would not support CSV mode, which pg_dump does not use. It might
>     have limitations, but its use goes far beyond just pg_dump, both
>     in theory and practice.
>
>
>
>
> "copy {subquery} to" is a general-purpose exporter that makes use of 
> those additional features.  Sure, they also work for the narrowed case 
> of "copy {relation/table} to" but I make my claim on the very fact 
> that {relation} cannot be stuff like foreign tables or partitioned 
> tables, which pg_dump has no need to target.
>
>


I don't believe that the premise supports the conclusion.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

  14. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    David G. Johnston <david.g.johnston@gmail.com> — 2025-03-29T18:58:22Z

    On Sat, Mar 29, 2025 at 11:56 AM Andrew Dunstan <andrew@dunslane.net> wrote:

> I don't believe that the premise supports the conclusion.
>
>
> Regardless, I do support this patch and probably any similar ones proposed
in the future.  Do you have an opinion on that?

David J.

  15. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Kirill Reshke <reshkekirill@gmail.com> — 2025-03-29T19:27:22Z

    On Sat, 29 Mar 2025 at 19:59, David G. Johnston
<david.g.johnston@gmail.com> wrote:
> Regardless, I do support this patch and probably any similar ones proposed in the future.  Do you have an opinion on that?
>
> David J.
>

I do also support what this patch aims to do, how do you like v1?

-- 
Best regards,
Kirill Reshke

  16. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    David G. Johnston <david.g.johnston@gmail.com> — 2025-03-29T19:49:16Z

    On Sat, Mar 29, 2025 at 12:27 PM Kirill Reshke <reshkekirill@gmail.com>
wrote:

> On Sat, 29 Mar 2025 at 19:59, David G. Johnston
> <david.g.johnston@gmail.com> wrote:
> > Regardless, I do support this patch and probably any similar ones
> proposed in the future.  Do you have an opinion on that?
> >
>
> I do also support what this patch aims to do, how do you like v1?
>
>
Seems reasonable, but I don't have enough experience with the codebase in
that area to submit a code review or be aware of any unexpected
side-effects or deficiencies.

David J.

  17. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Andrew Dunstan <andrew@dunslane.net> — 2025-03-29T20:01:53Z

    On 2025-03-29 Sa 2:58 PM, David G. Johnston wrote:
> On Sat, Mar 29, 2025 at 11:56 AM Andrew Dunstan <andrew@dunslane.net> 
> wrote:
>
>     I don't believe that the premise supports the conclusion.
>
>
> Regardless, I do support this patch and probably any similar ones 
> proposed in the future.  Do you have an opinion on that?
>
>

In principle I think it would be good to have COPY materialized_view TO ...


cheers


andrew


--
Andrew Dunstan
EDB:https://www.enterprisedb.com

  18. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2025-03-31T15:26:58Z

    
On 2025/03/30 5:01, Andrew Dunstan wrote:
> 
> On 2025-03-29 Sa 2:58 PM, David G. Johnston wrote:
>> On Sat, Mar 29, 2025 at 11:56 AM Andrew Dunstan <andrew@dunslane.net> wrote:
>>
>>     I don't believe that the premise supports the conclusion.
>>
>>
>> Regardless, I do support this patch and probably any similar ones proposed in the future.  Do you have an opinion on that?
>>
>>
> 
> In principle I think it would be good to have COPY materialized_view TO ...

I haven't found any reasons to object to this patch for now,
so I have no objections to this change.


Regarding the patch, here are some review comments:

+						errmsg("cannot copy from materialized view when the materialized view is not populated"),

How about including the object name for consistency with
other error messages in BeginCopyTo(), like this?

	errmsg("cannot copy from unpopulated materialized view \"%s\"",
		   RelationGetRelationName(rel)),


+						errhint("Use the REFRESH MATERIALIZED VIEW command populate the materialized view first."));

There seems to be a missing "to" just after "command".
Should it be "Use the REFRESH MATERIALIZED VIEW command to
populate the materialized view first."? Or we could simplify
the hint to match what SELECT on an unpopulated materialized
view logs: "Use the REFRESH MATERIALIZED VIEW command.".


The copy.sgml documentation should clarify that COPY TO can
be used with a materialized view only if it is populated.


Wouldn't it be beneficial to add a regression test to check
whether COPY matview TO works as expected?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  19. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2025-04-01T03:12:45Z

    On Mon, Mar 31, 2025 at 11:27 PM Fujii Masao
<masao.fujii@oss.nttdata.com> wrote:
>
> Regarding the patch, here are some review comments:
>
> +                                               errmsg("cannot copy from materialized view when the materialized view is not populated"),
>
> How about including the object name for consistency with
> other error messages in BeginCopyTo(), like this?
>
>         errmsg("cannot copy from unpopulated materialized view \"%s\"",
>                    RelationGetRelationName(rel)),
>
>
> +                                               errhint("Use the REFRESH MATERIALIZED VIEW command populate the materialized view first."));
>
> There seems to be a missing "to" just after "command".
> Should it be "Use the REFRESH MATERIALIZED VIEW command to
> populate the materialized view first."? Or we could simplify
> the hint to match what SELECT on an unpopulated materialized
> view logs: "Use the REFRESH MATERIALIZED VIEW command.".
>
based on your suggestion, i changed it to:

            if (!RelationIsPopulated(rel))
                ereport(ERROR,
                        errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
                        errmsg("cannot copy from unpopulated
materialized view \"%s\"",
                                    RelationGetRelationName(rel)),
                        errhint("Use the REFRESH MATERIALIZED VIEW
command to populate the materialized view first."));


>
> The copy.sgml documentation should clarify that COPY TO can
> be used with a materialized view only if it is populated.
>
"COPY TO can be used only with plain tables, not views, and does not
copy rows from child tables or child partitions"
i changed it to
"COPY TO can be used with plain tables and materialized views, not
regular views, and does not copy rows from child tables or child
partitions"

Another alternative wording I came up with:
"COPY TO can only be used with plain tables and materialized views,
not regular views. It also does not copy rows from child tables or
child partitions."


>
> Wouldn't it be beneficial to add a regression test to check
> whether COPY matview TO works as expected?
sure.

  20. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    David G. Johnston <david.g.johnston@gmail.com> — 2025-04-01T03:50:59Z

    On Mon, Mar 31, 2025 at 8:13 PM jian he <jian.universality@gmail.com> wrote:

> On Mon, Mar 31, 2025 at 11:27 PM Fujii Masao
>
> >
> > The copy.sgml documentation should clarify that COPY TO can
> > be used with a materialized view only if it is populated.
> >
> "COPY TO can be used only with plain tables, not views, and does not
> copy rows from child tables or child partitions"
> i changed it to
> "COPY TO can be used with plain tables and materialized views, not
> regular views, and does not copy rows from child tables or child
> partitions"
>
> Another alternative wording I came up with:
> "COPY TO can only be used with plain tables and materialized views,
> not regular views. It also does not copy rows from child tables or
> child partitions."
>
>
Those don't address the "populated" aspect of the materialized view.

I'm unclear ATM (can check later if needed) if populated means "non-empty"
or simply "had refresh executed at least once on it".  i.e., if the refresh
produces zero rows stored in the MV is it still populated?  I'm guessing
yes; and this only pertains to "WITH NO DATA", which itself already calls
out that "...and cannot be queried until RMV is used".  I find it of
marginal usefulness to bring that distinction over to COPY TO absent people
showing up confused about the error message, which likely will be quite
rare.  That said I'd probably settle with:

COPY TO can only be used with plain tables and populated
materialized views. It does not copy rows from child tables
or child partitions (i.e., copy table to copies the same rows as
select * from only table). The syntax COPY (select * from table) TO ...
can be used to dump all of the rows in an inheritance hierarchy,
partitioned table, or foreign table; as well as ordinary view results.

Curious about sequences; no way to name an index here.

I'm second-guessing why "composite type" shows up in the glossary under
"Relation"...though that originally came up IIRC discussing namespaces.

David J.

  21. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    vignesh C <vignesh21@gmail.com> — 2025-04-01T09:45:28Z

    On Tue, 1 Apr 2025 at 08:43, jian he <jian.universality@gmail.com> wrote:
>
> On Mon, Mar 31, 2025 at 11:27 PM Fujii Masao
> <masao.fujii@oss.nttdata.com> wrote:
> >
> > Regarding the patch, here are some review comments:
> >
> > +                                               errmsg("cannot copy from materialized view when the materialized view is not populated"),
> >
> > How about including the object name for consistency with
> > other error messages in BeginCopyTo(), like this?
> >
> >         errmsg("cannot copy from unpopulated materialized view \"%s\"",
> >                    RelationGetRelationName(rel)),
> >
> >
> > +                                               errhint("Use the REFRESH MATERIALIZED VIEW command populate the materialized view first."));
> >
> > There seems to be a missing "to" just after "command".
> > Should it be "Use the REFRESH MATERIALIZED VIEW command to
> > populate the materialized view first."? Or we could simplify
> > the hint to match what SELECT on an unpopulated materialized
> > view logs: "Use the REFRESH MATERIALIZED VIEW command.".
> >
> based on your suggestion, i changed it to:
>
>             if (!RelationIsPopulated(rel))
>                 ereport(ERROR,
>                         errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
>                         errmsg("cannot copy from unpopulated
> materialized view \"%s\"",
>                                     RelationGetRelationName(rel)),
>                         errhint("Use the REFRESH MATERIALIZED VIEW
> command to populate the materialized view first."));
>
>
> >
> > The copy.sgml documentation should clarify that COPY TO can
> > be used with a materialized view only if it is populated.
> >
> "COPY TO can be used only with plain tables, not views, and does not
> copy rows from child tables or child partitions"
> i changed it to
> "COPY TO can be used with plain tables and materialized views, not
> regular views, and does not copy rows from child tables or child
> partitions"
>
> Another alternative wording I came up with:
> "COPY TO can only be used with plain tables and materialized views,
> not regular views. It also does not copy rows from child tables or
> child partitions."

One thing I noticed was that if the materialized view is not refreshed
user will get stale data:
postgres=# create table t1(c1 int);
CREATE TABLE
postgres=# create materialized view mv2 as select * from t1;
SELECT 0

postgres=# insert into t1 values(10);
INSERT 0 1
postgres=# select * from t1;
 c1
----
 10
(1 row)

-- Before refresh the data will not be selected
postgres=# copy mv2 to stdout with (header);
c1

-- After refresh the data will be available
postgres=# refresh materialized view mv2;
REFRESH MATERIALIZED VIEW
postgres=# copy mv2 to stdout with (header);
c1
10

Should we document this?

The following can be changed to keep it consistent:
+copy matview1(id) TO stdout with (header);
+copy matview2 TO stdout with (header);
To:
COPY matview1(id) TO stdout with (header);
COPY matview2 TO stdout  with (header);

Regards,
Vignesh

  22. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Kirill Reshke <reshkekirill@gmail.com> — 2025-04-01T10:18:55Z

    On Tue, 1 Apr 2025, 11:45 vignesh C, <vignesh21@gmail.com> wrote:

>
> One thing I noticed was that if the materialized view is not refreshed
> user will get stale data
>
> Should we document this?
>


Does this patch alter thus behaviour? User will get stale data even on
HEAD, why should we take a care within this thread?

>

  23. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    vignesh C <vignesh21@gmail.com> — 2025-04-01T13:52:27Z

    On Tue, 1 Apr 2025 at 15:49, Kirill Reshke <reshkekirill@gmail.com> wrote:
>
> On Tue, 1 Apr 2025, 11:45 vignesh C, <vignesh21@gmail.com> wrote:
>>
>>
>> One thing I noticed was that if the materialized view is not refreshed
>> user will get stale data
>>
>> Should we document this?
>
> Does this patch alter thus behaviour? User will get stale data even on HEAD, why should we take a care within this thread?

We are not changing the existing behavior. However, since copying data
from large tables can take a significant amount of time, would it be
helpful to add a cautionary note advising users to refresh the
materialized view before running copy command to avoid stale data?
This could prevent users from realizing the issue only after running
the copy operation, which would then require them to run it again. If
you think this is already obvious, then the note may not be necessary.

Regards,
Vignesh

  24. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Kirill Reshke <reshkekirill@gmail.com> — 2025-04-01T14:14:31Z

    On Tue, 1 Apr 2025 at 15:52, vignesh C <vignesh21@gmail.com> wrote:
>
> On Tue, 1 Apr 2025 at 15:49, Kirill Reshke <reshkekirill@gmail.com> wrote:
> >
> > On Tue, 1 Apr 2025, 11:45 vignesh C, <vignesh21@gmail.com> wrote:
> >>
> >>
> >> One thing I noticed was that if the materialized view is not refreshed
> >> user will get stale data
> >>
> >> Should we document this?
> >
> > Does this patch alter thus behaviour? User will get stale data even on HEAD, why should we take a care within this thread?
>
> We are not changing the existing behavior. However, since copying data
> from large tables can take a significant amount of time, would it be
> helpful to add a cautionary note advising users to refresh the
> materialized view before running copy command to avoid stale data?
> This could prevent users from realizing the issue only after running
> the copy operation, which would then require them to run it again.

Yes, agree, +1 on that.

> If
> you think this is already obvious, then the note may not be necessary.

I don't think this is already obvious, but my objection is that we
should maybe discuss this as a separate issue (in a separate patch).
Looks like fixing this together with code commit is too much at once.
I prefer a one-commit-for-one-purpose style.

> Regards,
> Vignesh



-- 
Best regards,
Kirill Reshke

  25. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    David G. Johnston <david.g.johnston@gmail.com> — 2025-04-01T14:27:24Z

    On Tue, Apr 1, 2025 at 6:52 AM vignesh C <vignesh21@gmail.com> wrote:

>
> We are not changing the existing behavior. However, since copying data
> from large tables can take a significant amount of time, would it be
> helpful to add a cautionary note advising users to refresh the
> materialized view before running copy command to avoid stale data?
>
>
No, for the same reason the caveat about WITH NO DATA need not be mentioned
either.  These are the inherent properties/trade-offs of using a
materialized view versus a normal view.  They are discussed when talking
about the creation and usage of materialized views specifically.  Features
that interact with materialized views consistent with these two documented
properties do not need to point out that fact. i.e., the existing
non-documenting of those two points in SELECT is appropriate, and should be
emulated in COPY.  The output doesn't change if you write "copy table"
instead of "copy (select * from table)" so nothing needs to be pointed out.

David J.

  26. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2025-04-02T15:19:58Z

    
On 2025/04/01 12:12, jian he wrote:
> On Mon, Mar 31, 2025 at 11:27 PM Fujii Masao
> <masao.fujii@oss.nttdata.com> wrote:
>>
>> Regarding the patch, here are some review comments:
>>
>> +                                               errmsg("cannot copy from materialized view when the materialized view is not populated"),
>>
>> How about including the object name for consistency with
>> other error messages in BeginCopyTo(), like this?
>>
>>          errmsg("cannot copy from unpopulated materialized view \"%s\"",
>>                     RelationGetRelationName(rel)),
>>
>>
>> +                                               errhint("Use the REFRESH MATERIALIZED VIEW command populate the materialized view first."));
>>
>> There seems to be a missing "to" just after "command".
>> Should it be "Use the REFRESH MATERIALIZED VIEW command to
>> populate the materialized view first."? Or we could simplify
>> the hint to match what SELECT on an unpopulated materialized
>> view logs: "Use the REFRESH MATERIALIZED VIEW command.".
>>
> based on your suggestion, i changed it to:

Thanks for updating the patch!

> 
>              if (!RelationIsPopulated(rel))
>                  ereport(ERROR,
>                          errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
>                          errmsg("cannot copy from unpopulated
> materialized view \"%s\"",
>                                      RelationGetRelationName(rel)),
>                          errhint("Use the REFRESH MATERIALIZED VIEW
> command to populate the materialized view first."));

I think it's better to use the same hint message as the one output by
"COPY (SELECT * FROM <unpopulated matview>) TO",
specifically: "Use the REFRESH MATERIALIZED VIEW command," for consistency.


>> The copy.sgml documentation should clarify that COPY TO can
>> be used with a materialized view only if it is populated.
>>
> "COPY TO can be used only with plain tables, not views, and does not
> copy rows from child tables or child partitions"
> i changed it to
> "COPY TO can be used with plain tables and materialized views, not
> regular views, and does not copy rows from child tables or child
> partitions"

It would be clearer to specify that "COPY TO" applies to *populated*
materialized views rather than just "materialized views"?


> Another alternative wording I came up with:
> "COPY TO can only be used with plain tables and materialized views,
> not regular views. It also does not copy rows from child tables or
> child partitions."

If we split the first description into two as you suggested,
I'm tempted to propose the following improvements to enhance
the overall descriptions:

-------------
"COPY TO" can be used with plain tables and populated materialized views. For example, "COPY table TO" copies the same rows as "SELECT * FROM ONLY table." However, it doesn't directly support other relation types, such as partitioned tables, inheritance child tables, or views. To copy all rows from these relations, use "COPY (SELECT * FROM table) TO."
-------------


>> Wouldn't it be beneficial to add a regression test to check
>> whether COPY matview TO works as expected?
> sure.

The tests seem to have been placed under the category "COPY FROM ... DEFAULT",
which feels a bit misaligned. How about adding them to the end of copy.sql instead?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  27. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    jian he <jian.universality@gmail.com> — 2025-04-03T08:53:02Z

    On Wed, Apr 2, 2025 at 11:20 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
> >
> >              if (!RelationIsPopulated(rel))
> >                  ereport(ERROR,
> >                          errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
> >                          errmsg("cannot copy from unpopulated
> > materialized view \"%s\"",
> >                                      RelationGetRelationName(rel)),
> >                          errhint("Use the REFRESH MATERIALIZED VIEW
> > command to populate the materialized view first."));
>
> I think it's better to use the same hint message as the one output by
> "COPY (SELECT * FROM <unpopulated matview>) TO",
> specifically: "Use the REFRESH MATERIALIZED VIEW command," for consistency.
>
ok.

>
> >> The copy.sgml documentation should clarify that COPY TO can
> >> be used with a materialized view only if it is populated.
> >>
> > "COPY TO can be used only with plain tables, not views, and does not
> > copy rows from child tables or child partitions"
> > i changed it to
> > "COPY TO can be used with plain tables and materialized views, not
> > regular views, and does not copy rows from child tables or child
> > partitions"
>
> It would be clearer to specify that "COPY TO" applies to *populated*
> materialized views rather than just "materialized views"?
>
>
> > Another alternative wording I came up with:
> > "COPY TO can only be used with plain tables and materialized views,
> > not regular views. It also does not copy rows from child tables or
> > child partitions."
>
> If we split the first description into two as you suggested,
> I'm tempted to propose the following improvements to enhance
> the overall descriptions:
>
> -------------
> "COPY TO" can be used with plain tables and populated materialized views. For example, "COPY table TO" copies the same rows as "SELECT * FROM ONLY table." However, it doesn't directly support other relation types, such as partitioned tables, inheritance child tables, or views. To copy all rows from these relations, use "COPY (SELECT * FROM table) TO."
> -------------
>

your wording makes sense to me.
I try to ensure that the changing part in copy.sgml the line width
is less than 80 characters.
but I also want to make sure "<>" "</>" within the same line.
so after the change it becomes:

   <para>
    <command>COPY TO</command> can be used with plain
    tables and populated materialized views.
    For example,
    <literal>COPY <replaceable class="parameter">table</replaceable>
TO</literal>
    copies the same rows as
    <literal>SELECT * FROM ONLY <replaceable
class="parameter">table</replaceable></literal>.
    However it doesn't directly support other relation types,
    such as partitioned tables, inheritance child tables, or views.



> The tests seem to have been placed under the category "COPY FROM ... DEFAULT",
> which feels a bit misaligned. How about adding them to the end of copy.sql instead?
>
ok.

  28. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2025-04-03T11:46:59Z

    
On 2025/04/03 17:53, jian he wrote:
> On Wed, Apr 2, 2025 at 11:20 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>>
>>>               if (!RelationIsPopulated(rel))
>>>                   ereport(ERROR,
>>>                           errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
>>>                           errmsg("cannot copy from unpopulated
>>> materialized view \"%s\"",
>>>                                       RelationGetRelationName(rel)),
>>>                           errhint("Use the REFRESH MATERIALIZED VIEW
>>> command to populate the materialized view first."));
>>
>> I think it's better to use the same hint message as the one output by
>> "COPY (SELECT * FROM <unpopulated matview>) TO",
>> specifically: "Use the REFRESH MATERIALIZED VIEW command," for consistency.
>>
> ok.
> 
>>
>>>> The copy.sgml documentation should clarify that COPY TO can
>>>> be used with a materialized view only if it is populated.
>>>>
>>> "COPY TO can be used only with plain tables, not views, and does not
>>> copy rows from child tables or child partitions"
>>> i changed it to
>>> "COPY TO can be used with plain tables and materialized views, not
>>> regular views, and does not copy rows from child tables or child
>>> partitions"
>>
>> It would be clearer to specify that "COPY TO" applies to *populated*
>> materialized views rather than just "materialized views"?
>>
>>
>>> Another alternative wording I came up with:
>>> "COPY TO can only be used with plain tables and materialized views,
>>> not regular views. It also does not copy rows from child tables or
>>> child partitions."
>>
>> If we split the first description into two as you suggested,
>> I'm tempted to propose the following improvements to enhance
>> the overall descriptions:
>>
>> -------------
>> "COPY TO" can be used with plain tables and populated materialized views. For example, "COPY table TO" copies the same rows as "SELECT * FROM ONLY table." However, it doesn't directly support other relation types, such as partitioned tables, inheritance child tables, or views. To copy all rows from these relations, use "COPY (SELECT * FROM table) TO."
>> -------------
>>
> 
> your wording makes sense to me.
> I try to ensure that the changing part in copy.sgml the line width
> is less than 80 characters.
> but I also want to make sure "<>" "</>" within the same line.
> so after the change it becomes:
> 
>     <para>
>      <command>COPY TO</command> can be used with plain
>      tables and populated materialized views.
>      For example,
>      <literal>COPY <replaceable class="parameter">table</replaceable>
> TO</literal>
>      copies the same rows as
>      <literal>SELECT * FROM ONLY <replaceable
> class="parameter">table</replaceable></literal>.
>      However it doesn't directly support other relation types,
>      such as partitioned tables, inheritance child tables, or views.
> 
> 
> 
>> The tests seem to have been placed under the category "COPY FROM ... DEFAULT",
>> which feels a bit misaligned. How about adding them to the end of copy.sql instead?
>>
> ok.

Thanks for updating the patch! I made some minor cosmetic changes
and updated the commit log. The revised patch is attached.

Unless there are any objections, I'll proceed with committing it.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

  29. Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

    Fujii Masao <masao.fujii@oss.nttdata.com> — 2025-04-04T13:39:06Z

    
On 2025/04/03 20:46, Fujii Masao wrote:
> Thanks for updating the patch! I made some minor cosmetic changes
> and updated the commit log. The revised patch is attached.
> 
> Unless there are any objections, I'll proceed with committing it.

I've pushed the patch. Thanks!

As a follow-up, it might be worth enhancing psql to also
complete materialized view names after "COPY".

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION