RE: [PoC] pg_upgrade: allow to upgrade publisher node

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>

From: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
To: 'Amit Kapila' <amit.kapila16@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Bruce Momjian <bruce@momjian.us>, Julien Rouhaud <rjuju123@gmail.com>, vignesh C <vignesh21@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>
Date: 2023-08-18T03:12:19Z
Lists: pgsql-hackers
Dear Amit,

> > I have not used other plugins, but forcibly renamed the shared object file.
> > I would test by plugins like wal2json[1] if more cases are needed.
> >
> > 1. created logical replication slots on old node
> >   SELECT * FROM pg_create_logical_replication_slot('test', 'test_decoding')
> > 2. stopped the old nde
> > 3. forcibly renamed the so file. I used following script:
> >   sudo mv /path/to/test_decoding.so /path/to//test\"_decoding.so
> > 4. executed pg_upgrade and failed. Outputs what I got were:
> >
> > ```
> > Checking for presence of required libraries                 fatal
> >
> 
> Your test sounds reasonable but there is no harm in testing wal2json
> or some other plugin just to mimic the actual production scenario.
> Additionally, it would give us better coverage for the patch by
> testing out-of-core plugins for some other tests as well.

I've tested by using wal2json, decoder_raw[1], and my small decoder. The results were
the same: pg_upgrade correctly raised an ERROR. Following demo shows the case for wal2json.

In this test, the plugin was installed only on the old node and a slot was created.
Below shows the created slot:

```
(Old)=# SELECT slot_name, plugin FROM pg_replication_slots
slot_name |  plugin  
-----------+----------
 test      | wal2json
(1 row)
```

And I confirmed that the plugin worked well via pg_logical_slot_get_changes()
(This was needed to move forward the confirmed_flush_lsn)

```
(Old)=# INSERT INTO foo VALUES (1)
INSERT 0 1
(Old)=# SELECT * FROM pg_logical_slot_get_changes('test', NULL, NULL);
   lsn    | xid |                                                               data                                          
                     
----------+-----+-------------------------------------------------------------------------------------------------------------
---------------------
 0/63C8A8 | 731 | {"change":[{"kind":"insert","schema":"public","table":"foo","columnnames":["id"],"columntypes":["integer"],"
columnvalues":[1]}]}
(1 row)
```

Then the pg_upgrade was executed but failed, same as the previous example.

```
Checking for presence of required libraries                 fatal

Your installation references loadable libraries that are missing from the
new installation.  You can add these libraries to the new installation,
or remove the functions using them from the old installation.  A list of
problem libraries is in the file:
    data_N3/pg_upgrade_output.d/20230818T030006.675/loadable_libraries.txt
Failure, exiting
```

In the loadable_libraries.txt, it mentioned that wal2json was not installed to new directory.

```
could not load library "wal2json": ERROR:  could not access file "wal2json": No such file or directory
In database: postgres
```

Note that upgrade was done if the plugin was installed to new binary too.

Acknowledgement: Thank you Michael and Euler for creating great plugins!

[1]: https://github.com/michaelpq/pg_plugins

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Commits

  1. Fix issues in binary_upgrade_logical_slot_has_caught_up().

  2. Fix a random failure in 003_logical_slots.pl.

  3. Fix a test in 003_logical_slots.

  4. Fix uninitialized slot array access during the upgrade.

  5. Fix the test 003_logical_slots.

  6. Commit b195e6d482 forgot to update meson.build.

  7. Use shorter file names in the upgrade logical slots test.

  8. Migrate logical slots to the new node during an upgrade.

  9. Flush logical slots to disk during a shutdown checkpoint if required.

  10. Prevent possibility of panics during shutdown checkpoint.