Re: backup manifests

tushar <tushar.ahuja@enterprisedb.com>

From: tushar <tushar.ahuja@enterprisedb.com>

To: Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>, Suraj Kharage <suraj.kharage@enterprisedb.com>

Cc: Robert Haas <robertmhaas@gmail.com>, Rushabh Lathia <rushabh.lathia@gmail.com>, Tels <nospam-pg-abuse@bloodgate.com>, David Steele <david@pgmasters.net>, Andrew Dunstan <andrew.dunstan@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Jeevan Chalke <jeevan.chalke@enterprisedb.com>, vignesh C <vignesh21@gmail.com>

Date: 2020-03-05T12:05:28Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Try to avoid compiler warnings in optimized builds.
- 05021a2c0cd2 13.0 landed
Fix option related issues in pg_verifybackup.
- 0a89e93bfaa6 13.0 landed
Add index term for backup manifest in documentation.
- 4db819ba4039 13.0 landed
Code review for backup manifest.
- a2ac73e7be7a 13.0 landed
Document the backup manifest file format.
- 149f2ae88ab0 13.0 landed
Fix typo in pg_validatebackup documentation.
- c4f82a779d26 13.0 landed
Exclude backup_manifest file that existed in database, from BASE_BACKUP.
- 1ec50a81ec0a 13.0 landed
Msys2 tweaks for pg_validatebackup corruption test
- c3e4cbaab936 13.0 landed
Fix resource management bug with replication=database.
- 3e0d80fd8d3d 13.0 cited
Be more careful about time_t vs. pg_time_t in basebackup.c.
- db1531cae009 13.0 cited
pg_validatebackup: Fix 'make clean' to remove tmp_check.
- 9f8f881caa0f 13.0 landed
pg_validatebackup: Also use perl2host in TAP tests.
- 460314db08e8 13.0 landed
Generate backup manifests for base backups, and validate them.
- 0d8c9c1210c4 13.0 landed
Add checksum helper functions.
- c12e43a2e0d4 13.0 landed
pg_waldump: Add a --quiet option.
- ac44367efbef 13.0 landed
Catversion bump for b9b408c48724
- afb5465e0cfc 13.0 cited
pg_basebackup: Refactor code for reading COPY and tar data.
- 431ba7bebf13 13.0 landed
Use a ResourceOwner to track buffer pins in all cases.
- 3cb646264e8c 12.0 cited
Use ARMv8 CRC instructions where available.
- f044d71e331d 11.0 cited
Logical replication support for initial data copy
- 7c4f52409a8c 10.0 cited
Use Intel SSE 4.2 CRC instructions where available.
- 3dc2d62d0486 9.5.0 cited
Switch to CRC-32C in WAL and other places.
- 5028f22f6eb0 9.5.0 cited
Remove support for 64-bit CRC.
- 404bc51cde9d 9.5.0 cited
Change CRCs in WAL records from 64bit to 32bit for performance reasons.
- 21fda22ec46d 8.1.0 cited

There is one small observation if we use slash (/) with option -i then 
not getting the desired result

Steps to reproduce -
==============

[centos@tushar-ldap-docker bin]$ ./pg_basebackup -D test

[centos@tushar-ldap-docker bin]$ touch test/*pg_notify*/dummy_file

--working
[centos@tushar-ldap-docker bin]$ ./pg_validatebackup   
--ignore=*pg_notify*  test
pg_validatebackup: * manifest_checksum = 
be9b72e1320c6c34c131533de19371a10dd5011940181724e43277f786026c7b
pg_validatebackup: backup successfully verified

--not working

[centos@tushar-ldap-docker bin]$ ./pg_validatebackup   
--ignore=*pg_notify/*  test
pg_validatebackup: * manifest_checksum = 
be9b72e1320c6c34c131533de19371a10dd5011940181724e43277f786026c7b
pg_validatebackup: error: "pg_notify/dummy_file" is present on disk but 
not in the manifest

regards,

On 3/5/20 3:40 PM, tushar wrote:
> Hi,
>
> There is one scenario  where  i somehow able to run pg_validatebackup 
> successfully but when i tried to start the server , it is failing
>
> Steps to reproduce -
> --create 2 base backup directory
> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D db1
> [centos@tushar-ldap-docker bin]$ ./pg_basebackup -D db2
>
> --run pg_validatebackup , use backup_manifest of db1 directory 
> against  db2/  . Will get an error
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m 
> db1/backup_manifest db2/
> pg_validatebackup: * manifest_checksum = 
> 5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
> pg_validatebackup: error: checksum mismatch for file "backup_label"
> --copy the backup_level of db1 to db2 folder
> [centos@tushar-ldap-docker bin]$ cp db1/backup_label db2/.
>
> --run pg_validatebackup .. working fine
> [centos@tushar-ldap-docker bin]$ ./pg_validatebackup -m 
> db1/backup_manifest db2/
> pg_validatebackup: * manifest_checksum = 
> 5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
> pg_validatebackup: backup successfully verified
> [centos@tushar-ldap-docker bin]$
>
> --try to start the server
> [centos@tushar-ldap-docker bin]$ ./pg_ctl -D db2 start -o '-p 7777'
> waiting for server to start....2020-03-05 15:33:53.471 IST [24049] 
> LOG:  starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by 
> gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
> 2020-03-05 15:33:53.471 IST [24049] LOG:  listening on IPv6 address 
> "::1", port 7777
> 2020-03-05 15:33:53.471 IST [24049] LOG:  listening on IPv4 address 
> "127.0.0.1", port 7777
> 2020-03-05 15:33:53.473 IST [24049] LOG:  listening on Unix socket 
> "/tmp/.s.PGSQL.7777"
> 2020-03-05 15:33:53.476 IST [24050] LOG:  database system was 
> interrupted; last known up at 2020-03-05 15:32:51 IST
> 2020-03-05 15:33:53.573 IST [24050] LOG:  invalid checkpoint record
> 2020-03-05 15:33:53.573 IST [24050] FATAL:  could not locate required 
> checkpoint record
> 2020-03-05 15:33:53.573 IST [24050] HINT:  If you are restoring from a 
> backup, touch 
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/recovery.signal" and 
> add required recovery options.
>     If you are not restoring from a backup, try removing the file 
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label".
>     Be careful: removing 
> "/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label" will 
> result in a corrupt cluster if restoring from a backup.
> 2020-03-05 15:33:53.574 IST [24049] LOG:  startup process (PID 24050) 
> exited with exit code 1
> 2020-03-05 15:33:53.574 IST [24049] LOG:  aborting startup due to 
> startup process failure
> 2020-03-05 15:33:53.575 IST [24049] LOG:  database system is shut down
>  stopped waiting
> pg_ctl: could not start server
> Examine the log output.
> [centos@tushar-ldap-docker bin]$
>
> regards,
>
>
> On 3/5/20 1:09 PM, Rajkumar Raghuwanshi wrote:
>> Hi,
>>
>> In a negative test scenario, if I changed size to -1 in 
>> backup_manifest, pg_validatebackup giving
>> error with a random size number.
>>
>> [edb@localhost bin]$ ./pg_basebackup -p 5551 -D /tmp/bold 
>> --manifest-checksum 'SHA256'
>> [edb@localhost bin]$ ./pg_validatebackup /tmp/bold
>> pg_validatebackup: backup successfully verified
>>
>> --change a file size to -1 and generate new checksum.
>> [edb@localhost bin]$ vi /tmp/bold/backup_manifest
>> [edb@localhost bin]$ shasum -a256 /tmp/bold/backup_manifest
>> c3d7838cbbf991c6108f9c1ab78f673c20d8073114500f14da6ed07ede2dc44a 
>>  /tmp/bold/backup_manifest
>> [edb@localhost bin]$ vi /tmp/bold/backup_manifest
>>
>> [edb@localhost bin]$ ./pg_validatebackup /tmp/bold
>> pg_validatebackup: error: "global/4183" has size 0 on disk but size 
>> *18446744073709551615* in the manifest
>>
>> Thanks & Regards,
>> Rajkumar Raghuwanshi
>>
>>
>> On Thu, Mar 5, 2020 at 9:37 AM Suraj Kharage 
>> <suraj.kharage@enterprisedb.com 
>> <mailto:suraj.kharage@enterprisedb.com>> wrote:
>>
>>
>>     On Wed, Mar 4, 2020 at 7:21 PM tushar
>>     <tushar.ahuja@enterprisedb.com
>>     <mailto:tushar.ahuja@enterprisedb.com>> wrote:
>>
>>         Hi,
>>
>>         There is a scenario in which i add something inside the
>>         pg_tablespace directory , i am getting an error like-
>>
>>         pg_validatebackup: * manifest_checksum =
>>         77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
>>         pg_validatebackup: error:
>>         "pg_tblspc/16385/*PG_13_202002271*/test" is present on disk
>>         but not in the manifest
>>
>>         but if i remove 'PG_13_202002271 ' directory then there is no
>>         error
>>
>>         [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data
>>         pg_validatebackup: * manifest_checksum =
>>         77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
>>         pg_validatebackup: backup successfully verified
>>
>>
>>     This seems expected considering current design as we don't log
>>     the directory entries in backup_manifest. In your case, you have
>>     tablespace with no objects (empty tablespace) then
>>     backup_manifest does not have any entry for this hence when you
>>     remove this tablespace directory, validator could not detect it.
>>
>>     We can either document it or add the entry for directories in the
>>     manifest. Robert may have a better idea on this.
>>
>>     -- 
>>     -- 
>>
>>     Thanks & Regards,
>>     Suraj kharage,
>>     EnterpriseDB Corporation,
>>     The Postgres Database Company.
>>
>
> -- 
> regards,tushar
> EnterpriseDBhttps://www.enterprisedb.com/
> The Enterprise PostgreSQL Company


-- 
regards,tushar
EnterpriseDB  https://www.enterprisedb.com/
The Enterprise PostgreSQL Company