alter table xxx set unlogged take long time

Hi ,
We have PG v13.6 in RHEL8.4, we try to set table unlogged before load data.  There are a lot of existing data in this table, when 'alter table xxx  set unlogged', we found it take long time and spend time on IO datafileread.  Is it expected?

Thanks,

James