Drawbacks of using BYTEA for PK?

Are there any drawbacks of using BYTEA for PK compared to using a 
primitive/atomic data types like INT/SERIAL? (like significant 
performance hit, peculiar FK behaviour, etc).

I plan to use BYTEA for GUID (of course, temporarily I hope, until 
PostgreSQL officially supports GUID data type), since it seems to be the 
most convenient+compact compared to other data types currently 
available. I use GUIDs for most PK columns.

-- 
dave