Thread

Re: [HACKERS] data compression/encryption

Maurice Gittens <mgittens@gits.nl> — 1998-04-30T06:46:22Z

-----Original Message-----
From: Brett McCormick <brett@work.chicken.org>
To: pgsql-hackers@hub.org <pgsql-hackers@hub.org>
Date: donderdag 30 april 1998 10:42
Subject: [HACKERS] data compression/encryption


>
>from todo:
>
>Allow compression of large fields or a compressed field type
>
>I like this idea.  Should be pretty easy too.  Are we interested in
>putting this in the distribution, or as a contrib?  I could easily
>create a compressed field type like the text type.  However, how do
>you actually get the data in there?  Assuming you're trying to get
>around the 8k tuple limit, there's still the 8k query length.  Does
>copy do ok with >8k tuples (assuming the resulting tuple size is < 8k).
>
>Compression of large objects is also a good idea, but I'm not sure how
>it would be implemented, or how it would affect reads/writes (you
>can't really seek with zlib, which is what I would use).

>
>I've also been thinking about data encryption.  Assuming it would be
>too hard & long to revamp or add a new storage manager and actually
>encrypt the pages themselves, we can encrypt what gets stored in the
>field, and either have a type for it, or a function.  What about the
>idea of a 'data translator', a function which would act as a filter
>between the in/out functions and the actual storage of data on disk/in
>memory.  So that it could be applied to fields which would then be
>automagically compressed.

I've been looking at how information is stored at the lowest level in
postgresql,
and if I'm not mistaken compressing and/or encrypting of items on a
page is doable. Since items can be shuffeled around on a page without
changing their tid.

I haven't given much thought to how such functionality could be presented
to the user.

Regards,
    Maurice.