Thread

Re: [HACKERS] Disk block size issues.

Bruce Momjian <maillist@candle.pha.pa.us> — 1998-01-09T17:28:30Z

> 
> > > A few things that I have noticed will be affected by allowing the
> > > disk block size to be other than 8k. (4k, 8k, 16k or 32k)
> > > 
> > > 1. Rules
> > > 
> > > The rule system currently stores plans as tuples in pg_rewrite.
> > > Making the block size smaller will accordingly reduce the size of
> > > the rules you can create.
> > 
> > I say make it match the given block size at compile time.
> 
> For now it does.  There's a comment in rewriteDefine.c though that
> indicates the original pg coders thought about putting the stored
> plans into large objects if 8k was too limiting.

Yep, I saw that too.

> Could be nice to have the type limits stored in a system table so
> the user or a program could query the limits of the current db.

Someday.

> 
> > > 2. Attribute limits
> > > 
> > > Should the size limits of the varchar/char be driven by the chosen
> > > block size?
> > 
> > Yes, they should be calculated based on the compile block size.
> > ...
> > Just make the max size based on the block size.
> > ...
> > This is an interesting point.  While we can compute most of the changes
> > at compile time, we will have to communicate with clients that were
> > compiled with different max limits.
> > 
> > I recommend we increase the max client buffer size to what we believe is
> > the largest block size anyone would ever reasonably choose.  That way,
> > all can communicate.  I recommend you contact Peter Mount for JDBC,
> > Openlink for ODBC, and all the other client maintainers and let them
> > know the changes will be in 6.3 so they can be ready with new version
> > when 6.3 starts beta on February 1.
> 
> So the buffer size will be defined in one place also that they should all
> reference when compiling or running?  In include/config.h I assume?

Yes, in config.h, and let's call it PG... so it is clear, and everything
can key off of that.

> 
> This could be difficult for the ODBC and JDBC drivers to determine
> automagically since they are usually compiled on different systems that
> the postgres src.

I think they will need to handle the maximum size someone could ever
choose.  Let's face it, 32k or 64k is not too much to ask for a buffer. 
I just hope there are not too many of them.  I only see it in one place
in libpq.  The others are malloc'ed based on how big the result is when
it comes back from the socket.

I recommend we add a test in config.h to make sure they do not set the
max size greater than some predefined limit, and mention why we test
there (for clients).  The interface/* files will not use the backend
block size, but will use another config.h define called PGMAXBLCKSZ, or
something like that, so they can interoperate will all backends.

> 
> Other stuff...
> 
> Could the block size be made into a command line option, like "-k 8192"?

Too scary for me.

> 
> Would only require that the BLCKSZ define become a variable and that it
> be passed to the backends too.  Much easier than having to recompile/install
> postgres to change the block size.  Could have multiple postmasters running
> different block-sized databases without having to have a binary around for
> each size.

Yes, we could do that, but if they ever start the postmaster with a
different value, he is lost.  I thought because of the bit fields and
cases where BLCKSZ is used in macros to define sized arrays that we
can't make it variable.

I think we should make it a config.h constant for now, but I am not firm
on this.

> 
> Renaming BLCKSZ...
> 
> How about PG_BLOCK_SIZE?  Or if it's made a variable, DiskBlockSize, keeping
> it in the tradition of SortMem, ShowStats, etc.

I like that new name.

-- 
Bruce Momjian
maillist@candle.pha.pa.us