Re: [BUGS] BUG #1347: Bulk Import stopps after a while (

Markus Schaber wrote:
> Hi, Oliver,
> 
> On Tue, 14 Dec 2004 09:58:16 +1300
> Oliver Jowett <oliver@opencloud.com> wrote:
> 
> 
>>Also, it's never going to be 100% without a separate thread, as the 
>>server can spontaneously generate output (e.g. because of NOTIFY) 
>>regardless of how careful we are with our queries.

I think my analysis here was wrong. It doesn't matter if the server 
unexpectedly fills up its output pipe so long as the driver doesn't do 
the same.

> Sadly, we cannot abandon compatibility with older JDK releases, this
> would allow us to use the new asynchroneous I/O interfaces.

Hm, is async I/O standardized yet? I thought JSR203 ("more NIO") covered 
this, but it's still in the early stages according to jcp.org.

> Async I/O is a great tool when in the right hands, you might google for
> Felix von Leitner and his "fnord" and "gatling" servers that even beat
> the kernel based khttpd. 

Current NIO isn't async I/O, but it's still fairly good (use at least 
1.4.2_06, though, or you can have GC problems when collecting direct 
buffers). We have written several resource adaptors that use it and they 
perform well. The main win is that you don't need N threads for N 
clients. That can't really be applied to the JDBC driver easily as the 
API means that the client controls most of the threading, but the 
ability to get at non-blocking primitives would still be useful.

I wonder if we can have some threads-or-NIO abstraction .. the main 
thing we need is the ability to call out to some other code to do reads 
either when we discover our writes will block (NIO) or when we have data 
ready to read (threads). We then read and process messages until we 
would block (NIO) or we *do* block (threads).

I doubt I'll have time to look at this in the near future though.

-O