Re: Speed dblink using alternate libpq tuple storage
Kyotaro Horiguchi <horiguchi.kyotaro@oss.ntt.co.jp>
From: Kyotaro HORIGUCHI <horiguchi.kyotaro@oss.ntt.co.jp>
To: markokr@gmail.com
Cc: greg@2ndquadrant.com, pgsql-hackers@postgresql.org, mmoncure@gmail.com, shigeru.hanada@gmail.com
Date: 2012-02-28T08:04:44Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix COPY FROM for null marker strings that correspond to invalid encoding.
- e8476f46fc84 9.2.0 cited
-
Improve labeling of pg_test_fsync open_sync test output.
- 2bbd88f8f841 9.2.0 cited
Attachments
- libpq_rowproc_20120228.patch (text/x-patch) patch
- libpq_rowproc_doc_20120228.patch (text/x-patch) patch
- dblink_use_rowproc_20120228.patch (text/x-patch) patch
- early_exit_20120228.diff (text/x-patch) patch
This is the new version of the patch.
It is not rebased to the HEAD because of a build error.
> It's better to restore old two-path error handling.
I restorerd "OOM and save result" route. But it seems needed to
get back any amount of memory on REAL OOM as the comment in
original code says.
So I restored the meaning of rp == 0 && errMsg == NULL as REAL
OOM which is to throw the async result away and the result will
be preserved if errMsg is not NULL. `unknown error' has been
removed.
As the result, if row processor returns 0 the parser skips to the
end of rows and returns the working result or an error result
according to whether errMsg is set or not in the row processor.
> I don't think that should be required. Just use a dummy msg.
Considering the above, pqAddRow is also restored to leave errMsg
NULL on OOM.
> There is still one EOF in v3 getAnotherTuple() -
> pqGetInt(tupnfields), please turn that one also to
> protocolerror.
pqGetInt() returns EOF only when it wants additional reading from
network if the parameter `bytes' is appropreate. Non-zero return
from it seems should be handled as EOF, not a protocol error. The
one point I had modified bugilly is also restored. The so-called
'protocol error' has been vanished eventually.
> Instead use ("%s", errmsg) as argument there. libpq code
> is noisy enough, no need to add more.
done
Is there someting left undone?
By the way, I noticed that dblink always says that the current
connection is 'unnamed' in messages the errors in
dblink_record_internal@dblink. I could see that
dblink_record_internal defines the local variable conname = NULL
and pass it to dblink_res_error to display the error message. But
no assignment on it in the function.
It seemed properly shown when I added the code to set conname
from PG_GETARG_TEXT_PP(0) if available, in other words do that
just after DBLINK_GET_CONN/DBLINK_GET_NAMED_CONN's. It seems the
dblink's manner... This is not included in this patch.
Furthurmore dblink_res_error looks only into returned PGresult to
display the error and always says only `Error occurred on dblink
connection..: could not execute query'..
Is it right to consider this as follows?
- dblink is wrong in error handling. A client of libpq should
see PGconn by PQerrorMessage() if (or regardless of whether?)
PGresult says nothing about error.
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center