Horton
Menu

Thread

  1. Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-11T01:05:30Z

    I've been experimenting with using a different parser (one which is
more Oracle compatible).  While the parser is not yet ready for prime
time, I thought I would send out the code used to select the parser to
gauge its level of acceptability.  If this patch isn't going to fly,
then my approach to an alternate parser isn't going to fly either.

After this patch is applied, you can switch to a new parser by doing
something like this:

create function my_parser(opaque, opaque, opaque) returns opaque
  as '..../parser.so' language 'c';
set parser = my_parser;

After you do this, all subsequent input will be interpreted using the
specified parser.  Note that you may want to leave yourself an escape
hatch of some sort to set the parser back to Postgres standard.

If this patch is accepted, then some further work needs to be done to
set the parser for SPI calls, so that it is possible for the user to
change the parser while still using ordinary PL/pgSQL.

I would appreciate any feedback.

Ian

Index: src/backend/tcop/postgres.c
===================================================================
RCS file: /home/projects/pgsql/cvsroot/pgsql/src/backend/tcop/postgres.c,v
retrieving revision 1.230
diff -u -r1.230 postgres.c
--- src/backend/tcop/postgres.c	2001/08/04 00:14:43	1.230
+++ src/backend/tcop/postgres.c	2001/08/11 00:53:08
@@ -58,9 +58,11 @@
 #include "tcop/utility.h"
 #include "storage/proc.h"
 #include "utils/exc.h"
+#include "utils/fcache.h"
 #include "utils/guc.h"
 #include "utils/memutils.h"
 #include "utils/ps_status.h"
+#include "utils/syscache.h"
 #ifdef MULTIBYTE
 #include "mb/pg_wchar.h"
 #endif
@@ -118,6 +120,10 @@
 */
 int			XfuncMode = 0;
 
+char *parser_function_name = 0;
+static bool update_parser_function_fcache = true;
+static FunctionCachePtr parser_function_fcache = NULL;
+
 /* ----------------------------------------------------------------
  *		decls for routines only used in this file
  * ----------------------------------------------------------------
@@ -389,8 +395,23 @@
 
 	if (Show_parser_stats)
 		ResetUsage();
+
+	if (update_parser_function_fcache)
+		assign_parser(parser_function_name);
+
+	if (parser_function_fcache == NULL)
+		raw_parsetree_list = parser(query_string, typev, nargs);
+	else
+	{
+		Datum	result;
+
+		parser_function_fcache->fcinfo.arg[0] = PointerGetDatum(query_string);
+		parser_function_fcache->fcinfo.arg[1] = PointerGetDatum(typev);
+		parser_function_fcache->fcinfo.arg[2] = Int32GetDatum(nargs);
 
-	raw_parsetree_list = parser(query_string, typev, nargs);
+		result = FunctionCallInvoke(&parser_function_fcache->fcinfo);
+		raw_parsetree_list = (List *) DatumGetPointer(result);
+	}
 
 	if (Show_parser_stats)
 	{
@@ -399,6 +420,82 @@
 	}
 
 	return raw_parsetree_list;
+}
+
+/*
+ * Check that we can find a parser function.  This is called when the
+ * user assigns a value to the `parser' variable.
+ */
+bool
+check_parser(const char *proposed)
+{
+	HeapTuple	tup;
+	Oid			argtypes[FUNC_MAX_ARGS];
+
+	if (proposed[0] == '\0' || strcmp(proposed, "postgres") == 0)
+		return true;
+
+	/* We can't check this unless we have started running.  */
+	if (! IsNormalProcessingMode())
+		return true;
+
+	memset(argtypes, 0, sizeof argtypes);
+	tup = SearchSysCache(PROCNAME,
+						 PointerGetDatum(proposed),
+						 Int32GetDatum(3),
+						 PointerGetDatum(argtypes),
+						 0);
+	if (! HeapTupleIsValid(tup))
+		return false;
+	ReleaseSysCache(tup);
+	return true;
+}
+
+/*
+ * Assign a new parser function.
+ */
+void
+assign_parser(const char *value)
+{
+	FunctionCachePtr	fcache;
+	HeapTuple			tup;
+	Oid					argtypes[FUNC_MAX_ARGS];
+	Oid					oid;
+
+	/* We can't update parser_function_fcache until we have started
+	 * running.
+	 */
+	if (! IsNormalProcessingMode())
+	{
+		update_parser_function_fcache = true;
+		return;
+	}
+
+	if (value[0] == '\0' || strcmp(value, "postgres") == 0)
+		fcache = NULL;
+	else
+	{
+		memset(argtypes, 0, sizeof argtypes);
+		tup = SearchSysCache(PROCNAME,
+							 PointerGetDatum(value),
+							 Int32GetDatum(3),
+							 PointerGetDatum(argtypes),
+							 0);
+		if (! HeapTupleIsValid(tup))
+			elog(ERROR, "parser function %s does not exist", value);
+
+		oid = tup->t_data->t_oid;
+
+		ReleaseSysCache(tup);
+
+		fcache = init_fcache(oid, 3, TopMemoryContext);
+	}
+
+	if (parser_function_fcache != NULL)
+		pfree(parser_function_fcache);
+	parser_function_fcache = fcache;
+
+	update_parser_function_fcache = false;
 }
 
 /*
Index: src/backend/utils/misc/guc.c
===================================================================
RCS file: /home/projects/pgsql/cvsroot/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.45
diff -u -r1.45 guc.c
--- src/backend/utils/misc/guc.c	2001/07/05 15:19:40	1.45
+++ src/backend/utils/misc/guc.c	2001/08/11 00:53:11
@@ -376,6 +376,9 @@
 	{"krb_server_keyfile", PGC_POSTMASTER, &pg_krb_server_keyfile,
 	PG_KRB_SRVTAB, NULL, NULL},
 
+	{"parser", PGC_USERSET, &parser_function_name, "",
+	 check_parser, assign_parser},
+
 #ifdef ENABLE_SYSLOG
 	{"syslog_facility", PGC_POSTMASTER, &Syslog_facility,
 	"LOCAL0", check_facility, NULL},
Index: src/include/tcop/tcopprot.h
===================================================================
RCS file: /home/projects/pgsql/cvsroot/pgsql/src/include/tcop/tcopprot.h,v
retrieving revision 1.41
diff -u -r1.41 tcopprot.h
--- src/include/tcop/tcopprot.h	2001/06/08 21:16:48	1.41
+++ src/include/tcop/tcopprot.h	2001/08/11 00:53:12
@@ -31,6 +31,8 @@
 extern bool HostnameLookup;
 extern bool ShowPortNumber;
 
+extern char *parser_function_name;
+
 #ifndef BOOTSTRAP_INCLUDE
 
 extern List *pg_parse_and_rewrite(char *query_string,
@@ -39,6 +41,8 @@
 extern void pg_exec_query_string(char *query_string,
 					 CommandDest dest,
 					 MemoryContext parse_context);
+extern bool check_parser(const char *proposed);
+extern void assign_parser(const char *value);
 
 #endif	 /* BOOTSTRAP_INCLUDE */

  2. Re: Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-11T15:50:30Z

    Ian Lance Taylor <ian@airs.com> writes:
> I've been experimenting with using a different parser (one which is
> more Oracle compatible).

Hmm.  What we have here is a mechanism to swap out the entire
backend/parser/ subdirectory --- and nothing else.  Somehow this seems
like the wrong granularity.  parser/ is an awful lot of code to replace
to make a few random tweaks that don't affect query semantics.  Since
you aren't changing the querytree representation nor any of the rewrite/
plan/execute pipeline, it's hard to see how you can do more with this
than very marginal syntax hacks.  But if that's all you want to do,
seems like replacing pieces of the parser/semantic analyzer is the right
mechanism, not the whole durn thing.

> Note that you may want to leave yourself an escape
> hatch of some sort to set the parser back to Postgres standard.
> If this patch is accepted, then some further work needs to be done to
> set the parser for SPI calls, so that it is possible for the user to
> change the parser while still using ordinary PL/pgSQL.

I think both of these issues would need to be addressed before, not
after, considering the patch for acceptance.  In particular, how do we
cater for both plpgsql and a true "PL/SQL" language that uses the Oracle
parser?

> +	{
> +		Datum	result;
> +
> +		parser_function_fcache->fcinfo.arg[0] = PointerGetDatum(query_string);
> +		parser_function_fcache->fcinfo.arg[1] = PointerGetDatum(typev);
> +		parser_function_fcache->fcinfo.arg[2] = Int32GetDatum(nargs);
 
> -	raw_parsetree_list = parser(query_string, typev, nargs);
> +		result = FunctionCallInvoke(&parser_function_fcache->fcinfo);
> +		raw_parsetree_list = (List *) DatumGetPointer(result);
> +	}

Use FunctionCall3() to hide the cruft here.

> +		fcache = init_fcache(oid, 3, TopMemoryContext);

This is a tad odd.  Why don't you just do fmgr_info and store an FmgrInfo
structure?  You have no use for the rest of an executor fcache
structure.

The update_parser_function_fcache business bothers me, too.  I see the
problem: it doesn't work to do this lookup when postgresql.conf is read
in the postmaster.  However, I really don't like the notion of disabling
check_parser and allowing a possibly-bogus value to be assigned on
faith.  (If the function name *is* bogus, your code can never recover;
at the very least you ought to clear update_parser_function_fcache
before failing.)  Given the extent to which the parser is necessarily
tied to the rest of the system, I'm not sure there's any value in
allowing it to be implemented as a dynamic-link function.  I'd be more
than half inclined to go with a lower-tech solution wherein you expect
the alternate parser to be permanently linked in and known to the
check_parser and assign_parser subroutines.  Then the state variable is
just a C function pointer, and assign_parser looks something like

#ifdef ORACLE_PARSER_SUPPORTED
	if (strcasecmp(value, "Oracle") == 0)
		parser_fn = oracle_parser;
	else
#endif

			regards, tom lane

  3. Re: Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-11T17:01:58Z

    Tom Lane <tgl@sss.pgh.pa.us> writes:

> Ian Lance Taylor <ian@airs.com> writes:
> > I've been experimenting with using a different parser (one which is
> > more Oracle compatible).
> 
> Hmm.  What we have here is a mechanism to swap out the entire
> backend/parser/ subdirectory --- and nothing else.  Somehow this seems
> like the wrong granularity.  parser/ is an awful lot of code to replace
> to make a few random tweaks that don't affect query semantics.  Since
> you aren't changing the querytree representation nor any of the rewrite/
> plan/execute pipeline, it's hard to see how you can do more with this
> than very marginal syntax hacks.  But if that's all you want to do,
> seems like replacing pieces of the parser/semantic analyzer is the right
> mechanism, not the whole durn thing.

This patch doesn't actually replace the entire backend/parser
subdirectory.  It mainly only replaces scan.l and gram.y.  This is
because the code in postgres.c still passes the result of the replaced
parser to pg_analyze_and_rewrite().

You are absolutely correct that at this point this can only do
marginal syntax hacks.  But since my goal is to support Oracle syntax,
marginal syntax hacks are just what is needed.  For example, it's hard
to tweak the existing parser to support Oracle, because the set of
reserved words is different, and because Oracle uses different names
for datatypes.  Also, CREATE FUNCTION in Oracle doesn't use a literal
string, it actually switches to a different parser while parsing the
function.  It's not impossible to hack these into the Postgres parser;
just hard, and a complex maintenance problem.

It's true that increased Oracle compatibility will require code
changes in other parts of the backend.

> > Note that you may want to leave yourself an escape
> > hatch of some sort to set the parser back to Postgres standard.
> > If this patch is accepted, then some further work needs to be done to
> > set the parser for SPI calls, so that it is possible for the user to
> > change the parser while still using ordinary PL/pgSQL.
> 
> I think both of these issues would need to be addressed before, not
> after, considering the patch for acceptance.  In particular, how do we
> cater for both plpgsql and a true "PL/SQL" language that uses the Oracle
> parser?

I agree that these issues need to be addressed before the patch is
accepted.  I want to get a sense of whether I am on the right track at
all.

As noted above, PL/SQL needs to be more closely tied to the Oracle
parser than PL/pgSQL is to the Postgres parser.

I have been thinking in terms of a parser stack.  Code which uses SPI
could push the correct parser on top of the stack, and pop it off when
done.  Perhaps SPI would always use the postgres parser unless
explicitly directed otherwise.  The PL/SQL language interface would
direct otherwise.

Thanks for your other comments.  I went with a dynamic link approach
because it seemed minimally intrusive.  Using a straight function
pointer is certainly easier.

Ian

  4. Re: Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-11T20:35:14Z

    Ian Lance Taylor <ian@airs.com> writes:
> This patch doesn't actually replace the entire backend/parser
> subdirectory.  It mainly only replaces scan.l and gram.y.  This is
> because the code in postgres.c still passes the result of the replaced
> parser to pg_analyze_and_rewrite().

Oh, of course, how silly of me.  I was thinking that that call did the
analyze step too, but you're correct that it does not.  Okay, replacing
lexer+syntaxer is a more reasonable chunk-size.  (AFAIK there's no good
way to replace just part of a yacc/bison grammar on the fly, so you
couldn't go to a finer grain anyway, could you?)

			regards, tom lane

  5. Re: [PATCHES] Select parser at runtime

    Peter Eisentraut <peter_e@gmx.net> — 2001-08-11T20:59:49Z

    Ian Lance Taylor writes:

> I've been experimenting with using a different parser (one which is
> more Oracle compatible).

Why aren't you trying to add the missing pieces to the existing parser?

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

  6. Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-11T23:35:49Z

    Peter Eisentraut <peter_e@gmx.net> writes:

> Ian Lance Taylor writes:
> 
> > I've been experimenting with using a different parser (one which is
> > more Oracle compatible).
> 
> Why aren't you trying to add the missing pieces to the existing parser?

Because my goal is a parser which can accept Oracle code directly, and
Oracle does not use the same SQL syntax as Postgres.  They are, of
course, very similar, but it is not the case that all the differences
are items missing from the Postgres parser.  Some of them are items
which Postgres does in a different, typically more standards-
conforming, way.

For example: the datatypes have different names; the set of reserved
words is different; Oracle uses a weird syntax for outer joins.

Ian

  7. Re: Re: [PATCHES] Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-12T00:24:06Z

    Ian Lance Taylor <ian@airs.com> writes:
> For example: the datatypes have different names; the set of reserved
> words is different; Oracle uses a weird syntax for outer joins.

Is it really possible to fix these things strictly in the parser
(ie, without any semantic analysis)?  For example, I don't quite see
how you're going to translate Oracle-style outer joins to SQL standard
style without figuring out which fields belong to which relations.
Keep in mind the cardinal rule for the parsing step: Thou Shalt Not
Do Any Database Access (because the parser must work even in
transaction-aborted state, else how do we recognize ROLLBACK command?)

			regards, tom lane

  8. Re: [PATCHES] Select parser at runtime

    Peter Eisentraut <peter_e@gmx.net> — 2001-08-12T00:26:51Z

    Ian Lance Taylor writes:

> Because my goal is a parser which can accept Oracle code directly, and
> Oracle does not use the same SQL syntax as Postgres.  They are, of
> course, very similar, but it is not the case that all the differences
> are items missing from the Postgres parser.  Some of them are items
> which Postgres does in a different, typically more standards-
> conforming, way.

I'm not sure whether I like the notion of having to maintain multiple
parsers in the future.  We have always been quite fair in accepting
extensions and aliases for compatibility, so I don't see a problem there.
Then again, we're implemented an SQL server, not an Oracle server.  If you
want to convert your application there's this ora2pg thing.

> For example: the datatypes have different names; the set of reserved
> words is different;

Unless you have implemented a different parsing algorithm or want to rip
out features you're going to have a hard time changing the set of reserved
words.

> Oracle uses a weird syntax for outer joins.

We had already rejected this idea.  The earlier this disappears from the
face of the earth the better.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

  9. Re: Re: [PATCHES] Select parser at runtime

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-08-12T01:58:35Z

    > > Oracle uses a weird syntax for outer joins.
> 
> We had already rejected this idea.  The earlier this disappears from the
> face of the earth the better.

Sure, but until that happens, if we can easily give people portability,
why not?  My idea was to have a gram.y that rewrote _some_ of the
commands, and output another query string that could then be passed into
the existing gram.y.  Kind of crazy, but it may limit the duplication
of code in the two gram.y's.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  10. Re: Re: [PATCHES] Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-12T02:47:14Z

    Peter Eisentraut <peter_e@gmx.net> writes:
>> Oracle uses a weird syntax for outer joins.

> We had already rejected this idea.  The earlier this disappears from the
> face of the earth the better.

Now now, what about our goal of Postgres world domination?  Gonna be
tough to get there unless we can assimilate Oracle users ;-)

I do want to keep that brain-damaged syntax at arm's length, which
suggests a separate parser rather than merging it into our main
parser.  I'm not convinced a translation can be done at the grammar
level with no semantic analysis --- but if Ian thinks he can do it,
let him try...

			regards, tom lane

  11. Re: Re: [PATCHES] Select parser at runtime

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-08-12T04:25:28Z

    > Peter Eisentraut <peter_e@gmx.net> writes:
> >> Oracle uses a weird syntax for outer joins.
> 
> > We had already rejected this idea.  The earlier this disappears from the
> > face of the earth the better.
> 
> Now now, what about our goal of Postgres world domination?  Gonna be
> tough to get there unless we can assimilate Oracle users ;-)

"Fortress PostgreSQL",  Lamar Owen said.  We do have fortress feel,
don't we?


> I do want to keep that brain-damaged syntax at arm's length, which
> suggests a separate parser rather than merging it into our main
> parser.  I'm not convinced a translation can be done at the grammar
> level with no semantic analysis --- but if Ian thinks he can do it,
> let him try...

My guess is that he is going to need some changes in the /parser but
those will be minor and triggered by some Oracle flag.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  12. Re: Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-12T06:27:28Z

    Tom Lane <tgl@sss.pgh.pa.us> writes:

> Ian Lance Taylor <ian@airs.com> writes:
> > For example: the datatypes have different names; the set of reserved
> > words is different; Oracle uses a weird syntax for outer joins.
> 
> Is it really possible to fix these things strictly in the parser
> (ie, without any semantic analysis)?  For example, I don't quite see
> how you're going to translate Oracle-style outer joins to SQL standard
> style without figuring out which fields belong to which relations.
> Keep in mind the cardinal rule for the parsing step: Thou Shalt Not
> Do Any Database Access (because the parser must work even in
> transaction-aborted state, else how do we recognize ROLLBACK command?)

I admit that I haven't sorted out the outer join thing yet.  The
others are easy enough.

Ian

  13. Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-12T06:31:39Z

    Peter Eisentraut <peter_e@gmx.net> writes:

> I'm not sure whether I like the notion of having to maintain multiple
> parsers in the future.  We have always been quite fair in accepting
> extensions and aliases for compatibility, so I don't see a problem there.
> Then again, we're implemented an SQL server, not an Oracle server.  If you
> want to convert your application there's this ora2pg thing.

An approach such as ora2pg solves a specific problem.  It doesn't
really solve the general problem of people familiar with Oracle who
want to use Postgres.  It doesn't at all solve my problem, which is to
have an application which can easily speak to either Oracle or
Postgres.

> > For example: the datatypes have different names; the set of reserved
> > words is different;
> 
> Unless you have implemented a different parsing algorithm or want to rip
> out features you're going to have a hard time changing the set of reserved
> words.

That's why I'm implementing a different parsing algorithm.

Ian

  14. Re: Re: [PATCHES] Select parser at runtime

    Peter Eisentraut <peter_e@gmx.net> — 2001-08-12T16:23:38Z

    Tom Lane writes:

> Now now, what about our goal of Postgres world domination?  Gonna be
> tough to get there unless we can assimilate Oracle users ;-)

In order to achieve world domination you don't want to offer
compatibility, otherwise your users could move back and forth easily.
What you want is static conversion tools so people can move to your
product but not back to others.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

  15. Re: Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-12T19:19:32Z

    Peter Eisentraut <peter_e@gmx.net> writes:

> Tom Lane writes:
> 
> > Now now, what about our goal of Postgres world domination?  Gonna be
> > tough to get there unless we can assimilate Oracle users ;-)
> 
> In order to achieve world domination you don't want to offer
> compatibility, otherwise your users could move back and forth easily.
> What you want is static conversion tools so people can move to your
> product but not back to others.

I disagree.  To achieve world domination you should lower to barriers
to adoption as much as possible, and then keep people with you due to
the superiority of your product.  If the barriers to adoption are
high, people won't take the risk, and won't discover the superiority.

Incompatible syntax is a barrier to adoption because people fear the
time required to learn the new syntax, and they fear adopting Postgres
and then discovering after three months of enhancements to their
Postgres code that Postgres won't do the job and they have to switch
back.

Ian

  16. Re: Re: [PATCHES] Select parser at runtime

    Bruce Momjian <pgman@candle.pha.pa.us> — 2001-08-12T21:22:35Z

    > Tom Lane <tgl@sss.pgh.pa.us> writes:
> 
> > Ian Lance Taylor <ian@airs.com> writes:
> > > For example: the datatypes have different names; the set of reserved
> > > words is different; Oracle uses a weird syntax for outer joins.
> > 
> > Is it really possible to fix these things strictly in the parser
> > (ie, without any semantic analysis)?  For example, I don't quite see
> > how you're going to translate Oracle-style outer joins to SQL standard
> > style without figuring out which fields belong to which relations.
> > Keep in mind the cardinal rule for the parsing step: Thou Shalt Not
> > Do Any Database Access (because the parser must work even in
> > transaction-aborted state, else how do we recognize ROLLBACK command?)
> 
> I admit that I haven't sorted out the outer join thing yet.  The
> others are easy enough.
> 

Another idea is to put the Oracle stuff in gram.y, but use #ifdef or
something to mark the Oracle parts, and run gram.y through yacc/bison
with the Oracle defines visiable, and another time to create a second
parse state machine without Oracle.  I think Jan did that for something
like that once.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

  17. Re: Re: [PATCHES] Select parser at runtime

    Justin Clift <justin@postgresql.org> — 2001-08-12T22:17:17Z

    Hi guys,

Not sure if Peter was joking, but Ian's approach sounds much more
user-friendly.

Getting Oracle users to convert to PostgreSQL then be "stuck-with-it"
because they can't afford the migration elsewhere is not the right
approach.

PostgreSQL is a really good product, and the best way to emphasise it is
"here's PostgreSQL, people use it coz it *works better*".

And that's definitely achieveable.

:)

Regards and best wishes,

Justin Clift


Ian Lance Taylor wrote:
> 
> Peter Eisentraut <peter_e@gmx.net> writes:
> 
> > Tom Lane writes:
> >
> > > Now now, what about our goal of Postgres world domination?  Gonna be
> > > tough to get there unless we can assimilate Oracle users ;-)
> >
> > In order to achieve world domination you don't want to offer
> > compatibility, otherwise your users could move back and forth easily.
> > What you want is static conversion tools so people can move to your
> > product but not back to others.
> 
> I disagree.  To achieve world domination you should lower to barriers
> to adoption as much as possible, and then keep people with you due to
> the superiority of your product.  If the barriers to adoption are
> high, people won't take the risk, and won't discover the superiority.
> 
> Incompatible syntax is a barrier to adoption because people fear the
> time required to learn the new syntax, and they fear adopting Postgres
> and then discovering after three months of enhancements to their
> Postgres code that Postgres won't do the job and they have to switch
> back.
> 
> Ian
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly

-- 
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
     - Indira Gandhi

  18. Re: Re: [PATCHES] Select parser at runtime

    Vince Vielhaber <vev@michvhf.com> — 2001-08-13T02:21:55Z

    On Mon, 13 Aug 2001, Justin Clift wrote:

> Hi guys,
>
> Not sure if Peter was joking, but Ian's approach sounds much more
> user-friendly.
>
> Getting Oracle users to convert to PostgreSQL then be "stuck-with-it"
> because they can't afford the migration elsewhere is not the right
> approach.

If you think that people are going to flock to PostgreSQL from Oracle
simply because it's a drop in replacement, I want some of whatever it
is you're drinking!

An Oracle compatibility mode wouldn't be a bad idea, but at what cost
and at how much effort?  What are you going to do with incompatible
reserved words?  Who do you expect to do it?  How soon?  I've seen
alot of projects try to make themselves "user-friendly" only to suffer
in the end from what they lost in the effort.

Personally I'd prefer a PostgreSQL that was as SQL92 and beyond as it
could possibly be rather than some of this and some of that.

Vince.
-- 
==========================================================================
Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
         56K Nationwide Dialup from $16.00/mo at Pop4 Networking
        Online Campground Directory    http://www.camping-usa.com
       Online Giftshop Superstore    http://www.cloudninegifts.com
==========================================================================

  19. Re: Re: [PATCHES] Select parser at runtime

    Ross J. Reedstrom <reedstrm@rice.edu> — 2001-08-13T03:13:46Z

    On Sun, Aug 12, 2001 at 10:21:55PM -0400, Vince Vielhaber wrote:
> 
> If you think that people are going to flock to PostgreSQL from Oracle
> simply because it's a drop in replacement, I want some of whatever it
> is you're drinking!
> 
> An Oracle compatibility mode wouldn't be a bad idea, but at what cost
> and at how much effort?  What are you going to do with incompatible
> reserved words?  Who do you expect to do it?  How soon?  I've seen
> alot of projects try to make themselves "user-friendly" only to suffer
> in the end from what they lost in the effort.
> 
> Personally I'd prefer a PostgreSQL that was as SQL92 and beyond as it
> could possibly be rather than some of this and some of that.

Compatability modes have a good side and a bad side: they make it easy for
DBAs to move existing installations to your product at little cost, then
slowly upgrade to using native features. The OpenACS project would have
loved to have this possibility. Many of our current Oracle compatability
features came about to ease such ports, right?

However, they suffer from the problem of allowing continued operation in
'compatability' mode: there is no incentive for DB using project to use
the native interfaces or capabilites, even if they are better. This
is how OS/2's Windows compatabilty mode killed it in the long term:
the 'defacto standard' problem. Would the OpenACS project even exist,
if a full Oracle mode had existed at the time?

So, I think you're right, in the long run it's best to conform to an open
standard than try to chase a commercial product.

Ross

P.S. Of course, as an open project, anyone can write anything they
want. If we end up with a 'better Oracle than Oracle', so be it.

  20. Re: Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-13T04:51:55Z

    Vince Vielhaber <vev@michvhf.com> writes:

> An Oracle compatibility mode wouldn't be a bad idea, but at what cost
> and at how much effort?

That is why I focused on the relatively minor changes to Postgres
required to hook in an alternate parser.  I personally would not
expect the mainline Postgres team to worry about Oracle support.  But
if an Oracle parser can be decoupled from the mainline of Postgres
work, then most of the cost will be paid by the people who care about
it.  (Not all of the cost, because some communication will be required
when the parse tree nodes are changed.)

Along these lines, I don't think Bruce's suggestion of modifications
to the Postgres gram.y is a good idea, because it causes the Oracle
parser to add an ongoing cost to the Postgres parser.

Ian

  21. Re: Re: [PATCHES] Select parser at runtime

    Justin Clift <justin@postgresql.org> — 2001-08-13T13:25:42Z

    Hi Vince,

The point I'll make is this :

People who presently have installations on Oracle will be more inclined
to test/trial PostgreSQL if they know the learning curve is much less
than say, migrating to DB2 would be (or some other database without
specific Oracle-transition compatibilities).

Sure, they might move their installations to
PostgreSQL-with-an-Oracle-like-parser and then never convert them to
pure PostgreSQL.  So?  Does it matter?  Probably not, they're still
using PostgreSQL.  I'm pretty sure over time newer projects and
installations would become more PostgreSQL oriented as the DBA's gained
more experience and understanding of PostgreSQL's strengths.  i.e. 
"Chalk up a win."

Vince Vielhaber wrote:
> 
> On Mon, 13 Aug 2001, Justin Clift wrote:
> 
> > Hi guys,
> >
> > Not sure if Peter was joking, but Ian's approach sounds much more
> > user-friendly.
> >
> > Getting Oracle users to convert to PostgreSQL then be "stuck-with-it"
> > because they can't afford the migration elsewhere is not the right
> > approach.
> 
> If you think that people are going to flock to PostgreSQL from Oracle
> simply because it's a drop in replacement, I want some of whatever it
> is you're drinking!

If PostgreSQL was truly a drop-in-replacement then cost and good
reputation (especially over the coming years) would mean a lot of places
would use us instead of Oracle.  Presently though, we're not a
drop-in-replacement.

> An Oracle compatibility mode wouldn't be a bad idea, but at what cost
> and at how much effort?  What are you going to do with incompatible
> reserved words?  Who do you expect to do it?  How soon?  I've seen
> alot of projects try to make themselves "user-friendly" only to suffer
> in the end from what they lost in the effort.

The cost and effort is purely voluntary.  :)  i.e. $0-ish cost, and
heaps of effort.

> Personally I'd prefer a PostgreSQL that was as SQL92 and beyond as it
> could possibly be rather than some of this and some of that.

I don't see how having alternate parsers available, maintained and
updated by those interested in them, is a bad thing.  Certainly don't
see how it detracts from the main effort.

???

> Vince.
> --
> ==========================================================================
> Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
>          56K Nationwide Dialup from $16.00/mo at Pop4 Networking
>         Online Campground Directory    http://www.camping-usa.com
>        Online Giftshop Superstore    http://www.cloudninegifts.com
> ==========================================================================

Regards and best wishes,

Justin Clift

-- 
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
     - Indira Gandhi

  22. Re: Re: [PATCHES] Select parser at runtime

    Jan Wieck <janwieck@yahoo.com> — 2001-08-13T13:55:59Z

    Ian Lance Taylor wrote:
> Along these lines, I don't think Bruce's suggestion of modifications
> to the Postgres gram.y is a good idea, because it causes the Oracle
> parser to add an ongoing cost to the Postgres parser.

    Bruce,  Tom  and  I discussed these issues during our time in
    San Diego last month.

    If we want to have both parsers available at runtime we  need
    to  replace the YY (case-insensitive) prefix in the generated
    files per parser and call the right one from tcop.   Now  for
    some flex/bison combo's at least the prefix switches (to have
    something different than YY) don't work reliable. There  will
    be  some  global  YY-objects  left, causing linkage problems.
    That's why PL/pgSQL's scanner/parser's C-code is run  through
    sed(1).

    If  Bruce's  suggestion  of having both parsers in one source
    with #ifdef, #else, #endif is better  than  separate  sources
    depends  mainly  on  how big the differences finally will be.
    Doesn't really  bother  me.  Maybe  we  could  start  with  a
    combined  one  and  separate  later if it turns out that they
    drift apart too much?


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

  23. Re: Re: [PATCHES] Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-13T13:57:17Z

    Ian Lance Taylor <ian@airs.com> writes:
> ... most of the cost will be paid by the people who care about
> it.  (Not all of the cost, because some communication will be required
> when the parse tree nodes are changed.)

> Along these lines, I don't think Bruce's suggestion of modifications
> to the Postgres gram.y is a good idea, because it causes the Oracle
> parser to add an ongoing cost to the Postgres parser.

And managing grammar changes and parse-tree-node changes is not an
ongoing cost?  I beg to differ.  We do that a lot, and keeping multiple
grammar files in sync is not a pleasant prospect.  (Look at ecpg ---
it's a major pain to keep it in sync with the main parser, even though
it only shares productions and not output code.  Worse, I have zero
confidence that it actually *is* in sync.)

If the grammar changes are small and localized, I think Bruce's #ifdef
approach might well be the way to go.  However, we're speculating in
a vacuum here, not having seen the details of the changes needed.

			regards, tom lane

  24. Re: Re: [PATCHES] Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-13T14:23:24Z

    Jan Wieck <JanWieck@Yahoo.com> writes:
>     files per parser and call the right one from tcop.   Now  for
>     some flex/bison combo's at least the prefix switches (to have
>     something different than YY) don't work reliable. There  will
>     be  some  global  YY-objects  left, causing linkage problems.
>     That's why PL/pgSQL's scanner/parser's C-code is run  through
>     sed(1).

The only reason plpgsql's parser is still run through sed is that
I haven't gotten around to changing it ;-).  The main system has
depended on -P for awhile, and we've seen no reports of trouble.

(This is not unrelated to the fact that we now ship pre-yacced and
pre-lexed .c files, no doubt.  Only people who pull from CVS ever
rebuild the files at all, and we tell them they must use up-to-date
flex and bison.  This policy seems to work a lot better than the old
way of trying to work with whatever broken tools a particular platform
might have...)

			regards, tom lane

  25. Re: Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-13T17:08:46Z

    Jan Wieck <JanWieck@Yahoo.com> writes:

>     If we want to have both parsers available at runtime we  need
>     to  replace the YY (case-insensitive) prefix in the generated
>     files per parser and call the right one from tcop.   Now  for
>     some flex/bison combo's at least the prefix switches (to have
>     something different than YY) don't work reliable. There  will
>     be  some  global  YY-objects  left, causing linkage problems.
>     That's why PL/pgSQL's scanner/parser's C-code is run  through
>     sed(1).

This is a solved problem.  gdb, for example, links together four
different Yacc-based parsers, without even using bison's -p option.

Ian

  26. Re: Re: [PATCHES] Select parser at runtime

    Ian Lance Taylor <ian@airs.com> — 2001-08-13T17:33:54Z

    Tom Lane <tgl@sss.pgh.pa.us> writes:

> Ian Lance Taylor <ian@airs.com> writes:
> > ... most of the cost will be paid by the people who care about
> > it.  (Not all of the cost, because some communication will be required
> > when the parse tree nodes are changed.)
> 
> > Along these lines, I don't think Bruce's suggestion of modifications
> > to the Postgres gram.y is a good idea, because it causes the Oracle
> > parser to add an ongoing cost to the Postgres parser.
> 
> And managing grammar changes and parse-tree-node changes is not an
> ongoing cost?  I beg to differ.  We do that a lot, and keeping multiple
> grammar files in sync is not a pleasant prospect.  (Look at ecpg ---
> it's a major pain to keep it in sync with the main parser, even though
> it only shares productions and not output code.  Worse, I have zero
> confidence that it actually *is* in sync.)

Parse tree node changes are definitely an ongoing cost, as I mention
in the first quoted paragraph.  However, I would see this as a
communication issue from the people maintaining the core parser to the
people maintaining the Oracle parser.  Perhaps it will be possible to
better formalize the parse tree node interface.

(This approach is from my gcc experience.  The various gcc parsers (C,
C++, etc.) generate tree nodes.  The structure of the tree nodes does
change from time to time, forcing all the other parsers to change.
This is generally driven by the needs of some parser.  Different
groups of people maintain each parser.)

I'm not sure what you mean by managing grammar changes, although
perhaps I am reading too much into that.  The Oracle grammar is set by
Oracle, and will not change even if the Postgres grammar changes.

> If the grammar changes are small and localized, I think Bruce's #ifdef
> approach might well be the way to go.  However, we're speculating in
> a vacuum here, not having seen the details of the changes needed.

I've been trying to avoid the details, because that is just going to
raise another discussion which I honestly believe is orthogonal to
this discussion.  However, a quick sketch: proper handling of Oracle
reserved words is very difficult in a Yacc-based parser, because the
set of reserved words effectively changes on a contextual basis.  The
Postgres grammar uses various workarounds for this, but doesn't handle
everything entirely correctly from the point of view of the Oracle
language.  Therefore, my prototype Oracle grammar is actually a
recursive descent parser, not Yacc-based at all.

Ian

  27. Re: Re: [PATCHES] Select parser at runtime

    Peter Eisentraut <peter_e@gmx.net> — 2001-08-13T17:49:38Z

    Ian Lance Taylor writes:

> I'm not sure what you mean by managing grammar changes, although
> perhaps I am reading too much into that.  The Oracle grammar is set by
> Oracle, and will not change even if the Postgres grammar changes.

Things like VACUUM and ANALYZE, which you will have to keep unless you
want to implement an Oracle storage manager as well. ;-)

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

  28. Re: Re: [PATCHES] Select parser at runtime

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-08-13T17:58:36Z

    Peter Eisentraut <peter_e@gmx.net> writes:
> Things like VACUUM and ANALYZE, which you will have to keep unless you
> want to implement an Oracle storage manager as well. ;-)

Evidently Ian is just interested in a parser that could be used by
Oracle-compatible applications, which'd not be invoking such operations
anyway.  Maintenance scripts would have to use the regular PG parser.
That doesn't seem unreasonable.

Based on his further explanation, it seems that tracking grammar changes
wouldn't be an issue, but tracking parsetree changes definitely would
be.  I'm also still concerned about whether this can be done within the
parse step (no database access).

			regards, tom lane