Re: replace strtok()

Ranier Vilela <ranier.vf@gmail.com>

From: Ranier Vilela <ranier.vf@gmail.com>
To: Peter Eisentraut <peter@eisentraut.org>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-06-18T11:43:23Z
Lists: pgsql-hackers

Attachments

Em ter., 18 de jun. de 2024 às 04:18, Peter Eisentraut <peter@eisentraut.org>
escreveu:

> Under the topic of getting rid of thread-unsafe functions in the backend
> [0], here is a patch series to deal with strtok().
>
> Of course, strtok() is famously not thread-safe and can be replaced by
> strtok_r().  But it also has the wrong semantics in some cases, because
> it considers adjacent delimiters to be one delimiter.  So if you parse
>
>      SCRAM-SHA-256$<iterations>:<salt>$<storedkey>:<serverkey>
>
> with strtok(), then
>
>      SCRAM-SHA-256$$<iterations>::<salt>$$<storedkey>::<serverkey>
>
> parses just the same.  In many cases, this is arguably wrong and could
> hide mistakes.
>
> So I'm suggesting to use strsep() in those places.  strsep() is
> nonstandard but widely available.
>
> There are a few places where strtok() has the right semantics, such as
> parsing tokens separated by whitespace.  For those, I'm using strtok_r().
>
> A reviewer job here would be to check whether I made that distinction
> correctly in each case.
>
> On the portability side, I'm including a port/ replacement for strsep()
> and some workaround to get strtok_r() for Windows.  I have included
> these here as separate patches for clarity.
>
+1 For making the code thread-safe.
But I would like to see more const char * where this is possible.

For example, in pg_locale.c
IMO, the token variable can be const char *.

At least strchr expects a const char * as the first parameter.

I found another implementation of strsep, it seems lighter to me.
I will attach it for consideration, however, I have not done any testing.

best regards,
Ranier Vilela

Commits

  1. Fix memory leaks from incorrect strsep() uses

  2. Fix strsep() use for SCRAM secrets parsing

  3. Replace remaining strtok() with strtok_r()

  4. Windows replacement for strtok_r()

  5. Replace some strtok() with strsep()

  6. Add port/ replacement for strsep()