BUG #14826: Malformed array dimensions for empty arrays in return column "lexemes" of ts_debug()

The following bug has been logged on the website:

Bug reference:      14826
Logged by:          Erwin Brandstetter
Email address:      brsaweda@gmail.com
PostgreSQL version: 9.6.5
Operating system:   any
Description:        

Empty strings in this column seem to be "out of this world" - comparison to
'{}'::text[] returns FALSE:

test=# SELECT token, lexemes, lexemes = '{}'::text[]
test-# FROM   ts_debug('a title');
 token | lexemes | ?column?
-------+---------+----------
 a     | {}      | f
       |         |
 title | {titl}  | f

The root of the problem seems to be malformed array dimensions.
array_dims(lexemes) returns [1:0], while I would expect NULL:

test=# SELECT token, dictionaries, lexemes
test-#      , array_dims(lexemes) AS lex_dims
test-#      , array_dims(dictionaries) AS dict_dims
test-# FROM   ts_debug('a title');
 token |  dictionaries  | lexemes | lex_dims | dict_dims
-------+----------------+---------+----------+-----------
 a     | {english_stem} | {}      | [1:0]    | [1:1]
       | {}             |         |          |
 title | {english_stem} | {titl}  | [1:1]    | [1:1]

I tested in versions 8.4 to 10 Beta, always the same result.

Related question on dba.stackexchange:
https://dba.stackexchange.com/questions/186715/how-do-i-select-arrays-that-are-not-empty


Regards
Erwin Brandstetter