Horton

Thread

Commits

  1. Support shared libraries on Android (using make)

  2. meson: Map host_system android to linux

  3. meson: portname was set too early

  1. Build versionless .so for Android

    Matthias Kuhn <matthias@opengis.ch> — 2023-12-31T05:08:04Z

    Hi,

In order to ship libpq on Android, it needs to be built without a version
suffix on the .so file.
The attached patch will make sure that no version is added when built for
Android.
I was wondering if there are a) any comments on the approach and if I
should be handed in for a commitfest (currently waiting for the cooldown
period after account activation, I am not sure how long that is)

Thank you for any feedback
Matthias

  2. Re: Build versionless .so for Android

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-12-31T06:19:48Z

    Matthias Kuhn <matthias@opengis.ch> writes:
> In order to ship libpq on Android, it needs to be built without a version
> suffix on the .so file.
> The attached patch will make sure that no version is added when built for
> Android.

This ... seems incredibly brain-dead.  Does Android really not cope
with version-to-version ABI changes?

			regards, tom lane

  3. Re: Build versionless .so for Android

    Michael Paquier <michael@paquier.xyz> — 2023-12-31T06:24:27Z

    On Sun, Dec 31, 2023 at 06:08:04AM +0100, Matthias Kuhn wrote:
> In order to ship libpq on Android, it needs to be built without a version
> suffix on the .so file.
> The attached patch will make sure that no version is added when built for
> Android.

FWIW, I have mixed feelings about patching the code to treat
android-linux as an exception while changing nothing in the
documentation to explain why this is required.  A custom patch may
serve you better here, and note that you did not even touch the meson
paths.  See libpq_c_args in src/interfaces/libpq/meson.build as one
example.  That's just my opinion, of course, still there are no
buildfarm members that would cover your patch.

> I was wondering if there are a) any comments on the approach and if I
> should be handed in for a commitfest (currently waiting for the cooldown
> period after account activation, I am not sure how long that is)

Such discussions are adapted in a commit fest entry.  No idea if there
is a cooldown period after account creation before being able to touch
the CF app contents, though.
--
Michael

  4. Re: Build versionless .so for Android

    Joe Conway <mail@joeconway.com> — 2023-12-31T19:16:59Z

    On 12/31/23 01:24, Michael Paquier wrote:
> On Sun, Dec 31, 2023 at 06:08:04AM +0100, Matthias Kuhn wrote:
>> I was wondering if there are a) any comments on the approach and if I
>> should be handed in for a commitfest (currently waiting for the cooldown
>> period after account activation, I am not sure how long that is)
> 
> Such discussions are adapted in a commit fest entry.  No idea if there
> is a cooldown period after account creation before being able to touch
> the CF app contents, though.


FWIW I have expedited the cooldown period, so Matthias can log in right 
away.

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  5. Re: Build versionless .so for Android

    Matthias Kuhn <matthias@opengis.ch> — 2024-01-01T05:25:26Z

    Thanks for all the comments and help.

I have added the patch to the January CF.

It looks like meson does not currently support building for android, the
following output is what I get (but I have actually no experience with
meson):

    meson.build:320:2: ERROR: Problem encountered: unknown host system:
android

Please find an attached patch which also includes a documentation section.
I am happy to adjust if needed.

Kind regards
Matthias

  6. Re: Build versionless .so for Android

    Robert Haas <robertmhaas@gmail.com> — 2024-01-02T17:43:01Z

    On Sun, Dec 31, 2023 at 1:24 AM Michael Paquier <michael@paquier.xyz> wrote:
> FWIW, I have mixed feelings about patching the code to treat
> android-linux as an exception while changing nothing in the
> documentation to explain why this is required.  A custom patch may
> serve you better here, and note that you did not even touch the meson
> paths.  See libpq_c_args in src/interfaces/libpq/meson.build as one
> example.  That's just my opinion, of course, still there are no
> buildfarm members that would cover your patch.

It's reasonable to want good comments -- the one in the patch (1)
doesn't explain why this is required and (2) suggests that it is only
needed when cross-compiling which seems surprising and (3) has a typo.
But if it's true that this is needed, I tentatively think we might do
better to take the patch than force people to carry it out of tree.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  7. Re: Build versionless .so for Android

    Matthias Kuhn <matthias@opengis.ch> — 2024-01-05T00:00:05Z

    Hi,

Attached a patch with a (hopefully) better wording of the comment.

I have unsuccessfully tried to find an official source for this policy.
So for reference some discussions about the topic:

-
https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk
-
https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files

Please let me know if I can help in any way

Matthias

On Tue, Jan 2, 2024 at 6:43 PM Robert Haas <robertmhaas@gmail.com> wrote:

> On Sun, Dec 31, 2023 at 1:24 AM Michael Paquier <michael@paquier.xyz>
> wrote:
> > FWIW, I have mixed feelings about patching the code to treat
> > android-linux as an exception while changing nothing in the
> > documentation to explain why this is required.  A custom patch may
> > serve you better here, and note that you did not even touch the meson
> > paths.  See libpq_c_args in src/interfaces/libpq/meson.build as one
> > example.  That's just my opinion, of course, still there are no
> > buildfarm members that would cover your patch.
>
> It's reasonable to want good comments -- the one in the patch (1)
> doesn't explain why this is required and (2) suggests that it is only
> needed when cross-compiling which seems surprising and (3) has a typo.
> But if it's true that this is needed, I tentatively think we might do
> better to take the patch than force people to carry it out of tree.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
>

  8. Re: Build versionless .so for Android

    Robert Haas <robertmhaas@gmail.com> — 2024-01-05T14:44:55Z

    On Thu, Jan 4, 2024 at 7:00 PM Matthias Kuhn <matthias@opengis.ch> wrote:
> Attached a patch with a (hopefully) better wording of the comment.
>
> I have unsuccessfully tried to find an official source for this policy.
> So for reference some discussions about the topic:
>
> - https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk
> - https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files

This is interesting, but these links are also very old.

I like the new wording of the comment better, but I think we need more
confirmation that this is correct before committing it. Anyone else
here familiar with building on Android?

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  9. Re: Build versionless .so for Android

    Peter Eisentraut <peter@eisentraut.org> — 2024-01-05T14:57:23Z

    On 05.01.24 01:00, Matthias Kuhn wrote:
> Attached a patch with a (hopefully) better wording of the comment.
> 
> I have unsuccessfully tried to find an official source for this policy.
> So for reference some discussions about the topic:
> 
> - 
> https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk <https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk>
> - 
> https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files <https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files>
  What I would like to see is a specific thing that you are trying to do 
that doesn't work.  Probably, you are writing a program that is meant to 
run on Android, and you are linking it (provide command line), and then 
what happens?  The linking fails?  It fails to run?  What is the error? 
Can you provide a minimal example?  And so on.

  10. Re: Build versionless .so for Android

    Peter Eisentraut <peter@eisentraut.org> — 2024-01-05T15:00:01Z

    On 01.01.24 06:25, Matthias Kuhn wrote:
> It looks like meson does not currently support building for android, the 
> following output is what I get (but I have actually no experience with 
> meson):
> 
>      meson.build:320:2: ERROR: Problem encountered: unknown host system: 
> android

FWIW, the meson source code contains numerous mentions of an 'android' 
platform, so it seems like this is expected to work.

  11. Re: Build versionless .so for Android

    Andres Freund <andres@anarazel.de> — 2024-01-05T20:34:06Z

    Hi,

On 2024-01-05 16:00:01 +0100, Peter Eisentraut wrote:
> On 01.01.24 06:25, Matthias Kuhn wrote:
> > It looks like meson does not currently support building for android, the
> > following output is what I get (but I have actually no experience with
> > meson):
> > 
> >      meson.build:320:2: ERROR: Problem encountered: unknown host system:
> > android
> 
> FWIW, the meson source code contains numerous mentions of an 'android'
> platform, so it seems like this is expected to work.

This error is from our code, not meson's - the simplest fix would be to just
map android to linux, similar to how dragonflybsd is mapped to netbsd.  Blind
attempt attached.

It looks like that might be all that's required, as it looks like meson knows
that android doesn't properly deal with soversion.

Greetings,

Andres Freund

  12. Re: Build versionless .so for Android

    Andres Freund <andres@anarazel.de> — 2024-01-05T21:46:58Z

    Hi,

On 2024-01-05 15:57:23 +0100, Peter Eisentraut wrote:
> On 05.01.24 01:00, Matthias Kuhn wrote:
> > Attached a patch with a (hopefully) better wording of the comment.
> > 
> > I have unsuccessfully tried to find an official source for this policy.
> > So for reference some discussions about the topic:
> > 
> > - https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk <https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk>
> > - https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files <https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files>
>  What I would like to see is a specific thing that you are trying to do that
> doesn't work.  Probably, you are writing a program that is meant to run on
> Android, and you are linking it (provide command line), and then what
> happens?  The linking fails?  It fails to run?  What is the error? Can you
> provide a minimal example?  And so on.

I looked around for a bit, and couldn't really find good documentation around
this. The best I found is
https://developer.android.com/ndk/guides/abis#native-code-in-app-packages
which does strongly imply that the library names aren't versioned.

Greetings,

Andres Freund

  13. Re: Build versionless .so for Android

    Matthias Kuhn <matthias@opengis.ch> — 2024-01-14T11:37:00Z

    What I try to do is packaging an app with androiddeployqt which fails with
an error:

The bundled library lib/libpq.so.5 doesn't end with .so. Android only
supports versionless libraries ending with the .so suffix.

This error was introduced in response to this issue which contains hints
about the underlying problem:

https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346

I hope this sheds some light
Matthias

On Fri, Jan 5, 2024, 21:57 Peter Eisentraut <peter@eisentraut.org> wrote:

> On 05.01.24 01:00, Matthias Kuhn wrote:
> > Attached a patch with a (hopefully) better wording of the comment.
> >
> > I have unsuccessfully tried to find an official source for this policy.
> > So for reference some discussions about the topic:
> >
> > -
> >
> https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk
> <
> https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk
> >
> > -
> >
> https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files
> <
> https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files
> >
>   What I would like to see is a specific thing that you are trying to do
> that doesn't work.  Probably, you are writing a program that is meant to
> run on Android, and you are linking it (provide command line), and then
> what happens?  The linking fails?  It fails to run?  What is the error?
> Can you provide a minimal example?  And so on.
>
>

  14. Re: Build versionless .so for Android

    Peter Eisentraut <peter@eisentraut.org> — 2024-01-18T08:27:55Z

    On 14.01.24 12:37, Matthias Kuhn wrote:
> What I try to do is packaging an app with androiddeployqt which fails 
> with an error:
> 
> The bundled library lib/libpq.so.5 doesn't end with .so. Android only 
> supports versionless libraries ending with the .so suffix.
> 
> This error was introduced in response to this issue which contains hints 
> about the underlying problem:
> 
> https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346 
> <https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346>

I think the scenario there is using dlopen(), possibly via Qt, possibly 
via Java, so it's a very specific scenario, not necessarily indicative 
of a general requirement on Android, and apparently not using build-time 
linking.  It's quite conceivable that this issue would also exist with 
Qt on other platforms.

As I mentioned before, Meson has Android support.  Some people there 
clearly thought about it.  So I suggest you try building PostgreSQL for 
your Android environment using Meson and see what it produces.(*)  If 
the output files are the same as the autoconf/make build, then I would 
think your scenario is nonstandard.  If the output files are different, 
then we should check that and consider changes to get them to match.

It's of course possible that Meson is wrong, too, but then we need to 
have a broader analysis, because the implicit goal is to keep the two 
build systems for PostgreSQL consistent.

(*) - This means specifically that the installation trees produced by 
"make install-world-bin" and "meson install" should produce exactly the 
same set of files (same names in the same directories, not necessarily 
the same contents).

  15. Re: Build versionless .so for Android

    Matthias Kuhn <matthias@opengis.ch> — 2024-01-19T10:08:59Z

    Hi

On Thu, Jan 18, 2024 at 9:27 AM Peter Eisentraut <peter@eisentraut.org>
wrote:

> On 14.01.24 12:37, Matthias Kuhn wrote:
> > What I try to do is packaging an app with androiddeployqt which fails
> > with an error:
> >
> > The bundled library lib/libpq.so.5 doesn't end with .so. Android only
> > supports versionless libraries ending with the .so suffix.
> >
> > This error was introduced in response to this issue which contains hints
> > about the underlying problem:
> >
> > https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346
> > <https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346>
>
> I think the scenario there is using dlopen(), possibly via Qt, possibly
> via Java, so it's a very specific scenario, not necessarily indicative
> of a general requirement on Android, and apparently not using build-time
> linking.  It's quite conceivable that this issue would also exist with
> Qt on other platforms.
>
>
I haven't experienced this issue with Qt on any other platform (and I build
and run the same stack on windows, linux, macos and ios).
Also the links posted earlier in this thread hint to the same limitation of
Android, unrelated to Qt.


> As I mentioned before, Meson has Android support.  Some people there
> clearly thought about it.  So I suggest you try building PostgreSQL for
> your Android environment using Meson and see what it produces.(*)  If
> the output files are the same as the autoconf/make build, then I would
> think your scenario is nonstandard.  If the output files are different,
> then we should check that and consider changes to get them to match.
>
> It's of course possible that Meson is wrong, too, but then we need to
> have a broader analysis, because the implicit goal is to keep the two
> build systems for PostgreSQL consistent.
>

When trying to build with meson, including the patch which was provided by
Andres Freud (thanks),
I am currently stuck with the following error:

Configuring pg_config_ext.h using configuration

../src/tgresql-16-685bc9fc97.clean/src/include/meson.build:12:0: ERROR:
File port/android.h does not exist.

Kind regards
Matthias


>
> (*) - This means specifically that the installation trees produced by
> "make install-world-bin" and "meson install" should produce exactly the
> same set of files (same names in the same directories, not necessarily
> the same contents).
>
>

  16. Re: Build versionless .so for Android

    Peter Eisentraut <peter@eisentraut.org> — 2024-01-19T14:00:40Z

    On 19.01.24 11:08, Matthias Kuhn wrote:
> When trying to build with meson, including the patch which was provided 
> by Andres Freud (thanks),
> I am currently stuck with the following error:
> 
> Configuring pg_config_ext.h using configuration
> 
> ../src/tgresql-16-685bc9fc97.clean/src/include/meson.build:12:0: ERROR: 
> File port/android.h does not exist.

I think there is actually small bug in the meson setup.

In the top-level meson.build, line 166 has

     portname = host_system

but then later around line 190, host_system is changed for some cases 
(including the android case that was added in the proposed patch).  So 
that won't work, and probably also won't work for the existing cases 
there.  The portname assignment needs to be moved later in the file. 
Maybe you can try that on your own.

  17. Re: Build versionless .so for Android

    Matthias Kuhn <matthias@opengis.ch> — 2024-01-19T17:12:24Z

    Thanks,

This patch brought it further and indeed, the resulting libpq.so file is
unversioned when built for Android while it's versioned when built for
linux.

Matthias

On Fri, Jan 19, 2024 at 3:00 PM Peter Eisentraut <peter@eisentraut.org>
wrote:

> On 19.01.24 11:08, Matthias Kuhn wrote:
> > When trying to build with meson, including the patch which was provided
> > by Andres Freud (thanks),
> > I am currently stuck with the following error:
> >
> > Configuring pg_config_ext.h using configuration
> >
> > ../src/tgresql-16-685bc9fc97.clean/src/include/meson.build:12:0: ERROR:
> > File port/android.h does not exist.
>
> I think there is actually small bug in the meson setup.
>
> In the top-level meson.build, line 166 has
>
>      portname = host_system
>
> but then later around line 190, host_system is changed for some cases
> (including the android case that was added in the proposed patch).  So
> that won't work, and probably also won't work for the existing cases
> there.  The portname assignment needs to be moved later in the file.
> Maybe you can try that on your own.
>
>

  18. Re: Build versionless .so for Android

    Peter Eisentraut <peter@eisentraut.org> — 2024-01-23T19:42:54Z

    On 19.01.24 18:12, Matthias Kuhn wrote:
> Thanks,
> 
> This patch brought it further and indeed, the resulting libpq.so file is 
> unversioned when built for Android while it's versioned when built for 
> linux.

Ok, I have committed all of this now:

- Fix for correct order of host_system and portname assignment in 
meson.build.

- Patch from Andres to map android to linux in meson.build.

- Makefile.shlib support for Android (based on your patch, but I 
reworked it a bit).

So this should all work for you now.