Horton

Thread

Commits

  1. Improve prep_buildtree

  1. Improve prep_buildtree

    Peter Eisentraut <peter@eisentraut.org> — 2025-07-30T08:07:32Z

    Since I started using meson regularly, I have put the build directory 
under the source directory, which appears to be the recommended 
convention.  See for example

https://mesonbuild.com/Quick-guide.html#compiling-a-meson-project

Then, if I also want to create a build directory using configure/make in 
parallel to that (for example, to test the makefiles, if adding new 
source files), then prep_buildtree is a bit stupid about that.  It 
processes all directories under the top-level source directory, 
including existing other build trees, and so it makes a copy of existing 
build trees under the new build tree.  This isn't actually harmful, but 
it seems pretty stupid, and it also makes prep_buildtree slower and 
slower the more build trees you have.

I have never been much of a vpath user before meson, so I don't have 
much experience with it, but I suspect that it was previously the 
convention to have the build tree outside of the source tree, in which 
case this wouldn't have been a problem.

To fix this, I first tried to devise a way to detect whether a given 
directory is a build directory.  But that seemed pretty complicated. 
Instead, I chose the simpler solution that we just enumerate the source 
subdirectories that we know about (config, contrib, doc, src).  That way 
we can also remove the special handling to exclude the .git directory 
and make the find command a bit simpler.

See the attached patch.

Maybe people can try this with their local workflows to make sure it 
still works for them, but I don't expect any particular problems, other 
than perhaps some shell or find command portability issues.  Note, this 
only runs if you do a vpath configure/make build, not an in-tree 
configure/make build, and of course not if you use meson.

  2. Re: Improve prep_buildtree

    Nazir Bilal Yavuz <byavuz81@gmail.com> — 2025-07-30T08:28:57Z

    Hi,

Thank you for working on this!

On Wed, 30 Jul 2025 at 11:07, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> To fix this, I first tried to devise a way to detect whether a given
> directory is a build directory.  But that seemed pretty complicated.
> Instead, I chose the simpler solution that we just enumerate the source
> subdirectories that we know about (config, contrib, doc, src).  That way
> we can also remove the special handling to exclude the .git directory
> and make the find command a bit simpler.

Meson looks for ${builddir}/meson-private/coredata.dat file to
understand if the directory is a build directory [1]. I am just
wondering if you tried this and found it complicated or did you try
something else.

[1] https://github.com/mesonbuild/meson/blob/b4d32763940ae67d1ed36952112b3d94ba9f03a7/mesonbuild/mcompile.py#L30C1-L35C82

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

  3. Re: Improve prep_buildtree

    Peter Eisentraut <peter@eisentraut.org> — 2025-07-30T09:31:58Z

    On 30.07.25 10:28, Nazir Bilal Yavuz wrote:
> On Wed, 30 Jul 2025 at 11:07, Peter Eisentraut <peter@eisentraut.org> wrote:
>>
>> To fix this, I first tried to devise a way to detect whether a given
>> directory is a build directory.  But that seemed pretty complicated.
>> Instead, I chose the simpler solution that we just enumerate the source
>> subdirectories that we know about (config, contrib, doc, src).  That way
>> we can also remove the special handling to exclude the .git directory
>> and make the find command a bit simpler.
> 
> Meson looks for ${builddir}/meson-private/coredata.dat file to
> understand if the directory is a build directory [1]. I am just
> wondering if you tried this and found it complicated or did you try
> something else.

I tried things along this line, but it's not easy to have a shell "find" 
command handle that kind of logic.

  4. Re: Improve prep_buildtree

    Nazir Bilal Yavuz <byavuz81@gmail.com> — 2025-07-30T10:58:43Z

    Hi,

On Wed, 30 Jul 2025 at 12:32, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> On 30.07.25 10:28, Nazir Bilal Yavuz wrote:
> > On Wed, 30 Jul 2025 at 11:07, Peter Eisentraut <peter@eisentraut.org> wrote:
> >>
> >> To fix this, I first tried to devise a way to detect whether a given
> >> directory is a build directory.  But that seemed pretty complicated.
> >> Instead, I chose the simpler solution that we just enumerate the source
> >> subdirectories that we know about (config, contrib, doc, src).  That way
> >> we can also remove the special handling to exclude the .git directory
> >> and make the find command a bit simpler.
> >
> > Meson looks for ${builddir}/meson-private/coredata.dat file to
> > understand if the directory is a build directory [1]. I am just
> > wondering if you tried this and found it complicated or did you try
> > something else.
>
> I tried things along this line, but it's not easy to have a shell "find"
> command handle that kind of logic.

That makes sense. I noticed one difference, patched prep_buildtree
does not create a symbolic link for the top Makefile in the builddir.

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

  5. Re: Improve prep_buildtree

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-30T13:34:58Z

    Peter Eisentraut <peter@eisentraut.org> writes:
> On 30.07.25 10:28, Nazir Bilal Yavuz wrote:
>> Meson looks for ${builddir}/meson-private/coredata.dat file to
>> understand if the directory is a build directory [1]. I am just
>> wondering if you tried this and found it complicated or did you try
>> something else.

> I tried things along this line, but it's not easy to have a shell "find" 
> command handle that kind of logic.

For this purpose, I think it's better for us to rely on things we
control than things Meson controls.  Adding a new top-level directory
is rare enough that I don't see a problem with needing to update
prep_buildtree.

			regards, tom lane

  6. Re: Improve prep_buildtree

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-30T15:37:38Z

    On Wed, Jul 30, 2025 at 6:35 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> For this purpose, I think it's better for us to rely on things we
> control than things Meson controls.  Adding a new top-level directory
> is rare enough that I don't see a problem with needing to update
> prep_buildtree.

+1. (This approach should also keep prep_buildtree from recursing into
my git worktrees and vpaths, so I'm excited to give it a try!)

--Jacob

  7. Re: Improve prep_buildtree

    Peter Eisentraut <peter@eisentraut.org> — 2025-07-30T18:56:20Z

    On 30.07.25 12:58, Nazir Bilal Yavuz wrote:
> Hi,
> 
> On Wed, 30 Jul 2025 at 12:32, Peter Eisentraut <peter@eisentraut.org> wrote:
>>
>> On 30.07.25 10:28, Nazir Bilal Yavuz wrote:
>>> On Wed, 30 Jul 2025 at 11:07, Peter Eisentraut <peter@eisentraut.org> wrote:
>>>>
>>>> To fix this, I first tried to devise a way to detect whether a given
>>>> directory is a build directory.  But that seemed pretty complicated.
>>>> Instead, I chose the simpler solution that we just enumerate the source
>>>> subdirectories that we know about (config, contrib, doc, src).  That way
>>>> we can also remove the special handling to exclude the .git directory
>>>> and make the find command a bit simpler.
>>>
>>> Meson looks for ${builddir}/meson-private/coredata.dat file to
>>> understand if the directory is a build directory [1]. I am just
>>> wondering if you tried this and found it complicated or did you try
>>> something else.
>>
>> I tried things along this line, but it's not easy to have a shell "find"
>> command handle that kind of logic.
> 
> That makes sense. I noticed one difference, patched prep_buildtree
> does not create a symbolic link for the top Makefile in the builddir.

Ok, here is an updated version that takes care of that, too.

  8. Re: Improve prep_buildtree

    Nazir Bilal Yavuz <byavuz81@gmail.com> — 2025-07-31T07:24:34Z

    Hi,

On Wed, 30 Jul 2025 at 21:56, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> On 30.07.25 12:58, Nazir Bilal Yavuz wrote:
> >
> > That makes sense. I noticed one difference, patched prep_buildtree
> > does not create a symbolic link for the top Makefile in the builddir.
>
> Ok, here is an updated version that takes care of that, too.

Thanks. v2 LGTM.

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

  9. Re: Improve prep_buildtree

    Peter Eisentraut <peter@eisentraut.org> — 2025-08-04T07:14:17Z

    On 31.07.25 09:24, Nazir Bilal Yavuz wrote:
> On Wed, 30 Jul 2025 at 21:56, Peter Eisentraut <peter@eisentraut.org> wrote:
>>
>> On 30.07.25 12:58, Nazir Bilal Yavuz wrote:
>>>
>>> That makes sense. I noticed one difference, patched prep_buildtree
>>> does not create a symbolic link for the top Makefile in the builddir.
>>
>> Ok, here is an updated version that takes care of that, too.
> 
> Thanks. v2 LGTM.

committed