Horton

Thread

  1. timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> — 2025-04-08T16:10:15Z

    Hi
in continuation of "Ideas about presenting data coming from sensors"

https://www.postgresql.org/message-id/flat/8d2dd92a-da16-435b-a38e-fe72191fc9d1%40cloud.gatewaynet.com

we got the system working in single tables fashion (3 kinds of them), 
since no timeseries solution seemed to fit 100% all the requirements at 
the time, or simply because I didn't have the time to evaluate all the 
existing options.

Fast forward today, in a few months we got almost 63M rows , but this 
will increase exponentially since new vessels will be configured to send 
their sensor's data.

After an initial idea with timescaledb, I tried to install pg_timeseries 
today, and give it a try.

pg_timeseries does not seem active and their "columnar" requirement 
seems to have stuck due to citus not having been updated to postgresql 
17. Stopper.

timescaledb seemed mature, but also exotic, allow me the term. No way to 
use native logical replication, shortage of options to run on premise or 
self hosted, which leaves us with those options :

a) stick with timescaledb in their cloud offering and try to bridge the 
two systems (ours and the new timescaledb instance)

b) convert to native partitioning and just try to manage via partman, 
forgetting for the moment incremental views and columnar store, or maybe 
try to introduce some functionality from pg_ivm + pgcron

So the question : are those are our only options? google says so but is 
this really the case ?

thank you.

  2. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Amitabh Kant <amitabhkant@gmail.com> — 2025-04-08T17:37:32Z

    On Tue, Apr 8, 2025 at 9:40 PM Achilleas Mantzios - cloud <
a.mantzios@cloud.gatewaynet.com> wrote:

> Hi
> <snip>
> timescaledb seemed mature, but also exotic, allow me the term. No way to
> use native logical replication, shortage of options to run on premise or
> self hosted, which leaves us with those options :
>
> <snip>
>
>
I cannot comment on the applicability of timescaledb in your context, but
running it on premise/self-hosted has posed no problems, at least on
Debian.

If I understood your query incorrectly, please ignore.

Amitabh

  3. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> — 2025-04-08T17:59:52Z

    On 8/4/25 20:37, Amitabh Kant wrote:
> On Tue, Apr 8, 2025 at 9:40 PM Achilleas Mantzios - cloud 
> <a.mantzios@cloud.gatewaynet.com> wrote:
>
>     Hi
>     <snip>
>     timescaledb seemed mature, but also exotic, allow me the term. No
>     way to
>     use native logical replication, shortage of options to run on
>     premise or
>     self hosted, which leaves us with those options :
>
>     <snip>
>
> I cannot comment on the applicability of timescaledb in your context, 
> but running it on premise/self-hosted has posed no problems, at least 
> on Debian.
>
> If I understood your query incorrectly, please ignore.
Thank you, I meant the paid/supported service not the community version. 
Which of the two do you use?
>
> Amitabh

  4. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Amitabh Kant <amitabhkant@gmail.com> — 2025-04-09T01:50:56Z

    On Tue, Apr 8, 2025 at 11:29 PM Achilleas Mantzios <
a.mantzios@cloud.gatewaynet.com> wrote:

>
> On 8/4/25 20:37, Amitabh Kant wrote:
>
> On Tue, Apr 8, 2025 at 9:40 PM Achilleas Mantzios - cloud <
> a.mantzios@cloud.gatewaynet.com> wrote:
>
>> Hi
>> <snip>
>> timescaledb seemed mature, but also exotic, allow me the term. No way to
>> use native logical replication, shortage of options to run on premise or
>> self hosted, which leaves us with those options :
>>
>> <snip>
>>
>>
> I cannot comment on the applicability of timescaledb in your context, but
> running it on premise/self-hosted has posed no problems, at least on
> Debian.
>
> If I understood your query incorrectly, please ignore.
>
> Thank you, I meant the paid/supported service not the community version.
> Which of the two do you use?
>
>
>
I use the community version.

  5. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Brent Wood <pcreso@yahoo.com> — 2025-04-09T02:23:51Z

    I also use the free community edition on internal servers, but under Ubuntu. No issues and very good performance.
Brent Wood...

Yahoo Mail: Search, organise, conquer 
 
  On Wed, 9 Apr 2025 at 13:51, Amitabh Kant<amitabhkant@gmail.com> wrote:   On Tue, Apr 8, 2025 at 11:29 PM Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> wrote:
  

 
 On 8/4/25 20:37, Amitabh Kant wrote:
  
  On Tue, Apr 8, 2025 at 9:40 PM Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com> wrote:  
Hi
 <snip>
 timescaledb seemed mature, but also exotic, allow me the term. No way to 
 use native logical replication, shortage of options to run on premise or 
 self hosted, which leaves us with those options :
 
 <snip>
 
 
   I cannot comment on the applicability of timescaledb in your context, but running it on premise/self-hosted has posed no problems, at least on Debian.  
  If I understood your query incorrectly, please ignore.   
 Thank you, I meant the paid/supported service not the community version. Which of the two do you use?
 
   



I use the community version.  

  6. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> — 2025-04-09T06:20:37Z

    On 4/9/25 04:50, Amitabh Kant wrote:
>
>     Thank you, I meant the paid/supported service not the community
>     version. Which of the two do you use?
>>
> I use the community version.


On 4/9/25 05:23, Brent Wood wrote:

> I also use the free community edition on internal servers, but under 
> Ubuntu. No issues and very good performance.
>
> Brent Wood...
>

Thanks Amitabh abd Brent, how do you plan to cope with future upgrades 
based on logical replication ? Do you run timescale on a 
dedicated/separate system from the rest of your PostgreSQL cluster(s)?

  7. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Amitabh Kant <amitabhkant@gmail.com> — 2025-04-10T05:25:19Z

    On Wed, Apr 9, 2025 at 11:50 AM Achilleas Mantzios - cloud <
a.mantzios@cloud.gatewaynet.com> wrote:

>
> On 4/9/25 04:50, Amitabh Kant wrote:
>
> Thank you, I meant the paid/supported service not the community version.
>> Which of the two do you use?
>>
>>
>> I use the community version.
>
>
> On 4/9/25 05:23, Brent Wood wrote:
>
> I also use the free community edition on internal servers, but under
> Ubuntu. No issues and very good performance.
>
> Brent Wood...
>
>
> Thanks Amitabh abd Brent, how do you plan to cope with future upgrades
> based on logical replication ? Do you run timescale on a dedicated/separate
> system from the rest of your PostgreSQL cluster(s)?
>

I don't use the inbuilt logical replication. I depend upon pgBackRest for
my backups. My database is mainly time series data, and runs on the same
systems.

  8. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> — 2025-04-10T12:17:27Z

    Hi Brent

how do you plan to cope with future upgrades based on logical 
replication ? Do you run timescale on a dedicated/separate system from 
the rest of your PostgreSQL cluster(s)?

On 10/4/25 08:25, Amitabh Kant wrote:
> On Wed, Apr 9, 2025 at 11:50 AM Achilleas Mantzios - cloud 
> <a.mantzios@cloud.gatewaynet.com> wrote:
>
>
>     On 4/9/25 04:50, Amitabh Kant wrote:
>>
>>         Thank you, I meant the paid/supported service not the
>>         community version. Which of the two do you use?
>>>
>>     I use the community version.
>
>
>     On 4/9/25 05:23, Brent Wood wrote:
>
>>     I also use the free community edition on internal servers, but
>>     under Ubuntu. No issues and very good performance.
>>
>>     Brent Wood...
>>
>
>     Thanks Amitabh abd Brent, how do you plan to cope with future
>     upgrades based on logical replication ? Do you run timescale on a
>     dedicated/separate system from the rest of your PostgreSQL cluster(s)?
>
>
> I don't use the inbuilt logical replication. I depend upon pgBackRest 
> for my backups. My database is mainly time series data, and runs on 
> the same systems.
thanks

  9. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Brent Wood <brent.wood@niwa.co.nz> — 2025-04-12T21:21:49Z

    Our Timescale dbs are on individual hosts, separate from our main Postgres servers.

So standalone systems to be upgraded individually as required.

Brent Wood

Principal Technician, Fisheries
NIWA
DDI:  +64 (4) 3860529
________________________________
From: Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
Sent: Friday, April 11, 2025 00:17
To: pcreso@yahoo.com <pcreso@yahoo.com>
Cc: pgsql-general@lists.postgresql.org <pgsql-general@lists.postgresql.org>
Subject: Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm


Hi Brent

how do you plan to cope with future upgrades based on logical replication ? Do you run timescale on a dedicated/separate system from the rest of your PostgreSQL cluster(s)?

On 10/4/25 08:25, Amitabh Kant wrote:
On Wed, Apr 9, 2025 at 11:50 AM Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com<mailto:a.mantzios@cloud.gatewaynet.com>> wrote:


On 4/9/25 04:50, Amitabh Kant wrote:
Thank you, I meant the paid/supported service not the community version. Which of the two do you use?

I use the community version.


On 4/9/25 05:23, Brent Wood wrote:

I also use the free community edition on internal servers, but under Ubuntu. No issues and very good performance.

Brent Wood...


Thanks Amitabh abd Brent, how do you plan to cope with future upgrades based on logical replication ? Do you run timescale on a dedicated/separate system from the rest of your PostgreSQL cluster(s)?

I don't use the inbuilt logical replication. I depend upon pgBackRest for my backups. My database is mainly time series data, and runs on the same systems.
thanks
[https://www.niwa.co.nz/static/niwa-2018-horizontal-180.png] <https://www.niwa.co.nz>
Brent Wood
Principal Technician - GIS and Spatial Data Management
Programme Leader - Environmental Information Delivery
+64-4-386-0529

National Institute of Water & Atmospheric Research Ltd (NIWA)
301 Evans Bay Parade Hataitai Wellington New Zealand
Connect with NIWA: niwa.co.nz<https://www.niwa.co.nz> Facebook<https://www.facebook.com/nzniwa> LinkedIn<https://www.linkedin.com/company/niwa> Twitter<https://twitter.com/niwa_nz> Instagram<https://www.instagram.com/niwa_science> YouTube<https://www.youtube.com/channel/UCJ-j3MLMg1H59Ak2UaNLL3A>
To ensure compliance with legal requirements and to maintain cyber security standards, NIWA's IT systems are subject to ongoing monitoring, activity logging and auditing. This monitoring and auditing service may be provided by third parties. Such third parties can access information transmitted to, processed by and stored on NIWA's IT systems.
Note: This email is intended solely for the use of the addressee and may contain information that is confidential or subject to legal professional privilege. If you receive this email in error please immediately notify the sender and delete the email.

  10. Re: timescaledb vs NULL vs pg_timeseries vs partman + pgcron + pg_ivm

    Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> — 2025-04-13T05:13:41Z

    Thank you Brent and Amitabh


On 13/4/25 00:21, Brent Wood wrote:
> Our Timescale dbs are on individual hosts, separate from our main 
> Postgres servers.
>
> So standalone systems to be upgraded individually as required.
>
> Brent Wood
>
> Principal Technician, Fisheries
> NIWA
> DDI:  +64 (4) 3860529
> ------------------------------------------------------------------------
> *From:* Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
> *Sent:* Friday, April 11, 2025 00:17
> *To:* pcreso@yahoo.com <pcreso@yahoo.com>
> *Cc:* pgsql-general@lists.postgresql.org 
> <pgsql-general@lists.postgresql.org>
> *Subject:* Re: timescaledb vs NULL vs pg_timeseries vs partman + 
> pgcron + pg_ivm
>
> Hi Brent
>
> how do you plan to cope with future upgrades based on logical 
> replication ? Do you run timescale on a dedicated/separate system from 
> the rest of your PostgreSQL cluster(s)?
>
> On 10/4/25 08:25, Amitabh Kant wrote:
>> On Wed, Apr 9, 2025 at 11:50 AM Achilleas Mantzios - cloud 
>> <a.mantzios@cloud.gatewaynet.com 
>> <mailto:a.mantzios@cloud.gatewaynet.com>> wrote:
>>
>>
>>     On 4/9/25 04:50, Amitabh Kant wrote:
>>>
>>>         Thank you, I meant the paid/supported service not the
>>>         community version. Which of the two do you use?
>>>>
>>>     I use the community version.
>>
>>
>>     On 4/9/25 05:23, Brent Wood wrote:
>>
>>>     I also use the free community edition on internal servers, but
>>>     under Ubuntu. No issues and very good performance.
>>>
>>>     Brent Wood...
>>>
>>
>>     Thanks Amitabh abd Brent, how do you plan to cope with future
>>     upgrades based on logical replication ? Do you run timescale on a
>>     dedicated/separate system from the rest of your PostgreSQL
>>     cluster(s)?
>>
>>
>> I don't use the inbuilt logical replication. I depend upon pgBackRest 
>> for my backups. My database is mainly time series data, and runs on 
>> the same systems.
> thanks
> <https://www.niwa.co.nz> 	Brent Wood
> Principal Technician - GIS and Spatial Data Management
> Programme Leader - Environmental Information Delivery
> +64-4-386-0529
>
> National Institute of Water & Atmospheric Research Ltd (NIWA)
> 301 Evans Bay Parade Hataitai Wellington New Zealand
> *Connect with NIWA:* niwa.co.nz <https://www.niwa.co.nz> Facebook 
> <https://www.facebook.com/nzniwa> LinkedIn 
> <https://www.linkedin.com/company/niwa> Twitter 
> <https://twitter.com/niwa_nz> Instagram 
> <https://www.instagram.com/niwa_science> YouTube 
> <https://www.youtube.com/channel/UCJ-j3MLMg1H59Ak2UaNLL3A>
>
> To ensure compliance with legal requirements and to maintain cyber 
> security standards, NIWA's IT systems are subject to ongoing 
> monitoring, activity logging and auditing. This monitoring and 
> auditing service may be provided by third parties. Such third parties 
> can access information transmitted to, processed by and stored on 
> NIWA's IT systems.
> Note: This email is intended solely for the use of the addressee and 
> may contain information that is confidential or subject to legal 
> professional privilege. If you receive this email in error please 
> immediately notify the sender and delete the email.