Horton

Re: Storing thousands of csv files in postgresql

Ion Alberdi <ion.alberdi@pricemoov.com>

View thread View on postgr.es
From: Ion Alberdi <ion.alberdi@pricemoov.com>
To: Marcos Pegoraro <marcos@f10.com.br>
Cc: Erik Brandsberg <erik@heimdalldata.com>, pgsql-sql <pgsql-sql@lists.postgresql.org>
Date: 2022-02-15T20:43:42Z
Lists: pgsql-sql 
Hi Marcos,

>All those thousands of CSV files have a completely unknown structure ?
Exactly

Le mar. 15 févr. 2022 à 21:28, Marcos Pegoraro <marcos@f10.com.br> a écrit :

> I'm going to challenge that the question is not one for Postgres, but you
>> should be asking "What filesystem is best suited to having many files in
>> the same directory" and let the technology deal with the problem.  Postgres
>> is just dependent on the filesystem for this behavior.  And to that answer,
>> I believe, is XFS.
>>
>> He didn't say how many different schemas will have.
> All those thousands of CSV files have a completely unknown structure ?
> Or you'll have just dozens or hundreds of structures ?
>
> Marcos
>