points by lysecret 15 hours ago | 0 comments
Want to hijack this someone made experience recently with https://ducklake.select/ ? We need to setup a warehouse soon and this looks enticing.
[–] denom 14 hours ago
Ducklake works well. So far it is still under active development (post AWS acquisition). There is support for local disk-based and remote S3 based datalakes, so integration tests become really easy. Seems stable and all the bugs I've found have already been patched.
[–] lysecret 11 hours ago
Any risks in terms of using it on gcp/azure?
[–] viccis 12 hours ago
Praying for the poor Ducklake employees at AWS to keep whatever demon works there at bay who insists that every single data product has to be bolted onto AWS Glue (and its various Data Catalog, Data Lake, etc. partners in crime) in the most clumsy way possible. And please keep it away from Athena, possibly the worst major query engine in terms of support for query features and file formats. I don't want to see AWS-era Ducklake become another failure like S3 Table Buckets are (imo at least).
[–] Zizizizz 15 hours ago
I'd personally love to try this in earnest as well
Not that I'd use it, I want to avoid Databricks - the Duck ecosystem is far more compelling for my modest needs, and handles local develop a lot better (IMO).