It’s funny. Snowflake internally (and even more externally) take great efforts to hide the details. However, in doing so I find it harder to determine best practices.
Much of what you’ve written here is applicable to Snowflake under the hood - which is not surprising as the top tier computing research is a very small pool of incredibly intelligent people.
As Magamet said. Another great article. Thanks for sharing.
Hey, were you able to use hidden partitioning on object storage like S3? I tried it, but I had to explicitly include the 'modified' column in the query for the partitioning to work correctly. Other permutations I tried didn't seem to work.
example:
CREATE TABLE device_data_modelling.test
WITH (
format = 'PARQUET',
location = 's3://...,
table_type = 'ICEBERG',
partitioning = ARRAY['partition_date'],
is_external=False
)
I have to pass the partition_date in the select for this to recognise as the valid partitioning col
It’s funny. Snowflake internally (and even more externally) take great efforts to hide the details. However, in doing so I find it harder to determine best practices.
Much of what you’ve written here is applicable to Snowflake under the hood - which is not surprising as the top tier computing research is a very small pool of incredibly intelligent people.
As Magamet said. Another great article. Thanks for sharing.
Hey, were you able to use hidden partitioning on object storage like S3? I tried it, but I had to explicitly include the 'modified' column in the query for the partitioning to work correctly. Other permutations I tried didn't seem to work.
example:
CREATE TABLE device_data_modelling.test
WITH (
format = 'PARQUET',
location = 's3://...,
table_type = 'ICEBERG',
partitioning = ARRAY['partition_date'],
is_external=False
)
I have to pass the partition_date in the select for this to recognise as the valid partitioning col
Thanks for the article. How can I explicitly choose whether it will be positional or equality delete files created?