AWS hints, tips and how-tos. Things that have been picked while building out our AWS environment.
Redshift - Show Tables in schema
Get link
Facebook
X
Pinterest
Email
Other Apps
select t.table_name
from information_schema.tables t
where t.table_schema = 'myschema'-- put your schema name here
and t.table_type = 'BASE TABLE'
orderby t.table_name;
The S3 folder delusion When you AWS console you can create folders to group objects but this is just a delusion deliberately created by AWS to simplify the usage. In reality, S3 has a flat structure and all the objects are on the same level. Here’s the excerpt from AWS documentation that states this fact: In Amazon S3, buckets and objects are the primary resources, where objects are stored in buckets. Amazon S3 has a flat structure with no hierarchy like you would see in a typical file system. However, for the sake of organizational simplicity, the Amazon S3 console supports the folder concept as a means of grouping objects. Amazon S3 does this by using key name prefixes for objects. So essentially AWS is just smart enough to recognize the standard folder notation we’ve been using for ages to make this things easier for us. All about S3 objects.
S3 has no rename functionality. You need to copy to a new 'file' and delete the original. In python using boto3 s3 = boto3 . resource ( 's3' ) s3 . Object ( 'my_bucket' , 'my_file_new' ). copy_from ( CopySource = 'my_bucket/my_file_old' ) s3 . Object ( 'my_bucket' , 'my_file_old' ). delete ()
From the CLI you can delete all files recursively of the same name (/mydata.csv) using This is for cases where a file of the same name appears in subfolders of a bucket and you want to delete all occurences of that file. If you have bucket versioning turned on then you will not actually lose the file forever. aws s3 rm s3://mybucket/toplevelfolder/ --recursive --exclude "*" --include "*/mydata.csv" If you are paranoid about screwing things up there’s an optional – dryrun parameter that generates a report of what will be deleted, without actually deleting. aws s3 rm s3:// mybucket/toplevelfolder / --dryrun --recursive --exclude "*" --include "*/metadata.csv" (dryrun) delete: s3:// mybucket/toplevelfolder /part_2/mydata.csv (dryrun) delete: s3:// mybucket/toplevelfolder /part_3/mydata.csv
Comments
Post a Comment