Pinned Loading

1. Pyathena + pandas Pyathena + pandas

   1

   from pyathena import connect

   2

   import pandas as pd

   3

   conn = connect(region_name='us-west-2', work_group='my-workgroup', s3_staging_dir='s3://bucket')

   4

   df = pd.read_sql("""SELECT * FROM "db"."table";""", conn)

2. Boto3 - Start a batch job Boto3 - Start a batch job

   1

   import boto3

   2

   3

   def run_batch_job_submit(job_name, job_definition, job_queue):

   4

       try:

   5

           batch = boto3.client(service_name='batch')

3. Add a parquet file to athena Add a parquet file to athena

   1

   from pyathena import connect

   2

   3

   def add_table_to_athena(path, db, table, work_group, columns_schema, staging_dir, region_name):

   4

       # The LOCATION has to end with / or then the sub-parquet files will not be found and the table will not be created.

   5

       query = f"""CREATE EXTERNAL TABLE IF NOT EXISTS `{db}`.`{table}` ({columns_schema})

4. How to "change" a DynamoDB table name How to "change" a DynamoDB table name

   1

   Overall you will create a restoration point, create a new table from this restoration point, then delete the old table.

   2

   3

   1. Select the table you for which you would like to change the name. 

   4

   2. Go to backup tab

   5

   3. Create an On demand backup.