Skip to content

mdobydullah/python-read-parquet-file

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Python Read Parquet File

Read Parquet file and write it to an SQL or ClickHouse database.

Create Python Virtual Environment

# Linux/macOS
python3 -m venv venv
source venv/bin/activate

# Windows
python -m venv venv
venv\Scripts\activate

# Conda
conda create --name myenv python=3.11
conda activate myenv

Install Required Libraries

pip install pandas pyarrow sqlalchemy

For ClickHouse:

pip install clickhouse-connect

Config

Update config.py file. Keep parquet or any gz.parquet files in the data directory.

Read Parquet file

python read.py

Write to Database

python write.py

Read from Database

python query.py

About

Read Parquet file and write it to an SQL or ClickHouse database.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages