Python code for research project funded by the National Council for Scientific and Technological Development (CNPq) from August 2020 to August 2021.
You can check the projects Abstract (English) and Presentation Video (Portuguese) submitted to the 29° SIICUSP (USP International Symposium of Undergraduate Research).
The code presented in integracao.ipynb consists in a series of processing steps of around 500 csv files regarding medical appointments in the municipality of São Paulo, in Brazil (which sums up to around 150 million rows and 80 GB of data). The goal was to convert this data to a relational database structure. PostgreSQL was used for storing the converted data. The whole process is summarized in the flowchart bellow.
