This project aims to analyze the change in research volume for orphan diseases in the United States in the past 20 years.
Tech used: Python (pandas), Jupyter Notebook, Palantir Foundry The data was first downloaded from the clinicalTrials.gov Website, containing all registered clinical trials (583,713 rows). To limit the research to the United States the 'locations' row was unnested and all rows containing non US locations where filtered. Remaining were 191,587 US only trials. The orphanData table was downloaded from Kaggle, containing 11,456 rows.
After cleaning both datasets they were joined together, this was done using Palantir Foundry, because of the computing power needed. The project is currently not yet done, so these steps will come later.
For a detailed overview of all the datasets, their origin, and the prepreation done go to /data/README.md