This topic is part of the FactCheck research project.
A core aspect of FactCheck is the generation of statistical insights and metrics from crawled fact data. Your task is to reimplement our existing statistics API as a scalable stand-alone application using an (ideally Python-based) technology stack of your choice (e.g., PySpark, Pandas, NumPy). The key steps will involve...
- Fact Data Exploration: Explore our existing fact database and familiarize yourself with our data model.
- Fact Data Extraction: Extract thousands of fact data from our existing database to be used as your starting dataset, and adapt the data schema if needed.
- Metrics: Develop new or refine existing metrics from the fact data.
- API Reimplementation: Rebuild the statistics API from the ground up. Optionally, you may also create an interface that showcases its abilities.
- Deploy and Test: Deploy and test your newly developed solution alongside our server using Docker.
If needed, a suitable virtual machine will be provided to you. Depending on your strengths and interests, you may focus on...
- select data science aspects - e.g., orchestration, generation of statistics, data wrangling
- the safe and dynamic execution of semantically rich statistics
- creating a frontend (e.g., a dashboard, a Jupyter or Marimo notebook) to visualize your metrics
Technologies: Python; CouchDB; PySpark; Pandas; NumPy; Jupyter Notebooks; marimo; Docker; Apache Airflow
Tags: Aichinger; FactCheck; Data Science
