Skip to content

NFDI4Energy/Data-Anonymization-Service

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Anonymization Service

SDV based tabular data anonymization tool with a 4-step workflow:

  1. Upload data (CSV / XLS / XLSX) 1_Dataupload
  2. Detect PII 2_Scan
  3. Configure rules (exempt_columns / force_pii_columns) 3_Config
  4. Generate, preview, compare, and download synthetic data in csv format 4_result

Project Structure

  • src/
    • api.py - FastAPI endpoints
    • generate_metada.py - metadata generation and config save
    • Gaussian_Coupla_syntheticdata_generator.py - SDV synthetic data pipeline
  • ui/ - frontend page and step scripts
  • results/ - runtime outputs
    • uploaded_data/ - uploaded files converted to CSV
    • meta_data/ - generated metadata and user config
    • GaussianCopula_results/ - synthetic data, quality reports, scores
  • raw_data/ - optional mounted input folder (Docker use)

Requirements

  • Python 3.9+
  • Dependencies in requirements-api.txt
  • spaCy model:
    • python -m spacy download en_core_web_sm

Install through Docker

1) Prerequisites

2) Run

git clone https://github.com/gyt197/Data-Anonymization-Service.git
cd Data-Anonymization-Service
docker compose up --build

3) Access

App: http://localhost:8000

4) Stop

docker compose down

Docker Setup

Install from Github

1) Clone the repository

git clone https://github.com/gyt197/Data-Anonymization-Service.git

2) Navigate into the project

cd Data-Anonymization-Service

3) Create a virtual environment

python -m venv venv

4) Activate the virtual environment

On Windows

my-env\Scripts\activate

5) Install dependencies

pip install -r requirements-api.txt
pip install sdv python-multipart
python -m spacy download en_core_web_sm

6) Run the server

uvicorn src.api:app --host 0.0.0.0 --port 8000 --reload

License

MIT License

This project is licensed under the MIT License - see the LICENSE file for details.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors