datascience
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
jgenero da46e7f39a Merge branch 'feature/feature/xsl-notes-correction' into develop 2 years ago
actes-princiers Merge commit '54ca0996801b9959abcfdae785c2ddbfa25b48ec' into feature/feature/xsl-notes-correction 2 years ago
.gitignore update publication procedure 2 years ago
MiseEnProd.txt update MiseEnProd.txt 2 years ago
README.md Prise en compte de Berry et Anjou dans le pipeline etl 2 years ago
clean_intermediate_data.sh update requirements 3 years ago
config.yml install kedro 3 years ago
data_registry_add.sh add data registry utility 3 years ago
data_registry_update.sh add data registry utility 3 years ago
git_remote_add_data_registry.sh Prise en compte de Berry et Anjou dans le pipeline etl 2 years ago

README.md

Actes princiers -- data transformations

Project Name

human readable name : Actes Princiers

The project name 'Actes Princiers' has been applied to:

  • The project title in datascience/actes-princiers/README.md
  • The folder created for your project in datascience/actes-princiers
  • The project's python package in datascience/actes-princiers/src/actes_princiers

A best-practice setup includes initialising git and creating a virtual environment before running 'pip install -r src/requirements.txt'

Getting started

  • Install a virtual environment : python -m venv .venv
  • Enable the virtual environment : source .venv/bin/activate
  • install kedro pip install kedro
  • Install the packages and libraries pip install -r src/requirements.txt

go to actes-princiers's folder

Then open a terminal in the actes-princiers's folder and launch jupyter : kedro jupyter notebook or start the ipython prompt : kedro ipython

Launching the pipelines

go to actes-princiers's folder

Open a terminal in the actes-princiers's folder and launch kedro

kedro run

or launch a specific node in the pipeline with:

kedro run --nodes=<node_name>

or a search by tags with:

kedro run --tags=<tag_name>

The current tags are:

  • kedro run --tags="etl_transform": launches the XML to JSON transformations
  • kedro run --tags="populate_database": populates the mongodb distant database on the target server

Visualizing the pipelines

you shall install kedro-viz before

install kedro viz with

pip install kedro-viz

Then launch the command

kedro viz

tips

You need to reload Kedro variables by calling %reload_kedro in your notebook and re-run the code snippet