|
|
2 years ago | |
|---|---|---|
| actes-princiers | 2 years ago | |
| .gitignore | 2 years ago | |
| README.md | 2 years ago | |
| clean_intermediate_data.sh | 3 years ago | |
| config.yml | 3 years ago | |
| data_registry_add.sh | 3 years ago | |
| data_registry_update.sh | 3 years ago | |
| git_remote_add_data_registry.sh | 2 years ago | |
README.md
Actes princiers -- data transformations
Project Name
human readable name : Actes Princiers
The project name 'Actes Princiers' has been applied to:
- The project title in
datascience/actes-princiers/README.md - The folder created for your project in
datascience/actes-princiers - The project's python package in
datascience/actes-princiers/src/actes_princiers
A best-practice setup includes initialising git and creating a virtual environment before running 'pip install -r src/requirements.txt'
Getting started
- Install a virtual environment :
python -m venv .venv - Enable the virtual environment :
source .venv/bin/activate - install kedro
pip install kedro - Install the packages and libraries
pip install -r src/requirements.txt
go to actes-princiers's folder
Then open a terminal in the actes-princiers's folder
and launch jupyter : kedro jupyter notebook
or start the ipython prompt : kedro ipython
Launching the pipelines
go to actes-princiers's folder
Open a terminal in the actes-princiers's folder and launch kedro
kedro run
or launch a specific node in the pipeline with:
kedro run --nodes=<node_name>
or a search by tags with:
kedro run --tags=<tag_name>
The current tags are:
kedro run --tags="etl_transform": launches the XML to JSON transformationskedro run --tags="populate_database": populates the mongodb distant database on the target server
Visualizing the pipelines
you shall install kedro-viz before
install kedro viz with
pip install kedro-viz
Then launch the command
kedro viz
tips
You need to reload Kedro variables by calling %reload_kedro in your notebook and re-run the code snippet