You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

67 lines
1.7 KiB
Markdown

# Actes princiers -- data transformations
3 years ago
3 years ago
## Project Name
human readable name : `Actes Princiers`
The project name 'Actes Princiers' has been applied to:
3 years ago
- The project title in `datascience/actes-princiers/README.md`
- The folder created for your project in `datascience/actes-princiers`
3 years ago
- The project's python package in `datascience/actes-princiers/src/actes_princiers`
A best-practice setup includes initialising git and creating a virtual environment before running 'pip install -r src/requirements.txt'
3 years ago
## Getting started
3 years ago
3 years ago
- Install a virtual environment : `python -m venv .venv`
3 years ago
- Enable the virtual environment : `source .venv/bin/activate`
3 years ago
- install kedro `pip install kedro`
3 years ago
- Install the packages and libraries `pip install -r src/requirements.txt`
3 years ago
3 years ago
**go to `actes-princiers`'s folder**
3 years ago
Then open a terminal in the `actes-princiers`'s folder
and launch jupyter : `kedro jupyter notebook`
3 years ago
or start the ipython prompt : `kedro ipython`
3 years ago
3 years ago
## Launching the pipelines
3 years ago
**go to `actes-princiers`'s folder**
3 years ago
Open a terminal in the `actes-princiers`'s folder and launch kedro
3 years ago
`kedro run`
3 years ago
or launch a specific node in the pipeline with:
`kedro run --nodes=<node_name>`
3 years ago
3 years ago
or a search by tags with:
3 years ago
`kedro run --tags=<tag_name>`
3 years ago
The current tags are:
- `kedro run --tags="etl_transform"`: launches the XML to JSON transformations
- `kedro run --tags="populate_database"`: populates the mongodb distant database
on the target server
3 years ago
## Visualizing the pipelines
**you shall install kedro-viz before**
install kedro viz with
3 years ago
`pip install kedro-viz`
3 years ago
Then launch the command
3 years ago
`kedro viz`
3 years ago
3 years ago
## tips
3 years ago
3 years ago
You need to reload Kedro variables by calling `%reload_kedro` in your notebook and re-run the code snippet
3 years ago
3 years ago