install kedro
gwen
3 years ago
parent
148e3bdcac
commit
23af8d9eed
@ -0,0 +1 @@
|
||||
.venv
|
||||
@ -1,92 +1,41 @@
|
||||
# datascience
|
||||
# Actes princiers -- refactoring datascience
|
||||
|
||||
## Getting started
|
||||
|
||||
- Install the virtual environment : `python3 -m venv .venv`
|
||||
- Enable the virtual environment : `source .venv/bin/activate`
|
||||
- Install the packages and libraries `pip install -r src/requirements.txt`
|
||||
|
||||
## Getting started
|
||||
## Developper's rules and guidelines
|
||||
|
||||
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
|
||||
Declare any dependencies in `src/requirements.txt` for `pip` installation.
|
||||
|
||||
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
|
||||
To install them, run: `pip install -r src/requirements.txt`
|
||||
|
||||
## Add your files
|
||||
## Project dependencies
|
||||
|
||||
- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
|
||||
- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
|
||||
To generate or update the dependency requirements for your project:
|
||||
|
||||
```
|
||||
cd existing_repo
|
||||
git remote add origin https://gitlab.huma-num.fr/gremond/datascience.git
|
||||
git branch -M main
|
||||
git push -uf origin main
|
||||
kedro build-reqs
|
||||
```
|
||||
|
||||
## Integrate with your tools
|
||||
|
||||
- [ ] [Set up project integrations](https://gitlab.huma-num.fr/gremond/datascience/-/settings/integrations)
|
||||
|
||||
## Collaborate with your team
|
||||
|
||||
- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
|
||||
- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
|
||||
- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
|
||||
- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
|
||||
- [ ] [Automatically merge when pipeline succeeds](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
|
||||
|
||||
## Test and Deploy
|
||||
|
||||
Use the built-in continuous integration in GitLab.
|
||||
|
||||
- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
|
||||
- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing(SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
|
||||
- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
|
||||
- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
|
||||
- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
|
||||
|
||||
***
|
||||
|
||||
# Editing this README
|
||||
|
||||
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.
|
||||
|
||||
## Suggestions for a good README
|
||||
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
|
||||
|
||||
## Name
|
||||
Choose a self-explaining name for your project.
|
||||
|
||||
## Description
|
||||
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
|
||||
|
||||
## Badges
|
||||
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
|
||||
|
||||
## Visuals
|
||||
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
|
||||
|
||||
## Installation
|
||||
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
|
||||
|
||||
## Usage
|
||||
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
|
||||
This will `pip-compile` the contents of `src/requirements.txt` into a new file `src/requirements.lock`. You can see the output of the resolution by opening `src/requirements.lock`.
|
||||
|
||||
## Support
|
||||
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
|
||||
After this, if you'd like to update your project requirements, please update `src/requirements.txt` and re-run `kedro build-reqs`.
|
||||
|
||||
## Roadmap
|
||||
If you have ideas for releases in the future, it is a good idea to list them in the README.
|
||||
[Further information about project dependencies](https://kedro.readthedocs.io/en/stable/kedro_project_setup/dependencies.html#project-specific-dependencies)
|
||||
|
||||
## Contributing
|
||||
State if you are open to contributions and what your requirements are for accepting them.
|
||||
|
||||
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
|
||||
Project Name
|
||||
============
|
||||
|
||||
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
|
||||
human readable name : `Actes Princiers`
|
||||
|
||||
## Authors and acknowledgment
|
||||
Show your appreciation to those who have contributed to the project.
|
||||
The project name 'Actes Princiers' has been applied to:
|
||||
|
||||
## License
|
||||
For open source projects, say how it is licensed.
|
||||
- The project title in `datascience/actes-princiers/README.md`
|
||||
- The folder created for your project in `datascience/actes-princiers`
|
||||
- The project's python package in `datascience/actes-princiers/src/actes_princiers`
|
||||
|
||||
## Project status
|
||||
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
|
||||
A best-practice setup includes initialising git and creating a virtual environment before running 'pip install -r src/requirements.txt'
|
||||
|
||||
@ -0,0 +1,159 @@
|
||||
##########################
|
||||
# KEDRO PROJECT
|
||||
|
||||
# ignore all local configuration
|
||||
conf/local/**
|
||||
!conf/local/.gitkeep
|
||||
.telemetry
|
||||
|
||||
# ignore potentially sensitive credentials files
|
||||
conf/**/*credentials*
|
||||
|
||||
# ignore everything in the following folders
|
||||
data/**
|
||||
logs/**
|
||||
|
||||
# except their sub-folders
|
||||
!data/**/
|
||||
!logs/**/
|
||||
|
||||
# also keep all .gitkeep files
|
||||
!.gitkeep
|
||||
|
||||
# also keep the example dataset
|
||||
!data/01_raw/iris.csv
|
||||
|
||||
|
||||
##########################
|
||||
# Common files
|
||||
|
||||
# IntelliJ
|
||||
.idea/
|
||||
*.iml
|
||||
out/
|
||||
.idea_modules/
|
||||
|
||||
### macOS
|
||||
*.DS_Store
|
||||
.AppleDouble
|
||||
.LSOverride
|
||||
.Trashes
|
||||
|
||||
# Vim
|
||||
*~
|
||||
.*.swo
|
||||
.*.swp
|
||||
|
||||
# emacs
|
||||
*~
|
||||
\#*\#
|
||||
/.emacs.desktop
|
||||
/.emacs.desktop.lock
|
||||
*.elc
|
||||
|
||||
# JIRA plugin
|
||||
atlassian-ide-plugin.xml
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
### Python template
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
.cache
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
*.cover
|
||||
.hypothesis/
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
.static_storage/
|
||||
.media/
|
||||
local_settings.py
|
||||
|
||||
# Flask stuff:
|
||||
instance/
|
||||
.webassets-cache
|
||||
|
||||
# Scrapy stuff:
|
||||
.scrapy
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
target/
|
||||
|
||||
# Jupyter Notebook
|
||||
.ipynb_checkpoints
|
||||
|
||||
# IPython
|
||||
.ipython/profile_default/history.sqlite
|
||||
.ipython/profile_default/startup/README
|
||||
|
||||
# pyenv
|
||||
.python-version
|
||||
|
||||
# celery beat schedule file
|
||||
celerybeat-schedule
|
||||
|
||||
# SageMath parsed files
|
||||
*.sage.py
|
||||
|
||||
# Environments
|
||||
.env
|
||||
.envrc
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# mkdocs documentation
|
||||
/site
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
@ -0,0 +1,122 @@
|
||||
# Actes Princiers
|
||||
|
||||
## Overview
|
||||
|
||||
This is your new Kedro project, which was generated using `Kedro 0.18.10`.
|
||||
|
||||
Take a look at the [Kedro documentation](https://docs.kedro.org) to get started.
|
||||
|
||||
## Rules and guidelines
|
||||
|
||||
In order to get the best out of the template:
|
||||
|
||||
* Don't remove any lines from the `.gitignore` file we provide
|
||||
* Make sure your results can be reproduced by following a data engineering convention
|
||||
* Don't commit data to your repository
|
||||
* Don't commit any credentials or your local configuration to your repository. Keep all your credentials and local configuration in `conf/local/`
|
||||
|
||||
## How to install dependencies
|
||||
|
||||
Declare any dependencies in `src/requirements.txt` for `pip` installation and `src/environment.yml` for `conda` installation.
|
||||
|
||||
To install them, run:
|
||||
|
||||
```
|
||||
pip install -r src/requirements.txt
|
||||
```
|
||||
|
||||
## How to run your Kedro pipeline
|
||||
|
||||
You can run your Kedro project with:
|
||||
|
||||
```
|
||||
kedro run
|
||||
```
|
||||
|
||||
## How to test your Kedro project
|
||||
|
||||
Have a look at the file `src/tests/test_run.py` for instructions on how to write your tests. You can run your tests as follows:
|
||||
|
||||
```
|
||||
kedro test
|
||||
```
|
||||
|
||||
To configure the coverage threshold, go to the `.coveragerc` file.
|
||||
|
||||
## Project dependencies
|
||||
|
||||
To generate or update the dependency requirements for your project:
|
||||
|
||||
```
|
||||
kedro build-reqs
|
||||
```
|
||||
|
||||
This will `pip-compile` the contents of `src/requirements.txt` into a new file `src/requirements.lock`. You can see the output of the resolution by opening `src/requirements.lock`.
|
||||
|
||||
After this, if you'd like to update your project requirements, please update `src/requirements.txt` and re-run `kedro build-reqs`.
|
||||
|
||||
[Further information about project dependencies](https://docs.kedro.org/en/stable/kedro_project_setup/dependencies.html#project-specific-dependencies)
|
||||
|
||||
## How to work with Kedro and notebooks
|
||||
|
||||
> Note: Using `kedro jupyter` or `kedro ipython` to run your notebook provides these variables in scope: `context`, `catalog`, and `startup_error`.
|
||||
>
|
||||
> Jupyter, JupyterLab, and IPython are already included in the project requirements by default, so once you have run `pip install -r src/requirements.txt` you will not need to take any extra steps before you use them.
|
||||
|
||||
### Jupyter
|
||||
To use Jupyter notebooks in your Kedro project, you need to install Jupyter:
|
||||
|
||||
```
|
||||
pip install jupyter
|
||||
```
|
||||
|
||||
After installing Jupyter, you can start a local notebook server:
|
||||
|
||||
```
|
||||
kedro jupyter notebook
|
||||
```
|
||||
|
||||
### JupyterLab
|
||||
To use JupyterLab, you need to install it:
|
||||
|
||||
```
|
||||
pip install jupyterlab
|
||||
```
|
||||
|
||||
You can also start JupyterLab:
|
||||
|
||||
```
|
||||
kedro jupyter lab
|
||||
```
|
||||
|
||||
### IPython
|
||||
And if you want to run an IPython session:
|
||||
|
||||
```
|
||||
kedro ipython
|
||||
```
|
||||
|
||||
### How to convert notebook cells to nodes in a Kedro project
|
||||
You can move notebook code over into a Kedro project structure using a mixture of [cell tagging](https://jupyter-notebook.readthedocs.io/en/stable/changelog.html#release-5-0-0) and Kedro CLI commands.
|
||||
|
||||
By adding the `node` tag to a cell and running the command below, the cell's source code will be copied over to a Python file within `src/<package_name>/nodes/`:
|
||||
|
||||
```
|
||||
kedro jupyter convert <filepath_to_my_notebook>
|
||||
```
|
||||
> *Note:* The name of the Python file matches the name of the original notebook.
|
||||
|
||||
Alternatively, you may want to transform all your notebooks in one go. Run the following command to convert all notebook files found in the project root directory and under any of its sub-folders:
|
||||
|
||||
```
|
||||
kedro jupyter convert --all
|
||||
```
|
||||
|
||||
### How to ignore notebook output cells in `git`
|
||||
To automatically strip out all output cell contents before committing to `git`, you can run `kedro activate-nbstripout`. This will add a hook in `.git/config` which will run `nbstripout` before anything is committed to `git`.
|
||||
|
||||
> *Note:* Your output cells will be retained locally.
|
||||
|
||||
## Package your Kedro project
|
||||
|
||||
[Further information about building project documentation and packaging your project](https://docs.kedro.org/en/stable/tutorial/package_a_project.html)
|
||||
@ -0,0 +1,26 @@
|
||||
# What is this for?
|
||||
|
||||
This folder should be used to store configuration files used by Kedro or by separate tools.
|
||||
|
||||
This file can be used to provide users with instructions for how to reproduce local configuration with their own credentials. You can edit the file however you like, but you may wish to retain the information below and add your own section in the [Instructions](#Instructions) section.
|
||||
|
||||
## Local configuration
|
||||
|
||||
The `local` folder should be used for configuration that is either user-specific (e.g. IDE configuration) or protected (e.g. security keys).
|
||||
|
||||
> *Note:* Please do not check in any local configuration to version control.
|
||||
|
||||
## Base configuration
|
||||
|
||||
The `base` folder is for shared configuration, such as non-sensitive and project-related configuration that may be shared across team members.
|
||||
|
||||
WARNING: Please do not put access credentials in the base configuration folder.
|
||||
|
||||
## Instructions
|
||||
|
||||
|
||||
|
||||
|
||||
## Need help?
|
||||
|
||||
[Find out more about configuration from the Kedro documentation](https://docs.kedro.org/en/stable/kedro_project_setup/configuration.html).
|
||||
@ -0,0 +1,4 @@
|
||||
# Here you can define all your data sets by using simple YAML syntax.
|
||||
#
|
||||
# Documentation for this file format can be found in "The Data Catalog"
|
||||
# Link: https://docs.kedro.org/en/stable/data/data_catalog.html
|
||||
@ -0,0 +1,41 @@
|
||||
version: 1
|
||||
|
||||
disable_existing_loggers: False
|
||||
|
||||
formatters:
|
||||
simple:
|
||||
format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
|
||||
handlers:
|
||||
console:
|
||||
class: logging.StreamHandler
|
||||
level: INFO
|
||||
formatter: simple
|
||||
stream: ext://sys.stdout
|
||||
|
||||
info_file_handler:
|
||||
class: logging.handlers.RotatingFileHandler
|
||||
level: INFO
|
||||
formatter: simple
|
||||
filename: info.log
|
||||
maxBytes: 10485760 # 10MB
|
||||
backupCount: 20
|
||||
encoding: utf8
|
||||
delay: True
|
||||
|
||||
rich:
|
||||
class: kedro.logging.RichHandler
|
||||
rich_tracebacks: True
|
||||
# Advance options for customisation.
|
||||
# See https://docs.kedro.org/en/stable/logging/logging.html#project-side-logging-configuration
|
||||
# tracebacks_show_locals: False
|
||||
|
||||
loggers:
|
||||
kedro:
|
||||
level: INFO
|
||||
|
||||
actes_princiers:
|
||||
level: INFO
|
||||
|
||||
root:
|
||||
handlers: [rich, info_file_handler]
|
||||
@ -0,0 +1,224 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
|
||||
# actes_princiers documentation build
|
||||
# configuration file, created by sphinx-quickstart.
|
||||
#
|
||||
# This file is execfile()d with the current directory set to its
|
||||
# containing dir.
|
||||
#
|
||||
# Note that not all possible configuration values are present in this
|
||||
# autogenerated file.
|
||||
#
|
||||
# All configuration values have a default; values that are commented out
|
||||
# serve to show the default.
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
import re
|
||||
|
||||
from kedro.framework.cli.utils import find_stylesheets
|
||||
|
||||
from actes_princiers import __version__ as release
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = "actes_princiers"
|
||||
author = "Kedro"
|
||||
|
||||
# The short X.Y version.
|
||||
version = re.match(r"^([0-9]+\.[0-9]+).*", release).group(1)
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#
|
||||
# needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.napoleon",
|
||||
"sphinx_autodoc_typehints",
|
||||
"sphinx.ext.doctest",
|
||||
"sphinx.ext.todo",
|
||||
"sphinx.ext.coverage",
|
||||
"sphinx.ext.mathjax",
|
||||
"sphinx.ext.ifconfig",
|
||||
"sphinx.ext.viewcode",
|
||||
"sphinx.ext.mathjax",
|
||||
"nbsphinx",
|
||||
"myst_parser",
|
||||
"sphinx_copybutton",
|
||||
]
|
||||
|
||||
# enable autosummary plugin (table of contents for modules/classes/class
|
||||
# methods)
|
||||
autosummary_generate = True
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ["_templates"]
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
#
|
||||
source_suffix = {".rst": "restructuredtext", ".md": "markdown"}
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = "index"
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path .
|
||||
exclude_patterns = ["_build", "**.ipynb_checkpoints"]
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = "sphinx"
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = "sphinx_rtd_theme"
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#
|
||||
html_theme_options = {"collapse_navigation": False, "style_external_links": True}
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ["_static"]
|
||||
|
||||
# Custom sidebar templates, must be a dictionary that maps document names
|
||||
# to template names.
|
||||
#
|
||||
# The default sidebars (for documents that don't match any pattern) are
|
||||
# defined by theme itself. Builtin themes are using these templates by
|
||||
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
|
||||
# 'searchbox.html']``.
|
||||
#
|
||||
# html_sidebars = {}
|
||||
|
||||
html_show_sourcelink = False
|
||||
|
||||
# -- Options for HTMLHelp output ---------------------------------------------
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = "actes_princiersdoc"
|
||||
|
||||
# -- Options for LaTeX output ------------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#
|
||||
# 'papersize': 'letterpaper',
|
||||
#
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#
|
||||
# 'pointsize': '10pt',
|
||||
#
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#
|
||||
# 'preamble': '',
|
||||
#
|
||||
# Latex figure (float) alignment
|
||||
#
|
||||
# 'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(
|
||||
master_doc,
|
||||
"actes_princiers.tex",
|
||||
"actes_princiers Documentation",
|
||||
"Kedro",
|
||||
"manual",
|
||||
)
|
||||
]
|
||||
|
||||
# -- Options for manual page output ------------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [
|
||||
(
|
||||
master_doc,
|
||||
"actes_princiers",
|
||||
"actes_princiers Documentation",
|
||||
[author],
|
||||
1,
|
||||
)
|
||||
]
|
||||
|
||||
# -- Options for Texinfo output ----------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(
|
||||
master_doc,
|
||||
"actes_princiers",
|
||||
"actes_princiers Documentation",
|
||||
author,
|
||||
"actes_princiers",
|
||||
"Project actes_princiers codebase.",
|
||||
"Data-Science",
|
||||
)
|
||||
]
|
||||
|
||||
# -- Options for todo extension ----------------------------------------------
|
||||
|
||||
# If true, `todo` and `todoList` produce output, else they produce nothing.
|
||||
todo_include_todos = False
|
||||
|
||||
# -- Extension configuration -------------------------------------------------
|
||||
|
||||
# nbsphinx_prolog = """
|
||||
# see here for prolog/epilog details:
|
||||
# https://nbsphinx.readthedocs.io/en/0.3.1/prolog-and-epilog.html
|
||||
# """
|
||||
|
||||
# -- NBconvert kernel config -------------------------------------------------
|
||||
nbsphinx_kernel_name = "python3"
|
||||
|
||||
|
||||
def remove_arrows_in_examples(lines):
|
||||
for i, line in enumerate(lines):
|
||||
lines[i] = line.replace(">>>", "")
|
||||
|
||||
|
||||
def autodoc_process_docstring(app, what, name, obj, options, lines):
|
||||
remove_arrows_in_examples(lines)
|
||||
|
||||
|
||||
def skip(app, what, name, obj, skip, options):
|
||||
if name == "__init__":
|
||||
return False
|
||||
return skip
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.connect("autodoc-process-docstring", autodoc_process_docstring)
|
||||
app.connect("autodoc-skip-member", skip)
|
||||
# add Kedro stylesheets
|
||||
for stylesheet in find_stylesheets():
|
||||
app.add_css_file(stylesheet)
|
||||
@ -0,0 +1,19 @@
|
||||
.. actes_princiers documentation master file, created by sphinx-quickstart.
|
||||
You can adapt this file completely to your liking, but it should at least
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Welcome to project actes_princiers's API docs!
|
||||
=============================================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
modules
|
||||
|
||||
|
||||
Indices and tables
|
||||
==================
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`modindex`
|
||||
* :ref:`search`
|
||||
@ -0,0 +1,17 @@
|
||||
[tool.kedro]
|
||||
package_name = "actes_princiers"
|
||||
project_name = "Actes Princiers"
|
||||
kedro_init_version = "0.18.10"
|
||||
|
||||
[tool.isort]
|
||||
profile = "black"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
addopts = """
|
||||
--cov-report term-missing \
|
||||
--cov src/actes_princiers -ra"""
|
||||
|
||||
[tool.coverage.report]
|
||||
fail_under = 0
|
||||
show_missing = true
|
||||
exclude_lines = ["pragma: no cover", "raise NotImplementedError"]
|
||||
@ -0,0 +1,3 @@
|
||||
[flake8]
|
||||
max-line-length=88
|
||||
extend-ignore=E203
|
||||
@ -0,0 +1,4 @@
|
||||
"""Actes Princiers
|
||||
"""
|
||||
|
||||
__version__ = "0.1"
|
||||
@ -0,0 +1,47 @@
|
||||
"""Actes Princiers file for ensuring the package is executable
|
||||
as `actes-princiers` and `python -m actes_princiers`
|
||||
"""
|
||||
import importlib
|
||||
from pathlib import Path
|
||||
|
||||
from kedro.framework.cli.utils import KedroCliError, load_entry_points
|
||||
from kedro.framework.project import configure_project
|
||||
|
||||
|
||||
def _find_run_command(package_name):
|
||||
try:
|
||||
project_cli = importlib.import_module(f"{package_name}.cli")
|
||||
# fail gracefully if cli.py does not exist
|
||||
except ModuleNotFoundError as exc:
|
||||
if f"{package_name}.cli" not in str(exc):
|
||||
raise
|
||||
plugins = load_entry_points("project")
|
||||
run = _find_run_command_in_plugins(plugins) if plugins else None
|
||||
if run:
|
||||
# use run command from installed plugin if it exists
|
||||
return run
|
||||
# use run command from `kedro.framework.cli.project`
|
||||
from kedro.framework.cli.project import run
|
||||
|
||||
return run
|
||||
# fail badly if cli.py exists, but has no `cli` in it
|
||||
if not hasattr(project_cli, "cli"):
|
||||
raise KedroCliError(f"Cannot load commands from {package_name}.cli")
|
||||
return project_cli.run
|
||||
|
||||
|
||||
def _find_run_command_in_plugins(plugins):
|
||||
for group in plugins:
|
||||
if "run" in group.commands:
|
||||
return group.commands["run"]
|
||||
|
||||
|
||||
def main(*args, **kwargs):
|
||||
package_name = Path(__file__).parent.name
|
||||
configure_project(package_name)
|
||||
run = _find_run_command(package_name)
|
||||
run(*args, **kwargs)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -0,0 +1,16 @@
|
||||
"""Project pipelines."""
|
||||
from __future__ import annotations
|
||||
|
||||
from kedro.framework.project import find_pipelines
|
||||
from kedro.pipeline import Pipeline
|
||||
|
||||
|
||||
def register_pipelines() -> dict[str, Pipeline]:
|
||||
"""Register the project's pipelines.
|
||||
|
||||
Returns:
|
||||
A mapping from pipeline names to ``Pipeline`` objects.
|
||||
"""
|
||||
pipelines = find_pipelines()
|
||||
pipelines["__default__"] = sum(pipelines.values())
|
||||
return pipelines
|
||||
@ -0,0 +1,41 @@
|
||||
"""Project settings. There is no need to edit this file unless you want to change values
|
||||
from the Kedro defaults. For further information, including these default values, see
|
||||
https://kedro.readthedocs.io/en/stable/kedro_project_setup/settings.html."""
|
||||
|
||||
# Instantiated project hooks.
|
||||
# For example, after creating a hooks.py and defining a ProjectHooks class there, do
|
||||
# from actes_princiers.hooks import ProjectHooks
|
||||
# HOOKS = (ProjectHooks(),)
|
||||
|
||||
# Installed plugins for which to disable hook auto-registration.
|
||||
# DISABLE_HOOKS_FOR_PLUGINS = ("kedro-viz",)
|
||||
|
||||
# Class that manages storing KedroSession data.
|
||||
# from kedro.framework.session.store import BaseSessionStore
|
||||
# SESSION_STORE_CLASS = BaseSessionStore
|
||||
# Keyword arguments to pass to the `SESSION_STORE_CLASS` constructor.
|
||||
# SESSION_STORE_ARGS = {
|
||||
# "path": "./sessions"
|
||||
# }
|
||||
|
||||
# Directory that holds configuration.
|
||||
# CONF_SOURCE = "conf"
|
||||
|
||||
# Class that manages how configuration is loaded.
|
||||
# from kedro.config import OmegaConfigLoader
|
||||
# CONFIG_LOADER_CLASS = OmegaConfigLoader
|
||||
# Keyword arguments to pass to the `CONFIG_LOADER_CLASS` constructor.
|
||||
# CONFIG_LOADER_ARGS = {
|
||||
# "config_patterns": {
|
||||
# "spark" : ["spark*/"],
|
||||
# "parameters": ["parameters*", "parameters*/**", "**/parameters*"],
|
||||
# }
|
||||
# }
|
||||
|
||||
# Class that manages Kedro's library components.
|
||||
# from kedro.framework.context import KedroContext
|
||||
# CONTEXT_CLASS = KedroContext
|
||||
|
||||
# Class that manages the Data Catalog.
|
||||
# from kedro.io import DataCatalog
|
||||
# DATA_CATALOG_CLASS = DataCatalog
|
||||
@ -0,0 +1,14 @@
|
||||
black~=22.0
|
||||
flake8>=3.7.9, <5.0
|
||||
ipython>=7.31.1, <8.0; python_version < '3.8'
|
||||
ipython~=8.10; python_version >= '3.8'
|
||||
isort~=5.0
|
||||
jupyter~=1.0
|
||||
jupyterlab_server>=2.11.1, <2.16.0
|
||||
jupyterlab~=3.0, <3.6.0
|
||||
kedro~=0.18.10
|
||||
kedro-telemetry~=0.2.0
|
||||
nbstripout~=0.4
|
||||
pytest-cov~=3.0
|
||||
pytest-mock>=1.7.1, <2.0
|
||||
pytest~=7.2
|
||||
@ -0,0 +1,39 @@
|
||||
from setuptools import find_packages, setup
|
||||
|
||||
entry_point = (
|
||||
"actes-princiers = actes_princiers.__main__:main"
|
||||
)
|
||||
|
||||
|
||||
# get the dependencies and installs
|
||||
with open("requirements.txt", encoding="utf-8") as f:
|
||||
# Make sure we strip all comments and options (e.g "--extra-index-url")
|
||||
# that arise from a modified pip.conf file that configure global options
|
||||
# when running kedro build-reqs
|
||||
requires = []
|
||||
for line in f:
|
||||
req = line.split("#", 1)[0].strip()
|
||||
if req and not req.startswith("--"):
|
||||
requires.append(req)
|
||||
|
||||
setup(
|
||||
name="actes_princiers",
|
||||
version="0.1",
|
||||
packages=find_packages(exclude=["tests"]),
|
||||
entry_points={"console_scripts": [entry_point]},
|
||||
install_requires=requires,
|
||||
extras_require={
|
||||
"docs": [
|
||||
"docutils<0.18.0",
|
||||
"sphinx~=3.4.3",
|
||||
"sphinx_rtd_theme==0.5.1",
|
||||
"nbsphinx==0.8.1",
|
||||
"nbstripout~=0.4",
|
||||
"sphinx-autodoc-typehints==1.11.1",
|
||||
"sphinx_copybutton==0.3.1",
|
||||
"ipykernel>=5.3, <7.0",
|
||||
"Jinja2<3.1.0",
|
||||
"myst-parser~=0.17.2",
|
||||
]
|
||||
},
|
||||
)
|
||||
@ -0,0 +1,41 @@
|
||||
"""
|
||||
This module contains an example test.
|
||||
|
||||
Tests should be placed in ``src/tests``, in modules that mirror your
|
||||
project's structure, and in files named test_*.py. They are simply functions
|
||||
named ``test_*`` which test a unit of logic.
|
||||
|
||||
To run the tests, run ``kedro test`` from the project root directory.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from kedro.framework.project import settings
|
||||
from kedro.config import ConfigLoader
|
||||
from kedro.framework.context import KedroContext
|
||||
from kedro.framework.hooks import _create_hook_manager
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def config_loader():
|
||||
return ConfigLoader(conf_source=str(Path.cwd() / settings.CONF_SOURCE))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def project_context(config_loader):
|
||||
return KedroContext(
|
||||
package_name="actes_princiers",
|
||||
project_path=Path.cwd(),
|
||||
config_loader=config_loader,
|
||||
hook_manager=_create_hook_manager(),
|
||||
)
|
||||
|
||||
|
||||
# The tests below are here for the demonstration purpose
|
||||
# and should be replaced with the ones testing the project
|
||||
# functionality
|
||||
class TestProjectContext:
|
||||
def test_project_path(self, project_context):
|
||||
assert project_context.project_path == Path.cwd()
|
||||
@ -0,0 +1,4 @@
|
||||
output_dir: ~/actes-princiers
|
||||
project_name: Actes Princiers
|
||||
repo_name: actes-princiers
|
||||
python_package: actes_princiers
|
||||
Loading…
Reference in New Issue