Rename my_configuration to mycfg for brevity

This commit is contained in:
Dima Gerasimov 2019-12-19 19:45:25 +00:00 committed by Dmitrii Gerasimov
parent cd804091c3
commit 066641a4ce
31 changed files with 65 additions and 70 deletions

View file

@ -15,7 +15,7 @@ jobs:
# - run: python3 -m mypy --namespace-packages my
# - run: python3 -m pylint -E my
# TODO need to keep full my_configuration.py as example? to dummy run CI
# TODO need to keep full mycfg.py as example? to dummy run CI

View file

@ -25,17 +25,17 @@ Short example to give you an idea: "which subreddits I find most interesting?"
| QuantifiedSelf | 28 |
* Setting up
** =my_configuration= package for private paths/repositories (optional)
** =mycfg= package for private paths/repositories (optional)
If you're not planning to use private configuration (some modules don't need it) you can skip straight to the next step. Still, I'd recommend you to read anyway.
First you need to tell the package where to look for your data and external repositories, which is done though a separate (private) package named ~my_configuration~.
First you need to tell the package where to look for your data and external repositories, which is done though a separate (private) package named ~mycfg~.
You can see example in ~my_configuration_template~. You can copy it somewhere else and modify to your needs.
You can see example in ~mycfg_template~. You can copy it somewhere else and modify to your needs.
Some explanations:
#+begin_src bash :exports results :results output
for x in $(find my_configuration_template/ | grep -v -E 'mypy_cache|.git|__pycache__'); do
for x in $(find mycfg_template/ | grep -v -E 'mypy_cache|.git|__pycache__'); do
if [[ -L "$x" ]]; then
echo "l $x -> $(readlink $x)"
elif [[ -d "$x" ]]; then
@ -49,9 +49,9 @@ Some explanations:
#+RESULTS:
#+begin_example
d my_configuration_template/
d my_configuration_template/my_configuration
f my_configuration_template/my_configuration/__init__.py
d mycfg_template/
d mycfg_template/mycfg
f mycfg_template/mycfg/__init__.py
---
class paths:
"""
@ -60,15 +60,15 @@ f my_configuration_template/my_configuration/__init__.py
class hypexport:
export_dir = '/tmp/my_demo/backups/hypothesis'
---
d my_configuration_template/my_configuration/repos
l my_configuration_template/my_configuration/repos/hypexport -> /tmp/my_demo/hypothesis_repo
d mycfg_template/mycfg/repos
l mycfg_template/mycfg/repos/hypexport -> /tmp/my_demo/hypothesis_repo
#+end_example
As you can see, generally you specify fixed paths (e.g. to backup directory) in ~__init__.py~.
Feel free to add other files as well though to organize better, it's a real python package after all!
Some things (e.g. links to external packages like [[https://github.com/karlicoss/hypexport][hypexport]]) are specified as normal symlinks in ~repos~ directory.
That way you get easy imports (e.g. =import my_configuration.repos.hypexport.model=) and proper IDE integration.
That way you get easy imports (e.g. =import mycfg.repos.hypexport.model=) and proper IDE integration.
# TODO link to post about exports?
** =with_my= helper script
@ -76,7 +76,7 @@ Next, point =with_my= script to your private configuration:
#+begin_src bash
cp with_my.example with_my
vim with_my # specify path to your my_configuration (if you want to use it)
vim with_my # specify path to your mycfg (if you want to use it)
#+end_src
It's also convenient to put =with_my= somewhere in your system path so you can run it from anywhere.
@ -113,8 +113,8 @@ or, set up as ~mypy.ini~ file:
#+begin_src
[mypy]
mypy_path=/path/to/my_configuration_dir
mypy_path=/path/to/mycfg_dir
#+end_src
# TODO hmm, if package isn't using my_configuration then we don't really need it?
# TODO hmm, if package isn't using mycfg then we don't really need it?

View file

@ -35,9 +35,9 @@ def run():
with_my = 'my_repo/with_my'
copy('my_repo/with_my.example', with_my)
my_configuration_root = abspath('my_repo/my_configuration_template')
mycfg_root = abspath('my_repo/mycfg_template')
# edit the config and set path to private configuration
my = Path(with_my).read_text().replace("MY_CONFIGURATION_DIR = ''", "MY_CONFIGURATION_DIR = '{}'".format(my_configuration_root))
my = Path(with_my).read_text().replace("MYCFG_DIR = ''", "MYCFG_DIR = '{}'".format(mycfg_root))
Path(with_my).write_text(my)
#

View file

@ -13,23 +13,23 @@ Various thoughts on organizing
- not mypy/pylint friendly at all?
- Second alternative:
symlinks in my_configuration and direct import?
symlinks in mycfg and direct import?
+ mypy/pylint friendly
? keeping a symlink to model.py is not much worse than hardcoding path. so it's ok I guess
* Thoughts on organizing imports
- First way:
import my_configuration.hypexport_model as hypexport
works, but my_configuration is scattered across the repository?
import mycfg.hypexport_model as hypexport
works, but mycfg is scattered across the repository?
Second way:
from . import my_configuration?
from . import mycfg?
doesn't seem to work with subpackages?
right, perhaps symlinking is a good idea after all?...
Third way:
import my_configuration.repos.hypexport.model as hypexport
import mycfg.repos.hypexport.model as hypexport
works, but MYPYPATH doesn't seem to be happy...
ok, --namespace-packages solves it..

View file

@ -1,13 +1,7 @@
# TODO just eval setup file to populate paths etc?
# TODO note sure if it would
# TODO maybe just import everything?
# TODO how to make it mypy friendly? maybe defensive import? or mypy config? or interface file?
try:
import my_configuration
paths = my_configuration.paths # type: ignore
import mycfg
except ImportError:
import warnings
warnings.warn("my_configuration package isn't found! That might result in issues")
paths = None # type: ignore
warnings.warn("mycfg package isn't found! That might result in issues")

View file

@ -14,7 +14,7 @@ from kython import dictify
# TODO vendorize in my. pkg? It's quite handy...
from kython.klogging import LazyLogger
from my_configuration import paths
from mycfg import paths
logger = LazyLogger('bluemaestro', level=logging.DEBUG)

View file

@ -8,7 +8,7 @@ from kython import listify
from kython.org import parse_org_date
from kython.kerror import Res, echain
from my_configuration import paths
from mycfg import paths
import pandas as pd # type: ignore

View file

@ -1,7 +1,7 @@
from typing import Callable, Union, List
from my_configuration import paths
from my_configuration.repos.kobuddy.src.kobuddy import *
from mycfg import paths
from mycfg.repos.kobuddy.src.kobuddy import *
set_databases(paths.kobuddy.export_dir)

View file

@ -4,7 +4,7 @@ import re
from typing import Tuple, Iterator, List, Union
from my_configuration.holidays_data import HOLIDAYS_DATA
from mycfg.holidays_data import HOLIDAYS_DATA
# pip3 install workalendar

View file

@ -8,8 +8,8 @@ import pytz
from ..common import get_files
from my_configuration import paths
import my_configuration.repos.ghexport.model as ghexport
from mycfg import paths
import mycfg.repos.ghexport.model as ghexport
def get_logger():

View file

@ -3,8 +3,8 @@ from shutil import rmtree
from tempfile import TemporaryDirectory
from typing import Iterator, Union
import my_configuration.repos.fbmessengerexport.model as messenger
from my_configuration import paths
import mycfg.repos.fbmessengerexport.model as messenger
from mycfg import paths
def _dump_helper(model: messenger.Model, tdir: Path) -> None:

View file

@ -1,7 +1,8 @@
from . import paths
from .common import listify
from ._rss import Subscription
from mycfg import paths
import json
from pathlib import Path
from typing import Dict, List

View file

@ -1,7 +1,8 @@
from . import paths
from .common import listify
from ._rss import Subscription
from mycfg import paths
import json
from pathlib import Path
from typing import Dict, List

View file

@ -22,7 +22,7 @@ def get_logger():
def _get_exports() -> List[Path]:
from my_configuration import paths
from mycfg import paths
return get_files(paths.foursquare.export_path, '*.json')

View file

@ -6,9 +6,9 @@ from .common import group_by_key, the, cproperty, PathIsh, import_file
try:
# TODO might be worth having a special mode for type checking with and without my_configuration?
# TODO might be worth having a special mode for type checking with and without mycfg?
# TODO could somehow use typing.TYPE_CHECKING for that?
import my_configuration.repos.hypexport.model as hypexport
import mycfg.repos.hypexport.model as hypexport
Highlight = hypexport.Highlight
Model = hypexport.Model
except:
@ -26,7 +26,7 @@ class Config(NamedTuple):
if ep is not None:
return Path(ep)
else:
from my_configuration import paths
from mycfg import paths
return Path(paths.hypothesis.export_path)
@property
@ -37,7 +37,7 @@ class Config(NamedTuple):
else:
global Model
global Highlight
import my_configuration.repos.hypexport.model as hypexport
import mycfg.repos.hypexport.model as hypexport
# TODO a bit hacky.. not sure how to make it both mypy and runtime safe..
Model = hypexport.Model
Highlight = hypexport.Highlight
@ -52,7 +52,7 @@ def configure(*, export_path: Optional[PathIsh]=None, hypexport_path: Optional[P
hypexport_path_=hypexport_path,
)
# TODO for the purposes of mypy, try importing my_configuration anyway?
# TODO for the purposes of mypy, try importing mycfg anyway?
# return type for this method as well
# TODO check if it works at runtime..
def get_model() -> Model:

View file

@ -3,7 +3,7 @@ Uses instapaper API data export JSON file.
Set via
- ~configure~ method
- or in ~my_configuration.instpaper.export_path~
- or in ~mycfg.instpaper.export_path~
TODO upload my exporter script to github..
"""
@ -29,8 +29,8 @@ def _get_files():
# TODO use helper method from common to get json[s]?
export_path = _export_path
if export_path is None:
# fallback to my_configuration
# TODO import my_configuration?
# fallback to mycfg
# TODO import mycfg?
from . import paths
export_path = paths.instapaper.export_path
return list(sorted(Path(export_path).glob('*.json')))

View file

@ -7,7 +7,7 @@ from pathlib import Path
import logging
import pytz
from my_configuration import paths
from mycfg import paths
BDIR = paths.jawbone.export_dir

View file

@ -87,7 +87,7 @@ def iter_useful(data_file: str):
# TODO <<< hmm. these files do contain deep and light sleep??
# also steps stats??
from my_configuration import paths
from mycfg import paths
p = paths.jawbone.export_dir / 'old_csv'
# TODO with_my?

View file

@ -7,7 +7,7 @@ import json
import pytz
from my_configuration import paths
from mycfg import paths
# TODO Json type?
# TODO memoised properties?

View file

@ -6,8 +6,7 @@ from typing import Iterator, List, NamedTuple
from ..common import get_files
# TODO eh. rename to my_cfg? easier to type
from my_configuration import paths
from mycfg import paths
def _get_last():
return max(get_files(paths.imdb.export_path, glob='*.csv'))

View file

@ -8,7 +8,7 @@ from kython.kompress import open as kopen
from ..common import get_files
from my_configuration import paths
from mycfg import paths
def _get_last_takeout():

View file

@ -1,12 +1,12 @@
#!/usr/bin/env python3
from . import paths
from .common import import_file
from pathlib import Path
# path to pdfannots (https://github.com/0xabu/pdfannots)
pdfannots = import_file(paths.pdfs.pdfannots_py)
import mycfg.repos.pdfannots.pdfannots as pdfannots
from mycfg import paths
from datetime import datetime

View file

@ -41,7 +41,7 @@ class Article(NamedTuple):
# TODO add tags?
# TODO integrate with my_configuration
# TODO integrate with mycfg
def get_articles(json_path: Path) -> Sequence[Article]:
import json
raw = json.loads(json_path.read_text())['list']

View file

@ -2,8 +2,8 @@
from pathlib import Path, PosixPath
from typing import List, Sequence, Mapping
from my_configuration import paths
import my_configuration.repos.rexport.model as rexport
from mycfg import paths
import mycfg.repos.rexport.model as rexport
# TODO Move this to kython.kompress?

View file

@ -9,7 +9,7 @@ from .common import get_files
# TODO get rid of it
from kython import group_by_cmp # type: ignore
from my_configuration import paths
from mycfg import paths
def get_logger():
@ -17,11 +17,11 @@ def get_logger():
def _get_exports() -> List[Path]:
from my_configuration import paths
from mycfg import paths
return get_files(paths.rescuetime.export_path, '*.json')
import my_configuration.repos.rescuexport.model as rescuexport
import mycfg.repos.rescuexport.model as rescuexport
Model = rescuexport.Model

View file

@ -5,7 +5,7 @@ See https://help.twitter.com/en/managing-your-account/how-to-download-your-twitt
Expects path to be set
- via ~configure~ (before calling anything else)
- or in ~my_configuration.twitter.export_path~
- or in ~mycfg.twitter.export_path~
"""
@ -30,8 +30,8 @@ def configure(*, export_path: Optional[PathIsh]=None) -> None:
def _get_export() -> Path:
export_path = _export_path
if export_path is None:
# fallback to my_configuration
from . import paths
# fallback to mycfg
from mycfg import paths
export_path = paths.twitter.export_path
p = Path(export_path)
if p.is_dir():

View file

@ -2,7 +2,7 @@ from datetime import datetime
import json
from typing import NamedTuple, Iterator, Dict, Union, Sequence, Optional
from my_configuration import paths
from mycfg import paths
class Favorite(NamedTuple):

View file

@ -13,7 +13,7 @@ Feel free to use your preferred way of managing these packages otherwise.
# can be empty if you're not planning to use modules that use private configuration
# otherwise see readme to learn how to set it
MY_CONFIGURATION_DIR = ''
MYCFG_DIR = ''
######
@ -39,10 +39,10 @@ if __name__ == '__main__':
val = val + ':' + path
os.environ[envvar] = val
# TODO wonder why py.typed file in my_configuration didn't help?
# TODO wonder why py.typed file in mycfg didn't help?
for v in ['MYPYPATH', 'PYTHONPATH']:
upd(v, MY_DIR)
upd(v, MY_CONFIGURATION_DIR)
upd(v, MYCFG_DIR)
rest = sys.argv[1:]
os.execvp(rest[0], rest)