120 lines
4.7 KiB
Org Mode
120 lines
4.7 KiB
Org Mode
[[https://circleci.com/gh/karlicoss/my/tree/master][https://circleci.com/gh/karlicoss/my/tree/master.svg?style=svg]]
|
|
|
|
Python interface into my life.
|
|
|
|
In short, this package provides programmatic access to my personal data and knowledge.
|
|
Gory details of getting data, parsing, etc. are abstracted away and you get nice and familiar Python objects.
|
|
It makes it easier to access, work with, analyze and combine data and leverage on existing libraries for data analysis like Pandas, Matplotlib, etc.
|
|
|
|
This particular setup might not necessarily be most convenient for you to use, perhaps it's more of a concept of how you can organize, access and use personal data.
|
|
But it definitely works for me, so hopefully that would help you and serve as as source of inspiration.
|
|
|
|
The readme is more of a setup manual, I'm writing about motivation and specific usecases [[https://beepb00p.xyz/mypkg.html][here]].
|
|
Short example to give you an idea: "which subreddits I find most interesting?"
|
|
|
|
#+begin_src python :python "with_my python3" :exports both
|
|
from my.reddit import get_saves
|
|
from collections import Counter
|
|
saves = get_saves()
|
|
return Counter(s.subreddit for s in saves).most_common(3)
|
|
#+end_src
|
|
|
|
#+RESULTS:
|
|
| orgmode | 42 |
|
|
| AskReddit | 29 |
|
|
| QuantifiedSelf | 28 |
|
|
|
|
* Setting up
|
|
** =my_configuration= package for private paths/repositories (optional)
|
|
If you're not planning to use private configuration (some modules don't need it) you can skip straight to the next step. Still, I'd recommend you to read anyway.
|
|
|
|
First you need to tell the package where to look for your data and external repositories, which is done though a separate (private) package named ~my_configuration~.
|
|
|
|
You can see example in ~my_configuration_template~. You can copy it somewhere else and modify to your needs.
|
|
|
|
Some explanations:
|
|
|
|
#+begin_src bash :exports results :results output
|
|
for x in $(find my_configuration_template/ | grep -v -E 'mypy_cache|.git|__pycache__'); do
|
|
if [[ -L "$x" ]]; then
|
|
echo "l $x -> $(readlink $x)"
|
|
elif [[ -d "$x" ]]; then
|
|
echo "d $x"
|
|
else
|
|
echo "f $x"
|
|
(echo "---"; cat "$x"; echo "---" ) | sed 's/^/ /'
|
|
fi
|
|
done
|
|
#+end_src
|
|
|
|
#+RESULTS:
|
|
#+begin_example
|
|
d my_configuration_template/
|
|
d my_configuration_template/my_configuration
|
|
f my_configuration_template/my_configuration/__init__.py
|
|
---
|
|
class paths:
|
|
"""
|
|
Feel free to remore hypexport if you don't need it/add your own custom settings and use them
|
|
"""
|
|
class hypexport:
|
|
export_dir = '/tmp/my_demo/backups/hypothesis'
|
|
---
|
|
d my_configuration_template/my_configuration/repos
|
|
l my_configuration_template/my_configuration/repos/hypexport -> /tmp/my_demo/hypothesis_repo
|
|
#+end_example
|
|
|
|
As you can see, generally you specify fixed paths (e.g. to backup directory) in ~__init__.py~.
|
|
Feel free to add other files as well though to organize better, it's a real python package after all!
|
|
|
|
Some things (e.g. links to external packages like [[https://github.com/karlicoss/hypexport][hypexport]]) are specified as normal symlinks in ~repos~ directory.
|
|
That way you get easy imports (e.g. =import my_configuration.repos.hypexport.model=) and proper IDE integration.
|
|
|
|
# TODO link to post about exports?
|
|
** =with_my= helper script
|
|
Next, point =with_my= script to your private configuration:
|
|
|
|
#+begin_src bash
|
|
cp with_my.example with_my
|
|
vim with_my # specify path to your my_configuration (if you want to use it)
|
|
#+end_src
|
|
|
|
It's also convenient to put =with_my= somewhere in your system path so you can run it from anywhere.
|
|
|
|
** Dependencies
|
|
Dependencies are different for specific modules you're planning to use, so it's hard to specify.
|
|
Generally you can just try using the module and then install missing packages via ~pip install --user~, should be fairly straightforward.
|
|
|
|
* Usage examples
|
|
If you run your script with ~with_my~ wrapper, you'd have ~my~ in ~PYTHONPATH~ which gives you access to your data from within the script.
|
|
|
|
- accessing Kobo books
|
|
|
|
#+begin_src bash
|
|
with_my python3 -c 'import my.books.kobo as kobo; print(kobo.get_todos())'
|
|
#+end_src
|
|
|
|
- if you have [[https://github.com/karlicoss/orger][orger]] installed, you can use its Polar module to render all highlights as org-mode file as easy as:
|
|
#+begin_src bash
|
|
with_my orger/modules/polar.py --to polar.org
|
|
#+end_src
|
|
|
|
- read/run [[./demo.py][demo.py]] for a full demonstration of setting up Hypothesis (it uses public annotations data from Github)
|
|
|
|
|
|
* Linting
|
|
|
|
#+begin_src bash
|
|
# see https://github.com/python/mypy/issues/1645 for --namespace-packages explanation
|
|
with_my mypy --namespace-packages my
|
|
#+end_src
|
|
|
|
or, set up as ~mypy.ini~ file:
|
|
|
|
#+begin_src
|
|
[mypy]
|
|
mypy_path=/path/to/my_configuration_dir
|
|
#+end_src
|
|
|
|
|
|
# TODO hmm, if package isn't using my_configuration then we don't really need it?
|