Commit graph

88 commits

Author SHA1 Message Date
purarue
a7f05c2cad doc: spelling fixes 2024-11-26 21:51:40 +00:00
purarue
7ab6f0d5cb chore: update urls 2024-10-30 20:12:00 +00:00
Dima Gerasimov
477b7e8fd3 docs: minor update to overlays docs 2024-03-13 00:46:18 +00:00
Dima Gerasimov
a8f8858cb1 docs: document more experiments with overlays in docs 2023-12-22 02:54:36 +00:00
Dima Gerasimov
adbc0e73a2 docs: add note about directly checking overlays with mypy 2023-12-22 02:54:36 +00:00
Dima Gerasimov
84d835962d docs: some documentation/thoughts on properly implementing overlay packages 2023-12-20 02:51:27 +00:00
karlicoss
bd27bd4c24 docs: add documentation on logging during HPI module development 2023-10-29 00:50:22 +01:00
Sean Breckenridge
2a46341ce2 my.core.logging: compatibility with HPI_LOGS
re-adds a removed check for HPI_LOGS, add some docs

fix the checks for browserexport/takeout logs to
use the computed level from my.core.logging
2023-09-07 02:36:26 +01:00
Dima Gerasimov
661714f1d9 core/logging: overhaul and many improvements -- mainly to deprecate abandoned logzero
- generally saner/cleaner logger initialization

  In particular now it doesn't override logging level specified by the user code prior to instantiating the logger.

  Also remove the `LazyLogger` hack, doesn't seem like it's necessary when the above is implemented.

- get rid of `logzero` which is archived and abandoned now, use `colorlog` for coloured logging formatter

- allow configuring log level via shell via `LOGGING_LEVEL_module_name=<level>`

  E.g. `LOGGING_LEVEL_rescuexport_dal=WARNING LOGGING_LEVEL_my_rescuetime=debug ./script.py`

- port `AddExceptionTraceback` from HPI/promnesia

- port `CollapseLogsHandler` from HPI/promnesia

  Also allow configuring from the shell, e.g. `LOGGING_COLLAPSE=<level>`

- add support for `enlighten` progress bar, so it can be shared between different projects

  See https://github.com/Rockhopper-Technologies/enlighten#readme

  This allows nice CLI progressbars, e.g. for parallel processing of different files from HPI:

    ghexport.dal[111]  29%|████████████████████████████████████████████████████████████████▏              |  29/100 [00:03<00:07, 10.03 files/s]
    rexport.dal[comments]  17%|████████████████████████████████████▋                                      | 115/682 [00:03<00:14, 39.15 files/s]
    my.instagram.android   0%|▎                                                                           |    3/2631 [00:02<34:50, 1.26 files/s]

  Currently off by default, and hidden behind an env variable (`ENLIGHTEN_ENABLE=true`)
2023-06-21 18:42:15 +01:00
seanbreckenridge
7a32302d66
query: add --warn-exceptions, dateparser, docs (#290)
* query: add --warn-exceptions, dateparser, docs

added --warn-exceptions (like --raise-exceptions/--drop-exceptions, but
lets you pass a warn_func if you want to customize how the exceptions are
handled. By default this creates a logger in main and logs the exception

added dateparser as a fallback if its installed (it's not a strong dependency, but
I mentioned in the docs that it's useful for parsing dates/times)

added docs for query, and a few examples

--output gpx respects the --{drop,warn,raise}--exceptions flags, have
an example of that in the docs as well
2023-04-18 00:15:35 +01:00
Kian-Meng Ang
d2ef23fcb4 docs: fix typos
found via `codespell -L copie,datas,pres,fo,tooks,noo,ue,ket,frop`
2023-03-27 03:02:35 +01:00
seanbreckenridge
79eeab2128
cli completion doc updates, hide legacy import warning (#279)
* core/cli: hide warnings when autocompleting

* link to completion in setup/troubleshooting
* update completion docs to make source path clear
2023-03-06 21:36:36 +00:00
Sean Breckenridge
435cb020f9 add example for denylist, update ci 2023-02-28 20:55:12 +00:00
seanbreckenridge
98b086f746
location fallback (#263)
see https://github.com/karlicoss/HPI/issues/262

* move home to fallback/via_home.py
* move via_ip to fallback
* add fallback model
* add stub via_ip file
* add fallback_locations for via_ip
* use protocol for locations
* estimate_from helper, via_home estimator, all.py
* via_home: add accuracy, cache history
* add datasources to gpslogger/google_takeout
* tz/via_location.py: update import to fallback
* denylist docs/installation instructions
* tz.via_location: let user customize cachew refresh time
* add via_ip.estimate_location using binary search
* use estimate_location in via_home.get_location
* tests: add gpslogger to location config stub
* tests: install tz related libs in test env
* tz: add regression test for broken windows dates

* vendorize bisect_left from python src
doesnt have a 'key' parameter till python3.10
2023-02-28 04:30:06 +00:00
Sean Breckenridge
7925ec81b6 docs: browser - fix examples for config 2022-08-29 00:03:32 +01:00
seanbreckenridge
0ce44bf0d1
doctor: better quick option propogation for stats (#239)
doctor: better quick option propogation for stats

* use contextmanager for quick stats instead of editing global state
  directly
* send quick to lots of stat related functions, so they
could possibly be used without doctor, if someone wanted to
* if a stats function has a 'quick' kwarg, send the value
there as well
* add an option to sort locations in my.time.tz.via_location
2022-05-02 00:13:05 +01:00
Sean Breckenridge
f43eedd52a docs: describe the all.py/import_source pattern 2022-04-27 07:57:16 +01:00
seanbreckenridge
2cb836181b
location: add all.py, using takeout/gpslogger/ip (#237)
* location: add all.py, using takeout/gpslogger/ip, update docs
2022-04-26 21:11:35 +01:00
Sean Breckenridge
66a00c6ada docs: add docs for google_takeout_parser 2022-04-25 02:52:34 +01:00
Sean Breckenridge
16c777b45a my.config: catch possible nested config errors 2022-04-10 16:51:15 +01:00
seanbreckenridge
9e5cd60ff2
browser: parse browser history using browserexport (#216)
* browser: parse browser history using browserexport

from seanbreckenridge/HPI module:
1fba8ccf2f/my/browser/export.py
2022-02-13 23:56:05 +00:00
Sean Breckenridge
059c4ae791 docs: add link to template 2022-02-11 09:33:03 +00:00
Sean Breckenridge
a791b25650 core/cli: add --debug flag, add HPI_LOGS to docs 2022-02-11 09:31:10 +00:00
Sean Breckenridge
3f4fb64d56
core: drop py36 support, update docs for reddit (#193)
* docs: update references to my.reddit
* ci: remove 3.6, add 3.9
2022-01-27 08:26:15 +00:00
Sean Breckenridge
8033b5cdbd docs/reddit: fix code block 2021-12-19 17:20:12 +00:00
Sean Breckenridge
4364484192 docs: add hpi query example, links to other repos
also updated the MODULE_DESIGN docs to mention the
current workaround for converting single file
modules to namespace packages through the
deprecation warning
2021-12-19 17:20:12 +00:00
Sean Breckenridge
8422c6e420
my.reddit: refactor into module that supports pushshift/gdpr (#179)
* initial pushshift/rexport merge implementation, using id for merging
* smarter module deprecation warning using regex
* add `RedditBase` from promnesia
* `import_source` helper for gracefully handing mixin data sources
2021-10-31 20:39:04 +00:00
Sean Breckenridge
4a04c09f31 docs: fix copy-paste errors/spelling mistakes 2021-07-10 10:56:23 +01:00
Dima Gerasimov
2a4bddea79 polar: move to top level, add page support 2021-05-29 13:26:01 +01:00
Sean Breckenridge
c1b70cd90e
docs: add some documentation on module design (#160) 2021-04-11 16:53:43 +01:00
Dima Gerasimov
ad1cc71b0f readme: update 2021-02-20 01:39:55 +00:00
Dima Gerasimov
56d5587c20 CI: clean up tox config a bit, get rid of custom lint script 2021-02-18 02:04:40 +00:00
Dima Gerasimov
f40b804833 update setup documentation 2020-11-01 02:02:43 +01:00
Dima Gerasimov
1849a66f08 general: get rid of example_config & use demo/stub my.config instead 2020-11-01 02:02:43 +01:00
Dima Gerasimov
3a9e3e080f my.time.tz: implement different policies for localizing 2020-11-01 01:51:10 +01:00
Dima Gerasimov
4666378f7e my.location.home: simplify config format, make it a bit more robust + tests 2020-10-12 09:05:11 +02:00
Dima Gerasimov
dfea664f57 add my.location.home, use it as location/timezone fallback 2020-10-09 22:09:19 +02:00
Dima Gerasimov
ba9acc3445 my.location: let takeout provider be in a separate my.location.google; add CI test & enable mypy 2020-10-08 21:31:26 +02:00
Dima Gerasimov
6242307d7a my.bluemaestro: run against testdata, add on CI 2020-10-04 01:40:52 +02:00
Dima Gerasimov
e63c159b80 my.body.exercise: add more annotations & ci check 2020-10-03 18:24:08 +02:00
Dima Gerasimov
d8841d0d7a my.endomondo: add fake data generator, test mypy 2020-10-02 00:37:08 +02:00
Dima Gerasimov
1c20eb27aa CI: add mypy checks for my.reddit, my.pocket and my.github.ghexport 2020-09-30 23:33:06 +02:00
Dima Gerasimov
0682919449 general: use module dependencies as proper PIP packages + fallback 2020-09-30 23:33:06 +02:00
Dima Gerasimov
dc642b5a6d my.instapaper: add stat; add mypy checks on CI 2020-09-29 20:43:34 +02:00
Dima Gerasimov
3404b3fcf1 my.instapaper: use instapexport from PIP package 2020-09-29 20:43:34 +02:00
Dima Gerasimov
24fb983399 ci: add mypy for my.hypothesis 2020-09-29 19:44:45 +02:00
Dima Gerasimov
deefa9fbbc Use hypexport package in demo.py, clean up tox 2020-09-29 19:44:45 +02:00
Adrien Lacquemant
5b2cc577f2 Correct command to create config file 2020-08-20 20:14:55 +01:00
Dima Gerasimov
821eb47c93 kobo: BREAKING changes. Use kobuddy module directly, rename export_dir to export_path.
Hopefully this makes a lot of sense in the first place, and not that many users, so deserves breaking.
2020-06-04 22:50:52 +01:00
Dima Gerasimov
3d7844b711 core: support '' for explicitly set empty path set 2020-06-01 23:45:26 +01:00