Commit graph

47 commits

Author SHA1 Message Date
Dima Gerasimov
75639a3d5e tox: some prep for potentially using uv on CI instead of pip
see https://github.com/karlicoss/HPI/issues/391
2024-09-22 20:10:52 +01:00
Dima Gerasimov
6a18f47c37 my.github.gdpr/my.zulip.organization: use kompress support for tar.gz if it's available
otherwise fall back onto unpacking into tmp dir via my.core.structure
2024-09-18 23:35:03 +01:00
Dima Gerasimov
a7439c7846 general: move assert_never to my.core.compat as it's in stdlib from 3.11
rely on typing-extensions for fallback

introducing typing-extensions dependency without fallback, should be ok since it's in the top 10 of popular packages
2024-08-16 10:22:29 +01:00
Dima Gerasimov
fb8e9909a4 tests: simplify tests for my.core.serialize a bit and simplify tox file 2024-08-07 01:08:39 +01:00
Dima Gerasimov
751ed02f43 tests: pin pytest version to <8 for now, having some test collection errors
https://docs.pytest.org/en/stable/changelog.html#collection-changes
2024-03-13 00:46:18 +00:00
karlicoss
e547acfa59 general: update minimal cachew version
had quite a few useful fixes/performance optimizations since
2023-11-07 21:24:56 +00:00
karlicoss
fe26efaea8 core/kompress: move vendorized to _deprecated, use kompress library directly 2023-10-12 23:47:05 +01:00
Dima Gerasimov
0512488241 ci: sync configs to pymplate
- add python3.12
- add ruff
2023-10-06 02:24:01 +01:00
Sean Breckenridge
ff84d8fc88 core/cli: update vendored completion files
update required click version to 8.1
so we dont regenerate the vendored completions
wrong in the future
2023-09-07 00:01:27 +01:00
Dima Gerasimov
661714f1d9 core/logging: overhaul and many improvements -- mainly to deprecate abandoned logzero
- generally saner/cleaner logger initialization

  In particular now it doesn't override logging level specified by the user code prior to instantiating the logger.

  Also remove the `LazyLogger` hack, doesn't seem like it's necessary when the above is implemented.

- get rid of `logzero` which is archived and abandoned now, use `colorlog` for coloured logging formatter

- allow configuring log level via shell via `LOGGING_LEVEL_module_name=<level>`

  E.g. `LOGGING_LEVEL_rescuexport_dal=WARNING LOGGING_LEVEL_my_rescuetime=debug ./script.py`

- port `AddExceptionTraceback` from HPI/promnesia

- port `CollapseLogsHandler` from HPI/promnesia

  Also allow configuring from the shell, e.g. `LOGGING_COLLAPSE=<level>`

- add support for `enlighten` progress bar, so it can be shared between different projects

  See https://github.com/Rockhopper-Technologies/enlighten#readme

  This allows nice CLI progressbars, e.g. for parallel processing of different files from HPI:

    ghexport.dal[111]  29%|████████████████████████████████████████████████████████████████▏              |  29/100 [00:03<00:07, 10.03 files/s]
    rexport.dal[comments]  17%|████████████████████████████████████▋                                      | 115/682 [00:03<00:14, 39.15 files/s]
    my.instagram.android   0%|▎                                                                           |    3/2631 [00:02<34:50, 1.26 files/s]

  Currently off by default, and hidden behind an env variable (`ENLIGHTEN_ENABLE=true`)
2023-06-21 18:42:15 +01:00
Dima Gerasimov
a445d2cbfe general: python3.7 will reach EOL soon, remove its support 2023-05-16 01:18:30 +01:00
seanbreckenridge
98b086f746
location fallback (#263)
see https://github.com/karlicoss/HPI/issues/262

* move home to fallback/via_home.py
* move via_ip to fallback
* add fallback model
* add stub via_ip file
* add fallback_locations for via_ip
* use protocol for locations
* estimate_from helper, via_home estimator, all.py
* via_home: add accuracy, cache history
* add datasources to gpslogger/google_takeout
* tz/via_location.py: update import to fallback
* denylist docs/installation instructions
* tz.via_location: let user customize cachew refresh time
* add via_ip.estimate_location using binary search
* use estimate_location in via_home.get_location
* tests: add gpslogger to location config stub
* tests: install tz related libs in test env
* tz: add regression test for broken windows dates

* vendorize bisect_left from python src
doesnt have a 'key' parameter till python3.10
2023-02-28 04:30:06 +00:00
Dima Gerasimov
637982a5ba ci: update ci configs
- add windows runner
- update actions versions
- other minor enhancements
2022-05-03 10:16:01 +01:00
Sean Breckenridge
6185942f78 core/cli: autocomplete module names 2022-02-23 12:15:00 -01:00
Sean Breckenridge
3f4fb64d56
core: drop py36 support, update docs for reddit (#193)
* docs: update references to my.reddit
* ci: remove 3.6, add 3.9
2022-01-27 08:26:15 +00:00
Sean Breckenridge
fb49243005
core: add hpi query command (#157)
- restructure query code for cli, some test fixes
- initial query_range implementation

    refactored functions in query some more
    to allow re-use in range_range, select()
    pretty much just calls out to a bunch
    of handlers now
2021-04-06 17:19:58 +01:00
Sean Breckenridge
349ab78fca
core/cli: switch to using click library #155
everything is backwards-compatible with the previous
interface, the only minor changes were to the doctor cmd
which can now accept more than one item to run,
and the --skip-config-check to skip the config_ok
check if the user specifies to

added a test using click.testing.CliRunner (tests
the CLI in an isolated environment), though
additional tests which aren't testing the CLI
itself (parsing arguments or decorator behaviour)
can just call the functions themselves, as they
no longer accept a argparser.Namespace and instead
accept the direct arguments
2021-04-04 10:06:59 +01:00
Sean Breckenridge
eb26cf8633
my.core.serialize: orjson with additional default and _serialize hook (#140)
basic orjson serialize, json.dumps fallback

Lots of surrounding changes from this discussion:
0593c69056
2021-03-20 00:48:03 +00:00
Dima Gerasimov
ad1cc71b0f readme: update 2021-02-20 01:39:55 +00:00
Dima Gerasimov
56d5587c20 CI: clean up tox config a bit, get rid of custom lint script 2021-02-18 02:04:40 +00:00
Dima Gerasimov
df9a7f7390 core.pandas: add check for 'error' column + add empty one by default 2021-02-14 16:20:38 +00:00
Dima Gerasimov
1849a66f08 general: get rid of example_config & use demo/stub my.config instead 2020-11-01 02:02:43 +01:00
Dima Gerasimov
ced93e6942 reflect cachew changes of exception handling and temporary suppression 2020-10-08 21:22:02 +02:00
Dima Gerasimov
24fb983399 ci: add mypy for my.hypothesis 2020-09-29 19:44:45 +02:00
Dima Gerasimov
8019389ccb cli: move doctor to core, add doc 2020-05-25 10:17:40 +01:00
Dima Gerasimov
2ede5b3a5c cli: add config check command 2020-05-25 09:49:57 +01:00
Dima Gerasimov
2a9fd54c12 Improve documentation for some modules 2020-05-17 21:56:58 +01:00
Dima Gerasimov
647b6087dd add main HPI executable 2020-05-14 23:01:50 +01:00
Dima Gerasimov
d7abff03fc add dataclasses dependency for python<3.7 2020-05-10 22:43:02 +01:00
Dima Gerasimov
505a2b22ae fix my.config.repos stub 2020-05-07 08:39:30 +01:00
Dima Gerasimov
636060db57 Simplify config discovery: get rid of the hacky stub and reimport proper config automatically 2020-05-05 22:22:32 +01:00
Dima Gerasimov
4cceccd787 add test for dynamic config attributes (import my.cfg as config) 2020-05-04 22:08:58 +01:00
Dima Gerasimov
fe763c3c04 Fix my.config handling during mypy 2020-05-04 19:52:18 +01:00
Dima Gerasimov
1f07e1a2a8 enable mypy on CI for core stuff 2020-05-04 19:52:18 +01:00
Dima Gerasimov
78dbbd3c55 prettify emfit provider 2020-05-03 13:42:31 +01:00
Dima Gerasimov
04605d9c09 attemtp to use post versions 2020-04-24 22:01:15 +01:00
Dima Gerasimov
4e861eb2b2 fix namespace packages finding 2020-04-24 21:44:03 +01:00
Dima Gerasimov
185fa9aabd update readme 2020-04-18 16:16:54 +01:00
Dima Gerasimov
ea00c2afdb name PyPi package HPI instead of 'my' 2020-04-18 15:26:13 +01:00
Dima Gerasimov
7fe6520575 Switch to github CI 2020-04-18 15:21:23 +01:00
Dima Gerasimov
46f69a8911 install subpackages properly 2020-04-15 22:25:58 +01:00
Dima Gerasimov
968c448013 Update usage docs 2020-04-12 15:01:03 +01:00
Dima Gerasimov
f766c8abe5 updates to the setup guide 2020-04-12 13:42:03 +01:00
Dima Gerasimov
be7c4d2098 WIP on using ~/.config/my/my.config 2020-04-11 20:32:26 +01:00
Dima Gerasimov
fe0521b6dc install mycfg stub along with the package 2020-02-05 20:45:44 +00:00
Dima Gerasimov
3ee2deecae Use tox 2019-12-19 20:14:26 +00:00
Dima Gerasimov
e72ad57f91 Add initial setup.py 2019-11-17 10:44:14 +00:00