Commit graph

62 commits

Author SHA1 Message Date
Dima Gerasimov
71fdeca5e1 ci: update mypy config and make ruff config more consistent with other projects 2024-08-31 02:17:49 +01:00
Dima Gerasimov
9fd4227abf ruff: enable RET/PIE/PLW 2024-08-28 04:06:32 +01:00
Dima Gerasimov
72cc8ff3ac ruff: enable B warnings (mainly suppressed exceptions and unused variables) 2024-08-28 04:06:32 +01:00
Dima Gerasimov
b594377a59 ruff: enable RUF ruleset 2024-08-28 04:06:32 +01:00
Dima Gerasimov
664c40e3e8 ruff: enable FBT rules to detect boolean arguments use without kwargs 2024-08-28 04:06:32 +01:00
Dima Gerasimov
118c2d4484 ruff: enable UP ruleset for detecting python deprecations 2024-08-28 04:06:32 +01:00
Dima Gerasimov
7bfce72b7c core: cleanup/sort imports according to ruff check --select I 2024-08-16 11:38:13 +01:00
Dima Gerasimov
7023088d13 core.common: deprecate outdated LazyLogger alias 2024-08-16 10:22:29 +01:00
Dima Gerasimov
c45c51af22 core.common: move stats-related stuff to my.core.stats and add more thorough tests/docs
deprecate core.common.stat and core.common.Stats with backwards compatibility
2024-08-16 10:22:29 +01:00
Dima Gerasimov
32aa87b3ec dcotor: make compileall check a bit more defensive 2023-10-27 02:38:22 +01:00
Sean Breckenridge
2a46341ce2 my.core.logging: compatibility with HPI_LOGS
re-adds a removed check for HPI_LOGS, add some docs

fix the checks for browserexport/takeout logs to
use the computed level from my.core.logging
2023-09-07 02:36:26 +01:00
Sean Breckenridge
fcaa7c1561 core/cli: allow user to bypass PEP 668
when installing dependencies with 'hpi module install',
this now lets a user pass '--break-system-packages' (or '-B'),
which passes the same option down to pip, to allow the user
to bypass PEP 668 and install packages that could possibly
conflict with system packages.
2023-08-10 01:41:43 +01:00
Dima Gerasimov
c34656e8fb general: update mypy config, seems that logs of type: ignore aren't necessary anymore 2023-05-16 01:18:30 +01:00
seanbreckenridge
7a32302d66
query: add --warn-exceptions, dateparser, docs (#290)
* query: add --warn-exceptions, dateparser, docs

added --warn-exceptions (like --raise-exceptions/--drop-exceptions, but
lets you pass a warn_func if you want to customize how the exceptions are
handled. By default this creates a logger in main and logs the exception

added dateparser as a fallback if its installed (it's not a strong dependency, but
I mentioned in the docs that it's useful for parsing dates/times)

added docs for query, and a few examples

--output gpx respects the --{drop,warn,raise}--exceptions flags, have
an example of that in the docs as well
2023-04-18 00:15:35 +01:00
seanbreckenridge
40de162fab
cli: add option to output locations to gpx files (#286)
* cli: add option to output locations to gpx files
2023-04-15 00:31:11 +01:00
Kian-Meng Ang
d2ef23fcb4 docs: fix typos
found via `codespell -L copie,datas,pres,fo,tooks,noo,ue,ket,frop`
2023-03-27 03:02:35 +01:00
Sean Breckenridge
db2cd00bed try removing parallel on mac to prevent CI failure 2023-02-28 20:55:12 +00:00
Dima Gerasimov
bb5ad2b6ac core: make hpi install more defensive, just warn on no requirements
this is useful for backwards compatibility if modules remove their requirements
2023-02-07 01:57:00 +00:00
Sean Breckenridge
54e6fe6ab5 ci: try disabling parallel pip installs on windows 2022-12-17 21:07:30 +00:00
Dima Gerasimov
119b295d71 core: allow legacy modules to be used in 'hpi module install' for backwards compatibility
but show warning

kinda hacky, but hopefully we will simplify it further when we have more such legacy modules
2022-06-07 22:59:08 +01:00
Dima Gerasimov
f0397b00ff core/main: experimental --parallel flag for hpi module install 2022-06-06 09:49:15 +01:00
Dima Gerasimov
5f0231c5ee core/main: allow passing multiple packages to 'module install'/'module requires' subcommands 2022-06-06 09:49:15 +01:00
Dima Gerasimov
016f28250b general: initial flake8 checks (for now manual)
fix fairly uncontroversial stuff in my.core like
- line spacing, which isn't too annoying (e.g. unlike many inline whitespace checks that break vertical formatting)
- unused imports/variables
- too broad except
2022-06-05 22:28:38 +01:00
Dima Gerasimov
186f561018 core: some cleanup for core/init and doctor; fix issue with compileall 2022-06-02 14:21:21 +01:00
seanbreckenridge
0ce44bf0d1
doctor: better quick option propogation for stats (#239)
doctor: better quick option propogation for stats

* use contextmanager for quick stats instead of editing global state
  directly
* send quick to lots of stat related functions, so they
could possibly be used without doctor, if someone wanted to
* if a stats function has a 'quick' kwarg, send the value
there as well
* add an option to sort locations in my.time.tz.via_location
2022-05-02 00:13:05 +01:00
Sean Breckenridge
16c777b45a my.config: catch possible nested config errors 2022-04-10 16:51:15 +01:00
Sean Breckenridge
6185942f78 core/cli: autocomplete module names 2022-02-23 12:15:00 -01:00
Sean Breckenridge
a791b25650 core/cli: add --debug flag, add HPI_LOGS to docs 2022-02-11 09:31:10 +00:00
seanbreckenridge
7bf316eb9a
core/source: use import error (#211)
core/source: use import error

uses the more broad ImportError
instead of ModuleNotFoundError

reasoning being if some submodule
(the one I'm configuring currently is
my.twitter.twint) doesn't have additional
imports from another parser/DAL, but it
still has a config block, the user would
have to create a stub-config block in their
config to use the all.py file
2022-02-10 08:57:52 +00:00
Dima Gerasimov
0e891a267f doctor: suggest config documentation in case of ImportError from config
doesn't help in all cases but perhaps helpful anyway

relevant: https://github.com/karlicoss/HPI/issues/109
2022-02-02 23:46:46 +00:00
Sean Breckenridge
03dd1271f4 cli/query: add short flags, stream affects pprint
adds some short flags as CLI flags for convenience
the --stream flag previously only affected json, but
I can imagine '-o pprint -s -l 5' to print the first
5 items from some function could be useful as well
2022-01-27 08:50:57 +00:00
Sean Breckenridge
3f4fb64d56
core: drop py36 support, update docs for reddit (#193)
* docs: update references to my.reddit
* ci: remove 3.6, add 3.9
2022-01-27 08:26:15 +00:00
Sean Breckenridge
4a04c09f31 docs: fix copy-paste errors/spelling mistakes 2021-07-10 10:56:23 +01:00
Sean Breckenridge
43cfb2742f
cli/query: bugfix, convert output to list (#170)
* cli/query: bugfix, convert output to list to keep it backwards compatible
2021-04-28 21:19:49 +01:00
Sean Breckenridge
fa7474c087 cli/query: add --stream flag
allows you to do something like

hpi query --stream my.reddit.comments
to stream the JSON objects one per line, makes
it nicer to pipe into 'jq'/'fzf' instead
of having to process the giant list
at the end
2021-04-28 18:23:16 +01:00
Sean Breckenridge
0278f2b68d cli/query: improve fallback behaviour/error msg 2021-04-24 06:15:59 +01:00
Sean Breckenridge
4b4cb7cb5b cli/query: bugfix where datetime was ignored 2021-04-19 20:21:17 +01:00
Sean Breckenridge
277f0e3988
cli/query: interactive fallback, improve guess_stats (#163) 2021-04-19 18:57:42 +01:00
Sean Breckenridge
fb49243005
core: add hpi query command (#157)
- restructure query code for cli, some test fixes
- initial query_range implementation

    refactored functions in query some more
    to allow re-use in range_range, select()
    pretty much just calls out to a bunch
    of handlers now
2021-04-06 17:19:58 +01:00
Sean Breckenridge
349ab78fca
core/cli: switch to using click library #155
everything is backwards-compatible with the previous
interface, the only minor changes were to the doctor cmd
which can now accept more than one item to run,
and the --skip-config-check to skip the config_ok
check if the user specifies to

added a test using click.testing.CliRunner (tests
the CLI in an isolated environment), though
additional tests which aren't testing the CLI
itself (parsing arguments or decorator behaviour)
can just call the functions themselves, as they
no longer accept a argparser.Namespace and instead
accept the direct arguments
2021-04-04 10:06:59 +01:00
Dima Gerasimov
8d6f691824 core: feature: guess module stats from typing annotations 2021-03-15 10:27:18 +00:00
Dima Gerasimov
0534c5c57d cli: add 'hpi module install' and 'hpi module requires'
ci: use hpi module install; remove explicit module links

relevant:

- https://github.com/karlicoss/HPI/issues/12
- https://github.com/karlicoss/HPI/issues/79
2021-02-18 02:04:40 +00:00
Dima Gerasimov
4ad4f34cda core: improve mypy coverage 2021-02-18 02:04:40 +00:00
Dima Gerasimov
df9a7f7390 core.pandas: add check for 'error' column + add empty one by default 2021-02-14 16:20:38 +00:00
Dima Gerasimov
29ad315578 core/cli: some enhacements for frendlier errors/default config
see https://github.com/karlicoss/HPI/issues/110
2020-11-23 20:46:26 +01:00
Dima Gerasimov
ed4b6c409f doctor & stat improvements
- doctor: log when there is no stats() function and suggest to use it
- check that stats() result isn't None
- warn when there is not data in the iterable
2020-11-01 02:02:43 +01:00
Dima Gerasimov
2619af7ae7 doctor: warn if the default config is used 2020-11-01 02:02:43 +01:00
Dima Gerasimov
209cffb476 doctor: print import order 2020-10-09 23:22:00 +02:00
Dima Gerasimov
bdb5dcd221 my.calendar.holidays: cleanup + ci/stats + split off private data handling to https://github.com/karlicoss/hpi-personal-overlay 2020-10-09 23:22:00 +02:00
Dima Gerasimov
ed25fc2eeb cli: tabulate warnings for cleaner visual output; add --quick flag for doctor 2020-09-30 21:54:09 +02:00