fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1148 commits 3 branches 41 tags 2.5 MiB
Commit graph

25 commits

Author SHA1 Message Date
purarue
7ab6f0d5cb chore: update urls 2024-10-30 20:12:00 +00:00
Dima Gerasimov
8496d131e7 general: migrate modules to use 3.9 features 2024-10-19 23:41:22 +01:00
Dima Gerasimov
72cc8ff3ac ruff: enable B warnings (mainly suppressed exceptions and unused variables) 2024-08-28 04:06:32 +01:00
Dima Gerasimov
c45c51af22 core.common: move stats-related stuff to my.core.stats and add more thorough tests/docs 
deprecate core.common.stat and core.common.Stats with backwards compatibility
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
973c4205df core: cleanup deprecations, exclude from type checking and show runtime warnings 
among affected things:

- core.common.assert_never
- core.common.cproperty
- core.common.isoparse
- core.common.mcachew
- core.common.the
- core.common.tzdatetime
- core.compat.sqlite_backup
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
4f7c9b4a71 core: move split compat/legacy modules into hpi_compat and compat 2023-10-28 20:06:54 +01:00
Dima Gerasimov
1f61e853c9 reddit.rexport: experiment with using optional cpu pool (used by all of HPI) 
Enabled by the env variable, specifying how many cores to dedicate, e.g.

HPI_CPU_POOL=4 hpi query ...
 2023-10-25 02:06:45 +01:00
Dima Gerasimov
9ffce1b696 reddit.rexport: add accessors for subreddits, multireddits and profile 2023-10-19 02:26:28 +01:00
Dima Gerasimov
28d2450a21 reddit.rexport: some cleanup, move get_events stuff into personal overlay 2023-10-19 02:26:28 +01:00
Dima Gerasimov
dff31455f1 general: switch to make_logger in a few modules, use a bit more consistent logging, rely on default INFO level 2023-06-21 18:42:15 +01:00
Dima Gerasimov
c12224af74 misc: replace uses of pytz.utc with timezone.utc where it makes sense 2023-06-09 03:31:13 +01:00
Dima Gerasimov
fe88380499 general: switch to using native 3.8 versions for cached_property/Literal/Protocol instead of compat 2023-05-16 01:18:30 +01:00
Dima Gerasimov
c34656e8fb general: update mypy config, seems that logs of type: ignore aren't necessary anymore 2023-05-16 01:18:30 +01:00
Kian-Meng Ang
d2ef23fcb4 docs: fix typos 
found via `codespell -L copie,datas,pres,fo,tooks,noo,ue,ket,frop`
 2023-03-27 03:02:35 +01:00
Dima Gerasimov
3faebdd629 core: add Protocol/TypedDict to compat 2022-06-02 14:21:21 +01:00
Dima Gerasimov
9461df6aa5 general: extract the hack to warn of legacy imports and fallback to core/legacy.py 
use it both in my.fbmessenger and my.reddit

if in the future any new modules need to be switched to namespace package structure with all.py it should make it easy to do

related:
- https://github.com/karlicoss/HPI/issues/12
- https://github.com/karlicoss/HPI/issues/89
- https://github.com/karlicoss/HPI/issues/102
 2022-06-01 23:27:34 +01:00
Dima Gerasimov
73e57b52d1 general: cleanup -- remove main and executable bit where it's not necessary 2022-05-31 22:16:05 +01:00
Sean Breckenridge
823668ca5c make reddit.rexport logs info by default 
can always be configured with HPI_LOGS
having this on debug makes hpi doctor
quite verbose
 2022-02-02 00:35:54 +00:00
Dima Gerasimov
be21606075 my.reddit: better handling for legacy reddit config 
prior to this change it would error with

    @dataclass
>   class pushshift_config(uconfig.pushshift):
E   AttributeError: type object 'test_config' has no attribute 'pushshift'
 2021-12-24 18:02:37 +00:00
Dima Gerasimov
5e9cc2a6a0 my.reddit: enable CI tests 2021-12-24 18:02:37 +00:00
Sean Breckenridge
074b8685d6 reddit: pass logger to cachew 
so that HPI_LOGS can be used to interact
with this module, to check if cachew
is working properly
 2021-12-19 18:25:50 +00:00
Sean Breckenridge
d006339ab4 reddit: fix spelling mistakes 2021-11-03 20:18:10 +00:00
Sean Breckenridge
d6c484f321 reddit: ensure rexport isnt pointing to repo 2021-10-31 21:47:10 +00:00
Sean Breckenridge
5d2eadbcc6
reddit: swap inheritance order for Protocol (#183) 2021-10-31 21:24:16 +00:00
Sean Breckenridge
8422c6e420
my.reddit: refactor into module that supports pushshift/gdpr (#179) 
* initial pushshift/rexport merge implementation, using id for merging
* smarter module deprecation warning using regex
* add `RedditBase` from promnesia
* `import_source` helper for gracefully handing mixin data sources
 2021-10-31 20:39:04 +00:00