fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1155 commits 3 branches 41 tags 2.5 MiB
Commit graph

104 commits

Author SHA1 Message Date
Dima Gerasimov
8496d131e7 general: migrate modules to use 3.9 features 2024-10-19 23:41:22 +01:00
karlicoss
d3f9a8e8b6
core: migrate code to benefit from 3.9 stuff (#401) 
for now keeping ruff on 3.8 target version, need to sort out modules as well
 2024-10-19 20:55:09 +01:00
Dima Gerasimov
d58453410c ruff: process remaining existing checks and suppress the annoying ones 2024-08-28 04:06:32 +01:00
Dima Gerasimov
72cc8ff3ac ruff: enable B warnings (mainly suppressed exceptions and unused variables) 2024-08-28 04:06:32 +01:00
Dima Gerasimov
664c40e3e8 ruff: enable FBT rules to detect boolean arguments use without kwargs 2024-08-28 04:06:32 +01:00
Dima Gerasimov
d244c7cc4e ruff: enable and fix C4 ruleset 2024-08-28 04:06:32 +01:00
Dima Gerasimov
c08ddbc781 general: small updates for typing while trying out pyright 2024-08-28 04:06:32 +01:00
karlicoss
5ec357915b core.common: add test for classproperty 2024-08-17 13:05:56 +01:00
karlicoss
245ad22057 core.common: bring back asdict backwards compat -- was used in orger 2024-08-17 13:05:56 +01:00
Dima Gerasimov
7bfce72b7c core: cleanup/sort imports according to ruff check --select I 2024-08-16 11:38:13 +01:00
Dima Gerasimov
7023088d13 core.common: deprecate outdated LazyLogger alias 2024-08-16 10:22:29 +01:00
Dima Gerasimov
614c929f95 core.common: move Json, datetime_aware, datetime_naive, is_namedtuple, asdict to my.core.types 2024-08-16 10:22:29 +01:00
Dima Gerasimov
7f8a502310 core.common: move assert_subpackage to my.core.internal 2024-08-16 10:22:29 +01:00
Dima Gerasimov
88f3c17c27 core.common: move mime-related stuff to my.core.mime 
no backward compat, unlikely it was used by anyone else
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
c45c51af22 core.common: move stats-related stuff to my.core.stats and add more thorough tests/docs 
deprecate core.common.stat and core.common.Stats with backwards compatibility
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
18529257e7 core.common: move DummyExecutor to core.common.utils.concurrent 
without backwards compat, unlikely it's been used by anyone
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
bcc4c15304 core: cleanup my.core.common.unique_everseen 
- move to my.core.utils.itertools
- more robust check for hashable types -- now checks in runtime (since the one based on types purely isn't necessarily sound)
- add more testing
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
06084a8787 my.core.common: move warn_if_empty to my.core.utils.itertools, cleanup and add more tests 2024-08-16 10:22:29 +01:00
Dima Gerasimov
770dba5506 core.common: move away import related stuff to my.core.utils.imports 
moving without backward compatibility, since it's extremely unlikely they are used for any external modules

in fact, unclear if these methods still have much value at all, but keeping for now just in case
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
66c08a6c80 core.common: move listify to core.utils.itertools, use better typing annotations for it 
also some minor refactoring of my.rss
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
c64d7f5b67 core: cleanup itertool style helpers 
- deprecate group_by_key, should use itertool.bucket instead
- move make_dict and ensure_unique to my.core.utils.itertools
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
973c4205df core: cleanup deprecations, exclude from type checking and show runtime warnings 
among affected things:

- core.common.assert_never
- core.common.cproperty
- core.common.isoparse
- core.common.mcachew
- core.common.the
- core.common.tzdatetime
- core.compat.sqlite_backup
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
a7439c7846 general: move assert_never to my.core.compat as it's in stdlib from 3.11 
rely on typing-extensions for fallback

introducing typing-extensions dependency without fallback, should be ok since it's in the top 10 of popular packages
 2024-08-16 10:22:29 +01:00
Dima Gerasimov
069264ce52 core.common: get rid of deprecated utcfromtimestamp 2024-08-10 17:46:30 +01:00
Dima Gerasimov
0e6dd32afe ci: minor fixes after mypy update 2024-08-03 16:18:32 +01:00
karlicoss
51209c547e my.twitter.android: refactor into a proper module 
for now only extracting bookmarks, will use it for some time and see how it goes
 2023-12-24 00:49:07 +00:00
Dima Gerasimov
a843407e40 core/compat: move fromisoformat to .core.compat module 2023-11-19 23:45:08 +00:00
karlicoss
71cb66df5f core: add helper for more_iterable to check that all types involved are hashable 
Otherwise unique_everseen performance may degrade to quadratic rather than linear

For now hidden behind HPI_CHECK_UNIQUE_EVERSEEN flag

also switch some modules to use it
 2023-10-31 01:02:17 +00:00
karlicoss
70bf51a125 core/stats: exclude contextmanagers from guess_stats 2023-10-28 00:08:32 +01:00
karlicoss
86ea605aec core/stats: enable processing input files, report first and last filename 
can be useful for quick investigation/testing setup
 2023-10-22 00:47:36 +01:00
karlicoss
c335c0c9d8 core/stats: report datetime of first item in addition to last 
quite useful for quickly determining time span of a data source
 2023-10-22 00:47:36 +01:00
karlicoss
a60d69fb30 core/stats: get rid of duplicated keys for 'auto stats' 
previously:
```
{'iter_data': {'iter_data': {'count': 9, 'last': datetime.datetime(2020, 1, 3, 1, 1, 1)}}}
```

after
```
{'iter_data': {'count': 9, 'last': datetime.datetime(2020, 1, 3, 1, 1, 1)}}
```
 2023-10-22 00:47:36 +01:00
karlicoss
c63e80ce94 core: more consistent handling of zip archives in get_files + tests 2023-10-20 21:13:59 +01:00
karlicoss
fe26efaea8 core/kompress: move vendorized to _deprecated, use kompress library directly 2023-10-12 23:47:05 +01:00
karlicoss
68289c1be3 general: fix ignores after mypy version update 2023-10-12 23:47:05 +01:00
Dima Gerasimov
5fe21240b4 core: move mcachew into my.core.cachew; use better typing annotations (copied from cachew) 2023-06-08 01:29:49 +01:00
Dima Gerasimov
fe88380499 general: switch to using native 3.8 versions for cached_property/Literal/Protocol instead of compat 2023-05-16 01:18:30 +01:00
Dima Gerasimov
c34656e8fb general: update mypy config, seems that logs of type: ignore aren't necessary anymore 2023-05-16 01:18:30 +01:00
Dima Gerasimov
d464b1e607 core: implement more methods for ZipPath and better support for get_files 2023-04-03 22:58:54 +01:00
Kian-Meng Ang
d2ef23fcb4 docs: fix typos 
found via `codespell -L copie,datas,pres,fo,tooks,noo,ue,ket,frop`
 2023-03-27 03:02:35 +01:00
Dima Gerasimov
7098d6831f fix mypy in _identity 
seems easier to just ignore considering it's "internal" function

also a couple of tests to make sure it infers types correctly
 2022-10-19 00:06:23 +01:00
Dima Gerasimov
016f28250b general: initial flake8 checks (for now manual) 
fix fairly uncontroversial stuff in my.core like
- line spacing, which isn't too annoying (e.g. unlike many inline whitespace checks that break vertical formatting)
- unused imports/variables
- too broad except
 2022-06-05 22:28:38 +01:00
Dima Gerasimov
7a1b7b1554 core/general: add assert_never + typing annotations for dataset 2022-06-03 23:49:27 +01:00
Dima Gerasimov
4e59a65f9a core/general: move cached_property into compat, use standard implementation from python3.8 2022-05-31 14:08:50 +01:00
seanbreckenridge
0ce44bf0d1
doctor: better quick option propogation for stats (#239) 
doctor: better quick option propogation for stats

* use contextmanager for quick stats instead of editing global state
  directly
* send quick to lots of stat related functions, so they
could possibly be used without doctor, if someone wanted to
* if a stats function has a 'quick' kwarg, send the value
there as well
* add an option to sort locations in my.time.tz.via_location
 2022-05-02 00:13:05 +01:00
Dima Gerasimov
73c9e46c4c core: better support for compressed stuff, add .tar.gz 2022-01-27 22:58:33 +00:00
Sean Breckenridge
7493770d4d core: remove vendorized py37 isoformat code 2022-01-27 19:25:42 +00:00
Sean Breckenridge
3f4fb64d56
core: drop py36 support, update docs for reddit (#193) 
* docs: update references to my.reddit
* ci: remove 3.6, add 3.9
 2022-01-27 08:26:15 +00:00
Dima Gerasimov
dd928964e6 general: fix mypy errors after mypy and pytz stubs updates 
see 968fd6d01d/stubs/pytz/pytz/tzinfo.pyi (L6)
it says all concrete instances should not be None
 2021-12-19 18:53:29 +00:00
Sean Breckenridge
821bc08a23
core/structure: help locate/extract gdpr exports (#175) 
* core/structure: help locate/extract gdpr exports

* ci: add install-types to install stub packages
 2021-07-08 00:44:55 +01:00