Commit graph

22 commits

Author SHA1 Message Date
Dima Gerasimov
1215181af5 core: move stuff from tests/demo.py to my/core/tests/test_config.py
also clean all this up a bit
2024-08-25 20:49:56 +01:00
Dima Gerasimov
7bfce72b7c core: cleanup/sort imports according to ruff check --select I 2024-08-16 11:38:13 +01:00
Dima Gerasimov
7023088d13 core.common: deprecate outdated LazyLogger alias 2024-08-16 10:22:29 +01:00
Dima Gerasimov
614c929f95 core.common: move Json, datetime_aware, datetime_naive, is_namedtuple, asdict to my.core.types 2024-08-16 10:22:29 +01:00
Dima Gerasimov
2b0f92c883 my.core: deprecate Path/dataclass imports from my.core during type checking
runtime still works for backwards compatibility
2024-08-16 10:22:29 +01:00
Dima Gerasimov
c45c51af22 core.common: move stats-related stuff to my.core.stats and add more thorough tests/docs
deprecate core.common.stat and core.common.Stats with backwards compatibility
2024-08-16 10:22:29 +01:00
Dima Gerasimov
06084a8787 my.core.common: move warn_if_empty to my.core.utils.itertools, cleanup and add more tests 2024-08-16 10:22:29 +01:00
Dima Gerasimov
a7439c7846 general: move assert_never to my.core.compat as it's in stdlib from 3.11
rely on typing-extensions for fallback

introducing typing-extensions dependency without fallback, should be ok since it's in the top 10 of popular packages
2024-08-16 10:22:29 +01:00
karlicoss
68289c1be3 general: fix ignores after mypy version update 2023-10-12 23:47:05 +01:00
Dima Gerasimov
661714f1d9 core/logging: overhaul and many improvements -- mainly to deprecate abandoned logzero
- generally saner/cleaner logger initialization

  In particular now it doesn't override logging level specified by the user code prior to instantiating the logger.

  Also remove the `LazyLogger` hack, doesn't seem like it's necessary when the above is implemented.

- get rid of `logzero` which is archived and abandoned now, use `colorlog` for coloured logging formatter

- allow configuring log level via shell via `LOGGING_LEVEL_module_name=<level>`

  E.g. `LOGGING_LEVEL_rescuexport_dal=WARNING LOGGING_LEVEL_my_rescuetime=debug ./script.py`

- port `AddExceptionTraceback` from HPI/promnesia

- port `CollapseLogsHandler` from HPI/promnesia

  Also allow configuring from the shell, e.g. `LOGGING_COLLAPSE=<level>`

- add support for `enlighten` progress bar, so it can be shared between different projects

  See https://github.com/Rockhopper-Technologies/enlighten#readme

  This allows nice CLI progressbars, e.g. for parallel processing of different files from HPI:

    ghexport.dal[111]  29%|████████████████████████████████████████████████████████████████▏              |  29/100 [00:03<00:07, 10.03 files/s]
    rexport.dal[comments]  17%|████████████████████████████████████▋                                      | 115/682 [00:03<00:14, 39.15 files/s]
    my.instagram.android   0%|▎                                                                           |    3/2631 [00:02<34:50, 1.26 files/s]

  Currently off by default, and hidden behind an env variable (`ENLIGHTEN_ENABLE=true`)
2023-06-21 18:42:15 +01:00
Dima Gerasimov
ab7135d42f core: experimental import of my._init_hook to configure logging/warnings/env variables 2023-06-21 03:32:46 +01:00
Dima Gerasimov
016f28250b general: initial flake8 checks (for now manual)
fix fairly uncontroversial stuff in my.core like
- line spacing, which isn't too annoying (e.g. unlike many inline whitespace checks that break vertical formatting)
- unused imports/variables
- too broad except
2022-06-05 22:28:38 +01:00
Dima Gerasimov
7a1b7b1554 core/general: add assert_never + typing annotations for dataset 2022-06-03 23:49:27 +01:00
Dima Gerasimov
86497f9b13 new: basic arbtt module 2021-02-25 19:56:35 +00:00
Dima Gerasimov
63c825ab81 my.stackexchange: use GDPR data for votes 2020-12-11 07:02:16 +01:00
Dima Gerasimov
fa5e181cf8 core: minor helpers for error handling 2020-10-21 01:29:29 +02:00
Dima Gerasimov
06ee72bc30 core: more type annotations 2020-10-03 18:24:08 +02:00
Dima Gerasimov
fd41caa640 core: add __NOT_HPI_MODULE__ flag to mark utility files etc
(more of an intermediate solution perhaps)
2020-09-30 21:54:09 +02:00
Dima Gerasimov
1cc4eb5d8d core: add helper for computing stats; use it in modules 2020-06-04 22:19:34 +01:00
Dima Gerasimov
4b22d17188 core: add @warn_if_empty decorator 2020-05-25 00:25:33 +01:00
Dima Gerasimov
f3d5064ff2 polar: allow properly specifying polar_dir, with ~ as a default 2020-05-15 08:18:47 +01:00
Dima Gerasimov
6235e6ffae Make my.core a proper package (for brevity purposes) 2020-05-14 23:01:50 +01:00