Dima Gerasimov
8496d131e7
general: migrate modules to use 3.9 features
2024-10-19 23:41:22 +01:00
Dima Gerasimov
6a18f47c37
my.github.gdpr/my.zulip.organization: use kompress support for tar.gz if it's available
...
otherwise fall back onto unpacking into tmp dir via my.core.structure
2024-09-18 23:35:03 +01:00
Dima Gerasimov
d0df8e8f2d
ruff: enable PLR rules and fix bug in my.github.gdpr._is_bot
2024-08-28 04:06:32 +01:00
Dima Gerasimov
d244c7cc4e
ruff: enable and fix C4 ruleset
2024-08-28 04:06:32 +01:00
karlicoss
f28f68b14b
general: enhancle logging for various modules
2023-10-29 22:32:07 +00:00
Dima Gerasimov
642e3b14d5
my.github.gdpr: some minor enhancements
...
- better error context
- handle some unknown files
- handle user=None in some cases
- cleanup imports
2023-08-24 23:46:23 +01:00
Dima Gerasimov
dff31455f1
general: switch to make_logger in a few modules, use a bit more consistent logging, rely on default INFO level
2023-06-21 18:42:15 +01:00
Dima Gerasimov
c34656e8fb
general: update mypy config, seems that logs of type: ignore aren't necessary anymore
2023-05-16 01:18:30 +01:00
Dima Gerasimov
049820c827
my.github.gdpr: support uncompressed .tar.gz files
...
related to https://github.com/karlicoss/HPI/issues/20
2022-05-31 22:16:05 +01:00
Dima Gerasimov
1b4ca6ad1b
github.gdpr: prepare for using .tag.gz
2022-05-31 22:16:05 +01:00
Maxim Efremov
80c5be7293
Adding bots file type to reduce parsing issues
2022-05-02 08:53:46 +01:00
Dima Gerasimov
5ef2775265
my.github: some work in progress on generating consistent ids
...
sadly it seems that there are at several issues:
- gdpr has less detailed data so it's hard to generate a proper ID at times
- sometimes there is a small (1s?) discrepancy between created_at between same event in GDPR an API
- some API events can have duplicate payload, but different id, which violates uniqueness
2021-04-02 20:09:53 +01:00
Sean Breckenridge
5ecd4b4810
cleanup; remove unused imports
2021-04-02 08:38:06 +01:00
Sean Breckenridge
02a9fb5e8f
github.gdpr: parse project files
...
also fixed a typo in commit_comments
2021-03-15 12:40:22 +00:00
Dima Gerasimov
1cc4eb5d8d
core: add helper for computing stats; use it in modules
2020-06-04 22:19:34 +01:00
Dima Gerasimov
a267aeec5b
github: add config templates + docs
...
- ghexport: use export_path (export_dir is still supported)
2020-06-01 23:33:34 +01:00
Dima Gerasimov
ca39187c63
github: DEPRECATE my.coding.github
...
Instead my.github.all should be used (still backward compatible)
The reasons are
a) I don't feel that grouping (i.e. my.coding.*) makes much sense
b) using .all pattern (same way as twitter) allows for more composable and cleaner separation of GDPR and API data
2020-06-01 22:49:31 +01:00
