fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1120 commits 3 branches 41 tags 2.5 MiB
Commit graph

35 commits

Author SHA1 Message Date
Dima Gerasimov
72cc8ff3ac ruff: enable B warnings (mainly suppressed exceptions and unused variables) 2024-08-28 04:06:32 +01:00
Dima Gerasimov
d244c7cc4e ruff: enable and fix C4 ruleset 2024-08-28 04:06:32 +01:00
Dima Gerasimov
a5c04e789a twitter.archive: deduplicate results via json.dumps 
this speeds up processing quite a bit, from 40s to 20s for me, plus removes tons of identical outputs

interesting enough, using raw object without json.dumps as key brings unique_everseen to crawl...
 2023-10-24 01:54:30 +01:00
karlicoss
8c2d1c9463 general: use less explicit kompress boilerplate in modules 
now get_files/kompress library can handle it transparently
 2023-10-20 21:13:59 +01:00
Dima Gerasimov
dff31455f1 general: switch to make_logger in a few modules, use a bit more consistent logging, rely on default INFO level 2023-06-21 18:42:15 +01:00
Dima Gerasimov
fe88380499 general: switch to using native 3.8 versions for cached_property/Literal/Protocol instead of compat 2023-05-16 01:18:30 +01:00
Dima Gerasimov
c34656e8fb general: update mypy config, seems that logs of type: ignore aren't necessary anymore 2023-05-16 01:18:30 +01:00
Dima Gerasimov
c63177e186 general/ci: clean up mypy-misc pipeline, only exclude specific files instead 
marked some module configs which aren't really ready for public use as type: ignore
 2023-02-21 00:20:58 +00:00
Dima Gerasimov
5f1d41fa52 my.twitter.archive: fix for newer format (tweets filename changed to tweets.js) 2022-10-19 00:06:23 +01:00
Dima Gerasimov
ca91be8154 twitter.archive: fix legacy config detection 
apparently .name contains the parent module so previously it was throwing the exception instead
 2022-10-19 00:06:23 +01:00
Dima Gerasimov
4e59a65f9a core/general: move cached_property into compat, use standard implementation from python3.8 2022-05-31 14:08:50 +01:00
Dima Gerasimov
711157e0f5 my.twitter.archive: switch to zippath, add config section, better mypy coverage 2022-05-31 14:08:50 +01:00
Dima Gerasimov
bb6201bf2d my.twitter.archive: expand entities in tweet text 2022-05-31 01:28:11 +01:00
Dima Gerasimov
1e2fc3bec7 twitter.archive: unescape stuff like &lt/&gt 2022-05-31 01:28:11 +01:00
Dima Gerasimov
de7972be05 twitter: add permalink to Talon objects; extract shared method 2022-05-31 01:28:11 +01:00
Sean Breckenridge
62832a6756 twitter/archive: set default logger to warning 2022-02-09 23:18:24 +00:00
Sean Breckenridge
b6fa26b899 twitter/archive: update deprecated imports 2022-02-09 23:18:24 +00:00
Dima Gerasimov
b9852f45cf twitter: use import_source and proper merging for tweets from different sources 
+ use proper datetime_aware for created_at
 2022-02-08 20:45:10 +00:00
Dima Gerasimov
afdf9d4334 twitter: initial talon module, processing data from Talon android app 2022-02-08 20:45:10 +00:00
Sean Breckenridge
5ecd4b4810 cleanup; remove unused imports 2021-04-02 08:38:06 +01:00
Dima Gerasimov
15789a4149 kyhton.kompress: move to core (with a fallback, used in promnesia) 2020-10-29 03:13:18 +01:00
Dima Gerasimov
fbaa8e0b44 core: add warnings helper to highlight warnings so they are more visible in the output 2020-09-27 17:47:30 +02:00
Dima Gerasimov
1cc4eb5d8d core: add helper for computing stats; use it in modules 2020-06-04 22:19:34 +01:00
Dima Gerasimov
a267aeec5b github: add config templates + docs 
- ghexport: use export_path (export_dir is still supported)
 2020-06-01 23:33:34 +01:00
Dima Gerasimov
f5267d05d7 my.twitter.archive: rename config (preserving bckwd compatibility for now) 2020-05-24 13:06:52 +01:00
Dima Gerasimov
b99b2f3cfa core: add warning when get_files returns no files, my.twitter.archive: make more defensive in case of no archives 2020-05-24 12:51:23 +01:00
Dima Gerasimov
b7662378a2 docs: minor updates 2020-05-22 19:38:14 +01:00
Dima Gerasimov
03773a7b2c twitter module: prettify top level twitter.all 2020-05-22 19:00:02 +01:00
Dima Gerasimov
e92ca215e3 Adapt takeout and twitter configs to the new pattern 
Works fairly well so far?
 2020-05-10 15:56:57 +01:00
Dima Gerasimov
8b8a85e8c3 kompress.kopen improvements 
- tests
- uniform handling for bytes/str, always return utf8 str by default
 2020-05-04 08:37:36 +01:00
Dima Gerasimov
51ae8601b4 Update docstrings and add links 2020-04-26 16:50:06 +01:00
Dima Gerasimov
96a850faf9 remove unnecessary methods from twitter provider 2020-04-20 08:38:01 +01:00
Dima Gerasimov
81986b0624 support likes from twint 2020-04-14 23:01:44 +01:00
Dima Gerasimov
30b6918a8d unified view for twitter data 2020-04-14 22:05:47 +01:00
Dima Gerasimov
56b6ab9aaf move twitter stuff to twitter subdir 2020-04-14 21:38:21 +01:00
Renamed from my/twitter.py (Browse further)