fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
956 commits 3 branches 41 tags 2.5 MiB
Commit graph

956 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dima Gerasimov
8288032b1c my.telegram.telegram_backup: support optional extra_where and optional media info extraction for Promnesia 2023-03-27 03:27:13 +01:00
Dima Gerasimov
74710b339a telegram_backup: order messages by date and users/chats by id for determinism 2023-03-27 03:27:13 +01:00
Kian-Meng Ang
d2ef23fcb4 docs: fix typos 
found via `codespell -L copie,datas,pres,fo,tooks,noo,ue,ket,frop`
 2023-03-27 03:02:35 +01:00
Dima Gerasimov
919c84fb5a my.instagram: better unification of like messages/reactions 2023-03-27 02:16:17 +01:00
Dima Gerasimov
9aadbb504b my.instagram.android: properly extract our own user 2023-03-27 02:16:17 +01:00
Dima Gerasimov
8f7d14e7c6 my.instagram: somewhat mad merging mechanism to correlate gdpr and android exports 2023-03-27 02:16:17 +01:00
Dima Gerasimov
e7be680841 my.instagram.gdpr: handle missing message content defensively 2023-03-27 02:16:17 +01:00
Dima Gerasimov
347cd1ef77 my.fbmessenger: add Sender protocol for consistency 2023-03-17 00:33:22 +00:00
Dima Gerasimov
58d2e25a42 ci: suppress some mypy issues after upgrade 2023-03-17 00:33:22 +00:00
Dima Gerasimov
bef832cbff my.fbmessenger.export: remove legacy dump_chat_history code 2023-03-17 00:33:22 +00:00
Dima Gerasimov
0a05b27266 my.fbmessenger.android: set timezone to utc 2023-03-17 00:33:22 +00:00
Dima Gerasimov
457797bdfb my.bumble.android: better handling for missing conversation id in database 2023-03-17 00:33:22 +00:00
Dima Gerasimov
9db5f318fb my.twitter.twint: use dict row factory instead of sqlite Row 
otherwise it's not json serializable
 2023-03-17 00:33:22 +00:00
seanbreckenridge
79eeab2128
cli completion doc updates, hide legacy import warning (#279) 
* core/cli: hide warnings when autocompleting

* link to completion in setup/troubleshooting
* update completion docs to make source path clear
 2023-03-06 21:36:36 +00:00
seanbreckenridge
9d231a8ea9
google_takeout: add semantic location history (#278) 
* google_takeout: add semantic location history
 2023-03-04 18:36:10 +00:00
Dima Gerasimov
a4c713664e core.logging: sync logging helper with Promnesia, adds more goodies 
- print exception traceback by default when using logger.exception
- COLLAPSE_DEBUG_LOGS env variable
 2023-03-03 21:14:11 +00:00
Dima Gerasimov
bee17d932b fbmessenger.android: use Optional name, best to leave for the consumer to decide how to behave when it's unavailable 
e.g. using  <NAME UNAVAILABLE> was causing issues when used as zulip contact name
 2023-03-03 21:14:11 +00:00
Dima Gerasimov
4dfc4029c3 core.kompress: proper support for read_text/read_bytes against zstd/xz archives 2023-03-03 21:14:11 +00:00
Dima Gerasimov
b94904f5ee core.kompress: support .zst extension, seems more conventional than .zstd 2023-03-03 21:14:11 +00:00
Sean Breckenridge
db2cd00bed try removing parallel on mac to prevent CI failure 2023-02-28 20:55:12 +00:00
Sean Breckenridge
a70118645b my.ip.common: remove REQUIRES 
no reason to have it there since its
__NOT_HPI_MODULE__, so is not discoverable anyways
 2023-02-28 20:55:12 +00:00
Sean Breckenridge
f36bc6144b tox: use my.ip.all, sort hpi installs 2023-02-28 20:55:12 +00:00
Sean Breckenridge
435cb020f9 add example for denylist, update ci 2023-02-28 20:55:12 +00:00
seanbreckenridge
98b086f746
location fallback (#263) 
see https://github.com/karlicoss/HPI/issues/262

* move home to fallback/via_home.py
* move via_ip to fallback
* add fallback model
* add stub via_ip file
* add fallback_locations for via_ip
* use protocol for locations
* estimate_from helper, via_home estimator, all.py
* via_home: add accuracy, cache history
* add datasources to gpslogger/google_takeout
* tz/via_location.py: update import to fallback
* denylist docs/installation instructions
* tz.via_location: let user customize cachew refresh time
* add via_ip.estimate_location using binary search
* use estimate_location in via_home.get_location
* tests: add gpslogger to location config stub
* tests: install tz related libs in test env
* tz: add regression test for broken windows dates

* vendorize bisect_left from python src
doesnt have a 'key' parameter till python3.10
 2023-02-28 04:30:06 +00:00
Dima Gerasimov
6dc5e7575f vk_messages_backup: add unique_everseen to prevent duplicate messages 2023-02-28 03:55:44 +00:00
Dima Gerasimov
a7099e2efc vk_messages_backup: more correct handling of group chats & better chat ids 2023-02-28 03:55:44 +00:00
Dima Gerasimov
02c98143d5 vk_messages_backup: better structure & exract richer information 2023-02-28 03:55:44 +00:00
Dima Gerasimov
130c273513 my.telegram.telegram_backup enhancements 
- add chat handle
- add permalink
- more precise types
 2023-02-21 02:44:18 +00:00
Dima Gerasimov
07e7c62d02 general/ci: mypy check tests 2023-02-21 00:20:58 +00:00
Dima Gerasimov
c63177e186 general/ci: clean up mypy-misc pipeline, only exclude specific files instead 
marked some module configs which aren't really ready for public use as type: ignore
 2023-02-21 00:20:58 +00:00
Dima Gerasimov
eff9c02886 my.fbmessenger.android: add optional facebook_id 2023-02-20 20:14:15 +00:00
Dima Gerasimov
af874d2d75 my.fbmessenger.android: minor refactoring, comments & error handling 2023-02-20 20:14:15 +00:00
Dima Gerasimov
6493859ba5 my.telegram: initial module from telegram_backup 2023-02-19 01:20:38 +00:00
Dima Gerasimov
6594ad24dc my.tinder.android: speedup unique_everseen by adding unsafe_hash 2023-02-19 01:20:38 +00:00
Dima Gerasimov
458633ea96 my.tinder.android: add a bit of logging 2023-02-19 01:20:38 +00:00
Dima Gerasimov
0e884fe166 core/modules: switch away from using override_config to tmp_config in some tests & faka data generators 2023-02-09 02:35:09 +00:00
Dima Gerasimov
5ac5636e7f core: better support for ad-hoc configs 
properly reload/unload the relevant modules so hopefully no more weird hacks should be required

relevant
- https://github.com/karlicoss/promnesia/issues/340
- https://github.com/karlicoss/HPI/issues/46
 2023-02-09 02:35:09 +00:00
Dima Gerasimov
fb0c1289f0 my.fbmessenger.export: use context manager to properly close sqlite connection 2023-02-08 02:18:00 +00:00
Dima Gerasimov
bb5ad2b6ac core: make hpi install more defensive, just warn on no requirements 
this is useful for backwards compatibility if modules remove their requirements
 2023-02-07 01:57:00 +00:00
Dima Gerasimov
5c82d0faa9 switch from using dataset to raw sqlite3 module 
dataset is kinda unmaintaned and currently broken due to sqlalchemy 2.0 changes

resolves https://github.com/karlicoss/HPI/issues/264
 2023-02-07 01:57:00 +00:00
Dima Gerasimov
9c432027b5 instagram.android: fix missing id 2023-02-07 01:57:00 +00:00
karlicoss
11b6e51c90 ci: fix tox config 
seems that after version 4.0 it's necessary to specify environments to run
previosly it was picking them up automatically
 2023-01-31 00:31:54 +00:00
Sean Breckenridge
54e6fe6ab5 ci: try disabling parallel pip installs on windows 2022-12-17 21:07:30 +00:00
Sean Breckenridge
ad52e131a0 google.takeout.parser: recreate cache on upgrade 
https://github.com/seanbreckenridge/google_takeout_parser/pull/37
 2022-12-17 21:07:30 +00:00
Sean Breckenridge
716a2c82ba core/serialize: serialize stdlib Decimal class 2022-10-19 00:07:30 +01:00
Dima Gerasimov
7098d6831f fix mypy in _identity 
seems easier to just ignore considering it's "internal" function

also a couple of tests to make sure it infers types correctly
 2022-10-19 00:06:23 +01:00
Dima Gerasimov
5f1d41fa52 my.twitter.archive: fix for newer format (tweets filename changed to tweets.js) 2022-10-19 00:06:23 +01:00
Dima Gerasimov
ca91be8154 twitter.archive: fix legacy config detection 
apparently .name contains the parent module so previously it was throwing the exception instead
 2022-10-19 00:06:23 +01:00
Dima Gerasimov
c8cf0272f9 instagram.gdpr: use new path to personal information 2022-10-19 00:06:23 +01:00
Sean Breckenridge
7925ec81b6 docs: browser - fix examples for config 2022-08-29 00:03:32 +01:00