fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1147 commits 3 branches 41 tags 2.5 MiB
Commit graph

1147 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dima Gerasimov
bef832cbff my.fbmessenger.export: remove legacy dump_chat_history code 2023-03-17 00:33:22 +00:00
Dima Gerasimov
0a05b27266 my.fbmessenger.android: set timezone to utc 2023-03-17 00:33:22 +00:00
Dima Gerasimov
457797bdfb my.bumble.android: better handling for missing conversation id in database 2023-03-17 00:33:22 +00:00
Dima Gerasimov
9db5f318fb my.twitter.twint: use dict row factory instead of sqlite Row 
otherwise it's not json serializable
 2023-03-17 00:33:22 +00:00
seanbreckenridge
79eeab2128
cli completion doc updates, hide legacy import warning (#279) 
* core/cli: hide warnings when autocompleting

* link to completion in setup/troubleshooting
* update completion docs to make source path clear
 2023-03-06 21:36:36 +00:00
seanbreckenridge
9d231a8ea9
google_takeout: add semantic location history (#278) 
* google_takeout: add semantic location history
 2023-03-04 18:36:10 +00:00
Dima Gerasimov
a4c713664e core.logging: sync logging helper with Promnesia, adds more goodies 
- print exception traceback by default when using logger.exception
- COLLAPSE_DEBUG_LOGS env variable
 2023-03-03 21:14:11 +00:00
Dima Gerasimov
bee17d932b fbmessenger.android: use Optional name, best to leave for the consumer to decide how to behave when it's unavailable 
e.g. using  <NAME UNAVAILABLE> was causing issues when used as zulip contact name
 2023-03-03 21:14:11 +00:00
Dima Gerasimov
4dfc4029c3 core.kompress: proper support for read_text/read_bytes against zstd/xz archives 2023-03-03 21:14:11 +00:00
Dima Gerasimov
b94904f5ee core.kompress: support .zst extension, seems more conventional than .zstd 2023-03-03 21:14:11 +00:00
Sean Breckenridge
db2cd00bed try removing parallel on mac to prevent CI failure 2023-02-28 20:55:12 +00:00
Sean Breckenridge
a70118645b my.ip.common: remove REQUIRES 
no reason to have it there since its
__NOT_HPI_MODULE__, so is not discoverable anyways
 2023-02-28 20:55:12 +00:00
Sean Breckenridge
f36bc6144b tox: use my.ip.all, sort hpi installs 2023-02-28 20:55:12 +00:00
Sean Breckenridge
435cb020f9 add example for denylist, update ci 2023-02-28 20:55:12 +00:00
seanbreckenridge
98b086f746
location fallback (#263) 
see https://github.com/karlicoss/HPI/issues/262

* move home to fallback/via_home.py
* move via_ip to fallback
* add fallback model
* add stub via_ip file
* add fallback_locations for via_ip
* use protocol for locations
* estimate_from helper, via_home estimator, all.py
* via_home: add accuracy, cache history
* add datasources to gpslogger/google_takeout
* tz/via_location.py: update import to fallback
* denylist docs/installation instructions
* tz.via_location: let user customize cachew refresh time
* add via_ip.estimate_location using binary search
* use estimate_location in via_home.get_location
* tests: add gpslogger to location config stub
* tests: install tz related libs in test env
* tz: add regression test for broken windows dates

* vendorize bisect_left from python src
doesnt have a 'key' parameter till python3.10
 2023-02-28 04:30:06 +00:00
Dima Gerasimov
6dc5e7575f vk_messages_backup: add unique_everseen to prevent duplicate messages 2023-02-28 03:55:44 +00:00
Dima Gerasimov
a7099e2efc vk_messages_backup: more correct handling of group chats & better chat ids 2023-02-28 03:55:44 +00:00
Dima Gerasimov
02c98143d5 vk_messages_backup: better structure & exract richer information 2023-02-28 03:55:44 +00:00
Dima Gerasimov
130c273513 my.telegram.telegram_backup enhancements 
- add chat handle
- add permalink
- more precise types
 2023-02-21 02:44:18 +00:00
Dima Gerasimov
07e7c62d02 general/ci: mypy check tests 2023-02-21 00:20:58 +00:00
Dima Gerasimov
c63177e186 general/ci: clean up mypy-misc pipeline, only exclude specific files instead 
marked some module configs which aren't really ready for public use as type: ignore
 2023-02-21 00:20:58 +00:00
Dima Gerasimov
eff9c02886 my.fbmessenger.android: add optional facebook_id 2023-02-20 20:14:15 +00:00
Dima Gerasimov
af874d2d75 my.fbmessenger.android: minor refactoring, comments & error handling 2023-02-20 20:14:15 +00:00
Dima Gerasimov
6493859ba5 my.telegram: initial module from telegram_backup 2023-02-19 01:20:38 +00:00
Dima Gerasimov
6594ad24dc my.tinder.android: speedup unique_everseen by adding unsafe_hash 2023-02-19 01:20:38 +00:00
Dima Gerasimov
458633ea96 my.tinder.android: add a bit of logging 2023-02-19 01:20:38 +00:00
Dima Gerasimov
0e884fe166 core/modules: switch away from using override_config to tmp_config in some tests & faka data generators 2023-02-09 02:35:09 +00:00
Dima Gerasimov
5ac5636e7f core: better support for ad-hoc configs 
properly reload/unload the relevant modules so hopefully no more weird hacks should be required

relevant
- https://github.com/karlicoss/promnesia/issues/340
- https://github.com/karlicoss/HPI/issues/46
 2023-02-09 02:35:09 +00:00
Dima Gerasimov
fb0c1289f0 my.fbmessenger.export: use context manager to properly close sqlite connection 2023-02-08 02:18:00 +00:00
Dima Gerasimov
bb5ad2b6ac core: make hpi install more defensive, just warn on no requirements 
this is useful for backwards compatibility if modules remove their requirements
 2023-02-07 01:57:00 +00:00
Dima Gerasimov
5c82d0faa9 switch from using dataset to raw sqlite3 module 
dataset is kinda unmaintaned and currently broken due to sqlalchemy 2.0 changes

resolves https://github.com/karlicoss/HPI/issues/264
 2023-02-07 01:57:00 +00:00
Dima Gerasimov
9c432027b5 instagram.android: fix missing id 2023-02-07 01:57:00 +00:00
karlicoss
11b6e51c90 ci: fix tox config 
seems that after version 4.0 it's necessary to specify environments to run
previosly it was picking them up automatically
 2023-01-31 00:31:54 +00:00
Sean Breckenridge
54e6fe6ab5 ci: try disabling parallel pip installs on windows 2022-12-17 21:07:30 +00:00
Sean Breckenridge
ad52e131a0 google.takeout.parser: recreate cache on upgrade 
https://github.com/seanbreckenridge/google_takeout_parser/pull/37
 2022-12-17 21:07:30 +00:00
Sean Breckenridge
716a2c82ba core/serialize: serialize stdlib Decimal class 2022-10-19 00:07:30 +01:00
Dima Gerasimov
7098d6831f fix mypy in _identity 
seems easier to just ignore considering it's "internal" function

also a couple of tests to make sure it infers types correctly
 2022-10-19 00:06:23 +01:00
Dima Gerasimov
5f1d41fa52 my.twitter.archive: fix for newer format (tweets filename changed to tweets.js) 2022-10-19 00:06:23 +01:00
Dima Gerasimov
ca91be8154 twitter.archive: fix legacy config detection 
apparently .name contains the parent module so previously it was throwing the exception instead
 2022-10-19 00:06:23 +01:00
Dima Gerasimov
c8cf0272f9 instagram.gdpr: use new path to personal information 2022-10-19 00:06:23 +01:00
Sean Breckenridge
7925ec81b6 docs: browser - fix examples for config 2022-08-29 00:03:32 +01:00
Dima Gerasimov
119b295d71 core: allow legacy modules to be used in 'hpi module install' for backwards compatibility 
but show warning

kinda hacky, but hopefully we will simplify it further when we have more such legacy modules
 2022-06-07 22:59:08 +01:00
Sean Breckenridge
dbd15a7ee8 source: propogate help url for config errors 2022-06-07 21:33:38 +01:00
Dima Gerasimov
cef9b4c6d3 ci: try using --parallel install for mypy pipeline 
`time tox -e mypy-misc` (removed the actual mypy call)

before (each module in a separate 'hpi install' command)
```
real	1m45.901s
user	1m19.555s
sys	0m5.491s
```

in a single 'hpi install' command (multiple modules)
```
real	1m31.252s
user	1m6.028s
sys	0m5.065s
```

single 'hpi install' command with --parallel
```
real	0m15.674s
user	0m50.986s
sys	0m3.249s
```
 2022-06-06 09:49:15 +01:00
Dima Gerasimov
f0397b00ff core/main: experimental --parallel flag for hpi module install 2022-06-06 09:49:15 +01:00
Dima Gerasimov
5f0231c5ee core/main: allow passing multiple packages to 'module install'/'module requires' subcommands 2022-06-06 09:49:15 +01:00
Dima Gerasimov
016f28250b general: initial flake8 checks (for now manual) 
fix fairly uncontroversial stuff in my.core like
- line spacing, which isn't too annoying (e.g. unlike many inline whitespace checks that break vertical formatting)
- unused imports/variables
- too broad except
 2022-06-05 22:28:38 +01:00
Dima Gerasimov
fd0c65d176 my.tinder: initial module for android databases 2022-06-04 17:16:28 +01:00
Dima Gerasimov
b9d788efd0 some enhancements for facebook/instagram modules 
figured out that datetimes are naive
better username handling + investigation of thread names
 2022-06-04 17:16:28 +01:00
Sean Breckenridge
7323e99504 zulip: add stats function 2022-06-04 10:04:33 +01:00