fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
796 commits 3 branches 41 tags 2.5 MiB
Commit graph

55 commits

Author SHA1 Message Date
Sean Breckenridge
821bc08a23
core/structure: help locate/extract gdpr exports (#175) 
* core/structure: help locate/extract gdpr exports

* ci: add install-types to install stub packages
 2021-07-08 00:44:55 +01:00
Dima Gerasimov
0517f7ffb8 core/influxdb: add main method to create influx measurement and fill with values 
allows running something like

    python3 -m my.core.influxdb populate my.zotero
 2021-04-27 20:10:10 +01:00
Dima Gerasimov
393ed0d9ce core: set _max_workers for dummy concurrent pool 2021-04-22 11:11:39 +01:00
Sean Breckenridge
fb49243005
core: add hpi query command (#157) 
- restructure query code for cli, some test fixes
- initial query_range implementation

    refactored functions in query some more
    to allow re-use in range_range, select()
    pretty much just calls out to a bunch
    of handlers now
 2021-04-06 17:19:58 +01:00
Dima Gerasimov
b306ccc839 core: add ensure_unique iterator transfromation 2021-04-02 20:09:53 +01:00
Dima Gerasimov
ad177a1ccd my.pdfs: cleanup/refactor 
- modernize:
  - add REQUIRES spec for pdfannots library
  - config dataclass/config stub
  - stats function
  - absolute my.core imports in anticipation of splitting core
- use 'paths' instead of 'roots' (better reflects the semantics), use get_files
  backward compatible via config migration
- properly run tests/mypy
 2021-04-01 17:27:06 +01:00
Dima Gerasimov
5c38872efc core: add DummyExecutor to make it easier to debug concurrent code with Pools 2021-04-01 17:27:06 +01:00
Sean Breckenridge
eb26cf8633
my.core.serialize: orjson with additional default and _serialize hook (#140) 
basic orjson serialize, json.dumps fallback

Lots of surrounding changes from this discussion:
0593c69056
 2021-03-20 00:48:03 +00:00
Dima Gerasimov
ca4d58e4e7 core: add helper to 'freeze' dataclasses, in order to derive a schema from the properties 2021-02-25 19:56:35 +00:00
Dima Gerasimov
86497f9b13 new: basic arbtt module 2021-02-25 19:56:35 +00:00
Dima Gerasimov
271cd7feef core/cachew: use cache_dir in mcachew if it wasn't specified by the user 2021-02-21 19:51:58 +00:00
Dima Gerasimov
9afe1811a5 core/cachew: special handling for None in order to preserve cache_dir() path 
+ add 'suffix' argument for more straighforward logic
 2021-02-21 19:51:58 +00:00
Dima Gerasimov
da3c1c9b74 core/cachew: rely on ~/.cache for default cache path 
- rely on appdirs for default cache path instead of hardcoded /var/tmp/cachew
  technically backwards incompatible, but no action needed
  you might want to clean /var/tmp/cachew after updating

- use default cache path (e.g. ~/.cache) by default
  see https://github.com/ActiveState/appdirs#some-example-output for more info
  *warning*: things will be cached by default now (used to be uncached before)

- treat cache_dir = None in the config
  *warning*: kind of backwards incompatible.. but again nothing disasterous
 2021-02-21 19:51:58 +00:00
Dima Gerasimov
4ad4f34cda core: improve mypy coverage 2021-02-18 02:04:40 +00:00
Dima Gerasimov
f102101b39 core/windows: fix get_files and its tests 2021-02-16 06:40:42 +00:00
Dima Gerasimov
6239879245 core: add more tests for stat/datetime guessing 2021-02-14 16:20:38 +00:00
Dima Gerasimov
4012f9b7c2 core: more generic functions to jsonify data, rescuetime: fix influxdb filling 2021-02-14 16:20:38 +00:00
Dima Gerasimov
07f901e1e5 core: helpers for automatic dataframes from sequences of NamedTuple/dataclass 
also use in my.rescuetime
 2021-02-14 16:20:38 +00:00
Dima Gerasimov
df9a7f7390 core.pandas: add check for 'error' column + add empty one by default 2021-02-14 16:20:38 +00:00
Dima Gerasimov
ddea816a49 my.stackexchange: use proper pip package, add stat 
+ 'anonymous' mode for stat() function
 2020-12-11 07:02:16 +01:00
Dima Gerasimov
8abe66526d my.photos: minor fixes/configcleanup + speedup 2020-11-25 04:47:30 +01:00
Dima Gerasimov
29ad315578 core/cli: some enhacements for frendlier errors/default config 
see https://github.com/karlicoss/HPI/issues/110
 2020-11-23 20:46:26 +01:00
Dima Gerasimov
ed4b6c409f doctor & stat improvements 
- doctor: log when there is no stats() function and suggest to use it
- check that stats() result isn't None
- warn when there is not data in the iterable
 2020-11-01 02:02:43 +01:00
Dima Gerasimov
96be32aa51 mypy-friendly compat functions handling 2020-11-01 01:51:10 +01:00
Dima Gerasimov
3a9e3e080f my.time.tz: implement different policies for localizing 2020-11-01 01:51:10 +01:00
Dima Gerasimov
15789a4149 kyhton.kompress: move to core (with a fallback, used in promnesia) 2020-10-29 03:13:18 +01:00
Dima Gerasimov
cc127f1876 kython.klogging 
- move to core
- add a proper description why it's useful
- make default level INFO
- use HPI_LOGS variable for easier log level control (abdc6df1ea)
 2020-10-29 03:13:18 +01:00
Dima Gerasimov
2a2478bfa9 core: update cachew annotations 
orgmode: expose method to construct cacheable note
 2020-10-21 01:29:29 +02:00
Dima Gerasimov
6a1a006202 core: add DataFrame support to stat 2020-10-11 18:44:37 +02:00
Dima Gerasimov
ced93e6942 reflect cachew changes of exception handling and temporary suppression 2020-10-08 21:22:02 +02:00
Dima Gerasimov
06ee72bc30 core: more type annotations 2020-10-03 18:24:08 +02:00
Dima Gerasimov
e34c04ebc8 core.cachew: make disabled_cachew defensive 2020-09-18 23:50:40 +01:00
Sean Breckenridge
78489157a1 fix spelling mistakes 2020-09-06 20:44:28 +01:00
Dima Gerasimov
6515d1430f core: experimental guessing for last objects' date 2020-08-20 21:30:52 +01:00
Dima Gerasimov
a9ae6dbb7f core: add error count to stats helper 2020-07-30 22:37:21 +01:00
Dima Gerasimov
92307d5f3d bluemaestro: support new databases as well 2020-07-28 20:32:35 +01:00
Dima Gerasimov
092aef88ce core: detect compression, wrap in CPath if necessary 2020-07-26 21:31:26 +01:00
Dima Gerasimov
1cc4eb5d8d core: add helper for computing stats; use it in modules 2020-06-04 22:19:34 +01:00
Dima Gerasimov
3d7844b711 core: support '' for explicitly set empty path set 2020-06-01 23:45:26 +01:00
Dima Gerasimov
d7aff1be3f github: start moving to a proper artbitrated module 2020-06-01 22:49:31 +01:00
Dima Gerasimov
248e48dc30 core: improve types for warn_if_empty 
ok, works with this advice https://github.com/python/mypy/issues/1927 + overloads
 2020-05-25 01:23:30 +01:00
Dima Gerasimov
216944b3cd core: improvements for warnings, twitter/rss: try using @warn_if_empty 2020-05-25 00:56:03 +01:00
Dima Gerasimov
616ffb457e core: user overloads to type @warn_if_empty properly.. 2020-05-25 00:25:33 +01:00
Dima Gerasimov
e3a71ea6c6 my.core: more work on typing @warn_if_empty, extra test 2020-05-25 00:25:33 +01:00
Dima Gerasimov
4b22d17188 core: add @warn_if_empty decorator 2020-05-25 00:25:33 +01:00
Dima Gerasimov
b99b2f3cfa core: add warning when get_files returns no files, my.twitter.archive: make more defensive in case of no archives 2020-05-24 12:51:23 +01:00
Dima Gerasimov
c8bdbfd69f core: expand '~' in get_files & import_dir 2020-05-18 22:43:27 +01:00
Dima Gerasimov
403ec18385 core/modules: get rid of set_repo uses, it was just complicating everythin 2020-05-18 21:33:52 +01:00
Dima Gerasimov
0f80e9d5e6 ok, seems that import_dir is a bit saner 2020-05-18 21:04:38 +01:00
Dima Gerasimov
87ad9d38bb polar: add test for orger integration 2020-05-15 09:52:18 +01:00