fz0x1/HPI
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
946 commits 3 branches 41 tags 2.5 MiB
Commit graph

946 commits

This branch
This branch
All branches
Author SHA1 Message Date
Sean Breckenridge
821bc08a23
core/structure: help locate/extract gdpr exports (#175) 
* core/structure: help locate/extract gdpr exports

* ci: add install-types to install stub packages
 2021-07-08 00:44:55 +01:00
Dima Gerasimov
8ca88bde2e polar: backward compatibility for my.reading.polar 2021-05-29 13:26:01 +01:00
Dima Gerasimov
2a4bddea79 polar: move to top level, add page support 2021-05-29 13:26:01 +01:00
Sean Breckenridge
e8be20dcb5
core: add tmp_dir for global access to a tmp dir (#173) 
* core: add tmp_dir for global access to a tmp dir
 2021-05-17 00:28:26 +01:00
Sean Breckenridge
b64a11cc69
smscalls: allow multiple backup dirs (#172) 
* smscalls: allow multiple backup dirs
* add smscalls to my.config, add test to CI
 2021-05-14 01:35:36 +01:00
Sean Breckenridge
014494059d smscalls: add REQUIRES block to install lxml 2021-05-10 19:51:20 +01:00
Sean Breckenridge
43cfb2742f
cli/query: bugfix, convert output to list (#170) 
* cli/query: bugfix, convert output to list to keep it backwards compatible
 2021-04-28 21:19:49 +01:00
Sean Breckenridge
fa7474c087 cli/query: add --stream flag 
allows you to do something like

hpi query --stream my.reddit.comments
to stream the JSON objects one per line, makes
it nicer to pipe into 'jq'/'fzf' instead
of having to process the giant list
at the end
 2021-04-28 18:23:16 +01:00
Sean Breckenridge
d71383ddee stats/is_data_provider: ignore 'inputs' func 2021-04-28 18:00:49 +01:00
Dima Gerasimov
68019c80db core/influx: reuse _locate_functions_or_prompt to choose the data provider 2021-04-27 20:10:10 +01:00
Dima Gerasimov
0517f7ffb8 core/influxdb: add main method to create influx measurement and fill with values 
allows running something like

    python3 -m my.core.influxdb populate my.zotero
 2021-04-27 20:10:10 +01:00
Sean Breckenridge
0278f2b68d cli/query: improve fallback behaviour/error msg 2021-04-24 06:15:59 +01:00
Dima Gerasimov
491bef83bc bluemaestro: make defensive, yield Exception for measurements 2021-04-22 11:11:39 +01:00
Dima Gerasimov
2611e237a3 my.orgmode: add stat function 2021-04-22 11:11:39 +01:00
Dima Gerasimov
393ed0d9ce core: set _max_workers for dummy concurrent pool 2021-04-22 11:11:39 +01:00
Sean Breckenridge
4b4cb7cb5b cli/query: bugfix where datetime was ignored 2021-04-19 20:21:17 +01:00
Sean Breckenridge
277f0e3988
cli/query: interactive fallback, improve guess_stats (#163) 2021-04-19 18:57:42 +01:00
Dima Gerasimov
91eed15a75 my.zotero: extract top level item's tags 2021-04-13 18:05:49 +01:00
Dima Gerasimov
68d3385468 my.zotero: handle colors & extract human readable 2021-04-13 18:05:49 +01:00
Dima Gerasimov
1ef2c5619e my.zotero: initial version 2021-04-13 18:05:49 +01:00
Sean Breckenridge
c1b70cd90e
docs: add some documentation on module design (#160) 2021-04-11 16:53:43 +01:00
Sean Breckenridge
f559e7cb89 my.coding.commits: fix misspelling/add warning 2021-04-07 19:59:27 +01:00
Sean Breckenridge
fb49243005
core: add hpi query command (#157) 
- restructure query code for cli, some test fixes
- initial query_range implementation

    refactored functions in query some more
    to allow re-use in range_range, select()
    pretty much just calls out to a bunch
    of handlers now
 2021-04-06 17:19:58 +01:00
Dima Gerasimov
b94120deaf core/sqlite: add compat version for backup() for python3.6 2021-04-05 08:37:07 +01:00
Dima Gerasimov
f09ca17560 core/sqlite: move tests to separate module, pickling during Pool.submit can't handle importing :( 2021-04-05 08:37:07 +01:00
Dima Gerasimov
e99e8725b1 core/sqlite: add a helper to do an im-memory snapshot of db including WAL 
+ add a bunch of tests for different WAL behaviours
 2021-04-05 08:37:07 +01:00
Dima Gerasimov
f2a339f755 core/sqlite: extract immutable connection helper 
use in bluemaestro/zotero modules
 2021-04-05 08:37:07 +01:00
Sean Breckenridge
349ab78fca
core/cli: switch to using click library #155 
everything is backwards-compatible with the previous
interface, the only minor changes were to the doctor cmd
which can now accept more than one item to run,
and the --skip-config-check to skip the config_ok
check if the user specifies to

added a test using click.testing.CliRunner (tests
the CLI in an isolated environment), though
additional tests which aren't testing the CLI
itself (parsing arguments or decorator behaviour)
can just call the functions themselves, as they
no longer accept a argparser.Namespace and instead
accept the direct arguments
 2021-04-04 10:06:59 +01:00
Dima Gerasimov
5ef2775265 my.github: some work in progress on generating consistent ids 
sadly it seems that there are at several issues:

- gdpr has less detailed data so it's hard to generate a proper ID at times
- sometimes there is a small (1s?) discrepancy between created_at between same event in GDPR an API
- some API events can have duplicate payload, but different id, which violates uniqueness
 2021-04-02 20:09:53 +01:00
Dima Gerasimov
386234970b my.github.ghexport: handle more event types, more consisten body handling 2021-04-02 20:09:53 +01:00
Dima Gerasimov
b306ccc839 core: add ensure_unique iterator transfromation 2021-04-02 20:09:53 +01:00
Sean Breckenridge
c31641b4eb
core: discovery_pure; allow multiple package roots (#152) 
* core: discovery_pure; allow multiple package roots

iterates over my.__path__._path if possible
to discover additional paths to iterate over

else defaults to the path relative to
the current file
 2021-04-02 15:46:18 +01:00
Sean Breckenridge
5ecd4b4810 cleanup; remove unused imports 2021-04-02 08:38:06 +01:00
Sean Breckenridge
a11a3af597 commits: reduce possibility of path conflicts 2021-04-02 07:20:35 +01:00
Dima Gerasimov
edf6e5d50b my.pdfs: rely on pdfannots for created date extraction/parsing 2021-04-01 17:27:06 +01:00
Dima Gerasimov
ad177a1ccd my.pdfs: cleanup/refactor 
- modernize:
  - add REQUIRES spec for pdfannots library
  - config dataclass/config stub
  - stats function
  - absolute my.core imports in anticipation of splitting core
- use 'paths' instead of 'roots' (better reflects the semantics), use get_files
  backward compatible via config migration
- properly run tests/mypy
 2021-04-01 17:27:06 +01:00
Dima Gerasimov
e7604c188e my.pdfs: reorganize tests a bit, fix mypy 2021-04-01 17:27:06 +01:00
Dima Gerasimov
5c38872efc core: add DummyExecutor to make it easier to debug concurrent code with Pools 2021-04-01 17:27:06 +01:00
Sean Breckenridge
3118891c03
my.core.query: initial implementation (#143) 
in particular `my.core.query.select`: a function to query, order, sort and filter items from one or more sources
 2021-03-28 07:52:50 +01:00
Sean Breckenridge
d47f3c28aa my.core.serialize: support serializing Paths 2021-03-28 06:55:03 +01:00
Sean Breckenridge
1b36bd4379 fix spelling mistakes 2021-03-28 06:53:24 +01:00
Dima Gerasimov
29384aef44 my.goodreads: cleanup, rename from my.reading.goodrads & use proper pip dependency 
related:
- https://github.com/karlicoss/HPI/issues/79
- 10d8cc86a1
 2021-03-26 05:06:53 +00:00
Sean Breckenridge
1cdef6f40a fix mypy errors 
this fixes two distinct mypy errors

one where NamedTuple/dataclassees can't be
defined locally
https://github.com/python/mypy/issues/7281

which happens when you run mypy like
mypy -p my.core on warm cache

the second error is the core/types.py file shadowing the
stdlib types module
 2021-03-22 06:34:07 +00:00
ddrone
2bbbff0021 Update README.org 2021-03-21 15:16:21 +00:00
ddrone
8e868391fd Update README.org 
*muffled Yung Lean playing in the distance*
 2021-03-21 15:16:21 +00:00
Sean Breckenridge
eb26cf8633
my.core.serialize: orjson with additional default and _serialize hook (#140) 
basic orjson serialize, json.dumps fallback

Lots of surrounding changes from this discussion:
0593c69056
 2021-03-20 00:48:03 +00:00
Sean Breckenridge
02a9fb5e8f github.gdpr: parse project files 
also fixed a typo in commit_comments
 2021-03-15 12:40:22 +00:00
Dima Gerasimov
a1a24ffbc3 my.coding.commits: more cleanup 
Followup of https://github.com/karlicoss/HPI/pull/132

- add REQUIRES section
- use 'commits' config section & add proper schema
- use dedicated subdirectory for cache
 2021-03-15 10:33:46 +00:00
Dima Gerasimov
ec8b0e9170 my.coding.commits: actually test on CI, add config stub 2021-03-15 10:33:46 +00:00
Dima Gerasimov
8d6f691824 core: feature: guess module stats from typing annotations 2021-03-15 10:27:18 +00:00