Commit graph

589 commits

Author SHA1 Message Date
karlicoss
0bcc5952c7
Merge pull request #62 from karlicoss/updates
updates: core & kobo
2020-06-04 22:55:15 +01:00
Dima Gerasimov
821eb47c93 kobo: BREAKING changes. Use kobuddy module directly, rename export_dir to export_path.
Hopefully this makes a lot of sense in the first place, and not that many users, so deserves breaking.
2020-06-04 22:50:52 +01:00
Dima Gerasimov
db852b3927 kobo: move away from my.books 2020-06-04 22:20:48 +01:00
Dima Gerasimov
1cc4eb5d8d core: add helper for computing stats; use it in modules 2020-06-04 22:19:34 +01:00
karlicoss
a94b64c273
Merge pull request #61 from karlicoss/updates
github module: cleanup and proper modular layout
2020-06-01 23:52:07 +01:00
Dima Gerasimov
3d7844b711 core: support '' for explicitly set empty path set 2020-06-01 23:45:26 +01:00
Dima Gerasimov
a267aeec5b github: add config templates + docs
- ghexport: use export_path (export_dir is still supported)
2020-06-01 23:33:34 +01:00
Dima Gerasimov
ca39187c63 github: DEPRECATE my.coding.github
Instead my.github.all should be used (still backward compatible)

The reasons are
a) I don't feel that grouping (i.e. my.coding.*) makes much sense
b) using .all pattern (same way as twitter) allows for more composable and cleaner separation of GDPR and API data
2020-06-01 22:49:31 +01:00
Dima Gerasimov
d7aff1be3f github: start moving to a proper artbitrated module 2020-06-01 22:49:31 +01:00
Matthew Reishus
67cf4d0c04 my.coding.github ignores some events emitted by bots.
I use a service called dependabot ( https://dependabot.com/ ).  It
automatically creates pull requests in my repositories to upgrade
dependencies.  The modern front end javascript world moves really
quickly; projects have a ton of dependencies that are updating all the
time, so there are a lot of these pull requests.

Also, the PRs it makes have a lot of info in them.  Here's an example
one: https://github.com/mreishus/spades/pull/180 .  If you hit the
arrows, you can see it includes a lot of text in "Changelog" and
"Commits".  Now check out the list of closed PRs this project has:
https://github.com/mreishus/spades/pulls?q=is%3Apr+is%3Aclosed

Once I got everything working with my.coding.github, my Github.org
(using orger) was huge: 5MB.  I wanted to get rid of the dependabot
stuff, since it's mostly junk I'm not too interested it, and I got it
down to 130K (from 5MB) just from this commit.

Here's an example of an event I'm filtering out:
I'm looking to see if the "user" contains a "[bot]" tag in it.

  {
    "type": "pull_request",
    "url": "https://github.com/mreishus/spades/pull/96",
    "user": "https://github.com/dependabot-preview[bot]",
    "repository": "https://github.com/mreishus/spades",
    "title": "Bump axios from 0.19.1 to 0.19.2 in /frontend",
    "body": "Bumps [axios](https://github.com/axios/axios) from 0.19.1 to 0.19.2.\n<details>\n<summary>Release notes</summary [cut 5000 characters]
    "base": {
      "ref": "master",
      "sha": "a47687762887c6e5c0d5d0a38c3c9697f09cbcd6",
      "user": "https://github.com/mreishus",
      "repo": "https://github.com/mreishus/spades"
    },
    "head": {
      "ref": "dependabot/npm_and_yarn/frontend/axios-0.19.2",
      "sha": "0e79d0220002cb54cd40e13a40addcc0d0a01482",
      "user": "https://github.com/mreishus",
      "repo": "https://github.com/mreishus/spades"
    },
    "assignee": "https://github.com/mreishus",
    "assignees": [
      "https://github.com/mreishus"
    ],
    "milestone": null,
    "labels": [
      "https://github.com/mreishus/spades/labels/dependencies",
      "https://github.com/mreishus/spades/labels/javascript"
    ],
    "review_requests": [

    ],
    "work_in_progress": false,
    "merged_at": null,
    "closed_at": "2020-01-25T14:40:27Z",
    "created_at": "2020-01-22T13:37:17Z"
  },

Maybe this should be a config option, but I didn't know how to make them
cleanly in HPI, and I'm not sure if anyone would ever want this stuff.
2020-06-01 16:22:07 +01:00
Dima Gerasimov
f175acc848 pocket: reuse pockexport data access layer
BREAKING CHANGE! Data parsing was switched to pockexport.
This would help to keep it consistent across different apps in the future.

When you update, you'll need to:

- clone pockexport (latest version)
- set pockexport repository in your config (see doc/MODULES.org)
2020-05-27 08:42:47 +01:00
Dima Gerasimov
6453ff415d docs: somewhat acceptable data flow diagrams 2020-05-26 22:51:50 +01:00
Dima Gerasimov
150a6a8cb7 docs: wip on better explanation of configs/diagram 2020-05-26 22:51:50 +01:00
karlicoss
04eca6face
Merge pull request #55 from karlicoss/updates
cli updates: doctor mode
2020-05-25 12:30:18 +01:00
Dima Gerasimov
e351c8ba49 cli: add 'config init' command 2020-05-25 12:25:41 +01:00
Dima Gerasimov
7bd7cc9228 cli: integrate with stats reported by the modules 2020-05-25 11:46:30 +01:00
Dima Gerasimov
d890599c7c cli: add checks for importing modules 2020-05-25 11:41:44 +01:00
Dima Gerasimov
8019389ccb cli: move doctor to core, add doc 2020-05-25 10:17:40 +01:00
Dima Gerasimov
dab29a44b5 cli: detect config properly in mypy check 2020-05-25 10:04:58 +01:00
Dima Gerasimov
2ede5b3a5c cli: add config check command 2020-05-25 09:49:57 +01:00
karlicoss
ce8cd5b52c
Merge pull request #54 from karlicoss/updates
core: update warnings, add warn_if_empty decorator fore move defensive data sources
2020-05-25 01:28:42 +01:00
Dima Gerasimov
248e48dc30 core: improve types for warn_if_empty
ok, works with this advice https://github.com/python/mypy/issues/1927 + overloads
2020-05-25 01:23:30 +01:00
Dima Gerasimov
216944b3cd core: improvements for warnings, twitter/rss: try using @warn_if_empty 2020-05-25 00:56:03 +01:00
Dima Gerasimov
616ffb457e core: user overloads to type @warn_if_empty properly.. 2020-05-25 00:25:33 +01:00
Dima Gerasimov
e3a71ea6c6 my.core: more work on typing @warn_if_empty, extra test 2020-05-25 00:25:33 +01:00
Dima Gerasimov
4b22d17188 core: add @warn_if_empty decorator 2020-05-25 00:25:33 +01:00
karlicoss
af814df8e9
Merge pull request #53 from karlicoss/upd
make my.twitter.all easier to override
2020-05-24 23:02:57 +01:00
Dima Gerasimov
f5267d05d7 my.twitter.archive: rename config (preserving bckwd compatibility for now) 2020-05-24 13:06:52 +01:00
Dima Gerasimov
b99b2f3cfa core: add warning when get_files returns no files, my.twitter.archive: make more defensive in case of no archives 2020-05-24 12:51:23 +01:00
Dima Gerasimov
b7662378a2 docs: minor updates 2020-05-22 19:38:14 +01:00
Dima Gerasimov
03773a7b2c twitter module: prettify top level twitter.all 2020-05-22 19:00:02 +01:00
karlicoss
c410daa484
Merge pull request #52 from karlicoss/updates
Updates
2020-05-18 23:40:58 +01:00
Dima Gerasimov
02ba71a91d documentation: generate tables of content, better navigation 2020-05-18 23:31:55 +01:00
Dima Gerasimov
c8bdbfd69f core: expand '~' in get_files & import_dir 2020-05-18 22:43:27 +01:00
Dima Gerasimov
403ec18385 core/modules: get rid of set_repo uses, it was just complicating everythin 2020-05-18 21:33:52 +01:00
Dima Gerasimov
0f80e9d5e6 ok, seems that import_dir is a bit saner 2020-05-18 21:04:38 +01:00
Dima Gerasimov
44aa062756 tests: thinking about external repositories 2020-05-18 20:42:10 +01:00
karlicoss
41c5b34006
Merge pull request #51 from karlicoss/updates
Improve documentation for some modules
2020-05-17 22:10:58 +01:00
Dima Gerasimov
c0bbb4eaf2 misc: get rid of SimpleNamespace uses 2020-05-17 22:05:23 +01:00
Dima Gerasimov
2a9fd54c12 Improve documentation for some modules 2020-05-17 21:56:58 +01:00
karlicoss
c07ea0a600
Merge pull request #50 from karlicoss/polar
polar module updates
2020-05-17 14:01:49 +01:00
Dima Gerasimov
65138808e7 polar: handle few more attributes defensively 2020-05-15 13:17:02 +01:00
Dima Gerasimov
8277b33c18 polar: add highlight colors 2020-05-15 12:52:22 +01:00
Dima Gerasimov
3d8002c8c9 polar: support configuring defensive behaviour, support for highlight tags 2020-05-15 12:40:15 +01:00
Dima Gerasimov
844ebf28c1 polar: extract book tags 2020-05-15 11:49:30 +01:00
Dima Gerasimov
759b0e1324 polar: expose a proper filename 2020-05-15 10:11:09 +01:00
Dima Gerasimov
87ad9d38bb polar: add test for orger integration 2020-05-15 09:52:18 +01:00
Dima Gerasimov
0f27071dcc polar: minor improvements, konsume: more type annotations 2020-05-15 09:07:23 +01:00
Dima Gerasimov
f3d5064ff2 polar: allow properly specifying polar_dir, with ~ as a default 2020-05-15 08:18:47 +01:00
Dima Gerasimov
8f86d7706b core: use appdirs for ~/.config detection 2020-05-15 08:18:47 +01:00