Commit graph

618 commits

Author SHA1 Message Date
Dima Gerasimov
99e50f0afe core: experiments with attaching datetime information to errors
my.body.weigth: put datetime information in the error rows
2020-09-09 21:37:15 +01:00
Dima Gerasimov
743312a87b my.body.blood: prettify, add stat() 2020-09-09 21:37:15 +01:00
Dima Gerasimov
efea669a3e my.location: some cleanup and speedups 2020-09-09 21:37:15 +01:00
Dima Gerasimov
65781dd152 emfit: patch up timezone for correct local sleep time 2020-09-09 21:37:15 +01:00
Dima Gerasimov
d9bbf7cbf0 emfit: propagate errors properly, expose dataframe 2020-09-09 21:37:15 +01:00
Sean Breckenridge
78489157a1 fix spelling mistakes 2020-09-06 20:44:28 +01:00
Dima Gerasimov
07dd61ca6a my.emfit: move data access layer bits to emfitexport 2020-08-20 21:30:52 +01:00
Dima Gerasimov
975f9dd110 rescuetime: get rid of kython, use cachew 2020-08-20 21:30:52 +01:00
Dima Gerasimov
6515d1430f core: experimental guessing for last objects' date 2020-08-20 21:30:52 +01:00
karlicoss
cde5502151
Merge pull request #74 from thetomcraig/pdfs-process-filelist
Add "filelist" parameter to annotated_pdfs
2020-08-20 21:08:57 +01:00
Adrien Lacquemant
5b2cc577f2 Correct command to create config file 2020-08-20 20:14:55 +01:00
Tom Craig
5dc62ff085 Add tests for pdfs 2020-08-16 13:36:36 -07:00
Tom Craig
882ceb62fc Add a "filelist" paramter to annotated_pdfs 2020-08-16 12:57:20 -07:00
Dima Gerasimov
626ee994bf twint: open database in read only mode 2020-07-31 12:22:13 +01:00
Dima Gerasimov
4920defe12 vk: add messages processing 2020-07-31 12:22:13 +01:00
Dima Gerasimov
c54d85037c core: add base cachew directory 2020-07-31 12:22:13 +01:00
Dima Gerasimov
10a8ebaae4 vk: move favorites module to a subpackage, add stat 2020-07-30 22:37:21 +01:00
Dima Gerasimov
4ee89b85ee reddit: add stats() 2020-07-30 22:37:21 +01:00
Dima Gerasimov
a9ae6dbb7f core: add error count to stats helper 2020-07-30 22:37:21 +01:00
Dima Gerasimov
92307d5f3d bluemaestro: support new databases as well 2020-07-28 20:32:35 +01:00
Dima Gerasimov
9d45eb0559 bluemaestro: make iterative, add stat() 2020-07-28 20:32:35 +01:00
Tom Craig
fdaae59b59 Add .get to call for d[date] 2020-07-27 21:33:44 +01:00
Dima Gerasimov
092aef88ce core: detect compression, wrap in CPath if necessary 2020-07-26 21:31:26 +01:00
Dima Gerasimov
77deef98de reddit: more consistent handling for events 2020-07-26 21:31:26 +01:00
Dima Gerasimov
031b1278eb reddit: cleanup cachew wrapper a bit 2020-07-26 21:31:26 +01:00
Dima Gerasimov
6b548c24c1 doctor: better mypy detection 2020-07-26 21:31:26 +01:00
Dima Gerasimov
5eecd8721d cli: check specific module with doctor; print help on no command 2020-07-06 21:40:41 +01:00
Dima Gerasimov
49d25a75ae core: use immutable mode in dataset helper 2020-07-06 21:40:41 +01:00
Dima Gerasimov
4fc33a9ed2 core: add helper for opening read-only database 2020-07-06 21:40:41 +01:00
karlicoss
0bcc5952c7
Merge pull request #62 from karlicoss/updates
updates: core & kobo
2020-06-04 22:55:15 +01:00
Dima Gerasimov
821eb47c93 kobo: BREAKING changes. Use kobuddy module directly, rename export_dir to export_path.
Hopefully this makes a lot of sense in the first place, and not that many users, so deserves breaking.
2020-06-04 22:50:52 +01:00
Dima Gerasimov
db852b3927 kobo: move away from my.books 2020-06-04 22:20:48 +01:00
Dima Gerasimov
1cc4eb5d8d core: add helper for computing stats; use it in modules 2020-06-04 22:19:34 +01:00
karlicoss
a94b64c273
Merge pull request #61 from karlicoss/updates
github module: cleanup and proper modular layout
2020-06-01 23:52:07 +01:00
Dima Gerasimov
3d7844b711 core: support '' for explicitly set empty path set 2020-06-01 23:45:26 +01:00
Dima Gerasimov
a267aeec5b github: add config templates + docs
- ghexport: use export_path (export_dir is still supported)
2020-06-01 23:33:34 +01:00
Dima Gerasimov
ca39187c63 github: DEPRECATE my.coding.github
Instead my.github.all should be used (still backward compatible)

The reasons are
a) I don't feel that grouping (i.e. my.coding.*) makes much sense
b) using .all pattern (same way as twitter) allows for more composable and cleaner separation of GDPR and API data
2020-06-01 22:49:31 +01:00
Dima Gerasimov
d7aff1be3f github: start moving to a proper artbitrated module 2020-06-01 22:49:31 +01:00
Matthew Reishus
67cf4d0c04 my.coding.github ignores some events emitted by bots.
I use a service called dependabot ( https://dependabot.com/ ).  It
automatically creates pull requests in my repositories to upgrade
dependencies.  The modern front end javascript world moves really
quickly; projects have a ton of dependencies that are updating all the
time, so there are a lot of these pull requests.

Also, the PRs it makes have a lot of info in them.  Here's an example
one: https://github.com/mreishus/spades/pull/180 .  If you hit the
arrows, you can see it includes a lot of text in "Changelog" and
"Commits".  Now check out the list of closed PRs this project has:
https://github.com/mreishus/spades/pulls?q=is%3Apr+is%3Aclosed

Once I got everything working with my.coding.github, my Github.org
(using orger) was huge: 5MB.  I wanted to get rid of the dependabot
stuff, since it's mostly junk I'm not too interested it, and I got it
down to 130K (from 5MB) just from this commit.

Here's an example of an event I'm filtering out:
I'm looking to see if the "user" contains a "[bot]" tag in it.

  {
    "type": "pull_request",
    "url": "https://github.com/mreishus/spades/pull/96",
    "user": "https://github.com/dependabot-preview[bot]",
    "repository": "https://github.com/mreishus/spades",
    "title": "Bump axios from 0.19.1 to 0.19.2 in /frontend",
    "body": "Bumps [axios](https://github.com/axios/axios) from 0.19.1 to 0.19.2.\n<details>\n<summary>Release notes</summary [cut 5000 characters]
    "base": {
      "ref": "master",
      "sha": "a47687762887c6e5c0d5d0a38c3c9697f09cbcd6",
      "user": "https://github.com/mreishus",
      "repo": "https://github.com/mreishus/spades"
    },
    "head": {
      "ref": "dependabot/npm_and_yarn/frontend/axios-0.19.2",
      "sha": "0e79d0220002cb54cd40e13a40addcc0d0a01482",
      "user": "https://github.com/mreishus",
      "repo": "https://github.com/mreishus/spades"
    },
    "assignee": "https://github.com/mreishus",
    "assignees": [
      "https://github.com/mreishus"
    ],
    "milestone": null,
    "labels": [
      "https://github.com/mreishus/spades/labels/dependencies",
      "https://github.com/mreishus/spades/labels/javascript"
    ],
    "review_requests": [

    ],
    "work_in_progress": false,
    "merged_at": null,
    "closed_at": "2020-01-25T14:40:27Z",
    "created_at": "2020-01-22T13:37:17Z"
  },

Maybe this should be a config option, but I didn't know how to make them
cleanly in HPI, and I'm not sure if anyone would ever want this stuff.
2020-06-01 16:22:07 +01:00
Dima Gerasimov
f175acc848 pocket: reuse pockexport data access layer
BREAKING CHANGE! Data parsing was switched to pockexport.
This would help to keep it consistent across different apps in the future.

When you update, you'll need to:

- clone pockexport (latest version)
- set pockexport repository in your config (see doc/MODULES.org)
2020-05-27 08:42:47 +01:00
Dima Gerasimov
6453ff415d docs: somewhat acceptable data flow diagrams 2020-05-26 22:51:50 +01:00
Dima Gerasimov
150a6a8cb7 docs: wip on better explanation of configs/diagram 2020-05-26 22:51:50 +01:00
karlicoss
04eca6face
Merge pull request #55 from karlicoss/updates
cli updates: doctor mode
2020-05-25 12:30:18 +01:00
Dima Gerasimov
e351c8ba49 cli: add 'config init' command 2020-05-25 12:25:41 +01:00
Dima Gerasimov
7bd7cc9228 cli: integrate with stats reported by the modules 2020-05-25 11:46:30 +01:00
Dima Gerasimov
d890599c7c cli: add checks for importing modules 2020-05-25 11:41:44 +01:00
Dima Gerasimov
8019389ccb cli: move doctor to core, add doc 2020-05-25 10:17:40 +01:00
Dima Gerasimov
dab29a44b5 cli: detect config properly in mypy check 2020-05-25 10:04:58 +01:00
Dima Gerasimov
2ede5b3a5c cli: add config check command 2020-05-25 09:49:57 +01:00
karlicoss
ce8cd5b52c
Merge pull request #54 from karlicoss/updates
core: update warnings, add warn_if_empty decorator fore move defensive data sources
2020-05-25 01:28:42 +01:00