`time tox -e mypy-misc` (removed the actual mypy call)
before (each module in a separate 'hpi install' command)
```
real 1m45.901s
user 1m19.555s
sys 0m5.491s
```
in a single 'hpi install' command (multiple modules)
```
real 1m31.252s
user 1m6.028s
sys 0m5.065s
```
single 'hpi install' command with --parallel
```
real 0m15.674s
user 0m50.986s
sys 0m3.249s
```
- use portable separators
- paths should be prepended with r' (so backwards slash isn't treated as escaping)
- sqlite connections should be closed (otherwise windows fails to remove the underlying db file)
- workaround for emojis via PYTHONUTF8=1 test for now
- make ZipPath portable
- properly use tox python environment everywhere
this was causing issues on Windows
e.g.
WARNING: test command found but not installed in testenv
cmd: C:\hostedtoolcache\windows\Python\3.9.12\x64\python3.EXE
* initial pushshift/rexport merge implementation, using id for merging
* smarter module deprecation warning using regex
* add `RedditBase` from promnesia
* `import_source` helper for gracefully handing mixin data sources
* my.core.serialize: simplejson support, more types
I added a couple extra checks to the default function,
serializing datetime, dates and dataclasses (incase
orjson isn't installed)
(copied from below)
if orjson couldn't be imported, try simplejson
This is included for compatibility reasons because orjson
is rust-based and compiling on rarer architectures may not work
out of the box
as an example, I've been having issues getting it to install
on my phone (termux/android)
unlike the builtin JSON modue which serializes NamedTuples as lists
(even if you provide a default function), simplejson correctly
serializes namedtuples to dictionaries
this just gives another option to people, simplejson is pure python
so no one should have issues with that. orjson is still way faster,
so still preferable if its easy and theres a precompiled build
for your architecture (which there typically is)
If you're ever running this with simplejson installed and not orjson,
its pretty easy to tell as the JSON styling is different; orjson has
no spaces between tokens, simplejson puts spaces between tokens. e.g.
simplejson: {"a": 5, "b": 10}
orjson: {"a":5,"b":10}
- modernize:
- add REQUIRES spec for pdfannots library
- config dataclass/config stub
- stats function
- absolute my.core imports in anticipation of splitting core
- use 'paths' instead of 'roots' (better reflects the semantics), use get_files
backward compatible via config migration
- properly run tests/mypy
- rely on appdirs for default cache path instead of hardcoded /var/tmp/cachew
technically backwards incompatible, but no action needed
you might want to clean /var/tmp/cachew after updating
- use default cache path (e.g. ~/.cache) by default
see https://github.com/ActiveState/appdirs#some-example-output for more info
*warning*: things will be cached by default now (used to be uncached before)
- treat cache_dir = None in the config
*warning*: kind of backwards incompatible.. but again nothing disasterous
