fz0x1/jrnl
1
0
Fork 0
mirror of https://github.com/jrnl-org/jrnl.git synced 2025-05-10 08:38:32 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Allow runtime configuration overrides from the commandline (#1169)

Browse source 
Add --config-override feature

* add test and argument handler for runtime override of configurations.
* identify location to apply override in "main"
* update gitignore
* remove unneeded import
* add jrnl interface test for overriden configurations
* trivial whitespace change
* implement runtime override
* make format
* refactor override unittest
* clean up unused import
* start writing integration test
* add linewrap override scenario
* implement editor override step
* add dev dependencies on pytest -mock and -cov
* make format
* remove unused imports
* make format
* rename --override to --config-override
* move override implementation into own module
* begin TDD of dot notated overrides
* rewrite behavior scenario
* implement recursive config overrides
* clean up unittests
* iterate on behave step
* make format
* cleanup
* move override behave tests out of core
* refactor recursive code
* make format
* code cleanup
* remove unused import
* update test config
* rewrite test for better mock call expect
* make format
* binary search misbehaving windows test
* unittest multiple overrides
* uncomment dot notation unittest
* add multiple override scenario spec
* make format
* make format
* update unittests for new syntax
* update integ tests for new syntax
* update gitignore
* guard override application
* deserialize function as return type
* make format
* organize deserialization unittests
* better, more specific behave tests
* test different editor launch commands
* formatting
* handle datatypes in deserialization and update helptext
* stick to config convention in testbed
* update tests ith better verifications
* make format
* space
* review feedbac
* make format
* skip on win
* update deps
* update tests with better verifications
make format
space
review feedbac
* skip on win
* update deps
* refactor deserialization
organize test_parse_args
make format
* skip on win
* refactor deserialization
organize test_parse_args
make format
* update tests ith better verifications
* make format
* space
* make format
* document apply_overrides
* update gitignore
* document config-override enhancement
* Simplify config override syntax (#5)
* update tests and expected behavior
* clean up arg parsing tests
* update deserialization
* update deserialization
* config argparse action
* update override application logic
* update tests; delete unused imports
* override param must be list
* update docstring
* update test input to SUT
* update remaining override unittests
* make format
* forgot to update CLI syntax
* update documentation to sphinx style
* variable renames
* Lockfile merge (#7)
* Add brew and gitter badges to README
* Update changelog [ci skip]
* Make journal selection behavior more consistent when there's a colon with no date (#1164)
* Simplify config override syntax (#8)
* update tests and expected behavior
* clean up arg parsing tests
* update deserialization
* update deserialization
* config argparse action
* update override application logic
* update tests; delete unused imports
* override param must be list
* update docstring
* update test input to SUT
* update remaining override unittests
* make format
* forgot to update CLI syntax
* formatting
* Update pyproject.toml
* update lockfile to remove pytest-cov and pytest-mock deps
* update docs
* reuse existing mock; delete unneeded code
* move overrides earlier in the execution
use existing configs instead of custom
make format
clean up imports
* update for passworded access
context.parser -> parsed_args
* test that no editor is launched
* remove unnecessary mocks
* rename variable for intent
* reinstate getpass deletion
* update gitignore
* capture failure mode
* remove unneeded imports
* renamed variable
* delete redundant step
* comment on step
* clean up step behavior description
* [WIP] lock down journal access behavior
* skip -> wip
* correct command for overriding journal via dot keys
* update wip test for updating a "temp" journal and then reading baack its entries
* remove "mock" from poetry file
* make CI happy
* complex behavior sequence for default journal override
* separate out smaller pieces of logic
test that apply_overrides acts on base configuration and not the copy
* defer modification of loaded configuration to update_config
remove unused fixtures
delete complicated UT since behavior is covered in overrides.feature integ test
delete redundant UT
* Update .gitignore
* remove skip_win
* forward override unpacking to yaml library
* merge config override step with existing config_var step in core
delete config_override step
unify step description syntax
* delete unused and redundant code
* rebases are hard
* remove wip tag from test
* remove skipped tests for windows
* Address code review
yield -> return
remove needless copy
adjust spacing
re-inline args return
reset packaging info to e6c0a16342
revert package version for this PR
* consolidate imports
* Defer config_override unpacking to dict *after* base config is loaded
store cli overrides without unpacking just yet
move deserialize_config_args to config module
delete custom Action class for config operations
apply [k,v] -> {k, v} for each override
update test data
update import
* rename deserialize_config_args to better express intent
make format
This commit is contained in:
Suhas 2021-03-02 21:47:57 -05:00 • committed by GitHub
parent b99cebcee6
commit 4f79803885
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
12 changed files with 526 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
.gitignore vendored
View file

@ -54,3 +54,10 @@ exp/
_extras/
*.sublime-*
site/
.vscode/settings.json
coverage.xml
.vscode/launch.json
.coverage
.vscode/tasks.json
todo.txt

23
docs/advanced.md
View file

@ -62,6 +62,29 @@ and can be edited with a plain text editor.
Or use the built-in prompt or an external editor to compose your
entries.
### Modifying Configurations from the Command line
You can override a configuration field for the current instance of `jrnl` using `--config-override CONFIG_KEY CONFIG_VALUE` where `CONFIG_KEY` is a valid configuration field, specified in dot-notation and `CONFIG_VALUE` is the (valid) desired override value.
You can specify multiple overrides as multiple calls to `--config-override`.
!!! note
These overrides allow you to modify ***any*** field of your jrnl configuration. We trust that you know what you are doing.
#### Examples:
``` sh
#Create an entry using the `stdin` prompt, for rapid logging
jrnl --config-override editor ""
#Populate a project's log
jrnl --config-override journals.todo "$(git rev-parse --show-toplevel)/todo.txt" todo find my towel
#Pass multiple overrides
jrnl --config-override display_format fancy --config-override linewrap 20 \
--config-override colors.title green
```
## Multiple journal files
You can configure `jrnl`to use with multiple journals (eg.

27
docs/recipes.md
View file

@ -154,6 +154,33 @@ only field 1.
jrnl -on "$(jrnl --short | shuf -n 1 | cut -d' ' -f1,2)"
```
### Launch a terminal for rapid logging
You can use this to launch a terminal that is the `jrnl` stdin prompt so you can start typing away immediately.
```bash
jrnl now --config-override editor:""
```
Bind this to a keyboard shortcut.
Map `Super+Alt+J` to launch the terminal with jrnl prompt
- **xbindkeys**
In your `.xbindkeysrc`
```ini
Mod4+Mod1+j
alacritty -t floating-jrnl -e jrnl now --config-override editor:"",
```
- **I3 WM** Launch a floating terminal with the `jrnl` prompt
```ini
bindsym Mod4+Mod1+j exec --no-startup-id alacritty -t floating-jrnl -e jrnl --config-override editor:""
for_window[title="floating *"] floating enable
```
## External editors
Configure your preferred external editor by updating the `editor` option

98
features/overrides.feature Normal file
View file

@ -0,0 +1,98 @@
Feature: Implementing Runtime Overrides for Select Configuration Keys
Scenario: Override configured editor with built-in input === editor:''
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl --config-override editor ''"
Then the stdin prompt should have been called
Scenario: Postconfig commands with overrides
Given We use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl --decrypt --config-override highlight false --config-override editor nano"
Then the config should have "highlight" set to "bool:false"
And no editor should have been called
Scenario: Override configured linewrap with a value of 23
Given we use the config "simple.yaml"
And we use the password "test" if prompted
When we run "jrnl -2 --config-override linewrap 23 --format fancy"
Then the output should be
"""
┎─────╮2013-06-09 15:39
┃ My ╘═══════════════╕
┃ fir st ent ry. │
┠╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
┃ Everything is │
┃ alright │
┖─────────────────────┘
┎─────╮2013-06-10 15:40
┃ Lif ╘═══════════════╕
┃ e is goo d. │
┠╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
┃ But I'm better. │
┖─────────────────────┘
"""
Scenario: Override color selections with runtime overrides
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl -1 --config-override colors.body blue"
Then the config should have "colors.body" set to "blue"
Scenario: Apply multiple config overrides
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl -1 --config-override colors.body green --config-override editor 'nano'"
Then the config should have "colors.body" set to "green"
And the config should have "editor" set to "nano"
Scenario Outline: Override configured editor
Given we use the config "basic_encrypted.yaml"
And we use the password "test" if prompted
When we run "jrnl --config-override editor '<editor>'"
Then the editor <editor> should have been called
Examples: Editor Commands
| editor |
| nano |
| vi -c startinsert |
| code -w |
Scenario: Override default journal
Given we use the config "basic_dayone.yaml"
And we use the password "test" if prompted
When we run "jrnl --debug --config-override journals.default features/journals/simple.journal 20 Mar 2000: The rain in Spain comes from clouds"
Then we should get no error
And we should see the message "Entry added"
When we run "jrnl -3 --debug --config-override journals.default features/journals/simple.journal"
Then the output should be
"""
2000-03-20 09:00 The rain in Spain comes from clouds
2013-06-09 15:39 My first entry.
| Everything is alright
2013-06-10 15:40 Life is good.
| But I'm better.
"""
Scenario: Make an entry into an overridden journal
Given we use the config "basic_dayone.yaml"
And we use the password "test" if prompted
When we run "jrnl --config-override journals.temp features/journals/simple.journal temp Sep 06 1969: @say Ni"
Then we should get no error
And we should see the message "Entry added"
When we run "jrnl --config-override journals.temp features/journals/simple.journal temp -3"
Then the output should be
"""
1969-09-06 09:00 @say Ni
2013-06-09 15:39 My first entry.
| Everything is alright
2013-06-10 15:40 Life is good.
| But I'm better.
"""

41
features/steps/core.py
View file

@ -3,6 +3,7 @@
import ast
from collections import defaultdict
from jrnl.args import parse_args
import os
from pathlib import Path
import re
@ -13,8 +14,11 @@ from behave import given
from behave import then
from behave import when
import keyring
import toml
import yaml
from yaml.loader import FullLoader
import jrnl.time
from jrnl import Journal
@ -23,6 +27,7 @@ from jrnl import plugins
from jrnl.cli import cli
from jrnl.config import load_config
from jrnl.os_compat import split_args
from jrnl.override import apply_overrides, _recursively_apply
try:
import parsedatetime.parsedatetime_consts as pdt
@ -114,8 +119,15 @@ def read_value_from_string(string):
return ast.literal_eval(string)
# Takes strings like "bool:true" or "int:32" and coerces them into proper type
t, value = string.split(":")
value = {"bool": lambda v: v.lower() == "true", "int": int, "str": str}[t](value)
string_parts = string.split(":")
if len(string_parts) > 1:
type = string_parts[0]
value = string_parts[1:][0] # rest of the text
value = {"bool": lambda v: v.lower() == "true", "int": int, "str": str}[type](
value
)
else:
value = string_parts[0]
return value
@ -315,6 +327,7 @@ def run_with_input(context, command, inputs=""):
text = iter([inputs])
args = split_args(command)[1:]
context.args = args
def _mock_editor(command):
context.editor_command = command
@ -397,8 +410,13 @@ def run(context, command, text=""):
if "cache_dir" in context and context.cache_dir is not None:
cache_dir = os.path.join("features", "cache", context.cache_dir)
command = command.format(cache_dir=cache_dir)
if "config_path" in context and context.config_path is not None:
with open(context.config_path, "r") as f:
cfg = yaml.load(f, Loader=FullLoader)
context.jrnl_config = cfg
args = split_args(command)
context.args = args[1:]
def _mock_editor(command):
context.editor_command = command
@ -604,14 +622,29 @@ def journal_exists(context, journal_name="default"):
@then('the config should have "{key}" set to "{value}"')
@then('the config for journal "{journal}" should have "{key}" set to "{value}"')
def config_var(context, key, value="", journal=None):
key_as_vec = key.split(".")
if "args" in context:
parsed = parse_args(context.args)
overrides = parsed.config_override
value = read_value_from_string(value or context.text or "")
configuration = load_config(context.config_path)
if journal:
configuration = configuration["journals"][journal]
assert key in configuration
assert configuration[key] == value
if overrides:
with patch.object(
jrnl.override, "_recursively_apply", wraps=_recursively_apply
) as spy_recurse:
configuration = apply_overrides(overrides, configuration)
runtime_cfg = spy_recurse.call_args_list[0][0][0]
else:
runtime_cfg = configuration
# extract the value of the desired key from the configuration after overrides have been applied
for k in key_as_vec:
runtime_cfg = runtime_cfg["%s" % k]
assert runtime_cfg == value
@then('the config for journal "{journal}" should not have "{key}" set')

77
features/steps/override.py Normal file
View file

@ -0,0 +1,77 @@
from jrnl.jrnl import run
from unittest import mock
# from __future__ import with_statement
from jrnl.args import parse_args
from behave import then
from features.steps.core import _mock_getpass, _mock_time_parse
@then("the editor {editor} should have been called")
@then("No editor should have been called")
def editor_override(context, editor=None):
def _mock_write_in_editor(config):
editor = config["editor"]
journal = "features/journals/journal.jrnl"
context.tmpfile = journal
print("%s has been launched" % editor)
return journal
if "password" in context:
password = context.password
else:
password = ""
# fmt: off
# see: https://github.com/psf/black/issues/664
with \
mock.patch("jrnl.jrnl._write_in_editor", side_effect=_mock_write_in_editor(context.jrnl_config)) as mock_write_in_editor, \
mock.patch("sys.stdin.isatty", return_value=True), \
mock.patch('getpass.getpass',side_effect=_mock_getpass(password)), \
mock.patch("jrnl.time.parse", side_effect = _mock_time_parse(context)), \
mock.patch("jrnl.config.get_config_path", side_effect=lambda: context.config_path), \
mock.patch("jrnl.install.get_config_path", side_effect=lambda: context.config_path) \
:
try :
parsed_args = parse_args(context.args)
run(parsed_args)
context.exit_status = 0
context.editor = mock_write_in_editor
expected_config = context.jrnl_config
expected_config['editor'] = '%s'%editor
expected_config['journal'] ='features/journals/journal.jrnl'
if editor is not None:
assert mock_write_in_editor.call_count == 1
assert mock_write_in_editor.call_args[0][0]['editor']==editor
else:
# Expect that editor is *never* called
mock_write_in_editor.assert_not_called()
except SystemExit as e:
context.exit_status = e.code
# fmt: on
@then("the stdin prompt should have been called")
def override_editor_to_use_stdin(context):
try:
with mock.patch(
"sys.stdin.read",
return_value="Zwei peanuts walk into a bar und one of zem was a-salted",
) as mock_stdin_read, mock.patch(
"jrnl.install.load_or_install_jrnl", return_value=context.jrnl_config
), mock.patch(
"jrnl.Journal.open_journal",
spec=False,
return_value="features/journals/journal.jrnl",
), mock.patch(
"getpass.getpass", side_effect=_mock_getpass("test")
):
parsed_args = parse_args(context.args)
run(parsed_args)
context.exit_status = 0
mock_stdin_read.assert_called_once()
except SystemExit as e:
context.exit_status = e.code

23
jrnl/args.py
View file

@ -314,6 +314,29 @@ def parse_args(args=[]):
help=argparse.SUPPRESS,
)
config_overrides = parser.add_argument_group(
"Config file override",
textwrap.dedent("Apply a one-off override of the config file option"),
)
config_overrides.add_argument(
"--config-override",
dest="config_override",
action="append",
type=str,
nargs=2,
default=[],
metavar="CONFIG_KV_PAIR",
help="""
Override configured key-value pair with CONFIG_KV_PAIR for this command invocation only.
Examples: \n
\t - Use a different editor for this jrnl entry, call: \n
\t jrnl --config-override editor: "nano" \n
\t - Override color selections\n
\t jrnl --config-override colors.body blue --config-override colors.title green
""",
)
# Handle '-123' as a shortcut for '-n 123'
num = re.compile(r"^-(\d+)$")
args = [num.sub(r"-n \1", arg) for arg in args]

26
jrnl/config.py
View file

@ -19,6 +19,32 @@ XDG_RESOURCE = "jrnl"
DEFAULT_JOURNAL_NAME = "journal.txt"
DEFAULT_JOURNAL_KEY = "default"
YAML_SEPARATOR = ": "
def make_yaml_valid_dict(input: list) -> dict:
"""
Convert a two-element list of configuration key-value pair into a flat dict.
The dict is created through the yaml loader, with the assumption that
"input[0]: input[1]" is valid yaml.
:param input: list of configuration keys in dot-notation and their respective values.
:type input: list
:return: A single level dict of the configuration keys in dot-notation and their respective desired values
:rtype: dict
"""
assert len(input) == 2
# yaml compatible strings are of the form Key:Value
yamlstr = YAML_SEPARATOR.join(input)
runtime_modifications = yaml.load(yamlstr, Loader=yaml.FullLoader)
return runtime_modifications
def save_config(config):
config["version"] = __version__

7
jrnl/jrnl.py
View file

@ -16,6 +16,7 @@ from .editor import get_text_from_editor
from .editor import get_text_from_stdin
from .exception import UserAbort
from . import time
from .override import apply_overrides
def run(args):
@ -37,6 +38,12 @@ def run(args):
try:
config = install.load_or_install_jrnl()
original_config = config.copy()
# Apply config overrides
overrides = args.config_override
if overrides:
config = apply_overrides(overrides, config)
args = get_journal_name(args, config)
config = scope_config(config, args.journal_name)
except UserAbort as err:

65
jrnl/override.py Normal file
View file

@ -0,0 +1,65 @@
from .config import update_config, make_yaml_valid_dict
# import logging
def apply_overrides(overrides: list, base_config: dict) -> dict:
"""Unpack CLI provided overrides into the configuration tree.
:param overrides: List of configuration key-value pairs collected from the CLI
:type overrides: list
:param base_config: Configuration Loaded from the saved YAML
:type base_config: dict
:return: Configuration to be used during runtime with the overrides applied
:rtype: dict
"""
cfg_with_overrides = base_config.copy()
for pairs in overrides:
pairs = make_yaml_valid_dict(pairs)
key_as_dots, override_value = _get_key_and_value_from_pair(pairs)
keys = _convert_dots_to_list(key_as_dots)
cfg_with_overrides = _recursively_apply(
cfg_with_overrides, keys, override_value
)
update_config(base_config, cfg_with_overrides, None)
return base_config
def _get_key_and_value_from_pair(pairs):
key_as_dots, override_value = list(pairs.items())[0]
return key_as_dots, override_value
def _convert_dots_to_list(key_as_dots):
keys = key_as_dots.split(".")
keys = [k for k in keys if k != ""] # remove empty elements
return keys
def _recursively_apply(tree: dict, nodes: list, override_value) -> dict:
"""Recurse through configuration and apply overrides at the leaf of the config tree
Credit to iJames on SO: https://stackoverflow.com/a/47276490 for algorithm
Args:
config (dict): Configuration to modify
nodes (list): Vector of override keys; the length of the vector indicates tree depth
override_value (str): Runtime override passed from the command-line
"""
key = nodes[0]
if len(nodes) == 1:
tree[key] = override_value
else:
next_key = nodes[1:]
next_node = _get_config_node(tree, key)
_recursively_apply(next_node, next_key, override_value)
return tree
def _get_config_node(config: dict, key: str):
if key in config:
pass
else:
config[key] = None
return config[key]

79
tests/test_override.py Normal file
View file

@ -0,0 +1,79 @@
import pytest
from jrnl.override import (
apply_overrides,
_recursively_apply,
_get_config_node,
_get_key_and_value_from_pair,
_convert_dots_to_list,
)
@pytest.fixture()
def minimal_config():
cfg = {
"colors": {"body": "red", "date": "green"},
"default": "/tmp/journal.jrnl",
"editor": "vim",
"journals": {"default": "/tmp/journals/journal.jrnl"},
}
return cfg
def test_apply_override(minimal_config):
overrides = [["editor", "nano"]]
apply_overrides(overrides, minimal_config)
assert minimal_config["editor"] == "nano"
def test_override_dot_notation(minimal_config):
overrides = [["colors.body", "blue"]]
cfg = apply_overrides(overrides=overrides, base_config=minimal_config)
assert cfg["colors"] == {"body": "blue", "date": "green"}
def test_multiple_overrides(minimal_config):
overrides = [
["colors.title", "magenta"],
["editor", "nano"],
["journals.burner", "/tmp/journals/burner.jrnl"],
] # as returned by parse_args, saved in parser.config_override
cfg = apply_overrides(overrides, minimal_config)
assert cfg["editor"] == "nano"
assert cfg["colors"]["title"] == "magenta"
assert "burner" in cfg["journals"]
assert cfg["journals"]["burner"] == "/tmp/journals/burner.jrnl"
def test_recursively_apply():
cfg = {"colors": {"body": "red", "title": "green"}}
cfg = _recursively_apply(cfg, ["colors", "body"], "blue")
assert cfg["colors"]["body"] == "blue"
def test_get_config_node(minimal_config):
assert len(minimal_config.keys()) == 4
assert _get_config_node(minimal_config, "editor") == "vim"
assert _get_config_node(minimal_config, "display_format") == None
def test_get_kv_from_pair():
pair = {"ab.cde": "fgh"}
k, v = _get_key_and_value_from_pair(pair)
assert k == "ab.cde"
assert v == "fgh"
class TestDotNotationToList:
def test_unpack_dots_to_list(self):
keys = "a.b.c.d.e.f"
keys_list = _convert_dots_to_list(keys)
assert len(keys_list) == 6
def test_sequential_delimiters(self):
k = "g.r..h.v"
k_l = _convert_dots_to_list(k)
assert len(k_l) == 4

57
tests/test_parse_args.py
View file

@ -3,6 +3,7 @@ import shlex
import pytest
from jrnl.args import parse_args
from jrnl.config import make_yaml_valid_dict
def cli_as_dict(str):
@ -35,6 +36,7 @@ def expected_args(**kwargs):
"strict": False,
"tags": False,
"text": [],
"config_override": [],
}
return {**default_args, **kwargs}
@ -205,6 +207,31 @@ def test_version_alone():
assert cli_as_dict("--version") == expected_args(preconfig_cmd=preconfig_version)
def test_editor_override():
parsed_args = cli_as_dict('--config-override editor "nano"')
assert parsed_args == expected_args(config_override=[["editor", "nano"]])
def test_color_override():
assert cli_as_dict("--config-override colors.body blue") == expected_args(
config_override=[["colors.body", "blue"]]
)
def test_multiple_overrides():
parsed_args = cli_as_dict(
'--config-override colors.title green --config-override editor "nano" --config-override journal.scratchpad "/tmp/scratchpad"'
)
assert parsed_args == expected_args(
config_override=[
["colors.title", "green"],
["editor", "nano"],
["journal.scratchpad", "/tmp/scratchpad"],
]
)
# @see https://github.com/jrnl-org/jrnl/issues/520
@pytest.mark.parametrize(
"cli",
@ -233,3 +260,33 @@ def test_and_ordering(cli):
def test_edit_ordering(cli):
result = expected_args(edit=True, text=["second", "@oldtag", "@newtag"])
assert cli_as_dict(cli) == result
class TestDeserialization:
@pytest.mark.parametrize(
"input_str",
[
["editor", "nano"],
["colors.title", "blue"],
["default", "/tmp/egg.txt"],
],
)
def test_deserialize_multiword_strings(self, input_str):
runtime_config = make_yaml_valid_dict(input_str)
assert runtime_config.__class__ == dict
assert input_str[0] in runtime_config.keys()
assert runtime_config[input_str[0]] == input_str[1]
def test_deserialize_multiple_datatypes(self):
cfg = make_yaml_valid_dict(["linewrap", "23"])
assert cfg["linewrap"] == 23
cfg = make_yaml_valid_dict(["encrypt", "false"])
assert cfg["encrypt"] == False
cfg = make_yaml_valid_dict(["editor", "vi -c startinsert"])
assert cfg["editor"] == "vi -c startinsert"
cfg = make_yaml_valid_dict(["highlight", "true"])
assert cfg["highlight"] == True