Allow runtime configuration overrides from the commandline (#1169)

Add --config-override feature

* add test and argument handler for runtime override of configurations.
* identify location to apply override in "main"
* update gitignore
* remove unneeded import
* add jrnl interface test for overriden configurations
* trivial whitespace change
* implement runtime override
* make format
* refactor override unittest
* clean up unused import
* start writing integration test
* add linewrap override scenario
* implement editor override step
* add dev dependencies on pytest -mock and -cov
* make format
* remove unused imports
* make format
* rename --override to --config-override
* move override implementation into own module
* begin TDD of dot notated overrides
* rewrite behavior scenario
* implement recursive config overrides
* clean up unittests
* iterate on behave step
* make format
* cleanup
* move override behave tests out of core
* refactor recursive code
* make format
* code cleanup
* remove unused import
* update test config
* rewrite test for better mock call expect
* make format
* binary search misbehaving windows test
* unittest multiple overrides
* uncomment dot notation unittest
* add multiple override scenario spec
* make format
* make format
* update unittests for new syntax
* update integ tests for new syntax
* update gitignore
* guard override application
* deserialize function as return type
* make format
* organize deserialization unittests
* better, more specific behave tests
* test different editor launch commands
* formatting
* handle datatypes in deserialization and update helptext
* stick to config convention in testbed
* update tests ith better verifications
* make format
* space
* review feedbac
* make format
* skip on win
* update deps
* update tests with better verifications
make format
space
review feedbac
* skip on win
* update deps
* refactor deserialization
organize test_parse_args
make format
* skip on win
* refactor deserialization
organize test_parse_args
make format
* update tests ith better verifications
* make format
* space
* make format
* document apply_overrides
* update gitignore
* document config-override enhancement
* Simplify config override syntax (#5)
* update tests and expected behavior
* clean up arg parsing tests
* update deserialization
* update deserialization
* config argparse action
* update override application logic
* update tests; delete unused imports
* override param must be list
* update docstring
* update test input to SUT
* update remaining override unittests
* make format
* forgot to update CLI syntax
* update documentation to sphinx style
* variable renames
* Lockfile merge (#7)
* Add brew and gitter badges to README
* Update changelog [ci skip]
* Make journal selection behavior more consistent when there's a colon with no date (#1164)
* Simplify config override syntax (#8)
* update tests and expected behavior
* clean up arg parsing tests
* update deserialization
* update deserialization
* config argparse action
* update override application logic
* update tests; delete unused imports
* override param must be list
* update docstring
* update test input to SUT
* update remaining override unittests
* make format
* forgot to update CLI syntax
* formatting
* Update pyproject.toml
* update lockfile to remove pytest-cov and pytest-mock deps
* update docs
* reuse existing mock; delete unneeded code
* move overrides earlier in the execution
use existing configs instead of custom
make format
clean up imports
* update for passworded access
context.parser -> parsed_args
* test that no editor is launched
* remove unnecessary mocks
* rename variable for intent
* reinstate getpass deletion
* update gitignore
* capture failure mode
* remove unneeded imports
* renamed variable
* delete redundant step
* comment on step
* clean up step behavior description
* [WIP] lock down journal access behavior
* skip -> wip
* correct command for overriding journal via dot keys
* update wip test for updating a "temp" journal and then reading baack its entries
* remove "mock" from poetry file
* make CI happy
* complex behavior sequence for default journal override
* separate out smaller pieces of logic
test that apply_overrides acts on base configuration and not the copy
* defer modification of loaded configuration to update_config
remove unused fixtures
delete complicated UT since behavior is covered in overrides.feature integ test
delete redundant UT
* Update .gitignore
* remove skip_win
* forward override unpacking to yaml library
* merge config override step with existing config_var step in core
delete config_override step
unify step description syntax
* delete unused and redundant code
* rebases are hard
* remove wip tag from test
* remove skipped tests for windows
* Address code review
yield -> return
remove needless copy
adjust spacing
re-inline args return
reset packaging info to e6c0a16342
revert package version for this PR
* consolidate imports
* Defer config_override unpacking to dict *after* base config is loaded
store cli overrides without unpacking just yet
move deserialize_config_args to config module
delete custom Action class for config operations
apply [k,v] -> {k, v} for each override
update test data
update import
* rename deserialize_config_args to better express intent
make format

jrnl/args.py

@@ -314,6 +314,29 @@ def parse_args(args=[]):
         help=argparse.SUPPRESS,
     )
 
+    config_overrides = parser.add_argument_group(
+        "Config file override",
+        textwrap.dedent("Apply a one-off override of the config file option"),
+    )
+    config_overrides.add_argument(
+        "--config-override",
+        dest="config_override",
+        action="append",
+        type=str,
+        nargs=2,
+        default=[],
+        metavar="CONFIG_KV_PAIR",
+        help="""
+        Override configured key-value pair with CONFIG_KV_PAIR for this command invocation only.
+
+        Examples: \n
+        \t - Use a different editor for this jrnl entry, call: \n
+            \t jrnl --config-override editor: "nano" \n
+        \t - Override color selections\n
+           \t jrnl --config-override colors.body blue --config-override colors.title green
+        """,
+    )
+
     # Handle '-123' as a shortcut for '-n 123'
     num = re.compile(r"^-(\d+)$")
     args = [num.sub(r"-n \1", arg) for arg in args]

jrnl/config.py

@@ -19,6 +19,32 @@ XDG_RESOURCE = "jrnl"
 DEFAULT_JOURNAL_NAME = "journal.txt"
 DEFAULT_JOURNAL_KEY = "default"
 
+YAML_SEPARATOR = ": "
+
+
+def make_yaml_valid_dict(input: list) -> dict:
+
+    """
+
+    Convert a two-element list of configuration key-value pair into a flat dict.
+
+    The dict is created through the yaml loader, with the assumption that
+    "input[0]: input[1]" is valid yaml.
+
+    :param input: list of configuration keys in dot-notation and their respective values.
+    :type input: list
+    :return: A single level dict of the configuration keys in dot-notation and their respective desired values
+    :rtype: dict
+    """
+
+    assert len(input) == 2
+
+    # yaml compatible strings are of the form Key:Value
+    yamlstr = YAML_SEPARATOR.join(input)
+    runtime_modifications = yaml.load(yamlstr, Loader=yaml.FullLoader)
+
+    return runtime_modifications
+
 
 def save_config(config):
     config["version"] = __version__

jrnl/jrnl.py

@@ -16,6 +16,7 @@ from .editor import get_text_from_editor
 from .editor import get_text_from_stdin
 from .exception import UserAbort
 from . import time
+from .override import apply_overrides
 
 
 def run(args):
@@ -37,6 +38,12 @@ def run(args):
     try:
         config = install.load_or_install_jrnl()
         original_config = config.copy()
+
+        # Apply config overrides
+        overrides = args.config_override
+        if overrides:
+            config = apply_overrides(overrides, config)
+
         args = get_journal_name(args, config)
         config = scope_config(config, args.journal_name)
     except UserAbort as err:

jrnl/override.py

@@ -0,0 +1,65 @@
+from .config import update_config, make_yaml_valid_dict
+
+# import logging
+def apply_overrides(overrides: list, base_config: dict) -> dict:
+    """Unpack CLI provided overrides into the configuration tree.
+
+    :param overrides: List of configuration key-value pairs collected from the CLI
+    :type overrides: list
+    :param base_config: Configuration Loaded from the saved YAML
+    :type base_config: dict
+    :return: Configuration to be used during runtime with the overrides applied
+    :rtype: dict
+    """
+    cfg_with_overrides = base_config.copy()
+    for pairs in overrides:
+
+        pairs = make_yaml_valid_dict(pairs)
+        key_as_dots, override_value = _get_key_and_value_from_pair(pairs)
+        keys = _convert_dots_to_list(key_as_dots)
+        cfg_with_overrides = _recursively_apply(
+            cfg_with_overrides, keys, override_value
+        )
+
+    update_config(base_config, cfg_with_overrides, None)
+    return base_config
+
+
+def _get_key_and_value_from_pair(pairs):
+    key_as_dots, override_value = list(pairs.items())[0]
+    return key_as_dots, override_value
+
+
+def _convert_dots_to_list(key_as_dots):
+    keys = key_as_dots.split(".")
+    keys = [k for k in keys if k != ""]  # remove empty elements
+    return keys
+
+
+def _recursively_apply(tree: dict, nodes: list, override_value) -> dict:
+    """Recurse through configuration and apply overrides at the leaf of the config tree
+
+    Credit to iJames on SO: https://stackoverflow.com/a/47276490 for algorithm
+
+    Args:
+        config (dict): Configuration to modify
+        nodes (list): Vector of override keys; the length of the vector indicates tree depth
+        override_value (str): Runtime override passed from the command-line
+    """
+    key = nodes[0]
+    if len(nodes) == 1:
+        tree[key] = override_value
+    else:
+        next_key = nodes[1:]
+        next_node = _get_config_node(tree, key)
+        _recursively_apply(next_node, next_key, override_value)
+
+    return tree
+
+
+def _get_config_node(config: dict, key: str):
+    if key in config:
+        pass
+    else:
+        config[key] = None
+    return config[key]