fz0x1/pricehist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: fz0x1:master
Branches Tags
fz0x1:master
fz0x1:1.4.12
fz0x1:1.4.11
fz0x1:1.4.10
fz0x1:1.4.9
fz0x1:1.4.8
fz0x1:1.4.7
fz0x1:1.4.6
fz0x1:1.4.5
fz0x1:1.4.4
fz0x1:1.4.3
fz0x1:1.4.2
fz0x1:1.4.1
fz0x1:1.4.0
fz0x1:1.3.0
fz0x1:1.2.4
fz0x1:1.2.3
fz0x1:1.2.2
fz0x1:1.2.1
fz0x1:1.2.0
fz0x1:1.1.0
fz0x1:1.0.1
fz0x1:1.0.0
fz0x1:0.1.7
fz0x1:0.1.6
fz0x1:0.1.5
fz0x1:0.1.4
fz0x1:0.1.3
fz0x1:0.1.2
fz0x1:0.1.1
fz0x1:0.1.0
..
compare: fz0x1:0.1.7
Branches Tags
fz0x1:master
fz0x1:1.4.12
fz0x1:1.4.11
fz0x1:1.4.10
fz0x1:1.4.9
fz0x1:1.4.8
fz0x1:1.4.7
fz0x1:1.4.6
fz0x1:1.4.5
fz0x1:1.4.4
fz0x1:1.4.3
fz0x1:1.4.2
fz0x1:1.4.1
fz0x1:1.4.0
fz0x1:1.3.0
fz0x1:1.2.4
fz0x1:1.2.3
fz0x1:1.2.2
fz0x1:1.2.1
fz0x1:1.2.0
fz0x1:1.1.0
fz0x1:1.0.1
fz0x1:1.0.0
fz0x1:0.1.7
fz0x1:0.1.6
fz0x1:0.1.5
fz0x1:0.1.4
fz0x1:0.1.3
fz0x1:0.1.2
fz0x1:0.1.1
fz0x1:0.1.0

No commits in common. "master" and "0.1.7" have entirely different histories.
master ... 0.1.7

81 changed files with 2230 additions and 5426 deletions
Show stats Expand all files Collapse all files

1
.gitignore vendored
View file

@ -3,4 +3,3 @@
dist/
.coverage
htmlcov/
.tox/

14
.gitlab-ci.yml
View file

@ -14,20 +14,10 @@ before_script:
- pip install poetry
- poetry install
pre-commit:
script:
- make pre-commit
test:
script:
- poetry run isort src tests --check
- poetry run black src tests --check
- poetry run pytest
test-live:
script:
- tests/live.sh
coverage:
script:
- poetry run coverage run --source=pricehist -m pytest
- poetry run coverage report
coverage: '/^TOTAL.+?(\d+\%)$/'

36
Makefile
View file

@ -2,47 +2,23 @@
help: ## List make targets
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.PHONY: format
lint: ## Lint source code
poetry run flake8
format: ## Format source code
poetry run isort .
poetry run black .
.PHONY: lint
lint: ## Lint source code
poetry run flake8 src tests
.PHONY: test
test: ## Run tests
poetry run pytest --color=yes
test: ## Run non-live tests
poetry run pytest -m "not live" --color=yes
.PHONY: test-live
test-live: ## Run live tests
tests/live.sh
poetry run pytest -m live --color=yes
.PHONY: coverage
coverage: ## Generate and open coverage report
poetry run coverage run --source=pricehist -m pytest
poetry run coverage html
xdg-open htmlcov/index.html
.PHONY: install-pre-commit-hook
install-pre-commit-hook: ## Install the git pre-commit hook
echo -e "#!/bin/bash\nmake pre-commit" > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
.PHONY: pre-commit
pre-commit: ## Checks to run before each commit
poetry run isort src tests --check
poetry run black src tests --check
poetry run flake8 src tests
.PHONY: tox
tox: ## Run tests via tox
poetry run tox
.PHONY: fetch-iso-data
fetch-iso-data: ## Fetch the latest copy of the ISO 4217 currency data
wget -O src/pricehist/resources/list-one.xml \
https://www.six-group.com/dam/download/financial-information/data-center/iso-currrency/lists/list-one.xml
wget -O src/pricehist/resources/list-three.xml \
https://www.six-group.com/dam/download/financial-information/data-center/iso-currrency/lists/list-three.xml

425
README.md
View file

@ -8,23 +8,19 @@ support for multiple data sources and output formats.
[![PyPI version](https://badge.fury.io/py/pricehist.svg)](https://badge.fury.io/py/pricehist)
[![Downloads](https://pepy.tech/badge/pricehist)](https://pepy.tech/project/pricehist)
[![License](https://img.shields.io/pypi/l/pricehist)](https://gitlab.com/chrisberkhout/pricehist/-/blob/master/LICENSE)
[![Code style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgitlab.com%2Fchrisberkhout%2Fpricehist&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=hits&edge_flat=false)](https://hits.seeyoufarm.com)
## Installation
Install via pip or
[pipx](https://pypa.github.io/pipx/).
Install via [pip](https://pip.pypa.io/en/stable/) or
[pipx](https://pypa.github.io/pipx/):
```
```bash
pipx install pricehist
```
## Sources
- **`alphavantage`**: [Alpha Vantage](https://www.alphavantage.co/)
- **`bankofcanada`**: [Bank of Canada daily exchange rates](https://www.bankofcanada.ca/valet/docs)
- **`coinbasepro`**: [Coinbase Pro](https://pro.coinbase.com/)
- **`coindesk`**: [CoinDesk Bitcoin Price Index](https://www.coindesk.com/coindesk-api)
- **`coinmarketcap`**: [CoinMarketCap](https://coinmarketcap.com/)
- **`ecb`**: [European Central Bank Euro foreign exchange reference rates](https://www.ecb.europa.eu/stats/exchange/eurofxref/html/index.en.html)
@ -34,69 +30,43 @@ pipx install pricehist
- **`beancount`**: [Beancount](http://furius.ca/beancount/)
- **`csv`**: [Comma-separated values](https://en.wikipedia.org/wiki/Comma-separated_values)
- **`json`**: [JSON](https://en.wikipedia.org/wiki/JSON)
- **`jsonl`**: [JSON lines](https://en.wikipedia.org/wiki/JSON_streaming)
- **`gnucash-sql`**: [GnuCash](https://www.gnucash.org/) SQL
- **`ledger`**: [Ledger](https://www.ledger-cli.org/) and [hledger](https://hledger.org/)
## Reactions
## Examples
> This is my new favourite price fetcher, by far.
> -- _Simon Michael, creator of [hledger](https://hledger.org/) ([ref](https://groups.google.com/g/hledger/c/SCLbNiKl9D8/m/0ReYmDppAAAJ))_
Show usage information:
> This is great!
> -- _Martin Blais, creator of [Beancount](https://beancount.github.io/) ([ref](https://groups.google.com/g/beancount/c/cCJc9OhIlNg/m/QGRvNowcAwAJ))_
## How to
### Fetch prices
Fetch prices by choosing a source, a pair and, optionally, a time interval.
```
pricehist fetch ecb EUR/AUD -s 2021-01-04 -e 2021-01-08
```bash
pricehist -h
```
```
date,base,quote,amount,source,type
2021-01-04,EUR,AUD,1.5928,ecb,reference
2021-01-05,EUR,AUD,1.5927,ecb,reference
2021-01-06,EUR,AUD,1.5824,ecb,reference
2021-01-07,EUR,AUD,1.5836,ecb,reference
2021-01-08,EUR,AUD,1.5758,ecb,reference
usage: pricehist [-h] [--version] [-vvv] {sources,source,fetch} ...
Fetch historical price data
optional arguments:
-h, --help show this help message and exit
--version show version information
-vvv, --verbose show all log messages
commands:
{sources,source,fetch}
sources list sources
source show source details
fetch fetch prices
```
The default output format is CSV, which is suitable for use in spreadsheets and
with other tools. For example, you can generate a price chart from the command
line as follows (or using [an alias](https://gitlab.com/-/snippets/2163031)).
```
pricehist fetch coindesk BTC/USD -s 2021-01-01 | \
sed 1d | \
cut -d, -f1,4 | \
gnuplot -p -e '
set datafile separator ",";
set xdata time;
set timefmt "%Y-%m-%d";
set format x "%b\n%Y";
plot "/dev/stdin" using 1:2 with lines title "BTC/USD"
'
```
![BTC/USD prices](https://gitlab.com/chrisberkhout/pricehist/-/raw/master/example-gnuplot.png)
### Show usage information
Add `-h` to any command to see usage information.
Show usage information for the `fetch` command:
```
pricehist fetch -h
```
```
usage: pricehist fetch SOURCE PAIR [-h] [-vvv] [-t TYPE] [-s DATE | -sx DATE] [-e DATE | -ex DATE]
[-o beancount|csv|json|jsonl|gnucash-sql|ledger] [--invert] [--quantize INT]
[-o beancount|csv|gnucash-sql|ledger] [--invert] [--quantize INT]
[--fmt-base SYM] [--fmt-quote SYM] [--fmt-time TIME] [--fmt-decimal CHAR] [--fmt-thousands CHAR]
[--fmt-symbol rightspace|right|leftspace|left] [--fmt-datesep CHAR]
[--fmt-csvdelim CHAR] [--fmt-jsonnums]
[--fmt-symbol rightspace|right|leftspace|left] [--fmt-datesep CHAR] [--fmt-csvdelim CHAR]
positional arguments:
SOURCE the source identifier
@ -121,16 +91,30 @@ optional arguments:
--fmt-symbol LOCATION commodity symbol placement in output (default: rightspace)
--fmt-datesep CHAR date separator in output (default: '-')
--fmt-csvdelim CHAR field delimiter for CSV output (default: ',')
--fmt-jsonnums numbers not strings for JSON output (default: False)
```
### Choose and customize the output format
As the output format you can choose one of `beancount`, `csv`, `json`, `jsonl`,
`ledger` or `gnucash-sql`.
Fetch prices after 2021-01-04, ending 2021-01-15, as CSV:
```bash
pricehist fetch ecb EUR/AUD -sx 2021-01-04 -e 2021-01-15 -o csv
```
pricehist fetch ecb EUR/AUD -s 2021-01-04 -e 2021-01-08 -o ledger
```
date,base,quote,amount,source,type
2021-01-05,EUR,AUD,1.5927,ecb,reference
2021-01-06,EUR,AUD,1.5824,ecb,reference
2021-01-07,EUR,AUD,1.5836,ecb,reference
2021-01-08,EUR,AUD,1.5758,ecb,reference
2021-01-11,EUR,AUD,1.5783,ecb,reference
2021-01-12,EUR,AUD,1.5742,ecb,reference
2021-01-13,EUR,AUD,1.5734,ecb,reference
2021-01-14,EUR,AUD,1.5642,ecb,reference
2021-01-15,EUR,AUD,1.568,ecb,reference
```
In Ledger format:
```bash
pricehist fetch ecb EUR/AUD -s 2021-01-01 -o ledger | head
```
```
P 2021-01-04 00:00:00 EUR 1.5928 AUD
@ -138,313 +122,27 @@ P 2021-01-05 00:00:00 EUR 1.5927 AUD
P 2021-01-06 00:00:00 EUR 1.5824 AUD
P 2021-01-07 00:00:00 EUR 1.5836 AUD
P 2021-01-08 00:00:00 EUR 1.5758 AUD
P 2021-01-11 00:00:00 EUR 1.5783 AUD
P 2021-01-12 00:00:00 EUR 1.5742 AUD
P 2021-01-13 00:00:00 EUR 1.5734 AUD
P 2021-01-14 00:00:00 EUR 1.5642 AUD
P 2021-01-15 00:00:00 EUR 1.568 AUD
```
Formatting options let you control certain details of the output.
Generate SQL for a GnuCash database and apply it immediately:
```
pricehist fetch ecb EUR/AUD -s 2021-01-04 -e 2021-01-08 -o ledger \
--fmt-time '' --fmt-datesep / --fmt-base € --fmt-quote $ --fmt-symbol left
```
```
P 2021/01/04 € $1.5928
P 2021/01/05 € $1.5927
P 2021/01/06 € $1.5824
P 2021/01/07 € $1.5836
P 2021/01/08 € $1.5758
```
### Fetch new prices only
You can update an existing file without refetching the prices you already have.
First find the date of the last price, then fetch from there, drop the header
line if present and append the rest to the existing file.
```
last=$(tail -1 prices-eur-usd.csv | cut -d, -f1)
pricehist fetch ecb EUR/USD -sx $last -o csv | sed 1d >> prices-eur-usd.csv
```
### Load prices into GnuCash
You can generate SQL for a GnuCash database and apply it immediately with one
of the following commands.
```
```bash
pricehist fetch ecb EUR/AUD -s 2021-01-01 -o gnucash-sql | sqlite3 Accounts.gnucash
pricehist fetch ecb EUR/AUD -s 2021-01-01 -o gnucash-sql | mysql -u username -p -D databasename
pricehist fetch ecb EUR/AUD -s 2021-01-01 -o gnucash-sql | psql -U username -d databasename -v ON_ERROR_STOP=1
```
Beware that the GnuCash project itself does not support integration at the
database level, so there is a risk that the SQL generated by `pricehist` will
be ineffective or even damaging for some version of GnuCash. In practice, this
strategy has been used successfully by other projects. Reading the SQL and
keeping regular database backups is recommended.
## Design choices
The GnuCash database must already contain commodities with mnemonics matching
the base and quote of new prices, otherwise the SQL will fail without making
changes.
Each price entry is given a GUID based on its content (date, base, quote,
source, type and amount) and existing GUIDs are skipped in the final insert, so
you can apply identical or overlapping SQL files multiple times without
creating duplicate entries in the database.
### Show source information
The `source` command shows information about a source.
```
pricehist source alphavantage
```
```
ID : alphavantage
Name : Alpha Vantage
Description : Provider of market data for stocks, forex and cryptocurrencies
URL : https://www.alphavantage.co/
Start : 1995-01-01
Types : close, open, high, low, adjclose, mid
Notes : Alpha Vantage has data on...
```
Available symbols can be listed for most sources, either as full pairs or as
separate base and quote symbols that will work in certain combinations.
```
pricehist source ecb --symbols
```
```
EUR/AUD Euro against Australian Dollar
EUR/BGN Euro against Bulgarian Lev
EUR/BRL Euro against Brazilian Real
EUR/CAD Euro against Canadian Dollar
EUR/CHF Euro against Swiss Franc
...
```
It may also be possible to search for symbols.
```
pricehist source alphavantage --search Tesla
```
```
TL0.DEX Tesla, Equity, XETRA, EUR
TL0.FRK Tesla, Equity, Frankfurt, EUR
TSLA34.SAO Tesla, Equity, Brazil/Sao Paolo, BRL
TSLA Tesla Inc, Equity, United States, USD
TXLZF Tesla Exploration Ltd, Equity, United States, USD
```
### Inspect source interactions
You can see extra information by adding the verbose option (`--verbose` or
`-vvv`), including `curl` commands that reproduce each request to a source.
```
pricehist fetch coindesk BTC/USD -s 2021-01-01 -e 2021-01-05 -vvv
```
```
DEBUG Began pricehist run at 2021-08-12 14:38:26.630357.
DEBUG Starting new HTTPS connection (1): api.coindesk.com:443
DEBUG https://api.coindesk.com:443 "GET /v1/bpi/historical/close.json?currency=USD&start=2021-01-01&end=2021-01-05 HTTP/1.1" 200 319
DEBUG curl -X GET -H 'Accept: */*' -H 'Accept-Encoding: gzip, deflate' -H 'Connection: keep-alive' -H 'User-Agent: python-requests/2.25.1' --compressed 'https://api.coindesk.com/v1/bpi/historical/close.json?currency=USD&start=2021-01-01&end=2021-01-05'
DEBUG Available data covers the interval [2021-01-01--2021-01-05], as requested.
date,base,quote,amount,source,type
2021-01-01,BTC,USD,29391.775,coindesk,close
2021-01-02,BTC,USD,32198.48,coindesk,close
2021-01-03,BTC,USD,33033.62,coindesk,close
2021-01-04,BTC,USD,32017.565,coindesk,close
2021-01-05,BTC,USD,34035.0067,coindesk,close
DEBUG Ended pricehist run at 2021-08-12 14:38:26.709428.
```
Running a logged `curl` command shows exactly what data is returned by the
source.
```
pricehist fetch coindesk BTC/USD -s 2021-01-01 -e 2021-01-05 -vvv 2>&1 \
| grep '^DEBUG curl' | sed 's/^DEBUG //' | bash | jq .
```
```json
{
"bpi": {
"2021-01-01": 29391.775,
"2021-01-02": 32198.48,
"2021-01-03": 33033.62,
"2021-01-04": 32017.565,
"2021-01-05": 34035.0067
},
"disclaimer": "This data was produced from the CoinDesk Bitcoin Price Index. BPI value data returned as USD.",
"time": {
"updated": "Jan 6, 2021 00:03:00 UTC",
"updatedISO": "2021-01-06T00:03:00+00:00"
}
}
```
### Use via `bean-price`
Beancount users may wish to use `pricehist` sources via `bean-price`. To do so,
ensure the `pricehist` package is installed in an accessible location.
You can fetch the latest price directly from the command line.
```
bean-price -e "USD:pricehist.beanprice.coindesk/BTC:USD"
```
```
2021-08-18 price BTC:USD 44725.12 USD
```
You can fetch a series of prices by providing a Beancount file as input.
```
; input.beancount
2021-08-14 commodity BTC
price: "USD:pricehist.beanprice.coindesk/BTC:USD:close"
```
```
bean-price input.beancount --update --update-rate daily --inactive --clear-cache
```
```
2021-08-14 price BTC 47098.2633 USD
2021-08-15 price BTC 47018.9017 USD
2021-08-16 price BTC 45927.405 USD
2021-08-17 price BTC 44686.3333 USD
2021-08-18 price BTC 44725.12 USD
```
Adding `-v` will print progress information, `-vv` will print debug information,
including that from `pricehist`.
A source map specification for `bean-price` has the form
`<currency>:<module>/[^]<ticker>`. Additional `<module>/[^]<ticker>` parts can
be appended, separated by commas.
The module name will be of the form `pricehist.beanprice.<source_id>`.
The ticker symbol will be of the form `BASE:QUOTE:TYPE`.
Any non-alphanumeric characters except the equals sign (`=`), hyphen (`-`),
period (`.`), or parentheses (`(` or `)`) are special characters that need to
be encoded as their a two-digit hexadecimal code prefixed with an underscore,
because `bean-price` ticker symbols don't allow all the characters used by
`pricehist` pairs.
[This page](https://replit.com/@chrisberkhout/bpticker) will do it for you.
For example, the Yahoo! Finance symbol for the Dow Jones Industrial Average is
`^DJI`, and would have the source map specification
`USD:pricehist.beanprice.yahoo/_5eDJI`, or for the daily high price
`USD:pricehist.beanprice.yahoo/_5eDJI::high`.
### Use as a library
You may find `pricehist`'s source classes useful in your own scripts.
```
$ python
Python 3.9.6 (default, Jun 30 2021, 10:22:16)
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pricehist.series import Series
>>> from pricehist.sources.ecb import ECB
>>> series = ECB().fetch(Series("EUR", "AUD", "reference", "2021-01-04", "2021-01-08"))
>>> series.prices
[Price(date='2021-01-04', amount=Decimal('1.5928')), Price(date='2021-01-05', amount=Decimal('1.5927')), Price(date='2021-01-06', amount=Decimal('1.5824')), Price(date='2021-01-07', amount=Decimal('1.5836')), Price(date='2021-01-08', amount=Decimal('1.5758'))]
```
A subclass of `pricehist.exceptions.SourceError` will be raised for any error.
### Contribute
Contributions are welcome! If you discover a bug or want to work on a
non-trivial change, please open a
[GitLab issue](https://gitlab.com/chrisberkhout/pricehist/-/issues)
to discuss it.
Run `make install-pre-commit-hook` set up local pre-commit checks.
Set up your editor to run
[isort](https://pycqa.github.io/isort/),
[Black](https://black.readthedocs.io/en/stable/) and
[Flake8](https://flake8.pycqa.org/en/latest/),
or run them manually via `make format lint`.
## Terminology
A **source** is an upstream service that can provide a series of prices.
Each **series** of prices is for one pair and price type.
The [**pair**](https://en.wikipedia.org/wiki/Currency_pair) is made up of a
base and a quote, each given as a symbol. Sometimes you will give the base
only, and the quote will be determined with information from the source. The
available pairs, the symbols used in them and the available price types all
depend on the particular source used.
The **base** is the currency or commodity being valued. Each price expresses
the value of one unit of the base.
The **quote** is the unit used to express the value of the base.
A **symbol** is a code or abbreviation for a currency or commodity.
The **prices** in a series each have a date and an amount.
The **amount** is the number of units of the quote that are equal to one unit
of the base.
Consider the following command.
```
pricehist fetch coindesk BTC/USD --type close
```
- **`coindesk`** is the ID of the CoinDesk Bitcoin Price Index source.
- **`BTC`** is the symbol for Bitcoin, used here as the base.
- **`USD`** is the symbol for the United States Dollar, used here as the quote.
- **`BTC/USD`** is the pair Bitcoin against United States Dollar.
- **`close`** is the price type for the last price of each day.
A BTC/USD price of the amount 29,391.775 can be written as
"BTC/USD = 29391.775" or "BTC 29391.775 USD", and means that one Bitcoin is
worth 29,391.775 United States Dollars.
## Initial design choices
To keep things simple, `pricehist` provides only univariate time series of
daily historical prices. It doesn't provide other types of market, financial or
economic data, real-time prices, or other temporal resolutions. Multiple or
multivariate series require multiple invocations.
## Potential features
In the future, `pricehist` may be extended to cover some of the following
features:
- **Time of day**: Sources sometimes provide specific times for each day's
high/low prices and these could be preserved for output. This would require
changes to how dates are handled internally, clarification of time zone
handling and extension of the time formatting option.
- **Alternate resolutions**: Some sources can provide higher or lower
resolution data, such as hourly or weekly. These could be supported where
available. For other cases an option could be provided for downsampling data
before output.
- **Real-time prices**: These generally come from different source endpoints
than the historical data. Real-time prices will usually have a different
price type, such as `last`, `bid` or `ask`. Support for real-time prices
would allow adding sources that don't provide historical data. Start and end
times are irrelevant when requesting real-time prices. A "follow" option
could continuously poll for new prices.
- **Related non-price data**: Trading volume, spreads, split and dividend
events and other related data could be supported. The base/quote/type model
used for prices would work for some of this. Other things may require
extending the model.
- **Multivariate series**: Would allow, for example, fetching
high/low/open/close prices in a single invocation.
- **`format` command**: A command for rewriting existing CSV data into one of
the other output formats.
To keep things simple, at least for now, `pricehist` provides only univariate
time series of daily historical prices. It doesn't provide other types of
market, financial or economic data, real-time prices, or other temporal
resolutions. Multiple or multivariate series require multiple invocations.
## Alternatives
@ -459,3 +157,14 @@ method for fetching historical prices.
The GnuCash wiki documents [wrapper scripts](https://wiki.gnucash.org/wiki/Stocks/get_prices)
for the [Finance::QuoteHist](https://metacpan.org/pod/Finance::QuoteHist) Perl
module.
Other projects with related goals include:
* [`hledger-stockquotes`](https://github.com/prikhi/hledger-stockquotes):
A CLI addon for hledger that reads a journal file and pulls the historical prices for commodities.
* [`ledger_get_prices`](https://github.com/nathankot/ledger-get-prices):
Uses Yahoo Finance to generate a price database based on your current Ledger commodities and time period.
* [LedgerStockUpdate](https://github.com/adchari/LedgerStockUpdate):
Locates any stocks you have in your ledger-cli file, then generates a price database of those stocks.
* [`market-prices`](https://github.com/barrucadu/hledger-scripts#market-prices):
Downloads market values of commodities from a few different sources.

BIN
example-gnuplot.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 32 KiB

1228
poetry.lock generated
View file

File diff suppressed because it is too large Load diff

18
pyproject.toml
View file

@ -1,6 +1,6 @@
[tool.poetry]
name = "pricehist"
version = "1.4.12"
version = "0.1.7"
description = "Fetch and format historical price data"
authors = ["Chris Berkhout <chris@chrisberkhout.com>"]
license = "MIT"
@ -10,25 +10,23 @@ homepage = "https://gitlab.com/chrisberkhout/pricehist"
repository = "https://gitlab.com/chrisberkhout/pricehist"
include = [
"LICENSE",
"example-gnuplot.png",
]
[tool.poetry.dependencies]
python = "^3.8.1"
python = "^3.9"
requests = "^2.25.1"
lxml = "^5.1.0"
lxml = "^4.6.2"
cssselect = "^1.1.0"
curlify = "^2.2.1"
[tool.poetry.dev-dependencies]
pytest = "^8.3.2"
black = "^22.10.0"
flake8 = "^7.1.0"
pytest = "^6.2.2"
black = "^20.8b1"
flake8 = "^3.9.1"
isort = "^5.8.0"
responses = "^0.13.3"
coverage = "^5.5"
pytest-mock = "^3.6.1"
tox = "^3.24.3"
[build-system]
requires = ["poetry-core>=1.0.0"]
@ -42,4 +40,6 @@ profile = "black"
multi_line_output = 3
[tool.pytest.ini_options]
markers = []
markers = [
"live: makes a live request to a source"
]

2
src/pricehist/__init__.py
View file

@ -1 +1 @@
__version__ = "1.4.12"
__version__ = "0.1.7"

77
src/pricehist/beanprice/__init__.py
View file

@ -1,77 +0,0 @@
import re
from datetime import date, datetime, timedelta, timezone
from decimal import Decimal
from typing import List, NamedTuple, Optional
from pricehist import exceptions
from pricehist.series import Series
SourcePrice = NamedTuple(
"SourcePrice",
[
("price", Decimal),
("time", Optional[datetime]),
("quote_currency", Optional[str]),
],
)
def source(pricehist_source):
class Source:
def get_latest_price(self, ticker: str) -> Optional[SourcePrice]:
time_end = datetime.combine(date.today(), datetime.min.time())
time_begin = time_end - timedelta(days=7)
prices = self.get_prices_series(ticker, time_begin, time_end)
if prices:
return prices[-1]
else:
return None
def get_historical_price(
self, ticker: str, time: datetime
) -> Optional[SourcePrice]:
prices = self.get_prices_series(ticker, time, time)
if prices:
return prices[-1]
else:
return None
def get_prices_series(
self,
ticker: str,
time_begin: datetime,
time_end: datetime,
) -> Optional[List[SourcePrice]]:
base, quote, type = self._decode(ticker)
start = time_begin.date().isoformat()
end = time_end.date().isoformat()
local_tz = datetime.now(timezone.utc).astimezone().tzinfo
user_tz = time_begin.tzinfo or local_tz
try:
series = pricehist_source.fetch(Series(base, quote, type, start, end))
except exceptions.SourceError:
return None
return [
SourcePrice(
price.amount,
datetime.fromisoformat(price.date).replace(tzinfo=user_tz),
series.quote,
)
for price in series.prices
]
def _decode(self, ticker):
# https://github.com/beancount/beanprice/blob/b05203/beanprice/price.py#L166
parts = [
re.sub(r"_[0-9a-fA-F]{2}", lambda m: chr(int(m.group(0)[1:], 16)), part)
for part in ticker.split(":")
]
base, quote, candidate_type = (parts + [""] * 3)[0:3]
type = candidate_type or pricehist_source.types()[0]
return (base, quote, type)
return Source

4
src/pricehist/beanprice/alphavantage.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.alphavantage import AlphaVantage
Source = beanprice.source(AlphaVantage())

4
src/pricehist/beanprice/bankofcanada.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.bankofcanada import BankOfCanada
Source = beanprice.source(BankOfCanada())

4
src/pricehist/beanprice/coinbasepro.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.coinbasepro import CoinbasePro
Source = beanprice.source(CoinbasePro())

4
src/pricehist/beanprice/coindesk.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.coindesk import CoinDesk
Source = beanprice.source(CoinDesk())

4
src/pricehist/beanprice/coinmarketcap.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.coinmarketcap import CoinMarketCap
Source = beanprice.source(CoinMarketCap())

4
src/pricehist/beanprice/ecb.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.ecb import ECB
Source = beanprice.source(ECB())

4
src/pricehist/beanprice/exchangeratehost.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.exchangeratehost import ExchangeRateHost
Source = beanprice.source(ExchangeRateHost())

4
src/pricehist/beanprice/yahoo.py
View file

@ -1,4 +0,0 @@
from pricehist import beanprice
from pricehist.sources.yahoo import Yahoo
Source = beanprice.source(Yahoo())

147
src/pricehist/cli.py
View file

@ -10,13 +10,13 @@ from pricehist.format import Format
from pricehist.series import Series
def cli(argv=sys.argv):
def cli(args=None, output_file=sys.stdout):
start_time = datetime.now()
logger.init()
parser = build_parser()
args = parser.parse_args(argv[1:])
args = parser.parse_args()
if args.verbose:
logger.show_debug()
@ -25,105 +25,97 @@ def cli(argv=sys.argv):
try:
if args.version:
print(f"pricehist {__version__}")
print(f"pricehist {__version__}", file=output_file)
elif args.command == "sources":
result = sources.formatted()
print(result)
print(result, file=output_file)
elif args.command == "source" and args.symbols:
result = sources.by_id[args.source].format_symbols()
print(result, end="")
print(result, file=output_file, end="")
elif args.command == "source" and args.search:
result = sources.by_id[args.source].format_search(args.search)
print(result, end="")
print(result, file=output_file, end="")
elif args.command == "source":
total_width = shutil.get_terminal_size().columns
result = sources.by_id[args.source].format_info(total_width)
print(result)
print(result, file=output_file)
elif args.command == "fetch":
source = sources.by_id[args.source]
output = outputs.by_type[args.output]
if args.end < args.start:
parser.error(
f"The end date '{args.end}' preceeds the start date '{args.start}'!"
)
if args.type not in source.types():
parser.error(
f"The requested price type '{args.type}' is not "
f"recognized by the {source.id()} source!"
if args.start:
start = args.start
else:
start = source.start()
logging.info(f"Using the source default start date of {start}.")
if args.end < start:
logging.critical(
f"The end date '{args.end}' preceeds the start date '{start}'!"
)
sys.exit(1)
series = Series(
base=source.normalizesymbol(args.pair[0]),
quote=source.normalizesymbol(args.pair[1]),
type=args.type,
start=args.start,
type=args.type or (source.types() + ["(none)"])[0],
start=start,
end=args.end,
)
if series.type not in source.types():
logging.critical(
f"The requested price type '{series.type}' is not "
f"recognized by the {source.id()} source!"
)
sys.exit(1)
fmt = Format.fromargs(args)
result = fetch(series, source, output, args.invert, args.quantize, fmt)
print(result, end="")
print(result, end="", file=output_file)
else:
parser.print_help()
parser.print_help(file=sys.stderr)
except BrokenPipeError:
logging.debug("The output pipe was closed early.")
finally:
logging.debug(f"Ended pricehist run at {datetime.now()}.")
def valid_pair(s):
base, quote = (s + "/").split("/")[0:2]
if base == "":
msg = f"No base found in the requested pair '{s}'."
raise argparse.ArgumentTypeError(msg)
return (base, quote)
def valid_date(s):
if s == "today":
return today()
try:
return datetime.strptime(s, "%Y-%m-%d").date().isoformat()
except ValueError:
msg = f"Not a valid YYYY-MM-DD date: '{s}'."
raise argparse.ArgumentTypeError(msg)
def valid_date_before(s):
return (
datetime.strptime(valid_date(s), "%Y-%m-%d").date() - timedelta(days=1)
).isoformat()
def valid_date_after(s):
return (
datetime.strptime(valid_date(s), "%Y-%m-%d").date() + timedelta(days=1)
).isoformat()
def valid_char(s):
if len(s) == 1:
return s
else:
msg = f"Not a single character: '{s}'."
raise argparse.ArgumentTypeError(msg)
def today():
return datetime.now().date().isoformat()
def build_parser():
def valid_pair(s):
base, quote = (s + "/").split("/")[0:2]
if base == "":
msg = f"No base found in the requested pair '{s}'."
raise argparse.ArgumentTypeError(msg)
return (base, quote)
def valid_date(s):
if s == "today":
return today()
try:
return datetime.strptime(s, "%Y-%m-%d").date().isoformat()
except ValueError:
msg = f"Not a valid YYYY-MM-DD date: '{s}'."
raise argparse.ArgumentTypeError(msg)
def previous_valid_date(s):
return (
datetime.strptime(valid_date(s), "%Y-%m-%d").date() - timedelta(days=1)
).isoformat()
def following_valid_date(s):
return (
datetime.strptime(valid_date(s), "%Y-%m-%d").date() + timedelta(days=1)
).isoformat()
def today():
return datetime.now().date().isoformat()
def valid_char(s):
if len(s) == 1:
return s
else:
msg = f"Not a single character: '{s}'."
raise argparse.ArgumentTypeError(msg)
def formatter(prog):
return argparse.HelpFormatter(prog, max_help_position=50)
class SetSourceDefaults(argparse.Action):
def __call__(self, parser, namespace, value, option_string=None):
source = sources.by_id[value]
setattr(namespace, self.dest, value)
if getattr(namespace, "type") is None:
setattr(namespace, "type", source.types()[0])
if getattr(namespace, "start") is None:
setattr(namespace, "start", source.start())
default_fmt = Format()
parser = argparse.ArgumentParser(
prog="pricehist",
@ -205,7 +197,7 @@ def build_parser():
"[--fmt-base SYM] [--fmt-quote SYM] [--fmt-time TIME] "
"[--fmt-decimal CHAR] [--fmt-thousands CHAR] "
"[--fmt-symbol rightspace|right|leftspace|left] [--fmt-datesep CHAR] "
"[--fmt-csvdelim CHAR] [--fmt-jsonnums]"
"[--fmt-csvdelim CHAR]"
),
formatter_class=formatter,
)
@ -214,7 +206,6 @@ def build_parser():
metavar="SOURCE",
type=str,
choices=sources.by_id.keys(),
action=SetSourceDefaults,
help="the source identifier",
)
fetch_parser.add_argument(
@ -235,7 +226,7 @@ def build_parser():
dest="type",
metavar="TYPE",
type=str,
help="price type, e.g. close (default: first for source)",
help="price type, e.g. close",
)
fetch_start_group = fetch_parser.add_mutually_exclusive_group(required=False)
fetch_start_group.add_argument(
@ -251,7 +242,7 @@ def build_parser():
"--startx",
dest="start",
metavar="DATE",
type=valid_date_after,
type=following_valid_date,
help="start date, exclusive",
)
@ -270,7 +261,7 @@ def build_parser():
"--endx",
dest="end",
metavar="DATE",
type=valid_date_before,
type=previous_valid_date,
help="end date, exclusive",
)
@ -353,11 +344,5 @@ def build_parser():
type=valid_char,
help=f"field delimiter for CSV output (default: '{default_fmt.csvdelim}')",
)
fetch_parser.add_argument(
"--fmt-jsonnums",
dest="formatjsonnums",
action="store_true",
help=f"numbers not strings for JSON output (default: {default_fmt.jsonnums})",
)
return parser

4
src/pricehist/exceptions.py
View file

@ -52,7 +52,7 @@ class InvalidType(SourceError, ValueError):
class CredentialsError(SourceError):
"""Access credentials are unavailable or invalid."""
def __init__(self, keys, source, msg=""):
def __init__(self, keys, source):
self.keys = keys
self.source = source
message = (
@ -61,8 +61,6 @@ class CredentialsError(SourceError):
f"correctly. Run 'pricehist source {source.id()}' for more "
f"information about credentials."
)
if msg:
message += f" {msg}"
super(CredentialsError, self).__init__(message)

4
src/pricehist/fetch.py
View file

@ -80,7 +80,5 @@ def _cov_description(
f"and ends {end_uncovered} day{s(end_uncovered)} earlier "
f"than requested"
)
elif start_uncovered == 0 and end_uncovered == 0:
return "as requested"
else:
return "which doesn't match the request"
return "as requested"

2
src/pricehist/format.py
View file

@ -11,7 +11,6 @@ class Format:
symbol: str = "rightspace"
datesep: str = "-"
csvdelim: str = ","
jsonnums: bool = False
@classmethod
def fromargs(cls, args):
@ -28,7 +27,6 @@ class Format:
symbol=if_not_none(args.formatsymbol, default.symbol),
datesep=if_not_none(args.formatdatesep, default.datesep),
csvdelim=if_not_none(args.formatcsvdelim, default.csvdelim),
jsonnums=if_not_none(args.formatjsonnums, default.jsonnums),
)
def format_date(self, date):

25
src/pricehist/isocurrencies.py
View file

@ -8,8 +8,8 @@ currencies are included and countries with no universal currency are ignored.
The data is read from vendored copies of the XML files published by the
maintainers of the standard:
* :file:`list-one.xml` (current currencies & funds)
* :file:`list-three.xml` (historical currencies & funds)
* :file:`list_one.xml` (current currencies & funds)
* :file:`list_three.xml` (historical currencies & funds)
Classes:
@ -24,8 +24,7 @@ Functions:
"""
from dataclasses import dataclass, field
from importlib.resources import files
from typing import List
from importlib.resources import read_binary
from lxml import etree
@ -37,34 +36,26 @@ class ISOCurrency:
minor_units: int = None
name: str = None
is_fund: bool = False
countries: List[str] = field(default_factory=list)
countries: list[str] = field(default_factory=list)
historical: bool = False
withdrawal_date: str = None
def current_data_date():
one = etree.fromstring(
files("pricehist.resources").joinpath("list-one.xml").read_bytes()
)
one = etree.fromstring(read_binary("pricehist.resources", "list_one.xml"))
return one.cssselect("ISO_4217")[0].attrib["Pblshd"]
def historical_data_date():
three = etree.fromstring(
files("pricehist.resources").joinpath("list-three.xml").read_bytes()
)
three = etree.fromstring(read_binary("pricehist.resources", "list_three.xml"))
return three.cssselect("ISO_4217")[0].attrib["Pblshd"]
def by_code():
result = {}
one = etree.fromstring(
files("pricehist.resources").joinpath("list-one.xml").read_bytes()
)
three = etree.fromstring(
files("pricehist.resources").joinpath("list-three.xml").read_bytes()
)
one = etree.fromstring(read_binary("pricehist.resources", "list_one.xml"))
three = etree.fromstring(read_binary("pricehist.resources", "list_three.xml"))
for entry in three.cssselect("HstrcCcyNtry") + one.cssselect("CcyNtry"):
if currency := _parse(entry):

1
src/pricehist/logger.py
View file

@ -23,7 +23,6 @@ def init():
handler.setFormatter(Formatter())
logging.root.addHandler(handler)
logging.root.setLevel(logging.INFO)
logging.getLogger("charset_normalizer").disabled = True
def show_debug():

3
src/pricehist/outputs/__init__.py
View file

@ -1,7 +1,6 @@
from .beancount import Beancount
from .csv import CSV
from .gnucashsql import GnuCashSQL
from .json import JSON
from .ledger import Ledger
default = "csv"
@ -9,8 +8,6 @@ default = "csv"
by_type = {
"beancount": Beancount(),
"csv": CSV(),
"json": JSON(),
"jsonl": JSON(jsonl=True),
"gnucash-sql": GnuCashSQL(),
"ledger": Ledger(),
}

2
src/pricehist/outputs/baseoutput.py
View file

@ -8,4 +8,4 @@ from pricehist.sources.basesource import BaseSource
class BaseOutput(ABC):
@abstractmethod
def format(self, series: Series, source: BaseSource, fmt: Format) -> str:
pass # pragma: nocover
pass

30
src/pricehist/outputs/gnucashsql.py
View file

@ -40,9 +40,9 @@ Classes:
import hashlib
import logging
from datetime import datetime, timezone
from datetime import datetime
from decimal import Decimal
from importlib.resources import files
from importlib.resources import read_text
from pricehist import __version__
from pricehist.format import Format
@ -58,7 +58,6 @@ class GnuCashSQL(BaseOutput):
self._warn_about_backslashes(
{
"date": fmt.format_date("1970-01-01"),
"time": fmt.time,
"base": base,
"quote": quote,
@ -119,18 +118,13 @@ class GnuCashSQL(BaseOutput):
"well."
)
sql = (
files("pricehist.resources")
.joinpath("gnucash.sql")
.read_text()
.format(
version=__version__,
timestamp=datetime.now(timezone.utc).isoformat()[:-6] + "Z",
base=self._sql_str(base),
quote=self._sql_str(quote),
values_comment=values_comment,
values=values,
)
sql = read_text("pricehist.resources", "gnucash.sql").format(
version=__version__,
timestamp=datetime.utcnow().isoformat() + "Z",
base=self._sql_str(base),
quote=self._sql_str(quote),
values_comment=values_comment,
values=values,
)
return sql
@ -141,7 +135,7 @@ class GnuCashSQL(BaseOutput):
logging.warning(
f"Before running this SQL, check the formatting of the "
f"{self._english_join(hits)} strings. "
f"SQLite treats backslashes in strings as plain characters, but "
f"SQLite treats backslahes in strings as plain characters, but "
f"MariaDB/MySQL and PostgreSQL may interpret them as escape "
f"codes."
)
@ -174,9 +168,9 @@ class GnuCashSQL(BaseOutput):
denom = str(1)
else:
numerator = sign + "".join([str(d) for d in tup.digits])
denom = str(10**-tup.exponent)
denom = str(10 ** -tup.exponent)
fit = self._fit_in_int64(Decimal(numerator), Decimal(denom))
return (numerator, denom, fit)
def _fit_in_int64(self, *numbers):
return all(n >= -(2**63) and n <= (2**63) - 1 for n in numbers)
return all(n >= -(2 ** 63) and n <= (2 ** 63) - 1 for n in numbers)

57
src/pricehist/outputs/json.py
View file

@ -1,57 +0,0 @@
"""
JSON output
Date, number and base/quote formatting options will be respected.
Classes:
JSON
"""
import io
import json
from pricehist.format import Format
from .baseoutput import BaseOutput
class JSON(BaseOutput):
def __init__(self, jsonl=False):
self.jsonl = jsonl
def format(self, series, source, fmt=Format()):
data = []
output = io.StringIO()
base = fmt.base or series.base
quote = fmt.quote or series.quote
for price in series.prices:
date = fmt.format_date(price.date)
if fmt.jsonnums:
amount = float(price.amount)
else:
amount = fmt.format_num(price.amount)
data.append(
{
"date": date,
"base": base,
"quote": quote,
"amount": amount,
"source": source.id(),
"type": series.type,
}
)
if self.jsonl:
for row in data:
json.dump(row, output, ensure_ascii=False)
output.write("\n")
else:
json.dump(data, output, ensure_ascii=False, indent=2)
output.write("\n")
return output.getvalue()

6
src/pricehist/resources/gnucash.sql
View file

@ -35,10 +35,10 @@ WHERE tp.base = g1.mnemonic
AND tp.guid NOT IN (SELECT guid FROM prices)
;
-- Show the final relevant rows of the main prices table
SELECT 'final' AS status, p.* FROM prices p WHERE p.guid IN (SELECT guid FROM new_prices) ORDER BY p.date;
-- Show the summary.
SELECT * FROM summary;
-- Show the final relevant rows of the main prices table
SELECT 'final' AS status, p.* FROM prices p WHERE p.guid IN (SELECT guid FROM new_prices) ORDER BY p.date;
COMMIT;

24
src/pricehist/resources/list-one.xml → src/pricehist/resources/list_one.xml
View file

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ISO_4217 Pblshd="2023-01-01">
<ISO_4217 Pblshd="2018-08-29">
<CcyTbl>
<CcyNtry>
<CtryNm>AFGHANISTAN</CtryNm>
@ -413,9 +413,9 @@
</CcyNtry>
<CcyNtry>
<CtryNm>CROATIA</CtryNm>
<CcyNm>Euro</CcyNm>
<Ccy>EUR</Ccy>
<CcyNbr>978</CcyNbr>
<CcyNm>Kuna</CcyNm>
<Ccy>HRK</Ccy>
<CcyNbr>191</CcyNbr>
<CcyMnrUnts>2</CcyMnrUnts>
</CcyNtry>
<CcyNtry>
@ -1493,13 +1493,6 @@
<CcyNbr>694</CcyNbr>
<CcyMnrUnts>2</CcyMnrUnts>
</CcyNtry>
<CcyNtry>
<CtryNm>SIERRA LEONE</CtryNm>
<CcyNm>Leone</CcyNm>
<Ccy>SLE</Ccy>
<CcyNbr>925</CcyNbr>
<CcyMnrUnts>2</CcyMnrUnts>
</CcyNtry>
<CcyNtry>
<CtryNm>SINGAPORE</CtryNm>
<CcyNm>Singapore Dollar</CcyNm>
@ -1708,7 +1701,7 @@
<CcyMnrUnts>3</CcyMnrUnts>
</CcyNtry>
<CcyNtry>
<CtryNm>TÜRKİYE</CtryNm>
<CtryNm>TURKEY</CtryNm>
<CcyNm>Turkish Lira</CcyNm>
<Ccy>TRY</Ccy>
<CcyNbr>949</CcyNbr>
@ -1826,13 +1819,6 @@
<CcyNbr>928</CcyNbr>
<CcyMnrUnts>2</CcyMnrUnts>
</CcyNtry>
<CcyNtry>
<CtryNm>VENEZUELA (BOLIVARIAN REPUBLIC OF)</CtryNm>
<CcyNm>Bolívar Soberano</CcyNm>
<Ccy>VED</Ccy>
<CcyNbr>926</CcyNbr>
<CcyMnrUnts>2</CcyMnrUnts>
</CcyNtry>
<CcyNtry>
<CtryNm>VIET NAM</CtryNm>
<CcyNm>Dong</CcyNm>

11
src/pricehist/resources/list-three.xml → src/pricehist/resources/list_three.xml
View file

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ISO_4217 Pblshd="2023-01-01">
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ISO_4217 Pblshd="2018-08-20">
<HstrcCcyTbl>
<HstrcCcyNtry>
<CtryNm>AFGHANISTAN</CtryNm>
@ -253,13 +253,6 @@
<CcyNbr>191</CcyNbr>
<WthdrwlDt>2015-06</WthdrwlDt>
</HstrcCcyNtry>
<HstrcCcyNtry>
<CtryNm>CROATIA</CtryNm>
<CcyNm>Kuna</CcyNm>
<Ccy>HRK</Ccy>
<CcyNbr>191</CcyNbr>
<WthdrwlDt>2023-01</WthdrwlDt>
</HstrcCcyNtry>
<HstrcCcyNtry>
<CtryNm>CYPRUS</CtryNm>
<CcyNm>Cyprus Pound</CcyNm>

3
src/pricehist/series.py
View file

@ -1,6 +1,5 @@
from dataclasses import dataclass, field, replace
from decimal import Decimal, getcontext
from typing import List
from pricehist.price import Price
@ -12,7 +11,7 @@ class Series:
type: str
start: str
end: str
prices: List[Price] = field(default_factory=list)
prices: list[Price] = field(default_factory=list)
def invert(self):
return replace(

12
src/pricehist/sources/__init__.py
View file

@ -1,6 +1,4 @@
from .alphavantage import AlphaVantage
from .bankofcanada import BankOfCanada
from .coinbasepro import CoinbasePro
from .coindesk import CoinDesk
from .coinmarketcap import CoinMarketCap
from .ecb import ECB
@ -8,15 +6,7 @@ from .yahoo import Yahoo
by_id = {
source.id(): source
for source in [
AlphaVantage(),
BankOfCanada(),
CoinbasePro(),
CoinDesk(),
CoinMarketCap(),
ECB(),
Yahoo(),
]
for source in [AlphaVantage(), CoinDesk(), CoinMarketCap(), ECB(), Yahoo()]
}

94
src/pricehist/sources/alphavantage.py
View file

@ -5,11 +5,10 @@ import logging
import os
from datetime import datetime, timedelta
from decimal import Decimal
from typing import List, Tuple
import requests
from pricehist import __version__, exceptions
from pricehist import exceptions
from pricehist.price import Price
from .basesource import BaseSource
@ -17,7 +16,6 @@ from .basesource import BaseSource
class AlphaVantage(BaseSource):
QUERY_URL = "https://www.alphavantage.co/query"
API_KEY_NAME = "ALPHAVANTAGE_API_KEY"
def id(self):
return "alphavantage"
@ -38,26 +36,24 @@ class AlphaVantage(BaseSource):
return ["close", "open", "high", "low", "adjclose", "mid"]
def notes(self):
keystatus = "already set" if self._apikey(require=False) else "not yet set"
keystatus = "already set" if self._apikey(require=False) else "NOT YET set"
return (
"Alpha Vantage has data on digital (crypto) currencies, physical "
"(fiat) currencies and stocks.\n"
"You should obtain a free API key from "
"https://www.alphavantage.co/support/#api-key and set it in "
f"the {self.API_KEY_NAME} environment variable ({keystatus}), "
"otherwise, pricehist will attempt to use a generic key.\n"
"An API key is required. One can be obtained for free from "
"https://www.alphavantage.co/support/#api-key and should be made "
"available in the ALPHAVANTAGE_API_KEY environment variable "
f"({keystatus}).\n"
"The PAIR for currencies should be in BASE/QUOTE form. The quote "
"symbol must always be for a physical currency. The --symbols option "
"will list all digital and physical currency symbols.\n"
"The PAIR for stocks is the stock symbol only. The quote currency "
f"will be determined automatically. {self._stock_symbols_message()}\n"
"The price type 'adjclose' is only available for stocks, and "
"requires an access key for which premium endpoints are unlocked.\n"
"Beware that digital currencies quoted in non-USD currencies may "
"be converted from USD data at one recent exchange rate rather "
"than using historical rates.\n"
"Alpha Vantage's standard API rate limit is 25 requests per day. "
"Note that retrieving prices for one stock consumes two API calls."
"The price type 'adjclose' is only available for stocks.\n"
"Alpha Vantage's standard API call frequency limits is 5 calls per "
"minute and 500 per day, so you may need to pause between successive "
"commands. Note that retrieving prices for one stock requires two "
"calls."
)
def _stock_symbols_message(self):
@ -169,13 +165,14 @@ class AlphaVantage(BaseSource):
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
self._raise_for_generic_errors(data)
if type(data) == dict and "Note" in data and "call frequency" in data["Note"]:
raise exceptions.RateLimit(data["Note"])
expected_keys = ["1. symbol", "2. name", "3. type", "4. region", "8. currency"]
if (
type(data) is not dict
type(data) != dict
or "bestMatches" not in data
or type(data["bestMatches"]) is not list
or type(data["bestMatches"]) != list
or not all(k in m for k in expected_keys for m in data["bestMatches"])
):
raise exceptions.ResponseParsingError("Unexpected content.")
@ -185,13 +182,8 @@ class AlphaVantage(BaseSource):
def _stock_data(self, series):
output_quote = self._stock_currency(series.base) or "UNKNOWN"
if series.type == "adjclose":
function = "TIME_SERIES_DAILY_ADJUSTED"
else:
function = "TIME_SERIES_DAILY"
params = {
"function": function,
"function": "TIME_SERIES_DAILY_ADJUSTED",
"symbol": series.base,
"outputsize": self._outputsize(series.start),
"apikey": self._apikey(),
@ -212,7 +204,8 @@ class AlphaVantage(BaseSource):
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
self._raise_for_generic_errors(data)
if type(data) == dict and "Note" in data and "call frequency" in data["Note"]:
raise exceptions.RateLimit(data["Note"])
if "Error Message" in data:
if output_quote == "UNKNOWN":
@ -229,8 +222,7 @@ class AlphaVantage(BaseSource):
"high": entries["2. high"],
"low": entries["3. low"],
"close": entries["4. close"],
"adjclose": "5. adjusted close" in entries
and entries["5. adjusted close"],
"adjclose": entries["5. adjusted close"],
}
for day, entries in reversed(data["Time Series (Daily)"].items())
}
@ -263,9 +255,10 @@ class AlphaVantage(BaseSource):
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
self._raise_for_generic_errors(data)
if type(data) == dict and "Note" in data and "call frequency" in data["Note"]:
raise exceptions.RateLimit(data["Note"])
if type(data) is not dict or "Time Series FX (Daily)" not in data:
if type(data) != dict or "Time Series FX (Daily)" not in data:
raise exceptions.ResponseParsingError("Unexpected content.")
normalized_data = {
@ -304,17 +297,18 @@ class AlphaVantage(BaseSource):
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
self._raise_for_generic_errors(data)
if type(data) == dict and "Note" in data and "call frequency" in data["Note"]:
raise exceptions.RateLimit(data["Note"])
if type(data) is not dict or "Time Series (Digital Currency Daily)" not in data:
if type(data) != dict or "Time Series (Digital Currency Daily)" not in data:
raise exceptions.ResponseParsingError("Unexpected content.")
normalized_data = {
day: {
"open": entries["1. open"],
"high": entries["2. high"],
"low": entries["3. low"],
"close": entries["4. close"],
"open": entries[f"1a. open ({series.quote})"],
"high": entries[f"2a. high ({series.quote})"],
"low": entries[f"3a. low ({series.quote})"],
"close": entries[f"4a. close ({series.quote})"],
}
for day, entries in reversed(
data["Time Series (Digital Currency Daily)"].items()
@ -323,39 +317,21 @@ class AlphaVantage(BaseSource):
return normalized_data
def _apikey(self, require=True):
key = os.getenv(self.API_KEY_NAME)
key_name = "ALPHAVANTAGE_API_KEY"
key = os.getenv(key_name)
if require and not key:
generic_key = f"pricehist_{__version__}"
logging.debug(
f"{self.API_KEY_NAME} not set. "
f"Defaulting to generic key '{generic_key}'."
)
return generic_key
raise exceptions.CredentialsError([key_name], self)
return key
def _raise_for_generic_errors(self, data):
if type(data) is dict:
if "Information" in data and "daily rate limits" in data["Information"]:
raise exceptions.RateLimit(data["Information"])
if (
"Information" in data
and "unlock" in data["Information"]
and "premium" in data["Information"]
):
msg = "You were denied access to a premium endpoint."
raise exceptions.CredentialsError([self.API_KEY_NAME], self, msg)
if "Error Message" in data and "apikey " in data["Error Message"]:
raise exceptions.CredentialsError([self.API_KEY_NAME], self)
def _physical_symbols(self) -> List[Tuple[str, str]]:
def _physical_symbols(self) -> list[(str, str)]:
url = "https://www.alphavantage.co/physical_currency_list/"
return self._get_symbols(url, "Physical: ")
def _digital_symbols(self) -> List[Tuple[str, str]]:
def _digital_symbols(self) -> list[(str, str)]:
url = "https://www.alphavantage.co/digital_currency_list/"
return self._get_symbols(url, "Digital: ")
def _get_symbols(self, url, prefix) -> List[Tuple[str, str]]:
def _get_symbols(self, url, prefix) -> list[(str, str)]:
try:
response = self.log_curl(requests.get(url))
except Exception as e:

118
src/pricehist/sources/bankofcanada.py
View file

@ -1,118 +0,0 @@
import dataclasses
import json
from decimal import Decimal
import requests
from pricehist import exceptions
from pricehist.price import Price
from .basesource import BaseSource
class BankOfCanada(BaseSource):
def id(self):
return "bankofcanada"
def name(self):
return "Bank of Canada"
def description(self):
return "Daily exchange rates of the Canadian dollar from the Bank of Canada"
def source_url(self):
return "https://www.bankofcanada.ca/valet/docs"
def start(self):
return "2017-01-03"
def types(self):
return ["default"]
def notes(self):
return (
"Currently, only daily exchange rates are supported. They are "
"published once each business day by 16:30 ET. "
"All Bank of Canada exchange rates are indicative rates only.\n"
"To request support for other data provided by the "
"Bank of Canada Valet Web Services, please open an "
"issue in pricehist's Gitlab project. "
)
def symbols(self):
url = "https://www.bankofcanada.ca/valet/lists/series/json"
try:
response = self.log_curl(requests.get(url))
except Exception as e:
raise exceptions.RequestError(str(e)) from e
try:
response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
data = json.loads(response.content)
series_names = data["series"].keys()
fx_series_names = [
n for n in series_names if len(n) == 8 and n[0:2] == "FX"
]
results = [
(f"{n[2:5]}/{n[5:9]}", data["series"][n]["description"])
for n in sorted(fx_series_names)
]
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
if not results:
raise exceptions.ResponseParsingError("Expected data not found")
else:
return results
def fetch(self, series):
if len(series.base) != 3 or len(series.quote) != 3:
raise exceptions.InvalidPair(series.base, series.quote, self)
series_name = f"FX{series.base}{series.quote}"
data = self._data(series, series_name)
prices = []
for o in data.get("observations", []):
prices.append(Price(o["d"], Decimal(o[series_name]["v"])))
return dataclasses.replace(series, prices=prices)
def _data(self, series, series_name):
url = f"https://www.bankofcanada.ca/valet/observations/{series_name}/json"
params = {
"start_date": series.start,
"end_date": series.end,
"order_dir": "asc",
}
try:
response = self.log_curl(requests.get(url, params=params))
except Exception as e:
raise exceptions.RequestError(str(e)) from e
code = response.status_code
text = response.text
try:
result = json.loads(response.content)
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
if code == 404 and "not found" in text:
raise exceptions.InvalidPair(series.base, series.quote, self)
elif code == 400 and "End date must be greater than the Start date" in text:
raise exceptions.BadResponse(result["message"])
else:
try:
response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
return result

7
src/pricehist/sources/basesource.py
View file

@ -1,7 +1,6 @@
import logging
from abc import ABC, abstractmethod
from textwrap import TextWrapper
from typing import List, Tuple
import curlify
@ -31,7 +30,7 @@ class BaseSource(ABC):
pass # pragma: nocover
@abstractmethod
def types(self) -> List[str]:
def types(self) -> list[str]:
pass # pragma: nocover
@abstractmethod
@ -42,10 +41,10 @@ class BaseSource(ABC):
return str.upper()
@abstractmethod
def symbols(self) -> List[Tuple[str, str]]:
def symbols(self) -> list[(str, str)]:
pass # pragma: nocover
def search(self, query) -> List[Tuple[str, str]]:
def search(self, query) -> list[(str, str)]:
pass # pragma: nocover
@abstractmethod

164
src/pricehist/sources/coinbasepro.py
View file

@ -1,164 +0,0 @@
import dataclasses
import json
from datetime import datetime, timedelta, timezone
from decimal import Decimal
import requests
from pricehist import exceptions
from pricehist.price import Price
from .basesource import BaseSource
class CoinbasePro(BaseSource):
def id(self):
return "coinbasepro"
def name(self):
return "Coinbase Pro"
def description(self):
return "The Coinbase Pro feed API provides market data to the public."
def source_url(self):
return "https://docs.pro.coinbase.com/"
def start(self):
return "2015-07-20"
def types(self):
return ["mid", "open", "high", "low", "close"]
def notes(self):
return (
"This source uses Coinbase's Pro APIs, not the v2 API.\n"
"No key or other authentication is requried because it only uses "
"the feed APIs that provide market data and are public."
)
def symbols(self):
products_url = "https://api.pro.coinbase.com/products"
currencies_url = "https://api.pro.coinbase.com/currencies"
try:
products_response = self.log_curl(requests.get(products_url))
currencies_response = self.log_curl(requests.get(currencies_url))
except Exception as e:
raise exceptions.RequestError(str(e)) from e
try:
products_response.raise_for_status()
currencies_response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
products_data = json.loads(products_response.content)
currencies_data = json.loads(currencies_response.content)
currencies = {c["id"]: c for c in currencies_data}
results = []
for i in sorted(products_data, key=lambda i: i["id"]):
base = i["base_currency"]
quote = i["quote_currency"]
base_name = currencies[base]["name"] if currencies[base] else base
quote_name = currencies[quote]["name"] if currencies[quote] else quote
results.append((f"{base}/{quote}", f"{base_name} against {quote_name}"))
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
if not results:
raise exceptions.ResponseParsingError("Expected data not found")
else:
return results
def fetch(self, series):
data = []
for seg_start, seg_end in self._segments(series.start, series.end):
data.extend(self._data(series.base, series.quote, seg_start, seg_end))
prices = []
for item in data:
prices.append(Price(item["date"], self._amount(item, series.type)))
return dataclasses.replace(series, prices=prices)
def _segments(self, start, end, length=290):
start = datetime.fromisoformat(start).date()
end = max(datetime.fromisoformat(end).date(), start)
segments = []
seg_start = start
while seg_start <= end:
seg_end = min(seg_start + timedelta(days=length - 1), end)
segments.append((seg_start.isoformat(), seg_end.isoformat()))
seg_start = seg_end + timedelta(days=1)
return segments
def _data(self, base, quote, start, end):
product = f"{base}-{quote}"
url = f"https://api.pro.coinbase.com/products/{product}/candles"
params = {
"start": start,
"end": end,
"granularity": "86400",
}
try:
response = self.log_curl(requests.get(url, params=params))
except Exception as e:
raise exceptions.RequestError(str(e)) from e
code = response.status_code
text = response.text
if code == 400 and "aggregations requested exceeds" in text:
raise exceptions.BadResponse("Too many data points requested.")
elif code == 400 and "start must be before end" in text:
raise exceptions.BadResponse("The end can't preceed the start.")
elif code == 400 and "is too old" in text:
raise exceptions.BadResponse("The requested interval is too early.")
elif code == 404 and "NotFound" in text:
raise exceptions.InvalidPair(base, quote, self)
elif code == 429:
raise exceptions.RateLimit(
"The rate limit has been exceeded. For more information see "
"https://docs.pro.coinbase.com/#rate-limit."
)
else:
try:
response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
result = reversed(
[
{
"date": self._ts_to_date(candle[0]),
"low": candle[1],
"high": candle[2],
"open": candle[3],
"close": candle[4],
}
for candle in json.loads(response.content)
if start <= self._ts_to_date(candle[0]) <= end
]
)
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
return result
def _ts_to_date(self, ts):
return datetime.fromtimestamp(ts, tz=timezone.utc).date().isoformat()
def _amount(self, item, type):
if type in ["mid"]:
high = Decimal(str(item["high"]))
low = Decimal(str(item["low"]))
return sum([high, low]) / 2
else:
return Decimal(str(item[type]))

7
src/pricehist/sources/coindesk.py
View file

@ -1,6 +1,5 @@
import dataclasses
import json
import logging
from decimal import Decimal
import requests
@ -20,9 +19,7 @@ class CoinDesk(BaseSource):
def description(self):
return (
"WARNING: This source is deprecated. Data stops at 2022-07-10.\n"
"The documentation URL now redirects to the main page.\n"
"An average of Bitcoin prices across leading global exchanges.\n"
"An average of Bitcoin prices across leading global exchanges. \n"
"Powered by CoinDesk, https://www.coindesk.com/price/bitcoin"
)
@ -67,8 +64,6 @@ class CoinDesk(BaseSource):
return results
def fetch(self, series):
logging.warning("This source is deprecated. Data stops at 2022-07-10.")
if series.base != "BTC" or series.quote in ["BTC", "XBT"]:
# BTC is the only valid base.
# BTC as the quote will return BTC/USD, which we don't want.

207
src/pricehist/sources/coinmarketcap.py
View file

@ -2,7 +2,6 @@ import dataclasses
import json
from datetime import datetime, timezone
from decimal import Decimal
from functools import lru_cache
import requests
@ -33,16 +32,13 @@ class CoinMarketCap(BaseSource):
def notes(self):
return (
"This source makes unoffical use of endpoints that power "
"CoinMarketCap's public web interface.\n"
"CoinMarketCap currency symbols are not necessarily unique. "
"Each symbol you give will be coverted an ID by checking fiat and "
"metals first, then crypto by CoinMarketCap rank. "
"The symbol data is hard-coded for fiat and metals, but fetched "
"live for crypto.\n"
"You can directly use IDs, which can be listed via the --symbols "
"option. For example, 'ETH/BTC' is 'id=1027/id=1'. "
"The corresponding symbols will be used in output, when available."
"This source makes unoffical use of endpoints that power CoinMarketCap's "
"public web interface. The price data comes from a public equivalent of "
"the OHLCV Historical endpoint found in CoinMarketCap's official API.\n"
"CoinMarketCap currency symbols are not necessarily unique, so it "
"is recommended that you use IDs, which can be listed via the "
"--symbols option. For example, 'ETH/BTC' is 'id=1027/id=1'. The "
"corresponding symbols will be used in output."
)
def symbols(self):
@ -59,10 +55,9 @@ class CoinMarketCap(BaseSource):
prices = []
for item in data.get("quotes", []):
d = item["timeOpen"][0:10]
amount = self._amount(item["quote"], series.type)
if amount is not None:
prices.append(Price(d, amount))
d = item["time_open"][0:10]
amount = self._amount(next(iter(item["quote"].values())), series.type)
prices.append(Price(d, amount))
output_base, output_quote = self._output_pair(series.base, series.quote, data)
@ -71,21 +66,21 @@ class CoinMarketCap(BaseSource):
)
def _data(self, series):
url = "https://api.coinmarketcap.com/data-api/v3.1/cryptocurrency/historical"
url = "https://web-api.coinmarketcap.com/v1/cryptocurrency/ohlcv/historical"
params = {}
if series.base.startswith("ID="):
params["id"] = series.base[3:]
else:
params["id"] = self._id_from_symbol(series.base, series)
params["symbol"] = series.base
if series.quote.startswith("ID="):
params["convertId"] = series.quote[3:]
params["convert_id"] = series.quote[3:]
else:
params["convertId"] = self._id_from_symbol(series.quote, series)
params["convert"] = series.quote
params["timeStart"] = int(
params["time_start"] = int(
int(
datetime.strptime(series.start, "%Y-%m-%d")
.replace(tzinfo=timezone.utc)
@ -94,14 +89,12 @@ class CoinMarketCap(BaseSource):
- 24 * 60 * 60
# Start one period earlier since the start is exclusive.
)
params["timeEnd"] = int(
params["time_end"] = int(
datetime.strptime(series.end, "%Y-%m-%d")
.replace(tzinfo=timezone.utc)
.timestamp()
) # Don't round up since it's inclusive of the period covering the end time.
params["interval"] = "daily"
try:
response = self.log_curl(requests.get(url, params=params))
except Exception as e:
@ -120,6 +113,26 @@ class CoinMarketCap(BaseSource):
series.base, series.quote, self, "Bad quote ID."
)
elif code == 400 and 'Invalid value for \\"convert\\"' in text:
raise exceptions.InvalidPair(
series.base, series.quote, self, "Bad quote symbol."
)
elif code == 400 and "must be older than" in text:
if series.start <= series.end:
raise exceptions.BadResponse("The start date must be in the past.")
else:
raise exceptions.BadResponse(
"The start date must preceed or match the end date."
)
elif (
code == 400
and "must be a valid ISO 8601 timestamp or unix time" in text
and series.start < "2001-09-11"
):
raise exceptions.BadResponse("The start date can't preceed 2001-09-11.")
try:
response.raise_for_status()
except Exception as e:
@ -130,18 +143,7 @@ class CoinMarketCap(BaseSource):
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
if (
"status" in parsed
and "error_code" in parsed["status"]
and parsed["status"]["error_code"] == "500"
and "The system is busy" in parsed["status"]["error_message"]
):
raise exceptions.BadResponse(
"The server indicated a general error. "
"There may be problem with your request."
)
if type(parsed) is not dict or "data" not in parsed:
if type(parsed) != dict or "data" not in parsed:
raise exceptions.ResponseParsingError("Unexpected content.")
elif len(parsed["data"]) == 0:
@ -153,154 +155,39 @@ class CoinMarketCap(BaseSource):
return parsed["data"]
def _amount(self, data, type):
if type in ["mid"] and data["high"] is not None and data["low"] is not None:
if type in ["mid"]:
high = Decimal(str(data["high"]))
low = Decimal(str(data["low"]))
return sum([high, low]) / 2
elif type in data and data[type] is not None:
return Decimal(str(data[type]))
else:
return None
return Decimal(str(data[type]))
def _output_pair(self, base, quote, data):
data_base = data["symbol"]
symbols = {i["id"]: (i["symbol"] or i["code"]) for i in self._symbol_data()}
data_quote = None
if len(data["quotes"]) > 0:
data_quote = symbols[int(data["quotes"][0]["quote"]["name"])]
data_quote = next(iter(data["quotes"][0]["quote"].keys()))
lookup_quote = None
if quote.startswith("ID="):
symbols = {i["id"]: (i["symbol"] or i["code"]) for i in self._symbol_data()}
lookup_quote = symbols[int(quote[3:])]
output_base = data_base
output_quote = data_quote or lookup_quote or quote
output_quote = lookup_quote or data_quote or quote
return (output_base, output_quote)
def _id_from_symbol(self, symbol, series):
for i in self._symbol_data():
if i["symbol"] == symbol:
return i["id"]
raise exceptions.InvalidPair(
series.base, series.quote, self, f"Invalid symbol '{symbol}'."
)
@lru_cache(maxsize=1)
def _symbol_data(self):
base_url = "https://api.coinmarketcap.com/data-api/v1/"
base_url = "https://web-api.coinmarketcap.com/v1/"
fiat_url = f"{base_url}fiat/map?include_metals=true"
crypto_url = f"{base_url}cryptocurrency/map?sort=cmc_rank"
fiat = self._get_json_data(fiat_url)
crypto = self._get_json_data(crypto_url)
# fmt: off
fiat = [
{"id": 2781, "symbol": "USD", "name": "United States Dollar"},
{"id": 3526, "symbol": "ALL", "name": "Albanian Lek"},
{"id": 3537, "symbol": "DZD", "name": "Algerian Dinar"},
{"id": 2821, "symbol": "ARS", "name": "Argentine Peso"},
{"id": 3527, "symbol": "AMD", "name": "Armenian Dram"},
{"id": 2782, "symbol": "AUD", "name": "Australian Dollar"},
{"id": 3528, "symbol": "AZN", "name": "Azerbaijani Manat"},
{"id": 3531, "symbol": "BHD", "name": "Bahraini Dinar"},
{"id": 3530, "symbol": "BDT", "name": "Bangladeshi Taka"},
{"id": 3533, "symbol": "BYN", "name": "Belarusian Ruble"},
{"id": 3532, "symbol": "BMD", "name": "Bermudan Dollar"},
{"id": 2832, "symbol": "BOB", "name": "Bolivian Boliviano"},
{"id": 3529, "symbol": "BAM", "name": "Bosnia-Herzegovina Convertible Mark"}, # noqa: E501
{"id": 2783, "symbol": "BRL", "name": "Brazilian Real"},
{"id": 2814, "symbol": "BGN", "name": "Bulgarian Lev"},
{"id": 3549, "symbol": "KHR", "name": "Cambodian Riel"},
{"id": 2784, "symbol": "CAD", "name": "Canadian Dollar"},
{"id": 2786, "symbol": "CLP", "name": "Chilean Peso"},
{"id": 2787, "symbol": "CNY", "name": "Chinese Yuan"},
{"id": 2820, "symbol": "COP", "name": "Colombian Peso"},
{"id": 3534, "symbol": "CRC", "name": "Costa Rican Colón"},
{"id": 2815, "symbol": "HRK", "name": "Croatian Kuna"},
{"id": 3535, "symbol": "CUP", "name": "Cuban Peso"},
{"id": 2788, "symbol": "CZK", "name": "Czech Koruna"},
{"id": 2789, "symbol": "DKK", "name": "Danish Krone"},
{"id": 3536, "symbol": "DOP", "name": "Dominican Peso"},
{"id": 3538, "symbol": "EGP", "name": "Egyptian Pound"},
{"id": 2790, "symbol": "EUR", "name": "Euro"},
{"id": 3539, "symbol": "GEL", "name": "Georgian Lari"},
{"id": 3540, "symbol": "GHS", "name": "Ghanaian Cedi"},
{"id": 3541, "symbol": "GTQ", "name": "Guatemalan Quetzal"},
{"id": 3542, "symbol": "HNL", "name": "Honduran Lempira"},
{"id": 2792, "symbol": "HKD", "name": "Hong Kong Dollar"},
{"id": 2793, "symbol": "HUF", "name": "Hungarian Forint"},
{"id": 2818, "symbol": "ISK", "name": "Icelandic Króna"},
{"id": 2796, "symbol": "INR", "name": "Indian Rupee"},
{"id": 2794, "symbol": "IDR", "name": "Indonesian Rupiah"},
{"id": 3544, "symbol": "IRR", "name": "Iranian Rial"},
{"id": 3543, "symbol": "IQD", "name": "Iraqi Dinar"},
{"id": 2795, "symbol": "ILS", "name": "Israeli New Shekel"},
{"id": 3545, "symbol": "JMD", "name": "Jamaican Dollar"},
{"id": 2797, "symbol": "JPY", "name": "Japanese Yen"},
{"id": 3546, "symbol": "JOD", "name": "Jordanian Dinar"},
{"id": 3551, "symbol": "KZT", "name": "Kazakhstani Tenge"},
{"id": 3547, "symbol": "KES", "name": "Kenyan Shilling"},
{"id": 3550, "symbol": "KWD", "name": "Kuwaiti Dinar"},
{"id": 3548, "symbol": "KGS", "name": "Kyrgystani Som"},
{"id": 3552, "symbol": "LBP", "name": "Lebanese Pound"},
{"id": 3556, "symbol": "MKD", "name": "Macedonian Denar"},
{"id": 2800, "symbol": "MYR", "name": "Malaysian Ringgit"},
{"id": 2816, "symbol": "MUR", "name": "Mauritian Rupee"},
{"id": 2799, "symbol": "MXN", "name": "Mexican Peso"},
{"id": 3555, "symbol": "MDL", "name": "Moldovan Leu"},
{"id": 3558, "symbol": "MNT", "name": "Mongolian Tugrik"},
{"id": 3554, "symbol": "MAD", "name": "Moroccan Dirham"},
{"id": 3557, "symbol": "MMK", "name": "Myanma Kyat"},
{"id": 3559, "symbol": "NAD", "name": "Namibian Dollar"},
{"id": 3561, "symbol": "NPR", "name": "Nepalese Rupee"},
{"id": 2811, "symbol": "TWD", "name": "New Taiwan Dollar"},
{"id": 2802, "symbol": "NZD", "name": "New Zealand Dollar"},
{"id": 3560, "symbol": "NIO", "name": "Nicaraguan Córdoba"},
{"id": 2819, "symbol": "NGN", "name": "Nigerian Naira"},
{"id": 2801, "symbol": "NOK", "name": "Norwegian Krone"},
{"id": 3562, "symbol": "OMR", "name": "Omani Rial"},
{"id": 2804, "symbol": "PKR", "name": "Pakistani Rupee"},
{"id": 3563, "symbol": "PAB", "name": "Panamanian Balboa"},
{"id": 2822, "symbol": "PEN", "name": "Peruvian Sol"},
{"id": 2803, "symbol": "PHP", "name": "Philippine Peso"},
{"id": 2805, "symbol": "PLN", "name": "Polish Złoty"},
{"id": 2791, "symbol": "GBP", "name": "Pound Sterling"},
{"id": 3564, "symbol": "QAR", "name": "Qatari Rial"},
{"id": 2817, "symbol": "RON", "name": "Romanian Leu"},
{"id": 2806, "symbol": "RUB", "name": "Russian Ruble"},
{"id": 3566, "symbol": "SAR", "name": "Saudi Riyal"},
{"id": 3565, "symbol": "RSD", "name": "Serbian Dinar"},
{"id": 2808, "symbol": "SGD", "name": "Singapore Dollar"},
{"id": 2812, "symbol": "ZAR", "name": "South African Rand"},
{"id": 2798, "symbol": "KRW", "name": "South Korean Won"},
{"id": 3567, "symbol": "SSP", "name": "South Sudanese Pound"},
{"id": 3573, "symbol": "VES", "name": "Sovereign Bolivar"},
{"id": 3553, "symbol": "LKR", "name": "Sri Lankan Rupee"},
{"id": 2807, "symbol": "SEK", "name": "Swedish Krona"},
{"id": 2785, "symbol": "CHF", "name": "Swiss Franc"},
{"id": 2809, "symbol": "THB", "name": "Thai Baht"},
{"id": 3569, "symbol": "TTD", "name": "Trinidad and Tobago Dollar"},
{"id": 3568, "symbol": "TND", "name": "Tunisian Dinar"},
{"id": 2810, "symbol": "TRY", "name": "Turkish Lira"},
{"id": 3570, "symbol": "UGX", "name": "Ugandan Shilling"},
{"id": 2824, "symbol": "UAH", "name": "Ukrainian Hryvnia"},
{"id": 2813, "symbol": "AED", "name": "United Arab Emirates Dirham"},
{"id": 3571, "symbol": "UYU", "name": "Uruguayan Peso"},
{"id": 3572, "symbol": "UZS", "name": "Uzbekistan Som"},
{"id": 2823, "symbol": "VND", "name": "Vietnamese Dong"},
]
metals = [
{"id": 3575, "symbol": "XAU", "name": "Gold Troy Ounce"},
{"id": 3574, "symbol": "XAG", "name": "Silver Troy Ounce"},
{"id": 3577, "symbol": "XPT", "name": "Platinum Ounce"},
{"id": 3576, "symbol": "XPD", "name": "Palladium Ounce"},
]
# fmt: on
return fiat + metals + crypto
return crypto + fiat
def _get_json_data(self, url, params={}):
try:
@ -318,7 +205,7 @@ class CoinMarketCap(BaseSource):
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
if type(parsed) is not dict or "data" not in parsed:
if type(parsed) != dict or "data" not in parsed:
raise exceptions.ResponseParsingError("Unexpected content.")
elif len(parsed["data"]) == 0:

122
src/pricehist/sources/exchangeratehost.py
View file

@ -1,122 +0,0 @@
import dataclasses
import json
from decimal import Decimal
import requests
from pricehist import exceptions
from pricehist.price import Price
from .basesource import BaseSource
class ExchangeRateHost(BaseSource):
def id(self):
return "exchangeratehost"
def name(self):
return "exchangerate.host Exchange rates API"
def description(self):
return (
"Exchange rates API is a simple and lightweight free service for "
"current and historical foreign exchange rates & crypto exchange "
"rates."
)
def source_url(self):
return "https://exchangerate.host/"
def start(self):
return "1999-01-01"
def types(self):
return ["close"]
def notes(self):
return ""
def symbols(self):
url = "https://api.coindesk.com/v1/bpi/supported-currencies.json"
try:
response = self.log_curl(requests.get(url))
except Exception as e:
raise exceptions.RequestError(str(e)) from e
try:
response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
data = json.loads(response.content)
relevant = [i for i in data if i["currency"] not in ["BTC", "XBT"]]
results = [
(f"BTC/{i['currency']}", f"Bitcoin against {i['country']}")
for i in sorted(relevant, key=lambda i: i["currency"])
]
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
if not results:
raise exceptions.ResponseParsingError("Expected data not found")
else:
return results
def fetch(self, series):
if series.base != "BTC" or series.quote in ["BTC", "XBT"]:
# BTC is the only valid base.
# BTC as the quote will return BTC/USD, which we don't want.
# XBT as the quote will fail with HTTP status 500.
raise exceptions.InvalidPair(series.base, series.quote, self)
data = self._data(series)
prices = []
for (d, v) in data.get("bpi", {}).items():
prices.append(Price(d, Decimal(str(v))))
return dataclasses.replace(series, prices=prices)
def _data(self, series):
url = "https://api.coindesk.com/v1/bpi/historical/close.json"
params = {
"currency": series.quote,
"start": series.start,
"end": series.end,
}
try:
response = self.log_curl(requests.get(url, params=params))
except Exception as e:
raise exceptions.RequestError(str(e)) from e
code = response.status_code
text = response.text
if code == 404 and "currency was not found" in text:
raise exceptions.InvalidPair(series.base, series.quote, self)
elif code == 404 and "only covers data from" in text:
raise exceptions.BadResponse(text)
elif code == 404 and "end date is before" in text and series.end < series.start:
raise exceptions.BadResponse("End date is before start date.")
elif code == 404 and "end date is before" in text:
raise exceptions.BadResponse("The start date must be in the past.")
elif code == 500 and "No results returned from database" in text:
raise exceptions.BadResponse(
"No results returned from database. This can happen when data "
"for a valid quote currency (e.g. CUP) doesn't go all the way "
"back to the start date, and potentially for other reasons."
)
else:
try:
response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
result = json.loads(response.content)
except Exception as e:
raise exceptions.ResponseParsingError(str(e)) from e
return result

113
src/pricehist/sources/yahoo.py
View file

@ -1,3 +1,4 @@
import csv
import dataclasses
import json
import logging
@ -70,39 +71,61 @@ class Yahoo(BaseSource):
series.base, series.quote, self, "Don't specify the quote currency."
)
data = self._data(series)
quote = data["chart"]["result"][0]["meta"]["currency"]
offset = data["chart"]["result"][0]["meta"]["gmtoffset"]
timestamps = data["chart"]["result"][0]["timestamp"]
adjclose_data = data["chart"]["result"][0]["indicators"]["adjclose"][0]
rest_data = data["chart"]["result"][0]["indicators"]["quote"][0]
amounts = {**adjclose_data, **rest_data}
quote, history = self._data(series)
prices = [
Price(date, amount)
for i in range(len(timestamps))
if (date := self._ts_to_date(timestamps[i] + offset)) <= series.end
if (amount := self._amount(amounts, series.type, i)) is not None
Price(row["date"], amount)
for row in history
if (amount := self._amount(row, series.type))
]
return dataclasses.replace(series, quote=quote, prices=prices)
def _ts_to_date(self, ts) -> str:
return datetime.fromtimestamp(ts, tz=timezone.utc).date().isoformat()
def _amount(self, amounts, type, i):
if type == "mid" and amounts["high"] != "null" and amounts["low"] != "null":
return sum([Decimal(amounts["high"][i]), Decimal(amounts["low"][i])]) / 2
elif amounts[type] != "null" and amounts[type][i] is not None:
return Decimal(amounts[type][i])
def _amount(self, row, type):
if type == "mid" and row["high"] != "null" and row["low"] != "null":
return sum([Decimal(row["high"]), Decimal(row["low"])]) / 2
else:
return None
return Decimal(row[type])
def _data(self, series) -> dict:
base_url = "https://query1.finance.yahoo.com/v8/finance/chart"
def _data(self, series) -> (dict, csv.DictReader):
base_url = "https://query1.finance.yahoo.com/v7/finance"
headers = {"User-Agent": f"pricehist/{__version__}"}
url = f"{base_url}/{series.base}"
spark_url = f"{base_url}/spark"
spark_params = {
"symbols": series.base,
"range": "1d",
"interval": "1d",
"indicators": "close",
"includeTimestamps": "false",
"includePrePost": "false",
}
try:
spark_response = self.log_curl(
requests.get(spark_url, params=spark_params, headers=headers)
)
except Exception as e:
raise exceptions.RequestError(str(e)) from e
code = spark_response.status_code
text = spark_response.text
if code == 404 and "No data found for spark symbols" in text:
raise exceptions.InvalidPair(
series.base, series.quote, self, "Symbol not found."
)
try:
spark_response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
spark = json.loads(spark_response.content)
quote = spark["spark"]["result"][0]["response"][0]["meta"]["currency"]
except Exception as e:
raise exceptions.ResponseParsingError(
"The spark data couldn't be parsed. "
) from e
start_ts = int(
datetime.strptime(series.start, "%Y-%m-%d")
@ -115,37 +138,36 @@ class Yahoo(BaseSource):
.timestamp()
) + (
24 * 60 * 60
) # some symbols require padding on the end timestamp
) # round up to include the last day
params = {
"symbol": series.base,
history_url = f"{base_url}/download/{series.base}"
history_params = {
"period1": start_ts,
"period2": end_ts,
"interval": "1d",
"events": "capitalGain%7Cdiv%7Csplit",
"events": "history",
"includeAdjustedClose": "true",
"formatted": "true",
"userYfid": "true",
"lang": "en-US",
"region": "US",
}
try:
response = self.log_curl(requests.get(url, params=params, headers=headers))
history_response = self.log_curl(
requests.get(history_url, params=history_params, headers=headers)
)
except Exception as e:
raise exceptions.RequestError(str(e)) from e
code = response.status_code
text = response.text
code = history_response.status_code
text = history_response.text
if code == 404 and "No data found, symbol may be delisted" in text:
raise exceptions.InvalidPair(
series.base, series.quote, self, "Symbol not found."
)
elif code == 400 and "Data doesn't exist" in text:
if code == 400 and "Data doesn't exist" in text:
raise exceptions.BadResponse(
"No data for the given interval. Try requesting a larger interval."
)
elif code == 404 and "Timestamp data missing" in text:
raise exceptions.BadResponse(
"Data missing. The given interval may be for a gap in the data "
@ -153,21 +175,18 @@ class Yahoo(BaseSource):
)
try:
response.raise_for_status()
history_response.raise_for_status()
except Exception as e:
raise exceptions.BadResponse(str(e)) from e
try:
data = json.loads(response.content)
history_lines = history_response.content.decode("utf-8").splitlines()
history_lines[0] = history_lines[0].lower().replace(" ", "")
history = csv.DictReader(history_lines, delimiter=",")
except Exception as e:
raise exceptions.ResponseParsingError(
"The data couldn't be parsed. "
) from e
raise exceptions.ResponseParsingError(str(e)) from e
if "timestamp" not in data["chart"]["result"][0]:
raise exceptions.BadResponse(
"No data for the given interval. "
"There may be a problem with the symbol or the interval."
)
if history_lines[0] != "date,open,high,low,close,adjclose,volume":
raise exceptions.ResponseParsingError("Unexpected CSV format")
return data
return (quote, history)

169
tests/live.sh
View file

@ -1,169 +0,0 @@
#!/bin/bash
# These are basic happy path tests that run pricehist from the command line and
# confirm that the results come out as expected. They help ensure that the main
# endpoints for each source are still working.
# Run this from the project root.
export ALPHAVANTAGE_API_KEY="TEST_KEY_$RANDOM"
cmd_prefix="poetry run"
passed=0
failed=0
skipped=0
run_test(){
name=$1
cmd=$2
expected=$3
echo "TEST: $name"
echo " Action: $cmd"
echo -n " Result: "
full_cmd="$cmd_prefix $cmd"
actual=$($full_cmd 2>&1)
if [[ "$actual" == "$expected" ]]; then
passed=$((passed+1))
echo "passed, output as expected"
else
failed=$((failed+1))
echo "failed, output differs as follows..."
echo
diff <(echo "$expected") <(echo "$actual")
fi
echo
}
skip_test(){
name=$1
cmd=$2
echo "TEST: $name"
echo " Action: $cmd"
echo " Result: SKIPPED!"
skipped=$((skipped+1))
echo
}
report(){
total=$((passed+failed))
if [[ "$skipped" -eq "0" ]]; then
skipped_str="none"
else
skipped_str="$skipped"
fi
if [[ "$failed" -eq "0" ]]; then
echo "SUMMARY: $passed tests passed, none failed, $skipped_str skipped"
else
echo "SUMMARY: $failed/$total tests failed, $skipped_str skipped"
exit 1
fi
}
name="Alpha Vantage stocks"
cmd="pricehist fetch alphavantage TSLA -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,TSLA,USD,729.7700,alphavantage,close
2021-01-05,TSLA,USD,735.1100,alphavantage,close
2021-01-06,TSLA,USD,755.9800,alphavantage,close
2021-01-07,TSLA,USD,816.0400,alphavantage,close
2021-01-08,TSLA,USD,880.0200,alphavantage,close
END
run_test "$name" "$cmd" "$expected"
name="Alpha Vantage physical currency"
cmd="pricehist fetch alphavantage AUD/EUR -s 2021-01-11 -e 2021-01-14"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-11,AUD,EUR,0.63374,alphavantage,close
2021-01-12,AUD,EUR,0.63684,alphavantage,close
2021-01-13,AUD,EUR,0.63686,alphavantage,close
2021-01-14,AUD,EUR,0.63984,alphavantage,close
END
run_test "$name" "$cmd" "$expected"
name="Alpha Vantage digital currency"
cmd="pricehist fetch alphavantage BTC/USD -s 2024-07-01 -e 2024-07-05"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2024-07-01,BTC,USD,62830.13000000,alphavantage,close
2024-07-02,BTC,USD,62040.22000000,alphavantage,close
2024-07-03,BTC,USD,60145.01000000,alphavantage,close
2024-07-04,BTC,USD,57042.14000000,alphavantage,close
2024-07-05,BTC,USD,56639.43000000,alphavantage,close
END
run_test "$name" "$cmd" "$expected"
name="Bank of Canada"
cmd="pricehist fetch bankofcanada CAD/USD -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,CAD,USD,0.7843,bankofcanada,default
2021-01-05,CAD,USD,0.7870,bankofcanada,default
2021-01-06,CAD,USD,0.7883,bankofcanada,default
2021-01-07,CAD,USD,0.7870,bankofcanada,default
2021-01-08,CAD,USD,0.7871,bankofcanada,default
END
run_test "$name" "$cmd" "$expected"
name="Coinbase Pro"
cmd="pricehist fetch coinbasepro BTC/EUR -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,BTC,EUR,24127,coinbasepro,mid
2021-01-05,BTC,EUR,26201.31,coinbasepro,mid
2021-01-06,BTC,EUR,28527.005,coinbasepro,mid
2021-01-07,BTC,EUR,31208.49,coinbasepro,mid
2021-01-08,BTC,EUR,32019,coinbasepro,mid
END
skip_test "$name" "$cmd" "$expected"
name="CoinDesk Bitcoin Price Index v1"
cmd="pricehist fetch coindeskbpi BTC/USD -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,BTC,USD,31431.6123,coindeskbpi,close
2021-01-05,BTC,USD,34433.6065,coindeskbpi,close
2021-01-06,BTC,USD,36275.7563,coindeskbpi,close
2021-01-07,BTC,USD,39713.5079,coindeskbpi,close
2021-01-08,BTC,USD,40519.4486,coindeskbpi,close
END
skip_test "$name" "$cmd" "$expected"
name="CoinMarketCap"
cmd="pricehist fetch coinmarketcap BTC/EUR -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,BTC,EUR,25322.5034586073,coinmarketcap,mid
2021-01-05,BTC,EUR,26318.9928757682,coinmarketcap,mid
2021-01-06,BTC,EUR,28570.9945210226,coinmarketcap,mid
2021-01-07,BTC,EUR,31200.8342706036,coinmarketcap,mid
2021-01-08,BTC,EUR,32157.05279624555,coinmarketcap,mid
END
run_test "$name" "$cmd" "$expected"
name="European Central Bank"
cmd="pricehist fetch ecb EUR/JPY -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,EUR,JPY,126.62,ecb,reference
2021-01-05,EUR,JPY,126.25,ecb,reference
2021-01-06,EUR,JPY,127.03,ecb,reference
2021-01-07,EUR,JPY,127.13,ecb,reference
2021-01-08,EUR,JPY,127.26,ecb,reference
END
run_test "$name" "$cmd" "$expected"
name="Yahoo! Finance"
cmd="pricehist fetch yahoo TSLA -s 2021-01-04 -e 2021-01-08"
read -r -d '' expected <<END
date,base,quote,amount,source,type
2021-01-04,TSLA,USD,243.2566680908203125,yahoo,adjclose
2021-01-05,TSLA,USD,245.0366668701171875,yahoo,adjclose
2021-01-06,TSLA,USD,251.9933319091796875,yahoo,adjclose
2021-01-07,TSLA,USD,272.013336181640625,yahoo,adjclose
2021-01-08,TSLA,USD,293.339996337890625,yahoo,adjclose
END
run_test "$name" "$cmd" "$expected"
report

0
tests/pricehist/outputs/__init__.py
View file

44
tests/pricehist/outputs/test_beancount.py
View file

@ -1,44 +0,0 @@
from decimal import Decimal
import pytest
from pricehist.format import Format
from pricehist.outputs.beancount import Beancount
from pricehist.price import Price
from pricehist.series import Series
@pytest.fixture
def out():
return Beancount()
@pytest.fixture
def series():
prices = [
Price("2021-01-01", Decimal("24139.4648")),
Price("2021-01-02", Decimal("26533.576")),
Price("2021-01-03", Decimal("27001.2846")),
]
return Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03", prices)
def test_format_basics(out, series, mocker):
source = mocker.MagicMock()
result = out.format(series, source, Format())
assert result == (
"2021-01-01 price BTC 24139.4648 EUR\n"
"2021-01-02 price BTC 26533.576 EUR\n"
"2021-01-03 price BTC 27001.2846 EUR\n"
)
def test_format_custom(out, series, mocker):
source = mocker.MagicMock()
fmt = Format(base="XBT", quote="EURO", thousands=".", decimal=",", datesep="/")
result = out.format(series, source, fmt)
assert result == (
"2021/01/01 price XBT 24.139,4648 EURO\n"
"2021/01/02 price XBT 26.533,576 EURO\n"
"2021/01/03 price XBT 27.001,2846 EURO\n"
)

50
tests/pricehist/outputs/test_csv.py
View file

@ -1,50 +0,0 @@
from decimal import Decimal
import pytest
from pricehist.format import Format
from pricehist.outputs.csv import CSV
from pricehist.price import Price
from pricehist.series import Series
@pytest.fixture
def out():
return CSV()
@pytest.fixture
def series():
prices = [
Price("2021-01-01", Decimal("24139.4648")),
Price("2021-01-02", Decimal("26533.576")),
Price("2021-01-03", Decimal("27001.2846")),
]
return Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03", prices)
def test_format_basics(out, series, mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="sourceid")
result = out.format(series, source, Format())
assert result == (
"date,base,quote,amount,source,type\n"
"2021-01-01,BTC,EUR,24139.4648,sourceid,close\n"
"2021-01-02,BTC,EUR,26533.576,sourceid,close\n"
"2021-01-03,BTC,EUR,27001.2846,sourceid,close\n"
)
def test_format_custom(out, series, mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="sourceid")
fmt = Format(
base="XBT", quote="€", thousands=".", decimal=",", datesep="/", csvdelim="/"
)
result = out.format(series, source, fmt)
assert result == (
"date/base/quote/amount/source/type\n"
'"2021/01/01"/XBT/€/24.139,4648/sourceid/close\n'
'"2021/01/02"/XBT/€/26.533,576/sourceid/close\n'
'"2021/01/03"/XBT/€/27.001,2846/sourceid/close\n'
)

140
tests/pricehist/outputs/test_gnucashsql.py
View file

@ -1,140 +0,0 @@
import dataclasses
import logging
import re
from decimal import Decimal
import pytest
from pricehist.format import Format
from pricehist.outputs.gnucashsql import GnuCashSQL
from pricehist.price import Price
from pricehist.series import Series
@pytest.fixture
def out():
return GnuCashSQL()
@pytest.fixture
def series():
prices = [
Price("2021-01-01", Decimal("24139.4648")),
Price("2021-01-02", Decimal("26533.576")),
Price("2021-01-03", Decimal("27001.2846")),
]
return Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03", prices)
@pytest.fixture
def src(mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="coindesk")
return source
def test_format_base_and_quote(out, series, src):
result = out.format(series, src, Format())
base, quote = re.findall(r"WHERE mnemonic = (.*) LIMIT", result, re.MULTILINE)
assert base == "'BTC'"
assert quote == "'EUR'"
def test_format_new_price_values(out, series, src):
result = out.format(series, src, Format())
values = re.search(
r"\(guid, date, base, quote, source, type, "
r"value_num, value_denom\) VALUES\n([^;]*);",
result,
re.MULTILINE,
)[1]
assert values == (
"('0c4c01bd0a252641b806ce46f716f161', '2021-01-01 00:00:00', "
"'BTC', 'EUR', 'coindesk', 'close', 241394648, 10000),\n"
"('47f895ddfcce18e2421387e0e1b636e9', '2021-01-02 00:00:00', "
"'BTC', 'EUR', 'coindesk', 'close', 26533576, 1000),\n"
"('0d81630c4ac50c1b9b7c8211bf99c94e', '2021-01-03 00:00:00', "
"'BTC', 'EUR', 'coindesk', 'close', 270012846, 10000)\n"
)
def test_format_customized(out, series, src):
fmt = Format(
base="XBT",
quote="EURO",
datesep="/",
time="23:59:59",
)
result = out.format(series, src, fmt)
base, quote = re.findall(r"WHERE mnemonic = (.*) LIMIT", result, re.MULTILINE)
values = re.search(
r"\(guid, date, base, quote, source, type, "
r"value_num, value_denom\) VALUES\n([^;]*);",
result,
re.MULTILINE,
)[1]
assert base == "'XBT'"
assert quote == "'EURO'"
assert values == (
"('448173eef5dea23cea9ff9d5e8c7b07e', '2021/01/01 23:59:59', "
"'XBT', 'EURO', 'coindesk', 'close', 241394648, 10000),\n"
"('b6c0f4474c91c50e8f65b47767f874ba', '2021/01/02 23:59:59', "
"'XBT', 'EURO', 'coindesk', 'close', 26533576, 1000),\n"
"('2937c872cf0672863e11b9f46ee41e09', '2021/01/03 23:59:59', "
"'XBT', 'EURO', 'coindesk', 'close', 270012846, 10000)\n"
)
def test_format_escaping_of_strings(out, series, src):
result = out.format(series, src, Format(base="B'tc''n"))
base, quote = re.findall(r"WHERE mnemonic = (.*) LIMIT", result, re.MULTILINE)
assert base == "'B''tc''''n'"
def test_format_insert_commented_out_if_no_values(out, series, src):
empty_series = dataclasses.replace(series, prices=[])
result = out.format(empty_series, src, Format())
(
"-- INSERT INTO new_prices (guid, date, base, quote, source, type, "
"value_num, value_denom) VALUES\n"
"-- \n"
"-- ;\n"
) in result
def test_format_warns_about_backslash(out, series, src, caplog):
with caplog.at_level(logging.WARNING):
out.format(series, src, Format(quote="EU\\RO"))
r = caplog.records[0]
assert r.levelname == "WARNING"
assert "backslashes in strings" in r.message
def test__english_join_other_cases(out):
assert out._english_join([]) == ""
assert out._english_join(["one"]) == "one"
assert out._english_join(["one", "two"]) == "one and two"
assert out._english_join(["one", "two", "three"]) == "one, two and three"
def test_format_warns_about_out_of_range_numbers(out, series, src, caplog):
too_big_numerator = Decimal("9223372036854.775808")
s = dataclasses.replace(series, prices=[Price("2021-01-01", too_big_numerator)])
with caplog.at_level(logging.WARNING):
out.format(s, src, Format())
r = caplog.records[0]
assert r.levelname == "WARNING"
assert "outside of the int64 range" in r.message
def test__rational_other_exponent_cases(out):
assert out._rational(Decimal("9223372036854e6")) == (
"9223372036854000000",
"1",
True,
)
assert out._rational(Decimal("9223372036854e-6")) == (
"9223372036854",
"1000000",
True,
)

168
tests/pricehist/outputs/test_json.py
View file

@ -1,168 +0,0 @@
from decimal import Decimal
from textwrap import dedent
import pytest
from pricehist.format import Format
from pricehist.outputs.json import JSON
from pricehist.price import Price
from pricehist.series import Series
@pytest.fixture
def json_out():
return JSON()
@pytest.fixture
def jsonl_out():
return JSON(jsonl=True)
@pytest.fixture
def series():
prices = [
Price("2021-01-01", Decimal("24139.4648")),
Price("2021-01-02", Decimal("26533.576")),
Price("2021-01-03", Decimal("27001.2846")),
]
return Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03", prices)
def test_format_basics(json_out, series, mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="sourceid")
result = json_out.format(series, source, Format())
assert (
result
== dedent(
"""
[
{
"date": "2021-01-01",
"base": "BTC",
"quote": "EUR",
"amount": "24139.4648",
"source": "sourceid",
"type": "close"
},
{
"date": "2021-01-02",
"base": "BTC",
"quote": "EUR",
"amount": "26533.576",
"source": "sourceid",
"type": "close"
},
{
"date": "2021-01-03",
"base": "BTC",
"quote": "EUR",
"amount": "27001.2846",
"source": "sourceid",
"type": "close"
}
]
"""
).strip()
+ "\n"
)
def test_format_basic_jsonl(jsonl_out, series, mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="sourceid")
result = jsonl_out.format(series, source, Format())
assert (
result
== dedent(
"""
{"date": "2021-01-01", "base": "BTC", "quote": "EUR", "amount": "24139.4648", "source": "sourceid", "type": "close"}
{"date": "2021-01-02", "base": "BTC", "quote": "EUR", "amount": "26533.576", "source": "sourceid", "type": "close"}
{"date": "2021-01-03", "base": "BTC", "quote": "EUR", "amount": "27001.2846", "source": "sourceid", "type": "close"}
""" # noqa
).strip()
+ "\n"
)
def test_format_custom(json_out, series, mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="sourceid")
fmt = Format(base="XBT", quote="€", thousands=".", decimal=",", datesep="/")
result = json_out.format(series, source, fmt)
assert (
result
== dedent(
"""
[
{
"date": "2021/01/01",
"base": "XBT",
"quote": "€",
"amount": "24.139,4648",
"source": "sourceid",
"type": "close"
},
{
"date": "2021/01/02",
"base": "XBT",
"quote": "€",
"amount": "26.533,576",
"source": "sourceid",
"type": "close"
},
{
"date": "2021/01/03",
"base": "XBT",
"quote": "€",
"amount": "27.001,2846",
"source": "sourceid",
"type": "close"
}
]
"""
).strip()
+ "\n"
)
def test_format_numbers(json_out, series, mocker):
source = mocker.MagicMock()
source.id = mocker.MagicMock(return_value="sourceid")
fmt = Format(jsonnums=True)
result = json_out.format(series, source, fmt)
assert (
result
== dedent(
"""
[
{
"date": "2021-01-01",
"base": "BTC",
"quote": "EUR",
"amount": 24139.4648,
"source": "sourceid",
"type": "close"
},
{
"date": "2021-01-02",
"base": "BTC",
"quote": "EUR",
"amount": 26533.576,
"source": "sourceid",
"type": "close"
},
{
"date": "2021-01-03",
"base": "BTC",
"quote": "EUR",
"amount": 27001.2846,
"source": "sourceid",
"type": "close"
}
]
"""
).strip()
+ "\n"
)

52
tests/pricehist/outputs/test_ledger.py
View file

@ -1,52 +0,0 @@
from decimal import Decimal
import pytest
from pricehist.format import Format
from pricehist.outputs.ledger import Ledger
from pricehist.price import Price
from pricehist.series import Series
@pytest.fixture
def out():
return Ledger()
@pytest.fixture
def series():
prices = [
Price("2021-01-01", Decimal("24139.4648")),
Price("2021-01-02", Decimal("26533.576")),
Price("2021-01-03", Decimal("27001.2846")),
]
return Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03", prices)
def test_format_basics(out, series, mocker):
source = mocker.MagicMock()
result = out.format(series, source, Format())
assert result == (
"P 2021-01-01 00:00:00 BTC 24139.4648 EUR\n"
"P 2021-01-02 00:00:00 BTC 26533.576 EUR\n"
"P 2021-01-03 00:00:00 BTC 27001.2846 EUR\n"
)
def test_format_custom(out, series, mocker):
source = mocker.MagicMock()
fmt = Format(
base="XBT",
quote="€",
time="23:59:59",
thousands=".",
decimal=",",
symbol="left",
datesep="/",
)
result = out.format(series, source, fmt)
assert result == (
"P 2021/01/01 23:59:59 XBT €24.139,4648\n"
"P 2021/01/02 23:59:59 XBT €26.533,576\n"
"P 2021/01/03 23:59:59 XBT €27.001,2846\n"
)

66
tests/pricehist/sources/test_alphavantage.py
View file

@ -9,7 +9,7 @@ import pytest
import requests
import responses
from pricehist import __version__, exceptions
from pricehist import exceptions
from pricehist.price import Price
from pricehist.series import Series
from pricehist.sources.alphavantage import AlphaVantage
@ -48,9 +48,6 @@ search_url = re.compile(
r"https://www\.alphavantage\.co/query\?function=SYMBOL_SEARCH.*"
)
stock_url = re.compile(
r"https://www\.alphavantage\.co/query\?function=TIME_SERIES_DAILY&.*"
)
adj_stock_url = re.compile(
r"https://www\.alphavantage\.co/query\?function=TIME_SERIES_DAILY_ADJUSTED.*"
)
physical_url = re.compile(r"https://www\.alphavantage\.co/query\?function=FX_DAILY.*")
@ -59,26 +56,14 @@ digital_url = re.compile(
)
rate_limit_json = (
'{ "Information": "'
"Thank you for using Alpha Vantage! Our standard API rate limit is 25 "
"requests per day. Please subscribe to any of the premium plans at "
"https://www.alphavantage.co/premium/ to instantly remove all daily rate "
"limits."
'{ "Note": "'
"Thank you for using Alpha Vantage! Our standard API call frequency is 5 "
"calls per minute and 500 calls per day. Please visit "
"https://www.alphavantage.co/premium/ if you would like to target a higher "
"API call frequency."
'" }'
)
premium_json = (
'{ "Information": "Thank you for using Alpha Vantage! This is a premium '
"endpoint and there are multiple ways to unlock premium endpoints: (1) "
"become a holder of Alpha Vantage Coin (AVC), an Ethereum-based "
"cryptocurrency that provides various utility & governance functions "
"within the Alpha Vantage ecosystem (AVC mining guide: "
"https://www.alphatournament.com/avc_mining_guide/) to unlock all "
"premium endpoints, (2) subscribe to any of the premium plans at "
"https://www.alphavantage.co/premium/ to instantly unlock all premium "
'endpoints" }'
)
@pytest.fixture
def physical_list_ok(requests_mock):
@ -114,13 +99,6 @@ def ibm_ok(requests_mock):
yield requests_mock
@pytest.fixture
def ibm_adj_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "ibm-partial-adj.json").read_text()
requests_mock.add(responses.GET, adj_stock_url, body=json, status=200)
yield requests_mock
@pytest.fixture
def euraud_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "eur-aud-partial.json").read_text()
@ -304,7 +282,7 @@ def test_fetch_stock_known(src, type, search_ok, ibm_ok):
stock_req = ibm_ok.calls[1].request
assert search_req.params["function"] == "SYMBOL_SEARCH"
assert search_req.params["keywords"] == "IBM"
assert stock_req.params["function"] == "TIME_SERIES_DAILY"
assert stock_req.params["function"] == "TIME_SERIES_DAILY_ADJUSTED"
assert stock_req.params["symbol"] == "IBM"
assert stock_req.params["outputsize"] == "full"
assert (series.base, series.quote) == ("IBM", "USD")
@ -337,17 +315,14 @@ def test_fetch_stock_types_all_available(src, search_ok, ibm_ok):
opn = src.fetch(Series("IBM", "", "open", "2021-01-04", "2021-01-08"))
hgh = src.fetch(Series("IBM", "", "high", "2021-01-04", "2021-01-08"))
low = src.fetch(Series("IBM", "", "low", "2021-01-04", "2021-01-08"))
adj = src.fetch(Series("IBM", "", "adjclose", "2021-01-04", "2021-01-08"))
mid = src.fetch(Series("IBM", "", "mid", "2021-01-04", "2021-01-08"))
assert cls.prices[0].amount == Decimal("123.94")
assert opn.prices[0].amount == Decimal("125.85")
assert hgh.prices[0].amount == Decimal("125.9174")
assert low.prices[0].amount == Decimal("123.04")
assert mid.prices[0].amount == Decimal("124.4787")
def test_fetch_stock_types_adj_available(src, search_ok, ibm_adj_ok):
adj = src.fetch(Series("IBM", "", "adjclose", "2021-01-04", "2021-01-08"))
assert adj.prices[0].amount == Decimal("120.943645029")
assert mid.prices[0].amount == Decimal("124.4787")
def test_fetch_stock_type_mid_is_mean_of_low_and_high(src, search_ok, ibm_ok):
@ -426,13 +401,6 @@ def test_fetch_stock_rate_limit(src, type, search_ok, requests_mock):
assert "rate limit" in str(e.value)
def test_fetch_stock_premium(src, search_ok, requests_mock):
requests_mock.add(responses.GET, adj_stock_url, body=premium_json)
with pytest.raises(exceptions.CredentialsError) as e:
src.fetch(Series("IBM", "", "adjclose", "2021-01-04", "2021-01-08"))
assert "denied access to a premium endpoint" in str(e.value)
def test_fetch_physical_known(src, type, physical_list_ok, euraud_ok):
series = src.fetch(Series("EUR", "AUD", type, "2021-01-04", "2021-01-08"))
req = euraud_ok.calls[1].request
@ -657,22 +625,8 @@ def test_fetch_bad_pair_quote_non_physical(src, type, physical_list_ok):
assert "quote must be a physical currency" in str(e.value)
def test_fetch_api_key_defaults_to_generic(
src, type, physical_list_ok, euraud_ok, monkeypatch
):
def test_fetch_api_key_missing(src, type, physical_list_ok, monkeypatch):
monkeypatch.delenv(api_key_name)
src.fetch(Series("EUR", "AUD", type, "2021-01-04", "2021-01-08"))
req = euraud_ok.calls[-1].request
assert req.params["apikey"] == f"pricehist_{__version__}"
def test_fetch_api_key_invalid(src, type, physical_list_ok, requests_mock):
body = (
'{ "Error Message": "the parameter apikey is invalid or missing. Please '
"claim your free API key on (https://www.alphavantage.co/support/#api-key). "
'It should take less than 20 seconds." }'
)
requests_mock.add(responses.GET, physical_url, body=body)
with pytest.raises(exceptions.CredentialsError) as e:
src.fetch(Series("EUR", "AUD", type, "2021-01-04", "2021-01-08"))
assert "unavailable or invalid" in str(e.value)

105
tests/pricehist/sources/test_alphavantage/btc-aud-partial.json
View file

@ -10,53 +10,88 @@
},
"Time Series (Digital Currency Daily)": {
"2021-01-09": {
"1. open": "55074.06950240",
"2. high": "56150.17720000",
"3. low": "52540.71680000",
"4. close": "54397.30924680",
"5. volume": "75785.97967500"
"1a. open (AUD)": "55074.06950240",
"1b. open (USD)": "40586.96000000",
"2a. high (AUD)": "56150.17720000",
"2b. high (USD)": "41380.00000000",
"3a. low (AUD)": "52540.71680000",
"3b. low (USD)": "38720.00000000",
"4a. close (AUD)": "54397.30924680",
"4b. close (USD)": "40088.22000000",
"5. volume": "75785.97967500",
"6. market cap (USD)": "75785.97967500"
},
"2021-01-08": {
"1. open": "53507.50941120",
"2. high": "56923.63300000",
"3. low": "49528.31000000",
"4. close": "55068.43820140",
"5. volume": "139789.95749900"
"1a. open (AUD)": "53507.50941120",
"1b. open (USD)": "39432.48000000",
"2a. high (AUD)": "56923.63300000",
"2b. high (USD)": "41950.00000000",
"3a. low (AUD)": "49528.31000000",
"3b. low (USD)": "36500.00000000",
"4a. close (AUD)": "55068.43820140",
"4b. close (USD)": "40582.81000000",
"5. volume": "139789.95749900",
"6. market cap (USD)": "139789.95749900"
},
"2021-01-07": {
"1. open": "49893.81535840",
"2. high": "54772.88310000",
"3. low": "49256.92200000",
"4. close": "53507.23802320",
"5. volume": "132825.70043700"
"1a. open (AUD)": "49893.81535840",
"1b. open (USD)": "36769.36000000",
"2a. high (AUD)": "54772.88310000",
"2b. high (USD)": "40365.00000000",
"3a. low (AUD)": "49256.92200000",
"3b. low (USD)": "36300.00000000",
"4a. close (AUD)": "53507.23802320",
"4b. close (USD)": "39432.28000000",
"5. volume": "132825.70043700",
"6. market cap (USD)": "132825.70043700"
},
"2021-01-06": {
"1. open": "46067.47523820",
"2. high": "50124.29161740",
"3. low": "45169.81872000",
"4. close": "49893.81535840",
"5. volume": "127139.20131000"
"1a. open (AUD)": "46067.47523820",
"1b. open (USD)": "33949.53000000",
"2a. high (AUD)": "50124.29161740",
"2b. high (USD)": "36939.21000000",
"3a. low (AUD)": "45169.81872000",
"3b. low (USD)": "33288.00000000",
"4a. close (AUD)": "49893.81535840",
"4b. close (USD)": "36769.36000000",
"5. volume": "127139.20131000",
"6. market cap (USD)": "127139.20131000"
},
"2021-01-05": {
"1. open": "43408.17136500",
"2. high": "46624.45840000",
"3. low": "40572.50600000",
"4. close": "46067.47523820",
"5. volume": "116049.99703800"
"1a. open (AUD)": "43408.17136500",
"1b. open (USD)": "31989.75000000",
"2a. high (AUD)": "46624.45840000",
"2b. high (USD)": "34360.00000000",
"3a. low (AUD)": "40572.50600000",
"3b. low (USD)": "29900.00000000",
"4a. close (AUD)": "46067.47523820",
"4b. close (USD)": "33949.53000000",
"5. volume": "116049.99703800",
"6. market cap (USD)": "116049.99703800"
},
"2021-01-04": {
"1. open": "44779.08784700",
"2. high": "45593.18400000",
"3. low": "38170.72220000",
"4. close": "43406.76014740",
"5. volume": "140899.88569000"
"1a. open (AUD)": "44779.08784700",
"1b. open (USD)": "33000.05000000",
"2a. high (AUD)": "45593.18400000",
"2b. high (USD)": "33600.00000000",
"3a. low (AUD)": "38170.72220000",
"3b. low (USD)": "28130.00000000",
"4a. close (AUD)": "43406.76014740",
"4b. close (USD)": "31988.71000000",
"5. volume": "140899.88569000",
"6. market cap (USD)": "140899.88569000"
},
"2021-01-03": {
"1. open": "43661.51206300",
"2. high": "47191.80858340",
"3. low": "43371.85965060",
"4. close": "44779.08784700",
"5. volume": "120957.56675000"
"1a. open (AUD)": "43661.51206300",
"1b. open (USD)": "32176.45000000",
"2a. high (AUD)": "47191.80858340",
"2b. high (USD)": "34778.11000000",
"3a. low (AUD)": "43371.85965060",
"3b. low (USD)": "31962.99000000",
"4a. close (AUD)": "44779.08784700",
"4b. close (USD)": "33000.05000000",
"5. volume": "120957.56675000",
"6. market cap (USD)": "120957.56675000"
}
}
}

81
tests/pricehist/sources/test_alphavantage/ibm-partial-adj.json
View file

@ -1,81 +0,0 @@
{
"Meta Data": {
"1. Information": "Daily Time Series with Splits and Dividend Events",
"2. Symbol": "IBM",
"3. Last Refreshed": "2021-07-20",
"4. Output Size": "Full size",
"5. Time Zone": "US/Eastern"
},
"Time Series (Daily)": {
"2021-01-11": {
"1. open": "127.95",
"2. high": "129.675",
"3. low": "127.66",
"4. close": "128.58",
"5. adjusted close": "125.471469081",
"6. volume": "5602466",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-08": {
"1. open": "128.57",
"2. high": "129.32",
"3. low": "126.98",
"4. close": "128.53",
"5. adjusted close": "125.422677873",
"6. volume": "4676487",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-07": {
"1. open": "130.04",
"2. high": "130.46",
"3. low": "128.26",
"4. close": "128.99",
"5. adjusted close": "125.871556982",
"6. volume": "4507382",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-06": {
"1. open": "126.9",
"2. high": "131.88",
"3. low": "126.72",
"4. close": "129.29",
"5. adjusted close": "126.164304226",
"6. volume": "7956740",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-05": {
"1. open": "125.01",
"2. high": "126.68",
"3. low": "124.61",
"4. close": "126.14",
"5. adjusted close": "123.090458157",
"6. volume": "6114619",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-04": {
"1. open": "125.85",
"2. high": "125.9174",
"3. low": "123.04",
"4. close": "123.94",
"5. adjusted close": "120.943645029",
"6. volume": "5179161",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2020-12-31": {
"1. open": "124.22",
"2. high": "126.03",
"3. low": "123.99",
"4. close": "125.88",
"5. adjusted close": "122.836743878",
"6. volume": "3574696",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
}
}
}

42
tests/pricehist/sources/test_alphavantage/ibm-partial.json
View file

@ -11,43 +11,71 @@
"1. open": "127.95",
"2. high": "129.675",
"3. low": "127.66",
"4. close": "128.58"
"4. close": "128.58",
"5. adjusted close": "125.471469081",
"6. volume": "5602466",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-08": {
"1. open": "128.57",
"2. high": "129.32",
"3. low": "126.98",
"4. close": "128.53"
"4. close": "128.53",
"5. adjusted close": "125.422677873",
"6. volume": "4676487",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-07": {
"1. open": "130.04",
"2. high": "130.46",
"3. low": "128.26",
"4. close": "128.99"
"4. close": "128.99",
"5. adjusted close": "125.871556982",
"6. volume": "4507382",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-06": {
"1. open": "126.9",
"2. high": "131.88",
"3. low": "126.72",
"4. close": "129.29"
"4. close": "129.29",
"5. adjusted close": "126.164304226",
"6. volume": "7956740",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-05": {
"1. open": "125.01",
"2. high": "126.68",
"3. low": "124.61",
"4. close": "126.14"
"4. close": "126.14",
"5. adjusted close": "123.090458157",
"6. volume": "6114619",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2021-01-04": {
"1. open": "125.85",
"2. high": "125.9174",
"3. low": "123.04",
"4. close": "123.94"
"4. close": "123.94",
"5. adjusted close": "120.943645029",
"6. volume": "5179161",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
},
"2020-12-31": {
"1. open": "124.22",
"2. high": "126.03",
"3. low": "123.99",
"4. close": "125.88"
"4. close": "125.88",
"5. adjusted close": "122.836743878",
"6. volume": "3574696",
"7. dividend amount": "0.0000",
"8. split coefficient": "1.0"
}
}
}

246
tests/pricehist/sources/test_bankofcanada.py
View file

@ -1,246 +0,0 @@
import logging
import os
from datetime import datetime
from decimal import Decimal
from pathlib import Path
import pytest
import requests
import responses
from pricehist import exceptions
from pricehist.price import Price
from pricehist.series import Series
from pricehist.sources.bankofcanada import BankOfCanada
@pytest.fixture
def src():
return BankOfCanada()
@pytest.fixture
def type(src):
return src.types()[0]
@pytest.fixture
def requests_mock():
with responses.RequestsMock() as mock:
yield mock
@pytest.fixture
def series_list_url():
return "https://www.bankofcanada.ca/valet/lists/series/json"
def fetch_url(series_name):
return f"https://www.bankofcanada.ca/valet/observations/{series_name}/json"
@pytest.fixture
def series_list_json():
dir = Path(os.path.splitext(__file__)[0])
return (dir / "series-partial.json").read_text()
@pytest.fixture
def series_list_response_ok(requests_mock, series_list_url, series_list_json):
requests_mock.add(responses.GET, series_list_url, body=series_list_json, status=200)
yield requests_mock
@pytest.fixture
def recent_response_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "recent.json").read_text()
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), body=json, status=200)
yield requests_mock
@pytest.fixture
def all_response_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "all-partial.json").read_text()
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), body=json, status=200)
yield requests_mock
def test_normalizesymbol(src):
assert src.normalizesymbol("cad") == "CAD"
assert src.normalizesymbol("usd") == "USD"
def test_metadata(src):
assert isinstance(src.id(), str)
assert len(src.id()) > 0
assert isinstance(src.name(), str)
assert len(src.name()) > 0
assert isinstance(src.description(), str)
assert len(src.description()) > 0
assert isinstance(src.source_url(), str)
assert src.source_url().startswith("http")
assert datetime.strptime(src.start(), "%Y-%m-%d")
assert isinstance(src.types(), list)
assert len(src.types()) > 0
assert isinstance(src.types()[0], str)
assert len(src.types()[0]) > 0
assert isinstance(src.notes(), str)
def test_symbols(src, series_list_response_ok):
syms = src.symbols()
assert ("CAD/USD", "Canadian dollar to US dollar daily exchange rate") in syms
assert len(syms) > 3
def test_symbols_requests_logged(src, series_list_response_ok, caplog):
with caplog.at_level(logging.DEBUG):
src.symbols()
assert any(
["DEBUG" == r.levelname and "curl " in r.message for r in caplog.records]
)
def test_symbols_not_found(src, requests_mock, series_list_url):
requests_mock.add(responses.GET, series_list_url, body='{"series":{}}', status=200)
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "data not found" in str(e.value)
def test_symbols_network_issue(src, requests_mock, series_list_url):
requests_mock.add(
responses.GET,
series_list_url,
body=requests.exceptions.ConnectionError("Network issue"),
)
with pytest.raises(exceptions.RequestError) as e:
src.symbols()
assert "Network issue" in str(e.value)
def test_symbols_bad_status(src, requests_mock, series_list_url):
requests_mock.add(responses.GET, series_list_url, status=500)
with pytest.raises(exceptions.BadResponse) as e:
src.symbols()
assert "Server Error" in str(e.value)
def test_symbols_parsing_error(src, requests_mock, series_list_url):
requests_mock.add(responses.GET, series_list_url, body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "while parsing data" in str(e.value)
def test_fetch_known_pair(src, type, recent_response_ok):
series = src.fetch(Series("CAD", "USD", type, "2021-01-01", "2021-01-07"))
req = recent_response_ok.calls[0].request
assert req.params["order_dir"] == "asc"
assert req.params["start_date"] == "2021-01-01"
assert req.params["end_date"] == "2021-01-07"
assert series.prices[0] == Price("2021-01-04", Decimal("0.7843"))
assert series.prices[-1] == Price("2021-01-07", Decimal("0.7870"))
assert len(series.prices) == 4
def test_fetch_requests_logged(src, type, recent_response_ok, caplog):
with caplog.at_level(logging.DEBUG):
src.fetch(Series("CAD", "USD", type, "2021-01-01", "2021-01-07"))
assert any(
["DEBUG" == r.levelname and "curl " in r.message for r in caplog.records]
)
def test_fetch_long_hist_from_start(src, type, all_response_ok):
series = src.fetch(Series("CAD", "USD", type, src.start(), "2021-01-07"))
assert series.prices[0] == Price("2017-01-03", Decimal("0.7443"))
assert series.prices[-1] == Price("2021-01-07", Decimal("0.7870"))
assert len(series.prices) > 13
def test_fetch_from_before_start(src, type, requests_mock):
body = """{ "observations": [] }"""
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), status=200, body=body)
series = src.fetch(Series("CAD", "USD", type, "2000-01-01", "2017-01-01"))
assert len(series.prices) == 0
def test_fetch_to_future(src, type, all_response_ok):
series = src.fetch(Series("CAD", "USD", type, "2021-01-01", "2100-01-01"))
assert len(series.prices) > 0
def test_wrong_dates_order(src, type, requests_mock):
body = """{ "message": "The End date must be greater than the Start date." }"""
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), status=400, body=body)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("CAD", "USD", type, "2021-01-07", "2021-01-01"))
assert "End date must be greater" in str(e.value)
def test_fetch_in_future(src, type, requests_mock):
body = """{ "observations": [] }"""
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), status=200, body=body)
series = src.fetch(Series("CAD", "USD", type, "2030-01-01", "2030-01-07"))
assert len(series.prices) == 0
def test_fetch_empty(src, type, requests_mock):
requests_mock.add(
responses.GET, fetch_url("FXCADUSD"), body="""{"observations":{}}"""
)
series = src.fetch(Series("CAD", "USD", type, "2021-01-03", "2021-01-03"))
assert len(series.prices) == 0
def test_fetch_no_quote(src, type):
with pytest.raises(exceptions.InvalidPair):
src.fetch(Series("CAD", "", type, "2021-01-01", "2021-01-07"))
def test_fetch_unknown_pair(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url("FXCADAFN"),
status=404,
body="""{
"message": "Series FXCADAFN not found.",
"docs": "https://www.bankofcanada.ca/valet/docs"
}""",
)
with pytest.raises(exceptions.InvalidPair):
src.fetch(Series("CAD", "AFN", type, "2021-01-01", "2021-01-07"))
def test_fetch_network_issue(src, type, requests_mock):
body = requests.exceptions.ConnectionError("Network issue")
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), body=body)
with pytest.raises(exceptions.RequestError) as e:
src.fetch(Series("CAD", "USD", type, "2021-01-01", "2021-01-07"))
assert "Network issue" in str(e.value)
def test_fetch_bad_status(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url("FXCADUSD"),
status=500,
body="""{"message": "Some other reason"}""",
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("CAD", "USD", type, "2021-01-01", "2021-01-07"))
assert "Internal Server Error" in str(e.value)
def test_fetch_parsing_error(src, type, requests_mock):
requests_mock.add(responses.GET, fetch_url("FXCADUSD"), body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("CAD", "USD", type, "2021-01-01", "2021-01-07"))
assert "while parsing data" in str(e.value)

101
tests/pricehist/sources/test_bankofcanada/all-partial.json
View file

@ -1,101 +0,0 @@
{
"terms": {
"url": "https://www.bankofcanada.ca/terms/"
},
"seriesDetail": {
"FXCADUSD": {
"label": "CAD/USD",
"description": "Canadian dollar to US dollar daily exchange rate",
"dimension": {
"key": "d",
"name": "date"
}
}
},
"observations": [
{
"d": "2017-01-03",
"FXCADUSD": {
"v": "0.7443"
}
},
{
"d": "2017-01-04",
"FXCADUSD": {
"v": "0.7510"
}
},
{
"d": "2017-01-05",
"FXCADUSD": {
"v": "0.7551"
}
},
{
"d": "2017-01-06",
"FXCADUSD": {
"v": "0.7568"
}
},
{
"d": "2017-01-09",
"FXCADUSD": {
"v": "0.7553"
}
},
{
"d": "2017-01-10",
"FXCADUSD": {
"v": "0.7568"
}
},
{
"d": "2017-01-11",
"FXCADUSD": {
"v": "0.7547"
}
},
{
"d": "2020-12-29",
"FXCADUSD": {
"v": "0.7809"
}
},
{
"d": "2020-12-30",
"FXCADUSD": {
"v": "0.7831"
}
},
{
"d": "2020-12-31",
"FXCADUSD": {
"v": "0.7854"
}
},
{
"d": "2021-01-04",
"FXCADUSD": {
"v": "0.7843"
}
},
{
"d": "2021-01-05",
"FXCADUSD": {
"v": "0.7870"
}
},
{
"d": "2021-01-06",
"FXCADUSD": {
"v": "0.7883"
}
},
{
"d": "2021-01-07",
"FXCADUSD": {
"v": "0.7870"
}
}
]
}

41
tests/pricehist/sources/test_bankofcanada/recent.json
View file

@ -1,41 +0,0 @@
{
"terms": {
"url": "https://www.bankofcanada.ca/terms/"
},
"seriesDetail": {
"FXCADUSD": {
"label": "CAD/USD",
"description": "Canadian dollar to US dollar daily exchange rate",
"dimension": {
"key": "d",
"name": "date"
}
}
},
"observations": [
{
"d": "2021-01-04",
"FXCADUSD": {
"v": "0.7843"
}
},
{
"d": "2021-01-05",
"FXCADUSD": {
"v": "0.7870"
}
},
{
"d": "2021-01-06",
"FXCADUSD": {
"v": "0.7883"
}
},
{
"d": "2021-01-07",
"FXCADUSD": {
"v": "0.7870"
}
}
]
}

272
tests/pricehist/sources/test_bankofcanada/series-partial.json
View file

@ -1,272 +0,0 @@
{
"terms": {
"url": "https://www.bankofcanada.ca/terms/"
},
"series": {
"FXAUDCAD": {
"label": "AUD/CAD",
"description": "Australian dollar to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXAUDCAD"
},
"FXBRLCAD": {
"label": "BRL/CAD",
"description": "Brazilian real to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXBRLCAD"
},
"FXCNYCAD": {
"label": "CNY/CAD",
"description": "Chinese renminbi to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCNYCAD"
},
"FXEURCAD": {
"label": "EUR/CAD",
"description": "European euro to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXEURCAD"
},
"FXHKDCAD": {
"label": "HKD/CAD",
"description": "Hong Kong dollar to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXHKDCAD"
},
"FXINRCAD": {
"label": "INR/CAD",
"description": "Indian rupee to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXINRCAD"
},
"FXIDRCAD": {
"label": "IDR/CAD",
"description": "Indonesian rupiah to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXIDRCAD"
},
"FXJPYCAD": {
"label": "JPY/CAD",
"description": "Japanese yen to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXJPYCAD"
},
"FXMYRCAD": {
"label": "MYR/CAD",
"description": "Malaysian ringgit to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXMYRCAD"
},
"FXMXNCAD": {
"label": "MXN/CAD",
"description": "Mexican peso to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXMXNCAD"
},
"FXNZDCAD": {
"label": "NZD/CAD",
"description": "New Zealand dollar to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXNZDCAD"
},
"FXNOKCAD": {
"label": "NOK/CAD",
"description": "Norwegian krone to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXNOKCAD"
},
"FXPENCAD": {
"label": "PEN/CAD",
"description": "Peruvian new sol to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXPENCAD"
},
"FXRUBCAD": {
"label": "RUB/CAD",
"description": "Russian ruble to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXRUBCAD"
},
"FXSARCAD": {
"label": "SAR/CAD",
"description": "Saudi riyal to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXSARCAD"
},
"FXSGDCAD": {
"label": "SGD/CAD",
"description": "Singapore dollar to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXSGDCAD"
},
"FXZARCAD": {
"label": "ZAR/CAD",
"description": "South African rand to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXZARCAD"
},
"FXKRWCAD": {
"label": "KRW/CAD",
"description": "South Korean won to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXKRWCAD"
},
"FXSEKCAD": {
"label": "SEK/CAD",
"description": "Swedish krona to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXSEKCAD"
},
"FXCHFCAD": {
"label": "CHF/CAD",
"description": "Swiss franc to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCHFCAD"
},
"FXTWDCAD": {
"label": "TWD/CAD",
"description": "Taiwanese dollar to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXTWDCAD"
},
"FXTHBCAD": {
"label": "THB/CAD",
"description": "Thai baht to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXTHBCAD"
},
"FXTRYCAD": {
"label": "TRY/CAD",
"description": "Turkish lira to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXTRYCAD"
},
"FXGBPCAD": {
"label": "GBP/CAD",
"description": "UK pound sterling to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXGBPCAD"
},
"FXUSDCAD": {
"label": "USD/CAD",
"description": "US dollar to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXUSDCAD"
},
"FXVNDCAD": {
"label": "VND/CAD",
"description": "Vietnamese dong to Canadian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXVNDCAD"
},
"FXCADAUD": {
"label": "CAD/AUD",
"description": "Canadian dollar to Australian dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADAUD"
},
"FXCADBRL": {
"label": "CAD/BRL",
"description": "Canadian dollar to Brazilian real daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADBRL"
},
"FXCADCNY": {
"label": "CAD/CNY",
"description": "Canadian dollar to Chinese renminbi daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADCNY"
},
"FXCADEUR": {
"label": "CAD/EUR",
"description": "Canadian dollar to European euro daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADEUR"
},
"FXCADHKD": {
"label": "CAD/HKD",
"description": "Canadian dollar to Hong Kong dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADHKD"
},
"FXCADINR": {
"label": "CAD/INR",
"description": "Canadian dollar to Indian rupee daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADINR"
},
"FXCADIDR": {
"label": "CAD/IDR",
"description": "Canadian dollar to Indonesian rupiah daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADIDR"
},
"FXCADJPY": {
"label": "CAD/JPY",
"description": "Canadian dollar to Japanese yen daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADJPY"
},
"FXCADMYR": {
"label": "CAD/MYR",
"description": "Canadian dollar to Malaysian ringgit daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADMYR"
},
"FXCADMXN": {
"label": "CAD/MXN",
"description": "Canadian dollar to Mexican peso daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADMXN"
},
"FXCADNZD": {
"label": "CAD/NZD",
"description": "Canadian dollar to New Zealand dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADNZD"
},
"FXCADNOK": {
"label": "CAD/NOK",
"description": "Canadian dollar to Norwegian krone daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADNOK"
},
"FXCADPEN": {
"label": "CAD/PEN",
"description": "Canadian dollar to Peruvian new sol daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADPEN"
},
"FXCADRUB": {
"label": "CAD/RUB",
"description": "Canadian dollar to Russian ruble daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADRUB"
},
"FXCADSAR": {
"label": "CAD/SAR",
"description": "Canadian dollar to Saudi riyal daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADSAR"
},
"FXCADSGD": {
"label": "CAD/SGD",
"description": "Canadian dollar to Singapore dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADSGD"
},
"FXCADZAR": {
"label": "CAD/ZAR",
"description": "Canadian dollar to South African rand daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADZAR"
},
"FXCADKRW": {
"label": "CAD/KRW",
"description": "Canadian dollar to South Korean won daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADKRW"
},
"FXCADSEK": {
"label": "CAD/SEK",
"description": "Canadian dollar to Swedish krona daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADSEK"
},
"FXCADCHF": {
"label": "CAD/CHF",
"description": "Canadian dollar to Swiss franc daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADCHF"
},
"FXCADTWD": {
"label": "CAD/TWD",
"description": "Canadian dollar to Taiwanese dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADTWD"
},
"FXCADTHB": {
"label": "CAD/THB",
"description": "Canadian dollar to Thai baht daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADTHB"
},
"FXCADTRY": {
"label": "CAD/TRY",
"description": "Canadian dollar to Turkish lira daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADTRY"
},
"FXCADGBP": {
"label": "CAD/GBP",
"description": "Canadian dollar to UK pound sterling daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADGBP"
},
"FXCADUSD": {
"label": "CAD/USD",
"description": "Canadian dollar to US dollar daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADUSD"
},
"FXCADVND": {
"label": "CAD/VND",
"description": "Canadian dollar to Vietnamese dong daily exchange rate",
"link": "https://www.bankofcanada.ca/valet/series/FXCADVND"
},
"INDINF_GRACE_Q": {
"label": "Foreign demand for Canadian non-commodity exports (GRACE) (2007=100)",
"description": "Foreign demand for Canadian non-commodity exports (GRACE) (2007=100)",
"link": "https://www.bankofcanada.ca/valet/series/INDINF_GRACE_Q"
}
}
}

183
tests/pricehist/sources/test_basesource.py
View file

@ -1,183 +0,0 @@
import logging
from typing import List, Tuple
import pytest
from pricehist.series import Series
from pricehist.sources.basesource import BaseSource
class TestSource(BaseSource):
def id(self) -> str:
return ""
def name(self) -> str:
return ""
def description(self) -> str:
return ""
def source_url(self) -> str:
return ""
def start(self) -> str:
return ""
def types(self) -> List[str]:
return []
def notes(self) -> str:
return ""
def symbols(self) -> List[Tuple[str, str]]:
return []
def fetch(self, series: Series) -> Series:
pass
@pytest.fixture
def src():
return TestSource()
def test_normalizesymbol_default_uppercase(src):
assert src.normalizesymbol("eur") == "EUR"
def test_format_symbols_one(src, mocker):
src.symbols = mocker.MagicMock(return_value=[("A", "Description")])
assert src.format_symbols() == "A Description\n"
def test_format_symbols_many(src, mocker):
src.symbols = mocker.MagicMock(
return_value=[
("A", "Description"),
("BB", "Description longer"),
("CCC", "Description longer again"),
("DDDD", f"Description {'very '*15}long"),
]
)
assert src.format_symbols() == (
"A Description\n"
"BB Description longer\n"
"CCC Description longer again\n"
"DDDD Description very very very very very very very very "
"very very very very very very very long\n"
)
def test_format_search(src, mocker):
src.search = mocker.MagicMock(
return_value=[
("A", "Description"),
("BB", "Description longer"),
("CCC", "Description longer again"),
("DDDD", f"Description {'very '*15}long"),
]
)
assert src.format_search("some query") == (
"A Description\n"
"BB Description longer\n"
"CCC Description longer again\n"
"DDDD Description very very very very very very very very "
"very very very very very very very long\n"
)
def test_format_search_not_possible(src, mocker, caplog):
src.search = mocker.MagicMock(return_value=None)
with caplog.at_level(logging.INFO):
with pytest.raises(SystemExit) as e:
src.format_search("some query")
assert e.value.code == 1
r = caplog.records[0]
assert r.levelname == "ERROR"
assert "Symbol search is not possible for" in r.message
def test_format_search_no_results(src, mocker, caplog):
src.search = mocker.MagicMock(return_value=[])
with caplog.at_level(logging.INFO):
results = src.format_search("some query")
r = caplog.records[0]
assert r.levelname == "INFO"
assert "No results found" in r.message
assert results == ""
def test_format_info_skips_renderes_all_fields(src, mocker):
src.id = mocker.MagicMock(return_value="sourceid")
src.name = mocker.MagicMock(return_value="Source Name")
src.description = mocker.MagicMock(return_value="Source description.")
src.source_url = mocker.MagicMock(return_value="https://example.com/")
src.start = mocker.MagicMock(return_value="2021-01-01")
src.types = mocker.MagicMock(return_value=["open", "close"])
src.notes = mocker.MagicMock(return_value="Notes for user.")
output = src.format_info()
assert output == (
"ID : sourceid\n"
"Name : Source Name\n"
"Description : Source description.\n"
"URL : https://example.com/\n"
"Start : 2021-01-01\n"
"Types : open, close\n"
"Notes : Notes for user."
)
def test_format_info_skips_empty_fields(src, mocker):
src.notes = mocker.MagicMock(return_value="")
output = src.format_info()
assert "Notes" not in output
def test_format_info_wraps_long_values_with_indent(src, mocker):
notes = (
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do "
"eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim "
"ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut "
"aliquip ex ea commodo consequat."
)
src.notes = mocker.MagicMock(return_value=notes)
output = src.format_info(total_width=60)
assert output == (
"Notes : Lorem ipsum dolor sit amet, consectetur\n"
" adipiscing elit, sed do eiusmod tempor\n"
" incididunt ut labore et dolore magna aliqua.\n"
" Ut enim ad minim veniam, quis nostrud\n"
" exercitation ullamco laboris nisi ut aliquip\n"
" ex ea commodo consequat."
)
def test_format_info_newline_handling(src, mocker):
notes = (
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do "
"eiusmod tempor incididunt ut labore.\n"
"Ut enim ad minim veniam.\n"
"\n"
"Quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea "
"commodo consequat."
)
src.notes = mocker.MagicMock(return_value=notes)
output = src.format_info(total_width=60)
assert output == (
"Notes : Lorem ipsum dolor sit amet, consectetur\n"
" adipiscing elit, sed do eiusmod tempor\n"
" incididunt ut labore.\n"
" Ut enim ad minim veniam.\n"
"\n"
" Quis nostrud exercitation ullamco laboris nisi\n"
" ut aliquip ex ea commodo consequat."
)
def test_format_info_does_not_wrap_source_url(src, mocker):
url = "https://www.example.com/longlonglonglonglonglonglonglong/"
src.source_url = mocker.MagicMock(return_value=url)
output = src.format_info(total_width=60)
assert output == (
"URL : https://www.example.com/longlonglonglonglonglonglonglong/"
)

334
tests/pricehist/sources/test_coinbasepro.py
View file

@ -1,334 +0,0 @@
import logging
import os
import re
from datetime import datetime
from decimal import Decimal
from pathlib import Path
import pytest
import requests
import responses
from pricehist import exceptions
from pricehist.price import Price
from pricehist.series import Series
from pricehist.sources.coinbasepro import CoinbasePro
@pytest.fixture
def src():
return CoinbasePro()
@pytest.fixture
def type(src):
return src.types()[0]
@pytest.fixture
def requests_mock():
with responses.RequestsMock() as mock:
yield mock
@pytest.fixture
def products_url():
return "https://api.pro.coinbase.com/products"
@pytest.fixture
def currencies_url():
return "https://api.pro.coinbase.com/currencies"
def product_url(base, quote):
return f"https://api.pro.coinbase.com/products/{base}-{quote}/candles"
@pytest.fixture
def products_json():
return (Path(os.path.splitext(__file__)[0]) / "products-partial.json").read_text()
@pytest.fixture
def currencies_json():
return (Path(os.path.splitext(__file__)[0]) / "currencies-partial.json").read_text()
@pytest.fixture
def products_response_ok(requests_mock, products_url, products_json):
requests_mock.add(responses.GET, products_url, body=products_json, status=200)
yield requests_mock
@pytest.fixture
def currencies_response_ok(requests_mock, currencies_url, currencies_json):
requests_mock.add(responses.GET, currencies_url, body=currencies_json, status=200)
yield requests_mock
@pytest.fixture
def recent_response_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "recent.json").read_text()
requests_mock.add(responses.GET, product_url("BTC", "EUR"), body=json, status=200)
yield requests_mock
@pytest.fixture
def multi_response_ok(requests_mock):
url1 = re.compile(
r"https://api\.pro\.coinbase\.com/products/BTC-EUR/candles\?start=2020-01-01.*"
)
url2 = re.compile(
r"https://api\.pro\.coinbase\.com/products/BTC-EUR/candles\?start=2020-10-17.*"
)
json1 = (
Path(os.path.splitext(__file__)[0]) / "2020-01-01--2020-10-16.json"
).read_text()
json2 = (
Path(os.path.splitext(__file__)[0]) / "2020-10-17--2021-01-07.json"
).read_text()
requests_mock.add(responses.GET, url1, body=json1, status=200)
requests_mock.add(responses.GET, url2, body=json2, status=200)
yield requests_mock
@pytest.fixture
def response_empty(requests_mock):
requests_mock.add(
responses.GET,
product_url("BTC", "EUR"),
status=200,
body="[]",
)
def test_normalizesymbol(src):
assert src.normalizesymbol("btc") == "BTC"
assert src.normalizesymbol("usd") == "USD"
def test_metadata(src):
assert isinstance(src.id(), str)
assert len(src.id()) > 0
assert isinstance(src.name(), str)
assert len(src.name()) > 0
assert isinstance(src.description(), str)
assert len(src.description()) > 0
assert isinstance(src.source_url(), str)
assert src.source_url().startswith("http")
assert datetime.strptime(src.start(), "%Y-%m-%d")
assert isinstance(src.types(), list)
assert len(src.types()) > 0
assert isinstance(src.types()[0], str)
assert len(src.types()[0]) > 0
assert isinstance(src.notes(), str)
def test_symbols(src, products_response_ok, currencies_response_ok):
syms = src.symbols()
assert ("BTC/EUR", "Bitcoin against Euro") in syms
assert len(syms) > 2
def test_symbols_requests_logged(
src, products_response_ok, currencies_response_ok, caplog
):
with caplog.at_level(logging.DEBUG):
src.symbols()
matching = filter(
lambda r: "DEBUG" == r.levelname and "curl " in r.message,
caplog.records,
)
assert len(list(matching)) == 2
def test_symbols_not_found(src, requests_mock, products_url, currencies_response_ok):
requests_mock.add(responses.GET, products_url, body="[]", status=200)
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "data not found" in str(e.value)
def test_symbols_network_issue(
src, requests_mock, products_response_ok, currencies_url
):
requests_mock.add(
responses.GET,
currencies_url,
body=requests.exceptions.ConnectionError("Network issue"),
)
with pytest.raises(exceptions.RequestError) as e:
src.symbols()
assert "Network issue" in str(e.value)
def test_symbols_bad_status(src, requests_mock, products_url, currencies_response_ok):
requests_mock.add(responses.GET, products_url, status=500)
with pytest.raises(exceptions.BadResponse) as e:
src.symbols()
assert "Server Error" in str(e.value)
def test_symbols_parsing_error(
src, requests_mock, products_response_ok, currencies_url
):
requests_mock.add(responses.GET, currencies_url, body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "while parsing data" in str(e.value)
def test_fetch_known_pair(src, type, recent_response_ok):
series = src.fetch(Series("BTC", "EUR", type, "2021-01-01", "2021-01-07"))
req = recent_response_ok.calls[0].request
assert req.params["granularity"] == "86400"
assert req.params["start"] == "2021-01-01"
assert req.params["end"] == "2021-01-07"
assert series.prices[0] == Price("2021-01-01", Decimal("23881.35"))
assert series.prices[-1] == Price("2021-01-07", Decimal("31208.49"))
assert len(series.prices) == 7
def test_fetch_types_all_available(src, recent_response_ok):
mid = src.fetch(Series("BTC", "EUR", "mid", "2021-01-01", "2021-01-07"))
opn = src.fetch(Series("BTC", "EUR", "open", "2021-01-01", "2021-01-07"))
hgh = src.fetch(Series("BTC", "EUR", "high", "2021-01-01", "2021-01-07"))
low = src.fetch(Series("BTC", "EUR", "low", "2021-01-01", "2021-01-07"))
cls = src.fetch(Series("BTC", "EUR", "close", "2021-01-01", "2021-01-07"))
assert mid.prices[0].amount == Decimal("23881.35")
assert opn.prices[0].amount == Decimal("23706.73")
assert hgh.prices[0].amount == Decimal("24250")
assert low.prices[0].amount == Decimal("23512.7")
assert cls.prices[0].amount == Decimal("24070.97")
def test_fetch_type_mid_is_mean_of_low_and_high(src, recent_response_ok):
mid = src.fetch(Series("BTC", "EUR", "mid", "2021-01-01", "2021-01-07")).prices
low = src.fetch(Series("BTC", "EUR", "low", "2021-01-01", "2021-01-07")).prices
hgh = src.fetch(Series("BTC", "EUR", "high", "2021-01-01", "2021-01-07")).prices
assert all(
[
mid[i].amount == (sum([low[i].amount, hgh[i].amount]) / 2)
for i in range(0, 7)
]
)
def test_fetch_requests_logged(src, type, recent_response_ok, caplog):
with caplog.at_level(logging.DEBUG):
src.fetch(Series("BTC", "EUR", type, "2021-01-01", "2021-01-07"))
assert any(
["DEBUG" == r.levelname and "curl " in r.message for r in caplog.records]
)
def test_fetch_long_hist_multi_segment(src, type, multi_response_ok):
series = src.fetch(Series("BTC", "EUR", type, "2020-01-01", "2021-01-07"))
assert series.prices[0] == Price("2020-01-01", Decimal("6430.175"))
assert series.prices[-1] == Price("2021-01-07", Decimal("31208.49"))
assert len(series.prices) > 3
def test_fetch_from_before_start(src, type, requests_mock):
body = '{"message":"End is too old"}'
requests_mock.add(responses.GET, product_url("BTC", "EUR"), status=400, body=body)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "EUR", type, "1960-01-01", "1960-01-07"))
assert "too early" in str(e.value)
def test_fetch_in_future(src, type, response_empty):
series = src.fetch(Series("BTC", "EUR", type, "2100-01-01", "2100-01-07"))
assert len(series.prices) == 0
def test_fetch_wrong_dates_order_alledged(src, type, requests_mock):
# Is actually prevented in argument parsing and inside the source.
body = '{"message":"start must be before end"}'
requests_mock.add(responses.GET, product_url("BTC", "EUR"), status=400, body=body)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "EUR", type, "2021-01-07", "2021-01-01"))
assert "end can't preceed" in str(e.value)
def test_fetch_too_many_data_points_alledged(src, type, requests_mock):
# Should only happen if limit is reduced or calculated segments lengthened
body = "aggregations requested exceeds"
requests_mock.add(responses.GET, product_url("BTC", "EUR"), status=400, body=body)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "EUR", type, "2021-01-07", "2021-01-01"))
assert "Too many data points" in str(e.value)
def test_fetch_rate_limit(src, type, requests_mock):
body = "Too many requests"
requests_mock.add(responses.GET, product_url("BTC", "EUR"), status=429, body=body)
with pytest.raises(exceptions.RateLimit) as e:
src.fetch(Series("BTC", "EUR", type, "2021-01-07", "2021-01-01"))
assert "rate limit has been exceeded" in str(e.value)
def test_fetch_empty(src, type, response_empty):
series = src.fetch(Series("BTC", "EUR", type, "2000-01-01", "2000-01-07"))
assert len(series.prices) == 0
def test_fetch_unknown_base(src, type, requests_mock):
body = '{"message":"NotFound"}'
requests_mock.add(
responses.GET, product_url("UNKNOWN", "EUR"), status=404, body=body
)
with pytest.raises(exceptions.InvalidPair):
src.fetch(Series("UNKNOWN", "EUR", type, "2021-01-01", "2021-01-07"))
def test_fetch_unknown_quote(src, type, requests_mock):
body = '{"message":"NotFound"}'
requests_mock.add(responses.GET, product_url("BTC", "XZY"), status=404, body=body)
with pytest.raises(exceptions.InvalidPair):
src.fetch(Series("BTC", "XZY", type, "2021-01-01", "2021-01-07"))
def test_fetch_no_quote(src, type, requests_mock):
body = '{"message":"NotFound"}'
requests_mock.add(responses.GET, product_url("BTC", ""), status=404, body=body)
with pytest.raises(exceptions.InvalidPair):
src.fetch(Series("BTC", "", type, "2021-01-01", "2021-01-07"))
def test_fetch_unknown_pair(src, type, requests_mock):
body = '{"message":"NotFound"}'
requests_mock.add(responses.GET, product_url("ABC", "XZY"), status=404, body=body)
with pytest.raises(exceptions.InvalidPair):
src.fetch(Series("ABC", "XZY", type, "2021-01-01", "2021-01-07"))
def test_fetch_network_issue(src, type, requests_mock):
body = requests.exceptions.ConnectionError("Network issue")
requests_mock.add(responses.GET, product_url("BTC", "EUR"), body=body)
with pytest.raises(exceptions.RequestError) as e:
src.fetch(Series("BTC", "EUR", type, "2021-01-01", "2021-01-07"))
assert "Network issue" in str(e.value)
def test_fetch_bad_status(src, type, requests_mock):
requests_mock.add(
responses.GET, product_url("BTC", "EUR"), status=500, body="Some other reason"
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "EUR", type, "2021-01-01", "2021-01-07"))
assert "Internal Server Error" in str(e.value)
def test_fetch_parsing_error(src, type, requests_mock):
requests_mock.add(responses.GET, product_url("BTC", "EUR"), body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("BTC", "EUR", type, "2021-01-01", "2021-01-07"))
assert "while parsing data" in str(e.value)

18
tests/pricehist/sources/test_coinbasepro/2020-01-01--2020-10-16.json
View file

@ -1,18 +0,0 @@
[
[
1602806400,
9588,
9860,
9828.84,
9672.41,
1068.08144123
],
[
1577836800,
6388.91,
6471.44,
6400.02,
6410.22,
491.94797816
]
]

18
tests/pricehist/sources/test_coinbasepro/2020-10-17--2021-01-07.json
View file

@ -1,18 +0,0 @@
[
[
1609977600,
29516.98,
32900,
29818.73,
32120.19,
5957.46980324
],
[
1602892800,
9630.1,
9742.61,
9675.29,
9706.33,
385.03505036
]
]

141
tests/pricehist/sources/test_coinbasepro/currencies-partial.json
View file

@ -1,141 +0,0 @@
[
{
"id": "BTC",
"name": "Bitcoin",
"min_size": "0.00000001",
"status": "online",
"message": "",
"max_precision": "0.00000001",
"convertible_to": [],
"details": {
"type": "crypto",
"symbol": "₿",
"network_confirmations": 3,
"sort_order": 20,
"crypto_address_link": "https://live.blockcypher.com/btc/address/{{address}}",
"crypto_transaction_link": "https://live.blockcypher.com/btc/tx/{{txId}}",
"push_payment_methods": [
"crypto"
],
"group_types": [
"btc",
"crypto"
],
"display_name": "",
"processing_time_seconds": 0,
"min_withdrawal_amount": 0.0001,
"max_withdrawal_amount": 2400
}
},
{
"id": "DOGE",
"name": "Dogecoin",
"min_size": "1",
"status": "online",
"message": "",
"max_precision": "0.1",
"convertible_to": [],
"details": {
"type": "crypto",
"symbol": "",
"network_confirmations": 60,
"sort_order": 29,
"crypto_address_link": "https://dogechain.info/address/{{address}}",
"crypto_transaction_link": "",
"push_payment_methods": [
"crypto"
],
"group_types": [],
"display_name": "",
"processing_time_seconds": 0,
"min_withdrawal_amount": 1,
"max_withdrawal_amount": 17391300
}
},
{
"id": "ETH",
"name": "Ether",
"min_size": "0.00000001",
"status": "online",
"message": "",
"max_precision": "0.00000001",
"convertible_to": [],
"details": {
"type": "crypto",
"symbol": "Ξ",
"network_confirmations": 35,
"sort_order": 25,
"crypto_address_link": "https://etherscan.io/address/{{address}}",
"crypto_transaction_link": "https://etherscan.io/tx/0x{{txId}}",
"push_payment_methods": [
"crypto"
],
"group_types": [
"eth",
"crypto"
],
"display_name": "",
"processing_time_seconds": 0,
"min_withdrawal_amount": 0.001,
"max_withdrawal_amount": 7450
}
},
{
"id": "EUR",
"name": "Euro",
"min_size": "0.01",
"status": "online",
"message": "",
"max_precision": "0.01",
"convertible_to": [],
"details": {
"type": "fiat",
"symbol": "€",
"network_confirmations": 0,
"sort_order": 2,
"crypto_address_link": "",
"crypto_transaction_link": "",
"push_payment_methods": [
"sepa_bank_account"
],
"group_types": [
"fiat",
"eur"
],
"display_name": "",
"processing_time_seconds": 0,
"min_withdrawal_amount": 0,
"max_withdrawal_amount": 0
}
},
{
"id": "GBP",
"name": "British Pound",
"min_size": "0.01",
"status": "online",
"message": "",
"max_precision": "0.01",
"convertible_to": [],
"details": {
"type": "fiat",
"symbol": "£",
"network_confirmations": 0,
"sort_order": 3,
"crypto_address_link": "",
"crypto_transaction_link": "",
"push_payment_methods": [
"uk_bank_account",
"swift_lhv",
"swift"
],
"group_types": [
"fiat",
"gbp"
],
"display_name": "",
"processing_time_seconds": 0,
"min_withdrawal_amount": 0,
"max_withdrawal_amount": 0
}
}
]

62
tests/pricehist/sources/test_coinbasepro/products-partial.json
View file

@ -1,62 +0,0 @@
[
{
"id": "BTC-EUR",
"base_currency": "BTC",
"quote_currency": "EUR",
"base_min_size": "0.0001",
"base_max_size": "200",
"quote_increment": "0.01",
"base_increment": "0.00000001",
"display_name": "BTC/EUR",
"min_market_funds": "10",
"max_market_funds": "600000",
"margin_enabled": false,
"fx_stablecoin": false,
"post_only": false,
"limit_only": false,
"cancel_only": false,
"trading_disabled": false,
"status": "online",
"status_message": ""
},
{
"id": "ETH-GBP",
"base_currency": "ETH",
"quote_currency": "GBP",
"base_min_size": "0.001",
"base_max_size": "1400",
"quote_increment": "0.01",
"base_increment": "0.00000001",
"display_name": "ETH/GBP",
"min_market_funds": "10",
"max_market_funds": "1000000",
"margin_enabled": false,
"fx_stablecoin": false,
"post_only": false,
"limit_only": false,
"cancel_only": false,
"trading_disabled": false,
"status": "online",
"status_message": ""
},
{
"id": "DOGE-EUR",
"base_currency": "DOGE",
"quote_currency": "EUR",
"base_min_size": "1",
"base_max_size": "690000",
"quote_increment": "0.0001",
"base_increment": "0.1",
"display_name": "DOGE/EUR",
"min_market_funds": "5.0",
"max_market_funds": "100000",
"margin_enabled": false,
"fx_stablecoin": false,
"post_only": false,
"limit_only": false,
"cancel_only": false,
"trading_disabled": false,
"status": "online",
"status_message": ""
}
]

58
tests/pricehist/sources/test_coinbasepro/recent.json
View file

@ -1,58 +0,0 @@
[
[
1609977600,
29516.98,
32900,
29818.73,
32120.19,
5957.46980324
],
[
1609891200,
27105.01,
29949,
27655.04,
29838.52,
4227.05067035
],
[
1609804800,
24413.62,
27989,
26104.4,
27654.01,
4036.27720179
],
[
1609718400,
22055,
26199,
25624.7,
26115.94,
6304.41029978
],
[
1609632000,
24500,
27195.46,
25916.75,
25644.41,
4975.13927959
],
[
1609545600,
22000,
27000,
24071.26,
25907.35,
7291.88538639
],
[
1609459200,
23512.7,
24250,
23706.73,
24070.97,
1830.04655405
]
]

293
tests/pricehist/sources/test_coinmarketcap.py
View file

@ -36,10 +36,9 @@ def requests_mock():
yield mock
crypto_url = (
"https://api.coinmarketcap.com/data-api/v1/cryptocurrency/map?sort=cmc_rank"
)
fetch_url = "https://api.coinmarketcap.com/data-api/v3.1/cryptocurrency/historical"
crypto_url = "https://web-api.coinmarketcap.com/v1/cryptocurrency/map?sort=cmc_rank"
fiat_url = "https://web-api.coinmarketcap.com/v1/fiat/map?include_metals=true"
fetch_url = "https://web-api.coinmarketcap.com/v1/cryptocurrency/ohlcv/historical"
@pytest.fixture
@ -49,6 +48,13 @@ def crypto_ok(requests_mock):
yield requests_mock
@pytest.fixture
def fiat_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "fiat-partial.json").read_text()
requests_mock.add(responses.GET, fiat_url, body=json, status=200)
yield requests_mock
@pytest.fixture
def recent_id_id_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "recent-id1-id2782.json").read_text()
@ -56,6 +62,36 @@ def recent_id_id_ok(requests_mock):
yield requests_mock
@pytest.fixture
def recent_id_sym_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "recent-id1-aud.json").read_text()
requests_mock.add(responses.GET, fetch_url, body=json, status=200)
yield requests_mock
@pytest.fixture
def recent_sym_id_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "recent-btc-id2782.json").read_text()
requests_mock.add(responses.GET, fetch_url, body=json, status=200)
yield requests_mock
@pytest.fixture
def recent_sym_sym_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "recent-btc-aud.json").read_text()
requests_mock.add(responses.GET, fetch_url, body=json, status=200)
yield requests_mock
@pytest.fixture
def long_sym_sym_ok(requests_mock):
json = (
Path(os.path.splitext(__file__)[0]) / "long-btc-aud-partial.json"
).read_text()
requests_mock.add(responses.GET, fetch_url, body=json, status=200)
yield requests_mock
def test_normalizesymbol(src):
assert src.normalizesymbol("btc") == "BTC"
assert src.normalizesymbol("id=1") == "ID=1"
@ -84,31 +120,63 @@ def test_metadata(src):
assert isinstance(src.notes(), str)
def test_symbols(src, crypto_ok):
def test_symbols(src, crypto_ok, fiat_ok):
syms = src.symbols()
assert ("id=1", "BTC Bitcoin") in syms
assert ("id=2782", "AUD Australian Dollar") in syms
assert len(syms) > 2
def test_symbols_request_logged(src, crypto_ok, caplog):
def test_symbols_requests_logged(src, crypto_ok, fiat_ok, caplog):
with caplog.at_level(logging.DEBUG):
src.symbols()
logged_requests = 0
for r in caplog.records:
if r.levelname == "DEBUG" and "curl " in r.message:
logged_requests += 1
assert logged_requests == 1
assert logged_requests == 2
def test_symbols_crypto_not_found(src, requests_mock):
def test_symbols_fiat_not_found(src, requests_mock):
requests_mock.add(responses.GET, fiat_url, body="{}", status=200)
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "Unexpected content" in str(e.value)
def test_symbols_fiat_network_issue(src, requests_mock):
requests_mock.add(
responses.GET,
fiat_url,
body=requests.exceptions.ConnectionError("Network issue"),
)
with pytest.raises(exceptions.RequestError) as e:
src.symbols()
assert "Network issue" in str(e.value)
def test_symbols_fiat_bad_status(src, requests_mock):
requests_mock.add(responses.GET, fiat_url, status=500)
with pytest.raises(exceptions.BadResponse) as e:
src.symbols()
assert "Server Error" in str(e.value)
def test_symbols_fiat_parsing_error(src, requests_mock):
requests_mock.add(responses.GET, fiat_url, body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "while parsing data" in str(e.value)
def test_symbols_crypto_not_found(src, requests_mock, fiat_ok):
requests_mock.add(responses.GET, crypto_url, body="{}", status=200)
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "Unexpected content" in str(e.value)
def test_symbols_crypto_network_issue(src, requests_mock):
def test_symbols_crypto_network_issue(src, requests_mock, fiat_ok):
requests_mock.add(
responses.GET,
crypto_url,
@ -119,14 +187,14 @@ def test_symbols_crypto_network_issue(src, requests_mock):
assert "Network issue" in str(e.value)
def test_symbols_crypto_bad_status(src, requests_mock):
def test_symbols_crypto_bad_status(src, requests_mock, fiat_ok):
requests_mock.add(responses.GET, crypto_url, status=500)
with pytest.raises(exceptions.BadResponse) as e:
src.symbols()
assert "Server Error" in str(e.value)
def test_symbols_crypto_parsing_error(src, requests_mock):
def test_symbols_crypto_parsing_error(src, requests_mock, fiat_ok):
requests_mock.add(responses.GET, crypto_url, body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
@ -134,59 +202,59 @@ def test_symbols_crypto_parsing_error(src, requests_mock):
def test_symbols_no_data(src, type, requests_mock):
requests_mock.add(responses.GET, crypto_url, body='{"data": []}')
requests_mock.add(responses.GET, fiat_url, body='{"data": []}')
with pytest.raises(exceptions.ResponseParsingError) as e:
src.symbols()
assert "Empty data section" in str(e.value)
def test_fetch_known_pair_id_id(src, type, recent_id_id_ok, crypto_ok):
def test_fetch_known_pair_id_id(src, type, recent_id_id_ok, crypto_ok, fiat_ok):
series = src.fetch(Series("ID=1", "ID=2782", type, "2021-01-01", "2021-01-07"))
req = recent_id_id_ok.calls[0].request
assert req.params["id"] == "1"
assert req.params["convertId"] == "2782"
assert req.params["convert_id"] == "2782"
assert (series.base, series.quote) == ("BTC", "AUD")
assert len(series.prices) == 7
def test_fetch_known_pair_id_sym(src, type, recent_id_id_ok, crypto_ok):
def test_fetch_known_pair_id_sym(src, type, recent_id_sym_ok):
series = src.fetch(Series("ID=1", "AUD", type, "2021-01-01", "2021-01-07"))
req = recent_id_id_ok.calls[1].request
req = recent_id_sym_ok.calls[0].request
assert req.params["id"] == "1"
assert req.params["convertId"] == "2782"
assert req.params["convert"] == "AUD"
assert (series.base, series.quote) == ("BTC", "AUD")
assert len(series.prices) == 7
def test_fetch_known_pair_sym_id(src, type, recent_id_id_ok, crypto_ok):
def test_fetch_known_pair_sym_id(src, type, recent_sym_id_ok, crypto_ok, fiat_ok):
series = src.fetch(Series("BTC", "ID=2782", type, "2021-01-01", "2021-01-07"))
req = recent_id_id_ok.calls[1].request
assert req.params["id"] == "1"
assert req.params["convertId"] == "2782"
req = recent_sym_id_ok.calls[0].request
assert req.params["symbol"] == "BTC"
assert req.params["convert_id"] == "2782"
assert (series.base, series.quote) == ("BTC", "AUD")
assert len(series.prices) == 7
def test_fetch_known_pair_sym_sym(src, type, recent_id_id_ok, crypto_ok):
def test_fetch_known_pair_sym_sym(src, type, recent_sym_sym_ok):
series = src.fetch(Series("BTC", "AUD", type, "2021-01-01", "2021-01-07"))
req = recent_id_id_ok.calls[1].request
assert req.params["id"] == "1"
assert req.params["convertId"] == "2782"
req = recent_sym_sym_ok.calls[0].request
assert req.params["symbol"] == "BTC"
assert req.params["convert"] == "AUD"
assert len(series.prices) == 7
def test_fetch_requests_and_receives_correct_times(
src, type, recent_id_id_ok, crypto_ok
src, type, recent_id_id_ok, crypto_ok, fiat_ok
):
series = src.fetch(Series("ID=1", "ID=2782", type, "2021-01-01", "2021-01-07"))
req = recent_id_id_ok.calls[0].request
assert req.params["timeStart"] == str(timestamp("2020-12-31")) # back one period
assert req.params["timeEnd"] == str(timestamp("2021-01-07"))
assert series.prices[0] == Price("2021-01-01", Decimal("37914.35060237985"))
assert series.prices[-1] == Price("2021-01-07", Decimal("49369.66288590665"))
assert req.params["time_start"] == str(timestamp("2020-12-31")) # back one period
assert req.params["time_end"] == str(timestamp("2021-01-07"))
assert series.prices[0] == Price("2021-01-01", Decimal("37914.350602379853"))
assert series.prices[-1] == Price("2021-01-07", Decimal("49370.064689585612"))
def test_fetch_requests_logged(src, type, crypto_ok, recent_id_id_ok, caplog):
def test_fetch_requests_logged(src, type, recent_sym_sym_ok, caplog):
with caplog.at_level(logging.DEBUG):
src.fetch(Series("BTC", "AUD", type, "2021-01-01", "2021-01-07"))
assert any(
@ -194,20 +262,20 @@ def test_fetch_requests_logged(src, type, crypto_ok, recent_id_id_ok, caplog):
)
def test_fetch_types_all_available(src, crypto_ok, recent_id_id_ok):
def test_fetch_types_all_available(src, recent_sym_sym_ok):
mid = src.fetch(Series("BTC", "AUD", "mid", "2021-01-01", "2021-01-07"))
opn = src.fetch(Series("BTC", "AUD", "open", "2021-01-01", "2021-01-07"))
hgh = src.fetch(Series("BTC", "AUD", "high", "2021-01-01", "2021-01-07"))
low = src.fetch(Series("BTC", "AUD", "low", "2021-01-01", "2021-01-07"))
cls = src.fetch(Series("BTC", "AUD", "close", "2021-01-01", "2021-01-07"))
assert mid.prices[0].amount == Decimal("37914.35060237985")
assert opn.prices[0].amount == Decimal("37658.1146368474")
assert mid.prices[0].amount == Decimal("37914.350602379853")
assert opn.prices[0].amount == Decimal("37658.83948707033")
assert hgh.prices[0].amount == Decimal("38417.9137031205")
assert low.prices[0].amount == Decimal("37410.7875016392")
assert cls.prices[0].amount == Decimal("38181.9913330076")
assert low.prices[0].amount == Decimal("37410.787501639206")
assert cls.prices[0].amount == Decimal("38181.99133300758")
def test_fetch_type_mid_is_mean_of_low_and_high(src, crypto_ok, recent_id_id_ok):
def test_fetch_type_mid_is_mean_of_low_and_high(src, recent_sym_sym_ok):
mid = src.fetch(Series("BTC", "AUD", "mid", "2021-01-01", "2021-01-07")).prices
low = src.fetch(Series("BTC", "AUD", "low", "2021-01-01", "2021-01-07")).prices
hgh = src.fetch(Series("BTC", "AUD", "high", "2021-01-01", "2021-01-07")).prices
@ -219,24 +287,80 @@ def test_fetch_type_mid_is_mean_of_low_and_high(src, crypto_ok, recent_id_id_ok)
)
def test_fetch_empty(src, type, crypto_ok, requests_mock):
def test_fetch_long_hist_from_start(src, type, long_sym_sym_ok):
series = src.fetch(Series("BTC", "AUD", type, src.start(), "2021-01-07"))
assert series.prices[0] == Price("2013-04-28", Decimal("130.45956234123247"))
assert series.prices[-1] == Price("2021-01-07", Decimal("49370.064689585612"))
assert len(series.prices) > 13
def test_fetch_from_before_start(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
status=400,
body="""{ "status": { "error_code": 400, "error_message":
"\\"time_start\\" must be a valid ISO 8601 timestamp or unix time value",
} }""",
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "AUD", type, "2001-09-10", "2001-10-01"))
assert "start date can't preceed" in str(e.value)
def test_fetch_to_future(src, type, recent_sym_sym_ok):
series = src.fetch(Series("BTC", "AUD", type, "2021-01-01", "2100-01-01"))
assert len(series.prices) > 0
def test_fetch_in_future(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
status=400,
body="""{
"status": {
"error_code": 400,
"error_message": "\\"time_start\\" must be older than \\"time_end\\"."
}
}""",
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "AUD", type, "2030-01-01", "2030-01-07"))
assert "start date must be in the past" in str(e.value)
def test_fetch_reversed_dates(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
status=400,
body="""{
"status": {
"error_code": 400,
"error_message": "\\"time_start\\" must be older than \\"time_end\\"."
}
}""",
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "AUD", type, "2021-01-07", "2021-01-01"))
assert "start date must preceed or match the end" in str(e.value)
def test_fetch_empty(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
body="""{
"status": {
"error_code": 0,
"error_message": null
},
"data": {
"id": 1,
"name": "Bitcoin",
"symbol": "BTC",
"timeEnd": "1228348799",
"quotes": []
},
"status": {
"timestamp": "2024-08-03T09:31:52.719Z",
"error_code": "0",
"error_message": "SUCCESS",
"elapsed": "14",
"credit_count": 0
}
}""",
)
@ -244,36 +368,63 @@ def test_fetch_empty(src, type, crypto_ok, requests_mock):
assert len(series.prices) == 0
def test_fetch_bad_base_sym(src, type, crypto_ok):
with pytest.raises(exceptions.InvalidPair) as e:
def test_fetch_bad_base_sym(src, type, requests_mock):
requests_mock.add(responses.GET, fetch_url, body='{"data":{}}')
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("NOTABASE", "USD", type, "2021-01-01", "2021-01-07"))
assert "Invalid symbol 'NOTABASE'" in str(e.value)
assert "quote currency symbol can't be found" in str(e.value)
assert "other reasons" in str(e.value)
def test_fetch_bad_quote_sym(src, type, crypto_ok):
with pytest.raises(exceptions.InvalidPair) as e:
src.fetch(Series("BTC", "NOTAQUOTE", type, "2021-01-01", "2021-01-07"))
assert "Invalid symbol 'NOTAQUOTE'" in str(e.value)
def test_fetch_bad_response(src, type, crypto_ok, requests_mock):
def test_fetch_bad_quote_sym(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
status=200,
status=400,
body="""{
"status": {
"timestamp": "2024-08-03T09:42:43.699Z",
"error_code": "500",
"error_message": "The system is busy, please try again later!",
"elapsed": "0",
"credit_count": 0
}
"status": {
"error_code": 400,
"error_message": "Invalid value for \\"convert\\": \\"NOTAQUOTE\\""
}
}""",
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("ID=987654321", "USD", type, "2021-01-01", "2021-01-07"))
assert "general error" in str(e.value)
with pytest.raises(exceptions.InvalidPair) as e:
src.fetch(Series("BTC", "NOTAQUOTE", type, "2021-01-01", "2021-01-07"))
assert "Bad quote symbol" in str(e.value)
def test_fetch_bad_base_id(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
status=400,
body="""{
"status": {
"error_code": 400,
"error_message": "No items found."
}
}""",
)
with pytest.raises(exceptions.InvalidPair) as e:
src.fetch(Series("ID=20000", "USD", type, "2021-01-01", "2021-01-07"))
assert "Bad base ID" in str(e.value)
def test_fetch_bad_quote_id(src, type, requests_mock):
requests_mock.add(
responses.GET,
fetch_url,
status=400,
body="""{
"status": {
"error_code": 400,
"error_message": "Invalid value for \\"convert_id\\": \\"20000\\""
}
}""",
)
with pytest.raises(exceptions.InvalidPair) as e:
src.fetch(Series("BTC", "ID=20000", type, "2021-01-01", "2021-01-07"))
assert "Bad quote ID" in str(e.value)
def test_fetch_no_quote(src, type):
@ -281,7 +432,7 @@ def test_fetch_no_quote(src, type):
src.fetch(Series("BTC", "", type, "2021-01-01", "2021-01-07"))
def test_fetch_network_issue(src, type, crypto_ok, requests_mock):
def test_fetch_network_issue(src, type, requests_mock):
body = requests.exceptions.ConnectionError("Network issue")
requests_mock.add(responses.GET, fetch_url, body=body)
with pytest.raises(exceptions.RequestError) as e:
@ -289,21 +440,21 @@ def test_fetch_network_issue(src, type, crypto_ok, requests_mock):
assert "Network issue" in str(e.value)
def test_fetch_bad_status(src, type, crypto_ok, requests_mock):
def test_fetch_bad_status(src, type, requests_mock):
requests_mock.add(responses.GET, fetch_url, status=500, body="Some other reason")
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("BTC", "AUD", type, "2021-01-01", "2021-01-07"))
assert "Internal Server Error" in str(e.value)
def test_fetch_parsing_error(src, type, crypto_ok, requests_mock):
def test_fetch_parsing_error(src, type, requests_mock):
requests_mock.add(responses.GET, fetch_url, body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("BTC", "AUD", type, "2021-01-01", "2021-01-07"))
assert "while parsing data" in str(e.value)
def test_fetch_unexpected_json(src, type, crypto_ok, requests_mock):
def test_fetch_unexpected_json(src, type, requests_mock):
requests_mock.add(responses.GET, fetch_url, body='{"notdata": []}')
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("BTC", "AUD", type, "2021-01-01", "2021-01-07"))

30
tests/pricehist/sources/test_coinmarketcap/fiat-partial.json Normal file
View file

@ -0,0 +1,30 @@
{
"status": {
"timestamp": "2021-07-16T10:08:13.272Z",
"error_code": 0,
"error_message": null,
"elapsed": 1,
"credit_count": 0,
"notice": null
},
"data": [
{
"id": 2781,
"name": "United States Dollar",
"sign": "$",
"symbol": "USD"
},
{
"id": 2782,
"name": "Australian Dollar",
"sign": "$",
"symbol": "AUD"
},
{
"id": 3575,
"name": "Gold Troy Ounce",
"symbol": "",
"code": "XAU"
}
]
}

255
tests/pricehist/sources/test_coinmarketcap/long-btc-aud-partial.json Normal file
View file

@ -0,0 +1,255 @@
{
"status": {
"timestamp": "2021-07-17T16:16:11.926Z",
"error_code": 0,
"error_message": null,
"elapsed": 2262,
"credit_count": 0,
"notice": null
},
"data": {
"id": 1,
"name": "Bitcoin",
"symbol": "BTC",
"quotes": [
{
"time_open": "2013-04-28T00:00:00.000Z",
"time_close": "2013-04-28T23:59:59.999Z",
"time_high": "2013-04-28T18:50:02.000Z",
"time_low": "2013-04-28T20:15:02.000Z",
"quote": {
"AUD": {
"open": null,
"high": 132.39216797540558,
"low": 128.52695670705936,
"close": 130.52908647526473,
"volume": 0,
"market_cap": 1447740447.626921,
"timestamp": "2013-04-28T23:59:00.000Z"
}
}
},
{
"time_open": "2013-04-29T00:00:00.000Z",
"time_close": "2013-04-29T23:59:59.999Z",
"time_high": "2013-04-29T13:15:01.000Z",
"time_low": "2013-04-29T05:20:01.000Z",
"quote": {
"AUD": {
"open": 130.75666236543535,
"high": 142.67970067891736,
"low": 129.9456943366951,
"close": 139.77370978254794,
"volume": 0,
"market_cap": 1550883729.329852,
"timestamp": "2013-04-29T23:59:00.000Z"
}
}
},
{
"time_open": "2013-04-30T00:00:00.000Z",
"time_close": "2013-04-30T23:59:59.999Z",
"time_high": "2013-04-30T08:25:02.000Z",
"time_low": "2013-04-30T18:55:01.000Z",
"quote": {
"AUD": {
"open": 139.2515230635335,
"high": 141.93391873626476,
"low": 129.37940647790543,
"close": 134.06635802469137,
"volume": 0,
"market_cap": 1488052782.6003087,
"timestamp": "2013-04-30T23:59:00.000Z"
}
}
},
{
"time_open": "2013-05-01T00:00:00.000Z",
"time_close": "2013-05-01T23:59:59.999Z",
"time_high": "2013-05-01T00:15:01.000Z",
"time_low": "2013-05-01T19:55:01.000Z",
"quote": {
"AUD": {
"open": 134.06635802469137,
"high": 134.88573849160971,
"low": 104.93911468163968,
"close": 113.79243056489595,
"volume": 0,
"market_cap": 1263451603.6864119,
"timestamp": "2013-05-01T23:59:00.000Z"
}
}
},
{
"time_open": "2013-05-02T00:00:00.000Z",
"time_close": "2013-05-02T23:59:59.999Z",
"time_high": "2013-05-02T14:25:01.000Z",
"time_low": "2013-05-02T14:30:02.000Z",
"quote": {
"AUD": {
"open": 113.19910247390133,
"high": 122.60835462135991,
"low": 90.08385249759387,
"close": 102.63388848353591,
"volume": 0,
"market_cap": 1139905858.2089553,
"timestamp": "2013-05-02T23:59:00.000Z"
}
}
},
{
"time_open": "2013-05-03T00:00:00.000Z",
"time_close": "2013-05-03T23:59:59.999Z",
"time_high": "2013-05-03T05:30:02.000Z",
"time_low": "2013-05-03T03:05:01.000Z",
"quote": {
"AUD": {
"open": 103.64842454394694,
"high": 105.43929629649027,
"low": 77.03544845551335,
"close": 94.77409346519293,
"volume": 0,
"market_cap": 1052933070.3412836,
"timestamp": "2013-05-03T23:59:00.000Z"
}
}
},
{
"time_open": "2013-05-04T00:00:00.000Z",
"time_close": "2013-05-04T23:59:59.999Z",
"time_high": "2013-05-04T07:15:01.000Z",
"time_low": "2013-05-04T06:50:01.000Z",
"quote": {
"AUD": {
"open": 95.11343656595025,
"high": 111.49893348846227,
"low": 89.68392476245879,
"close": 109.07504363001745,
"volume": 0,
"market_cap": 1212251854.2757416,
"timestamp": "2013-05-04T23:59:00.000Z"
}
}
},
{
"time_open": "2021-01-01T00:00:00.000Z",
"time_close": "2021-01-01T23:59:59.999Z",
"time_high": "2021-01-01T12:38:43.000Z",
"time_low": "2021-01-01T00:16:43.000Z",
"quote": {
"AUD": {
"open": 37658.83948707033,
"high": 38417.9137031205,
"low": 37410.787501639206,
"close": 38181.99133300758,
"volume": 52943282221.028366,
"market_cap": 709720173049.5383,
"timestamp": "2021-01-01T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-02T00:00:00.000Z",
"time_close": "2021-01-02T23:59:59.999Z",
"time_high": "2021-01-02T19:49:42.000Z",
"time_low": "2021-01-02T00:31:44.000Z",
"quote": {
"AUD": {
"open": 38184.98611600682,
"high": 43096.681197423015,
"low": 37814.17187096531,
"close": 41760.62923079505,
"volume": 88214867181.97835,
"market_cap": 776278147177.8037,
"timestamp": "2021-01-02T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-03T00:00:00.000Z",
"time_close": "2021-01-03T23:59:59.999Z",
"time_high": "2021-01-03T07:47:38.000Z",
"time_low": "2021-01-03T00:20:45.000Z",
"quote": {
"AUD": {
"open": 41763.41015117659,
"high": 44985.93247585023,
"low": 41663.204350601605,
"close": 42511.10646879765,
"volume": 102011582370.28117,
"market_cap": 790270288834.0249,
"timestamp": "2021-01-03T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-04T00:00:00.000Z",
"time_close": "2021-01-04T23:59:59.999Z",
"time_high": "2021-01-04T04:07:42.000Z",
"time_low": "2021-01-04T10:19:42.000Z",
"quote": {
"AUD": {
"open": 42548.61349648768,
"high": 43360.96165147421,
"low": 37133.98436952697,
"close": 41686.38761359174,
"volume": 105824510346.65779,
"market_cap": 774984045201.7122,
"timestamp": "2021-01-04T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-05T00:00:00.000Z",
"time_close": "2021-01-05T23:59:59.999Z",
"time_high": "2021-01-05T22:44:35.000Z",
"time_low": "2021-01-05T06:16:41.000Z",
"quote": {
"AUD": {
"open": 41693.07321807638,
"high": 44403.79487147647,
"low": 39221.81167941294,
"close": 43790.067253370056,
"volume": 87016490203.50436,
"market_cap": 814135603090.2502,
"timestamp": "2021-01-05T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-06T00:00:00.000Z",
"time_close": "2021-01-06T23:59:59.999Z",
"time_high": "2021-01-06T23:57:36.000Z",
"time_low": "2021-01-06T00:25:38.000Z",
"quote": {
"AUD": {
"open": 43817.35864984641,
"high": 47186.65232598287,
"low": 43152.60281764236,
"close": 47115.85365360005,
"volume": 96330948324.8061,
"market_cap": 876019742889.9551,
"timestamp": "2021-01-06T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-07T00:00:00.000Z",
"time_close": "2021-01-07T23:59:59.999Z",
"time_high": "2021-01-07T18:17:42.000Z",
"time_low": "2021-01-07T08:25:51.000Z",
"quote": {
"AUD": {
"open": 47128.02139328098,
"high": 51833.478207775144,
"low": 46906.65117139608,
"close": 50686.90986207153,
"volume": 109124136558.20264,
"market_cap": 942469208700.134,
"timestamp": "2021-01-07T23:59:06.000Z"
}
}
}
]
}
}

136
tests/pricehist/sources/test_coinmarketcap/recent-btc-aud.json Normal file
View file

@ -0,0 +1,136 @@
{
"status": {
"timestamp": "2021-07-16T10:42:32.013Z",
"error_code": 0,
"error_message": null,
"elapsed": 20,
"credit_count": 0,
"notice": null
},
"data": {
"id": 1,
"name": "Bitcoin",
"symbol": "BTC",
"quotes": [
{
"time_open": "2021-01-01T00:00:00.000Z",
"time_close": "2021-01-01T23:59:59.999Z",
"time_high": "2021-01-01T12:38:43.000Z",
"time_low": "2021-01-01T00:16:43.000Z",
"quote": {
"AUD": {
"open": 37658.83948707033,
"high": 38417.9137031205,
"low": 37410.787501639206,
"close": 38181.99133300758,
"volume": 52943282221.028366,
"market_cap": 709720173049.5383,
"timestamp": "2021-01-01T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-02T00:00:00.000Z",
"time_close": "2021-01-02T23:59:59.999Z",
"time_high": "2021-01-02T19:49:42.000Z",
"time_low": "2021-01-02T00:31:44.000Z",
"quote": {
"AUD": {
"open": 38184.98611600682,
"high": 43096.681197423015,
"low": 37814.17187096531,
"close": 41760.62923079505,
"volume": 88214867181.97835,
"market_cap": 776278147177.8037,
"timestamp": "2021-01-02T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-03T00:00:00.000Z",
"time_close": "2021-01-03T23:59:59.999Z",
"time_high": "2021-01-03T07:47:38.000Z",
"time_low": "2021-01-03T00:20:45.000Z",
"quote": {
"AUD": {
"open": 41763.41015117659,
"high": 44985.93247585023,
"low": 41663.204350601605,
"close": 42511.10646879765,
"volume": 102011582370.28117,
"market_cap": 790270288834.0249,
"timestamp": "2021-01-03T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-04T00:00:00.000Z",
"time_close": "2021-01-04T23:59:59.999Z",
"time_high": "2021-01-04T04:07:42.000Z",
"time_low": "2021-01-04T10:19:42.000Z",
"quote": {
"AUD": {
"open": 42548.61349648768,
"high": 43360.96165147421,
"low": 37133.98436952697,
"close": 41686.38761359174,
"volume": 105824510346.65779,
"market_cap": 774984045201.7122,
"timestamp": "2021-01-04T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-05T00:00:00.000Z",
"time_close": "2021-01-05T23:59:59.999Z",
"time_high": "2021-01-05T22:44:35.000Z",
"time_low": "2021-01-05T06:16:41.000Z",
"quote": {
"AUD": {
"open": 41693.07321807638,
"high": 44403.79487147647,
"low": 39221.81167941294,
"close": 43790.067253370056,
"volume": 87016490203.50436,
"market_cap": 814135603090.2502,
"timestamp": "2021-01-05T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-06T00:00:00.000Z",
"time_close": "2021-01-06T23:59:59.999Z",
"time_high": "2021-01-06T23:57:36.000Z",
"time_low": "2021-01-06T00:25:38.000Z",
"quote": {
"AUD": {
"open": 43817.35864984641,
"high": 47186.65232598287,
"low": 43152.60281764236,
"close": 47115.85365360005,
"volume": 96330948324.8061,
"market_cap": 876019742889.9551,
"timestamp": "2021-01-06T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-07T00:00:00.000Z",
"time_close": "2021-01-07T23:59:59.999Z",
"time_high": "2021-01-07T18:17:42.000Z",
"time_low": "2021-01-07T08:25:51.000Z",
"quote": {
"AUD": {
"open": 47128.02139328098,
"high": 51833.478207775144,
"low": 46906.65117139608,
"close": 50686.90986207153,
"volume": 109124136558.20264,
"market_cap": 942469208700.134,
"timestamp": "2021-01-07T23:59:06.000Z"
}
}
}
]
}
}

136
tests/pricehist/sources/test_coinmarketcap/recent-btc-id2782.json Normal file
View file

@ -0,0 +1,136 @@
{
"status": {
"timestamp": "2021-07-16T10:42:27.169Z",
"error_code": 0,
"error_message": null,
"elapsed": 19,
"credit_count": 0,
"notice": null
},
"data": {
"id": 1,
"name": "Bitcoin",
"symbol": "BTC",
"quotes": [
{
"time_open": "2021-01-01T00:00:00.000Z",
"time_close": "2021-01-01T23:59:59.999Z",
"time_high": "2021-01-01T12:38:43.000Z",
"time_low": "2021-01-01T00:16:43.000Z",
"quote": {
"2782": {
"open": 37658.83948707033,
"high": 38417.9137031205,
"low": 37410.787501639206,
"close": 38181.99133300758,
"volume": 52943282221.028366,
"market_cap": 709720173049.5383,
"timestamp": "2021-01-01T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-02T00:00:00.000Z",
"time_close": "2021-01-02T23:59:59.999Z",
"time_high": "2021-01-02T19:49:42.000Z",
"time_low": "2021-01-02T00:31:44.000Z",
"quote": {
"2782": {
"open": 38184.98611600682,
"high": 43096.681197423015,
"low": 37814.17187096531,
"close": 41760.62923079505,
"volume": 88214867181.97835,
"market_cap": 776278147177.8037,
"timestamp": "2021-01-02T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-03T00:00:00.000Z",
"time_close": "2021-01-03T23:59:59.999Z",
"time_high": "2021-01-03T07:47:38.000Z",
"time_low": "2021-01-03T00:20:45.000Z",
"quote": {
"2782": {
"open": 41763.41015117659,
"high": 44985.93247585023,
"low": 41663.204350601605,
"close": 42511.10646879765,
"volume": 102011582370.28117,
"market_cap": 790270288834.0249,
"timestamp": "2021-01-03T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-04T00:00:00.000Z",
"time_close": "2021-01-04T23:59:59.999Z",
"time_high": "2021-01-04T04:07:42.000Z",
"time_low": "2021-01-04T10:19:42.000Z",
"quote": {
"2782": {
"open": 42548.61349648768,
"high": 43360.96165147421,
"low": 37133.98436952697,
"close": 41686.38761359174,
"volume": 105824510346.65779,
"market_cap": 774984045201.7122,
"timestamp": "2021-01-04T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-05T00:00:00.000Z",
"time_close": "2021-01-05T23:59:59.999Z",
"time_high": "2021-01-05T22:44:35.000Z",
"time_low": "2021-01-05T06:16:41.000Z",
"quote": {
"2782": {
"open": 41693.07321807638,
"high": 44403.79487147647,
"low": 39221.81167941294,
"close": 43790.067253370056,
"volume": 87016490203.50436,
"market_cap": 814135603090.2502,
"timestamp": "2021-01-05T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-06T00:00:00.000Z",
"time_close": "2021-01-06T23:59:59.999Z",
"time_high": "2021-01-06T23:57:36.000Z",
"time_low": "2021-01-06T00:25:38.000Z",
"quote": {
"2782": {
"open": 43817.35864984641,
"high": 47186.65232598287,
"low": 43152.60281764236,
"close": 47115.85365360005,
"volume": 96330948324.8061,
"market_cap": 876019742889.9551,
"timestamp": "2021-01-06T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-07T00:00:00.000Z",
"time_close": "2021-01-07T23:59:59.999Z",
"time_high": "2021-01-07T18:17:42.000Z",
"time_low": "2021-01-07T08:25:51.000Z",
"quote": {
"2782": {
"open": 47128.02139328098,
"high": 51833.478207775144,
"low": 46906.65117139608,
"close": 50686.90986207153,
"volume": 109124136558.20264,
"market_cap": 942469208700.134,
"timestamp": "2021-01-07T23:59:06.000Z"
}
}
}
]
}
}

136
tests/pricehist/sources/test_coinmarketcap/recent-id1-aud.json Normal file
View file

@ -0,0 +1,136 @@
{
"status": {
"timestamp": "2021-07-16T10:42:24.612Z",
"error_code": 0,
"error_message": null,
"elapsed": 57,
"credit_count": 0,
"notice": null
},
"data": {
"id": 1,
"name": "Bitcoin",
"symbol": "BTC",
"quotes": [
{
"time_open": "2021-01-01T00:00:00.000Z",
"time_close": "2021-01-01T23:59:59.999Z",
"time_high": "2021-01-01T12:38:43.000Z",
"time_low": "2021-01-01T00:16:43.000Z",
"quote": {
"AUD": {
"open": 37658.83948707033,
"high": 38417.9137031205,
"low": 37410.787501639206,
"close": 38181.99133300758,
"volume": 52943282221.028366,
"market_cap": 709720173049.5383,
"timestamp": "2021-01-01T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-02T00:00:00.000Z",
"time_close": "2021-01-02T23:59:59.999Z",
"time_high": "2021-01-02T19:49:42.000Z",
"time_low": "2021-01-02T00:31:44.000Z",
"quote": {
"AUD": {
"open": 38184.98611600682,
"high": 43096.681197423015,
"low": 37814.17187096531,
"close": 41760.62923079505,
"volume": 88214867181.97835,
"market_cap": 776278147177.8037,
"timestamp": "2021-01-02T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-03T00:00:00.000Z",
"time_close": "2021-01-03T23:59:59.999Z",
"time_high": "2021-01-03T07:47:38.000Z",
"time_low": "2021-01-03T00:20:45.000Z",
"quote": {
"AUD": {
"open": 41763.41015117659,
"high": 44985.93247585023,
"low": 41663.204350601605,
"close": 42511.10646879765,
"volume": 102011582370.28117,
"market_cap": 790270288834.0249,
"timestamp": "2021-01-03T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-04T00:00:00.000Z",
"time_close": "2021-01-04T23:59:59.999Z",
"time_high": "2021-01-04T04:07:42.000Z",
"time_low": "2021-01-04T10:19:42.000Z",
"quote": {
"AUD": {
"open": 42548.61349648768,
"high": 43360.96165147421,
"low": 37133.98436952697,
"close": 41686.38761359174,
"volume": 105824510346.65779,
"market_cap": 774984045201.7122,
"timestamp": "2021-01-04T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-05T00:00:00.000Z",
"time_close": "2021-01-05T23:59:59.999Z",
"time_high": "2021-01-05T22:44:35.000Z",
"time_low": "2021-01-05T06:16:41.000Z",
"quote": {
"AUD": {
"open": 41693.07321807638,
"high": 44403.79487147647,
"low": 39221.81167941294,
"close": 43790.067253370056,
"volume": 87016490203.50436,
"market_cap": 814135603090.2502,
"timestamp": "2021-01-05T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-06T00:00:00.000Z",
"time_close": "2021-01-06T23:59:59.999Z",
"time_high": "2021-01-06T23:57:36.000Z",
"time_low": "2021-01-06T00:25:38.000Z",
"quote": {
"AUD": {
"open": 43817.35864984641,
"high": 47186.65232598287,
"low": 43152.60281764236,
"close": 47115.85365360005,
"volume": 96330948324.8061,
"market_cap": 876019742889.9551,
"timestamp": "2021-01-06T23:59:06.000Z"
}
}
},
{
"time_open": "2021-01-07T00:00:00.000Z",
"time_close": "2021-01-07T23:59:59.999Z",
"time_high": "2021-01-07T18:17:42.000Z",
"time_low": "2021-01-07T08:25:51.000Z",
"quote": {
"AUD": {
"open": 47128.02139328098,
"high": 51833.478207775144,
"low": 46906.65117139608,
"close": 50686.90986207153,
"volume": 109124136558.20264,
"market_cap": 942469208700.134,
"timestamp": "2021-01-07T23:59:06.000Z"
}
}
}
]
}
}

191
tests/pricehist/sources/test_coinmarketcap/recent-id1-id2782.json
View file

@ -1,129 +1,136 @@
{
"status": {
"timestamp": "2021-07-16T10:42:21.065Z",
"error_code": 0,
"error_message": null,
"elapsed": 17,
"credit_count": 0,
"notice": null
},
"data": {
"id": 1,
"name": "Bitcoin",
"symbol": "BTC",
"timeEnd": "1575503999",
"quotes": [
{
"timeOpen": "2021-01-01T00:00:00.000Z",
"timeClose": "2021-01-01T23:59:59.999Z",
"timeHigh": "2021-01-01T12:38:43.000Z",
"timeLow": "2021-01-01T00:16:43.000Z",
"time_open": "2021-01-01T00:00:00.000Z",
"time_close": "2021-01-01T23:59:59.999Z",
"time_high": "2021-01-01T12:38:43.000Z",
"time_low": "2021-01-01T00:16:43.000Z",
"quote": {
"name": "2782",
"open": 37658.1146368474,
"high": 38417.9137031205,
"low": 37410.7875016392,
"close": 38181.9913330076,
"volume": 52901492931.8344367080,
"marketCap": 709159975413.2388897949,
"timestamp": "2021-01-01T23:59:59.999Z"
"2782": {
"open": 37658.83948707033,
"high": 38417.9137031205,
"low": 37410.787501639206,
"close": 38181.99133300758,
"volume": 52943282221.028366,
"market_cap": 709720173049.5383,
"timestamp": "2021-01-01T23:59:06.000Z"
}
}
},
{
"timeOpen": "2021-01-02T00:00:00.000Z",
"timeClose": "2021-01-02T23:59:59.999Z",
"timeHigh": "2021-01-02T19:49:42.000Z",
"timeLow": "2021-01-02T00:31:44.000Z",
"time_open": "2021-01-02T00:00:00.000Z",
"time_close": "2021-01-02T23:59:59.999Z",
"time_high": "2021-01-02T19:49:42.000Z",
"time_low": "2021-01-02T00:31:44.000Z",
"quote": {
"name": "2782",
"open": 38184.9861160068,
"high": 43096.6811974230,
"low": 37814.1718709653,
"close": 41760.6292307951,
"volume": 88214867181.9830439141,
"marketCap": 776278147177.8037261338,
"timestamp": "2021-01-02T23:59:59.999Z"
"2782": {
"open": 38184.98611600682,
"high": 43096.681197423015,
"low": 37814.17187096531,
"close": 41760.62923079505,
"volume": 88214867181.97835,
"market_cap": 776278147177.8037,
"timestamp": "2021-01-02T23:59:06.000Z"
}
}
},
{
"timeOpen": "2021-01-03T00:00:00.000Z",
"timeClose": "2021-01-03T23:59:59.999Z",
"timeHigh": "2021-01-03T07:47:38.000Z",
"timeLow": "2021-01-03T00:20:45.000Z",
"time_open": "2021-01-03T00:00:00.000Z",
"time_close": "2021-01-03T23:59:59.999Z",
"time_high": "2021-01-03T07:47:38.000Z",
"time_low": "2021-01-03T00:20:45.000Z",
"quote": {
"name": "2782",
"open": 41763.4101511766,
"high": 44985.9324758502,
"low": 41663.2043506016,
"close": 42534.0538859236,
"volume": 102253005977.1115650988,
"marketCap": 792140565709.1701340036,
"timestamp": "2021-01-03T23:59:59.999Z"
"2782": {
"open": 41763.41015117659,
"high": 44985.93247585023,
"low": 41663.204350601605,
"close": 42511.10646879765,
"volume": 102011582370.28117,
"market_cap": 790270288834.0249,
"timestamp": "2021-01-03T23:59:06.000Z"
}
}
},
{
"timeOpen": "2021-01-04T00:00:00.000Z",
"timeClose": "2021-01-04T23:59:59.999Z",
"timeHigh": "2021-01-04T04:07:42.000Z",
"timeLow": "2021-01-04T10:19:42.000Z",
"time_open": "2021-01-04T00:00:00.000Z",
"time_close": "2021-01-04T23:59:59.999Z",
"time_high": "2021-01-04T04:07:42.000Z",
"time_low": "2021-01-04T10:19:42.000Z",
"quote": {
"name": "2782",
"open": 42548.6134964877,
"high": 43347.7527651400,
"low": 37111.8678479690,
"close": 41707.4890765162,
"volume": 105251252720.3013091567,
"marketCap": 770785910830.3801120744,
"timestamp": "2021-01-04T23:59:59.999Z"
"2782": {
"open": 42548.61349648768,
"high": 43360.96165147421,
"low": 37133.98436952697,
"close": 41686.38761359174,
"volume": 105824510346.65779,
"market_cap": 774984045201.7122,
"timestamp": "2021-01-04T23:59:06.000Z"
}
}
},
{
"timeOpen": "2021-01-05T00:00:00.000Z",
"timeClose": "2021-01-05T23:59:59.999Z",
"timeHigh": "2021-01-05T22:44:35.000Z",
"timeLow": "2021-01-05T06:16:41.000Z",
"time_open": "2021-01-05T00:00:00.000Z",
"time_close": "2021-01-05T23:59:59.999Z",
"time_high": "2021-01-05T22:44:35.000Z",
"time_low": "2021-01-05T06:16:41.000Z",
"quote": {
"name": "2782",
"open": 41693.0732180764,
"high": 44406.6531914952,
"low": 39220.9654861842,
"close": 43777.4560620835,
"volume": 88071174132.6445648582,
"marketCap": 824003338903.4613958343,
"timestamp": "2021-01-05T23:59:59.999Z"
"2782": {
"open": 41693.07321807638,
"high": 44403.79487147647,
"low": 39221.81167941294,
"close": 43790.067253370056,
"volume": 87016490203.50436,
"market_cap": 814135603090.2502,
"timestamp": "2021-01-05T23:59:06.000Z"
}
}
},
{
"timeOpen": "2021-01-06T00:00:00.000Z",
"timeClose": "2021-01-06T23:59:59.999Z",
"timeHigh": "2021-01-06T23:57:36.000Z",
"timeLow": "2021-01-06T00:25:38.000Z",
"time_open": "2021-01-06T00:00:00.000Z",
"time_close": "2021-01-06T23:59:59.999Z",
"time_high": "2021-01-06T23:57:36.000Z",
"time_low": "2021-01-06T00:25:38.000Z",
"quote": {
"name": "2782",
"open": 43798.3790529373,
"high": 47185.7303335186,
"low": 43152.6028176424,
"close": 47114.9330444897,
"volume": 96948095813.7503737302,
"marketCap": 881631993096.0701475336,
"timestamp": "2021-01-06T23:59:59.999Z"
"2782": {
"open": 43817.35864984641,
"high": 47186.65232598287,
"low": 43152.60281764236,
"close": 47115.85365360005,
"volume": 96330948324.8061,
"market_cap": 876019742889.9551,
"timestamp": "2021-01-06T23:59:06.000Z"
}
}
},
{
"timeOpen": "2021-01-07T00:00:00.000Z",
"timeClose": "2021-01-07T23:59:59.999Z",
"timeHigh": "2021-01-07T18:17:42.000Z",
"timeLow": "2021-01-07T08:25:51.000Z",
"time_open": "2021-01-07T00:00:00.000Z",
"time_close": "2021-01-07T23:59:59.999Z",
"time_high": "2021-01-07T18:17:42.000Z",
"time_low": "2021-01-07T08:25:51.000Z",
"quote": {
"name": "2782",
"open": 47128.0213932810,
"high": 51832.6746004172,
"low": 46906.6511713961,
"close": 50660.9643451606,
"volume": 108451040396.2660095877,
"marketCap": 936655898949.2177196744,
"timestamp": "2021-01-07T23:59:59.999Z"
"2782": {
"open": 47128.02139328098,
"high": 51833.478207775144,
"low": 46906.65117139608,
"close": 50686.90986207153,
"volume": 109124136558.20264,
"market_cap": 942469208700.134,
"timestamp": "2021-01-07T23:59:06.000Z"
}
}
}
]
},
"status": {
"timestamp": "2024-08-02T18:23:21.586Z",
"error_code": "0",
"error_message": "SUCCESS",
"elapsed": "212",
"credit_count": 0
}
}

181
tests/pricehist/sources/test_yahoo.py
View file

@ -36,28 +36,31 @@ def requests_mock():
yield mock
def url(base):
return f"https://query1.finance.yahoo.com/v8/finance/chart/{base}"
spark_url = "https://query1.finance.yahoo.com/v7/finance/spark"
def history_url(base):
return f"https://query1.finance.yahoo.com/v7/finance/download/{base}"
@pytest.fixture
def spark_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "tsla-spark.json").read_text()
requests_mock.add(responses.GET, spark_url, body=json, status=200)
yield requests_mock
@pytest.fixture
def recent_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "tsla-recent.json").read_text()
requests_mock.add(responses.GET, url("TSLA"), body=json, status=200)
json = (Path(os.path.splitext(__file__)[0]) / "tsla-recent.csv").read_text()
requests_mock.add(responses.GET, history_url("TSLA"), body=json, status=200)
yield requests_mock
@pytest.fixture
def long_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "ibm-long-partial.json").read_text()
requests_mock.add(responses.GET, url("IBM"), body=json, status=200)
yield requests_mock
@pytest.fixture
def with_null_ok(requests_mock):
json = (Path(os.path.splitext(__file__)[0]) / "inrx-with-null.json").read_text()
requests_mock.add(responses.GET, url("INR=X"), body=json, status=200)
json = (Path(os.path.splitext(__file__)[0]) / "ibm-long-partial.csv").read_text()
requests_mock.add(responses.GET, history_url("IBM"), body=json, status=200)
yield requests_mock
@ -95,57 +98,53 @@ def test_symbols(src, caplog):
assert any(["Find the symbol of interest on" in r.message for r in caplog.records])
def test_fetch_known(src, type, recent_ok):
def test_fetch_known(src, type, spark_ok, recent_ok):
series = src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
req = recent_ok.calls[0].request
assert req.params["events"] == "capitalGain%7Cdiv%7Csplit"
assert req.params["includeAdjustedClose"] == "true"
spark_req = recent_ok.calls[0].request
hist_req = recent_ok.calls[1].request
assert spark_req.params["symbols"] == "TSLA"
assert hist_req.params["events"] == "history"
assert hist_req.params["includeAdjustedClose"] == "true"
assert (series.base, series.quote) == ("TSLA", "USD")
assert len(series.prices) == 5
def test_fetch_requests_and_receives_correct_times(src, type, recent_ok):
def test_fetch_requests_and_receives_correct_times(src, type, spark_ok, recent_ok):
series = src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
req = recent_ok.calls[0].request
assert req.params["period1"] == str(timestamp("2021-01-04"))
assert req.params["period2"] == str(timestamp("2021-01-09")) # rounded up one
assert req.params["interval"] == "1d"
assert series.prices[0] == Price("2021-01-04", Decimal("243.2566680908203125"))
assert series.prices[-1] == Price("2021-01-08", Decimal("293.339996337890625"))
hist_req = recent_ok.calls[1].request
assert hist_req.params["period1"] == str(timestamp("2021-01-04"))
assert hist_req.params["period2"] == str(timestamp("2021-01-09")) # rounded up one
assert hist_req.params["interval"] == "1d"
assert series.prices[0] == Price("2021-01-04", Decimal("729.770020"))
assert series.prices[-1] == Price("2021-01-08", Decimal("880.020020"))
def test_fetch_ignores_any_extra_row(src, type, recent_ok):
series = src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-07"))
assert series.prices[0] == Price("2021-01-04", Decimal("243.2566680908203125"))
assert series.prices[-1] == Price("2021-01-07", Decimal("272.013336181640625"))
def test_fetch_requests_logged(src, type, recent_ok, caplog):
def test_fetch_requests_logged(src, type, spark_ok, recent_ok, caplog):
with caplog.at_level(logging.DEBUG):
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
logged_requests = 0
for r in caplog.records:
if r.levelname == "DEBUG" and "curl " in r.message:
logged_requests += 1
assert logged_requests == 1
assert logged_requests == 2
def test_fetch_types_all_available(src, recent_ok):
def test_fetch_types_all_available(src, spark_ok, recent_ok):
adj = src.fetch(Series("TSLA", "", "adjclose", "2021-01-04", "2021-01-08"))
opn = src.fetch(Series("TSLA", "", "open", "2021-01-04", "2021-01-08"))
hgh = src.fetch(Series("TSLA", "", "high", "2021-01-04", "2021-01-08"))
low = src.fetch(Series("TSLA", "", "low", "2021-01-04", "2021-01-08"))
cls = src.fetch(Series("TSLA", "", "close", "2021-01-04", "2021-01-08"))
mid = src.fetch(Series("TSLA", "", "mid", "2021-01-04", "2021-01-08"))
assert adj.prices[0].amount == Decimal("243.2566680908203125")
assert opn.prices[0].amount == Decimal("239.82000732421875")
assert hgh.prices[0].amount == Decimal("248.163330078125")
assert low.prices[0].amount == Decimal("239.0633392333984375")
assert cls.prices[0].amount == Decimal("243.2566680908203125")
assert mid.prices[0].amount == Decimal("243.61333465576171875")
assert adj.prices[0].amount == Decimal("729.770020")
assert opn.prices[0].amount == Decimal("719.460022")
assert hgh.prices[0].amount == Decimal("744.489990")
assert low.prices[0].amount == Decimal("717.190002")
assert cls.prices[0].amount == Decimal("729.770020")
assert mid.prices[0].amount == Decimal("730.839996")
def test_fetch_type_mid_is_mean_of_low_and_high(src, recent_ok):
def test_fetch_type_mid_is_mean_of_low_and_high(src, spark_ok, recent_ok):
mid = src.fetch(Series("TSLA", "", "mid", "2021-01-04", "2021-01-08")).prices
hgh = src.fetch(Series("TSLA", "", "high", "2021-01-04", "2021-01-08")).prices
low = src.fetch(Series("TSLA", "", "low", "2021-01-04", "2021-01-08")).prices
@ -157,29 +156,22 @@ def test_fetch_type_mid_is_mean_of_low_and_high(src, recent_ok):
)
def test_fetch_from_before_start(src, type, long_ok):
def test_fetch_from_before_start(src, type, spark_ok, long_ok):
series = src.fetch(Series("IBM", "", type, "1900-01-01", "2021-01-08"))
assert series.prices[0] == Price("1962-01-02", Decimal("1.5133211612701416015625"))
assert series.prices[-1] == Price("2021-01-08", Decimal("103.2923736572265625"))
assert series.prices[0] == Price("1962-01-02", Decimal("1.837710"))
assert series.prices[-1] == Price("2021-01-08", Decimal("125.433624"))
assert len(series.prices) > 9
def test_fetch_skips_dates_with_nulls(src, type, with_null_ok):
series = src.fetch(Series("INR=X", "", type, "2017-07-10", "2017-07-12"))
assert series.prices[0] == Price("2017-07-10", Decimal("64.61170196533203125"))
assert series.prices[1] == Price("2017-07-12", Decimal("64.52559661865234375"))
assert len(series.prices) == 2
def test_fetch_to_future(src, type, recent_ok):
def test_fetch_to_future(src, type, spark_ok, recent_ok):
series = src.fetch(Series("TSLA", "", type, "2021-01-04", "2100-01-08"))
assert len(series.prices) > 0
def test_fetch_no_data_in_past(src, type, requests_mock):
def test_fetch_no_data_in_past(src, type, spark_ok, requests_mock):
requests_mock.add(
responses.GET,
url("TSLA"),
history_url("TSLA"),
status=400,
body=(
"400 Bad Request: Data doesn't exist for "
@ -191,10 +183,10 @@ def test_fetch_no_data_in_past(src, type, requests_mock):
assert "No data for the given interval" in str(e.value)
def test_fetch_no_data_in_future(src, type, requests_mock):
def test_fetch_no_data_in_future(src, type, spark_ok, requests_mock):
requests_mock.add(
responses.GET,
url("TSLA"),
history_url("TSLA"),
status=400,
body=(
"400 Bad Request: Data doesn't exist for "
@ -206,10 +198,10 @@ def test_fetch_no_data_in_future(src, type, requests_mock):
assert "No data for the given interval" in str(e.value)
def test_fetch_no_data_on_weekend(src, type, requests_mock):
def test_fetch_no_data_on_weekend(src, type, spark_ok, requests_mock):
requests_mock.add(
responses.GET,
url("TSLA"),
history_url("TSLA"),
status=404,
body="404 Not Found: Timestamp data missing.",
)
@ -221,7 +213,30 @@ def test_fetch_no_data_on_weekend(src, type, requests_mock):
def test_fetch_bad_sym(src, type, requests_mock):
requests_mock.add(
responses.GET,
url("NOTABASE"),
spark_url,
status=404,
body="""{
"spark": {
"result": null,
"error": {
"code": "Not Found",
"description": "No data found for spark symbols"
}
}
}""",
)
with pytest.raises(exceptions.InvalidPair) as e:
src.fetch(Series("NOTABASE", "", type, "2021-01-04", "2021-01-08"))
assert "Symbol not found" in str(e.value)
def test_fetch_bad_sym_history(src, type, spark_ok, requests_mock):
# In practice the spark history requests should succeed or fail together.
# This extra test ensures that a failure of the the history part is handled
# correctly even if the spark part succeeds.
requests_mock.add(
responses.GET,
history_url("NOTABASE"),
status=404,
body="404 Not Found: No data found, symbol may be delisted",
)
@ -236,23 +251,61 @@ def test_fetch_giving_quote(src, type):
assert "quote currency" in str(e.value)
def test_fetch_network_issue(src, type, requests_mock):
def test_fetch_spark_network_issue(src, type, requests_mock):
body = requests.exceptions.ConnectionError("Network issue")
requests_mock.add(responses.GET, url("TSLA"), body=body)
requests_mock.add(responses.GET, spark_url, body=body)
with pytest.raises(exceptions.RequestError) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "Network issue" in str(e.value)
def test_fetch_bad_status(src, type, requests_mock):
requests_mock.add(responses.GET, url("TSLA"), status=500, body="Some other reason")
def test_fetch_spark_bad_status(src, type, requests_mock):
requests_mock.add(responses.GET, spark_url, status=500, body="Some other reason")
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "Internal Server Error" in str(e.value)
def test_fetch_parsing_error(src, type, requests_mock):
requests_mock.add(responses.GET, url("TSLA"), body="")
def test_fetch_spark_parsing_error(src, type, requests_mock):
requests_mock.add(responses.GET, spark_url, body="NOT JSON")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "spark data couldn't be parsed" in str(e.value)
def test_fetch_spark_unexpected_json(src, type, requests_mock):
requests_mock.add(responses.GET, spark_url, body='{"notdata": []}')
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "spark data couldn't be parsed" in str(e.value)
def test_fetch_history_network_issue(src, type, spark_ok, requests_mock):
body = requests.exceptions.ConnectionError("Network issue")
requests_mock.add(responses.GET, history_url("TSLA"), body=body)
with pytest.raises(exceptions.RequestError) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "Network issue" in str(e.value)
def test_fetch_history_bad_status(src, type, spark_ok, requests_mock):
requests_mock.add(
responses.GET, history_url("TSLA"), status=500, body="Some other reason"
)
with pytest.raises(exceptions.BadResponse) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "Internal Server Error" in str(e.value)
def test_fetch_history_parsing_error(src, type, spark_ok, requests_mock):
requests_mock.add(responses.GET, history_url("TSLA"), body="")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "error occurred while parsing data from the source" in str(e.value)
def test_fetch_history_unexpected_csv_format(src, type, spark_ok, requests_mock):
requests_mock.add(responses.GET, history_url("TSLA"), body="BAD HEADER\nBAD DATA")
with pytest.raises(exceptions.ResponseParsingError) as e:
src.fetch(Series("TSLA", "", type, "2021-01-04", "2021-01-08"))
assert "Unexpected CSV format" in str(e.value)

11
tests/pricehist/sources/test_yahoo/ibm-long-partial.csv Normal file
View file

@ -0,0 +1,11 @@
Date,Open,High,Low,Close,Adj Close,Volume
1962-01-02,7.713333,7.713333,7.626667,7.626667,1.837710,390000
1962-01-03,7.626667,7.693333,7.626667,7.693333,1.853774,292500
1962-01-04,7.693333,7.693333,7.613333,7.616667,1.835299,262500
1962-01-05,7.606667,7.606667,7.453333,7.466667,1.799155,367500
1962-01-08,7.460000,7.460000,7.266667,7.326667,1.765422,547500
2021-01-04,125.849998,125.919998,123.040001,123.940002,120.954201,5179200
2021-01-05,125.010002,126.680000,124.610001,126.139999,123.101204,6114600
2021-01-06,126.900002,131.880005,126.720001,129.289993,126.175316,7956700
2021-01-07,130.039993,130.460007,128.259995,128.990005,125.882545,4507400
2021-01-08,128.570007,129.320007,126.980003,128.529999,125.433624,4676200
1 Date Open High Low Close Adj Close Volume
2 1962-01-02 7.713333 7.713333 7.626667 7.626667 1.837710 390000
3 1962-01-03 7.626667 7.693333 7.626667 7.693333 1.853774 292500
4 1962-01-04 7.693333 7.693333 7.613333 7.616667 1.835299 262500
5 1962-01-05 7.606667 7.606667 7.453333 7.466667 1.799155 367500
6 1962-01-08 7.460000 7.460000 7.266667 7.326667 1.765422 547500
7 2021-01-04 125.849998 125.919998 123.040001 123.940002 120.954201 5179200
8 2021-01-05 125.010002 126.680000 124.610001 126.139999 123.101204 6114600
9 2021-01-06 126.900002 131.880005 126.720001 129.289993 126.175316 7956700
10 2021-01-07 130.039993 130.460007 128.259995 128.990005 125.882545 4507400
11 2021-01-08 128.570007 129.320007 126.980003 128.529999 125.433624 4676200

249
tests/pricehist/sources/test_yahoo/ibm-long-partial.json
View file

@ -1,249 +0,0 @@
{
"chart": {
"result": [
{
"meta": {
"currency": "USD",
"symbol": "IBM",
"exchangeName": "NYQ",
"fullExchangeName": "NYSE",
"instrumentType": "EQUITY",
"firstTradeDate": -252322200,
"regularMarketTime": 1726257602,
"hasPrePostMarketData": true,
"gmtoffset": -14400,
"timezone": "EDT",
"exchangeTimezoneName": "America/New_York",
"regularMarketPrice": 214.79,
"fiftyTwoWeekHigh": 216.08,
"fiftyTwoWeekLow": 212.13,
"regularMarketDayHigh": 216.08,
"regularMarketDayLow": 212.13,
"regularMarketVolume": 4553547,
"longName": "International Business Machines Corporation",
"shortName": "International Business Machines",
"chartPreviousClose": 7.291,
"priceHint": 2,
"currentTradingPeriod": {
"pre": {
"timezone": "EDT",
"end": 1726234200,
"start": 1726214400,
"gmtoffset": -14400
},
"regular": {
"timezone": "EDT",
"end": 1726257600,
"start": 1726234200,
"gmtoffset": -14400
},
"post": {
"timezone": "EDT",
"end": 1726272000,
"start": 1726257600,
"gmtoffset": -14400
}
},
"dataGranularity": "1d",
"range": "",
"validRanges": [
"1d",
"5d",
"1mo",
"3mo",
"6mo",
"1y",
"2y",
"5y",
"10y",
"ytd",
"max"
]
},
"timestamp": [
-252322200,
-252235800,
-252149400,
-252063000,
-251803800,
1609770600,
1609857000,
1609943400,
1610029800,
1610116200
],
"events": {
"dividends": {
"-249298200": {
"amount": 0.000956,
"date": -249298200
},
"-241439400": {
"amount": 0.000956,
"date": -241439400
},
"-233577000": {
"amount": 0.000956,
"date": -233577000
},
"-225797400": {
"amount": 0.000956,
"date": -225797400
},
"-217848600": {
"amount": 0.001275,
"date": -217848600
},
"1573137000": {
"amount": 1.548757,
"date": 1573137000
},
"1581085800": {
"amount": 1.548757,
"date": 1581085800
},
"1588858200": {
"amount": 1.558317,
"date": 1588858200
},
"1596807000": {
"amount": 1.558317,
"date": 1596807000
},
"1604932200": {
"amount": 1.558317,
"date": 1604932200
}
},
"splits": {
"-177417000": {
"date": -177417000,
"numerator": 5.0,
"denominator": 4.0,
"splitRatio": "5:4"
},
"-114345000": {
"date": -114345000,
"numerator": 3.0,
"denominator": 2.0,
"splitRatio": "3:2"
},
"-53343000": {
"date": -53343000,
"numerator": 2.0,
"denominator": 1.0,
"splitRatio": "2:1"
},
"107530200": {
"date": 107530200,
"numerator": 5.0,
"denominator": 4.0,
"splitRatio": "5:4"
},
"297091800": {
"date": 297091800,
"numerator": 4.0,
"denominator": 1.0,
"splitRatio": "4:1"
},
"864826200": {
"date": 864826200,
"numerator": 2.0,
"denominator": 1.0,
"splitRatio": "2:1"
},
"927811800": {
"date": 927811800,
"numerator": 2.0,
"denominator": 1.0,
"splitRatio": "2:1"
}
}
},
"indicators": {
"quote": [
{
"close": [
7.2912678718566895,
7.3550028800964355,
7.281707763671875,
7.138305187225342,
7.00446081161499,
118.48948669433594,
120.59273529052734,
123.60420989990234,
123.31739807128906,
122.87763214111328
],
"low": [
7.2912678718566895,
7.2912678718566895,
7.2785210609436035,
7.125557899475098,
6.9471001625061035,
117.62906646728516,
119.13002014160156,
121.14722442626953,
122.61949920654297,
121.39579010009766
],
"open": [
7.374124050140381,
7.2912678718566895,
7.3550028800964355,
7.272148132324219,
7.131930828094482,
120.31549072265625,
119.5124282836914,
121.3193130493164,
124.32122039794922,
122.9158706665039
],
"high": [
7.374124050140381,
7.3550028800964355,
7.3550028800964355,
7.272148132324219,
7.131930828094482,
120.38240814208984,
121.1089859008789,
126.08030700683594,
124.7227554321289,
123.63288879394531
],
"volume": [
407940,
305955,
274575,
384405,
572685,
5417443,
6395872,
8322708,
4714740,
4891305
]
}
],
"adjclose": [
{
"adjclose": [
1.5133211612701416,
1.5265485048294067,
1.5113375186920166,
1.4815733432769775,
1.4537923336029053,
99.60364532470703,
101.37164306640625,
103.90313720703125,
103.66202545166016,
103.29237365722656
]
}
]
}
}
],
"error": null
}
}

119
tests/pricehist/sources/test_yahoo/inrx-with-null.json
View file

@ -1,119 +0,0 @@
{
"chart": {
"result": [
{
"meta": {
"currency": "INR",
"symbol": "INR=X",
"exchangeName": "CCY",
"fullExchangeName": "CCY",
"instrumentType": "CURRENCY",
"firstTradeDate": 1070236800,
"regularMarketTime": 1726284616,
"hasPrePostMarketData": false,
"gmtoffset": 3600,
"timezone": "BST",
"exchangeTimezoneName": "Europe/London",
"regularMarketPrice": 83.89,
"fiftyTwoWeekHigh": 83.89,
"fiftyTwoWeekLow": 83.89,
"regularMarketDayHigh": 83.89,
"regularMarketDayLow": 83.89,
"regularMarketVolume": 0,
"longName": "USD/INR",
"shortName": "USD/INR",
"chartPreviousClose": 64.6117,
"priceHint": 4,
"currentTradingPeriod": {
"pre": {
"timezone": "BST",
"start": 1726182000,
"end": 1726182000,
"gmtoffset": 3600
},
"regular": {
"timezone": "BST",
"start": 1726182000,
"end": 1726268340,
"gmtoffset": 3600
},
"post": {
"timezone": "BST",
"start": 1726268340,
"end": 1726268340,
"gmtoffset": 3600
}
},
"dataGranularity": "1d",
"range": "",
"validRanges": [
"1d",
"5d",
"1mo",
"3mo",
"6mo",
"1y",
"2y",
"5y",
"10y",
"ytd",
"max"
]
},
"timestamp": [
1499641200,
1499727600,
1499814000,
1499900400
],
"indicators": {
"quote": [
{
"open": [
64.6155014038086,
null,
64.55549621582031,
64.46800231933594
],
"volume": [
0,
null,
0,
0
],
"low": [
64.41000366210938,
null,
64.3499984741211,
64.33999633789062
],
"close": [
64.61170196533203,
null,
64.52559661865234,
64.36499786376953
],
"high": [
64.6155014038086,
null,
64.56999969482422,
64.48419952392578
]
}
],
"adjclose": [
{
"adjclose": [
64.61170196533203,
null,
64.52559661865234,
64.36499786376953
]
}
]
}
}
],
"error": null
}
}

6
tests/pricehist/sources/test_yahoo/tsla-recent.csv Normal file
View file

@ -0,0 +1,6 @@
Date,Open,High,Low,Close,Adj Close,Volume
2021-01-04,719.460022,744.489990,717.190002,729.770020,729.770020,48638200
2021-01-05,723.659973,740.840027,719.200012,735.109985,735.109985,32245200
2021-01-06,758.489990,774.000000,749.099976,755.979980,755.979980,44700000
2021-01-07,777.630005,816.989990,775.200012,816.039978,816.039978,51498900
2021-01-08,856.000000,884.489990,838.390015,880.020020,880.020020,75055500
1 Date Open High Low Close Adj Close Volume
2 2021-01-04 719.460022 744.489990 717.190002 729.770020 729.770020 48638200
3 2021-01-05 723.659973 740.840027 719.200012 735.109985 735.109985 32245200
4 2021-01-06 758.489990 774.000000 749.099976 755.979980 755.979980 44700000
5 2021-01-07 777.630005 816.989990 775.200012 816.039978 816.039978 51498900
6 2021-01-08 856.000000 884.489990 838.390015 880.020020 880.020020 75055500

126
tests/pricehist/sources/test_yahoo/tsla-recent.json
View file

@ -1,126 +0,0 @@
{
"chart": {
"result": [
{
"meta": {
"currency": "USD",
"symbol": "TSLA",
"exchangeName": "NMS",
"fullExchangeName": "NasdaqGS",
"instrumentType": "EQUITY",
"firstTradeDate": 1277818200,
"regularMarketTime": 1726257600,
"hasPrePostMarketData": true,
"gmtoffset": -14400,
"timezone": "EDT",
"exchangeTimezoneName": "America/New_York",
"regularMarketPrice": 230.29,
"fiftyTwoWeekHigh": 232.664,
"fiftyTwoWeekLow": 226.32,
"regularMarketDayHigh": 232.664,
"regularMarketDayLow": 226.32,
"regularMarketVolume": 59096538,
"longName": "Tesla, Inc.",
"shortName": "Tesla, Inc.",
"chartPreviousClose": 235.223,
"priceHint": 2,
"currentTradingPeriod": {
"pre": {
"timezone": "EDT",
"start": 1726214400,
"end": 1726234200,
"gmtoffset": -14400
},
"regular": {
"timezone": "EDT",
"start": 1726234200,
"end": 1726257600,
"gmtoffset": -14400
},
"post": {
"timezone": "EDT",
"start": 1726257600,
"end": 1726272000,
"gmtoffset": -14400
}
},
"dataGranularity": "1d",
"range": "",
"validRanges": [
"1d",
"5d",
"1mo",
"3mo",
"6mo",
"1y",
"2y",
"5y",
"10y",
"ytd",
"max"
]
},
"timestamp": [
1609770600,
1609857000,
1609943400,
1610029800,
1610116200
],
"indicators": {
"quote": [
{
"open": [
239.82000732421875,
241.22000122070312,
252.8300018310547,
259.2099914550781,
285.3333435058594
],
"close": [
243.2566680908203,
245.0366668701172,
251.9933319091797,
272.0133361816406,
293.3399963378906
],
"high": [
248.163330078125,
246.94667053222656,
258.0,
272.3299865722656,
294.8299865722656
],
"low": [
239.06333923339844,
239.73333740234375,
249.6999969482422,
258.3999938964844,
279.46331787109375
],
"volume": [
145914600,
96735600,
134100000,
154496700,
225166500
]
}
],
"adjclose": [
{
"adjclose": [
243.2566680908203,
245.0366668701172,
251.9933319091797,
272.0133361816406,
293.3399963378906
]
}
]
}
}
],
"error": null
}
}

77
tests/pricehist/sources/test_yahoo/tsla-spark.json Normal file
View file

@ -0,0 +1,77 @@
{
"spark": {
"result": [
{
"symbol": "TSLA",
"response": [
{
"meta": {
"currency": "USD",
"symbol": "TSLA",
"exchangeName": "NMS",
"instrumentType": "EQUITY",
"firstTradeDate": 1277818200,
"regularMarketTime": 1626465603,
"gmtoffset": -14400,
"timezone": "EDT",
"exchangeTimezoneName": "America/New_York",
"regularMarketPrice": 644.22,
"chartPreviousClose": 650.6,
"priceHint": 2,
"currentTradingPeriod": {
"pre": {
"timezone": "EDT",
"start": 1626422400,
"end": 1626442200,
"gmtoffset": -14400
},
"regular": {
"timezone": "EDT",
"start": 1626442200,
"end": 1626465600,
"gmtoffset": -14400
},
"post": {
"timezone": "EDT",
"start": 1626465600,
"end": 1626480000,
"gmtoffset": -14400
}
},
"dataGranularity": "1d",
"range": "1d",
"validRanges": [
"1d",
"5d",
"1mo",
"3mo",
"6mo",
"1y",
"2y",
"5y",
"10y",
"ytd",
"max"
]
},
"timestamp": [
1626442200,
1626465603
],
"indicators": {
"quote": [
{
"close": [
644.22,
644.22
]
}
]
}
}
]
}
],
"error": null
}
}

141
tests/pricehist/test_beanprice.py
View file

@ -1,141 +0,0 @@
import importlib
from datetime import date, datetime, timedelta, timezone
from decimal import Decimal
import pytest
from pricehist import beanprice, exceptions, sources
from pricehist.price import Price
from pricehist.series import Series
@pytest.fixture
def series():
series = Series(
"BTC",
"USD",
"high",
"2021-01-01",
"2021-01-03",
prices=[
Price("2021-01-01", Decimal("1.1")),
Price("2021-01-02", Decimal("1.2")),
Price("2021-01-03", Decimal("1.3")),
],
)
return series
@pytest.fixture
def pricehist_source(mocker, series):
mock = mocker.MagicMock()
mock.types = mocker.MagicMock(return_value=["close", "high", "low"])
mock.fetch = mocker.MagicMock(return_value=series)
return mock
@pytest.fixture
def source(pricehist_source):
return beanprice.source(pricehist_source)()
@pytest.fixture
def ltz():
return datetime.now(timezone.utc).astimezone().tzinfo
def test_get_prices_series(pricehist_source, source, ltz):
ticker = "BTC:USD:high"
begin = datetime(2021, 1, 1, tzinfo=ltz)
end = datetime(2021, 1, 3, tzinfo=ltz)
result = source.get_prices_series(ticker, begin, end)
pricehist_source.fetch.assert_called_once_with(
Series("BTC", "USD", "high", "2021-01-01", "2021-01-03")
)
assert result == [
beanprice.SourcePrice(Decimal("1.1"), datetime(2021, 1, 1, tzinfo=ltz), "USD"),
beanprice.SourcePrice(Decimal("1.2"), datetime(2021, 1, 2, tzinfo=ltz), "USD"),
beanprice.SourcePrice(Decimal("1.3"), datetime(2021, 1, 3, tzinfo=ltz), "USD"),
]
def test_get_prices_series_exception(pricehist_source, source, ltz, mocker):
pricehist_source.fetch = mocker.MagicMock(
side_effect=exceptions.RequestError("Message")
)
ticker = "_5eDJI::low"
begin = datetime(2021, 1, 1, tzinfo=ltz)
end = datetime(2021, 1, 3, tzinfo=ltz)
result = source.get_prices_series(ticker, begin, end)
assert result is None
def test_get_prices_series_special_chars(pricehist_source, source, ltz):
ticker = "_5eDJI::low"
begin = datetime(2021, 1, 1, tzinfo=ltz)
end = datetime(2021, 1, 3, tzinfo=ltz)
source.get_prices_series(ticker, begin, end)
pricehist_source.fetch.assert_called_once_with(
Series("^DJI", "", "low", "2021-01-01", "2021-01-03")
)
def test_get_prices_series_price_type(pricehist_source, source, ltz):
ticker = "TSLA"
begin = datetime(2021, 1, 1, tzinfo=ltz)
end = datetime(2021, 1, 3, tzinfo=ltz)
source.get_prices_series(ticker, begin, end)
pricehist_source.fetch.assert_called_once_with(
Series("TSLA", "", "close", "2021-01-01", "2021-01-03")
)
def test_get_historical_price(pricehist_source, source, ltz):
ticker = "BTC:USD:high"
time = datetime(2021, 1, 3, tzinfo=ltz)
result = source.get_historical_price(ticker, time)
pricehist_source.fetch.assert_called_once_with(
Series("BTC", "USD", "high", "2021-01-03", "2021-01-03")
)
assert result == beanprice.SourcePrice(
Decimal("1.3"), datetime(2021, 1, 3, tzinfo=ltz), "USD"
)
def test_get_historical_price_none_available(pricehist_source, source, ltz, mocker):
pricehist_source.fetch = mocker.MagicMock(
return_value=Series("BTC", "USD", "high", "2021-01-03", "2021-01-03", prices=[])
)
ticker = "BTC:USD:high"
time = datetime(2021, 1, 3, tzinfo=ltz)
result = source.get_historical_price(ticker, time)
assert result is None
def test_get_latest_price(pricehist_source, source, ltz):
ticker = "BTC:USD:high"
start = datetime.combine((date.today() - timedelta(days=7)), datetime.min.time())
today = datetime.combine(date.today(), datetime.min.time())
result = source.get_latest_price(ticker)
pricehist_source.fetch.assert_called_once_with(
Series("BTC", "USD", "high", start.date().isoformat(), today.date().isoformat())
)
assert result == beanprice.SourcePrice(
Decimal("1.3"), datetime(2021, 1, 3, tzinfo=ltz), "USD"
)
def test_get_latest_price_none_available(pricehist_source, source, ltz, mocker):
pricehist_source.fetch = mocker.MagicMock(
return_value=Series("BTC", "USD", "high", "2021-01-01", "2021-01-03", prices=[])
)
ticker = "BTC:USD:high"
result = source.get_latest_price(ticker)
assert result is None
def test_all_sources_available_for_beanprice():
for identifier in sources.by_id.keys():
importlib.import_module(f"pricehist.beanprice.{identifier}").Source()

166
tests/pricehist/test_cli.py
View file

@ -1,166 +0,0 @@
import argparse
import pytest
from pricehist import __version__, cli, sources
def w(string):
return string.split(" ")
def test_valid_pair():
assert cli.valid_pair("BTC/AUD") == ("BTC", "AUD")
assert cli.valid_pair("BTC/AUD/ignored") == ("BTC", "AUD")
assert cli.valid_pair("SYM") == ("SYM", "")
assert cli.valid_pair("SYM/") == ("SYM", "")
with pytest.raises(argparse.ArgumentTypeError):
cli.valid_pair("/SYM")
with pytest.raises(argparse.ArgumentTypeError):
cli.valid_pair("")
def test_valid_date():
assert cli.valid_date("today") == cli.today()
assert cli.valid_date("2021-12-30") == "2021-12-30"
with pytest.raises(argparse.ArgumentTypeError) as e:
cli.valid_date("2021-12-40")
assert "Not a valid" in str(e.value)
def test_valid_date_before():
assert cli.valid_date_before("2021-12-30") == "2021-12-29"
with pytest.raises(argparse.ArgumentTypeError) as e:
cli.valid_date_before("2021-12-40")
assert "Not a valid" in str(e.value)
def test_valid_date_after():
assert cli.valid_date_after("2021-12-30") == "2021-12-31"
with pytest.raises(argparse.ArgumentTypeError) as e:
cli.valid_date_after("2021-12-40")
assert "Not a valid" in str(e.value)
def test_valid_char():
assert cli.valid_char(",") == ","
with pytest.raises(argparse.ArgumentTypeError):
cli.valid_char("")
with pytest.raises(argparse.ArgumentTypeError):
cli.valid_char("12")
def test_cli_no_args_shows_usage(capfd):
cli.cli(w("pricehist"))
out, err = capfd.readouterr()
assert "usage: pricehist" in out
assert "optional arguments:" in out or "options:" in out
assert "commands:" in out
def test_cli_help_shows_usage_and_exits(capfd):
with pytest.raises(SystemExit) as e:
cli.cli(w("pricehist -h"))
assert e.value.code == 0
out, err = capfd.readouterr()
assert "usage: pricehist" in out
assert "optional arguments:" in out or "options:" in out
assert "commands:" in out
def test_cli_verbose(capfd, mocker):
cli.cli(w("pricehist --verbose"))
out, err = capfd.readouterr()
assert "Ended pricehist run at" in err
def test_cli_version(capfd):
cli.cli(w("pricehist --version"))
out, err = capfd.readouterr()
assert f"pricehist {__version__}\n" == out
def test_cli_sources(capfd):
cli.cli(w("pricehist sources"))
out, err = capfd.readouterr()
for source_id in sources.by_id.keys():
assert source_id in out
def test_cli_source(capfd):
expected = sources.by_id["ecb"].format_info() + "\n"
cli.cli(w("pricehist source ecb"))
out, err = capfd.readouterr()
assert out == expected
def test_cli_source_symbols(capfd, mocker):
sources.by_id["ecb"].symbols = mocker.MagicMock(
return_value=[("EUR/AUD", "Euro against Australian Dollar")]
)
cli.cli(w("pricehist source ecb --symbols"))
out, err = capfd.readouterr()
assert out == "EUR/AUD Euro against Australian Dollar\n"
def test_cli_source_search(capfd, mocker):
sources.by_id["alphavantage"].search = mocker.MagicMock(
return_value=[("TSLA", "Tesla Inc, Equity, United States, USD")]
)
cli.cli(w("pricehist source alphavantage --search TSLA"))
out, err = capfd.readouterr()
assert out == "TSLA Tesla Inc, Equity, United States, USD\n"
def test_cli_source_fetch(capfd, mocker):
formatted_result = "P 2021-01-01 00:00:00 BTC 24139.4648 EUR\n"
cli.fetch = mocker.MagicMock(return_value=formatted_result)
argv = w("pricehist fetch coindesk BTC/EUR -s 2021-01-01 -e 2021-01-01 -o ledger")
cli.cli(argv)
out, err = capfd.readouterr()
assert out == formatted_result
def test_cli_source_fetch_invalid_start(capfd, mocker):
argv = w("pricehist fetch coindesk BTC/EUR -s 2021-01-01 -e 2020-12-01")
with pytest.raises(SystemExit) as e:
cli.cli(argv)
assert e.value.code != 0
out, err = capfd.readouterr()
assert "end date '2020-12-01' preceeds the start date" in err
def test_cli_source_fetch_invalid_type(capfd, mocker):
argv = w("pricehist fetch coindesk BTC/EUR -t notype")
with pytest.raises(SystemExit) as e:
cli.cli(argv)
assert e.value.code != 0
out, err = capfd.readouterr()
assert "price type 'notype' is not recognized" in err
def test_cli_source_fetch_sets_source_defaults(mocker):
cli.fetch = mocker.MagicMock(return_value="")
cli.cli(w("pricehist fetch coindesk BTC/EUR"))
captured_series = cli.fetch.call_args.args[0]
assert captured_series.start == sources.by_id["coindesk"].start()
assert captured_series.type == sources.by_id["coindesk"].types()[0]
def test_cli_source_fetch_normalizes_symbols(mocker):
cli.fetch = mocker.MagicMock(return_value="")
cli.cli(w("pricehist fetch coindesk btc/eur"))
captured_series = cli.fetch.call_args.args[0]
assert captured_series.base == "BTC"
assert captured_series.quote == "EUR"
def test_cli_source_fetch_handles_brokenpipeerror(caplog, mocker):
cli.fetch = mocker.MagicMock(side_effect=BrokenPipeError())
cli.cli(w("pricehist fetch coindesk BTC/EUR --verbose"))
assert any(
[
"DEBUG" == r.levelname and "output pipe was closed early" in r.message
for r in caplog.records
]
)

6
tests/pricehist/test_fetch.py
View file

@ -36,7 +36,7 @@ def output(mocker):
@pytest.fixture
def fmt():
def fmt(mocker):
return Format()
@ -65,7 +65,7 @@ def test_fetch_returns_formatted_output(source, res_series, output, fmt, mocker)
def test_fetch_inverts_if_requested(source, res_series, output, fmt, mocker):
req_series = Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03")
inv_series = mocker.MagicMock()
inv_series = mocker.MagicMock(res_series)
res_series.invert = mocker.MagicMock(return_value=inv_series)
fetch(req_series, source, output, invert=True, quantize=None, fmt=fmt)
@ -76,7 +76,7 @@ def test_fetch_inverts_if_requested(source, res_series, output, fmt, mocker):
def test_fetch_quantizes_if_requested(source, res_series, output, fmt, mocker):
req_series = Series("BTC", "EUR", "close", "2021-01-01", "2021-01-03")
qnt_series = mocker.MagicMock()
qnt_series = mocker.MagicMock(res_series)
res_series.quantize = mocker.MagicMock(return_value=qnt_series)
fetch(req_series, source, output, invert=False, quantize=2, fmt=fmt)

1
tests/pricehist/test_format.py
View file

@ -14,7 +14,6 @@ def test_fromargs():
"formatdatesep": None,
"formatcsvdelim": None,
"formatbase": None,
"formatjsonnums": None,
}
args = namedtuple("args", arg_values.keys())(**arg_values)
fmt = Format.fromargs(args)

9
tox.ini
View file

@ -1,9 +0,0 @@
[tox]
isolated_build = True
envlist = py38,py39
[testenv]
deps = poetry
commands =
poetry install
poetry run make test