add cli

Author: marzzzello

.vscode/launch.json

@@ -0,0 +1,17 @@
+{
+    // Verwendet IntelliSense zum Ermitteln möglicher Attribute.
+    // Zeigen Sie auf vorhandene Attribute, um die zugehörigen Beschreibungen anzuzeigen.
+    // Weitere Informationen finden Sie unter https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Python: Modul",
+            "type": "python",
+            "request": "launch",
+            "module": "pytr",
+            "args": [
+                "portfolio"
+            ]
+        }
+    ]
+}

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2020 nborrmann
+Copyright (c) 2020 marzzzello
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,195 +1,54 @@
-# Trade Republic API
+# pytr: Use TradeRepublic in terminal
 
 This is a library for the private API of the Trade Republic online brokerage. I am not affiliated with Trade Republic Bank GmbH.
 
 ## Installation
 
-```
-pip install git+https://github.com/ptrstn/pytr
-```
+Install with `pip install pytr`
 
-## Authentication
-
-First you need to perform a device reset - a private key will be generated that pins your "device". The private key is saved to your keyfile. This procedure will log you out from your mobile device.
+Or you can clone the repo like so:
 
-```python
-from py_tr import TradeRepublicApi
-tr = TradeRepublicApi(phone_no="+4900000000000", pin="0000", keyfile='keyfile.pem')
-tr.initiate_device_reset()
-tr.complete_device_reset("0000") # Substitute the 2FA token that is sent to you via SMS.
+```sh
+git clone https://github.com/marzzzello/pytr.git
+cd pytr
+pip install .
 ```
 
-If no credentials are supplied the library will look for them in the file `~/.pytr/credentials` (the first line must contain the phone number, the second line the pin). If no keyfile is supplied the library will default to `~/.pytr/keyfile.pem`.
-
-## Api Subscriptions
-
-The Trade Republic API works fully asynchronously via Websocket. You subscribe to a 'topic', by sending a request using the `tr.subscribe(payload)` call (or any of the helper methods provided by the library). This will return a `subscription_id`, that you can use to identify responses belonging to this subscription. 
-
-After subscribing you will get one initial response on that subscription and an update whenever the response changes. E.g. market data subscriptions can update multiple times per second, but also the portfolio or watchlist subscriptions will receive an update if the positions change. 
-
-To receive the next response on the websocket, call `tr.recv()`. This will return a tuple consisting of 
-1. the `subscription_id` that this response belongs to
-1. a dictionary that contains all the subscription parameters
-1. the response dictionary
-
-If the Api replies with an error, a `TradeRepublicError` (also containing all three values) will be raised.
-
-To unsubscribe from a topic, call `tr.unsubscribe(subscription_id)`.
-
-The library does bookkeeping on current subscriptions in the `tr.subscriptions` dictionary, mapping `subscription_id` to a dictionary of the subscription parameters. 
-
-Sample code:
-
-```python
-import asyncio
-from py_tr import TradeRepublicApi
-
-tr = TradeRepublicApi(phone_no="+4900000000000", pin="0000", keyfile='keyfile.pem')
-
-async def my_loop():
-    cash_subscription_id = await tr.cash()
-    await tr.ticker("DE0007236101", "LSX")
-    await tr.ticker("DE0007100000", "LSX")
-
-    while True:
-        subscription_id, subscription, response = await tr.recv()
-
-        # Identify response by subscription_id:
-        if cash_subscription_id == subscription_id:
-            print(f"Cash balance is {response}")
-
-        # Or identify response by subscription type:
-        if subscription["type"] == "ticker":
-            print(f"Current tick for {subscription['id']} is {response}")
+## Usage
 
-asyncio.get_event_loop().run_until_complete(my_loop())
 ```
+$ pytr help
+usage: pytr [-h] [-s {bash,zsh}] [-v {warning,info,debug}]
+             {help,login,portfolio,dl_docs,set_price_alarms} ...
 
-## Blocking Api Calls
+positional arguments:
+  {help,login,portfolio,dl_docs,set_price_alarms}
+                        Desired action to perform
+    help                Print this help message
+    login               Check if credentials file exists. If not create it and
+                        ask for input. Try to login. Ask for device reset if
+                        needed
+    portfolio           Show current portfolio
+    dl_docs             Download all pdf documents from the timeline and sort
+                        them into folders
+    set_price_alarms    Set price alarms based on diff from current price
 
-For convenience the library provides a helper function that communicates in a blocking manner:
-
-```python
-portfolio = tr.run_blocking(tr.portfolio(), timeout=5.0)
+optional arguments:
+  -h, --help            show this help message and exit
+  -s {bash,zsh}, --print-completion {bash,zsh}
+                        print shell completion script
+  -v {warning,info,debug}, --verbosity {warning,info,debug}
+                        Set verbosity level
 ```
 
-This will subscribe to a topic, return the first response and immediately unsubscribe. If no response is returned this 
-will time out after a default of five seconds. You can also prefix any method with `blocking_` to achieve the same result (eg: `tr.blocking_portfolio(timeout=5)`).
-
-*Warning*: `tr.run_blocking()` will silently drop all messages belonging to different subscriptions, therefore do not use both approaches at the same time.
-
-## All Subscriptions
-
-The following subscriptions are supported by this library:
-
-### Portfolio
-```python
-tr.portfolio()
-tr.cash()
-tr.available_cash_for_payout()
-tr.portfolio_status()
-tr.portfolio_history(timeframe)
-tr.experience()
-```
-### Watchlist
-```python
-tr.watchlist()
-tr.add_watchlist(isin)
-tr.remove_watchlist(isin)
-```
-### Market Data
-```python
-tr.instrument_details(isin)
-tr.instrument_suitability(isin)
-tr.stock_details(isin)
-tr.ticker(isin, exchange="LSX")
-tr.performance(isin, exchange="LSX")
-tr.performance_history(isin, timeframe, exchange="LSX", resolution=None)
-```
-### Timeline
-```python
-tr.timeline(after=None)
-tr.timeline_detail(timeline_id)
-tr.timeline_detail_order(order_id)
-tr.timeline_detail_savings_plan(savings_plan_id)
-```
-### Search
-```python
-tr.search_tags()
-tr.search_suggested_tags(query)
-tr.search(query, asset_type="stock", page=1, page_size=20, aggregate=False, only_savable=False,
-          filter_index=None, filter_country=None, filter_sector=None, filter_region=None)
-tr.search_derivative(underlying_isin, product_type)
-```
-### Orders
+## Authentication
 
-Be careful, these methods can create actual live trades.
+First you need to perform a device reset - a private key will be generated that pins your "device". The private key is saved to your keyfile. This procedure will log you out from your mobile device.
 
-```python
-tr.order_overview()
-tr.cash_available_for_order()
-tr.size_available_for_order(isin, exchange)
-tr.price_for_order(isin, exchange, order_type)
-tr.market_order(isin, exchange, order_type, size, expiry, sell_fractions, expiry_date=None, warnings_shown=None)
-tr.limit_order(isin, exchange, order_type, size, limit, expiry, expiry_date=None, warnings_shown=None)
-tr.stop_market_order(isin, exchange, order_type, size, stop, expiry, expiry_date=None, warnings_shown=None)
-tr.cancel_order(order_id)
-```
-### Savings Plans
-```python
-tr.savings_plan_overview()
-tr.savings_plan_parameters(isin)
-tr.create_savings_plan(isin, amount, interval, start_date, start_date_type, start_date_value)
-tr.change_savings_plan(savings_plan_id, isin, amount, interval, start_date, start_date_type, start_date_value)
-tr.cancel_savings_plan(savings_plan_id)
-```
-### Price Alarms
-```python
-tr.price_alarm_overview()
-tr.create_price_alarm(isin, price)
-tr.cancel_price_alarm(price_alarm_id)
+```sh
+$ pytr login
+$ # or
+$ pytr login --phone_no +49123456789 --pin 1234
 ```
-### News
-```python
-tr.news(isin)
-tr.news_subscriptions()
-tr.subscribe_news(isin)
-tr.unsubscribe_news(isin)
-```
-### Other
-```python
-tr.motd()
-tr.neon_cards()
-```
-### REST calls
-These Api calls are not asynchronous, but plain old rest calls. 
-```python
-tr.settings()
-tr.order_cost(isin, exchange, order_mode, order_type, size, sell_fractions)
-tr.savings_plan_cost(isin, amount, interval)
-tr.payout(amount)
-tr.confirm_payout(process_id, code)
-```
-Payouts need two-factor-authentication: the `payout()` call will respond with a process_id and trigger an SMS with a code. Confirm the payout by calling `confirm_payout()` with the process_id and code.
-
-### Parameters
-* **isin** `string`: the *International Securities Identification Number* 
-* **timeframe** `string`: allowed values are `"1d"`, `"5d"`, `"1m"`, `"3m"`, `"6m"`, `"1y"`, `"5y"`, `"max"` 
-* **exchange** `string`: identifies a stock exchange, usually `"LSX"` for *Lang & Schwarz Exchange*, other allowed values can be seen in the `instrument_details()` call 
-* **resolution** `int` (optional): resolution for timeseries in milliseconds, minimum seems to be 60,000 
-* **asset_type** `string`: allowed values are `"stock"`, `"fund"`, `"derivative"`
-* **order_type** `string`: allowed values are `"buy"` or `"sell"` 
-* **size** `int`: how many shares to trade 
-* **sell_fractions** `bool`: sell remaining fractional shares
-* **limit** `float`: limit price 
-* **stop** `float`: stop price 
-* **expiry** `string`: allowed values are `"gfd"` (good for day), `"gtd"` (good till date) and `"gtc"` (good till cancelled)
-* **expiry_date** `string` (optional): if expiry is `"gtd"`, specify a date in the format `"yyyy-mm-dd"`
-* **warnings_shown** `list of strings` (optional): may contain one or more of the following values: `"targetMarket"`, `"userExperience"`, `"unknown"` - however an empty list also seems to always be accepted
-* **amount** `int` savings plan amount in euro
-* **interval** `string` interval for savings plan execution, allowed values are `"everySecondWeek"`, `"weekly"`, `"twoPerMonth"`, `"monthly"`, `"quarterly"`
-* **start_date** `string` first execution date for savings plans, in format `"yyyy-mm-dd"`
-* **start_date_type** `string` allowed values are `"dayOfMonth"`, `"weekday"`
-* **start_date_value** `int` either the day of month (0-30), or the weekday (0-6) on which to execute the savings plan
 
-Allowed values for search filters can be found using the `search_tags()` call.
+If no arguments are supplied pytr will look for them in the file `~/.pytr/credentials` (the first line must contain the phone number, the second line the pin). If the file doesn't exist pytr will ask for for the phone number and pin.

pytr/__init__.py

@@ -0,0 +1,16 @@
+#!/usr/bin/env python3
+import logging
+
+from pytr.main import main
+
+if __name__ == '__main__':
+    try:
+        main()
+    except KeyboardInterrupt:
+        log = logging.getLogger(__name__)
+        log.info('Exiting...')
+        exit()
+    except Exception as e:
+        log = logging.getLogger(__name__)
+        log.fatal(e)
+        raise

pytr/__main__.py

@@ -0,0 +1,76 @@
+import json
+import os
+import pathlib
+import sys
+
+from pygments import highlight, lexers, formatters
+
+from pytr.api import TradeRepublicApi
+from pytr.utils import get_logger
+
+
+def get_settings(tr):
+    formatted_json = json.dumps(tr.settings(), indent=2)
+    if sys.stdout.isatty():
+        colorful_json = highlight(formatted_json, lexers.JsonLexer(), formatters.TerminalFormatter())
+        return colorful_json
+    else:
+        return formatted_json
+
+
+def reset(tr):
+    tr.initiate_device_reset()
+    print("You should have received a SMS with a token. Please type it in:")
+    token = input()
+    tr.complete_device_reset(token)
+    print("Reset done")
+
+
+def login(phone_no=None, pin=None):
+    """
+    Check if credentials file exists else create it.
+    If no parameters are set but are needed then ask for input
+    Try to login. Ask for device reset if needed
+    """
+    home = pathlib.Path.home()
+    credentials_file = os.path.join(home, ".pytr", "credentials")
+    log = get_logger(__name__)
+
+    if os.path.isfile(credentials_file):
+        log.info("Found credentials file")
+        with open(credentials_file) as f:
+            lines = f.readlines()
+        phone_no = lines[0].strip()
+        pin = lines[1].strip()
+        log.info(f"Phone: {phone_no}, PIN: {pin}")
+    else:
+        log.info("Credentials file not found")
+        if phone_no is None:
+            print("Please enter your TradeRepbulic phone number in the format +49123456678:")
+            phone_no = input()
+        if pin is None:
+            print("Please enter your TradeRepbulic pin:")
+            pin = input()
+
+        with open(credentials_file, "w") as f:
+            f.writelines([phone_no + "\n", pin + "\n"])
+
+        log.info(f"Saved credentials in {credentials_file}")
+
+    # use ~/.pytr/credentials and ~/.pytr/keyfile.pem
+    tr = TradeRepublicApi()
+
+    try:
+        tr.login()
+    except (KeyError, AttributeError):
+        # old keyfile or no keyfile
+        print("Error logging in. Reset device? (y)")
+        confirmation = input()
+        if confirmation == "y":
+            reset(tr)
+        else:
+            print("Cancelling reset")
+            exit(1)
+    log.info("Logged in")
+    log.debug(get_settings(tr))
+    return tr

pytr/account.py

@@ -0,0 +1,100 @@
+import os
+import re
+
+from concurrent.futures import as_completed
+from requests_futures.sessions import FuturesSession
+
+from pytr.utils import preview, Timeline
+
+
+class DL:
+    def __init__(self, tr, output_path, headers={'User-Agent': 'pytr'}):
+        self.tr = tr
+        self.output_path = output_path
+        self.headers = headers
+
+        self.session = FuturesSession()
+        self.futures = []
+
+        self.docs_request = 0
+        self.done = 0
+
+        self.tl = Timeline(self.tr)
+
+    async def dl_loop(self):
+        await self.tl.get_next_timeline()
+
+        while True:
+            _subscription_id, subscription, response = await self.tr.recv()
+
+            if subscription["type"] == "timeline":
+                await self.tl.get_next_timeline(response)
+            elif subscription["type"] == "timelineDetail":
+                await self.tl.timelineDetail(response, self)
+            else:
+                print(f"unmatched subscription of type '{subscription['type']}':\n{preview(response)}")
+
+    def dl_doc(self, doc, titleText, subtitleText, subfolder=None):
+        """
+        send asynchronous request, append future with filepath to self.futures
+        """
+        doc_url = doc["action"]["payload"]
+
+        date = doc["detail"]
+        iso_date = "-".join(date.split(".")[::-1])
+
+        # extract time from subtitleText
+        time = re.findall("um (\\d+:\\d+) Uhr", subtitleText)
+        if time == []:
+            time = ""
+        else:
+            time = f" {time[0]}"
+
+        if subfolder is not None:
+            directory = os.path.join(self.output_path, subfolder)
+        else:
+            directory = self.output_path
+
+        # If doc_type is something like "Kosteninformation 2", then strip the 2 and save it in doc_type_num
+        doc_type = doc['title'].rsplit(" ")
+        if doc_type[-1].isnumeric() is True:
+            doc_type_num = f" {doc_type.pop()}"
+        else:
+            doc_type_num = ""
+
+        doc_type = " ".join(doc_type)
+        filepath = os.path.join(directory, doc_type, f"{iso_date}{time} {titleText}{doc_type_num}.pdf")
+
+        # if response['titleText'] == "Shopify":
+        #    print(json.dumps(response))
+
+        if os.path.isfile(filepath) is False:
+            self.docs_request += 1
+            future = self.session.get(doc_url)
+            future.filepath = filepath
+            self.futures.append(future)
+        else:
+            print("file {filepath} already exists. Skipping...")
+
+    def work_responses(self):
+        """
+        process responses of async requests
+        """
+        for future in as_completed(self.futures):
+            r = future.result()
+            os.makedirs(os.path.dirname(future.filepath), exist_ok=True)
+            with open(future.filepath, "wb") as f:
+                f.write(r.content)
+                self.done += 1
+
+                print(f"done: {self.done:>3}/{self.docs_request} {os.path.basename(future.filepath)}")
+
+                if self.done == self.docs_request:
+                    print("Done.")
+                    exit(0)
+
+    def dl_all(output_path):
+        """
+        todo
+        """
+        pass

src/py_tr/py_tr.py pytr/api.py

@@ -0,0 +1,140 @@
+#!/usr/bin/env python
+
+import argparse
+import asyncio
+import signal
+
+
+import shtab
+
+from pytr.utils import get_logger
+from pytr.dl import DL
+from pytr.account import login
+from pytr.portfolio import Portfolio
+
+
+# async def my_loop(tr, dl):
+#     # await tr.subscribe({"type": "unsubscribeNews"})
+#     # await tr.order_overview()
+
+#     # await tr.timeline_detail("98d13dc6-5bd3-43c8-b74a-dae4e7728f4f")
+
+#     # await tr.ticker("DE0007236101", "LSX")
+#     # await tr.ticker("DE0007100000", "LSX")
+
+#     while True:
+#         _subscription_id, subscription, response = await tr.recv()
+
+#         # Identify response by subscription_id:
+#         #   if portfolio_subscription_id == subscription_id:
+
+#         if subscription["type"] == "orders":
+#             print(f"Orders: {response}")
+
+#         # Or identify response by subscription type:
+#         elif subscription["type"] == "ticker":
+#             print(f"Current tick for {subscription['id']} is {response}")
+
+#         else:
+#             print(f"unmatched subscription of type '{subscription['type']}':\n{preview(response)}")
+
+
+def get_main_parser():
+    parser = argparse.ArgumentParser()
+    shtab.add_argument_to(parser, ["-s", "--print-completion"])  # magic!
+
+    parser.add_argument(
+        '-v', '--verbosity', help='Set verbosity level', choices=['warning', 'info', 'debug'], default='info'
+    )
+    subparsers = parser.add_subparsers(help='Desired action to perform', dest='command')
+
+    # help
+    subparsers.add_parser('help', help='Print this help message')
+
+    # Create parent subparser for {dl_docs, check}-parsers with common arguments
+    parent_parser = argparse.ArgumentParser(add_help=False)
+
+    # Subparsers based on parent
+
+    parser_login = subparsers.add_parser(
+        'login',
+        parents=[parent_parser],
+        help='Check if credentials file exists. If not create it and ask for input. Try to login. Ask for device reset if needed',
+    )
+    parser_login.add_argument('-n', '--phone_no', help='TradeRepbulic phone number (international format)')
+    parser_login.add_argument('-p', '--pin', help='TradeRepbulic pin')
+
+    subparsers.add_parser('portfolio', parents=[parent_parser], help='Show current portfolio')
+
+    parser_dl_docs = subparsers.add_parser(
+        'dl_docs',
+        parents=[parent_parser],
+        help='Download all pdf documents from the timeline and sort them into folders',
+    )
+    parser_dl_docs.add_argument('output', help='Output directory', metavar='PATH')
+
+    parser_set_price_alarms = subparsers.add_parser(
+        'set_price_alarms', parents=[parent_parser], help='Set price alarms based on diff from current price'
+    )
+    parser_set_price_alarms.add_argument(
+        '-p',
+        '--percent',
+        help='Percentage +/-',
+        # choices=range(-1000, 1001),
+        metavar='[-1000 ... 1000]',
+        type=int,
+        default=-10,
+    )
+    return parser
+
+
+def exit_gracefully(signum, frame):
+    # restore the original signal handler as otherwise evil things will happen
+    # in input when CTRL+C is pressed, and our signal handler is not re-entrant
+    global original_sigint
+    signal.signal(signal.SIGINT, original_sigint)
+
+    try:
+        if input("\nReally quit? (y/n)> ").lower().startswith('y'):
+            exit(1)
+
+    except KeyboardInterrupt:
+        print("Ok ok, quitting")
+        exit(1)
+
+    # restore the exit gracefully handler here
+    signal.signal(signal.SIGINT, exit_gracefully)
+
+
+def main():
+    # store the original SIGINT handler
+    global original_sigint
+    original_sigint = signal.getsignal(signal.SIGINT)
+    signal.signal(signal.SIGINT, exit_gracefully)
+
+    parser = get_main_parser()
+    args = parser.parse_args()
+    # print(vars(args))
+
+    log = get_logger(__name__, args.verbosity)
+    log.setLevel(args.verbosity.upper())
+    log.debug('logging is set to debug')
+
+    if args.command == "login":
+        login(phone_no=args.phone_no, pin=args.pin)
+
+    elif args.command == "dl_docs":
+        dl = DL(login(), args.output)
+        asyncio.get_event_loop().run_until_complete(dl.dl_loop())
+    elif args.command == "set_price_alarms":
+        # TODO
+        pass
+    elif args.command == "portfolio":
+        p = Portfolio(login())
+        p.get()
+    else:
+        parser.print_help()
+
+
+if __name__ == '__main__':
+    main()

pytr/dl.py

@@ -0,0 +1,68 @@
+import asyncio
+from pytr.utils import preview
+
+
+class Portfolio:
+    def __init__(self, tr):
+        self.tr = tr
+
+    async def portfolio_loop(self):
+        recv = 0
+        await self.tr.portfolio()
+        await self.tr.cash()
+        # await self.tr.available_cash_for_payout()
+
+        while True:
+            _subscription_id, subscription, response = await self.tr.recv()
+
+            if subscription["type"] == "portfolio":
+                recv += 1
+                self.portfolio = response
+            elif subscription["type"] == "cash":
+                recv += 1
+                self.cash = response
+            # elif subscription["type"] == "availableCashForPayout":
+            #     recv += 1
+            #     self.payoutCash = response
+            else:
+                print(f"unmatched subscription of type '{subscription['type']}':\n{preview(response)}")
+
+            if recv == 2:
+                return
+
+    def overview(self):
+        for x in ["netValue", "unrealisedProfit", "unrealisedProfitPercent", "unrealisedCost"]:
+            print(f"{x:24}: {self.portfolio[x]:>10.2f}")
+        print()
+
+        print("ISIN            avgCost *   quantity =    buyCost ->   netValue       diff   %-diff")
+        totalBuyCost = 0.0
+        totalNetValue = 0.0
+        positions = self.portfolio["positions"]
+        for pos in sorted(positions, key=lambda x: x['netValue'], reverse=True):
+            buyCost = pos["unrealisedAverageCost"] * pos["netSize"]
+            diff = pos["netValue"] - buyCost
+            diffP = ((pos["netValue"] / buyCost) - 1) * 100
+            totalBuyCost += buyCost
+            totalNetValue += pos["netValue"]
+
+            print(
+                f"{pos['instrumentId']} {pos['unrealisedAverageCost']:>10.2f} * {pos['netSize']:>10.2f} = {buyCost:>10.2f} -> {pos['netValue']:>10.2f} {diff:>10.2f} {diffP:>7.1f}%"
+            )
+
+        print("ISIN            avgCost *   quantity =    buyCost ->   netValue       diff   %-diff")
+        print()
+
+        diff = totalNetValue - totalBuyCost
+        diffP = ((totalNetValue / totalBuyCost) - 1) * 100
+        print(f"Depot {totalBuyCost:>43.2f} -> {totalNetValue:>10.2f} {diff:>10.2f} {diffP:>7.1f}%")
+
+        cash = self.cash[0]['amount']
+        currency = self.cash[0]['currencyId']
+        print(f"Cash {currency} {cash:>40.2f} -> {cash:>10.2f}")
+        print(f"Total {cash+totalBuyCost:>43.2f} -> {cash+totalNetValue:>10.2f}")
+
+    def get(self):
+        asyncio.get_event_loop().run_until_complete(self.portfolio_loop())
+
+        self.overview()

pytr/main.py

@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+
+import logging
+import coloredlogs
+import json
+
+
+log_level = None
+
+
+def get_logger(name=__name__, verbosity=None):
+    """
+    Colored logging
+
+    :param name: logger name (use __name__ variable)
+    :param verbosity:
+    :return: Logger
+    """
+    global log_level
+    if verbosity is not None:
+        if log_level is None:
+            log_level = verbosity
+        else:
+            raise RuntimeError('Verbosity has already been set.')
+
+    shortname = name.replace('fdroid_mirror_monitor.', '')
+    logger = logging.getLogger(shortname)
+
+    # no logging of libs (and fix double logs because of fdroidserver)
+    logger.propagate = False
+
+    fmt = '%(asctime)s %(threadName)-12s %(name)-7s %(levelname)-8s %(message)s'
+    datefmt = '%Y-%m-%d %H:%M:%S%z'
+
+    fs = {
+        'asctime': {'color': 'green'},
+        'hostname': {'color': 'magenta'},
+        'levelname': {'color': 'red', 'bold': True},
+        'name': {'color': 'magenta'},
+        'programname': {'color': 'cyan'},
+        'username': {'color': 'yellow'},
+    }
+
+    ls = {
+        'critical': {'color': 'red', 'bold': True},
+        'debug': {'color': 'green'},
+        'error': {'color': 'red'},
+        'info': {},
+        'notice': {'color': 'magenta'},
+        'spam': {'color': 'green', 'faint': True},
+        'success': {'color': 'green', 'bold': True},
+        'verbose': {'color': 'blue'},
+        'warning': {'color': 'yellow'},
+    }
+
+    coloredlogs.install(level=log_level, logger=logger, fmt=fmt, datefmt=datefmt, level_styles=ls, field_styles=fs)
+
+    return logger
+
+
+def preview(response, num_lines=5):
+    lines = json.dumps(response, indent=2).splitlines()
+    head = "\n".join(lines[:num_lines])
+    tail = len(lines) - num_lines
+
+    if tail <= 0:
+        return f"{head}\n"
+    else:
+        return f"{head}\n{tail} more lines hidden"
+
+
+class Timeline:
+    def __init__(self, tr):
+        self.tr = tr
+
+        self.received_detail = 0
+        self.requested_detail = 0
+
+    async def get_next_timeline(self, response=None):
+        """
+        Get timelines and save time in global list timelines.
+        Extract id of timeline events and save them in global list timeline_detail_ids
+        """
+
+        if response is None:
+            # empty response / first timeline
+            print("Awaiting #1  timeline")
+            self.timelines = []
+            self.timeline_detail_ids = []
+            self.timeline_events = []
+            await self.tr.timeline()
+        else:
+            self.timelines.append(response)
+            try:
+                after = response["cursors"]["after"]
+            except KeyError:
+                # last timeline is reached
+                print(f"Received #{len(self.timelines):<2} (last) timeline")
+                await self.get_timeline_details(5)
+            else:
+                print(f"Received #{len(self.timelines):<2} timeline, awaiting #{len(self.timelines)+1:<2} timeline")
+                await self.tr.timeline(after)
+
+            # print(json.dumps(response))
+            for event in response["data"]:
+                self.timeline_events.append(event)
+                self.timeline_detail_ids.append(event["data"]["id"])
+
+    async def get_timeline_details(self, num_torequest):
+        self.requested_detail += num_torequest
+
+        while num_torequest > 0:
+            num_torequest -= 1
+            try:
+                event = self.timeline_events.pop()
+            except IndexError:
+                return
+            else:
+                await self.tr.timeline_detail(event["data"]["id"])
+
+    async def timelineDetail(self, response, dl):
+
+        self.received_detail += 1
+
+        if self.received_detail == self.requested_detail:
+            await self.get_timeline_details(5)
+
+        timeline_detail_id = response["id"]
+        for event in self.timeline_events:
+            if timeline_detail_id == event["data"]["id"]:
+                self.timeline_events.remove(event)
+
+        print(f"len timeline_events: {len(self.timeline_events)}")
+
+        print(f"R: {self.received_detail}/{len(self.timeline_detail_ids)}")
+
+        if response["subtitleText"] == "Sparplan":
+            isSavingsPlan = True
+        else:
+            isSavingsPlan = False
+            # some savingsPlan don't have the subtitleText == "Sparplan" but there are actions just for savingsPans
+            for section in response["sections"]:
+                if section["type"] == "actionButtons":
+                    for button in section["data"]:
+                        if button["action"]["type"] in ["editSavingsPlan", "deleteSavingsPlan"]:
+                            isSavingsPlan = True
+                            break
+
+        print(f"Detail: {response['titleText']} -- {response['subtitleText']} -- istSparplan: {isSavingsPlan}")
+
+        for section in response["sections"]:
+            if section["type"] == "documents":
+                for doc in section["documents"]:
+
+                    # save all savingsplan documents in a subdirectory
+                    if isSavingsPlan:
+                        dl.dl_doc(doc, response['titleText'], response["subtitleText"], subfolder="Sparplan")
+                    else:
+                        dl.dl_doc(doc, response['titleText'], response["subtitleText"])
+
+        if self.received_detail == len(self.timeline_detail_ids):
+            print("received all details, downloading docs..")
+            dl.work_responses()
+        else:
+            print(f"r: {self.received_detail}/{len(self.timeline_detail_ids)} - istSparplan: {isSavingsPlan}")

pytr/portfolio.py

@@ -1,28 +1,40 @@
-import setuptools
+from os import path
+from setuptools import setup
 
-with open("README.md", "r") as fh:
-    long_description = fh.read()
 
-setuptools.setup(
-    name="py_tr",
-    version="1.0.2",
-    author="Nils Borrmann",
-    author_email="n.borrmann@googlemail.com",
+def readme():
+    this_directory = path.abspath(path.dirname(__file__))
+    with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f:
+        return f.read()
+
+
+setup(
+    name='pytr',
+    version='0.0.1',
+    description='Use TradeRepublic in terminal',
+    long_description=readme(),
+    long_description_content_type='text/markdown',
+    url='https://gitlab.com/marzzzello/pytr/',
+    author='marzzzello',
+    author_email='853485-marzzzello@users.noreply.gitlab.com',
     license='MIT',
-    description="Unoffical Python Interface for the Trade Republic API",
-    long_description=long_description,
-    long_description_content_type="text/markdown",
-    url="https://github.com/nborrmann/pytr",
+    packages=['pytr'],
+    python_requires='>=3.5',
+    entry_points={
+        'console_scripts': [
+            'pytr = pytr.main:main',
+        ],
+    },
+    #  scripts=['traderep'],
+    # install_requires=['py_tr'],
+    install_requires=['coloredlogs', 'ecdsa', 'pygments', 'requests_futures', 'shtab', 'websockets'],
     classifiers=[
         "License :: OSI Approved :: MIT License",
-        "Programming Language :: Python :: 3",
+        'Programming Language :: Python :: 3 :: Only',
         "Operating System :: OS Independent",
-        "Development Status :: 5 - Production/Stable",
+        'Development Status :: 3 - Alpha',
         "Topic :: Office/Business :: Financial",
-        "Topic :: Office/Business :: Financial :: Investment"
+        "Topic :: Office/Business :: Financial :: Investment",
     ],
-    install_requires=["requests", "websockets", "ecdsa"],
-    python_requires='>=3.5',
-    package_dir={'': 'src'},
-    packages=setuptools.find_packages('src'),
+    zip_safe=False,
 )