commit a8daf9c8b11ea781d832b603c0e5d5943321888d Author: Nicolás Sánchez Date: Thu Oct 24 23:33:47 2024 -0300 Initial commit diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..911fc49 --- /dev/null +++ b/.gitignore @@ -0,0 +1,14 @@ +__pycache__/ +.vscode/ +api_credentials.db +credentials.py +DCAv2.code-workspace +profits/my_database.db +profits/profits_database.db +utils/certs/ +utils/__pycache__/ +utils/close.py +utils/credentials.py +utils/set_exchange.py +utils/stuff.py + diff --git a/changelog.txt b/changelog.txt new file mode 100755 index 0000000..c6e3df8 --- /dev/null +++ b/changelog.txt @@ -0,0 +1,1144 @@ +2024.10.20: +. Insted of querying the average fill on the initial order, now the initial price is calculated using total_quote/total_base. +. Using create_market_buy_order_with_cost with Gate.io (Because there may be a bug when generating simulated market buy orders.) + +2024.10.18: +. Improved slippage behavior in take_profit_routine. + +2024.10.01: +. Improvements when checking for liquidity and restarting the bot because of lack of. + +2024.09.27: +. Modified some strings. + +2024.09.25: +. Optimized Binance's open orders fetching routine (Be sure to keep Binance's lap time >= 2 seconds to avoid hitting API limits, increase if banned.) + Every fetch_open_orders call have a weight of 80, you have 6000/minute available, the rest of the exchanges don't give a damn. + +2024.09.24: +. Optimized Binance's price fetch routine. + +2024.08.26: +. In check_status, if the safety order is none, it assigns it an empty order. + +2024.08.15: +. Removed "paused_pairs" list and endpoints and added "paused_traders" to global_status. +. Switched "trader will be paused" and "trader will be resumed" strings. + +2024.08.14: +. Added automatic status file backup. + +2024.08.11: +. Removed some unnecessary variable clearings in unwrapped_switch_to_short. + +2024.08.07a: +. Added an extra confirmation prior to starting the instance. +. Added percentage to switch to long. + +2024.08.07: +. Added extra exception handling code to update_status to catch an annoying bug. + +2024.08.05d: +. Minor wrap up. + +2024.08.05c: +. Further optimizations of previous point. + +2024.08.05b: +. In new_limit_order and market orders, returns a get_order(new_order) instead of directly what the endpoint returns. + +2024.08.05: +. Changes in market and simulated_market orders to mitigate Kucoin "order does not exist" error. + +2024.08.04: +. Now both exceptions in update_status write to log. + +2024.07.21: +. In a short trader, if the take profit price is bigger than the old_long price, it displays it in bright green. + +2024.07.17: +. Reverted previous change: It was ugly. + +2024.07.15: +. Added long/short trader amount to status bar. + +2024.07.08: +. Added tp order and safety order to the status dict. + +2024.07.07: +. Extended dynamic_so_deviance to include short traders. +. Implemented clip_value() +. The order of the traders on screen is now shorts first. + +2024.07.03: +. Bias implemented (Safety order price variation) + +2024.07.01: +. Small list comprehension refactor in load_keys_from_db. +. Now it rewrites the config file as soon as a new pair is imported. + +2024.06.24: +. Fixed a small bug in generate_status_strings. + +2024.06.23: +. Added a flag to /view_old_long endpoint, if true it will load the old_long data from the old_long file, else it will display it from the status dict. +. After switching to short, last_call is cancelled. + +2024.06.10: +. New API endpoint: /reload_safety_order. + +2024.06.06: +. Splash screen now has its own function. +. Implemented max_short_safety_orders: default safety order value for short traders. Default is 45, can be overriden in config_dict by setting "max_short_safety_orders" key. + +2024.06.05: +. More docstring edits, type checking and type hints :D + +2024.06.04: +. Refactoring in new_limit_order. +. Added some docstrings. + +2024.06.03: +. Added some type-checking code to linear_space. +. Now it forces a Telegram notification when a trader is paused because of exceeded retries limit. + +2024.05.31: +. Small change in import_trader: Now it loads no_of_safety_orders prior to loading the orders. +. Now it does not takes into account a partial SO fill in take_profit_routine. + +2024.05.26: +. Exchange name is now displayed in bright white (where available). +. When the old take profit order is partially filled, the log entry now includes the order id. +. Added paused_traders endpoint +. Initial refactor of import_trader: No longer a class, simplified it as a function. +. Added some type-checking code to the unwrapped API functions. + +2024.05.24: +. Now the trader calculates minimum_order_size_allowed with a simplified ceil statement. + +2024.05.23: +. Changed Kucoin minimum_quote_size calculation. + +2024.05.22: +. "There is no safety order to cancel" message no longer gets into log file. +. Cooldown between trades reduced to 2s. + +2024.05.21: +. Now it removes old_long file when switching to long. + +2024.05.20: +. Added "view_old_long" endpoint. + +2024.05.18: +. Added all_markets method to broker. It returns a dictionary of all the markets of the exchange. +. In the add_pair endpoint, now it checks first if the market exists and if it's active prior to trying to start the trader. +. Added no_retries flag to several methods in broker class. + +2024.05.17: +. Removed unused offline_workers references. +. Enabled autoswitch when safety orders are exhausted. + +2024.05.16: +. Removed SSL implementation and rate limiter, that tasks will be handled by Nginx from now on. +. Removed wraps, refactored api-key implementation. +. Fixed a small bug in check_duplicate_profit_in_db that ocurred when a trading pair got its first profit. + +2024.05.15: +. Removed CORS (not needed anymore) and added a simple rate limiter. If memory leak continues, try waitress. +. Partial profits partially implemented: + . When a take profit order is partially filled and later a new safety order is sent, the profit of that operation was not taken into account + when calculating the total profit at deal close. + . Only for long traders, at least for the time being. + . Joint-code, test later. + +2024.05.14: +. When adding quote, status_dict wasn't saved to disk, so if an interruption ocurred the old take profit order id remained in the status dictionary. + +2024.05.12: +. Initial implementation of API refactoring. +. Changed name of main file from dca_runtime_wAPI.py to main.py. + +2024.05.09: +. Minor refactoring in calculate_safety_prices, with sample code of future changes. + +2024.04.30: +. screen_buffer is now global, for future API access. + +2024.04.29: +. Implemented check_for_duplicate_profit_in_db() + +2024.04.28: +. Minor refactoring when removing keys from the status dictionary. + +2024.04.26: +. Unwrapped all API calls to prepare for migration to new API. + +2024.04.25: +. Added even more pause descriptors and exceptions (hunting pause bug only present on testnet) + +2024.04.24: +. Added a few more pause descriptors + +2024.04.23: +. Initial screen_buffer implementation. + +2024.04.20: +. Small detail on draw_line. + +2024.04.17: +. Removed newly unused telegram_bot_sendmessage() from broker. + +2024.04.16: +. Changed some variable names in profit_to_db. + +2024.04.15: +. Added get_write_order_history in broker. + +2024.04.14: +. Fixed bug while toggling Telegram notifications. + +2024.04.13: +. Consolidation of profit_db changes. + +2024.04.12: +. The exception that occurs when there is an error writing status file does not add an entry to the log. + +2024.04.11: +. Initial implementation of profit_to_db. + +2024.04.10a: +. Moved get_min_base_size, get_min_quote_size and get_step_size to broker. + +2024.04.10: +. Changed a conditional in return_status. +. Changed min_base_size calculation for Kucoin. + +2024.04.09: +. Removed market from status_dict. Since market() calls on CCXT are local, it does not make any sense to cache them. +. Added deal_order_history to the status_dict, which will be saved in a future implementation of profit_db. + +2024.04.08: +. In new_limit_order: replaced tries50 traders. +. Modified some strings +. Fixed a bug that led to online_pairs not be cleared on each iteration +. Removed old code and adaptative_so_amount (and its api endpoint), not useful at all + +0.9i: +. Added handling for a specific division by zero error (rare) + +0.9h: +. Modified a few variable declarations +. Now it restarts the pair when the TP order is missing + +0.9g: +. Modified log strings +. Added a Telegram message when switching back from short to long bot +. Fixed an order bug in liquidate_base() + +0.9f: +. Modified a few strings +. Added a log output when rewriting the broker config file + +0.9e: +. Added an endpoint that returns the last trader time + +0.9d: +. Added an endpoint that returns server time + +0.9c: +. Now it sends a Telegram message if a pair errors out. Mostly for debugging, probably will be removed later. + +0.9b: +. Changed a few self.quit for self.restart + +0.9a: +. Fixed a bug when processing old long info +. Now it searches for the .oldlong file if it can't find the info in status_dict when switching from short to long +. Added a config option for attempting pair restart +. Commented out some offline pairs restart code +. Enabled the change in 0.8o + +0.9: +. Now it sends a Telegram notification when a trader errors out + +0.8t: +. restart_pair() now cancels pending buy orders for long traders +. Implemented automatic trader restart on error + +0.8s: +. Now it shows one decimal less. + +0.8r: +. Modified return_status() so it does not return any number in scientific notation + +0.8q: +. Updated empty_order +. Added a pause after cancelling an order +. Added two more decimals to the displayed prices + +0.8p: +. Some minor refactorings here and there +. Removed a few spaces in the console output +. Prices shown in console output now to precision +. Chasing a stop_when_profit bug + +0.8o: +. Refactored return_status() +. Now it rereads the total amount of base on the exchange in the event of a take profit order rejected for lack of base. + This could be nasty, will follow it closely + +0.8n: +. Removed exceeded pairs notice on console +. Removed profit from console view. Might replace it with the amount of deals closed later. +. Added the exchange to the exceeded pair Telegram notification +. Added an API endpoint to switch the check of old long prices on or off. +. Fixed a bug in the missing pairs function that caused it to return an empty list +. Fixed a bug that colored an extra character in short mode +. Added exception handling to update_status +. Fixed alignment of pair and safety order info + +0.8m: +. Added pair restart +. Now the minimum amount for cleanup is double the minimum order size imposed by the exchange (may increase the amount of open orders on the exchange) +. Added a Telegram warning when the short price exceeds the old long price and a notice on the console output +. Refactored the Telegram messaging methods +. Added is_short to status +. Allowed multiple restart attempts of a bot (specially useful on Gate.io) + +0.8l: +. Fixed a bug which led to an empty safety price table + +0.8k: +. Minor change in error handling in exchange_wrapper + +0.8j: +. Added an API endpoint that returns the pairs that are included in the config file but not running + +0.8i: +. Fixed a bug when simulating market orders +. Minor refactorings +. Fixed unnecessary fee substraction when the trader is short + +0.8h: +. base_profit now can't be less than zero (maybe that could have been a problem at a later stage?) +. Added colors to health_index +. Corrected a bug when detecting duplicate profit reports +. Added a message that shows the amount of loose change available when taking profit + +0.8g: +. Now it displays a message when sending a new safety order +. Removed cosine table. It was stupid. +. Now it removes the pair from the config file when the last deal is closed +. Programmable last call added. +. Added support for FTX when calculating minimum order sizes +. Added timestamp to "free base" screen message +. Now it displays order size as an integer when possible when starting a short bot +. Added base_change to take into account rounding errors in short traders depleting the base amount + +0.8f: +. Added a check for duplicate profit reporting + +0.8e: +. Now it calculates on the fly the optimal order size for short bots +. Refactored some code in exchange_wrapper + +0.8d: +. Reimplemented switch_to_long +. Added a few docstrings +. Now it double checks if the pair is really offline when restarting an offline pair + +0.8c: +. Uncommented self.safety_order_index+1 in new_so_routine +. Doubled the amount of retries when starting a bot + +0.8b: +. Added deal price to the profit message displayed onscreen +. Upon error, get_open_orders only returns None + +0.8a: +. Modified the trader pause procedure +. Fixed a bug that was present when importing a single trader +. Modified the step size when switching to short + +0.7u: +. Fixed adaptative_so_amount when importing trader + +0.7t: +. Added a list of paused traders to be displayed onscreen if any are paused +. Fixed a route typo when saving old long info +. Added adaptative so amount: + - When switching to short mode, if the base is not enough, instead of returning an error it decreases the safety order amount + to calculate the order size. + - This is helpful in situations when the minimum order size of the pair is too big, or the trader long order size was too small. + - The lower limit is hardcoded to half the maximum amount of safety orders. + - I can't think of a scenario where having less orders than that is optimal. +. Changed return API string when toggling cleanup + + +0.7s: +. Changed the order of the conditionals when checking for autoswitch within the take profit routine +. Now when a trader switches to short, it writes the tp info on a different file, in case the trader crashes and restarts +. To the old_tp entry, a timestamp was added +. Removed duplicate method in trader.py +. Added an endpoint to toggle trader pause +. Improved some error logging + +0.7r: +. Added a pause character to indicate paused pairs. + +0.7q: +. Modified the removal routine of "last call" marked pairs. +. Commented out some lines dealing with old specific bugs. + +0.7p: +. Removed write_to_disk conditional, now it only writes to disk if told to. +. Fixed a wrong argument passed to fetch_market. + +0.7o: +. Fixed a wrong color bug. + +0.7n: +. Writes updated status file as soon as it finishes importing all the values. + +0.7m: +. In switch_to_long, toggled the is_short flag after messing with the config files. +. Removed unused precision variable from trader object. +. Minor string editing. +. Price now shows bright green if, within a short bot, the current price is larger than the old take profit price. + +0.7l: +. Commented out global status file loading. +. Removed some unnecesary ticker requests in get_min_base_size method. +. Added a double check before liquidating base. +. Added a switch_to_long API endpoint (Mostly for testing, but it might be useful someday). + +0.7k: +. Rounded to 6 decimals the base amount of the telegram notification. +. In that same message, it was needed to substract 1 from safety_order_index. +. Refactored the calculation of minimal base size (Thanks, CCXT) +. Minor refactorings here and there + +0.7j: +. Uncommented a few lines +. Now it checks if the pair is already running when adding pairs through API +. Added trader pause in market_close, add_quote, close_trader, global_close + +0.7i: +. Minor bug fixes and refactorings +. Fixed importing didn't took into account the original start_time and deal_start_time +. Switches on the self.pause flag in take_profit_routine and new_so_routine methods +. Moved the autoswitch routine to check_status + +0.7h: +. Added a notice when autoswitch is on + +0.7g: +. Now the pause flag of a trader is on by default, it only switches it off once it's properly started +. Refactored offline pairs routine + +0.7f: +. Now checks also for autoswitch when closing a short deal + +0.7e: +. Added a pause flag to not process orders when an API call manhandles the trader + +0.7d: +. Added safety order count to the Telegram take profit message + +0.7c: +. Divided dca_classes into trader and exchange_wrapper modules +. Implemented truncate method to easily round down numbers to precision, in preparation to implement a sans-ccxt version +. Reduced retry wait times in exchange_wrapper +. Now it also indicates the amount of base that was not sold (happens on some exchanges when tick size is too high) + +.0.7b: +. Changed the order of operations in switch_to_short to account for possible bad outcomes +. Changed the code when fetching market info due to inconsistencies between different exchange implementations in CCXT + (E.g., the fetch_ticker method returns ticker size info on FTX but not on Binance) + +.0.7a: +. new_limit_order now returns None when it encounters errors +. Fixed duplicated cleanup message +. First implementation of switch to short +. First implementation of autoswitch (off by default) +. Removed redundant variables +. Reduces safety order deviance when switching to short + +.0.6h: +. Changed the first conditional in take_profit_routine() +. Changed the order of sending take profit order and safety order, so if there are not enough funds the bots don't close any orders + +.0.6g: +. Some fixes on the telegram notification text +. Removed a restriction when importing traders +. Removed bug-hunting code in check_status (if safety_order_index==0) +. Refactoring at "#Check if SO order is filled" + +.0.6f: +. Status dictionary didn't have deal start time and trader start time. Commented out for now. +. Added price to telegram message + +.0.6e: +. Fixed importing bots with all SOs filled. +. Previous bug not fixed at all. Retrying. +. Short traders now are displayed in yellow. +. More changes dealing with safety orders. + +.0.6d: +. Fixed a bug when adding SOs. + +.0.6c: +. Fixed a few lines that substracted fees in quote when it wasn't necessary. + +.0.6b: +. When they were no more opened orders on the exchange, there was a bug that kept the script running as if life was normal. Fixed that. + +.0.6a: +. Some cosmetic changes regarding short bots. +. Fixed market close commands for compatibility with short bots. + +.0.6: +. Preliminary support for short bots. Market close API commands not compatible yet. + +.0.5f: +. Cleanup routine now only sweeps half of what it used to be. DCAing dust might be a good idea. +. Removed some keys from global_status["config"] that should not be public. +. Fixed two flags that signal showing more or less info on the terminal window. +. Reformatted a few strings. +. Now it displays the filled safety orders instead of the open safety order number. + +0.5e: +. Now it doesn't try to cancel a non-existent safety order when the deal that closes have consumed all safety orders. + +0.5d: +. Fixed a minor bug that miscounted the maximum safety orders when importing a trader that had more than the default value. + +0.5c: +. Serialized trader startup +. Added the endpoints for toggling less_info and even_less_info +. Updated some sleep timers +. Added two more decimals when displaying prices + +0.5b: +. global_status api endpoint used to return telegram keys. Not anymore. +. Small cleanup here and there +. Added a space in write_to_log method + +0.5a: +. Several minor bug fixes +. Removed the "Press Ctrl+C..." message +. Moved Telegram data to exchange config file + +0.5: +. First API implementation +. Exchange config file now has a host and port entry +. Several bug fixes and exception handling routines implemented + +0.4q: +. Removed a bug in the pair removal process +. Added a check for empty order list + +0.4p: +. In the config file, the time between restarts attempts now is specified in seconds, rather than minutes. +. Fixed? a bug that caused a restart of a pair that was marked for deletion + +0.4o: +. get_tp_level: Implemented dynamic take profit percentage (between +0.5% and -0.5% of config file's tp_level) + The first order will always be +0.5%, the first third without change (this is where almost all deals end up closing), + the second third -0.25% and the last third -0.5% +. get_tp_level_v2: Also there's an alternative available: it generates a cosine table and uses it as a continuous modifier for the tp_level. + Due to the nature of the cosine function, it is a bit more agressive when setting the percentage in the first few tp orders. + Probably worth a try if the markets are green, perhaps a bit too greedy if the markets are on a downtrend. + +.0.4n: +. Code cleanup (automatically recognizing when there is the need to simulate a market order) + +.0.4m: +. Code cleanup +. Added a loop to fetch_orders() +. Removed write_try() +. When simulating market orders, the initial order was not canceled if retries were exhausted. + +.0.4l: +. Modified a few market order calls that did not support Gate.io +. Removed average deal uptime. + +.0.4k: +. Doubled the amount of retries when there's an API bug (Because of a lovely exchange that starts with K and ends with ucoin) +. Added "clear offline pair list" command +. "Remove pair" now also deletes the pair if it is included in the offline pairs list +. The onscreen take profit message now truncates the number to 4 decimals +. Added an exception counter. The idea is to see which errors pop up more often and try to eliminate unnecessary exception catchers +. Fixed an error when adding funds. It didn't work in OKX as it didn't discount the fees of the buy order (in base currency) +. Zero-balance Kucoin bug handled by retrying the order. Commenting out that rounding stuff. +. Added average deal uptime. +. Added Gate.io support (simulating market orders mostly) + +.0.4j: +. Added a few time.sleep(0.5) when iterating for an order, because Kucoin. +. Enabled cleanup for Binance +. Minor code cleanup + +.0.4i: +. Added a few exceptions trying to address the 0 balance bug mentioned in the code +. Added a looped exception to get_order + +.0.4h: +. Eliminated a few time.sleep() that were unnecessary +. Added a looped exception to the start bot routine. I suspect that kucoin balance sometimes hits 0 (The reason? Maybe the balance goes to + 0 momentarily while setting up an order) and that is what is making the orders fail. If that is true, the best solution would be to + implement async-await, so it does not try to open two orders at the same time. +. Added a message on screen when a deal is closed + +.0.4g: +. Corrected a small error that serialized the initialization of the traders in all exchanges. + +.0.4f: +. Implemented rounding routine. + +.0.4e: +. Exchange name now appears in uppercase. +. Corrected a wild carriage return when enabling the "even less info" mode. +. Changed the take profit routine so it only calculates profits if there are no errors with the tp order. +. Added a price list dictionary to attempt to reduce API load (It loads all prices at once instead of fetching them one by one) + Some exchanges do not like this and prefer the old way. +. A few changes in new_so_routine to try to catch a bug that causes the new tp order not to be sent because of not enough base + If that does not work, I'll implement a decreasing rounding routine, just like in the previous version. + +.0.4d: +. Changes in the draw_line function + +.0.4c: +. Added a command to remove a pair without closing the orders +. Serialized the start of the bots for kucoin. Also added an "if open" clause when cancelling orders, also for kucoin. IH8U kucoin! + +.0.4b: +. Added another tp check + +.0.4a: +. Now it prints the exchange's name on the screen. +. Now it also sends the profit order id to telegram +. Minor bug fixes + +.0.4: +. General cleanup and bug-hunting. +. Eliminated a few redundant commands +. Ready for primetime? + +.v0.3d: +.dca_classes.py: + . Modified get_single_price. Now it does not return a tuple but a float. +.dca_runtime.py: + . Made the necessary changes to accomodate the update above. + +v0.3c: +.dca_classes.py: + . Modified send_new_safety_order() in order to catch a bug that results in 0/30 safety orders in an imported_trader +.dca_runtime.py: + . draw_line(): line width back to 80 + +v0.3b: +.dca_classes.py: + . No changes +.dca_runtime.py: + . You can now save added pairs to the main config file + +v0.3a: +.dca_classes.py: + . No changes +.dca_runtime.py + . Now it isn't necessary to input the pair without the / in some commands + +v0.3: +.dca_classes.py: + . Import bot implemented: + . Created a child class imported_trader + . Now it writes the status_dict to a file +.dca_runtime.py: + . Import bot implemented + +v0.2c: +.dca_classes.py: + . No changes +.dca_runtime.py: + . Multithreaded "close all pairs and quit" + +v0.2b: +.dca_classes.py: + . No changes +.dca_runtime.py: + . Fixed bugs in cleanup command. + . Line width is now 90 characters. + +v0.2a: +.dca_classes.py: + . No changes +.dca_runtime.py + . Multiple corrections to address issues found after testing. + +v0.2: +.dca_classes.py: + . No changes +.dca_runtime.py: + . Finished implementing commands. Now to test them + +v0.1d: +.dca_classes.py: + . No changes +.dca_runtime.py: + . Pair now is shown in blue + . Attempt to reconnect counter now does not show negative numbers + . Implemented more commands + +v0.1c: +.dca_classes.py: + . No changes +.dca_runtime.py: + . Added the ability to set a timer to attempt to restart an instance + +v0.1b: +. dca_classes.py: + . Now it reloads the config file every time it closes a deal. +. dca_runtime.py: + . When you add an instance, now it adds it to an intermediate list before adding it to the main running_instances list. + Sometimes it would trigger an error (Threads can only be started once) if you pressed Ctrl-C at the exact wrong time. + +v0.1a: +. Initial VPS deployment \ No newline at end of file diff --git a/configs/example_exchange_config.json b/configs/example_exchange_config.json new file mode 100755 index 0000000..fda9617 --- /dev/null +++ b/configs/example_exchange_config.json @@ -0,0 +1,25 @@ +{ + "exchange": "binance", + "is_sandbox": true, + "simulate_market_orders": false, + "key": "", + "secret": "", + "pairs": [ + "BTCUSDT", + "BNBUSDT", + "ETHUSDT", + "XRPUSDT", + "TRXUSDT", + "LTCUSDT" + ], + "reconnect": 30, + "lap_time": 1, + "host": "0.0.0.0", + "port": "5006", + "telegram": false, + "bot_token": "", + "bot_chatID": "", + "attempt_to_restart": true, + "default_order_size": 15, + "unified_order_query": true +} diff --git a/configs/example_trader_config.json b/configs/example_trader_config.json new file mode 100755 index 0000000..e2c4e3f --- /dev/null +++ b/configs/example_trader_config.json @@ -0,0 +1,21 @@ +{ + "pair": "BCC/USDT", + "order_size": 15, + "tp_level": 1.02, + "no_of_safety_orders": 23, + "safety_order_deviance": 2, + "safety_order_scale": 0.0105, + "write_logs": false, + "calculate_fees": true, + "cleanup": true, + "telegram": true, + "tp_mode": 3, + "tp_table": [], + "is_short": true, + "autoswitch": true, + "attempt_restart": true, + "check_old_long_price": true, + "dynamic_so_deviance": true, + "dsd_range": 1, + "bias": -0.5 +} diff --git a/duster.py b/duster.py new file mode 100644 index 0000000..e6db56b --- /dev/null +++ b/duster.py @@ -0,0 +1,217 @@ +import time +import json +import csv + +class duster: + def __init__(self,broker,status_info,current_price,importing=False): + ''' + Initializes duster. + + broker: broker object from exchange_wrapper + status_info: old status_dict dictionary of the trader or old duster_status from an old duster + importing: Boolean value, if True, the status_info dictionary is already formated as a duster. If not, it's formatted as an old status_dict dictionary from a trader. + ''' + + self.broker = broker + if not importing: + order = self.broker.get_order(status_info["tp_order_id"],status_info["market"]["symbol"]) + self.duster_status = {"duster_id": status_info["tp_order_id"], + "pair": status_info["market"]["symbol"], + "amount_spent": status_info["quote_spent"], + "current_price": current_price, + "deal_close_price": order["price"], + "volume_on_close": order["price"]*order["amount"], + "start_time": time.time() + } + else: + self.duster_status = status_info + + self.quit = False + self.pause = False #Maybe useful in the future + + + def __str__(self): + ''' + BASE/QUOTE | order_id followed | current_price | deal_close_price | total_volume_on_close | pct_to_profit | uptime + ''' + + yellow = "\033[0;33;40m" + green = "\033[0;32;40m" + red = "\033[0;31;40m" + cyan = "\033[0;36;40m" + white = "\033[0;37;40m" + + decimals = 11 + low_percentage = 1 + mid_percentage = 10 + high_percentage = 20 + + #safety_order_string = f"{x.status_dict['so_amount']-1}/{x.config_dict['no_of_safety_orders']}".rjust(5) + + #Check if necessary + mid_price = 0 + high_price = 0 + if self.duster_status["price"] is not None: + mid_price = self.duster_status["current_price"] + if self.duster_status["deal_close_price"] is not None: + high_price = self.duster_status["deal_close_price"] + + mid_boundary = '{:.20f}'.format(mid_price)[:decimals].center(decimals) + high_boundary = '{:.20f}'.format(high_price)[:decimals].center(decimals) + percentage_to_profit = 100 + pct_to_profit_str = "XX.XX" + if self.duster_status not in [0,None] and self.duster_status["current_price"]!=0: + diff = abs(self.duster_status["deal_close_price"]-self.duster_status["current_price"]) + percentage_to_profit = diff/self.duster_status["current_price"]*100 + + #Formatting (on-screen percentage not longer than 4 digits) + pct_to_profit_str = "{:.2f}".format(percentage_to_profit) + if len(pct_to_profit_str)==4: + pct_to_profit_str = f" {pct_to_profit_str}" + elif len(pct_to_profit_str)==6: + pct_to_profit_str = pct_to_profit_str[:5] + + p = "*PAUSED*" if self.pause==True else "" + price_color = white + pair_color = cyan + + #Set percentage's color + pct_color = white + if percentage_to_profitmid_percentage: + pct_color = yellow + if percentage_to_profit>high_percentage: + pct_color = red + + prices = f"{price_color}{mid_boundary}{white}|{green}{high_boundary}{white}" + percentage = f"{pct_color}{pct_to_profit_str}%{white}" + return f"{p}{pair_color}{self.duster_status['pair'].center(13)}{white}| {self.duster_status['id']} |{prices}| {self.duster_status['volume_on_close']} | {percentage} | Uptime: {self.seconds_to_time(time.time()-self.duster_status['start_time'])}" + + + def set_current_price(self,price: float): + ''' + Writes the current price on the status dictionary + ''' + try: + self.duster_status["current_price"] = float(price) + except Exception as e: + self.broker.logger.log_this(f"Error in duster. Set_current_price received a wrong value. {e}",1,self.duster_status["pair"]) + return 0 + + + def get_duster_status(self): + ''' + Returns the status dictionary + ''' + + return self.duster_status + + + def save_duster_status(self): + ''' + Save duster status in a file. + ''' + + try: + with open(f"status/{self.duster_status['id']}.duster_status","w") as status_file: + status_file.write(json.dumps(self.duster_status, indent=4)) + return 0 + except Exception as e: + self.broker.logger.log_this(f"Exception while writing duster status file: {e}",1,self.duster_status["pair"]) + return 1 + + + def load_duster_status(self): + ''' + Loads duster status to the status dictionary. + ''' + + try: + with open(f"status/{self.duster_status['id']}.duster_status") as status_file: + self.duster_status = json.load(status_file) + return 0 + except Exception as e: + self.broker.logger.log_this(f"Exception while reading duster status file: {e}",1,self.duster_status["pair"]) + return 1 + + + def check_duster_status(self,open_orders): + ''' + Checks duster status + ''' + + if open_orders==[] or self.pause: + return 0 + + if self.duster_status["id"] not in open_orders: + order = self.broker.get_order(self.duster_status["id"],self.duster_status["pair"]) + if order["status"]=="": + self.broker.logger.log_this(f"In check_duster_status, get_order returned an empty order",1,self.duster_status["pair"]) + return 1 + elif order["status"]=="closed": + self.report_profit(order) + self.quit = True + + return 0 + + + def report_profit(self,closed_order): + ''' + Saves profit in the profit file and sends the telegram notification. + ''' + + _, fees_paid = self.parse_fees(closed_order) + profit = closed_order["cost"]-self.duster_status["amount_spent"]-fees_paid + + #Write to file + profit_filename = f"profits/{self.duster_status['pair'].remove('/')}.profit" + try: + with open(profit_filename,"a") as profit_file: + profit_writer = csv.writer(profit_file, delimiter=",") + profit_writer.writerow([time.strftime("%Y-%m-%d"), profit, closed_order["id"]]) + except Exception as e: + self.broker.logger.log_this(f"Exception in profit_to_file: {e}",1,self.duster_status["pair"]) + + #Send notification + self.broker.logger.log_this(f"Duster closed a deal. Profit: {'{:.4f}'.format(profit)} {self.duster_status['pair'].split('/')[1]}",0,self.duster_status["pair"]) + + return 0 + + + def parse_fees(self,order): + ''' + Returns the fees paid ordered in "base,quote" + ''' + basefee = 0 + quotefee = 0 + base,quote = self.duster_status["pair"].split("/") + + if self.broker.get_exchange_name()=="binance": #CCXT still to this day does not take Binance fees into account. + try: + fee_rate = self.broker.fetch_market["maker"] if order["type"]=="limit" else self.broker.fetch_market["taker"] + except Exception as e: + self.broker.logger.log_this(f"Exception fetching market information: {e}. Using default fee rate of 0.1%",1,f"{base}{quote}") + fee_rate = 0.001 + if order["side"]=="buy": + basefee = order["filled"]*float(fee_rate) + elif order["side"]=="sell": + #To be implemented + #Maybe: + #quotefee = order["cost"]*float(fee_rate) + pass + return basefee,quotefee + + for x in order["fees"]: + if x["currency"]==base: + basefee+=float(x["cost"]) + if x["currency"]==quote: + quotefee+=float(x["cost"]) + return basefee,quotefee + + + def seconds_to_time(self,total_seconds): + ''' + Takes an int or float as an input and it returns a D:HH:MM:SS formatted string. + ''' + return f"{int(total_seconds/86400)}:" + '%02d:%02d:%02d' % (int((total_seconds % 86400) / 3600), int((total_seconds % 3600) / 60), int(total_seconds % 60)) \ No newline at end of file diff --git a/exchange_wrapper.py b/exchange_wrapper.py new file mode 100755 index 0000000..53dc26d --- /dev/null +++ b/exchange_wrapper.py @@ -0,0 +1,928 @@ +import json +import time +import requests +import credentials +import sqlite3 +from copy import deepcopy + + +class broker: + def __init__(self,exchange,read_config,config_filename): + self.config_filename = config_filename + self.read_config = read_config + self.exchange = exchange + self.last_price = 0 + self.wait_time = 1 #Default wait time for API breathing room + self.cooldown_multiplier = 2 #Cooldown multiplier of the value above between trader restarts or when slippage is exceeded (this should be in the config file) + self.empty_order = {"id": "", "status": "", "filled": 0, "remaining": 0, "price": 0, "cost": 0, "fees": [], "symbol": ""} + self.retries = read_config["retries"] if "retries" in self.read_config else 10 + self.slippage_default_threshold = self.read_config["slippage_default_threshold"] if "slippage_default_threshold" in read_config else .03 + self.logger = logger(self.read_config) + self.write_order_history = True #This should be a toggle in config_file + + self.database_connection = sqlite3.connect("profits/profits_database.db") + self.database_cursor = self.database_connection.cursor() + self.database_cursor.execute(''' + CREATE TABLE IF NOT EXISTS profits_table ( + timestamp REAL PRIMARY KEY, + pair TEXT, + amount REAL, + exchange_name TEXT, + order_id TEXT, + order_history TEXT + ) + ''') + self.database_connection.commit() + self.database_connection.close() + + self.exchange.load_markets() + + + def all_markets(self,no_retries=False): + retries = self.retries + while retries>0: + try: + return self.exchange.load_markets() + except Exception as e: + self.logger.log_this(f"Exception in reload_markets: {e}") + if no_retries: + break + retries-=1 + time.sleep(self.wait_time) + return {} + + + def reload_markets(self): + try: + self.exchange.load_markets(reload=True) + return 0 + except Exception as e: + self.logger.log_this(f"Exception in reload_markets: {e}") + return 1 + + + def write_profit_to_db(self,dataset,no_retries=False): + ''' + dataset format: (timestamp,pair,amount,exchange_name,order_id,order_history) + ''' + retries = self.retries + while retries>0: + try: + database_connection = sqlite3.connect("profits/profits_database.db") + database_cursor = database_connection.cursor() + database_cursor.execute('INSERT INTO profits_table VALUES(?, ?, ?, ?, ?, ?)', dataset) + database_connection.commit() + database_connection.close() + except Exception as e: + self.logger.log_this(f"Exception in write_profit_to_db: {e}") + if no_retries: + break + retries-=1 + time.sleep(self.wait_time) + return 0 + return 1 + + + def check_for_duplicate_profit_in_db(self,order,no_retries=False): + ''' + SQLite implementation of check_for_duplicate_profit(): + Compares the id of the last profit order with the one in the database. + ''' + retries = self.retries + while retries>0: + try: + database_connection = sqlite3.connect("profits/profits_database.db") + database_cursor = database_connection.cursor() + database_cursor.execute(f"SELECT * FROM profits_table WHERE pair = '{order['symbol']}' ORDER BY timestamp DESC LIMIT 1;") + rows = database_cursor.fetchall() + database_connection.close() + if rows==[]: + return False + return order["id"]==rows[0][4] + except Exception as e: + self.logger.log_this(f"Exception in check_for_duplicate_profit_in_db: {e}",1) + if no_retries: + break + retries-=1 + time.sleep(self.wait_time) + return False + + + def get_write_order_history(self): + return self.write_order_history + + def get_cooldown_multiplier(self): + return self.cooldown_multiplier + + def set_cooldown_multiplier(self, value:int): + self.cooldown_multiplier = value + return 0 + + def get_default_order_size(self): + return self.read_config["default_order_size"] + + + def set_default_order_size(self,size): + try: + self.read_config["default_order_size"] = float(size) + except Exception as e: + self.logger.log_this(f"Exception in set_default_order_size: {e}",1) + return 1 + return 0 + + + def get_slippage_default_threshold(self): + return self.slippage_default_threshold + + + def set_slippage_default_threshold(self,threshold): + try: + self.slippage_default_threshold = float(threshold) + return 0 + except Exception as e: + self.logger.log_this(f"Exception in set_slippage_default_threshold: {e}") + return 1 + + + def get_retries(self): + return self.retries + + + def set_retries(self,amount): + try: + self.retries = int(amount) + return 0 + except Exception as e: + self.logger.log_this(f"Exception in set_retries: {e}") + return 1 + + + def get_empty_order(self): + return self.empty_order + + + def get_exchange_name(self): + return self.read_config["exchange"] + + + def set_wait_time(self,sec): + ''' + Sets the default wait time between some API calls + ''' + try: + new_time = float(sec) + except Exception as e: + self.logger.log_this(f"Exception in set_wait_time: {e}") + return 1 + self.wait_time = new_time + return 0 + + + def get_wait_time(self): + ''' + Returns the default wait time between some API calls + ''' + return self.wait_time + + + def get_config(self): + return deepcopy(self.read_config) + + + def set_config(self,new_config): + self.read_config = deepcopy(new_config) + return 0 + + + def reload_config_file(self): + try: + with open(self.config_filename) as f: + self.read_config = json.load(f) + except Exception as e: + self.logger.log_this(f"Exception while reading the config file: {e}",1) + + + def add_pair_to_config(self,pair): + if pair not in self.read_config["pairs"]: + self.read_config["pairs"].append(pair) + return 0 + return 1 + + + def remove_pair_from_config(self,pair): + try: + if pair in self.read_config["pairs"]: + self.read_config["pairs"].remove(pair) + return 0 + self.logger.log_this("Pair does not exist - Can't remove from read_config",1,pair) + return 2 + except Exception as e: + self.logger.log_this(f"Problems removing pair: {e}",1,pair) + return 1 + + + def get_pairs(self): + return self.read_config["pairs"] + + + def clear_pairs(self): + self.read_config["pairs"].clear() + return 0 + + + def get_lap_time(self): + return self.read_config["lap_time"] + + + def set_lap_time(self,new_lap_time): + try: + self.read_config["lap_time"]=float(new_lap_time) + return 0 + except Exception as e: + self.logger.log_this(f"Can't set new lap time. {new_lap_time} is an invalid entry. Exception: {e}",1) + return 1 + + + def rewrite_config_file(self): + try: + with open(f"{self.config_filename}","w") as f: + f.write(json.dumps(self.read_config, indent=4)) + return 0 + except Exception as e: + self.logger.log_this(f"Problems writing the config file. Exception: {e}",1) + return 1 + + + def get_order_book(self,symbol,no_retries=False): + ''' + Returns the complete orderbook + ''' + retries = self.retries + while retries>0: + try: + return self.exchange.fetch_order_book(symbol) + except Exception as e: + self.logger.log_this(f"Exception in get_order_book: {e}",1) + if no_retries: + break + time.sleep(self.wait_time) + retries-=1 + return {} + + + def find_minimum_viable_price(self,order_book,amount,side): + suma = 0 + data = order_book["bids"] if side=="sell" else order_book["asks"] + for x in data: + suma += x[1] + if suma>=amount: + return x[0] + + + def get_prices(self,pair_list=None,no_retries=False): + ''' + Returns the closing prices of all the pairs in the pair_list list + + :param pair_list: list of pairs to get prices for + :param no_retries: if True, the function will not retry if it fails + :return: dictionary {pair: price} + ''' + + retries = self.retries + while retries>0: + try: + if self.read_config["exchange"]=="binance": + a = self.exchange.fetch_last_prices(pair_list) + return {x: a[x]["price"] for x in a.keys()} + else: + #a = self.exchange.fetch_tickers(pair_list) + a = self.exchange.fetch_tickers() + #return {x.upper(): a[x]["close"] for x in a.keys() if x.upper() in pair_list} + if pair_list is None: + return {x: a[x]["close"] for x in a.keys()} + return {x: a[x]["close"] for x in a.keys() if x in pair_list} + except Exception as e: + self.logger.log_this(f"Exception in get_prices: {e}",1) + if no_retries: + break + time.sleep(self.wait_time) + retries-=1 + return {} + + + def get_ticker_price(self,symbol,no_retries=False): + ''' + Returns the closing price of a trading pair + + :param symbol: trading pair symbol + :param no_retries: if True, will not retry if exception occurs + :return: closing price of trading pair + ''' + + retries = self.retries + while retries>0: + try: + pair = symbol + a = self.exchange.fetch_ticker(pair) + self.last_price = a["close"] + return self.last_price + except Exception as e: + self.logger.log_this(f"Exception in get_ticker_price: {e}",1) + if no_retries: + break + time.sleep(self.wait_time) + retries-=1 + return self.last_price + + + def get_mid_price(self,symbol): + ''' + Retrieves the orderbook and returns the average price [(top bid + top ask)/2] + + :param symbol: the symbol to get the mid price for + :return: the mid price + ''' + + orderbook = self.get_order_book(symbol) + if orderbook=={}: + self.logger.log_this("Can't fetch orderbook (from get_mid_price)",1,symbol) + return self.get_ticker_price(symbol) + try: + mid_price = (orderbook["asks"][0][0]+orderbook["bids"][0][0])/2 + except Exception as e: + self.logger.log_this(f"Exception getting mid_price: {e}",1,symbol) + return self.get_ticker_price(symbol) + return self.price_to_precision(symbol,mid_price) + + + def get_coins_balance(self,no_retries=False): + ''' + Retrieves the balance of all coins on the exchange + + :param no_retries: if True, it will not retry on failure + :return: list of all coins and their balance on the exchange + ''' + + retries = self.retries + while retries>0: + try: + return self.exchange.fetch_balance() + except Exception as e: + self.logger.log_this(f"Exception in get_coins_balance: {e}",1) + if no_retries: + break + time.sleep(self.wait_time) + retries-=1 + return [] + + + def fetch_full_orders(self,pairs=None) -> list: + ''' + Returns a list of all orders on the exchange + + :param pairs: list of pairs to get orders for + :return: list of orders + ''' + + if pairs is None: + pairs = [] + try: + orders = [] + if self.read_config["exchange"]=="binance": + orders = self.get_opened_orders_binance(pairs) + else: + orders = self.get_opened_orders() + return [] if orders is None else orders + except Exception as e: + self.logger.log_this(f"Exception in fetch_full_orders: {e}",2) + return [] + + + def fetch_open_orders(self,pairs=None) -> list: + ''' + Returns a list of IDs of all open orders on the exchange + + :param pairs: list of pairs to get opened orders + :return: list of IDs of all open orders + ''' + + if pairs is None: + pairs = [] + try: + id_list = [] + if self.read_config["exchange"]=="binance": + orders = self.get_opened_orders_binance(pairs) + else: + orders = self.get_opened_orders() + if orders!=[]: + id_list.extend(x["id"] for x in orders) + return id_list + except Exception as e: + self.logger.log_this(f"Exception in fetch_open_orders: {e}",2) + return [] + + + def get_opened_orders(self,no_retries=False): #It should return a list of all opened orders + ''' + Returns a list of all the orders on the exchange + + :param pairs: list of pairs + :return: list of all the open orders on the exchange + ''' + + retries = self.retries + while retries>0: + try: + return self.exchange.fetch_open_orders() + except Exception as e: + self.logger.log_this(f"Exception in get_opened_orders: {e}",1) + if no_retries: + break + time.sleep(self.wait_time) + retries-=1 + return [] + + + def get_opened_orders_binance(self,pairs): + ''' + Returns a list of all the open orders on the exchange + + :param pairs: list of pairs + :return: list of all the open orders on the exchange + ''' + + try: + if "unified_order_query" in self.read_config and self.read_config["unified_order_query"] is True: + return self.exchange.fetch_open_orders() + result = [] + for pair in pairs: + a = self.exchange.fetch_open_orders(pair) + result.extend(iter(a)) + return result + except Exception as e: + self.logger.log_this(f"Exception in get_opened_orders_binance: {e}",1) + return [] + + + def cancel_order(self,id,symbol,no_retries=False): + ''' + Receives an order id and cancels the corresponding order + + :param id: order id + :param symbol: pair + :param no_retries: if True, the function will not retry to cancel the order + :return: 0 if order was succesfully canceled, 1 if not + ''' + + pair = symbol + tries = self.retries//2 + while tries>0: + try: + while self.get_order(id,pair)["status"]=="open": + self.exchange.cancel_order(id,symbol=pair) + time.sleep(self.wait_time) + return 0 + except Exception as e: + if self.get_order(id,pair)["status"]=="canceled": + return 0 + self.logger.log_this(f"Exception in cancel_order: id {id} - exception: {e}",1) + if no_retries: + break + time.sleep(self.wait_time) + tries-=1 + return 1 + + + def amount_to_precision(self,pair,amount): + try: + return float(self.exchange.amount_to_precision(pair,amount)) + except Exception as e: + self.logger.log_this(f"Can't convert amount {amount} to precision. Exception: {e}",1,pair) + return amount + + + def price_to_precision(self,pair,price): + try: + return float(self.exchange.price_to_precision(pair,price)) + except Exception as e: + self.logger.log_this(f"Can't convert price {price} to precision. Exception: {e}",1,pair) + return price + + + def cost_to_precision(self,pair,amount): + try: + return float(self.exchange.cost_to_precision(pair,amount)) + except Exception as e: + self.logger.log_this(f"Can't convert cost {amount} to precision. Exception: {e}",1,pair) + return amount + + + def new_simulated_market_order(self,symbol,size,side,amount_in_base=False,no_retries=False): + ''' + TODO: Emulating Market Orders With Limit Orders + + It is also possible to emulate a market order with a limit order. + + WARNING this method can be risky due to high volatility, use it at your own risk and only use it when you know really well what you're doing! + + Most of the time a market sell can be emulated with a limit sell at a very low price – the exchange will automatically make it a taker order for market price + (the price that is currently in your best interest from the ones that are available in the order book). When the exchange detects that you're selling for a very low price + it will automatically offer you the best buyer price available from the order book. That is effectively the same as placing a market sell order. Thus market orders can be + emulated with limit orders (where missing). + + The opposite is also true – a market buy can be emulated with a limit buy for a very high price. Most exchanges will again close your order for best available price, + that is, the market price. + + However, you should never rely on that entirely, ALWAYS test it with a small amount first! You can try that in their web interface first to verify the logic. You can sell + the minimal amount at a specified limit price (an affordable amount to lose, just in case) and then check the actual filling price in trade history. + + :param symbol: The symbol of the asset you want to place a market order for. + :param size: The size of the order you want to place. + :param side: The side of the order you want to place (buy or sell) + :param amount_in_base: Signals is the size parameter is nominated in base or quote currency + :param no_retries: If True, the function will not try to fetch the order again if it fails + ''' + + retries = self.retries//2 + pair = symbol + while retries>0: + try: + if self.read_config["exchange"]=="gateio" and side=="buy" and not amount_in_base: + new_order = self.exchange.create_market_buy_order_with_cost(pair, size) + else: + order_book = self.get_order_book(symbol) + if order_book=={}: + self.logger.log_this(f"new_simulated_market_order. Order book returned an empty dictionary",1,symbol) + return self.empty_order + if amount_in_base or side!="buy": + base_amount = self.amount_to_precision(pair,size) + else: + avg_price = self.average_price_depth(order_book,size,"sell") + base_amount = size/avg_price if avg_price is not None else size/self.get_ticker_price(symbol) + price = self.find_minimum_viable_price(order_book,base_amount,side) + #Maybe check for slippage here instead of within the trader itself? idk + new_order = self.exchange.create_order(pair,"limit",side,base_amount,price) + time.sleep(self.wait_time) + return self.get_order(new_order["id"],pair) + except Exception as e: + self.logger.log_this(f"new_simulated_market_order exception: {e}",1,symbol) + if no_retries: + break + time.sleep(self.wait_time) + retries -= 1 + return self.empty_order + + + def weighted_average(self,prices,weights): + ''' + Given a list of prices and a list of weights, returns the weighted average of those prices. + + :param prices: list of prices + :param weights: list of weights + ''' + + return sum(prices[i]*weights[i] for i in range(len(prices)))/sum(weights) + + + def average_price_depth(self,order_book,size,side): + ''' + Given the size of the order in quote, it returns the average price that a market BUY order of that amount would get in the + current orderbook. + + :param order_book: the orderbook + :param size: the size of the order in quote + :param side: the side of the order + ''' + + quote = 0 + prices = [] + weights = [] + dataset = order_book["asks"] if side=="buy" else order_book["bids"] + for x in dataset: + prices.append(x[0]) + weights.append(x[1]) + quote+=x[1]*x[0] + if quote>=size: + #Now we calculate the weighted average + return self.weighted_average(prices,weights) + return None + + + def new_market_order(self,symbol,size,side,amount_in_base=False,no_retries=False): #It should send a new market order to the exchange + ''' + Sends a new market order to the exchange. + + :param symbol: The symbol of the asset. + :param size: The size of the order. + :param side: The side of the order. + :param amount_in_base: Whether the amount is nominated in base or quote currency. + :param no_retries: If True, the function will not try to fetch the order again if it fails + ''' + + if self.read_config["simulate_market_orders"]: + return self.new_simulated_market_order(symbol,size,side,amount_in_base=amount_in_base) + retries = self.retries + pair = symbol + while retries>0: + try: + if side=="buy": + to_buy = float(size) + if not amount_in_base: + order_book = self.get_order_book(symbol) + if order_book=={}: + self.logger.log_this(f"new_market_order - Orderbook request returned an empty dictionary - Not using orderbook. Switching to a less precise method.",1,symbol) + price = self.get_ticker_price(symbol) + else: + price = self.average_price_depth(order_book,size,side) + if price is None: + #Something failed, back to the basics + self.logger.log_this(f"new_market_order - average_price_depth returned None. Switching to a less precise method.",1,symbol) + price = self.get_ticker_price(symbol) + to_buy = float(size)/price + amount = self.amount_to_precision(pair,to_buy) + else: + amount = self.amount_to_precision(pair,size) #Market sell orders are ALWAYS nominated in baseexchange.create_order(pair,"market",side,amount) + + #self.logger.log_this(f"Order to be sent: {side} {amount}",1,pair) + order_to_send = self.exchange.create_order(pair,"market",side,amount) + time.sleep(self.wait_time) + return self.get_order(order_to_send["id"],pair) + # Because Kucoin "order does not exist" problem + #if order_to_send["amount"] is not None: # + # return self.get_order(order_to_send["id"],pair) # + #self.logger.log_this(f"Error sending order: Null order returned",2,pair) # + #self.cancel_order(order_to_send["id"],symbol,no_retries=True) # + #retries-=1 # + except Exception as e: + self.logger.log_this(f"Exception in new_market_order: {e}",1,pair) + if no_retries: + break + time.sleep(self.wait_time) + retries-=1 + return None + + + def not_enough_balance_error(self, error_object): + ''' + Checks if the error is a balance error. + Receives an error object. + Returns True if the error is a balance error, False otherwise. + + :param error_object: The error object. + :return: Boolean value. + ''' + + error_text = str(error_object) + return "insufficient" in error_text.lower() or "BALANCE_NOT_ENOUGH" in error_text or "Low available balance" in error_text + + + def new_limit_order(self,symbol,size,side,price,no_retries=False): + ''' + Sends a new limit order. + + :param symbol: The symbol of the order. + :param size: The size of the order. + :param side: The side of the order. + :param price: The price of the order. + :param no_retries: If True, the function will not retry to send the order if there is an error. + ''' + + tries = self.retries + pair = symbol + while tries>=0: + try: + order_to_send = self.exchange.create_order(pair,"limit",side,self.amount_to_precision(pair,size),price) + time.sleep(self.wait_time) + return self.get_order(order_to_send["id"],pair) + #if order_to_send["amount"] is not None: # Because Kucoin etc etc + # return self.get_order(order_to_send["id"],pair) # + #self.logger.log_this(f"Error sending order: Null order returned",2,pair) # + #self.cancel_order(order_to_send["id"],symbol,no_retries=True) # + #retries-=1 + + except Exception as e: + self.logger.log_this(f"Exception in new_limit_order - Side: {side} - Size: {size} - {self.amount_to_precision(pair,size)} - Exception: {e}",1,symbol) + if self.not_enough_balance_error(e): + if tries<=self.retries//2: #Halves the amount of retries if there is a balance error. + return 1 + if self.get_exchange_name()=="binance": + #If the exchange is Binance, it doesn't try that hard to resend the order: Since CCXT doesn't handle binance fees at all, + #instead of guesstimating the fees, it's easier and more precise to query the exchange for the remaining base currency + #and send the order with that amount. + tries-=1 + if no_retries: + break + tries-=1 + time.sleep(self.wait_time) + return None + + + def get_order(self,id,symbol,no_retries=False): + ''' + Gets an order from the exchange. + :param id: The id of the order. + :param symbol: The symbol of the order. + :param no_retries: If True, the function will not try to fetch the order again if it fails. + :return: The order. + ''' + if id=="": + return self.empty_order + tries = self.retries + pair = symbol + while tries>0: + try: + return self.exchange.fetch_order(id,symbol=pair) + except Exception as e: + self.logger.log_this(f"Exception in get_order: {e}",1,symbol) + if no_retries: + break + time.sleep(self.wait_time) + tries -=1 + return self.empty_order + + + def fetch_market(self,symbol,no_retries=False): + ''' + Returns the market. + :param symbol: The symbol of the market. + :param no_retries: If True, the function will not retry if an exception occurs. + :return: The market information. + ''' + tries = self.retries + pair = symbol + while tries>0: + try: + return self.exchange.market(pair) + except Exception as e: + self.logger.log_this(f"Exception in fetch_market: {e}",1,symbol) + if no_retries: + break + time.sleep(self.wait_time) + tries-=1 + return None + + + def get_ticker(self,symbol,no_retries=False): + ''' + Returns the ticker information. + :param symbol: The trading pair. + :param no_retries: If True, the function will not retry if an exception occurs. + :return: The ticker information. + ''' + tries = self.retries + pair = symbol + while tries>0: + try: + return self.exchange.fetch_ticker(pair) + except Exception as e: + self.logger.log_this(f"Exception in get_ticker: {e}") + if no_retries: + break + tries-=1 + time.sleep(self.wait_time) + return None + + + def get_min_base_size(self,pair): + ''' + Returns the minimum order base size that the exchange supports. + + :param pair: pair + :return: minimum order base size + ''' + + market = self.fetch_market(pair) + if market is None: + return None + if self.get_exchange_name() in ["okex","kucoin"]: + return float(market["limits"]["amount"]["min"]) + elif self.get_exchange_name() in ["gateio"]: + return (float(market["limits"]["cost"]["min"])+1)/self.get_ticker_price(pair) + elif self.get_exchange_name()=="binance": + for line in market["info"]["filters"]: + if line["filterType"] == "NOTIONAL": + return (float(line["minNotional"])+1)/self.get_ticker_price(pair) + return None + + + def get_min_quote_size(self,pair): + ''' + Returns the minimum order size in quote currency. + + :param pair: pair + :return: minimum order quote size + ''' + + market = self.fetch_market(pair) + if market is None: + return None + if self.get_exchange_name()=="binance": + for line in market["info"]["filters"]: + if line["filterType"] == "NOTIONAL": + #return self.broker.amount_to_precision(pair,(float(line["minNotional"]))) + return float(line["minNotional"]) + elif self.get_exchange_name()=="gateio": + #return self.cost_to_precision(pair,float(market["info"]["min_base_amount"])*self.broker.get_mid_price(pair)) + return float(market["limits"]["cost"]["min"]) + elif self.get_exchange_name() in ["okex","kucoin"]: + return self.cost_to_precision(pair,float(market["limits"]["amount"]["min"])*self.get_ticker_price(pair)) + return None + + + def get_step_size(self,pair): + ''' + Returns the step size of the market + + :param pair: pair + :return: step size + + ''' + market = self.fetch_market(pair) + if market is None: + return None + try: + if self.get_exchange_name()=="binance": + for filter in market["info"]["filters"]: + if filter["filterType"]=="LOT_SIZE": + return float(filter["stepSize"]) + elif self.get_exchange_name()=="kucoin": + return float(market["info"]["baseIncrement"]) + elif self.get_exchange_name() in ["gateio","okex"]: + return float(market["precision"]["amount"]) + except Exception as e: + self.logger.log_this(f"Exception in get_step_size: {e}",1,pair) + return None + + +class logger: + def __init__(self,broker_config): + self.broker_config = broker_config + self.exchange_name = self.broker_config["exchange"] + self.tg_credentials = credentials.get_credentials("telegram") + + + def set_telegram_notifications(self, toggle): + try: + self.broker_config["telegram"] = bool(toggle) + except Exception as e: + self.log_this(f"Error in set_telegram_notifications",1) + return 1 + return 0 + + + def send_tg_message(self,message,ignore_config=False): + ''' + Sends a Telegram message + ''' + tg_credentials = credentials.get_credentials("telegram") + send_text = f"https://api.telegram.org/bot{tg_credentials['token']}/sendMessage?chat_id={tg_credentials['chatid']}&parse_mode=Markdown&text={message}" + output = None + if self.broker_config["telegram"] or ignore_config: + output = requests.get(send_text,timeout=5).json() #5 seconds timeout. This could also be a tunable. + if not output["ok"]: + self.log_this(f"Error in send_tg_message: {output}") + return 1 + return 0 + + + def log_this(self,message,level=2,pair=None): + ''' + Level 0: Screen, log file and Telegram + Level 1: Screen and log file + Level 2: Screen only + ''' + + pair_data = "" if pair is None else f"{pair} | " + text = time.strftime(f"[%Y/%m/%d %H:%M:%S] | {pair_data}{message}") + + print(text) + + if level<2: + try: + with open(f"logs/{self.exchange_name}.log","a") as log_file: + log_file.write(text+"\n") + log_file.close() + except Exception as e: + print("Can't write log file") + print(e) + + if level<1: + self.send_tg_message(f"{self.broker_config['exchange'].capitalize()} | {pair_data}{message}",ignore_config=level==-1) + + return 0 + + + #def log_this_API_fail(self,message): + # ''' + # Records the message object to a log file. + # ''' + # + # try: + # with open(f"logs/{self.exchange_name}.api_errors.log","a") as log_file: + # log_file.write(message+"\n") + # except Exception as e: + # print("Can't write API log file") + # print(e) + # + # return 0 + \ No newline at end of file diff --git a/logs/binance.log b/logs/binance.log new file mode 100644 index 0000000..b7dce29 --- /dev/null +++ b/logs/binance.log @@ -0,0 +1,4 @@ +[2024/07/15 14:40:54] | Empty log file +[2024/07/15 14:40:54] | Empty log file +[2024/07/15 14:40:54] | Empty log file +[2024/07/15 14:40:54] | Empty log file diff --git a/logs/gateio.log b/logs/gateio.log new file mode 100644 index 0000000..1a16f33 --- /dev/null +++ b/logs/gateio.log @@ -0,0 +1,4 @@ +[2024/07/15 14:40:47] | Empty log file +[2024/07/15 14:40:47] | Empty log file +[2024/07/15 14:40:47] | Empty log file +[2024/07/15 14:40:47] | Empty log file diff --git a/logs/kucoin.log b/logs/kucoin.log new file mode 100644 index 0000000..1a16f33 --- /dev/null +++ b/logs/kucoin.log @@ -0,0 +1,4 @@ +[2024/07/15 14:40:47] | Empty log file +[2024/07/15 14:40:47] | Empty log file +[2024/07/15 14:40:47] | Empty log file +[2024/07/15 14:40:47] | Empty log file diff --git a/logs/okex.log b/logs/okex.log new file mode 100644 index 0000000..7c7655e --- /dev/null +++ b/logs/okex.log @@ -0,0 +1,4 @@ +[2024/07/15 14:40:51] | Empty log file +[2024/07/15 14:40:51] | Empty log file +[2024/07/15 14:40:51] | Empty log file +[2024/07/15 14:40:51] | Empty log file diff --git a/logs/reset_log_files.py b/logs/reset_log_files.py new file mode 100644 index 0000000..a353b7f --- /dev/null +++ b/logs/reset_log_files.py @@ -0,0 +1,20 @@ +import time +import sys + +text_string = time.strftime(f"[%Y/%m/%d %H:%M:%S] | Empty log file") + +files_to_reset = ["binance", "gateio", "kucoin","okex"] + +if len(sys.argv)>1 and sys.argv[1] in files_to_reset: + files_to_reset = [sys.argv[1]] + +for file in files_to_reset: + with open(f"{file}.log","w") as log_file: + print(f"Resetting {file}") + log_file.write(text_string+"\n") + log_file.write(text_string+"\n") + log_file.write(text_string+"\n") + log_file.write(text_string+"\n") + log_file.write(text_string+"\n") + +print("Done") \ No newline at end of file diff --git a/main.py b/main.py new file mode 100644 index 0000000..1c8fbb8 --- /dev/null +++ b/main.py @@ -0,0 +1,1674 @@ +import datetime +import json +import os +import sys +import time +from threading import Thread + +import sqlite3 +import ccxt + +from flask import Flask, jsonify, request + +import exchange_wrapper +import trader + + +''' +In case the permissions of the certificate changes, reset them this way: +/ sudo su +# chmod -R 755 /etc/letsencrypt/live/ +# chmod -R 755 /etc/letsencrypt/archive/ +# ll /etc/letsencrypt/ +''' + +version = "2024.10.20" + +''' +Color definitions. If you want to change them, check the reference at https://en.wikipedia.org/wiki/ANSI_escape_code#Colors +''' +yellow = "\033[0;33;40m" +green = "\033[0;32;40m" +red = "\033[0;31;40m" +blue = "\033[0;34;40m" +cyan = "\033[0;36;40m" +bright_white = "\033[0;97;40m" +bright_green = "\033[0;92;40m" +white = "\033[0;37;40m" + + +def seconds_to_time(total_seconds: float) -> str: + ''' + Takes an int or float as an input and it returns a D:HH:MM:SS formatted string. + ''' + + time_delta = datetime.timedelta(seconds=total_seconds) + + hours = time_delta.seconds//3600 + remainder = time_delta.seconds%3600 + minutes = remainder//60 + seconds = remainder%60 + + return f"{time_delta.days}:{hours:02d}:{minutes:02d}:{seconds:02d}" + + +def time_to_unix(year: str, month: str, day: str) -> int: + ''' + Takes three integer values as an input and returns the unix time corresponding to that input + ''' + try: + return int(time.mktime(datetime.date(int(year), int(month), int(day)).timetuple())) + except Exception as e: + broker.logger.log_this(f"{e}") + return 0 + + +def import_instance(pair: str) -> int: + broker.logger.log_this(f"Importing {pair}") + #with open(f"status/{pair}.status", "r") as f: + # status_file_contents = json.load(f) + with open(f"configs/{pair}.json", "r") as g: + config_file_contents = json.load(g) + instances_to_add.append(trader.trader(broker,config_file_contents,is_import=True)) + if pair not in tickers: + tickers.append(pair) + return 0 + + +def add_instance(base: str, quote: str) -> int: + #Check if the pair is already running + pair = f"{base}{quote}" + for x in running_instances: + if f"{x.base}{x.quote}"==pair: + broker.logger.log_this(f"Pair already running, duplicate instances are not allowed",1,pair) + return 1 + + #Check if config file already exists; if not, generate a new one + if not os.path.isfile(f"configs/{pair}.json"): + broker.logger.log_this(f"Config file does not exist. Generating...",1,pair) + details = generate_config_file(base,quote) + with open(f"configs/{pair}.json","w") as cf: + cf.write(json.dumps(details, indent=4)) + else: + with open(f"configs/{pair}.json", "r") as cf: + details = json.load(cf) + + #Initialize the trader object and add the pair to the tickers list + instances_to_add.append(trader.trader(broker,details)) + if pair not in tickers: + tickers.append(pair) + + return 0 + + +def initialize_instance(pair: str) -> int: + ''' + Loads the pair config file and initializes the trader object by adding it to the running instances list + ''' + with open(f"configs/{pair}.json", "r") as y: + config_details = json.load(y) + broker.logger.log_this(f"Initializing {pair}") + running_instances.append(trader.trader(broker,config_details)) + if pair not in tickers: + tickers.append(pair) + return 0 + + +def generate_config_file(base: str, quote: str) -> dict: + ''' + Generates a config file with default values for a given pair and returns that content in dictionary form. + TODO: Add a pair check against exchange's tickers data to properly support BASEQUOTE input format (without a slash) + 1. load tickers + 2. search for pair in dictionary + 3. assign proper base and quote values from the dictionary + ''' + return {"pair": f"{base}/{quote}", + "order_size": broker.get_default_order_size(), + "tp_level": 1.02, + "no_of_safety_orders": 30, + "safety_order_deviance": 2, + "safety_order_scale": 0.0105, + "write_logs": True, + "cleanup": True, + "telegram": True, + "tp_mode": 3, + "tp_table": [], + "is_short": False, + "autoswitch": False, + "check_old_long_price": True, + "attempt_restart": True, + "dynamic_so_deviance": True, + "dsd_range": 1, + "slippage_default_threshold": .02 + } + + +def set_exchange(config: dict): + ''' + Takes the config dictionary as an input and returns the exchange object properly configured + ''' + timeout = 10000 + if config["exchange"]=="binance": + exchange_class = getattr(ccxt, "binance") + exchange = exchange_class({ + "apiKey": config["key"], + "secret": config["secret"], + "timeout": timeout, + "enableRateLimit": True + }) + exchange.options["warnOnFetchOpenOrdersWithoutSymbol"] = False + if config["is_sandbox"]: + exchange.set_sandbox_mode(True) + elif config["exchange"]=="kucoin": + exchange_class = getattr(ccxt, "kucoin") + exchange = exchange_class({ + "apiKey": config["key"], + "secret": config["secret"], + "password": config["password"], + "timeout": timeout, + "enableRateLimit": True + }) + elif config["exchange"]=="okex": + exchange_class = getattr(ccxt, "okx") + exchange = exchange_class({ + "apiKey": config["key"], + "secret": config["secret"], + "password": config["password"], + "timeout": timeout, + "enableRateLimit": True + }) + elif config["exchange"]=="gateio": + exchange_class = getattr(ccxt, "gateio") + exchange = exchange_class({ + "apiKey": config["key"], + "secret": config["secret"], + "timeout": timeout, + "enableRateLimit": True + }) + elif config["exchange"]=="hitbtc": + exchange_class = getattr(ccxt, "hitbtc") + exchange = exchange_class({ + "apiKey": config["key"], + "secret": config["secret"], + "timeout": timeout, + "enableRateLimit": True + }) + elif config["exchange"]=="poloniex": + exchange_class = getattr(ccxt, "poloniex") + exchange = exchange_class({ + "apiKey": config["key"], + "secret": config["secret"], + "timeout": timeout, + "enableRateLimit": True + }) + else: + print(f"{time.strftime('[%Y/%m/%d %H:%M:%S]')} | Exchange not known or misspelled") + return None + exchange.options['createMarketBuyOrderRequiresPrice'] = False + return exchange + + +def restart_pair_no_json(base: str, quote: str) -> int: + try: + order_list = broker.fetch_full_orders(tickers) + for x in running_instances: + if f"{base}/{quote}"==x.pair: + x.pause = True + #Backing up old status file + x.write_status_file(True) + #Here, we could open a duster (if needed) + for order in order_list: + if order["symbol"]==f"{base}/{quote}" and x.is_short and order["side"]=="sell": + broker.logger.log_this(f"Cancelling old sell orders",2,f"{base}/{quote}") + broker.cancel_order(order["id"],order["symbol"]) + elif order["symbol"]==f"{base}/{quote}" and not x.is_short and order["side"]=="buy": + broker.logger.log_this(f"Cancelling old buy orders",2,f"{base}/{quote}") + broker.cancel_order(order["id"],order["symbol"]) + running_instances.remove(x) + add_instance(base,quote) + return 0 + return 1 + except Exception as e: + broker.logger.log_this(f"Exception in restart_pair_no_json: {e}",1,f"{base}/{quote}") + return 1 + + +def main_loop(): + global last_market_reload + global reload_interval + global screen_buffer + #global paused_pairs + + while True: + #Restart traders that have the restart flag raised and remove traders that have the quit flag raised + for x in running_instances: + if x.restart and x.config_dict["attempt_restart"]: + broker.logger.log_this(f"Restarting trader...",1,x.pair) + restart_pair_no_json(x.base,x.quote) + if x.quit: + #Here, check if a duster is needed + broker.logger.log_this(f"Quit flag raised, removing pair.",0,x.pair) + if f"{x.base}{x.quote}" in tickers: + tickers.remove(f"{x.base}{x.quote}") + broker.remove_pair_from_config(f"{x.base}{x.quote}") + broker.rewrite_config_file() + if x.pair in worker_status: + del(worker_status[x.pair]) + running_instances.remove(x) + + #Adds pending traders + if bool(instances_to_add): + for x in instances_to_add: + running_instances.append(x) + instances_to_add.clear() + + #Prepares the trader threads + open_orders = broker.fetch_open_orders(tickers) + pairs_to_fetch = [] + online_pairs = [] + for x in running_instances: + threads.append(Thread(target=x.check_status,args=(open_orders,))) + online_pairs.append(f"{x.base}{x.quote}") + pairs_to_fetch.append(x.pair) + + #Here, append the dusters' pairs to pairs_to_fetch, if missing. + # + # + # + # + # + + #Start the trader threads + for t in threads: + try: + t.start() + except Exception as e: + broker.logger.log_this(f"Error starting thread - {e}") + + #Wait for the trader threads to complete + for t in threads: + try: + t.join() + except Exception as e: + broker.logger.log_this(f"Error joining thread: {e}") + threads.clear() + + #Fetch prices + price_list = broker.get_prices(pairs_to_fetch) + + #Here, assign the prices to the dusters (if any) + + curr = 0 + top = 0 + for x in running_instances: + if not x.is_short: + curr += int(x.status_dict["so_amount"]) # For the safety order occupancy percentage calculation + top += int(x.config_dict["no_of_safety_orders"]) # It shows the percentage of safety orders not filled + if not x.quit: #Why? Maybe to protect return_status() from weird errors if the trader errored out? + try: + if x.pair in price_list and price_list[x.pair] is not None: + x.status_dict["price"] = price_list[x.pair] + except Exception as e: + broker.logger.log_this(f"Exception while querying for pair price, key not present on price_list dictionary: {e}",1,x.pair) + #if "status_string" in x.status_dict: + # screen_buffer.append(x.status_dict["status_string"]) + worker_status[x.pair] = x.status_dict + + #Clear the screen buffer + screen_buffer.clear() + + #Append worker data to screen buffer, shorts first. + for x in running_instances: + if x.is_short and "status_string" in x.status_dict: + screen_buffer.append(x.status_dict["status_string"]) + for x in running_instances: + if not x.is_short and "status_string" in x.status_dict: + screen_buffer.append(x.status_dict["status_string"]) + + #Updates some global status variables prior to deletion of those + global_status["online_workers"] = online_pairs.copy() + + #Check for paused pairs + global_status["paused_traders"] = [x.pair for x in running_instances if x.pause] + if global_status["paused_traders"]: + screen_buffer.append(f"{cyan}Paused pairs: {list(global_status['paused_traders'])}{white}") + + #Check for paused pairs + for x in running_instances: + if x.pause: + screen_buffer.append(f"{x.pair} paused: {x.status_dict['pause_reason']}") + + #Prints general info + instance_uptime = int(time.time()) - instance_start_time + #long_traders = len([None for x in running_instances if not x.is_short]) + #short_traders = len([None for x in running_instances if x.is_short]) + screen_buffer.append(time.strftime(f"[%Y/%m/%d %H:%M:%S] | {len(running_instances)} traders online | Instance uptime: {seconds_to_time(instance_uptime)}")) + + if top==0: #To protect from division by zero when there are no traders active + so_index=100 + else: + so_index = round(curr/top*100,2) + + color = red + if so_index<70: + color = yellow + if so_index<35: + color = green + is_testnet = "TESTNET " if broker.get_config()["is_sandbox"] else "" + screen_buffer.append(f"{bright_white}{broker.get_config()['exchange'].upper()} {is_testnet}{white}| DCAv2 {version} | CCXT v{ccxt.__version__} | Safety order occupancy: {color}{so_index}%{white}") + screen_buffer.append(blue + "="*80 + white) + + #Print screen buffer + for line in screen_buffer: + print(line) + + #Updates global status and remove keys that should not be public + global_status["instance_uptime"] = instance_uptime + global_status["config"] = broker.get_config() + for item in ["bot_chatID", "bot_token", "key", "secret", "password"]: + if item in global_status["config"]: + del(global_status["config"][item]) + + #Toggle pauses + if toggle_pauses: + for instance in running_instances: + if instance.pair in toggle_pauses: + instance.pause = not instance.pause + toggle_pauses.clear() + + #Checks if market reload is due + if time.time()-last_market_reload>reload_interval: + broker.reload_markets() + last_market_reload = time.time() + + #Take a deep breath and start all over again + time.sleep(broker.get_lap_time()) + + +def load_keys_from_db(file_name: str) -> list: + ''' + Load valid API keys + ''' + #valid_keys = [] + + database_connection = sqlite3.connect(file_name) + database_cursor = database_connection.cursor() + database_cursor.execute("SELECT * FROM credentials_table") + data = database_cursor.fetchall() + database_connection.close() + + valid_keys = [line[1] for line in data] + #for line in data: + # valid_keys.append(line[1]) + + return valid_keys + + +def display_splashscreen(): + ''' + Display splash screen + ''' + + print(""" + ###### ##### # ##### + # # # # # # # # + # # # # # # + # # # # # # # ##### + # # # ####### # # # + # # # # # # # # # + ###### ##### # # ## ####### + """) + + print(time.strftime(f"[%Y/%m/%d %H:%M:%S] | DCAv2 version {version}")) + print(time.strftime(f"[%Y/%m/%d %H:%M:%S] | Using CCXT version {ccxt.__version__}")) + + return None + + +######################### +######### API ########### +######################### + +base_api = Flask(__name__) + +@base_api.route("/global_status", methods=['GET']) +def return_global_status(): + ''' + GET request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_global_status() + return jsonify({'Error': 'API key invalid'}), 401 + +#@base_api.route("/paused_traders", methods=['GET']) +#def return_paused_status(): +# ''' +# GET request +# +# Parameters: +# None +# ''' +# if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: +# return unwrapped_paused_traders() +# return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/worker_status", methods=['GET']) +def return_worker_status(): + ''' + GET request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + base = request.args.get("base") + quote = request.args.get("quote") + return unwrapped_return_worker_status(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/view_old_long", methods=["GET"]) +def return_old_long(): + ''' + GET request + + Parameters: + base: str + quote: str + from_file: bool + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + base = request.args.get("base") + quote = request.args.get("quote") + from_file = request.args.get("from_file") + return unwrapped_view_old_long(base,quote,from_file) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/get_all_worker_status", methods=['GET']) +def return_all_worker_status(): + ''' + GET request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_return_all_worker_status() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/add_pair", methods=['POST']) +def add_pair(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_add_pair(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/remove_pair", methods=['POST']) +def remove_pair(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_remove_pair(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/restart_pair", methods=['POST']) +def restart_pair(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_restart_pair(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/import_pair", methods=['POST']) +def import_pair(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_import_pair(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/switch_to_long", methods=['POST']) +def switch_to_long(): + ''' + POST request + + Parameters: + base: str + quote: str + calculate_profits: int + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + calculate_profits = data["calculate_profits"] + return unwrapped_switch_to_long(base,quote,calculate_profits) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/switch_to_short", methods=['POST']) +def switch_to_short(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_switch_to_short(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/load_old_long", methods=['POST']) +def load_old_long(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_load_old_long(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/add_so", methods=['POST']) +def add_so(): + ''' + POST request + + Parameters: + base: str + quote: str + amount: int + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + amount = data["amount"] + return unwrapped_add_safety_orders(base,quote,amount) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/mod_tp_level", methods=['POST']) +def mod_tp_level(): + ''' + POST request + + Parameters: + base: str + quote: str + amount: float + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + amount = data["amount"] + return unwrapped_mod_tp_level(base,quote,amount) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/mod_global_tp_level", methods=['POST']) +def mod_global_tp_level(): + ''' + POST request + + Parameters: + amount: float + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + amount = data["amount"] + return unwrapped_mod_global_tp_level(amount) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/last_call", methods=['POST']) +def last_call(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_last_call(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/deferred_last_call", methods=['POST']) +def deferred_last_call(): + ''' + POST request + + Parameters: + base: str + quote: str + yyyymmdd: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + yyyymmdd = data["yyyymmdd"] + return unwrapped_deferred_last_call(base,quote,yyyymmdd) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/toggle_pause", methods=['POST']) +def toggle_pause(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_toggle_pause(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/global_last_call", methods=['POST']) +def global_last_call(): + ''' + POST request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_global_last_call() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/add_quote", methods=['POST']) +def add_quote(): + ''' + POST request + + Parameters: + base: str + quote: str + amount: float + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + amount = data["amount"] + return unwrapped_add_quote(base,quote,amount) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/missing_pairs", methods=['GET']) +def missing_pairs(): + ''' + GET request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_missing_pairs() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/toggle_cleanup", methods=['POST']) +def toggle_cleanup(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_toggle_cleanup(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/toggle_autoswitch", methods=['POST']) #type:ignore +def toggle_autoswitch(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_toggle_autoswitch(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/toggle_check_old_long_price", methods=['POST'])#type:ignore +def toggle_check_old_long_price(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_toggle_check_old_long_price(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/switch_quote_currency", methods=['POST']) +def switch_quote_currency(): + ''' + POST request + + Parameters: + base: str + quote: str + new_quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + new_quote = data["new_quote"] + return unwrapped_switch_quote_currency(base,quote,new_quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/toggle_restart", methods=['POST']) +def toggle_restart(): + ''' + POST request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_toggle_restart() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/toggle_telegram", methods=['POST']) +def toggle_telegram(): + ''' + POST request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_toggle_telegram() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/server_time", methods=['GET']) +def server_time(): + ''' + GET request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_server_time() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/trader_time", methods=['GET']) +def trader_time(): + ''' + GET request + + Parameters: + None + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_trader_time() + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/edit_loop_wait_time", methods=['POST']) +def loop_wait_time(): + ''' + POST request + + Parameters: + wait_time: float + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + wait_time = data["wait_time"] + return unwrapped_loop_wait_time(wait_time) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/edit_call_wait_time", methods=['POST']) +def call_wait_time(): + ''' + POST request + + Parameters: + wait_time: float + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + wait_time = data["wait_time"] + return unwrapped_call_wait_time(wait_time) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/reload_markets", methods=['POST']) +def reload_markets(): + ''' + POST request + + Parameters: + base: str + quote: str + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + return unwrapped_reload_markets() + + return jsonify({'Error': 'API key invalid'}), 401 + +@base_api.route("/reload_safety_order", methods=['POST']) +def reload_safety_order(): + ''' + POST request + + Parameters: + None + + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + if request.json is None: + return jsonify({'Error': 'request.json is None'}) + data = request.json + base = data["base"] + quote = data["quote"] + return unwrapped_reload_safety_order(base,quote) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + + +def run_API(): + base_api.run(host="0.0.0.0", port=broker.get_config()["port"]) + + + +########################### +# Unwrapped API functions # +########################### + +def unwrapped_global_status(): + return jsonify(global_status) + +def unwrapped_return_worker_status(base,quote): + if f"{base}/{quote}" in worker_status: + return jsonify(worker_status[f"{base}/{quote}"]) + return jsonify({"Error": "Worker does not exist"}) + +def unwrapped_return_all_worker_status(): + return jsonify(worker_status) + +def unwrapped_add_pair(base,quote): + try: + #Check if the trader is already running + for x in running_instances: + if f"{base}/{quote}"==x.pair: + broker.logger.log_this(f"Pair already running",1,f"{base}/{quote}") + return jsonify({"Error": "Pair already running"}) + + #Check if the market exists and it's open + markets = broker.exchange.load_markets() + if f"{base}/{quote}" not in markets: + broker.logger.log_this(f"Market does not exist",1,f"{base}/{quote}") + return jsonify({"Error": "Market does not exist"}) + if not markets[f"{base}/{quote}"]["active"]: + broker.logger.log_this(f"Market is inactive",1,f"{base}/{quote}") + return jsonify({"Error": "Market is inactive"}) + + broker.logger.log_this(f"Initializing trader",2,f"{base}/{quote}") + add_instance(base,quote) + + broker.add_pair_to_config(f"{base}{quote}") + + #Adding the pair to the config file (if it's not there yet) + #1. Read the config file + with open(sys.argv[1],"r") as f: + temp_details = json.load(f) + if base+quote not in temp_details["pairs"]: + #2. Save the current config file as .bak + with open(f"{sys.argv[1]}.bak","w") as c: + c.write(json.dumps(temp_details, indent=4)) + #3. Add the pair to the right list + temp_details["pairs"].append(f"{base}{quote}") + #4. Write the config file + with open(sys.argv[1],"w") as c: + c.write(json.dumps(temp_details, indent=4)) + broker.logger.log_this(f"Broker's config file updated",2,f"{base}/{quote}") + else: + broker.logger.log_this(f"Pair already included in the config file",2,f"{base}/{quote}") + return jsonify({"Success": "Pair added"}) + except Exception as e: + broker.logger.log_this(f"Exception while initializing new instance: {e}",1,f"{base}/{quote}") + return jsonify({"Error": "Error initializing new instance."}) + +def unwrapped_remove_pair(base,quote): + try: + for x in running_instances: + if f"{base}/{quote}"==x.pair: + x.quit = True + return jsonify({"Success": "Pair to be removed"}) + except Exception as e: + broker.logger.log_this(f"Exception while removing instance: {e}",1,f"{base}/{quote}") + return jsonify({"Error": "Halp"}) + +def unwrapped_restart_pair(base,quote): + if restart_pair_no_json(base,quote)==0: + return jsonify({"Success": "Pair restarted"}) + return jsonify({"Error": "Halp"}) + +def unwrapped_import_pair(base,quote): + try: + import_instance(base+quote) + broker.add_pair_to_config(f"{base}{quote}") + broker.rewrite_config_file() + broker.logger.log_this(f"Done",2,f"{base}/{quote}") + return jsonify({"Success": "Pair imported successfully"}) + except Exception as e: + broker.logger.log_this(f"Exception while importing instance: {e}",1,f"{base}/{quote}") + return jsonify({"Error": "Error importing instance"}) + +def unwrapped_switch_to_long(base,quote,calculate_profits): + ''' + Switches a pair to long mode. + If calculate_profits is 0, it does not calculate the profits + ''' + ignore_old_long = int(calculate_profits)!=1 + #Close trader and orders and pull info our of the orders + if f"{base}{quote}" not in broker.get_pairs(): + return jsonify({"Error": "Pair not running"}) + for x in running_instances: + if f"{base}/{quote}"==x.pair: + x.pause = True + if x.switch_to_long(ignore_old_long=ignore_old_long)==1: + return jsonify({"Error": "Error in switch_to_long()"}) + if x.start_bot()==1: + x.quit = True + return jsonify({"Error": "Error switching to long mode (wAPI)"}) + return jsonify({"Success": "Pair switched to long mode"}) + return jsonify({"Error": "Pair not found"}) + +def unwrapped_switch_to_short(base,quote): + ''' + Switches a pair to short mode. + ''' + #Close trader and orders and pull info our of the orders + if f"{base}{quote}" not in broker.get_pairs(): + return jsonify({"Error": "Pair not running"}) + for x in running_instances: + if f"{base}/{quote}"==x.pair and x.switch_to_short()==1: + return jsonify({"Error": "Error in switch_to_short()"}) + + #Restart instance + try: + broker.logger.log_this(f"Reinitializing trader",2,f"{base}/{quote}") + for x in running_instances: + if f"{base}/{quote}"==x.pair: + #Clearing some variables + #x.fees_paid_in_quote = 0 + #x.fees_paid_in_base = 0 + x.tp_order = x.broker.empty_order + x.so = x.broker.empty_order + #x.safety_price_table.clear() #This clearing (probably) is the origin of the update_status error after switching to short. + #x.safety_order_index = 0 + + #Reloading config file + x.config_dict = x.reload_config_dict() + + #Enabling autoswitch + x.config_dict["autoswitch"] = True + if x.start_bot()==1: + x.quit = True + return jsonify({"Error": "Error switching to short mode (wAPI)"}) + return jsonify({"Success": "Pair switched to short mode"}) + except Exception as e: + broker.logger.log_this(f"Exception while reinitializing instance: {e}",1,f"{base}/{quote}") + return jsonify({"Error": "Can't initialize trader"}) + return jsonify({"Error": "Halp"}) + +def unwrapped_load_old_long(base,quote): + ''' + Loads an old_long file to the status dictionary of a trader + ''' + #Load the file + try: + with open(f"{base}{quote}.oldlong") as ol: + old_long = json.load(ol) + except Exception as e: + broker.logger.log_this(f"Exception while loading old_long file: {e}",1,f"{base}/{quote}") + return jsonify({"Error": "old_long file of that pair does not exist."}) + + #Check that the file has the proper keys + if not any(["tp_price" in old_long, "tp_amount" in old_long, "quote_spent" in old_long, "datetime" in old_long]): + broker.logger.log_this(f"old_long file invalid: keys missing.",1,f"{base}/{quote}") + return jsonify({"Error": "File is invalid or missing keys"}) + + #Maybe here we could also check that the keys have the proper values + + #Creates (or modifies) a key in the status dictionary and assigns the contents of the file to that same key. + for x in running_instances: + if x.pair==f"{base}/{quote}": + x.status_dict["old_long"]=old_long + x.update_status(True) + return jsonify({"Success": "old_long file loaded to status_dict"}) + return jsonify({"Error": "Pair not found"}) + +def unwrapped_view_old_long(base,quote,from_file): + ''' + Returns the content of an old_long file + ''' + try: + if int(from_file)==1: + with open(f"{base}{quote}.oldlong") as ol: + old_long = json.load(ol) + return jsonify(old_long) + for x in running_instances: + if f"{base}/{quote}"==x.pair: + return jsonify(x.status_dict["old_long"]) + return jsonify({"Error": "Pair not found"}) + except Exception as e: + broker.logger.log_this(f"Exception while viewing old_long info: {e}",1,f"{base}/{quote}") + return jsonify({"Error": f"{e}"}) + +def unwrapped_add_safety_orders(base,quote,amount): + ''' + Increases the amount of safety orders that a trader can use. Once the current deal is closed, the value returns to the one in the config + file. + ''' + try: + for x in running_instances: + if f"{base}/{quote}"==x.pair: + x.pause = True + #x.no_of_safety_orders += int(amount) + x.config_dict["no_of_safety_orders"]+=int(amount) + broker.logger.log_this("Recalculating safety price table...",1,f"{base}/{quote}") + x.safety_price_table = x.calculate_safety_prices(x.start_price,x.config_dict["no_of_safety_orders"],x.config_dict["safety_order_deviance"]) + broker.logger.log_this(f"Done. Added {amount} safety orders",1,f"{base}/{quote}") + x.update_status(True) + x.pause = False + return jsonify({"Success": f"Done. Added {amount} safety orders"}) + return jsonify({"Error": "Pair not found"}) + except Exception as e: + broker.logger.log_this(f"{e}",2,f"{base}/{quote}") + return jsonify({"Error": "Error adding safety orders"}) + +def unwrapped_mod_tp_level(base,quote,amount): + ''' + Modifies the take profit percentage of a pair. It applies the new percentage only after a new TP order is sent. + ''' + try: + for x in running_instances: + if f"{base}/{quote}"==x.pair: + x.config_dict["tp_level"]=float(amount) + broker.logger.log_this("Done. The change will take effect when the next take profit order is placed",2,f"{base}/{quote}") + return jsonify({"Success": "Success. The change will take effect when the next TP order is placed"}) + except Exception: + broker.logger.log_this("Error changing percentage. Ignoring...",2,f"{base}/{quote}") + return jsonify({"Error": "Error changing percentage"}) + +def unwrapped_mod_global_tp_level(amount): + ''' + Modifies the take profit percentage of all pairs. It applies the new percentage only after a new TP order is sent. + ''' + for x in running_instances: + try: + x.config_dict["tp_level"]=float(amount) + broker.logger.log_this("Done. The change will take effect when the next take profit order is placed",2) + except Exception: + broker.logger.log_this("Error changing percentage. Ignoring.",2) + return jsonify({"Success": "Success. The change will take effect when the next TP order is placed"}) + +def unwrapped_last_call(base,quote): + ''' + Signals the trader to stop opening new deals once the current one is closed. + ''' + try: + if f"{base}{quote}" in broker.get_pairs(): + #read_config["pairs"].remove(base+quote) + for x in running_instances: + if f"{base}/{quote}"==x.pair: + x.stop_when_profit = not x.stop_when_profit + x.update_status(True) + if x.stop_when_profit: + return jsonify({"Success": "Trader scheduled to go offline when profit is reached"}) + return jsonify({"Success": "Last call cancelled"}) + return jsonify({"Error": "Trader does not exist"}) + except Exception: + return jsonify({"Error": "Halp"}) + +def unwrapped_deferred_last_call(base,quote,yyyymmdd): + ''' + Programs the trader to not open new deals from a certain future date. Like a VCR. + ''' + try: + year = yyyymmdd[:4] + month = yyyymmdd[4:6] + day = yyyymmdd[6:8] + limit = time_to_unix(year,month,day) + if limit==0: + return jsonify({"Error": "Can't convert date to unix"}) + for x in running_instances: + if f"{base}{quote}"==x.pair: + x.config_dict["stop_time"] = limit + #save config file to disk + x.broker.rewrite_config_file() + return jsonify({"Success": f"Trader scheduled to go offline when profit is reached after {limit}"}) + except Exception: + return jsonify({"Error": "Halp"}) + +def unwrapped_toggle_pause(base,quote): + ''' + Toggles the pause flag of a trader. + When a trader is paused, no new safety orders are sent to the exchange and the take profit order is unmonitored. + Although it could be useful to close the trader if the tp order is filled anyway. + ''' + try: + toggle_pauses.append(f"{base}/{quote}") + for instance in running_instances: + if instance.pair==f"{base}/{quote}": + if instance.pause: + return jsonify({"Success": "Trader will be resumed"}) + return jsonify({"Success": "Trader will be paused"}) + return jsonify({"Error": "Trader does not exist"}) + except Exception: + return jsonify({"Error": "Halp"}) + +def unwrapped_global_last_call(): + ''' + Signals all traders to stop opening new trades when the current ones closes. + ''' + try: + if broker.get_pairs!=[]: + broker.clear_pairs() + for x in running_instances: + x.stop_when_profit = True + broker.logger.log_this("Modified flag",2,f"{x.base}/{x.quote}") + return jsonify({"Success": "All traders scheduled to go offline when profit is reached"}) + else: + for x in running_instances: + x.stop_when_profit = False + broker.logger.log_this("Modified flag",2,f"{x.base}/{x.quote}") + broker.add_pair_to_config(f"{x.base}{x.quote}") + return jsonify({"Success": "Last call canceled"}) + except Exception: + return jsonify({"Error": "Halp"}) + +def unwrapped_add_quote(base,quote,amount): + ''' + Adds more funds to the deal, bringing down the average buy price in the meantime. + I do not recommend to use this, it's preferable to switch to short mode, but you do you. + Maybe it's more useful in a bull market's high volatility moment. + ''' + for x in running_instances: + if f"{base}/{quote}"==x.pair: + if x.is_short: + return jsonify({"Error": "Quote can't be added to short bots"}) + x.pause = True + new_average_price = (x.total_amount_of_quote+float(amount))/(x.total_amount_of_base+(float(amount)/x.status_dict["price"])) + broker.logger.log_this(f"Your new average buy price will be {new_average_price} {x.quote}",2,f"{base}/{quote}") + broker.logger.log_this(f"Your new take profit price price will be {new_average_price*x.get_tp_level()} {x.quote}",2,f"{base}/{quote}") + new_order = broker.new_market_order(x.pair,float(amount),"buy") + if new_order is None: + broker.logger.log_this("Error: Market order returned None",2,f"{base}/{quote}") + x.pause = False + return jsonify({"Error": "Market order returned None"}) + while True: + time.sleep(broker.get_wait_time()) + returned_order = broker.get_order(new_order["id"],x.pair) + if returned_order==broker.empty_order: + broker.logger.log_this("Problems sending the order",2,f"{base}/{quote}") + x.pause = False + return jsonify({"Error": "Problems sending the order"}) + elif returned_order["status"]=="expired": + x.pause = False + return jsonify({"Error": "New order expired"}) + elif returned_order["status"]=="closed": + broker.logger.log_this("Order sent",2,f"{base}/{quote}") + new_fees_in_base, new_fees_in_quote = x.parse_fees(returned_order) + x.fees_paid_in_base += new_fees_in_base + x.fees_paid_in_quote += new_fees_in_quote + x.total_amount_of_base = x.total_amount_of_base + returned_order["filled"] - new_fees_in_base + x.total_amount_of_quote += returned_order["cost"] + broker.logger.log_this("Cancelling old take profit order and sending a new one",2,f"{base}/{quote}") + attempts = 5 + while broker.cancel_order(x.tp_order["id"],x.pair)==1: + broker.logger.log_this("Can't cancel old take profit order, retrying...",2,f"{base}/{quote}") + time.sleep(broker.get_wait_time()) + attempts-=1 + if attempts==0: + broker.logger.log_this("Can't cancel old take profit order, cancelling...",2,f"{base}/{quote}") + x.pause = False + return jsonify({"Error": "Can't cancel old take profit order."}) + x.take_profit_price = x.total_amount_of_quote/x.total_amount_of_base*x.get_tp_level() + x.tp_order = broker.new_limit_order(x.pair,x.total_amount_of_base,"sell",x.take_profit_price) + x.update_status(True) + break + else: + broker.logger.log_this("Waiting for initial order to get filled",2,f"{base}/{quote}") + broker.logger.log_this(f"{returned_order}",2,f"{base}/{quote}") + time.sleep(broker.get_wait_time()) + x.pause = False + broker.logger.log_this("Done",2,f"{base}/{quote}") + return jsonify({"Success": "Quote added successfully"}) + return jsonify({"Error": "Something horrible happened :S"}) + +def unwrapped_missing_pairs(): + ''' + Returns a list of the pairs that are not running that are included in the config file + ''' + try: + missing_pairs = broker.get_pairs() + for trader in running_instances: + if f"{trader.base}{trader.quote}" in missing_pairs: + missing_pairs.remove(f"{trader.base}{trader.quote}") + return jsonify({"Success": missing_pairs}) + except Exception as e: + broker.logger.log_this(f"Exception while querying for missing pairs: {e}",1) + return jsonify({"Error": "Error fetching running pairs"}) + +#def unwrapped_paused_traders(): +# ''' +# Returns a list of paused pairs +# ''' +# return jsonify({"paused_traders":global_status["paused_traders"]}) + +def unwrapped_toggle_cleanup(base,quote): + try: + pair_to_toggle = f"{base}/{quote}" + for x in running_instances: + if pair_to_toggle==x.pair: + x.config_dict["cleanup"] = not x.config_dict["cleanup"] + if x.config_dict["cleanup"]: + return jsonify({"Success": "Cleanup turned ON"}) + return jsonify({"Success": "Cleanup turned OFF"}) + except Exception as e: + broker.logger.log_this(f"Exception while toggling cleanup: {e}",1,f"{base}{quote}") + return jsonify({"Error": "Halp"}) + return jsonify({"Error": "Task succesfully failed"}) + +def unwrapped_toggle_autoswitch(base,quote): + try: + pair_to_toggle = f"{base}/{quote}" + for x in running_instances: + if pair_to_toggle==x.pair: + if x.config_dict["autoswitch"]: + broker.logger.log_this("Autoswitch turned OFF",1,f"{base}/{quote}") + x.config_dict["autoswitch"] = False + return jsonify({"Success": "Autoswitch it's now OFF"}) + else: + broker.logger.log_this("Autoswitch turned ON",1,f"{base}/{quote}") + x.config_dict["autoswitch"] = True + return jsonify({"Success": "Autoswitch it's now ON"}) + except Exception as e: + broker.logger.log_this(f"Exception while toggling autoswitch: {e}",1,f"{base}{quote}") + return jsonify({"Error": "Halp"}) + +def unwrapped_toggle_check_old_long_price(base,quote): + try: + pair_to_toggle = f"{base}/{quote}" + for x in running_instances: + if pair_to_toggle==x.pair: + if x.config_dict["check_old_long_price"]: + broker.logger.log_this("Check OFF",1,f"{base}/{quote}") + x.config_dict["check_old_long_price"] = False + return jsonify({"Success": "Old long price check turned OFF"}) + else: + broker.logger.log_this("Check ON",1,f"{base}/{quote}") + x.config_dict["check_old_long_price"] = True + return jsonify({"Success": "Old long price check turned ON"}) + except Exception as e: + broker.logger.log_this(f"Exception while toggling check_old_long_price: {e}",1,f"{base}{quote}") + return jsonify({"Error": "Halp"}) + +def unwrapped_switch_quote_currency(base,quote,new_quote): + try: + pair_to_switch = f"{base}/{quote}" + for trader in running_instances: + if pair_to_switch==trader.pair: + #Pause the trader + trader.pause = True + + #Call x.switch_quote_currency + if trader.switch_quote_currency(new_quote)==1: + return jsonify({"Error": "Swap failed. Check log files for details."}) + + #Resume the trader + trader.pause = False + return jsonify({"Success": "Mission successful"}) + return jsonify({"Error": "Trader not found"}) + except Exception as e: + broker.logger.log_this(f"Exception while switching quote currency: {e}",1,f"{base}{quote}") + return jsonify({"Error": "Halp"}) + +def unwrapped_toggle_restart(): + new_config = broker.get_config() + new_config["attempt_to_restart"] = not new_config["attempt_to_restart"] + broker.set_config(new_config) + return jsonify({"Success": "attempt_to_restart toggled successfully"}) + +def unwrapped_toggle_telegram(): + new_config = broker.get_config() + new_config["telegram"] = not new_config["telegram"] + broker.set_config(new_config) + broker.logger.set_telegram_notifications(new_config["telegram"]) + toggle = "ON" if new_config["telegram"] else "OFF" + return jsonify({"Success": f"Telegram successfully toggled {toggle}"}) + +def unwrapped_server_time(): + return jsonify({"Time": time.time()}) + +def unwrapped_trader_time(): + try: + return jsonify({"Time": max(x.last_time_seen for x in running_instances)}) + except Exception as e: + broker.logger.log_this(f"Exception while retrieving trader_time: {e}",1) + return jsonify({"Error": str(e)}) + +def unwrapped_loop_wait_time(wait_time): + broker.set_lap_time(wait_time) + broker.logger.log_this("Done!") + return jsonify({"Success": "Lap time modified successfully"}) + +def unwrapped_call_wait_time(wait_time): + ''' + Modifies the time between some API calls and retries. + ''' + broker.set_wait_time(wait_time) + broker.logger.log_this("Done!") + return jsonify({"Success": "Call wait time modified successfully"}) + +def unwrapped_reload_markets(): + try: + broker.reload_markets() + return jsonify({"Success": "Markets reloaded successfully"}) + except Exception as e: + broker.logger.log_this(f"Exception while reloading markets: {e}",1) + return jsonify({"Error": "Markets couldn't be reloaded"}) + +def unwrapped_reload_safety_order(base,quote): + try: + for trader in running_instances: + if trader.pair==f"{base}/{quote}": + trader.reload_safety_order() + return jsonify({"Success": "Safety order reloaded successfully"}) + except Exception as e: + broker.logger.log_this(f"Exception while reloading safety order: {e}",1) + return jsonify({"Error": "Safety order couldn't be reloaded"}) + + +if __name__=="__main__": + + #Logo + display_splashscreen() + + #Setup some variables + instance_start_time = int(time.time()) + last_market_reload = time.time() + reload_interval = 3600 #Market reload interval (in seconds) + + #Loading config file + print(time.strftime("[%Y/%m/%d %H:%M:%S] | Loading config file...")) + try: + with open(sys.argv[1]) as f: + read_config = json.load(f) + except Exception as e: + print(e) + print("Wrong syntax. Correct syntax is 'python3 dcaruntime.py xxxxx.json', xxxxx.json being the config file.") + os._exit(1) + + #Check for import or load + import_mode = True + if "--first_start" in sys.argv: + import_mode = False + print(time.strftime("[%Y/%m/%d %H:%M:%S] | Initializing in FIRST START MODE, press enter to start...")) + else: + print(time.strftime("[%Y/%m/%d %H:%M:%S] | Initializing in IMPORT MODE, press enter to start...")) + input() + + #Load exchange config + exchange = set_exchange(read_config) + if exchange is None: + print("Error initializing exchange. Check spelling and/or the exchange configuration file.") + os._exit(1) + + #Creating the broker object + print(time.strftime(f"[%Y/%m/%d %H:%M:%S] | Connecting to {str(exchange)}...")) + broker = exchange_wrapper.broker(exchange,read_config,sys.argv[1]) #Also passes the config filename + + #Declaring some variables + running_instances = [] + open_orders = [] + instances_to_add = [] + online_pairs = [] + toggle_pauses = [] + tickers = [] + threads = [] + screen_buffer = [] + worker_status = {} + global_status = { + "name": broker.get_config()["exchange"].upper(), + "instance_uptime": 0, + "online_workers": [], + "paused_traders": [], + "version": version, + "ccxt_version": f"{ccxt.__version__}", + "config": broker.get_config() + } + + #Remove some keys that should not be public + for item in ["bot_chatID", "bot_token", "key", "secret", "password"]: + global_status["config"].pop(item,None) + + #Load valid API keys + valid_keys = load_keys_from_db("utils/api_credentials.db") + + #Initialize instances + if not import_mode: + toggle = input(f"This will initialize {len(broker.get_pairs())} instances, proceed? (Y/n) ") + if toggle not in ["Y","y",""]: + broker.logger.log_this("Aborting initialization",2) + os._exit(1) + #broker.logger.log_this(f"Initializing {len(broker.get_pairs())} instances",2) + for x in broker.get_pairs(): + initialize_instance(x) + else: + toggle = input(f"This will import {len(broker.get_pairs())} instances, proceed? (Y/n) ") + if toggle not in ["Y","y",""]: + broker.logger.log_this("Aborting import",2) + os._exit(1) + #broker.logger.log_this(f"Importing {len(broker.get_pairs())} instances",2) + for x in broker.get_pairs(): + import_instance(x) + broker.logger.log_this(f"All instances imported!",2) + + #Threads to run: main loop and flask api + main_threads = [Thread(target=main_loop,args=()),Thread(target=run_API,args=())] + + #Iterate indefinitely: + for m in main_threads: + m.start() + + #As always... fuck you 3commas! diff --git a/profits/db_read.py b/profits/db_read.py new file mode 100644 index 0000000..fd863c1 --- /dev/null +++ b/profits/db_read.py @@ -0,0 +1,25 @@ +import sqlite3 +import time +import json + +# Connect to the SQLite database +conn = sqlite3.connect('profits/profits_database.db') +cursor = conn.cursor() + +# Execute a SELECT query to retrieve data from the database +cursor.execute('SELECT * FROM profits_table') + +# Fetch all rows from the result set +rows = cursor.fetchall() + +# Process the fetched rows +#for row in rows: + #human_time = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(int(row[0]))) + #print(human_time,row[1],round(row[2],2),row[3]) # Or do whatever processing you need + +data = json.loads(rows[-3][-1]) +print(data[-1]) +print(f"Number of entries: {len(rows)}") + +# Close the connection +conn.close() \ No newline at end of file diff --git a/profits/dump_old_profits_to_db.py b/profits/dump_old_profits_to_db.py new file mode 100644 index 0000000..15ecc88 --- /dev/null +++ b/profits/dump_old_profits_to_db.py @@ -0,0 +1,77 @@ +''' +Format to dump: (pair,timestamp,profit,exchange_name,order_id,order_history) +''' + + +import os +from datetime import datetime +import json +import sqlite3 +import sys + + +#Make a dictionary with the format "pair":"exchange" instead of this garbage +if sys.argv[1]=="--testnet": + exchanges = ["binance_testnet"] +else: + exchanges = ["binance","gateio","kucoin","okex"] +recognized_pairs = {} +profits_path = "../profits/" +configs_path = "../configs/" +results = [] +dates_already_present = [] +order_id_list = [] +amount_of_files_to_process = 0 + +#Load pair information from the exchange(s) configuration file(s) +for exchange_file in exchanges: + with open(f"{configs_path}{exchange_file}.json") as f: + coins_in_exchange_config = json.load(f)["pairs"] + for coin in coins_in_exchange_config: + recognized_pairs[coin] = exchange_file + +#Connect to db +connection = sqlite3.connect('profits_database.db') +cursor = connection.cursor() + +#Extract order_id data from db +cursor.execute('SELECT * FROM profits_table') +rows = cursor.fetchall() +for row in rows: + order_id_list.append(row[4]) + + +for archivo in os.listdir(profits_path): + if archivo.endswith(".profits"): + amount_of_files_to_process+=1 + +#Format data +count = 1 +for archivo in os.listdir(profits_path): + if archivo.endswith(".profits"): + coin = archivo.split(".")[0] + coin = f"{coin[-50:-4]}/{coin[-4:]}" + print(f"Processing {count} out of {amount_of_files_to_process} files ({coin})") + with open(f"{profits_path}{archivo}", "r") as csvfile: + for x in csvfile: + date,amount,order_id = x.split(",") + unix_time = int(datetime.strptime(date,"%Y-%m-%d").timestamp()) + while unix_time in dates_already_present: + unix_time+=1 + dates_already_present.append(unix_time) + exchange = recognized_pairs[coin] if coin in recognized_pairs else "unknown" + stripped_order_id = order_id.strip() + + complete_line = (unix_time,coin,float(amount),exchange,stripped_order_id,"") + if stripped_order_id not in order_id_list: + results.append(complete_line) + count+=1 + + +#Dump data to db +#for row in results: +# cursor.execute('INSERT INTO profits_table VALUES(?, ?, ?, ?, ?, ?)', row) +#connection.commit() +#connection.close() + +print(f"Done. Added {len(results)} rows") \ No newline at end of file diff --git a/profits/last_n_deals.py b/profits/last_n_deals.py new file mode 100644 index 0000000..98a82d7 --- /dev/null +++ b/profits/last_n_deals.py @@ -0,0 +1,21 @@ +import sys +import time +import sqlite3 + +try: + amount_of_deals = sys.argv[1] +except Exception as e: + print(e) + print("Usage: python3 last_n_deals.py int") + sys.exit() + +connection = sqlite3.connect('profits_database.db') +cursor = connection.cursor() +cursor.execute(f'SELECT * FROM profits_table ORDER BY timestamp DESC LIMIT {amount_of_deals}') +deals = cursor.fetchall() +connection.close() + + +for line in deals[::-1]: + human_time = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(int(line[0]))) + print(human_time,line[1],round(line[2],2),line[3]) \ No newline at end of file diff --git a/profits/monthly_details.py b/profits/monthly_details.py new file mode 100644 index 0000000..5334d68 --- /dev/null +++ b/profits/monthly_details.py @@ -0,0 +1,52 @@ +''' +Usage: python3 monthly_details.py YYYY-MM + +Generates a csv file per exchange with details of profit per pair in the format "pair,profit" +''' + +import sqlite3 +import csv +import sys + +try: + month = sys.argv[1] +except Exception as e: + print(e) + print("Usage: python3 monthly_details.py YYYY-MM") + sys.exit() + +result = {"binance":[],"gateio":[],"kucoin":[],"okex":[]} + +#Connect to db +connection = sqlite3.connect("profits_database.db") +cursor = connection.cursor() + +query = """SELECT exchange_name, pair, strftime('%Y-%m', datetime(timestamp, 'unixepoch')) AS month, SUM(amount) AS total_profit +FROM profits_table +GROUP BY exchange_name, pair, month;""" +print("Querying the database") +cursor.execute(query) +by_exchange = cursor.fetchall() + +to_binance = [] +to_gateio = [] +to_kucoin = [] +to_okex = [] + +print("Generating reports") +for item in by_exchange: + if item[0] in result and item[2]==month: + result[item[0]].append((item[1],item[3])) + +#Descending order +for item in result.copy(): + result[item].sort(key=lambda tup: tup[1], reverse=True) + +print("Writing reports to disk") +for item in result: + with open(f"{item}_{month}_report.csv","w",newline="") as csv_file: + csv_writer = csv.writer(csv_file) + csv_writer.writerows(result[item]) + +print("done") + diff --git a/profits/monthlypair.py b/profits/monthlypair.py new file mode 100755 index 0000000..214f994 --- /dev/null +++ b/profits/monthlypair.py @@ -0,0 +1,44 @@ +import os, sys, json + +total = 0 + +try: + month = sys.argv[1] + year = sys.argv[2] +except: + print ("Wrong syntax. 'python3 monthlypair.py MM YY'") + sys.exit() + +with open(year + month+".csv","w",newline="") as f: + for archivo in os.listdir(): + if archivo.endswith(".profits"): + with open(archivo, "r") as csvfile: + for x in csvfile: + date,amount,_ = x.split(",") + if date.split("-")[1]==month and date.split("-")[0]==year: + total += float(amount) + #result.append([archivo[:-8],total]) + if total != 0: + print(archivo[:-8]+","+str(total)) + f.write(archivo[:-8]+","+str(total)+"\n") + total = 0 + +# Per-dollar section: + +with open(year+month+"_per_dollar.csv","w",newline="") as f: + for archivo in os.listdir(): + if archivo.endswith(".profits"): + with open(archivo, "r") as csvfile: + for x in csvfile: + date,amount,_ = x.split(",") + if date.split("-")[1]==month and date.split("-")[0]==year: + total += float(amount) + #result.append([archivo[:-8],total]) + if total != 0: + #Here load the order size from the config file + file_name = archivo.split(".")[0] + with open("/home/nsanc/DCA2_live/configs/"+file_name+".json") as g: + read_config = json.load(g) + print(archivo[:-8]+","+str(round(total/read_config["order_size"],2))) + f.write(archivo[:-8]+","+str(round(total/read_config["order_size"],2))+"\n") + total = 0 diff --git a/profits/per_exchange.py b/profits/per_exchange.py new file mode 100755 index 0000000..0f6349a --- /dev/null +++ b/profits/per_exchange.py @@ -0,0 +1,102 @@ +import os, sys, json + +total = 0 + +try: + month = sys.argv[1] + year = sys.argv[2] +except Exception as e: + print ("Wrong syntax. 'python3 per_exchange.py MM YY'") + print(e) + sys.exit() + +binance_filename = f"binance{year}{month}.csv" +kucoin_filename = f"kucoin{year}{month}.csv" +okex_filename = f"okex{year}{month}.csv" +gateio_filename = f"gateio{year}{month}.csv" + +with open(binance_filename,"w") as binance: + pass +with open(kucoin_filename,"w") as binance: + pass +with open(okex_filename,"w") as binance: + pass +with open(gateio_filename,"w") as binance: + pass + +#hitbtc_filename = f"hitbtc{year}{month}.csv" +#poloniex_filename = f"poloniex{year}{month}.csv" + +with open("../configs/binance.json") as r: + binance_config = json.load(r) +with open("../configs/kucoin.json") as r: + kucoin_config = json.load(r) +with open("../configs/okex.json") as r: + okex_config = json.load(r) +with open("../configs/gateio.json") as r: + gateio_config = json.load(r) +#with open("../configs/hitbtc.json") as r: +# hitbtc_config = json.load(r) +#with open("../configs/poloniex.json") as r: +# poloniex_config = json.load(r) + + +binance_sum = 0 +kucoin_sum = 0 +gateio_sum = 0 +okex_sum = 0 + +for archivo in os.listdir(): + if archivo.endswith(".profits"): + with open(archivo, "r") as csvfile: + for x in csvfile: + date,amount,_ = x.split(",") + if date[:4]==year and date[5:7]==month: + total += float(amount) + pair = archivo.split(".")[0] + if total!=0: + if pair in binance_config["pairs"]: + binance_sum+=total + with open(binance_filename,"a") as binance: + binance.write(f"{pair},{round(total,2)}\n") + elif pair in kucoin_config["pairs"]: + with open(kucoin_filename,"a") as kucoin: + kucoin.write(f"{pair},{round(total,2)}\n") + kucoin_sum+=total + elif pair in okex_config["pairs"]: + with open(okex_filename,"a") as okex: + okex.write(f"{pair},{round(total,2)}\n") + okex_sum+=total + elif pair in gateio_config["pairs"]: + with open(gateio_filename,"a") as gateio: + gateio.write(f"{pair},{round(total,2)}\n") + gateio_sum+=total + total = 0 + +#print("Totals:") +total_sum = binance_sum+gateio_sum+kucoin_sum+okex_sum +print("===========================") +print(f"Binance: {round(binance_sum,2)} USDT ({round(binance_sum/total_sum*100,2)}%)") +print(f"Gate.io: {round(gateio_sum,2)} USDT ({round(gateio_sum/total_sum*100,2)}%)") +print(f"KuCoin: {round(kucoin_sum,2)} USDT ({round(kucoin_sum/total_sum*100,2)}%)") +print(f"OKX: {round(okex_sum,2)} USDT ({round(okex_sum/total_sum*100,2)}%)") +print("===========================") + +try: + if sys.argv[3]=="--withdrawals": + print("With Gate.io:") + total_to_withdraw = 1200 + print(f"Binance: {int(total_to_withdraw*binance_sum/total_sum)} USDT") + print(f"Gate.io: {int(total_to_withdraw*gateio_sum/total_sum)} USDT") + print(f"KuCoin: {int(total_to_withdraw*kucoin_sum/total_sum)} USDT") + print(f"OKX: {int(total_to_withdraw*okex_sum/total_sum)} USDT") + print("===========================") + print("Without Gate.io:") + partial_sum = binance_sum+kucoin_sum+okex_sum + total_to_withdraw = 1200 + print(f"Binance: {int(total_to_withdraw*binance_sum/partial_sum)} USDT") + print(f"KuCoin: {int(total_to_withdraw*kucoin_sum/partial_sum)} USDT") + print(f"OKX: {int(total_to_withdraw*okex_sum/partial_sum)} USDT") + print("===========================") +except Exception as e: + pass #prograem \ No newline at end of file diff --git a/profits/per_exchange_report.py b/profits/per_exchange_report.py new file mode 100644 index 0000000..8df766a --- /dev/null +++ b/profits/per_exchange_report.py @@ -0,0 +1,89 @@ +''' +Display last three months of profits grouped by exchange +''' + +import sqlite3 +import datetime + +linewidth = 40 +exchanges = ["binance","gateio","kucoin","okex"] + +#Connect to db +connection = sqlite3.connect("profits_database.db") +cursor = connection.cursor() + +cursor.execute("""SELECT + exchange_name, + CASE + WHEN strftime('%Y-%m', timestamp, 'unixepoch', '-3 hours') = strftime('%Y-%m', 'now', 'localtime') THEN strftime('%Y-%m', 'now', 'localtime') + WHEN strftime('%Y-%m', timestamp, 'unixepoch', '-3 hours') = strftime('%Y-%m', 'now', 'localtime', '-1 month') THEN strftime('%Y-%m', 'now', 'localtime', '-1 month') + WHEN strftime('%Y-%m', timestamp, 'unixepoch', '-3 hours') = strftime('%Y-%m', 'now', 'localtime', '-2 month') THEN strftime('%Y-%m', 'now', 'localtime', '-2 month') + ELSE 'Other Months' + END AS month_group, + SUM(amount) AS total_amount + FROM + profits_table + WHERE + strftime('%s', 'now') - timestamp <= 365 * 24 * 60 * 60 -- 365 days in seconds + GROUP BY + exchange_name, month_group + ORDER BY + exchange_name, month_group;""") +by_exchange = cursor.fetchall() + + +# Get the current date +current_date = datetime.date.today() + +# Get the current month in YYYY-MM format +current_month = current_date.strftime("%Y-%m") + +# Get the last three months +last_three_months = [current_month] +for i in range(3): + # Calculate the date for the previous month + previous_month = current_date.replace(day=1) - datetime.timedelta(days=current_date.day) + previous_month = previous_month.replace(month=previous_month.month - i) + + # Format the previous month to YYYY-MM format + previous_month_str = previous_month.strftime("%Y-%m") + + # Append the previous month to the list + last_three_months.append(previous_month_str) + + +#Now we got the month list and the db data, let's present it in a readable fashion +print("Revenue per month:") +print("="*linewidth) +for yearmonth in last_three_months[:3][::-1]: + print(f"{yearmonth}:") + total = 0 + for exchange in exchanges: + if exchange=="binance": + for item in by_exchange: + if item[0]=="binance" and item[1]==yearmonth: + binance_total = item[2] + total+=item[2] + elif exchange=="gateio": + for item in by_exchange: + if item[0]=="gateio" and item[1]==yearmonth: + gateio_total = item[2] + total+=item[2] + elif exchange=="kucoin": + for item in by_exchange: + if item[0]=="kucoin" and item[1]==yearmonth: + kucoin_total = item[2] + total+=item[2] + elif exchange=="okex": + for item in by_exchange: + if item[0]=="okex" and item[1]==yearmonth: + okex_total = item[2] + total+=item[2] + print(f"Binance: {round(binance_total,2)} ({round(binance_total/total*100,2)}%)") + print(f"Gate.io: {round(gateio_total,2)} ({round(gateio_total/total*100,2)}%)") + print(f"KuCoin: {round(kucoin_total,2)} ({round(kucoin_total/total*100,2)}%)") + print(f"OKX: {round(okex_total,2)} ({round(okex_total/total*100,2)}%)") + print("-"*linewidth) + print(f"Total: {round(total,2)}") + print("="*linewidth) + diff --git a/profits/profit_report.py b/profits/profit_report.py new file mode 100644 index 0000000..8799f4e --- /dev/null +++ b/profits/profit_report.py @@ -0,0 +1,136 @@ +''' +Profits report from db +''' + +import sqlite3 +import calendar +import datetime + + +exchanges = ["binance","gateio","kucoin","okex"] +line_width = 40 + +#Connect to db +connection = sqlite3.connect("profits_database.db") +cursor = connection.cursor() + +#Last 60 days query +cursor.execute("""SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, + SUM(amount) AS total_amount + FROM profits_table + WHERE strftime('%s', 'now') - timestamp <= 60 * 24 * 60 * 60 -- 60 days in seconds + GROUP BY day_utc3 + ORDER BY day_utc3;""") +last_60_days_rows = cursor.fetchall() + +#Last 30 days query +#cursor.execute("""SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, +cursor.execute("""SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, + SUM(amount) AS total_amount + FROM profits_table + WHERE strftime('%s', 'now') - timestamp <= 30 * 24 * 60 * 60 -- 30 days in seconds;""") +last_30_days = cursor.fetchall() + +#Last 7 days query +cursor.execute("""SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, + SUM(amount) AS total_amount + FROM profits_table + WHERE strftime('%s', 'now') - timestamp <= 7 * 24 * 60 * 60 -- 7 days in seconds;""") +last_7_days = cursor.fetchall() + +#Last n months query +cursor.execute("""SELECT strftime('%Y-%m', timestamp, 'unixepoch', '-3 hours') AS year_month_utc3, + SUM(amount) AS total_amount + FROM profits_table + WHERE strftime('%s', 'now') - timestamp <= 18 * 30 * 24 * 60 * 60 -- 18 months in seconds + GROUP BY year_month_utc3 + ORDER BY year_month_utc3;""") +last_n_months_rows = cursor.fetchall() + +#Yearly totals +cursor.execute("""SELECT strftime('%Y', timestamp, 'unixepoch', '-3 hours') AS year_utc3, + SUM(amount) AS total_amount + FROM profits_table + WHERE strftime('%s', 'now') - timestamp <= 24 * 365 * 60 * 60 -- 365 days in seconds + GROUP BY year_utc3 + ORDER BY year_utc3;""") +yearly_totals = cursor.fetchall() + +print("="*line_width) +print("Last 60 days:") +print("-"*line_width) +for row in last_60_days_rows: + print(f"{row[0]}: {round(row[1],2)}") +print("="*line_width) +print("Last 18 months:") +print("-"*line_width) +for row in last_n_months_rows[1:]: + print(f"{row[0]}: {round(row[1],2)}") +print("-"*line_width) +print(f"Last 30 days average: {round(last_30_days[0][1]/30,2)}") +print(f"Last 7 days average: {round(last_7_days[0][1]/7,2)}") +cursor.execute("""SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, + SUM(amount) AS total_amount + FROM profits_table + WHERE timestamp > strftime('%s', 'now', 'start of month', 'utc') + GROUP BY day_utc3;""") +last_month = cursor.fetchall() + +#The projection calculation is: the amount of profit so far in the month + the averages of the last 30 days and the last 7 days times the number of days left in the month. +days_in_month = calendar.monthrange(datetime.date.today().year, datetime.date.today().month)[1] +daily_combined_media = (last_30_days[0][1]/30+last_7_days[0][1]/7)/2 +current_amount = last_n_months_rows[-1][1] +days_past_this_month = int(last_60_days_rows[-1][0][8:10]) +projection_calculation = current_amount + daily_combined_media*(days_in_month-days_past_this_month) + +print(f"This month projection: {round(projection_calculation,2)}") +print("="*line_width) +print("Per exchange:") +print("-"*line_width) +cursor.execute("""SELECT + exchange_name, + CASE + WHEN strftime('%Y-%m', timestamp, 'unixepoch', '-3 hours') = strftime('%Y-%m', 'now', 'localtime') THEN 'This Month' + WHEN strftime('%Y-%m', timestamp, 'unixepoch', '-3 hours') = strftime('%Y-%m', 'now', 'localtime', '-1 month') THEN 'Last Month' + ELSE 'Other Months' + END AS month_group, + SUM(amount) AS total_amount + FROM + profits_table + WHERE + strftime('%s', 'now') - timestamp <= 60 * 24 * 60 * 60 -- 60 days in seconds + GROUP BY + exchange_name, month_group + ORDER BY + exchange_name, month_group;""") + +#So type checking goes away +binance_amount = 0 +gateio_amount = 0 +kucoin_amount = 0 +okex_amount = 0 + +by_exchange = cursor.fetchall() +for row in by_exchange: + if row[0]=="binance": + if row[1]=="This Month": + binance_amount = row[2] + elif row[0]=="gateio": + if row[1]=="This Month": + gateio_amount = row[2] + elif row[0]=="kucoin": + if row[1]=="This Month": + kucoin_amount = row[2] + elif row[0]=="okex": + if row[1]=="This Month": + okex_amount = row[2] + +total_amount = binance_amount+gateio_amount+kucoin_amount+okex_amount + +print(f"Binance: {round(binance_amount,2)} USDT ({round(binance_amount/total_amount*100,2)}%)") +print(f"Gate.io: {round(gateio_amount,2)} USDT ({round(gateio_amount/total_amount*100,2)}%)") +print(f"KuCoin: {round(kucoin_amount,2)} USDT ({round(kucoin_amount/total_amount*100,2)}%)") +print(f"OKX: {round(okex_amount,2)} USDT ({round(okex_amount/total_amount*100,2)}%)") +print("="*line_width) + + diff --git a/profits/profit_since_short.py b/profits/profit_since_short.py new file mode 100755 index 0000000..5c11cef --- /dev/null +++ b/profits/profit_since_short.py @@ -0,0 +1,42 @@ +import os, sys, json, datetime + +pair = sys.argv[1] + +def load_old_long(pair): + #Load old_long info + try: + with open(f"../status/{pair}.oldlong") as f: + return json.load(f) + except Exception as e: + print(e) + return None + +def load_old_long_from_status(pair): + #Load old_long info + try: + with open(f"../status/{pair}.status") as f: + return json.load(f)["old_long"] + except Exception as e: + print(e) + return None + +old_long = load_old_long(pair) +if old_long is None: + old_long = load_old_long_from_status(pair) +if old_long is None: + sys.exit(0) + +#Get time of switch to unix time +old_date = old_long["datetime"][1:11] +time_of_switch = datetime.datetime.strptime(old_date,"%Y/%m/%d").timestamp() + +#Calculate profits +total = 0 +with open(f"{pair}.profits") as csvfile: + for x in csvfile: + [date,amount,_] = x.split(",") + time_of_profit = datetime.datetime.strptime(date,"%Y-%m-%d").timestamp() + if time_of_profit>=time_of_switch: + total += float(amount) + +print(f"Profits since {old_date}: {round(total,2)}") diff --git a/profits/profits_since_date.py b/profits/profits_since_date.py new file mode 100644 index 0000000..0502cc6 --- /dev/null +++ b/profits/profits_since_date.py @@ -0,0 +1,37 @@ +''' +Usage: python3 profits_since_date.py BASE/QUOTE YYYY-MM-DD +''' + +import sqlite3 +import sys +import datetime + +try: + pair = sys.argv[1].replace("/","") + since_date = sys.argv[2] +except Exception as e: + print(e) + print("Usage: python3 profits_since_date.py BASE/QUOTE YYYY-MM-DD") + sys.exit() + +#Connect to db +connection = sqlite3.connect("../profits/profits_database.db") +cursor = connection.cursor() + +linux_time = datetime.datetime.strptime(since_date,"%Y-%m-%d").timestamp() + +#Query database +query = f"""SELECT pair, SUM(amount) AS total_profit + FROM profits_table + WHERE timestamp >= '{linux_time}' + GROUP BY pair;""" +cursor.execute(query) +query_result = cursor.fetchall() + +#Calculate profits +total = 0 +for item in query_result: + if item[0].replace("/","")==pair: + total = item[1] + +print(f"Profits since {since_date}: {round(total,2)}") \ No newline at end of file diff --git a/profits/profits_that_day.py b/profits/profits_that_day.py new file mode 100644 index 0000000..3425c5a --- /dev/null +++ b/profits/profits_that_day.py @@ -0,0 +1,27 @@ +''' +Returns the profits of a certain day, grouped by pair. + +Usage: python3 profits_that_day.py YYYY-MM-DD +''' + +import sqlite3 +import sys + +try: + date = sys.argv[1] +except Exception as e: + print(e) + print("Usage: python3 profits_that_day.py YYYY-MM-DD") + sys.exit() + +#Connect to db +connection = sqlite3.connect("profits_database.db") +cursor = connection.cursor() + +query = f"SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, SUM(amount) AS total_amount FROM profits_table GROUP BY day_utc3 ORDER BY day_utc3;" +cursor.execute(query) +result = cursor.fetchall() + +for item in result: + if item[0]==date: + print(f"Profits on {date}: {round(item[1],2)}") \ No newline at end of file diff --git a/trader.py b/trader.py new file mode 100755 index 0000000..fbd2caf --- /dev/null +++ b/trader.py @@ -0,0 +1,1806 @@ +import csv +import json +import time +import os + +class trader: + def __init__(self, broker, config_dict: dict, is_import: bool = False): + self.pause = True #Signals the trader to not process order info when an API call manhandles the trader + #True by default, once the trader is started the start_bot method toggles it + self.quit = False #If true, it doesn't restart the bot when profit is reached. + self.restart = False + + self.broker = broker + self.tp_order = self.broker.get_empty_order() + self.so = self.broker.get_empty_order() + self.config_dict = config_dict + self.pair = self.config_dict["pair"] + self.market = self.broker.fetch_market(self.pair) + self.market_load_time = int(time.time()) + self.market_reload_period = 86400 #Market reload period in seconds + self.base,self.quote = self.pair.split("/") + self.is_short = self.config_dict["is_short"] + self.profit_table = self.config_dict["tp_table"] + self.max_short_safety_orders = 45 + if "max_short_safety_orders" in config_dict: + self.max_short_safety_orders = config_dict["max_short_safety_orders"] + self.start_time = int(time.time()) + self.total_amount_of_quote=0 + self.total_amount_of_base=1 + self.take_profit_price=0 + self.safety_price_table=[0] + self.fees_paid_in_base=0 + self.fees_paid_in_quote=0 + self.deal_start_time = 0 + self.last_time_seen = time.time() + self.start_price = 0 + self.safety_order_index = 0 + self.status_dict = { + "quote_spent": 0, + "base_bought": 0, + "so_amount": 0, + #"max_so_amount": config_dict["no_of_safety_orders"], + "take_profit_price": 1, + "next_so_price": 1, + #"acc_profit": 0, + "tp_order_id": "", + "take_profit_order": {}, + "so_order_id": "", + "safety_order": {}, + "safety_price_table": [], + "pause_reason": "", + "deal_uptime": 0, #In seconds + "total_uptime": 0, #In seconds + "price": 0, + "deal_order_history": [] + } + self.warnings = { + "short_price_exceeds_old_long": False, + "speol_notified": False + } + if "stop_time" in self.config_dict and int(self.config_dict["stop_time"]) int: + self.market_load_time = period + return 0 + + + def get_market_reload_period(self) -> float: + return self.market_reload_period + + + def reload_safety_order(self) -> int: + ''' + Reloads the safety order. + ''' + + self.so = self.broker.get_order(self.status_dict["so_order_id"],self.pair) + return 0 + + + def start_bot(self) -> int: + ''' + Initializes the trader. + ''' + #Perhaps we should search for open buy orders from a crashed trader and cancel them? + + #Reset some variables + self.safety_order_index = 0 + self.status_dict["deal_order_history"].clear() + + #Reloads the market + new_market_data = self.broker.fetch_market(self.pair) + if new_market_data is not None: + self.market = new_market_data + + self.pause = True + self.status_dict["pause_reason"] = "start_bot" + + if self.is_short: + self.broker.logger.log_this("Calculating optimal order size...",2,self.pair) + + #Get minimum order size from exchange + self.broker.logger.log_this("Fetching minimum order size...",2,self.pair) + min_base_size = self.broker.get_min_base_size(self.pair) + if min_base_size is None: + self.broker.logger.log_this("Can't fetch the minimum order size",1,self.pair) + return 1 + + #Fetch the amount of free base available on the exchange + self.broker.logger.log_this("Fetching free base currency on the exchange...",2,self.pair) + free_base = self.fetch_free_base() + if free_base is None: + self.broker.logger.log_this("Can't fetch the amount of base at the exchange",1,self.pair) + return 1 + + #Buy missing base sold because of rounding errors (rare) + if "old_long" in self.status_dict: + diff = self.status_dict["old_long"]["tp_amount"] - free_base + if diff>min_base_size: + diff = self.broker.amount_to_precision(self.pair,diff) + self.broker.logger.log_this(f"Buying missing {diff} {self.base}",1,self.pair) + self.broker.new_market_order(self.pair,diff,"buy",amount_in_base=True) + time.sleep(self.broker.get_wait_time()*2) + #Re-quering for the amount of base currency on the exchange + free_base = self.fetch_free_base() + if free_base is None: + self.broker.logger.log_this("Can't fetch the amount of base at the exchange",1,self.pair) + return 1 + + #Calculate order size and amount of safety orders + self.broker.logger.log_this("Calculating the order size...",2,self.pair) + order_size,no_of_safety_orders = self.calculate_order_size(free_base,min_base_size,self.max_short_safety_orders) + if order_size is None or no_of_safety_orders is None: + self.broker.logger.log_this("Can't calculate optimal size",1,self.pair) + return 1 + self.config_dict["order_size"] = order_size + self.config_dict["no_of_safety_orders"] = no_of_safety_orders + self.broker.logger.log_this(f"Order size: {self.broker.amount_to_precision(self.pair,order_size)}. Amount of safety orders: {no_of_safety_orders}",2,self.pair) + + #Write the changes to the config file + with open(f"configs/{self.base}{self.quote}.json","w") as g: + g.write(json.dumps(self.config_dict, indent=4)) + + else: + #Check order size + self.status_dict["pause_reason"] = "start_bot - checking order size" + self.broker.logger.log_this("Checking for order size",2,self.pair) + minimum_order_size_allowed = self.broker.get_min_quote_size(self.pair) + if minimum_order_size_allowed is not None and minimum_order_size_allowed>self.config_dict["order_size"]: + self.broker.logger.log_this(f"Order size too small. Minimum order size is {minimum_order_size_allowed} {self.quote}",1,self.pair) + if minimum_order_size_allowed dict: + ''' + Reloads the config dictionary from disk + + :return: dict + ''' + config_filename = f"configs/{self.base}{self.quote}.json" + with open(config_filename,"r") as y: + config_dict = json.load(y) + if self.config_dict["autoswitch"]: + config_dict["autoswitch"] = True + return config_dict + + + + def update_status(self, write_to_disk: bool) -> int: + ''' + Updates the status dictionary + + :param write_to_disk: bool - If True, writes the status file to disk. + :return: int + ''' + try: + if self.tp_order is not None: #Type checking + self.status_dict["tp_order_id"]=self.tp_order["id"] + self.status_dict["take_profit_order"]=self.tp_order + if self.so is not None: #Type checking + self.status_dict["so_order_id"]=self.so["id"] + self.status_dict["safety_order"]=self.so + if self.tp_order is not None and self.tp_order["price"] is not None: + self.status_dict["take_profit_price"]=self.tp_order["price"] + + try: + self.status_dict["next_so_price"]=self.safety_price_table[self.safety_order_index] #List index out of range bug + except Exception as e: + self.broker.logger.log_this(f"Is safety_price_table populated? Exception: {e} | Safety price table: {self.safety_price_table} | Safety order index: {self.safety_order_index}",1,self.pair) + + if self.so is not None and self.so["price"] is not None and self.so!=self.broker.get_empty_order(): + self.status_dict["next_so_price"]=self.so["price"] + self.status_dict["is_short"]=self.is_short + self.status_dict["quote_spent"]=self.total_amount_of_quote + self.status_dict["base_bought"]=self.total_amount_of_base + self.status_dict["so_amount"]=self.safety_order_index + self.status_dict["no_of_safety_orders"]=self.config_dict["no_of_safety_orders"] + self.status_dict["take_profit_price"]=self.take_profit_price + self.status_dict["safety_price_table"]=self.safety_price_table + self.status_dict["deal_uptime"]=int(time.time()) - self.deal_start_time + self.status_dict["total_uptime"]=int(time.time()) - self.start_time + self.status_dict["fees_paid_in_base"]=self.fees_paid_in_base + self.status_dict["fees_paid_in_quote"]=self.fees_paid_in_quote + self.status_dict["start_price"]=self.start_price + self.status_dict["tp_mode"]=self.config_dict["tp_mode"] + self.status_dict["profit_table"]=self.config_dict["tp_table"] + self.status_dict["start_time"]=self.start_time + self.status_dict["deal_start_time"]=self.deal_start_time + self.status_dict["stop_when_profit"]=self.stop_when_profit + except Exception as e: + self.broker.logger.log_this(f"Can't update status dictionary. Exception: {e}",1,self.pair) + + if write_to_disk: + self.write_status_file() + #try: + # if write_to_disk: + # json_object = json.dumps(self.status_dict, indent=4) + # with open(f"status/{self.base}{self.quote}.status", "w") as c: + # c.write(json_object) + #except Exception as e: + # self.broker.logger.log_this(f"Can't write status file to disk. Exception: {e}",1,self.pair) + return 0 + + def write_status_file(self,is_backup:bool=False): + try: + json_object = json.dumps(self.status_dict, indent=4) + file_name = f"{self.base}{self.quote}.status" + if is_backup: + self.broker.logger.log_this("Backing up status file...",2,self.pair) + file_name = time.strftime(f"{self.base}{self.quote}_%Y-%m-%d_%H:%M:%S.backup_status") + with open(f"status/{file_name}", "w") as c: + c.write(json_object) + except Exception as e: + self.broker.logger.log_this(f"Can't write status file to disk. Exception: {e}",1,self.pair) + + def dca_cost_calculator(self, order_size: float, amount_of_so: int, scalar: float) -> float: + ''' + Returns the maximum amount of currency that can be used by a trader, given the initial order size + + :param order_size: float + :param amount_of_so: int + :param scalar: float + :return: float + ''' + total = order_size + for i in range(1,amount_of_so+1): + total+=self.gib_so_size(order_size,i,scalar) + return total + + + def return_optimal_order_size(self, amount: float, min_size: float, amount_of_safety_orders: int, scalar: float) -> float: + ''' + Calculates the optimal order size for a short bot, according to the amount passed as a parameter. + Due to performance issues, the step size that is used is 1/10th of the minimum order size. + + :param amount: float + :param min_size: float + :param amount_of_safety_orders: int + :param scalar: float + :return: float + ''' + total_size = float(min_size) + + #Calculate optimal step size + self.broker.logger.log_this("Calculating optimal step size...",2,self.pair) + #step = self.get_step_size() + #if step is None: + # step = min_size + #if step==0: + # step = min_size + #self.broker.logger.log_this(f"Step size is {step}",2,self.pair) + + divisor = 10 + while divisor>0: + #step = self.broker.amount_to_precision(self.pair,min_size/divisor) + step = min_size/divisor + if step!=0: #When using amount_to_precision, this comes handy. + break + divisor-=1 + + #if step==0: + # step = self.broker.amount_to_precision(self.pair,min_size) + previous_size = 0 + while True: #This loop should have a safeguard + self.broker.logger.log_this(f"Calculating optimal order size ...",2,self.pair) + total_cost = self.dca_cost_calculator(total_size,amount_of_safety_orders,scalar) + if total_cost>=amount: + return previous_size + previous_size = total_size + total_size+=step + + + def parse_fees(self, order: dict) -> tuple: + ''' + Returns the fees paid ordered in "base,quote" + Note: CCXT does not detail the fees paid if the exchange is Binance. + ''' + + basefee = 0 + quotefee = 0 + + #Uncomment if you want to guesstimate Binance's fees. + #if self.broker.get_exchange_name()=="binance": + # #Fees of buy orders are charged in base currency, fees of sell orders are charged in quote currency. + # try: + # fee_rate = self.market["maker"] if order["type"]=="limit" else self.market["taker"] + # except Exception as e: + # self.broker.logger.log_this(f"Exception fetching market information: {e}. Using default fee rate of 0.1%",1,self.pair) + # fee_rate = 0.001 + # + # if order["side"]=="buy": + # basefee = order["filled"]*float(fee_rate) + # elif order["side"]=="sell": + # quotefee = order["cost"]*float(fee_rate) + # return basefee,quotefee + + for x in order["fees"]: + if x["currency"]==self.base: + basefee+=float(x["cost"]) + if x["currency"]==self.quote: + quotefee+=float(x["cost"]) + return basefee,quotefee + + + def do_cleanup(self) -> int: + ''' + Checks for any remaining base currency balance on the exchange + If it finds some and that amount is enough, it sends a sell order at the take profit price + It was implemented because some deals close with a little amount of base remaining + and it tends to pile up overtime + A more elegant solution would be to take note of the amount and the price at the moment of the deal closing that lead to + that small amount of change to appear, to make possible to calculate an optimal sell price of the remaining assets + instead of brute forcing it this way. + For smaller bots that might be overengineering it a bit anyway + ''' + + if self.is_short: #Short bots do not need cleanup + return 0 + balance_to_clean = self.fetch_free_base() + if balance_to_clean is None: + self.broker.logger.log_this("Can't fetch free base",1,self.pair) + return 1 + balance_to_clean /= 2 #Maybe it's a good idea, sort of DCAing the dust. + min_base_size = self.broker.get_min_base_size(self.pair) + if min_base_size is not None and balance_to_clean >= min_base_size: + self.broker.logger.log_this(f"Balance to clean: {balance_to_clean} {self.base}",2,self.pair) + self.broker.logger.log_this("Sending cleanup order...",2,self.pair) + cleanup_order = self.broker.new_limit_order(self.pair,balance_to_clean,"sell",self.take_profit_price) + if cleanup_order not in [None,self.broker.get_empty_order()]: + self.broker.logger.log_this("Cleanup successful",2,self.pair) + return 0 + self.broker.logger.log_this("Problems with the cleanup order",1,self.pair) + return 1 + self.broker.logger.log_this("No cleanup needed",2,self.pair) + return 0 + + + def calculate_order_size(self, free_base: float, min_base_size: float, amount_of_so: int = 30) -> tuple: + ''' + Calculates the optimal order size and the amount of safety orders from the free_base and minimum base size + ''' + + optimal_order_size = 0 + minimum_amount_of_safety_orders = 1 #This variable could be a config knob + while amount_of_so>minimum_amount_of_safety_orders: + optimal_order_size = self.return_optimal_order_size(free_base,min_base_size,amount_of_so,self.config_dict["safety_order_scale"]) #safety_order_scale: safety order growth factor + if optimal_order_size!=0: + self.broker.logger.log_this(f"Optimal order size is {optimal_order_size}",2,self.pair) + break + amount_of_so-=1 + if optimal_order_size==0: + self.broker.logger.log_this("Not enough base to switch. Order size would be too small",1,self.pair) + self.pause = False + self.status_dict["pause_reason"] = "" + return None,None + if optimal_order_size int: + ''' + This method modifies the config file of a pair to convert it to short. + It automagically sets the order size according to the funds available. + If not enough funds, it returns 1 + ''' + + if self.is_short: #Check if bot is already a short bot + return 1 + + #Let's do some type checking first + if self.tp_order is None: + self.broker.logger.log_this("Take profit order is None, can't switch to short",1,self.pair) + return 1 + if self.so is None: + self.broker.logger.log_this("Safety order is None, can't switch to short",1,self.pair) + return 1 + + #Pauses bot + self.pause = True + self.status_dict["pause_reason"] = "switch_to_short" + + #Read the config file + self.broker.logger.log_this("Reading config file",2,self.pair) + try: + with open(f"configs/{self.base}{self.quote}.json","r") as f: + old_config = json.load(f) + except Exception as e: + self.broker.logger.log_this(f"Error. Can't read the config file. Can't switch mode. Exception: {e}",1,self.pair) + self.pause = False + self.status_dict["pause_reason"] = "" + return 1 + + #Calculate if there is enough base and, if so, calculate the optimal order size + + #Fetch the real amount of available base + self.broker.logger.log_this(f"Fetching available {self.base}",2,self.pair) + free_base = self.fetch_free_base() + if free_base is None: + return 1 + + #Fetch the minimal order size + min_base_size = self.broker.get_min_base_size(self.pair) + if min_base_size is None: + self.broker.logger.log_this("Error. Can't fetch market info from the exchange",1,self.pair) + self.pause = False + self.status_dict["pause_reason"] = "" + return 1 + + #Check if there is enough base + if self.broker.amount_to_precision(self.pair,free_base+self.tp_order["amount"])<=min_base_size: + self.broker.logger.log_this("Error. Not enough base currency",1,self.pair) + self.pause = False + self.status_dict["pause_reason"] = "" + return 1 + + #Calculate order size + self.broker.logger.log_this("Calculating optimal order size",2,self.pair) + optimal_order_size,amount_of_so = self.calculate_order_size(free_base+self.tp_order["amount"],min_base_size,amount_of_so=self.max_short_safety_orders) + if optimal_order_size is None or amount_of_so is None: + return 1 + self.broker.logger.log_this(f"New order size: {optimal_order_size}",2,self.pair) + self.broker.logger.log_this(f"Amount of safety orders: {amount_of_so}",2,self.pair) + + #Close old orders + self.broker.logger.log_this("Switching trader mode to short",2,self.pair) + self.broker.logger.log_this("Closing orders...",2,self.pair) + if self.broker.cancel_order(self.tp_order["id"],self.pair)==1: + self.broker.logger.log_this("Can't cancel the take profit order. Can't switch mode",1,self.pair) + self.pause = False + self.status_dict["pause_reason"] = "" + return 1 + if self.so["id"]!="": + self.broker.cancel_order(self.so["id"],self.pair) + + #Save the old take profit order info for later and saves it to a different file in case of trader crash + self.broker.logger.log_this("Saving state in status_dict",2,self.pair) + self.status_dict["old_long"] = {"tp_price": self.tp_order["price"], + "tp_amount": self.tp_order["amount"], + "quote_spent": self.total_amount_of_quote, + "fees_paid_in_quote": self.fees_paid_in_quote, + "datetime": time.strftime("[%Y/%m/%d %H:%M:%S]") + } + try: + with open(f"status/{self.base}{self.quote}.oldlong","w") as s: + s.write(json.dumps(self.status_dict["old_long"],indent=4)) + except Exception as e: + self.broker.logger.log_this(f"Exception while saving old_long file: {e}",1,self.pair) + + #Modify config file accordingly + self.broker.logger.log_this("Modifying config file and saving a backup",2,self.pair) + try: + with open(f"configs/{self.base}{self.quote}.bak","w") as c: + c.write(json.dumps(old_config, indent=4)) + old_config["is_short"] = True + old_config["order_size"] = optimal_order_size #Now calculated on-the-fly at the start of every deal + old_config["no_of_safety_orders"] = amount_of_so + #4. Write the config file + with open(f"configs/{self.base}{self.quote}.json","w") as c: + c.write(json.dumps(old_config, indent=4)) + self.broker.logger.log_this("Config file updated",2,self.pair) + except Exception as e: + self.broker.logger.log_this(f"Error. Can't write the config file. Exception: {e}",1,self.pair) + #self.pause = False + return 1 + self.stop_when_profit = False + self.is_short = True + self.broker.logger.log_this("Done configuring. Starting bot...",2,self.pair) + return 0 + + + def switch_to_long(self, from_take_profit: bool = False, ignore_old_long: bool = False) -> int: + ''' + Takes a short bot and changes the mode to long. + Only does it if the current bot was previously a long one. + ''' + + #Check if it's not already long + if not self.is_short: + self.broker.logger.log_this("Can't switch a long trader to long, there's nothing to do",1,self.pair) + return 1 + + #Check if the orders are OK + if self.tp_order is None: + self.broker.logger.log_this("Take profit order is None, can't switch to short",1,self.pair) + return 1 + if self.so is None: + self.broker.logger.log_this("Safety order is None, can't switch to short",1,self.pair) + return 1 + + #Send Telegram message + self.broker.logger.log_this("Attempting to switch to long bot",0,self.pair) + + if not ignore_old_long and "old_long" not in self.status_dict: + self.broker.logger.log_this("Can't find old long info on status_dict, searching for oldlong file",1,self.pair) + try: + with open(f"status/{self.base}{self.quote}.oldlong") as f: + self.status_dict["old_long"] = json.load(f) + except Exception as e: + #self.write_to_log(time.strftime(f"[%Y/%m/%d %H:%M:%S] | {self.pair} | Can't find old long file")) + self.broker.logger.log_this(f"Can't file oldlong file. Exception: {e}",1,self.pair) + return 1 + + #Cancel open orders + if not from_take_profit: + #I think this exception handler is al pedo. + try: + self.broker.cancel_order(self.so["id"],self.pair) + except Exception as e: + self.broker.logger.log_this(f"Error in cancel_order while cancelling safety order. Exception: {e}",1,self.pair) + #Also this one. + try: + self.broker.cancel_order(self.tp_order["id"],self.pair) + except Exception as e: + self.broker.logger.log_this(f"Error in cancel_order while cancelling take profit order. Exception: {e}",1,self.pair) + + #Liquidate base + self.liquidate_base(ignore_profits=ignore_old_long) + + #switch config files and set self.is_short to False + try: + with open(f"configs/{self.base}{self.quote}.bak") as c: + old_config = json.load(c) + with open(f"configs/{self.base}{self.quote}.json","w") as c: + c.write(json.dumps(old_config, indent=4)) + self.is_short = False + except Exception as e: + self.broker.logger.log_this(f"Exception in switch_to_long while switching config files: {e}",1,self.pair) + return 1 + + #Remove old_long file (if it exists) + if os.path.isfile(f"status/{self.base}{self.quote}.oldlong"): + self.broker.logger.log_this("Removing old_long file...",2,self.pair) + os.remove(f"status/{self.base}{self.quote}.oldlong") + + #Set up a few variables + self.fees_paid_in_quote = 0 + self.fees_paid_in_base = 0 + self.tp_order = self.broker.get_empty_order() + self.so = self.broker.get_empty_order() + self.safety_price_table = [0] + self.safety_order_index = 0 + + #Disabling autoswitch + #self.config_dict["autoswitch"] = False + + #Done. Ready for start_bot + return 0 + + + def liquidate_base(self, ignore_profits: bool = True) -> int: + ''' + Fetches the amount of free base on the exchange, sells the entirety of the amount and calculates profits + Sends the Telegram message and writes the profits to disk + ''' + + #Find out the amount of free base + free_base = self.fetch_free_base() + if free_base is None: + self.broker.logger.log_this("Can't fetch free base",1,self.pair) + return 1 + #send market order selling the total amount of base in the last take profit short order + order = self.broker.new_market_order(self.pair,free_base,"sell") + tries = self.broker.get_retries() + while True: + time.sleep(self.broker.get_wait_time()) + market_tp_order = self.broker.get_order(order["id"],self.pair) + if market_tp_order["status"]=="closed": + _, fees_paid = self.parse_fees(market_tp_order) + break + tries-=1 + if tries==0: + self.broker.logger.log_this("Liquidation order not filling. Skipping base liquidation",1,self.pair) + return 1 + + #calculate profits + if not ignore_profits: + profit = market_tp_order["cost"] - self.status_dict["old_long"]["quote_spent"] - self.status_dict["old_long"]["fees_paid_in_quote"] - fees_paid + #self.status_dict["acc_profit"] += profit + + #Add profits to file and send telegram notifying profits + self.profit_to_file(profit,market_tp_order["id"]) #This is not used anymore, but it's still here since it does not take almost any disk space and/or CPU time. + self.profit_to_db(profit,market_tp_order["id"],self.broker.get_write_order_history()) + self.broker.logger.log_this(f"Switch successful. Profit: {round(profit,2)} {self.quote}",0,self.pair) + self.broker.logger.log_this(f"Sell price: {market_tp_order['price']} {self.quote}",0,self.pair) + self.broker.logger.log_this(f"Order ID: {market_tp_order['id']}",0,self.pair) + return 0 + + + def take_profit_routine(self, filled_order: dict) -> int: + ''' + When profit is reached, this method is called to handle the profit calculations, the closing of orders, + the reporting and the restart of the trader. + ''' + + self.pause = True #To stop the main thread to iterate through this bot's orders (just in case) + self.status_dict["pause_reason"] = "take_profit_routine - order handling" #start_bot will set this flag to False again once it starts + + #Let's do some type checking first + if self.tp_order is None: + error_string = time.strftime(f"[%Y/%m/%d %H:%M:%S] | {self.pair} | TP order is None") + self.status_dict["pause_reason"] = error_string + self.broker.logger.log_this("Error. Take profit order is None, pair will be restarted",0,self.pair) + self.write_status_file(True) + self.restart = True + return 1 + if self.so is None: + error_string = time.strftime(f"[%Y/%m/%d %H:%M:%S] | {self.pair} | Safety order is None") + self.status_dict["pause_reason"] = error_string + self.broker.logger.log_this("Error. Safety order is None",1,self.pair) + self.so = self.broker.get_empty_order() + + #Save the order + self.status_dict["deal_order_history"].append(filled_order) + + # Cancel the current safety order (first check if there is something to cancel) + already_counted = False + if self.so["id"]=="": + self.broker.logger.log_this("There is no safety order to cancel",2,self.pair) + elif self.broker.cancel_order(self.so["id"],self.pair)==1: + self.broker.logger.log_this("Old safety order probably filled. Can't cancel.",1,self.pair) + closed_order = self.broker.get_order(self.so["id"],self.pair) + if closed_order!=self.broker.get_empty_order() and closed_order["status"]=="closed": + self.total_amount_of_base = self.total_amount_of_base + closed_order["filled"] + self.total_amount_of_quote = self.total_amount_of_quote + closed_order["cost"] + #Save the order + self.status_dict["deal_order_history"].append(closed_order) + already_counted = True + + #IF NOT SHORT - Check if the SO was partially filled. If so, add the amounts to total_amount_of_base and total_amount_of_quote + #Suggestion: Would it be feasible to send a market sell order for the amount of base bought on the old safety order? + #Or is it better to leave this amount of change to the cleanup routine? + if not self.is_short and self.so["id"]!="" and not already_counted: + old_so_order = self.broker.get_order(self.so["id"],self.pair) + if old_so_order["filled"]>0: + self.broker.logger.log_this(f"Old safety order is partially filled, ID: {old_so_order['id']}",1,self.pair) + self.status_dict["deal_order_history"].append(old_so_order) + #Uncomment the next two lines if you do not want to ignore the partial fill + #self.total_amount_of_base = self.total_amount_of_base + old_so_order["filled"] + #self.total_amount_of_quote = self.total_amount_of_quote + old_so_order["cost"] + + #Hypothetical market order code below: + #DO NOT UPDATE total_amount_of_base and total_amount_of_quote + #If there is not enough base for an order, we'll accumulate it and let the cleanup routine deal with that. + # floor_amount = self.broker.get_min_base_size(self.pair) + # if floor_amount is not None and old_so_order["filled"]>floor_amount: + # self.broker.logger.log_this(f"Sending sell order for partially filled funds: {old_so_order["filled"]} {self.base}",1,self.pair) + # loose_change_order = broker.new_market_order(self.pair,old_so_order["filled"],"sell",amount_in_base=True) + # retries = self.broker.get_retries() + # while retries>0: + # time.sleep(self.broker.get_wait_time()) + # order_to_fill = self.broker.get_order(loose_change_order["id"],self.pair) + # if order_to_fill["status"]=="closed" and order_to_fill["filled"]>0: + # profit_with_fees = order_to_fill["filled"]-old_so_order["cost"]-self.parse_fees(order_to_fill)[1] + # self.broker.logger.log_this(f"Trader closed a loose change deal. Profit: {profit_with_fees} {self.quote}",0,self.pair) + # if profit_with_fees>0: + # self.profit_to_file(profit_with_fees,order_to_fill["id"]) + # retries-=1 + + if not self.broker.check_for_duplicate_profit_in_db(filled_order): + self.status_dict["pause_reason"] = "calculating profit" + # Calculate the profit + if self.is_short: + profit = self.total_amount_of_quote-filled_order["cost"]-self.fees_paid_in_quote-self.parse_fees(filled_order)[1] + else: + profit = filled_order["cost"]-self.total_amount_of_quote-self.fees_paid_in_quote-self.parse_fees(filled_order)[1] + if "partial_profit" in self.status_dict: + profit+=self.status_dict["partial_profit"] + + #Checks if some base was left over. + base_profit = max(self.total_amount_of_base-filled_order["filled"],0) #To avoid negative numbers in base_profit + + # Write the profit to file and send telegram message + if profit>0: #Negative profits are not saved because the cleanup takes care of the unsold base currency (the notorious small change issue that plagues some exchanges) + self.profit_to_file(profit,filled_order["id"]) + self.profit_to_db(profit,filled_order["id"],self.broker.get_write_order_history()) + else: #For logging purposes + self.broker.logger.log_this(f"NEGATIVE PROFIT - Total amount of base: {self.total_amount_of_base}, base in the order: {filled_order['amount']}, base filled: {filled_order['filled']}, base 'profit': {base_profit}",1,self.pair) + self.telegram_bot_sendprofit(profit,filled_order,base_profit=base_profit) + + # Print profit message on screen + extra = ' and {:.4f}'.format(base_profit) + f" {self.base}" if base_profit>0 else "" + self.broker.logger.log_this(f"Trader closed a deal. Profit: {'{:.4f}'.format(profit)} {self.quote}{extra}",2,self.pair) + self.broker.logger.log_this(f"Fill price: {filled_order['price']} {self.quote}",2,self.pair) + self.broker.logger.log_this(f"Safety orders triggered: {self.safety_order_index-1}",2,self.pair) + + #Check if it needs to switch the trader mode + self.status_dict["pause_reason"] = "check for autoswitch" + #If it's a short bot that used to be long AND autoswitch is enabled + if self.is_short and self.config_dict["autoswitch"] and "old_long" in self.status_dict and filled_order["average"]>self.status_dict["old_long"]["tp_price"]: + #Checking the price again to prevent being fooled by anomalous spikes + #It could be done above instead of using filled_order["average"], bit this way eases the API load a little bit, + #since every time (except the last one) it won't be bigger anyway. + if float(self.broker.get_ticker_price(self.pair))>self.status_dict["old_long"]["tp_price"]: + #Sell all base (market), report the profits and restart the trader + self.status_dict["pause_reason"] = "automatic_switch" + if self.switch_to_long(from_take_profit=True)==1: + self.write_status_file(True) + self.restart = True + self.broker.logger.log_this("Error, switch_to_long returned 1, pair will be restarted",0,self.pair) + return 1 + + + self.status_dict["pause_reason"] = "take_profit_routine - check time limit" + #Checks if there is a time limit for the trader + if "stop_time" in self.config_dict and time.time()>int(self.config_dict["stop_time"]): + self.stop_when_profit = True + + self.status_dict["pause_reason"] = "take_profit_routine - if stop_when_profit" + if self.stop_when_profit: #Signal to stop when profit + self.broker.logger.log_this("Pair shutting down. So long and thanks for all the fish",0,self.pair) + self.quit = True + return 1 + + # Clear variables and reload config_dict + #self.pause = False + self.status_dict["pause_reason"] = "take_profit_routine - var cleanup" + self.fees_paid_in_quote = 0 + self.fees_paid_in_base = 0 + self.tp_order = self.broker.get_empty_order() + self.so = self.broker.get_empty_order() + self.safety_price_table.clear() + #self.safety_order_index = 0 + self.config_dict = self.reload_config_dict() + + self.status_dict["pause_reason"] = "take_profit_routine - Cooldown" + self.broker.logger.log_this(f"Cooldown period...",2,self.pair) + time.sleep(self.broker.get_wait_time()*self.broker.get_cooldown_multiplier()) + + try: + percentage_difference = abs(filled_order['price']-self.broker.get_mid_price(self.pair))/filled_order["price"] + if percentage_difference>self.broker.get_slippage_default_threshold(): + #self.broker.logger.log_this(f"Slippage threshold exceeded, {self.broker.get_wait_time()*self.broker.get_cooldown_multiplier()}s cooldown.",1,self.pair) + self.broker.logger.log_this(f"Slippage threshold exceeded, waiting for cooldown and restarting pair",1,self.pair) + time.sleep(self.broker.get_wait_time()*self.broker.get_cooldown_multiplier()) + self.pause = False + self.restart = True + self.write_status_file(True) + return 1 + except Exception as e: + self.broker.logger.log_this(f"Exception while calculating percentage difference: {e}",1,self.pair) + self.broker.logger.log_this(f"Resuming...",2,self.pair) + + #Restarting the trader + self.status_dict["pause_reason"] = "take_profit_routine - restart_bot call" + restart_bot = self.start_bot() + self.status_dict["pause_reason"] = "take_profit_routine - restart_bot call - start_bot() called" + #retries = self.broker.get_retries() + if restart_bot==0: + return 0 + elif restart_bot==1: + self.pause = False + self.write_status_file(True) + self.restart = True + self.broker.logger.log_this("Error in trader, start_bot returned 1. Trader will be restarted",1,self.pair) + return 1 + elif restart_bot==2: + self.pause = False + self.write_status_file(True) + self.restart = True + self.broker.logger.log_this("Error in trader, start_bot returned 2 (Initial order never got filled). Trader will be restarted",1,self.pair) + return 2 + elif restart_bot==3: + self.pause = False + self.write_status_file(True) + self.restart = True + self.broker.logger.log_this("Error in trader, start_bot returned 3 (Slippage exceeded). Trader will be restarted",1,self.pair) + return 3 + else: + self.broker.logger.log_this(f"Error restarting trader, trader will be removed. Error code {restart_bot}",0,self.pair) + self.write_status_file(True) + self.quit = True + return 1 + + + def sum_filled_amounts(self, order: dict) -> int: + ''' + Adds the amount filled and the cost of an order to the totals. + ''' + # Add up the last order fees + new_fees_base = 0 #For type checking compliance + new_fees_quote = 0 + + # Update the total_amount_of_quote and total_amount_of_base variables + if order["id"]!="": #Necessary check when adding SOs + new_fees_base,new_fees_quote = self.parse_fees(order) + self.fees_paid_in_quote += new_fees_quote + self.total_amount_of_base = self.total_amount_of_base + order["filled"] - new_fees_base + self.total_amount_of_quote = self.total_amount_of_quote + order["cost"] + + # Done + return 0 + + + def new_so_routine(self, filled_order: dict, send_new_so: bool) -> int: + ''' + Handles all the bureaucracy prior and after sending a new safety order + :param filled_order: dict + :param send_new_so: bool + :return: 0 OK, 1 not enough funds, if can't cancel old TP, can't send new TP + ''' + + #Let's do some type checking first + if self.tp_order is None: + self.broker.logger.log_this("Take profit order is None, can't send a new safety order",1,self.pair) + return 1 + if self.so is None: + self.broker.logger.log_this("Safety order is None, can't send a new safety order",1,self.pair) + return 1 + + self.pause = True + self.status_dict["pause_reason"] = "new_so_routine" + + # Save the order + self.status_dict["deal_order_history"].append(filled_order) + + # Cancel the tp order + if self.broker.cancel_order(self.tp_order["id"],self.pair)==1: + error_string = f"{self.pair} | {self.tp_order['id']} | Old TP order probably filled. Can't cancel. This trader should be restarted" + self.broker.logger.log_this(f"Old take profit order is probably filled, can't cancel. This trader should be restarted. Order ID: {self.tp_order['id']}",1,self.pair) + self.status_dict["pause_reason"] = error_string + return 2 + + # Add the amount filled in the last safety order to the totals + self.sum_filled_amounts(filled_order) + + #Cooldown + time.sleep(self.broker.get_wait_time()) + + # Send the new safety order. If all expected safety orders are filled, it assigns an empty order to self.so + if send_new_so: + self.broker.logger.log_this("Sending a new safety order",2,self.pair) + if self.send_new_safety_order(self.status_dict["order_size"])==1: + error_string = "Problems sending the new safety order. Maybe not enough funds?" + self.broker.logger.log_this(error_string,1,self.pair) + self.status_dict["pause_reason"] = error_string + self.sum_filled_amounts(filled_order) + #self.so = self.broker.get_empty_order() + return 1 + else: + self.so = self.broker.get_empty_order() + self.safety_order_index+=1 + + # Check if the old tp order was partially filled. If so, update the previous two variables accordingly + # TODO: This should also be taken into account for the profit calculation + # Do the partial profit calculation and save it for later + old_tp_order = self.broker.get_order(self.tp_order["id"],self.pair) + if old_tp_order["filled"]>0: + self.broker.logger.log_this(f"Old take profit order is partially filled, id {old_tp_order['id']}",1,self.pair) + self.status_dict["deal_order_history"].append(old_tp_order) + #self.total_amount_of_base = old_tp_order["remaining"] + # Partial profit calculation + #if not self.is_short: + # current_deal_price = self.total_amount_of_base/self.total_amount_of_quote + # self.status_dict["partial_profit"] = old_tp_order["cost"]-(old_tp_order["filled"]*current_deal_price)-self.parse.fees(old_tp_order)[1] + # self.update_status(True) + # + # Maybe here we shouldn't substract fees yet, but add them up to the check. + # + self.total_amount_of_base = self.total_amount_of_base - old_tp_order["filled"] - self.parse_fees(old_tp_order)[0] + self.total_amount_of_quote = self.total_amount_of_quote - old_tp_order["cost"]# + self.parse_fees(old_tp_order)[1] + self.fees_paid_in_quote += self.parse_fees(old_tp_order)[1] + #self.fees_paid_in_base += self.parse_fees(old_tp_order)[0] + + #Cooldown + time.sleep(self.broker.get_wait_time()) + + # Send the new take profit order + if self.send_new_tp_order()==1: + error_string = "Problems sending the new take profit order" + self.broker.logger.log_this("Problems sending the new take profit order",1,self.pair) + self.status_dict["pause_reason"] = error_string + return 3 + + # Update the status_dict and that's it + self.update_status(True) + self.pause = False + self.status_dict["pause_reason"] = "" + return 0 + + + def check_old_long_price(self) -> int: + ''' + Checks if short price exceeds old long price. If so, send a Telegram message + ''' + price_exceeds = False + if "old_long" in self.status_dict: + price_exceeds = self.status_dict["price"]>float(self.status_dict["old_long"]["tp_price"]) + if price_exceeds: + self.warnings["short_price_exceeds_old_long"] = True + else: + self.warnings["short_price_exceeds_old_long"] = False + self.warnings["speol_notified"] = False + if not self.warnings["speol_notified"] and price_exceeds: + #Only notify one time AND if autoswitch is off + self.warnings["speol_notified"] = True + if not self.config_dict["autoswitch"]: + message = f"{self.base}@{self.status_dict['price']} ({str(self.broker.exchange)}), exceeds old long price of {self.status_dict['old_long']['tp_price']}" + self.broker.logger.log_this(message,0,self.pair) + return 0 + + + def check_slippage(self, threshold: float, order_size: float) -> bool: + ''' + Checks that the slippage is between range. Returns True if threshold is exceeded, False otherwise. + ''' + order_book = self.broker.get_order_book(self.pair,no_retries=True) + if order_book=={}: + self.broker.logger.log_this("Can't fetch orderbook",1,self.pair) + return False + suma = 0 + try: + mid_price = (order_book["asks"][0][0]+order_book["bids"][0][0])/2 + amount = (order_size/mid_price)*2 #Extra margin to avoid surprises. + first_price = order_book["asks"][0][0] + for x in order_book["asks"]: + suma += x[1] + if suma>=amount: + last_price = x[0] + break + if last_price-first_price>first_price*threshold: + #Threshold exceeded + return True + return False + except Exception as e: + self.broker.logger.log_this(f"Exception in check_slippage: {e}",1,self.pair) + return False + + + def check_status(self,open_orders: list) -> int: #Should I change the order? Check the SO first? + ''' + Main routine. It checks for closed orders and proceeds accordingly. + ''' + + #It does not even try if it receives an empty list or the worker is paused + if open_orders==[] or self.pause: + self.update_status(False) + return 0 + + #Checks if the orders are valid + if self.tp_order is None: + self.broker.logger.log_this("Take profit order is None",1,self.pair) + return 1 + if self.so is None: + #Attempt to reload the safety order from the status dict? + self.broker.logger.log_this("Safety order is None",1,self.pair) + self.so = self.broker.get_empty_order() + return 1 + if self.tp_order["id"]=="": + self.broker.logger.log_this(f"Take profit order missing. Stopping bot. Order ID: {self.tp_order['id']}",1,self.pair) + self.broker.cancel_order(self.so["id"],self.pair) + if self.config_dict["attempt_restart"]: + self.write_status_file(True) + self.restart = True + self.broker.logger.log_this("Raising restart flag: take profit order missing, trader will be restarted",0,self.pair) + else: + self.broker.logger.log_this("Take profit order missing. Trader restart disabled.",2,self.pair) + return 1 + + #Checks if the take profit order is filled + if self.tp_order["id"] not in open_orders: + tp_status = self.broker.get_order(self.tp_order["id"],self.pair) + if tp_status["status"]=="closed": + if tp_status["filled"]>0: + return self.take_profit_routine(tp_status) + self.broker.logger.log_this(f"Take profit order closed but not filled, 0 filled. Stopping bot. Order ID: {self.tp_order['id']}",1,self.pair) + #Cancelling safety order and stopping bot + self.broker.cancel_order(self.so["id"],self.pair) + if self.config_dict["attempt_restart"]: + self.write_status_file(True) + self.restart = True + self.broker.logger.log_this("Take profit order closed but not filled, trader will be restarted.",0,self.pair) + else: + self.broker.logger.log_this("Take profit order closed but not filled, trader restart disabled.",1,self.pair) + return 1 + elif tp_status["status"]=="canceled": + #TODO: Here, if the safety order is still open, we could resend the tp order. + if self.config_dict["attempt_restart"]: + self.broker.logger.log_this("Take profit order canceled. Restarting the bot.",1,self.pair) + self.write_status_file(True) + self.restart = True + else: + self.broker.logger.log_this("Take profit order canceled. Trader restart disabled.",1,self.pair) + return 1 + elif tp_status["status"]=="": + self.broker.logger.log_this(f"Take profit order search returned empty order. Order ID: {tp_status['id']}",1,self.pair) + return 1 + + # Check if safety order is filled + if self.so["id"] not in open_orders and self.safety_order_index<=self.config_dict["no_of_safety_orders"]: + #so_status = self.so + #if self.so["id"]!="": + so_status = self.broker.get_order(self.so["id"],self.pair) + tp_order_status = self.broker.get_order(self.tp_order["id"],self.pair) + + #Now we check 2 things: + #1. That the prior safety order status is indeed closed (or canceled) + #2. That the take profit order is still opened (if not, the deal must have closed, both orders closing is quite common in high variance scenarios) + if so_status["status"] in ["closed", "canceled", ""] and tp_order_status["status"]=="open": + #Switch to short if all safety orders are sent and autoswitch is enabled. + #May get into trouble if the trader is short of funds + if not self.is_short and self.safety_order_index==self.config_dict["no_of_safety_orders"] and self.config_dict["autoswitch"]: + self.switch_to_short() + self.write_status_file(True) + self.restart = True + return 0 + a = self.new_so_routine(so_status,self.safety_order_index float: + ''' + Returns the correct take profit percentage, according to the strategy (config_dict["tp_mode"]): + 0. Fixed percentage + 1. Variable percentage (+0.5% to -0.5% of the fixed percentage) + 2. Custom percentage table + 3. Linear percentage table + ''' + tp_level = 1 + if self.is_short or self.config_dict["tp_mode"]==0: #Fixed take profit percentage + tp_level = self.config_dict["tp_level"] + elif self.config_dict["tp_mode"]==1: #Variable percentage + limit = self.config_dict["no_of_safety_orders"]/3 + if order_index<=1: + tp_level = self.config_dict["tp_level"]+0.005 + elif order_index<=limit: + tp_level = self.config_dict["tp_level"] + elif limit<=order_index<=limit*2: + tp_level = self.config_dict["tp_level"]-0.0025 + else: + tp_level = self.config_dict["tp_level"]-0.005 + elif self.config_dict["tp_mode"]==2: + if ["tp_table"] in self.config_dict: + if len(self.config_dict["tp_table"])>=order_index: + tp_level = self.config_dict["tp_table"][order_index] #Custom percentage table + tp_level = self.config_dict["tp_table"][-1] + tp_level = self.config_dict["tp_level"] + elif self.config_dict["tp_mode"]==3: #Linear percentage table + profit_table = self.linear_space(self.config_dict["tp_level"]+0.005,self.config_dict["tp_level"]-0.005,self.config_dict["no_of_safety_orders"]) + tp_level = profit_table[-1] + if order_index str: + ''' + Returns a D:HH:MM:SS representation of total_seconds + ''' + return f"{int(total_seconds / 86400)}:" + '%02d:%02d:%02d' % (int(total_seconds % 86400 / 3600), int(total_seconds % 3600 / 60), int(total_seconds % 60)) + + + def adjust_base(self): + time.sleep(self.broker.get_wait_time()) + new_balance = self.broker.get_coins_balance() + if bool(new_balance): + self.broker.logger.log_this(f"Adjusting base amount to {new_balance['free'][self.base]}, total balance: {new_balance['total'][self.base]}",1,self.pair) + return new_balance["free"][self.base] + return None + + + def send_new_tp_order(self, multiplier: float = 1) -> int: + ''' + Calculates the correct take profit price and sends the order to the exchange + ''' + tries = self.broker.get_retries() + while tries>0: + if self.total_amount_of_base==0: + self.broker.logger.log_this("Amount of base equals 0, can't send take profit order",1,self.pair) + return 1 + if self.is_short: + self.take_profit_price = self.total_amount_of_quote/self.total_amount_of_base*(1-(self.get_tp_level(self.safety_order_index,multiplier)-1)) + self.tp_order = self.broker.new_limit_order(self.pair,self.total_amount_of_base,"buy",self.take_profit_price) + else: + self.take_profit_price = self.total_amount_of_quote/self.total_amount_of_base*self.get_tp_level(self.safety_order_index,multiplier) + self.tp_order = self.broker.new_limit_order(self.pair,self.total_amount_of_base,"sell",self.take_profit_price) + if self.tp_order==1: #This means that there was a miscalculation of base currency amount, let's correct it. + if self.is_short: #If in short mode, we don't recalculate anything. + return 1 + adjusted = self.adjust_base() + if adjusted is not None: + self.total_amount_of_base = adjusted + self.tp_order = None #Just to be able to iterate + if self.tp_order not in [None,self.broker.get_empty_order()]: + return 0 + tries-=1 + time.sleep(self.broker.get_wait_time()) + self.broker.logger.log_this("Problems sending take profit order",1,self.pair) + return 1 + + + def profit_to_file(self, amount: float, orderid: str) -> int: + ''' + Saves the profit to the corresponding profit file + DEPRECATED. Use profit_to_db instead. + ''' + try: + with open(self.profit_filename,"a") as profit_file: + profit_writer = csv.writer(profit_file, delimiter=",") + profit_writer.writerow([time.strftime("%Y-%m-%d"), amount, orderid]) + except Exception as e: + self.broker.logger.log_this(f"Exception in profit_to_file: {e}",1,self.pair) + return 0 + + + def profit_to_db(self, amount: float, orderid: str, write_deal_order_history: bool = False) -> int: + ''' + Saves the profit to the db in the format (pair,timestamp,profit,exchange_name,order_id,order_history) + ''' + retries = self.broker.get_retries() + while retries>0: + try: + order_history = json.dumps(self.status_dict["deal_order_history"]) if write_deal_order_history else "" + dataset = (time.time(),self.pair,amount,self.broker.get_exchange_name(),str(orderid),order_history) + return self.broker.write_profit_to_db(dataset) + except Exception as e: + self.broker.logger.log_this(f"Exception while writing profit: {e}",1,self.pair) + retries-=1 + time.sleep(self.broker.get_wait_time()) + return 1 + + + def send_new_safety_order(self, size: float) -> int: + ''' + Sends a new safety order to the exchange + ''' + so_size = self.gib_so_size(size,self.safety_order_index+1,self.config_dict["safety_order_scale"]) #safety_order_scale: safety order growth factor + if self.is_short: + new_order = self.broker.new_limit_order(self.pair,so_size,"sell",self.safety_price_table[self.safety_order_index+1]) + else: + new_order = self.broker.new_limit_order(self.pair,so_size/self.safety_price_table[self.safety_order_index+1],"buy",self.safety_price_table[self.safety_order_index+1]) + if new_order==1: + self.so = self.broker.get_empty_order() + self.broker.logger.log_this("Not enough balance to send a new safety order",1,self.pair) + #elif new_order in [None,self.broker.get_empty_order()] #MAYUBE THIS CONDITIONAL IS BETTER + elif new_order is None: + self.so = None + return 1 + else: + self.so = new_order + self.safety_order_index+=1 + return 0 + + + def telegram_bot_sendprofit(self,profit,order,base_profit=0) -> int: + ''' + Sends the Telegram notification when profit is met + ''' + try: + extra = f" and {round(base_profit,6)} {self.base}" if base_profit>0 else "" + message = f"{self.pair} closed a {'short' if self.is_short else 'long'} trade.\nProfit: {round(profit,6)} {self.quote}{extra}\nSafety orders triggered: {self.safety_order_index-1}\nTake profit price: {order['price']} {self.quote}\nTrade size: {round(order['cost'],2)} {self.quote}\nDeal uptime: {self.seconds_to_time(self.status_dict['deal_uptime'])}\nOrder ID: {order['id']}\nExchange: {self.broker.get_exchange_name().capitalize()}\n" + self.broker.logger.send_tg_message(message) + return 0 + except Exception as e: + self.broker.logger.log_this(f"Exception in telegram_bot_sendprofit: {e}",1,self.pair) + return 1 + + + def gib_so_size(self, starting_order_size: float, so_number: int, scaling_factor: float) -> float: + ''' + Returns the correct safety order size depending on the number + Scaling factor example: 5% = 0.0105 + ''' + order_size = starting_order_size + for _ in range(so_number): + order_size = order_size*scaling_factor*100 + return order_size + + + def clip_value(self,value,lower_limit,upper_limit): + ''' + Clips a value to a given range + ''' + + if valueupper_limit: + return upper_limit + + return value + + + def calculate_safety_prices(self, start_price: float, no_of_safety_orders: int, safety_order_deviance: float) -> list: + ''' + Generates a table of safety order's prices + ''' + safety_price_table = [start_price] + if self.config_dict["dynamic_so_deviance"]:# and no_of_safety_orders>=30: + #if self.config_dict["dynamic_so_deviance"] and not self.is_short: + #bias should be a real number between -1 and 1 (1>n>-1, NOT 1=>n>=-1) + #If bias -> 1, more space between the first orders, if -> -1, more space between the last orders, if 0, no change.. + if "bias" in self.config_dict: + deviance_factor = safety_order_deviance*self.clip_value(self.config_dict["bias"],-.99,.99) + so_deviance_table = self.linear_space(safety_order_deviance+deviance_factor,safety_order_deviance-deviance_factor,no_of_safety_orders) + else: + #Old way of calculating deviance + so_deviance_table = self.linear_space(safety_order_deviance-self.config_dict["dsd_range"],safety_order_deviance+self.config_dict["dsd_range"],no_of_safety_orders) + so_deviance_table.extend([so_deviance_table[-1]]*2) #This extra entries are needed in the next for loop + else: + so_deviance_table = [safety_order_deviance]*(no_of_safety_orders+2) + + multiplier = -1 if self.is_short else 1 + for y in range(1, no_of_safety_orders+2): #+2 instead of the expected +1 because of a bug when updating the status dict. It could be any value, if we add SOs the table is recalculated anyway + safety_price_table.append(safety_price_table[-1]-multiplier*(safety_price_table[0]*so_deviance_table[y-1]/100)) + + return safety_price_table + + + def linear_space(self, start: float, stop: float, amount: int) -> list: + ''' + Numpy's linspace local implementation. + Implemented here because: + - This is the only piece of code needed from Numpy + - Only executed when calculating the safety order table, so there's no need for outstanding performance. + ''' + result = [start] + if amount in [0,1]: + return result + step = (start-stop)/(amount-1) + for _ in range(1,amount): + result.append(result[-1]-step) + return result + + + def switch_quote_currency(self, new_quote: str) -> int: + ''' + Replaces the open orders with new updated orders, updates the config file and the status. + ''' + #First let's check if the market exists + market = self.broker.fetch_market(f"{self.base}/{new_quote}") + if market is None: + self.broker.logger.log_this("Market might not exist",1,self.pair) + return 1 + if "active" in market and not market["active"]: + self.broker.logger.log_this("Market is closed",1,self.pair) + return 1 + if self.tp_order is None: + self.broker.logger.log_this("Take profit order is None",1,self.pair) + return 1 + if self.so is None: + self.broker.logger.log_this("Safety order is None",1,self.pair) + return 1 + + #Replace the current take profit order with a new one with new quote currency + self.broker.logger.log_this("Replacing take profit order",2,self.pair) + self.tp_order = self.quote_currency_replace_order(self.tp_order,new_quote) + if self.tp_order==self.broker.get_empty_order(): + return 1 + + #Replace the current safety order (if any) with a new one with the new quote currency + if self.so!=self.broker.get_empty_order(): + self.broker.logger.log_this("Replacing safety order",2,self.pair) + self.so = self.quote_currency_replace_order(self.so,new_quote) + if self.so==self.broker.get_empty_order(): + return 1 + + #Calls switch_quote_currency_config + self.broker.logger.log_this("Modifying config file",2,self.pair) + self.quote_currency_switch_configs(new_quote) + + #Updates status_dict + self.broker.logger.log_this("Updating status file",2,self.pair) + self.update_status(True) + + #Done + self.broker.logger.log_this("Quote swap successful",2,self.pair) + return 0 + + + def quote_currency_replace_order(self, old_order: dict, new_quote: str) -> dict: + ''' + Cancels the order and returns the new updated order + ''' + #Cancels the old order + if self.broker.cancel_order(old_order["id"],self.pair)==1: + self.broker.logger.log_this(f"Can't cancel old order {old_order['id']}",1,self.pair) + return self.broker.get_empty_order() + + #Sends the new order + return self.broker.new_limit_order(f"{self.base}/{new_quote}",old_order["amount"],old_order["side"],old_order["price"]) + + + def quote_currency_switch_configs(self, new_quote: str) -> int: + ''' + Updates the broker config file, changes all the variables and writes the new pair config file + ''' + #Change broker config file + self.broker.remove_pair_from_config(f"{self.base}{self.quote}") + self.broker.add_pair_to_config(f"{self.base}{new_quote}") + + if self.broker.rewrite_config_file()==1: + #Error writing broker config file, undoing changes + self.broker.logger.log_this("Error writing new broker config file",1,self.pair) + self.quote_currency_undo_changes(new_quote,self.quote,False) + return 1 + + #Change pair-related variables + old_quote = self.quote + self.quote = new_quote + self.config_dict["pair"] = f"{self.base}/{self.quote}" + self.profit_filename = f"profits/{self.base}{self.quote}.profits" + self.log_filename = f"logs/{self.base}{self.quote}.log" + self.pair = self.config_dict["pair"] + + #If there is an old_long file, also copy it + if self.is_short and "old_long" in self.status_dict: + try: + with open(f"status/{self.base}{self.quote}.oldlong","w") as c: + c.write(json.dumps(self.status_dict["old_long"], indent=4)) + except Exception as e: + self.broker.logger.log_this(f"Exception while writing new old_long file: {e}",1,self.pair) + + #Write the new config file + try: + with open(f"configs/{self.base}{self.quote}.json","w") as c: + c.write(json.dumps(self.config_dict, indent=4)) + except Exception as e: + self.broker.logger.log_this(f"Exception while writing new trader config file: {e}",1,self.pair) + #Undoing changes + self.quote_currency_undo_changes(new_quote,old_quote,True) + return 1 + + #Done + return 0 + + + def quote_currency_undo_changes(self, new_quote: str, old_quote: str, write_broker_file: bool = False) -> int: + ''' + Revert changes made by switch_quote_currency() + ''' + #Switching variables + self.quote = old_quote + self.broker.remove_pair_from_config(f"{self.base}{new_quote}") + self.broker.add_pair_to_config(f"{self.base}{self.quote}") + + self.config_dict["pair"] = f"{self.base}/{self.quote}" + self.profit_filename = f"profits/{self.base}{self.quote}.profits" + self.log_filename = f"logs/{self.base}{self.quote}.log" + self.pair = self.config_dict["pair"] + + #Writing config file + if write_broker_file and self.broker.rewrite_config_file()==1: + self.broker.logger.log_this("Error in quote_currency_undo_changed: error writing new broker config file",1,self.pair) + + #Done + return 0 + + + def generate_status_strings(self) -> str: + ''' + Returns the status string properly formatted for screen output + ''' + yellow = "\033[0;33;40m" + green = "\033[0;32;40m" + red = "\033[0;31;40m" + blue = "\033[0;34;40m" + cyan = "\033[0;36;40m" + bright_white = "\033[0;97;40m" + bright_green = "\033[0;92;40m" + white = "\033[0;37;40m" + + def draw_line(price,min_value,max_value,break_even): + ''' + It draws the progress bar according to the inputs: + * If the price is bigger or equal to the break even price, the line's color is green + * If it's lower, the line's color is red + * All the way to the left, new safety order + * All the way to the right, profit! + ''' + try: + value = int(((price-min_value)/(max_value-min_value))*80) + except Exception as e: + self.broker.logger.log_this(f"{e}") + value = 1 + if min_value=break_even else red + else: + color = red if price>=break_even else green + return f"{color}{'='*value}{white}{'='*max(0,(80-value))}"[:100] + + decimals = 11 + low_percentage = 1 + mid_percentage = 10 + high_percentage = 20 + + safety_order_string = f"{self.status_dict['so_amount']-1}/{self.config_dict['no_of_safety_orders']}".rjust(5) + + #Check if necessary + low_price = 0 + mid_price = 0 + high_price = 0 + if self.status_dict["next_so_price"] is not None: + low_price = self.status_dict["next_so_price"] + if self.status_dict["price"] is not None: + mid_price = self.status_dict["price"] + if self.status_dict["take_profit_price"] is not None: + high_price = self.status_dict["take_profit_price"] + + low_boundary = '{:.20f}'.format(low_price)[:decimals].center(decimals) + mid_boundary = '{:.20f}'.format(mid_price)[:decimals].center(decimals) + high_boundary = '{:.20f}'.format(high_price)[:decimals].center(decimals) + + percentage_to_profit = 100 + pct_to_profit_str = "XX.XX" + if self.status_dict not in [0,None] and self.status_dict["price"]!=0: + diff = abs(self.status_dict["take_profit_price"]-self.status_dict["price"]) + percentage_to_profit = diff/self.status_dict["price"]*100 + + #Formatting (on-screen percentage not longer than 4 digits) + pct_to_profit_str = "{:.2f}".format(percentage_to_profit) + if len(pct_to_profit_str)==4: + pct_to_profit_str = f" {pct_to_profit_str}" + elif len(pct_to_profit_str)==6: + pct_to_profit_str = pct_to_profit_str[:5] + + line3 = "" + if self.total_amount_of_base!=0: + line3 = draw_line(self.status_dict["price"],self.status_dict["next_so_price"],self.status_dict["take_profit_price"],self.total_amount_of_quote/self.total_amount_of_base) + p = "*PAUSED*" if self.pause==True else "" + price_color = white + target_price_color = green + pair_color = cyan + if self.is_short: + price_color = white + pair_color = yellow + if "old_long" in self.status_dict: + if self.status_dict["price"]>self.status_dict["old_long"]["tp_price"]: + price_color = bright_green + if self.status_dict["take_profit_price"]>self.status_dict["old_long"]["tp_price"]: + target_price_color = bright_green + + #Set percentage's color + pct_color = white + if percentage_to_profitmid_percentage: + pct_color = yellow + if percentage_to_profit>high_percentage: + pct_color = red + + prices = f"{red}{low_boundary}{white}|{price_color}{mid_boundary}{white}|{target_price_color}{high_boundary}{white}|{pct_color}{pct_to_profit_str}%{white}" + line1 = f"{p}{pair_color}{self.pair.center(13)}{white}| {safety_order_string} |{prices}| Uptime: {self.seconds_to_time(self.status_dict['deal_uptime'])}" + if self.config_dict["autoswitch"]: + line1 = f"{line1} | AUTO" + if self.is_short and "old_long" in self.status_dict: + try: + percentage_to_switch = (self.status_dict["old_long"]["tp_price"]-self.status_dict["price"])*100/self.status_dict["price"] + #line1 = f"{line1} {round(percentage_to_switch,2)}%" + multiplier = int(percentage_to_switch/100)+1 + if multiplier>1: + line1 = f"{line1}x{multiplier}" + except ZeroDivisionError as e: + print(e) + if "stop_time" in self.config_dict and time.time()<=int(self.config_dict["stop_time"]): + line1 = f"{line1} | PROGRAMMED LAST DEAL" + if self.stop_when_profit==True: + line1 = f"{line1} | LAST DEAL" + + return f"{white}{line1}\n{line3}{white}" + + + def load_imported_trader(self) -> int: + ''' + Loads status dictionary, orders and sets up variables + ''' + #Load status dict + try: + with open(f"status/{self.base}{self.quote}.status") as sd: + self.status_dict = json.load(sd) + except Exception as e: + self.broker.logger.log_this(f"Exception: Couldn't load status dict. Aborting {e}",1,self.pair) + self.quit = True + return 1 + + self.status_dict["pause_reason"] = "Importing trader" + + #Load variables + self.total_amount_of_quote = self.status_dict["quote_spent"] + self.total_amount_of_base = self.status_dict["base_bought"] + self.safety_order_index = self.status_dict["so_amount"] + self.config_dict["no_of_safety_orders"] = self.status_dict["no_of_safety_orders"] #If this is not loaded from status_dict, it will ignore if safety orders were added at runtime + self.take_profit_price = self.status_dict["take_profit_price"] + self.safety_price_table = self.status_dict["safety_price_table"] + self.fees_paid_in_base = self.status_dict["fees_paid_in_base"] + self.fees_paid_in_quote = self.status_dict["fees_paid_in_quote"] + self.start_price = self.status_dict["start_price"] + self.start_time = self.status_dict["start_time"] + self.deal_start_time = self.status_dict["deal_start_time"] + self.stop_when_profit = self.status_dict["stop_when_profit"] + if "deal_order_history" not in self.status_dict: #No longer needed? + self.status_dict["deal_order_history"] = [] + + #Load take profit order + self.tp_order = self.broker.get_order(self.status_dict["tp_order_id"],self.pair) + if self.tp_order==self.broker.get_empty_order(): + self.broker.logger.log_this("Couldn't load take profit order (broker returned empty order). Aborting.",1,self.pair) + self.quit = True + return 1 + + #Load safety order + self.so = self.broker.get_order(self.status_dict["so_order_id"],self.pair) + if self.so==self.broker.get_empty_order() and self.safety_order_index1: + print(x,pairs[x]) + orders = exchange.fetch_open_orders(symbol=x) + for order in orders: + if order["side"]=="buy": + print(order) + found = True +if not found: + print("No duplicates found") \ No newline at end of file diff --git a/utils/commander.py b/utils/commander.py new file mode 100644 index 0000000..13f8d9a --- /dev/null +++ b/utils/commander.py @@ -0,0 +1,591 @@ +import requests +import sys +import json +import credentials + +try: + if sys.argv[1]=="--testnet": + is_testnet = True + string_to_add = "TESTNET " + api_key = credentials.get_credentials("testnet_api_key")["key"] + base_url = credentials.get_url("testnet") #type: ignore + exchanges = {"Binance":"/binance"} + elif sys.argv[1]=="--mainnet": + is_testnet = False + string_to_add = "MAINNET " + api_key = credentials.get_credentials("mainnet_api_key")["key"] + base_url = credentials.get_url("mainnet") #type: ignore + exchanges = {"binance":"/binance", "gate.io":"/gateio", "kucoin":"/kucoin", "okx":"/okex"} + else: + print(f"Unrecognized parameter {sys.argv[1]}") + sys.exit() +except Exception as e: + print(e) + sys.exit() + +headers = {'X-API-KEY': api_key} + +command_list = f'''{string_to_add}COMMANDS: + +INSTANCE +1) global_status 2) missing_pairs 3) server_time +4) trader_time 5) toggle_restart 6) toggle_telegram +7) mod_global_tp_level 8) global_last_call 9) edit_loop_wait_time +10) edit_call_wait_time 11) reload_markets 12) fetch_full_log +13) paused_traders 14) fetch_log + +TRADERS +51) worker_status 52) get_all_worker_status +53) add_pair 54) remove_pair 55) restart_pair +56) import_pair 57) switch_to_short 58) switch_to_long +59) load_old_long 60) add_so 61) add_quote +62) mod_tp_level 63) last_call 64) deferred_last_call +65) toggle_pause 66) toggle_cleanup 67) toggle_autoswitch +68) toggle_check_old_long_price 69) switch_quote_currency +70) reload_safety_order 71) view_old_long + +98) Change broker 99) Exit +''' + +def validate_pair(trading_pair): + return "/" in trading_pair and len(trading_pair)>3 + +def validate_float_or_int(number): + ''' + Validates if the number can be interpreted as a float or an int + ''' + try: + number = str(float(number)) + return True + except Exception: + return False + +def validate_int(number): + ''' + Validates if the number can be interpreted as an integer + ''' + try: + new_number = int(number) + if str(new_number) == str(number): + return True + return False + except Exception: + return False + +def select_exchange(exchanges): + ''' + Selects the exchange to use + ''' + + selection = input("Enter exchange: (Binance, Gate.io, KuCoin, OKX) ").lower() + for item in exchanges: + if selection in item.lower(): + return item + print("Invalid input") + sys.exit() + +if __name__=="__main__": + + if len(exchanges)==1: + selection = list(exchanges.keys())[0] + else: + selection = select_exchange(exchanges) + #selection = input("Enter exchange: (Binance, Gate.io, KuCoin, OKX) ").lower() + #for item in exchanges: + # if selection in item.lower(): + # selection = item + # break + #print("Invalid input") + #sys.exit() + port = exchanges[selection] + + + print("DCAv2 COMMANDER") + if not is_testnet: + print("WARNING: RUNNING ON MAINNET") + + while True: + print("="*80) + print(f"Exchange: {selection}") + print(command_list) + + + #When entering the command, it shows a brief description and requests for parameters. + command = input("Your input: ") + + try: + command = int(command) + except Exception as e: + print(e) + + if command==99: + print("Goodbye") + sys.exit() + + elif command==98: + #while True: + # selection = input("Enter exchange: (Binance, Gate.io, KuCoin, OKX) ").lower() + # if selection not in exchanges: + # print("Invalid input") + # port = exchanges[selection] + # break + selection = select_exchange(exchanges) + port = exchanges[selection] + print(f"New exchange selected: {selection}") + + + ###################### + ###### INSTANCE ###### + ###################### + + elif command==1: + print("global_status returns a dictionary of the global status of the instance") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/global_status" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==2: + print("missing_pairs returns a list of pairs that are in the config file of the instance") + print("but are not running.") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/missing_pairs" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==3: + print("server_time returns the linux time of the server") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/server_time" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==4: + print("trader_time return the last time of the traders was active") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/trader_time" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==5: + print("toggle_restart controls if the instance will attempt to restart failed traders or not") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/toggle_restart" + print(json.loads(requests.post(url, headers=headers).content)) + input("Press ENTER to continue ") + + elif command==6: + print("toggle_telegram turns on or off the Telegram notifications") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/toggle_telegram" + print(json.loads(requests.post(url, headers=headers).content)) + input("Press ENTER to continue ") + + elif command==7: + print("mod_global_tp_level modifies the percentage of profit of all the traders") + print("Example: 1.02 is equal to 2% profit") + new_profit_level = input("Desired profit level: ") + if not validate_float_or_int(new_profit_level): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/mod_global_tp_level" + parameters = {"amount": new_profit_level} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==8: + print("global_last_call signals all traders to cease operation when profit is reached") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/global_last_call" + print(json.loads(requests.post(url, headers=headers).content)) + input("Press ENTER to continue ") + + elif command==9: + print("edit_loop_wait_time modifies the pause the instance takes after processing the open orders") + print("instance fetch the orders -> instance sends the orders to the traders ->") + print("instance waits for the traders to complete their tasks -> instance waits seconds") + print("The input value can be an integer or a float") + new_wait_time = input("Desired wait time: ") + if not validate_float_or_int(new_wait_time): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/edit_loop_wait_time" + parameters = {"wait_time": new_wait_time} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==10: + print("edit_call_wait_time modifies the pause that the traders take between some API calls") + print("This aims to reduce the load on the API endpoints of the broker.") + print("The input value can be an integer or a float") + new_wait_time = input("Desired call wait time: ") + if not validate_float_or_int(new_wait_time): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/edit_call_wait_time" + parameters = {"wait_time": new_wait_time} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==11: + print("reload_markets forces CCXT to renew all the market information") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/reload_markets" + print(json.loads(requests.post(url, headers=headers).content)) + input("Press ENTER to continue ") + + elif command==12: + print("fetch_full_log displays the log of an instance.") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}/statistics_server/fetch_full_log?exchange_name={port[1:]}&width={100}" + for item in json.loads(requests.get(url, headers=headers).content)["line"]: + print(item) + input("Press ENTER to continue ") + + elif command==13: + print("paused_traders returns a list of paused traders.") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/paused_traders" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==14: + print("fetch_log displays the last n log entries of an instance.") + amount = input("Amount of lines? ") + if not validate_float_or_int(amount): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}/statistics_server/fetch_log?exchange_name={port[1:]}&width={100}&amount={amount}" + for item in json.loads(requests.get(url, headers=headers).content)["line"]: + print(item) + input("Press ENTER to continue ") + + + ###################### + ####### TRADER ####### + ###################### + + elif command==51: + print("worker_status return the status dictionary of the trader") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + base,quote = trading_pair.split("/") + url = f"{base_url}{port}/worker_status?base={base}"e={quote}" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==52: + print("get_all_worker_status returns a dictionary of all the status dictionaries of all active trader") + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/get_all_worker_status" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") + + elif command==53: + print("add_pair add a trader to the instance.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/add_pair" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==54: + print("remove_pair terminates a running trader from the instance.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/remove_pair" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==55: + print("restart_pair terminates and restarts a trader from the instance.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/restart_pair" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==56: + print("import_pair imports a trader to the instance.") + print("In order for the importing to be successful, a status file must exist in the status directory ") + print("and the take profit order must be open.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/import_pair" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==57: + print("switch_to_short changes the mode of operation of a trader from long mode (the default one) to short mode.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/switch_to_short" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==58: + print("switch_to_long changes the mode of operation of a trader from short mode to the default long mode") + print("It takes an extra parameter flag: 0 to ignore the profit calculation from the switch and 1 to do that calculation") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + calculation = input("Profit calculation? 0: ignore, 1: calculate ") + if not validate_pair(trading_pair): + print("The input is invalid") + break + if int(calculation) not in [0,1]: + print("The input for the calculation flag is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/switch_to_long" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote, + "calculate_profits": calculation} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==59: + print("load_old_long load to the status dictionary the contents of an old_long file") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/load_old_long" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==60: + print("add_so extends the safety order limit of a trader") + print("You can also use negative numbers to substract to that limit") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + amount = input("Amount of safety orders to add/remove: ") + if not validate_pair(trading_pair): + print("The input is invalid") + break + if not validate_int(amount): + print("The amount entered is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/add_so" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote, + "amount": amount} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==61: + print("add_quote adds a lump sum of quote currency to the deal.") + print("This is not possible to do on a short trader") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + amount = input("Amount of quote to add: ") + if not validate_pair(trading_pair): + print("The input is invalid") + break + if not validate_float_or_int(amount): + print("The amount entered is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/add_quote" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote, + "amount": amount} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==62: + print("mod_tp_level modifies the profit percentage of a trader") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + new_profit_level = input("Desired profit level: ") + if not validate_pair(trading_pair): + print("The input is invalid") + break + if not validate_float_or_int(new_profit_level): + print("The amount entered is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/mod_tp_level" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote, + "amount": new_profit_level} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==63: + print("last_call signals a trader to cease operation when profit is reached") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/last_call" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==64: + print("deferred_last_call signals a trader to cease operation when profit is reached after certain date") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + yyyymmdd = input("Input date (YYYYMMDD) ") + if not validate_pair(trading_pair): + print("The input is invalid") + break + if len(yyyymmdd)!=8: + print("Date format is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/deferred_last_call" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote, + "yyyymmdd": yyyymmdd} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==65: + print("toggle_pause pauses or unpauses a trader") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/toggle_pause" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==66: + print("toggle_cleanup enables or disables the cleanup routine of a trader") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/toggle_cleanup" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==67: + print("toggle_autoswitch enables or disables the automatic switch to long of a short trader once certain conditions are met.") + print("This is only valid in a short trader, of course.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/toggle_autoswitch" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==68: + print("toggle_check_old_long_price enables or disables the verification of the current price exceeding the old long price.") + print("This is only valid in a short trader, of course.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/toggle_check_old_long_price" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==69: + print("switch_quote_currency changes the quote currency of a running trader.") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + new_quote = input("Input new quote currency: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if len(new_quote)==0: + print("The quote currency is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/switch_quote_currency" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote, + "new_quote": new_quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==70: + print("reload_safety_order reloads the safety order to the reader using the order id present in the status dictionary") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + if input("Proceed? (Y/n) ") in ["Y","y",""]: + url = f"{base_url}{port}/reload_safety_order" + base,quote = trading_pair.split("/") + parameters = {"base": base, + "quote": quote} + print(json.loads(requests.post(url, headers=headers, json=parameters).content)) + input("Press ENTER to continue ") + + elif command==71: + print("Views the old_long information") + trading_pair = input("Input trader in the format BASE/QUOTE: ").upper() + if not validate_pair(trading_pair): + print("The input is invalid") + break + from_file = 0 if input("From file? (y/N) ") in ["N","n",""] else 1 + if input("Proceed? (Y/n) ") in ["Y","y",""]: + base,quote = trading_pair.split("/") + url = f"{base_url}{port}/view_old_long?base={base}"e={quote}&from_file={from_file}" + print(json.loads(requests.get(url,headers=headers).content)) + input("Press ENTER to continue ") diff --git a/utils/generate_keys.py b/utils/generate_keys.py new file mode 100644 index 0000000..f251249 --- /dev/null +++ b/utils/generate_keys.py @@ -0,0 +1,19 @@ +import uuid +import sqlite3 + + +users = ["user1", "user2", "user3", "user4"] +keys = [] + +for user in users: + keys.append([user,str(uuid.uuid4())]) + + +database_connection = sqlite3.connect("../api_credentials.db") +database_cursor = database_connection.cursor() +database_cursor.execute("CREATE TABLE IF NOT EXISTS credentials_table (user TEXT, key TEXT)") + +for pair in keys: + database_cursor.execute('INSERT INTO credentials_table VALUES(?, ?)', pair) +database_connection.commit() +database_connection.close() \ No newline at end of file diff --git a/utils/is_it_running.py b/utils/is_it_running.py new file mode 100755 index 0000000..6678b35 --- /dev/null +++ b/utils/is_it_running.py @@ -0,0 +1,13 @@ +import sys +import json + +exchanges = ["binance","okex","kucoin","gateio"] +pais = [] +for exchange in exchanges: + with open(f"../configs/{exchange}.json") as f: + pairs = json.load(f)["pairs"] + for pair in pairs: + if pair in [sys.argv[1],sys.argv[1].replace("/","")]: + print(f"{pair} is already running on {exchange}") + sys.exit(1) +print("Pair is not running") \ No newline at end of file diff --git a/utils/read_key_db.py b/utils/read_key_db.py new file mode 100644 index 0000000..2dc723f --- /dev/null +++ b/utils/read_key_db.py @@ -0,0 +1,15 @@ +import sqlite3 + +valid_keys = [] + +database_connection = sqlite3.connect("../api_credentials.db") +database_cursor = database_connection.cursor() + +database_cursor.execute("SELECT * FROM credentials_table") +data = database_cursor.fetchall() + + +for line in data: + valid_keys.append(line[1]) + +print(valid_keys) \ No newline at end of file diff --git a/utils/recreate_orders.py b/utils/recreate_orders.py new file mode 100644 index 0000000..b7ef628 --- /dev/null +++ b/utils/recreate_orders.py @@ -0,0 +1,83 @@ +''' +If the broker cancelled all your orders just because (THANK YOU KUCOIN), this script will resend all the orders and change the status dictionary values accordingly. +Note: since version 2024.07.08, both the safety and the take profit order are included in the status file; this will make resending the orders easier. +''' + + +import ccxt +import json +import sys + +def gib_so_size(starting_order_size: float, so_number: int, scaling_factor: float) -> float: + ''' + Returns the correct safety order size depending on the number + Scaling factor example: 5% = 0.0105 + ''' + order_size = starting_order_size + for _ in range(so_number): + order_size = order_size*scaling_factor*100 + return order_size + +with open(f"../configs/kucoin.json") as k: + config_file = json.load(k) + +exchange_class = getattr(ccxt, "kucoin") +exchange = exchange_class({ + "apiKey": config_file["key"], + "secret": config_file["secret"], + "password": config_file["password"], + "timeout": 30000, + "enableRateLimit": True +}) + +pair = sys.argv[1] + +with open(f"../status/{pair}.status") as f: + status_contents = json.loads(f.read()) + +with open(f"../configs/{pair}.json") as c: + config = json.loads(c.read()) + + +if not config["is_short"]: + buy_order_amount = gib_so_size(status_contents["order_size"],status_contents["so_amount"]+1,config["safety_order_scale"]) #Next safety order + buy_order_price = status_contents["next_so_price"] #Next safety order price + + sell_order_amount = status_contents["base_bought"] #Take profit order + sell_order_price = status_contents["take_profit_price"] #Take profit order price +else: + sell_order_amount = gib_so_size(status_contents["order_size"],status_contents["so_amount"]+1,config["safety_order_scale"]) #Next safety order + sell_order_price = status_contents["next_so_price"] #Next safety order price + + buy_order_amount = status_contents["base_bought"] #Take profit order + buy_order_price = status_contents["take_profit_price"] #Take profit order price + +print(f"Pair: {pair}") +pair_mode = "Short" if config["is_short"] else "Long" +print(f"Mode: {pair_mode}") +print(f"Buy order amount: {buy_order_amount}") +print(f"Buy order price: {buy_order_price}") +print(f"Sell order amount: {sell_order_amount}") +print(f"Sell order price: {sell_order_price}") + +input("Proceed? ") + +print("Sending buy order") +buy_order = exchange.create_order(config["pair"],"limit","buy",buy_order_amount,buy_order_price) +print(f"Buy order id: {buy_order['id']}") + +print("Sending sell order") +sell_order = exchange.create_order(config["pair"],"limit","sell",sell_order_amount,sell_order_price) +print(f"Sell order id: {sell_order['id']}") + +if not config["is_short"]: + status_contents["tp_order_id"] = sell_order["id"] + status_contents["so_order_id"] = buy_order["id"] +else: + status_contents["tp_order_id"] = buy_order["id"] + status_contents["so_order_id"] = sell_order["id"] + +json_object = json.dumps(status_contents, indent=4) +with open(f"../status/{pair}.status","w") as f: + f.write(json_object) + \ No newline at end of file diff --git a/utils/short_report.py b/utils/short_report.py new file mode 100644 index 0000000..d269eaf --- /dev/null +++ b/utils/short_report.py @@ -0,0 +1,53 @@ +import sqlite3 +import os +import sys +import json +import datetime + +pair = sys.argv[1].replace("/","") + +#Connect to db +connection = sqlite3.connect("../profits/profits_database.db") +cursor = connection.cursor() + +#Load old_long info +try: + with open(f"../status/{pair}.oldlong") as f: + old_long = json.load(f) +except Exception as e: + print(e) + print("No old_long file") + os._exit(0) + +#Get time of switch to unix time +old_date = old_long["datetime"][1:11] +linux_time = datetime.datetime.strptime(old_date,"%Y/%m/%d").timestamp() + +#Query database +query = f"""SELECT pair, SUM(amount) AS total_profit + FROM profits_table + WHERE timestamp >= '{linux_time}' + GROUP BY pair;""" +cursor.execute(query) +query_result = cursor.fetchall() + +#Calculate profits +total = 0 +for item in query_result: + if item[0].replace("/","")==pair: + total = item[1] + +print(f"Profits since switch ({old_date}): {round(total,2)}") +print(f"Profit needed to cover expenses: {round(old_long['quote_spent'],2)}") +print(f"Difference: {round(old_long['quote_spent']-total,2)}") + +try: + with open(f"../status/{pair}.status") as f: + status_file = json.load(f) +except Exception as e: + print(e) + print("No status file") + os._exit(0) + +print(f"Old long price: {old_long['tp_price']}") +print(f"Current price: {status_file['price']}") \ No newline at end of file diff --git a/utils/statistics_server_v3.py b/utils/statistics_server_v3.py new file mode 100644 index 0000000..a04d478 --- /dev/null +++ b/utils/statistics_server_v3.py @@ -0,0 +1,473 @@ +import sqlite3 +import sys +from flask import Flask, jsonify, request + + +''' +In case the certificate's permissions suddenly change (in auto renewal, for example), reset them this way: +/ sudo su +# chmod -R 755 /etc/letsencrypt/live/ +# chmod -R 755 /etc/letsencrypt/archive/ +# ll /etc/letsencrypt/ (to verify permissions) +''' + +cache_requests = False +if len(sys.argv)>1 and sys.argv[1]=="--cache_requests": + cache_requests = True + + +profits_database = "../profits/profits_database.db" +hashes_db = {"fetch_last_n_deals":0, + "fetch_last_n_deals_without_history":0, + "fetch_full_log":0, + "fetch_log":0, + "daily_totals":0, + "daily_totals_by_pair":0, + "monthly_totals":0, + "monthly_totals_by_pair":0, + "get_averages":0, + "total_profit":0, + "total_profit_by_pair":0} + + +def load_keys_from_db(file_name): + #valid_keys = [] + + connection = sqlite3.connect(file_name) + cursor = connection.cursor() + cursor.execute("SELECT * FROM credentials_table") + data = cursor.fetchall() + connection.close() + + valid_keys = [line[1] for line in data] + #for line in data: + # valid_keys.append(line[1]) + + return valid_keys + + +def query_total_profit(pair=None): + ''' + Returns total profit of the trading pair. + If no pair specified, returns the grand total of all pairs. + ''' + connection = sqlite3.connect(profits_database) + cursor = connection.cursor() + + if pair is None: + query = "SELECT SUM(amount) AS total_profit FROM profits_table" + cursor.execute(query) + connection.close() + query_result = cursor.fetchall() + return query_result[0][0] + else: + query = """SELECT pair, SUM(amount) AS total_profit + FROM profits_table + GROUP BY pair;""" + cursor.execute(query) + connection.close() + query_result = cursor.fetchall() + for item in query_result: + if item[0].replace("/","")==pair: + return item[1] + return 0 + + +def query_daily_totals(pair=None): + ''' + Returns a dictionary of daily totals of the trading pair. + If no pair specified, returns the totals of all pairs. + ''' + #Connect to db + connection = sqlite3.connect(profits_database) + cursor = connection.cursor() + + result = {} + + if pair is None: + query = """SELECT strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, + SUM(amount) AS total_profit + FROM profits_table + GROUP BY day_utc3;""" + cursor.execute(query) + query_result = cursor.fetchall() + connection.close() + for item in query_result: + result[item[0]] = item[1] + else: + query = """SELECT pair, strftime('%Y-%m-%d', timestamp, 'unixepoch', '-3 hours') AS day_utc3, + SUM(amount) AS total_profit + FROM profits_table + GROUP BY pair, day_utc3;""" + cursor.execute(query) + query_result = cursor.fetchall() + connection.close() + for item in query_result: + if item[0].replace("/","")==pair: + result[item[1]] = item[2] + return result + + +def query_monthly_totals(pair=None): + ''' + Returns a dictionary of monthly totals of the trading pair. + If no pair specified, returns the totals of all pairs. + ''' + #Connect to db + connection = sqlite3.connect(profits_database) + cursor = connection.cursor() + + result = {} + + if pair is None: + query = """SELECT strftime('%Y-%m', datetime(timestamp, 'unixepoch', '-3 hours')) AS month, + SUM(amount) AS total_profit + FROM profits_table + GROUP BY month;""" + cursor.execute(query) + query_result = cursor.fetchall() + connection.close() + for item in query_result: + result[item[0]] = item[1] + else: + query = f"""SELECT pair, strftime('%Y-%m', datetime(timestamp, 'unixepoch', '-3 hours')) AS month, + SUM(amount) AS total_profit + FROM profits_table + GROUP BY pair, month;""" + cursor.execute(query) + query_result = cursor.fetchall() + connection.close() + for item in query_result: + if item[0].replace("/","")==pair: + result[item[1]] = item[2] + return result + + +def last_n_deals(n): + ''' + Returns a list of the latest n deals + ''' + connection = sqlite3.connect(profits_database) + cursor = connection.cursor() + cursor.execute(f"SELECT * FROM profits_table ORDER BY timestamp DESC LIMIT {n}") + result = cursor.fetchall() + connection.close() + + return result + + +def last_n_deals_without_history(n): + ''' + Like last_n_deals, but without returning the order history. Useful in bandwidth-restricted scenarios. + ''' + + return [(row[0],row[1],row[2],row[3],row[4],"") for row in last_n_deals(n)] + + +def last_n_lines(file_name,width,amount=4,full_log=False): + + file_contents = [] + result = [] + + with open(file_name) as f: + file_contents = f.readlines() + + if full_log: + for line in file_contents: + result.append(line.strip()) + return result,len(file_contents) + + for line in file_contents[::-1][:amount]: + #trimmed = f"{line[0]}{line[12:21]}{line[23:]}".strip() + #result.append(trimmed[:width]) + trimmed = line.strip() + result.append(trimmed[:width]) + if len(trimmed)>width: + result.append(trimmed[width:width*2]) + return result[:amount],len(file_contents) + + +stats_api = Flask(__name__) + + +@stats_api.route("/clear_caches") +def clear_hashes(): + global hashes_db + + ''' + GET request + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + hashes_db = {"fetch_last_n_deals":0, + "fetch_last_n_deals_without_history":0, + "fetch_full_log":0, + "fetch_log":0, + "daily_totals":0, + "daily_totals_by_pair":0, + "monthly_totals":0, + "monthly_totals_by_pair":0, + "get_averages":0, + "total_profit":0, + "total_profit_by_pair":0} + return jsonify({"Done":0}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/fetch_last_n_deals") +def fetch_last_n_deals(): + ''' + GET request + Parameter: 'amount_of_deals' -> int + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + parameter = request.args.get("amount_of_deals") + response_value = last_n_deals(parameter) + if not cache_requests: + return jsonify({"last_deals": response_value}) + response_hash = hash(str({"last_deals": response_value})) + if hashes_db["fetch_last_n_deals"]!=response_hash: + hashes_db["fetch_last_n_deals"] = response_hash + return jsonify({"last_deals": response_value}) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return jsonify({"last_deals":""}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/fetch_last_n_deals_without_history") +def fetch_last_n_deals_without_history(): + ''' + GET request + Parameter: 'amount_of_deals' -> int + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + parameter = request.args.get("amount_of_deals") + #return jsonify({"last_deals": last_n_deals_without_history(parameter)}) + response_value = last_n_deals_without_history(parameter) + if not cache_requests: + return jsonify({"last_deals": response_value}) + response_hash = hash(str({"last_deals": response_value})) + if hashes_db["fetch_last_n_deals_without_history"]!=response_hash: + hashes_db["fetch_last_n_deals_without_history"] = response_hash + return jsonify({"last_deals": response_value}) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return jsonify({"last_deals":""}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/fetch_full_log") +def fetch_full_log(): + ''' + GET request + Parameters: 'exchange_name" -> string + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + exchange_name = request.args.get("exchange_name") + width = 0 + last_lines,amount_of_lines = last_n_lines(f"../logs/{exchange_name}.log",width,0,full_log=True) + if not cache_requests: + return jsonify({"line": last_lines, "amount_of_lines": amount_of_lines}) + response_hash = hash(str({"line": last_lines, "amount_of_lines": amount_of_lines})) + if hashes_db["fetch_full_log"]!=response_hash: + hashes_db["fetch_full_log"] = response_hash + return jsonify({"line": last_lines, "amount_of_lines": amount_of_lines}) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return {"line": [""]*width,"amount_of_lines": 0} + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/fetch_log") +def fetch_log(): + ''' + GET request + Parameters: 'exchange_name" -> string + 'width' -> int + 'amount' -> int + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + exchange_name = request.args.get("exchange_name") + width = int(request.args.get("width")) # type: ignore + amount = int(request.args.get("amount")) # type: ignore + last_lines,total_amount_of_lines = last_n_lines(f"../logs/{exchange_name}.log",width,amount) + if not cache_requests: + return jsonify({"line": last_lines, "amount_of_lines": total_amount_of_lines}) + response_hash = hash(str({"line": last_lines, "amount_of_lines": total_amount_of_lines})) + if hashes_db["fetch_log"]!=response_hash: + hashes_db["fetch_log"] = response_hash + return jsonify({"line": last_lines, "amount_of_lines": total_amount_of_lines}) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return {"line": [""]*10,"amount_of_lines": 0} + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/daily_totals") +def get_daily_totals(): + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + daily_totals = query_daily_totals() + if not cache_requests: + return jsonify(daily_totals) + response_hash = hash(str(daily_totals)) + if hashes_db["daily_totals"]!=response_hash: + hashes_db["daily_totals"] = response_hash + return jsonify(daily_totals) + return jsonify({"no_changes": True}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/daily_totals_by_pair") +def get_daily_totals_by_pair(): + ''' + GET request + Parameters: 'base' -> string + 'quote' -> string + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + base = request.args.get("base") + quote = request.args.get("quote") + daily_totals = query_daily_totals(f"{base}{quote}") + if not cache_requests: + return jsonify(daily_totals) + response_hash = hash(str(daily_totals)) + if hashes_db["daily_totals_by_pair"]!=response_hash: + hashes_db["daily_totals_by_pair"] = response_hash + return jsonify(daily_totals) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/monthly_totals") +def get_monthly_totals(): + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + monthly_totals = query_monthly_totals() + if not cache_requests: + return jsonify(monthly_totals) + response_hash = hash(str(monthly_totals)) + if hashes_db["monthly_totals"]!=response_hash: + hashes_db["monthly_totals"] = response_hash + return jsonify(monthly_totals) + return jsonify({"no_changes": True}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/monthly_totals_by_pair") +def get_monthly_totals_by_pair(): + ''' + GET request + Parameters: 'base' -> string + 'quote' -> string + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + base = request.args.get("base") + quote = request.args.get("quote") + monthly_totals = query_monthly_totals(f"{base}{quote}") + if not cache_requests: + return jsonify(monthly_totals) + response_hash = hash(str(monthly_totals)) + if hashes_db["monthly_totals_by_pair"]!=response_hash: + hashes_db["monthly_totals_by_pair"] = response_hash + return jsonify(monthly_totals) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/get_averages") +def get_averages(): + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + daily_totals = query_daily_totals() + val_30 = 0 + val_7 = 0 + acc_30 = [] + acc_7 = [] + for x in sorted(daily_totals): + acc_30.append(daily_totals[x]) + acc_7.append(daily_totals[x]) + length_30 = min(30,len(acc_30)) #Last 30 days + length_7 = min(7,len(acc_7)) #Last 7 days + for _ in range(length_30): + val_30 += acc_30.pop() + for _ in range(length_7): + val_7 += acc_7.pop() + if not cache_requests: + return jsonify({"30_day": val_30/length_30, "7_day": val_7/length_7}) + response_hash = hash(str({"30_day": val_30/length_30, "7_day": val_7/length_7})) + if hashes_db["get_averages"]!=response_hash: + hashes_db["get_averages"] = response_hash + return jsonify({"30_day": val_30/length_30, "7_day": val_7/length_7}) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/total_profit") +def total_profit(): + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + total = query_total_profit() + if not cache_requests: + return jsonify({"Total profit": total}) + response_hash = hash(str({"Total profit": total})) + if hashes_db["total_profit"]!=response_hash: + hashes_db["total_profit"] = response_hash + return jsonify({"Total profit": total}) + return jsonify({"no_changes": True}) + return jsonify({'Error': 'API key invalid'}), 401 + + +@stats_api.route("/total_profit_by_pair") +def total_profit_by_pair(): + ''' + GET request + Parameters: 'base' -> string + 'quote' -> string + ''' + if "X-API-KEY" in request.headers and request.headers.get("X-API-KEY") in valid_keys: + try: + base = request.args.get("base") + quote = request.args.get("quote") + total = query_total_profit(f"{base}{quote}") + if not cache_requests: + return jsonify({"Total profit": total}) + response_hash = hash(str({"Total profit": total})) + if hashes_db["total_profit_by_pair"]!=response_hash: + hashes_db["total_profit_by_pair"] = response_hash + return jsonify({"Total profit": total}) + return jsonify({"no_changes": True}) + except Exception as e: + print(e) + return jsonify({'Error': 'Halp'}) + return jsonify({'Error': 'API key invalid'}), 401 + + + +if __name__=="__main__": + + # Load valid keys from database + valid_keys = load_keys_from_db("api_credentials.db") + + #Waitress + #serve(stats_api,host="0.0.0.0",port=5010) + + #Dev server + stats_api.run(host="0.0.0.0",port=5010) + \ No newline at end of file diff --git a/utils/todo.txt b/utils/todo.txt new file mode 100755 index 0000000..6912677 --- /dev/null +++ b/utils/todo.txt @@ -0,0 +1,29 @@ +Mandatory: +========= +0. Mobile app. +1. Stats webpage. +2. Instead of giving a list of order_ids to each trader, give a list of the open orders and that's it (for easier future development, partial order fills for example) +3. Deploying script, both for testnet and for mainnet. + + +Would be nice to have: +===================== +0. Trader order: alphabetical; by uptime; by safety orders, by percentage_to_completion. (Although this may be more suitable for the web and mobile apps) +1. Local implementation of amount_to_precision, cost_to_precision and price_to_precision. (Unless the plan is to continue to use CCXT forever) +2. Instead of cancelling and resending the take profit order, you could just edit it (Kucoin only supports editing on high frequency orders) + + +Maybe it's a good idea?: +======================= +0. A fraction of the take profit order is taken as the first order of the next deal. + * Starting at the second or third safety order? + * This would address the issue of big spreads making the first order very expensive. + * Can be problematic in parabolic runs, unless the take-profit order is not x% above the buy order, but above the current price. +1. DUSTERS. If a trade is interrupted but the take profit sell order is still open, open another trader (different type, dust-trader, it remembers the original trade) + that follows that order; when it closes, it takes note of the profit. + a. The ability to import the dusters after an interruption will be key. + b. The duster uses the order id as duster id + c. Order on screen: BASE/QUOTE | order_id followed | current_price | deal_close_price | total_volume_on_close | pct_to_profit | uptime + d. In status bar: Total funds to be released. + e. Change main screen: x traders online | y dusters online + f. Since they only need to monitor if one order is filled and the data is already locally available, the extra API load will be negligible.