How to store API keys when algo trading?

I write Python scripts for clients to “algo trade” at currency/stock exchange websites. My clients are typically running my scripts on traditional personal desktop PC’s. Usually using these PC’s for web browsing activities as well. The environment is always linux; usually debian. In the industry; Python is quite standard for algo trading in this manner; both institutionally and privately.

However, I can’t help but see a flaw in the security model.

Each exchange has a slightly different authentication method; but in short there is:

USER INPUTS:  api['secret']       # private key from exchange  USER CONFIG FILE CONTAINS:  api['key']          # public key from exchange api['exchange']     # name of exchange; ie "binance" api['symbol']       # market pair symbol in format BTC:USD api['uri']          # the url up to .com/  FROM USER INPUTS SCRIPT BUILDS REQUEST SPECIFIC:  api['nonce']        # time.time() at beginning of request api['endpoint']     # path/to/server/resource api['url']          # uri + endpoint api['method']       # GET, POST, or DELETE api['params']       # dict with request specific parameters api['data']         # str with request specific parameters api['headers']      # authentication signature specific to the request 

These requests are of the sort:

POST BUY/SELL DELETE BUY/SELL (CANCEL) GET ACCOUNT BALANCES GET OPEN ORDERS WITHDRAW FUNDS 

Here are some examples of the authentication methods I’ve written to some currency exchanges; they’re all quite standard in the forex/stock/crypto trading industry. Mostly in the generalized format:

api["header"] = {"signature": HMAC(SHA256(the_request_parameters))} 

examples:

def signed_request(api, signal):     """     Remote procedure call for authenticated exchange operations     api         : dict with keys for building external request     signal      : multiprocessing completion relay     """     api = lookup_url(api)     api["data"] = ""     if api["exchange"] == "coinbase":         api["data"] = json_dumps(api["params"]) if api["params"] else ""         api["params"] = None         message = (             str(api["nonce"]) + api["method"] + api["endpoint"] + api["data"]         ).encode("ascii")         secret = b64decode(api["secret"])         signature = hmac.new(secret, message, hashlib.sha256).digest()         signature = b64encode(signature).decode("utf-8")         api["headers"] = {             "Content-Type": "Application/JSON",             "CB-ACCESS-SIGN": signature,             "CB-ACCESS-TIMESTAMP": str(api["nonce"]),             "CB-ACCESS-KEY": api["key"],             "CB-ACCESS-PASSPHRASE": api["passphrase"],         }     elif api["exchange"] == "poloniex":         api["params"]["nonce"] = int(api["nonce"] * 1000)         message = urlencode(api["params"]).encode("utf-8")         secret = api["secret"].encode("utf-8")         signature = hmac.new(secret, message, hashlib.sha512).hexdigest()         api["headers"] = {             "Content-Type": "application/x-www-form-urlencoded",             "Key": api["key"],             "Sign": signature,         }     elif api["exchange"] == "binance":         api["params"]["timestamp"] = int(api["nonce"] * 1000)         api["params"]["signature"] = signature         message = urlencode(api["params"].items()).encode("utf-8")         secret = bytes(api["secret"].encode("utf-8"))         signature = hmac.new(secret, message, hashlib.sha256).hexdigest()         api["headers"] = {"X-MBX-APIKEY": api["key"]}     elif api["exchange"] == "bittrex":         api["params"]["apikey"] = api["key"]         api["params"]["nonce"] = int(api["nonce"] * 1000)         message = api["url"] + api["endpoint"] + urlencode(api["params"])         message = bytearray(message, "ascii")         secret = bytearray(api["secret"], "ascii")         signature = hmac.new(secret, message, hashlib.sha512).hexdigest()         api["headers"] = {}     elif api["exchange"] == "kraken":         api["data"] = api["params"][:]         api["params"] = {}         data["nonce"] = int(1000 * api["nonce"])         api["endpoint"] = "/2.1.0/private/" + api["endpoint"]         message = (str(data["nonce"]) + urlencode(data)).encode("ascii")         message = api["endpoint"].encode("ascii") + hashlib.sha256(message).digest()         secret = b64decode(api["secret"])         signature = b64encode(hmac.new(secret, message, hashlib.sha512).digest())         api["headers"] = {             "User-Agent": "krakenex/2.1.0",             "API-Key": api["key"],             "API-Sign": signature,         }     elif api["exchange"] == "bitfinex":         nonce = str(int(api["nonce"] * 1000))         api["endpoint"] = path = "v2/auth/r/orders"         api["data"] = json.dumps(api["params"])         api["params"] = {}         message = ("/api/" + api["endpoint"] + nonce + api["data"]).encode("utf8")         secret = api["secret"].encode("utf8")         signature = hmac.new(secret, message, hashlib.sha384).hexdigest()         api["headers"] = {             "bfx-nonce": nonce,             "bfx-apikey": api["key"],             "bfx-signature": signature,             "content-type": "application/json",         }      url = api["url"] + api["endpoint"]     ret = requests.request(         method=api["method"],         url=url,         data=api["data"],         params=api["params"],         headers=api["headers"],     )     response = ret.json() 

My clients often ask me, where to store the api["secret"]. In config file? In an environmental variable? Enter it manually at every restart and store it on physically on paper? I have no good answer and anything suggested… I quickly facepalm.

I set out to write an app – in python – to store the API keys:

primary feature:

  • given url and key input:

  • writes secret to clipboard w/ xclip

  • auto clears clipboard in 10 seconds

security features:

  • reads/writes AES CBC encrypted password json to text file

  • salt is 16 byte shake256 and generated in crypto secure manner w/ os.urandom

  • new salt after every return to main menu and exit to prevent dictionary attack

  • master password stretched to 400 megabytes to prevent GPU/FPGA attack

  • master password hashed iteratively 1,000,000 times via traditional salted pbkdf sha512 to prevent brute attack

  • only 3rd party module is “pycryptodome”; raises exception if deprecated “pycrypto” is found

  • sudo system password required to edit the script

My thought was my users could use this app to secure their keys; and in a scripting cryptographic sense… I’ve dotted my i’s and crossed my t’s… when they’re stored they’re stored. Period. I’m quite convinced you cannot break my encryption scheme.

You can find my app here or by googling: litepresence/CypherVault

But I still facepalm. I opened a reddit/r/security thread here to discuss my app and my facepalm was quickly validated.

In the end of things… no matter how I handle the api[“secret”] it ends up on RAM which can be dumped by malware and uploaded to an attacker.

When the user enters the secret to be encrypted… its on RAM. When the script decrypts the secret to sign a transaction… its on RAM.

Then hocus pocus… “my script” caused somebody to lose money because it was not “secure”; I have a sense of responsibility.

How can this be avoided? How can one securely store exchange API secrets for composing financial transactions – by bot – in scripting languages, on desktop machines, without at some point exposing them to your RAM, and potentially worse… your SWAP?