Password storage best practices
We have a system (.NET application + sql server) where in addition to regular user passwords we need to store passwords that are used by the application processes to access various other subsystems or 3rd party services, for example sftp or external interfaces. What are the best practices from security standpoint, dos and don'ts, for storing passwords? We can't "simply" store hashes like we do for user passwords because the passwords ultimately need to be passed to other systems without user intervention. Is storing them in sql server with some sort of encryption (column, table) with additional user restrictions adequate? If not, what are other options we should be looking at?
Correct assumption, if you have to pass them via cleartext, you have to store them either in cleartext, or reversible to cleartext. Best practice would be to store them encrypted, but the application then has to decrypt the passwords. Access to the stored passwords should be restricted via database permissions to the application only, and in the application via user privileges.
The issue is then where to store the secret key for the encryption. If it is stored in the database, a leaked dump would contain both the encrypted passwords and the key to decrypt them. If you store them in a config file on the application side you have to backup these configurations too. If you lose them, you lose the access, or at the very least have to re-set the passwords in the application with a new encryption key.
Honestly, the best way to do this would probably be via a dedicated software, i.e. a password manager with an API for the application. This way the stored passwords stay up to date (since everyone should use the password manager) and the application has access via the API and you don't have to roll out your own crypto.