“Our entire AWS account is gone. The call center is down, we can’t log in - it’s like it never existed! How do we get it back?” 

A multimillion-dollar services provider discovers a nightmare situation. They had terminated an employee, and in retaliation, that employee shut down all of their AWS environment service capabilities. The client was completely dead in the water; locked out of accessing their own account, unable to determine exactly what was going on and with no ability to restore their AWS capabilities to operational status. 

After some digging, it was discovered that the former employee had disabled all other AWS users associated with the account, then changed the email address and password associated with the root AWS account. This locked our client completely out of the account, and since everything was done with authorized root-level credentials, AWS couldn’t reverse the damage. 

Their only option was to get the login from the former (terminated) employee. But communication rapidly devolved to the point that law enforcement showed up at his door before the day was over. Once the account was back in the right hands, the service provider was able to quickly turn the AWS environment back on quickly, but it still cost over a full day. And honestly, they were lucky that the terminated employee hadn’t also “scorched the earth” by deleting everything in the AWS environment as well. Then everything would have been completely unrecoverable. There are literally scripts out there for doing this very thing (AWS Nuke for example). 

Here are a few security measures your company can take against a security issue like this one: 

1. Practice Least Privileges 

Create at least two AWS admins, then set up other AWS accounts for your employees with very restrictive limitations on what they can do. The idea here is simple - everyone should have exactly the permissions they need and nothing more.  

Most cloud computing systems allow very granular control of user privileges. The Admin or Root account on any system shouldn’t be used for ANY daily work. Multi-Factor authentication should be activated on the Root account, linked to a hard-token* and that token put in a secure location. 

For the truly security-minded folks: put the hard-token in a safe, put the hard-token in a secure deposit box at a bank, etc. 

Also, ensure that two people have enough access to create users and fix permissions - that way, someone can be out sick without grinding the company to a halt. But the idea here is that the fewer the better, too many people with such access also raises your risk of a bad actor. 

*Why not a soft-token, such as Google Authenticator on a cellphone? Because you’re back to a single point of failure and risk of a lost/stolen/compromised device.  

2. Multi-Factor Authentication 

Multi-Factor Authentication (MFA) attaches a secondary authentication to (all of) your account(s), not just the root (email and password being the primary). You have likely experienced this when you were texted a code while signing up for something like Facebook, Uber, SnapChat, etc.  

Turn it on everywhere that you can. Not just AWS accounts. 

There are many forms of MFA, including text messages, apps on your phone, physical hard-tokens, and encrypted thumb drives. 

It’s very important to have a backup as well. Most systems will give you a set of “backup codes” which normally work as single-use access. You can print them or put them in an encrypted note - but make sure you get them if they are an option. 

Had the company used multi-factor authentication (and used it correctly! *), this ex-employee would have never been able to log into the account and shut it down without them knowing about it. 

*Going back to the earlier point about soft-tokens, if the company had MFA setup utilizing a soft-token installed on the terminated employee’s cellphone, then they would have been right back to square one. 

3. Notifications (Email or SMS) when Root account is accessed. 

This is almost self-explanatory, but you need to have notifications set up so that anytime the root account is accessed that the appropriate team gets a heads-up to ensure that any root access is authorized. 

GuardDuty, CloudWatch, and CloudTrail are three services that can help in that regard. It is an AWS Identity and Access Management (IAM) best practice recommendation to do so. 

4. Offboarding Process 

Finally, ensure your company has a secure offboarding process. Write up a procedure and review it regularly (at least once annually). 

The goal should be to strip all privileges in minutes, certainly less than 8 hours. When an employee is terminated, they should walk out of the building with no access and not be allowed back on their corporate computer. 

Today, so many services exist that can become critical to a business’s operation. If you can afford to use something like Okta to manage these services you will have an easy off-button, but if not at least consider using your email provider (Google Apps and Outlook both provide this service). 

Ultimately it is up to you to protect your data. A few small steps can go a long way to ensuring one bad actor won’t negatively impact your business. InfusionPoints is an Advanced Consulting Partner within the AWS Partner Network (APN).  We can help navigate your way through best practices suggested above and beyond. 

Contact Us today so we can help you secure your network. 

​