With all the attacks going on recently, every single MSP owner I’ve spoken to has been on edge about making sure their networks are secure. And several have been experiencing serious attacks on their client networks. The reality of 2021 is that your team needs to be prepared for ransomware attacks and network breaches.
One of the biggest ways MSPs can help themselves make sure they are prepared is through planning out attacks. The most effective tool for this in my opinion would be running out tabletop scenarios.
As I’ve been helping our partners implement effective tabletop strategic meetings, one thing is obvious. Most MSPs lack break glass accounts and of those that have them, few have teams that understand why they’re in place and what they should be used for.
What do I mean by break glass?
Think of that school fire alarm back when you were in high school. To sound the alarm, you probably had to break through some covering on the lever. The term break glass specifically comes from having to break the glass to get to that alarm.
In tech land, break glass accounts are a quick way for someone without specific access privileges to gain access if needed.
What are some reasons to use a break glass account?
Account issues—you might need break glass accounts for forgotten usernames or passwords (for instance, if someone with an account takes an extended leave or vacation). Also, locked passwords, or no user account may both result in a need for someone to access timely information.
Authentication issues—if you experience an authentication system failure, you may need a way to access your platform through a different means. Note: in this case, you should be using a different form of authentication with these accounts.
Authorization problems—if an emergency thrusts someone on your team into a roll they lack access for, you may have to immediately use a break glass account for them to help with the situation at hand.
What roles should break glass accounts have?
Normally, a break glass account used for emergency purposes needs to be able to access the system and revert any controls or lockouts that are in place and preventing users from working. I’d recommend most often that these accounts have administrator privileges in place to ensure you’re able to fix any issues ASAP. Break glass accounts usually have high privileges. Information on accessing these accounts should be in secured vaults that few individuals have access to.
How many break glass accounts should you have?
Normally, I would recommend only having one account per platform. Typically, these accounts would have global admin rights, excluded from your normal MFA (have some other MFA in place).
Keep it simple.
Make sure that your procedure for break glass is simple, effective, and reliable. Remember, you are likely having someone inexperienced access the platform in an emergency setting (there’s already enough distractions, make sure your process is as easy as possible).
Here’s a simple process for establishing break glass accounts:
Pre-staging your account
Make sure you create your break glass accounts ahead of time and be careful about what access and auditing trail you have in place. Here are some considerations when setting up your accounts:
Username—use something obvious and meaningful here. For instance, breakglass1 may be appropriate and could stand an audit trail. Make sure the account name is clear enough to your team when auditing so that they can alert on its specific use.
Strong passwords—as with every account, you need a strong password here—especially since these accounts will have heightened privileges. BUT make sure that the password is not too difficult to enter manually to avoid trouble entering it in an emergency (for instance, consider avoiding O’s and 0’s).
Account privileges—while the account will probably have heightened privileges compared to an average user, make sure to set the minimum necessary privilege. You don’t need someone—especially someone without a lot of experience on this platform—having more ability than they need to get through your platform-specific break glass process.
Auditing is a must—you should be logging what happens on this account. This will document specifically what work was performed while on the account. In some platforms, emergency accounts have a greater level of auditing automatically turned on, but make sure that the audit trail is working.
Managing the Accounts
Break glass accounts need to be carefully managed to allow access when they are absolutely needed. You likely will need to have emergency account details available within information for accessing the break glass account so a user will know what to do in the event of a break glass situation. Below are some options for where break glass account information may be stored:
Literally behind glass. If you take a literal interpretation to break glass, you may decide to store information for accessing a break glass account behind glass (similar to the fire alarm). While this may seem old fashioned, it will most likely prevent your team from accessing the information unless critically necessary and provides indication that it was accessed.
You may also consider locked drawers only specific people can access (you may assign individual codes to audit who accessed the information.
In the case where you want two people to be responsible for a specific break glass scenario, you may delegate each with part of a key to open the combination. For instance, you have the account information behind two locks, each assigned to a specific person or role within your company.
Wherever you store the break glass information, make sure that the individual(s) responsible for accessing the information in the event of an emergency understands what they will need to do before an event occurs. They should understand the sensitivity of having this information and the priority of needing to use an emergency account. I wouldn’t just assign this responsibility to any old person in your company.
Monitoring Use of Break Glass Accounts
You will want to carefully monitor use and access of your break glass accounts. I recommend you audit any privileged accounts regularly to understand their use. Make sure your alerting system is capable (and has been tested for) alerting when break glass accounts are accessed and used at minimum.
Unacceptable use should be recorded and responded to. Think of these break glass accounts to your fire alarm. You test your alarm at least once a year to make sure it would work. If you are in a corporate setting, you must document that your tests were successful. That’s exactly how I’d treat your monitoring of these accounts.
Cleaning After An Event
One last consideration you should have planned out is how you will clean up the account in the event it has been used. Here are some considerations I want you to think about:
Disable or delete the account—to prevent re-use of the account and password, consider changing the credentials. Some systems may automatically deactivate the break glass account after one use or may only permit the account active a day or some discrete time period post- activation of the account.
Make sure you account for what was accessed—take a look at the audit trail and make sure that the correct person(s) accessed the account and information within that account.
Make any notices—if you experienced a breach of information or inappropriate access of information resulting in use of the break glass procedure, make sure you notify the appropriate parties to what was or may have been accessed.
Update you break glass procedure—revisit how the procedure went. If there were any hiccups, this would be the perfect time to revise your procedure to allow for smoother use the next time a break glass account is needed.