Law firms, businesses, the cloud, and security

January 19th, 2010 Clear2Go No comments
http://www.flickr.com/photos/room929/428260081/

http://www.flickr.com/photos/room929/428260081/

Nicole Garton-Jones submitted on slaw.ca today a post entitled Practicing Law on the Road: the Role of the Cloud and the Emergence of the Virtual Law Firm.  In it she highlights the idea of working remotely and using VoIP, Cloud computing and virtual desktops along with your PDA and laptop devices.  Especially when it comes to law firms, my experience is they are often slower to adopt to technological changes that other industries due to a combination of tradition and general need to follow government laws, and procedures enforced by their professional organizations.  It is nice to see a lawyer promoting these technologies, I think that is great for the legal industry.

In her post, she discusses cloud computing, laptops and PDAs and touches on the security.  I feel that the security needs to be given a much more serious discussion.  My experience consulting with small companies and law firms is that they typically do not give security enough time, consideration, or expertise before choosing a technology path.  There are many reasons for this, cost, resources, and time being the main factors.  It is usually discussed when a laptop with sensitive data goes missing, someone realizes there is a keystroke logger on their system, or their server data has been compromised and is leaking onto the Internet bypassing the firewall, IDS, anti-virus, and notice of the system administrators or third party companies hired to provide system administration and security.

Cloud computing offers many advantages and cost savings to companies.  It also brings with it the concern of your data being stored off-site, out of your direct control.  With large cloud computing vendors such as Amazon and Google, your data could be across the world in a foreign country and the laws that apply to the protection of that data probably differ from those in your home country.  This has been a topic of discussion for a while now in the Cloud computing arena.  One of the suggestions is to use a ‘private’ cloud.  This is typically a cloud that you own or have more control over where the data is stored.  For example, Canadian Cloud offers a guarantee that “…data are safe and secure on hardware located in Canada, and subject only to Canadian laws and regulations..”  This resolves international issues when it comes to control of data and is appealing.  However, there is much more to consider before choosing a provider.  While Amazon, Google and other large companies are international, they also have the size to attract security professionals that are very knowledgeable and current.  They can afford the resources to properly monitor against attacks to steal your data.  Google recently publicized the discovery of China conducing espionage on its systems.  Will a provider of a smaller cloud offering have the resources to detect such attacks?  If you install your own cloud, do you have the resources to hire individuals capable of detecting these types of attacks?  One could argue that not using Amazon or Google is less secure and you have more risk exposure.  My point is that companies and firms need to consciously assess these decisions based on the sensitivity of the information they are thinking about storing on a cloud system.

Laptop security is still as important weather the cloud is present or not.  It makes sense for an attacker to go after the weakest link and that is almost always the end user device.  Although one may suggest that all the information is on the virtual desktop on the cloud, there may be cases where data needs to be pulled locally.  If this is the case and the data is sensitive you will require encryption.  Even if data is not stored on the laptop ever and therefore there is no need for encryption and the management tasks it brings, installation of malware that will capture keystrokes and gather screen shots is invaluable on the laptop of a lawyer involved in a sensitive case.  This software exists in many places and is easily obtained and deployed.  Proper user device security does not go away.

Between iPhone and Blackberry, currently the Blackberry is much more secure than an iPhone.  Blackberry has the infrastructure including BES servers which allow enforcement of detailed security policies along with a robust management architecture.  BES servers offer the ability to remotely wipe a lost Blackberry as well as the ability to track the location of the phone remotely.  The Blackberry device itself has the ability to wipe all data via a menu option or by simply entering the wrong password a configurable number of times.   By comparison, the current iPhone can have a password in place, but bypassing it is easy once you have the physical device and security policies can be easily overridden by the user of the device.  I fully expect the iPhone to improve in this area as it targets the business market, but currently this is the general state of security with the iPhone.  A company that deploys iPhones or Blackberries needs to consider the type of data on these devices and the required security.  While many users prefer the iPhone over the Blackberry, you are making a security decision when you make this decision as well.  Best to make it consciously and understand the risks you are assuming with your firm and clients data.

Companies and firms need to consciously assess the security requirements of their data independent of any one technology.  Once this is completed, choose and deploy solutions and services that meet those requirements balancing off risk, cost, and convenience.  While there is no such thing as 100% security, you can consciously minimize this exposure, and manage the risk.

How confident is your company or firm that data stored on your local servers, cloud infrastructure, laptops, PDAs and other devices is secure, and can not be extracted or viewed without proper authorization?  If your data was being extracted or viewed without authorization would your security team detect it?  If not, why not?

Authorized to shutdown the data center, update

January 13th, 2010 Clear2Go No comments

I posted a couple weeks ago about operators monitoring systems and discovering a serious exploit in progress and determining what to do if no one was available to make a call such as shutting down a service.  What metrics are in place such as length of time, number of phone calls, seriousness of incident, that allow an individual to make a call that might affect the business confidently.  My example was one where it was discovered that a hacker was slowly siphoning off account information at a financial institution. I don’t know what this particular institutions procedures were, but turns out my fictional example happened.  Not surprised as it is a valid scenario in todays world, but thought it was worth commenting.

Categories: Incident Response Tags:

To follow or not to follow, that is the question

January 4th, 2010 Clear2Go No comments

followTheCatOn Slaw blog, there was a post today about some issues a few lawyers had when they ended up following an individual on Twitter. The post ends by effectively asking if people feel they should follow someone who follows them or not. I added my thoughts into the comments of that post, but thought that would be a good topic for a quick entry in my blog as I have pondered that question for a while.  I’ve added a little more detail here as to my criteria than the comment. The process is not cast in stone, rather a general set of guidelines that I typically use to make a decision.

My goal with social media is to connect and meet other interesting people.  As a general rule, I believe that when someone decides to follow you they are indicating they value your opinion and/or want to start some sort of on-line relationship with you. At least for a majority of people, I believe this to be true. Specifically in my areas of interest (security and networking), Twitter has been very valuable for me in building relationships, getting feedback, and keeping abreast of what is happening.  I also feel that the point of Twitter, Facebook, Linkedin and other social media sites is to connect with others, build relationships and trust.  Accomplishing that requires both parties to give, just like a relationship between two friends.  If it is one sided, what is the point?

That being said, there are those that will use social media for ‘bad’. Bad by my definition in this context, is to attempt to tweet me to death with useless information, send marketing links about products constantly, or use it as an automated tweeting tool where no real person is on the other side.

When someone follows me I typically do the following:

Check their twitter profile

Are others following them?  What is the ratio they have of followers to following?  If not many are following them, then I check how long they have been tweeting.  Maybe they are new.  The ratio of followers to following is a indication to me of how active they are and how interested they are in others.  A low follow rate may indicate they like to say things, but don’t like to hear opinions of others.  Not 100%, but an indicator.

Scan their tweets

I scan their previous tweets.  Are they informative and original or are they all just re-tweets.  Do they appear to be all just trying to sell products? Do they appear to be auto-generated?

Internet presence

Do they have an Internet presence such as a website, blog,  Facebook account,  Linkedin account?  If they have a website does it look legitimate?  Does the website or blog have information that is useful?  Are their opinions?  Is their an ‘about me’ area where they tell the reader about them.  This is extremely important to me.  I like to know who I am building a relationship with.  I don’t need big secrets about them, but a general concept of who you are, what you do, likes dislikes is helpful.  If I am going to read your posts, references to articles, I’d like to know that you are real and have some background and/or experience with the information you post.

General Internet search

I will search Google.  Do they post elsewhere?  Do they have comments and opinions?

Based on the information I find and feedback, I make a decision to follow or not.  This evaluation process is similar for blogs I add to my blog reader.  Again, this is not cast in stone.   There are a few that I follow that do not follow me back and that is fine.  However, for me that is the exception as opposed to the rule.

Do you have a criteria for who you follow on Twitter or what blogs you subscribe to?

Categories: Social Networking Tags:

The future belongs to people who take initiative

January 4th, 2010 Clear2Go No comments

sethGodin1Seth Godin was interviewed by Nora Young on Spark.  The interview can be found here.  The part of the talk where Seth describes how many of us were never trained to take initiative but to follow instructions and how that impacts us in our work made a lot of sense to me.  My favourite part was the section on emotional labour, the act of connecting to another human being and making a change even if it is not easy for you to do it in that moment.

A good talk for anyone in a leadership position.

Categories: Leadership and Management Tags:

Authorized to shutdown the data center

December 22nd, 2009 Clear2Go No comments

theBorderKissAndCrySladeEmergShutdown2The picture on the right is taken from a Canadian television series called “The Border“.  It follows a team of Canadian customs agents saving Canada from threats.  In this particular episode called “Kiss and Cry“, Slade who is their technical wizard agent discovers that the Chinese secret service has installed a trojan in their system allowing them to monitor their activities.  Upon investigation, discovery of the trojan, and a quick assessment of the risk, he immediately initiates a system wide shutdown of all services.  Given the sensitivity of data they have in their systems, the type of data their systems have access to,  and the nature of their business it was the right call, however I found it interesting that Slade made it.

Although this is a fictional television series, this scene got me thinking about my clients.  I can not think of any client large or small that is prepared for or has a single staff member onsite that could authorize a system wide shutdown quickly.  As an example, let’s take a large financial institution.  One of the technical staff is doing some routine system checks and discovers that every time a customer logs into their bank accounts, the customers login and password information along with other helpful data such as birth date and postal code is transmitted externally to a range of servers.  Being a large financial institution there is presently a new customer login average of one per second.  What should she do?  Should she shutdown all customer access immediately?  Should she investigate?  If she investigates, how long should she investigate for?  Can she get hold of someone who can authorize the shutdown?  What if that person is unavailable?  Can she make the call to shutdown services then?  It is obviously critical.  Should she keep trying others?  If so for how long?  If from discovery through investigation to authorization it takes 10 minutes, that is 600 client compromises in this scenario.

What is important is that the staff clearly understand what they can and can not do in any situation.  They need to feel comfortable they have done the right thing and will not be punished for doing what they ‘perceive’ as the right thing.  In the scenario above, if you asked your employees what they would do in this scenario, do you know what they would answer?  Would they be comfortable answering the questions above and more importantly would the business be comfortable with the answers and the risks associated with those responses?

I know many business people that would indicate this is fictional or ‘far fetched’.  While I would have agreed to some degree a few years ago, I wouldn’t today.  What I would suggest is that they go to a recent technical (not business) security conference or ask your technical team or consultants about latest research into threats and vulnerabilities and their availability.  Don’t ask the vendors (or at least be careful), they are trying to sell you results and are never as advanced as the bad guys.  Also keep in mind that even research is behind.  There are many malicious pieces of software that are ‘underground’, but you don’t need to look there.  Just look at some of the available off the shelf tools available for purchase.

Is your business realistically aware of the current threats to its data?  Are the risk assessments accurate?  Do you have the appropriately qualified staff and procedures in place to deal with current threats and do they have the appropriate authorization to make the necessary calls in the event of an emergency or unexpected event?  Is the business comfortable and accepting of the risk exposure associated with these decisions?

A simple and common network attack

December 17th, 2009 Clear2Go 2 comments

1930DictionaryIn working with large companies such as service providers, financial and manufacturing institutions, I have come across many common and simple attacks.  I will discuss one that I came across recently while planning for a project.  It is not a new attack as I and most other security professionals have encountered it many times.   The attack itself has been around for years now.  What amazes me is that regardless of how simple, common, and old the attack is I usually find it undetected on most networks.

Before walking through the attack, let me describe the steps used for this attack.  There are many  papers, books, courses and posts by security professionals on how to effectively detect and respond to attacks, the proper methodology, decision points and other variables.  These methods vary to different degrees in application, complexity and point of view.  For example, the methods and steps identified and taken by a first responder will be different than a security architect designing a system.    For the purposes of this post, I’ve chosen a simple set of steps:

  • Detection
  • Investigation
  • Scope
  • Assessment
  • Mitigation

Detection

I was working on a particular server and router.  I was planning a side project I have an interest in and wanted to check the configurations of the router and server to ensure it would support my project.  During the course of checking the server, I issued a command to check for the current connections being made to the server (netstat).
netstat1-cleansed1

What immediately jumped out at me was the ssh connection highlighted above in red.  Although SSH is permitted to this system, there is only 3 people that have access and all are members of the same ISP.  This connection was not part of the ISP netblocks.  It is possible someone could have been traveling and accessed it remotely but I was confident no one with access was in China  (where the IP is registered).  Regardless of the source address, the source port ‘36948′ was constantly changing every few seconds, indicating a new connections being spawned.

Investigation

After observing the constant connection attempts, a quick look at the server logs and some basic filtering revealed the following:

Nov 16 00:45:05 serverA sshd[5423]: Invalid user admin from 218.108.234.208
Nov 16 00:45:05 serverA sshd[5424]: input_userauth_request: invalid user admin
Nov 16 00:45:06 serverA sshd[5423]: Failed password for invalid user admin from 218.108.234.208 port 36910 ssh2
Nov 16 00:45:10 serverA sshd[5425]: Invalid user test from 218.108.234.208
Nov 16 00:45:10 serverA sshd[5426]: input_userauth_request: invalid user test
Nov 16 00:45:11 serverA sshd[5425]: Failed password for invalid user test from 218.108.234.208 port 38556 ssh2
Nov 16 00:45:14 serverA sshd[5427]: Invalid user guest from 218.108.234.208
Nov 16 00:45:14 serverA sshd[5428]: input_userauth_request: invalid user guest
Nov 16 00:45:16 serverA sshd[5427]: Failed password for invalid user guest from 218.108.234.208 port 40196 ssh2
Nov 16 00:45:19 serverA sshd[5429]: Invalid user webmaster from 218.108.234.208
Nov 16 00:45:19 serverA sshd[5430]: input_userauth_request: invalid user webmaster
Nov 16 00:45:22 serverA sshd[5429]: Failed password for invalid user webmaster from 218.108.234.208 port 41776 ssh2
Nov 16 00:45:31 serverA sshd[5434]: Invalid user oracle from 218.108.234.208
Nov 16 00:45:31 serverA sshd[5435]: input_userauth_request: invalid user oracle
Nov 16 00:45:33 serverA sshd[5434]: Failed password for invalid user oracle from 218.108.234.208 port 45829 ssh2
Nov 16 00:45:36 serverA sshd[5436]: Invalid user library from 218.108.234.208
Nov 16 00:45:36 serverA sshd[5437]: input_userauth_request: invalid user library
Nov 16 00:45:38 serverA sshd[5436]: Failed password for invalid user library from 218.108.234.208 port 47647 ssh2
Nov 16 00:45:41 serverA sshd[5438]: Invalid user info from 218.108.234.208
Nov 16 00:45:41 serverA sshd[5439]: input_userauth_request: invalid user info
Nov 16 00:45:43 serverA sshd[5438]: Failed password for invalid user info from 218.108.234.208 port 49440 ssh2
Nov 16 00:45:46 serverA sshd[5440]: Invalid user shell from 218.108.234.208
Nov 16 00:45:46 serverA sshd[5441]: input_userauth_request: invalid user shell
Nov 16 00:45:48 serverA sshd[5440]: Failed password for invalid user shell from 218.108.234.208 port 51218 ssh2
Nov 16 00:45:51 serverA sshd[5442]: Invalid user linux from 218.108.234.208
Nov 16 00:45:51 serverA sshd[5443]: input_userauth_request: invalid user linux
Nov 16 00:45:53 serverA sshd[5442]: Failed password for invalid user linux from 218.108.234.208 port 52953 ssh2
Nov 16 00:45:56 serverA sshd[5444]: Invalid user unix from 218.108.234.208
Nov 16 00:45:56 serverA sshd[5445]: input_userauth_request: invalid user unix
Nov 16 00:45:59 serverA sshd[5444]: Failed password for invalid user unix from 218.108.234.208 port 54704 ssh2
Nov 16 00:46:02 serverA sshd[5446]: Invalid user webadmin from 218.108.234.208
Nov 16 00:46:02 serverA sshd[5447]: input_userauth_request: invalid user webadmin
Nov 16 00:46:04 serverA sshd[5446]: Failed password for invalid user webadmin from 218.108.234.208 port 56994 ssh2
Nov 16 00:46:13 serverA sshd[5451]: Invalid user test from 218.108.234.208
Nov 16 00:46:13 serverA sshd[5452]: input_userauth_request: invalid user test
Nov 16 00:46:16 serverA sshd[5451]: Failed password for invalid user test from 218.108.234.208 port 60988 ssh2
Nov 16 00:46:24 serverA sshd[5456]: Invalid user admin from 218.108.234.208
Nov 16 00:46:24 serverA sshd[5457]: input_userauth_request: invalid user admin
Nov 16 00:46:27 serverA sshd[5456]: Failed password for invalid user admin from 218.108.234.208 port 36482 ssh2
Nov 16 00:46:30 serverA sshd[5458]: Invalid user guest from 218.108.234.208
Nov 16 00:46:30 serverA sshd[5459]: input_userauth_request: invalid user guest
Nov 16 00:46:32 serverA sshd[5458]: Failed password for invalid user guest from 218.108.234.208 port 38285 ssh2
Nov 16 00:46:35 serverA sshd[5460]: Invalid user master from 218.108.234.208
Nov 16 00:46:35 serverA sshd[5461]: input_userauth_request: invalid user master
Nov 16 00:46:37 serverA sshd[5460]: Failed password for invalid user master from 218.108.234.208 port 39898 ssh2
Nov 16 00:47:20 serverA sshd[5489]: Invalid user admin from 218.108.234.208
Nov 16 00:47:20 serverA sshd[5490]: input_userauth_request: invalid user admin
Nov 16 00:47:23 serverA sshd[5489]: Failed password for invalid user admin from 218.108.234.208 port 54777 ssh2
Nov 16 00:47:26 serverA sshd[5491]: Invalid user admin from 218.108.234.208
Nov 16 00:47:26 serverA sshd[5492]: input_userauth_request: invalid user admin
Nov 16 00:47:28 serverA sshd[5491]: Failed password for invalid user admin from 218.108.234.208 port 56536 ssh2
Nov 16 00:47:31 serverA sshd[5493]: Invalid user admin from 218.108.234.208
Nov 16 00:47:31 serverA sshd[5494]: input_userauth_request: invalid user admin
Nov 16 00:47:33 serverA sshd[5493]: Failed password for invalid user admin from 218.108.234.208 port 58262 ssh2
Nov 16 00:47:36 serverA sshd[5495]: Invalid user admin from 218.108.234.208
Nov 16 00:47:36 serverA sshd[5496]: input_userauth_request: invalid user admin
Nov 16 00:47:38 serverA sshd[5495]: Failed password for invalid user admin from 218.108.234.208 port 60006 ssh2
Nov 16 00:47:52 serverA sshd[5503]: Invalid user test from 218.108.234.208
Nov 16 00:47:52 serverA sshd[5504]: input_userauth_request: invalid user test
Nov 16 00:47:54 serverA sshd[5503]: Failed password for invalid user test from 218.108.234.208 port 36914 ssh2
Nov 16 00:47:57 serverA sshd[5505]: Invalid user test from 218.108.234.208
Nov 16 00:47:57 serverA sshd[5506]: input_userauth_request: invalid user test
Nov 16 00:47:59 serverA sshd[5505]: Failed password for invalid user test from 218.108.234.208 port 38498 ssh2
Nov 16 00:48:04 serverA sshd[5507]: Invalid user webmaster from 218.108.234.208
Nov 16 00:48:04 serverA sshd[5508]: input_userauth_request: invalid user webmaster
Nov 16 00:48:06 serverA sshd[5507]: Failed password for invalid user webmaster from 218.108.234.208 port 40506 ssh2
Nov 16 00:48:09 serverA sshd[5509]: Invalid user user from 218.108.234.208
Nov 16 00:48:09 serverA sshd[5510]: input_userauth_request: invalid user user
Nov 16 00:48:11 serverA sshd[5509]: Failed password for invalid user user from 218.108.234.208 port 42147 ssh2
Nov 16 00:48:14 serverA sshd[5511]: Invalid user username from 218.108.234.208
Nov 16 00:48:14 serverA sshd[5512]: input_userauth_request: invalid user username
Nov 16 00:48:16 serverA sshd[5511]: Failed password for invalid user username from 218.108.234.208 port 43771 ssh2
Nov 16 00:48:19 serverA sshd[5513]: Invalid user username from 218.108.234.208
Nov 16 00:48:19 serverA sshd[5514]: input_userauth_request: invalid user username
Nov 16 00:48:21 serverA sshd[5513]: Failed password for invalid user username from 218.108.234.208 port 45636 ssh2
Nov 16 00:48:24 serverA sshd[5515]: Invalid user user from 218.108.234.208
Nov 16 00:48:24 serverA sshd[5516]: input_userauth_request: invalid user user
Nov 16 00:48:26 serverA sshd[5515]: Failed password for invalid user user from 218.108.234.208 port 47217 ssh2
Nov 16 00:48:35 serverA sshd[5520]: Invalid user admin from 218.108.234.208
Nov 16 00:48:35 serverA sshd[5521]: input_userauth_request: invalid user admin
Nov 16 00:48:37 serverA sshd[5520]: Failed password for invalid user admin from 218.108.234.208 port 50752 ssh2
Nov 16 00:48:40 serverA sshd[5522]: Invalid user test from 218.108.234.208
Nov 16 00:48:40 serverA sshd[5523]: input_userauth_request: invalid user test
Nov 16 00:48:42 serverA sshd[5522]: Failed password for invalid user test from 218.108.234.208 port 52460 ssh2
Nov 16 00:49:05 serverA sshd[5536]: Invalid user danny from 218.108.234.208
Nov 16 00:49:05 serverA sshd[5537]: input_userauth_request: invalid user danny
Nov 16 00:49:07 serverA sshd[5536]: Failed password for invalid user danny from 218.108.234.208 port 32852 ssh2
Nov 16 00:49:10 serverA sshd[5538]: Invalid user sharon from 218.108.234.208
Nov 16 00:49:10 serverA sshd[5539]: input_userauth_request: invalid user sharon
Nov 16 00:49:12 serverA sshd[5538]: Failed password for invalid user sharon from 218.108.234.208 port 34547 ssh2
Nov 16 00:49:15 serverA sshd[5540]: Invalid user aron from 218.108.234.208
Nov 16 00:49:15 serverA sshd[5541]: input_userauth_request: invalid user aron
Nov 16 00:49:17 serverA sshd[5540]: Failed password for invalid user aron from 218.108.234.208 port 36174 ssh2
Nov 16 00:49:20 serverA sshd[5542]: Invalid user alex from 218.108.234.208
Nov 16 00:49:20 serverA sshd[5543]: input_userauth_request: invalid user alex
Nov 16 00:49:22 serverA sshd[5542]: Failed password for invalid user alex from 218.108.234.208 port 37737 ssh2
Nov 16 00:49:25 serverA sshd[5544]: Invalid user brett from 218.108.234.208
Nov 16 00:49:25 serverA sshd[5545]: input_userauth_request: invalid user brett
Nov 16 00:49:27 serverA sshd[5544]: Failed password for invalid user brett from 218.108.234.208 port 39340 ssh2
...............

From the server logs, we can determine:

  • Attack started at 00:45
  • Dictionary attack where the attacker is sequencing through names as well as common Unix account ids.
  • Rate is approximately 1 id every 1.5-2 seconds
  • Source port is reasonably random, or at least random enough to fool basic firewall and IPS technologies.

Scope

What other systems if any on the network are under attack?  To determine this quickly I logged onto an aggregation point and captured traffic that corresponded to the attack in progress for a few minutes.  Next, a command was run to filter the captured data to show the servers that were being attacked.

$ tcpdump -n -r ./sshBfAttack-ispView.cap "src net 218.108.234.0/24 and tcp[tcpflags] & (tcp-syn) != 0" | awk '{print $5}' | awk -F. '{print $1"."$2"."$3"."$4}' | sort -u
reading from file ./sshBfAttack-ispView.cap, link-type EN10MB (Ethernet)
xxx.x0.0.25
xxx.x0.0.4
xxx.x0.0.43
xxx.x0.12.100
xxx.x0.12.101
xxx.x0.12.103
xxx.x0.12.136
xxx.x0.12.142
xxx.x0.12.20
xxx.x0.12.29
$

We now have a list of current targets.  The filter above is a simple filter and it makes some basic assumptions.  Several filters were run on the traffic to ensure the scope of the attack but for the purposes of this post, the concept is what is important.  The type of filters and parameters of the filters one uses will depend on the type of attack, direction of the attack and other factors.

Assessment / mitigation

What most fear when they assess an attack are false positives of actions they perform.  An action that causes a valid request to be denied for example.  In the case of a company such as an Internet service provider, financial institution or any business that makes money using the Internet, this could be detrimental.  How a company mitigates or handles an attack really depends on many factors.  The type of attack, the behaviour of the attack,  the risk of stopping the attack,  the risk of letting the attack proceed are just some examples of questions that need to be asked and answered.

For this specific attack:

  • The servers being attacked contained no financial or personal data that was at risk to anyone.
  • One of the servers controls some password authentication features
  • The attack is external and coming from a specific IP address.
  • The service under attack is really not required for external access.

The solution was to deploy an access control list on the routers to not permit connections to that service from external sources.  This effectively mitigated the attack.

Conclusion and thoughts

What amazes me is that these dictionary type of attacks, regardless of service are very common.  Every step I have outlined here can be automated and should be, yet in so many cases this is not true.  I know many organizations that have spent thousands of dollars on projects, vendor equipment, security audits, and consultants, yet you take a look at their network and this simple, known,  attack is still present and goes on undetected.

Has your company spent time and money on security solutions such as audits, penetration tests, and products for security?  If you looked at your network or asked your security folks if the attack here would be automatically detected, reported, investigated and mitigated if it was present on your network would the answer be ‘yes’.  If not, why not?

Nov 15 10:38:00 flashpoint sshd[2924]: Invalid user webmaster from 200.87.171.78
Nov 15 10:38:00 flashpoint sshd[2925]: input_userauth_request: invalid user webmaster
Nov 15 10:38:02 flashpoint sshd[2924]: Failed password for invalid user webmaster from 200.87.171.78 port 53724 ssh2
Nov 15 10:38:18 flashpoint sshd[2933]: Invalid user sales from 200.87.171.78
Nov 15 10:38:18 flashpoint sshd[2934]: input_userauth_request: invalid user sales
Nov 15 10:38:20 flashpoint sshd[2933]: Failed password for invalid user sales from 200.87.171.78 port 54139 ssh2
Nov 15 10:38:24 flashpoint sshd[2935]: Invalid user admin from 200.87.171.78
Nov 15 10:38:24 flashpoint sshd[2936]: input_userauth_request: invalid user admin
Nov 15 10:38:26 flashpoint sshd[2935]: Failed password for invalid user admin from 200.87.171.78 port 54247 ssh2
Nov 15 10:38:30 flashpoint sshd[2937]: Invalid user andrea from 200.87.171.78
Nov 15 10:38:30 flashpoint sshd[2938]: input_userauth_request: invalid user andrea
Nov 15 10:38:32 flashpoint sshd[2937]: Failed password for invalid user andrea from 200.87.171.78 port 54347 ssh2
Nov 15 10:38:40 flashpoint sshd[2939]: Invalid user backup from 200.87.171.78
Nov 15 10:38:40 flashpoint sshd[2940]: input_userauth_request: invalid user backup
Nov 15 10:38:41 flashpoint sshd[2939]: Failed password for invalid user backup from 200.87.171.78 port 54462 ssh2
Nov 15 10:38:45 flashpoint sshd[2941]: Invalid user guest from 200.87.171.78
Nov 15 10:38:45 flashpoint sshd[2942]: input_userauth_request: invalid user guest
Nov 15 10:38:47 flashpoint sshd[2941]: Failed password for invalid user guest from 200.87.171.78 port 54613 ssh2
Nov 15 10:38:51 flashpoint sshd[2943]: Invalid user guest1 from 200.87.171.78
Nov 15 10:38:51 flashpoint sshd[2944]: input_userauth_request: invalid user guest1
Nov 15 10:38:53 flashpoint sshd[2943]: Failed password for invalid user guest1 from 200.87.171.78 port 54697 ssh2
Nov 15 10:38:57 flashpoint sshd[2945]: Invalid user guest2 from 200.87.171.78
Nov 15 10:38:57 flashpoint sshd[2946]: input_userauth_request: invalid user guest2
Nov 15 10:38:59 flashpoint sshd[2945]: Failed password for invalid user guest2 from 200.87.171.78 port 54798 ssh2
Nov 15 10:39:04 flashpoint sshd[2947]: Invalid user guest3 from 200.87.171.78
Nov 15 10:39:04 flashpoint sshd[2948]: input_userauth_request: invalid user guest3

What matters now

December 14th, 2009 Clear2Go No comments

whatMattersNowSeth Godin put together eBook entitled “What Matters Now”.  So far I have only read the first 30 pages.  He contacted a bunch of individuals and asked them to write a page expressing their thoughts and feelings on the future.  Several of the individuals are people I follow on a regular basis.  So far it has been a great read, especially this time of year.   If you are still interested, I’d suggest reading Seth’s blog entry or Michael Hyatt’s posts.  Both are much better writers than I and will do it the justice it deserves.  You can also download the eBook from links in their posts.

Categories: Leadership and Management Tags:

Globalive, Bell, Telus, Rogers and customer resentment

December 11th, 2009 Clear2Go 1 comment

wind1The Canadian Government announced this morning that Globalive (operating as Windmobile in Canada) is free to enter into the Canadian Market and compete with our Tier 1 providers in the wireless space.  Their ability to compete in Canada has been under fire by Rogers, Telus, and Bell for a while now.  What I am amazed at most is the responses I have been seeing on Twitter, instant messaging, comments on news articles and even a poll.  If it wasn’t obvious before now, Canadian consumers seem to be:

  • very happy that the CRTC decision was over turned and Globalive is allowed to compete in Canada.
  • Customer anger, frustration, and resentment are very obvious with Rogers, Bell, and Telus.

These feelings don’t just pop up.  They have obviously been building in consumers over time.   I hope this is a wake-up call for the providers.  The anger and frustration being expressed is serious and I am disappointed they either were too naive to see it building in their customers, or just didn’t care.  Either way, it will now probably directly affect them.  My hope is that they learn to value their customers thoughts and opinions in the future.  As a side note, I think this is the happiest I’ve seen Canadians with the current conservative government to date.

Categories: Mobile/Wireless, customer service Tags:

Derailed by Tim Irwin

December 3rd, 2009 Clear2Go No comments

derailedI was recently given via Michael Hyatt at Thomas Nelson Publishing a copy of the book Derailed.  The book was written by Tim Irwin.  In the book, Tim discusses what he feels are the reasons why leaders fail as leaders and gives insight into how to avoid these situations.

The first part of the book profiles 6 CEOs of major companies that failed as leaders.   Each leader is analyzed and what Tim feels are their weaknesses and the reasons why they were asked to resign from their position.  While one can argue that the opinions are subjective (and they are), I found his rational to be sounds and made sense.  Regardless of the subjectivity, anyone can learn from the mistakes of these profiled individuals and help themselves be a better leader.  I found myself identifying with the character flaws of these individuals.  I have seen them in many people I have worked with and even myself at times.

The rest of the book discusses the derailment process.  Finally, based on the profiles and the derailment process, Tim Irwin identifies and discusses five lessons that can be learned by anyone in a leadership role and ways to implement these lessons and keep yourself ‘in check’ as a leader.

I found Derailed extremely valuable for myself.  It was well worth the time to read and would suggest it to really anyone that interacts with other people at work.  Although the book profiles CEOs of large organizations, it is very applicable to anyone, even those not in an ‘official’ leadership position.

Investigation of encrypted traffic

November 23rd, 2009 Clear2Go No comments

onyx1As the traffic on the Internet becomes more and more encrypted due to privacy concerns, the need to protect data from third parties, prying eyes, marketers, service providers and others, behavioural profiling of network sessions will become more and more necessary.  Already, there are many products that claim to do behavioural profiling of network activity in varying degrees to assist with behaviour detection.  There is more and more active research in this area by vendors, law enforcement, bad guys and others.

I reviewed a report where it was indicated that because the data was encrypted it was impossible to determine anything useful.  This is not always the case, but I have seen this conclusion in reports and investigations many times when dealing with encrypted or unidentified data.  Aside from the marketing which says that if my Internet sessions are encrypted then one is safe (nothing could be further from the truth), many network administrators do not understand or have had much experience with behavioural profiling.  Behavioural profiling of networks can be very complex, and research is relatively new in this area.  To give some insight into how one might profile network sessions and show how one can use behavioural profiling to extract information, I decided to walk through a simple example and answer a simple question.  Specifically, what are the differences between an encrypted network session where one is watching a program or video (user providing no input), compared to an interactive type of network session where one is interacting (providing input)?  I used the SSH protocol to illustrate.

I used video over SSH to watch a program.  The program was approximately 24 minutes in duration and was hosted on a server at my ISP.   There were no problems watching the program, it didn’t pause or stop, and it was just like watching a typical television program (in fact I watched it on my flat screen TV).  I used a device to capture the traffic between the server hosting the program and my home for the entire duration of the program.  Finally, I captured an interactive SSH session which was me logged into a server at my ISP, where I was doing some coding and some shell commands.

Attempts to look at the actual data of either of these captures will be useless.  Since the data is encrypted, without access to the session keys knowing what was transmitted is close to if not impossible.  That being stated, what behaviour characteristics can we observe to tell us what might be going on?

I separated the direction of each of two captures which gave me 4 capture files, video received, video transmitted, interactive data received and interactive data transmitted.

Bandwidth

Received Transmitted Ratio
Video 193.2 MB 7.0 MB 0.036
Interactive 0.59 MB 0.58 MB 0.98

Looking at the chart above, the video watching has a much larger amount of data received than transmitted compared to the interactive session where a similar amount of data is transmitted and received.  Analysis of most video streaming and flows where downloading is occurring will yield a similar results.  The ratio of received to transmitted data will be high.  Interactive sessions tend to have a more balanced ratio of transmitted to received data compared to a video session.  This of course has dependencies on what the user is doing in the interactive session, but typically this has been the case in my experience.

Inter-packet timing

Another interesting metric is the time difference or delta between two packets.  Watching a video or listening to music, the delta between two packets tends to be small in comparison to an interactive type of session.  There are a few reason for this.  Since the video is being viewed, it is important to ensure that the data arrives in a timely manner so as to not have the video ‘freeze’ while being watched.   Some software attempts to write the video data to disk in advance of viewing to help mitigate this problem, but that leaves an exposure where an savvy individual can obtain a copy of the video by simply making a copy of the temporary file.  As a result, newer software tends to attempt to keep the data in memory and not write it to disk.  The result is the need to ensure a smooth delivery of data, minimizing delay between packets (known as Jitter).

Received (seconds)
Transmitted (seconds)
Maximum Mean Std Dev. Maximum Mean Std Dev.
Video 3.065 0.021 0.094 3.051 0.014 0.076
Interactive 4028.555 3.568 88.736 4028.544 2.162 69.137

I wrote a simple python script which will take as input a capture file, calculates the inter-packet timing for each pair of packets and then outputs among other information, the results you see in the table above.  The Maximum field is the largest time between packets, the mean is the average time between packets, and the standard deviation is a measure of how ‘different’ the inter packet times are from the ‘normal’.  For those that don’t know or wish to have a refresher in standard deviation, here is a good place to start. However, most languages and spreadsheets have functions to calculate this for you if you do not wish to learn the math.  In simple terms and using our specific example, if all the packets had the exact same time between them then the standard deviation would be 0.  The greater the difference in timing between packets, the greater the standard deviation will be.

Notice that the standard deviation is much higher for the interactive session then the video session.  Sessions that stream data, tend to have a low standard deviation for inter-packet timing.  If you think about it this makes sense, as an interactive session you can walk away from the computer, or the program could be waiting for input from the user so data transmission will fluctuate more.

Bandwidth, inter-packet timing, and methods such as standard deviation and mean are just a few things that can be used to narrow down what a particular subjects activities might be.  In corporate or law enforcement investigations, profiling network behaviour can be a useful tool to determine if you need to spend more time on the investigation or if you have the right target.  Using our example above,  suppose a corporation wants to determine which employees are watching streaming videos.  A scan of the network data reveals an individual who has encrypted sessions, but these sessions show a transmit / receive ratio that is in line with typical interactive sessions and not video sessions.  Also, the standard deviation of the inter-packet timing is higher for these sessions, then you can rule them out as an individual of interest immediately.  This has the advantage of focusing your investigation, not encroaching on privacy issues unnecessarily,  and saves time by allowing you to focus on the users that have network sessions with characteristics that fit the behaviour you are looking for.

For those of you that feel comfortable because the data is ‘encrypted’ it can be a false sense of security.  These are two of the many metrics and theorems that can be used on the data.  This area has active research and there are many products that will do this type of analysis in an automated fashion.  For those interested in this, although older now, this is a great paper where an experiment was conducted to determine what movie people were watching even though the movie data was encrypted.  They used behavioural data to fingerprint the movies, then applied the fingerprints to encrypted transmitted data.