JOHANNESBURG — For years, South African Haroon Meer worked as a renowned ethical hacker, being paid by companies to run so-called ‘pen’ tests (penetration tests) on their networks to determine their vulnerabilities. It’s a job that he relished in, but he soon began to realise that pen tests weren’t necessarily completely safeguarding companies. He then embarked on starting his own company called Thinkst which has gone on to develop pieces of technology that are now being used by some of the world’s top tech giants and Fortune 100 companies. His company has developed what is called the ‘Canary Honeypot’ device, which fools hackers into thinking that they’re snooping around a company’s servers when they actually aren’t. The devices then subsequently alert the owners of a breach and give them a chance to clamp down on the attack before it’s too late. Meer’s Canary team – which is spread across Johannesburg and Cape Town (Meer himself is based in Qatar) – has also developed what is called Canary Tokens. These are digital files that can be used to fool hackers into thinking that they’ve stumbled upon an important document, such as an excel sheet containing salary details. When the file is opened, it sets off an ‘alarm’ alerting you that your computer or server has been breached. Meer has become a celebrity in the global cyber security space for his thoughts and practices in the industry – he’s also been a regular speaker at the world’s biggest cyber security fair Black Hat. I had a fascinating discussion with him. – Gareth van Zyl
With me on the line from Durban is Haroon Meer. Haroon, it’s a pleasure to talk to you today. For years in SA you were a very well-known ethical hacker/‘pen’ tester, where you were basically paid to hack into people’s systems. You’ve now branched out into your own business called Thinkst and you’re living in Qatar. Can you tell is a bit more?
I spent the better part of 10 or 11 years, like you say, breaking into networks and breaking into applications and we were pretty lucky because in the early 2000s it was the start of the industry, so, nobody really was doing it much longer than we were. We made a good name for ourselves by working hard and being particularly ‘evil’ at what we did. We broke into companies all over SA and then all over the world. It’s really a nice career. It’s a fun career to learn lots of stuff in but, at some point, I made a decision about two things. One is that pen testing wasn’t making customers considerably better. So, you’d find a way to break into them every time and it wasn’t really moving the needle. Then the other, from a personal point of view, is that you spend every engagement pulling off really cool attacks (or really creative attacks) but you’re effectively almost always scribbling in the margins of other people’s’ work. The hope with starting Thinkst then was that we could use that type of creative energy to build products that actually help people defend themselves. That’s where we are right now. We’re pretty fortunate. We’ve got a good team, and we’ve got a good product that’s got good international attraction. So, we’re pretty happy.
And your team is all over the world, you’re in Qatar, some of them are in Cape Town and Johannesburg as well?
Exactly, and we’ve just started with a sales presence in the US. For the most part, we concentrate our development skills in Cape Town but at this point it seems like we’re heading for a pretty dispersed company.
I was fortunate enough to watch one of your talks last week in Johannesburg at an Internet Solutions (IS) event. There you spoke about something called ‘Canary tokens’. Now, you’ve also developed what’s called the ‘Canary Honeypot’ can you tell us more about that?
So, ‘honeypots’ are a pretty old concept right. Since the early days of the internet security, people have been talking about putting out fake systems so that attackers who broke in would reach out to the fake system and either waste their time or give you a heads up about what’s happening. So, the concept is really old, but what we found from years and years of pen testing, and then a good amount of consulting, that very few customers actually implemented honeypots. For instance, I’ve done hundreds of pen tests, and when we managed a company, together we did thousands of pen tests. But we very seldom have ever been caught. The statistic will shock you but I’m talking about single digits across thousands of engagements, and this ties into what is said in the industry which is that a company will typically be broken into for something like 200-300 days before they’ve even found outthat they’ve actually been hacked.
— Internet Solutions (@IS_global) September 6, 2017
The concept of a honeypot then seems like such a no brainer because if you have these honeypots laying around in your network, when an attacker breaks in and he moves laterally (as they do) at some point they trip over your honeypot and then you get a very strong signal that there’s badness on your network. You then don’t have to find out about it when you’re on CNN, or 200-300 days later. But the key question then becomes: If it’s such a logical idea then why is nobody doing it? Until recently, almost nobody that we tested would have them and the reason we hit upon regarding this is that people are struggling to keep their heads above water, managing the 1000 or 20000 hosts that they have. You can’t tell them: ‘Here’s a good idea set up these fake hosts and manage those.’ So, the idea with Canary is really a honeypot as a server. People are then able to install this honeypot in under 4 minutes. It sets up on the network, and it really disguises itself really well so it looks like a valid functioning Cisco router or a Windows server and it adopts all the personalities.
If it’s a Cisco server, it will run Cisco services, and if it’s a Windows server it will look like a Windows server with a real Windows share, which makes it totally believable and then it just sits there. It’s not more noise for admins until the day that someone has broken in and then goes and touches it, and you get your notification telling you that someone has touched your Canary. So, it explains the name, right. It’s like the canary in the coalmine that tells you that there’s badness, and Canary tokens are an offshoot of this as well. We offer Canary tokens as a completely free service: Anyone can go to Canarytokens.org to download them. Essentially, Canary tokens are little tokens that you get to sprinkle around in places that do pretty much the same thing as Canaries do, which is they become little tripwires on your network or in your applications that regular people shouldn’t touch. But if they are touched, they give you a warning that stuff is happening.
For example, if I go to Canarytokens.org, it will ask me for an email address and I’ll put in mine. Then I’ll say, as a reminder, ‘mum’s Dropbox’ and it would give me a PDF file, which I’d then leave in my mum’s Dropbox. I’ll then create another token and call it ‘mum’s Gmail’ and leave that one in her email. I can do this for thousands of them and at some point, when someone breaks into my mum’s PC and is browsing her Dropbox, I’ll get this warning that tells me, ‘someone has just tripped the Canary token in your mum’s Dropbox.’ So, it becomes this sort of low overhead deployment. It’s not something I actively think about and when it gets tripped I get a warning telling me bad stuff is happening and where it’s happening.
I know that it’s probably a sensitive issue to detail, especially in the cyber security world, but I believe that some of your clients, specifically for the Canary Honeypot, are among the major tech firms of the world. Can you give us an idea of how widely used your hotspot is or your honeypots are?
We’ve been pretty fortunate, in part, because we had a good name coming out and then because lots of the early guys who tested the honeypots really liked them. Most customers don’t like being named, and I know this is going to go public. But if you use the internet today, it’s a guarantee that you used a company that’s using Canary. In terms of some of the guys who allow us to us their names, we’ve got the likes of big media houses like Al Jazeera. So, Al Jazeera have hundreds of honeypots. If you break into their network it’s not clear whether you’re touching broadcast equipment or whether you’re touching an actual honeypot that will alert people. Then in the US, if you’re Fortune 100, or if you’re one of the Silicon Valley darlings, like probably the best thing to do is to search for Canary on Twitter because you’ll find the guys at Slack talking about us or the security guys at FC talking about us. They kind of end up being our best advertisements.
Obviously, the reasons why the honeypot is so widely used is because in the cyber security world you are also a major celebrity. In fact, you’ve been a regular speaker at an annual conference called Black Hat, which is the world’s premier gathering for cyber security experts. You were a speaker at this year’s event as well, correct?
Yes so, I’m hoping they buy the product because the product is good and maybe the little bit of celebrity in this niche helped nudge it along initially. We’ve been speaking at Black Hat for a long time. I think I first spoke in 2002, and we’ve been delivering research at Black Hat and DEF CON almost every year since then. We were there this year at Black Hat and in 2015 I actually keynoted Black Hat, which was pretty cool. Also, a reminder that I’m just getting old.
During your talk in Johannesburg last week you further highlighted how there needs to be a change in thinking regarding security software. In fact, you said that some security software that exists out there is really not that secure at all.
Yes, it’s a horrible, and I guess, a not so well-kept secret anymore about the security industry. The fact is that we haven’t really figured out how to write secure software at scale and most security companies have been using the same rotten development practices as everybody else. So, there’s a whole bunch of security software that’s out there that’s written by just about the same guys who are writing just about any other piece of software. Except you trust it and put it into the frontlines of your defence.
A long time ago, a famous hacker named ‘Mudge’ pointed out that a security device is not necessarily a secured device. Over the last while we’ve seen this point being hammered through. Recently, guys at Google’s Project Zero, which is a dedicated security team, have been aiming at antivirus software. They’ve literally been taking it to the cleaners; so, I’m not talking in this case about bypassing antivirus. I’m talking about the fact that you are running antivirus, becoming the attack factor that gets you compromised because; again, those are big bloated pieces of software written in low-level languages trying to pass everything that they can, which gives them a hard job and most of them are failing at that pretty badly.
Haroon, you’re on the coalface when it comes to the cyber security world. Are you worried about some of the threats that are coming out? It looks as if the cyber security aspect of the world is becoming more of a worry for executives out there?
Yes, so the obvious answer is yes, but people shouldn’t listen to me because I’ve got a vested interest in making it sound pretty bad. But in truth, I am worried and there’s multiple reasons for it. The one is that our dependence on it is growing. Like increasingly a bad software outbreak has far reaching implications. Recently with the NotPetya, coming out of the Ukraine, you saw cascading effects that were not expected. You saw shipping guys like Maersk saying this caused problems with actual logistics. So, somewhere on the docks of some country someone is waiting for a container that doesn’t show up because somewhere in the Ukraine someone wrote some malware.
That stuff is just increasing all the time and while you have these massive, complex cascade failures on the one hand, you’ve got the other problem that most companies actually don’t know how bad they are. Again, I’m not saying it to be alarmist. I’m saying it because one of the things I’m fond of saying is that software is a force multiplier. Most companies haven’t come to grips with this yet. There’s sociological problems around software that we haven’t figured out. In the old days, you’d go about figuring out where more of your company’s bodies were buried, like if you figured out more insider information on the company that could probably get the company in trouble. You’d move further up the line so, they’d promote you more and more. To the point where your fate and the company’s fate were intertwined, and so, you’d have all these execs who know where all the bodies are buried.
But today, probably one of the lowest paid guys in the company is your help desk admin or some of the secretaries and they get to read everyone’s mail, or they could if they wanted to. So, now you’ve got the lowest paid guy at the company who actually may have access to all the mail of all the execs. And again, sociologically, we just haven’t figured out how to handle that that stuff yet, and execs probably don’t know that this is happening. Like CEOs or boards don’t know that; actually, their email can be reset by the $2 an hour help desk guy and attacks are almost an extension of that.
I was also introduced to you by Ronnie Apteker, who is of course the founder of Internet Solutions. What is the relationship there between Ronnie, and you and your business?
Ronnie and some friends actually approached us just over a year ago after hearing about Canary. We had started a partnership with IS and Dimension Data so IS is actually selling a solution called IS Breach Detection, which runs on top of Canary. It’s our Canary device that’s powering that solution. Ronnie and a small group have actually come on as investors and advisors. Essentially, they’re now a small part of the company and we get to talk to some of their friends and they get to peek in another product that’s hopefully doing well.
It must be quite interesting having Ronnie’s insight onboard?
It is certainly interesting. He’s a fun guy with a lot of miles so, yes, it’s definitely worth it.
Where do you see Thinkst and Canary in the next 2 to 3 years?
Yeah, that’s a good question. So, for the most part, our aim is twofold. Internationally, we like our presence so, we like the fact that major US companies like us and really like our product. I joked before but if you go on Twitter you’d see that the chief security officer of Slack has two stickers on his laptop, one is Slack and the other is Canary so, it makes us proud that a SA company is out there waving the flag. We hope to expand that. We hope to really infringe ourselves as a player in the US market.
— Geoff Belknap (@geoffbelknap) March 25, 2016
Then at home what we’re trying to do is make sure we build the sort of company that we would always have wanted to work at. Lots of people have caught the start-up bug and throw down two small tables and beanbags, and I’m a lot less for that and a lot more for just trying to spur or make or create a culture in SA. I think SA has got a problem where we’ve churned out lots and lots of consumers and not a lot of producers. I’m hoping that companies like ours can start making people realise that we can make software too and we can play with the best in the world too. Yes, hopefully, we kick a lot more of that into action.
Haroon, it’s very interesting that you’re living in Qatar. Would you ever consider moving to Silicon Valley seeing as that’s where probably a lot of your clients are based?
It’s interesting. I obviously visit Silicon Valley a lot and I’ve got a lot of friends there and lots of our customers are there. In part I almost resist it because I don’t think we have to. If you talk to any venture capitalist he’ll tell you that you have to be there. But in a way, I feel like Silicon Valley becomes a bit of its own echo chamber and I think we’re managing to get the best of both worlds by visiting often enough and selling in there often enough but not actually, living there. I’m going to try to resist for as long as I can because I think we can win without it.
Haroon Meer, it’s been an absolute pleasure talking to you, and very fascinating stuff. Thanks for taking the time to chat to me today.
Thank you, thanks for your time.