Who will buy, our wonderful snake-oil?

February 8, 2014

“I am patient with stupidity but not with those who are proud of it.” — Edith Sitwell

In my previous job; which was online-moderation, I did some technical-support for a while. I didn’t last long because their naivety and lack of understanding of their basic profession simply astonished me. One of the things we had a huge difference of opinion about was trust. In my mind and experience, if you are going to trust somebody to login to a client’s accounts with the power of a moderator then you have to have some basic trust in them in the first place. In the three years that I worked for the company this went from being the default position to being a completely foreign concept. By the time I left, the utter contempt that the moderators were held in (and bear in mind that 80% of my job was moderation) actually shocked me.

When they took the conscious decision to stop trusting their moderators and started implementing procedures to deal with this, they also started assuming that this now made them secure. What’s more, they then took this completely naive and false view and used it to implement even more procedures in a similar vein. To cap it all, they then started selling themselves as experts in this field.

“Nothing in the world is more dangerous than sincere ignorance and conscientious stupidity.” — Martin Luther King, Jr.

As a former social-psychologist I think that Groupthink is one of the things that still fascinates me most about all forms of the security industry (and I include warfare in this too). It’s almost like a purpose built Petri-dish for watching terrible concepts grow to such a size that nobody can see they are based on nonsensical premises which anybody who dares to question is immediately mocked.

“Доверяй, но проверяй.” — Ronald Reagan.

Part of the problem I have seen over many years is an obsession with not trusting your own people. I think there is a British/American divide here to a degree since I remember the big kerfuffle when the Americans discovered that the nuclear missiles on the British submarines were only secured by cheap bicycle locks. The experienced security people I know understand that the initial vetting and ongoing good treatment of staff is much more important than any amount of internal security. If Edward Snowden had really wanted to leak those documents then it’s unlikely that increased security measures would have stopped him. Security may be effective for stopping short term whimsical abuses but is no substitute at all for trust and respect in stopping long-term ones. The more oppressive and divided you make a work environment and the more obstacles you put in the way; the more you lose the respect of the workforce. In fact the more inviting you make yourself to exactly the abuses you are attempting to stop.

I saw another moderation company advertise that every keypress their moderators made was monitored in real-time by supervisors – I suppose this may be something approaching a solution but who would want to work for a company that did that? Do they expect to employ happy competent and experienced professionals?

“To forget one’s purpose is the commonest form of stupidity.” — Friedrich Nietzsche

Back to my original topic – My former employers; who out of politeness I will leave nameless. I will take one example of which they seem to be particularly proud: their secure login system. When I started at the company, they had a system that would allow the moderators to login to the various clients they worked on with a single icon press. It was nice and convenient and people sometimes used it. It wasn’t compulsory, it wasn’t made for high-security and when it was broken for some reason; the moderators would just ask somebody for the direct login passwords and they would use them instead.

Then they got a new technical/security manager, with no technical background at all and no experience at all in either security or moderation. He had a lot of bright ideas and talked the talk well enough though; it would seem.

They replaced the former login system because the old one was closing down. This wasn’t a bad idea and I still was with them so far. It was a nice convenience to have! But then they made the mistake of assuming this was a secure answer to all their prayers; and this is where their problems started.

For a start, the system they used (Onelogin) was only ever as secure as the people using and managing it, and the person managing it didn’t have a clue what he was doing. Although it is possible to setup secure conduits and exchange passwords securely this was never considered to the point where it took about 10 seconds to get a password out of the system. One of my supervisors asked me to write them a guide to “hacking” Onelogin so they could get the passwords out to give to their team in fact. A copy is available here if you are interested in just how simple it is.

“Stupidity is a talent for misconception.” — Edgar Allan Poe

But that’s just the start of it. As in all good Groupthink scenarios, the problem gets worse and builds on the original misconception. Access to client sites was done via a company proxy, and the Onelogin system was secured so it could only be logged into via this proxy and via the “secure browser” which all the staff now had to use. This could have been well and good, except that the proxy-setup was apparently the most unreliable thing in the entire universe. It wasn’t the proxy itself which had a moderately good uptime, it was the fact that the authentication system was next to useless. When it crashed (which it did, often), the technical manager had to be woken up to restart it. Obviously he didn’t trust anybody else to have access to it. This meant that often enough the proxy was was inaccessible for hours and hours at a time. The company was a 24/7 moderation company with some high profile clients, and still had some project managers who actually cared about them; so what would then happen is that they told the moderators to use another browser without the proxy enabled and they were given the passwords needed to moderate. They were then told not to tell any technical-people because that obviously wasn’t allowed.

So now you have a company who has put a lot of time and effort into creating a system that they are convinced is secure, and a staff-base, being unable to actually do the jobs for which they are paid and being relied upon to do; finding ways to circumvent all of this simply so that they can work. They aren’t doing this maliciously; they are doing this because the security in place is actually hampering them doing their jobs.

“Only two things are infinite, the universe and human stupidity, and I’m not sure about the former.” — Albert Einstein

One of the best things about all of this is that, now the company has done all of this; being social-media gurus as they are, they are telling the world all about it. They have not only done collaborative videos with Onelogin; they have also now set up a company to tell other people how to spread the misconceptions even further.

So folks – If you ever want to set up a 24/7 moderation company, with some very high-profile clients with sites which are constantly under attack by bad-folks who thankfully haven’t yet discovered the three completely separate and incredibly easy-to-attack single points of failure which would bring the entire company down with no means at all of switching to any alternatives.  If you ever want to completely alienate your work force so they have no respect or loyalty to you at all any more and if you ever want anybody with any technical background at all to look at you as though you are a naked-emperor on acid; then I know just the advisors for you!


Practical Lessons in Humility

July 11, 2011

Back in 1987, a colleague and I were using the large computer systems of the University of Leicester to run rather a lot of number crunching. Leicester had a large, well connected and well managed VAX Cluster which was ideal as a platform to collect password files and analyse them for erm… Statistics. It’s probably fair to say that we didn’t have permission to be using the Leicester systems but life was somewhat different then.

We’d been using this machine for a few months and one day we spotted that something was amiss. The Systems Manager of the machine had activated process accounting which effectively meant that everything we had done had been logged and there was a complete audit-trail of our naughtiness.

This was a long time ago, in the days before writeable CDs but we had an equally bad adversary in the guise of the system console of the VAX 8600 we were squatting on. In those days consoles looked like this:

VAX System Console

We had a problem at this point, in that we didn’t really want the people at Leicester knowing what we’d been doing and at the same time we were somewhat stuck because it had all been printed out and you can’t remotely edit a teletypewriter trail. Although we’d turned off the accounting, and disabled any future accounting, we were in a little bit of a hole.

We pondered this for a while and came up with a nice and neat solution – We’d get the logs from the last week and we’d edit any traces of ourselves out – We’d then print this to the printer so that anybody looking back over it for however many days wouldn’t see us. A little bit of editing and a couple of hundred pages of printing later, we’d wiped all traces of our activity out. Taking the hint that we probably weren’t wanted, we politely left, breathing a small sigh of relief that we’d narrowly escaped detection.

A couple of years later, I was doing my postgraduate at Leicester University and I went for a drink with their Systems Manager, Pete Humble – I was a little more well known at this point as one of the “good guys” and we got to chatting about the old days and for some reason the subject of tracking a big hack a couple of years back came up. Pete told the story of how some people had been using his machines for nefarious activities but they’d always managed to evade any logging so nobody had a clue what they were doing.

Apparently he came up with the idea of setting process accounting on the miscreants and logging this all to the console. I grinned knowingly and smugly asked him how that went for him and was somewhat taken aback when he explained that those pesky kids had cleverly reprinted a few of days of sanitised logs to hide themselves BUT, that every night, he had marked the stack of paper on the edge with a thick red marker and when he saw that the evening had produced an extra foot or so of printout above the last red mark, he’d realised what had happened.

At that point, I bought the drinks for the rest of the evening and congratulated him on being the only person to have ever caught me, with something as apparently low tech as a red permanent marker pen.

Peet remained one of my best friends until he died a few years ago, and I like to think that he taught me a lot more than not to be too smug in underestimating lower-tech solutions.


Thank you Jeanette

August 22, 2010

Just as an amused aside for a Saturday afternoon – Back in 1998 I was building what was then Cellnet’s Genie System, it is now O2 but you can’t blame me for any of that, Genie was actually very good for its time.

I was bored one day and since Cellnet didn’t have anyone in charge of sex and pornography like BT then did, I couldn’t pass the buck on the username registration issue so I decided to do it myself. I sat down and made a list of rude words, I went round the office and asked everyone their favourite rude words, I polled IRC for various rude words and in the end, I had a pretty comprehensive list of rude words (which before you ask, I have now lost).

I decided to ignore the usual “Scunthorpe” issues, and just blanket ban registrations with any of them in – It wouldn’t tell them that they couldn’t use that name, it just told them that the username already existed.

I’d forgotten all about this about 6 months later until I got a call from the Cellnet support people up in Thurso – They had an issue with a user who couldn’t register and they’d tried to do it for her and still they couldn’t. They’d tried adding a 2, a 3 all the way up to 100 to her name and still, they kept getting told the user already existed. Nobody had been able to solve the problem, the poor user was upset that she had so many namesakes and I was just on the phone giggling which didn’t help at all.

I did fix it for her, as a one off, but to this day, the memory of poor old Jeanette Quimby still brings a smile to my face.


Learning from History

April 14, 2010

I first left the Commercial Security industry back in 1999 when I decided to move on and lose all my money with Recruitment.com. After that, I ended up being interviewed for a directorship of a company making Intrusion Detection Systems.

They asked me a question which surprised me – It shouldn’t have but it did – “You haven’t done anything in the industry for over four years now. What makes you think that your skills are relevant?”

Maybe this is a problem with the whole concept of Commercial Security or indeed anyone who takes a wide view of security as interconnecting social and technological parts – I don’t think people know what it is. I think they are looking for ongoing training courses and certifications that give you letters after your name pronouncing that some random software company considers you to be an expert in something. I don’t have any of these and really I don’t want them. On the engineering side I am not going to mend a Cisco router myself and on the policy side, the various qualifications seem utterly meaningless and at odds with one another. The people who create these certifications are trying to push a methodology and singular methodologies are not always a good way to go.

I explained politely that very little that I do has changed much in the last few hundred years – Security is security, always has been, always will be. The methods of delivery differ but the concepts don’t. You can learn more about my type of security from sitting in the middle of a large castle and pondering why they built it the way they did. The inhabitants of the castle needed to be able to come and go and get about their daily lives, there were markets, shops and whole villages inside the larger castles and these communities needed protecting from outsiders and insiders alike. There’s nothing really new conceptually. Take the Trojan Horse – This was first deployed in name over 3,000 years ago and even then one of Troy’s very experienced voices was ignored when he said they should burn it. A fine example of management-override in action which can still teach us more real-world lessons than countless courses and certificates.

It is in the commercial interests of various corporations to have us believe that they can create expert security people by giving them a training course and a certification. They can’t, no more than military basic training can create a good soldier. Sandhurst and Westpoint teach the classics to the military officers of the future for good reasons – A firm grounding in history, an appreciation that you can learn from the past and an understanding that the last 4,000 years of human development were not just a waste of time; these are the tools that somebody needs to become a good security all-rounder. Then they need the experience of applying it, experience of where it goes wrong and experience of learning to live with the fact that most of the time, nobody will listen to them. Don’t worry – History will also teach that this was always the case as Laocoön would have pointed out if he hadn’t been silenced permanently.