Category Archives: Maltego

Social Media Security Audits

I just noticed Foreign spies on LinkedIn trying to recruit civil servants by ‘befriending’ them before stealing British secrets.

MI5 have warned that ‘hostile intelligence services’ are clandestinely targeting Government employees through the popular online CV website.

Secret agents working for malign foreign powers, including Russia and China, have created fake profiles on the social networking service to lure unsuspecting victims.

In the elaborate scam – that wouldn’t be out of place in a James Bond novel – enemy spies are using bogus accounts on the website, described as like Facebook but for business professionals, to try and ‘find, connect with, cultivate and recruit’ current and former Government employees.

The Daily Mail doesn’t need to clickbait, that is an extraordinarily windy title, and I’m wondering if it’s legitimate British English to use the verb malign as an adjective. Editorial warts aside, this is a serious issue, and exploiting social media leaks is something I do on a weekly basis.

As an example of what an exploitation might look like, here are some sanitized versions of a real world engagements I’ve had over the last year.

A fortune 500 executive was receiving a steady flow of messages with sexual content. The source knew things about her work day, her children, and details on a recent decorating choice in her home. Police had been working on the assumption that her home might have been surveilled or intruded. We examined her social media which did include some personal details, but not enough to cover all knowledge the stalker displayed. Access to her private office was a requirement and a ‘barium meal’ placed in her trash can yielded criminal charges for a janitor.

A Bitcoin related fraud case involved a limited liability company represented by a couple of individuals who were also codefendants. The LLC was incorporated in Delaware, making its members essentially unassailable. Starting with a pair of Twitter accounts for the promoters, we identified a pool of a dozen common associates there, and from that starting point a parent company with both assets and ongoing revenue was identified.

A fraud case resulting in a RICO suit involved multiple entities in several U.S. states and one offshore haven. The domains were examined for commonalities using Maltego, historic domain information was retrieved with Domain Tools, and the fingerprints of a single technical staff member was found. Manual examination of the LinkedIn networks for the named defendants yielded a candidate for the technical staff member, who was successfully subpoenaed.

A defamation domain concealed behind Cloudflare was strongly suspected to be the effort of a competitor to the company being smeared. A direct approach involving Maltego and manual methods yielded no usable information. The social networks of the leadership of the competitor were examined with an eye on other business entities, yielding a collection of domains to inspect. The defamation domain was colocated on the same virtual private server as one of those businesses.

Limited liability companies protect businesses from direct litigation approach, just as Cloudflare protects web sites. No such facility exists for protecting one’s social network footprint and Cloudflare only protects web services, not the entire network attack surface.

An initial hardened front on a company’s incorporation isn’t the end of the road. If their web presence is hardened that makes things much tougher, but if any social network data is available there is usually some avenue that can be pursued.

I am available for defensive and offensive engagements in this area. I can pursue an individual or company for the sake of discovery, or I can turn a would-be intruder’s eye on your presence. You can start the process by contacting me, Neal Rauhauser, on LinkedIn.

The Militarist Galaxy

The United States has no stable foreign policy; our geographic isolation coupled with our tremendous economy have put us in a bi-stable configuration for the last hundred years. We isolate … until things get really bad, then we plunge into the fray. We grow weary of participating in wars started by others and we turn our attention to domestic issues.

We’re making some serious mistakes now. I could swear our irresponsible, road blocking Republican dominated House decided to basically zero out the budget of the State Department the other day. That is loopy even by the broad standards we apply to them in this fifteen month interim before they are forever escorted out of control of the legislative branch.

This goofy, full tilt denial of reality thinking flows in large part from the group described as militarists – those brave souls who’ve never seen an intervention plan they didn’t love at first sight. Clear objectives? A pragmatic approach? Nope, we’ve got more guns than anyone, and we can bomb and/or drone until we achieve compliance.

I recognize some of the names here and having been lurking around the edges of policy making I’ve actually met one of the larger nodes – R. James Woolsey and I had a chat about renewable ammonia at a conference in Chicago back in 2008. They don’t seem unreasonable in small doses, except for the very fringe ones like Frank Gaffney, but looking back over the last twelve years the results from one war of necessity and one adventure speak for themselves.

Militarist People

Militarist People

Militarists By Name

Militarists By Name

Militarist Organizations

Militarist Organizations

Militarist Organizations By Name

Militarist Organizations By Name

RightWeb is keeping small public dossiers on 325 individuals and I found about 110 organizations. I inspected the people earlier in Militarist Influence On Foreign Policy but the results from named entity recognition were a terrible snarl.

What you see above are the results of probing the organizations, then removing all the single mention entities. Below are the ones with high degree. Keep in mind that being here is just a measure of who gets mentioned most often – Foreign Policy Collectives: @LobeLog earlier today was about Jim Lobe’s team, and they’re certainly not militarist, they’re just more influential than I suspected when I started examining their social network.

Top People & Organizations

Top People & Organizations

The name that immediately draws my eye is Frank Gaffney, whom Grover Norquist famously dubbed a “sick little bigot“. He’s reportedly all over the content in the Groundswell leak, which puts him in the room with people manufacturing fake news in order to harass the Obama administration.

Lies are what got us into Bush’s adventure in Iraq and if left unchecked people like Gaffney will lie and spin us into another hopeless morass – for the moment it’s Syria, but Egypt is sliding towards chaos in the same fashion. This Groundswell leak may provide the necessary leverage to drive a wedge between this fringe thinker and the affairs of Capitol Hill, but we’ll have to wait while the journalists digest it all. If I am counting correctly they have published just three threads from the three hundred documents and only twenty minutes of an insider’s wire tap of what must have been a fairly long meeting. There has to be more in there and I’m checking @MotherJones daily to see what emerges next.

Thirty Three Key Hashtags

Foreign Policy Collectives: @LobeLog

Earlier this year I examined the social networks of a number of foreign policy oriented groups including Wikistrat and e-International Relations. This included probing their Twitter and LinkedIn usage. I also laid hands on RightWeb’s content and produced Militarist Influence On Foreign Policy, an exploration of the static profiles for over 300 militarists maintained by a watchdog organization.

Near the end of that process I subscribed to LobeLog, which I’ve found to be very good. Today I noticed that eight of their authors have Twitter profiles so I turned my system loose on them.

3,508 DIscussion Peers For Eight Authors

3,508 DIscussion Peers For Eight Authors

400 Frequent Discussion Partners

400 Frequent Discussion Partners

Forty Four Accounts To Watch

Forty Four Accounts To Watch

So these final forty four are people who are important to the discussion – I recognize some of them from foreign policy reading and I assume the rest are academics and policy people. The criteria here were those mentioned fifty or more times in the last 3,200 tweets.

@LobeLog Authors Influencee Network

@LobeLog Authors Influencee Network

I pasted the eight seed names into Maltego and then let my @Klout transforms work. I am a little surprised by the result – the only loop in here is the one I created in order to keep the original accounts near the center of the graph. This is an indication that the foreign policy discussion space is large. When we examine astroturf efforts we find self-referential loops by the second generation.

There were over 1,400 hashtags referenced.

1,400 Hashtags

1,400 Hashtags

My parser has improved quite a bit since the last time I did this and I quickly narrowed down to just thirty three key hashtags that were being used.

Thirty Three Key Hashtags

Thirty Three Key Hashtags

What have we learned here?

I typed eight names into a text file, issued a single command, and fifteen minutes later I had the data used to produce these graphs. We can tell which other accounts they talk to, weighted by frequency, and we can determine who they influence according to Klout. We can also tell which topics concern them based on hashtag use weighted by frequency.

What can we do next?

I recognized some of the names as I was filtering the large list and in the final I see one person I know in real life and another that I know from a mailing list. These people are richly interconnected in a fairly transparent fashion.

I think the next step will be doing this for the much larger group listed on RightWeb, but that’s taking a while as I am having to dig for their Twitter accounts. Once I have that I will do some sort of composite graph, putting in all the foreign policy people and organizations I have identified, and I’m going to try to sort them into cliques.

What I really need here are a few foreign policy watchers who already pay close attention and who would be willing to either provide me API access to their account, or run a secondary account specifically to create who’s who lists. I have considered using a passive approach, just milking public lists, but for this to work I think there is an additional level required when classifying accounts. Lists made by users at this level tend to be inclusive – all experts on a given topic, rather than breaking them down to their viewpoints.

Militarist Influence On Foreign Policy

I have occasionally read individual profiles on RightWeb, and they have this to say about their efforts to track the influence of those who advocate excessive application of military force:

Right Web is a program of the Institute for Policy Studies (IPS) that assesses the work of prominent organizations and individuals—both in and out of government—who promote militarist U.S. foreign and defense policies, with a special focus on the “war on terror” and the Middle East. Right Web aims to foster informed public debate about these policies with feature articles and profiles of individuals and organizations that examine political discourses and institutional allegiances over time.

I happened to be looking at a profile their yesterday morning and I noticed that the content was amenable to being processed using Maltego and the natural language processing features of Alchemy and OpenCalais. I was right about accessibility, but the well tended set of profiles on 325 individuals was so link rich that it choked Maltego.

RightWeb Profiles 325 Militarists

RightWeb Profiles 325 Militarists

I saved the profiles page and with about fifteen minutes work I got a CSV file containing the name of each person and the link to their profile. I think it took Maltego about ten minutes to process the calls to the external services which extracted a large set of names, locations, and phrases. Once the content was there the trouble began. I pruned obvious mistakes and merged entities that were clearly the same. As an example, some profiles mention “President Obama” and others says “Barack Obama”.

Even with this clean up the content was cumbersome – 325 individuals with between fifteen and twenty five entities found in each profile basically brings Maltego to a standstill. Trying to scroll in entity list is painful and the visualization modes freeze for minutes at a time, or simply fail to redraw all together.

RightWeb Location Data

RightWeb Location Data

There were a large number of countries and quite a few specific cities that appeared as locations. Afghanistan and Pakistan, Iraq and Iran, Egypt and Gaza; these were common pairings for the profiles. I sorted them by creating regional location points and linking them but all this accomplished was creating a dozen more nodes on an already overwhelming graph.

RightWeb Profiles & Mentions

RightWeb Profiles & Mentions

Selecting the Person entities that had been discovered and moving them along with the associated profiles to a new subgraph was equally problematic. I removed the Person entities with only one mention .. then two … and finally when I had eliminated everyone with fewer than ten mentions I had a graph that was tolerable to explore.

President Obama was mentioned often, as were both Bushes, Clinton, and Reagan. Funniest were #!5, Islamophobe Frank Gaffney and #16 Condoleezza Rice, with one less link. A man that Grover Norquist famously described as a “sick little bigot” swings as much weight as a former secretary of state? Other LoonWatch favorites mentioned requently include Pamela Geller and Robert Spencer.

Maltego was a good starting point thanks to the named entity recognition support but the size of the response chokes it. The next sensible thing to do is export the content, but a simple minded approach to handling it won’t yield a lot of value. Gephi can swallow a dataset that size, but this is a few groups of entities of a specific type, and that’s not really a place to use Gephi beyond initial reconnaissance. Sentinel Visualizer is the closest fit in my data visualization toolbox, but importation will be a lot of work – some of the linked individuals are there because they agree, while others get mention because they provide opposition.

This is the perennial problem – you can have content, you can get it into a system, but there is no substitute for a human who follows the happenings. Good tools expand the reach of good analysts, they are not a substitute for having and developing that good analyst in the first place.

Illuminating NightWatch

I subscribe to NightWatch, a nightly review of daily geopolitical events that is considered the best of breed. The interface is old school – just rich format email daily, and a web link that provides a list of former entries. No topic cloud, no search, no nothing – you have to pay attention on a daily basis.

I have long wanted to dig deeper, indexing the content, and today I struck out in a first experiment in this area. I wrote a python script to create URLs for each of the 156 watches published this year, imported them into Maltego, then ran the Alchemy and OpenCalais Named Entity Recognition transform.

NightWatch Named Entity Recognition

NightWatch Named Entity Recognition

There were a lot of singletons in the return and I guessed that many of them would be parsing errors, so I used the circular layout in order to select nodes with two or more links to move to a new graph.

NightWatch NER Circular Layout

NightWatch NER Circular Layout

Once I had the graph trimmed I examined the layout using ‘bubble view’. This looked promising, with nodes sized by degree and some underlying structure evident in the giant component.

NightWatch NER Entities

NightWatch NER Entities

But once I looked closely I was disappointed. These three section of the graph give you the idea that the system did a good job of sorting by region, but this is not the case – ‘Pakistan’ is on one side of the graph, while ‘Quetta’, perhaps the most violent city in the country, was on the other side.

NightWatch NER Topics #1

NightWatch NER Topics #1

NightWatch NER Topics #2

NightWatch NER Topics #2

NightWatch Topics #3

NightWatch Topics #3

This is a case of using the wrong tool to create something visually interesting, but that provides no insight. It’s useful to know how often topics get referenced, but when logically related items are spread all over the graph even that bit of aggregation of information is of dubious value.

This graph was created by accessing 156 URLs, each of which contains a date stamp. I could try the temporal analysis features in Sentinel Visualizer, but since we are trying to see concepts rather than forensic data, I am not sure that it’s the right solution. I probably need to sort out Gephi’s Graph Streaming plugin, but before I do that I’ll need to either extract the graph from Maltego or write something of my own to extract named entities from the NightWatch URLs.

Wikistrat Klout Influencers Using Maltego

Wikistrat bills itself as ” is the world’s first Massively Multiplayer Online Consultancy (MMOC)”. This is the largest of a handful of hive minds I have studied that focus on foreign policy. The blowout from the Egyptian army takeover is something I have known was coming for about two weeks thanks to my subscription to NightWatch. I am curious to see who among their analysts was first discussing it on Twitter, and who else they involved.

Wikistrat Analysts

Wikistrat Analysts

Things like this go through a progression, first the locals and knowledgeable observers in country detect something in the works, then the the well connected analysts begin to talk, then the specialty reporting outlets, and finally the topic broaches mainstream news. Somewhere in the mix, around the time the specialty outlets begin running the story the blogosphere will also pick it up and start playing with it. A while ago I wrote a Maltego transform for the premier reputation tracking system, Klout. I dusted it off this evening and applied it to Wikistrat. Only four out of the thirty accounts I have identified were not registered for Klout.

Wikistrat Analyst Influencers

Wikistrat Analyst Influencers

I have been looking at the Wikistrat social network for a while now and the Twitter contingent is not large, so I recognize most of them by sight, and many of their conversation partners from the various examinations I have performed. The first surprise that there were so few influencers that reached more than one member. I had assumed I would find at least some luminaries from the field, but this is not the case. It was also a bit surprising to see that there were no instances where one analyst influenced another. My take on that is that these guys don’t talk shop on Twitter.

Wikistrat High Degree Influencers

Wikistrat High Degree Influencers

Thinking that this set was a high value source for foreign policy information I went through and manually separated them into organization role account and humans. The role accounts aren’t interesting in this context, since they are mostly broadcasters rather than conversation partners who would have influencers.

Wikistrat Analysts Influential People & Orgzanizations

Wikistrat Analysts Influential People & Orgzanizations

Once I had just the people I collected second generation influencers.

Wikistrat Second Generation Influencers

Wikistrat Second Generation Influencers

And when I checked those influenced by the core group I found no feedback loop at all. I think there are two explanations for this. The first is that this is a large universe, many players, and we’d start to see feedback loops if we took another step back. The problem with this is … two steps is a lot in a professional environment. The six degrees of separation meme is based on a sociology study in the 1960s and it more or less holds up no matter how large the system goes. There is a study of the Microsoft Messenger network where they had a set of two hundred million people and the average path length was about six. A professional network ought to have something similar to the Erdős Number for mathematicians – with an average distance between two people being four to five hops.

The other explanation is a little easier to swallow – Klout is seeing interactions but foreign policy is dense and there is an industry specific jargon. If the system can’t interpret the content of tweets it is less likely to automatically select influencers, and our test set are people who aren’t manually adding influencers to their profiles.

Three Generations No Feedback

Three Generations No Feedback

Circular layouts are a way to see who is in the middle of the action and who is on the edge. Here we see a core of actors and some small clusters on the edge.

Three Generations Circular Layout

Three Generations Circular Layout

I selected the inner circle, moved it to its own graph, and used a force directed layout. This particular phase didn’t provide a lot of new information but in general this is a place you’d linger if you had a new, complex network you were trying to understand, so I include it for the sake of completeness.

Three Generations Influencers Core

Three Generations Influencers Core

And finally we get a bit of a payoff. I recognize @texasinfrica, this one turns up in all sorts of discussions. The other eight are a couple of role accounts and a handful of new people.

Nine New Players

Three Generations Nine New Players

What did we accomplish here? I can see a few things.

I’ve had this ability for a long time but tonight was the first time I’ve applied it to more than a few accounts for testing. This is a selection of all influencers, not just foreign policy sources, so we’re stuck with a manual slog if we want to constrain the results to just that sector. The Maltego transform servers are limping tonight, which I credit to the work created by the Egyptian coup, so I can’t get into what these guys are saying without putting them into my Twitter recorder. There are maybe two hundred accounts listed so it would be an overnight job to get them all recorded for the first time.

What comes next?

The Maltego Klout transform code can be adjusted to produce output suitable for Gephi with a very small amount of work. We could use the Klout number itself to weight Twitter accounts, we can graph the influencers or influencees, and we can pull topics and create an accounts to areas of expertise map.

The Klout API rules are much tougher than Twitter or LinkedIN, where you can just create an application at will. I had to explain what I was doing and they were really helpful – my account has ten times the API credits of the individual/desktop accounts, and they agreed that if I could show some unique uses involving Maltego they would work with users that needed the higher capacity. They also limit caching to five to seven days, but this not good for forensics work or for long term studies of how clusters of accounts change over time.

Once we work through the access and data retention issues there are some really cool things that can be done – like launching a brand new Twitter account and taking daily snapshots of friends and followers as well as the derived Klout attributes. This data could be used to feed Sentinel Visualizer or the Gephi streaming plugin, producing an animation of an accounts growth and expansion into new topics.

I suppose the first order of the day here is encapsulating anything that can be done with the API in both a Maltego transform and something to output CSV for use with Gephi and Sentinel Visualizer. I will do this, publish the code to my Github, and then pick out something to study and write it up here.

Sentinel Visualizer: Serious Network Analysis

The first time I ever touched a link analysis tool was late in 2010. I downloaded the Maltego Community Edition, clicked my way through entering a pair of Twitter accounts, and despite the rate limited demo API the very first transform I ever ran immediately showed me something important. I was hooked.

I have had a paid license for the last two years and if you check the Maltego category here you can find a variety of posts showing just some of it’s uses. I recently took the Coursera SNA class, which is an excellent introduction to social network analysis, and I got to spend a bit of time with Gephi which is a more general purpose tool. Anything I have done recently, like this visualization of terror group names and their locations of operation, would have been done with Gephi.

Centers Of Terror Activity

Centers Of Terror Activity

My writing here has started to pay off and for the next three months I have a chance to work with Sentinel Visualizer from FMS. This is a law enforcement/counter insurgency grade visualization tool that can do many things which these other two tools can not.

Sentinel Visualizer Social Network

Sentinel Visualizer Social Network

This is a social network for some bad guys. Each of them has a ring of associates, and then in between there are people who are fixers, facilitators, or financiers for their activities. This is something you could discover with pen, paper, and patience, or you can apply a visualization tool. This particular graph could just as well have come from Maltego.

What Sentinel Visualizer offers that lesser tools do not are capabilities in two new realms – geospatial and temporal information handling.

Sentinel Visualizer understands where things happen in explicit detail. Maltego does have a ‘place’ entity, but it’s a simple little thing, just a city and country name required, maybe you put a street address. Sentinel Visualizer understands latitude and longitude, it understands how to find things that are geographically near each other, and it can provide detailed information for use in Google Maps.

Sentinel Visualizer also understands when events happen in a flexible, powerful fashion. Time can be as specific as down to the second, or as broad as “some time last summer”. This network of bad guys didn’t just appear out of thin air, they met, one after the other. This can be visualized forward and backward in time, letting you examine what happened before, during, and after a specific event.

The system offers some other capabilities that aren’t a big deal for my specific project, since I’m the only one handling the data, but they do matter for larger scale use. Individual installs use a local SQL server, but the top level product uses a shared database server so multiple analysts can all use the same data. That system also supports free read only clients, so consumers can view finished content live in the system rather than waiting for static reports.

I’m very excited to have this opportunity to add a full featured LE/counter-insurgency grade tool to both my skill set and my resume. I can’t say anything about the details of the project itself, beyond the fact that it is something to do with food security, but the system is flexible enough that I will be able to round up other datasets and show off what it can do.