Ricky : Today we're going to talk about
the world of OSINT, or open-source
intelligence and how it can really help
boost your skills as a cyber practitioner,
whether you're a seasoned professional or
just starting out. To better understand
what OSINT is, let's take a look at how a
military commander begins a campaign.
First, he starts off by conducting
reconnaissance and gathering intel. He'll
then send out spies to collect as much
info on the enemy's strength and
disposition. If he's really smart, he
might also do the reverse and use the same
scouts to recon his own forces to
understand what kind of information the
enemy might be able to gather on him. In
the cyber world, it's no different.
Building situational awareness is crucial
to success, whether you're doing digital
investigations, red or blue team
operations, or looking for a place to
start in the world of cybersecurity. And
OSINT is one of the most accessible and
low-cost ways to do this. In this video,
we'll go through a conceptual overview of
OSINT, how it can benefit your skills as a
cyber professional, and some great
resources out there to start with.
So back in World War Two, the United
States had an intelligence Department
called the Office of Strategic Services,
or OSS, which was the precursor to the
CIA. The OSS had an entire Research and
Analysis branch dedicated to open-source
intelligence. They collected newspapers,
journals, press clippings, and radio
broadcast reports from all over the world,
just to hunt down photos or articles that
might give away crucial intelligence about
the enemy. From bomb craters to new
aircraft or battleships, these bits of
data, once pieced together, could be used
to assess an enemy or to verify other
sources of intel. But today, just about
everybody uses OSINT for different
purposes. Journalists writing news
reports. cybercriminals looking to scam
people, students in academia working on
research projects, employers scoping out
job candidates, law enforcement working on
crime cases, and much, much more. For
example, in 2015, the US Air Force was
able to launch a strike against an ISIS
headquarters building within 24 hours of a
fighter posting a selfie of himself on
social media. Back in the old days, the
challenge of OSINT was in gathering enough
information. In today's world, though,
we're drowning in it, and the challenge
lies in processing and analyzing
everything. The sources of data are
tremendous, covering everything from
satellite or street-level imagery, public
court records, social media posts, videos,
forum threads, news articles, data leaks,
website history, IP registration data, and
many more. Generally, there's two ways to
gather data and information: active and
passive. Active collection puts the
researcher in direct contact with a
target. This might mean physically
traveling somewhere, talking with someone,
dumpster diving, or scanning a system for
vulnerabilities. While these results can
be very accurate, there's a higher risk of
detection because of your direct
involvement. It also tends to be narrowly
scoped and may miss out on the bigger
picture. Passive collection, on the other
hand, focuses more on quiet observation of
data that's generated by a target.
Studying maps, listening to someone's
conversation, or finding vulnerabilities
by fingerprinting a device based on its
network traffic are all passive
techniques. OSINT largely falls under the
passive category since it can almost be
done from the comfort of your chair. You
can also remain anonymous, provided you're
practicing good OPSEC such as using
virtual machines, VPNs, research accounts,
and Tor. The downside to passive
collection is that it requires more
involved analysis and may not provide the
same quality of intelligence as active
collection. Let's say you went on vacation
to Japan and I wanted to learn more about
the trip. While it might be easier to ask
you how it went, you might think I'm being
nosy and not want to share much. But by
turning to open-source intelligence, I
could gather the photos you posted on
Facebook of the trip, and geolocate them
to see where you went. To find out harder
questions like why you went to those
places, I might have to do some map recon
or spend time studying your Twitter
account or blog to get some contextual
clues about your choices and thought
processes. There's actually a third
collection method we'll call semi-passive,
that falls somewhere between the previous
two. This involves leveraging a
third-party service's active collection
measures to perform passive analysis. For
example, one of my favorite tools is a
site called urlscan.io. Made by the threat
intelligence expert Johannes Gilger,
urlscan lets you input a target URL, like
maybe a suspected phishing link and the
service will provide you with detailed
analysis about the website by visiting it
on your behalf, or presenting you the scan
results performed by someone else. It's a
kind of in-between the active and passive
techniques, but can also be considered
OSINT. There are also many services that
require you to pay for access to premium
databases that aren't publicly available
because they source information in a
variety of ways. There's a controversial
company called Clearview AI that scrapes
search engines and social media platforms
for images of people's faces, building a
private facial-recognition database for
clients to access. While some people might
not consider private databases to be pure
OSINT, because they're not free, others
might consider them fair game and be
considered a semi-passive research method.
Now, with all this in mind, we're going to
talk about the two most important concepts
in OSINT: identifiers and pivoting.
Identifiers are unique keywords, tokens,
or artifacts that describe a piece of
data. Some examples include name, email,
birthday, IP address, MAC address, phone
number, geo coordinates, home address,
license plate, timeframe, picture, Bitcoin
address, password hash, hostname,
operating system, social media handle,
relationships, occupations, social media
username, hobbies, hacker handle, credit
card number, search query, or website. You
get the idea. These identifiers might
exist across many different datasets
scattered across the internet. When you're
conducting OSINT research, you may only
have one or two identifiers available to
work with. Just searching for information
based on a couple of them might not give
you the best intelligence. The real OSINT
magic comes from pivoting, which is
searching for the same identifier in
different datasets to correlate and
discover new identifiers about a research
target. For instance, a photo might
contain a unique landmark that you can
discover using Google Street View or
Mapillary that leads you to a house.
Searching for the address on public county
records can reveal the owner's name, which
can then be used to discover social media
accounts and email addresses. In this
case, we've pivoted from a photo to an
email address. In other cases, you may
want to pivot in the opposite direction,
which requires you to possibly chain
identifiers using different types of
open-source data. For a more formalized
approach, the RAND Corporation came out
with a great paper talking about
open-source intelligence, link in the
description below. They break down the
OSINT lifecycle into four stages:
collection, processing, exploitation, and
production. The collection stage involves
acquiring and storing data from a variety
of sources. In many cases, it's not
practical for individuals to hoard
terabytes of data, so this step might
involve just signing up for accounts and
building API keys to query services that
do store the data. Speaking of services, a
great one out there is IntelX.io. Built by
the Austrian security professional, Peter
Kleissner, IntelX scrapes, Pastebin, and
many other sources from the darknet for
breach data and other types of
information. They also host a bunch of
useful third-party search tools for
identifiers. You should definitely check
out IntelX since it's a great way to find
different identifiers that normal search
engines won't show. Now the next stage is
processing, which may involve translating
results or normalizing them into a common
format for collaboration. There's Google
Translate and a bunch of project
management tools out there that will come
in handy at this step. The exploitation
stage involves connecting the dots between
identifiers and analyzing results in a
broader context. A great tool to use here
is Maltego, which lets you perform graph
analysis between different identifiers,
almost like a digital version of a
detective's evidence board. You can also
use Hunchly, which is a web capture tool
that automatically saves pages you've
visited before to preserve a trail during
OSINT deep dive or investigation. It's
made by the security researcher and OSINT
wizard Justin Seitz, who's also the author
of Gray Hat in Black Hat Python. One OSINT
pitfall is that not all sources of
information are equally valid, since some
might contain bias or have questionable
origins. Authenticating the credibility of
data at this stage is an important, but
often overlooked part of OSINT. The last
stage is production, which involves
consolidating your findings into a useful
report and then sharing it with others. If
you're just starting out in cybersecurity,
practicing your open-source intelligence
gathering skills is a fantastic way to dip
your toes into the field, since it's
something that doesn't require heavy
technical knowledge or programming skills
to learn. OSINT is naturally
research-oriented, which helps you develop
the virtues of persistence and curiosity,
personality traits that are essential for
being successful in cyber. If you know how
to use Google search, start learning some
of the more advanced search operators
available. Esteban Boges, a cyber
researcher at SecurityTrails, wrote a
great article on using Google Dorking to
find sensitive information and potential
vulnerabilities indexed online, link in
the description below. Start with
mini-OSINT project such as trying to find
as much personal information on yourself
or your family. Try different tools to
make the process easier and automated.
Now, if you work as a penetration tester
or red teamer, OSINT is one of the first
methods you should turn to when performing
reconnaissance on a client. Companies are
made up of people, with a well-defined
hierarchy, you can uncover with formal and
informal relationships between them. You
should build out clear profiles that
include identifiers, interests, and
habits, because these can uncover clues to
weaknesses for exploitation. You may find
someone who habitually recycles passwords,
some of which already exist in in breach
dumps or have the answers to their account
security questions scattered across the
Internet. The right amount of due
diligence on people allows you to craft
more credible and trustworthy social
engineering pretexts or phishing emails.
For more technical targets like servers,
good OSINT can let you map out a company's
external-facing infrastructure, or even
uncover clues about its internal posture
and security policies. If you're on a blue
team, doing the exact same thing can paint
a better picture of what adversaries might
have already researched using OSINT and
possible attack channels they might use.
The amount of tools and resources out
there simply go on and on. But there's
three that I think are worth mentioning
when it comes to learning more about OSINT
and cybersecurity. The first is a book
called, "Hunting Cyber Criminals", which
is written by the daring Vinny Troia,
who's single-handedly tracked black hat
criminals across the Internet and shares
his tradecraft and process in this book.
The next is a podcast called, "The
Privacy, Security and OSINT Show", hosted
by Michael Bazzell, one of the foremost
experts when it comes to helping
celebrities, billionaires, and everyday
folks to disappear and find some peace of
mind. His expertise in cyber
investigations is world-class and is a
must follow. The last resource I want to
share is a website called bellingcat.com,
which is full of articles, guides and case
studies showcasing real-world OSINT
investigations. Check out Vinny, Michael
and Bellingcat as three places to get
started on your journey to becoming an
OSINT pro. So that's it for this video on
open source intelligence. There's simply
far too much information out there to
cover OSINT in just one video, but I
really hope you found this overview as a
helpful start to share with others you
know. Thanks so much for watching, and
I'll see you soon!
