Missing Persons Edition: Using ElasticSearch to Sift through Breached Databases

-

Using ElasticSearch to Search through Breached Databases

Organizations, companies, and governments will have their data stolen multiple times a year by hackers. Notable cases include LinkedIn (2016), Adobe (2013), and Experian (2015). The stolen database will circulate on the internet via dark web, forums, and torrent magnet links. As a missing person investigator, we can use this data to find a persons home address, phone number, password information, birth date, secondary email addresses and more. ElasticSearch can help us store and search billions worth of breach data.

ElasticSearch is an open-source, distributed, RESTful, search engine. ES (ElasticSearch) uses JVM and is built on top of Apache Lucene. ES is great for indexing large amounts of data, sifting through a large result set, and analyzing data.

In this video I will demostrate how to take breach data, convert it to ES-compliant JSON, import it, and then query for results.

In this video I will show you how to run React/Node.js as a frontend solution.

Github Link

The Github to the ES Parsing tools and frontend is here: ElasticSearchBreachDB

Tasks Index

Download ElasticSearch

Download ElasticSearch here: ElasticSearch Downloads

Create an Index

Display all Indexes

GET http://localhost:9200/_cat/indices

Count how many Documents are in an ES index

GET http://localhost:9200/breach-imgur/_count

Comments

Post a Comment

Popular posts from this blog

Palo Alto for GNS3 CCDC Tutorial

-
Image
Opening Hello fellow CCDC Competitors! Struggling to get a Palo-Alto VM? I know how you can get one! Email me at ASzampiasSWD@gmail.com. I'll tell you the steps to get a PA-VM for GNS3. Why am I giving this info to you? Wouldn't it make sense to hog the info to myself? Wouldn't my team have an advantage? There are have and have not universities when it comes to PaloAltos. I fell into the "have not" group. Our college didn't have access to PA-VMs and still doesn't. Even if PA Cybersecurity Academy is free, there is too much red tape. I can't change the outcome. I can only come up with this one solution. Another reason is Tri-C is a tryout team. I think its pretty unfair as a potential Firewall Admin I'm the only one with access. No one can challenge me unless they spend 2K for a PA-VM. It becomes a rich-person only position. I have a leftover PA-VM from 2018 I promised not to share. I couldn't share that with my challengers. But I f...
Read more

Trace Labs Global Missing Persons CTF V

-
Image
Trace Labs CTF V TraceLabs hosted the Global Missing Persons CTF V event on July 11, 2020. My team //synergy got 64th place out of 190 teams. Introduction This was my first Trace Labs CTF. I captained a team. The good thing about being a captain is you can control who is on the team. My rules were: Must have sock puppet accounts ready. Must have a VPN. Writing out this is a competitive team where we work the entire 6 hours. I was worried about a future team member spamming submissions and disqualifying the team. I thought my requirements would eliminate people who weren't serious. The team hit 4 members Friday evening pretty quickly. Notes My laptop was too slow for this event. My virtual machine froze and I had to restart. Restarting means I have to login to my accounts again. I'll use my desktop next time. The team used Slack and Trello. Our strategy was to research a person and rotate every hour. This strategy broke after 1 rotation. Some missin...
Read more

Release of CCDC ISE Manager Website

-
Image
I worked on the ISE (Inject Scoring Engine) website from 2017 to 2019. I made it to encourage teams to follow a merit-based, open admission tryout format. I identified a problem and made a solution for it. Problem: Call for Merit and Fairness on CCDC Rosters Solution: Github Project Solution: How to run a Team After Tri-C was finished with 2017 Midwest Wildcards I used httrack to grap the Inject PDF files. I didn't grap just the files, but the entire front-end code of the website. If you created the HTML and CSS of the original ISE website, tell me your name so I can give you credit! My goal was to make an almost-replica of the original. The database and back-end code is all guessing by me. I used a LAMP stack because I believe the original website is built on that. I did some fingerprinting and found the "ccdcadmin1" website was hosted on Apache and PHP. If you notice any bugs or errors let me know. If you want a feature added let me know. I will be working ...
Read more